NIST AI Risk Management Framework: turning Measure and Manage into retrievable evidence
The NIST AI Risk Management Framework, published as NIST AI 100-1 in January 2023, is a voluntary framework organised into four functions, Govern, Map, Measure and Manage, that describe outcomes for managing risk across an AI system's lifecycle.
Auditrek is not open yet, so anything on this page about what it does is what it is being built to do.
What the NIST AI risk management framework is
The NIST AI Risk Management Framework (AI RMF 1.0, published by the US National Institute of Standards and Technology as NIST AI 100-1 in January 2023) is a voluntary framework for identifying, measuring and managing risk in AI systems across their lifecycle. It is organised into four functions, Govern, Map, Measure and Manage, each broken into categories and subcategories that state outcomes to be achieved rather than controls to be installed. NIST publishes a companion Playbook of suggested actions and references alongside it, and has published profiles for specific contexts, including one for generative AI.
Three properties of the framework matter more than its table of contents.
- It is voluntary and there is no NIST certification. No accredited auditor issues a NIST AI RMF certificate, and no penalty attaches to not using it. Its value is as a shared vocabulary: when internal audit, a supervisory examiner, or an enterprise customer's third-party risk questionnaire asks how you manage AI risk, the framework supplies the structure of the answer.
- It is outcome-based, which shifts the burden onto you. A subcategory tells you that an outcome should hold. It does not tell you what document, log or record demonstrates that it held. You supply the evidence, and you decide what counts as sufficient.
- It is socio-technical and lifecycle-wide. The framework frames risk in terms of trustworthiness characteristics, including valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. These are not properties you measure once at release.
If you landed here because someone has asked you to adopt the framework, the practical work is narrower than the document. Govern and Map produce artefacts a compliance function already knows how to produce: policy, ownership, risk tolerance, an inventory, an intended-purpose statement. Measure and Manage produce assertions about what a deployed system did over time. This guide walks the four functions, then spends most of its length on the evidence gap in the second pair.
The four functions, and what each one actually produces
Govern: the function that runs continuously
Govern is cross-cutting rather than sequential. It covers policies and processes, accountability structures, roles and responsibilities, risk tolerance, workforce competence, and engagement with affected parties. Its outputs are organisational: an approved AI policy, a named owner per system, a documented tolerance for the kinds of error a system can make, and a record that the policy was applied rather than merely written.
Map: establish context before you measure anything
Map is where you write down intended purpose, deployment setting, affected populations, the human oversight design, known limitations, and the assumptions the system depends on. Map is also where categorisation happens, including whether a use case sits in the higher-risk band under whichever regime applies to you. Without Map, Measure has no reference point, because you cannot say a metric is acceptable without a stated purpose and tolerance.
Measure: analyse, benchmark and track
Measure covers selecting appropriate methods and metrics, evaluating the system against the trustworthiness characteristics, tracking identified risks over time, and gathering feedback about the effectiveness of measurement itself. The important word is track. Pre-deployment test results satisfy a fraction of Measure; the rest asks what the system has been doing in production since then, which is a question about records, not a question about test suites.
Manage: prioritise, respond, recover, communicate
Manage covers allocating resources to mapped and measured risks, deciding which risks to treat and which to accept, responding to and recovering from incidents, monitoring after deployment, and handling third-party components. Manage is where override handling, change history, escalation and decommissioning live. Every Manage claim is a claim about a sequence of events, which means it can only be evidenced by something ordered and complete.
Why Measure and Manage is where the evidence thins out
Most organisations can assemble a credible Govern and Map file in a quarter. The policy exists, the inventory spreadsheet gets owners, the intended-purpose statements get drafted by the people who built the systems. These are documents about intent, and documents about intent can be written after the fact without anyone being misled about their nature.
Measure and Manage are different in kind. They assert facts about a production system's behaviour across a period: that drift was monitored, that overrides were reviewed, that a model change was noticed, that an incident affected a bounded set of decisions. When a reviewer tests those assertions, the usual answer is one of four things, and none of them travels well.
- A monitoring dashboard. It shows the present state and perhaps a thirty-day window. It cannot reconstruct what a specific decision looked like in March, and its underlying rows are frequently aggregated away.
- Application logs. They exist, they are voluminous, and they are mutable by anyone with write access to the log store. They were designed for debugging, so they carry stack traces and raw payloads rather than a stable field set, and they are usually rotated out on a retention schedule set by an infrastructure team that was never told about an evidence requirement.
- A spreadsheet maintained by the model risk team. Accurate, well intentioned, and an entirely human artefact. It shows conclusions, not the events the conclusions were drawn from.
- A ticket trail. It proves someone discussed something. It does not prove what the system did.
The gap is not a tooling shortage, it is a definition problem. Nobody decided what the unit of evidence is. Once you decide that the unit is one inference event, recorded in a fixed field set, in an ordered sequence per system, the Measure and Manage subcategories stop being essays and start being queries over a table.
What a reviewer asks for, function by function
| Function | What a reviewer typically asks to see | What event-level decision records contribute |
|---|---|---|
| Govern | Approved policy, named owners, documented risk tolerance, an inventory, and evidence the policy was applied to each system in it | Shows that a system in the inventory was genuinely under recording, from what date, and under which owner, rather than being listed and unmonitored |
| Map | Intended purpose, deployment context, affected populations, known limitations, and the human oversight design | Shows that the configuration actually serving decisions matched the mapped design, through model id and version and through the presence of human override events |
| Measure | Chosen metrics and methods, pre-deployment results, ongoing production measurement, and how measurement fed back into decisions | Supplies the underlying rows: output and confidence values per model version across any date range, so a metric can be recomputed rather than taken on trust |
| Manage | Monitoring in operation, incident and override handling, change history, and decommissioning | Gives an ordered, append-only sequence per system, so override rate, the moment a model version changed, and the point a system stopped producing decisions are all readable from one stream |
Turning Measure and Manage outcomes into retrievable evidence
- Fix the boundary of each AI system
Every later claim is per system, so the boundary decides what your evidence can prove. A useful rule: one system is one decision type served to one population under one owner. A retrained model on the same pipeline is a new model version inside the same system. A different pipeline serving a different decision, even on the same infrastructure, is a different system with its own record stream and its own inventory entry.
- Name the unit of evidence
Declare in writing that the unit is the decision record: one inference event, one row. This single sentence resolves most of the ambiguity in Measure and Manage, because it converts open-ended questions about monitoring into questions about coverage, completeness and ordering of a table you can point at.
- Agree the field set before anyone writes code
A workable minimum is input hash, model id and version, output, confidence, timestamp and human override flag. Adding fields later is easy; discovering that eight months of records lack model version is not, because attribution across retrains becomes guesswork and every Measure claim about a specific model collapses.
- Capture at the boundary, not inside the model
The serving wrapper already knows all six fields at the moment it returns a response, so the capture call belongs there: a POST from the wrapper, or a webhook emitted by the platform. Nothing about the model, the framework or the hosting needs to change, which matters because a migration turns an evidence project into a platform project and platform projects slip.
- Make the sequence append-only and chained
Assign each record a sequence number within its system, and bind it to its predecessor by hashing the previous hash together with the record payload. An append-only chain gives you a property a normal table does not have: a record cannot be quietly edited in the middle without the recomputation diverging from that point onward.
- Verify by recomputation, never by a stored flag
A boolean column saying verified is worth nothing, because whoever could alter a record could alter the column. Verification means walking the chain from the first sequence, recomputing each hash from the stored previous hash and payload, and returning a status derived on the spot, along with the first sequence number where recomputation failed.
- Put the state in front of the person who will be asked
The people who build the pipeline are not the people who answer the examiner. Whoever signs the response needs four numbers without opening a query console: how many decisions are logged, how many systems are connected, when the last report was generated, and what the current verification status is.
- Write down what the records do not cover
Name the gaps in the same document as the evidence: systems not yet connected, the date coverage begins, fields not captured, and any period where capture failed. A reviewer who finds an undisclosed gap treats everything else as suspect. A disclosed gap with a date attached is a finding you are already managing.
The field set that carries the weight
Six fields do most of the work across Measure and Manage. Each exists for a reason worth understanding before you negotiate the schema with an engineering team.
Input hash rather than input
Storing the raw input to a credit, hiring or clinical triage model turns an evidence store into a second copy of your most sensitive data, held for longer than the original and read by more people. A hash of the canonicalised input preserves what you actually need, which is the ability to show that a specific later reconstruction of the input matches the one the model saw, without holding the input itself.
Model id and version
This field is what makes a measurement attributable. Without it, a confidence distribution shift across a quarter is uninterpretable, because you cannot tell a population change from a silent redeploy. With it, every Measure claim can be scoped to a version and a date range.
Output and confidence
The pair is the raw material for production measurement. Held per record over time, they let someone recompute the metric a reviewer is questioning rather than defending a number whose provenance nobody remembers.
Timestamp and human override flag
Ordering matters, and so does the record of whether a human changed the outcome. The override flag is the only field that evidences human oversight at event level: oversight design belongs in Map, but the assertion that oversight operated is a count of overrides, per system, per version, over a period.
Minimisation at the door, not in a policy
Inference payloads from credit, hiring and clinical pipelines will carry special-category personal data unless something actively stops them. A policy prohibiting it is not a control. Rejecting a payload at ingestion when it contains recognisable raw sensitive patterns, such as national insurance or social security number formats, diagnosis code patterns, or unredacted credit bureau fields, and pseudonymising subject identifiers by default, is a control. Your lawful basis for processing remains yours to establish with your own counsel and your data protection function, and it should be settled before the first pipeline is connected, not after.
What a hash chain proves, and what it does not
A hash chain is a modest, well understood mechanism, and its value comes from being honest about its limits.
What it gives you is detectable alteration. Each record stores the hash of its predecessor, and its own hash is computed over that previous hash together with its payload. Change one field in record four hundred of ten thousand, and recomputation from record four hundred onward no longer matches what is stored, so verification returns a broken status and names the first sequence where the divergence appears. That converts the question a reviewer is really asking, which is whether this record could have been edited after the event it describes, from a matter of trust in your access controls into a matter of arithmetic anyone can repeat.
What it does not give you is worth stating plainly.
- It does not prove the record is true. The chain protects a record against later modification. It says nothing about whether the model behaved as the payload describes, which is why the capture point matters and why capture should sit in the serving path's own wrapper rather than in a reporting job that reads a downstream copy.
- It does not by itself prove when a record was created. Existence at a point in time is a separate property, usually obtained from an external timestamping authority. Treat time-of-creation and non-alteration as two distinct assurances, and do not let a chain diagram imply both.
- It does not decide retention. The AI RMF sets no retention period, and neither does a hash chain. Retention is a decision for your counsel and your sector's record-keeping requirements, taken with awareness that a chain you truncate can only be verified from the earliest record you kept.
How the AI RMF sits alongside the EU AI Act, ISO 42001 and SR 11-7
Teams often meet the NIST framework at the same moment as two or three other regimes, and the relationship between them is simpler than the volume of vendor material suggests.
- NIST AI RMF is voluntary guidance from a US standards body. It is broad, outcome-based, and deliberately not a certification scheme. It is the most useful vocabulary for internal structure and for answering enterprise risk questionnaires.
- Regulation (EU) 2024/1689, the EU AI Act, is binding law with a phased application timeline. Among its provisions for high-risk systems are record-keeping and logging requirements in Article 12 and requirements on transparency and information provided to deployers in Article 13. Whether and how it applies to a given system is a question for counsel, not for a framework crosswalk.
- ISO/IEC 42001 is an AI management system standard, and unlike the AI RMF it is certifiable through accredited bodies, which is why boards often prefer it as a target when they want an external attestation.
- SR 11-7 is US supervisory guidance on model risk management, long predating the current AI wave, and it supplies the vocabulary that examiners in US financial services already use: model inventory, validation, ongoing monitoring, effective challenge.
The practical point is that these four regimes ask for overlapping evidence in different wrappers. Every one of them, somewhere in its text, asks you to show what a specific system did over a specific period, who could change that account afterwards, and how often a human intervened. Build the event-level record once, with a stable field set and an ordered chain, and you are answering the same question in four dialects rather than running four programmes.
What none of this settles
Two honest boundaries, because a guide that pretends otherwise is not much use to someone who will have to defend the result.
First, evidence is not judgement. A complete, verifiable record stream shows what a system did and that the account has not been altered. It does not tell you whether the metric you chose was the right metric, whether your risk tolerance was set defensibly, or whether a supervisory authority will find your documentation sufficient. Those are determinations for your own counsel and, where relevant, your regulator. Nobody can sell you a conclusion about your regulatory position, and any tool that implies it can is describing something it cannot deliver.
Second, the framework has no report format. NIST does not publish a filing template for the AI RMF, so there is nothing to generate against it the way you might generate a structured report for a named article of a regulation. Auditrek's report generator produces PDF and HTML output mapped to EU AI Act Article 12 and 13 fields for a chosen system and date range, with a single regulation and format set at launch, and every report carries an embedded disclaimer stating that it does not constitute legal advice and has not been reviewed by qualified regulatory counsel. For NIST work, the records, the field set and the verification result are what you draw on when you write your own Measure and Manage narrative.
If you have one quarter
Pick the single system that would be asked about first, usually the one making decisions about people. Write its boundary and field set on one page. Get capture running from its serving wrapper, using the platform's existing REST or webhook path rather than a migration, with documented payload mapping for whichever platform is serving it. Run a verification pass and record the result with a date. Then write the Measure and Manage sections of your file against a record stream that exists, and list the systems still uncovered as a dated, owned gap. That sequence is unglamorous, and it is the difference between a framework adoption and a file someone can stand behind.
Questions
Is the NIST AI Risk Management Framework mandatory?
No. It is voluntary guidance published by the US National Institute of Standards and Technology as NIST AI 100-1. There is no penalty for not adopting it. Organisations use it because it gives a defensible structure for internal governance and a common vocabulary for answering customer and auditor questions.
What are the four functions of the AI RMF?
Govern, Map, Measure and Manage. Govern is cross-cutting and covers policy, accountability and risk tolerance. Map establishes context, purpose and affected populations. Measure selects methods and metrics and tracks risk over time. Manage prioritises and treats risk, handles incidents and overrides, and monitors after deployment.
Is there a NIST AI RMF certification or audit?
NIST does not run a certification or auditor accreditation scheme for the framework. If an external attestation is what your board wants, ISO/IEC 42001 is the AI management system standard that accredited bodies certify against. The AI RMF is used for structure and self-assessment.
Does following the AI RMF satisfy the EU AI Act?
They are different instruments. The AI RMF is voluntary US guidance; Regulation (EU) 2024/1689 is binding law with its own provisions, including record-keeping and logging in Article 12 and transparency and information for deployers in Article 13. The evidence overlaps substantially, but whether your documentation meets any legal obligation is a determination for qualified counsel, not for a framework mapping.
How long should AI decision records be retained?
The AI RMF does not set a retention period. Retention is a decision for your counsel and your sector's record-keeping requirements, weighed against data minimisation duties. One practical constraint: if you hold records as a hash chain, verification can only run from the earliest record you still hold, so truncation and verifiability trade against each other.
Can I use my existing MLOps logs as Measure and Manage evidence?
Sometimes, but usually not without work. Debug logs tend to have an unstable field set, carry raw payloads you would rather not retain, sit on a rotation schedule set for operational reasons, and are writable by anyone with access to the log store. A purpose-built record with a fixed field set, pseudonymised identifiers and an append-only chain answers the questions a reviewer actually asks.
What is the difference between the AI RMF and the AI RMF Playbook?
The framework document states the functions, categories and subcategories as outcomes. The Playbook is a companion resource offering suggested actions, references and documentation ideas for achieving those outcomes. The Playbook is where you look when you know the outcome you need and want candidate practices for reaching it.
Does capturing decision records require changing my model or pipeline?
It should not. The serving wrapper already holds every field the record needs at the moment it returns a response, so capture can be a REST call or an emitted webhook from that point. Auditrek ingests over a generic REST and webhook layer with documented payload mapping guides for AWS SageMaker and Azure ML, so no model migration is involved.
Sources
- NIST AI Risk Management Framework
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1
- NIST AI RMF Playbook, AI Resource Center
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), official text
- ISO/IEC 42001 artificial intelligence management system standard
- SR 11-7: Guidance on Model Risk Management