Healthcare AI documentation requirements without raw clinical data
If you are documenting a clinical AI system, triage scoring, imaging prioritisation, readmission risk, the hard part is rarely the logging itself, it is logging enough to evidence what the model did without copying patient data into a second system.
Auditrek is not open yet, so anything on this page about what it does is what it is being built to do.
What healthcare teams are actually asking when they ask about AI documentation requirements
The question almost always carries two obligations in one hand.
The first is record-keeping and logging: what the system did, when it did it, on which model version, and whether a person intervened, retained across the operating life of the system. In the EU AI Act that sits under Article 12. The second is transparency and provision of information to deployers, the instructions-for-use side, which sits under Article 13. Auditrek is built for the first, the event-level record, and its report generator produces output structured against the fields of both. It is designed to support your documentation of those obligations. It does not assess whether your practices satisfy them, and every report it produces carries a visible footer stating that the report does not constitute legal advice and has not been reviewed by qualified regulatory counsel.
Where the evidence sits today
- Inference-level telemetry in the MLOps platform, usually AWS SageMaker or Azure ML, retained on a rolling window that was sized for debugging rather than for an inquiry two years later.
- Clinician overrides as free text in the clinical record or in a governance review spreadsheet, unjoined to the inference that triggered them.
- Model versions in a registry the compliance function cannot read, and which is rewritten on every redeploy.
- The governance narrative in committee minutes and slide decks, written after the fact by people reconstructing what happened.
Each artefact exists. None of them is append-only. Nothing binds one decision to one model version, one confidence value, one timestamp and one named override, which is the join a supervisory question asks for.
Why the obvious fix makes the problem bigger
The instinct is to pipe the inference payload into a governance tool and let the tool hold everything. In a clinical pipeline, the payload is the patient. Copying diagnosis codes, identifiers and note text into a vendor environment to satisfy a record-keeping obligation trades a documentation gap for a new processing relationship over special-category data, with its own lawful basis, contractual and residency questions to answer before the first event is logged.
The decision in front of you
Four routes are usually on the table: build an append-only store yourself on locked object storage, extend the existing clinical audit log to cover model events, buy a broad governance platform that expects payload ingestion and changes to the inference path, or log a minimised record beside the pipeline and keep the clinical data where it already lives. Auditrek is the fourth. It sits off the production inference path, accepts events over REST or a webhook, and refuses payloads that look like raw clinical content.
How a clinical model gets a defensible record
- Register the AI system and record the DPA
Each model becomes an ai_system in your org. No API key and no webhook secret are issued until a dpa_confirmation with a signed_at value exists for that system. The check runs server-side and returns a 403 if the DPA is unsigned, so a clinical pipeline cannot be connected ahead of the paperwork by clicking past a screen.
- Map the payload from SageMaker or Azure ML
Ingestion is a generic REST and webhook layer with documented payload mapping guides for AWS SageMaker and Azure ML. There are no bespoke platform SDKs in the first release, which is the point: your serving code keeps its current shape and an inference wrapper or an existing event hook emits the record.
- Hash the input before it leaves your network
The record field is input_hash, not input. You compute the SHA-256 of whatever the model saw, the feature vector, the note, the image reference, and send the digest. The hash is enough to demonstrate later that a specific input produced a specific output, and the clinical content never crosses the boundary.
- Let the ingestion gate reject what should not be there
POST /api/decisions inspects each payload for recognisable raw sensitive-field patterns, the named ones being SSN formats, diagnosis code patterns such as ICD-10, and unredacted credit bureau fields, and rejects the request with a 422 and a clear error body before any database write. Subject identifiers are pseudonymised by default rather than as an option someone has to remember to enable.
- Record the model, the confidence and the override
Each accepted event carries model id and version, the output, confidence, timestamp and a human override flag. The override flag is what turns human oversight from a claim in a policy document into a queryable field: you can count how often clinicians departed from the model, per system, per date range.
- Write it into the chain
Every record gets a sequence_number and a prev_hash from the last record for that ai_system, and a hash computed as sha256(prev_hash + payload), all written in one transaction. The store is append-only per system, so the way to add context to an earlier decision is a later record, not an edit.
- Verify, then generate the document
The verification endpoint walks the chain from sequence 1 and recomputes each hash, returning a tamper status of valid or broken with the first broken sequence number, rather than reading a stored flag. The dashboard shows decisions logged, systems connected, last report generated and verification status. The report generator then produces PDF or HTML for one system and one date range, structured against Article 12 and 13 fields, with the non-legal-advice disclaimer embedded.
Why this shape suits clinical AI in particular
The ledger holds no diagnosis codes
Rejection happens at the API boundary, before the write, so the assurance you give your data protection officer is a mechanism with a test behind it rather than a policy commitment about what engineers will remember not to send.
Pseudonymous by default, not by configuration
Subject identifiers are pseudonymised at ingestion as the default path, not a setting someone has to find and switch on before the first clinical event is logged.
The model version travels with the decision
A clinical question is usually about a specific case on a specific day. Because model id and version are fields on the record, and the record is fixed in sequence, you can answer which version was serving without reconstructing a deployment history from CI logs.
Clinician override becomes evidence
The human override flag is captured per decision, so oversight can be described with counts over a date range instead of being asserted in prose after the fact.
Tamper status is recomputed, not asserted
The verification endpoint rebuilds the hash chain server-side on request. That distinction matters in a room where someone asks what stops a record being quietly changed after an adverse event review.
Nothing changes in the inference path
Recording happens over REST or a webhook beside the pipeline. There is no model migration, no proxy in front of serving, and no new dependency between a clinical model and a vendor being reachable.
What is in the record and what stays with you
| Element of the decision | What the Auditrek record holds | What stays in your environment |
|---|---|---|
| Model input | A SHA-256 input hash you compute before sending | The clinical input itself: notes, images, codes, observations |
| Patient identity | A pseudonymised subject reference, applied by default at ingestion | The patient record, in the systems that already hold it |
| Model output | The output value, confidence and timestamp | Whatever narrative or interface the clinician actually saw |
| Model identity | Model id and version as reported at inference time | Weights, training data and registry access |
| Human involvement | A human override flag on the record | The clinician's free-text rationale in the clinical record |
| Integrity and ordering | sequence_number, prev_hash and SHA-256 hash per ai_system | Nothing extra: verification recomputes from stored records |
| Raw sensitive fields | Rejected with a 422 before any database write | They remain in the systems already governed to hold them |
What Auditrek does not do
Auditrek records what your AI system did. It does not triage, score or interpret patients, it does not evaluate model quality, and it does not tell you what any regulation requires of your organisation. The report templates are descriptive and structural: they set out which decisions were logged, by which system and version, over which period, with which override activity, and they carry a fixed disclaimer on every page stating that the output does not constitute legal advice and has not been reviewed by qualified regulatory counsel. Your counsel and your notified body decide what is sufficient. We give you the record they will ask to see.
If your work spans other regulated models, the argument is close but the evidence differs. There are separate pages on the append-only record behind an automated credit scoring decision and on decision records for underwriting and pricing models.
Questions
Does Auditrek store patient data?
It is built not to. Model inputs arrive as a SHA-256 hash you compute before the call, subject identifiers are pseudonymised by default at ingestion, and the ingestion API inspects each payload for recognisable raw sensitive-field patterns, the named ones being SSN formats, diagnosis code patterns such as ICD-10, and unredacted credit bureau fields, rejecting the request with a 422 before any database write.
What happens if an engineer accidentally sends a diagnosis code?
The request fails with a 422 and a clear error body, and nothing is written. The check is server-side in POST /api/decisions, so it applies regardless of which client or pipeline sent the event.
Will these reports satisfy a supervisory authority under the EU AI Act?
We do not make that claim and you should not accept it from any vendor. The report generator produces PDF and HTML output for one system and one date range, structured against Article 12 and 13 fields, and is designed to support your documentation of those obligations. Every report carries a visible footer stating that it does not constitute legal advice and has not been reviewed by qualified regulatory counsel. Sufficiency is a question for your regulatory counsel.
Can we connect a clinical pipeline without signing a data processing agreement first?
No. The connect endpoint checks server-side that a dpa_confirmation has a non-null signed_at before issuing an API key or webhook secret, and returns a 403 if it does not. The gate cannot be bypassed from the client.
How would we show a record was not edited after an incident review?
The verification endpoint walks the chain for that record's ai_system from sequence 1, recomputes each hash from the stored prev_hash and payload, and returns a tamper status of valid or broken with the first broken sequence number. It is a recomputation on request, not a stored flag someone could set.
Does this require changes to our serving stack?
Ingestion is a generic REST and webhook layer, with documented payload mapping guides for AWS SageMaker and Azure ML. There are no bespoke platform SDKs in the first release and nothing sits in the production inference path.
Which fields make up a decision record?
Input hash, model id and version, output, confidence, timestamp and a human override flag, stored per ai_system as SHA-256 hash-chained append-only records with a sequence number and the previous record's hash.
Is it available now?
Auditrek is pre-launch and running a waitlist. Sign-in is email and password, and the service is hosted on Amazon Web Services.