Evidence and Audit Readiness; proving you run AI safely

This is Part 4 of a four-part TQS series on “Securing AI Systems.” Read: Part 1 — Model Supply Chain, Part 2 — Red-Teaming & Evaluations, Part 3 — Runtime Defences.

From controls to credible evidence

Regulated buyers—and increasingly, all enterprise buyers—ask not just what controls you claim, but what evidenceproves they are working. In the EU, the AI Act makes this explicit for high-risk systems: providers must maintain technical documentation, implement automatic logging and run post-market monitoring. Even if your current use is not high-risk, designing your programme to those expectations shortens due-diligence cycles and avoids rework when scope evolves. EUR-Lex

Keep a live “what’s running” dossier

Maintain a concise page per live endpoint that a non-developer can read: the model and version, intended purpose and decision boundaries, known limitations, dependencies, last evaluation results and links to the MBOM and attestation. This turns scattered artefacts into a navigable dossier for auditors, security reviewers and account teams. ISO/IEC 42001 provides the governance backbone for such a system; NIST’s AI RMF and GenAI Profile offer concrete actions to populate it. ISO+2NIST+2

Logging that stands up in audits

Design logs to answer three questions: what happened, under whose authority, with which inputs. Capture model id/version, policy decisions, retrieved context identifiers (not raw content), tool calls and outcomes, correlation IDs across services and—where permitted—minimal PII to support investigations. Align retention with risk. Article 12 sets the direction: high-risk systems must enable automatic event recording over their lifetime, and providers must retain logs for at least six months unless other law specifies otherwise. Build this into your platforms now. Artificial Intelligence Act+1

Change control you can show

Keep a change register for models and guardrails. For each release, record the risk assessment, evaluation outcomes that justified promotion, approvals with separation of duties and the rollback plan. Sign artefacts and store signatures alongside the MBOM in your registry. NIST’s SSDF lays out the discipline; many internal auditors already use it as a reference for software and will expect the same for AI. NIST Computer Security Resource Center

Incidents: define, rehearse, improve

Define what counts as an AI incident in your organisation: privacy leakage, unfair outcomes, sustained hallucination, over-permissive tool use. Write playbooks that span product, engineering, security and communications. When incidents happen, run post-incident reviews that examine both the model and the surrounding controls, then feed lessons back into your evaluation suite and policies. The NIST GenAI Profile explicitly calls for continuous risk management and post-deployment monitoring; this is where it becomes operational. NIST Publications

Post-market monitoring as a habit, not a scramble

Track performance, safety and fairness by segment over time. If your user base shifts—new languages, new document types—re-evaluate and update thresholds. Article 72 requires providers of high-risk systems to establish and maintain a post-market monitoring system based on a documented plan; the Commission will publish a template. Aligning early gives you fewer surprises and better resilience. Artificial Intelligence Act+1

Linking security state across systems

Identity, device and session risk often sit outside the model boundary. Where you already exchange security events using Security Event Tokens (SET, RFC 8417)—or where your vendors support the OpenID Shared Signals Framework (SSF)—you can propagate risk into model access policies and broadcast model-related risk back to relying parties. This isn’t an AI Act requirement; it’s a practical way to make your Zero-Trust perimeter react to what models observe. IETF Datatracker+2RFC Editor+2

Packaging your assurance

Assemble an evidence pack that buyers and internal risk teams can consume in one sitting. Include: supply-chain artefacts (provenance statements, MBOMs), recent evaluation reports (task fitness, robustness, privacy, fairness), runtime controls (policy summaries, logging schemas, isolation diagrams), change control records and post-market monitoring outputs. Tie each item to a recognised standard—NIST AI RMF/GenAI Profile, ISO/IEC 42001, ISO/IEC 23894, NIST SSDF—so reviewers know what “good” looks like. The aim is not to drown them in artefacts; it is to show you can run AI competently, repeatedly and transparently. NIST Computer Security Resource Center+4NIST+4NIST Publications+4

Catch up on Part 1 — Model Supply Chain, Part 2 — Red-Teaming & Evaluations and Part 3 — Runtime Defences.

Sources

1. EU AI Act (Official Journal / consolidation). EUR-Lex
2. Article 11 — Technical documentation; Annex IV — required contents. Artificial Intelligence Act+2artificial-intelligence-act.com+2
3. Article 12 — Record-keeping (logging). Artificial Intelligence Act+1
4. Article 72 — Post-market monitoring; impact analysis. Artificial Intelligence Act+1
5. NIST AI RMF & GenAI Profile. NIST+1
6. ISO/IEC 42001 and ISO/IEC 23894. ISO+1
7. NIST SSDF (SP 800-218). NIST Computer Security Resource Center
8. RFC 8417 (Security Event Tokens) and OpenID SSF 1.0. IETF Datatracker+2RFC Editor+2