Article · Governance & Risk
How to Audit an AI Agent's Decisions
Auditing an AI agent means being able to take any decision it made, from any week, and reconstruct what it received, which rule it applied, what it did, and who authorized it. If you cannot do that for a specific case in minutes, you do not have an audit problem, you have a system problem, because the record that makes it possible is designed before you go live, not recovered afterwards. This article explains what that trace must hold, how it is reviewed without going case by case, and what an auditor will ask for when they come.
What the agent’s trace must record
A useful trace is not an unreadable technical dump, but the reconstructible account of each decision, written so that whoever has to answer for it can read it, and that person will be a director or an auditor, not an engineer. For each agent action, the record should hold:
| Element | What it answers |
|---|---|
| Input and context | What the agent received and what information it consulted to decide |
| Rule applied | Which process criterion it used and why the case fit it |
| Action executed | What it did exactly, on which system, with what result |
| Escalation or approval | If it went through a person, who signed and when |
| Identity and time | Which agent acted, with which credential, at what moment |
The column most often forgotten is the rule applied. Recording that the agent did something is easy; recording why it did it, against which process criterion, is what turns the log into an audit trail. That trace is produced by the technical layer I cover in AI agent observability. Here I deal with what you do with it when it is time to review.
The test of a good trace: reconstruct any case at random
The exam I apply is simple. I pick a consequential action from a few months back (a payment, a credit note, a customer reply) and ask for the whole story: which document triggered it, which rule validated it, who approved it if required, what result it had. A well-designed system returns those elements within the agreed time. If the answer requires an engineer to “interpret the logs” or reconstruct by hand what happened, the trace is incomplete, however detailed it looks.
This exam separates two things that get confused: storing data and being able to audit. Many systems store enormous amounts of information and still cannot answer a specific question about a specific case. The measure of a good trace is not its volume, it is that any case at random reconstructs end to end without archaeology.
How review works: by sampling, not case by case
Auditing is not looking at every decision, which is neither possible nor needed. You review by sampling, with logic much like a financial audit. You take a sample of cases, reconstruct each one fully, and look for three things: that the rule applied was correct, that the limits held, and that escalations happened when they should. The sampling rate follows the risk: denser on irreversible actions and new processes, lighter where months of history show stable behavior.
What you look for is not only the isolated error but the pattern: a rule that fails on a certain class of cases, a limit being brushed against too often, escalations that fail to fire when they should. Each finding turns into an adjustment of the process or of the agent’s permissions and controls. The audit is not a closing formality, but the mechanism by which the system learns where to tune. And it is the evidence that justifies widening autonomy, because without a reviewed record any widening of permissions is an act of faith.
What an auditor will ask you for
When an auditor, internal or external, looks at a process run by agents, the questions are not technical, they are the usual ones, applied to a new actor. It pays to arrive with the answer ready:
- Who is accountable for the process. A name, a person with authority to change the rule and to answer for the result. The agent is a tool, not a subject to hand responsibility to.
- A case reconstructed end to end. The one they choose, not the one you prepared. This is the test from the section above, and the one that fastest reveals whether the system is genuinely auditable.
- Evidence of the controls. That the limits existed and were applied: a case that exceeded a cap and was escalated, a permission the agent did not have and therefore did not execute.
- The change log. When a rule, a permission or the model changed, who approved it, and what effect it had. An agent with no change history is as opaque as one with no decision trace.
- The handling of personal data. What data the agent touches and on what basis, a ground that overlaps with AI agents and data privacy and is better settled before anyone asks.
If your provider cannot show you all of this about any case from last week, the compliance conversation is premature, because the problem comes earlier.
How it fits with permissions, security and governance
The audit is the evidence leg. The limits it audits are designed in permissions and controls. The threats the trace lets you detect are in agent security. And the framework that decides what gets audited, how often, and who is accountable is AI agent governance. If the board wants a certifiable reference for that management system, it is what ISO/IEC 42001 standardizes. Designing the trace from the start, rather than after the first incident, is what turns an agent operation into something a serious board can approve, and it is part of what shows up when preparing a company for agents. The full context of what an agent executes and with what guarantees is in AI agents for business.
Frequently Asked Questions
Can I audit an agent if I am not technical?
Yes, the same way you audit accounts without being an accountant: by demanding the record and sampling it. You ask for specific cases reconstructed end to end (what the agent saw, which rule it applied, who reviewed the escalations) and verify that the controls worked. Readability of the trace for a non-technical person is, in fact, one of the requirements you should demand.
What is the difference between keeping logs and being able to audit?
Keeping logs is accumulating information, and auditing is being able to answer a specific question about a specific case. A system can store enormous amounts of data and still not let you reconstruct why it made a decision. The test is not how much it keeps, but whether any case at random reconstructs fully without manual work.
How often should an agent’s decisions be reviewed?
Continuously by sampling, with density adjusted to risk: more frequent on irreversible actions and freshly deployed processes, lighter where history shows stability. On top of that, a full review every time the model or an important rule changes, because a change can alter behavior on cases that used to work.
What is the first thing an auditor will ask for?
That you reconstruct end to end a case of their choosing, and that you name who is accountable for the process. Those two answers give an immediate sense of whether the system is governable. The rest (evidence of controls, change log, data handling) rests on those first two existing.