How I decide when an AI coding agent is actually done

A practical AI coding agent workflow for turning done from a confident summary into a completion claim that another person can inspect.

In short

• An agent's final message is an interface, not a source of truth.

• Every completed run should state what changed, the evidence, the covered scope, and what was not verified.

• More autonomy is useful only where the evidence and the limits can expand with it.

The task list was empty. The process was still running

I was running an agent task when the work inventory reported that nothing remained, yet one minute later the operating system still showed a polling process alive, and more than 3 minutes after the run began that process was still there. Both reports were accurate. They described different things, which was the problem: the task system knew that its own list was empty, but nobody had checked whether the work had left anything running outside that list (the process registry and the operating system were two different boundaries). That incident changed how I use AI coding agents because the final response, useful as it is, cannot be the unit of progress when the same system may have edited the code, written the tests, updated the documentation, and produced the explanation. Every artifact can agree. Every artifact can also share the same mistaken interpretation of the original request (agreement is not independence).

The dangerous property isn't that an agent may be wrong. Developers have always been wrong. It is that an incomplete change can now arrive quickly with code, tests, documentation, and a persuasive account of the work all supporting one another, while the missing requirement sits outside the story they tell. Production became cheaper. Assurance did not. So what does done mean when the actor making the completion claim also produced most of the evidence behind it?

Why done broke for AI coding agents

Most software teams already have tests, pull requests, continuous integration, code review, acceptance criteria, and a Definition of Done, and none of those controls became obsolete when agents started writing code. The operating conditions changed instead: an agent can produce more code than a reviewer can understand line by line, lose part of a request during a long run, and continue fluently from whatever remains in context.

It can also run the narrow suite that covers its implementation and report that "the tests pass" without saying which tests never ran. Several agents introduce another seam because each one may finish its own task correctly while two changes collide on a shared file, identifier, contract, document, or external system (local correctness does not guarantee convergence).

The old workflow often treats completion as a property of the task card: the card reached the last column, therefore the work is done. Agentic software development needs a stronger relation in which the intended outcome connects to the work that implemented it, the check that observed it, and the decision that accepted it.

If that path breaks, the change may still be useful or almost complete. It isn't proven complete, and naming the break is cheaper than reconstructing it after the next agent has already built on top.

What should an AI coding agent report?

I now ask every finished run for four fields:

DONE
  What became true.

PROOF
  What was executed or observed, and where the result lives.

SCOPE
  The commit, environment, requirements, and surfaces covered.

NOT VERIFIED
  What was not checked, could not be checked, or remains uncertain.

This is the smallest version of Proof of Done, and each field closes a different escape route. DONE describes the resulting state rather than the agent's activity. "Edited three files" is activity, while "UTF-8 report export now preserves non-Latin customer names" is a result that you can challenge against a requirement, a test, and the running system.

PROOF names what was observed and gives it an address: a test by name, a command and exit code, a CI run, a commit, a trace, a deployment identifier, a screenshot tied to a scenario, or a production observation. The medium matters less than reachability because evidence that nobody else can resolve exists only inside the conversation that produced it (and that conversation will eventually disappear from working memory).

SCOPE prevents narrow success from silently becoming a broad claim. A passing check on one commit does not prove the next commit, a staging smoke test does not become production evidence, and Linux coverage says nothing about Windows unless the report explicitly establishes that relation.

NOT VERIFIED keeps uncertainty honest. NOT VERIFIED: none within the stated scope is a valid answer, but silence is not, because silence hides two opposite states: everything relevant was checked, or nobody checked what was omitted.

Which one happened? A blank field cannot tell you, and when you have to reopen the entire chat to find out, the report has already failed its job.

A practical Proof of Done example

Consider the usual agent report:

Done. Implemented UTF-8 export support. Tests pass.

It sounds complete. It is also difficult to inspect because it names neither the result's boundary nor the observation behind the claim.

Now compare it with this report:

DONE: UTF-8 report export implemented at 5f21ac3.
PROOF: test_export_preserves_utf8 passed in CI run 1842.
SCOPE: Linux / Node 24 / REQ-014.
NOT VERIFIED: Excel import on Windows.

The identifiers are illustrative, but the difference is operational. The first report asks for trust, while the second gives the reviewer places to disagree: if the test does not exist, the proof fails at a named address, and if the CI run belongs to another commit, the evidence no longer matches the scope. You don't need to understand every implementation detail before you can identify that mismatch (the address does the first part of the review).

Windows support may also have been part of the request. In that case the open limit blocks acceptance, not because the whole change failed, but because the report has finally exposed the exact requirement that remains uncovered (a missing receipt now has a location).

Would this report survive after the chat is gone? If you can still resolve the requirement, commit, test, and CI run, you can challenge the claim without depending on the agent's memory.

When you can open the requirement, follow it to the commit, inspect the named check, see the environment, and read the remaining limit without asking the original agent to explain itself, the report has become durable enough for a reviewer, another agent, or a release gate to use after the conversation has disappeared, and it has done so without preserving private reasoning or turning every intermediate thought into permanent context that the next run must pay to read.

The six states hidden inside done

One word currently carries at least six different engineering states:

State

What it establishes

generated

An artifact exists

executed

Code or a check ran

passed

A successful result was observed

verified

An appropriate check demonstrated that it can distinguish success from failure

validated

The result addresses the intended user or system outcome

accepted

Required coverage is closed and every remaining limit is explicit

The common jump is from generated directly to done: the feature exists, the page opens, or the command returned zero, so the task is closed. Once you name the intermediate states, partial work becomes less embarrassing and more useful because you can distinguish a change that was generated but not executed, passed but not independently verified, or implementation-verified while the product hypothesis remains unvalidated (one word no longer hides three decisions).

Those aren't synonyms for failure. They are accurate positions in the work, and an agent should be allowed to stop at one of them without promoting uncertainty into completion merely to produce a clean final message.

How to add Proof of Done to an AI coding workflow

The protocol works without a new platform or a large process. I use five steps, and I add machinery only after a real failure has shown where these five stop carrying enough evidence.

1. Define outcome and scope: Before the agent edits code, state what should become true and what it must not change. When you write "improve the export," you give the system room to make plausible product decisions nobody approved, while a bounded request names the visible behavior, the affected surface, and the exclusions (the non-goals are part of scope, not an appendix).

Acceptance criteria help only when they describe observable behavior. A checklist filled with implementation guesses merely moves the ambiguity into another file, where it acquires the appearance of a specification without becoming one.

1. Name evidence first: Decide how the change will be checked before the work starts by naming the test class, scenario, invariant, or observation that would support the claim. You don't need to invent exact fixtures for a system nobody has inspected yet because the agent may need repository context before those values become honest (the check needs a job before it needs its final data).

This separates evidence design from evidence fabrication. The implementation can still teach you that the planned check was wrong, but changing the check then becomes a visible decision rather than a silent convenience.

1. Bound authority: Autonomy should expand where failure is reversible and the boundary is mechanical, so a coding agent can edit a branch, run tests, and prepare a pull request without asking after every tool call. You can grant that freedom without granting permission to publish, spend money, destroy data, or expand scope merely because those actions are available through a tool.

The strongest guard is structural. A credential that cannot reach production is stronger than a paragraph telling the agent not to use production, because the credential still works as a limit after the paragraph has fallen out of context.

1. Require four fields: The report should be produced by the work rather than reconstructed afterwards: commands record their exit codes, tests publish their names, and deployments produce identifiers. Manual evidence added after the run is automation debt, and a protocol that takes longer to maintain than the change itself will eventually be skipped (the cost is evidence about the design).

2. Inspect freshness and residue: The producer of a claim should not be its only judge. Independence can come from a deterministic runner, a contract test maintained at another boundary, an isolated reviewer, an external service, or production telemetry, while a second agent counts as independent only when its evidence path differs from the first agent's path.

Proof also expires. I once had a paragraph claiming that four backlog requirements were still open, with citations to commit-pinned rows that did indeed say open; the citations resolved perfectly, but the current repository had already closed the requirements, and the statement remained stale for 2 days and 6 hours before I corrected it.

Reachable evidence can prove an old state.

Finally, inspect residue: background processes, temporary infrastructure, uncommitted files, queued jobs, open worktrees, and external operations that survived the task. The empty inventory and the live poller were both true because completion had been measured inside one boundary while residue lived outside it (nothing was lying, but the system was looking in the wrong place).

What Proof of Done does not mean

Proof of Done is not mathematical proof of correctness, which most product teams cannot produce and which would make the protocol useless as an everyday standard. It is a bounded claim: this outcome, on this change, in this environment, supported by these observations, with these limits.

It does not require the same pipeline for every change because proof depth should follow impact, irreversibility, uncertainty, and coordination. A typo and a production database migration should not cross the same gates, and if the protocol adds 20 minutes of recordkeeping to a 5-minute edit, the implementation is wrong.

It does not mean reviewing every generated line either. That would return much of the time the agent saved, while the useful move is to put routine assurance into mechanical checks and reserve human attention for meaning, architecture, visual judgment, external publication, destructive action, and acceptance.

It also does not prove that a product decision was right. A team may build and release exactly what it intended without knowing whether customers will use the result, which means delivery proof and product validation remain different evidence available at different times.

The smallest useful change

The next agent run does not need a new orchestration platform. I would start with a template: add the four fields to the result contract, require a real address in PROOF, refuse a blank NOT VERIFIED field, and then check whether the evidence belongs to the scope the report claims (nothing else has to change on day one).

I wrote The Proof of Done Manifesto, the standard for building software with AI agents, after these failures appeared in my own runs. The full document adds the intent, execution, and evidence graphs, three gate types, proof expiry, and a reference implementation, but the four-field protocol survives all of that tooling being removed.

Faster code generation compounds only when acceptance becomes cheaper too.