Question 4 feels like the hardest one to operationalize.
Curious whether keeping the raw provider payload + repair diff is mainly expensive in storage, or whether the bigger problem is deciding which inputs and intermediate state are actually worth preserving for attribution.
Mostly the latter. Storage is the easy cost to see; deciding what represents causality is harder.
A raw payload proves what the provider returned, but not which changed input moved the result. I currently preserve state at decision boundaries: retrieved evidence, normalized inputs, prompt/model/tool versions, and any repair that changed the output.
The raw payload is fallback evidence. The boundary diff is what should answer "why did this number change?"
That makes sense. The distinction between preserving evidence and preserving causality is the more interesting part here.
About
Most "human-in-the-loop" systems only add an approval button. The agent can still change or bypass the action. I built Agent Approval Gate to bind approval to the exact validated payload and leave a complete audit trail.
3 Comments
Question 4 feels like the hardest one to operationalize.
Curious whether keeping the raw provider payload + repair diff is mainly expensive in storage, or whether the bigger problem is deciding which inputs and intermediate state are actually worth preserving for attribution.