Just Because an AI Agent Can Do Something Doesn't Mean It Should Be Allowed to.
I've been thinking about a distinction that seems increasingly important as AI agents become more capable.
Capability is not authorization.
An AI coding agent might technically be able to:
- Delete a database.
- Modify production infrastructure.
- Change an authentication flow.
- Deploy an application.
- Rotate credentials.
- Merge a pull request.
- Change a security policy.
But technical capability doesn't mean the agent should be allowed to perform those actions.
We don't normally design security systems this way.
A human engineer may have access to a production environment.
That doesn't mean every action they can technically execute is automatically authorized.
There are policies.
Permissions.
Approvals.
Audit trails.
Separation of duties.
So why should AI agents be different?
This becomes particularly interesting when an AI agent moves from generating code to actually executing changes.
The old model was:
Human → AI → Code
The emerging model looks more like:
Human
↓
AI Agent
↓
Tools
↓
Infrastructure
And that creates a new control problem.
The AI may decide:
"I need to modify this production configuration."
But the system should be able to independently determine:
"Is this agent authorized to perform this specific action under the current policy?"
That's a fundamentally different architecture.
The model should reason.
The control plane should authorize.
The runtime should execute.
The audit system should record what happened.
This separation is becoming one of the principles I'm exploring in NAEOS.
AI Agent
|
| proposed action
↓
+---------------+
| Policy Engine |
+-------+-------+
|
Allow / Deny /
Require Approval
|
↓
Runtime
|
↓
Audit Evidence
The important part is that the AI doesn't get to decide its own authority.
Its reasoning can propose an action.
But authorization should be deterministic and external to the model.
I think this distinction will become increasingly important as we move from AI-assisted development toward autonomous engineering.
What actions should an AI agent never be able to authorize for itself?
Building autonomous AI agents requires a shift from technical capability to strict boundary setting, as giving models unrestricted execution power invites severe operational and ethical risks. True product reliability isn't measured by how many tasks an agent can perform unsupervised, but by how effectively human-in-the-loop guardrails prevent high-stakes edge cases from turning into catastrophic failures.
I run a chain of nine agents that publishes products without me, and I'd add one thing from the operating side: the three branches of your policy engine do not fail equally. Allow and Deny are deterministic and they hold. "Require Approval" is not a control — it's a control-shaped hole with a human in it, and the human degrades.
Concrete: my gate sends one plain-language question with two buttons to my phone. I approved something in 46 seconds without opening the context I was supposed to read. On a system I built myself, that exists for no other reason than to protect me. Nothing about the architecture failed. The authorization was external to the model, deterministic, audited — and worthless, because the deciding component was a person doing something else at the time.
Meanwhile the hard denials worked perfectly, including one I fought against: my browser tooling is granted read-only, so the agent can see a page but cannot click. It's a blunt rule that cost me real work. It also never once degraded.
So my answer to your question is less about which actions and more about which failures: an agent should never be able to self-authorize an action whose failure is silent. My worst incident wasn't an agent doing something forbidden. It was a publish step that failed and returned exit code 0, so the chain recorded a product as live that never existed. The policy engine would have happily allowed it — it was an authorized action that simply didn't happen.
Audit evidence of "action allowed" isn't evidence the action succeeded. If your audit layer records proposals and verdicts but not verified outcomes, you get a complete, signed, entirely fictional history.
This distinction also matters outside engineering. In the product we are building, AI can compare project information and flag a mismatch, but it should not decide on its own that the difference is an error. Sometimes that difference is intentional, and the context lives in a client conversation.
For me, AI should prepare and surface information. A person should approve anything that affects a client, money, or a final decision.
The key boundary seems to be irreversible or externally visible actions.
Would production, credential, and permission changes always require human approval?
Credential and permission changes — yes, always, no exceptions. Those are policy-defining, not just risky: get one wrong and every downstream authorization check is now working off bad data. Production config is the one that's genuinely graded for us: a feature flag toggle or a config value with an instant, obvious rollback gets auto-approved; anything touching schema, infra topology, or a change we can't cleanly diff before/after goes through approval. The distinction ended up being less "is it production" and more "can I show you exactly what changed in a way you'd trust without re-deriving it yourself."
This matches something I run into daily running an AI agent for real ops work — the split that's worked well in practice is a hard "never self-authorize" list rather than a graded policy: irreversible/destructive actions (deleting data, force-pushing, dropping infra), anything that moves money or touches payment/financial credentials, and anything that becomes visible to a third party the moment it executes (sending an email, publishing a post, merging a PR) — because you can't un-ring that bell even if the audit trail is perfect afterward. Everything reversible or purely local, the agent just does. The interesting edge case for us has been actions that are technically reversible but socially irreversible — like posting a comment on someone's behalf. The audit log proves what happened, but it doesn't undo the read receipt. Curious if your policy engine treats "externally visible" as its own risk dimension, separate from "destructive"?