Supervised autonomy: how AI agents should earn trust
Every AI agent demo ends the same way: "...and it does all of this automatically." Every experienced operator hears the same thing: "...and you'll find out what it broke later." The gap between those two reactions is the actual product problem in agentic software, and "full auto on day one" is the wrong answer to it.
Autonomy isn't a feature; it's a state of trust
When a human delegates to another human, autonomy is never granted up front. A new hire drafts the email and you read it before it goes out. After twenty good drafts, you stop reading. After fifty, they send without asking - for that kind of email. You still review the contract negotiation. Trust is earned per task, from evidence, and it stays revocable.
There's no reason software agents deserve a lower bar - especially agents acting on surfaces you can't easily undo: a reply posted publicly under your app's name, a release rolled out to real users, a price change on a live product.
The graduation model
A trust model worth copying has four properties:
- Start supervised, always. The agent drafts; the human approves. The default is a proposal, never an action.
- Measure the approvals. Approved-without-edits is the signal. Fifteen consecutive review replies approved untouched is evidence; a confident demo is not.
- Graduate per capability, not globally. An agent that has earned auto-posting review replies has earned nothing about rollouts. Each capability climbs its own ladder.
- Make it reversible and auditable. One switch back to supervised, and an immutable log of every action taken - so trust can be checked, not just felt.
Why supervision makes the agent better, not just safer
The approval loop isn't only a brake - it's training signal. Every edit the human makes is a correction the agent can learn from; every unedited approval confirms a pattern. An agent that skips the supervised phase skips the phase where it learns your voice, your risk tolerance, your app's specifics. Full-auto-first agents aren't just riskier; they plateau dumber.
The economics follow the trust
One more consequence worth making explicit: in this model, the thing worth paying for is autonomy, not visibility. Seeing what the agent proposes - the see-and-approve loop - should be free, because it's how trust gets built. What's valuable is the hours returned when a capability graduates and runs without you.
This is how Nalya is built
Nalya applies this model to the operational work of a live Play app. Review replies, listing changes - each starts supervised, each tracks its own approval record, each offers graduation only when the record supports it, and each can be demoted with one tap. Guardrails stay armed even at full auto: every action lands in the log, and even watch-only surfaces pull their weight - she tracks your staged rollouts and flags every status change. Autonomy you can grant, verify, and take back - that's the only kind worth shipping.