Autonomous AI without delegating accountability

01What an autonomous agent actually does

An autonomous AI agent does more than generate content. It can communicate with customers, update records, initiate transactions and make decisions with real business consequences. The moment it holds a credential, it is not a chatbot — it is an actor inside your systems.

That distinction is where most governance goes wrong. A model that drafts text has a review step built in by default: a human reads the output before it goes anywhere. An agent with an API key has no such step unless you build one. The output *is* the action.

02Five things every agent should have

This is the minimum. If you cannot name all five for an agent already running in your company, you do not have a governed agent — you have an outage waiting for a trigger.

Note what these five have in common: none of them is a model capability. They are all organisational decisions, and they are all cheaper to make before deployment than after an incident.

03Why "human in the loop" is not an answer

Simply saying there is a human in the loop is not enough. It is the most common answer to the accountability question and the least examined. The phrase describes a diagram, not a control.

For the human to be a real control, three things must be true at the moment of the decision, not in the design document:

1. They have the information. A prompt that says "Approve?" with no context is not a review — it is a rubber stamp with extra steps. The reviewer needs to see what the agent saw and what it intends to do.
2. They have the time. Ten approvals a day get read. Four hundred get clicked. If the approval rate is 99%, the loop has already degraded into a formality, and the one case that mattered went through with the rest.
3. They have the authority. The reviewer must be able to say no and have that stick — without escalating to the person whose quarterly target the agent is helping to hit.

And one more, the one people forget: the intervention has to happen before the action becomes irreversible. An approval screen shown after the email is sent, the payment cleared or the record overwritten is not a control. It is a receipt.

04The principle: authority is delegable, accountability is not

Companies can delegate tasks and decision-making authority to AI. They cannot delegate accountability. That is the whole argument, and it is not a philosophical point — it is how the world already treats every other kind of automation and every other kind of agent.

When a bank's system approves a loan it should not have, the bank answers for it. When a contractor's crew damages the building, the contractor answers for it. Nobody accepts "the system decided" as a defence, and nobody will accept "the model decided" either. Regulators will not, customers will not, and a court will not.

So the question is never *whether* someone is accountable — someone always is. The question is whether your company has decided who, in advance and on purpose, or whether it will find out during the incident.

05What this looks like when it is built properly

This is not theory for us. The pattern below is what we ship — for example in 1C Agent, an AI layer over a company's ERP where managers ask for things in plain business language and the agent acts inside the accounting system.

An ERP is a good stress test for the argument, because the blast radius is real: postings, registers, period close. The five requirements land as concrete mechanics:

• Scope becomes role-based permissions. The agent inherits the requester's rights. A manager asking for a report gets a report; the same request cannot quietly become a write to a register the person could not touch themselves.
• Limits become a permission gate. Read operations run. Writes and anything with financial consequence hit a boundary the prompt cannot argue its way past — because the check lives in the system, not in the instructions.
• Approval becomes plan review. The agent states what it intends to do, in business language, before it does it. The human approves the plan, not a yes/no popup — that is the difference between the reviewer having the information and not.
• The record becomes a full audit log. Every action is logged and reversible: who asked, what the agent proposed, what it read, what changed. Reconstructable months later by someone who was not in the room.
• The owner becomes a named person with a kill switch. Not a policy document. A person who can stop it now.

The counter-intuitive result is that this is the part that lets you go faster. Once the boundary is real, you stop negotiating every single request. Managers get safe self-service instead of a ticket queue, and the core of the accounting system is never touched.

06Where to start if you already have agents running

Most companies reading this are not at the design stage — they already have three or four agents in production that someone stood up quickly. That is fine. The retrofit is short:

1. Inventory. List every agent with a credential. Include the ones a single team built without telling anyone; those are the ones that matter.
2. Name an owner per agent. One person. If nobody will take it, that is your finding — an agent nobody will own is an agent nobody should be running.
3. Find the irreversible actions. For each agent, list what it can do that cannot be undone: money out, messages to customers, deletions, anything filed with an authority. That list is where approval gates go. Everything else can stay automatic.
4. Check the log. Try to reconstruct one real decision from last week using only the log. If you cannot, the log is telemetry, not an audit trail — fix it before scaling.
5. Cap it. Put hard limits on spending and data scope, enforced by the system. Do this even where you trust the agent, because the limit protects you against the prompt injection and the bad update as much as against the model.

07Controlled autonomy, not unlimited autonomy

The organisations that get this right will not necessarily deploy AI more slowly. They will scale it with greater confidence — because they know where the system can act, where it must stop and who remains responsible.

That is worth being precise about, because governance is usually sold as a tax on speed. It is the opposite. The teams that stall are the ones stuck in permanent pilot: every expansion reopens the same unanswered question, so nothing ever leaves the sandbox. The teams that move have already answered it once, in a form they can reuse for the next agent and the one after.

The future is not unlimited AI autonomy. It is controlled autonomy that businesses can trust, audit and scale.

08Frequently asked questions

Short answers to what comes up most.