Blog›Automatización

What a person should approve and what an agent can do alone

Reviewing everything does not work, and neither does leaving everything unattended. What research says about where to put human approval when you delegate to an AI agent.

Equipo Neural Fabrik5 min de lectura
What a person should approve and what an agent can do alone

The usual answer to the risk of an AI agent is «have a person review it». Research on automation has been showing for decades that reviewing everything does not work, and it gives pointers on where to put the review.

Autonomy is not a switch

The classic framework describes automation as a scale, not as a yes or a no. That scale is set separately for four functions: gathering information, analysing it, deciding and executing. A system can be highly autonomous when gathering data and not autonomous at all when executing (Parasuraman, Sheridan and Wickens, 2000).

Among the criteria they propose for setting the level in each function is the cost of the consequences of each decision or action. It is the best academic support for a simple idea: the line is drawn according to what happens if it goes wrong.

A recent paper translates this into the vocabulary of agents with five possible roles for the person: operating, collaborating, advising, approving or observing (Feng, McDonald and Zhang, 2025).

Why reviewing everything does not work

  • Whoever supervises ends up trusting. Complacency appears in novices and in experts and is not corrected by practice. Automation bias, which is accepting what the machine proposes as correct, is not avoided by training or by instructions either (Parasuraman and Manzey, 2010).
  • It gets worse with workload and time pressure. That is the conclusion of a review of 74 studies, which also points to what helps: emphasising the responsibility of the person who decides and showing information instead of a ready-made recommendation (Goddard, Roudsari and Wyatt, 2012).
  • People review less over time. In real usage data published by a vendor, 20% of new users’ sessions approve everything automatically, and more than 40% among experienced users (Anthropic, 2026).
  • Person plus AI does not always add up. A meta-analysis of 106 experiments found that, on average, the combination performs worse than the better of the two on its own. There were losses in decision tasks and gains in content creation (Vaccaro, Almaatouq and Malone, 2024).

An analysis of 41 public policies that require human oversight concludes that, as they are usually framed, people do not manage to exercise it and that they give a false sense of security (Green, 2022).

What happens when nobody is there

Leaving everything unattended does not work either.

  • In a simulated company, the best agent completed 30% of the tasks on its own, and the longest ones were beyond its reach (Xu et al., 2024). It is a laboratory setting, with models from 2024 and 2025.
  • Reliability falls with duration. The best models of early 2025 completed tasks of around 50 minutes with a 50% success rate; that horizon doubles roughly every seven months (Kwa et al., METR, 2025). These are programming tasks, and the authors warn that it is not clear how far this generalises.
  • In a real shop, an agent ran a small point of sale on its own. Its authors concluded that they would not hire it: it directed customers to a payment account it had made up and sold at a loss (Anthropic, 2025).

Where to draw the line

That same study of real usage measured that only 0.8% of agents’ actions were irreversible. Its example is sending an email to a customer. If what cannot be undone is such a small share, a person’s attention can be concentrated there without exhausting it.

It has to be said clearly: no study shows that irreversibility is the best criterion. It is a design principle, consistent with the 2000 framework and with the EU AI Act. For high-risk systems, its Article 14 asks that oversight be proportionate to the risk and to the autonomy, that whoever supervises be aware of the tendency to trust too much, and that they be able to disregard or reverse an output and stop the system.

A marketing or invoicing agent is not usually high-risk. The article serves as a reference, not as an obligation.

In practice

  • Few approvals, reserved for what cannot be taken back: publishing, sending to a customer, paying, touching a production system.
  • Each approval shows what is going to be done and with what data, not just an accept button.
  • Everything else is reviewed afterwards, by sampling, and whatever fails goes back through approval.

At Neural Fabrik the agents work this way: they do routine work on their own and what has consequences waits for a person. It does not eliminate error, and the evidence above explains why no design does.

Sources

  • Parasuraman, Sheridan and Wickens (2000). «A model for types and levels of human interaction with automation». IEEE Transactions on Systems, Man, and Cybernetics A 30(3). Link
  • Feng, McDonald and Zhang (2025). «Levels of Autonomy for AI Agents». arXiv:2506.12469
  • Parasuraman and Manzey (2010). «Complacency and Bias in Human Use of Automation». Human Factors 52(3). Link
  • Goddard, Roudsari and Wyatt (2012). «Automation bias: a systematic review». JAMIA 19(1). Link
  • Anthropic (2026). «Measuring AI agent autonomy in practice»
  • Vaccaro, Almaatouq and Malone (2024). «When combinations of humans and AI are useful». Nature Human Behaviour 8. Link
  • Green (2022). «The flaws of policies requiring human oversight of government algorithms». arXiv:2109.05067
  • Xu et al. (2024). «TheAgentCompany». arXiv:2412.14161
  • Kwa et al., METR (2025). «Measuring AI Ability to Complete Long Software Tasks». arXiv:2503.14499
  • Anthropic (2025). «Project Vend: Can Claude run a small shop?»
  • Regulation (EU) 2024/1689, Article 14. Official text

Quieres ver como funcionaria en tu empresa?

Te preparamos una demo personalizada con datos de ejemplo de tu sector.

Solicitar propuesta