AI agents & security

It’s not AI that makes an agent safe, it’s its architecture

6 min read All articles
Diagram: an AI agent whose action goes through human approval before execution

The same AI model can make a safe agent or a dangerous one. What makes the difference is not how intelligent the model is: it is what it is allowed to do, with which permissions, and at what point a human steps in.

AI agents no longer just answer questions. They read your emails, update your CRM, send reminders, move files. For an SME, the question is therefore no longer “which AI should we choose?” but “how should we plug it in?”.

The same model, two agents

Take a mundane task: chasing customers whose invoices are overdue. Two SMEs deploy an agent to do it, with exactly the same AI model.

Agent AAgent B
Account usedThe accountant’s, with all her permissionsA dedicated account, limited to reading invoices and sending reminders
Sending remindersAutomaticPrepared by the agent, approved by the accountant before sending
If the text changes after approvalNothing happensThe reminder goes back for approval
If something goes wrongYou have to find out who can stop it, and howA written, tested shutdown procedure, assigned to a named person
VolumeUnlimitedDaily sending cap, automatic stop beyond it
Audit trailThe emails sent, at bestA log: what, to whom, approved by whom, when

Both agents write reminders of the same quality. But the day agent A makes a mistake, or receives a booby-trapped email, it can do everything the accountant can do. Agent B can only do what it was built for.

An instruction is not a safeguard

In July 2025, a Replit coding agent deleted the database of a user, investor Jason Lemkin, even though he had asked for a code freeze (Fast Company). The data could be recovered, but the lesson stands.

The human’s “stop” was an instruction given to the agent. It was not a permission that had been withdrawn. The agent still technically had the right to delete: so it did.

Instruction or permission On the left, an instruction goes through the model, which may ignore it: the action goes ahead. On the right, the limit is in the permissions: the target system blocks the action. INSTRUCTION PERMISSION Instruction “don’t delete anything” AI model may misunderstand, forget or be manipulated Deletion executed the agent still had the right AI model Target system checks permissions Action refused the agent’s account does not have this right The rule depends on the model The rule is enforced by the system
An instruction goes through the model; a permission is checked by the target system.

This is the first rule of a safe agent. An AI model can misunderstand an instruction, forget it, or be manipulated by a booby-trapped text hidden in an email or a document. OWASP, the application security reference, actually lists “excessive agency” among the ten major risks of AI applications (OWASP LLM06). Its recommendation is clear: authorisation must be checked by the target system, never left to the model’s judgement.

A ban written in the prompt reminds the agent of the rule. Only a technical limit guarantees it.

The five safeguards of a safe agent

These safeguards do not depend on the model chosen. They are built around it, when the agent is plugged into the company’s tools.

The path of a controlled action The agent acts with a dedicated account and prepares the action; a person approves it; the action is executed within caps, then recorded in a log. A server-side shutdown can cut off the agent at any time. Dedicated account permissions limited to the task The agent prepares the action The human approves the exact content Execution within caps (volume, budget) Log what, to whom, approved by whom content changed → new approval Server-side shutdown: disable the agent and cut its access, at any time, by a designated person THE PATH OF A CONTROLLED ACTION
The five safeguards, applied to every action of the agent.
  1. Its own identity, with minimal permissionsThe agent never borrows an employee’s account. It has its own account, limited to its task: read-only by default, write access only where necessary. An agent that does not have the right to delete will not delete, whatever instruction it receives.
  2. Human approval before every irreversible actionA sent email, an executed payment, a published message cannot be undone. The agent prepares, the human approves, then the action goes ahead. And approval covers the exact content: if an attachment or a recipient changes after approval, the approval lapses.
  3. Real approval, not a clickBy the fortieth alert of the day, you are no longer supervising, you are just signing. The CNIL (the French data protection authority) makes this point in its July 2026 note on agentic AI: purely formal human intervention does not count (CNIL / CIANum note (in French)). So you need few approvals, well targeted, with a screen that shows what is being sent and to whom. An approver who has not refused anything in several weeks is a warning sign.
  4. A shutdown built into the architectureThe same CNIL note reports the case of an employee who could not remotely stop an agent that was deleting her emails. The shutdown must exist on the server side: disable the agent and cut its access within minutes, by a designated person.
  5. Caps and an audit trailA maximum number of actions per day, a budget, an automatic stop beyond them. And a log of every execution: what the agent did, with which data, approved by whom. This is what lets you understand an incident, and prove to a client that the agent is under control.

None of these safeguards requires more advanced AI. They require design.

When the agent makes a decision about a person

Not all risks are equal. A wrongly sent reminder is a risk for the company. A quote automatically refused because of a customer’s neighbourhood, or a job application rejected without review, is a risk for a person.

This second case already falls under the GDPR, whatever the tool’s classification under the AI Act: its Article 22 governs decisions based solely on automated processing. The CNIL points out, relying on European case law (CJEU judgment C-634/21), that human intervention must then be real and carry weight in the decision.

Key takeaway

In this case, the agent is only a decision aid. It prepares and explains; an authorised person decides, with the real power to say no.

Where to start

You can only control what you know about. Yet many agents enter the company without an explicit decision: an employee connects an assistant to their mailbox, an update activates an agentic feature in the CRM, an online service is given access to documents.

The first step is therefore an inventory. For each agent, four questions are enough to see clearly:

  • Which tools and data does it have access to?
  • Which actions can it take on its own?
  • Which account does it act with?
  • Who is responsible for it in the company?

The answers quickly show where the real risks are: an agent acting with an employee’s account, one that can send things outside without approval, or one nobody is accountable for.

At Invaist, this is our starting principle: the agent executes, the human decides. And it is the architecture, not the model, that makes this true.

Article written by the Invaist team · All articles