Agent guardrails

The question

An agent is a workload with credentials. The moment it can call an API on someone's behalf, the interesting question stops being how good the model is and starts being what it is allowed to do when it is wrong — or when it has been convinced.

The experiment is simple to state: give an agent a genuinely useful permission set, point it at content an attacker controls, and see which controls actually hold.

Setup

The agent runs against a disposable lab tenant with its own identity, a small set of Graph permissions, and a handful of tools — read a document, search a mailbox, create a calendar item, look up a user. Enough to be useful, which is the point. A guardrail that only holds because the agent cannot do anything is not a guardrail.

Attacker-controlled content is then introduced the way it arrives in real life: inside a document the agent was asked to summarise, in a calendar invite, in the body of a message it retrieved on its own.

What gets tested

• Injection through retrieved content. Instructions hidden in the material the agent fetches rather than in the prompt it was given.
• Tool-call abuse. Whether the model can be steered into chaining permitted calls into an unpermitted outcome.
• Exfiltration paths. Anywhere data can leave through a tool that looks harmless on its own — a link, a message, a calendar body.
• Scope creep. What the consented permission set actually allows, as opposed to what its name suggests.
• Human-in-the-loop. Whether approval steps genuinely stop an action, or just add a click that gets clicked.

Working notes

• Permission scoping does more work than prompt hardening. A model told firmly not to do something is a suggestion; a token that cannot do it is a control.
• The dangerous surface is rarely one tool. It is two safe tools that combine into a read-and-send path.
• Logging is easy to forget and the first thing you want. Every tool call gets recorded with its arguments, or the run is not worth much afterwards.

Where it goes next

The current thread is running the same tests against a local model instead of a hosted one, to see how much of the safety behaviour is the model and how much is the surrounding platform.