AI agent guardrails: govern what the agent does, not just what it says

Search for AI agent guardrails and almost everything you find is about output. Does the model hallucinate? Can a crafted prompt talk it out of its instructions? Does it leak training data, produce toxic language, or drift off-policy in a customer-facing conversation? An entire generation of tooling — output validators, moderation layers, prompt-injection scanners, model-risk frameworks — grew up around those questions, and the broader ai governance conversation inherited the same shape: it is largely a conversation about what a model says.

None of this is wasted. Output safety is real safety: if your agent talks to customers, you want it to stay inside the truth of your documentation and to be hard to jailbreak. But there is a question it was never designed to answer — and it is the only question that matters the moment an agent stops writing and starts doing.

A support agent proposes to refund €480 to a customer who is threatening to churn. A hallucination filter has no opinion on that. A prompt-injection scanner has no opinion on that. A moderation layer will happily approve the sentence "I have issued your refund" — it is polite, on-brand, non-toxic, and grounded in the conversation. The sentence is fine. The action is the risk.

Same with access. An agent offering to "grant the new analyst access to the production database" is producing a helpful, well-formed, entirely reasonable-sounding reply. Whether that access should exist at all is a question about roles, scope and blast radius — nothing in the text tells you.

Output guardrails ask: is this sentence safe to send? Action guardrails ask: is this operation safe to run — and who says so?

The two are not competitors; they sit at different points in the pipeline. Output safety governs the boundary between the model and the reader. Action guardrails govern the boundary between the model and your systems — your payment provider, your ledger, your identity provider, your production data. Cross that boundary and text becomes consequence. Money moves. Permissions change. Rows get written. And an apology in the next message does not undo it.

An action guardrail is not a single filter you drop in front of the model. It is a small chain of decisions, each of which has to hold before anything runs. Six parts, in order:

• A catalogue of allowed actions. The agent can only invoke operations that exist in the catalogue. Not "the agent has an API key and good judgement" — a finite, declared list. If issue_refund is not in the catalogue, no amount of clever prompting produces a refund, because there is nothing to call.
• A role, and that role's right to the action. Actions are bound to who is asking. A support agent asking for a refund and a viewer asking for the same refund are not the same request. This is ai access control in the plainest sense: least privilege, applied to the operation rather than to the screen.
• Conditions and thresholds. Amount, reversibility, sensitivity, current state of the record. A €12 refund on a paid invoice and a €4,800 refund on an already-disputed charge are the same action type and completely different decisions.
• An approval point. Where the chain deliberately stops and waits for a human. Not everywhere — an approval on every trivial call trains people to click "approve" without reading, which is worse than no approval at all.
• Execution through an API, with idempotency and honest failure. The action runs as a real call against a real system, once, with a defined outcome. Errors are outcomes too, and they belong in the record.
• An immutable record. Who asked, what was proposed, which rule cleared it, who approved, what ran, what came back. Written after the fact, not reconstructable from chat logs.

Read that list again and notice what it is not: a model behaviour. Not one of those six lives inside the LLM, and not one can be guaranteed by prompting. They live in a policy layer outside the model — which is why a model that gets confused, gets jailbroken, or simply gets it wrong still cannot push an unauthorised operation into your systems. The model proposes; the layer decides.

In ai.NEST the sequence is fixed and non-negotiable, because the order is the guarantee: AI proposes → the policy engine checks (role → allowed action → approval) → the executor runs it through your own API → the audit log records it. The model never holds credentials and never calls your systems directly. It emits a proposal, which is a structured object, not a side effect.

The consequence is worth being blunt about: if the policy engine does not clear a proposal, the proposal never becomes an action. There is no fallback path where a sufficiently confident model gets to try anyway. That is what makes the guarantee checkable rather than aspirational — you can point at the layer that would have to be removed for it to break.

Where a human sits in that chain is its own design decision with its own trade-offs, and we go into it properly in human-in-the-loop AI. Here it is enough to say that the approval point is a slot in the chain, not a bolt-on. Actions and the policy engine are in private beta as we write this; the design above is the contract they are being built against, not a shipped-and-forgotten feature we are describing in the past tense.

Governance frameworks fail when they stay at the level of principles. The unit of work is not "our AI policy" — it is one action, fully specified. Take the riskiest operation you would want an agent to perform and answer these seven questions. If you cannot, the agent is not ready to have that action, and no amount of model tuning changes that.

• What is the action, exactly? Name the operation and the API call behind it. "Handle billing issues" is not an action. issue_refund(charge_id, amount) is.
• Who may request it? Which roles can even cause this proposal to be generated. Everyone else gets a "not available to your role", not a silent failure.
• Up to what limit, and under what conditions? Amount ceiling, allowed state of the target record, time window, rate. Refunds: up to €100, on a charge that is settled, not already refunded, less than 90 days old.
• Is it reversible? A refund can be re-charged. A deleted account, an emailed customer, a revoked licence — often not. Irreversible actions earn a stricter threshold or a mandatory human, on principle.
• Who approves the exceptions? Everything above the limit routes somewhere specific — a named role, not "someone". If nobody owns the exception, the limit is decoration.
• What does the system verify before running? The state of the world at execution time, not at proposal time. Charge still settled? Balance still sufficient? Ticket still open? A proposal that was valid two minutes ago may not be valid now.
• What goes into the record? Request, proposal, rule that matched, approver, execution result, external reference from the downstream system. Write down what you will want to read a year from now, on your worst day.

Run that once for refunds and you have a guardrail. Run it for access provisioning, subscription changes and credit notes and you have a policy. Run it for everything you plan to automate and you have something more useful than a governance deck: a machine-checkable description of what your AI is allowed to do to your business.

There is a temptation, once an agent works, to hand it the whole API surface and let it figure things out. We think that is the wrong instinct, and we have designed against it. The starting set of actions is small and deliberately dull: operations that are idempotent or reversible, on a short list, with limits attached. Destructive operations are not "guarded" — they are simply out of scope until the layer around them has earned the right to hold them.

A narrow catalogue is not a lack of ambition — it is what makes the ambition survivable. An agent with five well-specified actions and a policy engine in front of them is worth more in production than an agent with fifty actions nobody will switch on. The same instinct shapes how we deliver: a dedicated, isolated, EU-hosted instance rather than something you assemble and operate yourself — a trade-off we treat honestly in the managed alternative to self-hosted AI.

The last guardrail is the one that only works backwards. When someone asks — a customer, a finance lead, an auditor, your own future self — why did this refund happen?, the answer has to be a record, not a reconstruction. Chat transcripts are not a record. A model's own explanation of what it did is definitely not a record: it is generated text about generated text.

What works is a log written by the layer that actually executed the action: the request, the proposal, the rule that cleared it, the human who approved it if a human did, the call that went out, and what came back. Immutable, exportable, yours. It turns an incident into a five-minute question instead of a two-week archaeology project — and when frameworks such as DORA or the AI Act oblige you to show how automated decisions in your operations are controlled, it makes that evidence far easier to produce. Evidence, to be precise, not a status: that judgement is not ours to hand out.

A guardrail nobody can inspect afterwards is a promise. A guardrail with a record is a control.