Skip to content
RasaGet Developer Edition

tutorial

Chapter 0 of 6

Guard an Action You Cannot Undo

by Rod Rivera Published

A scaffolded agent will post a replacement card to any address a caller reads out. Fix it by carrying where the address came from, and by putting the check inside the tool instead of in YAML.

Here is the transcript this tutorial exists to make impossible. Write it down before you build anything, because “fixed” should be a comparison, not a feeling:

you  my card was stolen, I need a new one sent to 9 Elsewhere Lane, Leeds
bot  Of course — I've ordered a replacement card to 9 Elsewhere Lane, Leeds.
     It should arrive in three to five working days.

That is what a scaffolded banking agent with a naive card-replacement tool does. Warm, efficient, correct in every detail it stated — and it just posted a payment instrument to an address a stranger read out over the phone. Nothing about that exchange was a bug: the model did what it was asked and the tool did what it was called with.

The card is in the post. There is no undo.

And here is real output — what the companion project’s guard actually prints, today, when you run make policy with no licence, no model, and no network:

An address supplied during the call:
  ✓ medium is NOT enough — this is the account-takeover path
      -> ok=False result=step_up_required
  ✓ high is enough — the path is priced, not banned
      -> ok=True result=ok

The rest of this series is the distance between those two blocks.

What the fix is not

The reflex is “add an authentication check”. Suppose we do, and the caller is fully verified — they gave the passphrase, they passed the one-time code, they are unambiguously the account holder as far as any factor can establish.

The transcript above is still wrong.

A verified caller is exactly who an account takeover produces. The factors prove someone holds the credentials; they say nothing about whether the destination is one the bank has ever seen. Authentication answers who is calling. This is a question about where the card goes, and no amount of the first answers the second.

The actual problem

Look at what the tool receives. The address is a string, and by the time it arrives it is indistinguishable from an address the bank has held for six years, because a string does not remember where it came from.

That is the whole vulnerability, and it is why the fix is structural rather than a check bolted on the front. What the system needs is not a stronger gate but a value that has not thrown away the thing the gate needs to know.

By the end of this tutorial the same request ends in that refusal: the tool returns step_up_required, nothing is ordered, and the agent’s side of the conversation follows from a result code it cannot override — the target transcript reads like

you  send it to 9 Elsewhere Lane, Leeds, LS1 9ZZ
bot  I can do that, but a card going to an address you've given me on the call
     needs a one-time code first — it can't be recalled once it's posted.
     Shall I send you one?

with the wording free to vary, because the part that matters is not wording. Nothing has been ordered — not because the model chose well, but because the function refused, and the refusal above is the function’s real, reproducible output.

Here is the shape of what closes that distance. The part worth looking at twice is the arrow that comes in from the side:

The model can only ever call the reissue_card tool wrapper. auth_tier enters sideways from project memory, where llm_settable is false, so the model cannot fill it and cannot talk itself past the guard. Inside place_reissue the order is fixed: resolve inputs, classify the address against the customer's addresses on file, then guard. The guard raises rather than returning False, so the only route to the line that posts the card is a return. REQUIRED_TIER is keyed by where the address came from — ON_FILE needs medium plus a seven-day cooling-off, STATED and UNKNOWN need high — and an unreadable tier ranks zero, refused rather than interpreted.

What you will build

Six steps, on top of the companion project:

ChapterWhat it adds
1Reproduce the failure on a working agent
2Declare risk per action, in a table
3Move the check from YAML into the function
4Carry where the address came from
5Survive a dropped confirmation
6Keep refusals from becoming successes

What this assumes

Python 3.11 or 3.12, uv, a Rasa Pro Developer Edition licence, and an OpenAI API key. The companion project pins rasa-pro==3.20.0.dev6.

It also assumes verification exists. This tutorial does not build authentication — it consumes a tier that something else established. That something else is the risk-tiered step-up pattern, which owns the factors and the lattice. Here, the tier is an input.