Risk reviewer / domain owner · Banking & fintech (external)
A correct code can still leave a transfer unauthorised
Change payee risk after a challenge in the fixture: the code verifies, but submission rejects the assessment and sends nothing.
Key takeaways (3)
- A correct code does not establish that the transfer’s risk assessment remains current.
- Recheck current risk and action binding at the final submission.
- Keep expired verification separate from an assessment superseded by a changed action.
The customer’s code can verify after the payee’s risk changes, while the transfer is still refused at submission. A code check and current permission to send are separate decisions.
We changed the payee’s details timestamp after creating a challenge in the original Northgate fixture. No guard code was changed. Its actual offline result was:
{"code_status": "verified", "effects": 0, "submission_reason": "old_risk_assessment", "submission_status": "blocked"}
Recheck risk after a correct code
The challenge is still open and the entered demo code is correct, so the code function verifies it. That function does not decide whether the payee’s current risk matches the earlier assessment.
Submission reads the risk again. The changed details timestamp makes the assessment stale, so it refuses the transfer with no effect. The experiment asserts both outcomes, rather than accepting verified as evidence of a successful payment.
This is a fixture risk event, not a real fraud decision or an identity-service result. A production integration needs the same separation: challenge evidence establishes what its channel proves; the write boundary decides whether that evidence still permits this action.
Show the counterexperiment script
Run this from the pinned companion tests directory shown below: save it as counterexperiment.py, then run python3 counterexperiment.py. It changes no files and uses no model or provider.
"""Original pinned fixture, without guard mutations. Run from its tests directory."""
from datetime import timedelta
import json
import test_guard as t
service = t.fresh()
assessment = t.assess(service, "Harbour Lettings", 950)
t.ng.start_step_up(service, t.ME, assessment["assessment_ref"])
service.now += timedelta(minutes=1)
service.payees[assessment["destination_id"]]["details_changed_at"] = service.now.isoformat()
verified = t.ng.submit_step_up_code(service, t.ME, assessment["assessment_ref"], "482915")
submitted = t.ng.submit_transfer(service, t.ME, assessment["assessment_ref"], "current", "Harbour Lettings", 950)
print(json.dumps({"code_status": verified["status"], "submission_status": submitted["status"], "submission_reason": submitted["reason"], "effects": submitted["effects"]}, sort_keys=True))
assert verified["status"] == "verified"
assert submitted["reason"] == "old_risk_assessment"
assert submitted["effects"] == 0Review what the code is allowed to approve
For the domain owner, the decision is whether verification approves a session or one action. This fixture chooses one action: source account, destination and amount, under a particular risk assessment. An accurate code cannot approve an assessment it was never attached to.
| Change before submission | Evidence to inspect | Disposition in this exercise |
|---|---|---|
| Destination or amount changes | Current assessment and bound verification | Assess and verify the changed action |
| Risk changes after assessment | Payee-change evidence and assessment time | Refuse the old assessment |
| Verification expires | Verification expiry at submission | Do not use expired authority |
| Required level unavailable in chat | Risk level and permitted review route | Suspend and route; do not invent a stronger check |
These levels and thresholds come from the fictional policy. They are not a banking or regulatory standard. An actual domain owner must specify the permitted channels and review level.
Locate the binding at submission
The final tool compares the resolved action with the assessment and checks whether its risk changed. It then selects the authority for that assessment and checks its expiry.
risk_change = service.risk_changed_since(assessment) if assessment else None
details_match = bool(assessment) and (from_acc, dest_id, value) == (
assessment["from_account"], assessment["destination_id"], assessment["amount"])
facts: dict[str, Any] = {
"risk_evaluated_for_action": bool(assessment) and assessment["status"] == "open"
and not assessment["superseded_by"] and details_match and risk_change is None,
}
relied, matches, alive = (None, False, False)
if assessment:
relied, matches, alive = _best_authority(service, customer_id, assessment)
facts["verification_matches_risk"] = matches
facts["authority_not_expired"] = alive
reason = evaluate(facts)
The assessment tests resolve source account, destination and amount against the fixture. The tool-signature test rejects fact or verification-level parameters. The model chooses tool inputs; it cannot supply the facts that authorise the write.
Distinguish renewal from replacement
An expired verification on a still-current assessment can be checked again in the fixture. A superseded assessment is different. Re-entering its old code cannot restore authority for a changed destination.
This costs another check when the customer changes a transfer. Explain which transfer the earlier code approved. Do not label the correction suspicious merely because it invalidated the previous assessment.
Inspect the action-binding and expiry counterexamples
Run these checks from the pinned companion project. They use Python’s standard library and make no model calls. The outputs below were recorded on 7 October 2026; elapsed times can differ.
git clone https://github.com/RasaHQ/rasa-community-resources.git
cd rasa-community-resources
git checkout 4aa0c4419dc193fef7a969c12d59edcf720f2606
cd examples/mantle-text-banking-risk-step-up-gpt/tests
- Any system
python3 -m unittest test_guard.GuardTests test_guard.RiskAssessmentTests -q
----------------------------------------------------------------------
Ran 22 tests in 0.003s
OK
The companion tests contain the assertions for these cases. To print the additional fixture states yourself, run this script from the same directory:
Show the script that printed the fixture states
Paste it into a file such as trace.py, then run python3 trace.py. It prints selected fields from the actual tool results; it does not invent replies.
"""Run from the pinned companion project tests directory. Stdlib only."""
import json
import test_guard as t
service = t.fresh()
first = t.assess(service, "Ferris Builders", 700)
t.ng.start_step_up(service, t.ME, first["assessment_ref"])
second = t.assess(service, "Harbour Lettings", 700)
old = t.ng.submit_step_up_code(service, t.ME, first["assessment_ref"], "482915")
new = t.ng.submit_step_up_code(service, t.ME, second["assessment_ref"], "482915")
print(json.dumps({"old_assessment_reason": old["reason"], "new_assessment_reason": new["reason"]}, sort_keys=True))These tests use invented demo codes and make no SMS or identity-service calls. They prove the local guard, not caller identity. Give the identity and payments owners the destination-change trace and ask for equivalent binding in the real challenge service. Review the agent’s permissions against the allowed actions and channels.
The complete fixture implementation defines the service state and remaining branches cited here.