Skip to content
RasaGet a free licence
AI team casebook

Risk reviewer / domain owner · Banking & fintech (external)

A correct code can still leave a transfer unauthorised

Change payee risk after a challenge in the fixture: the code verifies, but submission rejects the assessment and sends nothing.

by Rod Rivera

About 6 minutesCase study

Key takeaways (3)
  • A correct code does not establish that the transfer’s risk assessment remains current.
  • Recheck current risk and action binding at the final submission.
  • Keep expired verification separate from an assessment superseded by a changed action.

The customer’s code can verify after the payee’s risk changes, while the transfer is still refused at submission. A code check and current permission to send are separate decisions.

We changed the payee’s details timestamp after creating a challenge in the original Northgate fixture. No guard code was changed. Its actual offline result was:

{"code_status": "verified", "effects": 0, "submission_reason": "old_risk_assessment", "submission_status": "blocked"}

Recheck risk after a correct code

The challenge is still open and the entered demo code is correct, so the code function verifies it. That function does not decide whether the payee’s current risk matches the earlier assessment.

Submission reads the risk again. The changed details timestamp makes the assessment stale, so it refuses the transfer with no effect. The experiment asserts both outcomes, rather than accepting verified as evidence of a successful payment.

This is a fixture risk event, not a real fraud decision or an identity-service result. A production integration needs the same separation: challenge evidence establishes what its channel proves; the write boundary decides whether that evidence still permits this action.

Show the counterexperiment script

Run this from the pinned companion tests directory shown below: save it as counterexperiment.py, then run python3 counterexperiment.py. It changes no files and uses no model or provider.

"""Original pinned fixture, without guard mutations. Run from its tests directory."""
from datetime import timedelta
import json
import test_guard as t

service = t.fresh()
assessment = t.assess(service, "Harbour Lettings", 950)
t.ng.start_step_up(service, t.ME, assessment["assessment_ref"])
service.now += timedelta(minutes=1)
service.payees[assessment["destination_id"]]["details_changed_at"] = service.now.isoformat()
verified = t.ng.submit_step_up_code(service, t.ME, assessment["assessment_ref"], "482915")
submitted = t.ng.submit_transfer(service, t.ME, assessment["assessment_ref"], "current", "Harbour Lettings", 950)
print(json.dumps({"code_status": verified["status"], "submission_status": submitted["status"], "submission_reason": submitted["reason"], "effects": submitted["effects"]}, sort_keys=True))
assert verified["status"] == "verified"
assert submitted["reason"] == "old_risk_assessment"
assert submitted["effects"] == 0

Review what the code is allowed to approve

For the domain owner, the decision is whether verification approves a session or one action. This fixture chooses one action: source account, destination and amount, under a particular risk assessment. An accurate code cannot approve an assessment it was never attached to.

Change before submissionEvidence to inspectDisposition in this exercise
Destination or amount changesCurrent assessment and bound verificationAssess and verify the changed action
Risk changes after assessmentPayee-change evidence and assessment timeRefuse the old assessment
Verification expiresVerification expiry at submissionDo not use expired authority
Required level unavailable in chatRisk level and permitted review routeSuspend and route; do not invent a stronger check

These levels and thresholds come from the fictional policy. They are not a banking or regulatory standard. An actual domain owner must specify the permitted channels and review level.

Assess account, payee and amount Challenge bound to assessment Changed action: old challenge invalid Verify this assessment Recheck risk and expiry at submission
FigureChanged actions invalidate the earlier assessment and challenge

Locate the binding at submission

The final tool compares the resolved action with the assessment and checks whether its risk changed. It then selects the authority for that assessment and checks its expiry.

Source excerpt:


risk_change = service.risk_changed_since(assessment) if assessment else None
details_match = bool(assessment) and (from_acc, dest_id, value) == (
    assessment["from_account"], assessment["destination_id"], assessment["amount"])
facts: dict[str, Any] = {
    "risk_evaluated_for_action": bool(assessment) and assessment["status"] == "open"
    and not assessment["superseded_by"] and details_match and risk_change is None,
}
relied, matches, alive = (None, False, False)
if assessment:
    relied, matches, alive = _best_authority(service, customer_id, assessment)
facts["verification_matches_risk"] = matches
facts["authority_not_expired"] = alive
reason = evaluate(facts)

The assessment tests resolve source account, destination and amount against the fixture. The tool-signature test rejects fact or verification-level parameters. The model chooses tool inputs; it cannot supply the facts that authorise the write.

Distinguish renewal from replacement

An expired verification on a still-current assessment can be checked again in the fixture. A superseded assessment is different. Re-entering its old code cannot restore authority for a changed destination.

This costs another check when the customer changes a transfer. Explain which transfer the earlier code approved. Do not label the correction suspicious merely because it invalidated the previous assessment.

Inspect the action-binding and expiry counterexamples

Run these checks from the pinned companion project. They use Python’s standard library and make no model calls. The outputs below were recorded on 7 October 2026; elapsed times can differ.

git clone https://github.com/RasaHQ/rasa-community-resources.git
cd rasa-community-resources
git checkout 4aa0c4419dc193fef7a969c12d59edcf720f2606
cd examples/mantle-text-banking-risk-step-up-gpt/tests
Any system
python3 -m unittest test_guard.GuardTests test_guard.RiskAssessmentTests -q
----------------------------------------------------------------------
Ran 22 tests in 0.003s

OK

The companion tests contain the assertions for these cases. To print the additional fixture states yourself, run this script from the same directory:

Show the script that printed the fixture states

Paste it into a file such as trace.py, then run python3 trace.py. It prints selected fields from the actual tool results; it does not invent replies.

"""Run from the pinned companion project tests directory. Stdlib only."""
import json
import test_guard as t

service = t.fresh()
first = t.assess(service, "Ferris Builders", 700)
t.ng.start_step_up(service, t.ME, first["assessment_ref"])
second = t.assess(service, "Harbour Lettings", 700)
old = t.ng.submit_step_up_code(service, t.ME, first["assessment_ref"], "482915")
new = t.ng.submit_step_up_code(service, t.ME, second["assessment_ref"], "482915")
print(json.dumps({"old_assessment_reason": old["reason"], "new_assessment_reason": new["reason"]}, sort_keys=True))

These tests use invented demo codes and make no SMS or identity-service calls. They prove the local guard, not caller identity. Give the identity and payments owners the destination-change trace and ask for equivalent binding in the real challenge service. Review the agent’s permissions against the allowed actions and channels.

The complete fixture implementation defines the service state and remaining branches cited here.