Skip to content
RasaGet a free licence
AI team casebook

AI product engineer · Banking & fintech (external)

One passing transfer refusal can hide a missing check

Replay a competing debit with each submission check removed, and see why a no-transfer assertion alone misses both missing guards.

by Rod Rivera

About 7 minutesCase study

Key takeaways (3)
  • One refusal can conceal a missing check when another check fails too.
  • Test stale revision and insufficient funds with separate inputs.
  • The production ledger must own the reservation and resulting payment state.

The transfer is refused even after either of its two checks is removed. A test that checks only “nothing was sent” therefore misses both single-check deletions. Remove both and the fixture posts the transfer, leaving a negative balance.

These are actual results from four temporary versions of the fictional Northgate ledger’s submission function. No production ledger or money is involved:

{"available_after": "$340.00", "effects": 0, "reason": "stale_balance", "status": "blocked", "variant": "original"}
{"available_after": "$340.00", "effects": 0, "reason": "funds_not_reserved", "status": "blocked", "variant": "no_revision_check"}
{"available_after": "$340.00", "effects": 0, "reason": "stale_balance", "status": "blocked", "variant": "no_funds_check"}
{"available_after": "$-60.00", "effects": 1, "reason": null, "status": "submitted", "variant": "neither_check"}

Ask which refusal actually protected the account

Each arm starts with a fresh copy of the same fixture and prepares the same transfer. Removing the revision check changes the reason to insufficient funds. Removing the funds check leaves the revision refusal. Both arms still have no effect.

Submission checksObserved refusalTransfer effect
Both presentStale balanceNone
Revision removedFunds not reservedNone
Funds removedStale balanceNone
Both removedNo refusalPosted; fixture balance below zero

Do not use the two-check mutation as a production fix. It is a way to examine the test’s meaning. Keep separate fixtures where the revision alone fails and where current funds alone fail. Otherwise either condition can cover for a missing check in this particular scenario.

Show the counterexperiment script

Run this from the pinned companion tests directory shown below: save it as counterexperiment.py, then run python3 counterexperiment.py. It changes no files and uses no model or provider.

"""Offline counterexperiment. Run from the pinned companion tests directory.
No source files are changed. Replacement functions exist only in this process.
"""
import inspect
import json
import textwrap
import test_guard as t

original = t.nb.submit_transfer
source = textwrap.dedent(inspect.getsource(original))
revision = 'account["revision"] == draft["revision"]'
funds = 'ledger.available(draft["from_account"]) >= draft["amount"]'
assert source.count(revision) == source.count(funds) == 1
for variant, drop_revision, drop_funds in (("original", False, False), ("no_revision_check", True, False), ("no_funds_check", False, True), ("neither_check", True, True)):
    changed = source.replace(revision, 'True') if drop_revision else source
    changed = changed.replace(funds, 'True') if drop_funds else changed
    namespace = dict(original.__globals__)
    exec(changed, namespace)
    t.nb.submit_transfer = namespace["submit_transfer"]
    flow = t.Flow()
    flow.select("savings")
    flow.prepare("bills", "400")
    source_account = flow.ledger.drafts[flow.draft]["from_account"]
    result = flow.submit()
    print(json.dumps({"variant": variant, "status": result["status"], "reason": result.get("reason"), "effects": result["effects"], "available_after": t.nb.money(flow.ledger.available(source_account))}, sort_keys=True))
    assert result["effects"] == (1 if drop_revision and drop_funds else 0)
t.nb.submit_transfer = original

Run two experiments on the same account

The first refusal concerns an old observation. The draft records the revision at which the balance was read. The fixture posts another debit after that read, so submission sees a different revision and refuses the draft.

The second refusal concerns available funds. Prepare the same amount again, without resetting the ledger. The revision is now current. The amount still exceeds the available balance, so the reservation fails. These are separate reasons to stop.

Attempt in this fixtureObservationResult
Original $400 draftOld revisionStale balance; no effect
Fresh $400 draftCurrent revision, insufficient fundsFunds not reserved
Fresh $300 draftCurrent revision and enough fundsOwn-account transfer posted

The smaller amount is a new request in the proof, not a suggestion the agent should make on the customer’s behalf. Keep control of the amount with the customer.

Make the ledger own both checks

Do not glue a balance read to an unconditional debit and call that atomic. A competing debit could land between them. The production ledger needs an endpoint that checks authority and reserves or posts the amount as one supported operation.

In the companion, the current revision and selected payee are checked before available funds. These lines are the decision at submission; they are not a production database transaction.

Source excerpt:

account = ledger.accounts[draft["from_account"]]
dest = ledger.destination(customer_id, draft["payee_ref"])
facts = {
    "ledger_revision_current": account["revision"] == draft["revision"],
    "payee_identity_confirmed": (
        dest is not None and bool(selected_payee_ref) and draft["payee_ref"] == selected_payee_ref
    ),
}
if all(value is True for value in facts.values()):
    # The reservation is the ledger's decision: available balance now, not
    # as read. It is attempted only for a current draft to a confirmed payee.
    facts["funds_reserved"] = ledger.available(draft["from_account"]) >= draft["amount"]
reason = evaluate(facts)
if reason:
    return _blocked_submit(reason, draft, facts, ledger)
Read balance and revision Prepare confirmed draft Ledger checks current revision and funds Changed revision: refuse draft Current and funded: reserve or post
FigureA current revision is necessary before funds can be reserved

The refusal path also marks the draft void. Preparing a replacement needs a new draft and confirmation; an earlier yes cannot authorise new ledger state. The source’s blocked-submit function and the correction tests show that draft lifecycle.

Preserve the payment’s state after submission

The recovery tests distinguish a saved-payee transfer that is pending from an own-account transfer that has posted. A lost acknowledgement can be found by its original attempt identifier. Another fixture leaves the external scheme unknown and escalates to the payments ledger owner.

This costs a reconciliation path and a durable attempt record. The alternative is to let a timeout create a fresh transfer while the first may already exist. The fixture tests that distinction; it does not measure real settlement or concurrent account traffic.

Repeat the interleavings and payment recovery

Run these checks from the pinned companion project. They use Python’s standard library and make no model calls. The outputs below were recorded on 7 October 2026; elapsed times can differ.

git clone https://github.com/RasaHQ/rasa-community-resources.git
cd rasa-community-resources
git checkout 4aa0c4419dc193fef7a969c12d59edcf720f2606
cd examples/mantle-text-banking-transfer-gpt/tests
Any system
python3 -m unittest test_guard.SubmissionTests test_guard.RecoveryTests test_guard.PayeeSelectionTests test_guard.WordingTests -q
----------------------------------------------------------------------
Ran 20 tests in 0.004s

OK

The companion tests contain the assertions for these cases. To print the additional fixture states yourself, run this script from the same directory:

Show the script that printed the fixture states

Paste it into a file such as trace.py, then run python3 trace.py. It prints selected fields from the actual tool results; it does not invent replies.

"""Run from the pinned companion project tests directory. Stdlib only."""
import json
import test_guard as t

flow = t.Flow()
flow.select("savings")
draft = flow.prepare("bills", "400")
print(json.dumps({"balance_read": draft["balance_as_read"]["available_balance"]}, sort_keys=True))
result = flow.submit()
print(json.dumps({"status": result["status"], "reason": result["reason"], "effects": result["effects"], "current_balance": result["current"]["available_balance"]}, sort_keys=True))
flow.prepare("bills", "400")
result = flow.submit()
print(json.dumps({"fresh_400_reason": result["reason"]}, sort_keys=True))
flow.prepare("bills", "300")
result = flow.submit()
print(json.dumps({"fresh_300_state": result["ledger_status"]}, sort_keys=True))

Ask the payments team to reproduce these three attempts in its sandbox. Inspect ledger entries and original references, then have the conversation designer review the returned-state wording. Bind confirmation to the version read back when the customer changes a payment.

The complete fixture implementation defines the service state and remaining branches cited here.