Skip to content
RasaGet a free license
Menu
Guides for AI teams

Guide · AI product engineer

Can Symitar voice agents send a transfer before logging it?

A transfer-logging rule that all 18 existing tests accept even when the code breaks it, and the one added test that fails when the log entry is written after the bank call.

by Rasa Community

About 14 minutes to read

Key takeaways (3)
  • Write the pending log entry before the bank call and commit it, so a stopped program or a lost reply leaves a record that a transfer may have been sent.
  • A test suite that passes with the log entry written after the send does not test the rule; move the block and see.
  • Check the log from a second database connection inside the call to the bank, because only a committed row is visible there.

A voice agent confirms a bank transfer, and then the bank’s reply never comes back. Did the money move? The part of the agent that has to answer is the host: the example’s own code that sends a confirmed transfer to the bank. The host keeps a journal, a log in a small SQLite database, and it writes a pending entry there before it sends. If the program stops after the send, or the reply is lost, that entry tells the host that a transfer may have been sent. Without it the journal says nothing happened, and the caller could ask again. That window is hypothetical here: this tutorial made no live lost-reply run.

Your job as the developer is to know that your tests would notice if that order broke. The example’s host already writes the entry first, and it must, so there is nothing to fix in its code. The gap is in the tests. We moved the entry after the bank call on purpose, in a copy of the example. All 18 existing host tests still passed, and one test we added fails on the moved order. This is a mutation of the example, not a recorded bank incident.

Native-host tests (base commit 0e90ad8f, owned Linux Codespace, 17 November 2026)Original order: entry, then bank callEntry moved after the bank call
The 18 existing tests18 pass18 pass
With the added test (19 tests)19 pass18 pass, 1 fails: the added test

The changed copy can send a transfer before logging it. We add the test that would notice. This tutorial shows the order in the code, makes the change, and adds that test. The term write-ahead logging comes from database literature (see Sources). This tutorial does not claim that the example follows any particular published scheme.

What you will learn

  • Why a transfer needs a pending log entry before it is sent, not after.
  • What the example does when the bank’s answer is lost, and what a stopped program leaves behind.
  • How to prove a test suite checks the order, by reversing the order on purpose.
  • How to write a test that sees the entry from the bank’s side of the call.
  • What this safety rule costs.

The bank here is a local simulation of the Jack Henry Symitar core. The example’s README says all people, accounts, and amounts are fictional and that no banking request goes to a real credit union (README.md, lines 5 and 9). The tests in this tutorial do not even use that simulation: they use a stand-in bank client written in the test file (tests/test_native_host.py, lines 20 to 31).

1. See the rule in the code

The host sends a confirmed transfer in one method, _dispatch, in native_host.py (lines 188 to 215). Read it in this order:

  1. It builds the request body first (line 193). A bad body fails here, before anything is recorded.
  2. It writes the log entry with self.journal.reserve(...) (line 198). The entry is pending.
  3. Only then does it call the bank with self.client.call('postTransfer', body) (line 203).
  4. It tries to record the outcome with self.journal.result(...) (line 211).

A pending row is not a record that money moved. It is a safety marker that says “a send may be in progress, and nobody has seen the answer yet.”

Where the pending entry sits relative to the bank call
  1. The host builds the request body (line 193). A bad body fails here, before anything is recorded.
  2. It writes the pending entry and commits it (line 198).
  3. It calls the bank (line 203).
  4. It tries to record the outcome (line 211).
FigureWhere the pending entry sits relative to the bank call

The lower row is the reversed order from Step 2. The entry is still written, but after the call, which is too late for a program that stops between the two to leave a record.

What step 4 records depends on what happened in step 3. The outcome starts as unknown (line 201):

  • A reply whose status is success is recorded as accepted (line 204).
  • An error the bank’s client reports as an invalid request or a vendor rejection is recorded as rejected.
  • Any other error, any other exception, and a reply that is not success all leave the outcome unknown.

A test covers the last case: after a failure of that kind, the confirming turn, the one that attempts the send, returns unknown. The same test then checks that a changed review from a new session is refused, that reads still work, and that the host made exactly one send (tests/test_native_host.py, lines 153 to 162).

Recording can itself fail. journal.result raises an error if its database update fails or does not change exactly one pending row (journal.py, lines 124 to 139). The host catches that error, returns unknown to the caller anyway, and moves on (native_host.py, lines 210 to 213). In that case the row may still be pending. A row is also left pending when the program stops before step 4 runs. In both cases, the next time the journal starts it turns every pending row into unknown (journal.py, line 78), because it cannot prove whether the request was sent.

While a row is pending or unknown, the journal refuses to open another entry for the same caller (journal.py, lines 107 to 120). That refusal stops a second transfer after a lost reply.

2. Reverse the order on purpose

Take the code from Step 1 and move the reserve block, with its except JournalRefused branch, below the bank call and its error handling, so that it runs just before the outcome is recorded. This is the only edit. Both versions of the file are in the receipts: original and mutated.

Then run the example’s native-host tests, which cover this host. The runs below were recorded in an owned Linux Codespace with provider keys removed, at commit 0e90ad8f, before the test in Step 3 existed. Each block is the last five lines of one run’s output, unedited, and the link beside each label opens the full output.

Original order (full output):


----------------------------------------------------------------------
Ran 18 tests in 0.183s

OK

Entry moved after the bank call (full output):


----------------------------------------------------------------------
Ran 18 tests in 0.184s

OK

Every test passed with the order reversed. None of the 18 observes the entry while the bank call is in progress. The tests read the final results, so the reversed code passed them all.

3. Look from the bank’s side

The missing check is simple to state: when the bank call starts, the entry must already be saved. A test can do that by replacing the client’s call with a wrapper that, when the call is postTransfer, opens a second SQLite connection to the journal file and reads the rows. A second connection cannot see a row that was not committed, so it sees pending only if the host committed it first.

Pull request 256 added this test, test_pending_marker_is_committed_before_the_bank_client_is_called, to tests/test_native_host.py (merge commit ed23e9b6). Before you open it, try to write the observation yourself: what must the test read when the bank call starts, and from which database connection?

The observation we added

Its key lines are:

        async def observe_call(name, body):
            if name == 'postTransfer':
                # A separate connection cannot see an uncommitted marker.
                with sqlite3.connect(self.root / 'journal' / 'journal.sqlite3') as reader:
                    observed.append(reader.execute('SELECT state FROM operations').fetchall())
            return await original_call(name, body)

It then runs a reviewed transfer, confirms it, and checks three things: the result is accepted, the second connection saw exactly [('pending',)] during the call, and the client was called once.

4. Check that the new test notices

A new test that passes tells you little. Run it against the reversed order. These runs are from the same owned Codespace, at base commit 0e90ad8f. The blocks are unedited.

Original order, 19 tests, last five lines (full output):


----------------------------------------------------------------------
Ran 19 tests in 0.196s

OK

Entry moved after the bank call, 19 tests: the failure report from FAIL: to the end (full output). The empty list [] is what the second connection saw when the bank call began:

FAIL: test_pending_marker_is_committed_before_the_bank_client_is_called (test_native_host.NativeHostTests.test_pending_marker_is_committed_before_the_bank_client_is_called)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspaces/symitar-mutation/examples/jack-henry-symitar/tests/test_native_host.py", line 70, in test_pending_marker_is_committed_before_the_bank_client_is_called
    self.assertEqual(observed, [[('pending',)]],
AssertionError: Lists differ: [[]] != [[('pending',)]]

First differing element 0:
[]
[('pending',)]

- [[]]
+ [[('pending',)]] : the durable pending marker must be visible before the bank call

----------------------------------------------------------------------
Ran 19 tests in 0.197s

FAILED (failures=1)

The new test alone, original order (full output):


----------------------------------------------------------------------
Ran 1 test in 0.015s

OK

The new test alone, entry moved after the bank call (full output):


----------------------------------------------------------------------
Ran 1 test in 0.016s

FAILED (failures=1)
Code under testTests runFailed
Original order, with the new test190
Entry moved after the bank call, with the new test191, the new test
The new test alone, original order10
The new test alone, entry moved after the bank call11

There were no skipped tests in any of these runs. The new test fails only on the reversed order, and it fails alone: the other 18 still pass. The production file was restored and hash-checked after each change.

5. Ten other changes that the tests caught

The ordering was one change. We also removed or weakened, one at a time, ten other guards in the example’s journal, host, and reply code. For each one we ran only the test module for that code. Before any change, the four modules ran 64 tests, and all 64 passed. Each change made at least one test fail. Seven of them are in the journal and the host:

What we removed or weakenedFile and lineTests that failed
Turning pending rows into unknown on startjournal.py 781
The refusal while a caller is heldjournal.py 1182
The check that the journal’s epoch matchesjournal.py 1161
The rule that a result only updates a pending rowjournal.py 1351
unknown rows counting as a holdjournal.py 1033
pending rows counting as a holdjournal.py 1033
The requirement that the next user turn confirmshost.py 2031

The epoch is an id the journal creates when it is set up. reserve refuses a call whose epoch differs from the journal’s, so a host holding an old id cannot write into a journal that was reset.

The other three changes are in the example’s conversation and reply code (one in lib/conversation_policy.py, two in lib/reply_policy.py). Each also failed one test. They concern how repeated text and held replies are handled, which this tutorial does not cover. The matrix summary lists all ten with their failing tests, and the script shows how each was run.

That makes eleven changes in all: ten that a test caught, and the reversed order, which none of the existing tests caught. This is not a mutation score and does not cover every guard in the example or the full test suite.

6. What this rule costs

Recording the attempt first has a price. After a lost reply the row is unknown, and the journal has no method to clear it or check it against the bank (journal.py, lines 124 to 130). While it is held, the caller cannot start another transfer. The host chooses a stuck caller over a possible second transfer.

The example’s README describes an audited reset that starts a new fictional exercise. It says the reset never resolves or retries the earlier transfer (README.md, line 294). This tutorial does not cover the reset, and it does not say how a real bank would reconcile an unknown transfer.

7. What this does and does not show

The example’s journal is a real SQLite file with synchronous=FULL (journal.py, line 56), and the test reads that same file. That makes the check about the committed row, not about a variable inside the host.

This page proves one thing about one example’s code, by running its tests. The example also has a separate offline component proof, declared for the exact pinned commit ed23e9b6: 81 tests, from five selected test modules and not the full suite, ran with no skips on the locked Rasa 3.21.0.dev5 environment, with no credentials, model, speech service, or real bank involved. The component proof receipt names its decisive test, test_pending_marker_is_committed_before_the_bank_client_is_called, and its limits. It is offline and is a different run from the 18-test and 19-test runs above, which were recorded earlier at base commit 0e90ad8f and are not an ed23e9b6 rerun. The limits:

Not claimed: Trained agent, model, voice, vendor, or current global release acceptance.

In plain words, no trained agent, language model, speech service, real credit union, or claim about the current Rasa release took part. The other limits are about what the tests do:

  • The example’s code already writes the entry first. The gap was in the tests. Nothing here says the example ever sent a transfer before logging it.
  • The tests use a stand-in bank client, not the example’s simulated bank. Its failure switch exercises the unknown outcome, and no live lost-reply run was made.
  • The new test does not stop the program between the two lines. It shows that the row is already committed when the bank call begins, which is what a stopped program would rely on.
  • The ten other changes are a selection, not every guard.
  • The page covers no spoken run, no response time, and no historical incident.

Try it yourself

The recorded runs used a GitHub Codespace, which is a Linux machine in your browser. Other setups are outside the recording. Open a Codespace on the RasaHQ/rasa-community-resources repository. In its terminal, run:

git checkout ed23e9b6df468c99ebef29972cfa31c3a6b39d39
cd examples/jack-henry-symitar
uv sync --locked

Then run the native-host tests:

Any system
.venv/bin/python -B -m unittest discover -s tests -p test_native_host.py -v

The commit is ed23e9b6, and the test comes from pull request 256. Expect 19 tests to pass. Then move the reserve block in native_host.py as in Step 2, run the same tests again, and expect exactly one failure: the test from Step 3. Restore the file when you are done, for example with git checkout -- native_host.py if you made no other edits to it.

Sources