Choose an agent workflow worth building
by Rasa team
Start with a decision your team can make this week: choose one workflow whose outcome you can observe and whose mistakes you can contain. A convincing conversation is a weak substitute for a completed customer task.
This guide is for the AI product manager deciding what to build next. You will leave with a scope brief to discuss with an engineer and the person who handles the task today. No Rasa installation is required.
Work through one candidate
Consider Horizon Travel, a fictional business used here only as a worked example. Its support team answers itinerary questions and changes bookings. We have no measured business results for this example.
A first proposal is “automate travel support.” Replace it with “help a signed-in traveller find the change conditions for an existing booking, then request a human-assisted change.” The second statement exposes both an outcome and a boundary. The agent may explain a retrieved policy; it may not invent eligibility or commit a paid change.
| Decision | Example brief | Evidence to collect before building |
|---|---|---|
| User and task | Signed-in traveller checking one existing booking | Observe real support sessions with permission |
| Current outcome | A support colleague retrieves the applicable conditions | Record completion, waiting time, and common exceptions |
| Proposed outcome | Traveller sees the applicable conditions and can request help | Check source, booking identity, and task completion |
| Excluded action | Charge a fee or change the booking | Identify the system that actually authorizes changes |
| Accountable owner | Travel support lead | Confirm who accepts unresolved requests |
These are proposed boundaries, not claims about what a Rasa configuration automatically enforces. The engineer must connect the identity and booking tools to the actual systems of record.
Make the decision measurable
Use a task outcome rather than message volume: eligible requests completed with correct source information / eligible requests attempted. Agree what makes a request eligible before collecting results. Keep abandoned, incorrectly answered, and handed-off requests visible as separate counts; silently excluding failures improves a percentage without improving the experience.
Record the current human workflow using the same definition. If that baseline does not exist, collecting it is the next piece of work. Do not promise a savings percentage from a demo.
For the fictional booking example, a correct answer with the wrong booking identifier is a failed task. A correct handoff can be a successful outcome when a change requires staff authority. That distinction changes both the product requirement and the evaluation set.
Choose the smallest useful slice
Compare the candidate against three practical questions:
- Can the team obtain an authoritative answer and identify the customer safely?
- Can someone observe whether the task finished correctly?
- Can the team stop or hand off the interaction when a necessary dependency fails?
If any answer is unknown, put that uncertainty in the brief and assign an owner. A workflow with accessible data and a modest benefit may be a better first build than a high-volume workflow whose exceptions nobody can explain.
The awkward case is a customer who asks for both information and a binding change in one sentence. The brief must say which part can proceed and how the other part reaches a human. “The assistant should be helpful” does not decide this.
Take this to your team
Copy these fields into a one-page brief: user; task; current workflow; observable outcome; denominator; authoritative sources; allowed actions; excluded actions; handoff owner; stop condition; next evidence to collect. Ask an engineer to challenge feasibility and a support colleague to bring one exception.
If the team cannot agree on the outcome, postpone implementation and observe the workflow together. If it can, bring the brief and one unresolved dependency to Build Day. Use the guarded-action tutorial when the next slice introduces an action that cannot easily be undone.
The distinction between context, measurement, and response is informed by the NIST AI RMF Core. The scope brief and fictional example are editorial tools, not a validated business benchmark.