The Results tab of the Conversation Simulation page lists every simulation run and shows which agent answers matched the expected responses. Found in the agent sidebar under Simulation → Scenarios, then the Results tab. Opening a run shows its Simulation Result page.
The agent's answers and the match outcomes are produced by AI, so they can differ between two runs of the same scenario. Read a single result as one sample, not a guarantee.
Results Tab: Controls
| |
|---|
| Show Filters / Hide Filters | Opens or closes the filter panel. |
| Sort By | Orders the list by Run Date (default, newest first), Code, Status or Duration, Ascending or Descending. |
| Grid View / List View | Shows the runs as cards or as a table (columns Run, Status, Version, Started, Duration, Scenarios, Steps). On a narrow screen the table collapses into one line per run. |
Results Tab: Filters
| |
|---|
| Code | Shows only runs whose code matches what you type, for example "TE-0001". |
| Status | Shows only runs with the selected status: Completed, Failed, Running or Stopped. |
| Version | Shows only runs against one version of the agent: All (default), Draft or Published. |
| Run Date From / Run Date To | Shows only runs started within this date range. Both dates are inclusive (whole days). |
| Apply | Applies the filter values to the list. |
| Reset | Clears the filters and shows all runs again. |
Results Tab: Run List
| |
|---|
| Run card or row | One per run. Click it to open the Simulation Result page. |
| Code | The run's identifier, shown as #<code>. |
| Version | A chip, Draft or Published: the copy of the agent the run was made against. A run saved before versions existed shows Published. |
| Status | Completed, Failed, Running, Stopped or Not Yet Run. |
| Run date and duration | When the run started and how long it took (- while not available). |
| Scenarios and Steps | How many scenarios and how many steps the run contains. |
| Load More | Shown when more runs are available. Adds the next batch to the list. |
| Empty state | No simulation results yet ("Run your scenarios to see results here") when nothing has been run, or No Results Found ("Try adjusting your filters or date range to see more results") when the filters match nothing. |
| |
|---|
| Back | Returns to the Results tab. |
| Title | Simulation Result with the run code and start time. A small spinner shows while the page is refreshing. |
| Error | Shown in red when the run failed, with the reason and, where relevant, "Running it again may help." |
| Note | Shown when the version changed while the run was in progress: "The draft changed during this run. Later scenarios may have used newer settings." (Draft run) or "The agent was published during this run. Later scenarios may have used the newly published settings." (Published run). |
| Live updates paused | Shown while the run is Running after 10 minutes: "Live updates paused after 10 minutes — use refresh to check the latest status." The page refreshes by itself until then. |
| Refresh (icon) | Shown while the run is Running. Reloads the latest progress. |
| Stop | Shown while the run is Running. After you confirm ("Are you sure you want to stop this execution?"), stops the run. |
| Run Again | Shown when the run is not running. Starts a new run of the same scenarios and opens it. |
| Delete (trash icon) | Shown when the run is not running. Deletes this run after you confirm ("Are you sure you want to delete this execution? This action cannot be undone."). |
Simulation Result: Summary Cards
| |
|---|
| Status row | Above the cards: the run's current status chip, the word Simulated and the Draft or Published chip. |
| CRITERIA MATCH | How many evaluation criteria were matched out of the total across all steps, for example "8 of 10 matched", with a percentage and a ring (green at 70% or more, yellow from 30% to 69%, red below 30%). With exactly one criterion the title reads CRITERION MATCH. |
| STEPS | Total number of steps in the run. With exactly one step the title reads STEP. |
| TOOL CALLS | Total number of tool calls the agent made in the run, with "n blocked" in red when some were blocked, or "None blocked". With exactly one call the title reads TOOL CALL. |
| DURATION | Total run time. |
Simulation Result: Scenarios
A heading Scenarios (n) (Scenario (1) when the run has one) with one expandable row per scenario in the run.
| |
|---|
| Scenario row | Shows the status icon, scenario name, folder, how many steps were completed (for example "3/4 Steps completed") and its duration. |
| Failed at step #n | Shown when the scenario failed, pointing at the step where it stopped. The error reason is shown below the row. |
| n blocked calls | Shown when tool calls in this scenario were blocked. |
| Mocks used in this run | Opens the Mocks used in this run panel. Only shown when the mocks feature is on. |
| Match badge | Shown for completed scenarios: criteria matched out of the total for this scenario, for example "2/3 Matched". |
| Empty state | No Scenarios Found ("There's no scenario data available for this simulation") when the run has no scenario data. |
Mocks Used in This Run
A panel with the mock plan the run used, read-only. It describes what the run was set up with when it started, even if you have changed the mock sets since.
| |
|---|
| Scenario line | "Scenario:" and the name of the scenario the panel belongs to. |
| CLOCK card | The date and time the agent saw, with its time zone underneath, or Real time with "No simulated clock". |
| LIVE TOOLS card | The keys of the tools the scenario allowed to run live, or None with "Every tool call was mocked". |
| Set list | Shown only when the run used more than one mock set. Lists the sets grouped by level, the scenario first and then each step, so you can pick the one to browse. |
| Mock set header | The name and description of the set shown, and chips for where it applied, such as "Scenario · #1" or "Step 2 · #1": the level and the set's place in that level's list. |
| Mock set content | The set's tools, cases and responses, shown read-only, in the same layout as the Tool Mocks editor. |
| Messages | No Mocks Were Used ("This run did not use any mock sets, so there is no mock plan to show."), No Mock Sets Were Attached ("Only the clock and live tools above applied to this run."), and "Failed to load this run's mock plan." with a Try Again button. |
Simulation Result: Step Details
Expand a scenario to see each step as its own expandable card.
| |
|---|
| Step #n | The step number with its status icon and, once completed, a match badge for that step. |
| Error | Shown in red when the step failed, with the error message. |
| User message | The message sent to the agent, shown at the top of the card. |
| Expected Response | The answer you wrote in the scenario. |
| AI Agent Response | The agent's actual answer. Parts that support a criterion are highlighted in green, parts that contradict one in red. Knowledge references in the answer can be expanded to show the source text. |
| Evaluation Criteria (x/y Matched) | Shown for completed steps; expand or collapse it. See Criteria Display. |
| No Evaluation Criteria | Shown when the step had no criteria defined: "This step does not have evaluation criteria defined." |
Criteria Display
The Evaluation Criteria (x/y Matched) box lists every criterion of the step, one box each.
| |
|---|
| Green box | The criterion was matched. Under it is the quoted part of the agent's answer that supports it, marked with a speech bubble: 💬 "...excerpt...". |
| Grey box | The criterion was not matched, whether the answer contradicted it or did not mention it. Under it is the reason, marked with a light bulb: 💡 reason. |
There are no separate "contradicted" or "not mentioned" labels. The difference shows in the AI Agent Response: a contradicting part is highlighted in red there.
Shown inside each step card. Says No tool calls in this turn. when the agent called no tools.
| |
|---|
| Tool calls (n) | Expands the list of tool calls the agent made in this step, with a chip per outcome, for example Mocked 2. The list starts with the line "Tool calls are listed in the order the agent made them." |
| Tool call card | A numbered card on a vertical line. Shows the tool, an outcome chip, the arguments as small tags, and a line that says why the call ended that way. Click the card to expand it. |
| Live — may contain real data | A warning chip on Live calls: the response is real, so it may contain real data. |
| Expanded row | Arguments and Response of the call, as code. Also "Loading full trace...", "Response was too large to show in full.", "(none)" when empty, "No trace detail available for this call." and "Failed to load full trace detail. Try collapsing and expanding this row again." |
| Checked before the answer | Inside an opened Mocked call, when cases were checked first and did not match: one line per case, with the mock set, the case number and label and the reason, for example “Booking defaults” · Case 2 ‘Registered patient’: date is "2026-08-06", expected "2026-08-05". Every attached set's cases are checked first, in order (a step's sets, then the scenario's, top to bottom), and the first matching case answers. |
| Shortened list notes | "1 more checked case isn’t shown, to keep the run record small." (or "n more checked cases aren’t shown, ..."), and "Argument values aren’t shown, to keep the run record small." when the reasons name the operator instead of the values. |
| Previous rule note | "Answered under the previous mock rule: a set's Otherwise ended the search." On a Mocked call recorded before every set's cases were checked ahead of any Otherwise. Run again to see the current order. |
| Response Prompt note | "The tool's Response Prompt was applied on top of this response, as in production". |
| The agent was told the tool is unavailable. | A red note on a Blocked call. |
| Mock This Call | On a Blocked call. Opens a menu to pick where the new mock goes (see Mock This Call and Save as Case). Only shown when the mocks feature is on. |
| Save as Case | On a Mocked or Live call. Opens the same menu, to save the call as a mock case. Only shown when the mocks feature is on. |
| Orphan note | On a blocked call whose tool had an old Run Live setting: "An older <tool> was set to run live. That setting no longer applies to this tool." For an MCP or plugin tool it reads "An older <tool> (from before the <name> server was added again) was set to run live. That setting no longer applies to this tool." (or "plugin"). It comes with a Move it in Tool Policy → link. |
Outcome chips
The outcome chip tells you what happened to the call.
| |
|---|
| Mocked | A mock set answered the call, either with a case that matched or, when no case in any attached set matched, with the Otherwise response of the first set that answers by default. The line under the arguments says which. |
| Simulated | The call was not made; the tool returned a simulated success. |
| Live | The call really ran, against the real system. |
| Blocked | The call was not made. The agent was told the tool is unavailable. |
| Internal Tool | An internal tool that always runs. |
| Unknown | The outcome could not be determined. |
Answer lines
The line under the arguments says why the call ended that way. A status code, when there is one, comes first as a small tag.
| |
|---|
Case <id>… '<label>' · <mock set name> | A Mocked call that a case answered: the start of the case's id, its label when it has one, and the mock set. |
No case matched, used Otherwise · <case> · <mock set name> | A Mocked call that the Otherwise of a set answered, because no case matched. |
| No mock matched | A Mocked call that records no case or set. |
| Step-level mock set, Scenario-level mock set | Shown at the end of a mocked call's line: whether the set that answered was attached to the step or to the whole scenario. |
| Tool policy | The agent's Tool Policy decided a Live, Simulated or Blocked call. |
| Allowed live in this scenario | The scenario's Allow Live let the tool run. |
| Built-in default | The tool's built-in default decided: no Tool Policy rule covers the tool. Older runs that recorded a call as not reviewed show this too. |
| No line | An Internal Tool call, or a call recorded before the rule was kept. |
Mock This Call and Save as Case
Both buttons open a small menu asking where the mock should go. Then the scenario opens in its form with the mock set editor already filled in for this call.
| |
|---|
| New Private Set for This Step | "Only used when this exact step runs." |
| New Private Set for This Scenario | "Used by every step in this scenario." |
| Existing Set… | "Pick a set once the scenario opens." The scenario opens a panel, Add this case to a mock set, listing the sets attached to the step and to the scenario. Pick one and the case is added to it. A library set warns "Library set: saving it changes every scenario that uses it." If no sets are attached yet, the panel offers New Private Set: This Step and New Private Set: This Scenario instead. |
| Live call notice | Saving a Live call asks first: "This call ran live and may contain real data. Saving it as a case stores that response as the mock. Continue?" |
| Too large | When the response was too large to copy: "The response was too large to copy. Fill in the response yourself before saving this case." |
| Not mockable | When the tool cannot be mocked: "This tool can't be mocked, so this call can't be saved as a case." |
Related guide: Read the Results, Fix and Retest