Test Tools Safely with Mocks

Once your agent calls tools (your own API, an MCP server, another agent or a plugin), testing gets harder: a test run could read live data that changes every day, write to a real system, or fail because a service is down. Mocks fix that. In this step you write scripted answers for your tools, attach them to a conversation simulation, and run it, so every run gets the same predictable tool results and nothing real is touched.

In this tutorial you test how Atlas, Kirana Group's operations assistant, reports project status. Atlas looks up projects with the Custom API get_project_status from Connect Your API. The real project data changes as work moves on, so you pin it down with a mock set: KG-2041 is always On hold at the CFO stage, and every other code returns 404 Not Found. Then you run a two-step simulation against it.

Before you start

• Your agent has at least one tool: Connect Your API, Connect an MCP Server, Work With Other Agents or a plugin from Using Plugins. To follow along, Atlas has the get_project_status API.
• The agent is published. Simulations run against the published version, and the mock editor only offers the tools of that version. See Preview and Publish Changes.
• You know the basics of conversation simulations (steps, expected responses, criteria). If not, read Test Automation and Design Scenarios first.
• The Mocks item appears under Simulation in the agent sidebar. If you don't see it, Mocks is not switched on for your environment yet.

Why use mocks

During a simulation, every tool call goes through a gate that decides what happens to it. Without mocks, each kind of tool has a fixed default:

So without mocks, a scenario that depends on Atlas's Projects API can never pass, and a scenario that reads a live table can pass today and fail tomorrow. A mock gives the tool a scripted answer instead. That lets you:

• Test without touching real systems or real data. No real project records, no writes to a production system, no calls to a paid API.
• Get the same result on every run, so a failing run means the agent changed, not the data.
• Test the hard paths on purpose: an empty result, a "not found", a 500 error, a timeout, a project that is on hold. These are difficult to trigger with a real service.

Key terms

Step 1: Create a mock set

1. In the CMS, open Simulation → Mocks in the agent sidebar. The page is titled Tool Mocks.

2. Click New Mock Set.

3. Fill in the set:

   Field	Value for Atlas
   Name (required, up to 120 characters)	Projects API: KG-2041 on hold
   Description	get_project_status returns a fixed "On hold" answer for KG-2041 and 404 for any other code.

The Library chip next to the title means this set is shared: you can attach it to any conversation simulation of this agent. Used in 0 conversations changes once you attach it in Step 6. Duplicate makes a copy of the whole set.

Step 2: Add the tool you want to mock

1. Under Tools in this set, click Add tool.
2. Search for get_project_status and click it. The list shows the tools of the published agent, grouped by source (Custom API, MCP, Plugins, Built-in). Each entry shows what the tool does with data and what it would do in a simulation if you don't mock it. For example, searching for escalate shows the built-in escalate_conversation tool with the labels Writes data and Simulated success. The note at the bottom reminds you that these are the tools of the published Atlas, the version simulations run against.

The selected tool's card shows its type (Custom API, GET), its description and its Inputs (the arguments the AI fills in when it calls the tool; for get_project_status that is projectCode). You can add several tools to one set; select a tool in the list on the left to edit its cases.

If a tool is later removed from the published agent, the set shows Tool not found in the published agent. Use Re-bind to… to move its cases to another tool.

Step 3: Add a case with conditions

1. Click Add case. Give it a label so you recognize it later: KG-2041 is on hold.
2. Under When all of these match, click Add condition. Choose the argument projectCode, the comparison is, and type KG-2041. The field shows the argument's description from the API ("Kirana project code in the form KG-0000, for example KG-2041. Use upper case.") to help you type a valid value.
3. Add more conditions if needed. The case only answers when every condition holds. Arguments you don't list can have any value.

The comparisons you can choose depend on the argument's type:

Tips:

• Text comparisons ignore upper and lower case unless you switch on Aa (match case) on the condition. So kg-2041 also matches this case.
• is one of takes a comma-separated list; is between takes a minimum and a maximum.
• matches pattern uses a regular expression.
• A case with no conditions matches every call. Put it last, because cases are checked from top to bottom and a catch-all would hide the cases below it.
• For unusual inputs, open Advanced — Match raw arguments and paste a piece of JSON the call's arguments must contain. It is checked in addition to the conditions.

Step 4: Write the response

Under Respond with, write what the tool returns for this case. For Atlas, keep the status 200 OK, choose JSON and paste:

{
  "projectCode": "KG-2041",
  "name": "Surabaya warehouse expansion",
  "status": "On hold",
  "percentComplete": 62,
  "approval": {
    "stage": "CFO",
    "nextApprover": null
  },
  "latestUpdate": "Paused by the CFO until the revised steel quotation is approved."
}

The editor adapts to the tool:

• For a Custom API, choose the HTTP status in the field next to Respond with (default 200 OK). Error codes in the 5xx range reach the agent as an empty result, just like a real failing API.
• Switch between JSON and Text for the body. Some tools also offer a Form that fits the tool's output (for example rows and columns for a table query), with View as JSON for the raw body.
• Success and Error open ready-made templates, such as an empty list, "Not found (404)", "Unauthorized (401)", "Rate limited (429)", "Service down or timed out" for a Custom API, or "Query fails" and "No rows found" for a table query.
• For an MCP tool, choose Content or MCP error. For another agent, write the reply text.
• Suggest lets AI draft a response for the case. Format tidies the JSON.
• Use a real response calls the real tool once so you can copy its shape. It is only available for tools that only read data, such as a Custom API that uses GET; for other tools it is locked because calling them could create real data.

Response bodies can be up to 64 KB.

Step 5: Decide what happens to every other call

Below the cases, the Otherwise (every other call) box decides what the tool answers when no case matches. For Atlas, every other project code should look unknown:

1. Click Error and pick the not-found template, or set the status to 404 Not Found yourself.
2. Make sure the body is {"error":"Not found"} in JSON mode.
3. Open Advanced and keep Respond with a default.

The two options under Advanced:

• Respond with a default: the call gets the response you write in this box (here: 404 with {"error":"Not found"}).
• Keep looking: the call moves on to the next mock set, and if none answers, to the tool's default from the table in Why use mocks. In the tool list such tools are marked "Keeps looking".

Click Save. The set now appears on the Tool Mocks page with its description, 1 tool · 1 case, the tool chip get_project_status, and how many conversations use it.

Step 6: Attach the mock set to a conversation simulation

1. Open Simulation → Scenarios and create a conversation simulation with two steps. See Design Scenarios for the details.
   • Step 1, user message: What is the status of project KG-2041?
   • Step 2, user message: And project KG-9999?
2. Add the expected responses, then click Suggest All Criteria to let AI write the evaluation criteria for every step. Review them and remove any you don't want.
3. In the Tool mocks card, click Add mock set:
   • From library… attaches a shared set. Choose Projects API: KG-2041 on hold.
   • Create new makes a Private set that belongs only to this conversation.
4. Click Save. The CMS asks for a Conversation Name and a Folder. Enter Project status: KG-2041 on hold, unknown code, keep the folder Default, and click Continue.

Things to know:

• Sets attached here are used by every step. Drag the rows to change their order: the first set that answers a call wins.
• Each step also has its own Tool mocks (n) row with Add step mock set. A step's own sets are checked before the conversation's sets, so you can change one tool's answer for a single step (for example "the first lookup finds nothing, the second one finds the project").
• Date & time the agent sees: by default the agent sees the real time. Turn the switch on to fix the date, time and time zone for this simulation. Use it when tool arguments depend on dates ("tomorrow"), so the conditions keep matching on every run.
• In the row's ⋮ menu, Make a private copy (on a library set) gives this conversation its own copy that you can change without affecting other conversations, and Save to library (on a private set) shares it with the rest of the agent's simulations.

Step 7: Run the simulation

Click Run on the conversation. If the run could reach tools that are blocked in simulations and have no mock, an Attention dialog lists them first. For Atlas it says the run "will hit 3 blocked tools, which will return a policy error instead of actually running": Ledger (the agent from Work With Other Agents) and resolve-library-id and query-docs (the tools from Connect an MCP Server).

These questions are about project status, so Atlas should not need any of those tools. Click Run anyway. If a scenario does depend on a blocked tool, click Cancel and add a mock for it first.

Check the result

1. When the run is done, open it from the Results tab (see Run and Track). The conversation shows Completed, the folder Default, 2/2 Steps completed, the duration (here 14 s), and 5/6 Matched.

2. Click Mocks used in this run to see the exact mock sets the run used, frozen at the time of the run, and the simulated clock if you set one.
3. Expand Step #1 and open Tool calls. The call custom:get_project_status with the arguments {"projectCode":"KG-2041"} shows the chips Mocked and Conversation. The line underneath names the case 'KG-2041 is on hold', the mock set Projects API: KG-2041 on hold and the status 200.

Each call shows an outcome chip and where its answer came from:

Expand a call to see the full Arguments and Response. If the tool has a Response Prompt, the row notes that it was applied on top of the mocked response, just as in production. On a Mocked or Live call, Save as case turns the call into a new case (in a private set for this step, for the whole conversation, or in an existing set), which is the quickest way to build mocks from real behavior.

4. Look at the step's answer and criteria. In Step #1, Atlas reports the status On hold at approval stage CFO, 62% complete, and the latest update from the mock. Three of the four criteria match. The one that fails is Identifies KG-2041 as the Surabaya warehouse expansion project: the evaluation notes that the answer does not mention KG-2041 or name the project.

This is exactly what simulations are for. The mocked data was correct, so the failure points at the agent, not at the data. Atlas's Response Prompt tells it to start with the status and the approval stage, so it never says which project it is talking about. A reader with three projects open could mix them up. You can fix it in the Response Prompt of get_project_status (for example, "Start with the project code and name, then the status and the current approval stage..."), publish, and run the simulation again until it shows 6/6 Matched. Notice also the line "Budget: Not provided in the project status": the mock has no budget fields, and Atlas says so instead of inventing numbers.

5. Expand Step #2. The call for KG-9999 matches no case, so the Otherwise default answers with 404, and Atlas says the code is unknown and asks the user to check it, as the API's Rules tell it to. Both criteria of this step match.

If the result is not what you expect:

• The call shows Blocked instead of Mocked. The mock set is not attached to this conversation or step, or it mocks a different tool. Check the Tool mocks card and the tool name.
• The call shows Default for KG-2041. The condition does not match the argument the AI sent. Expand the call, compare the Arguments with the condition, and check whether Aa (match case) is on.
• The tool is missing from Add tool. Publish the agent after adding or renaming the tool.

Limits and good practice

• Mocks only apply to conversation simulations. Preview, the published agent and real channels always call the real tools.
• A mock set holds up to 50 tools, each tool up to 50 cases, each case up to 20 conditions.
• Web search and web scraping cannot be mocked; they run live in simulations.
• Mocks follow the published agent: publish after adding or renaming a tool, then update your mock sets.
• Keep one purpose per mock set ("happy path", "API down", "project on hold") and combine sets per scenario, rather than one huge set.
• Always test at least one error path (Error templates) so you know how your agent answers when a service fails. The 404 for KG-9999 above is one.

For the full list of fields on these screens, see the Help pages Mocks, Scenarios and Results.

Next steps

• Next in the journey: Remember Each User, the first step of Stage 4.
• Run and Track: run scenarios regularly and follow their status.
• Analyze and Improve: turn failed criteria into improvements.
• Previous step: Talk by Voice.