Skip to main content
Use AI Tests for repeatable, independent customer questions. Use Playground when you need a multi-turn conversation.
AI Tests uses the Active agent selected when the run starts. An inactive agent open in the editor is not automatically the test target.
AI Tests simulates some actions, but not every connected tool is guaranteed to be isolated. A passing result does not prove that an order, payment, message, customer update, or automation completed.
1

Create a test case

  1. Open AI Studio > AI Tests and select Add new test case.
  2. Enter the test case name and optional description.
  3. Add a Customer question and a specific Expected answer or behavior.
  4. Add an approved customer profile only when the question requires customer context.
  5. Select Create test case. Create test case form in AI Tests
2

Add representative questions

Select Add new item for each question. Include a normal request, an exception, missing information, and a request that should be handed to a person.
3

Run the test

  1. Confirm the intended agent is active.
  2. Select Run test.
  3. Wait for the server-side run to finish. While it runs, question editing is unavailable.
4

Review the results

Check the pass summary and each question’s Last result.Test case page with the pass summary and Last result for each questionOpen a question and compare Customer asked, Expected answer or behavior, AI answered, and AI judge.Question result showing the customer question, expected answer, AI answer, and AI judge note
  • Passed: the evaluated answer met the written expectation.
  • Needs attention: the answer may be incomplete or only partially match.
  • Failed: the answer materially conflicts with the expectation.
  • Couldn’t run: no completed result was available to score.
  • Not run: the question has no result yet.
  1. Decide whether the problem is the answer, knowledge, instructions, or expected result.
  2. Correct the appropriate configuration or test question.
  3. Confirm the intended agent is still active, then run the test again.
  4. Verify any consequential action separately before relying on it.
  • Run test is unavailable: add a question or wait for the current run to finish.
  • The wrong agent was tested: check which agent was active when the run started.
  • A run did not finish: inspect the error and available results before selecting Run again.
  • Deleting a test case or question also deletes its run history. Review the confirmation before removal.