AI Tests uses the Active agent selected when the run starts. An inactive agent open in the editor is not automatically the test target.
1
Create a test case
- Open AI Studio > AI Tests and select Add new test case.
- Enter the test case name and optional description.
- Add a Customer question and a specific Expected answer or behavior.
- Add an approved customer profile only when the question requires customer context.
-
Select Create test case.

2
Add representative questions
Select Add new item for each question. Include a normal request, an exception, missing information, and a request that should be handed to a person.
3
Run the test
- Confirm the intended agent is active.
- Select Run test.
- Wait for the server-side run to finish. While it runs, question editing is unavailable.
4
Review the results
Check the pass summary and each question’s Last result.
Open a question and compare Customer asked, Expected answer or behavior, AI answered, and AI judge.


Understand the result labels
Understand the result labels
- Passed: the evaluated answer met the written expectation.
- Needs attention: the answer may be incomplete or only partially match.
- Failed: the answer materially conflicts with the expectation.
- Couldn’t run: no completed result was available to score.
- Not run: the question has no result yet.
Improve and repeat
Improve and repeat
- Decide whether the problem is the answer, knowledge, instructions, or expected result.
- Correct the appropriate configuration or test question.
- Confirm the intended agent is still active, then run the test again.
- Verify any consequential action separately before relying on it.
Troubleshooting and test history
Troubleshooting and test history
- Run test is unavailable: add a question or wait for the current run to finish.
- The wrong agent was tested: check which agent was active when the run started.
- A run did not finish: inspect the error and available results before selecting Run again.
- Deleting a test case or question also deletes its run history. Review the confirmation before removal.

