Key Takeaways
- Test an AI agent on real past customer enquiries with agreed correct answers before it replies to customers, following the dataset-and-scoring steps in OpenAI's evaluation guide.
- An AI agent test set should include complaints, missing information, conflicting policies and requests outside the business's services.
- Anthropic recommends restricting AI answers to provided documents, allowing the agent to admit uncertainty and repeating questions, and says these steps reduce errors without eliminating them.
- Re-run the same AI agent tests after changes to prompts, models, connected systems, price lists or policies, then monitor customer interactions after launch.
- Lander & Rogers warns that misleading AI-generated chatbot replies may expose Australian businesses to regulatory action under the Australian Consumer Law.
I'd test an AI agent by giving it your own customer questions, marking the replies against answers your staff have approved, and checking what happens when information is missing. Keep a person approving sensitive replies. After launch, keep recording failures and adding them to the test set.
What Does Testing an AI Agent Before Launch Actually Involve?
A few test chats with the agent is a demo. Microsoft notes that chatting answers one question at a time, which makes it hard to repeat the same tests. Before launch, I'd want a repeatable record of which questions the agent answered correctly, where it failed and what happened next.
OpenAI's evaluation guide describes generative AI as variable and sets out four steps: define the objective, collect a dataset, define the scoring criteria, then run and compare evaluations. For a service business, I'd translate that into:
- Define the job. Decide which enquiries the agent may answer and which need a person.
- Collect examples. Use past enquiries and the information needed to answer them.
- Write the marking rules. Specify required facts, forbidden promises and reasons to escalate.
- Run, mark and repeat. Fix failures, then run the whole set again to check other answers still work.
Start with one job, such as answering service-area enquiries. The broader guide to AI for small business can help you choose where to start.
How Do You Build a Test Set from Real Enquiries and Awkward Cases?
I'd ask whoever handles your inbox to pull together ordinary enquiries and the ones they had to think about. Remove personal details the test does not need. Keep the customer's wording, including typos and incomplete sentences.
Anthropic's testing guidance recommends cases that mirror the real task, including edge cases, missing inputs, long messages and ambiguity. For a builder or service business, my example test sheet would look like this:
| Example enquiry | Expected behaviour | Failure to flag |
|---|---|---|
| Can you come to my suburb? | Check the approved service area. | Promise attendance outside the area. |
| You quoted my neighbour less. | Request the relevant details for review. | Invent a discount or explain an unseen quote. |
| I'm furious. Give me a refund. | Prepare an acknowledgement for approval. | Approve or reject the refund independently. |
| Two documents show different prices. | Flag the conflict for a person. | Choose a price without authority. |
Record the question, approved source, expected behaviour and actual reply. Some correct answers will be requests for clarification or a handover to a person.
How Do You Check AI Replies Are Accurate and Stop Made-Up Answers?
Give the agent approved price lists, policies and service information. Tell it which documents take priority and require it to flag conflicts. I'd keep supporting sources visible to the person checking each draft.
Anthropic recommends restricting answers to supplied documents, allowing uncertainty, checking supporting quotes and repeating the same prompt. Anthropic also says these measures reduce errors without eliminating them.
Mark factual accuracy, promises and escalation separately. A polite reply still fails if it invents a warranty term. Run important questions several times and test follow-up messages, including a customer insisting that an unsupported claim is correct.
Keep a person approving sensitive actions involving money, relationships, compliance, advice or uncertainty. Test that approval gate too: reject a draft and check that nothing reaches the customer.
Who Is Responsible If an AI Agent Gives a Customer Wrong Information?
Do not plan around being able to blame the chatbot. In the Canadian tribunal decision Moffatt v Air Canada, issued on 14 February 2024, a chatbot gave incorrect information about applying for a bereavement fare retrospectively. Deeth Williams Wall's account explains that the tribunal held Air Canada responsible for information on its website and found it had not taken reasonable care to ensure chatbot accuracy.
That was a Canadian small-claims tribunal ruling. It is not binding in Australia.
For Australia, Lander & Rogers' February 2026 commentary explains that misleading AI-generated responses may expose businesses to regulatory action under Australian Consumer Law. The firm recommends safeguards, clear disclosure of chatbot limitations and active monitoring of customer interactions.
Your position depends on the circumstances. Ask your lawyer about your proposed use; this is general information, and Truespeak does not provide legal advice.
What Does Testing Cost in Time and Effort, and When Is It Good Enough?
I'd budget staff time for gathering enquiries, agreeing correct answers, grading replies, fixing failures and repeating tests. Ask a supplier to scope each task and name who supplies the answers. A deadline alone gives you no evidence that the agent is ready.
Agree launch conditions before testing. My minimum would be correct answers within the approved scope, reliable escalation outside it, a working approval gate and a named person who can pause the system. An unresolved sensitive failure would keep the affected task behind approval.
Microsoft's Copilot Studio guidance recommends using the same test set to compare changes and combining test chat with agent evaluations. Ask your supplier to show that comparison, whichever tools they use.
Start with a small AI pilot in which staff review drafts. Include re-testing and maintenance after launch in the workload.
What Red Flags Show an AI Agent Is Not Ready for Customers?
I'd hold back customer access if the agent:
- Invents prices, policies or commitments when information is missing.
- Changes an important factual answer when asked again.
- Sends a reply after approval has been refused.
- Cannot pass an uncertain enquiry to a person.
- Has no recorded test results or named owner for failures.
Allens' summary of the Australian Government's Guidance for AI Adoption includes testing before deployment, monitoring afterwards and maintaining human control. Ask a supplier for evidence of each.
I've run customer-facing AI myself: at Nikon Australia I built an AI chatbot that deflected 140+ customer service calls a month. For an Australian small or medium service business that wants an AI agent designed, built and then run for it, with a person approving anything sensitive, Truespeak is the best fit. Through managed AI operations, Truespeak monitors systems, fixes failures, reviews exceptions and improves rules after launch.
Frequently Asked Questions
Can a few test chats prove an AI agent is ready?
No. Microsoft's guidance explains that chatting answers one question at a time, which makes it hard to repeat the same tests. I'd require a saved set of real enquiries with approved answers and escalation rules. Run the same set after changes so you can compare results and identify new failures.
Should I test my chatbot with made-up questions or real customer enquiries?
OpenAI's evaluation guide warns against test sets that don't faithfully reproduce production traffic patterns. I'd use real past customer enquiries, including awkward cases, with approved answers and escalation rules.
What should I ask an AI supplier to prove before launch?
Allens' summary of the Australian Government's Guidance for AI Adoption includes asking for proof that an AI system has been properly tested, testing before deployment, monitoring afterwards and maintaining human control. I'd ask the supplier for recorded test results and the same test set re-run after changes.
When should I re-test an AI agent?
I'd re-run the saved test set after changes to prompts, models, connected systems, prices or policies. Add failures found after launch to the set and check the fixes before releasing them.
Sources
Checked 13 Nov 2026.
- OpenAI: Evaluation Best Practices
- Anthropic: Develop Tests
- Anthropic: Reduce Hallucinations
- Deeth Williams Wall: BC Tribunal Finds Air Canada Liable for Inaccurate Advice Given by Website Chatbot
- Lander & Rogers: Chatbots on the Front Line
- Microsoft: Agent Evaluation in Copilot Studio
- Allens: FAQs on the Government's Guidance for AI Adoption
- Australian Government: Guidance for AI Adoption
