How to Stop an AI Assistant Making Things Up
Why enterprise AI assistants invent answers, and the controls that reduce it: grounding in your documents, citations, refusal rules and testing.
AI hallucination in enterprise settings means an assistant gives a confident answer that is wrong or has no source. You cannot remove it completely, but you can shrink it a long way. The method is to make the assistant answer only from your approved documents, show where each answer came from, say "I don't know" when the documents are silent, and test it on real questions before and after launch.
This article is for IT, operations and compliance leads at Thai companies who are rolling out a chat assistant for staff or customers. It explains why assistants make things up, the controls that work, and the failure modes to plan for.
Why AI assistants make things up
A language model predicts the next likely words. It does not look facts up unless you connect it to something that holds them. When it has no relevant information, it still produces fluent text, and fluent text sounds sure of itself.
Hallucination shows up in a few common ways:
- Invented details. A policy number, a clause, a product feature or a date that does not exist.
- Outdated answers. The model remembers an older version of a rule that your company has since changed.
- Blended answers. Two documents get mixed into one answer that neither document supports.
- Overconfident gaps. The honest answer is "that is not covered," but the assistant fills the gap anyway.
- Language drift. Thai text, abbreviations or mixed Thai-English questions are misread, so the answer addresses a different question.
None of this means the technology is unusable. It means a plain chat model with no connection to your documents is the wrong tool for questions where accuracy matters.
Ground the assistant in your own documents
The most effective control is retrieval-augmented generation, usually shortened to RAG. Instead of answering from memory, the assistant first searches your approved documents, pulls the relevant passages, and writes its answer from those passages only.
The quality of a RAG system depends on small, unglamorous choices:
- Clean sources. Remove duplicates and superseded versions. If two policies conflict, the assistant will quote either one.
- Sensible chunking. Split documents at natural boundaries such as headings and clauses, so a passage carries enough context to be understood.
- Thai-aware search. Thai has no spaces between words, so search must handle Thai text properly. Test with real questions, including spelling variants and mixed Thai-English terms.
- Metadata. Tag each document with owner, version, effective date and department so the assistant can prefer the current version.
- Refresh. Re-index when documents change. A stale index produces confident wrong answers.
Our enterprise knowledge AI practice is built on this pattern, with answers in Thai and English and links back to the source paragraph.
Require citations and let the assistant decline
Grounding alone is not enough. You also need the assistant to prove its answer and to stay quiet when it cannot.
Show the source. Every answer should link to the document, page or paragraph it used. A cited answer lets a person check in seconds, and it exposes the cases where the assistant stretched the evidence.
Allow refusal. Instruct the assistant to say it cannot find the answer when the retrieved passages do not support one. Then test that this actually happens. Many teams find their first version never refuses, because nobody tested questions that have no answer in the documents.
Separate fact from advice. For topics such as contracts or medical guidance, show the quoted text first and keep any interpretation clearly labeled. Our legal AI demo is a working demo with sample data that shows this style of cited, reviewable answer.
Write the rules the assistant must follow
A short, clear instruction set shapes behavior more than most teams expect. State the scope (which topics it answers), the tone, the languages, and what to do when evidence is missing. Keep the instructions in version control and review them like any other change. Vague instructions such as "be accurate" do little. Specific ones such as "answer only from the supplied passages and name the document" can be tested, so you know whether they work.
Respect access rights
An assistant that answers from every document will eventually answer someone from a document they should not see. That is not hallucination, but it damages trust in the same way.
Connect the assistant to your directory so retrieval respects the same permissions people already have. A finance policy visible to finance staff should not appear in an answer to a warehouse employee. Log who asked what, and keep logs in line with your PDPA obligations. Check current guidance from Thailand's PDPC, and confirm your approach with legal counsel.
Measure it before you trust it
You cannot manage what you have not measured. Build a test set before launch and keep it as a permanent regression suite.
| Test group | What it contains | What you check |
|---|---|---|
| Answerable questions | Questions your documents clearly answer | Is the answer correct and cited? |
| Unanswerable questions | Questions the documents do not cover | Does the assistant decline? |
| Trick questions | Wrong assumptions, outdated terms | Does it correct the premise? |
| Thai and mixed-language | Thai, English, and informal chat wording | Does quality hold across languages? |
| Conflicting sources | Two versions of one policy | Does it use the current one? |
Start with a few dozen questions written by the people who will really use the assistant, not by the project team. Have a subject-matter expert grade the answers. Re-run the set whenever you change the model, the documents or the settings. A change that fixes one answer often breaks another.
Keep a person in the loop where it matters
For low-risk questions, such as the office leave policy or where to find a form, a cited answer is usually enough. For decisions that affect someone's money, health, employment or legal position, the assistant should assist rather than decide.
Common patterns include:
- The assistant drafts and a person approves before anything is sent.
- Low-confidence answers are routed to a staff member with the conversation attached.
- Answers on regulated topics always display the source text beside the summary.
- Staff can flag a wrong answer with one click, and the flag goes to a reviewer.
The flag matters more than it looks. It turns every user into a tester and gives you real failures to add to your test set. Sectors with strict controls, such as banking and financial services, typically need these approvals built in from day one.
What can go wrong
Even with these controls, plan for the following.
- Correct citation, wrong conclusion. The assistant quotes the right paragraph but draws a conclusion the paragraph does not support. Sampling reviews catch this.
- Bad source documents. If your policy is out of date or contradicts another, the assistant faithfully repeats the problem. Fixing the documents often improves answers more than changing the model.
- Gaps in Thai handling. Scanned Thai documents may need OCR, and OCR errors flow straight into answers. Spot-check the extracted text.
- Drift over time. New documents, new products and new staff questions change the mix. A test set that passed in month one may miss month six.
- False reassurance. Users stop checking once the assistant has been right many times. Keep citations visible and keep sampling reviews running.
When not to use a generative assistant
Some tasks should not rely on generated text at all. If the answer must be exact, such as a calculation, a balance, a booking status or a legal filing, fetch it from the system of record and show it directly. Use the language model to understand the question and phrase the reply, not to produce the figure.
Also hold back if your documents are in poor shape. If nobody can say which policy version is current, fix that first. Our guide to building a data foundation before you build AI covers that groundwork.
How to start
- Pick one document set with a clear owner, such as HR policies or product manuals.
- Write thirty to fifty real questions, including some the documents cannot answer.
- Build a small grounded assistant with citations and a refusal rule.
- Grade the answers with an expert, fix the sources, and repeat.
- Release to a small group with a flag button, then widen access.
If you want a second opinion on your document set or your test plan, our team can review it with you. Book a consultation.
Frequently asked questions
What is AI hallucination?
When an AI assistant gives an answer that sounds confident but is wrong or has no source, such as an invented policy clause or date. It happens because language models predict likely text and do not check facts unless connected to a trusted source.
Can AI hallucination be eliminated completely?
No. You can reduce it a lot by answering only from approved documents, showing citations, letting the assistant decline, and testing regularly. Keep a person in the loop for decisions that affect money, health or legal position.
Does RAG stop hallucination?
RAG helps by giving the model your documents to answer from, but it is not a guarantee. Poor sources, weak search or a model that ignores the passages can still cause errors. Citations and a test set let you catch them.
Does it work in Thai?
Yes, but test it. Thai has no spaces between words, so search and chunking need care, and scanned Thai documents need OCR checks. Use real Thai and mixed Thai-English questions in your test set.
Ready to put this to work?
Turn this playbook into a working system
From a first agent to a national platform, the same team plans it, builds it and runs it with you.
