Why AI makes confident mistakes
Language models predict likely text. They do not check facts the way a person does. They can invent a statistic, a quote, a law or a link, and present it in perfect prose. The more specific the claim, the more carefully you need to check it.
The four checks before anything reaches a client
Run every output through four quick checks. It takes a few minutes and protects your reputation.
- Facts: are names, dates, laws and claims correct? Check them against a reliable source.
- Numbers: do the figures add up, and where do they come from?
- Tone: does it sound like the client, and is it appropriate for their audience?
- Fit: does it answer the brief, or did it drift?
Edit or start again?
If the structure is right and only details are wrong, edit. If the answer misunderstood the task, do not patch it: improve the prompt and generate again. Patching a bad draft usually takes longer than a fresh, better-briefed one.
Keep an error log
Note every mistake you catch: what it was, which tool, which type of task. After a few weeks you will see patterns, such as invented sources or wrong local rules, and you can add warnings to your prompts and checklists.
Example
A near miss
An AI draft for a dental practice said a procedure was "painless and risk-free". The four checks flagged it: it is a medical claim the practice could not make. The final text said "Most patients describe the procedure as comfortable; your dentist will explain the risks."
Prompt to try
Replace the [brackets] with your own details and review the result before using it.
Review the text below as a careful editor. List: 1) every factual claim that should be checked, 2) any number without a source, 3) anything that could be a legal, medical or financial promise, 4) parts that do not answer this brief: [brief]. Text: [paste].
Exercise
Take one AI answer from your work this week and run the four checks.
