Checking AI’s Work

AI answers arrive fluent, confident and occasionally wrong. This page gives you a routine for catching the errors, a method borrowed from fact-checkers, and a sense of when a quick glance is enough and when it is not.

In 2023 a New York lawyer filed a court brief citing six earlier judgments that supported his client. Neither the other side’s lawyers nor the judge could find them, because they did not exist: the lawyer had asked a chatbot, which invented the cases complete with names, quotations and page numbers, and he had not checked. He and a colleague were fined. (The full story is on What AI Gets Wrong.) The tool did what language models sometimes do. The failure was the missing habit. This page is about building that habit, so that AI makes your work faster without making it wrong.

Why fluent is not the same as correct

Consensus A large language model generates the most plausible continuation of your request. Usually plausible and true coincide; sometimes they do not, and the model produces a hallucination in exactly the same confident tone as a fact. What AI Gets Wrong explains why. Errors cluster in predictable places: exact numbers and dates, quotations, citations and links, niche or local facts, recent events, and arithmetic done "in its head". Tools that search the web or your documents — grounding — reduce the problem but do not remove it As of 2026: the model can still misread a source, or cite one that does not say what it claims.

So verification is not a sign of distrust in a particular product. It is the normal price of using a fast, fallible helper, the way a manager reads a new assistant's first drafts.

A verification routine

This guide’s framing Work down this ladder. For a casual answer you might stop after step one; for anything you will publish, sign or act on, go all the way.

  1. Pick out the claimsList the specific factual statements the answer depends on: names, dates, numbers, rules, quotations. Ask the AI itself: "List every factual claim in your answer as bullet points."
  2. Check that sources exist — and say itOpen every cited link or paper. Does it exist? Is it by whom the AI says? Does the passage actually support the claim? Fabricated or mismatched citations are among the most common serious errors.
  3. Recompute the numbersRedo any calculation you will rely on with a calculator or spreadsheet. Check units, percentages of what, and whether totals add up.
  4. Ask for uncertainty"Which parts of this are you least sure about, and why?" Models are imperfect judges of their own reliability, but the answer often points to the weak spots.
  5. Cross-check independentlyConfirm key points in an authoritative source — an official website, the original document, a reference book. A second AI model is a useful extra check, but agreement between models is not proof: they can share the same mistake.
  6. Watch for red flagsVery precise figures with no source; quotations that sound too neat; links that 404; dates that do not fit; an answer that changes when you ask again; and anything that conveniently confirms what you hoped.
Try this prompt

Review your previous answer as a strict fact-checker. For each factual claim, give: (1) the claim, (2) how confident you are — high, medium or low — and why, (3) the kind of source that would confirm it. Flag any number you did not calculate with a tool and any citation you cannot be sure exists. Do not add new claims.

Then do the checking yourself. This prompt helps you find where to look; it does not do the looking.

The SIFT method

Fact-checkers and librarians have a well-tested routine for anything you meet online, and it works just as well for an AI answer and the sources it points to. The digital-literacy researcher Mike Caulfield set out SIFT in 2019, building on studies of how professional fact-checkers read "laterally" — leaving a page to see what others say about it — rather than reading deeply within it. AI for Research and Learning shows the method in use when researching with AI, and Cutting Through the Noise applies the same habits to AI headlines and claims. Tap each move.

For AI work, the "T" is the move that saves you most often. If an assistant says "a 2022 study found…", trace it: find the study, read the abstract, and check that it really found that.

Practice: spot the planted errors

Each of the following is a sample AI answer containing one planted problem. Find it.

Spot the error

“Malaysia’s Personal Data Protection Act was passed in 2015 and applies to personal data processed in commercial transactions.” What is wrong?

It is the Personal Data Protection Act 2010 (amended in 2024). Dates are a classic slip — and the name of the Act gives it away.

“With a 15% discount, a RM2,400 laptop saves you RM320, so you pay RM2,080.” What is wrong?

Both figures are internally consistent, which makes the error easy to miss. Recompute the first step.

“Mount Kinabalu, at about 4,095 m, is the highest mountain in Southeast Asia.” What is wrong?

Kinabalu (Sabah) is Malaysia's highest peak, but Hkakabo Razi in Myanmar is far higher. A true fact stretched one step too far is a common pattern.

“Alan Turing proposed the imitation game in ‘Computing Machinery and Intelligence’, published in Nature in 1950.” What is wrong?

The paper appeared in the philosophy journal Mind. Citation details are where models most often blend facts.

An answer says: “According to Lim and Rahman (2021), Journal of Asian Retail Studies, 68% of shoppers prefer e-wallets.” What should you do first?

A full-looking citation is not evidence. Trace it; if you cannot find it, do not use it. (This citation was invented for this exercise.)

You ask the same question twice and get two different figures. What does this most likely tell you?

Inconsistency across attempts is a red flag that the answer is being generated, not retrieved.

Matching effort to stakes

Checking everything to the last detail would erase the time AI saves. The skill is calibration: light checks where errors are cheap and visible, full checks where they are costly or hidden.

Brainstorming, first drafts, rewording your own text, ideas for a party or a lesson. You are the filter: you will judge each idea anyway, and there is no hidden factual claim. A quick read for sense is enough.

Key point

The question is not "Is AI reliable?" but "What happens if this particular answer is wrong, and who would notice?" If the answer is "something serious, and nobody", verify fully.

Two habits make all of this lighter. First, ask for sources from the start ("cite the official source for each rule"), so checking means clicking rather than searching. Second, keep a short personal log of errors you catch: within weeks you will know where your tools are weak, and where you can relax.

// Pause and reflect

  1. Think of the last AI answer you used. Which claims in it did you check, and which did you take on trust?
  2. Which of your regular tasks fall into the "essential" tab? Who is the authoritative source for each?
  3. Which of the six red flags have you already met? What did you do?

Your notes are saved privately in this browser.

Try this

Ask an assistant for a one-page briefing on a topic you know well, with at least three sources. Run the full ladder: list the claims, open every source, recompute any number, ask for its confidence, and cross-check two claims in an official source. Count the problems you find, and note which step found each one.

Key terms on this page

Sources & further reading