How to Fact-Check an AI Draft: Catching Hallucinations and Fabricated Citations Before You Publish

An AI draft can hand you a number that sounds exactly right, a real person's name attached to words they never said, and a link that goes nowhere. None of that looks wrong on the page. Most teams fact-check AI drafts by skimming for anything odd, which catches the obvious errors and misses every confident one. This guide gives you one repeatable pass for AI content fact checking instead: turn the draft into an evidence ledger, check every claim against a real source, and end with a clear decision to keep, narrow, replace, or remove. If you have been publishing on trust so far, that is normal, and you can fix it this week.

Step 1: Freeze the draft and gather every source

Start by saving the exact version you are checking. Not the one you are still editing. The one you are checking.

Copy the title, body, tables, captions, footnotes, quotes, links, and any source list into a working file. Record today's date next to it. If the tool supplied citations, bring them along, but label them as unverified leads. Asking a model for its sources makes the next step faster. It does not make those sources real.

This matters more than it sounds. The fabricated citations AI systems produce rarely look fake, and a model can serve fluent prose, a plausible statistic, and a citation-shaped link while the sentence is simply false. NIST uses the word confabulation for this: a system confidently presenting content that is wrong, contradicts itself, or drifts from what you asked. Confidence is not evidence.

If your draft came out of DeepSmith, Produced Content is your checkpoint. The Writer researches, links internally and externally, and hands you publish-ready metadata, which is a strong starting point. It is not a promise that every claim has been independently proven. Same rule for Autowrite: a scheduled article still sits behind this gate.

Done when: you have a frozen copy and a complete inventory of sources. Nothing factual is hiding in a heading, an image caption, a table cell, a pull quote, or a link label.

Common mistake: checking one version while working from a source list that belongs to an older one. Freeze first. Log later edits as new checks.

Step 2: Break the prose into one claim per row

Here is where most teams go wrong, so take a breath and slow down for a minute. People verify a paragraph as a whole. One sentence can hold a correct statistic, a wrong interpretation of it, and an invented attribution, all at once.

Read line by line and split compound sentences into single checkable claims. Treat all of these as claims:

• A named person, company, product, place, date, or event.
• A statistic, percentage, ranking, comparison, superlative, trend, or cause-and-effect statement.
• A quote, a paraphrase, or anything attributed to a person or document.
• Anything introduced by "according to," "research shows," or "experts say."
• A claim about what a study, report, or dataset concluded.
• A time-sensitive claim: "current," "latest," "now," "as of."
• An implied fact inside a chart, table, caption, or headline.

Give each claim one row. A ledger that actually works has these fields:

Done when: every factual assertion has a row, and every citation is tied to at least one row. You can point from a sentence to a specific piece of evidence without relying on memory.

Common mistake: skipping the "boring" sentences. Background claims get read as settled fact, which is exactly why a wrong one does damage.

Step 3: Trace each claim back to the original evidence

Now you investigate. The sequence is short and you can hold it in your head: stop, investigate the source, find better coverage, trace the claim to the original.

Stop. Pause on anything unusually precise, surprising, or emotionally loaded. That pause is the whole skill.

Investigate the source. Who created it, why, when, and did they have the access or expertise to know this particular fact?

Find better coverage. Search reputable reporting, library databases, and established fact-checking organizations for widely circulated claims. Use them for context and for competing evidence, not as an automatic stand-in for the primary record.

Trace it to the original. Open the actual paper, report, dataset, filing, transcript, or recording, and compare it with the sentence in your draft.

Rank your evidence honestly:

• Primary: the original dataset, government report, official filing, research paper, transcript, recording, or first-party documentation.
• Secondary: reputable reporting, reviews, database records, professional fact-checks. Good for context and corroboration.
• Discovery only: search snippets, aggregators, quote sites, social posts, another AI answer. These help you find leads. They prove nothing.

The right source depends on the fact. A publisher record can show a paper exists, but you need the paper's methods for a scientific claim. A transcript settles exact wording. An official bio settles someone's job title better than a news mention does.

Done when: you have the original or the best available evidence, you know who made it and when, and your ledger holds the exact supporting passage rather than just a source title.

Common mistake: treating the model's own bibliography, a search snippet, or a page repeating the same unsourced line as independent confirmation.

Step 4: Recalculate and contextualize every statistic

Numbers are where an AI hallucination check earns its keep. A statistic is a measurement, not decoration, so go back to the original table, dataset, or methodology page and match all of this before you keep the wording:

1. Definition. What is actually being counted? Words like "customer," "user," and "market" often carry a special definition in the source.
2. Numerator and denominator. For any percentage, know what sits on each side. Percent and percentage points are not the same thing.
3. Population. Who was eligible to be counted? Sample size, geography, exclusions.
4. Reference period. When was it measured, not just when it was published. Those are different dates.
5. Units and scale. Currency, conversion, inflation treatment, thousands versus millions versus billions.
6. Method and uncertainty. Sampling notes, weighting, margin of error, caveats. Never present an estimate as a precise count.
7. Arithmetic. Recompute the percentages, totals, and growth rates yourself. Check rounding, signs, and decimal placement.
8. Meaning. Is a descriptive number being used to imply cause? Does the trend start at a conveniently chosen year?
9. Corroboration. For consequential or disputed numbers, check a second independent authority. When two sources disagree, compare definitions and methods. Do not average them.

Pro tip: paste the source's full sentence into your ledger before you copy the number into the draft. The surrounding words usually carry the population, timeframe, or limitation the AI summary dropped.

Done when: the ledger records the exact table, row, or page; definition, period, population, units, and method all match; you have recomputed the arithmetic; and the draft still carries the source's caveats.

Common mistakes: mixing millions and billions, swapping percent for percentage points, using a survey result as a population count, comparing nominal money to inflation-adjusted money, and turning a correlation into a cause.

Step 5: Verify every quote against the original

Quotes feel safe because they look specific. They are one of the easiest things for a model to get subtly wrong.

Treat a direct quote as a word-for-word evidence task:

1. Copy a distinctive phrase and search it inside quotation marks. If nothing comes back, try a shorter fragment, since the AI may have changed the punctuation.
2. Find the earliest or original publication, book, report, transcript, interview, or recording. A page that merely repeats the quote is not the original.
3. Match the speaker's exact name, role, organization, date, and setting. If the words come from a written document, say so rather than implying an interview.
4. Compare wording character by character. Watch for dropped words, ellipses, bracketed edits, and punctuation that shifts the meaning.
5. Read the surrounding context. Was the speaker answering something specific? Was the line conditional? Does the omission reverse the point?
6. Check the attribution on its own. A famous name is a warning sign when the quote lives only on quote sites and unsourced posts.
7. Decide the form. Keep quotation marks only when the exact wording is supported. If the idea holds but the wording does not, rewrite it as a clearly attributed paraphrase. If you cannot pin the meaning down, cut it.

Professional standards here are strict for a reason: quotes stay accurate and in context, they are not tidied up for grammar, and ellipses never change what the speaker meant.

Done when: the ledger holds the original wording, the source identity, the setting, the date, and the page or timestamp, and your draft matches all of it.

Common mistake: finding a page with the same words and stopping there. Models often surface a popular misattribution, or fuse a real quote to the wrong speaker.

Step 6: Test citation identity, links, and page content separately

This is the step where you verify AI citations properly, and the key idea is that a citation passes three separate tests, not one. It has to exist, it has to load, and it has to say what your sentence claims. Fabricated citations AI drafts carry usually fail one of those three while sailing through the other two.

Check the bibliographic identity

Compare the draft's title, authors, publisher, venue, date, edition, and document type against the publisher or database record. For scholarly work, search the exact DOI or citation in Crossref Metadata Search or its REST API and compare what comes back.

Two limits worth knowing. Crossref runs on metadata deposited by members, so fields vary and it does not give you full text. A DOI record can prove a work is registered and point you at the right landing page. It cannot prove the paper contains your claim. And a Crossref miss does not automatically mean a fabricated citation, because DOIs registered by other agencies may not appear there. Confirm through the resolver or the publisher before you call it invented.

Test the URL and follow the redirects

Open the link and record where you actually land, not just the first response. A link checker will find broken destinations for you, but it cannot tell you whether a page is authoritative or whether it supports the sentence.

Read the cited content

Use find-in-page or a PDF search for the exact number, phrase, or conclusion. Read enough around it to catch conditions and qualifiers, and to notice when a source is describing a claim rather than endorsing it. A source that exists and loads can still be the wrong source for your sentence.

Keep a stable evidence record

Record the final source identity, the date you checked, and the exact location of the evidence: page and section for a document, timestamp for video or audio. Keep a permitted local copy when a page is likely to change. You are building something reproducible, not a decorative bibliography.

Done when: every citation has passed identity, link, content, and relevance checks, and your ledger tells "real source, wrong claim" apart from "cannot locate the source" and "link needs repair."

Common mistake: adding a green check because the link returned 200. An HTTP status is a transport result, not a fact-check.

Step 7: Resolve conflicts and check freshness

You have rows. Now read them against each other, because contradictions only show up at this level.

Go looking for contrary evidence on purpose, especially for a superlative, a ranking, a strong negative, or any causal explanation. Ask whether your "independent" sources are actually independent, or whether they all copied one press release. Repetition is not corroboration.

When credible sources disagree, compare definitions, populations, timeframes, editions, and update dates before you decide. Do not average measurements that were never measuring the same thing.

Then sweep for anything that moves: prices, headcounts, leadership, laws, product features, and every "latest" or "current." Replace vague currency with an explicit date, because a snapshot dated is honest and a snapshot undated goes stale quietly.

Keep the uncertainty the evidence hands you. If the source supports a narrower claim, narrow the sentence. If two credible sources genuinely conflict and you cannot resolve it, describe the disagreement or drop the claim.

Done when: every row has a status and a reason, material conflicts are resolved or surfaced in the prose, and time-sensitive claims carry a defensible date.

Common mistake: treating a source's publication date as proof its underlying data is current.

Step 8: Apply a hard publish gate

Last step, and it is the shortest. Everything you do to fact-check AI drafts comes down to this moment: filter the ledger by status and act on each row. No maybes.

• Keep: the exact evidence supports the wording.
• Narrow: the evidence supports a smaller claim, a specific date, or a stated population. Rewrite to preserve the limit.
• Replace: the source is real but inappropriate, stale, unreachable, or mismatched.
• Remove: no adequate evidence, a fabricated source, a contradicted claim, or an unverified quote.
• Recheck later: a temporary server failure, or a claim that will update before you publish.

Then run one mechanical sweep of the finished draft. Search for every percentage sign, currency symbol, number, quotation mark, named source, DOI-like string, and for "according to," "study," "research," "latest," "first," "only," and "best." Reconcile that list against your ledger. On a high-stakes piece, have a second reviewer redo the checks on the riskiest rows and sign the log.

Done when: nothing is left marked "probably true," unresolved rows are removed or explicitly qualified, and the version you publish is the version you checked.

Common mistake: checking the draft, then approving a later edit that quietly reintroduced an unverified line.

What to do next

Pick one article you published in the last month and run steps 1 through 3 on it. Just those three. That partial AI hallucination check will tell you fast whether your real risk is invented numbers, mismatched citations, or dead links, and where to spend your attention on the next piece.

Then build the ledger into your workflow instead of your memory. AI content fact checking gets cheap once it is a checklist and expensive every time it is a judgement call. This pass works on drafts from any source, and grounding production in stored context helps at the front end too. Deep IQ keeps your company facts, product details, and a trusted-sources list in one place, so drafts start closer to your reality. It still does not replace the gate. Nothing does.

Want a production pipeline you can run this gate on top of? Start a DeepSmith free trial and see the whole flow, from idea to reviewable draft, before you commit.