AI Hallucinations in Legal Work: Why Lawyers Must Watch for Them

AI hallucinations in legal work are fabricated or misstated legal authorities that an AI model produces with the formatting and confidence of real law: cases that do not exist, quotations no court ever wrote, and real decisions cited for holdings they never reached. The AI Hallucination Cases database, maintained by researcher Damien Charlotin, has cataloged more than 2,100 such filings, 1,492 of them in the United States, and contract disputes are the largest single case category. Hallucinations in legal work are not confined to litigation research. They also reach contract drafting, where a model invents a market-standard position, a defined term, or a cross-reference the document does not contain. Filings containing them fall into four recorded types, and a minority draw a monetary sanction or a disciplinary referral. A fixed verification workflow catches them before a document is filed, and purpose-built legal AI for law firms narrows the rate without eliminating it.

Key takeaways

The core facts about AI hallucinations in legal work, measured rather than assumed:

1. A model hallucinates because it predicts plausible-looking text rather than retrieving a real decision from a database, and retrieval-grounded tools still get it wrong on a measured, nonzero share of queries.
2. The AI Hallucination Cases database has cataloged more than 2,100 filings containing hallucinated authorities worldwide, 1,492 in the United States.
3. Only one of the four recorded hallucination types, a citation to a case that does not exist, fails a simple existence check; the other three require reading the source.
4. Courts have imposed monetary sanctions in 380 of the cataloged filings and disciplinary referrals in 163, with consequences running from a warning on the record to dismissed appeals and bar referral.
5. A fixed, primary-source verification workflow catches hallucinated citations before filing, and the same discipline applies to checking an opponent’s citations.
6. Purpose-built legal AI narrows the hallucination rate measurably but does not eliminate it, and no tool substitutes for verifying the authority yourself.

[cta-1]

How common AI hallucinations are in legal filings

AI hallucinations in legal filings are common enough that courts and researchers now count them. The AI Hallucination Cases database maintained by researcher Damien Charlotin has cataloged 2,185 such filings worldwide as of October 2026, 1,492 of them in the United States, containing 6,484 separate hallucinated items. Monthly additions have run between 60 and 122 through 2026, a rising absolute count as adoption grows.

What a single month of Westlaw data adds

The Thomson Reuters Institute found a sharper, narrower signal: a single Westlaw search of US cases decided between June 30 and August 1, 2025 turned up 22 separate cases in which a court or an opposing party identified non-existent authorities in a filing. The errors are not concentrated in large commercial litigation:

1. Powhatan County School Board v. Skinger (E.D. Va.): a pro se filing citing 42 nonexistent authorities.
2. In re Marriage of Haibt (Colo. Ct. App.): a family-law appeal.
3. In re Martin (Bankr. N.D. Ill.): a bankruptcy matter.

Which tools show up in the data

The database’s tool column, where a tool is named in the decision, is dominated by general-purpose chatbots (ChatGPT or OpenAI tools named in 139 filings) but is not limited to them: Westlaw or CoCounsel appears in 18, Claude in 18, and LexisNexis in 13. A tool built for legal work narrows the rate; it does not remove the tool column from the data.

Why AI tools hallucinate fake case citations

AI tools hallucinate fake case citations because a large language model predicts the next token in a sequence rather than retrieving a record from a database of decisions. A citation is highly patterned text, so a model can assemble a plausible reporter volume, page number, court and year without any of them corresponding to a real decision. Retrieval does not remove the problem on its own. Stanford RegLab researchers found that Lexis+ AI and Ask Practical Law AI returned incorrect information more than 17 percent of the time and Westlaw AI-Assisted Research more than 34 percent of the time, even though all three retrieve from licensed legal databases before generating text.

Two distinct failure modes

The RegLab researchers name two distinct failure modes behind every hallucinated citation:

1. Incorrect: the response states the law wrongly outright.
2. Misgrounded: the response states the law correctly but cites a source that does not actually support it, a failure the researchers argue may be the more dangerous of the two because the citation exists and survives a naive existence check.

Their worked example: a system recited the “undue burden” standard for abortion restrictions as settled law after Dobbs v. Jackson Women’s Health Organization, 597 U.S. 215 (2022), overturned it.

The shapes hallucinations take in practice

In practice, the hallucinations that reach filings take a handful of recognizable shapes:

1. A citation to a case that was never decided.
2. A real case quoted for language it never contains.
3. A real holding stretched to cover facts it never addressed.
4. A statute or regulation cited as current when it has been repealed or superseded.

The four types of AI hallucinations recorded in case law

Only the first type, a fabricated authority, fails a simple existence check. A lawyer who confirms every citation is real has caught 3,583 of the 6,484 cataloged items and missed the other 2,901, which are a real case cited for the wrong thing, a quotation that does not exist, or law that is no longer good. Verification has to go past “does this case exist” to “does this case say what the brief says it says.”

High-profile AI hallucination cases in legal practice

Four AI hallucination court cases illustrate the range of consequences, from a formal warning on the record to a $5,000 sanction, and in one case liability for an organization’s own chatbot.

Mata v. Avianca and the pattern it set

Mata v. Avianca, Inc. (S.D.N.Y. 2023) set the pattern other courts now cite: the attorneys submitted a brief citing six fabricated cases, and the court imposed a $5,000 sanction after finding the attorneys had not personally confirmed a single one of the citations before filing, and had submitted fabricated case excerpts when the court directed them to produce the decisions. The public docket records the full sanctions opinion.

Park v. Kim: a referral instead of a fine

Park v. Kim (2d Cir. 2024) involved fake citations in an appellate brief and ended in a referral to the circuit’s grievance panel, not a monetary sanction, showing that referral and fine are independent consequences a court can impose separately or together.

Moffatt v. Air Canada: the organizational-reliance precedent

Moffatt v. Air Canada (B.C. Civ. Res. Trib. 2024) did not involve a court filing at all. The tribunal held the airline liable for its customer-service chatbot’s false statement about bereavement fares. The case matters outside litigation because it holds an organization to what its AI said, whether the AI appears in a brief, a chatbot transcript, or a contract.

Ko v. Li: the lightest consequence on the ladder

Ko v. Li (Ont. S.C.J. 2025) involved fake citations in a notice of motion and drew a formal warning, the lightest consequence on the ladder.

Where AI hallucinations show up in contract work

AI hallucinations in contract drafting look different from fabricated case citations, and contract disputes are the largest single case category in the hallucination database at 457 of the cataloged filings. In transactional work the model does not usually invent a case. It invents a market-standard position, states a defined term the agreement never defines, points to a section number that does not exist, or cites a statutory provision inside a compliance clause that was repealed.

Each failure mode carries its own practical consequence:

1. An invented market-standard claim (“this indemnification cap is standard for this deal size”) loses a negotiation on a false premise, because the other side may not catch the error either.
2. A phantom defined term, a capitalized phrase the model used as though the agreement defines it when it does not, creates an interpretive gap a counterparty can exploit later.
3. A broken cross-reference, a section number that points to the wrong clause or to nothing, survives execution unnoticed and surfaces only once a dispute forces someone to read the contract literally.
4. A repealed statutory citation inside a compliance representation becomes a warranty the client cannot actually satisfy, discovered only when a regulator or counterparty tests it.

Moffatt v. Air Canada, covered above, is why this exposure is not confined to litigation: a contract drafted with an AI-invented term or a broken cross-reference creates the same document-accuracy exposure before any court ever sees it.

What happens when a hallucinated citation reaches a judge

The AI Hallucination Cases database has cataloged monetary sanctions in 380 of its filings and disciplinary referrals in 163, and a court can impose more than one consequence at once:

1. A warning on the record, with no further penalty.
2. Fee-shifting to the party that filed the fabricated authority.
3. A monetary fine, ranging from token amounts to several thousand dollars.
4. A struck brief, which forces a corrected refiling.
5. A dismissed appeal, where the fabricated authority was central to the argument.
6. Referral to a state bar grievance committee.

The Rule 11 certification behind every filing

The signature on a federal filing already certifies the paper’s legal and factual contentions under Rule 11(b) of the Federal Rules of Civil Procedure; a hallucinated citation is a Rule 11(b) violation whether or not the attorney knew the citation was fabricated, since Rule 11 applies an objective reasonable-inquiry standard rather than a knowledge requirement.

How the ladder plays out in practice

The ladder is not keyed to the number of fabricated citations. Kaur v. Desso (N.D.N.Y.) drew a $1,000 fine and mandated AI continuing legal education after the attorney admitted knowing AI tools fabricate citations and filed under deadline pressure anyway.

The cost beyond the sanction

Beyond the individual sanction, hallucinated filings cost judicial time that courts have limited capacity to absorb, and each public incident erodes the credibility AI-assisted filings carry with the bench. Both costs compound the individual sanction rather than replacing it.

[cta-2]

A verification workflow that catches hallucinations before filing

To detect AI hallucinations in legal briefs, check every citation against a primary source before the document leaves your desk, in a fixed order rather than by spot check. The workflow assumes the model is wrong until a primary source says otherwise.

The eight-step check

1. Confirm the case exists in an official reporter or a verified database such as Westlaw, Lexis or a free court-record source, not the AI tool’s own output.
2. Open the actual opinion and read the passage the brief relies on; do not stop at the case name and citation format looking correct.
3. Confirm the authority supports the proposition it is cited for, not merely that the case is real. This is the step the four-types data above makes mandatory: fabrication is only one of four failure modes, and the other three survive a bare existence check.
4. Check that the cited holding has not been overturned, narrowed, or superseded, using a citator (Shepard’s or KeyCite) rather than assuming currency.
5. Verify every statutory or regulatory citation against the current official code database, not a summary the AI tool generated.
6. Confirm quotations character-for-character against the source text; a fabricated quotation can attach to a real, correctly cited case.
7. Document the verification in the file: which source was checked, when, and by whom.
8. Require a second reviewer’s sign-off on any brief drafted substantially with AI assistance before filing, separate from the drafting attorney.

At the firm level

The same discipline needs process support beyond the individual check:

1. A written AI-use policy that names which tools are approved for filings.
2. Mandatory training before anyone files AI-assisted work.
3. A periodic audit of filed briefs against this workflow, rather than a one-time rollout.

The National Center for State Courts frames this as “never trust, always verify,” and treats technical safeguards and risk ratings for specific tools as a complement to the verification steps, not a substitute for them.

What to do when you find a hallucinated citation, in your filing or an opponent’s

What to do if opposing counsel cites a hallucinated case depends on whose filing it is: the same documentation discipline applies whether the fabricated citation belongs to the other side or to you.

If it is the other side’s filing

When opposing counsel cites a case you cannot find, document your search before you raise it:

1. Which databases you checked.
2. What search terms you used.
3. When you ran the search.

Give opposing counsel the chance to correct the record once you have raised it, then bring it to the court with your search documented if they do not. The general duty of competence under Model Rule 1.1 requires reasonable diligence in verifying the law a matter depends on, which supports checking an opponent’s citations rather than assuming the court will catch a fabricated one.

If it is yours

The database’s response data shows what happens once a lawyer’s own hallucination surfaces: of the lawyers whose hallucinated citations were caught, 404 attempted to withdraw or correct the filing and 384 apologized, while 98 denied the error and 45 doubled down on the fabricated authority. Denial and doubling down draw harsher sanctions than a prompt correction, because they turn a drafting error into a candor problem.

The professional responsibility framework behind verification

A lawyer’s ethical duty to verify AI output runs through existing Model Rules rather than any new AI-specific rule. ABA Formal Opinion 512, issued in July 2024, applies competence, confidentiality, communication, supervision and candor to generative AI without creating a separate standard for it.

The five duties the opinion applies

1. Competence (Model Rule 1.1 and Comment 8): a lawyer must understand the technology well enough to use it reasonably, which the opinion treats as requiring independent verification of AI-generated legal analysis before relying on it.
2. Confidentiality (Rule 1.6): inputting client information into a general-purpose AI tool can constitute an unauthorized disclosure, depending on the tool’s data-retention and training practices.
3. Communication (Rule 1.4): a client may need to know that AI materially shaped the work product, particularly where the client’s own instructions or risk tolerance bear on the tool’s use.
4. Supervision (Rules 5.1 and 5.3): a supervising lawyer is responsible for a subordinate’s or a nonlawyer assistant’s AI-assisted work product, including an associate’s or a contract reviewer’s unverified output.
5. Candor to the tribunal (Rule 3.3): once a lawyer knows a filed citation is fabricated, the duty of candor requires correcting the record, independent of any sanctions motion.

Where state bars and individual judges go further

Individual judges go further than the ABA opinion through standing orders that require a signed certification confirming manual verification of AI-generated citations before filing, a requirement that sits alongside the separate state bar ethics rules on AI use that already bind every licensed lawyer. These judicial orders are issued court by court rather than under a single jurisdiction-wide mandate, so a lawyer filing in multiple courts cannot assume one court’s certification requirement carries over to another, and which courts currently require disclosure changes as new orders are issued.

What purpose-built legal AI changes, and what it does not

Purpose-built legal AI tools still hallucinate, at a lower measured rate than general chatbots and not at zero. Stanford RegLab measured general-purpose chatbots hallucinating on 58 to 82 percent of legal queries, against more than 17 percent for Lexis+ AI and Ask Practical Law AI and more than 34 percent for Westlaw AI-Assisted Research.

What narrows the gap in general

Grounding outputs in a defined, licensed document set and surfacing the source alongside the output narrows the hallucination gap across legal AI tools generally, letting a human check the output in one step rather than running a separate search. Better prompting lowers the rate further but does not remove the verification step; a well-specified prompt reduces the chance a model invents an answer, and it cannot make a retrieval failure disappear.

What Spellbook does and does not fix

Spellbook is a commercial-contract specialist, not a case-law research platform. It is not the fix for a hallucinated citation in a brief, because that is not the problem it is built to solve. Where it fits is the contract-drafting failure modes described above:

1. The Review feature grounds every suggestion in the customer’s own playbooks, precedents and data from 20M+ real contracts, rather than unverified general web content.
2. Clickable Citations link every suggested revision back to the source language in the contract, so a reviewer can check it without leaving Word.
3. Proofread surfaces definition and section-reference defects directly, including terms defined but never used, terms used but never defined, incorrect or missing section references, and leftover template content, which maps directly onto the broken-cross-reference and phantom-defined-term failures described above.

Try Spellbook free to see how the Review feature grounds contract review in your own playbooks and precedents, with every suggestion traceable back to its source.

Frequently asked questions

The recurring questions about AI hallucinations are how to spot a fake citation, which tools produce them most often, and whether the problem is improving.

How can I tell if an AI-generated case citation is fake?

Look it up in an official reporter or a verified legal database, not the AI tool’s own output, and read the actual opinion rather than stopping once the case name resolves. Existence is not enough: confirm the case also supports the point it is cited for, since three of the four recorded hallucination types involve a real case cited incorrectly rather than a case that does not exist.

Can ChatGPT hallucinate court cases?

Yes. General-purpose chatbots are the most frequently named tool in the AI Hallucination Cases database, appearing in 139 cataloged filings, more than any purpose-built legal research tool in the dataset.

Are some AI tools more prone to hallucinations than others?

Yes, measurably. Stanford RegLab found general-purpose chatbots hallucinating on 58 to 82 percent of legal queries, compared with more than 17 percent for Lexis+ AI and Ask Practical Law AI and more than 34 percent for Westlaw AI-Assisted Research. All three figures are nonzero, which is why verification applies regardless of which tool produced the draft.

Is the hallucination problem getting better?

Not by the measured trend. Monthly additions to the AI Hallucination Cases database ran between 60 and 122 filings through 2026, a rising absolute count even as individual tools improve, because adoption is growing faster than the error rate is falling.

What should I do if I discover I have cited a hallucinated case?

Correct it immediately: notify the court and opposing counsel, withdraw or amend the filing, and document how the error happened. Lawyers who move quickly to correct or withdraw draw lighter consequences than those who deny the error or let it stand.

Can courts detect AI hallucinations automatically?

Not reliably yet. Detection today depends on a human, typically opposing counsel or a clerk, manually verifying a citation and finding nothing, which is why the verification workflow above exists as a manual discipline rather than a tool a court runs for you.