ChatGPT for Lawyers: Where It Works and Where It Stops

ChatGPT performs well for lawyers when the material it needs is already inside the prompt, and it fails when the answer has to come from outside the prompt. That distinction decides which parts of a commercial lawyer's week ChatGPT can carry.

ChatGPT handles six recurring pieces of commercial legal work, and it handles each one differently. Below: how well it handles each task, with the conditions each result depends on, the measured error rates on legal queries, what a usable legal prompt has to carry, and the point at which the work belongs with software built for in-house legal teams instead of a general assistant.

[cta-1]

Key Takeaways

ChatGPT for lawyers is a competent summarizer and first-drafter and an unreliable researcher, and that split is structural, not a matter of prompting skill.

1. ChatGPT is strongest where the source document sits in the prompt: summarizing, comparing two versions of a clause, rewriting dense language in plain English, and drafting from a precedent the lawyer supplies.
2. ChatGPT is weakest where the answer has to be retrieved from outside the prompt, which is where fabricated citations, invented holdings and confident but wrong statutory summaries come from.
3. Measured hallucination rates on specific legal questions ran between 69% and 88% in the Stanford study described below, and the legal research products built to fix the problem still erred on a sizeable share of queries.
4. A prompt naming the role, the jurisdiction, the permitted context, the output format and the verification step produces a materially better draft, and changes nothing about the reliability of anything retrieved.
5. The verification step does not expire as the software improves.

What ChatGPT Does Well for Lawyers, and What It Does Not

What can ChatGPT do for lawyers? Six recurring things, each at a different level of reliability. The table below gives the task, how well ChatGPT handles it, the condition the result depends on, and the failure when the condition does not hold.

The pattern in that table predicts most of what a lawyer will get from the tool. ChatGPT is strong where the input is in front of it and weak where the answer has to come from somewhere else, which is a property of how the model produces text, not a gap a newer release closes. Two conditions sit above all six rows, and both are settled before any prompt is typed.

What Has to Be True Before Any of This Holds

1. Whether the use is permissible at all, which turns on the client's engagement terms, the firm's own policy, any standing order in the matter, and the duties of competence and confidentiality in the relevant jurisdiction.
2. Whether the material may be pasted in, which turns on what the account's plan does with the text and on whether the document identifies a client or a matter.
3. Whether the output has to be defensible to a client, a counterparty or a court, which sets how much verification the task needs before it leaves the lawyer's desk.

The duties of competence and confidentiality apply to the use of any tool, and whether a particular use is permissible is worked through in the guide to whether lawyers may use ChatGPT at all. Treat permissibility as a threshold question, not a disclaimer, because the answer changes what may go into the prompt.

The second question is narrower and more often answered wrongly. Pasting a contract into a consumer account is disclosure to a third party, and the retention and training behavior differs by plan, not by how the lawyer intends to use the output. The plan-level detail and the settings that change it are set out in the guide to what ChatGPT does with a pasted document.

Clause Research and Plain-English Translation

The clause-research row is where ChatGPT is most useful and most quietly wrong, and the line between the two is the difference between reading and retrieving:

1. Asking what a clause does, with the clause text in the prompt, is a reading task and the answer holds up.
2. Asking what the clause means in a named jurisdiction is a retrieval task, answered in the same confident register at a much lower hit rate.
3. Asking how a court has treated the clause is retrieval with a citation attached, which is the highest-risk request on the page.
4. Asking whether the clause is enforceable as drafted is a legal conclusion, which is a question for a lawyer rather than a prompt.

A force majeure clause shows the split cleanly. The model parses the trigger events and the notice mechanics in front of it, then states how a court treats a pandemic under that trigger with identical confidence and no reliable basis.

How Do Lawyers Use ChatGPT in Legal Work?

The usage that survives contact with practice shares one property: the lawyer supplies the material and the model supplies the language.

1. Summarizing an agreement, a document set or a long email chain into the points a deal lead needs
2. Producing a first draft of a clause or a short agreement from a precedent the lawyer pastes in
3. Rewriting dense contract language in plain English for a business stakeholder
4. Comparing two versions of the same clause and naming the substantive differences
5. Drafting internal correspondence, status notes and meeting summaries
6. Explaining an unfamiliar term of art well enough to ask a better question of a colleague
7. Producing a plain-language briefing note on an agreement for a stakeholder outside the legal team

What the Model Does Not Do

The model proposes language and the lawyer decides whether the language is right, which is the relationship a lawyer has with a junior associate's first attempt. This guide covers commercial contract work, so litigation workflows sit outside it. Predicting how a matter will resolve is excluded too, because the model has no evidenced basis for it.

Can ChatGPT Draft and Review Contracts?

ChatGPT can draft and review contracts to a first-pass standard, and the review half is the weaker of the two. Given the full text and a clear instruction about what to look for, it surfaces issues worth a lawyer's attention: a missing limitation of liability, an uncapped indemnity, a one-way termination right, an auto-renewal with a short notice window.

What it does not do is measure the agreement against the team's positions, because it does not hold them. A review that matters answers a narrower question than "what are the risks here", and that narrower question is whether this draft departs from what the team has already decided to accept.

How to Make a First-Pass Review Actionable

1. Give it the clause text and the position the team requires, instead of asking for risks in the abstract
2. Ask it to name what is absent, because silence is the defect a general risk scan reliably misses
3. Require a quotation from the document for every issue it reports, which exposes the invented ones
4. Treat anything it says about enforceability as a question for a lawyer, not an answer
5. Run it clause by clause on the provisions that carry the money, instead of once across the whole agreement
6. State the counterparty and the commercial context, because the same clause carries different weight on a vendor paper than on a customer one

How far ChatGPT can be trusted on the review half specifically is a longer question than this section takes on, and it is worked through in the guide to reviewing a contract with ChatGPT.

Can Lawyers Use ChatGPT to Write Legal Documents?

Yes, with a lawyer drafting from it rather than relying on it. The model predicts plausible contract language and does not retrieve the law that governs it, so generated provisions can be unenforceable or incomplete in ways that are not visible on the page.

That is a statement about provenance, not about writing quality. A document produced from a supplied precedent inherits whatever was in that precedent; a document produced from nothing inherits the drafting conventions of whatever sat in the training data, which is a weaker basis than any firm's own paper. The consequences are narrow and worth naming.

Where a Generated Document Goes Wrong

1. Defined terms survive from the source precedent and are left dangling when the clause they supported is cut
2. Cross-references renumber incorrectly, and the model restates them confidently
3. Governing-law and dispute-resolution language arrives in a plausible generic form that may not suit the jurisdiction
4. Commercial positions from the source deal survive into the new one unless they are called out
5. Schedules and annexes are referred to in the body and then never produced

ChatGPT remains a reasonable instrument for initial research framing, quick queries and basic contract drafts, provided a lawyer reads every line of what comes back.

Legal Research with ChatGPT

ChatGPT is a poor instrument for legal research, and this is the one row in the table where no amount of prompt discipline changes the result. ChatGPT does not search a legal database. It produces the most probable next text given the prompt, so a request for authority returns text shaped like authority, carrying citations shaped like citations.

Legal-specific research platforms such as Thomson Reuters's Westlaw and LexisNexis are subscription products that retrieve from curated collections rather than generating from training data. That is a real difference in kind, and it narrows the problem without removing it: products that retrieve before they generate still produce errors at the rates measured below, so a subscription is not a substitute for reading the authority. What ChatGPT can do in a research workflow is frame the question.

What ChatGPT Can Do in a Research Workflow

1. Explain a doctrine well enough that a lawyer knows what to search for
2. Draft the search terms and the structure of a research memo
3. Translate an unfamiliar area into the vocabulary its practitioners use
4. Summarize a statute or an opinion that the lawyer pastes in, which is a reading task rather than a retrieval one

Each of those is a starting point a lawyer then verifies in a primary source, and none of them is a citation a lawyer may rely on. The depth past that point belongs to a research platform and to the lawyer's own reading.

Can Lawyers Use ChatGPT for Client Communication?

ChatGPT for client communication works for lawyers at first-draft stage and fails at final-draft stage, and the reason is register, not accuracy. The general-purpose tool produces the shape of the right client email and fills the gaps with whatever is most agreeable, which is a different defect from a factual error and a worse one in a live negotiation. Using ChatGPT for client communication shifts the lawyer's job from writing the email to auditing it.

Asked to write a follow-up email, the model defaults to an accommodating tone, because accommodating text is the most probable text. On internal correspondence that is harmless. On an email to the other side of a live negotiation it is a liability.

What Actually Goes Wrong in Counterparty Correspondence

The failure is not tonal. The model invents a commitment the deal has not made. Asked to acknowledge a counterparty's markup politely, it writes that the client is happy to accommodate the revised indemnity, or that the timeline is workable, because those are the sentences that usually follow in correspondence of that shape. Nothing in the prompt said so, and nothing in the output marks it as new.

The Check That Catches It

1. List every commitment, concession and date in the output, then confirm each one against the deal record
2. Delete any sentence that characterizes the client's position instead of stating it
3. Specify the register in the prompt, because the default register is wrong for adversarial correspondence
4. Read the first and last lines separately, since that is where the invented warmth collects
5. Check any date the output introduces against the agreement, because the model fills gaps with plausible ones

Client-facing drafts are a softer case, and the same list applies to anything stating a position, a deadline or a fee.

[cta-2]

How Reliable Is ChatGPT for Legal Work?

ChatGPT is unreliable for legal work at a rate that has been measured. A January 2024 study from Stanford's RegLab and Institute for Human-Centered AI, testing GPT-3.5, Llama 2 and PaLM 2 across more than 200,000 legal queries, found hallucination rates between 69% and 88% on specific legal questions.

What the Measured Rates Say

1. Asked about a court's central holding, the models tested were wrong at least 75% of the time.
2. Asked whether two cases were in tension with each other, performance was no better than random guessing.
3. The failures were confident. The models did not signal uncertainty where they were wrong, which is what makes the rate dangerous and not merely inconvenient.
4. The rates were measured across more than 200,000 queries per model, so they describe the models' ordinary behavior, not a set of adversarial edge cases.

Those figures come from models several releases old, and the honest reading is not that a current version scores the same. The failure is structural to generating legal text, not a defect a version bump removes, and the evidence for that is what happened when researchers tested the tools built specifically to fix it.

Why a Newer Model Does Not Close the Gap

A follow-up study from the same Stanford group tested the retrieval-grounded legal research products, including Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI. Those systems retrieve real authority before generating an answer, which is the architectural fix for fabrication, and they still hallucinated on between 17% and 33% of queries. That is a large improvement over an ungrounded model. It is not a clean record. And it describes purpose-built legal software, not a general assistant.

A running index of court decisions in which AI-fabricated citations were identified is maintained as a public database, and the entries are not confined to self-represented litigants.

What Follows for a Working Lawyer

1. Treat every citation, holding and statutory quotation as unverified until it has been read in a primary source.
2. Treat the output's confidence as carrying no information about its accuracy.
3. Keep the verification step permanently, because the Stanford finding on grounded tools shows it does not expire as the software improves.
4. Record who verified what in the matter file, so the check survives a handover and does not have to be repeated from scratch.

The consequences of skipping that step are documented, not theoretical, and the best-known instance, in which a filing built on fabricated citations led to sanctions, is set out in the account of the lawyer who relied on ChatGPT in a federal filing.

How to Write a Legal Prompt for ChatGPT

A usable legal prompt for ChatGPT carries five things, and each one changes a specific property of the output. Writing a legal prompt for ChatGPT is mostly a question of what the prompt specifies, not how it is phrased, and a prompt that specifies less gets a more generic answer back. None of the five changes the reliability of anything the model had to retrieve: a better prompt produces a better draft and does not make a fabricated citation less likely. Thomson Reuters sets out a similar list in its own guidance on writing legal AI prompts.

The Role and the Reader

Naming the role sets the register and the level of detail. "You are reviewing this as in-house counsel for the buyer" produces a more useful read than the same request with no role, because it tells the model whose exposure matters. Naming who will read the output does the second half of the work, since a note for a deal lead and a note for a general counsel want different levels of qualification.

The Jurisdiction and Governing Law

Stating the governing law stops the model averaging across jurisdictions, which is the model's default behavior and the source of its most plausible errors. It does not make the answer authoritative. It narrows what the model generates and tells the lawyer what to verify.

The Context the Engagement Permits

Whether the document is in the prompt changes the error rate more than any other single variable. A review of a pasted agreement is a reading task; a review of a described agreement is a retrieval task, and the two carry very different error rates. The constraint is what the engagement and the account's plan permit, which is why the permissibility and confidentiality questions sit near the top of this guide rather than at the bottom.

The Output Format

Specifying the format separates output a lawyer can act on from output a lawyer has to re-read before acting on it. A table of clause, issue and required position is actionable. A request for comments on the agreement is not. Asking for a quotation from the source document against every point makes invented content visible.

The Verification Instruction

Asking the model to mark what it is uncertain about, and to separate what it read in the document from what it supplied, surfaces some of the fabrication. Some of it, not all, and a prompt instruction is not a control. The control is the lawyer reading the authority.

A Thin Prompt Against a Specified One

A thin prompt about a nondisclosure agreement (NDA), "Review this NDA and tell me if there are any issues", returns a generic list of contract risks. The same task, specified, leaves none of the five to chance:

1. The role and the side: reviewing as counsel for the disclosing party
2. The governing law: New York
3. The position to test against: a two-year confidentiality term and a mutual carve-out for independently developed material
4. The output format: one row per departure, with the clause text quoted
5. The verification instruction: separate what the clause says from what the model is supplying

The second version returns something a lawyer can take to a deal lead. Worked examples for the recurring tasks sit in the collection of prompts written for legal work, which is where to go for finished versions, not the anatomy.

When to Use Legal AI Instead of ChatGPT

Legal AI instead of ChatGPT is the right choice at three points in the work. Each is a property of the work, not a question of which tool writes better prose, and at each one the work cannot be done in a chat window: the revision has to arrive as a tracked edit in the counterparty's own file, the standard being applied is the team's own playbook, and the market question needs a measured answer, not a guess.

When the Revision Has to Come Back as Tracked Changes

A markup a counterparty will read has to arrive as tracked changes in the file they sent, with the rationale attached as a comment on the clause it concerns. ChatGPT returns text in a chat window, so the lawyer re-keys the revision into the document, which is where the time saving goes. Spellbook's Review feature works inside the agreement and writes proposed revisions back as marked edits with comments on the clauses they concern, as native Track Changes in Word and as suggestions in Google Docs, so the output of the review is the document itself, not a description of it.

When the Standard Is the Team's Own Playbook

A review measuring a draft against general drafting practice answers a question nobody asked. The question is whether this draft departs from the positions the team has already decided to hold, which requires those positions to sit in the system, not just in the prompt. The Draft feature applies a team's recorded positions and precedent language to a new agreement, which is the difference between a generic suggestion and the clause the team actually uses.

When "Is This Market" Needs a Real Answer

Whether a liability cap at one year of fees is market cannot be answered from training data, because the model has no principled way to separate a widely copied template from a widely accepted position. The Compare to Market feature measures the term against a set of real executed agreements and 2,300+ benchmark standards, which makes it a retrieval question with a measurable answer, not a judgment the model is guessing at.

What This Does Not Solve

None of that removes the verification step, and the Stanford finding on grounded legal tools is the reason to say so plainly: purpose-built legal software reduces the error rate and does not reach zero, and the output is reviewed by a lawyer like any other draft. Spellbook is a commercial-contract specialist, not a legal-research or litigation platform, so research, case analysis and litigation workflows sit outside it. The wider set of options across those workflows is surveyed in the overview of legal AI tools.

Try Spellbook free to see a contract review run against your own positions instead of against general drafting practice.

[cta-3]

Frequently Asked Questions

Is There a ChatGPT for Lawyers?

There is no legal edition of ChatGPT. Two different things are sometimes described that way: general assistants with a legal prompt applied to them, which behave exactly like the general tool, and purpose-built legal software that retrieves from curated contract or research collections before generating. The second category performs measurably better on retrieval tasks and still requires verification. A direct comparison of the general models on legal work is set out in the comparison of Claude and ChatGPT for legal use.

Which ChatGPT Should a Lawyer Reach For?

For work where the document is pasted into the prompt, the choice of plan matters more than the choice of model, because the plan determines what happens to the text. For work requiring retrieval, no consumer plan makes a general assistant a reliable source of authority. Settle the plan question against the confidentiality duty first and the model question second.

How Accurate Is ChatGPT in Legal Document Preparation?

Accuracy is high for language and low for anything the model had to supply from outside the prompt. Drafting from a supplied precedent is reliable at the sentence level. Statements about enforceability, jurisdiction-specific consequences and authority are not, and they arrive in the same confident register as the parts that are correct.

Is It Safe to Put Sensitive Legal Information Into ChatGPT?

Entering information that identifies a client or a matter into a consumer account is disclosure to a third party, and encryption in transit does not change that. What a plan does with the text, how long it retains it and whether it is used for training differ by plan, so the plan terms and the account settings are what to check before any document goes in.

Which Legal Tasks Are Worth Prompting at All?

The tasks worth prompting are the ones where the material is already in hand: a document to summarize, a clause to rewrite, a precedent to draft from, two versions to compare. The tasks not worth prompting are the ones that require the model to recall something, because that is where the error rates sit regardless of how well the request is written.

How Should Lawyers Verify a Response?

Pull up each citation and confirm three things separately: that the case or the section exists at the reference given, that it says what the response says it says, and that it has not been superseded. Then confirm every quotation against the document it is attributed to. Fabricated authority arrives in the same format as real authority, so format carries no signal.