Most Private AI for Lawyers: What Qualifies as Private

The most private AI for lawyers is an AI tool that retains no prompt and trains on no client work. Two configurations reach that bar: a model the firm runs on its own hardware, and a managed AI service under a contracted zero-data-retention arrangement. Neither is the default setting at OpenAI, Azure OpenAI or Anthropic.

The practical question for most law firms is not which AI is most private in theory but which private configuration the firm can verify, which of the five options compared below actually reaches it, and how to get the commitment in writing.

Key takeaways

1. The most private AI for lawyers is a locally run model or a managed service under contracted zero data retention.
2. The default at OpenAI, Azure OpenAI and Anthropic is retention, not deletion.
3. Zero data retention is scoped to certain endpoints and does not reach a firm’s own logs.
4. Verification is a setting a customer can read, not an assurance a vendor gives.

[cta-1]

AI data retention defaults at the major providers

The AI data retention default is retention. OpenAI, Azure OpenAI and Anthropic, the three providers behind most legal AI tools, all log prompts for abuse monitoring out of the box, and the zero-retention configuration is an option a customer has to apply for rather than a setting that ships switched on. The terms below govern the application programming interface, or API, the integration surface behind most legal AI products, not the consumer chat product bearing the same name.

Abuse-monitoring logs may contain prompts and responses, which is a different question from whether that content trains a model, and OpenAI makes the two commitments separately. The first row also governs the API rather than the consumer product, so a firm that has approved API access through a vendor has not thereby approved the behavior of the consumer version of ChatGPT on a lawyer’s own laptop.

Azure OpenAI answers the same abuse-monitoring question differently from OpenAI, and the practical difference that matters most to a legal buyer is who actually reviews the sampled content, not whether a sample is taken at all. Microsoft’s abuse-monitoring documentation states that a sample of prompts and completions may be selected for review. That review is conducted by automated means, including large language models, by default, and human reviewers look at material only as necessary. Human review is therefore the escalation rather than the default, and Microsoft describes it as running through Secure Access Workstations under just-in-time approval.

Anthropic sets retention by a control rather than by a tier, and the control has a floor rather than a zero. The minimum period an administrator can set is 30 days, and projects are retained indefinitely by default, which Anthropic states in its custom data retention documentation. A firm that sets 30 days and keeps its sensitive work inside projects has not shortened the retention of that work, because the chat retention period does not delete chats held inside a project.

What zero data retention does not cover

Zero data retention covers the provider’s abuse-monitoring logs and model training, and it does not cover everything the tool touches. OpenAI’s data controls documentation lists endpoints that are ineligible for zero data retention, among them conversations, assistants, threads, vector stores and files, and those endpoints may still store application state while zero data retention is enabled on the account. Data the tool sends on to a third-party service follows that service’s retention policy. The firm’s own application logs sit outside the commitment entirely.

OpenAI’s own data controls guide marks the ineligible surfaces explicitly:

1. Conversations, assistants and threads
2. Files
3. Vector stores
4. Batch jobs
5. Agents

The consequence is concrete. A feature that retrieves from a firm’s precedent set has to put that precedent somewhere, and a vector store is the usual answer, so the document is retained even under a zero-retention arrangement. The question for a vendor is which endpoints its product calls and which of those the commitment reaches.

The second gap opens whenever the tool calls something else. OpenAI states that data sent to a third-party server is subject to that server’s retention policies, so a research or browsing step inherits whatever the downstream service keeps. A tool that reaches out to the live web, which is the design of a search-first assistant such as Perplexity, makes that call on the firm’s behalf every time it answers.

A narrower gap survives even a signed arrangement. Anthropic states that zero data retention arrangements are subject to its approval and apply to eligible Anthropic APIs and products using the commercial organization API key, and that it still retains User Safety classifier results in order to enforce its Usage Policy, which its documentation on those arrangements sets out. Classifier results are not prompts, and a firm that has told a client it sends nothing retainable should know that something is retained.

The last gap is the firm’s own. Prompt text passing through a firm’s gateway, logging platform or document management system is governed by the firm’s retention schedule, and no provider commitment reaches it.

Private AI options for legal teams, compared

OpenLLM

A firm that already operates its own infrastructure can serve open-weights models behind an endpoint it controls with OpenLLM, the BentoML project that runs models such as Llama, Qwen, Phi and DeepSeek through one OpenAI-compatible interface. The repository describes it as a way to run those models as an OpenAI-compatible endpoint, built for Docker, Kubernetes and BentoCloud. That deployment target is the part a legal buyer reads carefully, because the same tool can run on a server in the firm or on rented cloud capacity, and only the first keeps prompts on hardware the firm controls.

Pros

1. No per-seat vendor agreement and no provider retention term to negotiate.
2. Model and version choice stays with the firm, so a legacy or on-premises requirement can be met directly.
3. Open-weights licensing, so running it carries no recurring per-token billing beyond infrastructure cost.

Cons

1. Model serving, patching, access control and breach response stay with the firm’s own platform team.
2. An engineering commitment rather than a legal tool: no redlining, drafting or playbook workflow built in.
3. No vendor retention commitment to point to in a client security questionnaire, because there is no vendor.

Jan

For a sole practitioner who wants a model on one machine with no account and no external call, the shortest path is Jan, an open-source desktop application that downloads and runs open-weights models directly. A document opened in it does not reach an API at all, which removes the retention question rather than answering it.

Pros

1. Open source, so the behavior can be inspected rather than taken on trust.
2. No subscription or usage-based billing, since there is no vendor relationship to bill against.
3. Works with a range of open-weights models, so a firm is not locked to one.

Cons

1. One machine sets the ceiling on model size and on speed.
2. A per-device installation, so each lawyer’s copy is updated and governed separately rather than through one shared workspace.
3. Device-level security becomes the whole control, including disk encryption and physical access.

Ollama

Technical teams that want a local runtime with a path to scale later tend to reach for Ollama, a runtime for open-weights models with broad community and tooling support. The vendor states that prompts run locally are never stored or trained on. The distinction a firm has to hold onto is that the same product offers a cloud option, so the privacy posture depends on which of the two a team is actually running on any given day.

Pros

1. Active open-source community, so models and documentation update quickly.
2. No per-seat vendor agreement, so cost scales with infrastructure rather than headcount.
3. Runs the same interface on a laptop or a shared server, so a pilot can scale without switching tools.

Cons

1. The cloud option re-introduces a provider into the chain, which is easy to overlook under either name.
2. A general model runtime rather than a legal workflow tool, so contract review stays manual.
3. Answer quality depends on which open-weights model a team selects and keeps updated.

Harvey

Among managed legal platforms, Harvey publishes one of the wider certification surfaces in the category, spanning information-security audits, regional attestations and AI-specific standards. Its security page states that Harvey requires zero data retention from its model providers and does not use customer inputs, outputs or uploaded documents to train underlying models, and its public subprocessor list names those providers, among them Microsoft, OpenAI, Google Cloud Platform, Amazon Web Services and Anthropic. The underlying security and privacy reports also sit behind a gated access request rather than published outright, so the specifics are confirmed in direct diligence rather than read off the site.

Pros

1. States a contractual zero-retention requirement on its model providers, per its published security page.
2. SOC 2 Type 2, ISO/IEC 27001, ISO/IEC 27701 and ISO/IEC 42001:2023 certifications.
3. GDPR, CCPA, IRAP and EU AI Act entries in one trust center.
4. Also lists the AIUC-1 AI-governance certification, audited by Schellman, alongside TX-RAMP, VPAT and WCAG entries.
5. Managed platform, so there is no model infrastructure for the firm to operate.

Cons

1. The zero-retention requirement has to hold across the ten model and infrastructure subprocessors its list names, so the diligence question is the whole chain rather than one provider.
2. The underlying security and privacy reports sit behind a gated access request, so confirming the scope and dates of a specific certification is a diligence step rather than a published fact.
3. Anthropic and Mistral models are listed for a narrower set of processing locations than the other providers, so model choice and data residency interact.

Spellbook

Commercial practices evaluating a managed option alongside the three local ones above can look at Spellbook, built specifically for contract work rather than as a general-purpose assistant. The fit is deliberately narrow: redlining and drafting commercial agreements in the editor where that work already happens, rather than a tool a practice adopts alongside its existing stack and then has to govern separately. For a firm whose confidentiality exposure sits in third-party paper, that narrowness keeps the retention question limited to one vendor relationship rather than several.

Pros

1. Managed platform, so there is no model infrastructure for the firm to operate.
2. Built for commercial-contract work, with redlining and drafting workflows already in place rather than a general-purpose chat interface.
3. Works in Word and Google Docs, so the document stays in the editor the lawyer already uses.

Cons

1. The retention commitment covers the provider leg, not the firm’s own tenant and gateway logs.
2. Custom Playbooks built with Customer Success take one to two weeks before firm standards are enforced.
3. A single-vendor relationship, so renewing or renegotiating one agreement is the whole lever rather than infrastructure the firm owns outright.

[cta-2]

Where AI prompts are processed and stored

AI prompts are processed and stored in two different places, and where a prompt is processed and where it is stored are two separate commitments a vendor makes. A global deployment may be processed in any geography where the relevant model is deployed; a data-zone deployment confines processing to the named zone; and under either one, data at rest sits in the geography the customer designated, which Microsoft sets out in its Azure OpenAI data-privacy documentation. A term that pins storage to a region says nothing about the geography where inference ran.

The gap is not theoretical for a commercial practice. A client whose own agreements carry a data-residency covenant may flow it down to counsel, and a covenant written about storage is not satisfied by a storage-only answer. Ask a vendor which deployment type it runs and in which zone, rather than whether data is held in a given country.

How to verify zero data retention

To verify zero data retention, check the setting rather than the claim. OpenAI and Azure OpenAI expose their retention state somewhere a customer can read directly; Anthropic’s is an administrative control an owner sets rather than a flag a customer reads back.

1. Read the Azure capability flag. The Azure data-privacy documentation cited above states that a value called `ContentLogging` appears in the capabilities list and is set to false when logging for abuse monitoring is off, and that the attribute is absent when storage is on. It reads from the portal or the command line, which makes it a retention claim a customer can confirm without asking anyone.
2. Open the OpenAI Data Retention tab. Approved organizations configure retention under Settings, then Organization, then Data controls, then Data Retention, and the setting applies per organization or per project. An organization-level yes is therefore not a project-level yes, and the project a vendor runs the firm’s work in is the one that matters.
3. Ask for the subprocessor list in writing. The commitment a vendor made to its model provider says nothing about the other services in its stack, and a subprocessor list is the only document that enumerates them.
4. Ask which endpoints the commitment covers, by name. A vendor that answers with a product name rather than an endpoint list has not answered the question.
5. Re-verify after any vendor change. A configuration confirmed at signature can be changed at a model migration, so the check belongs in the renewal cycle rather than in onboarding alone.

Consumer tiers are a separate exercise, because the retention control a company negotiated for its API traffic does not reach the chat window a lawyer opens in a browser. Retention in Google’s consumer Gemini tier is set by Google account controls rather than by an enterprise agreement, which means the setting travels with the individual lawyer rather than with the firm, and a firm that has approved an enterprise deployment has approved nothing about the browser tab beside it. Microsoft documents its consumer assistant separately from Azure OpenAI, so Copilot’s retention behavior for legal work does not follow from the Azure terms above.

The retention terms to get in writing

The retention terms that matter belong in the AI vendor contract rather than in the vendor’s documentation, because documentation is revised without notice and a contract is not. A scope term states which endpoints and product features the retention commitment covers. A subprocessor term names the downstream providers and requires notice before the vendor adds another. A training term bars use of customer content to train or evaluate models, and an exit term fixes what happens to any retained data at termination.

Each of the following needs a named owner on the firm side rather than a line in a questionnaire:

1. Scope. Name the endpoints and product features the retention commitment covers, and say what happens to the surfaces it does not cover.
2. Subprocessors. List the downstream providers by name, require written notice before an addition, and state the firm’s remedy if it objects.
3. Training and evaluation. Bar use of customer content to train, fine-tune or evaluate models, for the vendor’s own quality review as well as for the model provider’s.
4. Exit and deletion. Fix the deletion window at termination, name the artifacts covered, including logs and derived indexes, and require written confirmation that it happened.
5. Change control. Require notice before the vendor changes the retention posture or its model provider, because a configuration a firm verified once can be changed without the contract being touched.

A questionnaire answer is a representation at a point in time and a contract term is enforceable for the life of the engagement, which is the reason to move these out of diligence and into the agreement.

Legal workflows that privacy-first AI supports

Privacy-first AI earns its place in legal workflows where the document cannot leave the firm’s control: drafting a sensitive non-disclosure agreement, reviewing non-public acquisition documents, redlining under an express confidentiality obligation, reusing firm precedent, and running internal compliance checks. Each of those legal workflows asks something different of the retention posture, and the difference is usually the endpoint the tool calls rather than the document itself.

An agreement drafted in the editor and the same agreement uploaded for analysis carry one confidentiality obligation and two different retention exposures, because the second one touched a file endpoint. Matching the workflow to the endpoint is the useful diligence question, and a practice can answer it about its own tools without waiting for a vendor.

How to choose and deploy a private AI for your legal team

Choosing a private AI for a legal team is a procurement exercise before it is a technology one: the retention posture is settled in the agreement, and everything after that is rollout. The diligence questions come first, then a pilot that tests quality rather than privacy alone, then the written policy that governs what may be entered.

Sequence matters more than tool count. A firm comparing the legal AI tools on the market on feature lists reaches a shortlist it then has to re-diligence on retention, while a firm starting from the retention requirement arrives at a shorter list needing no second pass. Name the person who signs off each stage before the first demo, so no stage is skipped under deal pressure.

How to pilot and test a private AI tool

Pilot a private AI tool on the firm’s own precedent and expired templates, not on live client files, until the retention configuration has been confirmed in writing. Run the output beside a human reviewer on the same documents and record where the tool was wrong, how long each task took, and whether it held up on third-party paper rather than the firm’s own forms.

A pilot easily leaves out two measurements: time to a usable answer, measured by the lawyer who would do the work, and error type, because a tool that misses an unusual indemnity is a different risk from one that invents a clause reference, and a pilot recording only a count cannot tell the two apart.

Writing an AI governance policy for a law firm

An AI governance policy for a law firm has to answer four questions in writing: which tools are approved, what categories of client information may be entered into them, who is accountable for reviewing the output before it leaves the firm, and what happens when a lawyer enters something that should not have gone in. Name the owner of each answer rather than the committee.

In practice the security function owns vendor selection and renewal, legal leadership approves the scope of use, and the reviewing lawyer is accountable for anything that leaves the firm. A fourth answer belongs in the policy and is the one most often left out: a disclosure route a lawyer will actually use, with no disciplinary framing attached to self-reporting, so reporting stays low-friction rather than something to avoid. A tool comparison like this one stops at the vendor’s retention terms, while data privacy for law firms turns on the approvals, access controls and breach reporting a firm runs itself.

How Spellbook handles client data

Spellbook handles client data under negotiated zero data retention agreements with both OpenAI and Anthropic, which its security page states keep customer data in requests and responses out of persistent storage and in memory only for the duration of the request. The platform is SOC 2 Type II compliant, HIPAA compliant and assessed as low-risk under the EU AI Act, and it runs as a Microsoft Word add-in that authenticates through Microsoft accounts, so a firm can enforce multi-factor authentication inside its own Microsoft Entra tenant.

The retention point and the product design are connected. Spellbook’s Review feature surfaces risks and proposes redlines as native Word tracked changes, and the Draft feature returns language into the same document, so the analysis runs against the file the lawyer already has open and no separate repository is created for the firm to govern. Infrastructure runs on Amazon Web Services with data centers in Canada and the United States, and the Trust Center carries the subprocessor list the diligence sequence above asks for, all of it recorded in the security documentation.

The honest limitation is twofold. Spellbook is a commercial-contract specialist rather than a legal-research or litigation platform, so a practice whose AI requirement is case law research is evaluating the wrong category. And as a managed service it keeps a model provider in the trust chain, which is the trade against running a model on the firm’s own hardware: a contractual control the firm can verify on paper, in place of a technical control it has to operate itself. A firm that wants no provider in the chain should be running a local model and accepting the operational burden.

For a commercial practice that would rather hold a retention commitment it can read than operate a model it has to patch, the managed route is the shorter path. Try Spellbook free with a seven-day trial, with no sales call and no implementation project.

[cta-3]

Frequently asked questions

What makes an AI tool private enough for legal work?

An AI tool is private enough for legal work when the retention commitment is written into the agreement, covers every endpoint the tool calls, bars training on customer content, and can be verified from the provider’s own configuration surface. Encryption and a SOC 2 report are necessary and are not sufficient on their own, because both can be true of a service that still logs prompts.

Are self-hosted AI models more private than managed AI with zero data retention?

Self-hosted AI models are more private in the narrow sense that no prompt reaches a third party, and managed AI with zero data retention is often more private in practice, because the firm running its own model also inherits the patching, the access controls and the breach response. The honest comparison is between a control a firm can verify on paper and a control it has to operate itself.

Does OpenAI have zero data retention?

OpenAI offers zero data retention on the API, but not by default and not on every endpoint. Abuse-monitoring logs are generated for all API usage and retained for up to 30 days unless the organization is approved for zero data retention or modified abuse monitoring, and that approval is subject to prior approval by OpenAI and acceptance of additional requirements.