AI Tools and Privilege: What Your Infrastructure Must Show

Four federal opinions handed down between January and March 2026 have created something that did not exist six months ago: a live circuit split on whether using a commercial AI tool to process client materials constitutes a voluntary disclosure sufficient to waive work product protection. For managing partners, this is not a law review hypothetical. It is the kind of question that belongs in your next malpractice renewal conversation — and increasingly, it already does.

The split is stark. Two decisions, one from the Northern District of Illinois and one from the Southern District of New York, applied a strict third-party disclosure analysis: if a firm submitted privileged materials to an external LLM API under standard commercial terms, the court treated that submission no differently than handing documents to an outside consultant without a robust confidentiality structure. Privilege waived. Two other decisions — from the Northern District of California and the District of Massachusetts — took a more permissive view, holding that confidentiality provisions in enterprise AI contracts are functionally analogous to discovery vendor agreements, which courts have long found sufficient to preserve protection. Privilege survived, but only because the firms in those cases could prove the contractual terms and show logs of what data moved.

That last clause is the operative one. In every 2026 decision that went the firm's way, the outcome turned on documentary evidence of the data architecture — not on the vendor's marketing page, not on a terms-of-service printout retrieved the morning of the hearing, but on contemporaneous, auditable records of what left the firm's infrastructure, under what terms, and what did not. The AI tool's data architecture has become a privilege argument. And most firms cannot make it.

Why the Circuit Split Happened Now

The doctrinal foundation was always fragile. Work product protection under Rule 26(b)(3) survives disclosure only when disclosure is made to someone with a common legal interest or under circumstances that do not substantially increase the risk of opponent access. Courts have wrestled for decades with how that standard applies to e-discovery vendors, cloud storage providers, and litigation support platforms. The general answer — that sufficiently protective contracts preserve privilege — created a workable if imprecise framework.

Generative AI broke the framework in two ways.

First, the nature of the transmission changed. When a firm uploads documents to a Relativity workspace or an e-discovery vendor's SFTP server, the documents are stored and processed but not ingested into a statistical model. The vendor's infrastructure handles the files; no external system is trained on or shaped by the content. With most commercial LLM APIs, that distinction is murkier. Training data exclusions vary by contract tier, by provider, and by the specific API endpoint used. Firms that deployed GPT-4 through the standard OpenAI API — not the enterprise agreement — operated under terms that, at various points, permitted input use for model improvement. The Illinois court focused heavily on this point.

Second, the query itself carries privilege. In a traditional e-discovery workflow, the firm controls the work product: it is the attorney's selection and ranking of documents, the litigation strategy, the mental impressions. In an agentic AI workflow — where the AI is drafting discovery responses, synthesizing deposition prep, or generating privilege logs — the query, the context window, the retrieved chunks, and the model's response collectively reflect attorney work product. Courts are beginning to recognize that the submission is not just 'documents'; it is the attorney's analytical process externalized into a prompt. That is a harder case for the 'it's just like an e-discovery vendor' analogy.

The California and Massachusetts courts were persuaded by firms that could demonstrate architectural separation: full documents and agent-layer reasoning stayed inside the firm's controlled environment; only narrow, de-identified or minimized retrieval chunks were ever sent to the external model. The Illinois and SDNY courts were presented with firms that could show almost nothing — they used a SaaS tool, they clicked 'agree' on the terms of service, and they had no logs.

The Malpractice Carrier Is Already Asking

Insurance markets move faster than doctrine. Three of the five largest legal professional liability carriers have added AI-specific questionnaires to their 2026 renewal applications, according to broker disclosures reviewed in Q2 2026. The questions are not abstract:

• Which AI tools does the firm use to process client documents?
• What data may be transmitted to third-party model providers?
• Does the firm have a written AI data governance policy?
• Can the firm produce logs showing what client data was submitted to external systems?

A 'no' or 'we're not sure' answer to any of those questions is now a pricing variable. One managing partner at a 200-lawyer firm described a renewal conversation in which the carrier's underwriter specifically referenced the Q1 2026 decisions and asked whether the firm had reviewed its AI vendor contracts for training data provisions. The firm had not. The premium increased 18 percent.

That is the context in which the infrastructure question stops being an IT conversation and starts being a governance conversation.

What the Surviving Firms Had in Common

The two decisions that preserved privilege — California and Massachusetts — involved firms with meaningfully different AI tooling, but identical governance postures. Both could produce:

Neither firm was running on-premise infrastructure exclusively. Both used cloud-based components. But both had designed their systems so that the full corpus of client documents, the agent-layer reasoning, the vector index, and the workflow logic all resided within the firm's controlled environment — and only the minimal retrieved context necessary to answer a specific query was ever sent to an external model, under an enterprise agreement with a documented training exclusion.

That architectural distinction — full corpus plus agent layer under client control, minimal chunks to the model under explicit contractual terms — is the difference the courts found dispositive.

The Architectural Question Your Infrastructure Needs to Answer

Most commercial legal AI platforms were not designed with this evidentiary standard in mind. Tools like Harvey, CoCounsel, and Lexis+ Protege are sophisticated products with genuine enterprise-grade security commitments. The honest assessment is not that they 'send all your data out' — their enterprise agreements are more nuanced than that, and firms using them under proper enterprise terms have contractual protections worth something.

But the architecture of those platforms is not designed to give the firm architectural sovereignty over what the privileged question is about to leave the perimeter. The agent scaffolding, the retrieval logic, the vector index, the workflow layer — in a SaaS deployment, those run on the vendor's infrastructure. The firm cannot produce an independent log of what chunks were retrieved and sent to the model, because that process happens inside a system the firm does not operate or audit.

That matters for two distinct reasons:

1. You cannot log what you do not control. The California and Massachusetts firms survived privilege challenges in part because they could produce granular, timestamped, internally generated logs. A firm relying on a SaaS vendor's logging — assuming the vendor provides it at all — is in a weaker evidentiary position. Vendor logs are third-party records. Your own infrastructure logs are first-party evidence.

2. The enterprise agreement is necessary but not sufficient. Every sophisticated legal AI vendor will tell you their enterprise terms prohibit training on your data. That is true and it matters. But the Q1 2026 cases suggest courts are looking beyond the contractual representation to the architectural reality: did the firm actually have control over what it was submitting, or was it trusting a black-box system to make that determination? Contractual intent without architectural evidence is a thinner privilege argument than both together.

This is precisely the problem that a private AI deployment model is designed to solve — not by eliminating LLM API calls entirely, but by keeping the firm in architectural control of everything except the minimal inference step.

What 'Architectural Control' Actually Means in Practice

The phrase gets used loosely. Here is what it means in operational terms for a firm that needs to defend a privilege challenge:

The full document corpus stays inside the firm's infrastructure. Whether that is on-premise servers or a firm-controlled cloud tenancy, the complete client documents — the merger agreement, the privileged memo, the deposition transcript — never leave. They are indexed, chunked, embedded, and stored in a vector database that the firm operates.

The agentic scaffolding runs inside the perimeter. When an attorney asks a question, the retrieval logic — what to look for, how to rank results, what context to assemble — runs inside the firm's environment. The model is not reasoning over the full corpus; it is reasoning over the narrow slice that the firm's own retrieval system has selected and returned.

Only minimized chunks go to the LLM, under the firm's chosen API terms. This is the step that involves external infrastructure. A small number of retrieved passages — the minimum necessary to answer the specific query — are sent to the language model. The firm chooses which model, negotiates the contract terms, and logs the transmission independently.

Every step is logged and auditable by the firm. Not by the vendor. By the firm. What query was run, against which matter files, at what time, which chunks were retrieved, what was sent to the model, under which API agreement, and what was returned. That log is the evidentiary foundation of the privilege defense.

For firms researching this architecture in the context of both AI governance and core legal work, the AI for law firms guide and the firm's case search capabilities both operate under this model — retrieval and indexing fully within the firm's controlled environment, model inference on minimized context under firm-selected API terms.

The Emerging Compliance Checklist

Based on the Q1 2026 decisions and the malpractice carrier questionnaires now circulating, firms should be able to answer 'yes' to each of the following before the next renewal:

• Does the firm have a written AI data governance policy that specifies which tools may be used with privileged client materials and under what conditions?
• Can the firm identify, for any AI-assisted work product, which specific data was transmitted to an external model provider, when, and under what contractual terms?
• Does the firm's AI infrastructure produce first-party logs — not vendor-supplied records — of data transmissions involving client materials?
• Has the firm reviewed its AI vendor contracts for training data provisions, data processing addenda, and the specific API endpoints authorized for use with privileged materials?
• Does the firm's architecture separate the retrieval/index layer from the inference layer, so that full documents are never submitted to an external model even when an attorney queries against a matter file?

A 'no' to any of these is not necessarily disqualifying — the law is unsettled and courts are not applying uniform standards. But it is a gap that an opposing counsel with a privilege challenge will exploit, and it is a question a malpractice underwriter is already asking.

What Comes Next

The circuit split will not resolve quickly. A circuit court opinion — rather than district-level decisions — is probably 18 to 24 months away. In the interim, the risk is asymmetric: firms with strong architectural controls lose nothing if the permissive standard prevails; firms with weak controls face meaningful exposure if the strict standard takes hold in their jurisdiction.

The more immediate pressure will come from clients, not courts. General counsel at sophisticated companies are beginning to ask their outside firms about AI data governance as a matter of vendor due diligence — the same way they ask about cybersecurity controls. A managing partner at a firm without a coherent answer to 'what does your AI system do with our documents?' is in a structurally weaker position in that conversation than one who can describe an architecture in which the full document corpus never leaves a controlled environment.

The firms that move first on infrastructure governance will have two advantages: a defensible record if privilege is challenged, and a credible answer when the client asks. Neither of those advantages requires waiting for circuit court clarity.

If you are evaluating AI infrastructure ahead of your next malpractice renewal or a client due diligence inquiry, the architectural questions in this piece are a reasonable starting framework. The core issue is not which AI tool you use — it is whether your infrastructure can produce a contemporaneous, auditable record of what your AI system did with privileged materials, and whether that record shows that full client documents stayed under your control. Those are engineering decisions as much as legal ones, and they are worth reviewing with both your IT leadership and outside counsel before the question arrives from an underwriter or an opposing brief.