Zero Retention Isn't Data Sovereignty: The Gap in Legal AI Contracts

A managing partner at an AmLaw 100 firm recently told us his firm had reviewed 14 vendor contracts in the past year, and every single one contained some version of the sentence: "We do not retain your data." Not one of those contracts, when pressed, could produce a diagram showing where the full client document sat while the model was actually working on it.

That gap — between what the contract promises and what the architecture does — is now the single most consequential due diligence failure in legal AI procurement. Zero retention has become table stakes, repeated so often across RFP responses that it has stopped functioning as a differentiator. It answers a question about time. It says nothing about place. And for a profession built on privilege, confidentiality, and jurisdictional control, place is the question that matters.

Why "We Don't Retain Data" Has Become Meaningless

Zero retention means one specific thing: after the model finishes processing a request, the provider deletes its copy. That's it. It doesn't tell you:

• Whether the full document was uploaded in its entirety to a third-party inference endpoint before deletion
• Whether a retrieval index built from your firm's document corpus lives on vendor infrastructure, even temporarily
• Whether logs of that processing — which can themselves contain privileged content — are retained by the vendor, a subprocessor, or a fourth party
• Which jurisdiction that processing physically occurred in

A clause promising deletion after 24 hours or after the session ends is a temporal control. It says nothing about the spatial control — where the data traveled, whose infrastructure touched it, and under what legal regime that infrastructure operates. A document that transits through a data center in a jurisdiction with weak discovery protections for 90 seconds has already created exposure, regardless of what happens to the copy afterward.

This distinction matters more now than it did two years ago because legal AI has moved from single-shot summarization to agentic workflows — tools that read across dozens of documents, call external connectors, and maintain working memory across multi-step tasks. The more agentic the tool, the more surface area exists for full documents to pass through infrastructure the firm doesn't control, even briefly. Our agentic AI for law firms analysis covers this shift in more detail, but the short version: agentic scaffolding needs more data in motion, which means retention clauses alone cover a shrinking fraction of actual risk.

The Three Questions That Replace the Retention Clause

If retention terms are necessary but not sufficient, what should replace them in vendor evaluation? Three architectural questions, each with a specific, verifiable answer:

1. Where do full client documents live during processing? Not after processing. During. If the answer involves the document being uploaded whole to a vendor's cloud environment or a third-party model provider's endpoint, the retention clause is protecting you from persistence, not from exposure.

2. Where does the retrieval index sit? A retrieval-augmented system builds an index — embeddings, chunks, metadata — from your document corpus to make retrieval fast and accurate. That index is functionally a derivative of your privileged material. If it lives on vendor infrastructure, it's a standing asset outside firm control, subject to that vendor's security posture, subpoena exposure, and breach history.

3. Who holds the audit logs, and can they be pulled for privilege review? When general counsel needs to reconstruct exactly what data a system touched — for a privilege log, a malpractice inquiry, or a regulator's request — can the firm produce that record unilaterally? Or does it require a request to a vendor, a wait time, and a dependency on that vendor's own retention and access policies?

These are structural questions. They have yes/no answers backed by architecture diagrams, not legal prose subject to reinterpretation by a new general counsel at the vendor five years from now.

The Architecture Comparison That Actually Matters

Here is the breakdown that should replace "do you retain data" as the first question in every legal AI RFP:

| What reaches the LLM provider | Often the full prompt context, sometimes full documents | Only the minimal retrieved chunks needed for the specific query |

| Governing terms for LLM calls | Set by vendor's contract with the model provider | Set by the firm's own API agreement with the chosen model provider |

The critical row is the last one. In a well-architected system, something does leave firm infrastructure — that's unavoidable if you want to use a frontier model for reasoning. The question is what leaves, how much, and under whose terms. A full 40-page merger agreement leaving for summarization is a different risk profile than three retrieved paragraphs, stripped of surrounding context, sent under a data processing agreement the firm itself negotiated with the model provider.

This is the architecture behind private AI deployment: full client files, the retrieval index, permissions, and logs stay on the firm's own infrastructure. Only the minimal chunks needed to answer a specific question are sent to the selected LLM. Not "nothing leaves" — that claim doesn't survive contact with how transformer models actually work. What leaves is small, logged, scoped, and reviewable after the fact.

Reading the Current Field Through This Lens

Applying these three questions to the tools currently competing for AmLaw 200 budgets produces a more useful comparison than most vendor scorecards:

• Harvey and CoCounsel have built substantial trust with large firms on the strength of workflow design and model performance, and both have published statements on data handling and zero-retention terms with their underlying model providers. The open architectural question for any firm evaluating them is the same one that applies industry-wide: during a multi-document workflow, does the full corpus transit vendor or subprocessor infrastructure, and can the firm independently audit that path?
• Lexis+ Protege and Legora occupy similar ground — strong product experiences built on a SaaS model where the firm's documents and the vendor's infrastructure are more tightly coupled by design. That's not a flaw; it's a tradeoff firms should price explicitly against their risk appetite, the same way they'd evaluate any cloud DMS migration.
• Claude Cowork and general-purpose assistants like ChatGPT raise a distinct version of this question because they were built for broad knowledge-work use cases, not privilege-sensitive legal workflows specifically. The agentic connectors that make these tools powerful — file access, browsing, memory — are also the surface area where "where does the document actually go" gets harder to answer definitively. Firms piloting Cowork for legal workflows should read the specifics in our Claude Cowork privilege breakdown before assuming a general-purpose retention policy maps cleanly onto privilege review.

None of this is an argument that shared-cloud legal AI is unsound. It's an argument that the retention clause was never the right instrument to evaluate it. Firms with lower sovereignty requirements — smaller matters, non-privileged research, jurisdictions with less restrictive data transfer rules — may reasonably accept vendor-hosted architecture in exchange for faster deployment and lower integration overhead. The point is to make that trade consciously, with the architecture diagram in hand, not to accept it by default because the retention clause sounded reassuring.

Why This Is a Transfer Impact Assessment Problem, Not a Marketing Problem

Under GDPR and increasingly under U.S. state privacy frameworks modeled on it, a Transfer Impact Assessment requires documenting where data actually goes, not what a vendor promises to do with it once it arrives. A retention clause is a contractual commitment — revisable in the next renewal cycle, subject to reinterpretation if the vendor is acquired, and unenforceable in practice unless the firm has the leverage and legal budget to litigate a breach of contract after the fact.

An architecture where full documents never leave firm infrastructure is not a promise. It's a fact a security team can verify by looking at network traffic. It's a fact that survives a change in the vendor's legal team, a change in the vendor's ownership, or a change in the underlying model provider the vendor uses. This is precisely why data sovereignty is trending as the primary axis of legal AI evaluation, alongside — and in some RFPs, ahead of — accuracy benchmarks. Our analysis in your data is their moat covers why vendors have structural incentives to keep this question vague, and why firms need to force the specificity themselves.

For firms conducting this diligence now, the practical output should be a one-page architecture map for every AI vendor in the stack, alongside — not instead of — the contract. That map should show, for each tool: where documents sit during processing, where the index lives, who holds logs, and what specifically transits to any third-party model. If a vendor cannot produce that map on request, that is itself the answer to the diligence question.

What to Ask in the Next Vendor Review

Before the next contract renewal or RFP cycle, run this checklist against every legal AI tool currently deployed or under evaluation:

• Request an architecture diagram, not just a security addendum — show where full documents, the index, and logs physically sit at each stage of a typical workflow
• Ask whether the retrieval index is rebuilt on vendor infrastructure or the firm's own environment, and whether it persists between sessions
• Confirm whether audit logs are pullable by firm IT/GC unilaterally, or require a request to the vendor's support channel
• Ask specifically what data crosses to any third-party LLM provider — full documents, full context windows, or scoped chunks — and under whose API terms
• Map this architecture against your firm's actual jurisdictional exposure, not a generic template

Firms building this into procurement now are treating it the same way they'd treat a DMS migration or an e-discovery platform selection: as an infrastructure decision with a decade-long tail, not a feature comparison with a renewal-cycle horizon. Our broader AI for law firms guide walks through how to sequence this evaluation across practice groups with different sovereignty requirements.

The retention clause isn't wrong — it's just answering a smaller question than the one that matters. As legal AI shifts from single-document summarization toward agentic workflows that read across full matters, the architecture question — where does the corpus live, who holds the index, what specifically leaves — will only grow more consequential. Worth pulling the actual diagram from your current vendor before the next renewal, not just the clause.