Data Sovereignty in Legal AI: What AmLaw Firms Risk With Public Tools

Three managing partners walk into a board meeting. One has deployed a leading SaaS legal AI platform firm-wide. One has quietly signed an enterprise agreement with a hyperscaler for a private cloud instance. The third is still running Word macros. By the end of 2025, two of those three will face a client data inquiry they didn't anticipate. The question is which two.

The legal AI market crossed $1.2 billion in annual revenue in 2024 and is projected to reach $4.1 billion by 2028, according to Bloomberg Law's technology benchmarking data. Every major vendor—Harvey, CoCounsel, Lexis+ Protege, Legora, and now Anthropic's Claude-powered Cowork—is racing to capture firm spend before the architecture decisions calcify. And those architecture decisions, made under competitive pressure and vendor sales cycles, will determine which firms control their data and which ones discover, too late, that they don't.

This is not a polemic against any specific vendor. Harvey's contract analysis is genuinely impressive. CoCounsel's integration with Clio solves real workflow problems. Lexis+ Protege's citator-aware responses represent a meaningful step forward for research tools. The question is not whether these tools work. The question is what happens to your clients' documents, deal data, and privileged communications inside the architectural choices those tools require—and whether your firm's leadership has asked that question with enough precision to actually answer it.

The Architectural Question Most Firms Are Asking Wrong

The framing that dominates vendor pitches and bar association AI guidance alike is binary: data leaves your firm, or it doesn't. This framing is wrong, and it's leading firms to make decisions based on a false choice.

The more precise question is: which components of your AI stack leave firm-controlled infrastructure, under whose contractual terms, and at what granularity?

Every serious AI system—whether SaaS or on-premise—ultimately sends something to a large language model to generate a response. LLMs don't run on laptops. The relevant distinction is not whether data travels across a network. It's what data travels, at what stage of the pipeline, under what legal agreement, and who controls the surrounding infrastructure.

Here's how that breaks down across deployment architectures:

The critical insight: what reaches the LLM in a well-architected private deployment is a narrow slice—the retrieved passage or document excerpt most relevant to the query. Not the entire corpus. Not the client's deal folder. Not the underlying index. The full retrieval apparatus, the permission model, the audit trail, and the complete document store stay on infrastructure the firm controls.

This is fundamentally different from a SaaS architecture where your client's M&A documents are ingested into, indexed on, and queried against vendor-operated servers—even when the vendor's terms prohibit training on that data. The data is there. That's the exposure.

What "Data Stays With Us" Actually Means in SaaS Vendor Agreements

Reading a legal AI vendor's data processing agreement carefully is an exercise in discovering how much work the word "use" is doing.

Common SaaS AI terms distinguish between using data to train the model (usually prohibited for enterprise tiers) and using data for operational purposes—which can include debugging, quality assurance, safety monitoring, and abuse detection. Those operational carve-outs mean your clients' documents may be reviewed by vendor employees or contractors under circumstances that don't trigger the training prohibition.

Beyond the contractual text, there's the infrastructure reality. When a firm uploads 40,000 pages of due diligence documents to a SaaS legal AI platform:

• Those documents are parsed, chunked, and embedded on vendor servers
• The resulting vector index—a mathematical representation of the document's content—lives on vendor infrastructure indefinitely until deleted
• Every query against those documents generates retrieval logs that capture which document chunks were surfaced and when
• A breach, regulatory subpoena, or government request targeting the vendor reaches all of that—including documents from deals your client considers strictly confidential

The 2023 Samsung incident, where engineers inadvertently exposed proprietary source code through ChatGPT queries, is frequently cited. Less discussed but more directly relevant to law firms: any legal AI SaaS vendor is itself a high-value target for adversarial actors precisely because it aggregates sensitive documents from hundreds of firms. The concentration risk is not hypothetical.

For a longer analysis of how privilege intersects with AI tool architecture, the Heppner privilege analysis covers the emerging case law in detail.

The Sovereignty-Critical Workload Framework

Not every legal AI workload carries the same data sensitivity. A partner researching public case law on a novel statutory interpretation question has different exposure concerns than a team running AI-assisted privilege review on a DOJ investigation response.

Firms that are thinking clearly about this are starting to segment their AI workloads by sovereignty requirements:

Tier 1: Public or Low-Sensitivity Workloads

• Legal research against published case law
• Drafting from publicly available precedent
• General market intelligence and news summarization
• Internal knowledge management on non-client content

Appropriate for: SaaS tools, with normal vendor due diligence

Tier 2: Standard Client Matter Work

• Contract review and redlining on standard commercial matters
• Routine due diligence document review
• Client-specific research integrating matter files
• Standard litigation support and discovery categorization

Appropriate for: SaaS enterprise tiers with negotiated DPAs, or private deployment; depends on client sensitivity and industry

Tier 3: Sovereignty-Critical Workloads

• Regulatory investigations and government enforcement matters
• M&A due diligence for targets in sensitive industries (defense, health data, critical infrastructure)
• Trade secret litigation and IP disputes
• Multi-jurisdictional matters with EU data residency requirements
• Matters involving government clients with FedRAMP or ITAR constraints
• Privilege review at scale

Appropriate for: Private AI deployment with on-premise or private cloud infrastructure where the firm controls the full stack

Most AmLaw 200 firms doing an honest audit will find that Tier 3 workloads represent 20-35% of their matter volume by revenue—disproportionately concentrated in their highest-billing practice groups. The practices where AI would deliver the most leverage are precisely the practices where SaaS architecture creates the most exposure.

Why the Agentic Layer Is the New Battleground

The legal AI conversation in 2023 was largely about retrieval: which tool could find the right case, the right clause, the right precedent. The 2025 conversation is about agents—AI systems that don't just retrieve information but take sequences of actions, make decisions across multiple steps, and interact with firm systems over time.

This shift matters enormously for data sovereignty, and it's underappreciated in most firm-level AI governance discussions.

A retrieval-only system ingests documents and answers queries. An agentic system might:

• Pull a client's matter history from the DMS
• Cross-reference prior deal terms from a separate transaction database
• Draft a comparison memo
• Route it to the appropriate partner based on matter-specific rules
• Log the entire workflow for billing and compliance purposes

Each step in that chain involves data. In a SaaS agentic architecture, the orchestration layer—the system deciding what to retrieve, how to combine it, and what action to take next—runs on vendor infrastructure. That means the vendor's systems are making decisions about your client data, sequencing access to your matter files, and logging that activity.

In a private deployment, the agent orchestration layer runs on firm infrastructure. The firm controls what connectors are active, what data sources the agent can access, what gets logged, and what minimal context is ultimately sent to the LLM for inference. The intelligence of the agent—its decision-making, its retrieval logic, its workflow rules—stays home.

For a deeper look at how agentic systems are reshaping law firm economics and workflows, see the agentic AI for law firms analysis.

The Regulatory Pressure That's About to Make This Urgent

Data sovereignty in legal AI is transitioning from a best-practice conversation to a compliance-driven imperative. Several regulatory vectors are converging:

EU AI Act (effective August 2026 for high-risk systems): Legal AI used in consequential decisions—document review affecting litigation strategy, contract analysis affecting deal terms—may qualify as high-risk under Annex III. High-risk systems require detailed logging, human oversight mechanisms, and data governance documentation. Firms running SaaS tools will need to obtain that documentation from vendors; firms running private deployments own it.

State Bar AI guidance: The ABA's Formal Opinion 512 (2024) and parallel state bar guidance increasingly emphasize that competent supervision of AI requires understanding what the tool does with client data. Several state bars have explicitly flagged that uploading client documents to third-party AI systems without client consent may implicate Rule 1.6. The threshold for what requires consent is still being defined, but the direction is clear.

Client contract requirements: An accelerating number of sophisticated clients—financial institutions, pharmaceutical companies, government contractors—are inserting AI-specific provisions into outside counsel guidelines. These provisions often require firms to disclose which AI tools have access to matter data and to represent that client confidential information is not processed on third-party infrastructure without pre-approval. Firms that have deployed SaaS legal AI broadly may find themselves in technical breach of client agreements they signed before auditing their AI stack.

Cross-border data flows: Matters with EU data subjects, government clients, or operations in jurisdictions with data localization requirements (Brazil's LGPD, India's DPDP Act, China's PIPL) create specific infrastructure requirements that standard SaaS deployments may not satisfy without dedicated regional instances—at meaningfully higher cost.

The Honest Comparison: SaaS Convenience vs. Architectural Control

The case for SaaS legal AI is real and shouldn't be dismissed. Faster deployment, lower upfront infrastructure cost, vendor-managed updates, and broad feature development funded by large user bases are genuine advantages. For firms where Tier 3 workloads are a small fraction of revenue, or where a specific tool's capabilities are uniquely strong, a SaaS enterprise agreement with a well-negotiated DPA may be the right call.

But the cost comparison is rarely done completely. Firms evaluating legal AI total cost need to account for:

• Per-seat SaaS fees at scale (most leading platforms run $80-200+ per user per month at enterprise tiers)
• Data egress and storage costs as matter volume grows
• Audit and compliance overhead to document vendor data handling for client requests
• Workload segmentation costs—if sovereign workloads can't go through the SaaS tool, firms end up maintaining parallel workflows anyway
• Renegotiation leverage loss once the firm's entire workflow is built on a single vendor's proprietary orchestration layer

Private deployment has real costs too: infrastructure investment, IT operational overhead, and the need to select and integrate LLM API providers rather than relying on a vendor's pre-built model access. The honest answer is that private deployment is not always cheaper in year one. It is almost always more defensible in year three, when a client requests an audit of how their documents were handled, or when a vendor changes its terms, or when a matter's sensitivity escalates mid-stream.

For practice groups where AI is becoming central to delivery—case search, large-scale document review, multi-matter synthesis—the architectural decision is effectively a long-term infrastructure decision. Treating it as a software subscription choice is a category error.

What a Well-Architected Private Deployment Actually Looks Like

For firms evaluating the private path, the architecture that balances capability with sovereignty looks roughly like this:

On firm-controlled infrastructure:

• Document ingestion pipeline and preprocessing
• Embedding models and vector stores (indexed document representations)
• Retrieval and ranking layer
• Agent orchestration and workflow engine
• Connector layer (DMS, billing system, practice management integrations)
• Permission and access control engine
• Full audit logs

What leaves firm infrastructure:

• Only the specific retrieved chunks most relevant to a given query
• Transmitted to the firm's chosen LLM provider (OpenAI, Anthropic, Google, or a self-hosted open-weight model) under the firm's negotiated API terms
• Not the full document, not the index, not the client metadata

This architecture means the firm retains control over what matters most: the complete client document corpus, the retrieval intelligence that determines what the AI sees, and the workflow layer that sequences every action. The LLM—the component that actually requires significant compute infrastructure—can be accessed via API from a provider the firm has vetted and contracted with directly, under terms the firm controls.

For firms with the highest sovereignty requirements, self-hosted open-weight models (Llama 3, Mistral, Falcon) eliminate the external API call entirely—at the cost of higher infrastructure investment and some capability trade-offs versus frontier commercial models.

The AI for law firms guide walks through the full deployment decision framework in more detail.

The firms that will navigate the next 24 months of legal AI without a data incident or a client relations problem are not the ones who deployed fastest. They're the ones who asked the architectural question precisely enough to actually answer it—and who segmented their AI strategy to match deployment model to workload sensitivity. If you're an innovation lead or CIO at an AmLaw firm currently evaluating your AI stack, the most valuable exercise you can run right now is a workload audit: map your active matters, identify the Tier 3 workloads by revenue weight, and ask honestly whether your current deployment architecture gives you defensible answers when a client, a regulator, or a court asks exactly where their documents went.