Shared-Cloud Legal AI: Where Does Your Firm's Data Actually Go?

In 2021, someone in a routine model-review meeting asked a question that most legal AI vendors still haven't answered honestly in 2025: whose data made the model better, and who benefited from that improvement.

The answer, at the time, wasn't malicious. It was structural. A large model was being fine-tuned across client workloads, and the improvements flowed upward into a shared system — a system every competitor of that client could also access. Nobody in the room had designed it to work that way on purpose. It was just the default architecture of multi-tenant AI infrastructure. That distinction — bad defaults, not bad intent — is the entire reason this analysis exists.

Two years later, when legal AI became the industry's obsession, the public conversation skipped right past that question. Everyone was benchmarking chat interfaces, citation accuracy, and drafting speed. Almost nobody was asking the upstream question that actually determines risk exposure: where does retrieval happen, where do the full documents sit, and who holds the logs?

That's not a compliance footnote. For an AmLaw 200 firm, it's the entire architecture decision.

The UX Arms Race Is Hiding the Real Question

Walk into any legal AI demo in 2025 and you'll see the same pitch: faster drafting, better citations, a cleaner chat window. Vendors compete on the surface layer because it's the easiest thing to compare in a 30-minute sales call. It's also the least consequential decision a firm will make.

The consequential decision happens one layer down, in infrastructure most partners never see:

• Retrieval layer — the system that decides which documents or chunks get pulled to answer a query
• Vector store — where document embeddings live, and whether they're isolated per client or pooled
• Agent/connector layer — the scaffolding that lets an AI tool touch your DMS, email, and matter management systems
• Logs — the record of every query, every document touched, every output generated

A firm can switch its chat interface in a procurement cycle. It cannot easily unwind three years of matter data that has already flowed through a shared retrieval and logging layer. That asymmetry — easy to adopt, hard to reverse — is exactly why the architecture question needs to come before the UX question, not after it. Our AI for law firms guide breaks down this sequencing in more detail, but the short version is: evaluate the plumbing before you fall in love with the interface.

What a Law Firm's Real Asset Actually Is

Most legal AI marketing treats a law firm's data as if it were a static asset — a pile of contracts and briefs waiting to be indexed. That framing understates what's actually at risk.

A firm's real asset was never just the documents. It's the judgment layered on top of them: how a specific partner reasons through an ambiguous fact pattern, which precedent gets weighted heavily versus treated as persuasive-only, what fact combinations get flagged as litigation risk before a client even asks. That judgment is built over decades, billed at a premium, and functionally inseparable from the firm's competitive position.

When a retrieval or fine-tuning layer sits in shared infrastructure, that judgment doesn't stay contained to the matter it came from. It becomes a signal the underlying system can generalize from — improving a product that, structurally, also serves the firm across the street. Nobody has to intend that outcome for it to be the architecture's natural behavior. This is the same dynamic explored in Your Data Is Their Moat: in most SaaS AI models, the customer's proprietary usage is quietly what makes the vendor's product defensible against the next customer.

This is why the founding question for any serious legal AI evaluation shouldn't be "how fast can it draft." It should be: does the firm's thinking stay the firm's?

Where Does Your Data Actually Live? A Structural Comparison

The honest way to evaluate any legal AI product — shared-cloud SaaS, per-seat licenses, or private deployment — is to map exactly where each layer of the stack sits, not to take a vendor's security-page language at face value.

The important nuance here — and one that gets flattened in most competitive marketing — is that nobody avoids LLM providers entirely. RAGbase Legal, like Harvey, CoCounsel, Legora, Lexis+ Protégé, and Claude-based tools like Cowork, ultimately calls a foundation model to generate language. The difference isn't "we never touch OpenAI or Anthropic and they do." The difference is what leaves the building and under whose terms.

In a shared-cloud architecture, the retrieval and reasoning layer often lives with the vendor, meaning full documents, embeddings, and logs sit inside their infrastructure before a query ever reaches the LLM. In a private deployment model, the full corpus, the retrieval layer, the vector store, the connectors, and the logs stay on the firm's own infrastructure — and only the minimal retrieved chunks needed to answer a specific question get sent externally, to whichever LLM provider the firm has selected under its own negotiated API terms.

That's the distinction worth interrogating in procurement: full corpus and agent layer under client control, versus minimized chunks sent outward under the firm's chosen terms. One architecture makes data sovereignty structurally default. The other makes it a policy promise layered on top of infrastructure not designed for it.

Why Agentic AI Raises the Stakes

This question mattered when legal AI was mostly summarization and drafting. It matters considerably more now that agentic tools are being given standing permissions to act across a firm's systems — pulling from the DMS, drafting in matter folders, triggering workflows without a human clicking "send" on every step.

An agent with broad connector access isn't just retrieving a document to answer one question. It's operating with a persistent view into firm systems, which means the retrieval and permissions layer isn't a one-time data-flow decision — it's a standing architecture decision that compounds with every new integration a firm turns on. The tools explored in our coverage of agentic AI for law firms are powerful precisely because they act autonomously across systems — which is also exactly why the scaffolding underneath them deserves more scrutiny than the chat window on top.

The adoption data backs up how much is at stake here. Industry surveys have repeatedly found a gap between how many lawyers have tried generative AI tools and how many firms have actually operationalized them at scale — what we've called the 98% adoption gap. A meaningful driver of that gap isn't model quality. It's that IT and risk committees stall procurement once they start asking exactly the architecture questions this article raises, and vendors aren't always prepared with a clear answer.

Where Sovereignty Actually Matters — and Where It Doesn't

Not every workflow needs a fully sovereign architecture. A partner asking a general research question about a public statute doesn't carry the same risk profile as a cross-border M&A due diligence review touching three clients' confidential financials.

Sovereignty-critical workloads typically share a few traits:

• Privilege exposure — matters where opposing counsel could later argue that AI-assisted review compromised privilege protections, a scenario our analysis of Claude Cowork and privilege examines in detail
• Regulated or conflicted clients — financial services, healthcare, or government clients with contractual data-handling requirements that go beyond standard vendor security certifications
• Competitive-intelligence risk — matters where the firm's specific reasoning pattern (not just the underlying facts) would be valuable to a competing firm or client
• High-volume precedent work — large-scale case search and litigation analytics workflows where the retrieval index itself becomes a proprietary asset over time

For these workloads, private AI deployment isn't a defensive posture — it's the only architecture that makes data control a structural default rather than a contractual promise. For lower-stakes, high-volume drafting work, a shared-cloud tool may be perfectly adequate. The mistake most firms make is applying one procurement standard across every use case, rather than matching architecture to risk tier.

The Question to Ask Before the Contract, Not After

Most legal AI contracts get negotiated on price per seat, output quality benchmarks, and integration timelines. Few get negotiated on the retrieval and logging architecture — because most buying committees don't know to ask, and most sales teams aren't incentivized to volunteer it.

Before signing, a CIO or managing partner should be able to get a straight answer to five questions:

1. Where does the full document corpus live — vendor cloud, or firm infrastructure?
2. Is the vector store single-tenant, or logically separated inside shared infrastructure?
3. What specifically leaves firm infrastructure on each query — full documents, large context windows, or minimized chunks?
4. Under whose API terms does that data reach the LLM provider — the vendor's enterprise agreement, or the firm's own negotiated terms?
5. Is the platform model-agnostic, or contractually locked to one provider's data-handling terms?

If a vendor can't answer these in a procurement meeting with the same fluency they answer questions about drafting speed, that's the answer.

The firms getting this right aren't the ones avoiding AI adoption — they're the ones sequencing it correctly, matching sovereignty-critical workloads to architectures built for firm-side control, and reserving shared-cloud tools for the work where that trade-off makes sense. That's a harder conversation than comparing chat interfaces, but it's the one that actually protects the judgment a firm has spent decades building.