When a 40-partner Hong Kong firm decides to run its own AI infrastructure rather than license a per-seat tool, it makes industry headlines. When that same firm's leadership explains the decision was about data residency and document custody, not about wanting to operate GPU clusters, the nuance gets lost in the retelling. That's roughly what happened with Shum Wong & Co.'s reported Ridger/MUX deployment this month — a real, defensible decision that's now being used to argue every firm needs to become its own data center operator. It doesn't.
The sovereign AI debate that's dominating legal tech roundups this quarter is asking the right question — who controls the firm's AI stack? — and landing on the wrong answer. Ownership of infrastructure and ownership of architecture are not the same thing, and conflating them is sending General Counsel and managing partners down an expensive, unnecessary path.
The Shum Wong & Co. Case and What It Actually Proves
Shum Wong & Co.'s move to deploy Ridger alongside a MUX-based orchestration layer was framed in recent coverage as a bet on "sovereign AI" — the idea that a firm should own its compute, storage, model weights, and orchestration end-to-end rather than route client data through third-party SaaS. For a firm handling cross-border PRC-Hong Kong matters under Hong Kong's Personal Data (Privacy) Ordinance, with clients who explicitly prohibit document transmission to foreign-hosted platforms, that instinct is sound.
But look closer at what the deployment actually solved for: document custody, retrieval control, and auditability inside the firm's own environment. That's an architecture decision, not a procurement decision to buy racks of H100s. Conflating the two is where the current debate goes wrong — and where firms considering a similar move risk over-engineering a solution to a problem that doesn't require it.
The real lesson from Shum Wong & Co. isn't "build a data center." It's: the firm that controls its index, its connectors, and its logs controls its AI destiny — regardless of whose GPUs do the final inference step.
The False Binary Driving Bad Decisions
Most of the current commentary frames the choice as binary:
- Rent everything — subscribe to a per-seat legal AI SaaS platform (Harvey, Legora, CoCounsel, Lexis+ Protege, or general-purpose tools like Claude Cowork or ChatGPT Enterprise) and accept that client documents, prompts, and derived work product transit a vendor's cloud.
- Own everything — stand up GPU infrastructure, model-serving stacks, and storage in a firm-controlled data center, as the Ridger/MUX narrative implies.
This binary ignores the layer that actually determines sovereignty: the retrieval and agentic orchestration layer that sits between the raw documents and the model call. A firm can own that layer entirely — inside its own Azure, AWS, or on-prem tenancy — while still using any LLM provider it chooses for the final inference step, under API terms the firm negotiates itself.
The data supports this distinction. Industry estimates for a production-grade GPU cluster capable of serving a mid-size firm's document volumes (think 8-16 H100-class GPUs plus redundancy, networking, and cooling) run $2-6 million in upfront capex, before a single query is answered, plus $400,000-$900,000 annually in power, cooling, and specialized MLOps staffing that most firms' IT departments don't have in-house. That's a defensible investment for a hyperscaler or an AI lab. It's a very hard one to justify for a law firm whose core business is legal services, not infrastructure operations.
What Sovereignty Actually Requires — Layer by Layer
Breaking the AI stack into its component layers makes the real decision points visible:
| Layer | What it controls | Does it need to be firm-owned for sovereignty? |
|---|---|---|
| Raw document storage | Full client files, matter history | Yes — stays in firm's tenancy/on-prem |
| Vector index / retrieval | What's searchable, how it's chunked | Yes — stays in firm's tenancy/on-prem |
| Connectors (DMS, email, matter mgmt) | Access scope, sync logs | Yes — stays in firm's tenancy/on-prem |
| Permissions & ethical walls | Who can query what | Yes — stays in firm's tenancy/on-prem |
| Audit logs | Every query, every chunk retrieved | Yes — stays in firm's tenancy/on-prem |
| Agentic workflow orchestration | How tasks are sequenced, reviewed | Yes — stays in firm's tenancy/on-prem |
| Model inference (the LLM call) | Final reasoning/generation step | No — can be externalized to a chosen provider |
| Raw GPU compute | Physical hardware serving the model | No — doesn't affect document custody |
Seven of eight layers in that table determine whether a firm actually controls its data. Only one — raw compute — is what the "build a data center" argument insists firms must own. That's the oversimplification this debate needs corrected.
What Leaves the Firm's Infrastructure — And What Doesn't
The honest architectural distinction isn't "we never send anything to an LLM provider and competitors do." Every serious legal AI platform, RAGbase Legal included, ultimately calls a model — whether that's a provider's frontier model or an open-weight model the firm hosts itself. The distinction that matters is what gets sent, and from where.
In a properly architected private AI deployment, a partner's query against a 400-page credit agreement doesn't ship the full document to anyone. The retrieval layer — running inside the firm's own infrastructure — identifies the eight relevant clauses, chunks them to perhaps 2,000-4,000 tokens, and sends only that minimized payload to the selected model's API for the generation step. The other 396 pages never leave the firm's environment. Neither does the permission record showing who asked, the audit log showing what was retrieved, or the underlying index that makes future queries faster.
Compare that to the typical per-seat SaaS pattern, where the vendor's platform ingests, indexes, and stores the full document corpus on its own infrastructure as a precondition of the service working at all — because the retrieval and orchestration layer is the vendor's product. That's not a criticism of those platforms' security postures, which are often robust (SOC 2 Type II, encryption at rest, etc.). It's an architectural fact: the firm's full corpus lives somewhere the firm doesn't operate.
The gap between those two models is the entire basis of the sovereignty argument — and it's resolvable without a GPU purchase order.
The Capex Case Against DIY AI Infrastructure
Run the numbers on a 300-lawyer firm evaluating the "own everything" path versus a sovereignty-first managed deployment:
Building an AI-native data center:
- $3-5M initial infrastructure capex (compute, storage, networking)
- $600K-$1M annually in specialized staffing (MLOps engineers are a different hire than firm IT, and they're scarce — median AI infrastructure engineer compensation in major markets now exceeds $220K base)
- 9-18 month build-and-harden timeline before the first production workflow goes live
- Ongoing responsibility for model updates, security patching, and GPU refresh cycles as hardware depreciates on a 3-4 year curve
Private AI deployed inside the firm's existing cloud tenancy:
- Infrastructure runs in environments firms already operate or already trust (Azure, AWS, or on-prem), with no new data center footprint
- Deployment timelines measured in weeks, not years, because the firm isn't building a hardware stack from scratch
- Ongoing cost structured as a platform fee rather than a capital commitment with depreciation risk
- The retrieval index, connectors, and logs are provisioned once and owned by the firm indefinitely — no vendor lock-in on the data layer, even if the firm later changes which LLM provider it calls
The second path delivers the same custody guarantees the Ridger/MUX deployment was designed to achieve, at a fraction of the capital outlay and a fraction of the time-to-value. For most AmLaw 200 firms, that's not a close call.
Where This Actually Matters: Privilege, Cross-Border, and Conflicts
The sovereignty question isn't academic. It shows up in three recurring scenarios where firms are making real architecture decisions right now:
- Cross-border matters under data residency law. Firms with PRC, EU, or Middle East-domiciled clients increasingly face explicit contractual or regulatory bars on document transmission to foreign-hosted SaaS platforms. GDPR's cross-border transfer mechanisms and Hong Kong's PDPO both create real friction for shared-cloud legal AI tools that process documents on vendor infrastructure outside the client's jurisdiction.
- Privilege preservation. Work product and attorney-client privilege analyses increasingly scrutinize where derivative content (summaries, drafts, extracted clauses) is generated and stored, not just where the source document lives. An architecture where the full analysis trail — retrieval, reasoning, drafts — stays inside firm-controlled logs is materially easier to defend in a privilege challenge than one where that trail lives on a third party's servers.
- Conflicts and ethical walls at scale. As firms deploy agentic AI workflows that touch multiple matters, the permissioning layer — which agent can see which client's documents — has to be enforced at the retrieval layer, not just the application layer. That's only fully auditable when the firm controls that layer directly.
Each of these is a genuine driver toward the sovereignty side of the spectrum. None of them requires owning a data center.
The Pragmatic Middle Path
This is the gap RAGbase Legal is built to close. The platform deploys the full agentic scaffolding — connectors into DMS and email systems, the retrieval and vector index layer, permissioning, audit logging, and workflow orchestration — inside the firm's own infrastructure, whether that's a private cloud tenancy the firm already operates or an on-premise environment for firms with the strictest residency requirements. Full client documents never leave that environment. Only the retrieved, minimized chunks needed for a specific query are sent to the firm's chosen LLM provider for inference, under API terms the firm negotiates and controls — including the option to route through region-specific or sovereign-cloud model endpoints where regulatory requirements demand it.
This is also why the comparison across tools matters as much as the comparison across architectures. A firm evaluating case search and research workflows, drafting assistance, and due diligence automation should be asking not just "which tool has the best output quality" but "where does my document corpus live while this tool is working, and who can audit that." Our AI for law firms guide walks through that evaluation framework in more depth, including the total-cost comparison between per-seat SaaS licensing and infrastructure-light private deployment over a three-year horizon.
What Comes Next
Expect the sovereign AI conversation to intensify through 2027 as more firms outside Hong Kong and Singapore — markets with acute cross-border sensitivity — start asking the same questions domestically, driven less by regulation and more by competitive pressure once clients start requesting documented data-handling architecture in RFPs. The firms that win that conversation won't be the ones who built the biggest data center. They'll be the ones who can draw an accurate architecture diagram showing exactly what stays in-house and what, minimally, goes out.
That diagram is the actual deliverable of the sovereignty debate — not a capex commitment.
If your firm is weighing a build-vs-license decision on AI infrastructure, start by mapping your own stack against the eight-layer breakdown above. Most firms find that five or six of those layers can be brought under direct control immediately, without touching compute ownership — and that's where the real sovereignty gains are sitting, waiting to be claimed.
Frequently Asked Questions
Does a law firm need to build its own data center to achieve AI sovereignty?
What actually leaves a firm's infrastructure when using a sovereignty-first architecture?
Why did Shum Wong & Co. in Hong Kong deploy its own AI infrastructure instead of using a SaaS legal AI tool?
Related Articles
AI for Law Firms in 2026: The Complete Guide to Choosing, Deploying, and Owning Legal AI
Comprehensive guide to AI adoption for law firms in 2026 — agentic AI, proprietary vs SaaS, privilege implications, pricing, and the ownership model.
Your AI Vendor's Moat Is Your Data. Here's How to Take It Back.
How SaaS AI vendors build competitive moats from your firm's usage data — the shared learning paradox, the dilution problem, and why proprietary AI keeps the compounding advantage with you.
98% of AmLaw 200 Firms Use AI — But Most Still Can't Search Their Own Files
98% AI adoption, but most law firms still can't search their own institutional knowledge. The gap between external AI tools and internal document access — and how to close it.
The Hidden Cost of Legal AI: Why 300-Lawyer Firms Are Spending $4.3M on Tools That Can't Find Their Own Case Files
Legal AI subscriptions cost up to $4.3M/year for large firms, yet can't search internal case files. Compare SaaS costs vs proprietary AI ownership economics.
Agentic AI for Law Firms: What It Actually Means in 2026
What agentic AI actually means for law firms — plain-English definition, what the big players are doing, real deployment examples, and how custom agents differ from SaaS workflows.
The True Cost of Legal AI: SaaS Subscriptions, Hidden Fees, and the Ownership Alternative
The hidden costs of legal AI in 2026 — SaaS subscription economics, the efficiency penalty on billable hours, data sovereignty risks, and why proprietary AI changes the math.
RAGbase builds private AI systems for law firms: deployed on the firm's own infrastructure, zero data retention, full ownership.
See How RAGbase Works on Your Data
30-minute call. We scope your use case and show the system live.