The Six-Layer Legal AI Stack: Why Infrastructure Wins

Cumulative venture funding into legal AI crossed roughly $3 billion by the end of 2025. Three years into the buying cycle, a growing number of AmLaw 200 CIOs are running the same audit: after renewing a per-seat contract for the third straight year, what does the firm actually own? The answer, in most shared-cloud deployments, is a seat count, a usage dashboard, and a cancellation date. The retrieval index, the agent workflows, the permission logic, the audit trail — all of it lives on someone else's cloud, built around someone else's roadmap.

An infrastructure briefing circulating among legal tech buyers this month crystallizes why that matters. It frames legal AI not as a single product category but as a six-layer stack, and argues that the layer most vendors compete on — which frontier model they resell — is the layer that matters least. The durable moat sits one level down: who owns orchestration, retrieval, permissions, and logs. That thesis lines up with what we've argued at RAGbase Legal since day one, and it's worth walking through the stack layer by layer to show exactly where firms lose control — and what the alternative architecture looks like in practice.

The Six-Layer Stack, Explained

Every legal AI product, regardless of brand, is built from the same six functional layers. The difference between vendors isn't whether these layers exist — it's who operates them and where they run.

The pattern is consistent: Layer 1 is the only layer most buyers evaluate, and it's the one layer that's genuinely commoditized. GPT-4-class and Claude-class models are converging on benchmark performance, and switching costs between them are falling as function-calling and context-window standards mature. Layers 2 through 6, by contrast, are where switching costs actually accumulate — and where most legal AI vendors quietly build their lock-in.

Where the Competitive Field Actually Sits

The current market maps cleanly onto this stack, even though the marketing rarely says so.

Consumer AI assistants repurposed for legal work — the Claude Cowork category — operate almost entirely at Layer 1 and a thin slice of Layer 2. They're fast to pilot because there's no retrieval infrastructure to stand up, but that's also the limitation: without a firm-owned index at Layer 3, the assistant can't reliably ground answers in a firm's full document history, and without Layer 5 controls, ethical-wall enforcement becomes a prompt-engineering problem rather than an architectural guarantee.

Per-seat legal SaaS platforms — the Harvey, CoCounsel, Legora, and Lexis+ Protege category — go deeper. They build real orchestration and retrieval layers, which is why they outperform generic assistants on legal-specific tasks. Harvey, for instance, has raised roughly $300 million across its last disclosed rounds and reports deployments at over 500 law firms and legal departments; CoCounsel inherited Casetext's retrieval stack and Thomson Reuters' content licensing; Legora has emphasized collaborative, matter-centric workflows since its 2025 raises. But in each case, Layers 2 through 6 run on the vendor's cloud, under the vendor's pricing tier, with export and portability governed by the vendor's contract terms — not the firm's infrastructure policy.

This is the structural reality the briefing points to: the firm is renting the only layers that actually compound in value. Model access, the layer everyone markets, is the one layer a firm can always get elsewhere.

The Chunk-vs-Corpus Distinction That Actually Matters

The honest version of this argument is not "competitors send your data out and we never do." Every serious legal AI product — including RAGbase Legal — ultimately calls a frontier model, because that's where the generation happens. The architectural question that actually determines risk exposure is what leaves the firm's infrastructure, and how much of it.

In a shared-cloud deployment, the full document corpus — every contract, deposition transcript, and privileged memo a firm has ingested — sits on the vendor's servers so the vendor's retrieval layer can search it. When a lawyer asks a question, the vendor's system retrieves relevant passages and forwards them, along with the query, to the frontier model API the vendor has contracted with. The firm's data has already made two hops outside its own control before the model ever sees it: into the vendor's storage, and into the vendor's model relationship.

In a sovereign architecture, the sequence is different. The full corpus, the embeddings, the retrieval index, the permission checks, and the resulting logs all stay on the firm's own infrastructure — whether that's a private cloud tenant or fully on-premise. When a query runs, the retrieval layer pulls only the minimal set of chunks needed to answer it — typically a handful of paragraph-length passages — and sends those, under the firm's own API agreement with the model provider of its choice, to generate a response. The model sees a narrow slice of text for a few seconds. It never has standing access to the corpus, the index, or the firm's permission structure.

That's the real distinction: full corpus plus agent layer under client control, versus minimized chunks sent to a model under the firm's own terms. It's the difference between a vendor holding your document library and routing queries through it, versus your own infrastructure holding the library and making brief, scoped calls outward. Firms evaluating private AI deployment options should ask vendors to draw this data-flow diagram explicitly — most per-seat SaaS contracts don't, because the answer reveals how much sits outside the firm's walls.

Why Model Choice Is a Governance Decision, Not a Product

Treating the model as swappable isn't a technical nicety — it's a risk-management requirement that's becoming harder to ignore. Three dynamics are driving this:

• Model performance leapfrogging is now measured in months, not years. A firm locked into a single provider's model through its vendor contract inherits that provider's pace of improvement — and its outages, rate-limit changes, and pricing resets.
• Jurisdictional and client-mandated model restrictions are increasing. Some financial-services and government clients now specify which model providers are acceptable for matters involving their data, a requirement a single-model SaaS product structurally cannot satisfy without a full platform migration.
• Pricing volatility at the model layer is routinely absorbed by the vendor, then passed through. When frontier labs adjust token pricing, per-seat vendors have historically repriced their own subscriptions within two to three quarters — a markup the firm pays twice, once on the model and once on the vendor's margin.

A stack architected around swappable models at Layer 1, with everything else firm-owned, absorbs these shifts at the infrastructure level. A firm using RAGbase Legal can route a cross-border regulatory matter to one model provider and a domestic litigation matter to another, governed by the same permissions and logging layer, without touching a vendor relationship or renegotiating a contract. That flexibility doesn't exist in architectures where the model is bundled into the product.

What RAGbase Legal Owns at Each Layer

Mapping RAGbase Legal's architecture against the same six-layer framework makes the positioning concrete rather than rhetorical:

1. Model — Firm-selected and swappable. Frontier models from leading providers are available by API, chosen per practice group, matter sensitivity, or client mandate, not locked by platform.
2. Orchestration — Runs on the firm's own infrastructure. Agent workflows, multi-step research and drafting chains, and tool-calling logic are configurable by the firm's own innovation team, not hidden behind a vendor's black box.
3. Retrieval & Index — The firm's document corpus, embeddings, and vector indices — including specialized indices for case search across precedent and matter history — live on firm-controlled infrastructure and are never shared across tenants.
4. Connectors & Data — Integrates with the DMS, email, and matter systems the firm already runs, rather than requiring migration to a vendor-mandated data layer.
5. Permissions & Identity — Maps directly onto the firm's existing identity provider, ethical-wall policies, and conflicts database, so access control reflects the firm's actual governance structure rather than an approximation of it.
6. Logging & Audit — Every prompt, retrieval, and model call is logged on the firm's own infrastructure, immutable and exportable on the firm's terms — critical for privilege review and for responding to outside counsel guidelines that increasingly require AI-use disclosure.

The practical effect: a firm can change its model provider, its cloud region, or even its primary legal AI vendor relationship without re-architecting its retrieval index, re-training its permission mappings, or losing its audit history. That portability is the moat the briefing describes — not a marketing claim about which model is best this quarter.

What This Means for the Next Renewal Cycle

For managing partners and CIOs evaluating 2027 budgets, the six-layer framework reframes the real vendor-selection question. It's not "which product has the best model this quarter" — that answer changes every six months and will keep changing. It's "which layers will we still own when this contract ends."

A useful diligence exercise: ask any legal AI vendor to specify, layer by layer, where each one runs and who can export it. Ask what happens to the retrieval index, the agent configurations, and three years of query logs if the firm terminates. In most per-seat SaaS agreements, the honest answer is that those assets don't transfer — they're deleted or retained by the vendor, and the firm starts over with whatever comes next. Firms that have mapped their own AI maturity against this framework — a process covered in more depth in our AI for law firms guide — consistently find that infrastructure ownership, not model access, is the variable that determines long-term cost and flexibility.

The frontier-model layer will keep changing quarterly for the foreseeable future — that's a feature of the AI market, not a flaw to architect around. The firms that come out ahead over the next three renewal cycles won't be the ones that picked the best model in 2026. They'll be the ones that built the orchestration, retrieval, permissions, and logging layers on infrastructure they actually control, so that whichever model wins next can simply be plugged in. Worth mapping your current stack against these six layers before the next contract renewal — the gaps tend to be where the real cost, and the real risk, is hiding.