Legal AI Due Diligence: The Questions Procurement Misses

Ask ten managing partners how they're evaluating AI vendors this year, and nine will describe a bake-off: GPT-5 versus Claude versus Gemini, benchmarked on a handful of sample memos. It's the wrong evaluation. Model quality is a moving target that resets every few months — the leaderboard that mattered in January is stale by June. What doesn't reset is the architecture decision sitting underneath it: where your client files actually live, who can produce evidence of what happened to them, and whether you can leave without a migration project. Get that wrong, and it doesn't matter which model you picked.

The Procurement Trap: Why "Which Model" Is the Wrong First Question

"Which model?" is a question vendors love, because it's the one they're built to answer well. It puts the conversation on benchmarks, demos, and roadmap slides — all things that favor whoever presents last. It does not surface the question that actually determines your firm's exposure: what happens to a client's full document set once it enters the system.

This distinction matters more now than it did eighteen months ago, for two reasons. First, the vendor landscape has consolidated around a handful of architectures — Harvey, CoCounsel, Lexis+ Protégé, Legora, and newer entrants like Claude Cowork all route client data through some combination of vendor-hosted indices and third-party model APIs, with meaningful differences in what's retained where. Second, regulatory scrutiny has caught up. Between ABA Formal Opinion 512 and the EU AI Act's high-risk provisions phasing in through 2026, "the vendor handles that" is no longer a defensible answer to a bar complaint or a discovery request.

A useful diligence framework doesn't require your GC to understand transformer architecture. It requires three questions that any vendor should be able to answer in a single sentence — and most can't.

Question 1: Where Do Full Client Documents Actually Live?

Not "is it encrypted." Not "is it SOC 2 certified." Those are baseline hygiene, not architecture. The question is physical and contractual: when a client uploads a 400-page merger agreement, where does that full document sit, during processing and after the session ends?

There are three real answers, and they carry very different risk profiles:

• On the vendor's servers, indefinitely. The document is ingested, embedded, and stored in the vendor's vector database alongside other clients' data, retained per the vendor's terms (often 30-90 days minimum, sometimes indefinitely for "product improvement" unless the firm negotiates an opt-out).
• In a shared multi-tenant index. The document lives in infrastructure the vendor operates, logically separated from other tenants by permissions the firm cannot independently verify or audit.
• On infrastructure the firm controls. The document never leaves the firm's environment; only the minimal text chunks needed to answer a specific query are transmitted externally, to a model provider the firm selects.

Most per-seat legal SaaS tools fall into the first two categories by design — that's how the product economics work. The vendor needs your documents in their index to deliver instant retrieval across a multi-tenant platform. That's a legitimate architecture choice, and it's not inherently reckless. But it means the firm's confidentiality posture is now a function of the vendor's retention policy, breach history, and subcontractor chain — not the firm's own controls.

For sovereignty-critical matters — active litigation, regulatory investigations, anything with a protective order attached — that dependency is the risk itself, independent of whether the vendor has ever had an incident. A private AI deployment model inverts the default: full documents, the retrieval index, and permissions stay inside the firm's infrastructure, and only minimized, query-specific chunks are sent to the LLM provider under terms the firm negotiates directly. The model still does the reasoning. The firm still controls the corpus.

Question 2: Who Produces the Audit Trail When a Judge Asks?

This is the question most vendor RFPs never ask, and it's the one with real courtroom consequences. When a fabricated or mis-cited authority shows up in a filing — and it keeps happening, with sanctions orders now numbering in the dozens across federal and state courts since 2023 — the question a judge asks isn't "which AI tool did you use." It's "show me the record of human review."

Under ABA Formal Opinion 512, lawyers retain non-delegable responsibility for supervising AI-generated work product and maintaining a reasonable understanding of how client information is handled by the tools they use. Under the EU AI Act, systems used in the administration of justice or supporting legal reasoning fall into high-risk categories carrying explicit logging, transparency, and human-oversight obligations as those provisions phase in through 2026. Neither framework accepts "the vendor has logs somewhere" as a substantive answer. The obligation runs to the lawyer and the firm, on demand, not to whichever subprocessor happens to retain server logs that quarter.

That creates a concrete diligence test: can your firm export, right now, a complete record of every AI-assisted citation in a given brief — the source document, the retrieved passage, the model output, and the reviewing attorney's sign-off — without opening a support ticket with the vendor?

If the answer requires a vendor's cooperation, a data request, or a subpoena to a third party, the firm doesn't control its own compliance posture. It's borrowing it. This is precisely the mechanism at issue in cases like Mata v. Avianca and its successors, and it's the throughline connecting well-documented privilege exposure analyses to the broader question of who actually holds the evidentiary chain of custody for AI-assisted work.

Question 3: Is Switching Models a Config Change or a Data Migration?

The third question is the one that reveals whether a firm has bought an AI system or a dependency wearing an AI label. Ask the vendor directly: if we want to move from Model A to Model B next quarter, what has to happen?

In a properly architected system, where the retrieval layer, the permission structure, and the document index are decoupled from any single model provider, switching models is a configuration change — you point the same retrieval pipeline at a different API endpoint, with the same documents, same permissions, same logs, untouched. It's largely invisible to end users.

In a tightly coupled, vendor-hosted system, switching models often means re-indexing the entire document corpus against a new embedding model, re-establishing permission mappings, and in some cases renegotiating the underlying contract because the model relationship is bundled into the platform license. That's not a technical inconvenience — it's a lock-in mechanism, whether or not the vendor frames it that way.

This question matters more over a three-year horizon than it does today, because model economics are volatile. Pricing per token has fallen sharply across major providers over the past two years, and the performance gap between frontier models narrows every few months. A firm locked into one vendor's model relationship inherits that vendor's pricing and roadmap decisions indefinitely. A firm that owns its retrieval layer can renegotiate, or switch providers outright, whenever the market moves.

Architecture Comparison: What Actually Determines Exposure

The table below maps the three questions against the two dominant architecture patterns in legal AI today.

Neither column is universally "correct." A firm running high-volume contract review across low-sensitivity vendor agreements may reasonably accept shared-cloud economics for speed. The failure mode is applying that same architecture, by default, to a matter under a protective order — because nobody asked the three questions before the pilot started.

What This Means Across the Current Vendor Landscape

The honest version of this comparison isn't "vendors send data out, we never do." Every serious legal AI product — including architecture that keeps documents on firm-controlled infrastructure — ultimately calls an LLM provider to generate reasoning over retrieved text. The question was never whether a model provider ever sees text. It's how much text, from where, and under whose contract.

Harvey, CoCounsel, Legora, and Lexis+ Protégé each bundle model access with a hosted retrieval layer — convenient for fast deployment, but the corpus and index live on their infrastructure. Claude Cowork represents a newer pattern: an agentic layer that can operate more flexibly across a firm's existing tools, which shifts some of these questions but doesn't eliminate them — the diligence framework above still applies to wherever the connectors and retrieval index actually sit. ChatGPT Enterprise, absent a dedicated legal retrieval layer, pushes the entire diligence burden back onto the firm's own connector and permissioning discipline.

The architectural alternative — full corpus, retrieval index, permissions, and logs on firm-controlled infrastructure, with only minimized chunks reaching the model provider — isn't a rejection of any of these tools. It's a different answer to the sovereignty question for the subset of matters where the firm cannot afford to have that answer be "ask the vendor." For firms running case search and diligence workflows across privileged material, that distinction is the actual product decision, not a footnote to it.

Building Your Own Diligence Checklist

A workable version of this framework fits on one page and gets asked before the pilot, not after the invoice:

• Document location: Get the vendor's data flow diagram in writing — not a security whitepaper, the actual diagram showing where full documents sit at each processing stage.
• Retention terms: Confirm in the contract, not the sales deck, how long documents and derived embeddings are retained after a matter closes.
• Audit export: Request a live demo of exporting a complete human-review audit trail for a single citation, with no vendor involvement in the export itself.
• Model portability: Ask what changes technically and contractually if the firm switches underlying models in twelve months.
• Subprocessor chain: Get the full list of subprocessors touching client data, not just the primary vendor's own infrastructure.

Firms that have built this out internally are worth comparing notes with — the checklist above is a starting point, not a finished standard, and the AI for law firms guide walks through how the underlying architecture choices map to specific practice-group risk profiles.

The model layer will keep changing — faster than most firms' three-year technology contracts. The architecture underneath it won't change on its own, which is exactly why it deserves the first question, not the third. Before the next vendor demo, ask where the documents live, who produces the audit trail, and what switching actually costs. The answers will tell you more about your exposure than any benchmark slide.