The 3 Questions Managing Partners Skip Before Buying Legal AI

Managing partners evaluating AI tools are running the wrong diligence process. They benchmark model quality, compare per-seat pricing, and ask which vendor has the best demo. Those are procurement questions. They tell you almost nothing about what happens to a client's merger agreement, litigation hold, or privileged memo once it's uploaded.

The question that actually matters — where does the file go, and who can prove what happened to it — rarely makes it into the RFP. That's a governance gap, and under current regulatory standards, it's an expensive one.

Why "Which Model?" Is the Wrong First Question

Model selection is a moving target. GPT-4, Claude, and Gemini leapfrog each other on benchmarks every few months, and whichever model tops the leaderboard this quarter will likely not top it next quarter. Firms that anchor their AI strategy to a specific model are optimizing for a variable that changes faster than their contract renewal cycle.

What doesn't change quarter to quarter is architecture. Where documents physically reside, who holds the encryption keys, how retrieval is scoped, and whether the firm or the vendor controls the audit trail — these are structural decisions, not features. Get them wrong, and no amount of model quality fixes the exposure.

This is also where most vendor conversations break down. Sales teams are trained to answer "is it secure?" with SOC 2 badges and encryption-at-rest claims. That's a different question from "where do the actual files sit, and who can retrieve them." A managing partner who accepts the first answer as a substitute for the second has skipped the diligence that matters.

The Three-Question Framework

Before signing anything, run every AI vendor — model-native or platform — through three questions. None require a technical background. All three determine your firm's actual exposure.

1. Where Do Full Client Documents Live, During Processing and After?

Not "is it encrypted." Not "is it SOC 2 certified." Physically, where does the file sit — during upload, during processing, and after the session ends?

There are three structurally different answers, and firms should be able to name which one applies to every tool in their stack:

• On the vendor's servers, retained indefinitely or per a retention policy the firm doesn't control. Common in consumer AI assistants and many per-seat legal SaaS tools where the full document is uploaded and indexed on vendor infrastructure.
• In a shared, multi-tenant index, where the firm's documents sit alongside other customers' data behind logical (not physical) separation.
• On infrastructure the firm controls, where the vector store, the permissions layer, and the full corpus never leave firm-owned or firm-designated environments — and only minimized, retrieved chunks are sent externally to answer a specific query.

That third model is the basis of a private AI deployment: the agentic scaffolding, the retrieval layer, the vector store, and the full client documents stay under the firm's control. What may leave the building is a handful of retrieved passages, sent to the selected LLM provider under terms the firm negotiated — not the underlying files.

This is not an argument that using a third-party model provider is inherently unsafe. RAGbase Legal itself can route queries to external LLM providers. The distinction that matters is what travels: full documents and metadata, or minimized, ephemeral chunks. A firm that can't answer this question for each tool in its stack has already lost the diligence exercise.

2. If a Judge Asks How a Citation Ended Up in a Brief, Who Produces the Audit Trail?

This is no longer a hypothetical. Courts have sanctioned attorneys for AI-fabricated citations in filings across multiple jurisdictions since 2023, and judges are increasingly requiring certifications of human review for AI-assisted work product.

ABA Formal Opinion 512 (July 2024) sets the standard: lawyers must exercise reasonable diligence over AI tools, which includes the ability to verify outputs and supervise the process that generated them. The EU AI Act's high-risk provisions, now phasing into force, impose parallel obligations on systems used in the administration of justice — including logging, traceability, and human oversight requirements that fall on the deploying organization, not just the model vendor.

Both frameworks converge on the same operational demand: the firm must produce evidence, on demand, showing who reviewed what, when, and against which source documents. "The vendor has logs somewhere" does not satisfy either standard, for three reasons:

• Access latency. If producing an audit trail requires opening a support ticket with the vendor, the firm cannot respond to a same-day discovery request or a judge's inquiry from the bench.
• Retention mismatch. Vendor log retention policies are built for the vendor's operational needs (debugging, billing), not for litigation holds or malpractice defense timelines that can run years.
• Chain of custody. A log the firm didn't generate and can't independently verify is weaker evidence than one the firm produced and controls end to end.

A defensible answer looks like this: the firm can pull a record showing the exact source documents retrieved, the specific passages surfaced to the model, the output generated, and the timestamp and identity of the reviewing attorney — without contacting anyone outside the firm. That's the standard case search and retrieval workflows need to meet if they're going to survive scrutiny in a malpractice claim or a Rule 11 challenge.

3. Is Switching Models a Config Change or a Data Migration?

This question exposes the difference between an AI system and a dependency.

If moving from one LLM to a newer or cheaper one requires re-indexing the firm's document corpus, re-permissioning every user and matter, or renegotiating a vendor contract, the firm hasn't bought an AI capability — it's bought a single-vendor commitment wearing an AI label. That matters commercially (pricing leverage evaporates) and operationally (the firm can't respond quickly when a better or more compliant model becomes available).

The test is simple: ask the vendor, in writing, what happens operationally if the firm wants to swap the underlying model next quarter. The answer reveals the architecture.

Firms that can't answer this question honestly are not making an AI decision — they're making a five-year infrastructure bet disguised as a software purchase.

What This Actually Determines: Exposure, Not Output Quality

Here's the strategic reframe: none of the three questions above are about which model writes the best first draft. They're about who controls the retrieval layer, the permissions layer, and the logs — because that's what determines a firm's exposure when something goes wrong, not the model name printed on the invoice.

Consider two firms using functionally similar AI assistants to draft a motion. Both get comparable output quality. But if one firm's documents sit in a shared vendor index with vendor-controlled logs, and the other firm's documents sit on infrastructure it controls with self-service audit trails, their risk profiles are not remotely similar — even though their invoices might look identical.

This is the core problem with evaluating legal AI the way firms evaluate ordinary SaaS: by feature checklist and price per seat. A hidden cost of that approach is that it defers the architecture question until after the contract is signed and the corpus is already indexed on someone else's infrastructure — at which point switching costs have already done their work.

Where the Major Platforms Land

The current field of legal AI tools splits roughly into three architectural camps, and it's worth naming them plainly rather than treating "legal AI" as one category.

Per-seat legal SaaS platforms (the CoCounsel and Lexis+ Protege category) bundle a specific model, a proprietary index, and a subscription in one package. They're fast to deploy and require no infrastructure lift, but the firm typically has limited visibility into where documents are indexed and little ability to swap the underlying model without a new contract.

Consumer-grade AI assistants repurposed for legal work — general-purpose chat tools without legal-specific retrieval scaffolding — offer the least architectural control. Full documents are frequently uploaded directly to vendor infrastructure with retention policies designed for consumer use cases, not litigation holds.

Agentic platforms with connector-based architectures, including tools like Claude Cowork and emerging entrants like Legora, sit closer to the third model described above: they can operate with retrieval scaffolding that limits what leaves the firm's environment. Whether a given deployment actually achieves that depends heavily on configuration — connector permissions, index location, and log ownership — not on the vendor's marketing description.

The honest takeaway is that vendor category is a weak proxy for actual exposure. Two firms using the same platform can have entirely different risk profiles based on how it's configured. That's why the three-question framework has to be applied per deployment, not once at the vendor-selection stage. Our AI for law firms guide breaks down deployment models in more depth for firms building this evaluation into their procurement process.

Building This Into Procurement, Not Just Diligence

The firms handling this well aren't running a one-time security review before signing. They're building the three questions into every renewal, every new connector approval, and every matter-type expansion, because the answers can change even when the vendor name doesn't — a vendor can change its data retention policy, add a new subprocessor, or shift its logging architecture in a product update the firm never sees.

A practical version of this discipline:

• Require documented answers to all three questions before any AI tool touches client documents, not just before enterprise-wide rollout.
• Re-verify at each contract renewal, since vendor architecture and subprocessor lists change more often than legal teams review them.
• Assign audit-trail ownership internally — a named person or team responsible for producing human-review evidence on 24-hour notice, independent of vendor support timelines.
• Test the migration question empirically during any pilot: actually attempt to change a configuration setting and time how long it takes.

None of this requires new technical expertise on the management committee. It requires treating architecture as a governance question with the same seriousness as conflicts checks or trust accounting — because increasingly, it carries comparable liability.

The model landscape will keep shifting — a new release every few months, a new leaderboard topper, a new vendor claiming superiority. None of that changes what a firm needs to be able to answer when a judge, a regulator, or a malpractice carrier asks where a document went and who reviewed the output before it went out the door. Firms that build their diligence around architecture rather than model marketing will be answering that question with evidence. Firms that didn't ask it will be answering it with a support ticket.