When AI Gets It Wrong: The Malpractice Math Law Firms Can't Ignore

In June 2023, two Levidow, Levidow & Oberman attorneys filed a brief in the Southern District of New York that cited six cases—Varghese v. China Southern Airlines, Shaboon v. Egyptair, Petersen v. Iran Air, and three others. None of them existed. A federal judge subsequently fined each attorney $5,000, referred the matter for disciplinary proceedings, and wrote an opinion that has now been read by virtually every bar ethics committee in the country. That case, Mata v. Avianca, was the legal profession's Chernobyl moment for AI—a highly visible failure that exposed a systemic risk hiding beneath the efficiency narrative.

Two years later, the problem has not been solved. It has scaled.

The same generative AI capabilities that can summarize a 400-page deposition in four minutes can also, with equal fluency and equal confidence, invent a circuit court ruling that supports your client's position—complete with a plausible docket number, a real-sounding judge's name, and a holding that fits the facts perfectly. The fabrication is not a bug that will be patched in the next model update. It is a structural property of how large language models work. And for managing partners, CIOs, and innovation leads deploying AI at scale across their firms, understanding why this happens—and which architectural choices actually reduce the risk—is no longer optional.

The Hallucination Problem Is Not Shrinking. It's Professionalizing.

Early hallucination incidents were easy to dismiss as user error—attorneys using consumer ChatGPT without proper supervision, treating a general-purpose chatbot as a Westlaw replacement. That framing let the profession off the hook for about eighteen months. It no longer holds.

As of early 2025, courts in at least fourteen federal jurisdictions have addressed AI-fabricated citations in filed documents. The American Bar Association's Formal Opinion 512 (2024) explicitly confirmed that attorneys using AI tools retain full professional responsibility for verifying AI-generated content—there is no "the AI told me" defense. Meanwhile, at least two major legal malpractice insurers—including a carrier covering more than 800 AmLaw firms—have introduced AI-specific endorsements that create coverage gaps when a claim arises directly from unverified AI output.

The incidents are also climbing the credibility ladder. Early cases involved solo practitioners or small firms with minimal AI governance. More recent sanctions have touched attorneys at regional firms with 50+ lawyers and, in one unreported matter shared by a claims manager at a legal malpractice insurer, a partner at a firm inside the AmLaw 200. The pattern is consistent: a time-pressured attorney, a persuasive AI output, an inadequate verification step, and a judge who noticed.

The financial exposure math is not complicated:

None of these are theoretical. All have been documented in the past 24 months.

Why LLMs Hallucinate—and Why "Better Models" Don't Fully Fix It

Understanding the architectural root cause matters because it shapes which solutions actually work.

Large language models do not retrieve facts. They predict the next token based on patterns learned during training. When asked about Varghese v. China Southern Airlines, ChatGPT-4 did not search a database and come up empty. It generated what a real case citation looks like, drawing on the statistical patterns of thousands of real cases it had processed. The output was linguistically indistinguishable from a genuine citation because it was built from the same linguistic DNA as genuine citations.

Model improvements help at the margin. GPT-4o and Claude 3.5 Sonnet hallucinate less frequently on well-documented legal questions than GPT-3.5 did. But the hallucination rate for obscure or jurisdiction-specific legal questions remains materially non-zero across all frontier models—and in legal practice, it is precisely the obscure, jurisdiction-specific question where the stakes are highest and the attorney is most likely to rely on AI because their own knowledge is thinnest.

Retrieval-Augmented Generation (RAG) is the architectural intervention that actually addresses this. By grounding model responses in retrieved source documents—real cases, real statutes, real contract language from the firm's own corpus—RAG constrains the model to reason from verified text rather than invent from pattern memory. The hallucination rate drops sharply for in-scope questions because the model has actual source material to work with. The remaining risk is mis-attribution or mis-reading of real documents, which is a qualitatively different (and more detectable) failure mode.

But RAG architectures are not all equivalent. The critical question is: who controls the retrieval layer, the index, and the source corpus?

The Architecture That Determines Your Risk Exposure

This is where the conversation shifts from AI literacy to enterprise risk management.

Consider the typical workflow of a public SaaS legal AI tool versus a private, on-premise deployment with a controlled RAG layer:

Public SaaS legal AI (common pattern):

• Attorney submits a research query
• Query goes to provider's infrastructure
• Provider's retrieval layer searches provider-controlled indexes (which may include publicly licensed legal databases, web content, or the provider's aggregated training corpus)
• Retrieved chunks + query go to LLM
• Response returns to attorney
• Firm has visibility into: the query and the response
• Firm does not have visibility into: what was in the retrieval index, why specific chunks were selected, whether the source documents are current, or how the provider's model was fine-tuned

Private on-premise RAG deployment (RAGbase Legal architecture):

• Attorney submits a research query
• Query hits the firm's own agentic scaffolding, running on firm infrastructure
• Retrieval layer searches firm-controlled vector stores—populated with the firm's chosen legal sources, client documents, and internal work product, with permission controls enforced at the index level
• Only the minimal retrieved chunks needed to answer the question are sent to the selected LLM provider under the firm's chosen API terms
• Response returns through firm infrastructure, with full audit log of: query, retrieved chunks, source documents, model used, and timestamp
• Firm has visibility into: everything except the LLM's internal computation
• Firm controls: the corpus, the index, the permissions, the retrieval logic, the audit trail, and the API relationship with the model provider

The distinction matters for hallucination risk in a specific, auditable way. When a RAGbase Legal deployment retrieves a passage from Westlaw, Fastcase, or the firm's own precedent library and passes it to Claude or GPT-4o, the model is working with verified text. When the response is wrong, the audit log shows which retrieved chunk the model was working from. That traceability enables a partner to identify the failure point, correct it, and demonstrate to a court or insurer that the firm had a reasonable verification process in place.

That demonstrable process is increasingly what separates a sanctions risk from a defensible workflow.

For a deeper technical breakdown of how private deployment architectures work in practice, see our private AI deployment overview.

What "Controlled Sources" Actually Means in Practice

The phrase "grounded in verified sources" gets used loosely in legal AI marketing. It is worth being precise about what it requires.

Source quality: The retrieval index must contain current, authenticated legal materials. A vector store populated with legal content scraped from the open web in 2022 is not a controlled source—it is a differently-shaped hallucination risk. Firm-negotiated feeds from Westlaw, Lexis, or Bloomberg Law, ingested under contract and updated on a defined schedule, are controlled sources.

Source currency: Statutes are amended. Regulations change. Cases are overruled. A retrieval system that does not track document version and update frequency will confidently retrieve good law that is now bad law. The firm's retrieval infrastructure—not the LLM provider's—must own this maintenance obligation.

Source scope: Not every question should be answered from the same index. A question about Delaware corporate law should retrieve from a different corpus than a question about a client's employment agreements. Permission-aware retrieval—where the system enforces matter-level and client-level access controls before returning any document—is not a nice-to-have. It is a conflict-of-interest and confidentiality requirement.

RAGbase Legal's case search infrastructure is built around these three requirements: authenticated source feeds, version-tracked indexes, and permission-enforced retrieval. The retrieval layer lives on the firm's infrastructure. The LLM sees only what the retrieval layer decides it should see.

The Governance Gap That Existing Tools Leave Open

Harvey, CoCounsel, Lexis+ Protege, and Legora are serious products built by serious teams, and they have meaningfully reduced the hallucination rates associated with unguided use of general-purpose LLMs. This is a genuine contribution. The honest framing is not that these tools are reckless—it is that their architecture creates a specific governance gap that certain workloads cannot tolerate.

That gap is auditability of the retrieval layer.

When a Harvey-generated research memo contains an error, the attorney knows what question was asked and what answer was returned. What they typically cannot reconstruct—without provider cooperation—is: which documents were retrieved to ground that answer, whether those documents were the current version of the cited authority, and why the retrieval system selected those documents over others that might have been in the index.

For most routine research tasks at most firms, this gap is acceptable. The efficiency gains are real, the error rates are low, and the supervising attorney catches the failures before they reach the court.

For sovereignty-critical workloads—major litigation with exposure above eight figures, M&A due diligence where a missed regulatory issue can kill a deal, regulatory matters where an incorrect citation to agency guidance can constitute a misrepresentation—the acceptable error rate is effectively zero, and the auditability requirement is non-negotiable.

This is where the architectural distinction becomes a business decision, not a philosophical one. It is also why the most sophisticated users of tools like Harvey are not choosing between Harvey and a private deployment—they are using Harvey for high-volume, lower-stakes research and routing sovereignty-critical workloads to infrastructure they fully control.

For a comprehensive framework on making this workload-routing decision, the AI for law firms guide walks through the full risk stratification model.

What Malpractice Carriers Are Actually Watching

The insurance market is often the most efficient early-warning system for emerging professional liability risks, because underwriters price what they believe will generate claims.

Conversations with legal malpractice underwriters in Q1 2025 reveal a consistent pattern: carriers are not categorically penalizing AI use—they are penalizing unauditable AI use. Firms that can demonstrate a documented AI governance framework, including defined retrieval sources, mandatory human review protocols, and an audit trail linking AI output to source documents, are being treated favorably relative to firms that cannot.

Some specific carrier postures observed in the current renewal cycle:

• Coverage exclusions for claims arising from AI-generated work product submitted without documented verification steps (two major carriers, confirmed)
• Premium discounts of 5–12% for firms that can demonstrate AI governance documentation comparable to e-discovery proportionality standards (one carrier, confirmed)
• Questionnaires on AI tool usage now standard in renewal applications for firms above 50 attorneys (multiple carriers)
• Reservation of rights language in claim responses where AI use cannot be ruled out as a contributing factor (emerging, not yet standard)

The practical implication: the cost of an auditable AI architecture is now directly offset by insurance market dynamics. A firm that can show a carrier its retrieval logs, source provenance, and review workflows is a materially different risk than a firm that cannot. That difference is beginning to show up in premium pricing.

The malpractice risk from unreliable AI is not going to resolve itself through better models or more user training alone. Frontier LLMs will continue to hallucinate on the questions that matter most—the obscure, the jurisdiction-specific, the high-stakes—because that is a structural property of how they work, not a calibration problem. The firms that manage this risk effectively over the next three years will be those that treat AI architecture as a governance decision, not just a technology procurement decision.

The key questions for your next AI governance review: Can you produce, on demand, a full audit trail showing which source documents grounded any AI-assisted work product submitted to a court or client? Does your retrieval layer enforce matter-level permissions before documents reach the model? And do you have a documented, workload-specific protocol for distinguishing which matters can tolerate SaaS-tier AI and which require the full corpus to stay under your control?

If any of those questions produce hesitation, the architecture—not the model—is where the conversation needs to start.