AI Legal Research Accuracy

What the Stanford study tested

A team from Stanford RegLab and the Stanford Institute for Human-Centered AI tested legal research tools from LexisNexis and Thomson Reuters, along with GPT-4. The authors used over 200 open-ended legal queries. The dataset was built by hand and pre-registered. The 2024 preprint reported that Lexis+ AI and Ask Practical Law AI gave incorrect answers more than 17% of the time, and that Westlaw AI-Assisted Research hallucinated more than 34% of the time.

The study counted two error types. One is an incorrect answer. The other is a misgrounded answer, where a real source does not support the claim. The authors saw misgrounding as potentially more harmful, because the citation looks trustworthy.

The peer-reviewed version

The study was published in the Journal of Empirical Legal Studies in 2025, volume 22, pages 216 to 242. A secondary summary of the abstract gives a range of 17% to 33%. I could not confirm the journal's exact Westlaw figure, so the numbers above come from the 2024 preprint. Check the journal version before you cite a specific rate. Use a citator or a legal database to confirm each case's history. A tool's summary can omit a later reversal, so that check belongs in a separate system.

What vendors said, and what changed

LexisNexis had promoted hallucination-free linked legal citations. The authors said clear evidence did not support that claim. The study first tested Ask Practical Law AI rather than Westlaw's AI-Assisted Research product, which drew criticism. Stanford said it would add the Westlaw product to its results. The authors also noted that tool responses changed during the study. Treat any vendor claim of zero hallucinations as a claim to test.

What to do in practice

Open every case or statute an AI tool cites. Confirm that it exists, says what the tool claims and is still good law. Check the pinpoint page. A citation that looks right but does not support the sentence is the most serious error.

Tools that show quoted passages help the check. Hyperresearch Answers returns quotes matched word for word against their pages, which confirms the quote appears in the source. It does not confirm that a legal conclusion is correct. A lawyer still has to read the source and decide. Keep a short log of each AI-assisted citation, with the date checked and the reviewer's initials. That record shows the verification happened.

Checklist

• Open every cited case or statute before you rely on it.
• Confirm each pinpoint page supports the sentence it backs.
• Check that each authority is still good law in a citator.
• Log which tool produced each citation for later review.
• Have an attorney sign off on any legal conclusion.

Questions & answers

Sources for this entry

• Stanford HAI: AI on trial, legal models hallucinate
• Stanford Law: Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
• Journal of Empirical Legal Studies 22(2), 2025 (citation record)

Vendor descriptions are attributed to the vendor. Prices change; confirm with each vendor.

Continue this reading path

• Legal & policy
• AI-Powered Research Tool
• Resource directory

Cite this entry