The evidence
The law, made up.
A chatbot will draft your contract in seconds (fake clauses, the wrong state's rules, citations to cases that don't exist) and never once tell you it guessed. It's not a hunch. Every figure below is sourced.
How often it gets the law wrong.
The four numbers that survive scrutiny: independent studies and public court records, not vendor marketing.
of the time, general AI chatbots hallucinate on legal questions, getting the law wrong more often than right (GPT-4 58%, Llama 2 88%).
Even the AI tools built for lawyers hallucinate, despite being marketed as "hallucination-free" (Lexis+ AI >17%, Westlaw AI >34%).
court cases have already been caught citing AI-fabricated law, and the count is climbing two to three every day.
of even the best AI contract drafts is legally inadequate, and the chatbot won't warn you what's missing.
Don't take our word for it.
The people who build these tools (and the courts that see the fallout) say it plainly.
People have a very high degree of trust in ChatGPT, which is interesting, because AI hallucinates. It should be the tech that you don't trust that much.
You will not rely on Output… as a substitute for professional advice.
Hallucinations remain a fundamental challenge for all large language models.
AI hallucination caused lawyers to submit briefs with citations to non-existent cases.
"Uncritical reliance" on AI output can violate a lawyer's duty of competence.
ChatGPT can make mistakes. Check important info.
What it quietly gets wrong.
Not typos: silent, document-killing defects. Documented, real-world examples of what a chatbot leaves out.
A clause that's void where you live
Chatbots routinely draft "no competing work for two years" language that is void in California and other states, because the bot applies one jurisdiction's rule everywhere.
Missing your state's signing rule
An AI-generated will omitted New York's required attestation clause: two witnesses, 30-day window. The defect was found in probate, too late to fix.
A hole your competitor walks through
A ChatGPT-drafted IC agreement left out a non-circumvention clause and agency-authority limits, leaving the business exposed to having its customers poached.
Unenforceable terms, missing disclosures
AI-drafted leases "may include terms that are unenforceable in your state, or miss mandatory disclosures," like lead paint. The tenant relationship is built on sand.
Fake law, real bills.
When invented law reaches a courtroom, it costs money. A sample from the public record.
And it's not just lawyers. Self-represented, non-lawyer filers are now the largest group getting caught: 957, versus 639 attorneys. The people most likely to trust a chatbot with their own legal work are the ones getting burned most.
Against free, $15 looks expensive.
Against a lawsuit, it's the cheapest insurance you'll buy.
The average U.S. contract dispute runs about $29,000; for small businesses, the median climbs toward $91,000. A void non-compete, a will that fails in probate, a missing carve-out: set that against a $15–$30 document, and the math isn't close.
Not generated. Governed.
This isn’t an argument against AI. It’s an argument against letting a chatbot author the law.
ArtiEsq uses AI too, but only to assemble, never to decide. Here's the difference that makes.
A lawyer built the framework
A licensed attorney designs every document's structure and required provisions in advance. The AI never writes the law; it fills your answers into a framework that was lawyered first.
Your state's real rules, wired in
The actual statutes for your jurisdiction are built into each document. No "one rule applied everywhere," no clause that's quietly void where you live.
No fabricated clauses or citations
Because the AI assembles from a governed template instead of generating from scratch, there's nothing to hallucinate: no made-up cases, no missing carve-outs, no guesses.
Stop guessing. Start with one document: free.
See an attorney-designed, state-specific document for yourself. No account needed.
Find my documentArtiEsq provides AI-assembled legal document templates designed by a licensed attorney. It is self-help software, not a law firm. The documents do not constitute legal advice and do not create an attorney-client relationship. For complex matters, consult a licensed attorney in your jurisdiction.
Sources
- Stanford RegLab / HAI, Large Legal Fictions: Profiling Legal Hallucinations in LLMs, Journal of Legal Analysis (2024).
- Stanford HAI, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, J. Empirical Legal Studies (2025).
- Damien Charlotin, AI Hallucination Cases Database, figures as of June 23, 2026.
- legalbenchmarks.ai, Benchmarking Humans & AI in Contract Drafting, Phase 2 (Sept 2025).
- Sam Altman, remarks on the OpenAI podcast (June 2025).
- OpenAI: Terms of Use; Why Language Models Hallucinate (Sept 2025).
- Chief Justice John G. Roberts Jr., 2023 Year-End Report on the Federal Judiciary.
- American Bar Association, Formal Opinion 512 (July 2024).
- Dott Legal (2025); Rogers Sevastianos & Bante (Dec 2025); Spellbook (2025). Documented drafting-failure examples.
- Sanctions: Couvrette v. Wisnovsky (D. Or. 2026); Jordan v. Chicago Housing Authority (2025); Noland v. Land of the Free (Cal. Ct. App. 2025); Nippon Life v. OpenAI (2026).
- Cost figures: WorldCC/IACCM contract-dispute data; Hone Law / Everlaw–ACC small-business litigation medians.
Statistics are attributed to their sources and were current as of June 2026; figures for fast-moving trackers may have changed since.