Stanford’s Research on AI Tools in Law: What Lawyers Should Know

The Five Key Takeaways from This Blog

• Researchers at Stanford have found that on average, 1 in 6 queries by supposedly “hallucination-free” legal-assistance AI tools yielded hallucinatory language. 
• Some A.I. companies are claiming to offer lawyer-assistant A.I. that is hallucination-free, but there is little evidence to suggest that such A.I. even exists.
• These findings should be of interest to virtually all lawyers planning on using A.I. in their profession. 
• Overreliance on A.I. in the legal profession has landed many lawyers in hot water, threatening their careers. 
• This should be a wake-up call across all professions, but especially in law, that the proper use of A.I. should involve fact-checking and a healthy skepticism of what the A.I. claims is true. 

Hallucinating in the Courtroom

The Tools Are Still Hallucinating

The reality is that most supposedly hallucination-free AI tools fail 1 in 6 benchmark queries, as Stanford researchers found.

Stanford elaborates on the kinds of hallucinations. They can be either outright wrong, such as saying that the First Amendment of the United States Constitution allows for Americans to speed past stop signs as a free expression of speech, in the form of replying “No thanks” to the sign saying “Stop”. This kind of hallucination is just incorrect. 

The other kind of hallucination is misgrounded, meaning the actual meat of the statement may be A-grade, but it is supported with the wrong (or just fictional) information. The already classic example of this in A.I. is a chatbot that accurately demonstrates a law as it would be carried out in a courtroom, but it uses a fictional case in the demonstration. 

Here’s a more specific, funny example of misgrounding: claiming a Monopoly player can collect $200 for passing Go, citing the First Amendment as the rule’s basis.

However, Stanford found that legal AI tools were more accurate in answering open-ended legal questions than general-purpose chatbots like ChatGPT.

Why A Chatbot May Hallucinate (and What You Can Do About It)

If you are a lawyer wondering how to use A.I. responsibly (which, we remind you, will be essential for your professional self-preservation within your career field, should you choose to use A.I.), then it will do well to consider why an A.I. may hallucinate. 

Well, as it turns out, the reasons are various. 

One of the reasons may just be in the training data. 

Many chatbots are trained on a virtual enormity of written text. Some of this text may come from peer-reviewed legal journals with up-to-date knowledge.

Other data, meanwhile, may come from the largely unregulated parts of the Internet, where information of varying accuracy is posted. One example is the website forum Reddit’s comment sections, where multiple answers to a question may all be incorrect.

This mix of accurate and inaccurate info in chatbot training data can lead to inaccurate responses.

Claims of thoroughly vetting all AI training data should be met with skepticism due to the sheer volume involved.

The best thing for a lawyer to do here is to always fact-check the output of A.I. Even if the A.I. is correct most of the time, those times it is incorrect can cost you big.

Another reason for hallucination just has to do with the nature of generative A.I. itself.

Such systems are not purely information-retrieval platforms, which seems more plagiaristic. But, by design, they will try to find creative solutions to a problem put to it.

This may take the form of, for example, making up an unreal case to support an accurate description of a real law.

Always make sure to investigate AI-cited sources to avoid supporting real-life cases with fictional references.