An AI ‘hallucination’ turned up in a B.C. court case. Experts say it’s a wake-up call

Vancouver tech lawyer Ryan Black’s work with video game companies put him in a position to watch the rise of artificial intelligence in the industry.

Now he finds himself on the front lines again as his own profession grapples with the technology.

“The degree to which it was impacting game studios really surprised people,” said Black, who helped the Law Society of British Columbia draft advice for lawyers about the use of AI.

“The generative (AI) revolution kind of has really hit people really hard in terms of, ‘Oh my gosh, we have to really pay attention to this now,’ so I would say that it’s a new thing for a lot of people,” he said referring to the type of technology that can create arguments and essays based on prompts from a user.

“It doesn’t surprise me that lawyers don’t know a lot about it.” 

The rise of generative AI tools like ChatGPT, he said, is a “revolutionary change to the practice of law,” but a recent ruling by the B.C. Supreme Court shows lawyers must use the technology cautiously and skeptically, legal experts say. 

In a costs ruling released Feb. 20 related to a child custody case, it was revealed that Vancouver lawyer Chong Ke had used ChatGPT to prepare material submitted in the case. 

The material included citations to cases that don’t exist, something her opponent in the case called an AI “hallucination.”

Ke told the court that discovering that the cited cases were fictitious was “mortifying,” and she quickly informed the Law Society and admitted a “lack of knowledge of the risks” of using AI to draft court submissions. 

“I am now aware of the dangers of relying on Al generated materials,” Ke said in an affidavit. “I understand that this issue has arisen in other jurisdictions and that the Law Society has published materials in recent months intended to alert lawyers in B.C. to these dangers.” 

Ke apologized to the court and her fellow lawyers.

Her lawyer John Forstrom said in an email that the case “has provoked significant public interest, but the substance of what happened is otherwise unremarkable.”

 “I’m not sure that the case has any significant implications regarding the use of generative AI in court proceedings generally,” Forstrom said. 

“Ms. Ke’s use of AI in this case was an acknowledged mistake. The question if or how generative AI might appropriately be employed in legal work did not arise.”

The society is now investigating Ke’s conduct, spokeswoman Christine Tam said in an email. 

“While recognizing the potential benefits of using AI in the delivery of legal services, the Law Society has also issued guidance to lawyers on the appropriate use of AI and expects lawyers to comply with the standards of conduct expected of a competent lawyer if they do rely on AI in serving their clients,” Tam said. 

The law society’s guidance, issued in late 2023, urges lawyers to seek training in the use of the technology, and be aware of confidentiality issues around data security, plagiarism and copyright concerns, and potential bias in materials produced by the technology. 

Law societies and courts in other provinces and territories have also produced guidance on the use of AI. For instance, the Supreme Court of Yukon said in a June 2023 practice direction that if any lawyer relies on AI “for their legal research or submissions in any matter and in any form,” they must tell the court. 

For Black, with the firm DLA Piper, the use of AI is causing a lot of “necessary angst about relying on a tool like this to do any real heavy lifting.”

Black said delivering justice requires the impartiality of a “human peer,” capable of evaluating and making important legally binding decisions. 

He said he’s encountered lawyers and judges who are either “completely dialed into it, to completely averse to it, to completely agnostic to it.”

He said he’s been “impressed by the pace of the technology,” but the need for caution and skepticism around any materials generated by the material is essential for lawyers now and into the future. 

Reflecting on the Ke case and others like it, Black said tools like ChatGPT are “really good autocorrect tools that do a fantastic job of relating text to other text, but they have no understanding of the world, they have no understanding of reality.”

UBC law professor Kristen Thomasen said in an interview that the B.C. Supreme Court case shows not only the limitations of the technology, but also the need for lawyers and other professionals “to be critical of the technologies that they’re using.”

Thomasen said evaluating the strengths and weaknesses of technology has to be done “in spite of what is often a lot of hype.” 

She said it’s important not to delegate work that requires a human element to a computer system in “high stakes” professions like law and policing where new, potentially problematic technologies should be approached and employed with caution. 

Thomasen said the technology has been described as a “living thing” or an existential threat to humanity, or thought of as a “superhuman ghost in the machine,” but despite being highly sophisticated, it’s just doing math based on data scraped from the internet. 

She said that stepping back from seeing it as a “person” would help institutions, students and teachers better understand what the technology actually does. 

“As we see how it progresses, I think it makes sense to then, kind of like the law societies, keep developing more refined and detailed guidelines or rules as we gain a better understanding of what the technology looks like,” she said. 

The judge in the case that involved Ke said it would be “prudent” for her to tell the court and opposing lawyers if any other material employed AI technology like ChatGPT. 

“As this case has unfortunately made clear, generative AI is still no substitute for the professional expertise that the justice system requires of lawyers,” Justice David Masuhara wrote in his costs ruling. “Competence in the selection and use of any technology tools, including those powered by AI, is critical. The integrity of the justice system requires no less.” 

Black said artificial intelligence technology isn’t going away, and any rules developed now will likely need changing due to the “breakneck speed” of its evolution. 

“We are for sure now in a world where AI will exist,” he said. “There is no un-ringing this bell as far as I’m concerned.”

This report by The Canadian Press was first published Feb. 29, 2024.

Darryl Greer, The Canadian Press