AI ‘gone awry’ made up case law, a Manitoba court says. Justice experts say it highlights a lack of access

At first, the appeal court filing may have seemed pretty standard.

A Manitoba woman who had a ruling against her in a foreclosure case was challenging the judge’s decision with the province’s top court, laying out her arguments and supporting legal cases.

But when the bank’s lawyers tried to look up some of those cases, they couldn’t find them. In their response, they said the Brandon, Man., woman’s filing misquoted paragraphs, improperly summarized decisions and, in some cases, relied on case law “that does not exist.”

“The materials … appear to be generated by some form of artificial intelligence gone awry,” the Manitoba Court of Appeal said in dismissing the woman’s case this month.

While cases of generative AI “hallucinating” fake case law when used for legal documents have come up in other jurisdictions, including Ontario and British Columbia, Manitoba hasn’t seen the same kinds of headlines.

But CBC has learned the Brandon woman’s case isn’t the only one in the province.

The Law Society of Manitoba says it’s also dealt with a lawyer who was put under supervision after being caught using AI that put forward unreliable case law. That case involved personal “extenuating circumstances,” executive director Leah Kosokowsky said.

Though cases of lawyers misusing AI exist, one expert says the cases he’s tracked across Canada largely involve people representing themselves.

Tom Macintosh Zheng, a Toronto-based lawyer and co-founder of Courtready — a website that offers tools including a national database of cases involving AI-hallucinated case law — said more than 80 per cent of cases in the database involve self-represented parties, like the woman in Brandon.

Experts like Macintosh Zheng say they aren’t surprised the issue continues to show up in courts across Canada, especially among self-represented people, in part because of how easy it is to use AI chatbots.

“I don’t want to overstate it,” the law society’s Kosokowsky said, but “I expect that it will increase.”

But they say the case also highlights a larger issue with access to justice, with barriers like the cost of hiring a lawyer leading regular people to turn to AI to help prepare sometimes complex legal documents — potentially without realizing the risks.

Manitoba prosecutor Ben Wickstrom says services like Legal Aid are under-resourced, creating another barrier.

“The system, unfortunately, often ends up relying on people to represent themselves,” said Wickstrom, who is also vice-president of prosecutions with the Manitoba Association of Crown Attorneys.

“They’re looking for ways to make the best argument for themselves. And there’s this technology that’s out there that is sort of purporting to be able to put it all together.”

In the Brandon foreclosure case, the woman said preparing her appeal was “challenging due to my self-represented status and lack of legal guidance.”

“The legal clinic I usually rely upon does not have lawyers trained in judgment appeals, and there are no resources to assist self-represented litigants with the appeal process,” she wrote in an affidavit.

More education needed

AI has “been sold to us as this do-everything tool,” said Winnipeg defence lawyer Christopher Gamby, but the Brandon case shows the risks of not double-checking its work.

“At the very least, I think that it’s going to throw some cold water on whatever argument you’re trying to make, if there’s a bunch of false citations in there,” said Gamby, who is also communications director for the Criminal Defence Lawyers Association of Manitoba.

Experts say the phenomenon also puts a strain on lawyers and courts already struggling with heavy caseloads. In the Brandon case, the appeal court said the bank’s lawyers “spent needless time and energy trying to verify the cases cited.”

Manitoba’s Court of King’s Bench issued a practice direction requiring disclosure of AI use in 2023 and has developed an internal AI committee to develop guidelines for use. But a Manitoba Courts spokesperson said the provincial and appeal courts are still “in the process of exploring a response to the issues around the use of AI.”

Lawyer Macintosh Zheng said the Brandon case shows more needs to be done to address the problem — and while some people across Canada have been punished for misusing AI in court, he thinks the way forward is clear.

“The data shows that despite the courts sanctioning folks, and putting out rule after rule and practice direction after practice direction, the trend has not stopped. It has, in fact, accelerated,” Macintosh Zheng said.

“And so for the Manitoba courts, it is time to start educating folks about what the risks are.”