Reliability

'Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools' by Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning and Daniel E. Ho in (2025) Journal of Empirical Legal Studies comments 

 In the legal profession, the recent integration of large language models (LLMs) into research and writing tools presents both unprecedented opportunities and significant challenges (Kite-Jackson 2023). These systems promise to perform complex legal tasks, but their adoption remains hindered by a critical flaw: their tendency to generate incorrect or misleading information, a phenomenon generally known as “hallucination” (Dahl et al. 2024). 

As some lawyers have learned the hard way, hallucinations are not merely a theoretical concern (Weiser and Bromwich 2023). In one highly publicized case, a New York lawyer faced sanctions for citing ChatGPT-invented fictional cases in a legal brief (Weiser 2023); many similar incidents have since been documented (Weiser and Bromwich 2023). In his 2023 annual report on the judiciary, Chief Justice John Roberts specifically noted the risk of “hallucinations” as a barrier to the use of AI in legal practice (Roberts 2023). 

Recently, however, legal technology providers such as LexisNexis and Thomson Reuters (parent company of Westlaw) have claimed to mitigate, if not entirely solve, hallucination risk (Casetext 2023; LexisNexis 2023b; Thomson Reuters 2023, inter alia). They say their use of sophisticated techniques such as retrieval-augmented generation (RAG) largely prevents hallucination in legal research tasks. ... 

However, none of these bold proclamations have been accompanied by empirical evidence. Moreover, the term “hallucination” itself is often left undefined in marketing materials, leading to confusion about which risks these tools genuinely mitigate. This study seeks to address these gaps by evaluating the performance of AI-driven legal research tools offered by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-­Assisted Research and Ask Practical Law AI) and, for comparison, GPT-4. 

Our findings, summarized in Figure 1, reveal a more nuanced reality than the one presented by these providers: while RAG appears to improve the performance of language models in answering legal queries, the hallucination problem persists at significant levels. To offer one simple example, shown in the top left panel of Figure 2, the Westlaw system claims that a paragraph in the Federal Rules of Bankruptcy Procedure (FRBP) states that deadlines are jurisdictional. But no such paragraph exists, and the underlying claim is itself unlikely to be true in light of the Supreme Court's holding in Kontrick v. Ryan, 540 U.S. 443, 447– 48 & 448 n.3 (2004), which held that FRBP deadlines under a related provision were not jurisdictional. 

We also document substantial variation in system performance. LexisNexis's Lexis+ AI is the highest-performing system we test, answering 65% of our queries accurately. Westlaw's AI-Assisted Research is accurate 42% of the time, but hallucinates nearly twice as often as the other legal tools we test. And Thomson Reuters's Ask Practical Law AI provides incomplete answers (refusals or ungrounded responses; see Section 4.3) on more than 60% of our queries, the highest rate among the systems we tested. ... 

Our article makes four key contributions. First, we conduct the first systematic assessment of leading AI tools for real-world legal research tasks. Second, we manually construct a preregistered dataset of over 200 legal queries for identifying and understanding vulnerabilities in legal AI tools. We run these queries on LexisNexis (Lexis+ AI), Thomson Reuters (Ask Practical Law AI), Westlaw (AI-Assisted Research), and GPT-4 and manually review their outputs for accuracy and fidelity to authority. Third, we offer a detailed typology to refine the understanding of “hallucinations,” which enables us to rigorously assess the claims made by AI service providers. Last, we not only uncover limitations of current technologies, but also characterize the reasons that they fail. These results inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains an important open question for the responsible integration of AI into law.  

The rest of this work is organized as follows. Section 2 provides an overview of the rise of AI in law and discusses the central challenge of hallucinations. Section 3 describes the potential and limitations of RAG systems to reduce hallucinations. Section 4 proposes a framework for evaluating hallucinations in a legal RAG system. Because legal research commonly requires the inclusion of citations, we define a hallucination as a response that contains either incorrect information or a false assertion that a source supports a proposition. Section 5 details our methodology to evaluate the performance of AI-based legal research tools (legal AI tools). Section 6 presents our results. We find that legal RAG can reduce hallucinations compared to general-­purpose AI systems (here, GPT-4), but hallucinations remain substantial, wide-ranging, and potentially insidious. Section 7 discusses the limitations of our study and the challenges of evaluating proprietary legal AI systems, which have far more restrictive conditions of use than AI systems available in other domains. Section 8 discusses the implications for legal practice and legal AI companies. Section 9 concludes with implications of our findings for legal practice.

Magna

In Ponzio v Ashley Services Group Limited [2025] FedCFamC2G 289 the Court addressed claims by Ponzio, stating 

In relation to Magna Carta the Court observes (adapting slightly what it said in Ledger Acquisitions Australia MB Pty Ltd v Kiefer [2014] FCCA 2216 at [59]- [66] per Judge Lucev) that: 

(a) the Charter of Magna Carta was signed by King John of England “in the meadow that is called Runnymede between Windsor and Staines” on 15 June 1215. Only three of the original clauses remain in effect in the United Kingdom, one of those clauses being cl 39 which became cl 29 in 1225. The current enactment dates from 1297; 

(b) the Immigration Restriction Act 1901 (Cth) (“Restriction Act”) was challenged in Chia Gee & Ors v Martin [1905] HCA 70; (1905) 3 CLR 649; (1905) 12 ALR 425 (“Chia Gee”) as “unconstitutional, on the basis that its provisions were contrary to the provisions of Magna Charta and the Statutes which had since confirmed it”: Chia Gee, CLR at 652-653 per Griffith CJ. Sir Samuel Griffith, the first Chief Justice of the High Court of Australia, and arguably the principal drafter of what became the Constitution of the Commonwealth of Australia (“Constitution”), brooked no argument on this contention, dismissing it in a single sentence: “The contention that a law of the Commonwealth is invalid because it is not in conformity with Magna Charta is not one for serious refutation”: Chia Gee, CLR at 653 per Griffith CJ The other two initial Justices of the High Court of Australia, Barton and O’Connor JJ, contented themselves with concurring with the Chief Justice: Chia Gee, CLR at 654 per Griffith CJ. Barton J, who was the first Prime Minister of the Commonwealth, and O’Connor J, were both involved in the Constitutional conventions which led to the drafting of the Constitution, Barton extensively so. Such was the authority of the first three Justices of the High Court of Australia that no more needed to be said; 

(c) in Ex parte Walsh and Johnson; in re Yates [1925] HCA 53; (1925) 37 CLR 36 (“Walsh and Johnson”), also a case concerning the Restriction Act, Isaacs J discussed the Constitutional significance of Magna Carta in an Australian context. Referring to cl 29 of Magna Carta Isaacs J said, Walsh and Johnson, CLR at 79, that:

The chapter, ... recognises three basic principles, namely, (1) primarily every free man has an inherent individual right to his life, liberty, property and citizenship; (2) his individual rights must always yield to the necessities of the general welfare at the will of the State; (3) the law of the land is the only mode by which the State can so declare its will. 

(d) Isaacs J recognised that personal liberty and property give way to a declaration by the State (in this case the Commonwealth) of the law of the land, observing, Walsh and Johnson, CLR at 79, that: These principles taken together form one united conception for the necessary adjustment of the individual and social rights and duties of the members of the State. 

(e) in Skyring v Federal Commissioner of Taxation (1991) 23 ATR 84; (1991) ATC 4028 (“Skyring”) the Full Court of the Federal Court of Australia, faced with a contention that Commonwealth tax legislation violated property rights secured by Magna Carta, held that the power conferred on the Commonwealth Parliament by the taxation power in s 51(ii) of the Constitution, to legislate with respect to taxation, extends to the imposition of taxation and its collection, even though it has the effect of requiring the person on which taxation is levied to pay the tax out of property which he owns: Skyring, ATR at 87 per Gummow, Einfeld and Heerey JJ; 

(f) in Arnold & Anor v State Bank of South Australia & Ors (1992) 38 FCR 484 (“Arnold”) the appellants sought to attack a mortgage on the basis that the debt secured by the mortgage involved the creation by the respondent bank of a book entry credit at no cost to itself. Magna Carta was invoked as guaranteeing the rights of the appellants to their matrimonial home and livelihood. Challenges were also made on the basis of passages from the Bible, and in particular those striking at usury. The Full Court of the Federal Court of Australia, in dismissing the appellants’ appeal, did not specifically refer to Magna Carta in its reasoning, but approved what had been said in two recent cases before single Judges of the Federal Court: Arnold at 485-486 per Burchett, Hill and Drummond JJ, including in Fisher & Anor v Westpac Banking Corporation & Ors (unreported, Federal Court of Australia, No 64/92, 18 August 1992) (“Fisher”) 

(g) in Fisher the plaintiffs sought to set aside a claim made by a bank under a mortgage to their matrimonial home on the basis that the matrimonial home was guaranteed not to be abrogated from or interfered with by anyone by reason of authority derived ultimately from Magna Carta. Similar pleas were also made by reference to biblical authority. In the Federal Court of Australia French J, like the first Chief Justice of the High Court of Australia in Chia Gee, dismissed the plea by reference to Magna Carta in a single sentence, Fisher at p 15 per French J, as follows:

In relation to the remaining pleas based on the Magna Carta and the Bible, it is sufficient to say they disclose no legally tenable cause of action. It follows that Magna Carta has no apparent operation in the circumstances of this case, the law of the land in relation to the unlawful termination action being that set out in the FW Act, and to the extent that this issue concerns the process and procedure of this Court, then as set in the FCFCA Act and the GFL Rules.