AI Hallucination Goes to Court: A Global Mapping of Emerging Legal Issues
The following is a guest post on the FPF blog authored by Wenlong Li, Research Professor, Zhejiang University Guanghua Law School and 2026 PPPM Awardee. The guest post reflects the opinion of the author only and does not necessarily reflect the position or views of FPF and our stakeholder communities. FPF provides this platform to foster diverse perspectives and informed discussion.
AI hallucination was once treated largely as a quirk of generative systems: occasionally amusing, often embarrassing, and generally filed away as an unavoidable technical limitation. That framing is now wearing thin. Hallucinated content is no longer confined to private exchanges in which a model produces a bizarre or obviously implausible answer. Across a small but steadily growing body of disputes and regulatory complaints emerging by 2026, false or unsupported generated claims have begun to surface in search interfaces, customer-service chatbots, public-facing AI accounts, self-published media, professional commentary and data protection complaints. As generated text moves out of the private chatbox and into public, commercial and institutional settings, it is increasingly being pulled into law.
This blogpost takes stock of these emerging developments at a global level. In the absence of a centralised database or other comprehensive source (note that existing databases of AI hallucinations like here and here mainly document fake citations or unreliable AI-generated materials submitted in judicial proceedings which fall outside the scope of this blogpost), the cases discussed here were assembled through ongoing monitoring of legal and policy developments, targeted searches, and the routine tracking of professional and social-media feeds. This exercise does not amount to a systematic review in the strict academic sense, nor does it claim to have captured every relevant dispute. Its purpose is to map a fast-moving field and identify the legal fault lines along which AI hallucination is beginning to be framed as a governable problem.
Even with those limitations, the sample is sufficiently broad to show how legal systems are beginning to construct hallucination through different causes of action, theories of responsibility and remedial pathways. These cases raise recurring questions about attribution, reliance, verification, interface design, reputational injury and data accuracy, while exposing disagreement over which actors should bear responsibility when generated falsehoods are produced, adopted, repeated or allowed to circulate.
What is emerging from this early global landscape is a patchwork of responses across different legal fields, shaped by the interface through which the statement is delivered, the actor who deploys or adopts it, the nature of the resulting harm, and the remedial imagination of the relevant legal regime. This pluralism brings several doctrinal fault lines into view:
- private interaction versus public externalisation;
- user misuse versus institutional adoption;
- warning versus reasonable reliance;
- individual correction versus system-level intervention; and
- model output versus the design and governance of the surrounding service.
Seen in this light, hallucination governance encompasses model accuracy and the social and institutional placement of generated text. A hallucination becomes legally salient when it is externalised, amplified, institutionalised, relied upon or made difficult to challenge and correct. The emerging law of hallucination appears to ask who put the statement into circulation, how it was framed, what expectations of reliability were created, who was in a position to verify it and who should bear the consequences when those safeguards broke down.
For the purposes of this blog post, “AI hallucination” is used in a deliberately limited sense to describe machine-generated text that presents a verifiable factual claim as though it were grounded, despite that claim being false, unsupported or not reliably traceable to relevant evidence. The term remains contested and should not be taken literally – i.e., it does not attribute human perception, belief or deceptive intent to the model. Its usefulness lies not in attributing human mental states to machines, but in identifying a recurring class of outputs whose apparent factuality may invite reliance even where the underlying evidential basis is absent or defective.
Hallucination enters law through different doors
Before asking where AI hallucination belongs doctrinally, it is useful to consider how it is entering legal systems in practice. Five of the matters discussed arose in China, although that distribution should not be read as evidence that hallucination causes more disputes there than elsewhere. These cases cover a wider range of AI-mediated interactions than is often recognised in English-language discussions. They include AI-assisted commercial fabrication (（2024）浙0108民初10311号), reputational harms (（2025）粤0192民初23599号), and allegedly inaccurate answers generated in private conversations (（2025）浙0192民初18143号) or public-facing AI searches (（2025）苏01民终14271号). The Chinese cases involving public dissemination by users, taken together, have tended to foreground user-side responsibility: when a person generates, elaborates upon and publicly disseminates an AI-assisted fabrication, at what point should that person bear legal responsibility for failing to verify it?
Table 1. Overview of 11 judicial and regulatory matters concerning AI-generated misinformation. Dates identify the procedural events specified in each row. Allegations, preliminary rulings and final dispositions are distinguished; inclusion does not imply that liability was established.
China: When the user becomes the publisher
Two Chinese decisions place responsibility squarely on users who converted generated text into public-facing content. In (2024)浙0108民初10311号, a self-media operator prompted an AI system to write an article presenting an unrelated company as an Alibaba subsidiary, added Alibaba branding and published the article without a prominent AI-content label. The Hangzhou Binjiang District People’s Court held that the operator remained responsible for verifying the article and found, somewhat puzzlingly, that the conduct constituted unfair competition. In (2025)粤0192民初23599号, a finance commentator supplied an AI application with an unverified 1,700-Chinese-character document, asked it to expand the material into an article of more than 10,000 Chinese characters, and published the resulting allegations of corporate misconduct as “original”. The Guangzhou Internet Court found reputational infringement and stressed a heightened, full-chain duty of care arising from the publisher’s professional identity, control over the prompts and decision to disseminate the output.
China: When the provider answers for the answer
The provider-facing cases reached different outcomes on materially different facts. In (2025)浙0192民初18143号, an AI application gave a user inaccurate information about a college campus and then generated statements promising compensation if the answer proved wrong. The Hangzhou Internet Court rejected the claim: the generated compensation promise was not attributable to the provider as a binding expression of intention in the circumstances, the provider had given warnings and adopted reasonable mitigation measures, and the user proved neither reliance nor loss. By contrast, in (2025)苏01民终14271号, Baidu’s AI answer falsely associated an identifiable lawyer with imprisonment and an explosion-related offence. The Nanjing courts distinguished the synthesized AI answer from search suggestions and related-search terms, treated the former as a direct platform-generated presentation, and ordered Baidu to apologize for the reputational harm.
Germany: Output under an institutional banner
Several European and North American disputes likewise concern companies that incorporated conversational or text-generating systems into their services and presented the output under an institutional banner. Germany presently offers the clearest examples involving public-facing AI statements delivered with an air of factual authority. In the Google AI Overview proceedings (26 O 869/26), the Munich I Regional Court granted a preliminary injunction after an AI-generated search summary linked two publishers to fraud, subscription traps and other dubious practices. The court treated the summary as Google’s own content because it reorganised source material into a newly worded answer and asserted connections that the cited sources did not make. In Campact v X.AI LLC (re: Grok) (324 O 461/25), the Hamburg Regional Court restrained X.AI from repeating Grok’s false statement that Campact depended heavily on federal funding. Publication through the public @grok account made the output attributable to its operator. Both proceedings thus focus on the authority created by the provider’s design and presentation of generated statements, rather than on the fact of automation alone.
Canada: The chatbot as part of the service
Canada’s Moffatt v Air Canada, 2024 BCCRT 149, fits within a more service-oriented frame. The dispute did not concern public reputation or the processing of personal data, but customer reliance on incorrect information about bereavement fares supplied by a chatbot embedded in the airline’s official service environment. The chatbot told the passenger that he could apply retrospectively for a reduced bereavement fare after completing his journey; he relied on that representation, only for Air Canada subsequently to refuse the request on the basis that its actual policy required the application to be made before travel. The statement came through a channel that the company itself had put in front of its customers and invited them to use. Echoing issues that later surfaced in the German cases, the chatbot’s response appeared alongside links to the airline’s actual policy, yet the tribunal declined to treat the mere availability of accurate information elsewhere as sufficient to shift the burden of verification onto the consumer. Air Canada could not, in effect, disaggregate its chatbot from the rest of its website and insist that the customer should have worked out which of the company’s own contradictory representations was authoritative.
The case therefore turns, in part, on how verification costs should be distributed between platforms and users, and on what a reasonable average customer is entitled to take at face value when information is delivered through an organisation’s own interface. Although the system at issue should not necessarily be equated technologically with today’s large language model applications, the decision remains an important functional precursor, showing how established service and misrepresentation principles may allocate responsibility when an organisation automates communication without ensuring that the information supplied through that channel is reliable.
The U.S.: Doctrine tested
The emerging U.S. landscape illustrates the diversity of legal responses to AI hallucination, spanning defamation, trademark infringement, consumer protection and professional licensing, with courts and regulators examining how generated falsehoods acquire credibility, enter commercial relationships and become attributable to particular actors. Walters v OpenAI, 23-A-04860-2, illustrates the limits that established defamation doctrine can place on claims arising from generated falsehoods. The dispute arose after a journalist used ChatGPT in connection with litigation materials and received a fabricated account falsely stating that radio host Mark Walters had been accused of embezzling funds from a gun-rights organisation. Crucially, however, the journalist did not simply accept and republish the answer. He recognised that something was amiss, checked the underlying material and did not disseminate the allegation to a wider audience. When Walters subsequently sued OpenAI for defamation, the court declined to treat the deployment of a potentially fallible generative system as a basis for strict liability, instead filtering the alleged harm through the conventional requirements of defamation. Unlike the German AI Overview and Grok disputes, the hallucinated statement had not been disseminated through a public-facing search result or organisational social-media account or taken up as an authoritative answer; nor had an intermediary converted it into a wider publication. The decision therefore raises a broader distributive question of how much of the cost of verifying generated content should fall on the provider, the immediate user or the person who encounters the output. Its answer may be regarded as business-friendly, but it also illustrates how existing doctrine can hold back the impulse to treat every false AI-generated statement as actionable merely because it was produced by an AI system.
Four U.S. matters still in motion
Beyond Walters, at least four emerging legal developments merit brief attention. In Keene v Google, No. 25 C 11431, AI Overviews allegedly falsely described James Keene as serving a life sentence for murder, and the U.S. District Court for Northern District of Illinois allowed on 14 September 2026 claims concerning two Overviews to proceed while dismissing two others without prejudice, leaving liability undetermined. In Advance Local Media v Cohere, No. 25-cv-01305, publishers alleged that the service generated fabricated articles bearing their names and trademarks, and US District Court for Southern District of New York allowed the challenged claims to proceed, extending the attribution issues examined here to false publisher provenance. Pennsylvania’s action against Character Technologies, No. 220 MD 2026, concerns a chatbot that allegedly claimed to be a licensed psychiatrist and supplied a fictitious medical licence number, and the Commonwealth sought injunctive relief, but no substantive judicial ruling has been delivered. Like Air Canada, it concerns apparently authoritative service communications, approached here through professional licensing. Lastly, in In the Matter of Rytr, Docket No. C-4806, the Federal Trade Commission (FTC) challenged a service generating consumer reviews containing invented product details and experiences, initially obtaining a consent order (i.e., a binding legal agreement between opposing parties that is approved and signed by a judge or regulatory agency to resolve a dispute without a trial under US law) that it subsequently reopened on 22 December 2025. In light of the nature of these emerging developments comprising unresolved litigation and an administrative order subsequently withdrawn, thye are acknowledged here but remain outside the following comparative analysis, with their fuller implications left for future research.
The EU: Data protection as stress test
In the EU, the data protection complaints brought by noyb shift the centre of gravity – but very slowly. It is important to note, though, that these are complaints filed with Data Protection Authorities, still pending at the time of writing. In lieu of beginning with publication, reliance or reputational injury, they ask whether generated personal data that are demonstrably inaccurate can be reconciled with the requirements of the EU’s General Data Protection Regulation (GDPR) concerning accuracy, access, rectification and erasure. In C078, ChatGPT repeatedly produced an incorrect date of birth for an identifiable individual, while OpenAI was allegedly unable or unwilling to correct the underlying output in a way that would prevent the false information from recurring. The complaint therefore tests a deceptively simple proposition that if a controller cannot reliably correct inaccurate personal data, what does compliance with the GDPR’s accuracy and rectification principles mean in practice. C097 pushes the same problem into considerably more sensitive territory. There, ChatGPT allegedly generated a fabricated account portraying a Norwegian man as having murdered two of his children and attempted to kill a third, while combining those invented criminal allegations with some accurate information about his family. It illustrates how generative systems can produce highly damaging synthetic narratives by weaving together accurate and fabricated personal information in ways that are difficult for the data subject to anticipate or correct. The contrast between the two complaints is therefore analytically useful: C078 concerns something close to an ordinary biographical inaccuracy, whereas C097 concerns an allegation carrying obvious and potentially severe reputational, social and psychological consequences.
As it has done in other areas of data protection law, noyb is using strategic complaints to push regulators – and, eventually, courts – to spell out how data protection principles apply to a new technological setting. This intervention is especially significant as the regulatory position remains unsettled. The European Data Protection Board’s Opinion on AI models, alongside the report from its ChatGPT Taksforce of May 2024, addresses the application of GDPR principles to the development and deployment of AI models but leaves difficult questions about output-level accuracy, rectification and recurrence far from resolved. Earlier enforcement, including the Italian Garante’s action against OpenAI in 2023, likewise brought data accuracy into view without settling how a controller should respond when a generative model can reproduce an inaccurate claim even after that claim has been challenged. Thus, the noyb complaints serve as a useful stress test for a regulatory framework designed around more conventional forms of personal-data processing, asking whether rights of correction and erasure remain meaningful when the object to be corrected is a probabilistic capacity to generate the same falsehood again.
Hallucination has no native legal home
As this global mapping shows, courts, tribunals, complainants and regulators are translating a related set of problems involving false or unsupported AI-generated factual claimsinto a range of inherited legal categories, including defamation, unfair competition, negligent misrepresentation, personality rights, data accuracy, trademark, profesional licensing, consumer protection, search and platform responsibility and so on. Each of these frameworks catches hold of one dimension of the problem, but none fully contains it. A fabricated criminal allegation may give rise to a defamation claim. An invented corporate affiliation may be taken up through unfair competition law. An inaccurate fare policy supplied by an airline chatbot can be framed as negligent misrepresentation or service-provider liability. A false date of birth generated about a public figure raises questions of inaccurate personal data. An AI-generated search summary that associates a company with fraud may trigger claims for injunctive relief grounded in personality rights.
The breadth of the harms associated with hallucinated content therefore raises a wider regulatory question: whether existing legal frameworks can be adapted sufficiently to govern the phenomenon, or whether more targeted forms of intervention will eventually be required.More critically, each legal frame brings with it a different account of what has gone wrong, who has been harmed, which actor bears responsibility and what form of remedy should follow. Whereas defamation law centres reputational injury, data protection law currently focuses on the accuracy and correction of personal data, with the potential to bring a broader set of commitments to bear, such as fairness, data subject rights (notably rectification and erasure), as well as how the obligations under Article 25 GDPR implement those principles through data protection by design. Misrepresentation turns on reliance, and unfair competition law looks to market conduct and commercial harm. Platform and service-liability regimes shift attention towards system design, deployment and control.
The choice of legal category thereby shapes the outcome of a dispute and the way hallucination itself is understood At this stage of legal development, AI hallucination is best understood as a family of problems arising from the interaction between model design, interface design, user behaviour, institutional authority and downstream reliance. In other words, what turns a technical failure into a legal problem is often the chain of decisions through which an unreliable output is surfaced, endorsed, acted upon or left uncorrected.
Hallucination becomes legally contestable when it travels and is taken up
The jurisdictional comparison above shows that different legal systems are encountering hallucination through different factual and legal settings. Beneath that geographical variation, however, a more coherent pattern begins to emerge. The legal significance of hallucination depends less on falsity alone than on two related questions: how far the generated content travels, and the extent to which an institution has taken it up as part of its own service or public voice.
Figure 1. Clustering of the AI hallucination cases based on externalisation and institutional/individual adoption
Axis one: How far the output travels
To bring that pattern into view, the cases can be organised along two axes. The first is the degree of externalisation, that is, whether an output remains within a relatively private interaction between a user and a model, or is pushed outwards through publication, republication, search results, social-media accounts or other forms of public circulation. It captures a consequential point that hallucination becomes more legally volatile once it leaves the chatbox. A false answer encountered and immediately rejected by one user is not equivalent to the same answer being repackaged as an investigative article, attributed to a professional commentator, displayed in response to a public search query or broadcast through an official organisational account. Apart from enlarging the audience, publication can also change the output’s social status. The Chinese self-media cases consequently occupy the high-externalisation, high-user-adoption area of the figure: the users selected generated material and presented it as substantive content through their own publications. Their adoption supplied an additional basis of apparent authority, whether through professional identity, branding or an “original” label. This differs from circulating an output merely to question, criticise or expose its unreliability.
Axis two: Whether an institution takes it up
The second axis concerns adoption by the communicating actor, whether an individual user or an organisation: the extent to which generated content is presented as that actor’s own substantive communication. Relevant indications include editorial selection, branding, professional presentation and integration into an official service channel. Technical control over a system is relevant context, but does not alone establish adoption of every answer it produces. Nor does a model’s invocation of another organisation’s name or professional credentials establish endorsement by that organisation or profession. Air Canada, Baidu, Google and X all raise variations of the same question: when does generate text cease to look like an answer produced for a user and begin to function as part of an institution’s own communicative infrastructure?
The search cases are particularly revealing because they expose a transition from retrieval to generation. Traditional search engines could plausibly describe themselves as organising access to third-party material. Features such as autocomplete (albeit addressed only briefly in the Google AI Overview case), snippets and related searches had already complicated that account, but AI-generated answers bring the difficulty fully into the open.
As the Munich I Regional Court put it (in para 43), once a search provider synthesises source material into a newly worded answer, structures it into a conclusion and places it prominently above or alongside conventional results, it becomes harder to maintain that the provider is merely pointing the user elsewhere. Quite the opposite, it was considered to be the speech of the AI providers! The Hamburg court’s approach to Grok further brings out a related point. The AI-generated nature of the statement did not necessarily weaken its apparent factual force. Indeed, the court held (in para 6) that presenting the service as fact-based—and publishing its response through an official account—could strengthen the authority that an ordinary user attributed to the answer. Hence, AI providers may not be the ‘authors’ in a conventional literary sense, but they have designed the answer layer, controlled its presentation and benefited from making the response appear immediate and useful. In both settings, the interface transformed fragments of information into a new communicative act.
Two one-to-one interactions, two outcomes
The cross-jurisdictional comparison between Moffatt v Air Canada and the Hangzhou’s campus-information case also shows the limits of institutional adoption as a stand-alone basis for liability. In both disputes, the inaccurate statement remained largely within a one-to-one service interaction and was delivered through an interface controlled by the provider. Yet the relationships surrounding those outputs were materially different. Air Canada’s chatbot purported to explain the airline’s own fare policy through an official customer-service channel, and the passenger relied on that information when arranging travel. In the Hangzhou case, by contrast, the service generated inaccurate information about university locations and later produced compensation-like statements, but the user detected the errors before acting upon them; the court found neither a binding promise nor an established loss and also took account of the provider’s warnings and mitigation measures. The divergence thus turns on whether the provider controlled or adopted the interface, whether the service created a sufficiently concrete and legally protected basis for reliance, the consequences that followed from that reliance, and the technical nature of the system itself. Moffatt concerned an automated chatbot that should not necessarily be treated as equivalent to a contemporary generative-AI system, whereas the Hangzhou dispute arose from the open-ended and probabilistic behaviour associated with generative models—a distinction that may shape both what users can reasonably expect and what precautions providers can realistically be required to take.
Walters v OpenAI lies closer to the opposite corner. The false statement arose within a limited interaction, was recognised as unreliable and was not passed on to a wider audience. Traditional defamation doctrine could therefore resist the attempt to convert the mere availability of a fallible system into strict liability. The case does not establish that private hallucinations are legally irrelevant but shows that falsity becomes more difficult to translate into a cause of action where the output has neither travelled nor been institutionally adopted in a manner that produces reliance or legally cognisable harm.
Where the quadrants run out
It is critical to point out that these axes do not automatically determine liability: low externalisation does not rule liability out, just as a high degree of provider control does not automatically make every false answer actionable. Their value lies instead in showing how an initially probabilistic model output can be transformed into a statement for which the law may expect someone to answer. The newer US proceedings also expose limits to a simple quadrant classification. Keene concerns generated statements presented within a public-facing search service, whereas Cohere raises the distinct problem of apparent authority borrowed from publishers through false attribution. The Pennsylvania proceeding concerns asserted professional authority within a conversational encounter, with attribution to the provider remaining contested. Rytr concerns the provision of functionality for possible downstream use, rather than a single established trajectory of publication and adoption. The figure therefore distinguishes the presentation of an output from a legal determination that the actor presenting it bears responsibility.
How much the falsehood costs
A further variable cuts across all four quadrants: the sensitivity and foreseeable harmfulness of the generated content. A wrong date of birth, an invented university campus and a fabricated criminal conviction are all factual inaccuracies, but they do not carry the same capacity to damage reputation, distort consequential decisions or resist later correction. The difference between the Nanjing and Hangzhou cases, as well as between noyb’s ordinary biographical-accuracy complaint and its complaint concerning a fabricated allegation of child murderer, suggests that legal expectations may become more demanding as the output grows more sensitive. Duties of verification, intervention and correction may therefore be calibrated not only according to how far a statement travels and who has adopted it, but also according to the gravity of what it alleges.
Above all, hallucination is unlikely to be engineered away (in the short run at least) before generative systems are woven more deeply into the infrastructures through which people search, transact, communicate and make consequential decisions. The law will therefore have to govern a persistent condition of probabilistic systems deployed in institutions that trade on authority and trust. It might be gleaned from these comparative accounts that the legal risk associated with text-to-text hallucination does not arise primarily from the existence of error inside the model. It arises when social and institutional mechanisms amplify that error. Ranking systems, interface design, professional status, account identity, “original” labels, product integration and commercial dissemination can dress probabilistic language up as a determinate factual assertion. They turn generated text into something that appears worth believing, relying upon or passing on. These emerging cases offer no single answer, but they point towards a recurring consideration, i.e., that the more an actor makes generated text useful, authoritative and difficult to distinguish from an institutional statement, the less plausible it becomes to fall back on the claim that the machine simply made it up.