LLM Hallucinations and Algorithmic Defect: Product Liability under the Consumer Protection Act 2019

I. The Technological Reality of Stochastic Fabrication in Large Language Models

Large Language Models (LLMs) such as GPT-4, Claude, Gemini, and proprietary enterprise architectures do not retrieve verified factual truths from curated knowledge graphs. At their computational core, these systems are autoregressive neural networks trained to predict the next most probable token in a sequence based on statistical weight distributions learned during pre-training. Consequently, when an LLM is queried on complex legal, medical, or financial matters, it frequently synthesizes fluent, highly persuasive, and authoritative assertions that are entirely devoid of factual truth—a phenomenon scientifically designated as 'hallucination'.

While AI developers historically characterized hallucinations as harmless research eccentricities, the commercial reality of 2026 presents catastrophic legal risks. LLMs are now directly integrated into commercial consumer products, automated legal research databases, clinical decision support software, and consumer-facing banking portals. When an enterprise chatbot fabricates a non-existent airline refund policy, recommends a toxic pharmaceutical dosage, or cites fictitious judicial rulings, the resulting injury causes tangible financial, physical, and reputational damage.

In Indian jurisprudence, the critical legal threshold has arrived: Can Large Language Models be categorized as 'products' under statutory product liability regimes? And does the probabilistic generation of false or harmful information constitute an actionable 'defect' under the Consumer Protection Act, 2019? This treatise provides an authoritative examination of algorithmic defect, developer strict liability, and judicial accountability.

II. The Statutory Architecture of Product Liability under CPA 2019

Prior to 2019, Indian product liability law was a fragmented patchwork of common law tort principles, contractual warranties under the Sale of Goods Act 1930, and basic consumer negligence doctrines. The enactment of the Consumer Protection Act, 2019 (CPA 2019) fundamentally revolutionized this landscape by dedicating an entire statutory chapter—Chapter VI (Sections 82 through 87)—to strict statutory product liability.

Under Section 2(34) of the CPA 2019, a 'product' is defined as any article or goods, substance or raw material or any extended cycle of such product, which may be in gaseous, liquid, or solid state, possessing intrinsic value, and produced or manufactured for commercial exchange. While traditionalists argue that software is an intangible service, Indian commercial jurisprudence—anchored in the landmark Supreme Court ruling in *Tata Consultancy Services v. State of Andhra Pradesh* (2005 1 SCC 308)—has firmly settled that computer software, whether canned, branded, or customized, constitutes 'goods' and therefore qualifies as a statutory product.

Section 2(10) defines a 'defect' as any fault, imperfection, or shortcoming in the quality, quantity, potency, purity, or standard which is required to be maintained by or under any law for the time being in force or under any contract, express or implied. Under Section 84, a product manufacturer is held strictly liable in a product liability action if the product contains a manufacturing defect, a design defect, a deviation from manufacturing specifications, or fails to contain adequate instructions or warnings for correct usage.

III. Deconstructing Algorithmic Design Defect and Inadequate Warnings

In an AI product liability claim, the plaintiff is not required to prove traditional negligence or malicious intent (mens rea). Liability under Chapter VI of the CPA 2019 is strict. To establish an 'algorithmic design defect' under Section 84(1)(b), the claimant must demonstrate that the LLM architecture possessed intrinsic structural vulnerabilities that rendered it unreasonably dangerous or prone to hazardous fabrications under foreseeable consumer usage.

AI laboratories frequently attempt to evade design defect liability by arguing that hallucinations are mathematically inevitable byproducts of transformer architectures. However, from a statutory perspective, marketing a machine that is inherently incapable of distinguishing truth from statistical probability—without integrating deterministic truth-grounding mechanisms such as Retrieval-Augmented Generation (RAG) or semantic guardrails—constitutes a prima facie design defect. If a safer alternative design was technologically feasible and the developer failed to implement it, design liability is established.

Furthermore, under Section 84(1)(e), a manufacturer is liable if the product fails to contain adequate instructions or warnings. Vague, boilerplate disclaimers buried deep within user terms—such as 'AI may make mistakes'—do not satisfy statutory warning standards when the model is simultaneously marketed as an expert legal, medical, or corporate advisory tool. Warnings must be prominent, contextual, and commensurate with the gravity of foreseeable harm.

IV. Professional Negligence and Vicarious Liability for Algorithmic Tools

A parallel dimension of liability arises when licensed professionals—such as advocates, chartered accountants, and medical practitioners—rely on hallucinating AI models in discharging their professional duties. In legal practice, presenting hallucinated case citations before an Indian High Court or the Supreme Court constitutes professional misconduct under the Advocates Act 1961 and contempt of court. A practicing advocate owes an independent, non-delegable duty to the court to verify the authenticity of every statutory citation.

In the medical domain, if a physician utilizes an AI diagnostic tool that hallucinates a benign diagnosis for an aggressive malignancy, the practitioner cannot shift sole culpability onto the software vendor. Under the *Jacob Mathew v. State of Punjab* doctrine, the standard of care required of a medical professional is that of a skilled practitioner possessing ordinary competence. Blind reliance on an unverified algorithmic recommendation constitutes actionable medical negligence under Section 106 of the Bharatiya Nyaya Sanhita 2023 and the Consumer Protection Act.

However, where the professional exercises reasonable care but is misled by an opaque, latent software defect that could not be detected through ordinary diligence, a secondary right of indemnification and product liability claim arises directly against the AI software developer under Section 84 and Section 86.

V. Statutory Defenses and the Limits of 'State-of-the-Art' Immunity

Section 87 of the CPA 2019 enumerates specific statutory exceptions to product liability. A product manufacturer cannot be held liable if the product was misused, altered, or modified by the consumer, or if the product was used in a place or in a manner that was contrary to clear and express warnings. AI developers invariably invoke these provisions, asserting that prompt engineering by the user constituted an unauthorized modification or misuse of the model.

In global tort law, developers frequently raise the 'State-of-the-Art' defense, asserting that the state of scientific and technical knowledge at the time the model was released was not such as to enable the existence of the defect to be discovered. However, unlike certain European liability directives, the Indian Consumer Protection Act 2019 does not contain an express, broad state-of-the-art statutory exemption. If a product causes personal injury, property damage, or financial loss due to a demonstrable defect, strict liability attaches regardless of whether the defect represented the frontier of global research.

Moreover, developers who deploy AI agents with autonomous execution capabilities—such as automated trading bots or direct database-modifying agents—cross the threshold from informational tools to autonomous commercial actors. When an agent acts on a hallucination to execute an unauthorized transaction, the developer and deployer face joint and several liability under both contract and tort principles.

VI. Chambers Advisory: Algorithmic Product Safety and Consumer Litigation

Advocate Ritik Khurana Chambers provides specialized litigation counsel and risk advisory at the intersection of consumer protection law and artificial intelligence engineering. For enterprise AI developers and SaaS providers, our chambers structures comprehensive Product Safety Protocols, designs legally robust tiered warning architectures, and conducts pre-deployment liability audits under CPA 2019 standards.

For corporate enterprises and consumers who have suffered catastrophic financial, reputational, or physical injury as a consequence of defective algorithmic outputs, our trial practice initiates statutory product liability claims before the National Consumer Disputes Redressal Commission (NCDRC) and State Commissions, securing substantial compensatory damages and mandatory recall orders.

As generative systems become deeply embedded in societal infrastructure, developers must recognize that legal liability is not an abstract concept; it is the ultimate statutory guarantee that artificial intelligence serves human welfare with uncompromising accuracy and accountability.