Who Is Liable When AI Gives Bad Advice?

AI tools can produce remarkably convincing answers, even when the information behind those answers is inaccurate. For the growing number of lawyers, accountants, financial advisors and other professionals who incorporate AI into their work, this creates a significant risk-management challenge. If inaccurate AI output leads to bad advice, you could be held liable, and your professional liability insurance policy may not cover the resulting claim.

When AI Mistakes Have Real-World Consequences

Recent examples show how harmful AI advice can be.

• A supermarket in New Zealand used AI to generate meal plans, but according to The Guardian, the AI produced recipes that would have lethal results if used, including one that would create chlorine gas.
• New York City launched a chatbot to help business owners. However, according to Fast Company, the chatbot sometimes encouraged business owners to violate the law. In one example, the chatbot said it was fine to fire a worker who complained about sexual harassment. In another example, it told a restaurant it could serve cheese with rat bites, although it warned that the customers should be informed.
• The National Eating Disorders Association embedded a chatbot into its hotline. According to com, the chatbot recommended weight loss, counting calories and other actions that could be harmful to people with eating disorders.

How AI Can Lead to Professional Negligence

For professionals, there are multiple ways AI usage could result in poor advice.

• Client-facing AI platforms may deliver bad advice directly to clients. If you offer self-service AI tools to clients, there’s a risk that these tools will provide inaccurate or potentially harmful advice. For example, an AI-powered investment platform that helps users manage their portfolios could provide incorrect information about assets, resulting in poor investment decisions. Likewise, an AI-powered chatbot used by a tax advisor may give clients incorrect or misleading information regarding allowable deductions, resulting in improper deductions that trigger IRS audits.
• Professionals who rely on AI may pass bad advice on to clients. In theory, the “human in the middle” approach should reduce the risk of poor AI advice being passed on to clients. However, this only works if the professional assesses all outputs critically. In the absence of a careful review, inaccuracies and potentially harmful information may go unnoticed. For example, a lawyer who uses AI to generate wills, trusts and other important documents but doesn’t check the outputs carefully may end up providing documents that contain errors that could contribute to legal battles and family feuds. Likewise, financial advisors who use AI to create investment strategies could provide advice that does not match the clients’ financial needs.

Business leaders may decide that the advantages of AI outweigh the risks, but AI usage requires appropriate oversight. Organizations can reduce their risks by understanding where employees and client-facing systems are using AI, determining when human review is required and establishing clear standards for verifying AI-generated information before it influences professional services.

Businesses and Professionals May Be Held Liable for AI Outputs

You didn’t create the AI model that generated bad advice, but you may have taken on liability by using the AI. There have already been real-life cases of businesses and professionals being held accountable for AI outputs.

In one example, Air Canada lost a small claims court case after its chatbot provided incorrect information regarding the airline’s policies for bereavement fares. According to Forbes, the airline tried to argue that the chatbot provided a link to the correct information, but the tribunal ruled that the airline did not adequately explain that the information provided by the chatbot should not be trusted and the incident amounted to negligent misrepresentation. The airline had to pay $812.02 in damages and court fees. Although this is a small claim, it shows that businesses can be held responsible for the things AI chatbots say.

In another example, a lawyer was fined $5,000 and held in contempt of court after using AI to generate an appeal for his client’s murder conviction. According to The Verge, the appeal included AI-fabricated witnesses and policy testimony. This is not the first time a lawyer has landed in hot water for submitting legal documents littered with AI hallucinations – something the judge pointed out.

According to the 16^th Annual Lawyers’ Professional Liability Claims Survey from EPIC Law Firm Group, legal malpractice claims are increasing, and AI is part of the reason: seven out of 13 insurers polled say they’ve seen an increase in AI-related claims in the past year.

Professional Liability Insurance May Not Cover Every AI-Related Claim

If you’re sued for professional negligence stemming from AI usage, you may seek coverage under your professional liability insurance policy. Whether coverage is available will depend on the terms of your policy and the details of the claim.

First, there’s the question of what constitutes a professional service under the terms of the policy. When the bad advice originates from a client-facing automated platform or chatbot, the insurer may argue that the claim is not covered because it’s not connected to professional services provided by the insured.

Second, some insurers are starting to add exclusions, restrictions or sublimits for AI-related losses. Until recently, professional liability insurance policies were typically silent on the matter of AI, but this is changing quickly.

For professionals, uncertainty about insurance coverage can make AI mistakes even more consequential. Now is the time to understand where AI is being used across your organization and whether your risk-management practices and insurance coverage have kept pace. What AI tools are you using? What safeguards do you have in place? And how would your insurance policy respond to an AI-related claim?

Heffernan Insurance Brokers can help you review your insurance coverage, identify potential gaps and understand how your policies may respond to emerging AI risks. Learn more about AI liability.