AI Liability: Managing Privacy Risks and Confidential Data Leaks

Is AI exposing your company to privacy and data risks? When a company or its employees provide sensitive information to AI systems without appropriate safeguards, data breaches, leaks and unauthorized disclosures are possible. AI tools may also infringe on the privacy of employees or clients by collecting and storing data without consent. As companies integrate AI more deeply into their operations, these risks can create regulatory, financial and reputational consequences that business leaders should not overlook.

AI Tools Can Expose Confidential Data

To get a good AI output, you need a strong prompt. Sometimes this requires detailed information or even uploading documents. If the information provided to the AI is sensitive or confidential, data privacy becomes a concern.

Breaches and leaks have already occurred. In one incident, PCMag says Claude chats appeared in Google search results for a short period. In another, Bitdefender reports that some ChatGPT users inadvertently made shared conversations searchable through Google after enabling a discoverability feature. OpenAI subsequently removed the feature, citing the risk that users could unintentionally share information they did not mean to make public.

For businesses and professionals, unintended disclosures like these could cause serious problems. Consider the following scenarios:

• A programmer is writing code for a confidential project. He wants to get AI feedback on new code, so he describes the project and uploads the code to the AI. A glitch in the AI platform allows other users to see his chat logs, potentially exposing the company’s trade secrets.
• A doctor is having difficulty diagnosing a patient. She enters all the information into an AI program and gets good suggestions on possible diagnoses. However, the chat is made public, exposing personal and identifiable details about the patient. The disclosure exposes protected health information, potentially triggering HIPAA compliance and liability concerns.

When AI Use Creates Privacy Concerns

Some AI tools collect and store data as part of their functions. If this occurs without the consent of the people involved, privacy becomes an issue.

Consider the following scenarios:

• A company installs an AI-powered monitoring system that uses facial recognition or other biometric identifiers to track employees. Employees allege that the company collected biometric information without the notices and consent required by applicable law.
• A doctor uses AI to summarize appointments and assist with billing and electronic health records. When patients realize AI was listening to and recording their appointments without their knowledge, they argue it’s a privacy violation.

Some lawsuits have already occurred. According to Carey & Associates, P.C., drivers of vehicles equipped with Lytx monitoring technology alleged that the system used machine vision and AI to collect biometric information in violation of the Illinois Biometric Information Privacy Act. Lytx denied wrongdoing but agreed to establish a $4.25 million settlement fund.

Meanwhile, Mobile Health News says a patient has filed a proposed class action lawsuit against a health system that he says recorded a medical visit without consent, in violation of California privacy laws.

Controlling Data and Privacy Risks When Using AI

Data and privacy concerns do not need to prevent businesses from moving forward with AI, but the risks do deserve attention.

• Conduct an inventory of your current AI usage. It’s important to know what you’re using and how you’re using it so you can determine what your current exposures are and whether you have sufficient controls in place.
• Use private, secure AI platforms. Before providing confidential information to an AI platform, understand how it stores, processes and uses your data, including whether inputs may be used for model training.
• Create a written policy on AI employee use. Even if your company only uses a secure platform, your employees may be putting company data at risk by using public AI systems without approval. To prevent this, create a clear written policy that includes which programs employees can use, what information they can enter or upload, when approval is required and the consequences of noncompliance.
• Review relevant data privacy laws. Being familiar with applicable privacy laws will help you avoid accidental violations. Consider whether laws in other states apply, for example, because you have employees or customers there.
• Evaluate AI vendors. Before adopting an AI tool, review the vendor’s security and privacy practices. Understand what data the vendor collects, where it is stored, who can access it, how long it is retained and whether subcontractors or other third parties may have access.
• Secure consent when needed. If you are using AI to monitor employees or record interactions with clients, determine whether consent is needed under applicable privacy laws. Seek legal guidance if you are uncertain.
• Review your insurance. If your business faces a claim involving AI-related privacy violations or data disclosure, will your existing insurance respond? Don’t assume it will without reviewing the policy language. Depending on the circumstances, coverage may potentially exist under cyber, professional liability or other policies, while AI-specific exclusions, sublimits or other restrictions may narrow coverage.

As AI becomes more integrated into business operations, privacy and data exposures are likely to evolve. Reviewing your insurance program can help you understand where your coverage may apply and where potential gaps remain. Heffernan Insurance Brokers can help you evaluate your existing coverage to determine whether additional coverage may be warranted. Learn more about AI liability.