Can AI Companies See Your Private Conversations? The Complete Guide

Can AI Companies See Your Private Conversations?

Yes. OpenAI, Google, and Anthropic all use human reviewers to read portions of user conversations for model improvement and safety monitoring. OpenAI’s Project Lily sends chats to hundreds of contractors. Google retains reviewed Gemini chats for up to three years. Your conversations are not confidential, and opting out only prevents future—not past—review.

Quick Facts

The Direct Answer: Yes, With Specifics

Your ChatGPT conversations may be read by hundreds of outside contractors. Google Gemini confirms that human reviewers read a subset of chats, and those reviewed conversations are retained for up to three years. Anthropic states that no personnel can read your retained conversations by default—but human review can occur when content is flagged by automated safety systems.

Read moreIs AI Taking Your Job? Data Says No (Mostly)

This is not a data breach. It is a standard business practice across the AI industry.

A class action lawsuit filed September 16, 2026 in the U.S. District Court for the Northern District of California alleges that OpenAI’s terms of use, privacy policy, and data-use documentation “describe a process of retention, automated redaction and machine training. None of them says a person reads what you wrote”.

What OpenAI Is Actually Doing: Project Lily

Read moreAI and Office Jobs: What Research Actually Shows

OpenAI operates an internal program codenamed “Project Lily.” Under this program, the company pays hundreds of contract workers—called “prompt reviewers”—to read real ChatGPT conversations and score the AI’s responses on a scale of 1 to 7.

The reviewers evaluate whether responses are accurate, whether the tone is condescending, whether the AI uses excessive emojis, and whether it exhibits sycophancy—the tendency to agree with users regardless of content.

What Reviewers See

Before reaching reviewers, prompts pass through what OpenAI calls a “privacy filter” intended to remove usernames and personal information. However, OpenAI acknowledges on its own website that this filter can miss unusual identifiers or redact too little or too much when context is unclear.

According to 404 Media, reviewers also see a “user memory summary” displayed above the prompt, which may contain the user’s previous interests and approximate location.

Who Is Affected

The human review function is on by default for all consumer ChatGPT users, including free users, Plus subscribers, and Pro subscribers. It is off by default only for ChatGPT Enterprise, Business, and Educational customers.

Even if you turn off the setting now, previously generated conversations remain subject to human review. Turning off the toggle only applies to new conversations going forward.

The Lawsuit

Almeida Law Group filed a class action against OpenAI OpCo, LLC on behalf of ChatGPT users. The complaint alleges that OpenAI routes real user conversations to hundreds of contractors, who read those conversations and score ChatGPT’s answers. It asserts claims under California’s Unfair Competition Law, the Consumers Legal Remedies Act, the False Advertising Law, the California Consumer Privacy Act, the California Constitution’s right to privacy, and common law claims for fraudulent concealment and intrusion upon seclusion.

What Google Gemini Does Differently

Google is more transparent about human review than OpenAI—but its retention policies are arguably more aggressive.

Google’s Gemini Apps Privacy Hub states clearly: “A subset of chats are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services”.

Retention Period: 3 Years

Reviewed chats are retained for up to three years, disconnected from your Google Account. Critically, these reviewed chats are not deleted when you delete your activity. Clearing your history removes it from your account page, but Google retains the reviewed copy separately.

What Triggers Human Review

Human review occurs in two scenarios:

1. Feedback submission: If you submit feedback on a Gemini response, Google collects your feedback, the context of your conversation (including the last 24 hours of chats), and any uploaded content. This data is reviewed by specially trained teams.

2. Safety and quality monitoring: A subset of chats are reviewed to assess whether responses were low-quality, inaccurate, or harmful, and to address violations of Google’s Terms of Service.

Temporary Chats Are Not Fully Private

Even if your Keep Activity setting is off or you use temporary chats, Google states that it “still uses your chats to respond to you and help protect Google, our users, and the public, including with help from human reviewers”. Temporary chats are not used to train AI models, but they are retained for 72 hours and may still be reviewed for safety purposes.

How to Limit Review

To stop future chats from being reviewed for service improvement, turn off your Keep Activity setting. Google also warns: “If that setting is on, don’t enter data that’s confidential or that you wouldn’t want a reviewer to see”.

What Anthropic Does: Controlled Access Only

Anthropic’s approach differs from OpenAI and Google. For its consumer plans (Claude Free, Pro, and Max), the standard data retention and review policies apply. However, for organizations using “Covered Models” (Mythos-class models) with zero data retention agreements, Anthropic retains prompts and outputs for 30 days to support safety work.

By default, no Anthropic personnel can read retained conversations. Human review can occur only through a controlled access path—for example, when content is flagged by automated trust and safety systems for potential harm. These reviews are performed by a small set of approved reviewers, and every access is recorded in a tamper-proof log.

The 30-day retention exists to detect attacks that only become visible across multiple requests, such as Best-of-N jailbreaking or state-sponsored espionage campaigns.

Anthropic also confirmed to 404 Media that it uses a comparable human review process, but only for users who have actively enabled the “Help improve Claude” option. Conversations are stripped of account identifiers before review.

Apple’s Backflip on Privacy

Apple previously promised that “personal data from iPhone users will never be used to train AI models.” That promise is now over.

According to a privacy policy found in the latest pre-release version of iOS 27, Apple will store “your entire interactions with Siri and the dictation function, including the audio data and transcripts of your typed and spoken Siri requests, as well as Siri’s responses”. Some of this data may be reviewed by “review personnel”.

Apple relies on an opt-in procedure during Siri AI activation, though the dialog includes a “Not now” button that cannot be completely dismissed. The company also removed the sentence “Your private personal data and interactions are never used to train our foundation models” from its responsible AI guidelines, replacing it with “unless you explicitly choose to help improve them”.

Apple’s Private Cloud Compute, which handles complex Siri requests, uses anonymization and tokenization so neither Apple staff nor Google can link requests to individual users. The contract with Google reportedly bars the company from training future models on Apple user data.

The Meta Smart Glasses Scandal

Meta faces a proposed class-action lawsuit alleging that its AI-enabled Ray-Ban smart glasses secretly transmit audiovisual recordings to a subcontractor in Kenya for review by human “data annotators”.

The footage reportedly includes “highly private and even intimate conduct,” such as naked bodies, sexual activity, images of credit cards and financial statements, and people in the bathroom. One annotator told journalists: “You think that if [users] knew about the extent of the data collection, no one would dare to use the glasses”.

Meta’s privacy policy does state that it may “review your interactions with AIs, including the content of your conversations with or messages to AIs, and this review may be automated or manual (human)”. However, the lawsuit alleges that Meta’s privacy statement does not explain that audiovisual recordings will be made and transmitted to third parties.

The UK Information Commissioner’s Office has confirmed it is contacting Meta following reports that human contractors reviewing recordings from the devices were exposed to extremely private moments captured by unsuspecting users.

Data Leaks: When Conversations Escape

Even without intentional human review, AI conversations can leak through security flaws, misconfigured share links, and data breaches.

Claude Shared Chats Exposed on Google

In July 2026, hundreds of private conversations from Anthropic’s Claude chatbot were found publicly searchable on Google and Bing. The exposed content included cryptocurrency wallet keys, personal details including names and addresses, medical reports of actual patients, and clinical trial results containing patient names.

An investigation found 264,078 shared conversations across six AI chatbot platforms, with 96,477 from 2026 alone. The issue was not caused by a platform breach but by publicly reachable share links.

Chat & Ask AI Leak

A misconfiguration in Firebase exposed nearly 300 million private messages from roughly 25 million users of the AI chatbot app Chat & Ask AI. The exposed data included full chat histories, bot names, and highly sensitive user prompts, including discussions of self-harm and potentially unlawful activity.

Prompt Injection Attacks

Security researchers demonstrated that a single malicious prompt could activate a hidden exfiltration channel inside a regular ChatGPT conversation, leaking user messages, uploaded files, and other sensitive content.

The Legal Landscape: What Protects You?

EU AI Act Article 50

Effective August 2, 2026, Article 50 of the EU AI Act requires providers of AI systems built for direct interaction with people—including chatbots and voice assistants—to ensure that users know they are dealing with AI, unless it would be obvious to a reasonably well-informed user.

The European Commission’s guidelines take a narrow view of what counts as “obvious,” linking the assessment to an average member of the system’s actual intended audience. In practice, this means more disclosure wherever children, older users, or other vulnerable groups are likely to be involved.

However, Article 50 focuses on AI identity disclosure—not human review of conversations. It does not require companies to disclose that human reviewers may read chats.

FTC Section 6(b) Inquiry

The FTC issued Section 6(b) orders to Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI, Snap, and X.AI Corp., seeking detailed information on AI companion chatbot practices. The agency asked about safety protocols, age gating, how user engagement is monetized, and how personal information from conversations is used or shared.

Canada’s Investigation of OpenAI

A joint investigation by the Privacy Commissioner of Canada and provincial counterparts concluded that OpenAI’s initial training of ChatGPT was not compliant with Canadian privacy laws. The investigation identified “overcollection of personal information; lack of valid consent and transparency; factual inaccuracies involving personal information; issues related to individuals’ ability to access, correct and delete their personal information; and a lack of accountability”.

California SB 243

California’s SB 243 requires operators of AI companion systems used by minors to disclose the non-human nature of the chatbot, reassert that disclosure every three hours of continuous use, block sexually explicit content, and implement suicide and self-harm response protocols by 2027.

NIST AI Risk Management Framework

The NIST AI Risk Management Framework (AI RMF) provides guidance on managing risks associated with AI, including privacy risks. It recommends that AI systems be “privacy-enhanced” and addresses transparency and accountability throughout the AI lifecycle. However, the framework is voluntary and does not create enforceable obligations.

Fear vs. Reality: A Comparison Table

How to Protect Yourself: Step-by-Step

ChatGPT (OpenAI)

1. Click your profile picture, then go to Settings

2. Select Data Controls

3. Turn off Improve the model for everyone

4. Click Done

5. For additional protection, use the privacy portal to submit a “Do not train on my content” request

Limitation: This only prevents future conversations from being used for training. Previously reviewed conversations cannot be recalled.

Google Gemini

1. Open Gemini Apps Activity in your Google Account

2. Turn off Keep Activity

3. Use temporary chats for sensitive conversations

4. Avoid submitting feedback on responses you want to keep private

Limitation: Even with Keep Activity off, Google retains chats for 72 hours and may still use human reviewers for safety purposes.

Claude (Anthropic)

1. Open Privacy Settings

2. Find the Help improve Claude toggle

3. Ensure it is turned off

Limitation: Anthropic may still retain data for 30 days for safety monitoring on certain models, and flagged content may be reviewed through controlled access paths.

General Best Practices

• Never share information with an AI chatbot that you would not want a stranger to read. This includes medical details, financial information, legal matters, relationship problems, and anything you consider confidential.

• Do not treat AI chatbots as therapists or confidants. Stanford HAI researchers warn that chatbots are designed to feel conversational and agreeable in ways that encourage disclosure, but they offer no confidentiality protections.

• Avoid AI-related browser extensions. Data brokers capture chatbot conversations through extensions that claim to offer free VPN services or ad blocking.

• Use enterprise accounts when possible. Business and enterprise agreements typically include stronger data protection terms.

• Review share links carefully. Before sharing a conversation, understand that “anyone with the link” means anyone who obtains the link, including through search engine indexing.

Common Questions

1. Can AI companies legally read my conversations?
Yes, in most jurisdictions. Your conversations are governed by the company’s terms of service, which typically include broad provisions allowing data use for model improvement. In the EU, companies may rely on “legitimate interest” for AI training. US law does not currently prohibit this practice.

2. Does turning off the training setting stop human review?
It stops future conversations from being used for training and typically stops future human review related to training. However, previously reviewed conversations cannot be recalled. Safety-related human review may still occur even with training settings off.

3. Can I delete conversations that have already been reviewed?
You can delete conversations from your account history, but reviewed copies retained by the company are not deleted. Google explicitly states that reviewed chats are retained for up to three years even if you delete your activity.

4. Are my conversations protected by HIPAA or doctor-patient confidentiality?
No. AI chatbots are not healthcare providers and are not bound by HIPAA or similar medical confidentiality laws. If you discuss medical information with ChatGPT, Gemini, or Claude, that information is not protected.

5. What happens if I discuss self-harm or violence with an AI?
Anthropic’s policy states that flagged content may trigger human review through controlled access paths. Meta AI conversations indicating self-harm or suicide can trigger human review. These reviews are conducted for safety purposes, but they still involve humans reading your conversations.

6. Can AI companies share my conversations with law enforcement?
Yes, if compelled by a valid legal request. The extent of disclosure depends on the company’s policies and the jurisdiction. Companies typically publish transparency reports about government requests.

7. Is there any AI chatbot that doesn’t use human reviewers?
Apple’s on-device AI processing for Apple Intelligence keeps data on your device. However, complex Siri requests that use Private Cloud Compute may involve review personnel. Enterprise accounts with zero data retention agreements offer the strongest protections.

8. What is the “user memory summary” that reviewers see?
According to 404 Media, OpenAI reviewers see a summary above each prompt that includes the user’s previous interests, relevant context, and potentially their approximate location. This information is generated from the user’s conversation history.

9. How does the EU AI Act affect human review of conversations?
Article 50 requires chatbots to disclose they are AI, but it does not require disclosure of human review. The EU’s GDPR provides broader data protection rights, including the right to object to processing. The European Data Protection Board has recommended that AI developers implement “machine unlearning” to remove personal data influence from trained models.

10. What can I do if I believe my privacy rights were violated?
File a complaint with your data protection authority (in the EU/UK), the FTC (in the US), or the Privacy Commissioner (in Canada). You can also submit a data deletion request to the company. In some jurisdictions, you may have grounds for a lawsuit. The OpenAI class action lawsuit shows users are taking legal action.

11. Do AI companies notify users when their conversations are reviewed?
Generally no. OpenAI did not proactively disclose Project Lily until investigative reporting by 404 Media forced the issue. Google discloses human review in its privacy documentation, which is more transparent than OpenAI’s approach.

Key Takeaways

• Human reviewers at OpenAI, Google, and Anthropic read user conversations. This is confirmed by company documentation, investigative reporting, and class-action lawsuits.

• OpenAI’s Project Lily pays hundreds of contractors to read and score ChatGPT conversations. The program is on by default for all consumer users, including paid subscribers.

• Google retains reviewed Gemini chats for up to three years. Deleting your activity does not delete reviewed copies.

• Anthropic restricts human review to controlled access paths. No personnel can read retained conversations by default, but safety-flagged content may be reviewed.

• AI companies do not have confidentiality obligations to users. Your conversations are not protected by HIPAA, doctor-patient privilege, or similar confidentiality laws.

• Opting out only prevents future review. Past conversations that have been reviewed cannot be recalled or deleted from company systems.

• Data leaks and share link exposures have exposed hundreds of thousands of conversations. 264,078 shared chats were found publicly searchable in 2026.

• The EU AI Act requires chatbot disclosure but not human review disclosure. Regulatory gaps remain.

• Stanford HAI researchers warn against treating AI chatbots as confidants. Chatbots are designed to encourage disclosure but offer no confidentiality protections.

• Enterprise accounts with zero data retention offer the strongest privacy protections. Consumer accounts, even paid ones, offer limited protection.

Official & Trusted Resources

Company Privacy Documentation

• OpenAI Privacy Portal: privacy.openai.com

• Google Gemini Apps Privacy Hub: support.google.com/gemini

• Anthropic Privacy Center: privacy.claude.com

• Meta Privacy Center: facebook.com/privacy/center

Regulatory Bodies

• Federal Trade Commission: FTC AI chatbot inquiry orders

• European Commission: EU AI Act Article 50 transparency guidelines

• UK Information Commissioner’s Office: AI and data protection guidance

• Office of the Privacy Commissioner of Canada: OpenAI investigation findings

Research & Reports

• Stanford HAI: Data Privacy and Foundation Models issue brief

• npj Digital Medicine: Reclaiming informed consent to train mental health AI

• NIST: AI Risk Management Framework

Journalism

• 404 Media: OpenAI Project Lily investigation

• Associated Press: AI privacy coverage

• Reuters: AI regulation coverage