Chatbots and Privacy: Confidences and Legal Access

Conversations with public AI have already been included in legal cases, and some companies are scrutinizing internal messaging to train their models. While more than half of survey respondents are unaware of the use of training, researchers are raising concerns about the memorization of LLMs. There are settings available, as well as local alternatives without cloud dependency.

Conversations Already Used in Justice and Work

Transcripts from ChatGPT were used in an investigation into a double murder in South Carolina earlier this year, as well as in an arson case in Palisades last year. A conversation with Snapchat's AI also served as evidence in a murder trial in 2024. Sam Altman, CEO of OpenAI, indicated that sensitive exchanges with ChatGPT may need to be produced in the event of legal proceedings. Providers like Anthropic, Google, and OpenAI may be legally required to hand over chat data to authorities, a reality known to only 25% of respondents in the DuckDuckGo survey. In the workplace, monitoring extends beyond social media: AI accounts used through professional access can be reviewed by employers, and sharing internal information in a private chat is considered a violation, although two out of five workers have done so according to a recent survey. Several companies, including Walmart, Delta, T-Mobile, Chevron, and Starbucks, use AI systems to assess employee positivity or toxicity by analyzing platforms like Slack or Teams, and some organizations request to publish messages in public channels to feed model training.

Reused Data for Training Models

Unless opted out, exchanges with chatbots are recorded and can be reused for training AI models. Last year, hundreds of conversations with Claude from Anthropic appeared in Google search results, demonstrating the persistence of this data. AI training relies on large volumes of content, whether published materials, accessible audio or video recordings, or user discussions. By 2025, six companies—Amazon (Nova), Anthropic (Claude), Google (Gemini), Meta (Meta AI), Microsoft (Copilot), and OpenAI (ChatGPT)—have reintroduced user chats for additional training, without consistently offering an opt-out option.

Technical Risks: Unintentional Disclosures and Memorization

A researcher from Wake Forest University has shown that agents relying on language models can disclose data, intentionally or not. For Uttara Ananthakrishnan, a professor at the University of Washington, the memorization of data by LLMs is a critical point: a model can provide two people with the same response containing identifying elements, without being aware of it.

What Users Perceive (or Ignore)

More than half of respondents to the DuckDuckGo survey are unaware, or unsure, that their conversations are used for AI training, and nearly three in five express discomfort with this usage; 30% feel very uncomfortable. One-third of individuals report sharing secrets with chatbots that they do not share with close friends or professionals, a figure that rises to 56% among AI enthusiasts. Among parents, 43% say they reveal more to AI than to trusted individuals, including their children. Many are unaware of the potential uses of their data or the settings available to prevent their recording. Nearly 40% have no trust in companies, and almost 50% have no trust in the U.S. government to protect this data.

Commercial Narrative and Privacy Reality

DuckDuckGo considers messages that attempt to present chatbots as safe confidants to be misleading, as exchanges are generally not private. MemX, a memory AI agent, summarizes this reality by describing these discussions as commercial records, not confidential consultations. In practice, shared information can be reused for training, accessed by authorities upon request, or available to an employer in a professional context.

Reducing the Footprint: Settings and Local Alternatives

Many services offer settings to prevent exchanges from being stored for training, and a guide by Blake Stimac outlines how to proceed for Gemini, Claude, ChatGPT, and Grok; at Meta, the process resembles more of a request than an effective opt-out. It is recommended to treat chats as public, avoid detailing mental states, and regularly export data to check what is stored. Offline and local options, such as Jan.ai, Ollama, LM Studio, PrivateGPT, or GPT4All, allow users to avoid the cloud. Uttara Ananthakrishnan emphasizes that chat interfaces create a sense of intimacy that encourages sharing very personal thoughts, even as some fear that they may inadvertently become accessible archives.