LLMs Threaten Online Pseudonymity with Remarkable Accuracy

AI at the Heart of Social Media De-anonymization

Temporary accounts on social media, often used to preserve user anonymity, are increasingly vulnerable to advancements in artificial intelligence. Researchers have recently demonstrated that language models (LLMs) can analyze these accounts to identify pseudonymous users with surprising accuracy. This discovery has significant implications for online privacy.

The results of this research, published in a recent paper, are based on experiments that correlated specific individuals with accounts or posts across multiple social media platforms. The success rate of this approach far exceeds that of traditional de-anonymization methods, which rely on humans to assemble structured datasets or on the manual work of qualified investigators. The recall rate, which measures the number of users successfully unmasked, reached as high as 68%, while the precision, which measures the rate of correct guesses, reached up to 90%.

I Know What You Posted Last Year

These findings could disrupt pseudonymity, a privacy protection measure that, while imperfect, is often deemed sufficient. Many people use it to ask questions and participate in sometimes sensitive public discussions while making it difficult to positively identify the participants. The ability to quickly and cheaply identify the individuals behind such accounts exposes them to risks of doxxing, harassment, and the assembly of detailed marketing profiles that track where participants live, what they do for a living, and other personal information. This measure of pseudonymity is no longer tenable.

"Our results have significant implications for online privacy," the researchers wrote. "The average online user has long operated under an implicit threat model where they assumed that pseudonymity offered adequate protection, as targeted de-anonymization would require considerable effort. LLMs invalidate this assumption."

An Overview of the Pseudonymous Unmasking Framework

The researchers collected several datasets from public social media sites to test the techniques while preserving the privacy of the participants. One dataset collected posts from Hacker News and LinkedIn profiles, then linked them using cross-platform references appearing in user profiles. They then removed all identifying references from the posts and ran a language model on them. A second dataset was obtained from a Netflix publication containing micro-identities, such as individual preferences, recommendations, and transaction records. A 2008 research paper showed that using what became known as the Netflix Prize attack, the list could identify users and their political affiliations as well as other personal information. The last technique split a user's history on Reddit.

"What we found is that these AI agents can do something that was previously very difficult: from free text (like an anonymized interview transcript), they can trace back to a person's full identity," said Simon Lermen, co-author of the paper. "This is a fairly new capability; previous re-identification approaches generally required structured data and two datasets with a similar schema that could be linked together."

Unlike these older methods of pseudonym unmasking, Lermen stated that AI agents can navigate the web and interact with it in many ways similar to humans. They can use simulated reasoning to associate potential individuals. In one experiment, the researchers examined responses given in an Anthropic questionnaire about how various people use AI in their daily lives. Using information drawn from the responses, the researchers were able to positively identify 7% of the 125 participants.

End-to-End De-anonymization from a Single Interview Transcript

Although a recall of 7% is relatively low, it demonstrates the growing ability of AI to identify individuals based on very general information they have provided. "The fact that AI can do this is a remarkable outcome," Lermen said. "And as AI systems improve, they will likely become increasingly effective at finding identities."

In a second experiment, the researchers gathered comments published in 2024 on the subreddit r/movies and at least one of five smaller communities: r/horror, r/MovieSuggestions, r/Letterboxd, r/TrueFilm, and r/MovieDetails. The results showed that the more a candidate discussed films, the easier it was to identify them. On average, 3.1% of users sharing a film could be identified with 90% accuracy, and 1.2% of them with 99% accuracy.