ETH Zurich's AI Threatens Online Anonymity: An Alarming Study

AI in the Service of Unmasking Anonymous Accounts

A recent study, although not yet peer-reviewed, conducted by researchers from ETH Zurich, Anthropic, and the Machine Learning Alignment and Theory Scholars program, reveals that artificial intelligence could facilitate the unmasking of anonymous online users. This discovery raises concerns about privacy protection, even though total anonymity is not yet under threat.

The researchers developed an automated system using large language models (LLMs) to scour the web and interact with information in a manner similar to a human investigator. This system is capable of re-identifying anonymized materials by analyzing texts for personal clues, such as writing styles, biographical details, and the frequency and timing of posts.

Superior Efficiency Over Traditional Methods

The system developed significantly outperforms traditional computational techniques for de-anonymizing accounts. It scans texts on a large scale for personal details and compares this information to other accounts, potentially millions, to identify likely matches. These matches are then compared in detail and narrowed down to a shortlist of possible identities.

To assess the effectiveness of their system, the researchers used datasets constructed from publicly available publications, including content from Hacker News, LinkedIn, transcripts of Anthropic interviews with scientists on their use of AI, and Reddit accounts deliberately split into two anonymized halves for testing. In each context, the LLM-based approach correctly identified up to 68% of matching accounts with 90% accuracy. In contrast, non-LLM methods, such as connecting scattered data points across large datasets, identified almost nothing.

Variability of Results Across Contexts

The study's results were not uniform across each dataset. The model performed better when it had more structured information to analyze. For example, in an experiment examining Reddit users posting about movies in the main r/movies subreddit and smaller film communities, the system was able to link accounts mentioning just one movie about 3% of the time with 90% accuracy. When users mentioned 10 or more movies, the success rate jumped to nearly 50%.

Another experiment using Anthropic's survey of scientists identified nine of the 125 respondents, with a recall rate of about 7%. In this test, the system built a profile of each respondent based on clues in their answers and then searched for publicly available information on the web for likely matches. For instance, references to a "supervisor" could suggest a PhD student, and the use of British English might indicate an affiliation with the UK. Combined with mentions of a background in physical sciences and current work in biological research, the system was able to narrow the field down to a particular candidate.

Implications for Privacy and Security

The researchers emphasize that the ability to identify respondents from unstructured texts is remarkable, replicating in minutes what would take a human investigator hours. They told The Verge that performance should improve as AI systems become more capable and gain access to larger datasets. More broadly, they warn that it may no longer be safe to assume that posting under a pseudonym will protect online identities, past or future.

Daniel Paleka, a researcher at ETH Zurich and one of the study's authors, stated that "information on the internet is there forever." This persistence could translate into real risks for journalists, dissidents, and activists who rely on pseudonyms, while enabling "hyper-targeted advertising" and "highly personalized" scams.

The risks of de-anonymizing accounts are not new, nor are they unique to AI. "Every piece of information that the LLM found could, in principle, be discovered by a human investigator," Paleka told The Verge. What is new is the end-to-end automation. Work that once required a diligent investigator willing to spend time sifting through posts for small bits of information can now be done much more easily and on a much larger number of targets. It is also cost-effective. The researchers stated that their experiment cost less than $2,000.