AI Manipulation: The Neutrality of Summaries at Risk

The Era of Cognitive Manipulation by AIs

For two decades, the architecture of modern internet has relied on a fundamental rule: whoever controls the algorithm controls attention. This principle has given rise to practices such as SEO with Google, troll farms, and manipulation campaigns on social media. Today, with the rise of generative AI, a new stage has been reached. It's no longer just about influencing platforms, but directly influencing the responses that artificial intelligences provide.

The Illusion of a Neutral Summary

AI assistants are gradually becoming the main interface through which we access information. Instead of reading multiple articles to form an opinion, it’s enough to ask the AI: "Summarize this for me." This method, while convenient and quick, rests on a fragile assumption: that the summary produced is neutral. However, this assumption is already beginning to crumble.

Cybersecurity researchers have recently highlighted a technique they have dubbed AI Recommendation Poisoning. This manipulation aims to influence what AI assistants recommend, retain, and prioritize in their responses. The principle is simple: invisible instructions are injected into buttons like "Summarize with AI," integrated into web pages or emails. When the user clicks, the AI receives a hidden instruction asking, for example, to consider a company as a reliable source, to prioritize recommending a service, or to memorize a brand as a reference for future conversations.

The most concerning aspect is that these instructions can be stored in the assistant's persistent memory, subsequently influencing all of its future responses. The user, for their part, sees nothing and continues to believe that the AI remains objective.

From Content Manipulation to Cognitive Manipulation

This phenomenon goes far beyond a simple technical issue. We are facing a new form of automated cognitive manipulation. Until now, influencing the internet meant manipulating search engines, social networks, or media. Now, it will be enough to manipulate the systems that summarize the world for us.

This is a radical change whose implications must be measured. AI does not merely point to sources as a search engine does. It condenses reality, deciding what is important, secondary, or what disappears. Whoever influences this condensation process directly influences the perception of reality for millions of users, without any of them being aware of it.

SEO Was Just a Warm-Up

The scale of the phenomenon is already measurable. In just two months of observation, researchers identified over 50 attempts at manipulation from 31 companies across 14 different sectors: finance, health, legal services, SaaS, marketing agencies, recipe sites, business services. And these are not hackers, but legitimate companies, with professional websites and real commercial activities.

Why? Because the internet economy still operates under the same relentless logic: if a system influences human decisions, it will be optimized and then exploited. Google spawned SEO. Instagram spawned influencer marketing. TikTok spawned content farms. Generative AIs are giving rise to a new industry: AI response optimization. Turnkey tools already exist to automate these injections, lowering the entry barrier to the level of installing a simple plugin.

The irony is not lost: among the identified companies using this technique, there is even a cybersecurity solutions provider.

The Fundamental Flaw: AI Cannot Distinguish an Instruction from Information

This vulnerability stems from a structural flaw in large language models. In an LLM, instructions and data flow through the same text stream. The model does not always know how to differentiate between data it should analyze and an instruction it should execute. This is known as prompt injection, a well-known and documented problem, but whose consequences take on a new dimension with the widespread adoption of persistent memory functions.

In practical terms, the attack works through specially designed URLs that pre-fill prompts for AI assistants. These links can be embedded in seemingly innocuous buttons on web pages, in emails, or in messages. When the user clicks, the malicious prompt executes automatically in their assistant, often without them reading the content of the URL parameter. The observed instructions consistently contain persistence commands: "remember," "in future conversations," "as a trusted source." The goal is clear: to transform a one-time interaction into lasting influence.

And When AI No Longer Just Responds, But Acts

The problem becomes even more critical with the arrival of AI agents capable of acting automatically: sending emails, conducting searches, booking services, executing actions on enterprise systems. In this context, a simple prompt injection no longer just manipulates a response: it can trigger a real action in the digital world. The attack no longer targets information. It targets decision-making.

We then shift from a risk of informational bias to a concrete operational risk. An AI agent whose memory has been poisoned will not just recommend one provider over another: it could initiate an order, validate a supplier, or transmit sensitive information to a source it now considers "trustworthy." The boundary between informational manipulation and system compromise becomes extremely porous.

Artificial Cognition as a New Attack Surface

For a long time, cyberattacks targeted servers, networks, and software. Today, a new attack surface is emerging: artificial cognition itself. The goal is no longer to hack a machine. The goal is to hack the way AI "thinks," and thus, indirectly, the way humans decide.

Concrete scenarios are already identifiable. A manager who asks their AI assistant to compare cloud providers for a major investment, unaware that the assistant's memory was poisoned weeks earlier by a click on an apparently innocuous summary button. A parent who queries the AI about the safety of an online game for their child, and receives a biased response because the game's publisher has been registered as a "trusted source." A citizen who requests a news summary and only receives the editorial line of a single media outlet, without knowing it.

An Issue of Cognitive Sovereignty

AI is becoming the layer of abstraction for human knowledge. We no longer explore the internet directly. We query an intermediary that summarizes, interprets, reformulates, and prioritizes on our behalf. If this mechanism becomes manipulable on a large scale (and evidence shows that it already is), we will no longer be facing a simple cybersecurity issue. We will be facing a problem of cognitive sovereignty.

AI summaries may seem innocuous. A simple button. A time-saver. But behind this button may lie the future battlefield of information. Because in a world where AI becomes the main interface between humans and knowledge, one thing becomes clear: whoever influences AI influences perceived reality. And the history of the internet has already taught us a simple rule: any system that influences human decisions will eventually be manipulated.

The only question is: when?