Cloudflare: Fine Management of AI Bots Starting September 2026
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisis ton rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Cloudflare: Fine Management of AI Bots Starting September 2026
Cloudflare is replacing its global blocking of AI bots with granular controls for search, training, and agent crawlers.
Cloudflare is providing all its clients with new tools to control AI bot access to their websites. Since July 2024, Cloudflare clients have been able to block all AI crawlers with a single click. Now, instead of a blanket ban, site owners can choose from three categories:
- Search: pertains to indexing by search engines.
- Training: pertains to data collection for training AI models.
- Agent: pertains to bots acting on behalf of users, such as ChatGPT.
The new AI bot controls in the Cloudflare dashboard allow site owners to manage access for Search, Training, and Agent bots separately.
Starting September 15, 2026, Training and Agent bots will be blocked by default on pages containing advertisements, as advertising indicates that a site desires human visitors. Search crawlers will still be permitted. Versatile crawlers like Googlebot, which perform both search and training functions, will be treated according to the strictest rule. The new options will also be available for users on the free plan.
Cloudflare encourages bot operators to separate their crawlers by purpose. Cloudflare's CEO, Matthew Prince, previously criticized Google for combining its search crawler with its AI crawler. In June 2026, Prince noted that bot traffic had surpassed human traffic on the Internet for the first time, a milestone he did not expect to reach until late 2027.
New Bot Database for More Granular Control
For enterprise clients, Cloudflare is also launching BotBase, a searchable database of all known bots integrated directly into the dashboard. It shows how each bot is classified and how it uses content, whether by linking pages or reproducing them in full.
Cloudflare is also changing the rules for verified bots. Previously, all verified bots were automatically allowed. In the future, a bot's category will determine whether it gains access. The "Verified" label will no longer be sufficient: bot operators will need to prove that they identify honestly and do not abuse the access granted to them.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.