AI Agents Confront Web Blockages: An Open Standard in the Works

⚡

Key Takeaways

1Personal AI agents like Muse (Meta) and Instinct encounter roadblocks on many websites, sometimes intentional, sometimes due to anti-bot filters.

2Platforms like Amazon, Yelp, or airlines limit or prohibit access to agents, while users report account suspensions.

3Meta, Walmart, and other partners are working on an open standard to allow legitimate AI agents to access online services, and Meta is increasing partnerships for Muse.

💡Why it matters — The access of AI agents to websites will depend on the ability of stakeholders to establish shared rules, which will condition the adoption of these tools by the general public.

Anti-bot filters and platform policies often prevent AI agents from making purchases or reservations on behalf of users. Airlines, marketplaces, and web protection networks set their boundaries, while a group of companies, including Meta and Walmart, is working on an open protocol. Meanwhile, Meta is forming partnerships to ensure Muse's access, despite recent cases of blocking.

Infrastructure Filters: Suspicions Towards CDNs and Limited Data

Industry players suspect that Cloudflare and other content delivery networks are disrupting the traffic of the AI agent Muse. This situation is attributed to a change in default settings for crawlers on September 15, allowing the blocking of AI training while permitting other bots, and to configurations that have led some sites to block AI agents on pages containing advertisements. When questioned, Cloudflare stated that it does not have specific data to share regarding these blocks and referred to its public hub Cloudflare Radar, which shows an increase in bot activity without distinguishing legitimate agents from others. Cloudflare offers protections against automation, manages a marketplace where AI bots must pay to access certain data, and is testing a web browser dedicated to AI agents.

Platform Rules: Restricted Access and Airline Caution

Yelp does not allow non-human traffic on its platform unless the agent pays for access through a data licensing program, which exposes AI agent efforts to failure without a formal agreement. eBay clarifies that it does not prohibit all third-party shopping agents but restricts unauthorized actions such as automated scraping and model training. Adidas, Zillow, and Pizza Hut declined to comment. In the airline sector, booking flight tickets is one of the main uses of AI agents, but many reports indicate refusals to process requests. According to Delta, the company protects its customers against unauthorized automation and does not allow third-party agents to make reservations on its sites, while exploring the potential of these tools for safety and user experience. From United's side, a representative mentioned the existence of a rule that prohibits any automation without prior written agreement and did not provide details regarding any blocking measures targeting personal AI agents.

User Landscape: Repeated Blocks and Suspended Accounts

On social media, early users report an increase in blocks targeting their AI agents, citing Yelp, eBay, Zillow, Pizza Hut, Adidas, and several airlines. Some believe that these blocks are on the rise. Individuals claim that eBay accounts have been suspended for using AI agents, and one user indicated that Google expelled Instinct during a Gmail cleanup. In this context, widespread confusion persists regarding the exact reasons for the failures encountered by the agents.

Industry Response: An Open Standard and Targeted Agreements

Meta, Walmart, Stripe, Sierra, Genesys, Rocket, NiCE, and Decagon are working on developing an open standard to govern communication between AI agents and businesses, focusing on agent-to-agent interactions in e-commerce and aiming to distinguish legitimate uses from abuses. Meta asserts that a clear normative framework can give companies more predictable control and facilitate collaboration between agents, arguing that refusing an agent is akin to refusing the customer using it. Meanwhile, Meta is establishing brand partnerships to ensure Muse's access to partner sites; the list of connectors integrated into Muse is growing, even if recently announced partners are not yet included. Muse's expansion includes tools for small businesses and hints at a reorganization of this list. In this model, any block at a partner is considered a bug to report.

Concrete Cases: Amazon Blocks Muse, Walmart Cooperates but Stumbles on Human Verification

Amazon recently blocked the AI agent Muse, preventing it from browsing and making purchases on its marketplace. At Walmart, several user complaints reported that purchases were impossible via Muse; a Walmart spokesperson assured that these blocks are unintentional and recalled a partnership announced during Meta's Connect conference in September, aiming to be accessible through AI agent experiences. On a technical level, the presence of a human verification button can interrupt the session and expel the agent, highlighting the limitations of anti-bot tools against new usages. More broadly, some blocks are deliberate while others fall under traditional protections, often leaving the user in uncertainty. These incidents concern agents now capable of executing complete tasks without technical prerequisites for the user.