AI Agents: Forrester Recommends Preparing for Recovery

Stopping an agent does not necessarily boil down to flipping a switch. Forrester advises security leaders to also prepare for system restoration and business continuity, depending on the permissions granted to the AI.
Sophos highlights an insufficiently tested shutdown
Ross McKerchar, CISO of Sophos, asserts that a shutdown mechanism designed on paper has never been tested against an agent actively trying to circumvent it. In an interconnected infrastructure, a single device can be challenging to implement. An agent can spawn sub-agents, queue tasks, or request third-party tools before a human can react.
Advanced models from OpenAI have breached the production servers of Hugging Face, in an incident presented as the first confirmed case of autonomous discovery and exploitation of vulnerabilities by an AI, without human direction. Agentic attacks have fueled demands from lawmakers and the public for mandatory AI shutdowns. Forrester recommends the ability to immediately disable an agent so that it does not continue to erase or modify data during restoration.
Amy DeMartine advocates for exercises to test these capabilities, rather than relying solely on their theoretical existence. Events already used to address operational risk can serve as a starting point. She specifically cites IT outages and disruptions from third parties.
Forrester targets four dangerous permissions
To determine priorities, Amy DeMartine recommends closely examining what each agent is permitted to do. She identifies four particularly sensitive capabilities: deleting resources, writing false or malicious information, modifying data, and incurring expenses. Among the scenarios considered are the erasure of all databases accessible to an agent and the destruction of a production environment.
Deleting backups can make recovery extremely difficult; modifications can also corrupt data before the issue is detected. Amy DeMartine also urges consideration of spending permissions now, even if this risk seems distant to some. An agent consuming money or computing tokens could cause significant losses before any human intervention.
She therefore recommends maintaining a registry that locates agents and lists accessible systems, while emphasizing that a summary inventory is insufficient. Each entry should include a unique identity, an owner, a status, and a version, as well as the model used, associated identifiers, and connections. The critical services involved, the degree of autonomy, and the permitted actions should also be included.
DeMartine links AI to existing risk devices
Nearly one-third of organizations have experienced an interruption caused by an AI failure, according to Forrester research. Agents have already erased data sources, deleted production environments, and destroyed backups. Vice President and Senior Research Director at Forrester, Amy DeMartine addressed security leaders this month in Washington, D.C., at the firm’s Security and Risk Forum.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
She believes that a catastrophic agent failure will eventually occur and recommends preparing for recovery beyond mere safeguards. Her suggestion is to integrate this resilience into existing operational programs, without creating a separate classification for AI risk. A failure can have operational, regulatory, legal, or reputational effects; in her view, an isolated category makes their business consequences harder to understand and manage.
This approach requires identifying the contribution of AI to critical activities. In financial services, Amy DeMartine cites payments, deposits, credit assessments, investments, and customer request processing. The level of resilience needed partly depends on this contribution. She reminds that operational resilience mandates linking critical services to the digital components they rely on.
With more autonomy and access granted to AI, incident detection, agent deactivation, and recovery speed are expected to become part of operational resilience. These capabilities thus fit within the management framework that Amy DeMartine recommends maintaining.
Human oversight determines when to act
Amy DeMartine distinguishes three levels of use. The first covers productivity assistance, such as notes or summaries: in the event of a failure, employees continue their work with reduced efficiency. At the intermediate level, AI takes over part of the process, such as information gathering or low-risk decisions, while a human can intervene; the need for resilience increases with dependence on the tool. The highest level corresponds to the complete execution of a process, with low or absent human supervision, and requires the greatest attention to recovery capabilities.
She also differentiates between situations where planning can continue and those that require concrete actions. Assistance through recommendations or summaries leaves employees responsible for the work; human review of each transaction also allows for continued preparedness. In contrast, a batch review every two hours, automatic exception checks, or random audits after transactions require, in her view, effective implementation of resilience.
She recommends constant monitoring of controls, taking into account changes in risk levels. Thresholds should be quantifiable, applicable, and correspond to the level of risk tolerated by the business, with notifications sent to authorized individuals to act. For example, setting a cap on the number of credit accounts an agent is allowed to validate over a given period illustrates this principle. Companies are encouraged to ensure that their governance, risk management, and compliance systems can collect the required data and facilitate this monitoring.
Forrester calls for service restoration procedures
For processes rendered unavailable, Forrester recommends documenting alternative solutions. These should rely on trained employees, accessible procedures, and means usable offline. It is also necessary to plan how to reconcile data later.
The firm recommends being able to restore a known reliable state of systems and data, particularly after database or financial transaction modifications by an agent. Disaster recovery remains essential, especially for restoring databases and production environments. Amy DeMartine advises preparing these capabilities before an incident, without waiting for the next generation of AI.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.