AI Agents: Context Engineering, Key to Their Effectiveness

Optimizing Context, a Valuable and Limited Resource for AI Agents

In the field of artificial intelligence, failures are often not due to the intrinsic capability of the models, but rather to how context is developed, transmitted, and maintained. This issue is complex because the field is rapidly evolving and the techniques used are still maturing. Much of this work relies on an experimental approach, heavily influenced by the specific context, constraints, and environment in which the systems operate.

In my research on multi-agent systems, I have observed a recurring pattern: the performance of models depends less on the amount of context provided and more on the precision with which that context is structured. This article aims to share my learnings regarding context management, viewed as a limited resource. It involves deciding which information to include or exclude and how to structure this data so that agents remain consistent, efficient, and reliable over time.

Context Engineering

Context engineering involves providing the appropriate information, tools, and formats to a large language model (LLM) to accomplish a specific task. Successful context engineering entails determining the smallest set of high-signal tokens, thereby maximizing the likelihood that the LLM will produce a quality outcome.

In practice, this generally translates into four main actions:

• Offloading information to external systems, allowing the model not to handle everything internally.
• Dynamically retrieving necessary information rather than preloading everything.
• Isolating context to prevent subtasks from contaminating each other.
• Reducing history to retain only what will be useful for the future.

A common problem is context pollution, where an overload of unnecessary, conflicting, or redundant information distracts the LLM.

Context Compaction

Context compaction is an effective response to context pollution. When a model approaches the limit of its contextual capacity, it summarizes its content and opens a new context window with that summary. This method is particularly useful for long-duration tasks, allowing the model to continue its work without significant performance degradation.

Recent research on context folding proposes an alternative: agents actively manage their working context. An agent can split to handle a subtask and then fold back once it is completed, condensing the intermediate steps while retaining a concise summary of the results obtained.

Communication Between Agents

As task complexity increases, teams are turning to multi-agent systems. A common mistake is to assume that more agents mean more shared context. In reality, sharing a vast transcript among sub-agents can hinder specialization. Each agent then reads everything, inheriting the mistakes of others and bearing the same contextual cost repeatedly.

The solution lies in managing communication as a state transfer through well-defined interfaces, rather than as shared memory. For discrete tasks with clear inputs and outputs, agents should exchange artifacts rather than raw traces.

KV Cache Penalty

When generating text with AI models, many identical calculations are often repeated. The KV cache is an optimization technique that speeds up this process by storing crucial information from previous steps, thus avoiding the need to recalculate everything.

However, in multi-agent systems, if each agent shares the same context, it overloads the model with irrelevant details, leading to a significant KV cache penalty. Agents working on the same task need to communicate, but this should not be done through memory sharing.

Keeping the Agent's Toolset Small and Relevant

The choice of tools is a context problem disguised as a capacity problem. As an agent accumulates more tools, the action space becomes harder to navigate. This has consequences. Tool schemas must be much more distinct than most people realize. Tools should be well understood and have minimal overlap in their functionalities.

Good compaction must therefore preserve facts that continue to constrain future actions. Otherwise, you end up with a concise summary that looks good to a human but is useless for an agent.

Context engineering is still evolving, and there is no one-size-fits-all method to achieve it. If not managed well, context grows, drifts, and ultimately collapses under its own weight. If managed well, context becomes the difference between an agent that merely responds and an agent capable of reasoning, adapting, and remaining consistent across long and complex tasks.