ContextFusion Reduces 60 to 99% of Tokens by Optimizing Context

A new tool aims to streamline what is sent to language models by retaining only the essentials. It combines compact representations, budget-constrained selection, and multi-vendor adaptations. Cuts of 60 to 99% of tokens have been achieved, with clarifications on cases where the approach is not suitable.

Budget-Constrained Selection, Cache, and Deltas to Limit Useful Volume

The cache-sensitive assembly distinguishes stable elements, such as system instructions, citation maps, and reusable blocks, from dynamic content like volatile or real-time data. In agent mode, delta merging allows for the identification of additions, updates, deletions, and unchanged elements to optimize cache reuse. A compression layer offers several levels, ranging from JSON minification to field pruning and compacting citations into identifiers. The core optimization relies on an NP-hard problem, but the tool claims execution times of under 100 ms for typical loads thanks to indexing and heuristics. However, ContextFusion specifies that optimization adds an overhead of 50 to 200 ms, making it unsuitable for real-time streaming requiring latency under 100 ms. Upstream, retrieval is performed by classifying the query, conducting a top-100 lexical search, and then a quick rerank to form the candidate set.

Usage Scope: Multi-Vendor, Tight Costs, and Agents, with Limits

ContextFusion is presented as relevant for multi-vendor configurations, allowing the use of the same pipeline with output formats tailored to each vendor. It is also highlighted for production where cost control is essential, with a claimed reduction of 60 to 99% of tokens. The management of agent conversations benefits from delta merging, which aims to avoid wasting repeated context. The tool supports the unification of heterogeneous sources such as PDFs, images, code, and spreadsheets, and meets latency requirements through precalculation and caching. However, ContextFusion notes that it may be superfluous for simple one-turn question-answer cases with small documents if a satisfactory RAG framework is already in use, or if real-time streaming with latency under 100 ms is needed, due to the optimization overhead of 50 to 200 ms.

The Problem Addressed: Tight Budgets, Risks, Cache, and Diversity

The project highlights the limitations of an approach based solely on semantic similarity, especially when the budget is 4000 tokens against 50 relevant pieces of 500 tokens each. Some elements may present both high utility and significant risk, such as outdated documentation. Others are reusable via cache, while some must remain fresh. Diversity is also sought to avoid sending multiple versions of the same information. ContextFusion's multi-objective planner formulates selection as a constrained optimization problem, weighing utility, risk, token cost, latency, cacheability, and diversity within a budget limit. This approach relies on a 0/1 knapsack applied to block representation variants.

Technical Chain: Ingestion, Normalization, Representations, and Vendor Adaptation

The processing chain begins with the ingestion of various formats, including PDF, DOCX, CSV, JSON, images via OCR, code, and Markdown. A normalization step converts all sources into ContextBlock objects with common fields such as source type, content hash, creation date, and metadata. The representation layer precalculates compact variants for each block, such as universal_summary, qa_extractive, code_signature, and agent_condensed. The pipeline then stores fingerprints, summaries, token statistics, retrieval features, and compact variants in a cache folder. Payloads are compiled according to the targeted vendor, for example in chat.completions format for OpenAI, with XML citations for Anthropic, via the local API for Ollama, or an openai_compatible wrapper. Retrieval prepares a candidate set through a lexical approach followed by a quick rerank.

Transparency and Claimed Results, with Published Benchmarks

The ContextFusion web interface displays detailed metrics such as ingested files, selected blocks, and total token volume, as well as the retained compact variants. For each block, it indicates the source, type of representation, utility score, and estimated token count. An exact preview of the context transmitted to the model is provided, along with an optional comparison of the model's response. A rare level of transparency has been highlighted compared to most RAG tools, which are often perceived as black boxes. They claim a reduction of 60 to 99% of tokens without loss of response quality. Benchmarks have been conducted with Claude Sonnet 4.6 on production-like loads, with publicly accessible methodology.

Getting Started: NPM, Python, Docker, and Configuration Example

For JavaScript, a dependency-free NPM wrapper allows installation and configuration via npx, the creation of an API key file, and the execution of optimizations, notably with the provider anthropic and the model claude-sonnet-4-6, as well as launching a web interface on a defined port. In Python, the context-portfolio-optimizer package is installed via pip, configured with a .env file, and then executed via command line with a token budget and specific queries or tasks, such as explaining code functions, including with gpt-5-mini and parameterized budgets. A Docker image can be built and launched to process data with a given budget. An example code illustrates integration via PipelineRunner and AnthropicAdapter, with a YAML configuration detailing budgets by categories and scoring weights, and the use of the model claude-sonnet-4-6 on sources combining PDF, Python code, and CSV. The tool emphasizes the ingestion of various formats, delta management in conversations, and budget-constrained selection.