Harness-1: The $20 Billion Agent Challenging GPT-5.4

Harness-1: The $20 Billion Agent That Challenges GPT-5.4

Why Do Existing Research Agents Stagnate?

Most retrieval agents are trained in an integral manner. The model generates queries, reads snippets, decides what is important, and retains all this context in an ever-expanding transcript. The policy learns everything: the search strategy, evidence tracking, deduplication, and even stopping conditions.

The problem is that reinforcement learning then tries to improve all of this at once. Semantic search decisions, such as "should I search for the 'merger date' or the 'acquisition year'," get mixed up with more basic management tasks. Have I seen this snippet before? Reinforcement learning ends up optimizing both, and honestly, they do not share the same learning dynamics. It becomes a bit chaotic.

Researchers call this the central design flaw. Their solution is simple: move state management out of the model and into a harness.

What Does the Harness Actually Do?

The state harness constitutes the main advancement. It runs the model like a state machine. It maintains these four persistent structures throughout each episode:

• A candidate pool that includes all compressed and deduplicated documents from all candidate searches.
• A curated set which is the final output with up to 30 identified documents marked with importance indicators (very_high, high, medium, low).
• A full-text store containing every piece of retrieved data, stored outside the machine's prompt.
• A proof graph which is a collection of self-extracted entities, their linking documents, and unique leads.

The proof graph part of this structure is quite clever. The regex extractor scans each piece of retrieved data for proper names, years, and dates. Linking documents containing two or more entities often found together are flagged as very high priority. Unique items mark potential follow-up searches. At each turn, the harness presents this information efficiently and compactly.

The Eight-Tool Interface

The eight model-based tools operate at each turn. At each turn, the model issues exactly one action.

A two-phase compression is applied to the output of the retrieval search phase. The first phase of compression uses Sentence-BM25 to rank all sentences and select the top 4 from each snippet. The second phase of compression is achieved through two-level deduplication: the first step is deduplication by snippet ID, and the second step is deduplication by content fingerprint. The policy never sees the raw retrieval output before the completion of the two-phase deduplication.

The design has paid off, as the model has kept its context clean. The model has only processed signals, and not all tokens are noise.

The Cold Start Problem (and Its Solution)

The first problem in retrieval training is determining how a policy learns to create a curated dataset from scratch, leading to randomness in the early episodes of the policy's RL. Since the initial state of the policy has no precedent to refine, it does not know how to curate. Consequently, the policy either throws everything into the curated set or curates nothing at all.

Harness-1 addresses this issue by using a hot start seed. After the harness successfully performs a search for the first time, it automatically generates a curated dataset using the top 8 re-evaluated results that have been labeled with a fairness score. Thus, the policy has a corrective function (refinement, increasing the value of quality documents and decreasing the quality of weak documents) rather than a primary function (removing all documents and starting from scratch).

This small change creates a significant amount of stability in training and demonstrates that curation is learned more easily through refinement than through creation.

How Training Works: SFT Then RL

There are two steps in the training pipeline that perform different types of work:

Step 1: Supervised Fine-Tuning

A teacher model (GPT-5.4) operates in the full active state harness and is trained with a broad set of diverse queries at this stage. After filtering out all underperforming trajectories, we are left with a total of 899 episodes that cover the correct use of the interface to train the model on how to call tools, structure actions, and update the curated set.

Step 2: Reinforcement Learning

In the second step of reinforcement learning, a CISPO policy is used with a reward function based solely on terminal rewards, and has a limit of 40 turns. The training data consisted of queries for financial documents (SEC), but the policies learned at this stage were generalizable to all 8 reference domains. The reward function has two major advantages:

• The first advantage is the separation of discovery and selection. Both elements are provided as independent rewards during the search and curation of a discovery (i.e., a relevant document is found and then curated).

• The second advantage is the addition of a diversity bonus for the tools used. This bonus is more significant than you might think.

Without the diversity bonus, the agent gets stuck in a loop. The agent constantly issues the same search query in slightly varied forms, fills the curated set with many similar items, and experiences a slowdown (0.53 curated recall). The agent learns to use grep_corpus, check, and read_document in addition to search_corpus when a diversity bonus is added, and consequently, the agent's recall score increases to 0.60 due to this single change.

Practical: Running Harness-1 Locally

Let's give it a try.

Currently, this repository uses uv for dependency management and vLLM for the service. You will need sufficient GPU VRAM to run a 20B model. For example, a single A100 (80 GB) will work very well. Alternatively, two A100 (40 GB) will work great using tensor parallelism if you have them.

Clone the repository and install it:

git clone https://github.com/pat-jj/harness-1.git

If you haven't installed uv, do it now:

uv sync --extra vllm

Note that downloading vLLM and its CUDA dependencies is done with the --extra vllm flag and may take some time during the first package download. If you skip this step, the inference script will not work due to its dependency on the vLLM server.

The first time you run an application with this model installed, it will download about 40 GB of weight from HuggingFace and set up a local server compatible with OpenAI using uvicorn. After starting uvicorn, you can open the server at http://0.0.0.0:8000, and you should be able to run your model.

uv run python inference/vllm_local_inference.py serve \
--model pat-jj/harness-1 \
--served-model-name harness-1

If you have two GPUs, you can add --tensor-parallel-size 2 to create a distribution between the two GPUs. Without this option, you will encounter memory issues with a 40 GB GPU.

Running step 3 means you can now issue a search query directly to the Harness-1 server. You need to format your search query as a structured query directed against a Chroma corpus. Here’s what a minimal test would look like, using the BrowseComp+ reference format:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
response = client.chat.completions.create(
model="harness-1",
"content": "Search for documents on the [EU AI Act](/dossier/regulation-ia) enforcement timeline for 2024."