NVIDIA AI-Q: Leader in DeepResearch Benchmarks

NVIDIA AI-Q, a deep research agent, has recently achieved the top position on the two main benchmarks for evaluating deep research agents: DeepResearch Bench and DeepResearch Bench II. With scores of 55.95 and 54.50 respectively, this achievement marks a significant advancement for open and portable deep research. The fact that a single configurable set dominates these two benchmarks demonstrates that models and tools accessible to developers can drive cutting-edge agentic research.

What Sets AI-Q Apart?

AI-Q is an open plan for creating AI agents that reason over enterprise and web data to provide well-cited answers. It offers a fully open and modular architecture that companies can own, inspect, customize, and configure according to their use case. The deep researcher is a workflow within the broader AI-Q plan, which includes intent routing, query clarification, and shallow search. The deep researcher adopts a multi-agent architecture consisting of a planner, a researcher, and an orchestrator, built on the NVIDIA NeMo Agent Toolkit and the NVIDIA Nemotron 3 Super-tuned models, with an optional ensemble and report refiner for maximum report quality. An ensemble—flexible by design, adjustable to your needs.

Why Winning Both Benchmarks Matters

DeepResearch Bench I and II evaluate research agents complementarily. DeepResearch Bench assesses report quality against a reference report based on dimensions such as completeness, depth of analysis, instruction following, and readability. Performing well here rewards well-structured narratives and strong synthesis. DeepResearch Bench II uses over 70 detailed binary rubrics per task to verify if an agent retrieves the right information (Information Recall), synthesizes it into high-level analyses (Analysis), and presents the results clearly (Presentation). Performing well here rewards granular factual accuracy and analytical rigor. Being at the top of both benchmarks means that the AI-Q deep researcher produces well-cited and well-crafted reports while mastering the underlying retrieval and reasoning.

Architecture at a Glance

The architecture of the AI-Q deep researcher behind these results relies on three components: Orchestrator, Planner, and Researcher. The orchestrator coordinates the research loop. The planner maps the informational landscape and designs an evidence-based research plan. The researcher deploys parallel specialists to gather and synthesize evidence from multiple analytical perspectives. Each agent can be powered by a different LLM. An optional ensemble runs multiple agents in parallel and merges their results for maximum report quality and information coverage.

Underlying Stack: NVIDIA and Deep Research

The same underlying stack powers both submissions to the ranking: open, reproducible, and built on: NVIDIA NeMo Agent Toolkit for wiring workflows, function logging, and evaluation. LangChain DeepAgents for the planner-researcher-orchestrator flow in multiple phases with middleware for sub-agents when applicable. NVIDIA Nemotron 3 LLMs powering the agent pipeline. The Nemotron models can be fine-tuned to excel in research synthesis and long-term tool calling. The core remains multi-step research (plan → gather → synthesize), web research (Tavily), and academic article search (Serper), as well as citation-supported reporting. Optionally, an ensemble layer and report refiner can be added for maximum report quality.

Key Ingredients in AI-Q

Four ingredients have been central to these results: Multi-agent architecture with evidence-based planning and specialized researchers, built on NVIDIA NeMo Agent Toolkit and LangChain DeepAgents. NVIDIA Nemotron 3 Super-tuned: approximately 67,000 SFT trajectories from a few seed datasets with research questions, filtered by a principle-based judge. Custom middleware for long-term reliability. The NeMo Agent Toolkit and LangChain middleware are extended with components that enhance reliability and robustness. Ensemble researcher and report refiner (optional): parallel pipeline outputs merged by an LLM, with a post-hoc refiner for maximum report quality. Each element is detailed in the following sections.

NVIDIA Nemotron 3 Super-tuned: Data and Training

A major factor in the results is a fine-tuned NVIDIA Nemotron-3-Super-120B-A12B model. We chose it for this workflow because it aligns well with multi-step agentic reasoning, tool usage, and citation-based report writing; fine-tuning on real research and synthesis trajectories makes it effective for planner, researcher, and orchestrator roles at scale.

Trajectory Generation

We collected research questions from several open-source datasets: approximately 17,000 questions from OpenScholar, 21,000 from ResearchQA, and 2,457 questions from Fathom-DeepResearch-SFT. We then generated about 80,000 trajectories for the entire workflow using the open-source GPT-OSS-120B model. Each trajectory covers the behavior of the planner, researcher, and orchestrator. Notably, these trajectories include real web research results from the Tavily and Serper APIs, allowing the model to learn to navigate and perform multi-step searches and syntheses on real data.

Principle-Based Filtering

Most trajectories were not completed in time or were halted due to exceeding the tool call limit, but for those that produced expected results, we applied additional filtering using the judge model. Completed trajectories were rated with the model nvidia/Qwen3-Nemotron-32B-GenRM-Principle, which predicts quality based on dimensions such as completeness, readability, accuracy, and relevance. After filtering, approximately 67,000 trajectories were retained for training.

Model: NVIDIA Nemotron-3-Super-120B-A12B

• Configuration: One epoch, 5,615 steps, approximately 25 hours on 16×8 NVIDIA H100 GPUs.

AI-Q Deep Researcher

The AI-Q deep researcher adopts a multi-agent architecture (Orchestrator, Planner, and Researcher) with iterative loops plan → gather → synthesize, citation management, and custom middleware for long-term reliability. An optional ensemble and report refiner layer can be activated for maximum report quality. The multi-agent design also serves as a long-context strategy: each sub-agent works within its own context window and only returns its synthesized output, so the orchestrator never sees the raw responses from the tools. This keeps the orchestrator's context focused and prevents lengthy and noisy search results from degrading its reasoning.

Orchestrator

Coordinates the entire research loop. Calls the Planner to produce an evidence-based research plan, then the Researcher multiple times with targeted research tasks derived from that plan. After the research, the orchestrator reviews the quality constraints of the plan, deploys targeted research to fill gaps, and drafts the long report. An optional refinement step makes modifications to the report based on the raw briefs from the researcher in a new context window—a second evidence retrieval point.

Planner

Operates in two phases. A Scout sub-agent first maps the informational landscape through broad searches. An Architect sub-agent then designs the research plan, including the report outline, targeted search queries, and quality constraints, while conducting its own research to validate structural choices. Evidence-based planning is essential for producing reliable, high-quality reports. Our planner knows the informational landscape before committing to a structure. It decides where to deepen and broaden based on what it has actually found, not on assumptions.

Researcher

Deploys multiple specialized sub-agents in parallel, each with a distinct perspective: Evidence Collector: facts, statistics, specific figures from authoritative sources. Mechanism Explorer: causal explanations, theoretical frameworks. Comparator: benchmarks, comparative data, trade-off analyses. Critic: counterarguments, limitations, failure cases. Horizon Scanner: recent developments, emerging trends. They share the same research tools but with different analytical frameworks. Diverse specialists researching the same topic often illuminate evidence that a single generalist would miss. The researcher synthesizes the specialists' results into a brief, unified, and cited document. An LLM then verifies this synthesis against the raw outputs of the specialists in a new context window, retrieving any relevant information.

Configuration-Driven Flexibility

Each component is interchangeable. LLMs, tools, and agent graphs can be configured via YAML. The planner, researcher, and orchestrator can each be powered by a different LLM. For the benchmark submission, a fine-tuned Nemotron 3 powers the researcher, which processes 4 times more tokens than the planner and orchestrator combined.

Custom Middleware for Long-Term Reliability

Each agent and sub-agent intertwines LLM and tool calls across many steps (often more than 32). At this scale, the system can fail in ways that short interactions never reveal. Our agent harness provides custom middleware to manage and mitigate these issues: Tool name sanitization: LLMs can hallucinate running tool names. This middleware applies model-based cleaning, alias resolution, and fuzzy matching to retrieve the intended tool. Reasoning-aware retry: LLMs with reasoning sometimes produce reflection tokens without tool calls or final responses, which could silently disrupt the agent's loop. The middleware detects this, preserves reasoning in context, and retries. Budget application: Each agent and sub-agent has its own tool call ceiling. When the limit is reached, the system...