NVIDIA NeMo Switchyard: 113 Channels Control Routing

The NeMo Switchyard router relies on a list of 113 chains and primarily confronts tool outputs, rarely the text entered by the user. Its confidence is calculated using a tanh function with a threshold of 0.5, which can lead to the ignoring of isolated signals. The entire system operates within a declarative architecture in TOML and can be reproduced in Python from the code constants.

The router interprets execution "debris," not the prompt

The router mainly reads traces and artifacts from an agent's execution rather than the initial prompt. Other approaches exist, such as prompt-based classifiers, learned routers, or gateway routing indexed on price and load. Operational reservations accompany this choice: the calibration of thresholds depends on a specific dataset, and the logic is sensitive to the transmission format of the turn depth. Activating an LLM classifier also adds costs and additional calls. Recommendations specify when to favor the scene router or, conversely, to resort to escalation modes or learned classification.

Heuristics and a 0.5 threshold guide the decision

The decision relies on hard-coded error severities and recovery heuristics, referred to as "rotation" and "exploration." The confidence score is calculated with a tanh function and a threshold set at 0.5, which imposes corroboration of signals. In this context, an isolated Python traceback may be disregarded. The agent loop's tempo sets the trade-offs at each LLM call, based on the observed tool traffic.

A Rust file and 113 chains drive the routing

The logic is based on 113 chains, of which only one is ever compared to the user prompt, and this phrase comes from Claude Code. The routing logic has been primarily hard-coded in a single Rust file that gathers twelve tables of static chains along with a compacting indicator. This process associates portions of chains with tool designations, shell commands, and tool results, and in the case of a compacted context, a single four-word phrase is linked to the conversation text. The architecture adopts a declarative approach using a TOML file that specifies clients, targets, and LLM paths, relying on a collection of algorithms and protocol conversion. The logic can be reproduced in Python by porting the functions and reading the tuning constants directly from the source.