Cohere Unveils North Mini Code: A Revolution for Developers

An Innovative Model for Developers

Cohere has recently introduced North Mini Code, a cutting-edge model designed for developers. This model, which includes 30 billion parameters, utilizes a Mixture-of-Experts architecture with 3 billion active parameters. It is now available on the Hugging Face platform under the Apache 2.0 license. North Mini Code is specifically tailored to meet the needs of agentic software engineering tasks, offering advanced coding capabilities.

This model marks the beginning of a new series of models developed by Cohere, aimed at optimizing complex workflows in software engineering. It is particularly suited for tasks requiring terminal interaction and the generation of high-quality code. On the Artificial Analysis Coding Index, North Mini Code achieves an impressive score of 33.4, outperforming models such as Qwen3.5 and Gemma 4, as well as much larger models like Nemotron 3 Super and Mistral Small 4.

Proven Quality and Robustness

The quality and robustness of North Mini Code are essential for coding agents operating in varied environments. Cohere trained this model using multiple structures, making it particularly reliable for coding agents like OpenCode. North Mini Code is based on a Transformer architecture with a decoder, integrating efficient attention that combines sliding window attention with RoPE and global attention without positional embeddings, in a 3:1 ratio.

The model employs a MoE feed-forward block with 128 experts, of which 8 are activated per token. Each expert is structured in a FFN block with SwiGLU activation. A router applies a sigmoid activation function to the logits before top-k selection, and a single dense layer precedes the sparse layers.

Rigorous Post-Training

To achieve coding excellence, North Mini Code underwent a supervised fine-tuning (SFT) process in two stages, followed by reinforcement learning with verifiable rewards (RLVR). The first stage of SFT focuses on integrating coding capabilities into a broader mix to ensure robustness and usability. The data used includes 70% trainable tokens, 43% agentic tool usage data, and 27% competitive or scientific programming data.

The second stage of SFT utilizes a mix of 4.5 billion tokens from reasoning-focused samples, with 61% of tokens being code. Cohere's data pipeline relies on containerized agentic coding environments, using over 70,000 verifiable tasks from 5,000 unique repositories. Context lengths of 64K and 128K are used respectively for the first and second stages of SFT, allowing for "long-to-long" cascade training.

Adaptability in Diverse Environments

North Mini Code's ability to adapt to different environments enhances its effectiveness in various software development contexts. Cohere introduced a small amount of additional benchmark data (6% of the SFT mix) during the second stage of SFT, which improved performance by 10% with the OpenCode environment while maintaining compatibility with SWE-Agent on SWE-Bench Verified.

Training on data from hybrid environments revealed minimal data conflict, suggesting that the skills required by different environments are complementary.

Asynchronous Reinforcement Learning

Coding agent deployments often require varying durations, making synchronous learning ineffective. Therefore, Cohere opted for asynchronous reinforcement learning, using a trainer in parallel with a vLLM for continuous deployments. The model employs CISPO, a log-likelihood objective with token-level importance sampling correction.

Cohere has implemented multi-environment online training on two types of tasks: terminal-based and software engineering tasks. Each training batch includes 512 deployments, with 8 deployments sampled per prompt.