Mistral Small 4: The AI Revolutionizing Coding and Reasoning

Mistral Small 4: The AI Revolutionizing Coding and Reasoning

What's New with Mistral Small 4?

Mistral Small 4 stands out for its ability to integrate three essential functions into a single model. Traditionally, different artificial intelligence systems were required to conduct conversations, perform complex analyses, or write code. Now, Mistral Small 4 allows for the simultaneous and effortless management of these tasks. It positions itself as a versatile assistant, capable of discussing, analyzing, and coding from a single access point.

Its innovative architecture, called Mixture-of-Experts (MoE), relies on a network of 128 experts. This system enables the model to select the four most suitable experts for each specific task, thereby optimizing its efficiency. Although the model boasts an impressive total of 119 billion parameters, only 6 to 6.5 billion are activated for a given request, which speeds up processing while reducing costs.

Key features include:

• Multimodal Input: Thanks to its vision module Pixtral, it can process both images and text.
• Long Context Window: Capable of handling up to 256,000 tokens of information, it is ideal for analyzing large documents.
• Open and Accessible: The model weights are available under the Apache 2.0 license, allowing for commercial use. It is open-source and accessible via APIs and partner platforms.
• Optimized Performance: Mistral reduces the time required to complete tasks by 40% and processes three times more requests per second than its predecessor.

Under the Hood: Architecture and Specifications

Mistral Small 4 combines a text decoder and a vision encoder. When an image is provided, the vision system Pixtral interprets it and transmits the information to the text model, which then generates a response. This design allows for a seamless fusion of visual and textual data.

Architectural details include:

• Decoder Stack: Composed of 36 transformer layers, with a hidden size of 4096 and 32 attention heads.
• MoE Details: 128 experts, with 4 activated per token, featuring a shared expert component to ensure consistency.
• Vision Component: The vision model Pixtral includes 24 layers and processes images with a patch size of 14.
• Vocabulary: The Tekken tokenizer has an extensive vocabulary of 131,072 tokens, supporting multiple languages and complex instructions.

Although the number of active parameters is relatively low, the overall size of the model requires significant memory. The 119B model requires large VRAM; the quantized version at 4 bits consumes about 60 GB, while the 16-bit version uses nearly 240 GB. This does not include the memory needed in the KV cache for long-context tasks.

Evaluations and Benchmarks

Mistral Small 4 does not just rely on a smart architecture; it also showcases quantified performance that supports its claims. The model emphasizes quality and efficiency, providing accurate responses with low latency and reduced costs, thanks to shorter outputs.

Efficiency: High Scores with Fewer Words

Across various benchmarks, Mistral Small 4 consistently shows a tendency to match or surpass leading models while using significantly fewer words.

• Mathematical Reasoning (AIME 2025): The model scores 93 in reasoning mode, equivalent to that of the much larger Qwen3.5 122B model. Its average output length in instructional mode is 3,900 characters, well below the nearly 15,000 characters of GPT-OSS 120B.
• Coding Tasks (LiveCodeBench): The model achieves a competitive score of 64, slightly surpassing GPT-OSS 120B (63). It produces code that is over 10 times shorter (2.1k characters versus 23.6k characters), demonstrating its efficiency in generating correct code without unnecessary verbiage.
• Long Context Reasoning (LCR): Mistral Small 4 receives a high score of 72, with an extremely short output of only 200 characters in instructional mode. This skill is remarkable for extracting responses from vast volumes of text.

A Generational Leap for Mistral

Comparisons with other models show that Mistral Small 4 represents a significant advancement over previous versions. It continually sets new internal standards for text and vision.

• Superior Reasoning: It outperforms Mistral models on challenging text tests with a score of 71.2 on GPQA Diamond and 78 on MMLU Pro.
• Visual Capabilities: The model also excels in vision tasks with a score of 60 in MMMU-Pro, surpassing earlier models like Mistral Small 3.2 and Medium 3.1.

Mistral Small 4 proves to be highly competitive and, in some cases, even surpasses larger internal models such as Magistral Medium 1.2 on demanding benchmarks when utilizing its advanced reasoning mode. This confirms that Mistral Small 4 lives up to its promise of delivering the best in reasoning and coding skills in a practical format.