Mistral: the Shieldstral Model Redefines AI Security

Mistral: the Shieldstral model redefines AI security

Mistral's open model Shieldstral, with 3 billion parameters, competes with much larger security models while being significantly lighter. According to a new paper, it is proposed to replace fixed security categories with yes or no questions that operators can define in real-time without needing to retrain the classifier.

Mistral claims that Shieldstral matches models three times its size on standard text security benchmarks and sets a new record for joint text and image classification.

Real-time rules to customize security checks

Many security models sort content using fixed taxonomies. The authors of the paper, including Mistral co-founder Guillaume Lample, highlight two issues with this approach:

• Public security datasets group risks too disparately to support a common taxonomy.
• The same rules do not fit all use cases. For example, content suitable for a cybersecurity tool could be harmful on a mental health platform.

Operators define review criteria in plain language. Shieldstral returns a token, which produces a security score between zero and one.

Synthetic data to manage new rules

Researchers combined about 54.1 million examples covering security, harmful content, and manipulation attempts into a unique format. They applied strict standards for targeted manipulation, moderate standards for general security data, and more lenient standards for response quality.

Each training example includes task instructions, a specific yes or no question, and the content to be reviewed. The response is a single token.

To teach Shieldstral finer distinctions, the team used another language model to rewrite safe text into dangerous variants. Each example also included a similar but different category that had to be rejected, allowing the model to separate closely related rules rather than making a broad judgment of safety or unsafety.

The authors created the adaptability test categories separately from the training set, using different names and levels of detail. None of the detailed test categories directly correspond to a training category, although 10 of the 12 broader classes have approximate equivalents.

The 3B model competes with a model nearly seven times larger

On combined text benchmarks, Shieldstral achieves an F1 score of 84.9%. The F1 score combines precision and recall into a single metric, with 100% representing a perfect score. This result matches that of GPT-OSS-Safeguard-20B, which is about seven times larger, and surpasses Qwen3Guard-8B at 84.0%, Nemotron-3.5-Safety-4B at 83.3%, and LlamaGuard-4-12B at 69.1%.

For images and text-image combinations, Shieldstral scores 83.8%, ahead of OmniGuard-7B at 77.6% and LlavaGuard-7B at 71.6%.

On text benchmarks, Shieldstral matches an OpenAI model approximately seven times larger.

Adaptability performance

GPT-OSS-Safeguard-20B leads the adaptability benchmark with 94.1%, compared to 91.3% for Shieldstral. This test uses rules that differ from the training categories or are entirely new. However, the authors consider Shieldstral to be more practical than GPT-OSS-Safeguard-20B and Nemotron-3.5-Safety, as the latter generate long sequences of intermediate reasoning that increase computational costs, while Shieldstral returns a single word.

On policies absent from training, Shieldstral lags behind the two models that generate intermediate reasoning before responding.

A model based on Ministral-3B

Shieldstral is based on Mistral's Ministral-3B with the vision encoder Pixtral. In a validation test with detailed categories, synthetic category data boosted the F1 score by 23.3 percentage points, which researchers believe is the main driver of the model's adaptability to new rules.

Shieldstral is available as an open-weight model under the Apache 2.0 license.

Custom rules to limit costly filtering errors

Security classifiers sit on either side of the main language model, filtering requests before processing and responses before they reach users. Operators can update these rules without retraining the main model. However, as each request passes through the classifier, its size, speed, and cost accumulate quickly.

Anthropic's Claude Fable 5 has demonstrated how poorly tuned filters can affect actual usage. An artificial analysis revealed that the system automatically redirected 8 to 9% of tasks to a weaker model. A medical physicist deemed Fable 5 unusable because their work often includes the word "nuclear." Other users reported that the system flagged MRI analysis as bioterrorism.

Anthropic tightened the filter after identifying a security issue and claims it has since blocked harmless coding tasks more frequently. Shieldstral gives operators more control over this trade-off. They can draft filtering criteria in real-time and tailor the filter to a specific application instead of adopting someone else's categories.

These classifiers are already playing an increasing role in the industry. OpenAI uses them for automatic age detection in ChatGPT and routes emotional requests through a security filter to stricter models. Claude Code uses a classifier to block external scripts, production deployments, and forced pushes. The review of Fable 5 by Anthropic also requires the company to store inputs and outputs for up to 30 days, or up to two years after rule violations.