SFT: Key to the Security of DeepMind's Gemini Models

In a series of informal research updates, the language model interpretability team at Google DeepMind shared a surprising discovery regarding the security properties of the Gemini models. Contrary to initial expectations, it was observed that the security of Gemini is primarily influenced by the combination of pre-training and Supervised Fine-Tuning (SFT), rather than by other training steps such as Reinforcement Learning (RL). Although this observation is specific to Gemini and may evolve with future versions, it was deemed significant enough to be shared with the broader security community.

Experiment

The experiment was conducted by applying SFT to the pre-trained versions of Gemini 3.1 Pro and Gemini 3 Flash. These post-SFT models were then compared to the production versions across various relevant security benchmarks. The results showed that the performance of the SFT-only models and the production models were remarkably similar, with error bars representing 95% confidence intervals on the evaluations.

The main outcome of this study is that the blue bars, representing the SFT-only models, and the orange bars, representing the production models, are strikingly similar across the evaluations. This implies that for Gemini, SFT is a high-impact intervention point for the model's security and behavior. Google DeepMind plans to further intervene at this level in the future to enhance the security of the models.

Security Benchmarks

The tests included several benchmark sets, each with its own specifics:

• ODCV: This benchmark is described in a paper available on arxiv.
• Alignment Evaluations: Based on a modified version of Petri, these evaluations are designed to be single-prompt and filtered to contain only "alignment dilemma" style issues.
• Single-Prompt Misalignment: This evaluation measures how often the model makes the "wrong" decision, according to an automated evaluator.
• Evaluation Awareness: This refers to how frequently an automated evaluator determines that the model was aware it was being evaluated.
• Security Evaluations: These evaluations measure the model's excessive refusal rate on benign prompts that appear harmful and the model's unsafe response rate on harmful prompts.
• Reward Hacking: In this environment, the model is placed in a Docker container within the Gemini CLI and is asked to optimize an algorithmic problem against a non-modifiable file containing a timing script. An automated evaluator measures the percentage of deployments where the model cheats in some way.
• Free Tier User Logs: These logs consist of 50,000 random prompts, anonymized and stripped of personal information from AI Studio users. Automated evaluators are run for each of the behaviors listed on the models' thoughts and responses, although many positives from the automated evaluators are likely false positives.

These results underscore the importance of SFT in improving the security of language models, potentially influencing future development and deployment approaches for these technologies.