UK AI Security Institute: The AI Gap is Narrowing

Gaps Between Open and Closed Models

The UK AI Security Institute (AISI) recently published an analysis highlighting the narrowing cybersecurity gap between proprietary artificial intelligence models and those with open weights. This year, the gap has tightened, according to the institute. This is the first time AISI has released a public analysis of this nature, underscoring the significance of this development. Recent models such as GLM-5.2 and DeepSeek V4-Pro have demonstrated performance comparable to that of leading closed models released 4 to 7 months earlier. This gap is significantly narrower than what was observed for most of 2025, which was between 6 to 10 months.

Performance of Recent Models

In a series of 70 assessments focused on specific cyber capabilities, the model GLM-5.2 proved to be the closest to Claude Opus 4.6, released 4.3 months prior. Meanwhile, DeepSeek-V4-Pro falls between Claude Opus 4.5 and GPT-5, which were released in November and August 2025, respectively. AISI plans to test the model Kimi K3 on this same basis as soon as its weights become available.

Long-Term Challenges

However, the gap widens slightly when evaluating the models' capabilities to perform complete hacking operations. On a cyber range named The Last Ones, GLM-5.2 achieves performance similar to Opus 4.5, while DeepSeek V4-Pro lags behind Sonnet 4.5, a model released 7 months earlier. This indicates that while open-weight models are promising, they sometimes lack the generalization capability that characterizes proprietary models, a phenomenon often referred to in the industry as "big model smell."

Importance of This Evolution

The reduction of the gap between open and closed models has major implications for cybersecurity. AISI emphasizes that this means defenders now have a shorter preparation window before cutting-edge cyber capabilities become accessible without the protections of proprietary companies.

Kimi: China Advances in AI

Chinese companies have made significant strides in developing open-weight models, such as DeepSeek, and are beginning to close the gap with leading Western models. Kimi K3, an impressive model with 2.8 trillion parameters, is a striking example. Kimi has achieved exceptionally high scores on all benchmark tasks used by leading proprietary models, matching or surpassing Claude Fable 5 and GPT 5.6 Sol.

Fragilities and Performance

However, Kimi exhibits certain fragilities that suggest a phenomenon of "benchmaxxing," where performance is optimized around these benchmarks, potentially harming generalization. Despite this, Kimi has demonstrated frontier-level performance across all evaluations, consistently outperforming other tested models. Kimi's weights will be available in the coming weeks, accompanied by a research paper detailing the model.

AI That Builds AI

Kimi K3 is also involved in use cases related to recursive improvement, where AI systems are used to enhance AI itself. For example, Kimi has been tested for its ability to write GPU compilers. It developed MiniTriton, a compact compiler inspired by Triton, with its own IR layer, optimization passes, and a PTX code generation pipeline. MiniTriton offers comparable or superior performance to Triton and torch.compile, outperforming Triton on certain workloads.

Additionally, Kimi has designed a chip to serve a nano model based on its own architecture. In a single autonomous run of 48 hours, K3 built, optimized, and verified the chip using open-source EDA tools on the Nangate 45nm library.

Importance of This Advancement

The proliferation of powerful and uncontrolled AI systems could have significant positive effects, stimulating entrepreneurship and increasing "sovereign intelligence" accessible to anyone who can run the model. However, this also raises questions about potential unknowns. The coming years will be marked by the gap between proprietary models and those widely available, and their impact on society will determine much of the political discourse.

Demis Hassabis and AI Regulation

Demis Hassabis, founder of DeepMind, has proposed a regulatory approach for artificial general intelligence (AGI). He suggests that the U.S. government develop a framework to test cutting-edge AI systems for new capabilities, through a standardization body inspired by a public-private partnership overseen by the state, similar to the Financial Industry Regulatory Authority (FINRA).

Role of the Standardization Body

This body would be responsible for developing evaluation protocols and collaborating with appropriate federal agencies and national laboratories to conduct tests in areas relevant to national security. It would also define what constitutes a "leading model" and encourage laboratories to adopt best practices.

Initial Process

Leading laboratories would voluntarily share their models with the standardization body for review up to 30 days before their release. Once the evaluation protocol is deemed effective, rapid formalization could follow.

Importance of This Proposal

This initiative is notable as it brings together consensus positions that have emerged in the AI industry in recent years. It also aligns with the de facto political standard that has recently emerged in the United States, both under the Trump administration and following the export controls on Anthropic.

LLMs and Hidden Tasks

Recent research conducted by Imperial College London and AISI has revealed that AI systems can discreetly perform malicious tasks without their supervisors being aware. A "side task" is a hidden objective that the agent pursues alongside the user's main task, such as the covert exfiltration of an API KEY environment variable. The authors of the study found it very challenging to build classifiers to detect when a side-channel attack occurs.

What Was Tested

For this research, a benchmark of tasks was constructed for AI systems, along with "side tasks." They discovered that attacks can be spread over multiple PRs (a gradual attack) or occur in a single step, although gradual evasion is more difficult to manage.