Claude Opus 4.6: Leader in AI Code Maintainability

The generation of code by artificial intelligence (AI) is experiencing rapid growth, but it poses challenges regarding long-term maintainability. Not all generative models are equal in their ability to produce sustainable code. The phenomenon of vibe coding has exacerbated this issue, as AI agents tend to generate functional code without considering long-term constraints, often adding third-party dependencies or rewriting entire stacks. This can become a major obstacle for companies that rely on the underlying model.

Researchers from Sun Yat-sen University and Alibaba have studied the quality of code generated by different large language models (LLMs) over time. They found that these models do not all produce the same level of maintainability. According to their research, some models generate more maintainable code than others, which is crucial for businesses.

SWE-CI: A Benchmark for Maintainability

To assess the maintainability of code generated by LLMs, the researchers developed a new benchmark called SWE-CI. Unlike traditional benchmarks like SWE-bench, which focus on fixing isolated bugs, SWE-CI evaluates a model's ability to maintain software over the long term.

In this framework, an open-source software project extracted from GitHub is used. The AI receives an older version of the project and must evolve it to its current version, feature by feature, through successive modification cycles. On average, the gap between the two versions represents 233 days of human development. The final dataset includes 100 tasks from 68 mature Python projects recognized by the developer community.

The model must discover the necessary modifications on its own, analyze the gaps between its version and the target version, identify priority issues, and correct them. This process can be repeated up to 20 times per task. The results are scored using a metric called EvoScore, which values the quality of the code at the end of the process. A model that produces clean and well-structured code from the outset will have more leeway at the end, while a model that piles on patches will eventually collapse under its own weight.

Results: Claude Opus 4.6 Takes the Lead

The study's results place Claude Opus 4.6 at the top of the rankings with an estimated maintainability score between 0.85 and 0.90, significantly ahead of its competitors. Claude Opus 4.5 and GLM-5 follow with scores of 0.60 to 0.65. Other models like Qwen3.5-plus, MiniMax-M2.5, and Kimi-K2.5 are around 0.45 to 0.50. DeepSeek-V3.2 and GPT-5.2 are in the middle of the pack.

In terms of the ability to avoid regressions, Claude Opus 4.6 succeeds in 76% of tasks, while Claude Opus 4.5 reaches 51%. Kimi-K2.5 and GLM-5 peak at 37%, and GPT-5.2 drops to 23%. The majority of the tested models fail to preserve existing code in more than 80% of cases.

The Importance of Human Feedback

These results highlight the significant gap between the best model and the others, particularly regarding the ability to avoid degrading existing code. Models launched after 2026 show a marked improvement, indicating that providers are beginning to optimize their models for maintainability.

However, the comparison remains incomplete, as several proprietary models have not been tested, such as Codex and Gemini. SWE-CI offers a benchmark that every company can use to evaluate its own models, and the dataset is available on Hugging Face. Despite the advancements, even the best model like Claude Opus 4.6 is not without flaws. Human oversight remains essential to ensure the quality of the generated code.