Brief IA
›

SusVibes: 11.8% of AI-generated solutions are secure

Use Cases·Tom Levy·

SusVibes: 11.8% of AI-generated solutions are secure

SusVibes: 11.8% of AI-generated solutions are secure
Key Takeaways
1In SusVibes, 11.8% of solutions were secure when agents did not receive security instructions.
2At Mimecast, according to Leslie Nielson, an identified person is accountable for every modification delivered with the help of AI.
3Darren Meyer, at Endor Labs, observes significant adoption among analysts, marketing teams, and researchers.
Why it matters : generated code can function while still harboring vulnerabilities.

In the SusVibes evaluation, programming agents without security guidelines produced secure solutions in only 11.8% of cases. However, their responses could be functional up to 57% of the time. For organizations, controls pertain to both imported components and human responsibilities, as well as applications created outside IT teams.

Budgets Lag Behind Software Development

Security budgets generally increase at a slower rate than software development, estimates Darren Meyer, research engineer at Endor Labs. According to him, the adoption of AI exacerbates this gap. Application security is often underfunded, while assisted programming accelerates development. Some decision-makers may still view the protection of AI systems as optional.

Darren Meyer believes that both quality tools and specialists in application security and products are necessary. These specialists must be able to collaborate with developers of various profiles and understand the technical, compliance, and security aspects of AI. The volume to be examined can exceed the capabilities of a fully manual review. Leslie Nielson, information security officer at Mimecast, an information security provider, warns that a reviewer overwhelmed with low-risk changes may end up validating without a thorough examination.

Darren Meyer also describes organizations divided between a strong commitment to AI and a marked refusal. These divisions can complicate the work of security leaders, including in the business sectors. Insufficiently explained restrictions risk being perceived as an impediment to innovation. Thomas Segura, senior technical writer at GitGuardian, advocates for a strategy that combines productivity with security requirements.

Imported Components Can Evade Controls

A dependency that is absent from the manifest, known as a "ghost," may not be detected by a scanner and could contain a vulnerability. Third-party libraries also represent a point of vigilance. Software composition analysis and tools dedicated to the supply chain can help identify the components used, their flaws, and the associated legal or compliance difficulties.

Alex Ștefănescu, a security expert and technology activist, warns about packages created by agents or completion functions, particularly in GitHub Copilot. A statistically plausible suite following an import instruction does not guarantee that a package exists on PyPI, he explains. According to Alex Ștefănescu, attackers register suggested names on npm or PyPI and add functions that give the packages a credible appearance. Importing them into an application can cause damage.

Machine learning models and datasets downloaded from Hugging Face or other platforms also present risks, regardless of code generation. According to Alex Ștefănescu, a backdoor may persist despite checks at both ends, activating during model integration and causing a leak of company information. The use of programming assistants can also lead to the inadvertent insertion of sensitive data into codebases.

Security leaders can establish procedures for downloading and integrating these external resources, then document them in the agents' guidelines when relevant. These procedures may include automatic searches for malware or backdoors. They can also limit allowed models to those from verified publishers or impose internal testing in isolated environments.

La lettre du soirGratuit · 19 h

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Mimecast Reserves Approval for People

At Mimecast, every change delivered with the assistance of AI is attributed to a person identified in the integration request, claims Leslie Nielson. Validation is exclusively in the hands of humans. He recommends combining automated checks with human assessment: his company automates repetitive checks on dependencies, secrets, and known vulnerability patterns. Access, authentication, and sensitive data, however, always go before a specialist, even in the absence of an automatic alert.

Bianca Lankford, vice president of engineering at Datadog, advises against granting prior trust to generated code. Security teams are already bringing static analysis, secret detection, and dependency checks closer to each integration request, even to the developer's publisher. According to her, some companies define authorized assistants, restrict their access, and train reviewers to spot outdated components, poorly validated inputs, and overly permissive settings.

For Bianca Lankford, the involvement of a specialist should depend on risk, not the origin of the code. She cites authentication, cryptography, payments, personal data, and infrastructure, with possible automated guidance based on code responsibility rules. The scale of changes, the addition of dependencies, and changes in permissions or network exposure are other signals.

Alex Ștefănescu distinguishes between software production and understanding its operation. According to this expert, maintained documentation, robust testing, and written records of significant architectural decisions can save time during a security incident or complex diagnosis. These resources should be produced and maintained by humans for other humans, rather than entrusted to automatic generation.

Business Teams Develop Their Own Applications

IT departments are increasingly turning to business users to create their applications, while Claude Code, Cursor, GitHub Copilot, and Amazon Q Developer are among the main assistants used by developers. Darren Meyer observes significant adoption among data analysts, marketing teams, and researchers. These teams, traditionally unfamiliar with developing their own software, are creating more small tools that integrate libraries and AI models, often without understanding the risks.

An IT manager recounted on Reddit his discovery of a young graduate practicing "vibe coding" several months after hiring him. According to his account, the young graduate had designed a web application under a domain he controlled, utilizing customer data and private contracts. The free offering from Supabase was used for data storage, while his GitHub repository directly fed the deployment on Cloudflare's serverless infrastructure.

The manager claims that the employee was working directly in production, with servers distributed around the world, while the company was located in the European Union. When questioned about code review and precautions taken, the employee reportedly only cited the private nature of the GitHub repository. The CEO then dismissed the IT manager's concerns.

Darren Meyer believes that this development outside of usual channels, combined with limited knowledge of application security, increases risks. Security leaders can reach out early to the relevant teams and consider tailored training for analysts, marketing professionals, and researchers on the dangers of AI tools and libraries.

SusVibes Tests the Security of Functional Code

SusVibes is based on 186 real feature requests from open-source projects. Danqing Wang from the Language Technologies Institute at Carnegie Mellon University, who participated in this evaluation, explains that the observed failures were not mere oversights. Detecting them required considering the possibilities of an attacker, beyond the expected functionality.

Katie Paxton-Fear, a security specialist at Semgrep, believes that the improvement of assistants in producing functional programs encourages developers to trust them more and reduce their vigilance. The volume of code produced contributes to security leaders' concerns. AI can identify defects in software, but it also introduces its own errors when writing code.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.