Boris Cherny Clarifies Requirements for Validating AI-Generated Code

Key Takeaways

1Boris Cherny distinguishes two approaches for AI-generated code: tolerance for the "black box" in low-risk prototypes, increased requirements for production

2At Anthropic, numerous safeguards are in place for production code: linting, testing, daily fuzzers, automated reviews

3The developer remains responsible for quality and can ask Claude to rewrite the codebase or wait for a new model

Why it matters : Regulating AI-generated code is becoming crucial to ensure the maintainability and reliability of enterprise software.

Faced with a developer's questions about the "friction" of AI-assisted coding, Boris Cherny offers a guiding principle: a tolerated black box for low-impact prototypes, but heightened requirements and multiple checks for production. He describes the protections used at Anthropic and reminds that developers are responsible for quality.

For production code, higher requirements and safeguards

For production code generated by Claude, Boris Cherny sets a higher standard than what is expected from a human. He describes, at Anthropic, an arsenal of protections: numerous linting rules, various tests, end-to-end tests driven by Claude, fuzzers powered by Claude run daily, as well as automated code reviews and security reviews. He warns that the absence of these safeguards exposes one to code that is difficult to maintain. According to him, the model increasingly facilitates these tasks through daily routines and Claude Code Review.

Prototypes: accepting the black box when the risk is low

Boris Cherny believes there is room for two approaches. For prototypes and other code intended to be discarded, he is willing to treat them as black boxes when the consequences of a failure are low, without aiming for perfection. This position contrasts with the more permissive idea of considering all code as a black box by only checking the output.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

A debate fueled by AI coding "friction"

A developer wrote to Boris Cherny, the creator of Claude Code, to highlight the "friction" caused by AI-assisted coding. Cherny published the full response after sharing the email he received on X on Thursday, titled "What to do about inefficiency?". He indicates that he receives such messages daily and tries to respond as much as possible. The developer asked him to arbitrate between two visions: a moderate use that speeds up coding while remaining reviewable and explainable, and an approach that treats code as a black box. He reports that many struggle to follow the moderate option for fear of their jobs, to the point of "forcing" practices. In the background, AI has changed the tasks of engineers, with some finding ease in it, while others experience it as a more solitary and tedious endeavor.

The developer, guardian of quality and possible outcomes

For Boris Cherny, the developer's mission is to maintain a level of code quality. AI leaders speak of a "mini CEO" role. He calls for acting as a manager, capable of stopping bad code. As a last resort, he suggests getting more involved with Claude, asking it to rewrite the codebase if necessary, or waiting for the next model.