Generative AI: Demos Can't Replace Teams

Generative model demonstrations are captivating, but they do not address integration with existing systems, legal and accessibility constraints, or the complexity of deployment. In the face of unrealistic expectations, some advice encourages engineers to explain the limitations of LLMs, adjust certain processes, and preserve what is essential: delivering safe and useful software.

Staying the Course: Producing Useful and Safe Software Remains the Priority

The core mission of engineers remains the creation of safe and useful products for users. Engineers are seen as the best positioned to accomplish this mission, even in the era of generative AI. The goal is to prevent enthusiasm for AI from leading the organization to pursue inappropriate objectives. Some management teams want to believe that an engineering team could be replaced by a few prompts, an idea that some companies are also helping to propagate. Explaining the real limitations of AI tools helps avoid the misconception that a prompt notebook can replace an engineering team.

Attractive Demos, Systems to Deliver Under Constraints

A functional demonstration masks the real complexity and takes place under specific conditions, without real users. However, a demo does not reflect what is involved in a production environment. Transitioning to production requires integrating the product with existing systems that include millions of lines of legacy code, meeting regulatory obligations and accessibility standards, as well as managing often complex deployment. Code generation is only part of the work, and sometimes after a demonstration, some mistakenly believe that engineers are no longer necessary.

Core Skills: Fundamentals of LLMs and Internal Education

Engineers have a superior ability to understand how AI models work. It is useful to master certain fundamentals: LLMs are probabilistic, the concept of tokens is essential, and the quality of training data influences outcomes. It is not necessary to be a researcher to acquire this knowledge, which often places engineers ahead of most decision-makers in AI. This expertise should be shared: explaining the difference between a demonstration and a production system, and clearly outlining the limitations of the tools, helps avoid misleading shortcuts.

Climate of Pressure, Process Adjustments, and Team Voices

In a context of layoffs and restructuring, engineers face comments about the speed of ChatGPT or the creation of applications over a weekend. These situations were experienced in 2024 and 2025, a period marked by enthusiasm for AI and increasing adoption of Claude and ChatGPT. In the face of this pressure, some practices may be reevaluated when they have become mere formalities; acknowledging that certain processes are too slow can provide the necessary credibility to advocate for important steps. Even if the pursuit of speed at all costs is deemed harmful by the author, frontline engineers must explain and defend the rationale behind practices, as no one else will do it for them.