Goldman Sachs: AI for Collective Productivity

A New Vision of AI at Goldman Sachs

Marco Argenti, the Chief Information Officer of Goldman Sachs, is leading a major transformation within the bank. With a team of 12,000 engineers, Goldman Sachs has integrated artificial intelligence tools into its workflows. Argenti compares the impact of AI to a "3D printing" of software, allowing for real-time prototype creation, which revolutionizes how engineers approach software development.

While many American companies are looking to measure their employees' use of AI, Goldman Sachs is taking a different approach. For example, JPMorgan uses dashboards to monitor AI-related activities of its employees, while Meta installs software to track keystrokes and mouse movements. These methods aim to train their AI systems by analyzing human interactions.

Team Performance Takes Priority

At Goldman Sachs, Marco Argenti prefers to evaluate the speed of teams using AI rather than focusing on individual performance. According to him, concentrating solely on individuals is akin to "missing the forest for the trees." He oversees a rapid transformation in how developers design software, emphasizing the speed at which an idea turns into a finished product.

Although the bank has access to data on individual tool usage, it prioritizes an inter-team overview. This approach aims to accelerate project timelines, improve quality control, and manage AI token consumption within budgetary constraints. Unlike others, Goldman has not implemented dashboards to compare AI adoption rates among colleagues.

The Importance of Collaboration

Argenti explains that measuring productivity at the individual level can be misleading. He compares it to observing a single player on the field without considering the entire team. For him, efficiency lies in collaboration and passing the ball, rather than in isolated performances.

Measuring developer productivity has always been a challenge for companies. Argenti emphasizes that there is no single, magical metric. For instance, counting lines of code is not a reliable measure of productivity. He compares this to a physical training program where vital signs are more revealing than isolated numbers.

Managing AI Token Costs

Another crucial aspect is managing costs related to AI tokens. Argenti notes that if token usage increases without an impact on output, it often means the company is still in an experimental phase. Goldman Sachs has identified a threshold where productivity begins to rise after a certain level of token consumption.

Engineers often use tokens to plan and create requirements documents before coding. This preparatory phase does not immediately produce code, but once the plan is established, code production accelerates, leading to increased token consumption.

Growing Enthusiasm for AI

According to Argenti, enthusiasm for AI has surpassed the initial fear among engineers. During a recent innovation meeting, he observed a sense of empowerment and liberation among participants. Although some were skeptical at first, this sentiment was often linked to a lack of understanding of the tool.

Towards a Real-Time Prototyping Culture

The impact of AI is also evident in how engineers present their work. Rather than relying on PowerPoint presentations, they bring concrete solutions and prototypes of new products. This ability to prototype in real time allows for instant modifications during meetings, transforming the dynamics of idea presentation and validation.

Today, Goldman Sachs engineers can modify a product in front of their colleagues, eliminating the delay between idea and prototype. Argenti describes this process as a "3D printing" of software, where creation and iteration occur almost simultaneously.

Debate on Tracking AI Usage

The debate on tracking AI usage is lively. As a manager, Argenti believes it is more effective to focus on metrics at the team level. Work is often carried out by hybrid teams, composed of agents and humans, and it is crucial to measure the speed of feature development.

By examining the workflow, it is possible to see how long it takes to go from idea to production. A team may have a certain backlog, but as soon as it starts to reduce that backlog, it indicates improvement.

Analyzing AI Effectiveness

To analyze the effectiveness of AI, it is essential not to limit oneself to individual metrics. Argenti compares this to a training program where changes in vital signs are observed rather than focusing on isolated numbers. If the indicators show improvement, it means the team is on the right track.

Token Costs and Tangible Results

The skyrocketing costs of tokens is an important topic. If token usage is high without an improvement in output, it indicates an experimental phase. Once a certain threshold is crossed, productivity begins to increase, showing that efforts are paying off.

Engineers use tokens to create implementation plans and requirements documents, which do not immediately produce code. However, once the plan is established, code production accelerates, demonstrating a correlation between token usage and tangible results.