Google Cloud and AI Security: Between Challenges and Vulnerabilities

At a recent event in Los Angeles, Francis de Souza spoke about the current state of artificial intelligence (AI) security in enterprises. With a calm approach, he asserted that a transition period is inevitable before reaching a more stable and secure situation. This transition is also underway at Google.

De Souza's main message, which has resonated for years among security professionals, is that security cannot be an afterthought, especially with the rise of AI. He emphasized that companies must adopt a platform approach from the very beginning of their AI journey. "Security is not something you can add later, and it’s not something you can leave to employees to handle on their own," he stated.

De Souza warned against the phenomenon of "shadow AI," where employees use consumer AI tools without adequate oversight from the organization. He stressed that companies must require platforms to offer security, governance, and auditability from the outset. "There is no AI strategy without a data strategy and a security strategy. They must go hand in hand," he added.

Although he mentioned Google Cloud, de Souza clarified that he was not solely promoting this service. In response to a comment about the promotional nature of his advice, he explained that Google is committed to a multicloud approach. According to him, even companies that think they operate on a single cloud are actually using SaaS applications and collaborating with business partners who may use different clouds. "It is important for companies to have a consistent security posture across clouds, across models," he asserted.

De Souza also highlighted that the threat landscape has evolved so fundamentally that old defensive models are now too slow. He noted that the average time between an initial breach and moving to the next stage of an attack has decreased from eight hours to just 22 seconds. The attack surface has also expanded well beyond the traditional network perimeter. "In addition to your usual estate, you now have models. You have data pipelines used to train the models. You have agents, you have prompts. All of this needs to be protected," he explained.

One specific threat that de Souza pointed out concerns agents operating within a company's internal systems. These agents can discover forgotten data repositories that no one has accounted for in years. "Many organizations have old SharePoint servers and access controls that they haven’t really updated, but that didn’t matter because no one really knew where they were. But agents wandering through your company will find these data assets and expose the data contained within," he warned.

To address these threats, de Souza advocates responding at machine speed with machine speed. "We are now seeing the emergence of a fully agentic, AI-native defense, where organizations can run agents to drive their defense," he stated. "Instead of having a human-led defense or even a human in the loop, you can now have humans overseeing a fully agentic defense." He added that this has become a matter of leadership, not just a technological issue. "It’s a board-level issue and an executive team issue. It’s not just a security team problem."

However, even as AI takes on a greater share of the defensive workload, qualified individuals to oversee it are becoming scarce. The vulnerabilities that AI itself introduces are multiplying faster than security teams can address them. "We are going to need people to deal with the bug-pocalypse," said Lea Kissner, Chief Information Security Officer at LinkedIn, adding that she does not expect the industry to sustainably understand AI security for at least a few years.

This brings us back to the platform providers themselves. The Register has published a series of reports over the past few weeks documenting a wave of Google Cloud developers hit with five-figure bills due to unauthorized API calls to the Gemini models—services that many of them had never intentionally used or activated. The cases followed a familiar pattern: API keys initially deployed for Google Maps, publicly placed according to Google’s own instructions, had silently become capable of accessing Gemini after Google expanded their scope without clearly disclosing the change.

Rod Danan, CEO of the interview preparation platform Prentus, reported that his bill reached $10,138 in about 30 minutes after attackers exploited his compromised API key. Isuru Fonseka, a Sydney-based developer whose account was also compromised, woke up to charges of around AUD 17,000 despite believing he had set a spending cap of $250.

What neither of them knew is that Google’s automated systems had upgraded their billing levels based on account history, raising their effective caps to as much as $100,000 without explicit consent.

Google refunded both after The Register published its initial report. However, Google told The Register that it does not intend to change its policy on automatic level upgrades, stating that it prioritizes preventing service outages over enforcing users' declared budget preferences.

There is the separate question of what happens when a developer tries to shut everything down. The Register reported this week on research from the security company Aikido, finding that even developers who catch a compromised key and delete it immediately may not be safe. According to Aikido's findings, attackers can apparently continue to use that key for up to 23 minutes as Google’s revocation propagates through its infrastructure. Aikido researcher Joseph Leon told The Register that during this window, success rates are unpredictable—in some minutes, over 90% of requests are still authenticated—and attackers can use this time to exfiltrate files and cached conversation data from Gemini.

Leon also noted that Google’s new credential formats do not seem to have the same issue: service account API credentials revoke in about five seconds, and the new AQ-prefixed key format for Gemini takes about a minute. "Both work at Google scale," he wrote in Aikido's related document. "Both suggest that this can be technically resolved for Google API keys as well." In summary, according to Leon, the 23-minute window is not an engineering constraint but a matter of priorities for the company.

This is worth considering when reading de Souza's advice, which is sound and should be taken very seriously. He is not wrong, but there is currently a gap between what platforms prescribe and how quickly they adapt themselves, and it is good to be aware of that as well.