Google and Genie Transform Street View into Immersive AI Worlds

Google DeepMind recently announced a major breakthrough by combining its Genie world model with Street View images. This innovation allows for the creation of interactive environments generated by artificial intelligence, based on real locations. Thanks to the vast database of Street View, users can explore virtual worlds grounded in reality.

A Training Ground for AI

The feature is currently in an experimental prototype phase and is accessible to Google AI Ultra subscribers. It is limited to locations in the United States and still presents some graphical imperfections. However, it offers a realistic training ground for AI agents, robots, and autonomous vehicles, thanks to the integration of Street View data.

Users can choose a location on a map, select a visual style such as "Ocean World," "Desert Sands," "Stone Age," or "B&W Film," and describe a character. The Genie 3 model then generates an interactive world from actual Street View sequences. This technology relies on "Maps Imagery Grounding," an interface already used by developers to create AI visuals from Street View data.

Impressive Demonstrations

To illustrate the capabilities of this technology, Google showcased demonstrations such as a flooded Golden Gate Bridge and the Fort Worth Stockyards in Texas styled like the 1920s. These examples not only highlight the technology's potential but also its current limitations, with blurry textures and sometimes surreal transitions.

DeepMind's SIMA 2 agent is already using Genie as a training ground, while Waymo employs it to simulate realistic street scenarios for autonomous vehicles. The connection to Street View allows these training sessions to be anchored to specific real-world locations.

Demonstrations and Access

The initial demonstrations resemble a mix of a technology showcase and a game preview. Bilawal Sidhu, a former product lead at Google for augmented/virtual reality and 3D maps, published a series of test prompts showing what it looks like in practice. He raced a Formula 1 car in Google Maps colors on the Las Vegas Strip, rode as a squirrel on a scooter in front of the Palais des Beaux-Arts, piloted a boat on Lady Bird Lake in Austin, and walked through the White House using indoor Street View data.

Towards Global Expansion

The Genie project with Street View anchoring is being rolled out globally for Google AI Ultra subscribers, a premium service priced at $200 per month. Although the feature is currently limited to the United States, Google plans for future expansion. DeepMind views this project as a research prototype, and the team is already working on an improved version.

In summary, this innovation from Google DeepMind could redefine how AI agents and robots are trained, providing them with realistic environments grounded in real data.