Google Launches TorchTPU, Amazon Pushes TorchNeuron Against Nvidia

Key Takeaways

1Google releases TorchTPU as open source to bring PyTorch to its TPUs and Google Cloud

2Amazon launches TorchNeuron for its Trainium chip and contributes to opening PyTorch to other hardware

3PyTorch has historically been tied to Nvidia GPUs, which still dominate the industry

Why it matters : These initiatives facilitate the migration to alternatives to Nvidia GPUs, but do not guarantee a massive shift in the short term.

Google has open-sourced TorchTPU and deployed tools to attract PyTorch projects to its TPUs and Google Cloud. The day before, Amazon released TorchNeuron for its Trainium chip and is working to decouple PyTorch from Nvidia hardware. The common goal is to facilitate porting, without guaranteeing a massive abandonment of Nvidia GPUs.

Google Focuses on Compatibility and Migration Assistants

Google claims to have collaborated directly with Meta, Hugging Face, Inferact, and RadixArk, as well as other players, to prepare TorchTPU. According to Google, the features and libraries of TorchTPU have been designed to work with these AI development tools, similar to the original PyTorch. vLLM is developed by Inferact while SGLang was designed by RadixArk. Google also offers the TPU agent suite, aimed at facilitating the migration of developers' applications to its own chips and cloud, while also supporting error correction. Additionally, LiteRT, which allows AI to run on phones, laptops, or embedded systems, has received an update to improve compatibility with PyTorch applications.

Porting PyTorch to TPUs and Trainium Without Heavy Rewriting

TorchTPU enables AI applications to run on Tensor Processing Units, Google's AI chips. It facilitates the transfer of existing PyTorch applications to TPUs and Google Cloud, thanks to a simple configuration change, with no rewriting required. Amazon, on its part, announced TorchNeuron on Monday, which aims for the same goal for its Trainium chip and AWS, also without rewriting applications. Amazon specifies that it is submitting code to the main PyTorch project to decouple certain features from Nvidia chips, making the tool more open to other hardware by default.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Why PyTorch Remains Nvidia GPU-Oriented

PyTorch is a widely used open-source solution among developers and startups for creating, training, and running AI models. According to some sources, it is used in 85% of AI applications. Meta launched PyTorch in 2016, at a time when Nvidia GPUs were already the benchmark for AI research, leading to the development of many functions specifically for these processors. In 2022, Meta entrusted PyTorch to the Linux Foundation, a nonprofit entity that oversees many major open-source projects.

A Power Dynamic That Doesn't Shift Overnight

The majority of AI applications are still designed to run by default on Nvidia GPUs. Amazon and Google are looking to reduce this advantage, but the ability to migrate easily does not mean that developers will massively abandon Nvidia hardware. Google is joining this dynamic by aligning with Amazon's efforts to challenge Nvidia's dominant position, with the announcement of TorchTPU as open source. TorchTPU is the version of PyTorch adapted by Google.