Google unveils two new TPU chips for secure and fast AI

Google introduces two new variants of its 8th generation TPUs. The 8tx targets confidential training and inference with hardware security, while the 8i Max doubles the memory, compute, and bandwidth of the 8i. These launches come as hyperscalers develop their own chips to reduce their reliance on Nvidia.

Cloud giants design their own chips, Nvidia under pressure

Microsoft, Amazon, OpenAI, and Anthropic are developing their own AI-dedicated semiconductors and associated software to lessen their dependence on Nvidia. Google is also strengthening its chip manufacturing strategy to limit the use of external components in its cloud business. This trend occurs against the backdrop of Nvidia facing a challenging period, after its CEO Jensen Huang acknowledged having an affair with someone suspected of espionage for China.

TPU 8tx: confidential training and inference with AMD

The TPU 8tx, designed in collaboration with AMD and based on the TPU 8t, is part of the eighth generation of chips introduced in April. Google states that this chip is intended for the confidential training and execution of AI models. Security is built directly into the hardware layer to protect and isolate sensitive data and model weights, even during use. The TPU 8tx features a custom inference network that allows reasoning to occur in parallel with training, enabling distillation and training of smaller models on the fly. According to Google, this capability can be useful for training a model on proprietary data without having to transfer that data between different devices.

TPU 8i Max: doubled memory and throughput for inference

The TPU 8i Max combines two inference-focused TPU 8i chips, launched in April, into a single component. This configuration doubles the memory capacity, processing power, and bandwidth compared to a TPU 8i, allowing AI models to process a broader range of information during queries. Google presented these 8th generation TPU variants to meet the needs of both confidential inference and workloads requiring increased memory.