AMD Revolutionizes Local AI with Its Cloudless Mini PC

AMD recently unveiled a mini PC capable of running gigantic artificial intelligence (AI) models locally, eliminating the need for cloud services or monthly subscriptions. This development could represent a significant advancement for users looking to reduce costs while maintaining data privacy.

At the heart of this innovation is the Ryzen AI Max+ 395 processor, also known by its code name Strix Halo. This processor is integrated into machines like the GMKtec EVO-X2 and features unified memory of up to 128 GB. A portion of this memory is allocated to the graphics processor, allowing for up to 96 GB of VRAM. This configuration enables the loading of AI models much larger than those handled by most consumer graphics cards.

AMD claims that its mini PC can run models with up to 235 billion parameters locally, using free tools like Ollama. This means users can keep their data on their own machines, without sending it to a remote server, and without being constrained by subscriptions.

In tests conducted by AMD, this configuration outperformed an RTX 5080 in certain AI inference tasks, although this depends on the specific scenarios used. However, when compared to high-end cards like the RTX 5090, some AI models still run slower.

For businesses handling sensitive data, this solution offers a clear advantage: documents remain local, and costs associated with cloud services can be significantly reduced. A specialized consultant reported having replaced his expensive GPU rentals with this type of configuration, thereby improving the profitability of his operations.

Nevertheless, not everything is perfect. Performance varies depending on the models used, and the ROCm software, AMD's ecosystem for AI, still needs to catch up to CUDA, Nvidia's platform. Despite these challenges, AMD's mini PC demonstrates that running giant models locally is increasingly feasible on compact machines.