AirLLM on Mac M4: Qwen3.8-27B Fails Due to I/O Limit

On a Mac Mini M4 equipped with 16 GB, an attempt to run Qwen3.8‑27B with AirLLM failed. According to the author, the software pathway and the targeted architecture are incompatible, and disk access becomes the limiting factor. The measured throughput drops to 16.4 seconds per token, approximately 219 tokens/hour, with 53.79 GB read per token.

Limited Throughput: 53.79 GB Read Per Token, 219 Tokens/Hour

AirLLM needs to read about 53.79 GB from the disk to generate each token. The measured time reaches 16.4 seconds per token, which translates to about 219 tokens per hour. Under these conditions, long responses requiring reasoning become very slow. According to the author, on Apple Silicon, the main limitation comes from input/output.

MLX Routing and Crash During Layer Counting

On macOS, AirLLM systematically redirects models to an MLX/Llama implementation, without calling the specific class for Qwen3.8. The substring separation logic mistakenly links thousands of tensors, and then execution fails during layer counting due to incompatible naming conventions, after about 55.56 GB has been downloaded. Even if the routing were corrected, MLX persistence produces structures that the torch streaming engine cannot read, and the Gated DeltaNet architecture would require a specific backend.

AirLLM Promises and Failure on Mac Mini 16 GB

AirLLM claims the ability to run 70 billion parameter models on a 4 GB GPU and states that Qwen3.8‑27B would only require 3.33 GB. The author attempted to launch Qwen3.8‑27B with AirLLM on a Mac Mini M4 with 16 GB of unified memory, but without success. Ultimately, he recommends using a smaller GGUF quant with the Metal backend of llama.cpp.