Apple Silicon: The Hidden Energy Cost of Local LLMs Revealed

Apple Silicon: The Hidden Energy Cost of Local LLMs Revealed

Operating Cost of a Local LLM

A recent article from Towards Data Science, titled How Much Does It Actually Cost to Run a Local LLM?, highlighted the energy cost associated with using a local language model. The study revealed that the most expensive model to operate was not the largest one, but rather the one whose cost per token depended on throughput rather than the number of parameters.

I do not have an RTX 3090. I use a Mac Studio M3 Ultra with 96 GB of unified memory. Therefore, I wanted to see if the study's results also applied to my machine. I measured the energy consumption of five models in sustained generation, using the actual wall power and pricing it at my electricity rate of $0.31/kWh. The result: the surprise is even more pronounced on Apple Silicon. My 120 billion parameter model costs about five times less per token than a smaller model.

Measurement Methodology

The tool used for these measurements is called TokenWatt. It is an OpenAI-compatible proxy specifically designed for Apple Silicon. It sends each request to a local inference server while measuring energy consumption. TokenWatt reads the power rails of the chip via Apple's IOReport interface, without requiring special permissions or external instruments. It subtracts an idle baseline to measure the marginal cost of the request, and it evaluates the result according to your electricity rate.

To ensure the reliability of the measurements, I calibrated TokenWatt with a Shelly Plug US Gen4 that measures actual wall power. Each figure presented here comes with a real margin of error, ranging from ±2.6% to ±4.5%.

Results of Measured Models

The measurements were conducted in sustained generation loops for three durations (120, 360, and 720 seconds), with a 15-minute idle time between each run. Here are the results:

| Model | Parameters | Type | Footprint | tok/s | Cost ($/1M out) | |----------------------|------------|-------|-----------|-------|------------------| | Qwen3.5-4B | 4B | Dense | 2.9 GB | 133.8 | $0.063 | | Qwen3.6-35B-A3B | 35B | MoE | 35 GB | 76.0 | $0.087 | | Qwen3-Coder-Next | ~80B | MoE | 60 GB | 65.0 | $0.103 | | gpt-oss-120b | 120B | MoE | 59 GB | 74.0 | $0.109 | | Qwen3.6-27B | 27B | Dense | 28 GB | 21.5 | $0.554 |

Cost Analysis

The cheapest model is the smallest one, but the most expensive model, by a factor of five to nine, is the dense model with 27 billion parameters. The 120 billion model is five times cheaper. The operating cost of a model does not simply depend on the number of parameters, but on how the data is processed.

Why the Large Model is More Advantageous

The energy cost per token is determined by the power consumed and the generation throughput. The dense 27B model consumes 138 watts while generating only 21.5 tokens per second. In contrast, mixture-of-experts (MoE) models activate only a fraction of their parameters for each token, which reduces the amount of data moved and improves speed and energy efficiency.

Real-World Impact

A sustained loop benchmark does not always reflect the reality of daily use. I also recorded my actual inference requests over a month, totaling about 6,300 requests. The results show that the dense 27B model costs about ten times more per token than the MoE models.

Considerations Specific to Apple Silicon

• Memory Cost: On a unified memory machine, keeping a large model in memory consumes energy even between tokens.

• Memory Limits: My 120B model uses about 59 GB, which approaches the practical limit of my Mac. Machines with 192 GB or 256 GB could run larger models.

• Marginal Cost: The measurements only account for marginal electricity, not the total cost of the Mac, which is much higher depending on its usage.

These results are useful for comparing the operating costs of language models on Apple Silicon machines.