Kimi K3: China Bets on Memory for AI

Kimi K3: An AI Model That Focuses on Memory

The Kimi K3 model from Moonshot AI, unveiled on July 16, has garnered attention due to its impressive number of parameters: 2.8 trillion. This figure places the K3 at the forefront of open-weight models, a category where no other model had previously reached such a level. In the field of artificial intelligence, models are often ranked by size, and the K3 falls into the so-called "3T" category, an unprecedented threshold for publicly accessible models.

This flagship model from Moonshot AI made a spectacular leap, increasing from 1 trillion to 2.8 trillion parameters in a single release. This jump far exceeds the previous record of 1.6 trillion held by the DeepSeek V4 Pro. While the number of parameters is often highlighted, it only tells part of the story of how the K3 operates.

A Strategy to Bypass American Restrictions

The increase in the number of parameters in the K3 could be interpreted as a response to American restrictions on computing capabilities. However, Moonshot AI's technical blog suggests a different approach: the K3 does not circumvent computational constraints but shifts them by exchanging computation for memory at nearly every level of its design.

This strategy deserves close examination, as computation and memory are not interchangeable constraints, and their accessibility varies for a Chinese lab.

The Memory Challenge in the Kimi K3 Model

Two main factors determine the cost of running a large AI model: the amount of computation needed to generate each word and the amount of model that must be ready and accessible at all times. The first factor is computation, while the second is memory. Export restrictions on chips initially hit China hard, and the design of the K3 appears to be an attempt to reduce the reliance on computation.

The main change introduced by Moonshot AI is a technique called "mixture-of-experts." Instead of using the entire model for each word, the K3 is divided into 896 specialized sections, using only 16 at a time, or about 1.8% of the total. This significantly reduces the computation per word. However, the memory remains unchanged, as the 2.8 trillion parameters must be loaded and ready for use at any moment.

To reduce the memory footprint, Moonshot AI trained the K3 to operate with a precision of four bits per parameter, instead of the usual sixteen. This method, known as "quantization-aware training," was applied as early as the fine-tuning phase. Moonshot AI claims this choice was made for "broad hardware compatibility," which seems to be a strategy to avoid using non-Nvidia silicon. The savings are significant: an independent analysis estimates that the model requires about 1.4 TB in this format, compared to 5.6 TB at full precision.

Kimi Delta Attention: An Innovation to Reduce Memory Costs

Another change, called "Kimi Delta Attention," aims to reduce another memory cost. When a model processes a very long document, it accumulates a cache of everything it has already read. At K3's announced limit of one million tokens, equivalent to several thousand pages, this cache becomes larger than the model itself in memory.

Moonshot AI is particularly clear about the commercial importance of this approach. The company has contributed to a caching code for the open-source vLLM service project, stating that this combination allows for competitive pricing of the K3 despite its size. Moonshot recommends running the K3 on 64 or more accelerators, connected closely enough to function as a single pool.

This approach is similar to Huawei's CloudMatrix systems, indicating that the real workaround lies in the ability to pool memory across many individual chips. Training-level computation cannot be assembled in the same way.

The Hardware Behind the Kimi K3

Moonshot AI's blog does not specify whether this pooling occurs on Chinese silicon. Chip-level testing has been conducted on Nvidia H200S and on a "GPGPU from an alternative supplier," which the company does not name. Other results have been evaluated on an Nvidia L20, the reduced card sold in China under export rules.

The blog does not clarify where the H200 hardware is located, and the U.S. House of Representatives passed a law in January to close the offshore cloud rental loophole that had allowed Chinese companies to access restricted accelerators remotely. Memory, not processing power, is where the Chinese chip industry is most behind.

During trade negotiations in August 2025, Beijing requested a relaxation of restrictions on wide-band memory rather than on lithography tools or access to TSMC, a clear signal of what officials consider truly constraining. Domestic production of this memory is projected to be around two million stacks this year, sufficient for about 250,000 to 300,000 Huawei Ascend 910C-class chips, while SMIC has wafer capacity for over a million.

Deployment of the Kimi K3 Model: Challenges and Opportunities

For companies in the region, the practical question is not whether the K3 tops a ranking, but whether an open-weight model of this size is deployable. Asian companies are turning to open weights for three reasons: price, data sovereignty, and coverage of regional languages. Banks and insurers across Southeast Asia have tested self-hosted open models to ensure that records never leave their own systems.

The weights will be available on July 27, and any organization that can afford the hardware will be able to download, modify, and run the K3 within its own walls. The question is how many can do so. Moonshot recommends serving the model on 64 or more accelerators connected as a single pool, and the weights alone represent about 1.4 TB in the format in which it is shipped, according to an independent analysis, before accounting for the memory needed to process a long document.

This is a data center commitment, not a server room. For most companies, the practical outcome is to rent dedicated capacity rather than own it. Nevertheless, this keeps data within the country and under contract, which most regional regulators require. What it does not deliver is independence from infrastructure providers, which initially attracted many buyers to open weights in the first place.

The Technical and Financial Challenges of the Kimi K3

The software is not ready either. The two main architectural changes in the K3 are recent enough that standard open-source tools for running models do not yet support them, and Moonshot says it is working with inference partners and open-source maintainers to align technical details before release. Teams planning a self-hosted deployment should consider the launch date and the date of use as different things.

The price of the K3 has also evolved. It is launched with a maximum reasoning effort as the only option, with lower modes to follow, so that long reasoning chains and retried steps add up quickly.

Performance and Limitations of the Kimi K3 Model

Arena placed the K3 at the top of its frontend code evaluation with 1,679 points, ahead of Fable 5, during blind tests by developers, as reported by Tom's Hardware. This result is real. It is also a benchmark in the field.

Moonshot itself is more measured than its headlines. The company states that the overall performance of the K3 is still below that of Claude Fable 5 and GPT 5.6 Sol, and lists three limitations: the generation quality can become very unstable if a harness fails to return historical reflection content, the model can make unexpected decisions on behalf of a user when intent is ambiguous, and it shows a notable user experience gap compared to Fable 5 and GPT 5.6 Sol.

Its own footnotes also reveal that Fable 5 encountered fallback solutions on 35% of tasks in Moonshot's SWE Marathon evaluation, which may have affected the measured score of that model. Everything else remains a first-party claim. No figures for the K3 can be independently verified until the weights are public.

Bank of America analysts, led by Alex Liu, stated in a note that the K3 shows that large-scale pre-training combined with architectural work can still yield significant gains for flagship Chinese models despite computational constraints, which is the sober version of the argument, and closer to what the blog supports.

The direction of evolution is not in dispute. July 27 is when we will discover how many K3 belong to this column.