Huawei HiFloat4: AI Advancement Amidst Tech Restrictions

HiFloat4 from Huawei: A Significant Advancement in AI

Huawei has recently highlighted its new precision format, HiFloat4, which has been tested against MXFP4, a similar format developed by the Open Compute Project. This 4-bit format is designed for training and inference in artificial intelligence, and the results show that it outperforms its competitor. This advancement reflects the growing interest of Chinese companies in developing low-precision data formats that are specifically tailored to their hardware platforms.

The main objective of this innovation is to enable efficient pre-training of large language models (LLMs) using specialized AI accelerators while adhering to strict power constraints. Huawei is particularly focused on its Ascend neural processing units (NPUs), which are designed to handle deep learning workloads.

Researchers trained three types of models on Huawei's Ascend chips: OpenPangu-1B, Llama3-8B, and Qwen3-MoE-30B. They observed that the larger the models, the more HiFloat4 reduced the loss error compared to a full precision BF16 reference. In comparison, HiFloat4 consistently outperformed MXFP4.

The results show that HiFloat4 achieves a relative loss of about 1.0%, while MXFP4 records a loss of 1.5% compared to a full precision reference. For the Llama and Qwen models, HiFloat4 presents an error gap of less than 1% compared to the reference.

The Strategic Importance of HiFloat4

HiFloat4 represents a step towards even lower precision than HiFloat8, illustrating the ongoing effort by Huawei and other Chinese chip manufacturers to maximize the efficiency of their technologies. This initiative comes in the context of export restrictions that limit China's access to cutting-edge technologies, making the development of low-precision formats suited to their own hardware crucial.

Anthropic and the Automation of AI Research

In another area of AI, researchers from the Anthropic Fellows program have explored the possibility of automating artificial intelligence research. They successfully created autonomous AI agents capable of proposing ideas, conducting experiments, and iterating on open research problems, including how to train a powerful model using only the supervision of a weaker model.

The researchers tested this approach using generalization methods. Two human researchers spent seven days iterating on four of the most promising generalization methods, achieving a PGR score of 0.23.

After an additional five days of research, the AI agents nearly closed the performance gap, reaching a final PGR of 0.97. The cost of this research amounted to approximately $18,000 in tokens and model training fees.

How AI Agents Work

AI agents operate in independent environments but can communicate and learn from one another. They have access to common helper functions for model training and inference, and can autonomously propose hypotheses, design risk-reduction experiments, analyze data, and train models.

Limitations and Implications of Automation

While the automation of AI research appears promising, researchers emphasize that human intervention is necessary to maintain diversity in research directions. This raises questions about the ability of machines to effectively propose their own research directions, which could transform AI research and lead to the expansion of a machine economy.

Comparison of Chinese and American Models

A recent study compared the Chinese model Kimi K2.5 to American models such as DeepSeek V3.2, Claude Opus 4.5, and GPT 5.2. The results show that K2.5 exhibits similar dual-use capabilities to GPT 5.2, but with fewer refusals on CBRNE-related requests.

Regarding biological tasks, K2.5 shows a lower refusal rate. In terms of cyber capabilities, K2.5 is considered a decent model but not an expert. In terms of alignment, K2.5 displays less aligned behavior than GPT-5.2 and Claude Opus 4.5. Finally, the K2.5 model has a higher refusal rate on sensitive Chinese political topics compared to Claude Opus 4.5 and GPT-5.2 Pro.

Researchers also demonstrated that with a bit of computing power, they could reduce the built-in protections in Kimi K2.5, raising concerns about the security and use of these models.