Downloading an Open-Weight AI Model Involves Technical Choices

Downloading weights allows you to run an AI model at home, without a subscription and while keeping your data local. However, this choice comes with hardware constraints, varying licenses, and security concerns. Here’s an overview of the available options and precautions to consider.

Reduced Control After Release and Debate on Safeguards

Once the weights are distributed, the developer has limited leverage over subsequent uses. A company can, at best, cut access to an online service in case of abuse, but publishing a safer replacement does not update the already downloaded copies. The freedom to modify a model also facilitates the weakening or removal of protections. Pierre Stock, VP of Science at Mistral, believes that safeguards are necessary for the openness of weights and that they can be partially circumvented at the cost of an effort comparable to adjusting a model. Conversely, access to the weights provides independent researchers the opportunity to audit systems closely and directly test fixes. Pierre Stock emphasizes that the openness of weights fosters ecosystem progress and argues against a situation where a third party could cut access to a used model.

Local Use: Privacy, Performance, and Cost vs. Cloud

Running a model locally allows for exchanges without a subscription and keeps prompts and files off a provider's servers. The capabilities of an open-weight model cover email drafting, document summarization, local file analysis, and programming assistance, similar to cloud services. However, the speed and length of the context that can be processed depend on the model and machine resources. Computers with ample memory or a powerful GPU facilitate large models, with benchmarks being a Mac with 16 GB or 36 GB of RAM, while smaller models can run on 8 GB. Before purchasing a new machine, the cost should be compared to a $20 monthly subscription. Some models are also accessible via developer websites, such as DeepSeek and Kimi, at the cost of sending prompts to the provider. Additionally, hosted services like ChatGPT, Gemini, and Claude offer free plans. Open weights provide the ability to keep sensitive documents on the device and maintain a model even if the online version disappears.

Who Offers What: Chinese Dominance and US/EU Offerings

Chinese models dominate the rankings of Artificial Analysis for open weights, with DeepSeek, Kimi K3 from Moonshot AI, and Alibaba's Qwen3.8 family, all available for download. On the American side, OpenAI distributes distinct gpt-oss models separate from ChatGPT, Google offers Gemma, Meta announces Muse Glimmer and Muse Spark within Llama, and Nvidia releases Nemotron. Startups are enriching the offering, such as Reflection with Beam, whose weights are available, and Thinking Machines Lab with Inkling. In Europe, Mistral AI makes Mistral Large 3 and the Ministral 3 family downloadable, and offers Mistral Large 4, Le Chonk, with its weights. Not all of these models are suitable for a laptop: some require server-grade hardware, although smaller variants work on PCs, phones, or other devices.

What Does Weight Openness Bring and What Are the Risks

Downloading the weights of a model allows for local execution, provided you have compatible software and sufficient hardware. This capability extends beyond chatbots to image and video. Access does not equate to unlimited permissions or total transparency regarding the model's genesis. Even offline, a system can hallucinate and carry biases from its training. Publishers sometimes release weights without sharing training data, which limits external verification, and openness does not imply better performance than a closed model. Companies, including Microsoft, advocate for weight accessibility, arguing that it helps security teams respond to attackers using AI, following reports of potential restrictions in the United States targeting Chinese models.

Publishing Model Weights Does Not Guarantee Code Openness

Making weights available is not sufficient to qualify a project as open source according to the Open Source Initiative: the training and execution code must be shared, along with sufficient information about the data so that a qualified person can reconstruct a similar model. Open source guarantees use, study, modification, and sharing. The OSI does not require the publication of training data, while the Free Software Foundation believes that such data should be free to use and modify. Some licenses for open-weight models limit commercial use, which falls outside the OSI's scope, and copyleft clauses may require maintaining the same terms when sharing derivative versions.

Installing and Adapting: Consumer Tools and Permitted Adjustments

Applications such as LM Studio, Ollama, and Jan facilitate the installation and execution of open-weight models, via a chatbot-like interface on Mac or PC. When the license allows, additional training enables the adaptation of a model without starting from scratch. The ecosystem is also evolving towards embedded variations: Gemma models include versions for phones and laptops. Mistral offers a range that includes models specialized by use and device. According to Pierre Stock, audio models can even fit on a smartwatch.