Alibaba and Qwen 3: Local AI Accessible to All

The Rise of Large Language Models
Setting up your own large language model may seem complex, but the future is promising. Cutting-edge AI models are increasingly threatened by strict export controls or rising API costs. As this technology integrates into daily life, the open-source movement is not just a philosophical preference but a necessary mechanism to keep AI in the hands of ordinary users.
Parity has not yet been reached; proprietary models from major tech labs retain a significant advantage in terms of raw performance. However, there is reason to hope that the gap will close quickly. An independent community of researchers and developers is tirelessly working to ensure that this technology is accessible to anyone with a computer. Today, the foundations for true democratization are already in place: you can run a highly capable model entirely on your own laptop. The experiment that follows consists of finding a large language model that can run entirely on a laptop and using it for the simple tasks normally assigned to a large lab model.
Qwen 3 8B: A Powerful Model at Your Fingertips
The model chosen is Qwen 3 8B, installed on a MacBook Air and run entirely offline: the language model lives on the machine itself rather than in a distant data center. The Qwen model family has been trained by Alibaba (the Chinese company) and is fully open-source, available online for anyone to download. The model has 9 billion parameters and takes up about 6 GB of RAM when loaded.
What follows is a practical guide, from A to Z, for running a true local language model on an Apple Silicon Mac, including the necessary terminal commands. But before opening the terminal, it’s important to discuss the rationale behind this approach.
Why Install a Local Model?
Most of the time, cloud models are better and easier to use. An 8 billion parameter model on a laptop does not surpass cutting-edge AI, and massive cloud models remain the right choice for heavy tasks.
However, price fluctuations and sovereignty wars surrounding AI could make open-source and local models highly relevant for a future where access to technology makes a huge difference. Every time you use Claude or ChatGPT, you send your data to remote servers where access can be blocked at any time.
The notion of “digital sovereignty” is a grand phrase for a very ordinary desire: users want to own what reads their most sensitive thoughts, just as they own a physical notebook or keep money at home.
A local model addresses this need clearly in the AI world. Once downloaded, nothing leaves the machine. No API keys, no changes in terms of service, no discreet data retention policies. You can unplug the Wi-Fi card, and it continues to work. For the very sensitive part of your work, this can be worth the entry cost.
People love to say that local models “democratize” AI. That is not yet the case. Running this setup still assumes that you own a €1,500 laptop with massive unified memory and that you are comfortable with the command line. It’s a narrow and fortunate slice of the world.
However, the trajectory is becoming more democratized. Two years ago, running a decent offline model required a dedicated workstation and serious technical pain. Today, it takes a few hours and 5 GB of disk space.
The Machine and Specifications
The setup was built on a MacBook Air M4 with 24 GB of unified memory and about 235 GB of free storage. It was a fresh start: no Homebrew, no Python environment nightmares.
The number that matters here is 24 GB. Apple Silicon's “unified memory” is the magic trick that makes Macs so exceptionally good for this. Since the CPU and GPU share exactly the same pool of memory, the massive weights of neural networks don’t need to be slowly transferred back and forth.
An 8 billion parameter model takes about 5 GB on disk and occupies about 6 GB in memory when loaded. On a 24 GB machine, that’s very comfortable. You could run a 14 billion parameter model and still keep dozens of browser tabs open. (If you have an 8 GB Mac, stick to 1.5 billion or 3 billion parameter models and close your other applications).
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
There are a dozen ways to run local AI, and most of them require you to worry about compilation flags and dependency trees. You shouldn’t have to do that.
Ollama is an open-source framework and tool that works simply. It’s a single binary that bundles a highly optimized model executor (llama.cpp using Apple’s Metal for GPU acceleration), a Docker-style model registry, and a local HTTP API. You install it, pull a model, and communicate with it. That’s it!
Step 1: Install Ollama (No Need for Homebrew)
Ollama comes as a standard macOS application in a zip file. The command-line interface (CLI) is secretly inside the application bundle, so it can be set up entirely by hand.
- Download the Apple Silicon version:
curl -L -o Ollama-darwin.zip https://ollama.com/download/Ollama-darwin.zip
- Unzip and move the application to your Applications folder:
unzip -o -q Ollama-darwin.zip
mv Ollama.app /Applications/
If you don’t know how to open the terminal, go to your Mac applications and search for “terminal.”
Step 2: Add Ollama to Your PATH
To avoid fighting with sudo permissions in /usr/local/bin, a symbolic link to the CLI is created in a local directory the user owns: it’s just a handy shortcut to speed up installation and get the LLM running.
- Create a local bin directory and link the CLI:
mkdir -p ~/.local/bin
ln -sf /Applications/Ollama.app/Contents/Resources/ollama ~/.local/bin/ollama
- Make this permanent in your zsh profile:
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc
- Apply it to your current shell:
export PATH="$HOME/.local/bin:$PATH"
ollama --version
Step 3: Start the Server
Ollama runs a lightweight server in the background to expose the API and manage your computer’s memory.
- Start the server and log the output:
mkdir -p ~/.ollama/logs
nohup ollama serve > ~/.ollama/logs/serve.log 2>&1 &
- Check if it’s active:
curl -s http://127.0.0.1:11434/api/version
If the above command returns a “version,” Ollama is set up!
Step 4: Pull the Model
It’s as simple as that:
ollama pull qwen3:8b
Go make yourself a coffee. The download is about 5.2 GB.
After running ollama list, you will see the model available for you.
Step 5: Interact with Your Computer's New Digital Brain
You have three distinct ways to interact with your new local model.
- Interactive Chat (the easiest)
ollama run qwen3:8b
Running the above command will launch the interactive chat. In default mode, the model will display “thinking tokens,” something that is usually abstract and hidden in most commercial tools.
A first test is to ask the local model what it thinks about open-source models.
- On-the-fly Terminal Commands To interact with your local model, you can also provide the question outside of interactive mode:
ollama run qwen3:8b "write a python script that tells me how many vowels a word has"
Here’s the script that the local language model built:
# Ask the user for a word
word = input("Enter a word: ")
# Define the set of vowels
vowels = {'a', 'e', 'i', 'o', 'u'}
# Initialize a counter
count = 0
# Convert the word to lowercase and check each character
for char in word.lower():
if char in vowels:
count += 1
# Display the result
print(f"Number of vowels: {count}")
- The HTTP API (for scripts and applications) You can also use this outside of terminal commands. If you are comfortable with Python, you can create any local script using your local model:
import json, urllib.request
req = urllib.request.Request(
"http://127.0.0.1:11434/api/generate",
data=json.dumps({
"model": "qwen3:8b",
"[prompt](/glossaire/prompt)": "Give me three uses of a local LLM.",
"stream": False,
"headers": {"Content-Type": "application/json"},
})
)
print(json.loads(urllib.request.urlopen(req).read())["response"])
Here’s the model’s response after running this Python script: Sure! Here are three common and practical uses of a local LLM (large language model):
-
Personalized Assistance and Productivity: A local LLM can act as a private AI assistant, helping with tasks like email drafting, scheduling, note-taking, and even coding. Since it runs locally, it preserves user privacy and does not rely on internet connectivity.
-
Content Creation and Language Processing: You can use a local LLM to generate creative content such as blog posts, stories, scripts, or marketing copy. It can also assist with language translation, grammar checking, and text summarization.
-
Custom Applications and Integration: A local LLM can be integrated into custom applications or workflows, such as chatbots, customer support systems, or data analysis tools. This allows for tailored solutions without exposing sensitive data to external servers.
Refining the Experience — Taming the “Thinking Tokens”
Qwen 3 is a hybrid reasoning model. By default, it generates a verbose block <think>...</think> describing its thought process before providing the final answer.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.