Alibaba and Qwen 3: Local AI Accessible to All

The Rise of Large Language Models

Setting up your own large language model may seem complex, but the future is promising. Cutting-edge AI models are increasingly threatened by strict export controls or rising API costs. As this technology integrates into our daily lives, the open-source movement is not just a philosophical preference but a necessary mechanism to keep AI in the hands of ordinary users.

We are not yet at parity; proprietary models from major tech labs retain a significant advantage in terms of raw performance. However, we can hope that the gap will close quickly. An independent community of researchers and developers is tirelessly working to ensure that this technology is accessible to anyone with a computer. Today, the foundations for true democratization are already in place: you can run a highly capable model entirely on your own laptop. For this experience, I decided to find a large language model that could run entirely on my laptop and use it for the simple tasks I would normally assign to a large lab model.

Qwen 3 8B: A Powerful Model at Your Fingertips

We will install Qwen 3 8B on my MacBook Air, run it entirely offline, and finally have a language model on my own machine rather than in a distant data center. The Qwen model family has been trained by Alibaba (the Chinese company) and is fully open-source, available online for anyone to download. The model has 9 billion parameters and takes up about 6 GB of RAM when loaded.

What follows is a practical guide, from A to Z, for running a true local language model on an Apple Silicon Mac, including the necessary terminal commands. But before we open the terminal, it’s important to discuss the rationale behind this approach.

Why Install a Local Model?

Most of the time, cloud models are better and easier to use. I won’t pretend that an 8 billion parameter model on a laptop surpasses cutting-edge AI. That’s not the case, and I will continue to use massive cloud models for heavy tasks.

However, price fluctuations and sovereignty wars surrounding AI could make open-source and local models highly relevant for a future where access to technology makes a huge difference. Every time you use Claude or ChatGPT, you send your data to remote servers where access can be blocked at any time.

The notion of “digital sovereignty” is a grand phrase for a very ordinary desire: we want to own what reads our most sensitive thoughts, just as we own a physical notebook or keep money at home.

A local model addresses this need clearly in the AI world. Once downloaded, nothing leaves the machine. No API keys, no changes in terms of service, no discreet data retention policies. You can unplug the Wi-Fi card, and it continues to work. For the very sensitive part of your work, this can be worth the entry cost.

People love to say that local models “democratize” AI. I wish that were true, but we are not there yet. Running this setup still assumes that you own a €1,500 laptop with massive unified memory and that you are comfortable with the command line. It’s a narrow and fortunate slice of the world.

However, the trajectory is becoming more democratized. Two years ago, running a decent offline model required a dedicated workstation and serious technical pain. This weekend, it took me a few hours and 5 GB of disk space.

The Machine and Specifications

I built this on a MacBook Air M4 with 24 GB of unified memory and about 235 GB of free storage. It was a fresh start: no Homebrew, no Python environment nightmares.

The number that matters here is 24 GB. Apple Silicon's “unified memory” is the magic trick that makes Macs so exceptionally good for this. Since the CPU and GPU share exactly the same pool of memory, the massive weights of neural networks don’t need to be slowly transferred back and forth.

An 8 billion parameter model takes about 5 GB on disk and occupies about 6 GB in memory when loaded. On a 24 GB machine, that’s very comfortable. You could run a 14 billion parameter model and still keep dozens of browser tabs open. (If you have an 8 GB Mac, stick to 1.5 billion or 3 billion parameter models and close your other applications).

There are a dozen ways to run local AI, and most of them require you to worry about compilation flags and dependency trees. You shouldn’t have to do that.

Ollama is an open-source framework and tool that works simply. It’s a single binary that bundles a highly optimized model executor (llama.cpp using Apple’s Metal for GPU acceleration), a Docker-style model registry, and a local HTTP API. You install it, pull a model, and communicate with it. That’s it!

Step 1: Install Ollama (No Need for Homebrew)

Ollama comes as a standard macOS application in a zip file. The command-line interface (CLI) is secretly inside the application bundle, so we can set it up entirely by hand.

• Download the Apple Silicon version:

curl -L -o Ollama-darwin.zip https://ollama.com/download/Ollama-darwin.zip

• Unzip and move the application to your Applications folder:

unzip -o -q Ollama-darwin.zip
mv Ollama.app /Applications/

If you don’t know how to open the terminal, go to your Mac applications and search for “terminal.”

Step 2: Add Ollama to Your PATH

I didn’t want to fight with sudo permissions in /usr/local/bin, so I created a symbolic link to the CLI in a local directory I own — it’s just a handy shortcut to speed up installation and get the LLM running.

• Create a local bin directory and link the CLI:

mkdir -p ~/.local/bin
ln -sf /Applications/Ollama.app/Contents/Resources/ollama ~/.local/bin/ollama

• Make this permanent in your zsh profile:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc

• Apply it to your current shell:

export PATH="$HOME/.local/bin:$PATH"
ollama --version

Step 3: Start the Server

Ollama runs a lightweight server in the background to expose the API and manage your computer’s memory.

• Start the server and log the output:

mkdir -p ~/.ollama/logs
nohup ollama serve > ~/.ollama/logs/serve.log 2>&1 &

• Check if it’s active:

curl -s http://127.0.0.1:11434/api/version

If the above command returns a “version,” Ollama is set up!

Step 4: Pull the Model

It’s as simple as that:

ollama pull qwen3:8b

Go make yourself a coffee. The download is about 5.2 GB.

After running ollama list, you will see the model available for you.

Step 5: Interact with Your Computer's New Digital Brain

You have three distinct ways to interact with your new local model.

1. Interactive Chat (the easiest)

ollama run qwen3:8b

Running the above command will launch the interactive chat. In default mode, the model will display “thinking tokens,” something that is usually abstract and hidden in most commercial tools.

I will start by asking my local model what it thinks about open-source models.

1. On-the-fly Terminal Commands To interact with your local model, you can also provide the question outside of interactive mode:

ollama run qwen3:8b "write a python script that tells me how many vowels a word has"

Here’s the script that our local language model built:

# Ask the user for a word
word = input("Enter a word: ")
# Define the set of vowels
vowels = {'a', 'e', 'i', 'o', 'u'}
# Initialize a counter
count = 0
# Convert the word to lowercase and check each character
for char in word.lower():
    if char in vowels:
        count += 1
# Display the result
print(f"Number of vowels: {count}")

1. The HTTP API (for scripts and applications) You can also use this outside of terminal commands. If you are comfortable with Python, you can create any local script using your local model:

import json, urllib.request
req = urllib.request.Request(
    "http://127.0.0.1:11434/api/generate",
    data=json.dumps({
        "model": "qwen3:8b",
        "[prompt](/glossaire/prompt)": "Give me three uses of a local LLM.",
        "stream": False,
        "headers": {"Content-Type": "application/json"},
    })
)
print(json.loads(urllib.request.urlopen(req).read())["response"])

Here’s the model’s response after running this Python script: Sure! Here are three common and practical uses of a local LLM (large language model):

• Personalized Assistance and Productivity: A local LLM can act as a private AI assistant, helping with tasks like email drafting, scheduling, note-taking, and even coding. Since it runs locally, it preserves user privacy and does not rely on internet connectivity.

• Content Creation and Language Processing: You can use a local LLM to generate creative content such as blog posts, stories, scripts, or marketing copy. It can also assist with language translation, grammar checking, and text summarization.

• Custom Applications and Integration: A local LLM can be integrated into custom applications or workflows, such as chatbots, customer support systems, or data analysis tools. This allows for tailored solutions without exposing sensitive data to external servers.

Let me know if you would like examples of how to implement these uses!

Refining the Experience — Taming the “Thinking Tokens”

Qwen 3 is a hybrid reasoning model. By default, it generates a verbose block <think>...</think> describing its thought process before providing the final answer.