Mythos and llama.cpp: Revolutionizing Local Coding

Introduction to the Qwythos-9B-Claude-Mythos-5-1M Model

The Qwythos-9B-Claude-Mythos-5-1M model is a reasoning and coding tool equipped with 9 billion parameters. Based on Qwen3.5, it is specifically designed for local coding workflows, agent development, and tasks requiring extensive context. Its uniqueness lies in its ability to operate efficiently on consumer-grade hardware while providing sufficient performance for practical coding tasks.

Local Execution with llama.cpp

To run this model locally, we use llama.cpp. This guide explains how to install and configure this tool to run the Qwythos model enhanced by Mythos. The installation is done on an RTX 4070 Ti Super with 16 GB of VRAM, allowing for comfortable management of Q6_K MTP quantization. For users with an 8 GB GPU, it is recommended to start with the Q4_K_M variant, which offers a good balance between quality, speed, and memory usage.

Installing llama.cpp

Start by installing the official command-line interface for llama.cpp. This will give you access to the llama command, including llama serve, which we will use to run the model locally.

curl -LsSf https://llama.app/install.sh | sh

Next, add the installation directory to your shell path so that your terminal can find the llama command:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

To confirm that the installation was successful, run:

llama --help

If everything is installed correctly, you should see the available commands and options for llama.cpp.

Setting Up a Hugging Face Cache Directory

Set the Hugging Face cache to a location with enough free storage space. This is particularly useful on cloud machines where the default home directory has limited disk space.

export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"

To keep it across new terminal sessions, add it to your shell configuration:

echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
source ~/.bashrc

Starting the Qwythos MTP Model

Run the Q6_K MTP GGUF model with all available GPU layers:

llama serve -hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K" \
--alias "qwythos-9b-mtp" \
--host 0.0.0.0 \
--n-gpu-layers all \
--ctx-size 100000 \
--batch-size 1024 \
--ubatch-size 512 \
--flash-attn on \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--spec-type draft-mtp \
--spec-draft-n-max 6 \
--threads-batch 24 \
--repeat-penalty 1.05

The first time you run this command, llama.cpp will download the model files from Hugging Face. After that, it will load the model into GPU memory and start a local server.

Once the model is running, open the local interface in your browser:

http://localhost:8910

This also gives you a local API endpoint compatible with OpenAI at:

http://localhost:8910/v1

The Q6_K variant offers better output quality but also uses more memory. If you encounter VRAM or RAM issues, first reduce --ctx-size. If that is not enough, try the Q4_K_M variant instead.

Here are the most important options in the command:

• --n-gpu-layers all: Offloads all supported layers to the GPU.
• --ctx-size 100000: Allocates a context window of 100,000 tokens. Reduce this if you are low on VRAM or RAM.
• --cache-type-k q8_0 and --cache-type-v q8_0: Reduces KV cache memory usage while maintaining good quality.
• --spec-type draft-mtp: Enables speculative MTP decoding.
• --spec-draft-n-max 6: Allows up to six speculative tokens per step.

On my RTX 4070 Ti Super, I was getting about 81.74 tokens per second. In my test, the model was also able to reason through the task, call the necessary tools, and return the latest gold price when used in the local agent workflow.

Installing Pi and Integrating with llama.cpp

Pi can connect to the local llama.cpp server and use the model as a coding agent. The pi-llama plugin is designed to connect Pi to a running local llama.cpp server without needing an external API key.

curl -fsSL https://pi.dev/install.sh | sh

Install the llama.cpp plugin for Pi:

pi install git:github.com/huggingface/pi-llama

Create a small test project:

mkdir new-project

By default, the plugin looks for llama.cpp on port 8080. In this guide, our local server runs on port 8910, so we need to set the correct local API address before starting Pi:

export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"

In Pi, type:

pi start

Select the alias of the local model:

You should see the model listed in Pi, and once selected, you can start using it as a local coding agent.

Testing the Mythos-Enhanced Coding Model with Pi

It is now time to test the local model on real coding tasks within Pi. I used two simple yet practical tasks: a browser game and a small Python CLI tool.

Creating a Simple Browser Game

Start with a prompt asking Pi to create a small browser game called "Beat the AI".

• Create a simple browser game called "Beat the AI".
• The player has 30 seconds to answer short pattern recognition questions. Each correct answer increases the score by one. Display a countdown timer, a score display, a progress bar, and a final results screen.
• Use only HTML, CSS, and pure JavaScript.
• Generate at least three types of questions, such as numerical patterns, quick math, and verbal logic.
• Add a restart button after the game ends.
• Make the interface fun and neat.
• Keep all code easy to understand and avoid unnecessary files.

Test the game in the browser before finishing.

In my test, Pi created the complete game in a single HTML file with embedded CSS and JavaScript, keeping the project simple and easy to inspect.

Pi then summarized what it built, including the 30-second countdown, score tracking, progress bar, question types, and the restart button.

After that, I opened the game in the browser to check that it worked correctly.

The result was a neat little game with:

• A countdown timer
• Multiple types of questions
• A restart flow after the game ends

This is a good example of how the model can handle a complete front-end task locally without needing an external API.

Creating a Python CLI Tool to Convert CSV to Excel

For the second test, ask Pi to build a small Python CLI tool that converts a CSV file into an Excel .xlsx file.

• Create a simple Python CLI tool that converts a CSV file into an Excel .xlsx file, accepts input and output file paths as arguments, preserves column headers, validates missing files, and prints clear success or error messages. Before finishing, create a small dummy CSV file with sample data, use it to test the CLI tool, check that the Excel file is created correctly, and then summarize the test results.

Pi created a Python script named csv2excel.py, generated a sample CSV file, executed the test command, and then summarized the results.

python csv2excel.py sample_data.csv output.xlsx

In my run, the summary showed that the script:

• Accepted input and output file paths.
• Preserved column headers.
• Correctly converted the sample data.
• Handled missing files with clear error messages.
• Properly managed missing output paths.

I also opened the generated Excel file to confirm that the output was correct.

The sample output correctly preserved the rows and headers, confirming that the CLI tool worked as intended.

Final Thoughts

I truly appreciate this small local coding model. It is fast, sufficiently accurate for daily coding tasks, and does not require a huge amount of VRAM to be useful. You can use it with Pi, Claude Code, OpenCode, or any other coding environment that supports OpenAI or Anthropic-compatible endpoints.

For basic front-end applications, Python scripts, CLI tools, and rapid prototypes, it works surprisingly well. You can create useful projects locally without relying on external APIs or paying for each request.

For even better results, I highly recommend adding web search skills, Context7, and other useful integrations from Pi. You can also check out my comprehensive guide on optimizing your Pi coding agent setup here: How to Set Up Kimi K2.7 Code with Pi: The Ultimate AI Coding Environment.

Abid Ali Awan (@1abidaliawan) is a certified data science professional who enjoys building machine learning models. Currently, he focuses on content creation and writing technical blogs about machine learning and data science technologies. Abid holds a Master's in Technology Management and a Bachelor's in Telecommunications Engineering. His vision is to build an AI product using a graphical neural network for students suffering from mental health issues.