Fine-Tuning: How Llama and Mistral Learn New Tasks

Understanding Fine-Tuning for Beginners

What is Pre-training?

Imagine you start with a newly created model, equipped with millions or even billions of parameters initially assigned random values. If you want to teach it a specific task, like categorizing movies into different genres, it would first need to learn the entire English language from scratch. This proves impossible, especially if you only have a limited dataset. It's a bit like trying to teach biology to a child who hasn't yet mastered the basics of language or fundamental scientific concepts.

Pre-training solves this problem by allowing the model to learn complex and general elements all at once, relying on an immense amount of data. This phase requires considerable computational and data resources. Once this step is completed, you have a model that has already absorbed the language. During this phase, the model learns a simple skill: predicting the next word. It is presented with a snippet of text with a hidden word, and it must guess what comes next. Correct predictions result in a small loss, while incorrect ones lead to a larger loss, prompting the model to adjust.

For example, if presented with the sentence "The cat is sitting on the ____", the model learns that "rug" is a far more likely answer than "car." Repeating this training across billions of sentences, books, and articles allows the model to become an excellent predictor of subsequent words while absorbing grammar, facts, reasoning patterns, and much more. After pre-training, the model already understands the language, which allows for building each subsequent task on this foundation rather than starting from scratch. This is why these models are often referred to as foundation models.

In general, you do not pre-train a model yourself. You download the final result, a pre-trained model like Llama, Mistral, or Qwen, and go from there. This brings us to our main topic: fine-tuning.

What is Fine-Tuning?

Many beginners think that once a model has been trained, its parameters are fixed forever. In reality, having a pre-trained model means that the parameters have been adjusted to "good values" that encode intelligence and work well for general tasks. Once you have this model, you can adapt this intelligence to your specific needs using task-specific data — this is called fine-tuning. The data requirements at this stage are much lower than for pre-training, as you only need examples for the task that interests you.

This is very similar to training chefs in a culinary school: once they join a restaurant, they acquire skills specific to that establishment. Since we are not starting from scratch here, it is less costly — much like training someone who has already graduated from culinary school is less demanding than training someone who is completely new. The diagram below summarizes the difference between pre-training and fine-tuning.

How Does Fine-Tuning Work?

We have already discussed predicting the next token and the pre-training process. Now, let's examine the fine-tuning loop.

You present the model with a task-specific data example — for instance, a movie — and ask it to categorize it. It makes a guess, which you compare to the ideal answer. You slightly adjust the parameters, then repeat the process until it improves at the task in question. Two major elements distinguish fine-tuning from pre-training:

• Data → It is small, high-quality, and task-specific, unlike the entirety of the internet.

• Learning Rate → It is low, with few passes, because we want the model to adapt without losing its general skills.

Two Common Types of Fine-Tuning

Although definitions vary online depending on how many parameters you want to adjust, fine-tuning generally falls into two categories:

• Full Fine-Tuning: In this scenario, every parameter of your model can be modified. You run the loop mentioned earlier, and all billions of parameters are slightly adjusted for your task. The main drawback of this method is the memory required — you need enough memory to hold and update the entire model, which, for a large language model, requires substantial hardware. There is also an increased risk of catastrophic forgetting, where the model becomes proficient in the specific task but loses its general capabilities.

• Parameter-Efficient Fine-Tuning (PEFT): Instead of updating every parameter in the network, PEFT techniques freeze the base model — every original parameter remains unchanged — and introduce a small set of new trainable parameters. Only these are modified. There are various techniques for this, such as LoRA, QLoRA, and prompt tuning, but the details go beyond the scope of this article. PEFT requires less memory and training time, with a lower risk of forgetting previously acquired knowledge. For most fine-tunings of large language models, this is the default choice.

Is Fine-Tuning Always the Solution?

Fine-tuning is ideal for teaching models a new skill, style, behavior, or task, but it is not the only tool — and often not the first one you should consider. Sometimes, a better prompt can solve your problem without requiring training. Similarly, when it makes more sense to look up information online or in a database at the time of the query, retrieval-augmented generation (RAG) is a better option, especially when facts are numerous or change frequently. These approaches are not in competition; in practice, most systems use them together. It’s good to keep this in mind before committing to a full fine-tuning cycle.

Additional Resources

If you want to practice fine-tuning specifically with LoRA, here are some recommended resources:

• Hugging Face PEFT: The standard open-source library for LoRA, QLoRA, prompt tuning, and more. Start with the documentation and the repository.

• Hugging Face TRL: Pairs with PEFT and provides you with a ready-to-use SFTTrainer for the supervised fine-tuning loop.

• Unsloth: The most user-friendly path for LoRA/QLoRA, with free notebooks on Colab and Kaggle, training ~2× faster, and much lower VRAM.

• Axolotl: Once you are comfortable, a popular configuration-based (YAML) tool to run fine-tuning pipelines without writing much code.

• The original LoRA paper: "LoRA: Low-Rank Adaptation of Large Language Models."

• The QLoRA paper: "QLoRA: Efficient Fine-Tuning of LLMs."

For a good first project, take a small instructive model (like an Llama 8B, Qwen, or Gemma), open an Unsloth QLoRA notebook, fine-tune it on a few hundred clean examples from your task, and observe the decrease in training loss. Once you've done it once, every term in this article will seem much more concrete.