Prompt, RAG, Fine-Tuning: Avoid Costly Pitfalls for SMEs

Understanding the Real Needs of SMEs in AI

In the world of small and medium-sized enterprises, the terms prompt, RAG, and fine-tuning often lead to confusion. This misunderstanding can be costly, both financially and in terms of wasted time. In reality, what an SME needs rarely requires fine-tuning, an operation that often turns out to be more complex and expensive than necessary.

During initial meetings with executives, it is not uncommon to hear: “We need an AI trained on our data.” This request, while legitimate, often rests on a misconception that specific training equates to seriousness and professionalism. However, in 9 out of 10 cases, the answer is to wait. What these executives describe is often a solution that costs 20 times more than what is actually necessary and takes 4 months to implement.

The Prompt: A Simple and Effective Solution

The prompt is how you communicate with the AI model. It consists of the instructions you give it, as well as the context you provide when asking a question. While this may seem basic, it often meets the overwhelming majority of SMEs' needs.

Take the example of an accounting firm wanting to automate the sorting of its client emails by type of request. Instead of launching an expensive project costing €30,000, a precise prompt was drafted. By using 5 examples of already classified emails, this solution was integrated into their system in a week, for the simple cost of the API subscription.

Situations where the prompt suffices include drafting quotes, summarizing reports, responding to standard emails, rephrasing texts, or extracting information from a document. If your need boils down to “explain well to the model what you want and give it one or two examples,” then the prompt is all you need.

RAG: AI Serving Your Documents

The term RAG, which stands for "retrieval-augmented generation," may seem intimidating, but the concept is simple. Imagine an assistant whom you do not ask to memorize your 4,000 documents. Instead, you pair it with a librarian. For each question asked, this librarian will search for the 3 most relevant passages in your files and provide them to the AI, which then responds based on this information.

This approach is necessary when the AI needs to rely on your material, for example, for a support chatbot connected to your knowledge base, searching through hundreds of contracts, or an assistant responding about your internal procedures. One of the great advantages of RAG is that when you add a document, it is immediately taken into account without requiring reprogramming. You maintain control over what the AI can cite, and you know where each response comes from.

Implementing RAG can take a few weeks, depending on the volume of documents to be processed. It is a solution that sits above the prompt and covers nearly all remaining cases.

Fine-Tuning: A Rarely Necessary Option

Fine-tuning involves retraining the model on your own examples so that it adopts a specific format or tone sustainably. This is what many executives envision when they talk about training on their data. However, this is where I often hit the brakes.

For fine-tuning to be justified, three conditions must be met simultaneously: a very narrow and repetitive task, a large volume of data, and thousands of clean, well-labeled examples. Most SMEs meet none of these conditions.

A commonly overlooked trap is that the day your process changes, everything needs to be retrained. Moreover, every few months, a new base model is released, often better than your specialized version from six months ago. Your investment thus ages quickly. The highest cost in this process is data preparation, a long and manual task that few people enjoy.

Avoiding Costly Mistakes

The scenario I see most often is that of a company jumping straight to fine-tuning because it seems serious, neglecting the prompt and RAG. The common outcome is an expenditure of €40,000 and 4 months to achieve a result that a setup costing €200 per month could have delivered in 2 weeks. With, on top of that, a frozen model that will need retraining at the first evolution.

The correct order is the opposite. Start with the prompt. If that is not enough, add RAG. Fine-tuning should be the last resort, once you have proof that nothing else suffices.

Making the Right Decisions

To choose the right approach, ask yourself these questions in order:

1. Does the model just need good instructions and one or two examples? Then it’s a prompt. Stop there.
2. Does it need to draw from your up-to-date documents and cite its sources? Then it’s RAG.
3. Does it need to reproduce a very specific format or tone thousands of times a day, with labeled examples at hand? Only then does fine-tuning come into play. And still: test RAG first.

If you get stuck on the first question, that’s already great news for your budget.

Investing Wisely

The uncomfortable truth that undermines flashy presentations is that the model is rarely the problem. Your useful budget should be allocated elsewhere: in organizing your data (a clear client folder is better than a boosted model), in well-written and tested prompts, and in connecting the AI to your existing tools, such as your CRM, your email, your files.

A startup SME needs a good prompt, sometimes a bit of RAG, and someone to connect all of this to its daily operations. The day you truly need fine-tuning, you will know. Your data will tell you, as will your volume. In the meantime, keep the €40,000.