RAG or Fine-Tuning: Choosing for Domain Adaptation

Two main approaches dominate the adaptation of models to business contexts: RAG and fine-tuning. They do not address the same issues. On one hand, retrieval updates knowledge and ensures traceability; on the other, training targets behavior. Here’s what each option allows, what it does not do, and how to deploy it in practice.

Specific business needs impose traceability and timestamped procedures

In an operational environment, every response must be traceable to a source document that can be accessed at any time. An on-call policy allows for 15 minutes for the primary responder to acknowledge receipt before escalating to the secondary responder, followed by 10 minutes before alerting the team leader. Incidents affecting payment or checkout trigger an immediate alert to the team leader, regardless of acknowledgments. Runbooks outline specific actions, such as checking the Postgres replication lag with a dedicated SQL query, promoting the standby if the lag is under 30 seconds, immediately updating the connection string, and refraining from switching if the lag exceeds 5 minutes while escalating to the database team. Post-mortems document causes and remedies, such as a case from March 2026 where a reduction of the connection pool from 100 to 20 led to exhaustion, corrected by reverting to the original size, adding an alert, and requiring a second reviewer on the platform side before merging.

RAG addresses evolving knowledge, not behavioral shortcomings

Retrieval-augmented generation does not modify the model: it injects, at inference, the most relevant passages from a document base along with the user's query. This approach targets large and changing corpora and allows reliance on explicit references. However, it does not correct tone, formatting discipline, or the use of industry-specific vocabulary. In an engineering context where runbooks and post-mortems are continuously enriched, RAG is suitable because knowledge evolves weekly and responses must remain traceable. In terms of implementation, the proposed example requires Python 3.10 or higher, the installation of scikit-learn and anthropic, as well as an API key from Anthropic.

Training on examples shapes the model's responses

Training on input/output examples adapts the model so that the desired behavior becomes the default, making the injection of patterns at inference unnecessary. The predominant LoRA and QLoRA techniques train an adapter that often comprises less than 1% of the model's parameters, reducing the effort to a few hundred dollars and a few hours, instead of a complete retraining. Fine-tuning adjusts style, structure, and pattern recognition, but does not reliably provide factual knowledge; the recall of specific facts learned during training remains uncertain, especially for granular elements. Therefore, fine-tuning should be viewed as a tool for behavior rather than knowledge.

Implementing simple retrieval: sentence segmentation and TF-IDF

A minimal RAG pipeline can segment documents at sentence boundaries to avoid breaking procedural steps, grouping two sentences per fragment by default. Each fragment is represented by a structure containing the document ID, title, and text. The index builds TF-IDF vectors on the fragments and calculates cosine similarity with the query. The search returns, ranked by decreasing relevance, the top three fragments by default.