A Local Multilingual Ranking Pipeline with BGE-M3

A multilingual text classifier can be trained without multiplying models by language. By relying on embeddings produced locally by BGE-M3 via Ollama and Scikit-LLM, a logistic regression learns from reviews in English and Spanish. This all runs without a paid API, on a local machine or in a notebook.

Common embeddings make language secondary for the classifier

BGE-M3 has been pre-trained on data covering over 100 languages. In this setup, English and Spanish sentences expressing the same meaning produce nearly identical embeddings. Once these vectors are provided to a logistic regression, the rating classification no longer truly depends on the language. The BGE-M3 model is open-source and designed to handle multilingual information when generating embeddings.

Bilingual Amazon reviews, encoded on 5 levels

The Amazon Multi-language Reviews dataset provides reviews rated on a 5-star scale, encoded in labels from 0 to 4. A total of 2000 texts in English and Spanish is loaded for training purposes. It is recommended to maintain a balanced sample between languages and to randomly shuffle before splitting the sets. The English and Spanish subsets are each shuffled with a seed of 42 and restricted to 1000 examples, then combined into a single DataFrame and reshuffled with random_state 42. The features X correspond to the text column and the labels y to the label. The total displayed is 2000 samples, with a class distribution printed as 0:444, 3:410, 2:404, 4:380, and 1:362.

A local and free inference chain with Ollama and Scikit-LLM

The stated goal is a 100% free execution, particularly in a notebook, avoiding paid APIs like OpenAI. Ollama provides models that can be used locally, and Scikit-LLM is configured to communicate with an Ollama server running BGE-M3. The server is launched in the background, a 5-second wait is applied for initialization, and then the model bge-m3 is retrieved. Scikit-LLM is pointed to http://localhost:11434/v1/ and receives a dummy key, required by the client but ignored by Ollama. Starting Ollama in the background is presented as the simplest way in a cloud environment, though it is not essential in a local IDE. The installation requires scikit-llm and datasets==2.19.1, as well as zstd and the Ollama installation script.

Vectorizing with BGE-M3, training with logistic regression, and evaluating

The pipeline consists of two steps: a GPTVectorizer from Scikit-LLM configured with the bge-m3 model to produce the embeddings, followed by a LogisticRegression classifier. The data is split with 80% for training and 20% for testing, using a seed of 42. The vectorizer is set with a batch_size of 32, and the logistic regression with max_iter 1000 and random_state 42, before training the pipeline on the training set. Evaluation is conducted on the test set using the predictions from the pipeline and a classification report. The setup forms a multilingual classifier without multiplying models by language, relying on a free local configuration.