Scikit-LLM and Ollama: Free Text Classification

In a world where access to advanced language models can often be costly, a new method is emerging to democratize this technology. Thanks to Ollama, it is now possible to run language models locally, such as Llama 3, Mistral, and Gemma, without having to pay for API calls.

To get started, it is essential to install Ollama on your machine. Once installed, you can download popular open-source models. For example, to obtain Llama 3, simply run the following command in your terminal:

ollama run llama3

Alternatives like Mistral or Gemma can also be downloaded in a similar manner. Once the model is running, it is ready to receive local API calls. To keep the model active in the background, just type "/bye" in the terminal.

It is recommended to use an IDE to run this tutorial, as it facilitates interaction with Ollama. Before executing the Python code, make sure to install the necessary libraries:

pip install scikit-learn pandas scikit-llm

If you encounter a "Module not found" error while running the Python code, try installing the dependencies one by one.

The Scikit-LLM library plays a crucial role in redirecting requests to this local server. This is done by configuring Scikit-LLM to use Ollama's default port, thereby eliminating the need for costly API keys. Scikit-LLM requires a default key to pass internal checks, but this string will be ignored in practice with Ollama.

SKLLMConfig.set_gpt_url("http://localhost:11434/v1")
SKLLMConfig.set_openai_key("local-ollama-is-free")

With this configuration, it is possible to build a zero-shot text classifier. This process uses the ZeroShotGPTClassifier class from Scikit-LLM.

To illustrate this process, a dataset is prepared with user reviews and their corresponding categories. The model is then trained and tested on this data, demonstrating how a local language model can be used for classification tasks without additional costs.

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)
clf = ZeroShotGPTClassifier(model="custom_url::llama3")
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)

To display the model's inference results, the following code can be used:

for review, prediction in zip(X_test, predictions):
    print(f"Review text:  '{review}'")
    print(f"Predicted tag: {prediction}")
    print("-" * 50)

The results show that even with a small dataset, it is possible to achieve effective classification predictions. This approach offers an economical and accessible alternative for using advanced language models in machine learning applications.

To run the Python script, you can launch it from your terminal. For example, if you named it local_classification.py, use the following command:

python local_classification.py

By following all these steps, you can use advanced language models locally and for free, representing a significant advancement for developers and researchers looking to reduce their costs.