Scikit-Ollama: Revolutionizing Classification with Ollama Local

Introduction to Scikit-Ollama

Integrating large language models (LLMs) into traditional machine learning workflows has become not only feasible but is also transforming the way we work with these models. Traditionally, companies and developers relied on commercial cloud APIs, often limited by quotas and data privacy issues. However, the emergence of solutions like scikit-ollama is a game changer. This library, which heavily builds on scikit-llm, connects the simplicity of scikit-learn with the power of LLMs, particularly those that are free and operate locally on Ollama.

Scikit-ollama stands out for its ability to combine the user-friendly interface of scikit-learn with locally running Ollama models to perform zero-shot text classification. By integrating these models directly into local workflows, it eliminates reliance on cloud services, thereby reducing costs and data privacy risks. This allows users to leverage the power of advanced language models while maintaining full control over their data.

This article explores how to set up this integration to build a highly practical zero-shot classifier for sentiment prediction on movie reviews, using a local Llama 3 model running on your machine.

Preparing the Environment

Before getting started, it is crucial to ensure that your development environment is ready to use scikit-ollama. This library requires Python 3.9 or a later version. To check the installed version of Python, run the following command in your terminal:

python --version

If your Python version is earlier than 3.9, you will need to update before proceeding. Once this check is complete, install scikit-ollama with the following command:

pip install scikit-ollama

With the installation complete, we can move on to the next step, which is loading the necessary datasets for our project.

Loading the Data

Scikit-LLM offers a datasets module that contains various datasets for classification. For this example, we will use a text dataset for sentiment classification of movie reviews. Here’s how to load this data and display an example review with its sentiment label:

from skllm.datasets import get_classification_dataset

# Loading a demo dataset for sentiment analysis containing movie reviews
# The expected labels are: "positive", "negative", "neutral"
X, y = get_classification_dataset()
print(f"Example text: {X[0]} \nLabel: {y[0]}")

This code extracts a dataset that will be used to train our model to predict the sentiments expressed in movie reviews. The example output shows a movie review with an associated sentiment label, illustrating how the model can be effectively used to analyze sentiments.

Installing and Configuring Ollama

To use scikit-ollama, it is necessary to have Ollama installed locally on your machine. Follow the provided instructions to install Ollama and make sure to download the model you wish to use. To download a model, run the following command in your terminal:

ollama pull <MODEL_NAME>

In our case, we will use the model llama3:latest. Ensure that it is correctly installed before proceeding. This step is crucial as it guarantees that the model is available locally for use in classification tasks without relying on an internet connection or external service.

Initializing the Zero-Shot Classifier

Once the Ollama model is installed, we can proceed to initialize the classifier. The following code imports the ZeroShotOllamaClassifier class from scikit-ollama and instantiates a sentiment classifier backed by the llama3:latest model:

from skollama.models.ollama.classification.zero_shot import ZeroShotOllamaClassifier

# Initializing the classifier with our local Ollama model: llama3:latest
clf = ZeroShotOllamaClassifier(model="llama3:latest")

It is important to note that llama3:latest is a general language model, originally designed to do much more than classify text. However, by using it with scikit-ollama, we can transform our classification task into a syntactically constrained text generation prompt, allowing the model to function like a traditional machine learning model.

Applying the Model: Fit and Predict

Applying the model follows the traditional machine learning ritual of two steps: fit and predict. In the context of zero-shot classification, calling fit() does not update the model's weights. Instead, it records the candidate classification labels, guiding the model for contextual learning:

# "Fitting" the model simply involves providing the list of candidate labels
clf.fit(None, ["positive", "negative", "neutral"])

When you call the predict() method and pass a set of text reviews, the local instance of Ollama processes each entry as a prompt and generates predictions that correspond to one of the zero-shot classification labels.

The following code generates predictions on the dataset and prints the first three results:

# Generating and displaying predictions on our dataset
predictions = clf.predict(X)
for text, prediction in zip(X[:3], predictions[:3]):
    print(f"Text: '{text}'")
    print(f"Predicted sentiment: {prediction}\n")

Each prediction is the result of an in-depth analysis performed by the local model, which uses its natural language processing capabilities to determine the sentiment expressed in each movie review. This demonstrates the model's ability to perform complex tasks autonomously, without requiring weight updates or additional training.

Conclusion

By using scikit-ollama, we have demonstrated how to replace cloud-based LLM APIs with local Ollama models to perform inference tasks. This approach eliminates subscription fees and ensures that sensitive textual data remains on your machine. The scikit-ollama library encapsulates this local integration elegantly, making it available as a simple scikit-learn pipeline.