Small AI Models: A Revolution in Agent Creation

Local Small AI Models Revolutionize the Creation of Intelligent Agents

Introduction

In the past, creating artificial intelligence (AI) agents was a domain reserved for technology giants, requiring costly and complex infrastructures. Developers had to rely on expensive cloud APIs and powerful servers to design these agents. However, this image has radically changed with the advent of small language models (SLMs), which now allow anyone, even novices, to build functional AI agents directly on a personal computer.

These SLMs are lightweight enough to operate without an Internet connection after initial setup, thus eliminating recurring costs associated with APIs. They offer reasoning, planning, and response capabilities while being compact enough to run on standard machines. This article will guide you through creating a local AI agent using tools like Ollama and LangChain/LangGraph, whether you are a Python beginner or an intermediate developer exploring AI.

What is an AI Agent?

An AI agent is a sophisticated program that uses a language model to think, make decisions, and act with the goal of achieving a specific objective. Unlike traditional chatbots, which merely respond to messages, an AI agent is capable of:

• Breaking down a complex task into simpler steps.
• Choosing the appropriate tool or action for each step.
• Using the results obtained to inform subsequent steps.
• Continuing the process until the task is completed.

Imagine the difference between a simple calculator, which waits for your instructions, and a personal assistant, which anticipates your needs, determines the necessary steps, and executes them autonomously.

A basic AI agent consists of three main components:

| Part | Function | |------|----------| | Brain (LLM/SLM) | Understands the input and decides on actions to take | | Memory | Retains the context of previous interactions | | Tools | External functions that the agent can use, such as searching or calculating |

What are Small Language Models?

Small language models (SLMs) are AI models trained on vast text datasets, similar to large models like GPT-4, but designed to be much more compact. While GPT-4 may contain hundreds of billions of parameters, SLMs such as Phi-3, Mistral 7B, or Llama 3.2 (3B) range from 1 billion to 13 billion parameters. This reduction in size makes them suitable for execution on personal computers equipped with modern CPUs or consumer GPUs.

Here are some examples of popular SLMs:

| Model | Developer | Size | Recommended Use | |-------|-----------|------|-----------------| | Phi-3 Mini | Microsoft | 3.8B | Fast reasoning, low memory consumption | | Mistral 7B | Mistral AI | 7B | General tasks, following instructions | | Llama 3.2 (3B) | Meta | 3B | Balanced performance |

For those unsure about which model to choose, Phi-3 Mini and Llama 3.2 (3B) are well-documented and accessible options, perfect for getting started on local machines.

Why Run AI Agents Locally?

You might wonder why not simply use cloud services like the OpenAI API or Google Gemini. Here are some reasons why local SLMs are an appealing alternative:

• Cost savings on API fees. Cloud services often charge per token or per request, which can quickly become expensive if your agent makes numerous requests. In contrast, once set up, local models operate without additional fees.

• Enhanced privacy. Using a cloud API means your data leaves your device, which can raise privacy concerns, especially for sensitive information like medical records or business data. Local models keep all data on your device.

• Offline operation. Even without an Internet connection, your AI agent continues to function, offering increased flexibility.

• Total control. You have the freedom to choose the model, set parameters, and customize your agent's behavior without being limited by usage policies or rate restrictions.

• In-depth learning. Setting up and running models locally forces you to understand their internal workings, enriching your development skills.

Tools You Will Use

To build your local AI agent, you will use the following tools:

• Ollama: This open-source tool allows you to download and run language models on your local machine with a single command, simplifying setup.

• LangChain / LangGraph: LangChain is a popular framework for creating applications based on language models. LangGraph, an extension of LangChain, facilitates the creation of agent workflows, defining how your agent thinks and acts step by step using a graph-based structure.

Setting Up Your Environment

Before you start coding your agent, you need to set up your working environment.

Step 1: Install Ollama

Go to ollama.com to download the installer for your operating system (Windows, Mac, or Linux). Once installed, open your terminal and download a model with the following command:

ollama pull phi3

This will download the Phi-3 Mini model to your machine. To verify it is working correctly, run:

ollama run phi3

You should see a prompt allowing you to interact directly with the model. Type /bye to exit.

Step 2: Install Python Libraries

Create a virtual environment and install the necessary packages:

python -m venv agent-env

For Linux/Mac:

source agent-env/bin/activate

On Windows:

agent-env\Scripts\activate

Then install the required libraries:

pip install langchain langchain-ollama langgraph

Make sure to use Python 3.9 or a newer version. Check your version with:

python --version

Building Your First Local AI Agent

Now let's create a simple agent capable of answering questions and using a basic tool, such as a calculator.

In your agent.py file, insert the following code:

from langchain_ollama import OllamaLLM
from langchain.agents import AgentExecutor, create_react_agent
from langchain.tools import tool
from langchain import hub

# Step 1: Load the local model via Ollama
llm = OllamaLLM(model="phi3")

# Step 2: Define a simple tool -- a calculator
@tool
def calculator(expression: str) -> str:
    """Evaluates a basic mathematical expression. Input must be a valid mathematical expression in Python."""
    try:
        result = eval(expression)
        return str(result)
    except Exception as e:
        return f"Error: {str(e)}"

# Step 3: Group the tools
tools = [calculator]

# Step 4: Load a ReAct prompt model
prompt = hub.pull("hwchase17/react")

# Step 5: Create the agent
agent = create_react_agent(llm=llm, tools=tools, prompt=prompt)

# Step 6: Wrap in an executor to manage the agent's loop
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Step 7: Run the agent
response = agent_executor.invoke({
    "input": "What is 245 multiplied by 18, then divided by 5?"
})

print("\n--- Agent's Response ---")
print(response["output"])

Here’s what happens in this script:

• The OllamaLLM class connects to the locally running Phi-3 model.
• The @tool decorator transforms a standard Python function into a tool that the agent can call.
• The create_react_agent function uses the ReAct model, a method where the agent reasons about the problem and then acts using a tool, repeatedly, until it arrives at an answer.
• AgentExecutor manages the reasoning, action, and observation loop of the results.

To run the script, use the following command:

python agent.py

You will see the agent's reasoning process displayed in the terminal before it produces the final answer.

Adding Memory and Tools to Your Agent

For an agent to be truly useful, it must remember past conversations. Here’s how to add conversation memory and a second tool, such as a knowledge base search.

In your agent_with_memory.py file:

from langchain_ollama import OllamaLLM
from langchain.agents import AgentExecutor, create_react_agent
from langchain.tools import tool
from langchain.memory import ConversationBufferMemory
from langchain import hub

llm = OllamaLLM(model="phi3")

# Tool 1: Calculator
@tool
def calculator(expression: str) -> str:
    """Evaluates a basic mathematical expression."""
    try:
        return str(eval(expression))
    except Exception as e:
        return f"Error: {str(e)}"

# Tool 2: Simulated knowledge base search
@tool
def knowledge_base(query: str) -> str:
    """Searches for information in a local knowledge base."""
    kb = {
        "python": "Python is a beginner-friendly programming language widely used in AI and data science.",
        "AI agent": "An AI agent is a program that uses a language model to reason and take actions.",
        "ollama": "Ollama is a tool for running language models locally on your computer.",
    }
    for key in kb:
        if key in query.lower():

Conclusion

By following these steps, you can create a local AI agent that not only answers questions but also remembers past conversations and uses tools to enrich its responses. This paves the way for more complex and personalized applications while ensuring privacy and control over your data.