Gemma 4 and Claude Code: Local Programming Revolution

A New Era for Local Agentic Programming

Imagine a scenario where a multi-agent workflow is capable of reading files, writing patches, executing tests, and repeating these actions across four different services, all while making 400 API calls in a single afternoon. An alert message informs you that you have exceeded your plan's limit. Each token represents a cost, each prompt sends your code to a third-party server, and rate limits interrupt prolonged sessions. The only solution seems to be to pay more.

Gemma 4: A Significant Technological Advancement

The Gemma 4 model, with its 26 billion parameters, activates only 3.8 billion of these parameters at each pass. It achieves a score of 77.1% on LiveCodeBench v6 and 86.4% on τ2-bench, benchmarks that specifically evaluate a model's ability to use tools, execute steps, and manage errors in a complex workflow. Compared to its predecessor, Gemma 3 27B, which only reached 6.6% on these same tests, this update represents a major advancement. This means that Gemma 4 can execute an agentic loop with Claude Code without constantly distorting its function call parameters.

Building an Effective Tech Stack

This article details the construction of a complete tech stack: Ollama to serve Gemma 4 locally, the Modelfile to avoid context window failures in agentic sessions, the settings.json file that connects Claude Code to the local endpoint, a verification script to ensure everything works correctly before applying it to real code, and an honest analysis of potential issues and their solutions. This article is aimed at engineers who already have an understanding of large language models (LLMs) and the costs associated with agentic loops.

Gemma 4: A More Permissive License

Released on April 2, 2026 under Apache 2.0, Gemma 4 is Google DeepMind's most performant open-weight model to date. It is available in four variants: E2B (2B effective), E4B (4B effective), 26B MoE, and 31B Dense. The 26B MoE model uses 128 small experts and activates only 8 per token plus one shared expert, offering quality close to 31B at a significantly reduced computational cost.

Previous versions of Gemma were under a custom Google license with ambiguous commercial use restrictions, often seen as a barrier by corporate legal teams. With the Apache 2.0 license, Gemma 4 allows for easier integration into internal tools, product deployment, or execution in production pipelines without requiring extensive legal review.

Gemma 4's Performance in Numbers

The performance of Gemma 4 is measured across several key benchmarks:

• Gemma 4 31B Dense
• τ2-bench for agentic tool usage
• LiveCodeBench v6
• AIME 2026 for mathematics

Hardware Requirements for Gemma 4

Before downloading an 18 GB model, it is crucial to understand the hardware requirements. The Gemma 4 family has been designed to fit a wide range of devices, from edge devices to workstations, and the four variants reflect this diversity.

Installing Ollama, Gemma 4, and Claude Code

Step 1: Install Ollama

For macOS and Linux, installation is done in a single line:

curl -fsSL https://ollama.com/install.sh | sh

It is essential to check that the version is 0.14.0+ for support of the Anthropic Messages API. The endpoint compatible with Anthropic was added in January 2026, and the expected version of Ollama is 0.22.x or higher (as of May 2026).

For Windows, you need to download the native installer from https://ollama.com. Using WSL2 is recommended for GPU passthrough on Windows.

After installation, Ollama starts as a background service on port 11434. To check that it is running correctly:

curl http://localhost:11434

The expected response is that Ollama is running.

Step 2: Download Gemma 4

For the 26B MoE, recommended for this setup, the download is approximately 18 GB:

ollama pull gemma4:26b

During the download, it is advisable to monitor the progress to ensure everything is proceeding correctly.

It is also possible to download the 31B for comparison on capable hardware:

ollama pull gemma4:31b

Once the download is complete, it should display gemma4:26b with the size and modification date.

Step 3: Install Claude Code

Prerequisites: Node.js 18 or later. Check the version:

node --version

Install the Claude Code CLI globally:

npm install -g @anthropic-ai/claude-code

Verify the installation:

claude --version

With Ollama running and Gemma 4 downloaded, the next step is to export the environment variables and launch Claude Code.

The default context window of Ollama for Gemma 4 is 4K tokens, while the actual context window of Gemma 4 is 128K–256K. This 4K default is not a suggestion; it is what Ollama will use unless you override it. In an agentic session, Claude Code that reads source files, retains conversation history, and maintains tool call results across multiple turns, 4K tokens deplete quickly.

Without context replacement, Claude Code loses track of the content of the files being edited, forgets previous instructions, and produces fragmented changes. For example, when an agent tries to refactor a 200-line service class, it forgets that the second half exists. The agent does not raise an error but works with an incomplete view of the file, producing partially correct output that breaks downstream.

The solution is a Modelfile that integrates the correct context size and other inference parameters into a named model variant. Create this file:

~/.ollama/Modelfiles/gemma4-claude

• Gemma 4 26B MoE variant adjusted for Claude Code agentic sessions.
• Integrates the context window, temperature, and system prompt into the model.
• Ensures that each Claude Code session starts with the correct configuration.

Create the Modelfiles directory if it does not exist:

mkdir -p ~/.ollama/Modelfiles

Save the content of the Modelfile above to this path, then build:

ollama create gemma4-claude -f ~/.ollama/Modelfiles/gemma4-claude

Check that the variant has been created; it should display gemma4-claude alongside gemma4:26b.

Quick test: check that it loads and responds:

ollama run gemma4-claude "What is the time complexity of binary search and why?"

Expect a clear and concise technical response within seconds.

Connecting Claude Code to the Local Model

With the model variant built, the configuration connects Claude Code to Ollama. Two environment variables are essential, but three additional variables prevent the most common failure modes.

The Anthropic-compatible endpoint of Ollama is at http://localhost:11434, not at http://localhost:11434/v1. The /v1 path is the OpenAI-compatible layer of Ollama. Claude Code uses the Anthropic Messages API protocol, which maps to the root endpoint. Using the /v1 path will produce authentication errors or unexpected behavior.

Global Settings — ~/.claude/settings.json

This configuration applies to every Claude Code session across all projects. It is the right choice unless you frequently switch between local and cloud models per project.

{
  "ANTHROPIC_BASE_URL": "http://localhost:11434",
  "ANTHROPIC_AUTH_TOKEN": "ollama",
  "ANTHROPIC_API_KEY": "",
  "ANTHROPIC_MODEL": "gemma4-claude",
  "ANTHROPIC_DEFAULT_SONNET_MODEL": "gemma4-claude",
  "ANTHROPIC_DEFAULT_HAIKU_MODEL": "gemma4-claude",
  "ANTHROPIC_DEFAULT_OPUS_MODEL": "gemma4-claude",
  "CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS": "1"
}

Each variable has its importance:

• ANTHROPIC_BASE_URL redirects all API calls from Claude Code from Anthropic's servers to your local Ollama instance.

• ANTHROPIC_AUTH_TOKEN must be set to a non-empty string; Ollama ignores the value, but Claude Code requires the header to be present.

• ANTHROPIC_API_KEY: "" explicitly empties the key so that Claude Code cannot fall back to a real Anthropic API key if one is set in your shell environment. Without this, a misconfigured ANTHROPIC_BASE_URL could silently fail on the paid API.

• ANTHROPIC_MODEL is the main model name that Claude Code sends in requests. Set this to your custom Modelfile variant, gemma4-claude and not gemma4:26b. The raw model tag does not carry the context window replacement.

• ANTHROPIC_DEFAULT_SONNET_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL, and ANTHROPIC_DEFAULT_OPUS_MODEL: Claude Code internally routes different types of tasks to different model levels. By setting all three to the same local model, you ensure that each request lands on your Ollama instance.