Running Local LLMs with Ollama and OpenClaw

I remember the first time I tried to run a local LLM. I spent hours fighting with Python environments, CUDA versions, and weird model weight formats. It was a headache. Ollama changed that for me. It is the easiest way to get high-quality models running on your own hardware without paying for API tokens. Integrating it with OpenClaw makes things even better because it handles the discovery and configuration for you.

If you want to keep your data private or just want to experiment without a credit card, this is the way to go. I recommend using tool-capable models like Qwen 2.5 or Llama 3.3 to get the most out of OpenClaw’s agent features.

What You’ll Need

• Ollama installed and running (ollama.ai).
• A tool-capable model downloaded (like qwen2.5-coder:32b).

Quick Start

You can get this running in about five minutes. Here is the fastest path to a working local setup.

1. Pull your models

Open your terminal and grab a model that supports tools. I like these two for general tasks:

ollama pull qwen2.5-coder:32b
ollama pull llama3.3

2. Enable the provider

OpenClaw looks for an environment variable to trigger auto-discovery. Since Ollama is local, any value works for the key.

# Set this in your terminal or .env file
export OLLAMA_API_KEY="ollama-local"

3. Configure your agent

Update your openclaw.json5 (or your config file) to use the new model. Use the ollama/ prefix so OpenClaw knows which provider to call.

{
  agents: {
    defaults: {
      model: {
        primary: "ollama/qwen2.5-coder:32b",
        fallbacks: ["ollama/llama3.3"]
      },
    },
  },
}

4. Verify the setup

Check if OpenClaw sees your local models:

openclaw models list

Troubleshooting

Ollama not detected

If OpenClaw can’t find your models, check two things. First, make sure Ollama is actually running by visiting http://localhost:11434 in your browser. Second, ensure you haven’t defined an explicit models.providers.ollama block in your config, as that turns off the auto-discovery feature.

Corrupted responses

If you see weird text or tool names like memory_get leaking into the chat, it is likely a streaming issue. Some local models struggle with streaming tool calls. I have disabled streaming by default for Ollama to prevent this. If you manually turned it on and see garbled text, set it back to false:

{
  agents: {
    defaults: {
      models: {
        "ollama/qwen2.5-coder:32b": {
          streaming: false,
        },
      },
    },
  },
}

Connection refused

This usually means Ollama is stopped or running on a different port. Restart it with ollama serve. If you are running Ollama on a different machine, you will need to use an explicit configuration to point to the correct IP address.

No models in the list

OpenClaw only shows models that report tool support during auto-discovery. If your model is missing, it might not support tools natively. You can still use it by defining it manually in the models.providers.ollama section of your config file.

Still having trouble getting your local models to talk to your agents? Talk to our AI Setup Assistant to get back on track.

What’s Next

• Model Selection — Learn how to choose the right model for your task.
• Model Providers — See how to mix local models with cloud APIs.
• Configuration — View the full list of setup options.
• Agent Roles — Set up specialized agents for your local models.