Notes

Claude Code With Ollama Setup

January 31, 2026

I tried claude-code-router with Ollama and it didn’t really work with Ollama due to mismatching input/output formats. Even Claude Code with Anthropic doesn’t work well out-of-the box with local (not cloud) models. Instead, at least in my experience, you have to do some more setup to get a useful Claude Code CLI working with Ollama.

The below is a from-scratch setup that makes Claude Code run with a locally running gpt-oss:20b model on Ollama. It’s not as strong as cloud-based SOTA models but at least this runs locally with 32GB of memory and an Nvidia RTX 4090:

  1. Install ollama and claude code

    curl -fsSL https://claude.ai/install.sh | bash
curl -fsSL https://ollama.com/install.sh | sh

  2. Declare a new ollama model with expanded context:

    # create this file with the name "Modelfile"
FROM gpt-oss:20b
PARAMETER num_ctx 65536

  3. Create model

    ollama create gpt-oss-64k -f Modelfile

  4. Set env vars

    # Recommended: add this to ~/.bashrc
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434

  5. Run claude code

    claude --model gpt-oss-64k

  6. Test prompts:

    > list files in current directory
> create a python script that calculates the first 10 fibonacci numbers called fib.py
> Run fib.py and show its output