BYO LLM key

Self-host runs against the bundled Ollama by default. If you'd rather point at a managed provider with your own API key — for higher quality without a GPU, or to use a model the local hardware can't run — flip a couple of env vars.

Why you'd do this

• Quality. GPT-4o / Claude Sonnet still beat any 8B-quantized local model on long delegation chains and analytical reasoning.
• Speed without GPU. Managed providers run on much bigger hardware than your laptop; ~3–10x faster generation.
• Cost. Free on small workloads (Anthropic's $5 monthly credit, OpenAI's free-tier embeddings) but scales with usage.

You stay self-host: data still lives in your Postgres, the agent runs on your hardware, only the LLM API call goes outbound. Air-gap operators should ignore this page entirely.

Switch at install

When the curl … | sh installer asks:

Use the bundled Ollama LLM (default) or a managed API key?
  [1] Bundled Ollama — recommended. ~5 GB first-boot pull.
  [2] OpenAI / Anthropic (BYO key)
Choice [1]: 2

Pick 2, choose provider, paste key. The installer writes the right env vars to .env.

Switch on an existing install

cd ccd
# Edit .env, append:
AI_LLM_PROVIDER=openai
AI_OPENAI_API_KEY=sk-…

# Or for Anthropic:
AI_LLM_PROVIDER=anthropic
AI_ANTHROPIC_API_KEY=sk-ant-…

# Restart so ai-services picks up the new env:
docker compose restart ai-services

Ollama keeps running but stops being called. You can stop the service entirely if you want to free the RAM:

docker compose stop ollama ollama-preload

(Don't docker compose down — that takes everything down. stop leaves the other services running.)

Embeddings

The LLM provider and the embeddings provider are independent. Common configurations:

Set independently:

AI_LLM_PROVIDER=anthropic
AI_EMBEDDINGS_PROVIDER=openai
AI_OPENAI_API_KEY=sk-…
AI_ANTHROPIC_API_KEY=sk-ant-…

A note on embedding dimensions

The knowledge-graph embedding column is vector(1536). OpenAI's text-embedding-3-small produces 1536-dim vectors — perfect fit. Ollama's default nomic-embed-text produces 768 — doesn't fit, so semantic search falls back to pg_trgm fuzzy matching. Results are still useful, just less precise.

If you care about semantic search precision and want to stay local:

• Use OpenAI for embeddings only (AI_EMBEDDINGS_PROVIDER=openai). Generation can still be Ollama. Cost is negligible — embeddings are pennies per million tokens.

A future migration parameterizes the column dim so any model fits — see the roadmap.

Cost expectations

These are ballpark — exact numbers depend on your usage pattern.

The cloud edition's managed brain proxy bills these to your subscription instead of your API keys.