Hardware sizing

The self-host bundle's footprint is dominated by the local LLM. Everything else (Node services, Postgres, Redis) fits in under 2 GB of RAM.

Minimum

What you need to boot the default config (Llama 3.1 8B Q4_K_M + nomic-embed-text):

Resource	Floor	Notes
System RAM	16 GB	Llama 3.1 8B warm ≈ 6 GB; OS + everything else ≈ 8 GB; leaves a slim safety margin.
Free disk	8 GB	~5 GB for models, ~2 GB for Docker images, ~1 GB scratch.
CPU	x86_64 or arm64, 4+ cores	Ollama is multi-threaded; 4 cores keeps generation usable on CPU.
Network	50 Mbps for first pull	Models are pulled from ollama.com on first boot.

Comfortable

If you have the budget, here's what makes life easier:

Resource	Recommended	Why
System RAM	32 GB	Lets you swap in a 13B or 70B model without re-engineering.
Free disk	50 GB	Headroom for multiple cached models + Docker image churn.
CPU	8+ cores or any modern GPU	Generation rate scales linearly with core count up to ~16.
GPU	NVIDIA 8 GB+ or Apple Silicon	~10x faster generation. Optional.

Model size cheatsheet

Model	Disk	Warm RAM	Tokens/sec (CPU 8-core)
`llama3.1:8b-instruct-q4_K_M` (default)	4.7 GB	~6 GB	8–12
`qwen2.5:7b-instruct-q4_K_M`	4.4 GB	~5.5 GB	10–14
`phi3:medium-128k-instruct-q4_K_M`	8.5 GB	~10 GB	6–10
`llama3.1:70b-instruct-q4_K_M`	40 GB	~45 GB	1–3 (CPU); 25+ on a 24 GB GPU
`mixtral:8x7b-instruct-q4_K_M`	26 GB	~30 GB	4–8 (CPU); 20+ on GPU

Smaller models trade quality for speed. The default 8B handles the orchestrator's multi-step tool calling well; the 70B is noticeably better at long delegation chains and analytical reasoning if you have the hardware.

Swapping models

cd ccd
# Edit .env, change:
#   AI_OLLAMA_MODEL=llama3.1:70b-instruct-q4_K_M
docker compose up -d ollama-preload

The preload sidecar re-runs and pulls the new model into the cache volume. The original model isn't deleted — switch back any time without re-paying the pull cost.

GPU

NVIDIA

Install the NVIDIA Container Toolkit.

Uncomment the deploy.resources.reservations.devices block in docker-compose.yml:

ollama:
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

docker compose up -d ollama.

Ollama auto-detects CUDA and offloads layers to the GPU.

Apple Silicon

Docker Desktop on M-series Macs doesn't pass through the Metal API to containers. For GPU acceleration:

Install Ollama natively on the host: brew install ollama then ollama serve.
Skip the ollama service in docker-compose.yml (comment it out).
Point AI_OLLAMA_BASE_URL at the host: http://host.docker.internal:11434/v1.

This gets you Metal acceleration — roughly 2–3x faster generation on an M2 vs Docker-confined CPU.

When you're CPU-bound

If you only have CPU and generation feels slow:

Drop to a smaller model first (qwen2.5:7B is competitive with Llama 8B at ~80% the RAM).
Lower OLLAMA_KEEP_ALIVE to 30s if you don't generate often — the model gets unloaded, freeing RAM for other workloads, at the cost of a longer first-token after pause.
Avoid the 70B / Mixtral tier on CPU — they're meaningful only with a GPU.