Ollama setup
Ollama pull downloads a model; ollama run starts an interactive session; ollama serve exposes the OpenAI-compatible API on port 11434.
Hub: Ollama.
Pull and run
ollama pull llama3.2
ollama run llama3.2
# Loaded models and memory
ollama ps
# API server (default :11434)
ollama serveModel sizing by VRAM
| VRAM | Practical models | Fit |
|---|---|---|
| 8 GB | 3B–7B Q4 | Exploration |
| 16 GB | 7B–13B | Most IDE tasks |
| 24 GB+ | 14B–32B, 70B Q | Heavier work; still below frontier |
Benchmark quants (Q4, Q5, Q8) on your prompts — Q4 that fails review costs more than API tokens saved.
Unload models
ollama stop llama3.2Loading five models simultaneously causes VRAM thrashing and unpredictable latency.
Related
Last updated on