Zed local models

Zed supports local inference through Ollama and custom OpenAI-compatible endpoints. GSC queries like zed llama.cpp, zed vllm, zed litellm, and zed rag usually mean wiring a local stack instead of cloud APIs.

Hub: Tokenminning in Zed. Stack guides: Ollama, llama.cpp, vLLM.

Ollama (fastest path)

ollama pull llama3.2

In Zed assistant settings, point the provider to your Ollama baseURL (default http://localhost:11434). Zed sends OpenAI-compatible requests — Ollama runs llama.cpp under the hood.

Use for: exploration, air-gapped work, drafts before cloud commit.

llama.cpp server

Run llama-server and set Zed’s provider base URL to http://localhost:8080/v1. Same pattern as Ollama but you control GGUF paths and --fit-target memory sizing.

vLLM

For a shared local vLLM instance on a dev GPU:

• baseURL: http://localhost:8000/v1
• Pick a model name the server exposes
• Cap concurrent Zed users if the GPU is shared

zed vllm queries are usually about this OpenAI-compatible URL swap — not a Zed-specific vLLM plugin.

LiteLLM gateway

zed litellm setups route Zed through a LiteLLM  proxy for one URL that fans out to many providers or local backends. Useful for team routing and spend caps — adds latency; avoid double gateways (LiteLLM + OpenRouter) unless required.

RAG and retrieval

zed rag is not a separate Zed product feature — it means assistant retrieval over your codebase or attached context. Prefer:

• Narrow @file / @symbol references — Context guide
• Trim indexed surface before adding external RAG pipelines
• For app-level RAG, see Output and RAG

Hybrid routing

Log which backend served each task for cost attribution.

Related

• Zed hub
• Token usage
• Local inference