LiteLLM proxy setup

Run LiteLLM as an OpenAI-compatible proxy on port 4000. Clients point baseURL at the proxy instead of calling providers directly.

Hub: Tokenminning with LiteLLM.

Minimal config.yaml

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
 
general_settings:
  master_key: sk-<your-master-key>
  database_url: postgresql://user:pass@localhost:5432/litellm

Start the proxy:

litellm --config config.yaml --port 4000

Postgres is required for budgets & spend.

Client wiring

from openai import OpenAI
 
client = OpenAI(
    api_key="sk-<virtual-key>",
    base_url="http://localhost:4000",
)
 
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

Zed integration

Point Zed’s OpenAI-compatible provider at http://localhost:4000. See Zed local models for the full pattern.

Avoid double gateways

Pick one gateway per client.

Spending guardrails

• Issue virtual keys — never distribute the master key
• Set max_budget at key creation
• Run behind auth on shared networks — an open proxy is an open relay
• Log x-litellm-response-cost in your app middleware

Related

• LiteLLM hub
• Budgets & spend
• LiteLLM proxy docs