Skip to Content
Self-hostingOllamaOpenAI API

Ollama OpenAI API

Ollama exposes /v1/chat/completions compatible with the OpenAI SDK. Set baseURL to http://localhost:11434/v1.

Hub: Ollama.

Client example

import OpenAI from "openai";
 
const client = new OpenAI({
  baseURL: "http://localhost:11434/v1",
  apiKey: "ollama", // required by SDK; Ollama ignores it
});
 
const response = await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Summarize this function in one line." }],
  max_tokens: 128,
});

Always set max_tokens — local models do not cap output by default.

Metering

Tag spans inference.backend=ollama. Compare token counts to Narev  cloud rates for the same tier — local is not free, it trades API cost for GPU time.

Last updated on
SetupIDE integration