Tokenminning with Cloudflare AI Gateway

Cloudflare AI Gateway  adds observability and control in front of your AI providers — analytics, caching, rate limiting, and model fallback with one line of configuration.

Hub for the Cloudflare AI Gateway cluster. Best fit for apps already on Cloudflare Workers or needing edge-level caching and throttling.

Quick checklist

1. Open AI Gateway → Analytics for requests, tokens, and cost — analytics.
2. Enable caching for identical requests — caching.
3. Set rate limits per gateway route — rate limits.
4. Configure model fallback for resilience without frontier defaults.
5. Default to cheap models in your Worker or app code.

Typical impact: 20–50% from edge cache hits on repeat requests; rate limits prevent runaway agent loops.

How to reduce Cloudflare AI Gateway spending

1. Measure first

AI Gateway Analytics shows requests, tokens, cost, and errors per provider and model.

→ Analytics

2. Cap spend

Set rate limits on gateway routes. Throttle CI and eval workloads separately.

→ Rate limits

3. Route cheap

Default to mini/haiku models in your application. Use request retry and fallback  for resilience — cheap model first.

4. Cache at edge

Enable gateway caching to serve identical requests from Cloudflare’s cache.

→ Caching

5. Trim upstream bloat

Reduce context in Workers and agents — Context hygiene.

6. Guardrail in prod

Separate gateway instances for dev, staging, and production with different rate limits.

How Cloudflare AI Gateway bills

AI Gateway itself is included on all Cloudflare plans. You pay provider rates for inference. Caching reduces provider calls — saving both cost and latency.

Guides

Connect your stack

const response = await fetch(
  "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${OPENAI_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: "Hello" }],
    }),
  },
);

See Get started  for full setup.

Anti-patterns

Related

• Gateways
• Cloudflare AI Gateway docs