Cloudflare AI Gateway caching
Cloudflare AI Gateway caching serves identical requests from Cloudflare’s edge cache instead of calling the provider again — faster responses and lower provider bills.
Hub: Tokenminning with Cloudflare AI Gateway. Concept: Prompt caching.
Enable caching
In the Cloudflare dashboard → AI Gateway → your gateway → Caching:
- Enable caching for the gateway route
- Set TTL (time-to-live) for cached responses
- Send identical requests — second call hits cache
See Caching docs .
What gets cached
| Request type | Cache benefit |
|---|---|
| Identical prompt + model + params | High — full response cached |
| Same system prompt, different user message | Low — full body differs |
| Streaming responses | Config-dependent |
Gateway caching deduplicates identical requests. For multi-turn agents with changing user messages, stabilize prefixes and rely on provider-side prefix caching too.
Cache vs provider prefix cache
| Layer | Matches | Savings |
|---|---|---|
| AI Gateway edge cache | Identical full request | Zero provider call |
| Provider prefix cache | Stable system prompt prefix | Cheaper input tokens |
| Semantic cache | Similar meaning | App middleware |
Spending guardrails
- Enable caching on classification and extraction endpoints (high repeat rate)
- Set reasonable TTL — stale cache is cheaper than stale answers
- Monitor cache hit rate in Analytics
- Do not cache user-specific responses with PII unless TTL is very short
Troubleshooting
Cache hit rate near zero — requests differ slightly (timestamps in prompt, random seeds). Normalize prompts before sending.
Stale responses — lower TTL or disable caching for real-time features.
Related
Last updated on