Helicone caching
Helicone prompt caching stores repeat prompt prefixes at the gateway layer — reducing input token cost when system prompts and tool schemas stay stable across turns.
Hub: Tokenminning with Helicone. Concept: Prompt caching.
Enable prompt caching
Configure prompt caching in the Helicone dashboard for your gateway or proxy route. Identical prefixes across requests hit the cache instead of re-billing full input tokens.
Stabilize the prefix
- Static first — system prompt, tool definitions
- Dynamic last — user messages, retrieved chunks
- Do not edit system block every agent turn
Same discipline as OpenRouter caching — gateway cache helps only when prefixes are stable.
Cache vs semantic
| Layer | Matches | Where |
|---|---|---|
| Helicone prompt cache | Exact prefix reuse | Gateway proxy |
| Provider prefix cache | Provider-side | cached_tokens in response |
| Semantic cache | Similar meaning | Your app — Semantic caching |
Measure cache effectiveness
Compare input token counts in the Requests tab before and after stabilizing system prompts. Cache hits show lower prompt token counts on repeat calls.
Spending guardrails
- Enable caching on routes with stable system prompts (agents, RAG)
- Skip caching on one-shot classification endpoints (low benefit)
- Pin tool schemas — MCP bloat kills cache hit rate
- Review cache hit rate monthly in dashboard analytics
Related
Last updated on