Prompt caching lets a provider reuse the computed state of a prompt prefix across requests, and bill that reused portion at a fraction of the normal input price. It is the single easiest discount most API users never claim.
When caching pays off
Caching shines when many requests share a long, stable prefix: a system prompt with detailed instructions, a fixed set of few-shot examples, or a document being questioned repeatedly. The savings scale with prefix length and request frequency.
When it does not
Cache entries expire after minutes of inactivity, so low-traffic applications rarely hit the cache. And if your prompt prefix changes on every call, for example because you inject a timestamp at the top, you break the cache without noticing. Put volatile data at the end of the prompt, keep the stable part first, and check your provider’s usage dashboard to confirm the cached tokens actually show up.
Leave a Reply