Measuring Cached Token Usage Without Double Counting
A cache-hit percentage is only useful when its denominator is clear. Learn how to reconcile token reuse, request counts, and task cost.
Read the articleSubject collection / 2 articles
Token usage
Explore the token usage questions that connect these guides.
Usage records need definitions. Input, cached input, output, cache writes, and storage can represent different accounting categories in different services. This collection explains how to retain the original measurement and avoid confusing an observed quantity with an estimate.
Follow a task through its individual requests, including retries and intermediate work. Identify which quantities are subsets and which are separate. Keep the model, method, and relevant definitions attached to the record so a future comparison does not mix values that only happen to share a name.
Articles about token usage
Prompt Caching for LLMs: Design a Stable Prefix That Earns Its Keep
A stable prompt prefix can make repeated context reusable. The benefit depends on useful repetition, careful versioning, and measured behavior.
Read the articleTake the next step.
Start with the AI Token Usage Cache Index guide to place the terminology in context. Then explore Tokens Cache Index for a connected design question.
Read across the index.
A topic can cross several systems. Browse the complete Lab for the surrounding articles, or use the 22-topic directory to find a different entry point.