A report that says an application has a high cache-hit rate can conceal several different measurements. It might count requests that reused any input, the fraction of input tokens read from a cache, or the share of eligible reference material reused. Those numbers answer different questions. None, by itself, tells you how much a completed task cost.
Accurate reporting begins by defining the unit, understanding the provider's counters, and keeping a consistent comparison. This guide uses a fully hypothetical workload to show the arithmetic. The examples are teaching numbers, not CacheIndex.com usage statistics or provider prices.
Decide what the report should help you change
Choose a practical decision before assembling a dashboard. You might want to know whether a prompt revision reduced repeated processing, whether long idle periods make explicit caching uneconomical, or whether one document workflow should use a smaller context bundle. A measurement becomes useful when it can distinguish among those choices.
Use at least three levels of accounting: the individual API request, the user-visible task, and the reporting period. A task may involve several requests, retries, and tool calls. Summarizing only successful final requests can hide work that the application still performed and may have paid for.
The AI token-usage index guide describes a useful reporting vocabulary. Keep its categories explicit in your own records, even when a provider uses different names for equivalent concepts.
Read the counter definitions before adding them
Do not assume that a field called input tokens means uncached input. One API may report a total with cache reads included as a subset; another may report separate input categories. Adding every visible counter can therefore count the same work twice. Preserve the original usage object and document your normalization rule.
For a concrete source, Google's Gemini context-caching documentation describes reported cached-token usage and explains that explicit-cache costs can depend on token quantity and storage duration, alongside other input and output charges. The accounting model must reflect the specific endpoint and pricing arrangement you use.
Create a short data dictionary. State whether each field is a total, a subset, or a separate category; its unit; and whether it describes creation, reuse, or generated output. Include the API version or documentation reference used when writing that definition.
Calculate request hits and token reuse separately
Consider a hypothetical set of twenty requests. Every request contains ten thousand input tokens. Fifteen requests each reuse eight thousand input tokens, while five reuse none. Across the set, total input is two hundred thousand tokens and cached reads are one hundred twenty thousand tokens.
Two denominators, two answers
The request-hit rate is fifteen divided by twenty, or seventy-five percent. The cached-input share is one hundred twenty thousand divided by two hundred thousand, or sixty percent. Both calculations are correct. The first describes how often some reuse occurred. The second describes how much input was reused.
The remaining eighty thousand tokens are input not read from cache. Depending on the API's billing definitions, some may be charged as cache creation and some as ordinary input. Do not combine them into a single cost category until that distinction has been resolved.
Label the metrics precisely. A chart titled “cache hits” invites ambiguity. “Requests with reported cache reads” and “cached reads as a share of total input” make the denominator visible.
Use a cost equation that matches the categories
For the same hypothetical workload, suppose the eighty thousand non-read tokens consist of forty thousand cache-write tokens and forty thousand ordinary input tokens. Suppose output totals twenty thousand tokens. We will use invented accounting units per million tokens: four for ordinary input, five for writes, one for reads, and eight for output.
The input calculation is 0.04 × 4, plus 0.04 × 5, plus 0.12 × 1. That equals 0.48 accounting units. Output contributes 0.02 × 8, or 0.16 units. The combined token charge is therefore 0.64 units before any additional costs.
Compare matched totals
A matched hypothetical uncached workload would charge all two hundred thousand input tokens at four units per million, plus the same output. Its total would be 0.96 units. The difference is 0.32 units, approximately one third of that matched baseline. These rates illustrate arithmetic only.
Include setup, retention, and failed work
The previous comparison remains incomplete if the workload also incurs storage charges, separate setup calls, or retries. Add those items to the relevant side of the comparison. Avoid counting a cache write both inside the request categories and again as an independent setup charge unless both charges really exist.
If retained context incurs a time-based cost, record the quantity and actual billable duration. A cache created for a short review session but retained through a long idle interval may have a different cost profile from the same cache removed when the session ends. Establish the relevant duration from usage or billing records.
Track failed attempts separately from successful task outcomes. This lets a team see whether lower per-request costs were offset by more requests needed to finish the same work. It also prevents a reliability regression from disappearing inside an attractive average.
Make the baseline comparable
A fair baseline preserves the work the user wanted. Compare the same question set, source documents, expected answer quality, and completion criteria. If one version answers shorter questions or generates much less output, identify that difference before attributing the change to caching.
Measure cold starts and repeated sessions explicitly. A benchmark made entirely of immediate repeat requests can exaggerate how much reuse an ordinary workflow achieves. Include realistic pauses, new documents, updates, and the mix of long and short tasks the application actually handles.
The stable-prefix design guide explains how prompt structure can create useful repetition. Test that structure independently from other improvements when possible, so the result tells you which change to keep.
Build a small, useful measurement record
For each request, retain a request identifier, task identifier, timestamp, model identifier, prompt revision, normalized usage categories, and outcome. Add timing measurements if latency matters. A content digest or bundle identifier can help group repeated context without copying the raw prompt into every analytics record.
For each task, aggregate total work and record whether it met the completion criteria. For each reporting period, show counts, token totals, cost estimates, and the proportion of records with complete usage data. Missing usage is an unknown quantity, not automatically zero.
Keep estimates distinguishable from reconciled billing. A local cost model may omit contract discounts or a specialized billing category. It can still guide engineering decisions if the assumptions are visible and the gap against invoices is investigated.
Handle empty groups explicitly. If a period contains no eligible requests, display the reuse rate as unavailable instead of dividing by zero or showing a misleading zero percent. When combining groups of different sizes, sum their token counts before calculating the overall token share. Averaging the displayed percentages would give a tiny workload the same influence as a large one.
Interpret changes before optimizing a percentage
A higher cached-input share can be a poor outcome if the application began sending unnecessary context. A lower share can be a good outcome if a smaller prompt completes the same task with less total work. Pair reuse metrics with total input per task, total cost per task, and quality checks.
When a metric changes suddenly, first check reporting changes: missing fields, duplicate events, modified grouping, or a new endpoint. Next check workload composition and prompt revisions. Only then treat the difference as evidence of changed cache behavior.
Keep the analysis focused on actionable causes. “More cache misses” is a symptom. “Reference bundles now change on every request because they include a generated timestamp” identifies a fix worth testing.
Conclusion: measure reuse and outcomes together
Good cached-token accounting separates request frequency, token volume, and cost. Define each counter, preserve raw evidence, use a matched baseline, and include the full task lifecycle. The most useful result is a report that explains what changed and whether the application delivered the same useful outcome with less work.
Continue the reading path through the subjects covered in this article.
