Tokens Cache Index

The idea, in context

A token count is meaningful only in context. The same visible material can be represented differently by different tokenizers, and a cached input count describes a particular processing path. A tokens cache index should connect the count to a versioned source, the relevant model or tokenizer, and the type of usage observed. This separates a planning estimate from provider-reported usage and makes later comparisons easier to explain. A token identifier is also different from a digital asset or payment token.

Label estimates and observations

Use different fields for a local token estimate and a usage value returned by a service. Record how the estimate was produced. Avoid silently mixing them in totals; a change in tokenizer or request assembly can otherwise look like a cache-performance change.

Preserve the request boundary

Track stable instructions, retrieved context, conversation history, and new user input as distinct components in your own analysis. The provider may report a different aggregation. Keep its raw usage response alongside your normalized record so the accounting can be audited.

Interpret reuse before cost

A cache hit is an event, while cached tokens describe a quantity. Neither alone is a bill. To estimate cost, apply the actual rate categories and units for the provider and model you use, then include any write, storage, or other applicable charges separately.