AI Cache Index

The idea, in context

AI systems can reuse different layers of work: prepared prompt context, retrieved documents, embeddings, or previously produced responses. An AI cache index is a way to describe those layers without treating them as interchangeable. Give each entry a clear purpose and an owner. A prompt prefix may remain useful across many requests while a finished answer becomes unsuitable after a document update. The essential question is which assumptions must stay true for a reuse decision to be correct.

Choose the unit of reuse

List the material being retained and the computation it avoids. Distinguish provider-managed prompt caching from your own response store. Keep separate measurements for each layer so a cached prompt does not accidentally get reported as an answer returned without generation.

Version every meaningful dependency

Consider model identity, system instructions, retrieval corpus, tools, output schema, and access scope. Store the relevant versions alongside the entry. When one changes, a namespace switch can make previous work ineligible without requiring every physical object to disappear immediately.

Evaluate before widening reuse

Begin with exact matches and a small set of representative tasks. Compare cached outputs with freshly computed results. If you explore approximate matching, include near-misses that look similar but have different answers, such as conflicting dates, customer permissions, or product editions.