Skip to content

API reference

Stop paying full price for the same tokens.

Divergence dataclass

fix property

What to change so the next request repeats this one's prefix.

Prompt

Collect segments in any order; render with an adapter. Output order is fixed: tools, system, context, examples, history, retrieved, user.

call_site is any name you choose for the place in your code that makes this call, such as "support.answer" or "rag.answer". There is no fixed list. Use one per prompt template, not one per user or per request. It groups usage per call site in Stats, record() and cachemaxxing report, and the report finds drift by comparing consecutive requests of the same call site. It also names Gemini explicit caches, and it is part of prompt_cache_key on OpenAI models before GPT-5.6.

tools(tools, *, sort=True)

Tool definitions in your provider's format; sorted by name unless sort=False.

context(docs, *, reorderable=False)

Static reference material. reorderable=True sorts the parts so their order can't drift.

messages(messages)

Your whole chat as an OpenAI-style list of {"role": ..., "content": ...} dicts, for any provider. Leading system/developer messages become the system prompt; the rest, latest user message included, is the conversation. Same as .system(...).history(...).

history(messages)

Earlier turns, append-only, never modified: your provider's message format, or {"role", "content"} text turns (converted for Gemini).

retrieved(chunks)

Per-request context (RAG chunks, time, user profile). Placed after history, verbatim.

prefix_hash()

16 hex chars that change exactly when the tools or the static text change.

anthropic(model, *, ttl='5m', diagnose=False)

kwargs for anthropic client.messages.create(max_tokens=..., **kwargs). diagnose=True opts in to Anthropic's cache diagnostics; pass the previous response's id on later turns and usage(response).miss_reason says what changed (system_changed, tools_changed, ...).

openai(model, *, ttl='30m', cache_key=None, diagnose=False, explicit=None)

kwargs for openai client.responses.create(**kwargs). diagnose= asks OpenAI (GPT-5.6+) to compare with that response; usage(response).miss_reason then says why a cache missed.

Azure OpenAI: model is your deployment's name. Explicit breakpoints are chosen from the name, so pass explicit=True for a GPT-5.6+ deployment with another name, and explicit=False for a provisioned one.

openai_chat(model, *, ttl='30m', cache_key=None, diagnose=False, explicit=None)

kwargs for openai client.chat.completions.create(**kwargs) and OpenAI-compatible servers. explicit works as in openai().

gemini(model, *, cached_content=None)

kwargs for google-genai client.models.generate_content(**kwargs). Implicit caching unless cached_content (from gemini_cache) is given; then the cached fields are left out.

gemini_prefix()

The cacheable part of a Gemini request: what an explicit cache holds.

litellm(model, *, ttl=None, explicit=False)

kwargs for litellm.completion(**kwargs) (or the LiteLLM Proxy via the OpenAI SDK), model included. Cache fields follow the route: Claude gets cache_control, openai/ gets openai_chat(), gemini/ stays implicit unless explicit=True (LiteLLM then creates an explicit cache for the static prefix).

Stats

Usage totals per call site, in memory. Safe to share between threads.

Give add each call's request too and Stats diagnoses misses as they happen. When a call read less than half of its input from cache, it compares the request with the previous one from the same call site, counts the cause in drift, shows the top cause in table(), and logs a warning with the exact change and the fix the first time each cause shows up for a call site. warn=False keeps the diagnosis and drops the warnings.

add(call_site, u, request=None)

Count one call. With request (the kwargs you sent), also diagnose it: returns the Divergence when the call missed and differs from the previous request in a way that explains it, else None.

Usage dataclass

attributes(call_site=None, *, prefix='cachemaxxing.')

Flat metrics for a trace: Langfuse or LangSmith metadata, OpenTelemetry span attributes (span.set_attributes(u.attributes("support.answer"))). Unknown values are left out.

canonical_json(obj)

Same data, same bytes: sorted keys, fixed separators, non-ASCII kept.

should_cache(prefix_tokens, expected_reuses, provider, model, *, interval_s=60.0)

Whether caching a prefix of prefix_tokens pays when it is reused expected_reuses times, interval_s seconds apart (start to start).

gemini_cache(client, p, model, *, expected_reuses, interval_s=60.0, prefix_tokens=None)

Name of a live explicit cache holding p's static prefix, created only if should_cache() says it pays. Use it as p.gemini(model, cached_content=name). None means stay on implicit caching, which is also what happens, with a warning on the cachemaxxing logger, when the caches API fails. The prefix is counted exactly with client.models.count_tokens (once per prefix) unless prefix_tokens is given.

gemini_cache_async(client, p, model, *, expected_reuses, interval_s=60.0, prefix_tokens=None) async

Async twin of gemini_cache, using client.aio.

record(call_site, request, response, *, redact=None)

One JSONL line for cachemaxxing report. The library never writes files; you do. call_site is your name for the place in your code that made the call (see Prompt).

With redact, the line holds no prompt or output text: every string in the request goes through redact (except model names, roles, block types and tool names), and the response is reduced to its token counts and cost. Raises ValueError if the response can't be read (see usage).