API reference¶
Stop paying full price for the same tokens.
Prompt
¶
Collect segments in any order; render with an adapter. Output order is fixed: tools, system, context, examples, history, retrieved, user.
call_site is any name you choose for the place in your code that makes this call, such as
"support.answer" or "rag.answer". There is no fixed list. Use one per prompt template, not one per user
or per request. It groups usage per call site in Stats, record() and cachemaxxing report, and the
report finds drift by comparing consecutive requests of the same call site. It also names Gemini
explicit caches, and it is part of prompt_cache_key on OpenAI models before GPT-5.6.
tools(tools, *, sort=True)
¶
Tool definitions in your provider's format; sorted by name unless sort=False.
context(docs, *, reorderable=False)
¶
Static reference material. reorderable=True sorts the parts so their order can't drift.
messages(messages)
¶
Your whole chat as an OpenAI-style list of {"role": ..., "content": ...} dicts, for any provider. Leading system/developer messages become the system prompt; the rest, latest user message included, is the conversation. Same as .system(...).history(...).
history(messages)
¶
Earlier turns, append-only, never modified: your provider's message format, or {"role", "content"} text turns (converted for Gemini).
retrieved(chunks)
¶
Per-request context (RAG chunks, time, user profile). Placed after history, verbatim.
prefix_hash()
¶
16 hex chars that change exactly when the tools or the static text change.
anthropic(model, *, ttl='5m', diagnose=False)
¶
kwargs for anthropic client.messages.create(max_tokens=..., **kwargs).
diagnose=True opts in to Anthropic's cache diagnostics; pass the previous response's id on later turns
and usage(response).miss_reason says what changed (system_changed, tools_changed, ...).
openai(model, *, ttl='30m', cache_key=None, diagnose=False, explicit=None)
¶
kwargs for openai client.responses.create(**kwargs). diagnose=
Azure OpenAI: model is your deployment's name. Explicit breakpoints are chosen from the name, so pass
explicit=True for a GPT-5.6+ deployment with another name, and explicit=False for a provisioned one.
openai_chat(model, *, ttl='30m', cache_key=None, diagnose=False, explicit=None)
¶
kwargs for openai client.chat.completions.create(**kwargs) and OpenAI-compatible servers.
explicit works as in openai().
gemini(model, *, cached_content=None)
¶
kwargs for google-genai client.models.generate_content(**kwargs). Implicit caching unless
cached_content (from gemini_cache) is given; then the cached fields are left out.
gemini_prefix()
¶
The cacheable part of a Gemini request: what an explicit cache holds.
litellm(model, *, ttl=None, explicit=False)
¶
kwargs for litellm.completion(**kwargs) (or the LiteLLM Proxy via the OpenAI SDK), model included.
Cache fields follow the route: Claude gets cache_control, openai/ gets openai_chat(), gemini/ stays
implicit unless explicit=True (LiteLLM then creates an explicit cache for the static prefix).
Stats
¶
Usage totals per call site, in memory. Safe to share between threads.
Give add each call's request too and Stats diagnoses misses as they happen. When a call read less than half
of its input from cache, it compares the request with the previous one from the same call site, counts the
cause in drift, shows the top cause in table(), and logs a warning with the exact change and the fix the
first time each cause shows up for a call site. warn=False keeps the diagnosis and drops the warnings.
add(call_site, u, request=None)
¶
Count one call. With request (the kwargs you sent), also diagnose it: returns the Divergence when
the call missed and differs from the previous request in a way that explains it, else None.
Usage
dataclass
¶
attributes(call_site=None, *, prefix='cachemaxxing.')
¶
Flat metrics for a trace: Langfuse or LangSmith metadata, OpenTelemetry span attributes
(span.set_attributes(u.attributes("support.answer"))). Unknown values are left out.
canonical_json(obj)
¶
Same data, same bytes: sorted keys, fixed separators, non-ASCII kept.
should_cache(prefix_tokens, expected_reuses, provider, model, *, interval_s=60.0)
¶
Whether caching a prefix of prefix_tokens pays when it is reused expected_reuses times,
interval_s seconds apart (start to start).
gemini_cache(client, p, model, *, expected_reuses, interval_s=60.0, prefix_tokens=None)
¶
Name of a live explicit cache holding p's static prefix, created only if should_cache() says it pays.
Use it as p.gemini(model, cached_content=name). None means stay on implicit caching, which is also what
happens, with a warning on the cachemaxxing logger, when the caches API fails.
The prefix is counted exactly with client.models.count_tokens (once per prefix) unless prefix_tokens is given.
gemini_cache_async(client, p, model, *, expected_reuses, interval_s=60.0, prefix_tokens=None)
async
¶
Async twin of gemini_cache, using client.aio.
record(call_site, request, response, *, redact=None)
¶
One JSONL line for cachemaxxing report. The library never writes files; you do.
call_site is your name for the place in your code that made the call (see Prompt).
With redact, the line holds no prompt or output text: every string in the request goes through
redact (except model names, roles, block types and tool names), and the response is reduced to its
token counts and cost. Raises ValueError if the response can't be read (see usage).