Changelog¶
All notable changes are documented here. The format follows Keep a Changelog and the project uses Semantic Versioning.
Unreleased¶
0.11.0 - 2026-11-05¶
Added¶
- Azure OpenAI support, with a new guide.
Prompt.openaiandopenai_chattakeexplicit=:Truefor a GPT-5.6+ deployment whose name doesn't say so,Falsefor a provisioned deployment.CacheMiddleware,CachedModelandcache_fieldstake it too, and now work withAzureChatOpenAI. Tested live ongpt-6-luna;examples/chat.py azureruns it. Cost estimates use OpenAI's list prices. - A
conversationhint for a difference in an earlier turn of the conversation, with its own advice. - Prices for
gpt-6.1-terraandgpt-6-sol.
Fixed¶
- False alarms in diagnosis when users take turns on one call site, the normal case in production.
Stats,report, the dashboard and the LangChain wrappers reported "the cached prefix broke" for calls that read 98–99% from cache, because the previous request was another user's. Now only calls that read less than half of their input from cache are diagnosed, and a differing turn isn't reported as a broken prefix. Hit rates and the--min-hit-rategate were never affected. est_costandsavednow use OpenAI's higher prices for prompts over 272K tokens, ongpt-6-luna,gpt-6.1-solandgpt-6-astra.
Changed¶
- All provider prices were re-checked on 2026-11-05.
0.10.0 - 2026-10-31¶
Added¶
- Automatic diagnosis:
stats.add(call_site, usage(resp), request=kw).Statscompares each request with the previous one from the same call site. The first time something breaks the cached prefix, it logs a warning on thecachemaxxinglogger with where the requests diverge, the text before and after, and how to fix it. It then counts each cause instats.drift, andstats.table()shows the top cause per call site.Stats(warn=False)keeps the diagnosis without the warnings.add()returns theDivergencewhen a call broke the prefix.CacheMiddlewareandCachedModeldiagnose automatically. Divergence.fix: what to change for each kind of divergence. Log warnings,cachemaxxing report,cachemaxxing diffand the dashboard all show the same advice.- A tutorial that takes a support bot with a timestamp at the top of its prompt from 0% to 98% cached input, with real output, plus
examples/tutorial.py.
Changed¶
cachemaxxing reportprints, under its table, the most frequent cause for each call site that missed, with the first example and the fix.cachemaxxing diffadds afix:line.- The "prefix matched but nothing was read" drift is now labelled
prefix ok, no cache read. The explanation moved to the diagnosis below the table.
0.9.0 - 2026-10-31¶
This is the release candidate for 1.0. The public API is now frozen.
Added¶
- A "Running in production" guide covering rollout, what happens when something fails, logs, concurrency, costs and the stability policy.
- The public API is listed in
design/api.md, andtests/api.txtfreezes every public name and signature. From 1.0, renames and removals go through a deprecation first. SECURITY.mdwith private vulnerability reporting,CONTRIBUTING.md, and issue templates for bugs and wrong prices.
Changed¶
- The development status is now Beta.
Fixed¶
gemini_cacheandgemini_cache_asyncno longer make your call fail when the Gemini caches API fails. They returnNone, so the call uses implicit caching, and log a warning on thecachemaxxinglogger.
0.5.0 - 2026-10-31¶
Added¶
record(..., redact=fn)writes a log line without prompt or output text. Every string in the request goes throughfn, except model names, roles, block types and tool names. The response is reduced to its token counts and cost.reportgives the same hit rates, costs and drifting fields as on a plain log.CacheMiddlewareandCachedModeltake the sameredact=for theiron_recordlines.
Changed¶
- The
report --htmldashboard draws at most the last 5,000 calls of each call site, and says so when there are more, so the page stays small for any log size. The totals and the table still count every call. - The dashboard page has a Content-Security-Policy, so the browser itself blocks any network request.
Fixed¶
Statsis safe to share between threads. Before, concurrentadd()calls could lose counts.- Concurrent
gemini_cacheandgemini_cache_asynccalls for the same prefix now create one explicit cache. Before, each caller could create its own, and each was billed for storage.
0.4.1 - 2026-10-31¶
Fixed¶
CacheMiddlewareandCachedModelno longer make a model call fail when adding cache fields or measuring the reply fails, including when youron_recordraises. The call goes through, and a warning goes to thecachemaxxinglogger. A wrong setting, such as a TTL the provider doesn't offer, still raises before anything is sent.CacheMiddlewarenow also checks the TTL when it is created.usage()now raises onlyValueErrorfor responses it can't read. Before, some unexpected field types raisedAttributeErrororTypeError.
Changed¶
cachemaxxing reportreads the log line by line instead of loading it whole. It skips lines it can't read and counts them on stderr; before, the first one stopped the report. Add--strictto stop at the first unreadable line, with exit code 2.
0.4.0 - 2026-10-31¶
Added¶
cachemaxxing report LOG --html PATHwrites a self-contained dashboard page. It shows the totals, the hit rate of every call, a sortable table of call sites, and each divergence in context with a fix. The page makes no network requests.- A landing page at the docs root, and a demo dashboard built from a real log. The docs home moved to
overview/. Usage.saved: USD saved compared with the same call without caching. It is negative when cache writes cost more than the reads saved.Statsandreportshow asaved $column.Usage.attributes(): a call's cache metrics as flat keys for Langfuse or LangSmith metadata and OpenTelemetry span attributes.Prompt.messages(chat)takes a chat kept as an OpenAI-style list of{"role", "content"}messages and renders it for any provider. Leading system messages become the cached system prompt. Gemini gets these turns converted to its own format.- Simple examples:
examples/chat.py, a plain message list, andexamples/langchain_chat.py, withSystemMessage,HumanMessageandAIMessage. The quickstart now shows where each provider's cache markers go and the exact request that is sent. call_siteis documented in the API reference and in the diagnosing guide.CacheMiddlewareandCachedModelattach these metrics to every reply asresponse_metadata["cachemaxxing"].
0.3.0 - 2026-10-31¶
Added¶
diagnose=onanthropic()andopenai()requests the provider's own cache-miss verdict (Anthropic cache diagnostics, OpenAI prompt-cache diagnostics).usage(...).miss_reasoncarries it, andStatsandreportshow it per call site.cachemaxxing.count.prefix_tokens(): exact prefix token counts from the Anthropic, OpenAI or google-genai client.- GitHub Action (
uses: ashhadahsan/cachemaxxing@v0.3.0): fails the job when a call site's hit rate drops belowmin-hit-rate, and writes the report to the job summary.
Changed¶
- Benchmarks v2:
- a third arm, a clean prompt using only the provider's own automatic caching
- RAG and LangChain-agent workloads
claude-sonnet-5-5andgpt-6.1-sol- every arm starts cold with its own prompt
- costs compared on the input side, since output length varies with the answer and not with caching
gemini_cache()counts the prefix exactly (once per prefix) instead of estimating at ~4 characters per token. On the benchmark prompt the estimate was 17% below Gemini's count (and 41% below Anthropic's).
0.2.0 - 2026-10-30¶
Added¶
- Logo (
docs/assets/logo.svg), used in the README and as the docs site's logo and favicon. cachemaxxing.langchain.CacheMiddlewarefor LangChain 1.x / LangGraph agents: per-provider cache fields on every model call (OpenAI GPT-5.6+ breakpoints, Anthropic markers and automatic caching), usage per call site, and anon_recordhook forcachemaxxing report. New extra:cachemaxxing[langchain].cachemaxxing.langchain.CachedModel(wrap a chat model for hand-built LangGraph graphs and LCEL chains) andcache_fields()(the low-level piece).examples/langchain_agent.pyandexamples/langgraph_custom.py.
Fixed¶
usage()on a LangChainAIMessagereturned all zeros; it now readsusage_metadatafor every provider integration.diffshows short values whole instead of a window cut mid-word.
0.1.0 - 2026-10-30¶
Added¶
Prompt: segments ordered from most to least stable (tools, system, context, examples, history, retrieved, user), with deterministic serialization.- Adapters:
anthropic()(cache_control at stability boundaries),openai()/openai_chat()(explicit breakpoints on GPT-5.6+,prompt_cache_keyon older models),gemini()(implicit by default),litellm()(route-aware markers). usage(): one normalized shape for Anthropic, OpenAI Responses and Chat, Gemini and LiteLLM responses, with list-price cost estimates;Statsper call site;record()for JSONL logs.diff(): the first point where two requests diverge, with hints (timestamp, uuid, tool order, key order, ...).should_cache(): whether explicit caching pays, per provider and model.gemini_cache()/gemini_cache_async(): create or reuse Gemini explicit caches only when they pay.- CLI:
cachemaxxing diff A Bandcachemaxxing report LOG --min-hit-rate X. - Provider facts table with sources, verified 2026-10-30.
- Benchmarks: naive vs cachemaxxing on claude-haiku-5-5, gpt-6-luna and gemini-3.8-flash (8-turn conversations, cold start, one customer per conversation), with results JSON and a chart.
examples/quickstart.pyfor Anthropic, OpenAI, Gemini and LiteLLM.examples/rag_chat.py: a multi-turn RAG chat with per-turn retrieval, append-only history,diffbetween turns, JSONL logging and aStatstable.