maf-cachebench 0.1.0__tar.gz

maf_cachebench-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SOKOLAI BV
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

maf_cachebench-0.1.0/PKG-INFO

@@ -0,0 +1,125 @@
+Metadata-Version: 2.4
+Name: maf-cachebench
+Version: 0.1.0
+Summary: Benchmark for Microsoft Agent Framework compaction strategies: what each one costs in prompt-cache hits and what it loses, measured on a replayed conversation or a live agent.
+Keywords: benchmark,compaction,prompt-cache,agent-framework,microsoft-agent-framework
+Author: SOKOLAI BV
+Author-email: SOKOLAI BV <info@sokolai.com>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Typing :: Typed
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Testing
+Requires-Dist: agent-framework-core>=1.20.0,<1.21
+Requires-Dist: maf-compaction>=0.0.0,<0.2
+Requires-Dist: httpx>=0.27,<1
+Requires-Dist: agent-framework-foundry>=1.14.0,<2 ; extra == 'foundry'
+Requires-Dist: azure-identity>=1.17,<2 ; extra == 'foundry'
+Requires-Dist: agent-framework-mistral>=1.0.0b260918,<2 ; extra == 'mistral'
+Requires-Dist: agent-framework-ollama>=1.0.0b261002,<2 ; extra == 'ollama'
+Requires-Dist: ollama>=0.5.3,<0.6 ; extra == 'ollama'
+Requires-Dist: agent-framework-openai>=1.15.0,<2 ; extra == 'openai'
+Requires-Dist: azure-identity>=1.17,<2 ; extra == 'openai'
+Requires-Dist: tiktoken>=0.8,<1 ; extra == 'tiktoken'
+Requires-Python: >=3.12, <3.15
+Project-URL: Homepage, https://www.sokol.ai
+Project-URL: Source, https://github.com/sokolaidev/maf-extensions
+Project-URL: Issues, https://github.com/sokolaidev/maf-extensions/issues
+Provides-Extra: foundry
+Provides-Extra: mistral
+Provides-Extra: ollama
+Provides-Extra: openai
+Provides-Extra: tiktoken
+Description-Content-Type: text/markdown
+
+# maf-cachebench
+
+[![PyPI](https://img.shields.io/pypi/v/maf-cachebench)](https://pypi.org/project/maf-cachebench/) [![Python](https://img.shields.io/pypi/pyversions/maf-cachebench)](https://pypi.org/project/maf-cachebench/) [![License](https://img.shields.io/badge/license-MIT-green)](https://github.com/sokolaidev/maf-extensions/blob/main/LICENSE)
+
+> **Experimental.** Releases before 1.0 may change or remove APIs. Importing this package emits `MafCachebenchExperimentalWarning`.
+
+A benchmark for [Microsoft Agent Framework](https://aka.ms/AgentFramework) compaction strategies. It measures what each strategy costs in provider prompt-cache hits and what it destroys while doing it: the framework's own strategies and the ones in [`maf-compaction`](https://pypi.org/project/maf-compaction/), twenty in all, on the same conversation, priced at the provider's cached and uncached rates.
+
+This is an independent package. It is not affiliated with or endorsed by Microsoft.
+
+## What it measures
+
+Provider prompt caches match on an exact prefix, and every compaction strategy rewrites history inside that prefix, so compaction breaks the cache by construction. The question is what that costs and what it saves. The benchmark plants facts in a conversation's tool results, runs the conversation under each strategy, and then asks for the facts back from the compacted context. Each strategy gets a cost, a cache hit rate, and a count of facts it kept, and the strategies are ranked on cost behind an accuracy bar.
+
+Two harnesses answer two questions:
+
+- `cachebench` replays a scripted transcript. Every provider and strategy replays the same scripted structure, with a fixed-width per-cell salt isolating its cache prefix. These controlled workloads support comparisons across providers. It answers how a *prompt* caches.
+- `cachebench_live` drives a real agent, with real replies and real tool calls, so compaction acts on the history an agent would accumulate. Its numbers compare strategies within one model. It answers what a strategy costs an agent.
+
+## Install
+
+```bash
+pip install maf-cachebench[openai]        # Azure OpenAI, OpenRouter and the Azure Responses route
+pip install maf-cachebench[foundry]       # Microsoft Foundry project endpoints
+pip install maf-cachebench[mistral]
+pip install maf-cachebench[ollama]
+pip install maf-cachebench[tiktoken]      # then pass --tokenizer tiktoken for exact counts
+```
+
+Five commands land on the path: `cachebench`, `cachebench_live`, `cachebench_advise`, `cachebench_recall` and `cachebench_summary`. Each prints its own `--help`.
+
+## Pointing it at a provider
+
+A provider is selected as `provider` or `provider:model`, so two models on one provider can sit in one run. Credentials come from the environment:
+
+| Provider | Variables |
+|---|---|
+| `azure` | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_CHAT_COMPLETION_MODEL` |
+| `azure-responses` | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_RESPONSES_MODEL`, and a working `DefaultAzureCredential` (Entra authentication) |
+| `foundry` | `FOUNDRY_PROJECT_ENDPOINT`, `FOUNDRY_MODEL`, and a working `DefaultAzureCredential` |
+| `openrouter` | `OPENROUTER_API_KEY`, `OPENROUTER_MODEL`; pin `OPENROUTER_PROVIDER_ORDER`, or you measure the router |
+| `mistral` | `MISTRAL_API_KEY`, `MISTRAL_CHAT_MODEL` |
+| `ollama` | `OLLAMA_MODEL`, optionally `OLLAMA_HOST` and `OLLAMA_API_KEY`; this provider caches and never reports it |
+
+Prices are passed as `--price-input`, `--price-cached` and `--price-output` per million tokens, except on OpenRouter where they are discovered.
+
+## A first run
+
+```bash
+# see the plan and the prompt sizes without spending anything
+cachebench_live foundry:gpt-5.6-luna --dry-run --strategies none,anchored,tool_summary_anchored
+
+# a live cell: a 120K window, a conversation sized to 1.5 times it, five seeds, records kept
+cachebench_live foundry:gpt-5.6-luna --agent harness --context-window 120000 --fill 1.5 --repeats 5 \
+  --strategies none,tool_summary_anchored,tool_and_user_summary_anchored --summarizer-provider foundry:gpt-5.6-luna \
+  --price-input 0.20 --price-cached 0.02 --price-output 1.20 --results-jsonl runs/cell.jsonl
+
+# render the table again from the records, calling nothing
+cachebench_live --from-jsonl runs/cell.jsonl
+```
+
+`--results-jsonl` is not optional in practice: a cell runs for hours, and the file is appended one record per seed the moment it is scored, so a stopped run keeps what it paid for. `--from-jsonl` rebuilds the table and the verdict from the records with no provider configured, and files that measured the same cell merge into one table.
+
+## Reading the table
+
+Rows that keep at least `--min-correctness` of the control's accuracy come first, cheapest first; the rest follow below a line. The columns to read, in order:
+
+- `seed$` is what the conversation cost, the strategy's own summariser calls included and the probes left out. The ranking and the verdict are on it.
+- `seed$+-` is the spread between the cheapest and dearest seed. A gap smaller than this is not a result, and the verdict line says `NOT SUPPORTED` when that happens.
+- `seed hit%` is the share of the conversation's input the provider served from its cache, probes excluded. This is what compaction did to the cache.
+- `facts` and `lost` are the planted facts that survived into the compacted context, and the ones compaction removed.
+- `acc1` and `acc2` are the model's accuracy on the scoped and the combined closing questions, asked from the same restored snapshot.
+- `dq` marks a row that sent a prompt larger than the window it stood in for. A cell that disqualifies at all leaves the ranking.
+- `flags` is where a row says it is not measuring what its name claims. Read it before the money columns.
+
+The full option reference, the flags legend and the measurement design are in the [benchmark documentation](https://github.com/sokolaidev/maf-extensions/blob/main/docs/compaction/cachebench.md), and the findings in the [compaction overview](https://github.com/sokolaidev/maf-extensions/blob/main/docs/compaction/README.md).
+
+## What it depends on
+
+`agent-framework-core`, pinned to one minor because the benchmark reads the framework's private `_compaction` helpers; `maf-compaction` for the strategies it was built to measure; and `httpx` for the advisor's price lookup. The provider clients are extras.
+
+## Licence
+
+MIT. See the [LICENSE](https://github.com/sokolaidev/maf-extensions/blob/main/LICENSE).
+
+All commands default to the dependency-free estimator. Install the `tiktoken` extra and pass `--tokenizer tiktoken` for BPE counts. Explicit price rates must be finite and non-negative.

maf_cachebench-0.1.0/README.md

@@ -0,0 +1,86 @@
+# maf-cachebench
+
+[![PyPI](https://img.shields.io/pypi/v/maf-cachebench)](https://pypi.org/project/maf-cachebench/) [![Python](https://img.shields.io/pypi/pyversions/maf-cachebench)](https://pypi.org/project/maf-cachebench/) [![License](https://img.shields.io/badge/license-MIT-green)](https://github.com/sokolaidev/maf-extensions/blob/main/LICENSE)
+
+> **Experimental.** Releases before 1.0 may change or remove APIs. Importing this package emits `MafCachebenchExperimentalWarning`.
+
+A benchmark for [Microsoft Agent Framework](https://aka.ms/AgentFramework) compaction strategies. It measures what each strategy costs in provider prompt-cache hits and what it destroys while doing it: the framework's own strategies and the ones in [`maf-compaction`](https://pypi.org/project/maf-compaction/), twenty in all, on the same conversation, priced at the provider's cached and uncached rates.
+
+This is an independent package. It is not affiliated with or endorsed by Microsoft.
+
+## What it measures
+
+Provider prompt caches match on an exact prefix, and every compaction strategy rewrites history inside that prefix, so compaction breaks the cache by construction. The question is what that costs and what it saves. The benchmark plants facts in a conversation's tool results, runs the conversation under each strategy, and then asks for the facts back from the compacted context. Each strategy gets a cost, a cache hit rate, and a count of facts it kept, and the strategies are ranked on cost behind an accuracy bar.
+
+Two harnesses answer two questions:
+
+- `cachebench` replays a scripted transcript. Every provider and strategy replays the same scripted structure, with a fixed-width per-cell salt isolating its cache prefix. These controlled workloads support comparisons across providers. It answers how a *prompt* caches.
+- `cachebench_live` drives a real agent, with real replies and real tool calls, so compaction acts on the history an agent would accumulate. Its numbers compare strategies within one model. It answers what a strategy costs an agent.
+
+## Install
+
+```bash
+pip install maf-cachebench[openai]        # Azure OpenAI, OpenRouter and the Azure Responses route
+pip install maf-cachebench[foundry]       # Microsoft Foundry project endpoints
+pip install maf-cachebench[mistral]
+pip install maf-cachebench[ollama]
+pip install maf-cachebench[tiktoken]      # then pass --tokenizer tiktoken for exact counts
+```
+
+Five commands land on the path: `cachebench`, `cachebench_live`, `cachebench_advise`, `cachebench_recall` and `cachebench_summary`. Each prints its own `--help`.
+
+## Pointing it at a provider
+
+A provider is selected as `provider` or `provider:model`, so two models on one provider can sit in one run. Credentials come from the environment:
+
+| Provider | Variables |
+|---|---|
+| `azure` | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_CHAT_COMPLETION_MODEL` |
+| `azure-responses` | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_RESPONSES_MODEL`, and a working `DefaultAzureCredential` (Entra authentication) |
+| `foundry` | `FOUNDRY_PROJECT_ENDPOINT`, `FOUNDRY_MODEL`, and a working `DefaultAzureCredential` |
+| `openrouter` | `OPENROUTER_API_KEY`, `OPENROUTER_MODEL`; pin `OPENROUTER_PROVIDER_ORDER`, or you measure the router |
+| `mistral` | `MISTRAL_API_KEY`, `MISTRAL_CHAT_MODEL` |
+| `ollama` | `OLLAMA_MODEL`, optionally `OLLAMA_HOST` and `OLLAMA_API_KEY`; this provider caches and never reports it |
+
+Prices are passed as `--price-input`, `--price-cached` and `--price-output` per million tokens, except on OpenRouter where they are discovered.
+
+## A first run
+
+```bash
+# see the plan and the prompt sizes without spending anything
+cachebench_live foundry:gpt-5.6-luna --dry-run --strategies none,anchored,tool_summary_anchored
+
+# a live cell: a 120K window, a conversation sized to 1.5 times it, five seeds, records kept
+cachebench_live foundry:gpt-5.6-luna --agent harness --context-window 120000 --fill 1.5 --repeats 5 \
+  --strategies none,tool_summary_anchored,tool_and_user_summary_anchored --summarizer-provider foundry:gpt-5.6-luna \
+  --price-input 0.20 --price-cached 0.02 --price-output 1.20 --results-jsonl runs/cell.jsonl
+
+# render the table again from the records, calling nothing
+cachebench_live --from-jsonl runs/cell.jsonl
+```
+
+`--results-jsonl` is not optional in practice: a cell runs for hours, and the file is appended one record per seed the moment it is scored, so a stopped run keeps what it paid for. `--from-jsonl` rebuilds the table and the verdict from the records with no provider configured, and files that measured the same cell merge into one table.
+
+## Reading the table
+
+Rows that keep at least `--min-correctness` of the control's accuracy come first, cheapest first; the rest follow below a line. The columns to read, in order:
+
+- `seed$` is what the conversation cost, the strategy's own summariser calls included and the probes left out. The ranking and the verdict are on it.
+- `seed$+-` is the spread between the cheapest and dearest seed. A gap smaller than this is not a result, and the verdict line says `NOT SUPPORTED` when that happens.
+- `seed hit%` is the share of the conversation's input the provider served from its cache, probes excluded. This is what compaction did to the cache.
+- `facts` and `lost` are the planted facts that survived into the compacted context, and the ones compaction removed.
+- `acc1` and `acc2` are the model's accuracy on the scoped and the combined closing questions, asked from the same restored snapshot.
+- `dq` marks a row that sent a prompt larger than the window it stood in for. A cell that disqualifies at all leaves the ranking.
+- `flags` is where a row says it is not measuring what its name claims. Read it before the money columns.
+
+The full option reference, the flags legend and the measurement design are in the [benchmark documentation](https://github.com/sokolaidev/maf-extensions/blob/main/docs/compaction/cachebench.md), and the findings in the [compaction overview](https://github.com/sokolaidev/maf-extensions/blob/main/docs/compaction/README.md).
+
+## What it depends on
+
+`agent-framework-core`, pinned to one minor because the benchmark reads the framework's private `_compaction` helpers; `maf-compaction` for the strategies it was built to measure; and `httpx` for the advisor's price lookup. The provider clients are extras.
+
+## Licence
+
+MIT. See the [LICENSE](https://github.com/sokolaidev/maf-extensions/blob/main/LICENSE).
+
+All commands default to the dependency-free estimator. Install the `tiktoken` extra and pass `--tokenizer tiktoken` for BPE counts. Explicit price rates must be finite and non-negative.

maf_cachebench-0.1.0/pyproject.toml

@@ -0,0 +1,116 @@
+[project]
+name = "maf-cachebench"
+version = "0.1.0"
+description = "Benchmark for Microsoft Agent Framework compaction strategies: what each one costs in prompt-cache hits and what it loses, measured on a replayed conversation or a live agent."
+readme = "README.md"
+requires-python = ">=3.12,<3.15"
+license = "MIT"
+license-files = ["LICENSE"]
+keywords = [
+    "benchmark",
+    "compaction",
+    "prompt-cache",
+    "agent-framework",
+    "microsoft-agent-framework",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Typing :: Typed",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Topic :: Software Development :: Testing",
+]
+dependencies = [
+    "agent-framework-core>=1.20.0,<1.21",
+    "maf-compaction>=0.0.0,<0.2",
+    "httpx>=0.27,<1",
+]
+
+[[project.authors]]
+name = "SOKOLAI BV"
+email = "info@sokolai.com"
+
+[project.optional-dependencies]
+openai = [
+    "agent-framework-openai>=1.15.0,<2",
+    "azure-identity>=1.17,<2",
+]
+foundry = [
+    "agent-framework-foundry>=1.14.0,<2",
+    "azure-identity>=1.17,<2",
+]
+mistral = ["agent-framework-mistral>=1.0.0b260918,<2"]
+ollama = [
+    "agent-framework-ollama>=1.0.0b261002,<2",
+    "ollama>=0.5.3,<0.6",
+]
+tiktoken = ["tiktoken>=0.8,<1"]
+
+[project.scripts]
+cachebench = "maf_cachebench:main"
+cachebench_live = "maf_cachebench:live_main"
+cachebench_advise = "maf_cachebench:advise_main"
+cachebench_recall = "maf_cachebench:recall_main"
+cachebench_summary = "maf_cachebench:summary_main"
+
+[project.urls]
+Homepage = "https://www.sokol.ai"
+Source = "https://github.com/sokolaidev/maf-extensions"
+Issues = "https://github.com/sokolaidev/maf-extensions/issues"
+
+[tool.uv.sources.maf-compaction]
+workspace = true
+
+[tool.uv.build-backend]
+module-name = "maf_cachebench"
+module-root = "src"
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+
+[tool.ruff.lint]
+extend-select = [
+    "I",
+    "UP",
+    "D100",
+    "D101",
+    "D103",
+    "D104",
+    "E501",
+    "W505",
+    "TD",
+    "FIX",
+    "ERA001",
+]
+
+[tool.ruff.lint.per-file-ignores]
+"tests/**" = [
+    "D101",
+    "D103",
+    "E501",
+    "W505",
+]
+"samples/**" = [
+    "D101",
+    "D103",
+    "E501",
+    "W505",
+]
+
+[tool.ruff.lint.pycodestyle]
+max-doc-length = 100
+
+[tool.pyright]
+include = ["src"]
+typeCheckingMode = "strict"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[build-system]
+requires = ["uv_build>=0.11.24,<0.12.0"]
+build-backend = "uv_build"

maf_cachebench-0.1.0/pyproject.toml.orig

@@ -0,0 +1,94 @@
+[project]
+name = "maf-cachebench"
+version = "0.1.0"
+description = "Benchmark for Microsoft Agent Framework compaction strategies: what each one costs in prompt-cache hits and what it loses, measured on a replayed conversation or a live agent."
+readme = "README.md"
+requires-python = ">=3.12,<3.15"
+authors = [{ name = "SOKOLAI BV", email = "info@sokolai.com" }]
+license = "MIT"
+license-files = ["LICENSE"]
+keywords = [
+    "benchmark",
+    "compaction",
+    "prompt-cache",
+    "agent-framework",
+    "microsoft-agent-framework",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Typing :: Typed",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Topic :: Software Development :: Testing",
+]
+dependencies = [
+    # The strategies under measurement, and the private `_compaction` helpers both packages
+    # read; the same minor as maf-compaction pins, for the same reason.
+    "agent-framework-core>=1.20.0,<1.21",
+    # Include the unreleased workspace version and the first release line.
+    "maf-compaction>=0.0.0,<0.2",
+    # Model prices are fetched from OpenRouter for the advisor's cost table.
+    "httpx>=0.27,<1",
+]
+
+[project.optional-dependencies]
+# One extra per provider route; a run needs only the client it points at. The floors are the
+# client versions the published measurements were made with; the Mistral and Ollama clients
+# have only pre-release versions on the index, so their floors name one.
+openai = ["agent-framework-openai>=1.15.0,<2", "azure-identity>=1.17,<2"]
+foundry = ["agent-framework-foundry>=1.14.0,<2", "azure-identity>=1.17,<2"]
+mistral = ["agent-framework-mistral>=1.0.0b260918,<2"]
+# The Ollama route builds the client's own async HTTP client, so that package is named too.
+ollama = ["agent-framework-ollama>=1.0.0b261002,<2", "ollama>=0.5.3,<0.6"]
+tiktoken = ["tiktoken>=0.8,<1"]
+
+[project.scripts]
+cachebench = "maf_cachebench:main"
+cachebench_live = "maf_cachebench:live_main"
+cachebench_advise = "maf_cachebench:advise_main"
+cachebench_recall = "maf_cachebench:recall_main"
+cachebench_summary = "maf_cachebench:summary_main"
+
+[project.urls]
+Homepage = "https://www.sokol.ai"
+Source = "https://github.com/sokolaidev/maf-extensions"
+Issues = "https://github.com/sokolaidev/maf-extensions/issues"
+
+[tool.uv.sources]
+maf-compaction = { workspace = true }
+
+[tool.uv.build-backend]
+module-name = "maf_cachebench"
+module-root = "src"
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+
+[tool.ruff.lint]
+extend-select = ["I", "UP", "D100", "D101", "D103", "D104", "E501", "W505", "TD", "FIX", "ERA001"]
+
+[tool.ruff.lint.per-file-ignores]
+# ruff resolves [tool.ruff] per file by the NEAREST ancestor pyproject.toml that has one, so
+# this package carries its own copy rather than inheriting one from the workspace root.
+# The length rules join the docstring pair: test fixtures and probe scripts are lenient, not
+# API surface.
+"tests/**" = ["D101", "D103", "E501", "W505"]
+"samples/**" = ["D101", "D103", "E501", "W505"]
+
+[tool.ruff.lint.pycodestyle]
+max-doc-length = 100
+
+[tool.pyright]
+include = ["src"]
+typeCheckingMode = "strict"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[build-system]
+requires = ["uv_build>=0.11.24,<0.12.0"]
+build-backend = "uv_build"

maf_cachebench-0.1.0/src/maf_cachebench/__init__.py

@@ -0,0 +1,236 @@
+"""Compaction and prompt-cache interaction benchmark for Agent Framework.
+
+Provider prompt caches match on exact prefixes, and every compaction strategy works by
+excluding or rewriting messages inside an existing history. Compaction therefore breaks
+the cached prefix by construction, and the question this package answers empirically is
+what that costs: how much of each prompt stays reusable under a given strategy, how much
+of that reusable prefix a given provider actually serves from cache, and whether
+compacting earlier saves more prompt tokens than it loses in cache reads.
+
+Measurements replay deterministic scripted content and structure across providers and
+strategies, with fixed-width per-cell salts isolating their cache prefixes.
+"""
+
+import importlib.metadata
+
+from ._advise_cli import main as advise_main
+from ._advisor import ModelPricing, StrategyCost, Verdict, advise, cost_of, fetch_openrouter_pricing
+from ._cli import build_parser, main, run_benchmark
+from ._fill import FillPlan, plan_fill
+from ._live import (
+    AGENT_KINDS,
+    DEFAULT_COMBINED_REPEATS,
+    DEFAULT_PROBE_REPEATS,
+    IdentifiedHistoryProvider,
+    LiveOutcome,
+    MeteredClient,
+    ModelCall,
+    ProbeOutcome,
+    SummarizerUsage,
+    UsageRecorder,
+    build_live_agent,
+    build_live_scenario,
+    find_nested_strategy,
+    make_lookup_tool,
+    probe_count,
+    restore_state,
+    run_live,
+    score_combined_samples,
+    score_samples,
+    serialize_history,
+    snapshot_state,
+    unretrieved_facts,
+    wants_client_side_history,
+)
+from ._live_cli import main as live_main
+from ._metrics import (
+    common_message_prefix,
+    percentile,
+    serialize_message,
+    summarize_cell,
+    token_counts,
+)
+from ._providers import (
+    PROVIDER_SPECS,
+    ProviderRuntime,
+    ProviderSpec,
+    build_provider,
+    parse_provider_selector,
+    prompt_cache_key,
+    prompt_cache_key_options,
+    provider_names,
+)
+from ._recall import (
+    Contradiction,
+    FactOutcome,
+    PlantedFact,
+    RecallScenario,
+    RecallScore,
+    build_recall_scenario,
+    score_answer,
+)
+from ._recall_cli import main as recall_main
+from ._records import (
+    SCHEMA_VERSION,
+    CellParams,
+    SeedRecord,
+    StrategySettings,
+    WorkloadSettings,
+    append_seed_record,
+    group_by_cell,
+    read_seed_records,
+)
+from ._report import render_summary_table, write_records_jsonl, write_summary_csv
+from ._runner import CallOutcome, ProviderCaller, TurnCaller, run_cell, unsupported_option
+from ._strategies import (
+    STRATEGIES_FORCING_RECORDS,
+    STRATEGIES_NEEDING_SUMMARIZER,
+    STRATEGY_BUILDERS,
+    StrategyOptions,
+    build_strategy,
+    forces_records,
+    resolve_context_window,
+    strategy_names,
+)
+from ._summary import JointOutcome, JointVerdict, recommend, relative_correctness
+from ._summary_cli import main as summary_main
+from ._tokenizers import TOKENIZER_NAMES, build_tokenizer
+from ._transcripts import (
+    DEFAULT_SYSTEM_TOKENS,
+    TRANSCRIPT_PRESETS,
+    TranscriptPreset,
+    build_preset,
+    build_transcript,
+)
+from ._types import CellKey, CellSummary, Transcript, TranscriptTurn, TurnRecord
+
+try:
+    __version__ = importlib.metadata.version(__name__)
+except importlib.metadata.PackageNotFoundError:
+    __version__ = "0.1.0"  # Fallback for development mode
+
+__all__ = [
+    "AGENT_KINDS",
+    "DEFAULT_COMBINED_REPEATS",
+    "DEFAULT_PROBE_REPEATS",
+    "DEFAULT_SYSTEM_TOKENS",
+    "PROVIDER_SPECS",
+    "SCHEMA_VERSION",
+    "STRATEGIES_FORCING_RECORDS",
+    "STRATEGIES_NEEDING_SUMMARIZER",
+    "STRATEGY_BUILDERS",
+    "TOKENIZER_NAMES",
+    "TRANSCRIPT_PRESETS",
+    "CallOutcome",
+    "CellKey",
+    "CellParams",
+    "CellSummary",
+    "Contradiction",
+    "FactOutcome",
+    "FillPlan",
+    "IdentifiedHistoryProvider",
+    "JointOutcome",
+    "JointVerdict",
+    "LiveOutcome",
+    "MeteredClient",
+    "ModelCall",
+    "SummarizerUsage",
+    "ModelPricing",
+    "PlantedFact",
+    "ProbeOutcome",
+    "ProviderCaller",
+    "ProviderRuntime",
+    "ProviderSpec",
+    "RecallScenario",
+    "RecallScore",
+    "SeedRecord",
+    "StrategyCost",
+    "StrategyOptions",
+    "StrategySettings",
+    "Transcript",
+    "TranscriptPreset",
+    "TranscriptTurn",
+    "TurnCaller",
+    "TurnRecord",
+    "UsageRecorder",
+    "Verdict",
+    "WorkloadSettings",
+    "MafCachebenchExperimentalWarning",
+    "__version__",
+    "advise",
+    "advise_main",
+    "append_seed_record",
+    "build_live_agent",
+    "build_live_scenario",
+    "build_parser",
+    "build_preset",
+    "build_provider",
+    "build_recall_scenario",
+    "build_strategy",
+    "build_tokenizer",
+    "build_transcript",
+    "common_message_prefix",
+    "cost_of",
+    "fetch_openrouter_pricing",
+    "find_nested_strategy",
+    "forces_records",
+    "group_by_cell",
+    "live_main",
+    "main",
+    "make_lookup_tool",
+    "parse_provider_selector",
+    "percentile",
+    "plan_fill",
+    "probe_count",
+    "prompt_cache_key",
+    "prompt_cache_key_options",
+    "provider_names",
+    "read_seed_records",
+    "recall_main",
+    "recommend",
+    "relative_correctness",
+    "render_summary_table",
+    "resolve_context_window",
+    "restore_state",
+    "run_benchmark",
+    "run_cell",
+    "run_live",
+    "score_answer",
+    "score_combined_samples",
+    "score_samples",
+    "serialize_history",
+    "serialize_message",
+    "snapshot_state",
+    "strategy_names",
+    "summarize_cell",
+    "summary_main",
+    "token_counts",
+    "unretrieved_facts",
+    "unsupported_option",
+    "wants_client_side_history",
+    "write_records_jsonl",
+    "write_summary_csv",
+]
+
+# Experimental package: importing it emits a UserWarning rather than a FutureWarning, so a host
+# running under `python -W error` can still import it.
+import warnings as _warnings
+
+
+class MafCachebenchExperimentalWarning(UserWarning):
+    """Warning category for maf-cachebench's experimental-package notice."""
+
+
+def _warn_experimental() -> None:
+    message = (
+        "maf_cachebench is experimental and may change or be removed in future versions "
+        "without notice."
+    )
+    try:
+        _warnings.warn(message, category=MafCachebenchExperimentalWarning, stacklevel=2)
+    except MafCachebenchExperimentalWarning:
+        # Deliberate: under `-W error` an informational notice must not fail the import.
+        pass
+
+
+_warn_experimental()