tokenaudit 0.2.0__tar.gz → 0.5.0__tar.gz

tokenaudit-0.5.0/CHANGELOG.md

@@ -0,0 +1,126 @@
+# Changelog
+
+## 0.5.0 — 2026-11-12
+
+From the [TradingAgents on Claude case study](docs/case-studies/tradingagents-claude.md):
+
+- New detector `unread_cache_writes`: requests that write to the cache when
+  nothing later re-sends what they wrote, so the write premium buys nothing.
+  HIGH when the session's writes cost more than its reads saved. Defers to
+  prefix_drift, ttl_expiry, tool_schema_churn, breakpoint_limit and
+  lookback_overflow, which explain a missed read better.
+- `missing_cache_markers` counts tool definitions in the shared prefix (they
+  come first in an Anthropic prompt, and different tools share nothing) and
+  sizes each repeat from the earlier request's reported tokens instead of
+  characters. On the Claude runs its estimate went from 2k-5k to 8k-22k tokens
+  per run, in line with what caching actually read back.
+- `import tokenaudit.counterfactual` on its own no longer fails with a
+  circular import.
+
+## 0.4.0 — 2026-11-12
+
+Upgrading adds findings to traces with these patterns, and changes the
+impact of some existing ones: re-baseline `tokenaudit check` after upgrading.
+
+New detectors, each with a positive corpus fixture and near-miss negatives:
+
+- `misordered_shared_context`: the same reports or documents sent to several
+  agents, each behind its own role text, so the prefix cache never reuses
+  them.
+- `cross_session_drift`: a date, user or ticker near the start of a long
+  prompt, which keeps it from being reused across sessions.
+- `tool_schema_bloat`: tool definitions sent on every request and never
+  called by any request carrying the same tool set.
+- `missing_cache_markers`: a provider that only caches on request
+  (Anthropic), a session with no markers and no cache activity, and requests
+  re-sending a prefix an earlier one sent within the cache lifetime.
+
+The first two find the TradingAgents problems the case study diagnosed by
+hand. The captured runs, previously reported as clean, now show both
+findings and a 25% health floor (was 100%).
+
+Pricing:
+
+- Every finding is priced with its model's read and write rates from the
+  rulebook. A request that wrote the cache is charged the write premium
+  (1.25x on Anthropic's 5-minute cache) instead of full price, and the
+  assumption line names the rates used.
+- `diagnose` and the HTML report show a dollar range when the rulebook has a
+  verified input price for the model.
+
+Ingest:
+
+- Raw provider logs: JSONL or a JSON array of logged request/response pairs
+  from the OpenAI Responses, OpenAI Chat Completions and Anthropic Messages
+  APIs. Anthropic's exclusive token counts are converted; records without
+  timestamps are flagged and time-based detectors abstain.
+- Tool definitions (name and size) and the tools each response called, from
+  OpenInference, `gen_ai.tool.definitions` in current OTel GenAI, and raw
+  logs.
+- Per-line digests kept in memory for the new detectors; `ingest` output
+  leaves them out.
+
+Also:
+
+- `diagnose` and `check` report a cross-session section when given more
+  than one session.
+- Demo: a `render.yaml` blueprint for one-click deploys, and raw provider
+  logs are accepted as uploads.
+- The capture runner records Anthropic runs (`--provider anthropic`);
+  `experiments/tradingagents-claude/` is ready to run with an API key.
+- The corpus has 30 fixtures (was 16).
+
+## 0.2.0 — 2026-11-10
+
+- Fix the GitHub Action for external users. It installed the latest PyPI
+  release, and 0.1.0 has no `check` command. It now installs the code at the
+  ref it is used from, so `@v0.2.0` runs 0.2.0. The `version` input still
+  pins a PyPI release.
+- Rulebook re-verified against current provider docs (version 2026.11.10):
+  - Anthropic: Opus, Sonnet and Haiku 5.5, Fable and Mythos 5.1/5.2.
+    Per-model read multipliers (0.05x and 0.025x) and base input prices.
+  - OpenAI: GPT-5.6 sizes, GPT-6 and GPT-6.1, 12h TTL. Pre-5.6 models no
+    longer publish a fixed minimum, so it is recorded as unknown.
+  - Google: added Gemini 4 Argon and 3.8 Flash; removed 3.7 and 3.5 Flash,
+    which are no longer in Google's table.
+- `sub_minimum` abstains when a model has no published minimum.
+- Release workflow: pushing a `vX.Y.Z` tag publishes to PyPI.
+- The sdist no longer includes the experiment traces.
+- **`ingest_text(text, source)`**: ingest trace content already in memory
+  (an upload, a stream) — nothing touches disk. `ingest_path` now wraps it.
+- **Hosted demo** (`demo/`, not part of the package): a one-page FastAPI
+  app — drop a trace, get the report; two prerendered samples (the
+  synthetic prefix-drift fixture and the real captured TradingAgents
+  runs). Uploads are parsed in memory and never stored or logged.
+- **CI cost-regression gate**: `tokenaudit check <traces> --baseline
+  baseline.json` fails a build (exit 1) on a new HIGH finding or a drop in
+  cache read rate / health floor beyond `--max-regression` points; create
+  or refresh the baseline with `--update`. Ships as a composite GitHub
+  Action (`action.yml`) with a markdown summary written to the job page
+  (and to a file via `--markdown`, for PR comments). See `docs/ci.md`.
+
+## 0.1.0 — 2026-08-30
+
+First usable release.
+
+- **Ingest** four real trace dialects into one normalized accounting model:
+  OTel GenAI current semconv, legacy flat names (older openllmetry),
+  OpenInference, and the OTel Python SDK's `to_json()` JSONL — with
+  layered-span deduplication (framework + SDK instrumentors double-report
+  the same call), a data-quality gate that refuses to price malformed
+  telemetry, and prompt privacy by construction (block hashes and lengths,
+  never text).
+- **Nine deterministic detectors** — prefix drift, sub-minimum cache
+  blocks, model switch, TTL expiry, breakpoint limit, lookback overflow,
+  tool-schema churn, context bloat, retry storm — each with evidence
+  pointers, bounded impact ranges with stated assumptions, and abstention
+  (with a reason) as a first-class outcome distinct from "checked, clean".
+- **Provider caching rulebook** packaged as sourced, dated JSON (Anthropic,
+  OpenAI, Google), with a CI staleness alarm and a rendered comparison
+  table (`tokenaudit rulebook`).
+- **Evaluation** against a byte-reproducible labeled corpus, published in
+  `eval/results.md`; per-detector corpus records ship inside the package
+  and appear on findings.
+- **CLI**: `tokenaudit diagnose` (text, `--json`, self-contained `--html`
+  report, `--sample`), `tokenaudit ingest`, `tokenaudit rulebook`.
+- Zero runtime dependencies. Python 3.11+.

{tokenaudit-0.2.0 → tokenaudit-0.5.0}/DECISIONS.md

@@ -39,3 +39,26 @@ Three calls made together for the 0.1.0 release, all in service of the same prin
 **The cache health score is `realized / achievable` savings** — realized cache savings over realized-plus-what-the-findings-quantify. Both halves are computed the same way, so there are no weights to defend; overlapping findings make the denominator conservative, so the score is presented as a floor and says so.
 
 **Findings carry a corpus record, not a confidence number.** A precision float computed from a 16-fixture corpus would be theater. The package ships the raw record instead — "3/4 planted cases found, 0 false alarms in N negatives" — generated by the eval harness and drift-tested, so what appears next to a finding is exactly what was measured.
+
+## 007 — Line-level digests, and detectors that look across sessions (2026-11-12)
+
+**Decision:** ingest keeps a truncated SHA-256 and length for every line of every message, in memory only, next to the existing whole-message digest. Two detectors use them: `misordered_shared_context` within a session and `cross_session_drift` across sessions.
+
+**Why:** the TradingAgents diagnosis was done by hand, because whole-message hashes can't see the same reports inside two different messages. The tool called those runs clean while the case study showed a 4x improvement. Lines are fine enough to find shared content and to pin a date to one line, and coarse enough to stay fast.
+
+**Boundaries:** line digests never leave the process (`tokenaudit ingest` drops them), because a hash of a short line is easy to guess. Cross-session findings count toward a CI gate's findings but not toward the health floor, since whether sessions share a cache depends on when they ran; the low impact bound only counts sessions that started within the model's cache lifetime of each other. Each new detector defers to an existing one where they overlap: a short edit near the start of a prompt is prefix drift, and a value that changes on every call within a session is prefix drift too.
+
+## 008 — Price against the fix, per model; new detectors yield to older ones (2026-11-12)
+
+**Decision:** a finding's impact is what the affected tokens cost minus what they would have cost under the fix, using the model's read and write multipliers from the rulebook. A request that wrote the cache is charged the write premium, not full price. Dollars appear only when the rulebook has a verified input price; tiered prices (Haiku 5.5) stay in tokens.
+
+**Why:** a flat discount overstated waste on models with cheaper reads and ignored that a miss on Anthropic often costs more than a plain request, because it writes the cache. Keeping the rates in one module means a rulebook update reprices every detector at once.
+
+**Boundaries:** `tool_schema_bloat` prints tool names, which are identifiers a developer chose rather than prompt text. It counts a tool as used if any request with the same tool set called it, so a shared registry where every tool is used by some agent is not flagged; telling agents apart needs more than a trace carries. `missing_cache_markers` only counts block-aligned prefixes repeated within the cache lifetime, since a breakpoint sits at a block's end and a later repeat would have missed anyway. It leaves growing conversations to `context_bloat` and byte-identical re-sends to `retry_storm`, whose fixes fit those cases better.
+
+## 009 — Report cache writes that buy nothing; size prefixes from real token counts (2026-11-12)
+
+**Decision:** `unread_cache_writes` reports writes that no later request in the session sends again, priced at the write premium, and calls a session HIGH when that premium exceeds what its reads saved. It stays quiet when another detector already explains why the cache wasn't read. Prefix sizes in `missing_cache_markers` come from the earlier request's reported input tokens, scaled by its share of the request's characters.
+
+**Why:** the Claude case study turned caching on for every TradingAgents request and paid 7.2% more, and 0.4.0 called those runs clean. Caching is only cheaper when something reads the write back, and "turn on caching" is the fix people reach for first, so the tool has to say when it backfires. Character counts were off by 2x on prompts made of numbers and JSON, which made the missing-markers estimate a fraction of what caching delivered.
+

{tokenaudit-0.2.0 → tokenaudit-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tokenaudit
-Version: 0.2.0
+Version: 0.5.0
 Summary: Root-cause diagnosis of wasted LLM tokens, from the traces your agents already emit
 Project-URL: Repository, https://github.com/yashmyeole/tokenaudit
 Project-URL: Issues, https://github.com/yashmyeole/tokenaudit/issues
@@ -29,7 +29,7 @@ Description-Content-Type: text/markdown
 
 **Root-cause diagnosis of wasted LLM tokens — from the traces your agents already emit.**
 
-> Status: alpha (0.2.0). Interfaces may change before 1.0.
+> Status: alpha (0.5.0). Interfaces may change before 1.0.
 
 Every LLM cost tool can tell you *how much* you spent and *who* spent it. Almost none can tell you *why a specific request wasted money*. The biggest lever on an LLM bill — prompt caching, up to a 10× discount on repeated input — fails silently: change one byte of a cached prefix, drop below a model's minimum cacheable size, let a timer expire, and the provider just bills you full price. No error, no warning. There are [documented](https://barndoor.ai/what-happens-to-your-llm-bill-when-prompt-caching-fails/) [cases](https://github.com/TauricResearch/TradingAgents/issues/750) of agents running at ~0% cache hit rate for months, at 30–40% unnecessary cost, because of a single line of prompt construction.
 
@@ -37,21 +37,26 @@ tokenaudit reads the traces agents already produce and, for each expensive reque
 
 It works: diagnosing one of those documented cases produced a 9-file patch that took TradingAgents' cache hit rate from **7.5% to 32.4% (4.3×)** in a controlled, pre-registered before/after experiment — the [first case study](docs/case-studies/tradingagents.md).
 
+On Claude it found the opposite problem: switching on caching for every request made the same agent's input bill 7.2% higher, because most of its prompts are sent once and every cache write costs 1.25×. The [second case study](docs/case-studies/tradingagents-claude.md) shows where the money went and what to cache instead.
+
 ## Quickstart
 
     pip install tokenaudit
     tokenaudit diagnose --sample          # diagnose the bundled demo trace
     tokenaudit diagnose mytraces/ --html report.html
 
-Zero runtime dependencies; reads OTLP/JSON files and OTel SDK JSONL. Try it
+Zero runtime dependencies; reads OTLP/JSON files, OTel SDK JSONL, and raw
+OpenAI or Anthropic request/response logs. Try it
 on this repo's real captured runs: `tokenaudit diagnose corpus/captured/runs`.
-Prefer a browser? The [demo app](demo/) diagnoses a dropped trace; uploads
-are parsed in memory and never stored.
+Prefer a browser? The [live demo](https://tokenaudit-demo.onrender.com)
+diagnoses a dropped trace; uploads are parsed in memory and never stored. It
+runs on a free instance that sleeps when idle, so the first load can take a
+minute.
 
 The plan, in one line each:
 
 - Ingest standard traces — OpenTelemetry GenAI in all three of its cache-field naming generations, OpenInference, raw provider SDK logs — into one normalized accounting model.
-- Run a library of deterministic detectors for the silent failure modes: prefix drift, sub-minimum cache blocks, model switches, TTL expiry, breakpoint and lookback limits, tool-schema churn and bloat, context bloat, retry storms.
+- Run a library of deterministic detectors for the silent failure modes: prefix drift, sub-minimum cache blocks, model switches, TTL expiry, breakpoint and lookback limits, tool-schema churn and bloat, context bloat, retry storms, shared content sent behind different openings, per-session values that stop a prompt being reused across sessions, prompts that never ask a cache-on-request provider to cache them, and cache writes nothing reads back.
 - Price each finding against a counterfactual — what the bill would have been under the fix — as a bounded range, never a point guess.
 - Publish per-detector precision and recall against a labeled corpus, so you know how much to trust each finding.
 
@@ -118,11 +123,13 @@ If you know a tool that already does this, please open an issue — I would genu
 - [x] `tokenaudit diagnose` CLI, HTML report, cache health score
 - [x] CI cost-regression mode: fail a build that tanks the cache hit rate ([docs/ci.md](docs/ci.md))
 - [x] Demo app: drag a trace in, get the report ([demo/](demo/))
-- [x] Case studies on public agents, with reproduction commands ([TradingAgents: 4.3× cache hit rate](docs/case-studies/tradingagents.md))
-- [ ] Detectors for misordered shared context and cross-run prefix drift
-- [ ] Counterfactual pricing with per-model read rates and dollar ranges
-- [ ] Raw provider SDK log ingest, tool-schema bloat detector
-- [ ] Public demo deployment, case study on an Anthropic-model agent
+- [x] Case studies on public agents, with reproduction commands ([TradingAgents: 4.3× cache hit rate](docs/case-studies/tradingagents.md), [TradingAgents on Claude: when caching raised the bill](docs/case-studies/tradingagents-claude.md))
+- [x] Detectors for misordered shared context and cross-session drift: they now find the TradingAgents problems on their own ([case study, section 6](docs/case-studies/tradingagents.md#6-what-tokenaudit-reports-now))
+- [x] Counterfactual pricing with per-model read and write rates and dollar ranges
+- [x] Raw provider SDK log ingest (OpenAI Responses, Chat Completions, Anthropic Messages), tool-schema bloat detector
+- [x] A detector for prompts that never ask Anthropic to cache them
+- [x] Public demo deployment: [tokenaudit-demo.onrender.com](https://tokenaudit-demo.onrender.com)
+- [x] Case study on an Anthropic-model agent ([TradingAgents on Claude](docs/case-studies/tradingagents-claude.md))
 
 ## Development
 

{tokenaudit-0.2.0 → tokenaudit-0.5.0}/README.md

@@ -2,7 +2,7 @@
 
 **Root-cause diagnosis of wasted LLM tokens — from the traces your agents already emit.**
 
-> Status: alpha (0.2.0). Interfaces may change before 1.0.
+> Status: alpha (0.5.0). Interfaces may change before 1.0.
 
 Every LLM cost tool can tell you *how much* you spent and *who* spent it. Almost none can tell you *why a specific request wasted money*. The biggest lever on an LLM bill — prompt caching, up to a 10× discount on repeated input — fails silently: change one byte of a cached prefix, drop below a model's minimum cacheable size, let a timer expire, and the provider just bills you full price. No error, no warning. There are [documented](https://barndoor.ai/what-happens-to-your-llm-bill-when-prompt-caching-fails/) [cases](https://github.com/TauricResearch/TradingAgents/issues/750) of agents running at ~0% cache hit rate for months, at 30–40% unnecessary cost, because of a single line of prompt construction.
 
@@ -10,21 +10,26 @@ tokenaudit reads the traces agents already produce and, for each expensive reque
 
 It works: diagnosing one of those documented cases produced a 9-file patch that took TradingAgents' cache hit rate from **7.5% to 32.4% (4.3×)** in a controlled, pre-registered before/after experiment — the [first case study](docs/case-studies/tradingagents.md).
 
+On Claude it found the opposite problem: switching on caching for every request made the same agent's input bill 7.2% higher, because most of its prompts are sent once and every cache write costs 1.25×. The [second case study](docs/case-studies/tradingagents-claude.md) shows where the money went and what to cache instead.
+
 ## Quickstart
 
     pip install tokenaudit
     tokenaudit diagnose --sample          # diagnose the bundled demo trace
     tokenaudit diagnose mytraces/ --html report.html
 
-Zero runtime dependencies; reads OTLP/JSON files and OTel SDK JSONL. Try it
+Zero runtime dependencies; reads OTLP/JSON files, OTel SDK JSONL, and raw
+OpenAI or Anthropic request/response logs. Try it
 on this repo's real captured runs: `tokenaudit diagnose corpus/captured/runs`.
-Prefer a browser? The [demo app](demo/) diagnoses a dropped trace; uploads
-are parsed in memory and never stored.
+Prefer a browser? The [live demo](https://tokenaudit-demo.onrender.com)
+diagnoses a dropped trace; uploads are parsed in memory and never stored. It
+runs on a free instance that sleeps when idle, so the first load can take a
+minute.
 
 The plan, in one line each:
 
 - Ingest standard traces — OpenTelemetry GenAI in all three of its cache-field naming generations, OpenInference, raw provider SDK logs — into one normalized accounting model.
-- Run a library of deterministic detectors for the silent failure modes: prefix drift, sub-minimum cache blocks, model switches, TTL expiry, breakpoint and lookback limits, tool-schema churn and bloat, context bloat, retry storms.
+- Run a library of deterministic detectors for the silent failure modes: prefix drift, sub-minimum cache blocks, model switches, TTL expiry, breakpoint and lookback limits, tool-schema churn and bloat, context bloat, retry storms, shared content sent behind different openings, per-session values that stop a prompt being reused across sessions, prompts that never ask a cache-on-request provider to cache them, and cache writes nothing reads back.
 - Price each finding against a counterfactual — what the bill would have been under the fix — as a bounded range, never a point guess.
 - Publish per-detector precision and recall against a labeled corpus, so you know how much to trust each finding.
 
@@ -91,11 +96,13 @@ If you know a tool that already does this, please open an issue — I would genu
 - [x] `tokenaudit diagnose` CLI, HTML report, cache health score
 - [x] CI cost-regression mode: fail a build that tanks the cache hit rate ([docs/ci.md](docs/ci.md))
 - [x] Demo app: drag a trace in, get the report ([demo/](demo/))
-- [x] Case studies on public agents, with reproduction commands ([TradingAgents: 4.3× cache hit rate](docs/case-studies/tradingagents.md))
-- [ ] Detectors for misordered shared context and cross-run prefix drift
-- [ ] Counterfactual pricing with per-model read rates and dollar ranges
-- [ ] Raw provider SDK log ingest, tool-schema bloat detector
-- [ ] Public demo deployment, case study on an Anthropic-model agent
+- [x] Case studies on public agents, with reproduction commands ([TradingAgents: 4.3× cache hit rate](docs/case-studies/tradingagents.md), [TradingAgents on Claude: when caching raised the bill](docs/case-studies/tradingagents-claude.md))
+- [x] Detectors for misordered shared context and cross-session drift: they now find the TradingAgents problems on their own ([case study, section 6](docs/case-studies/tradingagents.md#6-what-tokenaudit-reports-now))
+- [x] Counterfactual pricing with per-model read and write rates and dollar ranges
+- [x] Raw provider SDK log ingest (OpenAI Responses, Chat Completions, Anthropic Messages), tool-schema bloat detector
+- [x] A detector for prompts that never ask Anthropic to cache them
+- [x] Public demo deployment: [tokenaudit-demo.onrender.com](https://tokenaudit-demo.onrender.com)
+- [x] Case study on an Anthropic-model agent ([TradingAgents on Claude](docs/case-studies/tradingagents-claude.md))
 
 ## Development
 

{tokenaudit-0.2.0 → tokenaudit-0.5.0}/corpus/captured/runner.py

@@ -1,10 +1,11 @@
 """Capture one real TradingAgents run as OpenTelemetry spans in a JSONL file.
 
 Zero-modification capture: TradingAgents' code is untouched. Instrumentation
-wraps the OpenAI SDK and LangChain at process level — the same zero-integration
-posture tokenaudit itself takes. Spans are written line-by-line using the OTel
-Python SDK's own ``ReadableSpan.to_json()`` rendering, with no custom mapping,
-so the captured dialect is exactly what the SDK produces in the wild.
+wraps the provider SDK (OpenAI or Anthropic) and LangChain at process level,
+the same zero-integration posture tokenaudit itself takes. Spans are written
+line-by-line using the OTel Python SDK's own ``ReadableSpan.to_json()``
+rendering, with no custom mapping, so the captured dialect is exactly what
+the SDK produces in the wild.
 
 This script is executed by ``capture_tradingagents.sh`` inside a throwaway
 virtualenv on the machine that holds the API key. It is not part of the
@@ -66,16 +67,22 @@ def build_provider(trace_file: Path, run_id: str, ta_commit: str) -> TracerProvi
     return provider
 
 
-def instrument(provider: TracerProvider) -> list[str]:
+def instrument(provider: TracerProvider, llm_provider: str = "openai") -> list[str]:
     """Attach OpenInference instrumentors; return the ones that activated."""
     active: list[str] = []
     try:
-        from openinference.instrumentation.openai import OpenAIInstrumentor
+        if llm_provider == "anthropic":
+            from openinference.instrumentation.anthropic import AnthropicInstrumentor
 
-        OpenAIInstrumentor().instrument(tracer_provider=provider)
-        active.append("openinference-instrumentation-openai")
+            AnthropicInstrumentor().instrument(tracer_provider=provider)
+            active.append("openinference-instrumentation-anthropic")
+        else:
+            from openinference.instrumentation.openai import OpenAIInstrumentor
+
+            OpenAIInstrumentor().instrument(tracer_provider=provider)
+            active.append("openinference-instrumentation-openai")
     except Exception as exc:
-        print(f"warning: OpenAI instrumentor not active: {exc}", file=sys.stderr)
+        print(f"warning: {llm_provider} SDK instrumentor not active: {exc}", file=sys.stderr)
     try:
         from openinference.instrumentation.langchain import LangChainInstrumentor
 
@@ -88,12 +95,14 @@ def instrument(provider: TracerProvider) -> list[str]:
     return active
 
 
-def run_analysis(ticker: str, date: str, deep: str, quick: str, rounds: int) -> str:
+def run_analysis(
+    ticker: str, date: str, deep: str, quick: str, rounds: int, llm_provider: str = "openai"
+) -> str:
     from tradingagents.default_config import DEFAULT_CONFIG
     from tradingagents.graph.trading_graph import TradingAgentsGraph
 
     config = dict(DEFAULT_CONFIG)
-    config["llm_provider"] = "openai"
+    config["llm_provider"] = llm_provider
     config["deep_think_llm"] = deep
     config["quick_think_llm"] = quick
     config["max_debate_rounds"] = rounds
@@ -119,6 +128,7 @@ def main() -> None:
     parser.add_argument("--deep", default="gpt-5.4-mini")
     parser.add_argument("--quick", default="gpt-5.4-mini")
     parser.add_argument("--rounds", type=int, default=1)
+    parser.add_argument("--provider", choices=["openai", "anthropic"], default="openai")
     parser.add_argument("--ta-commit", default="unknown")
     parser.add_argument("--smoke", action="store_true", help="plumbing test, no API calls")
     args = parser.parse_args()
@@ -134,6 +144,7 @@ def main() -> None:
         "smoke": args.smoke,
         "ticker": args.ticker,
         "trade_date": args.date,
+        "llm_provider": args.provider,
         "models": {"deep_think": args.deep, "quick_think": args.quick},
         "max_debate_rounds": args.rounds,
         "tradingagents_commit": args.ta_commit,
@@ -146,8 +157,11 @@ def main() -> None:
                 "tradingagents",
                 "opentelemetry-sdk",
                 "openinference-instrumentation-openai",
+                "openinference-instrumentation-anthropic",
                 "openinference-instrumentation-langchain",
                 "openai",
+                "anthropic",
+                "langchain-anthropic",
                 "langchain-core",
             )
         },
@@ -164,8 +178,10 @@ def main() -> None:
         manifest["instrumentors"] = []
         decision = "smoke"
     else:
-        manifest["instrumentors"] = instrument(provider)
-        decision = run_analysis(args.ticker, args.date, args.deep, args.quick, args.rounds)
+        manifest["instrumentors"] = instrument(provider, args.provider)
+        decision = run_analysis(
+            args.ticker, args.date, args.deep, args.quick, args.rounds, args.provider
+        )
         (args.out / "decision.txt").write_text(decision + "\n", encoding="utf-8")
 
     provider.force_flush()

{tokenaudit-0.2.0 → tokenaudit-0.5.0}/corpus/synthetic/fixtures/breakpoint_limit.label.json

@@ -11,13 +11,18 @@
   ],
   "must_not_contain": [
     "context_bloat",
+    "cross_session_drift",
     "lookback_overflow",
+    "misordered_shared_context",
+    "missing_cache_markers",
     "model_switch",
     "prefix_drift",
     "retry_storm",
     "sub_minimum",
+    "tool_schema_bloat",
     "tool_schema_churn",
-    "ttl_expiry"
+    "ttl_expiry",
+    "unread_cache_writes"
   ],
   "evidence": {
     "marked_blocks": 6,

tokenaudit-0.5.0/corpus/synthetic/fixtures/cache_writes_reused.label.json

@@ -0,0 +1,38 @@
+{
+  "fixture": "cache_writes_reused",
+  "schema": "otel-genai-current",
+  "provider_shape": "anthropic",
+  "models": [
+    "claude-sonnet-4-5"
+  ],
+  "content_captured": true,
+  "contains": [],
+  "must_not_contain": [
+    "breakpoint_limit",
+    "context_bloat",
+    "cross_session_drift",
+    "lookback_overflow",
+    "misordered_shared_context",
+    "missing_cache_markers",
+    "model_switch",
+    "prefix_drift",
+    "retry_storm",
+    "sub_minimum",
+    "tool_schema_bloat",
+    "tool_schema_churn",
+    "ttl_expiry",
+    "unread_cache_writes"
+  ],
+  "evidence": {
+    "calls": 6,
+    "written_never_read": 0
+  },
+  "notes": "Three agents, each called twice, with only the stable system prompt marked. Near-miss: every write is read back by the agent's next call, so a correct tool reports nothing here.",
+  "totals": {
+    "calls": 6,
+    "input_tokens": 12114,
+    "cache_read_tokens": 6021,
+    "cache_creation_tokens": 6021,
+    "output_tokens": 1200
+  }
+}