@ictechgy/context-guard 0.4.9 → 0.4.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/README.ko.md +59 -31
- package/README.md +85 -36
- package/docs/benchmark-fixtures/token-savings-12task-baseline.prompt.example.md +7 -0
- package/docs/benchmark-fixtures/token-savings-12task-contextguard.prompt.example.md +7 -0
- package/docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl +24 -0
- package/docs/benchmark-fixtures/token-savings-12task.tasks.example.json +182 -0
- package/docs/benchmark-fixtures/token-savings-12task.variants.example.json +10 -0
- package/docs/benchmark-workflow-examples.md +3 -0
- package/docs/benchmark-workflows/context-pack-byte-proxy.example.json +278 -137
- package/docs/benchmark-workflows/measured-token-workflow.example.json +279 -138
- package/docs/benchmark-workflows/provider-cache-telemetry.example.json +279 -138
- package/docs/distribution.md +10 -7
- package/docs/experimental-benchmark-fixtures.md +30 -6
- package/package.json +4 -6
- package/packaging/homebrew/context-guard.rb.template +1 -1
- package/plugins/context-guard/.claude-plugin/plugin.json +1 -1
- package/plugins/context-guard/README.ko.md +20 -14
- package/plugins/context-guard/README.md +26 -17
- package/plugins/context-guard/bin/context-guard +147 -25
- package/plugins/context-guard/bin/context-guard-artifact +884 -79
- package/plugins/context-guard/bin/context-guard-audit +33 -2
- package/plugins/context-guard/bin/context-guard-bench +1542 -31
- package/plugins/context-guard/bin/context-guard-cache-score +665 -0
- package/plugins/context-guard/bin/context-guard-compress +146 -1
- package/plugins/context-guard/bin/context-guard-cost +790 -6
- package/plugins/context-guard/bin/context-guard-experiments +463 -26
- package/plugins/context-guard/bin/context-guard-failed-nudge +9 -2
- package/plugins/context-guard/bin/context-guard-filter +163 -7
- package/plugins/context-guard/bin/context-guard-guard-read +3 -0
- package/plugins/context-guard/bin/context-guard-pack +892 -49
- package/plugins/context-guard/bin/context-guard-rewrite-bash +3 -0
- package/plugins/context-guard/bin/context-guard-sanitize-output +76 -12
- package/plugins/context-guard/bin/context-guard-setup +165 -31
- package/plugins/context-guard/bin/context-guard-statusline +490 -283
- package/plugins/context-guard/bin/context-guard-statusline-merged +5 -0
- package/plugins/context-guard/bin/context-guard-tool-prune +480 -53
- package/plugins/context-guard/bin/context-guard-trim-output +288 -41
- package/plugins/context-guard/brief/README.md +5 -5
- package/plugins/context-guard/lib/context_guard_commands.py +230 -0
- package/plugins/context-guard/skills/setup/SKILL.md +1 -0
- package/context-guard-kit/README.md +0 -91
- package/context-guard-kit/benchmark_runner.py +0 -2401
- package/context-guard-kit/claude_transcript_cost_audit.py +0 -2346
- package/context-guard-kit/context_compress.py +0 -695
- package/context-guard-kit/context_escrow.py +0 -935
- package/context-guard-kit/context_filter.py +0 -637
- package/context-guard-kit/context_guard_cli.py +0 -325
- package/context-guard-kit/context_guard_diet.py +0 -1711
- package/context-guard-kit/context_pack.py +0 -2713
- package/context-guard-kit/cost_guard.py +0 -2349
- package/context-guard-kit/experimental_registry.py +0 -4348
- package/context-guard-kit/failed_attempt_nudge.py +0 -567
- package/context-guard-kit/guard_large_read.py +0 -690
- package/context-guard-kit/hook_secret_patterns.py +0 -43
- package/context-guard-kit/read_symbol.py +0 -483
- package/context-guard-kit/rewrite_bash_for_token_budget.py +0 -501
- package/context-guard-kit/sanitize_output.py +0 -725
- package/context-guard-kit/settings.example.json +0 -67
- package/context-guard-kit/setup_wizard.py +0 -2515
- package/context-guard-kit/statusline.sh +0 -362
- package/context-guard-kit/statusline_merged.sh +0 -157
- package/context-guard-kit/tool_schema_pruner.py +0 -837
- package/context-guard-kit/trim_command_output.py +0 -1449
package/README.md
CHANGED
|
@@ -1,20 +1,22 @@
|
|
|
1
1
|
# ContextGuard
|
|
2
2
|
|
|
3
|
-
ContextGuard is a local-first context management toolkit for AI coding and tool agents. It
|
|
3
|
+
ContextGuard is a local-first context management toolkit for AI coding and tool-using agents. It ships as a Claude Code plugin first: install it once, enable it per project, and roll it back when needed.
|
|
4
|
+
|
|
5
|
+
It helps trim noisy output, steer agents toward symbol-level reads, nudge repeated failures, redact secret-like patterns, and measure usage. The same guardrails extend to other agents through local helper commands and advisory brief-mode rule snippets.
|
|
4
6
|
|
|
5
7
|
- Korean documentation: [`README.ko.md`](README.ko.md)
|
|
6
8
|
- Static landing page: [GitHub Pages](https://ictechgy.github.io/context-guard/) ([source](docs/index.html))
|
|
7
9
|
|
|
8
10
|
## TL;DR
|
|
9
11
|
|
|
10
|
-
|
|
12
|
+
Installation and activation are deliberately separate. Installing ContextGuard only makes local helpers or Claude plugin skills available; configuration changes happen only when you run an explicit setup command.
|
|
11
13
|
|
|
12
14
|
| If you use... | Install | Activate |
|
|
13
15
|
| --- | --- | --- |
|
|
14
16
|
| Claude Code | `/plugin marketplace add ictechgy/context-guard` then `/plugin install context-guard@context-guard` | Run `/context-guard:setup` inside the project. |
|
|
15
17
|
| Codex CLI or any terminal-first agent | `npm install -g @ictechgy/context-guard` or one-shot `npx @ictechgy/context-guard ...` | `context-guard setup --agent codex --scope project --with-init --with-skill --plan`, then rerun with `--yes`. |
|
|
16
|
-
| Other rule-file agents | npm/npx install above | `context-guard setup --agent gemini,cursor,windsurf,cline,copilot --scope project --with-init --plan`, then apply only the agents you want. |
|
|
17
|
-
| macOS/Homebrew users |
|
|
18
|
+
| Other rule-file agents | Use the npm/npx install path above. | `context-guard setup --agent gemini,cursor,windsurf,cline,copilot --scope project --with-init --plan`, then apply only the agents you want. |
|
|
19
|
+
| macOS/Homebrew users | Release path: `brew install ictechgy/tap/context-guard` | Same `context-guard setup ...` commands after install. |
|
|
18
20
|
|
|
19
21
|
Common commands:
|
|
20
22
|
|
|
@@ -27,16 +29,18 @@ context-guard setup --agent claude --scope user --verify --json # read-only use
|
|
|
27
29
|
context-guard setup --agent claude --scope user --plan
|
|
28
30
|
```
|
|
29
31
|
|
|
30
|
-
Project scope is the default. User-level setup is opt-in, requires an explicit agent for writes, records backups and rollback metadata, and never runs during package installation. Use `context-guard doctor` or `context-guard setup --verify` for a read-only health check before applying setup
|
|
32
|
+
Project scope is the default. User-level setup is opt-in, requires an explicit agent for writes, records backups and rollback metadata, and never runs during package installation. Use `context-guard doctor` or `context-guard setup --verify` for a read-only health check before applying setup. `doctor` reports next commands and makes no changes. Setup looks for bundled or checkout-local helpers first; it does not trust arbitrary `PATH` helpers unless you explicitly pass `--allow-path-helper-fallback` for a known-good install.
|
|
33
|
+
|
|
34
|
+
Distribution and helper trust boundaries are conservative too: npm exposes only canonical `context-guard`/`context-guard-*` bin links, legacy `claude-*` wrappers remain package files for path-based migration, command manifests are treated as literal data instead of executable Python, and the macOS visibility helper is discovered from bundled/resource/executable-relative paths or an absolute explicit override with a minimal child environment. Current working directories, relative overrides, symlinked helpers, arbitrary `PATH`, and ambient shell environment are not trusted by default.
|
|
31
35
|
|
|
32
|
-
ContextGuard is intentionally conservative about savings claims. It reduces common sources of context bloat and provides benchmark tooling so you can measure before
|
|
36
|
+
ContextGuard is intentionally conservative about savings claims. It reduces common sources of context bloat and provides benchmark tooling so you can measure before-and-after results on your own tasks. It does **not** promise a fixed token or cost reduction for every repository.
|
|
33
37
|
|
|
34
38
|
## Claude Code first, other agents too
|
|
35
39
|
|
|
36
|
-
ContextGuard ships
|
|
40
|
+
ContextGuard ships as a Claude Code plugin first, which is still the fastest path to value for Claude users. After installation, the same local-first guardrails can be reused by other AI coding and tool-using agents through:
|
|
37
41
|
|
|
38
42
|
- **Local helper commands** (`context-guard-*`) that run as plain shell commands, independent of any specific agent.
|
|
39
|
-
- **Advisory brief-mode rule snippets** you install into an agent's own instruction file (`AGENTS.md`, `GEMINI.md`, `.cursorrules`, Copilot instructions, and similar rule files) and remove by deleting the marker-delimited block.
|
|
43
|
+
- **Advisory brief-mode rule snippets** that you install into an agent's own instruction file (`AGENTS.md`, `GEMINI.md`, `.cursorrules`, Copilot instructions, and similar rule files) and remove by deleting the marker-delimited block.
|
|
40
44
|
- **Dry-run cross-agent setup** that writes only local files, backs up before changing anything, and applies only with explicit approval.
|
|
41
45
|
|
|
42
46
|
Current setup surfaces:
|
|
@@ -54,14 +58,14 @@ Current setup surfaces:
|
|
|
54
58
|
|
|
55
59
|
## How ContextGuard reduces token waste
|
|
56
60
|
|
|
57
|
-
ContextGuard does not
|
|
61
|
+
ContextGuard does not lower model prices by itself. It reduces avoidable context before it reaches an AI coding agent, then gives you signals to measure whether the change helped.
|
|
58
62
|
|
|
59
63
|
| Waste path | ContextGuard guardrail |
|
|
60
64
|
| --- | --- |
|
|
61
65
|
| Whole-file reads for one function | Suggest search, symbol slices, bounded outlines, and small line ranges before a full read. |
|
|
62
66
|
| Long test, build, search, or diff output | Trim output, emit structured digests, or store large logs locally and return compact receipts. |
|
|
63
67
|
| Repeated failing commands | Warn after repeated Bash failures so the agent changes strategy before more stale logs enter context. |
|
|
64
|
-
| Secret-like or noisy terminal output | Apply best-effort
|
|
68
|
+
| Secret-like or noisy terminal output | Apply best-effort pattern-based redaction for common credential patterns and sensitive-looking paths before output is copied into context. |
|
|
65
69
|
| Unknown token/cost hotspots | Surface statusline signals, transcript audits, and matched benchmark reports for before/after evidence. |
|
|
66
70
|
| Anthropic API requests that may miss prompt cache | `context-guard cost preflight` estimates input size, breakpoint-level cache risk, and low/mid/high cost ranges before a call; default mode warns only. |
|
|
67
71
|
| Volatile context before stable prompt prefixes | Audit bounded redacted prompt-segment hashes and flag likely cache-unfriendly prompt layouts without exposing raw prompt text. |
|
|
@@ -73,10 +77,10 @@ ContextGuard complements provider and semantic caches, and sits next to prompt c
|
|
|
73
77
|
|
|
74
78
|
| Tool category | Saves by | ContextGuard relationship |
|
|
75
79
|
| --- | --- | --- |
|
|
76
|
-
| Provider prompt/context caching | Reusing stable prompt prefixes. | Complementary; ContextGuard helps keep the changing tail of context smaller and cleaner, `context-guard-audit` can flag likely volatile prefix layouts, and `context-guard cost` can warn when an Anthropic request is likely to
|
|
80
|
+
| Provider prompt/context caching | Reusing stable prompt prefixes. | Complementary; ContextGuard helps keep the changing tail of context smaller and cleaner, `context-guard-audit` can flag likely volatile prefix layouts, and `context-guard cost` can warn when an Anthropic request is likely to cache-write instead of cache-read. |
|
|
77
81
|
| Semantic response cache | Reusing answers to identical or similar requests. | Complementary; ContextGuard does not serve cached AI answers. |
|
|
78
82
|
| Prompt/context compression | Shortening text that is already selected for the model. | Adjacent; ContextGuard trims and summarizes local output, but does not promise lossless semantic compression. |
|
|
79
|
-
| Experimental planners and local runtimes |
|
|
83
|
+
| Experimental planners and local runtimes | Default-off and explicit-command-only; covers local-proxy plans and gate records plus narrow local runtimes for caller-supplied context-diff, visual evidence-pack, learned-compression, and self-hosted metrics evidence. | The local proxy `record` command starts no listener and forwards no traffic; `serve local-proxy` binds and forwards only literal loopback IPs for one bounded request; `--response-sandbox` can replace a safe UTF-8 upstream body with a compact local artifact rehydration envelope. Compressor/model execution, OCR/crop services, external forwarding, credential persistence, and hosted-savings claims stay out of scope until a separate evidence gate and future PR allow them. |
|
|
80
84
|
| ContextGuard | Avoiding unnecessary files, logs, repeated failures, and noisy output before they enter agent context. | Local guardrails, reversible artifacts, and measurement. |
|
|
81
85
|
|
|
82
86
|
Related patterns that informed the design:
|
|
@@ -95,13 +99,14 @@ Three deterministic levels ship under [`plugins/context-guard/brief/`](plugins/c
|
|
|
95
99
|
|
|
96
100
|
## What to measure
|
|
97
101
|
|
|
98
|
-
|
|
102
|
+
If you need a savings claim, measure it on your own tasks:
|
|
99
103
|
|
|
100
104
|
- full-file reads versus symbol or line-range reads
|
|
101
105
|
- raw logs versus digest output or artifact receipts
|
|
102
106
|
- transcript hotspots reported by `context-guard-audit`, including `cache_friendliness` prompt-layout signals and `cache_layout_advice` experiment priorities
|
|
103
107
|
- statusline `cache` / `reuse` as observed transcript/provider-cache signals, not savings caused by ContextGuard
|
|
104
108
|
- `context-guard cost preflight` estimates for Anthropic request JSON, followed by `context-guard cost observe` using provider usage fields (`cache_creation_input_tokens`, `cache_read_input_tokens`) after the call
|
|
109
|
+
- static prompt/request cache layout checks from `context-guard-cache-score`, including optional user-supplied cache write/read multiplier amortization risk; its char/4 token estimates and warnings are advisory only until provider usage fields confirm real cache hits
|
|
105
110
|
- matched successful baseline/variant runs from `context-guard-bench`
|
|
106
111
|
- large tool/MCP catalogs versus `context-guard-tool-prune` top-k reports plus receipt retrieval
|
|
107
112
|
- optional experimental lanes in [`research/experimental-token-reduction-radar.md`](research/experimental-token-reduction-radar.md); fixture-only starters in [`docs/experimental-benchmark-fixtures.md`](docs/experimental-benchmark-fixtures.md) use the same matched-task benchmark gates before any savings claim
|
|
@@ -111,13 +116,14 @@ When you need a savings claim, measure it on your own tasks:
|
|
|
111
116
|
- It does not guarantee a fixed token or cost reduction.
|
|
112
117
|
- It does not send work to external AI providers to save model tokens.
|
|
113
118
|
- It does not mutate global Claude settings during install.
|
|
119
|
+
- It does not execute command manifests as code or trust arbitrary `PATH`/current-working-directory helpers during setup or packaged smoke checks.
|
|
114
120
|
- It does not replace real before/after measurement when you need a savings claim.
|
|
115
|
-
- Local RAM/disk receipts can reduce what you send next, but they do **not** replace Anthropic's provider prompt cache or guarantee cache hits. Recheck Anthropic prompt-caching and pricing docs before release or billing claims: https://docs.anthropic.com/en/build-with-claude/prompt-caching and https://platform.claude.com/docs/en/about-claude/pricing.
|
|
116
|
-
- Experimental helpers are mostly dry-run checker/planner surfaces, including a design-only external-forwarding opt-in gate. Explicit local runtimes exist only for caller-supplied context-diff replacement payloads, caller-supplied visual crop/OCR evidence packs, caller-supplied learned-compression prose candidates, self-hosted metrics JSONL sidecar records, local-proxy runtime-gate JSONL records, and one-shot `serve local-proxy` loopback forwarding plus optional shifted-cost diagnostic JSONL rows for successful forwarded requests.
|
|
121
|
+
- Local RAM/disk receipts can help reduce what you send next, but they do **not** replace Anthropic's provider prompt cache or guarantee cache hits. Recheck Anthropic prompt-caching and pricing docs before release or billing claims: https://docs.anthropic.com/en/build-with-claude/prompt-caching and https://platform.claude.com/docs/en/about-claude/pricing.
|
|
122
|
+
- Experimental helpers are mostly dry-run checker/planner surfaces, including a design-only external-forwarding opt-in gate. Explicit local runtimes exist only for caller-supplied context-diff replacement payloads, caller-supplied visual crop/OCR evidence packs, caller-supplied learned-compression prose candidates, self-hosted metrics JSONL sidecar records, local-proxy runtime-gate JSONL records, and one-shot `serve local-proxy` loopback forwarding with a private ready-file nonce, optional `--response-sandbox` compact artifact envelopes for safe UTF-8 responses, plus optional shifted-cost diagnostic JSONL rows for successful forwarded requests.
|
|
117
123
|
- ContextGuard does not ship learned/synthetic compressor execution, embeddings, rerankers, model calls, generated replacement text, screenshot capture, image cropping, OCR execution, image parsing, external OCR/image services, self-hosted KV/latent inference optimization beyond explicit local metrics recording, or broader proxy forwarding beyond literal-loopback, one-request HTTP forwarding with credential material blocked.
|
|
118
124
|
- It does not alias the old `/claude-token-optimizer:*` Claude Code slash-command namespace. Use `/context-guard:*` after installing this plugin.
|
|
119
125
|
|
|
120
|
-
Legacy local CLI wrappers (`claude-token-*`, `claude-read-symbol`, `claude-trim-output`, and `claude-sanitize-output`) still ship
|
|
126
|
+
Legacy local CLI wrappers (`claude-token-*`, `claude-read-symbol`, `claude-trim-output`, and `claude-sanitize-output`) still ship as package files under `plugins/context-guard/bin/` so existing plugin-path automation can migrate gradually. npm global/`npx` bin links intentionally expose only the canonical `context-guard`/`context-guard-*` commands; call the legacy wrappers by package/plugin path if you still need them.
|
|
121
127
|
|
|
122
128
|
## Features
|
|
123
129
|
|
|
@@ -125,16 +131,16 @@ Legacy local CLI wrappers (`claude-token-*`, `claude-read-symbol`, `claude-trim-
|
|
|
125
131
|
| --- | --- |
|
|
126
132
|
| Claude Code plugin skills | Guided setup, optimization, and transcript usage audits. |
|
|
127
133
|
| Project-local setup wizard | Applies recommended `.claude/settings.json` options without touching global settings. |
|
|
128
|
-
| Context
|
|
134
|
+
| Context management scanner | Finds missing guardrails, noisy hooks, broad reads, large context files, secret-like files, excessive MCP servers, and expensive defaults. |
|
|
129
135
|
| Structural-waste doctor | Opt-in local diagnostics for duplicate rules, stale imports, unused skill candidates, oversized tool schemas, and repeated read/tool-call loops. |
|
|
130
136
|
| Large-read guard and symbol reader | Nudges the agent toward `rg`, symbol reads, and small line ranges instead of full-file reads. |
|
|
131
137
|
| Output trimming and sanitizing | Keeps test, build, search, and diff output compact while redacting likely secrets before they enter agent context. |
|
|
132
138
|
| Declarative output filter | Opt-in JSON DSL for user-owned command filters with protected failure passthrough and validation before use. |
|
|
133
139
|
| Local artifact store | Saves large sanitized logs outside the conversation and returns compact receipts or exact requested slices. |
|
|
134
|
-
| Anthropic cost guard | `context-guard cost preflight/observe/ledger/compile` estimates cache
|
|
135
|
-
| Budgeted context packer | Assembles prioritized local file evidence into a byte-budgeted Markdown pack, can suggest a build-compatible manifest from local signals,
|
|
140
|
+
| Anthropic cost guard | `context-guard cost preflight/observe/ledger/compile` estimates cache risk and cost ranges. `context-guard route-advisor` summarizes local total-cost and batchability route candidates, stores only keyed HMAC fingerprints where a ledger is used, and stays passive unless `--enforce` is explicit. |
|
|
141
|
+
| Budgeted context packer | Assembles prioritized local file evidence into a byte-budgeted Markdown pack, can suggest a build-compatible manifest from local signals, adds `--explain` for compact local selection reasons plus bounded repo-map metadata, and adds opt-in `--adaptive-k` / `--symbol-memory` advisory metadata. |
|
|
136
142
|
| Tool/MCP schema pruner | Emits bounded top-k tool/schema advisory reports from local catalogs with compact receipts and full sanitized payload retrieval. |
|
|
137
|
-
| Conservative stdin compressor | Shrinks selected JSON, diffs, logs, search output, code, and prose with observed byte evidence and estimated token proxies. |
|
|
143
|
+
| Conservative stdin compressor | Shrinks selected JSON, diffs, logs, search output, code, and prose with observed byte evidence and estimated token proxies; `--mode readable` adds an opt-in readable prose preview with exact fallback guidance. |
|
|
138
144
|
| Protected-zone policy receipts | Opt-in `context-guard-compress --protected-policy` and `context-guard cost compile` metadata mark code/diff/path/hash/JSON/literal zones as structural-only with exact retrieval guidance. |
|
|
139
145
|
| Repeated-failure nudge | Warns after repeated Bash failures so the agent changes strategy before stale logs fill the context. |
|
|
140
146
|
| Statusline, audit, and benchmarks | Shows context/cache/cost signals, finds usage and cache-friendliness hotspots, and records conservative before/after evidence. |
|
|
@@ -170,7 +176,7 @@ Setup is explicit, project-local, and reversible. The plugin does not configure
|
|
|
170
176
|
|
|
171
177
|
## Install with npm/npx
|
|
172
178
|
|
|
173
|
-
The npm package exposes a canonical `context-guard` command plus
|
|
179
|
+
The npm package exposes a canonical `context-guard` command plus `context-guard-*` helper commands. Package installation is passive: there is no `postinstall` setup hook and no config write until you run `context-guard setup` yourself. npm global/`npx` bin links intentionally expose only canonical `context-guard`/`context-guard-*` commands; legacy `claude-*` wrapper files remain packaged for explicit path-based migration but are not advertised as executable bin aliases. If setup cannot find bundled or checkout-local helpers, `PATH` fallback remains disabled by default; use `--allow-path-helper-fallback` only for trusted helper directories after `context-guard doctor` or `setup --verify` confirms the plan.
|
|
174
180
|
|
|
175
181
|
```bash
|
|
176
182
|
npm install -g @ictechgy/context-guard
|
|
@@ -214,7 +220,7 @@ context-guard setup --agent claude --scope user --verify --json
|
|
|
214
220
|
|
|
215
221
|
Both modes are read-only configuration checks. `doctor` reports recommended next commands, and `setup --verify` checks whether setup is complete without applying changes. With `--json`, the report is written to stdout.
|
|
216
222
|
|
|
217
|
-
### Scan context
|
|
223
|
+
### Scan context management
|
|
218
224
|
|
|
219
225
|
```bash
|
|
220
226
|
./plugins/context-guard/bin/context-guard-diet scan .
|
|
@@ -245,10 +251,12 @@ The optional Read guard uses a progressive path for oversized files: search firs
|
|
|
245
251
|
|
|
246
252
|
```bash
|
|
247
253
|
long-command 2>&1 | ./plugins/context-guard/bin/context-guard-artifact store --command "long-command" --json
|
|
254
|
+
./plugins/context-guard/bin/context-guard-artifact search "ERROR" --json
|
|
255
|
+
./plugins/context-guard/bin/context-guard-artifact receipt <artifact_id> --json
|
|
248
256
|
./plugins/context-guard/bin/context-guard-artifact get <artifact_id> --lines 1:80
|
|
249
257
|
```
|
|
250
258
|
|
|
251
|
-
Artifact mode is for capture and retrieval. It stores sanitized output under `.context-guard/artifacts` by default and can still read legacy `.claude-token-optimizer/artifacts` receipts from before the rebrand. JSON receipts include line-numbered top-error receipts, duplicate-line groups,
|
|
259
|
+
Artifact mode is for capture, sandbox search, and retrieval. It stores sanitized output under `.context-guard/artifacts` by default and can still read legacy `.claude-token-optimizer/artifacts` receipts from before the rebrand. JSON receipts include line-numbered top-error receipts, duplicate-line groups, sanitized bounded `suggested_queries`, and an `output_sandbox` envelope with a stable `contextguard-artifact:<id>` handle. Use `context-guard-artifact receipt <artifact_id> --json` to rehydrate metadata-only handles without returning content, then fetch the smallest useful exact slice instead of replaying the full log. `search` scans the local sanitized artifact sandbox by literal substring, returns capped match/context records, and includes `context-guard-artifact get ... --lines START:END` rehydration commands for omitted detail. For custom `--dir` values, raw private paths stay redacted by default; rerun with the same `--dir`, or pass `search --show-paths` when you explicitly want a directly executable local command. The search report is local-only and does not make hosted token/cost savings claims. When `--max-lines` accompanies a `--lines START:END` selector, it caps lines returned within that range; it does not expand the selector. Preserve the producer command's exit code yourself when using shell pipelines in release checks, or use `context-guard-trim-output -- ...` when exit-code preservation is the primary requirement.
|
|
252
260
|
|
|
253
261
|
### Build a budgeted context pack
|
|
254
262
|
|
|
@@ -259,11 +267,11 @@ Artifact mode is for capture and retrieval. It stores sanitized output under `.c
|
|
|
259
267
|
--diff HEAD \
|
|
260
268
|
--manifest-out suggested-pack.json \
|
|
261
269
|
--pack-out context-pack.md \
|
|
262
|
-
--budget-bytes 12000 --json --explain
|
|
270
|
+
--budget-bytes 12000 --json --explain --adaptive-k --symbol-memory
|
|
263
271
|
# Or run the two explicit steps:
|
|
264
272
|
./plugins/context-guard/bin/context-guard-pack suggest \
|
|
265
273
|
--root . --query "review failing tests" --diff HEAD \
|
|
266
|
-
--manifest-out suggested-pack.json --budget-bytes 12000 --json
|
|
274
|
+
--manifest-out suggested-pack.json --budget-bytes 12000 --json --adaptive-k --adaptive-k-policy recall
|
|
267
275
|
./plugins/context-guard/bin/context-guard-pack build \
|
|
268
276
|
--root . --manifest suggested-pack.json --budget-bytes 12000 --json
|
|
269
277
|
./plugins/context-guard/bin/context-guard-pack slice --root . --path README.md --lines 1:40 --json
|
|
@@ -276,6 +284,8 @@ A few boundaries are intentional:
|
|
|
276
284
|
- Add `--explain` for compact deterministic local selection/build reasons in JSON or text output.
|
|
277
285
|
- `--explain` may include bounded `repo_map` metadata: sampled byte/token-proxy tree entries, category-only secret-risk counts, signature-first file hints, explain-only graph ranks, and exact `slice`/symbol retrieval hints.
|
|
278
286
|
- Explain metadata does not change the manifest, pack body, receipt, or byte budget. It does not use network/model/embedding calls, and token values remain local `chars_div_4` proxies rather than provider-token or savings claims.
|
|
287
|
+
- Add `--adaptive-k` to `suggest` or `auto` for advisory-only shrink/expand top-k metadata derived from local score distribution, byte-budget fit, and clamped score-mass recall/precision proxies. Use `--adaptive-k-policy balanced|recall|precision` plus optional `--adaptive-k-min-recall-proxy` / `--adaptive-k-min-precision-proxy` gates to choose a local recommendation policy; gate failures are metadata-only (`pass|failed`). The adaptive block includes capped selected/omitted evidence and structured source-verification hints, never applies the recommendation automatically, and does not change the manifest, pack body, receipt, or byte budget.
|
|
288
|
+
- Add `--symbol-memory` to `auto` for repo-map-derived symbol/graph advisory metadata with exact `slice` / `read-symbol` verification hints. It is source-verification guidance only and does not change the manifest, pack body, receipt, or byte budget.
|
|
279
289
|
- `--manifest-out` writes a build-compatible manifest; `--pack-out` saves the rendered pack.
|
|
280
290
|
- `context-guard-pack suggest` is the lower-level additive local-only planning step. It ranks candidate files and line ranges from `--query`, `--diff`, repeated `--files`, and optional sanitized `--output` / `--test-output` files under `--root`, then writes a manifest that `build --manifest` can consume.
|
|
281
291
|
- `context-guard-pack build` assembles prioritized local file evidence into a Markdown body whose rendered UTF-8 bytes stay within `--budget-bytes`. JSON output records included, partial, duplicate, unsafe, missing, and budget-omitted sources.
|
|
@@ -290,10 +300,32 @@ The packer uses deterministic standard-library heuristics only: no network, mode
|
|
|
290
300
|
--catalog tools.json \
|
|
291
301
|
--query "review failing tests" \
|
|
292
302
|
--top 5 --budget-bytes 12000 --json
|
|
303
|
+
./plugins/context-guard/bin/context-guard-tool-prune defer-report \
|
|
304
|
+
--catalog tools.json \
|
|
305
|
+
--query "review failing tests" \
|
|
306
|
+
--core-top 3 --deferred-top 20 --json
|
|
293
307
|
./plugins/context-guard/bin/context-guard-tool-prune get <receipt_id> --tool read_file --json
|
|
294
308
|
```
|
|
295
309
|
|
|
296
|
-
`context-guard-tool-prune` ranks a local tool or MCP catalog with deterministic lexical heuristics and emits a bounded top-k advisory report. Inline selected schemas respect an observed UTF-8 byte budget, and omitted or budget-skipped schemas remain recoverable from a compact local receipt plus a separate sanitized payload under `.context-guard/tool-prune`. This is advisory only: it does not mutate MCP configuration, and token counts remain estimated proxies rather than measured provider savings.
|
|
310
|
+
`context-guard-tool-prune` ranks a local tool or MCP catalog with deterministic lexical heuristics and emits a bounded top-k advisory report. Inline selected schemas respect an observed UTF-8 byte budget, and omitted or budget-skipped schemas remain recoverable from a compact local receipt plus a separate sanitized payload under `.context-guard/tool-prune`. `defer-report` uses the same receipt path to split a catalog into core inline tools plus deferred tool stubs and namespace summaries, and reports gross deferred-schema plus net initial-report char/4 proxy accounting so you can see what moved out of the first prompt. This is advisory only: it does not mutate MCP configuration, does not configure native provider tool search, and token counts remain estimated proxies rather than measured provider savings.
|
|
311
|
+
|
|
312
|
+
### Score static prompt cacheability
|
|
313
|
+
|
|
314
|
+
```bash
|
|
315
|
+
./plugins/context-guard/bin/context-guard-cache-score --input prompt.json --provider openai --json
|
|
316
|
+
./plugins/context-guard/bin/context-guard cache-score --input prompt.txt --provider anthropic --json
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
`context-guard-cache-score` is a local static lint for prompt/request layout. It estimates total and cacheable-prefix size with a tokenizer-free char/4 proxy, warns about dynamic-looking values near the prefix, and records provider caveats for OpenAI, Anthropic, Gemini, or a generic threshold. Optional `--expected-reuses`, `--cache-write-multiplier`, and `--cache-read-multiplier` inputs add an advisory amortization-risk section using user-supplied economics only. It does not call providers, store raw prompts, estimate prices from bundled defaults, observe cache hits, or prove token/cost savings; verify real cache behavior with provider usage telemetry.
|
|
320
|
+
|
|
321
|
+
### Advise on total cost, batchability, and routing
|
|
322
|
+
|
|
323
|
+
```bash
|
|
324
|
+
./plugins/context-guard/bin/context-guard route-advisor --workload workload.json --json
|
|
325
|
+
./plugins/context-guard/bin/context-guard-cost route-advisor --feature batch_api=true --feature structured_outputs=true --json < workload.json
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
`context-guard route-advisor` is a local, passive advisor. It reads caller-supplied workload JSON, provider feature declarations, usage telemetry, and shifted external/local costs, then emits total-cost accounting, batchability blockers, and candidate routes such as batch API, prompt-cache prefix preservation, structured outputs, or cheaper-model evaluation. It does not start a queue, call providers, refresh pricing docs, or treat bundled provider feature knowledge as authoritative; unknown or caller-supplied features are marked recheck-required. Treat recommendations as candidates only. Hosted token or cost savings claims require matched successful tasks, non-inferior quality, and shifted-cost evidence.
|
|
297
329
|
|
|
298
330
|
### Compress selected local text conservatively
|
|
299
331
|
|
|
@@ -301,19 +333,22 @@ The packer uses deterministic standard-library heuristics only: no network, mode
|
|
|
301
333
|
git diff | ./plugins/context-guard/bin/context-guard-compress --json
|
|
302
334
|
pytest -q 2>&1 | ./plugins/context-guard/bin/context-guard-compress --type log
|
|
303
335
|
cat evidence.txt | ./plugins/context-guard/bin/context-guard-compress --json --protected-policy
|
|
336
|
+
cat sanitized-prose.txt | ./plugins/context-guard/bin/context-guard-compress --json --type prose --mode readable
|
|
304
337
|
```
|
|
305
338
|
|
|
306
339
|
`context-guard-compress` classifies sanitized stdin as JSON, diff, log, search output, code, or prose, then applies deterministic reductions such as JSON compaction, diff context folding, duplicate log/search line collapse, and whitespace normalization. It never claims observed model-token savings; byte counts are observed, token counts are labeled as estimates, and lossy receipts point you back to `context-guard-artifact store` for exact retrieval.
|
|
307
340
|
|
|
308
341
|
Add `--protected-policy` when the input may contain semantic-sensitive zones such as code fences, diffs, identifiers, numeric constants, hashes, paths, stack frames, quoted strings, or JSON keys. The flag does not change default compressor behavior; it adds `protected_zone_policy` and `transform_policy` metadata that denies semantic/paraphrase rewrites, allows only structural transforms plus artifact retrieval, and stores only class/count policy metadata rather than raw protected spans.
|
|
309
342
|
|
|
343
|
+
Add `--mode readable` only for sanitized prose previews. It uses a deterministic sentence window, blocks prompt-like or high-risk protected signals, stores no raw protected spans, and marks exact fallback retrieval as required before edits or claims. It does not run learned compressors, models, embeddings, or rerankers.
|
|
344
|
+
|
|
310
345
|
### Trim or summarize command output
|
|
311
346
|
|
|
312
347
|
```bash
|
|
313
348
|
./plugins/context-guard/bin/context-guard-trim-output --max-lines 120 -- npm test
|
|
314
349
|
```
|
|
315
350
|
|
|
316
|
-
Use `--digest markdown` or `--digest json` for a compact semantic digest instead of head/tail logs. Digest mode keeps status, exit code, truncation counts, runner failure facts, a sanitized failure signature, duplicate-line groups, representative lines, redaction counts, and suggested next queries while preserving the wrapped command exit code. Add `--artifact-receipt` with digest mode when you want the exact sanitized full output stored locally as a `context-guard-artifact` receipt; re-expand with the emitted `context-guard-artifact get ...`
|
|
351
|
+
Use `--digest markdown` or `--digest json` for a compact semantic digest instead of head/tail logs. Digest mode keeps status, exit code, truncation counts, runner failure facts, a sanitized failure signature, duplicate-line groups, representative lines, redaction counts, and suggested next queries while preserving the wrapped command exit code. Add `--artifact-receipt` with digest mode when you want the exact sanitized full output stored locally as a `context-guard-artifact` receipt; keep the emitted `contextguard-artifact:<id>` handle in agent context and re-expand with the emitted `context-guard-artifact receipt/get/search ...` commands before relying on omitted details. Wrapped commands time out after 600 seconds by default; tune this with `--timeout-seconds`.
|
|
317
352
|
|
|
318
353
|
### Sanitize search and diff output
|
|
319
354
|
|
|
@@ -375,20 +410,25 @@ These fields can flag likely volatile content near the prompt prefix, stable-pre
|
|
|
375
410
|
```bash
|
|
376
411
|
./plugins/context-guard/bin/context-guard-bench \
|
|
377
412
|
--tasks bench/tasks.json --variants bench/variants.json --csv bench/results.csv \
|
|
378
|
-
--ledger-jsonl bench/cost-shift.jsonl --report-json bench/report.json
|
|
413
|
+
--ledger-jsonl bench/cost-shift.jsonl --report-json bench/report.json \
|
|
414
|
+
--dashboard-md bench/dashboard.md
|
|
379
415
|
```
|
|
380
416
|
|
|
417
|
+
For deterministic local replay before a live provider run, add `--evidence-jsonl docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl` and, for the 12-task fixture, `--baseline-variant baseline_full_context_fixture`. Replay mode skips provider and `success_command` execution, writes the same CSV/report/dashboard surfaces, and marks synthetic/manual evidence as non-public-claim-eligible.
|
|
418
|
+
|
|
381
419
|
Read the report through its claim boundaries before writing any savings statement:
|
|
382
420
|
|
|
383
421
|
- Successful baseline/variant runs are compared by real tokens and `cost_usd + external_cost_usd`; byte reductions stay proxy evidence.
|
|
384
422
|
- Token-savings claims require `primary_tokens_measured` on both sides of a matched task.
|
|
385
423
|
- `matched_pair_evidence` links each successful task bucket to the transform, measurement availability, quality gate, and claim boundary.
|
|
424
|
+
- `default_matrix` classifies trimming, artifact escrow, tool pruning, cache advice, adaptive-k, and optional compression as `default-on`, `advisory`, `experimental`, or `reject/rework` from the same matched evidence. The matrix is report-only: it does not change runtime defaults or authorize hosted token/cost savings claims.
|
|
425
|
+
- `public_claim_readiness` is the authoritative release/public-claim gate. It remains false unless matched successful tasks, provider-measured primary tokens/cost, quality non-inferiority, shifted-cost accounting, explicit confidence/failure notes, and complete provider-export provenance all pass; unsupported hosted savings claims are forbidden when `claim_allowed` is false.
|
|
386
426
|
- `wall_time_seconds`, `provider_cached_tokens`, and `provider_cached_tokens_measured` are diagnostic telemetry, not proof of ContextGuard-caused token or cost savings.
|
|
387
427
|
- Optional `self_hosted_metrics` from provider payloads are stored as per-row JSONL ledger sidecars, kept out of CSV/report summaries, and must not be folded into hosted API token/cost savings claims.
|
|
388
428
|
- If cost fields are zero or unavailable, the report can still mark token savings but will not claim shifted-cost savings.
|
|
389
429
|
- CSV schemas are strict; after upgrading the benchmark helper, start a new `--csv` file or migrate the header named in the mismatch error.
|
|
390
430
|
|
|
391
|
-
See [`docs/benchmark-report.example.json`](docs/benchmark-report.example.json) for a minimal report-shape example, [`docs/benchmark-workflow-examples.md`](docs/benchmark-workflow-examples.md) for workflow-specific synthetic examples, and [`docs/experimental-benchmark-fixtures.md`](docs/experimental-benchmark-fixtures.md) for fixture-only experimental task/variant starters.
|
|
431
|
+
See [`docs/benchmark-report.example.json`](docs/benchmark-report.example.json) for a minimal report-shape example, [`docs/benchmark-workflow-examples.md`](docs/benchmark-workflow-examples.md) for workflow-specific synthetic examples, and [`docs/experimental-benchmark-fixtures.md`](docs/experimental-benchmark-fixtures.md) for fixture-only experimental task/variant starters plus synthetic evidence replay.
|
|
392
432
|
|
|
393
433
|
### Manage experimental opt-ins
|
|
394
434
|
|
|
@@ -408,7 +448,7 @@ context-guard experiments record self-hosted-metrics-ledger --ledger-jsonl .cont
|
|
|
408
448
|
context-guard experiments plan local-proxy --json --bind-host 127.0.0.1 --target-host 127.0.0.1 --runtime-gate-ack
|
|
409
449
|
context-guard experiments plan local-proxy-external-forwarding --external-forwarding-intent --external-forwarding-design-ack --allow-host api.example.com --allow-scheme https --credential-redaction-policy strip-sensitive-headers --provider-evidence-boundary diagnostic-only-provider-measured-required --threat-model-note "Only user-owned HTTPS endpoint; sensitive headers are stripped before any future forwarding." --json
|
|
410
450
|
context-guard experiments record local-proxy-runtime-gate --ledger-jsonl .context-guard/local-proxy-gates.jsonl --bind-host 127.0.0.1 --target-host 127.0.0.1 --runtime-gate-ack --json
|
|
411
|
-
context-guard experiments serve local-proxy --bind-host 127.0.0.1 --bind-port 18080 --target-host 127.0.0.1 --target-port 18081 --runtime-gate-ack --forwarding-gate-ack --once --diagnostic-ledger-jsonl .context-guard/local-proxy-diagnostics.jsonl --json
|
|
451
|
+
context-guard experiments serve local-proxy --bind-host 127.0.0.1 --bind-port 18080 --target-host 127.0.0.1 --target-port 18081 --runtime-gate-ack --forwarding-gate-ack --once --ready-file .context-guard/local-proxy-ready.json --response-sandbox --response-artifact-dir .context-guard/artifacts --diagnostic-ledger-jsonl .context-guard/local-proxy-diagnostics.jsonl --json
|
|
412
452
|
context-guard experiments enable output-receipt-trim --root .
|
|
413
453
|
context-guard experiments disable output-receipt-trim --root .
|
|
414
454
|
```
|
|
@@ -417,7 +457,7 @@ The local-proxy examples are intentionally split by side effect:
|
|
|
417
457
|
|
|
418
458
|
- `plan local-proxy` produces advisory metadata only; it does not enable forwarding.
|
|
419
459
|
- `record local-proxy-runtime-gate` appends one localhost-only gate row and still starts no listener, forwards no traffic, persists no API keys, and makes no hosted-savings claim.
|
|
420
|
-
- `serve local-proxy` is the separate MVP. It requires both runtime and forwarding acknowledgements plus `--once`, binds only a literal loopback IP, forwards only to a literal loopback IP target, blocks credential-bearing requests, uses byte/time limits, uses literal IPs instead of hostname DNS targets, does not persist API keys, and does not support external forwarding, CONNECT/TLS proxying, or hosted-savings claims.
|
|
460
|
+
- `serve local-proxy` is the separate MVP. It requires both runtime and forwarding acknowledgements plus `--once`, a private `--ready-file` nonce handoff for the forwarding client, binds only a literal loopback IP, forwards only to a literal loopback IP target, blocks credential-bearing requests, uses byte/time limits, uses literal IPs instead of hostname DNS targets, does not persist API keys, and does not support external forwarding, CONNECT/TLS proxying, or hosted-savings claims. Optional `--response-sandbox` is a mediated response mode, not transparent forwarding: it artifacts only safe UTF-8 upstream response text and returns a compact JSON envelope with `contextguard-artifact:<id>` and rehydration commands; binary, sensitive, oversized, or blocked responses are not artifacted.
|
|
421
461
|
- With `--diagnostic-ledger-jsonl`, `serve` appends one shifted-cost diagnostic row only after a successful forwarded request. The row stores hashes/metadata rather than raw headers, request bodies, response bodies, or hosted-savings evidence.
|
|
422
462
|
- `plan local-proxy-external-forwarding` is a dry-run design gate only. It requires explicit external intent, design acknowledgement, HTTPS host allowlist, threat model notes, credential redaction policy, and provider-evidence boundary, but starts no listener, performs no DNS lookup, calls no external service, forwards no traffic, persists no credentials, and does not ship an external proxy forwarding runtime.
|
|
423
463
|
|
|
@@ -431,19 +471,26 @@ Shipped experimental checker/planner surfaces, plus explicit local context-diff,
|
|
|
431
471
|
| `visual-crop-ocr` | Dry-run visual evidence advice plus an explicit `emit visual-crop-ocr` runtime for caller-supplied evidence packs. | `emit` requires a full visual evidence receipt, missed-context note, and complete user-supplied crop and/or OCR evidence; ContextGuard does not capture screenshots, crop images, run OCR, parse images, call external services, write files, or support hosted token/cost savings claims. |
|
|
432
472
|
| `learned-compression` | Deny-by-default policy checks plus an explicit `emit learned-compression` runtime for caller-supplied compact prose candidates with verified exact fallback content. | `emit` requires sanitized trusted prose, protected-signal denial, a verified local fallback artifact matching the input, and a smaller caller-supplied prose candidate; ContextGuard does not run compressors, embeddings, rerankers, model calls, subprocesses, external services, generated replacement text, or hosted savings claims. |
|
|
433
473
|
| `self-hosted-metrics-ledger` | Dry-run preview plus an explicit `record ... --ledger-jsonl` runtime for local/model-server latency, memory, quality, energy, throughput, and local-cost metrics. | The dry-run preview does not write a ledger; the explicit record command writes only local JSONL sidecars and still does not support hosted API token/cost savings claims. |
|
|
434
|
-
| `local-proxy` | Localhost-only advisory metadata, design-only `plan local-proxy-external-forwarding` review for future external forwarding, an explicit `record local-proxy-runtime-gate --ledger-jsonl` runtime for one local gate row, an explicit one-shot `serve local-proxy` loopback forwarding MVP, and optional `--diagnostic-ledger-jsonl` shifted-cost diagnostics for successful forwarded requests. | `plan` writes no ledger. `record` writes only after localhost-only metadata and `--runtime-gate-ack`; it starts no listener, forwards no traffic, and performs no DNS lookup. `serve` additionally requires `--forwarding-gate-ack --once`, literal loopback bind/target IPs, nonzero ports, bounded bytes/timeouts, and credential-free requests; it performs no external forwarding, no CONNECT/TLS proxying, no API-key persistence, and no hosted-savings claim. `--diagnostic-ledger-jsonl` writes only successful-forward diagnostics with no raw headers/bodies and no hosted-savings claim. `plan local-proxy-external-forwarding` emits threat-model/allowlist/redaction/provider-evidence design metadata only and still performs no DNS lookup, external service call, traffic forwarding, credential persistence, or hosted-savings claim. |
|
|
474
|
+
| `local-proxy` | Localhost-only advisory metadata, design-only `plan local-proxy-external-forwarding` review for future external forwarding, an explicit `record local-proxy-runtime-gate --ledger-jsonl` runtime for one local gate row, an explicit one-shot `serve local-proxy` loopback forwarding MVP, optional `--response-sandbox` compact artifact envelopes, and optional `--diagnostic-ledger-jsonl` shifted-cost diagnostics for successful forwarded requests. | `plan` writes no ledger. `record` writes only after localhost-only metadata and `--runtime-gate-ack`; it starts no listener, forwards no traffic, and performs no DNS lookup. `serve` additionally requires `--forwarding-gate-ack --once`, a private `--ready-file` nonce handoff, literal loopback bind/target IPs, nonzero ports, bounded bytes/timeouts, and credential-free requests; it performs no external forwarding, no CONNECT/TLS proxying, no API-key persistence, and no hosted-savings claim. `--response-sandbox` can store safe UTF-8 response text as a sanitized local artifact receipt and return a compact envelope with redacted rehydration command templates; it does not claim hosted token/cost savings. `--diagnostic-ledger-jsonl` writes only successful-forward diagnostics with no raw headers/bodies and no hosted-savings claim. `plan local-proxy-external-forwarding` emits threat-model/allowlist/redaction/provider-evidence design metadata only and still performs no DNS lookup, external service call, traffic forwarding, credential persistence, or hosted-savings claim. |
|
|
435
475
|
|
|
436
476
|
## What is not yet shipped
|
|
437
477
|
|
|
438
|
-
These are directions the project has
|
|
478
|
+
These are directions the project has tracked, not committed features. Nothing here ships unless documented elsewhere in the repository.
|
|
479
|
+
|
|
480
|
+
ContextGuard does not yet ship:
|
|
439
481
|
|
|
440
|
-
-
|
|
482
|
+
- learned/synthetic compressor execution or generated replacement text beyond the caller-supplied learned candidate emitter
|
|
483
|
+
- generated crop/OCR or visual-token pruning runtime beyond the caller-supplied visual evidence-pack emitter
|
|
484
|
+
- self-hosted KV/latent optimization beyond explicit local metrics recording
|
|
485
|
+
- external, daemon, or credential-bearing proxy forwarding beyond the one-shot literal-loopback local proxy MVP
|
|
486
|
+
|
|
487
|
+
See the [experimental token-reduction radar](research/experimental-token-reduction-radar.md) and [fixture-only experimental benchmark starters](docs/experimental-benchmark-fixtures.md). Those lanes remain experimental/non-shipped under the later-roadmap gate until matched successful tasks, failure-rate guardrails, human-correction tracking, shifted-cost accounting, provider-measured token/cost evidence, and separate future PR gates justify any hosted API savings claim or broader runtime feature claim.
|
|
441
488
|
|
|
442
489
|
## Repository layout
|
|
443
490
|
|
|
444
491
|
- `.claude-plugin/marketplace.json` — Claude Code marketplace manifest.
|
|
445
492
|
- `plugins/context-guard/` — installable Claude Code plugin package.
|
|
446
|
-
- `context-guard-kit/` —
|
|
493
|
+
- `context-guard-kit/` — checkout-local Python/Bash helper sources. npm packages ship synchronized `plugins/context-guard/bin` and `plugins/context-guard/lib` copies instead of duplicating this source tree.
|
|
447
494
|
- `docs/index.html` — static landing page for the project.
|
|
448
495
|
- `tests/` — regression tests for helper behavior.
|
|
449
496
|
|
|
@@ -477,6 +524,8 @@ export PATH="$PWD/plugins/context-guard/bin:$PATH"
|
|
|
477
524
|
context-guard-setup --plan
|
|
478
525
|
```
|
|
479
526
|
|
|
527
|
+
Do not rely on `PATH` lookup for generated hooks by default. The setup wizard records explicit bundled or checkout-local helper paths; `--allow-path-helper-fallback` is only for trusted external installs and validates the resolved helper path, symlink state, and bounded identity probe before writing commands. The macOS app helper follows the same trust model: no launch-CWD discovery, no relative override paths, and no inherited ambient shell environment beyond the allowlisted values it needs to start.
|
|
528
|
+
|
|
480
529
|
## Release checks
|
|
481
530
|
|
|
482
531
|
Before publishing or merging release-sensitive changes, run the copy check and both gates:
|
|
@@ -487,7 +536,7 @@ python3 scripts/prepublish_check.py
|
|
|
487
536
|
python3 scripts/release_smoke.py
|
|
488
537
|
```
|
|
489
538
|
|
|
490
|
-
When a helper under `context-guard-kit/` changes, run `python3 scripts/sync_plugin_copies.py --write` before the gates. `sync_plugin_copies.py --check` verifies the exact-copy contract up front; `prepublish_check.py` verifies package invariants, synchronized plugin binaries, manifests, diagnostic redaction, and the regression suite. `release_smoke.py` executes representative packaged entrypoints from `plugins/context-guard/bin` in a temporary project so broken CLI wiring is caught before publish. See [docs/release-runbook.md](docs/release-runbook.md) for the full release workflow, evidence checklist, quad-review requirement, and rollback checklist.
|
|
539
|
+
When a helper under `context-guard-kit/` changes, run `python3 scripts/sync_plugin_copies.py --write` before the gates. `sync_plugin_copies.py --check` verifies the maintainer-facing exact-copy contract up front. npm packages intentionally ship only the synchronized plugin-local `plugins/context-guard/bin` entrypoints and `plugins/context-guard/lib` helpers to avoid duplicate implementation payloads, and the npm bin map intentionally omits legacy `claude-*` wrapper aliases. Command manifests are loaded as literal assignments for release and runtime checks; executable Python, imports, functions, or shadow manifests are rejected. `prepublish_check.py` verifies package invariants, synchronized plugin binaries, manifests, diagnostic redaction, and the regression suite. `release_smoke.py` executes representative packaged entrypoints from `plugins/context-guard/bin` in a temporary project so broken CLI wiring is caught before publish. See [docs/release-runbook.md](docs/release-runbook.md) for the full release workflow, evidence checklist, quad-review requirement, and rollback checklist.
|
|
491
540
|
|
|
492
541
|
Versioned release notes live in [CHANGELOG.md](CHANGELOG.md); the prepublish gate requires an entry matching the plugin manifest version before publishing.
|
|
493
542
|
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
# Fixture-only baseline full context prompt
|
|
2
|
+
|
|
3
|
+
This synthetic baseline prompt represents the unoptimized full-context side of a future matched benchmark. It is dry-run-only and not a hosted API token/cost savings claim. A real run must replace this prose with sanitized project evidence, a real success_command, provider-measured primary token/cost fields, matched successful tasks, a 10%p failure-rate guardrail, human corrections, and shifted-cost accounting.
|
|
4
|
+
|
|
5
|
+
Include exact protected evidence such as paths, IDs, stack frames, JSON keys, and artifact receipt fallback handles without semantic rewriting.
|
|
6
|
+
|
|
7
|
+
Metric checklist for real replacements: tokens_per_successful_task, total_cost_with_shift_usd, external_cost_usd, matched_successful_task evidence, 10%p failure-rate guardrail, and proxy-byte caveat.
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
# Fixture-only ContextGuard advisory-foundations prompt
|
|
2
|
+
|
|
3
|
+
This synthetic candidate prompt represents a future ContextGuard-assisted side using cache layout lint, core-vs-deferred tool schemas, artifact receipts, and claim-safe telemetry. It is dry-run-only and not a hosted API token/cost savings claim. A real run must preserve exact retrieval or re-expand paths for omitted context and compare only provider-measured matched successful tasks.
|
|
4
|
+
|
|
5
|
+
Byte reductions, cache-hit predictions, local latency, char/4 token proxies, and receipt counts are diagnostic only until primary provider token/cost telemetry and shifted external work are measured.
|
|
6
|
+
|
|
7
|
+
Metric checklist for real replacements: tokens_per_successful_task, total_cost_with_shift_usd, external_cost_usd, matched_successful_task evidence, 10%p failure-rate guardrail, and proxy-byte caveat.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
{"artifacts_used": 0, "bytes_after": 9450, "bytes_before": 9450, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_01_bugfix", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1715, "output_tokens": 229}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 11.17}
|
|
2
|
+
{"artifacts_used": 1, "bytes_after": 5481, "bytes_before": 9450, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_01_bugfix", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1131, "output_tokens": 210}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.13}
|
|
3
|
+
{"artifacts_used": 0, "bytes_after": 9900, "bytes_before": 9900, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_02_exploration", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1830, "output_tokens": 238}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 11.34}
|
|
4
|
+
{"artifacts_used": 1, "bytes_after": 5742, "bytes_before": 9900, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_02_exploration", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1207, "output_tokens": 218}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.26}
|
|
5
|
+
{"artifacts_used": 0, "bytes_after": 10350, "bytes_before": 10350, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_03_code_review", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1945, "output_tokens": 247}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 11.51}
|
|
6
|
+
{"artifacts_used": 1, "bytes_after": 6003, "bytes_before": 10350, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_03_code_review", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1283, "output_tokens": 227}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.39}
|
|
7
|
+
{"artifacts_used": 0, "bytes_after": 10800, "bytes_before": 10800, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_04_long_log_analysis", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2060, "output_tokens": 256}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 11.68}
|
|
8
|
+
{"artifacts_used": 1, "bytes_after": 6264, "bytes_before": 10800, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_04_long_log_analysis", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1359, "output_tokens": 235}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.52}
|
|
9
|
+
{"artifacts_used": 0, "bytes_after": 11250, "bytes_before": 11250, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_05_migration", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2175, "output_tokens": 265}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 11.85}
|
|
10
|
+
{"artifacts_used": 1, "bytes_after": 6525, "bytes_before": 11250, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_05_migration", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1435, "output_tokens": 243}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.65}
|
|
11
|
+
{"artifacts_used": 0, "bytes_after": 11700, "bytes_before": 11700, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_06_docs", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2290, "output_tokens": 274}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.02}
|
|
12
|
+
{"artifacts_used": 1, "bytes_after": 6785, "bytes_before": 11700, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_06_docs", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1511, "output_tokens": 252}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.78}
|
|
13
|
+
{"artifacts_used": 0, "bytes_after": 12150, "bytes_before": 12150, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_07_refactor", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2405, "output_tokens": 283}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.19}
|
|
14
|
+
{"artifacts_used": 1, "bytes_after": 7046, "bytes_before": 12150, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_07_refactor", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1587, "output_tokens": 260}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 11.91}
|
|
15
|
+
{"artifacts_used": 0, "bytes_after": 12600, "bytes_before": 12600, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_08_performance", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2520, "output_tokens": 292}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.36}
|
|
16
|
+
{"artifacts_used": 1, "bytes_after": 7307, "bytes_before": 12600, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_08_performance", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1663, "output_tokens": 268}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 12.04}
|
|
17
|
+
{"artifacts_used": 0, "bytes_after": 13050, "bytes_before": 13050, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_09_telemetry", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2635, "output_tokens": 301}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.53}
|
|
18
|
+
{"artifacts_used": 1, "bytes_after": 7568, "bytes_before": 13050, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_09_telemetry", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1739, "output_tokens": 276}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 12.17}
|
|
19
|
+
{"artifacts_used": 0, "bytes_after": 13500, "bytes_before": 13500, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_10_cache_layout", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2750, "output_tokens": 310}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.7}
|
|
20
|
+
{"artifacts_used": 1, "bytes_after": 7829, "bytes_before": 13500, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_10_cache_layout", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1815, "output_tokens": 285}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 12.3}
|
|
21
|
+
{"artifacts_used": 0, "bytes_after": 13950, "bytes_before": 13950, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_11_tool_schema", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2865, "output_tokens": 319}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 12.87}
|
|
22
|
+
{"artifacts_used": 1, "bytes_after": 8090, "bytes_before": 13950, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_11_tool_schema", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1890, "output_tokens": 293}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 12.43}
|
|
23
|
+
{"artifacts_used": 0, "bytes_after": 14400, "bytes_before": 14400, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 0, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_12_artifact_receipt", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 2980, "output_tokens": 328}, "turns": 3, "variant": "baseline_full_context_fixture", "wall_time_seconds": 13.04}
|
|
24
|
+
{"artifacts_used": 1, "bytes_after": 8352, "bytes_before": 14400, "corrections": 0, "cost_measured": false, "cost_usd": 0.0, "effort": "medium", "external_cost_measured": true, "external_cost_usd": 0.0, "external_tokens": 0, "external_tokens_measured": true, "hook_triggers": 1, "model": "sonnet", "notes": "synthetic fixture-only replay row; not provider measured and not public-claim eligible", "primary_tokens_measured": false, "provenance": {"capture_command_or_export_id": "docs/benchmark-fixtures/token-savings-12task.evidence.example.jsonl", "claim_scope": "local_replay_fixture_not_public_claim", "evidence_source_type": "synthetic_fixture"}, "provider_cached_tokens": 0, "provider_cached_tokens_measured": false, "schema_version": "contextguard.bench.run-evidence.v1", "success": true, "task_id": "token_savings_12_artifact_receipt", "tokens": {"cache_creation": 0, "cache_read": 0, "input_tokens": 1966, "output_tokens": 301}, "turns": 2, "variant": "fixture_only_contextguard_advisory_foundations", "wall_time_seconds": 12.56}
|