@softspark/ai-toolkit 4.15.0 → 4.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/AGENTS.md +117 -0
  2. package/CHANGELOG.md +43 -0
  3. package/README.md +19 -13
  4. package/app/.claude-plugin/plugin.json +1 -1
  5. package/app/ARCHITECTURE.md +4 -3
  6. package/app/hooks/_hook-io.sh +18 -3
  7. package/app/hooks/ai-toolkit-statusline.sh +30 -5
  8. package/app/hooks/filter-tool-output.sh +76 -0
  9. package/app/hooks/governance-capture.sh +1 -1
  10. package/app/hooks/guard-path.sh +2 -2
  11. package/app/hooks/post-tool-use.sh +5 -3
  12. package/app/hooks/pre-compact-save.sh +4 -3
  13. package/app/hooks/quality-gate.sh +12 -1
  14. package/app/hooks/revert-guard.sh +5 -2
  15. package/app/hooks/save-session.sh +4 -2
  16. package/app/hooks/session-end.sh +36 -4
  17. package/app/hooks/session-start.sh +11 -5
  18. package/app/hooks.json +10 -0
  19. package/app/output-filter-policy.json +15 -0
  20. package/app/skills/brand-voice/scripts/measure.py +7 -5
  21. package/benchmarks/ecosystem-doctor-snapshot.json +22 -22
  22. package/benchmarks/output-filter/README.md +11 -0
  23. package/benchmarks/output-filter/scenarios.json +25 -0
  24. package/bin/ai-toolkit.js +2 -0
  25. package/kb/history/completed/native-tool-output-filter-plan.md +517 -0
  26. package/kb/procedures/release-preparation-sop.md +6 -5
  27. package/kb/reference/architecture-overview.md +6 -5
  28. package/kb/reference/cli-reference.md +19 -2
  29. package/kb/reference/codex-cli-compatibility.md +1 -0
  30. package/kb/reference/copilot-compatibility.md +173 -0
  31. package/kb/reference/enterprise-config-guide.md +28 -2
  32. package/kb/reference/global-install-model.md +6 -2
  33. package/kb/reference/hooks-catalog.md +105 -16
  34. package/kb/reference/opencode-compatibility.md +1 -0
  35. package/kb/reference/supported-tools-registry.md +10 -5
  36. package/kb/reference/tool-output-filter.md +288 -0
  37. package/kb/reference/windows-support.md +4 -3
  38. package/llms-full.txt +1182 -40
  39. package/llms.txt +3 -0
  40. package/manifest.json +9 -6
  41. package/package.json +3 -2
  42. package/scripts/benchmark_output_filter.py +343 -0
  43. package/scripts/check_deps.py +16 -0
  44. package/scripts/claude_app.py +30 -2
  45. package/scripts/config_cli.py +4 -4
  46. package/scripts/config_lock.py +120 -14
  47. package/scripts/config_merger.py +103 -20
  48. package/scripts/config_resolver.py +22 -2
  49. package/scripts/config_validator.py +268 -16
  50. package/scripts/copilot_legacy_hashes.json +338 -0
  51. package/scripts/doctor.py +1 -0
  52. package/scripts/generate_codex_hooks.py +2 -0
  53. package/scripts/generate_copilot.py +464 -71
  54. package/scripts/generate_copilot_hooks.py +124 -7
  55. package/scripts/generate_gemini_hooks.py +33 -10
  56. package/scripts/generate_opencode_plugin.py +28 -12
  57. package/scripts/install_steps/ai_tools.py +115 -3
  58. package/scripts/install_steps/hooks.py +25 -1
  59. package/scripts/output_filter_cli.py +347 -0
  60. package/scripts/output_filter_hook.py +23 -0
  61. package/scripts/plugin_schema.py +27 -1
  62. package/scripts/schemas/ai-toolkit-config.schema.json +83 -5
  63. package/scripts/session_state.py +156 -42
  64. package/scripts/tool_output_filter/__init__.py +33 -0
  65. package/scripts/tool_output_filter/contracts.py +173 -0
  66. package/scripts/tool_output_filter/engine.py +260 -0
  67. package/scripts/tool_output_filter/hook_runtime.py +369 -0
  68. package/scripts/tool_output_filter/input.py +56 -0
  69. package/scripts/tool_output_filter/invariants.py +40 -0
  70. package/scripts/tool_output_filter/policy.py +153 -0
  71. package/scripts/tool_output_filter/profiles/__init__.py +68 -0
  72. package/scripts/tool_output_filter/profiles/repeat_lines.py +71 -0
  73. package/scripts/tool_output_filter/profiles/tap_success.py +154 -0
  74. package/scripts/tool_output_filter/recovery.py +846 -0
  75. package/scripts/tool_output_filter/telemetry.py +13 -0
  76. package/scripts/uninstall.py +96 -3
@@ -0,0 +1,517 @@
1
+ ---
2
+ title: "Implementation Plan: Native Tool Output Filter"
3
+ category: planning
4
+ service: ai-toolkit
5
+ tags:
6
+ - output-filter
7
+ - hooks
8
+ - recovery
9
+ - performance
10
+ - claude-code
11
+ doc_type: plan
12
+ status: completed
13
+ created: "2026-07-23"
14
+ last_updated: "2026-07-23"
15
+ completed: "2026-07-23"
16
+ completion: "100% of approved Claude Code scope; other runtimes remain manual-only by capability decision"
17
+ shipped_in: "Unreleased"
18
+ description: "Approved implementation plan and completion evidence for the original dependency-free ai-toolkit tool-output filter, including conservative profiles, exact recovery, telemetry, runtime capability gates, and adjacent repairs."
19
+ ---
20
+
21
+ # Implementation Plan: Native Tool Output Filter
22
+
23
+ ## Status
24
+
25
+ Completed on 2026-07-23 after user approval.
26
+
27
+ Completion evidence:
28
+
29
+ - 69 focused Python tests pass, including the production-wrapper benchmark.
30
+ - The 100-sample cold wrapper measures 63.565 ms p95 against the 75 ms gate.
31
+ - The 8 MiB profile case measures 4.354 ms p95 and 8,792,932 peak traced
32
+ bytes against the 150 ms and 41,943,040 byte gates.
33
+ - The strict repository validator reports 44 agents, 108 skills, and 1477
34
+ tests with zero errors or warnings.
35
+ - Ruff, mypy, ShellCheck, skill audit, generated artifacts, and the final
36
+ repository test gate pass for the changed surface.
37
+
38
+ ## Context
39
+
40
+ `ai-toolkit` currently controls assistant response length and reports real
41
+ Claude session tokens, but it does not transform live tool output before that
42
+ output reaches the model. The requested feature is an original, MIT-licensed,
43
+ dependency-free implementation inside `ai-toolkit`. RTK is research input only:
44
+ no runtime dependency, vendoring, translated code, copied filters, fixtures,
45
+ regex tables, CLI names, or configuration keys.
46
+
47
+ The first native integration targets Claude Code because current Claude hooks
48
+ support replacing successful tool output through
49
+ `PostToolUse.hookSpecificOutput.updatedToolOutput`. The transformer remains
50
+ strictly post-execution and cannot alter the command, arguments, environment,
51
+ working directory, permission decision, exit status, or signal.
52
+
53
+ This feature is separate from MCP `tools/list` description trimming. Hooks can
54
+ transform an executed tool result, but they do not intercept MCP catalog
55
+ metadata. [PATH: kb/history/completed/f2-mcp-trim-spike-20260504.md]
56
+ [PATH: kb/planning/mcp-context-trim-v4-prd.md]
57
+
58
+ Relevant local boundaries:
59
+
60
+ - Fixed lifecycle enforcement belongs in hooks. [PATH: CLAUDE.md:10]
61
+ - The current safety guards run in `PreToolUse`. [PATH: app/hooks.json:37]
62
+ - Hook runtime Python helpers are deployed explicitly.
63
+ [PATH: scripts/install_steps/hooks.py:71]
64
+ - Runtime hook schemas differ by editor.
65
+ [PATH: kb/reference/hooks-catalog.md:571]
66
+ - Existing output/token work intentionally uses native mechanisms.
67
+ [PATH: kb/history/completed/output-token-discipline-plan-20260504.md]
68
+ - Claude hook contract:
69
+ <https://code.claude.com/docs/en/hooks>
70
+
71
+ ## Scope
72
+
73
+ ### Included
74
+
75
+ - Python standard library implementation under `scripts/`.
76
+ - Pure post-execution transformation of successful textual Bash output.
77
+ - `off`, `observe`, and `safe` modes.
78
+ - Claude Code integration through `PostToolUse.updatedToolOutput`.
79
+ - Manual CLI for fixture inspection, status, recovery, and cleanup.
80
+ - Two initial deterministic profiles:
81
+ - `repeat-lines`: aggregate adjacent identical non-diagnostic lines and state
82
+ their multiplicity.
83
+ - `tap-success`: compact valid successful TAP while retaining plan,
84
+ directives, comments, totals, duration, and all diagnostic material.
85
+ - Byte and line savings, latency, outcome, profile ID, and profile version.
86
+ - Exact, bounded, session-scoped raw recovery when `safe` mode is enabled.
87
+ - Capability-gated adapters for additional runtimes after Phase 2.
88
+
89
+ ### Excluded
90
+
91
+ - RTK binaries, libraries, source code, filters, fixtures, configuration, or
92
+ branding.
93
+ - Command rewriting, `sh -c`, shell parsing, permission decisions, or command
94
+ execution by the filter.
95
+ - LLM-generated summaries or network calls.
96
+ - Project-defined regex filters in the first release.
97
+ - Failed commands, non-empty `stderr`, signals, TTY/streaming output, binary or
98
+ invalid text, pipes, redirects, security scanners, dependency audits,
99
+ deployment, migrations, permission failures, and destructive-command
100
+ diagnostics.
101
+ - `Read`, web results, arbitrary MCP results, and MCP `tools/list`.
102
+ - Claims that byte reduction equals billed-token or whole-session savings.
103
+
104
+ ## Architecture
105
+
106
+ ```text
107
+ PostToolUse payload
108
+ |
109
+ v
110
+ runtime adapter validates the native payload and output shape
111
+ |
112
+ +-- unsupported, disabled, unsafe, failed, or malformed --> no hook output
113
+ |
114
+ v
115
+ eligibility policy selects an explicit profile
116
+ |
117
+ v
118
+ pure deterministic transformer
119
+ |
120
+ v
121
+ invariant gate
122
+ - mandatory facts preserved
123
+ - no forbidden invention
124
+ - deterministic and idempotent
125
+ - at least 15% and 1 KiB smaller
126
+ |
127
+ +-- invariant failure --> exact raw passthrough + session circuit breaker
128
+ |
129
+ v
130
+ exact raw response written to bounded ephemeral recovery store
131
+ |
132
+ +-- recovery unavailable --> exact raw passthrough
133
+ |
134
+ v
135
+ hookSpecificOutput.updatedToolOutput
136
+ ```
137
+
138
+ ### Module layout
139
+
140
+ ```text
141
+ scripts/
142
+ tool_output_filter/
143
+ __init__.py
144
+ contracts.py
145
+ engine.py
146
+ hook_runtime.py
147
+ input.py
148
+ policy.py
149
+ invariants.py
150
+ recovery.py
151
+ telemetry.py
152
+ profiles/
153
+ __init__.py
154
+ repeat_lines.py
155
+ tap_success.py
156
+ output_filter_hook.py
157
+ output_filter_cli.py
158
+ benchmark_output_filter.py
159
+
160
+ app/
161
+ hooks/
162
+ filter-tool-output.sh
163
+ session-end.sh
164
+ hooks.json
165
+ output-filter-policy.json
166
+
167
+ tests/
168
+ fixtures/output-filter/
169
+ repeat-basic/
170
+ tap-basic/
171
+ tap-diagnostic/
172
+ ansi-adversarial/
173
+ test_tool_output_filter.py
174
+ test_tool_output_filter_cli.py
175
+ test_tool_output_filter_properties.py
176
+ test_tool_output_filter_recovery.py
177
+ test_tool_output_filter_benchmark.py
178
+ test_output_filter_config.bats
179
+ test_output_filter_hook.bats
180
+
181
+ kb/reference/
182
+ tool-output-filter.md
183
+ ```
184
+
185
+ ### Configuration contract
186
+
187
+ ```json
188
+ {
189
+ "toolOutputFilter": {
190
+ "mode": "off",
191
+ "profiles": ["repeat-lines", "tap-success"],
192
+ "maxInputBytes": 8388608,
193
+ "minSavingsBytes": 1024,
194
+ "minSavingsRatio": 0.15,
195
+ "recovery": {
196
+ "mode": "ephemeral",
197
+ "ttlMinutes": 60,
198
+ "maxSessionBytes": 33554432
199
+ }
200
+ }
201
+ }
202
+ ```
203
+
204
+ - `off`: no Python invocation and no metrics.
205
+ - `observe`: calculate an eligible result and metadata, but return no hook
206
+ output, so the model receives the original bytes.
207
+ - `safe`: replace only output that passes every invariant and has exact raw
208
+ recovery available.
209
+ - `AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1`: global emergency bypass.
210
+ - Three consecutive runtime or invariant failures disable filtering for the
211
+ current session and emit one bounded warning.
212
+ - All profiles remain explicit opt-in until GA evidence is reviewed.
213
+
214
+ ### Recovery and privacy
215
+
216
+ - Store the exact original tool-response object, not a redacted approximation.
217
+ - Location:
218
+ `~/.softspark/ai-toolkit/sessions/<repo-key>/output-filter/<session>/<opaque-id>`.
219
+ - Directory mode `0700`, file mode `0600`, exclusive no-follow creation,
220
+ opaque random identifiers, atomic publication, quota, and TTL.
221
+ - Never store raw commands, paths, output, environment values, or session IDs
222
+ in telemetry.
223
+ - Telemetry contains only profile ID/version, input/output byte and line
224
+ counts, latency, outcome, and fallback reason.
225
+ - Session end, explicit clean, uninstall, and expired-TTL cleanup remove only
226
+ ai-toolkit-owned recovery artifacts.
227
+ - If the platform cannot provide the secure-store contract, `safe` degrades to
228
+ raw passthrough. It must not silently become lossy.
229
+
230
+ ## Success Criteria
231
+
232
+ - [x] No new npm, pip, Cargo, system, or runtime dependency.
233
+ - [x] Shadow/observe mode is byte-identical for 100% of fixtures and native
234
+ hook-contract test payloads.
235
+ - [x] Any exception, timeout, unsupported encoding, unknown profile, malformed
236
+ payload, failed invariant, unavailable recovery store, non-zero exit,
237
+ signal, or non-empty `stderr` returns the exact original output.
238
+ - [x] Every omission is marked with profile/version, original and emitted
239
+ line counts, and an opaque exact-recovery handle.
240
+ - [x] Golden fixtures preserve 100% of declared mandatory facts and introduce
241
+ zero non-marker facts.
242
+ - [x] Filter output is deterministic, idempotent, and never larger than raw.
243
+ - [x] Eligible fixtures achieve median reduction of at least 30%; replacement
244
+ requires at least 15% and 1 KiB saved per call.
245
+ - [x] Hard input cap is 8 MiB; larger payloads pass through without parsing.
246
+ - [x] Algorithm p95 is at most 20 ms for 100 KiB and 150 ms for 8 MiB; cold
247
+ end-to-end hook invocation p95 is at most 75 ms on CI reference runners.
248
+ - [x] Peak traced Python allocation is at most three times input size plus
249
+ 16 MiB.
250
+ - [x] No regex has unbounded catastrophic backtracking.
251
+ - [x] Claude Code `safe` mode is opt-in and can be disabled globally without
252
+ reinstalling.
253
+ - [x] Additional runtimes activate only after their native result-replacement
254
+ contract has captured fixtures and passing integration tests.
255
+ - [x] Full repository validation and test suite are green.
256
+
257
+ ## Pre-Mortem
258
+
259
+ | Risk | Probability | Impact | Mitigation |
260
+ |---|---|---|---|
261
+ | A profile removes a discriminating fact | High | High | Mandatory-fact fixtures, adversarial sentinels, raw recovery, one regression disables the profile |
262
+ | A runtime changes its tool-output schema | Medium | High | Versioned adapters, strict shape validation, unknown shape returns no hook output |
263
+ | Filtering hides security or failure diagnostics | Medium | High | Explicit exclusion list, successful stdout-only MVP, failures always raw |
264
+ | Recovery leaks secrets or paths | Medium | High | Exact data only in opt-in ephemeral spool, `0700`/`0600`, no-follow, quota, TTL, complete cleanup |
265
+ | Hook latency degrades normal tool use | Medium | Medium | Bash fast-path, size threshold, hard cap, benchmark gates, circuit breaker |
266
+ | Command classification mishandles shell syntax | High | Medium | No command rewriting or execution, conservative allowlist, pipes/redirects bypass |
267
+ | Regex or parser behavior is locale/version-specific | Medium | High | Independently captured multi-version fixtures; malformed/localized output passes raw |
268
+ | Concurrent sessions corrupt metrics or recovery | Medium | Medium | Per-session directories, opaque IDs, atomic writes; do not reuse global `session-edits.json` |
269
+ | Runtime adapters diverge in safety behavior | High | High | Capability matrix and one adapter at a time; no guessed compatibility |
270
+ | Scope expands into MCP catalog proxying | Medium | Medium | Keep MCP trim as a separate PRD and release boundary |
271
+
272
+ ## Tasks
273
+
274
+ ### Phase 1: Observable vertical slice, no output mutation (M)
275
+
276
+ Success criteria:
277
+
278
+ - Claude PostToolUse payloads are parsed by a native Python stdlib engine.
279
+ - `off` and `observe` modes cannot change model-visible output.
280
+ - Independent fixture format and semantic oracle are operational.
281
+ - No raw tool output or command is persisted.
282
+
283
+ Tasks:
284
+
285
+ - [x] Define contracts, independent fixture provenance ledger, conservative
286
+ eligibility rules, and clean implementation boundary.
287
+ Owner: `infrastructure-architect`.
288
+ Files: `kb/reference/tool-output-filter.md`,
289
+ `tests/fixtures/output-filter/README.md`.
290
+ - [x] Implement pure engine, policy, invariants, `repeat-lines` candidate, and
291
+ `tap-success` candidate in observe-only mode.
292
+ Owner: `backend-specialist`.
293
+ Files: `scripts/tool_output_filter/**`.
294
+ - [x] Add manual `inspect`, `status`, recovery, and cleanup CLI.
295
+ Owner: `backend-specialist`.
296
+ Files: `scripts/output_filter_cli.py`, `bin/ai-toolkit.js`.
297
+ - [x] Add Claude PostToolUse adapter last in hook order and deploy the Python
298
+ runtime package with installed hooks.
299
+ Owner: `command-expert`.
300
+ Files: `app/hooks/filter-tool-output.sh`, `app/hooks.json`,
301
+ `scripts/install_steps/hooks.py`.
302
+ - [x] Add `toolOutputFilter` schema, merge, validation, and effective-policy
303
+ materialization; fix config-lock source/version/integrity staleness and
304
+ the Article VII reserved-number mismatch while those files are open.
305
+ Owner: `backend-specialist`.
306
+ Files: `scripts/schemas/ai-toolkit-config.schema.json`,
307
+ `scripts/config_merger.py`, `scripts/config_validator.py`,
308
+ `scripts/config_lock.py`, `app/output-filter-policy.json`.
309
+ - [x] Build unit, property, hook-contract, malformed, binary, Unicode,
310
+ ANSI/OSC, injection, huge-line, and concurrency tests.
311
+ Owner: `test-engineer`.
312
+ Files: `tests/test_tool_output_filter*.py`,
313
+ `tests/test_tool_output_filter*.bats`,
314
+ `tests/fixtures/output-filter/**`.
315
+
316
+ Rollback/scope cut:
317
+
318
+ - Remove the owned PostToolUse entry and deployed runtime package.
319
+ - If native payload fixtures are unstable, ship only the manual inspect CLI
320
+ and fixture oracle; do not enable the hook.
321
+
322
+ ### Phase 2: Claude Code safe mode with exact recovery (L)
323
+
324
+ Dependency: Phase 1 must pass all observe-mode and performance gates.
325
+
326
+ Success criteria:
327
+
328
+ - `safe` is explicit opt-in.
329
+ - Only successful, stdout-only, allowlisted text is replaceable.
330
+ - Exact raw recovery is available before replacement is emitted.
331
+ - Every failure path is byte-identical passthrough.
332
+
333
+ Tasks:
334
+
335
+ - [x] Promote `repeat-lines` and `tap-success` only after mandatory-fact,
336
+ idempotence, determinism, and minimum-savings gates pass.
337
+ Owner: `backend-specialist`.
338
+ Files: `scripts/tool_output_filter/profiles/**`,
339
+ `scripts/tool_output_filter/invariants.py`.
340
+ - [x] Implement bounded secure ephemeral recovery and exact `recover`/`clean`
341
+ CLI operations.
342
+ Owner: `security-architect`.
343
+ Files: `scripts/tool_output_filter/recovery.py`,
344
+ `scripts/output_filter_cli.py`.
345
+ - [x] Add session circuit breaker, metadata-only telemetry, TTL/quota cleanup,
346
+ and owned-artifact removal.
347
+ Owner: `backend-specialist`.
348
+ Files: `scripts/tool_output_filter/telemetry.py`,
349
+ `app/hooks/session-end.sh`, `scripts/uninstall.py`.
350
+ - [x] Verify safety precedence and that filtering cannot emit permission
351
+ decisions, execute output, or change original tool input/status.
352
+ Owner: `security-auditor`.
353
+ Files: `tests/test_tool_output_filter.py`,
354
+ `tests/test_tool_output_filter_recovery.py`,
355
+ `tests/test_output_filter_hook.bats`.
356
+ - [x] Add deterministic benchmark corpus and enforce latency, memory, maximum
357
+ input, and savings thresholds.
358
+ Owner: `performance-optimizer`.
359
+ Files: `scripts/benchmark_output_filter.py`,
360
+ `benchmarks/output-filter/**`.
361
+
362
+ Rollback/scope cut:
363
+
364
+ - Global disable returns the hook to observe/raw behavior immediately.
365
+ - Per-profile disable preserves other validated profiles.
366
+ - If exact secure recovery is unavailable on a platform, that platform stays
367
+ in `observe`; no weaker recovery implementation is accepted.
368
+ - If TAP cannot meet semantic gates, release only `repeat-lines`.
369
+
370
+ ### Phase 3: Capability-gated runtime expansion (L)
371
+
372
+ Dependency: Phase 2 must complete Claude safe-mode dogfooding without a
373
+ confirmed semantic loss.
374
+
375
+ Success criteria:
376
+
377
+ - Each runtime has a documented capability: native replacement, observe-only,
378
+ manual CLI, or unsupported.
379
+ - No adapter uses command rewriting or `additionalContext` as fake replacement.
380
+ - Each active adapter has captured native payload/output fixtures.
381
+
382
+ Tasks:
383
+
384
+ - [x] Build and document the capability matrix from current runtime contracts.
385
+ Owner: `technical-researcher`.
386
+ Files: `kb/reference/tool-output-filter.md`,
387
+ `kb/reference/supported-tools-registry.md`.
388
+ - [x] Classify adapters one by one and keep OpenCode, Cursor, and the remaining
389
+ runtimes manual-only because no independently verified native
390
+ result-replacement contract passed the release boundary.
391
+ Owner: `command-expert`.
392
+ Files: `scripts/generate_opencode_plugin.py`,
393
+ `scripts/generate_cursor_hooks.py`, runtime-specific fixture files.
394
+ - [x] Fix ignored OpenCode guard block propagation before adding its filter
395
+ adapter.
396
+ Owner: `backend-specialist`.
397
+ Files: `scripts/generate_opencode_plugin.py`,
398
+ `tests/test_opencode*.bats`.
399
+ - [x] Fix Gemini invalid-settings overwrite and non-atomic write before any
400
+ Gemini filter adapter is attempted.
401
+ Owner: `backend-specialist`.
402
+ Files: `scripts/generate_gemini_hooks.py`,
403
+ `tests/test_gemini.bats`.
404
+ - [x] Add per-runtime uninstall, rollback, user-hook preservation, failure, and
405
+ schema-drift tests.
406
+ Owner: `test-engineer`.
407
+ Files: `tests/test_hooks_per_editor.bats`,
408
+ `tests/test_*hooks*.bats`.
409
+
410
+ Rollback/scope cut:
411
+
412
+ - A runtime without safe native replacement remains manual CLI or
413
+ observe-only.
414
+ - Any runtime-specific regression removes only its owned adapter.
415
+ - Cross-runtime uniformity is not a release requirement.
416
+
417
+ ### Phase 4: Documentation and GA decision (M)
418
+
419
+ Dependency: Phases 1 and 2 are mandatory; Phase 3 may remain partial.
420
+
421
+ Success criteria:
422
+
423
+ - Dogfood evidence distinguishes eligible-output byte savings from actual
424
+ session token receipts.
425
+ - Documentation lists exclusions, recovery sensitivity, and bypass procedure.
426
+ - `safe` remains opt-in unless a separate user-approved promotion decision is
427
+ made.
428
+
429
+ Tasks:
430
+
431
+ - [x] Review available shadow/safe evidence, false-positive and
432
+ circuit-breaker paths, and latency. No production Claude JSONL receipt
433
+ evidence was collected in this implementation run, so documentation
434
+ makes no token-savings claim and `safe` remains opt-in.
435
+ Owner: `data-analyst`.
436
+ Files: `kb/reference/tool-output-filter.md`,
437
+ `kb/reference/stats.md`.
438
+ - [x] Update all hook counts, architecture references, CLI help, lifecycle
439
+ catalog, README, changelog, generated instruction surfaces, and release
440
+ notes.
441
+ Owner: `documenter`.
442
+ Files: `README.md`, `CLAUDE.md`, `ARCHITECTURE.md`,
443
+ `app/ARCHITECTURE.md`, `kb/reference/architecture-overview.md`,
444
+ `kb/reference/hooks-catalog.md`, `CHANGELOG.md`, `AGENTS.md`,
445
+ `llms.txt`, `llms-full.txt`, `package.json`, `plugin.json`.
446
+ - [x] Re-read the complete diff for orphaned references, dead code, missing
447
+ behavior coverage, stale docs, and unexpected generated changes.
448
+ Owner: `code-reviewer`.
449
+ Files: all changed files.
450
+
451
+ Rollback/scope cut:
452
+
453
+ - Release `observe` and manual CLI only if active-mode evidence is insufficient.
454
+ - Do not promote any profile to default without separate approval.
455
+
456
+ ## Dependencies
457
+
458
+ ```text
459
+ Phase 1 → Phase 2 → Phase 3 → Phase 4
460
+ └────────→ Phase 4 if runtime expansion is deferred
461
+ ```
462
+
463
+ There are no circular dependencies. Phase 1 is shippable as diagnostics,
464
+ Phase 2 as Claude-only opt-in filtering, and Phase 3 as incremental
465
+ runtime-by-runtime support.
466
+
467
+ ## Verification
468
+
469
+ Focused verification:
470
+
471
+ 1. `python3 -m unittest discover -s tests -p 'test_tool_output_filter*.py'`
472
+ 2. `bats tests/test_output_filter_config.bats tests/test_output_filter_hook.bats`
473
+ 3. `python3 scripts/benchmark_output_filter.py`
474
+ 4. `shellcheck --severity=warning app/hooks/*.sh`
475
+
476
+ Repository quality gates:
477
+
478
+ 1. `npm run generate:all`
479
+ 2. `python3 scripts/validate.py --strict`
480
+ 3. `python3 scripts/audit_skills.py --ci`
481
+ 4. `npm test`
482
+ 5. `git diff --check`
483
+ 6. Re-read `git diff` and confirm no orphaned references, missing tests, stale
484
+ docs, copied upstream expressions, or new external dependencies.
485
+
486
+ Expected result: every command exits `0`, focused semantic fixtures report zero
487
+ missing mandatory facts and zero forbidden inventions, active replacements meet
488
+ the savings threshold, and all unsafe paths remain exact passthrough.
489
+
490
+ ## Agent Assignments
491
+
492
+ | Responsibility | Agent | Model |
493
+ |---|---|---|
494
+ | Architecture and contracts | `infrastructure-architect` | Assigned tier, no override |
495
+ | Python engine and configuration | `backend-specialist` | Assigned tier, no override |
496
+ | Hook and runtime adapters | `command-expert` | Assigned tier, no override |
497
+ | Recovery threat model | `security-architect` | Assigned tier, no override |
498
+ | Security verification | `security-auditor` | Assigned tier, no override |
499
+ | Semantic and integration tests | `test-engineer` | Assigned tier, no override |
500
+ | Performance gates | `performance-optimizer` | Assigned tier, no override |
501
+ | Runtime contract research | `technical-researcher` | Assigned tier, no override |
502
+ | Evidence analysis | `data-analyst` | Assigned tier, no override |
503
+ | Documentation | `documenter` | Assigned tier, no override |
504
+ | Final diff review | `code-reviewer` | Assigned tier, no override |
505
+
506
+ ## Independent-Implementation Boundary
507
+
508
+ - Implement from this plan, local ai-toolkit requirements, native runtime
509
+ contracts, and independently captured command outputs.
510
+ - Do not consult or copy RTK source while implementing filters.
511
+ - Do not copy or transliterate identifiers, control flow, comments, regex
512
+ tables, filter order, fixtures, messages, configuration keys, default values,
513
+ benchmarks, or documentation wording.
514
+ - Record the origin and rationale of each filter rule in the fixture provenance
515
+ ledger.
516
+ - If any upstream expression is intentionally adapted, stop and perform an
517
+ Apache-2.0 attribution and NOTICE review before continuing.
@@ -3,9 +3,9 @@ title: "SOP: Release Preparation"
3
3
  category: procedures
4
4
  service: ai-toolkit
5
5
  tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
6
- version: "1.11.0"
6
+ version: "1.11.1"
7
7
  created: "2026-04-10"
8
- last_updated: "2026-06-02"
8
+ last_updated: "2026-07-24"
9
9
  description: "Step-by-step checklist for preparing a new ai-toolkit release — ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, and checksum-pin checks added in v2.8.0, the single-run npm test discipline added in v1.8.0, the ecosystem-sync gate added in v1.9.0, the registry-vs-generators drift gate added in v1.10.0, and the mandatory pre-tag ShellCheck gate added in v1.11.0 (publish.yml does not run ShellCheck, so a hook lint failure can publish while reddening main CI — see the v4.5.1 postmortem in Phase 5)."
10
10
  ---
11
11
 
@@ -335,9 +335,10 @@ grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
335
335
  **2. URL-sourced rules and hooks are checksum-pinned:**
336
336
 
337
337
  ```bash
338
- # On a machine that has consumed URL rules/hooks at least once:
339
- jq 'to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
340
- jq 'to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
338
+ # On a machine that has consumed URL rules/hooks at least once
339
+ # (schema_version 1: entries live under the .rules / .hooks key):
340
+ jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
341
+ jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
341
342
  ```
342
343
 
343
344
  - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
@@ -3,9 +3,9 @@ title: "AI Toolkit - Architecture Overview"
3
3
  category: reference
4
4
  service: ai-toolkit
5
5
  tags: [architecture, overview, design, structure]
6
- version: "1.4.7"
6
+ version: "1.5.0"
7
7
  created: "2026-03-23"
8
- last_updated: "2026-07-14"
8
+ last_updated: "2026-07-23"
9
9
  description: "Architecture of ai-toolkit: directory layout, Claude app export, global install model, editor-aware MCP install, Codex translation layer, skill tiers, and integration with projects."
10
10
  ---
11
11
 
@@ -285,7 +285,7 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
285
285
 
286
286
  ## Quality Hooks
287
287
 
288
- 28 entries across 14 lifecycle events. See [hooks-catalog.md](hooks-catalog.md) for full details.
288
+ 29 entries across 14 lifecycle events. See [hooks-catalog.md](hooks-catalog.md) for full details.
289
289
 
290
290
  | Hook | Trigger | Script | Action |
291
291
  |------|---------|--------|--------|
@@ -294,12 +294,13 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
294
294
  | Notification | Claude waiting for input | *(inline)* | macOS desktop notification |
295
295
  | PreToolUse | Before Bash | `guard-destructive.sh` | Block destructive commands |
296
296
  | PreToolUse | Before file ops (Bash, Read, Edit, Write, MultiEdit, Glob, Grep, NotebookEdit, mcp\_filesystem) | `guard-path.sh` | Block wrong-user path hallucination |
297
- | PreToolUse | Before Edit/Write/MultiEdit | `guard-config.sh` | Block config file edits without explicit acknowledgment |
297
+ | PreToolUse | Before Edit/Write/MultiEdit | `guard-config.sh` | Always block protected config edits; remove the hook deliberately when a change is authorized |
298
298
  | PreToolUse | Before Bash (git commit) | `commit-quality.sh` | Advisory Conventional Commits format check |
299
299
  | UserPromptSubmit | Before user prompt execution | `user-prompt-submit.sh` | Prompt governance reminder |
300
300
  | UserPromptSubmit | Before user prompt execution | `track-usage.sh` | Record skill invocations to stats.json |
301
301
  | PostToolUse | After edit/write tools | `post-tool-use.sh` | Lightweight validation reminders |
302
302
  | PostToolUse | After any tool | `governance-capture.sh` | Log security-sensitive operations |
303
+ | PostToolUse | After successful Bash | `filter-tool-output.sh` | Observe eligible text or replace it only after exact recovery succeeds; Claude Code only |
303
304
  | Stop | After response | `quality-check.sh` | Multi-language lint |
304
305
  | Stop | After response | `save-session.sh` | Persist session context |
305
306
  | Stop | Before final stop | `quality-gate.sh` | Block final response on lint/type errors |
@@ -309,7 +310,7 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
309
310
  | SubagentStop | Subagent completion | `subagent-stop.sh` | Handoff checklist for subagents |
310
311
  | PreCompact | Before compaction | `pre-compact.sh` | Save prioritized context: instincts > tasks > git state > decisions |
311
312
  | PreCompact | Before compaction | `pre-compact-save.sh` | Timestamped context snapshot to audit trail |
312
- | SessionEnd | Session end | `session-end.sh` | Persist handoff note for the next session |
313
+ | SessionEnd | Session end | `session-end.sh` | Clean owned output recovery and persist the next-session handoff note |
313
314
 
314
315
  Scripts at `~/.softspark/ai-toolkit/hooks/`. See [hooks-catalog.md](hooks-catalog.md) for details.
315
316
 
@@ -2,9 +2,9 @@
2
2
  title: "CLI Reference"
3
3
  category: reference
4
4
  service: ai-toolkit
5
- tags: [cli, commands, reference, install, update, plugin, mcp, telemetry]
5
+ tags: [cli, commands, reference, install, update, plugin, mcp, telemetry, output-filter]
6
6
  created: "2026-04-13"
7
- last_updated: "2026-07-10"
7
+ last_updated: "2026-07-23"
8
8
  description: "Complete CLI reference for all ai-toolkit commands, options, and flags."
9
9
  ---
10
10
 
@@ -75,6 +75,23 @@ Usage: ai-toolkit <command> [options]
75
75
  | `config create-base <name>` | Scaffold base config npm package |
76
76
  | `config check [path]` | CI enforcement gate (exit 0=pass, 1=fail, 2=no config; `--json`) |
77
77
 
78
+ ## Native Tool-Output Filter
79
+
80
+ | Command | Description |
81
+ |---------|-------------|
82
+ | `output-filter status [--policy PATH]` | Print the effective trusted project/global policy, or inspect an explicit policy |
83
+ | `output-filter inspect --profile repeat-lines\|tap-success` | Read stdin and print content-free candidate metadata without changing or echoing the input |
84
+ | `output-filter recover <handle> [--base-directory PATH] [--session-id ID]` | Print the exact stored native response object for an opaque recovery handle |
85
+ | `output-filter clean [--base-directory PATH] [--session-id ID]` | Remove owned filter artifacts for one session or all sessions in the current repository |
86
+ | `output-filter clean --session-id ID --expired` | Remove expired exact-response artifacts for one native session |
87
+ | `output-filter hook --policy PATH` | **Internal, not for manual use.** `PostToolUse` entry point that reads the native hook payload on stdin and emits any replacement. The installed hook `filter-tool-output.sh` calls the same runtime through the lean `output_filter_hook.py` process instead |
88
+
89
+ `off` is the default. Configure `toolOutputFilter.mode` in
90
+ `.softspark-toolkit.json`, then run `install --local` or `update --local`.
91
+ `AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1` bypasses active filtering immediately.
92
+ See [Native Tool Output Filter](tool-output-filter.md) for eligibility,
93
+ privacy, recovery, and runtime support.
94
+
78
95
  ## Project Registry
79
96
 
80
97
  | Command | Description |
@@ -240,3 +240,4 @@ The Codex compatibility path is verified by:
240
240
  - `kb/reference/skills-catalog.md`
241
241
  - `kb/reference/architecture-overview.md`
242
242
  - `kb/reference/global-install-model.md`
243
+ - `kb/reference/copilot-compatibility.md`