@softspark/ai-toolkit 4.15.0 → 4.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +117 -0
- package/CHANGELOG.md +43 -0
- package/README.md +19 -13
- package/app/.claude-plugin/plugin.json +1 -1
- package/app/ARCHITECTURE.md +4 -3
- package/app/hooks/_hook-io.sh +18 -3
- package/app/hooks/ai-toolkit-statusline.sh +30 -5
- package/app/hooks/filter-tool-output.sh +76 -0
- package/app/hooks/governance-capture.sh +1 -1
- package/app/hooks/guard-path.sh +2 -2
- package/app/hooks/post-tool-use.sh +5 -3
- package/app/hooks/pre-compact-save.sh +4 -3
- package/app/hooks/quality-gate.sh +12 -1
- package/app/hooks/revert-guard.sh +5 -2
- package/app/hooks/save-session.sh +4 -2
- package/app/hooks/session-end.sh +36 -4
- package/app/hooks/session-start.sh +11 -5
- package/app/hooks.json +10 -0
- package/app/output-filter-policy.json +15 -0
- package/app/skills/brand-voice/scripts/measure.py +7 -5
- package/benchmarks/ecosystem-doctor-snapshot.json +22 -22
- package/benchmarks/output-filter/README.md +11 -0
- package/benchmarks/output-filter/scenarios.json +25 -0
- package/bin/ai-toolkit.js +2 -0
- package/kb/history/completed/native-tool-output-filter-plan.md +517 -0
- package/kb/procedures/release-preparation-sop.md +6 -5
- package/kb/reference/architecture-overview.md +6 -5
- package/kb/reference/cli-reference.md +19 -2
- package/kb/reference/codex-cli-compatibility.md +1 -0
- package/kb/reference/copilot-compatibility.md +173 -0
- package/kb/reference/enterprise-config-guide.md +28 -2
- package/kb/reference/global-install-model.md +6 -2
- package/kb/reference/hooks-catalog.md +105 -16
- package/kb/reference/opencode-compatibility.md +1 -0
- package/kb/reference/supported-tools-registry.md +10 -5
- package/kb/reference/tool-output-filter.md +288 -0
- package/kb/reference/windows-support.md +4 -3
- package/llms-full.txt +1182 -40
- package/llms.txt +3 -0
- package/manifest.json +9 -6
- package/package.json +3 -2
- package/scripts/benchmark_output_filter.py +343 -0
- package/scripts/check_deps.py +16 -0
- package/scripts/claude_app.py +30 -2
- package/scripts/config_cli.py +4 -4
- package/scripts/config_lock.py +120 -14
- package/scripts/config_merger.py +103 -20
- package/scripts/config_resolver.py +22 -2
- package/scripts/config_validator.py +268 -16
- package/scripts/copilot_legacy_hashes.json +338 -0
- package/scripts/doctor.py +1 -0
- package/scripts/generate_codex_hooks.py +2 -0
- package/scripts/generate_copilot.py +464 -71
- package/scripts/generate_copilot_hooks.py +124 -7
- package/scripts/generate_gemini_hooks.py +33 -10
- package/scripts/generate_opencode_plugin.py +28 -12
- package/scripts/install_steps/ai_tools.py +115 -3
- package/scripts/install_steps/hooks.py +25 -1
- package/scripts/output_filter_cli.py +347 -0
- package/scripts/output_filter_hook.py +23 -0
- package/scripts/plugin_schema.py +27 -1
- package/scripts/schemas/ai-toolkit-config.schema.json +83 -5
- package/scripts/session_state.py +156 -42
- package/scripts/tool_output_filter/__init__.py +33 -0
- package/scripts/tool_output_filter/contracts.py +173 -0
- package/scripts/tool_output_filter/engine.py +260 -0
- package/scripts/tool_output_filter/hook_runtime.py +369 -0
- package/scripts/tool_output_filter/input.py +56 -0
- package/scripts/tool_output_filter/invariants.py +40 -0
- package/scripts/tool_output_filter/policy.py +153 -0
- package/scripts/tool_output_filter/profiles/__init__.py +68 -0
- package/scripts/tool_output_filter/profiles/repeat_lines.py +71 -0
- package/scripts/tool_output_filter/profiles/tap_success.py +154 -0
- package/scripts/tool_output_filter/recovery.py +846 -0
- package/scripts/tool_output_filter/telemetry.py +13 -0
- package/scripts/uninstall.py +96 -3
package/llms-full.txt
CHANGED
|
@@ -17,6 +17,7 @@
|
|
|
17
17
|
- [Plan: Ecosystem Deep Sweep — All 12 Supported Tools](kb/history/completed/ecosystem-deep-sweep-20260423.md)
|
|
18
18
|
- [Plan: Enterprise Config Inheritance — Multi-Repo Governance with `extends`](kb/history/completed/enterprise-config-inheritance-plan-20260412.md)
|
|
19
19
|
- [Spike: F2 MCP Context Trim — Hook Feasibility & Path Decision](kb/history/completed/f2-mcp-trim-spike-20260504.md)
|
|
20
|
+
- [Implementation Plan: Native Tool Output Filter](kb/history/completed/native-tool-output-filter-plan.md)
|
|
20
21
|
- [Plan: Offline-First SLM Profile — Lightweight Mode for Local Models](kb/history/completed/offline-slm-profile-plan-20260411.md)
|
|
21
22
|
- [Plan: Output & Token Discipline](kb/history/completed/output-token-discipline-plan-20260504.md)
|
|
22
23
|
- [How-To Guides](kb/howto/README.md)
|
|
@@ -38,6 +39,7 @@
|
|
|
38
39
|
- [AI Toolkit - Codex CLI Compatibility](kb/reference/codex-cli-compatibility.md)
|
|
39
40
|
- [Ecosystem Comparison](kb/reference/comparison.md)
|
|
40
41
|
- [Plan: Competitive Features — ai-toolkit](kb/reference/competitive-features-implementation.md)
|
|
42
|
+
- [AI Toolkit - GitHub Copilot Compatibility](kb/reference/copilot-compatibility.md)
|
|
41
43
|
- [Distribution Model](kb/reference/distribution-model.md)
|
|
42
44
|
- [Enterprise Config Inheritance Guide](kb/reference/enterprise-config-guide.md)
|
|
43
45
|
- [Extension API Reference](kb/reference/extension-api.md)
|
|
@@ -61,6 +63,7 @@
|
|
|
61
63
|
- [Usage Statistics](kb/reference/stats.md)
|
|
62
64
|
- [Supported Tools Registry](kb/reference/supported-tools-registry.md)
|
|
63
65
|
- [Config Sync](kb/reference/sync.md)
|
|
66
|
+
- [Native Tool Output Filter](kb/reference/tool-output-filter.md)
|
|
64
67
|
- [Unique Features & Differentiators](kb/reference/unique-features.md)
|
|
65
68
|
- [Windows Support](kb/reference/windows-support.md)
|
|
66
69
|
- [Troubleshooting](kb/troubleshooting/README.md)
|
|
@@ -1774,6 +1777,528 @@ Estimate in the PRD: ~8 working days.
|
|
|
1774
1777
|
|
|
1775
1778
|
---
|
|
1776
1779
|
|
|
1780
|
+
## kb/history/completed/native-tool-output-filter-plan.md
|
|
1781
|
+
|
|
1782
|
+
---
|
|
1783
|
+
title: "Implementation Plan: Native Tool Output Filter"
|
|
1784
|
+
category: planning
|
|
1785
|
+
service: ai-toolkit
|
|
1786
|
+
tags:
|
|
1787
|
+
- output-filter
|
|
1788
|
+
- hooks
|
|
1789
|
+
- recovery
|
|
1790
|
+
- performance
|
|
1791
|
+
- claude-code
|
|
1792
|
+
doc_type: plan
|
|
1793
|
+
status: completed
|
|
1794
|
+
created: "2026-07-23"
|
|
1795
|
+
last_updated: "2026-07-23"
|
|
1796
|
+
completed: "2026-07-23"
|
|
1797
|
+
completion: "100% of approved Claude Code scope; other runtimes remain manual-only by capability decision"
|
|
1798
|
+
shipped_in: "Unreleased"
|
|
1799
|
+
description: "Approved implementation plan and completion evidence for the original dependency-free ai-toolkit tool-output filter, including conservative profiles, exact recovery, telemetry, runtime capability gates, and adjacent repairs."
|
|
1800
|
+
---
|
|
1801
|
+
|
|
1802
|
+
# Implementation Plan: Native Tool Output Filter
|
|
1803
|
+
|
|
1804
|
+
## Status
|
|
1805
|
+
|
|
1806
|
+
Completed on 2026-07-23 after user approval.
|
|
1807
|
+
|
|
1808
|
+
Completion evidence:
|
|
1809
|
+
|
|
1810
|
+
- 69 focused Python tests pass, including the production-wrapper benchmark.
|
|
1811
|
+
- The 100-sample cold wrapper measures 63.565 ms p95 against the 75 ms gate.
|
|
1812
|
+
- The 8 MiB profile case measures 4.354 ms p95 and 8,792,932 peak traced
|
|
1813
|
+
bytes against the 150 ms and 41,943,040 byte gates.
|
|
1814
|
+
- The strict repository validator reports 44 agents, 108 skills, and 1477
|
|
1815
|
+
tests with zero errors or warnings.
|
|
1816
|
+
- Ruff, mypy, ShellCheck, skill audit, generated artifacts, and the final
|
|
1817
|
+
repository test gate pass for the changed surface.
|
|
1818
|
+
|
|
1819
|
+
## Context
|
|
1820
|
+
|
|
1821
|
+
`ai-toolkit` currently controls assistant response length and reports real
|
|
1822
|
+
Claude session tokens, but it does not transform live tool output before that
|
|
1823
|
+
output reaches the model. The requested feature is an original, MIT-licensed,
|
|
1824
|
+
dependency-free implementation inside `ai-toolkit`. RTK is research input only:
|
|
1825
|
+
no runtime dependency, vendoring, translated code, copied filters, fixtures,
|
|
1826
|
+
regex tables, CLI names, or configuration keys.
|
|
1827
|
+
|
|
1828
|
+
The first native integration targets Claude Code because current Claude hooks
|
|
1829
|
+
support replacing successful tool output through
|
|
1830
|
+
`PostToolUse.hookSpecificOutput.updatedToolOutput`. The transformer remains
|
|
1831
|
+
strictly post-execution and cannot alter the command, arguments, environment,
|
|
1832
|
+
working directory, permission decision, exit status, or signal.
|
|
1833
|
+
|
|
1834
|
+
This feature is separate from MCP `tools/list` description trimming. Hooks can
|
|
1835
|
+
transform an executed tool result, but they do not intercept MCP catalog
|
|
1836
|
+
metadata. [PATH: kb/history/completed/f2-mcp-trim-spike-20260504.md]
|
|
1837
|
+
[PATH: kb/planning/mcp-context-trim-v4-prd.md]
|
|
1838
|
+
|
|
1839
|
+
Relevant local boundaries:
|
|
1840
|
+
|
|
1841
|
+
- Fixed lifecycle enforcement belongs in hooks. [PATH: CLAUDE.md:10]
|
|
1842
|
+
- The current safety guards run in `PreToolUse`. [PATH: app/hooks.json:37]
|
|
1843
|
+
- Hook runtime Python helpers are deployed explicitly.
|
|
1844
|
+
[PATH: scripts/install_steps/hooks.py:71]
|
|
1845
|
+
- Runtime hook schemas differ by editor.
|
|
1846
|
+
[PATH: kb/reference/hooks-catalog.md:571]
|
|
1847
|
+
- Existing output/token work intentionally uses native mechanisms.
|
|
1848
|
+
[PATH: kb/history/completed/output-token-discipline-plan-20260504.md]
|
|
1849
|
+
- Claude hook contract:
|
|
1850
|
+
<https://code.claude.com/docs/en/hooks>
|
|
1851
|
+
|
|
1852
|
+
## Scope
|
|
1853
|
+
|
|
1854
|
+
### Included
|
|
1855
|
+
|
|
1856
|
+
- Python standard library implementation under `scripts/`.
|
|
1857
|
+
- Pure post-execution transformation of successful textual Bash output.
|
|
1858
|
+
- `off`, `observe`, and `safe` modes.
|
|
1859
|
+
- Claude Code integration through `PostToolUse.updatedToolOutput`.
|
|
1860
|
+
- Manual CLI for fixture inspection, status, recovery, and cleanup.
|
|
1861
|
+
- Two initial deterministic profiles:
|
|
1862
|
+
- `repeat-lines`: aggregate adjacent identical non-diagnostic lines and state
|
|
1863
|
+
their multiplicity.
|
|
1864
|
+
- `tap-success`: compact valid successful TAP while retaining plan,
|
|
1865
|
+
directives, comments, totals, duration, and all diagnostic material.
|
|
1866
|
+
- Byte and line savings, latency, outcome, profile ID, and profile version.
|
|
1867
|
+
- Exact, bounded, session-scoped raw recovery when `safe` mode is enabled.
|
|
1868
|
+
- Capability-gated adapters for additional runtimes after Phase 2.
|
|
1869
|
+
|
|
1870
|
+
### Excluded
|
|
1871
|
+
|
|
1872
|
+
- RTK binaries, libraries, source code, filters, fixtures, configuration, or
|
|
1873
|
+
branding.
|
|
1874
|
+
- Command rewriting, `sh -c`, shell parsing, permission decisions, or command
|
|
1875
|
+
execution by the filter.
|
|
1876
|
+
- LLM-generated summaries or network calls.
|
|
1877
|
+
- Project-defined regex filters in the first release.
|
|
1878
|
+
- Failed commands, non-empty `stderr`, signals, TTY/streaming output, binary or
|
|
1879
|
+
invalid text, pipes, redirects, security scanners, dependency audits,
|
|
1880
|
+
deployment, migrations, permission failures, and destructive-command
|
|
1881
|
+
diagnostics.
|
|
1882
|
+
- `Read`, web results, arbitrary MCP results, and MCP `tools/list`.
|
|
1883
|
+
- Claims that byte reduction equals billed-token or whole-session savings.
|
|
1884
|
+
|
|
1885
|
+
## Architecture
|
|
1886
|
+
|
|
1887
|
+
```text
|
|
1888
|
+
PostToolUse payload
|
|
1889
|
+
|
|
|
1890
|
+
v
|
|
1891
|
+
runtime adapter validates the native payload and output shape
|
|
1892
|
+
|
|
|
1893
|
+
+-- unsupported, disabled, unsafe, failed, or malformed --> no hook output
|
|
1894
|
+
|
|
|
1895
|
+
v
|
|
1896
|
+
eligibility policy selects an explicit profile
|
|
1897
|
+
|
|
|
1898
|
+
v
|
|
1899
|
+
pure deterministic transformer
|
|
1900
|
+
|
|
|
1901
|
+
v
|
|
1902
|
+
invariant gate
|
|
1903
|
+
- mandatory facts preserved
|
|
1904
|
+
- no forbidden invention
|
|
1905
|
+
- deterministic and idempotent
|
|
1906
|
+
- at least 15% and 1 KiB smaller
|
|
1907
|
+
|
|
|
1908
|
+
+-- invariant failure --> exact raw passthrough + session circuit breaker
|
|
1909
|
+
|
|
|
1910
|
+
v
|
|
1911
|
+
exact raw response written to bounded ephemeral recovery store
|
|
1912
|
+
|
|
|
1913
|
+
+-- recovery unavailable --> exact raw passthrough
|
|
1914
|
+
|
|
|
1915
|
+
v
|
|
1916
|
+
hookSpecificOutput.updatedToolOutput
|
|
1917
|
+
```
|
|
1918
|
+
|
|
1919
|
+
### Module layout
|
|
1920
|
+
|
|
1921
|
+
```text
|
|
1922
|
+
scripts/
|
|
1923
|
+
tool_output_filter/
|
|
1924
|
+
__init__.py
|
|
1925
|
+
contracts.py
|
|
1926
|
+
engine.py
|
|
1927
|
+
hook_runtime.py
|
|
1928
|
+
input.py
|
|
1929
|
+
policy.py
|
|
1930
|
+
invariants.py
|
|
1931
|
+
recovery.py
|
|
1932
|
+
telemetry.py
|
|
1933
|
+
profiles/
|
|
1934
|
+
__init__.py
|
|
1935
|
+
repeat_lines.py
|
|
1936
|
+
tap_success.py
|
|
1937
|
+
output_filter_hook.py
|
|
1938
|
+
output_filter_cli.py
|
|
1939
|
+
benchmark_output_filter.py
|
|
1940
|
+
|
|
1941
|
+
app/
|
|
1942
|
+
hooks/
|
|
1943
|
+
filter-tool-output.sh
|
|
1944
|
+
session-end.sh
|
|
1945
|
+
hooks.json
|
|
1946
|
+
output-filter-policy.json
|
|
1947
|
+
|
|
1948
|
+
tests/
|
|
1949
|
+
fixtures/output-filter/
|
|
1950
|
+
repeat-basic/
|
|
1951
|
+
tap-basic/
|
|
1952
|
+
tap-diagnostic/
|
|
1953
|
+
ansi-adversarial/
|
|
1954
|
+
test_tool_output_filter.py
|
|
1955
|
+
test_tool_output_filter_cli.py
|
|
1956
|
+
test_tool_output_filter_properties.py
|
|
1957
|
+
test_tool_output_filter_recovery.py
|
|
1958
|
+
test_tool_output_filter_benchmark.py
|
|
1959
|
+
test_output_filter_config.bats
|
|
1960
|
+
test_output_filter_hook.bats
|
|
1961
|
+
|
|
1962
|
+
kb/reference/
|
|
1963
|
+
tool-output-filter.md
|
|
1964
|
+
```
|
|
1965
|
+
|
|
1966
|
+
### Configuration contract
|
|
1967
|
+
|
|
1968
|
+
```json
|
|
1969
|
+
{
|
|
1970
|
+
"toolOutputFilter": {
|
|
1971
|
+
"mode": "off",
|
|
1972
|
+
"profiles": ["repeat-lines", "tap-success"],
|
|
1973
|
+
"maxInputBytes": 8388608,
|
|
1974
|
+
"minSavingsBytes": 1024,
|
|
1975
|
+
"minSavingsRatio": 0.15,
|
|
1976
|
+
"recovery": {
|
|
1977
|
+
"mode": "ephemeral",
|
|
1978
|
+
"ttlMinutes": 60,
|
|
1979
|
+
"maxSessionBytes": 33554432
|
|
1980
|
+
}
|
|
1981
|
+
}
|
|
1982
|
+
}
|
|
1983
|
+
```
|
|
1984
|
+
|
|
1985
|
+
- `off`: no Python invocation and no metrics.
|
|
1986
|
+
- `observe`: calculate an eligible result and metadata, but return no hook
|
|
1987
|
+
output, so the model receives the original bytes.
|
|
1988
|
+
- `safe`: replace only output that passes every invariant and has exact raw
|
|
1989
|
+
recovery available.
|
|
1990
|
+
- `AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1`: global emergency bypass.
|
|
1991
|
+
- Three consecutive runtime or invariant failures disable filtering for the
|
|
1992
|
+
current session and emit one bounded warning.
|
|
1993
|
+
- All profiles remain explicit opt-in until GA evidence is reviewed.
|
|
1994
|
+
|
|
1995
|
+
### Recovery and privacy
|
|
1996
|
+
|
|
1997
|
+
- Store the exact original tool-response object, not a redacted approximation.
|
|
1998
|
+
- Location:
|
|
1999
|
+
`~/.softspark/ai-toolkit/sessions/<repo-key>/output-filter/<session>/<opaque-id>`.
|
|
2000
|
+
- Directory mode `0700`, file mode `0600`, exclusive no-follow creation,
|
|
2001
|
+
opaque random identifiers, atomic publication, quota, and TTL.
|
|
2002
|
+
- Never store raw commands, paths, output, environment values, or session IDs
|
|
2003
|
+
in telemetry.
|
|
2004
|
+
- Telemetry contains only profile ID/version, input/output byte and line
|
|
2005
|
+
counts, latency, outcome, and fallback reason.
|
|
2006
|
+
- Session end, explicit clean, uninstall, and expired-TTL cleanup remove only
|
|
2007
|
+
ai-toolkit-owned recovery artifacts.
|
|
2008
|
+
- If the platform cannot provide the secure-store contract, `safe` degrades to
|
|
2009
|
+
raw passthrough. It must not silently become lossy.
|
|
2010
|
+
|
|
2011
|
+
## Success Criteria
|
|
2012
|
+
|
|
2013
|
+
- [x] No new npm, pip, Cargo, system, or runtime dependency.
|
|
2014
|
+
- [x] Shadow/observe mode is byte-identical for 100% of fixtures and native
|
|
2015
|
+
hook-contract test payloads.
|
|
2016
|
+
- [x] Any exception, timeout, unsupported encoding, unknown profile, malformed
|
|
2017
|
+
payload, failed invariant, unavailable recovery store, non-zero exit,
|
|
2018
|
+
signal, or non-empty `stderr` returns the exact original output.
|
|
2019
|
+
- [x] Every omission is marked with profile/version, original and emitted
|
|
2020
|
+
line counts, and an opaque exact-recovery handle.
|
|
2021
|
+
- [x] Golden fixtures preserve 100% of declared mandatory facts and introduce
|
|
2022
|
+
zero non-marker facts.
|
|
2023
|
+
- [x] Filter output is deterministic, idempotent, and never larger than raw.
|
|
2024
|
+
- [x] Eligible fixtures achieve median reduction of at least 30%; replacement
|
|
2025
|
+
requires at least 15% and 1 KiB saved per call.
|
|
2026
|
+
- [x] Hard input cap is 8 MiB; larger payloads pass through without parsing.
|
|
2027
|
+
- [x] Algorithm p95 is at most 20 ms for 100 KiB and 150 ms for 8 MiB; cold
|
|
2028
|
+
end-to-end hook invocation p95 is at most 75 ms on CI reference runners.
|
|
2029
|
+
- [x] Peak traced Python allocation is at most three times input size plus
|
|
2030
|
+
16 MiB.
|
|
2031
|
+
- [x] No regex has unbounded catastrophic backtracking.
|
|
2032
|
+
- [x] Claude Code `safe` mode is opt-in and can be disabled globally without
|
|
2033
|
+
reinstalling.
|
|
2034
|
+
- [x] Additional runtimes activate only after their native result-replacement
|
|
2035
|
+
contract has captured fixtures and passing integration tests.
|
|
2036
|
+
- [x] Full repository validation and test suite are green.
|
|
2037
|
+
|
|
2038
|
+
## Pre-Mortem
|
|
2039
|
+
|
|
2040
|
+
| Risk | Probability | Impact | Mitigation |
|
|
2041
|
+
|---|---|---|---|
|
|
2042
|
+
| A profile removes a discriminating fact | High | High | Mandatory-fact fixtures, adversarial sentinels, raw recovery, one regression disables the profile |
|
|
2043
|
+
| A runtime changes its tool-output schema | Medium | High | Versioned adapters, strict shape validation, unknown shape returns no hook output |
|
|
2044
|
+
| Filtering hides security or failure diagnostics | Medium | High | Explicit exclusion list, successful stdout-only MVP, failures always raw |
|
|
2045
|
+
| Recovery leaks secrets or paths | Medium | High | Exact data only in opt-in ephemeral spool, `0700`/`0600`, no-follow, quota, TTL, complete cleanup |
|
|
2046
|
+
| Hook latency degrades normal tool use | Medium | Medium | Bash fast-path, size threshold, hard cap, benchmark gates, circuit breaker |
|
|
2047
|
+
| Command classification mishandles shell syntax | High | Medium | No command rewriting or execution, conservative allowlist, pipes/redirects bypass |
|
|
2048
|
+
| Regex or parser behavior is locale/version-specific | Medium | High | Independently captured multi-version fixtures; malformed/localized output passes raw |
|
|
2049
|
+
| Concurrent sessions corrupt metrics or recovery | Medium | Medium | Per-session directories, opaque IDs, atomic writes; do not reuse global `session-edits.json` |
|
|
2050
|
+
| Runtime adapters diverge in safety behavior | High | High | Capability matrix and one adapter at a time; no guessed compatibility |
|
|
2051
|
+
| Scope expands into MCP catalog proxying | Medium | Medium | Keep MCP trim as a separate PRD and release boundary |
|
|
2052
|
+
|
|
2053
|
+
## Tasks
|
|
2054
|
+
|
|
2055
|
+
### Phase 1: Observable vertical slice, no output mutation (M)
|
|
2056
|
+
|
|
2057
|
+
Success criteria:
|
|
2058
|
+
|
|
2059
|
+
- Claude PostToolUse payloads are parsed by a native Python stdlib engine.
|
|
2060
|
+
- `off` and `observe` modes cannot change model-visible output.
|
|
2061
|
+
- Independent fixture format and semantic oracle are operational.
|
|
2062
|
+
- No raw tool output or command is persisted.
|
|
2063
|
+
|
|
2064
|
+
Tasks:
|
|
2065
|
+
|
|
2066
|
+
- [x] Define contracts, independent fixture provenance ledger, conservative
|
|
2067
|
+
eligibility rules, and clean implementation boundary.
|
|
2068
|
+
Owner: `infrastructure-architect`.
|
|
2069
|
+
Files: `kb/reference/tool-output-filter.md`,
|
|
2070
|
+
`tests/fixtures/output-filter/README.md`.
|
|
2071
|
+
- [x] Implement pure engine, policy, invariants, `repeat-lines` candidate, and
|
|
2072
|
+
`tap-success` candidate in observe-only mode.
|
|
2073
|
+
Owner: `backend-specialist`.
|
|
2074
|
+
Files: `scripts/tool_output_filter/**`.
|
|
2075
|
+
- [x] Add manual `inspect`, `status`, recovery, and cleanup CLI.
|
|
2076
|
+
Owner: `backend-specialist`.
|
|
2077
|
+
Files: `scripts/output_filter_cli.py`, `bin/ai-toolkit.js`.
|
|
2078
|
+
- [x] Add Claude PostToolUse adapter last in hook order and deploy the Python
|
|
2079
|
+
runtime package with installed hooks.
|
|
2080
|
+
Owner: `command-expert`.
|
|
2081
|
+
Files: `app/hooks/filter-tool-output.sh`, `app/hooks.json`,
|
|
2082
|
+
`scripts/install_steps/hooks.py`.
|
|
2083
|
+
- [x] Add `toolOutputFilter` schema, merge, validation, and effective-policy
|
|
2084
|
+
materialization; fix config-lock source/version/integrity staleness and
|
|
2085
|
+
the Article VII reserved-number mismatch while those files are open.
|
|
2086
|
+
Owner: `backend-specialist`.
|
|
2087
|
+
Files: `scripts/schemas/ai-toolkit-config.schema.json`,
|
|
2088
|
+
`scripts/config_merger.py`, `scripts/config_validator.py`,
|
|
2089
|
+
`scripts/config_lock.py`, `app/output-filter-policy.json`.
|
|
2090
|
+
- [x] Build unit, property, hook-contract, malformed, binary, Unicode,
|
|
2091
|
+
ANSI/OSC, injection, huge-line, and concurrency tests.
|
|
2092
|
+
Owner: `test-engineer`.
|
|
2093
|
+
Files: `tests/test_tool_output_filter*.py`,
|
|
2094
|
+
`tests/test_tool_output_filter*.bats`,
|
|
2095
|
+
`tests/fixtures/output-filter/**`.
|
|
2096
|
+
|
|
2097
|
+
Rollback/scope cut:
|
|
2098
|
+
|
|
2099
|
+
- Remove the owned PostToolUse entry and deployed runtime package.
|
|
2100
|
+
- If native payload fixtures are unstable, ship only the manual inspect CLI
|
|
2101
|
+
and fixture oracle; do not enable the hook.
|
|
2102
|
+
|
|
2103
|
+
### Phase 2: Claude Code safe mode with exact recovery (L)
|
|
2104
|
+
|
|
2105
|
+
Dependency: Phase 1 must pass all observe-mode and performance gates.
|
|
2106
|
+
|
|
2107
|
+
Success criteria:
|
|
2108
|
+
|
|
2109
|
+
- `safe` is explicit opt-in.
|
|
2110
|
+
- Only successful, stdout-only, allowlisted text is replaceable.
|
|
2111
|
+
- Exact raw recovery is available before replacement is emitted.
|
|
2112
|
+
- Every failure path is byte-identical passthrough.
|
|
2113
|
+
|
|
2114
|
+
Tasks:
|
|
2115
|
+
|
|
2116
|
+
- [x] Promote `repeat-lines` and `tap-success` only after mandatory-fact,
|
|
2117
|
+
idempotence, determinism, and minimum-savings gates pass.
|
|
2118
|
+
Owner: `backend-specialist`.
|
|
2119
|
+
Files: `scripts/tool_output_filter/profiles/**`,
|
|
2120
|
+
`scripts/tool_output_filter/invariants.py`.
|
|
2121
|
+
- [x] Implement bounded secure ephemeral recovery and exact `recover`/`clean`
|
|
2122
|
+
CLI operations.
|
|
2123
|
+
Owner: `security-architect`.
|
|
2124
|
+
Files: `scripts/tool_output_filter/recovery.py`,
|
|
2125
|
+
`scripts/output_filter_cli.py`.
|
|
2126
|
+
- [x] Add session circuit breaker, metadata-only telemetry, TTL/quota cleanup,
|
|
2127
|
+
and owned-artifact removal.
|
|
2128
|
+
Owner: `backend-specialist`.
|
|
2129
|
+
Files: `scripts/tool_output_filter/telemetry.py`,
|
|
2130
|
+
`app/hooks/session-end.sh`, `scripts/uninstall.py`.
|
|
2131
|
+
- [x] Verify safety precedence and that filtering cannot emit permission
|
|
2132
|
+
decisions, execute output, or change original tool input/status.
|
|
2133
|
+
Owner: `security-auditor`.
|
|
2134
|
+
Files: `tests/test_tool_output_filter.py`,
|
|
2135
|
+
`tests/test_tool_output_filter_recovery.py`,
|
|
2136
|
+
`tests/test_output_filter_hook.bats`.
|
|
2137
|
+
- [x] Add deterministic benchmark corpus and enforce latency, memory, maximum
|
|
2138
|
+
input, and savings thresholds.
|
|
2139
|
+
Owner: `performance-optimizer`.
|
|
2140
|
+
Files: `scripts/benchmark_output_filter.py`,
|
|
2141
|
+
`benchmarks/output-filter/**`.
|
|
2142
|
+
|
|
2143
|
+
Rollback/scope cut:
|
|
2144
|
+
|
|
2145
|
+
- Global disable returns the hook to observe/raw behavior immediately.
|
|
2146
|
+
- Per-profile disable preserves other validated profiles.
|
|
2147
|
+
- If exact secure recovery is unavailable on a platform, that platform stays
|
|
2148
|
+
in `observe`; no weaker recovery implementation is accepted.
|
|
2149
|
+
- If TAP cannot meet semantic gates, release only `repeat-lines`.
|
|
2150
|
+
|
|
2151
|
+
### Phase 3: Capability-gated runtime expansion (L)
|
|
2152
|
+
|
|
2153
|
+
Dependency: Phase 2 must complete Claude safe-mode dogfooding without a
|
|
2154
|
+
confirmed semantic loss.
|
|
2155
|
+
|
|
2156
|
+
Success criteria:
|
|
2157
|
+
|
|
2158
|
+
- Each runtime has a documented capability: native replacement, observe-only,
|
|
2159
|
+
manual CLI, or unsupported.
|
|
2160
|
+
- No adapter uses command rewriting or `additionalContext` as fake replacement.
|
|
2161
|
+
- Each active adapter has captured native payload/output fixtures.
|
|
2162
|
+
|
|
2163
|
+
Tasks:
|
|
2164
|
+
|
|
2165
|
+
- [x] Build and document the capability matrix from current runtime contracts.
|
|
2166
|
+
Owner: `technical-researcher`.
|
|
2167
|
+
Files: `kb/reference/tool-output-filter.md`,
|
|
2168
|
+
`kb/reference/supported-tools-registry.md`.
|
|
2169
|
+
- [x] Classify adapters one by one and keep OpenCode, Cursor, and the remaining
|
|
2170
|
+
runtimes manual-only because no independently verified native
|
|
2171
|
+
result-replacement contract passed the release boundary.
|
|
2172
|
+
Owner: `command-expert`.
|
|
2173
|
+
Files: `scripts/generate_opencode_plugin.py`,
|
|
2174
|
+
`scripts/generate_cursor_hooks.py`, runtime-specific fixture files.
|
|
2175
|
+
- [x] Fix ignored OpenCode guard block propagation before adding its filter
|
|
2176
|
+
adapter.
|
|
2177
|
+
Owner: `backend-specialist`.
|
|
2178
|
+
Files: `scripts/generate_opencode_plugin.py`,
|
|
2179
|
+
`tests/test_opencode*.bats`.
|
|
2180
|
+
- [x] Fix Gemini invalid-settings overwrite and non-atomic write before any
|
|
2181
|
+
Gemini filter adapter is attempted.
|
|
2182
|
+
Owner: `backend-specialist`.
|
|
2183
|
+
Files: `scripts/generate_gemini_hooks.py`,
|
|
2184
|
+
`tests/test_gemini.bats`.
|
|
2185
|
+
- [x] Add per-runtime uninstall, rollback, user-hook preservation, failure, and
|
|
2186
|
+
schema-drift tests.
|
|
2187
|
+
Owner: `test-engineer`.
|
|
2188
|
+
Files: `tests/test_hooks_per_editor.bats`,
|
|
2189
|
+
`tests/test_*hooks*.bats`.
|
|
2190
|
+
|
|
2191
|
+
Rollback/scope cut:
|
|
2192
|
+
|
|
2193
|
+
- A runtime without safe native replacement remains manual CLI or
|
|
2194
|
+
observe-only.
|
|
2195
|
+
- Any runtime-specific regression removes only its owned adapter.
|
|
2196
|
+
- Cross-runtime uniformity is not a release requirement.
|
|
2197
|
+
|
|
2198
|
+
### Phase 4: Documentation and GA decision (M)
|
|
2199
|
+
|
|
2200
|
+
Dependency: Phases 1 and 2 are mandatory; Phase 3 may remain partial.
|
|
2201
|
+
|
|
2202
|
+
Success criteria:
|
|
2203
|
+
|
|
2204
|
+
- Dogfood evidence distinguishes eligible-output byte savings from actual
|
|
2205
|
+
session token receipts.
|
|
2206
|
+
- Documentation lists exclusions, recovery sensitivity, and bypass procedure.
|
|
2207
|
+
- `safe` remains opt-in unless a separate user-approved promotion decision is
|
|
2208
|
+
made.
|
|
2209
|
+
|
|
2210
|
+
Tasks:
|
|
2211
|
+
|
|
2212
|
+
- [x] Review available shadow/safe evidence, false-positive and
|
|
2213
|
+
circuit-breaker paths, and latency. No production Claude JSONL receipt
|
|
2214
|
+
evidence was collected in this implementation run, so documentation
|
|
2215
|
+
makes no token-savings claim and `safe` remains opt-in.
|
|
2216
|
+
Owner: `data-analyst`.
|
|
2217
|
+
Files: `kb/reference/tool-output-filter.md`,
|
|
2218
|
+
`kb/reference/stats.md`.
|
|
2219
|
+
- [x] Update all hook counts, architecture references, CLI help, lifecycle
|
|
2220
|
+
catalog, README, changelog, generated instruction surfaces, and release
|
|
2221
|
+
notes.
|
|
2222
|
+
Owner: `documenter`.
|
|
2223
|
+
Files: `README.md`, `CLAUDE.md`, `ARCHITECTURE.md`,
|
|
2224
|
+
`app/ARCHITECTURE.md`, `kb/reference/architecture-overview.md`,
|
|
2225
|
+
`kb/reference/hooks-catalog.md`, `CHANGELOG.md`, `AGENTS.md`,
|
|
2226
|
+
`llms.txt`, `llms-full.txt`, `package.json`, `plugin.json`.
|
|
2227
|
+
- [x] Re-read the complete diff for orphaned references, dead code, missing
|
|
2228
|
+
behavior coverage, stale docs, and unexpected generated changes.
|
|
2229
|
+
Owner: `code-reviewer`.
|
|
2230
|
+
Files: all changed files.
|
|
2231
|
+
|
|
2232
|
+
Rollback/scope cut:
|
|
2233
|
+
|
|
2234
|
+
- Release `observe` and manual CLI only if active-mode evidence is insufficient.
|
|
2235
|
+
- Do not promote any profile to default without separate approval.
|
|
2236
|
+
|
|
2237
|
+
## Dependencies
|
|
2238
|
+
|
|
2239
|
+
```text
|
|
2240
|
+
Phase 1 → Phase 2 → Phase 3 → Phase 4
|
|
2241
|
+
└────────→ Phase 4 if runtime expansion is deferred
|
|
2242
|
+
```
|
|
2243
|
+
|
|
2244
|
+
There are no circular dependencies. Phase 1 is shippable as diagnostics,
|
|
2245
|
+
Phase 2 as Claude-only opt-in filtering, and Phase 3 as incremental
|
|
2246
|
+
runtime-by-runtime support.
|
|
2247
|
+
|
|
2248
|
+
## Verification
|
|
2249
|
+
|
|
2250
|
+
Focused verification:
|
|
2251
|
+
|
|
2252
|
+
1. `python3 -m unittest discover -s tests -p 'test_tool_output_filter*.py'`
|
|
2253
|
+
2. `bats tests/test_output_filter_config.bats tests/test_output_filter_hook.bats`
|
|
2254
|
+
3. `python3 scripts/benchmark_output_filter.py`
|
|
2255
|
+
4. `shellcheck --severity=warning app/hooks/*.sh`
|
|
2256
|
+
|
|
2257
|
+
Repository quality gates:
|
|
2258
|
+
|
|
2259
|
+
1. `npm run generate:all`
|
|
2260
|
+
2. `python3 scripts/validate.py --strict`
|
|
2261
|
+
3. `python3 scripts/audit_skills.py --ci`
|
|
2262
|
+
4. `npm test`
|
|
2263
|
+
5. `git diff --check`
|
|
2264
|
+
6. Re-read `git diff` and confirm no orphaned references, missing tests, stale
|
|
2265
|
+
docs, copied upstream expressions, or new external dependencies.
|
|
2266
|
+
|
|
2267
|
+
Expected result: every command exits `0`, focused semantic fixtures report zero
|
|
2268
|
+
missing mandatory facts and zero forbidden inventions, active replacements meet
|
|
2269
|
+
the savings threshold, and all unsafe paths remain exact passthrough.
|
|
2270
|
+
|
|
2271
|
+
## Agent Assignments
|
|
2272
|
+
|
|
2273
|
+
| Responsibility | Agent | Model |
|
|
2274
|
+
|---|---|---|
|
|
2275
|
+
| Architecture and contracts | `infrastructure-architect` | Assigned tier, no override |
|
|
2276
|
+
| Python engine and configuration | `backend-specialist` | Assigned tier, no override |
|
|
2277
|
+
| Hook and runtime adapters | `command-expert` | Assigned tier, no override |
|
|
2278
|
+
| Recovery threat model | `security-architect` | Assigned tier, no override |
|
|
2279
|
+
| Security verification | `security-auditor` | Assigned tier, no override |
|
|
2280
|
+
| Semantic and integration tests | `test-engineer` | Assigned tier, no override |
|
|
2281
|
+
| Performance gates | `performance-optimizer` | Assigned tier, no override |
|
|
2282
|
+
| Runtime contract research | `technical-researcher` | Assigned tier, no override |
|
|
2283
|
+
| Evidence analysis | `data-analyst` | Assigned tier, no override |
|
|
2284
|
+
| Documentation | `documenter` | Assigned tier, no override |
|
|
2285
|
+
| Final diff review | `code-reviewer` | Assigned tier, no override |
|
|
2286
|
+
|
|
2287
|
+
## Independent-Implementation Boundary
|
|
2288
|
+
|
|
2289
|
+
- Implement from this plan, local ai-toolkit requirements, native runtime
|
|
2290
|
+
contracts, and independently captured command outputs.
|
|
2291
|
+
- Do not consult or copy RTK source while implementing filters.
|
|
2292
|
+
- Do not copy or transliterate identifiers, control flow, comments, regex
|
|
2293
|
+
tables, filter order, fixtures, messages, configuration keys, default values,
|
|
2294
|
+
benchmarks, or documentation wording.
|
|
2295
|
+
- Record the origin and rationale of each filter rule in the fixture provenance
|
|
2296
|
+
ledger.
|
|
2297
|
+
- If any upstream expression is intentionally adapted, stop and perform an
|
|
2298
|
+
Apache-2.0 attribution and NOTICE review before continuing.
|
|
2299
|
+
|
|
2300
|
+
---
|
|
2301
|
+
|
|
1777
2302
|
## kb/history/completed/offline-slm-profile-plan-20260411.md
|
|
1778
2303
|
|
|
1779
2304
|
---
|
|
@@ -4999,9 +5524,9 @@ title: "SOP: Release Preparation"
|
|
|
4999
5524
|
category: procedures
|
|
5000
5525
|
service: ai-toolkit
|
|
5001
5526
|
tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
|
|
5002
|
-
version: "1.11.
|
|
5527
|
+
version: "1.11.1"
|
|
5003
5528
|
created: "2026-04-10"
|
|
5004
|
-
last_updated: "2026-
|
|
5529
|
+
last_updated: "2026-07-24"
|
|
5005
5530
|
description: "Step-by-step checklist for preparing a new ai-toolkit release — ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, and checksum-pin checks added in v2.8.0, the single-run npm test discipline added in v1.8.0, the ecosystem-sync gate added in v1.9.0, the registry-vs-generators drift gate added in v1.10.0, and the mandatory pre-tag ShellCheck gate added in v1.11.0 (publish.yml does not run ShellCheck, so a hook lint failure can publish while reddening main CI — see the v4.5.1 postmortem in Phase 5)."
|
|
5006
5531
|
---
|
|
5007
5532
|
|
|
@@ -5331,9 +5856,10 @@ grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
|
|
|
5331
5856
|
**2. URL-sourced rules and hooks are checksum-pinned:**
|
|
5332
5857
|
|
|
5333
5858
|
```bash
|
|
5334
|
-
# On a machine that has consumed URL rules/hooks at least once
|
|
5335
|
-
|
|
5336
|
-
jq 'to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/
|
|
5859
|
+
# On a machine that has consumed URL rules/hooks at least once
|
|
5860
|
+
# (schema_version 1: entries live under the .rules / .hooks key):
|
|
5861
|
+
jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
|
|
5862
|
+
jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
|
|
5337
5863
|
```
|
|
5338
5864
|
|
|
5339
5865
|
- [ ] Both queries return empty arrays (every URL entry has a `sha256`)
|
|
@@ -6318,9 +6844,9 @@ title: "AI Toolkit - Architecture Overview"
|
|
|
6318
6844
|
category: reference
|
|
6319
6845
|
service: ai-toolkit
|
|
6320
6846
|
tags: [architecture, overview, design, structure]
|
|
6321
|
-
version: "1.
|
|
6847
|
+
version: "1.5.0"
|
|
6322
6848
|
created: "2026-03-23"
|
|
6323
|
-
last_updated: "2026-07-
|
|
6849
|
+
last_updated: "2026-07-23"
|
|
6324
6850
|
description: "Architecture of ai-toolkit: directory layout, Claude app export, global install model, editor-aware MCP install, Codex translation layer, skill tiers, and integration with projects."
|
|
6325
6851
|
---
|
|
6326
6852
|
|
|
@@ -6600,7 +7126,7 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
|
|
|
6600
7126
|
|
|
6601
7127
|
## Quality Hooks
|
|
6602
7128
|
|
|
6603
|
-
|
|
7129
|
+
29 entries across 14 lifecycle events. See [hooks-catalog.md](hooks-catalog.md) for full details.
|
|
6604
7130
|
|
|
6605
7131
|
| Hook | Trigger | Script | Action |
|
|
6606
7132
|
|------|---------|--------|--------|
|
|
@@ -6609,12 +7135,13 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
|
|
|
6609
7135
|
| Notification | Claude waiting for input | *(inline)* | macOS desktop notification |
|
|
6610
7136
|
| PreToolUse | Before Bash | `guard-destructive.sh` | Block destructive commands |
|
|
6611
7137
|
| PreToolUse | Before file ops (Bash, Read, Edit, Write, MultiEdit, Glob, Grep, NotebookEdit, mcp\_filesystem) | `guard-path.sh` | Block wrong-user path hallucination |
|
|
6612
|
-
| PreToolUse | Before Edit/Write/MultiEdit | `guard-config.sh` |
|
|
7138
|
+
| PreToolUse | Before Edit/Write/MultiEdit | `guard-config.sh` | Always block protected config edits; remove the hook deliberately when a change is authorized |
|
|
6613
7139
|
| PreToolUse | Before Bash (git commit) | `commit-quality.sh` | Advisory Conventional Commits format check |
|
|
6614
7140
|
| UserPromptSubmit | Before user prompt execution | `user-prompt-submit.sh` | Prompt governance reminder |
|
|
6615
7141
|
| UserPromptSubmit | Before user prompt execution | `track-usage.sh` | Record skill invocations to stats.json |
|
|
6616
7142
|
| PostToolUse | After edit/write tools | `post-tool-use.sh` | Lightweight validation reminders |
|
|
6617
7143
|
| PostToolUse | After any tool | `governance-capture.sh` | Log security-sensitive operations |
|
|
7144
|
+
| PostToolUse | After successful Bash | `filter-tool-output.sh` | Observe eligible text or replace it only after exact recovery succeeds; Claude Code only |
|
|
6618
7145
|
| Stop | After response | `quality-check.sh` | Multi-language lint |
|
|
6619
7146
|
| Stop | After response | `save-session.sh` | Persist session context |
|
|
6620
7147
|
| Stop | Before final stop | `quality-gate.sh` | Block final response on lint/type errors |
|
|
@@ -6624,7 +7151,7 @@ Agents (code-reviewer, debugger, devops-implementer, ...)
|
|
|
6624
7151
|
| SubagentStop | Subagent completion | `subagent-stop.sh` | Handoff checklist for subagents |
|
|
6625
7152
|
| PreCompact | Before compaction | `pre-compact.sh` | Save prioritized context: instincts > tasks > git state > decisions |
|
|
6626
7153
|
| PreCompact | Before compaction | `pre-compact-save.sh` | Timestamped context snapshot to audit trail |
|
|
6627
|
-
| SessionEnd | Session end | `session-end.sh` |
|
|
7154
|
+
| SessionEnd | Session end | `session-end.sh` | Clean owned output recovery and persist the next-session handoff note |
|
|
6628
7155
|
|
|
6629
7156
|
Scripts at `~/.softspark/ai-toolkit/hooks/`. See [hooks-catalog.md](hooks-catalog.md) for details.
|
|
6630
7157
|
|
|
@@ -7031,9 +7558,9 @@ The reference benchmark set is intentionally curated:
|
|
|
7031
7558
|
title: "CLI Reference"
|
|
7032
7559
|
category: reference
|
|
7033
7560
|
service: ai-toolkit
|
|
7034
|
-
tags: [cli, commands, reference, install, update, plugin, mcp, telemetry]
|
|
7561
|
+
tags: [cli, commands, reference, install, update, plugin, mcp, telemetry, output-filter]
|
|
7035
7562
|
created: "2026-04-13"
|
|
7036
|
-
last_updated: "2026-07-
|
|
7563
|
+
last_updated: "2026-07-23"
|
|
7037
7564
|
description: "Complete CLI reference for all ai-toolkit commands, options, and flags."
|
|
7038
7565
|
---
|
|
7039
7566
|
|
|
@@ -7104,6 +7631,23 @@ Usage: ai-toolkit <command> [options]
|
|
|
7104
7631
|
| `config create-base <name>` | Scaffold base config npm package |
|
|
7105
7632
|
| `config check [path]` | CI enforcement gate (exit 0=pass, 1=fail, 2=no config; `--json`) |
|
|
7106
7633
|
|
|
7634
|
+
## Native Tool-Output Filter
|
|
7635
|
+
|
|
7636
|
+
| Command | Description |
|
|
7637
|
+
|---------|-------------|
|
|
7638
|
+
| `output-filter status [--policy PATH]` | Print the effective trusted project/global policy, or inspect an explicit policy |
|
|
7639
|
+
| `output-filter inspect --profile repeat-lines\|tap-success` | Read stdin and print content-free candidate metadata without changing or echoing the input |
|
|
7640
|
+
| `output-filter recover <handle> [--base-directory PATH] [--session-id ID]` | Print the exact stored native response object for an opaque recovery handle |
|
|
7641
|
+
| `output-filter clean [--base-directory PATH] [--session-id ID]` | Remove owned filter artifacts for one session or all sessions in the current repository |
|
|
7642
|
+
| `output-filter clean --session-id ID --expired` | Remove expired exact-response artifacts for one native session |
|
|
7643
|
+
| `output-filter hook --policy PATH` | **Internal, not for manual use.** `PostToolUse` entry point that reads the native hook payload on stdin and emits any replacement. The installed hook `filter-tool-output.sh` calls the same runtime through the lean `output_filter_hook.py` process instead |
|
|
7644
|
+
|
|
7645
|
+
`off` is the default. Configure `toolOutputFilter.mode` in
|
|
7646
|
+
`.softspark-toolkit.json`, then run `install --local` or `update --local`.
|
|
7647
|
+
`AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1` bypasses active filtering immediately.
|
|
7648
|
+
See [Native Tool Output Filter](tool-output-filter.md) for eligibility,
|
|
7649
|
+
privacy, recovery, and runtime support.
|
|
7650
|
+
|
|
7107
7651
|
## Project Registry
|
|
7108
7652
|
|
|
7109
7653
|
| Command | Description |
|
|
@@ -7421,6 +7965,7 @@ The Codex compatibility path is verified by:
|
|
|
7421
7965
|
- `kb/reference/skills-catalog.md`
|
|
7422
7966
|
- `kb/reference/architecture-overview.md`
|
|
7423
7967
|
- `kb/reference/global-install-model.md`
|
|
7968
|
+
- `kb/reference/copilot-compatibility.md`
|
|
7424
7969
|
|
|
7425
7970
|
---
|
|
7426
7971
|
|
|
@@ -8139,6 +8684,184 @@ None — all features are independent of external systems.
|
|
|
8139
8684
|
|
|
8140
8685
|
---
|
|
8141
8686
|
|
|
8687
|
+
## kb/reference/copilot-compatibility.md
|
|
8688
|
+
|
|
8689
|
+
---
|
|
8690
|
+
title: "AI Toolkit - GitHub Copilot Compatibility"
|
|
8691
|
+
category: reference
|
|
8692
|
+
service: ai-toolkit
|
|
8693
|
+
tags: [copilot, compatibility, install, skills, prompts, instructions, agents, hooks]
|
|
8694
|
+
version: "1.0.0"
|
|
8695
|
+
created: "2026-07-15"
|
|
8696
|
+
last_updated: "2026-07-15"
|
|
8697
|
+
description: "Reference for how ai-toolkit integrates with GitHub Copilot — the five .github/ surfaces, their runtime context-loading semantics, and why the same skill is emitted as both a prompt file and a skill directory."
|
|
8698
|
+
---
|
|
8699
|
+
|
|
8700
|
+
# AI Toolkit - GitHub Copilot Compatibility
|
|
8701
|
+
|
|
8702
|
+
## Summary
|
|
8703
|
+
|
|
8704
|
+
`ai-toolkit install --local --editors copilot` emits native GitHub Copilot
|
|
8705
|
+
customization files under `.github/`. Copilot is the toolkit's widest-surface
|
|
8706
|
+
editor target: a single skill can materialize into up to three distinct Copilot
|
|
8707
|
+
mechanisms (path-scoped instruction, invokable prompt, on-demand skill). All
|
|
8708
|
+
emitted paths are generated build artifacts and are `.gitignore`d — they are not
|
|
8709
|
+
committed source. This document explains what each surface is, **when each one
|
|
8710
|
+
enters the model context**, and why the same skill body appears in more than one
|
|
8711
|
+
file. Install/profile behavior is owned by `kb/reference/global-install-model.md`;
|
|
8712
|
+
config-path and capability tracking by `kb/reference/supported-tools-registry.md`.
|
|
8713
|
+
|
|
8714
|
+
## Generated Surfaces
|
|
8715
|
+
|
|
8716
|
+
| Surface | Path | Copilot mechanism |
|
|
8717
|
+
|---------|------|-------------------|
|
|
8718
|
+
| Repo-wide instructions | `.github/copilot-instructions.md` | Always-on repository custom instructions |
|
|
8719
|
+
| Path-scoped instructions | `.github/instructions/ai-toolkit-*.instructions.md` | Custom instructions gated by an `applyTo` glob |
|
|
8720
|
+
| Prompt files | `.github/prompts/ai-toolkit-*.prompt.md` | Reusable prompts, invoked as `/name` |
|
|
8721
|
+
| Native agents | `.github/agents/ai-toolkit-*.agent.md` | Custom agents in the agent picker |
|
|
8722
|
+
| Portable skills | `.github/skills/ai-toolkit-*/SKILL.md` (+ `reference/`, `scripts/`) | Agent Skills, injected on demand |
|
|
8723
|
+
| Lifecycle hooks | `.github/hooks/ai-toolkit.json` + runtime | Version-1 Copilot hooks (profile ≥ `standard`) |
|
|
8724
|
+
| Shared rules | root `AGENTS.md` | Read by Copilot code review and CLI |
|
|
8725
|
+
|
|
8726
|
+
## Surface Loading Semantics
|
|
8727
|
+
|
|
8728
|
+
The five customization surfaces do **not** all cost context the same way. This
|
|
8729
|
+
is the practical difference that governs token usage and any perceived
|
|
8730
|
+
"double loading":
|
|
8731
|
+
|
|
8732
|
+
| Surface | Enters context… | Passive cost |
|
|
8733
|
+
|---------|-----------------|--------------|
|
|
8734
|
+
| `copilot-instructions.md` | Every chat request in the repo | Always-on |
|
|
8735
|
+
| `*.instructions.md` with `applyTo: "**"` | Every request (glob matches all files) | Effectively always-on |
|
|
8736
|
+
| `*.instructions.md` with a scoped glob (e.g. `**/*.py`) | Only when a matching file is in context | Path-scoped |
|
|
8737
|
+
| `*.prompt.md` | **Only** when the user runs `/<name>` | None until invoked |
|
|
8738
|
+
| `SKILL.md` | **Only** when Copilot chooses to use the skill (progressive disclosure) | None until triggered |
|
|
8739
|
+
| `*.agent.md` | Only when that agent is selected in the picker | None until selected |
|
|
8740
|
+
|
|
8741
|
+
Instructions are auto-added to requests as soon as their `applyTo` glob matches;
|
|
8742
|
+
skills and prompts are pull-based, never injected passively.
|
|
8743
|
+
|
|
8744
|
+
## Prompt ↔ Skill Duplication
|
|
8745
|
+
|
|
8746
|
+
Every user-invocable skill is emitted **both** as a `.github/prompts/*.prompt.md`
|
|
8747
|
+
(so it is available as a `/slash-command`) **and** as a
|
|
8748
|
+
`.github/skills/<name>/SKILL.md` directory (so Copilot can auto-trigger it with
|
|
8749
|
+
its bundled `scripts/` and `reference/` assets). The two bodies are byte-identical
|
|
8750
|
+
after their frontmatter and Copilot execution-notes header; the differences are:
|
|
8751
|
+
|
|
8752
|
+
- the prompt carries only a `description`; the skill adds `name` and bundles the
|
|
8753
|
+
runnable assets the prompt does not ship;
|
|
8754
|
+
- the prompt's execution note treats the current request as task input, the
|
|
8755
|
+
skill's note resolves relative paths against its own directory.
|
|
8756
|
+
|
|
8757
|
+
**This is intentional, not a stale leftover, and does not cause a persistent
|
|
8758
|
+
double-load:**
|
|
8759
|
+
|
|
8760
|
+
- Neither surface is always-on. The prompt loads only on explicit `/invoke`; the
|
|
8761
|
+
skill loads only on trigger. Neither sits in `copilot-instructions.md` or in an
|
|
8762
|
+
always-matching `applyTo`, so the shared body carries **zero** passive context
|
|
8763
|
+
cost.
|
|
8764
|
+
- The same skill is registered from a **single** root (`.github/skills`), not
|
|
8765
|
+
duplicated across `.claude/skills` or `.agents/skills` in the same repo, so
|
|
8766
|
+
there is no duplicate skill registration.
|
|
8767
|
+
- The one edge case is a single turn where the user runs `/<name>` **and** Copilot
|
|
8768
|
+
autonomously pulls the matching skill in the same request. That is a one-shot
|
|
8769
|
+
redundancy of identical text — wasted tokens for that turn only, with no
|
|
8770
|
+
conflicting instructions and no persistent effect.
|
|
8771
|
+
|
|
8772
|
+
Neither surface can replace the other: the prompt has no bundled scanner or
|
|
8773
|
+
reference material, and the skill directory is not exposed as a slash command.
|
|
8774
|
+
The body is duplicated so each surface is self-contained.
|
|
8775
|
+
|
|
8776
|
+
## Compatibility Read Paths
|
|
8777
|
+
|
|
8778
|
+
Copilot also discovers project `.claude/skills` and `.agents/skills`, and
|
|
8779
|
+
personal `~/.agents/skills`. ai-toolkit still materializes self-contained native
|
|
8780
|
+
skills under `.github/skills` (and under the active Copilot config root for
|
|
8781
|
+
global installs) so that bundled assets and helper scripts remain available and
|
|
8782
|
+
`COPILOT_HOME` sessions do not depend on fallback discovery. The toolkit does not
|
|
8783
|
+
write the same skill into two roots at once, so fallback discovery never produces
|
|
8784
|
+
a duplicate registration.
|
|
8785
|
+
|
|
8786
|
+
## Generated, Git-Ignored Artifacts
|
|
8787
|
+
|
|
8788
|
+
All Copilot outputs are build products, regenerated on every install/generate,
|
|
8789
|
+
and listed in `.gitignore`:
|
|
8790
|
+
|
|
8791
|
+
- `.github/copilot-instructions.md`
|
|
8792
|
+
- `.github/instructions/`
|
|
8793
|
+
- `.github/prompts/`
|
|
8794
|
+
- `.github/agents/`
|
|
8795
|
+
- `.github/skills/`
|
|
8796
|
+
- `.github/hooks/`
|
|
8797
|
+
|
|
8798
|
+
Deleting them locally is safe (they are untracked and ignored); the next
|
|
8799
|
+
`ai-toolkit install --editors copilot` or generator run recreates them. The
|
|
8800
|
+
generator also cleans stale managed entries and byte-exact historical ai-toolkit
|
|
8801
|
+
files while preserving user-authored files.
|
|
8802
|
+
|
|
8803
|
+
## Install & Profiles
|
|
8804
|
+
|
|
8805
|
+
Authoritative behavior lives in `kb/reference/global-install-model.md`. In brief:
|
|
8806
|
+
|
|
8807
|
+
- **All profiles (including `minimal`)** emit root `AGENTS.md`,
|
|
8808
|
+
`.github/copilot-instructions.md`, native `.github/agents`, and portable
|
|
8809
|
+
self-contained `.github/skills`.
|
|
8810
|
+
- **`standard`, `strict`, `full`** additionally emit scoped `.github/instructions`,
|
|
8811
|
+
`.github/prompts`, and native version-1 `.github/hooks`.
|
|
8812
|
+
- **`minimal`** omits instructions, prompts, and hooks.
|
|
8813
|
+
|
|
8814
|
+
Moving an existing project down to `minimal` removes only marked or byte-exact
|
|
8815
|
+
historical ai-toolkit instructions/prompts/hooks; unmanaged project files stay.
|
|
8816
|
+
|
|
8817
|
+
## Auto-Detection
|
|
8818
|
+
|
|
8819
|
+
The installer treats Copilot as configured when any of these markers exist:
|
|
8820
|
+
`.github/copilot-instructions.md`, `.github/instructions`, `.github/prompts`,
|
|
8821
|
+
`.github/agents`, `.github/skills`, `.github/hooks`, `.github/mcp.json`.
|
|
8822
|
+
`ai-toolkit update` then picks up Copilot automatically.
|
|
8823
|
+
|
|
8824
|
+
## Generators & CLI
|
|
8825
|
+
|
|
8826
|
+
Copilot has no dedicated `ai-toolkit copilot-*` subcommand; it is produced by
|
|
8827
|
+
`ai-toolkit install --editors copilot` (add `--local` for project scope) or by
|
|
8828
|
+
running the generators directly:
|
|
8829
|
+
|
|
8830
|
+
- `scripts/generate_copilot.py` — instructions, prompts, native agents, and
|
|
8831
|
+
portable skill directories (`> .github/copilot-instructions.md` with no target
|
|
8832
|
+
argument; multi-surface emission with a target directory).
|
|
8833
|
+
- `scripts/generate_copilot_hooks.py` — version-1 hook config plus a
|
|
8834
|
+
self-contained repository/config-root runtime.
|
|
8835
|
+
|
|
8836
|
+
## Behavioral Limits
|
|
8837
|
+
|
|
8838
|
+
- Prompt files are available only in VS Code, Visual Studio, and JetBrains IDEs;
|
|
8839
|
+
GitHub.com and the CLI use instructions, skills, agents, and `AGENTS.md`.
|
|
8840
|
+
- Custom agents emit native `.agent.md` with `name` and `description`; `tools` is
|
|
8841
|
+
omitted rather than guessing editor-specific aliases.
|
|
8842
|
+
- Prompt and skill bodies strip Claude-only interpolation (`$ARGUMENTS`,
|
|
8843
|
+
`CLAUDE_SKILL_DIR`) and delegation APIs; hooks use the GitHub version-1 schema
|
|
8844
|
+
with camelCase event names.
|
|
8845
|
+
|
|
8846
|
+
## Verification
|
|
8847
|
+
|
|
8848
|
+
- `scripts/generate_copilot.py` / `generate_copilot_hooks.py` contract tests
|
|
8849
|
+
(`tests/test_copilot.bats`, `tests/test_copilot_hooks.bats`).
|
|
8850
|
+
- Release layout check in `kb/procedures/release-verification-sop.md` asserts the
|
|
8851
|
+
`.github/{agents,skills,instructions,prompts,hooks}` surfaces.
|
|
8852
|
+
- `validate.py --strict` + `audit_skills.py --ci` in CI.
|
|
8853
|
+
|
|
8854
|
+
## Related
|
|
8855
|
+
|
|
8856
|
+
- `kb/reference/global-install-model.md`
|
|
8857
|
+
- `kb/reference/supported-tools-registry.md`
|
|
8858
|
+
- `kb/reference/codex-cli-compatibility.md`
|
|
8859
|
+
- `kb/reference/opencode-compatibility.md`
|
|
8860
|
+
- `kb/reference/skills-catalog.md`
|
|
8861
|
+
- `kb/reference/agents-catalog.md`
|
|
8862
|
+
|
|
8863
|
+
---
|
|
8864
|
+
|
|
8142
8865
|
## kb/reference/distribution-model.md
|
|
8143
8866
|
|
|
8144
8867
|
---
|
|
@@ -8221,7 +8944,7 @@ tags:
|
|
|
8221
8944
|
- multi-repo
|
|
8222
8945
|
doc_type: reference
|
|
8223
8946
|
created: "2026-04-11"
|
|
8224
|
-
last_updated: "2026-
|
|
8947
|
+
last_updated: "2026-07-23"
|
|
8225
8948
|
description: "Comprehensive guide for setting up and using ai-toolkit configuration inheritance. Covers base config creation, project setup, enforcement rules, CI integration, and troubleshooting."
|
|
8226
8949
|
---
|
|
8227
8950
|
|
|
@@ -8288,7 +9011,9 @@ ai-toolkit config check # CI enforcement check
|
|
|
8288
9011
|
|-------|------|-------------|
|
|
8289
9012
|
| `extends` | string | Base config source (npm, git URL, local path) |
|
|
8290
9013
|
| `profile` | enum | `minimal`, `standard`, `strict`, `full`, `offline-slm` |
|
|
9014
|
+
| `toolOutputFilter` | object | Native Bash result filtering policy: mode, profiles, savings thresholds, and recovery limits |
|
|
8291
9015
|
| `agents` | object | `enabled`, `disabled`, `custom` arrays |
|
|
9016
|
+
| `plugins` | object | Resolved `enabled` and `disabled` plugin intent |
|
|
8292
9017
|
| `rules` | object | `inject`, `remove` arrays |
|
|
8293
9018
|
| `constitution` | object | `amendments` array (article 8+ only) |
|
|
8294
9019
|
| `enforce` | object | Non-overridable constraints (base configs only) |
|
|
@@ -8357,10 +9082,34 @@ Base configs can define non-overridable constraints via the `enforce` block:
|
|
|
8357
9082
|
| Constraint | Effect |
|
|
8358
9083
|
|------------|--------|
|
|
8359
9084
|
| `minHookProfile` | Projects cannot use a weaker hook profile |
|
|
8360
|
-
| `requiredPlugins` |
|
|
9085
|
+
| `requiredPlugins` | Adds each required name to the effective `plugins.enabled` intent and prevents projects from disabling it |
|
|
8361
9086
|
| `forbidOverride` | These components cannot be overridden |
|
|
8362
9087
|
| `requiredAgents` | Must be enabled in all projects |
|
|
8363
9088
|
|
|
9089
|
+
`requiredPlugins` is configuration enforcement, not an installer. It makes the
|
|
9090
|
+
effective intent explicit and causes validation to reject a conflicting
|
|
9091
|
+
`plugins.disabled` entry. Install the named plugin pack separately with
|
|
9092
|
+
`ai-toolkit plugin install <name>` or your organization deployment workflow.
|
|
9093
|
+
|
|
9094
|
+
### Native tool-output filtering
|
|
9095
|
+
|
|
9096
|
+
Projects can opt into the dependency-free output filter without changing the
|
|
9097
|
+
organization-wide default:
|
|
9098
|
+
|
|
9099
|
+
```json
|
|
9100
|
+
{
|
|
9101
|
+
"toolOutputFilter": {
|
|
9102
|
+
"mode": "observe",
|
|
9103
|
+
"profiles": ["repeat-lines", "tap-success"]
|
|
9104
|
+
}
|
|
9105
|
+
}
|
|
9106
|
+
```
|
|
9107
|
+
|
|
9108
|
+
`ai-toolkit install --local` materializes the effective policy as the managed
|
|
9109
|
+
`.claude/ai-toolkit-output-filter.json` file. `off` is the default, `observe`
|
|
9110
|
+
collects metadata without changing model-visible output, and `safe` permits
|
|
9111
|
+
replacement only when every invariant and exact-recovery check succeeds.
|
|
9112
|
+
|
|
8364
9113
|
### Overrides
|
|
8365
9114
|
|
|
8366
9115
|
Projects can override base settings, but must declare intent:
|
|
@@ -8865,9 +9614,9 @@ title: "Global Install Model"
|
|
|
8865
9614
|
category: reference
|
|
8866
9615
|
service: ai-toolkit
|
|
8867
9616
|
tags: [install, global, claude, codex, plugins, local-setup]
|
|
8868
|
-
version: "3.3.
|
|
9617
|
+
version: "3.3.1"
|
|
8869
9618
|
created: "2026-03-26"
|
|
8870
|
-
last_updated: "2026-07-
|
|
9619
|
+
last_updated: "2026-07-15"
|
|
8871
9620
|
description: "Reference description of Claude Code global install, Claude app plugin export, project-local editor setup, global Codex plugin layering, and command responsibilities in ai-toolkit."
|
|
8872
9621
|
---
|
|
8873
9622
|
|
|
@@ -9027,6 +9776,9 @@ Claude Code hooks do **not** live in project-local settings. They are merged onl
|
|
|
9027
9776
|
self-contained `.github/skills`. Profiles `standard`, `strict`, and `full`
|
|
9028
9777
|
add scoped `.github/instructions`, `.github/prompts`, and native version-1
|
|
9029
9778
|
`.github/hooks`. Profile `minimal` omits those three additional directories.
|
|
9779
|
+
When an existing project moves from `standard`, `strict`, or `full` to
|
|
9780
|
+
`minimal`, the installer removes only marked or byte-exact historical ai-toolkit
|
|
9781
|
+
instructions, prompts, and hooks. Unmanaged project files remain in place.
|
|
9030
9782
|
|
|
9031
9783
|
Global install emits personal instructions, scoped instructions, agents, and
|
|
9032
9784
|
skills under the active Copilot config root; profile `standard` and above adds
|
|
@@ -9085,6 +9837,7 @@ Global-only editor MCP configs are not written during `install --local`. Use `ai
|
|
|
9085
9837
|
- `kb/reference/distribution-model.md`
|
|
9086
9838
|
- `kb/reference/merge-friendly-install-model.md`
|
|
9087
9839
|
- `kb/reference/codex-cli-compatibility.md`
|
|
9840
|
+
- `kb/reference/copilot-compatibility.md`
|
|
9088
9841
|
- `kb/reference/mcp-editor-compatibility.md`
|
|
9089
9842
|
|
|
9090
9843
|
---
|
|
@@ -9301,9 +10054,9 @@ title: "Hooks Catalog"
|
|
|
9301
10054
|
category: reference
|
|
9302
10055
|
service: ai-toolkit
|
|
9303
10056
|
tags: [hooks, quality, safety, enforcement, settings.json]
|
|
9304
|
-
version: "1.
|
|
10057
|
+
version: "1.8.0"
|
|
9305
10058
|
created: "2026-03-27"
|
|
9306
|
-
last_updated: "2026-07-
|
|
10059
|
+
last_updated: "2026-07-23"
|
|
9307
10060
|
description: "Complete reference of all ai-toolkit hooks: events, scripts, installation, and runtime behavior."
|
|
9308
10061
|
---
|
|
9309
10062
|
|
|
@@ -9311,7 +10064,7 @@ description: "Complete reference of all ai-toolkit hooks: events, scripts, insta
|
|
|
9311
10064
|
|
|
9312
10065
|
## Overview
|
|
9313
10066
|
|
|
9314
|
-
ai-toolkit provides
|
|
10067
|
+
ai-toolkit provides 29 global hook entries across 14 lifecycle events that enforce quality, safety, and workflow rules across all Claude Code sessions, plus a separate `statusLine` command. Hooks are merged into `~/.claude/settings.json` on install, with logic in standalone scripts at `~/.softspark/ai-toolkit/hooks/`.
|
|
9315
10068
|
|
|
9316
10069
|
## Supported Surface
|
|
9317
10070
|
|
|
@@ -9327,7 +10080,11 @@ ai-toolkit update # re-copies scripts, re-merges (idempotent)
|
|
|
9327
10080
|
```
|
|
9328
10081
|
|
|
9329
10082
|
**File locations:**
|
|
10083
|
+
|
|
9330
10084
|
- Scripts: `~/.softspark/ai-toolkit/hooks/*.sh`
|
|
10085
|
+
- Output-filter runtime: `~/.softspark/ai-toolkit/scripts/output_filter_hook.py`, `output_filter_cli.py`, and `tool_output_filter/`
|
|
10086
|
+
- Global output-filter policy: `~/.softspark/ai-toolkit/hooks/output-filter-policy.json`
|
|
10087
|
+
- Managed project policy: `<project>/.claude/ai-toolkit-output-filter.json`
|
|
9331
10088
|
- Config: `~/.claude/settings.json` → `hooks` key
|
|
9332
10089
|
- Source: `ai-toolkit/app/hooks/*.sh` + `app/hooks.json`
|
|
9333
10090
|
|
|
@@ -9348,10 +10105,11 @@ ai-toolkit update # re-copies scripts, re-merges (idempotent)
|
|
|
9348
10105
|
3. Loads session context from the per-repo session store (if exists)
|
|
9349
10106
|
4. Loads active instincts from `.claude/instincts/*.md` (if any)
|
|
9350
10107
|
|
|
9351
|
-
By default the hook
|
|
9352
|
-
|
|
9353
|
-
|
|
9354
|
-
|
|
10108
|
+
By default the hook resets edit state for a new native session, preserves it
|
|
10109
|
+
when the source is `compact`, performs stale search-flag cleanup, and checks
|
|
10110
|
+
for updates without printing informational stdout. Set
|
|
10111
|
+
`AI_TOOLKIT_HOOK_VERBOSE=1` to print the startup reminders and loaded context
|
|
10112
|
+
for debugging; `AI_TOOLKIT_HOOK_QUIET=1` keeps it silent explicitly.
|
|
9355
10113
|
|
|
9356
10114
|
> **Session storage:** auto-generated session artifacts (context, handoff note,
|
|
9357
10115
|
> checkpoints, decisions) are stored **outside the project repo** under
|
|
@@ -9471,6 +10229,59 @@ arms the corrective Stop hook.
|
|
|
9471
10229
|
|
|
9472
10230
|
Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
9473
10231
|
|
|
10232
|
+
### PostToolUse (native Bash output filter): `filter-tool-output.sh`
|
|
10233
|
+
|
|
10234
|
+
| Field | Value |
|
|
10235
|
+
|-------|-------|
|
|
10236
|
+
| Event | `PostToolUse` |
|
|
10237
|
+
| Matcher | `Bash` |
|
|
10238
|
+
| Script | `~/.softspark/ai-toolkit/hooks/filter-tool-output.sh` |
|
|
10239
|
+
| Fires | After a successful Claude Code Bash tool call |
|
|
10240
|
+
|
|
10241
|
+
The hook is disabled by default and runs last among the installed
|
|
10242
|
+
`PostToolUse` handlers. Its modes are:
|
|
10243
|
+
|
|
10244
|
+
| Mode | Behavior |
|
|
10245
|
+
|------|----------|
|
|
10246
|
+
| `off` | Shell fast path exits before Python starts |
|
|
10247
|
+
| `observe` | Evaluates eligible output and writes content-free metadata, but emits no replacement |
|
|
10248
|
+
| `safe` | Replaces eligible output only after invariants pass and the exact native response is stored for recovery |
|
|
10249
|
+
|
|
10250
|
+
Only explicitly allowlisted test, lint, typecheck, and validation command
|
|
10251
|
+
shapes are eligible. Failed or interrupted tools, non-empty stderr, image or
|
|
10252
|
+
binary results, pipes, redirects, deployment, migrations, audits, security
|
|
10253
|
+
scanners, malformed payloads, unavailable recovery, and any uncertain case
|
|
10254
|
+
remain unchanged.
|
|
10255
|
+
|
|
10256
|
+
The per-project policy `<project>/.claude/ai-toolkit-output-filter.json` is
|
|
10257
|
+
honored only when **both** checks pass: the project root is registered in
|
|
10258
|
+
`~/.softspark/ai-toolkit/projects.json`, and the sibling regular file
|
|
10259
|
+
`<project>/.claude/.ai-toolkit-output-filter.owner` holds the ai-toolkit owner
|
|
10260
|
+
marker. `ai-toolkit install --local` writes both. Registration is required
|
|
10261
|
+
because the owner marker is a public constant, so a cloned or untrusted
|
|
10262
|
+
checkout must never be able to self-enable filtering by shipping its own
|
|
10263
|
+
marker. An unregistered project, a missing or foreign marker, or a symlinked
|
|
10264
|
+
project root or `.claude` directory falls back to the installed global policy
|
|
10265
|
+
at `~/.softspark/ai-toolkit/hooks/output-filter-policy.json`, which ships as
|
|
10266
|
+
`off`.
|
|
10267
|
+
|
|
10268
|
+
`jq` is a required system dependency (`python3 scripts/check_deps.py`
|
|
10269
|
+
verifies it alongside `python3`, `git`, and `node`); without it `guard-path.sh`
|
|
10270
|
+
blocks file tools rather than skipping path validation.
|
|
10271
|
+
|
|
10272
|
+
Recovery data is private and session-scoped under
|
|
10273
|
+
`~/.softspark/ai-toolkit/sessions/<repo-key>/output-filter/`. Telemetry records
|
|
10274
|
+
only profile/version, byte and line counts, latency, outcome, and fallback
|
|
10275
|
+
reason. Recovery-backed modes require a bounded native session ID containing
|
|
10276
|
+
only ASCII letters, digits, underscores, or hyphens. Three consecutive
|
|
10277
|
+
profile, invariant, or recovery safety failures open a session circuit
|
|
10278
|
+
breaker. Set `AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1` for an immediate bypass.
|
|
10279
|
+
|
|
10280
|
+
The replacement adapter is Claude Code-specific. Claude Chat/Cowork exports
|
|
10281
|
+
exclude it, and the generated hooks for other editors do not activate it.
|
|
10282
|
+
The `minimal` hook profile and `AI_TOOLKIT_DISABLED_HOOKS=filter-tool-output`
|
|
10283
|
+
bypass it before the Python runtime starts.
|
|
10284
|
+
|
|
9474
10285
|
### Stop (quality check) — `quality-check.sh`
|
|
9475
10286
|
|
|
9476
10287
|
| Field | Value |
|
|
@@ -9584,9 +10395,13 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
|
9584
10395
|
| Script | `~/.softspark/ai-toolkit/hooks/session-end.sh` |
|
|
9585
10396
|
| Fires | When a Claude session ends |
|
|
9586
10397
|
|
|
9587
|
-
**Action:**
|
|
10398
|
+
**Action:** Removes private output-filter artifacts and isolated edit state for
|
|
10399
|
+
the ending native session, then writes `session-end.md` to the per-repo session store
|
|
10400
|
+
(`~/.softspark/ai-toolkit/sessions/<repo-key>/`) with a lightweight handoff note
|
|
10401
|
+
for the next session.
|
|
9588
10402
|
|
|
9589
|
-
|
|
10403
|
+
The handoff note is skipped when `TOOLKIT_HOOK_PROFILE=minimal`; recovery and
|
|
10404
|
+
edit-state cleanup still run.
|
|
9590
10405
|
|
|
9591
10406
|
### TeammateIdle — inline
|
|
9592
10407
|
|
|
@@ -9611,7 +10426,7 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
|
9611
10426
|
| Script | `~/.softspark/ai-toolkit/hooks/guard-config.sh` |
|
|
9612
10427
|
| Fires | Before any file write/edit operation |
|
|
9613
10428
|
|
|
9614
|
-
**Action:**
|
|
10429
|
+
**Action:** Always blocks (exit 2) edits to protected linter and formatter configuration: ESLint, Prettier, TypeScript, Stylelint, Biome, `ruff.toml`, and Ruff sections in `pyproject.toml`. Prompt text and acknowledgment phrases do not bypass the guard. A user who explicitly authorizes a protected edit must deliberately remove the guard for that operation.
|
|
9615
10430
|
|
|
9616
10431
|
### SessionStart — `mcp-health.sh`
|
|
9617
10432
|
|
|
@@ -9633,7 +10448,12 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
|
9633
10448
|
| Script | `~/.softspark/ai-toolkit/hooks/governance-capture.sh` |
|
|
9634
10449
|
| Fires | After any tool use |
|
|
9635
10450
|
|
|
9636
|
-
**Action:** Non-blocking (always exits 0). Logs
|
|
10451
|
+
**Action:** Non-blocking (always exits 0). Logs recognized
|
|
10452
|
+
dangerous-command categories and writes to sensitive filenames to
|
|
10453
|
+
`~/.softspark/ai-toolkit/governance.log` with an ISO timestamp, normalized
|
|
10454
|
+
session ID, tool name, and a bounded category or path detail. It does not
|
|
10455
|
+
store the full command or file content. Skipped when
|
|
10456
|
+
`TOOLKIT_HOOK_PROFILE=minimal`.
|
|
9637
10457
|
|
|
9638
10458
|
### PreCompact — `pre-compact-save.sh`
|
|
9639
10459
|
|
|
@@ -9644,7 +10464,11 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
|
9644
10464
|
| Script | `~/.softspark/ai-toolkit/hooks/pre-compact-save.sh` |
|
|
9645
10465
|
| Fires | Before context compaction |
|
|
9646
10466
|
|
|
9647
|
-
**Action:** Saves a timestamped context snapshot to
|
|
10467
|
+
**Action:** Saves a timestamped context snapshot to
|
|
10468
|
+
`~/.softspark/ai-toolkit/compactions/YYYY-MM-DD_HH-MM-SS_<session>.txt`.
|
|
10469
|
+
Captures the bounded normalized session ID, working directory, and git branch.
|
|
10470
|
+
Provides an audit trail for compaction boundaries. Skipped when
|
|
10471
|
+
`TOOLKIT_HOOK_PROFILE=minimal`.
|
|
9648
10472
|
|
|
9649
10473
|
### PreToolUse (commit quality) — `commit-quality.sh`
|
|
9650
10474
|
|
|
@@ -9724,7 +10548,23 @@ First-match-wins per file. Built-in runners: `bats`, `pytest`, `vitest`, `jest`.
|
|
|
9724
10548
|
| Script | `~/.softspark/ai-toolkit/hooks/search-tracker.sh` |
|
|
9725
10549
|
| Fires | After any search-style tool call |
|
|
9726
10550
|
|
|
9727
|
-
**Action:** Clears `~/.softspark/ai-toolkit/state/search-required-<session_id>.flag`
|
|
10551
|
+
**Action:** Clears `~/.softspark/ai-toolkit/state/search-required-<session_id>.flag`
|
|
10552
|
+
for the bounded normalized native session. Identity lookup accepts
|
|
10553
|
+
`session_id`, Augment `conversation_id` / `conversationId`,
|
|
10554
|
+
`CLAUDE_SESSION_ID`, or the transcript basename before falling back to
|
|
10555
|
+
`default`. It pairs with `user-prompt-submit.sh` (sets the flag on long
|
|
10556
|
+
technical prompts only when a search provider is detected or strict mode is
|
|
10557
|
+
enabled) and `stop-search-check.sh` (blocks Stop if the calling session's flag
|
|
10558
|
+
is still set). Search provider detection parses actual MCP server names from
|
|
10559
|
+
`mcpServers`, `mcp_servers`, or `mcp` config blocks; hook matchers and
|
|
10560
|
+
permission allowlists do not count as providers. Codex Stop enforcement also
|
|
10561
|
+
scans the recent `$CODEX_HOME/log/codex-tui.log` window (default
|
|
10562
|
+
`~/.codex/log/codex-tui.log`) for `ToolCall:
|
|
10563
|
+
mcp__...__smart_query` and `tool.name="smart_query"`-style entries because
|
|
10564
|
+
Codex MCP tool calls may not fire the shared `PostToolUse` tracker. Together
|
|
10565
|
+
the hooks enforce the global CLAUDE.md GOLDEN RULE without breaking
|
|
10566
|
+
offline/no-RAG installs and without cross-session interference when multiple
|
|
10567
|
+
runtime windows run in parallel.
|
|
9728
10568
|
|
|
9729
10569
|
Non-blocking (exit 0). Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
9730
10570
|
|
|
@@ -9771,7 +10611,7 @@ Non-blocking (exit 0). Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
|
|
|
9771
10611
|
|
|
9772
10612
|
| Component | Purpose |
|
|
9773
10613
|
|-----------|---------|
|
|
9774
|
-
| `scripts/session_state.py` |
|
|
10614
|
+
| `scripts/session_state.py` | Isolated edit state keyed by a hash of the normalized native session ID. Reset on SessionStart and cleaned on SessionEnd. Read by revert-guard and quality-gate. |
|
|
9775
10615
|
| `scripts/test_cohesion.py` | Resolves changed paths → test commands via cohesion map. First-match-wins. Stdlib-only. |
|
|
9776
10616
|
| `app/hooks/test-cohesion-map.json` | Toolkit-default path → tests mapping (used when no project map exists). |
|
|
9777
10617
|
| `app/hooks/_locate-toolkit.sh` | Shared bash helper that exports `$TOOLKIT_DIR` for hooks needing scripts/. |
|
|
@@ -9805,8 +10645,9 @@ commands explicitly silent, and Codex-generated hooks plus Claude's bundled
|
|
|
9805
10645
|
```
|
|
9806
10646
|
~/.softspark/ai-toolkit/
|
|
9807
10647
|
├── rules/ # Registered rules (add-rule.sh)
|
|
9808
|
-
├── state/ # Per-session runtime state
|
|
9809
|
-
│ ├── session-edits
|
|
10648
|
+
├── state/ # Per-session runtime state
|
|
10649
|
+
│ ├── session-edits-<hash>.json # Isolated edit state, removed at SessionEnd
|
|
10650
|
+
│ ├── session-edits.json # Compatibility alias for the active session
|
|
9810
10651
|
│ ├── search-required-<sid>.flag # Per-session: set by user-prompt-submit, cleared by search-tracker/stop-search-check, GC'd at SessionStart (>60min)
|
|
9811
10652
|
│ ├── loaded-instructions.log # Audit trail of which rules entered context
|
|
9812
10653
|
│ └── test-cohesion-last.log # Last cohesion test command output
|
|
@@ -9878,6 +10719,7 @@ Beyond the global Claude Code hooks above, editor profiles emit native hook file
|
|
|
9878
10719
|
| Augment | `.augment/settings.json` (hooks block) | `generate_augment_hooks.py` | Claude-style events |
|
|
9879
10720
|
| GitHub Copilot | `.github/hooks/ai-toolkit.json`; user `$COPILOT_HOME/hooks/ai-toolkit.json` | `generate_copilot_hooks.py` | GitHub version 1, camelCase events (profile ≥ `standard`) |
|
|
9880
10721
|
| Codex CLI | `.codex/hooks.json`; user `$CODEX_HOME/hooks.json` | `generate_codex_hooks.py` | Native Codex schema, PascalCase events, command ownership markers |
|
|
10722
|
+
| OpenCode | `.opencode/plugins/ai-toolkit-hooks.js`; user `~/.config/opencode/plugins/ai-toolkit-hooks.js` | `generate_opencode_plugin.py` | Native JavaScript plugin hooks |
|
|
9881
10723
|
|
|
9882
10724
|
### Cursor hooks (`.cursor/hooks.json`)
|
|
9883
10725
|
|
|
@@ -11521,6 +12363,7 @@ The opencode integration is verified by:
|
|
|
11521
12363
|
- `kb/reference/skills-catalog.md`
|
|
11522
12364
|
- `kb/reference/agents-catalog.md`
|
|
11523
12365
|
- `kb/reference/codex-cli-compatibility.md`
|
|
12366
|
+
- `kb/reference/copilot-compatibility.md`
|
|
11524
12367
|
- `kb/reference/architecture-overview.md`
|
|
11525
12368
|
- `kb/reference/global-install-model.md`
|
|
11526
12369
|
- `kb/reference/mcp-editor-compatibility.md`
|
|
@@ -12284,9 +13127,9 @@ title: "Supported Tools Registry"
|
|
|
12284
13127
|
category: reference
|
|
12285
13128
|
service: ai-toolkit
|
|
12286
13129
|
tags: [editors, platforms, generators, integration, ecosystem]
|
|
12287
|
-
version: "1.
|
|
13130
|
+
version: "1.11.0"
|
|
12288
13131
|
created: "2026-04-23"
|
|
12289
|
-
last_updated: "2026-07-
|
|
13132
|
+
last_updated: "2026-07-23"
|
|
12290
13133
|
description: "Human-readable view of scripts/ecosystem_tools.json — the canonical list of tools ai-toolkit integrates with (Claude Code, Claude Chat/Cowork, and 11 editors), their documentation URLs, config paths, our generators, and tracked capability markers."
|
|
12291
13134
|
---
|
|
12292
13135
|
|
|
@@ -12314,6 +13157,7 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12314
13157
|
| Our generators | — (Claude Code is the primary target; toolkit content ships directly as `.md` files and `settings.json` merges) |
|
|
12315
13158
|
| Tracked hook events | Core: `SessionStart`, `SessionEnd`, `UserPromptSubmit`, `Notification`, `MessageDisplay`. Tool: `PreToolUse`, `PostToolUse`, `PostToolUseFailure`, `PostToolBatch`. Turn: `Stop`, `StopFailure`, `UserPromptExpansion`. Subagent: `SubagentStart`, `SubagentStop`. Compaction: `PreCompact`, `PostCompact`. Permissions: `PermissionRequest`, `PermissionDenied`. Elicitation: `Elicitation`, `ElicitationResult`. Teams: `TaskCreated`, `TaskCompleted`, `TeammateIdle`. Worktrees/env: `WorktreeCreate`, `WorktreeRemove`, `CwdChanged`, `FileChanged`, `ConfigChange`. Setup: `Setup`, `InstructionsLoaded` |
|
|
12316
13159
|
| Tracked handler types | `command`, `prompt`, `agent`, `mcp_tool`, `http` (POST event JSON to allowlisted URLs via `allowedHttpHookUrls`) |
|
|
13160
|
+
| Tool-output replacement | Native `PostToolUse.updatedToolOutput`; ai-toolkit exposes dependency-free `off`, byte-identical `observe`, and opt-in recoverable `safe` modes for eligible successful Bash text |
|
|
12317
13161
|
| Other capabilities | slash commands, MCP server/client, sub-agent, output style, `SKILL.md` (≥500 lines warn) |
|
|
12318
13162
|
| Version probe | `claude --version` |
|
|
12319
13163
|
| Notes | v2.1.169 added `disableBundledSkills` setting + `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS` env var (hides bundled skills/built-in slash commands from the model; toolkit skills in `.claude/skills/` are unaffected — useful when toolkit skills overlap built-ins) and `claude --safe-mode` / `CLAUDE_CODE_SAFE_MODE` (starts with hooks, skills, agents, and CLAUDE.md disabled — first isolation step when debugging toolkit rule enforcement). `fallbackModel` settings key (v2.1.166) noted as not-adopted (class C, no toolkit surface writes model settings). |
|
|
@@ -12333,6 +13177,7 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12333
13177
|
| Plugin layout | `.claude-plugin/plugin.json`, `skills/*/SKILL.md`, `agents/*.md`, `hooks/hooks.json`; ai-toolkit uses manifest paths under `claude-app/` for its generated app-only rules and hooks |
|
|
12334
13178
|
| Our generator | `scripts/claude_app.py` (`ai-toolkit claude-app export`) |
|
|
12335
13179
|
| Runtime split | Skills work in Chat (web/Desktop) and Cowork. Hooks and sub-agents run only in Cowork. Claude app does **not** scan Claude Code's `~/.claude/rules/`, `CLAUDE.md`, or `~/.claude/settings.json`. |
|
|
13180
|
+
| Tool-output replacement | Not shipped. The Claude Code-only filter hook and runtime are excluded from the app archive until Cowork has an independently verified replacement contract. |
|
|
12336
13181
|
| Install/update | Export the ZIP, upload it from `Customize > Plugins`, then paste the generated global-instructions file into `Settings > Cowork > Global instructions`. Re-export/re-upload after toolkit updates. |
|
|
12337
13182
|
|
|
12338
13183
|
---
|
|
@@ -12384,7 +13229,7 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12384
13229
|
| Our generators | `scripts/generate_copilot.py` (instructions, prompts, agents, and portable skill directories), `scripts/generate_copilot_hooks.py` (native version-1 hook config plus self-contained runtime) |
|
|
12385
13230
|
| Global install | `ai-toolkit install --editors copilot` writes instructions, agents, skills, and, for profile ≥ `standard`, native hooks below `$COPILOT_HOME` when set or `~/.copilot` otherwise. VS Code Copilot and GitHub.com use repo `.github/` files, so local emission remains required. |
|
|
12386
13231
|
| Tracked capabilities | `copilot-instructions.md`, Copilot Chat, Copilot Workspace, Copilot cloud agent, `applyTo`, custom agents, prompt files, `instructions.md`, `AGENTS.md`, MCP, skills, CLI hooks, `~/.copilot/` |
|
|
12387
|
-
| Compatibility notes | Custom agents use native `.agent.md` files with `name` and `description`; `tools` is omitted instead of guessing editor-specific aliases. Prompt and skill bodies remove Claude-only interpolation and delegation APIs. Hooks use the GitHub version-1 schema, camelCase event names, native decision payloads, and a repository/config-root-contained Python runtime instead of Claude hook scripts. Project MCP remains owned by the editor MCP sync path. Copilot code review also reads the nearest `AGENTS.md`; local install keeps its generated section separate from Codex/OpenCode sections. |
|
|
13232
|
+
| Compatibility notes | Custom agents use native `.agent.md` files with `name` and `description`; `tools` is omitted instead of guessing editor-specific aliases. Prompt and skill bodies remove Claude-only interpolation and delegation APIs. Hooks use the GitHub version-1 schema, camelCase event names, native decision payloads, and a repository/config-root-contained Python runtime instead of Claude hook scripts. Project MCP remains owned by the editor MCP sync path. Copilot code review also reads the nearest `AGENTS.md`; local install keeps its generated section separate from Codex/OpenCode sections. Surface loading semantics and the prompt/skill duplication are documented in `kb/reference/copilot-compatibility.md`. |
|
|
12388
13233
|
|
|
12389
13234
|
### Gemini CLI
|
|
12390
13235
|
|
|
@@ -12451,7 +13296,7 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12451
13296
|
| Our generators | `scripts/generate_augment.py`, `scripts/generate_augment_rules.py`, `scripts/generate_augment_agents.py` (profile=full), `scripts/generate_augment_commands.py` (profile=full), `scripts/generate_augment_hooks.py` (profile=full, HOME-scoped), `scripts/generate_augment_skills.py` (profile=full) |
|
|
12452
13297
|
| Global install | `ai-toolkit install --editors augment` writes `~/.augment/rules/ai-toolkit.md` **plus** (profile=full) `~/.augment/agents/`, `~/.augment/commands/`, and hooks in `~/.augment/settings.json` — all documented user-tier surfaces. A global-only Augment user previously got no hooks/agents/commands. Skills need no global emission: Auggie natively reads `~/.claude/skills/`. |
|
|
12453
13298
|
| Tracked capabilities | `.augment`, Agent mode, Next Edit, MCP, context engine, Auggie CLI, `always_apply`, `agent_requested`, subagents, custom commands, `SKILL.md`, `PreToolUse`, `PostToolUse`, `SessionStart`, `SessionEnd`, `Stop`, `Notification`, ACP Mode, plugins/marketplace |
|
|
12454
|
-
| Latest / notes | Auggie CLI v0.31.0. Since v0.30.0 (2026-06-25) `PreToolUse`/`PostToolUse` hooks fire during sub-agent sessions
|
|
13299
|
+
| Latest / notes | Auggie CLI v0.31.0. Since v0.30.0 (2026-06-25) `PreToolUse`/`PostToolUse` hooks fire during sub-agent sessions. The shared hook input adapter normalizes Augment's `conversation_id`, so edit and quality state remains isolated across parallel sub-agents. `Notification` is enum-only (doctor flipped its marker), with no dedicated handler documented, so class C (do NOT wire). Plugins/marketplace (`auggie plugin marketplace add`, `.augment-plugin`/`.claude-plugin` layouts, `enabledPlugins`/`autoUpdateMarketplaces` in settings) are documented but not yet a shipping target. |
|
|
12455
13300
|
| SPA caveat | Mintlify Next.js SPA; use `https://docs.augmentcode.com/<path>.md` siblings (discoverable via `/llms.txt`) for machine reads |
|
|
12456
13301
|
|
|
12457
13302
|
### Google Antigravity
|
|
@@ -12479,7 +13324,8 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12479
13324
|
| Config paths | **Instructions:** project `AGENTS.md` (root→cwd chain, closest wins) and global `~/.codex/AGENTS.md` (`$CODEX_HOME/AGENTS.md`; `~/.codex/AGENTS.override.md` takes precedence). NOTE: `~/AGENTS.md` is NOT a global-instruction surface — Codex only reads it if a session's cwd is exactly `$HOME`. Plus `.agents/skills/*/SKILL.md`, `.codex/agents/*.toml`, `~/.codex/agents/*.toml`, `.codex/hooks.json`, `~/.codex/hooks.json`, `.codex/config.toml` (project layers, root→cwd, closest wins, trusted projects only), `~/.codex/config.toml`. |
|
|
12480
13325
|
| Our generators | `scripts/generate_codex.py`, `scripts/generate_codex_agents.py` (native custom-agent TOML), `scripts/generate_codex_hooks.py`, `scripts/generate_codex_skills.py` (opt-in via `--codex-skills`) |
|
|
12481
13326
|
| Rules delivery | Universal coding rules are inlined into `AGENTS.md` (Codex reads instructions only from AGENTS.md, not `.agents/rules/`); language rules ship as `<lang>-rules` skills under `.agents/skills/`. Global install writes `~/.codex/AGENTS.md` (not `~/AGENTS.md`, which Codex never loads globally); plugin-pack rules are marker-injected into the same file. `project_doc_max_bytes` default is 32 KiB and Codex silently truncates AGENTS.md past that (see codex-cli-compatibility.md). |
|
|
12482
|
-
| Tracked hook events | Upstream canonical (codex-rs `HookEventName` enum): `PreToolUse`, `PostToolUse`, `PermissionRequest`, `PreCompact`, `PostCompact`, `SessionStart`, `UserPromptSubmit`, `SubagentStart`, `SubagentStop`, `Stop` (10 events). We wire 9
|
|
13327
|
+
| Tracked hook events | Upstream canonical (codex-rs `HookEventName` enum): `PreToolUse`, `PostToolUse`, `PermissionRequest`, `PreCompact`, `PostCompact`, `SessionStart`, `UserPromptSubmit`, `SubagentStart`, `SubagentStop`, `Stop` (10 events). We wire 9 through an explicit Codex map, including destructive-command and wrong-home path guards on both Bash `PreToolUse` and `PermissionRequest`. `PostCompact` is not wired (its only hook was the removed environment-snapshot probe). |
|
|
13328
|
+
| Tool-output replacement | Manual `ai-toolkit output-filter inspect` only. The Claude-specific `updatedToolOutput` adapter is not emitted into Codex hooks. |
|
|
12483
13329
|
| Tracked handler types | `command` (emitted by default; the only handler Codex actually runs). `prompt` and `agent` are parsed by Codex but NOT yet executed, so hand-authored handlers of those types are inert. |
|
|
12484
13330
|
| Other capabilities | `AGENTS.md`, `config.toml`, `mcp_servers`, sandbox policies, `.agents/skills/*/SKILL.md` (native Codex skill discovery path), `.codex/agents/*.toml` (native custom agents) |
|
|
12485
13331
|
| Version probe | `codex --version` |
|
|
@@ -12493,6 +13339,8 @@ The canonical data lives in **`scripts/ecosystem_tools.json`** and is consumed b
|
|
|
12493
13339
|
| Release notes | https://github.com/sst/opencode/releases (redirects to anomalyco/opencode) |
|
|
12494
13340
|
| Config paths | `opencode.json`, `.opencode/agents/*.md`, `.opencode/commands/*.md`, `.opencode/plugins/*`, `.opencode/skills/*/SKILL.md` (v1.14+), `AGENTS.md`; skill fallback discovery: `.claude/skills/`, `.agents/skills/`, `~/.config/opencode/skills/`, `~/.claude/skills/`, `~/.agents/skills/` |
|
|
12495
13341
|
| Our generators | `scripts/generate_opencode.py`, `scripts/generate_opencode_agents.py`, `scripts/generate_opencode_commands.py`, `scripts/generate_opencode_json.py`, `scripts/generate_opencode_plugin.py` |
|
|
13342
|
+
| Hook isolation | Tool hooks preserve native `sessionID` as normalized `session_id`; exit code 2 from a blocking pre-tool guard is raised back to OpenCode instead of being ignored. |
|
|
13343
|
+
| Tool-output replacement | Manual `ai-toolkit output-filter inspect` only. No active OpenCode output adapter is shipped. |
|
|
12496
13344
|
| Tracked plugin events | `session.created`, `session.compacted`, `session.deleted`, `message.updated`, `tool.execute.before`, `tool.execute.after`, `permission.asked`, `command.executed` |
|
|
12497
13345
|
| Other capabilities | `opencode.json` config, primary + subagent modes, `@`-mention subagents, `/`-invocation commands, MCP (local + remote), plugin hooks in JS/TS, native `SKILL.md` discovery with Claude-compatible fallback, `permission.skill.*` matrix |
|
|
12498
13346
|
| Version probe | `opencode --version` |
|
|
@@ -12630,6 +13478,299 @@ ai-toolkit sync --pull abc123def456 # Use gist ID from first push
|
|
|
12630
13478
|
|
|
12631
13479
|
---
|
|
12632
13480
|
|
|
13481
|
+
## kb/reference/tool-output-filter.md
|
|
13482
|
+
|
|
13483
|
+
---
|
|
13484
|
+
title: "Native Tool Output Filter"
|
|
13485
|
+
category: reference
|
|
13486
|
+
service: ai-toolkit
|
|
13487
|
+
tags: [output-filter, hooks, recovery, telemetry, claude-code]
|
|
13488
|
+
version: "1.0.0"
|
|
13489
|
+
created: "2026-07-23"
|
|
13490
|
+
last_updated: "2026-07-23"
|
|
13491
|
+
description: "Contract, configuration, safety boundaries, recovery, CLI, and runtime support for the native ai-toolkit output filter."
|
|
13492
|
+
---
|
|
13493
|
+
|
|
13494
|
+
# Native Tool Output Filter
|
|
13495
|
+
|
|
13496
|
+
## Overview
|
|
13497
|
+
|
|
13498
|
+
ai-toolkit includes an original, dependency-free filter for selected
|
|
13499
|
+
post-execution tool results. It is disabled by default and does not depend on,
|
|
13500
|
+
vendor, execute, or copy another output-filter package.
|
|
13501
|
+
|
|
13502
|
+
The active adapter targets Claude Code because its `PostToolUse` contract can
|
|
13503
|
+
replace a native tool response through
|
|
13504
|
+
`hookSpecificOutput.updatedToolOutput`. The replacement object retains the
|
|
13505
|
+
native response shape and changes only `stdout`. See the
|
|
13506
|
+
[Claude Code hooks reference](https://code.claude.com/docs/en/hooks).
|
|
13507
|
+
|
|
13508
|
+
The filter never changes the command, arguments, environment, working
|
|
13509
|
+
directory, permission decision, exit status, or signal. [PATH:
|
|
13510
|
+
scripts/tool_output_filter/hook_runtime.py] [PATH:
|
|
13511
|
+
scripts/tool_output_filter/engine.py]
|
|
13512
|
+
|
|
13513
|
+
## Modes
|
|
13514
|
+
|
|
13515
|
+
| Mode | Model-visible result | Recovery | Telemetry |
|
|
13516
|
+
|------|----------------------|----------|-----------|
|
|
13517
|
+
| `off` | Original | None | None |
|
|
13518
|
+
| `observe` | Original | No raw response | Content-free decision metadata |
|
|
13519
|
+
| `safe` | Replacement only after every gate passes | Exact native response saved first | Content-free decision metadata |
|
|
13520
|
+
|
|
13521
|
+
`off` is a shell fast path, so the Python runtime is not started. Any runtime
|
|
13522
|
+
error, malformed payload, unsafe command, failed invariant, unavailable secure
|
|
13523
|
+
storage, or insufficient saving leaves the original response unchanged.
|
|
13524
|
+
|
|
13525
|
+
Three consecutive profile, invariant, or recovery safety failures open a
|
|
13526
|
+
persistent session-scoped circuit breaker. One bounded Claude system message
|
|
13527
|
+
reports the bypass, then later results stay unchanged for that session.
|
|
13528
|
+
|
|
13529
|
+
## Configuration
|
|
13530
|
+
|
|
13531
|
+
Configure the project in `.softspark-toolkit.json`:
|
|
13532
|
+
|
|
13533
|
+
```json
|
|
13534
|
+
{
|
|
13535
|
+
"toolOutputFilter": {
|
|
13536
|
+
"mode": "observe",
|
|
13537
|
+
"profiles": ["repeat-lines", "tap-success"],
|
|
13538
|
+
"maxInputBytes": 8388608,
|
|
13539
|
+
"minSavingsBytes": 1024,
|
|
13540
|
+
"minSavingsRatio": 0.15,
|
|
13541
|
+
"recovery": {
|
|
13542
|
+
"mode": "ephemeral",
|
|
13543
|
+
"ttlMinutes": 60,
|
|
13544
|
+
"maxSessionBytes": 33554432
|
|
13545
|
+
}
|
|
13546
|
+
}
|
|
13547
|
+
}
|
|
13548
|
+
```
|
|
13549
|
+
|
|
13550
|
+
Run `ai-toolkit install --local` or `ai-toolkit update --local` to materialize
|
|
13551
|
+
the effective policy as:
|
|
13552
|
+
|
|
13553
|
+
```text
|
|
13554
|
+
<project>/.claude/ai-toolkit-output-filter.json
|
|
13555
|
+
<project>/.claude/.ai-toolkit-output-filter.owner
|
|
13556
|
+
```
|
|
13557
|
+
|
|
13558
|
+
The managed files use mode `0600`. The hook accepts a project policy only when
|
|
13559
|
+
the project root is registered in `~/.softspark/ai-toolkit/projects.json`
|
|
13560
|
+
**and** the regular owner marker matches ai-toolkit. Registration is the
|
|
13561
|
+
security boundary: the owner marker is a public constant, so requiring the
|
|
13562
|
+
registry stops a cloned or untrusted checkout from self-enabling filtering by
|
|
13563
|
+
committing its own marker. An unregistered project, a missing or foreign
|
|
13564
|
+
marker, or a symlinked project root or `.claude` directory falls back to the
|
|
13565
|
+
installed global policy at
|
|
13566
|
+
`~/.softspark/ai-toolkit/hooks/output-filter-policy.json`, which defaults to
|
|
13567
|
+
`off`. [PATH: app/hooks/filter-tool-output.sh] [PATH:
|
|
13568
|
+
scripts/install_steps/ai_tools.py]
|
|
13569
|
+
|
|
13570
|
+
`jq` is a required system dependency for the lifecycle hooks and is verified
|
|
13571
|
+
by `python3 scripts/check_deps.py` alongside `python3`, `git`, and `node`.
|
|
13572
|
+
|
|
13573
|
+
Before executing anything, the hook validates the resolved Python runtime path:
|
|
13574
|
+
it must be a readable regular file and must not be a symlink. A missing,
|
|
13575
|
+
non-regular, unreadable, or symlinked runtime makes the hook exit silently and
|
|
13576
|
+
leave the tool response unchanged, so a tampered or half-installed runtime
|
|
13577
|
+
cannot be invoked. The same regular-file rule applies to every policy file the
|
|
13578
|
+
hook reads. [PATH: app/hooks/filter-tool-output.sh]
|
|
13579
|
+
|
|
13580
|
+
Set `AI_TOOLKIT_OUTPUT_FILTER_DISABLE=1` for an immediate bypass without
|
|
13581
|
+
reinstalling. `AI_TOOLKIT_OUTPUT_FILTER_POLICY` may point the hook to an
|
|
13582
|
+
explicit regular policy file for controlled operational testing. The hook is
|
|
13583
|
+
also skipped by the `minimal` hook profile and may be listed in
|
|
13584
|
+
`AI_TOOLKIT_DISABLED_HOOKS`. `AI_TOOLKIT_OUTPUT_FILTER_HOOK_RUNTIME` is reserved
|
|
13585
|
+
for controlled runtime testing; the manual and cleanup CLI remains
|
|
13586
|
+
`output_filter_cli.py`.
|
|
13587
|
+
|
|
13588
|
+
## Eligibility
|
|
13589
|
+
|
|
13590
|
+
The Claude adapter considers only a completed `PostToolUse` event with:
|
|
13591
|
+
|
|
13592
|
+
- tool name `Bash`;
|
|
13593
|
+
- a non-empty native session ID of at most 160 ASCII letters, digits,
|
|
13594
|
+
underscores, or hyphens;
|
|
13595
|
+
- string `stdout`;
|
|
13596
|
+
- empty `stderr`;
|
|
13597
|
+
- `interrupted: false`;
|
|
13598
|
+
- `isImage: false`;
|
|
13599
|
+
- a command that matches a strict allowlist for test, lint, typecheck, or
|
|
13600
|
+
validation tools;
|
|
13601
|
+
- input at or below 8 MiB;
|
|
13602
|
+
- valid text without binary or terminal-control content.
|
|
13603
|
+
|
|
13604
|
+
The following always pass through unchanged:
|
|
13605
|
+
|
|
13606
|
+
- failed, interrupted, image, binary, invalid-text, TTY, or streaming results;
|
|
13607
|
+
- pipes, redirects, shell chaining, substitutions, and multiline commands;
|
|
13608
|
+
- deployment, release, migration, publish, destroy, audit, and security-scanner
|
|
13609
|
+
commands;
|
|
13610
|
+
- arbitrary Python scripts and unknown command shapes;
|
|
13611
|
+
- output with non-empty stderr;
|
|
13612
|
+
- unknown profiles or native payload shapes;
|
|
13613
|
+
- candidates that save less than both the configured byte and ratio threshold.
|
|
13614
|
+
|
|
13615
|
+
The command classifier is eligibility logic only. It never parses and
|
|
13616
|
+
re-executes a command. [PATH: scripts/tool_output_filter/hook_runtime.py]
|
|
13617
|
+
|
|
13618
|
+
## Profiles
|
|
13619
|
+
|
|
13620
|
+
### `repeat-lines`
|
|
13621
|
+
|
|
13622
|
+
Collapses only adjacent identical, non-diagnostic lines. It retains the first
|
|
13623
|
+
line and adds a versioned marker with the exact number of omitted copies.
|
|
13624
|
+
Warnings, failures, permissions, security diagnostics, blank lines, comments,
|
|
13625
|
+
existing filter markers, and control-bearing output are not collapsed.
|
|
13626
|
+
|
|
13627
|
+
### `tap-success`
|
|
13628
|
+
|
|
13629
|
+
Accepts only a strict, complete, successful TAP stream with a single plan and
|
|
13630
|
+
contiguous `ok` result numbers. It retains the TAP version, plan, directives,
|
|
13631
|
+
comments, totals, duration, and other summary lines. Diagnostics, `not ok`,
|
|
13632
|
+
non-zero failure summaries, malformed plans, gaps, duplicates, and unknown
|
|
13633
|
+
content reject the whole profile.
|
|
13634
|
+
|
|
13635
|
+
Both profiles are deterministic and idempotent. A safe replacement must remain
|
|
13636
|
+
smaller after the recovery marker is added. [PATH:
|
|
13637
|
+
scripts/tool_output_filter/profiles/] [PATH:
|
|
13638
|
+
tests/test_tool_output_filter_properties.py]
|
|
13639
|
+
|
|
13640
|
+
## Exact Recovery and Privacy
|
|
13641
|
+
|
|
13642
|
+
Before `safe` mode emits a replacement, it stores and reloads the complete
|
|
13643
|
+
native tool-response object. Equality must succeed before the hook prints
|
|
13644
|
+
`updatedToolOutput`.
|
|
13645
|
+
|
|
13646
|
+
```text
|
|
13647
|
+
~/.softspark/ai-toolkit/sessions/<repo-key>/
|
|
13648
|
+
└── output-filter/
|
|
13649
|
+
└── <hashed-session>/
|
|
13650
|
+
├── <opaque-handle>.json
|
|
13651
|
+
├── .circuit-state.json
|
|
13652
|
+
└── .telemetry.jsonl
|
|
13653
|
+
```
|
|
13654
|
+
|
|
13655
|
+
Recovery directories use `0700`; response, state, and telemetry files use
|
|
13656
|
+
`0600`. Creation and cleanup use pinned directory descriptors, no-follow
|
|
13657
|
+
operations, atomic publication, opaque random handles, a per-session quota,
|
|
13658
|
+
and TTL cleanup. If this secure contract is unavailable, `safe` mode returns
|
|
13659
|
+
the original response.
|
|
13660
|
+
|
|
13661
|
+
The replacement ends with a marker similar to:
|
|
13662
|
+
|
|
13663
|
+
```text
|
|
13664
|
+
[ai-toolkit-output-filter repeat-lines/v1; original_lines=500; emitted_lines=3; recovery=<opaque-handle>]
|
|
13665
|
+
```
|
|
13666
|
+
|
|
13667
|
+
The recovery file can contain everything returned by the tool, including
|
|
13668
|
+
secrets. Treat the session directory as sensitive. Telemetry never stores raw
|
|
13669
|
+
output, commands, paths, environment values, session IDs, or recovery handles.
|
|
13670
|
+
It contains only profile/version, input/output byte and line counts, latency,
|
|
13671
|
+
outcome, and a bounded fallback reason.
|
|
13672
|
+
|
|
13673
|
+
Session end, explicit cleanup, and global uninstall remove only validated
|
|
13674
|
+
ai-toolkit-owned filter artifacts. Foreign files and directories are
|
|
13675
|
+
preserved. [PATH: scripts/tool_output_filter/recovery.py] [PATH:
|
|
13676
|
+
app/hooks/session-end.sh] [PATH: scripts/uninstall.py]
|
|
13677
|
+
|
|
13678
|
+
## CLI
|
|
13679
|
+
|
|
13680
|
+
Inspect candidate savings without changing output:
|
|
13681
|
+
|
|
13682
|
+
```bash
|
|
13683
|
+
some-test-command | ai-toolkit output-filter inspect --profile repeat-lines
|
|
13684
|
+
some-tap-command | ai-toolkit output-filter inspect --profile tap-success
|
|
13685
|
+
```
|
|
13686
|
+
|
|
13687
|
+
The JSON report contains counts, eligibility, outcome, and fallback reason. It
|
|
13688
|
+
does not echo stdin.
|
|
13689
|
+
|
|
13690
|
+
Inspect the effective trusted project or global policy:
|
|
13691
|
+
|
|
13692
|
+
```bash
|
|
13693
|
+
ai-toolkit output-filter status
|
|
13694
|
+
ai-toolkit output-filter status --policy /path/to/materialized-policy.json
|
|
13695
|
+
```
|
|
13696
|
+
|
|
13697
|
+
Recover the exact native response object using the handle printed in a safe
|
|
13698
|
+
replacement:
|
|
13699
|
+
|
|
13700
|
+
```bash
|
|
13701
|
+
ai-toolkit output-filter recover <opaque-handle>
|
|
13702
|
+
```
|
|
13703
|
+
|
|
13704
|
+
The default lookup derives the current repository session directory. Advanced
|
|
13705
|
+
or test workflows can add `--base-directory PATH` or `--session-id ID`.
|
|
13706
|
+
|
|
13707
|
+
Clean the ending session, expired exact responses, or all filter artifacts for
|
|
13708
|
+
the current repository:
|
|
13709
|
+
|
|
13710
|
+
```bash
|
|
13711
|
+
ai-toolkit output-filter clean --session-id <native-session-id>
|
|
13712
|
+
ai-toolkit output-filter clean --session-id <native-session-id> --expired
|
|
13713
|
+
ai-toolkit output-filter clean
|
|
13714
|
+
```
|
|
13715
|
+
|
|
13716
|
+
`--expired` requires `--session-id`. Cleanup prints only the removed artifact
|
|
13717
|
+
count and scope.
|
|
13718
|
+
|
|
13719
|
+
## Runtime Capability Matrix
|
|
13720
|
+
|
|
13721
|
+
| Runtime | Active result replacement | Capability |
|
|
13722
|
+
|---------|---------------------------|------------|
|
|
13723
|
+
| Claude Code | Yes, opt-in | Native `PostToolUse.updatedToolOutput` adapter |
|
|
13724
|
+
| Claude Chat / Cowork | No | Plugin export explicitly excludes the Claude Code-only hook |
|
|
13725
|
+
| Cursor | No | Manual `output-filter inspect` only |
|
|
13726
|
+
| Windsurf / Devin | No | Manual `output-filter inspect` only |
|
|
13727
|
+
| GitHub Copilot | No | Manual `output-filter inspect` only |
|
|
13728
|
+
| Gemini CLI | No | Manual `output-filter inspect` only |
|
|
13729
|
+
| Cline | No | Manual `output-filter inspect` only |
|
|
13730
|
+
| Roo Code | No | Manual `output-filter inspect` only |
|
|
13731
|
+
| Aider | No | Manual `output-filter inspect` only |
|
|
13732
|
+
| Augment | No | Manual `output-filter inspect` only |
|
|
13733
|
+
| Google Antigravity | No | Manual `output-filter inspect` only |
|
|
13734
|
+
| Codex CLI | No | Manual `output-filter inspect` only |
|
|
13735
|
+
| OpenCode | No | Manual `output-filter inspect` only |
|
|
13736
|
+
|
|
13737
|
+
An editor hook, extra context message, or command wrapper is not treated as
|
|
13738
|
+
result replacement. A new adapter requires a verified native replacement
|
|
13739
|
+
contract and dedicated native payload tests.
|
|
13740
|
+
|
|
13741
|
+
## Benchmark Semantics
|
|
13742
|
+
|
|
13743
|
+
Run the deterministic offline corpus:
|
|
13744
|
+
|
|
13745
|
+
```bash
|
|
13746
|
+
PYTHONDONTWRITEBYTECODE=1 python3 scripts/benchmark_output_filter.py
|
|
13747
|
+
```
|
|
13748
|
+
|
|
13749
|
+
The benchmark measures profile p95 latency, production Bash-wrapper latency
|
|
13750
|
+
with a fresh Python process per sample in one native session, traced peak
|
|
13751
|
+
allocation, and eligible-output byte reduction. It uses 100 samples by default
|
|
13752
|
+
to avoid a one-sample p95 swing. The current gates are:
|
|
13753
|
+
|
|
13754
|
+
- at least 30% candidate byte reduction;
|
|
13755
|
+
- at most 20 ms p95 for profile inputs up to 100 KiB;
|
|
13756
|
+
- at most 150 ms p95 for the 8 MiB profile case;
|
|
13757
|
+
- at most 75 ms p95 for a cold end-to-end hook process;
|
|
13758
|
+
- peak traced allocation no greater than three input sizes plus 16 MiB.
|
|
13759
|
+
|
|
13760
|
+
Byte reduction is not billed-token savings and is not a whole-session cost
|
|
13761
|
+
claim. Measure actual model token receipts separately before changing the
|
|
13762
|
+
default mode. [PATH: scripts/benchmark_output_filter.py] [PATH:
|
|
13763
|
+
benchmarks/output-filter/]
|
|
13764
|
+
|
|
13765
|
+
## Related
|
|
13766
|
+
|
|
13767
|
+
- [Hooks Catalog](hooks-catalog.md)
|
|
13768
|
+
- [Supported Tools Registry](supported-tools-registry.md)
|
|
13769
|
+
- [Architecture Overview](architecture-overview.md)
|
|
13770
|
+
- [Output Token Discipline Plan](../history/completed/output-token-discipline-plan-20260504.md)
|
|
13771
|
+
|
|
13772
|
+
---
|
|
13773
|
+
|
|
12633
13774
|
## kb/reference/unique-features.md
|
|
12634
13775
|
|
|
12635
13776
|
---
|
|
@@ -12863,7 +14004,7 @@ category: reference
|
|
|
12863
14004
|
service: ai-toolkit
|
|
12864
14005
|
tags: [windows, wsl, install, uninstall, dependencies, hooks, security]
|
|
12865
14006
|
created: "2026-04-24"
|
|
12866
|
-
last_updated: "2026-07-
|
|
14007
|
+
last_updated: "2026-07-15"
|
|
12867
14008
|
description: "Windows support model for ai-toolkit: WSL, Git Bash, dependency detection, hooks, and fail-closed managed mutations."
|
|
12868
14009
|
---
|
|
12869
14010
|
|
|
@@ -12900,8 +14041,9 @@ Cross-platform hooks should keep the Bash entrypoint small and delegate complex
|
|
|
12900
14041
|
|
|
12901
14042
|
## Managed Mutation Safety
|
|
12902
14043
|
|
|
12903
|
-
Managed cleanup by `ai-toolkit uninstall`,
|
|
12904
|
-
native Codex hook generation require POSIX
|
|
14044
|
+
Managed cleanup by `ai-toolkit uninstall`, Copilot profile downgrade cleanup,
|
|
14045
|
+
external hook injection/removal, and native Codex hook generation require POSIX
|
|
14046
|
+
`dir_fd` and `O_NOFOLLOW` support.
|
|
12905
14047
|
These primitives are the platform prerequisite for pinning directories between
|
|
12906
14048
|
a trusted configuration root and the file being changed, so a symlink swap
|
|
12907
14049
|
cannot redirect a mutation outside that root. Absolute trusted roots are opened
|