@softspark/ai-toolkit 4.19.1 → 4.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +121 -0
- package/LICENSE +201 -21
- package/NOTICE +55 -0
- package/README.md +35 -21
- package/app/.claude-plugin/plugin.json +2 -2
- package/app/hooks/_hook-io.sh +4 -0
- package/app/hooks/_locate-toolkit.sh +4 -0
- package/app/hooks/_profile-check.sh +4 -0
- package/app/hooks/_search-capability.sh +4 -0
- package/app/hooks/_session-paths.sh +4 -0
- package/app/hooks/ai-toolkit-statusline.sh +4 -0
- package/app/hooks/commit-quality.sh +4 -0
- package/app/hooks/config-desync-guard.sh +4 -0
- package/app/hooks/governance-capture.sh +4 -0
- package/app/hooks/guard-config.sh +4 -0
- package/app/hooks/guard-destructive.sh +4 -0
- package/app/hooks/guard-path.sh +4 -0
- package/app/hooks/instructions-audit.sh +4 -0
- package/app/hooks/loop-guard.sh +4 -0
- package/app/hooks/mcp-health.sh +4 -0
- package/app/hooks/notify-waiting.sh +4 -0
- package/app/hooks/post-tool-use.sh +4 -0
- package/app/hooks/pre-compact-save.sh +4 -0
- package/app/hooks/pre-compact.sh +4 -0
- package/app/hooks/quality-check.sh +4 -0
- package/app/hooks/quality-gate.sh +4 -0
- package/app/hooks/revert-guard.sh +4 -0
- package/app/hooks/save-session.sh +4 -0
- package/app/hooks/search-tracker.sh +4 -0
- package/app/hooks/session-end.sh +4 -0
- package/app/hooks/session-start.sh +4 -0
- package/app/hooks/stop-search-check.sh +4 -0
- package/app/hooks/subagent-start.sh +4 -0
- package/app/hooks/subagent-stop.sh +4 -0
- package/app/hooks/test-cohesion.sh +4 -0
- package/app/hooks/track-usage.sh +4 -0
- package/app/hooks/user-prompt-submit.sh +4 -0
- package/app/plugins/README.md +15 -15
- package/app/plugins/enterprise-pack/hooks/output-style.sh +4 -0
- package/app/plugins/enterprise-pack/hooks/status-line.sh +4 -0
- package/app/plugins/memory-pack/hooks/observation-capture.sh +4 -0
- package/app/plugins/memory-pack/hooks/session-summary.sh +4 -0
- package/app/plugins/memory-pack/scripts/init_db.py +4 -0
- package/app/plugins/memory-pack/scripts/strip_private.py +4 -0
- package/app/skills/_lib/__init__.py +4 -0
- package/app/skills/_lib/detect_utils.py +4 -0
- package/app/skills/a11y-validate/scripts/a11y-scanner.py +4 -0
- package/app/skills/analyze/scripts/complexity.py +4 -0
- package/app/skills/brand-voice/scripts/measure.py +4 -0
- package/app/skills/build/scripts/detect-build.py +4 -0
- package/app/skills/ci/scripts/ci-detect.py +4 -0
- package/app/skills/commit/scripts/pre-commit-check.py +4 -0
- package/app/skills/cve-scan/scripts/cve_scan.py +4 -0
- package/app/skills/debug/scripts/error-parser.py +4 -0
- package/app/skills/deploy/scripts/pre_deploy_check.py +4 -0
- package/app/skills/docs/scripts/doc-inventory.py +4 -0
- package/app/skills/documentation-standards/SKILL.md +25 -5
- package/app/skills/explain/scripts/dependency-graph.py +4 -0
- package/app/skills/explore/scripts/visualize.py +4 -0
- package/app/skills/fix/scripts/error-classifier.py +4 -0
- package/app/skills/health/scripts/health_check.py +4 -0
- package/app/skills/hipaa-validate/scripts/hipaa_scan.py +4 -0
- package/app/skills/lint/scripts/detect-linters.py +4 -0
- package/app/skills/migrate/scripts/migration-status.py +4 -0
- package/app/skills/plugin-creator/SKILL.md +18 -8
- package/app/skills/pr/scripts/pr-summary.py +4 -0
- package/app/skills/refactor/scripts/refactor-scan.py +4 -0
- package/app/skills/review/scripts/diff-analyzer.py +4 -0
- package/app/skills/rollback/scripts/rollback_info.py +4 -0
- package/app/skills/seo-validate/scripts/seo-scanner.py +4 -0
- package/app/skills/skill-creator/templates/reasoning-engine/search.py +4 -0
- package/app/skills/test/scripts/detect-runner.py +4 -0
- package/app/skills/write-a-prd/scripts/poll.js +4 -0
- package/benchmarks/ecosystem-doctor-snapshot.json +3 -3
- package/benchmarks/run.py +4 -0
- package/bin/ai-toolkit.js +4 -0
- package/kb/{planning/drop-cascade-hooks-after-sunset.md → history/completed/drop-cascade-hooks-after-sunset-20260710.md} +24 -0
- package/kb/{reference/language-packs.md → history/completed/language-packs-removed-20260727.md} +32 -7
- package/kb/{planning/mcp-context-trim-v4-prd.md → history/completed/mcp-context-trim-v4-prd-obsoleted-20260727.md} +43 -9
- package/kb/history/completed/no-op-plugin-packs-removed-20260727.md +127 -0
- package/kb/history/completed/tool-output-token-reduction-closed-20260727.md +190 -0
- package/kb/procedures/release-preparation-sop.md +83 -5
- package/kb/reference/comparison.md +1 -1
- package/kb/reference/enterprise-config-guide.md +1 -1
- package/kb/reference/licensing.md +148 -0
- package/kb/reference/plugin-pack-conventions.md +24 -16
- package/llms-full.txt +1182 -530
- package/llms.txt +6 -3
- package/manifest.json +1 -1
- package/package.json +3 -2
- package/scripts/_common.py +4 -0
- package/scripts/add_rule.py +4 -0
- package/scripts/audit_skills.py +4 -0
- package/scripts/benchmark_config.py +4 -0
- package/scripts/benchmark_ecosystem.py +4 -0
- package/scripts/check_deps.py +4 -0
- package/scripts/claude_app.py +4 -0
- package/scripts/codex_skill_adapter.py +4 -0
- package/scripts/compile_slm.py +4 -0
- package/scripts/config_cli.py +4 -0
- package/scripts/config_lock.py +4 -0
- package/scripts/config_merger.py +4 -0
- package/scripts/config_resolver.py +4 -0
- package/scripts/config_scaffold.py +5 -1
- package/scripts/config_validator.py +4 -0
- package/scripts/create_skill.py +4 -0
- package/scripts/dir_rules_shared.py +4 -0
- package/scripts/doctor.py +4 -0
- package/scripts/ecosystem_doctor.py +4 -0
- package/scripts/eject.py +4 -0
- package/scripts/emission.py +4 -0
- package/scripts/evaluate_skills.py +4 -0
- package/scripts/frontmatter.py +4 -0
- package/scripts/generate_agents_md.py +4 -0
- package/scripts/generate_aider_conf.py +4 -0
- package/scripts/generate_antigravity.py +4 -0
- package/scripts/generate_augment.py +4 -0
- package/scripts/generate_augment_agents.py +4 -0
- package/scripts/generate_augment_commands.py +4 -0
- package/scripts/generate_augment_hooks.py +4 -0
- package/scripts/generate_augment_rules.py +4 -0
- package/scripts/generate_augment_skills.py +4 -0
- package/scripts/generate_cline.py +4 -0
- package/scripts/generate_cline_rules.py +4 -0
- package/scripts/generate_cline_skills.py +4 -0
- package/scripts/generate_codex.py +4 -0
- package/scripts/generate_codex_agents.py +4 -0
- package/scripts/generate_codex_hooks.py +4 -0
- package/scripts/generate_codex_skills.py +4 -0
- package/scripts/generate_conventions.py +4 -0
- package/scripts/generate_copilot.py +4 -0
- package/scripts/generate_copilot_hooks.py +4 -0
- package/scripts/generate_cursor_agents.py +4 -0
- package/scripts/generate_cursor_hooks.py +4 -0
- package/scripts/generate_cursor_mdc.py +4 -0
- package/scripts/generate_cursor_rules.py +4 -0
- package/scripts/generate_cursor_skills.py +4 -0
- package/scripts/generate_devin_hooks.py +4 -0
- package/scripts/generate_gemini.py +4 -0
- package/scripts/generate_gemini_commands.py +4 -0
- package/scripts/generate_gemini_hooks.py +4 -0
- package/scripts/generate_gemini_skills.py +4 -0
- package/scripts/generate_language_rules_skills.py +4 -0
- package/scripts/generate_llms_txt.py +4 -0
- package/scripts/generate_opencode.py +4 -0
- package/scripts/generate_opencode_agents.py +4 -0
- package/scripts/generate_opencode_commands.py +4 -0
- package/scripts/generate_opencode_json.py +4 -0
- package/scripts/generate_opencode_plugin.py +4 -0
- package/scripts/generate_roo_modes.py +4 -0
- package/scripts/generate_roo_rules.py +4 -0
- package/scripts/generate_windsurf.py +4 -0
- package/scripts/generate_windsurf_rules.py +4 -0
- package/scripts/generate_windsurf_skills.py +4 -0
- package/scripts/generator_base.py +4 -0
- package/scripts/harvest_ecosystem.py +4 -0
- package/scripts/hook_sources.py +4 -0
- package/scripts/inject_hook_cli.py +4 -0
- package/scripts/inject_mcp_cli.py +4 -0
- package/scripts/inject_rule_cli.py +4 -0
- package/scripts/inject_section_cli.py +4 -0
- package/scripts/injection.py +4 -0
- package/scripts/install.py +4 -0
- package/scripts/install_git_hooks.py +4 -0
- package/scripts/install_steps/__init__.py +4 -0
- package/scripts/install_steps/ai_tools.py +4 -0
- package/scripts/install_steps/detect_language.py +4 -0
- package/scripts/install_steps/hooks.py +4 -0
- package/scripts/install_steps/install_state.py +4 -0
- package/scripts/install_steps/markers.py +4 -0
- package/scripts/install_steps/project_registry.py +4 -0
- package/scripts/install_steps/symlinks.py +4 -0
- package/scripts/instruction_core.py +4 -0
- package/scripts/mcp_editors.py +4 -0
- package/scripts/mcp_manager.py +4 -0
- package/scripts/mcp_sources.py +4 -0
- package/scripts/merge-hooks.py +4 -0
- package/scripts/migrate.py +4 -0
- package/scripts/output_filter_retirement.py +4 -0
- package/scripts/pack_codebase.py +4 -0
- package/scripts/paths.py +4 -0
- package/scripts/plugin.py +4 -0
- package/scripts/plugin_schema.py +4 -0
- package/scripts/projects_cli.py +4 -0
- package/scripts/propagate_global.py +4 -0
- package/scripts/remove_rule.py +4 -0
- package/scripts/rule_sources.py +4 -0
- package/scripts/secure_fs.py +4 -0
- package/scripts/session_state.py +4 -0
- package/scripts/session_token_stats.py +4 -0
- package/scripts/skill_pointer.py +4 -0
- package/scripts/stats.py +4 -0
- package/scripts/sync.py +4 -0
- package/scripts/test_cohesion.py +4 -0
- package/scripts/uninstall.py +4 -0
- package/scripts/update_projects.py +4 -0
- package/scripts/url_fetch.py +4 -0
- package/scripts/validate.py +37 -1
- package/scripts/version_check.py +4 -0
- package/app/plugins/csharp-pack/README.md +0 -11
- package/app/plugins/csharp-pack/plugin.json +0 -18
- package/app/plugins/frontend-pack/README.md +0 -14
- package/app/plugins/frontend-pack/plugin.json +0 -22
- package/app/plugins/java-pack/README.md +0 -11
- package/app/plugins/java-pack/plugin.json +0 -18
- package/app/plugins/kotlin-pack/README.md +0 -11
- package/app/plugins/kotlin-pack/plugin.json +0 -18
- package/app/plugins/research-pack/README.md +0 -14
- package/app/plugins/research-pack/plugin.json +0 -22
- package/app/plugins/ruby-pack/README.md +0 -11
- package/app/plugins/ruby-pack/plugin.json +0 -18
- package/app/plugins/rust-pack/README.md +0 -11
- package/app/plugins/rust-pack/plugin.json +0 -18
- package/app/plugins/security-pack/README.md +0 -15
- package/app/plugins/security-pack/plugin.json +0 -23
- package/app/plugins/swift-pack/README.md +0 -11
- package/app/plugins/swift-pack/plugin.json +0 -18
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
3
|
+
# Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
|
|
4
|
+
# Source: https://github.com/softspark/ai-toolkit
|
|
5
|
+
|
|
2
6
|
"""SEO scanner -- pattern-matching heuristics for common SEO issues.
|
|
3
7
|
|
|
4
8
|
Stdlib only. No external dependencies.
|
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
3
|
+
# Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
|
|
4
|
+
# Source: https://github.com/softspark/ai-toolkit
|
|
5
|
+
|
|
2
6
|
"""Generic domain reasoning engine for ai-toolkit skills.
|
|
3
7
|
|
|
4
8
|
Multi-domain search across a categorized knowledge base with anti-pattern filtering.
|
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
3
|
+
# Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
|
|
4
|
+
# Source: https://github.com/softspark/ai-toolkit
|
|
5
|
+
|
|
2
6
|
"""Auto-detect test framework and runner from project configuration files.
|
|
3
7
|
|
|
4
8
|
Scans a project directory for well-known config files (pyproject.toml,
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"last_run": "2026-07-
|
|
2
|
+
"last_run": "2026-07-27T11:02:34Z",
|
|
3
3
|
"schema_version": 1,
|
|
4
4
|
"tools": {
|
|
5
5
|
"aider": {
|
|
@@ -66,7 +66,7 @@
|
|
|
66
66
|
}
|
|
67
67
|
},
|
|
68
68
|
"claude-app": {
|
|
69
|
-
"docs_hash": "
|
|
69
|
+
"docs_hash": "08443e412637cd52",
|
|
70
70
|
"headings": [
|
|
71
71
|
"Add global and folder instructions",
|
|
72
72
|
"Availability",
|
|
@@ -364,7 +364,7 @@
|
|
|
364
364
|
}
|
|
365
365
|
},
|
|
366
366
|
"gemini-cli": {
|
|
367
|
-
"docs_hash": "
|
|
367
|
+
"docs_hash": "6e930c2fa0b827f5",
|
|
368
368
|
"headings": [
|
|
369
369
|
"Breadcrumbs",
|
|
370
370
|
"Directory actions",
|
package/benchmarks/run.py
CHANGED
package/bin/ai-toolkit.js
CHANGED
|
@@ -22,6 +22,30 @@ description: "Completed cleanup of the deprecated Windsurf Cascade hooks generat
|
|
|
22
22
|
|
|
23
23
|
**Completed in v4.13.0 (2026-07-10).** The deprecated generator and install/test wiring were removed; the Devin hook generator remains.
|
|
24
24
|
|
|
25
|
+
> **Filed to history 2026-07-27.** The plan was finished in v4.13.0 but sat in
|
|
26
|
+
> `kb/planning/` for another two and a half weeks, where it read as open work.
|
|
27
|
+
> Before moving it, every claim in the Verification section below was re-run
|
|
28
|
+
> against the codebase:
|
|
29
|
+
>
|
|
30
|
+
> | Check | Result |
|
|
31
|
+
> |---|---|
|
|
32
|
+
> | `scripts/generate_windsurf_hooks.py` deleted | gone |
|
|
33
|
+
> | `grep -rn generate_windsurf_hooks scripts/ tests/` | no source references (one stale `.pyc` under gitignored `__pycache__`) |
|
|
34
|
+
> | `_HOOK_STEM_ALIAS = {"devin": "windsurf"}` retained in `validate.py` | present, line 882 |
|
|
35
|
+
> | windsurf `our_generators` in `scripts/ecosystem_tools.json` | Cascade generator absent, `generate_devin_hooks.py` present |
|
|
36
|
+
> | `kb/reference/supported-tools-registry.md` hooks-migration row | reads "Complete" in past tense |
|
|
37
|
+
> | `validate.py --strict` | 0 errors, 0 warnings |
|
|
38
|
+
>
|
|
39
|
+
> The work was real and complete. Only the filing was wrong.
|
|
40
|
+
>
|
|
41
|
+
> **Process note.** This is the second document found misfiled in
|
|
42
|
+
> `kb/planning/` on the same day; the other,
|
|
43
|
+
> [`mcp-context-trim-v4-prd-obsoleted-20260727.md`](mcp-context-trim-v4-prd-obsoleted-20260727.md),
|
|
44
|
+
> had gone obsolete because the platform shipped the fix first. A plan with a
|
|
45
|
+
> `trigger_date` needs a matching move-to-history step in the release that acts
|
|
46
|
+
> on it — otherwise `kb/planning/` accumulates work that is already done or no
|
|
47
|
+
> longer worth doing, and the next reader cannot tell which is which.
|
|
48
|
+
|
|
25
49
|
## Why this exists
|
|
26
50
|
|
|
27
51
|
Windsurf rebranded to Devin Desktop on 2026-06-02. The Cascade agent — and its
|
package/kb/{reference/language-packs.md → history/completed/language-packs-removed-20260727.md}
RENAMED
|
@@ -1,15 +1,40 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "AI Toolkit - Language Plugin Packs"
|
|
3
|
-
category:
|
|
2
|
+
title: "AI Toolkit - Language Plugin Packs (removed)"
|
|
3
|
+
category: planning
|
|
4
4
|
service: ai-toolkit
|
|
5
|
-
tags: [plugins, languages, rust, java, csharp, kotlin, swift, ruby]
|
|
6
|
-
|
|
5
|
+
tags: [plugins, languages, rust, java, csharp, kotlin, swift, ruby, removed]
|
|
6
|
+
doc_type: postmortem
|
|
7
|
+
status: completed
|
|
8
|
+
version: "1.1.0"
|
|
7
9
|
created: "2026-03-29"
|
|
8
|
-
last_updated: "2026-
|
|
9
|
-
description: "
|
|
10
|
+
last_updated: "2026-07-27"
|
|
11
|
+
description: "REMOVED 2026-07-27 in v4.20.0. The six language packs wrapped skills that already ship in core, so installing one put zero files on disk. The skills themselves (rust-patterns, java-patterns, csharp-patterns, kotlin-patterns, swift-patterns, ruby-patterns) were never in the packs and are unaffected."
|
|
10
12
|
---
|
|
11
13
|
|
|
12
|
-
# Language Plugin Packs
|
|
14
|
+
# Language Plugin Packs — REMOVED
|
|
15
|
+
|
|
16
|
+
> **Removed 2026-07-27 in v4.20.0. Nothing was lost.**
|
|
17
|
+
>
|
|
18
|
+
> The six packs described below each declared exactly one skill —
|
|
19
|
+
> `rust-patterns`, `java-patterns`, `csharp-patterns`, `kotlin-patterns`,
|
|
20
|
+
> `swift-patterns`, `ruby-patterns` — and every one of those skills lives in
|
|
21
|
+
> `app/skills/`, is part of the core install, and **still does**. The packs
|
|
22
|
+
> owned no files of their own beyond a `README.md`.
|
|
23
|
+
>
|
|
24
|
+
> Because `ai-toolkit install` links every core skill, installing a language
|
|
25
|
+
> pack put **zero** files on disk. Measured on both runtimes and all three
|
|
26
|
+
> profiles: `(0 file items)` every time.
|
|
27
|
+
>
|
|
28
|
+
> **If you used one of these packs, you lose nothing.** The skill it named is
|
|
29
|
+
> already installed and triggers on the same file types it always did — open a
|
|
30
|
+
> `.rs` file and `rust-patterns` still loads.
|
|
31
|
+
>
|
|
32
|
+
> Full measurement and the rule that now prevents a repeat:
|
|
33
|
+
> [`no-op-plugin-packs-removed-20260727.md`](no-op-plugin-packs-removed-20260727.md).
|
|
34
|
+
>
|
|
35
|
+
> The document below is preserved as written on 2026-03-29.
|
|
36
|
+
|
|
37
|
+
---
|
|
13
38
|
|
|
14
39
|
## Overview
|
|
15
40
|
|
|
@@ -10,23 +10,56 @@ tags:
|
|
|
10
10
|
- tokens
|
|
11
11
|
- v4
|
|
12
12
|
doc_type: plan
|
|
13
|
-
status:
|
|
13
|
+
status: obsoleted
|
|
14
14
|
created: "2026-05-04"
|
|
15
|
-
last_updated: "2026-
|
|
16
|
-
completion: "0%"
|
|
17
|
-
target_milestone: "v4.0"
|
|
15
|
+
last_updated: "2026-07-27"
|
|
16
|
+
completion: "0% — never started, and no longer worth starting"
|
|
17
|
+
target_milestone: "v4.0 (abandoned)"
|
|
18
18
|
predecessor:
|
|
19
19
|
- "kb/history/completed/output-token-discipline-plan-20260504.md"
|
|
20
20
|
- "kb/history/completed/f2-mcp-trim-spike-20260504.md"
|
|
21
|
-
description: "Local MCP proxy
|
|
21
|
+
description: "ABANDONED 2026-07-27. Local MCP proxy that would compress tool descriptions before they reach the model. Never built. Claude Code now defers MCP tool schemas by default — the catalog costs ~120 tokens of tool names, with schemas fetched on demand — so the 8-15k per-turn overhead this PRD was written to remove no longer exists. Kept as the record of a plan the platform solved first."
|
|
22
22
|
---
|
|
23
23
|
|
|
24
|
-
# PRD: MCP Context Trim v4.0
|
|
24
|
+
# PRD: MCP Context Trim v4.0 — ABANDONED
|
|
25
|
+
|
|
26
|
+
> **Abandoned 2026-07-27, never implemented.**
|
|
27
|
+
>
|
|
28
|
+
> **Why:** the premise expired. This PRD is built on one measured claim — that
|
|
29
|
+
> MCP tool descriptions sit in every turn's system prompt, costing 8–15k tokens
|
|
30
|
+
> per turn. Claude Code now **defers MCP tool schemas by default**: the model
|
|
31
|
+
> receives roughly 120 tokens of tool *names*, and full schemas are fetched on
|
|
32
|
+
> demand through tool search when a task actually needs one.
|
|
33
|
+
> ([Claude Code docs — context window](https://code.claude.com/docs/en/context-window),
|
|
34
|
+
> `ENABLE_TOOL_SEARCH=auto|false` controls the older eager behaviour.)
|
|
35
|
+
>
|
|
36
|
+
> The overhead this proxy was designed to remove is already gone. Building it
|
|
37
|
+
> now would add a supervised local daemon, an `.mcp.json` rewrite, a rollback
|
|
38
|
+
> path and five pre-mortem failure modes in order to compress a 120-token
|
|
39
|
+
> catalog.
|
|
40
|
+
>
|
|
41
|
+
> **How it was caught:** during the 2026-07-27 token-reduction review, a
|
|
42
|
+
> measurement of 1189 real sessions put the median startup context at 22,175
|
|
43
|
+
> tokens against ~7,850 in the documented reference shape. Attributing that gap
|
|
44
|
+
> showed the excess was skill and agent descriptions and rule files — not MCP.
|
|
45
|
+
> The MCP catalog was already deferred, exactly as the docs describe. Full
|
|
46
|
+
> context: [`tool-output-token-reduction-closed-20260727.md`](tool-output-token-reduction-closed-20260727.md).
|
|
47
|
+
>
|
|
48
|
+
> **What survives:** nothing in the architecture below is reusable, because it
|
|
49
|
+
> exists to solve a problem the host now solves. What survives is the process
|
|
50
|
+
> note — this PRD sat in `kb/planning/` for nearly three months while the
|
|
51
|
+
> platform shipped the fix, and nobody re-checked the premise. A plan blocked on
|
|
52
|
+
> a host limitation should carry a re-validation date, not wait indefinitely.
|
|
53
|
+
>
|
|
54
|
+
> Everything below is the document as written on 2026-05-04. It is preserved
|
|
55
|
+
> unedited so the reasoning stays auditable.
|
|
25
56
|
|
|
26
|
-
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
**Status:** Proposed *(as of 2026-05-04; see abandonment note above)*
|
|
27
60
|
**Target milestone:** v4.0
|
|
28
|
-
**Carved out of:** [`output-token-discipline-plan-20260504.md`](
|
|
29
|
-
**Spike basis:** [`f2-mcp-trim-spike-20260504.md`](
|
|
61
|
+
**Carved out of:** [`output-token-discipline-plan-20260504.md`](output-token-discipline-plan-20260504.md) (was Feature 2)
|
|
62
|
+
**Spike basis:** [`f2-mcp-trim-spike-20260504.md`](f2-mcp-trim-spike-20260504.md)
|
|
30
63
|
|
|
31
64
|
## Problem
|
|
32
65
|
|
|
@@ -156,3 +189,4 @@ Applied to each tool description in `tools/list` response:
|
|
|
156
189
|
| Date | Status | Author |
|
|
157
190
|
|------|--------|--------|
|
|
158
191
|
| 2026-05-04 | PRD drafted from spike conclusions, carved out of v3.2.0 plan | claude |
|
|
192
|
+
| 2026-07-27 | Abandoned without implementation. Claude Code began deferring MCP tool schemas by default, removing the per-turn overhead this proxy targeted. Moved from `kb/planning/` to `kb/history/completed/`. | claude |
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Removed: Nine Plugin Packs That Installed Nothing"
|
|
3
|
+
category: planning
|
|
4
|
+
service: ai-toolkit
|
|
5
|
+
tags:
|
|
6
|
+
- plugins
|
|
7
|
+
- plugin-pack
|
|
8
|
+
- measurement
|
|
9
|
+
- postmortem
|
|
10
|
+
- dead-code
|
|
11
|
+
doc_type: postmortem
|
|
12
|
+
status: completed
|
|
13
|
+
created: "2026-07-27"
|
|
14
|
+
last_updated: "2026-07-27"
|
|
15
|
+
shipped_in: "v4.20.0 (removal)"
|
|
16
|
+
description: "Nine of eleven plugin packs installed zero files, because every asset they declared already ships in the core install. Measured across both runtimes and all three profiles. Records what was removed, what survived, why the authoring guidance produced the problem, and the check that now prevents a repeat."
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Removed: Nine Plugin Packs That Installed Nothing
|
|
20
|
+
|
|
21
|
+
`csharp-pack`, `java-pack`, `kotlin-pack`, `ruby-pack`, `rust-pack`,
|
|
22
|
+
`swift-pack`, `frontend-pack`, `research-pack`, `security-pack` — removed in
|
|
23
|
+
v4.20.0. `memory-pack` and `enterprise-pack` stay.
|
|
24
|
+
|
|
25
|
+
## The measurement
|
|
26
|
+
|
|
27
|
+
`ai-toolkit install` into a throwaway `HOME`, then `plugin install <pack>`, on
|
|
28
|
+
each runtime. The number is what `plugin install` itself reports:
|
|
29
|
+
|
|
30
|
+
| Pack | claude | codex |
|
|
31
|
+
|---|---:|---:|
|
|
32
|
+
| `csharp-pack` | 0 | 0 |
|
|
33
|
+
| `java-pack` | 0 | 0 |
|
|
34
|
+
| `kotlin-pack` | 0 | 0 |
|
|
35
|
+
| `ruby-pack` | 0 | 0 |
|
|
36
|
+
| `rust-pack` | 0 | 0 |
|
|
37
|
+
| `swift-pack` | 0 | 0 |
|
|
38
|
+
| `research-pack` | 0 | 0 |
|
|
39
|
+
| `security-pack` | 0 | 0 |
|
|
40
|
+
| `frontend-pack` | 0 | **1** |
|
|
41
|
+
| **`memory-pack`** | **4** | **4** |
|
|
42
|
+
| **`enterprise-pack`** | **2** | **2** |
|
|
43
|
+
|
|
44
|
+
Cross-checked against the filesystem, not just the reported count: after core
|
|
45
|
+
install (108 skills, 44 agents, 4742 characters of hook config), installing
|
|
46
|
+
`rust-pack`, `security-pack`, `research-pack` or any language pack produced
|
|
47
|
+
`+0` skills, `+0` agents, `+0` bytes of `settings.json` and `+0` files under
|
|
48
|
+
`~/.softspark`.
|
|
49
|
+
|
|
50
|
+
Profiles make no difference. `minimal`, `standard` and `strict` all install the
|
|
51
|
+
same 108 skills and 44 agents, and packs add `+0` to each.
|
|
52
|
+
|
|
53
|
+
## Why
|
|
54
|
+
|
|
55
|
+
Every one of the nine declared only assets that already ship in core:
|
|
56
|
+
|
|
57
|
+
| Pack | Declared | All present in core? |
|
|
58
|
+
|---|---|---|
|
|
59
|
+
| `rust-pack` | `rust-patterns` | yes |
|
|
60
|
+
| `security-pack` | `review`, `security-patterns`, `panic`, `security-auditor`, `security-architect`, `code-reviewer` | yes |
|
|
61
|
+
| `research-pack` | `docs`, `research-mastery`, `plan`, `technical-researcher`, `fact-checker`, `search-specialist` | yes |
|
|
62
|
+
|
|
63
|
+
Since `ai-toolkit install` links every core skill and agent, a manifest that
|
|
64
|
+
names only core assets resolves to a set of things already installed. There is
|
|
65
|
+
nothing left to do, so nothing is done.
|
|
66
|
+
|
|
67
|
+
Eight of the nine owned no file but `README.md`. `security-pack`'s two hooks
|
|
68
|
+
were core's `guard-destructive.sh` and `quality-gate.sh`; `research-pack`'s was
|
|
69
|
+
core's `user-prompt-submit.sh`.
|
|
70
|
+
|
|
71
|
+
## The one that was not quite zero
|
|
72
|
+
|
|
73
|
+
`frontend-pack` installed exactly one file, on codex only:
|
|
74
|
+
`plugin-frontend-pack-post-tool-use.sh`. The pack owned no such file — it
|
|
75
|
+
declared core's `post-tool-use.sh`, which core's codex surface does not install,
|
|
76
|
+
so the pack copied it in under a pack-prefixed name.
|
|
77
|
+
|
|
78
|
+
That is a generic hook wearing a domain label, not frontend functionality. It
|
|
79
|
+
was removed with the pack. **If `post-tool-use` should run on codex, it belongs
|
|
80
|
+
in the core codex hook set**, and adding it there is a separate, honest change —
|
|
81
|
+
not a side effect of installing a pack named after a UI framework.
|
|
82
|
+
|
|
83
|
+
## The authoring rule that caused this
|
|
84
|
+
|
|
85
|
+
`app/skills/plugin-creator/SKILL.md` told pack authors:
|
|
86
|
+
|
|
87
|
+
> **MUST** reference existing toolkit assets before duplicating — packs extend,
|
|
88
|
+
> they do not fork
|
|
89
|
+
|
|
90
|
+
Read literally against a core install that ships everything, that instruction
|
|
91
|
+
produces a no-op every time. It was correct about avoiding forks and silent
|
|
92
|
+
about the pack needing to add anything. Nine packs followed it exactly.
|
|
93
|
+
|
|
94
|
+
The rule now reads that a pack **must install something the core install does
|
|
95
|
+
not**, with the verification spelled out, and the validation checklist carries a
|
|
96
|
+
line item requiring a non-zero file count on every runtime the pack claims.
|
|
97
|
+
|
|
98
|
+
## What survived, and what users lose
|
|
99
|
+
|
|
100
|
+
**Nothing.** Every skill and agent the nine packs named is a core asset, still
|
|
101
|
+
present, still installed, still triggering on the same conditions. A Rust
|
|
102
|
+
developer who had `rust-pack` installed keeps `rust-patterns`, because it was
|
|
103
|
+
never in the pack.
|
|
104
|
+
|
|
105
|
+
The two remaining packs are the two that own files:
|
|
106
|
+
|
|
107
|
+
- `memory-pack` — two hooks, `init_db.py`, `strip_private.py`, and its own
|
|
108
|
+
`mem-search` skill. Verified working the same day: driving
|
|
109
|
+
`observation-capture.sh` with a real payload wrote a row to the SQLite store.
|
|
110
|
+
- `enterprise-pack` — `status-line.sh` and `output-style.sh`, both its own.
|
|
111
|
+
|
|
112
|
+
## Process note
|
|
113
|
+
|
|
114
|
+
The removal was nearly made on a wrong measurement. The first pass concluded
|
|
115
|
+
"nine no-ops on every editor" after testing only four packs on the claude
|
|
116
|
+
surface. `tests/test_plugin.bats` contradicted it by asserting that
|
|
117
|
+
`frontend-pack` creates a file on codex — the test was right and the
|
|
118
|
+
measurement was incomplete. The full 11 × 2 matrix was only then run.
|
|
119
|
+
|
|
120
|
+
The existing test suite caught an error in a fresh measurement. That is worth
|
|
121
|
+
remembering next time a test looks like it is merely in the way of a cleanup.
|
|
122
|
+
|
|
123
|
+
## Related
|
|
124
|
+
|
|
125
|
+
- [Plugin Pack Conventions](../../reference/plugin-pack-conventions.md) — the rule this postmortem installed
|
|
126
|
+
- [Language Packs (removed)](language-packs-removed-20260727.md) — the reference doc for six of the nine
|
|
127
|
+
- [rtk-pack Retirement](rtk-pack-retirement-20260727.md) — the pack removed the day before, for a different reason
|
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Closed: Tool-Output Token Reduction — Three Attempts, One Ceiling"
|
|
3
|
+
category: planning
|
|
4
|
+
service: ai-toolkit
|
|
5
|
+
tags:
|
|
6
|
+
- token-reduction
|
|
7
|
+
- measurement
|
|
8
|
+
- postmortem
|
|
9
|
+
- prompt-caching
|
|
10
|
+
- context-window
|
|
11
|
+
- closed-line-of-work
|
|
12
|
+
doc_type: postmortem
|
|
13
|
+
status: completed
|
|
14
|
+
created: "2026-07-27"
|
|
15
|
+
last_updated: "2026-07-27"
|
|
16
|
+
description: "Closes the tool-output token-reduction line of work after a third measurement. Decomposes 1189 real sessions by cost: 84% is context being fed to the model, 14.7% is responses. Tool output is a small lever by construction and three independent attempts have now hit the same ceiling. Records what shipped (a 20.2% rag-mcp response trim, worth 0.49% of cost), what was killed by its own kill number, and the three measurement errors made on the way."
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Closed: Tool-Output Token Reduction
|
|
20
|
+
|
|
21
|
+
**Read this before proposing a fourth attempt.**
|
|
22
|
+
|
|
23
|
+
Three independent efforts have tried to cut tokens by shrinking what tools
|
|
24
|
+
return. All three were competently built. All three measured out near zero.
|
|
25
|
+
|
|
26
|
+
| Attempt | Shipped | Measured saving |
|
|
27
|
+
|---|---|---:|
|
|
28
|
+
| [Native tool-output filter](output-filter-retirement-20260726.md) | v4.16.0, removed v4.17.0 | **0.0000%** |
|
|
29
|
+
| [rtk-pack](rtk-pack-retirement-20260727.md) | v4.18.0, removed v4.19.0 | **0.0615%** |
|
|
30
|
+
| This review's only clean win (rag-mcp response trim) | rag-mcp, 2026-07-27 | **0.49%** |
|
|
31
|
+
|
|
32
|
+
This is not three unlucky implementations. It is one structural fact, measured
|
|
33
|
+
three different ways.
|
|
34
|
+
|
|
35
|
+
## Where the money actually is
|
|
36
|
+
|
|
37
|
+
1189 sessions with traffic, 83,352 assistant turns, priced at Opus list rates:
|
|
38
|
+
|
|
39
|
+
| Component | Share of cost |
|
|
40
|
+
|---|---:|
|
|
41
|
+
| `cache_read` | **63.1%** |
|
|
42
|
+
| `cache_creation` | 20.9% |
|
|
43
|
+
| `output` | 14.7% |
|
|
44
|
+
| `input` (uncached) | 1.3% |
|
|
45
|
+
|
|
46
|
+
**84% of cost is feeding context to the model.** Responses are 14.7%. That
|
|
47
|
+
closes "make the model write less" as a serious lever — the whole `brand-voice`
|
|
48
|
+
concise mode plays for a seventh of the bill.
|
|
49
|
+
|
|
50
|
+
Cache hit rate is **97.4%**. Published guidance treats 80–95% as the achievable
|
|
51
|
+
band, so there is nothing to win in cache tuning either.
|
|
52
|
+
|
|
53
|
+
Session shape: median 29 turns (mean 70.1), median peak context 71,276 tokens
|
|
54
|
+
(mean 114,712), median startup context 22,175 tokens.
|
|
55
|
+
|
|
56
|
+
## Why tool output cannot be the lever
|
|
57
|
+
|
|
58
|
+
Decomposed by amplification — every tool result is re-read on every turn that
|
|
59
|
+
follows it, so a result's true cost is its size times the turns remaining:
|
|
60
|
+
|
|
61
|
+
| Category | Share of `cache_read` |
|
|
62
|
+
|---|---:|
|
|
63
|
+
| Fixed startup prelude | 12.4% |
|
|
64
|
+
| `Read` results | 15.6% |
|
|
65
|
+
| `Bash` results | 6.7% |
|
|
66
|
+
| `rag-mcp` results | 3.8% |
|
|
67
|
+
| Unattributed (assistant text, thinking, user messages, reminders) | ~61% |
|
|
68
|
+
|
|
69
|
+
Every tool in the toolbox, amplified across every turn, is **26.1%** of
|
|
70
|
+
`cache_read`, and it is not compressible without losing what it says. The
|
|
71
|
+
original Phase 0 calculation reached the same place from the other direction:
|
|
72
|
+
tool results are 4.54% of input-token volume, so 4.54% is the arithmetic
|
|
73
|
+
ceiling for any mechanism operating on them.
|
|
74
|
+
|
|
75
|
+
Raw tool-result bytes, 113 MB across 1308 session files:
|
|
76
|
+
|
|
77
|
+
| Tool | Share of bytes |
|
|
78
|
+
|---|---:|
|
|
79
|
+
| `Read` | 62.2% |
|
|
80
|
+
| `Bash` | 26.1% |
|
|
81
|
+
| `rag-mcp` (3 tools) | 7.6% |
|
|
82
|
+
| everything else | <2% each |
|
|
83
|
+
|
|
84
|
+
## What shipped
|
|
85
|
+
|
|
86
|
+
**rag-mcp response compaction** — `compact_payload()` in
|
|
87
|
+
`app/rag-mcp-server/routes/kb_search.py`, applied to `smart_query`,
|
|
88
|
+
`hybrid_search_kb` and `get_document`, plus both sides of the smart_query cache
|
|
89
|
+
so a hit and a miss return identical bytes.
|
|
90
|
+
|
|
91
|
+
Verified by running the shipped function over 1356 real responses captured from
|
|
92
|
+
session logs: **20.2% smaller, with no field the agent acts on removed.**
|
|
93
|
+
That is 0.77% of `cache_read`, **0.49% of total cost**.
|
|
94
|
+
|
|
95
|
+
It drops request echo (`use_hyde`/`use_crag`/`use_multi_hop`), null result
|
|
96
|
+
columns, `_from_cache: false`, `total_documents_used` when it equals the result
|
|
97
|
+
count, and the part of `source_documents_used` that merely repeats
|
|
98
|
+
`results[].kb_id`. It deliberately keeps `file_path` (addresses Read/Edit, where
|
|
99
|
+
`kb_id` addresses get_document), `routing` (the only signal for which pipeline
|
|
100
|
+
ran), `score: 0.0`, and empty result sets. 16 unit tests, most of them asserting
|
|
101
|
+
what must survive rather than what gets removed.
|
|
102
|
+
|
|
103
|
+
## What was killed, and by what
|
|
104
|
+
|
|
105
|
+
A kill number was published before the work: *if the shipped changes do not cut
|
|
106
|
+
`cache_read` by 3%, stop and do not proceed to the behavioural changes.*
|
|
107
|
+
|
|
108
|
+
Result: **0.79%.** The threshold was not met, and the remaining items were not
|
|
109
|
+
built. The kill number did its job — this is the first of the three attempts
|
|
110
|
+
where it bound before code was written rather than after it shipped.
|
|
111
|
+
|
|
112
|
+
## Three measurement errors, and what they cost
|
|
113
|
+
|
|
114
|
+
Recorded because each one nearly produced a wrong decision, and because two of
|
|
115
|
+
them are the same class of error that produced the previous two failures.
|
|
116
|
+
|
|
117
|
+
**1. Duplicate reads: 11.1% was actually 0.1%.** The first pass keyed duplicate
|
|
118
|
+
detection on `file_path` alone, counting re-reads of *different ranges* of the
|
|
119
|
+
same file as waste. Re-keyed on `(file_path, offset, limit)`:
|
|
120
|
+
|
|
121
|
+
| Key | Duplicates | Bytes | Share of `Read` |
|
|
122
|
+
|---|---:|---:|---:|
|
|
123
|
+
| path only (wrong) | 1754 | 7,787,530 | 11.1% |
|
|
124
|
+
| path + offset + limit (right) | **36** | **90,975** | **0.1%** |
|
|
125
|
+
|
|
126
|
+
A whole planned deliverable — a session-scoped dedup hook, with a designed
|
|
127
|
+
mitigation for the post-compaction re-read hazard — rested on that 11.1%. It
|
|
128
|
+
does not exist.
|
|
129
|
+
|
|
130
|
+
**2. A 58% "trim" that was deleting document content.** An aggressive variant of
|
|
131
|
+
the rag-mcp compaction measured 58% smaller. It was whitelisting top-level keys
|
|
132
|
+
and thereby dropping `content` — which for `get_document` *is* the document,
|
|
133
|
+
30.5% of all bytes those endpoints return. Not a trim; data loss that looked
|
|
134
|
+
like a win. The honest figure is 20.2%.
|
|
135
|
+
|
|
136
|
+
**3. "77.5% of the rag-mcp response is overhead."** Roughly half of that
|
|
137
|
+
non-content mass is `kb_id`, `file_path`, `title` and `score` — fields the agent
|
|
138
|
+
uses. Removable overhead is about 20%, not 77%.
|
|
139
|
+
|
|
140
|
+
The common thread: **every one of these errors made the opportunity look bigger
|
|
141
|
+
than it is, and every one was caught only by decomposing before building.** The
|
|
142
|
+
filter retirement drew the same conclusion about premise validation; rtk-pack
|
|
143
|
+
drew it about installing the artifact. This adds a third: decompose the metric
|
|
144
|
+
before trusting its headline.
|
|
145
|
+
|
|
146
|
+
## What is left, and why it was not taken
|
|
147
|
+
|
|
148
|
+
| Option | Value | Why not |
|
|
149
|
+
|---|---:|---|
|
|
150
|
+
| Model routing | tens of % | Ruled out by the maintainer — Opus 5 stays |
|
|
151
|
+
| Trim the toolkit's own startup prelude | ~2.5% of cost | Ruled out — costs skill/agent discoverability |
|
|
152
|
+
| `Read` with ranges instead of whole files | up to 10.4% of `cache_read` | The only remaining item of size, and **not quality-neutral**: it is a behavioural change whose effect on correctness cannot be measured automatically |
|
|
153
|
+
| Unattributed ~61% of `cache_read` | unknown | Assistant text, thinking blocks, user messages, system reminders — no clean cut available |
|
|
154
|
+
|
|
155
|
+
For the record, the startup prelude was measured rather than guessed. Median
|
|
156
|
+
startup context is 22,175 tokens against ~7,850 in the documented reference
|
|
157
|
+
shape, and the gap is the toolkit's own:
|
|
158
|
+
|
|
159
|
+
| Component | Count | ~tokens |
|
|
160
|
+
|---|---:|---:|
|
|
161
|
+
| skill descriptions | 108 | 5,051 |
|
|
162
|
+
| agent descriptions | 44 | 2,750 |
|
|
163
|
+
| project rules | 5 | 3,802 |
|
|
164
|
+
| global rules | 6 | 3,133 |
|
|
165
|
+
| `CLAUDE.md` files | 3 | 2,338 |
|
|
166
|
+
| **total** | | **17,076** |
|
|
167
|
+
|
|
168
|
+
The tool built to reduce tokens is the single largest addition to every
|
|
169
|
+
session's context. That is worth knowing, and it is still only 2.5% of cost,
|
|
170
|
+
because the prelude is 12.4% of `cache_read` and only part of it is removable.
|
|
171
|
+
|
|
172
|
+
## The rule this leaves behind
|
|
173
|
+
|
|
174
|
+
**Do not open a fourth tool-output token-reduction effort without first
|
|
175
|
+
producing a measurement that beats 4.54%.** That is the ceiling on this workload
|
|
176
|
+
and it has now been approached from three directions. Any proposal in this space
|
|
177
|
+
must state, before any code, which share of *input token volume* it addresses —
|
|
178
|
+
not which share of tool output, not which share of Bash bytes.
|
|
179
|
+
|
|
180
|
+
If token cost genuinely needs to come down, the levers that are actually large
|
|
181
|
+
are model selection and turn count. Both are policy decisions, not engineering
|
|
182
|
+
projects, and neither is in this line of work.
|
|
183
|
+
|
|
184
|
+
## Related
|
|
185
|
+
|
|
186
|
+
- [Output Filter Retirement](output-filter-retirement-20260726.md) — attempt one, 0.0000%
|
|
187
|
+
- [rtk-pack Retirement](rtk-pack-retirement-20260727.md) — attempt two, 0.0615%
|
|
188
|
+
- [rtk Pack Integration](rtk-pack-integration-20260726.md) — the Phase 0 ceiling calculation
|
|
189
|
+
- [MCP Context Trim v4.0 — abandoned](mcp-context-trim-v4-prd-obsoleted-20260727.md) — a fourth idea in this space, killed by the platform shipping the fix first
|
|
190
|
+
- [Output & Token Discipline](output-token-discipline-plan-20260504.md) — the plan all of this descends from
|