@softspark/ai-toolkit 4.30.3 → 4.32.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/CHANGELOG.md +123 -0
  2. package/README.md +32 -25
  3. package/app/.claude-plugin/plugin.json +1 -1
  4. package/app/agents/code-reviewer.md +14 -0
  5. package/app/agents/frontend-specialist.md +27 -24
  6. package/app/agents/infrastructure-validator.md +2 -2
  7. package/app/claude-app/hooks/hooks.json +4 -2
  8. package/app/claude-app/skills/ai-toolkit-rules/SKILL.md +74 -17
  9. package/app/hooks/quality-gate.sh +9 -2
  10. package/app/hooks.json +4 -2
  11. package/app/personas/frontend-lead.md +20 -12
  12. package/app/rules/claude-toolkit-rules.md +4 -1
  13. package/app/rules/common/git-team.md +33 -0
  14. package/app/rules/common/git-workflow.md +6 -20
  15. package/app/rules/common/performance.md +25 -1
  16. package/app/rules/common/testing.md +7 -1
  17. package/app/rules/edit-discipline.md +37 -0
  18. package/app/skills/app-builder/SKILL.md +37 -6
  19. package/app/skills/claude-toolkit-rules/SKILL.md +37 -0
  20. package/app/skills/design-engineering/SKILL.md +87 -102
  21. package/app/skills/design-engineering/reference/animation-recipes.md +67 -0
  22. package/app/skills/design-engineering/reference/component-craft.md +28 -0
  23. package/app/skills/documentation-standards/SKILL.md +1 -1
  24. package/app/skills/edit-discipline/SKILL.md +50 -0
  25. package/app/skills/git-conventions/SKILL.md +18 -0
  26. package/app/skills/output-mode/SKILL.md +41 -0
  27. package/app/skills/quality-gates/SKILL.md +23 -0
  28. package/app/skills/review/SKILL.md +10 -0
  29. package/benchmarks/ecosystem-doctor-snapshot.json +17 -15
  30. package/bin/ai-toolkit.js +2 -0
  31. package/kb/procedures/{ecosystem-sync-sop.md → sop-ecosystem-sync.md} +2 -2
  32. package/kb/procedures/{maintenance-sop.md → sop-maintenance.md} +6 -3
  33. package/kb/procedures/{post-release-testing-sop.md → sop-post-release-testing.md} +3 -3
  34. package/kb/procedures/sop-pre-commit.md +96 -0
  35. package/kb/procedures/{release-preparation-sop.md → sop-release.md} +6 -6
  36. package/kb/reference/architecture-overview.md +1 -1
  37. package/kb/reference/cli-reference.md +3 -2
  38. package/kb/reference/copilot-compatibility.md +1 -1
  39. package/kb/reference/global-install-model.md +16 -3
  40. package/kb/reference/hooks-catalog.md +5 -3
  41. package/kb/reference/language-rules.md +28 -10
  42. package/kb/reference/licensing.md +2 -2
  43. package/kb/reference/quick-wins-implementation-summary.md +1 -1
  44. package/kb/reference/supported-tools-registry.md +2 -2
  45. package/kb/reference/unique-features.md +2 -1
  46. package/kb/troubleshooting/plugin-double-load.md +1 -1
  47. package/llms-full.txt +1016 -871
  48. package/llms.txt +6 -5
  49. package/manifest.json +2 -2
  50. package/package.json +6 -2
  51. package/scripts/benchmark_ecosystem.py +0 -1
  52. package/scripts/check_split.py +11 -9
  53. package/scripts/claude_app.py +5 -7
  54. package/scripts/codex_skill_adapter.py +4 -12
  55. package/scripts/compile_slm.py +10 -26
  56. package/scripts/doctor.py +322 -0
  57. package/scripts/evaluate_skills.py +1 -1
  58. package/scripts/frontmatter.py +452 -29
  59. package/scripts/generate_augment_rules.py +4 -4
  60. package/scripts/generate_cursor_mdc.py +2 -3
  61. package/scripts/generate_language_rules_skills.py +8 -14
  62. package/scripts/generate_llms_txt.py +1 -15
  63. package/scripts/generate_opencode_agents.py +0 -1
  64. package/scripts/generate_opencode_skills.py +2 -20
  65. package/scripts/generate_toolkit_rules_skills.py +189 -0
  66. package/scripts/generate_windsurf_rules.py +0 -1
  67. package/scripts/generator_base.py +0 -1
  68. package/scripts/inject_hook_cli.py +15 -2
  69. package/scripts/inject_mcp_cli.py +1 -2
  70. package/scripts/install.py +32 -1
  71. package/scripts/install_git_hooks.py +0 -1
  72. package/scripts/install_steps/ai_tools.py +65 -25
  73. package/scripts/install_steps/markers.py +6 -6
  74. package/scripts/install_steps/skill_scope.py +188 -0
  75. package/scripts/instruction_core.py +5 -8
  76. package/scripts/merge-hooks.py +13 -3
  77. package/scripts/pack_codebase.py +1 -1
  78. package/scripts/surface_manifest.py +6 -7
  79. package/scripts/validate.py +181 -12
  80. /package/kb/procedures/{release-verification-sop.md → sop-release-verification.md} +0 -0
package/llms-full.txt CHANGED
@@ -32,11 +32,12 @@
32
32
  - [Closed: Tool-Output Token Reduction](kb/history/completed/tool-output-token-reduction-closed-20260727.md)
33
33
  - [How-To Guides](kb/howto/README.md)
34
34
  - [Plan: Cloud Security Pack — Multi-Cloud Audit](kb/planning/cloud-security-pack-plan.md)
35
- - [SOP: Ecosystem Sync](kb/procedures/ecosystem-sync-sop.md)
36
- - [SOP: AI Toolkit Maintenance](kb/procedures/maintenance-sop.md)
37
- - [SOP: Post-Release Testing](kb/procedures/post-release-testing-sop.md)
38
- - [SOP: Release Preparation](kb/procedures/release-preparation-sop.md)
39
- - [SOP: Release Verification](kb/procedures/release-verification-sop.md)
35
+ - [SOP: Ecosystem Sync](kb/procedures/sop-ecosystem-sync.md)
36
+ - [SOP: AI Toolkit Maintenance](kb/procedures/sop-maintenance.md)
37
+ - [SOP: Post-Release Testing](kb/procedures/sop-post-release-testing.md)
38
+ - [SOP: Pre-Commit Gate](kb/procedures/sop-pre-commit.md)
39
+ - [SOP: Release Verification](kb/procedures/sop-release-verification.md)
40
+ - [SOP: Release Preparation](kb/procedures/sop-release.md)
40
41
  - [Agents Catalog](kb/reference/agents-catalog.md)
41
42
  - [Anti-Pattern Registry Format](kb/reference/anti-pattern-registry-format.md)
42
43
  - [AI Toolkit Architecture](kb/reference/architecture-overview.md)
@@ -95,6 +96,7 @@
95
96
  - **chaos**: Injects controlled faults for resilience testing on non-prod. Triggers: chaos, fault injection, latency injection, dependency kill, resilience test.
96
97
  - **ci**: Detect/generate/debug CI pipeline config (GitHub Actions, GitLab CI). Triggers: CI setup, build pipeline, GitHub Actions config, debug CI, GitLab CI.
97
98
  - **ci-cd-patterns**: CI/CD: GitHub Actions, GitLab CI, Jenkins, caching, blue-green, canary. Triggers: CI, CD, pipeline, GitHub Actions, workflow YAML, release, canary, rollout.
99
+ - **claude-toolkit-rules**: SoftSpark working agreement: never guess a home directory path, give at least three alternatives, and apply a devil's advocate critique to decisions. Triggers: toolkit, conventions, workflow, alternatives, review.
98
100
  - **clean-code**: Code quality: meaningful names, SRP, DRY, small functions, guard clauses, refactoring. Triggers: clean code, naming, code smell, SRP, DRY, long function, god class, dead code.
99
101
  - **command-creator**: Creates new Claude Code slash commands with frontmatter and validation. Triggers: new slash command, create command, command scaffold.
100
102
  - **commit**: Creates Conventional Commits with pre-commit validation. Triggers: commit, conventional commit, git commit, message.
@@ -115,12 +117,14 @@
115
117
  - **docs**: Generates/updates README, API docs, architecture notes. Triggers: docs, README, API docs, architecture note, documentation.
116
118
  - **documentation-standards**: KB conventions: YAML frontmatter, 10-category taxonomy (reference/howto/procedures/troubleshooting/best-practices/decisions/runbooks/planning/business/templates). Triggers: kb/, SOP, runbook, howto, frontmatter, knowledge base.
117
119
  - **ecommerce-patterns**: E-commerce: cart, checkout, payments (Stripe/Adyen), order state, inventory, promos, tax. Triggers: cart, checkout, SKU, payment, Stripe, Shopify, Medusa, Magento, coupon, refund.
120
+ - **edit-discipline**: Change files with the edit and write tools, never by rewriting them through bash (sed, awk, tee, heredoc, redirection), and show `git diff` before reporting a file-changing task as done. Triggers: edit, write, modify, refactor, patch, fix, diff, review changes.
118
121
  - **evaluate**: Evaluates RAG retrieval and LLM-as-judge metrics (faithfulness, relevancy, context precision). Triggers: measure RAG quality, knowledge gap, RAG eval, golden dataset.
119
122
  - **evolve**: Analyzes agent/skill failures, drafts prompt/permission fixes. Triggers: improve agent, refine skill, system prompt, optimize agent.
120
123
  - **explain**: Explains code/architecture with Mermaid diagrams and sequence flows. Triggers: what does X do, how does Y work, explain code, sequence diagram.
121
124
  - **explore**: Explores codebase structure, stack, and architecture. Triggers: explore codebase, project structure, stack overview, architecture map.
122
125
  - **fix**: Applies targeted fix to known bug/lint error, verifies with same command that surfaced it. Triggers: fix, apply fix, fix bug, fix lint, targeted fix.
123
126
  - **flutter-patterns**: Flutter/Dart: widgets, state mgmt (Riverpod/Bloc), navigation, platform channels. Triggers: Flutter, Dart, widget, Riverpod, Bloc, pubspec, hot reload.
127
+ - **git-conventions**: Conventional Commits only (feat, fix, docs, refactor, test, chore); no AI co-authorship trailer and no AI signature in a commit message. Triggers: commit, git, message, changelog, release, pull request.
124
128
  - **git-mastery**: Advanced Git: rebase, bisect, reflog, cherry-pick, worktrees, LFS. Triggers: rebase, bisect, cherry-pick, reflog, force push, merge conflict, worktree.
125
129
  - **golang-rules**: Go coding rules: style, patterns, security, testing. Triggers: .go, go.mod, go.sum, Gin, Echo, Gorilla, testing, gofmt.
126
130
  - **grill-me**: Stress-tests a plan via Socratic questioning down each decision branch. Triggers: stress-test, grill me, validate assumptions, challenge plan, socratic review.
@@ -147,6 +151,7 @@
147
151
  - **observability-patterns**: Observability: structured logs, metrics (RED/USE), tracing, SLO/SLI. Triggers: logging, metrics, Prometheus, Grafana, OpenTelemetry, trace, monitoring.
148
152
  - **onboard**: Sets up ai-toolkit in a project: symlinks, CLAUDE.md, intent interview. Triggers: onboard, setup project, install ai-toolkit, migrate project.
149
153
  - **orchestrate**: Coordinates multiple specialized agents in parallel. Triggers: orchestrate, multi-agent, parallel agents, coordinate agents.
154
+ - **output-mode**: Answer concisely: lead with the result, no preamble, tables over prose, no trailing restatement of a diff the reader can already see. Triggers: response style, verbosity, summary, explanation, report.
150
155
  - **panic**: Emergency kill switch — halts all agents via lockfile gate. Triggers: panic, stop everything, kill switch, halt agents, agents looping.
151
156
  - **performance-profiling**: Performance: golden signals, p50/p95/p99, flame graphs, load testing. Triggers: performance, slow, latency, p99, flame graph, bottleneck, memory leak.
152
157
  - **persona**: Switches engineering persona at runtime. Triggers: persona, switch role, backend-lead, frontend-lead, devops-eng, junior-dev.
@@ -160,6 +165,7 @@
160
165
  - **prompt-caching-patterns**: Anthropic API prompt caching: TTL, breakpoints, stacking, invalidation, hit rate. Triggers: prompt caching, cache_control, cache breakpoint, cache TTL, hit rate.
161
166
  - **python-rules**: Python coding rules: style, patterns, security, testing. Triggers: .py, .pyi, pyproject.toml, requirements.txt, Pipfile, FastAPI, Django, Flask, pytest, SQLAlchemy, ruff, mypy.
162
167
  - **qa-session**: Interactive QA: user reports bugs conversationally, agent files GitHub issues. Triggers: QA session, report bug, file issue, conversational QA, bug intake.
168
+ - **quality-gates**: Plan before work over an hour, and hold the gates: ruff clean, mypy --strict clean, pytest coverage above 70 percent, no secrets in code. Triggers: quality, lint, mypy, pytest, coverage, gate, definition of done.
163
169
  - **rag-patterns**: RAG: embeddings, chunking, hybrid search (BM25+vector), reranking, CRAG, multi-hop. Triggers: RAG, embedding, pgvector, Qdrant, Pinecone, Weaviate, reranker, semantic search.
164
170
  - **refactor**: Refactors code for quality and maintainability. Triggers: refactor, clean up, restructure, improve code, modernize.
165
171
  - **refactor-plan**: Creates detailed refactor plan with tiny commits via interview, files as GitHub RFC. Triggers: refactor plan, refactoring RFC, incremental refactor, safe steps.
@@ -6929,7 +6935,7 @@ Examples:
6929
6935
 
6930
6936
  ---
6931
6937
 
6932
- ## kb/procedures/ecosystem-sync-sop.md
6938
+ ## kb/procedures/sop-ecosystem-sync.md
6933
6939
 
6934
6940
  ---
6935
6941
  title: "SOP: Ecosystem Sync"
@@ -7186,21 +7192,21 @@ jobs:
7186
7192
 
7187
7193
  - [Supported Tools Registry](../reference/supported-tools-registry.md) — human-readable per-tool breakdown
7188
7194
  - [MCP Editor Compatibility](../reference/mcp-editor-compatibility.md) — MCP-specific adapter table
7189
- - [Maintenance SOP](maintenance-sop.md) — general toolkit upkeep
7190
- - [Release Preparation SOP](release-preparation-sop.md) — run the doctor before tagging
7195
+ - [Maintenance SOP](sop-maintenance.md) — general toolkit upkeep
7196
+ - [Release Preparation SOP](sop-release.md) — run the doctor before tagging
7191
7197
 
7192
7198
  ---
7193
7199
 
7194
- ## kb/procedures/maintenance-sop.md
7200
+ ## kb/procedures/sop-maintenance.md
7195
7201
 
7196
7202
  ---
7197
7203
  title: "SOP: AI Toolkit Maintenance"
7198
7204
  category: procedures
7199
7205
  service: ai-toolkit
7200
7206
  tags: [sop, maintenance, agents, skills, install]
7201
- version: "3.4.0"
7207
+ version: "3.5.0"
7202
7208
  created: "2026-03-23"
7203
- last_updated: "2026-08-06"
7209
+ last_updated: "2026-09-04"
7204
7210
  description: "Standard operating procedures for installing, maintaining, and evolving the ai-toolkit."
7205
7211
  ---
7206
7212
 
@@ -7424,7 +7430,9 @@ that runtime should receive the change.
7424
7430
  greets the user with a traceback, and one that reads stdin must answer an
7425
7431
  empty stdin with an error rather than blocking forever.
7426
7432
  3. Update `kb/reference/skills-catalog.md` and `app/ARCHITECTURE.md`
7427
- 4. Run `scripts/validate.py` — it checks the invocation, the body budget, and
7433
+ 4. Run `scripts/validate.py` — it checks the invocation, the body budget, the
7434
+ description budget (warn over 400 characters, error over 1024; unquoted
7435
+ descriptions containing `: ` or ` #` are rejected, use `>-`), and
7428
7436
  `reference/` link resolution
7429
7437
  5. Run `python3 scripts/surface_manifest.py` before the next release to adopt the
7430
7438
  new skill into the protected surface
@@ -7500,6 +7508,7 @@ bats tests/test_install.bats tests/test_codex.bats
7500
7508
  bats tests/test_claude_app.bats tests/test_hooks_per_editor.bats
7501
7509
  python3 scripts/claude_app.py verify
7502
7510
  python3 scripts/validate.py --strict
7511
+ npm run test:py # pytest over scripts/ logic incl. the frontmatter corpus test (dev venv, see CONTRIBUTING.md)
7503
7512
  ```
7504
7513
 
7505
7514
  ## Managing Plugins
@@ -7705,7 +7714,7 @@ What `uninstall` does:
7705
7714
 
7706
7715
  ---
7707
7716
 
7708
- ## kb/procedures/post-release-testing-sop.md
7717
+ ## kb/procedures/sop-post-release-testing.md
7709
7718
 
7710
7719
  ---
7711
7720
  title: "SOP: Post-Release Testing"
@@ -7725,7 +7734,7 @@ actually install, from npm, rather than the working tree.
7725
7734
 
7726
7735
  Sibling procedures exist for `jira-mcp` and `legal-pl-pack`; this is the
7727
7736
  ai-toolkit equivalent. It complements
7728
- [Release Verification](release-verification-sop.md), which checks the toolkit
7737
+ [Release Verification](sop-release-verification.md), which checks the toolkit
7729
7738
  from the maintainer's own installed copy. The difference that matters: this one
7730
7739
  never writes to the maintainer's `~/.claude` or `~/.softspark`.
7731
7740
 
@@ -8019,1201 +8028,1302 @@ print(f'removed {sb} ({n} files)')
8019
8028
 
8020
8029
  ## Related
8021
8030
 
8022
- - [Release Preparation](release-preparation-sop.md) — run before tagging
8023
- - [Release Verification](release-verification-sop.md) — the maintainer-install checks
8031
+ - [Release Preparation](sop-release.md) — run before tagging
8032
+ - [Release Verification](sop-release-verification.md) — the maintainer-install checks
8024
8033
  - [rtk-pack Retirement](../history/completed/rtk-pack-retirement-20260727.md) — what happened the one time this SOP was written and not run
8025
8034
 
8026
8035
  ---
8027
8036
 
8028
- ## kb/procedures/release-preparation-sop.md
8037
+ ## kb/procedures/sop-pre-commit.md
8029
8038
 
8030
8039
  ---
8031
- title: "SOP: Release Preparation"
8040
+ title: "SOP: Pre-Commit Gate"
8032
8041
  category: procedures
8033
8042
  service: ai-toolkit
8034
- tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
8035
- version: "1.15.0"
8036
- created: "2026-04-10"
8043
+ tags: [pre-commit, quality-gate, shellcheck, bats, validation, secrets]
8044
+ version: "1.0.0"
8045
+ created: "2026-09-02"
8037
8046
  last_updated: "2026-09-02"
8038
- description: "Step-by-step checklist for preparing a new ai-toolkit release — ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, branch CI, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, checksum-pin, ShellCheck, licensing, exact-tag assertions, and a green Ubuntu/macOS branch-CI gate before any release tag is created."
8047
+ description: "The local gate to run before every commit to ai-toolkit: staged-file and secret scan, ShellCheck, Python syntax, toolkit validation, and the Bats suite. Runs the same checks CI runs, so main does not go red after a push."
8039
8048
  ---
8040
8049
 
8041
- # SOP: Release Preparation
8050
+ # SOP: Pre-Commit Gate
8042
8051
 
8043
- Complete checklist for preparing a new `@softspark/ai-toolkit` release.
8044
- Run this **before** tagging. After tagging and publishing, run the
8045
- [Release Verification SOP](release-verification-sop.md) to smoke-test.
8052
+ ## Why this exists
8046
8053
 
8047
- **Pipeline:**
8048
- ```
8049
- Ecosystem Sync SOP (drift check + generator updates)
8050
-
8051
- Release Preparation (this SOP)
8052
-
8053
- git tag → CI publish → Release Verification SOP
8054
- ```
8054
+ CI runs on `main` and on pull requests, but it runs *after* the push. Every
8055
+ check below already exists in `.github/workflows/ci.yml`; running them locally
8056
+ first is what keeps `main` green and keeps a release from becoming a debug
8057
+ session. The v4.5.1 postmortem in `sop-release.md` is the case
8058
+ where a hook lint failure published while reddening `main` CI.
8055
8059
 
8056
- **Time:** 10-20 minutes (includes ecosystem sync review)
8060
+ This gate is not a substitute for `sop-release.md`. That SOP gates
8061
+ a *tag*; this one gates a *commit*.
8057
8062
 
8058
- ---
8063
+ ## Prerequisites
8059
8064
 
8060
- ## Quick Checklist (TL;DR)
8065
+ `shellcheck`, `python3` and `bats` must be on `PATH`. The suite is the one
8066
+ check that silently does nothing when its runner is absent, so install it
8067
+ before relying on this gate:
8061
8068
 
8062
8069
  ```bash
8063
- # 0. Ecosystem sync (mandatory for minor/major releases; optional for patch)
8064
- # Full procedure: kb/procedures/ecosystem-sync-sop.md
8065
- python3 scripts/ecosystem_doctor.py --format text > /tmp/eco-report.txt
8066
- cat /tmp/eco-report.txt
8067
- # If drift detected: stop here, follow ecosystem-sync-sop.md Phase 2-4 to
8068
- # classify each drift (A-F), update generators as needed, refresh snapshot,
8069
- # THEN resume this SOP.
8070
- python3 scripts/ecosystem_doctor.py --update # after all drift resolved
8071
-
8072
- # 1. Decide version bump
8073
- # patch (1.4.2 → 1.4.3): bugfix, typo, doc fix
8074
- # minor (1.4.2 → 1.5.0): new feature, new skill, new flag, any ecosystem-class-B/F change
8075
- # major (1.4.2 → 2.0.0): breaking change, any ecosystem-class-D removed path
8076
-
8077
- # 2. Sync version across all files
8078
- python3 scripts/sync_version.py X.Y.Z # if script exists, else manual
8079
-
8080
- # 3. Write CHANGELOG.md entry
8081
- # 4. Regenerate artifacts
8082
- python3 scripts/generate_agents_md.py > AGENTS.md
8083
- python3 scripts/generate_llms_txt.py > llms.txt
8084
- python3 scripts/generate_llms_txt.py --full > llms-full.txt
8085
-
8086
- # 5. Validate + audit + SARIF + shellcheck + test + ecosystem check
8087
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif && shellcheck --severity=warning app/hooks/*.sh && npm test
8088
-
8089
- # 5a. Supply-chain standard (v2.8.0+) — non-negotiable
8090
- grep -q -- '--provenance' .github/workflows/publish.yml || { echo "MISSING --provenance"; exit 1; }
8091
- grep -q 'id-token: write' .github/workflows/publish.yml || { echo "MISSING id-token: write"; exit 1; }
8092
- python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8093
-
8094
- # 5b. Ecosystem gate — snapshot must be current before tag
8095
- python3 scripts/ecosystem_doctor.py --offline --check || { echo "STALE ecosystem snapshot — re-run doctor"; exit 1; }
8096
-
8097
- # 5c. Licensing gate — SPDX headers, LICENSE, NOTICE, manifest consistency
8098
- npx bats tests/test_licensing.bats || { echo "LICENSING GATE FAILED"; exit 1; }
8099
-
8100
- # 6. Commit, push branch, and wait for the exact commit's full CI
8101
- git add -A && git commit -m "chore: release vX.Y.Z"
8102
- RELEASE_SHA=$(git rev-parse HEAD)
8103
- git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
8104
- RUN_ID=""
8105
- for ATTEMPT in 1 2 3 4 5; do
8106
- RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
8107
- --limit 1 --json databaseId --jq '.[0].databaseId')
8108
- [ -n "$RUN_ID" ] && break
8109
- echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
8110
- [ "$ATTEMPT" -eq 5 ] || sleep 60
8111
- done
8112
- test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
8113
- gh run watch "$RUN_ID" --exit-status \
8114
- || { echo "FAIL: release commit CI failed"; exit 1; }
8115
- for OS in ubuntu-latest macos-latest; do
8116
- gh run view "$RUN_ID" --json jobs \
8117
- --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
8118
- | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
8119
- done
8120
- test "$(git rev-parse HEAD)" = "$RELEASE_SHA" || { echo "FAIL: local HEAD changed during CI"; exit 1; }
8121
- git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
8122
- test "$(git rev-parse origin/main)" = "$RELEASE_SHA" || { echo "FAIL: origin/main changed during CI"; exit 1; }
8123
-
8124
- # 6a. Only green branch CI authorizes tag creation
8125
- git tag vX.Y.Z "$RELEASE_SHA"
8126
-
8127
- # 6b. Assert the tag before pushing it (v4.19.0 postmortem, Phase 7)
8128
- test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" || { echo "FAIL: tag not on tested release commit"; exit 1; }
8129
- git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" || { echo "FAIL: tag not on release commit"; exit 1; }
8130
-
8131
- # 6c. Push the single tag by full ref. Never --tags.
8132
- git push origin refs/tags/vX.Y.Z
8070
+ brew install bats-core shellcheck # macOS
8071
+ sudo apt-get install -y bats shellcheck # Debian / Ubuntu
8133
8072
  ```
8134
8073
 
8135
- ---
8074
+ ## The gate
8136
8075
 
8137
- ## Phase 0: Ecosystem Sync (MANDATORY for minor/major)
8076
+ Run all five from the repository root. Each mirrors a CI job.
8138
8077
 
8139
- Before touching version numbers, confirm the toolkit is aligned with the current state of every editor / platform it integrates with. Skipping this phase ships a release whose generators may lag a month-old CLI refactor, a rename of `.cursorrules` to `.cursor/rules/`, or a new hook event we do not yet emit.
8078
+ ```bash
8079
+ # 1. Staged files: secrets, large blobs, commit-type suggestion
8080
+ python3 app/skills/commit/scripts/pre-commit-check.py
8140
8081
 
8141
- **When this phase is mandatory:**
8142
- - Minor release (X.Y.0) — always
8143
- - Major release (X.0.0) — always
8144
- - Patch release (X.Y.Z) — only if the patch touches a generator or install flow
8082
+ # 2. ShellCheck hooks -- the CI job that has published while red before
8083
+ shellcheck --severity=warning app/hooks/*.sh app/plugins/*/hooks/*.sh
8145
8084
 
8146
- **When to skip:** pure doc-only patches, SOP edits, internal refactors that do not touch `scripts/generate_*` or `app/skills/*/SKILL.md`.
8085
+ # 3. Python syntax across every script the toolkit ships
8086
+ python3 -m py_compile scripts/*.py app/skills/*/scripts/*.py
8147
8087
 
8148
- ### 0.1 Run the doctor
8088
+ # 4. Toolkit integrity: agents, skills, registry drift, content quality
8089
+ npm run validate
8149
8090
 
8150
- ```bash
8151
- python3 scripts/ecosystem_doctor.py --format text | tee /tmp/eco-report.txt
8091
+ # 5. The suite. Run it ONCE -- see the single-run discipline below
8092
+ npm test
8152
8093
  ```
8153
8094
 
8154
- Output classifies every registered tool as **Clean**, **Drift**, or **Errored**.
8095
+ A non-zero exit from any of them is a stop, not a warning to note and push past.
8155
8096
 
8156
- ### 0.2 Act on drift
8097
+ ## Single-run discipline
8157
8098
 
8158
- For each drifting tool, follow [ecosystem-sync-sop.md](ecosystem-sync-sop.md) Phase 2-4:
8099
+ `npm test` runs the Bats suite with `--jobs 4`. Running it repeatedly to see
8100
+ whether a failure is "flaky" hides real ordering bugs and wastes minutes. Run
8101
+ it once. If it fails, read the failure and fix the cause; if the same test
8102
+ passes on a re-run without a code change, that instability is itself the bug
8103
+ and belongs in an issue, not in a retry.
8159
8104
 
8160
- | Drift class | Release impact |
8161
- |-------------|----------------|
8162
- | A (cosmetic reword) | No version impact — refresh snapshot, continue |
8163
- | B (new feature — integrate) | **Minor** version bump at minimum; new generator or extended generator |
8164
- | C (new feature — not adopted) | No impact — note in registry |
8165
- | D (deprecation) | **Minor** or **major** depending on user impact; add migration warning |
8166
- | E (feature promoted to default) | **Minor**; simplify generator, keep fallback comment |
8167
- | F (feature newly globally available) | **Minor**; may require new generator or new config path |
8105
+ ## What this gate does not cover
8168
8106
 
8169
- If any B/D/E/F changes land in this preparation pass, mention them explicitly in the CHANGELOG entry (Phase 3) under a `Ecosystem` subsection.
8107
+ - **Cross-platform.** The suite here runs on your machine only. Both
8108
+ `ubuntu-latest` and `macos-latest` must be green before a tag; that gate
8109
+ lives in `sop-release.md`, Phase 7, and is bound to the exact
8110
+ release commit.
8111
+ - **Required files.** The `required-files` CI job checks that every file the
8112
+ module template mandates is present and that the version is in sync across
8113
+ all four manifests. It is cheap and runs on every push.
8114
+ - **Provenance and SARIF.** Release-time concerns; see
8115
+ `sop-release.md`.
8170
8116
 
8171
- ### 0.3 Refresh snapshot
8117
+ ## Commit message
8172
8118
 
8173
- Once every drift is resolved (either by code change or by re-classifying as acceptable):
8119
+ Conventional Commits, and no AI co-authorship trailer:
8174
8120
 
8175
- ```bash
8176
- python3 scripts/ecosystem_doctor.py --update
8121
+ ```
8122
+ feat(scope): summary in the imperative
8123
+ fix(scope): summary in the imperative
8124
+ docs|refactor|test|chore(scope): ...
8177
8125
  ```
8178
8126
 
8179
- This writes the new baseline to `benchmarks/ecosystem-doctor-snapshot.json`. Commit it as part of the release commit.
8127
+ `pre-commit-check.py` suggests a type from the staged paths. It is a
8128
+ suggestion; the scope and the summary are yours.
8180
8129
 
8181
- ### 0.4 Gate
8130
+ ## Related
8182
8131
 
8183
- ```bash
8184
- python3 scripts/ecosystem_doctor.py --offline --check
8185
- ```
8132
+ - `sop-release.md` -- the pre-tag gate, including cross-platform CI
8133
+ - `sop-post-release-testing.md` -- what to verify after a release ships
8134
+ - `sop-release-verification.md` -- end-to-end smoke test of an installed build
8186
8135
 
8187
- Must exit `0`. If it exits `1`, the snapshot is stale — rerun Phase 0.3 or review the remaining drift.
8136
+ ---
8137
+
8138
+ ## kb/procedures/sop-release-verification.md
8188
8139
 
8140
+ ---
8141
+ title: "SOP: Release Verification"
8142
+ category: procedures
8143
+ service: ai-toolkit
8144
+ tags: [sop, verification, release, smoke-test, install, update, qa, provenance, sarif, dsh]
8145
+ version: "1.8.0"
8146
+ created: "2026-04-08"
8147
+ last_updated: "2026-09-01"
8148
+ description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit. Verifies CLI, native Codex and GitHub Copilot surfaces, explicit DSH lifecycle, Claude app export, doctor, validation, tests, eject, provenance, SARIF, and per-skill permissions."
8189
8149
  ---
8190
8150
 
8191
- ## Phase 1: Determine Version Bump
8151
+ # SOP: Release Verification
8192
8152
 
8193
- Follow [Semantic Versioning](https://semver.org/):
8153
+ End-to-end smoke test after installing or updating `@softspark/ai-toolkit`.
8154
+ Verifies all critical paths from the user's perspective.
8194
8155
 
8195
- | Change Type | Bump | Examples |
8196
- |-------------|------|---------|
8197
- | Bugfix, typo, doc-only | **patch** | Fix install flag, correct description |
8198
- | New feature, skill, agent, flag | **minor** | Add `/hipaa-validate`, add `--output json` |
8199
- | Breaking CLI change, removed skill, config format change | **major** | Rename `install` to `setup`, remove skill |
8156
+ **Use this SOP when:**
8157
+ - After `npm install -g @softspark/ai-toolkit@latest`
8158
+ - After `ai-toolkit update`
8159
+ - Before tagging a new version (`git tag`)
8160
+ - Before publishing to npm (`npm publish`)
8161
+ - As a smoke test in CI/CD
8200
8162
 
8201
- **Rule:** When in doubt, bump minor.
8163
+ **Prerequisites:**
8164
+ - Node.js >= 18, Python 3, `bats`, git
8165
+ - `@softspark/ai-toolkit` installed globally
8166
+
8167
+ **Time:** 10-15 minutes (full), 2 minutes (quick checklist)
8202
8168
 
8203
8169
  ---
8204
8170
 
8205
- ## Phase 2: Sync Version in All Files
8171
+ ## Quick Checklist (TL;DR)
8206
8172
 
8207
- The canonical version lives in `package.json`. These files **must** match:
8173
+ The 14 core commands below must pass. Releases that change DSH must also complete Phase 10.
8208
8174
 
8209
- ### Mandatory sync (every release)
8175
+ ```bash
8176
+ # Pre-commit (Phase 0)
8177
+ python3 scripts/generate_agents_md.py > AGENTS.md # 1. Regenerate AGENTS.md
8178
+ python3 scripts/generate_llms_txt.py > llms.txt # 2. Regenerate llms.txt
8179
+ python3 scripts/validate.py --strict # 3. Validation passed?
8180
+ npm test > /tmp/npm-test.log 2>&1 && grep -c '^ok ' /tmp/npm-test.log && ! grep -q '^not ok' /tmp/npm-test.log # 4. All tests passed? (single run, cached)
8210
8181
 
8211
- | File | Field | How to update |
8212
- |------|-------|---------------|
8213
- | `package.json` | `"version": "X.Y.Z"` | Edit directly |
8214
- | `manifest.json` | `"version": "X.Y.Z"` | Edit directly |
8215
- | `app/.claude-plugin/plugin.json` | `"version": "X.Y.Z"` | Edit directly |
8182
+ # Post-install verification (Phases 1-7)
8183
+ ai-toolkit --version # 5. Version OK?
8184
+ ai-toolkit status # 6. Status OK?
8185
+ ai-toolkit doctor # 7. Health check passed?
8186
+ ai-toolkit install --dry-run # 8. Global install OK?
8187
+ python3 scripts/audit_skills.py --ci # 9. Security audit clean?
8216
8188
 
8217
- ### Auto-synced (no manual action)
8189
+ # Supply-chain verification (Phase 8, v2.8.0+)
8190
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json,sys; assert json.load(sys.stdin)['version']=='2.1.0'; print('SARIF OK')" # 10. SARIF 2.1.0 well-formed?
8191
+ python3 scripts/audit_skills.py --permissions | head -30 # 11. Broad-access skills reviewed?
8192
+ npm view @softspark/ai-toolkit@X.Y.Z --json | python3 -c "import json,sys; d=json.load(sys.stdin); assert d['dist']['attestations']['provenance']['predicateType']=='https://slsa.dev/provenance/v1'; print('PROVENANCE OK')" # 12. Provenance attested on npm?
8193
+ python3 scripts/claude_app.py verify # 13. Claude Chat/Cowork plugin contract valid?
8218
8194
 
8219
- | File | Mechanism |
8220
- |------|-----------|
8221
- | `package-lock.json` | Regenerated by `npm install --package-lock-only` |
8195
+ # Deep-coverage verification (Phase 9, v3.0.0+)
8196
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8197
+ diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) && echo "OK: registry matches" # 14. Registry <-> generators drift?
8198
+ ```
8222
8199
 
8223
- ### Conditional sync (only if the doc was modified in this release)
8200
+ ---
8224
8201
 
8225
- | File | Field | When to update |
8226
- |------|-------|---------------|
8227
- | `kb/procedures/maintenance-sop.md` | frontmatter `version:` | If SOP content changed |
8228
- | `kb/reference/skills-catalog.md` | frontmatter `version:` | If skills added/removed |
8229
- | `kb/reference/agents-catalog.md` | frontmatter `version:` | If agents added/removed |
8230
- | `kb/reference/hooks-catalog.md` | frontmatter `version:` | If hooks changed |
8231
- | `kb/reference/architecture-overview.md` | frontmatter `version:` | If architecture changed |
8232
- | `kb/reference/distribution-model.md` | frontmatter `version:` | If install model changed |
8233
- | `kb/reference/global-install-model.md` | frontmatter `version:` | If install model changed |
8202
+ ## Phase 0: Pre-Commit & Pre-Push (2 min)
8234
8203
 
8235
- > **Note:** KB `version:` fields track the **document version**, not the toolkit version.
8236
- > Only bump them when the document content actually changes in this release.
8204
+ Run these commands **before every commit and push to main**. CI validates
8205
+ counts but does NOT auto-regenerate you must do it locally.
8237
8206
 
8238
- ### Count sync (if skills/agents/hooks changed)
8207
+ ```bash
8208
+ # 1. Regenerate generated artifacts
8209
+ python3 scripts/generate_agents_md.py > AGENTS.md
8210
+ python3 scripts/generate_llms_txt.py > llms.txt
8211
+ python3 scripts/generate_llms_txt.py --full > llms-full.txt
8239
8212
 
8240
- | File | What to check |
8241
- |------|---------------|
8242
- | `package.json` | `"description"` — skill/agent count |
8243
- | `README.md` | Badge counts, "What You Get" table |
8244
- | `app/ARCHITECTURE.md` | Section headings with counts |
8213
+ # 2. Validate everything (catches stale counts, missing assets)
8214
+ python3 scripts/validate.py --strict
8245
8215
 
8246
- > **Tip:** `validate.py --strict` catches count drift AND version mismatches
8247
- > (package.json vs manifest.json vs plugin.json) automatically.
8248
- > If validation passes, counts and versions are correct.
8216
+ # 3. Security audit
8217
+ python3 scripts/audit_skills.py --ci
8249
8218
 
8250
- ### Verification command
8219
+ # 4. Run tests
8220
+ npm test
8251
8221
 
8252
- After syncing, verify all mandatory files match:
8222
+ # 5. Stage and commit
8223
+ # AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated and
8224
+ # gitignored — do NOT commit them (regenerated at prepublishOnly). Only the
8225
+ # doc artifacts llms.txt / llms-full.txt are committed.
8226
+ git add llms.txt llms-full.txt
8227
+ git add -p # stage your other changes
8228
+ git commit -m "feat: your change description"
8229
+ ```
8230
+
8231
+ **Why local?** Branch protection on `main` requires PRs and status checks.
8232
+ CI cannot push directly to `main`, so generated artifacts must be committed
8233
+ by the developer as part of their PR.
8253
8234
 
8235
+ **One-liner (copy-paste):**
8254
8236
  ```bash
8255
- VERSION=$(python3 -c "import json; print(json.load(open('package.json'))['version'])")
8256
- echo "Target: $VERSION"
8257
- echo "manifest.json: $(python3 -c "import json; print(json.load(open('manifest.json'))['version'])")"
8258
- echo "plugin.json: $(python3 -c "import json; print(json.load(open('app/.claude-plugin/plugin.json'))['version'])")"
8259
- echo "package-lock.json: $(python3 -c "import json; print(json.load(open('package-lock.json'))['version'])")"
8237
+ python3 scripts/generate_agents_md.py > AGENTS.md && python3 scripts/generate_llms_txt.py > llms.txt && python3 scripts/generate_llms_txt.py --full > llms-full.txt && python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
8260
8238
  ```
8261
8239
 
8262
- All four must print the same version. If not, fix before proceeding.
8240
+ ---
8263
8241
 
8264
- ### Public surface review
8242
+ ## Phase 1: CLI & Version (1 min)
8265
8243
 
8266
8244
  ```bash
8267
- python3 scripts/surface_manifest.py --update
8268
- git diff app/surface.json
8245
+ ai-toolkit --version
8246
+ ai-toolkit --help
8247
+ which ai-toolkit
8269
8248
  ```
8270
8249
 
8271
- **Every line the diff removes is a breaking change.** Restore it, or take the
8272
- deprecation path in `BACKWARD_COMPATIBILITY.md` and add a `DECISIONS.md` entry
8273
- before the tag. Lines added are new surface being adopted into protection — that
8274
- needs no ceremony.
8275
-
8276
- Do not run `--update` to make a red build green. The check fails because something
8277
- users depend on disappeared; regenerating the manifest deletes the evidence, not
8278
- the problem.
8250
+ **Verify:**
8251
+ - [ ] `--version` returns correct semver (e.g., `1.4.0`)
8252
+ - [ ] `--help` displays full command list without errors
8253
+ - [ ] `which` points to global npm bin path
8279
8254
 
8280
- ### Skill body budget ratchet
8255
+ ---
8281
8256
 
8282
- `validate.py` prints the largest skill body on every run:
8257
+ ## Phase 2: Global Install & Status (2 min)
8283
8258
 
8284
- ```
8285
- Body budget: largest is <skill> at <N> bytes (warn 18000, error 20000)
8259
+ ```bash
8260
+ ai-toolkit install --dry-run
8261
+ ai-toolkit status
8286
8262
  ```
8287
8263
 
8288
- Once that number sits at least 2000 bytes under `SKILL_BODY_BUDGET_WARN`, lower
8289
- `SKILL_BODY_BUDGET_WARN` by 2000 in `scripts/validate.py` and ship the tightened
8290
- threshold with the release. Target floor is 12000.
8291
-
8292
- Two rules, both learned the hard way:
8293
-
8294
- - **Never lower a threshold in the same change that something violates it.** Split
8295
- the offending skill into `SKILL.md` + `reference/` first, prove it with
8296
- `python3 scripts/check_split.py <skill> --before <pre-split SKILL.md>`, then
8297
- tighten in a follow-up.
8298
- - **Never raise a threshold to make a red build green.** A body over budget means
8299
- detail that belongs in `reference/` is loading on every trigger match, including
8300
- the accidental ones. Raising the number hides the cost, it does not remove it.
8264
+ **Verify `--dry-run`:**
8265
+ - [ ] Agents: 44
8266
+ - [ ] Skills: 108
8267
+ - [ ] Hooks merged into settings.json
8268
+ - [ ] "Other AI Tools" lists documented global targets (with `--editors`): aider, antigravity, augment, cline, codex, copilot, cursor, gemini, opencode, roo, windsurf. Scope varies: Codex uses `$CODEX_HOME` (default `~/.codex`) plus `$HOME/.agents/skills`; Copilot uses `$COPILOT_HOME` (default `~/.copilot`); cursor has only `~/.cursor/hooks.json`; antigravity has the `~/.gemini/*/skills` pointer. Cursor and Antigravity rules remain project-only.
8301
8269
 
8302
- If the largest body has not moved since the previous release, leave the threshold
8303
- alone and say so in the release notes. A ratchet that never advances is honest;
8304
- one that advances past reality is not.
8270
+ **Verify `status`:**
8271
+ - [ ] Version matches expected
8272
+ - [ ] Profile: minimal/standard/strict
8273
+ - [ ] Modules: list of installed modules
8274
+ - [ ] Latest: up to date / update available
8305
8275
 
8306
8276
  ---
8307
8277
 
8308
- ## Phase 3: Write CHANGELOG Entry
8309
-
8310
- Add entry at the top of `CHANGELOG.md` (after the header, before previous release):
8311
-
8312
- ```markdown
8313
- ## vX.Y.Z — Short Title (YYYY-MM-DD)
8314
-
8315
- ### Added
8316
- - **Feature name** — description
8317
-
8318
- ### Changed
8319
- - **What changed** — old behavior → new behavior
8320
-
8321
- ### Fixed
8322
- - **Bug description** — what was broken and how it's fixed
8278
+ ## Phase 3: Doctor Health Check (1 min)
8323
8279
 
8324
- ### Removed
8325
- - **What was removed** — migration path if any
8280
+ ```bash
8281
+ ai-toolkit doctor
8326
8282
  ```
8327
8283
 
8328
- **Rules:**
8329
- - Use **bold** for feature names
8330
- - Start descriptions with a verb (Added, Changed, Fixed, Removed)
8331
- - Reference skill names with backticks and slash: `/hipaa-validate`
8332
- - Include script names: `scripts/hipaa_scan.py`
8333
- - Include count changes: `Skill count: 91 → 92`
8334
- - Date format: `YYYY-MM-DD`
8335
- - Title: short, descriptive, no version number repetition
8336
-
8337
- ### Update README "What's New" section
8338
-
8339
- **MANDATORY on every release.** Update the `## What's New in vX.Y.Z` section in `README.md`:
8340
-
8341
- 1. Change the heading version: `## What's New in vX.Y.Z`
8342
- 2. Replace bullet points with 3-5 highlights from this release
8343
- 3. **Keep only the latest version block.** Delete the previous `## What's New in vA.B.C` section(s). README is the shop window, not the archive — users see the current release, full history lives in `CHANGELOG.md`.
8344
- 4. Keep the `See [CHANGELOG.md](CHANGELOG.md) for full history.` link directly below the bullet list.
8284
+ **Expected sections (all OK):**
8285
+ - Environment: node, bash, python3, bats
8286
+ - Global Install: .claude exists, agents/skills symlinks (0 broken), settings.json hooks
8287
+ - Hook Scripts: all present and executable
8288
+ - Hook Configuration: 14 events registered
8289
+ - Generated Artifacts: AGENTS.md, llms.txt, llms-full.txt
8290
+ - Planned Assets: plugin.json, benchmarks, plugin packs
8291
+ - Benchmark Freshness: < 30 days
8292
+ - Stale Rules: all healthy
8345
8293
 
8346
- > **Warning:** This section is the first thing users see after the badges.
8347
- > A stale version here (e.g., "What's New in v2.1.3" when shipping v2.3.0)
8348
- > signals an unmaintained project. Do NOT skip this step.
8294
+ **Verify:**
8295
+ - [ ] `Errors: 0 | Warnings: 0`
8296
+ - [ ] `HEALTH CHECK PASSED`
8349
8297
 
8350
- > **Single-version rule:** README.md must contain **exactly one** `## What's New in vX.Y.Z` heading at any time. If you find multiple stacked (e.g. v2.6.1 + v2.6.0 + v2.5.0), that is a SOP drift — collapse to the latest on the next release commit.
8298
+ If doctor detects problems: `ai-toolkit doctor --fix` auto-repairs
8299
+ (broken symlinks, non-executable hooks, missing scripts, missing llms-full.txt).
8351
8300
 
8352
8301
  ---
8353
8302
 
8354
- ## Phase 4: Regenerate Artifacts
8355
-
8356
- Use the npm scripts, not the generators directly:
8303
+ ## Phase 4: Local Install (2 min)
8357
8304
 
8358
8305
  ```bash
8359
- npm run generate:agents # AI_TOOLKIT_NO_CUSTOM_RULES=1 python3 scripts/generate_agents_md.py > AGENTS.md
8360
- npm run generate:llms # llms.txt + llms-full.txt
8306
+ mkdir -p /tmp/ai-toolkit-verify && cd /tmp/ai-toolkit-verify
8307
+ git init -q
8308
+ ai-toolkit install --local --editors all --dry-run
8309
+ cd - && rm -rf /tmp/ai-toolkit-verify
8361
8310
  ```
8362
8311
 
8363
- `generate:agents` sets `AI_TOOLKIT_NO_CUSTOM_RULES=1`. Running
8364
- `generate_agents_md.py` bare picks up whatever is registered in the maintainer's
8365
- own `~/.softspark/ai-toolkit/rules/`, which then ships inside `AGENTS.md`.
8366
-
8367
- Check if anything actually changed:
8312
+ **Verify "Project-local" section:**
8313
+ - [ ] Would create: CLAUDE.md
8314
+ - [ ] Would create: .claude/settings.local.json
8315
+ - [ ] Would inject: .claude/constitution.md
8316
+ - [ ] Editors: all 11 listed (copilot, cursor, windsurf, cline, roo, aider, augment, antigravity, codex, gemini, opencode)
8317
+ - [ ] Would generate configs for each editor (legacy + directory-based)
8318
+ - [ ] Would install: .git/hooks/pre-commit
8319
+ - [ ] Would inject language rules (auto-detected)
8368
8320
 
8321
+ **Also test auto-detect (no --editors flag):**
8369
8322
  ```bash
8370
- git diff --stat AGENTS.md llms.txt llms-full.txt
8323
+ ai-toolkit install --local --dry-run
8324
+ # → Editors: none (empty project has no existing configs)
8371
8325
  ```
8372
8326
 
8373
- If no diff, the artifacts are already current. If there is a diff, stage them.
8374
-
8375
8327
  ---
8376
8328
 
8377
- ## Phase 5: Validate, Audit, Test
8378
-
8379
- Run the full quality gate:
8329
+ ## Phase 5: Validation & Security Audit (3 min)
8380
8330
 
8381
8331
  ```bash
8382
8332
  python3 scripts/validate.py --strict
8383
8333
  python3 scripts/audit_skills.py --ci
8384
- python3 scripts/audit_skills.py --sarif > audit.sarif # MANDATORY — GHAS ingest
8385
- python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8334
+ ```
8386
8335
 
8387
- # ShellCheck on hooks (added in 1.11.0). Mirrors the ci.yml "ShellCheck hooks"
8388
- # job. NOT run by validate.py, npm test, OR publish.yml so a hook with a
8389
- # ShellCheck warning passes every other gate AND still publishes on tag while
8390
- # turning main CI red. Run it here, before tagging.
8391
- shellcheck --severity=warning app/hooks/*.sh && echo "OK: shellcheck clean"
8336
+ **Verify validate.py:**
8337
+ - [ ] Agents: 44, Skills: 108, Tests: exactly the current README badge count
8338
+ - [ ] Hook events: 14, Hook scripts: >= 30
8339
+ - [ ] Plugin packs >= 10, KB documents >= 20
8340
+ - [ ] `Errors: 0 | Warnings: 0` → `VALIDATION PASSED`
8392
8341
 
8393
- # Registry / generator drift (added in 1.10.0). Meta-generators excluded.
8394
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8395
- diff \
8396
- <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) \
8397
- <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) \
8398
- && echo "OK: registry matches filesystem" \
8399
- || { echo "DRIFT: update supported-tools-registry.md before tagging"; exit 1; }
8342
+ **Verify audit_skills.py:**
8343
+ - [ ] `HIGH: 0` (MUST be zero — CI fails otherwise)
8344
+ - [ ] `WARN: 0`
8345
+ - [ ] `INFO: N` (acceptable — broad-access skills: orchestrate, swarm, workflow)
8400
8346
 
8401
- # Stage first IF this release adds or deletes a kb/ file. The test
8402
- # "npm package KB files match the tracked release set" compares `git ls-files
8403
- # kb` (the index) against what `npm pack` sees (the working tree), so an
8404
- # unstaged addition reads as "extra" and an unstaged deletion as "missing".
8405
- # Phase 6 stages, and it runs after this one, so the ordering fails the test
8406
- # for any release that touches the KB. Staging early costs nothing.
8407
- git status --porcelain kb/ | grep -qE '^(\?\?| D|\?M)' && git add -A kb/
8347
+ ---
8408
8348
 
8409
- # Run npm test ONCE, cache output, parse from file. The suite is 1400+ bats
8410
- # cases — rerunning it per check wastes minutes. Do not pipe npm test into
8411
- # tail/grep multiple times in the same session.
8349
+ ## Phase 6: Tests (3-5 min)
8350
+
8351
+ ```bash
8352
+ # Run ONCE, capture to file, then parse. Full suite is 669+ bats cases —
8353
+ # re-running it per check (tail / grep ok / grep not ok piped separately)
8354
+ # wastes minutes every release. Always cache the output.
8412
8355
  npm test > /tmp/npm-test.log 2>&1
8356
+ exit=$?
8413
8357
  tail -3 /tmp/npm-test.log
8414
- echo "ok: $(grep -c '^ok ' /tmp/npm-test.log) | not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8358
+ echo "ok: $(grep -c '^ok ' /tmp/npm-test.log)"
8359
+ echo "not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8360
+ echo "exit: $exit"
8415
8361
  ```
8416
8362
 
8417
- **Expected results:**
8418
- - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
8419
- - `audit_skills.py --ci`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
8420
- - `audit_skills.py --sarif`: valid JSON, non-empty `runs[0].tool.driver.rules`
8421
- - `audit_skills.py --permissions`: review `Skills with Bash + Write + Edit` list — any newly-added skill with broad access MUST be justified in the CHANGELOG entry
8422
- - `shellcheck --severity=warning app/hooks/*.sh`: no output, exit 0. A common false positive is `SC2034` on `INPUT` or env vars (e.g. `AI_TOOLKIT_HOOK_FORMAT`) that a *sourced* helper (`_hook-io.sh`) consumes — ShellCheck cannot see cross-file use. Fix with a `# shellcheck disable=SC2034` directive or `export`, matching `guard-destructive.sh`. Never tag with a red ShellCheck.
8423
- - Registry drift: `OK: registry matches filesystem`. If `DRIFT:` appears, add the missing `scripts/generate_*.py` rows to `kb/reference/supported-tools-registry.md` before tagging.
8424
- - `npm test`: `1..N` with zero `not ok` (read from the cached `/tmp/npm-test.log`, do not rerun)
8425
-
8426
- > **Why this matters (v4.5.1 postmortem):** `publish.yml` runs only `validate.py` + `npm test`, so it published v4.5.0 even though the `main` CI `ShellCheck hooks` job was red on two `SC2034` warnings in a new hook. The publish workflow does **not** depend on the CI workflow. Until that is fixed, ShellCheck is a manual pre-tag gate — run it here every time.
8363
+ **Verify:**
8364
+ - [ ] `exit == 0`
8365
+ - [ ] `ok == expected test count` (e.g., 945 on v3.0.0)
8366
+ - [ ] `not ok == 0`
8367
+ - [ ] Bats runs tests in parallel (4 jobs)
8368
+ - [ ] Groups: agents, autodetect, cli, generators, guards, hooks, inject,
8369
+ install, kb, mcp, readme, profiles, uninstall, validate
8427
8370
 
8428
- **One-liner:**
8371
+ **Anti-pattern — do NOT do this:**
8429
8372
  ```bash
8430
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > audit.sarif && shellcheck --severity=warning app/hooks/*.sh && diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE 'generate_agents_md\.py|generate_llms_txt\.py|generate_language_rules_skills\.py' | sort -u) && npm test
8373
+ # Runs the full suite THREE times. Adds 1-3 min and pressures CI capacity.
8374
+ npm test 2>&1 | tail -3
8375
+ npm test 2>&1 | grep -c '^ok '
8376
+ npm test 2>&1 | grep -c '^not ok'
8431
8377
  ```
8432
8378
 
8433
- **If tests fail:** Fix the issue, do NOT skip. Common failures:
8434
- - Stale counts → re-run `generate:all` or fix README/ARCHITECTURE
8435
- - Missing frontmatter add to new KB docs
8436
- - Broken symlink `ai-toolkit doctor --fix`
8379
+ **Key test areas:**
8380
+ - Guards: rm -rf, DROP TABLE, git push --force blocked
8381
+ - Install: idempotent, profiles, --only/--skip, orphan cleanup
8382
+ - Eject: real files (not symlinks), inlined rules
8383
+ - Uninstall: removes toolkit, preserves user content
8437
8384
 
8438
- ### Phase 5c: Licensing Gate (v4.20.0+)
8385
+ ---
8439
8386
 
8440
- The project is Apache-2.0. Attribution only works if the artefact actually
8441
- carries it, and every part of that is mechanically checkable.
8387
+ ## Phase 7: Eject (1 min)
8442
8388
 
8443
8389
  ```bash
8444
- # The whole gate, enforced in CI. Run it here so a failure is caught before tagging.
8445
- npx bats tests/test_licensing.bats
8390
+ mkdir -p /tmp/ai-toolkit-eject-test
8391
+ cd /tmp/ai-toolkit-eject-test
8392
+ ai-toolkit eject
8393
+ cd - && rm -rf /tmp/ai-toolkit-eject-test
8446
8394
  ```
8447
8395
 
8448
- The seven assertions, and why each exists:
8396
+ **Verify:**
8397
+ - [ ] Agents copied as real files (not symlinks)
8398
+ - [ ] Skills copied as real directories
8399
+ - [ ] Rules inlined into CLAUDE.md
8400
+ - [ ] constitution.md and ARCHITECTURE.md copied
8401
+ - [ ] `output-styles/` directory present (v2.7.1+)
8449
8402
 
8450
- | Check | Fails when |
8451
- |---|---|
8452
- | Every shipped source file carries an SPDX header | A new `.py`/`.sh`/`.js`/`.bats` file was added without one — the common case, and the reason this is a test rather than a habit |
8453
- | Headers name Apache-2.0 and nothing else | A file was copied in from an MIT/GPL source with its own header intact |
8454
- | **No** markdown file carries an SPDX header | Someone "helpfully" ran the header script over `app/skills/` — headers there sit above parsed frontmatter and bill every session for it |
8455
- | `LICENSE` is the complete Apache 2.0 text | The file was truncated or replaced with a summary |
8456
- | `NOTICE` carries attribution, the source URL, §4(d) and the MIT-era notice | The attribution mechanism was gutted |
8457
- | `LICENSE` **and** `NOTICE` ship in the npm package | `package.json` `files` lost an entry — a NOTICE that never reaches the consumer cannot satisfy §4(d) |
8458
- | Every manifest declaring a licence declares Apache-2.0 | `package.json`, `manifest.json`, `plugin.json` and `package-lock.json` drifted apart |
8403
+ ---
8459
8404
 
8460
- **Adding source files in this release?** The header goes *after* the shebang,
8461
- never before it. Short SPDX form:
8405
+ ## Phase 8: Supply-Chain Verification (2 min, v2.8.0+)
8462
8406
 
8463
- ```
8464
- # SPDX-License-Identifier: Apache-2.0
8465
- # Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
8466
- # Source: https://github.com/softspark/ai-toolkit
8467
- ```
8407
+ These checks enforce the v2.8.0 security standard on a freshly-published release. Run AFTER the `publish.yml` workflow completes on the tag.
8468
8408
 
8469
- `//` for JavaScript. Full convention and the reasoning behind the markdown
8470
- exclusion: [Licensing](../reference/licensing.md).
8409
+ ### 8.1 Provenance attestation on npm
8471
8410
 
8472
- **Changing the licence itself?** Do not hand-type the licence text. Take it
8473
- verbatim from a published copy and cross-verify against a second independent
8474
- copy before writing `LICENSE` — a rendered or summarised licence is not the
8475
- licence. Prior releases stay under their original terms; a licence change
8476
- applies going forward and revokes nothing already granted.
8411
+ ```bash
8412
+ VERSION="X.Y.Z" # the tag just published
8413
+ npm view "@softspark/ai-toolkit@${VERSION}" --json \
8414
+ | python3 -c "import json, sys; d=json.load(sys.stdin); att=d['dist'].get('attestations', {}); assert att.get('provenance', {}).get('predicateType') == 'https://slsa.dev/provenance/v1', f'NO PROVENANCE for {d[\"version\"]}'; print(f'PROVENANCE OK: {att[\"url\"]}')"
8415
+ ```
8477
8416
 
8478
- ### Phase 5a: Supply-Chain Hardening Verification (v2.8.0+)
8417
+ **Verify:**
8418
+ - [ ] Exit 0 and prints `PROVENANCE OK: https://registry.npmjs.org/...`
8419
+ - [ ] `https://www.npmjs.com/package/@softspark/ai-toolkit/v/${VERSION}` shows the green "Provenance" badge
8479
8420
 
8480
- These checks enforce the security standard introduced in v2.8.0. Do NOT tag a release until all pass.
8421
+ **If provenance is missing:** the `publish.yml` ran without `id-token: write` or `--provenance`. Restore them and cut a patch release a silently unsigned publish is a regression against the v2.8.0 standard.
8481
8422
 
8482
- **1. Publish workflow emits provenance:**
8423
+ ### 8.2 Audit SARIF output (for GHAS ingest)
8483
8424
 
8484
8425
  ```bash
8485
- grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
8426
+ python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif
8427
+ python3 -c "import json; d=json.load(open('/tmp/audit.sarif')); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']=='ai-toolkit-audit-skills'; print(f'SARIF OK: {len(d[\"runs\"][0][\"results\"])} results across {len(d[\"runs\"][0][\"tool\"][\"driver\"][\"rules\"])} rules')"
8486
8428
  ```
8487
8429
 
8488
- - [ ] Both markers present (`--provenance` flag + `id-token: write` permission)
8489
- - [ ] Any PR that changes `publish.yml` REQUIRES an approved security review
8430
+ **Verify:**
8431
+ - [ ] Valid SARIF 2.1.0
8432
+ - [ ] In the publishing repo, the CI job uploads `audit.sarif` via `github/codeql-action/upload-sarif@v3` so findings appear in the Security tab
8490
8433
 
8491
- **2. URL-sourced rules and hooks are checksum-pinned:**
8434
+ ### 8.3 Per-skill permissions report
8492
8435
 
8493
8436
  ```bash
8494
- # On a machine that has consumed URL rules/hooks at least once
8495
- # (schema_version 1: entries live under the .rules / .hooks key):
8496
- jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
8497
- jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
8437
+ python3 scripts/audit_skills.py --permissions | head -40
8498
8438
  ```
8499
8439
 
8500
- - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
8501
- - [ ] If not, run `ai-toolkit update` to backfill missing hashes before tagging
8440
+ **Verify:**
8441
+ - [ ] Bash skill count has NOT jumped unexpectedly since the previous release
8442
+ - [ ] Any newly added entry under `Skills with Bash + Write + Edit` matches a CHANGELOG bullet that justifies the broad scope
8443
+ - [ ] JSON form (`--permissions --json`) is available for automated drift dashboards
8502
8444
 
8503
- **3. Audit SARIF output is well-formed:**
8445
+ ### 8.4 URL-sourced rules/hooks are checksum-pinned
8446
+
8447
+ `sources.json` is an envelope of the form `{"schema_version": 1, "rules"|"hooks": {...}}`, so the jq filter must pick the nested map before piping into the pin assertion. The URL-count in the success message ignores local `path:`-only entries — only entries with a `url:` field are pinned and counted.
8504
8448
 
8505
8449
  ```bash
8506
- python3 scripts/audit_skills.py --sarif | python3 -c "import json, sys; d=json.load(sys.stdin); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']; print('SARIF OK')"
8450
+ jq '.rules // .hooks // {}' ~/.softspark/ai-toolkit/rules/sources.json 2>/dev/null \
8451
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'RULE PIN OK: {url_n} URL rules, all with sha256')"
8452
+ jq '.hooks // {}' ~/.softspark/ai-toolkit/hooks/external/sources.json 2>/dev/null \
8453
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'HOOK PIN OK: {url_n} URL hooks, all with sha256')"
8507
8454
  ```
8508
8455
 
8509
- - [ ] Prints `SARIF OK`
8510
- - [ ] If the script ever grows new rule classes, extend the SARIF `rules[]` coverage before releasing
8456
+ **Verify:**
8457
+ - [ ] Both commands print `... PIN OK`
8458
+ - [ ] If any entry is unpinned, the `register_url_source()` call missed passing `content=` — fix the call-site and retag
8511
8459
 
8512
- **4. Strict-pin mode passes on CI** (optional, recommended for stable branches):
8460
+ ### 8.5 Strict-pin smoke test (optional but recommended)
8513
8461
 
8514
8462
  ```bash
8515
8463
  AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
8516
8464
  ```
8517
8465
 
8466
+ **Verify:**
8518
8467
  - [ ] Exit 0, no `CHECKSUM CHANGED` line
8519
- - [ ] Any unexpected upstream change blocks the release until explicitly approved
8468
+ - [ ] If a checksum change was intentional (e.g. upstream rule update), document it in the CHANGELOG entry before tagging
8520
8469
 
8521
8470
  ---
8522
8471
 
8523
- ## Phase 6: Commit
8472
+ ## Phase 9: Deep-Coverage Checks (v3.0.0+)
8524
8473
 
8525
- Stage all release files:
8474
+ These verify the native-surface generators shipped in v3.0.0 actually emit the right files for the right profiles, and that the tool registry stays in sync with shipped generators.
8475
+
8476
+ > **Safety warning — HOME-scoped writes:** Running `--profile full` with `augment` in the editor list writes to `$HOME/.augment/settings.json` (Augment stores hooks under HOME, not per-project). Use `--dry-run` for verification unless you intend to carry ai-toolkit hook entries on this machine. The generator is marker-safe (only rewrites its own `_source: ai-toolkit` entries) but is still a side-effect.
8477
+
8478
+ ### 9.1 `--profile full` emits every native surface
8526
8479
 
8527
8480
  ```bash
8528
- git add package.json manifest.json app/.claude-plugin/plugin.json
8529
- git add package-lock.json
8530
- git add CHANGELOG.md
8531
- git add llms.txt llms-full.txt
8532
- # NOTE: AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated
8533
- # editor configs and are gitignored — do NOT commit them. `prepublishOnly` runs
8534
- # `npm run generate:all`, so the shipped package (which lists AGENTS.md in
8535
- # package.json `files`) gets a fresh copy at publish time.
8536
- git add -p # review and stage any other changes
8481
+ D=/tmp/aitk-profile-full-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8482
+ ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot \
8483
+ --profile full --dry-run 2>&1 \
8484
+ | grep -E "\\.cursor/(hooks\\.json|agents)|\\.devin/hooks\\.v1\\.json|\\.gemini/(settings\\.json|commands)|\\.augment/(agents|commands)|\\.codex/(agents|hooks)|\\.agents/skills|\\.github/(agents|skills|instructions|prompts|hooks)"
8537
8485
  ```
8538
8486
 
8539
- Commit:
8487
+ **Verify** — at least the following lines appear:
8488
+ - [ ] `.cursor/hooks.json` and `.cursor/agents/`
8489
+ - [ ] `.devin/hooks.v1.json`
8490
+ - [ ] `.gemini/settings.json` hooks AND `.gemini/commands/`
8491
+ - [ ] `.augment/agents/` + `.augment/commands/` + `$HOME/.augment/settings.json`
8492
+ - [ ] `.codex/hooks.json` + `.codex/hooks/` + `.codex/agents/`
8493
+ - [ ] `.agents/skills/` (Codex native discovery path)
8494
+ - [ ] `.github/agents/` + `.github/skills/` + `.github/instructions/` + `.github/prompts/` + `.github/hooks/`
8495
+
8496
+ ### 9.2 Codex skills are profile-independent
8540
8497
 
8541
8498
  ```bash
8542
- git commit -m "chore: release vX.Y.Z"
8499
+ D=/tmp/aitk-codex-skills-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8500
+ ai-toolkit install --local --editors codex --profile minimal --dry-run 2>&1 \
8501
+ | grep -q "Would generate: .agents/skills" && echo "OK: Codex skills emit at minimal"
8502
+ ai-toolkit install --local --editors codex --profile standard --codex-skills --dry-run 2>&1 \
8503
+ | grep -q "Would refresh: .agents/skills" && echo "OK: legacy flag explicitly refreshes the same catalog"
8543
8504
  ```
8544
8505
 
8545
- ---
8506
+ **Verify:**
8507
+ - [ ] Selecting Codex emits all skills under `.agents/skills/` at every profile
8508
+ - [ ] `--codex-skills` remains an explicit refresh alias; it is not required for installation
8509
+ - [ ] No profile emits `.codex/skills/`
8546
8510
 
8547
- ## Phase 7: Push Branch, Verify CI, Tag, and Push Tag
8511
+ ### 9.3 Breaking-change surfaces land on `--profile standard`
8512
+
8513
+ The standard profile must include:
8514
+ - Copilot scoped instructions, prompts, custom agents, portable skills, and hooks
8515
+ - Gemini hooks (`.gemini/settings.json`)
8548
8516
 
8549
8517
  ```bash
8550
- RELEASE_SHA=$(git rev-parse HEAD)
8551
- git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
8518
+ D=/tmp/aitk-breaking-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8519
+ ai-toolkit install --local --editors copilot,gemini --profile standard --dry-run 2>&1 \
8520
+ | tee /tmp/aitk-breaking.log
8521
+ grep -q "\\.github/instructions/" /tmp/aitk-breaking.log && echo "OK: Copilot dir layout at standard"
8522
+ grep -q "\\.github/agents/.*\\.github/skills/" /tmp/aitk-breaking.log && echo "OK: Copilot agents and skills at standard"
8523
+ grep -q "\\.github/hooks/" /tmp/aitk-breaking.log && echo "OK: Copilot hooks at standard"
8524
+ grep -q "\\.gemini/settings\\.json hooks" /tmp/aitk-breaking.log && echo "OK: Gemini hooks at standard"
8525
+ ```
8552
8526
 
8553
- # Bind the gate to the exact release commit. GitHub run registration is
8554
- # asynchronous, so retry at most five times and log every attempt.
8555
- RUN_ID=""
8556
- for ATTEMPT in 1 2 3 4 5; do
8557
- RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
8558
- --limit 1 --json databaseId --jq '.[0].databaseId')
8559
- [ -n "$RUN_ID" ] && break
8560
- echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
8561
- [ "$ATTEMPT" -eq 5 ] || sleep 60
8562
- done
8563
- test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
8564
- gh run watch "$RUN_ID" --exit-status \
8565
- || { echo "FAIL: release commit CI failed"; exit 1; }
8527
+ **Verify all four lines print `OK:`**. If any is missing, native Copilot or Gemini coverage has regressed.
8566
8528
 
8567
- # publish.yml is Ubuntu-only. Require both full CI matrix jobs before creating
8568
- # the tag so a green publish job cannot bypass a red macOS test.
8569
- for OS in ubuntu-latest macos-latest; do
8570
- gh run view "$RUN_ID" --json jobs \
8571
- --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
8572
- | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
8573
- done
8529
+ ### 9.4 Install is idempotent
8574
8530
 
8575
- # Refuse to tag if either local or remote main moved while CI was running.
8576
- test "$(git rev-parse HEAD)" = "$RELEASE_SHA" \
8577
- || { echo "FAIL: local HEAD changed during CI"; exit 1; }
8578
- git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
8579
- test "$(git rev-parse origin/main)" = "$RELEASE_SHA" \
8580
- || { echo "FAIL: origin/main changed during CI"; exit 1; }
8531
+ ```bash
8532
+ D=/tmp/aitk-idem-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8533
+ # Sort file paths before hashing find traversal order follows inode order,
8534
+ # which can shift between runs even when content is byte-identical, producing
8535
+ # false FAIL signals.
8536
+ ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
8537
+ SHA1=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
8538
+ ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
8539
+ SHA2=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
8540
+ [ "$SHA1" = "$SHA2" ] && echo "OK: idempotent" || echo "FAIL: install is not idempotent"
8541
+ ```
8581
8542
 
8582
- git tag vX.Y.Z "$RELEASE_SHA"
8543
+ **Verify:** prints `OK: idempotent`. A second run must produce byte-identical files in every managed path.
8583
8544
 
8584
- # Assert the tag before pushing it. Both checks are one line each and both
8585
- # have caught a real broken release.
8586
- test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" \
8587
- || { echo "FAIL: tag is not on the tested release commit"; exit 1; }
8588
- git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" \
8589
- || { echo "FAIL: tag is not on the chore: release commit"; exit 1; }
8545
+ ### 9.5 Live-install JSON outputs parse
8590
8546
 
8591
- # Push the single release tag by its full ref.
8592
- git push origin refs/tags/vX.Y.Z
8547
+ The bats suite validates JSON shape at generation time. This re-checks that what actually landed on disk after a live install parses without errors.
8548
+
8549
+ ```bash
8550
+ D=/tmp/aitk-json-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8551
+ ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot --profile full >/dev/null 2>&1
8552
+ for f in .cursor/hooks.json .devin/hooks.v1.json .gemini/settings.json .codex/hooks.json .github/hooks/ai-toolkit.json "$HOME/.augment/settings.json"; do
8553
+ [ -f "$f" ] && python3 -c "import json; json.load(open('$f'))" && echo "OK: $f"
8554
+ done
8593
8555
  ```
8594
8556
 
8595
- **Why branch CI comes before the tag (v4.30.2 postmortem).** The publish
8596
- workflow runs only on Ubuntu and can publish while the separate macOS matrix job
8597
- is red. `v4.30.2` exposed this with a macOS-only DSH timing failure. A release
8598
- commit must therefore pass the complete Ubuntu/macOS branch workflow before its
8599
- tag exists; a successful publish workflow is not a substitute for green CI.
8557
+ **Verify:** each emitted file prints `OK: <path>`. Any `json.decoder.JSONDecodeError` means the merge logic corrupted the output.
8600
8558
 
8601
- **Why the assertions (v4.19.0 postmortem).** v4.19.0 was tagged on a commit
8602
- that contained only a KB document and still carried `package.json` version
8603
- `4.18.0`; the actual release sat in the commit above it under a recycled
8604
- `fix:` message. `publish.yml` fired, tried to publish a version already on
8605
- npm, and failed. Nothing on npm, a tag pointing at the wrong tree, and the
8606
- only way out was rewriting a pushed commit. Both assertions above catch this
8607
- in under a second. Run them.
8559
+ ### 9.6 Registry / generator drift check
8608
8560
 
8609
- **Never `git push --tags`.** It pushes every local tag at once, and GitHub
8610
- suppresses tag-triggered workflow runs when many tags arrive in a single push
8611
- — the workflow silently does not fire and nothing publishes. Push the single
8612
- release tag by its full ref, as above. (Sibling evidence: this is exactly how
8613
- rag-mcp's `1.0.3` image build was skipped, when a `--tags` push carried 37
8614
- tags at once.)
8561
+ `kb/reference/supported-tools-registry.md` should enumerate every per-editor `scripts/generate_*.py` we ship. Meta-generators (`generate_agents_md.py`, `generate_llms_txt.py`) are excluded they produce docs/artifacts, not editor configs.
8615
8562
 
8616
- This triggers `.github/workflows/publish.yml` which:
8617
- 1. Runs `validate.py --strict`
8618
- 2. Runs `npm test`
8619
- 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z` with `--provenance` (SLSA v1 build attestation)
8563
+ ```bash
8564
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8565
+ REG=$(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u)
8566
+ FS=$(ls scripts/generate_*.py | grep -vE "$META" | sort -u)
8567
+ diff <(echo "$REG") <(echo "$FS") && echo "OK: registry matches filesystem" || echo "DRIFT: update supported-tools-registry.md"
8568
+ ```
8620
8569
 
8621
- **Provenance is non-negotiable.** If `id-token: write` permission or the `--provenance` flag is missing from `publish.yml`, fix it BEFORE tagging an unsigned release is a regression against the v2.8.0 standard.
8570
+ **Verify:** prints `OK: registry matches filesystem`. If not, add the missing rows to the registry before tagging the next release.
8622
8571
 
8623
- **After CI completes:** Run the [Release Verification SOP](release-verification-sop.md)
8624
- to smoke-test the published package AND verify the provenance attestation landed on npm.
8572
+ ### 9.7 Claude Chat / Cowork plugin validates and exports
8573
+
8574
+ ```bash
8575
+ python3 scripts/claude_app.py verify
8576
+ D=/tmp/aitk-claude-app-${RANDOM}
8577
+ python3 scripts/claude_app.py export --output "$D.zip" --no-custom-rules
8578
+ python3 -c "import zipfile; z=zipfile.ZipFile('$D.zip'); assert '.claude-plugin/plugin.json' in z.namelist(); assert 'claude-app/skills/ai-toolkit-rules/SKILL.md' in z.namelist(); print('OK: Claude app archive')"
8579
+ ```
8580
+
8581
+ **Verify:** the official validator exits 0; the archive contains the manifest,
8582
+ app-native rules skill, bundled agents/skills, and plugin-relative Cowork hooks.
8625
8583
 
8626
8584
  ---
8627
8585
 
8628
- ## Rollback
8586
+ ## Phase 10: Explicit DSH Qualification (v4.30.0+)
8629
8587
 
8630
- If a bad release was published:
8588
+ Run this phase whenever the release changes the `dsh` target, package pins, preset lifecycle, or DSH compatibility documentation. Use a new task-specific `DSH_HOME`; never replace `HOME` or reuse a regular profile.
8589
+
8590
+ Prerequisites: DSH `0.1.1-rc.2`, pnpm `>=11.7.0,<12.0.0`, Codex logged in through ChatGPT, Claude Code logged in natively, and GitHub Copilot CLI logged in natively. Do not supply provider API keys.
8631
8591
 
8632
8592
  ```bash
8633
- # Unpublish from npm (within 72h)
8634
- npm unpublish @softspark/ai-toolkit@X.Y.Z
8593
+ DSH_SMOKE_ROOT="$(mktemp -d)"
8594
+ export DSH_HOME="$DSH_SMOKE_ROOT/dsh-home"
8635
8595
 
8636
- # Or deprecate (preferred — doesn't break existing installs)
8637
- npm deprecate @softspark/ai-toolkit@X.Y.Z "Known issue: <description>. Use vA.B.C instead."
8596
+ ai-toolkit dsh install --profile web
8597
+ ai-toolkit dsh doctor --profile web
8598
+ dsh --profile web --host 127.0.0.1 --port 0 --no-open
8599
+ ```
8638
8600
 
8639
- # Delete tag
8640
- git tag -d vX.Y.Z
8641
- git push origin --delete vX.Y.Z
8601
+ In a new `softspark-orchestrator` session, select the `codex` provider and run two standalone marker prompts:
8602
+
8603
+ 1. `subagent_claude_code` returns an exact child marker, then the Codex parent returns its exact completion marker.
8604
+ 2. `subagent_gemini_copilot` returns an exact child marker, then the Codex parent returns its exact completion marker.
8605
+
8606
+ Stop DSH, then remove only the managed profile artifacts:
8607
+
8608
+ ```bash
8609
+ ai-toolkit dsh uninstall --profile web --yes
8642
8610
  ```
8643
8611
 
8612
+ **Verify:** both tool results have `isError=false`, both turns end as `completed`, `doctor` reports no recovery requirement before uninstall, and an unrelated preset fixture remains unchanged. Preserve only redacted event sequence evidence; never attach credentials, auth files, or full private prompts.
8613
+
8644
8614
  ---
8645
8615
 
8646
- ## Checklist Summary
8616
+ ## Troubleshooting
8647
8617
 
8648
- | # | Step | Command / Action | Pass Criteria |
8649
- |---|------|-----------------|---------------|
8650
- | 0a | Ecosystem drift check | `ecosystem_doctor.py --format text` | All tools Clean, or drift classified and resolved |
8651
- | 0b | Ecosystem snapshot refresh | `ecosystem_doctor.py --update` | `benchmarks/ecosystem-doctor-snapshot.json` updated |
8652
- | 0c | Ecosystem gate | `ecosystem_doctor.py --offline --check` | Exit 0 |
8653
- | 1 | Version bump type | Decide patch/minor/major | — |
8654
- | 2 | `package.json` version | Edit `"version"` | Matches target |
8655
- | 3 | `manifest.json` version | Edit `"version"` | Matches target |
8656
- | 4 | `plugin.json` version | Edit `"version"` | Matches target |
8657
- | 5 | `package-lock.json` | `npm install --package-lock-only` | Matches target |
8658
- | 6 | Count sync | Check `package.json` description, README | `validate.py` passes |
8659
- | 7 | CHANGELOG.md | Add release entry (incl. `Ecosystem` subsection if any B/D/E/F drift) | Entry exists for vX.Y.Z |
8660
- | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_llms_txt.py` | No unexpected diff |
8661
- | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
8662
- | 10 | Security audit (CI mode) | `audit_skills.py --ci` | 0 HIGH |
8663
- | 11 | Security audit (SARIF) | `audit_skills.py --sarif` | Valid SARIF 2.1.0 JSON |
8664
- | 12 | Per-skill permissions | `audit_skills.py --permissions` | New broad-access skills justified in CHANGELOG |
8665
- | 13 | ShellCheck hooks | `shellcheck --severity=warning app/hooks/*.sh` | Exit 0, no output (mirrors ci.yml; publish.yml does NOT run it) |
8666
- | 14 | Provenance flag check | `grep -- '--provenance' .github/workflows/publish.yml` | Present |
8667
- | 15 | Checksum-pin backfill | `sources.json` entries all have `sha256` | No unpinned URL sources |
8668
- | 15a | Licensing gate | `npx bats tests/test_licensing.bats` | 7/7 — SPDX headers, LICENSE, NOTICE, npm `files`, manifest consistency |
8669
- | 16 | Tests | `git add -A kb/` if the KB changed, then `npm test` | All pass |
8670
- | 17 | Commit | `git commit` | Clean working tree |
8671
- | 18 | Push branch | `git push origin main` | Exact release commit is on `origin/main` |
8672
- | 18a | Full branch CI | `gh run watch "$RUN_ID" --exit-status` plus matrix job assertions | Ubuntu and macOS Bats jobs both conclude `success` |
8673
- | 19 | Tag | `git tag vX.Y.Z` | Tag exists only after green branch CI |
8674
- | 19a | Tag is on tested SHA | `test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA"` | Exit 0 |
8675
- | 19b | Tag is on the release commit | `git show --no-patch --format=%s vX.Y.Z` | Reads `chore: release vX.Y.Z` |
8676
- | 20 | Push the single tag | `git push origin refs/tags/vX.Y.Z` | Publish CI triggered with `id-token: write`. Never `--tags`. |
8618
+ ### `ai-toolkit: command not found`
8619
+
8620
+ ```bash
8621
+ npm install -g @softspark/ai-toolkit
8622
+ # or check PATH:
8623
+ export PATH="$(npm config get prefix)/bin:$PATH"
8624
+ ```
8625
+
8626
+ ### Doctor: broken symlinks
8627
+
8628
+ ```bash
8629
+ ai-toolkit doctor --fix # auto-repair
8630
+ ai-toolkit update # or full re-install
8631
+ ```
8632
+
8633
+ ### Tests fail: missing bats
8634
+
8635
+ ```bash
8636
+ brew install bats-core # macOS
8637
+ npm install -g bats # cross-platform
8638
+ ```
8639
+
8640
+ ### validate.py: stale counts
8641
+
8642
+ README badges don't match the current agents/skills/tests counts.
8643
+ Update README.md and re-run.
8644
+
8645
+ ### Eject: missing skills
8646
+
8647
+ ```bash
8648
+ ai-toolkit update # re-link missing symlinks
8649
+ ai-toolkit eject /tmp/test # retry
8650
+ ```
8677
8651
 
8678
8652
  ---
8679
8653
 
8680
- ## kb/procedures/release-verification-sop.md
8654
+ ## Success Criteria
8655
+
8656
+ | Area | Criterion |
8657
+ |------|-----------|
8658
+ | CLI | `--version` correct, `--help` full list, `status` current |
8659
+ | Health | `doctor`: 0 errors, 0 warnings, PASSED |
8660
+ | Install | `--dry-run` correct counts, `--local` all configs |
8661
+ | Quality | `validate.py --strict`: PASSED |
8662
+ | Security (baseline) | `audit_skills.py --ci`: 0 HIGH |
8663
+ | Security (SARIF) | `audit_skills.py --sarif`: valid SARIF 2.1.0 with non-empty rules array |
8664
+ | Security (permissions) | `audit_skills.py --permissions`: broad-access skills unchanged or justified in CHANGELOG |
8665
+ | Supply chain | `dist.attestations.provenance.predicateType == https://slsa.dev/provenance/v1` on npm |
8666
+ | Supply chain | All `sources.json` URL entries carry a `sha256`; `AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run` passes |
8667
+ | Tests | `npm test`: N/N passed, 0 failures |
8668
+ | Eject | Standalone `.claude/` with real files AND `output-styles/` directory |
8669
+ | Guards | Destructive commands blocked |
8670
+ | Deep coverage | `--profile full` emits every documented native surface, including Codex agents/skills/hooks and Copilot agents/skills/hooks |
8671
+ | Profile contract | Codex skills emit at every profile without `--codex-skills`; Copilot standard emits agents, skills, instructions, prompts, and hooks |
8672
+ | Idempotence | Second `install` run produces byte-identical output in every managed path |
8673
+ | Live JSON | Every generated `.json` file on disk parses as valid JSON |
8674
+ | Registry | `supported-tools-registry.md` enumerates every `scripts/generate_*.py` we ship |
8681
8675
 
8682
8676
  ---
8683
- title: "SOP: Release Verification"
8677
+
8678
+ ## kb/procedures/sop-release.md
8679
+
8680
+ ---
8681
+ title: "SOP: Release Preparation"
8684
8682
  category: procedures
8685
8683
  service: ai-toolkit
8686
- tags: [sop, verification, release, smoke-test, install, update, qa, provenance, sarif, dsh]
8687
- version: "1.8.0"
8688
- created: "2026-04-08"
8689
- last_updated: "2026-09-01"
8690
- description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit. Verifies CLI, native Codex and GitHub Copilot surfaces, explicit DSH lifecycle, Claude app export, doctor, validation, tests, eject, provenance, SARIF, and per-skill permissions."
8684
+ tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
8685
+ version: "1.15.0"
8686
+ created: "2026-04-10"
8687
+ last_updated: "2026-09-02"
8688
+ description: "Step-by-step checklist for preparing a new ai-toolkit release ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, branch CI, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, checksum-pin, ShellCheck, licensing, exact-tag assertions, and a green Ubuntu/macOS branch-CI gate before any release tag is created."
8691
8689
  ---
8692
8690
 
8693
- # SOP: Release Verification
8694
-
8695
- End-to-end smoke test after installing or updating `@softspark/ai-toolkit`.
8696
- Verifies all critical paths from the user's perspective.
8691
+ # SOP: Release Preparation
8697
8692
 
8698
- **Use this SOP when:**
8699
- - After `npm install -g @softspark/ai-toolkit@latest`
8700
- - After `ai-toolkit update`
8701
- - Before tagging a new version (`git tag`)
8702
- - Before publishing to npm (`npm publish`)
8703
- - As a smoke test in CI/CD
8693
+ Complete checklist for preparing a new `@softspark/ai-toolkit` release.
8694
+ Run this **before** tagging. After tagging and publishing, run the
8695
+ [Release Verification SOP](sop-release-verification.md) to smoke-test.
8704
8696
 
8705
- **Prerequisites:**
8706
- - Node.js >= 18, Python 3, `bats`, git
8707
- - `@softspark/ai-toolkit` installed globally
8697
+ **Pipeline:**
8698
+ ```
8699
+ Ecosystem Sync SOP (drift check + generator updates)
8700
+
8701
+ Release Preparation (this SOP)
8702
+
8703
+ git tag → CI publish → Release Verification SOP
8704
+ ```
8708
8705
 
8709
- **Time:** 10-15 minutes (full), 2 minutes (quick checklist)
8706
+ **Time:** 10-20 minutes (includes ecosystem sync review)
8710
8707
 
8711
8708
  ---
8712
8709
 
8713
8710
  ## Quick Checklist (TL;DR)
8714
8711
 
8715
- The 14 core commands below must pass. Releases that change DSH must also complete Phase 10.
8716
-
8717
8712
  ```bash
8718
- # Pre-commit (Phase 0)
8719
- python3 scripts/generate_agents_md.py > AGENTS.md # 1. Regenerate AGENTS.md
8720
- python3 scripts/generate_llms_txt.py > llms.txt # 2. Regenerate llms.txt
8721
- python3 scripts/validate.py --strict # 3. Validation passed?
8722
- npm test > /tmp/npm-test.log 2>&1 && grep -c '^ok ' /tmp/npm-test.log && ! grep -q '^not ok' /tmp/npm-test.log # 4. All tests passed? (single run, cached)
8713
+ # 0. Ecosystem sync (mandatory for minor/major releases; optional for patch)
8714
+ # Full procedure: kb/procedures/sop-ecosystem-sync.md
8715
+ python3 scripts/ecosystem_doctor.py --format text > /tmp/eco-report.txt
8716
+ cat /tmp/eco-report.txt
8717
+ # If drift detected: stop here, follow sop-ecosystem-sync.md Phase 2-4 to
8718
+ # classify each drift (A-F), update generators as needed, refresh snapshot,
8719
+ # THEN resume this SOP.
8720
+ python3 scripts/ecosystem_doctor.py --update # after all drift resolved
8723
8721
 
8724
- # Post-install verification (Phases 1-7)
8725
- ai-toolkit --version # 5. Version OK?
8726
- ai-toolkit status # 6. Status OK?
8727
- ai-toolkit doctor # 7. Health check passed?
8728
- ai-toolkit install --dry-run # 8. Global install OK?
8729
- python3 scripts/audit_skills.py --ci # 9. Security audit clean?
8722
+ # 1. Decide version bump
8723
+ # patch (1.4.2 1.4.3): bugfix, typo, doc fix
8724
+ # minor (1.4.2 1.5.0): new feature, new skill, new flag, any ecosystem-class-B/F change
8725
+ # major (1.4.2 2.0.0): breaking change, any ecosystem-class-D removed path
8730
8726
 
8731
- # Supply-chain verification (Phase 8, v2.8.0+)
8732
- python3 scripts/audit_skills.py --sarif | python3 -c "import json,sys; assert json.load(sys.stdin)['version']=='2.1.0'; print('SARIF OK')" # 10. SARIF 2.1.0 well-formed?
8733
- python3 scripts/audit_skills.py --permissions | head -30 # 11. Broad-access skills reviewed?
8734
- npm view @softspark/ai-toolkit@X.Y.Z --json | python3 -c "import json,sys; d=json.load(sys.stdin); assert d['dist']['attestations']['provenance']['predicateType']=='https://slsa.dev/provenance/v1'; print('PROVENANCE OK')" # 12. Provenance attested on npm?
8735
- python3 scripts/claude_app.py verify # 13. Claude Chat/Cowork plugin contract valid?
8727
+ # 2. Sync version across all files
8728
+ python3 scripts/sync_version.py X.Y.Z # if script exists, else manual
8736
8729
 
8737
- # Deep-coverage verification (Phase 9, v3.0.0+)
8738
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8739
- diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) && echo "OK: registry matches" # 14. Registry <-> generators drift?
8730
+ # 3. Write CHANGELOG.md entry
8731
+ # 4. Regenerate artifacts
8732
+ python3 scripts/generate_agents_md.py > AGENTS.md
8733
+ python3 scripts/generate_llms_txt.py > llms.txt
8734
+ python3 scripts/generate_llms_txt.py --full > llms-full.txt
8735
+
8736
+ # 5. Validate + audit + SARIF + shellcheck + test + ecosystem check
8737
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif && shellcheck --severity=warning app/hooks/*.sh && npm test
8738
+
8739
+ # 5a. Supply-chain standard (v2.8.0+) — non-negotiable
8740
+ grep -q -- '--provenance' .github/workflows/publish.yml || { echo "MISSING --provenance"; exit 1; }
8741
+ grep -q 'id-token: write' .github/workflows/publish.yml || { echo "MISSING id-token: write"; exit 1; }
8742
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8743
+
8744
+ # 5b. Ecosystem gate — snapshot must be current before tag
8745
+ python3 scripts/ecosystem_doctor.py --offline --check || { echo "STALE ecosystem snapshot — re-run doctor"; exit 1; }
8746
+
8747
+ # 5c. Licensing gate — SPDX headers, LICENSE, NOTICE, manifest consistency
8748
+ npx bats tests/test_licensing.bats || { echo "LICENSING GATE FAILED"; exit 1; }
8749
+
8750
+ # 6. Commit, push branch, and wait for the exact commit's full CI
8751
+ git add -A && git commit -m "chore: release vX.Y.Z"
8752
+ RELEASE_SHA=$(git rev-parse HEAD)
8753
+ git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
8754
+ RUN_ID=""
8755
+ for ATTEMPT in 1 2 3 4 5; do
8756
+ RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
8757
+ --limit 1 --json databaseId --jq '.[0].databaseId')
8758
+ [ -n "$RUN_ID" ] && break
8759
+ echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
8760
+ [ "$ATTEMPT" -eq 5 ] || sleep 60
8761
+ done
8762
+ test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
8763
+ gh run watch "$RUN_ID" --exit-status \
8764
+ || { echo "FAIL: release commit CI failed"; exit 1; }
8765
+ for OS in ubuntu-latest macos-latest; do
8766
+ gh run view "$RUN_ID" --json jobs \
8767
+ --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
8768
+ | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
8769
+ done
8770
+ test "$(git rev-parse HEAD)" = "$RELEASE_SHA" || { echo "FAIL: local HEAD changed during CI"; exit 1; }
8771
+ git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
8772
+ test "$(git rev-parse origin/main)" = "$RELEASE_SHA" || { echo "FAIL: origin/main changed during CI"; exit 1; }
8773
+
8774
+ # 6a. Only green branch CI authorizes tag creation
8775
+ git tag vX.Y.Z "$RELEASE_SHA"
8776
+
8777
+ # 6b. Assert the tag before pushing it (v4.19.0 postmortem, Phase 7)
8778
+ test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" || { echo "FAIL: tag not on tested release commit"; exit 1; }
8779
+ git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" || { echo "FAIL: tag not on release commit"; exit 1; }
8780
+
8781
+ # 6c. Push the single tag by full ref. Never --tags.
8782
+ git push origin refs/tags/vX.Y.Z
8740
8783
  ```
8741
8784
 
8742
8785
  ---
8743
8786
 
8744
- ## Phase 0: Pre-Commit & Pre-Push (2 min)
8787
+ ## Phase 0: Ecosystem Sync (MANDATORY for minor/major)
8745
8788
 
8746
- Run these commands **before every commit and push to main**. CI validates
8747
- counts but does NOT auto-regenerate — you must do it locally.
8789
+ Before touching version numbers, confirm the toolkit is aligned with the current state of every editor / platform it integrates with. Skipping this phase ships a release whose generators may lag a month-old CLI refactor, a rename of `.cursorrules` to `.cursor/rules/`, or a new hook event we do not yet emit.
8790
+
8791
+ **When this phase is mandatory:**
8792
+ - Minor release (X.Y.0) — always
8793
+ - Major release (X.0.0) — always
8794
+ - Patch release (X.Y.Z) — only if the patch touches a generator or install flow
8795
+
8796
+ **When to skip:** pure doc-only patches, SOP edits, internal refactors that do not touch `scripts/generate_*` or `app/skills/*/SKILL.md`.
8797
+
8798
+ ### 0.1 Run the doctor
8748
8799
 
8749
8800
  ```bash
8750
- # 1. Regenerate generated artifacts
8751
- python3 scripts/generate_agents_md.py > AGENTS.md
8752
- python3 scripts/generate_llms_txt.py > llms.txt
8753
- python3 scripts/generate_llms_txt.py --full > llms-full.txt
8801
+ python3 scripts/ecosystem_doctor.py --format text | tee /tmp/eco-report.txt
8802
+ ```
8754
8803
 
8755
- # 2. Validate everything (catches stale counts, missing assets)
8756
- python3 scripts/validate.py --strict
8804
+ Output classifies every registered tool as **Clean**, **Drift**, or **Errored**.
8757
8805
 
8758
- # 3. Security audit
8759
- python3 scripts/audit_skills.py --ci
8806
+ ### 0.2 Act on drift
8760
8807
 
8761
- # 4. Run tests
8762
- npm test
8808
+ For each drifting tool, follow [sop-ecosystem-sync.md](sop-ecosystem-sync.md) Phase 2-4:
8763
8809
 
8764
- # 5. Stage and commit
8765
- # AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated and
8766
- # gitignored do NOT commit them (regenerated at prepublishOnly). Only the
8767
- # doc artifacts llms.txt / llms-full.txt are committed.
8768
- git add llms.txt llms-full.txt
8769
- git add -p # stage your other changes
8770
- git commit -m "feat: your change description"
8771
- ```
8810
+ | Drift class | Release impact |
8811
+ |-------------|----------------|
8812
+ | A (cosmetic reword) | No version impact refresh snapshot, continue |
8813
+ | B (new feature integrate) | **Minor** version bump at minimum; new generator or extended generator |
8814
+ | C (new feature — not adopted) | No impact — note in registry |
8815
+ | D (deprecation) | **Minor** or **major** depending on user impact; add migration warning |
8816
+ | E (feature promoted to default) | **Minor**; simplify generator, keep fallback comment |
8817
+ | F (feature newly globally available) | **Minor**; may require new generator or new config path |
8772
8818
 
8773
- **Why local?** Branch protection on `main` requires PRs and status checks.
8774
- CI cannot push directly to `main`, so generated artifacts must be committed
8775
- by the developer as part of their PR.
8819
+ If any B/D/E/F changes land in this preparation pass, mention them explicitly in the CHANGELOG entry (Phase 3) under a `Ecosystem` subsection.
8820
+
8821
+ ### 0.3 Refresh snapshot
8822
+
8823
+ Once every drift is resolved (either by code change or by re-classifying as acceptable):
8776
8824
 
8777
- **One-liner (copy-paste):**
8778
8825
  ```bash
8779
- python3 scripts/generate_agents_md.py > AGENTS.md && python3 scripts/generate_llms_txt.py > llms.txt && python3 scripts/generate_llms_txt.py --full > llms-full.txt && python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
8826
+ python3 scripts/ecosystem_doctor.py --update
8780
8827
  ```
8781
8828
 
8782
- ---
8829
+ This writes the new baseline to `benchmarks/ecosystem-doctor-snapshot.json`. Commit it as part of the release commit.
8783
8830
 
8784
- ## Phase 1: CLI & Version (1 min)
8831
+ ### 0.4 Gate
8785
8832
 
8786
8833
  ```bash
8787
- ai-toolkit --version
8788
- ai-toolkit --help
8789
- which ai-toolkit
8834
+ python3 scripts/ecosystem_doctor.py --offline --check
8790
8835
  ```
8791
8836
 
8792
- **Verify:**
8793
- - [ ] `--version` returns correct semver (e.g., `1.4.0`)
8794
- - [ ] `--help` displays full command list without errors
8795
- - [ ] `which` points to global npm bin path
8837
+ Must exit `0`. If it exits `1`, the snapshot is stale — rerun Phase 0.3 or review the remaining drift.
8796
8838
 
8797
8839
  ---
8798
8840
 
8799
- ## Phase 2: Global Install & Status (2 min)
8841
+ ## Phase 1: Determine Version Bump
8800
8842
 
8801
- ```bash
8802
- ai-toolkit install --dry-run
8803
- ai-toolkit status
8804
- ```
8843
+ Follow [Semantic Versioning](https://semver.org/):
8805
8844
 
8806
- **Verify `--dry-run`:**
8807
- - [ ] Agents: 44
8808
- - [ ] Skills: 108
8809
- - [ ] Hooks merged into settings.json
8810
- - [ ] "Other AI Tools" lists documented global targets (with `--editors`): aider, antigravity, augment, cline, codex, copilot, cursor, gemini, opencode, roo, windsurf. Scope varies: Codex uses `$CODEX_HOME` (default `~/.codex`) plus `$HOME/.agents/skills`; Copilot uses `$COPILOT_HOME` (default `~/.copilot`); cursor has only `~/.cursor/hooks.json`; antigravity has the `~/.gemini/*/skills` pointer. Cursor and Antigravity rules remain project-only.
8845
+ | Change Type | Bump | Examples |
8846
+ |-------------|------|---------|
8847
+ | Bugfix, typo, doc-only | **patch** | Fix install flag, correct description |
8848
+ | New feature, skill, agent, flag | **minor** | Add `/hipaa-validate`, add `--output json` |
8849
+ | Breaking CLI change, removed skill, config format change | **major** | Rename `install` to `setup`, remove skill |
8811
8850
 
8812
- **Verify `status`:**
8813
- - [ ] Version matches expected
8814
- - [ ] Profile: minimal/standard/strict
8815
- - [ ] Modules: list of installed modules
8816
- - [ ] Latest: up to date / update available
8851
+ **Rule:** When in doubt, bump minor.
8817
8852
 
8818
8853
  ---
8819
8854
 
8820
- ## Phase 3: Doctor Health Check (1 min)
8855
+ ## Phase 2: Sync Version in All Files
8821
8856
 
8822
- ```bash
8823
- ai-toolkit doctor
8824
- ```
8857
+ The canonical version lives in `package.json`. These files **must** match:
8825
8858
 
8826
- **Expected sections (all OK):**
8827
- - Environment: node, bash, python3, bats
8828
- - Global Install: .claude exists, agents/skills symlinks (0 broken), settings.json hooks
8829
- - Hook Scripts: all present and executable
8830
- - Hook Configuration: 14 events registered
8831
- - Generated Artifacts: AGENTS.md, llms.txt, llms-full.txt
8832
- - Planned Assets: plugin.json, benchmarks, plugin packs
8833
- - Benchmark Freshness: < 30 days
8834
- - Stale Rules: all healthy
8859
+ ### Mandatory sync (every release)
8835
8860
 
8836
- **Verify:**
8837
- - [ ] `Errors: 0 | Warnings: 0`
8838
- - [ ] `HEALTH CHECK PASSED`
8861
+ | File | Field | How to update |
8862
+ |------|-------|---------------|
8863
+ | `package.json` | `"version": "X.Y.Z"` | Edit directly |
8864
+ | `manifest.json` | `"version": "X.Y.Z"` | Edit directly |
8865
+ | `app/.claude-plugin/plugin.json` | `"version": "X.Y.Z"` | Edit directly |
8839
8866
 
8840
- If doctor detects problems: `ai-toolkit doctor --fix` auto-repairs
8841
- (broken symlinks, non-executable hooks, missing scripts, missing llms-full.txt).
8867
+ ### Auto-synced (no manual action)
8842
8868
 
8843
- ---
8869
+ | File | Mechanism |
8870
+ |------|-----------|
8871
+ | `package-lock.json` | Regenerated by `npm install --package-lock-only` |
8844
8872
 
8845
- ## Phase 4: Local Install (2 min)
8873
+ ### Conditional sync (only if the doc was modified in this release)
8874
+
8875
+ | File | Field | When to update |
8876
+ |------|-------|---------------|
8877
+ | `kb/procedures/sop-maintenance.md` | frontmatter `version:` | If SOP content changed |
8878
+ | `kb/reference/skills-catalog.md` | frontmatter `version:` | If skills added/removed |
8879
+ | `kb/reference/agents-catalog.md` | frontmatter `version:` | If agents added/removed |
8880
+ | `kb/reference/hooks-catalog.md` | frontmatter `version:` | If hooks changed |
8881
+ | `kb/reference/architecture-overview.md` | frontmatter `version:` | If architecture changed |
8882
+ | `kb/reference/distribution-model.md` | frontmatter `version:` | If install model changed |
8883
+ | `kb/reference/global-install-model.md` | frontmatter `version:` | If install model changed |
8884
+
8885
+ > **Note:** KB `version:` fields track the **document version**, not the toolkit version.
8886
+ > Only bump them when the document content actually changes in this release.
8887
+
8888
+ ### Count sync (if skills/agents/hooks changed)
8889
+
8890
+ | File | What to check |
8891
+ |------|---------------|
8892
+ | `package.json` | `"description"` — skill/agent count |
8893
+ | `README.md` | Badge counts, "What You Get" table |
8894
+ | `app/ARCHITECTURE.md` | Section headings with counts |
8895
+
8896
+ > **Tip:** `validate.py --strict` catches count drift AND version mismatches
8897
+ > (package.json vs manifest.json vs plugin.json) automatically.
8898
+ > If validation passes, counts and versions are correct.
8899
+
8900
+ ### Verification command
8901
+
8902
+ After syncing, verify all mandatory files match:
8846
8903
 
8847
8904
  ```bash
8848
- mkdir -p /tmp/ai-toolkit-verify && cd /tmp/ai-toolkit-verify
8849
- git init -q
8850
- ai-toolkit install --local --editors all --dry-run
8851
- cd - && rm -rf /tmp/ai-toolkit-verify
8905
+ VERSION=$(python3 -c "import json; print(json.load(open('package.json'))['version'])")
8906
+ echo "Target: $VERSION"
8907
+ echo "manifest.json: $(python3 -c "import json; print(json.load(open('manifest.json'))['version'])")"
8908
+ echo "plugin.json: $(python3 -c "import json; print(json.load(open('app/.claude-plugin/plugin.json'))['version'])")"
8909
+ echo "package-lock.json: $(python3 -c "import json; print(json.load(open('package-lock.json'))['version'])")"
8852
8910
  ```
8853
8911
 
8854
- **Verify "Project-local" section:**
8855
- - [ ] Would create: CLAUDE.md
8856
- - [ ] Would create: .claude/settings.local.json
8857
- - [ ] Would inject: .claude/constitution.md
8858
- - [ ] Editors: all 11 listed (copilot, cursor, windsurf, cline, roo, aider, augment, antigravity, codex, gemini, opencode)
8859
- - [ ] Would generate configs for each editor (legacy + directory-based)
8860
- - [ ] Would install: .git/hooks/pre-commit
8861
- - [ ] Would inject language rules (auto-detected)
8912
+ All four must print the same version. If not, fix before proceeding.
8913
+
8914
+ ### Public surface review
8862
8915
 
8863
- **Also test auto-detect (no --editors flag):**
8864
8916
  ```bash
8865
- ai-toolkit install --local --dry-run
8866
- # Editors: none (empty project has no existing configs)
8917
+ python3 scripts/surface_manifest.py --update
8918
+ git diff app/surface.json
8867
8919
  ```
8868
8920
 
8869
- ---
8921
+ **Every line the diff removes is a breaking change.** Restore it, or take the
8922
+ deprecation path in `BACKWARD_COMPATIBILITY.md` and add a `DECISIONS.md` entry
8923
+ before the tag. Lines added are new surface being adopted into protection — that
8924
+ needs no ceremony.
8870
8925
 
8871
- ## Phase 5: Validation & Security Audit (3 min)
8926
+ Do not run `--update` to make a red build green. The check fails because something
8927
+ users depend on disappeared; regenerating the manifest deletes the evidence, not
8928
+ the problem.
8872
8929
 
8873
- ```bash
8874
- python3 scripts/validate.py --strict
8875
- python3 scripts/audit_skills.py --ci
8930
+ ### Skill body budget ratchet
8931
+
8932
+ `validate.py` prints the largest skill body on every run:
8933
+
8934
+ ```
8935
+ Body budget: largest is <skill> at <N> bytes (warn 18000, error 20000)
8876
8936
  ```
8877
8937
 
8878
- **Verify validate.py:**
8879
- - [ ] Agents: 44, Skills: 108, Tests: exactly the current README badge count
8880
- - [ ] Hook events: 14, Hook scripts: >= 30
8881
- - [ ] Plugin packs >= 10, KB documents >= 20
8882
- - [ ] `Errors: 0 | Warnings: 0` → `VALIDATION PASSED`
8938
+ Once that number sits at least 2000 bytes under `SKILL_BODY_BUDGET_WARN`, lower
8939
+ `SKILL_BODY_BUDGET_WARN` by 2000 in `scripts/validate.py` and ship the tightened
8940
+ threshold with the release. Target floor is 12000.
8883
8941
 
8884
- **Verify audit_skills.py:**
8885
- - [ ] `HIGH: 0` (MUST be zero — CI fails otherwise)
8886
- - [ ] `WARN: 0`
8887
- - [ ] `INFO: N` (acceptable broad-access skills: orchestrate, swarm, workflow)
8942
+ Two rules, both learned the hard way:
8943
+
8944
+ - **Never lower a threshold in the same change that something violates it.** Split
8945
+ the offending skill into `SKILL.md` + `reference/` first, prove it with
8946
+ `python3 scripts/check_split.py <skill> --before <pre-split SKILL.md>`, then
8947
+ tighten in a follow-up.
8948
+ - **Never raise a threshold to make a red build green.** A body over budget means
8949
+ detail that belongs in `reference/` is loading on every trigger match, including
8950
+ the accidental ones. Raising the number hides the cost, it does not remove it.
8888
8951
 
8889
- ---
8952
+ If the largest body has not moved since the previous release, leave the threshold
8953
+ alone and say so in the release notes. A ratchet that never advances is honest;
8954
+ one that advances past reality is not.
8890
8955
 
8891
- ## Phase 6: Tests (3-5 min)
8956
+ ---
8892
8957
 
8893
- ```bash
8894
- # Run ONCE, capture to file, then parse. Full suite is 669+ bats cases —
8895
- # re-running it per check (tail / grep ok / grep not ok piped separately)
8896
- # wastes minutes every release. Always cache the output.
8897
- npm test > /tmp/npm-test.log 2>&1
8898
- exit=$?
8899
- tail -3 /tmp/npm-test.log
8900
- echo "ok: $(grep -c '^ok ' /tmp/npm-test.log)"
8901
- echo "not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8902
- echo "exit: $exit"
8903
- ```
8958
+ ## Phase 3: Write CHANGELOG Entry
8904
8959
 
8905
- **Verify:**
8906
- - [ ] `exit == 0`
8907
- - [ ] `ok == expected test count` (e.g., 945 on v3.0.0)
8908
- - [ ] `not ok == 0`
8909
- - [ ] Bats runs tests in parallel (4 jobs)
8910
- - [ ] Groups: agents, autodetect, cli, generators, guards, hooks, inject,
8911
- install, kb, mcp, readme, profiles, uninstall, validate
8960
+ Add entry at the top of `CHANGELOG.md` (after the header, before previous release):
8912
8961
 
8913
- **Anti-pattern — do NOT do this:**
8914
- ```bash
8915
- # Runs the full suite THREE times. Adds 1-3 min and pressures CI capacity.
8916
- npm test 2>&1 | tail -3
8917
- npm test 2>&1 | grep -c '^ok '
8918
- npm test 2>&1 | grep -c '^not ok'
8919
- ```
8962
+ ```markdown
8963
+ ## vX.Y.Z — Short Title (YYYY-MM-DD)
8920
8964
 
8921
- **Key test areas:**
8922
- - Guards: rm -rf, DROP TABLE, git push --force blocked
8923
- - Install: idempotent, profiles, --only/--skip, orphan cleanup
8924
- - Eject: real files (not symlinks), inlined rules
8925
- - Uninstall: removes toolkit, preserves user content
8965
+ ### Added
8966
+ - **Feature name** description
8926
8967
 
8927
- ---
8968
+ ### Changed
8969
+ - **What changed** — old behavior → new behavior
8928
8970
 
8929
- ## Phase 7: Eject (1 min)
8971
+ ### Fixed
8972
+ - **Bug description** — what was broken and how it's fixed
8930
8973
 
8931
- ```bash
8932
- mkdir -p /tmp/ai-toolkit-eject-test
8933
- cd /tmp/ai-toolkit-eject-test
8934
- ai-toolkit eject
8935
- cd - && rm -rf /tmp/ai-toolkit-eject-test
8974
+ ### Removed
8975
+ - **What was removed** — migration path if any
8936
8976
  ```
8937
8977
 
8938
- **Verify:**
8939
- - [ ] Agents copied as real files (not symlinks)
8940
- - [ ] Skills copied as real directories
8941
- - [ ] Rules inlined into CLAUDE.md
8942
- - [ ] constitution.md and ARCHITECTURE.md copied
8943
- - [ ] `output-styles/` directory present (v2.7.1+)
8978
+ **Rules:**
8979
+ - Use **bold** for feature names
8980
+ - Start descriptions with a verb (Added, Changed, Fixed, Removed)
8981
+ - Reference skill names with backticks and slash: `/hipaa-validate`
8982
+ - Include script names: `scripts/hipaa_scan.py`
8983
+ - Include count changes: `Skill count: 91 → 92`
8984
+ - Date format: `YYYY-MM-DD`
8985
+ - Title: short, descriptive, no version number repetition
8944
8986
 
8945
- ---
8987
+ ### Update README "What's New" section
8946
8988
 
8947
- ## Phase 8: Supply-Chain Verification (2 min, v2.8.0+)
8989
+ **MANDATORY on every release.** Update the `## What's New in vX.Y.Z` section in `README.md`:
8948
8990
 
8949
- These checks enforce the v2.8.0 security standard on a freshly-published release. Run AFTER the `publish.yml` workflow completes on the tag.
8991
+ 1. Change the heading version: `## What's New in vX.Y.Z`
8992
+ 2. Replace bullet points with 3-5 highlights from this release
8993
+ 3. **Keep only the latest version block.** Delete the previous `## What's New in vA.B.C` section(s). README is the shop window, not the archive — users see the current release, full history lives in `CHANGELOG.md`.
8994
+ 4. Keep the `See [CHANGELOG.md](CHANGELOG.md) for full history.` link directly below the bullet list.
8950
8995
 
8951
- ### 8.1 Provenance attestation on npm
8996
+ > **Warning:** This section is the first thing users see after the badges.
8997
+ > A stale version here (e.g., "What's New in v2.1.3" when shipping v2.3.0)
8998
+ > signals an unmaintained project. Do NOT skip this step.
8952
8999
 
8953
- ```bash
8954
- VERSION="X.Y.Z" # the tag just published
8955
- npm view "@softspark/ai-toolkit@${VERSION}" --json \
8956
- | python3 -c "import json, sys; d=json.load(sys.stdin); att=d['dist'].get('attestations', {}); assert att.get('provenance', {}).get('predicateType') == 'https://slsa.dev/provenance/v1', f'NO PROVENANCE for {d[\"version\"]}'; print(f'PROVENANCE OK: {att[\"url\"]}')"
8957
- ```
9000
+ > **Single-version rule:** README.md must contain **exactly one** `## What's New in vX.Y.Z` heading at any time. If you find multiple stacked (e.g. v2.6.1 + v2.6.0 + v2.5.0), that is a SOP drift — collapse to the latest on the next release commit.
8958
9001
 
8959
- **Verify:**
8960
- - [ ] Exit 0 and prints `PROVENANCE OK: https://registry.npmjs.org/...`
8961
- - [ ] `https://www.npmjs.com/package/@softspark/ai-toolkit/v/${VERSION}` shows the green "Provenance" badge
9002
+ ---
8962
9003
 
8963
- **If provenance is missing:** the `publish.yml` ran without `id-token: write` or `--provenance`. Restore them and cut a patch release — a silently unsigned publish is a regression against the v2.8.0 standard.
9004
+ ## Phase 4: Regenerate Artifacts
8964
9005
 
8965
- ### 8.2 Audit SARIF output (for GHAS ingest)
9006
+ Use the npm scripts, not the generators directly:
8966
9007
 
8967
9008
  ```bash
8968
- python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif
8969
- python3 -c "import json; d=json.load(open('/tmp/audit.sarif')); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']=='ai-toolkit-audit-skills'; print(f'SARIF OK: {len(d[\"runs\"][0][\"results\"])} results across {len(d[\"runs\"][0][\"tool\"][\"driver\"][\"rules\"])} rules')"
9009
+ npm run generate:agents # AI_TOOLKIT_NO_CUSTOM_RULES=1 python3 scripts/generate_agents_md.py > AGENTS.md
9010
+ npm run generate:llms # llms.txt + llms-full.txt
8970
9011
  ```
8971
9012
 
8972
- **Verify:**
8973
- - [ ] Valid SARIF 2.1.0
8974
- - [ ] In the publishing repo, the CI job uploads `audit.sarif` via `github/codeql-action/upload-sarif@v3` so findings appear in the Security tab
9013
+ `generate:agents` sets `AI_TOOLKIT_NO_CUSTOM_RULES=1`. Running
9014
+ `generate_agents_md.py` bare picks up whatever is registered in the maintainer's
9015
+ own `~/.softspark/ai-toolkit/rules/`, which then ships inside `AGENTS.md`.
8975
9016
 
8976
- ### 8.3 Per-skill permissions report
9017
+ Check if anything actually changed:
8977
9018
 
8978
9019
  ```bash
8979
- python3 scripts/audit_skills.py --permissions | head -40
9020
+ git diff --stat AGENTS.md llms.txt llms-full.txt
8980
9021
  ```
8981
9022
 
8982
- **Verify:**
8983
- - [ ] Bash skill count has NOT jumped unexpectedly since the previous release
8984
- - [ ] Any newly added entry under `Skills with Bash + Write + Edit` matches a CHANGELOG bullet that justifies the broad scope
8985
- - [ ] JSON form (`--permissions --json`) is available for automated drift dashboards
8986
-
8987
- ### 8.4 URL-sourced rules/hooks are checksum-pinned
8988
-
8989
- `sources.json` is an envelope of the form `{"schema_version": 1, "rules"|"hooks": {...}}`, so the jq filter must pick the nested map before piping into the pin assertion. The URL-count in the success message ignores local `path:`-only entries — only entries with a `url:` field are pinned and counted.
9023
+ If no diff, the artifacts are already current. If there is a diff, stage them.
8990
9024
 
8991
- ```bash
8992
- jq '.rules // .hooks // {}' ~/.softspark/ai-toolkit/rules/sources.json 2>/dev/null \
8993
- | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'RULE PIN OK: {url_n} URL rules, all with sha256')"
8994
- jq '.hooks // {}' ~/.softspark/ai-toolkit/hooks/external/sources.json 2>/dev/null \
8995
- | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'HOOK PIN OK: {url_n} URL hooks, all with sha256')"
8996
- ```
9025
+ ---
8997
9026
 
8998
- **Verify:**
8999
- - [ ] Both commands print `... PIN OK`
9000
- - [ ] If any entry is unpinned, the `register_url_source()` call missed passing `content=` — fix the call-site and retag
9027
+ ## Phase 5: Validate, Audit, Test
9001
9028
 
9002
- ### 8.5 Strict-pin smoke test (optional but recommended)
9029
+ Run the full quality gate:
9003
9030
 
9004
9031
  ```bash
9005
- AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
9006
- ```
9032
+ python3 scripts/validate.py --strict
9033
+ python3 scripts/audit_skills.py --ci
9034
+ python3 scripts/audit_skills.py --sarif > audit.sarif # MANDATORY — GHAS ingest
9035
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
9007
9036
 
9008
- **Verify:**
9009
- - [ ] Exit 0, no `CHECKSUM CHANGED` line
9010
- - [ ] If a checksum change was intentional (e.g. upstream rule update), document it in the CHANGELOG entry before tagging
9037
+ # ShellCheck on hooks (added in 1.11.0). Mirrors the ci.yml "ShellCheck hooks"
9038
+ # job. NOT run by validate.py, npm test, OR publish.yml — so a hook with a
9039
+ # ShellCheck warning passes every other gate AND still publishes on tag while
9040
+ # turning main CI red. Run it here, before tagging.
9041
+ shellcheck --severity=warning app/hooks/*.sh && echo "OK: shellcheck clean"
9011
9042
 
9012
- ---
9043
+ # Registry / generator drift (added in 1.10.0). Meta-generators excluded.
9044
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
9045
+ diff \
9046
+ <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) \
9047
+ <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) \
9048
+ && echo "OK: registry matches filesystem" \
9049
+ || { echo "DRIFT: update supported-tools-registry.md before tagging"; exit 1; }
9013
9050
 
9014
- ## Phase 9: Deep-Coverage Checks (v3.0.0+)
9051
+ # Stage first IF this release adds or deletes a kb/ file. The test
9052
+ # "npm package KB files match the tracked release set" compares `git ls-files
9053
+ # kb` (the index) against what `npm pack` sees (the working tree), so an
9054
+ # unstaged addition reads as "extra" and an unstaged deletion as "missing".
9055
+ # Phase 6 stages, and it runs after this one, so the ordering fails the test
9056
+ # for any release that touches the KB. Staging early costs nothing.
9057
+ git status --porcelain kb/ | grep -qE '^(\?\?| D|\?M)' && git add -A kb/
9015
9058
 
9016
- These verify the native-surface generators shipped in v3.0.0 actually emit the right files for the right profiles, and that the tool registry stays in sync with shipped generators.
9059
+ # Run npm test ONCE, cache output, parse from file. The suite is 1400+ bats
9060
+ # cases — rerunning it per check wastes minutes. Do not pipe npm test into
9061
+ # tail/grep multiple times in the same session.
9062
+ npm test > /tmp/npm-test.log 2>&1
9063
+ tail -3 /tmp/npm-test.log
9064
+ echo "ok: $(grep -c '^ok ' /tmp/npm-test.log) | not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
9065
+ ```
9017
9066
 
9018
- > **Safety warning — HOME-scoped writes:** Running `--profile full` with `augment` in the editor list writes to `$HOME/.augment/settings.json` (Augment stores hooks under HOME, not per-project). Use `--dry-run` for verification unless you intend to carry ai-toolkit hook entries on this machine. The generator is marker-safe (only rewrites its own `_source: ai-toolkit` entries) but is still a side-effect.
9067
+ **Expected results:**
9068
+ - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
9069
+ - `audit_skills.py --ci`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
9070
+ - `audit_skills.py --sarif`: valid JSON, non-empty `runs[0].tool.driver.rules`
9071
+ - `audit_skills.py --permissions`: review `Skills with Bash + Write + Edit` list — any newly-added skill with broad access MUST be justified in the CHANGELOG entry
9072
+ - `shellcheck --severity=warning app/hooks/*.sh`: no output, exit 0. A common false positive is `SC2034` on `INPUT` or env vars (e.g. `AI_TOOLKIT_HOOK_FORMAT`) that a *sourced* helper (`_hook-io.sh`) consumes — ShellCheck cannot see cross-file use. Fix with a `# shellcheck disable=SC2034` directive or `export`, matching `guard-destructive.sh`. Never tag with a red ShellCheck.
9073
+ - Registry drift: `OK: registry matches filesystem`. If `DRIFT:` appears, add the missing `scripts/generate_*.py` rows to `kb/reference/supported-tools-registry.md` before tagging.
9074
+ - `npm test`: `1..N` with zero `not ok` (read from the cached `/tmp/npm-test.log`, do not rerun)
9019
9075
 
9020
- ### 9.1 `--profile full` emits every native surface
9076
+ > **Why this matters (v4.5.1 postmortem):** `publish.yml` runs only `validate.py` + `npm test`, so it published v4.5.0 even though the `main` CI `ShellCheck hooks` job was red on two `SC2034` warnings in a new hook. The publish workflow does **not** depend on the CI workflow. Until that is fixed, ShellCheck is a manual pre-tag gate — run it here every time.
9021
9077
 
9078
+ **One-liner:**
9022
9079
  ```bash
9023
- D=/tmp/aitk-profile-full-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9024
- ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot \
9025
- --profile full --dry-run 2>&1 \
9026
- | grep -E "\\.cursor/(hooks\\.json|agents)|\\.devin/hooks\\.v1\\.json|\\.gemini/(settings\\.json|commands)|\\.augment/(agents|commands)|\\.codex/(agents|hooks)|\\.agents/skills|\\.github/(agents|skills|instructions|prompts|hooks)"
9080
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > audit.sarif && shellcheck --severity=warning app/hooks/*.sh && diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE 'generate_agents_md\.py|generate_llms_txt\.py|generate_language_rules_skills\.py' | sort -u) && npm test
9027
9081
  ```
9028
9082
 
9029
- **Verify** at least the following lines appear:
9030
- - [ ] `.cursor/hooks.json` and `.cursor/agents/`
9031
- - [ ] `.devin/hooks.v1.json`
9032
- - [ ] `.gemini/settings.json` hooks AND `.gemini/commands/`
9033
- - [ ] `.augment/agents/` + `.augment/commands/` + `$HOME/.augment/settings.json`
9034
- - [ ] `.codex/hooks.json` + `.codex/hooks/` + `.codex/agents/`
9035
- - [ ] `.agents/skills/` (Codex native discovery path)
9036
- - [ ] `.github/agents/` + `.github/skills/` + `.github/instructions/` + `.github/prompts/` + `.github/hooks/`
9083
+ **If tests fail:** Fix the issue, do NOT skip. Common failures:
9084
+ - Stale counts → re-run `generate:all` or fix README/ARCHITECTURE
9085
+ - Missing frontmatter → add to new KB docs
9086
+ - Broken symlink `ai-toolkit doctor --fix`
9037
9087
 
9038
- ### 9.2 Codex skills are profile-independent
9088
+ ### Phase 5c: Licensing Gate (v4.20.0+)
9089
+
9090
+ The project is Apache-2.0. Attribution only works if the artefact actually
9091
+ carries it, and every part of that is mechanically checkable.
9039
9092
 
9040
9093
  ```bash
9041
- D=/tmp/aitk-codex-skills-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9042
- ai-toolkit install --local --editors codex --profile minimal --dry-run 2>&1 \
9043
- | grep -q "Would generate: .agents/skills" && echo "OK: Codex skills emit at minimal"
9044
- ai-toolkit install --local --editors codex --profile standard --codex-skills --dry-run 2>&1 \
9045
- | grep -q "Would refresh: .agents/skills" && echo "OK: legacy flag explicitly refreshes the same catalog"
9094
+ # The whole gate, enforced in CI. Run it here so a failure is caught before tagging.
9095
+ npx bats tests/test_licensing.bats
9046
9096
  ```
9047
9097
 
9048
- **Verify:**
9049
- - [ ] Selecting Codex emits all skills under `.agents/skills/` at every profile
9050
- - [ ] `--codex-skills` remains an explicit refresh alias; it is not required for installation
9051
- - [ ] No profile emits `.codex/skills/`
9098
+ The seven assertions, and why each exists:
9052
9099
 
9053
- ### 9.3 Breaking-change surfaces land on `--profile standard`
9100
+ | Check | Fails when |
9101
+ |---|---|
9102
+ | Every shipped source file carries an SPDX header | A new `.py`/`.sh`/`.js`/`.bats` file was added without one — the common case, and the reason this is a test rather than a habit |
9103
+ | Headers name Apache-2.0 and nothing else | A file was copied in from an MIT/GPL source with its own header intact |
9104
+ | **No** markdown file carries an SPDX header | Someone "helpfully" ran the header script over `app/skills/` — headers there sit above parsed frontmatter and bill every session for it |
9105
+ | `LICENSE` is the complete Apache 2.0 text | The file was truncated or replaced with a summary |
9106
+ | `NOTICE` carries attribution, the source URL, §4(d) and the MIT-era notice | The attribution mechanism was gutted |
9107
+ | `LICENSE` **and** `NOTICE` ship in the npm package | `package.json` `files` lost an entry — a NOTICE that never reaches the consumer cannot satisfy §4(d) |
9108
+ | Every manifest declaring a licence declares Apache-2.0 | `package.json`, `manifest.json`, `plugin.json` and `package-lock.json` drifted apart |
9054
9109
 
9055
- The standard profile must include:
9056
- - Copilot scoped instructions, prompts, custom agents, portable skills, and hooks
9057
- - Gemini hooks (`.gemini/settings.json`)
9110
+ **Adding source files in this release?** The header goes *after* the shebang,
9111
+ never before it. Short SPDX form:
9058
9112
 
9059
- ```bash
9060
- D=/tmp/aitk-breaking-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9061
- ai-toolkit install --local --editors copilot,gemini --profile standard --dry-run 2>&1 \
9062
- | tee /tmp/aitk-breaking.log
9063
- grep -q "\\.github/instructions/" /tmp/aitk-breaking.log && echo "OK: Copilot dir layout at standard"
9064
- grep -q "\\.github/agents/.*\\.github/skills/" /tmp/aitk-breaking.log && echo "OK: Copilot agents and skills at standard"
9065
- grep -q "\\.github/hooks/" /tmp/aitk-breaking.log && echo "OK: Copilot hooks at standard"
9066
- grep -q "\\.gemini/settings\\.json hooks" /tmp/aitk-breaking.log && echo "OK: Gemini hooks at standard"
9067
9113
  ```
9114
+ # SPDX-License-Identifier: Apache-2.0
9115
+ # Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
9116
+ # Source: https://github.com/softspark/ai-toolkit
9117
+ ```
9118
+
9119
+ `//` for JavaScript. Full convention and the reasoning behind the markdown
9120
+ exclusion: [Licensing](../reference/licensing.md).
9121
+
9122
+ **Changing the licence itself?** Do not hand-type the licence text. Take it
9123
+ verbatim from a published copy and cross-verify against a second independent
9124
+ copy before writing `LICENSE` — a rendered or summarised licence is not the
9125
+ licence. Prior releases stay under their original terms; a licence change
9126
+ applies going forward and revokes nothing already granted.
9127
+
9128
+ ### Phase 5a: Supply-Chain Hardening Verification (v2.8.0+)
9068
9129
 
9069
- **Verify all four lines print `OK:`**. If any is missing, native Copilot or Gemini coverage has regressed.
9130
+ These checks enforce the security standard introduced in v2.8.0. Do NOT tag a release until all pass.
9070
9131
 
9071
- ### 9.4 Install is idempotent
9132
+ **1. Publish workflow emits provenance:**
9072
9133
 
9073
9134
  ```bash
9074
- D=/tmp/aitk-idem-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9075
- # Sort file paths before hashing — find traversal order follows inode order,
9076
- # which can shift between runs even when content is byte-identical, producing
9077
- # false FAIL signals.
9078
- ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
9079
- SHA1=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
9080
- ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
9081
- SHA2=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
9082
- [ "$SHA1" = "$SHA2" ] && echo "OK: idempotent" || echo "FAIL: install is not idempotent"
9135
+ grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
9083
9136
  ```
9084
9137
 
9085
- **Verify:** prints `OK: idempotent`. A second run must produce byte-identical files in every managed path.
9086
-
9087
- ### 9.5 Live-install JSON outputs parse
9138
+ - [ ] Both markers present (`--provenance` flag + `id-token: write` permission)
9139
+ - [ ] Any PR that changes `publish.yml` REQUIRES an approved security review
9088
9140
 
9089
- The bats suite validates JSON shape at generation time. This re-checks that what actually landed on disk after a live install parses without errors.
9141
+ **2. URL-sourced rules and hooks are checksum-pinned:**
9090
9142
 
9091
9143
  ```bash
9092
- D=/tmp/aitk-json-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9093
- ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot --profile full >/dev/null 2>&1
9094
- for f in .cursor/hooks.json .devin/hooks.v1.json .gemini/settings.json .codex/hooks.json .github/hooks/ai-toolkit.json "$HOME/.augment/settings.json"; do
9095
- [ -f "$f" ] && python3 -c "import json; json.load(open('$f'))" && echo "OK: $f"
9096
- done
9144
+ # On a machine that has consumed URL rules/hooks at least once
9145
+ # (schema_version 1: entries live under the .rules / .hooks key):
9146
+ jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
9147
+ jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
9097
9148
  ```
9098
9149
 
9099
- **Verify:** each emitted file prints `OK: <path>`. Any `json.decoder.JSONDecodeError` means the merge logic corrupted the output.
9100
-
9101
- ### 9.6 Registry / generator drift check
9150
+ - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
9151
+ - [ ] If not, run `ai-toolkit update` to backfill missing hashes before tagging
9102
9152
 
9103
- `kb/reference/supported-tools-registry.md` should enumerate every per-editor `scripts/generate_*.py` we ship. Meta-generators (`generate_agents_md.py`, `generate_llms_txt.py`) are excluded — they produce docs/artifacts, not editor configs.
9153
+ **3. Audit SARIF output is well-formed:**
9104
9154
 
9105
9155
  ```bash
9106
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
9107
- REG=$(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u)
9108
- FS=$(ls scripts/generate_*.py | grep -vE "$META" | sort -u)
9109
- diff <(echo "$REG") <(echo "$FS") && echo "OK: registry matches filesystem" || echo "DRIFT: update supported-tools-registry.md"
9156
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json, sys; d=json.load(sys.stdin); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']; print('SARIF OK')"
9110
9157
  ```
9111
9158
 
9112
- **Verify:** prints `OK: registry matches filesystem`. If not, add the missing rows to the registry before tagging the next release.
9159
+ - [ ] Prints `SARIF OK`
9160
+ - [ ] If the script ever grows new rule classes, extend the SARIF `rules[]` coverage before releasing
9113
9161
 
9114
- ### 9.7 Claude Chat / Cowork plugin validates and exports
9162
+ **4. Strict-pin mode passes on CI** (optional, recommended for stable branches):
9115
9163
 
9116
9164
  ```bash
9117
- python3 scripts/claude_app.py verify
9118
- D=/tmp/aitk-claude-app-${RANDOM}
9119
- python3 scripts/claude_app.py export --output "$D.zip" --no-custom-rules
9120
- python3 -c "import zipfile; z=zipfile.ZipFile('$D.zip'); assert '.claude-plugin/plugin.json' in z.namelist(); assert 'claude-app/skills/ai-toolkit-rules/SKILL.md' in z.namelist(); print('OK: Claude app archive')"
9165
+ AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
9121
9166
  ```
9122
9167
 
9123
- **Verify:** the official validator exits 0; the archive contains the manifest,
9124
- app-native rules skill, bundled agents/skills, and plugin-relative Cowork hooks.
9168
+ - [ ] Exit 0, no `CHECKSUM CHANGED` line
9169
+ - [ ] Any unexpected upstream change blocks the release until explicitly approved
9125
9170
 
9126
9171
  ---
9127
9172
 
9128
- ## Phase 10: Explicit DSH Qualification (v4.30.0+)
9129
-
9130
- Run this phase whenever the release changes the `dsh` target, package pins, preset lifecycle, or DSH compatibility documentation. Use a new task-specific `DSH_HOME`; never replace `HOME` or reuse a regular profile.
9173
+ ## Phase 6: Commit
9131
9174
 
9132
- Prerequisites: DSH `0.1.1-rc.2`, pnpm `>=11.7.0,<12.0.0`, Codex logged in through ChatGPT, Claude Code logged in natively, and GitHub Copilot CLI logged in natively. Do not supply provider API keys.
9175
+ Stage all release files:
9133
9176
 
9134
9177
  ```bash
9135
- DSH_SMOKE_ROOT="$(mktemp -d)"
9136
- export DSH_HOME="$DSH_SMOKE_ROOT/dsh-home"
9137
-
9138
- ai-toolkit dsh install --profile web
9139
- ai-toolkit dsh doctor --profile web
9140
- dsh --profile web --host 127.0.0.1 --port 0 --no-open
9178
+ git add package.json manifest.json app/.claude-plugin/plugin.json
9179
+ git add package-lock.json
9180
+ git add CHANGELOG.md
9181
+ git add llms.txt llms-full.txt
9182
+ # NOTE: AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated
9183
+ # editor configs and are gitignored — do NOT commit them. `prepublishOnly` runs
9184
+ # `npm run generate:all`, so the shipped package (which lists AGENTS.md in
9185
+ # package.json `files`) gets a fresh copy at publish time.
9186
+ git add -p # review and stage any other changes
9141
9187
  ```
9142
9188
 
9143
- In a new `softspark-orchestrator` session, select the `codex` provider and run two standalone marker prompts:
9189
+ Commit:
9144
9190
 
9145
- 1. `subagent_claude_code` returns an exact child marker, then the Codex parent returns its exact completion marker.
9146
- 2. `subagent_gemini_copilot` returns an exact child marker, then the Codex parent returns its exact completion marker.
9191
+ ```bash
9192
+ git commit -m "chore: release vX.Y.Z"
9193
+ ```
9147
9194
 
9148
- Stop DSH, then remove only the managed profile artifacts:
9195
+ ---
9196
+
9197
+ ## Phase 7: Push Branch, Verify CI, Tag, and Push Tag
9149
9198
 
9150
9199
  ```bash
9151
- ai-toolkit dsh uninstall --profile web --yes
9152
- ```
9200
+ RELEASE_SHA=$(git rev-parse HEAD)
9201
+ git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
9153
9202
 
9154
- **Verify:** both tool results have `isError=false`, both turns end as `completed`, `doctor` reports no recovery requirement before uninstall, and an unrelated preset fixture remains unchanged. Preserve only redacted event sequence evidence; never attach credentials, auth files, or full private prompts.
9203
+ # Bind the gate to the exact release commit. GitHub run registration is
9204
+ # asynchronous, so retry at most five times and log every attempt.
9205
+ RUN_ID=""
9206
+ for ATTEMPT in 1 2 3 4 5; do
9207
+ RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
9208
+ --limit 1 --json databaseId --jq '.[0].databaseId')
9209
+ [ -n "$RUN_ID" ] && break
9210
+ echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
9211
+ [ "$ATTEMPT" -eq 5 ] || sleep 60
9212
+ done
9213
+ test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
9214
+ gh run watch "$RUN_ID" --exit-status \
9215
+ || { echo "FAIL: release commit CI failed"; exit 1; }
9155
9216
 
9156
- ---
9217
+ # publish.yml is Ubuntu-only. Require both full CI matrix jobs before creating
9218
+ # the tag so a green publish job cannot bypass a red macOS test.
9219
+ for OS in ubuntu-latest macos-latest; do
9220
+ gh run view "$RUN_ID" --json jobs \
9221
+ --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
9222
+ | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
9223
+ done
9157
9224
 
9158
- ## Troubleshooting
9225
+ # Refuse to tag if either local or remote main moved while CI was running.
9226
+ test "$(git rev-parse HEAD)" = "$RELEASE_SHA" \
9227
+ || { echo "FAIL: local HEAD changed during CI"; exit 1; }
9228
+ git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
9229
+ test "$(git rev-parse origin/main)" = "$RELEASE_SHA" \
9230
+ || { echo "FAIL: origin/main changed during CI"; exit 1; }
9159
9231
 
9160
- ### `ai-toolkit: command not found`
9232
+ git tag vX.Y.Z "$RELEASE_SHA"
9161
9233
 
9162
- ```bash
9163
- npm install -g @softspark/ai-toolkit
9164
- # or check PATH:
9165
- export PATH="$(npm config get prefix)/bin:$PATH"
9234
+ # Assert the tag before pushing it. Both checks are one line each and both
9235
+ # have caught a real broken release.
9236
+ test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" \
9237
+ || { echo "FAIL: tag is not on the tested release commit"; exit 1; }
9238
+ git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" \
9239
+ || { echo "FAIL: tag is not on the chore: release commit"; exit 1; }
9240
+
9241
+ # Push the single release tag by its full ref.
9242
+ git push origin refs/tags/vX.Y.Z
9166
9243
  ```
9167
9244
 
9168
- ### Doctor: broken symlinks
9245
+ **Why branch CI comes before the tag (v4.30.2 postmortem).** The publish
9246
+ workflow runs only on Ubuntu and can publish while the separate macOS matrix job
9247
+ is red. `v4.30.2` exposed this with a macOS-only DSH timing failure. A release
9248
+ commit must therefore pass the complete Ubuntu/macOS branch workflow before its
9249
+ tag exists; a successful publish workflow is not a substitute for green CI.
9169
9250
 
9170
- ```bash
9171
- ai-toolkit doctor --fix # auto-repair
9172
- ai-toolkit update # or full re-install
9173
- ```
9251
+ **Why the assertions (v4.19.0 postmortem).** v4.19.0 was tagged on a commit
9252
+ that contained only a KB document and still carried `package.json` version
9253
+ `4.18.0`; the actual release sat in the commit above it under a recycled
9254
+ `fix:` message. `publish.yml` fired, tried to publish a version already on
9255
+ npm, and failed. Nothing on npm, a tag pointing at the wrong tree, and the
9256
+ only way out was rewriting a pushed commit. Both assertions above catch this
9257
+ in under a second. Run them.
9174
9258
 
9175
- ### Tests fail: missing bats
9259
+ **Never `git push --tags`.** It pushes every local tag at once, and GitHub
9260
+ suppresses tag-triggered workflow runs when many tags arrive in a single push
9261
+ — the workflow silently does not fire and nothing publishes. Push the single
9262
+ release tag by its full ref, as above. (Sibling evidence: this is exactly how
9263
+ rag-mcp's `1.0.3` image build was skipped, when a `--tags` push carried 37
9264
+ tags at once.)
9176
9265
 
9177
- ```bash
9178
- brew install bats-core # macOS
9179
- npm install -g bats # cross-platform
9180
- ```
9266
+ This triggers `.github/workflows/publish.yml` which:
9267
+ 1. Runs `validate.py --strict`
9268
+ 2. Runs `npm test`
9269
+ 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z` with `--provenance` (SLSA v1 build attestation)
9181
9270
 
9182
- ### validate.py: stale counts
9271
+ **Provenance is non-negotiable.** If `id-token: write` permission or the `--provenance` flag is missing from `publish.yml`, fix it BEFORE tagging — an unsigned release is a regression against the v2.8.0 standard.
9183
9272
 
9184
- README badges don't match the current agents/skills/tests counts.
9185
- Update README.md and re-run.
9273
+ **After CI completes:** Run the [Release Verification SOP](sop-release-verification.md)
9274
+ to smoke-test the published package AND verify the provenance attestation landed on npm.
9186
9275
 
9187
- ### Eject: missing skills
9276
+ ---
9277
+
9278
+ ## Rollback
9279
+
9280
+ If a bad release was published:
9188
9281
 
9189
9282
  ```bash
9190
- ai-toolkit update # re-link missing symlinks
9191
- ai-toolkit eject /tmp/test # retry
9283
+ # Unpublish from npm (within 72h)
9284
+ npm unpublish @softspark/ai-toolkit@X.Y.Z
9285
+
9286
+ # Or deprecate (preferred — doesn't break existing installs)
9287
+ npm deprecate @softspark/ai-toolkit@X.Y.Z "Known issue: <description>. Use vA.B.C instead."
9288
+
9289
+ # Delete tag
9290
+ git tag -d vX.Y.Z
9291
+ git push origin --delete vX.Y.Z
9192
9292
  ```
9193
9293
 
9194
9294
  ---
9195
9295
 
9196
- ## Success Criteria
9296
+ ## Checklist Summary
9197
9297
 
9198
- | Area | Criterion |
9199
- |------|-----------|
9200
- | CLI | `--version` correct, `--help` full list, `status` current |
9201
- | Health | `doctor`: 0 errors, 0 warnings, PASSED |
9202
- | Install | `--dry-run` correct counts, `--local` all configs |
9203
- | Quality | `validate.py --strict`: PASSED |
9204
- | Security (baseline) | `audit_skills.py --ci`: 0 HIGH |
9205
- | Security (SARIF) | `audit_skills.py --sarif`: valid SARIF 2.1.0 with non-empty rules array |
9206
- | Security (permissions) | `audit_skills.py --permissions`: broad-access skills unchanged or justified in CHANGELOG |
9207
- | Supply chain | `dist.attestations.provenance.predicateType == https://slsa.dev/provenance/v1` on npm |
9208
- | Supply chain | All `sources.json` URL entries carry a `sha256`; `AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run` passes |
9209
- | Tests | `npm test`: N/N passed, 0 failures |
9210
- | Eject | Standalone `.claude/` with real files AND `output-styles/` directory |
9211
- | Guards | Destructive commands blocked |
9212
- | Deep coverage | `--profile full` emits every documented native surface, including Codex agents/skills/hooks and Copilot agents/skills/hooks |
9213
- | Profile contract | Codex skills emit at every profile without `--codex-skills`; Copilot standard emits agents, skills, instructions, prompts, and hooks |
9214
- | Idempotence | Second `install` run produces byte-identical output in every managed path |
9215
- | Live JSON | Every generated `.json` file on disk parses as valid JSON |
9216
- | Registry | `supported-tools-registry.md` enumerates every `scripts/generate_*.py` we ship |
9298
+ | # | Step | Command / Action | Pass Criteria |
9299
+ |---|------|-----------------|---------------|
9300
+ | 0a | Ecosystem drift check | `ecosystem_doctor.py --format text` | All tools Clean, or drift classified and resolved |
9301
+ | 0b | Ecosystem snapshot refresh | `ecosystem_doctor.py --update` | `benchmarks/ecosystem-doctor-snapshot.json` updated |
9302
+ | 0c | Ecosystem gate | `ecosystem_doctor.py --offline --check` | Exit 0 |
9303
+ | 1 | Version bump type | Decide patch/minor/major | — |
9304
+ | 2 | `package.json` version | Edit `"version"` | Matches target |
9305
+ | 3 | `manifest.json` version | Edit `"version"` | Matches target |
9306
+ | 4 | `plugin.json` version | Edit `"version"` | Matches target |
9307
+ | 5 | `package-lock.json` | `npm install --package-lock-only` | Matches target |
9308
+ | 6 | Count sync | Check `package.json` description, README | `validate.py` passes |
9309
+ | 7 | CHANGELOG.md | Add release entry (incl. `Ecosystem` subsection if any B/D/E/F drift) | Entry exists for vX.Y.Z |
9310
+ | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_llms_txt.py` | No unexpected diff |
9311
+ | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
9312
+ | 10 | Security audit (CI mode) | `audit_skills.py --ci` | 0 HIGH |
9313
+ | 11 | Security audit (SARIF) | `audit_skills.py --sarif` | Valid SARIF 2.1.0 JSON |
9314
+ | 12 | Per-skill permissions | `audit_skills.py --permissions` | New broad-access skills justified in CHANGELOG |
9315
+ | 13 | ShellCheck hooks | `shellcheck --severity=warning app/hooks/*.sh` | Exit 0, no output (mirrors ci.yml; publish.yml does NOT run it) |
9316
+ | 14 | Provenance flag check | `grep -- '--provenance' .github/workflows/publish.yml` | Present |
9317
+ | 15 | Checksum-pin backfill | `sources.json` entries all have `sha256` | No unpinned URL sources |
9318
+ | 15a | Licensing gate | `npx bats tests/test_licensing.bats` | 7/7 — SPDX headers, LICENSE, NOTICE, npm `files`, manifest consistency |
9319
+ | 16 | Tests | `git add -A kb/` if the KB changed, then `npm test` | All pass |
9320
+ | 17 | Commit | `git commit` | Clean working tree |
9321
+ | 18 | Push branch | `git push origin main` | Exact release commit is on `origin/main` |
9322
+ | 18a | Full branch CI | `gh run watch "$RUN_ID" --exit-status` plus matrix job assertions | Ubuntu and macOS Bats jobs both conclude `success` |
9323
+ | 19 | Tag | `git tag vX.Y.Z` | Tag exists only after green branch CI |
9324
+ | 19a | Tag is on tested SHA | `test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA"` | Exit 0 |
9325
+ | 19b | Tag is on the release commit | `git show --no-patch --format=%s vX.Y.Z` | Reads `chore: release vX.Y.Z` |
9326
+ | 20 | Push the single tag | `git push origin refs/tags/vX.Y.Z` | Publish CI triggered with `id-token: write`. Never `--tags`. |
9217
9327
 
9218
9328
  ---
9219
9329
 
@@ -9801,7 +9911,7 @@ Three tiers determine how to approach a task:
9801
9911
  |------|-------|-----------|-------|
9802
9912
  | Task | `disable-model-invocation: true` | User via `/skill` only | 32 |
9803
9913
  | Hybrid | (neither) | User via `/skill` + agent knowledge | 31 |
9804
- | Knowledge | `user-invocable: false` | Claude auto-loads | 46 |
9914
+ | Knowledge | `user-invocable: false` | Claude auto-loads | 51 |
9805
9915
 
9806
9916
  ## Multi-Agent Execution
9807
9917
 
@@ -10346,7 +10456,7 @@ category: reference
10346
10456
  service: ai-toolkit
10347
10457
  tags: [cli, commands, reference, install, update, plugin, mcp, telemetry]
10348
10458
  created: "2026-04-13"
10349
- last_updated: "2026-08-19"
10459
+ last_updated: "2026-09-04"
10350
10460
  description: "Complete CLI reference for all ai-toolkit commands, options, and flags."
10351
10461
  ---
10352
10462
 
@@ -10361,6 +10471,7 @@ Usage: ai-toolkit <command> [options]
10361
10471
  | Command | Description |
10362
10472
  |---------|-------------|
10363
10473
  | `install` | First-time global install into `~/.claude/` + Cursor, Windsurf, Gemini |
10474
+ | `install --language-skills detected\|all` | `detected` (default): turn off `<lang>-rules`/`<lang>-patterns` skills for languages no registered project uses, via `skillOverrides` in `~/.claude/settings.json` (entries are tracked in `state.json` and restored when a project brings the language back; a user's own override is never touched); `all`: keep every language skill on. The choice persists across `install`/`update` |
10364
10475
  | `install --local` | Claude Code configs only; add `--editors all` or `--editors cursor,aider` for other tools |
10365
10476
  | `update` | Re-apply toolkit after `npm install -g @softspark/ai-toolkit@latest` |
10366
10477
  | `update --local` | Re-apply + auto-detect editors from existing project files |
@@ -10368,7 +10479,7 @@ Usage: ai-toolkit <command> [options]
10368
10479
  | `status` | Show installed modules and version |
10369
10480
  | `uninstall` | Remove toolkit from `~/.claude/` |
10370
10481
  | `validate` | Verify toolkit integrity (`--strict` for CI-grade, warnings = errors) |
10371
- | `doctor` | Diagnose install health, hooks, quick-win assets, and artifact drift |
10482
+ | `doctor` | Diagnose install health, hooks, quick-win assets, artifact drift, context budget (est. resident tokens of the skill/agent listings and user rules, plus skills with zero recorded use; read-only, prints the `skillOverrides` key to paste), and permission rules (`permissions.allow` wildcards on interpreters, task runners, package installs, `gh api`, `curl`, `git fetch`, destructive commands; warns only, never edits) |
10372
10483
  | `doctor --fix` | Auto-repair broken symlinks, missing hooks, stale artifacts |
10373
10484
  | `eject [dir]` | Export standalone config (no symlinks, no toolkit dependency) |
10374
10485
  | `claude-app export [--output FILE] [--no-custom-rules] [--verify]` | Build an uploadable Claude Chat/Desktop/Cowork plugin ZIP and global-instructions file |
@@ -11738,7 +11849,7 @@ running the generators directly:
11738
11849
 
11739
11850
  - `scripts/generate_copilot.py` / `generate_copilot_hooks.py` contract tests
11740
11851
  (`tests/test_copilot.bats`, `tests/test_copilot_hooks.bats`).
11741
- - Release layout check in `kb/procedures/release-verification-sop.md` asserts the
11852
+ - Release layout check in `kb/procedures/sop-release-verification.md` asserts the
11742
11853
  `.github/{agents,skills,instructions,prompts,hooks}` surfaces.
11743
11854
  - `validate.py --strict` + `audit_skills.py --ci` in CI.
11744
11855
 
@@ -12673,9 +12784,9 @@ title: "Global Install Model"
12673
12784
  category: reference
12674
12785
  service: ai-toolkit
12675
12786
  tags: [install, global, claude, codex, plugins, local-setup]
12676
- version: "3.4.0"
12787
+ version: "3.5.0"
12677
12788
  created: "2026-03-26"
12678
- last_updated: "2026-08-19"
12789
+ last_updated: "2026-09-04"
12679
12790
  description: "Reference description of Claude Code global install, Claude app plugin export, project-local editor setup, global Codex plugin layering, and command responsibilities in ai-toolkit."
12680
12791
  ---
12681
12792
 
@@ -12687,6 +12798,19 @@ description: "Reference description of Claude Code global install, Claude app pl
12687
12798
 
12688
12799
  That means one machine-level install provides agents, skills, hooks, constitution, and rule files to every project without committing toolkit boilerplate into each repository.
12689
12800
 
12801
+ Language knowledge skills are the one part of the global install that is
12802
+ scoped by evidence (v4.32.0). Every `<lang>-rules` / `<lang>-patterns` skill is
12803
+ still symlinked, but `install` and `update` read the languages detected across
12804
+ the projects registered in `projects.json` and set `skillOverrides: off` in
12805
+ `~/.claude/settings.json` for the others, so their descriptions stop loading
12806
+ into every session. The entries the toolkit wrote are listed in `state.json`
12807
+ under `managed_skill_overrides`; only those are ever removed again (when a
12808
+ newly registered project brings the language back), a user's own override is
12809
+ left alone. With no registered project on disk nothing is disabled.
12810
+ `--language-skills all` restores everything and persists the choice
12811
+ (`language_skill_scope` in `state.json`); `ai-toolkit doctor` reports the
12812
+ resulting listing size and zero-use skills.
12813
+
12690
12814
  Other editor targets are opt-in and only use documented file surfaces. Cursor
12691
12815
  rules stay project-local because Cursor's global user rules are managed through
12692
12816
  the settings UI, not a stable merge-safe file. Codex supports both project and
@@ -12729,7 +12853,7 @@ The `--profile` flag controls how much of each editor's native surface is activa
12729
12853
  |---------|-----------|----------|
12730
12854
  | `minimal` | Smallest editor surface. Copilot still receives its root instructions, native agents, and self-contained skills; Codex still receives instructions, agents, skills, and native safety hooks. | You want the smallest supported footprint. |
12731
12855
  | `standard` (default) | Claude Code + editor rule files. Includes Gemini and Antigravity hooks plus native Copilot instructions, agents, portable skills, and hooks. | Day-to-day installs. Most users. |
12732
- | `strict` | Everything in `standard` plus git-hook wiring for commit-time safety checks. | Solo dev or tight team with zero tolerance for drift. |
12856
+ | `strict` | Everything in `standard` plus git-hook wiring for commit-time safety checks and the `git-team` common rule (feature branches, PR size, required approval, review SLA). | Tight team with zero tolerance for drift. A solo maintainer releasing straight to `main` wants `standard`, which keeps the solo-safe `git-workflow` core only. |
12733
12857
  | `full` | Every native surface across every editor: hooks, sub-agents, custom commands, skill pointers for Cursor / Windsurf / Gemini / Augment / Antigravity. | You want maximum coverage and understand that each editor will carry generated files under its own layout. |
12734
12858
 
12735
12859
  Codex installs materialize the full skill catalog under `.agents/skills/`
@@ -13118,9 +13242,9 @@ title: "Hooks Catalog"
13118
13242
  category: reference
13119
13243
  service: ai-toolkit
13120
13244
  tags: [hooks, quality, safety, enforcement, settings.json]
13121
- version: "1.11.0"
13245
+ version: "1.12.0"
13122
13246
  created: "2026-03-27"
13123
- last_updated: "2026-08-19"
13247
+ last_updated: "2026-09-04"
13124
13248
  description: "Complete reference of all ai-toolkit hooks: events, scripts, installation, and runtime behavior."
13125
13249
  ---
13126
13250
 
@@ -13305,6 +13429,7 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
13305
13429
  | Matcher | *(all)* |
13306
13430
  | Script | `~/.softspark/ai-toolkit/hooks/quality-check.sh` |
13307
13431
  | Fires | After every Claude response |
13432
+ | Mode | Background (`"async": true`). The linter (`npx tsc`, `ruff`) was the bulk of a 3.3 s median Stop chain; its output is advisory and reaches you when it finishes instead of holding the turn open |
13308
13433
 
13309
13434
  **Action:** Runs language-appropriate linter:
13310
13435
  - Python: `ruff check .`
@@ -13323,6 +13448,7 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
13323
13448
  | Matcher | *(all)* |
13324
13449
  | Script | `~/.softspark/ai-toolkit/hooks/save-session.sh` |
13325
13450
  | Fires | After every Claude response |
13451
+ | Mode | Background (`"async": true`): writes a file and exits 0, nothing to wait for |
13326
13452
 
13327
13453
  **Action:** Writes enriched session context to the per-repo session store (`~/.softspark/ai-toolkit/sessions/<repo-key>/session-context.md`) for cross-session persistence. Captures:
13328
13454
  - Session ID and last assistant message (first 5 lines)
@@ -13340,7 +13466,7 @@ Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
13340
13466
  | Script | `~/.softspark/ai-toolkit/hooks/quality-gate.sh` |
13341
13467
  | Fires | Before Claude is allowed to finish a response |
13342
13468
 
13343
- **Action:** Runs lint/typecheck. **Blocks stopping (exit 2)** if errors found, so Claude must continue and fix the issues. Missing local tooling is reported as skipped rather than blocking the session.
13469
+ **Action:** Runs lint/typecheck. **Blocks stopping (exit 2)** if errors found, so Claude must continue and fix the issues. Missing local tooling is reported as skipped rather than blocking the session. Ruff runs only when the project configured it (`ruff.toml`, `.ruff.toml`, or a `[tool.ruff]` table in `pyproject.toml`); a `pyproject.toml` that only carries build or pytest/mypy metadata does not trigger `ruff check .` under whatever configuration the machine resolves (v4.32.0).
13344
13470
 
13345
13471
  Skipped when `TOOLKIT_HOOK_PROFILE=minimal`.
13346
13472
 
@@ -13938,9 +14064,9 @@ title: "Language Rules System"
13938
14064
  category: reference
13939
14065
  service: ai-toolkit
13940
14066
  tags: [rules, languages, coding-style, testing, patterns, security]
13941
- version: "2.1.0"
14067
+ version: "2.2.0"
13942
14068
  created: "2026-04-07"
13943
- last_updated: "2026-06-30"
14069
+ last_updated: "2026-09-04"
13944
14070
  description: "Reference for the language-specific rules system: 13 per-language rule sets shipped as knowledge skills, plus common rules installed as Claude Code path-scoped project rules."
13945
14071
  ---
13946
14072
 
@@ -13990,7 +14116,7 @@ app/rules/
13990
14116
  └── medplum/
13991
14117
  ```
13992
14118
 
13993
- **Total: 13 per-language directories × 5 files + 1 common directory × 5 files + 3 standalone files** (see README.md for canonical count). Per-language directories ship as `<lang>-rules` knowledge skills; the common directory is installed as Claude Code `.claude/rules/ai-toolkit-*.md` files.
14119
+ **Total: 13 per-language directories × 5 files + 1 common directory × 6 files (5 in every profile + `git-team` in `strict`) + 3 standalone files** (see README.md for canonical count). Per-language directories ship as `<lang>-rules` knowledge skills; the common directory is installed as Claude Code `.claude/rules/ai-toolkit-*.md` files.
13994
14120
 
13995
14121
  ## Supported Languages
13996
14122
 
@@ -14080,13 +14206,16 @@ Common rules are installed as path-scoped Claude Code rule files:
14080
14206
  ```
14081
14207
  .claude/rules/
14082
14208
  ├── ai-toolkit-coding-style.md
14209
+ ├── ai-toolkit-git-team.md # --profile strict only
14083
14210
  ├── ai-toolkit-git-workflow.md
14084
14211
  ├── ai-toolkit-performance.md
14085
14212
  ├── ai-toolkit-security.md
14086
14213
  └── ai-toolkit-testing.md
14087
14214
  ```
14088
14215
 
14089
- Each file starts with:
14216
+ A source rule may also carry `profiles:` (same block-list form). `git-team` declares `profiles: ["strict"]`: branching, pull-request, and review conventions for teams, kept out of `standard` so a solo maintainer who releases straight to `main` is not told to open PRs against themselves. Rerunning `install --local` with a different profile adds or removes the managed file. Only the Claude Code local install honours `profiles`; the Claude app export, `compile-slm`, and editor `lang-common` bundles still receive every common rule (see `DECISIONS.md`, 2026-09-04).
14217
+
14218
+ Each file's `paths` frontmatter is copied from the source rule in `app/rules/common/<category>.md`. A source rule without a `paths` block is always-on:
14090
14219
 
14091
14220
  ```yaml
14092
14221
  ---
@@ -14095,6 +14224,18 @@ paths:
14095
14224
  ---
14096
14225
  ```
14097
14226
 
14227
+ A source rule with a `paths` block is path-scoped, so Claude Code loads it only when a matching file is touched. As of v4.32.0 `testing` (`**/*.test.*`, `**/*.spec.*`, `**/test_*`, `**/*_test.*`, `**/tests/**`) and `performance` (source-file extensions plus `**/*.sql`) are scoped; `coding-style`, `git-workflow`, and `security` stay always-on because they carry prohibitions that must hold in every session. To change a scope, edit the source frontmatter; `validate.py` rejects inline lists and unquoted globs because the installer reads only the block form:
14228
+
14229
+ ```yaml
14230
+ ---
14231
+ language: common
14232
+ category: testing
14233
+ version: "1.1.0"
14234
+ paths:
14235
+ - "**/tests/**"
14236
+ ---
14237
+ ```
14238
+
14098
14239
  The project `CLAUDE.md` receives only a compact index between a single named marker (the per-language markers from v1.x are no longer used):
14099
14240
 
14100
14241
  ```
@@ -14102,10 +14243,13 @@ The project `CLAUDE.md` receives only a compact index between a single named mar
14102
14243
  # Language Rules
14103
14244
 
14104
14245
  Common ai-toolkit rules live in `.claude/rules/ai-toolkit-*.md`
14105
- with Claude Code `paths` frontmatter so they load when project files
14106
- are opened instead of expanding this CLAUDE.md at session startup.
14246
+ with Claude Code `paths` frontmatter instead of expanding this
14247
+ CLAUDE.md. Always-on rules load in every session; path-scoped rules
14248
+ load only when a matching file is touched.
14107
14249
 
14108
- Common rule files: `.claude/rules/ai-toolkit-coding-style.md`, ...
14250
+ Always-on: `.claude/rules/ai-toolkit-coding-style.md`, ...
14251
+
14252
+ Path-scoped: `.claude/rules/ai-toolkit-performance.md`, ...
14109
14253
 
14110
14254
  Language-specific rules live in `<lang>-rules` knowledge skills (e.g.
14111
14255
  `python-rules`, `typescript-rules`) and load automatically when their
@@ -14142,7 +14286,7 @@ Language rules are tracked as modules in `manifest.json`:
14142
14286
 
14143
14287
  | Module | Description |
14144
14288
  |--------|-------------|
14145
- | `rules-common` | Common coding rules (5 files), included in `standard` profile |
14289
+ | `rules-common` | Common coding rules (6 files: 5 in every profile, `git-team` in `strict` only), included in `standard` profile |
14146
14290
  | `rules-typescript` | TypeScript-specific rules |
14147
14291
  | `rules-python` | Python-specific rules |
14148
14292
  | `rules-golang` | Go-specific rules |
@@ -14163,9 +14307,9 @@ Language rules are tracked as modules in `manifest.json`:
14163
14307
  |---|---|---|---|
14164
14308
  | Source | `app/rules/common/` | `app/rules/<lang>/` | `app/skills/<name>/SKILL.md` |
14165
14309
  | Delivery to Claude | Path-scoped `.claude/rules/ai-toolkit-*.md` files (`--local`) + compact `CLAUDE.md` index | Generated as `<lang>-rules` knowledge skills, loaded contextually | Loaded contextually by description match |
14166
- | Visibility | Loaded when project files are opened | Loaded when triggers match (file extensions, framework names) | Loaded when triggers match |
14310
+ | Visibility | Always-on (`coding-style`, `git-workflow`, `security`) or loaded when a matching file is touched (`testing`, `performance`) | Loaded when triggers match (file extensions, framework names) | Loaded when triggers match |
14167
14311
  | Scope | Language-agnostic standards (security, git, testing, perf, style) | Per-language coding-style, frameworks, patterns, security, testing | Domain skills (testing, debugging, RAG, etc.) |
14168
- | Install | `ai-toolkit install --local` | Global install (skills directory is symlinked) | Global install |
14312
+ | Install | `ai-toolkit install --local` | Global install (skills directory is symlinked); skills for languages no registered project uses are turned off via `skillOverrides` (`--language-skills detected`, the default) unless `--language-skills all` was chosen | Global install |
14169
14313
  | Other editors | Inlined into editor-specific rule files | Inlined into editor-specific rule files (still full content, not skills) | N/A |
14170
14314
 
14171
14315
  Per-language content delivered as a knowledge skill is the same Markdown that other editors receive inlined. The split exists only for Claude, where the Agent Skills progressive-disclosure mechanism keeps the system prompt small.
@@ -14313,7 +14457,7 @@ MIT remnant would need each contributor's agreement.
14313
14457
 
14314
14458
  It is a test rather than a checklist line on purpose. This project has two
14315
14459
  same-day postmortems about SOPs that existed and were skipped; CI does not skip.
14316
- [Release Preparation](../procedures/release-preparation-sop.md) Phase 5c runs the
14460
+ [Release Preparation](../procedures/sop-release.md) Phase 5c runs the
14317
14461
  same gate before tagging so a failure surfaces before the tag, not after.
14318
14462
 
14319
14463
  ## If the licence ever changes again
@@ -14326,7 +14470,7 @@ source, cross-verify against a second independent copy, and only then write
14326
14470
  ## Related
14327
14471
 
14328
14472
  - [`LICENSE`](../../LICENSE), [`NOTICE`](../../NOTICE)
14329
- - [Release Preparation SOP](../procedures/release-preparation-sop.md) — Phase 5c
14473
+ - [Release Preparation SOP](../procedures/sop-release.md) — Phase 5c
14330
14474
  - [Distribution Model](distribution-model.md) — what ships and where
14331
14475
 
14332
14476
  ---
@@ -16006,7 +16150,7 @@ Updated baseline docs:
16006
16150
  - `kb/reference/skills-catalog.md`
16007
16151
  - `kb/reference/plugin-pack-conventions.md`
16008
16152
  - `kb/reference/claude-ecosystem-benchmark-snapshot.md`
16009
- - `kb/procedures/maintenance-sop.md`
16153
+ - `kb/procedures/sop-maintenance.md`
16010
16154
 
16011
16155
  ## Validation Evidence
16012
16156
 
@@ -16860,13 +17004,13 @@ See [DSH Compatibility](./dsh-compatibility.md) for commands, topology, subscrip
16860
17004
  2. Delete its section from this document.
16861
17005
  3. Delete its snapshot entry from `benchmarks/ecosystem-doctor-snapshot.json` (or let the next `--update` prune it — currently not pruned automatically).
16862
17006
  4. Decide whether to keep the generator (`scripts/generate_<tool>_*.py`) for backwards compatibility or delete it.
16863
- 5. Remove references from `README.md`, `manifest.json` `description` field, and `kb/procedures/maintenance-sop.md` `Supported editors` line.
17007
+ 5. Remove references from `README.md`, `manifest.json` `description` field, and `kb/procedures/sop-maintenance.md` `Supported editors` line.
16864
17008
 
16865
17009
  ---
16866
17010
 
16867
17011
  ## Related
16868
17012
 
16869
- - [Ecosystem Sync SOP](../procedures/ecosystem-sync-sop.md) — how to use the doctor
17013
+ - [Ecosystem Sync SOP](../procedures/sop-ecosystem-sync.md) — how to use the doctor
16870
17014
  - [MCP Editor Compatibility](./mcp-editor-compatibility.md) — MCP-specific subset
16871
17015
  - `scripts/ecosystem_tools.json` — source of truth
16872
17016
  - `scripts/ecosystem_doctor.py` — drift detector
@@ -16963,7 +17107,7 @@ category: reference
16963
17107
  service: ai-toolkit
16964
17108
  tags: [features, differentiators, constitution, hooks, security, tdd, memory]
16965
17109
  created: "2026-04-13"
16966
- last_updated: "2026-08-06"
17110
+ last_updated: "2026-09-04"
16967
17111
  description: "Detailed description of ai-toolkit's unique features: constitution enforcement, hooks system, security scanning, effort budgeting, quality gates, and more."
16968
17112
  ---
16969
17113
 
@@ -17078,6 +17222,7 @@ added after the thing it checks had already shipped broken.
17078
17222
  | **Public surface** (`surface_manifest.py` + `app/surface.json`) | a skill, agent, CLI command, frontmatter field, hook, KB category or pack disappearing | `BACKWARD_COMPATIBILITY.md` listed the surfaces; renaming one still left every check green |
17079
17223
  | **Script invocation** (`validate.py`) | a skill running its own script by any path other than `${CLAUDE_SKILL_DIR}`, or through the wrong interpreter | nine skills shipped invocations that resolved only in the repo, never on an installed machine |
17080
17224
  | **Skill body budget** (`validate.py`) | a `SKILL.md` body over 20,000 bytes; warns over 18,000 | the body loads on every trigger match, including accidental ones; three skills sat at 24–35 KB |
17225
+ | **Skill description budget** (`validate.py`) | a description over 1,024 characters (the Agent Skills cap; the runtime truncates past it), or an unquoted description containing `: ` or ` #` (strict YAML drops every field, `allowed-tools` included); warns over 400; applied to plugin-pack skills too | two installed skills sat at 791 and 883 characters and made up 8% of the model's skill listing on their own |
17081
17226
  | **Split integrity** (`check_split.py`) | a body → `reference/` refactor that lost a fenced code line, a required section, the description, or a working link | run by hand during a split; caught a heading corrupted inside a fenced example on first use |
17082
17227
 
17083
17228
  The surface check is deliberately one-directional: **removals fail, additions pass.**
@@ -17399,7 +17544,7 @@ Expect `Total plugin skills loaded: 0`, `Total plugin agents loaded: 0`, and the
17399
17544
  ## Related
17400
17545
 
17401
17546
  - `kb/reference/global-install-model.md`
17402
- - `kb/procedures/maintenance-sop.md`
17547
+ - `kb/procedures/sop-maintenance.md`
17403
17548
 
17404
17549
  ---
17405
17550