@softspark/ai-toolkit 4.30.2 → 4.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/CHANGELOG.md +57 -0
  2. package/README.md +27 -28
  3. package/app/.claude-plugin/plugin.json +1 -1
  4. package/app/agents/code-reviewer.md +14 -0
  5. package/app/agents/frontend-specialist.md +27 -24
  6. package/app/agents/infrastructure-validator.md +2 -2
  7. package/app/claude-app/skills/ai-toolkit-rules/SKILL.md +44 -1
  8. package/app/personas/frontend-lead.md +20 -12
  9. package/app/rules/claude-toolkit-rules.md +4 -1
  10. package/app/rules/edit-discipline.md +37 -0
  11. package/app/skills/app-builder/SKILL.md +37 -6
  12. package/app/skills/claude-toolkit-rules/SKILL.md +37 -0
  13. package/app/skills/design-engineering/SKILL.md +87 -102
  14. package/app/skills/design-engineering/reference/animation-recipes.md +67 -0
  15. package/app/skills/design-engineering/reference/component-craft.md +28 -0
  16. package/app/skills/documentation-standards/SKILL.md +1 -1
  17. package/app/skills/edit-discipline/SKILL.md +50 -0
  18. package/app/skills/git-conventions/SKILL.md +18 -0
  19. package/app/skills/output-mode/SKILL.md +41 -0
  20. package/app/skills/quality-gates/SKILL.md +23 -0
  21. package/app/skills/review/SKILL.md +10 -0
  22. package/kb/procedures/{ecosystem-sync-sop.md → sop-ecosystem-sync.md} +2 -2
  23. package/kb/procedures/{post-release-testing-sop.md → sop-post-release-testing.md} +3 -3
  24. package/kb/procedures/sop-pre-commit.md +96 -0
  25. package/kb/procedures/{release-preparation-sop.md → sop-release.md} +87 -26
  26. package/kb/reference/architecture-overview.md +1 -1
  27. package/kb/reference/copilot-compatibility.md +1 -1
  28. package/kb/reference/licensing.md +2 -2
  29. package/kb/reference/quick-wins-implementation-summary.md +1 -1
  30. package/kb/reference/supported-tools-registry.md +2 -2
  31. package/kb/troubleshooting/plugin-double-load.md +1 -1
  32. package/llms-full.txt +974 -806
  33. package/llms.txt +6 -5
  34. package/manifest.json +1 -1
  35. package/package.json +2 -1
  36. package/scripts/doctor.py +1 -1
  37. package/scripts/generate_toolkit_rules_skills.py +189 -0
  38. package/scripts/validate.py +1 -1
  39. /package/kb/procedures/{maintenance-sop.md → sop-maintenance.md} +0 -0
  40. /package/kb/procedures/{release-verification-sop.md → sop-release-verification.md} +0 -0
package/llms-full.txt CHANGED
@@ -32,11 +32,12 @@
32
32
  - [Closed: Tool-Output Token Reduction](kb/history/completed/tool-output-token-reduction-closed-20260727.md)
33
33
  - [How-To Guides](kb/howto/README.md)
34
34
  - [Plan: Cloud Security Pack — Multi-Cloud Audit](kb/planning/cloud-security-pack-plan.md)
35
- - [SOP: Ecosystem Sync](kb/procedures/ecosystem-sync-sop.md)
36
- - [SOP: AI Toolkit Maintenance](kb/procedures/maintenance-sop.md)
37
- - [SOP: Post-Release Testing](kb/procedures/post-release-testing-sop.md)
38
- - [SOP: Release Preparation](kb/procedures/release-preparation-sop.md)
39
- - [SOP: Release Verification](kb/procedures/release-verification-sop.md)
35
+ - [SOP: Ecosystem Sync](kb/procedures/sop-ecosystem-sync.md)
36
+ - [SOP: AI Toolkit Maintenance](kb/procedures/sop-maintenance.md)
37
+ - [SOP: Post-Release Testing](kb/procedures/sop-post-release-testing.md)
38
+ - [SOP: Pre-Commit Gate](kb/procedures/sop-pre-commit.md)
39
+ - [SOP: Release Verification](kb/procedures/sop-release-verification.md)
40
+ - [SOP: Release Preparation](kb/procedures/sop-release.md)
40
41
  - [Agents Catalog](kb/reference/agents-catalog.md)
41
42
  - [Anti-Pattern Registry Format](kb/reference/anti-pattern-registry-format.md)
42
43
  - [AI Toolkit Architecture](kb/reference/architecture-overview.md)
@@ -95,6 +96,7 @@
95
96
  - **chaos**: Injects controlled faults for resilience testing on non-prod. Triggers: chaos, fault injection, latency injection, dependency kill, resilience test.
96
97
  - **ci**: Detect/generate/debug CI pipeline config (GitHub Actions, GitLab CI). Triggers: CI setup, build pipeline, GitHub Actions config, debug CI, GitLab CI.
97
98
  - **ci-cd-patterns**: CI/CD: GitHub Actions, GitLab CI, Jenkins, caching, blue-green, canary. Triggers: CI, CD, pipeline, GitHub Actions, workflow YAML, release, canary, rollout.
99
+ - **claude-toolkit-rules**: SoftSpark working agreement: never guess a home directory path, give at least three alternatives, and apply a devil's advocate critique to decisions. Triggers: toolkit, conventions, workflow, alternatives, review.
98
100
  - **clean-code**: Code quality: meaningful names, SRP, DRY, small functions, guard clauses, refactoring. Triggers: clean code, naming, code smell, SRP, DRY, long function, god class, dead code.
99
101
  - **command-creator**: Creates new Claude Code slash commands with frontmatter and validation. Triggers: new slash command, create command, command scaffold.
100
102
  - **commit**: Creates Conventional Commits with pre-commit validation. Triggers: commit, conventional commit, git commit, message.
@@ -115,12 +117,14 @@
115
117
  - **docs**: Generates/updates README, API docs, architecture notes. Triggers: docs, README, API docs, architecture note, documentation.
116
118
  - **documentation-standards**: KB conventions: YAML frontmatter, 10-category taxonomy (reference/howto/procedures/troubleshooting/best-practices/decisions/runbooks/planning/business/templates). Triggers: kb/, SOP, runbook, howto, frontmatter, knowledge base.
117
119
  - **ecommerce-patterns**: E-commerce: cart, checkout, payments (Stripe/Adyen), order state, inventory, promos, tax. Triggers: cart, checkout, SKU, payment, Stripe, Shopify, Medusa, Magento, coupon, refund.
120
+ - **edit-discipline**: Change files with the edit and write tools, never by rewriting them through bash (sed, awk, tee, heredoc, redirection), and show `git diff` before reporting a file-changing task as done. Triggers: edit, write, modify, refactor, patch, fix, diff, review changes.
118
121
  - **evaluate**: Evaluates RAG retrieval and LLM-as-judge metrics (faithfulness, relevancy, context precision). Triggers: measure RAG quality, knowledge gap, RAG eval, golden dataset.
119
122
  - **evolve**: Analyzes agent/skill failures, drafts prompt/permission fixes. Triggers: improve agent, refine skill, system prompt, optimize agent.
120
123
  - **explain**: Explains code/architecture with Mermaid diagrams and sequence flows. Triggers: what does X do, how does Y work, explain code, sequence diagram.
121
124
  - **explore**: Explores codebase structure, stack, and architecture. Triggers: explore codebase, project structure, stack overview, architecture map.
122
125
  - **fix**: Applies targeted fix to known bug/lint error, verifies with same command that surfaced it. Triggers: fix, apply fix, fix bug, fix lint, targeted fix.
123
126
  - **flutter-patterns**: Flutter/Dart: widgets, state mgmt (Riverpod/Bloc), navigation, platform channels. Triggers: Flutter, Dart, widget, Riverpod, Bloc, pubspec, hot reload.
127
+ - **git-conventions**: Conventional Commits only (feat, fix, docs, refactor, test, chore); no AI co-authorship trailer and no AI signature in a commit message. Triggers: commit, git, message, changelog, release, pull request.
124
128
  - **git-mastery**: Advanced Git: rebase, bisect, reflog, cherry-pick, worktrees, LFS. Triggers: rebase, bisect, cherry-pick, reflog, force push, merge conflict, worktree.
125
129
  - **golang-rules**: Go coding rules: style, patterns, security, testing. Triggers: .go, go.mod, go.sum, Gin, Echo, Gorilla, testing, gofmt.
126
130
  - **grill-me**: Stress-tests a plan via Socratic questioning down each decision branch. Triggers: stress-test, grill me, validate assumptions, challenge plan, socratic review.
@@ -147,6 +151,7 @@
147
151
  - **observability-patterns**: Observability: structured logs, metrics (RED/USE), tracing, SLO/SLI. Triggers: logging, metrics, Prometheus, Grafana, OpenTelemetry, trace, monitoring.
148
152
  - **onboard**: Sets up ai-toolkit in a project: symlinks, CLAUDE.md, intent interview. Triggers: onboard, setup project, install ai-toolkit, migrate project.
149
153
  - **orchestrate**: Coordinates multiple specialized agents in parallel. Triggers: orchestrate, multi-agent, parallel agents, coordinate agents.
154
+ - **output-mode**: Answer concisely: lead with the result, no preamble, tables over prose, no trailing restatement of a diff the reader can already see. Triggers: response style, verbosity, summary, explanation, report.
150
155
  - **panic**: Emergency kill switch — halts all agents via lockfile gate. Triggers: panic, stop everything, kill switch, halt agents, agents looping.
151
156
  - **performance-profiling**: Performance: golden signals, p50/p95/p99, flame graphs, load testing. Triggers: performance, slow, latency, p99, flame graph, bottleneck, memory leak.
152
157
  - **persona**: Switches engineering persona at runtime. Triggers: persona, switch role, backend-lead, frontend-lead, devops-eng, junior-dev.
@@ -160,6 +165,7 @@
160
165
  - **prompt-caching-patterns**: Anthropic API prompt caching: TTL, breakpoints, stacking, invalidation, hit rate. Triggers: prompt caching, cache_control, cache breakpoint, cache TTL, hit rate.
161
166
  - **python-rules**: Python coding rules: style, patterns, security, testing. Triggers: .py, .pyi, pyproject.toml, requirements.txt, Pipfile, FastAPI, Django, Flask, pytest, SQLAlchemy, ruff, mypy.
162
167
  - **qa-session**: Interactive QA: user reports bugs conversationally, agent files GitHub issues. Triggers: QA session, report bug, file issue, conversational QA, bug intake.
168
+ - **quality-gates**: Plan before work over an hour, and hold the gates: ruff clean, mypy --strict clean, pytest coverage above 70 percent, no secrets in code. Triggers: quality, lint, mypy, pytest, coverage, gate, definition of done.
163
169
  - **rag-patterns**: RAG: embeddings, chunking, hybrid search (BM25+vector), reranking, CRAG, multi-hop. Triggers: RAG, embedding, pgvector, Qdrant, Pinecone, Weaviate, reranker, semantic search.
164
170
  - **refactor**: Refactors code for quality and maintainability. Triggers: refactor, clean up, restructure, improve code, modernize.
165
171
  - **refactor-plan**: Creates detailed refactor plan with tiny commits via interview, files as GitHub RFC. Triggers: refactor plan, refactoring RFC, incremental refactor, safe steps.
@@ -6929,7 +6935,7 @@ Examples:
6929
6935
 
6930
6936
  ---
6931
6937
 
6932
- ## kb/procedures/ecosystem-sync-sop.md
6938
+ ## kb/procedures/sop-ecosystem-sync.md
6933
6939
 
6934
6940
  ---
6935
6941
  title: "SOP: Ecosystem Sync"
@@ -7186,12 +7192,12 @@ jobs:
7186
7192
 
7187
7193
  - [Supported Tools Registry](../reference/supported-tools-registry.md) — human-readable per-tool breakdown
7188
7194
  - [MCP Editor Compatibility](../reference/mcp-editor-compatibility.md) — MCP-specific adapter table
7189
- - [Maintenance SOP](maintenance-sop.md) — general toolkit upkeep
7190
- - [Release Preparation SOP](release-preparation-sop.md) — run the doctor before tagging
7195
+ - [Maintenance SOP](sop-maintenance.md) — general toolkit upkeep
7196
+ - [Release Preparation SOP](sop-release.md) — run the doctor before tagging
7191
7197
 
7192
7198
  ---
7193
7199
 
7194
- ## kb/procedures/maintenance-sop.md
7200
+ ## kb/procedures/sop-maintenance.md
7195
7201
 
7196
7202
  ---
7197
7203
  title: "SOP: AI Toolkit Maintenance"
@@ -7705,7 +7711,7 @@ What `uninstall` does:
7705
7711
 
7706
7712
  ---
7707
7713
 
7708
- ## kb/procedures/post-release-testing-sop.md
7714
+ ## kb/procedures/sop-post-release-testing.md
7709
7715
 
7710
7716
  ---
7711
7717
  title: "SOP: Post-Release Testing"
@@ -7725,7 +7731,7 @@ actually install, from npm, rather than the working tree.
7725
7731
 
7726
7732
  Sibling procedures exist for `jira-mcp` and `legal-pl-pack`; this is the
7727
7733
  ai-toolkit equivalent. It complements
7728
- [Release Verification](release-verification-sop.md), which checks the toolkit
7734
+ [Release Verification](sop-release-verification.md), which checks the toolkit
7729
7735
  from the maintainer's own installed copy. The difference that matters: this one
7730
7736
  never writes to the maintainer's `~/.claude` or `~/.softspark`.
7731
7737
 
@@ -8019,1140 +8025,1302 @@ print(f'removed {sb} ({n} files)')
8019
8025
 
8020
8026
  ## Related
8021
8027
 
8022
- - [Release Preparation](release-preparation-sop.md) — run before tagging
8023
- - [Release Verification](release-verification-sop.md) — the maintainer-install checks
8028
+ - [Release Preparation](sop-release.md) — run before tagging
8029
+ - [Release Verification](sop-release-verification.md) — the maintainer-install checks
8024
8030
  - [rtk-pack Retirement](../history/completed/rtk-pack-retirement-20260727.md) — what happened the one time this SOP was written and not run
8025
8031
 
8026
8032
  ---
8027
8033
 
8028
- ## kb/procedures/release-preparation-sop.md
8034
+ ## kb/procedures/sop-pre-commit.md
8029
8035
 
8030
8036
  ---
8031
- title: "SOP: Release Preparation"
8037
+ title: "SOP: Pre-Commit Gate"
8032
8038
  category: procedures
8033
8039
  service: ai-toolkit
8034
- tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
8035
- version: "1.14.0"
8036
- created: "2026-04-10"
8037
- last_updated: "2026-08-06"
8038
- description: "Step-by-step checklist for preparing a new ai-toolkit release — ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, and checksum-pin checks added in v2.8.0, the single-run npm test discipline added in v1.8.0, the ecosystem-sync gate added in v1.9.0, the registry-vs-generators drift gate added in v1.10.0, the mandatory pre-tag ShellCheck gate added in v1.11.0 (publish.yml does not run ShellCheck, so a hook lint failure can publish while reddening main CI — see the v4.5.1 postmortem in Phase 5), the pre-push tag assertions added in v1.12.0 after v4.19.0 was tagged on the wrong commit (Phase 7), and the licensing gate added in v1.13.0 with the move to Apache-2.0 (Phase 5c)."
8040
+ tags: [pre-commit, quality-gate, shellcheck, bats, validation, secrets]
8041
+ version: "1.0.0"
8042
+ created: "2026-09-02"
8043
+ last_updated: "2026-09-02"
8044
+ description: "The local gate to run before every commit to ai-toolkit: staged-file and secret scan, ShellCheck, Python syntax, toolkit validation, and the Bats suite. Runs the same checks CI runs, so main does not go red after a push."
8039
8045
  ---
8040
8046
 
8041
- # SOP: Release Preparation
8047
+ # SOP: Pre-Commit Gate
8042
8048
 
8043
- Complete checklist for preparing a new `@softspark/ai-toolkit` release.
8044
- Run this **before** tagging. After tagging and publishing, run the
8045
- [Release Verification SOP](release-verification-sop.md) to smoke-test.
8049
+ ## Why this exists
8046
8050
 
8047
- **Pipeline:**
8048
- ```
8049
- Ecosystem Sync SOP (drift check + generator updates)
8050
-
8051
- Release Preparation (this SOP)
8052
-
8053
- git tag → CI publish → Release Verification SOP
8054
- ```
8051
+ CI runs on `main` and on pull requests, but it runs *after* the push. Every
8052
+ check below already exists in `.github/workflows/ci.yml`; running them locally
8053
+ first is what keeps `main` green and keeps a release from becoming a debug
8054
+ session. The v4.5.1 postmortem in `sop-release.md` is the case
8055
+ where a hook lint failure published while reddening `main` CI.
8055
8056
 
8056
- **Time:** 10-20 minutes (includes ecosystem sync review)
8057
+ This gate is not a substitute for `sop-release.md`. That SOP gates
8058
+ a *tag*; this one gates a *commit*.
8057
8059
 
8058
- ---
8060
+ ## Prerequisites
8059
8061
 
8060
- ## Quick Checklist (TL;DR)
8062
+ `shellcheck`, `python3` and `bats` must be on `PATH`. The suite is the one
8063
+ check that silently does nothing when its runner is absent, so install it
8064
+ before relying on this gate:
8061
8065
 
8062
8066
  ```bash
8063
- # 0. Ecosystem sync (mandatory for minor/major releases; optional for patch)
8064
- # Full procedure: kb/procedures/ecosystem-sync-sop.md
8065
- python3 scripts/ecosystem_doctor.py --format text > /tmp/eco-report.txt
8066
- cat /tmp/eco-report.txt
8067
- # If drift detected: stop here, follow ecosystem-sync-sop.md Phase 2-4 to
8068
- # classify each drift (A-F), update generators as needed, refresh snapshot,
8069
- # THEN resume this SOP.
8070
- python3 scripts/ecosystem_doctor.py --update # after all drift resolved
8071
-
8072
- # 1. Decide version bump
8073
- # patch (1.4.2 → 1.4.3): bugfix, typo, doc fix
8074
- # minor (1.4.2 → 1.5.0): new feature, new skill, new flag, any ecosystem-class-B/F change
8075
- # major (1.4.2 → 2.0.0): breaking change, any ecosystem-class-D removed path
8076
-
8077
- # 2. Sync version across all files
8078
- python3 scripts/sync_version.py X.Y.Z # if script exists, else manual
8079
-
8080
- # 3. Write CHANGELOG.md entry
8081
- # 4. Regenerate artifacts
8082
- python3 scripts/generate_agents_md.py > AGENTS.md
8083
- python3 scripts/generate_llms_txt.py > llms.txt
8084
- python3 scripts/generate_llms_txt.py --full > llms-full.txt
8085
-
8086
- # 5. Validate + audit + SARIF + shellcheck + test + ecosystem check
8087
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif && shellcheck --severity=warning app/hooks/*.sh && npm test
8088
-
8089
- # 5a. Supply-chain standard (v2.8.0+) — non-negotiable
8090
- grep -q -- '--provenance' .github/workflows/publish.yml || { echo "MISSING --provenance"; exit 1; }
8091
- grep -q 'id-token: write' .github/workflows/publish.yml || { echo "MISSING id-token: write"; exit 1; }
8092
- python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8093
-
8094
- # 5b. Ecosystem gate — snapshot must be current before tag
8095
- python3 scripts/ecosystem_doctor.py --offline --check || { echo "STALE ecosystem snapshot — re-run doctor"; exit 1; }
8096
-
8097
- # 5c. Licensing gate — SPDX headers, LICENSE, NOTICE, manifest consistency
8098
- npx bats tests/test_licensing.bats || { echo "LICENSING GATE FAILED"; exit 1; }
8099
-
8100
- # 6. Commit + tag + push
8101
- git add -A && git commit -m "chore: release vX.Y.Z"
8102
- git tag vX.Y.Z
8103
-
8104
- # 6a. Assert the tag before pushing it (v4.19.0 postmortem, Phase 7)
8105
- test "$(git rev-parse vX.Y.Z)" = "$(git rev-parse HEAD)" || { echo "FAIL: tag not on HEAD"; exit 1; }
8106
- git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" || { echo "FAIL: tag not on release commit"; exit 1; }
8107
-
8108
- # 6b. Branch first, then the single tag by full ref. Never --tags.
8109
- git push origin main
8110
- git push origin refs/tags/vX.Y.Z
8067
+ brew install bats-core shellcheck # macOS
8068
+ sudo apt-get install -y bats shellcheck # Debian / Ubuntu
8111
8069
  ```
8112
8070
 
8113
- ---
8071
+ ## The gate
8114
8072
 
8115
- ## Phase 0: Ecosystem Sync (MANDATORY for minor/major)
8073
+ Run all five from the repository root. Each mirrors a CI job.
8116
8074
 
8117
- Before touching version numbers, confirm the toolkit is aligned with the current state of every editor / platform it integrates with. Skipping this phase ships a release whose generators may lag a month-old CLI refactor, a rename of `.cursorrules` to `.cursor/rules/`, or a new hook event we do not yet emit.
8075
+ ```bash
8076
+ # 1. Staged files: secrets, large blobs, commit-type suggestion
8077
+ python3 app/skills/commit/scripts/pre-commit-check.py
8118
8078
 
8119
- **When this phase is mandatory:**
8120
- - Minor release (X.Y.0) — always
8121
- - Major release (X.0.0) — always
8122
- - Patch release (X.Y.Z) — only if the patch touches a generator or install flow
8079
+ # 2. ShellCheck hooks -- the CI job that has published while red before
8080
+ shellcheck --severity=warning app/hooks/*.sh app/plugins/*/hooks/*.sh
8123
8081
 
8124
- **When to skip:** pure doc-only patches, SOP edits, internal refactors that do not touch `scripts/generate_*` or `app/skills/*/SKILL.md`.
8082
+ # 3. Python syntax across every script the toolkit ships
8083
+ python3 -m py_compile scripts/*.py app/skills/*/scripts/*.py
8125
8084
 
8126
- ### 0.1 Run the doctor
8085
+ # 4. Toolkit integrity: agents, skills, registry drift, content quality
8086
+ npm run validate
8127
8087
 
8128
- ```bash
8129
- python3 scripts/ecosystem_doctor.py --format text | tee /tmp/eco-report.txt
8088
+ # 5. The suite. Run it ONCE -- see the single-run discipline below
8089
+ npm test
8130
8090
  ```
8131
8091
 
8132
- Output classifies every registered tool as **Clean**, **Drift**, or **Errored**.
8092
+ A non-zero exit from any of them is a stop, not a warning to note and push past.
8133
8093
 
8134
- ### 0.2 Act on drift
8094
+ ## Single-run discipline
8135
8095
 
8136
- For each drifting tool, follow [ecosystem-sync-sop.md](ecosystem-sync-sop.md) Phase 2-4:
8096
+ `npm test` runs the Bats suite with `--jobs 4`. Running it repeatedly to see
8097
+ whether a failure is "flaky" hides real ordering bugs and wastes minutes. Run
8098
+ it once. If it fails, read the failure and fix the cause; if the same test
8099
+ passes on a re-run without a code change, that instability is itself the bug
8100
+ and belongs in an issue, not in a retry.
8137
8101
 
8138
- | Drift class | Release impact |
8139
- |-------------|----------------|
8140
- | A (cosmetic reword) | No version impact — refresh snapshot, continue |
8141
- | B (new feature — integrate) | **Minor** version bump at minimum; new generator or extended generator |
8142
- | C (new feature — not adopted) | No impact — note in registry |
8143
- | D (deprecation) | **Minor** or **major** depending on user impact; add migration warning |
8144
- | E (feature promoted to default) | **Minor**; simplify generator, keep fallback comment |
8145
- | F (feature newly globally available) | **Minor**; may require new generator or new config path |
8102
+ ## What this gate does not cover
8146
8103
 
8147
- If any B/D/E/F changes land in this preparation pass, mention them explicitly in the CHANGELOG entry (Phase 3) under a `Ecosystem` subsection.
8104
+ - **Cross-platform.** The suite here runs on your machine only. Both
8105
+ `ubuntu-latest` and `macos-latest` must be green before a tag; that gate
8106
+ lives in `sop-release.md`, Phase 7, and is bound to the exact
8107
+ release commit.
8108
+ - **Required files.** The `required-files` CI job checks that every file the
8109
+ module template mandates is present and that the version is in sync across
8110
+ all four manifests. It is cheap and runs on every push.
8111
+ - **Provenance and SARIF.** Release-time concerns; see
8112
+ `sop-release.md`.
8148
8113
 
8149
- ### 0.3 Refresh snapshot
8114
+ ## Commit message
8150
8115
 
8151
- Once every drift is resolved (either by code change or by re-classifying as acceptable):
8116
+ Conventional Commits, and no AI co-authorship trailer:
8152
8117
 
8153
- ```bash
8154
- python3 scripts/ecosystem_doctor.py --update
8118
+ ```
8119
+ feat(scope): summary in the imperative
8120
+ fix(scope): summary in the imperative
8121
+ docs|refactor|test|chore(scope): ...
8155
8122
  ```
8156
8123
 
8157
- This writes the new baseline to `benchmarks/ecosystem-doctor-snapshot.json`. Commit it as part of the release commit.
8124
+ `pre-commit-check.py` suggests a type from the staged paths. It is a
8125
+ suggestion; the scope and the summary are yours.
8158
8126
 
8159
- ### 0.4 Gate
8127
+ ## Related
8160
8128
 
8161
- ```bash
8162
- python3 scripts/ecosystem_doctor.py --offline --check
8163
- ```
8129
+ - `sop-release.md` -- the pre-tag gate, including cross-platform CI
8130
+ - `sop-post-release-testing.md` -- what to verify after a release ships
8131
+ - `sop-release-verification.md` -- end-to-end smoke test of an installed build
8164
8132
 
8165
- Must exit `0`. If it exits `1`, the snapshot is stale — rerun Phase 0.3 or review the remaining drift.
8133
+ ---
8166
8134
 
8135
+ ## kb/procedures/sop-release-verification.md
8136
+
8137
+ ---
8138
+ title: "SOP: Release Verification"
8139
+ category: procedures
8140
+ service: ai-toolkit
8141
+ tags: [sop, verification, release, smoke-test, install, update, qa, provenance, sarif, dsh]
8142
+ version: "1.8.0"
8143
+ created: "2026-04-08"
8144
+ last_updated: "2026-09-01"
8145
+ description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit. Verifies CLI, native Codex and GitHub Copilot surfaces, explicit DSH lifecycle, Claude app export, doctor, validation, tests, eject, provenance, SARIF, and per-skill permissions."
8167
8146
  ---
8168
8147
 
8169
- ## Phase 1: Determine Version Bump
8148
+ # SOP: Release Verification
8170
8149
 
8171
- Follow [Semantic Versioning](https://semver.org/):
8150
+ End-to-end smoke test after installing or updating `@softspark/ai-toolkit`.
8151
+ Verifies all critical paths from the user's perspective.
8172
8152
 
8173
- | Change Type | Bump | Examples |
8174
- |-------------|------|---------|
8175
- | Bugfix, typo, doc-only | **patch** | Fix install flag, correct description |
8176
- | New feature, skill, agent, flag | **minor** | Add `/hipaa-validate`, add `--output json` |
8177
- | Breaking CLI change, removed skill, config format change | **major** | Rename `install` to `setup`, remove skill |
8153
+ **Use this SOP when:**
8154
+ - After `npm install -g @softspark/ai-toolkit@latest`
8155
+ - After `ai-toolkit update`
8156
+ - Before tagging a new version (`git tag`)
8157
+ - Before publishing to npm (`npm publish`)
8158
+ - As a smoke test in CI/CD
8178
8159
 
8179
- **Rule:** When in doubt, bump minor.
8160
+ **Prerequisites:**
8161
+ - Node.js >= 18, Python 3, `bats`, git
8162
+ - `@softspark/ai-toolkit` installed globally
8163
+
8164
+ **Time:** 10-15 minutes (full), 2 minutes (quick checklist)
8180
8165
 
8181
8166
  ---
8182
8167
 
8183
- ## Phase 2: Sync Version in All Files
8168
+ ## Quick Checklist (TL;DR)
8184
8169
 
8185
- The canonical version lives in `package.json`. These files **must** match:
8170
+ The 14 core commands below must pass. Releases that change DSH must also complete Phase 10.
8186
8171
 
8187
- ### Mandatory sync (every release)
8172
+ ```bash
8173
+ # Pre-commit (Phase 0)
8174
+ python3 scripts/generate_agents_md.py > AGENTS.md # 1. Regenerate AGENTS.md
8175
+ python3 scripts/generate_llms_txt.py > llms.txt # 2. Regenerate llms.txt
8176
+ python3 scripts/validate.py --strict # 3. Validation passed?
8177
+ npm test > /tmp/npm-test.log 2>&1 && grep -c '^ok ' /tmp/npm-test.log && ! grep -q '^not ok' /tmp/npm-test.log # 4. All tests passed? (single run, cached)
8188
8178
 
8189
- | File | Field | How to update |
8190
- |------|-------|---------------|
8191
- | `package.json` | `"version": "X.Y.Z"` | Edit directly |
8192
- | `manifest.json` | `"version": "X.Y.Z"` | Edit directly |
8193
- | `app/.claude-plugin/plugin.json` | `"version": "X.Y.Z"` | Edit directly |
8179
+ # Post-install verification (Phases 1-7)
8180
+ ai-toolkit --version # 5. Version OK?
8181
+ ai-toolkit status # 6. Status OK?
8182
+ ai-toolkit doctor # 7. Health check passed?
8183
+ ai-toolkit install --dry-run # 8. Global install OK?
8184
+ python3 scripts/audit_skills.py --ci # 9. Security audit clean?
8194
8185
 
8195
- ### Auto-synced (no manual action)
8186
+ # Supply-chain verification (Phase 8, v2.8.0+)
8187
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json,sys; assert json.load(sys.stdin)['version']=='2.1.0'; print('SARIF OK')" # 10. SARIF 2.1.0 well-formed?
8188
+ python3 scripts/audit_skills.py --permissions | head -30 # 11. Broad-access skills reviewed?
8189
+ npm view @softspark/ai-toolkit@X.Y.Z --json | python3 -c "import json,sys; d=json.load(sys.stdin); assert d['dist']['attestations']['provenance']['predicateType']=='https://slsa.dev/provenance/v1'; print('PROVENANCE OK')" # 12. Provenance attested on npm?
8190
+ python3 scripts/claude_app.py verify # 13. Claude Chat/Cowork plugin contract valid?
8196
8191
 
8197
- | File | Mechanism |
8198
- |------|-----------|
8199
- | `package-lock.json` | Regenerated by `npm install --package-lock-only` |
8192
+ # Deep-coverage verification (Phase 9, v3.0.0+)
8193
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8194
+ diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) && echo "OK: registry matches" # 14. Registry <-> generators drift?
8195
+ ```
8200
8196
 
8201
- ### Conditional sync (only if the doc was modified in this release)
8197
+ ---
8202
8198
 
8203
- | File | Field | When to update |
8204
- |------|-------|---------------|
8205
- | `kb/procedures/maintenance-sop.md` | frontmatter `version:` | If SOP content changed |
8206
- | `kb/reference/skills-catalog.md` | frontmatter `version:` | If skills added/removed |
8207
- | `kb/reference/agents-catalog.md` | frontmatter `version:` | If agents added/removed |
8208
- | `kb/reference/hooks-catalog.md` | frontmatter `version:` | If hooks changed |
8209
- | `kb/reference/architecture-overview.md` | frontmatter `version:` | If architecture changed |
8210
- | `kb/reference/distribution-model.md` | frontmatter `version:` | If install model changed |
8211
- | `kb/reference/global-install-model.md` | frontmatter `version:` | If install model changed |
8199
+ ## Phase 0: Pre-Commit & Pre-Push (2 min)
8212
8200
 
8213
- > **Note:** KB `version:` fields track the **document version**, not the toolkit version.
8214
- > Only bump them when the document content actually changes in this release.
8201
+ Run these commands **before every commit and push to main**. CI validates
8202
+ counts but does NOT auto-regenerate you must do it locally.
8215
8203
 
8216
- ### Count sync (if skills/agents/hooks changed)
8204
+ ```bash
8205
+ # 1. Regenerate generated artifacts
8206
+ python3 scripts/generate_agents_md.py > AGENTS.md
8207
+ python3 scripts/generate_llms_txt.py > llms.txt
8208
+ python3 scripts/generate_llms_txt.py --full > llms-full.txt
8217
8209
 
8218
- | File | What to check |
8219
- |------|---------------|
8220
- | `package.json` | `"description"` — skill/agent count |
8221
- | `README.md` | Badge counts, "What You Get" table |
8222
- | `app/ARCHITECTURE.md` | Section headings with counts |
8210
+ # 2. Validate everything (catches stale counts, missing assets)
8211
+ python3 scripts/validate.py --strict
8223
8212
 
8224
- > **Tip:** `validate.py --strict` catches count drift AND version mismatches
8225
- > (package.json vs manifest.json vs plugin.json) automatically.
8226
- > If validation passes, counts and versions are correct.
8213
+ # 3. Security audit
8214
+ python3 scripts/audit_skills.py --ci
8227
8215
 
8228
- ### Verification command
8216
+ # 4. Run tests
8217
+ npm test
8229
8218
 
8230
- After syncing, verify all mandatory files match:
8219
+ # 5. Stage and commit
8220
+ # AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated and
8221
+ # gitignored — do NOT commit them (regenerated at prepublishOnly). Only the
8222
+ # doc artifacts llms.txt / llms-full.txt are committed.
8223
+ git add llms.txt llms-full.txt
8224
+ git add -p # stage your other changes
8225
+ git commit -m "feat: your change description"
8226
+ ```
8227
+
8228
+ **Why local?** Branch protection on `main` requires PRs and status checks.
8229
+ CI cannot push directly to `main`, so generated artifacts must be committed
8230
+ by the developer as part of their PR.
8231
8231
 
8232
+ **One-liner (copy-paste):**
8232
8233
  ```bash
8233
- VERSION=$(python3 -c "import json; print(json.load(open('package.json'))['version'])")
8234
- echo "Target: $VERSION"
8235
- echo "manifest.json: $(python3 -c "import json; print(json.load(open('manifest.json'))['version'])")"
8236
- echo "plugin.json: $(python3 -c "import json; print(json.load(open('app/.claude-plugin/plugin.json'))['version'])")"
8237
- echo "package-lock.json: $(python3 -c "import json; print(json.load(open('package-lock.json'))['version'])")"
8234
+ python3 scripts/generate_agents_md.py > AGENTS.md && python3 scripts/generate_llms_txt.py > llms.txt && python3 scripts/generate_llms_txt.py --full > llms-full.txt && python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
8238
8235
  ```
8239
8236
 
8240
- All four must print the same version. If not, fix before proceeding.
8237
+ ---
8241
8238
 
8242
- ### Public surface review
8239
+ ## Phase 1: CLI & Version (1 min)
8243
8240
 
8244
8241
  ```bash
8245
- python3 scripts/surface_manifest.py --update
8246
- git diff app/surface.json
8242
+ ai-toolkit --version
8243
+ ai-toolkit --help
8244
+ which ai-toolkit
8247
8245
  ```
8248
8246
 
8249
- **Every line the diff removes is a breaking change.** Restore it, or take the
8250
- deprecation path in `BACKWARD_COMPATIBILITY.md` and add a `DECISIONS.md` entry
8251
- before the tag. Lines added are new surface being adopted into protection — that
8252
- needs no ceremony.
8253
-
8254
- Do not run `--update` to make a red build green. The check fails because something
8255
- users depend on disappeared; regenerating the manifest deletes the evidence, not
8256
- the problem.
8247
+ **Verify:**
8248
+ - [ ] `--version` returns correct semver (e.g., `1.4.0`)
8249
+ - [ ] `--help` displays full command list without errors
8250
+ - [ ] `which` points to global npm bin path
8257
8251
 
8258
- ### Skill body budget ratchet
8252
+ ---
8259
8253
 
8260
- `validate.py` prints the largest skill body on every run:
8254
+ ## Phase 2: Global Install & Status (2 min)
8261
8255
 
8256
+ ```bash
8257
+ ai-toolkit install --dry-run
8258
+ ai-toolkit status
8262
8259
  ```
8263
- Body budget: largest is <skill> at <N> bytes (warn 18000, error 20000)
8264
- ```
8265
-
8266
- Once that number sits at least 2000 bytes under `SKILL_BODY_BUDGET_WARN`, lower
8267
- `SKILL_BODY_BUDGET_WARN` by 2000 in `scripts/validate.py` and ship the tightened
8268
- threshold with the release. Target floor is 12000.
8269
8260
 
8270
- Two rules, both learned the hard way:
8271
-
8272
- - **Never lower a threshold in the same change that something violates it.** Split
8273
- the offending skill into `SKILL.md` + `reference/` first, prove it with
8274
- `python3 scripts/check_split.py <skill> --before <pre-split SKILL.md>`, then
8275
- tighten in a follow-up.
8276
- - **Never raise a threshold to make a red build green.** A body over budget means
8277
- detail that belongs in `reference/` is loading on every trigger match, including
8278
- the accidental ones. Raising the number hides the cost, it does not remove it.
8261
+ **Verify `--dry-run`:**
8262
+ - [ ] Agents: 44
8263
+ - [ ] Skills: 108
8264
+ - [ ] Hooks merged into settings.json
8265
+ - [ ] "Other AI Tools" lists documented global targets (with `--editors`): aider, antigravity, augment, cline, codex, copilot, cursor, gemini, opencode, roo, windsurf. Scope varies: Codex uses `$CODEX_HOME` (default `~/.codex`) plus `$HOME/.agents/skills`; Copilot uses `$COPILOT_HOME` (default `~/.copilot`); cursor has only `~/.cursor/hooks.json`; antigravity has the `~/.gemini/*/skills` pointer. Cursor and Antigravity rules remain project-only.
8279
8266
 
8280
- If the largest body has not moved since the previous release, leave the threshold
8281
- alone and say so in the release notes. A ratchet that never advances is honest;
8282
- one that advances past reality is not.
8267
+ **Verify `status`:**
8268
+ - [ ] Version matches expected
8269
+ - [ ] Profile: minimal/standard/strict
8270
+ - [ ] Modules: list of installed modules
8271
+ - [ ] Latest: up to date / update available
8283
8272
 
8284
8273
  ---
8285
8274
 
8286
- ## Phase 3: Write CHANGELOG Entry
8287
-
8288
- Add entry at the top of `CHANGELOG.md` (after the header, before previous release):
8289
-
8290
- ```markdown
8291
- ## vX.Y.Z — Short Title (YYYY-MM-DD)
8292
-
8293
- ### Added
8294
- - **Feature name** — description
8295
-
8296
- ### Changed
8297
- - **What changed** — old behavior → new behavior
8298
-
8299
- ### Fixed
8300
- - **Bug description** — what was broken and how it's fixed
8275
+ ## Phase 3: Doctor Health Check (1 min)
8301
8276
 
8302
- ### Removed
8303
- - **What was removed** — migration path if any
8277
+ ```bash
8278
+ ai-toolkit doctor
8304
8279
  ```
8305
8280
 
8306
- **Rules:**
8307
- - Use **bold** for feature names
8308
- - Start descriptions with a verb (Added, Changed, Fixed, Removed)
8309
- - Reference skill names with backticks and slash: `/hipaa-validate`
8310
- - Include script names: `scripts/hipaa_scan.py`
8311
- - Include count changes: `Skill count: 91 → 92`
8312
- - Date format: `YYYY-MM-DD`
8313
- - Title: short, descriptive, no version number repetition
8314
-
8315
- ### Update README "What's New" section
8316
-
8317
- **MANDATORY on every release.** Update the `## What's New in vX.Y.Z` section in `README.md`:
8318
-
8319
- 1. Change the heading version: `## What's New in vX.Y.Z`
8320
- 2. Replace bullet points with 3-5 highlights from this release
8321
- 3. **Keep only the latest version block.** Delete the previous `## What's New in vA.B.C` section(s). README is the shop window, not the archive — users see the current release, full history lives in `CHANGELOG.md`.
8322
- 4. Keep the `See [CHANGELOG.md](CHANGELOG.md) for full history.` link directly below the bullet list.
8281
+ **Expected sections (all OK):**
8282
+ - Environment: node, bash, python3, bats
8283
+ - Global Install: .claude exists, agents/skills symlinks (0 broken), settings.json hooks
8284
+ - Hook Scripts: all present and executable
8285
+ - Hook Configuration: 14 events registered
8286
+ - Generated Artifacts: AGENTS.md, llms.txt, llms-full.txt
8287
+ - Planned Assets: plugin.json, benchmarks, plugin packs
8288
+ - Benchmark Freshness: < 30 days
8289
+ - Stale Rules: all healthy
8323
8290
 
8324
- > **Warning:** This section is the first thing users see after the badges.
8325
- > A stale version here (e.g., "What's New in v2.1.3" when shipping v2.3.0)
8326
- > signals an unmaintained project. Do NOT skip this step.
8291
+ **Verify:**
8292
+ - [ ] `Errors: 0 | Warnings: 0`
8293
+ - [ ] `HEALTH CHECK PASSED`
8327
8294
 
8328
- > **Single-version rule:** README.md must contain **exactly one** `## What's New in vX.Y.Z` heading at any time. If you find multiple stacked (e.g. v2.6.1 + v2.6.0 + v2.5.0), that is a SOP drift — collapse to the latest on the next release commit.
8295
+ If doctor detects problems: `ai-toolkit doctor --fix` auto-repairs
8296
+ (broken symlinks, non-executable hooks, missing scripts, missing llms-full.txt).
8329
8297
 
8330
8298
  ---
8331
8299
 
8332
- ## Phase 4: Regenerate Artifacts
8333
-
8334
- Use the npm scripts, not the generators directly:
8300
+ ## Phase 4: Local Install (2 min)
8335
8301
 
8336
8302
  ```bash
8337
- npm run generate:agents # AI_TOOLKIT_NO_CUSTOM_RULES=1 python3 scripts/generate_agents_md.py > AGENTS.md
8338
- npm run generate:llms # llms.txt + llms-full.txt
8303
+ mkdir -p /tmp/ai-toolkit-verify && cd /tmp/ai-toolkit-verify
8304
+ git init -q
8305
+ ai-toolkit install --local --editors all --dry-run
8306
+ cd - && rm -rf /tmp/ai-toolkit-verify
8339
8307
  ```
8340
8308
 
8341
- `generate:agents` sets `AI_TOOLKIT_NO_CUSTOM_RULES=1`. Running
8342
- `generate_agents_md.py` bare picks up whatever is registered in the maintainer's
8343
- own `~/.softspark/ai-toolkit/rules/`, which then ships inside `AGENTS.md`.
8344
-
8345
- Check if anything actually changed:
8309
+ **Verify "Project-local" section:**
8310
+ - [ ] Would create: CLAUDE.md
8311
+ - [ ] Would create: .claude/settings.local.json
8312
+ - [ ] Would inject: .claude/constitution.md
8313
+ - [ ] Editors: all 11 listed (copilot, cursor, windsurf, cline, roo, aider, augment, antigravity, codex, gemini, opencode)
8314
+ - [ ] Would generate configs for each editor (legacy + directory-based)
8315
+ - [ ] Would install: .git/hooks/pre-commit
8316
+ - [ ] Would inject language rules (auto-detected)
8346
8317
 
8318
+ **Also test auto-detect (no --editors flag):**
8347
8319
  ```bash
8348
- git diff --stat AGENTS.md llms.txt llms-full.txt
8320
+ ai-toolkit install --local --dry-run
8321
+ # → Editors: none (empty project has no existing configs)
8349
8322
  ```
8350
8323
 
8351
- If no diff, the artifacts are already current. If there is a diff, stage them.
8352
-
8353
8324
  ---
8354
8325
 
8355
- ## Phase 5: Validate, Audit, Test
8356
-
8357
- Run the full quality gate:
8326
+ ## Phase 5: Validation & Security Audit (3 min)
8358
8327
 
8359
8328
  ```bash
8360
8329
  python3 scripts/validate.py --strict
8361
8330
  python3 scripts/audit_skills.py --ci
8362
- python3 scripts/audit_skills.py --sarif > audit.sarif # MANDATORY — GHAS ingest
8363
- python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8331
+ ```
8364
8332
 
8365
- # ShellCheck on hooks (added in 1.11.0). Mirrors the ci.yml "ShellCheck hooks"
8366
- # job. NOT run by validate.py, npm test, OR publish.yml so a hook with a
8367
- # ShellCheck warning passes every other gate AND still publishes on tag while
8368
- # turning main CI red. Run it here, before tagging.
8369
- shellcheck --severity=warning app/hooks/*.sh && echo "OK: shellcheck clean"
8333
+ **Verify validate.py:**
8334
+ - [ ] Agents: 44, Skills: 108, Tests: exactly the current README badge count
8335
+ - [ ] Hook events: 14, Hook scripts: >= 30
8336
+ - [ ] Plugin packs >= 10, KB documents >= 20
8337
+ - [ ] `Errors: 0 | Warnings: 0` → `VALIDATION PASSED`
8370
8338
 
8371
- # Registry / generator drift (added in 1.10.0). Meta-generators excluded.
8372
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8373
- diff \
8374
- <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) \
8375
- <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) \
8376
- && echo "OK: registry matches filesystem" \
8377
- || { echo "DRIFT: update supported-tools-registry.md before tagging"; exit 1; }
8339
+ **Verify audit_skills.py:**
8340
+ - [ ] `HIGH: 0` (MUST be zero — CI fails otherwise)
8341
+ - [ ] `WARN: 0`
8342
+ - [ ] `INFO: N` (acceptable — broad-access skills: orchestrate, swarm, workflow)
8378
8343
 
8379
- # Stage first IF this release adds or deletes a kb/ file. The test
8380
- # "npm package KB files match the tracked release set" compares `git ls-files
8381
- # kb` (the index) against what `npm pack` sees (the working tree), so an
8382
- # unstaged addition reads as "extra" and an unstaged deletion as "missing".
8383
- # Phase 6 stages, and it runs after this one, so the ordering fails the test
8384
- # for any release that touches the KB. Staging early costs nothing.
8385
- git status --porcelain kb/ | grep -qE '^(\?\?| D|\?M)' && git add -A kb/
8344
+ ---
8386
8345
 
8387
- # Run npm test ONCE, cache output, parse from file. The suite is 1400+ bats
8388
- # cases — rerunning it per check wastes minutes. Do not pipe npm test into
8389
- # tail/grep multiple times in the same session.
8346
+ ## Phase 6: Tests (3-5 min)
8347
+
8348
+ ```bash
8349
+ # Run ONCE, capture to file, then parse. Full suite is 669+ bats cases —
8350
+ # re-running it per check (tail / grep ok / grep not ok piped separately)
8351
+ # wastes minutes every release. Always cache the output.
8390
8352
  npm test > /tmp/npm-test.log 2>&1
8353
+ exit=$?
8391
8354
  tail -3 /tmp/npm-test.log
8392
- echo "ok: $(grep -c '^ok ' /tmp/npm-test.log) | not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8355
+ echo "ok: $(grep -c '^ok ' /tmp/npm-test.log)"
8356
+ echo "not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8357
+ echo "exit: $exit"
8393
8358
  ```
8394
8359
 
8395
- **Expected results:**
8396
- - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
8397
- - `audit_skills.py --ci`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
8398
- - `audit_skills.py --sarif`: valid JSON, non-empty `runs[0].tool.driver.rules`
8399
- - `audit_skills.py --permissions`: review `Skills with Bash + Write + Edit` list — any newly-added skill with broad access MUST be justified in the CHANGELOG entry
8400
- - `shellcheck --severity=warning app/hooks/*.sh`: no output, exit 0. A common false positive is `SC2034` on `INPUT` or env vars (e.g. `AI_TOOLKIT_HOOK_FORMAT`) that a *sourced* helper (`_hook-io.sh`) consumes — ShellCheck cannot see cross-file use. Fix with a `# shellcheck disable=SC2034` directive or `export`, matching `guard-destructive.sh`. Never tag with a red ShellCheck.
8401
- - Registry drift: `OK: registry matches filesystem`. If `DRIFT:` appears, add the missing `scripts/generate_*.py` rows to `kb/reference/supported-tools-registry.md` before tagging.
8402
- - `npm test`: `1..N` with zero `not ok` (read from the cached `/tmp/npm-test.log`, do not rerun)
8403
-
8404
- > **Why this matters (v4.5.1 postmortem):** `publish.yml` runs only `validate.py` + `npm test`, so it published v4.5.0 even though the `main` CI `ShellCheck hooks` job was red on two `SC2034` warnings in a new hook. The publish workflow does **not** depend on the CI workflow. Until that is fixed, ShellCheck is a manual pre-tag gate — run it here every time.
8360
+ **Verify:**
8361
+ - [ ] `exit == 0`
8362
+ - [ ] `ok == expected test count` (e.g., 945 on v3.0.0)
8363
+ - [ ] `not ok == 0`
8364
+ - [ ] Bats runs tests in parallel (4 jobs)
8365
+ - [ ] Groups: agents, autodetect, cli, generators, guards, hooks, inject,
8366
+ install, kb, mcp, readme, profiles, uninstall, validate
8405
8367
 
8406
- **One-liner:**
8368
+ **Anti-pattern — do NOT do this:**
8407
8369
  ```bash
8408
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > audit.sarif && shellcheck --severity=warning app/hooks/*.sh && diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE 'generate_agents_md\.py|generate_llms_txt\.py|generate_language_rules_skills\.py' | sort -u) && npm test
8370
+ # Runs the full suite THREE times. Adds 1-3 min and pressures CI capacity.
8371
+ npm test 2>&1 | tail -3
8372
+ npm test 2>&1 | grep -c '^ok '
8373
+ npm test 2>&1 | grep -c '^not ok'
8409
8374
  ```
8410
8375
 
8411
- **If tests fail:** Fix the issue, do NOT skip. Common failures:
8412
- - Stale counts → re-run `generate:all` or fix README/ARCHITECTURE
8413
- - Missing frontmatter add to new KB docs
8414
- - Broken symlink `ai-toolkit doctor --fix`
8376
+ **Key test areas:**
8377
+ - Guards: rm -rf, DROP TABLE, git push --force blocked
8378
+ - Install: idempotent, profiles, --only/--skip, orphan cleanup
8379
+ - Eject: real files (not symlinks), inlined rules
8380
+ - Uninstall: removes toolkit, preserves user content
8415
8381
 
8416
- ### Phase 5c: Licensing Gate (v4.20.0+)
8382
+ ---
8417
8383
 
8418
- The project is Apache-2.0. Attribution only works if the artefact actually
8419
- carries it, and every part of that is mechanically checkable.
8384
+ ## Phase 7: Eject (1 min)
8420
8385
 
8421
8386
  ```bash
8422
- # The whole gate, enforced in CI. Run it here so a failure is caught before tagging.
8423
- npx bats tests/test_licensing.bats
8387
+ mkdir -p /tmp/ai-toolkit-eject-test
8388
+ cd /tmp/ai-toolkit-eject-test
8389
+ ai-toolkit eject
8390
+ cd - && rm -rf /tmp/ai-toolkit-eject-test
8424
8391
  ```
8425
8392
 
8426
- The seven assertions, and why each exists:
8393
+ **Verify:**
8394
+ - [ ] Agents copied as real files (not symlinks)
8395
+ - [ ] Skills copied as real directories
8396
+ - [ ] Rules inlined into CLAUDE.md
8397
+ - [ ] constitution.md and ARCHITECTURE.md copied
8398
+ - [ ] `output-styles/` directory present (v2.7.1+)
8427
8399
 
8428
- | Check | Fails when |
8429
- |---|---|
8430
- | Every shipped source file carries an SPDX header | A new `.py`/`.sh`/`.js`/`.bats` file was added without one — the common case, and the reason this is a test rather than a habit |
8431
- | Headers name Apache-2.0 and nothing else | A file was copied in from an MIT/GPL source with its own header intact |
8432
- | **No** markdown file carries an SPDX header | Someone "helpfully" ran the header script over `app/skills/` — headers there sit above parsed frontmatter and bill every session for it |
8433
- | `LICENSE` is the complete Apache 2.0 text | The file was truncated or replaced with a summary |
8434
- | `NOTICE` carries attribution, the source URL, §4(d) and the MIT-era notice | The attribution mechanism was gutted |
8435
- | `LICENSE` **and** `NOTICE` ship in the npm package | `package.json` `files` lost an entry — a NOTICE that never reaches the consumer cannot satisfy §4(d) |
8436
- | Every manifest declaring a licence declares Apache-2.0 | `package.json`, `manifest.json`, `plugin.json` and `package-lock.json` drifted apart |
8400
+ ---
8437
8401
 
8438
- **Adding source files in this release?** The header goes *after* the shebang,
8439
- never before it. Short SPDX form:
8402
+ ## Phase 8: Supply-Chain Verification (2 min, v2.8.0+)
8440
8403
 
8441
- ```
8442
- # SPDX-License-Identifier: Apache-2.0
8443
- # Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
8444
- # Source: https://github.com/softspark/ai-toolkit
8445
- ```
8404
+ These checks enforce the v2.8.0 security standard on a freshly-published release. Run AFTER the `publish.yml` workflow completes on the tag.
8446
8405
 
8447
- `//` for JavaScript. Full convention and the reasoning behind the markdown
8448
- exclusion: [Licensing](../reference/licensing.md).
8406
+ ### 8.1 Provenance attestation on npm
8449
8407
 
8450
- **Changing the licence itself?** Do not hand-type the licence text. Take it
8451
- verbatim from a published copy and cross-verify against a second independent
8452
- copy before writing `LICENSE` — a rendered or summarised licence is not the
8453
- licence. Prior releases stay under their original terms; a licence change
8454
- applies going forward and revokes nothing already granted.
8408
+ ```bash
8409
+ VERSION="X.Y.Z" # the tag just published
8410
+ npm view "@softspark/ai-toolkit@${VERSION}" --json \
8411
+ | python3 -c "import json, sys; d=json.load(sys.stdin); att=d['dist'].get('attestations', {}); assert att.get('provenance', {}).get('predicateType') == 'https://slsa.dev/provenance/v1', f'NO PROVENANCE for {d[\"version\"]}'; print(f'PROVENANCE OK: {att[\"url\"]}')"
8412
+ ```
8455
8413
 
8456
- ### Phase 5a: Supply-Chain Hardening Verification (v2.8.0+)
8414
+ **Verify:**
8415
+ - [ ] Exit 0 and prints `PROVENANCE OK: https://registry.npmjs.org/...`
8416
+ - [ ] `https://www.npmjs.com/package/@softspark/ai-toolkit/v/${VERSION}` shows the green "Provenance" badge
8457
8417
 
8458
- These checks enforce the security standard introduced in v2.8.0. Do NOT tag a release until all pass.
8418
+ **If provenance is missing:** the `publish.yml` ran without `id-token: write` or `--provenance`. Restore them and cut a patch release a silently unsigned publish is a regression against the v2.8.0 standard.
8459
8419
 
8460
- **1. Publish workflow emits provenance:**
8420
+ ### 8.2 Audit SARIF output (for GHAS ingest)
8461
8421
 
8462
8422
  ```bash
8463
- grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
8423
+ python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif
8424
+ python3 -c "import json; d=json.load(open('/tmp/audit.sarif')); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']=='ai-toolkit-audit-skills'; print(f'SARIF OK: {len(d[\"runs\"][0][\"results\"])} results across {len(d[\"runs\"][0][\"tool\"][\"driver\"][\"rules\"])} rules')"
8464
8425
  ```
8465
8426
 
8466
- - [ ] Both markers present (`--provenance` flag + `id-token: write` permission)
8467
- - [ ] Any PR that changes `publish.yml` REQUIRES an approved security review
8427
+ **Verify:**
8428
+ - [ ] Valid SARIF 2.1.0
8429
+ - [ ] In the publishing repo, the CI job uploads `audit.sarif` via `github/codeql-action/upload-sarif@v3` so findings appear in the Security tab
8468
8430
 
8469
- **2. URL-sourced rules and hooks are checksum-pinned:**
8431
+ ### 8.3 Per-skill permissions report
8470
8432
 
8471
8433
  ```bash
8472
- # On a machine that has consumed URL rules/hooks at least once
8473
- # (schema_version 1: entries live under the .rules / .hooks key):
8474
- jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
8475
- jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
8434
+ python3 scripts/audit_skills.py --permissions | head -40
8476
8435
  ```
8477
8436
 
8478
- - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
8479
- - [ ] If not, run `ai-toolkit update` to backfill missing hashes before tagging
8437
+ **Verify:**
8438
+ - [ ] Bash skill count has NOT jumped unexpectedly since the previous release
8439
+ - [ ] Any newly added entry under `Skills with Bash + Write + Edit` matches a CHANGELOG bullet that justifies the broad scope
8440
+ - [ ] JSON form (`--permissions --json`) is available for automated drift dashboards
8480
8441
 
8481
- **3. Audit SARIF output is well-formed:**
8442
+ ### 8.4 URL-sourced rules/hooks are checksum-pinned
8443
+
8444
+ `sources.json` is an envelope of the form `{"schema_version": 1, "rules"|"hooks": {...}}`, so the jq filter must pick the nested map before piping into the pin assertion. The URL-count in the success message ignores local `path:`-only entries — only entries with a `url:` field are pinned and counted.
8482
8445
 
8483
8446
  ```bash
8484
- python3 scripts/audit_skills.py --sarif | python3 -c "import json, sys; d=json.load(sys.stdin); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']; print('SARIF OK')"
8447
+ jq '.rules // .hooks // {}' ~/.softspark/ai-toolkit/rules/sources.json 2>/dev/null \
8448
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'RULE PIN OK: {url_n} URL rules, all with sha256')"
8449
+ jq '.hooks // {}' ~/.softspark/ai-toolkit/hooks/external/sources.json 2>/dev/null \
8450
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'HOOK PIN OK: {url_n} URL hooks, all with sha256')"
8485
8451
  ```
8486
8452
 
8487
- - [ ] Prints `SARIF OK`
8488
- - [ ] If the script ever grows new rule classes, extend the SARIF `rules[]` coverage before releasing
8453
+ **Verify:**
8454
+ - [ ] Both commands print `... PIN OK`
8455
+ - [ ] If any entry is unpinned, the `register_url_source()` call missed passing `content=` — fix the call-site and retag
8489
8456
 
8490
- **4. Strict-pin mode passes on CI** (optional, recommended for stable branches):
8457
+ ### 8.5 Strict-pin smoke test (optional but recommended)
8491
8458
 
8492
8459
  ```bash
8493
8460
  AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
8494
8461
  ```
8495
8462
 
8463
+ **Verify:**
8496
8464
  - [ ] Exit 0, no `CHECKSUM CHANGED` line
8497
- - [ ] Any unexpected upstream change blocks the release until explicitly approved
8465
+ - [ ] If a checksum change was intentional (e.g. upstream rule update), document it in the CHANGELOG entry before tagging
8498
8466
 
8499
8467
  ---
8500
8468
 
8501
- ## Phase 6: Commit
8469
+ ## Phase 9: Deep-Coverage Checks (v3.0.0+)
8502
8470
 
8503
- Stage all release files:
8471
+ These verify the native-surface generators shipped in v3.0.0 actually emit the right files for the right profiles, and that the tool registry stays in sync with shipped generators.
8472
+
8473
+ > **Safety warning — HOME-scoped writes:** Running `--profile full` with `augment` in the editor list writes to `$HOME/.augment/settings.json` (Augment stores hooks under HOME, not per-project). Use `--dry-run` for verification unless you intend to carry ai-toolkit hook entries on this machine. The generator is marker-safe (only rewrites its own `_source: ai-toolkit` entries) but is still a side-effect.
8474
+
8475
+ ### 9.1 `--profile full` emits every native surface
8504
8476
 
8505
8477
  ```bash
8506
- git add package.json manifest.json app/.claude-plugin/plugin.json
8507
- git add package-lock.json
8508
- git add CHANGELOG.md
8509
- git add llms.txt llms-full.txt
8510
- # NOTE: AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated
8511
- # editor configs and are gitignored — do NOT commit them. `prepublishOnly` runs
8512
- # `npm run generate:all`, so the shipped package (which lists AGENTS.md in
8513
- # package.json `files`) gets a fresh copy at publish time.
8514
- git add -p # review and stage any other changes
8478
+ D=/tmp/aitk-profile-full-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8479
+ ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot \
8480
+ --profile full --dry-run 2>&1 \
8481
+ | grep -E "\\.cursor/(hooks\\.json|agents)|\\.devin/hooks\\.v1\\.json|\\.gemini/(settings\\.json|commands)|\\.augment/(agents|commands)|\\.codex/(agents|hooks)|\\.agents/skills|\\.github/(agents|skills|instructions|prompts|hooks)"
8515
8482
  ```
8516
8483
 
8517
- Commit:
8484
+ **Verify** — at least the following lines appear:
8485
+ - [ ] `.cursor/hooks.json` and `.cursor/agents/`
8486
+ - [ ] `.devin/hooks.v1.json`
8487
+ - [ ] `.gemini/settings.json` hooks AND `.gemini/commands/`
8488
+ - [ ] `.augment/agents/` + `.augment/commands/` + `$HOME/.augment/settings.json`
8489
+ - [ ] `.codex/hooks.json` + `.codex/hooks/` + `.codex/agents/`
8490
+ - [ ] `.agents/skills/` (Codex native discovery path)
8491
+ - [ ] `.github/agents/` + `.github/skills/` + `.github/instructions/` + `.github/prompts/` + `.github/hooks/`
8492
+
8493
+ ### 9.2 Codex skills are profile-independent
8518
8494
 
8519
8495
  ```bash
8520
- git commit -m "chore: release vX.Y.Z"
8496
+ D=/tmp/aitk-codex-skills-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8497
+ ai-toolkit install --local --editors codex --profile minimal --dry-run 2>&1 \
8498
+ | grep -q "Would generate: .agents/skills" && echo "OK: Codex skills emit at minimal"
8499
+ ai-toolkit install --local --editors codex --profile standard --codex-skills --dry-run 2>&1 \
8500
+ | grep -q "Would refresh: .agents/skills" && echo "OK: legacy flag explicitly refreshes the same catalog"
8521
8501
  ```
8522
8502
 
8523
- ---
8503
+ **Verify:**
8504
+ - [ ] Selecting Codex emits all skills under `.agents/skills/` at every profile
8505
+ - [ ] `--codex-skills` remains an explicit refresh alias; it is not required for installation
8506
+ - [ ] No profile emits `.codex/skills/`
8507
+
8508
+ ### 9.3 Breaking-change surfaces land on `--profile standard`
8524
8509
 
8525
- ## Phase 7: Tag and Push
8510
+ The standard profile must include:
8511
+ - Copilot scoped instructions, prompts, custom agents, portable skills, and hooks
8512
+ - Gemini hooks (`.gemini/settings.json`)
8526
8513
 
8527
8514
  ```bash
8528
- git tag vX.Y.Z
8515
+ D=/tmp/aitk-breaking-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8516
+ ai-toolkit install --local --editors copilot,gemini --profile standard --dry-run 2>&1 \
8517
+ | tee /tmp/aitk-breaking.log
8518
+ grep -q "\\.github/instructions/" /tmp/aitk-breaking.log && echo "OK: Copilot dir layout at standard"
8519
+ grep -q "\\.github/agents/.*\\.github/skills/" /tmp/aitk-breaking.log && echo "OK: Copilot agents and skills at standard"
8520
+ grep -q "\\.github/hooks/" /tmp/aitk-breaking.log && echo "OK: Copilot hooks at standard"
8521
+ grep -q "\\.gemini/settings\\.json hooks" /tmp/aitk-breaking.log && echo "OK: Gemini hooks at standard"
8522
+ ```
8529
8523
 
8530
- # Assert the tag before pushing it. Both checks are one line each and both
8531
- # have caught a real broken release.
8532
- test "$(git rev-parse vX.Y.Z)" = "$(git rev-parse HEAD)" \
8533
- || { echo "FAIL: tag is not on HEAD"; exit 1; }
8534
- git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" \
8535
- || { echo "FAIL: tag is not on the chore: release commit"; exit 1; }
8524
+ **Verify all four lines print `OK:`**. If any is missing, native Copilot or Gemini coverage has regressed.
8536
8525
 
8537
- # Push the branch, then the single release tag by its full ref.
8538
- git push origin main
8539
- git push origin refs/tags/vX.Y.Z
8540
- ```
8526
+ ### 9.4 Install is idempotent
8541
8527
 
8542
- **Why the assertions (v4.19.0 postmortem).** v4.19.0 was tagged on a commit
8543
- that contained only a KB document and still carried `package.json` version
8544
- `4.18.0`; the actual release sat in the commit above it under a recycled
8545
- `fix:` message. `publish.yml` fired, tried to publish a version already on
8546
- npm, and failed. Nothing on npm, a tag pointing at the wrong tree, and the
8547
- only way out was rewriting a pushed commit. Both assertions above catch this
8548
- in under a second. Run them.
8528
+ ```bash
8529
+ D=/tmp/aitk-idem-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8530
+ # Sort file paths before hashing find traversal order follows inode order,
8531
+ # which can shift between runs even when content is byte-identical, producing
8532
+ # false FAIL signals.
8533
+ ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
8534
+ SHA1=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
8535
+ ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
8536
+ SHA2=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
8537
+ [ "$SHA1" = "$SHA2" ] && echo "OK: idempotent" || echo "FAIL: install is not idempotent"
8538
+ ```
8549
8539
 
8550
- **Never `git push --tags`.** It pushes every local tag at once, and GitHub
8551
- suppresses tag-triggered workflow runs when many tags arrive in a single push
8552
- — the workflow silently does not fire and nothing publishes. Push the single
8553
- release tag by its full ref, as above. (Sibling evidence: this is exactly how
8554
- rag-mcp's `1.0.3` image build was skipped, when a `--tags` push carried 37
8555
- tags at once.)
8540
+ **Verify:** prints `OK: idempotent`. A second run must produce byte-identical files in every managed path.
8556
8541
 
8557
- This triggers `.github/workflows/publish.yml` which:
8558
- 1. Runs `validate.py --strict`
8559
- 2. Runs `npm test`
8560
- 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z` with `--provenance` (SLSA v1 build attestation)
8542
+ ### 9.5 Live-install JSON outputs parse
8561
8543
 
8562
- **Provenance is non-negotiable.** If `id-token: write` permission or the `--provenance` flag is missing from `publish.yml`, fix it BEFORE tagging an unsigned release is a regression against the v2.8.0 standard.
8544
+ The bats suite validates JSON shape at generation time. This re-checks that what actually landed on disk after a live install parses without errors.
8563
8545
 
8564
- **After CI completes:** Run the [Release Verification SOP](release-verification-sop.md)
8565
- to smoke-test the published package AND verify the provenance attestation landed on npm.
8546
+ ```bash
8547
+ D=/tmp/aitk-json-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8548
+ ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot --profile full >/dev/null 2>&1
8549
+ for f in .cursor/hooks.json .devin/hooks.v1.json .gemini/settings.json .codex/hooks.json .github/hooks/ai-toolkit.json "$HOME/.augment/settings.json"; do
8550
+ [ -f "$f" ] && python3 -c "import json; json.load(open('$f'))" && echo "OK: $f"
8551
+ done
8552
+ ```
8566
8553
 
8567
- ---
8554
+ **Verify:** each emitted file prints `OK: <path>`. Any `json.decoder.JSONDecodeError` means the merge logic corrupted the output.
8568
8555
 
8569
- ## Rollback
8556
+ ### 9.6 Registry / generator drift check
8570
8557
 
8571
- If a bad release was published:
8558
+ `kb/reference/supported-tools-registry.md` should enumerate every per-editor `scripts/generate_*.py` we ship. Meta-generators (`generate_agents_md.py`, `generate_llms_txt.py`) are excluded — they produce docs/artifacts, not editor configs.
8572
8559
 
8573
8560
  ```bash
8574
- # Unpublish from npm (within 72h)
8575
- npm unpublish @softspark/ai-toolkit@X.Y.Z
8561
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8562
+ REG=$(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u)
8563
+ FS=$(ls scripts/generate_*.py | grep -vE "$META" | sort -u)
8564
+ diff <(echo "$REG") <(echo "$FS") && echo "OK: registry matches filesystem" || echo "DRIFT: update supported-tools-registry.md"
8565
+ ```
8576
8566
 
8577
- # Or deprecate (preferred doesn't break existing installs)
8578
- npm deprecate @softspark/ai-toolkit@X.Y.Z "Known issue: <description>. Use vA.B.C instead."
8567
+ **Verify:** prints `OK: registry matches filesystem`. If not, add the missing rows to the registry before tagging the next release.
8579
8568
 
8580
- # Delete tag
8581
- git tag -d vX.Y.Z
8582
- git push origin --delete vX.Y.Z
8569
+ ### 9.7 Claude Chat / Cowork plugin validates and exports
8570
+
8571
+ ```bash
8572
+ python3 scripts/claude_app.py verify
8573
+ D=/tmp/aitk-claude-app-${RANDOM}
8574
+ python3 scripts/claude_app.py export --output "$D.zip" --no-custom-rules
8575
+ python3 -c "import zipfile; z=zipfile.ZipFile('$D.zip'); assert '.claude-plugin/plugin.json' in z.namelist(); assert 'claude-app/skills/ai-toolkit-rules/SKILL.md' in z.namelist(); print('OK: Claude app archive')"
8583
8576
  ```
8584
8577
 
8578
+ **Verify:** the official validator exits 0; the archive contains the manifest,
8579
+ app-native rules skill, bundled agents/skills, and plugin-relative Cowork hooks.
8580
+
8585
8581
  ---
8586
8582
 
8587
- ## Checklist Summary
8583
+ ## Phase 10: Explicit DSH Qualification (v4.30.0+)
8588
8584
 
8589
- | # | Step | Command / Action | Pass Criteria |
8590
- |---|------|-----------------|---------------|
8591
- | 0a | Ecosystem drift check | `ecosystem_doctor.py --format text` | All tools Clean, or drift classified and resolved |
8592
- | 0b | Ecosystem snapshot refresh | `ecosystem_doctor.py --update` | `benchmarks/ecosystem-doctor-snapshot.json` updated |
8593
- | 0c | Ecosystem gate | `ecosystem_doctor.py --offline --check` | Exit 0 |
8594
- | 1 | Version bump type | Decide patch/minor/major | — |
8595
- | 2 | `package.json` version | Edit `"version"` | Matches target |
8596
- | 3 | `manifest.json` version | Edit `"version"` | Matches target |
8597
- | 4 | `plugin.json` version | Edit `"version"` | Matches target |
8598
- | 5 | `package-lock.json` | `npm install --package-lock-only` | Matches target |
8599
- | 6 | Count sync | Check `package.json` description, README | `validate.py` passes |
8600
- | 7 | CHANGELOG.md | Add release entry (incl. `Ecosystem` subsection if any B/D/E/F drift) | Entry exists for vX.Y.Z |
8601
- | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_llms_txt.py` | No unexpected diff |
8602
- | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
8603
- | 10 | Security audit (CI mode) | `audit_skills.py --ci` | 0 HIGH |
8604
- | 11 | Security audit (SARIF) | `audit_skills.py --sarif` | Valid SARIF 2.1.0 JSON |
8605
- | 12 | Per-skill permissions | `audit_skills.py --permissions` | New broad-access skills justified in CHANGELOG |
8606
- | 13 | ShellCheck hooks | `shellcheck --severity=warning app/hooks/*.sh` | Exit 0, no output (mirrors ci.yml; publish.yml does NOT run it) |
8607
- | 14 | Provenance flag check | `grep -- '--provenance' .github/workflows/publish.yml` | Present |
8608
- | 15 | Checksum-pin backfill | `sources.json` entries all have `sha256` | No unpinned URL sources |
8609
- | 15a | Licensing gate | `npx bats tests/test_licensing.bats` | 7/7 — SPDX headers, LICENSE, NOTICE, npm `files`, manifest consistency |
8610
- | 16 | Tests | `git add -A kb/` if the KB changed, then `npm test` | All pass |
8611
- | 17 | Commit | `git commit` | Clean working tree |
8612
- | 18 | Tag | `git tag vX.Y.Z` | Tag exists |
8613
- | 18a | Tag is on HEAD | `test "$(git rev-parse vX.Y.Z)" = "$(git rev-parse HEAD)"` | Exit 0 |
8614
- | 18b | Tag is on the release commit | `git show --no-patch --format=%s vX.Y.Z` | Reads `chore: release vX.Y.Z` |
8615
- | 19 | Push branch, then the single tag | `git push origin main && git push origin refs/tags/vX.Y.Z` | CI triggered with `id-token: write`. Never `--tags`. |
8585
+ Run this phase whenever the release changes the `dsh` target, package pins, preset lifecycle, or DSH compatibility documentation. Use a new task-specific `DSH_HOME`; never replace `HOME` or reuse a regular profile.
8616
8586
 
8617
- ---
8587
+ Prerequisites: DSH `0.1.1-rc.2`, pnpm `>=11.7.0,<12.0.0`, Codex logged in through ChatGPT, Claude Code logged in natively, and GitHub Copilot CLI logged in natively. Do not supply provider API keys.
8618
8588
 
8619
- ## kb/procedures/release-verification-sop.md
8589
+ ```bash
8590
+ DSH_SMOKE_ROOT="$(mktemp -d)"
8591
+ export DSH_HOME="$DSH_SMOKE_ROOT/dsh-home"
8620
8592
 
8621
- ---
8622
- title: "SOP: Release Verification"
8623
- category: procedures
8624
- service: ai-toolkit
8625
- tags: [sop, verification, release, smoke-test, install, update, qa, provenance, sarif, dsh]
8626
- version: "1.8.0"
8627
- created: "2026-04-08"
8628
- last_updated: "2026-09-01"
8629
- description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit. Verifies CLI, native Codex and GitHub Copilot surfaces, explicit DSH lifecycle, Claude app export, doctor, validation, tests, eject, provenance, SARIF, and per-skill permissions."
8630
- ---
8593
+ ai-toolkit dsh install --profile web
8594
+ ai-toolkit dsh doctor --profile web
8595
+ dsh --profile web --host 127.0.0.1 --port 0 --no-open
8596
+ ```
8631
8597
 
8632
- # SOP: Release Verification
8598
+ In a new `softspark-orchestrator` session, select the `codex` provider and run two standalone marker prompts:
8633
8599
 
8634
- End-to-end smoke test after installing or updating `@softspark/ai-toolkit`.
8635
- Verifies all critical paths from the user's perspective.
8600
+ 1. `subagent_claude_code` returns an exact child marker, then the Codex parent returns its exact completion marker.
8601
+ 2. `subagent_gemini_copilot` returns an exact child marker, then the Codex parent returns its exact completion marker.
8636
8602
 
8637
- **Use this SOP when:**
8638
- - After `npm install -g @softspark/ai-toolkit@latest`
8639
- - After `ai-toolkit update`
8640
- - Before tagging a new version (`git tag`)
8641
- - Before publishing to npm (`npm publish`)
8642
- - As a smoke test in CI/CD
8603
+ Stop DSH, then remove only the managed profile artifacts:
8643
8604
 
8644
- **Prerequisites:**
8645
- - Node.js >= 18, Python 3, `bats`, git
8646
- - `@softspark/ai-toolkit` installed globally
8605
+ ```bash
8606
+ ai-toolkit dsh uninstall --profile web --yes
8607
+ ```
8647
8608
 
8648
- **Time:** 10-15 minutes (full), 2 minutes (quick checklist)
8609
+ **Verify:** both tool results have `isError=false`, both turns end as `completed`, `doctor` reports no recovery requirement before uninstall, and an unrelated preset fixture remains unchanged. Preserve only redacted event sequence evidence; never attach credentials, auth files, or full private prompts.
8649
8610
 
8650
8611
  ---
8651
8612
 
8652
- ## Quick Checklist (TL;DR)
8613
+ ## Troubleshooting
8653
8614
 
8654
- The 14 core commands below must pass. Releases that change DSH must also complete Phase 10.
8615
+ ### `ai-toolkit: command not found`
8655
8616
 
8656
8617
  ```bash
8657
- # Pre-commit (Phase 0)
8658
- python3 scripts/generate_agents_md.py > AGENTS.md # 1. Regenerate AGENTS.md
8659
- python3 scripts/generate_llms_txt.py > llms.txt # 2. Regenerate llms.txt
8660
- python3 scripts/validate.py --strict # 3. Validation passed?
8661
- npm test > /tmp/npm-test.log 2>&1 && grep -c '^ok ' /tmp/npm-test.log && ! grep -q '^not ok' /tmp/npm-test.log # 4. All tests passed? (single run, cached)
8662
-
8663
- # Post-install verification (Phases 1-7)
8664
- ai-toolkit --version # 5. Version OK?
8665
- ai-toolkit status # 6. Status OK?
8666
- ai-toolkit doctor # 7. Health check passed?
8667
- ai-toolkit install --dry-run # 8. Global install OK?
8668
- python3 scripts/audit_skills.py --ci # 9. Security audit clean?
8618
+ npm install -g @softspark/ai-toolkit
8619
+ # or check PATH:
8620
+ export PATH="$(npm config get prefix)/bin:$PATH"
8621
+ ```
8669
8622
 
8670
- # Supply-chain verification (Phase 8, v2.8.0+)
8671
- python3 scripts/audit_skills.py --sarif | python3 -c "import json,sys; assert json.load(sys.stdin)['version']=='2.1.0'; print('SARIF OK')" # 10. SARIF 2.1.0 well-formed?
8672
- python3 scripts/audit_skills.py --permissions | head -30 # 11. Broad-access skills reviewed?
8673
- npm view @softspark/ai-toolkit@X.Y.Z --json | python3 -c "import json,sys; d=json.load(sys.stdin); assert d['dist']['attestations']['provenance']['predicateType']=='https://slsa.dev/provenance/v1'; print('PROVENANCE OK')" # 12. Provenance attested on npm?
8674
- python3 scripts/claude_app.py verify # 13. Claude Chat/Cowork plugin contract valid?
8623
+ ### Doctor: broken symlinks
8675
8624
 
8676
- # Deep-coverage verification (Phase 9, v3.0.0+)
8677
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
8678
- diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) && echo "OK: registry matches" # 14. Registry <-> generators drift?
8625
+ ```bash
8626
+ ai-toolkit doctor --fix # auto-repair
8627
+ ai-toolkit update # or full re-install
8628
+ ```
8629
+
8630
+ ### Tests fail: missing bats
8631
+
8632
+ ```bash
8633
+ brew install bats-core # macOS
8634
+ npm install -g bats # cross-platform
8635
+ ```
8636
+
8637
+ ### validate.py: stale counts
8638
+
8639
+ README badges don't match the current agents/skills/tests counts.
8640
+ Update README.md and re-run.
8641
+
8642
+ ### Eject: missing skills
8643
+
8644
+ ```bash
8645
+ ai-toolkit update # re-link missing symlinks
8646
+ ai-toolkit eject /tmp/test # retry
8679
8647
  ```
8680
8648
 
8681
8649
  ---
8682
8650
 
8683
- ## Phase 0: Pre-Commit & Pre-Push (2 min)
8651
+ ## Success Criteria
8684
8652
 
8685
- Run these commands **before every commit and push to main**. CI validates
8686
- counts but does NOT auto-regenerate — you must do it locally.
8653
+ | Area | Criterion |
8654
+ |------|-----------|
8655
+ | CLI | `--version` correct, `--help` full list, `status` current |
8656
+ | Health | `doctor`: 0 errors, 0 warnings, PASSED |
8657
+ | Install | `--dry-run` correct counts, `--local` all configs |
8658
+ | Quality | `validate.py --strict`: PASSED |
8659
+ | Security (baseline) | `audit_skills.py --ci`: 0 HIGH |
8660
+ | Security (SARIF) | `audit_skills.py --sarif`: valid SARIF 2.1.0 with non-empty rules array |
8661
+ | Security (permissions) | `audit_skills.py --permissions`: broad-access skills unchanged or justified in CHANGELOG |
8662
+ | Supply chain | `dist.attestations.provenance.predicateType == https://slsa.dev/provenance/v1` on npm |
8663
+ | Supply chain | All `sources.json` URL entries carry a `sha256`; `AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run` passes |
8664
+ | Tests | `npm test`: N/N passed, 0 failures |
8665
+ | Eject | Standalone `.claude/` with real files AND `output-styles/` directory |
8666
+ | Guards | Destructive commands blocked |
8667
+ | Deep coverage | `--profile full` emits every documented native surface, including Codex agents/skills/hooks and Copilot agents/skills/hooks |
8668
+ | Profile contract | Codex skills emit at every profile without `--codex-skills`; Copilot standard emits agents, skills, instructions, prompts, and hooks |
8669
+ | Idempotence | Second `install` run produces byte-identical output in every managed path |
8670
+ | Live JSON | Every generated `.json` file on disk parses as valid JSON |
8671
+ | Registry | `supported-tools-registry.md` enumerates every `scripts/generate_*.py` we ship |
8672
+
8673
+ ---
8674
+
8675
+ ## kb/procedures/sop-release.md
8676
+
8677
+ ---
8678
+ title: "SOP: Release Preparation"
8679
+ category: procedures
8680
+ service: ai-toolkit
8681
+ tags: [sop, release, version, publish, changelog, semver, provenance, sarif, ecosystem, shellcheck]
8682
+ version: "1.15.0"
8683
+ created: "2026-04-10"
8684
+ last_updated: "2026-09-02"
8685
+ description: "Step-by-step checklist for preparing a new ai-toolkit release — ecosystem-sync drift check, version sync, changelog, artifact regeneration, validation, branch CI, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, checksum-pin, ShellCheck, licensing, exact-tag assertions, and a green Ubuntu/macOS branch-CI gate before any release tag is created."
8686
+ ---
8687
+
8688
+ # SOP: Release Preparation
8689
+
8690
+ Complete checklist for preparing a new `@softspark/ai-toolkit` release.
8691
+ Run this **before** tagging. After tagging and publishing, run the
8692
+ [Release Verification SOP](sop-release-verification.md) to smoke-test.
8693
+
8694
+ **Pipeline:**
8695
+ ```
8696
+ Ecosystem Sync SOP (drift check + generator updates)
8697
+
8698
+ Release Preparation (this SOP)
8699
+
8700
+ git tag → CI publish → Release Verification SOP
8701
+ ```
8702
+
8703
+ **Time:** 10-20 minutes (includes ecosystem sync review)
8704
+
8705
+ ---
8706
+
8707
+ ## Quick Checklist (TL;DR)
8687
8708
 
8688
8709
  ```bash
8689
- # 1. Regenerate generated artifacts
8710
+ # 0. Ecosystem sync (mandatory for minor/major releases; optional for patch)
8711
+ # Full procedure: kb/procedures/sop-ecosystem-sync.md
8712
+ python3 scripts/ecosystem_doctor.py --format text > /tmp/eco-report.txt
8713
+ cat /tmp/eco-report.txt
8714
+ # If drift detected: stop here, follow sop-ecosystem-sync.md Phase 2-4 to
8715
+ # classify each drift (A-F), update generators as needed, refresh snapshot,
8716
+ # THEN resume this SOP.
8717
+ python3 scripts/ecosystem_doctor.py --update # after all drift resolved
8718
+
8719
+ # 1. Decide version bump
8720
+ # patch (1.4.2 → 1.4.3): bugfix, typo, doc fix
8721
+ # minor (1.4.2 → 1.5.0): new feature, new skill, new flag, any ecosystem-class-B/F change
8722
+ # major (1.4.2 → 2.0.0): breaking change, any ecosystem-class-D removed path
8723
+
8724
+ # 2. Sync version across all files
8725
+ python3 scripts/sync_version.py X.Y.Z # if script exists, else manual
8726
+
8727
+ # 3. Write CHANGELOG.md entry
8728
+ # 4. Regenerate artifacts
8690
8729
  python3 scripts/generate_agents_md.py > AGENTS.md
8691
8730
  python3 scripts/generate_llms_txt.py > llms.txt
8692
8731
  python3 scripts/generate_llms_txt.py --full > llms-full.txt
8693
8732
 
8694
- # 2. Validate everything (catches stale counts, missing assets)
8695
- python3 scripts/validate.py --strict
8733
+ # 5. Validate + audit + SARIF + shellcheck + test + ecosystem check
8734
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif && shellcheck --severity=warning app/hooks/*.sh && npm test
8696
8735
 
8697
- # 3. Security audit
8698
- python3 scripts/audit_skills.py --ci
8736
+ # 5a. Supply-chain standard (v2.8.0+) — non-negotiable
8737
+ grep -q -- '--provenance' .github/workflows/publish.yml || { echo "MISSING --provenance"; exit 1; }
8738
+ grep -q 'id-token: write' .github/workflows/publish.yml || { echo "MISSING id-token: write"; exit 1; }
8739
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8699
8740
 
8700
- # 4. Run tests
8701
- npm test
8741
+ # 5b. Ecosystem gate — snapshot must be current before tag
8742
+ python3 scripts/ecosystem_doctor.py --offline --check || { echo "STALE ecosystem snapshot — re-run doctor"; exit 1; }
8702
8743
 
8703
- # 5. Stage and commit
8704
- # AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated and
8705
- # gitignored — do NOT commit them (regenerated at prepublishOnly). Only the
8706
- # doc artifacts llms.txt / llms-full.txt are committed.
8707
- git add llms.txt llms-full.txt
8708
- git add -p # stage your other changes
8709
- git commit -m "feat: your change description"
8744
+ # 5c. Licensing gate — SPDX headers, LICENSE, NOTICE, manifest consistency
8745
+ npx bats tests/test_licensing.bats || { echo "LICENSING GATE FAILED"; exit 1; }
8746
+
8747
+ # 6. Commit, push branch, and wait for the exact commit's full CI
8748
+ git add -A && git commit -m "chore: release vX.Y.Z"
8749
+ RELEASE_SHA=$(git rev-parse HEAD)
8750
+ git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
8751
+ RUN_ID=""
8752
+ for ATTEMPT in 1 2 3 4 5; do
8753
+ RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
8754
+ --limit 1 --json databaseId --jq '.[0].databaseId')
8755
+ [ -n "$RUN_ID" ] && break
8756
+ echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
8757
+ [ "$ATTEMPT" -eq 5 ] || sleep 60
8758
+ done
8759
+ test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
8760
+ gh run watch "$RUN_ID" --exit-status \
8761
+ || { echo "FAIL: release commit CI failed"; exit 1; }
8762
+ for OS in ubuntu-latest macos-latest; do
8763
+ gh run view "$RUN_ID" --json jobs \
8764
+ --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
8765
+ | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
8766
+ done
8767
+ test "$(git rev-parse HEAD)" = "$RELEASE_SHA" || { echo "FAIL: local HEAD changed during CI"; exit 1; }
8768
+ git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
8769
+ test "$(git rev-parse origin/main)" = "$RELEASE_SHA" || { echo "FAIL: origin/main changed during CI"; exit 1; }
8770
+
8771
+ # 6a. Only green branch CI authorizes tag creation
8772
+ git tag vX.Y.Z "$RELEASE_SHA"
8773
+
8774
+ # 6b. Assert the tag before pushing it (v4.19.0 postmortem, Phase 7)
8775
+ test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" || { echo "FAIL: tag not on tested release commit"; exit 1; }
8776
+ git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" || { echo "FAIL: tag not on release commit"; exit 1; }
8777
+
8778
+ # 6c. Push the single tag by full ref. Never --tags.
8779
+ git push origin refs/tags/vX.Y.Z
8710
8780
  ```
8711
8781
 
8712
- **Why local?** Branch protection on `main` requires PRs and status checks.
8713
- CI cannot push directly to `main`, so generated artifacts must be committed
8714
- by the developer as part of their PR.
8782
+ ---
8783
+
8784
+ ## Phase 0: Ecosystem Sync (MANDATORY for minor/major)
8785
+
8786
+ Before touching version numbers, confirm the toolkit is aligned with the current state of every editor / platform it integrates with. Skipping this phase ships a release whose generators may lag a month-old CLI refactor, a rename of `.cursorrules` to `.cursor/rules/`, or a new hook event we do not yet emit.
8787
+
8788
+ **When this phase is mandatory:**
8789
+ - Minor release (X.Y.0) — always
8790
+ - Major release (X.0.0) — always
8791
+ - Patch release (X.Y.Z) — only if the patch touches a generator or install flow
8792
+
8793
+ **When to skip:** pure doc-only patches, SOP edits, internal refactors that do not touch `scripts/generate_*` or `app/skills/*/SKILL.md`.
8794
+
8795
+ ### 0.1 Run the doctor
8715
8796
 
8716
- **One-liner (copy-paste):**
8717
8797
  ```bash
8718
- python3 scripts/generate_agents_md.py > AGENTS.md && python3 scripts/generate_llms_txt.py > llms.txt && python3 scripts/generate_llms_txt.py --full > llms-full.txt && python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
8798
+ python3 scripts/ecosystem_doctor.py --format text | tee /tmp/eco-report.txt
8719
8799
  ```
8720
8800
 
8721
- ---
8801
+ Output classifies every registered tool as **Clean**, **Drift**, or **Errored**.
8722
8802
 
8723
- ## Phase 1: CLI & Version (1 min)
8803
+ ### 0.2 Act on drift
8804
+
8805
+ For each drifting tool, follow [sop-ecosystem-sync.md](sop-ecosystem-sync.md) Phase 2-4:
8806
+
8807
+ | Drift class | Release impact |
8808
+ |-------------|----------------|
8809
+ | A (cosmetic reword) | No version impact — refresh snapshot, continue |
8810
+ | B (new feature — integrate) | **Minor** version bump at minimum; new generator or extended generator |
8811
+ | C (new feature — not adopted) | No impact — note in registry |
8812
+ | D (deprecation) | **Minor** or **major** depending on user impact; add migration warning |
8813
+ | E (feature promoted to default) | **Minor**; simplify generator, keep fallback comment |
8814
+ | F (feature newly globally available) | **Minor**; may require new generator or new config path |
8815
+
8816
+ If any B/D/E/F changes land in this preparation pass, mention them explicitly in the CHANGELOG entry (Phase 3) under a `Ecosystem` subsection.
8817
+
8818
+ ### 0.3 Refresh snapshot
8819
+
8820
+ Once every drift is resolved (either by code change or by re-classifying as acceptable):
8724
8821
 
8725
8822
  ```bash
8726
- ai-toolkit --version
8727
- ai-toolkit --help
8728
- which ai-toolkit
8823
+ python3 scripts/ecosystem_doctor.py --update
8729
8824
  ```
8730
8825
 
8731
- **Verify:**
8732
- - [ ] `--version` returns correct semver (e.g., `1.4.0`)
8733
- - [ ] `--help` displays full command list without errors
8734
- - [ ] `which` points to global npm bin path
8826
+ This writes the new baseline to `benchmarks/ecosystem-doctor-snapshot.json`. Commit it as part of the release commit.
8827
+
8828
+ ### 0.4 Gate
8829
+
8830
+ ```bash
8831
+ python3 scripts/ecosystem_doctor.py --offline --check
8832
+ ```
8833
+
8834
+ Must exit `0`. If it exits `1`, the snapshot is stale — rerun Phase 0.3 or review the remaining drift.
8735
8835
 
8736
8836
  ---
8737
8837
 
8738
- ## Phase 2: Global Install & Status (2 min)
8838
+ ## Phase 1: Determine Version Bump
8839
+
8840
+ Follow [Semantic Versioning](https://semver.org/):
8841
+
8842
+ | Change Type | Bump | Examples |
8843
+ |-------------|------|---------|
8844
+ | Bugfix, typo, doc-only | **patch** | Fix install flag, correct description |
8845
+ | New feature, skill, agent, flag | **minor** | Add `/hipaa-validate`, add `--output json` |
8846
+ | Breaking CLI change, removed skill, config format change | **major** | Rename `install` to `setup`, remove skill |
8847
+
8848
+ **Rule:** When in doubt, bump minor.
8849
+
8850
+ ---
8851
+
8852
+ ## Phase 2: Sync Version in All Files
8853
+
8854
+ The canonical version lives in `package.json`. These files **must** match:
8855
+
8856
+ ### Mandatory sync (every release)
8857
+
8858
+ | File | Field | How to update |
8859
+ |------|-------|---------------|
8860
+ | `package.json` | `"version": "X.Y.Z"` | Edit directly |
8861
+ | `manifest.json` | `"version": "X.Y.Z"` | Edit directly |
8862
+ | `app/.claude-plugin/plugin.json` | `"version": "X.Y.Z"` | Edit directly |
8863
+
8864
+ ### Auto-synced (no manual action)
8865
+
8866
+ | File | Mechanism |
8867
+ |------|-----------|
8868
+ | `package-lock.json` | Regenerated by `npm install --package-lock-only` |
8869
+
8870
+ ### Conditional sync (only if the doc was modified in this release)
8871
+
8872
+ | File | Field | When to update |
8873
+ |------|-------|---------------|
8874
+ | `kb/procedures/sop-maintenance.md` | frontmatter `version:` | If SOP content changed |
8875
+ | `kb/reference/skills-catalog.md` | frontmatter `version:` | If skills added/removed |
8876
+ | `kb/reference/agents-catalog.md` | frontmatter `version:` | If agents added/removed |
8877
+ | `kb/reference/hooks-catalog.md` | frontmatter `version:` | If hooks changed |
8878
+ | `kb/reference/architecture-overview.md` | frontmatter `version:` | If architecture changed |
8879
+ | `kb/reference/distribution-model.md` | frontmatter `version:` | If install model changed |
8880
+ | `kb/reference/global-install-model.md` | frontmatter `version:` | If install model changed |
8881
+
8882
+ > **Note:** KB `version:` fields track the **document version**, not the toolkit version.
8883
+ > Only bump them when the document content actually changes in this release.
8884
+
8885
+ ### Count sync (if skills/agents/hooks changed)
8886
+
8887
+ | File | What to check |
8888
+ |------|---------------|
8889
+ | `package.json` | `"description"` — skill/agent count |
8890
+ | `README.md` | Badge counts, "What You Get" table |
8891
+ | `app/ARCHITECTURE.md` | Section headings with counts |
8892
+
8893
+ > **Tip:** `validate.py --strict` catches count drift AND version mismatches
8894
+ > (package.json vs manifest.json vs plugin.json) automatically.
8895
+ > If validation passes, counts and versions are correct.
8896
+
8897
+ ### Verification command
8898
+
8899
+ After syncing, verify all mandatory files match:
8739
8900
 
8740
8901
  ```bash
8741
- ai-toolkit install --dry-run
8742
- ai-toolkit status
8902
+ VERSION=$(python3 -c "import json; print(json.load(open('package.json'))['version'])")
8903
+ echo "Target: $VERSION"
8904
+ echo "manifest.json: $(python3 -c "import json; print(json.load(open('manifest.json'))['version'])")"
8905
+ echo "plugin.json: $(python3 -c "import json; print(json.load(open('app/.claude-plugin/plugin.json'))['version'])")"
8906
+ echo "package-lock.json: $(python3 -c "import json; print(json.load(open('package-lock.json'))['version'])")"
8743
8907
  ```
8744
8908
 
8745
- **Verify `--dry-run`:**
8746
- - [ ] Agents: 44
8747
- - [ ] Skills: 108
8748
- - [ ] Hooks merged into settings.json
8749
- - [ ] "Other AI Tools" lists documented global targets (with `--editors`): aider, antigravity, augment, cline, codex, copilot, cursor, gemini, opencode, roo, windsurf. Scope varies: Codex uses `$CODEX_HOME` (default `~/.codex`) plus `$HOME/.agents/skills`; Copilot uses `$COPILOT_HOME` (default `~/.copilot`); cursor has only `~/.cursor/hooks.json`; antigravity has the `~/.gemini/*/skills` pointer. Cursor and Antigravity rules remain project-only.
8909
+ All four must print the same version. If not, fix before proceeding.
8750
8910
 
8751
- **Verify `status`:**
8752
- - [ ] Version matches expected
8753
- - [ ] Profile: minimal/standard/strict
8754
- - [ ] Modules: list of installed modules
8755
- - [ ] Latest: up to date / update available
8911
+ ### Public surface review
8912
+
8913
+ ```bash
8914
+ python3 scripts/surface_manifest.py --update
8915
+ git diff app/surface.json
8916
+ ```
8917
+
8918
+ **Every line the diff removes is a breaking change.** Restore it, or take the
8919
+ deprecation path in `BACKWARD_COMPATIBILITY.md` and add a `DECISIONS.md` entry
8920
+ before the tag. Lines added are new surface being adopted into protection — that
8921
+ needs no ceremony.
8922
+
8923
+ Do not run `--update` to make a red build green. The check fails because something
8924
+ users depend on disappeared; regenerating the manifest deletes the evidence, not
8925
+ the problem.
8926
+
8927
+ ### Skill body budget ratchet
8928
+
8929
+ `validate.py` prints the largest skill body on every run:
8930
+
8931
+ ```
8932
+ Body budget: largest is <skill> at <N> bytes (warn 18000, error 20000)
8933
+ ```
8934
+
8935
+ Once that number sits at least 2000 bytes under `SKILL_BODY_BUDGET_WARN`, lower
8936
+ `SKILL_BODY_BUDGET_WARN` by 2000 in `scripts/validate.py` and ship the tightened
8937
+ threshold with the release. Target floor is 12000.
8756
8938
 
8757
- ---
8939
+ Two rules, both learned the hard way:
8758
8940
 
8759
- ## Phase 3: Doctor Health Check (1 min)
8941
+ - **Never lower a threshold in the same change that something violates it.** Split
8942
+ the offending skill into `SKILL.md` + `reference/` first, prove it with
8943
+ `python3 scripts/check_split.py <skill> --before <pre-split SKILL.md>`, then
8944
+ tighten in a follow-up.
8945
+ - **Never raise a threshold to make a red build green.** A body over budget means
8946
+ detail that belongs in `reference/` is loading on every trigger match, including
8947
+ the accidental ones. Raising the number hides the cost, it does not remove it.
8760
8948
 
8761
- ```bash
8762
- ai-toolkit doctor
8763
- ```
8949
+ If the largest body has not moved since the previous release, leave the threshold
8950
+ alone and say so in the release notes. A ratchet that never advances is honest;
8951
+ one that advances past reality is not.
8764
8952
 
8765
- **Expected sections (all OK):**
8766
- - Environment: node, bash, python3, bats
8767
- - Global Install: .claude exists, agents/skills symlinks (0 broken), settings.json hooks
8768
- - Hook Scripts: all present and executable
8769
- - Hook Configuration: 14 events registered
8770
- - Generated Artifacts: AGENTS.md, llms.txt, llms-full.txt
8771
- - Planned Assets: plugin.json, benchmarks, plugin packs
8772
- - Benchmark Freshness: < 30 days
8773
- - Stale Rules: all healthy
8953
+ ---
8774
8954
 
8775
- **Verify:**
8776
- - [ ] `Errors: 0 | Warnings: 0`
8777
- - [ ] `HEALTH CHECK PASSED`
8955
+ ## Phase 3: Write CHANGELOG Entry
8778
8956
 
8779
- If doctor detects problems: `ai-toolkit doctor --fix` auto-repairs
8780
- (broken symlinks, non-executable hooks, missing scripts, missing llms-full.txt).
8957
+ Add entry at the top of `CHANGELOG.md` (after the header, before previous release):
8781
8958
 
8782
- ---
8959
+ ```markdown
8960
+ ## vX.Y.Z — Short Title (YYYY-MM-DD)
8783
8961
 
8784
- ## Phase 4: Local Install (2 min)
8962
+ ### Added
8963
+ - **Feature name** — description
8785
8964
 
8786
- ```bash
8787
- mkdir -p /tmp/ai-toolkit-verify && cd /tmp/ai-toolkit-verify
8788
- git init -q
8789
- ai-toolkit install --local --editors all --dry-run
8790
- cd - && rm -rf /tmp/ai-toolkit-verify
8791
- ```
8965
+ ### Changed
8966
+ - **What changed** old behavior → new behavior
8792
8967
 
8793
- **Verify "Project-local" section:**
8794
- - [ ] Would create: CLAUDE.md
8795
- - [ ] Would create: .claude/settings.local.json
8796
- - [ ] Would inject: .claude/constitution.md
8797
- - [ ] Editors: all 11 listed (copilot, cursor, windsurf, cline, roo, aider, augment, antigravity, codex, gemini, opencode)
8798
- - [ ] Would generate configs for each editor (legacy + directory-based)
8799
- - [ ] Would install: .git/hooks/pre-commit
8800
- - [ ] Would inject language rules (auto-detected)
8968
+ ### Fixed
8969
+ - **Bug description** what was broken and how it's fixed
8801
8970
 
8802
- **Also test auto-detect (no --editors flag):**
8803
- ```bash
8804
- ai-toolkit install --local --dry-run
8805
- # → Editors: none (empty project has no existing configs)
8971
+ ### Removed
8972
+ - **What was removed** — migration path if any
8806
8973
  ```
8807
8974
 
8808
- ---
8975
+ **Rules:**
8976
+ - Use **bold** for feature names
8977
+ - Start descriptions with a verb (Added, Changed, Fixed, Removed)
8978
+ - Reference skill names with backticks and slash: `/hipaa-validate`
8979
+ - Include script names: `scripts/hipaa_scan.py`
8980
+ - Include count changes: `Skill count: 91 → 92`
8981
+ - Date format: `YYYY-MM-DD`
8982
+ - Title: short, descriptive, no version number repetition
8809
8983
 
8810
- ## Phase 5: Validation & Security Audit (3 min)
8984
+ ### Update README "What's New" section
8811
8985
 
8812
- ```bash
8813
- python3 scripts/validate.py --strict
8814
- python3 scripts/audit_skills.py --ci
8815
- ```
8986
+ **MANDATORY on every release.** Update the `## What's New in vX.Y.Z` section in `README.md`:
8816
8987
 
8817
- **Verify validate.py:**
8818
- - [ ] Agents: 44, Skills: 108, Tests: exactly the current README badge count
8819
- - [ ] Hook events: 14, Hook scripts: >= 30
8820
- - [ ] Plugin packs >= 10, KB documents >= 20
8821
- - [ ] `Errors: 0 | Warnings: 0` → `VALIDATION PASSED`
8988
+ 1. Change the heading version: `## What's New in vX.Y.Z`
8989
+ 2. Replace bullet points with 3-5 highlights from this release
8990
+ 3. **Keep only the latest version block.** Delete the previous `## What's New in vA.B.C` section(s). README is the shop window, not the archive — users see the current release, full history lives in `CHANGELOG.md`.
8991
+ 4. Keep the `See [CHANGELOG.md](CHANGELOG.md) for full history.` link directly below the bullet list.
8822
8992
 
8823
- **Verify audit_skills.py:**
8824
- - [ ] `HIGH: 0` (MUST be zero CI fails otherwise)
8825
- - [ ] `WARN: 0`
8826
- - [ ] `INFO: N` (acceptable — broad-access skills: orchestrate, swarm, workflow)
8993
+ > **Warning:** This section is the first thing users see after the badges.
8994
+ > A stale version here (e.g., "What's New in v2.1.3" when shipping v2.3.0)
8995
+ > signals an unmaintained project. Do NOT skip this step.
8827
8996
 
8828
- ---
8997
+ > **Single-version rule:** README.md must contain **exactly one** `## What's New in vX.Y.Z` heading at any time. If you find multiple stacked (e.g. v2.6.1 + v2.6.0 + v2.5.0), that is a SOP drift — collapse to the latest on the next release commit.
8829
8998
 
8830
- ## Phase 6: Tests (3-5 min)
8999
+ ---
8831
9000
 
8832
- ```bash
8833
- # Run ONCE, capture to file, then parse. Full suite is 669+ bats cases —
8834
- # re-running it per check (tail / grep ok / grep not ok piped separately)
8835
- # wastes minutes every release. Always cache the output.
8836
- npm test > /tmp/npm-test.log 2>&1
8837
- exit=$?
8838
- tail -3 /tmp/npm-test.log
8839
- echo "ok: $(grep -c '^ok ' /tmp/npm-test.log)"
8840
- echo "not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8841
- echo "exit: $exit"
8842
- ```
9001
+ ## Phase 4: Regenerate Artifacts
8843
9002
 
8844
- **Verify:**
8845
- - [ ] `exit == 0`
8846
- - [ ] `ok == expected test count` (e.g., 945 on v3.0.0)
8847
- - [ ] `not ok == 0`
8848
- - [ ] Bats runs tests in parallel (4 jobs)
8849
- - [ ] Groups: agents, autodetect, cli, generators, guards, hooks, inject,
8850
- install, kb, mcp, readme, profiles, uninstall, validate
9003
+ Use the npm scripts, not the generators directly:
8851
9004
 
8852
- **Anti-pattern — do NOT do this:**
8853
9005
  ```bash
8854
- # Runs the full suite THREE times. Adds 1-3 min and pressures CI capacity.
8855
- npm test 2>&1 | tail -3
8856
- npm test 2>&1 | grep -c '^ok '
8857
- npm test 2>&1 | grep -c '^not ok'
9006
+ npm run generate:agents # AI_TOOLKIT_NO_CUSTOM_RULES=1 python3 scripts/generate_agents_md.py > AGENTS.md
9007
+ npm run generate:llms # llms.txt + llms-full.txt
8858
9008
  ```
8859
9009
 
8860
- **Key test areas:**
8861
- - Guards: rm -rf, DROP TABLE, git push --force blocked
8862
- - Install: idempotent, profiles, --only/--skip, orphan cleanup
8863
- - Eject: real files (not symlinks), inlined rules
8864
- - Uninstall: removes toolkit, preserves user content
8865
-
8866
- ---
9010
+ `generate:agents` sets `AI_TOOLKIT_NO_CUSTOM_RULES=1`. Running
9011
+ `generate_agents_md.py` bare picks up whatever is registered in the maintainer's
9012
+ own `~/.softspark/ai-toolkit/rules/`, which then ships inside `AGENTS.md`.
8867
9013
 
8868
- ## Phase 7: Eject (1 min)
9014
+ Check if anything actually changed:
8869
9015
 
8870
9016
  ```bash
8871
- mkdir -p /tmp/ai-toolkit-eject-test
8872
- cd /tmp/ai-toolkit-eject-test
8873
- ai-toolkit eject
8874
- cd - && rm -rf /tmp/ai-toolkit-eject-test
9017
+ git diff --stat AGENTS.md llms.txt llms-full.txt
8875
9018
  ```
8876
9019
 
8877
- **Verify:**
8878
- - [ ] Agents copied as real files (not symlinks)
8879
- - [ ] Skills copied as real directories
8880
- - [ ] Rules inlined into CLAUDE.md
8881
- - [ ] constitution.md and ARCHITECTURE.md copied
8882
- - [ ] `output-styles/` directory present (v2.7.1+)
9020
+ If no diff, the artifacts are already current. If there is a diff, stage them.
8883
9021
 
8884
9022
  ---
8885
9023
 
8886
- ## Phase 8: Supply-Chain Verification (2 min, v2.8.0+)
8887
-
8888
- These checks enforce the v2.8.0 security standard on a freshly-published release. Run AFTER the `publish.yml` workflow completes on the tag.
9024
+ ## Phase 5: Validate, Audit, Test
8889
9025
 
8890
- ### 8.1 Provenance attestation on npm
9026
+ Run the full quality gate:
8891
9027
 
8892
9028
  ```bash
8893
- VERSION="X.Y.Z" # the tag just published
8894
- npm view "@softspark/ai-toolkit@${VERSION}" --json \
8895
- | python3 -c "import json, sys; d=json.load(sys.stdin); att=d['dist'].get('attestations', {}); assert att.get('provenance', {}).get('predicateType') == 'https://slsa.dev/provenance/v1', f'NO PROVENANCE for {d[\"version\"]}'; print(f'PROVENANCE OK: {att[\"url\"]}')"
8896
- ```
9029
+ python3 scripts/validate.py --strict
9030
+ python3 scripts/audit_skills.py --ci
9031
+ python3 scripts/audit_skills.py --sarif > audit.sarif # MANDATORY GHAS ingest
9032
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
8897
9033
 
8898
- **Verify:**
8899
- - [ ] Exit 0 and prints `PROVENANCE OK: https://registry.npmjs.org/...`
8900
- - [ ] `https://www.npmjs.com/package/@softspark/ai-toolkit/v/${VERSION}` shows the green "Provenance" badge
9034
+ # ShellCheck on hooks (added in 1.11.0). Mirrors the ci.yml "ShellCheck hooks"
9035
+ # job. NOT run by validate.py, npm test, OR publish.yml — so a hook with a
9036
+ # ShellCheck warning passes every other gate AND still publishes on tag while
9037
+ # turning main CI red. Run it here, before tagging.
9038
+ shellcheck --severity=warning app/hooks/*.sh && echo "OK: shellcheck clean"
8901
9039
 
8902
- **If provenance is missing:** the `publish.yml` ran without `id-token: write` or `--provenance`. Restore them and cut a patch release — a silently unsigned publish is a regression against the v2.8.0 standard.
9040
+ # Registry / generator drift (added in 1.10.0). Meta-generators excluded.
9041
+ META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
9042
+ diff \
9043
+ <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) \
9044
+ <(ls scripts/generate_*.py | grep -vE "$META" | sort -u) \
9045
+ && echo "OK: registry matches filesystem" \
9046
+ || { echo "DRIFT: update supported-tools-registry.md before tagging"; exit 1; }
8903
9047
 
8904
- ### 8.2 Audit SARIF output (for GHAS ingest)
9048
+ # Stage first IF this release adds or deletes a kb/ file. The test
9049
+ # "npm package KB files match the tracked release set" compares `git ls-files
9050
+ # kb` (the index) against what `npm pack` sees (the working tree), so an
9051
+ # unstaged addition reads as "extra" and an unstaged deletion as "missing".
9052
+ # Phase 6 stages, and it runs after this one, so the ordering fails the test
9053
+ # for any release that touches the KB. Staging early costs nothing.
9054
+ git status --porcelain kb/ | grep -qE '^(\?\?| D|\?M)' && git add -A kb/
8905
9055
 
8906
- ```bash
8907
- python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif
8908
- python3 -c "import json; d=json.load(open('/tmp/audit.sarif')); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']=='ai-toolkit-audit-skills'; print(f'SARIF OK: {len(d[\"runs\"][0][\"results\"])} results across {len(d[\"runs\"][0][\"tool\"][\"driver\"][\"rules\"])} rules')"
9056
+ # Run npm test ONCE, cache output, parse from file. The suite is 1400+ bats
9057
+ # cases — rerunning it per check wastes minutes. Do not pipe npm test into
9058
+ # tail/grep multiple times in the same session.
9059
+ npm test > /tmp/npm-test.log 2>&1
9060
+ tail -3 /tmp/npm-test.log
9061
+ echo "ok: $(grep -c '^ok ' /tmp/npm-test.log) | not ok: $(grep -c '^not ok' /tmp/npm-test.log)"
8909
9062
  ```
8910
9063
 
8911
- **Verify:**
8912
- - [ ] Valid SARIF 2.1.0
8913
- - [ ] In the publishing repo, the CI job uploads `audit.sarif` via `github/codeql-action/upload-sarif@v3` so findings appear in the Security tab
9064
+ **Expected results:**
9065
+ - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
9066
+ - `audit_skills.py --ci`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
9067
+ - `audit_skills.py --sarif`: valid JSON, non-empty `runs[0].tool.driver.rules`
9068
+ - `audit_skills.py --permissions`: review `Skills with Bash + Write + Edit` list — any newly-added skill with broad access MUST be justified in the CHANGELOG entry
9069
+ - `shellcheck --severity=warning app/hooks/*.sh`: no output, exit 0. A common false positive is `SC2034` on `INPUT` or env vars (e.g. `AI_TOOLKIT_HOOK_FORMAT`) that a *sourced* helper (`_hook-io.sh`) consumes — ShellCheck cannot see cross-file use. Fix with a `# shellcheck disable=SC2034` directive or `export`, matching `guard-destructive.sh`. Never tag with a red ShellCheck.
9070
+ - Registry drift: `OK: registry matches filesystem`. If `DRIFT:` appears, add the missing `scripts/generate_*.py` rows to `kb/reference/supported-tools-registry.md` before tagging.
9071
+ - `npm test`: `1..N` with zero `not ok` (read from the cached `/tmp/npm-test.log`, do not rerun)
8914
9072
 
8915
- ### 8.3 Per-skill permissions report
9073
+ > **Why this matters (v4.5.1 postmortem):** `publish.yml` runs only `validate.py` + `npm test`, so it published v4.5.0 even though the `main` CI `ShellCheck hooks` job was red on two `SC2034` warnings in a new hook. The publish workflow does **not** depend on the CI workflow. Until that is fixed, ShellCheck is a manual pre-tag gate — run it here every time.
8916
9074
 
9075
+ **One-liner:**
8917
9076
  ```bash
8918
- python3 scripts/audit_skills.py --permissions | head -40
9077
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > audit.sarif && shellcheck --severity=warning app/hooks/*.sh && diff <(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u) <(ls scripts/generate_*.py | grep -vE 'generate_agents_md\.py|generate_llms_txt\.py|generate_language_rules_skills\.py' | sort -u) && npm test
8919
9078
  ```
8920
9079
 
8921
- **Verify:**
8922
- - [ ] Bash skill count has NOT jumped unexpectedly since the previous release
8923
- - [ ] Any newly added entry under `Skills with Bash + Write + Edit` matches a CHANGELOG bullet that justifies the broad scope
8924
- - [ ] JSON form (`--permissions --json`) is available for automated drift dashboards
8925
-
8926
- ### 8.4 URL-sourced rules/hooks are checksum-pinned
8927
-
8928
- `sources.json` is an envelope of the form `{"schema_version": 1, "rules"|"hooks": {...}}`, so the jq filter must pick the nested map before piping into the pin assertion. The URL-count in the success message ignores local `path:`-only entries — only entries with a `url:` field are pinned and counted.
8929
-
8930
- ```bash
8931
- jq '.rules // .hooks // {}' ~/.softspark/ai-toolkit/rules/sources.json 2>/dev/null \
8932
- | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'RULE PIN OK: {url_n} URL rules, all with sha256')"
8933
- jq '.hooks // {}' ~/.softspark/ai-toolkit/hooks/external/sources.json 2>/dev/null \
8934
- | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; url_n=sum(1 for v in d.values() if v.get('url')); print(f'HOOK PIN OK: {url_n} URL hooks, all with sha256')"
8935
- ```
9080
+ **If tests fail:** Fix the issue, do NOT skip. Common failures:
9081
+ - Stale counts re-run `generate:all` or fix README/ARCHITECTURE
9082
+ - Missing frontmatter add to new KB docs
9083
+ - Broken symlink `ai-toolkit doctor --fix`
8936
9084
 
8937
- **Verify:**
8938
- - [ ] Both commands print `... PIN OK`
8939
- - [ ] If any entry is unpinned, the `register_url_source()` call missed passing `content=` — fix the call-site and retag
9085
+ ### Phase 5c: Licensing Gate (v4.20.0+)
8940
9086
 
8941
- ### 8.5 Strict-pin smoke test (optional but recommended)
9087
+ The project is Apache-2.0. Attribution only works if the artefact actually
9088
+ carries it, and every part of that is mechanically checkable.
8942
9089
 
8943
9090
  ```bash
8944
- AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
9091
+ # The whole gate, enforced in CI. Run it here so a failure is caught before tagging.
9092
+ npx bats tests/test_licensing.bats
8945
9093
  ```
8946
9094
 
8947
- **Verify:**
8948
- - [ ] Exit 0, no `CHECKSUM CHANGED` line
8949
- - [ ] If a checksum change was intentional (e.g. upstream rule update), document it in the CHANGELOG entry before tagging
9095
+ The seven assertions, and why each exists:
8950
9096
 
8951
- ---
9097
+ | Check | Fails when |
9098
+ |---|---|
9099
+ | Every shipped source file carries an SPDX header | A new `.py`/`.sh`/`.js`/`.bats` file was added without one — the common case, and the reason this is a test rather than a habit |
9100
+ | Headers name Apache-2.0 and nothing else | A file was copied in from an MIT/GPL source with its own header intact |
9101
+ | **No** markdown file carries an SPDX header | Someone "helpfully" ran the header script over `app/skills/` — headers there sit above parsed frontmatter and bill every session for it |
9102
+ | `LICENSE` is the complete Apache 2.0 text | The file was truncated or replaced with a summary |
9103
+ | `NOTICE` carries attribution, the source URL, §4(d) and the MIT-era notice | The attribution mechanism was gutted |
9104
+ | `LICENSE` **and** `NOTICE` ship in the npm package | `package.json` `files` lost an entry — a NOTICE that never reaches the consumer cannot satisfy §4(d) |
9105
+ | Every manifest declaring a licence declares Apache-2.0 | `package.json`, `manifest.json`, `plugin.json` and `package-lock.json` drifted apart |
8952
9106
 
8953
- ## Phase 9: Deep-Coverage Checks (v3.0.0+)
9107
+ **Adding source files in this release?** The header goes *after* the shebang,
9108
+ never before it. Short SPDX form:
8954
9109
 
8955
- These verify the native-surface generators shipped in v3.0.0 actually emit the right files for the right profiles, and that the tool registry stays in sync with shipped generators.
9110
+ ```
9111
+ # SPDX-License-Identifier: Apache-2.0
9112
+ # Copyright 2024-2026 Lukasz Krzemien (biuro@softspark.eu)
9113
+ # Source: https://github.com/softspark/ai-toolkit
9114
+ ```
8956
9115
 
8957
- > **Safety warning — HOME-scoped writes:** Running `--profile full` with `augment` in the editor list writes to `$HOME/.augment/settings.json` (Augment stores hooks under HOME, not per-project). Use `--dry-run` for verification unless you intend to carry ai-toolkit hook entries on this machine. The generator is marker-safe (only rewrites its own `_source: ai-toolkit` entries) but is still a side-effect.
9116
+ `//` for JavaScript. Full convention and the reasoning behind the markdown
9117
+ exclusion: [Licensing](../reference/licensing.md).
8958
9118
 
8959
- ### 9.1 `--profile full` emits every native surface
9119
+ **Changing the licence itself?** Do not hand-type the licence text. Take it
9120
+ verbatim from a published copy and cross-verify against a second independent
9121
+ copy before writing `LICENSE` — a rendered or summarised licence is not the
9122
+ licence. Prior releases stay under their original terms; a licence change
9123
+ applies going forward and revokes nothing already granted.
8960
9124
 
8961
- ```bash
8962
- D=/tmp/aitk-profile-full-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8963
- ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot \
8964
- --profile full --dry-run 2>&1 \
8965
- | grep -E "\\.cursor/(hooks\\.json|agents)|\\.devin/hooks\\.v1\\.json|\\.gemini/(settings\\.json|commands)|\\.augment/(agents|commands)|\\.codex/(agents|hooks)|\\.agents/skills|\\.github/(agents|skills|instructions|prompts|hooks)"
8966
- ```
9125
+ ### Phase 5a: Supply-Chain Hardening Verification (v2.8.0+)
8967
9126
 
8968
- **Verify** at least the following lines appear:
8969
- - [ ] `.cursor/hooks.json` and `.cursor/agents/`
8970
- - [ ] `.devin/hooks.v1.json`
8971
- - [ ] `.gemini/settings.json` hooks AND `.gemini/commands/`
8972
- - [ ] `.augment/agents/` + `.augment/commands/` + `$HOME/.augment/settings.json`
8973
- - [ ] `.codex/hooks.json` + `.codex/hooks/` + `.codex/agents/`
8974
- - [ ] `.agents/skills/` (Codex native discovery path)
8975
- - [ ] `.github/agents/` + `.github/skills/` + `.github/instructions/` + `.github/prompts/` + `.github/hooks/`
9127
+ These checks enforce the security standard introduced in v2.8.0. Do NOT tag a release until all pass.
8976
9128
 
8977
- ### 9.2 Codex skills are profile-independent
9129
+ **1. Publish workflow emits provenance:**
8978
9130
 
8979
9131
  ```bash
8980
- D=/tmp/aitk-codex-skills-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
8981
- ai-toolkit install --local --editors codex --profile minimal --dry-run 2>&1 \
8982
- | grep -q "Would generate: .agents/skills" && echo "OK: Codex skills emit at minimal"
8983
- ai-toolkit install --local --editors codex --profile standard --codex-skills --dry-run 2>&1 \
8984
- | grep -q "Would refresh: .agents/skills" && echo "OK: legacy flag explicitly refreshes the same catalog"
9132
+ grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
8985
9133
  ```
8986
9134
 
8987
- **Verify:**
8988
- - [ ] Selecting Codex emits all skills under `.agents/skills/` at every profile
8989
- - [ ] `--codex-skills` remains an explicit refresh alias; it is not required for installation
8990
- - [ ] No profile emits `.codex/skills/`
8991
-
8992
- ### 9.3 Breaking-change surfaces land on `--profile standard`
9135
+ - [ ] Both markers present (`--provenance` flag + `id-token: write` permission)
9136
+ - [ ] Any PR that changes `publish.yml` REQUIRES an approved security review
8993
9137
 
8994
- The standard profile must include:
8995
- - Copilot scoped instructions, prompts, custom agents, portable skills, and hooks
8996
- - Gemini hooks (`.gemini/settings.json`)
9138
+ **2. URL-sourced rules and hooks are checksum-pinned:**
8997
9139
 
8998
9140
  ```bash
8999
- D=/tmp/aitk-breaking-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9000
- ai-toolkit install --local --editors copilot,gemini --profile standard --dry-run 2>&1 \
9001
- | tee /tmp/aitk-breaking.log
9002
- grep -q "\\.github/instructions/" /tmp/aitk-breaking.log && echo "OK: Copilot dir layout at standard"
9003
- grep -q "\\.github/agents/.*\\.github/skills/" /tmp/aitk-breaking.log && echo "OK: Copilot agents and skills at standard"
9004
- grep -q "\\.github/hooks/" /tmp/aitk-breaking.log && echo "OK: Copilot hooks at standard"
9005
- grep -q "\\.gemini/settings\\.json hooks" /tmp/aitk-breaking.log && echo "OK: Gemini hooks at standard"
9141
+ # On a machine that has consumed URL rules/hooks at least once
9142
+ # (schema_version 1: entries live under the .rules / .hooks key):
9143
+ jq '.rules | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
9144
+ jq '.hooks | to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
9006
9145
  ```
9007
9146
 
9008
- **Verify all four lines print `OK:`**. If any is missing, native Copilot or Gemini coverage has regressed.
9147
+ - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
9148
+ - [ ] If not, run `ai-toolkit update` to backfill missing hashes before tagging
9009
9149
 
9010
- ### 9.4 Install is idempotent
9150
+ **3. Audit SARIF output is well-formed:**
9011
9151
 
9012
9152
  ```bash
9013
- D=/tmp/aitk-idem-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9014
- # Sort file paths before hashing — find traversal order follows inode order,
9015
- # which can shift between runs even when content is byte-identical, producing
9016
- # false FAIL signals.
9017
- ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
9018
- SHA1=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
9019
- ai-toolkit install --local --editors cursor,gemini --profile full >/dev/null 2>&1
9020
- SHA2=$(find .cursor .gemini -type f -print0 | LC_ALL=C sort -z | xargs -0 shasum | shasum | awk '{print $1}')
9021
- [ "$SHA1" = "$SHA2" ] && echo "OK: idempotent" || echo "FAIL: install is not idempotent"
9153
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json, sys; d=json.load(sys.stdin); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']; print('SARIF OK')"
9022
9154
  ```
9023
9155
 
9024
- **Verify:** prints `OK: idempotent`. A second run must produce byte-identical files in every managed path.
9025
-
9026
- ### 9.5 Live-install JSON outputs parse
9156
+ - [ ] Prints `SARIF OK`
9157
+ - [ ] If the script ever grows new rule classes, extend the SARIF `rules[]` coverage before releasing
9027
9158
 
9028
- The bats suite validates JSON shape at generation time. This re-checks that what actually landed on disk after a live install parses without errors.
9159
+ **4. Strict-pin mode passes on CI** (optional, recommended for stable branches):
9029
9160
 
9030
9161
  ```bash
9031
- D=/tmp/aitk-json-${RANDOM} && mkdir -p "$D" && cd "$D" && git init -q
9032
- ai-toolkit install --local --editors cursor,windsurf,gemini,augment,codex,copilot --profile full >/dev/null 2>&1
9033
- for f in .cursor/hooks.json .devin/hooks.v1.json .gemini/settings.json .codex/hooks.json .github/hooks/ai-toolkit.json "$HOME/.augment/settings.json"; do
9034
- [ -f "$f" ] && python3 -c "import json; json.load(open('$f'))" && echo "OK: $f"
9035
- done
9162
+ AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
9036
9163
  ```
9037
9164
 
9038
- **Verify:** each emitted file prints `OK: <path>`. Any `json.decoder.JSONDecodeError` means the merge logic corrupted the output.
9165
+ - [ ] Exit 0, no `CHECKSUM CHANGED` line
9166
+ - [ ] Any unexpected upstream change blocks the release until explicitly approved
9039
9167
 
9040
- ### 9.6 Registry / generator drift check
9168
+ ---
9041
9169
 
9042
- `kb/reference/supported-tools-registry.md` should enumerate every per-editor `scripts/generate_*.py` we ship. Meta-generators (`generate_agents_md.py`, `generate_llms_txt.py`) are excluded — they produce docs/artifacts, not editor configs.
9170
+ ## Phase 6: Commit
9171
+
9172
+ Stage all release files:
9043
9173
 
9044
9174
  ```bash
9045
- META="generate_agents_md.py|generate_llms_txt.py|generate_language_rules_skills.py"
9046
- REG=$(grep -oE 'scripts/generate_[a-z_]+\.py' kb/reference/supported-tools-registry.md | sort -u)
9047
- FS=$(ls scripts/generate_*.py | grep -vE "$META" | sort -u)
9048
- diff <(echo "$REG") <(echo "$FS") && echo "OK: registry matches filesystem" || echo "DRIFT: update supported-tools-registry.md"
9175
+ git add package.json manifest.json app/.claude-plugin/plugin.json
9176
+ git add package-lock.json
9177
+ git add CHANGELOG.md
9178
+ git add llms.txt llms-full.txt
9179
+ # NOTE: AGENTS.md, GEMINI.md, and .github/copilot-instructions.md are generated
9180
+ # editor configs and are gitignored — do NOT commit them. `prepublishOnly` runs
9181
+ # `npm run generate:all`, so the shipped package (which lists AGENTS.md in
9182
+ # package.json `files`) gets a fresh copy at publish time.
9183
+ git add -p # review and stage any other changes
9049
9184
  ```
9050
9185
 
9051
- **Verify:** prints `OK: registry matches filesystem`. If not, add the missing rows to the registry before tagging the next release.
9052
-
9053
- ### 9.7 Claude Chat / Cowork plugin validates and exports
9186
+ Commit:
9054
9187
 
9055
9188
  ```bash
9056
- python3 scripts/claude_app.py verify
9057
- D=/tmp/aitk-claude-app-${RANDOM}
9058
- python3 scripts/claude_app.py export --output "$D.zip" --no-custom-rules
9059
- python3 -c "import zipfile; z=zipfile.ZipFile('$D.zip'); assert '.claude-plugin/plugin.json' in z.namelist(); assert 'claude-app/skills/ai-toolkit-rules/SKILL.md' in z.namelist(); print('OK: Claude app archive')"
9189
+ git commit -m "chore: release vX.Y.Z"
9060
9190
  ```
9061
9191
 
9062
- **Verify:** the official validator exits 0; the archive contains the manifest,
9063
- app-native rules skill, bundled agents/skills, and plugin-relative Cowork hooks.
9064
-
9065
9192
  ---
9066
9193
 
9067
- ## Phase 10: Explicit DSH Qualification (v4.30.0+)
9068
-
9069
- Run this phase whenever the release changes the `dsh` target, package pins, preset lifecycle, or DSH compatibility documentation. Use a new task-specific `DSH_HOME`; never replace `HOME` or reuse a regular profile.
9070
-
9071
- Prerequisites: DSH `0.1.1-rc.2`, pnpm `>=11.7.0,<12.0.0`, Codex logged in through ChatGPT, Claude Code logged in natively, and GitHub Copilot CLI logged in natively. Do not supply provider API keys.
9194
+ ## Phase 7: Push Branch, Verify CI, Tag, and Push Tag
9072
9195
 
9073
9196
  ```bash
9074
- DSH_SMOKE_ROOT="$(mktemp -d)"
9075
- export DSH_HOME="$DSH_SMOKE_ROOT/dsh-home"
9076
-
9077
- ai-toolkit dsh install --profile web
9078
- ai-toolkit dsh doctor --profile web
9079
- dsh --profile web --host 127.0.0.1 --port 0 --no-open
9080
- ```
9197
+ RELEASE_SHA=$(git rev-parse HEAD)
9198
+ git push origin main || { echo "FAIL: release commit push failed"; exit 1; }
9199
+
9200
+ # Bind the gate to the exact release commit. GitHub run registration is
9201
+ # asynchronous, so retry at most five times and log every attempt.
9202
+ RUN_ID=""
9203
+ for ATTEMPT in 1 2 3 4 5; do
9204
+ RUN_ID=$(gh run list --workflow ci.yml --event push --commit "$RELEASE_SHA" \
9205
+ --limit 1 --json databaseId --jq '.[0].databaseId')
9206
+ [ -n "$RUN_ID" ] && break
9207
+ echo "Waiting for CI run registration ($ATTEMPT/5)" >&2
9208
+ [ "$ATTEMPT" -eq 5 ] || sleep 60
9209
+ done
9210
+ test -n "$RUN_ID" || { echo "FAIL: CI run not found for release commit"; exit 1; }
9211
+ gh run watch "$RUN_ID" --exit-status \
9212
+ || { echo "FAIL: release commit CI failed"; exit 1; }
9213
+
9214
+ # publish.yml is Ubuntu-only. Require both full CI matrix jobs before creating
9215
+ # the tag so a green publish job cannot bypass a red macOS test.
9216
+ for OS in ubuntu-latest macos-latest; do
9217
+ gh run view "$RUN_ID" --json jobs \
9218
+ --jq ".jobs[] | select(.name == \"Bats test suite ($OS)\") | .conclusion" \
9219
+ | grep -qx success || { echo "FAIL: $OS test job not green"; exit 1; }
9220
+ done
9081
9221
 
9082
- In a new `softspark-orchestrator` session, select the `codex` provider and run two standalone marker prompts:
9222
+ # Refuse to tag if either local or remote main moved while CI was running.
9223
+ test "$(git rev-parse HEAD)" = "$RELEASE_SHA" \
9224
+ || { echo "FAIL: local HEAD changed during CI"; exit 1; }
9225
+ git fetch origin main || { echo "FAIL: origin/main refresh failed"; exit 1; }
9226
+ test "$(git rev-parse origin/main)" = "$RELEASE_SHA" \
9227
+ || { echo "FAIL: origin/main changed during CI"; exit 1; }
9083
9228
 
9084
- 1. `subagent_claude_code` returns an exact child marker, then the Codex parent returns its exact completion marker.
9085
- 2. `subagent_gemini_copilot` returns an exact child marker, then the Codex parent returns its exact completion marker.
9229
+ git tag vX.Y.Z "$RELEASE_SHA"
9086
9230
 
9087
- Stop DSH, then remove only the managed profile artifacts:
9231
+ # Assert the tag before pushing it. Both checks are one line each and both
9232
+ # have caught a real broken release.
9233
+ test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA" \
9234
+ || { echo "FAIL: tag is not on the tested release commit"; exit 1; }
9235
+ git show --no-patch --format=%s vX.Y.Z | grep -qx "chore: release vX.Y.Z" \
9236
+ || { echo "FAIL: tag is not on the chore: release commit"; exit 1; }
9088
9237
 
9089
- ```bash
9090
- ai-toolkit dsh uninstall --profile web --yes
9238
+ # Push the single release tag by its full ref.
9239
+ git push origin refs/tags/vX.Y.Z
9091
9240
  ```
9092
9241
 
9093
- **Verify:** both tool results have `isError=false`, both turns end as `completed`, `doctor` reports no recovery requirement before uninstall, and an unrelated preset fixture remains unchanged. Preserve only redacted event sequence evidence; never attach credentials, auth files, or full private prompts.
9094
-
9095
- ---
9242
+ **Why branch CI comes before the tag (v4.30.2 postmortem).** The publish
9243
+ workflow runs only on Ubuntu and can publish while the separate macOS matrix job
9244
+ is red. `v4.30.2` exposed this with a macOS-only DSH timing failure. A release
9245
+ commit must therefore pass the complete Ubuntu/macOS branch workflow before its
9246
+ tag exists; a successful publish workflow is not a substitute for green CI.
9096
9247
 
9097
- ## Troubleshooting
9248
+ **Why the assertions (v4.19.0 postmortem).** v4.19.0 was tagged on a commit
9249
+ that contained only a KB document and still carried `package.json` version
9250
+ `4.18.0`; the actual release sat in the commit above it under a recycled
9251
+ `fix:` message. `publish.yml` fired, tried to publish a version already on
9252
+ npm, and failed. Nothing on npm, a tag pointing at the wrong tree, and the
9253
+ only way out was rewriting a pushed commit. Both assertions above catch this
9254
+ in under a second. Run them.
9098
9255
 
9099
- ### `ai-toolkit: command not found`
9256
+ **Never `git push --tags`.** It pushes every local tag at once, and GitHub
9257
+ suppresses tag-triggered workflow runs when many tags arrive in a single push
9258
+ — the workflow silently does not fire and nothing publishes. Push the single
9259
+ release tag by its full ref, as above. (Sibling evidence: this is exactly how
9260
+ rag-mcp's `1.0.3` image build was skipped, when a `--tags` push carried 37
9261
+ tags at once.)
9100
9262
 
9101
- ```bash
9102
- npm install -g @softspark/ai-toolkit
9103
- # or check PATH:
9104
- export PATH="$(npm config get prefix)/bin:$PATH"
9105
- ```
9263
+ This triggers `.github/workflows/publish.yml` which:
9264
+ 1. Runs `validate.py --strict`
9265
+ 2. Runs `npm test`
9266
+ 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z` with `--provenance` (SLSA v1 build attestation)
9106
9267
 
9107
- ### Doctor: broken symlinks
9268
+ **Provenance is non-negotiable.** If `id-token: write` permission or the `--provenance` flag is missing from `publish.yml`, fix it BEFORE tagging — an unsigned release is a regression against the v2.8.0 standard.
9108
9269
 
9109
- ```bash
9110
- ai-toolkit doctor --fix # auto-repair
9111
- ai-toolkit update # or full re-install
9112
- ```
9270
+ **After CI completes:** Run the [Release Verification SOP](sop-release-verification.md)
9271
+ to smoke-test the published package AND verify the provenance attestation landed on npm.
9113
9272
 
9114
- ### Tests fail: missing bats
9273
+ ---
9115
9274
 
9116
- ```bash
9117
- brew install bats-core # macOS
9118
- npm install -g bats # cross-platform
9119
- ```
9275
+ ## Rollback
9120
9276
 
9121
- ### validate.py: stale counts
9277
+ If a bad release was published:
9122
9278
 
9123
- README badges don't match the current agents/skills/tests counts.
9124
- Update README.md and re-run.
9279
+ ```bash
9280
+ # Unpublish from npm (within 72h)
9281
+ npm unpublish @softspark/ai-toolkit@X.Y.Z
9125
9282
 
9126
- ### Eject: missing skills
9283
+ # Or deprecate (preferred — doesn't break existing installs)
9284
+ npm deprecate @softspark/ai-toolkit@X.Y.Z "Known issue: <description>. Use vA.B.C instead."
9127
9285
 
9128
- ```bash
9129
- ai-toolkit update # re-link missing symlinks
9130
- ai-toolkit eject /tmp/test # retry
9286
+ # Delete tag
9287
+ git tag -d vX.Y.Z
9288
+ git push origin --delete vX.Y.Z
9131
9289
  ```
9132
9290
 
9133
9291
  ---
9134
9292
 
9135
- ## Success Criteria
9293
+ ## Checklist Summary
9136
9294
 
9137
- | Area | Criterion |
9138
- |------|-----------|
9139
- | CLI | `--version` correct, `--help` full list, `status` current |
9140
- | Health | `doctor`: 0 errors, 0 warnings, PASSED |
9141
- | Install | `--dry-run` correct counts, `--local` all configs |
9142
- | Quality | `validate.py --strict`: PASSED |
9143
- | Security (baseline) | `audit_skills.py --ci`: 0 HIGH |
9144
- | Security (SARIF) | `audit_skills.py --sarif`: valid SARIF 2.1.0 with non-empty rules array |
9145
- | Security (permissions) | `audit_skills.py --permissions`: broad-access skills unchanged or justified in CHANGELOG |
9146
- | Supply chain | `dist.attestations.provenance.predicateType == https://slsa.dev/provenance/v1` on npm |
9147
- | Supply chain | All `sources.json` URL entries carry a `sha256`; `AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run` passes |
9148
- | Tests | `npm test`: N/N passed, 0 failures |
9149
- | Eject | Standalone `.claude/` with real files AND `output-styles/` directory |
9150
- | Guards | Destructive commands blocked |
9151
- | Deep coverage | `--profile full` emits every documented native surface, including Codex agents/skills/hooks and Copilot agents/skills/hooks |
9152
- | Profile contract | Codex skills emit at every profile without `--codex-skills`; Copilot standard emits agents, skills, instructions, prompts, and hooks |
9153
- | Idempotence | Second `install` run produces byte-identical output in every managed path |
9154
- | Live JSON | Every generated `.json` file on disk parses as valid JSON |
9155
- | Registry | `supported-tools-registry.md` enumerates every `scripts/generate_*.py` we ship |
9295
+ | # | Step | Command / Action | Pass Criteria |
9296
+ |---|------|-----------------|---------------|
9297
+ | 0a | Ecosystem drift check | `ecosystem_doctor.py --format text` | All tools Clean, or drift classified and resolved |
9298
+ | 0b | Ecosystem snapshot refresh | `ecosystem_doctor.py --update` | `benchmarks/ecosystem-doctor-snapshot.json` updated |
9299
+ | 0c | Ecosystem gate | `ecosystem_doctor.py --offline --check` | Exit 0 |
9300
+ | 1 | Version bump type | Decide patch/minor/major | — |
9301
+ | 2 | `package.json` version | Edit `"version"` | Matches target |
9302
+ | 3 | `manifest.json` version | Edit `"version"` | Matches target |
9303
+ | 4 | `plugin.json` version | Edit `"version"` | Matches target |
9304
+ | 5 | `package-lock.json` | `npm install --package-lock-only` | Matches target |
9305
+ | 6 | Count sync | Check `package.json` description, README | `validate.py` passes |
9306
+ | 7 | CHANGELOG.md | Add release entry (incl. `Ecosystem` subsection if any B/D/E/F drift) | Entry exists for vX.Y.Z |
9307
+ | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_llms_txt.py` | No unexpected diff |
9308
+ | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
9309
+ | 10 | Security audit (CI mode) | `audit_skills.py --ci` | 0 HIGH |
9310
+ | 11 | Security audit (SARIF) | `audit_skills.py --sarif` | Valid SARIF 2.1.0 JSON |
9311
+ | 12 | Per-skill permissions | `audit_skills.py --permissions` | New broad-access skills justified in CHANGELOG |
9312
+ | 13 | ShellCheck hooks | `shellcheck --severity=warning app/hooks/*.sh` | Exit 0, no output (mirrors ci.yml; publish.yml does NOT run it) |
9313
+ | 14 | Provenance flag check | `grep -- '--provenance' .github/workflows/publish.yml` | Present |
9314
+ | 15 | Checksum-pin backfill | `sources.json` entries all have `sha256` | No unpinned URL sources |
9315
+ | 15a | Licensing gate | `npx bats tests/test_licensing.bats` | 7/7 — SPDX headers, LICENSE, NOTICE, npm `files`, manifest consistency |
9316
+ | 16 | Tests | `git add -A kb/` if the KB changed, then `npm test` | All pass |
9317
+ | 17 | Commit | `git commit` | Clean working tree |
9318
+ | 18 | Push branch | `git push origin main` | Exact release commit is on `origin/main` |
9319
+ | 18a | Full branch CI | `gh run watch "$RUN_ID" --exit-status` plus matrix job assertions | Ubuntu and macOS Bats jobs both conclude `success` |
9320
+ | 19 | Tag | `git tag vX.Y.Z` | Tag exists only after green branch CI |
9321
+ | 19a | Tag is on tested SHA | `test "$(git rev-parse vX.Y.Z)" = "$RELEASE_SHA"` | Exit 0 |
9322
+ | 19b | Tag is on the release commit | `git show --no-patch --format=%s vX.Y.Z` | Reads `chore: release vX.Y.Z` |
9323
+ | 20 | Push the single tag | `git push origin refs/tags/vX.Y.Z` | Publish CI triggered with `id-token: write`. Never `--tags`. |
9156
9324
 
9157
9325
  ---
9158
9326
 
@@ -9740,7 +9908,7 @@ Three tiers determine how to approach a task:
9740
9908
  |------|-------|-----------|-------|
9741
9909
  | Task | `disable-model-invocation: true` | User via `/skill` only | 32 |
9742
9910
  | Hybrid | (neither) | User via `/skill` + agent knowledge | 31 |
9743
- | Knowledge | `user-invocable: false` | Claude auto-loads | 46 |
9911
+ | Knowledge | `user-invocable: false` | Claude auto-loads | 51 |
9744
9912
 
9745
9913
  ## Multi-Agent Execution
9746
9914
 
@@ -11677,7 +11845,7 @@ running the generators directly:
11677
11845
 
11678
11846
  - `scripts/generate_copilot.py` / `generate_copilot_hooks.py` contract tests
11679
11847
  (`tests/test_copilot.bats`, `tests/test_copilot_hooks.bats`).
11680
- - Release layout check in `kb/procedures/release-verification-sop.md` asserts the
11848
+ - Release layout check in `kb/procedures/sop-release-verification.md` asserts the
11681
11849
  `.github/{agents,skills,instructions,prompts,hooks}` surfaces.
11682
11850
  - `validate.py --strict` + `audit_skills.py --ci` in CI.
11683
11851
 
@@ -14252,7 +14420,7 @@ MIT remnant would need each contributor's agreement.
14252
14420
 
14253
14421
  It is a test rather than a checklist line on purpose. This project has two
14254
14422
  same-day postmortems about SOPs that existed and were skipped; CI does not skip.
14255
- [Release Preparation](../procedures/release-preparation-sop.md) Phase 5c runs the
14423
+ [Release Preparation](../procedures/sop-release.md) Phase 5c runs the
14256
14424
  same gate before tagging so a failure surfaces before the tag, not after.
14257
14425
 
14258
14426
  ## If the licence ever changes again
@@ -14265,7 +14433,7 @@ source, cross-verify against a second independent copy, and only then write
14265
14433
  ## Related
14266
14434
 
14267
14435
  - [`LICENSE`](../../LICENSE), [`NOTICE`](../../NOTICE)
14268
- - [Release Preparation SOP](../procedures/release-preparation-sop.md) — Phase 5c
14436
+ - [Release Preparation SOP](../procedures/sop-release.md) — Phase 5c
14269
14437
  - [Distribution Model](distribution-model.md) — what ships and where
14270
14438
 
14271
14439
  ---
@@ -15945,7 +16113,7 @@ Updated baseline docs:
15945
16113
  - `kb/reference/skills-catalog.md`
15946
16114
  - `kb/reference/plugin-pack-conventions.md`
15947
16115
  - `kb/reference/claude-ecosystem-benchmark-snapshot.md`
15948
- - `kb/procedures/maintenance-sop.md`
16116
+ - `kb/procedures/sop-maintenance.md`
15949
16117
 
15950
16118
  ## Validation Evidence
15951
16119
 
@@ -16799,13 +16967,13 @@ See [DSH Compatibility](./dsh-compatibility.md) for commands, topology, subscrip
16799
16967
  2. Delete its section from this document.
16800
16968
  3. Delete its snapshot entry from `benchmarks/ecosystem-doctor-snapshot.json` (or let the next `--update` prune it — currently not pruned automatically).
16801
16969
  4. Decide whether to keep the generator (`scripts/generate_<tool>_*.py`) for backwards compatibility or delete it.
16802
- 5. Remove references from `README.md`, `manifest.json` `description` field, and `kb/procedures/maintenance-sop.md` `Supported editors` line.
16970
+ 5. Remove references from `README.md`, `manifest.json` `description` field, and `kb/procedures/sop-maintenance.md` `Supported editors` line.
16803
16971
 
16804
16972
  ---
16805
16973
 
16806
16974
  ## Related
16807
16975
 
16808
- - [Ecosystem Sync SOP](../procedures/ecosystem-sync-sop.md) — how to use the doctor
16976
+ - [Ecosystem Sync SOP](../procedures/sop-ecosystem-sync.md) — how to use the doctor
16809
16977
  - [MCP Editor Compatibility](./mcp-editor-compatibility.md) — MCP-specific subset
16810
16978
  - `scripts/ecosystem_tools.json` — source of truth
16811
16979
  - `scripts/ecosystem_doctor.py` — drift detector
@@ -17338,7 +17506,7 @@ Expect `Total plugin skills loaded: 0`, `Total plugin agents loaded: 0`, and the
17338
17506
  ## Related
17339
17507
 
17340
17508
  - `kb/reference/global-install-model.md`
17341
- - `kb/procedures/maintenance-sop.md`
17509
+ - `kb/procedures/sop-maintenance.md`
17342
17510
 
17343
17511
  ---
17344
17512