@softspark/ai-toolkit 2.8.0 → 2.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/AGENTS.md +31 -31
  2. package/CHANGELOG.md +29 -1
  3. package/README.md +4 -5
  4. package/app/.claude-plugin/plugin.json +1 -1
  5. package/app/agents/system-governor.md +98 -8
  6. package/app/constitution.md +7 -1
  7. package/app/rules/common/coding-style.md +12 -2
  8. package/app/skills/api-patterns/SKILL.md +1 -1
  9. package/app/skills/app-builder/SKILL.md +1 -1
  10. package/app/skills/architecture-decision/SKILL.md +1 -1
  11. package/app/skills/ci-cd-patterns/SKILL.md +1 -1
  12. package/app/skills/clean-code/SKILL.md +6 -1
  13. package/app/skills/csharp-patterns/SKILL.md +1 -1
  14. package/app/skills/database-patterns/SKILL.md +1 -1
  15. package/app/skills/debugging-tactics/SKILL.md +1 -1
  16. package/app/skills/design-engineering/SKILL.md +1 -1
  17. package/app/skills/docker-devops/SKILL.md +1 -1
  18. package/app/skills/documentation-standards/SKILL.md +1 -1
  19. package/app/skills/ecommerce-patterns/SKILL.md +1 -1
  20. package/app/skills/flutter-patterns/SKILL.md +1 -1
  21. package/app/skills/git-mastery/SKILL.md +1 -1
  22. package/app/skills/hive-mind/SKILL.md +1 -1
  23. package/app/skills/java-patterns/SKILL.md +1 -1
  24. package/app/skills/kotlin-patterns/SKILL.md +1 -1
  25. package/app/skills/mcp-patterns/SKILL.md +1 -1
  26. package/app/skills/migration-patterns/SKILL.md +1 -1
  27. package/app/skills/observability-patterns/SKILL.md +1 -1
  28. package/app/skills/performance-profiling/SKILL.md +1 -1
  29. package/app/skills/plan-writing/SKILL.md +1 -1
  30. package/app/skills/rag-patterns/SKILL.md +1 -1
  31. package/app/skills/refactor-plan/SKILL.md +1 -0
  32. package/app/skills/research-mastery/SKILL.md +1 -1
  33. package/app/skills/ruby-patterns/SKILL.md +1 -1
  34. package/app/skills/rust-patterns/SKILL.md +1 -1
  35. package/app/skills/security-patterns/SKILL.md +1 -1
  36. package/app/skills/swift-patterns/SKILL.md +1 -1
  37. package/app/skills/testing-patterns/SKILL.md +1 -1
  38. package/app/skills/typescript-patterns/SKILL.md +1 -1
  39. package/app/skills/verification-before-completion/SKILL.md +7 -0
  40. package/kb/procedures/release-preparation-sop.md +72 -15
  41. package/kb/procedures/release-verification-sop.md +82 -7
  42. package/llms-full.txt +184 -52
  43. package/manifest.json +1 -1
  44. package/package.json +1 -1
  45. package/scripts/audit_skills.py +68 -0
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: java-patterns
3
- description: "Loaded when user asks about Java development patterns"
3
+ description: "Java development patterns: Spring Boot, CompletableFuture, records, sealed types, streams, JPA/Hibernate, Maven/Gradle, virtual threads (Loom). Triggers: Java, Spring, Spring Boot, JPA, Hibernate, Maven, Gradle, CompletableFuture, record type, sealed class, virtual thread. Load when writing or reviewing Java code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: kotlin-patterns
3
- description: "Loaded when user asks about Kotlin development patterns"
3
+ description: "Kotlin development patterns: coroutines, Flow, sealed classes, data classes, extension functions, null safety, Ktor, Jetpack Compose, KMP. Triggers: Kotlin, coroutine, Flow, suspend, Ktor, Android Kotlin, Jetpack Compose, sealed class, data class, KMP, kotlinx. Load when writing or reviewing Kotlin code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: mcp-patterns
3
- description: "Loaded when user asks about MCP servers or tool protocol design"
3
+ description: "MCP (Model Context Protocol) server design: tool schemas, resource patterns, transport selection (stdio/SSE), client configuration, error handling, capability negotiation. Triggers: MCP, Model Context Protocol, MCP server, MCP tool, MCP resource, JSON-RPC, stdio transport, SSE transport, Claude Desktop config, Cursor MCP. Load when building or integrating MCP servers."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: migration-patterns
3
- description: "Loaded when user asks about database migrations or zero-downtime deploys"
3
+ description: "Zero-downtime database migration patterns: expand-contract, double-write, backfill, blue-green schema changes, feature flags, rollback safety, online DDL. Triggers: migration, schema change, zero-downtime, expand-contract, double-write, backfill, ALTER TABLE, column rename, safe deploy, online DDL. Load when planning non-trivial DB schema changes."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: observability-patterns
3
- description: "Loaded when user asks about logging, metrics, or tracing patterns"
3
+ description: "Observability: structured logging, metrics (RED/USE/four golden signals), distributed tracing (OpenTelemetry), correlation IDs, log aggregation, SLO/SLI. Triggers: logging, log level, metrics, Prometheus, Grafana, OpenTelemetry, trace, span, structured log, observability, monitoring, SLO, SLI, alerting. Load when adding or reviewing logs, metrics, or traces."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: performance-profiling
3
- description: "Loaded when user asks about performance profiling or optimization"
3
+ description: "Performance measurement and optimization: four golden signals (latency/traffic/errors/saturation), p50/p95/p99, baseline-change-measure loop, flame graphs, load testing. Triggers: performance, slow, latency, p99, flame graph, profile, bottleneck, optimization, load test, benchmark, CPU profiling, memory leak. Load when diagnosing or optimizing slow code or services."
4
4
  effort: medium
5
5
  allowed-tools: Read, Grep
6
6
  user-invocable: false
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: plan-writing
3
- description: "Loaded when user asks to write an implementation plan or pre-mortem"
3
+ description: "Implementation plan and pre-mortem drafting: phase breakdown, success criteria, risks, rollback plan, acceptance tests, estimated effort. Triggers: implementation plan, pre-mortem, phased plan, project plan, task breakdown, success criteria, rollback strategy, risk register. Load when user asks to write a plan or pre-mortem document."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: rag-patterns
3
- description: "Loaded when user asks about RAG systems, embeddings, or vector search"
3
+ description: "RAG architecture: embeddings, chunking strategies, hybrid search (BM25 + vector), reranking, CRAG/self-correcting, multi-hop reasoning, evaluation metrics. Triggers: RAG, embedding, vector search, pgvector, Qdrant, Pinecone, Weaviate, chunking, reranker, retrieval, hybrid search, semantic search, knowledge base, cosine similarity. Load when building or tuning RAG systems."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -110,3 +110,4 @@ What is explicitly NOT part of this refactor.
110
110
  - No file paths or code snippets in the issue (durability)
111
111
  - File immediately via `gh issue create` — don't ask for review
112
112
  - Interview thoroughly before planning
113
+ - **Dead code cleanup is mandatory per step, not deferred** (Constitution Art. VI.1): every refactor step must leave the repo with zero orphaned references. "We'll delete the old code in a later step" is only acceptable for transitional double-write / expand-contract phases where both paths are temporarily live — and the cleanup step must be explicitly listed in the plan, not implied.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: research-mastery
3
- description: "Loaded when user asks to research, verify, or synthesize information"
3
+ description: "Hierarchical information retrieval following strict order: KB first (smart_query/crag_search), then MCP/Context7, then web search, then LLM knowledge as last resort. Triggers: research, find information, verify fact, synthesize sources, fact-check, cross-reference, multi-source, cite sources, investigate topic. Load when any task requires external or cross-source knowledge."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: ruby-patterns
3
- description: "Loaded when user asks about Ruby development patterns"
3
+ description: "Ruby and Rails development patterns: blocks, metaprogramming, ActiveRecord, Sidekiq, RSpec, Sorbet/RBS, Hanami, Roda, Rack middleware. Triggers: Ruby, Rails, ActiveRecord, Sidekiq, RSpec, gem, Gemfile, bundler, rake, Hanami, Sorbet. Load when writing or reviewing Ruby code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: rust-patterns
3
- description: "Loaded when user asks about Rust development patterns"
3
+ description: "Rust development patterns: ownership, borrowing, lifetimes, async (Tokio), error handling (Result/anyhow/thiserror), traits, macros, zero-cost abstractions, unsafe boundaries. Triggers: Rust, borrow checker, lifetime, Tokio, async Rust, cargo, trait, impl, Result, unsafe, lifetime annotation, clippy. Load when writing or reviewing Rust code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: security-patterns
3
- description: "Loaded when user asks about security, OWASP, or auth patterns"
3
+ description: "Application security: OWASP Top 10, authN/authZ, input validation, secrets management, TLS, CSRF/XSS/SQLi, session handling, JWT, rate limiting, CSP. Triggers: security, OWASP, auth, JWT, CSRF, XSS, SQL injection, secrets, encryption, TLS, authentication, authorization, CSP, CORS, password hashing. Load when touching auth code, handling user input, or doing security review."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: swift-patterns
3
- description: "Loaded when user asks about Swift or iOS development patterns"
3
+ description: "Swift and iOS development patterns: SwiftUI, Combine, async/await, property wrappers, actors, Swift Package Manager, Core Data, UIKit interop, @MainActor. Triggers: Swift, SwiftUI, Combine, iOS, Xcode, actor, property wrapper, Core Data, SPM, UIKit, @MainActor, @State, @Binding. Load when writing or reviewing Swift code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: testing-patterns
3
- description: "Loaded when user asks about testing strategy, fixtures, or mocking"
3
+ description: "Testing strategy and craft: pyramid vs trophy, unit/integration/e2e split, fixtures, mocks vs fakes vs stubs, AAA pattern, flaky test diagnosis, coverage goals, property-based testing. Triggers: test, testing strategy, fixture, mock, stub, AAA, unit test, integration test, e2e, Playwright, Cypress, flaky, coverage, TDD, test pyramid. Load when writing, reviewing, or designing test suites."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: typescript-patterns
3
- description: "Loaded when user asks about TypeScript patterns or type safety"
3
+ description: "TypeScript type safety patterns: strict mode, generics, conditional types, template literals, discriminated unions, branded types, Zod, satisfies operator, const assertions. Triggers: TypeScript, TS, generics, conditional type, utility type, strict, Zod, satisfies, discriminated union, type safety, type narrowing, template literal type. Load when writing or reviewing TypeScript code."
4
4
  effort: medium
5
5
  user-invocable: false
6
6
  allowed-tools: Read
@@ -54,6 +54,9 @@ Skip any step = lying, not verifying
54
54
  | Regression test works | Red-green cycle verified | Test passes once |
55
55
  | Agent completed | VCS diff shows changes | Agent reports "success" |
56
56
  | Requirements met | Line-by-line checklist | Tests passing |
57
+ | No dead code (Art. VI.1) | Grep for every removed/renamed symbol: 0 references | "I cleaned up what I touched" |
58
+ | Behavior change covered (Art. VI.2) | Integration test for the API surface + unit test + docs updated | Unit test on the helper only |
59
+ | Diff is clean (Art. VI.4) | Re-read full diff: no orphaned imports, no stale docs, no skipped fixes | "I only changed what I needed" |
57
60
 
58
61
  ## Red Flags — STOP
59
62
 
@@ -102,6 +105,10 @@ CORRECT: Agent reports success → Check VCS diff → Verify changes → Report
102
105
  WRONG: Trust agent report at face value
103
106
  ```
104
107
 
108
+ ## Constitutional Anchors
109
+
110
+ This skill enforces **Constitution Art. VI.4 (Verify Before Claiming Done)**. The diff re-read is not optional: before any completion claim, confirm no orphaned references, no missing test coverage for changed paths, no stale docs. A task is not done while any of those exist.
111
+
105
112
  ## The Bottom Line
106
113
 
107
114
  Run the command. Read the output. THEN claim the result.
@@ -2,11 +2,11 @@
2
2
  title: "SOP: Release Preparation"
3
3
  category: procedures
4
4
  service: ai-toolkit
5
- tags: [sop, release, version, publish, changelog, semver]
6
- version: "1.6.0"
5
+ tags: [sop, release, version, publish, changelog, semver, provenance, sarif]
6
+ version: "1.7.0"
7
7
  created: "2026-04-10"
8
- last_updated: "2026-04-17"
9
- description: "Step-by-step checklist for preparing a new ai-toolkit release — version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag."
8
+ last_updated: "2026-04-18"
9
+ description: "Step-by-step checklist for preparing a new ai-toolkit release — version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag. Includes mandatory Provenance, SARIF, and checksum-pin checks added in v2.8.0."
10
10
  ---
11
11
 
12
12
  # SOP: Release Preparation
@@ -42,8 +42,13 @@ python3 scripts/generate_codex_rules.py .
42
42
  python3 scripts/generate_llms_txt.py > llms.txt
43
43
  python3 scripts/generate_llms_txt.py --full > llms-full.txt
44
44
 
45
- # 5. Validate + audit + test
46
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
45
+ # 5. Validate + audit + SARIF + test
46
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif && npm test
47
+
48
+ # 5a. Supply-chain standard (v2.8.0+) — non-negotiable
49
+ grep -q -- '--provenance' .github/workflows/publish.yml || { echo "MISSING --provenance"; exit 1; }
50
+ grep -q 'id-token: write' .github/workflows/publish.yml || { echo "MISSING id-token: write"; exit 1; }
51
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
47
52
 
48
53
  # 6. Commit + tag + push
49
54
  git add -A && git commit -m "chore: release vX.Y.Z"
@@ -200,17 +205,21 @@ Run the full quality gate:
200
205
  ```bash
201
206
  python3 scripts/validate.py --strict
202
207
  python3 scripts/audit_skills.py --ci
208
+ python3 scripts/audit_skills.py --sarif > audit.sarif # MANDATORY — GHAS ingest
209
+ python3 scripts/audit_skills.py --permissions # review Bash/Write/Edit footprint
203
210
  npm test
204
211
  ```
205
212
 
206
213
  **Expected results:**
207
214
  - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
208
- - `audit_skills.py`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
215
+ - `audit_skills.py --ci`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
216
+ - `audit_skills.py --sarif`: valid JSON, non-empty `runs[0].tool.driver.rules`
217
+ - `audit_skills.py --permissions`: review `Skills with Bash + Write + Edit` list — any newly-added skill with broad access MUST be justified in the CHANGELOG entry
209
218
  - `npm test`: `1..N` with zero `not ok`
210
219
 
211
220
  **One-liner:**
212
221
  ```bash
213
- python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
222
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && python3 scripts/audit_skills.py --sarif > audit.sarif && npm test
214
223
  ```
215
224
 
216
225
  **If tests fail:** Fix the issue, do NOT skip. Common failures:
@@ -218,6 +227,48 @@ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci &&
218
227
  - Missing frontmatter → add to new KB docs
219
228
  - Broken symlink → `ai-toolkit doctor --fix`
220
229
 
230
+ ### Phase 5a: Supply-Chain Hardening Verification (v2.8.0+)
231
+
232
+ These checks enforce the security standard introduced in v2.8.0. Do NOT tag a release until all pass.
233
+
234
+ **1. Publish workflow emits provenance:**
235
+
236
+ ```bash
237
+ grep -E '\-\-provenance|id-token: write' .github/workflows/publish.yml
238
+ ```
239
+
240
+ - [ ] Both markers present (`--provenance` flag + `id-token: write` permission)
241
+ - [ ] Any PR that changes `publish.yml` REQUIRES an approved security review
242
+
243
+ **2. URL-sourced rules and hooks are checksum-pinned:**
244
+
245
+ ```bash
246
+ # On a machine that has consumed URL rules/hooks at least once:
247
+ jq 'to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/rules/sources.json
248
+ jq 'to_entries | map(select(.value.url != null and (.value.sha256 // "" | length) == 0))' ~/.softspark/ai-toolkit/hooks/external/sources.json
249
+ ```
250
+
251
+ - [ ] Both queries return empty arrays (every URL entry has a `sha256`)
252
+ - [ ] If not, run `ai-toolkit update` to backfill missing hashes before tagging
253
+
254
+ **3. Audit SARIF output is well-formed:**
255
+
256
+ ```bash
257
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json, sys; d=json.load(sys.stdin); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']; print('SARIF OK')"
258
+ ```
259
+
260
+ - [ ] Prints `SARIF OK`
261
+ - [ ] If the script ever grows new rule classes, extend the SARIF `rules[]` coverage before releasing
262
+
263
+ **4. Strict-pin mode passes on CI** (optional, recommended for stable branches):
264
+
265
+ ```bash
266
+ AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
267
+ ```
268
+
269
+ - [ ] Exit 0, no `CHECKSUM CHANGED` line
270
+ - [ ] Any unexpected upstream change blocks the release until explicitly approved
271
+
221
272
  ---
222
273
 
223
274
  ## Phase 6: Commit
@@ -250,10 +301,12 @@ git push origin main --tags
250
301
  This triggers `.github/workflows/publish.yml` which:
251
302
  1. Runs `validate.py --strict`
252
303
  2. Runs `npm test`
253
- 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z`
304
+ 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z` with `--provenance` (SLSA v1 build attestation)
305
+
306
+ **Provenance is non-negotiable.** If `id-token: write` permission or the `--provenance` flag is missing from `publish.yml`, fix it BEFORE tagging — an unsigned release is a regression against the v2.8.0 standard.
254
307
 
255
308
  **After CI completes:** Run the [Release Verification SOP](release-verification-sop.md)
256
- to smoke-test the published package.
309
+ to smoke-test the published package AND verify the provenance attestation landed on npm.
257
310
 
258
311
  ---
259
312
 
@@ -288,8 +341,12 @@ git push origin --delete vX.Y.Z
288
341
  | 7 | CHANGELOG.md | Add release entry | Entry exists for vX.Y.Z |
289
342
  | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_codex_rules.py`, `generate_llms_txt.py` | No unexpected diff |
290
343
  | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
291
- | 10 | Security audit | `audit_skills.py --ci` | 0 HIGH |
292
- | 11 | Tests | `npm test` | All pass |
293
- | 12 | Commit | `git commit` | Clean working tree |
294
- | 13 | Tag | `git tag vX.Y.Z` | Tag exists |
295
- | 14 | Push | `git push origin main --tags` | CI triggered |
344
+ | 10 | Security audit (CI mode) | `audit_skills.py --ci` | 0 HIGH |
345
+ | 11 | Security audit (SARIF) | `audit_skills.py --sarif` | Valid SARIF 2.1.0 JSON |
346
+ | 12 | Per-skill permissions | `audit_skills.py --permissions` | New broad-access skills justified in CHANGELOG |
347
+ | 13 | Provenance flag check | `grep -- '--provenance' .github/workflows/publish.yml` | Present |
348
+ | 14 | Checksum-pin backfill | `sources.json` entries all have `sha256` | No unpinned URL sources |
349
+ | 15 | Tests | `npm test` | All pass |
350
+ | 16 | Commit | `git commit` | Clean working tree |
351
+ | 17 | Tag | `git tag vX.Y.Z` | Tag exists |
352
+ | 18 | Push | `git push origin main --tags` | CI triggered with `id-token: write` |
@@ -2,11 +2,11 @@
2
2
  title: "SOP: Release Verification"
3
3
  category: procedures
4
4
  service: ai-toolkit
5
- tags: [sop, verification, release, smoke-test, install, update, qa]
6
- version: "1.1.0"
5
+ tags: [sop, verification, release, smoke-test, install, update, qa, provenance, sarif]
6
+ version: "1.2.0"
7
7
  created: "2026-04-08"
8
- last_updated: "2026-04-13"
9
- description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit — verifies CLI, install, doctor, validation, tests, and eject from user perspective."
8
+ last_updated: "2026-04-18"
9
+ description: "End-to-end smoke test after installing or updating @softspark/ai-toolkit — verifies CLI, install, doctor, validation, tests, eject, npm provenance attestation, SARIF audit, and per-skill permissions. Reflects the v2.8.0 supply-chain standard."
10
10
  ---
11
11
 
12
12
  # SOP: Release Verification
@@ -31,7 +31,7 @@ Verifies all critical paths from the user's perspective.
31
31
 
32
32
  ## Quick Checklist (TL;DR)
33
33
 
34
- 10 commands — if all pass, the release is ready:
34
+ 13 commands — if all pass, the release is ready:
35
35
 
36
36
  ```bash
37
37
  # Pre-commit (Phase 0)
@@ -47,6 +47,11 @@ ai-toolkit status # 7. Status OK?
47
47
  ai-toolkit doctor # 8. Health check passed?
48
48
  ai-toolkit install --dry-run # 9. Global install OK?
49
49
  python3 scripts/audit_skills.py --ci # 10. Security audit clean?
50
+
51
+ # Supply-chain verification (Phase 8, v2.8.0+)
52
+ python3 scripts/audit_skills.py --sarif | python3 -c "import json,sys; assert json.load(sys.stdin)['version']=='2.1.0'; print('SARIF OK')" # 11. SARIF 2.1.0 well-formed?
53
+ python3 scripts/audit_skills.py --permissions | head -30 # 12. Broad-access skills reviewed?
54
+ npm view @softspark/ai-toolkit@X.Y.Z --json | python3 -c "import json,sys; d=json.load(sys.stdin); assert d['dist']['attestations']['provenance']['predicateType']=='https://slsa.dev/provenance/v1'; print('PROVENANCE OK')" # 13. Provenance attested on npm?
50
55
  ```
51
56
 
52
57
  ---
@@ -230,6 +235,72 @@ cd - && rm -rf /tmp/ai-toolkit-eject-test
230
235
  - [ ] Skills copied as real directories
231
236
  - [ ] Rules inlined into CLAUDE.md
232
237
  - [ ] constitution.md and ARCHITECTURE.md copied
238
+ - [ ] `output-styles/` directory present (v2.7.1+)
239
+
240
+ ---
241
+
242
+ ## Phase 8: Supply-Chain Verification (2 min, v2.8.0+)
243
+
244
+ These checks enforce the v2.8.0 security standard on a freshly-published release. Run AFTER the `publish.yml` workflow completes on the tag.
245
+
246
+ ### 8.1 Provenance attestation on npm
247
+
248
+ ```bash
249
+ VERSION="X.Y.Z" # the tag just published
250
+ npm view "@softspark/ai-toolkit@${VERSION}" --json \
251
+ | python3 -c "import json, sys; d=json.load(sys.stdin); att=d['dist'].get('attestations', {}); assert att.get('provenance', {}).get('predicateType') == 'https://slsa.dev/provenance/v1', f'NO PROVENANCE for {d[\"version\"]}'; print(f'PROVENANCE OK: {att[\"url\"]}')"
252
+ ```
253
+
254
+ **Verify:**
255
+ - [ ] Exit 0 and prints `PROVENANCE OK: https://registry.npmjs.org/...`
256
+ - [ ] `https://www.npmjs.com/package/@softspark/ai-toolkit/v/${VERSION}` shows the green "Provenance" badge
257
+
258
+ **If provenance is missing:** the `publish.yml` ran without `id-token: write` or `--provenance`. Restore them and cut a patch release — a silently unsigned publish is a regression against the v2.8.0 standard.
259
+
260
+ ### 8.2 Audit SARIF output (for GHAS ingest)
261
+
262
+ ```bash
263
+ python3 scripts/audit_skills.py --sarif > /tmp/audit.sarif
264
+ python3 -c "import json; d=json.load(open('/tmp/audit.sarif')); assert d['version']=='2.1.0' and d['runs'][0]['tool']['driver']['name']=='ai-toolkit-audit-skills'; print(f'SARIF OK: {len(d[\"runs\"][0][\"results\"])} results across {len(d[\"runs\"][0][\"tool\"][\"driver\"][\"rules\"])} rules')"
265
+ ```
266
+
267
+ **Verify:**
268
+ - [ ] Valid SARIF 2.1.0
269
+ - [ ] In the publishing repo, the CI job uploads `audit.sarif` via `github/codeql-action/upload-sarif@v3` so findings appear in the Security tab
270
+
271
+ ### 8.3 Per-skill permissions report
272
+
273
+ ```bash
274
+ python3 scripts/audit_skills.py --permissions | head -40
275
+ ```
276
+
277
+ **Verify:**
278
+ - [ ] Bash skill count has NOT jumped unexpectedly since the previous release
279
+ - [ ] Any newly added entry under `Skills with Bash + Write + Edit` matches a CHANGELOG bullet that justifies the broad scope
280
+ - [ ] JSON form (`--permissions --json`) is available for automated drift dashboards
281
+
282
+ ### 8.4 URL-sourced rules/hooks are checksum-pinned
283
+
284
+ ```bash
285
+ jq '.rules // .hooks // {}' ~/.softspark/ai-toolkit/rules/sources.json 2>/dev/null \
286
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; print(f'RULE PIN OK: {len(d)} URL rules, all with sha256')"
287
+ jq '.hooks // {}' ~/.softspark/ai-toolkit/hooks/external/sources.json 2>/dev/null \
288
+ | python3 -c "import json, sys; d=json.load(sys.stdin) or {}; bad=[n for n,v in d.items() if v.get('url') and not v.get('sha256')]; assert not bad, f'UNPINNED: {bad}'; print(f'HOOK PIN OK: {len(d)} URL hooks, all with sha256')"
289
+ ```
290
+
291
+ **Verify:**
292
+ - [ ] Both commands print `... PIN OK`
293
+ - [ ] If any entry is unpinned, the `register_url_source()` call missed passing `content=` — fix the call-site and retag
294
+
295
+ ### 8.5 Strict-pin smoke test (optional but recommended)
296
+
297
+ ```bash
298
+ AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run
299
+ ```
300
+
301
+ **Verify:**
302
+ - [ ] Exit 0, no `CHECKSUM CHANGED` line
303
+ - [ ] If a checksum change was intentional (e.g. upstream rule update), document it in the CHANGELOG entry before tagging
233
304
 
234
305
  ---
235
306
 
@@ -279,7 +350,11 @@ ai-toolkit eject /tmp/test # retry
279
350
  | Health | `doctor`: 0 errors, 0 warnings, PASSED |
280
351
  | Install | `--dry-run` correct counts, `--local` all configs |
281
352
  | Quality | `validate.py --strict`: PASSED |
282
- | Security | `audit_skills.py --ci`: 0 HIGH |
353
+ | Security (baseline) | `audit_skills.py --ci`: 0 HIGH |
354
+ | Security (SARIF) | `audit_skills.py --sarif`: valid SARIF 2.1.0 with non-empty rules array |
355
+ | Security (permissions) | `audit_skills.py --permissions`: broad-access skills unchanged or justified in CHANGELOG |
356
+ | Supply chain | `dist.attestations.provenance.predicateType == https://slsa.dev/provenance/v1` on npm |
357
+ | Supply chain | All `sources.json` URL entries carry a `sha256`; `AI_TOOLKIT_STRICT_PIN=1 ai-toolkit update --dry-run` passes |
283
358
  | Tests | `npm test`: N/N passed, 0 failures |
284
- | Eject | Standalone .claude/ with real files |
359
+ | Eject | Standalone `.claude/` with real files AND `output-styles/` directory |
285
360
  | Guards | Destructive commands blocked |