@softspark/ai-toolkit 1.5.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/llms-full.txt CHANGED
@@ -14,7 +14,12 @@
14
14
  - [Best Practices](kb/best-practices/README.md)
15
15
  - [No Hardcoded Counts in Secondary Docs](kb/best-practices/no-hardcoded-counts.md)
16
16
  - [How-To Guides](kb/howto/README.md)
17
+ - [Plan: Cloud Security Pack — Multi-Cloud Audit](kb/planning/cloud-security-pack-plan.md)
18
+ - [Plan: Enterprise Config Inheritance — Multi-Repo Governance with `extends`](kb/planning/enterprise-config-inheritance-plan.md)
19
+ - [Plan: Local Dashboard — `ai-toolkit ui`](kb/planning/local-dashboard-plan.md)
20
+ - [Plan: Offline-First SLM Profile — Lightweight Mode for Local Models](kb/planning/offline-slm-profile-plan.md)
17
21
  - [SOP: Claude Toolkit Maintenance](kb/procedures/maintenance-sop.md)
22
+ - [SOP: Release Preparation](kb/procedures/release-preparation-sop.md)
18
23
  - [SOP: Release Verification](kb/procedures/release-verification-sop.md)
19
24
  - [Agents Catalog](kb/reference/agents-catalog.md)
20
25
  - [Anti-Pattern Registry Format](kb/reference/anti-pattern-registry-format.md)
@@ -266,6 +271,3624 @@ Step-by-step guides for common tasks. Guides will be added here as they are crea
266
271
 
267
272
  ---
268
273
 
274
+ ## kb/planning/cloud-security-pack-plan.md
275
+
276
+ ---
277
+ title: "Plan: Cloud Security Pack — Multi-Cloud Audit (GCP/AWS/Azure)"
278
+ category: planning
279
+ service: ai-toolkit
280
+ tags:
281
+ - cloud-security
282
+ - gcp
283
+ - aws
284
+ - azure
285
+ - firebase
286
+ - plugin-pack
287
+ - security-audit
288
+ - credentials
289
+ doc_type: plan
290
+ status: proposed
291
+ created: "2026-04-10"
292
+ last_updated: "2026-04-10"
293
+ completion: "0%"
294
+ council_review: "2026-04-10 — conditional FOR, scope reduction recommended"
295
+ description: "Plugin pack for deterministic, read-only security auditing of GCP (Firebase/Cloud Functions), AWS (S3/Lambda/IAM), and Azure (NSG/Functions/CosmosDB). Includes CLI credential management, static+live modes, false positive resolution, SARIF output, incremental scanning, and CI integration."
296
+ ---
297
+
298
+ # Plan: Cloud Security Pack — Multi-Cloud Audit
299
+
300
+ **Status:** Proposed
301
+ **Completion:** 0%
302
+ **Created:** 2026-04-10
303
+ **Origin:** Firebase RTDB/Firestore rules audit, Cloud Functions public exposure, false positive resolution for App Check/Gateway patterns
304
+ **Estimated Effort:** 5-6 weeks (council-revised from original 3-4 weeks)
305
+
306
+ ---
307
+
308
+ ## 1. Objective
309
+
310
+ Create `cloud-security-pack` plugin pack that provides deterministic, read-only security auditing for three major cloud providers. All scripts are stdlib-only Python with zero external dependencies. The pack includes a dedicated agent, multiple scan scripts, CLI credential management, and CI pipeline integration.
311
+
312
+ **Key design principles:**
313
+ - **Read-only** — never modifies cloud resources, only reads state
314
+ - **Deterministic** — reproducible results, no LLM-driven regex (same pattern as `hipaa_scan.py`)
315
+ - **False positive aware** — context graph resolves "public endpoint behind gateway/App Check/WAF"
316
+ - **CI-ready** — `--output json` + `--output sarif` (SARIF v2.1.0 for GitHub Advanced Security), exit code 1 on HIGH, 0 otherwise
317
+ - **Credential isolation** — keys stored in `~/.ai-toolkit/credentials/`, accessible only by this pack's scripts
318
+ - **Static-first** — static mode (no credentials) is the default, live mode is opt-in upgrade
319
+ - **Incremental** — `--changed` flag scans only files modified since last commit (PR workflow)
320
+ - **IaC via `terraform show -json`** — wraps Terraform's own JSON output instead of parsing HCL directly
321
+
322
+ ---
323
+
324
+ ## 2. Architecture Overview
325
+
326
+ ```
327
+ ai-toolkit credentials add gcp --file ~/sa-viewer.json
328
+ ai-toolkit credentials add aws --profile my-audit-profile
329
+ ai-toolkit credentials add azure --subscription abc-123
330
+
331
+ ┌──────────────────────────────────────────────────────┐
332
+ │ cloud-security-pack │
333
+ │ │
334
+ │ Agent: cloud-security-auditor │
335
+ │ Tools: Read, Grep, Glob, Bash (read-only commands) │
336
+ │ │
337
+ │ Skills: │
338
+ │ /cloud-security-audit (orchestrator) │
339
+ │ /firebase-rules-audit (GCP: rules) │
340
+ │ /cloud-functions-audit (GCP: CF + IAM) │
341
+ │ /aws-security-audit (AWS: S3/Lambda) │
342
+ │ /azure-security-audit (Azure: NSG/Fn) │
343
+ │ │
344
+ │ Scripts (stdlib Python, zero deps): │
345
+ │ gcp_auth.py (credential helper)│
346
+ │ firebase_rules_scan.py (static parser) │
347
+ │ cloud_functions_audit.py (CF IAM + context) │
348
+ │ aws_security_scan.py (S3/Lambda/IAM) │
349
+ │ azure_security_scan.py (NSG/Fn/RBAC) │
350
+ │ false_positive_resolver.py (context graph) │
351
+ │ sarif_formatter.py (SARIF v2.1.0) │
352
+ │ incremental.py (git diff filter) │
353
+ │ │
354
+ │ Modes: │
355
+ │ --static (no credentials, parse IaC/source) │
356
+ │ --live (credentials, deployed state) │
357
+ │ --output json|sarif (CI pipeline) │
358
+ │ --changed <ref> (incremental, static only) │
359
+ │ --explain <id> (remediation lookup) │
360
+ └──────────────────────────────────────────────────────┘
361
+
362
+ ### YAML Parsing Constraint (BLOCKER)
363
+
364
+ Python stdlib has NO YAML parser. This affects AWS static mode:
365
+ - CloudFormation templates (`.yaml`) — YAML
366
+ - `serverless.yml` — YAML
367
+ - SAM templates (`template.yaml`) — YAML
368
+
369
+ **Decision: JSON-only for static IaC parsing.** Rationale:
370
+ 1. CloudFormation supports both JSON and YAML — JSON variant parseable with `json` module
371
+ 2. `terraform show -json` outputs JSON — the primary Terraform path
372
+ 3. `serverless.yml` → recommend users run `sls print --format json` to convert
373
+ 4. Adding `pyyaml` breaks the stdlib-only constraint for the entire toolkit
374
+
375
+ **Practical impact:** ~30% of CloudFormation users use JSON, ~70% YAML. For YAML users, live mode (`--mode live`) still works (queries APIs directly, no file parsing needed). The `--explain` output will suggest `sls print --format json` conversion.
376
+
377
+ **v2 option:** Ship a vendored minimal YAML subset parser (~150 LOC, handles flat key-value and simple nested maps — enough for security-relevant fields like `Principal`, `Effect`, `authLevel`).
378
+ ```
379
+
380
+ ---
381
+
382
+ ## 3. Progress Tracking
383
+
384
+ | # | Feature | Priority | Status | Est. Time | Notes |
385
+ |---|---------|----------|--------|-----------|-------|
386
+ | 1.1 | CLI `credentials` command (add/list/remove/test) | P0 | Proposed | 2d | 0600 perms, allowlist wrapper |
387
+ | 1.1b | `credentials init` interactive wizard | P1 | Proposed | 1.5d | **Deferred to Milestone 2** (orchestration-review: saves 1.5d in M1 critical path) |
388
+ | 1.2 | `cloud-security-auditor` agent | P0 | Proposed | 1d | Agent definition |
389
+ | 1.3 | SARIF + incremental scan infrastructure | P0 | Proposed | 2d | `--output sarif`, `--changed` flag |
390
+ | 2.1 | `firebase-rules-audit` skill + script | P0 | Proposed | 4-5d | Recursive descent parser (orchestration-review: +1d vs regex) |
391
+ | 2.2 | `cloud-functions-audit` skill + script | P0 | Proposed | 3-4d | CF IAM + App Check context |
392
+ | 2.3 | False positive resolver (GCP context) | P0 | Proposed | 3-4d | Context graph engine — GCP only (~8 code paths). **+2d per provider** in later milestones (orchestration-review) |
393
+ | 3.1 | `aws-security-audit` skill + script | P1 | Proposed | 4-5d | S3/Lambda/IAM/SG, `terraform show -json` |
394
+ | 3.2 | `/cloud-security-audit` orchestrator + plugin.json | P1 | Proposed | 3d | Multi-provider orchestration + pack |
395
+ | 4.1 | `azure-security-audit` skill + script | P1 | Proposed | 4-5d | NSG/Functions/RBAC |
396
+ | 5.1 | Tests + CI integration docs | P1 | Proposed | 3d | Tests + SARIF + pipeline examples |
397
+ | 5.2 | Documentation (kb/) | P2 | Proposed | 1d | Checklists, patterns, KB |
398
+
399
+ **Phasing (full delivery, all 3 providers):**
400
+ - **Phase 1 (week 1-2):** Foundation + GCP — credentials CLI, agent, SARIF/incremental infra, firebase-rules, cloud-functions, false positive resolver (GCP context)
401
+ - **Phase 2 (week 3-4):** AWS + orchestrator — AWS security audit, `terraform show -json`, orchestrator skill, plugin pack, `credentials init`
402
+ - **Phase 3 (week 5-6):** Azure + polish — Azure security audit, false positive resolver (Azure context), full test suite, documentation
403
+
404
+ ---
405
+
406
+ ## 4. Dependency Graph
407
+
408
+ ```
409
+ Phase 1: Foundation + GCP (week 1-2)
410
+ ====================================
411
+ credentials CLI (1.1) ─────┐
412
+ ├──► firebase-rules-audit (2.1)
413
+ SARIF + incremental (1.3) ──┤
414
+ ├──► cloud-functions-audit (2.2) ──► false-positive-resolver (2.3)
415
+ agent definition (1.2) ─────┘
416
+
417
+ Phase 2: AWS + Orchestrator (week 3-4)
418
+ ======================================
419
+ credentials init (1.1b) ───┐
420
+ ├──► aws-security-audit (3.1) ─────► false-positive-resolver (+AWS context)
421
+ └──► orchestrator skill + plugin.json (3.2)
422
+
423
+ Phase 3: Azure + Polish (week 5-6)
424
+ ==================================
425
+ ├──► azure-security-audit (4.1) ───► false-positive-resolver (+Azure context)
426
+ └──► tests + docs (5.1, 5.2)
427
+ ```
428
+
429
+ **All 3 providers ship.** No conditional gates — full delivery in 6 weeks.
430
+
431
+ ---
432
+
433
+ ## 5. Detailed Implementation
434
+
435
+ ### Faza 1: Foundation (tydzien 1)
436
+
437
+ #### 1.1 CLI `credentials` Command
438
+
439
+ **Purpose:** Secure credential storage for cloud provider API access. Credentials live outside any project directory and are only accessible by this pack's scripts.
440
+
441
+ **CLI interface:**
442
+ ```bash
443
+ # GCP — Service Account JSON
444
+ ai-toolkit credentials add gcp --file ~/sa-viewer.json
445
+ ai-toolkit credentials add gcp --file ~/sa-viewer.json --project my-project-id
446
+
447
+ # GCP — use existing gcloud session (no file needed)
448
+ ai-toolkit credentials add gcp --gcloud --project my-project-id
449
+
450
+ # AWS — named profile (reads from ~/.aws/credentials)
451
+ ai-toolkit credentials add aws --profile audit-readonly
452
+ ai-toolkit credentials add aws --profile audit-readonly --region eu-west-1
453
+
454
+ # AWS — explicit keys (interactive, never on CLI args)
455
+ ai-toolkit credentials add aws --interactive
456
+
457
+ # Azure — subscription
458
+ ai-toolkit credentials add azure --subscription abc-123-def
459
+
460
+ # Azure — use existing az login session
461
+ ai-toolkit credentials add azure --az-cli
462
+
463
+ # Interactive guided setup (reduces onboarding from 4 steps to 1)
464
+ ai-toolkit credentials init # auto-detect provider, interactive wizard
465
+ ai-toolkit credentials init --provider gcp # skip auto-detect, go straight to GCP setup
466
+
467
+ # Management
468
+ ai-toolkit credentials list
469
+ ai-toolkit credentials remove gcp
470
+ ai-toolkit credentials remove aws
471
+ ai-toolkit credentials remove azure
472
+ ai-toolkit credentials test gcp # verify read-only access works
473
+ ai-toolkit credentials test aws
474
+ ```
475
+
476
+ **Storage structure:**
477
+ ```
478
+ ~/.ai-toolkit/
479
+ credentials/
480
+ gcp.json # SA key file (copied, chmod 0600)
481
+ gcp.meta.json # { project_id, added_at, method: "file"|"gcloud" }
482
+ aws.json # { profile, region, method: "profile"|"keys" }
483
+ azure.json # { subscription_id, method: "subscription"|"az-cli" }
484
+ ```
485
+
486
+ **`credentials init` interactive flow:**
487
+ 1. Auto-detect providers from project files (`firebase.json` → GCP, `*.tf` with `provider "aws"` → AWS, etc.)
488
+ 2. If multiple detected → ask user: "Found GCP and AWS markers. Which provider to configure first? [gcp/aws/both]"
489
+ 3. Per provider:
490
+ - GCP: "Do you have a Service Account JSON file? [y/n]" → if yes: ask path → if no: "Run `gcloud auth application-default login` and we'll use that"
491
+ - AWS: "Do you have a named profile in ~/.aws/credentials? [y/n]" → if yes: ask profile name → if no: "Run `aws configure` first"
492
+ - Azure: "Do you have an active `az login` session? [y/n]" → if yes: ask subscription ID → if no: "Run `az login` first"
493
+ 4. Run `credentials test` automatically after setup
494
+ 5. Generate `.cloud-security.json` scaffold with detected context
495
+
496
+ **Security requirements:**
497
+ - All credential files: `chmod 0600` (owner read/write only)
498
+ - Never log credential contents to stdout/stderr
499
+ - `credentials test` validates:
500
+ - Connection works (GCP: `gcloud auth list`, AWS: `aws sts get-caller-identity`, Azure: `az account show`)
501
+ - SA/role has **only read permissions** — **REFUSE to store** if write access detected (orchestration-review: warn-only is ignored by users). Override: `--force` flag with explicit acknowledgment
502
+ - Project/subscription exists
503
+ - `.gitignore`-proof — lives in `~/.ai-toolkit/`, never in project directory
504
+ - Scripts access credentials via `gcp_auth.py` helper — single entry point, no direct file reads
505
+
506
+ **Files to create/modify:**
507
+
508
+ | File | Action | Description |
509
+ |------|--------|-------------|
510
+ | `scripts/credentials_cli.py` | CREATE | CLI: add/list/remove/test credentials |
511
+ | `bin/ai-toolkit.js` | EDIT | Register `credentials` subcommand |
512
+ | `tests/test_credentials.bats` | CREATE | Tests: add, remove, permissions, test |
513
+
514
+ **Success Criteria:**
515
+ - [ ] `credentials add gcp --file` copies and secures SA key
516
+ - [ ] `credentials add aws --profile` stores profile reference
517
+ - [ ] `credentials add azure --subscription` stores subscription
518
+ - [ ] `credentials test` validates read access + warns on write perms
519
+ - [ ] `credentials list` shows providers without exposing secrets
520
+ - [ ] `credentials remove` cleans up securely
521
+ - [ ] All files created with 0600 permissions
522
+ - [ ] Tests: >= 8
523
+
524
+ ---
525
+
526
+ #### 1.3 SARIF Output + Incremental Scanning (Council addition)
527
+
528
+ **SARIF v2.1.0 output** — industry standard consumed by GitHub Advanced Security (inline PR annotations), VS Code SARIF Viewer, Azure DevOps, and SonarQube.
529
+
530
+ ```bash
531
+ /cloud-security-audit --output sarif > results.sarif
532
+
533
+ # GitHub Actions: upload SARIF for inline PR annotations
534
+ - uses: github/codeql-action/upload-sarif@v3
535
+ with:
536
+ sarif_file: results.sarif
537
+ ```
538
+
539
+ **SARIF structure (per finding):**
540
+ ```json
541
+ {
542
+ "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
543
+ "version": "2.1.0",
544
+ "runs": [{
545
+ "tool": { "driver": {
546
+ "name": "cloud-security-audit", "version": "1.0.0",
547
+ "rules": [{ "id": "GCP-CF-001", "shortDescription": { "text": "Public Cloud Function invoker" }, "helpUri": "https://cloud.google.com/functions/docs/securing" }]
548
+ } },
549
+ "results": [{
550
+ "ruleId": "GCP-CF-001",
551
+ "level": "error",
552
+ "message": { "text": "Cloud Function 'adminEndpoint' has allUsers invoker with no protection layer" },
553
+ "locations": [{ "physicalLocation": { "artifactLocation": { "uri": "functions/src/admin.ts" }, "region": { "startLine": 42 } } }],
554
+ "properties": { "resolved_severity": "HIGH", "context_chain": [], "provider": "gcp" }
555
+ }]
556
+ }]
557
+ }
558
+ ```
559
+
560
+ **Incremental scan mode** — scan only changed files since last commit:
561
+
562
+ ```bash
563
+ /cloud-security-audit --changed # files changed vs HEAD~1
564
+ /cloud-security-audit --changed HEAD~5 # files changed in last 5 commits
565
+ /cloud-security-audit --changed main # files changed vs main branch (PR workflow)
566
+ ```
567
+
568
+ Implementation: use `git diff --name-only <ref>` to get changed files, filter to relevant extensions (.rules, .tf, .json, .ts, .py, .bicep), scan only those. Falls back to full scan if no git repo detected.
569
+
570
+ **`--changed` applies to static mode ONLY.** Live mode always scans all deployed resources (it queries cloud APIs, not files). If user passes `--changed` with `--mode live`, emit warning: "Incremental scan applies to static analysis only. Live mode will scan all resources." and proceed with full live scan.
571
+
572
+ **`--explain` flag** — detailed remediation for a specific finding:
573
+
574
+ ```bash
575
+ /cloud-security-audit --explain GCP-CF-001
576
+ # Output: what the finding means, why it matters, exact fix steps,
577
+ # links to GCP documentation, CIS Benchmark reference
578
+ ```
579
+
580
+ **Files:**
581
+
582
+ | File | Action | Description |
583
+ |------|--------|-------------|
584
+ | `app/skills/cloud-security-audit/scripts/sarif_formatter.py` | CREATE | SARIF v2.1.0 output |
585
+ | `app/skills/cloud-security-audit/scripts/incremental.py` | CREATE | Git diff + file filter |
586
+ | `app/skills/cloud-security-audit/reference/rule-explanations.json` | CREATE | Per-rule remediation guides |
587
+
588
+ **Success Criteria:**
589
+ - [ ] `--output sarif` produces valid SARIF v2.1.0 JSON with `driver.rules[]` array (orchestration-review: GitHub silently drops annotations without rule metadata)
590
+ - [ ] SARIF upload to GitHub Advanced Security works (inline PR annotations)
591
+ - [ ] SARIF `level` mapping: HIGH→`error`, WARN→`warning`, INFO→`note`
592
+ - [ ] `--changed main` scans only PR-changed files
593
+ - [ ] `--explain <rule-id>` shows detailed remediation
594
+ - [ ] Tests: >= 6
595
+
596
+ ---
597
+
598
+ #### 1.2 Agent Definition: `cloud-security-auditor`
599
+
600
+ **File:** `app/agents/cloud-security-auditor.md`
601
+
602
+ ```markdown
603
+ ---
604
+ name: cloud-security-auditor
605
+ description: "Multi-cloud security auditor (GCP/AWS/Azure). Read-only deterministic scans
606
+ for IAM, network, storage, serverless, and compliance. False positive resolution
607
+ via security context graph."
608
+ model: opus
609
+ color: red
610
+ tools: Read, Grep, Glob, Bash
611
+ skills: security-patterns, cloud-security-audit
612
+ ---
613
+
614
+ # Cloud Security Auditor Agent
615
+
616
+ You are the **Cloud Security Auditor**. You perform read-only security assessments
617
+ across GCP, AWS, and Azure. You never modify cloud resources.
618
+
619
+ ## Core Philosophy
620
+ **"Read everything, change nothing. Context before verdict."**
621
+
622
+ ## Mandatory Protocol
623
+ Before any audit:
624
+ 1. Check credentials: `ai-toolkit credentials test <provider>`
625
+ 2. Determine mode: `--static` (IaC/source only) or `--live` (deployed state)
626
+ 3. Run deterministic scripts first, then interpret results
627
+
628
+ ## Responsibilities
629
+
630
+ ### 1. Static Analysis (no credentials needed)
631
+ - Parse IaC: Terraform (.tf), CloudFormation (.yaml/.json), Bicep (.bicep)
632
+ - Parse Firebase rules: firestore.rules, database.rules.json
633
+ - Parse source: Cloud Functions (onCall vs onRequest), Lambda handlers, Azure Functions
634
+ - Parse configs: firebase.json, serverless.yml, sam-template.yaml
635
+
636
+ ### 2. Live Analysis (credentials required, READ-ONLY)
637
+ - GCP: `gcloud` CLI commands (list, describe, get-iam-policy)
638
+ - AWS: `aws` CLI commands (s3api, lambda, iam, ec2 — get/list/describe only)
639
+ - Azure: `az` CLI commands (network nsg, functionapp, cosmosdb — list/show only)
640
+
641
+ ### 3. False Positive Resolution
642
+ Build security context graph before rendering verdict:
643
+ - Public endpoint → check: API Gateway? WAF? App Check? CDN?
644
+ - Open port → check: behind Load Balancer? VPN? private subnet?
645
+ - Broad IAM role → check: scoped to specific resource? temporary?
646
+
647
+ ## Allowed CLI Commands (WHITELIST — read-only only)
648
+
649
+ ### GCP
650
+ - gcloud functions list / describe / get-iam-policy
651
+ - gcloud projects get-iam-policy
652
+ - gcloud app-check services list
653
+ - gcloud firestore indexes list
654
+ - gcloud compute firewall-rules list
655
+ - gcloud run services list / describe
656
+ - firebase apps:list
657
+
658
+ ### AWS
659
+ - aws s3api get-bucket-policy / get-bucket-acl / get-public-access-block
660
+ - aws lambda get-policy / get-function-configuration / list-functions
661
+ - aws iam list-roles / list-policies / get-role / get-policy-version
662
+ - aws ec2 describe-security-groups / describe-network-acls
663
+ - aws apigateway get-rest-apis / get-resources
664
+ - aws elbv2 describe-load-balancers / describe-listeners
665
+
666
+ ### Azure
667
+ - az network nsg list / show / rule list
668
+ - az functionapp list / show / config show
669
+ - az cosmosdb list / show / keys list
670
+ - az role assignment list
671
+ - az webapp show / config show
672
+ - az network application-gateway list
673
+
674
+ ### NEVER ALLOWED
675
+ - Any create/update/delete/put/set/deploy/push command
676
+ - Any command that modifies state
677
+ - `gcloud auth activate-service-account` (credential pivot)
678
+ - `aws sts assume-role` (lateral movement)
679
+ - `az login` (session hijack)
680
+ - `terraform apply` / `terraform destroy`
681
+
682
+ ## Output Format
683
+ (see skill SKILL.md for detailed format)
684
+ ```
685
+
686
+ **SECURITY: Programmatic Bash Allowlist (orchestration-review P0 BLOCKER)**
687
+
688
+ The read-only whitelist above lives in the agent's system prompt. An LLM can be prompt-injected via malicious IaC files or source code comments. **The prompt-only approach is insufficient.**
689
+
690
+ **Required:** A shell wrapper script (`scripts/cloud_security_allowlist.sh`) that validates every CLI invocation against a compiled allowlist regex BEFORE execution, independent of the LLM:
691
+
692
+ ```bash
693
+ #!/bin/bash
694
+ # cloud_security_allowlist.sh — wraps Bash calls from cloud-security-auditor
695
+ # Rejects any command not matching read-only patterns
696
+
697
+ ALLOWED_PATTERNS=(
698
+ '^gcloud (functions|projects|app-check|firestore|compute|run) (list|describe|get-iam-policy|indexes)'
699
+ '^gcloud auth list$'
700
+ '^firebase apps:list'
701
+ '^aws (s3api|lambda|iam|ec2|apigateway|elbv2|sts|cloudfront) (get-|list-|describe-|generate-credential-report)'
702
+ '^aws sts get-caller-identity$'
703
+ '^az (network|functionapp|cosmosdb|role|webapp|storage) (list|show|rule list|config show|assignment list|account show)'
704
+ '^az account show$'
705
+ '^terraform show -json'
706
+ '^git diff --name-only'
707
+ '^python3 .*/scripts/.*\.(py)$'
708
+ )
709
+
710
+ CMD="$*"
711
+ for pattern in "${ALLOWED_PATTERNS[@]}"; do
712
+ if [[ "$CMD" =~ $pattern ]]; then
713
+ exec $CMD
714
+ fi
715
+ done
716
+
717
+ echo "BLOCKED: Command not in read-only allowlist: $CMD" >&2
718
+ exit 1
719
+ ```
720
+
721
+ The agent's `allowed-tools` in SKILL.md references this wrapper instead of raw Bash. All cloud CLI calls go through the allowlist.
722
+
723
+ **`terraform plan` risk (orchestration-review):** `terraform plan -json` executes providers and provisioners. A malicious `.tf` file with `local-exec` provisioner runs arbitrary code during plan. **Decision: use `terraform show -json` ONLY (reads existing state), NOT `terraform plan -json`.** Document this prominently.
724
+
725
+ **Files:**
726
+
727
+ | File | Action | Description |
728
+ |------|--------|-------------|
729
+ | `app/skills/cloud-security-audit/scripts/cloud_security_allowlist.sh` | CREATE | Bash allowlist wrapper |
730
+
731
+ **Success Criteria:**
732
+ - [ ] Agent file created in `app/agents/`
733
+ - [ ] Read-only command whitelist documented
734
+ - [ ] NEVER ALLOWED section explicit
735
+ - [ ] Programmatic Bash allowlist enforced (not just prompt)
736
+ - [ ] `terraform plan` excluded — only `terraform show -json` allowed
737
+ - [ ] Allowlist tested: blocked commands return exit 1
738
+
739
+ ---
740
+
741
+ ### Faza 2: GCP / Firebase (Phase 1, tydzien 1-2)
742
+
743
+ #### 2.1 Skill: `firebase-rules-audit`
744
+
745
+ **Purpose:** Static analysis of Firestore rules and RTDB rules. No credentials needed.
746
+
747
+ **What it scans:**
748
+
749
+ | Check | Severity | Description |
750
+ |-------|----------|-------------|
751
+ | `allow read, write: if true` | HIGH | World-readable/writable collection |
752
+ | `allow read: if true` without `write` guard | WARN | Public read — may be intentional |
753
+ | `allow write: if request.auth != null` without field validation | WARN | Authenticated but no field-level validation |
754
+ | Missing `request.resource.data` validation on writes | WARN | No schema enforcement |
755
+ | Wildcard collection `/{document=**}` with broad permissions | HIGH | Recursive wildcard + open access |
756
+ | RTDB `.read: true` or `.write: true` at root | HIGH | Entire database public |
757
+ | RTDB `.read: "auth != null"` without path scoping | WARN | All authenticated users can read everything |
758
+ | Timestamp/TTL rules missing for sensitive collections | WARN | No data lifecycle enforcement |
759
+ | `get()` / `exists()` cross-collection reads without auth check | WARN | Privilege escalation via rule chaining |
760
+ | Rules file > 256KB (approaching Firebase limit) | WARN | May hit deployment limit |
761
+
762
+ **Script:** `scripts/firebase_rules_scan.py`
763
+ - Parses `firestore.rules` via **recursive descent parser** (not regex — orchestration-review P1)
764
+ - Grammar: ~8 production rules (service, match, allow, function, condition)
765
+ - Tracks: brace depth, current match path, accumulated allow blocks
766
+ - Handles: nested `match` blocks, multi-line conditions with `&&`/`||`, custom `function` declarations
767
+ - Estimated: 550-650 LOC for parser + check logic
768
+ - Unsupported (documented): CEL ternary expressions, complex `get()`/`exists()` chains with computed paths
769
+ - Parses `database.rules.json` (JSON rules — stdlib `json` module)
770
+ - Outputs findings as JSON or text
771
+ - Exit code 1 on HIGH, 0 otherwise
772
+ - Supports `.cloud-security-ignore` for suppressions
773
+
774
+ **Reference file:** `reference/firebase-rules-patterns.md` — safe/unsafe patterns with examples
775
+
776
+ **Files:**
777
+
778
+ | File | Action | Description |
779
+ |------|--------|-------------|
780
+ | `app/skills/firebase-rules-audit/SKILL.md` | CREATE | Skill definition |
781
+ | `app/skills/firebase-rules-audit/scripts/firebase_rules_scan.py` | CREATE | Deterministic scanner |
782
+ | `app/skills/firebase-rules-audit/reference/firebase-rules-patterns.md` | CREATE | Safe/unsafe patterns |
783
+
784
+ **Success Criteria:**
785
+ - [ ] Parses `firestore.rules` — detects 10+ check patterns
786
+ - [ ] Parses `database.rules.json` — detects root-level open access
787
+ - [ ] `--output json` for CI
788
+ - [ ] `.cloud-security-ignore` support
789
+ - [ ] Tests: >= 10 (one per check pattern + edge cases)
790
+
791
+ ---
792
+
793
+ #### 2.2 Skill: `cloud-functions-audit`
794
+
795
+ **Purpose:** Audit Cloud Functions permissions and detect false positives.
796
+
797
+ **Static mode (no credentials):**
798
+
799
+ | Check | Severity | Description |
800
+ |-------|----------|-------------|
801
+ | `onRequest` handler without auth middleware | WARN | Potentially public — needs context |
802
+ | `onCall` handler (inherently authenticated) | INFO | Informational — callable is auth'd |
803
+ | Hardcoded API keys / secrets in source | HIGH | Secrets in code |
804
+ | CORS `origin: '*'` in CF handler | WARN | Unrestricted CORS |
805
+ | Missing rate limiting patterns | WARN | No throttling on public endpoint |
806
+ | `functions.https.onRequest` + no `validateFirebaseIdToken` | WARN | HTTP function without Firebase Auth check |
807
+
808
+ **Live mode (credentials required):**
809
+
810
+ | Check | Severity | CLI Command | Description |
811
+ |-------|----------|-------------|-------------|
812
+ | `allUsers` invoker on CF | CONTEXT | `gcloud functions get-iam-policy` | Public — resolve with context graph |
813
+ | `allAuthenticatedUsers` invoker | WARN | `gcloud functions get-iam-policy` | Any Google account can invoke |
814
+ | App Check enforcement status | CONTEXT | `gcloud app-check services list` | Feeds into false positive resolution |
815
+ | Deployed rules vs local rules diff | WARN | `gcloud firestore indexes` + local | Rules drift detection |
816
+ | Cloud Run public ingress | CONTEXT | `gcloud run services describe` | Public — resolve with context graph |
817
+ | Overly broad IAM roles on SA | HIGH | `gcloud projects get-iam-policy` | CF service account with editor/owner |
818
+
819
+ **Script:** `scripts/cloud_functions_audit.py`
820
+
821
+ **Files:**
822
+
823
+ | File | Action | Description |
824
+ |------|--------|-------------|
825
+ | `app/skills/cloud-functions-audit/SKILL.md` | CREATE | Skill definition |
826
+ | `app/skills/cloud-functions-audit/scripts/cloud_functions_audit.py` | CREATE | Scanner |
827
+ | `app/skills/cloud-functions-audit/scripts/gcp_auth.py` | CREATE | Credential loader |
828
+ | `app/skills/cloud-functions-audit/reference/false-positives-gcp.md` | CREATE | False positive patterns |
829
+
830
+ **Success Criteria:**
831
+ - [ ] Static: parses CF source for auth patterns
832
+ - [ ] Live: checks IAM bindings via `gcloud`
833
+ - [ ] False positive resolution for App Check + Gateway patterns
834
+ - [ ] Tests: >= 8
835
+
836
+ ---
837
+
838
+ #### 2.3 False Positive Resolver
839
+
840
+ **Purpose:** Central engine that resolves "is this actually a problem?" by building a security context graph.
841
+
842
+ **How it works:**
843
+ ```
844
+ Input: Finding { resource, severity, type }
845
+
846
+ Step 1: Gather context
847
+ ├── Check API Gateway routes (firebase.json rewrites, API Gateway configs)
848
+ ├── Check WAF/CDN (Cloudflare, CloudFront, Azure Front Door)
849
+ ├── Check App Check / AppArmor / Shield
850
+ ├── Check callable vs HTTP function type
851
+ ├── Check VPC / private subnet placement
852
+ └── Check Load Balancer + auth middleware
853
+
854
+ Step 2: Apply resolution rules
855
+ ├── Public CF + App Check ENFORCED → SUPPRESSED (protected)
856
+ ├── Public CF + API Gateway route → SUPPRESSED (gateway handles auth)
857
+ ├── Public CF + onCall() → SUPPRESSED (callable is auth'd by SDK)
858
+ ├── Public S3 + CloudFront OAI → SUPPRESSED (not directly accessible)
859
+ ├── Open SG port + ALB → SUPPRESSED (ALB handles TLS + auth)
860
+ ├── Open NSG + Application Gateway → SUPPRESSED (WAF handles filtering)
861
+ └── No context found → KEEP ORIGINAL SEVERITY
862
+
863
+ Step 3: Output
864
+ ├── Original severity
865
+ ├── Resolved severity (SUPPRESSED / DOWNGRADED / CONFIRMED)
866
+ ├── Context chain (what protections were found)
867
+ └── Confidence (high if multiple protections, low if single)
868
+ ```
869
+
870
+ **Output example:**
871
+ ```json
872
+ {
873
+ "resource": "processPayment",
874
+ "provider": "gcp",
875
+ "type": "cloud-function-public-invoker",
876
+ "original_severity": "HIGH",
877
+ "resolved_severity": "SUPPRESSED",
878
+ "confidence": "high",
879
+ "context_chain": [
880
+ { "layer": "app_check", "status": "ENFORCED", "source": "gcloud app-check services list" },
881
+ { "layer": "function_type", "status": "onCall", "source": "source:index.ts:42" },
882
+ { "layer": "api_gateway", "status": "ROUTED", "source": "firebase.json:rewrites" }
883
+ ],
884
+ "verdict": "3/3 protection layers active. Suppressing finding."
885
+ }
886
+ ```
887
+
888
+ **Script:** `scripts/false_positive_resolver.py`
889
+
890
+ **Resolution rules stored in:** `reference/resolution-rules.json`
891
+ ```json
892
+ {
893
+ "rules": [
894
+ {
895
+ "id": "gcp-cf-appcheck",
896
+ "finding_type": "cloud-function-public-invoker",
897
+ "provider": "gcp",
898
+ "context_required": ["app_check:ENFORCED"],
899
+ "action": "SUPPRESS",
900
+ "reason": "App Check enforced — only verified app instances can invoke"
901
+ },
902
+ {
903
+ "id": "gcp-cf-callable",
904
+ "finding_type": "cloud-function-public-invoker",
905
+ "provider": "gcp",
906
+ "context_required": ["function_type:onCall"],
907
+ "action": "SUPPRESS",
908
+ "reason": "onCall functions require Firebase Auth token from client SDK"
909
+ },
910
+ {
911
+ "id": "aws-s3-cloudfront-oai",
912
+ "finding_type": "s3-bucket-public-access",
913
+ "provider": "aws",
914
+ "context_required": ["cloudfront_oai:ACTIVE"],
915
+ "action": "SUPPRESS",
916
+ "reason": "Bucket accessed only via CloudFront Origin Access Identity"
917
+ },
918
+ {
919
+ "id": "azure-func-apigw",
920
+ "finding_type": "function-app-public",
921
+ "provider": "azure",
922
+ "context_required": ["application_gateway:ACTIVE"],
923
+ "action": "SUPPRESS",
924
+ "reason": "Function behind Application Gateway with WAF"
925
+ }
926
+ ]
927
+ }
928
+ ```
929
+
930
+ **Files:**
931
+
932
+ | File | Action | Description |
933
+ |------|--------|-------------|
934
+ | `app/skills/cloud-security-audit/scripts/false_positive_resolver.py` | CREATE | Context graph engine |
935
+ | `app/skills/cloud-security-audit/reference/resolution-rules.json` | CREATE | Configurable rules |
936
+
937
+ **Success Criteria:**
938
+ - [ ] Resolves GCP: App Check, Gateway, callable patterns
939
+ - [ ] Resolves AWS: CloudFront OAI, ALB, WAF patterns
940
+ - [ ] Resolves Azure: App Gateway, Front Door, VNET patterns
941
+ - [ ] JSON output with context chain
942
+ - [ ] User can add custom rules to `.cloud-security.json` `context` section
943
+ - [ ] Tests: >= 12 (4 per provider)
944
+
945
+ ---
946
+
947
+ ### Faza 3: AWS (Phase 2, tydzien 3-4)
948
+
949
+ #### 3.1 Skill: `aws-security-audit`
950
+
951
+ **Static mode (IaC parsing):**
952
+
953
+ | Check | Severity | Source | Description |
954
+ |-------|----------|--------|-------------|
955
+ | S3 bucket `"Effect": "Allow", "Principal": "*"` | HIGH | .tf / .json | Public bucket policy |
956
+ | S3 `BlockPublicAccess` all false | HIGH | .tf / .json | Public access not blocked |
957
+ | Lambda `resource-based policy` with `Principal: "*"` | HIGH | .tf / .json | Public Lambda |
958
+ | Security Group `0.0.0.0/0` ingress on non-80/443 | HIGH | .tf / .json | Open port to world |
959
+ | IAM policy with `Action: "*"` | HIGH | .tf / .json | God-mode IAM |
960
+ | IAM policy with `Resource: "*"` + sensitive actions | WARN | .tf / .json | Broad resource scope |
961
+ | Unencrypted RDS/DynamoDB | WARN | .tf / .json | Missing encryption at rest |
962
+ | CloudTrail disabled | HIGH | .tf / .json | No audit logging |
963
+ | Missing VPC Flow Logs | WARN | .tf / .json | No network monitoring |
964
+
965
+ **Live mode:**
966
+
967
+ | Check | CLI Command | Description |
968
+ |-------|-------------|-------------|
969
+ | S3 public buckets | `aws s3api get-public-access-block` | Per-bucket public access |
970
+ | Lambda public policies | `aws lambda get-policy` | Resource-based policies |
971
+ | Open Security Groups | `aws ec2 describe-security-groups` | Ingress from 0.0.0.0/0 |
972
+ | Overly permissive IAM | `aws iam list-roles` + `get-role` | Roles with admin/broad access |
973
+ | Unused IAM credentials | `aws iam generate-credential-report` | Stale access keys |
974
+ | API Gateway without auth | `aws apigateway get-rest-apis` | Endpoints without authorizer |
975
+
976
+ **Script:** `scripts/aws_security_scan.py`
977
+
978
+ **Files:**
979
+
980
+ | File | Action | Description |
981
+ |------|--------|-------------|
982
+ | `app/skills/aws-security-audit/SKILL.md` | CREATE | Skill definition |
983
+ | `app/skills/aws-security-audit/scripts/aws_security_scan.py` | CREATE | Scanner |
984
+ | `app/skills/aws-security-audit/scripts/aws_auth.py` | CREATE | Credential loader |
985
+ | `app/skills/aws-security-audit/reference/aws-security-checklist.md` | CREATE | CIS Benchmark mapping |
986
+ | `app/skills/aws-security-audit/reference/false-positives-aws.md` | CREATE | ALB/CloudFront/WAF patterns |
987
+
988
+ **Terraform approach (council revision):** Do NOT parse HCL directly (stdlib-only Python cannot handle heredocs, variable interpolation, modules, `for_each`). Instead wrap `terraform show -json` / `terraform plan -json` which outputs clean JSON. Fallback to flat-resource regex for projects without `terraform` CLI.
989
+
990
+ **Success Criteria:**
991
+ - [ ] Static: `terraform show -json` wrapper + CloudFormation/SAM JSON parsing
992
+ - [ ] Live: checks S3, Lambda, IAM, SG via `aws` CLI
993
+ - [ ] False positive resolution for CloudFront/ALB/WAF
994
+ - [ ] CIS Benchmark mapping in reference
995
+ - [ ] Tests: >= 10
996
+
997
+ ---
998
+
999
+ ### Faza 4: Azure (Phase 3, tydzien 5-6)
1000
+
1001
+ #### 4.1 Skill: `azure-security-audit`
1002
+
1003
+ **Static mode (IaC parsing):**
1004
+
1005
+ | Check | Severity | Source | Description |
1006
+ |-------|----------|--------|-------------|
1007
+ | NSG rule `0.0.0.0/0` source on management ports | HIGH | .tf / .bicep | Open RDP/SSH to world |
1008
+ | Function App `authLevel: "anonymous"` | WARN | source / .tf | Public Azure Function |
1009
+ | Cosmos DB `publicNetworkAccess: enabled` | WARN | .tf / .bicep | Public database access |
1010
+ | Storage Account `allowBlobPublicAccess: true` | HIGH | .tf / .bicep | Public blob storage |
1011
+ | Missing Key Vault references (hardcoded secrets) | HIGH | source | Secrets not in Key Vault |
1012
+ | Missing RBAC (classic co-admin model) | WARN | .tf | Legacy access model |
1013
+
1014
+ **Live mode:**
1015
+
1016
+ | Check | CLI Command | Description |
1017
+ |-------|-------------|-------------|
1018
+ | Open NSG rules | `az network nsg rule list` | Broad inbound rules |
1019
+ | Function App auth | `az functionapp show` + `config` | Auth level and provider |
1020
+ | Cosmos DB access | `az cosmosdb show` | Network access settings |
1021
+ | RBAC assignments | `az role assignment list` | Owner/Contributor sprawl |
1022
+ | Storage public access | `az storage account show` | Blob public access |
1023
+ | App Service auth | `az webapp auth show` | Auth configuration |
1024
+
1025
+ **Script:** `scripts/azure_security_scan.py`
1026
+
1027
+ **Files:**
1028
+
1029
+ | File | Action | Description |
1030
+ |------|--------|-------------|
1031
+ | `app/skills/azure-security-audit/SKILL.md` | CREATE | Skill definition |
1032
+ | `app/skills/azure-security-audit/scripts/azure_security_scan.py` | CREATE | Scanner |
1033
+ | `app/skills/azure-security-audit/scripts/azure_auth.py` | CREATE | Credential loader |
1034
+ | `app/skills/azure-security-audit/reference/azure-security-checklist.md` | CREATE | CIS Benchmark mapping |
1035
+ | `app/skills/azure-security-audit/reference/false-positives-azure.md` | CREATE | App Gateway/Front Door patterns |
1036
+
1037
+ **Success Criteria:**
1038
+ - [ ] Static: parses Terraform, Bicep, ARM templates
1039
+ - [ ] Live: checks NSG, Functions, CosmosDB, RBAC via `az` CLI
1040
+ - [ ] False positive resolution for App Gateway/Front Door/VNET
1041
+ - [ ] Tests: >= 10
1042
+
1043
+ ---
1044
+
1045
+ ### Faza 3 (cont.): Orchestration + Pack Integration (Phase 2, tydzien 3-4)
1046
+
1047
+ #### 3.2 Orchestrator Skill: `/cloud-security-audit`
1048
+
1049
+ **Purpose:** Single entry point that runs all provider audits detected in the project.
1050
+
1051
+ **Behavior:**
1052
+ 1. Auto-detect providers from project files:
1053
+ - `firebase.json` / `firestore.rules` / `.firebaserc` → GCP
1054
+ - `serverless.yml` / `template.yaml` / `*.tf` with `provider "aws"` → AWS
1055
+ - `*.bicep` / `*.tf` with `provider "azurerm"` / `azure-pipelines.yml` → Azure
1056
+ 2. Check available credentials: `ai-toolkit credentials list`
1057
+ 3. Run detected provider scans in parallel
1058
+ 4. Merge results through false positive resolver
1059
+ 5. Output unified report
1060
+
1061
+ **Skill frontmatter:**
1062
+ ```yaml
1063
+ ---
1064
+ name: cloud-security-audit
1065
+ description: "Multi-cloud security audit — auto-detects GCP/AWS/Azure and runs
1066
+ deterministic scans with false positive resolution"
1067
+ user-invocable: true
1068
+ effort: high
1069
+ disable-model-invocation: true
1070
+ context: fork
1071
+ agent: cloud-security-auditor
1072
+ argument-hint: "[path] [--provider gcp|aws|azure|auto] [--mode static|live] [--severity high|warn] [--output json]"
1073
+ allowed-tools: Read, Grep, Glob, Bash
1074
+ ---
1075
+ ```
1076
+
1077
+ **CLI usage:**
1078
+ ```bash
1079
+ /cloud-security-audit # auto-detect providers, static mode (default)
1080
+ /cloud-security-audit --provider gcp # GCP only
1081
+ /cloud-security-audit --provider aws,azure # AWS + Azure
1082
+ /cloud-security-audit --mode static # no credentials, IaC/source only (DEFAULT)
1083
+ /cloud-security-audit --mode live # deployed state (requires credentials)
1084
+ /cloud-security-audit --severity high # HIGH findings only
1085
+ /cloud-security-audit --output json # CI pipeline output
1086
+ /cloud-security-audit --output sarif # SARIF v2.1.0 for GitHub Advanced Security
1087
+ /cloud-security-audit --changed main # incremental: only files changed vs main
1088
+ /cloud-security-audit --explain GCP-CF-001 # detailed remediation for specific rule
1089
+ /cloud-security-audit src/functions/ # scan specific path
1090
+ ```
1091
+
1092
+ **Unified report format:**
1093
+ ```markdown
1094
+ ## Cloud Security Audit Report
1095
+
1096
+ ### Summary
1097
+ | Metric | GCP | AWS | Azure | Total |
1098
+ |--------|-----|-----|-------|-------|
1099
+ | Mode | live | static | n/a | — |
1100
+ | Resources scanned | 12 | 8 | 0 | 20 |
1101
+ | HIGH | 2 | 1 | 0 | 3 |
1102
+ | WARN | 4 | 3 | 0 | 7 |
1103
+ | SUPPRESSED (false positive) | 3 | 1 | 0 | 4 |
1104
+
1105
+ ### Findings (sorted by severity)
1106
+
1107
+ #### [HIGH] GCP: Cloud Function "adminEndpoint" — public invoker, no protection
1108
+ ...
1109
+
1110
+ #### [SUPPRESSED] GCP: Cloud Function "processPayment" — public invoker
1111
+ Context: App Check ENFORCED + onCall + API Gateway routed (3/3 layers)
1112
+ ...
1113
+ ```
1114
+
1115
+ ---
1116
+
1117
+ #### 3.2.1 Plugin Pack Manifest
1118
+
1119
+ **File:** `app/plugins/cloud-security-pack/plugin.json`
1120
+
1121
+ ```json
1122
+ {
1123
+ "name": "cloud-security-pack",
1124
+ "description": "Multi-cloud security auditing for GCP, AWS, and Azure",
1125
+ "version": "1.0.0",
1126
+ "domain": "cloud-security",
1127
+ "type": "plugin-pack",
1128
+ "status": "experimental",
1129
+ "requires": [],
1130
+ "includes": {
1131
+ "agents": ["cloud-security-auditor"],
1132
+ "skills": [
1133
+ "cloud-security-audit",
1134
+ "firebase-rules-audit",
1135
+ "cloud-functions-audit",
1136
+ "aws-security-audit",
1137
+ "azure-security-audit"
1138
+ ],
1139
+ "rules": [],
1140
+ "hooks": []
1141
+ },
1142
+ "credentials": {
1143
+ "supported_providers": ["gcp", "aws", "azure"],
1144
+ "setup_command": "ai-toolkit credentials add <provider>"
1145
+ }
1146
+ }
1147
+ ```
1148
+
1149
+ **Directory structure:**
1150
+ ```
1151
+ app/plugins/cloud-security-pack/
1152
+ ├── plugin.json
1153
+ ├── README.md
1154
+ └── (skills and agent live in core app/ dirs, referenced by name)
1155
+ ```
1156
+
1157
+ ---
1158
+
1159
+ ### Faza 5: Tests + Documentation (ongoing, ships with each milestone)
1160
+
1161
+ #### 5.1 Tests
1162
+
1163
+ | Test file | Count | Description |
1164
+ |-----------|-------|-------------|
1165
+ | `tests/test_credentials.bats` | 8+ | CLI credentials management |
1166
+ | `tests/test_firebase_rules_scan.py` | 10+ | Firestore/RTDB rules patterns |
1167
+ | `tests/test_cloud_functions_audit.py` | 8+ | CF static + live checks |
1168
+ | `tests/test_aws_security_scan.py` | 10+ | S3/Lambda/IAM/SG checks |
1169
+ | `tests/test_azure_security_scan.py` | 10+ | NSG/Functions/CosmosDB checks |
1170
+ | `tests/test_false_positive_resolver.py` | 12+ | Context graph resolution |
1171
+ | `tests/test_sarif_formatter.py` | 4+ | SARIF v2.1.0 output validation |
1172
+ | `tests/test_incremental.py` | 4+ | Git diff filtering, fallback |
1173
+ | `tests/test_credentials_init.py` | 4+ | Interactive wizard, auto-detect |
1174
+ | **Total** | **70+** | |
1175
+
1176
+ #### 5.2 Documentation
1177
+
1178
+ | File | Description |
1179
+ |------|-------------|
1180
+ | `kb/planning/cloud-security-pack-plan.md` | This document |
1181
+ | `kb/reference/cloud-security-checklist.md` | Unified multi-cloud checklist (created when Milestone 1 ships) |
1182
+ | Skills `reference/` dirs | Per-provider patterns and false positives |
1183
+
1184
+ ---
1185
+
1186
+ ## 6. Configuration & Suppression (Unified)
1187
+
1188
+ **Council revision:** merged `.cloud-security-ignore` + `.cloud-security-config` into a single `.cloud-security.json`. Two config files is one too many — developers expect one file.
1189
+
1190
+ Scaffold interactively: `ai-toolkit credentials init` generates this file.
1191
+
1192
+ **Suppression Governance (orchestration-review P0):**
1193
+ - **Wildcard ignores** (e.g., `GCP-CF-*`) REQUIRE a `justification` field — scanner refuses to suppress without one
1194
+ - **All ignore entries** require `justification` — enforced by schema validation
1195
+ - **CI diff detection:** When `.cloud-security.json` is modified in a PR, scanner emits a `SUPPRESSION_CHANGED` finding (severity: WARN) listing added/removed/modified ignore rules. This prevents silent suppression of vulnerabilities via committed config changes.
1196
+ - **In live mode:** context claims (e.g., `app_check_enforced: true`) are verified against actual cloud state. If claim doesn't match reality, emit `CONTEXT_MISMATCH` finding (severity: HIGH)
1197
+
1198
+ ```json
1199
+ {
1200
+ "$schema": "https://softspark.github.io/ai-toolkit/schemas/cloud-security.json",
1201
+
1202
+ "ignore": [
1203
+ { "rule": "GCP-CF-001:processPayment", "justification": "Behind App Check + API Gateway, verified 2026-04-10" },
1204
+ { "rule": "AWS-S3-001:static-assets", "justification": "Intentionally public static bucket, CloudFront OAI active" },
1205
+ { "rule": "GCP-CF-*", "justification": "REQUIRED for wildcard suppression — reviewed by @jacek in PR #142" }
1206
+ ],
1207
+
1208
+ "context": {
1209
+ "gcp": {
1210
+ "app_check_enforced": true,
1211
+ "api_gateway": "projects/my-proj/locations/us-central1/gateways/main",
1212
+ "known_public_functions": ["healthCheck", "webhookReceiver"]
1213
+ },
1214
+ "aws": {
1215
+ "waf_enabled": true,
1216
+ "cloudfront_distributions": ["E1234567890"],
1217
+ "known_public_buckets": ["static-assets-prod"]
1218
+ },
1219
+ "azure": {
1220
+ "front_door_enabled": true,
1221
+ "application_gateway": "my-app-gw",
1222
+ "known_public_functions": ["webhookHandler"]
1223
+ }
1224
+ }
1225
+ }
1226
+ ```
1227
+
1228
+ ---
1229
+
1230
+ ## 7. CI Pipeline Integration
1231
+
1232
+ ### Basic: JSON output + fail on HIGH
1233
+
1234
+ ```yaml
1235
+ - name: Cloud Security Audit
1236
+ run: |
1237
+ python3 scripts/cloud_security_audit.py \
1238
+ --mode static --output json --severity high \
1239
+ > security-report.json
1240
+ # Exit code 1 = HIGH findings → fail pipeline
1241
+ ```
1242
+
1243
+ ### Recommended: SARIF + GitHub Advanced Security (inline PR annotations)
1244
+
1245
+ ```yaml
1246
+ - name: Cloud Security Audit
1247
+ run: |
1248
+ python3 scripts/cloud_security_audit.py \
1249
+ --mode static --output sarif --changed ${{ github.event.pull_request.base.sha }} \
1250
+ > results.sarif
1251
+ continue-on-error: true
1252
+
1253
+ - name: Upload SARIF
1254
+ uses: github/codeql-action/upload-sarif@v3
1255
+ with:
1256
+ sarif_file: results.sarif
1257
+ ```
1258
+
1259
+ ### Live mode with credentials (hardened — orchestration-review P0)
1260
+
1261
+ ```yaml
1262
+ - name: Cloud Security Audit (Live)
1263
+ env:
1264
+ GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
1265
+ run: |
1266
+ TMPFILE=$(mktemp -m 0600)
1267
+ trap 'rm -f "$TMPFILE"' EXIT
1268
+ echo "$GCP_SA_KEY" > "$TMPFILE"
1269
+ ai-toolkit credentials add gcp --file "$TMPFILE"
1270
+ python3 scripts/cloud_security_audit.py \
1271
+ --mode live --output sarif --changed "${{ github.event.pull_request.base.sha }}" \
1272
+ > results.sarif
1273
+
1274
+ - name: Validate & Upload SARIF
1275
+ if: always()
1276
+ run: python3 -c "import json; d=json.load(open('results.sarif')); assert d.get('version')=='2.1.0', 'Invalid SARIF'"
1277
+ continue-on-error: false
1278
+
1279
+ - name: Upload SARIF
1280
+ uses: github/codeql-action/upload-sarif@v3
1281
+ with:
1282
+ sarif_file: results.sarif
1283
+ ```
1284
+
1285
+ **CI Security Notes (orchestration-review):**
1286
+ - `mktemp -m 0600` creates file with owner-only permissions (not world-readable `/tmp/sa.json`)
1287
+ - `trap 'rm -f' EXIT` ensures cleanup even on script failure
1288
+ - SARIF schema validated before upload to prevent injected/corrupted annotations
1289
+ - `${{ github.event.pull_request.base.sha }}` quoted to prevent shell injection via crafted refs
1290
+ - Consider GitHub OIDC workload identity federation instead of long-lived SA keys for production
1291
+
1292
+ ---
1293
+
1294
+ ## 8. Success Criteria (Overall)
1295
+
1296
+ | Metric | Target |
1297
+ |--------|--------|
1298
+ | Providers supported | 3 (GCP, AWS, Azure) |
1299
+ | Check patterns (total) | 40+ (GCP: 16, AWS: 15, Azure: 12) |
1300
+ | False positive rules | 10+ |
1301
+ | Output formats | 3 (text, JSON, SARIF v2.1.0) |
1302
+ | CLI commands | 7 (add, list, remove, test, init per provider) |
1303
+ | Scripts (stdlib Python) | 9 (auth + scanners + resolver + sarif + incremental) |
1304
+ | Tests | 70+ |
1305
+ | External dependencies | 0 (stdlib only, CLI tools: gcloud/aws/az/terraform) |
1306
+ | CI exit codes | 0=clean, 1=HIGH findings, 2=credential error |
1307
+ | Incremental scan | `--changed` flag works with git refs |
1308
+ | GitHub integration | SARIF upload → inline PR annotations |
1309
+
1310
+ ---
1311
+
1312
+ ## 9. Fix Strategy
1313
+
1314
+ **Same approach as HIPAA scanner v1: No auto-fix. Agent interprets and suggests.**
1315
+
1316
+ The deterministic scripts produce findings. The `cloud-security-auditor` agent then:
1317
+ 1. **Reads** the flagged file/resource to understand actual context
1318
+ 2. **Suggests** a specific fix (not generic advice — concrete code/config change)
1319
+ 3. **Never** auto-applies changes — the user reviews and applies manually
1320
+
1321
+ Examples:
1322
+ - Firestore rules finding → agent suggests the exact `allow read: if request.auth != null` rule change
1323
+ - Public CF finding → agent suggests adding `validateFirebaseIdToken` middleware with code snippet
1324
+ - Open S3 bucket → agent suggests the exact bucket policy JSON to add `BlockPublicAccess`
1325
+ - Broad IAM role → agent suggests the minimal policy document with only required permissions
1326
+
1327
+ **Why no auto-fix in v1:**
1328
+ - Cloud security fixes require project-specific knowledge (which SA, which bucket, which auth flow)
1329
+ - Wrong auto-fix on IAM can lock out real users
1330
+ - Firestore rules changes can break client apps
1331
+ - The agent's context-aware suggestion is more valuable than a blind auto-fix
1332
+
1333
+ **v2 option:** `--fix-mode suggest` generates a `.cloud-security-fixes.patch` file that users can review and apply with `git apply`.
1334
+
1335
+ ---
1336
+
1337
+ ## 10. Risks and Mitigation (updated)
1338
+
1339
+ | Risk | Probability | Impact | Mitigation |
1340
+ |------|-------------|--------|------------|
1341
+ | CLI tools (gcloud/aws/az) not installed | Medium | Medium | Graceful fallback to static-only mode, clear error message |
1342
+ | Cloud APIs change breaking audit commands | Low | Medium | Version-pin CLI output format parsing, test with CI |
1343
+ | False positive rules too aggressive (suppress real issues) | Low | High | Default to WARN not SUPPRESS, require `.cloud-security-config` for suppression |
1344
+ | Credential leakage in logs | Low | Critical | Never log credentials, 0600 perms, `/tmp` cleanup in CI |
1345
+ | Scope creep (too many checks) | Medium | Medium | Start with top-10 per provider, expand based on feedback |
1346
+ | YAML IaC not parseable (stdlib-only) | High | Medium | JSON-only for static IaC; live mode unaffected; v2: vendored YAML subset parser |
1347
+ | `--changed` confused with live mode | Low | Low | Explicit warning: incremental applies to static only |
1348
+
1349
+ ---
1350
+
1351
+ ## 11. Pre-Mortem
1352
+
1353
+ 1. **Firebase rules parser too simplistic** — Firestore rules use CEL-like syntax with nested `match` blocks. Parser needs proper recursive descent, not just regex. Mitigation: build minimal CEL parser or use line-by-line pattern matching with scope tracking.
1354
+ 2. **False positive resolver gives false confidence** — Users may trust SUPPRESSED status and miss real issues. Mitigation: always show context chain, require explicit `.cloud-security-config` for auto-suppression, default to WARN.
1355
+ 3. **AWS credential scope too broad** — User provides admin-level AWS profile. Mitigation: `credentials test` checks actual permissions, WARN if write access detected, suggest read-only IAM policy in docs.
1356
+ 4. **Three providers = 3x maintenance** — Each provider's CLI evolves independently. Mitigation: abstract provider interface, single test matrix, version tracking per provider.
1357
+ 5. **Terraform parsing incomplete** — HCL syntax is complex (modules, variables, conditionals). Mitigation: wrap `terraform show -json` instead of parsing HCL. Fallback to flat regex for projects without `terraform` CLI.
1358
+ 6. **SARIF adoption low** — Users may not know how to use SARIF with GitHub. Mitigation: provide copy-paste GitHub Actions workflow in docs and `--explain` for onboarding.
1359
+
1360
+ ---
1361
+
1362
+ ## 12. Council Review Summary (2026-04-10)
1363
+
1364
+ **Verdict:** CONDITIONAL FOR — implement with scope reduction.
1365
+ **Confidence:** MEDIUM (weighted score: FOR 3.1 vs AGAINST 2.9)
1366
+
1367
+ **Key insights applied to this plan:**
1368
+ - [x] Timeline revised from 3-4 → 5-6 weeks
1369
+ - [x] ~~Azure deferred to Milestone 3~~ → **reinstated: full 3-provider delivery**
1370
+ - [x] SARIF v2.1.0 output added — essential for GitHub Advanced Security integration
1371
+ - [x] Incremental scan mode added (`--changed`) — how developers actually use security tools
1372
+ - [x] `terraform show -json` wrapper instead of HCL parsing — realistic path
1373
+ - [x] Single config file `.cloud-security.json` (merged ignore + context)
1374
+ - [x] `credentials init` interactive wizard — reduce onboarding friction
1375
+ - [x] `--explain <rule-id>` for on-demand remediation guidance
1376
+ - [x] Static mode as default — zero-setup first experience
1377
+
1378
+ **Deferred to v2:**
1379
+ - Kubernetes/container security (separate pack candidate)
1380
+ - Secret scanning with entropy detection
1381
+ - Compliance framework mapping (SOC2, PCI-DSS, NIST 800-53)
1382
+ - Visual security dashboard in browser
1383
+ - GitHub PR comment integration beyond SARIF
1384
+ - Vendored YAML subset parser for CloudFormation YAML static scanning
1385
+
1386
+ **Council strongest agreement:** False positive resolver is the killer feature and primary differentiator vs Checkov/Trivy/Prowler. No existing tool combines deterministic scanning with context-aware resolution.
1387
+
1388
+ ---
1389
+
1390
+ ## 13. Orchestration Review Summary (2026-04-10)
1391
+
1392
+ **Agents:** tech-lead, security-architect, product-manager, code-reviewer (4 parallel)
1393
+
1394
+ **Verdict:** Plan structurally complete (14/14 elements). Three P0 security blockers identified and resolved.
1395
+
1396
+ **Applied changes:**
1397
+
1398
+ | # | Action | Source | Priority | Applied? |
1399
+ |---|--------|--------|----------|----------|
1400
+ | 1 | Programmatic Bash allowlist | security-architect | P0 | Yes — allowlist.sh + agent integration |
1401
+ | 2 | CI hardening (mktemp/trap/SARIF validation) | security-architect | P0 | Yes — CI examples rewritten |
1402
+ | 3 | Suppression governance (justification + diff detection) | security-architect | P0 | Yes — schema + SUPPRESSION_CHANGED finding |
1403
+ | 4 | Recursive descent parser for Firestore | code-reviewer | P1 | Yes — replaced regex approach, +1d estimate |
1404
+ | 5 | SARIF `driver.rules[]` for GitHub annotations | code-reviewer | P1 | Yes — schema + success criteria |
1405
+ | 6 | Task 4.1→3.2 numbering fix | tech-lead | P1 | Yes — renumbered |
1406
+ | 7 | `terraform plan` execution risk documented | security-architect | P1 | Yes — only `show -json` allowed |
1407
+ | 8 | `credentials init` deferred to M2 | code-reviewer | P2 | Yes — saves 1.5d in M1 |
1408
+ | 9 | False positive resolver budgeted +2d/provider | code-reviewer | P2 | Yes — estimate updated |
1409
+ | 10 | `SUPPRESSION_CHANGED` finding type | security-architect | P2 | Yes — in governance section |
1410
+
1411
+ **Market positioning (product-manager):**
1412
+ - Not competing with Checkov on check count (40 vs 3000)
1413
+ - Competing on: zero-noise (false positive resolver), zero-setup (static-first), IDE-native (10 platforms), AI interpretation
1414
+ - Target: developers in ai-toolkit ecosystem, not enterprise security teams
1415
+ - Value as ecosystem feature, not standalone product
1416
+
1417
+ **Timeline revision (code-reviewer):**
1418
+ - 1 person: 6-7 weeks realistic (was 5-6)
1419
+ - 2 people: 4-5 weeks (parallel GCP + AWS tracks)
1420
+ - All 3 providers ship in 6 weeks — no conditional gates
1421
+
1422
+ ---
1423
+
1424
+ ## 14. Next Actions
1425
+
1426
+ 1. [ ] Approve plan
1427
+ 2. [ ] Implement `credentials` CLI command (1.1) + Bash allowlist
1428
+ 3. [ ] Create `cloud-security-auditor` agent (1.2)
1429
+ 4. [ ] Implement SARIF formatter + incremental scan (1.3)
1430
+ 5. [ ] Implement `firebase-rules-audit` — recursive descent parser (2.1)
1431
+ 6. [ ] Implement `cloud-functions-audit` + false positive resolver GCP (2.2, 2.3)
1432
+ 7. [ ] Implement `aws-security-audit` + `terraform show -json` wrapper (3.1)
1433
+ 8. [ ] Implement orchestrator + plugin pack + `credentials init` (3.2, 1.1b)
1434
+ 9. [ ] Implement `azure-security-audit` + false positive resolver Azure (4.1)
1435
+ 10. [ ] Full test suite (70+) + documentation + release
1436
+
1437
+ ---
1438
+
1439
+ **Last Updated:** 2026-04-10
1440
+ **Council Reviewed:** 2026-04-10
1441
+ **Orchestration Reviewed:** 2026-04-10 (4 agents: tech-lead, security-architect, product-manager, code-reviewer)
1442
+
1443
+ ---
1444
+
1445
+ ## kb/planning/enterprise-config-inheritance-plan.md
1446
+
1447
+ ---
1448
+ title: "Plan: Enterprise Config Inheritance — Multi-Repo Governance with extends"
1449
+ category: planning
1450
+ service: ai-toolkit
1451
+ tags:
1452
+ - enterprise
1453
+ - multi-repo
1454
+ - config-inheritance
1455
+ - extends
1456
+ - governance
1457
+ - team-management
1458
+ - monorepo
1459
+ doc_type: plan
1460
+ status: proposed
1461
+ created: "2026-04-10"
1462
+ last_updated: "2026-04-10"
1463
+ completion: "0%"
1464
+ description: "Configuration inheritance system for ai-toolkit. Enables organizations to define a shared base config (agents, rules, hooks, profiles, constitution overrides) published as an npm package or local path, which individual projects extend via an `extends` field. Changes to the base config propagate automatically on `ai-toolkit update`. Targets enterprises managing 10-100+ repositories with uniform AI governance."
1465
+ ---
1466
+
1467
+ # Plan: Enterprise Config Inheritance — Multi-Repo Governance with `extends`
1468
+
1469
+ **Status:** Proposed
1470
+ **Completion:** 0%
1471
+ **Created:** 2026-04-10
1472
+ **Origin:** Organizations adopting ai-toolkit across 10-100+ repositories face a config synchronization problem — updating a rule or policy requires touching every repository individually. The `extends` pattern (popularized by ESLint, TypeScript, Prettier) solves this by establishing a single source of truth that projects inherit from.
1473
+ **Estimated Effort:** 5-7 weeks (1 person) — MVP (core engine + install integration) shippable in ~3.5 weeks
1474
+
1475
+ ---
1476
+
1477
+ ## 1. Objective
1478
+
1479
+ Create a configuration inheritance system where projects can extend a shared base config published as an npm package, a Git URL, or a local path. The base config defines organizational defaults (which agents to enable, which rules to enforce, which hooks to require, persona presets, and constitution amendments). Individual projects can override or supplement the base, creating a layered governance model.
1480
+
1481
+ **Key design principles:**
1482
+ - **Familiar pattern** — mirrors ESLint's `extends`, TypeScript's `extends`, and Prettier's shared configs
1483
+ - **npm-first distribution** — base configs are regular npm packages (e.g., `@mycompany/ai-toolkit-config`). Resolver shells out to `npm pack` CLI (respects `.npmrc` auth) — no hand-rolled npm client, preserves stdlib-only constraint
1484
+ - **Single extends in v1** — `"extends": "string"` only. Multi-base merge (`"extends": [...]`) deferred to v2 to avoid merge-ordering complexity (ESLint's multi-extends is a known source of confusion)
1485
+ - **Layered merge** — base → project, with explicit override semantics (`override: true` required for safety-critical overrides)
1486
+ - **Constitution immutable** — base constitution articles cannot be modified by projects, period. Projects can only ADD new articles (article 6+). No weakening detection heuristics — absolute immutability is simpler and safer
1487
+ - **Offline-capable** — resolved at `install`/`update` time, not at runtime
1488
+ - **Backward-compatible** — projects without `extends` work exactly as today (no breaking changes)
1489
+ - **Audit trail** — `state.json` records which base config was resolved and what was overridden
1490
+
1491
+ ---
1492
+
1493
+ ## 1a. Functional Requirements
1494
+
1495
+ | ID | Requirement | Priority | Success Metric |
1496
+ |----|-------------|----------|----------------|
1497
+ | FR1 | Resolve `extends` from npm package, git URL, local path | Must | 4 source types work |
1498
+ | FR2 | Deep merge base → project config with layered semantics | Must | Merge engine handles dict, list, scalar types |
1499
+ | FR3 | Constitution immutability — Articles I-V cannot be modified | Must | 100% block rate on modification attempts |
1500
+ | FR4 | Override validation with `override: true` + `justification` | Must | Missing justification → error |
1501
+ | FR5 | `enforce` block constraints (minHookProfile, requiredPlugins, forbidOverride, requiredAgents) | Must | All 4 constraint types enforced |
1502
+ | FR6 | Install/update integration — resolve extends during install | Must | `install --local` detects `.ai-toolkit.json` |
1503
+ | FR7 | `config diff` command — show project vs base differences | Must | All merge layers visible |
1504
+ | FR8 | `config validate` command — schema + enforcement validation | Must | Exit 0/1 for pass/fail |
1505
+ | FR9 | `config init` — interactive project config setup | Should | Guided flow produces valid `.ai-toolkit.json` |
1506
+ | FR10 | `config create-base` — scaffold npm base config package | Should | Ready-to-publish package with `package.json` |
1507
+ | FR11 | Lock file for reproducible installs | Should | Identical resolved config across team members |
1508
+ | FR12 | Audit trail in `state.json` | Should | Resolved version + overrides recorded |
1509
+ | FR13 | CI enforcement command (`config check`) | Could | Exit 0/1 for governance compliance |
1510
+ | FR14 | Multi-base extends (`"extends": [...]`) | Won't (v2) | Deferred — merge ordering complexity |
1511
+
1512
+ ---
1513
+
1514
+ ## 2. Architecture Overview
1515
+
1516
+ ```
1517
+ Organization Level (published once, consumed by all repos):
1518
+ ═══════════════════════════════════════════════════════════
1519
+
1520
+ @mycompany/ai-toolkit-config (npm package)
1521
+ ├── ai-toolkit.config.json ← base configuration
1522
+ ├── rules/
1523
+ │ ├── code-review-policy.md ← company-specific rules
1524
+ │ └── deployment-checklist.md
1525
+ ├── agents/
1526
+ │ └── compliance-auditor.md ← company-specific agent
1527
+ └── package.json
1528
+
1529
+ Project Level (per-repository):
1530
+ ══════════════════════════════
1531
+
1532
+ my-service/
1533
+ ├── .ai-toolkit.json ← project config with "extends"
1534
+ ├── .claude/
1535
+ │ ├── CLAUDE.md ← generated (base + project merged)
1536
+ │ └── settings.json ← generated (base hooks + project hooks merged)
1537
+ └── ...
1538
+
1539
+ Merge Pipeline:
1540
+ ══════════════
1541
+
1542
+ @mycompany/ai-toolkit-config ← Layer 0: organizational defaults
1543
+
1544
+
1545
+ ai-toolkit defaults (manifest.json) ← Layer 1: toolkit defaults
1546
+
1547
+
1548
+ .ai-toolkit.json ← Layer 2: project overrides
1549
+
1550
+
1551
+ Resolved Configuration ← Final: CLAUDE.md, settings.json, etc.
1552
+ ```
1553
+
1554
+ ### Config Resolution Order
1555
+
1556
+ ```
1557
+ 1. Load base config from "extends" (npm package, git URL, or local path)
1558
+ 2. Merge with ai-toolkit defaults (manifest.json profiles)
1559
+ 3. Apply project-level overrides from .ai-toolkit.json
1560
+ 4. Validate merged config (constitution immutability, schema validation)
1561
+ 5. Generate output files (CLAUDE.md, settings.json, agent symlinks, etc.)
1562
+ ```
1563
+
1564
+ ---
1565
+
1566
+ ## 3. Progress Tracking
1567
+
1568
+ | # | Feature | Priority | Status | Est. Time | Notes |
1569
+ |---|---------|----------|--------|-----------|-------|
1570
+ | 1.1 | `.ai-toolkit.json` schema definition | P0 | Proposed | 1d | JSON Schema with `extends` field |
1571
+ | 1.2 | Config resolver (npm, git, local path) | P0 | Proposed | 3d | Fetch + cache + validate base configs |
1572
+ | 1.3 | Merge engine (layered merge with override semantics) | P0 | Proposed | 3d | Deep merge with `override: true` gates |
1573
+ | 1.4 | Constitution immutability guard | P0 | Proposed | 1d | Block weakening of safety articles |
1574
+ | 2.1 | Install/update integration | P0 | Proposed | 2d | Resolve extends during install/update |
1575
+ | 2.2 | `ai-toolkit config diff` command | P0 | Proposed | 1.5d | Show project vs base differences — primary debugging tool |
1576
+ | 2.3 | `ai-toolkit config validate` command | P0 | Proposed | 1d | Validate .ai-toolkit.json schema + extends resolution |
1577
+ | 2.4 | `ai-toolkit config init` command | P1 | Proposed | 1.5d | Interactive project config setup |
1578
+ | 2.5 | `ai-toolkit config create-base` command | P1 | Proposed | 2d | Scaffold base config package |
1579
+ | 3.1 | Audit trail in state.json | P1 | Proposed | 1d | Record resolved config provenance |
1580
+ | 3.2 | Lock file (`.ai-toolkit.lock.json`) | P1 | Proposed | 1.5d | Pin resolved versions for reproducibility |
1581
+ | 3.3 | Base config scaffolder (npm package template) | P1 | Proposed | 1.5d | Ready-to-publish template |
1582
+ | 3.4 | CI enforcement (`ai-toolkit config check`) | P2 | Proposed | 1d | Verify project adheres to base + no unapproved overrides |
1583
+ | 4.1 | Tests | P1 | Proposed | 3d | Unit: resolution, merge, immutability, override, CLI commands. Integration: `install --local` with `.ai-toolkit.json` containing `extends`, verify resolved `CLAUDE.md` has base + project rules merged end-to-end |
1584
+ | 4.2 | Documentation | P1 | Proposed | 3d | Enterprise setup guide + all 9 docs per CLAUDE.md: README, CLAUDE.md, ARCHITECTURE.md, package.json, llms.txt, llms-full.txt, AGENTS.md, skills-catalog.md, architecture-overview.md |
1585
+
1586
+ **Phasing (MVP-first):**
1587
+ - **MVP Phase 1 (week 1-2):** Core engine — schema (1.1), resolver (1.2), merge engine (1.3), constitution guard (1.4)
1588
+ - **MVP Phase 2 (week 2-3):** Integration + diff — install integration (2.1), `config diff` (2.2), `config validate` (2.3), tests for above (~3.5 weeks = shippable MVP)
1589
+ - **Phase 3 (week 4-5):** CLI polish — `config init` (2.4), `config create-base` (2.5), scaffolder (3.3)
1590
+ - **Phase 4 (week 5-6):** Enterprise — audit trail (3.1), lock file (3.2), CI enforcement (3.4) (**gate behind real enterprise feedback**)
1591
+ - **Phase 5 (week 6-7):** Tests + documentation (4.1, 4.2) (3d docs — all 9 docs per CLAUDE.md rules)
1592
+
1593
+ > **Demand validation gate:** Ship MVP (Phases 1-2), announce, measure adoption. Only build Phase 4 (lock file, CI enforcement, audit trail) in response to confirmed enterprise demand.
1594
+
1595
+ ---
1596
+
1597
+ ## 4. Dependency Graph
1598
+
1599
+ ```
1600
+ MVP Phase 1: Core Engine (week 1-2)
1601
+ ====================================
1602
+ Schema definition (1.1) ──────┐
1603
+ ├──► Merge engine (1.3)
1604
+ Config resolver (1.2) ────────┤
1605
+ └──► Constitution guard (1.4)
1606
+
1607
+ MVP Phase 2: Integration + Diff (week 2-3)
1608
+ ============================================
1609
+ Install integration (2.1) ──┐
1610
+ ├──► config diff (2.2)
1611
+ └──► config validate (2.3)
1612
+ └──► MVP tests → SHIP
1613
+
1614
+ ═══ DEMAND VALIDATION GATE ═══
1615
+
1616
+ Phase 3: CLI Polish (week 4-5)
1617
+ ===============================
1618
+ ├──► config init (2.4)
1619
+ └──► create-base (2.5) + scaffolder (3.3)
1620
+
1621
+ Phase 4: Enterprise (week 5-6)
1622
+ ================================
1623
+ Audit trail (3.1) ──┐
1624
+ ├──► Lock file (3.2)
1625
+ └──► CI enforcement (3.4)
1626
+
1627
+ Phase 5: Polish (week 6-7)
1628
+ ===========================
1629
+ └──► Full tests + docs (4.1, 4.2)
1630
+ ```
1631
+
1632
+ ---
1633
+
1634
+ ## 5. Detailed Implementation
1635
+
1636
+ ### Phase 1: Core Engine (week 1-2)
1637
+
1638
+ #### 1.1 Configuration Schema (`.ai-toolkit.json`)
1639
+
1640
+ > **v1 scope:** The full schema below shows the target state. v1 implements only: `extends`, `profile`, `agents`, `rules`, `constitution`, and `enforce`. See section 6a for the v1/v2 field breakdown.
1641
+
1642
+ **Project-level config file:**
1643
+
1644
+ ```json
1645
+ {
1646
+ "$schema": "https://softspark.github.io/ai-toolkit/schemas/ai-toolkit-config.json",
1647
+
1648
+ "extends": "@mycompany/ai-toolkit-config",
1649
+
1650
+ "profile": "standard",
1651
+ "persona": "backend-lead",
1652
+ "hookProfile": "strict",
1653
+
1654
+ "agents": {
1655
+ "enabled": ["backend-specialist", "test-engineer", "debugger"],
1656
+ "disabled": ["game-developer", "mobile-developer"],
1657
+ "custom": ["./agents/compliance-auditor.md"]
1658
+ },
1659
+
1660
+ "skills": {
1661
+ "disabled": ["/deploy", "/rollback"],
1662
+ "custom": ["./skills/internal-deploy/"]
1663
+ },
1664
+
1665
+ "rules": {
1666
+ "inject": ["./rules/code-review-policy.md"],
1667
+ "remove": []
1668
+ },
1669
+
1670
+ "plugins": {
1671
+ "required": ["security-pack", "memory-pack"],
1672
+ "forbidden": []
1673
+ },
1674
+
1675
+ "languages": ["typescript", "python"],
1676
+
1677
+ "editors": ["cursor", "windsurf", "copilot"],
1678
+
1679
+ "constitution": {
1680
+ "amendments": [
1681
+ {
1682
+ "article": 6,
1683
+ "title": "Data Sovereignty",
1684
+ "text": "All code generation must comply with GDPR. No personal data in prompts. No PII in generated code comments."
1685
+ }
1686
+ ]
1687
+ },
1688
+
1689
+ "overrides": {
1690
+ "hooks": {
1691
+ "quality-check": {
1692
+ "override": true,
1693
+ "justification": "Company uses custom lint pipeline via Jenkins",
1694
+ "replacement": "skip"
1695
+ }
1696
+ }
1697
+ }
1698
+ }
1699
+ ```
1700
+
1701
+ **Base config (`ai-toolkit.config.json` in npm package):**
1702
+
1703
+ ```json
1704
+ {
1705
+ "$schema": "https://softspark.github.io/ai-toolkit/schemas/ai-toolkit-base-config.json",
1706
+ "name": "@mycompany/ai-toolkit-config",
1707
+ "version": "2.1.0",
1708
+ "description": "MyCompany standard AI coding config",
1709
+
1710
+ "extends": null,
1711
+
1712
+ "profile": "strict",
1713
+ "persona": "backend-lead",
1714
+ "hookProfile": "strict",
1715
+
1716
+ "agents": {
1717
+ "enabled": ["backend-specialist", "test-engineer", "code-reviewer", "security-auditor", "debugger", "documenter"],
1718
+ "disabled": ["game-developer"],
1719
+ "custom": ["./agents/compliance-auditor.md"]
1720
+ },
1721
+
1722
+ "rules": {
1723
+ "inject": [
1724
+ "./rules/code-review-policy.md",
1725
+ "./rules/deployment-checklist.md",
1726
+ "./rules/data-handling-policy.md"
1727
+ ]
1728
+ },
1729
+
1730
+ "plugins": {
1731
+ "required": ["security-pack"]
1732
+ },
1733
+
1734
+ "languages": ["typescript"],
1735
+
1736
+ "constitution": {
1737
+ "amendments": [
1738
+ {
1739
+ "article": 6,
1740
+ "title": "Data Sovereignty",
1741
+ "text": "All code generation must comply with GDPR. No personal data in prompts."
1742
+ },
1743
+ {
1744
+ "article": 7,
1745
+ "title": "Audit Compliance",
1746
+ "text": "All AI-generated code changes must be logged to the company audit system. The governance-capture hook must remain enabled."
1747
+ }
1748
+ ]
1749
+ },
1750
+
1751
+ "enforce": {
1752
+ "minHookProfile": "standard",
1753
+ "requiredPlugins": ["security-pack"],
1754
+ "forbidOverride": ["constitution", "guard-destructive", "guard-path"],
1755
+ "requiredAgents": ["security-auditor"]
1756
+ }
1757
+ }
1758
+ ```
1759
+
1760
+ **`enforce` section:** Base configs can define non-overridable constraints:
1761
+ - `minHookProfile` — projects cannot go below this profile
1762
+ - `requiredPlugins` — must be installed in all projects
1763
+ - `forbidOverride` — these components cannot be overridden
1764
+ - `requiredAgents` — must be enabled in all projects
1765
+
1766
+ ---
1767
+
1768
+ #### 1.2 Config Resolver
1769
+
1770
+ **Resolution sources:**
1771
+
1772
+ | Source | Syntax | Resolution |
1773
+ |--------|--------|------------|
1774
+ | npm package | `"extends": "@mycompany/ai-toolkit-config"` | `npm pack --pack-destination /tmp` + extract |
1775
+ | npm with version | `"extends": "@mycompany/ai-toolkit-config@^2.0.0"` | Version resolution via npm |
1776
+ | Git URL | `"extends": "git+https://github.com/myco/ai-config.git"` | `git clone --depth 1` to cache |
1777
+ | Local path | `"extends": "../shared-config"` | Resolve relative to project root |
1778
+ | ~~Multiple bases~~ | ~~`"extends": ["@mycompany/base", "@mycompany/typescript-extra"]`~~ | Deferred to v2 — multi-base merge ordering is a complexity trap |
1779
+
1780
+ **Cache directory:** `~/.ai-toolkit/config-cache/`
1781
+ ```
1782
+ ~/.ai-toolkit/config-cache/
1783
+ @mycompany/
1784
+ ai-toolkit-config/
1785
+ 2.1.0/
1786
+ ai-toolkit.config.json
1787
+ rules/
1788
+ agents/
1789
+ ```
1790
+
1791
+ **Resolution algorithm:**
1792
+ ```python
1793
+ def resolve_extends(extends_value: str, project_root: str) -> list[BaseConfig]:
1794
+ """Resolve extends chain into ordered list of base configs.
1795
+
1796
+ v1: single string only. Multi-base (list) deferred to v2.
1797
+ """
1798
+ configs = []
1799
+ for source in [extends_value]: # v2: support list[str]
1800
+ if source.startswith('@') or source.startswith('npm:'):
1801
+ config = resolve_npm(source)
1802
+ elif source.startswith('git+'):
1803
+ config = resolve_git(source)
1804
+ elif source.startswith('.') or source.startswith('/'):
1805
+ config = resolve_local(source, project_root)
1806
+ else:
1807
+ raise ConfigError(f"Unknown extends source: {source}")
1808
+
1809
+ # Recursive: base config may also have "extends"
1810
+ if config.extends:
1811
+ parent_configs = resolve_extends(config.extends, config.root)
1812
+ configs.extend(parent_configs)
1813
+
1814
+ configs.append(config)
1815
+
1816
+ return configs
1817
+
1818
+
1819
+ def resolve_extends(extends_value: str, project_root: str,
1820
+ _visited: set[str] | None = None) -> list[BaseConfig]:
1821
+ """Full signature with cycle detection via visited set."""
1822
+ if _visited is None:
1823
+ _visited = set()
1824
+ if extends_value in _visited:
1825
+ raise ConfigError(
1826
+ f"Circular extends detected: {extends_value} already in chain "
1827
+ f"{' → '.join(_visited)}. Check your base config's 'extends' field."
1828
+ )
1829
+ if len(_visited) >= 5:
1830
+ raise ConfigError(
1831
+ f"Extends chain too deep (max 5 levels). Chain: {' → '.join(_visited)}"
1832
+ )
1833
+ _visited.add(extends_value)
1834
+ # ... resolution logic as above, passing _visited to recursive calls
1835
+ ```
1836
+
1837
+ **Max recursion depth:** 5 levels (prevent circular extends). Circular detection via visited set.
1838
+
1839
+ **Offline handling:** If the npm/git source is unavailable:
1840
+ 1. Check cache (`~/.ai-toolkit/config-cache/`)
1841
+ 2. If cached version found → use with warning: "Using cached config v2.1.0 (offline)"
1842
+ 3. If not cached → error with instructions: "Run `ai-toolkit config update` when online"
1843
+
1844
+ ---
1845
+
1846
+ #### 1.3 Merge Engine
1847
+
1848
+ **Layered deep merge with explicit override semantics:**
1849
+
1850
+ ```python
1851
+ def merge_configs(base: dict, project: dict) -> dict:
1852
+ """Merge project config over base config with rules."""
1853
+ merged = {}
1854
+
1855
+ for key in set(base.keys()) | set(project.keys()):
1856
+ base_val = base.get(key)
1857
+ proj_val = project.get(key)
1858
+
1859
+ if proj_val is None:
1860
+ merged[key] = base_val
1861
+ elif base_val is None:
1862
+ merged[key] = proj_val
1863
+ elif key == 'constitution':
1864
+ merged[key] = merge_constitution(base_val, proj_val)
1865
+ elif key == 'agents':
1866
+ merged[key] = merge_agents(base_val, proj_val)
1867
+ elif key == 'rules':
1868
+ merged[key] = merge_rules(base_val, proj_val)
1869
+ elif key == 'overrides':
1870
+ merged[key] = validate_overrides(base, proj_val)
1871
+ elif isinstance(base_val, dict) and isinstance(proj_val, dict):
1872
+ merged[key] = merge_configs(base_val, proj_val)
1873
+ elif isinstance(base_val, list) and isinstance(proj_val, list):
1874
+ merged[key] = list(set(base_val + proj_val)) # union
1875
+ else:
1876
+ merged[key] = proj_val # project wins for scalars
1877
+
1878
+ return merged
1879
+ ```
1880
+
1881
+ **Agent merge rules:**
1882
+ ```python
1883
+ def merge_agents(base: dict, project: dict) -> dict:
1884
+ """Merge agent configs — project can enable/disable but not remove base-required."""
1885
+ merged_enabled = set(base.get('enabled', []))
1886
+
1887
+ # Project can add agents
1888
+ merged_enabled.update(project.get('enabled', []))
1889
+
1890
+ # Project can disable agents (unless base enforces them)
1891
+ for agent in project.get('disabled', []):
1892
+ if agent in base.get('enforce', {}).get('requiredAgents', []):
1893
+ raise ConfigError(
1894
+ f"Cannot disable '{agent}' — required by base config '{base['name']}'. "
1895
+ f"Contact your team lead to request an exemption."
1896
+ )
1897
+ merged_enabled.discard(agent)
1898
+
1899
+ return {
1900
+ 'enabled': sorted(merged_enabled),
1901
+ 'custom': base.get('custom', []) + project.get('custom', [])
1902
+ }
1903
+ ```
1904
+
1905
+ **Override validation:**
1906
+ ```python
1907
+ def validate_overrides(base: dict, overrides: dict) -> dict:
1908
+ """Validate project overrides against base enforcement rules."""
1909
+ forbidden = set(base.get('enforce', {}).get('forbidOverride', []))
1910
+
1911
+ for key, override in overrides.items():
1912
+ if key in forbidden:
1913
+ raise ConfigError(
1914
+ f"Cannot override '{key}' — forbidden by base config '{base['name']}'.\n"
1915
+ f"Forbidden overrides: {', '.join(sorted(forbidden))}\n"
1916
+ f"Contact your team lead to request an exemption."
1917
+ )
1918
+ if not override.get('override'):
1919
+ raise ConfigError(
1920
+ f"Override for '{key}' requires explicit 'override: true' + 'justification' field.\n"
1921
+ f"This ensures intentional deviation from organizational defaults."
1922
+ )
1923
+ if not override.get('justification'):
1924
+ raise ConfigError(
1925
+ f"Override for '{key}' requires a 'justification' field explaining why.\n"
1926
+ f"Example: \"Company uses custom lint pipeline via Jenkins\""
1927
+ )
1928
+
1929
+ return overrides
1930
+ ```
1931
+
1932
+ ---
1933
+
1934
+ #### 1.4 Constitution Immutability Guard
1935
+
1936
+ **Core rule:** Base constitution articles are absolutely immutable. Projects can only ADD new articles.
1937
+
1938
+ No weakening-detection heuristic (character count, semantic analysis) — these produce false positives and are gameable. Instead, the rule is simple and absolute: if an article number exists in the base, it cannot be modified by the project.
1939
+
1940
+ ```python
1941
+ def merge_constitution(base: dict, project: dict) -> dict:
1942
+ """Merge constitution — additions only, no modifications."""
1943
+ base_amendments = {a['article']: a for a in base.get('amendments', [])}
1944
+ proj_amendments = {a['article']: a for a in project.get('amendments', [])}
1945
+
1946
+ # Toolkit articles I-V are always immutable
1947
+ IMMUTABLE_ARTICLES = {1, 2, 3, 4, 5}
1948
+
1949
+ merged = dict(base_amendments)
1950
+
1951
+ for article_num, amendment in proj_amendments.items():
1952
+ if article_num in IMMUTABLE_ARTICLES:
1953
+ raise ConfigError(
1954
+ f"Cannot modify Constitution Article {article_num} — immutable.\n"
1955
+ f"Articles I-V are defined by ai-toolkit and cannot be overridden.\n"
1956
+ f"You can ADD new articles (article 6+)."
1957
+ )
1958
+ if article_num in base_amendments:
1959
+ # Base articles are immutable — projects cannot modify them
1960
+ raise ConfigError(
1961
+ f"Cannot modify Constitution Article {article_num} — "
1962
+ f"defined by base config '{base.get('name', 'unknown')}'.\n"
1963
+ f"Base articles are immutable. You can ADD new articles "
1964
+ f"with a higher article number."
1965
+ )
1966
+ merged[article_num] = amendment
1967
+
1968
+ return {'amendments': list(merged.values())}
1969
+ ```
1970
+
1971
+ ---
1972
+
1973
+ ### MVP Phase 2: Integration + Diff (week 2-3)
1974
+
1975
+ #### 2.1 Install/Update Integration
1976
+
1977
+ **Modified `install.py` flow:**
1978
+
1979
+ ```python
1980
+ # During install --local:
1981
+ # 1. Check for .ai-toolkit.json in project root
1982
+ # 2. If found and has "extends":
1983
+ # a. Resolve base config(s)
1984
+ # b. Merge base → project
1985
+ # c. Validate merged config
1986
+ # d. Generate files from merged config
1987
+ # 3. If not found: proceed with current behavior (backwards compatible)
1988
+ ```
1989
+
1990
+ **CLI flags:**
1991
+ ```bash
1992
+ ai-toolkit install --local # auto-detect .ai-toolkit.json
1993
+ ai-toolkit install --local --config ./custom.json # explicit config file
1994
+ ai-toolkit update --local # re-resolve extends + update
1995
+ ai-toolkit update --local --refresh-base # force re-fetch base config
1996
+ ```
1997
+
1998
+ ---
1999
+
2000
+ #### 2.2 `ai-toolkit config diff`
2001
+
2002
+ **Show differences between project config and base:**
2003
+
2004
+ ```bash
2005
+ ai-toolkit config diff
2006
+
2007
+ # Output:
2008
+ # Base: @mycompany/ai-toolkit-config@2.1.0
2009
+ #
2010
+ # Profile: strict (base) → standard (project) ⚠ OVERRIDE
2011
+ # Persona: backend-lead (base) → frontend-lead (project)
2012
+ # Hook Profile: strict (base) → strict (inherited)
2013
+ #
2014
+ # Agents:
2015
+ # + frontend-specialist (project adds)
2016
+ # - game-developer (base disables)
2017
+ # = security-auditor (base requires, cannot disable)
2018
+ #
2019
+ # Rules:
2020
+ # + ./rules/api-standards.md (project adds)
2021
+ # = code-review-policy.md (inherited from base)
2022
+ #
2023
+ # Constitution:
2024
+ # = Articles I-V (immutable)
2025
+ # = Article 6: Data Sovereignty (inherited from base)
2026
+ # + Article 8: API Standards (project adds)
2027
+ #
2028
+ # Overrides:
2029
+ # quality-check: SKIP (justification: "Custom Jenkins pipeline")
2030
+ ```
2031
+
2032
+ ---
2033
+
2034
+ #### 2.3 `ai-toolkit config validate`
2035
+
2036
+ ```bash
2037
+ ai-toolkit config validate
2038
+
2039
+ # Checks:
2040
+ # ✓ .ai-toolkit.json schema valid
2041
+ # ✓ extends: @mycompany/ai-toolkit-config@2.1.0 resolved
2042
+ # ✓ No forbidden overrides
2043
+ # ✓ Required plugins installed: security-pack
2044
+ # ✓ Required agents enabled: security-auditor
2045
+ # ✓ Constitution articles I-V intact
2046
+ # ✓ Hook profile meets minimum: standard ≥ standard
2047
+ # ✓ All custom rule files exist
2048
+ # ✓ All custom agent files exist
2049
+ ```
2050
+
2051
+ ---
2052
+
2053
+ ### Phase 3: CLI Polish (week 4-5)
2054
+
2055
+ #### 2.4 `ai-toolkit config init`
2056
+
2057
+ **Interactive project config setup:**
2058
+
2059
+ ```bash
2060
+ ai-toolkit config init
2061
+
2062
+ # Flow:
2063
+ # 1. "Does your organization have a shared ai-toolkit config? [y/n]"
2064
+ # → y: "npm package name or git URL:" → resolves + validates
2065
+ # → n: creates minimal .ai-toolkit.json without extends
2066
+ # 2. "Which profile? [minimal/standard/strict]" → default from base or standard
2067
+ # 3. "Which persona? [none/backend-lead/frontend-lead/devops-eng/junior-dev]"
2068
+ # 4. Auto-detect languages from project
2069
+ # 5. Auto-detect editors from project files
2070
+ # 6. Write .ai-toolkit.json
2071
+ # 7. Run ai-toolkit install --local
2072
+ ```
2073
+
2074
+ ---
2075
+
2076
+ #### 2.5 `ai-toolkit config create-base`
2077
+
2078
+ **Scaffold a base config package:**
2079
+
2080
+ ```bash
2081
+ ai-toolkit config create-base @mycompany/ai-toolkit-config
2082
+
2083
+ # Creates:
2084
+ # @mycompany-ai-toolkit-config/
2085
+ # ├── package.json (name, version, files, peerDependencies)
2086
+ # ├── ai-toolkit.config.json (base config with sane defaults)
2087
+ # ├── rules/ (empty, ready for company rules)
2088
+ # ├── agents/ (empty, ready for company agents)
2089
+ # └── README.md (setup instructions)
2090
+ ```
2091
+
2092
+ **Generated `package.json`:**
2093
+ ```json
2094
+ {
2095
+ "name": "@mycompany/ai-toolkit-config",
2096
+ "version": "1.0.0",
2097
+ "description": "Shared ai-toolkit configuration for MyCompany",
2098
+ "main": "ai-toolkit.config.json",
2099
+ "files": ["ai-toolkit.config.json", "rules/", "agents/"],
2100
+ "peerDependencies": {
2101
+ "@softspark/ai-toolkit": ">=1.5.0"
2102
+ },
2103
+ "keywords": ["ai-toolkit", "config", "shared"]
2104
+ }
2105
+ ```
2106
+
2107
+ ---
2108
+
2109
+ ### Phase 4: Enterprise Features (week 5-6)
2110
+
2111
+ #### 3.1 Audit Trail
2112
+
2113
+ **`state.json` additions:**
2114
+ ```json
2115
+ {
2116
+ "extends": {
2117
+ "source": "@mycompany/ai-toolkit-config",
2118
+ "version": "2.1.0",
2119
+ "resolved_at": "2026-04-10T10:30:00Z",
2120
+ "hash": "sha256:abc123...",
2121
+ "overrides_applied": [
2122
+ {
2123
+ "key": "hooks.quality-check",
2124
+ "action": "skip",
2125
+ "justification": "Custom Jenkins pipeline"
2126
+ }
2127
+ ]
2128
+ }
2129
+ }
2130
+ ```
2131
+
2132
+ ---
2133
+
2134
+ #### 3.2 Lock File (`.ai-toolkit.lock.json`)
2135
+
2136
+ **Purpose:** Pin the exact resolved version of base configs for reproducible installs across team members and CI.
2137
+
2138
+ ```json
2139
+ {
2140
+ "lockfileVersion": 1,
2141
+ "resolved": {
2142
+ "@mycompany/ai-toolkit-config": {
2143
+ "version": "2.1.0",
2144
+ "resolved": "https://registry.npmjs.org/@mycompany/ai-toolkit-config/-/ai-toolkit-config-2.1.0.tgz",
2145
+ "integrity": "sha512-abc123...",
2146
+ "cached": "~/.ai-toolkit/config-cache/@mycompany/ai-toolkit-config/2.1.0/"
2147
+ }
2148
+ },
2149
+ "generated_at": "2026-04-10T10:30:00Z",
2150
+ "ai_toolkit_version": "1.5.1"
2151
+ }
2152
+ ```
2153
+
2154
+ **Behavior:**
2155
+ - `ai-toolkit install --local` → uses lock file if present (like `npm ci`)
2156
+ - `ai-toolkit update --local` → re-resolves and updates lock file (like `npm install`)
2157
+ - `ai-toolkit update --local --refresh-base` → force re-fetch ignoring cache
2158
+ - `.ai-toolkit.lock.json` should be committed to git (team synchronization)
2159
+
2160
+ ---
2161
+
2162
+ #### 3.4 CI Enforcement
2163
+
2164
+ **`ai-toolkit config check` — for CI pipelines:**
2165
+
2166
+ ```bash
2167
+ ai-toolkit config check
2168
+
2169
+ # Exit codes:
2170
+ # 0 — project complies with base config
2171
+ # 1 — violations found (missing required plugins, forbidden overrides, etc.)
2172
+ # 2 — .ai-toolkit.json not found
2173
+ ```
2174
+
2175
+ **GitHub Actions example:**
2176
+ ```yaml
2177
+ - name: AI Toolkit Governance Check
2178
+ run: |
2179
+ npx @softspark/ai-toolkit config check
2180
+ npx @softspark/ai-toolkit config validate --strict
2181
+ ```
2182
+
2183
+ **What it checks:**
2184
+ 1. Required plugins are installed
2185
+ 2. Required agents are enabled
2186
+ 3. No forbidden overrides applied without exemption
2187
+ 4. Hook profile meets minimum
2188
+ 5. Constitution articles intact
2189
+ 6. Lock file up-to-date (warn if stale)
2190
+
2191
+ ---
2192
+
2193
+ ## 6. File Summary
2194
+
2195
+ | File | Action | LOC (est.) | Description |
2196
+ |------|--------|------------|-------------|
2197
+ | `scripts/config_resolver.py` | CREATE | ~400 | Resolve extends (npm, git, local path) |
2198
+ | `scripts/config_merger.py` | CREATE | ~350 | Layered merge engine |
2199
+ | `scripts/config_validator.py` | CREATE | ~200 | Schema + enforcement validation |
2200
+ | `scripts/config_scaffold.py` | CREATE | ~250 | create-base scaffolder |
2201
+ | `scripts/config_diff.py` | CREATE | ~200 | Diff viewer |
2202
+ | `scripts/config_check.py` | CREATE | ~150 | CI enforcement checker |
2203
+ | `scripts/install.py` | EDIT | +80 | Integrate extends resolution |
2204
+ | `bin/ai-toolkit.js` | EDIT | +40 | Register config subcommands |
2205
+ | `manifest.json` | EDIT | +10 | Schema references |
2206
+ | `kb/reference/enterprise-config-guide.md` | CREATE | ~300 | Enterprise setup guide |
2207
+ | `kb/reference/base-config-template/` | CREATE | ~200 | Scaffolded base config files |
2208
+ | `tests/test_config_resolver.bats` | CREATE | ~150 | Resolution tests |
2209
+ | `tests/test_config_merger.bats` | CREATE | ~200 | Merge + override tests |
2210
+ | `tests/test_config_immutability.bats` | CREATE | ~100 | Constitution guard tests |
2211
+ | `tests/test_config_cli.bats` | CREATE | ~150 | CLI command tests |
2212
+ | **Total** | | **~2780** | |
2213
+
2214
+ ---
2215
+
2216
+ ## 6a. Schema Scope (v1 vs v2)
2217
+
2218
+ v1 ships with a minimal schema. Each additional field adds merge logic, validation, diff output, and test surface. Expand based on real usage, not speculation.
2219
+
2220
+ | Field | v1 | v2 | Rationale |
2221
+ |-------|----|----|-----------|
2222
+ | `extends` | single string | array (multi-base) | Multi-base merge ordering is complex |
2223
+ | `profile` | yes | — | Core governance knob |
2224
+ | `agents` | yes | — | Most common customization |
2225
+ | `rules` | yes | — | Rule injection is existing feature |
2226
+ | `constitution` | yes | — | Key differentiator |
2227
+ | `enforce` | yes | — | Non-overridable constraints |
2228
+ | `skills` | — | yes | Less commonly customized at org level |
2229
+ | `plugins` | — | yes | Depends on plugin maturity |
2230
+ | `languages` | — | yes | Auto-detected, rarely org-level |
2231
+ | `editors` | — | yes | Auto-detected, rarely org-level |
2232
+ | `overrides` | — | yes | Complex, needs real-world feedback |
2233
+ | `hookProfile` / `persona` | — | yes | Low demand signal |
2234
+
2235
+ ---
2236
+
2237
+ ## 6b. Non-Functional Requirements
2238
+
2239
+ | Category | Requirement |
2240
+ |----------|-------------|
2241
+ | **Performance** | `install --local` with extends resolution < 5s (cached), < 15s (first fetch). Config merge < 100ms. |
2242
+ | **Offline** | Cached configs used when registry unavailable, with clear warning. |
2243
+ | **Security** | No secret exposure in config files or audit trail. npm auth via `.npmrc` (user-managed). `execFile` for npm CLI (no shell injection). |
2244
+ | **Error messages** | Every validation error includes: what failed, which config layer caused it, and what to do (e.g., "Contact your team lead to request an exemption"). |
2245
+ | **Backward compatibility** | 100% — projects without `.ai-toolkit.json` work exactly as today. Zero behavioral changes for existing users. |
2246
+ | **Maintainability** | Each new schema field requires: merge logic, validation, diff output, test. Budget 0.5d per new field. |
2247
+ | **Quality gates** | `ruff check scripts/config_*.py` (0 errors), `mypy --strict scripts/config_*.py` (0 errors). Run before every commit. |
2248
+ | **Type safety** | 100% public API type hints (all function signatures). >60% internal. Use `TypedDict` for config schemas, `dataclass` for resolved configs. |
2249
+
2250
+ ---
2251
+
2252
+ ## 7. Success Criteria (Overall)
2253
+
2254
+ | Metric | Target |
2255
+ |--------|--------|
2256
+ | Extends sources (v1) | 4 (npm, npm+version, git URL, local path) — single string only |
2257
+ | Merge depth | 5 levels max (recursive extends) |
2258
+ | Config schema | JSON Schema validated |
2259
+ | Constitution protection | 100% (Articles I-V immutable) |
2260
+ | Override justification | Required for all overrides |
2261
+ | Enforce constraints | 4 types (minHookProfile, requiredPlugins, forbidOverride, requiredAgents) |
2262
+ | Backward compatibility | 100% (projects without .ai-toolkit.json work as today) |
2263
+ | CI enforcement | Exit code 0/1 for governance compliance |
2264
+ | Lock file | Reproducible installs across team members |
2265
+ | Scaffold command | Ready-to-publish npm package template |
2266
+ | Tests | 30+ |
2267
+ | Offline resolution | Cached configs with warning |
2268
+
2269
+ ---
2270
+
2271
+ ## 8. Risks and Mitigation
2272
+
2273
+ | Risk | Probability | Impact | Mitigation |
2274
+ |------|-------------|--------|------------|
2275
+ | npm registry unavailable during install | Low | Medium | Cache + offline fallback with warning |
2276
+ | Circular extends chain | Low | High | Max depth 5 + visited set for cycle detection |
2277
+ | Base config breaks project | Medium | High | Lock file pins exact version; `ai-toolkit config diff` shows changes before update |
2278
+ | Override abuse (teams bypass governance) | Medium | Medium | `enforce.forbidOverride` + CI check + justification requirement |
2279
+ | Config schema too restrictive | Medium | Medium | Start with minimal enforcement, expand based on enterprise feedback |
2280
+ | Multiple base configs conflict | — | — | Deferred to v2 (single extends only in v1) |
2281
+ | Private npm registry authentication | Medium | Low | Use existing npm auth (`.npmrc`), document setup |
2282
+ | Git URL resolution slow | Low | Low | `--depth 1` clone, cache aggressively |
2283
+
2284
+ ---
2285
+
2286
+ ## 9. Pre-Mortem
2287
+
2288
+ 1. **"Config file fatigue"** — developers already have `.eslintrc`, `tsconfig.json`, `.prettierrc`. Another `.ai-toolkit.json` may feel like bloat. Mitigation: file is optional, all features work without it. The DX gain (organizational governance without per-repo updates) justifies the file.
2289
+ 2. **"Base config never gets updated"** — team lead creates base config, nobody maintains it. Mitigation: `ai-toolkit config check` in CI catches drift; lock file staleness warnings.
2290
+ 3. **"Override justification is annoying"** — developers will write "needed" as justification. Mitigation: CI check can enforce minimum justification length (>20 chars); code review culture catches low-effort justifications.
2291
+ 4. **"Merge semantics are confusing"** — "does project override or extend the base agent list?" Mitigation: explicit semantics documented in schema; `ai-toolkit config diff` shows exactly what happened.
2292
+ 5. **"Enterprise teams want RBAC on overrides"** — who can approve overrides? Mitigation: v1 uses justification text + code review; v2 could integrate with GitHub CODEOWNERS for override approval.
2293
+
2294
+ ---
2295
+
2296
+ ## 10. Market Positioning
2297
+
2298
+ **Target users:**
2299
+ 1. **Engineering managers** — enforce AI coding standards across 20+ repos without touching each one
2300
+ 2. **Security teams** — ensure constitution + security-auditor agent is always enabled
2301
+ 3. **Platform teams** — distribute company-specific agents, rules, and plugins via npm
2302
+ 4. **Compliance officers** — audit trail of what AI governance rules are active in each project
2303
+
2304
+ **Competitive advantage:** No existing AI coding toolkit supports configuration inheritance. This is a unique enterprise feature that transforms ai-toolkit from a developer tool into an organizational governance platform.
2305
+
2306
+ **Revenue potential:** Enterprise teams are the primary audience for paid support/consulting around ai-toolkit. Config inheritance is the feature that makes enterprise adoption manageable.
2307
+
2308
+ ---
2309
+
2310
+ ## 11. Next Actions
2311
+
2312
+ **MVP (ship first, ~3.5 weeks):**
2313
+ 1. [ ] Approve plan
2314
+ 2. [ ] Define `.ai-toolkit.json` JSON Schema — v1 scope only (1.1)
2315
+ 3. [ ] Implement config resolver (npm, git, local) with caching (1.2)
2316
+ 4. [ ] Implement merge engine with override validation (1.3)
2317
+ 5. [ ] Implement constitution immutability guard (1.4)
2318
+ 6. [ ] Integrate into install.py flow (2.1)
2319
+ 7. [ ] Create `config diff` viewer (2.2) — primary debugging tool
2320
+ 8. [ ] Create `config validate` checker (2.3)
2321
+ 9. [ ] Tests for above (4.1 partial)
2322
+ 10. [ ] **Ship MVP → announce → measure adoption**
2323
+
2324
+ **Post-MVP (if demand validated):**
2325
+ 11. [ ] Create `config init` interactive command (2.4)
2326
+ 12. [ ] Create `config create-base` scaffolder (2.5)
2327
+ 13. [ ] Add audit trail to state.json (3.1)
2328
+ 14. [ ] Implement lock file generation + resolution (3.2)
2329
+ 15. [ ] Create base config npm package template (3.3)
2330
+ 16. [ ] Create CI enforcement command `config check` (3.4)
2331
+ 17. [ ] Full tests + documentation — all 9 docs per CLAUDE.md (4.1, 4.2)
2332
+
2333
+ ---
2334
+
2335
+ ## 12. Future (v2)
2336
+
2337
+ | Feature | Rationale |
2338
+ |---------|-----------|
2339
+ | Multi-base extends (`"extends": [...]`) | Needs real-world feedback on merge ordering UX |
2340
+ | v1 deferred schema fields (skills, plugins, languages, editors, overrides, hookProfile, persona) | Expand based on actual enterprise requests |
2341
+ | RBAC on overrides (GitHub CODEOWNERS integration) | v1 uses justification + code review |
2342
+ | Semantic constitution analysis | Character-count heuristics removed in v1; revisit only if absolute immutability proves too restrictive |
2343
+ | `ai-toolkit config audit` (full governance report) | Depends on audit trail maturity |
2344
+
2345
+ ---
2346
+
2347
+ ## 13. Cross-Plan Dependencies
2348
+
2349
+ This plan shares modification targets with two other proposed plans:
2350
+
2351
+ | Shared File | This Plan | Local Dashboard Plan | Offline SLM Plan |
2352
+ |-------------|-----------|---------------------|-----------------|
2353
+ | `scripts/install.py` | +80 LOC (extends resolution) | — | +30 LOC (offline-slm profile) |
2354
+ | `manifest.json` | +10 LOC (schema refs) | — | +5 LOC (offline-slm profile) |
2355
+ | `bin/ai-toolkit.js` | +40 LOC (config subcommands) | +15 LOC (ui command) | +10 LOC (compile-slm command) |
2356
+
2357
+ **If implementing in parallel:** coordinate merge order for shared files. Recommended sequence: Offline SLM (smallest changes) → Enterprise Config → Dashboard (no install.py changes).
2358
+
2359
+ **Dashboard integration note:** If this plan ships before the Dashboard plan, the Dashboard's Config page (2.6) should display `.ai-toolkit.json` / `extends` status and the `config diff` output.
2360
+
2361
+ ---
2362
+
2363
+ **Last Updated:** 2026-04-10
2364
+
2365
+ ---
2366
+
2367
+ ## kb/planning/local-dashboard-plan.md
2368
+
2369
+ ---
2370
+ title: "Plan: Local Dashboard — ai-toolkit ui"
2371
+ category: planning
2372
+ service: ai-toolkit
2373
+ tags:
2374
+ - dashboard
2375
+ - developer-experience
2376
+ - web-ui
2377
+ - tui
2378
+ - configuration
2379
+ - visualization
2380
+ doc_type: plan
2381
+ status: proposed
2382
+ created: "2026-04-10"
2383
+ last_updated: "2026-04-10"
2384
+ completion: "0%"
2385
+ description: "Ephemeral local web dashboard for ai-toolkit. Provides visual management of agents, skills, hooks, plugins, stats, credentials, and configuration profiles. Zero external dependencies — stdlib Node.js server with embedded HTML/CSS/JS. Launched via `ai-toolkit ui`."
2386
+ ---
2387
+
2388
+ # Plan: Local Dashboard — `ai-toolkit ui`
2389
+
2390
+ **Status:** Proposed
2391
+ **Completion:** 0%
2392
+ **Created:** 2026-04-10
2393
+ **Origin:** DX friction — managing 44 agents, 91 skills, 21 hooks, 11 plugin packs, and multiple profiles via CLI only creates a steep learning curve for new adopters. Visual management lowers the adoption barrier.
2394
+ **Estimated Effort:** 6-7 weeks (1 person)
2395
+
2396
+ ---
2397
+
2398
+ ## 1. Objective
2399
+
2400
+ Create an `ai-toolkit ui` command that launches an ephemeral local HTTP server serving a single-page web dashboard. The dashboard provides visual management of all toolkit components — no more memorizing CLI flags.
2401
+
2402
+ **Key design principles:**
2403
+ - **Zero external dependencies** — stdlib Node.js `http` module, embedded HTML/CSS/JS (same approach as `visual-server.cjs` in `/write-a-prd`). Server lives in `app/dashboard/` (not `scripts/`) — aligns with `visual-server.cjs` precedent and the "scripts/ = Python stdlib only" convention
2404
+ - **Ephemeral** — auto-kills after 30 minutes of inactivity (matching existing companion pattern)
2405
+ - **Read-write** — reads state from `~/.ai-toolkit/state.json`, `~/.claude/settings.json`, `manifest.json`; writes config changes through existing CLI commands (never mutates files directly)
2406
+ - **Dark theme** — premium aesthetic, responsive, glassmorphism accents
2407
+ - **Offline** — no CDN dependencies, no external fonts, no analytics
2408
+ - **Port discovery** — starts on 3141, auto-increments if busy
2409
+
2410
+ ---
2411
+
2412
+ ## 1a. Functional Requirements
2413
+
2414
+ | ID | Requirement | Priority | Success Metric |
2415
+ |----|-------------|----------|----------------|
2416
+ | FR1 | HTTP server with auto-kill + port discovery | Must | Starts, auto-kills after 30 min, increments port if busy |
2417
+ | FR2 | Read API endpoints (status, agents, skills, hooks, plugins, stats, config, mcp) | Must | All 8 endpoints return valid JSON |
2418
+ | FR3 | Action execution via `POST /api/action` with SSE streaming | Must | Command output streamed, exit code returned |
2419
+ | FR4 | Overview page with health checks + component counts | Must | Live data, matches `ai-toolkit validate` output |
2420
+ | FR5 | Agents page with grid view + category filter | Must | All 44 agents parsed, 10 categories filterable |
2421
+ | FR6 | Skills page with type/effort filter + sortable table | Must | All 91 skills, 3 type badges |
2422
+ | FR7 | Hooks page with lifecycle diagram + profile toggle | Should | 21 hooks, 3 profiles toggleable |
2423
+ | FR8 | Plugins page with install/remove action buttons | Should | Action triggers CLI via POST /api/action |
2424
+ | FR9 | Config page with profile/persona management | Should | Reads + writes profiles via CLI |
2425
+ | FR10 | Stats page with inline SVG charts | Should | Bar chart + trend line minimum (v1) |
2426
+ | FR11 | MCP template browser with add/remove | Could | 25 templates browseable |
2427
+ | FR12 | Dark theme + glassmorphism CSS design system | Must | Zero external CSS/font deps |
2428
+ | FR13 | Client-side SPA routing (hash-based) | Must | Navigation without page reload |
2429
+ | FR14 | Responsive design (desktop, tablet, mobile) | Should | 3 breakpoints, usable on mobile |
2430
+ | FR15 | Command allowlist enforcement on /api/action | Must | Non-allowlisted commands → 403 |
2431
+
2432
+ ---
2433
+
2434
+ ## 2. Architecture Overview
2435
+
2436
+ ```
2437
+ ai-toolkit ui [--port 3141] [--no-auto-kill]
2438
+
2439
+ ┌──────────────────────────────────────────────────────────┐
2440
+ │ Local Dashboard │
2441
+ │ │
2442
+ │ Server: stdlib Node.js http (0 deps) │
2443
+ │ Port: 3141 (auto-increment if busy) │
2444
+ │ Auto-kill: 30 min idle (configurable) │
2445
+ │ │
2446
+ │ Pages: │
2447
+ │ / Overview + health │
2448
+ │ /agents 44 agents — grid view │
2449
+ │ /skills 91 skills — filterable table │
2450
+ │ /hooks 21 hooks — lifecycle diagram │
2451
+ │ /plugins 11 packs — install/remove │
2452
+ │ /config Profile / persona / modules │
2453
+ │ /stats Usage analytics + charts │
2454
+ │ /mcp MCP templates — add/remove │
2455
+ │ (credentials page deferred — requires cloud-security-pack CLI) │
2456
+ │ │
2457
+ │ API (JSON, internal): │
2458
+ │ GET /api/status Toolkit state │
2459
+ │ GET /api/agents Agent catalog │
2460
+ │ GET /api/skills Skill catalog │
2461
+ │ GET /api/hooks Hook definitions │
2462
+ │ GET /api/plugins Plugin packs + install state │
2463
+ │ GET /api/stats Usage statistics │
2464
+ │ GET /api/config Current configuration │
2465
+ │ GET /api/mcp MCP templates + installed │
2466
+ │ (credentials endpoint deferred — see Future section) │
2467
+ │ POST /api/action Execute CLI command │
2468
+ │ │
2469
+ │ Static assets (inlined in server.js): │
2470
+ │ HTML template (single page, client-side routing) │
2471
+ │ CSS (dark theme, glassmorphism, responsive) │
2472
+ │ JS (vanilla, fetch-based, no framework) │
2473
+ └──────────────────────────────────────────────────────────┘
2474
+ ```
2475
+
2476
+ ### Action Execution Model
2477
+
2478
+ All write operations go through `POST /api/action`:
2479
+
2480
+ ```json
2481
+ {
2482
+ "command": "plugin",
2483
+ "args": ["install", "memory-pack"]
2484
+ }
2485
+ ```
2486
+
2487
+ The server spawns `ai-toolkit <command> <args>` as a child process, streams stdout/stderr back via SSE (Server-Sent Events), and returns exit code. This ensures:
2488
+ 1. All existing validation logic in CLI scripts executes
2489
+ 2. No file mutation logic duplicated in the dashboard
2490
+ 3. Audit trail identical to CLI usage
2491
+
2492
+ ---
2493
+
2494
+ ## 3. Progress Tracking
2495
+
2496
+ | # | Feature | Priority | Status | Est. Time | Notes |
2497
+ |---|---------|----------|--------|-----------|-------|
2498
+ | 1.1 | HTTP server + auto-kill + port discovery | P0 | Proposed | 1d | stdlib Node.js, ephemeral lifecycle |
2499
+ | 1.2 | API layer — read endpoints (status, agents, skills, hooks, plugins, stats, config, mcp) | P0 | Proposed | 2d | Parse frontmatter, state.json, hooks.json |
2500
+ | 1.3 | API layer — action execution endpoint | P0 | Proposed | 1d | Spawn CLI + SSE streaming |
2501
+ | 2.1 | Overview page (health, component counts, version) | P0 | Proposed | 1.5d | Dashboard landing page |
2502
+ | 2.2 | Agents page (grid + detail view + category filter) | P0 | Proposed | 2d | Parse agent .md frontmatter |
2503
+ | 2.3 | Skills page (filterable table + type badges + effort) | P0 | Proposed | 2d | Task / hybrid / knowledge taxonomy |
2504
+ | 2.4 | Hooks page (lifecycle diagram + profile toggle) | P1 | Proposed | 2d | Visual event → script mapping |
2505
+ | 2.5 | Plugins page (install/remove/status cards) | P1 | Proposed | 2d | Action buttons trigger CLI |
2506
+ | 2.6 | Config page (profile/persona/modules checkboxes) | P1 | Proposed | 2d | Read/write profiles |
2507
+ | 2.7 | Stats page (usage charts, skill invocation heatmap) | P1 | Proposed | 3.5d | Hand-drawn SVG charts (bar, heatmap, trend), no chart library |
2508
+ | 2.8 | MCP page (template browser, add/remove) | P2 | Proposed | 1.5d | 25 MCP templates |
2509
+ | 2.9 | ~~Credentials page~~ | — | Deferred | — | Requires cloud-security-pack CLI commands (not yet implemented) |
2510
+ | 3.1 | CSS design system (dark theme, glassmorphism, responsive) | P0 | Proposed | 2d | Premium aesthetic, zero external deps |
2511
+ | 3.2 | Client-side routing + navigation | P0 | Proposed | 1d | Hash-based SPA routing |
2512
+ | 4.1 | CLI command registration (`ai-toolkit ui`) | P0 | Proposed | 0.5d | bin/ai-toolkit.js integration |
2513
+ | 4.2 | Tests (bats + node:test) | P1 | Proposed | 3d | Server lifecycle (bats) + API endpoints + SSE streaming (node:test) |
2514
+ | 4.3 | Documentation | P1 | Proposed | 2.5d | README, CLAUDE.md, ARCHITECTURE.md, package.json, llms.txt, llms-full.txt, AGENTS.md, skills-catalog.md, architecture-overview.md |
2515
+
2516
+ **Phasing:**
2517
+ - **Phase 1 (week 1-3):** Foundation — server, API, design system, overview page, agents page, skills page
2518
+ - **Phase 2 (week 3-5):** Interactive — hooks, plugins, config, action execution, stats
2519
+ - **Phase 3 (week 6-7):** Polish — MCP, responsive testing, tests, documentation (2.5d docs — all 9 docs per CLAUDE.md rules)
2520
+
2521
+ > **Demand validation gate:** Ship Phase 1 (server + overview + agents + skills) as MVP. Announce, measure adoption (track `ai-toolkit ui` invocations via `stats.json`). Only build Phase 2 interactive pages if usage > 10 sessions/week across early adopters.
2522
+
2523
+ ---
2524
+
2525
+ ## 4. Dependency Graph
2526
+
2527
+ ```
2528
+ Phase 1: Foundation (week 1-3)
2529
+ ================================
2530
+ HTTP server (1.1) ──────┐
2531
+ ├──► API read layer (1.2) ──► Overview page (2.1)
2532
+ CSS design system (3.1) ┤ ├──► Agents page (2.2)
2533
+ Client routing (3.2) ───┘ └──► Skills page (2.3)
2534
+
2535
+ Phase 2: Interactive (week 3-5)
2536
+ =================================
2537
+ API action layer (1.3) ─┐
2538
+ ├──► Hooks page (2.4)
2539
+ ├──► Plugins page (2.5)
2540
+ ├──► Config page (2.6)
2541
+ └──► Stats page (2.7)
2542
+
2543
+ Phase 3: Polish (week 6-7)
2544
+ ===========================
2545
+ ├──► MCP page (2.8)
2546
+ ├──► CLI registration (4.1)
2547
+ └──► Tests + docs (4.2, 4.3)
2548
+ ```
2549
+
2550
+ ---
2551
+
2552
+ ## 5. Detailed Implementation
2553
+
2554
+ ### Phase 1: Foundation (week 1-3)
2555
+
2556
+ #### 1.1 HTTP Server + Lifecycle
2557
+
2558
+ **File:** `app/dashboard/server.js`
2559
+
2560
+ ```javascript
2561
+ // Key design decisions:
2562
+ // 1. stdlib only — require('http'), require('fs'), require('path')
2563
+ // 2. Auto-kill timer — 30 min idle, reset on every request
2564
+ // 3. Port discovery — try 3141, increment until available
2565
+ // 4. Single-file deployment — HTML/CSS/JS embedded as template literals
2566
+ // 5. Same pattern as visual-brainstorming companion in /write-a-prd
2567
+
2568
+ const AUTO_KILL_MS = 30 * 60 * 1000; // 30 minutes
2569
+ const DEFAULT_PORT = 3141;
2570
+ const MAX_PORT_ATTEMPTS = 10;
2571
+ ```
2572
+
2573
+ **CLI interface:**
2574
+ ```bash
2575
+ ai-toolkit ui # open dashboard on port 3141
2576
+ ai-toolkit ui --port 4000 # custom port
2577
+ ai-toolkit ui --no-auto-kill # disable 30 min auto-kill
2578
+ ai-toolkit ui --open # auto-open browser (default: true)
2579
+ ```
2580
+
2581
+ **Lifecycle:**
2582
+ 1. Start HTTP server on available port
2583
+ 2. Open browser via `open` (macOS) / `xdg-open` (Linux)
2584
+ 3. Reset idle timer on every request
2585
+ 4. After 30 min inactivity → `process.exit(0)` with console message
2586
+ 5. `Ctrl+C` → graceful shutdown
2587
+
2588
+ **Security:**
2589
+ - Bind to `127.0.0.1` only (never `0.0.0.0`)
2590
+ - No authentication needed (localhost only, ephemeral)
2591
+ - POST `/api/action` validates command against allowlist (same commands as CLI)
2592
+ - No file uploads, no eval, no template injection
2593
+
2594
+ **Files:**
2595
+
2596
+ | File | Action | Description |
2597
+ |------|--------|-------------|
2598
+ | `app/dashboard/server.js` | CREATE | HTTP server + API + embedded UI |
2599
+ | `app/dashboard/api.js` | CREATE | API endpoint handlers |
2600
+ | `app/dashboard/assets.js` | CREATE | Embedded HTML/CSS/JS templates |
2601
+
2602
+ **Success Criteria:**
2603
+ - [ ] Server starts on port 3141 (or next available)
2604
+ - [ ] Auto-kills after 30 min idle
2605
+ - [ ] Browser auto-opens on launch
2606
+ - [ ] Binds to 127.0.0.1 only
2607
+ - [ ] `Ctrl+C` graceful shutdown
2608
+
2609
+ ---
2610
+
2611
+ #### 1.2 API Layer — Read Endpoints
2612
+
2613
+ All endpoints return JSON. Data sources:
2614
+
2615
+ | Endpoint | Source | Description |
2616
+ |----------|--------|-------------|
2617
+ | `GET /api/status` | `~/.ai-toolkit/state.json` + `manifest.json` | Version, profile, installed modules |
2618
+ | `GET /api/agents` | `app/agents/*.md` frontmatter | Name, description, tools, triggers, category |
2619
+ | `GET /api/skills` | `app/skills/*/SKILL.md` frontmatter | Name, type, effort, agent, description |
2620
+ | `GET /api/hooks` | `app/hooks.json` | Event, script, description, profile |
2621
+ | `GET /api/plugins` | `app/plugins/*/plugin.json` + state | Name, domain, status, installed?, components |
2622
+ | `GET /api/stats` | `~/.ai-toolkit/stats.json` | Skill invocation counts, dates |
2623
+ | `GET /api/config` | `~/.ai-toolkit/state.json` + settings | Profile, persona, modules, hook profile |
2624
+ | `GET /api/mcp` | `app/mcp-templates/*.json` + `.mcp.json` | Available templates, installed servers |
2625
+ | ~~`GET /api/credentials`~~ | — | Deferred — requires cloud-security-pack CLI |
2626
+
2627
+ **Frontmatter parser:** Reuse the YAML-subset parser pattern from existing `scripts/frontmatter.py` — port to JS (simple `---` delimited key-value extraction, covers all toolkit frontmatter which is flat YAML).
2628
+
2629
+ **Response format (example):**
2630
+ ```json
2631
+ {
2632
+ "agents": [
2633
+ {
2634
+ "name": "backend-specialist",
2635
+ "description": "Expert backend architect for Node.js, Python, PHP...",
2636
+ "tools": ["Read", "Write", "Edit", "Bash", "Grep", "Glob"],
2637
+ "triggers": ["backend", "server", "api", "endpoint"],
2638
+ "category": "development",
2639
+ "file": "app/agents/backend-specialist.md"
2640
+ }
2641
+ ],
2642
+ "meta": { "count": 44, "categories": 10 }
2643
+ }
2644
+ ```
2645
+
2646
+ **Success Criteria:**
2647
+ - [ ] All 8 read endpoints return valid JSON
2648
+ - [ ] Agent frontmatter parsed for all 44 agents
2649
+ - [ ] Skill frontmatter parsed for all 91 skills
2650
+ - [ ] Stats endpoint handles missing stats.json gracefully
2651
+
2652
+ ---
2653
+
2654
+ #### 1.3 API Layer — Action Execution
2655
+
2656
+ **Endpoint:** `POST /api/action`
2657
+
2658
+ ```json
2659
+ // Request
2660
+ {
2661
+ "command": "plugin",
2662
+ "args": ["install", "memory-pack"]
2663
+ }
2664
+
2665
+ // Response (SSE stream)
2666
+ event: stdout
2667
+ data: Installing memory-pack...
2668
+
2669
+ event: stdout
2670
+ data: ✓ Hooks installed
2671
+
2672
+ event: done
2673
+ data: {"exitCode": 0, "duration": 1234}
2674
+ ```
2675
+
2676
+ **Command allowlist:**
2677
+ ```javascript
2678
+ const ALLOWED_COMMANDS = [
2679
+ 'plugin install', 'plugin remove', 'plugin update', 'plugin clean',
2680
+ 'mcp add', 'mcp remove',
2681
+ 'install --profile', 'install --persona',
2682
+ 'update', 'validate', 'doctor', 'doctor --fix',
2683
+ 'stats', 'stats --reset',
2684
+ // 'credentials add/remove/test' — deferred until cloud-security-pack CLI ships
2685
+ ];
2686
+ ```
2687
+
2688
+ **Security:** Commands not in allowlist → 403. No shell injection — args passed as array to `execFile`, never interpolated into a string.
2689
+
2690
+ **Success Criteria:**
2691
+ - [ ] SSE streaming of command output
2692
+ - [ ] Exit code returned in final event
2693
+ - [ ] Command allowlist enforced
2694
+ - [ ] No shell injection possible
2695
+ - [ ] Concurrent commands rejected (one at a time)
2696
+
2697
+ ---
2698
+
2699
+ ### Phase 2: UI Pages (week 3-5)
2700
+
2701
+ #### 2.1 Overview Page
2702
+
2703
+ The landing page — first thing the user sees.
2704
+
2705
+ **Layout:**
2706
+ ```
2707
+ ┌─────────────────────────────────────────────────┐
2708
+ │ ai-toolkit v1.5.1 [● running] │
2709
+ ├─────────────────────────────────────────────────┤
2710
+ │ │
2711
+ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │
2712
+ │ │ 44 │ │ 91 │ │ 21 │ │ 11 │ │
2713
+ │ │agents│ │skills│ │hooks │ │packs │ │
2714
+ │ └──────┘ └──────┘ └──────┘ └──────┘ │
2715
+ │ │
2716
+ │ Profile: standard Persona: (none) │
2717
+ │ Hook Profile: standard │
2718
+ │ Node: v22.x Python: 3.12 │
2719
+ │ │
2720
+ │ ┌─ Health ──────────────────────────────────┐ │
2721
+ │ │ ✓ Constitution symlinked │ │
2722
+ │ │ ✓ Hooks installed (21/21) │ │
2723
+ │ │ ✓ Agents symlinked (44/44) │ │
2724
+ │ │ ⚠ 2 plugins not installed │ │
2725
+ │ │ ✓ MCP servers: 3 configured │ │
2726
+ │ └───────────────────────────────────────────┘ │
2727
+ │ │
2728
+ │ ┌─ Recent Activity ─────────────────────────┐ │
2729
+ │ │ /review — 12 invocations (last: 2h ago) │ │
2730
+ │ │ /commit — 8 invocations (last: 5h ago) │ │
2731
+ │ │ /test — 6 invocations (last: 1d ago) │ │
2732
+ │ └───────────────────────────────────────────┘ │
2733
+ └─────────────────────────────────────────────────┘
2734
+ ```
2735
+
2736
+ **Data sources:** `/api/status` + `/api/stats` + `/api/plugins`
2737
+
2738
+ **Success Criteria:**
2739
+ - [ ] Component counts rendered from live data
2740
+ - [ ] Health checks match `ai-toolkit validate` output
2741
+ - [ ] Recent activity from stats.json
2742
+ - [ ] Responsive on mobile-width screens
2743
+
2744
+ ---
2745
+
2746
+ #### 2.2 Agents Page
2747
+
2748
+ **Layout:** Grid of agent cards, filterable by category (10 categories).
2749
+
2750
+ **Card design:**
2751
+ ```
2752
+ ┌─────────────────────────────────┐
2753
+ │ 🔧 backend-specialist │
2754
+ │ │
2755
+ │ Expert backend architect for │
2756
+ │ Node.js, Python, PHP... │
2757
+ │ │
2758
+ │ Tools: Read, Write, Edit, Bash │
2759
+ │ Triggers: backend, server, api │
2760
+ │ │
2761
+ │ [View Definition] │
2762
+ └─────────────────────────────────┘
2763
+ ```
2764
+
2765
+ **Features:**
2766
+ - Category filter pills (development, security, data, infrastructure, etc.)
2767
+ - Search by name/description/trigger
2768
+ - Click card → modal with full agent definition (rendered markdown)
2769
+ - Agent count per category in filter pills
2770
+
2771
+ ---
2772
+
2773
+ #### 2.3 Skills Page
2774
+
2775
+ **Layout:** Filterable table with type badges.
2776
+
2777
+ | Skill | Type | Effort | Agent | Description |
2778
+ |-------|------|--------|-------|-------------|
2779
+ | /commit | task | medium | — | Structured commit with linting |
2780
+ | /review | hybrid | high | code-reviewer | Code review: quality, security |
2781
+ | clean-code | knowledge | — | — | Auto-loaded code quality patterns |
2782
+
2783
+ **Features:**
2784
+ - Filter by type (task / hybrid / knowledge)
2785
+ - Filter by effort (low / medium / high / max)
2786
+ - Search by name/description
2787
+ - Sort by any column
2788
+ - Badge colors: task=blue, hybrid=purple, knowledge=green
2789
+
2790
+ ---
2791
+
2792
+ #### 2.4 Hooks Page
2793
+
2794
+ **Layout:** Lifecycle diagram with profile toggle (minimal/standard/strict).
2795
+
2796
+ **Visualization:**
2797
+ ```
2798
+ Event Timeline (horizontal flow):
2799
+ PreToolUse ──► ToolUse ──► PostToolUse ──► Notification
2800
+ │ │ │ │
2801
+ guard-path.sh (tool exec) quality-check.sh track-usage.sh
2802
+ guard-destructive.sh session-context.sh
2803
+ ```
2804
+
2805
+ **Features:**
2806
+ - Lifecycle event diagram — visual mapping of event → hook script(s)
2807
+ - Profile toggle — show/hide hooks per profile (minimal hides most, strict shows all)
2808
+ - Hook detail panel — click hook name to see: script path, description, when it fires, which profile includes it
2809
+ - Profile diff — highlight which hooks are added/removed between profiles
2810
+ - Status indicators — green (installed), red (missing script), yellow (overridden by plugin)
2811
+
2812
+ **Data source:** `app/hooks.json` + `~/.ai-toolkit/hooks/` filesystem check
2813
+
2814
+ **Success Criteria:**
2815
+ - [ ] All 21 hooks rendered with correct lifecycle event
2816
+ - [ ] Profile toggle filters hooks correctly
2817
+ - [ ] Missing hook scripts shown with warning indicator
2818
+ - [ ] Hook detail panel shows script content preview
2819
+
2820
+ ---
2821
+
2822
+ #### 2.5 Plugins Page
2823
+
2824
+ **Layout:** Cards with install/remove buttons.
2825
+
2826
+ ```
2827
+ ┌─────────────────────────────────┐
2828
+ │ 🧩 memory-pack ● installed │
2829
+ │ │
2830
+ │ SQLite-based persistent memory │
2831
+ │ with FTS5 search across sessions│
2832
+ │ │
2833
+ │ Skills: 1 Hooks: 2 Agents: 0 │
2834
+ │ DB size: 2.4 MB Obs: 1,234 │
2835
+ │ │
2836
+ │ [Update] [Clean] [Remove] │
2837
+ └─────────────────────────────────┘
2838
+ ```
2839
+
2840
+ **Actions:** Install, update, remove, clean — all via `POST /api/action` → `ai-toolkit plugin <action> <name>`. SSE output shown in a slide-out console panel.
2841
+
2842
+ ---
2843
+
2844
+ #### 2.6 Config Page
2845
+
2846
+ **Layout:** Form with current configuration, editable.
2847
+
2848
+ **Sections:**
2849
+ 1. **Profile selector:** minimal / standard / strict (radio buttons)
2850
+ 2. **Persona selector:** none / backend-lead / frontend-lead / devops-eng / junior-dev
2851
+ 3. **Hook profile:** minimal / standard / strict
2852
+ 4. **Installed modules:** checkboxes for each module from manifest.json
2853
+ 5. **Language rules:** detected languages + override checkboxes
2854
+ 6. **Editor configs:** which editors are configured (read-only status)
2855
+
2856
+ **Save:** Generates and executes the equivalent `ai-toolkit install --profile X --persona Y --modules A,B,C` command.
2857
+
2858
+ ---
2859
+
2860
+ #### 2.7 Stats Page
2861
+
2862
+ **Layout:** Usage analytics with inline SVG charts.
2863
+
2864
+ **Charts (SVG, hand-drawn — no chart library):**
2865
+ 1. **Skill invocation bar chart** — top 15 most-used skills
2866
+ 2. **Invocation heatmap** — 7x24 grid (day of week × hour) showing when skills are used
2867
+ 3. **Effort distribution** — pie chart of low/medium/high/max invocations
2868
+ 4. **Trend line** — daily invocations over last 30 days
2869
+
2870
+ **Data source:** `~/.ai-toolkit/stats.json` (written by `track-usage.sh` hook)
2871
+
2872
+ **SVG implementation notes:**
2873
+ - All charts rendered as inline SVG strings in `assets.js` — no external chart library
2874
+ - Bar chart: `<rect>` elements with calculated heights, axis labels as `<text>`
2875
+ - Heatmap: 7x24 grid of `<rect>` with color intensity mapped to invocation count (0=transparent, max=accent-primary)
2876
+ - Trend line: `<polyline>` with data points as `<circle>`, area fill via `<polygon>`
2877
+ - Pie chart: `<path>` arcs calculated from percentages (use `Math.cos`/`Math.sin` for arc endpoints)
2878
+ - All charts use CSS custom properties for colors (respects design system tokens)
2879
+
2880
+ **Fallback:** If hand-drawn SVG proves too complex for 4 chart types in 3.5d budget, reduce to 2 charts (bar + trend) for v1 and defer heatmap + pie to v2.
2881
+
2882
+ **Success Criteria:**
2883
+ - [ ] All 4 chart types render from live stats.json data
2884
+ - [ ] Empty state when stats.json missing: "No usage data yet — skills will appear here after first use"
2885
+ - [ ] Charts responsive (SVG viewBox scales to container width)
2886
+ - [ ] Tooltip on hover showing exact counts
2887
+
2888
+ ---
2889
+
2890
+ ### Phase 3: Polish (week 6-7)
2891
+
2892
+ #### 3.1 CSS Design System
2893
+
2894
+ **Design tokens:**
2895
+ ```css
2896
+ :root {
2897
+ /* Dark theme palette */
2898
+ --bg-primary: #0f0f14;
2899
+ --bg-secondary: #1a1a24;
2900
+ --bg-card: rgba(255, 255, 255, 0.04);
2901
+ --bg-glass: rgba(255, 255, 255, 0.06);
2902
+ --border-glass: rgba(255, 255, 255, 0.08);
2903
+
2904
+ /* Accent colors (derived from SoftSpark brand) */
2905
+ --accent-primary: #6366f1; /* indigo */
2906
+ --accent-secondary: #8b5cf6; /* violet */
2907
+ --accent-success: #10b981; /* emerald */
2908
+ --accent-warning: #f59e0b; /* amber */
2909
+ --accent-danger: #ef4444; /* red */
2910
+
2911
+ /* Typography */
2912
+ --font-sans: system-ui, -apple-system, sans-serif;
2913
+ --font-mono: 'SF Mono', 'Cascadia Code', monospace;
2914
+
2915
+ /* Spacing scale */
2916
+ --space-xs: 4px;
2917
+ --space-sm: 8px;
2918
+ --space-md: 16px;
2919
+ --space-lg: 24px;
2920
+ --space-xl: 32px;
2921
+
2922
+ /* Glassmorphism */
2923
+ --glass-blur: 12px;
2924
+ --glass-bg: rgba(255, 255, 255, 0.05);
2925
+ --glass-border: rgba(255, 255, 255, 0.1);
2926
+ }
2927
+ ```
2928
+
2929
+ **Components:**
2930
+ - Cards with glassmorphism backdrop-filter
2931
+ - Badge pills (type, effort, status)
2932
+ - Sidebar navigation with active state
2933
+ - Console output panel (slide-out, monospace)
2934
+ - Modal overlays for detail views
2935
+ - Toast notifications for action results
2936
+ - Skeleton loading states
2937
+
2938
+ **Responsive breakpoints:**
2939
+ - Desktop: 1200px+ (sidebar + content)
2940
+ - Tablet: 768px-1199px (collapsed sidebar)
2941
+ - Mobile: <768px (hamburger menu, stacked cards)
2942
+
2943
+ ---
2944
+
2945
+ #### 4.1 CLI Command Registration
2946
+
2947
+ **File:** `bin/ai-toolkit.js` — add to `COMMANDS` and `SPECIAL_HANDLERS`:
2948
+
2949
+ ```javascript
2950
+ // COMMANDS
2951
+ 'ui': 'Launch local web dashboard for visual toolkit management',
2952
+
2953
+ // SPECIAL_HANDLERS
2954
+ 'ui': (args) => {
2955
+ const serverPath = path.join(TOOLKIT_DIR, 'app', 'dashboard', 'server.js');
2956
+ const child = require('child_process').spawn('node', [serverPath, ...args], {
2957
+ stdio: 'inherit',
2958
+ env: { ...process.env, TOOLKIT_DIR }
2959
+ });
2960
+ child.on('exit', (code) => process.exit(code || 0));
2961
+ },
2962
+ ```
2963
+
2964
+ **Files to modify:**
2965
+
2966
+ | File | Action | Description |
2967
+ |------|--------|-------------|
2968
+ | `bin/ai-toolkit.js` | EDIT | Register `ui` command |
2969
+ | `app/dashboard/server.js` | CREATE | HTTP server (main) |
2970
+ | `app/dashboard/api.js` | CREATE | API handlers |
2971
+ | `app/dashboard/assets.js` | CREATE | Embedded HTML/CSS/JS |
2972
+ | `app/dashboard/frontmatter.js` | CREATE | JS frontmatter parser |
2973
+ | `tests/test_dashboard.bats` | CREATE | Server lifecycle tests |
2974
+ | `tests/test_dashboard_api.bats` | CREATE | API endpoint tests |
2975
+
2976
+ ---
2977
+
2978
+ ## 6. File Summary
2979
+
2980
+ | File | Action | LOC (est.) | Description |
2981
+ |------|--------|------------|-------------|
2982
+ | `app/dashboard/server.js` | CREATE | ~300 | HTTP server, lifecycle, routing |
2983
+ | `app/dashboard/api.js` | CREATE | ~400 | All API endpoint handlers |
2984
+ | `app/dashboard/assets.js` | CREATE | ~2800 | Embedded HTML + CSS + JS. Split internally: `getOverviewHTML()`, `getAgentsHTML()`, etc. — one exported function per page. Single file, multiple functions (not multiple files — preserves single-`require` deployment). If file exceeds 3000 LOC, extract to `assets/` directory with `index.js` barrel |
2985
+ | `app/dashboard/frontmatter.js` | CREATE | ~80 | Frontmatter parser (JS port) |
2986
+ | `bin/ai-toolkit.js` | EDIT | +15 | Register `ui` command |
2987
+ | `tests/test_dashboard.bats` | CREATE | ~100 | Server lifecycle tests |
2988
+ | `tests/test_dashboard_api.bats` | CREATE | ~150 | API endpoint tests |
2989
+ | **Total** | | **~3845** | |
2990
+
2991
+ ---
2992
+
2993
+ ## 7. Success Criteria (Overall)
2994
+
2995
+ | Metric | Target |
2996
+ |--------|--------|
2997
+ | Pages | 8 (overview, agents, skills, hooks, plugins, config, stats, mcp) |
2998
+ | API endpoints | 9 (8 read + 1 action) |
2999
+ | External dependencies | 0 (stdlib Node.js only) |
3000
+ | Server startup time | < 500ms |
3001
+ | Auto-kill | 30 min idle (configurable) |
3002
+ | Responsive breakpoints | 3 (desktop, tablet, mobile) |
3003
+ | Tests | 25+ |
3004
+ | Browser support | Chrome, Firefox, Safari (modern, no IE) |
3005
+
3006
+ ---
3007
+
3008
+ ## 7a. Non-Functional Requirements
3009
+
3010
+ | Category | Requirement |
3011
+ |----------|-------------|
3012
+ | **Performance** | Server startup < 500ms. Page render < 200ms. API responses < 100ms. |
3013
+ | **Memory** | Max RSS < 100MB (including embedded assets). No unbounded caching. |
3014
+ | **Concurrency** | Max 1 concurrent action execution. Queue or reject additional requests with 429. |
3015
+ | **Security** | Bind `127.0.0.1` only. CSP header: `default-src 'self' 'unsafe-inline'`. No CORS headers (same-origin only). Command allowlist enforced server-side. `execFile` with array args (no shell). |
3016
+ | **Accessibility** | Keyboard navigation for all interactive elements. Focus management on page transitions. Minimum 4.5:1 contrast ratio (WCAG AA). ARIA labels on icon-only buttons. |
3017
+ | **Error handling** | Missing/corrupt data files → graceful empty state with message. Server crash → exit code 1 with stderr diagnostic. API errors → JSON `{ "error": "..." }` with HTTP status. |
3018
+ | **Graceful degradation** | If `stats.json` missing → stats page shows "No data yet". If `state.json` missing → overview shows defaults. If agent/skill .md unreadable → skip with warning in console. |
3019
+
3020
+ ---
3021
+
3022
+ ## 7b. Rollback & Feature Flag
3023
+
3024
+ The dashboard is purely additive — removing it is trivial:
3025
+ 1. Delete `app/dashboard/` directory
3026
+ 2. Remove `ui` from `COMMANDS` and `SPECIAL_HANDLERS` in `bin/ai-toolkit.js`
3027
+ 3. No config files, no state files, no hooks to clean up
3028
+
3029
+ **Disable without removal:** `ai-toolkit ui --disabled` could print "Dashboard disabled" and exit. Alternatively, skip registering the `ui` command in `bin/ai-toolkit.js` behind a `manifest.json` module flag so users can opt out via `--skip dashboard`.
3030
+
3031
+ ---
3032
+
3033
+ ## 7c. Testing Strategy
3034
+
3035
+ | Layer | Framework | Coverage |
3036
+ |-------|-----------|----------|
3037
+ | Server lifecycle (start, port discovery, auto-kill, shutdown) | bats | 5+ tests |
3038
+ | API endpoints (all 8 read + action) | `node:test` (stdlib) | 15+ tests |
3039
+ | SSE streaming (stdout/stderr/done events) | `node:test` | 3+ tests |
3040
+ | Command allowlist enforcement | `node:test` | 5+ tests |
3041
+ | Frontmatter parser edge cases | `node:test` | 5+ tests |
3042
+ | Integration: full pipeline (start → API → action → stop) | `node:test` | 3+ tests |
3043
+ | **Total** | | **36+** |
3044
+
3045
+ `node:test` is stdlib (Node 18+), zero dependencies. Bats tests are for CLI-level integration (process start/stop). JS tests cover API correctness.
3046
+
3047
+ ---
3048
+
3049
+ ## 7d. Discoverability
3050
+
3051
+ | Touchpoint | Action |
3052
+ |------------|--------|
3053
+ | `ai-toolkit install` output | Print banner: `Run 'ai-toolkit ui' to explore agents, skills, and plugins visually.` |
3054
+ | `ai-toolkit help` | Include `ui` in command list |
3055
+ | README.md | Screenshot/GIF of dashboard overview page |
3056
+ | First-run detection | If `stats.json` is empty, show a "Try the dashboard" suggestion after `ai-toolkit install` |
3057
+
3058
+ ---
3059
+
3060
+ ## 8. Risks and Mitigation
3061
+
3062
+ | Risk | Probability | Impact | Mitigation |
3063
+ |------|-------------|--------|------------|
3064
+ | Assets.js file too large (embedded HTML/CSS/JS) | Medium | Low | Split into multiple template modules, lazy-load pages |
3065
+ | Port conflict on 3141 | Low | Low | Auto-increment port, show clear message |
3066
+ | SSE not supported in old browsers | Low | Low | Fallback to polling for action results |
3067
+ | Frontmatter parser edge cases | Low | Medium | Match exact patterns used in existing toolkit metadata |
3068
+ | Config writes break installation | Low | High | All writes go through existing CLI commands — never direct file mutation |
3069
+ | Stats.json missing or empty | Medium | Low | Graceful empty state with "No usage data yet" message |
3070
+
3071
+ ---
3072
+
3073
+ ## 9. Pre-Mortem
3074
+
3075
+ 1. **"Too many features per page"** — Dashboard tries to show everything. Users may feel overwhelmed. Mitigation: progressive disclosure — overview page shows summary only, detail pages are opt-in.
3076
+ 2. **"Asset file becomes unmaintainable"** — ~2800 LOC of embedded HTML/CSS/JS is hard to iterate on. Mitigation: split into `assets/overview.js`, `assets/agents.js`, etc. with a build-free concatenation in server.js.
3077
+ 3. **"Nobody uses it"** — CLI users may prefer CLI. Mitigation: dashboard is opt-in, never required. Add `ai-toolkit ui` suggestion to `ai-toolkit install` output for new users.
3078
+ 4. **"Action execution feels disconnected"** — SSE console output may confuse users unfamiliar with CLI. Mitigation: rich UI feedback (progress bars, success/error toasts) layered on top of raw output.
3079
+ 5. **"Charts look bad without a library"** — Hand-drawn SVG charts may look amateur. Mitigation: keep charts simple (bar + heatmap), use consistent design tokens, test extensively.
3080
+
3081
+ ---
3082
+
3083
+ ## 10. Next Actions
3084
+
3085
+ 1. [ ] Approve plan
3086
+ 2. [ ] Create `app/dashboard/server.js` with HTTP server + lifecycle (1.1)
3087
+ 3. [ ] Create API layer — read endpoints (1.2)
3088
+ 4. [ ] Create CSS design system + client routing (3.1, 3.2)
3089
+ 5. [ ] Build Overview page (2.1)
3090
+ 6. [ ] Build Agents page (2.2)
3091
+ 7. [ ] Build Skills page (2.3)
3092
+ 8. [ ] Add action execution API (1.3)
3093
+ 9. [ ] Build Hooks, Plugins, Config pages (2.4, 2.5, 2.6)
3094
+ 10. [ ] Build Stats page with SVG charts (2.7)
3095
+ 11. [ ] Build MCP page (2.8)
3096
+ 12. [ ] Register CLI command (4.1)
3097
+ 13. [ ] Tests — bats + node:test (4.2)
3098
+ 14. [ ] Documentation — all 9 docs per CLAUDE.md rules (4.3)
3099
+
3100
+ ---
3101
+
3102
+ ## 11. Future (Deferred)
3103
+
3104
+ | Feature | Reason for deferral | Prerequisite |
3105
+ |---------|-------------------|--------------|
3106
+ | Credentials page (2.9) | CLI has no `credentials` commands yet | cloud-security-pack CLI implementation |
3107
+ | `ai-toolkit ui --static` | Generate a single standalone HTML file (no server) — covers 80% of catalog value at 20% complexity | Post-v1 evaluation of actual usage patterns |
3108
+ | Guided onboarding flow | Interactive wizard for new users | Post-v1, based on user feedback |
3109
+ | Light theme toggle | Dark-only for v1 | CSS variable architecture makes this easy later |
3110
+
3111
+ ---
3112
+
3113
+ ## 10. Market Positioning
3114
+
3115
+ **Target users:**
3116
+ 1. **New adopters** — developers evaluating ai-toolkit who want to understand what's included before committing to a profile
3117
+ 2. **Team leads** — visual overview of which agents, skills, and plugins are active across team setups
3118
+ 3. **Plugin explorers** — developers browsing available plugin packs without memorizing CLI commands
3119
+ 4. **Onboarding** — new team members getting oriented with the toolkit's capabilities
3120
+
3121
+ **Competitive advantage:** No existing AI coding toolkit provides a zero-dependency ephemeral web dashboard for visual management. Backstage is 1000x heavier (requires Kubernetes, PostgreSQL). TUI tools (mise, lazygit) lack visual richness. The ephemeral auto-kill design means zero operational burden.
3122
+
3123
+ **Discovery opportunity:** The dashboard serves as a self-documenting catalog — users discover agents and skills they didn't know existed, increasing toolkit utilization.
3124
+
3125
+ ---
3126
+
3127
+ ## 12. Cross-Plan Dependencies
3128
+
3129
+ This plan shares modification targets with two other proposed plans:
3130
+
3131
+ | Shared File | This Plan | Enterprise Config Plan | Offline SLM Plan |
3132
+ |-------------|-----------|----------------------|-----------------|
3133
+ | `bin/ai-toolkit.js` | +15 LOC (ui command) | +40 LOC (config subcommands) | +10 LOC (compile-slm command) |
3134
+
3135
+ **If Enterprise Config ships first:** Dashboard Config page (2.6) should display `.ai-toolkit.json` / `extends` status and `config diff` output. Add an API endpoint `GET /api/config/extends` reading resolved extends state from `state.json`.
3136
+
3137
+ **If Offline SLM ships first:** Dashboard Overview page (2.1) should display `offline-slm` profile status and link to compiled output. Stats page may show limited data (SLM providers don't emit hook-based stats).
3138
+
3139
+ ---
3140
+
3141
+ **Last Updated:** 2026-04-10
3142
+
3143
+ ---
3144
+
3145
+ ## kb/planning/offline-slm-profile-plan.md
3146
+
3147
+ ---
3148
+ title: "Plan: Offline-First SLM Profile — Lightweight Mode for Local Models"
3149
+ category: planning
3150
+ service: ai-toolkit
3151
+ tags:
3152
+ - offline
3153
+ - slm
3154
+ - small-language-models
3155
+ - ollama
3156
+ - lm-studio
3157
+ - profile
3158
+ - context-optimization
3159
+ - privacy
3160
+ doc_type: plan
3161
+ status: proposed
3162
+ created: "2026-04-10"
3163
+ last_updated: "2026-04-10"
3164
+ completion: "0%"
3165
+ description: "Lightweight profile for ai-toolkit optimized for Small Language Models (SLMs) running locally via Ollama, LM Studio, or similar. Compiles a minimal instruction set that fits within 4K-8K system prompt budgets while preserving critical safety guardrails. Targets air-gapped, privacy-first, and cost-sensitive development workflows."
3166
+ ---
3167
+
3168
+ # Plan: Offline-First SLM Profile — Lightweight Mode for Local Models
3169
+
3170
+ **Status:** Proposed
3171
+ **Completion:** 0%
3172
+ **Created:** 2026-04-10
3173
+ **Origin:** Enterprise IP security requirements (air-gapped environments), cost-sensitive solo developers, and the growing adoption of local models (Ollama, LM Studio, llamafile). Current toolkit emits 20K+ token system prompts that exceed SLM context windows and degrade small model performance.
3174
+ **Estimated Effort:** 4-5 weeks (1 person)
3175
+
3176
+ ---
3177
+
3178
+ ## 1. Objective
3179
+
3180
+ Create a `--profile offline-slm` install profile and a `scripts/compile_slm.py` compiler that produces a minimal, high-signal instruction set optimized for Small Language Models (8B-32B parameters). The compiled output preserves critical safety guardrails while stripping agent orchestration, multi-agent coordination, and complex skill routing that SLMs cannot handle.
3181
+
3182
+ **Key design principles:**
3183
+ - **Token budget** — compiled output fits within 4K tokens (system prompt), with optional 8K mode for larger SLMs
3184
+ - **Safety-preserved** — Constitution Articles I-V always included (non-negotiable)
3185
+ - **Single-agent focus** — no multi-agent orchestration, no /swarm, no /teams
3186
+ - **Deterministic compilation** — same input → same output, no LLM involved in compilation
3187
+ - **Model-aware** — detects model size from Ollama API or manual flag and adjusts verbosity
3188
+ - **Platform-agnostic** — outputs plain markdown consumable by any local inference engine
3189
+ - **Hooks stripped** — SLM providers don't support lifecycle hooks; rules compile into system prompt
3190
+
3191
+ ---
3192
+
3193
+ ## 1a. Functional Requirements
3194
+
3195
+ | ID | Requirement | Priority | Success Metric |
3196
+ |----|-------------|----------|----------------|
3197
+ | FR1 | Token counter (stdlib-only, ±10% accuracy target) | Must | Conservative estimate, no external deps |
3198
+ | FR2 | Component parser + scorer with safety-priority ranking | Must | Constitution=1.0, all components scored |
3199
+ | FR3 | Compression engine with 4 levels (ultra-light, light, standard, extended) | Must | Each level strips progressively less |
3200
+ | FR4 | Budget packer (greedy knapsack by score/size ratio) | Must | Output ≤ budget × 0.95 in all cases |
3201
+ | FR5 | Markdown emitter with safety-first structure | Must | Constitution always first in output |
3202
+ | FR6 | `--profile offline-slm` install integration | Must | `install.py` + `manifest.json` updated |
3203
+ | FR7 | `compile-slm` CLI command with flags | Must | `--budget`, `--model-size`, `--persona`, `--lang`, `--output`, `--format`, `--dry-run` |
3204
+ | FR8 | Constitution always included (non-negotiable) | Must | Compilation fails if constitution exceeds budget alone |
3205
+ | FR9 | Model size detection from Ollama API | Should | Auto-detect with graceful fallback to `14b` |
3206
+ | FR10 | Persona-aware compilation (boost relevant skills) | Should | Persona skills ranked higher |
3207
+ | FR11 | Language-aware compilation (include matching rules only) | Should | Non-matching language rules excluded |
3208
+ | FR12 | Integration guides for 4 platforms (Ollama, LM Studio, Aider, Continue.dev) | Should | Step-by-step setup per platform |
3209
+ | FR13 | Compile quality validator (post-compilation checks) | Should | FAIL on missing constitution, budget exceeded |
3210
+ | FR14 | 4 output formats (raw markdown, Ollama Modelfile, JSON string, Aider-compatible) | Should | Each format usable by target tool |
3211
+ | FR15 | `--dry-run` output showing included components + token counts | Should | Table: component, score, tokens, included? |
3212
+
3213
+ ---
3214
+
3215
+ ## 2. Architecture Overview
3216
+
3217
+ ```
3218
+ ai-toolkit install --profile offline-slm [--model-size 8b|14b|32b|70b]
3219
+ ai-toolkit compile-slm [--budget 4096] [--persona backend-lead] [--lang typescript]
3220
+
3221
+ ┌──────────────────────────────────────────────────────────┐
3222
+ │ offline-slm Profile │
3223
+ │ │
3224
+ │ Compiler: scripts/compile_slm.py │
3225
+ │ Input: full toolkit (agents, skills, rules, constitution)│
3226
+ │ Output: single compiled .md file for system prompt │
3227
+ │ │
3228
+ │ Token Budget Tiers: │
3229
+ │ ultra-light (2K) — safety + persona only │
3230
+ │ light (4K) — safety + persona + top skills + rules │
3231
+ │ standard (8K) — safety + persona + full skills + rules │
3232
+ │ extended (16K) — near-full toolkit (for 32B+ models) │
3233
+ │ │
3234
+ │ Output Files: │
3235
+ │ ~/.ai-toolkit/compiled/slm-system-prompt.md │
3236
+ │ ~/.ai-toolkit/compiled/slm-skills-reference.md │
3237
+ │ CLAUDE.md (or equivalent) — auto-generated │
3238
+ │ │
3239
+ │ Integration Targets: │
3240
+ │ Ollama (modelfile SYSTEM directive) │
3241
+ │ LM Studio (system prompt field) │
3242
+ │ llamafile (--system-prompt flag) │
3243
+ │ Open WebUI (system prompt setting) │
3244
+ │ Aider (--system-prompt-file flag) │
3245
+ │ Continue.dev (system prompt in config) │
3246
+ └──────────────────────────────────────────────────────────┘
3247
+ ```
3248
+
3249
+ ### Compilation Pipeline
3250
+
3251
+ ```
3252
+ Full Toolkit (20K+ tokens)
3253
+
3254
+
3255
+ ┌─────────────────┐
3256
+ │ 1. Parse Phase │ Read all agents, skills, rules, constitution
3257
+ └────────┬────────┘
3258
+
3259
+ ┌─────────────────┐
3260
+ │ 2. Rank Phase │ Score components by: safety criticality × usage frequency × persona relevance
3261
+ └────────┬────────┘
3262
+
3263
+ ┌─────────────────┐
3264
+ │ 3. Compress Phase│ Strip: examples, rationalization tables, related skills, verbose headers
3265
+ └────────┬────────┘
3266
+
3267
+ ┌─────────────────┐
3268
+ │ 4. Budget Phase │ Pack highest-scoring components until token budget reached
3269
+ └────────┬────────┘
3270
+
3271
+ ┌─────────────────┐
3272
+ │ 5. Emit Phase │ Write compiled .md + integration instructions
3273
+ └─────────────────┘
3274
+ ```
3275
+
3276
+ ---
3277
+
3278
+ ## 3. Progress Tracking
3279
+
3280
+ | # | Feature | Priority | Status | Est. Time | Notes |
3281
+ |---|---------|----------|--------|-----------|-------|
3282
+ | 1.1 | Token counter (tiktoken-free, word-based estimator) | P0 | Proposed | 0.5d | ~0.75 tokens/word heuristic (stdlib only) |
3283
+ | 1.2 | Component parser + scorer | P0 | Proposed | 2d | Parse frontmatter, score by criticality/frequency/persona |
3284
+ | 1.3 | Compression engine | P0 | Proposed | 2d | Strip examples, rationalization tables, headers |
3285
+ | 1.4 | Budget packer | P0 | Proposed | 1d | Greedy knapsack by score/size ratio |
3286
+ | 1.5 | Emitter (markdown output) | P0 | Proposed | 1d | Clean compiled .md file |
3287
+ | 2.1 | Profile integration (`--profile offline-slm`) | P0 | Proposed | 1.5d | Install.py + manifest.json + state.json |
3288
+ | 2.2 | CLI command (`ai-toolkit compile-slm`) | P0 | Proposed | 1d | Standalone compilation with flags |
3289
+ | 2.3 | Model size detection (Ollama API) | P1 | Proposed | 1d | Auto-detect model params from `ollama list` |
3290
+ | 2.4 | Persona-aware compilation | P1 | Proposed | 1.5d | Boost persona-relevant skills in ranking |
3291
+ | 2.5 | Language-aware compilation | P1 | Proposed | 1d | Include only matching language rules |
3292
+ | 3.1 | Integration guides (Ollama, LM Studio, Aider, Continue) | P1 | Proposed | 1.5d | Step-by-step per platform |
3293
+ | 3.2 | Compile quality validator | P1 | Proposed | 1d | Verify output covers constitution, fits budget |
3294
+ | 3.3 | Tests | P1 | Proposed | 3d | Unit: compilation determinism, budget compliance, 4 compression levels × 4 output formats, constitution guard. Integration: `compile-slm --model-size 8b`, verify output fits 2048 tokens + constitution present end-to-end. Target: 40+ tests |
3295
+ | 3.4 | Documentation | P1 | Proposed | 2.5d | All 9 docs per CLAUDE.md: README, CLAUDE.md, ARCHITECTURE.md, package.json, llms.txt, llms-full.txt, AGENTS.md, skills-catalog.md, architecture-overview.md + integration guide |
3296
+
3297
+ **Phasing:**
3298
+ - **Phase 1 (week 1-2):** Compiler — parser, scorer, compressor, packer, emitter
3299
+ - **Phase 2 (week 2-3):** Integration — profile, CLI, model detection, persona/language awareness
3300
+ - **Phase 3 (week 3-4):** Polish — integration guides, validator, tests, documentation
3301
+
3302
+ > **Demand validation gate:** Ship Phase 1 + basic Phase 2 (compiler + profile + CLI with `--budget` and `--model-size` flags) as MVP. Test with 3 real models (8B, 14B, 32B). Only build persona/language-aware compilation and platform-specific integration guides if MVP validation confirms output quality.
3303
+
3304
+ ---
3305
+
3306
+ ## 4. Dependency Graph
3307
+
3308
+ ```
3309
+ Phase 1: Compiler (week 1-2)
3310
+ ============================
3311
+ Token counter (1.1) ────┐
3312
+ ├──► Compression engine (1.3) ──► Budget packer (1.4) ──► Emitter (1.5)
3313
+ Component parser (1.2) ──┘
3314
+
3315
+ Phase 2: Integration (week 2-3)
3316
+ ================================
3317
+ Profile integration (2.1) ──┐
3318
+ ├──► CLI command (2.2)
3319
+ Model detection (2.3) ──────┤
3320
+ Persona-aware (2.4) ────────┤
3321
+ Language-aware (2.5) ────────┘
3322
+
3323
+ Phase 3: Polish (week 3-4)
3324
+ ===========================
3325
+ ├──► Integration guides (3.1)
3326
+ ├──► Compile validator (3.2)
3327
+ └──► Tests + docs (3.3, 3.4)
3328
+ ```
3329
+
3330
+ ---
3331
+
3332
+ ## 5. Detailed Implementation
3333
+
3334
+ ### Phase 1: Compiler Engine (week 1-2)
3335
+
3336
+ #### 1.1 Token Counter
3337
+
3338
+ **Stdlib-only token estimation** — no tiktoken, no external dependencies.
3339
+
3340
+ ```python
3341
+ def estimate_tokens(text: str) -> int:
3342
+ """Estimate token count from text without external dependencies.
3343
+
3344
+ Uses two heuristics and returns the higher (conservative) estimate:
3345
+ 1. Word-based: ~0.75 tokens/word for English prose
3346
+ 2. Char-based: ~1 token per 4 chars (more accurate for code-heavy content)
3347
+
3348
+ Accuracy target: ±10% vs tiktoken cl100k_base. To be validated on 50 toolkit files before shipping.
3349
+ """
3350
+ word_est = int(len(text.split()) * 0.75)
3351
+ char_est = len(text) // 4
3352
+ # Code blocks have higher token density — adjust
3353
+ code_blocks = text.count('```')
3354
+ code_penalty = code_blocks * 15
3355
+ return max(word_est, char_est) + code_penalty
3356
+ ```
3357
+
3358
+ **Why not tiktoken:** tiktoken requires a C extension and network download of the BPE file. This violates the stdlib-only constraint and fails in air-gapped environments (which is literally the target audience for this feature).
3359
+
3360
+ **Accuracy target:** ±10% vs tiktoken cl100k_base. Using `max(word, char)` gives a conservative estimate. We pack to budget × 0.95 (5% safety margin) to absorb estimation error.
3361
+
3362
+ ---
3363
+
3364
+ #### 1.2 Component Parser + Scorer
3365
+
3366
+ **Parse all toolkit components into a unified scoring table:**
3367
+
3368
+ ```python
3369
+ @dataclass
3370
+ class Component:
3371
+ name: str
3372
+ type: str # 'constitution', 'agent', 'skill', 'rule', 'hook-equivalent'
3373
+ source_file: str
3374
+ full_text: str
3375
+ compressed_text: str # after stripping (populated by compressor)
3376
+ tokens_full: int
3377
+ tokens_compressed: int
3378
+ score: float # 0.0 - 1.0
3379
+
3380
+ # Scoring factors
3381
+ safety_criticality: float # 0.0-1.0 (constitution=1.0, guard hooks=0.9)
3382
+ usage_frequency: float # 0.0-1.0 (from stats.json, normalized)
3383
+ persona_relevance: float # 0.0-1.0 (match against active persona)
3384
+ language_relevance: float # 0.0-1.0 (match against project language)
3385
+ ```
3386
+
3387
+ **Scoring formula:**
3388
+ ```python
3389
+ score = (
3390
+ safety_criticality * 0.40 + # Safety always dominates — non-negotiable content gets priority
3391
+ usage_frequency * 0.25 + # Frequently used = valuable — from stats.json invocation counts
3392
+ persona_relevance * 0.20 + # Persona-matched = valuable — e.g. backend-lead boosts API skills
3393
+ language_relevance * 0.15 # Language-matched = contextual — include only relevant rules
3394
+ )
3395
+ # Weight rationale: safety must dominate (0.40) to guarantee constitution + guard rules always fit.
3396
+ # Usage + persona (0.45 combined) ensure the most practical content fills remaining budget.
3397
+ # Language (0.15) is a tiebreaker — most projects use 1-2 languages.
3398
+ # Weights are compile-time constants in v1. If empirical testing (5 standard tasks across
3399
+ # 3 model sizes) shows suboptimal results, expose as --score-weights flag in v2.
3400
+ ```
3401
+
3402
+ **Fixed-score components (always included):**
3403
+
3404
+ | Component | Score | Reason |
3405
+ |-----------|-------|--------|
3406
+ | Constitution (Articles I-V) | 1.0 | Non-negotiable safety |
3407
+ | Guard hooks (destructive, path) | 0.95 | Core safety rules (compiled as text, not hooks) |
3408
+ | Active persona definition | 0.90 | User-selected identity |
3409
+ | Active language rules | 0.85 | Project-specific quality gates |
3410
+
3411
+ **Dynamic-score components:**
3412
+
3413
+ | Component | Base Score | Adjusted By |
3414
+ |-----------|-----------|-------------|
3415
+ | Individual skills | 0.3-0.7 | Usage frequency + persona fit |
3416
+ | Agent definitions | 0.2-0.6 | Persona relevance (only 1 agent in SLM mode) |
3417
+ | Knowledge skills | 0.2-0.5 | Language match + persona match |
3418
+ | Iron Law rules | 0.7 | Always high (quality enforcement) |
3419
+
3420
+ ---
3421
+
3422
+ #### 1.3 Compression Engine
3423
+
3424
+ **Strip low-signal content while preserving semantics:**
3425
+
3426
+ | Strip Target | Savings (est.) | Example |
3427
+ |-------------|---------------|---------|
3428
+ | `## Common Rationalizations` tables | 200-400 tokens/skill | 15 skills have these tables |
3429
+ | `## Related Skills` sections | 50-100 tokens/skill | Routing not useful for SLMs |
3430
+ | `## Verification Checklist` (keep 1-liner summary) | 100-200 tokens/agent | Compress to "Verify: tests pass, no placeholders" |
3431
+ | Markdown headers (collapse hierarchy) | 20-50 tokens/file | `### 2.1.3 Sub-feature` → plain paragraph |
3432
+ | Example code blocks (keep first, strip rest) | 100-500 tokens/skill | Keep 1 example max |
3433
+ | Frontmatter (YAML) | 50-100 tokens/file | Strip entirely from compiled output |
3434
+ | Agent `## Allowed CLI Commands` lists | 200-400 tokens/agent | Not needed when agent won't execute them |
3435
+ | Multi-agent coordination instructions | 300-500 tokens | SLM = single agent, no /orchestrate |
3436
+ | Effort-based budgeting rules | 100 tokens | SLM doesn't manage budgets |
3437
+
3438
+ **Compression levels:**
3439
+
3440
+ ```python
3441
+ COMPRESSION_LEVELS = {
3442
+ 'ultra-light': {
3443
+ 'strip_examples': True,
3444
+ 'strip_rationalizations': True,
3445
+ 'strip_related_skills': True,
3446
+ 'strip_verification': True,
3447
+ 'strip_agent_commands': True,
3448
+ 'strip_multi_agent': True,
3449
+ 'max_skills': 5, # Only top 5 skills by score
3450
+ 'max_agents': 0, # No agent definitions (persona only)
3451
+ 'include_rules': False,
3452
+ },
3453
+ 'light': {
3454
+ 'strip_examples': True,
3455
+ 'strip_rationalizations': True,
3456
+ 'strip_related_skills': True,
3457
+ 'strip_verification': 'summary', # 1-liner
3458
+ 'strip_agent_commands': True,
3459
+ 'strip_multi_agent': True,
3460
+ 'max_skills': 10,
3461
+ 'max_agents': 1, # Persona agent only
3462
+ 'include_rules': True,
3463
+ },
3464
+ 'standard': {
3465
+ 'strip_examples': 'first-only', # Keep 1 example
3466
+ 'strip_rationalizations': True,
3467
+ 'strip_related_skills': True,
3468
+ 'strip_verification': 'summary',
3469
+ 'strip_agent_commands': True,
3470
+ 'strip_multi_agent': True,
3471
+ 'max_skills': 20,
3472
+ 'max_agents': 3,
3473
+ 'include_rules': True,
3474
+ },
3475
+ 'extended': {
3476
+ 'strip_examples': 'first-only',
3477
+ 'strip_rationalizations': 'first-only',
3478
+ 'strip_related_skills': False,
3479
+ 'strip_verification': False,
3480
+ 'strip_agent_commands': False,
3481
+ 'strip_multi_agent': True, # Still stripped for SLMs
3482
+ 'max_skills': 40,
3483
+ 'max_agents': 5,
3484
+ 'include_rules': True,
3485
+ },
3486
+ }
3487
+ ```
3488
+
3489
+ ---
3490
+
3491
+ #### 1.4 Budget Packer
3492
+
3493
+ **Greedy knapsack algorithm:** Sort components by `score / compressed_tokens` ratio (value density), pack until budget exhausted.
3494
+
3495
+ ```python
3496
+ def pack_components(components: list[Component], budget: int) -> list[Component]:
3497
+ """Pack highest-value components into token budget."""
3498
+ # Fixed components always included (constitution, persona, language rules)
3499
+ fixed = [c for c in components if c.score >= 0.85]
3500
+ remaining_budget = budget - sum(c.tokens_compressed for c in fixed)
3501
+
3502
+ # Sort remaining by value density
3503
+ dynamic = sorted(
3504
+ [c for c in components if c.score < 0.85],
3505
+ key=lambda c: c.score / max(c.tokens_compressed, 1),
3506
+ reverse=True
3507
+ )
3508
+
3509
+ packed = list(fixed)
3510
+ for comp in dynamic:
3511
+ if comp.tokens_compressed <= remaining_budget:
3512
+ packed.append(comp)
3513
+ remaining_budget -= comp.tokens_compressed
3514
+
3515
+ return packed
3516
+ ```
3517
+
3518
+ **Budget validation:** After packing, verify total tokens ≤ budget × 0.95 (5% safety margin for tokenizer estimation error).
3519
+
3520
+ **Constitution budget guard:** Before packing dynamic components, verify that fixed components (constitution + persona + language rules) fit within the budget. If `sum(fixed.tokens_compressed) > budget`, fail with: `"Constitution + safety rules alone exceed {budget} token budget. Minimum safe budget: {required}. Use --budget {required} or higher."` This prevents silent omission of safety-critical content.
3521
+
3522
+ ---
3523
+
3524
+ #### 1.5 Emitter
3525
+
3526
+ **Output:** Single markdown file structured for maximum SLM comprehension.
3527
+
3528
+ ```markdown
3529
+ # AI Coding Assistant — System Instructions
3530
+
3531
+ ## Safety Rules (MANDATORY)
3532
+ [Compiled constitution — always first, highest attention position]
3533
+
3534
+ ## Your Identity
3535
+ [Compiled persona — who you are, what you focus on]
3536
+
3537
+ ## Coding Standards
3538
+ [Compiled language rules — active language only]
3539
+
3540
+ ## Key Skills
3541
+ [Top N skill summaries — compressed, actionable]
3542
+
3543
+ ## Quality Checklist
3544
+ [Compiled from Iron Laws + verification — bullet points only]
3545
+ ```
3546
+
3547
+ **Why this structure:**
3548
+ - Safety first = maximum attention weight in transformer architecture
3549
+ - Identity second = establishes persona before task instructions
3550
+ - Standards = project-specific rules that shape code output
3551
+ - Skills at end = reference material, lower attention needed
3552
+
3553
+ ---
3554
+
3555
+ ### Phase 2: Integration (week 2-3)
3556
+
3557
+ #### 2.1 Profile Integration
3558
+
3559
+ **manifest.json addition:**
3560
+ ```json
3561
+ {
3562
+ "profiles": {
3563
+ "offline-slm": ["core"],
3564
+ "offline-slm-extended": ["core", "agents"]
3565
+ }
3566
+ }
3567
+ ```
3568
+
3569
+ **Install behavior:**
3570
+ ```bash
3571
+ ai-toolkit install --profile offline-slm
3572
+
3573
+ # What happens:
3574
+ # 1. Standard install of core components
3575
+ # 2. Runs compile_slm.py with auto-detected settings
3576
+ # 3. Writes compiled output to ~/.ai-toolkit/compiled/
3577
+ # 4. Generates integration instructions for detected local model tools
3578
+ # 5. state.json records profile as "offline-slm"
3579
+ ```
3580
+
3581
+ **No hooks installed:** SLM providers (Ollama, LM Studio) don't support lifecycle hooks. The critical hook behavior (destructive command guard, path guard) is compiled into the system prompt text as rules.
3582
+
3583
+ ---
3584
+
3585
+ #### 2.2 CLI Command
3586
+
3587
+ ```bash
3588
+ ai-toolkit compile-slm # auto-detect model, default budget
3589
+ ai-toolkit compile-slm --budget 4096 # explicit token budget
3590
+ ai-toolkit compile-slm --budget 8192 --persona backend-lead # persona + budget
3591
+ ai-toolkit compile-slm --model-size 8b # auto-select budget for 8B model
3592
+ ai-toolkit compile-slm --model-size 32b # auto-select budget for 32B model
3593
+ ai-toolkit compile-slm --lang typescript,python # include specific language rules
3594
+ ai-toolkit compile-slm --output ./my-system-prompt.md # custom output path
3595
+ ai-toolkit compile-slm --dry-run # show what would be included + token counts (table format below)
3596
+
3597
+ # --dry-run output format:
3598
+ # Budget: 4096 tokens | Level: light | Persona: backend-lead
3599
+ # ┌────────────────────────────┬──────────┬────────┬──────────┐
3600
+ # │ Component │ Score │ Tokens │ Included │
3601
+ # ├────────────────────────────┼──────────┼────────┼──────────┤
3602
+ # │ Constitution (Articles I-V)│ 1.00 │ 420 │ YES │
3603
+ # │ Persona: backend-lead │ 0.90 │ 180 │ YES │
3604
+ # │ Rule: coding-style │ 0.85 │ 310 │ YES │
3605
+ # │ Skill: /review │ 0.68 │ 290 │ YES │
3606
+ # │ ... │ ... │ ... │ ... │
3607
+ # │ Skill: /deploy │ 0.22 │ 350 │ NO (budget)│
3608
+ # └────────────────────────────┴──────────┴────────┴──────────┘
3609
+ # Total: 3,840 / 4,096 tokens (93.7% utilization)
3610
+ ai-toolkit compile-slm --format ollama # output as Ollama Modelfile SYSTEM block
3611
+ ai-toolkit compile-slm --format json-string # JSON-escaped string (for config files)
3612
+ ai-toolkit compile-slm --format raw # plain markdown (default)
3613
+ ```
3614
+
3615
+ **Model size → budget mapping:**
3616
+
3617
+ Note: budget is about *effective instruction following capacity*, not context window. A 128K-context 8B model can *hold* 16K system prompt tokens, but cannot *follow* them reliably. Empirically, SLMs degrade when system prompt exceeds ~10-15% of their effective capacity.
3618
+
3619
+ ```python
3620
+ MODEL_BUDGETS = {
3621
+ '7b': {'budget': 2048, 'level': 'ultra-light'}, # Llama 3.1 8B, Mistral 7B
3622
+ '8b': {'budget': 2048, 'level': 'ultra-light'},
3623
+ '14b': {'budget': 4096, 'level': 'light'}, # Qwen 2.5 14B, Phi-3 14B
3624
+ '32b': {'budget': 8192, 'level': 'standard'}, # Qwen 2.5 32B, Mixtral 8x7B
3625
+ '70b': {'budget': 16384, 'level': 'extended'}, # Llama 3.1 70B
3626
+ }
3627
+ ```
3628
+
3629
+ ---
3630
+
3631
+ #### 2.3 Model Size Detection
3632
+
3633
+ **Auto-detect from Ollama:**
3634
+ ```python
3635
+ def detect_model_size() -> str | None:
3636
+ """Detect running model size from Ollama API."""
3637
+ try:
3638
+ # curl http://localhost:11434/api/tags
3639
+ resp = urllib.request.urlopen('http://localhost:11434/api/tags', timeout=2)
3640
+ data = json.loads(resp.read())
3641
+ models = data.get('models', [])
3642
+ if models:
3643
+ # Extract parameter count from model name: "llama3.1:8b" → "8b"
3644
+ latest = models[0]['name']
3645
+ match = re.search(r'(\d+)[bB]', latest)
3646
+ if match:
3647
+ return match.group(0).lower()
3648
+ except (urllib.error.URLError, TimeoutError, json.JSONDecodeError):
3649
+ pass
3650
+ return None
3651
+ ```
3652
+
3653
+ **Fallback:** If no model detected, use `14b` defaults (4K budget, light compression). User can override with `--model-size`.
3654
+
3655
+ ---
3656
+
3657
+ ### Phase 3: Polish (week 3-4)
3658
+
3659
+ #### 3.1 Integration Guides
3660
+
3661
+ Per-platform setup instructions generated by the compiler.
3662
+
3663
+ **Ollama:**
3664
+ ```bash
3665
+ # 1. Compile system prompt
3666
+ ai-toolkit compile-slm --format ollama --model-size 8b > Modelfile.ai-toolkit
3667
+
3668
+ # 2. Create custom model
3669
+ ollama create my-coder -f Modelfile.ai-toolkit
3670
+
3671
+ # 3. Use
3672
+ ollama run my-coder "implement the payment API"
3673
+ ```
3674
+
3675
+ **LM Studio:**
3676
+ ```
3677
+ 1. ai-toolkit compile-slm --model-size 14b
3678
+ 2. Open LM Studio → Chat → System Prompt
3679
+ 3. Paste contents of ~/.ai-toolkit/compiled/slm-system-prompt.md
3680
+ ```
3681
+
3682
+ **Aider:**
3683
+ ```bash
3684
+ ai-toolkit compile-slm --output .aider.system-prompt.md --model-size 32b
3685
+ aider --model ollama/qwen2.5-coder:32b --system-prompt-file .aider.system-prompt.md
3686
+ ```
3687
+
3688
+ **Continue.dev:**
3689
+ ```bash
3690
+ # 1. Compile to a local file
3691
+ ai-toolkit compile-slm --model-size 14b
3692
+
3693
+ # 2. In .continue/config.json, paste the compiled content into systemMessage
3694
+ # (Continue.dev does not support file references — content must be inline)
3695
+ # Use: ai-toolkit compile-slm --format json-string to get escaped output
3696
+ ```
3697
+
3698
+ ---
3699
+
3700
+ #### 3.2 Compile Quality Validator
3701
+
3702
+ Post-compilation checks:
3703
+
3704
+ | Check | Severity | Description |
3705
+ |-------|----------|-------------|
3706
+ | Constitution present | FAIL | Articles I-V must be in output |
3707
+ | Budget exceeded | FAIL | Token count > budget |
3708
+ | Persona missing (when specified) | WARN | Persona definition not included |
3709
+ | No language rules included | WARN | Project language not detected |
3710
+ | Less than 3 skills included | WARN | Very minimal — may be too sparse |
3711
+ | Output empty | FAIL | Compilation produced no content |
3712
+
3713
+ ---
3714
+
3715
+ ## 6. File Summary
3716
+
3717
+ | File | Action | LOC (est.) | Description |
3718
+ |------|--------|------------|-------------|
3719
+ | `scripts/compile_slm.py` | CREATE | ~500 | Main compiler — orchestrates pipeline: parse → score → compress → pack → emit. Contains `Component` dataclass, scorer, and budget packer |
3720
+ | `scripts/slm_token_counter.py` | CREATE | ~50 | Token estimation (stdlib only) — `estimate_tokens()` function used by compiler and validator |
3721
+ | `scripts/slm_compression.py` | CREATE | ~300 | Compression engine — strip/summarize functions per content type, compression level configs (`COMPRESSION_LEVELS` dict) |
3722
+ | `scripts/slm_integration.py` | CREATE | ~150 | Platform-specific output formatters — Ollama Modelfile, JSON-escaped string, raw markdown, Aider-compatible |
3723
+ | `bin/ai-toolkit.js` | EDIT | +10 | Register `compile-slm` command |
3724
+ | `scripts/install.py` | EDIT | +30 | Handle `--profile offline-slm` |
3725
+ | `manifest.json` | EDIT | +5 | Add offline-slm profile |
3726
+ | `kb/reference/offline-slm-guide.md` | CREATE | ~200 | Integration guides for all platforms |
3727
+ | `tests/test_compile_slm.bats` | CREATE | ~150 | Compilation tests |
3728
+ | `tests/test_slm_budgets.bats` | CREATE | ~80 | Budget compliance tests |
3729
+ | **Total** | | **~1575** | |
3730
+
3731
+ ---
3732
+
3733
+ ## 6a. Non-Functional Requirements
3734
+
3735
+ | Category | Requirement |
3736
+ |----------|-------------|
3737
+ | **Performance** | Compilation < 2 seconds. No network calls during compilation (all data local). |
3738
+ | **Accuracy** | Token estimation ±10% vs tiktoken cl100k_base. Budget compliance: output ≤ budget × 0.95. |
3739
+ | **Determinism** | Same input (agents, skills, rules, persona, language, budget) → identical output. No randomness. |
3740
+ | **Security** | Constitution Articles I-V always present in output — compilation fails if they exceed budget alone. |
3741
+ | **Offline** | Zero network dependencies. Ollama auto-detection gracefully fails to manual fallback. |
3742
+ | **Portability** | Output is plain markdown — consumable by any tool accepting a system prompt string/file. |
3743
+ | **Quality gates** | `ruff check scripts/compile_slm.py scripts/slm_*.py` (0 errors), `mypy --strict scripts/compile_slm.py scripts/slm_*.py` (0 errors). Run before every commit. |
3744
+ | **Type safety** | 100% public API type hints (all function signatures). `Component` dataclass fully typed. Scoring functions use typed parameters, not bare `dict`. |
3745
+
3746
+ ---
3747
+
3748
+ ## 6b. Cache Invalidation & Recompile Triggers
3749
+
3750
+ Compiled output (`~/.ai-toolkit/compiled/slm-system-prompt.md`) is a **derived artifact** — it must be recompiled when inputs change:
3751
+
3752
+ | Trigger | Action |
3753
+ |---------|--------|
3754
+ | `ai-toolkit update` | Auto-recompile if profile is `offline-slm` |
3755
+ | `ai-toolkit install --profile offline-slm` | Always compile |
3756
+ | Agent/skill/rule files changed (detected via mtime) | Warn: "Compiled SLM prompt may be stale. Run `ai-toolkit compile-slm`" |
3757
+ | Manual `ai-toolkit compile-slm` | Always recompile |
3758
+
3759
+ Compiled output includes a header comment: `<!-- Compiled: 2026-04-10T10:30:00Z | Budget: 4096 | Level: light | Persona: backend-lead -->` for staleness detection.
3760
+
3761
+ ---
3762
+
3763
+ ## 6b-bis. Rollback & Removal
3764
+
3765
+ The offline-slm feature is purely additive — removing it is trivial:
3766
+ 1. Delete `scripts/compile_slm.py`, `scripts/slm_token_counter.py`, `scripts/slm_compression.py`, `scripts/slm_integration.py`
3767
+ 2. Remove `"offline-slm"` and `"offline-slm-extended"` from `manifest.json` profiles
3768
+ 3. Remove `compile-slm` from `SCRIPT_COMMANDS` in `bin/ai-toolkit.js`
3769
+ 4. Delete `~/.ai-toolkit/compiled/` directory (user-side)
3770
+ 5. No hooks, no state files, no config entries to clean up
3771
+
3772
+ ---
3773
+
3774
+ ## 6c. Quality Gate Degradation Notice
3775
+
3776
+ **Important:** The `offline-slm` profile strips lifecycle hooks because SLM providers don't support them. This means:
3777
+
3778
+ - No pre-commit quality check (ruff/tsc/mypy)
3779
+ - No destructive command interception (guard hooks)
3780
+ - No session context preservation
3781
+
3782
+ Guard hook behavior is **compiled into the system prompt as text rules** — the SLM is *instructed* not to run destructive commands, but unlike hook-based enforcement, this is advisory, not blocking.
3783
+
3784
+ Documentation must clearly state: **"SLM mode trades enforcement for guidance. Safety rules are present but not machine-enforced."**
3785
+
3786
+ For teams needing enforcement, recommend `--profile offline-slm` combined with a Git pre-commit hook (`.git/hooks/pre-commit`) that runs lint/type-check independently of the AI tool.
3787
+
3788
+ ---
3789
+
3790
+ ## 7. Success Criteria (Overall)
3791
+
3792
+ | Metric | Target |
3793
+ |--------|--------|
3794
+ | Budget tiers | 4 (ultra-light 2K, light 4K, standard 8K, extended 16K) |
3795
+ | Token budget compliance | 100% (output ≤ budget in all cases) |
3796
+ | Constitution inclusion | 100% (always present, all 5 articles) |
3797
+ | Compilation time | < 2 seconds |
3798
+ | Model size auto-detection | Ollama API (with graceful fallback) |
3799
+ | Output formats | 4 (raw markdown, Ollama Modelfile, JSON-escaped string, Aider-compatible) |
3800
+ | Integration guides | 4 platforms (Ollama, LM Studio, Aider, Continue.dev) |
3801
+ | Persona support | 4 personas (backend-lead, frontend-lead, devops-eng, junior-dev) |
3802
+ | Language rule support | 13 languages (all existing rules) |
3803
+ | External dependencies | 0 (stdlib Python only) |
3804
+ | Tests | 40+ |
3805
+
3806
+ ---
3807
+
3808
+ ## 8. Risks and Mitigation
3809
+
3810
+ | Risk | Probability | Impact | Mitigation |
3811
+ |------|-------------|--------|------------|
3812
+ | Token estimation too inaccurate | Medium | Medium | Validate against tiktoken on 50 files, target ±10% |
3813
+ | Compiled prompt too compressed — loses meaning | Medium | High | Validate with 8B model on 5 standard tasks; if quality drops, increase minimum budget |
3814
+ | Ollama API changes | Low | Low | Graceful fallback to manual `--model-size` |
3815
+ | User expects full toolkit features with SLM | Medium | Medium | Clear documentation: "SLM mode = safety + coding standards, not multi-agent orchestration" |
3816
+ | Air-gapped environment can't run `ai-toolkit compile-slm` | Low | Medium | Pre-compile during `install` when network is available; compiled output is self-contained |
3817
+ | Constitution text changes break compiled cache | Low | Low | Recompile on every `ai-toolkit update` |
3818
+
3819
+ ---
3820
+
3821
+ ## 9. Pre-Mortem
3822
+
3823
+ 1. **"Compiled prompt is too generic"** — without the full agent definitions, the SLM may produce generic code that doesn't match project conventions. Mitigation: Language rules and persona have highest priority after constitution — they provide project-specific context.
3824
+ 2. **"Users expect /review to work with 8B model"** — Skill invocations won't be available via SLM providers that lack hook support. Mitigation: Compiled output includes skill *knowledge* as rules, not invocable commands. Clear docs: "Skills are compiled as coding standards, not slash commands."
3825
+ 3. **"Model detects wrong size"** — Ollama model naming is inconsistent (`llama3.1:8b` vs `codellama:7b-instruct`). Mitigation: regex extracts any `\d+[bB]` pattern; fallback to `14b` if ambiguous.
3826
+ 4. **"Other local inference tools emerge"** — Jan.ai, GPT4All, Tabby, etc. Mitigation: Raw markdown output works with any tool that accepts a system prompt file.
3827
+ 5. **"Nobody uses the feature"** — SLM adoption among professional developers may be niche. Mitigation: Low effort (3-4 weeks), high signaling value ("we support air-gapped environments"), enterprise sales enabler.
3828
+
3829
+ ---
3830
+
3831
+ ## 10. Market Positioning
3832
+
3833
+ **Target users:**
3834
+ 1. **Enterprise (air-gapped)** — financial services, defense, healthcare firms that cannot send code to cloud LLM APIs
3835
+ 2. **Privacy-conscious solo devs** — developers who don't want code leaving their machine
3836
+ 3. **Cost-sensitive teams** — startups that can't afford Anthropic/OpenAI API costs at scale
3837
+ 4. **Offline-first** — developers working on trains, planes, or in regions with poor connectivity
3838
+
3839
+ **Competitive advantage:** No existing AI coding toolkit provides a compilation pipeline that adapts its instruction set to model capacity. This is a first-mover feature.
3840
+
3841
+ ---
3842
+
3843
+ ## 11. Next Actions
3844
+
3845
+ 1. [ ] Approve plan
3846
+ 2. [ ] Implement token counter (1.1)
3847
+ 3. [ ] Implement component parser + scorer (1.2)
3848
+ 4. [ ] Implement compression engine (1.3)
3849
+ 5. [ ] Implement budget packer + emitter (1.4, 1.5)
3850
+ 6. [ ] Integrate profile into install.py + manifest.json (2.1)
3851
+ 7. [ ] Create CLI command `compile-slm` (2.2)
3852
+ 8. [ ] Add Ollama model detection (2.3)
3853
+ 9. [ ] Add persona + language aware compilation (2.4, 2.5)
3854
+ 10. [ ] Write integration guides for 4 platforms (3.1)
3855
+ 11. [ ] Compile quality validator (3.2)
3856
+ 12. [ ] Tests + documentation (3.3, 3.4)
3857
+
3858
+ ---
3859
+
3860
+ ## 12. Future
3861
+
3862
+ | Feature | Rationale |
3863
+ |---------|-----------|
3864
+ | Automatic recompile on file changes (file watcher) | v1 uses manual recompile + staleness warning |
3865
+ | Quality benchmarks (run 5 standard tasks, measure output quality per budget tier) | Validate compilation quality empirically before shipping |
3866
+ | Plugin-aware compilation (include memory-pack prompts if installed) | Depends on plugin system maturity |
3867
+ | Model-specific prompt templates (different SLM families prefer different instruction styles) | Needs empirical testing across model families |
3868
+ | `--profile offline-slm --enforce-git-hooks` (install git pre-commit hook for quality gates) | Compensates for stripped lifecycle hooks |
3869
+
3870
+ ---
3871
+
3872
+ ## 13. Cross-Plan Dependencies
3873
+
3874
+ This plan shares modification targets with two other proposed plans:
3875
+
3876
+ | Shared File | This Plan | Enterprise Config Plan | Local Dashboard Plan |
3877
+ |-------------|-----------|----------------------|---------------------|
3878
+ | `scripts/install.py` | +30 LOC (offline-slm profile) | +80 LOC (extends resolution) | — |
3879
+ | `manifest.json` | +5 LOC (offline-slm profile) | +10 LOC (schema refs) | — |
3880
+ | `bin/ai-toolkit.js` | +10 LOC (compile-slm command) | +40 LOC (config subcommands) | +15 LOC (ui command) |
3881
+
3882
+ **If implementing in parallel:** this plan has the smallest changes to shared files — merge first to minimize conflicts.
3883
+
3884
+ **Enterprise Config interaction:** If Enterprise Config ships, `compile-slm` should respect the `extends` chain — compile the merged config, not just local. Add a `--ignore-extends` flag for air-gapped environments without access to the base config.
3885
+
3886
+ ---
3887
+
3888
+ **Last Updated:** 2026-04-10
3889
+
3890
+ ---
3891
+
269
3892
  ## kb/procedures/maintenance-sop.md
270
3893
 
271
3894
  ---
@@ -275,7 +3898,7 @@ service: ai-toolkit
275
3898
  tags: [sop, maintenance, agents, skills, install]
276
3899
  version: "1.4.2"
277
3900
  created: "2026-03-23"
278
- last_updated: "2026-04-09"
3901
+ last_updated: "2026-04-10"
279
3902
  description: "Standard operating procedures for installing, maintaining, and evolving the ai-toolkit."
280
3903
  ---
281
3904
 
@@ -306,12 +3929,15 @@ ai-toolkit install --local --editors cursor,aider # specific editors onl
306
3929
 
307
3930
  Supported editors: `cursor`, `windsurf`, `cline`, `roo`, `aider`, `augment`, `copilot`, `antigravity`.
308
3931
 
309
- To restrict which language rules are injected into `CLAUDE.md`, use `--lang`:
3932
+ To restrict which language rules are injected, use `--lang`:
310
3933
 
311
3934
  ```bash
312
3935
  ai-toolkit install --local --lang python,typescript
3936
+ ai-toolkit install --local --lang python --editors all # language rules propagated to all editors
313
3937
  ```
314
3938
 
3939
+ When `--editors` is combined with `--lang` (or auto-detected languages), language rules are propagated to all configured editors as `ai-toolkit-lang-<lang>` files — not just Claude's `CLAUDE.md`. Similarly, registered custom rules (`~/.ai-toolkit/rules/`) are propagated to directory-based editor configs as `ai-toolkit-custom-<name>` files.
3940
+
315
3941
  **Note:** Hooks are global-only — merged into `~/.claude/settings.json` by `ai-toolkit install`. Project-local `--local` does not install hooks; any legacy `.claude/hooks.json` is removed automatically.
316
3942
 
317
3943
  **Input validation (v1.4.2):** `--only`, `--skip`, `--editors`, and `--lang` are validated on input; an invalid value exits with a clear error before any changes are made.
@@ -391,7 +4017,10 @@ ai-toolkit add-rule ./my-project-rules.md
391
4017
  # → copies to ~/.ai-toolkit/rules/my-project-rules.md
392
4018
 
393
4019
  ai-toolkit update
394
- # → injects the rule into ~/.claude/CLAUDE.md, ~/.cursor/rules, Windsurf, Gemini
4020
+ # → injects the rule into ~/.claude/CLAUDE.md and all global editor configs
4021
+
4022
+ ai-toolkit update --local
4023
+ # → also propagates as ai-toolkit-custom-<name> to directory-based editors (Cursor, Windsurf, Cline, Roo, Augment, Antigravity)
395
4024
  ```
396
4025
 
397
4026
  To unregister (removes from registry **and** strips the block from CLAUDE.md):
@@ -479,7 +4108,7 @@ State tracked in `~/.ai-toolkit/plugins.json`.
479
4108
 
480
4109
  Follow the `documentation-standards` knowledge skill (`app/skills/documentation-standards/SKILL.md`) for full spec. Quick checklist:
481
4110
 
482
- 1. **Choose category directory:** `kb/reference/`, `kb/howto/`, `kb/procedures/`, `kb/troubleshooting/`, or `kb/best-practices/`
4111
+ 1. **Choose category directory:** `kb/reference/`, `kb/howto/`, `kb/procedures/`, `kb/troubleshooting/`, `kb/best-practices/`, or `kb/planning/`
483
4112
  2. **Create file:** kebab-case name, no dates in filename
484
4113
  3. **Add frontmatter** with all 7 required fields: `title`, `category`, `service`, `tags`, `created`, `last_updated`, `description`
485
4114
  4. **Write in English**
@@ -601,6 +4230,288 @@ What `uninstall` does:
601
4230
 
602
4231
  ---
603
4232
 
4233
+ ## kb/procedures/release-preparation-sop.md
4234
+
4235
+ ---
4236
+ title: "SOP: Release Preparation"
4237
+ category: procedures
4238
+ service: ai-toolkit
4239
+ tags: [sop, release, version, publish, changelog, semver]
4240
+ version: "1.5.0"
4241
+ created: "2026-04-10"
4242
+ last_updated: "2026-04-10"
4243
+ description: "Step-by-step checklist for preparing a new ai-toolkit release — version sync, changelog, artifact regeneration, validation, and tagging. Run BEFORE every git tag."
4244
+ ---
4245
+
4246
+ # SOP: Release Preparation
4247
+
4248
+ Complete checklist for preparing a new `@softspark/ai-toolkit` release.
4249
+ Run this **before** tagging. After tagging and publishing, run the
4250
+ [Release Verification SOP](release-verification-sop.md) to smoke-test.
4251
+
4252
+ **Pipeline:**
4253
+ ```
4254
+ Release Preparation (this SOP) → git tag → CI publish → Release Verification SOP
4255
+ ```
4256
+
4257
+ **Time:** 5-10 minutes
4258
+
4259
+ ---
4260
+
4261
+ ## Quick Checklist (TL;DR)
4262
+
4263
+ ```bash
4264
+ # 1. Decide version bump
4265
+ # patch (1.4.2 → 1.4.3): bugfix, typo, doc fix
4266
+ # minor (1.4.2 → 1.5.0): new feature, new skill, new flag
4267
+ # major (1.4.2 → 2.0.0): breaking change
4268
+
4269
+ # 2. Sync version across all files
4270
+ python3 scripts/sync_version.py X.Y.Z # if script exists, else manual
4271
+
4272
+ # 3. Write CHANGELOG.md entry
4273
+ # 4. Regenerate artifacts
4274
+ python3 scripts/generate_agents_md.py > AGENTS.md
4275
+ python3 scripts/generate_llms_txt.py > llms.txt
4276
+ python3 scripts/generate_llms_txt.py --full > llms-full.txt
4277
+
4278
+ # 5. Validate + audit + test
4279
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
4280
+
4281
+ # 6. Commit + tag + push
4282
+ git add -A && git commit -m "chore: release vX.Y.Z"
4283
+ git tag vX.Y.Z
4284
+ git push origin main --tags
4285
+ ```
4286
+
4287
+ ---
4288
+
4289
+ ## Phase 1: Determine Version Bump
4290
+
4291
+ Follow [Semantic Versioning](https://semver.org/):
4292
+
4293
+ | Change Type | Bump | Examples |
4294
+ |-------------|------|---------|
4295
+ | Bugfix, typo, doc-only | **patch** | Fix install flag, correct description |
4296
+ | New feature, skill, agent, flag | **minor** | Add `/hipaa-validate`, add `--output json` |
4297
+ | Breaking CLI change, removed skill, config format change | **major** | Rename `install` to `setup`, remove skill |
4298
+
4299
+ **Rule:** When in doubt, bump minor.
4300
+
4301
+ ---
4302
+
4303
+ ## Phase 2: Sync Version in All Files
4304
+
4305
+ The canonical version lives in `package.json`. These files **must** match:
4306
+
4307
+ ### Mandatory sync (every release)
4308
+
4309
+ | File | Field | How to update |
4310
+ |------|-------|---------------|
4311
+ | `package.json` | `"version": "X.Y.Z"` | Edit directly |
4312
+ | `manifest.json` | `"version": "X.Y.Z"` | Edit directly |
4313
+ | `app/.claude-plugin/plugin.json` | `"version": "X.Y.Z"` | Edit directly |
4314
+
4315
+ ### Auto-synced (no manual action)
4316
+
4317
+ | File | Mechanism |
4318
+ |------|-----------|
4319
+ | `package-lock.json` | Regenerated by `npm install --package-lock-only` |
4320
+
4321
+ ### Conditional sync (only if the doc was modified in this release)
4322
+
4323
+ | File | Field | When to update |
4324
+ |------|-------|---------------|
4325
+ | `kb/procedures/maintenance-sop.md` | frontmatter `version:` | If SOP content changed |
4326
+ | `kb/reference/skills-catalog.md` | frontmatter `version:` | If skills added/removed |
4327
+ | `kb/reference/agents-catalog.md` | frontmatter `version:` | If agents added/removed |
4328
+ | `kb/reference/hooks-catalog.md` | frontmatter `version:` | If hooks changed |
4329
+ | `kb/reference/architecture-overview.md` | frontmatter `version:` | If architecture changed |
4330
+ | `kb/reference/distribution-model.md` | frontmatter `version:` | If install model changed |
4331
+ | `kb/reference/global-install-model.md` | frontmatter `version:` | If install model changed |
4332
+
4333
+ > **Note:** KB `version:` fields track the **document version**, not the toolkit version.
4334
+ > Only bump them when the document content actually changes in this release.
4335
+
4336
+ ### Count sync (if skills/agents/hooks changed)
4337
+
4338
+ | File | What to check |
4339
+ |------|---------------|
4340
+ | `package.json` | `"description"` — skill/agent count |
4341
+ | `README.md` | Badge counts, "What You Get" table |
4342
+ | `app/ARCHITECTURE.md` | Section headings with counts |
4343
+
4344
+ > **Tip:** `validate.py --strict` and `npm test` (metadata contract tests) catch
4345
+ > count drift automatically. If tests pass, counts are correct.
4346
+
4347
+ ### Verification command
4348
+
4349
+ After syncing, verify all mandatory files match:
4350
+
4351
+ ```bash
4352
+ VERSION=$(python3 -c "import json; print(json.load(open('package.json'))['version'])")
4353
+ echo "Target: $VERSION"
4354
+ echo "manifest.json: $(python3 -c "import json; print(json.load(open('manifest.json'))['version'])")"
4355
+ echo "plugin.json: $(python3 -c "import json; print(json.load(open('app/.claude-plugin/plugin.json'))['version'])")"
4356
+ echo "package-lock.json: $(python3 -c "import json; print(json.load(open('package-lock.json'))['version'])")"
4357
+ ```
4358
+
4359
+ All four must print the same version. If not, fix before proceeding.
4360
+
4361
+ ---
4362
+
4363
+ ## Phase 3: Write CHANGELOG Entry
4364
+
4365
+ Add entry at the top of `CHANGELOG.md` (after the header, before previous release):
4366
+
4367
+ ```markdown
4368
+ ## vX.Y.Z — Short Title (YYYY-MM-DD)
4369
+
4370
+ ### Added
4371
+ - **Feature name** — description
4372
+
4373
+ ### Changed
4374
+ - **What changed** — old behavior → new behavior
4375
+
4376
+ ### Fixed
4377
+ - **Bug description** — what was broken and how it's fixed
4378
+
4379
+ ### Removed
4380
+ - **What was removed** — migration path if any
4381
+ ```
4382
+
4383
+ **Rules:**
4384
+ - Use **bold** for feature names
4385
+ - Start descriptions with a verb (Added, Changed, Fixed, Removed)
4386
+ - Reference skill names with backticks and slash: `/hipaa-validate`
4387
+ - Include script names: `scripts/hipaa_scan.py`
4388
+ - Include count changes: `Skill count: 91 → 92`
4389
+ - Date format: `YYYY-MM-DD`
4390
+ - Title: short, descriptive, no version number repetition
4391
+
4392
+ ---
4393
+
4394
+ ## Phase 4: Regenerate Artifacts
4395
+
4396
+ ```bash
4397
+ python3 scripts/generate_agents_md.py > AGENTS.md
4398
+ python3 scripts/generate_llms_txt.py > llms.txt
4399
+ python3 scripts/generate_llms_txt.py --full > llms-full.txt
4400
+ ```
4401
+
4402
+ Check if anything actually changed:
4403
+
4404
+ ```bash
4405
+ git diff --stat AGENTS.md llms.txt llms-full.txt
4406
+ ```
4407
+
4408
+ If no diff, the artifacts are already current. If there is a diff, stage them.
4409
+
4410
+ ---
4411
+
4412
+ ## Phase 5: Validate, Audit, Test
4413
+
4414
+ Run the full quality gate:
4415
+
4416
+ ```bash
4417
+ python3 scripts/validate.py --strict
4418
+ python3 scripts/audit_skills.py --ci
4419
+ npm test
4420
+ ```
4421
+
4422
+ **Expected results:**
4423
+ - `validate.py`: `Errors: 0 | Warnings: 0 | VALIDATION PASSED`
4424
+ - `audit_skills.py`: `HIGH: 0 | WARN: 0` (INFO is acceptable)
4425
+ - `npm test`: `1..N` with zero `not ok`
4426
+
4427
+ **One-liner:**
4428
+ ```bash
4429
+ python3 scripts/validate.py --strict && python3 scripts/audit_skills.py --ci && npm test
4430
+ ```
4431
+
4432
+ **If tests fail:** Fix the issue, do NOT skip. Common failures:
4433
+ - Stale counts → re-run `generate:all` or fix README/ARCHITECTURE
4434
+ - Missing frontmatter → add to new KB docs
4435
+ - Broken symlink → `ai-toolkit doctor --fix`
4436
+
4437
+ ---
4438
+
4439
+ ## Phase 6: Commit
4440
+
4441
+ Stage all release files:
4442
+
4443
+ ```bash
4444
+ git add package.json manifest.json app/.claude-plugin/plugin.json
4445
+ git add package-lock.json
4446
+ git add CHANGELOG.md
4447
+ git add AGENTS.md llms.txt llms-full.txt
4448
+ git add -p # review and stage any other changes
4449
+ ```
4450
+
4451
+ Commit:
4452
+
4453
+ ```bash
4454
+ git commit -m "chore: release vX.Y.Z"
4455
+ ```
4456
+
4457
+ ---
4458
+
4459
+ ## Phase 7: Tag and Push
4460
+
4461
+ ```bash
4462
+ git tag vX.Y.Z
4463
+ git push origin main --tags
4464
+ ```
4465
+
4466
+ This triggers `.github/workflows/publish.yml` which:
4467
+ 1. Runs `validate.py --strict`
4468
+ 2. Runs `npm test`
4469
+ 3. Publishes to npm as `@softspark/ai-toolkit@X.Y.Z`
4470
+
4471
+ **After CI completes:** Run the [Release Verification SOP](release-verification-sop.md)
4472
+ to smoke-test the published package.
4473
+
4474
+ ---
4475
+
4476
+ ## Rollback
4477
+
4478
+ If a bad release was published:
4479
+
4480
+ ```bash
4481
+ # Unpublish from npm (within 72h)
4482
+ npm unpublish @softspark/ai-toolkit@X.Y.Z
4483
+
4484
+ # Or deprecate (preferred — doesn't break existing installs)
4485
+ npm deprecate @softspark/ai-toolkit@X.Y.Z "Known issue: <description>. Use vA.B.C instead."
4486
+
4487
+ # Delete tag
4488
+ git tag -d vX.Y.Z
4489
+ git push origin --delete vX.Y.Z
4490
+ ```
4491
+
4492
+ ---
4493
+
4494
+ ## Checklist Summary
4495
+
4496
+ | # | Step | Command / Action | Pass Criteria |
4497
+ |---|------|-----------------|---------------|
4498
+ | 1 | Version bump type | Decide patch/minor/major | — |
4499
+ | 2 | `package.json` version | Edit `"version"` | Matches target |
4500
+ | 3 | `manifest.json` version | Edit `"version"` | Matches target |
4501
+ | 4 | `plugin.json` version | Edit `"version"` | Matches target |
4502
+ | 5 | `package-lock.json` | `npm install --package-lock-only` | Matches target |
4503
+ | 6 | Count sync | Check `package.json` description, README | `validate.py` passes |
4504
+ | 7 | CHANGELOG.md | Add release entry | Entry exists for vX.Y.Z |
4505
+ | 8 | Regenerate artifacts | `generate_agents_md.py`, `generate_llms_txt.py` | No unexpected diff |
4506
+ | 9 | Validate | `validate.py --strict` | 0 errors, 0 warnings |
4507
+ | 10 | Security audit | `audit_skills.py --ci` | 0 HIGH |
4508
+ | 11 | Tests | `npm test` | All pass |
4509
+ | 12 | Commit | `git commit` | Clean working tree |
4510
+ | 13 | Tag | `git tag vX.Y.Z` | Tag exists |
4511
+ | 14 | Push | `git push origin main --tags` | CI triggered |
4512
+
4513
+ ---
4514
+
604
4515
  ## kb/procedures/release-verification-sop.md
605
4516
 
606
4517
  ---