pan-wizard 3.36.0 → 3.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +3645 -0
- package/README.md +36 -48
- package/agents/pan-conductor.md +2 -3
- package/agents/pan-counterfactual.md +1 -2
- package/agents/pan-debugger.md +14 -3
- package/agents/pan-design-checker.md +1 -2
- package/agents/pan-designer.md +1 -2
- package/agents/pan-document_code.md +1 -2
- package/agents/pan-hardener.md +2 -3
- package/agents/pan-integration-checker.md +1 -2
- package/agents/pan-knowledge.md +1 -2
- package/agents/pan-meta-reviewer.md +3 -3
- package/agents/pan-optimizer.md +4 -4
- package/agents/pan-phase-researcher.md +79 -488
- package/agents/pan-plan-checker.md +12 -2
- package/agents/pan-planner.md +10 -6
- package/agents/pan-previewer.md +1 -2
- package/agents/pan-project-researcher.md +70 -499
- package/agents/pan-release.md +1 -1
- package/agents/pan-reviewer.md +9 -3
- package/agents/pan-roadmapper.md +12 -7
- package/agents/pan-verifier.md +7 -2
- package/bin/install-lib.cjs +134 -14
- package/bin/install.js +59 -12
- package/commands/pan/army.md +1 -1
- package/commands/pan/autodev.md +76 -0
- package/commands/pan/exec-phase.md +3 -4
- package/commands/pan/focus-auto.md +14 -7
- package/commands/pan/focus-design.md +4 -5
- package/commands/pan/focus-docs.md +24 -0
- package/commands/pan/learn.md +13 -6
- package/commands/pan/milestone-audit.md +8 -1
- package/commands/pan/milestone-done.md +10 -5
- package/commands/pan/milestone-new.md +2 -2
- package/commands/pan/new-project.md +15 -12
- package/commands/pan/plan-phase.md +15 -9
- package/commands/pan/quick.md +1 -1
- package/commands/pan/review-deep.md +11 -5
- package/commands/pan/settings.md +6 -0
- package/commands/pan/verify-phase.md +6 -2
- package/docs/AGENTS.md +917 -0
- package/docs/ARCHITECTURE.md +1054 -0
- package/docs/CLI-REFERENCE.md +4582 -0
- package/docs/COMPARISON.md +116 -0
- package/docs/DEVELOPMENT.md +490 -0
- package/docs/ECOSYSTEM-REVIEW-2026-06.md +107 -0
- package/docs/ECOSYSTEM-REVIEW-2026-08.md +349 -0
- package/docs/EXAMPLES.md +584 -0
- package/docs/FAQ.md +254 -0
- package/docs/FIELD-HARVEST-2026-07.md +88 -0
- package/docs/FIELD-REPORT-army-2026-06.md +121 -0
- package/docs/HOOKS.md +638 -0
- package/docs/IMPROVEMENT-TODO.md +246 -0
- package/docs/INTERNALS.md +485 -0
- package/docs/MIGRATION-v2-to-v3.md +258 -0
- package/docs/SECURITY-HARDENING.md +230 -0
- package/docs/SKILLS-ENHANCEMENT-PLAN-V2.md +122 -0
- package/docs/SKILLS-FULL-TEXT.md +9685 -0
- package/docs/SKILLS-REFERENCE.md +1037 -0
- package/docs/TROUBLESHOOTING.md +1084 -0
- package/docs/USER-GUIDE.md +1437 -0
- package/docs/archive/SKILLS-ENHANCEMENT-PLAN.md +230 -0
- package/docs/audits/AUDIT-2026-08-e2e-ROADMAP.md +117 -0
- package/docs/audits/AUDIT-2026-08-e2e-VERIFICATION.md +693 -0
- package/docs/audits/AUDIT-2026-08-e2e.md +1546 -0
- package/docs/audits/OUTSTANDING-2026-08.md +275 -0
- package/docs/audits/STABILITY-TEST-2026-08.md +640 -0
- package/docs/branding/README-brand-draft.md +119 -0
- package/docs/branding/handoff/IMPLEMENTATION.md +205 -0
- package/docs/branding/handoff/README-brand-draft.md +119 -0
- package/docs/branding/handoff/image-prompts.md +46 -0
- package/docs/branding/image-prompts.md +46 -0
- package/docs/branding/nano-banana-prompts.txt +60 -0
- package/docs/context-monitor.md +13 -0
- package/docs/decisions/ADR-0001-strategic-feature-prioritization.md +83 -0
- package/docs/decisions/ADR-0002-copilot-cli-runtime.md +71 -0
- package/docs/decisions/ADR-0003-smart-execution.md +84 -0
- package/docs/decisions/ADR-0004-copilot-cli-interaction.md +89 -0
- package/docs/decisions/ADR-0005-command-naming.md +80 -0
- package/docs/decisions/ADR-0006-focus-commands.md +101 -0
- package/docs/decisions/ADR-0007-ship-v1.md +73 -0
- package/docs/decisions/ADR-0008-map-codebase-mermaid-togaf.md +80 -0
- package/docs/decisions/ADR-0009-production-deployment-checklist.md +68 -0
- package/docs/decisions/ADR-0010-standards-integration.md +79 -0
- package/docs/decisions/ADR-0011-help-commands-rewrite.md +64 -0
- package/docs/decisions/ADR-0012-contributors-attribution-index.md +42 -0
- package/docs/decisions/ADR-0013-strategic-feature-additions.md +53 -0
- package/docs/decisions/ADR-0014-internal-cleanup-code-quality.md +53 -0
- package/docs/decisions/ADR-0015-focus-auto-runner.md +296 -0
- package/docs/decisions/ADR-0016-enhanced-doc-sync.md +54 -0
- package/docs/decisions/ADR-0017-enhanced-focus-design.md +69 -0
- package/docs/decisions/ADR-0018-user-reality-testing.md +84 -0
- package/docs/decisions/ADR-0019-end-to-end-user-acceptance-testing.md +113 -0
- package/docs/decisions/ADR-0020-ai-drift-prevention.md +55 -0
- package/docs/decisions/ADR-0021-codebase-mapper-v2.md +76 -0
- package/docs/decisions/ADR-0022-lifecycle-completeness.md +63 -0
- package/docs/decisions/ADR-0023-opus-4-7-adoption.md +89 -0
- package/docs/decisions/ADR-0024-spec-b-v2-completion.md +137 -0
- package/docs/decisions/ADR-0025-behavioral-guardrails-layer.md +129 -0
- package/docs/decisions/ADR-0026-self-improvement-loop.md +171 -0
- package/docs/decisions/ADR-0027-doc-code-link-graph.md +164 -0
- package/docs/decisions/ADR-0028-unified-agents-skills-target.md +46 -0
- package/docs/decisions/ADR-0029-experiment-evaluator.md +76 -0
- package/docs/decisions/ADR-0030-planning-concurrency.md +37 -0
- package/docs/decisions/ADR-0031-focus-auto-backlog-source.md +47 -0
- package/docs/decisions/ADR-0032-squad-model.md +64 -0
- package/docs/decisions/ADR-0033-army-campaign.md +40 -0
- package/docs/decisions/ADR-0034-scheduled-campaigns.md +57 -0
- package/docs/decisions/ADR-0035-army-hud-dashboard.md +40 -0
- package/docs/decisions/ADR-0036-agentic-memory-architecture.md +106 -0
- package/docs/decisions/ADR-0037-pin-security-agents-to-opus.md +51 -0
- package/docs/decisions/ADR-0038-skill-aligned-decomposition.md +41 -0
- package/docs/decisions/ADR-0039-worktree-as-project-variant.md +37 -0
- package/docs/decisions/ADR-0040-memory-injection-defense.md +55 -0
- package/docs/decisions/ADR-0041-mcp-dual-era-bridge.md +47 -0
- package/docs/decisions/ADR-0042-design-parity-both-flows.md +45 -0
- package/docs/decisions/ADR-0043-addressable-planning-roots.md +56 -0
- package/docs/decisions/ADR-0044-bound-the-cached-context.md +104 -0
- package/docs/decisions/ADR-0045-agent-plugins-bundle.md +76 -0
- package/docs/decisions/ADR-0046-subagent-frontmatter-decisions.md +39 -0
- package/docs/decisions/ADR-0047-pan-harness.md +76 -0
- package/docs/decisions/ADR-0048-one-core-per-project.md +43 -0
- package/docs/decisions/ADR-0049-machine-verdicts-and-findings-ledger.md +116 -0
- package/docs/specs/ai_drift_prevention_featureai.md +286 -0
- package/docs/specs/codebase_mapper_v2_featureai.md +930 -0
- package/docs/specs/command_naming_optimization_featureai.md +930 -0
- package/docs/specs/comp_analysis_markdown_v1.md +149 -0
- package/docs/specs/continuous_autonomous_execution_featureai.md +304 -0
- package/docs/specs/contributors_attribution_index_featureai.md +476 -0
- package/docs/specs/copilot_cli_interaction_featureai.md +868 -0
- package/docs/specs/copilot_cli_runtime_featureai.md +647 -0
- package/docs/specs/custom-skills-assessment.md +461 -0
- package/docs/specs/deployment_audit_findings_featureai.md +243 -0
- package/docs/specs/doc_code_link_graph_featureai.md +253 -0
- package/docs/specs/documentation_alignment_cleanup_featureai.md +69 -0
- package/docs/specs/e2e_visual_testing_featureai.md +615 -0
- package/docs/specs/end_to_end_user_acceptance_testing_featureai.md +810 -0
- package/docs/specs/evidence-loop-plan.md +76 -0
- package/docs/specs/evidence_loop_featureai.md +497 -0
- package/docs/specs/focus-auto-optimize-category.md +449 -0
- package/docs/specs/focus-auto-optimize-superplan.md +188 -0
- package/docs/specs/focus_auto_runner_featureai.md +1102 -0
- package/docs/specs/focus_design_v2_featureai.md +197 -0
- package/docs/specs/focus_sync_v2_featureai.md +202 -0
- package/docs/specs/git_integration_overhaul_featureai.md +326 -0
- package/docs/specs/glm_zai_provider_featureai.md +200 -0
- package/docs/specs/googlecli_adoption_featureai.md +661 -0
- package/docs/specs/help_commands_rewrite_featureai.md +860 -0
- package/docs/specs/industry_analysis_strategic_additions_featureai.md +231 -0
- package/docs/specs/industry_comparison_and_feature_gaps_featureai.md +674 -0
- package/docs/specs/internal_cleanup_code_quality_featureai.md +486 -0
- package/docs/specs/investigations/README.md +31 -0
- package/docs/specs/investigations/digests/2026-09-28.md +47 -0
- package/docs/specs/investigations/digests/2026-10-08.md +62 -0
- package/docs/specs/investigations/digests/2026-10-10.md +61 -0
- package/docs/specs/investigations/features/debugger-red-loop-first.md +55 -0
- package/docs/specs/investigations/features/harness-cross-host-behaviour.md +93 -0
- package/docs/specs/investigations/features/optimize-apply-regression-gate.md +66 -0
- package/docs/specs/investigations/features/reviewer-smell-baseline.md +57 -0
- package/docs/specs/investigations/features/trace-failure-capture.md +69 -0
- package/docs/specs/lifecycle_completeness_featureai.md +754 -0
- package/docs/specs/local-first-ci.md +42 -0
- package/docs/specs/map_codebase_mermaid_togaf_featureai.md +805 -0
- package/docs/specs/market-delta-2026-09-superplan.md +310 -0
- package/docs/specs/market-ideas-2026-09.md +131 -0
- package/docs/specs/market-ideas-2026-10.md +428 -0
- package/docs/specs/market-ideas-ledger.md +177 -0
- package/docs/specs/memory-optimization-2026-10.md +285 -0
- package/docs/specs/multi-track-planning-roots.md +143 -0
- package/docs/specs/multi_model_routing_featureai.md +687 -0
- package/docs/specs/multi_model_routing_superplan.md +197 -0
- package/docs/specs/opus_47_existing_enhancements_featureai.md +247 -0
- package/docs/specs/opus_47_extended_features_featureai.md +363 -0
- package/docs/specs/opus_47_extended_features_v2_featureai.md +397 -0
- package/docs/specs/pan_focus_commands_featureai.md +2336 -0
- package/docs/specs/pan_zcode_mcp_bridge_featureai.md +120 -0
- package/docs/specs/product-excellence-superplan.md +315 -0
- package/docs/specs/product_excellence_program_2026_q4_featureai.md +365 -0
- package/docs/specs/production_deployment_readiness_featureai.md +433 -0
- package/docs/specs/production_readiness_ship_v1_featureai.md +698 -0
- package/docs/specs/real_test_enforcement_featureai.md +213 -0
- package/docs/specs/reality-check-2026-09-superplan.md +362 -0
- package/docs/specs/right-sized-ceremony.md +74 -0
- package/docs/specs/roadmap_and_competition_2026_q4_featureai.md +390 -0
- package/docs/specs/self_improvement_loop_featureai.md +875 -0
- package/docs/specs/skill-aligned-decomposition.md +155 -0
- package/docs/specs/standards_integration_featureai.md +833 -0
- package/docs/specs/testing-system-redesign-2026-09.md +267 -0
- package/docs/specs/user_reality_testing_system_featureai.md +788 -0
- package/docs/specs/watch-triggers.json +22 -0
- package/docs/specs/workflow-integration-redesign.md +742 -0
- package/docs/specs/workflow_state_consistency_featureai.md +183 -0
- package/hooks/dist/pan-context-monitor.js +9 -2
- package/hooks/dist/pan-cost-logger.js +5 -0
- package/hooks/dist/pan-guard.js +28 -5
- package/hooks/dist/pan-stop-guard.js +9 -9
- package/hooks/dist/pan-trace-logger.js +5 -0
- package/package.json +10 -2
- package/pan-wizard-core/bin/lib/agents-md.cjs +57 -8
- package/pan-wizard-core/bin/lib/autodev-report.cjs +180 -0
- package/pan-wizard-core/bin/lib/autodev.cjs +642 -0
- package/pan-wizard-core/bin/lib/ci.cjs +143 -0
- package/pan-wizard-core/bin/lib/commands.cjs +1 -1
- package/pan-wizard-core/bin/lib/config.cjs +31 -2
- package/pan-wizard-core/bin/lib/constants.cjs +27 -1
- package/pan-wizard-core/bin/lib/core.cjs +22 -10
- package/pan-wizard-core/bin/lib/cost.cjs +14 -4
- package/pan-wizard-core/bin/lib/cross-family.cjs +164 -0
- package/pan-wizard-core/bin/lib/debug-cases.cjs +104 -0
- package/pan-wizard-core/bin/lib/failure-class.cjs +81 -0
- package/pan-wizard-core/bin/lib/findings.cjs +34 -4
- package/pan-wizard-core/bin/lib/focus.cjs +1 -1
- package/pan-wizard-core/bin/lib/hosts.cjs +88 -0
- package/pan-wizard-core/bin/lib/init.cjs +7 -2
- package/pan-wizard-core/bin/lib/knowledge.cjs +2 -0
- package/pan-wizard-core/bin/lib/merge.cjs +50 -0
- package/pan-wizard-core/bin/lib/mutate.cjs +16 -5
- package/pan-wizard-core/bin/lib/next-action.cjs +112 -0
- package/pan-wizard-core/bin/lib/optimize.cjs +2 -2
- package/pan-wizard-core/bin/lib/phase.cjs +48 -4
- package/pan-wizard-core/bin/lib/plan-lint.cjs +150 -1
- package/pan-wizard-core/bin/lib/preview.cjs +23 -4
- package/pan-wizard-core/bin/lib/requirements-lint.cjs +130 -0
- package/pan-wizard-core/bin/lib/research-lint.cjs +150 -3
- package/pan-wizard-core/bin/lib/review-deep.cjs +14 -22
- package/pan-wizard-core/bin/lib/review-vote.cjs +105 -0
- package/pan-wizard-core/bin/lib/runner.cjs +2 -16
- package/pan-wizard-core/bin/lib/sizing.cjs +356 -0
- package/pan-wizard-core/bin/lib/squads.cjs +1 -1
- package/pan-wizard-core/bin/lib/state-write.cjs +247 -0
- package/pan-wizard-core/bin/lib/uat.cjs +149 -0
- package/pan-wizard-core/bin/lib/verdict.cjs +26 -3
- package/pan-wizard-core/bin/lib/verify-plan-set.cjs +30 -1
- package/pan-wizard-core/bin/lib/verify-preflight.cjs +2 -2
- package/pan-wizard-core/bin/lib/verify-red.cjs +37 -1
- package/pan-wizard-core/bin/lib/verify-retro.cjs +3 -2
- package/pan-wizard-core/bin/lib/verify.cjs +59 -4
- package/pan-wizard-core/bin/lib/worktree.cjs +70 -0
- package/pan-wizard-core/bin/pan-tools.cjs +168 -27
- package/pan-wizard-core/learnings/universal/autonomous-loop.md +2 -2
- package/pan-wizard-core/references/continuation-format.md +3 -3
- package/pan-wizard-core/references/guardrails.md +1 -1
- package/pan-wizard-core/references/model-profiles.md +2 -3
- package/pan-wizard-core/references/questioning.md +24 -0
- package/pan-wizard-core/references/ui-brand.md +4 -6
- package/pan-wizard-core/references/verdict-contract.md +2 -1
- package/pan-wizard-core/templates/config.json +5 -1
- package/pan-wizard-core/templates/debug.md +1 -0
- package/pan-wizard-core/templates/discovery.md +2 -2
- package/pan-wizard-core/templates/requirements.md +9 -0
- package/pan-wizard-core/workflows/add-phase.md +4 -6
- package/pan-wizard-core/workflows/assumptions.md +19 -1
- package/pan-wizard-core/workflows/diagnose-issues.md +2 -0
- package/pan-wizard-core/workflows/discuss-phase.md +4 -2
- package/pan-wizard-core/workflows/exec-lean.md +69 -0
- package/pan-wizard-core/workflows/exec-phase.md +36 -8
- package/{commands/pan → pan-wizard-core/workflows}/focus-doc-audit.md +13 -29
- package/{commands/pan → pan-wizard-core/workflows}/focus-drift-walking.md +3 -18
- package/{commands/pan → pan-wizard-core/workflows}/focus-exec.md +8 -23
- package/{commands/pan → pan-wizard-core/workflows}/focus-plan.md +6 -20
- package/{commands/pan → pan-wizard-core/workflows}/focus-scan.md +3 -16
- package/{commands/pan → pan-wizard-core/workflows}/focus-sync.md +7 -21
- package/pan-wizard-core/workflows/health.md +6 -0
- package/pan-wizard-core/workflows/help.md +31 -53
- package/pan-wizard-core/workflows/insert-phase.md +4 -6
- package/pan-wizard-core/workflows/learn.md +3 -3
- package/pan-wizard-core/workflows/milestone-audit.md +2 -2
- package/pan-wizard-core/workflows/milestone-new.md +38 -95
- package/pan-wizard-core/workflows/new-project.md +96 -258
- package/pan-wizard-core/workflows/optimize.md +107 -5
- package/pan-wizard-core/workflows/phase-tests.md +10 -0
- package/pan-wizard-core/workflows/plan-phase.md +86 -43
- package/pan-wizard-core/workflows/profile.md +50 -0
- package/pan-wizard-core/workflows/progress.md +10 -0
- package/pan-wizard-core/workflows/quick.md +34 -43
- package/pan-wizard-core/workflows/research-phase.md +118 -40
- package/pan-wizard-core/workflows/resume-project.md +18 -2
- package/pan-wizard-core/workflows/settings.md +21 -24
- package/pan-wizard-core/workflows/todo-check.md +6 -2
- package/pan-wizard-core/workflows/transition.md +22 -4
- package/pan-wizard-core/workflows/uat-walkthrough.md +90 -0
- package/pan-wizard-core/workflows/update.md +1 -1
- package/pan-wizard-core/workflows/verify-phase.md +2 -2
- package/scripts/build-agent-plugin.js +35 -15
- package/scripts/build-plugin.js +10 -1
- package/scripts/deprecate-old-versions.js +70 -3
- package/scripts/generate-skills-docs.py +1 -1
- package/scripts/git-hooks/pre-push +27 -0
- package/scripts/install-git-hooks.js +7 -5
- package/scripts/install-pre-push.cjs +38 -0
- package/scripts/land.cjs +151 -0
- package/scripts/local-ci.cjs +501 -0
- package/scripts/local-schedule.cjs +277 -0
- package/scripts/prompt-lint.cjs +32 -0
- package/scripts/prompt-rules.cjs +156 -0
- package/scripts/run-tests.cjs +7 -1
- package/scripts/test-quality-lint.cjs +3 -3
- package/scripts/test-surface.cjs +9 -2
- package/agents/pan-research-synthesizer.md +0 -246
- package/commands/pan/audit-deployment.md +0 -386
- package/commands/pan/dashboard.md +0 -22
- package/commands/pan/discord.md +0 -19
- package/commands/pan/mcp-bridge.md +0 -143
- package/commands/pan/milestone-cleanup.md +0 -19
- package/commands/pan/milestone-gaps.md +0 -35
- package/commands/pan/optimize.md +0 -115
- package/commands/pan/profile.md +0 -76
- package/commands/pan/research-phase.md +0 -189
- package/pan-wizard-core/bin/lib/bridge.cjs +0 -269
- package/pan-wizard-core/templates/research-project/architecture.md +0 -210
- package/pan-wizard-core/templates/research-project/features.md +0 -153
- package/pan-wizard-core/templates/research-project/pitfalls.md +0 -206
- package/pan-wizard-core/templates/research-project/stack.md +0 -126
- package/pan-wizard-core/templates/research-project/summary.md +0 -170
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Right-sized ceremony: PAN's cost and time on well-specified work
|
|
2
|
+
|
|
3
|
+
**Date:** `2026-10-24`, last amended `2026-10-25` · **Status:** Built; re-measurements below · **Owner decisions:** `2026-10-24` (right-sized by default; `quality` keeps the old behaviour; `--quality` per run)
|
|
4
|
+
|
|
5
|
+
## The problem
|
|
6
|
+
|
|
7
|
+
Three paired H10 runs (`docs/specs/product-excellence-superplan.md`, H10 row) compared PAN with Claude Code on its own, both on Opus 5.5, on precisely specified work. The bare host completed every rep. PAN completed the same work and cost much more:
|
|
8
|
+
|
|
9
|
+
| Pair | Bare host | PAN | PAN ÷ bare |
|
|
10
|
+
|---|---|---|---|
|
|
11
|
+
| Single module, `/pan:quick --full` (discount codes) | $0.32, 72 s | $2.82, 748 s | about 9× cost, 10× time |
|
|
12
|
+
| Two-phase roadmap, `autodev` | $0.62, 144 s | $19.19, 4,659 s (one rep; the next ran out of budget) | about 30× cost, 32× time |
|
|
13
|
+
|
|
14
|
+
## Where it went (instrumented runs, `2026-10-24`)
|
|
15
|
+
|
|
16
|
+
PAN's cost logger (`.planning/metrics/tokens.jsonl`) and autodev's ledger, on the same seeds, with PAN `3.36.0`:
|
|
17
|
+
|
|
18
|
+
**`/pan:quick --full`, $2.67, 585 s.** Planner 214 s and 22k output tokens; plan-checker 111 s; executor 123 s (the work itself); verifier 48 s. The orchestrating session cost about $1.44, $1.10 of it cache writes (each new agent context re-caches its prompt and the project files).
|
|
19
|
+
|
|
20
|
+
**One autodev phase pair, $10.00, 40 min.** plan-phase 1: $5.24 and 21 min. exec-phase 1: $2.86 and 9 min. plan-phase 2: $1.75 and 10 min. Per agent: the plan-checker ran four times (91k output tokens, about $2.06, 12 min) and the planner three times (68k, about $2.00, 12 min), because the checker sent the plans back for revision; the executor cost $0.63.
|
|
21
|
+
|
|
22
|
+
**Reading.** About 70% of PAN's cost and time was planning ceremony on work whose requirements already said exactly what to build: research, then a planner and a plan-checker revising a plan, serially, every agent on the launch model at high effort. A cheaper model alone helps less than it seems: Sonnet 5.5 is half Opus 5.5's price, not a fifth.
|
|
23
|
+
|
|
24
|
+
## What changed
|
|
25
|
+
|
|
26
|
+
| Lever | `balanced` (default) and `budget` | `quality` |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| Ceremony | **Auto-sized** (`pan-wizard-core/bin/lib/sizing.cjs`): a phase whose requirements (at least one) each have an acceptance criterion or name the artifact to build is **lean** up to three times `workflow.lean_max_requirements` (15 requirements; one session holds a lean phase, and the planner designs anything larger): above `lean_max_requirements` (5) the engine splits it into plans of at most that many tasks: no research, one short plan, the engine's plan check instead of the plan-checker agent (`pan-tools size plan-check`, which records the plan check exec-phase needs), and execution without the test-generation agent (one reviewer pass and the verifier still run; autodev runs the phase in one session, `exec-lean`). A quick task whose description names an artifact and runs to 15 words or more is lean: the engine writes its one-task plan (`pan-tools size quick-plan`) and no planner runs. Anything else is **full** | Full everywhere |
|
|
29
|
+
| Revision loop | One revision, then the user decides | Two revisions |
|
|
30
|
+
| Checker output | Verdict and blocking issues only, a word budget | Unbudgeted |
|
|
31
|
+
| Models | Plan-checker and researchers on the mid tier (Sonnet 5.5 on Claude) | Every agent on the reasoning tier |
|
|
32
|
+
|
|
33
|
+
`--quality` on `/pan:quick`, `/pan:plan-phase`, `/pan:exec-phase` and `autodev run` (passed to every unit) gives one run the full ceremony; `--full` does the same for one command. `workflow.ceremony: full` keeps the full ceremony for a project without changing its models. `/pan:new-project` and `/pan:settings` describe the choice.
|
|
34
|
+
|
|
35
|
+
**Not done: lower effort on checks.** Claude Code reads an agent's `effort:` from its file, and no workflow passes effort per run, so lowering it there would change `quality` too. It stays as it was.
|
|
36
|
+
|
|
37
|
+
**Unchanged on purpose:** the executor's prompt (a tier-0 cassette records it, and re-recording needs an API key), the verifier on every phase (the phase gate needs its verdict), and every deterministic gate.
|
|
38
|
+
|
|
39
|
+
## Re-measurement (`2026-10-24`, the same seeds, owner-approved $8 cap)
|
|
40
|
+
|
|
41
|
+
Every run passed its project's tests and every case of the hidden acceptance check (`check-discounts.cjs`, `check-tasks.cjs`).
|
|
42
|
+
|
|
43
|
+
| Task | PAN before | PAN after | Bare host | PAN ÷ bare, after |
|
|
44
|
+
|---|---|---|---|---|
|
|
45
|
+
| Discount codes | $2.67, 585 s (`/pan:quick --full`) | $0.44, 90 s (`/pan:quick`, lean: no planner, no checker) | $0.32, 76 s | 1.4× cost, 1.2× time |
|
|
46
|
+
| Two-phase roadmap, `autodev` | $19.19, 4,659 s (H10 rep 1) | $4.52, 1,003 s (both phases built; stopped at the $4.50 cap before phase 2's separate verify unit) | $0.54, 121 s | 8× cost, 8× time |
|
|
47
|
+
|
|
48
|
+
Autodev per unit after: plan-phase 1 $1.29 (256 s), exec-phase 1 $1.85 (466 s), plan-phase 2 $0.77 (143 s), exec-phase 2 $0.47 (109 s). No research and no plan-checker ran; one planner per phase and one executor per phase did.
|
|
49
|
+
|
|
50
|
+
**Step 2 (built after this measurement).** A lean phase is planned by the engine: `pan-tools size phase-plan` writes one task per requirement from the requirements' own text, checks the plan in code, records the plan check and commits it, and `autodev` runs it for a lean phase's `plan` unit with no session, so a lean phase costs one exec-phase session. `/pan:plan-phase` uses it too.
|
|
51
|
+
|
|
52
|
+
**Step 2 re-measured (`2026-10-24`, owner-approved $5 cap):** autodev on the same two-phase roadmap finished both phases (`done`) for **$3.09 in 691 s**, passing every hidden acceptance case: plan 1 by the engine ($0, 1 s), execute 1 $1.65 (353 s), plan 2 by the engine ($0, 1 s), execute 2 $1.44 (334 s). Against the bare host's $0.54 and 121 s that is **5.7× cost and 5.7× time, down from about 30×** ($19.19 and 4,659 s before right-sizing). What remains is the exec-phase session itself (the executor, the verifier and the orchestrating workflow), about $1.50 and six minutes a phase.
|
|
53
|
+
|
|
54
|
+
**Bug found by the run, fixed:** when the cap was spent mid-unit, a retry started a session with `--max-budget-usd 0.00`, which Claude Code refuses; a retry now stops with `budget_usd` first.
|
|
55
|
+
|
|
56
|
+
## Optimisation set 2 and the regression check (`2026-10-24`, owner-approved $10 cap)
|
|
57
|
+
|
|
58
|
+
**Change:** autodev's unit prompt now starts with the command (`/pan:exec-phase 1 (PAN autodev unit, unattended: …)`). The earlier transcript showed each unit spending its first turns finding the command, because inside a sentence Claude Code did not expand it; now the host expands it and the session starts at `init`. **Effect on cost: within noise** ($2.89 against $3.09 for the two-phase roadmap; units ran engine plan $0, execute $1.87, engine plan $0, execute $1.02). The exploration turns were cheap; what each session pays is its fixed context: about 90k tokens of cache writes (Claude Code's system prompt and tools, the workflow, the agents' results), about $0.45 a session, and PAN's MCP tool list is only about 2,300 tokens of it.
|
|
59
|
+
|
|
60
|
+
**Regression check, live:**
|
|
61
|
+
|
|
62
|
+
| Check | Result |
|
|
63
|
+
|---|---|
|
|
64
|
+
| Lean autodev, two phases | `done`, $2.89, 745 s; hidden acceptance check passed |
|
|
65
|
+
| `/pan:quick --full --quality` (the old path) | planner, plan-checker and verifier all ran; $2.00, 618 s; hidden check passed |
|
|
66
|
+
| `/pan:plan-phase 1` on a vague phase (`clarify-export`) | sized `full` by itself: the planner wrote two plans, no engine plan; the plan-checker did not run before the session's $2.50 cap |
|
|
67
|
+
|
|
68
|
+
Plus a fresh packed install on all five runtimes (no errors; `size` answers in each runtime's core; the engine plans a lean phase and exec-phase reads its plan check as `ok`) and the full local CI.
|
|
69
|
+
|
|
70
|
+
**Where the rest of autodev's cost is:** per phase, an orchestrating session (about $0.85: mostly that fixed context), the executor (about $0.40) and the verifier (about $0.17). Plain Claude Code pays one session's fixed context for the whole job. The levers left are structural: run consecutive lean phases in one execute session, or let the engine's gates stand in for the verifier on a lean phase (a quality trade the H4 corpus cannot yet price, since its defects are too easy).
|
|
71
|
+
|
|
72
|
+
**One session per lean phase (`2026-10-25`).** For a lean phase, autodev's execute unit no longer starts `/pan:exec-phase` (an orchestrating session with an 11k-token workflow that then spawns a separate executor). It hands the session `pan-wizard-core/workflows/exec-lean.md`, which does the executor's work in that session (the plan as specification, tests never weakened, `supersedes_tests` recorded as deviations, a commit per task, the summary and state updates) and then spawns the verifier and records its verdict (and, since the quality check below, one reviewer pass before it). The phase gate and `phase complete` stay in autodev. The verifier stays, so quality is unchanged; what goes is one session's fixed context and the orchestration turns. A full phase and `--quality` keep exec-phase.
|
|
73
|
+
|
|
74
|
+
**Does the lean path drop quality? (`2026-10-25`, owner-approved $3).** The same `pan-reviewer`, same prompt, over the lean brief build and a bare host's brief build, both 33/33 on the hidden check: lean 1 high, 1 medium, 1 low (a save that overwrites the live file, so a crash mid-write loses the shop; `Number.isInteger` past the safe range; dates not normalised); bare 1 high, 2 medium, 1 low (restock past the safe range saves a file `load` then rejects; unchecked totals; an empty file read as an empty shop with no flush before rename; two instances on one file). Equivalent builds, the lean one at 193 lines and 46 tests against 336 and 21. Reading: no quality drop on this evidence, and one defect in each build that neither the tests nor the verifier could see. So the lean path now keeps **one reviewer pass** (`exec-lean` step 4, and exec-phase's `code_review` on a lean phase): about $0.40 a phase; a `NEEDS_FIXES` gets one fix round, and what stays open is recorded as deferred, never passed silently. $2.08 spent (two reviews at $0.38 and $0.42, a bare rebuild at $1.28 that the harness's $1-a-session floor stopped after two of four sessions, the product already complete).
|
|
@@ -0,0 +1,390 @@
|
|
|
1
|
+
# /featureAI Spec: Q4 2026 Roadmap and Competitive Analysis
|
|
2
|
+
|
|
3
|
+
**Date:** `2026-10-11`
|
|
4
|
+
**Status:** Realigned `2026-10-12` — the owner set features first and exposure later. The program is now `product_excellence_program_2026_q4_featureai.md`; this spec's competitive analysis stands as background, its publication bet (F3) moves to that program's deferred Exposure phase, and F1 and F2 are absorbed there (R45 in W1; R46 by V9 and V1 in W1, with V2 in W2).
|
|
5
|
+
**Tree:** `feat/memory-optimization-2026-10` @ `4994ed5` (`3.34.0`, released `2026-10-10`) plus the same-day doc audit's working tree
|
|
6
|
+
**Supersedes:** `industry_comparison_and_feature_gaps_featureai.md` (`2026-02-28`) and `industry_analysis_strategic_additions_featureai.md` (`2026-03-03`) as the strategic read
|
|
7
|
+
**Evidence base (all `2026-10-11`):** the market idea scan with a feature-level read of Superpowers (ledger rows MI-110 to MI-126), the doc audit, the reality check (§11 of `docs/ECOSYSTEM-REVIEW-2026-08.md`), and a feature-level landscape of Spec Kit, BMAD, gsd-core and OpenSpec read from their READMEs, docs, releases APIs and registries. Queue ids (M, R) refer to the Addendum of `docs/specs/market-ideas-2026-10.md`.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Phase 0: Problem Framing
|
|
12
|
+
|
|
13
|
+
### 0.1 Problem statement
|
|
14
|
+
|
|
15
|
+
PAN Wizard is ahead of its category on rigour and nearly invisible in it. The reality check found a product that does what its docs say on all five runtimes. It has a behavioural harness that installs from a packed artifact, an MCP server with a forbidden-verb list and a human merge gate, a cost ledger priced from a dated rate table, and deterministic orchestration beside portable markdown workflows. No direct peer was found with all four. But `pan-wizard` drew 1,599 npm downloads from `2026-09-11` to `2026-10-10` (api.npmjs.org). OpenSpec drew about 2.9 million and BMAD about 77,500 in a comparable window. Superpowers is listed in three official plugin marketplaces; PAN is listed in none. Three things make this the quarter to act:
|
|
16
|
+
- **The peers are moving onto PAN's ground.** gsd-core shares PAN's `.planning/` file vocabulary and loop. BMAD v7 adds an eval tool. Superpowers runs an eval lab against real CLIs.
|
|
17
|
+
- **The distribution channels have opened.** Anthropic takes directory submissions through a developer portal, and Codex and Copilot ship default marketplaces.
|
|
18
|
+
- **A host is changing under PAN.** OpenCode 2 is now pushed to users and does not load PAN's plugin.
|
|
19
|
+
|
|
20
|
+
### 0.2 Scope
|
|
21
|
+
|
|
22
|
+
| In scope | Out of scope |
|
|
23
|
+
|---|---|
|
|
24
|
+
| A sequenced roadmap for `2026-10` to `2026-12`: correctness, publication, measured behaviour, plan quality | A sixth runtime (MI-017), standardising `.planning/` (MI-018), skills-marketplace listings on quality grounds alone (MI-020) |
|
|
25
|
+
| A competitive analysis of the direct peers and the host-tool natives | The IDE peers in `docs/COMPARISON.md`, whose rows did not change this quarter |
|
|
26
|
+
| Implement-ready specs for the roadmap's first items (F1, F2) and design-level specs for the strategic ones (F3, F4, F5) | Building anything: this spec feeds `/execplan` |
|
|
27
|
+
| The decisions only the owner can make, named and sized | Telemetry of any kind (MI-056) |
|
|
28
|
+
|
|
29
|
+
### 0.3 Success criteria (roadmap level)
|
|
30
|
+
|
|
31
|
+
```text
|
|
32
|
+
SC-1: PAN's plugin is installable from at least one official marketplace (Anthropic's directory, Codex's or Copilot's default marketplaces) by 2026-12-31, from a tracked artifact a test validates.
|
|
33
|
+
SC-2: Every capability claim in README.md and docs/COMPARISON.md is backed by code or a harness scenario; the claims audit of the next reality check finds no FALSE row and no WEAKENED row in PAN's own column.
|
|
34
|
+
SC-3: The OpenCode plugin loads on OpenCode 1.x and 2.x, proven by a live check on each.
|
|
35
|
+
SC-4: A failed test gate cannot complete a phase on any runtime, enforced by code that a test pins.
|
|
36
|
+
SC-5: At least one portable behavioural chain (exec-phase) passes the harness on a second host CLI (MI-104).
|
|
37
|
+
SC-6: UAT runs as a command on all five runtimes and feeds the diagnose loop (closes the COMPARISON "UAT only as a template" row).
|
|
38
|
+
SC-7: No regression: the full suite, every release gate and harness tier 0 stay green at each step.
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### 0.4 User stories
|
|
42
|
+
|
|
43
|
+
```text
|
|
44
|
+
As a developer choosing a planning framework, I want to find and install PAN from my host's marketplace,
|
|
45
|
+
so that I can try it in one step, instead of discovering it on npm and reading an install guide.
|
|
46
|
+
|
|
47
|
+
As an OpenCode user moved to OpenCode 2, I want PAN's compaction hand-off to keep working,
|
|
48
|
+
so that a long session resumes where it stopped, instead of losing PAN's position silently.
|
|
49
|
+
|
|
50
|
+
As a team lead relying on verification, I want a failing test suite to stop the phase in code,
|
|
51
|
+
so that a verifier's mistaken "passed" cannot ship broken work, instead of trusting the prompt.
|
|
52
|
+
|
|
53
|
+
As a product owner, I want to walk a phase's acceptance criteria with the agent and have gaps diagnosed,
|
|
54
|
+
so that UAT is part of the loop, instead of a template nobody runs.
|
|
55
|
+
|
|
56
|
+
As a maintainer, I want the portable chain proven on more than Claude Code,
|
|
57
|
+
so that "works on five runtimes" means behaviour, not only installation.
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Phase 1: Internal Reconnaissance (what the reality check established)
|
|
63
|
+
|
|
64
|
+
### 1.1 Strengths, with evidence
|
|
65
|
+
|
|
66
|
+
| Strength | Evidence |
|
|
67
|
+
|---|---|
|
|
68
|
+
| Installs and runs on all five runtimes | Packed-tarball install of all five in seconds; `validate deployment` clean on each; harness tier 0 all green except the Antigravity gate (no `agy`) |
|
|
69
|
+
| Fresh context per plan | `workflows/exec-phase.md` spawns `pan-executor` with paths only; tier-2 chain scenarios count reads from each executor's own transcript |
|
|
70
|
+
| Deterministic orchestration where control flow is knowable | Native Claude Code workflow scripts (`buildNativeWorkflowScripts` in `bin/install-lib.cjs`) beside portable markdown; the oracle measured parity (MI-002) |
|
|
71
|
+
| Engine over MCP with guard rails | `pan-wizard-core/mcp/`: fixed registry, `FORBIDDEN_VERB`, no-shell spawn, a per-request merge token; every resource reads cleanly on a bare project |
|
|
72
|
+
| Cost ledger correctness | Dated rate table re-read from the pricing page; long-prompt pricing (MI-107); per-agent transcript attribution; usage-window stop (MI-109) |
|
|
73
|
+
| Behavioural harness against packed installs | `harness/` tiers 0–2 with a findings ledger (ADR-0047) |
|
|
74
|
+
| Documentation trust | Counts live in one file; doc-lint and model-version lints gate the suite; the doc audit's prompt sweeps found real code bugs |
|
|
75
|
+
|
|
76
|
+
### 1.2 Weaknesses, with evidence
|
|
77
|
+
|
|
78
|
+
| Weakness | Evidence | Roadmap answer |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| Invisible | 1,599 npm downloads in the last month; no official marketplace listing; no GitHub stars recorded since the repository went public | F3 |
|
|
81
|
+
| Some guarantees are prompt-only | The verifier's test gate (RC3); the `/pan:army` merge gate (README says so); the MCP caps use caller-supplied counts | F2, and honest wording (done) |
|
|
82
|
+
| UAT does not run | No command writes a UAT file; `diagnose-issues.md` is loaded by no command; the native diagnose script exists on Claude Code only | F4 |
|
|
83
|
+
| Behaviour proven on one host | The model tiers of the harness run on Claude Code only (MI-104 plumbing built, parsers missing) | H3 |
|
|
84
|
+
| OpenCode 2 plugin break | v2's loader needs `setup` or `effect` (RC1) | F1 |
|
|
85
|
+
| Plan quality depends on prose | No Review Focus, no executor concerns state, some spawns inherit the session model | M39, M41, M42 |
|
|
86
|
+
|
|
87
|
+
### 1.3 Runtime matrix today
|
|
88
|
+
|
|
89
|
+
| Runtime | Commands/skills | Agents | Hooks / observers | MCP | Native workflows | Behaviour proven |
|
|
90
|
+
|---|---|---|---|---|---|---|
|
|
91
|
+
| Claude Code | ✅ | ✅ | ✅ full set | ✅ `.mcp.json` (global: printed command) | ✅ | ✅ tier 2 |
|
|
92
|
+
| Codex | ✅ `.agents/skills` | ✅ `.toml` | ✅ (async observers) | printed TOML snippet by design | — | install and MCP listing only |
|
|
93
|
+
| Gemini CLI | ✅ `.toml` | ✅ | partial: no monitor or loggers (no matching event) | ✅ | — | install and MCP listing only; live model runs blocked on an API key |
|
|
94
|
+
| OpenCode | ✅ | ✅ | plugin (compaction only; breaks on v2) | ✅ (v1 shape, which v2 normalises) | — | install, discovery and MCP listing |
|
|
95
|
+
| Copilot CLI | ✅ | ✅ `.agent.md` | ✅ | ✅ `.github/mcp.json` | — | install, MCP, a live compaction check |
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Phase 2: Competitive Analysis
|
|
100
|
+
|
|
101
|
+
### 2.1 Positioning
|
|
102
|
+
|
|
103
|
+
| Project | Position in its own words (paraphrased) | Centre of gravity |
|
|
104
|
+
|---|---|---|
|
|
105
|
+
| **PAN Wizard** | A planning, state and verification layer installed into five host CLIs | Rigour and portability: durable state, gated loop, measured behaviour |
|
|
106
|
+
| **Superpowers** (`obra/superpowers`, v7.0.0 `2026-10-10`) | A skills plugin that changes how the agent works: brainstorm, plan, subagent-driven development, TDD, review | Behavioural discipline through always-loaded skills; widest marketplace presence |
|
|
107
|
+
| **gsd-core** (`open-gsd/gsd-core`, v1.16.0 `2026-10-05`) | Lightweight meta-prompting and context engineering; fresh-context subagents through discuss, plan, execute, verify, ship | The closest twin: same `.planning/` vocabulary, many gates, cross-AI reviewer lanes |
|
|
108
|
+
| **Spec Kit** (`github/spec-kit`, v1.1.3 `2026-10-09`) | Structured processes and templates for any agent: spec-driven development, bug fixing, idea assessment | Breadth: widest agent coverage, extension catalog, workflow engine |
|
|
109
|
+
| **BMAD** (`bmad-code-org/BMAD-METHOD`, v6.12.2 `2026-10-11`; v7 preview) | Agile AI-driven development with product, architecture, UX, dev and test perspectives | Planning depth through personas; v7 moves to skills-native distribution |
|
|
110
|
+
| **OpenSpec** (`Fission-AI/OpenSpec`, v1.14.1 `2026-10-06`) | A lightweight spec layer so human and agent agree before code is written | Simplicity and reach: delta specs, a strict validator, the largest install base |
|
|
111
|
+
|
|
112
|
+
### 2.2 Capability matrix
|
|
113
|
+
|
|
114
|
+
Y = shipped · P = partial or weaker · N = not found on `2026-10-11` · B = beta or preview. PAN's column is from the code; the others are from each project's own README, docs and releases.
|
|
115
|
+
|
|
116
|
+
| Capability | PAN | Superpowers | gsd-core | Spec Kit | BMAD | OpenSpec |
|
|
117
|
+
|---|---|---|---|---|---|---|
|
|
118
|
+
| Durable planning state | Y `.planning/` | P dated spec and plan files, ephemeral run ledger | Y `.planning/` + a JSON state contract with transactions | Y `.specify/` per feature | Y `_bmad-output/`; v7 initiatives | Y `openspec/specs` + delta changes |
|
|
119
|
+
| Research before planning | Y researcher agents | P brainstorming with optional recon | Y researchers, claim provenance | Y plan Phase 0; `assess` extension | Y deep-recon packs | N (reads code first) |
|
|
120
|
+
| Plan verification | Y plan-checker loop | Y self-review + user review of the plan | Y plan-checker and several gates; plans reviewed by other AIs | Y `analyze`, checklists | Y readiness gate | Y `validate --strict` |
|
|
121
|
+
| Fresh context per unit of work | Y | Y per task (SDD) | Y per executor, parallel waves | N (agent's own session) | P (`bmad-build-auto`; bmad-loop) | N |
|
|
122
|
+
| Parallel worktrees | P Claude-only army | N by design (one worktree per plan) | Y on by default, several runtimes | P community extension | P bmad-loop (beta), five CLIs | N |
|
|
123
|
+
| Post-execution verification | Y verifier + test gate (prompt-enforced) | Y verification before completion, final review | Y goal-backward verify, invalidation on change, API coverage | Y `converge` loop | Y coverage audit, re-verification | Y `verify` (skipped ≠ passed) |
|
|
124
|
+
| UAT that runs | N template only | N | Y in `verify-work`; live-DOM UAT (off by default) | P bug extension's verified/partial/failed | P walkthrough skill | N |
|
|
125
|
+
| Review agents | Y reviewer, hardener, meta-reviewer | Y task + final reviewer | Y review pipeline, disposition ledger | P community | Y multi-layer parallel review | N |
|
|
126
|
+
| Behavioural evals of the framework | Y harness vs packed installs, findings ledger | Y Quorum lab vs real CLIs (separate repo) | P mutation testing, conformance tiers | P `selftest` extension | B `bmad-eval` (v7) | N (QA smoke harness unshipped) |
|
|
127
|
+
| Cost ledger / model routing | Y ledger, dated rates, profiles, usage-window stop | P per-role model guidance in prose | P profiles, failure escalation; no rate table | P per-step model | P handoff routing | N |
|
|
128
|
+
| MCP server | Y read tools + gated merge | N | Y read and write state (no merge gate found) | B read-only, experimental | N | N |
|
|
129
|
+
| Hooks | Y four runtimes + OpenCode plugin | Y session-start bootstrap | Y monitor and several guards | Y extension and native hooks | P customization hooks | N |
|
|
130
|
+
| Deterministic orchestration | Y native scripts on Claude Code | N | B Workflow-tool capability, Claude-only | Y workflow engine (YAML) | P bmad-loop (Python) | N |
|
|
131
|
+
| Runtimes targeted | 5 (+ Antigravity via `.agents/`) | ~15 harnesses | 16 (dropped Gemini CLI in v1.15.0) | 42 integration keys | 48 install targets (v6) | ~44 tools |
|
|
132
|
+
| Official marketplace listings | none | Claude, Codex-curated, Grok | none (self-hosted) | none | none (own marketplace) | none |
|
|
133
|
+
| Telemetry | none (MI-056) | opt-out logo ping in one tool | none (a recorded non-goal) | none found | none found | on by default, opt-out |
|
|
134
|
+
|
|
135
|
+
### 2.3 Momentum (last 90 days, from `2026-07-13`)
|
|
136
|
+
|
|
137
|
+
| Project | Releases (GitHub API) | Downloads per month (primary registry unless marked) | Stars (secondary) |
|
|
138
|
+
|---|---|---|---|
|
|
139
|
+
| PAN Wizard | 6 | npm 1,599 | none recorded |
|
|
140
|
+
| Superpowers | 5 | (plugin installs: ~1.15M official-marketplace installs, a scrape of Anthropic's endpoint, secondary) | ~297k |
|
|
141
|
+
| gsd-core | 11 | npm ~46.5k (+ ~60k on the legacy package) | ~10k |
|
|
142
|
+
| Spec Kit | 44 | PyPI ~84k (pypistats, secondary) | ~141k |
|
|
143
|
+
| BMAD | 4 on v6 (v7 unreleased) | npm ~77.5k | ~54k |
|
|
144
|
+
| OpenSpec | 11 | npm ~2.9M; Homebrew ~1.7k | ~72k |
|
|
145
|
+
|
|
146
|
+
### 2.4 Where PAN stands
|
|
147
|
+
|
|
148
|
+
- **Ahead (protect):**
|
|
149
|
+
- The harness against packed installs.
|
|
150
|
+
- The MCP merge gate. gsd-core's MCP writes state with no merge gate found.
|
|
151
|
+
- The dated cost ledger.
|
|
152
|
+
- Deterministic plus portable orchestration with a measured parity oracle.
|
|
153
|
+
- **Threat to these:** Superpowers' Quorum and BMAD's `bmad-eval` make "we test the framework itself" a contested claim within a quarter.
|
|
154
|
+
- **Parity:**
|
|
155
|
+
- Durable state (gsd-core matches it file for file).
|
|
156
|
+
- Plan verification.
|
|
157
|
+
- Fresh-context execution (Superpowers, gsd-core).
|
|
158
|
+
- Review agents (BMAD's layered review is richer on adversarial and intent layers).
|
|
159
|
+
- **Behind:**
|
|
160
|
+
- Distribution and reach on every measure.
|
|
161
|
+
- Running UAT (gsd-core).
|
|
162
|
+
- Parallel worktrees beyond Claude Code (gsd-core, bmad-loop).
|
|
163
|
+
- Cross-AI review (gsd-core reviewer lanes; MI-044 is WATCH).
|
|
164
|
+
- Runtime breadth. This is by decision (MI-017); it is a positioning choice, not a gap to close.
|
|
165
|
+
- **Converging signals:**
|
|
166
|
+
- Explicit model choice per dispatch (Superpowers).
|
|
167
|
+
- Verification invalidated by change (gsd-core; PAN has MI-067).
|
|
168
|
+
- Skipped is not passed (OpenSpec; PAN has MI-064).
|
|
169
|
+
- Evals of the framework (three peers).
|
|
170
|
+
- The read on `2026-09-10` holds: the market converged on PAN's thesis. The gap is publication, and now also proof of behaviour off Claude Code.
|
|
171
|
+
|
|
172
|
+
### 2.5 Differentiation statement
|
|
173
|
+
|
|
174
|
+
PAN is the planning, state and verification layer whose behaviour is measured against what users actually install, on every host it supports, with spend and merges under explicit control. The roadmap keeps every item inside that sentence. Anything that only widens reach (more runtimes, telemetry, marketplace spam) or only adds ceremony (personas, always-on bootstraps) is out.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## Phase 3: Design — the roadmap
|
|
179
|
+
|
|
180
|
+
### 3.1 Three strategic bets
|
|
181
|
+
|
|
182
|
+
1. **Get found (publication).** One official marketplace listing by year end, from a tracked artifact (F3). Every review since August names this gap. Two peers with smaller rigour have larger reach because of it.
|
|
183
|
+
2. **Make every guarantee true in code (trust).** The test gate (F2), UAT that runs (F4), the OpenCode 2 plugin (F1), explicit models on every spawn (M39). The audit this quarter showed the docs had drifted ahead of the code in exactly these places.
|
|
184
|
+
3. **Prove behaviour beyond one host (evidence).** The portable chain on a second host CLI (MI-104), a baseline arm before any prompt change (M43, MI-114), and live re-runs of the two open findings (R49). This keeps PAN's lead on evals while two peers build theirs.
|
|
185
|
+
|
|
186
|
+
### 3.2 Horizons
|
|
187
|
+
|
|
188
|
+
| Horizon | Window | Theme | Items | Exit |
|
|
189
|
+
|---|---|---|---|---|
|
|
190
|
+
| **H1** | `2026-10` | Correctness and trust | F1 (R45), F2 (R46), M39, R47, R48, the local-marketplace path (RC10) | SC-3, SC-4; no spawn inherits the session model |
|
|
191
|
+
| **H2** | `2026-11` | Publication and plan quality | F3 (MI-025, after the owner's decision), M41 Review Focus, M42 executor concerns, the Antigravity variant once `agy` confirms the manifest (MI-026) | SC-1 submitted; plans carry Review Focus |
|
|
192
|
+
| **H3** | `2026-12` | Measured behaviour | F4 UAT that runs, MI-104 second-host chain, M43 then M35, R49; F5 cross-AI review lane as a design spike | SC-5, SC-6; open harness findings resolved |
|
|
193
|
+
|
|
194
|
+
### 3.3 Decisions only the owner can make
|
|
195
|
+
|
|
196
|
+
| Decision | Blocks | Options | Recommendation |
|
|
197
|
+
|---|---|---|---|
|
|
198
|
+
| Plugin hosting | F3 | (a) a tracked plugin folder in this repo, committed by the release workflow, submitted through Anthropic's developer portal; (b) a second npm package with `source: npm` | (a): the portal tracks a public repo's folder or tag. It is the model the one listed peer uses (a pinned commit). It keeps a single package |
|
|
199
|
+
| Harness spend cap | MI-104, M43, M35, R49 | a per-run `--max-usd` | A standing monthly cap, recorded in `harness/README.md` **Decided `2026-10-12`: none** — per-run `--max-usd` (program §3.4) |
|
|
200
|
+
| Gemini API key | Gemini live checks (M33, MI-051) | a key in the environment for live gates only | Provide one, or accept "Gemini behaviour unproven" in COMPARISON |
|
|
201
|
+
| ADR-0049 acceptance | none in code; doc truth | accept or amend | Accept: M11 and M12 shipped on it in `3.32.0` **Accepted `2026-10-12`** |
|
|
202
|
+
|
|
203
|
+
### 3.4 What not to do (re-affirmed for the roadmap)
|
|
204
|
+
|
|
205
|
+
All the entries carried in the reality check hold. The roadmap adds four:
|
|
206
|
+
- **Do not chase runtime count.** Peers list 15 to 48 targets. PAN's answer is proven behaviour on its five (MI-017).
|
|
207
|
+
- **Do not add personas or an always-on bootstrap skill.** Both trade progressive disclosure for ceremony (MI-119).
|
|
208
|
+
- **Do not emit OpenCode v2 config shapes** while the published schema is v1.
|
|
209
|
+
- **Do not ship telemetry to measure reach.** Registry download counts and marketplace install counts are enough (MI-056).
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Phase 4: Specifications
|
|
214
|
+
|
|
215
|
+
### F1 — OpenCode plugin for v1 and v2 (R45, H1, P1, S)
|
|
216
|
+
|
|
217
|
+
**Problem:** OpenCode 2's loader requires a default export with `id` and `setup` (or `effect`) and throws "Plugin must export a default definition with an id and an effect or setup function". PAN exports `{ id, server }`, so the compaction hand-off is lost on v2. The documented fix (`opencode.ai/v2/docs/build/plugins`, migrate-v1 page) is one object carrying both: v1 calls `server()`, v2 calls `setup()`. It needs OpenCode ≥1.18.29.
|
|
218
|
+
|
|
219
|
+
**Files**
|
|
220
|
+
- `pan-wizard-core/opencode/pan-wizard.js`: add `setup(ctx)`.
|
|
221
|
+
- `tests/opencode-plugin.test.cjs`: the export-shape test changes from "`{ id, server }`, nothing else" to "`id`, `server`, `setup`", plus the cases below.
|
|
222
|
+
- `docs/HOOKS.md`, `docs/TROUBLESHOOTING.md`: one line each on v2.
|
|
223
|
+
|
|
224
|
+
**Implementation steps**
|
|
225
|
+
1. Factor the existing compaction body into `pushBlock(directory, target)`. It is fail-open and inert outside work in flight.
|
|
226
|
+
2. Add `function setup(ctx)`. It resolves the directory like `server` (`ctx.directory || ctx.worktree || process.cwd()`) and calls `ctx.session.hook('compaction', (e) => { … })`. The handler pushes the block onto `e.system` when that is an array. Otherwise it pushes onto `e.context` when that is an array, otherwise it does nothing. Wrap everything in try/catch.
|
|
227
|
+
3. Export `module.exports = { id: 'pan-wizard', server, setup }`. Keep CommonJS: the installer writes `{"type":"commonjs"}` above `plugins/`.
|
|
228
|
+
4. Live check on `@opencode/cli` 2.0.26 in a scratch home. Install PAN with `--opencode`, start a session with a phase in flight, and run `/compact`. The summary must carry PAN's block. Record the event's actual field names, because `system` was read from the binary, not the docs. If the field differs, change step 2 and its tests before shipping.
|
|
229
|
+
|
|
230
|
+
**Tests (at least 10)**
|
|
231
|
+
- Happy path:
|
|
232
|
+
- (1) `setup` registers exactly one `compaction` hook on a fake `ctx.session`.
|
|
233
|
+
- (2) Firing it with `{ system: [] }` pushes one block equal to the `server` path's block.
|
|
234
|
+
- (3) `server` behaviour is unchanged (the existing parity tests still pass).
|
|
235
|
+
- (4) `PAN_TRACK` routing works through `setup`.
|
|
236
|
+
- Edge cases:
|
|
237
|
+
- (5) No `ctx.session` or no `hook` function: `setup` returns without throwing.
|
|
238
|
+
- (6) An event with neither `system` nor `context`: no throw, no write.
|
|
239
|
+
- (7) No phase in flight: nothing pushed.
|
|
240
|
+
- (8) The block is truncated at the same cap.
|
|
241
|
+
- Error cases:
|
|
242
|
+
- (9) A throwing `hook` does not escape `setup`.
|
|
243
|
+
- (10) A `.planning/` read error inside the handler is swallowed.
|
|
244
|
+
- Runtime-specific:
|
|
245
|
+
- (11) The installer still copies the file byte for byte for OpenCode only.
|
|
246
|
+
- (12) Uninstall still removes it and keeps the user's plugins.
|
|
247
|
+
|
|
248
|
+
**Runtime matrix:** OpenCode 1.x (≥1.18.29) and 2.x ✅. The other four runtimes are unaffected (no plugin).
|
|
249
|
+
|
|
250
|
+
### F2 — The test gate enforced in code (R46, H1, P1, M)
|
|
251
|
+
|
|
252
|
+
**Problem:** The verifier's prompt says a failed or non-running suite means `gaps_found`. The recorder (`verdictFromVerificationFrontmatter` in `pan-wizard-core/bin/lib/verdict.cjs`) only warns `test_gate_failed_under_pass` and keeps `passed`. `workflows/exec-phase.md` routes on the word `findings record --raw` prints. A verifier that writes `status: passed` with `test_gate: failed` therefore completes the phase.
|
|
253
|
+
|
|
254
|
+
**Files**
|
|
255
|
+
- `pan-wizard-core/bin/lib/verdict.cjs`: the frontmatter adapter.
|
|
256
|
+
- The `findings record` command module (read `cmdFindingsRecord` first).
|
|
257
|
+
- `pan-wizard-core/bin/lib/verify-*.cjs`: the module behind `verify reconcile`.
|
|
258
|
+
- Tests: `tests/verdict*.test.cjs` and the reconcile tests.
|
|
259
|
+
- `README.md` (revert the interim wording to the stronger claim once true), `docs/CLI-REFERENCE.md`, `agents/pan-verifier.md` (unchanged rule, now backed).
|
|
260
|
+
|
|
261
|
+
**Implementation steps**
|
|
262
|
+
1. In the adapter, when `test_gate === 'failed'` and the outcome is pass, set the outcome to gaps. Add a finding of class `defect` ("the test gate failed under a passed status") and keep the warning text for the trace.
|
|
263
|
+
2. In `findings record --raw`, print the adjusted word. exec-phase's routing table then sends the phase to gaps without any prompt change.
|
|
264
|
+
3. In `verify reconcile`, treat the same case as a contradiction (exit 1). That covers the auto-advance gate in `transition.md`, which reads the file's own `status:`.
|
|
265
|
+
4. Leave `test_gate: skipped` alone. Skipped already surfaces through `not_checked` (MI-064). A missing `test_gate` stays a warning.
|
|
266
|
+
|
|
267
|
+
**Tests (at least 10)**
|
|
268
|
+
- Happy path:
|
|
269
|
+
- (1) passed + gate passed → passed.
|
|
270
|
+
- (2) gaps + gate failed → gaps.
|
|
271
|
+
- (3) passed + gate skipped → passed with `not_checked` intact.
|
|
272
|
+
- Edge cases:
|
|
273
|
+
- (4) passed + gate failed → gaps, plus one defect finding.
|
|
274
|
+
- (5) A missing `test_gate` → passed with the existing warning.
|
|
275
|
+
- (6) An unknown gate value → the existing warning, no downgrade.
|
|
276
|
+
- (7) CRLF frontmatter gives the same result.
|
|
277
|
+
- Error cases:
|
|
278
|
+
- (8) `reconcile` exits 1 on passed + gate failed and prints why.
|
|
279
|
+
- (9) A verification file with no status → `no_verdict` as today.
|
|
280
|
+
- Runtime-specific:
|
|
281
|
+
- (10) The exec-phase routing snippet, run in a sandbox, routes to gaps.
|
|
282
|
+
- (11) The native `pan-exec-waves` script's verification read takes the same path (check how it reads the status).
|
|
283
|
+
|
|
284
|
+
**Runtime matrix:** all five ✅. The CLI is shared; the change is in the engine, not in host-specific text.
|
|
285
|
+
|
|
286
|
+
### F3 — Publication pipeline (MI-025, H2, P5, L; needs the hosting decision)
|
|
287
|
+
|
|
288
|
+
**Design (option a, recommended):**
|
|
289
|
+
- A release-workflow job runs `npm run build:plugin`, then commits the result to a tracked `plugins/pan-wizard/` folder (or a `plugin` branch), tagged with the release.
|
|
290
|
+
- A root `.claude-plugin/marketplace.json` lists it with a relative source, so `claude plugin marketplace add oharms/PanWizard` works from GitHub.
|
|
291
|
+
- The Codex (`.agents/plugins/marketplace.json`) and Copilot (`.github/plugin/marketplace.json`) manifests point at the tracked Agent Plugins bundle, not the untracked `./dist/`.
|
|
292
|
+
- Submission goes to Anthropic's developer portal, then the Codex and Copilot default-marketplace processes, and Grok's marketplace, which consumes the Claude format.
|
|
293
|
+
|
|
294
|
+
**Gates:**
|
|
295
|
+
- `claude plugin validate` on the tracked folder.
|
|
296
|
+
- A harness live gate that adds the marketplace from the GitHub URL in a scratch home and lists PAN's commands.
|
|
297
|
+
- Gate 8 extended: the tracked folder matches a fresh build.
|
|
298
|
+
|
|
299
|
+
**Risks:**
|
|
300
|
+
- A tracked build doubles the diff on each release: a dedicated folder plus a release-only job keeps it out of feature PRs.
|
|
301
|
+
- Review latency at each marketplace: start with the one already shown to list peers.
|
|
302
|
+
|
|
303
|
+
**Runtime matrix:**
|
|
304
|
+
- Claude Code: plugin.
|
|
305
|
+
- Codex and Copilot: Agent Plugins bundle.
|
|
306
|
+
- Gemini and OpenCode: unchanged (npm installer).
|
|
307
|
+
- Antigravity: after MI-026.
|
|
308
|
+
|
|
309
|
+
### F4 — UAT that runs (H3, P4, L)
|
|
310
|
+
|
|
311
|
+
**Problem:**
|
|
312
|
+
- The UAT template (`templates/uat.md`) exists.
|
|
313
|
+
- The diagnose workflow (`workflows/diagnose-issues.md`) reads a UAT file and spawns `pan-debugger` per gap.
|
|
314
|
+
- The native `pan-diagnose-issues` script does the same on Claude Code.
|
|
315
|
+
- But no command writes a UAT file, so the chain never starts. The COMPARISON row is "UAT only as a template no command runs". gsd-core runs UAT inside its verify step.
|
|
316
|
+
|
|
317
|
+
**Design:**
|
|
318
|
+
1. Add a `--uat` mode to `/pan:verify-phase`, as a markdown workflow so it works on all five runtimes.
|
|
319
|
+
2. It reads the phase's success criteria from the roadmap slice and the verification report.
|
|
320
|
+
3. It walks the user through them one at a time (one question per message, the user's verdict recorded verbatim).
|
|
321
|
+
4. It writes `{phase}-uat.md` from the template, with `status: complete` or `status: gaps`, then hands any gaps to `diagnose-issues`: markdown everywhere, the native script on Claude Code.
|
|
322
|
+
5. In auto mode it never runs: a human verdict is the point. It prints the command instead.
|
|
323
|
+
|
|
324
|
+
**Gates:**
|
|
325
|
+
- A template round-trip test with `extractFrontmatter`.
|
|
326
|
+
- A workflow text test that the mode loads `diagnose-issues.md`.
|
|
327
|
+
- A tier-2 harness scenario with a scripted user (the MI-049 driver pattern) that marks one criterion failed and asserts a diagnosed `uat.md` and a gaps plan.
|
|
328
|
+
|
|
329
|
+
**Runtime matrix:** all five ✅ (markdown). The native diagnosis is Claude Code only, as today.
|
|
330
|
+
|
|
331
|
+
### F5 — Cross-AI review lane (H3 spike, P5, M; MI-044 WATCH → design)
|
|
332
|
+
|
|
333
|
+
**Problem:** gsd-core sends plans to other AIs through declared reviewer lanes, and BMAD lets an external model take a review layer. PAN's reviewers are all one model family.
|
|
334
|
+
|
|
335
|
+
**Design (spike only this quarter):**
|
|
336
|
+
- A `review.lanes` config entry names a host CLI already installed (for example `codex exec` or `gemini -p`).
|
|
337
|
+
- `/pan:review-deep` runs that lane's prompt on the diff in a read-only mode, and its findings go through the same `findings record` path.
|
|
338
|
+
|
|
339
|
+
**Spike exit:** one lane runs on this machine with a captured cost, and its findings parse into the ledger.
|
|
340
|
+
|
|
341
|
+
**Hard constraints:**
|
|
342
|
+
- No shell interpolation of the diff.
|
|
343
|
+
- A foreign CLI's cost must be readable (the MI-104 parser rule) or the lane is refused.
|
|
344
|
+
|
|
345
|
+
**Runtime matrix:** a reviewer lane runs from any host. Lanes are opt-in.
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
## Phase 5: Risk Assessment
|
|
350
|
+
|
|
351
|
+
| Risk | Impact | Likelihood | Mitigation |
|
|
352
|
+
|---|---|---|---|
|
|
353
|
+
| The v2 compaction event's fields differ from what was read in the binary | F1 ships a no-op on v2 | Medium | The live check in F1 step 4 gates the release; the handler tolerates both `system` and `context` |
|
|
354
|
+
| F2 turns a verifier's honest mistake into a stopped phase | Users see more `gaps_found` | Medium | That is the intended behaviour; the gap message names the failed gate and the fix path |
|
|
355
|
+
| Publication review rejects or delays the listing | SC-1 slips | Medium | Submit to the channel where a peer is already listed; keep the npm path working throughout |
|
|
356
|
+
| A tracked plugin build drifts from the source | Users install stale prompts | Low | Gate 8 extended to compare the tracked folder with a fresh build |
|
|
357
|
+
| Cross-host harness runs cost more than planned | H3 slips | Medium | Per-run `--max-usd` approvals; tier 0 stays free and green; cassette replay (program H2) keeps reruns free; model tiers run per release, not per commit |
|
|
358
|
+
| UAT mode stalls headless sessions | Auto runs hang | Low | `--uat` refuses auto mode and prints the command |
|
|
359
|
+
| Peers' eval labs out-pace PAN's harness | The "measured behaviour" claim weakens | Medium | H3 widens the harness to a second host; COMPARISON states dated, sourced verdicts only |
|
|
360
|
+
| OpenCode 1.x users break if PAN writes v2 shapes | Config rejected under the closed v1 schema | High if done | Recorded as a "What not to do" entry |
|
|
361
|
+
|
|
362
|
+
---
|
|
363
|
+
|
|
364
|
+
## Sources (read `2026-10-11`)
|
|
365
|
+
|
|
366
|
+
- Peers:
|
|
367
|
+
- Each project's README, docs and GitHub releases API (`published_at`): `obra/superpowers`, `open-gsd/gsd-core` (default branch `next`), `github/spec-kit`, `bmad-code-org/BMAD-METHOD` (v6 on `V6.12`, v7 on `main`), `Fission-AI/OpenSpec`.
|
|
368
|
+
- Contributor counts from the GitHub API link header.
|
|
369
|
+
- Registries:
|
|
370
|
+
- `api.npmjs.org/downloads/point/last-month/<package>` for `pan-wizard`, `bmad-method`, `@opengsd/gsd-core`, `get-shit-done-cc` and `@fission-ai/openspec`.
|
|
371
|
+
- `formulae.brew.sh` for OpenSpec.
|
|
372
|
+
- pypistats for Spec Kit (secondary).
|
|
373
|
+
- Marketplaces, raw JSON parsed:
|
|
374
|
+
- `anthropics/claude-plugins-official`
|
|
375
|
+
- `openai/plugins`
|
|
376
|
+
- `github/awesome-copilot`
|
|
377
|
+
- `github/copilot-plugins`
|
|
378
|
+
- `xai-org/plugin-marketplace`
|
|
379
|
+
- `cursor/plugins`
|
|
380
|
+
- OpenCode 2:
|
|
381
|
+
- `opencode.ai/v2/docs/{migrate-v1,build/plugins,permissions,config}`
|
|
382
|
+
- `opencode.ai/config.json`
|
|
383
|
+
- strings read (not run) from `@opencode/cli-linux-x64@2.0.26`
|
|
384
|
+
- PAN:
|
|
385
|
+
- `docs/ECOSYSTEM-REVIEW-2026-08.md` §11
|
|
386
|
+
- `docs/specs/market-ideas-ledger.md`
|
|
387
|
+
- `docs/specs/market-ideas-2026-10.md` (Addendum)
|
|
388
|
+
- `docs/COMPARISON.md`
|
|
389
|
+
- the code paths cited above.
|
|
390
|
+
- Secondary figures: stars, the Superpowers install count, and PyPI downloads. Each is marked where it appears.
|