pan-wizard 3.35.0 → 3.37.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (339) hide show
  1. package/CHANGELOG.md +3645 -0
  2. package/README.md +46 -52
  3. package/agents/pan-conductor.md +2 -3
  4. package/agents/pan-counterfactual.md +1 -2
  5. package/agents/pan-debugger.md +14 -3
  6. package/agents/pan-design-checker.md +1 -2
  7. package/agents/pan-designer.md +2 -3
  8. package/agents/pan-document_code.md +1 -2
  9. package/agents/pan-executor.md +25 -0
  10. package/agents/pan-hardener.md +4 -3
  11. package/agents/pan-integration-checker.md +1 -2
  12. package/agents/pan-knowledge.md +1 -2
  13. package/agents/pan-meta-reviewer.md +3 -3
  14. package/agents/pan-optimizer.md +4 -4
  15. package/agents/pan-phase-researcher.md +80 -475
  16. package/agents/pan-plan-checker.md +40 -3
  17. package/agents/pan-planner.md +21 -7
  18. package/agents/pan-previewer.md +3 -2
  19. package/agents/pan-project-researcher.md +72 -493
  20. package/agents/pan-release.md +1 -1
  21. package/agents/pan-reviewer.md +14 -3
  22. package/agents/pan-roadmapper.md +12 -7
  23. package/agents/pan-verifier.md +9 -4
  24. package/bin/install-lib.cjs +134 -14
  25. package/bin/install.js +59 -12
  26. package/commands/pan/army.md +1 -1
  27. package/commands/pan/autodev.md +76 -0
  28. package/commands/pan/cost.md +21 -1
  29. package/commands/pan/exec-phase.md +3 -4
  30. package/commands/pan/focus-auto.md +14 -7
  31. package/commands/pan/focus-design.md +4 -5
  32. package/commands/pan/focus-docs.md +24 -0
  33. package/commands/pan/learn.md +13 -6
  34. package/commands/pan/milestone-audit.md +8 -1
  35. package/commands/pan/milestone-done.md +10 -5
  36. package/commands/pan/milestone-new.md +2 -2
  37. package/commands/pan/new-project.md +15 -12
  38. package/commands/pan/plan-phase.md +15 -9
  39. package/commands/pan/preview.md +1 -1
  40. package/commands/pan/quick.md +1 -1
  41. package/commands/pan/review-deep.md +11 -5
  42. package/commands/pan/settings.md +6 -0
  43. package/commands/pan/verify-phase.md +6 -2
  44. package/docs/AGENTS.md +917 -0
  45. package/docs/ARCHITECTURE.md +1054 -0
  46. package/docs/CLI-REFERENCE.md +4582 -0
  47. package/docs/COMPARISON.md +116 -0
  48. package/docs/DEVELOPMENT.md +490 -0
  49. package/docs/ECOSYSTEM-REVIEW-2026-06.md +107 -0
  50. package/docs/ECOSYSTEM-REVIEW-2026-08.md +349 -0
  51. package/docs/EXAMPLES.md +584 -0
  52. package/docs/FAQ.md +254 -0
  53. package/docs/FIELD-HARVEST-2026-07.md +88 -0
  54. package/docs/FIELD-REPORT-army-2026-06.md +121 -0
  55. package/docs/HOOKS.md +638 -0
  56. package/docs/IMPROVEMENT-TODO.md +246 -0
  57. package/docs/INTERNALS.md +485 -0
  58. package/docs/MIGRATION-v2-to-v3.md +258 -0
  59. package/docs/SECURITY-HARDENING.md +230 -0
  60. package/docs/SKILLS-ENHANCEMENT-PLAN-V2.md +122 -0
  61. package/docs/SKILLS-FULL-TEXT.md +9685 -0
  62. package/docs/SKILLS-REFERENCE.md +1037 -0
  63. package/docs/TROUBLESHOOTING.md +1084 -0
  64. package/docs/USER-GUIDE.md +1437 -0
  65. package/docs/archive/SKILLS-ENHANCEMENT-PLAN.md +230 -0
  66. package/docs/audits/AUDIT-2026-08-e2e-ROADMAP.md +117 -0
  67. package/docs/audits/AUDIT-2026-08-e2e-VERIFICATION.md +693 -0
  68. package/docs/audits/AUDIT-2026-08-e2e.md +1546 -0
  69. package/docs/audits/OUTSTANDING-2026-08.md +275 -0
  70. package/docs/audits/STABILITY-TEST-2026-08.md +640 -0
  71. package/docs/branding/README-brand-draft.md +119 -0
  72. package/docs/branding/handoff/IMPLEMENTATION.md +205 -0
  73. package/docs/branding/handoff/README-brand-draft.md +119 -0
  74. package/docs/branding/handoff/image-prompts.md +46 -0
  75. package/docs/branding/image-prompts.md +46 -0
  76. package/docs/branding/nano-banana-prompts.txt +60 -0
  77. package/docs/context-monitor.md +13 -0
  78. package/docs/decisions/ADR-0001-strategic-feature-prioritization.md +83 -0
  79. package/docs/decisions/ADR-0002-copilot-cli-runtime.md +71 -0
  80. package/docs/decisions/ADR-0003-smart-execution.md +84 -0
  81. package/docs/decisions/ADR-0004-copilot-cli-interaction.md +89 -0
  82. package/docs/decisions/ADR-0005-command-naming.md +80 -0
  83. package/docs/decisions/ADR-0006-focus-commands.md +101 -0
  84. package/docs/decisions/ADR-0007-ship-v1.md +73 -0
  85. package/docs/decisions/ADR-0008-map-codebase-mermaid-togaf.md +80 -0
  86. package/docs/decisions/ADR-0009-production-deployment-checklist.md +68 -0
  87. package/docs/decisions/ADR-0010-standards-integration.md +79 -0
  88. package/docs/decisions/ADR-0011-help-commands-rewrite.md +64 -0
  89. package/docs/decisions/ADR-0012-contributors-attribution-index.md +42 -0
  90. package/docs/decisions/ADR-0013-strategic-feature-additions.md +53 -0
  91. package/docs/decisions/ADR-0014-internal-cleanup-code-quality.md +53 -0
  92. package/docs/decisions/ADR-0015-focus-auto-runner.md +296 -0
  93. package/docs/decisions/ADR-0016-enhanced-doc-sync.md +54 -0
  94. package/docs/decisions/ADR-0017-enhanced-focus-design.md +69 -0
  95. package/docs/decisions/ADR-0018-user-reality-testing.md +84 -0
  96. package/docs/decisions/ADR-0019-end-to-end-user-acceptance-testing.md +113 -0
  97. package/docs/decisions/ADR-0020-ai-drift-prevention.md +55 -0
  98. package/docs/decisions/ADR-0021-codebase-mapper-v2.md +76 -0
  99. package/docs/decisions/ADR-0022-lifecycle-completeness.md +63 -0
  100. package/docs/decisions/ADR-0023-opus-4-7-adoption.md +89 -0
  101. package/docs/decisions/ADR-0024-spec-b-v2-completion.md +137 -0
  102. package/docs/decisions/ADR-0025-behavioral-guardrails-layer.md +129 -0
  103. package/docs/decisions/ADR-0026-self-improvement-loop.md +171 -0
  104. package/docs/decisions/ADR-0027-doc-code-link-graph.md +164 -0
  105. package/docs/decisions/ADR-0028-unified-agents-skills-target.md +46 -0
  106. package/docs/decisions/ADR-0029-experiment-evaluator.md +76 -0
  107. package/docs/decisions/ADR-0030-planning-concurrency.md +37 -0
  108. package/docs/decisions/ADR-0031-focus-auto-backlog-source.md +47 -0
  109. package/docs/decisions/ADR-0032-squad-model.md +64 -0
  110. package/docs/decisions/ADR-0033-army-campaign.md +40 -0
  111. package/docs/decisions/ADR-0034-scheduled-campaigns.md +57 -0
  112. package/docs/decisions/ADR-0035-army-hud-dashboard.md +40 -0
  113. package/docs/decisions/ADR-0036-agentic-memory-architecture.md +106 -0
  114. package/docs/decisions/ADR-0037-pin-security-agents-to-opus.md +51 -0
  115. package/docs/decisions/ADR-0038-skill-aligned-decomposition.md +41 -0
  116. package/docs/decisions/ADR-0039-worktree-as-project-variant.md +37 -0
  117. package/docs/decisions/ADR-0040-memory-injection-defense.md +55 -0
  118. package/docs/decisions/ADR-0041-mcp-dual-era-bridge.md +47 -0
  119. package/docs/decisions/ADR-0042-design-parity-both-flows.md +45 -0
  120. package/docs/decisions/ADR-0043-addressable-planning-roots.md +56 -0
  121. package/docs/decisions/ADR-0044-bound-the-cached-context.md +104 -0
  122. package/docs/decisions/ADR-0045-agent-plugins-bundle.md +76 -0
  123. package/docs/decisions/ADR-0046-subagent-frontmatter-decisions.md +39 -0
  124. package/docs/decisions/ADR-0047-pan-harness.md +76 -0
  125. package/docs/decisions/ADR-0048-one-core-per-project.md +43 -0
  126. package/docs/decisions/ADR-0049-machine-verdicts-and-findings-ledger.md +116 -0
  127. package/docs/specs/ai_drift_prevention_featureai.md +286 -0
  128. package/docs/specs/codebase_mapper_v2_featureai.md +930 -0
  129. package/docs/specs/command_naming_optimization_featureai.md +930 -0
  130. package/docs/specs/comp_analysis_markdown_v1.md +149 -0
  131. package/docs/specs/continuous_autonomous_execution_featureai.md +304 -0
  132. package/docs/specs/contributors_attribution_index_featureai.md +476 -0
  133. package/docs/specs/copilot_cli_interaction_featureai.md +868 -0
  134. package/docs/specs/copilot_cli_runtime_featureai.md +647 -0
  135. package/docs/specs/custom-skills-assessment.md +461 -0
  136. package/docs/specs/deployment_audit_findings_featureai.md +243 -0
  137. package/docs/specs/doc_code_link_graph_featureai.md +253 -0
  138. package/docs/specs/documentation_alignment_cleanup_featureai.md +69 -0
  139. package/docs/specs/e2e_visual_testing_featureai.md +615 -0
  140. package/docs/specs/end_to_end_user_acceptance_testing_featureai.md +810 -0
  141. package/docs/specs/evidence-loop-plan.md +76 -0
  142. package/docs/specs/evidence_loop_featureai.md +497 -0
  143. package/docs/specs/focus-auto-optimize-category.md +449 -0
  144. package/docs/specs/focus-auto-optimize-superplan.md +188 -0
  145. package/docs/specs/focus_auto_runner_featureai.md +1102 -0
  146. package/docs/specs/focus_design_v2_featureai.md +197 -0
  147. package/docs/specs/focus_sync_v2_featureai.md +202 -0
  148. package/docs/specs/git_integration_overhaul_featureai.md +326 -0
  149. package/docs/specs/glm_zai_provider_featureai.md +200 -0
  150. package/docs/specs/googlecli_adoption_featureai.md +661 -0
  151. package/docs/specs/help_commands_rewrite_featureai.md +860 -0
  152. package/docs/specs/industry_analysis_strategic_additions_featureai.md +231 -0
  153. package/docs/specs/industry_comparison_and_feature_gaps_featureai.md +674 -0
  154. package/docs/specs/internal_cleanup_code_quality_featureai.md +486 -0
  155. package/docs/specs/investigations/README.md +31 -0
  156. package/docs/specs/investigations/digests/2026-09-28.md +47 -0
  157. package/docs/specs/investigations/digests/2026-10-08.md +62 -0
  158. package/docs/specs/investigations/digests/2026-10-10.md +61 -0
  159. package/docs/specs/investigations/features/debugger-red-loop-first.md +55 -0
  160. package/docs/specs/investigations/features/harness-cross-host-behaviour.md +93 -0
  161. package/docs/specs/investigations/features/optimize-apply-regression-gate.md +66 -0
  162. package/docs/specs/investigations/features/reviewer-smell-baseline.md +57 -0
  163. package/docs/specs/investigations/features/trace-failure-capture.md +69 -0
  164. package/docs/specs/lifecycle_completeness_featureai.md +754 -0
  165. package/docs/specs/local-first-ci.md +42 -0
  166. package/docs/specs/map_codebase_mermaid_togaf_featureai.md +805 -0
  167. package/docs/specs/market-delta-2026-09-superplan.md +310 -0
  168. package/docs/specs/market-ideas-2026-09.md +131 -0
  169. package/docs/specs/market-ideas-2026-10.md +428 -0
  170. package/docs/specs/market-ideas-ledger.md +177 -0
  171. package/docs/specs/memory-optimization-2026-10.md +285 -0
  172. package/docs/specs/multi-track-planning-roots.md +143 -0
  173. package/docs/specs/multi_model_routing_featureai.md +687 -0
  174. package/docs/specs/multi_model_routing_superplan.md +197 -0
  175. package/docs/specs/opus_47_existing_enhancements_featureai.md +247 -0
  176. package/docs/specs/opus_47_extended_features_featureai.md +363 -0
  177. package/docs/specs/opus_47_extended_features_v2_featureai.md +397 -0
  178. package/docs/specs/pan_focus_commands_featureai.md +2336 -0
  179. package/docs/specs/pan_zcode_mcp_bridge_featureai.md +120 -0
  180. package/docs/specs/product-excellence-superplan.md +315 -0
  181. package/docs/specs/product_excellence_program_2026_q4_featureai.md +365 -0
  182. package/docs/specs/production_deployment_readiness_featureai.md +433 -0
  183. package/docs/specs/production_readiness_ship_v1_featureai.md +698 -0
  184. package/docs/specs/real_test_enforcement_featureai.md +213 -0
  185. package/docs/specs/reality-check-2026-09-superplan.md +362 -0
  186. package/docs/specs/right-sized-ceremony.md +74 -0
  187. package/docs/specs/roadmap_and_competition_2026_q4_featureai.md +390 -0
  188. package/docs/specs/self_improvement_loop_featureai.md +875 -0
  189. package/docs/specs/skill-aligned-decomposition.md +155 -0
  190. package/docs/specs/standards_integration_featureai.md +833 -0
  191. package/docs/specs/testing-system-redesign-2026-09.md +267 -0
  192. package/docs/specs/user_reality_testing_system_featureai.md +788 -0
  193. package/docs/specs/watch-triggers.json +22 -0
  194. package/docs/specs/workflow-integration-redesign.md +742 -0
  195. package/docs/specs/workflow_state_consistency_featureai.md +183 -0
  196. package/hooks/dist/pan-context-monitor.js +9 -2
  197. package/hooks/dist/pan-cost-logger.js +5 -0
  198. package/hooks/dist/pan-guard.js +28 -5
  199. package/hooks/dist/pan-stop-guard.js +9 -9
  200. package/hooks/dist/pan-trace-logger.js +5 -0
  201. package/package.json +10 -2
  202. package/pan-wizard-core/bin/lib/agents-md.cjs +57 -8
  203. package/pan-wizard-core/bin/lib/autodev-report.cjs +180 -0
  204. package/pan-wizard-core/bin/lib/autodev.cjs +642 -0
  205. package/pan-wizard-core/bin/lib/ci.cjs +143 -0
  206. package/pan-wizard-core/bin/lib/commands.cjs +1 -1
  207. package/pan-wizard-core/bin/lib/config.cjs +41 -5
  208. package/pan-wizard-core/bin/lib/constants.cjs +27 -1
  209. package/pan-wizard-core/bin/lib/core.cjs +31 -14
  210. package/pan-wizard-core/bin/lib/cost.cjs +160 -4
  211. package/pan-wizard-core/bin/lib/cross-family.cjs +164 -0
  212. package/pan-wizard-core/bin/lib/debug-cases.cjs +104 -0
  213. package/pan-wizard-core/bin/lib/doc-lint.cjs +5 -3
  214. package/pan-wizard-core/bin/lib/failure-class.cjs +81 -0
  215. package/pan-wizard-core/bin/lib/findings.cjs +148 -9
  216. package/pan-wizard-core/bin/lib/focus.cjs +1 -1
  217. package/pan-wizard-core/bin/lib/git.cjs +1 -1
  218. package/pan-wizard-core/bin/lib/hosts.cjs +88 -0
  219. package/pan-wizard-core/bin/lib/init.cjs +7 -2
  220. package/pan-wizard-core/bin/lib/knowledge.cjs +2 -0
  221. package/pan-wizard-core/bin/lib/merge.cjs +50 -0
  222. package/pan-wizard-core/bin/lib/mutate.cjs +268 -0
  223. package/pan-wizard-core/bin/lib/next-action.cjs +112 -0
  224. package/pan-wizard-core/bin/lib/optimize.cjs +2 -2
  225. package/pan-wizard-core/bin/lib/phase-gate.cjs +15 -9
  226. package/pan-wizard-core/bin/lib/phase-remove.cjs +91 -11
  227. package/pan-wizard-core/bin/lib/phase.cjs +85 -13
  228. package/pan-wizard-core/bin/lib/plan-lint.cjs +310 -0
  229. package/pan-wizard-core/bin/lib/preview.cjs +26 -4
  230. package/pan-wizard-core/bin/lib/requirements-lint.cjs +130 -0
  231. package/pan-wizard-core/bin/lib/research-lint.cjs +268 -0
  232. package/pan-wizard-core/bin/lib/review-deep.cjs +14 -22
  233. package/pan-wizard-core/bin/lib/review-vote.cjs +105 -0
  234. package/pan-wizard-core/bin/lib/roadmap.cjs +1 -0
  235. package/pan-wizard-core/bin/lib/runner.cjs +2 -16
  236. package/pan-wizard-core/bin/lib/sizing.cjs +356 -0
  237. package/pan-wizard-core/bin/lib/squads.cjs +1 -1
  238. package/pan-wizard-core/bin/lib/state-write.cjs +247 -0
  239. package/pan-wizard-core/bin/lib/test-gate.cjs +433 -0
  240. package/pan-wizard-core/bin/lib/uat.cjs +149 -0
  241. package/pan-wizard-core/bin/lib/usage.cjs +104 -0
  242. package/pan-wizard-core/bin/lib/verdict.cjs +26 -3
  243. package/pan-wizard-core/bin/lib/verify-plan-set.cjs +30 -1
  244. package/pan-wizard-core/bin/lib/verify-preflight.cjs +2 -2
  245. package/pan-wizard-core/bin/lib/verify-red.cjs +174 -0
  246. package/pan-wizard-core/bin/lib/verify-retro.cjs +3 -2
  247. package/pan-wizard-core/bin/lib/verify-scope.cjs +143 -2
  248. package/pan-wizard-core/bin/lib/verify.cjs +86 -31
  249. package/pan-wizard-core/bin/lib/worktree.cjs +70 -0
  250. package/pan-wizard-core/bin/pan-tools.cjs +257 -28
  251. package/pan-wizard-core/learnings/universal/autonomous-loop.md +2 -2
  252. package/pan-wizard-core/mcp/orchestrator.cjs +4 -1
  253. package/pan-wizard-core/references/continuation-format.md +3 -3
  254. package/pan-wizard-core/references/design-methodology.md +1 -1
  255. package/pan-wizard-core/references/guardrails.md +1 -1
  256. package/pan-wizard-core/references/handoff-decisions.md +1 -1
  257. package/pan-wizard-core/references/model-profiles.md +2 -3
  258. package/pan-wizard-core/references/questioning.md +24 -0
  259. package/pan-wizard-core/references/ui-brand.md +4 -6
  260. package/pan-wizard-core/references/verdict-contract.md +2 -1
  261. package/pan-wizard-core/templates/config.json +10 -2
  262. package/pan-wizard-core/templates/debug.md +1 -0
  263. package/pan-wizard-core/templates/design.md +5 -4
  264. package/pan-wizard-core/templates/discovery.md +2 -2
  265. package/pan-wizard-core/templates/phase-prompt.md +7 -0
  266. package/pan-wizard-core/templates/requirements.md +9 -0
  267. package/pan-wizard-core/templates/research.md +10 -0
  268. package/pan-wizard-core/templates/summary-complex.md +5 -1
  269. package/pan-wizard-core/templates/summary-minimal.md +4 -0
  270. package/pan-wizard-core/templates/summary-standard.md +5 -1
  271. package/pan-wizard-core/templates/summary.md +8 -1
  272. package/pan-wizard-core/workflows/add-phase.md +4 -6
  273. package/pan-wizard-core/workflows/assumptions.md +19 -1
  274. package/pan-wizard-core/workflows/diagnose-issues.md +2 -0
  275. package/pan-wizard-core/workflows/discuss-phase.md +4 -2
  276. package/pan-wizard-core/workflows/exec-lean.md +69 -0
  277. package/pan-wizard-core/workflows/exec-phase.md +79 -15
  278. package/pan-wizard-core/workflows/execute-plan.md +3 -1
  279. package/{commands/pan → pan-wizard-core/workflows}/focus-doc-audit.md +13 -29
  280. package/{commands/pan → pan-wizard-core/workflows}/focus-drift-walking.md +3 -18
  281. package/{commands/pan → pan-wizard-core/workflows}/focus-exec.md +8 -23
  282. package/{commands/pan → pan-wizard-core/workflows}/focus-plan.md +6 -20
  283. package/{commands/pan → pan-wizard-core/workflows}/focus-scan.md +3 -16
  284. package/{commands/pan → pan-wizard-core/workflows}/focus-sync.md +7 -21
  285. package/pan-wizard-core/workflows/health.md +6 -0
  286. package/pan-wizard-core/workflows/help.md +114 -77
  287. package/pan-wizard-core/workflows/insert-phase.md +4 -6
  288. package/pan-wizard-core/workflows/learn.md +3 -3
  289. package/pan-wizard-core/workflows/milestone-audit.md +2 -2
  290. package/pan-wizard-core/workflows/milestone-new.md +38 -95
  291. package/pan-wizard-core/workflows/new-project.md +96 -258
  292. package/pan-wizard-core/workflows/optimize.md +107 -5
  293. package/pan-wizard-core/workflows/phase-tests.md +10 -0
  294. package/pan-wizard-core/workflows/plan-phase.md +106 -45
  295. package/pan-wizard-core/workflows/profile.md +50 -0
  296. package/pan-wizard-core/workflows/progress.md +10 -0
  297. package/pan-wizard-core/workflows/quick.md +34 -43
  298. package/pan-wizard-core/workflows/remove-phase.md +11 -1
  299. package/pan-wizard-core/workflows/research-phase.md +118 -40
  300. package/pan-wizard-core/workflows/resume-project.md +18 -2
  301. package/pan-wizard-core/workflows/settings.md +33 -19
  302. package/pan-wizard-core/workflows/todo-check.md +6 -2
  303. package/pan-wizard-core/workflows/transition.md +22 -4
  304. package/pan-wizard-core/workflows/uat-walkthrough.md +90 -0
  305. package/pan-wizard-core/workflows/update.md +1 -1
  306. package/pan-wizard-core/workflows/verify-phase.md +39 -46
  307. package/scripts/build-agent-plugin.js +35 -15
  308. package/scripts/build-help.cjs +106 -0
  309. package/scripts/build-plugin.js +10 -1
  310. package/scripts/deprecate-old-versions.js +70 -3
  311. package/scripts/generate-skills-docs.py +1 -1
  312. package/scripts/git-hooks/pre-push +27 -0
  313. package/scripts/install-git-hooks.js +7 -5
  314. package/scripts/install-pre-push.cjs +38 -0
  315. package/scripts/land.cjs +151 -0
  316. package/scripts/local-ci.cjs +501 -0
  317. package/scripts/local-schedule.cjs +277 -0
  318. package/scripts/mutation-probe.cjs +7 -80
  319. package/scripts/prompt-lint.cjs +32 -0
  320. package/scripts/prompt-rules.cjs +156 -0
  321. package/scripts/run-tests.cjs +7 -1
  322. package/scripts/test-quality-lint.cjs +3 -3
  323. package/scripts/test-surface.cjs +175 -6
  324. package/agents/pan-research-synthesizer.md +0 -246
  325. package/commands/pan/audit-deployment.md +0 -386
  326. package/commands/pan/dashboard.md +0 -22
  327. package/commands/pan/discord.md +0 -19
  328. package/commands/pan/mcp-bridge.md +0 -143
  329. package/commands/pan/milestone-cleanup.md +0 -19
  330. package/commands/pan/milestone-gaps.md +0 -35
  331. package/commands/pan/optimize.md +0 -115
  332. package/commands/pan/profile.md +0 -76
  333. package/commands/pan/research-phase.md +0 -189
  334. package/pan-wizard-core/bin/lib/bridge.cjs +0 -269
  335. package/pan-wizard-core/templates/research-project/architecture.md +0 -204
  336. package/pan-wizard-core/templates/research-project/features.md +0 -147
  337. package/pan-wizard-core/templates/research-project/pitfalls.md +0 -200
  338. package/pan-wizard-core/templates/research-project/stack.md +0 -120
  339. package/pan-wizard-core/templates/research-project/summary.md +0 -170
@@ -0,0 +1,875 @@
1
+ # featureAI: Cross-Project Self-Improvement Loop — Idea → External Build → Learn → Promote → Ship
2
+
3
+ > **Generated**: 2026-04-27
4
+ > **Mode**: `--full` (all phases)
5
+ > **Feature**: `/pan:experiment` family — autonomous external builds drive PAN's own evolution
6
+ > **Target version**: v3.7.0 (XL feature, ships across 3-4 waves)
7
+ > **Predecessors**: focus_auto_runner_featureai.md (autonomous loop), googlecli_adoption_featureai.md (behavioral guardrails layer), Session 33 Playwright VSCode integration patterns
8
+ > **Status**: design — awaiting acceptance
9
+
10
+ ---
11
+
12
+ ## Phase 0: Problem Framing & Demand
13
+
14
+ ### 0.1 Problem Statement
15
+
16
+ PAN Wizard ships v3.5+ with a **self-learning system inside a project** — `pan-trace-logger` captures every agent spawn, `/pan:learn` analyzes the session via `pan-optimizer`, `/pan:optimize apply` writes back to that project's `.planning/memory/`. The loop is closed *for one project at a time*.
17
+
18
+ The PAN tool itself does not learn from this. Each PAN release ships frozen behavioral rules (`references/guardrails.md`, workflow text, agent instructions). Real-world session data — the gold mine of "what actually goes wrong when AI agents drive PAN" — stays trapped in user projects' `.planning/optimization/` directories. We dogfood PAN on PAN itself (`d:\PanWizard\.planning/`), but those learnings are **per-session memory**, not part of the shipped product. The next release of PAN does not know what the previous release's sessions discovered.
19
+
20
+ This matters NOW because:
21
+
22
+ - v3.5 just shipped the trace + learn + optimize trio. The pieces exist. What's missing is the **promote step** that lifts findings from one session's memory into shipped artifacts.
23
+ - v3.6 (just shipped) added `references/guardrails.md` — a human-authored behavioral file consumed by every workflow. This proves the *delivery channel* for shipped behavioral content. We need only build the *production channel* (AI-derived patterns that flow into shipped files).
24
+ - The MEMORY note "Cross-Session Learning (PATTERNS.md auto-capture)" was deferred for exactly this reason: needs a design spec. This is that spec.
25
+ - Real-world AI coding tooling (Cursor Background Agents, Aider's MCP integrations, Claude Code's Skills) is converging on session telemetry as a product feedback loop. PAN's existing trace infrastructure is unique among workflow tools; not closing the meta-loop wastes the lead.
26
+
27
+ **The cost of not building this:** PAN's quality plateau is gated by what its maintainers can hand-author into `references/`, `workflows/`, and agent instructions. Every session's lessons stay local. PAN's improvement velocity is bounded by maintainer attention, not by accumulated session evidence.
28
+
29
+ ### 0.2 Demand Evidence
30
+
31
+ | Evidence | Source | Finding |
32
+ |----------|--------|---------|
33
+ | **User-stated explicit request** | This conversation | "design into pan a system whereby we build and then call learn and then we bring the file back to optimise PAN" |
34
+ | **MEMORY note** | `MEMORY.md` "Known Deferred Items" | "Cross-Session Learning (PATTERNS.md auto-capture): L effort, needs design spec" |
35
+ | **Existing infrastructure** | `optimize.cjs`, `pan-optimizer.md`, `pan-trace-logger.js`, `/pan:learn`, `/pan:optimize` | Trace + analyze + apply already work intra-project. Promote + ship are the missing ~30% |
36
+ | **Existing delivery channel** | `references/guardrails.md` shipped in v3.6.0 to all 5 runtimes | Proves AI-readable shipped reference docs work; same path serves AI-derived patterns |
37
+ | **Existing process automation** | `tests/scenarios/*.test.cjs` already spawn fresh installs in temp dirs via `createScenarioRunner` | Cross-process orchestration is a known-cost operation in this codebase |
38
+ | **Existing VSCode automation** | `tests/e2e/playwright.config.mjs`, `tests/e2e/vscode-helpers.mjs` (Session 33) | Precedent for driving VSCode programmatically — though the simpler `claude` CLI subprocess path covers the autonomous use case |
39
+ | **Industry convergence** | Cursor Background Agents, Aider --watch-files, Copilot Coding Agent | Autonomous external project execution is a 2026 standard; PAN's gap is closing the *learning* side, not just the execution side |
40
+
41
+ **Demand status:** STRONG — direct user request + deferred-with-design-spec marker + the prerequisite infrastructure is all in place.
42
+
43
+ ### 0.3 Scope Definition
44
+
45
+ | In Scope | Out of Scope (and why) |
46
+ |----------|------------------------|
47
+ | Idea-input format (`idea.md` schema) PAN can consume as a Phase-0 spec | Multi-stakeholder idea collection workflow — single user, single idea, single experiment |
48
+ | Experiment scaffolding: create isolated project folder outside source repo with `.planning/` and idea doc | Experiments inside the source repo — installer guard already prohibits self-install; same logic applies |
49
+ | External AI agent invocation via subprocess (`claude -p`, `codex`, `gemini`, `opencode`) | VSCode UI automation as the *primary* path — adds Playwright dependency to the runtime; reserve as advanced opt-in |
50
+ | Stream observation: tail external project's `state.md`, `current-agent-id.txt`, summary files; surface progress in PAN session | Bidirectional control — external agent runs autonomously; PAN observes but doesn't intervene mid-build (cleaner failure semantics) |
51
+ | Telemetry harvest: copy `.planning/optimization/`, `agent-history.json`, traces back to PAN source as `experiments/<id>/` | Cross-experiment aggregation — one experiment, one harvest; aggregation is a separate v3.8+ feature |
52
+ | `/pan:learn --experiment <id>` runs `pan-optimizer` against harvested data | Re-running pan-optimizer continuously during the experiment — single post-build analysis is sufficient |
53
+ | `pan-tools learn promote --pattern <id> --scope <universal\|internal>` writes findings to `pan-wizard-core/learnings/` | Auto-promote (no human review) — risky; manual review step is the v3.7 design; auto-promote is a v3.8 ramp |
54
+ | Two-tier output: `learnings/universal/` (shipped to all installs) vs `learnings/internal/` (PAN-source-only) | Three or more tiers — universal/internal binary is sufficient |
55
+ | Workflows reference `learnings/universal/*.md` like they reference `references/*.md` | New runtime adapter logic — `learnings/` ships via the same glob that ships `references/` |
56
+ | Budget cap + timeout + circuit breaker on external runs (inherits focus-auto pattern) | Cost-tracking integration — `cost.cjs` already records spend; cross-experiment cost analysis is v3.8 |
57
+ | Tests: scenario coverage for the experiment lifecycle (init → run → harvest → promote) | Integration test that drives a real Claude Code subprocess — too slow for CI; mocked subprocess is sufficient |
58
+
59
+ ### 0.4 Success Criteria (Measurable)
60
+
61
+ ```
62
+ SC-1: /pan:experiment new <slug> --idea <file> creates an isolated experiment folder OUTSIDE the source repo with .planning/ and idea.md
63
+ SC-2: /pan:experiment run <id> spawns an external AI agent (Claude/Codex/Gemini/OpenCode) pointed at the experiment folder
64
+ SC-3: Experiment runner streams progress back to PAN session (state.md changes, summary.md appearances)
65
+ SC-4: Loop stops on ANY of: build success (final summary written), budget exhausted, timeout, circuit breaker (3 consecutive task failures)
66
+ SC-5: /pan:experiment harvest <id> copies .planning/optimization/ + agent-history.json + state.md from experiment folder back to d:\PanWizard\experiments\<id>\
67
+ SC-6: /pan:learn --experiment <id> runs pan-optimizer over harvested data, writes report to experiments/<id>/learnings/report-*.md
68
+ SC-7: pan-tools learn promote --pattern <id> --scope universal extracts a finding to pan-wizard-core/learnings/universal/<topic>.md
69
+ SC-8: pan-tools learn promote --pattern <id> --scope internal extracts a finding to pan-wizard-core/learnings/internal/<topic>.md (NOT shipped to user installs)
70
+ SC-9: Workflows reference learnings/universal/*.md the same way they reference references/*.md (already proven in v3.6.0)
71
+ SC-10: Installer ships pan-wizard-core/learnings/universal/ to all 5 runtimes; learnings/internal/ stays in source only
72
+ SC-11: Tests: 30+ new tests across experiment.test.cjs, learn-promote.test.cjs, scenarios/learnings-installed.test.cjs
73
+ SC-12: Zero regressions in existing 2588 tests
74
+ SC-13: A complete loop (idea.md → experiment run → harvest → learn → promote → re-install verifies pattern shipped) executes end-to-end in < 10 minutes for a trivial idea
75
+ SC-14: Spec works regardless of which runtime drives the external project (Claude/Codex/Gemini/OpenCode); GitHub Copilot CLI is opt-in (no headless mode)
76
+ ```
77
+
78
+ ### 0.5 User Stories
79
+
80
+ ```
81
+ As a PAN maintainer with an idea for a small CLI tool,
82
+ I want to /pan:experiment new my-idea --idea my-idea.md, run it externally,
83
+ have it produce a real working project AND telemetry on every PAN-driven decision,
84
+ and then /pan:learn promote the most-cited findings into PAN's shipped behavioral docs,
85
+ so that the next PAN release literally got smarter from this build,
86
+ instead of having to manually transcribe lessons from .planning/optimization/ into references/guardrails.md.
87
+
88
+ As a PAN maintainer reviewing a fresh experiment harvest,
89
+ I want a clear universal-vs-internal scope decision per finding,
90
+ so that PAN-specific quirks (e.g., "always commit individually" because of the source repo's hooks)
91
+ don't pollute generalizable patterns shipped to user installs,
92
+ instead of relying on judgment under deadline.
93
+
94
+ As a PAN maintainer running 5 experiments overnight on different ideas,
95
+ I want each experiment isolated in its own folder, with its own budget cap,
96
+ and a single command (/pan:experiment harvest --all) to bring telemetry back the next morning,
97
+ so that overnight automation is trustworthy,
98
+ instead of waking up to a runaway external process or merged-state corruption.
99
+
100
+ As a PAN user (downstream consumer),
101
+ I want PAN's shipped behavioral docs to evolve based on actual usage signals,
102
+ so that recurring AI failure modes get codified as guardrails before they bite me,
103
+ instead of running into the same shortcut shortcut my teammates already encountered last quarter.
104
+ ```
105
+
106
+ ---
107
+
108
+ ## Phase 1: Internal Reconnaissance
109
+
110
+ ### 1.1 Existing Pieces (already shipped)
111
+
112
+ The 70% that exists:
113
+
114
+ | Capability | Module / file | Notes |
115
+ |------------|---------------|-------|
116
+ | Per-session trace logging | `hooks/pan-trace-logger.js` (5th hook, SubagentStop) | Auto-creates day-scoped sessions, logs `decision:agent_completion` events |
117
+ | Trace storage | `pan-wizard-core/bin/lib/optimize.cjs` `getOptimizeDir()` | `.planning/optimization/traces/sess_<id>/` |
118
+ | Pattern analysis | `pan-wizard-core/bin/lib/optimize.cjs` `analyzeEvents()` | Frequency maps, agent stats, error rates, redundancy detection |
119
+ | Report generation | `pan-wizard-core/bin/lib/optimize.cjs` `generateLocalReport()` + `pan-optimizer` agent | Structured findings with `## Auto-Apply Actions` JSON block |
120
+ | Apply reports | `pan-wizard-core/bin/lib/optimize.cjs` `applyReportRecommendations()` | Writes to `.planning/memory/`, project config |
121
+ | `/pan:learn` command | `commands/pan/learn.md` + `pan-wizard-core/workflows/learn.md` | Invokes pan-optimizer, shows score |
122
+ | `/pan:optimize` command | `commands/pan/optimize.md` + `optimize.cjs` cmd functions | trace init/end/status/list, learn, apply, list, stats |
123
+ | Behavioral content delivery | `pan-wizard-core/references/*.md` (14 files), workflow cross-references | Proven channel for AI-readable shipped content (v3.6.0) |
124
+ | External-process orchestration | `tests/helpers.cjs` `createScenarioRunner()` | Spawns fresh installer in temp dir, returns runner with `run()` + `cleanup()` |
125
+ | VSCode-specific automation | `tests/e2e/vscode-helpers.mjs` (Session 33) | Playwright + @vscode/test-electron — exists but unnecessary for autonomous CLI runs |
126
+ | Budget + circuit-breaker pattern | `pan-wizard-core/bin/lib/focus.cjs` (focus-auto) | Triple safety: per-cycle budget + cumulative cap + max cycles + zero-progress halt |
127
+
128
+ ### 1.2 Missing Pieces (the 30% to build)
129
+
130
+ | Gap | Module / artifact | Effort |
131
+ |-----|------------------|--------|
132
+ | Experiment lifecycle data | `pan-wizard-core/bin/lib/experiment.cjs` (28th core module) | M (4 pts) |
133
+ | Experiment commands | `commands/pan/experiment.md` (52nd command) | S (2 pts) |
134
+ | External agent runner | `pan-wizard-core/bin/lib/runner.cjs` (29th core module) — subprocess invocation, output streaming | M (4 pts) |
135
+ | Idea-doc schema | `pan-wizard-core/templates/idea.md` (27th template) | XS (1 pt) |
136
+ | Harvest logic | `experiment.cjs` `harvestExperiment()` | S (2 pts) |
137
+ | `pan-experiment-runner` agent | `agents/pan-experiment-runner.md` (21st agent) — drives the external instance | M (4 pts) |
138
+ | Promote logic | `pan-wizard-core/bin/lib/optimize.cjs` `promotePattern()` | M (4 pts) |
139
+ | Two-tier learnings dir | `pan-wizard-core/learnings/universal/` and `pan-wizard-core/learnings/internal/` | XS (1 pt) |
140
+ | Installer ships `learnings/universal/` only | `bin/install.js` (verify glob; likely already covers any new top-level dir under pan-wizard-core/) | XS (1 pt) |
141
+ | Workflow re-references | exec-phase, plan-phase, focus-exec workflows reference `learnings/universal/` | S (2 pts) |
142
+ | Tests | `experiment.test.cjs`, `runner.test.cjs`, `learn-promote.test.cjs`, `scenarios/learnings-installed.test.cjs` | M-L (10-15 pts) |
143
+ | Documentation | USER-GUIDE, ARCHITECTURE updates + ADR-0026 | S (2 pts) |
144
+
145
+ **Total estimated effort:** ~30-40 pts. XL feature; ships across 3-4 sessions/waves.
146
+
147
+ ### 1.3 Architectural Constraints
148
+
149
+ | Constraint | Source | Impact |
150
+ |------------|--------|--------|
151
+ | Source repo never receives a PAN install | CLAUDE.md "Self-Protection Gate" | Experiments MUST live outside `d:\PanWizard\` |
152
+ | Manifest tracks every shipped file | Existing v3.5+ behavior | New `learnings/universal/*.md` files appear in manifest checksum |
153
+ | 5-runtime parity is mandatory | CLAUDE.md "5 Target Runtimes" | Shipped learnings install identically to all 5 runtime dirs |
154
+ | Zero runtime deps | Existing project rule | Subprocess invocation uses `child_process.spawn` (Node built-in) — no new packages |
155
+ | Behavioral content is markdown + frontmatter | `references/`, `workflows/`, `agents/` patterns | `learnings/universal/*.md` follows the same shape |
156
+
157
+ ### 1.4 Cross-Runtime Compatibility
158
+
159
+ | Runtime | External invocation | Headless support | Notes |
160
+ |---------|---------------------|-------------------|-------|
161
+ | Claude Code | `claude -p "<prompt>"` or via Skills/MCP | ✅ headless via `-p` flag | Primary target |
162
+ | Codex CLI | `codex exec "<prompt>"` (or similar) | ✅ | Verify exact flag in Codex docs at implementation time |
163
+ | Gemini CLI | `gemini -p "<prompt>"` | ✅ | Verify flag |
164
+ | OpenCode | `opencode "<prompt>"` | ✅ | Verify flag |
165
+ | GitHub Copilot CLI | (no general-purpose subprocess prompt mode known) | ❌ | Opt-out for now; document limitation |
166
+
167
+ The runner module abstracts these via a per-runtime adapter pattern (similar to existing installer adapters). Configuration: `pan-tools experiment run <id> --runtime claude` (defaults to the runtime PAN was installed for).
168
+
169
+ ---
170
+
171
+ ## Phase 2: Source Pattern Analysis
172
+
173
+ ### 2.1 Patterns Adopted
174
+
175
+ **From `focus.cjs` (focus-auto runner — Session 26):**
176
+ - Triple safety net: per-cycle budget + cumulative budget + max cycles + zero-progress circuit breaker
177
+ - Run state in `.planning/focus/auto-run.json` — cross-session resumability via `--continue`
178
+ - `determineStopReason()` enumerated stop reasons returned in JSON
179
+
180
+ The experiment runner inherits all of these. State file: `.planning/experiments/<id>/run-state.json`.
181
+
182
+ **From `optimize.cjs` (circular optimization loop — Session 31):**
183
+ - Trace session lifecycle (`initTraceSession`, `logTraceEvent`, `endTraceSession`)
184
+ - Local analysis (`analyzeEvents`) producing structured findings
185
+ - Auto-apply JSON block at the end of optimizer reports
186
+ - `applyReportRecommendations()` for surgical writes
187
+
188
+ The promote step extends the apply pattern: instead of writing to `.planning/memory/` (intra-project), it writes to `pan-wizard-core/learnings/{universal,internal}/<topic>.md` (source repo, shippable on next release).
189
+
190
+ **From `tests/helpers.cjs` (`createScenarioRunner` — Session 18):**
191
+ - `child_process.execSync` invocation pattern
192
+ - Temp dir lifecycle (`mkdtempSync` + `rmSync` cleanup)
193
+ - Per-runtime adapter via `RUNTIME_DIR` map
194
+
195
+ The runner module generalizes this from "install + run command" to "scaffold + drive autonomous build."
196
+
197
+ **From v3.6.0 (`references/guardrails.md` shipped to 5 runtimes):**
198
+ - Single-file behavioral content shipped via `references/` directory
199
+ - Workflows reference by relative path
200
+ - All 5 runtimes get the file through existing installer glob
201
+
202
+ The `learnings/universal/` directory mirrors this pattern exactly — same delivery channel, AI-derived rather than human-authored content.
203
+
204
+ ### 2.2 Patterns Explicitly NOT Adopted
205
+
206
+ - **Auto-promote without human review.** Risky in v3.7. The promote step is gated on a human pressing the trigger (`pan-tools learn promote --pattern <id>`). Auto-promote ramps in v3.8+ once we have N=10+ successful manual promotes to learn the filter.
207
+ - **VSCode UI automation as primary path.** Playwright + @vscode/test-electron infrastructure exists (Session 33), but using it for autonomous external runs adds: (a) GUI dependency, (b) headless display setup, (c) cross-platform display drivers. CLI subprocess is sufficient for the autonomous case. VSCode UI automation stays for testing purposes.
208
+ - **Bidirectional control mid-run.** External agent runs autonomously; PAN observes via file-tail, doesn't inject prompts. Cleaner failure semantics — if the external agent gets stuck, the timeout/circuit-breaker fires; PAN doesn't try to "save" it. Rationale: every intervention path is a new bug surface.
209
+ - **Multi-experiment aggregation.** v3.7 ships single-experiment promote. Aggregation across N experiments is v3.8 — needs more design (which findings are universal vs idiosyncratic to N specific experiments?).
210
+
211
+ ---
212
+
213
+ ## Phase 3: Design
214
+
215
+ ### 3.1 Architecture Overview
216
+
217
+ ```
218
+ PAN source repo (d:\PanWizard\)
219
+ ──────────────────────────────────
220
+ 1. /pan:experiment new my-idea
221
+ ↓
222
+ 2. Creates D:\pan-experiments\my-idea\
223
+ ├── .planning/
224
+ │ ├── state.md
225
+ │ ├── idea.md ← from --idea <file>
226
+ │ └── ...
227
+ └── (PAN install for chosen runtime)
228
+
229
+ 3. /pan:experiment run my-idea --runtime claude
230
+ ↓
231
+ runner.cjs spawns: claude -p "/pan:new-project --auto @idea.md"
232
+ ↓
233
+ External Claude Code session runs autonomously inside the experiment folder.
234
+ Builds the idea using the same PAN workflows we just shipped (with guardrails!)
235
+
236
+ 4. Trace logger inside experiment captures every agent spawn.
237
+ PAN session tails experiment's state.md / agent-history.json / summaries.
238
+
239
+ 5. /pan:experiment harvest my-idea
240
+ ↓
241
+ Copies experiment's .planning/optimization/, agent-history.json,
242
+ state.md, summary files BACK to source repo:
243
+ d:\PanWizard\experiments\my-idea\
244
+ ├── traces/
245
+ ├── reports/
246
+ └── snapshot/
247
+
248
+ 6. /pan:learn --experiment my-idea
249
+ ↓
250
+ pan-optimizer (existing agent) reads experiments/my-idea/, generates:
251
+ experiments/my-idea/learnings/report-<timestamp>.md
252
+ with ## Findings + ## Auto-Apply Actions JSON
253
+
254
+ 7. pan-tools learn promote --pattern <id> --scope universal
255
+ ↓
256
+ Extracts the finding into:
257
+ pan-wizard-core/learnings/universal/<topic>.md
258
+
259
+ 8. /pan:check → npm test → git commit → v3.7.x release
260
+ ↓
261
+ Next user install gets the finding shipped under
262
+ .claude/pan-wizard-core/learnings/universal/<topic>.md
263
+ Workflows already reference learnings/universal/ — auto-loaded.
264
+ ```
265
+
266
+ ### 3.2 Module Plan
267
+
268
+ #### `pan-wizard-core/bin/lib/experiment.cjs` (new — 28th module)
269
+
270
+ ```js
271
+ // Pure-ish functions. Side effects only at write boundaries.
272
+
273
+ const PAN_EXPERIMENTS_ROOT_DEFAULT = path.join(os.homedir(), 'pan-experiments');
274
+ // Configurable via .planning/config.json `experiments_root`
275
+
276
+ function newExperiment(slug, opts) {
277
+ // opts: { ideaPath, runtime, root, budget }
278
+ // 1. Validate slug (lowercase, hyphens, max 40 chars)
279
+ // 2. Compute path: <root>/<slug>
280
+ // 3. Refuse if path exists (no clobber)
281
+ // 4. Refuse if path is inside source repo (PAN_SOURCE_ROOT guard)
282
+ // 5. mkdir, copy idea.md to <path>/.planning/idea.md (or write template)
283
+ // 6. Run installer: node <source>/bin/install.js --<runtime> --local at <path>
284
+ // 7. Write <path>/.planning/experiments/<id>/manifest.json with metadata
285
+ // 8. Return { experiment_id, path, runtime, idea_path, created_at }
286
+ }
287
+
288
+ function listExperiments(opts) {
289
+ // List all experiments under root, with status, runtime, created_at
290
+ }
291
+
292
+ function getExperimentManifest(slug, opts) {
293
+ // Read <root>/<slug>/.planning/experiments/<slug>/manifest.json
294
+ }
295
+
296
+ function harvestExperiment(slug, opts) {
297
+ // 1. Locate experiment path
298
+ // 2. Source: <experiment>/.planning/optimization/, agent-history.json, state.md, phases/
299
+ // 3. Destination: d:\PanWizard\experiments\<slug>\
300
+ // 4. Copy with directory structure preserved
301
+ // 5. Write harvest manifest with timestamp + size + checksum
302
+ // 6. Return { harvested_files, total_bytes, harvest_path }
303
+ }
304
+
305
+ function pruneExperiment(slug, opts) {
306
+ // After harvest + promote, optionally delete the experiment folder
307
+ // Soft delete: rename to <slug>-archived-<timestamp>
308
+ }
309
+
310
+ module.exports = {
311
+ newExperiment, listExperiments, getExperimentManifest,
312
+ harvestExperiment, pruneExperiment,
313
+ PAN_EXPERIMENTS_ROOT_DEFAULT,
314
+ };
315
+ ```
316
+
317
+ #### `pan-wizard-core/bin/lib/runner.cjs` (new — 29th module)
318
+
319
+ ```js
320
+ // External agent subprocess orchestration.
321
+ // Per-runtime adapter pattern.
322
+
323
+ const RUNTIME_RUNNERS = {
324
+ claude: { bin: 'claude', args: (prompt) => ['-p', prompt] },
325
+ codex: { bin: 'codex', args: (prompt) => ['exec', prompt] },
326
+ gemini: { bin: 'gemini', args: (prompt) => ['-p', prompt] },
327
+ opencode: { bin: 'opencode', args: (prompt) => [prompt] },
328
+ copilot: null, // unsupported (no headless prompt mode)
329
+ };
330
+
331
+ function runExperiment(slug, opts) {
332
+ // opts: { runtime, prompt, timeout, budget, onProgress }
333
+ // 1. Validate runtime support
334
+ // 2. Build command args
335
+ // 3. spawn child_process.spawn() in experiment dir
336
+ // 4. Stream stdout/stderr via opts.onProgress callback (line-buffered)
337
+ // 5. Watch experiment's .planning/state.md / agent-history.json for status
338
+ // 6. Enforce timeout
339
+ // 7. Enforce circuit breaker (3 consecutive task failures)
340
+ // 8. Persist run-state.json after every status change
341
+ // 9. Return { exit_code, status, elapsed_ms, summaries_count, stop_reason }
342
+ }
343
+
344
+ function tailExperimentState(slug, opts) {
345
+ // File-watcher loop on state.md, agent-history.json
346
+ // Emits `state_change`, `agent_spawned`, `agent_completed`, `summary_written` events
347
+ // Used by runExperiment internally; also exposed for monitoring without re-running
348
+ }
349
+
350
+ function stopExperiment(slug) {
351
+ // Read PID from run-state.json, send SIGTERM, fall back to SIGKILL
352
+ }
353
+
354
+ module.exports = { runExperiment, tailExperimentState, stopExperiment, RUNTIME_RUNNERS };
355
+ ```
356
+
357
+ #### `pan-wizard-core/bin/lib/optimize.cjs` (extend existing — add `promotePattern`)
358
+
359
+ ```js
360
+ // New export, alongside existing analyzeEvents, generateLocalReport, etc.
361
+
362
+ function promotePattern(reportPath, patternId, opts) {
363
+ // opts: { scope: 'universal'|'internal', topic: string, sourceRoot: string }
364
+ // 1. Read report file, extract the named pattern from ## Findings
365
+ // 2. Validate pattern has: id, summary, evidence (>= 1 trace event), proposed_rule
366
+ // 3. Compute output path:
367
+ // universal → <sourceRoot>/pan-wizard-core/learnings/universal/<topic>.md
368
+ // internal → <sourceRoot>/pan-wizard-core/learnings/internal/<topic>.md
369
+ // 4. Append (or create) the topic file with the new pattern
370
+ // 5. Front-matter: pattern_ids list, last_updated, source_experiments list
371
+ // 6. Return { promoted_to: path, pattern_id, scope, topic }
372
+ }
373
+
374
+ function listPromotedPatterns(sourceRoot) {
375
+ // Walk learnings/{universal,internal}/, return inventory
376
+ }
377
+
378
+ function unpromotePattern(topicFile, patternId) {
379
+ // Remove a pattern from a topic file (for rollback)
380
+ }
381
+ ```
382
+
383
+ ### 3.3 Command Surface
384
+
385
+ #### `commands/pan/experiment.md` (new — 52nd command)
386
+
387
+ ```
388
+ /pan:experiment new <slug> --idea <path> [--runtime <r>] [--root <path>] [--budget <pts>]
389
+ /pan:experiment list [--status <pending|running|done|failed>]
390
+ /pan:experiment run <slug> [--runtime <r>] [--timeout <sec>] [--no-stream]
391
+ /pan:experiment status <slug>
392
+ /pan:experiment stop <slug>
393
+ /pan:experiment harvest <slug> [--prune]
394
+ /pan:experiment harvest --all
395
+ /pan:experiment archive <slug>
396
+ /pan:experiment delete <slug> --confirm
397
+ ```
398
+
399
+ CLI subcommands (`pan-tools experiment <sub>`):
400
+ - `experiment new <slug> --idea <path>`
401
+ - `experiment list`
402
+ - `experiment manifest <slug>`
403
+ - `experiment run <slug>` (or via `runner.cjs` directly)
404
+ - `experiment harvest <slug>`
405
+ - `experiment prune <slug>`
406
+
407
+ #### Extended `/pan:learn` and `/pan:optimize`
408
+
409
+ ```
410
+ /pan:learn --experiment <slug> # NEW: run pan-optimizer over harvested experiment
411
+ pan-tools learn promote --pattern <id> --scope universal --topic <name>
412
+ pan-tools learn unpromote --pattern <id> --topic <name>
413
+ pan-tools learn list-promoted
414
+ ```
415
+
416
+ ### 3.4 New Agent: `pan-experiment-runner`
417
+
418
+ ```yaml
419
+ ---
420
+ name: pan-experiment-runner
421
+ description: Drives an external AI coding session against an experiment folder. Tails state, observes summaries, decides when the build is "done enough." Read-only relative to PAN source; writes only to <experiment-folder>/.planning/.
422
+ tools: Read, Bash, Glob, Grep
423
+ color: orange
424
+ thinking: enabled
425
+ thinking_budget: 6000
426
+ ---
427
+
428
+ <role>
429
+ You are the experiment runner. You drive an AUTONOMOUS external Claude Code (or other CLI) instance against an isolated experiment folder. Your job: observe, not intervene.
430
+
431
+ You are NOT the agent doing the build. You are the watchdog. The external instance does the actual work.
432
+
433
+ **You may NOT:**
434
+ - Edit files in the experiment folder
435
+ - Inject prompts into the external instance
436
+ - Modify the source repo
437
+
438
+ **You MAY:**
439
+ - Read the experiment's .planning/ state
440
+ - Tail the experiment's agent-history.json
441
+ - Decide when to declare success / failure / timeout based on stop conditions
442
+ - Update PAN's .planning/experiments/<slug>/run-state.json
443
+ - Surface progress to the orchestrating user
444
+ </role>
445
+
446
+ <stop_conditions>
447
+ 1. **success**: experiment's state.md status = "done" AND the milestone summary file exists
448
+ 2. **budget_exhausted**: external instance's cumulative trace event count > opts.budget * SPEND_RATIO
449
+ 3. **timeout**: elapsed wall-clock > opts.timeout
450
+ 4. **circuit_breaker**: 3 consecutive agent failures in agent-history.json
451
+ 5. **manual_stop**: PAN session writes <experiment>/.planning/STOP signal file
452
+ </stop_conditions>
453
+
454
+ <observations_to_log>
455
+ Every state transition writes a trace event in PAN's experiment record:
456
+ - agent_spawned (with task description)
457
+ - agent_completed (with status, elapsed)
458
+ - summary_written (with phase, plan)
459
+ - state_change (with old → new status)
460
+ - stop_condition_hit (with reason)
461
+ </observations_to_log>
462
+ ```
463
+
464
+ ### 3.5 Idea Document Schema
465
+
466
+ `pan-wizard-core/templates/idea.md`:
467
+
468
+ ```markdown
469
+ ---
470
+ title: <one-line idea title>
471
+ created: <ISO-8601>
472
+ created_by: <user>
473
+ runtime_preference: claude # claude|codex|gemini|opencode
474
+ budget: 80 # XS-XL points
475
+ priority: medium # low|medium|high
476
+ ---
477
+
478
+ # Idea: <title>
479
+
480
+ ## Problem
481
+ What user pain does this address?
482
+
483
+ ## Success Criteria
484
+ What does "this works" look like? Be concrete.
485
+
486
+ ## Scope
487
+ | In Scope | Out of Scope |
488
+ |----------|--------------|
489
+ | ... | ... |
490
+
491
+ ## Constraints
492
+ - Tech stack
493
+ - Dependencies
494
+ - Deadlines
495
+ - Runtime support requirements
496
+
497
+ ## Reference Material (optional)
498
+ - @path/to/related-doc.md
499
+ - https://link/to/inspiration
500
+
501
+ ## Notes (optional)
502
+ Free-form context the external agent should know.
503
+ ```
504
+
505
+ ### 3.6 Two-Tier Learnings Layout
506
+
507
+ ```
508
+ pan-wizard-core/
509
+ ├── references/ # human-authored, shipped to all installs (existing — guardrails, tdd, etc.)
510
+ ├── learnings/ # NEW
511
+ │ ├── universal/ # AI-derived, shipped to all installs (consumed by user-project workflows)
512
+ │ │ ├── exec-patterns.md
513
+ │ │ ├── plan-patterns.md
514
+ │ │ ├── verification-patterns.md
515
+ │ │ └── debugging-patterns.md
516
+ │ └── internal/ # AI-derived, source-only (consumed when working on PAN itself)
517
+ │ ├── pan-dev-patterns.md
518
+ │ └── installer-patterns.md
519
+ └── ...
520
+ ```
521
+
522
+ Each topic file structure:
523
+
524
+ ```markdown
525
+ ---
526
+ topic: exec-patterns
527
+ last_updated: 2026-04-30
528
+ patterns:
529
+ - id: P-001
530
+ summary: Always run npm test:all before phase complete
531
+ promoted_at: 2026-04-30
532
+ source_experiments: [my-idea-1, my-idea-2]
533
+ - id: P-002
534
+ summary: When verification fails on missing artifact, scaffold first then fix
535
+ promoted_at: 2026-05-02
536
+ source_experiments: [my-idea-3]
537
+ ---
538
+
539
+ # Exec Patterns (AI-derived)
540
+
541
+ This file is auto-maintained by `pan-tools learn promote`. Each pattern was
542
+ extracted from one or more experiment runs (see source_experiments). Patterns
543
+ are advisory — the orchestrator should weight them against current context.
544
+
545
+ ## P-001 — Always run `npm test:all` before phase complete
546
+
547
+ **Evidence:** 14 trace events across experiments my-idea-1, my-idea-2 where
548
+ `/pan:verify-phase` passed but a downstream scenario test caught a regression.
549
+
550
+ **Rule:** Before marking a phase complete, run the full test suite, not just
551
+ the phase's own tests. If `test:all` script exists, prefer it over `test`.
552
+
553
+ **Applies in:** exec-phase, focus-exec, milestone-done
554
+
555
+ ## P-002 — ...
556
+ ```
557
+
558
+ ### 3.7 Installer Changes
559
+
560
+ `bin/install.js` glob already covers any new top-level dir under `pan-wizard-core/`. Verify:
561
+
562
+ ```js
563
+ // Around bin/install.js line ~XXX (verify exact location at impl)
564
+ const corePaths = [
565
+ 'pan-wizard-core/bin',
566
+ 'pan-wizard-core/workflows',
567
+ 'pan-wizard-core/templates',
568
+ 'pan-wizard-core/references',
569
+ 'pan-wizard-core/learnings/universal', // NEW — explicitly include
570
+ // 'pan-wizard-core/learnings/internal' INTENTIONALLY OMITTED
571
+ ];
572
+ ```
573
+
574
+ Adding `learnings/universal` explicitly (instead of relying on glob) makes the universal-vs-internal split discoverable in install.js. Tests assert `learnings/internal/` is NOT in any install dir.
575
+
576
+ ### 3.8 Workflow Cross-References
577
+
578
+ Following the v3.6.0 pattern, 4 workflows gain a reference to `learnings/universal/`:
579
+
580
+ ```markdown
581
+ <required_reading>
582
+ @~/.claude/pan-wizard-core/references/guardrails.md
583
+ @~/.claude/pan-wizard-core/learnings/universal/ <!-- NEW: directory glob loads all topic files -->
584
+ </required_reading>
585
+ ```
586
+
587
+ Workflows updated:
588
+ - `exec-phase.md`
589
+ - `plan-phase.md`
590
+ - `verify-phase.md`
591
+ - `execute-plan.md`
592
+
593
+ ### 3.9 End-to-End Flow Walkthrough
594
+
595
+ User runs:
596
+
597
+ ```bash
598
+ # 1. Capture an idea
599
+ echo "# Idea: Build a tiny markdown linter CLI" > my-idea.md
600
+ # (fill in problem, success, scope, constraints)
601
+
602
+ # 2. Scaffold experiment
603
+ pan-tools experiment new md-lint --idea my-idea.md --runtime claude --budget 60
604
+ # → creates D:/pan-experiments/md-lint/, installs PAN for claude, copies idea.md
605
+
606
+ # 3. Run the experiment (autonomous external Claude Code)
607
+ pan-tools experiment run md-lint
608
+ # → spawns: claude -p "/pan:new-project --auto @.planning/idea.md"
609
+ # inside D:/pan-experiments/md-lint/
610
+ # → tails its state.md; PAN session prints periodic progress
611
+
612
+ # 4. After ~10-30 min, experiment completes (or hits a stop condition)
613
+ pan-tools experiment harvest md-lint
614
+ # → copies optimization/, agent-history, state, summaries to
615
+ # d:/PanWizard/experiments/md-lint/
616
+
617
+ # 5. Run pan-optimizer over harvested data
618
+ /pan:learn --experiment md-lint
619
+ # → produces d:/PanWizard/experiments/md-lint/learnings/report-<ts>.md
620
+
621
+ # 6. Review report. Promote select findings:
622
+ pan-tools learn promote --pattern P-007 --scope universal --topic exec-patterns
623
+ # → appends pattern to pan-wizard-core/learnings/universal/exec-patterns.md
624
+
625
+ pan-tools learn promote --pattern P-013 --scope internal --topic pan-dev-patterns
626
+ # → appends to pan-wizard-core/learnings/internal/pan-dev-patterns.md
627
+
628
+ # 7. Verify shipping behavior
629
+ npm run test:all # asserts learnings/universal/ ships, internal/ does not
630
+ node bin/install.js --claude --local # in d:/pantesting, verify universal/ present, internal/ absent
631
+
632
+ # 8. Ship as v3.7.x
633
+ # CHANGELOG entry. ADR entry. Commit. Tag.
634
+ ```
635
+
636
+ ### 3.10 Implementation Plan (4 Waves Across v3.7.0 → v3.7.3)
637
+
638
+ | Wave | Version | Items | Pts | Days |
639
+ |------|---------|-------|-----|------|
640
+ | W1: Experiment scaffolding + idea schema | v3.7.0 | experiment.cjs (new + list + manifest), idea.md template, experiment.md command (subset), tests | 12 | 1 |
641
+ | W2: External runner + observation | v3.7.1 | runner.cjs, pan-experiment-runner agent, run/status/stop subcommands, file-tail, run-state.json | 10 | 1 |
642
+ | W3: Harvest + extended /pan:learn | v3.7.2 | experiment.harvestExperiment(), `/pan:learn --experiment`, harvest/prune subcommands, scenario tests | 8 | 1 |
643
+ | W4: Promote + ship | v3.7.3 | optimize.promotePattern(), learn promote/unpromote/list-promoted, learnings/{universal,internal}/ tier, installer + workflow refs, ADR-0026 | 10 | 1 |
644
+
645
+ **Total: ~40 pts, 4 sessions, ships across 4 patch versions OR consolidated as v3.7.0 if shipped as a single wave.**
646
+
647
+ Recommendation: ship as **v3.7.0 single wave** (40 pts is doable in one focused session with the spec already written). Each wave above can also be a separate session if the user prefers staged rollout.
648
+
649
+ ### 3.11 Test Plan
650
+
651
+ Target: 30+ new tests.
652
+
653
+ #### Unit tests
654
+
655
+ - `tests/experiment.test.cjs` (8-10 tests): newExperiment validation, root path, slug rules, manifest, list, prune
656
+ - `tests/runner.test.cjs` (6-8 tests): runtime adapter selection, mocked subprocess spawn, timeout enforcement, circuit breaker, run-state persistence
657
+ - `tests/learn-promote.test.cjs` (6-8 tests): promotePattern shape, scope routing, topic file append, unpromote round-trip, list-promoted
658
+
659
+ #### Scenario tests
660
+
661
+ - `tests/scenarios/learnings-installed.test.cjs` (10 tests): `learnings/universal/*.md` present in all 5 runtime installs, `learnings/internal/` ABSENT in all 5 runtime installs (negative assertion), idea.md template ships, experiment.md command ships
662
+ - `tests/scenarios/experiment-lifecycle.test.cjs` (5 tests): mocked end-to-end — new → manifest → harvest → promote — without spawning a real external process
663
+
664
+ #### Integration tests (optional, opt-in via env flag)
665
+
666
+ - `tests/integration/real-experiment.test.cjs` — gated by `PAN_REAL_EXPERIMENT=1`, runs an actual `claude -p` against a trivial idea. Skipped in CI by default; available for manual smoke before each release.
667
+
668
+ ### 3.12 Documentation
669
+
670
+ - `docs/USER-GUIDE.md` — new "Self-Improvement Loop" section under Advanced Features, walking through the end-to-end flow
671
+ - `docs/ARCHITECTURE.md` — new "v3.7.0 additions — Self-Improvement Loop" section listing the 2 new modules + new agent + 2 new top-level dirs
672
+ - `docs/decisions/ADR-0026-self-improvement-loop.md` — records: (a) external-process model (subprocess vs VSCode), (b) two-tier learnings split (universal vs internal), (c) manual promote (defer auto-promote to v3.8), (d) experiment root outside source repo
673
+ - `CHANGELOG.md` — v3.7.0 entry
674
+
675
+ ---
676
+
677
+ ## Phase 4: Specification — Ready-to-Implement
678
+
679
+ ### Files to Create
680
+
681
+ 1. `pan-wizard-core/bin/lib/experiment.cjs` (new — 28th core module, ~400 LOC)
682
+ 2. `pan-wizard-core/bin/lib/runner.cjs` (new — 29th core module, ~300 LOC)
683
+ 3. `pan-wizard-core/templates/idea.md` (new — 27th template)
684
+ 4. `pan-wizard-core/learnings/universal/.gitkeep` (placeholder — first promote creates topic files)
685
+ 5. `pan-wizard-core/learnings/internal/.gitkeep` (placeholder)
686
+ 6. `pan-wizard-core/learnings/README.md` (explains universal vs internal split)
687
+ 7. `commands/pan/experiment.md` (new — 52nd command, full subcommand surface)
688
+ 8. `agents/pan-experiment-runner.md` (new — 21st agent)
689
+ 9. `tests/experiment.test.cjs`
690
+ 10. `tests/runner.test.cjs`
691
+ 11. `tests/learn-promote.test.cjs`
692
+ 12. `tests/scenarios/learnings-installed.test.cjs`
693
+ 13. `tests/scenarios/experiment-lifecycle.test.cjs`
694
+ 14. `docs/decisions/ADR-0026-self-improvement-loop.md`
695
+
696
+ ### Files to Modify
697
+
698
+ 1. `pan-wizard-core/bin/lib/optimize.cjs` — add `promotePattern`, `listPromotedPatterns`, `unpromotePattern` exports
699
+ 2. `pan-wizard-core/bin/pan-tools.cjs` — register `experiment` and `learn promote/unpromote/list-promoted` subcommand routes
700
+ 3. `commands/pan/learn.md` — extend `--experiment <slug>` flag + promote subcommand surface
701
+ 4. `pan-wizard-core/workflows/learn.md` — add experiment-aware path
702
+ 5. `pan-wizard-core/workflows/exec-phase.md` — reference `learnings/universal/`
703
+ 6. `pan-wizard-core/workflows/plan-phase.md` — reference `learnings/universal/`
704
+ 7. `pan-wizard-core/workflows/verify-phase.md` — reference `learnings/universal/`
705
+ 8. `pan-wizard-core/workflows/execute-plan.md` — reference `learnings/universal/`
706
+ 9. `bin/install.js` — explicit `learnings/universal` in shipped paths; tests assert `learnings/internal` is NOT shipped
707
+ 10. `bin/install-lib.cjs` — verify glob handles new dir; add helper if needed
708
+ 11. `package.json` — bump to 3.7.0
709
+ 12. `CHANGELOG.md` — v3.7.0 entry
710
+ 13. `docs/USER-GUIDE.md` — new section
711
+ 14. `docs/ARCHITECTURE.md` — new section
712
+ 15. `MEMORY.md` — refresh counts (29 modules, 21 agents, 52 commands, ~2620 tests)
713
+
714
+ ### Step Order (recommended for single-session ship)
715
+
716
+ 1. Write `experiment.cjs` + `runner.cjs` skeletons with all exports stubbed
717
+ 2. Write all 5 test files (TDD-first; tests fail)
718
+ 3. Implement experiment.cjs functions (newExperiment → list → manifest → harvest → prune)
719
+ 4. Implement runner.cjs (RUNTIME_RUNNERS map, runExperiment with mocked subprocess for tests)
720
+ 5. Implement promotePattern in optimize.cjs
721
+ 6. Wire pan-tools.cjs dispatcher
722
+ 7. Run unit tests — green
723
+ 8. Create commands/pan/experiment.md, agents/pan-experiment-runner.md
724
+ 9. Add workflow cross-references
725
+ 10. Add `learnings/universal/`, `learnings/internal/` dirs with .gitkeep + README
726
+ 11. Update bin/install.js for two-tier delivery
727
+ 12. Run scenario tests — green (verifies shipping behavior)
728
+ 13. Manual install verification in d:\pantesting (5 runtimes)
729
+ 14. Manual smoke test: spawn a real `claude -p` against a trivial idea (opt-in integration)
730
+ 15. Update CHANGELOG, USER-GUIDE, ARCHITECTURE, MEMORY
731
+ 16. Write ADR-0026
732
+ 17. Bump version to 3.7.0
733
+ 18. Commit + tag
734
+
735
+ ### Test Coverage Summary
736
+
737
+ | File | Test count | Type |
738
+ |------|------------|------|
739
+ | experiment.test.cjs | 10 | Unit |
740
+ | runner.test.cjs | 8 | Unit |
741
+ | learn-promote.test.cjs | 8 | Unit |
742
+ | scenarios/learnings-installed.test.cjs | 10 | Scenario |
743
+ | scenarios/experiment-lifecycle.test.cjs | 5 | Scenario |
744
+ | **Total new** | **~41** | — |
745
+
746
+ ### Runtime Matrix
747
+
748
+ | Runtime | Experiment runner support | Receives `learnings/universal/` | Receives `learnings/internal/` |
749
+ |---------|---------------------------|--------------------------------|--------------------------------|
750
+ | Claude | ✅ via `claude -p` | ✅ | ❌ (intentional — source-only) |
751
+ | Codex | ✅ via `codex exec` | ✅ | ❌ |
752
+ | Gemini | ✅ via `gemini -p` | ✅ | ❌ |
753
+ | OpenCode | ✅ via `opencode` | ✅ | ❌ |
754
+ | GitHub Copilot | ❌ runner unsupported (no headless mode) | ✅ | ❌ |
755
+
756
+ GitHub Copilot users can still consume promoted learnings; they just can't drive an experiment runner. Documented limitation.
757
+
758
+ ---
759
+
760
+ ## Phase 5: Risk Assessment
761
+
762
+ | Risk | Impact | Likelihood | Mitigation |
763
+ |------|--------|------------|------------|
764
+ | Spawned external agent runs unbounded, consumes cost | High | Medium | Triple safety: timeout (default 30 min) + budget cap (default 80 pts) + circuit breaker (3 consecutive failures). Test coverage for each. |
765
+ | Experiment folder pollutes user disk | Low | Medium | Default root `~/pan-experiments/`; configurable. Document `pan-tools experiment delete` for cleanup. |
766
+ | Promote step writes garbage into shipped learnings | High | Medium | **Manual review gate** — promote requires explicit `--pattern <id>`; no auto-promote in v3.7. Tests verify file shape; ADR-0026 records the manual-only design. |
767
+ | Universal vs internal split is wrong (PAN-internal pattern shipped as universal) | Medium | High | Filter heuristic in promote: detect references to `pan-wizard-core/`, `bin/install.js`, `commands/pan/*` paths → suggest `--scope internal`. Human can override. |
768
+ | External agent doesn't return useful telemetry (e.g., gemini -p doesn't trigger PAN trace logger) | Medium | Medium | The trace logger fires on Claude Code SubagentStop; non-Claude runtimes log differently. Start with Claude as the primary; document fallback as "no auto-trace, manual harvest" for v3.7. |
769
+ | `claude -p` flag may not exist or differ across versions | Medium | Medium | Implement adapter map. Test against current Claude Code version at impl time. Document required Claude Code version in `references/`. |
770
+ | Cross-platform subprocess differences (Windows vs Unix) | Medium | Low | Use Node's `child_process.spawn` with `shell: false`. Existing scenario tests already cover Windows; new tests use same pattern. |
771
+ | Harvest copies sensitive data | Medium | Low | Document: idea.md is the only "sensitive" file; harvest excludes it by default; `--include-idea` opt-in flag for full archive. |
772
+ | `learnings/internal/` accidentally ships | High | Low | Negative test in `scenarios/learnings-installed.test.cjs`: assert NOT in any install dir. Installer test runs in CI on every change. |
773
+ | Cumulative learnings grow unbounded | Low | Medium | Each topic file has a soft size cap (10KB recommendation); patterns can be retired via `unpromote`. v3.8+ add an automatic age-based prune. |
774
+ | pan-experiment-runner agent intervenes in external instance | High | Low | Agent definition explicitly forbids editing experiment files. Agent tools list excludes Edit/Write. Tests assert no writes outside `<experiment>/.planning/`. |
775
+ | Test count grows fast — CI time increases | Low | High | New tests are mostly unit (fast). Integration test gated by env flag (excluded from CI). Mocked subprocess in scenario tests stays under 10s. |
776
+ | Spec/impl mismatch on runtime headless flags | Medium | Medium | Implementation phase verifies each runtime's actual headless invocation. Adapter map updates accordingly. ADR-0026 records what was verified vs assumed. |
777
+
778
+ ### Backward Compatibility
779
+
780
+ - All proposals are additive. No breaking changes to existing commands, workflows, agents, tests.
781
+ - `learnings/` is a new top-level directory under `pan-wizard-core/`. Existing v3.6.x installs upgrading to v3.7.0 receive the new dir + universal contents via standard `pan update` flow.
782
+ - Workflows reference `learnings/universal/` AFTER existing `references/` references. If the dir is missing (manual install corruption), workflows degrade gracefully — the `@~/.claude/.../learnings/universal/` glob just resolves to no files.
783
+ - `optimize.cjs` extensions (`promotePattern`, etc.) are new exports. Existing `analyzeEvents`, `generateLocalReport`, etc. unchanged.
784
+
785
+ ### What's Explicitly NOT in v3.7.0
786
+
787
+ - **Auto-promote** without human review — defer to v3.8+ once 10+ manual promotes inform the filter
788
+ - **Cross-experiment aggregation** — defer to v3.8+ (different design problem)
789
+ - **VSCode UI automation as runner path** — keep CLI subprocess; VSCode automation stays for tests
790
+ - **Bidirectional control mid-run** — observation-only by design
791
+ - **GitHub Copilot CLI runner** — no headless prompt mode known; document limitation
792
+ - **Experiment-driven evaluation** (LLM-as-judge over the experiment's outputs) — separate eval feature, see googlecli_adoption_featureai.md notes
793
+ - **Sandboxed/containerized experiments** — process isolation via separate folder is sufficient for v3.7; container isolation if needed later
794
+
795
+ ---
796
+
797
+ ## Appendix A: Open Questions (resolve before W4)
798
+
799
+ 1. **Should `learnings/universal/` topic files be human-editable, or AI-only?** Recommendation: AI-only via `promote/unpromote`. Human edits create drift between topic frontmatter `pattern_ids` and body content. For human-authored content, use `references/`.
800
+
801
+ 2. **Should the experiment runner use a single per-runtime adapter, or call the user's installed `claude` PATH?** Recommendation: PATH lookup (`which claude`). Document as a prerequisite in USER-GUIDE.
802
+
803
+ 3. **Should `harvest --all` interleave reports or keep per-experiment dirs?** Recommendation: per-experiment dirs (cleaner traceability). `learn promote --aggregate` is a v3.8 feature for cross-experiment patterns.
804
+
805
+ 4. **Should `learnings/universal/exec-patterns.md` REPLACE or AUGMENT `references/guardrails.md`?** Recommendation: augment. Guardrails are human-authored canonical rules; learnings are AI-derived advisory patterns. Workflows reference both.
806
+
807
+ 5. **Should runtime adapters for codex/gemini/opencode ship in v3.7.0 or claude-only?** Recommendation: claude-only in W2 (v3.7.1); add other adapters in v3.7.4 follow-up. Faster initial ship; same surface.
808
+
809
+ 6. **Should we add a `pan-tools experiment replay <slug>` to re-run from a specific phase?** Recommendation: defer to v3.8. Replay needs idempotency design that v3.7 doesn't need.
810
+
811
+ ---
812
+
813
+ ## Appendix B: Mapping to Existing PAN Surface
814
+
815
+ | New element | Closest existing analog | Drift risk |
816
+ |-------------|-------------------------|------------|
817
+ | `experiment.cjs` | `focus.cjs` (auto-runner) | Low — same lifecycle pattern |
818
+ | `runner.cjs` | `tests/helpers.cjs createScenarioRunner()` | Low — generalizes existing pattern |
819
+ | `pan-experiment-runner` agent | `pan-optimizer` agent | Low — same observation-only stance |
820
+ | `learnings/universal/` | `references/` | None — same delivery channel |
821
+ | `learnings/internal/` | `pan-wizard-core/.dev-only/` (doesn't exist) | New concept; needs negative test |
822
+ | `idea.md` template | `requirements.md`, `project_spec_*.md` (existing patterns) | Low — same shape |
823
+ | `pan-tools experiment new/run/harvest` | `pan-tools focus auto/scan/plan/exec` | Low — same subcommand pattern |
824
+ | `pan-tools learn promote` | `pan-tools optimize apply` | Medium — new write-target (source repo, not project) |
825
+
826
+ ---
827
+
828
+ ## Appendix C: Timeline & Dependencies
829
+
830
+ ```
831
+ v3.6.0 ✓ shipped 2026-04-27 (behavioral guardrails layer)
832
+ │
833
+ ├─ unblocks: shipping behavioral content via references/ pattern (proven)
834
+ │
835
+ v3.7.0 (this spec) — Self-Improvement Loop
836
+ │
837
+ ├─ W1: experiment scaffolding (12 pts)
838
+ ├─ W2: runner + observation (10 pts)
839
+ ├─ W3: harvest + extended /pan:learn (8 pts)
840
+ ├─ W4: promote + ship (10 pts)
841
+ │
842
+ └─ unblocks:
843
+ ├─ v3.8.0: auto-promote (rules-based filter, AI-confidence threshold)
844
+ ├─ v3.8.0: cross-experiment aggregation
845
+ ├─ v3.9.0: container-isolated experiments (if needed for security)
846
+ └─ v3.9.0: experiment-replay-from-phase (idempotency design)
847
+ ```
848
+
849
+ ---
850
+
851
+ ## Verification Checklist (pre-merge for v3.7.0)
852
+
853
+ - [ ] `pan-wizard-core/bin/lib/experiment.cjs` exists, exports newExperiment / listExperiments / harvestExperiment / pruneExperiment
854
+ - [ ] `pan-wizard-core/bin/lib/runner.cjs` exists, exports runExperiment / tailExperimentState / stopExperiment / RUNTIME_RUNNERS
855
+ - [ ] `pan-wizard-core/bin/lib/optimize.cjs` exports promotePattern / listPromotedPatterns / unpromotePattern (added)
856
+ - [ ] `commands/pan/experiment.md` registered, runs all 8 subcommands
857
+ - [ ] `agents/pan-experiment-runner.md` exists, has correct tool list (Read/Bash/Glob/Grep — NO Edit/Write)
858
+ - [ ] `pan-wizard-core/templates/idea.md` exists with required frontmatter fields
859
+ - [ ] `pan-wizard-core/learnings/universal/` exists; ships to all 5 runtime installs
860
+ - [ ] `pan-wizard-core/learnings/internal/` exists; does NOT ship to any install (negative test passes)
861
+ - [ ] 4 long workflows reference `learnings/universal/`
862
+ - [ ] `bin/install.js` explicit two-tier handling verified by tests
863
+ - [ ] `npm test` passes, 41 new tests added
864
+ - [ ] `npm run test:scenarios` passes, including learnings shipping tests
865
+ - [ ] Manual install in d:\pantesting for 2+ runtimes confirms learnings/universal/ shipped, learnings/internal/ absent
866
+ - [ ] Manual smoke: real `claude -p` experiment runs end-to-end on a trivial idea, harvest + promote complete the loop
867
+ - [ ] CHANGELOG.md, USER-GUIDE.md, ARCHITECTURE.md updated
868
+ - [ ] ADR-0026 committed
869
+ - [ ] MEMORY.md refreshed (29 modules, 21 agents, 52 commands, ~2620 tests)
870
+ - [ ] `package.json` version is 3.7.0
871
+ - [ ] All v3.6.0 behavioral guardrails still in effect (regression check)
872
+
873
+ ---
874
+
875
+ **End of spec. Awaiting acceptance to begin v3.7.0 W1.**