session-orchestrator 3.16.0 → 3.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (220) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/.claude-plugin/plugin.json +1 -1
  3. package/.codex-plugin/plugin.json +1 -1
  4. package/.cursor/rules/030-wave-execution.mdc +17 -1
  5. package/CHANGELOG.md +130 -412
  6. package/README.md +17 -12
  7. package/SECURITY.md +190 -27
  8. package/agents/AGENTS.md +20 -3
  9. package/agents/code-implementer.md +6 -6
  10. package/agents/db-specialist.md +1 -1
  11. package/agents/qa-strategist.md +31 -6
  12. package/agents/schemas/qa-strategist.schema.json +27 -0
  13. package/agents/schemas/test-writer.schema.json +60 -2
  14. package/agents/security-reviewer.md +1 -1
  15. package/agents/session-reviewer.md +1 -1
  16. package/agents/test-writer.md +29 -10
  17. package/agents/ui-developer.md +1 -1
  18. package/commands/contract-version-bump.md +28 -0
  19. package/commands/portfolio.md +1 -1
  20. package/docs/README.md +2 -1
  21. package/docs/USER-GUIDE.md +8 -3
  22. package/docs/ci-setup.md +121 -7
  23. package/docs/codex-setup.md +1 -1
  24. package/docs/components.md +7 -7
  25. package/docs/cursor-setup.md +22 -9
  26. package/docs/events-schema.md +5 -1
  27. package/docs/instruction-delivery.md +444 -0
  28. package/docs/pi-setup.md +1 -1
  29. package/docs/rule-authoring.md +58 -9
  30. package/docs/session-config-reference.md +306 -6
  31. package/docs/session-config-template.md +66 -3
  32. package/docs/telemetry/telemetry-claims.md +204 -0
  33. package/docs/telemetry.md +158 -0
  34. package/hooks/_lib/guard-source-loader.mjs +467 -0
  35. package/hooks/_lib/lock-bootstrap.mjs +21 -0
  36. package/hooks/_lib/vcs-create-matcher.mjs +119 -0
  37. package/hooks/config-protection.mjs +0 -0
  38. package/hooks/enforce-commands.mjs +10 -2
  39. package/hooks/hooks-codex.json +1 -1
  40. package/hooks/hooks-cursor.json +11 -2
  41. package/hooks/hooks-pi.json +10 -0
  42. package/hooks/hooks.json +21 -1
  43. package/hooks/on-session-end.mjs +178 -18
  44. package/hooks/on-session-start.mjs +23 -0
  45. package/hooks/post-bash-write-verify.mjs +977 -0
  46. package/hooks/post-subagent-discovery-validator.mjs +256 -41
  47. package/hooks/pre-bash-destructive-guard.mjs +525 -160
  48. package/hooks/pre-bash-issue-budget.mjs +167 -0
  49. package/hooks/pre-bash-sessions-ledger-guard.mjs +627 -0
  50. package/hooks/pre-bash-templates-first.mjs +96 -63
  51. package/hooks/skill-invocation-telemetry.mjs +109 -10
  52. package/hooks/subagent-telemetry.mjs +527 -37
  53. package/package.json +16 -3
  54. package/pi/prompts/contract-version-bump.md +12 -0
  55. package/rules/README.md +32 -0
  56. package/scripts/archive-closed-prds.mjs +12 -22
  57. package/scripts/autopilot-multi.mjs +103 -20
  58. package/scripts/backfill-abandoned-sessions.mjs +160 -4
  59. package/scripts/check-doc-consistency.sh +17 -1
  60. package/scripts/compute-grounding-injection.sh +18 -3
  61. package/scripts/dialectic-deriver.mjs +7 -2
  62. package/scripts/eval-session.mjs +50 -9
  63. package/scripts/fleet-instruction-scan.mjs +141 -0
  64. package/scripts/lib/auto-dialectic.mjs +11 -2
  65. package/scripts/lib/auto-dream.mjs +16 -5
  66. package/scripts/lib/autopilot/mr-draft.mjs +31 -1
  67. package/scripts/lib/autopilot/worktree-pipeline.mjs +113 -5
  68. package/scripts/lib/backlog-scan.mjs +39 -6
  69. package/scripts/lib/blocked-commands-policy.mjs +340 -0
  70. package/scripts/lib/build-live-signals.mjs +7 -4
  71. package/scripts/lib/ci-status-banner.mjs +75 -12
  72. package/scripts/lib/claude-md-budget-lint.mjs +283 -34
  73. package/scripts/lib/command-blocker.mjs +1013 -58
  74. package/scripts/lib/config/config-protection.mjs +2 -1
  75. package/scripts/lib/config/context-coverage.mjs +82 -0
  76. package/scripts/lib/config/drift-check.mjs +9 -1
  77. package/scripts/lib/config/gitlab-portfolio.mjs +1 -1
  78. package/scripts/lib/config/issue-budget.mjs +123 -0
  79. package/scripts/lib/config/moc-staleness.mjs +98 -0
  80. package/scripts/lib/config/reconcile.mjs +21 -0
  81. package/scripts/lib/config/section-extractor.mjs +121 -1
  82. package/scripts/lib/config/worktree-orphans.mjs +138 -0
  83. package/scripts/lib/config-schema.mjs +23 -3
  84. package/scripts/lib/config.mjs +32 -0
  85. package/scripts/lib/context-coverage-banner.mjs +223 -0
  86. package/scripts/lib/convergence-monitor.mjs +49 -3
  87. package/scripts/lib/description-surface.mjs +535 -0
  88. package/scripts/lib/dispatcher/enumerate.mjs +136 -30
  89. package/scripts/lib/dispatcher/rank.mjs +22 -8
  90. package/scripts/lib/ecosystem-wizard/config-writer.mjs +26 -24
  91. package/scripts/lib/ecosystem-wizard/wizard-prompt.mjs +1 -1
  92. package/scripts/lib/eval/engine.mjs +47 -5
  93. package/scripts/lib/events.mjs +59 -7
  94. package/scripts/lib/evolve/autonomy-verdict.mjs +5 -0
  95. package/scripts/lib/evolve/autopilot-effectiveness.mjs +54 -7
  96. package/scripts/lib/gates/gate-full.mjs +15 -3
  97. package/scripts/lib/gates/gate-helpers.mjs +132 -6
  98. package/scripts/lib/gitlab-ops/stale-mr-sweep.mjs +28 -8
  99. package/scripts/lib/gitlab-portfolio/aggregator.mjs +8 -2
  100. package/scripts/lib/gitlab-portfolio/cli.mjs +1 -1
  101. package/scripts/lib/handover-gate.mjs +7 -3
  102. package/scripts/lib/harness-audit/categories/category4.mjs +22 -5
  103. package/scripts/lib/instruction-budget-guard.mjs +402 -51
  104. package/scripts/lib/io.mjs +345 -10
  105. package/scripts/lib/issue-budget.mjs +269 -0
  106. package/scripts/lib/issue-close-strip-labels.mjs +39 -9
  107. package/scripts/lib/label-scope.mjs +47 -0
  108. package/scripts/lib/learnings/schema.mjs +43 -3
  109. package/scripts/lib/lock-reaper.mjs +1 -2
  110. package/scripts/lib/memory-proposals/schema.mjs +36 -1
  111. package/scripts/lib/moc-staleness-banner.mjs +267 -0
  112. package/scripts/lib/peer-discovery.mjs +645 -0
  113. package/scripts/lib/pi-hook-bridge.mjs +146 -17
  114. package/scripts/lib/product-repo-detect.mjs +9 -8
  115. package/scripts/lib/project-hygiene.mjs +432 -0
  116. package/scripts/lib/quality-gate.mjs +167 -0
  117. package/scripts/lib/recommendations-v0.mjs +1 -1
  118. package/scripts/lib/reconcile/eligibility.mjs +1 -1
  119. package/scripts/lib/reconcile/emitter.mjs +23 -4
  120. package/scripts/lib/reconcile/engine.mjs +147 -39
  121. package/scripts/lib/reconcile/idempotency.mjs +114 -14
  122. package/scripts/lib/reconcile-nudge-banner.mjs +65 -9
  123. package/scripts/lib/resource-probe/evaluate.mjs +70 -4
  124. package/scripts/lib/resource-probe.mjs +19 -0
  125. package/scripts/lib/rule-loader.mjs +6 -0
  126. package/scripts/lib/scope-baseline.mjs +564 -0
  127. package/scripts/lib/scope-gate.mjs +399 -98
  128. package/scripts/lib/session-close-backfill.mjs +61 -6
  129. package/scripts/lib/session-end/phase-skip.mjs +1 -0
  130. package/scripts/lib/session-end/worktree-orphan-sweep.mjs +252 -0
  131. package/scripts/lib/session-id.mjs +221 -41
  132. package/scripts/lib/session-lock.mjs +304 -6
  133. package/scripts/lib/session-schema/constants.mjs +22 -3
  134. package/scripts/lib/session-schema/filters.mjs +88 -0
  135. package/scripts/lib/session-schema/validator.mjs +16 -0
  136. package/scripts/lib/session-schema.mjs +1 -0
  137. package/scripts/lib/sessions-integrity-banner.mjs +294 -0
  138. package/scripts/lib/sessions-staleness-banner.mjs +121 -12
  139. package/scripts/lib/skill-evolution/idempotency.mjs +135 -16
  140. package/scripts/lib/skill-evolution/mr-opener.mjs +9 -1
  141. package/scripts/lib/skill-health/join.mjs +35 -9
  142. package/scripts/lib/spiral-carryover.mjs +142 -30
  143. package/scripts/lib/state-md/mission-status.mjs +53 -3
  144. package/scripts/lib/subagents-schema.mjs +43 -9
  145. package/scripts/lib/telemetry/anon-id.mjs +141 -0
  146. package/scripts/lib/telemetry/consent.mjs +299 -0
  147. package/scripts/lib/telemetry/paths.mjs +27 -0
  148. package/scripts/lib/telemetry/queue.mjs +287 -0
  149. package/scripts/lib/telemetry/schema.mjs +384 -0
  150. package/scripts/lib/telemetry/sync.mjs +312 -0
  151. package/scripts/lib/test-runner/issue-reconcile.mjs +53 -13
  152. package/scripts/lib/tests-src-ratio.mjs +484 -0
  153. package/scripts/lib/validate/check-agents.mjs +56 -0
  154. package/scripts/lib/validate/check-hooks-symmetry.mjs +244 -10
  155. package/scripts/lib/validate/check-rules.mjs +217 -35
  156. package/scripts/lib/validate/check-test-value-bans.mjs +782 -0
  157. package/scripts/lib/validate/check-unicode-safety.mjs +1 -0
  158. package/scripts/lib/validate-vendored-rules.mjs +10 -2
  159. package/scripts/lib/vault-archive.mjs +17 -2
  160. package/scripts/lib/vault-backfill/glab.mjs +8 -0
  161. package/scripts/lib/vault-mirror/process.mjs +30 -0
  162. package/scripts/lib/vault-mirror/render-sessions.mjs +293 -36
  163. package/scripts/lib/vault-status/board-writer.mjs +63 -5
  164. package/scripts/lib/vault-status/narrative-mirror.mjs +13 -7
  165. package/scripts/lib/vcs-repo-spec.mjs +362 -0
  166. package/scripts/lib/wave-resource-gate.mjs +115 -11
  167. package/scripts/lib/worktree/listing.mjs +44 -7
  168. package/scripts/mcp-server.sh +32 -6
  169. package/scripts/measure-context-overhead.sh +151 -0
  170. package/scripts/memory-propose.mjs +72 -9
  171. package/scripts/print-applicable-rules.mjs +51 -12
  172. package/scripts/release.mjs +534 -0
  173. package/scripts/run-quality-gate.mjs +123 -5
  174. package/scripts/telemetry.mjs +250 -0
  175. package/scripts/validate-wave-scope.mjs +182 -17
  176. package/scripts/vault-integration-watcher.mjs +32 -10
  177. package/skills/_shared/config-reading.md +2 -2
  178. package/skills/bootstrap/fast-template.md +1 -1
  179. package/skills/claude-md-drift-check/checker.mjs +145 -28
  180. package/skills/contract-version-bump/SKILL.md +219 -0
  181. package/skills/discovery/SKILL.md +4 -4
  182. package/skills/discovery/issue-templates.md +11 -11
  183. package/skills/discovery/probes-audit.md +1 -1
  184. package/skills/discovery/probes-feature.md +1 -1
  185. package/skills/discovery/probes-session.md +26 -5
  186. package/skills/ecosystem-health/SKILL.md +1 -1
  187. package/skills/ecosystem-health/wizard.md +4 -4
  188. package/skills/evolve/SKILL.md +1 -0
  189. package/skills/gitlab-ops/SKILL.md +20 -12
  190. package/skills/gitlab-portfolio/SKILL.md +2 -2
  191. package/skills/hook-development/SKILL.md +1 -1
  192. package/skills/mode-selector/SKILL.md +1 -1
  193. package/skills/npm-publish/SKILL.md +97 -0
  194. package/skills/plan/SKILL.md +5 -5
  195. package/skills/plan/mode-feature.md +4 -4
  196. package/skills/plan/mode-new.md +10 -10
  197. package/skills/plan/mode-retro.md +1 -1
  198. package/skills/quality-gates/SKILL.md +1 -1
  199. package/skills/reconcile/SKILL.md +21 -4
  200. package/skills/session-end/SKILL.md +108 -14
  201. package/skills/session-end/discovery-scan.md +4 -2
  202. package/skills/session-end/drift-operations.md +4 -4
  203. package/skills/session-end/metrics-collection.md +13 -0
  204. package/skills/session-end/phase-3-2-docs-verification.md +1 -1
  205. package/skills/session-end/phase-3-6-tail.md +2 -1
  206. package/skills/session-end/plan-verification.md +5 -2
  207. package/skills/session-end/vault-operations.md +1 -1
  208. package/skills/session-end/verification-checklist.md +1 -1
  209. package/skills/session-plan/SKILL.md +6 -2
  210. package/skills/session-plan/wave-template.md +2 -0
  211. package/skills/session-start/SKILL.md +149 -7
  212. package/skills/session-start/phase-4-5-resource-health.md +15 -2
  213. package/skills/test-runner/SKILL.md +2 -2
  214. package/skills/vault-sync/SKILL.md +1 -1
  215. package/skills/vault-sync/package-lock.json +3 -3
  216. package/skills/vault-sync/validator.mjs +228 -40
  217. package/skills/wave-executor/SKILL.md +5 -2
  218. package/skills/wave-executor/circuit-breaker.md +2 -0
  219. package/skills/wave-executor/wave-loop.md +163 -10
  220. package/templates/_shared/loop.md +4 -4
@@ -75,6 +75,16 @@ stale-issue-days: 30 # issue-age threshold for triage flag
75
75
 
76
76
  Read by: `skills/session-start/SKILL.md`, `skills/ecosystem-health/SKILL.md`, `skills/gitlab-ops/SKILL.md`.
77
77
 
78
+ ## Auto-Skill Dispatch
79
+
80
+ ```yaml
81
+ auto-skill-dispatch: false # opt-in; phrase-match meta-skill — see skills/using-orchestrator/SKILL.md
82
+ ```
83
+
84
+ Top-level scalar (not nested). When `true`, entry-point skills invoke `skills/using-orchestrator/SKILL.md` once before their own Phase 1 to detect implicit slash-command intent in the user's first message (bilingual EN/DE phrase map, confidence-scored, AUQ-disambiguated on close ties). Default `false` — zero behavior change until opted in.
85
+
86
+ Read by: `skills/using-orchestrator/SKILL.md`, `skills/_shared/bootstrap-gate.md`.
87
+
78
88
  ## Quality Gates
79
89
 
80
90
  ```yaml
@@ -91,7 +101,7 @@ Read by: `skills/quality-gates/SKILL.md`, `skills/session-end/SKILL.md`. The thr
91
101
  ## Discovery
92
102
 
93
103
  ```yaml
94
- discovery-on-close: auto # session-type aware: housekeeping=false, feature/deep=true (#264)
104
+ discovery-on-close: auto # auto = true for every session type (was housekeeping=false until 2026-07-29)
95
105
  discovery-probes: [all] # all | code | infra | ui | arch | session | audit | vault | feature
96
106
  discovery-exclude-paths: [] # globs (e.g. "vendor/**", "dist/**")
97
107
  discovery-severity-threshold: low # critical | high | medium | low
@@ -101,6 +111,17 @@ discovery-parallelism: 5 # 1..16 probe agents in parallel
101
111
 
102
112
  Read by: `skills/discovery/SKILL.md` and the probe modules under `skills/discovery/probes-*.md`.
103
113
 
114
+ ## Issue Budget
115
+
116
+ ```yaml
117
+ issue-budget:
118
+ max-per-session: 12 # non-exempt issues one session may create (0 = block all)
119
+ mode: strict # strict | warn | off
120
+ overflow: collect-issue # collect-issue | vault-note
121
+ ```
122
+
123
+ Quantity cap, not a quality filter — `discovery-*-threshold` above cannot bound creation volume. `priority::critical`, the carryover class and `broken-window` issues are exempt. Read by: `hooks/pre-bash-issue-budget.mjs`, `scripts/lib/spiral-carryover.mjs`, `skills/session-end/SKILL.md` Phase 5 Step 3b.
124
+
104
125
  ## Persistence & Safety
105
126
 
106
127
  ```yaml
@@ -115,6 +136,12 @@ enforcement-gates:
115
136
  path-guard: true
116
137
  command-guard: true
117
138
  post-edit-validate: true
139
+ bash-write-verify: true # PostToolUse/Bash working-tree diff, warn-only (#915)
140
+ # bash-write-guard: true # ⚠ INVERTED DEFAULT — this is the ONE gate that is
141
+ # OFF when the key is absent. Every other key above
142
+ # means "enabled unless set to false"; this one runs
143
+ # only on a literal `true`. Leaving it out does NOT
144
+ # leave Bash writes guarded (#800/#915).
118
145
  allow-destructive-ops: false # disables destructive-command guard when true
119
146
  reasoning-output: false # opt-in STATE:/PLAN: agent transparency markers
120
147
  grounding-check: true # session-end Phase 1.1a planned-vs-touched diff
@@ -124,7 +151,9 @@ max-turns: auto # housekeeping=8, feature=15, deep=25
124
151
  auto-commit-per-wave: false # opt-in: commit after each wave's Quality-Lite PASS (default false; V3.6 plumbing)
125
152
  ```
126
153
 
127
- Read by: `skills/session-start/SKILL.md`, `skills/session-end/SKILL.md`, `hooks/pre-edit-scope.mjs`, `hooks/pre-bash-destructive-guard.mjs`, `hooks/pre-bash-enforce-commands.mjs`, `hooks/post-edit-validate.mjs`.
154
+ Read by: `skills/session-start/SKILL.md`, `skills/session-end/SKILL.md`, `hooks/pre-edit-scope.mjs`, `hooks/pre-bash-destructive-guard.mjs`, `hooks/pre-bash-enforce-commands.mjs`, `hooks/post-edit-validate.mjs`, `hooks/post-bash-write-verify.mjs`.
155
+
156
+ > **The `bash-write-guard` gate is the single inverted default in this block.** `hooks/enforce-commands.mjs` runs it only when `gates['bash-write-guard'] === true` is set literally; a missing key means the gate is OFF, unlike `path-guard`/`command-guard`/`post-edit-validate`/`bash-write-verify`, where a missing key means ON. Assuming otherwise is exactly the gap #915 was filed for — `hooks/enforce-scope.mjs` gates only `Edit`/`Write`/`MultiEdit`, so with `bash-write-guard` off nothing pre-checks the path of a `echo x > out-of-scope.mjs`. Full rationale (including the measured false-positive rate that justifies keeping it opt-in) in `docs/session-config-reference.md` § enforcement-gates.
128
157
 
129
158
  ## Resource Awareness (env-aware)
130
159
 
@@ -284,7 +313,7 @@ Read by: `scripts/lib/config/handover-gate.mjs`, `skills/session-end/SKILL.md` P
284
313
 
285
314
  ## Broken-Window Budget
286
315
 
287
- Opt-in gate in `/close` (session-end Phase 2.6). When enabled, it aggregates THIS session's "knowingly-broken shipments" (echo-stubs shipped under `enforcement: warn`, Phase 2.3/2.5 "Override and close" choices, MED/LOW findings routed to "Unresolved Review Findings", wave-level overridden findings) and files ONE hard-terminated closure issue per item (labels `broken-window` + `priority:high`, hard due-date). Non-blocking and idempotent (issue #730, Epic H / H5).
316
+ Opt-in gate in `/close` (session-end Phase 2.6). When enabled, it aggregates THIS session's "knowingly-broken shipments" (echo-stubs shipped under `enforcement: warn`, Phase 2.3/2.5 "Override and close" choices, MED/LOW findings routed to "Unresolved Review Findings", wave-level overridden findings) and files ONE hard-terminated closure issue per item (labels `broken-window` + `priority::high`, hard due-date). Non-blocking and idempotent (issue #730, Epic H / H5).
288
317
 
289
318
  ```yaml
290
319
  broken-window-budget:
@@ -466,6 +495,21 @@ docs-staleness:
466
495
  mode: warn # strict | warn | off
467
496
  thresholds:
468
497
  living: 90 # days — single tier; severity escalates at 1×/2×/3× threshold
498
+ moc-staleness:
499
+ # Parser gotcha: this key line must carry NO inline comment.
500
+ enabled: false # opt-in — <vault>/08-topics/*-moc.md staleness banner (session-start Phase 4)
501
+ thresholds:
502
+ moc: 90 # days — frontmatter `updated:` threshold; missing/unparseable is EXCLUDED, not reported
503
+ mode: warn # warn | off
504
+ context-coverage:
505
+ # Parser gotcha: this key line must carry NO inline comment.
506
+ enabled: false # opt-in — registered 01-projects/ folders lacking context.md AND _passive.md
507
+ mode: warn # warn | off
508
+ worktree-orphans:
509
+ # Parser gotcha: this key line must carry NO inline comment.
510
+ enabled: false # opt-in — session-end Phase 4b sweep; CANDIDATES ONLY, never auto-deletes (PSA-003)
511
+ base-branch: main # validated — a leading-dash value is rejected and falls back to main
512
+ mode: warn # warn | off
469
513
  ```
470
514
 
471
515
  Read by: `skills/discovery/probes/docs-staleness.mjs`, `scripts/lib/config/docs-staleness.mjs`.
@@ -665,6 +709,8 @@ enforcement-gates:
665
709
  path-guard: true
666
710
  command-guard: true
667
711
  post-edit-validate: true
712
+ bash-write-verify: true # PostToolUse/Bash working-tree diff, warn-only (#915)
713
+ # bash-write-guard: true # ⚠ INVERTED DEFAULT: absent = OFF (opposite of the keys above)
668
714
  allow-destructive-ops: false
669
715
  reasoning-output: false
670
716
  grounding-check: true
@@ -820,6 +866,18 @@ docs-staleness:
820
866
  mode: warn
821
867
  thresholds:
822
868
  living: 90
869
+ moc-staleness:
870
+ enabled: false
871
+ thresholds:
872
+ moc: 90
873
+ mode: warn
874
+ context-coverage:
875
+ enabled: false
876
+ mode: warn
877
+ worktree-orphans:
878
+ enabled: false
879
+ base-branch: main
880
+ mode: warn
823
881
 
824
882
  # CLAUDE.md drift check
825
883
  drift-check:
@@ -867,6 +925,11 @@ loop-guard:
867
925
  instruction-budget:
868
926
  enabled: true # always-on directive-budget banner; off-by-config silences it
869
927
  ceiling: 480 # structural-directive ceiling (baseline ~457; ratchet guards growth)
928
+ byte-ceiling: 114000 # byte ceiling for the SAME corpus (baseline ~108589, +5% headroom —
929
+ # deliberately the same relative slack the directive ceiling carries,
930
+ # so neither axis is accidentally the stricter one). Either axis alone
931
+ # puts the banner over budget: a 9 KB prose rule with three bullets is
932
+ # invisible to the directive count but costs real payload (#931a).
870
933
  mode: warn # warn (surface banner) | off (silent no-op)
871
934
 
872
935
  # Config-protection guard (ecc-analysis / #622)
@@ -0,0 +1,204 @@
1
+ # Telemetry Claims — Provenance & Methodology
2
+
3
+ > **Not the opt-in usage-telemetry client.** This document explains the
4
+ > methodology behind the maintainer's **local, private** metrics aggregates
5
+ > (`.orchestrator/metrics/*.jsonl`, gitignored by design) used in marketing
6
+ > copy such as "645 orchestrated sessions." It is a separate data flow from
7
+ > the plugin's optional, strictly opt-in anonymous usage-telemetry client —
8
+ > see [docs/telemetry.md](../telemetry.md) for what that client collects,
9
+ > its kill switches, and where the data goes.
10
+
11
+ > Cross-repo aggregate over the maintainer's private session corpus as of 2026-06.
12
+ > Not independently auditable: the per-session records this aggregate is computed
13
+ > from are gitignored (privacy by default). This is a **snapshot, not a live
14
+ > counter** — re-running the math on a later date yields different numbers.
15
+
16
+ _As-of: 2026-06 (maintainer-reported)_
17
+
18
+ This document explains the telemetry figures used in public-facing material (e.g.
19
+ agenticbuilders.at) so the claim is verifiable in *method* even though the
20
+ absolute owner numbers cannot be reproduced from any public artifact. No marketing
21
+ copy lives here — only what each number means, where it comes from, and how it is
22
+ derived.
23
+
24
+ ---
25
+
26
+ ## Headline claim
27
+
28
+ > **645 orchestrated sessions · 1,680 documented learnings · 7,700+ agent runs
29
+ > (98.8% cleanly completed) · up to 34 parallel agents · 17 repos · as of 2026-06**
30
+
31
+ Every figure above is a **cross-repo aggregate** computed over the local,
32
+ **gitignored** JSONL metric files of several of the maintainer's private repos:
33
+
34
+ ```
35
+ .orchestrator/metrics/sessions.jsonl
36
+ .orchestrator/metrics/learnings.jsonl
37
+ .orchestrator/metrics/subagents.jsonl
38
+ .orchestrator/metrics/events.jsonl
39
+ ```
40
+
41
+ These files are listed in `.gitignore` by design (privacy by default). No public
42
+ commit, release, or CI artifact contains them, so the headline totals are
43
+ **maintainer-reported** and not independently auditable. What *is* reproducible is
44
+ the measurement recipe — see [Reproduce it yourself](#reproduce-it-yourself).
45
+
46
+ ---
47
+
48
+ ## Claims table
49
+
50
+ | Claim | Source file | What one record means | How aggregated | As-of |
51
+ | --- | --- | --- | --- | --- |
52
+ | 645 orchestrated sessions | `sessions.jsonl` | 1 line = 1 complete `/session` -> `/close` cycle | `wc -l` summed across all contributing repos | 2026-06 |
53
+ | 1,680 documented learnings | `learnings.jsonl` | 1 line = 1 extracted, confidence-scored learning (written by `/evolve`) | `wc -l` summed across repos | 2026-06 |
54
+ | 7,700+ agent runs | `subagents.jsonl` | 1 dispatched subagent run (see note below on `start`/`stop` pairing) | `wc -l` (or `start`-event count) summed across repos | 2026-06 |
55
+ | 98.8% cleanly completed | `subagents.jsonl` (+ `events.jsonl`) | completion status of a dispatched run | completed runs / total dispatched runs — *maintainer-reported, see methodology* | 2026-06 |
56
+ | up to 34 parallel agents | `subagents.jsonl` / `events.jsonl` | peak count of agents dispatched concurrently | max over time-windowed `start`/`stop` overlap | 2026-06 |
57
+ | 17 repos | n/a (corpus scope) | a repo that contributed at least one `sessions.jsonl` line | count of contributing repos | 2026-06 |
58
+
59
+ ---
60
+
61
+ ## In-repo test suite — the "10,000+ tests" badge
62
+
63
+ Distinct from the private-corpus figures above, the README **Tests** badge and the
64
+ "10,000+ vitest tests run on every commit" line count **this repository's own test
65
+ suite** — a **public, CI-verifiable** number, not a maintainer-reported aggregate.
66
+
67
+ | Claim | What it counts | How measured | As-of |
68
+ | --- | --- | --- | --- |
69
+ | 10,000+ vitest tests | executed test cases across `tests/**/*.test.mjs` | `npm test` prints the exact runtime total; the static floor is countable without running the suite (below) | 2026-06 |
70
+
71
+ Both numbers reproduce in a fresh checkout:
72
+
73
+ ```bash
74
+ find tests -name '*.test.mjs' | wc -l # test files -> 475
75
+ grep -rohE '\b(it|test)\(' tests | wc -l # static test defs -> ~9,871
76
+ grep -rohE '\b(it|test|describe)\.each\b' tests | wc -l # parameterized blocks -> 93
77
+ ```
78
+
79
+ The static `it(` / `test(` count (~9,871 across 475 files) is a **floor**: the 93
80
+ `it.each` / `test.each` parameterized blocks each expand to multiple executed cases
81
+ at runtime, so the **case count vitest reports on `npm test` is 10,000+**. Unlike the
82
+ private-corpus figures above, this one is fully auditable — run `npm test` in this
83
+ checkout and read vitest's summary line.
84
+
85
+ ## Methodology
86
+
87
+ ### Session / learning / agent-run counts
88
+
89
+ Each total is the sum of `wc -l` over the corresponding JSONL file across every
90
+ contributing repo:
91
+
92
+ ```bash
93
+ # per repo, then summed by hand across the corpus
94
+ wc -l .orchestrator/metrics/sessions.jsonl # -> orchestrated sessions
95
+ wc -l .orchestrator/metrics/learnings.jsonl # -> documented learnings
96
+ wc -l .orchestrator/metrics/subagents.jsonl # -> agent-run records
97
+ ```
98
+
99
+ The JSONL format is append-only (one JSON object per line), so a line count is a
100
+ faithful event count. There is no de-duplication step across repos — each repo
101
+ contributes its own disjoint slice.
102
+
103
+ ### "Up to 34 parallel agents"
104
+
105
+ This is the **peak** number of agents dispatched at the same time, not an average.
106
+ It is read off the time windows in `subagents.jsonl` / `events.jsonl`: each agent
107
+ run has a `start` and a `stop` record with timestamps; the peak is the maximum
108
+ number of runs whose `[start, stop]` intervals overlap at any instant. Structurally
109
+ this equals `agents-per-wave x concurrently-running waves` at the busiest moment of
110
+ the busiest session. The session config governing fan-out (`agents-per-wave`,
111
+ `waves`) lives in each repo's `CLAUDE.md` Session Config block.
112
+
113
+ > **Note (2026-07-03, refs #724):** the small-batch dispatch default (3–4 `Agent()`
114
+ > calls per message, `wave-loop.md § Dispatch Agents`) structurally lowers future
115
+ > instantaneous peaks — agents within a wave now start in staggered batches rather
116
+ > than a single simultaneous fan-out, so the overlapping-interval peak trends below
117
+ > the historical `agents-per-wave × waves` ceiling this figure was read off.
118
+
119
+ ### "98.8% cleanly completed"
120
+
121
+ Defined as **runs that completed cleanly / all dispatched runs**, where a clean
122
+ completion is a run that reached its terminal `stop` with a `complete` status (as
123
+ opposed to `spiral`- or `failed`-classified outcomes the orchestrator records for
124
+ runs that loop or error out). This ratio is **maintainer-reported**: it is computed
125
+ over the owner's full private corpus, and the exact per-run status classification
126
+ is not present in every repo's local slice.
127
+
128
+ > **Caveat (verified against this repo's own slice):** in *this* public repo,
129
+ > `subagents.jsonl` records only `event: "start"` / `event: "stop"` markers (no
130
+ > per-run `status` field). The completion percentage therefore cannot be
131
+ > re-derived from this repo alone; it is an attribute of the owner's larger
132
+ > private corpus and is labelled maintainer-reported accordingly.
133
+
134
+ ---
135
+
136
+ ## Reproduce it yourself
137
+
138
+ Skeptics cannot reproduce the *owner's absolute totals* (the source records are
139
+ gitignored), but the **measurement pattern is fully reproducible** on your own
140
+ machine. After running N of your own sessions:
141
+
142
+ ```bash
143
+ wc -l .orchestrator/metrics/sessions.jsonl # your orchestrated sessions
144
+ wc -l .orchestrator/metrics/learnings.jsonl # your documented learnings
145
+ wc -l .orchestrator/metrics/subagents.jsonl # your agent runs
146
+ ```
147
+
148
+ Each command returns *your* numbers using the exact same definitions in the claims
149
+ table above. As a worked example, this very repo's local slice at the time of
150
+ writing reports:
151
+
152
+ ```
153
+ 29 .orchestrator/metrics/sessions.jsonl
154
+ 97 .orchestrator/metrics/learnings.jsonl
155
+ 1796 .orchestrator/metrics/subagents.jsonl
156
+ ```
157
+
158
+ (maintainer-reported, verifiable by running the commands above in this checkout).
159
+ The point is that the *method* is transparent and runs identically everywhere; only
160
+ the maintainer's aggregate magnitude stays private.
161
+
162
+ ---
163
+
164
+ ## Framing & limits
165
+
166
+ These figures are the **maintainer's own telemetry**. They are **not** Anthropic
167
+ benchmarks, not Anthropic-published limits, and not a claim about what Claude or any
168
+ model "can do" in general. They describe how this orchestrator was operated across
169
+ one maintainer's private repos.
170
+
171
+ In particular, **Anthropic does not document a hard parallel-agent limit**; the
172
+ "up to 34 parallel agents" figure is an observed peak of *this* tooling's fan-out,
173
+ not a platform ceiling. For readers who want vendor-side context on multi-agent
174
+ cost and design (cited only for orientation, not as a source for the numbers
175
+ above):
176
+
177
+ - Multi-agent token cost — "agents typically use about 4x more tokens than chat
178
+ interactions, and multi-agent systems use about 15x more tokens." Anthropic,
179
+ *How we built our multi-agent research system* (2025-06-13).
180
+ <https://www.anthropic.com/engineering/built-multi-agent-research-system>
181
+ - The orchestrator-workers pattern this tool implements is described in Anthropic,
182
+ *Building effective agents* (2024-12-19).
183
+ <https://www.anthropic.com/engineering/building-effective-agents>
184
+
185
+ No other Anthropic claims are made or implied here.
186
+
187
+ ---
188
+
189
+ ## Privacy
190
+
191
+ What is **never published** from the source corpus:
192
+
193
+ - Absolute filesystem paths
194
+ - Hostnames (`.local` / `.lan` / `.internal`, etc.)
195
+ - Repository names and VCS URLs (org/repo paths)
196
+ - Prompts, session contents, or any free-form text
197
+ - Emails, git-author identities, tokens, IP addresses
198
+
199
+ Only aggregate counts and definitions leave the private corpus. This mirrors the
200
+ anonymization pattern already enforced for shared learnings by
201
+ `scripts/export-hw-learnings.mjs` (see also hardware-patterns; vault-archived; regenerated via `npm run share:hw-learnings`),
202
+ which strips paths, IPs, VCS URLs, hostnames, emails, git authors, and token-shaped
203
+ strings, and exports only structured fields — never free-form user text. The same
204
+ "counts out, records stay home" principle applies to every figure in this document.
@@ -0,0 +1,158 @@
1
+ # Telemetry
2
+
3
+ Session Orchestrator ships an **optional, strictly opt-in** anonymous
4
+ usage-telemetry client. This page is the transparency contract: exactly what
5
+ is collected, what is never collected, how consent works, every kill switch,
6
+ where the data goes, and how long it is kept. Nothing here is aspirational —
7
+ it is the locked v1 schema and consent precedence this plugin implements.
8
+
9
+ ## TL;DR
10
+
11
+ - **Off by default.** Nothing is sent until you explicitly consent.
12
+ - **One prompt, ever.** An interactive session asks once, at most; the
13
+ answer is saved locally and never asked again (until you reset it).
14
+ - **Trivial to turn off**, at any time, with an environment variable or a
15
+ one-line CLI command — no restart, no config-file archaeology.
16
+ - **No CI/headless sends, ever.** Non-interactive sessions never prompt and
17
+ never send, regardless of prior consent state.
18
+ - **Fully open source.** The client code and the ingest-server code both
19
+ live in this repository — nothing is a black box.
20
+ - **Not the same thing as the local metrics used in marketing claims.** See
21
+ [Relationship to `telemetry-claims.md`](#relationship-to-telemetry-claimsmd)
22
+ below.
23
+
24
+ ## What we collect
25
+
26
+ When telemetry is enabled and a batch is flushed, the payload is built from
27
+ a strict field whitelist — nothing outside this list is ever included, and a
28
+ projection unit test enforces the drop of any non-whitelisted input field.
29
+
30
+ | Field | Meaning |
31
+ |---|---|
32
+ | `record_kind` | Always `"usage-ping"` for this record type. |
33
+ | `schema_version` | Currently `1`. Additive-only evolution within a version — see [Schema evolution](#schema-evolution). |
34
+ | `anon_id` | A random UUID, not derived from any machine identifier. Rotates every 90 days; the old ID is discarded, not linked to the new one. |
35
+ | `sent_at` | Timestamp of the flush. |
36
+ | `plugin_version` | The installed plugin's semver. |
37
+ | `platform` | One of `claude`, `codex`, `cursor`, `pi`, `other`. |
38
+ | `os` | Operating system family (e.g. `darwin`, `linux`, `win32`). |
39
+ | `arch` | CPU architecture (e.g. `arm64`, `x64`). |
40
+ | `node_major` | Major Node.js version in use. |
41
+ | `ci` | Boolean — whether the run was detected as a CI environment. |
42
+ | `fleet` | Boolean — whether this send came from an operator's own fleet-mode host (`owner.yaml` opt-in), as opposed to an external install. |
43
+ | `session_type` | One of `housekeeping`, `feature`, `deep`, `other`. |
44
+ | `duration_bucket` | One of `<15m`, `15-60m`, `1-3h`, `>3h` — a coarse bucket, never an exact duration. |
45
+ | `skills[]` | Names of invoked skills, filtered against the shipped plugin roster — any name not in that roster becomes `"other"`. |
46
+ | `commands[]` | Same filtering rule as `skills[]`. |
47
+
48
+ ## What we never collect
49
+
50
+ This list is a hard invariant, not a deferral:
51
+
52
+ - No repository names, no file paths, no git remotes.
53
+ - No prompts, no session transcripts, no free-form text of any kind.
54
+ - No command arguments — only whitelisted command/skill *names*, and only
55
+ from the shipped roster (anything else is reduced to `"other"`).
56
+ - No hostnames.
57
+ - No IP addresses stored. The ingest server uses the requester's IP
58
+ **transiently, in memory, only** to enforce a per-IP rate limit — it is
59
+ never written to disk, and access logging is disabled on the telemetry
60
+ vhost.
61
+ - No email addresses, no git author identity, no account identifiers.
62
+
63
+ If a skill or command name isn't part of the plugin's own shipped roster —
64
+ including any custom or third-party skill you've added locally — it never
65
+ leaves your machine; it is projected to `"other"` before the payload is
66
+ built.
67
+
68
+ ## Consent & kill switches
69
+
70
+ Precedence, highest wins:
71
+
72
+ 1. **`DO_NOT_TRACK`** — any non-empty value except `0`/`false` disables
73
+ telemetry unconditionally. This is the industry-standard signal and
74
+ overrides everything else, including a fleet force-enable.
75
+ 2. **`SO_TELEMETRY_DISABLED=1`** — explicit per-shell disable.
76
+ 3. **`SO_TELEMETRY=1`** — explicit per-shell force-enable (used for fleet
77
+ testing without touching the consent file).
78
+ 4. **`owner.yaml` `telemetry.enabled`** — host-local fleet-mode opt-in (see
79
+ below); has no effect on a machine without that file.
80
+ 5. **Saved consent** — `~/.config/session-orchestrator/telemetry.json`,
81
+ written the first time you answer the consent prompt. Never inside any
82
+ repository, never committed.
83
+ 6. **First-run prompt** — shown at most once, **interactively only**. A
84
+ headless or CI invocation never shows this prompt and never sends
85
+ telemetry, regardless of any saved state.
86
+
87
+ If the consent file is corrupt or unreadable, the client fails **closed**:
88
+ telemetry state degrades to "no consent" (nothing sent) rather than
89
+ guessing, with a one-line stderr hint pointing at the CLI below.
90
+
91
+ **CLI:**
92
+
93
+ ```bash
94
+ node scripts/telemetry.mjs status # show current consent + kill-switch state
95
+ node scripts/telemetry.mjs enable # opt in
96
+ node scripts/telemetry.mjs disable # opt out
97
+ node scripts/telemetry.mjs show # print the last built payload, don't send
98
+ ```
99
+
100
+ **Debug flag:** set `SO_TELEMETRY_DEBUG=1` to print the exact payload that
101
+ *would* be sent to stderr instead of sending it — useful for verifying the
102
+ whitelist projection yourself before ever trusting it.
103
+
104
+ **Fleet mode.** An operator running many repos on one host can set
105
+ `telemetry.enabled: true` (and optionally `telemetry.fleet: true`) in their
106
+ own `owner.yaml` — a host-local, never-committed file outside every repo —
107
+ to enable telemetry across all adopted repos without a per-repo prompt.
108
+ Records sent this way carry `fleet: true`. `DO_NOT_TRACK` and
109
+ `SO_TELEMETRY_DISABLED=1` still win over fleet mode in the same shell.
110
+
111
+ ## Where it goes
112
+
113
+ Consented payloads are sent as a batched `POST` to:
114
+
115
+ ```
116
+ https://telemetry.session-orchestrator.com/v1/records
117
+ ```
118
+
119
+ This endpoint is operated by the plugin's maintainer. The server-side code
120
+ is open source in this same repository, under `server/ingest/` — a
121
+ dependency-light Node service that validates the payload against the
122
+ per-`record_kind` schema, stores it in SQLite, and rejects anything that
123
+ doesn't fit the schema (unknown `record_kind`, oversized body, or a schema
124
+ violation). There is no third-party analytics vendor in this path — no
125
+ Segment, no Mixpanel, no Google Analytics.
126
+
127
+ The send is fire-and-forget with a short timeout; if the endpoint is
128
+ unreachable, the batch queues locally (bounded size, oldest entries dropped
129
+ first) and retries later. Telemetry never blocks or slows down a session
130
+ beyond that short timeout budget.
131
+
132
+ ## Retention
133
+
134
+ - **Raw records:** kept 24 months, then pruned. The retention window exists
135
+ to support year-over-year product decisions (what to deepen, what to
136
+ sunset) without keeping data indefinitely.
137
+ - **Aggregates:** kept indefinitely; aggregates carry no record-level
138
+ identifiers by construction.
139
+ - **Anonymous ID rotation:** every 90 days, independent of retention — a
140
+ rotated ID cannot be linked back to the one it replaced.
141
+
142
+ ## Schema evolution
143
+
144
+ The schema is **additive-only** within a given `schema_version`: new
145
+ optional fields may appear, but no field is ever repurposed or removed
146
+ without a version bump. The server accepts both the current and the
147
+ immediately previous `schema_version`, so a slightly-outdated client is
148
+ never hard-broken by a server-side schema update.
149
+
150
+ ## Relationship to `telemetry-claims.md`
151
+
152
+ This page describes the **opt-in, client-side usage-telemetry pipeline**
153
+ above. It is a distinct data flow from
154
+ [`docs/telemetry/telemetry-claims.md`](telemetry/telemetry-claims.md), which
155
+ documents the methodology behind the maintainer's separate **local, private**
156
+ metrics aggregates (`.orchestrator/metrics/*.jsonl`, gitignored, never
157
+ transmitted anywhere) used in marketing claims such as "645 orchestrated
158
+ sessions." Neither pipeline feeds the other.