@luminascale/shepherd 0.0.0-stage → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (203) hide show
  1. package/.agents/agents/README.md +12 -0
  2. package/.agents/agents/planner.md +38 -0
  3. package/.agents/agents/reviewer.md +38 -0
  4. package/.agents/agents/scout.md +43 -0
  5. package/.agents/agents/worker.md +33 -0
  6. package/.agents/skills/smoke-test/SKILL.md +155 -0
  7. package/.agents/skills/smoke-test/references/forensics.md +46 -0
  8. package/.agents/skills/smoke-test/scripts/live-smoke-poll.mjs +130 -0
  9. package/.github/workflows/publish.yml +77 -0
  10. package/.pi/prompts/release.md +21 -0
  11. package/.prettierrc +10 -0
  12. package/AGENTS.md +88 -0
  13. package/CONTRIBUTING.md +177 -0
  14. package/LICENSE +201 -0
  15. package/README.md +424 -2
  16. package/docs/README.md +25 -0
  17. package/docs/guides/agent-definitions.md +42 -0
  18. package/docs/guides/diagnostics.md +199 -0
  19. package/docs/guides/tool-reference.md +55 -0
  20. package/docs/manual-tests/ping-pong-peer-messaging.md +147 -0
  21. package/docs/plans/agent-labels.md +257 -0
  22. package/docs/plans/architecture.md +110 -0
  23. package/docs/plans/settings-scope.md +163 -0
  24. package/docs/plans/tool-surface-split.md +278 -0
  25. package/docs/reference/dictionary +26 -0
  26. package/docs/releasing.md +47 -0
  27. package/docs/specs/005-agent-messaging-and-task-lifecycle/baseline.md +115 -0
  28. package/docs/specs/005-agent-messaging-and-task-lifecycle/handoff.md +564 -0
  29. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase10-notes.md +109 -0
  30. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase6-notes.md +69 -0
  31. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase7-notes.md +100 -0
  32. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase8-notes.md +127 -0
  33. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase9-notes.md +104 -0
  34. package/docs/specs/005-agent-messaging-and-task-lifecycle/plan.md +1093 -0
  35. package/docs/specs/005-agent-messaging-and-task-lifecycle/spec.md +583 -0
  36. package/docs/specs/005-agent-messaging-and-task-lifecycle/tasks.md +838 -0
  37. package/docs/specs/006-shepherd-tool-rendering/plan.md +175 -0
  38. package/docs/specs/006-shepherd-tool-rendering/spec.md +469 -0
  39. package/index.ts +599 -0
  40. package/package.json +112 -4
  41. package/scripts/capture-system-prompt.mjs +45 -0
  42. package/scripts/extract-pi-system-prompt.mjs +277 -0
  43. package/scripts/host-impact.mjs +74 -0
  44. package/scripts/resolve-agent.mjs +43 -0
  45. package/scripts/show-shepherd-orchestrator.mjs +58 -0
  46. package/scripts/show-shepherd-prompt.mjs +96 -0
  47. package/scripts/verify-suites.mjs +67 -0
  48. package/specs/001-artifacts/spec.md +436 -0
  49. package/specs/002-simplify-agent-orchestration/plan.md +287 -0
  50. package/specs/002-simplify-agent-orchestration/spec.md +323 -0
  51. package/specs/002-simplify-agent-orchestration/tasks.md +212 -0
  52. package/specs/003-orchestrator-session-artifacts/plan.md +396 -0
  53. package/specs/003-orchestrator-session-artifacts/spec.md +453 -0
  54. package/specs/003-orchestrator-session-artifacts/tasks.md +141 -0
  55. package/specs/004-asynchronous-prompt-watchers/plan.md +139 -0
  56. package/specs/004-asynchronous-prompt-watchers/spec.md +202 -0
  57. package/specs/004-asynchronous-prompt-watchers/tasks.md +126 -0
  58. package/specs/005-per-workspace-settings/plan.md +230 -0
  59. package/specs/005-per-workspace-settings/spec.md +368 -0
  60. package/specs/005-per-workspace-settings/tasks.md +178 -0
  61. package/specs/006-dual-runtime-herdr-durable/intent.md +86 -0
  62. package/specs/006-dual-runtime-herdr-durable/journal.md +80 -0
  63. package/specs/006-dual-runtime-herdr-durable/plan.md +242 -0
  64. package/specs/006-dual-runtime-herdr-durable/spec.md +389 -0
  65. package/specs/006-dual-runtime-herdr-durable/tasks.md +199 -0
  66. package/specs/007-unwatched-completion-notice/journal.md +145 -0
  67. package/specs/007-unwatched-completion-notice/plan.md +186 -0
  68. package/specs/007-unwatched-completion-notice/spec.md +284 -0
  69. package/specs/007-unwatched-completion-notice/tasks.md +147 -0
  70. package/specs/008-remote-machine-sheep/intent.md +174 -0
  71. package/specs/008-remote-machine-sheep/journal.md +57 -0
  72. package/specs/008-remote-machine-sheep/prerequisites.md +219 -0
  73. package/specs/009-hc-runtime/intent.md +44 -0
  74. package/specs/009-hc-runtime/plan.md +49 -0
  75. package/specs/009-hc-runtime/spec.md +118 -0
  76. package/specs/009-hc-runtime/tasks.md +62 -0
  77. package/specs/010-unseen-watch-parity/intent.md +57 -0
  78. package/specs/010-unseen-watch-parity/journal.md +172 -0
  79. package/specs/010-unseen-watch-parity/plan.md +74 -0
  80. package/specs/010-unseen-watch-parity/spec.md +191 -0
  81. package/specs/010-unseen-watch-parity/tasks.md +119 -0
  82. package/specs/011-hc-background-poll/intent.md +57 -0
  83. package/specs/011-hc-background-poll/journal.md +95 -0
  84. package/specs/011-hc-background-poll/plan.md +87 -0
  85. package/specs/011-hc-background-poll/spec.md +141 -0
  86. package/specs/011-hc-background-poll/tasks.md +83 -0
  87. package/specs/012-steer-default/intent.md +12 -0
  88. package/specs/012-steer-default/journal.md +40 -0
  89. package/specs/012-steer-default/plan.md +7 -0
  90. package/specs/012-steer-default/spec.md +42 -0
  91. package/specs/012-steer-default/tasks.md +6 -0
  92. package/specs/013-opencode-shepherd/handoff-t011.md +94 -0
  93. package/specs/013-opencode-shepherd/intent.md +59 -0
  94. package/specs/013-opencode-shepherd/plan.md +204 -0
  95. package/specs/013-opencode-shepherd/spec.md +258 -0
  96. package/specs/013-opencode-shepherd/tasks.md +259 -0
  97. package/src/core/artifact-sessions.ts +835 -0
  98. package/src/core/config.ts +329 -0
  99. package/src/core/discovery.ts +342 -0
  100. package/src/core/herdr.ts +1 -0
  101. package/src/core/host.ts +77 -0
  102. package/src/core/ids.ts +78 -0
  103. package/src/core/lifecycle.ts +1 -0
  104. package/src/core/messaging.ts +1 -0
  105. package/src/core/migration.ts +55 -0
  106. package/src/core/orchestration.ts +1 -0
  107. package/src/core/types.ts +143 -0
  108. package/src/core/unseen-completions.ts +114 -0
  109. package/src/extension/cli.ts +417 -0
  110. package/src/extension/config.ts +1 -0
  111. package/src/extension/settings-ui.ts +296 -0
  112. package/src/extension/shepherd-done.ts +768 -0
  113. package/src/extension/shepherd.ts +2548 -0
  114. package/src/extension/system-prompt.ts +38 -0
  115. package/src/extension/unseen-completions.ts +237 -0
  116. package/src/opencode/actions.ts +794 -0
  117. package/src/opencode/bridge.ts +777 -0
  118. package/src/opencode/display.ts +36 -0
  119. package/src/opencode/fieldnotes.ts +60 -0
  120. package/src/opencode/host.ts +62 -0
  121. package/src/opencode/orphans.ts +162 -0
  122. package/src/opencode/server.ts +458 -0
  123. package/src/runtimes/backend.ts +119 -0
  124. package/src/runtimes/detect.ts +102 -0
  125. package/src/runtimes/hc/hc.ts +527 -0
  126. package/src/runtimes/hc/runtime.ts +1898 -0
  127. package/src/runtimes/herdr/herdr.ts +912 -0
  128. package/src/runtimes/herdr/lifecycle.ts +1712 -0
  129. package/src/runtimes/herdr/messaging.ts +703 -0
  130. package/src/runtimes/herdr/orchestration.ts +1583 -0
  131. package/src/runtimes/herdr/runtime.ts +224 -0
  132. package/src/runtimes/runtime.ts +133 -0
  133. package/src/runtimes/settle-observations.ts +114 -0
  134. package/test/fixtures/context-project/AGENTS.md +2 -0
  135. package/test/fixtures/home/.agents/agents/same.md +6 -0
  136. package/test/fixtures/home/.agents/agents/shared-only.md +6 -0
  137. package/test/fixtures/home/.pi/agent/agents/invalid-thinking.md +6 -0
  138. package/test/fixtures/home/.pi/agent/agents/model-empty.md +7 -0
  139. package/test/fixtures/home/.pi/agent/agents/model-explicit.md +7 -0
  140. package/test/fixtures/home/.pi/agent/agents/omit-context-absent.md +5 -0
  141. package/test/fixtures/home/.pi/agent/agents/omit-context-false.md +6 -0
  142. package/test/fixtures/home/.pi/agent/agents/omit-context-invalid.md +6 -0
  143. package/test/fixtures/home/.pi/agent/agents/omit-context-true.md +6 -0
  144. package/test/fixtures/home/.pi/agent/agents/omit-false.md +6 -0
  145. package/test/fixtures/home/.pi/agent/agents/omit-invalid.md +6 -0
  146. package/test/fixtures/home/.pi/agent/agents/omit-pi-docs.md +7 -0
  147. package/test/fixtures/home/.pi/agent/agents/omit-true.md +6 -0
  148. package/test/fixtures/home/.pi/agent/agents/only-user.md +9 -0
  149. package/test/fixtures/home/.pi/agent/agents/same.md +6 -0
  150. package/test/fixtures/proj/.agents/agents/proj-shared.md +6 -0
  151. package/test/fixtures/proj/.pi/agents/project-pi.md +6 -0
  152. package/test/fixtures/proj/.pi/agents/same.md +6 -0
  153. package/test/helpers/fake-clock.mjs +39 -0
  154. package/test/helpers/feature-contracts.mjs +44 -0
  155. package/test/helpers/test-utils.mjs +44 -0
  156. package/test/verify-artifact-sessions.mjs +258 -0
  157. package/test/verify-backend.mjs +297 -0
  158. package/test/verify-child-mislabel.mjs +135 -0
  159. package/test/verify-child-surface.mjs +321 -0
  160. package/test/verify-cli.mjs +197 -0
  161. package/test/verify-command-ux.mjs +100 -0
  162. package/test/verify-compat.mjs +32 -0
  163. package/test/verify-cross-host-notes.mjs +128 -0
  164. package/test/verify-delegate.mjs +161 -0
  165. package/test/verify-detect.mjs +151 -0
  166. package/test/verify-discovery.mjs +239 -0
  167. package/test/verify-done-nudge.mjs +238 -0
  168. package/test/verify-effective-prompt.mjs +98 -0
  169. package/test/verify-hc-matrix.mjs +246 -0
  170. package/test/verify-hc-poll.mjs +260 -0
  171. package/test/verify-host-docs.mjs +35 -0
  172. package/test/verify-host-impact.mjs +49 -0
  173. package/test/verify-ids.mjs +141 -0
  174. package/test/verify-launch.mjs +183 -0
  175. package/test/verify-message-routing.mjs +560 -0
  176. package/test/verify-messaging.mjs +303 -0
  177. package/test/verify-migration-notice.mjs +69 -0
  178. package/test/verify-multi-wait.mjs +42 -0
  179. package/test/verify-opencode-display.mjs +47 -0
  180. package/test/verify-opencode-fieldnotes.mjs +121 -0
  181. package/test/verify-opencode-notice.mjs +77 -0
  182. package/test/verify-opencode-nudge.mjs +116 -0
  183. package/test/verify-opencode-orphans.mjs +160 -0
  184. package/test/verify-opencode-surface.mjs +107 -0
  185. package/test/verify-packaging.mjs +82 -0
  186. package/test/verify-parent-surface.mjs +118 -0
  187. package/test/verify-phase0.mjs +95 -0
  188. package/test/verify-registry.mjs +124 -0
  189. package/test/verify-rename-migration.mjs +111 -0
  190. package/test/verify-rendering.mjs +379 -0
  191. package/test/verify-session-boundary.mjs +146 -0
  192. package/test/verify-settings.mjs +260 -0
  193. package/test/verify-stale-wait.mjs +339 -0
  194. package/test/verify-status-widget.mjs +377 -0
  195. package/test/verify-status.mjs +172 -0
  196. package/test/verify-task-completion.mjs +156 -0
  197. package/test/verify-task-failures.mjs +201 -0
  198. package/test/verify-task-mislabel.mjs +143 -0
  199. package/test/verify-task-registry.mjs +194 -0
  200. package/test/verify-task-watchers.mjs +217 -0
  201. package/test/verify-tool-schemas.mjs +120 -0
  202. package/test/verify-unseen-completions.mjs +1211 -0
  203. package/test/verify-watchers.mjs +181 -0
@@ -0,0 +1,191 @@
1
+ # Spec — Unseen Watch Parity (herdr pane + hc)
2
+
3
+ ## Status
4
+
5
+ Proposed. No behavior change yet. Continuation of 009 (hc runtime),
6
+ converging the hc settle path onto the 007 (unwatched completion notice)
7
+ contract. Spec-only: no `src` changes in this spec; implementation follows
8
+ one worker per phase (see `plan.md`).
9
+
10
+ ## Background: the gap
11
+
12
+ 007 built the guarantee from three parts:
13
+
14
+ - `UnseenCompletionLedger` (`src/core/unseen-completions.ts`) + singleton
15
+ `unseenCompletionLedger` — queue of un-noticed settlements.
16
+ - Registry settlement observer — `onTaskSettled(listener)` in
17
+ `src/runtimes/herdr/orchestration.ts`, firing `(result, { hadWatchers,
18
+ agent?, label?, source? })` after the watcher fan-out.
19
+ - Notice bridge (`src/extension/unseen-completions.ts`) —
20
+ `configureUnseenCompletionBridge(pi, { coalesceMs?, ledger? })`:
21
+ subscribes to the observer on `session_start`, tracks busy/idle
22
+ (`agent_start` / `agent_settled`, preferring `ctx.isIdle()`), delivers
23
+ at the `agent_before_settle` boundary (`{ entries: [entry], continue:
24
+ true }` on `completed` runs only) or via one debounced
25
+ `pi.sendMessage(notice, { triggerTurn: true })` after
26
+ `UNSEEN_NOTICE_COALESCE_MS = 250`.
27
+
28
+ The hc runtime (`src/runtimes/hc/runtime.ts`, `settleTask` ~line 812)
29
+ currently bypasses the observer: it calls `notifyHcTaskWatchers(result)`
30
+ for its own fan-out and then records **directly** into the shared ledger.
31
+ Consequences:
32
+
33
+ 1. No idle wake is scheduled for hc settles — `scheduleIdleWake()` only
34
+ runs inside the bridge's Herdr-observer callback. An hc settle while
35
+ idle records an entry that waits for the next `agent_settled` tick or
36
+ boundary instead of one coalesced wake.
37
+ 2. No `source` tag — the `delegate-submission-failure` exclusion (007
38
+ §1/§2: a delegate call that fails before the child ever receives the
39
+ task settles `failed` but is already visible in the `shepherd_delegate`
40
+ tool result) is not passed through on hc, risking a false-positive
41
+ notice for an error the Shepherd already saw.
42
+ 3. The bridge subscribes to exactly one registry
43
+ (`lifecycleRegistry.onTaskSettled` from
44
+ `src/runtimes/herdr/orchestration.ts`); the hc settle point has no
45
+ `onTaskSettled` equivalent for the bridge (or tests) to subscribe to.
46
+
47
+ ## Contract
48
+
49
+ ### 1. No-watch done → settle-or-idle-wake, both backends
50
+
51
+ `delegate` with no subsequent `watch`, followed by child
52
+ `shepherd_done`, must notify the Shepherd identically on Herdr panes and
53
+ hc terminals:
54
+
55
+ - **Shepherd busy** → one `agent_before_settle` continuation
56
+ (`completed` outcome only): a single `custom_message` entry with
57
+ `customType: "shepherd.task.unseen"`, drained task ids, `continue:
58
+ true`. Aborted/errored runs return nothing; entries stay pending for
59
+ the idle path.
60
+ - **Shepherd idle** → exactly one debounced
61
+ `pi.sendMessage(notice, { triggerTurn: true })` per burst, after
62
+ `UNSEEN_NOTICE_COALESCE_MS`. Entries drain only after the synchronous
63
+ send returns without throwing; a throw leaves entries pending for retry.
64
+ - Notice content is backend-agnostic and identical: header `Shepherd
65
+ notice — N task(s) settled with no watcher attached:`, per-entry
66
+ `- <taskId> (<agent>[, "label"]): <status> — <summary|error>`, footer
67
+ `Full results: shepherd_watch({ id: [ "id", ... ] })`, with
68
+ `details: { taskIds, settlements }` (never `completions` — that key
69
+ collides with the watcher collapsed formatter).
70
+ - `shepherd_watch` after the notice still retrieves the settled result
71
+ synchronously and clears the entry (authoritative retrieval clear in the
72
+ `watch` action, not the watcher notifier — late-watch deliveries never
73
+ reach the notifier).
74
+
75
+ ### 2. Observer parity on hc
76
+
77
+ The hc settle point gains an `onTaskSettled` hook with the same semantics
78
+ as the Herdr registry observer (007 §1):
79
+
80
+ - Fires once per actual transition into a terminal task state, after the
81
+ hc watcher fan-out, with a shallow copy of the result and `{ hadWatchers,
82
+ agent?, label?, source? }`.
83
+ - `hadWatchers` is true only when at least one hc watcher callback
84
+ actually ran for the settlement.
85
+ - `source: "delegate-submission-failure"` marks the hc delegate-failure
86
+ settle path; the ledger ignores it (the error is already in the
87
+ `shepherd_delegate` tool result).
88
+ - Listeners run inside `try/catch` after the fan-out; listener failure
89
+ never affects settlement; no subscriber means identical behavior.
90
+ - Prompt (legacy) settlements are not observed.
91
+
92
+ The bridge subscribes through the **active runtime's registry** (resolved
93
+ per command, same as 009's `resolveBackend`) instead of importing the
94
+ Herdr module directly — the seam 007 §1 anticipated ("its registry
95
+ equivalent must expose `onTaskSettled` with the same semantics; the
96
+ extension subscribes through the active runtime's registry"). The hc
97
+ direct-ledger write is removed; hc settles flow through observer →
98
+ `ledger.record` → `scheduleIdleWake`, exactly like Herdr.
99
+
100
+ ### 3. Timeout-vs-done race: inbox drains before the deadline wins
101
+
102
+ A tracked reply deadline (`shepherd_delegate` `timeout`) and a child
103
+ `shepherd_done` racing each other settle the task exactly once, on both
104
+ backends:
105
+
106
+ - First settlement wins. The task registry's already-settled guard
107
+ (Herdr `settleTask` early-return on non-running state; hc `if
108
+ (record.result)` guard) makes the second settle a no-op — same rule as
109
+ today, now asserted for both runtimes.
110
+ - The deadline timer MUST drain the parent inbox before settling. hc
111
+ `drainBrokerInbox()` runs only on tool calls (`message`/`watch`/
112
+ `status`/`read`/`close`) — an idle parent with no watch makes none,
113
+ so a `task_done` envelope can sit in the broker inbox while the bare
114
+ `setTimeout(deadline)` fires `timed_out` first. The fix: the deadline
115
+ callback drains first and settles timeout only if still no result.
116
+ A done already in the inbox before the deadline then wins (it was
117
+ truly first); a done arriving after keeps losing (correct first-wins).
118
+ The `watch`-loop deadline check already drains first — the timer is
119
+ brought to the same order.
120
+ - In practice a late `shepherd_done` still loses: it arrives at an
121
+ already-terminal task and is ignored (child sees the settled state,
122
+ not a second completion).
123
+ - The losing side's work is not lost: the agent transcript remains
124
+ readable via `shepherd_read`, and the winning settlement's text/error
125
+ names the outcome. No unseen entry is recorded for the ignored loser
126
+ (it was never a settlement).
127
+ - If the timeout settles with no watcher attached, §1 applies: the
128
+ `timed_out`/`blocked` entry is itself an unseen settlement and is
129
+ notified once.
130
+
131
+ ### 4. Close cancels pending watch
132
+
133
+ Closing an agent with an outstanding delegated task (`shepherd_close`)
134
+ cancels the task on both backends:
135
+
136
+ - Settlement is `cancelled`; the ledger never records `cancelled`
137
+ (007 §2), so no unseen entry and no wake/boundary continuation arise
138
+ from the close itself.
139
+ - `clearForAgent(handle.id)` after a successful `runtime.close` drops any
140
+ still-pending unseen entries for that agent and reports the dropped
141
+ task ids in the close tool result (retrievable-before-close via
142
+ `shepherd_watch` remains the Shepherd's choice).
143
+ - A `shepherd_watch` already blocked on the task resolves with the
144
+ cancellation rather than hanging; no new watch may attach to a closed
145
+ agent's tasks.
146
+
147
+ ### 5. Verification: unseen matrix case for hc
148
+
149
+ Extend the unseen suite (`test/verify-unseen-completions.mjs`) with an hc
150
+ matrix case alongside the `verify-hc-matrix.mjs` pattern (009 Phase 1.2):
151
+
152
+ - Same prompts on both runtimes: `delegate` + no `watch` +
153
+ settle-as-`done` produces byte-identical notice text modulo task/agent
154
+ ids, on Herdr and hc settle paths.
155
+ - No id leak: notices, tool results, and widget rows on the hc path
156
+ contain opaque ids only — no raw `t4` / `w1:p1`, no socket paths, no
157
+ `HERDR_*` / `HC_*` values.
158
+ - `delegate-submission-failure` on hc records nothing.
159
+ - Timeout-vs-done race on hc: settle-then-settle keeps the first result;
160
+ transcript readable; single (winner-only) unseen entry when unwatched.
161
+ - Close on hc: `cancelled`, ledger clean, close result reports dropped
162
+ ids.
163
+ - Deterministic: injected small `coalesceMs` + isolated ledger instance,
164
+ fake pi surface (capture `on` / `registerMessageRenderer` /
165
+ `sendMessage`), no live spawn — same discipline as the existing unseen
166
+ bridge cases.
167
+
168
+ ## Non-goals
169
+
170
+ - Auto-watch on `delegate` (rejected in 007, stays rejected on hc).
171
+ - Child-side `shepherd_done` return or `task_done_ack` envelope changes.
172
+ - Broker envelope/schema changes; new model-facing tools; new settings;
173
+ `shepherd_status` unseen-count (still a possible follow-up, not this
174
+ spec).
175
+ - Persisting the ledger across sessions (in-memory, session-scoped, both
176
+ backends).
177
+ - Changing watched-task delivery (`steer`), watcher batching, legacy
178
+ prompt watchers, or lifecycle timing.
179
+
180
+ ## Invariants (007 carried forward, both backends)
181
+
182
+ - Settlement is authoritative; ledger and notice never change task state
183
+ and never invent a completion.
184
+ - The settlement observer cannot slow or break settlement (either
185
+ runtime).
186
+ - Watched deliveries byte-for-byte unchanged.
187
+ - No `continue: true` after an aborted or errored run.
188
+ - Notify-once per entry; no continuation loops.
189
+ - No tool-schema, prompt, settings, broker, or fieldnote changes.
190
+ - `registerShepherdTools` behavior unchanged; existing suites pass
191
+ untouched.
@@ -0,0 +1,119 @@
1
+ # Tasks — 010 unseen-watch parity
2
+
3
+ Work in order. One worker per phase — phases run strictly sequential (no
4
+ parallel implementation workers; both phases touch the settle/notify
5
+ seam). A task is done only when its implementation + verification are
6
+ complete. This spec is `010-unseen-watch-parity` (spec.md contract,
7
+ plan.md design). Continues 009, converges onto the 007 contract.
8
+
9
+ **This task is spec-only: no `src` changes.** The checkboxes below bind
10
+ the later implementation.
11
+
12
+ ## Overnight run + testing (no reload)
13
+
14
+ - The main Pi session cannot reload extensions, so never rely on in-place
15
+ reload for live verification (same rule as 009).
16
+ - For live tests needing new code: `hc run --bg` a fresh herdr-core agent
17
+ (full `pi` path), then have that child call `shepherd_*`
18
+ (`spawn`/`delegate`/`watch`/`done`) to spawn agents — a fresh Pi session
19
+ with updated code.
20
+ - Keep `npm test` green in the current session for unit checks; use the
21
+ fresh `hc` child only for live spawn/delegate/watch proof.
22
+
23
+ ## Phase 0 — hc observer hook, no behavior change
24
+
25
+ - [x] **0.1 Add `onTaskSettled` to the hc settle path**
26
+ - In `src/runtimes/hc/runtime.ts`: listener set + `onTaskSettled`
27
+ returning unsubscribe, mirroring
28
+ `src/runtimes/herdr/orchestration.ts` `onTaskSettled`.
29
+ - Fire after the hc watcher fan-out (`notifyHcTaskWatchers`), with a
30
+ shallow result copy and fresh `{ hadWatchers, agent?, label?,
31
+ source? }`; `hadWatchers` true only when a watcher callback actually
32
+ ran; agent handle captured for `agent`/`label`; listeners inside
33
+ `try/catch`; fires only on the transition to terminal; prompt paths
34
+ untouched.
35
+ - `settleTask` gains the optional internal `source` options argument;
36
+ the hc `delegateAgent` failure path passes `source:
37
+ "delegate-submission-failure"`.
38
+ - Bridge NOT rewired yet; direct ledger write stays until Phase 1, so
39
+ behavior is unchanged.
40
+
41
+ - [x] **0.2 Verify Phase 0**
42
+ - `npm test` green; no existing suite modified.
43
+ - Observer unit cases (hc side, in the unseen suite file or a scratch
44
+ harness): settle with watcher → `hadWatchers: true`; without →
45
+ `false` + agent/label; `source` passthrough; listener throw does not
46
+ break `settleTask`; unsubscribe stops delivery.
47
+
48
+ ## Phase 1 — bridge via active runtime; parity live
49
+
50
+ - [x] **1.1 Subscribe through the resolved runtime**
51
+ - `src/extension/unseen-completions.ts`: replace the direct Herdr
52
+ `lifecycleRegistry` import with subscription through the active
53
+ runtime's registry (per-command `resolveBackend` result; see plan.md
54
+ open question 1 for placement).
55
+ - Session scoping unchanged: subscribe on `session_start`
56
+ (idempotent), release on `session_shutdown`; busy/idle tracking
57
+ (`agent_start` / `agent_settled`, `ctx.isIdle()` preferred),
58
+ `UNSEEN_NOTICE_COALESCE_MS` debounce, `triggerTurn` idle wake,
59
+ `agent_before_settle` boundary continuation (`completed` only), no
60
+ pre-entry `canContinue` check (#16), drain-on-hand-to-pi, best-effort
61
+ send (throw keeps pending).
62
+ - Remove the hc direct-ledger write: hc settles flow observer →
63
+ `record` → `scheduleIdleWake`, exactly like Herdr. The bridge owns
64
+ the only `record` call sites.
65
+
66
+ - [x] **1.2 Verify Phase 1**
67
+ - No-watch done → settle-or-idle-wake on BOTH backends: busy → one
68
+ boundary continuation; idle → exactly one debounced wake per burst;
69
+ notice text backend-agnostic (ids differ, wording identical).
70
+ - `npm test` green, existing suites untouched.
71
+
72
+ ## Phase 2 — matrix verification, docs, landing
73
+
74
+ - [x] **2.1 Unseen matrix case for hc** (in
75
+ `test/verify-unseen-completions.mjs`; cross-link `verify-hc-matrix.mjs`)
76
+ - Same prompts both runtimes: delegate + no watch + settle-as-done →
77
+ byte-identical notice text modulo task/agent ids, Herdr and hc.
78
+ - No id leak on hc: notices, tool results, widget rows carry opaque ids
79
+ only (no raw `t4` / `w1:p1`, socket paths, `HERDR_*` / `HC_*`).
80
+ - `delegate-submission-failure` on hc records nothing.
81
+ - Timeout-vs-done race on hc: first settlement wins (deadline wins),
82
+ loser ignored, transcript readable via `shepherd_read`, single
83
+ winner-only unseen entry when unwatched.
84
+ - Close-cancels-watch on hc: `cancelled`, ledger clean,
85
+ `clearForAgent` ids reported in the close result, blocked watch
86
+ resolves with cancellation.
87
+ - Deterministic: injected small `coalesceMs`, isolated ledger, fake pi
88
+ surface, no live spawn.
89
+
90
+ - [x] **2.2 Docs + live proof**
91
+ - One-row parity note where 007 documented the notice (README `###
92
+ Delegate, then watch` + `docs/guides/tool-reference.md`
93
+ `shepherd_watch` entry): the notice holds on both backends.
94
+ - Live smoke (throwaway script, both backends): delegate with no watch;
95
+ boundary notice + `shepherd_watch` retrieval; idle wake fires once;
96
+ close drops + reports; race keeps winner + transcript.
97
+
98
+ - [x] **2.3 Full suite + landing**
99
+ - `npm test` green with all existing suites untouched.
100
+ - Record scope, evidence, and the landing note in a `journal.md`
101
+ before merging (follow the 007 journal precedent).
102
+
103
+ ## Phase 3 — deadline drains inbox before settling timeout (live bug)
104
+
105
+ - [x] **3.1 Drain-first deadline timer on hc**
106
+ - In `src/runtimes/hc/runtime.ts` delegate deadline `setTimeout`:
107
+ call `drainBrokerInbox()` first, settle `timed_out` only if the
108
+ record still has no result (mirrors the `watch`-loop
109
+ drain-then-check order). No other behavior change.
110
+ - Done-in-inbox-before-deadline wins; genuinely late done still
111
+ loses (first-wins by arrival). `delegate-submission-failure` path
112
+ unchanged.
113
+
114
+ - [x] **3.2 Regression test (deterministic, no live spawn)**
115
+ - Inbox has queued `task_done` + expired deadline → `completed`,
116
+ no timeout entry. Empty inbox + expired deadline → `timed_out`
117
+ unseen entry. Late done after terminal timeout ignored, transcript
118
+ readable, single winner-only entry.
119
+ - `npm test` green, existing suites untouched.
@@ -0,0 +1,57 @@
1
+ # Intent — hc background poll (011)
2
+
3
+ ## Why this exists
4
+
5
+ Spec 010 closed the settle/notify seam: every hc settlement flows through
6
+ the observer into the shared ledger, and the Phase 3 deadline timer drains
7
+ first so a queued done beats an expired timeout. But settling still
8
+ requires *something to drain the parent inbox*, and on hc the only drains
9
+ are `message` tool calls and the delegate deadline timer. `watch`
10
+ (non-blocking registration) and `status` (read-only view) never drain.
11
+
12
+ Consequence, proven live in the 010 smoke runs: a child finishes,
13
+ `task_done` sits in `parent/inbox`, and with an idle Shepherd that sends
14
+ no messages, nothing settles until the delegate deadline fires — 10
15
+ minutes in the smoke, up to the 60-minute default setting in production.
16
+ The unseen notice (007/010) then arrives a full timeout late, and
17
+ `watch`/`status` report stale pending/running the whole time.
18
+
19
+ The pane backend never has this problem: it polls the parent inbox on a
20
+ 250 ms background timer (`parentBrokerTimer` in
21
+ `src/runtimes/herdr/lifecycle.ts`, unref'd, started with the broker,
22
+ stopped on shutdown). hc needs the same heartbeat behind the same
23
+ surface — no new tools, no schema change, no new settings.
24
+
25
+ ## Who it serves
26
+
27
+ 1. **Idle Shepherd (hc)** — delegates, keeps working, never messages.
28
+ Gets the settlement (and any unseen notice) within ~250 ms of the
29
+ child finishing, not at the next message or the far-away deadline.
30
+ 2. **Watch/status callers** — `watch` registration and `status` keep
31
+ their current semantics, but the registry they read is drained in the
32
+ background, so they stop reporting stale pending/running.
33
+ 3. **Maintainer** — one drain function, two triggers (tool calls as
34
+ today, plus the timer). No new machinery: the timer only calls the
35
+ existing `drainBrokerInbox`, and settlement still flows through the
36
+ 010 observer into the shared ledger.
37
+
38
+ ## What success looks like
39
+
40
+ - Same smoke as 010, minus the `message`: `spawn -> delegate -> (no
41
+ watch, no message)` settles `completed` within seconds of
42
+ `shepherd_done`, and the watcher/unseen notice follows at once.
43
+ - No junk output: the background poll creates no broker dirs, writes no
44
+ diagnostics on the happy path, and never keeps the host process alive
45
+ (unref'd, stops when quiescent).
46
+ - Deterministic suite: no wall-clock waits — an exported tick/start/stop
47
+ seam drives the timer logic in unit tests, same precedent as 010
48
+ Phase 0 exporting `notifyHcTaskSettledObservers` for testability.
49
+
50
+ ## Guiding values
51
+
52
+ 1. **Mirror Herdr, don't invent.** 250 ms, unref'd, single idempotent
53
+ timer, stop on quiescence/shutdown. Boring on purpose.
54
+ 2. **Settle path untouched.** The timer triggers drains; it never settles
55
+ directly, never reorders, never invents completions.
56
+ 3. **No new surface.** No tools, no schema, no settings, no broker
57
+ changes. Wiring only, like 010.
@@ -0,0 +1,95 @@
1
+ # Journal — 011 hc background poll
2
+
3
+ ## Scope (single turn, per user request)
4
+
5
+ Full spec in one turn: poller wiring + drain-first watch/status (Phase 0),
6
+ deterministic suite + docs + live smoke (Phase 1), landing (Phase 2).
7
+
8
+ ## What changed
9
+
10
+ - `src/runtimes/hc/runtime.ts`
11
+ - Hoisted the drain pipeline (`settleTask`, `resolveReplyForTask`,
12
+ `openPendingRequest`, `writeTaskDoneDiagnostic`,
13
+ `applyTaskDoneEnvelope`, `applyMessageEnvelope`,
14
+ `settleDueReplyTimeouts`) from the `createHcRuntime` closure to
15
+ module scope — verified closure-free (module maps/services only).
16
+ The closure keeps a thin `drainBrokerInbox()` wrapper, so all four
17
+ existing call sites (delegate, message, watch loop, deadline timer)
18
+ behave exactly as before.
19
+ - New module poller: `HC_DRAIN_POLL_MS = 250` (mirrors Herdr),
20
+ single idempotent unref'd timer, `registerHcPollSession` /
21
+ `startHcInboxPoller` / `stopHcInboxPoller` /
22
+ `isHcInboxPollerRunning` / `pollHcInboxOnce` /
23
+ `drainHcInboxForSession` / `hcInboxPollerStateForTests`.
24
+ `delegate` registers its session at task creation. Tick drains the
25
+ registered session, then stops at quiescence (no unsettled tasks).
26
+ - `registerHcPollSession` resolves through `ownerSessionKey` exactly
27
+ like the drains (explicit id → env session → process fallback),
28
+ never skipped: the tick always polls the same store tool calls use.
29
+ Found live: the first version validated only the passed arg, so an
30
+ env-session runtime (the smoke) never started the poller.
31
+ - `src/extension/shepherd.ts`: hc `watch` registration and `status`
32
+ view drain first via `drainHcInboxForSession` (best-effort try/catch,
33
+ returns false without touching anything when no session resolves —
34
+ a read can never mint a broker). Return shapes unchanged.
35
+ - `test/verify-hc-poll.mjs` (new, wired as `hc-poll:test` at the end of
36
+ the `test` chain): 7 assertions, no live spawn, no wall-clock waits —
37
+ seam lifecycle (idempotent start, unref, stop), quiescence stop,
38
+ no-mint guarantee, full tick→poll→apply pump with a real on-disk
39
+ broker (queued `task_done` consumed to `processed/` + dropped
40
+ diagnostic), latest-registration-wins.
41
+ - Docs: one row each in `README.md` (delegate-watch paragraph) and
42
+ `docs/guides/tool-reference.md` (watch entry).
43
+
44
+ ## Design finding: single session, not a set
45
+
46
+ The first cut tracked a *set* of poll sessions. Live-trace reasoning
47
+ showed the module caches exactly one broker: alternating session keys
48
+ would mint a fresh store on every `ensureBroker` call, so a multi-key
49
+ tick would poll fresh empty stores and never find completions. The hc
50
+ runtime is already single-session-per-process (module `generationId`
51
+ baked into every id), so the poller tracks one key, latest wins. This
52
+ is load-bearing, not style — recorded here so nobody "generalizes" it
53
+ back into a set.
54
+
55
+ ## Live evidence (throwaway `/tmp/011-live-smoke.mjs`, fresh process)
56
+
57
+ Real `hc run --bg` worker, `delegate` with a 30-minute deadline (so the
58
+ Phase-3 deadline drain cannot confound), then **no watch, no message,
59
+ no status calls** — observation only via `describeHcStatusView` (pure
60
+ map read) and direct inbox dir stats:
61
+ `spawn → delegate → poller running → done arrives → settled 3 s later`
62
+ (granularity-bounded; tick is 250 ms), close clean, zero new
63
+ `hc-parent-*` junk stores. Note: the status view only carries
64
+ *unsettled* tasks, so a delegated task vanishing from the view while
65
+ the agent is known IS the settlement signal.
66
+
67
+ ## Verification
68
+
69
+ - `npm test` exit 0 (676 PASS lines), all pre-existing suites untouched,
70
+ `hc-poll:test` green at the end of the chain.
71
+ - Live smoke passed (above).
72
+
73
+ ## Landing
74
+
75
+ Single commit on `feat/011-hc-background-poll`, fast-forward merged to
76
+ `main`. 010's branch is already merged; 011 builds directly on it.
77
+
78
+ ## Review follow-up (reviewer: review-010-011, same day)
79
+
80
+ - Critical 1 (shutdown wiring): `stopHcInboxPoller()` is now called
81
+ from both `shutdownHcStaleWaitMonitor` and `shutdownHcTaskWatchers`
82
+ (they are currently caller-less export seams; the wiring honors the
83
+ contract for when they are called). Unref + quiescence remain the
84
+ live guarantees.
85
+ - Critical 2 (real settlement): new `hcInjectTestAgent` /
86
+ `hcInjectTestTask` (/Remove) seams drive a real known-task settle
87
+ through the tick and the 010 observer, plus the keep-alive converse
88
+ (open task keeps the poller alive; removal restores quiescence).
89
+ - Warning (b) delegate-start: covered by the live smoke (poller
90
+ observed running right after `delegate`), not the unit suite —
91
+ `delegate` needs a live child, so the suite asserts register
92
+ semantics and the smoke asserts the wiring.
93
+ - Warning (d): `writeTaskDoneDiagnostic` and `applyTaskDoneEnvelope`
94
+ now take the drain's `active` broker; diagnostics land beside the
95
+ drained store, never the module-cached one.
@@ -0,0 +1,87 @@
1
+ # Plan — 011 hc background poll
2
+
3
+ ## Target
4
+
5
+ One timer, existing drain, both wired and removable:
6
+
7
+ ```text
8
+ src/runtimes/hc/runtime.ts background poller (start/stop/tick) + tick uses
9
+ session-keyed broker only; bare-ensureBroker ban
10
+ src/runtimes/herdr/lifecycle.ts UNCHANGED (reference implementation only)
11
+ test/verify-hc-poll.mjs NEW deterministic suite (no live spawn,
12
+ no wall-clock waits)
13
+ package.json / test chain wire the new suite (script + chain entry)
14
+ docs one-row note where 007/010 documented notices
15
+ (README delegate-watch paragraph +
16
+ tool-reference shepherd_watch entry)
17
+ ```
18
+
19
+ ## Design decisions
20
+
21
+ 1. **Mirror Herdr's shape exactly.** Herdr: module `parentBrokerTimer`,
22
+ `startParentBrokerMonitor` (idempotent, 250 ms, `unref`), stopped on
23
+ shutdown/quiescence. hc: module `hcDrainTimer` (or equivalent),
24
+ started on first `delegate` (first open task — `tasks` already
25
+ records `deadlineAt`, so "open" is cheap to compute), stopped when
26
+ zero unsettled tasks remain, restarted by the next `delegate`.
27
+ 250 ms fixed, unref'd — no new setting (boring rule).
28
+ 2. **Tick = one `drainBrokerInbox()` call, nothing else.** It already
29
+ drains the parent inbox, applies completions/replies through the
30
+ 010 paths, and settles due reply timeouts. The tick adds no logic;
31
+ settlement still flows observer → ledger → bridge exactly as today.
32
+ Separately, hc `watch`-registration drains first (before the
33
+ non-blocking `hcTaskWatcherService.watch`) and `describeHcStatusView`
34
+ drains first: same drain function, synchronous, return shapes
35
+ unchanged — an already-settled task reports `completed` instead of
36
+ stale `pending`/`running`.
37
+ 3. **Session-keyed broker only.** The tick resolves the broker the same
38
+ way tool-call drains do (the adapter's `sessionId`); if no session
39
+ key is available it skips the tick. It never calls bare
40
+ `ensureBroker()`. This is the 010 follow-up invariant made
41
+ structural: grep must show exactly one bare `ensureBroker()` left
42
+ in the module (none — every call site passes a session id; verify
43
+ by inspection in Phase 0).
44
+ 4. **Test seam, not timers, in suites.** Export start/stop plus a
45
+ synchronous `tickHcDrain`-style entry (name TBD in Phase 0;
46
+ precedent: 010 exporting `notifyHcTaskSettledObservers` "so observer
47
+ parity is unit-testable without a live terminal"). Suites drive the
48
+ seam with isolated ledgers/registries; no `setInterval` fires in
49
+ tests, no wall-clock waits.
50
+ 5. **Shutdown joins the existing seams.** `shutdownHcTaskWatchers()`
51
+ and `shutdownHcStaleWaitMonitor()` already exist as module exports;
52
+ the poller stops alongside them (and is also safe to leave stopped:
53
+ quiescence stops it anyway, and `unref` means it can never hold the
54
+ process open).
55
+ 6. **One worker per phase.** The timer touches the drain/settle seam
56
+ that 010 just stabilized: Phase 0 (wiring) and Phase 1
57
+ (verification) run strictly sequential, no parallel implementation
58
+ workers.
59
+
60
+ ## Order
61
+
62
+ Phase 0 — poller wiring + drain-first watch/status, no behavior
63
+ contract change beyond promptness/freshness:
64
+ timer + start/stop + tick seam + bare-`ensureBroker` audit (every
65
+ settle-path call site passes a session id); drain-first hc watch
66
+ registration + `describeHcStatusView`; `npm test` green, no
67
+ existing suite modified.
68
+
69
+ Phase 1 — deterministic suite (`verify-hc-poll.mjs`) + package.json
70
+ wiring + docs one-row + live smoke (throwaway, both backends:
71
+ delegate with no watch and no message settles in ~1 s on hc, ~250 ms
72
+ on Herdr as today) + journal + landing.
73
+
74
+ ## Open questions
75
+
76
+ 1. Start on `delegate` vs `spawn`? Decided: `delegate` (no completion
77
+ can arrive before the first task exists; avoids waking the timer
78
+ for agents that never receive work).
79
+ 2. Stop condition: zero unsettled tasks — or also require zero live
80
+ agents? Proposal: tasks only (agents without tasks cannot settle
81
+ anything). Cheap either way; decide in Phase 0.
82
+ 3. Tick seam shape: exported `tick` function vs injectable interval
83
+ with `0` meaning manual? Proposal: exported tick + start/stop
84
+ (simplest, mirrors the 010 observer-hook precedent).
85
+ 4. ~~Should `watch`-registration/`status` also drain first?~~ Decided:
86
+ yes — drain-first in both paths (return shapes unchanged). The
87
+ 250 ms poll is still required for idle parents that make no calls.