@luminascale/shepherd 0.0.0-stage → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (204) hide show
  1. package/.agents/agents/README.md +12 -0
  2. package/.agents/agents/planner.md +38 -0
  3. package/.agents/agents/reviewer.md +38 -0
  4. package/.agents/agents/scout.md +43 -0
  5. package/.agents/agents/worker.md +33 -0
  6. package/.agents/skills/smoke-test/SKILL.md +155 -0
  7. package/.agents/skills/smoke-test/references/forensics.md +46 -0
  8. package/.agents/skills/smoke-test/scripts/live-smoke-poll.mjs +130 -0
  9. package/.github/workflows/publish.yml +77 -0
  10. package/.pi/prompts/release.md +21 -0
  11. package/.prettierrc +10 -0
  12. package/AGENTS.md +88 -0
  13. package/CONTRIBUTING.md +177 -0
  14. package/LICENSE +201 -0
  15. package/README.md +397 -2
  16. package/docs/README.md +26 -0
  17. package/docs/guides/agent-definitions.md +42 -0
  18. package/docs/guides/diagnostics.md +199 -0
  19. package/docs/guides/installation.md +61 -0
  20. package/docs/guides/tool-reference.md +55 -0
  21. package/docs/manual-tests/ping-pong-peer-messaging.md +147 -0
  22. package/docs/plans/agent-labels.md +257 -0
  23. package/docs/plans/architecture.md +110 -0
  24. package/docs/plans/settings-scope.md +163 -0
  25. package/docs/plans/tool-surface-split.md +278 -0
  26. package/docs/reference/dictionary +26 -0
  27. package/docs/releasing.md +47 -0
  28. package/docs/specs/005-agent-messaging-and-task-lifecycle/baseline.md +115 -0
  29. package/docs/specs/005-agent-messaging-and-task-lifecycle/handoff.md +564 -0
  30. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase10-notes.md +109 -0
  31. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase6-notes.md +69 -0
  32. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase7-notes.md +100 -0
  33. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase8-notes.md +127 -0
  34. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase9-notes.md +104 -0
  35. package/docs/specs/005-agent-messaging-and-task-lifecycle/plan.md +1093 -0
  36. package/docs/specs/005-agent-messaging-and-task-lifecycle/spec.md +583 -0
  37. package/docs/specs/005-agent-messaging-and-task-lifecycle/tasks.md +838 -0
  38. package/docs/specs/006-shepherd-tool-rendering/plan.md +175 -0
  39. package/docs/specs/006-shepherd-tool-rendering/spec.md +469 -0
  40. package/index.ts +602 -0
  41. package/package.json +112 -4
  42. package/scripts/capture-system-prompt.mjs +45 -0
  43. package/scripts/extract-pi-system-prompt.mjs +277 -0
  44. package/scripts/host-impact.mjs +74 -0
  45. package/scripts/resolve-agent.mjs +43 -0
  46. package/scripts/show-shepherd-orchestrator.mjs +58 -0
  47. package/scripts/show-shepherd-prompt.mjs +96 -0
  48. package/scripts/verify-suites.mjs +67 -0
  49. package/specs/001-artifacts/spec.md +436 -0
  50. package/specs/002-simplify-agent-orchestration/plan.md +287 -0
  51. package/specs/002-simplify-agent-orchestration/spec.md +323 -0
  52. package/specs/002-simplify-agent-orchestration/tasks.md +212 -0
  53. package/specs/003-orchestrator-session-artifacts/plan.md +396 -0
  54. package/specs/003-orchestrator-session-artifacts/spec.md +453 -0
  55. package/specs/003-orchestrator-session-artifacts/tasks.md +141 -0
  56. package/specs/004-asynchronous-prompt-watchers/plan.md +139 -0
  57. package/specs/004-asynchronous-prompt-watchers/spec.md +202 -0
  58. package/specs/004-asynchronous-prompt-watchers/tasks.md +126 -0
  59. package/specs/005-per-workspace-settings/plan.md +230 -0
  60. package/specs/005-per-workspace-settings/spec.md +368 -0
  61. package/specs/005-per-workspace-settings/tasks.md +178 -0
  62. package/specs/006-dual-runtime-herdr-durable/intent.md +86 -0
  63. package/specs/006-dual-runtime-herdr-durable/journal.md +80 -0
  64. package/specs/006-dual-runtime-herdr-durable/plan.md +242 -0
  65. package/specs/006-dual-runtime-herdr-durable/spec.md +389 -0
  66. package/specs/006-dual-runtime-herdr-durable/tasks.md +199 -0
  67. package/specs/007-unwatched-completion-notice/journal.md +145 -0
  68. package/specs/007-unwatched-completion-notice/plan.md +186 -0
  69. package/specs/007-unwatched-completion-notice/spec.md +284 -0
  70. package/specs/007-unwatched-completion-notice/tasks.md +147 -0
  71. package/specs/008-remote-machine-sheep/intent.md +174 -0
  72. package/specs/008-remote-machine-sheep/journal.md +57 -0
  73. package/specs/008-remote-machine-sheep/prerequisites.md +219 -0
  74. package/specs/009-hc-runtime/intent.md +44 -0
  75. package/specs/009-hc-runtime/plan.md +49 -0
  76. package/specs/009-hc-runtime/spec.md +118 -0
  77. package/specs/009-hc-runtime/tasks.md +62 -0
  78. package/specs/010-unseen-watch-parity/intent.md +57 -0
  79. package/specs/010-unseen-watch-parity/journal.md +172 -0
  80. package/specs/010-unseen-watch-parity/plan.md +74 -0
  81. package/specs/010-unseen-watch-parity/spec.md +191 -0
  82. package/specs/010-unseen-watch-parity/tasks.md +119 -0
  83. package/specs/011-hc-background-poll/intent.md +57 -0
  84. package/specs/011-hc-background-poll/journal.md +95 -0
  85. package/specs/011-hc-background-poll/plan.md +87 -0
  86. package/specs/011-hc-background-poll/spec.md +141 -0
  87. package/specs/011-hc-background-poll/tasks.md +83 -0
  88. package/specs/012-steer-default/intent.md +12 -0
  89. package/specs/012-steer-default/journal.md +40 -0
  90. package/specs/012-steer-default/plan.md +7 -0
  91. package/specs/012-steer-default/spec.md +42 -0
  92. package/specs/012-steer-default/tasks.md +6 -0
  93. package/specs/013-opencode-shepherd/handoff-t011.md +94 -0
  94. package/specs/013-opencode-shepherd/intent.md +59 -0
  95. package/specs/013-opencode-shepherd/plan.md +204 -0
  96. package/specs/013-opencode-shepherd/spec.md +258 -0
  97. package/specs/013-opencode-shepherd/tasks.md +263 -0
  98. package/src/core/artifact-sessions.ts +835 -0
  99. package/src/core/config.ts +329 -0
  100. package/src/core/discovery.ts +342 -0
  101. package/src/core/herdr.ts +1 -0
  102. package/src/core/host.ts +77 -0
  103. package/src/core/ids.ts +78 -0
  104. package/src/core/lifecycle.ts +1 -0
  105. package/src/core/messaging.ts +1 -0
  106. package/src/core/migration.ts +55 -0
  107. package/src/core/orchestration.ts +1 -0
  108. package/src/core/types.ts +143 -0
  109. package/src/core/unseen-completions.ts +114 -0
  110. package/src/extension/cli.ts +417 -0
  111. package/src/extension/config.ts +1 -0
  112. package/src/extension/settings-ui.ts +296 -0
  113. package/src/extension/shepherd-done.ts +768 -0
  114. package/src/extension/shepherd.ts +2548 -0
  115. package/src/extension/system-prompt.ts +38 -0
  116. package/src/extension/unseen-completions.ts +237 -0
  117. package/src/opencode/actions.ts +794 -0
  118. package/src/opencode/bridge.ts +777 -0
  119. package/src/opencode/display.ts +36 -0
  120. package/src/opencode/fieldnotes.ts +60 -0
  121. package/src/opencode/host.ts +62 -0
  122. package/src/opencode/orphans.ts +162 -0
  123. package/src/opencode/server.ts +458 -0
  124. package/src/runtimes/backend.ts +119 -0
  125. package/src/runtimes/detect.ts +102 -0
  126. package/src/runtimes/hc/hc.ts +527 -0
  127. package/src/runtimes/hc/runtime.ts +1898 -0
  128. package/src/runtimes/herdr/herdr.ts +912 -0
  129. package/src/runtimes/herdr/lifecycle.ts +1712 -0
  130. package/src/runtimes/herdr/messaging.ts +703 -0
  131. package/src/runtimes/herdr/orchestration.ts +1583 -0
  132. package/src/runtimes/herdr/runtime.ts +224 -0
  133. package/src/runtimes/runtime.ts +133 -0
  134. package/src/runtimes/settle-observations.ts +114 -0
  135. package/test/fixtures/context-project/AGENTS.md +2 -0
  136. package/test/fixtures/home/.agents/agents/same.md +6 -0
  137. package/test/fixtures/home/.agents/agents/shared-only.md +6 -0
  138. package/test/fixtures/home/.pi/agent/agents/invalid-thinking.md +6 -0
  139. package/test/fixtures/home/.pi/agent/agents/model-empty.md +7 -0
  140. package/test/fixtures/home/.pi/agent/agents/model-explicit.md +7 -0
  141. package/test/fixtures/home/.pi/agent/agents/omit-context-absent.md +5 -0
  142. package/test/fixtures/home/.pi/agent/agents/omit-context-false.md +6 -0
  143. package/test/fixtures/home/.pi/agent/agents/omit-context-invalid.md +6 -0
  144. package/test/fixtures/home/.pi/agent/agents/omit-context-true.md +6 -0
  145. package/test/fixtures/home/.pi/agent/agents/omit-false.md +6 -0
  146. package/test/fixtures/home/.pi/agent/agents/omit-invalid.md +6 -0
  147. package/test/fixtures/home/.pi/agent/agents/omit-pi-docs.md +7 -0
  148. package/test/fixtures/home/.pi/agent/agents/omit-true.md +6 -0
  149. package/test/fixtures/home/.pi/agent/agents/only-user.md +9 -0
  150. package/test/fixtures/home/.pi/agent/agents/same.md +6 -0
  151. package/test/fixtures/proj/.agents/agents/proj-shared.md +6 -0
  152. package/test/fixtures/proj/.pi/agents/project-pi.md +6 -0
  153. package/test/fixtures/proj/.pi/agents/same.md +6 -0
  154. package/test/helpers/fake-clock.mjs +39 -0
  155. package/test/helpers/feature-contracts.mjs +44 -0
  156. package/test/helpers/test-utils.mjs +44 -0
  157. package/test/verify-artifact-sessions.mjs +258 -0
  158. package/test/verify-backend.mjs +297 -0
  159. package/test/verify-child-mislabel.mjs +135 -0
  160. package/test/verify-child-surface.mjs +321 -0
  161. package/test/verify-cli.mjs +197 -0
  162. package/test/verify-command-ux.mjs +100 -0
  163. package/test/verify-compat.mjs +32 -0
  164. package/test/verify-cross-host-notes.mjs +128 -0
  165. package/test/verify-delegate.mjs +161 -0
  166. package/test/verify-detect.mjs +151 -0
  167. package/test/verify-discovery.mjs +239 -0
  168. package/test/verify-done-nudge.mjs +238 -0
  169. package/test/verify-effective-prompt.mjs +98 -0
  170. package/test/verify-hc-matrix.mjs +246 -0
  171. package/test/verify-hc-poll.mjs +260 -0
  172. package/test/verify-host-docs.mjs +66 -0
  173. package/test/verify-host-impact.mjs +49 -0
  174. package/test/verify-ids.mjs +141 -0
  175. package/test/verify-launch.mjs +183 -0
  176. package/test/verify-message-routing.mjs +560 -0
  177. package/test/verify-messaging.mjs +303 -0
  178. package/test/verify-migration-notice.mjs +74 -0
  179. package/test/verify-multi-wait.mjs +42 -0
  180. package/test/verify-opencode-display.mjs +47 -0
  181. package/test/verify-opencode-fieldnotes.mjs +121 -0
  182. package/test/verify-opencode-notice.mjs +77 -0
  183. package/test/verify-opencode-nudge.mjs +116 -0
  184. package/test/verify-opencode-orphans.mjs +160 -0
  185. package/test/verify-opencode-surface.mjs +107 -0
  186. package/test/verify-packaging.mjs +82 -0
  187. package/test/verify-parent-surface.mjs +118 -0
  188. package/test/verify-phase0.mjs +95 -0
  189. package/test/verify-registry.mjs +124 -0
  190. package/test/verify-rename-migration.mjs +111 -0
  191. package/test/verify-rendering.mjs +379 -0
  192. package/test/verify-session-boundary.mjs +146 -0
  193. package/test/verify-settings.mjs +260 -0
  194. package/test/verify-stale-wait.mjs +339 -0
  195. package/test/verify-status-widget.mjs +377 -0
  196. package/test/verify-status.mjs +172 -0
  197. package/test/verify-task-completion.mjs +156 -0
  198. package/test/verify-task-failures.mjs +201 -0
  199. package/test/verify-task-mislabel.mjs +143 -0
  200. package/test/verify-task-registry.mjs +194 -0
  201. package/test/verify-task-watchers.mjs +217 -0
  202. package/test/verify-tool-schemas.mjs +120 -0
  203. package/test/verify-unseen-completions.mjs +1211 -0
  204. package/test/verify-watchers.mjs +181 -0
@@ -0,0 +1,564 @@
1
+ # Handoff: Asynchronous Shepherd Tasks and Agent Messaging
2
+
3
+ ## Session state
4
+
5
+ - Branch: `feat/005-agent-messaging-and-task-lifecycle`
6
+ - Repository: `MikkelKappelPersson/shepherd`
7
+ - Current HEAD: tip of the feature branch — `b2c3b59` plus the Phase 5 failure-test commit and this handoff update (hashes shift when this doc changes; use `git log --oneline`)
8
+ - Working tree: contains the Phase 11 launch-tool fix, verification updates, and documentation/checklist edits
9
+ - Last full verification: `npm test` passed with zero failures
10
+
11
+ This handoff is for continuing implementation on another computer. Start by
12
+ checking out the branch and reading this file, `spec.md`, `plan.md`, and
13
+ `tasks.md` in this directory.
14
+
15
+ ## Completed work
16
+
17
+ ### Phase 0 — Baseline and guardrails
18
+
19
+ Completed and committed in `a0dab9f`.
20
+
21
+ Added:
22
+
23
+ - `baseline.md`
24
+ - Reusable temporary-directory and fake-identity helpers
25
+ - Manual/scoped clock helpers
26
+ - Centralized future tool/surface/task-state contracts
27
+ - Explicit-settlement guardrail tests
28
+ - Phase 0 test command in `package.json`
29
+
30
+ The baseline confirms that the current legacy prompt path uses sidecars and
31
+ Herdr state observations, while the new task path must not infer completion
32
+ from those events.
33
+
34
+ ### Phase 1 — Task registry and task state
35
+
36
+ Completed and committed in `ae68588`.
37
+
38
+ Implemented in `src/core/orchestration.ts`:
39
+
40
+ - Session-scoped opaque task IDs
41
+ - `TaskRecord`, `TaskResult`, and settlement types
42
+ - `created`, `running`, `waiting`, `completed`, `blocked`, `failed`,
43
+ `cancelled`, and `timed_out` states
44
+ - One active tracked task per agent
45
+ - Pending request tracking
46
+ - Waiting timestamps
47
+ - Task deadlines and timeout timers
48
+ - Stale-notification metadata
49
+ - Artifact association fields and settlement callbacks
50
+ - Ownership validation
51
+ - Idempotent terminal settlement
52
+ - Cancellation and task snapshots
53
+
54
+ Tests are in `test/verify-task-registry.mjs`.
55
+
56
+ ### Phase 2 — Parent-owned mailbox transport
57
+
58
+ Completed and committed in `e48666d`.
59
+
60
+ Implemented in `src/core/messaging.ts`:
61
+
62
+ - Protected session mailbox directories
63
+ - Parent and per-child inboxes
64
+ - Atomic JSON envelope publication
65
+ - Envelope validation
66
+ - Parent/child/peer routing
67
+ - Child capabilities
68
+ - Session, broker, sender, and target validation
69
+ - Duplicate acknowledgement markers
70
+ - Malformed-envelope quarantine
71
+ - Queue-depth, byte-size, and content-length limits
72
+ - Processed/rejected message retention
73
+ - Cleanup protection until children are gone
74
+
75
+ Tests are in `test/verify-messaging.mjs`.
76
+
77
+ ### Phase 3 — Child-side Shepherd surface
78
+
79
+ Completed and committed in `44b5ae8`.
80
+
81
+ Extended `src/extension/shepherd-done.ts` with:
82
+
83
+ - Child `shepherd_message` tool
84
+ - Child `shepherd_done` tool
85
+ - Broker capability loading from environment
86
+ - Child inbox polling
87
+ - `followUp` and `steer` delivery through `pi.sendUserMessage`
88
+ - Correlated sender/message/task/thread/reply metadata
89
+ - Runtime diagnostics for failed local delivery
90
+ - Explicit task-done envelopes
91
+ - Tracked-task protection against normal `agent_end` completion
92
+
93
+ Extended `src/core/herdr.ts` with launch-time broker and task environment
94
+ wiring plus task instructions explaining explicit completion and replies.
95
+
96
+ Tests are in `test/verify-child-surface.mjs`; launch coverage is in
97
+ `test/verify-launch.mjs`.
98
+
99
+ ### Phase 4 — Tracked delegation
100
+
101
+ Implemented and committed in `b2c3b59`.
102
+
103
+ Implemented:
104
+
105
+ - Parent `shepherd_delegate` tool
106
+ - `LifecycleDelegateParams`
107
+ - Parent broker creation and lazy monitor lifecycle
108
+ - Pre-launch opaque agent ID reservation
109
+ - Child capability registration during agent startup
110
+ - `delegateAgent()` in `src/core/lifecycle.ts`
111
+ - Non-blocking task-envelope submission
112
+ - Task artifact reservation/start/finalization callback
113
+ - Publication failure rollback
114
+ - Closed-agent and active-task checks
115
+ - Updated parent tool schema/surface tests
116
+
117
+ The delegation path is:
118
+
119
+ ```text
120
+ shepherd_spawn
121
+ -> parent broker registers child capability
122
+ -> child starts with broker/task environment
123
+ shepherd_delegate
124
+ -> task record is reserved
125
+ -> task envelope is queued to the child inbox
126
+ -> task transitions to running
127
+ -> delegate returns taskId without waiting
128
+ ```
129
+
130
+ The old `shepherd_prompt` path remains separate and is still legacy behavior.
131
+ Its compatibility mapping to task IDs is deliberately deferred.
132
+
133
+ ### Phase 5 — Explicit completion and failures
134
+
135
+ Complete.
136
+
137
+ Implemented in `b2c3b59` and the task-failure tests:
138
+
139
+ - Parent broker polling via `processParentBrokerMessages()`
140
+ - `task_done` envelope consumption
141
+ - Child ownership validation
142
+ - Unknown/foreign task rejection
143
+ - Rejection of successful completion while required requests are pending
144
+ - Explicit completion summary/timestamp persistence
145
+ - Idempotent task settlement
146
+ - Task artifact finalization callback
147
+ - Task deadline timeout settlement
148
+ - Pane disappearance failure detection
149
+ - Provider-error detection from pane output
150
+ - Close-to-task-cancellation behavior
151
+ - Normal tracked-task `agent_end` no longer claims success
152
+ - Task ID inclusion in retained sidecars
153
+
154
+ Tests:
155
+
156
+ - `test/verify-task-completion.mjs`: owning-child completion, foreign-child
157
+ rejection, duplicate completion, pending-request rejection, blocked
158
+ completion, idle-child + explicit completion, and no-premature-completion
159
+ from `agent_end`/`agent_settled`/idle state.
160
+ - `test/verify-task-failures.mjs`: provider-error failure, unexpected pane
161
+ exit failure, close-to-cancellation, and deadline timeout. Uses a
162
+ Node-script fake `herdr` on PATH backed by a JSON state file so `pane
163
+ list`, `pane close`, and `agent read` observations are deterministic.
164
+
165
+ Deferred (tracked in `tasks.md`):
166
+
167
+ - Preserve useful child output in failure results (completion-diagnostics
168
+ follow-up).
169
+ - Preserve runtime observations for status output (Phase 9).
170
+
171
+ ### Phase 6 — Asynchronous message routing
172
+
173
+ Complete.
174
+
175
+ Implemented:
176
+
177
+ - Parent `shepherd_message` tool (`src/extension/shepherd.ts`, schema in
178
+ `src/core/types.ts`) with flat root object: `target` + `message` required;
179
+ `taskId`, `threadId`, `replyTo`, `expectsReply`, `delivery` optional.
180
+ - `sendParentMessage()` and the parent message notifier/bridge in
181
+ `src/core/lifecycle.ts`; child message delivery via
182
+ `configureParentMessageNotifications()` and `sendUserMessage` delivery in
183
+ `src/extension/shepherd.ts` / `shepherd-done.ts`. Child requests and replies
184
+ wake the parent at the next safe turn boundary; ordinary child messages are
185
+ queued passively.
186
+ - Task registry request/reply correlation in `src/core/orchestration.ts`
187
+ (`openPendingRequest`, `resolveReplyForTask`, `clearPendingReply`) with a
188
+ single outstanding request per task.
189
+ - Child-side `runtime` request-open mirror and reply routing through the
190
+ parent broker, so a busy peer's queued question keeps the sender task in
191
+ the `waiting` state and a matching reply returns it to `running`.
192
+ - Reply deadline tracking and `blocked` settlement on timeout.
193
+ - Provenance preserved via `originSenderId` on the relayed reply.
194
+ - Delivery failure diagnostic surfaced to the parent for failed local
195
+ `sendUserMessage`.
196
+
197
+ Tests: `test/verify-message-routing.mjs` (parent/child, busy/idle recipient,
198
+ request creation and timeout, valid/invalid/duplicate/origin-preserving
199
+ replies, delivery failures, unknown/closed targets, task ownership check).
200
+
201
+ See `phase6-notes.md` in this directory for the routing and correlation
202
+ decisions.
203
+
204
+ ### Phase 7 — Task-aware `shepherd_watch`
205
+
206
+ Complete.
207
+
208
+ Implemented:
209
+
210
+ - `LifecycleRegistry.watchTasks()` (plus the task side of `settleTask`
211
+ settlement) in `src/core/orchestration.ts`: one-shot, synchronous,
212
+ exactly-once per task, driven directly from settlement — no polling.
213
+ `settleTask` is the single authoritative hook, so `waiting`, `idle`,
214
+ `agent_end`, and `agent_settled` can never fire a task watcher.
215
+ - `TaskWatcherService` in `src/core/lifecycle.ts`: coalescing, ordered
216
+ delivery with `unref()`'d flush timers and `shutdown()`.
217
+ - `shepherd_watch` now accepts task ids (primary) and legacy prompt ids
218
+ (compat, routed to the existing prompt watcher); agent ids and pane ids
219
+ are rejected with a clear error; a call cannot mix both kinds.
220
+ - Parent `shepherd.task.completion` message + bridge that sends a
221
+ follow-up with `triggerTurn: true` keyed by task id.
222
+
223
+ Tests: `test/verify-task-watchers.mjs` (non-blocking registration, no
224
+ completion from `waiting`, terminal-only settlement across completed/failed/
225
+ blocked/cancelled/timed_out, coalesced input order, independent watchers,
226
+ rejections, bridge delivery and shutdown suppression).
227
+
228
+ See `phase7-notes.md` in this directory for the design decisions.
229
+
230
+ ### Phase 8 — Stale-wait monitoring
231
+
232
+ Complete.
233
+
234
+ Implemented:
235
+
236
+ - `staleWaitThreshold` setting in `src/extension/config.ts` (minutes, default
237
+ `5`; `0` disables reminders; invalid values fall back to the default).
238
+ Wired into `OVERRIDABLE_FIELDS` so it participates in user + project layer
239
+ resolution, the `/shepherd settings` menu (`src/extension/settings-ui.ts`
240
+ adds a `Stale wait reminder` row with an `off` option), and the README
241
+ settings table plus a short stale-wait explanation.
242
+ - `LifecycleRegistry.markStaleNotified(taskId)` (orchestration) stamps the
243
+ episode with a wall-clock time so the monitor issues at most one reminder
244
+ per episode. `staleNotifiedAt` is cleared by `setTaskRunning`,
245
+ `resolvePendingRequest`, `resolveReplyForTask`, and `settleTask` — a reply
246
+ (or any terminal event) resets the state, a re-entry into `waiting` later
247
+ can produce a new reminder.
248
+ - `StaleWaitMonitor` (lifecycle.ts) + the `staleWaitMonitor` singleton and
249
+ `configureStaleWaitNotifications()` / `shutdownStaleWaitMonitor()`. The
250
+ monitor only starts when a task opens a pending reply
251
+ (`sendParentMessage` and the child `runtime` mirror call
252
+ `staleWaitMonitor.kick()`), polls at 1s (unref'd), inspects *task* state
253
+ (not raw agent state), and emits at most one notification per episode with
254
+ full context (task, owner, description, question, pending request id,
255
+ target, target state, elapsed wait, threshold).
256
+ - The parent bridge (`configureStaleWaitBridge` in
257
+ `src/extension/shepherd.ts`) registers a `shepherd.stale.wait` custom
258
+ message renderer and delivers via `sendMessage` with
259
+ `deliverAs: 'followUp'` and **no** turn trigger, so the idle parent sees
260
+ the nudge without being forced to act; the parent only wakes on a real
261
+ `shepherd_watch` completion or on a user interaction.
262
+ - `index.ts` toggles `setStaleWaitSessionActive` on session
263
+ start/shutdown and calls `shutdownStaleWaitMonitor()` on shutdown so the
264
+ timer is released with the session.
265
+ - Reply deadline (already present, Phase 6) settles a task as `blocked` if
266
+ the reply never arrives; the stale reminder is strictly informational and
267
+ never changes task state.
268
+
269
+ Tests: `test/verify-stale-wait.mjs` (threshold crossing, one notification per
270
+ episode, content coverage, reset after a reply, completed + idle (no task)
271
+ agents stay quiet, stale-wait does not settle a task, monitor starts/stops
272
+ with the waiting set, disabled threshold, and bridge delivery + shutdown
273
+ suppression).
274
+
275
+ See `phase8-notes.md` in this directory for the design decisions.
276
+
277
+ ### Phase 9 — Status and TUI integration
278
+
279
+ Complete.
280
+
281
+ Implemented:
282
+
283
+ - `AgentStatus` in `src/core/orchestration.ts` now carries a `task` view
284
+ (`id`, `state`, `waitingSince`/`waitingMs`, `pendingRequestMessageId`,
285
+ `waitingRecipient`, `stale`) computed by a new
286
+ `LifecycleRegistry.taskStatusForAgent()`; process state and task state stay
287
+ independent (an idle process owns the waiting task; a completed task leaves
288
+ the process `done` with no `task` field).
289
+ - `src/extension/shepherd.ts` `status` action exposes a compact public task
290
+ view (`waitingMs`, `pendingRequest` (renamed from the long field),
291
+ `waitingOn`, `stale`) and names both process and task state in the summary
292
+ text. Herdr pane ids remain internal diagnostics (not part of the public
293
+ `status` result).
294
+ - `src/core/herdr.ts` adds `activeTasksByPane()` (in-memory registry only,
295
+ safe for every snapshot tick) and `workingOrWaitingSubagents()`: an owned
296
+ pane is listed when the process is not idle **or** it still owns an open
297
+ tracked task, so an idle child parked on a required reply is never filtered
298
+ out as "done". `workingSubagents()` remains as a working-only projection.
299
+ - The status widget in `index.ts` now renders task state distinctly: the
300
+ header counts `N working · N waiting`, rows show the **task** state while a
301
+ task is open (a working process with a waiting task renders `waiting`),
302
+ and waiting rows append ` ⏳<m>m ←<recipient>` (elapsed wait + who is
303
+ expected to answer). A stale episode renders as `waiting (stale)`. Pane and
304
+ task ids are not painted into rows.
305
+ - README: "Reading status" section documents independent process/task state,
306
+ the task fields, and the widget's idle-but-waiting projection.
307
+
308
+ Tests: `test/verify-status.mjs` (process/task independence for idle+waiting,
309
+ working+running, and no-task agents; reply clears waiting/stale; completion
310
+ removes the task view; doAction public mapping keeps opaque ids and never
311
+ leaks pane data). `test/verify-status-widget.mjs` now covers the idle process
312
+ with a waiting (stale) task, working process with a running task, completed
313
+ task removal, session ownership filtering, and phase-1/2 glyph/width checks
314
+ across three snapshot phases.
315
+
316
+ See `phase9-notes.md` in this directory for the design decisions.
317
+
318
+ "
319
+ ### Phase 10 — Compatibility decisions
320
+
321
+ Decisions (spec "Compatibility" section):
322
+
323
+ - `shepherd_prompt`: **retained as a deprecated compatibility path**, not
324
+ removed. Rationale: removing it would break every transcript and prompt
325
+ template that still references it during migration, and its one-prompt-per-
326
+ turn semantics remain correct for fire-and-forget one-turn messages. The tool
327
+ label, description, promptSnippet, and parameter descriptions now say it is
328
+ deprecated and name the migration target (`shepherd_delegate` +
329
+ `shepherd_watch`), so the model migrates instead of defaulting to it. Its
330
+ completion semantics remain one-child-turn-completion; that is documented,
331
+ not silently changed, so it cannot be mistaken for task tracking.
332
+ - `shepherd_wait`: **hard-removed from the model-facing surface.** Its blocking
333
+ behavior obscured intermediate `waiting` state and encouraged the Shepherd
334
+ to remain visibly working while it waited. Use `shepherd_watch` for
335
+ non-blocking terminal notifications and `shepherd_status` for intermediate
336
+ task state. The legacy prompt path is observed through `shepherd_watch`.
337
+ - `shepherd_watch`, `shepherd_delegate`, `shepherd_message`, `shepherd_done`:
338
+ unchanged (already documented in phases 6–8).
339
+
340
+ ## Important current architecture
341
+
342
+ ### Parent task completion
343
+
344
+ `ensureParentBroker()` in `src/core/lifecycle.ts` lazily creates a parent broker
345
+ and starts a 250ms unref'd monitor. The monitor calls
346
+ `processParentBrokerMessages()`.
347
+
348
+ The parent currently handles these inbox events:
349
+
350
+ - `task_done`: validates and settles the task
351
+ - `message` / `reply`: routed to the parent Pi session as a custom Shepherd message (followUp). Requests with `expectsReply` and replies trigger a parent turn at the next safe boundary; ordinary messages remain passive. A `reply` also resolves the matching pending request on the sender's task and is relayed back to the asker's child inbox
352
+ - `runtime` with `requestOpen`: opens the task's pending request (task → waiting) so a busy recipient's queued question keeps the sender task open
353
+ - Pane disappearance, provider error, and reply-deadline expiry settle tasks as failed / failed / blocked
354
+
355
+ ### Child task completion
356
+
357
+ The child extension receives `PI_SHEPHERD_TASK_ID`. A normal assistant turn
358
+ ending while this variable is present does not write a successful completion
359
+ sidecar and does not shut down the child. The child must call `shepherd_done`.
360
+
361
+ Provider errors still write a sidecar and may be detected by the parent runtime
362
+ monitor.
363
+
364
+ ### Task state invariant
365
+
366
+ Do not weaken this invariant:
367
+
368
+ ```text
369
+ Pi turn ends -> runtime observation only
370
+ agent_end -> runtime observation only
371
+ agent_settled -> runtime observation only
372
+ Herdr idle -> runtime observation only
373
+ shepherd_done -> normal task completion
374
+ ```
375
+
376
+ An idle child may own a `waiting` task.
377
+
378
+ ### Transport behavior
379
+
380
+ The filesystem broker uses:
381
+
382
+ ```text
383
+ <broker>/broker.json
384
+ <broker>/parent/inbox/*.json
385
+ <broker>/agents/<hashed-agent-id>/manifest.json
386
+ <broker>/agents/<hashed-agent-id>/inbox/*.json
387
+ <broker>/processed/*
388
+ <broker>/rejected/*
389
+ <broker>/acks/*
390
+ ```
391
+
392
+ `pollParentInbox()` and `pollChildInbox()` move consumed files to processed
393
+ storage and create acknowledgement markers. Child and parent runtime adapters
394
+ are responsible for making handler operations idempotent.
395
+
396
+ ## Remaining work
397
+
398
+ Consult `tasks.md` for the authoritative checkbox state. Phases 5, 6, 7, 8, 9, and 10
399
+ are complete. Phase 11 (full verification and release readiness) is in progress:
400
+
401
+ ### Phase 11 status (as of this update)
402
+
403
+ - Automated verification: all 15 items done — `npm test` passes cleanly.
404
+ - Live Herdr matrix started. Verified live: Herdr session attach, persistent
405
+ spawn, non-blocking delegation, status during `running`, close of an
406
+ agent with an open task (task settles `cancelled`, artifact finalized,
407
+ exactly one watcher notification), temporary launch-dir cleanup after pane
408
+ confirmation.
409
+ - **Live bug found and fixed:** `writePiLaunchFiles` passed agent-defined
410
+ `tools:` frontmatter straight to `--tools`, which is a full allowlist — so
411
+ children of agents like bundled `scout` (`read,grep,find,ls`) lost the
412
+ child-surface tools `shepherd_message` and `shepherd_done` and could not
413
+ message peers or complete tasks. Fix: `CHILD_SURFACE_TOOLS` is always
414
+ appended to the launch `--tools` list. `test/verify-launch.mjs` asserts the
415
+ allowlist now contains `read,shepherd_message,shepherd_done`.
416
+ - The fix is on disk; a RUNNING Shepherd parent only picks it up on extension
417
+ reload (sessions load TS at startup). Live spawns from a pre-reload parent
418
+ still launch children without the child tools.
419
+ - Completed live matrix items: canonical wait/reply/done flow with stale-wait
420
+ notification, status during `waiting` and after completion, busy-planner
421
+ interleaving, both planner/scout closure directions, task timeout, provider
422
+ model-error observation, pane ownership rejection, temporary-resource cleanup,
423
+ and parent shutdown. After the parent was quit and resumed, the persistent
424
+ survivor remained visible and idle; the previous opaque lifecycle handle was
425
+ rejected by the resumed parent.
426
+
427
+ ## Live-matrix findings (Phase 11)
428
+
429
+ 1. **`--tools` allowlist stripped child tools (fixed).** See "Live bug found and
430
+ fixed" above. `CHILD_SURFACE_TOOLS` is now always appended to the launch
431
+ `--tools` list; live launch scripts confirmed to carry
432
+ `read,grep,find,ls,shepherd_message,shepherd_done`.
433
+ 2. **Child peer addressing requires the opaque agent id.** A child cannot
434
+ address a sibling by display name (`target "planner"` →
435
+ `invalid_target: Unknown child target "planner"`). Peer routing resolves
436
+ only the hashed broker manifest directory for a registered agent id. The
437
+ parent must supply the peer's opaque id in the task text. This is per the
438
+ spec (parent-mediated peer delivery, opaque ids), but is a documentation/
439
+ discovery gap — a child has no way to enumerate sibling ids. Candidate for
440
+ a follow-up (e.g. a parent-injected roster or a peer listing for the child
441
+ surface).
442
+ 3. **`task_done` cannot succeed while a tracked reply is pending.** If a task
443
+ is `waiting` on an outstanding request (`pendingReplyMessageId` set), a
444
+ `shepherd_done` with `completed` is ignored (task stays waiting) until the
445
+ matching reply arrives or the task is settled via `blocked`/`failed`.
446
+ The reply's `replyTo` must equal the pending message id exactly.
447
+
448
+ Phase 11 live verification is complete. The canonical completion flow,
449
+ status-after-completion, planner/scout closure interleavings, busy-planner
450
+ interleaving, timeout, provider/model error observation, pane-ownership
451
+ rejection, temporary-resource cleanup, and parent shutdown were verified live.
452
+ After the parent was quit and resumed, the persistent survivor remained visible
453
+ and idle, while the previous opaque lifecycle handle was rejected by the
454
+ resumed parent. The invalid-model run exposed a readiness race:
455
+ `shepherd_spawn` can report success if Herdr briefly detects the child before pi
456
+ exits; the pane then becomes `unknown` with the model error and completion
457
+ sentinel visible. This is recorded as a follow-up hardening issue, not hidden as
458
+ a clean spawn pass.
459
+
460
+ The worker timeout also demonstrated that a late child message can be rejected
461
+ or accepted as an ordinary message after the task is terminal without changing
462
+ the timed-out result. Child parent aliases are exact lowercase values
463
+ (`shepherd`/`parent`); the live `Shepherd.` attempt was rejected with a structured
464
+ `invalid_target` response. Peer routing likewise requires the opaque registered
465
+ agent id, not a display name.
466
+
467
+ Note: a temporary `.shepherd/config.json` delta setting `staleWaitThreshold: 1`
468
+ was a no-op because the user config's `settingsScope` is `"user"` (project
469
+ deltas are only read when scope is `"project"`); the live stale-wait
470
+ notification fired at the 5-minute default, which validated the monitor anyway.
471
+ The scratch delta file has been removed; user config is untouched.
472
+
473
+ The only remaining live gate is parent shutdown. A fresh idle persistent child
474
+ labelled `p11 parent-shutdown survivor` is currently open for that check. End
475
+ this parent session, start a new Shepherd in the same project, confirm the
476
+ survivor remains visible/idle and that the new session cannot claim or close the
477
+ old session's pane, then update `tasks.md` and the specification status.
478
+
479
+ README and active user-facing documentation now describe the tracked task,
480
+ message, watch, status, compatibility, fieldnote, ownership, and stale-wait
481
+ behavior.
482
+
483
+ ## Useful commands
484
+
485
+ Run focused tests:
486
+
487
+ ```bash
488
+ npm run phase0:test
489
+ npm run task-registry:test
490
+ npm run messaging:test
491
+ npm run child-surface:test
492
+ npm run delegate:test
493
+ npm run task-completion:test
494
+ npm run task-failures:test
495
+ npm run message-routing:test
496
+ npm run task-watchers:test
497
+ npm run stale-wait:test
498
+ npm run compat:test
499
+ npm run status:test
500
+ npm run status-widget:test
501
+ npm run launch:test
502
+ ```
503
+
504
+ Run everything:
505
+
506
+ ```bash
507
+ npm test
508
+ ```
509
+
510
+ Inspect state:
511
+
512
+ ```bash
513
+ git status --short
514
+ git branch --show-current
515
+ git log --oneline --decorate -10
516
+ ```
517
+
518
+ ## Commit history for this feature
519
+
520
+ ```text
521
+ (test: cover phase five task failure paths) <- tip after the Phase 5 test work
522
+ b2c3b59 feat: wire tracked delegation and task completion
523
+ 44b5ae8 feat: add child messaging and completion surface
524
+ e48666d feat: add parent-owned messaging mailbox
525
+ ae68588 feat: add tracked task registry
526
+ a0dab9f test: establish async task lifecycle phase zero
527
+ 986d474 docs: specify agent messaging and task lifecycle
528
+ ```
529
+
530
+ ## Continuation checklist
531
+
532
+ - [x] Read this handoff.
533
+ - [x] Read `spec.md`.
534
+ - [x] Read `plan.md`.
535
+ - [x] Read the current checkbox state in `tasks.md`.
536
+ - [x] Confirm the feature branch is checked out.
537
+ - [x] Run `npm test` before making further changes.
538
+ - [x] Finish remaining Phase 5 tests.
539
+ - [x] Implement Phase 6 parent message routing.
540
+ - [x] Implement Phase 7 task-aware `shepherd_watch`.
541
+ - [x] Implement Phase 8 stale-wait monitoring + `staleWaitThreshold` setting.
542
+ - [x] Implement Phase 9 task-aware status and TUI projection.
543
+ - [x] Make the Phase 10 compatibility decisions (shepherd_prompt deprecated;
544
+ shepherd_watch-only completion observation) and update active documentation.
545
+ - [x] Keep explicit `shepherd_done` completion authoritative.
546
+ - [x] Update this handoff if architecture or compatibility decisions change.
547
+
548
+ ## Spec close-out (2026-09-01)
549
+
550
+ The spec is closed as delivered. Phases 0–10 are implemented; the full test
551
+ suite passes and the `docs/manual-tests/ping-pong-peer-messaging.md` smoke
552
+ test passes live, including watcher-completion delivery before cleanup
553
+ (steered watcher delivery, issue #10).
554
+
555
+ Deferred work is tracked out-of-band rather than left open in this spec:
556
+
557
+ - Preserve useful child output in failure results —
558
+ https://github.com/MikkelKappelPersson/shepherd/issues/11
559
+
560
+ Housekeeping at close: the spec directory moved from
561
+ `docs/specs/001-agent-messaging-and-task-lifecycle` to
562
+ `docs/specs/005-agent-messaging-and-task-lifecycle`, and the branch was
563
+ renamed from `feature/001-agent-messaging-and-task-lifecycle` to
564
+ `feat/005-agent-messaging-and-task-lifecycle`.
@@ -0,0 +1,109 @@
1
+ # Phase 10 — Artifacts, cleanup, compatibility, and documentation (design notes)
2
+
3
+ Phase 10 closes the feature by (a) confirming the artifact lifecycle follows
4
+ task settlement, (b) confirming shutdown/cleanup behavior, (c) making the
5
+ explicit compatibility decisions for `shepherd_prompt` and `shepherd_wait`,
6
+ and (d) updating the active documentation.
7
+
8
+ ## Artifacts
9
+
10
+ Delegate-time allocation was already in place from Phase 4: `delegateAgent`
11
+ reserves exactly one artifact per delegated task (mode `single`, task text
12
+ attached), marks it `running` with the taskId, and attaches a finalizer via
13
+ `attachTaskArtifact`. Ordinary messages never allocate artifacts — only
14
+ `delegateAgent` calls `reserveArtifacts`, so a `shepherd_message` cannot
15
+ create a note. "Open while waiting" is free because the note stays
16
+ `running` (only the frontmatter status changes; agents append freely) and
17
+ `settleTask` is the single authoritative finalization point:
18
+
19
+ - completed -> `completed`
20
+ - blocked (reply deadline, parent failure, ...) -> `failed`
21
+ - failed -> `failed`
22
+ - cancelled (close/cancel) -> `cancelled`
23
+ - timed_out -> `timed-out`
24
+
25
+ `settleTask` guards duplicate settlement (`if (record.settled) return
26
+ record.result!`), so repeated `shepherd_done`/reply/timeout/cancellation
27
+ events cannot double-finalize; `onSettled` runs exactly once and is wrapped
28
+ so an artifact failure never breaks task settlement. Final summaries
29
+ (`result.text`) and errors (`result.error`) are preserved in the note's
30
+ "## Shepherd orchestration" section, so "preserve final summaries and
31
+ errors" holds. Useful request/message references: the finalizer metadata
32
+ carries only the taskId (one note per task, not per chat — messages that
33
+ are correlated with a task use the task's artifact, not a new one).
34
+
35
+ Verified by `verify-delegate.mjs` (allocation + start + cancel finalization)
36
+ and `verify-artifact-sessions.mjs` (all four terminal artifact statuses +
37
+ final output/error sections + in-memory mirroring).
38
+
39
+ ## Cleanup and shutdown
40
+
41
+ All of the following were already implemented in Phases 4–8:
42
+
43
+ - parent shutdown: `shutdownParentBroker` stops the parent mailbox poll and,
44
+ via `stopParentBrokerMonitor`, the stale-wait monitor;
45
+ - child shutdown: the child surface (`shepherd-done.ts`) closes its own
46
+ broker channel on session end;
47
+ - persisted task outcomes survive parent shutdown because they are on-disk
48
+ (session.json + note files);
49
+ - a closing agent cancels its active task (`registry.close()` ->
50
+ `cancelTask()`, rc 130) and the task settlement clears its pending
51
+ requests; duplicate events are idempotent (settleTask guard);
52
+ - created-pane ownership protects foreign panes (existing invariant,
53
+ verified by the launch suite); mailbox consumption is parent-only
54
+ (broker capability is parent-scoped; children get a child capability).
55
+
56
+ No new cleanup code was required in Phase 10; these behaviors are pinned by
57
+ existing tests plus the compat suite's "timeout never touches the task"
58
+ check.
59
+
60
+ ## Compatibility decisions
61
+
62
+ `shepherd_prompt`
63
+ RETAINED, DEPRECATED. Fire-and-forget one-turn submission stays available so
64
+ existing transcripts/templates keep working, but every model-facing field
65
+ (label, description, promptSnippet, message parameter) now states it is a
66
+ deprecated compatibility path and names the migration target
67
+ (`shepherd_delegate` + `shepherd_watch`). Its one-turn completion semantics
68
+ are unchanged and documented, so they are not "silently active": the docs
69
+ say exactly what it does and what to use instead.
70
+
71
+ `shepherd_wait`
72
+ RETAINED, UPGRADED to be task-aware. It now accepts task ids (preferred) or
73
+ legacy prompt ids (unchanged path); mixing them in one call is rejected,
74
+ mirroring shepherd_watch. Task waits:
75
+ - resolve via the task-watcher registry hook (no polling), in input order
76
+ including for tasks that settle out of order;
77
+ - resolve instantly for already-settled tasks;
78
+ - a wait timeout is a wait-level timeout: it rejects with
79
+ LifecycleError('timeout', ...) and removes the watcher. The underlying
80
+ tasks keep running (their own deadlines remain authoritative) and the
81
+ error message points the model at shepherd_watch.
82
+ No task or message depends on the parent calling shepherd_wait; the tool
83
+ description says shepherd_watch is preferred for new orchestration.
84
+
85
+ Implementation: `LifecycleRegistry.waitForTasks(handles, timeoutMs)` plus a
86
+ public `removeWatcher(watcherId)`; `watchTasks` additionally delivers
87
+ already-settled tasks synchronously in input order (Set-callback form, used by
88
+ waitForTasks) while keeping the legacy single-callback contract intact.
89
+
90
+ Verification: `test/verify-compat.mjs` (descriptions/pointers, schema
91
+ accepts both id kinds with flat-object root, in-order resolution, instant
92
+ already-settled, timeout leaves the task untouched, unknown-id rejection,
93
+ doAction public task mapping with returnCode).
94
+
95
+ ## Documentation
96
+
97
+ - README: new "Task delegation workflow" section (delegate ->
98
+ message/watch -> done, with a delegate+watch example and a child-to-child
99
+ message example), the task/message id explanation, idle-child-with-waiting-
100
+ task behavior, followUp vs steer, stale-wait reminders, and explicit
101
+ compatibility status for shepherd_prompt/shepherd_wait.
102
+ - docs/guides/tool-reference.md: delegate/message/done tools added;
103
+ prompt/wait/watch rows updated to reflect deprecation, task ids, and
104
+ mixing rules.
105
+ - docs/reference/dictionary: entries for task, task id, message, message id,
106
+ request/reply correlation, waiting tasks (idle child), and the deprecated
107
+ tools.
108
+ - docs/plans/architecture.md: task/message surface and broker added; the
109
+ "completed implementation" list now lists the tracked-task primitives.