@luminascale/shepherd 0.0.0-stage → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (204) hide show
  1. package/.agents/agents/README.md +12 -0
  2. package/.agents/agents/planner.md +38 -0
  3. package/.agents/agents/reviewer.md +38 -0
  4. package/.agents/agents/scout.md +43 -0
  5. package/.agents/agents/worker.md +33 -0
  6. package/.agents/skills/smoke-test/SKILL.md +155 -0
  7. package/.agents/skills/smoke-test/references/forensics.md +46 -0
  8. package/.agents/skills/smoke-test/scripts/live-smoke-poll.mjs +130 -0
  9. package/.github/workflows/publish.yml +77 -0
  10. package/.pi/prompts/release.md +21 -0
  11. package/.prettierrc +10 -0
  12. package/AGENTS.md +88 -0
  13. package/CONTRIBUTING.md +177 -0
  14. package/LICENSE +201 -0
  15. package/README.md +397 -2
  16. package/docs/README.md +26 -0
  17. package/docs/guides/agent-definitions.md +42 -0
  18. package/docs/guides/diagnostics.md +199 -0
  19. package/docs/guides/installation.md +61 -0
  20. package/docs/guides/tool-reference.md +55 -0
  21. package/docs/manual-tests/ping-pong-peer-messaging.md +147 -0
  22. package/docs/plans/agent-labels.md +257 -0
  23. package/docs/plans/architecture.md +110 -0
  24. package/docs/plans/settings-scope.md +163 -0
  25. package/docs/plans/tool-surface-split.md +278 -0
  26. package/docs/reference/dictionary +26 -0
  27. package/docs/releasing.md +47 -0
  28. package/docs/specs/005-agent-messaging-and-task-lifecycle/baseline.md +115 -0
  29. package/docs/specs/005-agent-messaging-and-task-lifecycle/handoff.md +564 -0
  30. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase10-notes.md +109 -0
  31. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase6-notes.md +69 -0
  32. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase7-notes.md +100 -0
  33. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase8-notes.md +127 -0
  34. package/docs/specs/005-agent-messaging-and-task-lifecycle/phase9-notes.md +104 -0
  35. package/docs/specs/005-agent-messaging-and-task-lifecycle/plan.md +1093 -0
  36. package/docs/specs/005-agent-messaging-and-task-lifecycle/spec.md +583 -0
  37. package/docs/specs/005-agent-messaging-and-task-lifecycle/tasks.md +838 -0
  38. package/docs/specs/006-shepherd-tool-rendering/plan.md +175 -0
  39. package/docs/specs/006-shepherd-tool-rendering/spec.md +469 -0
  40. package/index.ts +602 -0
  41. package/package.json +112 -4
  42. package/scripts/capture-system-prompt.mjs +45 -0
  43. package/scripts/extract-pi-system-prompt.mjs +277 -0
  44. package/scripts/host-impact.mjs +74 -0
  45. package/scripts/resolve-agent.mjs +43 -0
  46. package/scripts/show-shepherd-orchestrator.mjs +58 -0
  47. package/scripts/show-shepherd-prompt.mjs +96 -0
  48. package/scripts/verify-suites.mjs +67 -0
  49. package/specs/001-artifacts/spec.md +436 -0
  50. package/specs/002-simplify-agent-orchestration/plan.md +287 -0
  51. package/specs/002-simplify-agent-orchestration/spec.md +323 -0
  52. package/specs/002-simplify-agent-orchestration/tasks.md +212 -0
  53. package/specs/003-orchestrator-session-artifacts/plan.md +396 -0
  54. package/specs/003-orchestrator-session-artifacts/spec.md +453 -0
  55. package/specs/003-orchestrator-session-artifacts/tasks.md +141 -0
  56. package/specs/004-asynchronous-prompt-watchers/plan.md +139 -0
  57. package/specs/004-asynchronous-prompt-watchers/spec.md +202 -0
  58. package/specs/004-asynchronous-prompt-watchers/tasks.md +126 -0
  59. package/specs/005-per-workspace-settings/plan.md +230 -0
  60. package/specs/005-per-workspace-settings/spec.md +368 -0
  61. package/specs/005-per-workspace-settings/tasks.md +178 -0
  62. package/specs/006-dual-runtime-herdr-durable/intent.md +86 -0
  63. package/specs/006-dual-runtime-herdr-durable/journal.md +80 -0
  64. package/specs/006-dual-runtime-herdr-durable/plan.md +242 -0
  65. package/specs/006-dual-runtime-herdr-durable/spec.md +389 -0
  66. package/specs/006-dual-runtime-herdr-durable/tasks.md +199 -0
  67. package/specs/007-unwatched-completion-notice/journal.md +145 -0
  68. package/specs/007-unwatched-completion-notice/plan.md +186 -0
  69. package/specs/007-unwatched-completion-notice/spec.md +284 -0
  70. package/specs/007-unwatched-completion-notice/tasks.md +147 -0
  71. package/specs/008-remote-machine-sheep/intent.md +174 -0
  72. package/specs/008-remote-machine-sheep/journal.md +57 -0
  73. package/specs/008-remote-machine-sheep/prerequisites.md +219 -0
  74. package/specs/009-hc-runtime/intent.md +44 -0
  75. package/specs/009-hc-runtime/plan.md +49 -0
  76. package/specs/009-hc-runtime/spec.md +118 -0
  77. package/specs/009-hc-runtime/tasks.md +62 -0
  78. package/specs/010-unseen-watch-parity/intent.md +57 -0
  79. package/specs/010-unseen-watch-parity/journal.md +172 -0
  80. package/specs/010-unseen-watch-parity/plan.md +74 -0
  81. package/specs/010-unseen-watch-parity/spec.md +191 -0
  82. package/specs/010-unseen-watch-parity/tasks.md +119 -0
  83. package/specs/011-hc-background-poll/intent.md +57 -0
  84. package/specs/011-hc-background-poll/journal.md +95 -0
  85. package/specs/011-hc-background-poll/plan.md +87 -0
  86. package/specs/011-hc-background-poll/spec.md +141 -0
  87. package/specs/011-hc-background-poll/tasks.md +83 -0
  88. package/specs/012-steer-default/intent.md +12 -0
  89. package/specs/012-steer-default/journal.md +40 -0
  90. package/specs/012-steer-default/plan.md +7 -0
  91. package/specs/012-steer-default/spec.md +42 -0
  92. package/specs/012-steer-default/tasks.md +6 -0
  93. package/specs/013-opencode-shepherd/handoff-t011.md +94 -0
  94. package/specs/013-opencode-shepherd/intent.md +59 -0
  95. package/specs/013-opencode-shepherd/plan.md +204 -0
  96. package/specs/013-opencode-shepherd/spec.md +258 -0
  97. package/specs/013-opencode-shepherd/tasks.md +263 -0
  98. package/src/core/artifact-sessions.ts +835 -0
  99. package/src/core/config.ts +329 -0
  100. package/src/core/discovery.ts +342 -0
  101. package/src/core/herdr.ts +1 -0
  102. package/src/core/host.ts +77 -0
  103. package/src/core/ids.ts +78 -0
  104. package/src/core/lifecycle.ts +1 -0
  105. package/src/core/messaging.ts +1 -0
  106. package/src/core/migration.ts +55 -0
  107. package/src/core/orchestration.ts +1 -0
  108. package/src/core/types.ts +143 -0
  109. package/src/core/unseen-completions.ts +114 -0
  110. package/src/extension/cli.ts +417 -0
  111. package/src/extension/config.ts +1 -0
  112. package/src/extension/settings-ui.ts +296 -0
  113. package/src/extension/shepherd-done.ts +768 -0
  114. package/src/extension/shepherd.ts +2548 -0
  115. package/src/extension/system-prompt.ts +38 -0
  116. package/src/extension/unseen-completions.ts +237 -0
  117. package/src/opencode/actions.ts +794 -0
  118. package/src/opencode/bridge.ts +777 -0
  119. package/src/opencode/display.ts +36 -0
  120. package/src/opencode/fieldnotes.ts +60 -0
  121. package/src/opencode/host.ts +62 -0
  122. package/src/opencode/orphans.ts +162 -0
  123. package/src/opencode/server.ts +458 -0
  124. package/src/runtimes/backend.ts +119 -0
  125. package/src/runtimes/detect.ts +102 -0
  126. package/src/runtimes/hc/hc.ts +527 -0
  127. package/src/runtimes/hc/runtime.ts +1898 -0
  128. package/src/runtimes/herdr/herdr.ts +912 -0
  129. package/src/runtimes/herdr/lifecycle.ts +1712 -0
  130. package/src/runtimes/herdr/messaging.ts +703 -0
  131. package/src/runtimes/herdr/orchestration.ts +1583 -0
  132. package/src/runtimes/herdr/runtime.ts +224 -0
  133. package/src/runtimes/runtime.ts +133 -0
  134. package/src/runtimes/settle-observations.ts +114 -0
  135. package/test/fixtures/context-project/AGENTS.md +2 -0
  136. package/test/fixtures/home/.agents/agents/same.md +6 -0
  137. package/test/fixtures/home/.agents/agents/shared-only.md +6 -0
  138. package/test/fixtures/home/.pi/agent/agents/invalid-thinking.md +6 -0
  139. package/test/fixtures/home/.pi/agent/agents/model-empty.md +7 -0
  140. package/test/fixtures/home/.pi/agent/agents/model-explicit.md +7 -0
  141. package/test/fixtures/home/.pi/agent/agents/omit-context-absent.md +5 -0
  142. package/test/fixtures/home/.pi/agent/agents/omit-context-false.md +6 -0
  143. package/test/fixtures/home/.pi/agent/agents/omit-context-invalid.md +6 -0
  144. package/test/fixtures/home/.pi/agent/agents/omit-context-true.md +6 -0
  145. package/test/fixtures/home/.pi/agent/agents/omit-false.md +6 -0
  146. package/test/fixtures/home/.pi/agent/agents/omit-invalid.md +6 -0
  147. package/test/fixtures/home/.pi/agent/agents/omit-pi-docs.md +7 -0
  148. package/test/fixtures/home/.pi/agent/agents/omit-true.md +6 -0
  149. package/test/fixtures/home/.pi/agent/agents/only-user.md +9 -0
  150. package/test/fixtures/home/.pi/agent/agents/same.md +6 -0
  151. package/test/fixtures/proj/.agents/agents/proj-shared.md +6 -0
  152. package/test/fixtures/proj/.pi/agents/project-pi.md +6 -0
  153. package/test/fixtures/proj/.pi/agents/same.md +6 -0
  154. package/test/helpers/fake-clock.mjs +39 -0
  155. package/test/helpers/feature-contracts.mjs +44 -0
  156. package/test/helpers/test-utils.mjs +44 -0
  157. package/test/verify-artifact-sessions.mjs +258 -0
  158. package/test/verify-backend.mjs +297 -0
  159. package/test/verify-child-mislabel.mjs +135 -0
  160. package/test/verify-child-surface.mjs +321 -0
  161. package/test/verify-cli.mjs +197 -0
  162. package/test/verify-command-ux.mjs +100 -0
  163. package/test/verify-compat.mjs +32 -0
  164. package/test/verify-cross-host-notes.mjs +128 -0
  165. package/test/verify-delegate.mjs +161 -0
  166. package/test/verify-detect.mjs +151 -0
  167. package/test/verify-discovery.mjs +239 -0
  168. package/test/verify-done-nudge.mjs +238 -0
  169. package/test/verify-effective-prompt.mjs +98 -0
  170. package/test/verify-hc-matrix.mjs +246 -0
  171. package/test/verify-hc-poll.mjs +260 -0
  172. package/test/verify-host-docs.mjs +66 -0
  173. package/test/verify-host-impact.mjs +49 -0
  174. package/test/verify-ids.mjs +141 -0
  175. package/test/verify-launch.mjs +183 -0
  176. package/test/verify-message-routing.mjs +560 -0
  177. package/test/verify-messaging.mjs +303 -0
  178. package/test/verify-migration-notice.mjs +74 -0
  179. package/test/verify-multi-wait.mjs +42 -0
  180. package/test/verify-opencode-display.mjs +47 -0
  181. package/test/verify-opencode-fieldnotes.mjs +121 -0
  182. package/test/verify-opencode-notice.mjs +77 -0
  183. package/test/verify-opencode-nudge.mjs +116 -0
  184. package/test/verify-opencode-orphans.mjs +160 -0
  185. package/test/verify-opencode-surface.mjs +107 -0
  186. package/test/verify-packaging.mjs +82 -0
  187. package/test/verify-parent-surface.mjs +118 -0
  188. package/test/verify-phase0.mjs +95 -0
  189. package/test/verify-registry.mjs +124 -0
  190. package/test/verify-rename-migration.mjs +111 -0
  191. package/test/verify-rendering.mjs +379 -0
  192. package/test/verify-session-boundary.mjs +146 -0
  193. package/test/verify-settings.mjs +260 -0
  194. package/test/verify-stale-wait.mjs +339 -0
  195. package/test/verify-status-widget.mjs +377 -0
  196. package/test/verify-status.mjs +172 -0
  197. package/test/verify-task-completion.mjs +156 -0
  198. package/test/verify-task-failures.mjs +201 -0
  199. package/test/verify-task-mislabel.mjs +143 -0
  200. package/test/verify-task-registry.mjs +194 -0
  201. package/test/verify-task-watchers.mjs +217 -0
  202. package/test/verify-tool-schemas.mjs +120 -0
  203. package/test/verify-unseen-completions.mjs +1211 -0
  204. package/test/verify-watchers.mjs +181 -0
@@ -0,0 +1,178 @@
1
+ # Tasks: Per-workspace self-contained Shepherd settings
2
+
3
+ Implement in order. Keep the current settings behavior passing until the new
4
+ resolution model is fully covered. Do not change lifecycle tool schemas or
5
+ opaque id contracts as part of this work.
6
+
7
+ ## Phase 1 — Configuration core
8
+
9
+ - [x] **1.1 Define the new settings shape**
10
+ - Replace `settingsScope`/`ConfigScope` with effective `projectScope`.
11
+ - Keep all existing Shepherd value fields and defaults unchanged.
12
+ - Define separate project-overridable and user-only field lists.
13
+ - Keep `confirmProjectAgents` user-owned as a security control.
14
+
15
+ - [x] **1.2 Add project activation parsing**
16
+ - Parse boolean `projectScope` only from project configuration.
17
+ - Honor legacy project `settingsScope: "project"` as active.
18
+ - Give a real boolean flag precedence over the legacy string.
19
+ - Ignore user-file scope flags for resolution.
20
+ - Treat absent, invalid, and malformed project activation as inactive.
21
+
22
+ - [x] **1.3 Implement per-workspace resolution**
23
+ - Load user values over built-in defaults.
24
+ - Resolve the project file for the supplied cwd only.
25
+ - When active, resolve project values over built-in defaults without user
26
+ values leaking into missing project fields.
27
+ - Preserve the trusted user-layer `confirmProjectAgents` value even when
28
+ project scope is active.
29
+ - Return `projectScope: false` for user-layer results and no-cwd results.
30
+
31
+ - [x] **1.4 Implement self-contained persistence**
32
+ - Make user saves write values only.
33
+ - Make project saves write every project-overridable field plus
34
+ `projectScope`; never write `confirmProjectAgents` to the project file.
35
+ - Preserve `created` reporting and mtime-cache updates.
36
+ - Remove delta comparison and `loadProjectDelta()` after callers migrate.
37
+
38
+ - [x] **1.5 Implement parked-value lifecycle**
39
+ - Add `loadProjectFileValues()` with built-in fallback for missing fields.
40
+ - Add `deactivateProjectScope()`.
41
+ - Preserve values on deactivation.
42
+ - Drop legacy `settingsScope` when normalizing a project file.
43
+ - Never delete the project file during deactivation.
44
+
45
+ - [x] **1.6 Preserve existing migrations**
46
+ - Keep `settings.json` → `config.json` migration behavior.
47
+ - Keep timeout millisecond-to-minute migration behavior.
48
+ - Ensure user saves strip both old and new scope keys.
49
+
50
+ ## Phase 2 — Settings UI
51
+
52
+ - [x] **2.1 Convert the scope menu item**
53
+ - Rename the item id to `projectScope`.
54
+ - Display `project`, `user (project file dormant)`, or `user` as applicable.
55
+ - Ensure the current display value is included in the cycle values.
56
+
57
+ - [x] **2.2 Implement activation and deactivation actions**
58
+ - Activation reads parked values and writes a complete project config with
59
+ `projectScope: true`.
60
+ - Missing project files are created with a creation notification.
61
+ - User selection calls `deactivateProjectScope()` and preserves the file.
62
+ - Resync the menu after each successful scope change.
63
+
64
+ - [x] **2.3 Route value edits by effective scope**
65
+ - Save user values only while project scope is inactive.
66
+ - Save complete project values while project scope is active.
67
+ - Always save `confirmProjectAgents` to the user file.
68
+ - Label or describe `confirmProjectAgents` as user-only in the menu.
69
+ - Preserve the existing fieldnotes and project-agent notifications.
70
+ - Preserve error notifications for failed writes.
71
+
72
+ ## Phase 3 — Runtime integration
73
+
74
+ - [x] **3.1 Migrate config API callers**
75
+ - Remove `settingsScope`, `ConfigScope`, and `loadProjectDelta` references.
76
+ - Verify all `loadSettings()` calls pass the appropriate cwd.
77
+ - Store the owning workspace cwd on agent/task records.
78
+ - Replace `process.cwd()` in reply-deadline and stale-wait settings lookups
79
+ with the owning agent/task cwd.
80
+ - Keep session-start fieldnote initialization cwd-aware.
81
+
82
+ - [x] **3.2 Verify effective runtime settings**
83
+ - Confirm delegation/prompt timeout uses the active workspace settings.
84
+ - Confirm stale-wait threshold uses the active workspace settings.
85
+ - Confirm agent scope and bundled-agent inclusion use the active workspace.
86
+ - Confirm project-agent confirmation uses the trusted user-layer setting;
87
+ project config cannot disable it.
88
+ - Confirm tab/process retention and emoji rendering still use settings.
89
+
90
+ ## Phase 4 — Config and integration tests
91
+
92
+ - [x] **4.1 Rewrite the config-layer test**
93
+ - Test defaults and `projectScope: false`.
94
+ - Test values-only user persistence.
95
+ - Test ignored and stripped user scope keys.
96
+ - Test dormant false/keyless/invalid/malformed project files.
97
+ - Test active project isolation from user values.
98
+ - Test per-workspace independence.
99
+ - Test legacy project string compatibility.
100
+
101
+ - [x] **4.2 Test full project writes**
102
+ - Verify project saves contain all project-overridable fields plus the
103
+ boolean flag, and omit `confirmProjectAgents`.
104
+ - Verify fresh project creation.
105
+ - Verify missing active-project fields use built-in defaults.
106
+ - Verify explicit runtime values still take precedence where applicable.
107
+
108
+ - [x] **4.3 Test parked values and deactivation**
109
+ - Verify activation preserves dormant project values.
110
+ - Verify deactivation writes `false` rather than deleting the file.
111
+ - Verify deactivation preserves values and removes legacy scope keys.
112
+ - Verify repeated deactivation and missing-file deactivation are no-ops.
113
+ - Verify keyless and legacy-string object deactivation normalizes to
114
+ `projectScope: false`.
115
+ - Verify invalid parked values fall back to built-in defaults and unknown
116
+ values are normalized away on a complete project save.
117
+
118
+ - [x] **4.4 Update dependent fixtures**
119
+ - Update status-widget fixtures to use `projectScope: true`.
120
+ - Update command-UX fixtures to use the new file shapes.
121
+ - Add/adjust fieldnotes, timeout, stale-wait, discovery, confirmation, and
122
+ widget coverage.
123
+ - Add a child/task cwd test where the parent process cwd differs from the
124
+ workspace cwd.
125
+ - Cover timeout unit migration, stale thresholds at/below zero, non-object
126
+ project files, and filesystem failure paths.
127
+
128
+ ## Phase 5 — Repository hygiene and documentation
129
+
130
+ - [x] **5.1 Update ignore rules**
131
+ - Keep `.shepherd` runtime sessions ignored.
132
+ - Allow an explicit `.shepherd/config.json` to be tracked.
133
+ - Verify generated fieldnotes remain ignored.
134
+ - Verify `.shepherd/config.json` is trackable in a temporary git repository
135
+ and remains present after a fresh checkout.
136
+
137
+ - [x] **5.2 Update user documentation**
138
+ - Rewrite the README settings section for the new model.
139
+ - Document the per-workspace boolean activation flag.
140
+ - Document self-contained project files and built-in fallback behavior.
141
+ - Document the migration behavior and repository-commit guidance.
142
+ - Add the repository-agent security boundary: project files cannot disable
143
+ the user-owned confirmation gate.
144
+
145
+ - [x] **5.3 Update maintainer documentation**
146
+ - Update `AGENTS.md`.
147
+ - Mark the old settings-scope plan as superseded or link to this spec.
148
+ - Add this spec/plan to any applicable documentation index.
149
+
150
+ ## Phase 6 — Verification and completion
151
+
152
+ - [x] **6.1 Run automated verification**
153
+ - Run `git diff --check`.
154
+ - Run `npm run settings:test`.
155
+ - Run `npm test`.
156
+
157
+ - [ ] **6.2 Perform live settings verification**
158
+ - Test creation, activation, edit, deactivation, and reactivation through
159
+ `/shepherd settings`.
160
+ - Test two independent workspaces.
161
+ - Test hand-edited mtime reload with a changed file mtime; document the
162
+ same-mtime cache limitation or strengthen cache invalidation if required.
163
+ - Test fieldnotes session snapshot behavior.
164
+ - Test active project files with missing and invalid fields.
165
+ - Test timeout and stale-wait resolution with a child/task cwd different from
166
+ the parent process cwd.
167
+
168
+ - [x] **6.3 Review migration and security behavior**
169
+ - Confirm keyless legacy project files do not silently activate.
170
+ - Confirm committed project settings are clearly documented as
171
+ repository-controlled.
172
+ - Confirm no user scope pointer can affect another workspace.
173
+ - Confirm a committed project file cannot set `confirmProjectAgents: false`
174
+ to suppress project-agent confirmation.
175
+
176
+ - [x] **6.4 Complete the specification**
177
+ - Update `spec.md` status after implementation and verification.
178
+ - Record any intentional deviations or migration caveats.
@@ -0,0 +1,86 @@
1
+ # Intent — Dual-runtime Shepherd (Herdr + Durable)
2
+
3
+ ## Why this exists
4
+
5
+ pi-shepherd already works: an orchestrator herds one specialist per task in
6
+ visible Herdr panes. One team runs test-driver agents against the app under
7
+ test, and on crash a different-context reviewer takes over. That workflow
8
+ must not break.
9
+
10
+ A second need has appeared alongside it: a company test agent that runs
11
+ overnight. Multiple users, different models per user (free local vs. company
12
+ Copilot), test plans submitted in the evening, reports read in the morning.
13
+ Herdr panes alone cannot survive a 2am reboot, deduplicate a retried submit,
14
+ account usage per model, or run headless. That is what durable execution is
15
+ for.
16
+
17
+ This spec exists to serve **both** without forking the project into two
18
+ mental models.
19
+
20
+ ## Who it serves
21
+
22
+ 1. **Today's Herdr user** — drives shepherd interactively, watches sheep
23
+ work, reads panes. Fieldnotes are the agents' shared memory, not a
24
+ human deliverable. Must notice zero change by default.
25
+ 2. **Overnight test owner** — submits a test plan, sleeps, reads a report.
26
+ Needs resume-after-crash, no double test runs, per-user model routing,
27
+ and a readable morning artifact. Never opens a Herdr pane if they do not
28
+ want to.
29
+ 3. **Maintainer (us)** — one codebase, one tool surface, one agent format.
30
+ No parallel `herdr_*` vs `durable_*` universe to document and test twice.
31
+
32
+ ## What success looks like
33
+
34
+ - The same orchestrator prompt — _"run plan A with test-driver, on crash
35
+ hand fieldnote + logs to crash-reviewer, return both"_ — runs unchanged
36
+ on Herdr by day and on durable by night.
37
+ - A user with a free local model and a user with a Copilot subscription use
38
+ the same agents; only the Shepherd session's default model differs.
39
+ - A host reboot at 2am is a non-event: reopen, `resume()`, agent memory
40
+ intact, HTML report still renders. Same `requestId` never runs the test
41
+ suite twice.
42
+ - Morning handoff is a templated HTML file owned by the app, rendered
43
+ from job state. `shepherd.md` is never the human report — it is the
44
+ shepherd/sheep working memory that the report was built from.
45
+ - A newcomer reading the README still learns "shepherd herds sheep in
46
+ Herdr" first. Durable appears as one settings row: _night pen_.
47
+
48
+ ## Guiding values
49
+
50
+ 1. **Herdr-first.** Visibility is the product. Panes, `herd/spawn/read`,
51
+ 🐑 widget stay default. Fieldnotes stay the agents' working memory —
52
+ they survive compaction and restarts and stay the agents' truth —
53
+ never the human handoff.
54
+ 2. **Same shepherd, different field.** Durable is a backend, not a rebrand.
55
+ Tool names, opaque IDs, agent Markdown, and handoff habits do not fork.
56
+ 3. **No magic.** Shepherd stays low-level and explicit:
57
+ `spawn -> delegate -> watch -> done`. No hidden workflow modes, no
58
+ invented completions, no closing panes we did not create.
59
+ 4. **Agent memory, not human handoff.** `shepherd.md` + per-agent notes
60
+ are for shepherd/sheep to track what is going on. If a detail only
61
+ exists inside SQLite, it does not count as shared agent context — docs
62
+ mirror to files. Human handoff is a separate templated HTML file with
63
+ nothing to do with shepherd internals.
64
+ 5. **Boring by default.** No SQLite required, no new deps for Herdr-only
65
+ installs, no migration, no rewrite. Durable is opt-in and clearly
66
+ experimental.
67
+
68
+ ## Deliberate temptations to resist
69
+
70
+ - Adding a second tool family (`durable_spawn`, ...). One surface, backend
71
+ switches underneath.
72
+ - Leaking durable vocabulary (`conversationId`, `pi.live`) into user docs
73
+ or tool results.
74
+ - Making panes write-capable viewers of durable state (dual-write). Panes
75
+ in `auto` mode tail; execution stays in one place.
76
+ - Marking test-driving tools `replay: safe` to make demos pass. Rerunning
77
+ side effects silently is worse than reporting `interrupted`.
78
+ - Letting a committed project config disable trust gates or force durable
79
+ on for everyone. Backend choice follows the same user-owned safety rules
80
+ as agents.
81
+
82
+ ## When we are done
83
+
84
+ We can delete this file's doubts: shepherd still means herding inspectable
85
+ sheep — panes by day, committed views by night — it just also works while
86
+ we sleep.
@@ -0,0 +1,80 @@
1
+ # Journal — 006 dual-runtime Shepherd (Herdr + Durable)
2
+
3
+ ## Status — 2026-10-18
4
+
5
+ - Spec restructured to the agreed `src/runtimes/` layout, with review
6
+ fixes folded in (`c4f0daa`, `65c0011`).
7
+ - **Scope decision: implement Phase 0 only.** Land the 2-runtimes-1-core
8
+ boundary (runtime interface + `src/core/ids.ts` + Herdr rename-only move
9
+ + temporary shims) with zero behavior change.
10
+ - Phases 1–4 (durable backend, message/status parity, fieldnotes bridge,
11
+ auto viewer, docs/live proof) are **not** implemented at this point and
12
+ remain future work on this spec.
13
+ - **Merge condition:** when Phase 0 is green — `npm test` plus live Herdr
14
+ checks (idle spawn, non-blocking prompt, single + multi-wait, status,
15
+ close, ownership refusal, focus preservation; the project-agent gate is
16
+ covered by the suite rather than live, to avoid mutating user settings)
17
+ — merge `feat/006-dual-runtime-herdr-durable` back into `main`.
18
+ Unfinished Phase 1+ work must not be carried to `main`.
19
+ - Herdr-only install stays dependency-clean: no `pi-durable` /
20
+ `@earendil-works/chord` added in Phase 0.
21
+
22
+ ## Decisions
23
+
24
+ - Layout: `src/core/` = shared domain only; `src/runtimes/runtime.ts` =
25
+ `ShepherdRuntime` interface; `src/runtimes/herdr/` = Herdr
26
+ implementation; `src/runtimes/durable/` = future; `src/extension/` =
27
+ pi-coding-agent surface.
28
+ - Phase 0 keeps re-export shims at the old
29
+ `src/core/{herdr,orchestration,lifecycle,messaging}.ts` and
30
+ `src/extension/config.ts` paths so existing `verify-*.mjs` imports stay
31
+ untouched. Shims are removed (with import-path updates) in task 1.5,
32
+ i.e. after a durable path exists.
33
+ - The settings store moves to `src/core/config.ts` so runtimes never
34
+ import the extension layer; `/shepherd settings` UI stays in
35
+ `src/extension/`.
36
+ - The runtime interface is lifecycle-only. Discovery stays shared;
37
+ `herd`/`agents`/`prune`/widget keep runtime introspection accessors.
38
+
39
+ ## Log
40
+
41
+ - 2026-10-18: spec intent/plan/tasks landed (`4960f35`).
42
+ - 2026-10-18: adopted `src/runtimes/` layout + shim policy (`c4f0daa`).
43
+ - 2026-10-18: review fixes — ID codec task, one-contract wording, tool
44
+ list, task 1.5 scope (`65c0011`).
45
+ - 2026-10-18: scope narrowed to Phase 0 until green; merge to `main` on
46
+ success.
47
+ - 2026-10-18: Phase 0 implementation (tasks 0.1–0.3) via worker sheep:
48
+ `src/runtimes/runtime.ts` (pure 7-method contract), `src/core/ids.ts`
49
+ (byte-identical mint/validation, wired into orchestration/lifecycle),
50
+ rename-only moves into `src/runtimes/herdr/` + `src/core/config.ts`,
51
+ `export *` shims at old paths, `src/runtimes/herdr/runtime.ts` adapter,
52
+ derived `delegate`/`close` routed in `doAction`, new `verify-ids.mjs`.
53
+ Full suite green (465 PASS, 31 sub-suites).
54
+ - 2026-10-18: review (reviewer-02): no critical findings; rename-only
55
+ proof, shim export coverage, ID parity and routing equivalence all
56
+ verified. Findings triaged: `read()` trailing-newline off-by-one and
57
+ contract-doc clarity fixed (reviewer-03 delta): `watch` rejection
58
+ semantics, `waitingOn` as opaque `MessageId`, terminal proc states. The
59
+ unrouted `spawn`/`message`/`status`/`read` adapter methods and the
60
+ single umbrella routing comment are recorded as an explicit Phase 0
61
+ deferral in `spec.md` and `tasks.md` 0.3 (partial routing is correct
62
+ under the zero-behavior-change rule). Residuals accepted as pre-existing
63
+ or task-1.5 scope: `closeAgent` record cleanup may race pane teardown;
64
+ the `src/extension/config.ts` shim bypasses `verify-settings.mjs`'s
65
+ cache-busting query until 1.5.
66
+ - 2026-10-18: live Herdr smoke (`/tmp/phase0-live-smoke.mjs`, not
67
+ committed): worker/`reviewer` sheep over the real Herdr server — idle
68
+ spawn, focus preservation, non-blocking prompt (1.3 s), delegate +
69
+ single wait, concurrent multi-wait across two sheep, status projection,
70
+ close, ownership refusal (`Refusing to close an unowned pane.`), raw
71
+ pane id rejected as handle. The smoke exposed one real adapter bug:
72
+ `spawn` resolved to the internal handle record instead of the opaque id
73
+ string; fixed, and the rerun is **13/13 green**. Project-agent gate is covered by the
74
+ suite (`verify-discovery`/`verify-command-ux`) rather than live, to
75
+ avoid mutating the user settings file. Stale pane registrations left by
76
+ the pre-existing `closeAgent` cleanup race were removed with `shepherd
77
+ prune`.
78
+ - 2026-10-18: **Phase 0 landed on `main`** (`42182eb`, fast-forward from
79
+ `feat/006-dual-runtime-herdr-durable`); `npm test` green on `main`
80
+ (465 PASS, 32 sub-suites). Phases 1–4 remain future work on this spec.
@@ -0,0 +1,242 @@
1
+ # Implementation Plan: Dual-runtime Shepherd (Herdr + Durable)
2
+
3
+ This plan implements `spec.md` + `intent.md` by refactoring pi-shepherd into
4
+ **2 runtimes, 1 core**: one pure orchestration core with two executors
5
+ (Herdr panes today, durable conversations/tasks tomorrow) behind the
6
+ unchanged `shepherd_*` surface.
7
+
8
+ ## Target surface
9
+
10
+ No model-facing change. Same tools, same opaque IDs, same agent Markdown:
11
+
12
+ ```text
13
+ shepherd_spawn(agent, {cwd, label, placement}) -> agent ID
14
+ shepherd_delegate(agentID, task, {requestId}) -> task ID
15
+ shepherd_message(target, message, {taskId, expectsReply, replyTo}) -> message ID
16
+ shepherd_watch({id: taskID | taskIDs}) -> completion notification(s)
17
+ shepherd_status(agentID) -> {proc, task, waitingOn, stale}
18
+ shepherd_close({id: agentID})
19
+ shepherd_read(agent, {lines}) -> tail
20
+ child: shepherd_message(parent|peer) + shepherd_done (only normal success)
21
+ ```
22
+
23
+ Backend selects inside, never as `durable_*` tools:
24
+
25
+ ```text
26
+ backend: "herdr" (default) | "durable" | "auto"
27
+ ```
28
+
29
+ `auto` = execute on durable, mirror read-only tail panes in Herdr when
30
+ reachable. Panes never write in `auto`.
31
+
32
+ ## Design decisions
33
+
34
+ - One behavioral contract, two implementations: `delegate = submit +
35
+ watch`, `expectsReply -> waiting`, `staleWait` info-only + `timeout ->
36
+ blocked`, 1-turn `done-nudge` budget, and the close-ownership rule are
37
+ specified once and implemented per runtime — Herdr keeps today's code,
38
+ durable reimplements the contract. Shared pure helpers (`discovery.ts`,
39
+ `ids.ts`, fieldnotes paths, settings store, tool-schema types) live in
40
+ `src/core/`, which imports neither the `herdr` CLI nor `pi-durable`.
41
+ - Runtimes are thin executors of one interface (`src/runtimes/runtime.ts`):
42
+ `spawn / delegate / message / watch / status / close / read`.
43
+ Herdr-runtime is moved current code; durable-runtime is new over
44
+ `Harness / createConversation / submit(requestId) / wait / abort /
45
+ viewState / watchEvents / taskGraph`.
46
+ - Layout: `src/core/` keeps shared domain only (`discovery.ts`,
47
+ `artifact-sessions.ts`, `types.ts`, new `ids.ts`, settings store moved
48
+ from `extension/config.ts` to `core/config.ts`). `src/runtimes/herdr/`
49
+ receives `herdr.ts + orchestration.ts + lifecycle.ts` plus `messaging.ts`
50
+ as support, rename-only; `src/runtimes/durable/` is new;
51
+ `src/extension/` remains the pi adapter. Temporary re-export shims at
52
+ the old `src/core/*` and `src/extension/config.ts` paths keep the Phase 0
53
+ suite untouched; the Phase 1 import-path task drops them.
54
+ - Two adapters, one core: `src/extension/*` stays on
55
+ `ExtensionAPI from '@earendil-works/pi-coding-agent'`; new
56
+ `src/runtimes/durable/extension.ts` exposes `defineExtension({name:"shepherd"})`
57
+ with `(args, api, chordCtx)` signatures. Shared agent frontmatter
58
+ (`name/description/tools/model/thinking/omit-*`) maps to
59
+ `configure({model, thinkingLevel, tools, cwd})` on durable.
60
+ - One sheep = one owned durable conversation
61
+ (`ownership:{kind:"task", taskId}`, fresh `pi.provider` identity,
62
+ `background:true` anchors for `stayOpen` persistent workers). Mirrors
63
+ pi-durable `22-subagent-foreground / 23-subagent-background`.
64
+ - `shepherd.md` + per-agent notes are **agent memory, not human report**:
65
+ shepherd/sheep working memory surviving compaction, waiting, close,
66
+ restart. Durable `ShepherdNotes` doc mirrors them in the same `commit()`;
67
+ files remain `tail -f`-able via `env`. Morning human handoff is a
68
+ separate app-owned templated HTML file rendered from the job report doc —
69
+ never `shepherd.md`, never a doc dump, never `/export`. Out of shepherd
70
+ scope except to guarantee the memory it was built from survived.
71
+ - Replay: app-driving tools are NOT `replay:"safe"`. Crash-interrupt yields
72
+ `interrupted` + partial output, never silent rerun. Only idempotent
73
+ readers may opt in with keyed `executeTool` reattach.
74
+ - `@earendil-works/pi-durable` + `@earendil-works/chord` are optional
75
+ peerDeps. Herdr-only installs load no durable code, grow no size,
76
+ require no SQLite.
77
+ - `placement` ignored for durable execution, honored for `auto` viewer
78
+ panes (viewers are `stayOpen:false`, input-refusing).
79
+ - `confirmProjectAgents` stays user-owned; backend choice follows the same
80
+ rule (project file cannot force durable on).
81
+
82
+ ## Phase 0 — Extract runtime boundary (no behavior change)
83
+
84
+ ### 0.1 Define `src/runtimes/runtime.ts`
85
+
86
+ Opaque types only: `AgentHandle / TaskId / MessageId / Completion /
87
+ RuntimeStatus`. No Herdr identifiers, no durable identifiers in the public
88
+ contract. Document mapping: Herdr `paneId` OR durable
89
+ `{conversationId, submissionId, taskId}` internally.
90
+
91
+ ### 0.2 Extract `src/core/ids.ts`
92
+
93
+ Move ID minting (`shepherd-<kind>-<session>-<uuid>`) and handle
94
+ parse/validation out of `LifecycleRegistry` into a pure codec both
95
+ runtimes call. `orchestration.ts` keeps handles and registries only.
96
+ Formats and `invalid_handle` messages stay identical.
97
+
98
+ ### 0.3 Move Herdr code
99
+
100
+ Move current `src/core/herdr.ts + orchestration.ts + lifecycle.ts +
101
+ messaging.ts` rename-only into `src/runtimes/herdr/` (messaging is support,
102
+ not an interface implementer), and the settings store
103
+ `src/extension/config.ts` → `src/core/config.ts`. `src/extension/shepherd.ts`
104
+ and `index.ts` route lifecycle calls through the interface; discovery stays
105
+ shared and herd/prune/widget keep runtime introspection accessors. No
106
+ schema change, no prompt change. Leave re-export shims at the old
107
+ `src/core/*` and `src/extension/config.ts` paths so existing `verify-*.mjs`
108
+ passes untouched — that is the proof the boundary is right.
109
+
110
+ Keep: `created-panes.json` ownership source, `--no-focus` backgrounds,
111
+ temp-file lifetime (delete only after pane gone), session-scoped registries
112
+ with clear invalid-handle errors, `shepherd_done` 1-nudge rule
113
+ (`PI_SHEPHERD_DONE_NUDGES`, `0` disables).
114
+
115
+ ### 0.4 Verify boundary
116
+
117
+ Existing `verify-*.mjs` suite passes untouched, live Herdr checks pass
118
+ (idle spawn, non-blocking prompt, single + multi-wait, status, close,
119
+ ownership refusal, project-agent gate), Herdr-only install evaluates no
120
+ new dependency.
121
+
122
+ ## Phase 1 — Durable spawn/delegate/watch/done (foreground)
123
+
124
+ ### 1.1 Scaffold `src/runtimes/durable/`
125
+
126
+ Add `@earendil-works/pi-durable` + `@earendil-works/chord` as optional
127
+ peerDeps (Herdr-only installs never import them). Create
128
+ `src/runtimes/durable/runtime.ts` (interface over `Harness`) and
129
+ `src/runtimes/durable/extension.ts` (`defineExtension` with the same tool
130
+ surface + docs).
131
+
132
+ ### 1.2 Sheep-as-conversation
133
+
134
+ `spawn`: `tx.createConversation({ownership:{kind:"task",taskId}})` +
135
+ `configure()` from agent def. `model:null/default` inherits Shepherd session
136
+ model (per-user default: free local vs. Copilot); pinned `model` overrides
137
+ (e.g. triage local, deep review Copilot). `thinking` clamped per model.
138
+
139
+ `delegate`: `conversation.submit({type:"input", content, requestId})`,
140
+ `requestId` defaults to shepherd task ID for overnight dedup.
141
+
142
+ Child `shepherd_done` completes owner task; parent `watch` =
143
+ `submission.wait() / harness.waitForTask()`. Steered delivery preserved:
144
+ mid-turn inject between tool rounds, idle triggers new turn.
145
+
146
+ Harness wiring: `Harness.open(storage)` with `MemoryStorage` (tests) or
147
+ `openNodeSqliteStorage(path)` / `openNodeJsonlStorage(dir)` (service).
148
+ `harness.resume()` on boot, `harness.closed.then(reopen if failed)`.
149
+ `settings.compaction/progress/retry/toolExecution` pass-through, no shepherd
150
+ opinions except defaults that keep 8h logs alive.
151
+
152
+ ### 1.3 Replay policy
153
+
154
+ App-driving tools stay non-`replay:"safe"`: crash-interrupted calls yield
155
+ `interrupted` + partial output, never silent rerun. Only idempotent
156
+ readers opt in with keyed `executeTool` reattach.
157
+
158
+ ### 1.4 Matrix fixture
159
+
160
+ Add `test/verify-durable-matrix.mjs`: same test-driver -> crash-reviewer
161
+ fixture on both backends (`MemoryStorage + fauxProvider` for durable).
162
+ Asserts identical orchestrator prompts produce identical handoffs, opaque-ID
163
+ contract holds (pane IDs and conversation IDs both rejected as tool IDs),
164
+ same `requestId` never double-runs.
165
+
166
+ ### 1.5 Update import/path references + drop shims
167
+
168
+ Rewrite imports in `test/verify-*.mjs` and root `index.ts` from
169
+ `src/core/{herdr,orchestration,lifecycle,messaging}.ts` and
170
+ `src/extension/config.ts` to `src/runtimes/herdr/*` and `src/core/config.ts`;
171
+ update non-import path strings (e.g. `test/verify-status-widget.mjs`
172
+ module-path assertions) and the `index.ts` header comment. Assertions stay
173
+ unchanged. Delete the Phase 0 shims and refresh layout references in
174
+ `AGENTS.md` and `docs/plans/architecture.md`. `npm test` green with no
175
+ old-path survivors.
176
+
177
+ ## Phase 2 — Message/status/read/close/timeouts
178
+
179
+ `message` without model impact -> `submit({type:"write"})` (immediate even
180
+ mid-run, except behind queued reset). With impact -> steer/follow-up queued
181
+ in `docs[pi.inbox]`. `expectsReply:true` -> durable `waiting` + existing
182
+ `staleWaitThreshold` single reminder + `timeout` abort -> `blocked`.
183
+
184
+ `status` from `viewState() (entries, pi.live, pi.inbox)` + `taskGraph()`:
185
+ `working` iff live generation/tools running, `waiting` iff parked on reply,
186
+ with `waitingOn + stale`. Preserve idle-process-but-waiting-task visibility
187
+ the widget already handles.
188
+
189
+ `read` = committed entries + `pi.live` partials/output tail (cap like
190
+ `watch()` 100-frame collapse). `close/abort` bottom-up, queued inputs
191
+ withdrawn, queued writes stay. `background:true` anchors = `stayOpen`.
192
+
193
+ `done-nudge` maps to durable `onYield` on `GenerationTask`, same 1-turn
194
+ budget, then stall escalates as ordinary message.
195
+
196
+ ## Phase 3 — Fieldnotes bridge + settings + auto viewer
197
+
198
+ `ShepherdNotes = defineDoc({kind:"shepherd.notes", scope:"conversation",
199
+ history:"rewindable", fork:"asOf"})`. Note + entry in one `commit()`.
200
+ File mirror via `env` for agent continuity/debugging. `fieldnotes`
201
+ session-scoped (new session after toggle). Keep gitignore:
202
+
203
+ ```gitignore
204
+ .shepherd/*
205
+ !.shepherd/config.json
206
+ ```
207
+
208
+ Settings: `backend` + `durableStorage` (`sqlite:path` / `jsonl:dir` /
209
+ `memory`) in user file + active project file
210
+ (`projectScope:true` pattern from spec 005). `/shepherd settings` gains two
211
+ rows. `auto` viewer: read-only Herdr panes fed by `watchEvents()`
212
+ snapshot + deltas, never accepting input (avoids dual-write).
213
+
214
+ Reporter contract (app-owned, documented not implemented): HTML renderer
215
+ reads job report doc (e.g. `app.test-report`) + logs, not shepherd internals.
216
+ Shepherd guarantees agent memory; app guarantees human template.
217
+
218
+ ## Phase 4 — Tests, docs, live verification
219
+
220
+ Pure tests where possible (ID codec, registries, `requestId` dedup,
221
+ waiting/stale/timeout matrix, notes-doc mirror + `fork:asOf`, nudge
222
+ budget). `npm test` + Herdr-only install pulls no durable dep.
223
+
224
+ Docs: README gains one backend section (Herdr-first, durable experimental),
225
+ agent-definition guide notes dual mapping, diagnostics guide covers
226
+ `extract:system-prompt` for both adapters.
227
+
228
+ Live: `/shepherd herd/spawn/read/settings` on herdr; overnight dry-run on
229
+ durable (submit evening, kill service mid-bash, `reopen+resume` morning,
230
+ unsafe -> `interrupted`, safe-keyed -> reattached, report renders).
231
+
232
+ ## Completion checklist
233
+
234
+ - [ ] `runtime.ts` + `ids.ts` extracted, Herdr moved with zero behavior
235
+ change; Phase 0 shims dropped and import paths updated in Phase 1.
236
+ - [ ] Durable `spawn/delegate/watch/done` foreground works on memory+sqlite.
237
+ - [ ] `message/expectsReply/status/read/close/timeout/staleWait` parity.
238
+ - [ ] Notes doc <-> file mirror + `fork:asOf` handoff.
239
+ - [ ] `backend/durableStorage` settings + `auto` read-only viewer.
240
+ - [ ] Reporter contract documented; shepherd never emits human HTML.
241
+ - [ ] Matrix tests + crash-resume + `npm test` green, Herdr-only clean.
242
+ - [ ] README/docs updated, Herdr still hero and default.