@jr2/orchestrator 0.5.1 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (350) hide show
  1. package/console/components/app.ts +55 -15
  2. package/console/components/bottom-drawer.ts +259 -0
  3. package/console/components/drawer.ts +14 -6
  4. package/console/components/fleet.ts +36 -13
  5. package/console/components/machine-pane.ts +61 -5
  6. package/console/components/nav.ts +54 -8
  7. package/console/conversation.ts +161 -0
  8. package/console/login.ts +123 -0
  9. package/console/main.ts +600 -60
  10. package/console/store.ts +611 -64
  11. package/console/style.css +231 -0
  12. package/docs/agents-block.md +2 -2
  13. package/docs/concepts/agents-and-turns.md +142 -29
  14. package/docs/concepts/architecture.md +60 -34
  15. package/docs/concepts/callers-and-hooks.md +205 -0
  16. package/docs/concepts/charts.md +13 -8
  17. package/docs/concepts/composition.md +10 -1
  18. package/docs/concepts/converge.md +7 -2
  19. package/docs/concepts/durable-runs.md +43 -16
  20. package/docs/concepts/effects.md +133 -50
  21. package/docs/concepts/gates.md +95 -25
  22. package/docs/concepts/machines-and-workflows.md +14 -4
  23. package/docs/concepts/pools-and-sources.md +193 -17
  24. package/docs/concepts/secrets-and-trust.md +186 -62
  25. package/docs/concepts/the-menu.md +14 -0
  26. package/docs/concepts/workspaces.md +135 -56
  27. package/docs/get-started/a-model-decides.md +14 -13
  28. package/docs/get-started/an-agent-in-a-sandbox.md +46 -35
  29. package/docs/get-started/connect-a-model.md +1 -1
  30. package/docs/get-started/create-an-instance.md +17 -14
  31. package/docs/get-started/have-your-assistant-write-a-machine.md +111 -40
  32. package/docs/get-started/install.md +4 -0
  33. package/docs/guides/add-tools-to-the-sandbox.md +72 -15
  34. package/docs/guides/answer-a-gate-from-a-forge.md +304 -0
  35. package/docs/guides/answer-a-gate.md +74 -37
  36. package/docs/guides/bind-open-parts.md +29 -12
  37. package/docs/guides/call-an-mcp-tool-from-a-machine.md +572 -0
  38. package/docs/guides/cancel-a-run.md +26 -16
  39. package/docs/guides/cut-a-worktree.md +560 -0
  40. package/docs/guides/fix-a-failed-converge.md +19 -7
  41. package/docs/guides/give-a-caller-a-scope.md +203 -0
  42. package/docs/guides/give-an-agent-mcp-tools.md +422 -0
  43. package/docs/guides/iterate-on-a-machine-on-kind.md +10 -9
  44. package/docs/guides/let-an-agent-write-a-chart.md +14 -7
  45. package/docs/guides/look-inside-a-sandbox.md +115 -42
  46. package/docs/guides/move-from-kind-to-a-shared-cluster.md +16 -0
  47. package/docs/guides/open-the-console.md +63 -14
  48. package/docs/guides/push-and-sign-a-branch.md +67 -21
  49. package/docs/guides/read-a-conversation.md +225 -0
  50. package/docs/guides/run-a-chart-by-name.md +17 -16
  51. package/docs/guides/run-a-command-or-call-a-url.md +84 -41
  52. package/docs/guides/run-one-worker-per-item.md +34 -27
  53. package/docs/guides/set-an-agents-access-and-dials.md +69 -22
  54. package/docs/guides/size-a-workspace.md +19 -6
  55. package/docs/guides/start-a-run-for-each-alert.md +290 -0
  56. package/docs/guides/start-a-run-from-a-run.md +18 -16
  57. package/docs/guides/start-a-run.md +24 -18
  58. package/docs/guides/steer-an-agent.md +332 -0
  59. package/docs/guides/use-a-hosted-provider.md +7 -1
  60. package/docs/guides/watch-a-run.md +146 -117
  61. package/docs/guides/write-a-hook-kind.md +238 -0
  62. package/docs/guides/write-a-tool-as-an-mcp-server.md +351 -0
  63. package/docs/guides/write-a-workflow.md +18 -8
  64. package/docs/llms.txt +110 -23
  65. package/docs/reference/authoring-surface/agent.md +35 -12
  66. package/docs/reference/authoring-surface/answer.md +20 -4
  67. package/docs/reference/authoring-surface/arg.md +28 -6
  68. package/docs/reference/authoring-surface/basic.md +1 -1
  69. package/docs/reference/authoring-surface/bearer.md +1 -1
  70. package/docs/reference/authoring-surface/branchname.md +15 -0
  71. package/docs/reference/authoring-surface/carry.md +23 -0
  72. package/docs/reference/authoring-surface/chartschema.md +1 -1
  73. package/docs/reference/authoring-surface/chartschemafor.md +1 -1
  74. package/docs/reference/authoring-surface/command.md +37 -22
  75. package/docs/reference/authoring-surface/customize.md +4 -1
  76. package/docs/reference/authoring-surface/defineconfig.md +1 -1
  77. package/docs/reference/authoring-surface/defineevent.md +1 -1
  78. package/docs/reference/authoring-surface/dispatch.md +5 -4
  79. package/docs/reference/authoring-surface/fail.md +18 -5
  80. package/docs/reference/authoring-surface/file.md +37 -17
  81. package/docs/reference/authoring-surface/frompick.md +1 -1
  82. package/docs/reference/authoring-surface/gate.md +118 -0
  83. package/docs/reference/authoring-surface/group-agents.md +15 -0
  84. package/docs/reference/authoring-surface/group-charts.md +16 -0
  85. package/docs/reference/authoring-surface/group-composition.md +25 -0
  86. package/docs/reference/authoring-surface/group-config.md +17 -0
  87. package/docs/reference/authoring-surface/group-effects.md +48 -0
  88. package/docs/reference/authoring-surface/group-failure.md +13 -0
  89. package/docs/reference/authoring-surface/group-gates.md +16 -0
  90. package/docs/reference/authoring-surface/group-machine-and-events.md +15 -0
  91. package/docs/reference/authoring-surface/group-steering.md +12 -0
  92. package/docs/reference/authoring-surface/group-types-these-signatures-name.md +137 -0
  93. package/docs/reference/authoring-surface/group-workspaces-and-repos.md +15 -0
  94. package/docs/reference/authoring-surface/http.md +109 -9
  95. package/docs/reference/authoring-surface/isfailure.md +1 -1
  96. package/docs/reference/authoring-surface/isready.md +14 -0
  97. package/docs/reference/authoring-surface/jr2setup.md +14 -4
  98. package/docs/reference/authoring-surface/mcp.md +107 -0
  99. package/docs/reference/authoring-surface/open.md +41 -8
  100. package/docs/reference/authoring-surface/openbody.md +1 -1
  101. package/docs/reference/authoring-surface/paramsof.md +23 -5
  102. package/docs/reference/authoring-surface/pool.md +79 -11
  103. package/docs/reference/authoring-surface/push.md +31 -13
  104. package/docs/reference/authoring-surface/readyof.md +21 -0
  105. package/docs/reference/authoring-surface/reread.md +19 -0
  106. package/docs/reference/authoring-surface/secret.md +26 -5
  107. package/docs/reference/authoring-surface/sentby.md +1 -1
  108. package/docs/reference/authoring-surface/skip.md +22 -0
  109. package/docs/reference/authoring-surface/source.md +59 -4
  110. package/docs/reference/authoring-surface/steer.md +93 -0
  111. package/docs/reference/authoring-surface/text.md +1 -1
  112. package/docs/reference/authoring-surface/type-agentdefinition.md +16 -3
  113. package/docs/reference/authoring-surface/type-agentmcpserver.md +54 -0
  114. package/docs/reference/authoring-surface/type-agentslots.md +4 -1
  115. package/docs/reference/authoring-surface/type-anyframepath.md +14 -0
  116. package/docs/reference/authoring-surface/type-anyworktreepath.md +15 -0
  117. package/docs/reference/authoring-surface/type-bindactors.md +14 -0
  118. package/docs/reference/authoring-surface/type-binding.md +6 -7
  119. package/docs/reference/authoring-surface/type-caller.md +40 -0
  120. package/docs/reference/authoring-surface/type-callerrecord.md +32 -0
  121. package/docs/reference/authoring-surface/type-carry.md +19 -0
  122. package/docs/reference/authoring-surface/type-chainlink.md +3 -3
  123. package/docs/reference/authoring-surface/type-commandinput.md +4 -3
  124. package/docs/reference/authoring-surface/type-commandoptions.md +0 -22
  125. package/docs/reference/authoring-surface/type-commandoutput.md +23 -5
  126. package/docs/reference/authoring-surface/type-customize.md +20 -3
  127. package/docs/reference/authoring-surface/type-delivered.md +17 -0
  128. package/docs/reference/authoring-surface/type-deliveredevent.md +18 -0
  129. package/docs/reference/authoring-surface/type-dispatchdefinition.md +4 -3
  130. package/docs/reference/authoring-surface/type-dispatchoutput.md +9 -2
  131. package/docs/reference/authoring-surface/type-doorlessinput.md +19 -0
  132. package/docs/reference/authoring-surface/type-effectbinding.md +12 -2
  133. package/docs/reference/authoring-surface/type-filebody.md +4 -8
  134. package/docs/reference/authoring-surface/type-framepath.md +14 -0
  135. package/docs/reference/authoring-surface/type-gatebinding.md +15 -0
  136. package/docs/reference/authoring-surface/type-gatehooks.md +12 -0
  137. package/docs/reference/authoring-surface/type-gatelook.md +37 -0
  138. package/docs/reference/authoring-surface/type-gateoutcome.md +32 -0
  139. package/docs/reference/authoring-surface/type-gatereply.md +19 -0
  140. package/docs/reference/authoring-surface/type-gateslot.md +21 -0
  141. package/docs/reference/authoring-surface/type-gateslotinput.md +13 -0
  142. package/docs/reference/authoring-surface/type-gateslotparts.md +18 -0
  143. package/docs/reference/authoring-surface/type-gateslots.md +12 -0
  144. package/docs/reference/authoring-surface/type-gatetypesat.md +13 -0
  145. package/docs/reference/authoring-surface/type-hostinjectedinput.md +3 -5
  146. package/docs/reference/authoring-surface/type-httpdefinition.md +6 -3
  147. package/docs/reference/authoring-surface/type-httpfileinput.md +20 -0
  148. package/docs/reference/authoring-surface/type-httpoutput.md +8 -2
  149. package/docs/reference/authoring-surface/type-jr2config.md +37 -2
  150. package/docs/reference/authoring-surface/type-lateroutcome.md +21 -0
  151. package/docs/reference/authoring-surface/type-lends.md +5 -1
  152. package/docs/reference/authoring-surface/type-lookoutput.md +14 -0
  153. package/docs/reference/authoring-surface/type-mcpcalloutput.md +78 -0
  154. package/docs/reference/authoring-surface/type-mcpnotreturned.md +42 -0
  155. package/docs/reference/authoring-surface/type-mcpoptions.md +47 -0
  156. package/docs/reference/authoring-surface/type-mcpoutput.md +50 -0
  157. package/docs/reference/authoring-surface/type-mcpreadoutput.md +53 -0
  158. package/docs/reference/authoring-surface/type-mcpremoteserver.md +50 -0
  159. package/docs/reference/authoring-surface/type-mcpserver.md +41 -0
  160. package/docs/reference/authoring-surface/type-mcpstdioserver.md +42 -0
  161. package/docs/reference/authoring-surface/type-namedsecret.md +4 -4
  162. package/docs/reference/authoring-surface/type-namestart.md +77 -0
  163. package/docs/reference/authoring-surface/type-noslotworkspaceoptions.md +1 -6
  164. package/docs/reference/authoring-surface/type-notworktreekeys.md +12 -0
  165. package/docs/reference/authoring-surface/type-openbodydeclaration.md +1 -1
  166. package/docs/reference/authoring-surface/type-openbodyoutput.md +1 -1
  167. package/docs/reference/authoring-surface/type-opencommandinput.md +7 -2
  168. package/docs/reference/authoring-surface/type-permissivenoslotworkspaceoptions.md +1 -6
  169. package/docs/reference/authoring-surface/type-permissiveworkspaceoptions.md +5 -10
  170. package/docs/reference/authoring-surface/type-poolnextinput.md +22 -0
  171. package/docs/reference/authoring-surface/type-poolspec.md +11 -6
  172. package/docs/reference/authoring-surface/type-pushinput.md +22 -3
  173. package/docs/reference/authoring-surface/type-pushoutput.md +33 -8
  174. package/docs/reference/authoring-surface/type-pushsourcespec.md +38 -0
  175. package/docs/reference/authoring-surface/type-readsourcespec.md +47 -0
  176. package/docs/reference/authoring-surface/type-reposlot.md +6 -6
  177. package/docs/reference/authoring-surface/type-reread.md +18 -0
  178. package/docs/reference/authoring-surface/type-runsconfig.md +41 -0
  179. package/docs/reference/authoring-surface/type-sandboxoptions.md +11 -4
  180. package/docs/reference/authoring-surface/type-sender.md +26 -1
  181. package/docs/reference/authoring-surface/type-sendercallerfield.md +14 -0
  182. package/docs/reference/authoring-surface/type-size.md +1 -1
  183. package/docs/reference/authoring-surface/type-skip.md +19 -0
  184. package/docs/reference/authoring-surface/type-slothandles.md +5 -13
  185. package/docs/reference/authoring-surface/type-sourcehooks.md +23 -0
  186. package/docs/reference/authoring-surface/type-sourcein.md +25 -0
  187. package/docs/reference/authoring-surface/type-sourcereply.md +18 -0
  188. package/docs/reference/authoring-surface/type-sourcespec.md +26 -17
  189. package/docs/reference/authoring-surface/type-sourceverdict.md +49 -0
  190. package/docs/reference/authoring-surface/type-standingpoolmachine.md +12 -0
  191. package/docs/reference/authoring-surface/type-standingpoolspec.md +51 -0
  192. package/docs/reference/authoring-surface/type-steerargs.md +20 -0
  193. package/docs/reference/authoring-surface/type-thinkinglevel.md +1 -1
  194. package/docs/reference/authoring-surface/type-usercontainer.md +1 -1
  195. package/docs/reference/authoring-surface/type-workspaced.md +30 -9
  196. package/docs/reference/authoring-surface/type-workspacehandles.md +5 -6
  197. package/docs/reference/authoring-surface/type-workspaceoptions.md +1 -13
  198. package/docs/reference/authoring-surface/type-worktreefound.md +16 -0
  199. package/docs/reference/authoring-surface/type-worktreeinput.md +56 -0
  200. package/docs/reference/authoring-surface/type-worktreekey.md +14 -0
  201. package/docs/reference/authoring-surface/type-worktreeoptions.md +29 -0
  202. package/docs/reference/authoring-surface/type-worktreeoutput.md +45 -0
  203. package/docs/reference/authoring-surface/type-worktreepath.md +16 -0
  204. package/docs/reference/authoring-surface/type-worktreeready.md +37 -0
  205. package/docs/reference/authoring-surface/type-worktreerefused.md +21 -0
  206. package/docs/reference/authoring-surface/workspace.md +66 -38
  207. package/docs/reference/authoring-surface/worktree.md +71 -0
  208. package/docs/reference/authoring-surface.md +88 -147
  209. package/docs/reference/cli.md +413 -74
  210. package/docs/reference/config.md +248 -6
  211. package/docs/reference/decisions/0001-operator-owned-generic-sandbox.md +10 -9
  212. package/docs/reference/decisions/0002-duplex-streaming-actor.md +81 -14
  213. package/docs/reference/decisions/0004-worktree-provisioning-model.md +27 -21
  214. package/docs/reference/decisions/0005-sandbox-pod-composition.md +39 -25
  215. package/docs/reference/decisions/0006-agent-control-surface.md +6 -3
  216. package/docs/reference/decisions/0007-durable-machine-state.md +1 -1
  217. package/docs/reference/decisions/0009-cli-and-instance-interface.md +150 -49
  218. package/docs/reference/decisions/0010-bdd-acceptance-tests.md +5 -3
  219. package/docs/reference/decisions/0011-workflow-defined-events.md +256 -12
  220. package/docs/reference/decisions/0012-workspace-wrapper-machine.md +39 -32
  221. package/docs/reference/decisions/0013-the-agent-reaches-its-machine-through-a-container-it-cannot-read.md +54 -30
  222. package/docs/reference/decisions/0014-observation-is-open-run-state-is-not.md +53 -27
  223. package/docs/reference/decisions/0015-authoring-surface-absorbs-the-mechanism.md +26 -16
  224. package/docs/reference/decisions/0016-agent-turn-mechanics-are-internal.md +9 -7
  225. package/docs/reference/decisions/0017-pool-over-a-source-port.md +226 -10
  226. package/docs/reference/decisions/0018-instance-agents-are-definitions-jr2-assembles-the-harness.md +15 -7
  227. package/docs/reference/decisions/0019-one-converging-command-against-the-current-context.md +53 -27
  228. package/docs/reference/decisions/0021-workspace-continuity-is-a-lease-that-answers-back.md +16 -2
  229. package/docs/reference/decisions/0022-observation-is-a-level-triggered-feed.md +4 -4
  230. package/docs/reference/decisions/0023-the-harness-prints-the-conversation.md +124 -43
  231. package/docs/reference/decisions/0024-an-agents-turn-ends-with-the-state-that-asked-for-it.md +36 -38
  232. package/docs/reference/decisions/0025-cancel-ends-a-run-stop-parks-it.md +10 -5
  233. package/docs/reference/decisions/0027-the-harness-is-jr2s-own-server-flue-retires-the-wire-stays.md +16 -7
  234. package/docs/reference/decisions/0028-what-an-agent-may-do-to-the-workspace-is-part-of-its-definition.md +36 -19
  235. package/docs/reference/decisions/0029-a-menu-is-fixed-for-a-conversation-and-a-turn-is-told-its-allowed-picks.md +18 -10
  236. package/docs/reference/decisions/0030-a-snapshot-names-the-machine-it-was-written-under.md +3 -2
  237. package/docs/reference/decisions/0031-menu-only-agents-run-on-the-instance-harness.md +23 -15
  238. package/docs/reference/decisions/0032-the-console-unlocks-with-a-callers-credential.md +101 -0
  239. package/docs/reference/decisions/0033-a-machine-declares-the-input-that-starts-it.md +22 -22
  240. package/docs/reference/decisions/0034-the-console-renders-with-preact-and-the-server-erases-the-types.md +6 -6
  241. package/docs/reference/decisions/0037-an-instance-builds-its-sandbox-images-jr2-injects-the-harness.md +13 -12
  242. package/docs/reference/decisions/0049-a-machine-carries-its-parts-and-composes-by-invoke.md +23 -23
  243. package/docs/reference/decisions/0050-what-code-names-config-declares-what-the-cli-names-discovery-finds.md +6 -1
  244. package/docs/reference/decisions/0051-a-repo-is-a-slot-on-the-workspace-and-a-cache-on-the-node.md +57 -46
  245. package/docs/reference/decisions/0052-a-sandbox-lands-where-an-ordinary-pod-lands-and-the-cache-follows-it.md +5 -0
  246. package/docs/reference/decisions/0053-a-fetch-inside-the-pod-asks-the-node-cache-and-the-cache-asks-the-remote.md +20 -8
  247. package/docs/reference/decisions/0054-the-kit-ships-machines-and-a-shipped-machine-leaves-its-parts-open.md +55 -40
  248. package/docs/reference/decisions/0055-a-release-is-a-pushed-tag-and-the-publish-guard-is-the-credential.md +14 -13
  249. package/docs/reference/decisions/0057-a-turn-is-its-frame-its-dials-and-whether-it-continues.md +31 -24
  250. package/docs/reference/decisions/0058-a-harness-answers-the-orchestrator-alone.md +18 -10
  251. package/docs/reference/decisions/0059-a-harness-holds-stand-ins-and-the-custodian-holds-the-keys.md +137 -86
  252. package/docs/reference/decisions/0062-jr2-briefs-the-agent-on-its-seat-never-on-its-task.md +165 -91
  253. package/docs/reference/decisions/0063-the-orchestrator-watches-the-cluster.md +7 -7
  254. package/docs/reference/decisions/0064-a-workspace-waits-for-capacity.md +2 -2
  255. package/docs/reference/decisions/0065-a-persist-survives-a-crash-and-history-is-kept-for-a-set-time.md +109 -0
  256. package/docs/reference/decisions/0066-a-model-tier-runs-a-real-model-and-a-change-to-what-it-reads-runs-the-tier.md +53 -23
  257. package/docs/reference/decisions/0067-an-effect-is-an-actor-slot-and-runs-at-most-once-unless-marked-idempotent.md +221 -153
  258. package/docs/reference/decisions/0068-an-effector-holds-an-effects-secrets-and-signs-its-wire.md +161 -96
  259. package/docs/reference/decisions/0069-the-orchestrator-is-a-kit-image-and-an-instance-publishes-its-assembly.md +32 -9
  260. package/docs/reference/decisions/0070-a-machine-lends-its-parts-to-a-chart.md +48 -33
  261. package/docs/reference/decisions/0071-a-run-starts-a-run-through-a-dispatch-effect.md +56 -30
  262. package/docs/reference/decisions/0072-the-docs-ship-inside-the-kit-and-an-instances-agents-md-indexes-them.md +35 -25
  263. package/docs/reference/decisions/0073-a-machine-fails-a-run-with-fail-and-an-uncaught-failure-ends-it-failed.md +1 -1
  264. package/docs/reference/decisions/0075-a-machine-cuts-a-worktree-and-the-workspace-only-clones.md +296 -0
  265. package/docs/reference/decisions/0076-an-agents-control-credential-lives-only-where-no-workspace-code-runs.md +347 -0
  266. package/docs/reference/decisions/0077-jr2-requires-landlock-on-every-sandbox-node.md +79 -0
  267. package/docs/reference/decisions/0078-a-caller-comes-from-one-authenticator-seam-and-acts-within-its-scopes.md +285 -0
  268. package/docs/reference/decisions/0079-an-agent-holds-mcp-tools-a-machine-calls-them-as-an-effect-and-the-machine-is-the-approval.md +198 -0
  269. package/docs/reference/decisions.md +9 -4
  270. package/docs/reference/glossary.md +138 -45
  271. package/docs/reference/hooks.md +1077 -0
  272. package/docs/reference/http-api.md +988 -123
  273. package/docs/reference/machines.md +0 -1
  274. package/docs/why.md +7 -5
  275. package/package.json +10 -3
  276. package/src/actor.ts +194 -42
  277. package/src/agent.ts +56 -6
  278. package/src/ambient.ts +3 -5
  279. package/src/assembly-checks.ts +530 -4
  280. package/src/callers.ts +245 -0
  281. package/src/chart.ts +238 -26
  282. package/src/charts.ts +9 -2
  283. package/src/config.ts +566 -7
  284. package/src/conversations.ts +453 -0
  285. package/src/custodian.lua +17 -5
  286. package/src/custodian.ts +257 -77
  287. package/src/customize.ts +154 -7
  288. package/src/dispatch-effect.ts +142 -26
  289. package/src/durability.ts +2 -0
  290. package/src/duration.ts +20 -0
  291. package/src/effect-checks.ts +52 -4
  292. package/src/effect.ts +401 -132
  293. package/src/effector-wire.ts +278 -30
  294. package/src/fail.ts +17 -4
  295. package/src/gate-brand.ts +16 -0
  296. package/src/gate.ts +634 -48
  297. package/src/harness-client.ts +84 -35
  298. package/src/harness-env.ts +33 -0
  299. package/src/held-secrets.ts +197 -39
  300. package/src/holds.ts +5 -2
  301. package/src/hook-kinds/discord.ts +92 -0
  302. package/src/hook-kinds/door.ts +107 -0
  303. package/src/hook-kinds/github.ts +89 -0
  304. package/src/hook-kinds/gitlab.ts +72 -0
  305. package/src/hook-kinds/hook.ts +7 -0
  306. package/src/hook-kinds/index.ts +465 -0
  307. package/src/hook-kinds/slack.ts +151 -0
  308. package/src/hook-kinds/webhook.ts +113 -0
  309. package/src/hooks.ts +506 -0
  310. package/src/http-effect.ts +164 -70
  311. package/src/http.ts +933 -99
  312. package/src/index.ts +30 -5
  313. package/src/instance.ts +196 -26
  314. package/src/internal.ts +23 -3
  315. package/src/kube-client.ts +6 -3
  316. package/src/machine-doc.ts +29 -5
  317. package/src/mcp-effect.ts +424 -0
  318. package/src/model-keys.ts +24 -0
  319. package/src/mount-digest.ts +41 -7
  320. package/src/names.ts +82 -7
  321. package/src/notices.ts +576 -59
  322. package/src/open-body.ts +110 -30
  323. package/src/open-effect.ts +17 -4
  324. package/src/open.ts +40 -7
  325. package/src/parts.ts +125 -35
  326. package/src/party.ts +35 -1
  327. package/src/pick.ts +43 -9
  328. package/src/pool.ts +1172 -172
  329. package/src/registration.ts +249 -17
  330. package/src/repo-identity.ts +7 -3
  331. package/src/repos.ts +20 -2
  332. package/src/run-host.ts +1614 -190
  333. package/src/run-list.ts +191 -0
  334. package/src/sandbox-effect.ts +217 -66
  335. package/src/sandbox-kube.ts +116 -21
  336. package/src/secret-placement.ts +248 -35
  337. package/src/server.ts +53 -4
  338. package/src/setup.ts +158 -38
  339. package/src/snapshot-store.ts +565 -48
  340. package/src/steer.ts +111 -0
  341. package/src/stub-harness.ts +6 -2
  342. package/src/tokens.ts +362 -46
  343. package/src/vocabulary.ts +23 -4
  344. package/src/wire.ts +27 -10
  345. package/src/workspace.ts +314 -259
  346. package/src/worktree.ts +249 -0
  347. package/docs/reference/authoring-surface/type-noslotspec.md +0 -26
  348. package/docs/reference/authoring-surface/type-workspacespec.md +0 -37
  349. package/docs/reference/decisions/0032-the-console-unlocks-with-the-instance-token.md +0 -61
  350. package/docs/reference/decisions/0065-a-persist-survives-a-crash-and-history-is-a-week.md +0 -65
@@ -1,13 +1,14 @@
1
1
  ---
2
2
  title: Gates
3
- description: How a run waits for an event from a person, the CLI, the Console, a webhook or a CI job.
3
+ description: How a run waits for an event from a person, the CLI, the Console, a Hook or a CI job.
4
4
  ---
5
5
 
6
6
  # Gates
7
7
 
8
8
  A [Gate](../reference/glossary.md#gate) is an input that a run waits for from outside the run. A person, a CI job or a
9
- webhook can send it. A state of a [Machine](../reference/glossary.md#machine) opens a Gate when it invokes the `gate`
10
- actor, and the Gate closes when the state exits. While the Gate is open, a caller can send one of its events.
9
+ [Hook](../reference/glossary.md#hook) can send it. A state of a [Machine](../reference/glossary.md#machine) opens a Gate
10
+ when it invokes the `gate` actor, and the Gate closes when the state exits. While the Gate is open, a caller can send
11
+ one of its events.
11
12
 
12
13
  ## Why Gates exist
13
14
 
@@ -105,7 +106,8 @@ export const machine = jr2Setup({
105
106
  ```
106
107
 
107
108
  You do not import the `gate` actor. `jr2Setup` adds it, and a state invokes it as `invoke: { src: "gate" }`. Its input
108
- is optional. It can hold `meta`, and an authored `gate` id.
109
+ is optional. It can hold `meta`, and an authored `gate` id. A Gate that hears a Hook is a Gate slot, which you declare
110
+ with `gate({ … })` (see [A Gate that hears a Hook](#a-gate-that-hears-a-hook)).
109
111
 
110
112
  ### The id
111
113
 
@@ -133,29 +135,36 @@ that the run takes from the Gate now. jr2 refuses an event that is not in `accep
133
135
 
134
136
  ### The meta
135
137
 
136
- `meta` is data for the caller. It tells a person what to decide, and it lets a webhook find the correct Gate. In the
137
- example, the `review` Gate shows the version and the URL of the CI job. jr2 does not read `meta`.
138
+ `meta` is data for the caller. It tells a person what to decide, and it lets the handler of a Gate slot find the correct
139
+ Gate. In the example, the `review` Gate shows the version and the URL of the CI job. jr2 does not read the `meta` of a
140
+ plain Gate. A Gate slot checks its `meta` against its schema when it opens.
138
141
 
139
142
  ### The callers
140
143
 
141
- A caller finds the open Gates of a run with [`GET /runs/:runId`](../reference/http-api.md#get-runs-runid). Each Gate
142
- gives its id, its events with their input schemas, `acceptsNow` and `meta` (see
143
- [`GateView`](../reference/http-api.md#type-gateview)). Then the caller sends one event, with its input. These callers
144
- use the same route,
144
+ A caller finds the open Gates of all the live runs that it can answer with
145
+ [`GET /gates`](../reference/http-api.md#get-gates), the oldest first, and the open Gates of one run with
146
+ [`GET /runs/:runId`](../reference/http-api.md#get-runs-runid). Each Gate gives its id, its events with their input
147
+ schemas, `acceptsNow` and `meta` (see [`GateView`](../reference/http-api.md#type-gateview)). Then the caller sends one
148
+ event, with its input. These callers use the same route,
145
149
  [`POST /runs/:runId/gates/:gate/events`](../reference/http-api.md#post-runs-runid-gates-gate-events), with the
146
150
  [Instance](../reference/glossary.md#instance) token:
147
151
 
148
- - **The [jr2 CLI](../reference/glossary.md#jr2-cli).** [`jr2 status`](../reference/cli.md#status) prints the open Gates
149
- of a run under `gates`. [`jr2 send`](../reference/cli.md#send) with `--gate` sends an event to one of them.
150
- - **The [Console](../reference/glossary.md#console).** When you type the Instance token, the Attention tab shows a card
151
- for each open Gate. The card has a form for each event.
152
- - **A webhook or a CI job.** Any client that can reach the [Orchestrator](../reference/glossary.md#orchestrator), and
153
- that holds the Instance token, can send to a Gate. It can call the route, or run `jr2 send` with `--url` and
152
+ - **The [jr2 CLI](../reference/glossary.md#jr2-cli).** [`jr2 gates`](../reference/cli.md#gates) prints the open Gates of
153
+ all the live runs that you can answer, the oldest first, each with the time when it opened.
154
+ [`jr2 status`](../reference/cli.md#status) prints the open Gates of one run under `gates`.
155
+ [`jr2 send`](../reference/cli.md#send) with `--gate` sends an event to one of them.
156
+ - **The [Console](../reference/glossary.md#console).** When you type a token with the `answer` scope, or log in, the
157
+ Attention tab shows a card for each open Gate of the Workflows of that scope. The card has a form for each event.
158
+ - **A CI job.** Any client that can reach the [Orchestrator](../reference/glossary.md#orchestrator), and that holds a
159
+ credential with the `answer` scope, can send to a Gate. It can call the route, or run `jr2 send` with `--url` and
154
160
  `JR2_TOKEN`.
161
+ - **A Hook.** A Gate slot's handlers turn a delivery of an outside service into one of the Gate's events (see below).
155
162
 
156
- The [Sender](../reference/glossary.md#sender) of the event is the Gate, by its invoke id, and not the person or the
157
- system behind it. The invoke id is not always the id that a caller uses. [The Menu](the-menu.md) tells how to use the
158
- Sender.
163
+ The [Sender](../reference/glossary.md#sender) of the event is the Gate, by its invoke id. The invoke id is not always
164
+ the id that a caller uses. The Sender also carries the [Caller](../reference/glossary.md#caller) that answered: `via`,
165
+ `name`, `onBehalfOf` when the Caller gives it, and `checked`, the fields that jr2 verified.
166
+ [The Sender](the-menu.md#the-sender) tells how to use the Sender, and why you must not trust an `onBehalfOf` that
167
+ `checked` does not name.
159
168
 
160
169
  ### A run that waits
161
170
 
@@ -173,7 +182,62 @@ After each answer, the Agent can pick again, so it can ask and get answers with
173
182
  on the pick. For example, `guard: ({ context }) => context.replies < 1` allows one question only. A guard that reads
174
183
  only the context removes the pick from the [Allowed picks](the-menu.md#allowed-picks).
175
184
 
176
- A Pool that has a `wake` event serves it as a Gate with the id `source` (see [Pools and Sources](pools-and-sources.md)).
185
+ A Pool that has a `wake` event, or a Source that hears a Hook, serves a Gate with the id `source`. A Hook's delivery
186
+ reaches the Source's handlers through it (see [Pools and Sources](pools-and-sources.md#a-hook-reaches-a-source)).
187
+
188
+ ## A Gate that hears a Hook
189
+
190
+ A [Hook](../reference/glossary.md#hook) checks that a delivery comes from its service, and decides nothing
191
+ ([Callers and Hooks](callers-and-hooks.md) tells what the door of a Hook does). A Gate decides what the delivery means
192
+ for its run. Thus a Gate that hears a Hook is an extension of the Gate, and not a second concept. You declare it as a
193
+ Gate slot in `actors`, as `actors: { review: gate({ meta, delivers, hooks, look, reply }) }`, and a state invokes it by
194
+ its key, as `invoke: { src: "review" }`.
195
+
196
+ - **`meta`** is a zod object. The Gate checks the state's `meta` against it when it opens, and its handlers and its look
197
+ read the `meta` typed.
198
+ - **`delivers`** lists the events that the handlers and the look can give. An event whose def does not set `audience` is
199
+ `"external"` here, so an Agent below the Gate does not have it on its Menu. `createMachine` refuses an event of
200
+ `delivers` that the Gate does not take at the state that invokes it.
201
+ - **`hooks`** has a handler for each [Hook kind](../reference/glossary.md#hook-kind) and event key, such as
202
+ `github: { "pull_request_review.submitted": handler }`. A delivery goes to each open Gate whose handlers hear its kind
203
+ and key. The handler compares the payload with the `meta`, and gives one event of `delivers`, or `undefined`. It is
204
+ pure and synchronous.
205
+ - **`look`** reads the fact that the Gate waits for. It reads once after the Gate opens, again after each restart of the
206
+ Orchestrator, and every `every` (default `"10m"`) while the Gate is open, until the run takes the event that `until`
207
+ gives. So a delivery that came before the Gate opened, or that was lost in an outage, does not stop the run. The read
208
+ is an `http` Effect marked `idempotent: true`, or a promise. A looked event has the Gate as its Sender and no Caller,
209
+ and the run's feed shows `looked`. `look: "none"` is for a Gate that hears a command, such as `/retry`, which no read
210
+ can find. A packaged Machine leaves its look `open`, and the composer binds it with
211
+ `customize(machine, { gates: { <slot>: { look } } })`.
212
+ - **`reply`** gives the text that the person who acted reads, for a kind whose answer a person reads: a Slack command or
213
+ click, or a Discord interaction. `reply(outcome, { meta, caller })` is pure and synchronous. `outcome.state` is
214
+ `delivered`, with the event, or `refused`, with the reason. It runs after the run answers. It does not run for a Gate
215
+ whose handler gave `undefined`, because that Gate was not a target. It runs for every other target, and a handler that
216
+ threw is one of them: its `outcome` is `refused`, with the reason. It gives plain text or `undefined`. The Hook kind
217
+ escapes the text and shows it only to the person who acted.
218
+
219
+ ### The answer that a person reads
220
+
221
+ For a chat command or click, the Hook's door answers 200 with one text in the kind's envelope, not the outcome JSON:
222
+
223
+ - The door joins the `reply` text of each target. A Source's text comes first, then a Gate's. In each, the order is by
224
+ Workflow name and then by run id. The door drops an empty text and a text that is the same as one before it. The text
225
+ of a Source's `skipped` verdict shows only when no other target gave a text.
226
+ - When no target gives a text, the door writes one: for example `Nothing in jr2 waits for this now.`, or that jr2 got
227
+ the delivery before.
228
+ - Where the door's rules give 429 or 503, the person gets 200, and a line of the text says why. The hook log keeps the
229
+ status of the rules.
230
+ - The door cuts the text to the limit of the kind (Discord: 2000 characters; Slack: 4000).
231
+ - Slack shows no text in the answer to a click. Thus the door answers the click with an empty 200, and then posts the
232
+ text to the click's `response_url` through the Instance Effector. A post that fails is a `follow-up` line in
233
+ [`jr2 hooks <name>`](../reference/cli.md#hooks).
234
+
235
+ A message that everyone in the channel sees is a change of shared state. Write it as an `http` Effect of the Machine.
236
+
237
+ A Chart can invoke a Gate slot that its Open body lends in `lends.gates` (see [Charts](charts.md)). The Chart must
238
+ declare each event of the slot's `delivers` in its `events`.
239
+
240
+ [Answer a Gate from a forge](../guides/answer-a-gate-from-a-forge.md) shows a full example.
177
241
 
178
242
  ## Rules
179
243
 
@@ -189,20 +253,26 @@ A Pool that has a `wake` event serves it as a Gate with the id `source` (see [Po
189
253
  - jr2 refuses a second open Gate with the same id in one run.
190
254
  - `jr2 up` warns about a Gate above an Agent's state that takes an event that the Agent can also reach. The Gate's
191
255
  caller can send the event, and the Agent cannot.
256
+ - `jr2 up` warns about a Gate slot that hears a Hook and has no look and no `after` on its state, and about a look that
257
+ a package left `open` and that nobody bound. A person can still answer that Gate with `jr2 send`.
258
+ - A Gate slot's handler that throws, or that gives an event that the run does not take now, never faults the run. The
259
+ Hook's answer names the Gate in `refused`, and the run's feed shows `hook-refused`.
192
260
 
193
261
  ## Read next
194
262
 
195
- - [Answer a Gate](../guides/answer-a-gate.md)
263
+ - [Answer a Gate](../guides/answer-a-gate.md) and [Answer a Gate from a forge](../guides/answer-a-gate-from-a-forge.md)
196
264
  - [Open the Console](../guides/open-the-console.md)
197
- - [The Menu](the-menu.md), [Pools and Sources](pools-and-sources.md) and [Durable runs](durable-runs.md)
265
+ - [The Menu](the-menu.md), [Callers and Hooks](callers-and-hooks.md), [Pools and Sources](pools-and-sources.md) and
266
+ [Durable runs](durable-runs.md)
198
267
  - Reference: [`jr2 send`](../reference/cli.md#send), [`jr2 status`](../reference/cli.md#status),
199
268
  [`GET /runs/:runId`](../reference/http-api.md#get-runs-runid),
200
269
  [`POST /runs/:runId/gates/:gate/events`](../reference/http-api.md#post-runs-runid-gates-gate-events),
201
270
  [`GateView`](../reference/http-api.md#type-gateview) and [`sentBy`](../reference/authoring-surface/sentby.md)
202
- - [ADR-0011](../reference/decisions/0011-workflow-defined-events.md): a Gate is an addressable resource on a run
271
+ - [ADR-0011](../reference/decisions/0011-workflow-defined-events.md): a Gate is an addressable resource on a run, and a
272
+ Gate slot hears a Hook and looks before it waits
203
273
  - [ADR-0013](../reference/decisions/0013-the-agent-reaches-its-machine-through-a-container-it-cannot-read.md): an Agent
204
274
  cannot send to a Gate, and a send that the run does not take is refused
205
275
  - [ADR-0015](../reference/decisions/0015-authoring-surface-absorbs-the-mechanism.md): the position of an `on` key gives
206
276
  its event to a Gate or to an Agent
207
- - [ADR-0032](../reference/decisions/0032-the-console-unlocks-with-the-instance-token.md): the Console sends to a Gate
208
- with the Instance token
277
+ - [ADR-0032](../reference/decisions/0032-the-console-unlocks-with-a-callers-credential.md): the Console sends to a Gate
278
+ with a Caller's credential
@@ -143,6 +143,13 @@ has a door of its own. It does not use the door of the Machine inside it, becaus
143
143
  item. Thus the wrapper declares the door, with its `input` option. `workspace()` refuses a body that has a zod `input`,
144
144
  also a body that `setup()` made.
145
145
 
146
+ A `workspace()` that a parent Machine invokes parses its door too, before it makes a pod. When the parent passes less
147
+ than the door requires, the invoke fails with `workspace input invalid, before any pod: <zod message>`, and the parent's
148
+ `onError` takes it. This is a defect of the parent, so no pod is spent on it. The body gets the parsed value, as it does
149
+ on a root run: defaults are applied and unknown keys are removed. A `pool()` with a door does the same, and its error
150
+ starts `pool input invalid, before any worker`. The door of a plain child Machine is a type, and nothing parses it at
151
+ run time.
152
+
146
153
  ## The Vocabulary
147
154
 
148
155
  The Vocabulary of a Machine is the list of the workflow events that it takes. Each event comes from
@@ -153,7 +160,8 @@ jr2 makes three things from the Vocabulary:
153
160
 
154
161
  - **The Menu of each Agent.** Each event that an Agent can send is a tool for its model. The description of the event is
155
162
  the description of the tool. [The Menu](the-menu.md) tells how.
156
- - **The events that a Gate accepts.** `jr2 send`, the Console and a webhook can send only those events.
163
+ - **The events that a Gate accepts.** `jr2 send`, the Console and a [Hook](../reference/glossary.md#hook) can send only
164
+ those events.
157
165
  - **The check of each delivery.** jr2 parses the payload of each event with its schema before the Machine gets the
158
166
  event.
159
167
 
@@ -225,13 +233,13 @@ in the run to the feeds, also those of each child Machine.
225
233
 
226
234
  You see an Emit in these places:
227
235
 
228
- - `jr2 run`, and `jr2 logs -f`, write each Emit on stderr while they are attached to the run. The line holds `emit`,
229
- then the Emit as JSON. `jr2 logs` without `-f` writes no Emit.
236
+ - `jr2 run` and `jr2 watch` write each Emit while they are attached to the run. The line holds `emit`, then the Emit as
237
+ JSON. `jr2 run` and `jr2 watch --json` write it on stderr; `jr2 watch` writes it on stdout.
230
238
  - The Activity tab of the Console shows the type of each Emit that comes while the page is open.
231
239
  - The [Harness](../reference/glossary.md#harness) of each [Workspace](../reference/glossary.md#workspace) writes the
232
240
  Emits of its run in its log, beside the conversation of its Agents.
233
241
  - The feed of a run, [`GET /runs/:runId/events`](../reference/http-api.md#get-runs-runid-events), carries each Emit with
234
- its payload. This feed needs the Instance token.
242
+ its payload. This feed needs the `read` scope for the Workflow of the run.
235
243
 
236
244
  The feeds that need no token carry the type of an Emit, but not its payload. The payload is your data, as the context of
237
245
  a run is.
@@ -253,6 +261,8 @@ moves a Machine.
253
261
  - A payload schema is a flat `z.object`, and it has no `sender` field. jr2 writes `sender` on each event that a party
254
262
  sends.
255
263
  - When a Machine declares a door, the Orchestrator refuses a start whose input does not agree with it.
264
+ - A `workspace()` or a `pool()` under a parent parses its door before it makes a pod. An input that does not agree fails
265
+ the invoke.
256
266
  - The body of a `workspace()` declares no door. `workspace()` refuses a body that declares one.
257
267
  - The output of a run is the root `output` of its Machine. jr2 refuses an `output` on a final state directly below the
258
268
  root when the root has no `output`.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  title: Pools and Sources
3
- description: How a Pool runs one worker Machine for each work item that a Source gives, and how its run ends.
3
+ description: How a Pool runs one worker for each item of a Source, how a Hook brings items, and how its run ends.
4
4
  ---
5
5
 
6
6
  # Pools and Sources
@@ -31,7 +31,7 @@ example runs with no model. A worker is usually a Machine with an [Agent](../ref
31
31
  ```ts
32
32
  import { assign, fromPromise } from "xstate";
33
33
  import { z } from "zod";
34
- import { defineEvent, jr2Setup, pool, source } from "@jr2/orchestrator";
34
+ import { defineEvent, jr2Setup, pool, source, type PoolNextInput } from "@jr2/orchestrator";
35
35
 
36
36
  type Ticket = { id: string; title: string; needs: string[] };
37
37
 
@@ -50,7 +50,7 @@ const claimed = new Set<string>();
50
50
 
51
51
  // The Source: give the next ready ticket, or say how many tickets are not ready.
52
52
  const tickets = source<Ticket>({
53
- next: fromPromise(async ({ input }: { input: { active: string[] } }) => {
53
+ next: fromPromise(async ({ input }: { input: PoolNextInput }) => {
54
54
  const done = (id: string) => claimed.has(id) && !input.active.includes(id);
55
55
  const waiting = backlog.filter((t) => !claimed.has(t.id));
56
56
  const ready = waiting.find((t) => t.needs.every(done));
@@ -166,9 +166,9 @@ not a status in the output: a run that waits has no output yet.
166
166
  input to a worker. `itemInput` can copy a part of it into the worker's input.
167
167
  - The run input of a Pool is its own `input` schema, not the input of its worker. `cap` and `itemInput` read it.
168
168
  - The Gates of a worker show in the run. The id of each one starts with the id of its item, such as `T-1.review`.
169
- - The Pool takes the `wake` event of the Source at a Gate named `source`, for the whole run. A webhook or
170
- [`jr2 send`](../reference/cli.md#send) sends it. When the Pool is inside a different Machine, the Gate is
171
- `<path>.source`.
169
+ - The Pool takes the `wake` event of the Source at a Gate named `source`, for the whole run.
170
+ [`jr2 send`](../reference/cli.md#send) sends it. A Hook reaches the Source's handlers through the same Gate. When the
171
+ Pool is inside a different Machine, the Gate is `<path>.source`.
172
172
  - The [Vocabulary](../reference/glossary.md#vocabulary) of a Pool is only its `wake` event. Each worker keeps its own.
173
173
 
174
174
  A Pool is a Machine. It can be the root of a Workflow, or a different Machine can compose it.
@@ -177,11 +177,22 @@ A Pool is a Machine. It can be the root of a Workflow, or a different Machine ca
177
177
 
178
178
  ## Sources
179
179
 
180
- [`source(spec)`](../reference/authoring-surface/source.md) declares where a Pool gets its items. A Source has three
181
- parts ([`SourceSpec`](../reference/authoring-surface/type-sourcespec.md)):
180
+ [`source(spec)`](../reference/authoring-surface/source.md) declares where a Pool gets its items. A Source has one of
181
+ three forms:
182
182
 
183
- - `next`: an [xstate](https://xstate.js.org) promise actor that claims the next ready item. Its input is `{ active }`,
184
- the ids of the items that workers have now.
183
+ | Form | What it has | How the Pool gets items |
184
+ | ---- | ------------------------------ | ---------------------------------------------------------------------------------------- |
185
+ | Pull | `next` | The Pool calls `next`, and `next` gives one item at a time. |
186
+ | Read | `read` and `items` | The Pool reads the whole set, and starts the items that are new, one for each slot. |
187
+ | Push | `item` (a zod schema), `hooks` | Each [Hook](../reference/glossary.md#hook) delivery carries one item. The Pool holds it. |
188
+
189
+ ### Pull
190
+
191
+ A pull Source has three parts ([`SourceSpec`](../reference/authoring-surface/type-sourcespec.md)):
192
+
193
+ - `next`: an [xstate](https://xstate.js.org) promise actor that claims the next ready item. Its input is
194
+ `{ active, settled, input }`: the ids of the items that workers have now, the ids of the finished and failed items,
195
+ and the input of the Pool. Thus a Source can fan out over a plan with no state of its own.
185
196
  - `wake`: an optional event, made with [`defineEvent`](../reference/authoring-surface/defineevent.md), that tells the
186
197
  Pool that new work is ready.
187
198
  - `pollEvery`: an optional time in milliseconds. While the Pool waits, it calls `next` again at this interval.
@@ -193,6 +204,166 @@ ready. Without `open`, the Pool cannot tell a finished run from a blocked one. T
193
204
 
194
205
  A Source can be a queue, a generator, or a set that `next` queries again each time.
195
206
 
207
+ ### Read
208
+
209
+ A read Source reads the whole set each time. `read` is an `http` Effect marked `idempotent: true`, so that the read can
210
+ use a secret that the Instance Effector holds, or a promise. Its input is the input of the Pool. `items(output)` gives
211
+ the items in its answer, in order. Without `items`, the answer must be the list of items.
212
+
213
+ - The Pool reads when the run starts, when a worker finishes, on `wake`, when a Hook's handler gives `reread()`, and
214
+ after `pollEvery` milliseconds while it waits.
215
+ - One read fills each free slot. The Pool does not start an item whose id a worker has, or that finished in this run.
216
+ The items that do not get a slot stay in the Source: the next read finds them again.
217
+ - When a `reread()`, a `wake` or a finished worker comes during a read, the Pool reads one more time after that read.
218
+ Thus no change is lost.
219
+ - A read that fails gives no items. The next read tries again.
220
+ - An item whose `itemId` throws or gives no id is skipped, and the read starts the other items. The run's feed has a
221
+ `pool` line `skipped` that says which item and why, one time in the run's life.
222
+
223
+ The read is the truth. A Hook delivery only says "read again". An item that changed while it waited is read as it is
224
+ now.
225
+
226
+ ### Push
227
+
228
+ A push Source has no read. Each item comes in a Hook delivery, and the handler gives `carry(item)`. The Pool parses the
229
+ item with the `item` schema and holds it in its run, in the order of arrival. When a slot is free, the Pool starts the
230
+ oldest held item. Only a standing Pool of runs takes a push Source (see [A Pool of runs](#a-pool-of-runs)), because a
231
+ Pool that ends when no item is ready would end at once.
232
+
233
+ `hold` is the most items that the Pool holds, 1000 by default. A held item is in the run's saved state before the Hook
234
+ gets its answer, so a restart does not lose it.
235
+
236
+ ### A Hook reaches a Source
237
+
238
+ The `hooks` of a Source have the shape of a [Gate's](gates.md) handlers: a handler for each Hook kind and event key. A
239
+ handler reads `{ payload, caller, delivery, input }`. It is pure and synchronous, and it gives one of three answers:
240
+
241
+ - [`reread()`](../reference/authoring-surface/reread.md): read the Source again now. A pull or a read Source gives it.
242
+ - [`carry(item)`](../reference/authoring-surface/carry.md): here is the item. A push Source gives it.
243
+ - [`skip(reason)`](../reference/authoring-surface/skip.md): this delivery brings nothing, because of `reason`.
244
+
245
+ A handler never gives `undefined`. A handler that throws is a skip, with the error as its reason. The Hook reaches the
246
+ Source through the Pool's `source` Gate, so the Hook's `may.answer` must name the Pool's Workflow.
247
+
248
+ The answer of the Hook's door has a line in `sources` for each Source that heard the delivery. Its `verdict` is one of
249
+ these:
250
+
251
+ | `verdict` | What it means |
252
+ | ----------- | ------------------------------------------------------------------------------------------------ |
253
+ | `held` | The Pool holds the carried item. `key` is its id. |
254
+ | `existing` | The Pool holds the key already, a worker has it, or a run of the callee has it (`run` names it). |
255
+ | `hold-full` | The Pool holds `hold` items. When each Source says `hold-full`, the answer is 429. |
256
+ | `reread` | The Source reads again. |
257
+ | `skipped` | The handler gave `skip(reason)`, or threw. `reason` says why. |
258
+ | `paused` | The standing Pool is paused, and it holds nothing. `jr2 run <workflow>` resumes it. |
259
+
260
+ A 429 does not mark the delivery as seen, so the service can send it again. For a chat command or a click, the person
261
+ gets 200 and a text that says why ([The answer that a person reads](gates.md#the-answer-that-a-person-reads)).
262
+
263
+ A Source can also give `reply(verdict, { caller })`: the text that the person who acted on a chat service reads, from
264
+ its verdict. `verdict.verdict` is one of the names above, except `paused`. `held`, `existing` and `hold-full` also carry
265
+ the `item` and its `key`. The text of a `skipped` verdict shows only when no Gate or other Source gave a text, because a
266
+ skip says "not for me". See [The answer that a person reads](gates.md#the-answer-that-a-person-reads).
267
+
268
+ ## A Pool of runs
269
+
270
+ The worker of a Pool can be a [`dispatch(...)`](../reference/authoring-surface/dispatch.md) slot in place of a Machine.
271
+ Then each item is a run of its own of the slot's Workflow: it has its own row in `jr2 runs`, its own cancel and its own
272
+ Workspaces. The example starts one `triage` run for each alert that an alert service sends:
273
+
274
+ ```ts
275
+ import { z } from "zod";
276
+ import { carry, dispatch, pool, skip, source } from "@jr2/orchestrator";
277
+ import type {} from "../hooks/sentry.ts";
278
+
279
+ // The door of the triage Workflow: each alert is one run of it.
280
+ const triageDoor = z.object({ issue: z.string(), title: z.string() });
281
+
282
+ // A push Source: each Hook delivery carries an alert, and the Pool holds it until a slot is free.
283
+ const alerts = source({
284
+ item: z.object({ issue: z.string(), title: z.string(), level: z.string() }),
285
+ hooks: {
286
+ sentry: {
287
+ "issue.created": ({ payload }) => {
288
+ const issue = payload.data.issue;
289
+ if (issue.level !== "error" && issue.level !== "fatal") return skip(`level is ${issue.level}`);
290
+ return carry({ issue: issue.id, title: issue.title, level: issue.level });
291
+ },
292
+ },
293
+ },
294
+ });
295
+
296
+ // A standing Pool of runs: one triage run for each alert, at most 8 live at once.
297
+ export const machine = pool(dispatch("triage", { input: triageDoor, wait: true }), {
298
+ id: "alerts",
299
+ standing: true,
300
+ source: alerts,
301
+ itemId: (alert) => `sentry-${alert.issue}`,
302
+ cap: 8,
303
+ itemInput: (alert) => ({ issue: alert.issue, title: alert.title }),
304
+ });
305
+ ```
306
+
307
+ jr2 ships no kind for this alert service. Thus the Instance makes the `sentry` kind with `webhook()` from
308
+ `@jr2/orchestrator/hooks/webhook`, in a module of its own: `hooks/sentry.ts`. `jr2.config.ts` imports it for the Hook,
309
+ and the Workflow imports it for the types of its handlers.
310
+
311
+ A Pool of runs is different from a Pool of child Machines:
312
+
313
+ - **The key.** `itemId(item)` is the key of the item's run in the callee Workflow. Each start with the same key gets the
314
+ same run while the run's row lives: while the run is live, and for `runs.retain` after it ends. Thus an item never
315
+ starts two runs, also after a restart. `jr2 status` of the run prints the key, and the Hook delivery that carried the
316
+ item.
317
+ - **It stands.** A Pool of runs needs `standing: true`. It never ends by itself, and it has no output. When it has no
318
+ work, it waits for a Hook delivery, a `reread()`, a `wake` or its `pollEvery`.
319
+ - **`cap` counts live runs.** The `dispatch` slot must say `wait: true`, so that the Pool knows when a run ends. A run
320
+ that ends in any way frees its slot. A run that ends `failed`, or a start that the callee's door refuses, ends only
321
+ its item: the run is not the Pool's child. The run's feed has a `pool` line for each start and each end.
322
+ - **Independent runs.** A cancel of the Pool does not cancel its runs. `customize()` does not reach into them.
323
+
324
+ ### jr2 keeps the standing run
325
+
326
+ Nobody starts the run of a standing Pool. jr2 starts it at the publish that adds the Workflow, and keeps one live run of
327
+ it after that:
328
+
329
+ - **A restart keeps it.** The restored run keeps its held items and its live runs. jr2 starts no second one.
330
+ - **A publish hands it over.** At each publish, jr2 starts a new run under the new Assembly and ends the old run `done`.
331
+ `jr2 status <old run>` prints `handed over to: <new run>`. The new run takes the held items, parsed again by the new
332
+ `item` schema (an item that does not parse is dropped, with a `carry-refused` line on the feed), and it sends each
333
+ active key again: the key finds its run, so no item starts a second run. jr2 also hands the run over when it has taken
334
+ 500 items or is one day old, and when its Assembly can no longer be read (drift).
335
+ - **An `error` is handed over after a back-off** of 1 s, then 4 s, then 16 s. A run that ends `failed` is handed over as
336
+ a run that ends `error` is. While the Pool waits, it has no live run, and the Hook's door answers 503 to a delivery
337
+ for it, so the service sends it again. For a chat command or a click, the person gets 200 and a text that says why
338
+ ([The answer that a person reads](gates.md#the-answer-that-a-person-reads)).
339
+ - **A bound with no run started is handed over after the back-off too.** When a run takes its 500 items and none of them
340
+ starts a run (the callee's door refuses each one, for example), jr2 ends the run and waits before the next run. The
341
+ Pool knows an item that ends with no run only during the run that took it. Without the wait, each new run would take
342
+ the same items again at once, with no end.
343
+ - **jr2 pauses a Pool whose items start no run.** One rule counts a run that ends `error`, ends `failed`, or takes its
344
+ bound: when no item that the run took itself starts a run, the run counts. An active key that a hand-over carried to
345
+ the run does not count, because its run started before. A run in which an item that it took starts a run starts the
346
+ count again. After 3 hand-overs from counted runs in a row, jr2 pauses the Pool when the next counted run ends.
347
+ `jr2 status` and the Orchestrator's log say which end paused the Pool.
348
+ - **A cancel is a pause.** `jr2 send <run> --event CANCEL` ends the run `cancelled`, and jr2 starts no run of the
349
+ Workflow at a restart or a publish. A delivery gets the verdict `paused`, and the Pool holds nothing. `jr2 status`
350
+ prints the Pool as paused, with the number of items it holds.
351
+ - **`jr2 run <workflow>` resumes it.** It is the one start of a standing Workflow that jr2 accepts: the new run takes
352
+ the held items and the active keys of the paused run, and `jr2 status` prints `resumed by:` with the Caller.
353
+ `jr2 run <workflow> --drop-held` resumes it with no held item. While the Pool is live, `jr2 run` of it is refused.
354
+
355
+ `jr2 up` prints the chain from each Hook to each standing Pool and its callee (`standing: sentry → alerts → triage`),
356
+ and the state of each standing Pool after the publish. It warns when two standing Pools dispatch to one Workflow: `cap`
357
+ is for each Pool, so the caps add.
358
+
359
+ `pool()` and the load walk refuse a Pool that cannot run, and the message tells the fix:
360
+
361
+ - A worker that is a Workflow name. Write `dispatch("<name>", { input, wait: true })`.
362
+ - A Pool of runs without `standing: true`, a `dispatch` slot without `wait: true` or with a `target`, and a Pool of runs
363
+ without `cap`.
364
+ - `standing: true` on a Pool of child Machines, and a push Source on a Pool that does not stand.
365
+ - A standing Pool that is not the top Machine of its Workflow. Put it in a file of its own in `workflows/`.
366
+
196
367
  ## Work Source and the Ready-set
197
368
 
198
369
  A [Work Source](../reference/glossary.md#work-source) is a Source over an issue tracker, such as GitHub issues or Jira.
@@ -210,21 +381,26 @@ the issue tracker.
210
381
  ## Rules
211
382
 
212
383
  - `itemId` gives each item an id of its own, and the same id each time. The Pool uses it as the id of the worker and as
213
- the key in `items`.
214
- - `next` must not give an item that a worker has now, or an item that a worker finished. The Pool gives `next` the
215
- active ids only. Your Source keeps the finished items.
216
- - `cap` is a number, or a function of the run input. Without `cap`, one worker runs at a time.
384
+ the key in `items`. An item whose `itemId` throws or gives no id is skipped: a push Source's verdict is `skipped`, and
385
+ a read Source puts a `skipped` line on the feed. In a pull Source, the run ends `error`.
386
+ - `next` must not give an item in `active` or in `settled`.
387
+ - `cap` is a number, or a function of the run input. Without `cap`, one worker runs at a time. A Pool of runs must state
388
+ `cap`.
217
389
  - `onDrained` takes only `"final"`: the run ends.
218
- - The Pool calls `next` again on `wake` or after `pollEvery` only while it waits: workers run and no item is ready.
390
+ - The Pool calls `next` again on `wake`, on `reread()` or after `pollEvery` only while it waits: workers run and no item
391
+ is ready. A `wake` or a `reread()` that comes while `next` runs makes the Pool call `next` once more.
219
392
  - A `task` in a Pool needs a `branch` in its input. Make it in `itemInput`, for example from the item id.
220
393
 
221
394
  ## Read next
222
395
 
223
- - [Run one worker per item](../guides/run-one-worker-per-item.md)
396
+ - [Run one worker per item](../guides/run-one-worker-per-item.md) and
397
+ [Start a run for each alert](../guides/start-a-run-for-each-alert.md)
398
+ - [Callers and Hooks](callers-and-hooks.md)
224
399
  - [Composition](composition.md)
225
400
  - [Gates](gates.md)
226
401
  - Reference: [`pool()`](../reference/authoring-surface/pool.md), [`source()`](../reference/authoring-surface/source.md),
227
- [`PoolSpec`](../reference/authoring-surface/type-poolspec.md),
402
+ [`carry()`](../reference/authoring-surface/carry.md), [`reread()`](../reference/authoring-surface/reread.md),
403
+ [`skip()`](../reference/authoring-surface/skip.md), [`PoolSpec`](../reference/authoring-surface/type-poolspec.md),
228
404
  [`SourceSpec`](../reference/authoring-surface/type-sourcespec.md),
229
405
  [`SourceNextResult`](../reference/authoring-surface/type-sourcenextresult.md)
230
406
  - [ADR-0017](../reference/decisions/0017-pool-over-a-source-port.md): the run loop is a Pool over a Source