@powerhousedao/reactor-workflow 6.2.3-dev.54 → 6.2.3-dev.56

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -87,10 +87,10 @@ rather than restated here.
87
87
  **Workflow execution is a singleton pinned to one reactor.** The engine forks
88
88
  child processes, so no browser reactor can compose it; the hazard a guard is
89
89
  needed for is two Node replicas over one run journal. It is not theoretical:
90
- opening the journal runs `recoverOrphanedRuns` and `recoverAbandonedRuns`,
91
- which close out every RUNNING and PENDING run that is not in **this** process's
92
- in-flight set — so a second replica booting marks the first one's live runs
93
- FAILED, and then both arm every trigger and both poll it.
90
+ opening the journal fails every RUNNING run that is not in **this** process's
91
+ in-flight set, and starting the runtime admits every PENDING one — so a second
92
+ replica booting fails the first one's live runs, runs its queued ones a second
93
+ time, and then both arm every trigger and both poll it.
94
94
 
95
95
  The guard is a durable claim on the journal's own database
96
96
  (`reactor/singleton-lease.ts`, one row in `workflow_singleton`), taken by the host
@@ -122,13 +122,14 @@ the sweeps and the supervisor are per process.
122
122
  still on its way in, is recorded once. Every trigger firing (an operation, a
123
123
  piece trigger's items, a schedule slot, a webhook delivery) is journaled as a
124
124
  PENDING run with its payload before it fires. If its source will not deliver
125
- it again, the refusal fails that run, so it can be rerun. If it will (a sync
125
+ it again, the refusal leaves that run PENDING, and the next owner starts it;
126
+ a manual fire's caller hears the refusal, so its run is failed. If it will (a sync
126
127
  webhook answered 503 for the sender to retry, or piece items whose cursor the
127
128
  stopped trigger put back), the run is cancelled as redelivered. A firing
128
129
  refused before its run was written is left to its source: the operation is
129
130
  replayed, the sender retries, the schedule slot stays due, the trigger's
130
131
  cursor is put back or never moved. A close-out write that fails leaves the
131
- run PENDING, and the next owner fails it when it opens the journal. Without a
132
+ run PENDING, and the next owner starts it. Without a
132
133
  working journal a firing that still runs is unrecorded, and its refusal is
133
134
  logged as an error.
134
135
  - A renewal that fails or hangs is retried every 5s. A holder that has gone
@@ -270,6 +271,19 @@ transport finds that child's code by walking up to this package's own
270
271
  `package.json` and reading `dist/worker-entry.js`, so `pnpm build` must have run
271
272
  before anything executes a piece — including the suites here.
272
273
 
274
+ Children come from one warm pool (`worker/pool.ts`, ADR 0007). Every piece
275
+ request leases an idle child, preferring one that has the piece loaded, and
276
+ returns it when it settles: a step, a trigger hook, an editor's options or
277
+ describe. A run holds no child between steps. Trigger hooks have children of
278
+ their own, so a burst of runs can't delay a poll, and so does the editor (one
279
+ child, which exits when idle). A child is retired when it
280
+ dies or times out, when a request switches TLS verification off, past 80% of
281
+ its heap budget, after `PH_WORKFLOWS_WORKER_MAX_REQUESTS` requests, or after a
282
+ minute idle above `PH_WORKFLOWS_WORKER_MIN`.
283
+
284
+ Piece code is trusted: what a piece keeps in module state carries over to the
285
+ next request its child serves, whichever workflow sent it.
286
+
273
287
  That entry is `src/worker/entry.ts`: the piece worker from `src/pieces`, plus
274
288
  `ctx.reactor` (`src/worker/reactor.ts`), which needs `@powerhousedao/reactor`
275
289
  and so lives outside the piece layer. For an action, trigger or option resolver
@@ -413,6 +427,18 @@ too. Unless an author changed them, a workflow now:
413
427
  That is what the fields have said since the first schema; what changed is that
414
428
  they are true.
415
429
 
430
+ - **Every run is admitted twice**: by its workflow's own `concurrency`, then
431
+ across workflows (`reactor/run-dispatcher.ts`). The second admits at most
432
+ `PH_WORKFLOWS_RUNS_IN_FLIGHT` runs at once, and none while the main thread's
433
+ event-loop utilization is above `PH_WORKFLOWS_ADMIT_MAX_ELU`: that thread
434
+ caps throughput, so a run admitted past it only adds latency. One run is
435
+ always admitted when none is running. A run waiting at either gate stays
436
+ PENDING, a manual fire included, so a crash leaves it for the next boot.
437
+ - **The next boot starts what a stopped reactor journaled.** Its PENDING runs
438
+ are admitted when the runtime starts, oldest first, through both gates. A
439
+ run admitted at `FIRE_CRASH_BUDGET` (3) boots without ever starting is
440
+ failed instead, so a fire that takes the reactor down is not replayed
441
+ forever. RUNNING runs are still failed at open, and rerunnable.
416
442
  - **Concurrency is process-local**, which is exactly right: workflow execution
417
443
  is a singleton pinned to one reactor (see **Placement** above), so this
418
444
  process is the deployment's whole run set. A firing SINGLETON drops is
@@ -450,8 +476,11 @@ they are true.
450
476
  changed or not), and a **disable then re-enable**. Both work across a restart.
451
477
  A park blocks the registrations of the version that failed and older ones
452
478
  only: a newer version always arms, and lifts the park it outlived. Every park
453
- write runs on the trigger supervisor's lane, with enable and disable, so a
454
- park cannot land between an enable's check and its write. An unresolvable
479
+ write runs on the workflow's trigger lane, with its enable and disable, so a
480
+ park cannot land between an enable's check and its write. Lanes are per
481
+ workflow: a slow hook delays only its own workflow, and a tick fires due
482
+ schedules first, then runs polls alongside each other, as many as
483
+ `PH_WORKFLOWS_TRIGGER_SLOTS`. An unresolvable
455
484
  piece leaves a PARKED row alone as well, rather than turning it ERROR and
456
485
  letting the ERROR row's own retry arm it.
457
486
  - **`retryOn` empty means every error is retryable.** The schema reads "error
@@ -563,8 +592,14 @@ explanation behind it.
563
592
  | ------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------- |
564
593
  | `PH_WORKFLOWS_SECRETS_MASTER_KEY` | generated key file | 64 hex chars (32 bytes) encrypting connection secrets at rest (`reactor/secret-store.ts`) |
565
594
  | `PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES` | unset | Addresses or CIDRs a piece may reach, widening the default policy (`reactor/lib.ts`) |
566
- | `PH_WORKFLOWS_RUN_CONCURRENCY` | `4` | Runs executing at once; one forked node child each (`worker/pool.ts`) |
567
- | `PH_WORKFLOWS_RUN_QUEUE_DEPTH` | `0` | Runs that may wait for a slot before new ones are refused; `0` waits without limit |
595
+ | `PH_WORKFLOWS_RUN_CONCURRENCY` | from the host | Most piece children at once (`worker/pool.ts`); unset, sized from the cores and memory, at most 32 |
596
+ | `PH_WORKFLOWS_RUN_QUEUE_DEPTH` | `0` | Requests that may wait for a child before new ones are refused; `0` waits without limit |
597
+ | `PH_WORKFLOWS_WORKER_MEMORY_MB` | `256` | Each child's V8 heap (`--max-old-space-size`); a child past 80% of it is retired |
598
+ | `PH_WORKFLOWS_WORKER_MAX_REQUESTS` | `1000` | Requests a child serves before it is retired |
599
+ | `PH_WORKFLOWS_WORKER_MIN` | `1` | Idle children kept warm; `0` lets every idle child exit |
600
+ | `PH_WORKFLOWS_TRIGGER_SLOTS` | `1` | Children reserved for trigger hooks |
601
+ | `PH_WORKFLOWS_RUNS_IN_FLIGHT` | 8 × the pool | Runs admitted at once across workflows (`reactor/run-dispatcher.ts`) |
602
+ | `PH_WORKFLOWS_ADMIT_MAX_ELU` | `0.8` | Main-thread event-loop utilization above which no new run is admitted; one always runs |
568
603
  | `PH_WORKFLOWS_MAX_QUEUED_FIRINGS` | `100` | Firings of ONE workflow that may wait for its concurrency slot; past it a firing is journaled CANCELLED (`reactor/run-gate.ts`) |
569
604
  | `PH_WORKFLOWS_POLL_INTERVAL_MS` | `60000` | Cadence for a polling trigger that names none of its own |
570
605
  | `PH_WORKFLOWS_WEBHOOK_RECONCILE_MS` | `900000` | How often a webhook trigger re-registers with its provider |
@@ -577,6 +612,7 @@ explanation behind it.
577
612
 
578
613
  Each numeric one parses as `Number(raw) || default`: a value that is not a
579
614
  positive number falls back silently rather than failing at boot.
615
+ `PH_WORKFLOWS_WORKER_MIN` also takes `0`.
580
616
 
581
617
  **Run retention is ON by default, at 30 days.** It used to be opt-in, which
582
618
  meant unbounded growth on every host that did not know to set the variable —