@powerhousedao/reactor-workflow 6.2.3-dev.49 → 6.2.3-dev.51

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -52,6 +52,10 @@ rather than restated here.
52
52
  `apfile://` refs, the size ceiling and a host-injected fetcher have no
53
53
  upstream equivalent. An `ApFile` a processor builds is flattened to a plain
54
54
  object at that boundary: a class instance does not survive the worker IPC.
55
+ Its `base64` is computed on first read, as upstream's getter is, so a piece
56
+ that only reads `data` holds one copy of the file. A `streaming: true` FILE
57
+ prop gets upstream's `ApStreamingFile` instead: `{ filename, extension,
58
+ size, body }`, with `body` read from the staged file or the network.
55
59
  - **Prop validation**, from the same place. Before an action's `run()` or any
56
60
  trigger hook but `onDisable`, a prop left unset takes its `defaultValue` and
57
61
  the coerced values go through the engine's `validateProperty`. A failure is
@@ -78,6 +82,93 @@ rather than restated here.
78
82
  the text of an HTML error page reach the run journal. Redaction runs last,
79
83
  over the formatter's output as well.
80
84
 
85
+ ## Placement: one reactor runs workflows
86
+
87
+ **Workflow execution is a singleton pinned to one reactor.** The engine forks
88
+ child processes, so no browser reactor can compose it; the hazard a guard is
89
+ needed for is two Node replicas over one run journal. It is not theoretical:
90
+ opening the journal runs `recoverOrphanedRuns` and `recoverAbandonedRuns`,
91
+ which close out every RUNNING and PENDING run that is not in **this** process's
92
+ in-flight set — so a second replica booting marks the first one's live runs
93
+ FAILED, and then both arm every trigger and both poll it.
94
+
95
+ The guard is a durable claim on the journal's own database
96
+ (`reactor/singleton-lease.ts`, one row in `workflow_singleton`), taken by the host
97
+ **before** the runtime is built, and refused by name when another live process
98
+ holds it (`WorkflowSingletonConflictError`). The
99
+ `trigger_state.lease_owner` / `lease_expires_at` columns stay in the schema,
100
+ always null and unread: they were per trigger, the wrong granularity, since
101
+ the sweeps and the supervisor are per process.
102
+
103
+ - The lease is valid for 60s and renewed every 20s from the moment it is
104
+ claimed, so a slow compose cannot outlast it. A compose that fails after the
105
+ claim releases it, and so does `stop()`, so the next boot owns workflows
106
+ immediately instead of waiting out the TTL.
107
+ - An **expired** lease is taken over: a killed process does not lock workflows
108
+ out until a human intervenes. A claim under the holder's own owner name may
109
+ take it over earlier, once the holder's heartbeat has been silent for two
110
+ renewal periods (40s), but never from a holder that is still renewing: the
111
+ new process opening the journal would fail the old one's live runs.
112
+ - Each claim records a random **instance** token, never configured. Heartbeat
113
+ and release match on owner and instance, so during a rolling deploy under
114
+ one stable owner name the old process can neither renew nor delete the new
115
+ process's lease.
116
+ - A heartbeat that finds the lease held by another claim logs an **error**
117
+ naming the holder, stops renewing and calls `onLost`. The host shuts the
118
+ runtime down: no trigger, webhook or manual run starts after that, and the
119
+ host reports its triggers unavailable. It does not re-claim; workflows come
120
+ back on the next boot, since re-arming needs a fresh compose.
121
+ - A firing the shutdown refuses, whether queued for its concurrency slot or
122
+ still on its way in, is recorded once. Every trigger firing (an operation, a
123
+ piece trigger's items, a schedule slot, a webhook delivery) is journaled as a
124
+ PENDING run with its payload before it fires. If its source will not deliver
125
+ it again, the refusal fails that run, so it can be rerun. If it will (a sync
126
+ webhook answered 503 for the sender to retry, or piece items whose cursor the
127
+ stopped trigger put back), the run is cancelled as redelivered. A firing
128
+ refused before its run was written is left to its source: the operation is
129
+ replayed, the sender retries, the schedule slot stays due, the trigger's
130
+ cursor is put back or never moved. A close-out write that fails leaves the
131
+ run PENDING, and the next owner fails it when it opens the journal. Without a
132
+ working journal a firing that still runs is unrecorded, and its refusal is
133
+ logged as an error.
134
+ - A renewal that fails or hangs is retried every 5s. A holder that has gone
135
+ 30s without a renewal it knows landed reports itself lost
136
+ (`reason: "unrenewable"`), before a stale same-owner claim (40s) or expiry
137
+ (60s) could take the lease: journal writes are best-effort, so it would
138
+ otherwise keep running workflows beside the next owner. Switchboard then
139
+ goes down through its fatal shutdown, since nobody else runs workflows,
140
+ and the restarted process re-claims; an embedded host without one stays
141
+ without workflows and logs that a restart is needed.
142
+ - Over a journal no other process can open (embedded PGlite) nothing can take
143
+ the lease, so the holder never fences itself; a stalled renewal is logged.
144
+ - Nothing fences the journal itself: writes do not check the lease, so a
145
+ process keeps writing until it notices the loss.
146
+ - **A refused claim does not take the API down.** The host boots WITHOUT the
147
+ workflow runtime and warns, naming the current owner: no trigger fires here
148
+ and the workflow GraphQL face is absent, while inspection, GraphQL, sync, MCP
149
+ and every drive serve normally. The one thing this process must not do is run
150
+ workflows against a journal a live process owns; aborting the whole boot over
151
+ it turned "not allowed to run one component" into an outage — and, with a
152
+ random owner name, into a crash loop for the TTL after every unclean kill.
153
+ The host retries the claim every 20s and starts workflows once it succeeds,
154
+ so a rolling deploy whose slots have different owner names does not end
155
+ with nobody running them. This is only for a host that never composed: one
156
+ that lost the lease after composing stays without workflows until restart.
157
+ - **The default owner is stable**: `<hostname>/<fingerprint of the journal's
158
+ storage location>`. So one deployment slot restarting re-claims its OWN lease
159
+ once the killed process's heartbeat is stale (40s) rather than waiting out
160
+ the whole TTL. A genuine second replica still differs
161
+ by hostname or by the journal it points at. The storage location is hashed,
162
+ never printed: it can be a Postgres URL with credentials, and the owner name
163
+ goes into a database row and every log line about the lease. The case a stable
164
+ name cannot separate is two processes on ONE host over ONE journal, which is a
165
+ misconfiguration those two already share — and it is not silent: the loser's
166
+ heartbeat finds the lease taken and says so by name.
167
+ - `PH_WORKFLOWS_SINGLETON_OWNER` is still the operator's contract and overrides
168
+ the derived name. Set it per deployment slot when the hostname is not stable
169
+ (a fresh container id each deploy) or when two slots share a journal on
170
+ purpose.
171
+
81
172
  ## How the host composes it
82
173
 
83
174
  The engine names no host type. `WorkflowRuntimeHostDeps` (`src/reactor/host.ts`)
@@ -102,6 +193,41 @@ next boot instead of vanishing; a fresh registration starts at head, so history
102
193
  is never replayed. `onOperations` journals a matched fire before it returns, so
103
194
  the cursor never passes an event that is not yet durable.
104
195
 
196
+ **A fire that crashes the reactor is bounded.** The durable cursor is what
197
+ makes an operation written while the runtime was down catch up — and it is also
198
+ what re-delivers, on every boot, an operation whose fire takes the process down
199
+ before anything is journaled (the EPIPE boot loop). So the dedupe row counts
200
+ **deliveries**, committed before the risky work, which is the only way a crash
201
+ that leaves nothing behind can be counted at all:
202
+
203
+ - A delivery whose claim already holds a run id is an ordinary duplicate and is
204
+ suppressed, as before. So is one whose key fired with no run linked — run
205
+ without a journal row, its run erased, or written before claims existed —
206
+ which holds the `FIRED_WITHOUT_RUN_ID` marker rather than NULL.
207
+ - A delivery whose claim holds **no** run id is retried: the previous attempt
208
+ died before it journaled anything, and losing a legitimate trigger to a
209
+ transient store failure would be worse than the loop.
210
+ - Past `FIRE_CRASH_BUDGET` (3) such deliveries the fire is **abandoned**, with
211
+ a FAILED run naming the loop — visible, and rerunnable once the cause is
212
+ fixed, instead of a reactor that crashes on every boot and says nothing. The
213
+ FAILED run is linked to the key, so later deliveries are duplicates.
214
+
215
+ The **count** and the **claim** are deliberately different writes. Counting is
216
+ its own committed transaction, because a delivery that leaves nothing behind
217
+ has to be countable. Claiming is `run_id`, set under a `WHERE run_id IS NULL`
218
+ guard in the SAME transaction as the run row: the guard takes the row's lock,
219
+ so of two concurrent deliveries of one operation the second blocks until the
220
+ first commits and then matches no row — exactly one delivery fires, and a crash
221
+ in between rolls the claim back with the run it failed to journal. Were the
222
+ bump to ride along inside the claim transaction, a run insert that takes the
223
+ process down would roll the count back too and the budget could never reach its
224
+ limit.
225
+
226
+ Log writes on the piece-log and run-failure paths are truncated before the
227
+ write (`MAX_LOG_LINE_CHARS`): a piece error carrying an HTML error page is a
228
+ multi-megabyte write to a pipe that may be blocked, and bounding the write is
229
+ cheaper than handling the throw.
230
+
105
231
  A workflow document's `DELETE_DOCUMENT` disarms it as disabling does: its
106
232
  deliveries stop once the operation is indexed, a piece trigger's `onDisable`
107
233
  runs, and then its trigger row, its FLOW `ctx.store` partition, its webhook
@@ -117,7 +243,8 @@ runs `run_document` ties to it, the runs whose trigger payload names it as
117
243
  workflow's own runs (`run.workflow_id`, test runs included), and every rerun
118
244
  of those, transitively. Their
119
245
  `step_execution` and `run_document` rows go with them; a `trigger_dedupe` row
120
- they claimed keeps its key and loses its `run_id`. A run erased while it is
246
+ they claimed keeps its key, with `run_id` set to the `FIRED_WITHOUT_RUN_ID`
247
+ marker so a redelivery is still a duplicate. A run erased while it is
121
248
  still executing journals nothing more from this process. The read model's
122
249
  fence is `"skip"`: the journal lives on the relational handle, not the
123
250
  reactor's, so a run the purge races can still be written after the marker was
@@ -244,6 +371,150 @@ Which port a step takes changes which steps run, never the order steps are
244
371
  reached in. The studio's step outline (`stepOutline` in the workflow editor)
245
372
  lists steps in this order.
246
373
 
374
+ ## The workflow policy
375
+
376
+ A workflow document carries a `policy` block, and every field in it was schema
377
+ and editor only until W3.3: nothing read `concurrency`, `runTimeoutSeconds`,
378
+ `defaultRetry` or `onFailure`, so an author who set them got no behaviour and
379
+ no warning. They are enforced now, and the fields that are **not** are marked
380
+ `NOT YET ENFORCED` in the document model's own SDL rather than left to look
381
+ live. `reactor/policy.ts` resolves the block. A definition with **no** policy
382
+ at all enforces nothing, but only hand-built definitions lack one: `policy` is
383
+ non-null in the workflow model since v1, so every workflow document, old or
384
+ new, carries one.
385
+
386
+ | Field | Where | Behaviour |
387
+ | -------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------- |
388
+ | `concurrency` | `reactor/run-gate.ts` | SINGLETON drops a firing while a run is active; QUEUE serialises; PARALLEL runs concurrently |
389
+ | `maxParallelRuns` | `reactor/run-gate.ts` | Bounds PARALLEL; null is unbounded. SINGLETON and QUEUE are 1 by definition |
390
+ | `runTimeoutSeconds` | `pieces/engine/coordinator.ts` | A run deadline from FIRING time, checked between steps and bounding every retry wait; expiry ends the run CANCELLED |
391
+ | `defaultRetry`, step `retry` | `pieces/engine/retry.ts` | Attempts, backoff, delays and `retryOn`; attempts land on the step's journal row |
392
+ | `onFailure` | `reactor/service.ts` | On a trigger's failed run: PARK parks the trigger; NOTIFY logs at error level; IGNORE does nothing. A failed manual run or rerun changes nothing |
393
+ | `maxSuspensionDays` | — | **Not enforced**: nothing suspends. Waitpoints, `run.pause` and `generateResumeUrl` all throw |
394
+ | `retainRunsDays` | — | **Not enforced** per workflow; `PH_WORKFLOWS_RUN_RETENTION_DAYS` is the journal-wide control |
395
+ | `journalAsDocument` | — | **Not enforced**: the journal is relational, and there is no run document model |
396
+ | step `idempotencyKeyExpression` | — | **Not enforced**: a fire dedupes on its trigger operation or a trigger item's `_dedupe_key` |
397
+
398
+ **Enforcing the policy changes every deployed workflow.** The model's initial
399
+ values are `concurrency: QUEUE`, `onFailure: PARK` and
400
+ `runTimeoutSeconds: 3600`, and documents written before enforcement carry them
401
+ too. Unless an author changed them, a workflow now:
402
+
403
+ 1. **Serialises its runs** (QUEUE): a firing waits while a run of the same
404
+ workflow executes.
405
+ 2. **Parks its trigger on the first terminal failure of a run it fired**
406
+ (PARK; a failed manual run or rerun does not count): the trigger
407
+ stops firing until the workflow is re-published or re-enabled.
408
+ 3. **Has a one-hour run deadline counted from firing** (3600s): queue wait
409
+ included; past it the run ends CANCELLED, before its next step.
410
+ 4. **Cancels firings past 100 waiters** (`PH_WORKFLOWS_MAX_QUEUED_FIRINGS`):
411
+ an overflowing firing is journaled CANCELLED instead of run.
412
+
413
+ That is what the fields have said since the first schema; what changed is that
414
+ they are true.
415
+
416
+ - **Concurrency is process-local**, which is exactly right: workflow execution
417
+ is a singleton pinned to one reactor (see **Placement** above), so this
418
+ process is the deployment's whole run set. A firing SINGLETON drops is
419
+ journaled as a CANCELLED run rather than discarded — a firing that vanished
420
+ is indistinguishable from a trigger that never fired.
421
+ - **The QUEUE is bounded**, at `PH_WORKFLOWS_MAX_QUEUED_FIRINGS` waiting
422
+ firings per workflow (100). QUEUE means latency, not failure — but an
423
+ unbounded queue means neither: a document-event trigger on a busy type
424
+ enqueues faster than the workflow runs, every waiter holds its payload and
425
+ its promise, and the lane grows until the process dies. A firing that
426
+ overflows the depth is journaled CANCELLED exactly as a SINGLETON refusal is.
427
+ - **A sync webhook answers a refused firing without a 500.** A firing refused
428
+ as parked, by SINGLETON, or as stale after its wait gets 409; one refused by
429
+ a full queue or that waited past its deadline gets 429. A provider retries a
430
+ 500, and each retry would journal another CANCELLED run; 500 stays for real
431
+ failures.
432
+ - **A firing that waited reads the workflow again before it runs.** If it is
433
+ no longer ENABLED, was re-published, or was parked while the firing waited,
434
+ the firing is journaled CANCELLED without running a step.
435
+ - **The run deadline starts at FIRING time, not at admission.** Queue time is
436
+ part of the time the run took: a firing that waits past its
437
+ `runTimeoutSeconds` for a slot is CANCELLED without executing a single step,
438
+ rather than running its side effect long after the timeout that was supposed
439
+ to bound it. The document read counts too.
440
+ - **PARKED holds for every trigger kind, and a restart does not clear it.** The
441
+ park is a runtime override of the document's enabled-ness: parking writes a
442
+ `workflow_park` row naming the published version that failed, never the
443
+ document, so the document still says ENABLED and re-arming from it — which is
444
+ what a reboot does for every workflow it finds — would un-park the broken
445
+ workflow and resume firing it. A schedule or piece trigger's `trigger_state`
446
+ row also turns PARKED, so the supervisor stops polling it; a document-event,
447
+ document-lifecycle or webhook trigger is left unregistered; and any firing
448
+ that still arrives is journaled CANCELLED. Only two things clear a park: a
449
+ **re-publish** (the published version moves past the one that failed, trigger
450
+ changed or not), and a **disable then re-enable**. Both work across a restart.
451
+ A park blocks the registrations of the version that failed and older ones
452
+ only: a newer version always arms, and lifts the park it outlived. Every park
453
+ write runs on the trigger supervisor's lane, with enable and disable, so a
454
+ park cannot land between an enable's check and its write. An unresolvable
455
+ piece leaves a PARKED row alone as well, rather than turning it ERROR and
456
+ letting the ERROR row's own retry arm it.
457
+ - **`retryOn` empty means every error is retryable.** The schema reads "error
458
+ classes that are retryable; everything else fails terminally on attempt 1",
459
+ but the shipped default is an empty list, and taking that literally would
460
+ make every `maxAttempts` a lie. A non-empty entry matches the error's class
461
+ name exactly or appears anywhere in its message, case-insensitively, so both
462
+ `["HostCallTimeoutError"]` and `["429"]` work.
463
+ - `maxAttempts` is clamped to 10 and one backoff wait to 5 minutes: each
464
+ attempt re-runs a side effect and holds the run's worker slot.
465
+ - **Only the attempt is retried, never the resolution.** A step's input is
466
+ resolved once, before the first attempt, because resolution reads the scope
467
+ and nothing an attempt changes: an `UnresolvedReferenceError` names a key
468
+ that will not exist on attempt five either. A resolution failure fails the
469
+ step immediately, with no backoff waits spent on it.
470
+ - **A retry wait is clipped to the run deadline, and the deadline wins.** A
471
+ backoff longer than the time left is served out only as far as the deadline,
472
+ and then the step gets no further attempt: running one would be a side effect
473
+ after the run was already over. The attempt that failed is still journaled,
474
+ no error port is taken — nothing downstream may run after the run has ended —
475
+ and the run reads CANCELLED, since the clock stopped it rather than the
476
+ workflow failing.
477
+ - **A run its deadline cancelled can be rerun**, like a FAILED one: it is
478
+ journaled with the error name `RunDeadlineExceeded`, and the rerun replays
479
+ the steps that finished. A firing refused before it ran (parked, SINGLETON,
480
+ stale, queue full, expired in the queue) is CANCELLED without that name and
481
+ is not rerunnable; fire the workflow again instead.
482
+ - A step that DECLARES a `retry` block overrides `defaultRetry`, whatever the
483
+ block resolves to — `{maxAttempts: 1}` is an author saying "not this one".
484
+ - An **INDETERMINATE** step is never retried and never replayed: a retry would
485
+ be a second write. A rerun does execute it again, so it can write twice. See
486
+ **Indeterminate steps** below.
487
+
488
+ ## Indeterminate steps
489
+
490
+ A piece's call of its host is capped (`PH_WORKFLOWS_HOST_CALL_TIMEOUT_MS`, 10s,
491
+ raised to the step's own `timeoutSeconds` when that is longer; a trigger hook
492
+ or a design-time call takes the same cap, raised to its own timeout), and always
493
+ clipped to end a margin before the step's kill deadline, so the call's own
494
+ timeout is what the step reports. A **writing** call that times out —
495
+ `store.put`, `store.delete` — may well have been committed, so the step records
496
+ `INDETERMINATE` rather than FAILED: reporting a failure for a write that landed
497
+ is a claim nobody can stand behind, and it was happening (the 10s cap against a
498
+ dispatch under load). A reactor write still unfinished at the step deadline
499
+ (`ReactorJobPendingError`) is INDETERMINATE the same way. A read that times out
500
+ is an ordinary failure, and a call with no time left before the deadline is
501
+ not sent at all.
502
+
503
+ An INDETERMINATE step **takes no port**, so no error branch claims to have
504
+ handled it, and the run fails naming the state. It is not retried, and a rerun
505
+ does not replay it — only SUCCEEDED and REPLAYED steps replay. A rerun
506
+ **executes it again**, so a write that did land the first time is made a second
507
+ time; check its target before rerunning such a run. Workflow Studio renders it
508
+ in its own tone.
509
+
510
+ **Design-time tests carry it too.** `testStep` reports
511
+ `SUCCEEDED | FAILED | INDETERMINATE`, a trigger test whose hook made an
512
+ unconfirmed host call records INDETERMINATE, and such a test run is journaled
513
+ FAILED rather than green — the same answer a real run gives. An INDETERMINATE
514
+ last test is not a sample either: a draft step reading it is told to test that
515
+ block again, since it has no confirmed output to stand on. Collapsing the three
516
+ states to two is how a write-unconfirmed test came to read as a pass.
517
+
247
518
  ## Expressions
248
519
 
249
520
  Every string in a step's config is a template, nested strings in objects and
@@ -294,21 +565,65 @@ explanation behind it.
294
565
  | `PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES` | unset | Addresses or CIDRs a piece may reach, widening the default policy (`reactor/lib.ts`) |
295
566
  | `PH_WORKFLOWS_RUN_CONCURRENCY` | `4` | Runs executing at once; one forked node child each (`worker/pool.ts`) |
296
567
  | `PH_WORKFLOWS_RUN_QUEUE_DEPTH` | `0` | Runs that may wait for a slot before new ones are refused; `0` waits without limit |
568
+ | `PH_WORKFLOWS_MAX_QUEUED_FIRINGS` | `100` | Firings of ONE workflow that may wait for its concurrency slot; past it a firing is journaled CANCELLED (`reactor/run-gate.ts`) |
297
569
  | `PH_WORKFLOWS_POLL_INTERVAL_MS` | `60000` | Cadence for a polling trigger that names none of its own |
298
570
  | `PH_WORKFLOWS_WEBHOOK_RECONCILE_MS` | `900000` | How often a webhook trigger re-registers with its provider |
299
571
  | `PH_WORKFLOWS_WEBHOOK_TIMEOUT_MS` | `30000` | How long a sync-mode delivery holds the provider's socket |
300
572
  | `PH_WORKFLOWS_PIECE_MAX_FILE_BYTES` | `8388608` | File-size ceiling for FILE-property hydration and `ctx.files.write` |
301
- | `PH_WORKFLOWS_RUN_RETENTION_DAYS` | unset (off) | Deletes finished runs older than this many days (`reactor/run-retention.ts`) |
573
+ | `PH_WORKFLOWS_HOST_CALL_TIMEOUT_MS` | `10000` | Cap on one call a piece makes of its host; raised to the step's own timeout when that is longer, clipped to the step deadline (`activepieces/context/limits.ts`) |
574
+ | `PH_WORKFLOWS_ATTACHMENT_CACHE_BYTES` | `1073741824` | Downloaded attachments kept on disk by content (`engine/attachment-cache.ts`); `0` is off |
575
+ | `PH_WORKFLOWS_RUN_RETENTION_DAYS` | `30` | Deletes finished runs older than this many days; `0`/`off` keeps everything (`reactor/run-retention.ts`) |
576
+ | `PH_WORKFLOWS_SINGLETON_OWNER` | `<host>/<journal hash>` | Names this process as the workflow singleton's owner; the default is stable per slot, so a restart re-claims once the old heartbeat is stale (`reactor/singleton-lease.ts`) |
302
577
 
303
578
  Each numeric one parses as `Number(raw) || default`: a value that is not a
304
579
  positive number falls back silently rather than failing at boot.
305
580
 
306
- **Run retention is off by default**: the run journal keeps every run. With
307
- `PH_WORKFLOWS_RUN_RETENTION_DAYS` set, a sweep runs when the journal opens and
308
- hourly after, deleting runs that finished before the window together with their
309
- step executions and run documents, 500 runs per transaction. Unfinished runs
310
- are never pruned. The same sweep drops trigger dedupe keys older than the
311
- longest dedupe TTL (24h); a deleted workflow's keys go when it is deleted.
581
+ **Run retention is ON by default, at 30 days.** It used to be opt-in, which
582
+ meant unbounded growth on every host that did not know to set the variable —
583
+ measured at 743MB in three days. A journal is diagnostic, so a default window
584
+ is the honest setting and "keep everything" is a deliberate choice:
585
+ `PH_WORKFLOWS_RUN_RETENTION_DAYS=0` (or `off`, `never`, `false`, `none`) turns
586
+ it off. A value that is not a positive number falls back to the **default**
587
+ rather than to off — a typo must not remove the bound the variable exists to
588
+ set.
589
+
590
+ A sweep runs when the journal opens and hourly after, deleting runs that
591
+ finished before the window together with their step executions and run
592
+ documents, 500 runs per transaction. Unfinished runs are never pruned. The same
593
+ sweep drops trigger dedupe keys older than the longest dedupe TTL (24h); a
594
+ deleted workflow's keys go when it is deleted.
595
+
596
+ **Row width is capped too**: a step's input and output, and a run's trigger
597
+ payload, are each truncated past `STEP_PAYLOAD_MAX_BYTES` (256KB) to a marker
598
+ holding the original byte count and the head of the serialized JSON. The marker
599
+ is keyed on a **reserved** key whose value is a versioned sentinel, so a
600
+ payload cannot be mistaken for one — the predicate used to duck-type
601
+ `{truncated, bytes, prefix}`, which is exactly the shape of a truncation report
602
+ a piece might legitimately return, and being mistaken for a marker makes a
603
+ side-effectful step re-run. Rows written before the sentinel are still read, by
604
+ their exact key set.
605
+
606
+ A SUCCEEDED step whose output was truncated **replays** on rerun rather than
607
+ re-executing: it had side effects. Its output is explicitly unavailable, so a
608
+ later step that reads it fails the rerun by name (`UnavailableValueError`)
609
+ instead of being handed a marker. A step whose piece declares a reactor
610
+ **read** has no side effect to repeat, so it re-executes and produces its
611
+ output again.
612
+
613
+ The fact **survives further reruns**. The rerun's own REPLAYED row journals the
614
+ truncation marker again (the record's `journaledOutput`), so a second rerun of
615
+ the rerun still reads a marker rather than a NULL it would take for an ordinary
616
+ replay with no output. The record handed back to a caller keeps `output`
617
+ absent either way: a marker must never sit where real data goes.
618
+
619
+ The refusal is **deep, and on every route out of resolution**. The unavailable
620
+ wrapper carries its reason on a symbol, which anything that serializes it drops
621
+ — so a path that lands one level ABOVE the wrapper (`{{steps.charge}}`, or a
622
+ bare `{{steps}}`) would have handed the piece an ordinary-looking object whose
623
+ `output` became `{}` across the worker boundary, reason and all. Resolution
624
+ therefore checks the value it is about to return at any depth, and
625
+ `resolveStepInput` checks the whole resolved input again before it crosses to
626
+ the piece.
312
627
 
313
628
  **The secrets key is not optional in production.** Unset, `loadKey` generates
314
629
  `./.ph/secrets.key` — relative to the working directory, like the bundle cache
@@ -372,6 +687,32 @@ Token requests leave from the reactor process, so they are held to the egress
372
687
  policy pieces run under: `https` to a public address, or an address named in
373
688
  `PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES`, over `http` too.
374
689
 
690
+ ## Files and attachments
691
+
692
+ Bytes never cross the worker's IPC channel; host and worker share
693
+ `.ph/ap-attachment-staging/<uuid>`, one directory per step, removed when the
694
+ step returns. Directories a crashed host left behind are swept at startup.
695
+
696
+ - **In.** The host downloads every `attachment://` ref in a step's config into
697
+ the step's directory, counting bytes against
698
+ `PH_WORKFLOWS_PIECE_MAX_FILE_BYTES` as they arrive. Downloads land in a
699
+ content-addressed cache first (`.ph/ap-attachment-cache`), so a later step
700
+ or run reuses them; a step gets a copy (a clone where the filesystem has
701
+ them), never the cache file. A URL is fetched by the worker, and its body is
702
+ counted as it arrives too.
703
+ - **Out.** `ctx.files.write` takes a Buffer or a Readable and writes it into
704
+ the step's directory under the ceiling. After the step the host ingests each
705
+ file, hashing and uploading it from disk, and refuses any path the step did
706
+ not write: outside its directory, a symlink or a hard link.
707
+ - **Who may read.** A run reads the refs its own steps wrote, and the refs its
708
+ journaled steps already held on a rerun or a step test. Anything else is the
709
+ host's call (`canReadAttachmentRef`), made against the documents the run was
710
+ handed: the trigger's and those its steps read. Switchboard allows a ref
711
+ when the run's user may read one of those documents and that document
712
+ references it, as its download route does; a cached copy is served only
713
+ after the same check.
714
+ - **Time.** Staging and ingest count against the step's timeout.
715
+
375
716
  ## Known missing features
376
717
 
377
718
  This is what a piece can declare or call that this engine does not run. It is tracked in
@@ -468,6 +809,19 @@ releases what an earlier enable registered. The reason reads
468
809
  every reactor: the reactor is the project, as it is for `ctx.store`'s
469
810
  PROJECT scope.
470
811
 
812
+ **Files**
813
+
814
+ - Trigger hooks cannot read `attachment://` refs, and a trigger's
815
+ `ctx.files.write` returns an inline data URI rather than a ref.
816
+ - A webhook body that is not JSON or form data is decoded as UTF-8 text, so
817
+ binary and multipart deliveries are corrupted.
818
+ - A step cannot read back a file it wrote: its `apfile://` token only becomes
819
+ a ref once the step returns.
820
+ - A Buffer or `ApFile` returned in an output is serialized as a JSON number
821
+ array, not ingested as a file.
822
+ - A written file's content type is inferred from ten extensions; others are
823
+ `application/octet-stream`.
824
+
471
825
  **Piece**
472
826
 
473
827
  - `deprecated` is not in the descriptor.