@powerhousedao/reactor-workflow 6.2.3-dev.49 → 6.2.3-dev.50
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +274 -8
- package/dist/index.d.ts +263 -32
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1293 -107
- package/dist/index.js.map +1 -1
- package/dist/{piece-registry-CpiVMWNv.js → piece-registry-BKRLK7Kv.js} +1192 -90
- package/dist/piece-registry-BKRLK7Kv.js.map +1 -0
- package/dist/{piece-registry-wQBhKJUl.d.ts → piece-registry-Ds8WkewF.d.ts} +232 -99
- package/dist/piece-registry-Ds8WkewF.d.ts.map +1 -0
- package/dist/testing.d.ts +1 -1
- package/dist/testing.js +91 -3
- package/dist/testing.js.map +1 -0
- package/dist/worker-entry.js +5685 -285
- package/dist/worker-entry.js.map +1 -1
- package/package.json +16 -8
- package/dist/loader-Gkx6I_Bo.js +0 -91
- package/dist/loader-Gkx6I_Bo.js.map +0 -1
- package/dist/piece-registry-CpiVMWNv.js.map +0 -1
- package/dist/piece-registry-wQBhKJUl.d.ts.map +0 -1
- package/dist/reactor-rpc-BqAjCK1p.js +0 -1134
- package/dist/reactor-rpc-BqAjCK1p.js.map +0 -1
- package/dist/worker-entry.d.ts +0 -2
package/README.md
CHANGED
|
@@ -52,6 +52,10 @@ rather than restated here.
|
|
|
52
52
|
`apfile://` refs, the size ceiling and a host-injected fetcher have no
|
|
53
53
|
upstream equivalent. An `ApFile` a processor builds is flattened to a plain
|
|
54
54
|
object at that boundary: a class instance does not survive the worker IPC.
|
|
55
|
+
Its `base64` is computed on first read, as upstream's getter is, so a piece
|
|
56
|
+
that only reads `data` holds one copy of the file. A `streaming: true` FILE
|
|
57
|
+
prop gets upstream's `ApStreamingFile` instead: `{ filename, extension,
|
|
58
|
+
size, body }`, with `body` read from the staged file or the network.
|
|
55
59
|
- **Prop validation**, from the same place. Before an action's `run()` or any
|
|
56
60
|
trigger hook but `onDisable`, a prop left unset takes its `defaultValue` and
|
|
57
61
|
the coerced values go through the engine's `validateProperty`. A failure is
|
|
@@ -102,6 +106,41 @@ next boot instead of vanishing; a fresh registration starts at head, so history
|
|
|
102
106
|
is never replayed. `onOperations` journals a matched fire before it returns, so
|
|
103
107
|
the cursor never passes an event that is not yet durable.
|
|
104
108
|
|
|
109
|
+
**A fire that crashes the reactor is bounded.** The durable cursor is what
|
|
110
|
+
makes an operation written while the runtime was down catch up — and it is also
|
|
111
|
+
what re-delivers, on every boot, an operation whose fire takes the process down
|
|
112
|
+
before anything is journaled (the EPIPE boot loop). So the dedupe row counts
|
|
113
|
+
**deliveries**, committed before the risky work, which is the only way a crash
|
|
114
|
+
that leaves nothing behind can be counted at all:
|
|
115
|
+
|
|
116
|
+
- A delivery whose claim already holds a run id is an ordinary duplicate and is
|
|
117
|
+
suppressed, as before. So is one whose key fired with no run linked — run
|
|
118
|
+
without a journal row, its run erased, or written before claims existed —
|
|
119
|
+
which holds the `FIRED_WITHOUT_RUN_ID` marker rather than NULL.
|
|
120
|
+
- A delivery whose claim holds **no** run id is retried: the previous attempt
|
|
121
|
+
died before it journaled anything, and losing a legitimate trigger to a
|
|
122
|
+
transient store failure would be worse than the loop.
|
|
123
|
+
- Past `FIRE_CRASH_BUDGET` (3) such deliveries the fire is **abandoned**, with
|
|
124
|
+
a FAILED run naming the loop — visible, and rerunnable once the cause is
|
|
125
|
+
fixed, instead of a reactor that crashes on every boot and says nothing. The
|
|
126
|
+
FAILED run is linked to the key, so later deliveries are duplicates.
|
|
127
|
+
|
|
128
|
+
The **count** and the **claim** are deliberately different writes. Counting is
|
|
129
|
+
its own committed transaction, because a delivery that leaves nothing behind
|
|
130
|
+
has to be countable. Claiming is `run_id`, set under a `WHERE run_id IS NULL`
|
|
131
|
+
guard in the SAME transaction as the run row: the guard takes the row's lock,
|
|
132
|
+
so of two concurrent deliveries of one operation the second blocks until the
|
|
133
|
+
first commits and then matches no row — exactly one delivery fires, and a crash
|
|
134
|
+
in between rolls the claim back with the run it failed to journal. Were the
|
|
135
|
+
bump to ride along inside the claim transaction, a run insert that takes the
|
|
136
|
+
process down would roll the count back too and the budget could never reach its
|
|
137
|
+
limit.
|
|
138
|
+
|
|
139
|
+
Log writes on the piece-log and run-failure paths are truncated before the
|
|
140
|
+
write (`MAX_LOG_LINE_CHARS`): a piece error carrying an HTML error page is a
|
|
141
|
+
multi-megabyte write to a pipe that may be blocked, and bounding the write is
|
|
142
|
+
cheaper than handling the throw.
|
|
143
|
+
|
|
105
144
|
A workflow document's `DELETE_DOCUMENT` disarms it as disabling does: its
|
|
106
145
|
deliveries stop once the operation is indexed, a piece trigger's `onDisable`
|
|
107
146
|
runs, and then its trigger row, its FLOW `ctx.store` partition, its webhook
|
|
@@ -117,7 +156,8 @@ runs `run_document` ties to it, the runs whose trigger payload names it as
|
|
|
117
156
|
workflow's own runs (`run.workflow_id`, test runs included), and every rerun
|
|
118
157
|
of those, transitively. Their
|
|
119
158
|
`step_execution` and `run_document` rows go with them; a `trigger_dedupe` row
|
|
120
|
-
they claimed keeps its key
|
|
159
|
+
they claimed keeps its key, with `run_id` set to the `FIRED_WITHOUT_RUN_ID`
|
|
160
|
+
marker so a redelivery is still a duplicate. A run erased while it is
|
|
121
161
|
still executing journals nothing more from this process. The read model's
|
|
122
162
|
fence is `"skip"`: the journal lives on the relational handle, not the
|
|
123
163
|
reactor's, so a run the purge races can still be written after the marker was
|
|
@@ -244,6 +284,150 @@ Which port a step takes changes which steps run, never the order steps are
|
|
|
244
284
|
reached in. The studio's step outline (`stepOutline` in the workflow editor)
|
|
245
285
|
lists steps in this order.
|
|
246
286
|
|
|
287
|
+
## The workflow policy
|
|
288
|
+
|
|
289
|
+
A workflow document carries a `policy` block, and every field in it was schema
|
|
290
|
+
and editor only until W3.3: nothing read `concurrency`, `runTimeoutSeconds`,
|
|
291
|
+
`defaultRetry` or `onFailure`, so an author who set them got no behaviour and
|
|
292
|
+
no warning. They are enforced now, and the fields that are **not** are marked
|
|
293
|
+
`NOT YET ENFORCED` in the document model's own SDL rather than left to look
|
|
294
|
+
live. `reactor/policy.ts` resolves the block. A definition with **no** policy
|
|
295
|
+
at all enforces nothing, but only hand-built definitions lack one: `policy` is
|
|
296
|
+
non-null in the workflow model since v1, so every workflow document, old or
|
|
297
|
+
new, carries one.
|
|
298
|
+
|
|
299
|
+
| Field | Where | Behaviour |
|
|
300
|
+
| -------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------- |
|
|
301
|
+
| `concurrency` | `reactor/run-gate.ts` | SINGLETON drops a firing while a run is active; QUEUE serialises; PARALLEL runs concurrently |
|
|
302
|
+
| `maxParallelRuns` | `reactor/run-gate.ts` | Bounds PARALLEL; null is unbounded. SINGLETON and QUEUE are 1 by definition |
|
|
303
|
+
| `runTimeoutSeconds` | `pieces/engine/coordinator.ts` | A run deadline from FIRING time, checked between steps and bounding every retry wait; expiry ends the run CANCELLED |
|
|
304
|
+
| `defaultRetry`, step `retry` | `pieces/engine/retry.ts` | Attempts, backoff, delays and `retryOn`; attempts land on the step's journal row |
|
|
305
|
+
| `onFailure` | `reactor/service.ts` | On a trigger's failed run: PARK parks the trigger; NOTIFY logs at error level; IGNORE does nothing. A failed manual run or rerun changes nothing |
|
|
306
|
+
| `maxSuspensionDays` | — | **Not enforced**: nothing suspends. Waitpoints, `run.pause` and `generateResumeUrl` all throw |
|
|
307
|
+
| `retainRunsDays` | — | **Not enforced** per workflow; `PH_WORKFLOWS_RUN_RETENTION_DAYS` is the journal-wide control |
|
|
308
|
+
| `journalAsDocument` | — | **Not enforced**: the journal is relational, and there is no run document model |
|
|
309
|
+
| step `idempotencyKeyExpression` | — | **Not enforced**: a fire dedupes on its trigger operation or a trigger item's `_dedupe_key` |
|
|
310
|
+
|
|
311
|
+
**Enforcing the policy changes every deployed workflow.** The model's initial
|
|
312
|
+
values are `concurrency: QUEUE`, `onFailure: PARK` and
|
|
313
|
+
`runTimeoutSeconds: 3600`, and documents written before enforcement carry them
|
|
314
|
+
too. Unless an author changed them, a workflow now:
|
|
315
|
+
|
|
316
|
+
1. **Serialises its runs** (QUEUE): a firing waits while a run of the same
|
|
317
|
+
workflow executes.
|
|
318
|
+
2. **Parks its trigger on the first terminal failure of a run it fired**
|
|
319
|
+
(PARK; a failed manual run or rerun does not count): the trigger
|
|
320
|
+
stops firing until the workflow is re-published or re-enabled.
|
|
321
|
+
3. **Has a one-hour run deadline counted from firing** (3600s): queue wait
|
|
322
|
+
included; past it the run ends CANCELLED, before its next step.
|
|
323
|
+
4. **Cancels firings past 100 waiters** (`PH_WORKFLOWS_MAX_QUEUED_FIRINGS`):
|
|
324
|
+
an overflowing firing is journaled CANCELLED instead of run.
|
|
325
|
+
|
|
326
|
+
That is what the fields have said since the first schema; what changed is that
|
|
327
|
+
they are true.
|
|
328
|
+
|
|
329
|
+
- **Concurrency is process-local**, which is exactly right: workflow execution
|
|
330
|
+
is a singleton pinned to one reactor, so this
|
|
331
|
+
process is the deployment's whole run set. A firing SINGLETON drops is
|
|
332
|
+
journaled as a CANCELLED run rather than discarded — a firing that vanished
|
|
333
|
+
is indistinguishable from a trigger that never fired.
|
|
334
|
+
- **The QUEUE is bounded**, at `PH_WORKFLOWS_MAX_QUEUED_FIRINGS` waiting
|
|
335
|
+
firings per workflow (100). QUEUE means latency, not failure — but an
|
|
336
|
+
unbounded queue means neither: a document-event trigger on a busy type
|
|
337
|
+
enqueues faster than the workflow runs, every waiter holds its payload and
|
|
338
|
+
its promise, and the lane grows until the process dies. A firing that
|
|
339
|
+
overflows the depth is journaled CANCELLED exactly as a SINGLETON refusal is.
|
|
340
|
+
- **A sync webhook answers a refused firing without a 500.** A firing refused
|
|
341
|
+
as parked, by SINGLETON, or as stale after its wait gets 409; one refused by
|
|
342
|
+
a full queue or that waited past its deadline gets 429. A provider retries a
|
|
343
|
+
500, and each retry would journal another CANCELLED run; 500 stays for real
|
|
344
|
+
failures.
|
|
345
|
+
- **A firing that waited reads the workflow again before it runs.** If it is
|
|
346
|
+
no longer ENABLED, was re-published, or was parked while the firing waited,
|
|
347
|
+
the firing is journaled CANCELLED without running a step.
|
|
348
|
+
- **The run deadline starts at FIRING time, not at admission.** Queue time is
|
|
349
|
+
part of the time the run took: a firing that waits past its
|
|
350
|
+
`runTimeoutSeconds` for a slot is CANCELLED without executing a single step,
|
|
351
|
+
rather than running its side effect long after the timeout that was supposed
|
|
352
|
+
to bound it. The document read counts too.
|
|
353
|
+
- **PARKED holds for every trigger kind, and a restart does not clear it.** The
|
|
354
|
+
park is a runtime override of the document's enabled-ness: parking writes a
|
|
355
|
+
`workflow_park` row naming the published version that failed, never the
|
|
356
|
+
document, so the document still says ENABLED and re-arming from it — which is
|
|
357
|
+
what a reboot does for every workflow it finds — would un-park the broken
|
|
358
|
+
workflow and resume firing it. A schedule or piece trigger's `trigger_state`
|
|
359
|
+
row also turns PARKED, so the supervisor stops polling it; a document-event,
|
|
360
|
+
document-lifecycle or webhook trigger is left unregistered; and any firing
|
|
361
|
+
that still arrives is journaled CANCELLED. Only two things clear a park: a
|
|
362
|
+
**re-publish** (the published version moves past the one that failed, trigger
|
|
363
|
+
changed or not), and a **disable then re-enable**. Both work across a restart.
|
|
364
|
+
A park blocks the registrations of the version that failed and older ones
|
|
365
|
+
only: a newer version always arms, and lifts the park it outlived. Every park
|
|
366
|
+
write runs on the trigger supervisor's lane, with enable and disable, so a
|
|
367
|
+
park cannot land between an enable's check and its write. An unresolvable
|
|
368
|
+
piece leaves a PARKED row alone as well, rather than turning it ERROR and
|
|
369
|
+
letting the ERROR row's own retry arm it.
|
|
370
|
+
- **`retryOn` empty means every error is retryable.** The schema reads "error
|
|
371
|
+
classes that are retryable; everything else fails terminally on attempt 1",
|
|
372
|
+
but the shipped default is an empty list, and taking that literally would
|
|
373
|
+
make every `maxAttempts` a lie. A non-empty entry matches the error's class
|
|
374
|
+
name exactly or appears anywhere in its message, case-insensitively, so both
|
|
375
|
+
`["HostCallTimeoutError"]` and `["429"]` work.
|
|
376
|
+
- `maxAttempts` is clamped to 10 and one backoff wait to 5 minutes: each
|
|
377
|
+
attempt re-runs a side effect and holds the run's worker slot.
|
|
378
|
+
- **Only the attempt is retried, never the resolution.** A step's input is
|
|
379
|
+
resolved once, before the first attempt, because resolution reads the scope
|
|
380
|
+
and nothing an attempt changes: an `UnresolvedReferenceError` names a key
|
|
381
|
+
that will not exist on attempt five either. A resolution failure fails the
|
|
382
|
+
step immediately, with no backoff waits spent on it.
|
|
383
|
+
- **A retry wait is clipped to the run deadline, and the deadline wins.** A
|
|
384
|
+
backoff longer than the time left is served out only as far as the deadline,
|
|
385
|
+
and then the step gets no further attempt: running one would be a side effect
|
|
386
|
+
after the run was already over. The attempt that failed is still journaled,
|
|
387
|
+
no error port is taken — nothing downstream may run after the run has ended —
|
|
388
|
+
and the run reads CANCELLED, since the clock stopped it rather than the
|
|
389
|
+
workflow failing.
|
|
390
|
+
- **A run its deadline cancelled can be rerun**, like a FAILED one: it is
|
|
391
|
+
journaled with the error name `RunDeadlineExceeded`, and the rerun replays
|
|
392
|
+
the steps that finished. A firing refused before it ran (parked, SINGLETON,
|
|
393
|
+
stale, queue full, expired in the queue) is CANCELLED without that name and
|
|
394
|
+
is not rerunnable; fire the workflow again instead.
|
|
395
|
+
- A step that DECLARES a `retry` block overrides `defaultRetry`, whatever the
|
|
396
|
+
block resolves to — `{maxAttempts: 1}` is an author saying "not this one".
|
|
397
|
+
- An **INDETERMINATE** step is never retried and never replayed: a retry would
|
|
398
|
+
be a second write. A rerun does execute it again, so it can write twice. See
|
|
399
|
+
**Indeterminate steps** below.
|
|
400
|
+
|
|
401
|
+
## Indeterminate steps
|
|
402
|
+
|
|
403
|
+
A piece's call of its host is capped (`PH_WORKFLOWS_HOST_CALL_TIMEOUT_MS`, 10s,
|
|
404
|
+
raised to the step's own `timeoutSeconds` when that is longer; a trigger hook
|
|
405
|
+
or a design-time call takes the same cap, raised to its own timeout), and always
|
|
406
|
+
clipped to end a margin before the step's kill deadline, so the call's own
|
|
407
|
+
timeout is what the step reports. A **writing** call that times out —
|
|
408
|
+
`store.put`, `store.delete` — may well have been committed, so the step records
|
|
409
|
+
`INDETERMINATE` rather than FAILED: reporting a failure for a write that landed
|
|
410
|
+
is a claim nobody can stand behind, and it was happening (the 10s cap against a
|
|
411
|
+
dispatch under load). A reactor write still unfinished at the step deadline
|
|
412
|
+
(`ReactorJobPendingError`) is INDETERMINATE the same way. A read that times out
|
|
413
|
+
is an ordinary failure, and a call with no time left before the deadline is
|
|
414
|
+
not sent at all.
|
|
415
|
+
|
|
416
|
+
An INDETERMINATE step **takes no port**, so no error branch claims to have
|
|
417
|
+
handled it, and the run fails naming the state. It is not retried, and a rerun
|
|
418
|
+
does not replay it — only SUCCEEDED and REPLAYED steps replay. A rerun
|
|
419
|
+
**executes it again**, so a write that did land the first time is made a second
|
|
420
|
+
time; check its target before rerunning such a run. Workflow Studio renders it
|
|
421
|
+
in its own tone.
|
|
422
|
+
|
|
423
|
+
**Design-time tests carry it too.** `testStep` reports
|
|
424
|
+
`SUCCEEDED | FAILED | INDETERMINATE`, a trigger test whose hook made an
|
|
425
|
+
unconfirmed host call records INDETERMINATE, and such a test run is journaled
|
|
426
|
+
FAILED rather than green — the same answer a real run gives. An INDETERMINATE
|
|
427
|
+
last test is not a sample either: a draft step reading it is told to test that
|
|
428
|
+
block again, since it has no confirmed output to stand on. Collapsing the three
|
|
429
|
+
states to two is how a write-unconfirmed test came to read as a pass.
|
|
430
|
+
|
|
247
431
|
## Expressions
|
|
248
432
|
|
|
249
433
|
Every string in a step's config is a template, nested strings in objects and
|
|
@@ -294,21 +478,64 @@ explanation behind it.
|
|
|
294
478
|
| `PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES` | unset | Addresses or CIDRs a piece may reach, widening the default policy (`reactor/lib.ts`) |
|
|
295
479
|
| `PH_WORKFLOWS_RUN_CONCURRENCY` | `4` | Runs executing at once; one forked node child each (`worker/pool.ts`) |
|
|
296
480
|
| `PH_WORKFLOWS_RUN_QUEUE_DEPTH` | `0` | Runs that may wait for a slot before new ones are refused; `0` waits without limit |
|
|
481
|
+
| `PH_WORKFLOWS_MAX_QUEUED_FIRINGS` | `100` | Firings of ONE workflow that may wait for its concurrency slot; past it a firing is journaled CANCELLED (`reactor/run-gate.ts`) |
|
|
297
482
|
| `PH_WORKFLOWS_POLL_INTERVAL_MS` | `60000` | Cadence for a polling trigger that names none of its own |
|
|
298
483
|
| `PH_WORKFLOWS_WEBHOOK_RECONCILE_MS` | `900000` | How often a webhook trigger re-registers with its provider |
|
|
299
484
|
| `PH_WORKFLOWS_WEBHOOK_TIMEOUT_MS` | `30000` | How long a sync-mode delivery holds the provider's socket |
|
|
300
485
|
| `PH_WORKFLOWS_PIECE_MAX_FILE_BYTES` | `8388608` | File-size ceiling for FILE-property hydration and `ctx.files.write` |
|
|
301
|
-
| `
|
|
486
|
+
| `PH_WORKFLOWS_HOST_CALL_TIMEOUT_MS` | `10000` | Cap on one call a piece makes of its host; raised to the step's own timeout when that is longer, clipped to the step deadline (`activepieces/context/limits.ts`) |
|
|
487
|
+
| `PH_WORKFLOWS_ATTACHMENT_CACHE_BYTES` | `1073741824` | Downloaded attachments kept on disk by content (`engine/attachment-cache.ts`); `0` is off |
|
|
488
|
+
| `PH_WORKFLOWS_RUN_RETENTION_DAYS` | `30` | Deletes finished runs older than this many days; `0`/`off` keeps everything (`reactor/run-retention.ts`) |
|
|
302
489
|
|
|
303
490
|
Each numeric one parses as `Number(raw) || default`: a value that is not a
|
|
304
491
|
positive number falls back silently rather than failing at boot.
|
|
305
492
|
|
|
306
|
-
**Run retention is
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
493
|
+
**Run retention is ON by default, at 30 days.** It used to be opt-in, which
|
|
494
|
+
meant unbounded growth on every host that did not know to set the variable —
|
|
495
|
+
measured at 743MB in three days. A journal is diagnostic, so a default window
|
|
496
|
+
is the honest setting and "keep everything" is a deliberate choice:
|
|
497
|
+
`PH_WORKFLOWS_RUN_RETENTION_DAYS=0` (or `off`, `never`, `false`, `none`) turns
|
|
498
|
+
it off. A value that is not a positive number falls back to the **default**
|
|
499
|
+
rather than to off — a typo must not remove the bound the variable exists to
|
|
500
|
+
set.
|
|
501
|
+
|
|
502
|
+
A sweep runs when the journal opens and hourly after, deleting runs that
|
|
503
|
+
finished before the window together with their step executions and run
|
|
504
|
+
documents, 500 runs per transaction. Unfinished runs are never pruned. The same
|
|
505
|
+
sweep drops trigger dedupe keys older than the longest dedupe TTL (24h); a
|
|
506
|
+
deleted workflow's keys go when it is deleted.
|
|
507
|
+
|
|
508
|
+
**Row width is capped too**: a step's input and output, and a run's trigger
|
|
509
|
+
payload, are each truncated past `STEP_PAYLOAD_MAX_BYTES` (256KB) to a marker
|
|
510
|
+
holding the original byte count and the head of the serialized JSON. The marker
|
|
511
|
+
is keyed on a **reserved** key whose value is a versioned sentinel, so a
|
|
512
|
+
payload cannot be mistaken for one — the predicate used to duck-type
|
|
513
|
+
`{truncated, bytes, prefix}`, which is exactly the shape of a truncation report
|
|
514
|
+
a piece might legitimately return, and being mistaken for a marker makes a
|
|
515
|
+
side-effectful step re-run. Rows written before the sentinel are still read, by
|
|
516
|
+
their exact key set.
|
|
517
|
+
|
|
518
|
+
A SUCCEEDED step whose output was truncated **replays** on rerun rather than
|
|
519
|
+
re-executing: it had side effects. Its output is explicitly unavailable, so a
|
|
520
|
+
later step that reads it fails the rerun by name (`UnavailableValueError`)
|
|
521
|
+
instead of being handed a marker. A step whose piece declares a reactor
|
|
522
|
+
**read** has no side effect to repeat, so it re-executes and produces its
|
|
523
|
+
output again.
|
|
524
|
+
|
|
525
|
+
The fact **survives further reruns**. The rerun's own REPLAYED row journals the
|
|
526
|
+
truncation marker again (the record's `journaledOutput`), so a second rerun of
|
|
527
|
+
the rerun still reads a marker rather than a NULL it would take for an ordinary
|
|
528
|
+
replay with no output. The record handed back to a caller keeps `output`
|
|
529
|
+
absent either way: a marker must never sit where real data goes.
|
|
530
|
+
|
|
531
|
+
The refusal is **deep, and on every route out of resolution**. The unavailable
|
|
532
|
+
wrapper carries its reason on a symbol, which anything that serializes it drops
|
|
533
|
+
— so a path that lands one level ABOVE the wrapper (`{{steps.charge}}`, or a
|
|
534
|
+
bare `{{steps}}`) would have handed the piece an ordinary-looking object whose
|
|
535
|
+
`output` became `{}` across the worker boundary, reason and all. Resolution
|
|
536
|
+
therefore checks the value it is about to return at any depth, and
|
|
537
|
+
`resolveStepInput` checks the whole resolved input again before it crosses to
|
|
538
|
+
the piece.
|
|
312
539
|
|
|
313
540
|
**The secrets key is not optional in production.** Unset, `loadKey` generates
|
|
314
541
|
`./.ph/secrets.key` — relative to the working directory, like the bundle cache
|
|
@@ -372,6 +599,32 @@ Token requests leave from the reactor process, so they are held to the egress
|
|
|
372
599
|
policy pieces run under: `https` to a public address, or an address named in
|
|
373
600
|
`PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES`, over `http` too.
|
|
374
601
|
|
|
602
|
+
## Files and attachments
|
|
603
|
+
|
|
604
|
+
Bytes never cross the worker's IPC channel; host and worker share
|
|
605
|
+
`.ph/ap-attachment-staging/<uuid>`, one directory per step, removed when the
|
|
606
|
+
step returns. Directories a crashed host left behind are swept at startup.
|
|
607
|
+
|
|
608
|
+
- **In.** The host downloads every `attachment://` ref in a step's config into
|
|
609
|
+
the step's directory, counting bytes against
|
|
610
|
+
`PH_WORKFLOWS_PIECE_MAX_FILE_BYTES` as they arrive. Downloads land in a
|
|
611
|
+
content-addressed cache first (`.ph/ap-attachment-cache`), so a later step
|
|
612
|
+
or run reuses them; a step gets a copy (a clone where the filesystem has
|
|
613
|
+
them), never the cache file. A URL is fetched by the worker, and its body is
|
|
614
|
+
counted as it arrives too.
|
|
615
|
+
- **Out.** `ctx.files.write` takes a Buffer or a Readable and writes it into
|
|
616
|
+
the step's directory under the ceiling. After the step the host ingests each
|
|
617
|
+
file, hashing and uploading it from disk, and refuses any path the step did
|
|
618
|
+
not write: outside its directory, a symlink or a hard link.
|
|
619
|
+
- **Who may read.** A run reads the refs its own steps wrote, and the refs its
|
|
620
|
+
journaled steps already held on a rerun or a step test. Anything else is the
|
|
621
|
+
host's call (`canReadAttachmentRef`), made against the documents the run was
|
|
622
|
+
handed: the trigger's and those its steps read. Switchboard allows a ref
|
|
623
|
+
when the run's user may read one of those documents and that document
|
|
624
|
+
references it, as its download route does; a cached copy is served only
|
|
625
|
+
after the same check.
|
|
626
|
+
- **Time.** Staging and ingest count against the step's timeout.
|
|
627
|
+
|
|
375
628
|
## Known missing features
|
|
376
629
|
|
|
377
630
|
This is what a piece can declare or call that this engine does not run. It is tracked in
|
|
@@ -468,6 +721,19 @@ releases what an earlier enable registered. The reason reads
|
|
|
468
721
|
every reactor: the reactor is the project, as it is for `ctx.store`'s
|
|
469
722
|
PROJECT scope.
|
|
470
723
|
|
|
724
|
+
**Files**
|
|
725
|
+
|
|
726
|
+
- Trigger hooks cannot read `attachment://` refs, and a trigger's
|
|
727
|
+
`ctx.files.write` returns an inline data URI rather than a ref.
|
|
728
|
+
- A webhook body that is not JSON or form data is decoded as UTF-8 text, so
|
|
729
|
+
binary and multipart deliveries are corrupted.
|
|
730
|
+
- A step cannot read back a file it wrote: its `apfile://` token only becomes
|
|
731
|
+
a ref once the step returns.
|
|
732
|
+
- A Buffer or `ApFile` returned in an output is serialized as a JSON number
|
|
733
|
+
array, not ingested as a file.
|
|
734
|
+
- A written file's content type is inferred from ten extensions; others are
|
|
735
|
+
`application/octet-stream`.
|
|
736
|
+
|
|
471
737
|
**Piece**
|
|
472
738
|
|
|
473
739
|
- `deprecated` is not in the descriptor.
|