@powerhousedao/reactor-workflow 6.2.3-dev.29 → 6.2.3-dev.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -60,6 +60,13 @@ rather than restated here.
60
60
  not parse reaches the piece as that text, as coercion already hands it on;
61
61
  and every declared prop is checked, so a required one absent from the input
62
62
  fails, where upstream's processor checks only the keys it was given.
63
+ A DYNAMIC prop's children are checked in the host, before the worker is
64
+ asked: the editor writes them as the step's `propertySettings[].schema`
65
+ in the SET_STEP_CONFIG (or SET_TRIGGER) of the edit they belong to, and
66
+ the run reads them from the published step. A value that is not an
67
+ object, or lacks a required child, fails the step (or parks the trigger)
68
+ naming each missing field. Nothing stores whether a step is complete: the
69
+ editor computes it where it shows it, and gates Publish on it.
63
70
  - **The SSRF table**, likewise from `./host`. `worker/egress.ts` classifies an
64
71
  address with `ssrfIpClassifier.isBlockedIp`; the connect-time socket and DNS
65
72
  hooks, the per-request policy and the allow-lists are ours. The one range the
@@ -94,6 +101,15 @@ next boot instead of vanishing; a fresh registration starts at head, so history
94
101
  is never replayed. `onOperations` journals a matched fire before it returns, so
95
102
  the cursor never passes an event that is not yet durable.
96
103
 
104
+ A workflow document's `DELETE_DOCUMENT` disarms it as disabling does: its
105
+ deliveries stop once the operation is indexed, a piece trigger's `onDisable`
106
+ runs, and then its trigger row, its FLOW `ctx.store` partition, its webhook
107
+ token and its dedupe keys are deleted. A restart finds nothing to re-arm.
108
+
109
+ `runsPage` pages newest first on when a run was journaled (`enqueued_at`),
110
+ which starting a PENDING run leaves alone, so a run keeps its place between
111
+ pages. `startedAt` is when it began executing.
112
+
97
113
  The webhook endpoints are registered under the `@powerhousedao/workflow`
98
114
  namespace, which this package exports as `WORKFLOW_PACKAGE_NAME`.
99
115
 
@@ -104,6 +120,122 @@ transport finds that child's code by walking up to this package's own
104
120
  `package.json` and reading `dist/worker-entry.js`, so `pnpm build` must have run
105
121
  before anything executes a piece — including the suites here.
106
122
 
123
+ ## Blocks
124
+
125
+ A step names its block the way Activepieces does, with three fields:
126
+ `pieceName`, `pieceVersion` and `actionName`. The trigger has `pieceName`,
127
+ `pieceVersion` and `triggerName`. `pieceVersion` is always an exact semver; the
128
+ workflow model refuses anything else. A block's identity is the piece and the
129
+ name (`blockKey` in `@powerhousedao/pieces-framework/block-type`), so the same
130
+ action at another version is the same block.
131
+
132
+ ### The core piece
133
+
134
+ The engine's own blocks are the built-in piece `@powerhousedao/piece-core`, in
135
+ `src/pieces/core/`. Its actions are `branch` and `assert`; its triggers are
136
+ `manual`, `schedule` and `webhook`. It is written with `createPiece`,
137
+ `createAction` and `createTrigger`, and its version is this package's version.
138
+
139
+ - The runtime registers it itself (`src/pieces/builtin.ts`); it is always
140
+ installed and never comes from a package list.
141
+ - It is host-bound, like `@powerhousedao/piece-reactor`: it always runs the
142
+ installed copy.
143
+ - It is this package's own code, so it is described and run in the reactor
144
+ process, not in the worker. Its descriptor comes from the same
145
+ `buildDescriptor` as any piece, which also reads its output ports and form
146
+ hints (`showWhen`, `emptyChoice`, the trigger's `display: "schedule"`).
147
+ - `branch` and `assert` run through `ActivepiecesBlockExecutor` like any piece
148
+ action, with the same resolution and journaling. `branch` leaves on `true`
149
+ or `false` by its result.
150
+ - The triggers are fed by the host: `reactor/schedule.ts`, `reactor/webhook.ts`
151
+ and the `fire` mutation. Their hooks are never called.
152
+
153
+ ## Block resolution
154
+
155
+ One policy (`reactor/block-resolver.ts`) resolves every block, for steps,
156
+ trigger arming, design-time descriptors, output trees, connection checks and
157
+ step tests. A block whose version is not an exact semver resolves to `missing`.
158
+
159
+ 1. **Candidates.** The installed package piece (`local`). For a name outside
160
+ `@activepieces/`, the configured registry's `GET /pieces/<name>/versions`
161
+ (`registry`); npm's packument is read only when there is no registry or it
162
+ answers 404 (`npm`), so a name the registry owns never comes from npm. For
163
+ `@activepieces/*`, the npm packument (`activepieces`: fetched from their
164
+ CDN, then npm). Listings are cached for five minutes; a source that does not
165
+ answer within two seconds contributes nothing.
166
+ 2. **Exact** wins, `local` first on a tie, then `registry`, `activepieces`,
167
+ `npm`.
168
+ 3. **Otherwise the closest** (`rankClosestVersions`): the highest of the same
169
+ major (the same minor for `0.x`) at or above the pin is `compatible`;
170
+ anything else is `fallback`.
171
+ A candidate without the block's action or trigger is skipped for the next
172
+ in the same order (`Skipped 0.1.0: it has no action "send_request"` in the
173
+ note). At most five candidates are described per resolution.
174
+ 4. **Host-bound pieces** (`@powerhousedao/piece-core`,
175
+ `@powerhousedao/piece-reactor`) always run the installed copy, with match
176
+ `installed`.
177
+ 5. **Missing** is the only failure: no source has the piece, or none of the
178
+ candidates described has an action or trigger of that name.
179
+
180
+ A version mismatch never blocks. The resolution is journaled on the step
181
+ (`piece_version`, `piece_source`, `version_match`, `version_note`) and on the
182
+ trigger state; a run's `warnings` counts its `fallback` steps and its edges on
183
+ ports their source never takes. `blockResolutions(workflowId)` reports the
184
+ resolution for every draft block before anything runs. The bundle is
185
+ fetched from the chosen source only, and cached under `<cache>/<source>/`.
186
+
187
+ ## Execution order
188
+
189
+ The coordinator (`pieces/engine/coordinator.ts`) runs one step at a time. It
190
+ passes over the `steps` array in order until a pass changes nothing:
191
+
192
+ - **Edges are decided by their source.** The trigger's edges are decided when
193
+ the run starts, on its `next` port. A step's edges are all decided once the
194
+ step runs or is skipped. An edge is taken when its port is the one the step
195
+ took and its condition, if any, holds.
196
+ - **A step waits for every inbound edge.** Once all are decided, the step runs
197
+ if any of them was taken, and is skipped otherwise. A join is an OR.
198
+ - **Siblings run in array order.** Two steps that become ready in the same pass
199
+ run in the order the `steps` array lists them, whatever their ports. A step
200
+ listed before the step that feeds it waits for the next pass.
201
+ - **Entries.** A step with no inbound edges runs only when the workflow has no
202
+ trigger. With a trigger, it never runs.
203
+ - **Never reached.** A step in a cycle, or fed by an edge from a step that does
204
+ not exist, is skipped when the run ends. So is every step not yet reached
205
+ when a step fails with no `error` edge taken.
206
+
207
+ Which port a step takes changes which steps run, never the order steps are
208
+ reached in. The studio's step outline (`stepOutline` in the workflow editor)
209
+ lists steps in this order.
210
+
211
+ ## Expressions
212
+
213
+ Every string in a step's config is a template, nested strings in objects and
214
+ arrays included. A field's `propertySettings[].mode` is editor-only: it picks
215
+ the control (the typed one, or the free expression box) and has no effect at
216
+ run time.
217
+
218
+ - **Literal braces.** `\{{` is a literal `{{`: `"Dear \{{name}}"` reaches the
219
+ piece as `Dear {{name}}`.
220
+ - **Paths.** `trigger.payload.x`, `steps.<key>.output.y`, `steps.<key>.error`
221
+ and `variables.<key>`. Brackets read keys that are not plain names, and array
222
+ indexes: `steps.fetch.output.headers["content.type"]`, `output.items[0]`.
223
+ - **Raw or text.** When the trimmed field is exactly one `{{…}}`, the field
224
+ takes the raw value (a number stays a number, an object an object). Any other
225
+ string is text, and each expression is interpolated: `null` as the empty
226
+ string, objects and arrays as JSON.
227
+ - **Unresolved references fail the step.** A path that names nothing fails with
228
+ `Unresolved reference {{steps.x.output.y}}`. The optional form
229
+ `{{steps.x.output.y?}}` resolves to `null` instead. A path whose value is
230
+ `null` is not missing.
231
+ - **Fallbacks.** `a || b || 'default'` takes the first term whose value is not
232
+ `null` or `""`. A missing path falls through to the next term; the last term
233
+ fails when missing, unless it is a literal or ends in `?`. Literals are
234
+ single- or double-quoted, with `\` escapes.
235
+
236
+ Edge conditions are templates too. A condition with an unresolved reference
237
+ fails the run.
238
+
107
239
  ## `./testing`
108
240
 
109
241
  `@powerhousedao/reactor-workflow/testing` re-exports the piece layer: the
@@ -130,11 +262,18 @@ explanation behind it.
130
262
  | `PH_WORKFLOWS_WEBHOOK_RECONCILE_MS` | `900000` | How often a webhook trigger re-registers with its provider |
131
263
  | `PH_WORKFLOWS_WEBHOOK_TIMEOUT_MS` | `30000` | How long a sync-mode delivery holds the provider's socket |
132
264
  | `PH_WORKFLOWS_PIECE_MAX_FILE_BYTES` | `8388608` | File-size ceiling for FILE-property hydration and `ctx.files.write` |
265
+ | `PH_WORKFLOWS_RUN_RETENTION_DAYS` | unset (off) | Deletes finished runs older than this many days (`reactor/run-retention.ts`) |
133
266
 
134
267
  Each numeric one parses as `Number(raw) || default`: a value that is not a
135
268
  positive number falls back silently rather than failing at boot.
136
269
 
137
- One of them carries a caveat worth knowing before a deployment depends on it.
270
+ **Run retention is off by default**: the run journal keeps every run. With
271
+ `PH_WORKFLOWS_RUN_RETENTION_DAYS` set, a sweep runs when the journal opens and
272
+ hourly after, deleting runs that finished before the window together with their
273
+ step executions and run documents, 500 runs per transaction. Unfinished runs
274
+ are never pruned. The same sweep drops trigger dedupe keys older than the
275
+ longest dedupe TTL (24h); a deleted workflow's keys go when it is deleted.
276
+
138
277
  **The secrets key is not optional in production.** Unset, `loadKey` generates
139
278
  `./.ph/secrets.key` — relative to the working directory, like the bundle cache
140
279
  and the attachment staging dir. A host whose working directory does not survive
@@ -180,6 +319,23 @@ only. There is no second setting: a package installed from that registry
180
319
  already ships pieces that run in the worker, so a bundle fetched from it is no
181
320
  more trusted than one that arrived inside a package.
182
321
 
322
+ ## OAuth2 connections
323
+
324
+ An OAUTH2 connection brings its own app. `startOAuth` (`reactor/oauth.ts`)
325
+ builds the provider's authorize URL from the piece's `PieceAuth.OAuth2`
326
+ (`{prop}` placeholders filled from the connection's config, PKCE when the
327
+ piece asks for it) and records a pending attempt in the `oauth` namespace,
328
+ good for one exchange within 10 minutes. The host serves the redirect —
329
+ Switchboard at `<workflow package base>/oauth/callback` — and passes what the
330
+ provider sent to `completeOAuth`, which exchanges the code, stores the token
331
+ set as a managed secret named `token` on the connection and runs the
332
+ connection check. Resolution refreshes the token 15 minutes before it
333
+ expires and rotates that secret in place.
334
+
335
+ Token requests leave from the reactor process, so they are held to the egress
336
+ policy pieces run under: `https` to a public address, or an address named in
337
+ `PH_WORKFLOWS_EGRESS_ALLOW_ADDRESSES`, over `http` too.
338
+
183
339
  ## Known missing features
184
340
 
185
341
  This is what a piece can declare or call that this engine does not run. It is tracked in
@@ -200,13 +356,22 @@ releases what an earlier enable registered. The reason reads
200
356
 
201
357
  **Triggers**
202
358
 
203
- - `TriggerStrategy.APP_WEBHOOK` and `context.app.createListeners`: the
204
- listeners are never read, so deliveries are refused (#3081).
205
- - `renewConfiguration` / `onRenew`: rejected when the strategy is not
206
- `NONE`, as `renewConfiguration` (#3090).
359
+ - `TriggerStrategy.APP_WEBHOOK` and `context.app.createListeners`: rejected,
360
+ as `TriggerStrategy APP_WEBHOOK` (#3081). There is no app-level endpoint or
361
+ listener table to route a delivery by. A strategy the engine does not know,
362
+ or none at all, is rejected the same way. One helper decides the strategy
363
+ (`triggerDelivery` in `@powerhousedao/pieces-framework/workflow`), and a
364
+ trigger whose descriptor cannot be read is `ERROR` with a retry, never polled.
365
+ - `renewConfiguration` / `onRenew`: a `WEBHOOK` trigger's `CRON` strategy
366
+ runs `onRenew` on that cron, in UTC. The next renewal time is stored on the
367
+ trigger row, so it survives a restart. A failed `onRenew` sets
368
+ `renew_error` (`renewError` in `triggerStates`), apart from the poll's
369
+ `last_error`, and retries with backoff capped at the next cron slot. The
370
+ trigger stays `ENABLED`. Any other strategy but `NONE`, or a cron that does not parse, is
371
+ rejected as `renewConfiguration` (#3090).
207
372
  - `TriggerStrategy.MANUAL` on a piece trigger: rejected, as
208
- `TriggerStrategy.MANUAL` (#3091). The engine's own `core#manual` is not a
209
- piece trigger and is unaffected.
373
+ `TriggerStrategy.MANUAL` (#3091). The core piece's `manual` trigger is fed by
374
+ the host's `fire` mutation and is unaffected.
210
375
  - Every `WEBHOOK` trigger's `run()` is called every 15 minutes without a
211
376
  `payload`, as a reconciliation sweep. A `run` that only maps the delivery
212
377
  either fails or fires a spurious run (#3090).
@@ -225,8 +390,17 @@ releases what an earlier enable registered. The reason reads
225
390
  - `auth` as an array runs through the method whose type matches the
226
391
  connection's. The piece is refused only when none of its methods can run,
227
392
  with the first method's reason.
228
- - OAuth2 and OIDC: rejected, as `OAuth2 auth` and `OIDC auth` (#3091). Their
229
- connections are refused at check and run too.
393
+ - OAuth2 runs with the connection's own app only: the authorization-code
394
+ grant, with the `client_id` in the connection's config and the
395
+ `client_secret` in its secret refs. The `client_credentials` grant is
396
+ rejected, as `OAuth2 client credentials` (#3091). There are no
397
+ operator-configured or Powerhouse-hosted apps, and no Activepieces
398
+ `CLOUD_OAUTH2` / `PLATFORM_OAUTH2` connections.
399
+ - A token is refreshed at most once at a time per process. Replicas are not
400
+ coordinated, so two can refresh one token at once; a provider that rotates
401
+ refresh tokens may then revoke one of them.
402
+ - OIDC: rejected, as `OIDC auth` (#3091). Its connections are refused at
403
+ check and run too.
230
404
  - `server` in `validate` and `getConnectionIdentifier` is a throwing stub.
231
405
  - A CUSTOM_AUTH value's props reach the piece as stored, not coerced: a
232
406
  `Property.Number` prop arrives as the string it was entered as.