pure-effect 0.13.0 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +64 -0
- package/README.md +296 -124
- package/index.d.ts +1118 -134
- package/index.js +1029 -462
- package/package.json +3 -1
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
- Inject context without touching function signatures
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
11
|
- OpenTelemetry-ready via lifecycle hooks
|
|
12
|
-
- Zero dependencies, about
|
|
12
|
+
- Zero dependencies, about 7 KB minified and gzipped
|
|
13
13
|
- Works in JavaScript and TypeScript (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
@@ -27,6 +27,7 @@
|
|
|
27
27
|
- [Which Errors Are Data](#which-errors-are-data)
|
|
28
28
|
- [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context)
|
|
29
29
|
- [Why Pure Effect](#why-pure-effect)
|
|
30
|
+
- [Load Tests](#load-tests)
|
|
30
31
|
- [API Reference](#api-reference)
|
|
31
32
|
- [Limitations](#limitations)
|
|
32
33
|
|
|
@@ -38,7 +39,7 @@ npm install pure-effect
|
|
|
38
39
|
|
|
39
40
|
## Quick Start
|
|
40
41
|
|
|
41
|
-
A
|
|
42
|
+
A user registration flow. Each step receives the value the previous step produced. An I/O step is a pair: the call to make, and a `next` function that receives the answer and decides what happens next.
|
|
42
43
|
|
|
43
44
|
```js
|
|
44
45
|
import { Success, Failure, Command, effectPipe, runEffect } from 'pure-effect';
|
|
@@ -50,8 +51,8 @@ const validateRegistration = (input) => {
|
|
|
50
51
|
return Success(input);
|
|
51
52
|
};
|
|
52
53
|
|
|
53
|
-
//
|
|
54
|
-
// Name the commands
|
|
54
|
+
// The next two functions return a Command object. They do not call the database.
|
|
55
|
+
// Name the commands: traces, replay matching, and telemetry spans use the names.
|
|
55
56
|
const ensureEmailAvailable = (input) => {
|
|
56
57
|
const cmdFindUser = () => db.findUser(input.email);
|
|
57
58
|
const next = (found) => (found ? Failure('Email already in use.') : Success(input));
|
|
@@ -81,7 +82,7 @@ async function registerUser(input) {
|
|
|
81
82
|
|
|
82
83
|
## Testing Without Mocks
|
|
83
84
|
|
|
84
|
-
|
|
85
|
+
Pipelines return plain objects, so you can check what the code will do without running it.
|
|
85
86
|
|
|
86
87
|
Validation runs as soon as you build the flow, so testing it needs nothing else:
|
|
87
88
|
|
|
@@ -113,33 +114,34 @@ assert.equal(step2.cmd.name, 'cmdSaveUser');
|
|
|
113
114
|
// The database was never touched.
|
|
114
115
|
```
|
|
115
116
|
|
|
116
|
-
Write both kinds
|
|
117
|
+
Write both kinds, because they catch different bugs. Only step tests see the value passed from one step to the next.
|
|
117
118
|
|
|
118
|
-
|
|
119
|
+
For example, if the email guard returned `Success(true)` instead of `Success(input)`, the step test would fail: it expects the input back and gets `true`. The flow test would still pass, because `cmdFindUser` is still the first call and `cmdSaveUser` the second, while the flow saves `true` to the database instead of the user.
|
|
119
120
|
|
|
120
|
-
|
|
121
|
+
A step tested on its own returns a `Failure` without `initialInput`. That is why the assertion above is `Failure('Email already in use.')`, while the validation one is `Failure('Invalid email.', badInput)`: `effectPipe` adds the input when it builds a flow.
|
|
121
122
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
Nothing can see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them.
|
|
123
|
+
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them.
|
|
125
124
|
|
|
126
125
|
## How It Works
|
|
127
126
|
|
|
128
|
-
A flow is a chain of pairs: an I/O call
|
|
127
|
+
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop.
|
|
129
128
|
|
|
130
|
-
Recording and replay
|
|
129
|
+
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
131
130
|
|
|
132
|
-
Building a flow runs the pure steps
|
|
131
|
+
Building a flow runs the pure steps right away: `registerUserFlow(badInput)` returns the validation `Failure` synchronously, so the tests above do not need `runEffect`. Only Commands need it.
|
|
133
132
|
|
|
134
133
|
## Time-Travel Debugging
|
|
135
134
|
|
|
136
|
-
Record what each Command returned, then feed those results back into the
|
|
135
|
+
Record what each Command returned, then feed those results back into the flow to retrace the path a request took, with no database or network.
|
|
137
136
|
|
|
138
137
|
```js
|
|
139
138
|
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
140
139
|
|
|
141
|
-
// Record a real run.
|
|
142
|
-
const { result, trace } = await recordEffect(checkoutFlow, input, {
|
|
140
|
+
// Record a real run. The context's flowName names the flow in the trace.
|
|
141
|
+
const { result, trace } = await recordEffect(checkoutFlow, input, {
|
|
142
|
+
version: process.env.BUILD_ID,
|
|
143
|
+
context: { flowName: 'checkout' }
|
|
144
|
+
});
|
|
143
145
|
|
|
144
146
|
// Later, somewhere else, with nothing connected:
|
|
145
147
|
await timeTravel(checkoutFlow, trace);
|
|
@@ -147,14 +149,31 @@ await timeTravel(checkoutFlow, trace);
|
|
|
147
149
|
|
|
148
150
|
```
|
|
149
151
|
Replaying 'checkout' (3 recorded steps)
|
|
150
|
-
Initial input: {
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
152
|
+
Initial input: {
|
|
153
|
+
"cartId": "cart_abc123",
|
|
154
|
+
"promoCode": "FREE_YEAR_VIP"
|
|
155
|
+
}
|
|
156
|
+
Step 1: cmdFetchCart returned in 41.277ms {
|
|
157
|
+
"totalAmount": "120.00"
|
|
158
|
+
}
|
|
159
|
+
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
160
|
+
"isValid": true,
|
|
161
|
+
"discountValue": 100
|
|
162
|
+
}
|
|
163
|
+
Step 3: cmdChargeCard threw in 181.507ms {
|
|
164
|
+
"code": "invalid_amount",
|
|
165
|
+
"name": "Error",
|
|
166
|
+
"message": "Amount must be non-zero."
|
|
167
|
+
}
|
|
154
168
|
Replay finished with state: Failure
|
|
169
|
+
Error: {
|
|
170
|
+
"code": "invalid_amount",
|
|
171
|
+
"name": "Error",
|
|
172
|
+
"message": "Amount must be non-zero."
|
|
173
|
+
}
|
|
155
174
|
```
|
|
156
175
|
|
|
157
|
-
A production incident becomes a
|
|
176
|
+
A production incident becomes a regression test, with no mocks or fixtures:
|
|
158
177
|
|
|
159
178
|
```js
|
|
160
179
|
it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
@@ -164,9 +183,9 @@ it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
|
164
183
|
});
|
|
165
184
|
```
|
|
166
185
|
|
|
167
|
-
The test
|
|
186
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay raises a `TimeParadox` naming where it diverged. If the error handling changes, the assertion fails.
|
|
168
187
|
|
|
169
|
-
|
|
188
|
+
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) raises no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
170
189
|
|
|
171
190
|
```js
|
|
172
191
|
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
@@ -179,23 +198,28 @@ it('incident 8f3a fixed: a 100% promo checks out without a charge', async () =>
|
|
|
179
198
|
});
|
|
180
199
|
```
|
|
181
200
|
|
|
182
|
-
For every other recording
|
|
201
|
+
For every other recording, `unreached` should be empty, so a fix that skips a step by accident fails the test.
|
|
202
|
+
|
|
203
|
+
This works when the removed step is the last one the flow reaches. Remove a step from the middle and every later step moves to a different path, so the replay stops at a `TimeParadox`. In that case `unreached` lists the steps the replay never got to, not the steps the fix made unnecessary.
|
|
204
|
+
|
|
205
|
+
**Replay checks the path, not the values.** It shows that the flow asks for the same Commands in the same order and handles the same answers. It cannot check what the flow computes, because every Command's result comes from the recording, not from the new code. If you fix how a refund amount is calculated and replay the bad run, the charge step gets the old amount from the recording: the replay reports `Success`, returns the value from before the fix, and flags nothing. Test a change to a value against the pure step that computes it.
|
|
183
206
|
|
|
184
|
-
**A trace records what each Command returned, not
|
|
207
|
+
**A trace records what each Command returned, not the arguments it was called with.** Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds the arguments that Command ran with. You can inspect them in a debugger, but you cannot assert on them. Storing arguments would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
185
208
|
|
|
186
|
-
**
|
|
209
|
+
**Put anything that varies between runs inside a Command.** For replay, the current time or a random ID counts as I/O. Wrapped in a Command, it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay and drifts from the trace.
|
|
187
210
|
|
|
188
|
-
**
|
|
211
|
+
**To check for this,** record a flow, send the trace through JSON as your storage would, replay it, and compare the outcomes. A step that computes a new value on each run shows up as a `TimeParadox` if it changes which Commands run, or as a different final value. Going through JSON also catches a result that storage cannot keep, such as a `Date`, a class instance, or `undefined` in an array, which is where a replay from memory and a replay from storage part ways:
|
|
189
212
|
|
|
190
213
|
```js
|
|
191
214
|
const { result, trace } = await recordEffect(registerUserFlow, input);
|
|
192
|
-
const
|
|
193
|
-
|
|
215
|
+
const stored = JSON.parse(JSON.stringify(trace)); // what your storage gives back
|
|
216
|
+
const { result: replayed } = await replayEffect(registerUserFlow(input), stored);
|
|
217
|
+
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive JSON
|
|
194
218
|
```
|
|
195
219
|
|
|
196
|
-
|
|
220
|
+
This also works for a `Failure`: an error read back from a trace keeps its message, name, cause, custom properties, and an `AggregateError`'s list of errors, and is deep-equal to the one the Command threw, including one whose `name` or `cause` was set after it was created. The exception is a custom error class, which comes back as a plain `Error` with that class's name; compare `error.name` and `error.message` instead.
|
|
197
221
|
|
|
198
|
-
**
|
|
222
|
+
**You choose the trace format.** `replayEffect` also accepts a resolver function instead of a trace, so you can replay from OpenTelemetry spans, a log pipeline, or a database table, not only from the JSON that `recorder` produces.
|
|
199
223
|
|
|
200
224
|
```js
|
|
201
225
|
// A resolver answers one question: what did production get back for this step?
|
|
@@ -205,7 +229,7 @@ await replayEffect(checkoutFlow(input), resolve);
|
|
|
205
229
|
|
|
206
230
|
## Recording in Production
|
|
207
231
|
|
|
208
|
-
`recordEffect`
|
|
232
|
+
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup (see `examples/recording-example.js`).
|
|
209
233
|
|
|
210
234
|
```js
|
|
211
235
|
import { configureEffect } from 'pure-effect';
|
|
@@ -223,11 +247,17 @@ configureEffect(
|
|
|
223
247
|
);
|
|
224
248
|
```
|
|
225
249
|
|
|
226
|
-
|
|
250
|
+
By default, successful runs are held in memory and then discarded. A run that rejects because your own code threw, such as a `TypeError` after an API changed the shape of its response, is offered to `keep` as a `Failure` carrying the thrown error, so by default its trace is kept, and the run still rejects. Replaying that trace reproduces the throw. If `keep` or the sink throws, for example because `JSON.stringify` meets a value it cannot serialize, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`. `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. `maxEntries` caps the length of a trace and reports the overflow as `dropped`. The sink can write a trace to S3 or a database column as JSON, as long as your Commands return plain data.
|
|
251
|
+
|
|
252
|
+
The example reads the input from the flow itself, where `effectPipe` stores it, so build the outermost flow with `effectPipe`. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one. The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
253
|
+
|
|
254
|
+
**A trace keeps data, not objects.** A result is copied when it is recorded, and copied again each time a replay hands it to the flow, so neither a later step nor a replay can change what the trace says. The copy keeps values but not classes: a money object or a database entity comes back as a plain object without its methods, and a `Buffer` as a plain byte array. An error returned inside a result keeps its message but loses properties such as `code`. Writing the trace as JSON loses more: a `Date` becomes a string, a `Map` becomes `{}`, an error inside a result becomes `{}`, `undefined` in an array becomes `null`, and a `BigInt` makes the sink throw.
|
|
255
|
+
|
|
256
|
+
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`, and rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
227
257
|
|
|
228
258
|
## Passing Runtime Context
|
|
229
259
|
|
|
230
|
-
Some values come from the framework
|
|
260
|
+
Some values come from the framework (an authenticated tenant, a request ID, environment config) rather than from the data being processed. `Ask` lets a step read the `context` object passed to `runEffect` without passing it through every function:
|
|
231
261
|
|
|
232
262
|
```js
|
|
233
263
|
import { Success, Failure, Command, Ask, effectPipe, runEffect } from 'pure-effect';
|
|
@@ -240,7 +270,9 @@ const findProduct = (productId) =>
|
|
|
240
270
|
|
|
241
271
|
app.post('/checkout', async (req, res) => {
|
|
242
272
|
const result = await runEffect(checkoutFlow(req.body.productId), { tenant: req.tenant });
|
|
243
|
-
res.json(result);
|
|
273
|
+
if (result.type === 'Success') return res.json(result.value);
|
|
274
|
+
// A Failure also carries the flow's input, so send the client only an error meant for it.
|
|
275
|
+
res.status(400).json({ error: typeof result.error === 'string' ? result.error : 'Checkout failed.' });
|
|
244
276
|
});
|
|
245
277
|
```
|
|
246
278
|
|
|
@@ -248,19 +280,14 @@ Recording stores the context alongside the trace, so `Ask` replays with the valu
|
|
|
248
280
|
|
|
249
281
|
## Retrying Transient Failures
|
|
250
282
|
|
|
251
|
-
`Retry` runs part of a flow again when
|
|
252
|
-
|
|
253
|
-
**Wrap the Command that fails, not the pipeline.** Every attempt re-runs the whole wrapped tree, including Commands that already succeeded:
|
|
283
|
+
`Retry` runs part of a flow again when its I/O fails, meaning a Command's function throws. It does not retry a `Failure` that a step returned: that means the flow decided to stop, so the `Failure` is passed on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency. An option set to `undefined` keeps its default, so settings read from configuration can leave a key out. To reuse the same settings, share an object:
|
|
254
284
|
|
|
255
285
|
```js
|
|
256
|
-
|
|
257
|
-
Retry(
|
|
258
|
-
|
|
259
|
-
// Correct: only the step that fails transiently is retried.
|
|
260
|
-
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), { attempts: 3 }))(order);
|
|
286
|
+
const flakyNetwork = { attempts: 3, delay: 200, backoff: 2 };
|
|
287
|
+
Retry(fetchPrice(sku), flakyNetwork);
|
|
261
288
|
```
|
|
262
289
|
|
|
263
|
-
|
|
290
|
+
The retry settings are a plain object, so you can inspect and assert on them without running anything:
|
|
264
291
|
|
|
265
292
|
```js
|
|
266
293
|
import { Success, Failure, Command, Retry, runEffect } from 'pure-effect';
|
|
@@ -279,15 +306,49 @@ assert.equal(weatherFn.type, 'Retry');
|
|
|
279
306
|
assert.equal(weatherFn.options.attempts, 3);
|
|
280
307
|
```
|
|
281
308
|
|
|
282
|
-
|
|
309
|
+
**Wrap the Command that fails, not the pipeline.** Each attempt re-runs everything inside the `Retry`, including Commands that already succeeded:
|
|
310
|
+
|
|
311
|
+
```js
|
|
312
|
+
// Dangerous: a flaky receipt step charges the customer again on every attempt.
|
|
313
|
+
Retry(effectPipe(chargeCard, sendReceipt)(order), { attempts: 3 });
|
|
314
|
+
|
|
315
|
+
// Correct: only the step that fails transiently is retried.
|
|
316
|
+
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), { attempts: 3 }))(order);
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
Wrapping a pipeline is safe only when every Command in it is idempotent (safe to run more than once).
|
|
320
|
+
|
|
321
|
+
**Wrapping one Command is not always enough.** A Command's `next` is inside the `Retry` too, so if `next` continues the flow, everything after it is retried as well. This looks like it follows the advice above, but it does not:
|
|
322
|
+
|
|
323
|
+
```js
|
|
324
|
+
// Dangerous: `next` continues the booking, so every later step is inside the Retry.
|
|
325
|
+
Retry(
|
|
326
|
+
Command(cmdHoldSeat, (seat) => chargeCard(trip, seat)),
|
|
327
|
+
{ attempts: 2 }
|
|
328
|
+
);
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
If a later Command's function throws, the `Retry` runs again, which repeats the seat hold and the charge. With two of these nested, a failure in the last step of a booking charges the card nine times.
|
|
332
|
+
|
|
333
|
+
Instead, leave the retried Command with its default `next` and continue in a later pipeline step, outside the `Retry`:
|
|
334
|
+
|
|
335
|
+
```js
|
|
336
|
+
// Safe: the tree the Retry repeats is that one Command.
|
|
337
|
+
effectPipe(
|
|
338
|
+
() => Retry(Command(cmdHoldSeat), { attempts: 2 }),
|
|
339
|
+
(seat) => chargeCard(trip, seat)
|
|
340
|
+
);
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
When every attempt fails, `runEffect` returns a `Failure` with this error:
|
|
283
344
|
|
|
284
345
|
```text
|
|
285
346
|
{ retryExhausted: true, lastError: <the last error>, attempts: 3 }
|
|
286
347
|
```
|
|
287
348
|
|
|
288
|
-
In TypeScript, a `Retry`
|
|
349
|
+
In TypeScript, a `Retry` adds `RetryExhaustedError` to the pipeline's error union, next to the error type of the steps it wraps, since a `Failure` those steps return passes through unchanged. `lastError` is typed `unknown`: it is whatever the Command's function threw, and nothing in the types says what that is, so check it before using it, for example with `instanceof Error`.
|
|
289
350
|
|
|
290
|
-
**
|
|
351
|
+
**Use `onExhausted` to fall back when every attempt fails.** The fallback runs instead of returning the `Failure`. If it succeeds, its value continues down the pipeline. If it fails, its own `Failure` is returned as is:
|
|
291
352
|
|
|
292
353
|
```js
|
|
293
354
|
const fetchPrice = (sku) =>
|
|
@@ -297,13 +358,13 @@ const fetchPrice = (sku) =>
|
|
|
297
358
|
});
|
|
298
359
|
```
|
|
299
360
|
|
|
300
|
-
Fallback steps are recorded
|
|
361
|
+
Fallback steps are recorded, so a replay repeats the fallback too. A fallback never starts in a `Parallel` branch that has already been cancelled. With `onExhausted` set, the `Retry` adds the fallback's error type to the TypeScript error union instead of `RetryExhaustedError`.
|
|
301
362
|
|
|
302
|
-
Every attempt is
|
|
363
|
+
Every attempt is recorded, so a replay repeats the same sequence of failures, without the delays.
|
|
303
364
|
|
|
304
365
|
## Running Effects in Parallel
|
|
305
366
|
|
|
306
|
-
`Parallel` runs several flows at the same time and passes their results to `next` as an
|
|
367
|
+
`Parallel` runs several flows at the same time and passes their results to `next` as an array, in order. If a branch fails, the other branches are cancelled, `next` is not called, and that branch's `Failure` is returned.
|
|
307
368
|
|
|
308
369
|
```js
|
|
309
370
|
import { Success, Command, Parallel } from 'pure-effect';
|
|
@@ -312,11 +373,43 @@ const loadProfile = (userId) =>
|
|
|
312
373
|
Parallel([getUser(userId), getPermissions(userId)], ([user, permissions]) => Success({ user, permissions }));
|
|
313
374
|
```
|
|
314
375
|
|
|
315
|
-
`next` is optional,
|
|
376
|
+
`next` is optional, as with `Command`. Without it, `Parallel(effects)` returns the array of values.
|
|
377
|
+
|
|
378
|
+
Every branch can read the `Ask` context.
|
|
379
|
+
|
|
380
|
+
**Options for batches.** The second argument can be either `next` or an options object:
|
|
381
|
+
|
|
382
|
+
```js
|
|
383
|
+
// At most 5 branches in flight, for a gateway that rate limits.
|
|
384
|
+
Parallel(subscriptions.map(billOne), { limit: 5 });
|
|
385
|
+
|
|
386
|
+
// Every branch runs to completion, and `next` receives the outcomes rather than the values.
|
|
387
|
+
Parallel(subscriptions.map(billOne), { limit: 5, settled: true });
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end and `next` receives one `Success` or `Failure` per branch, in order, so a failed record is something you can count instead of the end of the job. In TypeScript, write `settled: true` where you call `Parallel`, or mark a shared options object `as const`: since `settled` decides what `next` receives, a value the compiler only knows as `true` or `false` is refused.
|
|
316
391
|
|
|
317
|
-
|
|
392
|
+
```js
|
|
393
|
+
Parallel(subscriptions.map(billOne), (outcomes) => Success(outcomes.map(summarize)), { limit: 5, settled: true });
|
|
394
|
+
```
|
|
318
395
|
|
|
319
|
-
|
|
396
|
+
A `Failure` in the list carries its branch's input when that branch was built with `effectPipe`, and no input otherwise, so match outcomes to records by their position in the list. When you report the outcomes, pick the fields you need rather than logging the whole object:
|
|
397
|
+
|
|
398
|
+
```js
|
|
399
|
+
Parallel(work, (outcomes) => Success(outcomes.map((o) => (o.type === 'Success' ? o.value : o.error.message))), {
|
|
400
|
+
settled: true
|
|
401
|
+
});
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
Logging the outcomes as they are would include each branch's input, which for a registration or login contains credentials.
|
|
405
|
+
|
|
406
|
+
The outcomes are plain `Success` and `Failure` objects. Returning one of them from `next` is the same as returning any other `Failure`: the flow stops, and `Retry` does not run it again.
|
|
407
|
+
|
|
408
|
+
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
409
|
+
|
|
410
|
+
`limit` caps how many branches run at once; the rest start as others finish. Results and recorded paths stay in array order, so a limit changes only the pacing, and a trace recorded with a limit replays the same without one. A `limit` that is not a positive integer throws a `TypeError`.
|
|
411
|
+
|
|
412
|
+
**Cancelling is a request, not a guarantee.** A cancelled branch starts no new Commands: a three-step branch whose first step is running when a sibling fails finishes that step and stops. To stop the running step itself, its function has to accept the `AbortSignal` it is given and pass it on to the I/O:
|
|
320
413
|
|
|
321
414
|
```js
|
|
322
415
|
// Cancellable: the request is aborted the moment a sibling branch fails.
|
|
@@ -326,13 +419,15 @@ const fetchProfile = (userId) => Command((signal) => fetch(`/users/${userId}`, {
|
|
|
326
419
|
const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${userId}`).then((r) => r.json()));
|
|
327
420
|
```
|
|
328
421
|
|
|
329
|
-
Outside a `Parallel
|
|
422
|
+
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
423
|
+
|
|
424
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay raises a `TimeParadox` naming it.
|
|
330
425
|
|
|
331
426
|
## Composing Larger Flows
|
|
332
427
|
|
|
333
|
-
`effectPipe`
|
|
428
|
+
`effectPipe` runs steps in a straight line. Branching and joining use the same pieces:
|
|
334
429
|
|
|
335
|
-
**A step can return a sub-pipeline.** `effectPipe(...)(value)` returns an Effect like any other, so a step can branch into
|
|
430
|
+
**A step can return a sub-pipeline.** `effectPipe(...)(value)` returns an Effect like any other, so a step can branch into another flow. A `Failure` in that flow stops the outer pipeline too:
|
|
336
431
|
|
|
337
432
|
```js
|
|
338
433
|
const processOrder = (order) => (order.isGift ? giftFlow(order) : standardFlow(order));
|
|
@@ -340,7 +435,7 @@ const processOrder = (order) => (order.isGift ? giftFlow(order) : standardFlow(o
|
|
|
340
435
|
const fulfillment = effectPipe(validateOrder, processOrder, scheduleShipping);
|
|
341
436
|
```
|
|
342
437
|
|
|
343
|
-
**
|
|
438
|
+
**Use `Parallel` to join values that do not depend on each other.** When a later step needs several such values, fetch them at the same time. Without `next`, the results arrive as an array, in order:
|
|
344
439
|
|
|
345
440
|
```js
|
|
346
441
|
const loadCheckout = effectPipe(
|
|
@@ -349,7 +444,7 @@ const loadCheckout = effectPipe(
|
|
|
349
444
|
);
|
|
350
445
|
```
|
|
351
446
|
|
|
352
|
-
**
|
|
447
|
+
**Join values that depend on each other locally.** When step B needs step A's result and step C needs both, pass both forward in one value. A small pipeline inside the step does this without nested callbacks:
|
|
353
448
|
|
|
354
449
|
```js
|
|
355
450
|
const applyLoyaltyDiscount = (orderId) =>
|
|
@@ -360,13 +455,13 @@ const applyLoyaltyDiscount = (orderId) =>
|
|
|
360
455
|
)(orderId);
|
|
361
456
|
```
|
|
362
457
|
|
|
363
|
-
**
|
|
458
|
+
**Pass only what the next steps need.** Avoid one object that collects everything computed so far. When a step's input is all it gets, it cannot depend on a value from far upstream without showing it, stale fields do not linger, and two steps cannot clash over the same key. Data also lives only as long as the flow needs it, which matters for credentials and personal data. In TypeScript, it keeps the pipeline type-checked: if an upstream step changes what it returns, the step that reads the value fails to compile instead of getting `undefined` at runtime.
|
|
364
459
|
|
|
365
460
|
## Which Errors Are Data
|
|
366
461
|
|
|
367
|
-
|
|
462
|
+
Pure Effect has no catch, on purpose. A flow's outcomes are of two kinds, and each is written differently.
|
|
368
463
|
|
|
369
|
-
**An outcome the flow handles is data.** A Command's `next` receives the result and can
|
|
464
|
+
**An outcome the flow handles is data.** A Command's `next` receives the result and can return any Effect, including a fallback pipeline. When the I/O can reject, catch the error inside the Command's function and return it as a value:
|
|
370
465
|
|
|
371
466
|
```js
|
|
372
467
|
const fetchCachedPrice = (sku) => {
|
|
@@ -379,16 +474,51 @@ const fetchPrice = (sku) => {
|
|
|
379
474
|
pricing
|
|
380
475
|
.get(sku)
|
|
381
476
|
.then((price) => ({ ok: true, price }))
|
|
382
|
-
.catch((error) => ({ ok: false, error }));
|
|
477
|
+
.catch((error) => ({ ok: false, code: error.code, message: error.message }));
|
|
383
478
|
return Command(cmdFetchLivePrice, (r) => (r.ok ? Success({ ...r.price, stale: false }) : fetchCachedPrice(sku)));
|
|
384
479
|
};
|
|
385
480
|
```
|
|
386
481
|
|
|
387
|
-
The failed
|
|
482
|
+
The failed attempt is recorded as that Command's result, with the error's `code` and `message`, so a replay takes the same fallback branch. Copy the fields you need rather than returning the error itself: a trace keeps data, not objects, so an error stored inside a result loses its properties (see [Recording in Production](#recording-in-production)).
|
|
483
|
+
|
|
484
|
+
**A `Failure` means abort.** It stops the flow and goes back to the code that called `runEffect`, which decides what it means: an HTTP status, a queue retry, an alert. Nothing inside the flow can catch a `Failure`, so a reader never has to look elsewhere for a handler.
|
|
485
|
+
|
|
486
|
+
Rule of thumb: if you would handle it, return it as a value; if you would only report it, return a `Failure`. The one exception is retry exhaustion, which happens inside `Retry`; handle it with the `onExhausted` option in [Retrying Transient Failures](#retrying-transient-failures).
|
|
487
|
+
|
|
488
|
+
**Catch or retry, not both.** `Retry` only reacts to a function that throws. If you catch a 503 inside the function and return it as a value, then wrap the step in `Retry`, it makes one call and never retries. Catch the error when the flow should handle it as data; let it throw when `Retry` should try again.
|
|
489
|
+
|
|
490
|
+
```js
|
|
491
|
+
// Handled as data: one call, and `next` decides what the miss means.
|
|
492
|
+
Command(
|
|
493
|
+
function cmdFetchPrice() {
|
|
494
|
+
return pricing
|
|
495
|
+
.get(sku)
|
|
496
|
+
.then((price) => ({ ok: true, price }))
|
|
497
|
+
.catch((error) => ({ ok: false, code: error.code, message: error.message }));
|
|
498
|
+
},
|
|
499
|
+
(r) => (r.ok ? Success(r.price) : fetchCachedPrice(sku))
|
|
500
|
+
);
|
|
501
|
+
|
|
502
|
+
// Retried: the function lets the rejection out, so the interpreter sees an I/O fault.
|
|
503
|
+
Retry(
|
|
504
|
+
Command(function cmdFetchPrice() {
|
|
505
|
+
return pricing.get(sku);
|
|
506
|
+
}),
|
|
507
|
+
{ attempts: 3 }
|
|
508
|
+
);
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
`runEffect` tells these cases apart:
|
|
388
512
|
|
|
389
|
-
|
|
513
|
+
| what happened | what it is | what `Retry` does | what `onExhausted` sees |
|
|
514
|
+
| --- | --- | --- | --- |
|
|
515
|
+
| a step returned `Failure(...)`, or an `onBeforeCommand` hook threw | an abort | nothing; it propagates at once, unwrapped | nothing |
|
|
516
|
+
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
517
|
+
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
518
|
+
| an `onStep` hook threw after the Command's function succeeded | a bug | nothing; it is thrown, not returned | nothing |
|
|
519
|
+
| an `onStep` hook returned `undefined` after the Command's function returned a value | a bug | nothing; it is thrown, not returned | nothing |
|
|
390
520
|
|
|
391
|
-
|
|
521
|
+
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is treated as a bug: `runEffect` rejects with the thrown error instead of returning a `Failure`.
|
|
392
522
|
|
|
393
523
|
## TypeScript: Typed Errors and Context
|
|
394
524
|
|
|
@@ -409,7 +539,9 @@ if (result.type === 'Failure') {
|
|
|
409
539
|
}
|
|
410
540
|
```
|
|
411
541
|
|
|
412
|
-
Value types
|
|
542
|
+
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
543
|
+
|
|
544
|
+
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
413
545
|
|
|
414
546
|
### Typed context with `Ask`
|
|
415
547
|
|
|
@@ -424,15 +556,36 @@ const findProduct = (productId: string): Effect<Product, 'not_found', AppContext
|
|
|
424
556
|
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
425
557
|
```
|
|
426
558
|
|
|
559
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object.
|
|
560
|
+
|
|
427
561
|
## Why Pure Effect
|
|
428
562
|
|
|
429
|
-
**vs. Temporal and durable execution (Restate, Inngest):** The closest relatives,
|
|
563
|
+
**vs. Temporal and durable execution (Restate, Inngest):** The closest relatives, built on the same idea: record what every step returned and replay it. Those engines store the history on a server and resume workflows automatically after a crash. Pure Effect leaves storage and restarts to your application, and there is no infrastructure to run.
|
|
564
|
+
|
|
565
|
+
**vs. Effect-TS (and fp-ts):** A full ecosystem with fibers, streaming, schema validation, structured concurrency, and more, with a steep learning curve and its own vocabulary. Pure Effect takes only the idea of describing side effects as data, and does less: testable pipelines, context injection, retry, parallel execution, and replayable traces. If you need fibers or streaming, use Effect-TS.
|
|
430
566
|
|
|
431
|
-
**vs.
|
|
567
|
+
**vs. plain async/await with mocks:** A mock can pass every test while behaving differently from the real driver. Here, business logic never does I/O, so there is nothing to mock.
|
|
432
568
|
|
|
433
|
-
**
|
|
569
|
+
**When to use something else:** If your code does little async I/O, or testing and debugging production are not problems for you, plain async/await is simpler.
|
|
434
570
|
|
|
435
|
-
|
|
571
|
+
## Load Tests
|
|
572
|
+
|
|
573
|
+
Timings are from Node 22 on a MacBook Pro with M4 Pro CPU:
|
|
574
|
+
|
|
575
|
+
**A web service under load.** A `node:http` service ran signup and checkout flows, where each checkout was a `Parallel` of a retried stock reservation and a card charge, with the OpenTelemetry and recording examples installed.
|
|
576
|
+
|
|
577
|
+
- 800 requests at once, across 4 tenants: every response and every recorded trace belonged to its own request, and no tenant, user, or request ID crossed between runs.
|
|
578
|
+
- The ~3,500 telemetry spans formed 800 request traces, with every span inside its own request's trace.
|
|
579
|
+
- Side effects matched the outcomes: 320 charges and 320 orders, and 80 declined cards stopped 30 stock reservations that were still running.
|
|
580
|
+
- 30,000 runs in a row kept memory between 10 and 11 MB, and left nothing running afterwards.
|
|
581
|
+
- A background worker that ran 20,000 jobs in one flow, looping through a Command's `next`, finished in under ~400 ms without holding on to memory.
|
|
582
|
+
- Stopping the server with flows still running, or clients hanging up mid-request: every flow that had started finished and was recorded, with no unhandled rejections.
|
|
583
|
+
|
|
584
|
+
**Batches and parallel work.**
|
|
585
|
+
|
|
586
|
+
- A settled `Parallel` over 200 records with `limit: 7` never had more than 7 records in flight, and returned the outcomes in order.
|
|
587
|
+
- 5,000 branches in one `Parallel` ran in ~25 ms.
|
|
588
|
+
- Running a Command through `runEffect` costs about 0.2 microseconds, against 0.06 for a plain `await`.
|
|
436
589
|
|
|
437
590
|
## API Reference
|
|
438
591
|
|
|
@@ -450,11 +603,11 @@ Returns `{ type: 'Failure', error, initialInput }`. Stops the pipeline immediate
|
|
|
450
603
|
|
|
451
604
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
452
605
|
|
|
453
|
-
- `cmd`: A function (sync or async) that performs the side effect. Inside a `Parallel` branch it is called with an `AbortSignal` that fires when a sibling branch fails; elsewhere it is called with no arguments.
|
|
454
|
-
- `next`: Receives the result of `cmd` and returns the next Effect. Optional, defaulting to `(result) => Success(result)
|
|
606
|
+
- `cmd`: A function (sync or async) that performs the side effect. Inside a `Parallel` branch it is called with an `AbortSignal` that fires when a sibling branch fails; elsewhere it is called with no arguments. A function passed by name gets the signal as its first argument, so wrap one that takes an optional argument (see [Limitations](#limitations)).
|
|
607
|
+
- `next`: Receives the result of `cmd` and returns the next Effect. Optional, defaulting to `(result) => Success(result)`.
|
|
455
608
|
- `meta`: Optional metadata, passed to `onBeforeCommand`. A string `meta.name` becomes the Command's identity. Otherwise, the name of the function is used (`cmd.name`).
|
|
456
609
|
|
|
457
|
-
**Every Command needs
|
|
610
|
+
**Every Command needs a name.** Test assertions, trace entries, replay matching, and telemetry spans all use it. It is chosen in this order:
|
|
458
611
|
|
|
459
612
|
```js
|
|
460
613
|
Command(cmdFn, next, { name: 'chargeCard' }); // 1. meta.name, independent of how cmdFn was written
|
|
@@ -464,7 +617,7 @@ Command(function cmdChargeCard() {
|
|
|
464
617
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
465
618
|
```
|
|
466
619
|
|
|
467
|
-
|
|
620
|
+
Use `meta.name` in code that gets minified, since minifiers rename functions and would rename every step in every trace. Otherwise, naming the function is fine, and is what the examples do.
|
|
468
621
|
|
|
469
622
|
#### `Ask(nextFn)`
|
|
470
623
|
|
|
@@ -474,14 +627,22 @@ Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nex
|
|
|
474
627
|
|
|
475
628
|
Returns `{ type: 'Retry', effect, options, next }`.
|
|
476
629
|
|
|
477
|
-
- `options.attempts`: Max retries, not counting the first try (default: `3`).
|
|
478
|
-
- `options.delay`: Ms before the first retry (default: `100`).
|
|
479
|
-
- `options.backoff`: Multiplier applied to delay on each attempt (default: `1`, flat).
|
|
630
|
+
- `options.attempts`: Max retries, not counting the first try (default: `3`). Must be a positive integer; anything else, including `0`, throws a `TypeError`. To handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
631
|
+
- `options.delay`: Ms before the first retry (default: `100`). Must be a finite number of 0 or more, or it throws a `TypeError`.
|
|
632
|
+
- `options.backoff`: Multiplier applied to the delay on each attempt (default: `1`, flat). Must be a finite number of 0 or more, or it throws a `TypeError`.
|
|
633
|
+
|
|
634
|
+
An option set to `undefined` keeps its default.
|
|
635
|
+
|
|
480
636
|
- `options.onExhausted(error)`: Runs a fallback Effect when every attempt has failed, receiving `{ retryExhausted, lastError, attempts }`. The fallback's success feeds `next`; its failure propagates unwrapped. Per-use only. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
481
637
|
|
|
482
|
-
#### `Parallel(effects, next?)`
|
|
638
|
+
#### `Parallel(effects, next?, options?)`
|
|
639
|
+
|
|
640
|
+
Returns `{ type: 'Parallel', effects, next, options }`. Runs all effects at the same time. `next` receives the array of success values, in order, and is optional, defaulting to `(values) => Success(values)` as with `Command`. The first branch to fail cancels the others and its `Failure` is returned; `next` is not called. Which branch fails first depends on timing, so it is recorded, and a replay returns the same one. Each branch's Commands receive an `AbortSignal` as their only argument, so I/O that accepts it can be stopped while running; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
641
|
+
|
|
642
|
+
The second argument can be `next` or the options, so `Parallel(effects, { limit: 5 })` works.
|
|
483
643
|
|
|
484
|
-
|
|
644
|
+
- `limit`: the most branches running at once. Results and recorded paths stay in array order, so a limit changes only the pacing. A value that is not a positive integer throws a `TypeError`.
|
|
645
|
+
- `settled`: run every branch to the end and pass `next` one outcome per branch, `Success` or `Failure`, in array order. No branch cancels the others, and the `Parallel` never fails because of a branch. An `EffectTypeError` still escapes, because a malformed flow is a bug, not a branch outcome.
|
|
485
646
|
|
|
486
647
|
### Building pipelines
|
|
487
648
|
|
|
@@ -496,7 +657,7 @@ A step does not have to use the value it receives. In JavaScript, closing over s
|
|
|
496
657
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
497
658
|
```
|
|
498
659
|
|
|
499
|
-
In TypeScript
|
|
660
|
+
In TypeScript, types stop being checked at that step, because a function that ignores its parameter puts no constraint on the step before it. See [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context). Passing the value through every step keeps the whole pipeline checked, which is why the [Quick Start](#quick-start) does it.
|
|
500
661
|
|
|
501
662
|
One shape to avoid in either language:
|
|
502
663
|
|
|
@@ -507,7 +668,7 @@ One shape to avoid in either language:
|
|
|
507
668
|
};
|
|
508
669
|
```
|
|
509
670
|
|
|
510
|
-
|
|
671
|
+
A Command is only data, so one that is created and not returned never runs, and any `Failure` it would have produced is lost. Return the Command, and have its `next` return the value the rest of the pipeline needs.
|
|
511
672
|
|
|
512
673
|
### Running a flow
|
|
513
674
|
|
|
@@ -516,26 +677,28 @@ Commands are data, so a Command that is constructed and discarded never runs, an
|
|
|
516
677
|
Walks the flow, runs each Command with `async/await`, resolves `Ask` with the supplied `context`, and returns the final `Success` or `Failure`.
|
|
517
678
|
|
|
518
679
|
- `context`: Passed to `Ask`'s next function and to `onBeforeCommand`. `context.flowName` names the workflow in telemetry.
|
|
519
|
-
- `callConfig`: Per-call `onStep`, `onRun`,
|
|
520
|
-
- `onRun` fires
|
|
680
|
+
- `callConfig`: Per-call `onStep`, `onRun`, and `onBeforeCommand`, added to the `configureEffect` wiring unless `inherit: false`, which ignores that wiring for the run. A `retry` key throws a `TypeError`: retry options are per-use, passed to `Retry`. See [`configureEffect`](#configureeffectconfigs).
|
|
681
|
+
- `onRun` fires once per `runEffect` call. Retry attempts run inside that one span.
|
|
521
682
|
|
|
522
|
-
A step that returns something other than an Effect is a bug
|
|
683
|
+
A step that returns something other than an Effect is a bug, so `runEffect` throws an `EffectTypeError` naming the step instead of returning a `Failure`:
|
|
523
684
|
|
|
524
685
|
```
|
|
525
686
|
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
526
687
|
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
527
688
|
```
|
|
528
689
|
|
|
529
|
-
The same check catches a missing `return`, a Command's next function returning a plain value, and `runEffect(flow)` where `runEffect(flow(input))` was meant.
|
|
690
|
+
The same check catches a missing `return`, a Command's next function returning a plain value, a step or next function written as `async` (it returns a Promise, so do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their own arguments as the flow is built, before any of its I/O runs: `Command(db.findUser(email))`, which makes the call while building the flow, `Command(fn, { name })` with the metadata where `next` goes, a `Parallel` given something other than an array of Effects, and an `effectPipe` step that is not a function each throw an `EffectTypeError` naming the mistake. A Command whose function throws is still a `Failure`. A throw from a `next` function or a pure step is also a bug, and `runEffect` rejects with the error as thrown.
|
|
530
691
|
|
|
531
692
|
#### `configureEffect(...configs)`
|
|
532
693
|
|
|
533
|
-
- `onRun(effect, pipeline, flowName)` wraps the entire workflow; must `await pipeline()
|
|
534
|
-
- `onStep(name, type, op)` wraps each Command; must `await op()` and return its result. Returning a value _without_ calling `op()` is how replay works.
|
|
535
|
-
- `
|
|
536
|
-
- `
|
|
694
|
+
- `onRun(effect, pipeline, flowName)` wraps the entire workflow; must `await pipeline()` and return its result.
|
|
695
|
+
- `onStep(name, type, op, path)` wraps each Command; must `await op()` and return its result. A hook that calls another hook passes `path` on, since a replay matches steps on it. `op()` returns a promise, even for a synchronous Command. Returning a value _without_ calling `op()` is how replay works. A throw after `op()` succeeded is a bug in the hook: the run rejects, and `Retry` does not run the Command again. So is returning `undefined` after `op()` returned a value, which is what a hook that forgot its `return` does: the run rejects with a `TypeError` instead of continuing as though the Command had returned nothing. A throw without calling `op()` counts as the Command failing.
|
|
696
|
+
- `onStep` also wraps each `Parallel`, with `name` and `type` both `'Parallel'`. Its `op()` runs the branches, so a hook must call it; a hook that returns without calling it makes the run reject with a `TypeError`. It returns which branch, if any, cancelled the others, such as `{ cancelled: true, branch: 0 }`, and it returns even when a branch threw, since the run rejects only after the hook has returned. Telemetry gets one span per `Parallel`, with the spans of its branches' Commands inside it, marked as an error when the `Parallel` was cancelled.
|
|
697
|
+
- `onBeforeCommand(command, context)` fires before each Command; throw to abort. The run returns a `Failure` carrying the thrown error, and `Retry` does not retry it.
|
|
537
698
|
|
|
538
|
-
|
|
699
|
+
It only configures hooks. Retry options are passed to `Retry(effect, options)`, and a `retry` key here throws a `TypeError`.
|
|
700
|
+
|
|
701
|
+
Each call adds a layer of hooks on top of those already installed and returns a function that removes that layer. Passing several configurations to one call is the same as calling it once for each, so these two forms are equivalent:
|
|
539
702
|
|
|
540
703
|
```js
|
|
541
704
|
configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
@@ -545,6 +708,8 @@ configureEffect(telemetryHooks());
|
|
|
545
708
|
configureEffect(recordingHooks({ sink }));
|
|
546
709
|
```
|
|
547
710
|
|
|
711
|
+
This is the order the hooks run in during one `runEffect`:
|
|
712
|
+
|
|
548
713
|
```
|
|
549
714
|
runEffect(flow(input))
|
|
550
715
|
│
|
|
@@ -562,11 +727,9 @@ runEffect(flow(input))
|
|
|
562
727
|
│ │
|
|
563
728
|
│ ┌─ recording.onRun returns
|
|
564
729
|
├─ telemetry.onRun returns
|
|
565
|
-
│
|
|
566
|
-
retry: { ...telemetry.retry, ...recording.retry } later configurations win
|
|
567
730
|
```
|
|
568
731
|
|
|
569
|
-
Removing a layer
|
|
732
|
+
Removing a layer removes only that layer, even if others were installed after it, so a library can add its own hooks and remove them later without affecting the application's. Calling `configureEffect()` with no arguments removes every layer. A call whose arguments are all `undefined`, such as a conditional `configureEffect(flag ? hooks : undefined)`, installs nothing and removes nothing.
|
|
570
733
|
|
|
571
734
|
```js
|
|
572
735
|
const remove = configureEffect(telemetryHooks());
|
|
@@ -574,7 +737,7 @@ const remove = configureEffect(telemetryHooks());
|
|
|
574
737
|
remove();
|
|
575
738
|
```
|
|
576
739
|
|
|
577
|
-
**
|
|
740
|
+
**Per-call hooks.** By default, the hooks in a `callConfig` are merged with the configured ones by the same rules: configured wrappers go outside, and configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out of the run. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
578
741
|
|
|
579
742
|
```
|
|
580
743
|
inherit: true (default) inherit: false
|
|
@@ -582,53 +745,60 @@ remove();
|
|
|
582
745
|
onRun C.onRun (none)
|
|
583
746
|
onBeforeCommand C.onBeforeCommand (none)
|
|
584
747
|
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
585
|
-
retry { ...C.retry, ...K.retry } { ...defaults, ...K.retry }
|
|
586
748
|
```
|
|
587
749
|
|
|
588
|
-
`recordEffect` inherits, so recording inside an application that already has tracing
|
|
750
|
+
`recordEffect` inherits, so recording inside an application that already has tracing still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
589
751
|
|
|
590
752
|
### Recording and replay
|
|
591
753
|
|
|
592
754
|
#### `recorder(options?)`
|
|
593
755
|
|
|
594
|
-
Returns `{ onStep, entries, toTrace }`. Pass `onStep` to `runEffect`
|
|
756
|
+
Returns `{ onStep, entries, toTrace }`. Pass `onStep` to `runEffect` to record what every Command returned. A recorder holds the steps of every run it sees, so installed with `configureEffect` for a whole application it mixes requests into one trace, which a replay refuses; give each run its own recorder, as `examples/recording-example.js` does. `toTrace(meta)` packages the trace, and it copies the `initialInput` and `context` you give it at the moment you call it. A value that cannot be copied whole, such as a context holding a logger function, is copied around the parts that cannot be copied, which are kept as they are. If a Command can change either, for example an ORM save that adds an id to the object it was given, call `toTrace` with them before the run and take `trace` and `dropped` from a second call afterwards, as `recordEffect` does.
|
|
595
757
|
|
|
596
|
-
Each entry is `{ command, path, result, durationMs }`, or `{ command, path, error, durationMs }` when the Command threw, so a trace also
|
|
758
|
+
Each entry is `{ command, path, result, durationMs }`, or `{ command, path, threw: true, error, durationMs }` when the Command threw, so a trace also shows which step was slow. `threw` is what marks a step that threw, since a JSON copy of the trace drops `error` when the Command threw `undefined`; an entry with `error` and no `threw`, as older traces have, still counts as one. `path` is the Command's position in the flow, which is what a replay matches on. Each `Parallel` adds one entry, `{ command: 'Parallel', path, result }`, whose result says which branch, if any, cancelled the others: `{ cancelled: false }`, `{ cancelled: true, branch: 0 }`, or `branch: null` when an enclosing `Parallel` cancelled it. `redact` is not called for it, since it holds no data from your flow. Results are copied when recorded, so a later step that changes a returned object does not change the trace, and copied again when a replay hands them to the flow. In a value that cannot be copied whole, such as an object holding a function, the parts that cannot be copied are kept as they are. The copy keeps data but not classes; see [Recording in Production](#recording-in-production). Recording never changes the outcome of a run: if `redact` throws, the step is recorded as `'[redaction failed]'` and the flow carries on.
|
|
597
759
|
|
|
598
|
-
- `options.redact(value, name, kind)`:
|
|
760
|
+
- `options.redact(value, name, kind)`: Removes sensitive data from a trace. It receives every value the trace stores, and `kind` says which one it is. Each value is a copy, so `redact` can delete fields in place or return a new value, and the flow still sees what it was given:
|
|
599
761
|
|
|
600
762
|
```js
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
763
|
+
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
764
|
+
const mask = (value, field) =>
|
|
765
|
+
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
766
|
+
|
|
767
|
+
const redact = (value, name, kind) => {
|
|
768
|
+
if (kind === 'initialInput') return mask(value, 'password');
|
|
769
|
+
if (kind === 'context') return mask(value, 'authToken');
|
|
770
|
+
if (kind === 'error') return mask(value, 'attempted');
|
|
771
|
+
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
772
|
+
};
|
|
773
|
+
recorder({ redact });
|
|
774
|
+
|
|
775
|
+
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
776
|
+
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
609
777
|
```
|
|
610
778
|
|
|
611
|
-
|
|
779
|
+
Check the value before replacing a field in it. `{ ...value, email: '[redacted]' }` turns a `null` into an object, so a lookup that found nothing replays as one that found a user, and the flow takes the other branch. It also turns an error thrown as a string into an object holding its letters.
|
|
780
|
+
|
|
781
|
+
For `'result'` and `'error'`, `name` is the Command's name; for `'initialInput'` and `'context'`, it is the kind. Redacting `initialInput` rarely breaks replay, since Commands are not run; it matters only if a step branches on the removed field. Redacting `context` breaks `Ask` replay if a step reads what you removed.
|
|
612
782
|
|
|
613
783
|
- `options.maxEntries`: Cap trace length; overflow is counted in `dropped`.
|
|
614
784
|
- `options.stack`: Record stack traces for thrown errors (off by default).
|
|
615
785
|
|
|
616
786
|
#### `recordEffect(flowFn, initialInput, options?)`
|
|
617
787
|
|
|
618
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts
|
|
788
|
+
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts. To record an application without changing call sites, use the wiring in `examples/recording-example.js`, which gives each run its own recorder.
|
|
619
789
|
|
|
620
790
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
621
791
|
|
|
622
|
-
Replays a flow, feeding recorded results to Commands instead of running them. Returns `{ result, unreached }`: the flow's outcome, and the recorded entries the flow never asked for (empty when every step was reached). A flow that stops early
|
|
792
|
+
Replays a flow, feeding recorded results to Commands instead of running them. Returns `{ result, unreached }`: the flow's outcome, and the recorded entries the flow never asked for (empty when every step was reached). A flow that stops early raises no `TimeParadox`, so `unreached` is where that shows up. With a resolver, only `{ result }` is returned, since a resolver cannot list what it holds.
|
|
623
793
|
|
|
624
|
-
- `traceOrResolver`: a trace (or bare entries array) to replay directly, or a resolver function for traces stored in some other shape. A resolver returns `{ result }`, `{ error }`, or `undefined` if the step is unrecorded. A malformed trace rejects with a `ReplayError`.
|
|
625
|
-
- `options.context`: context for `Ask
|
|
626
|
-
- `options.onMissing`: `'throw'` (default) fails on an unrecorded step; `'execute'` runs the real Command, giving a recorded prefix with a live tail.
|
|
794
|
+
- `traceOrResolver`: a trace (or bare entries array) to replay directly, or a resolver function for traces stored in some other shape. A resolver returns `{ result }`, `{ error }`, or `undefined` if the step is unrecorded. A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`: answering with `{ result }` holding the recorded cancellation replays it as production decided, and anything else replays that `Parallel` by timing, as before. A malformed trace rejects with a `ReplayError`.
|
|
795
|
+
- `options.context`: context for `Ask`. With a trace it defaults to the context the trace recorded, so pass one only to replay with a different one. A resolver has no recorded context, so pass it one if the flow reads `Ask`.
|
|
796
|
+
- `options.onMissing`: `'throw'` (default) fails on an unrecorded step; `'execute'` runs the real Command, giving a recorded prefix with a live tail. Use it for a trace that was cut short, such as one capped by `maxEntries`. After a flow changes shape, for example when a Command is newly wrapped in `Retry`, its steps sit at new positions, so `'execute'` would run all of them live and leave the recorded ones unused: record the flow again instead.
|
|
627
797
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
628
|
-
- `options.hooks` (default `false`): run the replay inside the
|
|
629
|
-
- `options.onResolved(step, outcome)`: observe each replayed step.
|
|
798
|
+
- `options.hooks` (default `false`): run the replay inside the configured hooks, so they see the replayed steps, a configured recorder included. When off, the configured hooks are skipped, so a replay cannot reach a telemetry backend or a trace sink.
|
|
799
|
+
- `options.onResolved(step, outcome)`: observe each replayed step. If it throws, the replay stops there and `replayEffect` rejects with that error; it never changes a step's outcome.
|
|
630
800
|
|
|
631
|
-
A trace whose entries carry no `path` (written by hand, or recorded before paths existed) is matched
|
|
801
|
+
A trace whose entries carry no `path` (written by hand, or recorded before paths existed) is matched by position, which works for a sequential flow; a `Parallel` step in such a trace is refused with a `ReplayError`, since the order branches finish in cannot tell them apart.
|
|
632
802
|
|
|
633
803
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
634
804
|
|
|
@@ -636,10 +806,12 @@ Replays a trace and narrates each step with its recorded duration, naming any re
|
|
|
636
806
|
|
|
637
807
|
## Limitations
|
|
638
808
|
|
|
639
|
-
- **`Retry` repeats the
|
|
640
|
-
- **Cancelling a `Parallel` branch
|
|
641
|
-
- **
|
|
642
|
-
- **
|
|
643
|
-
- **
|
|
644
|
-
- **A
|
|
645
|
-
-
|
|
809
|
+
- **`Retry` repeats everything it wraps, including a Command's `next`.** When a Command's function throws, every Command inside the `Retry` runs again, including the ones that already succeeded. If a retried Command's `next` continues the flow, everything after it is retried too. Give a retried Command the default `next` and continue in a later pipeline step, or make sure every Command it reaches is safe to run more than once. A `Failure` a step returned is not retried at all, so a function that catches its own error and returns it as a value is never retried: see [Which Errors Are Data](#which-errors-are-data).
|
|
810
|
+
- **Cancelling a `Parallel` branch cannot stop everything.** A cancelled branch starts no new Commands, and a function that uses the `AbortSignal` it is given can be stopped while running. A function that ignores the signal runs to completion, so a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch to finish before returning, so no cancelled work is still running after the `Failure` is returned.
|
|
811
|
+
- **Inside a `Parallel`, a Command's function gets the `AbortSignal` as its first argument.** A function passed by name that has an optional first argument takes the signal for it, so the same Command behaves differently inside a `Parallel` than outside one: `Command(nanoid)` returns an empty ID, since `nanoid(size = 21)` reads the signal as the size, and a repository helper with a default page size gets the signal as its page size. Wrap such a function, as in `Command(() => nanoid())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
812
|
+
- **A `Failure` carries everything.** It holds the full error and the `initialInput` that `effectPipe` attached, and neither is trimmed, because tests and debugging need both. `redact` keeps sensitive data out of a **trace**. Keeping it out of your **logs** is up to you: log `result.error` rather than the whole `Failure`. The same goes for the outcomes a settled `Parallel` passes to `next`, which can carry their branch's input; for a login or registration flow, that input holds credentials.
|
|
813
|
+
- **Replay checks the path, not the values.** Replaying an old trace cannot check a fix that changes what a flow computes, because every Command's result comes from the recording: the replay hands the step the value from before the fix, reports `Success`, and flags nothing. It can check a fix that changes which Commands run. A removed step shows up in `unreached` only if it was the last step the flow reached; removed from the middle, it shifts the later paths and the replay stops at a `TimeParadox`.
|
|
814
|
+
- **A trace keeps data, not objects.** Recorded results come back without their classes: a money object or a database entity as a plain object, a `Buffer` as a plain byte array, and an error inside a result without properties such as `code`. Through JSON, a `Date` also comes back as a string and a `Map` as `{}`. A flow whose `next` calls a method on a result, or branches on one of those properties, replays differently from production. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|
|
815
|
+
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two concurrent requests does not, because a trace records one flow. `Parallel` branches replay with the results each branch saw, and a cancelled `Parallel` with the branch that cancelled it, but not the order they ran in, so replay cannot settle a race between branches that share state.
|
|
816
|
+
- **Configuration belongs to one copy of the library.** `configureEffect` stores hooks in module state, so two copies of the library in one process (from a dual ESM and CJS setup, two versions in the dependency tree, or a worker thread) each have their own. Hooks configured in one do not apply to the other, and nothing warns you.
|
|
817
|
+
- **A Command needs a name.** `meta.name` survives minification. `cmd.name` does not, unless the minifier is set to keep function names.
|