pure-effect 0.19.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +148 -119
- package/index.d.ts +26 -19
- package/index.js +119 -39
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [How It Works](#how-it-works) · [Testing Without Mocks](#testing-without-mocks) · [Coming from async/await](#coming-from-asyncawait) · [Writing Flows as Generators](#writing-flows-as-generators)
|
|
18
18
|
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production) · [Hooks](#hooks)
|
|
19
19
|
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Writing Loops](#writing-loops) · [Which Errors Are Data](#which-errors-are-data) · [Generator Flows in Depth](#generator-flows-in-depth) · [TypeScript](#typescript-typed-errors-and-context)
|
|
20
|
-
- **Reference:** [Why Pure Effect](#why-pure-effect) · [
|
|
20
|
+
- **Reference:** [Why Pure Effect](#why-pure-effect) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
21
|
|
|
22
22
|
The six sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: recording, retries, loops, steps that run at the same time, or TypeScript.
|
|
23
23
|
|
|
@@ -124,7 +124,7 @@ assert.equal(commandName(step2), 'cmdSaveUser');
|
|
|
124
124
|
// The database was never touched.
|
|
125
125
|
```
|
|
126
126
|
|
|
127
|
-
Write both kinds: only a step test sees the value passed from one step to the next.
|
|
127
|
+
Write both kinds: only a step test sees the value passed from one step to the next. Suppose the email guard returned `Success(true)` instead of `Success(input)`. The step test would fail. The flow test would still pass, since the calls and their order are unchanged, while the flow saved `true` instead of the user.
|
|
128
128
|
|
|
129
129
|
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
130
130
|
|
|
@@ -145,7 +145,7 @@ Hand it a recorded run instead, and each Command gets what it returned the first
|
|
|
145
145
|
import { recordEffect } from 'pure-effect';
|
|
146
146
|
|
|
147
147
|
// A real run returns its result, and a trace of what each Command returned.
|
|
148
|
-
const { result: live, trace } = await recordEffect(registerUserFlow
|
|
148
|
+
const { result: live, trace } = await recordEffect(registerUserFlow(input));
|
|
149
149
|
|
|
150
150
|
// Anywhere else, with nothing connected, the same flow gets the same answers.
|
|
151
151
|
const { result: replayed } = await replayEffect(registerUserFlow(trace.initialInput), trace);
|
|
@@ -160,7 +160,7 @@ In TypeScript, a walk needs a check at each step that the step is a Command; [Te
|
|
|
160
160
|
|
|
161
161
|
## Coming from async/await
|
|
162
162
|
|
|
163
|
-
A flow does what `async`/`await` code does, as data
|
|
163
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command: which call to make, and what to do with the answer. Each thing you would write with `async`/`await` has an equivalent:
|
|
164
164
|
|
|
165
165
|
| `async`/`await` | Generator flow | Pipeline |
|
|
166
166
|
| --- | --- | --- |
|
|
@@ -214,9 +214,9 @@ The flow's test of the same branch hands one step the answer, with nothing to re
|
|
|
214
214
|
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
215
215
|
```
|
|
216
216
|
|
|
217
|
-
Each step is a place a test can start, so a
|
|
217
|
+
Each step is a place a test can start, so a test reaches a branch by calling the step it lives in. A test of an `async` function has to run everything before the branch, with every call on the way replaced. The mocks do check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
218
218
|
|
|
219
|
-
You can switch one function at a time
|
|
219
|
+
You can switch one function at a time. Keep its name and parameters, run the flow in its body, and turn a `Failure` back into a throw, so the code that calls it does not change. The mocked test above passes against this version too:
|
|
220
220
|
|
|
221
221
|
```js
|
|
222
222
|
// Callers still await registerUserAsync and catch what it throws.
|
|
@@ -259,7 +259,7 @@ assert.equal(onboarded.value.workspace.ownerId, onboarded.value.user.id);
|
|
|
259
259
|
|
|
260
260
|
A pipeline would need a step that passes the user and the workspace on together; see [Composing Larger Flows](#composing-larger-flows). Keep decisions in step functions all the same, as `registerUserFlowGen` does, so each can be tested on its own.
|
|
261
261
|
|
|
262
|
-
The body can run more than once:
|
|
262
|
+
The body can run more than once: for each `Retry` attempt and each run, as an `async` function does when called again, and when a test walks a second branch. So keep it to its own variables, and get `Date.now()` or a random ID from a Command, so a replay gets the recorded one. [Generator Flows in Depth](#generator-flows-in-depth) covers when it runs again, how a `catch` or `finally` in it behaves when a step stops the flow, and the mistakes `effectGen` catches.
|
|
263
263
|
|
|
264
264
|
### Testing a generator flow
|
|
265
265
|
|
|
@@ -282,7 +282,7 @@ Record what each Command returned, then feed those results back into the flow to
|
|
|
282
282
|
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
283
283
|
|
|
284
284
|
// Record a real run. The context's flowName names the flow in the trace.
|
|
285
|
-
const { result, trace } = await recordEffect(checkoutFlow
|
|
285
|
+
const { result, trace } = await recordEffect(checkoutFlow(input), {
|
|
286
286
|
version: process.env.BUILD_ID,
|
|
287
287
|
context: { flowName: 'checkout' }
|
|
288
288
|
});
|
|
@@ -327,7 +327,9 @@ it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
|
327
327
|
});
|
|
328
328
|
```
|
|
329
329
|
|
|
330
|
-
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
330
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
331
|
+
|
|
332
|
+
A step the trace lacks ends the replay in a `Failure` holding a `ReplayError`. A trace lacks a step that comes after the end of the recording, one a hook vetoed in production, and one the recorder dropped under `maxEntries`.
|
|
331
333
|
|
|
332
334
|
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
333
335
|
|
|
@@ -369,29 +371,31 @@ A trace is plain JSON. This one is from a checkout whose card charge timed out o
|
|
|
369
371
|
- A number counts the Commands, `Retry`s and `Parallel`s the flow reached, from `0`. A `Success` or an `Ask` between them adds nothing.
|
|
370
372
|
- `r` and a number is one attempt of the `Retry` at that position, `r0` being the first try, and `rf` is its `onExhausted` fallback.
|
|
371
373
|
- `p` and a number is one branch of the `Parallel` at that position, from `p0`. The `Parallel` has an entry of its own, at a path such as `2p`, which records whether a branch cancelled the others.
|
|
372
|
-
- `/` goes one level in
|
|
374
|
+
- `/` goes one level in. `1r1/0` is the first step of the second try of the `Retry` at position 1. `2p1/0` is the first step of the second branch of the `Parallel` at position 2.
|
|
373
375
|
|
|
374
376
|
### What a replay cannot check
|
|
375
377
|
|
|
376
378
|
A replay proves that the flow asks for the same Commands in the same order and handles the same answers. Every result comes from the recording, so it cannot check:
|
|
377
379
|
|
|
378
|
-
- **What the flow computes.** Fix how a refund amount is calculated and replay the bad run
|
|
379
|
-
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix
|
|
380
|
+
- **What the flow computes.** Fix how a refund amount is calculated and replay the bad run: the charge step still gets the old amount from the recording. The replay reports `Success`, returns the value from before the fix, and flags nothing. Test a value change against the pure step that computes it.
|
|
381
|
+
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix. Catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
380
382
|
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
381
383
|
|
|
382
|
-
When a Command's result changes shape, rename the Command. A trace matches it by name
|
|
384
|
+
When a Command's result changes shape, rename the Command. A trace matches it by name. So under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
383
385
|
|
|
384
386
|
### Keeping a replay faithful
|
|
385
387
|
|
|
386
|
-
Put anything that varies between runs inside a Command. For replay, the current time or a random ID counts as I/O
|
|
388
|
+
Put anything that varies between runs inside a Command. For replay, the current time or a random ID counts as I/O. In a Command it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay, and drifts from the trace.
|
|
389
|
+
|
|
390
|
+
To check a flow, record it, replay the trace, and compare the outcomes. A step that computes a fresh value shows up as a different final value, or as a `TimeParadox` if it changes which Commands run. A trace holds each result in the form JSON stores it, so this also catches a result your storage cannot keep, such as a `Date` the flow calls a method on:
|
|
387
391
|
|
|
388
392
|
```js
|
|
389
|
-
const { result, trace } = await recordEffect(registerUserFlow
|
|
393
|
+
const { result, trace } = await recordEffect(registerUserFlow(input));
|
|
390
394
|
const { result: replayed } = await replayEffect(registerUserFlow(input), trace);
|
|
391
395
|
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive storage
|
|
392
396
|
```
|
|
393
397
|
|
|
394
|
-
A `Failure` compares the same way when the Command threw a plain `Error
|
|
398
|
+
A `Failure` compares the same way when the Command threw a plain `Error`. It comes back with its message, name, cause, custom properties and an `AggregateError`'s list of errors, even a `name` or `cause` set after it was created. An error of any other class, built-in ones included, comes back as a plain `Error` carrying that class's name. So `deepEqual` from `node:assert/strict` refuses the replayed `TypeError` that `fetch` rejects with when the network fails. Where a Command can throw one, compare `error.name` and `error.message` instead.
|
|
395
399
|
|
|
396
400
|
### Replaying from your own storage
|
|
397
401
|
|
|
@@ -430,19 +434,31 @@ configureEffect(
|
|
|
430
434
|
|
|
431
435
|
What the example does with each run:
|
|
432
436
|
|
|
433
|
-
- **Which runs it keeps.** `keep` decides
|
|
434
|
-
- **
|
|
437
|
+
- **Which runs it keeps.** `keep` decides. By default it keeps a run that ended in a `Failure`, and discards a successful one after holding it in memory.
|
|
438
|
+
- **When your own code throws.** A `TypeError` after an API changed the shape of its response, say, makes the run reject. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept, and replaying it reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded. That run did no I/O, so its input reproduces it.
|
|
439
|
+
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out and marked `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
435
440
|
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
436
441
|
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
437
442
|
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
438
443
|
|
|
439
|
-
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` built, so build the outermost flow with
|
|
444
|
+
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` or `effectGen` built, so build the outermost flow with one of them. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
440
445
|
|
|
441
446
|
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
442
447
|
|
|
443
|
-
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow
|
|
448
|
+
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow. So neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes:
|
|
449
|
+
|
|
450
|
+
| Recorded | Replayed as |
|
|
451
|
+
| ----------------------------------- | --------------------------------------------------- |
|
|
452
|
+
| a money object or a database entity | a plain object, without its methods |
|
|
453
|
+
| a `Date` | a string |
|
|
454
|
+
| a `Map` or a `Set` | `{}` |
|
|
455
|
+
| a `Buffer` | `{ type: 'Buffer', data }` |
|
|
456
|
+
| `undefined` in an array | `null` |
|
|
457
|
+
| an error returned inside a result | its own fields, such as `code`, without its message |
|
|
444
458
|
|
|
445
|
-
|
|
459
|
+
A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it. So a context holding a lazy client is left out whole.
|
|
460
|
+
|
|
461
|
+
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`. Rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
446
462
|
|
|
447
463
|
### Keeping secrets out of a trace
|
|
448
464
|
|
|
@@ -467,7 +483,7 @@ assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing fou
|
|
|
467
483
|
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
468
484
|
```
|
|
469
485
|
|
|
470
|
-
A replay rebuilds the flow from the redacted input and context
|
|
486
|
+
A replay rebuilds the flow from the redacted input and context. So a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does. A context field an `Ask` reads should stay.
|
|
471
487
|
|
|
472
488
|
## Hooks
|
|
473
489
|
|
|
@@ -571,9 +587,9 @@ app.post('/checkout', async (req, res) => {
|
|
|
571
587
|
});
|
|
572
588
|
```
|
|
573
589
|
|
|
574
|
-
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run
|
|
590
|
+
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run. So a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, belongs in a Command, whose result the trace records each time.
|
|
575
591
|
|
|
576
|
-
Keep the context to plain values such as IDs, flags and settings. Recording copies it into every trace, so a database client or connection pool in it puts its connection string
|
|
592
|
+
Keep the context to plain values such as IDs, flags and settings. Recording copies it into every trace, so a database client or connection pool in it puts its connection string there. One whose properties throw when read, as a lazy client's can, leaves the whole context out, and a replay then needs it passed in. Reach a service from the Command's function, as `findProduct` reaches `db`. To run a flow against fakes or a test database, have the function that builds it take the services, and pass them in the test:
|
|
577
593
|
|
|
578
594
|
```js
|
|
579
595
|
// Takes its catalog, so a test can run it against a fake.
|
|
@@ -595,7 +611,7 @@ A walk or a replay never runs a Command's function, so this is the test that cat
|
|
|
595
611
|
`Retry` runs part of a flow again when a Command's function throws. A `Failure` a step returned is not retried: the flow decided to stop, so the `Failure` passes on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency; to reuse settings, share an object:
|
|
596
612
|
|
|
597
613
|
```js
|
|
598
|
-
const flakyNetwork = {
|
|
614
|
+
const flakyNetwork = { retries: 3, delay: 200, backoff: 2 };
|
|
599
615
|
Retry(fetchPrice(sku), flakyNetwork);
|
|
600
616
|
```
|
|
601
617
|
|
|
@@ -609,23 +625,23 @@ const fetchWeather = (city) => {
|
|
|
609
625
|
fetch(`https://example-weather-api.com/v1/current?city=${city}`).then((r) => r.json());
|
|
610
626
|
return Retry(
|
|
611
627
|
Command(cmdFetchWeather, (data) => (data.error ? Failure(data.error) : Success(data))),
|
|
612
|
-
{
|
|
628
|
+
{ retries: 3, delay: 200, backoff: 2 } // 200ms, 400ms, 800ms
|
|
613
629
|
);
|
|
614
630
|
};
|
|
615
631
|
|
|
616
632
|
const weatherFn = fetchWeather('Tokyo');
|
|
617
633
|
assert.equal(weatherFn.type, 'Retry');
|
|
618
|
-
assert.equal(weatherFn.options.
|
|
634
|
+
assert.equal(weatherFn.options.retries, 3);
|
|
619
635
|
```
|
|
620
636
|
|
|
621
637
|
**Wrap the Command that fails, not the pipeline.** Each attempt runs everything inside the `Retry` again, including Commands that already succeeded:
|
|
622
638
|
|
|
623
639
|
```js
|
|
624
640
|
// Dangerous: a flaky receipt step charges the customer again on every attempt.
|
|
625
|
-
Retry(effectPipe(chargeCard, sendReceipt)(order), {
|
|
641
|
+
Retry(effectPipe(chargeCard, sendReceipt)(order), { retries: 3 });
|
|
626
642
|
|
|
627
643
|
// Correct: only the step that fails transiently is retried.
|
|
628
|
-
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), {
|
|
644
|
+
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), { retries: 3 }))(order);
|
|
629
645
|
```
|
|
630
646
|
|
|
631
647
|
Wrapping a pipeline is safe only when every Command in it is idempotent (safe to run more than once).
|
|
@@ -636,7 +652,7 @@ Wrapping a pipeline is safe only when every Command in it is idempotent (safe to
|
|
|
636
652
|
// Dangerous: `next` continues the booking, so every later step is inside the Retry.
|
|
637
653
|
Retry(
|
|
638
654
|
Command(cmdHoldSeat, (seat) => chargeCard(trip, seat)),
|
|
639
|
-
{
|
|
655
|
+
{ retries: 2 }
|
|
640
656
|
);
|
|
641
657
|
```
|
|
642
658
|
|
|
@@ -645,7 +661,7 @@ If a later Command's function throws, the `Retry` holds the seat and charges the
|
|
|
645
661
|
```js
|
|
646
662
|
// Safe: the tree the Retry repeats is that one Command.
|
|
647
663
|
effectPipe(
|
|
648
|
-
() => Retry(Command(cmdHoldSeat), {
|
|
664
|
+
() => Retry(Command(cmdHoldSeat), { retries: 2 }),
|
|
649
665
|
(seat) => chargeCard(trip, seat)
|
|
650
666
|
);
|
|
651
667
|
```
|
|
@@ -653,7 +669,7 @@ effectPipe(
|
|
|
653
669
|
When every attempt fails, `runEffect` returns a `Failure` with this error:
|
|
654
670
|
|
|
655
671
|
```text
|
|
656
|
-
{ retryExhausted: true, lastError: <the last error>,
|
|
672
|
+
{ retryExhausted: true, lastError: <the last error>, retries: 3 }
|
|
657
673
|
```
|
|
658
674
|
|
|
659
675
|
In TypeScript, a `Retry` adds `RetryExhaustedError` to the error union, beside the errors of the steps it wraps. `lastError` is typed `unknown`, since it is whatever the Command's function threw, so check it before using it, for example with `instanceof Error`.
|
|
@@ -663,8 +679,8 @@ In TypeScript, a `Retry` adds `RetryExhaustedError` to the error union, beside t
|
|
|
663
679
|
```js
|
|
664
680
|
const fetchPrice = (sku) =>
|
|
665
681
|
Retry(fetchLivePrice(sku), {
|
|
666
|
-
|
|
667
|
-
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError,
|
|
682
|
+
retries: 3,
|
|
683
|
+
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError, retries }
|
|
668
684
|
});
|
|
669
685
|
```
|
|
670
686
|
|
|
@@ -695,7 +711,7 @@ Parallel(subscriptions.map(billOne), { limit: 5 });
|
|
|
695
711
|
Parallel(subscriptions.map(billOne), { limit: 5, settled: true });
|
|
696
712
|
```
|
|
697
713
|
|
|
698
|
-
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end and `next` receives one `Success` or `Failure` per branch, in order
|
|
714
|
+
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end, and `next` receives one `Success` or `Failure` per branch, in order. A failed record is then something you can count instead of the end of the job. In TypeScript, write `settled: true` where you call `Parallel`, or mark a shared options object `as const`. Since `settled` decides what `next` receives, a value the compiler only knows as `true` or `false` is refused.
|
|
699
715
|
|
|
700
716
|
```js
|
|
701
717
|
Parallel(subscriptions.map(billOne), (outcomes) => Success(outcomes.map(summarize)), { limit: 5, settled: true });
|
|
@@ -711,7 +727,7 @@ Parallel(work, (outcomes) => Success(outcomes.map((o) => (o.type === 'Success' ?
|
|
|
711
727
|
|
|
712
728
|
The outcomes are plain `Success` and `Failure` objects. Returning one of them from `next` is the same as returning any other `Failure`: the flow stops, and `Retry` does not run it again.
|
|
713
729
|
|
|
714
|
-
**Collecting some failures and stopping on others.** `settled` decides for every failure at once.
|
|
730
|
+
**Collecting some failures and stopping on others.** `settled` decides for every failure at once. Sometimes a batch should note a missing record and go on, but stop everything when its credentials expire. Leave `settled` off and decide in each branch: return what should be collected as a value inside a `Success`, and what should stop the batch as a `Failure`, which cancels the other branches:
|
|
715
731
|
|
|
716
732
|
```js
|
|
717
733
|
// A missing user is counted; an expired token stops the batch.
|
|
@@ -747,9 +763,13 @@ const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${user
|
|
|
747
763
|
|
|
748
764
|
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
749
765
|
|
|
766
|
+
A function gets the signal only when it declares a parameter without a default value. One written as `(...args)` or `({ signal } = {})`, or wrapped by a helper that hides its parameters, gets none, so once started it runs to the end. A parameter with a default value does not count, so `Command(nanoid)`, whose size is written `size = 21`, works everywhere.
|
|
767
|
+
|
|
768
|
+
That rule catches a function passed by name whose first parameter is optional and has no default. `function listUsers(page) { page ??= 1; return db.users(page); }` gets the signal as its page inside a `Parallel`, and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
769
|
+
|
|
750
770
|
### Undoing what succeeded when a branch fails
|
|
751
771
|
|
|
752
|
-
Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped.
|
|
772
|
+
Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped. Sometimes one of them did something that has to be undone, such as charging a card for an order whose stock then ran out. The code that called `runEffect` no longer has the charge to refund. Use `settled: true`, so `next` sees every outcome, and have it return the steps that undo whatever succeeded, followed by the failure:
|
|
753
773
|
|
|
754
774
|
```js
|
|
755
775
|
const reserveStock = (order) => {
|
|
@@ -811,9 +831,9 @@ A settled `Parallel` cancels nothing, so both steps always run to the end, and t
|
|
|
811
831
|
|
|
812
832
|
### Errors no branch returned
|
|
813
833
|
|
|
814
|
-
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type
|
|
834
|
+
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type. So give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)). Otherwise it falls into the last error it expects, and reports a vetoed charge as a declined card.
|
|
815
835
|
|
|
816
|
-
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data))
|
|
836
|
+
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)). To stop a batch, read the operator's switch in a Command whose `next` returns a `Failure`, rather than vetoing in `onBeforeCommand`:
|
|
817
837
|
|
|
818
838
|
```js
|
|
819
839
|
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
@@ -833,7 +853,11 @@ Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed
|
|
|
833
853
|
|
|
834
854
|
### Replaying a cancelled `Parallel`
|
|
835
855
|
|
|
836
|
-
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
856
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
857
|
+
|
|
858
|
+
If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
859
|
+
|
|
860
|
+
A generator flow in a cancelled branch can run its `catch` and `finally` one step later in a replay than it did in production. A trace marks where a branch stopped only by its Commands. So a branch cancelled during a `Retry`'s wait, just before an `Ask`, or at a `Retry` or `Parallel` with no Command in it replays as though it stopped at its next Command.
|
|
837
861
|
|
|
838
862
|
## Composing Larger Flows
|
|
839
863
|
|
|
@@ -894,7 +918,7 @@ const findUnpaidInvoices = (customerId) => {
|
|
|
894
918
|
// Second part: the loop's body, a flow of its own for one invoice.
|
|
895
919
|
const chargeInvoice = (invoice) => {
|
|
896
920
|
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
897
|
-
return Retry(Command(cmdChargeInvoice), {
|
|
921
|
+
return Retry(Command(cmdChargeInvoice), { retries: 3 });
|
|
898
922
|
};
|
|
899
923
|
|
|
900
924
|
const sendReceipt = (receipt) => {
|
|
@@ -925,7 +949,7 @@ assert.deepEqual(await runEffect(payAll([])), Success([]));
|
|
|
925
949
|
|
|
926
950
|
Each invoice is paid after the one before it has finished, and the first that fails stops the rest from starting, as in the `async` version. When the payments can overlap, a higher `limit` runs several at a time.
|
|
927
951
|
|
|
928
|
-
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one, since a `Retry` runs everything it wraps again (see [Retrying Transient Failures](#retrying-transient-failures))
|
|
952
|
+
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one, since a `Retry` runs everything it wraps again (see [Retrying Transient Failures](#retrying-transient-failures)). Around the whole loop it would charge the invoices already paid. Around the whole body, a receipt that failed to send would charge the card again.
|
|
929
953
|
|
|
930
954
|
The `Parallel`'s `next` receives what each body returned, in order, so a total or a summary of the loop goes there: `Parallel(invoices.map(payInvoice), (receipts) => Success(receipts.length), { limit: 1 })`. To keep going past a failure, as a `try`/`catch` inside the loop would, add `settled: true`: every invoice is tried, and `next` receives one `Success` or `Failure` per invoice. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
931
955
|
|
|
@@ -949,7 +973,7 @@ assert.equal(fromLoop.result.error.retryExhausted, true);
|
|
|
949
973
|
assert.deepEqual(fromLoop.result, fromParallel.result);
|
|
950
974
|
```
|
|
951
975
|
|
|
952
|
-
It stops at the first invoice that fails, as the `Parallel` with `limit: 1` does. Use the `Parallel` instead when
|
|
976
|
+
It stops at the first invoice that fails, as the `Parallel` with `limit: 1` does. Use the `Parallel` instead when payments can overlap, with a higher `limit`. Use it too when the loop has to keep going past a failure, with `settled: true`, since a `catch` in the body cannot recover. The two forms record differently, so a trace recorded from one replays against the other as a `TimeParadox`.
|
|
953
977
|
|
|
954
978
|
Sometimes getting the list is a loop of its own. Paging through an API with a cursor is one: each request needs the cursor from the page before, so the requests cannot be listed in advance. Write that loop as an ordinary `async` function, call it from one Command, and map over the list it returns, as above:
|
|
955
979
|
|
|
@@ -1022,7 +1046,7 @@ Retry(
|
|
|
1022
1046
|
Command(function cmdFetchPrice() {
|
|
1023
1047
|
return pricing.get(sku);
|
|
1024
1048
|
}),
|
|
1025
|
-
{
|
|
1049
|
+
{ retries: 3 }
|
|
1026
1050
|
);
|
|
1027
1051
|
```
|
|
1028
1052
|
|
|
@@ -1034,18 +1058,24 @@ Retry(
|
|
|
1034
1058
|
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
1035
1059
|
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
1036
1060
|
|
|
1037
|
-
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is
|
|
1061
|
+
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is a bug, and is thrown as it is rather than returned as a `Failure`. `runEffect` rejects with it, or, from a step before the first Command, the call that builds the flow throws it.
|
|
1038
1062
|
|
|
1039
1063
|
### A malformed flow
|
|
1040
1064
|
|
|
1041
|
-
A flow that is not made of Effects is a bug, so it throws an `EffectTypeError` that says where, rather than returning a `Failure`. The steps before a flow's first Command run as it is built
|
|
1065
|
+
A flow that is not made of Effects is a bug, so it throws an `EffectTypeError` that says where, rather than returning a `Failure`. The steps before a flow's first Command run as it is built. So a mistake in one of them is thrown by the call that builds the flow, as `registerUserFlow(input)` would throw if its first step returned a plain object:
|
|
1042
1066
|
|
|
1043
1067
|
```
|
|
1044
1068
|
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
1045
1069
|
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
1046
1070
|
```
|
|
1047
1071
|
|
|
1048
|
-
A mistake after the first Command makes `runEffect` reject. That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments where they are built, before their own I/O runs
|
|
1072
|
+
A mistake after the first Command makes `runEffect` reject. That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments where they are built, before their own I/O runs. Each of these throws an `EffectTypeError` there:
|
|
1073
|
+
|
|
1074
|
+
- `Command(db.findUser(email))`, which makes the call while the flow is built
|
|
1075
|
+
- `Command(fn, { name })`, with the metadata in `next`'s place
|
|
1076
|
+
- a `Parallel` or `effectPipe` given the wrong thing
|
|
1077
|
+
|
|
1078
|
+
One built inside a `next` is checked when that `next` runs.
|
|
1049
1079
|
|
|
1050
1080
|
## Generator Flows in Depth
|
|
1051
1081
|
|
|
@@ -1053,7 +1083,7 @@ What [Writing Flows as Generators](#writing-flows-as-generators) leaves out: whe
|
|
|
1053
1083
|
|
|
1054
1084
|
### What runs once, and what runs again
|
|
1055
1085
|
|
|
1056
|
-
A run goes through the body once, top to bottom, as `async` code does.
|
|
1086
|
+
A run goes through the body once, top to bottom, as `async` code does. Running the flow again starts the body from the top, as calling an `async` function again does. Each `Retry` attempt, each run of one flow value, and each `Parallel` branch holding it gets a fresh build, so every Command in it sees the same values. A test that hands an earlier step a second answer, to walk another branch, runs the body again from the top too, feeding it the answers before that step:
|
|
1057
1087
|
|
|
1058
1088
|
```js
|
|
1059
1089
|
const flow = registerUserFlowGen(input);
|
|
@@ -1066,13 +1096,13 @@ assert.deepEqual(flow.next({ id: 1 }), Failure('Email already in use.'));
|
|
|
1066
1096
|
assert.equal(commandName(flow.next(null)), 'cmdSaveUser');
|
|
1067
1097
|
```
|
|
1068
1098
|
|
|
1069
|
-
So keep the body to its own variables.
|
|
1099
|
+
So keep the body to its own variables. A body run again for a walk has to ask for the same steps in the same order, and one that asks for a different one stops with an `EffectTypeError` naming both. `Date.now()`, a random ID, or a variable outside the body can change between runs. Get the first two from a Command, as a pipeline does, so a replay gets the recorded value. Pass the third in the flow's input.
|
|
1070
1100
|
|
|
1071
1101
|
### How a generator flow stops
|
|
1072
1102
|
|
|
1073
1103
|
`return` ends the flow. A plain value is its `Success`, `return Failure(error)` stops it with that error, and returning a Command or any other Effect runs it, as `return fetch(url)` does in an `async` function. A `throw` in the body is a bug, as one in a pipeline step is, and `runEffect` rejects with it.
|
|
1074
1104
|
|
|
1075
|
-
When a step stops the flow, because its Command threw or it returned a `Failure`, the body's `catch` and `finally` blocks run, as they would for an `await` that rejected. The `catch` gets an error saying where the flow stopped, not the original, which
|
|
1105
|
+
When a step stops the flow, because its Command threw or it returned a `Failure`, the body's `catch` and `finally` blocks run, as they would for an `await` that rejected. The `catch` gets an error saying where the flow stopped, not the original, which stays the flow's `Failure`. So it cannot branch on the original's fields: `e.code` and the like are `undefined`, and `e.name` is `FlowStopped`. It can log, but it cannot recover:
|
|
1076
1106
|
|
|
1077
1107
|
```js
|
|
1078
1108
|
const unreliableDb = {
|
|
@@ -1112,6 +1142,10 @@ await assert.rejects(runEffect(saveLater({ id: 1 })), /kept going after Command
|
|
|
1112
1142
|
|
|
1113
1143
|
Handle an error the flow expects inside the Command's function, and return it as data for the body to branch on; recover from one that keeps happening with `Retry`'s `onExhausted`. See [Which Errors Are Data](#which-errors-are-data). Cleanup for the whole run goes in a `try`/`finally` around `runEffect`, as it does for a pipeline.
|
|
1114
1144
|
|
|
1145
|
+
A body's `finally` does not always run. It does not run when a bug or a `TimeParadox` rejects the run. In a `Parallel` where one branch threw, another branch that stops for its own reason before the rejection reaches it may still run its `catch` and `finally`, depending on timing.
|
|
1146
|
+
|
|
1147
|
+
A step in a body's `finally` is not supported. It runs when the body ends normally, and is refused when a step stopped the body. But if an error is passing through the body when its `finally` reaches a step, and the flow is cancelled before that step starts, the error is lost.
|
|
1148
|
+
|
|
1115
1149
|
### Mistakes it catches
|
|
1116
1150
|
|
|
1117
1151
|
| Written | What happens |
|
|
@@ -1147,17 +1181,17 @@ if (result.type === 'Failure') {
|
|
|
1147
1181
|
}
|
|
1148
1182
|
```
|
|
1149
1183
|
|
|
1150
|
-
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object
|
|
1184
|
+
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object. A step that cannot fail, such as a Command without a `next`, adds nothing.
|
|
1151
1185
|
|
|
1152
|
-
**The union covers only the `Failure`s your steps return.** A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure`, and its error is not in the union. So where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error
|
|
1186
|
+
**The union covers only the `Failure`s your steps return.** A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure`, and its error is not in the union. So where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error. The same goes for the outcomes a settled `Parallel` hands `next`. One fallback covers the whole flow: catch inside a Command's function only what the flow can handle, and let the rest throw, which is also what lets `Retry` act on it.
|
|
1153
1187
|
|
|
1154
|
-
To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error
|
|
1188
|
+
To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error`. TypeScript refuses `instanceof Error` on an error type made only of strings, or on a flow that declares none.
|
|
1155
1189
|
|
|
1156
1190
|
To type what a Command's function returns, such as a JSON response, write the function's result type, as in `Command((): Promise<User | null> => fetchJson(url), next)`. `Command<User | null>(…)` also compiles, but with some type arguments given TypeScript infers none of the others, so that Command's errors are typed `unknown`, which hides the rest of the union.
|
|
1157
1191
|
|
|
1158
1192
|
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
1159
1193
|
|
|
1160
|
-
`Failure` keeps a literal exactly as written, but `Success` widens it, as TypeScript does for any object: `Success({ status: 'queued' })` holds a `status` of type `string`. A later step that tells values apart by such a field then refuses it, often several steps from where the value was built
|
|
1194
|
+
`Failure` keeps a literal exactly as written, but `Success` widens it, as TypeScript does for any object: `Success({ status: 'queued' })` holds a `status` of type `string`. A later step that tells values apart by such a field then refuses it, often several steps from where the value was built. So give the type there, as in `Success<Queued>({ status: 'queued', jobId })`.
|
|
1161
1195
|
|
|
1162
1196
|
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
1163
1197
|
|
|
@@ -1174,7 +1208,7 @@ const findProduct = (productId: string): Effect<Product, 'not_found', AppContext
|
|
|
1174
1208
|
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
1175
1209
|
```
|
|
1176
1210
|
|
|
1177
|
-
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object
|
|
1211
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object. It checks one passed as a variable against what the flow reads, as it checks one written in place, so a variable whose type lacks a field the flow reads does not compile. `recordEffect`, `replayEffect` and `timeTravel` check a context the same way, and `replayEffect` given a resolver requires one, since a resolver holds none.
|
|
1178
1212
|
|
|
1179
1213
|
### Generator flows
|
|
1180
1214
|
|
|
@@ -1193,7 +1227,7 @@ const registerUser = effectGen(function* (input: { email: string }) {
|
|
|
1193
1227
|
|
|
1194
1228
|
### Testing a flow in TypeScript
|
|
1195
1229
|
|
|
1196
|
-
A flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure
|
|
1230
|
+
A flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure`. So `.cmd` and `.next` compile only after the test checks that the step is a Command. `node:assert`'s `assert(step.type === 'Command')` is such a check, and TypeScript follows it. Jest's and Vitest's `expect` is not, so a walk needs a check at every step, and a small helper makes that one line per step with any test framework:
|
|
1197
1231
|
|
|
1198
1232
|
```ts
|
|
1199
1233
|
import assert from 'node:assert/strict';
|
|
@@ -1211,7 +1245,7 @@ const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
|
1211
1245
|
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
1212
1246
|
```
|
|
1213
1247
|
|
|
1214
|
-
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns
|
|
1248
|
+
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns. So where [Testing Without Mocks](#testing-without-mocks) passes `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
1215
1249
|
|
|
1216
1250
|
## Why Pure Effect
|
|
1217
1251
|
|
|
@@ -1223,25 +1257,6 @@ For a step that returns its Command directly, as in `ensureEmailAvailable(input)
|
|
|
1223
1257
|
|
|
1224
1258
|
**When to use something else:** If your code does little async I/O, or testing and debugging production are not problems for you, plain async/await is simpler.
|
|
1225
1259
|
|
|
1226
|
-
## Load Tests
|
|
1227
|
-
|
|
1228
|
-
Timings are from Node 22 on a MacBook Pro with M4 Pro CPU:
|
|
1229
|
-
|
|
1230
|
-
**A web service under load.** A `node:http` service ran signup and checkout flows, where each checkout was a `Parallel` of a retried stock reservation and a card charge, with the OpenTelemetry and recording examples installed.
|
|
1231
|
-
|
|
1232
|
-
- 800 requests at once, across 4 tenants: every response and every recorded trace belonged to its own request, and no tenant, user, or request ID crossed between runs.
|
|
1233
|
-
- The ~3,500 telemetry spans formed 800 request traces, with every span inside its own request's trace.
|
|
1234
|
-
- Side effects matched the outcomes: 320 charges and 320 orders, and 80 declined cards stopped 30 stock reservations that were still running.
|
|
1235
|
-
- 30,000 runs in a row kept memory between 10 and 11 MB, and left nothing running afterwards.
|
|
1236
|
-
- A background worker that ran 20,000 jobs in one flow, looping through a Command's `next`, finished in under ~400 ms without holding on to memory.
|
|
1237
|
-
- Stopping the server with flows still running, or clients hanging up mid-request: every flow that had started finished and was recorded, with no unhandled rejections.
|
|
1238
|
-
|
|
1239
|
-
**Batches and parallel work.**
|
|
1240
|
-
|
|
1241
|
-
- A settled `Parallel` over 200 records with `limit: 7` never had more than 7 records in flight, and returned the outcomes in order.
|
|
1242
|
-
- 5,000 branches in one `Parallel` ran in ~25 ms.
|
|
1243
|
-
- Running a Command through `runEffect` costs about 0.2 microseconds, against 0.06 for a plain `await`.
|
|
1244
|
-
|
|
1245
1260
|
## API Reference
|
|
1246
1261
|
|
|
1247
1262
|
Every `options` argument and hook configuration below throws for a name it does not read or a value it cannot use, so a misspelt option fails instead of running with its default. `Retry` and `Parallel` throw an `EffectTypeError` where they are built.
|
|
@@ -1260,7 +1275,7 @@ Returns `{ type: 'Failure', error }`. Stops the pipeline immediately.
|
|
|
1260
1275
|
|
|
1261
1276
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
1262
1277
|
|
|
1263
|
-
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch,
|
|
1278
|
+
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch, one that declares a parameter without a default value gets an `AbortSignal` that fires when a sibling fails; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1264
1279
|
- `next`: receives `cmd`'s result and returns the next Effect; by default `(result) => Success(result)`.
|
|
1265
1280
|
- `meta`: metadata passed to `onBeforeCommand`. A string `meta.name` names the Command.
|
|
1266
1281
|
|
|
@@ -1274,7 +1289,9 @@ Command(function cmdChargeCard() {
|
|
|
1274
1289
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
1275
1290
|
```
|
|
1276
1291
|
|
|
1277
|
-
Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1292
|
+
- Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1293
|
+
- Otherwise naming the function is enough. An arrow assigned to a `const` first, as in `const cmdChargeCard = () => api.charge()`, takes the `const`'s name.
|
|
1294
|
+
- An inline arrow does not. A replay cannot tell two `'anonymous'` Commands apart, so a refactor that swaps them replays without complaint, each handed the other's recorded result.
|
|
1278
1295
|
|
|
1279
1296
|
#### `commandName(command)`
|
|
1280
1297
|
|
|
@@ -1286,24 +1303,23 @@ Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nex
|
|
|
1286
1303
|
|
|
1287
1304
|
#### `Retry(effect, options?)`
|
|
1288
1305
|
|
|
1289
|
-
Returns `{ type: 'Retry', effect, options, next }`.
|
|
1290
|
-
|
|
1291
|
-
- `options.attempts`: Max retries, not counting the first try (default: `3`). Must be a positive integer; anything else, including `0`, throws. To handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
1292
|
-
- `options.delay`: Ms before the first retry (default: `100`). Must be a finite number of 0 or more.
|
|
1293
|
-
- `options.backoff`: Multiplier applied to the delay on each attempt (default: `1`, flat). Must be a finite number of 0 or more.
|
|
1294
|
-
|
|
1295
|
-
An option set to `undefined` keeps its default.
|
|
1306
|
+
Returns `{ type: 'Retry', effect, options, next }`. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
1296
1307
|
|
|
1297
|
-
- `options.
|
|
1308
|
+
- `options.retries`: tries after the first, a positive integer (default `3`), so `2` makes at most three calls. Anything else throws, `0` included: to handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
1309
|
+
- `options.delay`: milliseconds before the first retry, a finite number of 0 or more (default `100`).
|
|
1310
|
+
- `options.backoff`: the multiplier applied to the delay on each attempt, a finite number of 0 or more (default `1`, flat).
|
|
1311
|
+
- `options.onExhausted(error)`: a fallback Effect to run when every attempt has failed, given `{ retryExhausted, lastError, retries }`. Its success feeds `next`; its failure is returned unwrapped. Per use only.
|
|
1312
|
+
- An option set to `undefined` keeps its default.
|
|
1298
1313
|
|
|
1299
1314
|
#### `Parallel(effects, next?, options?)`
|
|
1300
1315
|
|
|
1301
|
-
Returns `{ type: 'Parallel', effects, next, options }
|
|
1302
|
-
|
|
1303
|
-
The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
1316
|
+
Returns `{ type: 'Parallel', effects, next, options }`. Runs the branches at the same time and hands `next` their values in order; by default `(values) => Success(values)`. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1304
1317
|
|
|
1318
|
+
- The first branch to fail cancels the others, and its `Failure` is the result, with `next` skipped. Which branch that is depends on timing, so it is recorded, and a replay returns the same one.
|
|
1319
|
+
- Each branch's Commands get an `AbortSignal` to pass to their I/O.
|
|
1320
|
+
- The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
1305
1321
|
- `limit`: the most branches running at once, a positive integer. Results and recorded paths stay in array order, so it changes only the pacing.
|
|
1306
|
-
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order
|
|
1322
|
+
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order, so no branch cancels the others or fails the `Parallel`. A bug still rejects the run: a step or `next` that throws, or returns something other than an Effect, is not a branch outcome.
|
|
1307
1323
|
|
|
1308
1324
|
### Building pipelines
|
|
1309
1325
|
|
|
@@ -1316,7 +1332,7 @@ Composes steps into a pipeline: each step gets the previous step's `Success` val
|
|
|
1316
1332
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
1317
1333
|
```
|
|
1318
1334
|
|
|
1319
|
-
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it
|
|
1335
|
+
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it. Passing the value through every step, as the [Quick Start](#quick-start) does, keeps the whole pipeline checked; see [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context).
|
|
1320
1336
|
|
|
1321
1337
|
In either language, a Command a step builds and does not return never runs, and any `Failure` it would have produced is lost:
|
|
1322
1338
|
|
|
@@ -1346,21 +1362,25 @@ Wraps an Effect for `yield*` inside `effectGen`, which gives back its value: `co
|
|
|
1346
1362
|
|
|
1347
1363
|
#### `runEffect(effect, context?, callConfig?)`
|
|
1348
1364
|
|
|
1349
|
-
Runs a flow and resolves with its `Success` or `Failure`.
|
|
1365
|
+
Runs a flow and resolves with its `Success` or `Failure`. See [Which Errors Are Data](#which-errors-are-data).
|
|
1350
1366
|
|
|
1351
1367
|
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
1352
1368
|
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the configured hooks, or used alone with `inherit: false`; see [Hooks](#hooks).
|
|
1353
|
-
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure`.
|
|
1369
|
+
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure`.
|
|
1370
|
+
- A malformed flow, or a throw from a `next` or a pure step, is a bug. The call that builds the flow throws it before the first Command, and `runEffect` rejects with it after.
|
|
1354
1371
|
|
|
1355
1372
|
#### `configureEffect(...configs)`
|
|
1356
1373
|
|
|
1357
|
-
Adds a layer of hooks for the whole process and returns a function that removes that layer only.
|
|
1374
|
+
Adds a layer of hooks for the whole process and returns a function that removes that layer only. See [Hooks](#hooks).
|
|
1358
1375
|
|
|
1359
|
-
-
|
|
1376
|
+
- Several configurations passed to one call form one layer.
|
|
1377
|
+
- `configureEffect()` with no arguments removes every layer, and a call whose arguments are all `undefined` changes nothing.
|
|
1378
|
+
- Throws a `TypeError`, before installing anything, for a configuration that is not an object, a key other than the three below (`retry` included), or a hook that is not a function (`null` included).
|
|
1379
|
+
- `onRun(effect, pipeline, flowName, initialInput)`: wraps the whole run once, retries included. It must `await pipeline()` and return its result. `initialInput` is what the flow was called with when `effectPipe` or `effectGen` built it, and `undefined` otherwise; pass it on to any hook it calls.
|
|
1360
1380
|
- `onStep(name, type, op, path)`: wraps each Command, and each `Parallel` with `name` and `type` both `'Parallel'`. It must `await op()` and return its result, and pass `path` on to any hook it calls, since a replay matches steps on it.
|
|
1361
1381
|
- For a Command, `op()` returns a promise, even for a synchronous function. Returning a value without calling `op()` answers for the Command, which is how replay works, and throwing without calling it counts as the Command failing.
|
|
1362
|
-
- A throw after `op()` succeeded, or `undefined`
|
|
1363
|
-
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }
|
|
1382
|
+
- A throw after `op()` succeeded, or `undefined` in place of its value (a forgotten `return` or `await`), is a bug in the hook: the run rejects, and `Retry` does not run the Command again.
|
|
1383
|
+
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }`. A branch that threw adds `threw: true`.
|
|
1364
1384
|
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it. A vetoed Command leaves no entry in a trace, so a replay of that run fails where it was vetoed; see [Errors no branch returned](#errors-no-branch-returned).
|
|
1365
1385
|
|
|
1366
1386
|
### Recording and replay
|
|
@@ -1370,7 +1390,7 @@ Adds a layer of hooks for the whole process and returns a function that removes
|
|
|
1370
1390
|
Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run, passed as `runEffect(flow, context, { onStep })`. Give each run its own recorder: one shared by a whole application mixes requests into one trace, which a replay refuses. `recordEffect` and the [recording example](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js) both do this for you.
|
|
1371
1391
|
|
|
1372
1392
|
- `onStep`: the hook. Recording never changes a run's outcome.
|
|
1373
|
-
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. `path
|
|
1393
|
+
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. A replay matches each on its `path`, the step's position in the flow.
|
|
1374
1394
|
|
|
1375
1395
|
```text
|
|
1376
1396
|
{ command, path, result, durationMs } a Command that returned
|
|
@@ -1381,24 +1401,29 @@ Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run,
|
|
|
1381
1401
|
because 'redact' threw on it or the 'copy' failed
|
|
1382
1402
|
```
|
|
1383
1403
|
|
|
1384
|
-
-
|
|
1404
|
+
- An older entry with `error` and no `threw` still counts as a throw.
|
|
1405
|
+
- `toTrace(meta)`: the entries as a trace, with `meta`'s `initialInput`, `context`, `flowName` and `version`, copied into the form JSON stores when it is called. If a Command can change the input or the context, as an ORM save that adds an id does, call it before the run as well, as `recordEffect` does.
|
|
1385
1406
|
- `options.redact(value, name, kind)`: returns what the trace stores in place of `value`. `kind` is `'result'`, `'error'`, `'initialInput'` or `'context'`, and `name` is the Command's name, or the kind. See [Keeping secrets out of a trace](#keeping-secrets-out-of-a-trace).
|
|
1386
1407
|
- `options.maxEntries`: the most entries a trace keeps; the rest are counted in `dropped`, and a replay stops at the first step the trace lacks.
|
|
1387
1408
|
- `options.stack`: record stack traces for thrown errors (off by default).
|
|
1388
|
-
- A value `redact` throws on, or one JSON cannot hold, is left out and marked `unrecorded
|
|
1409
|
+
- A value `redact` throws on, or one JSON cannot hold, is left out and marked `unrecorded`: on its entry, or for the input and context in the trace's `unrecorded`, as in `{ context: 'copy' }`. A replay stops there with a `ReplayError` rather than hand the flow a stand-in. See [Recording in Production](#recording-in-production) for what a copy keeps.
|
|
1389
1410
|
|
|
1390
|
-
#### `recordEffect(
|
|
1411
|
+
#### `recordEffect(effect, options?)`
|
|
1391
1412
|
|
|
1392
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts
|
|
1413
|
+
Runs a flow for real while recording, returning `{ result, trace }`. The trace holds the flow's input when `effectPipe` or `effectGen` built it, so `timeTravel` can build the flow again. Accepts `recorder` options plus `context` and `version`. For tests and scripts; to record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
1393
1414
|
|
|
1394
1415
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
1395
1416
|
|
|
1396
|
-
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for.
|
|
1417
|
+
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for. See [Time-Travel Debugging](#time-travel-debugging).
|
|
1397
1418
|
|
|
1398
|
-
-
|
|
1399
|
-
- A
|
|
1400
|
-
- `
|
|
1401
|
-
-
|
|
1419
|
+
- A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`.
|
|
1420
|
+
- A malformed trace rejects with a `ReplayError`.
|
|
1421
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver: a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of. See [Replaying from your own storage](#replaying-from-your-own-storage).
|
|
1422
|
+
- Any other answer from a resolver, `null` included, or a throw from it, rejects the replay. With a resolver, `unreached` is absent.
|
|
1423
|
+
- A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`. Answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1424
|
+
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`; a trace whose context could not be recorded refuses to replay without one.
|
|
1425
|
+
- `options.onMissing`: `'throw'` (default) stops at a step the trace lacks; `'execute'` runs its Command for real, so use it only where Commands reach test doubles or only read.
|
|
1426
|
+
- A trace that dropped entries refuses `'execute'`. A flow that changed shape, as when a Command is newly wrapped in `Retry` or moved into a new `Parallel` branch, stops at a `TimeParadox` even under `'execute'`.
|
|
1402
1427
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
1403
1428
|
- `options.hooks` (default `false`): run the replay inside the configured hooks, a configured recorder included, so they see the replayed steps. Off, a replay cannot reach a telemetry backend or a trace sink.
|
|
1404
1429
|
- `options.onResolved(step, outcome)`: observes each replayed step. A throw stops the replay, and `replayEffect` rejects with it; it never changes an outcome.
|
|
@@ -1406,19 +1431,23 @@ Runs a flow with each Command answered from a trace instead of run, and returns
|
|
|
1406
1431
|
|
|
1407
1432
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
1408
1433
|
|
|
1409
|
-
Replays a trace and narrates each step with its recorded duration
|
|
1434
|
+
Replays a trace and narrates each step with its recorded duration. Returns `{ result, unreached }`, as `replayEffect` does for a trace.
|
|
1435
|
+
|
|
1436
|
+
- Names any recorded steps that were never reached.
|
|
1437
|
+
- Warns when the trace's `version` differs from `options.version`, and when steps are named `'anonymous'`.
|
|
1438
|
+
- Refuses a trace whose input could not be recorded, since the flow is rebuilt from it.
|
|
1439
|
+
- `options.context` overrides the trace's; `options.log` replaces `console.log`.
|
|
1410
1440
|
|
|
1411
1441
|
## Limitations
|
|
1412
1442
|
|
|
1413
|
-
- **`Retry` repeats everything it wraps, including a Command's `next`.**
|
|
1414
|
-
- **Cancelling a `Parallel` branch cannot stop
|
|
1415
|
-
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.**
|
|
1416
|
-
- **`redact` covers a trace, not a result.** A `Failure` holds the full error,
|
|
1417
|
-
- **Replay checks the path, not the values.** Every
|
|
1418
|
-
- **A trace keeps data, not objects.** A
|
|
1419
|
-
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two
|
|
1420
|
-
- **Configuration belongs to one copy of the library.**
|
|
1421
|
-
- **A Command needs a name.** `meta.name`
|
|
1443
|
+
- **`Retry` repeats everything it wraps, including a Command's `next`.** Commands that already succeeded run again, and a retried Command whose `next` continues the flow repeats the rest of the flow with it. Wrap only the Command that fails, and continue in a later step; see [Retrying Transient Failures](#retrying-transient-failures). A function that catches its own error is never retried; see [Which Errors Are Data](#which-errors-are-data).
|
|
1444
|
+
- **Cancelling a `Parallel` branch cannot stop a call already running.** A cancelled branch starts no new Commands, but a function that ignores its `AbortSignal`, or is never given one, runs to the end. So a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch before it returns; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1445
|
+
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.** So `Command(listUsers)`, where `listUsers(page)` treats `page` as optional, gets the signal as its page inside a `Parallel` and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1446
|
+
- **`redact` covers a trace, not a result.** A `Failure` holds the full error, and an error can quote what a Command sent, as a database's duplicate-key error quotes the key. Keeping it out of your logs is up to you.
|
|
1447
|
+
- **Replay checks the path, not the values.** Every result comes from the recording, so a replay cannot check a fix that changes what a flow computes. A step removed from the middle of a flow stops the replay at a `TimeParadox`; see [What a replay cannot check](#what-a-replay-cannot-check).
|
|
1448
|
+
- **A trace keeps data, not objects.** A replayed value is what JSON stores: a class instance loses its methods, a `Date` becomes a string, and a thrown error comes back as a plain `Error`, so check `error.name` rather than `instanceof`. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|
|
1449
|
+
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two requests does not, because a trace records one flow. Nor does a race between branches that share state: branches replay with the results they saw, not the order they ran in. A cancelled generator branch can stop one step later in a replay; see [Replaying a cancelled `Parallel`](#replaying-a-cancelled-parallel).
|
|
1450
|
+
- **Configuration belongs to one copy of the library.** Hooks live in module state, so two copies in one process (a dual ESM and CJS setup, two versions in the dependency tree, or a worker thread) do not share them, and nothing warns you.
|
|
1451
|
+
- **A Command needs a name that survives minification.** `meta.name` does. `cmd.name` does not, unless the minifier keeps function names.
|
|
1422
1452
|
- **A generator flow's body can run more than once,** and code in it that changes something outside it, such as a counter or a log line, does so again; see [What runs once, and what runs again](#what-runs-once-and-what-runs-again).
|
|
1423
|
-
- **A generator
|
|
1424
|
-
- **A generator body's `finally` does not always run.** It does not run when a bug or a `TimeParadox` rejects the run. In a `Parallel` where one branch threw, another branch that stops for its own reason before the rejection reaches it may still run its `catch` and `finally`, depending on timing. A step in a body's `finally` is not supported: it runs when the body ends normally and is refused when a step stopped the body, but if an error is passing through the body when its `finally` reaches a step, and the flow is cancelled before that step starts, the error is lost. Put cleanup in a `try`/`finally` around `runEffect`.
|
|
1453
|
+
- **A generator body's `finally` does not always run.** It does not run when a bug or a `TimeParadox` rejects the run. Put cleanup in a `try`/`finally` around `runEffect`; see [How a generator flow stops](#how-a-generator-flow-stops).
|