pure-effect 0.19.0 → 0.20.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -1
- package/README.md +959 -657
- package/RULES.md +43 -0
- package/examples/recording-example.js +6 -3
- package/index.d.ts +29 -20
- package/index.js +153 -58
- package/package.json +4 -3
package/README.md
CHANGED
|
@@ -6,20 +6,20 @@
|
|
|
6
6
|
|
|
7
7
|
- Replay a production failure locally, with no database and no network
|
|
8
8
|
- No mocks needed to test async pipelines
|
|
9
|
-
-
|
|
9
|
+
- Read request values, such as the tenant, from any step with `Ask`, without passing them along
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
|
-
- OpenTelemetry-ready via lifecycle hooks
|
|
11
|
+
- OpenTelemetry-ready via lifecycle hooks, with a reference integration to copy
|
|
12
12
|
- Zero dependencies, about 11 KB minified and gzipped
|
|
13
|
-
- Works in JavaScript, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
13
|
+
- Works in JavaScript on Node 22 or later, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
16
16
|
|
|
17
|
-
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [
|
|
18
|
-
- **
|
|
19
|
-
- **
|
|
20
|
-
- **Reference:** [Why Pure Effect](#why-pure-effect) · [
|
|
17
|
+
- **Getting started:** [Installation](#installation) · [The Core](#the-core) · [Quick Start](#quick-start) · [Coming from async/await](#coming-from-asyncawait) · [Writing Flows as Generators](#writing-flows-as-generators) · [Testing Without Mocks](#testing-without-mocks) · [Rules for Writing a Flow](#rules-for-writing-a-flow)
|
|
18
|
+
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Writing Loops](#writing-loops) · [Which Errors Are Data](#which-errors-are-data) · [Generator Flows in Depth](#generator-flows-in-depth) · [Parallel in Depth](#parallel-in-depth) · [TypeScript](#typescript-typed-errors-and-context)
|
|
19
|
+
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production) · [Hooks](#hooks) · [Replay in Depth](#replay-in-depth)
|
|
20
|
+
- **Reference:** [Why Pure Effect](#why-pure-effect) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
21
|
|
|
22
|
-
The
|
|
22
|
+
The seven sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: retries, loops, steps that run at the same time, TypeScript, or recording.
|
|
23
23
|
|
|
24
24
|
## Installation
|
|
25
25
|
|
|
@@ -27,22 +27,30 @@ The six sections under Getting started are enough to write and test a flow. Come
|
|
|
27
27
|
npm install pure-effect
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
+
## The Core
|
|
31
|
+
|
|
32
|
+
Every flow is built from these:
|
|
33
|
+
|
|
34
|
+
| Piece | What it is |
|
|
35
|
+
| --- | --- |
|
|
36
|
+
| `Success(value)` | A step that succeeded, carrying its value on to the next step. |
|
|
37
|
+
| `Failure(error)` | A step that stops the flow, carrying its error to the caller. |
|
|
38
|
+
| `Command(cmd, next)` | A call to make, and a function that turns its answer into the next step. |
|
|
39
|
+
| `effectPipe(...)`, `effectGen(...)` | Join steps into a flow: as a pipeline, or written like `async` code with `yield* step(...)`. |
|
|
40
|
+
| `runEffect(flow)` | Runs the flow, the only place the calls are made, and returns a `Success` or a `Failure`. |
|
|
41
|
+
|
|
42
|
+
Everything else adds to these. `Ask` reads request values, `Retry` and `Parallel` wrap steps, and recording and replay watch `runEffect` make its calls.
|
|
43
|
+
|
|
30
44
|
## Quick Start
|
|
31
45
|
|
|
32
|
-
|
|
46
|
+
Five steps take you from a plain `async` function to a tested flow. The example registers a user.
|
|
33
47
|
|
|
34
|
-
|
|
35
|
-
import { Success, Failure, Command, effectPipe, runEffect } from 'pure-effect';
|
|
48
|
+
**1. Describe each call as a Command.** A Command holds a call to make, and a `next` function that turns its answer into the next step. Building one makes no call.
|
|
36
49
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
if (!input.email.includes('@')) return Failure('Invalid email.');
|
|
40
|
-
if (input.password.length < 8) return Failure('Password too short.');
|
|
41
|
-
return Success(input);
|
|
42
|
-
};
|
|
50
|
+
```js
|
|
51
|
+
import { Success, Failure, Command, Retry, effectPipe, effectGen, step, runEffect } from 'pure-effect';
|
|
43
52
|
|
|
44
|
-
//
|
|
45
|
-
// Name the commands: traces, replay matching, and telemetry spans use the names.
|
|
53
|
+
// Name each Command's function: traces, replay and telemetry refer to the Command by it.
|
|
46
54
|
const ensureEmailAvailable = (input) => {
|
|
47
55
|
const cmdFindUser = () => db.findUser(input.email);
|
|
48
56
|
const next = (found) => (found ? Failure('Email already in use.') : Success(input));
|
|
@@ -54,27 +62,27 @@ const saveUser = (input) => {
|
|
|
54
62
|
const cmdSaveUser = () => db.saveUser(input);
|
|
55
63
|
return Command(cmdSaveUser);
|
|
56
64
|
};
|
|
65
|
+
```
|
|
57
66
|
|
|
58
|
-
|
|
59
|
-
const registerUserFlow = (input) => effectPipe(validateRegistration, ensureEmailAvailable, saveUser)(input);
|
|
67
|
+
Keep a Command's function to the one call. Tests never run it, so nothing else in it gets tested.
|
|
60
68
|
|
|
61
|
-
|
|
62
|
-
async function registerUser(input) {
|
|
63
|
-
const result = await runEffect(registerUserFlow(input));
|
|
69
|
+
**2. Join the steps into a flow.** A step with no I/O returns `Success` or `Failure` itself. `effectPipe` hands each step the value the step before it produced.
|
|
64
70
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
+
```js
|
|
72
|
+
// Pure. No I/O, instantly testable.
|
|
73
|
+
const validateRegistration = (input) => {
|
|
74
|
+
if (!input.email.includes('@')) return Failure('Invalid email.');
|
|
75
|
+
if (input.password.length < 8) return Failure('Password too short.');
|
|
76
|
+
return Success(input);
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
// Validation runs as the flow is built. No I/O happens until runEffect.
|
|
80
|
+
const registerUserFlow = (input) => effectPipe(validateRegistration, ensureEmailAvailable, saveUser)(input);
|
|
71
81
|
```
|
|
72
82
|
|
|
73
83
|
The same flow can be written as a generator, with `yield* step(...)` where `async` code writes `await`. [Writing Flows as Generators](#writing-flows-as-generators) covers what to know.
|
|
74
84
|
|
|
75
85
|
```js
|
|
76
|
-
import { effectGen, step } from 'pure-effect';
|
|
77
|
-
|
|
78
86
|
const registerUserFlowGen = effectGen(function* (input) {
|
|
79
87
|
yield* step(validateRegistration(input));
|
|
80
88
|
yield* step(ensureEmailAvailable(input));
|
|
@@ -82,103 +90,72 @@ const registerUserFlowGen = effectGen(function* (input) {
|
|
|
82
90
|
});
|
|
83
91
|
```
|
|
84
92
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop. A generator flow builds the same Commands: `effectGen` runs your function up to each `yield*` and continues it with the answer.
|
|
88
|
-
|
|
89
|
-
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
90
|
-
|
|
91
|
-
Building a flow runs the pure steps right away: `registerUserFlow(badInput)` returns the validation `Failure` synchronously, so the tests below do not need `runEffect`. Only Commands need it.
|
|
93
|
+
**3. Return a `Failure`, and throw only when trying again could help.** The two ways to fail do different things:
|
|
92
94
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
A flow is plain objects, so a test can check what it will do without running it. Validation runs as soon as the flow is built, so testing it needs nothing else:
|
|
95
|
+
- An outcome the flow decides, such as an invalid email or one already in use, is a `Failure` a step returns. The flow stops there, and nothing retries it.
|
|
96
|
+
- A call that broke, such as a timeout or a dropped connection, is an error the Command's function throws. `Retry` makes the call again:
|
|
96
97
|
|
|
97
98
|
```js
|
|
98
|
-
|
|
99
|
-
|
|
99
|
+
// Up to three calls. Wrap only the Command: Retry runs everything it wraps again.
|
|
100
|
+
const saveUserWithRetry = (input) => Retry(saveUser(input), { retries: 2 });
|
|
100
101
|
```
|
|
101
102
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
```js
|
|
105
|
-
const input = { email: 'test@test.com', password: 'password123' };
|
|
106
|
-
|
|
107
|
-
// The email is free, so the step passes the input on.
|
|
108
|
-
assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
103
|
+
Either way, the caller gets a `Failure` once the flow gives up. A call that throws outside a `Retry` ends the run in a `Failure` holding the error it threw.
|
|
109
104
|
|
|
110
|
-
|
|
111
|
-
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
112
|
-
```
|
|
105
|
+
After a `Retry` gives up, the `Failure` holds `{ retryExhausted, lastError, retries }`, with the last error thrown as `lastError`. When a call throws an error the flow should decide on, such as a declined card, catch it inside the Command's function and return it; [Which Errors Are Data](#which-errors-are-data) shows how.
|
|
113
106
|
|
|
114
|
-
|
|
107
|
+
**4. Test it without mocks.** A flow is plain objects. Hand a step the answer you choose and check what it returns:
|
|
115
108
|
|
|
116
109
|
```js
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
assert.equal(commandName(step1), 'cmdFindUser');
|
|
121
|
-
|
|
122
|
-
const step2 = step1.next(null); // pretend no user was found
|
|
123
|
-
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
124
|
-
// The database was never touched.
|
|
110
|
+
// The email is taken, so the flow stops here. No database was touched.
|
|
111
|
+
const taken = ensureEmailAvailable({ email: 'taken@test.com', password: 'password123' }).next({ id: 1 });
|
|
112
|
+
assert.deepEqual(taken, Failure('Email already in use.'));
|
|
125
113
|
```
|
|
126
114
|
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
130
|
-
|
|
131
|
-
To run the whole flow instead of walking it, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command by name. Nothing is called and no retry waits; answering `{ error }` makes that Command throw, which is how a test reaches a `Retry`'s fallback:
|
|
132
|
-
|
|
133
|
-
```js
|
|
134
|
-
import { replayEffect } from 'pure-effect';
|
|
135
|
-
|
|
136
|
-
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
137
|
-
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
138
|
-
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
139
|
-
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
140
|
-
```
|
|
115
|
+
[Testing Without Mocks](#testing-without-mocks) shows each kind of test, and how to run a whole flow with no I/O.
|
|
141
116
|
|
|
142
|
-
|
|
117
|
+
**5. Run it.** `runEffect` makes the calls and returns a `Success` or a `Failure`. It is the only place a call is made, so it sits at the edge of your app:
|
|
143
118
|
|
|
144
119
|
```js
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
// A real run returns its result, and a trace of what each Command returned.
|
|
148
|
-
const { result: live, trace } = await recordEffect(registerUserFlow, input);
|
|
120
|
+
async function registerUser(input) {
|
|
121
|
+
const result = await runEffect(registerUserFlow(input));
|
|
149
122
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
123
|
+
if (result.type === 'Success') {
|
|
124
|
+
console.log('User created:', result.value);
|
|
125
|
+
} else {
|
|
126
|
+
console.error('Error:', result.error);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
153
129
|
```
|
|
154
130
|
|
|
155
|
-
[Time-Travel Debugging](#time-travel-debugging)
|
|
156
|
-
|
|
157
|
-
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them, with the services passed in as [Passing Runtime Context](#passing-runtime-context) shows.
|
|
158
|
-
|
|
159
|
-
In TypeScript, a walk needs a check at each step that the step is a Command; [Testing a flow in TypeScript](#testing-a-flow-in-typescript) has a helper that makes that one line.
|
|
131
|
+
That is a whole flow. [Rules for Writing a Flow](#rules-for-writing-a-flow) lists the twelve rules that cover most of what goes wrong, and [Time-Travel Debugging](#time-travel-debugging) shows how to record a run in production and replay it on your machine. In TypeScript, [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context) covers typing a flow's errors and context, handling every error, and testing a flow.
|
|
160
132
|
|
|
161
133
|
## Coming from async/await
|
|
162
134
|
|
|
163
|
-
A flow does what `async`/`await` code does, as data
|
|
135
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command: which call to make, and what to do with the answer. Each thing you would write with `async`/`await` has an equivalent.
|
|
136
|
+
|
|
137
|
+
Where the two styles differ:
|
|
164
138
|
|
|
165
139
|
| `async`/`await` | Generator flow | Pipeline |
|
|
166
140
|
| --- | --- | --- |
|
|
167
|
-
| `await db.findUser(email)` | `yield* step(findUser(email))` | `Command(
|
|
141
|
+
| `await db.findUser(email)` | `yield* step(findUser(email))` | `Command(cmdFindUser, next)` |
|
|
168
142
|
| the code after the `await` | the next `yield*` | the Command's `next` |
|
|
169
143
|
| an `if` on a result | a step that returns `Success` or `Failure` | a branch in `next` |
|
|
170
144
|
| `return value` | `return value` | `Success(value)` |
|
|
171
145
|
| `throw` to stop | a step that returns `Failure(error)` | `Failure(error)` |
|
|
172
|
-
| `try`/`catch` around a call | a catch inside the Command's function | the same |
|
|
173
|
-
| a loop that retries a call | `yield* step(Retry(...))` | `Retry(...)` |
|
|
174
|
-
| `await Promise.all([...])` | `yield* step(Parallel([...]))` | `Parallel([...])` |
|
|
175
146
|
| a `for` loop with an `await` inside | the same loop, with `yield*` | `Parallel(list.map(...), { limit: 1 })` |
|
|
176
|
-
| a loop that keeps going past a failure | `Parallel(..., { settled: true })` | the same |
|
|
177
147
|
| a call that needs two earlier results | two variables | a named step that passes both on |
|
|
178
|
-
| a value from the request | `yield* step(Ask())` | `Ask` |
|
|
179
|
-
| `Date.now()` or a random ID | a Command | a Command |
|
|
180
148
|
|
|
181
|
-
|
|
149
|
+
The rest are the same in both styles, and a generator flow runs each one with `yield* step(...)`:
|
|
150
|
+
|
|
151
|
+
- `try`/`catch` around a call: a catch inside the Command's function
|
|
152
|
+
- a loop that retries a call: `Retry`
|
|
153
|
+
- `await Promise.all([...])`: `Parallel`
|
|
154
|
+
- a loop that keeps going past a failure: `Parallel` with `settled: true`
|
|
155
|
+
- a value from the request: `Ask`
|
|
156
|
+
- `Date.now()` or a random ID: a Command
|
|
157
|
+
|
|
158
|
+
`cmdFindUser` is the named function `() => db.findUser(email)`, and `findUser` stands for a step that returns the Command, as `ensureEmailAvailable` does in the Quick Start. A `catch` around `yield*` can log and rethrow, but it cannot recover; see [Which Errors Are Data](#which-errors-are-data).
|
|
182
159
|
|
|
183
160
|
One rule is behind both columns: each decision lives in a small step function that receives the answer it depends on and returns `Success` or `Failure`, so a test can call it directly. A pipeline joins those steps with `effectPipe`; a generator flow puts them in order with `yield*`. An `if` around a `yield*` works, but it can be tested only by walking the whole flow, so keep the body to putting steps in order. A `for` loop that runs a step for each item decides nothing, so it is fine.
|
|
184
161
|
|
|
@@ -193,30 +170,9 @@ async function registerUserAsync(input) {
|
|
|
193
170
|
}
|
|
194
171
|
```
|
|
195
172
|
|
|
196
|
-
The difference shows in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace
|
|
197
|
-
|
|
198
|
-
```js
|
|
199
|
-
import { mock } from 'node:test';
|
|
200
|
-
|
|
201
|
-
const input = { email: 'test@test.com', password: 'password123' };
|
|
202
|
-
|
|
203
|
-
mock.method(db, 'findUser', async () => ({ id: 1 }));
|
|
204
|
-
const saveUser = mock.method(db, 'saveUser', async (user) => user);
|
|
205
|
-
|
|
206
|
-
await assert.rejects(registerUserAsync(input), { message: 'Email already in use.' });
|
|
207
|
-
assert.equal(saveUser.mock.callCount(), 0);
|
|
208
|
-
mock.restoreAll();
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
The flow's test of the same branch hands one step the answer, with nothing to replace, wait for, or put back:
|
|
212
|
-
|
|
213
|
-
```js
|
|
214
|
-
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
Each step is a place a test can start, so a branch is tested by calling the step it lives in, where a test of an `async` function runs everything before the branch, with every call on the way replaced. The mocks do check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
173
|
+
The difference shows in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace `db.findUser` and `db.saveUser` with mocks, run the function, and put them back. The flow's test hands the one step the answer, `ensureEmailAvailable(input).next({ id: 1 })`, with nothing to replace or put back. The mocks do check one thing that test cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
218
174
|
|
|
219
|
-
You can switch one function at a time
|
|
175
|
+
You can switch one function at a time. Keep its name and parameters, run the flow in its body, and turn a `Failure` back into a throw, so the code that calls it does not change:
|
|
220
176
|
|
|
221
177
|
```js
|
|
222
178
|
// Callers still await registerUserAsync and catch what it throws.
|
|
@@ -230,11 +186,13 @@ async function registerUserAsync(input) {
|
|
|
230
186
|
await assert.rejects(registerUserAsync({ email: 'bad-email', password: '123' }), { message: 'Invalid email.' });
|
|
231
187
|
```
|
|
232
188
|
|
|
189
|
+
If the flow has a `Retry`, a `Failure` from one that gave up holds `{ retryExhausted, lastError, retries }`; throw its `lastError` to rethrow what the call threw, as the old function did.
|
|
190
|
+
|
|
233
191
|
A loop becomes a step that gets the list and a `Parallel` over it, as the table shows; [Writing Loops](#writing-loops) translates one.
|
|
234
192
|
|
|
235
193
|
## Writing Flows as Generators
|
|
236
194
|
|
|
237
|
-
A generator flow is written like `async` code, with `yield* step(...)` where `async` code writes `await`. `effectGen` turns it into the same Commands a pipeline builds, so recording, replay, `Retry`, `Parallel` and the tests
|
|
195
|
+
A generator flow is written like `async` code, with `yield* step(...)` where `async` code writes `await`. `effectGen` turns it into the same Commands a pipeline builds, so recording, replay, `Retry`, `Parallel` and the tests in [Testing Without Mocks](#testing-without-mocks) work the same for both. The Quick Start's `registerUserFlowGen` is one. The same steps in the same order record the same paths in either form, so a trace recorded from a pipeline replays against its generator version. A loop written as a `for` loop rather than a `Parallel` records differently; see [Writing Loops](#writing-loops).
|
|
238
196
|
|
|
239
197
|
### Where a generator helps
|
|
240
198
|
|
|
@@ -259,328 +217,132 @@ assert.equal(onboarded.value.workspace.ownerId, onboarded.value.user.id);
|
|
|
259
217
|
|
|
260
218
|
A pipeline would need a step that passes the user and the workspace on together; see [Composing Larger Flows](#composing-larger-flows). Keep decisions in step functions all the same, as `registerUserFlowGen` does, so each can be tested on its own.
|
|
261
219
|
|
|
262
|
-
The body can run more than once:
|
|
220
|
+
The body can run more than once: for each `Retry` attempt and each run, as an `async` function does when called again, and when a test walks a second branch. So keep it to its own variables, and get `Date.now()` or a random ID from a Command, so a replay gets the recorded one. [Generator Flows in Depth](#generator-flows-in-depth) covers when it runs again, how a `catch` or `finally` in it behaves when a step stops the flow, and the mistakes `effectGen` catches.
|
|
263
221
|
|
|
264
|
-
|
|
222
|
+
## Testing Without Mocks
|
|
265
223
|
|
|
266
|
-
|
|
224
|
+
A flow is plain objects, so a test can check what it will do without running it. Validation runs as soon as the flow is built, so testing it needs nothing else:
|
|
267
225
|
|
|
268
226
|
```js
|
|
269
|
-
const
|
|
270
|
-
|
|
271
|
-
const { result } = await replayEffect(registerUserFlowGen(input), answerByName);
|
|
272
|
-
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
227
|
+
const badInput = { email: 'bad-email', password: '123' };
|
|
228
|
+
assert.deepEqual(registerUserFlow(badInput), Failure('Invalid email.'));
|
|
273
229
|
```
|
|
274
230
|
|
|
275
|
-
|
|
231
|
+
Next, test the steps. Hand a step an answer and check what it returns:
|
|
232
|
+
|
|
233
|
+
```js
|
|
234
|
+
const input = { email: 'test@test.com', password: 'password123' };
|
|
276
235
|
|
|
277
|
-
|
|
236
|
+
// The email is free, so the step passes the input on.
|
|
237
|
+
assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
278
238
|
|
|
279
|
-
|
|
239
|
+
// The email is taken, so the flow stops here.
|
|
240
|
+
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Then test the flow: the right calls, in the right order. Handing each Command's `next` the answer you choose, and checking the step it returns, is called walking the flow; nothing runs. `commandName` gives the name a Command is recorded under:
|
|
280
244
|
|
|
281
245
|
```js
|
|
282
|
-
import {
|
|
246
|
+
import { commandName } from 'pure-effect';
|
|
283
247
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
version: process.env.BUILD_ID,
|
|
287
|
-
context: { flowName: 'checkout' }
|
|
288
|
-
});
|
|
248
|
+
const step1 = registerUserFlow(input);
|
|
249
|
+
assert.equal(commandName(step1), 'cmdFindUser');
|
|
289
250
|
|
|
290
|
-
//
|
|
291
|
-
|
|
251
|
+
const step2 = step1.next(null); // pretend no user was found
|
|
252
|
+
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
253
|
+
// The database was never touched.
|
|
292
254
|
```
|
|
293
255
|
|
|
294
|
-
|
|
295
|
-
Replaying 'checkout' (3 recorded steps)
|
|
296
|
-
Initial input: {
|
|
297
|
-
"cartId": "cart_abc123",
|
|
298
|
-
"promoCode": "FREE_YEAR_VIP"
|
|
299
|
-
}
|
|
300
|
-
Step 1: cmdFetchCart returned in 41.277ms {
|
|
301
|
-
"totalAmount": "120.00"
|
|
302
|
-
}
|
|
303
|
-
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
304
|
-
"isValid": true,
|
|
305
|
-
"discountValue": 100
|
|
306
|
-
}
|
|
307
|
-
Step 3: cmdChargeCard threw in 181.507ms {
|
|
308
|
-
"code": "invalid_amount",
|
|
309
|
-
"name": "Error",
|
|
310
|
-
"message": "Amount must be non-zero."
|
|
311
|
-
}
|
|
312
|
-
Replay finished with state: Failure
|
|
313
|
-
Error: {
|
|
314
|
-
"code": "invalid_amount",
|
|
315
|
-
"name": "Error",
|
|
316
|
-
"message": "Amount must be non-zero."
|
|
317
|
-
}
|
|
318
|
-
```
|
|
256
|
+
Write both kinds: only a step test sees the value passed from one step to the next. Suppose the email guard returned `Success(true)` instead of `Success(input)`. The step test would fail. The flow test would still pass, since the calls and their order are unchanged, while the flow saved `true` instead of the user.
|
|
319
257
|
|
|
320
|
-
A
|
|
258
|
+
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
321
259
|
|
|
322
|
-
|
|
323
|
-
it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
324
|
-
const { result } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
325
|
-
assert.equal(result.type, 'Failure');
|
|
326
|
-
assert.equal(result.error.code, 'invalid_amount');
|
|
327
|
-
});
|
|
328
|
-
```
|
|
260
|
+
A generator flow is tested the same way. Its decisions are in step functions, which get the step tests, and a walk or a whole-flow run works on `registerUserFlowGen` as it does on `registerUserFlow`. A walk straight through runs the body once, and walking a second branch runs it again.
|
|
329
261
|
|
|
330
|
-
|
|
262
|
+
[Testing a whole flow](#testing-a-whole-flow) shows how to run a whole flow in a test instead, `Retry` and `Parallel` included, with no I/O.
|
|
331
263
|
|
|
332
|
-
|
|
264
|
+
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them, with the services passed in as [Passing Runtime Context](#passing-runtime-context) shows.
|
|
333
265
|
|
|
334
|
-
|
|
335
|
-
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
336
|
-
const { result, unreached } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
337
|
-
assert.equal(result.type, 'Success');
|
|
338
|
-
assert.deepEqual(
|
|
339
|
-
unreached.map((e) => e.command),
|
|
340
|
-
['cmdChargeCard']
|
|
341
|
-
);
|
|
342
|
-
});
|
|
343
|
-
```
|
|
266
|
+
In TypeScript, a walk needs a check at each step that the step is a Command; [Testing a flow in TypeScript](#testing-a-flow-in-typescript) has a helper that makes that one line.
|
|
344
267
|
|
|
345
|
-
|
|
268
|
+
### Testing a whole flow
|
|
346
269
|
|
|
347
|
-
|
|
270
|
+
To run a whole flow in a test instead of walking it, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command. Nothing is called and no retry waits. The function gets each step the flow asks for, with its `name`, and returns `{ result }` for the Command to return, `{ error }` for it to throw, or `undefined` for a step it has no answer for, which ends the replay in a `Failure` holding a `ReplayError`:
|
|
348
271
|
|
|
349
|
-
|
|
272
|
+
```js
|
|
273
|
+
import { replayEffect } from 'pure-effect';
|
|
350
274
|
|
|
351
|
-
|
|
352
|
-
{
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
"initialInput": { "cartId": "cart_1" },
|
|
356
|
-
"context": { "flowName": "checkout" },
|
|
357
|
-
"dropped": 0,
|
|
358
|
-
"trace": [
|
|
359
|
-
{ "command": "cmdFetchCart", "path": "0", "result": { "total": 120 }, "durationMs": 41.3 },
|
|
360
|
-
{ "command": "cmdChargeCard", "path": "1r0/0", "threw": true,
|
|
361
|
-
"error": { "__error": true, "name": "Error", "message": "gateway timeout", "code": "ETIMEDOUT" }, "durationMs": 5002.1 },
|
|
362
|
-
{ "command": "cmdChargeCard", "path": "1r1/0", "result": { "chargeId": "ch_1" }, "durationMs": 181.5 }
|
|
363
|
-
]
|
|
364
|
-
}
|
|
275
|
+
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
276
|
+
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
277
|
+
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
278
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
365
279
|
```
|
|
366
280
|
|
|
367
|
-
`
|
|
281
|
+
Answering `{ error }` is how a test reaches a `Retry`'s next attempt or its `onExhausted` fallback. An answer takes the place of the Command's function, its `catch` included: for a function that catches an error and returns it, answer with what it returns, not `{ error }`. To answer each attempt differently, give a name a list of answers and hand them out in turn. Here the Quick Start's `saveUserWithRetry` fails once, then goes through:
|
|
368
282
|
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
283
|
+
```js
|
|
284
|
+
const answers = { cmdSaveUser: [{ error: new Error('connection reset') }, { result: { id: 1, ...input } }] };
|
|
285
|
+
const inTurn = (step) => answers[step.name]?.shift();
|
|
286
|
+
const { result } = await replayEffect(saveUserWithRetry(input), inTurn);
|
|
287
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
288
|
+
```
|
|
373
289
|
|
|
374
|
-
|
|
290
|
+
The function is also asked about each `Parallel`, under the name `'Parallel'`; answering `undefined` lets its branches run as written. A step also carries its `path`, its position in the flow, which [Reading a trace](#reading-a-trace) explains; answer by path to tell apart two Commands with the same name.
|
|
375
291
|
|
|
376
|
-
|
|
292
|
+
## Rules for Writing a Flow
|
|
377
293
|
|
|
378
|
-
|
|
379
|
-
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix: catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
380
|
-
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
294
|
+
Twelve rules cover most of what goes wrong. The same rules ship in the package as `RULES.md`, with an index of where each task is covered.
|
|
381
295
|
|
|
382
|
-
|
|
296
|
+
<!-- rules:start -->
|
|
383
297
|
|
|
384
|
-
|
|
298
|
+
1. **Put every call that does I/O inside a Command, and keep its function to that one call:** `Command(() => db.findUser(email))`, never `step(db.findUser(email))`. Tests never run the function, so anything else in it goes untested. See [Testing Without Mocks](#testing-without-mocks).
|
|
299
|
+
2. **Name every Command,** with a named function such as `const cmdFindUser = () => ...`, or with `meta.name`. Traces, replay and telemetry use the name, and a minifier can strip a function's name. See [`Command`](#commandcmdfn-nextfn-meta).
|
|
300
|
+
3. **Put each decision in a step function** that receives one value and returns `Success`, `Failure` or the next Command, so a test can call it directly. See [Coming from async/await](#coming-from-asyncawait).
|
|
301
|
+
4. **Return an outcome the caller should see as `Failure(error)`; never throw it.** A `Failure` stops the flow, is never retried, and only the code that calls `runEffect` acts on it. See [Which Errors Are Data](#which-errors-are-data).
|
|
302
|
+
5. **A `catch` in a generator flow cannot recover.** It can log and rethrow, and it sees a `FlowStopped` error saying where the flow stopped, not the original. To handle an error, catch it inside the Command's function and return it as data. See [How a generator flow stops](#how-a-generator-flow-stops).
|
|
303
|
+
6. **Let a Command's function throw only when `Retry` should try again.** An error it catches and returns is data, and `Retry` never sees it. To retry some errors and not others, rethrow the first kind and return the second. A throw also skips any undo the Command's `next` would have run. See [Which Errors Are Data](#which-errors-are-data).
|
|
304
|
+
7. **Wrap only the Command that fails in `Retry`, and continue in a later step.** `Retry` runs everything it wraps again, the Command's `next` included. `retries: 2` means up to three calls, and `onExhausted` is where you give up. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
305
|
+
8. **Write a loop as a step that gets the list, then `Parallel(list.map(f), { limit: 1 })`.** `limit` caps how many run at once, and `settled: true` keeps going past a failure. A generator flow can use a `for` loop with `yield*`. See [Writing Loops](#writing-loops).
|
|
306
|
+
9. **Get the time, random IDs and anything else that changes between runs from a Command.** A replay then gets the recorded value, and a generator body, which can run more than once, sees the same value each time. See [What runs once, and what runs again](#what-runs-once-and-what-runs-again).
|
|
307
|
+
10. **Return plain data from Commands, and keep the context to plain values.** A trace stores JSON, so a `Date` comes back as a string. Reach a service from the Command's function, and hand it to the function that builds the flow. See [Passing Runtime Context](#passing-runtime-context).
|
|
308
|
+
11. **Clean up in a `try`/`finally` around `runEffect`,** not inside the flow. See [How a generator flow stops](#how-a-generator-flow-stops).
|
|
309
|
+
12. **Check a replay's error, not only its type.** A replay of a flow that changed shape ends in a `Failure` holding a `TimeParadox`, the error naming where the flow and the trace part ways, so a test that only expects a `Failure` passes by accident. See [Time-Travel Debugging](#time-travel-debugging).
|
|
385
310
|
|
|
386
|
-
|
|
311
|
+
<!-- rules:end -->
|
|
387
312
|
|
|
388
|
-
|
|
389
|
-
const { result, trace } = await recordEffect(registerUserFlow, input);
|
|
390
|
-
const { result: replayed } = await replayEffect(registerUserFlow(input), trace);
|
|
391
|
-
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive storage
|
|
392
|
-
```
|
|
313
|
+
## Passing Runtime Context
|
|
393
314
|
|
|
394
|
-
|
|
315
|
+
Some values come from the framework (an authenticated tenant, a request ID, environment config) rather than from the data being processed. `Ask` lets a step read the `context` object passed to `runEffect` without passing it through every function:
|
|
395
316
|
|
|
396
|
-
|
|
317
|
+
```js
|
|
318
|
+
import { Success, Failure, Command, Ask, effectPipe, runEffect } from 'pure-effect';
|
|
397
319
|
|
|
398
|
-
|
|
320
|
+
const findProduct = (productId) =>
|
|
321
|
+
Ask((ctx) => {
|
|
322
|
+
const cmdFindProduct = () => db[ctx.tenant].findProduct(productId);
|
|
323
|
+
return Command(cmdFindProduct, (product) => (product ? Success(product) : Failure('Product not found.')));
|
|
324
|
+
});
|
|
399
325
|
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
326
|
+
app.post('/checkout', async (req, res) => {
|
|
327
|
+
const result = await runEffect(checkoutFlow(req.body.productId), { tenant: req.tenant });
|
|
328
|
+
if (result.type === 'Success') return res.json(result.value);
|
|
329
|
+
// A Failure holds the full error, so send the client only an error meant for it.
|
|
330
|
+
// Answer the errors the flow returns by name, and anything else, such as a database outage, as a server error.
|
|
331
|
+
if (result.error === 'Product not found.') return res.status(400).json({ error: result.error });
|
|
332
|
+
res.status(500).json({ error: 'Checkout failed.' });
|
|
333
|
+
});
|
|
406
334
|
```
|
|
407
335
|
|
|
408
|
-
|
|
336
|
+
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run. So a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, belongs in a Command, whose result the trace records each time.
|
|
409
337
|
|
|
410
|
-
|
|
338
|
+
Keep the context to plain values such as IDs, flags and settings. Recording copies it into every trace, so a database client or connection pool in it puts its connection string there. One whose properties throw when read, as a lazy client's can, leaves the whole context out, and a replay then needs it passed in. Reach a service from the Command's function, as `findProduct` reaches `db`. To run a flow against fakes or a test database, have the function that builds it take the services, and pass them in the test:
|
|
411
339
|
|
|
412
340
|
```js
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
configureEffect(
|
|
419
|
-
telemetryHooks(),
|
|
420
|
-
recordingHooks({
|
|
421
|
-
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
422
|
-
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
423
|
-
redact: (value, name, kind) =>
|
|
424
|
-
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
425
|
-
maxEntries: 500,
|
|
426
|
-
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
427
|
-
})
|
|
428
|
-
);
|
|
429
|
-
```
|
|
430
|
-
|
|
431
|
-
What the example does with each run:
|
|
432
|
-
|
|
433
|
-
- **Which runs it keeps.** `keep` decides, and by default keeps a run that ended in a `Failure`; successful runs are held in memory and then discarded. When your own code throws, such as a `TypeError` after an API changed the shape of its response, the run rejects. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept. Replaying that trace reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded; that run did no I/O, so its input reproduces it.
|
|
434
|
-
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out of the trace, which marks where with `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
435
|
-
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
436
|
-
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
437
|
-
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
438
|
-
|
|
439
|
-
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` built, so build the outermost flow with `effectPipe`. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
440
|
-
|
|
441
|
-
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
442
|
-
|
|
443
|
-
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow, so neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes: a money object or a database entity comes back as a plain object without its methods, a `Date` as a string, a `Map` or a `Set` as `{}`, a `Buffer` as `{ type: 'Buffer', data }`, `undefined` in an array as `null`, and an error returned inside a result with its own fields, such as `code`, but without its message. A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it, so a context holding a lazy client is left out whole.
|
|
444
|
-
|
|
445
|
-
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`, and rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
446
|
-
|
|
447
|
-
### Keeping secrets out of a trace
|
|
448
|
-
|
|
449
|
-
`redact` sees everything a trace stores: each result, each thrown error, the flow's input and the context, told apart by `kind`. It gets a copy in the form the trace stores, so changing it in place is safe, and what it returns is stored in that form too. A thrown `Error` arrives as a plain object holding its `name`, `message` and own fields, and any object returned for it replays as an `Error`.
|
|
450
|
-
|
|
451
|
-
Check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
452
|
-
|
|
453
|
-
```js
|
|
454
|
-
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
455
|
-
const mask = (value, field) =>
|
|
456
|
-
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
457
|
-
|
|
458
|
-
const redact = (value, name, kind) => {
|
|
459
|
-
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
460
|
-
if (kind === 'context') return mask(value, 'authToken');
|
|
461
|
-
if (kind === 'error') return mask(value, 'attempted');
|
|
462
|
-
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
463
|
-
};
|
|
464
|
-
recorder({ redact });
|
|
465
|
-
|
|
466
|
-
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
467
|
-
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
468
|
-
```
|
|
469
|
-
|
|
470
|
-
A replay rebuilds the flow from the redacted input and context, so a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does, and a context field an `Ask` reads should stay.
|
|
471
|
-
|
|
472
|
-
## Hooks
|
|
473
|
-
|
|
474
|
-
Hooks see every run, and recording, replay and the telemetry example are built on them. There are three: `onRun` wraps a whole run, `onStep` wraps each Command and each `Parallel`, and `onBeforeCommand` runs before each Command and can veto it by throwing. A hook that only watches must `await` what it wraps and return its result; [`configureEffect`](#configureeffectconfigs) lists each hook's arguments.
|
|
475
|
-
|
|
476
|
-
`configureEffect` adds a layer of hooks for the whole process and returns a function that removes it. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's:
|
|
477
|
-
|
|
478
|
-
```js
|
|
479
|
-
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
480
|
-
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
481
|
-
remove();
|
|
482
|
-
```
|
|
483
|
-
|
|
484
|
-
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
485
|
-
|
|
486
|
-
```
|
|
487
|
-
runEffect(flow(input))
|
|
488
|
-
│
|
|
489
|
-
├─ telemetry.onRun first configuration: outermost
|
|
490
|
-
│ └─ recording.onRun last configuration: innermost
|
|
491
|
-
│ │
|
|
492
|
-
│ │ for each Command:
|
|
493
|
-
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
494
|
-
│ ├─ recording.onBeforeCommand
|
|
495
|
-
│ ├─ telemetry.onStep
|
|
496
|
-
│ │ └─ recording.onStep closest to the Command
|
|
497
|
-
│ │ └─ cmd()
|
|
498
|
-
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
499
|
-
│ ├─ telemetry.onStep returns from the inside out
|
|
500
|
-
│ │
|
|
501
|
-
│ ┌─ recording.onRun returns
|
|
502
|
-
├─ telemetry.onRun returns
|
|
503
|
-
```
|
|
504
|
-
|
|
505
|
-
**Knowing which request a step belongs to.** `onStep` and `onRun` get no context, so a hook that logs per request finds the request the way the rest of your logging does. An `AsyncLocalStorage` scope entered around `runEffect` reaches every hook the run calls, and the recording example uses one to give each run its own recorder:
|
|
506
|
-
|
|
507
|
-
```js
|
|
508
|
-
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
509
|
-
|
|
510
|
-
const requestScope = new AsyncLocalStorage();
|
|
511
|
-
const stepLog = [];
|
|
512
|
-
|
|
513
|
-
// Logs each step with its request and duration, and returns what the step returned.
|
|
514
|
-
const removeStepLog = configureEffect({
|
|
515
|
-
onStep: async (name, type, op, path) => {
|
|
516
|
-
const started = performance.now();
|
|
517
|
-
try {
|
|
518
|
-
return await op();
|
|
519
|
-
} finally {
|
|
520
|
-
const { requestId } = requestScope.getStore() ?? {};
|
|
521
|
-
stepLog.push({ requestId, name, path, ms: performance.now() - started });
|
|
522
|
-
}
|
|
523
|
-
}
|
|
524
|
-
});
|
|
525
|
-
|
|
526
|
-
await requestScope.run({ requestId: 'req_1' }, () => runEffect(registerUserFlow(input)));
|
|
527
|
-
assert.deepEqual(
|
|
528
|
-
stepLog.map((line) => [line.requestId, line.name]),
|
|
529
|
-
[
|
|
530
|
-
['req_1', 'cmdFindUser'],
|
|
531
|
-
['req_1', 'cmdSaveUser']
|
|
532
|
-
]
|
|
533
|
-
);
|
|
534
|
-
removeStepLog();
|
|
535
|
-
```
|
|
536
|
-
|
|
537
|
-
A Command that `onBeforeCommand` vetoes never reaches `onStep`, so log a veto where you make it. In tests, call `configureEffect()` with no arguments in a `beforeEach`, so a test that fails before removing its hooks does not leave them installed for the next.
|
|
538
|
-
|
|
539
|
-
**Per-call hooks.** `runEffect`'s third argument takes hooks for that call only. They are merged with the configured ones by the same rules: the configured wrappers run outside them, and the configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
540
|
-
|
|
541
|
-
```
|
|
542
|
-
inherit: true (default) inherit: false
|
|
543
|
-
|
|
544
|
-
onRun C.onRun (none)
|
|
545
|
-
onBeforeCommand C.onBeforeCommand (none)
|
|
546
|
-
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
547
|
-
```
|
|
548
|
-
|
|
549
|
-
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
550
|
-
|
|
551
|
-
## Passing Runtime Context
|
|
552
|
-
|
|
553
|
-
Some values come from the framework (an authenticated tenant, a request ID, environment config) rather than from the data being processed. `Ask` lets a step read the `context` object passed to `runEffect` without passing it through every function:
|
|
554
|
-
|
|
555
|
-
```js
|
|
556
|
-
import { Success, Failure, Command, Ask, effectPipe, runEffect } from 'pure-effect';
|
|
557
|
-
|
|
558
|
-
const findProduct = (productId) =>
|
|
559
|
-
Ask((ctx) => {
|
|
560
|
-
const cmdFindProduct = () => db[ctx.tenant].findProduct(productId);
|
|
561
|
-
return Command(cmdFindProduct, (product) => (product ? Success(product) : Failure('Product not found.')));
|
|
562
|
-
});
|
|
563
|
-
|
|
564
|
-
app.post('/checkout', async (req, res) => {
|
|
565
|
-
const result = await runEffect(checkoutFlow(req.body.productId), { tenant: req.tenant });
|
|
566
|
-
if (result.type === 'Success') return res.json(result.value);
|
|
567
|
-
// A Failure holds the full error, so send the client only an error meant for it.
|
|
568
|
-
// Answer the errors the flow returns by name, and anything else, such as a database outage, as a server error.
|
|
569
|
-
if (result.error === 'Product not found.') return res.status(400).json({ error: result.error });
|
|
570
|
-
res.status(500).json({ error: 'Checkout failed.' });
|
|
571
|
-
});
|
|
572
|
-
```
|
|
573
|
-
|
|
574
|
-
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run, so read a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, in a Command, whose result the trace records each time.
|
|
575
|
-
|
|
576
|
-
Keep the context to plain values such as IDs, flags and settings. Recording copies it into every trace, so a database client or connection pool in it puts its connection string in the trace, and one whose properties throw when read, as a lazy client's can, leaves the whole context out, so a replay needs it passed in. Reach a service from the Command's function, as `findProduct` reaches `db`. To run a flow against fakes or a test database, have the function that builds it take the services, and pass them in the test:
|
|
577
|
-
|
|
578
|
-
```js
|
|
579
|
-
// Takes its catalog, so a test can run it against a fake.
|
|
580
|
-
const makeFindProduct = (catalog) => (productId) => {
|
|
581
|
-
const cmdFindProduct = () => catalog.findProduct(productId);
|
|
582
|
-
return Command(cmdFindProduct, (product) => (product ? Success(product) : Failure('Product not found.')));
|
|
583
|
-
};
|
|
341
|
+
// Takes its catalog, so a test can run it against a fake.
|
|
342
|
+
const makeFindProduct = (catalog) => (productId) => {
|
|
343
|
+
const cmdFindProduct = () => catalog.findProduct(productId);
|
|
344
|
+
return Command(cmdFindProduct, (product) => (product ? Success(product) : Failure('Product not found.')));
|
|
345
|
+
};
|
|
584
346
|
|
|
585
347
|
// The fake answers by the ID it receives, so the test also checks what the Command's function sends.
|
|
586
348
|
const findInFakeCatalog = makeFindProduct({ findProduct: async (id) => (id === 'abc' ? { id } : null) });
|
|
@@ -595,37 +357,38 @@ A walk or a replay never runs a Command's function, so this is the test that cat
|
|
|
595
357
|
`Retry` runs part of a flow again when a Command's function throws. A `Failure` a step returned is not retried: the flow decided to stop, so the `Failure` passes on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency; to reuse settings, share an object:
|
|
596
358
|
|
|
597
359
|
```js
|
|
598
|
-
const flakyNetwork = {
|
|
360
|
+
const flakyNetwork = { retries: 3, delay: 200, backoff: 2 };
|
|
599
361
|
Retry(fetchPrice(sku), flakyNetwork);
|
|
600
362
|
```
|
|
601
363
|
|
|
602
|
-
The settings are a plain object, so a test can check them without running anything
|
|
364
|
+
The settings are a plain object, so a test can check them without running anything. The `Retry` wraps only the Command, and the decision about its answer comes in the next step, outside the `Retry`:
|
|
603
365
|
|
|
604
366
|
```js
|
|
605
|
-
import { Success, Failure, Command, Retry,
|
|
367
|
+
import { Success, Failure, Command, Retry, effectPipe } from 'pure-effect';
|
|
606
368
|
|
|
607
369
|
const fetchWeather = (city) => {
|
|
608
370
|
const cmdFetchWeather = () =>
|
|
609
371
|
fetch(`https://example-weather-api.com/v1/current?city=${city}`).then((r) => r.json());
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
{ attempts: 3, delay: 200, backoff: 2 } // 200ms, 400ms, 800ms
|
|
613
|
-
);
|
|
372
|
+
// Four calls in all, waiting 200ms, 400ms, then 800ms.
|
|
373
|
+
return Retry(Command(cmdFetchWeather), { retries: 3, delay: 200, backoff: 2 });
|
|
614
374
|
};
|
|
615
375
|
|
|
376
|
+
const requireWeather = (data) => (data.error ? Failure(data.error) : Success(data));
|
|
377
|
+
const weatherFlow = (city) => effectPipe(fetchWeather, requireWeather)(city);
|
|
378
|
+
|
|
616
379
|
const weatherFn = fetchWeather('Tokyo');
|
|
617
380
|
assert.equal(weatherFn.type, 'Retry');
|
|
618
|
-
assert.equal(weatherFn.options.
|
|
381
|
+
assert.equal(weatherFn.options.retries, 3);
|
|
619
382
|
```
|
|
620
383
|
|
|
621
384
|
**Wrap the Command that fails, not the pipeline.** Each attempt runs everything inside the `Retry` again, including Commands that already succeeded:
|
|
622
385
|
|
|
623
386
|
```js
|
|
624
387
|
// Dangerous: a flaky receipt step charges the customer again on every attempt.
|
|
625
|
-
Retry(effectPipe(chargeCard, sendReceipt)(order), {
|
|
388
|
+
Retry(effectPipe(chargeCard, sendReceipt)(order), { retries: 3 });
|
|
626
389
|
|
|
627
390
|
// Correct: only the step that fails transiently is retried.
|
|
628
|
-
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), {
|
|
391
|
+
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), { retries: 3 }))(order);
|
|
629
392
|
```
|
|
630
393
|
|
|
631
394
|
Wrapping a pipeline is safe only when every Command in it is idempotent (safe to run more than once).
|
|
@@ -636,7 +399,7 @@ Wrapping a pipeline is safe only when every Command in it is idempotent (safe to
|
|
|
636
399
|
// Dangerous: `next` continues the booking, so every later step is inside the Retry.
|
|
637
400
|
Retry(
|
|
638
401
|
Command(cmdHoldSeat, (seat) => chargeCard(trip, seat)),
|
|
639
|
-
{
|
|
402
|
+
{ retries: 2 }
|
|
640
403
|
);
|
|
641
404
|
```
|
|
642
405
|
|
|
@@ -645,7 +408,7 @@ If a later Command's function throws, the `Retry` holds the seat and charges the
|
|
|
645
408
|
```js
|
|
646
409
|
// Safe: the tree the Retry repeats is that one Command.
|
|
647
410
|
effectPipe(
|
|
648
|
-
() => Retry(Command(cmdHoldSeat), {
|
|
411
|
+
() => Retry(Command(cmdHoldSeat), { retries: 2 }),
|
|
649
412
|
(seat) => chargeCard(trip, seat)
|
|
650
413
|
);
|
|
651
414
|
```
|
|
@@ -653,7 +416,7 @@ effectPipe(
|
|
|
653
416
|
When every attempt fails, `runEffect` returns a `Failure` with this error:
|
|
654
417
|
|
|
655
418
|
```text
|
|
656
|
-
{ retryExhausted: true, lastError: <the last error>,
|
|
419
|
+
{ retryExhausted: true, lastError: <the last error>, retries: 3 }
|
|
657
420
|
```
|
|
658
421
|
|
|
659
422
|
In TypeScript, a `Retry` adds `RetryExhaustedError` to the error union, beside the errors of the steps it wraps. `lastError` is typed `unknown`, since it is whatever the Command's function threw, so check it before using it, for example with `instanceof Error`.
|
|
@@ -663,13 +426,17 @@ In TypeScript, a `Retry` adds `RetryExhaustedError` to the error union, beside t
|
|
|
663
426
|
```js
|
|
664
427
|
const fetchPrice = (sku) =>
|
|
665
428
|
Retry(fetchLivePrice(sku), {
|
|
666
|
-
|
|
667
|
-
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError,
|
|
429
|
+
retries: 3,
|
|
430
|
+
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError, retries }
|
|
668
431
|
});
|
|
669
432
|
```
|
|
670
433
|
|
|
671
434
|
Every attempt and the fallback are recorded, so a replay repeats the same failures and the same fallback, without the delays. A fallback never starts in a `Parallel` branch that has already been cancelled. With `onExhausted` set, the fallback's error type takes the place of `RetryExhaustedError` in the TypeScript error union.
|
|
672
435
|
|
|
436
|
+
### Waiting as long as the server asks
|
|
437
|
+
|
|
438
|
+
`Retry` waits a fixed `delay`, so it cannot wait as long as a server names, as a rate limit's `Retry-After` does. Catch that answer inside the Command's function and return it as data, as [Which Errors Are Data](#which-errors-are-data) shows, then have `next` return a Command that waits that long, followed by the same request again. A replay answers the wait from the trace like any other Command, so it does not wait.
|
|
439
|
+
|
|
673
440
|
## Running Effects in Parallel
|
|
674
441
|
|
|
675
442
|
`Parallel` runs several flows at the same time and passes their results to `next` as an array, in order. If a branch fails, the other branches are cancelled, `next` is not called, and that branch's `Failure` is returned.
|
|
@@ -695,7 +462,7 @@ Parallel(subscriptions.map(billOne), { limit: 5 });
|
|
|
695
462
|
Parallel(subscriptions.map(billOne), { limit: 5, settled: true });
|
|
696
463
|
```
|
|
697
464
|
|
|
698
|
-
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end and `next` receives one `Success` or `Failure` per branch, in order
|
|
465
|
+
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end, and `next` receives one `Success` or `Failure` per branch, in order. A failed record is then something you can count instead of the end of the job. In TypeScript, write `settled: true` where you call `Parallel`, or mark a shared options object `as const`. Since `settled` decides what `next` receives, a value the compiler only knows as `true` or `false` is refused.
|
|
699
466
|
|
|
700
467
|
```js
|
|
701
468
|
Parallel(subscriptions.map(billOne), (outcomes) => Success(outcomes.map(summarize)), { limit: 5, settled: true });
|
|
@@ -711,7 +478,7 @@ Parallel(work, (outcomes) => Success(outcomes.map((o) => (o.type === 'Success' ?
|
|
|
711
478
|
|
|
712
479
|
The outcomes are plain `Success` and `Failure` objects. Returning one of them from `next` is the same as returning any other `Failure`: the flow stops, and `Retry` does not run it again.
|
|
713
480
|
|
|
714
|
-
**Collecting some failures and stopping on others.** `settled` decides for every failure at once.
|
|
481
|
+
**Collecting some failures and stopping on others.** `settled` decides for every failure at once. Sometimes a batch should note a missing record and go on, but stop everything when its credentials expire. Leave `settled` off and decide in each branch: return what should be collected as a value inside a `Success`, and what should stop the batch as a `Failure`, which cancels the other branches:
|
|
715
482
|
|
|
716
483
|
```js
|
|
717
484
|
// A missing user is counted; an expired token stops the batch.
|
|
@@ -747,9 +514,13 @@ const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${user
|
|
|
747
514
|
|
|
748
515
|
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
749
516
|
|
|
517
|
+
A function gets the signal only when it declares a parameter without a default value. One written as `(...args)` or `({ signal } = {})`, or wrapped by a helper that hides its parameters, gets none, so once started it runs to the end. A parameter with a default value does not count, so `Command(nanoid)`, whose size is written `size = 21`, works everywhere.
|
|
518
|
+
|
|
519
|
+
That rule catches a function passed by name whose first parameter is optional and has no default. `function listUsers(page) { page ??= 1; return db.users(page); }` gets the signal as its page inside a `Parallel`, and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
520
|
+
|
|
750
521
|
### Undoing what succeeded when a branch fails
|
|
751
522
|
|
|
752
|
-
Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped.
|
|
523
|
+
Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped. Sometimes one of them did something that has to be undone, such as charging a card for an order whose stock then ran out. The code that called `runEffect` no longer has the charge to refund. Use `settled: true`, so `next` sees every outcome, and have it return the steps that undo whatever succeeded, followed by the failure:
|
|
753
524
|
|
|
754
525
|
```js
|
|
755
526
|
const reserveStock = (order) => {
|
|
@@ -809,31 +580,7 @@ assert.deepEqual(
|
|
|
809
580
|
|
|
810
581
|
A settled `Parallel` cancels nothing, so both steps always run to the end, and the undoing starts once both have finished. The undo steps are part of the flow, so a trace records them and a replay repeats them. An undo step that fails ends the flow with its own error rather than the original one; wrap it in `Retry` if it can fail for a moment.
|
|
811
582
|
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type, so give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)), rather than letting it fall into the last error it expects, which would report a vetoed charge as a declined card.
|
|
815
|
-
|
|
816
|
-
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)), and stop a batch by reading the operator's switch in a Command whose `next` returns a `Failure`, rather than by vetoing in `onBeforeCommand`:
|
|
817
|
-
|
|
818
|
-
```js
|
|
819
|
-
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
820
|
-
const cmdBillingHalted = () => flags.get('billing.halted');
|
|
821
|
-
const billUnlessHalted = (subscription) =>
|
|
822
|
-
Command(cmdBillingHalted, (halted) => (halted ? Failure('halted') : billOne(subscription)));
|
|
823
|
-
|
|
824
|
-
Parallel(subscriptions.map(billUnlessHalted), (outcomes) => Success(outcomes.map(summarize)), {
|
|
825
|
-
limit: 5,
|
|
826
|
-
settled: true
|
|
827
|
-
});
|
|
828
|
-
```
|
|
829
|
-
|
|
830
|
-
The halt is then one of the branch's declared errors, typed like a declined card, and the trace records each read of the switch, so the batch replays. A vetoed Command leaves no entry in the trace, so a replay of a vetoed batch fails at the first veto.
|
|
831
|
-
|
|
832
|
-
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
833
|
-
|
|
834
|
-
### Replaying a cancelled `Parallel`
|
|
835
|
-
|
|
836
|
-
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
583
|
+
Under `settled`, a branch whose Command throws reaches `next` as a `Failure`, so the undo still runs. When one call follows another instead, as a shipment follows a charge, the later Command's function has to catch its own error and return it as data: a throw stops the flow before its `next` runs, and the charge stays. [Translating `try`/`catch`](#translating-trycatch) shows that shape.
|
|
837
584
|
|
|
838
585
|
## Composing Larger Flows
|
|
839
586
|
|
|
@@ -872,6 +619,40 @@ When the lookup is written in place rather than reused, the Command's own `next`
|
|
|
872
619
|
|
|
873
620
|
In a generator flow, most of this comes for free. A step that branches into another flow is yielded like any other, as in `yield* step(processOrder(order))`. Values that do not depend on each other are joined with `const [cart, user] = yield* step(Parallel([fetchCart(input.cartId), fetchUser(input.userId)]))`. Values that do are variables, with no step to pass them on together; see [Where a generator helps](#where-a-generator-helps). Keep each decision in a step function that takes only what it needs, as the advice above says for a pipeline's steps.
|
|
874
621
|
|
|
622
|
+
### Ending a flow early without an error
|
|
623
|
+
|
|
624
|
+
Not every early end is an error. A webhook that has already handled an event should answer `duplicate` and do nothing else. A pipeline stops only at a `Failure`, so the step that decides ends the flow by what it returns: a `Success` holding the result, or the rest of the flow. Make it the last step of its pipeline, since a `Success` anywhere else passes on to the next step:
|
|
625
|
+
|
|
626
|
+
```js
|
|
627
|
+
const requireOrderId = (event) => (event.orderId ? Success(event) : Failure('Missing order ID.'));
|
|
628
|
+
|
|
629
|
+
// The rest of the flow, for an event seen for the first time.
|
|
630
|
+
const markOrderPaid = (event) => {
|
|
631
|
+
const cmdMarkOrderPaid = () => orderStore.markPaid(event.orderId);
|
|
632
|
+
return Command(cmdMarkOrderPaid, () => Success({ status: 'processed' }));
|
|
633
|
+
};
|
|
634
|
+
|
|
635
|
+
// Claiming the event decides: a duplicate ends the flow here, with a result rather than an error.
|
|
636
|
+
const claimEvent = (event) => {
|
|
637
|
+
const cmdClaimEvent = () => eventLog.claim(event.eventId);
|
|
638
|
+
return Command(cmdClaimEvent, (firstTime) => (firstTime ? markOrderPaid(event) : Success({ status: 'duplicate' })));
|
|
639
|
+
};
|
|
640
|
+
|
|
641
|
+
const handleEvent = (event) => effectPipe(requireOrderId, claimEvent)(event);
|
|
642
|
+
|
|
643
|
+
assert.deepEqual(
|
|
644
|
+
await runEffect(handleEvent({ eventId: 'evt_1', orderId: 'order_1' })),
|
|
645
|
+
Success({ status: 'processed' })
|
|
646
|
+
);
|
|
647
|
+
assert.deepEqual(
|
|
648
|
+
await runEffect(handleEvent({ eventId: 'evt_seen', orderId: 'order_1' })),
|
|
649
|
+
Success({ status: 'duplicate' })
|
|
650
|
+
);
|
|
651
|
+
assert.deepEqual(await runEffect(handleEvent({ eventId: 'evt_2' })), Failure('Missing order ID.'));
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
When the rest of the flow is longer, the step returns it as a pipeline of its own, as `processOrder` does above. In a generator flow, `return` ends the flow anywhere, but keep the decision in a step all the same: `return yield* step(claimEvent(event))`.
|
|
655
|
+
|
|
875
656
|
## Writing Loops
|
|
876
657
|
|
|
877
658
|
A loop is written in two parts: a step that gets the list, and a `Parallel` over it, with `limit: 1` so the items run one after another, as in a `for` loop. The `async` version is usually split the same way. Paying a customer's unpaid invoices in order, and stopping at the first that fails:
|
|
@@ -894,7 +675,7 @@ const findUnpaidInvoices = (customerId) => {
|
|
|
894
675
|
// Second part: the loop's body, a flow of its own for one invoice.
|
|
895
676
|
const chargeInvoice = (invoice) => {
|
|
896
677
|
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
897
|
-
return Retry(Command(cmdChargeInvoice), {
|
|
678
|
+
return Retry(Command(cmdChargeInvoice), { retries: 3 });
|
|
898
679
|
};
|
|
899
680
|
|
|
900
681
|
const sendReceipt = (receipt) => {
|
|
@@ -925,7 +706,7 @@ assert.deepEqual(await runEffect(payAll([])), Success([]));
|
|
|
925
706
|
|
|
926
707
|
Each invoice is paid after the one before it has finished, and the first that fails stops the rest from starting, as in the `async` version. When the payments can overlap, a higher `limit` runs several at a time.
|
|
927
708
|
|
|
928
|
-
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one, since a `Retry` runs everything it wraps again (see [Retrying Transient Failures](#retrying-transient-failures))
|
|
709
|
+
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one, since a `Retry` runs everything it wraps again (see [Retrying Transient Failures](#retrying-transient-failures)). Around the whole loop it would charge the invoices already paid. Around the whole body, a receipt that failed to send would charge the card again.
|
|
929
710
|
|
|
930
711
|
The `Parallel`'s `next` receives what each body returned, in order, so a total or a summary of the loop goes there: `Parallel(invoices.map(payInvoice), (receipts) => Success(receipts.length), { limit: 1 })`. To keep going past a failure, as a `try`/`catch` inside the loop would, add `settled: true`: every invoice is tried, and `next` receives one `Success` or `Failure` per invoice. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
931
712
|
|
|
@@ -949,7 +730,7 @@ assert.equal(fromLoop.result.error.retryExhausted, true);
|
|
|
949
730
|
assert.deepEqual(fromLoop.result, fromParallel.result);
|
|
950
731
|
```
|
|
951
732
|
|
|
952
|
-
It stops at the first invoice that fails, as the `Parallel` with `limit: 1` does. Use the `Parallel` instead when
|
|
733
|
+
It stops at the first invoice that fails, as the `Parallel` with `limit: 1` does. Use the `Parallel` instead when payments can overlap, with a higher `limit`. Use it too when the loop has to keep going past a failure, with `settled: true`, since a `catch` in the body cannot recover. The two forms record differently, so a trace recorded from one replays against the other as a `TimeParadox`.
|
|
953
734
|
|
|
954
735
|
Sometimes getting the list is a loop of its own. Paging through an API with a cursor is one: each request needs the cursor from the page before, so the requests cannot be listed in advance. Write that loop as an ordinary `async` function, call it from one Command, and map over the list it returns, as above:
|
|
955
736
|
|
|
@@ -971,7 +752,7 @@ const listAllOrders = () => {
|
|
|
971
752
|
};
|
|
972
753
|
```
|
|
973
754
|
|
|
974
|
-
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test.
|
|
755
|
+
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test. This is rule 1 applied to a loop: the Command's function is the one call, `fetchAllOrders()`, and the loop lives in a function of its own that an integration test covers.
|
|
975
756
|
|
|
976
757
|
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
977
758
|
|
|
@@ -997,222 +778,748 @@ const fetchPrice = (sku) => {
|
|
|
997
778
|
};
|
|
998
779
|
```
|
|
999
780
|
|
|
1000
|
-
The failed attempt is recorded as that Command's result, with the error's `code` and `message`, so a replay takes the same fallback branch. Copy the fields you need rather than returning the error itself: a trace keeps data, not objects, so an error stored inside a result loses its message (see [Recording in Production](#recording-in-production)).
|
|
781
|
+
The failed attempt is recorded as that Command's result, with the error's `code` and `message`, so a replay takes the same fallback branch. Copy the fields you need rather than returning the error itself: a trace keeps data, not objects, so an error stored inside a result loses its message (see [Recording in Production](#recording-in-production)).
|
|
782
|
+
|
|
783
|
+
**A `Failure` stops the flow.** It goes back to the code that called `runEffect`, which decides what it means: an HTTP status, a queue retry, an alert. Nothing inside the flow can catch a `Failure`, so a reader never has to look elsewhere for a handler.
|
|
784
|
+
|
|
785
|
+
Rule of thumb: if you would handle it, return it as a value; if you would only report it, return a `Failure`. The one exception is retry exhaustion, which happens inside `Retry`; handle it with the `onExhausted` option in [Retrying Transient Failures](#retrying-transient-failures).
|
|
786
|
+
|
|
787
|
+
**Catch or retry, not both.** `Retry` only reacts to a function that throws. If you catch a 503 inside the function and return it as a value, then wrap the step in `Retry`, it makes one call and never retries. Catch the error when the flow should handle it as data; let it throw when `Retry` should try again.
|
|
788
|
+
|
|
789
|
+
```js
|
|
790
|
+
// Handled as data: one call, and `next` decides what the miss means.
|
|
791
|
+
Command(
|
|
792
|
+
function cmdFetchPrice() {
|
|
793
|
+
return pricing
|
|
794
|
+
.get(sku)
|
|
795
|
+
.then((price) => ({ ok: true, price }))
|
|
796
|
+
.catch((error) => ({ ok: false, code: error.code, message: error.message }));
|
|
797
|
+
},
|
|
798
|
+
(r) => (r.ok ? Success(r.price) : fetchCachedPrice(sku))
|
|
799
|
+
);
|
|
800
|
+
|
|
801
|
+
// Retried: the function lets the rejection out, so the interpreter sees an I/O fault.
|
|
802
|
+
Retry(
|
|
803
|
+
Command(function cmdFetchPrice() {
|
|
804
|
+
return pricing.get(sku);
|
|
805
|
+
}),
|
|
806
|
+
{ retries: 3 }
|
|
807
|
+
);
|
|
808
|
+
```
|
|
809
|
+
|
|
810
|
+
To retry some errors and not others, do both in one function: return the errors the flow handles, and rethrow the rest so `Retry` tries again. Here a charge the gateway refuses is handled, and anything else, such as a timeout, is retried:
|
|
811
|
+
|
|
812
|
+
```js
|
|
813
|
+
const chargeAmount = (amount) => {
|
|
814
|
+
const cmdChargeAmount = () =>
|
|
815
|
+
gateway.charge(amount).catch((error) => {
|
|
816
|
+
if (error.code === 'invalid_amount') return { refused: true, code: error.code };
|
|
817
|
+
throw error; // may pass on a later try
|
|
818
|
+
});
|
|
819
|
+
return Retry(Command(cmdChargeAmount), { retries: 2 });
|
|
820
|
+
};
|
|
821
|
+
|
|
822
|
+
// A later step turns the handled error into the flow's outcome.
|
|
823
|
+
const requireCharge = (charge) => (charge.refused ? Failure(`Charge refused: ${charge.code}`) : Success(charge));
|
|
824
|
+
const pay = (amount) => effectPipe(chargeAmount, requireCharge)(amount);
|
|
825
|
+
|
|
826
|
+
assert.deepEqual(await runEffect(pay(0)), Failure('Charge refused: invalid_amount'));
|
|
827
|
+
assert.deepEqual(await runEffect(pay(25)), Success({ chargeId: 'ch_1', amount: 25 }));
|
|
828
|
+
```
|
|
829
|
+
|
|
830
|
+
`runEffect` tells these cases apart:
|
|
831
|
+
|
|
832
|
+
| what happened | what it is | what `Retry` does | what `onExhausted` sees | what the caller's `Failure` holds |
|
|
833
|
+
| --- | --- | --- | --- | --- |
|
|
834
|
+
| a step returned `Failure(...)`, or an `onBeforeCommand` hook threw | a stop | nothing; it propagates at once, unwrapped | nothing | the error returned or thrown |
|
|
835
|
+
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone | the error thrown, or, once a `Retry` gives up, `{ retryExhausted, lastError, retries }` |
|
|
836
|
+
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing | nothing; `runEffect` rejects |
|
|
837
|
+
|
|
838
|
+
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is a bug, and is thrown as it is rather than returned as a `Failure`. `runEffect` rejects with it, or, from a step before the first Command, the call that builds the flow throws it.
|
|
839
|
+
|
|
840
|
+
### Translating `try`/`catch`
|
|
841
|
+
|
|
842
|
+
A `catch` around `yield*` can log and rethrow, but it cannot recover, and it sees a `FlowStopped` error rather than the original. So each `try`/`catch` in an `async` function moves to where its error happens. The four common shapes:
|
|
843
|
+
|
|
844
|
+
- **Carry on past a failed call.** Catch inside the Command's function and return the error as data, as below.
|
|
845
|
+
- **Undo the calls that succeeded when one running beside them fails.** Run the calls in a `Parallel` with `settled: true`, and have its `next` undo the ones that succeeded; see [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails).
|
|
846
|
+
- **Undo an earlier call when a later one fails.** Catch inside the later Command's function and return the error as data, and have its `next` return the Command that undoes the earlier one, as in the second example below. A later Command that throws instead stops the flow before its `next` runs, so nothing is undone.
|
|
847
|
+
- **Retry, then give up.** Wrap the Command in `Retry`, and put what the `catch` did after the last try in `onExhausted`; see [Retrying Transient Failures](#retrying-transient-failures).
|
|
848
|
+
|
|
849
|
+
A welcome email whose failure the flow reports but does not stop for:
|
|
850
|
+
|
|
851
|
+
```js
|
|
852
|
+
// Takes the mailer, so a test can hand it one that fails.
|
|
853
|
+
const makeSendWelcome = (mailer) => (user) => {
|
|
854
|
+
const cmdSendWelcome = () =>
|
|
855
|
+
mailer
|
|
856
|
+
.sendWelcome(user.email)
|
|
857
|
+
.then(() => ({ sent: true }))
|
|
858
|
+
.catch((error) => ({ sent: false, reason: error.message }));
|
|
859
|
+
return Command(cmdSendWelcome);
|
|
860
|
+
};
|
|
861
|
+
|
|
862
|
+
const failingMailer = {
|
|
863
|
+
sendWelcome: async () => {
|
|
864
|
+
throw new Error('smtp down');
|
|
865
|
+
}
|
|
866
|
+
};
|
|
867
|
+
const welcome = await runEffect(makeSendWelcome(failingMailer)({ email: 'test@test.com' }));
|
|
868
|
+
assert.deepEqual(welcome, Success({ sent: false, reason: 'smtp down' }));
|
|
869
|
+
```
|
|
870
|
+
|
|
871
|
+
A reservation made after a charge, which refunds the charge when the reservation fails:
|
|
872
|
+
|
|
873
|
+
```js
|
|
874
|
+
const refundCharge = (charge, reason) => {
|
|
875
|
+
const cmdRefundCharge = () => payments.refund(charge.chargeId);
|
|
876
|
+
return Command(cmdRefundCharge, () => Failure(`Reservation failed: ${reason}`));
|
|
877
|
+
};
|
|
878
|
+
|
|
879
|
+
const reserveStock = (charge) => {
|
|
880
|
+
const cmdReserveStock = () =>
|
|
881
|
+
warehouse
|
|
882
|
+
.reserve(charge.orderId)
|
|
883
|
+
.then(() => ({ reserved: true }))
|
|
884
|
+
.catch((error) => ({ reserved: false, reason: error.message }));
|
|
885
|
+
return Command(cmdReserveStock, (reservation) =>
|
|
886
|
+
reservation.reserved ? Success(charge) : refundCharge(charge, reservation.reason)
|
|
887
|
+
);
|
|
888
|
+
};
|
|
889
|
+
|
|
890
|
+
const reservation = await runEffect(reserveStock({ orderId: 'order_1', chargeId: 'ch_1' }));
|
|
891
|
+
assert.deepEqual(reservation, Failure('Reservation failed: out of stock'));
|
|
892
|
+
```
|
|
893
|
+
|
|
894
|
+
### A malformed flow
|
|
895
|
+
|
|
896
|
+
A flow that is not made of Effects is a bug, so it throws an `EffectTypeError` that says where, rather than returning a `Failure`. The steps before a flow's first Command run as it is built. So a mistake in one of them is thrown by the call that builds the flow, as `registerUserFlow(input)` would throw if its first step returned a plain object:
|
|
897
|
+
|
|
898
|
+
```
|
|
899
|
+
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
900
|
+
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
901
|
+
```
|
|
902
|
+
|
|
903
|
+
A mistake after the first Command makes `runEffect` reject. That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments where they are built, before their own I/O runs. Each of these throws an `EffectTypeError` there:
|
|
904
|
+
|
|
905
|
+
- `Command(db.findUser(email))`, which makes the call while the flow is built
|
|
906
|
+
- `Command(fn, { name })`, with the metadata in `next`'s place
|
|
907
|
+
- a `Parallel` or `effectPipe` given the wrong thing
|
|
908
|
+
|
|
909
|
+
One built inside a `next` is checked when that `next` runs.
|
|
910
|
+
|
|
911
|
+
## Generator Flows in Depth
|
|
912
|
+
|
|
913
|
+
What [Writing Flows as Generators](#writing-flows-as-generators) leaves out: when a body runs again, what a `catch` or `finally` in it can do, and the mistakes `effectGen` catches.
|
|
914
|
+
|
|
915
|
+
### What runs once, and what runs again
|
|
916
|
+
|
|
917
|
+
A run goes through the body once, top to bottom, as `async` code does. Running the flow again starts the body from the top, as calling an `async` function again does. Each `Retry` attempt, each run of one flow value, and each `Parallel` branch holding it gets a fresh build, so every Command in it sees the same values. A test that hands an earlier step a second answer, to walk another branch, runs the body again from the top too, feeding it the answers before that step:
|
|
918
|
+
|
|
919
|
+
```js
|
|
920
|
+
const flow = registerUserFlowGen(input);
|
|
921
|
+
assert.equal(commandName(flow), 'cmdFindUser');
|
|
922
|
+
|
|
923
|
+
// The first answer finds a user, so the flow stops there.
|
|
924
|
+
assert.deepEqual(flow.next({ id: 1 }), Failure('Email already in use.'));
|
|
925
|
+
|
|
926
|
+
// A second answer to the same step runs the body again from the top, and this time it goes on to save.
|
|
927
|
+
assert.equal(commandName(flow.next(null)), 'cmdSaveUser');
|
|
928
|
+
```
|
|
929
|
+
|
|
930
|
+
So keep the body to its own variables. A body run again for a walk has to ask for the same steps in the same order, and one that asks for a different one stops with an `EffectTypeError` naming both. `Date.now()`, a random ID, or a variable outside the body can change between runs. Get the first two from a Command, as a pipeline does, so a replay gets the recorded value. Pass the third in the flow's input.
|
|
931
|
+
|
|
932
|
+
### How a generator flow stops
|
|
933
|
+
|
|
934
|
+
`return` ends the flow. A plain value is its `Success`, `return Failure(error)` stops it with that error, and returning a Command or any other Effect runs it, as `return fetch(url)` does in an `async` function. A `throw` in the body is a bug, as one in a pipeline step is, and `runEffect` rejects with it.
|
|
935
|
+
|
|
936
|
+
When a step stops the flow, because its Command threw or it returned a `Failure`, the body's `catch` and `finally` blocks run, as they would for an `await` that rejected. The `catch` gets an error saying where the flow stopped, not the original, which stays the flow's `Failure`. So it cannot branch on the original's fields: `e.code` and the like are `undefined`, and `e.name` is `FlowStopped`. It can log, but it cannot recover:
|
|
937
|
+
|
|
938
|
+
```js
|
|
939
|
+
const unreliableDb = {
|
|
940
|
+
saveUser: async () => {
|
|
941
|
+
throw new Error('connection reset');
|
|
942
|
+
}
|
|
943
|
+
};
|
|
944
|
+
|
|
945
|
+
const saveUser = effectGen(function* (user) {
|
|
946
|
+
try {
|
|
947
|
+
const cmdSaveUser = () => unreliableDb.saveUser(user);
|
|
948
|
+
return yield* step(Command(cmdSaveUser));
|
|
949
|
+
} catch (error) {
|
|
950
|
+
// Runs when the save throws. It can log, and it has to let the error pass.
|
|
951
|
+
throw error;
|
|
952
|
+
}
|
|
953
|
+
});
|
|
954
|
+
|
|
955
|
+
const result = await runEffect(saveUser({ id: 1 }));
|
|
956
|
+
assert.equal(result.error.message, 'connection reset');
|
|
957
|
+
```
|
|
958
|
+
|
|
959
|
+
A `catch` that returns or carries on, one that throws a different error, and a `finally` that runs another step all stop the run with an `EffectTypeError` that says so:
|
|
960
|
+
|
|
961
|
+
```js
|
|
962
|
+
const saveLater = effectGen(function* (user) {
|
|
963
|
+
try {
|
|
964
|
+
const cmdSaveUser = () => unreliableDb.saveUser(user);
|
|
965
|
+
return yield* step(Command(cmdSaveUser));
|
|
966
|
+
} catch {
|
|
967
|
+
return 'saved later';
|
|
968
|
+
}
|
|
969
|
+
});
|
|
970
|
+
|
|
971
|
+
await assert.rejects(runEffect(saveLater({ id: 1 })), /kept going after Command 'cmdSaveUser' stopped it/);
|
|
972
|
+
```
|
|
973
|
+
|
|
974
|
+
Handle an error the flow expects inside the Command's function, and return it as data for the body to branch on; recover from one that keeps happening with `Retry`'s `onExhausted`. See [Which Errors Are Data](#which-errors-are-data). Cleanup for the whole run goes in a `try`/`finally` around `runEffect`, as it does for a pipeline.
|
|
975
|
+
|
|
976
|
+
A body's `finally` does not always run. It does not run when a bug or a `TimeParadox` rejects the run. In a `Parallel` where one branch threw, another branch that stops for its own reason before the rejection reaches it may still run its `catch` and `finally`, depending on timing.
|
|
977
|
+
|
|
978
|
+
A step in a body's `finally` is not supported. It runs when the body ends normally, and is refused when a step stopped the body. But if an error is passing through the body when its `finally` reaches a step, and the flow is cancelled before that step starts, the error is lost.
|
|
979
|
+
|
|
980
|
+
### Mistakes it catches
|
|
981
|
+
|
|
982
|
+
| Written | What happens |
|
|
983
|
+
| --- | --- |
|
|
984
|
+
| `yield step(x)`, without the star | an `EffectTypeError`: write `yield*` |
|
|
985
|
+
| `yield x` | an `EffectTypeError`: write `yield* step(x)` |
|
|
986
|
+
| `yield* x`, without `step` | the engine's `TypeError`, which on V8 becomes an `EffectTypeError` saying to write `yield* step(x)` |
|
|
987
|
+
| `const found = step(x)`, then `found.id` | an `EffectTypeError`: `step(...)` was used as a value |
|
|
988
|
+
| `[...step(x)]` or `for (const v of step(x))` | an `EffectTypeError`: `step(...)` was iterated outside `yield*` |
|
|
989
|
+
| `step(db.findUser(email))` | an `EffectTypeError`: put the call in a Command |
|
|
990
|
+
| `effectGen(async function* () {})` | an `EffectTypeError`: write `function*` |
|
|
991
|
+
|
|
992
|
+
In TypeScript, most of these do not compile; see [TypeScript](#typescript-typed-errors-and-context).
|
|
993
|
+
|
|
994
|
+
## Parallel in Depth
|
|
995
|
+
|
|
996
|
+
What [Running Effects in Parallel](#running-effects-in-parallel) leaves out: errors no branch returned, and how a cancelled `Parallel` replays.
|
|
997
|
+
|
|
998
|
+
### Errors no branch returned
|
|
999
|
+
|
|
1000
|
+
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type. So give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)). Otherwise it falls into the last error it expects, and reports a vetoed charge as a declined card.
|
|
1001
|
+
|
|
1002
|
+
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)). To stop a batch, read the operator's switch in a Command whose `next` returns a `Failure`, rather than vetoing in `onBeforeCommand`:
|
|
1003
|
+
|
|
1004
|
+
```js
|
|
1005
|
+
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
1006
|
+
const cmdBillingHalted = () => flags.get('billing.halted');
|
|
1007
|
+
const billUnlessHalted = (subscription) =>
|
|
1008
|
+
Command(cmdBillingHalted, (halted) => (halted ? Failure('halted') : billOne(subscription)));
|
|
1009
|
+
|
|
1010
|
+
Parallel(subscriptions.map(billUnlessHalted), (outcomes) => Success(outcomes.map(summarize)), {
|
|
1011
|
+
limit: 5,
|
|
1012
|
+
settled: true
|
|
1013
|
+
});
|
|
1014
|
+
```
|
|
1015
|
+
|
|
1016
|
+
The halt is then one of the branch's declared errors, typed like a declined card, and the trace records each read of the switch, so the batch replays. A vetoed Command leaves no entry in the trace, so a replay of a vetoed batch fails at the first veto.
|
|
1017
|
+
|
|
1018
|
+
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
1019
|
+
|
|
1020
|
+
### Replaying a cancelled `Parallel`
|
|
1021
|
+
|
|
1022
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
1023
|
+
|
|
1024
|
+
If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
1025
|
+
|
|
1026
|
+
A generator flow in a cancelled branch can run its `catch` and `finally` one step later in a replay than it did in production. A trace marks where a branch stopped only by its Commands. So a branch cancelled during a `Retry`'s wait, just before an `Ask`, or at a `Retry` or `Parallel` with no Command in it replays as though it stopped at its next Command.
|
|
1027
|
+
|
|
1028
|
+
## TypeScript: Typed Errors and Context
|
|
1029
|
+
|
|
1030
|
+
The bundled declarations need TypeScript 5.1 or later.
|
|
1031
|
+
|
|
1032
|
+
### Error union across pipeline steps
|
|
1033
|
+
|
|
1034
|
+
Each step in `effectPipe` carries its own error type. The compiler collects them into a union automatically:
|
|
1035
|
+
|
|
1036
|
+
```ts
|
|
1037
|
+
type ValidationError = 'invalid_email' | 'weak_password';
|
|
1038
|
+
type ApiError = 'network_timeout' | 'rate_limited';
|
|
1039
|
+
|
|
1040
|
+
const validate = (input: { email: string }): Effect<{ email: string }, ValidationError> => { ... };
|
|
1041
|
+
const submit = (input: { email: string }): Effect<{ id: number }, ApiError> => { ... };
|
|
1042
|
+
|
|
1043
|
+
const result = await runEffect(effectPipe(validate, submit)({ email: 'user@example.com' }));
|
|
1044
|
+
if (result.type === 'Failure') {
|
|
1045
|
+
result.error; // 'invalid_email' | 'weak_password' | 'network_timeout' | 'rate_limited'
|
|
1046
|
+
}
|
|
1047
|
+
```
|
|
1048
|
+
|
|
1049
|
+
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object. A step that cannot fail, such as a Command without a `next`, adds nothing.
|
|
1050
|
+
|
|
1051
|
+
**The union covers only the `Failure`s your steps return.** A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure`, and its error is not in the union. So where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error. The same goes for the outcomes a settled `Parallel` hands `next`. One fallback covers the whole flow: catch inside a Command's function only what the flow can handle, and let the rest throw, which is also what lets `Retry` act on it.
|
|
1052
|
+
|
|
1053
|
+
To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error`. TypeScript refuses `instanceof Error` on an error type made only of strings, or on a flow that declares none.
|
|
1054
|
+
|
|
1055
|
+
To type what a Command's function returns, such as a JSON response, write the function's result type, as in `Command((): Promise<User | null> => fetchJson(url), next)`. `Command<User | null>(…)` also compiles, but with some type arguments given TypeScript infers none of the others, so that Command's errors are typed `unknown`, which hides the rest of the union.
|
|
1056
|
+
|
|
1057
|
+
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
1058
|
+
|
|
1059
|
+
`Failure` keeps a literal exactly as written, but `Success` widens it, as TypeScript does for any object: `Success({ status: 'queued' })` holds a `status` of type `string`. A later step that tells values apart by such a field then refuses it, often several steps from where the value was built. So give the type there, as in `Success<Queued>({ status: 'queued', jobId })`.
|
|
1060
|
+
|
|
1061
|
+
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
1062
|
+
|
|
1063
|
+
### Handling every error
|
|
1064
|
+
|
|
1065
|
+
A `switch` over a flow's error union can make TypeScript refuse a case you forgot. Check first for a thrown error, which reaches `result.error` without being in the union, and end the `switch` with a `never`, so a case added to the union later is a compile error until it is handled. TypeScript narrows a `switch` only over a union, so with a single error type, leave out the `default`:
|
|
1066
|
+
|
|
1067
|
+
```ts
|
|
1068
|
+
type BillingError = { type: 'NoSubscription' } | { type: 'CardDeclined'; code: string };
|
|
1069
|
+
|
|
1070
|
+
const result = await runEffect(billTenant(tenantId), context);
|
|
1071
|
+
if (result.type === 'Failure') {
|
|
1072
|
+
const thrown: unknown = result.error;
|
|
1073
|
+
if (thrown instanceof Error) return serverError(thrown); // a Command's function threw
|
|
1074
|
+
switch (result.error.type) {
|
|
1075
|
+
case 'NoSubscription':
|
|
1076
|
+
return skipTenant();
|
|
1077
|
+
case 'CardDeclined':
|
|
1078
|
+
return notifyDecline(result.error.code);
|
|
1079
|
+
default: {
|
|
1080
|
+
const unhandled: never = result.error;
|
|
1081
|
+
return serverError(unhandled);
|
|
1082
|
+
}
|
|
1083
|
+
}
|
|
1084
|
+
}
|
|
1085
|
+
```
|
|
1086
|
+
|
|
1087
|
+
### Typed context with `Ask`
|
|
1088
|
+
|
|
1089
|
+
`Effect<T, E, Ctx>` carries a third type parameter for the context object:
|
|
1090
|
+
|
|
1091
|
+
```ts
|
|
1092
|
+
type AppContext = { tenant: string; requestId: string };
|
|
1093
|
+
|
|
1094
|
+
const findProduct = (productId: string): Effect<Product, 'not_found', AppContext> =>
|
|
1095
|
+
Ask<Product, 'not_found', AppContext>((ctx) => { ... });
|
|
1096
|
+
|
|
1097
|
+
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
1098
|
+
```
|
|
1099
|
+
|
|
1100
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object. It checks one passed as a variable against what the flow reads, as it checks one written in place, so a variable whose type lacks a field the flow reads does not compile. `recordEffect`, `replayEffect` and `timeTravel` check a context the same way, and `replayEffect` given a resolver requires one, since a resolver holds none.
|
|
1101
|
+
|
|
1102
|
+
### Generator flows
|
|
1103
|
+
|
|
1104
|
+
`effectGen` infers everything from the body. Each `yield*` is typed by its own step, the error union collects every step's errors and every `Failure` the body returns, and the context collects what every step reads:
|
|
1105
|
+
|
|
1106
|
+
```ts
|
|
1107
|
+
const registerUser = effectGen(function* (input: { email: string }) {
|
|
1108
|
+
const found = yield* step(findUser(input.email)); // User | null
|
|
1109
|
+
if (found) return Failure('EMAIL_TAKEN');
|
|
1110
|
+
return yield* step(saveUser(input));
|
|
1111
|
+
});
|
|
1112
|
+
// (input: { email: string }) => Effect<User, 'EMAIL_TAKEN' | the errors findUser and saveUser declare>
|
|
1113
|
+
```
|
|
1114
|
+
|
|
1115
|
+
`yield* x` without `step`, reading a property of a step, and an `async function*` do not compile, and a body that returns an untyped value keeps its steps' errors. Returning a step or a Promise compiles, so that a generic helper returning a value of its type parameter does too, and fails when it runs. `yield step(x)` without the star does, and fails when it runs. Name the context `Ask()` reads as `Ask<AppContext>()`. Given a function, `Ask`'s first type argument is the value instead, so annotate the parameter: `Ask((ctx: AppContext) => ...)`.
|
|
1116
|
+
|
|
1117
|
+
### A flow with no input
|
|
1118
|
+
|
|
1119
|
+
A flow takes its input as one parameter. For one that needs none, such as a nightly job that gets everything from its Commands, type the parameter `void`, so the flow is called with no argument. In a pipeline, that is the first step's parameter. Without it, `nightlyReport()` fails to compile with "Expected 1 arguments, but got 0":
|
|
1120
|
+
|
|
1121
|
+
```ts
|
|
1122
|
+
const nightlyReport = effectGen(function* (_input: void) {
|
|
1123
|
+
const users = yield* step(listUsers());
|
|
1124
|
+
return users.length;
|
|
1125
|
+
});
|
|
1126
|
+
|
|
1127
|
+
const result = await runEffect(nightlyReport());
|
|
1128
|
+
|
|
1129
|
+
// The same as a pipeline.
|
|
1130
|
+
const nightlyCount = effectPipe(
|
|
1131
|
+
(_input: void) => listUsers(),
|
|
1132
|
+
(users) => Success(users.length)
|
|
1133
|
+
);
|
|
1134
|
+
|
|
1135
|
+
const counted = await runEffect(nightlyCount());
|
|
1136
|
+
```
|
|
1137
|
+
|
|
1138
|
+
### Testing a flow in TypeScript
|
|
1139
|
+
|
|
1140
|
+
A flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure`. So `.cmd` and `.next` compile only after the test checks that the step is a Command. `node:assert`'s `assert(step.type === 'Command')` is such a check, and TypeScript follows it. Jest's and Vitest's `expect` is not, so a walk needs a check at every step, and a small helper makes that one line per step with any test framework:
|
|
1141
|
+
|
|
1142
|
+
```ts
|
|
1143
|
+
import assert from 'node:assert/strict';
|
|
1144
|
+
import { commandName } from 'pure-effect';
|
|
1145
|
+
import type { Effect } from 'pure-effect';
|
|
1146
|
+
|
|
1147
|
+
// Fails unless the step is the Command named, and returns it typed as one.
|
|
1148
|
+
function assertCommand<T, E, C>(step: Effect<T, E, C>, name: string) {
|
|
1149
|
+
assert(step.type === 'Command', `expected ${name}, got ${step.type}`);
|
|
1150
|
+
assert.equal(commandName(step), name);
|
|
1151
|
+
return step;
|
|
1152
|
+
}
|
|
1153
|
+
|
|
1154
|
+
const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
1155
|
+
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
1156
|
+
```
|
|
1157
|
+
|
|
1158
|
+
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns. So where [Testing Without Mocks](#testing-without-mocks) passes `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
1159
|
+
|
|
1160
|
+
## Time-Travel Debugging
|
|
1161
|
+
|
|
1162
|
+
Record what each Command returned, then feed those results back into the flow to retrace the path a request took, with no database or network.
|
|
1163
|
+
|
|
1164
|
+
Record a run with `recordEffect`, and `replayEffect` hands each Command what it returned the first time. That is how a run recorded in production replays on your machine, with nothing connected:
|
|
1165
|
+
|
|
1166
|
+
```js
|
|
1167
|
+
import { recordEffect, replayEffect } from 'pure-effect';
|
|
1168
|
+
|
|
1169
|
+
// A real run returns its result, and a trace of what each Command returned.
|
|
1170
|
+
const { result: live, trace } = await recordEffect(registerUserFlow(input));
|
|
1171
|
+
|
|
1172
|
+
// Anywhere else, with nothing connected, the same flow gets the same answers.
|
|
1173
|
+
const { result: replayed } = await replayEffect(registerUserFlow(trace.initialInput), trace);
|
|
1174
|
+
assert.deepEqual(replayed, live);
|
|
1175
|
+
```
|
|
1176
|
+
|
|
1177
|
+
In production, record each run with a version and a flow name, and replay its trace anywhere with `timeTravel`, which narrates each step:
|
|
1178
|
+
|
|
1179
|
+
```js
|
|
1180
|
+
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
1181
|
+
|
|
1182
|
+
// Record a real run. The context's flowName names the flow in the trace.
|
|
1183
|
+
const { result, trace } = await recordEffect(checkoutFlow(input), {
|
|
1184
|
+
version: process.env.BUILD_ID,
|
|
1185
|
+
context: { flowName: 'checkout' }
|
|
1186
|
+
});
|
|
1187
|
+
|
|
1188
|
+
// Later, somewhere else, with nothing connected:
|
|
1189
|
+
await timeTravel(checkoutFlow, trace);
|
|
1190
|
+
```
|
|
1191
|
+
|
|
1192
|
+
```
|
|
1193
|
+
Replaying 'checkout' (3 recorded steps)
|
|
1194
|
+
Initial input: {
|
|
1195
|
+
"cartId": "cart_abc123",
|
|
1196
|
+
"promoCode": "FREE_YEAR_VIP"
|
|
1197
|
+
}
|
|
1198
|
+
Step 1: cmdFetchCart returned in 41.277ms {
|
|
1199
|
+
"totalAmount": "120.00"
|
|
1200
|
+
}
|
|
1201
|
+
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
1202
|
+
"isValid": true,
|
|
1203
|
+
"discountValue": 100
|
|
1204
|
+
}
|
|
1205
|
+
Step 3: cmdChargeCard threw in 181.507ms {
|
|
1206
|
+
"code": "invalid_amount",
|
|
1207
|
+
"name": "Error",
|
|
1208
|
+
"message": "Amount must be non-zero."
|
|
1209
|
+
}
|
|
1210
|
+
Replay finished with state: Failure
|
|
1211
|
+
Error: {
|
|
1212
|
+
"code": "invalid_amount",
|
|
1213
|
+
"name": "Error",
|
|
1214
|
+
"message": "Amount must be non-zero."
|
|
1215
|
+
}
|
|
1216
|
+
```
|
|
1217
|
+
|
|
1218
|
+
A production incident becomes a regression test, with no mocks:
|
|
1219
|
+
|
|
1220
|
+
```js
|
|
1221
|
+
it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
1222
|
+
const { result } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
1223
|
+
assert.equal(result.type, 'Failure');
|
|
1224
|
+
assert.equal(result.error.code, 'invalid_amount');
|
|
1225
|
+
});
|
|
1226
|
+
```
|
|
1227
|
+
|
|
1228
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
1229
|
+
|
|
1230
|
+
A step the trace lacks ends the replay in a `Failure` holding a `ReplayError`. A trace lacks a step that comes after the end of the recording, one a hook vetoed in production, and one the recorder dropped under `maxEntries`.
|
|
1231
|
+
|
|
1232
|
+
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
1233
|
+
|
|
1234
|
+
```js
|
|
1235
|
+
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
1236
|
+
const { result, unreached } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
1237
|
+
assert.equal(result.type, 'Success');
|
|
1238
|
+
assert.deepEqual(
|
|
1239
|
+
unreached.map((e) => e.command),
|
|
1240
|
+
['cmdChargeCard']
|
|
1241
|
+
);
|
|
1242
|
+
});
|
|
1243
|
+
```
|
|
1244
|
+
|
|
1245
|
+
For every other recording, `unreached` should be empty, so a fix that skips a step by accident fails the test. This works when the removed step is the last one the flow reaches. Remove a step from the middle and every later step moves to a different path, so the replay stops at a `TimeParadox`, and `unreached` lists the steps it never got to. [Proving a fix that changes the flow's shape](#proving-a-fix-that-changes-the-flows-shape) shows how to test such a fix.
|
|
1246
|
+
|
|
1247
|
+
A test can also run a whole flow with no recording at all, answering each Command itself; see [Testing a whole flow](#testing-a-whole-flow).
|
|
1248
|
+
|
|
1249
|
+
### Reading a trace
|
|
1250
|
+
|
|
1251
|
+
A trace is plain JSON. This one is from a checkout whose card charge timed out once and then went through:
|
|
1252
|
+
|
|
1253
|
+
```text
|
|
1254
|
+
{
|
|
1255
|
+
"flowName": "checkout",
|
|
1256
|
+
"version": "b42",
|
|
1257
|
+
"initialInput": { "cartId": "cart_1" },
|
|
1258
|
+
"context": { "flowName": "checkout" },
|
|
1259
|
+
"dropped": 0,
|
|
1260
|
+
"trace": [
|
|
1261
|
+
{ "command": "cmdFetchCart", "path": "0", "result": { "total": 120 }, "durationMs": 41.3 },
|
|
1262
|
+
{ "command": "cmdChargeCard", "path": "1r0/0", "threw": true,
|
|
1263
|
+
"error": { "__error": true, "name": "Error", "message": "gateway timeout", "code": "ETIMEDOUT" }, "durationMs": 5002.1 },
|
|
1264
|
+
{ "command": "cmdChargeCard", "path": "1r1/0", "result": { "chargeId": "ch_1" }, "durationMs": 181.5 }
|
|
1265
|
+
]
|
|
1266
|
+
}
|
|
1267
|
+
```
|
|
1268
|
+
|
|
1269
|
+
`trace` holds one entry per step, in the order the steps finished; [`recorder`](#recorderoptions) lists each kind of entry. A replay matches each step by its `path`, its position in the flow, and a `TimeParadox` names the path where the flow and the trace part ways:
|
|
1270
|
+
|
|
1271
|
+
- A number counts the Commands, `Retry`s and `Parallel`s the flow reached, from `0`. A `Success` or an `Ask` between them adds nothing.
|
|
1272
|
+
- `r` and a number is one attempt of the `Retry` at that position, `r0` being the first try, and `rf` is its `onExhausted` fallback.
|
|
1273
|
+
- `p` and a number is one branch of the `Parallel` at that position, from `p0`. The `Parallel` has an entry of its own, at a path such as `2p`, which records whether a branch cancelled the others.
|
|
1274
|
+
- `/` goes one level in. `1r1/0` is the first step of the second try of the `Retry` at position 1. `2p1/0` is the first step of the second branch of the `Parallel` at position 2.
|
|
1275
|
+
|
|
1276
|
+
### What a replay cannot check
|
|
1001
1277
|
|
|
1002
|
-
|
|
1278
|
+
A replay proves that the flow asks for the same Commands in the same order and handles the same answers. Every result comes from the recording, so it cannot check:
|
|
1003
1279
|
|
|
1004
|
-
|
|
1280
|
+
- **What the flow computes.** Fix how a refund amount is calculated and replay the bad run: the charge step still gets the old amount from the recording. The replay reports `Success`, returns the value from before the fix, and flags nothing. Test a value change against the pure step that computes it.
|
|
1281
|
+
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix. Catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
1282
|
+
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
1005
1283
|
|
|
1006
|
-
|
|
1284
|
+
When a Command's result changes shape, rename the Command. A trace matches it by name. So under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
1285
|
+
|
|
1286
|
+
## Recording in Production
|
|
1287
|
+
|
|
1288
|
+
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup. The two files imported below ship in the package's `examples` folder, `node_modules/pure-effect/examples`, and in the repository's [examples folder](https://github.com/aycangulez/pure-effect/blob/main/examples); copy them into your project.
|
|
1007
1289
|
|
|
1008
1290
|
```js
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
.get(sku)
|
|
1014
|
-
.then((price) => ({ ok: true, price }))
|
|
1015
|
-
.catch((error) => ({ ok: false, code: error.code, message: error.message }));
|
|
1016
|
-
},
|
|
1017
|
-
(r) => (r.ok ? Success(r.price) : fetchCachedPrice(sku))
|
|
1018
|
-
);
|
|
1291
|
+
import { randomUUID } from 'node:crypto';
|
|
1292
|
+
import { configureEffect } from 'pure-effect';
|
|
1293
|
+
import { recordingHooks } from './recording-example.js';
|
|
1294
|
+
import { telemetryHooks } from './opentelemetry-example.js';
|
|
1019
1295
|
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
|
|
1023
|
-
|
|
1024
|
-
|
|
1025
|
-
|
|
1296
|
+
configureEffect(
|
|
1297
|
+
telemetryHooks(),
|
|
1298
|
+
recordingHooks({
|
|
1299
|
+
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
1300
|
+
// Attached to each trace, so timeTravel warns when it replays one against another build.
|
|
1301
|
+
version: process.env.BUILD_ID,
|
|
1302
|
+
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
1303
|
+
redact: (value, name, kind) =>
|
|
1304
|
+
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
1305
|
+
maxEntries: 500,
|
|
1306
|
+
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
1307
|
+
})
|
|
1026
1308
|
);
|
|
1027
1309
|
```
|
|
1028
1310
|
|
|
1029
|
-
|
|
1311
|
+
What the example does with each run:
|
|
1030
1312
|
|
|
1031
|
-
|
|
1032
|
-
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1313
|
+
- **Which runs it keeps.** `keep` decides. By default it keeps a run that ended in a `Failure`, and discards a successful one after holding it in memory.
|
|
1314
|
+
- **When your own code throws.** A `TypeError` after an API changed the shape of its response, say, makes the run reject. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept, and replaying it reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded. That run did no I/O, so its input reproduces it.
|
|
1315
|
+
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out and marked `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
1316
|
+
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
1317
|
+
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
1318
|
+
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
1036
1319
|
|
|
1037
|
-
|
|
1320
|
+
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` or `effectGen` built, so build the outermost flow with one of them. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
1038
1321
|
|
|
1039
|
-
|
|
1322
|
+
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
1040
1323
|
|
|
1041
|
-
A
|
|
1324
|
+
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow. So neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes:
|
|
1042
1325
|
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1326
|
+
| Recorded | Replayed as |
|
|
1327
|
+
| ----------------------------------- | --------------------------------------------------- |
|
|
1328
|
+
| a money object or a database entity | a plain object, without its methods |
|
|
1329
|
+
| a `Date` | a string |
|
|
1330
|
+
| a `Map` or a `Set` | `{}` |
|
|
1331
|
+
| a `Buffer` | `{ type: 'Buffer', data }` |
|
|
1332
|
+
| `undefined` in an array | `null` |
|
|
1333
|
+
| an error returned inside a result | its own fields, such as `code`, without its message |
|
|
1047
1334
|
|
|
1048
|
-
A
|
|
1335
|
+
A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it. So a context holding a lazy client is left out whole.
|
|
1049
1336
|
|
|
1050
|
-
|
|
1337
|
+
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`. Rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
1051
1338
|
|
|
1052
|
-
|
|
1339
|
+
### Keeping secrets out of a trace
|
|
1053
1340
|
|
|
1054
|
-
|
|
1341
|
+
`redact` sees everything a trace stores: each result, each thrown error, the flow's input and the context, told apart by `kind`. It gets a copy in the form the trace stores, so changing it in place is safe, and what it returns is stored in that form too. A thrown `Error` arrives as a plain object holding its `name`, `message` and own fields, and any object returned for it replays as an `Error`.
|
|
1055
1342
|
|
|
1056
|
-
|
|
1343
|
+
Check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
1057
1344
|
|
|
1058
1345
|
```js
|
|
1059
|
-
|
|
1060
|
-
|
|
1346
|
+
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
1347
|
+
const mask = (value, field) =>
|
|
1348
|
+
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
1061
1349
|
|
|
1062
|
-
|
|
1063
|
-
|
|
1350
|
+
const redact = (value, name, kind) => {
|
|
1351
|
+
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
1352
|
+
if (kind === 'context') return mask(value, 'authToken');
|
|
1353
|
+
if (kind === 'error') return mask(value, 'attempted');
|
|
1354
|
+
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
1355
|
+
};
|
|
1356
|
+
recorder({ redact });
|
|
1064
1357
|
|
|
1065
|
-
|
|
1066
|
-
assert.equal(
|
|
1358
|
+
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
1359
|
+
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
1067
1360
|
```
|
|
1068
1361
|
|
|
1069
|
-
|
|
1362
|
+
A replay rebuilds the flow from the redacted input and context. So a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does. A context field an `Ask` reads should stay.
|
|
1070
1363
|
|
|
1071
|
-
|
|
1364
|
+
## Hooks
|
|
1072
1365
|
|
|
1073
|
-
|
|
1366
|
+
Hooks see every run, and recording, replay and the telemetry example are built on them. There are three: `onRun` wraps a whole run, `onStep` wraps each Command and each `Parallel`, and `onBeforeCommand` runs before each Command and can veto it by throwing. A hook that only watches must `await` what it wraps and return its result; [`configureEffect`](#configureeffectconfigs) lists each hook's arguments.
|
|
1074
1367
|
|
|
1075
|
-
|
|
1368
|
+
`configureEffect` adds a layer of hooks for the whole process and returns a function that removes it. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's:
|
|
1076
1369
|
|
|
1077
1370
|
```js
|
|
1078
|
-
const
|
|
1079
|
-
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
};
|
|
1371
|
+
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
1372
|
+
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
1373
|
+
remove();
|
|
1374
|
+
```
|
|
1083
1375
|
|
|
1084
|
-
|
|
1085
|
-
try {
|
|
1086
|
-
const cmdSaveUser = () => unreliableDb.saveUser(user);
|
|
1087
|
-
return yield* step(Command(cmdSaveUser));
|
|
1088
|
-
} catch (error) {
|
|
1089
|
-
// Runs when the save throws. It can log, and it has to let the error pass.
|
|
1090
|
-
throw error;
|
|
1091
|
-
}
|
|
1092
|
-
});
|
|
1376
|
+
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
1093
1377
|
|
|
1094
|
-
|
|
1095
|
-
|
|
1378
|
+
```
|
|
1379
|
+
runEffect(flow(input))
|
|
1380
|
+
│
|
|
1381
|
+
├─ telemetry.onRun first configuration: outermost
|
|
1382
|
+
│ └─ recording.onRun last configuration: innermost
|
|
1383
|
+
│ │
|
|
1384
|
+
│ │ for each Command:
|
|
1385
|
+
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
1386
|
+
│ ├─ recording.onBeforeCommand
|
|
1387
|
+
│ ├─ telemetry.onStep
|
|
1388
|
+
│ │ └─ recording.onStep closest to the Command
|
|
1389
|
+
│ │ └─ cmd()
|
|
1390
|
+
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
1391
|
+
│ ├─ telemetry.onStep returns from the inside out
|
|
1392
|
+
│ │
|
|
1393
|
+
│ ┌─ recording.onRun returns
|
|
1394
|
+
├─ telemetry.onRun returns
|
|
1096
1395
|
```
|
|
1097
1396
|
|
|
1098
|
-
|
|
1397
|
+
**Knowing which request a step belongs to.** `onStep` and `onRun` get no context, so a hook that logs per request finds the request the way the rest of your logging does. An `AsyncLocalStorage` scope entered around `runEffect` reaches every hook the run calls, and the recording example uses one to give each run its own recorder:
|
|
1099
1398
|
|
|
1100
1399
|
```js
|
|
1101
|
-
|
|
1102
|
-
|
|
1103
|
-
|
|
1104
|
-
|
|
1105
|
-
|
|
1106
|
-
|
|
1400
|
+
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
1401
|
+
|
|
1402
|
+
const requestScope = new AsyncLocalStorage();
|
|
1403
|
+
const stepLog = [];
|
|
1404
|
+
|
|
1405
|
+
// Logs each step with its request and duration, and returns what the step returned.
|
|
1406
|
+
const removeStepLog = configureEffect({
|
|
1407
|
+
onStep: async (name, type, op, path) => {
|
|
1408
|
+
const started = performance.now();
|
|
1409
|
+
try {
|
|
1410
|
+
return await op();
|
|
1411
|
+
} finally {
|
|
1412
|
+
const { requestId } = requestScope.getStore() ?? {};
|
|
1413
|
+
stepLog.push({ requestId, name, path, ms: performance.now() - started });
|
|
1414
|
+
}
|
|
1107
1415
|
}
|
|
1108
1416
|
});
|
|
1109
1417
|
|
|
1110
|
-
await
|
|
1418
|
+
await requestScope.run({ requestId: 'req_1' }, () => runEffect(registerUserFlow(input)));
|
|
1419
|
+
assert.deepEqual(
|
|
1420
|
+
stepLog.map((line) => [line.requestId, line.name]),
|
|
1421
|
+
[
|
|
1422
|
+
['req_1', 'cmdFindUser'],
|
|
1423
|
+
['req_1', 'cmdSaveUser']
|
|
1424
|
+
]
|
|
1425
|
+
);
|
|
1426
|
+
removeStepLog();
|
|
1111
1427
|
```
|
|
1112
1428
|
|
|
1113
|
-
|
|
1114
|
-
|
|
1115
|
-
### Mistakes it catches
|
|
1116
|
-
|
|
1117
|
-
| Written | What happens |
|
|
1118
|
-
| --- | --- |
|
|
1119
|
-
| `yield step(x)`, without the star | an `EffectTypeError`: write `yield*` |
|
|
1120
|
-
| `yield x` | an `EffectTypeError`: write `yield* step(x)` |
|
|
1121
|
-
| `yield* x`, without `step` | the engine's `TypeError`, which on V8 becomes an `EffectTypeError` saying to write `yield* step(x)` |
|
|
1122
|
-
| `const found = step(x)`, then `found.id` | an `EffectTypeError`: `step(...)` was used as a value |
|
|
1123
|
-
| `[...step(x)]` or `for (const v of step(x))` | an `EffectTypeError`: `step(...)` was iterated outside `yield*` |
|
|
1124
|
-
| `step(db.findUser(email))` | an `EffectTypeError`: put the call in a Command |
|
|
1125
|
-
| `effectGen(async function* () {})` | an `EffectTypeError`: write `function*` |
|
|
1126
|
-
|
|
1127
|
-
In TypeScript, most of these do not compile; see [TypeScript](#typescript-typed-errors-and-context).
|
|
1128
|
-
|
|
1129
|
-
## TypeScript: Typed Errors and Context
|
|
1429
|
+
A Command that `onBeforeCommand` vetoes never reaches `onStep`, so log a veto where you make it. In tests, call `configureEffect()` with no arguments in a `beforeEach`, so a test that fails before removing its hooks does not leave them installed for the next.
|
|
1130
1430
|
|
|
1131
|
-
|
|
1431
|
+
**Per-call hooks.** `runEffect`'s third argument takes hooks for that call only. They are merged with the configured ones by the same rules: the configured wrappers run outside them, and the configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
1132
1432
|
|
|
1133
|
-
|
|
1433
|
+
```
|
|
1434
|
+
inherit: true (default) inherit: false
|
|
1134
1435
|
|
|
1135
|
-
|
|
1436
|
+
onRun C.onRun (none)
|
|
1437
|
+
onBeforeCommand C.onBeforeCommand (none)
|
|
1438
|
+
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
1439
|
+
```
|
|
1136
1440
|
|
|
1137
|
-
|
|
1138
|
-
type ValidationError = 'invalid_email' | 'weak_password';
|
|
1139
|
-
type ApiError = 'network_timeout' | 'rate_limited';
|
|
1441
|
+
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
1140
1442
|
|
|
1141
|
-
|
|
1142
|
-
const submit = (input: { email: string }): Effect<{ id: number }, ApiError> => { ... };
|
|
1443
|
+
## Replay in Depth
|
|
1143
1444
|
|
|
1144
|
-
|
|
1145
|
-
if (result.type === 'Failure') {
|
|
1146
|
-
result.error; // 'invalid_email' | 'weak_password' | 'network_timeout' | 'rate_limited'
|
|
1147
|
-
}
|
|
1148
|
-
```
|
|
1445
|
+
What [Time-Travel Debugging](#time-travel-debugging) leaves out: proving a fix that changes a flow's shape, keeping a replay faithful, and replaying from storage other than a trace.
|
|
1149
1446
|
|
|
1150
|
-
|
|
1447
|
+
### Proving a fix that changes the flow's shape
|
|
1151
1448
|
|
|
1152
|
-
|
|
1449
|
+
A fix that removes or moves a step in the middle of a flow changes the path of every step after it, so its old trace stops at a `TimeParadox`. Here the incident is a checkout that tried to charge a free cart, and the fix skips the charge, so the save moves to where the charge was:
|
|
1153
1450
|
|
|
1154
|
-
|
|
1451
|
+
```js
|
|
1452
|
+
const fetchCart = (cartId) => {
|
|
1453
|
+
const cmdFetchCart = () => carts.find(cartId);
|
|
1454
|
+
return Command(cmdFetchCart);
|
|
1455
|
+
};
|
|
1155
1456
|
|
|
1156
|
-
|
|
1457
|
+
const chargeCart = (cart) => {
|
|
1458
|
+
const cmdChargeCard = () => gateway.charge(cart.total);
|
|
1459
|
+
return Command(cmdChargeCard, () => Success(cart));
|
|
1460
|
+
};
|
|
1157
1461
|
|
|
1158
|
-
|
|
1462
|
+
const saveOrder = (cart) => {
|
|
1463
|
+
const cmdSaveOrder = () => orderStore.save(cart);
|
|
1464
|
+
return Command(cmdSaveOrder);
|
|
1465
|
+
};
|
|
1159
1466
|
|
|
1160
|
-
|
|
1467
|
+
// Production ran this, and recorded the charge failing.
|
|
1468
|
+
const checkoutBefore = (cartId) => effectPipe(fetchCart, chargeCart, saveOrder)(cartId);
|
|
1469
|
+
const { trace: incident } = await recordEffect(checkoutBefore('cart_free'));
|
|
1161
1470
|
|
|
1162
|
-
|
|
1471
|
+
// The fix: no charge when there is nothing to charge.
|
|
1472
|
+
const chargeIfDue = (cart) => (cart.total === 0 ? Success(cart) : chargeCart(cart));
|
|
1473
|
+
const checkoutFixed = (cartId) => effectPipe(fetchCart, chargeIfDue, saveOrder)(cartId);
|
|
1163
1474
|
|
|
1164
|
-
|
|
1475
|
+
// Against the old trace, the fixed flow stops where the save took the charge's place.
|
|
1476
|
+
const { result: paradox } = await replayEffect(checkoutFixed(incident.initialInput), incident);
|
|
1477
|
+
assert.equal(paradox.error.name, 'TimeParadox');
|
|
1478
|
+
```
|
|
1165
1479
|
|
|
1166
|
-
|
|
1480
|
+
Prove such a fix in three parts. Test the decision that changed, as a pure step. Then walk the fixed flow from the incident's input, handing each step the answer production got for it, for as long as the fix leaves the steps as they were:
|
|
1167
1481
|
|
|
1168
|
-
```
|
|
1169
|
-
|
|
1482
|
+
```js
|
|
1483
|
+
import { commandName } from 'pure-effect';
|
|
1170
1484
|
|
|
1171
|
-
|
|
1172
|
-
|
|
1485
|
+
// The decision that changed: a free cart is not charged.
|
|
1486
|
+
assert.deepEqual(chargeIfDue({ id: 'cart_free', total: 0 }), Success({ id: 'cart_free', total: 0 }));
|
|
1173
1487
|
|
|
1174
|
-
|
|
1488
|
+
// Given the cart production fetched, the fixed flow saves the order without charging it.
|
|
1489
|
+
const fetchedCart = incident.trace.find((entry) => entry.command === 'cmdFetchCart').result;
|
|
1490
|
+
const first = checkoutFixed(incident.initialInput);
|
|
1491
|
+
assert.equal(commandName(first), 'cmdFetchCart');
|
|
1492
|
+
assert.equal(commandName(first.next(fetchedCart)), 'cmdSaveOrder');
|
|
1175
1493
|
```
|
|
1176
1494
|
|
|
1177
|
-
|
|
1495
|
+
The walk takes production's answers only up to the change. Past it, each recorded answer belongs to a step that has moved, so the trace has nothing more to say about the fixed flow. Last, once the fix ships, record the incident again, so its regression test is a plain replay once more.
|
|
1178
1496
|
|
|
1179
|
-
###
|
|
1180
|
-
|
|
1181
|
-
`effectGen` infers everything from the body. Each `yield*` is typed by its own step, the error union collects every step's errors and every `Failure` the body returns, and the context collects what every step reads:
|
|
1497
|
+
### Keeping a replay faithful
|
|
1182
1498
|
|
|
1183
|
-
|
|
1184
|
-
const registerUser = effectGen(function* (input: { email: string }) {
|
|
1185
|
-
const found = yield* step(findUser(input.email)); // User | null
|
|
1186
|
-
if (found) return Failure('EMAIL_TAKEN');
|
|
1187
|
-
return yield* step(saveUser(input));
|
|
1188
|
-
});
|
|
1189
|
-
// (input: { email: string }) => Effect<User, 'EMAIL_TAKEN' | the errors findUser and saveUser declare>
|
|
1190
|
-
```
|
|
1499
|
+
Put anything that varies between runs inside a Command. For replay, the current time or a random ID counts as I/O. In a Command it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay, and drifts from the trace.
|
|
1191
1500
|
|
|
1192
|
-
|
|
1501
|
+
To check a flow, record it, replay the trace, and compare the outcomes. A step that computes a fresh value shows up as a different final value, or as a `TimeParadox` if it changes which Commands run. A trace holds each result in the form JSON stores it, so this also catches a result your storage cannot keep, such as a `Date` the flow calls a method on:
|
|
1193
1502
|
|
|
1194
|
-
|
|
1503
|
+
```js
|
|
1504
|
+
const { result, trace } = await recordEffect(registerUserFlow(input));
|
|
1505
|
+
const { result: replayed } = await replayEffect(registerUserFlow(input), trace);
|
|
1506
|
+
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive storage
|
|
1507
|
+
```
|
|
1195
1508
|
|
|
1196
|
-
A
|
|
1509
|
+
A `Failure` compares the same way when the Command threw a plain `Error`. It comes back with its message, name, cause, custom properties and an `AggregateError`'s list of errors, even a `name` or `cause` set after it was created. An error of any other class, built-in ones included, comes back as a plain `Error` carrying that class's name. So `deepEqual` from `node:assert/strict` refuses the replayed `TypeError` that `fetch` rejects with when the network fails. Where a Command can throw one, compare `error.name` and `error.message` instead.
|
|
1197
1510
|
|
|
1198
|
-
|
|
1199
|
-
import assert from 'node:assert/strict';
|
|
1200
|
-
import { commandName } from 'pure-effect';
|
|
1201
|
-
import type { Effect } from 'pure-effect';
|
|
1511
|
+
### Replaying from your own storage
|
|
1202
1512
|
|
|
1203
|
-
|
|
1204
|
-
function assertCommand<T, E, C>(step: Effect<T, E, C>, name: string) {
|
|
1205
|
-
assert(step.type === 'Command', `expected ${name}, got ${step.type}`);
|
|
1206
|
-
assert.equal(commandName(step), name);
|
|
1207
|
-
return step;
|
|
1208
|
-
}
|
|
1513
|
+
`replayEffect` also accepts a resolver function instead of a trace, so you can replay from OpenTelemetry spans, a log pipeline, or a database table, not only from the JSON that `recorder` produces. Whatever you store needs each step's `path`, which `onStep` receives, and its result; the telemetry example records neither, since it keeps values off spans.
|
|
1209
1514
|
|
|
1210
|
-
|
|
1211
|
-
|
|
1515
|
+
```js
|
|
1516
|
+
// A resolver answers one question: what did production get back for this step? Look the step up by its path, its
|
|
1517
|
+
// position in the flow. `step.index` counts steps in the order they finished, which shifts around a Parallel.
|
|
1518
|
+
const outputs = new Map(mySpans.map((span) => [span.attributes.path, span.attributes.output]));
|
|
1519
|
+
const resolve = (step) => (outputs.has(step.path) ? { result: outputs.get(step.path) } : undefined);
|
|
1520
|
+
await replayEffect(checkoutFlow(input), resolve);
|
|
1212
1521
|
```
|
|
1213
1522
|
|
|
1214
|
-
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns, so where [Testing Without Mocks](#testing-without-mocks) passes `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
1215
|
-
|
|
1216
1523
|
## Why Pure Effect
|
|
1217
1524
|
|
|
1218
1525
|
**vs. Temporal and durable execution (Restate, Inngest):** The closest relatives, built on the same idea: record what every step returned and replay it. Those engines store the history on a server and resume workflows automatically after a crash. Pure Effect leaves storage and restarts to your application, and there is no infrastructure to run.
|
|
@@ -1223,25 +1530,6 @@ For a step that returns its Command directly, as in `ensureEmailAvailable(input)
|
|
|
1223
1530
|
|
|
1224
1531
|
**When to use something else:** If your code does little async I/O, or testing and debugging production are not problems for you, plain async/await is simpler.
|
|
1225
1532
|
|
|
1226
|
-
## Load Tests
|
|
1227
|
-
|
|
1228
|
-
Timings are from Node 22 on a MacBook Pro with M4 Pro CPU:
|
|
1229
|
-
|
|
1230
|
-
**A web service under load.** A `node:http` service ran signup and checkout flows, where each checkout was a `Parallel` of a retried stock reservation and a card charge, with the OpenTelemetry and recording examples installed.
|
|
1231
|
-
|
|
1232
|
-
- 800 requests at once, across 4 tenants: every response and every recorded trace belonged to its own request, and no tenant, user, or request ID crossed between runs.
|
|
1233
|
-
- The ~3,500 telemetry spans formed 800 request traces, with every span inside its own request's trace.
|
|
1234
|
-
- Side effects matched the outcomes: 320 charges and 320 orders, and 80 declined cards stopped 30 stock reservations that were still running.
|
|
1235
|
-
- 30,000 runs in a row kept memory between 10 and 11 MB, and left nothing running afterwards.
|
|
1236
|
-
- A background worker that ran 20,000 jobs in one flow, looping through a Command's `next`, finished in under ~400 ms without holding on to memory.
|
|
1237
|
-
- Stopping the server with flows still running, or clients hanging up mid-request: every flow that had started finished and was recorded, with no unhandled rejections.
|
|
1238
|
-
|
|
1239
|
-
**Batches and parallel work.**
|
|
1240
|
-
|
|
1241
|
-
- A settled `Parallel` over 200 records with `limit: 7` never had more than 7 records in flight, and returned the outcomes in order.
|
|
1242
|
-
- 5,000 branches in one `Parallel` ran in ~25 ms.
|
|
1243
|
-
- Running a Command through `runEffect` costs about 0.2 microseconds, against 0.06 for a plain `await`.
|
|
1244
|
-
|
|
1245
1533
|
## API Reference
|
|
1246
1534
|
|
|
1247
1535
|
Every `options` argument and hook configuration below throws for a name it does not read or a value it cannot use, so a misspelt option fails instead of running with its default. `Retry` and `Parallel` throw an `EffectTypeError` where they are built.
|
|
@@ -1260,7 +1548,7 @@ Returns `{ type: 'Failure', error }`. Stops the pipeline immediately.
|
|
|
1260
1548
|
|
|
1261
1549
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
1262
1550
|
|
|
1263
|
-
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch,
|
|
1551
|
+
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch, one that declares a parameter without a default value gets an `AbortSignal` that fires when a sibling fails; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1264
1552
|
- `next`: receives `cmd`'s result and returns the next Effect; by default `(result) => Success(result)`.
|
|
1265
1553
|
- `meta`: metadata passed to `onBeforeCommand`. A string `meta.name` names the Command.
|
|
1266
1554
|
|
|
@@ -1274,7 +1562,9 @@ Command(function cmdChargeCard() {
|
|
|
1274
1562
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
1275
1563
|
```
|
|
1276
1564
|
|
|
1277
|
-
Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1565
|
+
- Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1566
|
+
- Otherwise naming the function is enough. An arrow assigned to a `const` first, as in `const cmdChargeCard = () => api.charge()`, takes the `const`'s name.
|
|
1567
|
+
- An inline arrow does not. A replay cannot tell two `'anonymous'` Commands apart, so a refactor that swaps them replays without complaint, each handed the other's recorded result.
|
|
1278
1568
|
|
|
1279
1569
|
#### `commandName(command)`
|
|
1280
1570
|
|
|
@@ -1286,24 +1576,23 @@ Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nex
|
|
|
1286
1576
|
|
|
1287
1577
|
#### `Retry(effect, options?)`
|
|
1288
1578
|
|
|
1289
|
-
Returns `{ type: 'Retry', effect, options, next }`.
|
|
1579
|
+
Returns `{ type: 'Retry', effect, options, next }`. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
1290
1580
|
|
|
1291
|
-
- `options.
|
|
1292
|
-
- `options.delay`:
|
|
1293
|
-
- `options.backoff`:
|
|
1294
|
-
|
|
1295
|
-
An option set to `undefined` keeps its default.
|
|
1296
|
-
|
|
1297
|
-
- `options.onExhausted(error)`: Runs a fallback Effect when every attempt has failed, receiving `{ retryExhausted, lastError, attempts }`. The fallback's success feeds `next`; its failure propagates unwrapped. Per-use only. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
1581
|
+
- `options.retries`: tries after the first, a positive integer (default `3`), so `2` makes at most three calls. Anything else throws, `0` included: to handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
1582
|
+
- `options.delay`: milliseconds before the first retry, a finite number of 0 or more (default `100`).
|
|
1583
|
+
- `options.backoff`: the multiplier applied to the delay on each attempt, a finite number of 0 or more (default `1`, flat).
|
|
1584
|
+
- `options.onExhausted(error)`: a fallback Effect to run when every attempt has failed, given `{ retryExhausted, lastError, retries }`. Its success feeds `next`; its failure is returned unwrapped. Per use only.
|
|
1585
|
+
- An option set to `undefined` keeps its default.
|
|
1298
1586
|
|
|
1299
1587
|
#### `Parallel(effects, next?, options?)`
|
|
1300
1588
|
|
|
1301
|
-
Returns `{ type: 'Parallel', effects, next, options }
|
|
1302
|
-
|
|
1303
|
-
The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
1589
|
+
Returns `{ type: 'Parallel', effects, next, options }`. Runs the branches at the same time and hands `next` their values in order; by default `(values) => Success(values)`. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1304
1590
|
|
|
1591
|
+
- The first branch to fail cancels the others, and its `Failure` is the result, with `next` skipped. Which branch that is depends on timing, so it is recorded, and a replay returns the same one.
|
|
1592
|
+
- Each branch's Commands get an `AbortSignal` to pass to their I/O.
|
|
1593
|
+
- The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
1305
1594
|
- `limit`: the most branches running at once, a positive integer. Results and recorded paths stay in array order, so it changes only the pacing.
|
|
1306
|
-
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order
|
|
1595
|
+
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order, so no branch cancels the others or fails the `Parallel`. A bug still rejects the run: a step or `next` that throws, or returns something other than an Effect, is not a branch outcome.
|
|
1307
1596
|
|
|
1308
1597
|
### Building pipelines
|
|
1309
1598
|
|
|
@@ -1316,7 +1605,7 @@ Composes steps into a pipeline: each step gets the previous step's `Success` val
|
|
|
1316
1605
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
1317
1606
|
```
|
|
1318
1607
|
|
|
1319
|
-
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it
|
|
1608
|
+
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it. Passing the value through every step, as the [Quick Start](#quick-start) does, keeps the whole pipeline checked; see [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context).
|
|
1320
1609
|
|
|
1321
1610
|
In either language, a Command a step builds and does not return never runs, and any `Failure` it would have produced is lost:
|
|
1322
1611
|
|
|
@@ -1346,21 +1635,25 @@ Wraps an Effect for `yield*` inside `effectGen`, which gives back its value: `co
|
|
|
1346
1635
|
|
|
1347
1636
|
#### `runEffect(effect, context?, callConfig?)`
|
|
1348
1637
|
|
|
1349
|
-
Runs a flow and resolves with its `Success` or `Failure`.
|
|
1638
|
+
Runs a flow and resolves with its `Success` or `Failure`. See [Which Errors Are Data](#which-errors-are-data).
|
|
1350
1639
|
|
|
1351
1640
|
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
1352
1641
|
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the configured hooks, or used alone with `inherit: false`; see [Hooks](#hooks).
|
|
1353
|
-
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure
|
|
1642
|
+
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure` holding the error thrown; under a `Retry` that gave up, it holds `{ retryExhausted, lastError, retries }`.
|
|
1643
|
+
- A malformed flow, or a throw from a `next` or a pure step, is a bug. The call that builds the flow throws it before the first Command, and `runEffect` rejects with it after.
|
|
1354
1644
|
|
|
1355
1645
|
#### `configureEffect(...configs)`
|
|
1356
1646
|
|
|
1357
|
-
Adds a layer of hooks for the whole process and returns a function that removes that layer only.
|
|
1647
|
+
Adds a layer of hooks for the whole process and returns a function that removes that layer only. See [Hooks](#hooks).
|
|
1358
1648
|
|
|
1359
|
-
-
|
|
1649
|
+
- Several configurations passed to one call form one layer.
|
|
1650
|
+
- `configureEffect()` with no arguments removes every layer, and a call whose arguments are all `undefined` changes nothing.
|
|
1651
|
+
- Throws a `TypeError`, before installing anything, for a configuration that is not an object, a key other than the three below (`retry` included), or a hook that is not a function (`null` included).
|
|
1652
|
+
- `onRun(effect, pipeline, flowName, initialInput)`: wraps the whole run once, retries included. It must `await pipeline()` and return its result. `initialInput` is what the flow was called with when `effectPipe` or `effectGen` built it, and `undefined` otherwise; pass it on to any hook it calls.
|
|
1360
1653
|
- `onStep(name, type, op, path)`: wraps each Command, and each `Parallel` with `name` and `type` both `'Parallel'`. It must `await op()` and return its result, and pass `path` on to any hook it calls, since a replay matches steps on it.
|
|
1361
1654
|
- For a Command, `op()` returns a promise, even for a synchronous function. Returning a value without calling `op()` answers for the Command, which is how replay works, and throwing without calling it counts as the Command failing.
|
|
1362
|
-
- A throw after `op()` succeeded, or `undefined`
|
|
1363
|
-
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }
|
|
1655
|
+
- A throw after `op()` succeeded, or `undefined` in place of its value (a forgotten `return` or `await`), is a bug in the hook: the run rejects, and `Retry` does not run the Command again.
|
|
1656
|
+
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }`. A branch that threw adds `threw: true`.
|
|
1364
1657
|
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it. A vetoed Command leaves no entry in a trace, so a replay of that run fails where it was vetoed; see [Errors no branch returned](#errors-no-branch-returned).
|
|
1365
1658
|
|
|
1366
1659
|
### Recording and replay
|
|
@@ -1370,7 +1663,7 @@ Adds a layer of hooks for the whole process and returns a function that removes
|
|
|
1370
1663
|
Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run, passed as `runEffect(flow, context, { onStep })`. Give each run its own recorder: one shared by a whole application mixes requests into one trace, which a replay refuses. `recordEffect` and the [recording example](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js) both do this for you.
|
|
1371
1664
|
|
|
1372
1665
|
- `onStep`: the hook. Recording never changes a run's outcome.
|
|
1373
|
-
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. `path
|
|
1666
|
+
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. A replay matches each on its `path`, the step's position in the flow.
|
|
1374
1667
|
|
|
1375
1668
|
```text
|
|
1376
1669
|
{ command, path, result, durationMs } a Command that returned
|
|
@@ -1381,24 +1674,29 @@ Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run,
|
|
|
1381
1674
|
because 'redact' threw on it or the 'copy' failed
|
|
1382
1675
|
```
|
|
1383
1676
|
|
|
1384
|
-
-
|
|
1677
|
+
- An older entry with `error` and no `threw` still counts as a throw.
|
|
1678
|
+
- `toTrace(meta)`: the entries as a trace, with `meta`'s `initialInput`, `context`, `flowName` and `version`, copied into the form JSON stores when it is called. If a Command can change the input or the context, as an ORM save that adds an id does, call it before the run as well, as `recordEffect` does.
|
|
1385
1679
|
- `options.redact(value, name, kind)`: returns what the trace stores in place of `value`. `kind` is `'result'`, `'error'`, `'initialInput'` or `'context'`, and `name` is the Command's name, or the kind. See [Keeping secrets out of a trace](#keeping-secrets-out-of-a-trace).
|
|
1386
1680
|
- `options.maxEntries`: the most entries a trace keeps; the rest are counted in `dropped`, and a replay stops at the first step the trace lacks.
|
|
1387
1681
|
- `options.stack`: record stack traces for thrown errors (off by default).
|
|
1388
|
-
- A value `redact` throws on, or one JSON cannot hold, is left out and marked `unrecorded
|
|
1682
|
+
- A value `redact` throws on, or one JSON cannot hold, is left out and marked `unrecorded`: on its entry, or for the input and context in the trace's `unrecorded`, as in `{ context: 'copy' }`. A replay stops there with a `ReplayError` rather than hand the flow a stand-in. See [Recording in Production](#recording-in-production) for what a copy keeps.
|
|
1389
1683
|
|
|
1390
|
-
#### `recordEffect(
|
|
1684
|
+
#### `recordEffect(effect, options?)`
|
|
1391
1685
|
|
|
1392
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts
|
|
1686
|
+
Runs a flow for real while recording, returning `{ result, trace }`. The trace holds the flow's input when `effectPipe` or `effectGen` built it, so `timeTravel` can build the flow again. Accepts `recorder` options plus `context` and `version`. For tests and scripts; to record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
1393
1687
|
|
|
1394
1688
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
1395
1689
|
|
|
1396
|
-
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for.
|
|
1690
|
+
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for. See [Time-Travel Debugging](#time-travel-debugging).
|
|
1397
1691
|
|
|
1398
|
-
-
|
|
1399
|
-
- A
|
|
1400
|
-
- `
|
|
1401
|
-
-
|
|
1692
|
+
- A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`.
|
|
1693
|
+
- A malformed trace rejects with a `ReplayError`.
|
|
1694
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver: a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of. See [Testing a whole flow](#testing-a-whole-flow) and [Replaying from your own storage](#replaying-from-your-own-storage).
|
|
1695
|
+
- Any other answer from a resolver, `null` included, or a throw from it, rejects the replay. With a resolver, `unreached` is absent.
|
|
1696
|
+
- A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`. Answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1697
|
+
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`; a trace whose context could not be recorded refuses to replay without one.
|
|
1698
|
+
- `options.onMissing`: `'throw'` (default) stops at a step the trace lacks; `'execute'` runs its Command for real, so use it only where Commands reach test doubles or only read.
|
|
1699
|
+
- A trace that dropped entries refuses `'execute'`. A flow that changed shape, as when a Command is newly wrapped in `Retry` or moved into a new `Parallel` branch, stops at a `TimeParadox` even under `'execute'`.
|
|
1402
1700
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
1403
1701
|
- `options.hooks` (default `false`): run the replay inside the configured hooks, a configured recorder included, so they see the replayed steps. Off, a replay cannot reach a telemetry backend or a trace sink.
|
|
1404
1702
|
- `options.onResolved(step, outcome)`: observes each replayed step. A throw stops the replay, and `replayEffect` rejects with it; it never changes an outcome.
|
|
@@ -1406,19 +1704,23 @@ Runs a flow with each Command answered from a trace instead of run, and returns
|
|
|
1406
1704
|
|
|
1407
1705
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
1408
1706
|
|
|
1409
|
-
Replays a trace and narrates each step with its recorded duration
|
|
1707
|
+
Replays a trace and narrates each step with its recorded duration. Returns `{ result, unreached }`, as `replayEffect` does for a trace.
|
|
1708
|
+
|
|
1709
|
+
- Names any recorded steps that were never reached.
|
|
1710
|
+
- Warns when the trace's `version` differs from `options.version`, and when steps are named `'anonymous'`.
|
|
1711
|
+
- Refuses a trace whose input could not be recorded, since the flow is rebuilt from it.
|
|
1712
|
+
- `options.context` overrides the trace's; `options.log` replaces `console.log`.
|
|
1410
1713
|
|
|
1411
1714
|
## Limitations
|
|
1412
1715
|
|
|
1413
|
-
- **`Retry` repeats everything it wraps, including a Command's `next`.**
|
|
1414
|
-
- **Cancelling a `Parallel` branch cannot stop
|
|
1415
|
-
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.**
|
|
1416
|
-
- **`redact` covers a trace, not a result.** A `Failure` holds the full error,
|
|
1417
|
-
- **Replay checks the path, not the values.** Every
|
|
1418
|
-
- **A trace keeps data, not objects.** A
|
|
1419
|
-
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two
|
|
1420
|
-
- **Configuration belongs to one copy of the library.**
|
|
1421
|
-
- **A Command needs a name.** `meta.name`
|
|
1716
|
+
- **`Retry` repeats everything it wraps, including a Command's `next`.** Commands that already succeeded run again, and a retried Command whose `next` continues the flow repeats the rest of the flow with it. Wrap only the Command that fails, and continue in a later step; see [Retrying Transient Failures](#retrying-transient-failures). A function that catches its own error is never retried; see [Which Errors Are Data](#which-errors-are-data).
|
|
1717
|
+
- **Cancelling a `Parallel` branch cannot stop a call already running.** A cancelled branch starts no new Commands, but a function that ignores its `AbortSignal`, or is never given one, runs to the end. So a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch before it returns; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1718
|
+
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.** So `Command(listUsers)`, where `listUsers(page)` treats `page` as optional, gets the signal as its page inside a `Parallel` and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1719
|
+
- **`redact` covers a trace, not a result.** A `Failure` holds the full error, and an error can quote what a Command sent, as a database's duplicate-key error quotes the key. Keeping it out of your logs is up to you.
|
|
1720
|
+
- **Replay checks the path, not the values.** Every result comes from the recording, so a replay cannot check a fix that changes what a flow computes. A step removed from the middle of a flow stops the replay at a `TimeParadox`; see [What a replay cannot check](#what-a-replay-cannot-check).
|
|
1721
|
+
- **A trace keeps data, not objects.** A replayed value is what JSON stores: a class instance loses its methods, a `Date` becomes a string, and a thrown error comes back as a plain `Error`, so check `error.name` rather than `instanceof`. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|
|
1722
|
+
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two requests does not, because a trace records one flow. Nor does a race between branches that share state: branches replay with the results they saw, not the order they ran in. A cancelled generator branch can stop one step later in a replay; see [Replaying a cancelled `Parallel`](#replaying-a-cancelled-parallel).
|
|
1723
|
+
- **Configuration belongs to one copy of the library.** Hooks live in module state, so two copies in one process (a dual ESM and CJS setup, two versions in the dependency tree, or a worker thread) do not share them, and nothing warns you.
|
|
1724
|
+
- **A Command needs a name that survives minification.** `meta.name` does. `cmd.name` does not, unless the minifier keeps function names.
|
|
1422
1725
|
- **A generator flow's body can run more than once,** and code in it that changes something outside it, such as a counter or a log line, does so again; see [What runs once, and what runs again](#what-runs-once-and-what-runs-again).
|
|
1423
|
-
- **A generator
|
|
1424
|
-
- **A generator body's `finally` does not always run.** It does not run when a bug or a `TimeParadox` rejects the run. In a `Parallel` where one branch threw, another branch that stops for its own reason before the rejection reaches it may still run its `catch` and `finally`, depending on timing. A step in a body's `finally` is not supported: it runs when the body ends normally and is refused when a step stopped the body, but if an error is passing through the body when its `finally` reaches a step, and the flow is cancelled before that step starts, the error is lost. Put cleanup in a `try`/`finally` around `runEffect`.
|
|
1726
|
+
- **A generator body's `finally` does not always run.** It does not run when a bug or a `TimeParadox` rejects the run. Put cleanup in a `try`/`finally` around `runEffect`; see [How a generator flow stops](#how-a-generator-flow-stops).
|