pure-effect 0.20.0 → 0.20.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -1
- package/README.md +711 -438
- package/RULES.md +43 -0
- package/examples/recording-example.js +6 -3
- package/index.d.ts +3 -1
- package/index.js +34 -19
- package/package.json +4 -3
package/README.md
CHANGED
|
@@ -6,20 +6,20 @@
|
|
|
6
6
|
|
|
7
7
|
- Replay a production failure locally, with no database and no network
|
|
8
8
|
- No mocks needed to test async pipelines
|
|
9
|
-
-
|
|
9
|
+
- Read request values, such as the tenant, from any step with `Ask`, without passing them along
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
|
-
- OpenTelemetry-ready via lifecycle hooks
|
|
11
|
+
- OpenTelemetry-ready via lifecycle hooks, with a reference integration to copy
|
|
12
12
|
- Zero dependencies, about 11 KB minified and gzipped
|
|
13
|
-
- Works in JavaScript, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
13
|
+
- Works in JavaScript on Node 22 or later, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
16
16
|
|
|
17
|
-
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [
|
|
18
|
-
- **
|
|
19
|
-
- **
|
|
17
|
+
- **Getting started:** [Installation](#installation) · [The Core](#the-core) · [Quick Start](#quick-start) · [Coming from async/await](#coming-from-asyncawait) · [Writing Flows as Generators](#writing-flows-as-generators) · [Testing Without Mocks](#testing-without-mocks) · [Rules for Writing a Flow](#rules-for-writing-a-flow)
|
|
18
|
+
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Writing Loops](#writing-loops) · [Which Errors Are Data](#which-errors-are-data) · [Generator Flows in Depth](#generator-flows-in-depth) · [Parallel in Depth](#parallel-in-depth) · [TypeScript](#typescript-typed-errors-and-context)
|
|
19
|
+
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production) · [Hooks](#hooks) · [Replay in Depth](#replay-in-depth)
|
|
20
20
|
- **Reference:** [Why Pure Effect](#why-pure-effect) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
21
|
|
|
22
|
-
The
|
|
22
|
+
The seven sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: retries, loops, steps that run at the same time, TypeScript, or recording.
|
|
23
23
|
|
|
24
24
|
## Installation
|
|
25
25
|
|
|
@@ -27,22 +27,30 @@ The six sections under Getting started are enough to write and test a flow. Come
|
|
|
27
27
|
npm install pure-effect
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
+
## The Core
|
|
31
|
+
|
|
32
|
+
Every flow is built from these:
|
|
33
|
+
|
|
34
|
+
| Piece | What it is |
|
|
35
|
+
| --- | --- |
|
|
36
|
+
| `Success(value)` | A step that succeeded, carrying its value on to the next step. |
|
|
37
|
+
| `Failure(error)` | A step that stops the flow, carrying its error to the caller. |
|
|
38
|
+
| `Command(cmd, next)` | A call to make, and a function that turns its answer into the next step. |
|
|
39
|
+
| `effectPipe(...)`, `effectGen(...)` | Join steps into a flow: as a pipeline, or written like `async` code with `yield* step(...)`. |
|
|
40
|
+
| `runEffect(flow)` | Runs the flow, the only place the calls are made, and returns a `Success` or a `Failure`. |
|
|
41
|
+
|
|
42
|
+
Everything else adds to these. `Ask` reads request values, `Retry` and `Parallel` wrap steps, and recording and replay watch `runEffect` make its calls.
|
|
43
|
+
|
|
30
44
|
## Quick Start
|
|
31
45
|
|
|
32
|
-
|
|
46
|
+
Five steps take you from a plain `async` function to a tested flow. The example registers a user.
|
|
33
47
|
|
|
34
|
-
|
|
35
|
-
import { Success, Failure, Command, effectPipe, runEffect } from 'pure-effect';
|
|
48
|
+
**1. Describe each call as a Command.** A Command holds a call to make, and a `next` function that turns its answer into the next step. Building one makes no call.
|
|
36
49
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
if (!input.email.includes('@')) return Failure('Invalid email.');
|
|
40
|
-
if (input.password.length < 8) return Failure('Password too short.');
|
|
41
|
-
return Success(input);
|
|
42
|
-
};
|
|
50
|
+
```js
|
|
51
|
+
import { Success, Failure, Command, Retry, effectPipe, effectGen, step, runEffect } from 'pure-effect';
|
|
43
52
|
|
|
44
|
-
//
|
|
45
|
-
// Name the commands: traces, replay matching, and telemetry spans use the names.
|
|
53
|
+
// Name each Command's function: traces, replay and telemetry refer to the Command by it.
|
|
46
54
|
const ensureEmailAvailable = (input) => {
|
|
47
55
|
const cmdFindUser = () => db.findUser(input.email);
|
|
48
56
|
const next = (found) => (found ? Failure('Email already in use.') : Success(input));
|
|
@@ -54,27 +62,27 @@ const saveUser = (input) => {
|
|
|
54
62
|
const cmdSaveUser = () => db.saveUser(input);
|
|
55
63
|
return Command(cmdSaveUser);
|
|
56
64
|
};
|
|
65
|
+
```
|
|
57
66
|
|
|
58
|
-
|
|
59
|
-
const registerUserFlow = (input) => effectPipe(validateRegistration, ensureEmailAvailable, saveUser)(input);
|
|
67
|
+
Keep a Command's function to the one call. Tests never run it, so nothing else in it gets tested.
|
|
60
68
|
|
|
61
|
-
|
|
62
|
-
async function registerUser(input) {
|
|
63
|
-
const result = await runEffect(registerUserFlow(input));
|
|
69
|
+
**2. Join the steps into a flow.** A step with no I/O returns `Success` or `Failure` itself. `effectPipe` hands each step the value the step before it produced.
|
|
64
70
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
+
```js
|
|
72
|
+
// Pure. No I/O, instantly testable.
|
|
73
|
+
const validateRegistration = (input) => {
|
|
74
|
+
if (!input.email.includes('@')) return Failure('Invalid email.');
|
|
75
|
+
if (input.password.length < 8) return Failure('Password too short.');
|
|
76
|
+
return Success(input);
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
// Validation runs as the flow is built. No I/O happens until runEffect.
|
|
80
|
+
const registerUserFlow = (input) => effectPipe(validateRegistration, ensureEmailAvailable, saveUser)(input);
|
|
71
81
|
```
|
|
72
82
|
|
|
73
83
|
The same flow can be written as a generator, with `yield* step(...)` where `async` code writes `await`. [Writing Flows as Generators](#writing-flows-as-generators) covers what to know.
|
|
74
84
|
|
|
75
85
|
```js
|
|
76
|
-
import { effectGen, step } from 'pure-effect';
|
|
77
|
-
|
|
78
86
|
const registerUserFlowGen = effectGen(function* (input) {
|
|
79
87
|
yield* step(validateRegistration(input));
|
|
80
88
|
yield* step(ensureEmailAvailable(input));
|
|
@@ -82,103 +90,72 @@ const registerUserFlowGen = effectGen(function* (input) {
|
|
|
82
90
|
});
|
|
83
91
|
```
|
|
84
92
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop. A generator flow builds the same Commands: `effectGen` runs your function up to each `yield*` and continues it with the answer.
|
|
88
|
-
|
|
89
|
-
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
90
|
-
|
|
91
|
-
Building a flow runs the pure steps right away: `registerUserFlow(badInput)` returns the validation `Failure` synchronously, so the tests below do not need `runEffect`. Only Commands need it.
|
|
93
|
+
**3. Return a `Failure`, and throw only when trying again could help.** The two ways to fail do different things:
|
|
92
94
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
A flow is plain objects, so a test can check what it will do without running it. Validation runs as soon as the flow is built, so testing it needs nothing else:
|
|
95
|
+
- An outcome the flow decides, such as an invalid email or one already in use, is a `Failure` a step returns. The flow stops there, and nothing retries it.
|
|
96
|
+
- A call that broke, such as a timeout or a dropped connection, is an error the Command's function throws. `Retry` makes the call again:
|
|
96
97
|
|
|
97
98
|
```js
|
|
98
|
-
|
|
99
|
-
|
|
99
|
+
// Up to three calls. Wrap only the Command: Retry runs everything it wraps again.
|
|
100
|
+
const saveUserWithRetry = (input) => Retry(saveUser(input), { retries: 2 });
|
|
100
101
|
```
|
|
101
102
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
```js
|
|
105
|
-
const input = { email: 'test@test.com', password: 'password123' };
|
|
106
|
-
|
|
107
|
-
// The email is free, so the step passes the input on.
|
|
108
|
-
assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
103
|
+
Either way, the caller gets a `Failure` once the flow gives up. A call that throws outside a `Retry` ends the run in a `Failure` holding the error it threw.
|
|
109
104
|
|
|
110
|
-
|
|
111
|
-
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
112
|
-
```
|
|
105
|
+
After a `Retry` gives up, the `Failure` holds `{ retryExhausted, lastError, retries }`, with the last error thrown as `lastError`. When a call throws an error the flow should decide on, such as a declined card, catch it inside the Command's function and return it; [Which Errors Are Data](#which-errors-are-data) shows how.
|
|
113
106
|
|
|
114
|
-
|
|
107
|
+
**4. Test it without mocks.** A flow is plain objects. Hand a step the answer you choose and check what it returns:
|
|
115
108
|
|
|
116
109
|
```js
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
assert.equal(commandName(step1), 'cmdFindUser');
|
|
121
|
-
|
|
122
|
-
const step2 = step1.next(null); // pretend no user was found
|
|
123
|
-
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
124
|
-
// The database was never touched.
|
|
110
|
+
// The email is taken, so the flow stops here. No database was touched.
|
|
111
|
+
const taken = ensureEmailAvailable({ email: 'taken@test.com', password: 'password123' }).next({ id: 1 });
|
|
112
|
+
assert.deepEqual(taken, Failure('Email already in use.'));
|
|
125
113
|
```
|
|
126
114
|
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
130
|
-
|
|
131
|
-
To run the whole flow instead of walking it, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command by name. Nothing is called and no retry waits; answering `{ error }` makes that Command throw, which is how a test reaches a `Retry`'s fallback:
|
|
132
|
-
|
|
133
|
-
```js
|
|
134
|
-
import { replayEffect } from 'pure-effect';
|
|
135
|
-
|
|
136
|
-
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
137
|
-
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
138
|
-
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
139
|
-
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
140
|
-
```
|
|
115
|
+
[Testing Without Mocks](#testing-without-mocks) shows each kind of test, and how to run a whole flow with no I/O.
|
|
141
116
|
|
|
142
|
-
|
|
117
|
+
**5. Run it.** `runEffect` makes the calls and returns a `Success` or a `Failure`. It is the only place a call is made, so it sits at the edge of your app:
|
|
143
118
|
|
|
144
119
|
```js
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
// A real run returns its result, and a trace of what each Command returned.
|
|
148
|
-
const { result: live, trace } = await recordEffect(registerUserFlow(input));
|
|
120
|
+
async function registerUser(input) {
|
|
121
|
+
const result = await runEffect(registerUserFlow(input));
|
|
149
122
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
123
|
+
if (result.type === 'Success') {
|
|
124
|
+
console.log('User created:', result.value);
|
|
125
|
+
} else {
|
|
126
|
+
console.error('Error:', result.error);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
153
129
|
```
|
|
154
130
|
|
|
155
|
-
[Time-Travel Debugging](#time-travel-debugging)
|
|
156
|
-
|
|
157
|
-
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them, with the services passed in as [Passing Runtime Context](#passing-runtime-context) shows.
|
|
158
|
-
|
|
159
|
-
In TypeScript, a walk needs a check at each step that the step is a Command; [Testing a flow in TypeScript](#testing-a-flow-in-typescript) has a helper that makes that one line.
|
|
131
|
+
That is a whole flow. [Rules for Writing a Flow](#rules-for-writing-a-flow) lists the twelve rules that cover most of what goes wrong, and [Time-Travel Debugging](#time-travel-debugging) shows how to record a run in production and replay it on your machine. In TypeScript, [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context) covers typing a flow's errors and context, handling every error, and testing a flow.
|
|
160
132
|
|
|
161
133
|
## Coming from async/await
|
|
162
134
|
|
|
163
|
-
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command: which call to make, and what to do with the answer. Each thing you would write with `async`/`await` has an equivalent
|
|
135
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command: which call to make, and what to do with the answer. Each thing you would write with `async`/`await` has an equivalent.
|
|
136
|
+
|
|
137
|
+
Where the two styles differ:
|
|
164
138
|
|
|
165
139
|
| `async`/`await` | Generator flow | Pipeline |
|
|
166
140
|
| --- | --- | --- |
|
|
167
|
-
| `await db.findUser(email)` | `yield* step(findUser(email))` | `Command(
|
|
141
|
+
| `await db.findUser(email)` | `yield* step(findUser(email))` | `Command(cmdFindUser, next)` |
|
|
168
142
|
| the code after the `await` | the next `yield*` | the Command's `next` |
|
|
169
143
|
| an `if` on a result | a step that returns `Success` or `Failure` | a branch in `next` |
|
|
170
144
|
| `return value` | `return value` | `Success(value)` |
|
|
171
145
|
| `throw` to stop | a step that returns `Failure(error)` | `Failure(error)` |
|
|
172
|
-
| `try`/`catch` around a call | a catch inside the Command's function | the same |
|
|
173
|
-
| a loop that retries a call | `yield* step(Retry(...))` | `Retry(...)` |
|
|
174
|
-
| `await Promise.all([...])` | `yield* step(Parallel([...]))` | `Parallel([...])` |
|
|
175
146
|
| a `for` loop with an `await` inside | the same loop, with `yield*` | `Parallel(list.map(...), { limit: 1 })` |
|
|
176
|
-
| a loop that keeps going past a failure | `Parallel(..., { settled: true })` | the same |
|
|
177
147
|
| a call that needs two earlier results | two variables | a named step that passes both on |
|
|
178
|
-
| a value from the request | `yield* step(Ask())` | `Ask` |
|
|
179
|
-
| `Date.now()` or a random ID | a Command | a Command |
|
|
180
148
|
|
|
181
|
-
|
|
149
|
+
The rest are the same in both styles, and a generator flow runs each one with `yield* step(...)`:
|
|
150
|
+
|
|
151
|
+
- `try`/`catch` around a call: a catch inside the Command's function
|
|
152
|
+
- a loop that retries a call: `Retry`
|
|
153
|
+
- `await Promise.all([...])`: `Parallel`
|
|
154
|
+
- a loop that keeps going past a failure: `Parallel` with `settled: true`
|
|
155
|
+
- a value from the request: `Ask`
|
|
156
|
+
- `Date.now()` or a random ID: a Command
|
|
157
|
+
|
|
158
|
+
`cmdFindUser` is the named function `() => db.findUser(email)`, and `findUser` stands for a step that returns the Command, as `ensureEmailAvailable` does in the Quick Start. A `catch` around `yield*` can log and rethrow, but it cannot recover; see [Which Errors Are Data](#which-errors-are-data).
|
|
182
159
|
|
|
183
160
|
One rule is behind both columns: each decision lives in a small step function that receives the answer it depends on and returns `Success` or `Failure`, so a test can call it directly. A pipeline joins those steps with `effectPipe`; a generator flow puts them in order with `yield*`. An `if` around a `yield*` works, but it can be tested only by walking the whole flow, so keep the body to putting steps in order. A `for` loop that runs a step for each item decides nothing, so it is fine.
|
|
184
161
|
|
|
@@ -193,30 +170,9 @@ async function registerUserAsync(input) {
|
|
|
193
170
|
}
|
|
194
171
|
```
|
|
195
172
|
|
|
196
|
-
The difference shows in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace
|
|
197
|
-
|
|
198
|
-
```js
|
|
199
|
-
import { mock } from 'node:test';
|
|
200
|
-
|
|
201
|
-
const input = { email: 'test@test.com', password: 'password123' };
|
|
202
|
-
|
|
203
|
-
mock.method(db, 'findUser', async () => ({ id: 1 }));
|
|
204
|
-
const saveUser = mock.method(db, 'saveUser', async (user) => user);
|
|
205
|
-
|
|
206
|
-
await assert.rejects(registerUserAsync(input), { message: 'Email already in use.' });
|
|
207
|
-
assert.equal(saveUser.mock.callCount(), 0);
|
|
208
|
-
mock.restoreAll();
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
The flow's test of the same branch hands one step the answer, with nothing to replace, wait for, or put back:
|
|
212
|
-
|
|
213
|
-
```js
|
|
214
|
-
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
Each step is a place a test can start, so a test reaches a branch by calling the step it lives in. A test of an `async` function has to run everything before the branch, with every call on the way replaced. The mocks do check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
173
|
+
The difference shows in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace `db.findUser` and `db.saveUser` with mocks, run the function, and put them back. The flow's test hands the one step the answer, `ensureEmailAvailable(input).next({ id: 1 })`, with nothing to replace or put back. The mocks do check one thing that test cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
218
174
|
|
|
219
|
-
You can switch one function at a time. Keep its name and parameters, run the flow in its body, and turn a `Failure` back into a throw, so the code that calls it does not change
|
|
175
|
+
You can switch one function at a time. Keep its name and parameters, run the flow in its body, and turn a `Failure` back into a throw, so the code that calls it does not change:
|
|
220
176
|
|
|
221
177
|
```js
|
|
222
178
|
// Callers still await registerUserAsync and catch what it throws.
|
|
@@ -230,11 +186,13 @@ async function registerUserAsync(input) {
|
|
|
230
186
|
await assert.rejects(registerUserAsync({ email: 'bad-email', password: '123' }), { message: 'Invalid email.' });
|
|
231
187
|
```
|
|
232
188
|
|
|
189
|
+
If the flow has a `Retry`, a `Failure` from one that gave up holds `{ retryExhausted, lastError, retries }`; throw its `lastError` to rethrow what the call threw, as the old function did.
|
|
190
|
+
|
|
233
191
|
A loop becomes a step that gets the list and a `Parallel` over it, as the table shows; [Writing Loops](#writing-loops) translates one.
|
|
234
192
|
|
|
235
193
|
## Writing Flows as Generators
|
|
236
194
|
|
|
237
|
-
A generator flow is written like `async` code, with `yield* step(...)` where `async` code writes `await`. `effectGen` turns it into the same Commands a pipeline builds, so recording, replay, `Retry`, `Parallel` and the tests
|
|
195
|
+
A generator flow is written like `async` code, with `yield* step(...)` where `async` code writes `await`. `effectGen` turns it into the same Commands a pipeline builds, so recording, replay, `Retry`, `Parallel` and the tests in [Testing Without Mocks](#testing-without-mocks) work the same for both. The Quick Start's `registerUserFlowGen` is one. The same steps in the same order record the same paths in either form, so a trace recorded from a pipeline replays against its generator version. A loop written as a `for` loop rather than a `Parallel` records differently; see [Writing Loops](#writing-loops).
|
|
238
196
|
|
|
239
197
|
### Where a generator helps
|
|
240
198
|
|
|
@@ -261,315 +219,103 @@ A pipeline would need a step that passes the user and the workspace on together;
|
|
|
261
219
|
|
|
262
220
|
The body can run more than once: for each `Retry` attempt and each run, as an `async` function does when called again, and when a test walks a second branch. So keep it to its own variables, and get `Date.now()` or a random ID from a Command, so a replay gets the recorded one. [Generator Flows in Depth](#generator-flows-in-depth) covers when it runs again, how a `catch` or `finally` in it behaves when a step stops the flow, and the mistakes `effectGen` catches.
|
|
263
221
|
|
|
264
|
-
|
|
222
|
+
## Testing Without Mocks
|
|
265
223
|
|
|
266
|
-
|
|
224
|
+
A flow is plain objects, so a test can check what it will do without running it. Validation runs as soon as the flow is built, so testing it needs nothing else:
|
|
267
225
|
|
|
268
226
|
```js
|
|
269
|
-
const
|
|
270
|
-
|
|
271
|
-
const { result } = await replayEffect(registerUserFlowGen(input), answerByName);
|
|
272
|
-
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
227
|
+
const badInput = { email: 'bad-email', password: '123' };
|
|
228
|
+
assert.deepEqual(registerUserFlow(badInput), Failure('Invalid email.'));
|
|
273
229
|
```
|
|
274
230
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
## Time-Travel Debugging
|
|
278
|
-
|
|
279
|
-
Record what each Command returned, then feed those results back into the flow to retrace the path a request took, with no database or network.
|
|
231
|
+
Next, test the steps. Hand a step an answer and check what it returns:
|
|
280
232
|
|
|
281
233
|
```js
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
// Record a real run. The context's flowName names the flow in the trace.
|
|
285
|
-
const { result, trace } = await recordEffect(checkoutFlow(input), {
|
|
286
|
-
version: process.env.BUILD_ID,
|
|
287
|
-
context: { flowName: 'checkout' }
|
|
288
|
-
});
|
|
234
|
+
const input = { email: 'test@test.com', password: 'password123' };
|
|
289
235
|
|
|
290
|
-
//
|
|
291
|
-
|
|
292
|
-
```
|
|
236
|
+
// The email is free, so the step passes the input on.
|
|
237
|
+
assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
293
238
|
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
Initial input: {
|
|
297
|
-
"cartId": "cart_abc123",
|
|
298
|
-
"promoCode": "FREE_YEAR_VIP"
|
|
299
|
-
}
|
|
300
|
-
Step 1: cmdFetchCart returned in 41.277ms {
|
|
301
|
-
"totalAmount": "120.00"
|
|
302
|
-
}
|
|
303
|
-
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
304
|
-
"isValid": true,
|
|
305
|
-
"discountValue": 100
|
|
306
|
-
}
|
|
307
|
-
Step 3: cmdChargeCard threw in 181.507ms {
|
|
308
|
-
"code": "invalid_amount",
|
|
309
|
-
"name": "Error",
|
|
310
|
-
"message": "Amount must be non-zero."
|
|
311
|
-
}
|
|
312
|
-
Replay finished with state: Failure
|
|
313
|
-
Error: {
|
|
314
|
-
"code": "invalid_amount",
|
|
315
|
-
"name": "Error",
|
|
316
|
-
"message": "Amount must be non-zero."
|
|
317
|
-
}
|
|
239
|
+
// The email is taken, so the flow stops here.
|
|
240
|
+
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
318
241
|
```
|
|
319
242
|
|
|
320
|
-
|
|
243
|
+
Then test the flow: the right calls, in the right order. Handing each Command's `next` the answer you choose, and checking the step it returns, is called walking the flow; nothing runs. `commandName` gives the name a Command is recorded under:
|
|
321
244
|
|
|
322
245
|
```js
|
|
323
|
-
|
|
324
|
-
const { result } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
325
|
-
assert.equal(result.type, 'Failure');
|
|
326
|
-
assert.equal(result.error.code, 'invalid_amount');
|
|
327
|
-
});
|
|
328
|
-
```
|
|
329
|
-
|
|
330
|
-
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
331
|
-
|
|
332
|
-
A step the trace lacks ends the replay in a `Failure` holding a `ReplayError`. A trace lacks a step that comes after the end of the recording, one a hook vetoed in production, and one the recorder dropped under `maxEntries`.
|
|
246
|
+
import { commandName } from 'pure-effect';
|
|
333
247
|
|
|
334
|
-
|
|
248
|
+
const step1 = registerUserFlow(input);
|
|
249
|
+
assert.equal(commandName(step1), 'cmdFindUser');
|
|
335
250
|
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
assert.equal(result.type, 'Success');
|
|
340
|
-
assert.deepEqual(
|
|
341
|
-
unreached.map((e) => e.command),
|
|
342
|
-
['cmdChargeCard']
|
|
343
|
-
);
|
|
344
|
-
});
|
|
251
|
+
const step2 = step1.next(null); // pretend no user was found
|
|
252
|
+
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
253
|
+
// The database was never touched.
|
|
345
254
|
```
|
|
346
255
|
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
### Reading a trace
|
|
350
|
-
|
|
351
|
-
A trace is plain JSON. This one is from a checkout whose card charge timed out once and then went through:
|
|
256
|
+
Write both kinds: only a step test sees the value passed from one step to the next. Suppose the email guard returned `Success(true)` instead of `Success(input)`. The step test would fail. The flow test would still pass, since the calls and their order are unchanged, while the flow saved `true` instead of the user.
|
|
352
257
|
|
|
353
|
-
|
|
354
|
-
{
|
|
355
|
-
"flowName": "checkout",
|
|
356
|
-
"version": "b42",
|
|
357
|
-
"initialInput": { "cartId": "cart_1" },
|
|
358
|
-
"context": { "flowName": "checkout" },
|
|
359
|
-
"dropped": 0,
|
|
360
|
-
"trace": [
|
|
361
|
-
{ "command": "cmdFetchCart", "path": "0", "result": { "total": 120 }, "durationMs": 41.3 },
|
|
362
|
-
{ "command": "cmdChargeCard", "path": "1r0/0", "threw": true,
|
|
363
|
-
"error": { "__error": true, "name": "Error", "message": "gateway timeout", "code": "ETIMEDOUT" }, "durationMs": 5002.1 },
|
|
364
|
-
{ "command": "cmdChargeCard", "path": "1r1/0", "result": { "chargeId": "ch_1" }, "durationMs": 181.5 }
|
|
365
|
-
]
|
|
366
|
-
}
|
|
367
|
-
```
|
|
258
|
+
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
368
259
|
|
|
369
|
-
|
|
260
|
+
A generator flow is tested the same way. Its decisions are in step functions, which get the step tests, and a walk or a whole-flow run works on `registerUserFlowGen` as it does on `registerUserFlow`. A walk straight through runs the body once, and walking a second branch runs it again.
|
|
370
261
|
|
|
371
|
-
|
|
372
|
-
- `r` and a number is one attempt of the `Retry` at that position, `r0` being the first try, and `rf` is its `onExhausted` fallback.
|
|
373
|
-
- `p` and a number is one branch of the `Parallel` at that position, from `p0`. The `Parallel` has an entry of its own, at a path such as `2p`, which records whether a branch cancelled the others.
|
|
374
|
-
- `/` goes one level in. `1r1/0` is the first step of the second try of the `Retry` at position 1. `2p1/0` is the first step of the second branch of the `Parallel` at position 2.
|
|
262
|
+
[Testing a whole flow](#testing-a-whole-flow) shows how to run a whole flow in a test instead, `Retry` and `Parallel` included, with no I/O.
|
|
375
263
|
|
|
376
|
-
|
|
264
|
+
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them, with the services passed in as [Passing Runtime Context](#passing-runtime-context) shows.
|
|
377
265
|
|
|
378
|
-
|
|
266
|
+
In TypeScript, a walk needs a check at each step that the step is a Command; [Testing a flow in TypeScript](#testing-a-flow-in-typescript) has a helper that makes that one line.
|
|
379
267
|
|
|
380
|
-
|
|
381
|
-
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix. Catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
382
|
-
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
268
|
+
### Testing a whole flow
|
|
383
269
|
|
|
384
|
-
|
|
270
|
+
To run a whole flow in a test instead of walking it, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command. Nothing is called and no retry waits. The function gets each step the flow asks for, with its `name`, and returns `{ result }` for the Command to return, `{ error }` for it to throw, or `undefined` for a step it has no answer for, which ends the replay in a `Failure` holding a `ReplayError`:
|
|
385
271
|
|
|
386
|
-
|
|
272
|
+
```js
|
|
273
|
+
import { replayEffect } from 'pure-effect';
|
|
387
274
|
|
|
388
|
-
|
|
275
|
+
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
276
|
+
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
277
|
+
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
278
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
279
|
+
```
|
|
389
280
|
|
|
390
|
-
|
|
281
|
+
Answering `{ error }` is how a test reaches a `Retry`'s next attempt or its `onExhausted` fallback. An answer takes the place of the Command's function, its `catch` included: for a function that catches an error and returns it, answer with what it returns, not `{ error }`. To answer each attempt differently, give a name a list of answers and hand them out in turn. Here the Quick Start's `saveUserWithRetry` fails once, then goes through:
|
|
391
282
|
|
|
392
283
|
```js
|
|
393
|
-
const { result,
|
|
394
|
-
const
|
|
395
|
-
|
|
284
|
+
const answers = { cmdSaveUser: [{ error: new Error('connection reset') }, { result: { id: 1, ...input } }] };
|
|
285
|
+
const inTurn = (step) => answers[step.name]?.shift();
|
|
286
|
+
const { result } = await replayEffect(saveUserWithRetry(input), inTurn);
|
|
287
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
396
288
|
```
|
|
397
289
|
|
|
398
|
-
|
|
290
|
+
The function is also asked about each `Parallel`, under the name `'Parallel'`; answering `undefined` lets its branches run as written. A step also carries its `path`, its position in the flow, which [Reading a trace](#reading-a-trace) explains; answer by path to tell apart two Commands with the same name.
|
|
399
291
|
|
|
400
|
-
|
|
292
|
+
## Rules for Writing a Flow
|
|
401
293
|
|
|
402
|
-
|
|
294
|
+
Twelve rules cover most of what goes wrong. The same rules ship in the package as `RULES.md`, with an index of where each task is covered.
|
|
403
295
|
|
|
404
|
-
|
|
405
|
-
// A resolver answers one question: what did production get back for this step? Look the step up by its path, its
|
|
406
|
-
// position in the flow. `step.index` counts steps in the order they finished, which shifts around a Parallel.
|
|
407
|
-
const outputs = new Map(mySpans.map((span) => [span.attributes.path, span.attributes.output]));
|
|
408
|
-
const resolve = (step) => (outputs.has(step.path) ? { result: outputs.get(step.path) } : undefined);
|
|
409
|
-
await replayEffect(checkoutFlow(input), resolve);
|
|
410
|
-
```
|
|
296
|
+
<!-- rules:start -->
|
|
411
297
|
|
|
412
|
-
|
|
298
|
+
1. **Put every call that does I/O inside a Command, and keep its function to that one call:** `Command(() => db.findUser(email))`, never `step(db.findUser(email))`. Tests never run the function, so anything else in it goes untested. See [Testing Without Mocks](#testing-without-mocks).
|
|
299
|
+
2. **Name every Command,** with a named function such as `const cmdFindUser = () => ...`, or with `meta.name`. Traces, replay and telemetry use the name, and a minifier can strip a function's name. See [`Command`](#commandcmdfn-nextfn-meta).
|
|
300
|
+
3. **Put each decision in a step function** that receives one value and returns `Success`, `Failure` or the next Command, so a test can call it directly. See [Coming from async/await](#coming-from-asyncawait).
|
|
301
|
+
4. **Return an outcome the caller should see as `Failure(error)`; never throw it.** A `Failure` stops the flow, is never retried, and only the code that calls `runEffect` acts on it. See [Which Errors Are Data](#which-errors-are-data).
|
|
302
|
+
5. **A `catch` in a generator flow cannot recover.** It can log and rethrow, and it sees a `FlowStopped` error saying where the flow stopped, not the original. To handle an error, catch it inside the Command's function and return it as data. See [How a generator flow stops](#how-a-generator-flow-stops).
|
|
303
|
+
6. **Let a Command's function throw only when `Retry` should try again.** An error it catches and returns is data, and `Retry` never sees it. To retry some errors and not others, rethrow the first kind and return the second. A throw also skips any undo the Command's `next` would have run. See [Which Errors Are Data](#which-errors-are-data).
|
|
304
|
+
7. **Wrap only the Command that fails in `Retry`, and continue in a later step.** `Retry` runs everything it wraps again, the Command's `next` included. `retries: 2` means up to three calls, and `onExhausted` is where you give up. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
305
|
+
8. **Write a loop as a step that gets the list, then `Parallel(list.map(f), { limit: 1 })`.** `limit` caps how many run at once, and `settled: true` keeps going past a failure. A generator flow can use a `for` loop with `yield*`. See [Writing Loops](#writing-loops).
|
|
306
|
+
9. **Get the time, random IDs and anything else that changes between runs from a Command.** A replay then gets the recorded value, and a generator body, which can run more than once, sees the same value each time. See [What runs once, and what runs again](#what-runs-once-and-what-runs-again).
|
|
307
|
+
10. **Return plain data from Commands, and keep the context to plain values.** A trace stores JSON, so a `Date` comes back as a string. Reach a service from the Command's function, and hand it to the function that builds the flow. See [Passing Runtime Context](#passing-runtime-context).
|
|
308
|
+
11. **Clean up in a `try`/`finally` around `runEffect`,** not inside the flow. See [How a generator flow stops](#how-a-generator-flow-stops).
|
|
309
|
+
12. **Check a replay's error, not only its type.** A replay of a flow that changed shape ends in a `Failure` holding a `TimeParadox`, the error naming where the flow and the trace part ways, so a test that only expects a `Failure` passes by accident. See [Time-Travel Debugging](#time-travel-debugging).
|
|
413
310
|
|
|
414
|
-
|
|
311
|
+
<!-- rules:end -->
|
|
312
|
+
|
|
313
|
+
## Passing Runtime Context
|
|
314
|
+
|
|
315
|
+
Some values come from the framework (an authenticated tenant, a request ID, environment config) rather than from the data being processed. `Ask` lets a step read the `context` object passed to `runEffect` without passing it through every function:
|
|
415
316
|
|
|
416
317
|
```js
|
|
417
|
-
import {
|
|
418
|
-
import { configureEffect } from 'pure-effect';
|
|
419
|
-
import { recordingHooks } from './recording-example.js';
|
|
420
|
-
import { telemetryHooks } from './opentelemetry-example.js';
|
|
421
|
-
|
|
422
|
-
configureEffect(
|
|
423
|
-
telemetryHooks(),
|
|
424
|
-
recordingHooks({
|
|
425
|
-
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
426
|
-
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
427
|
-
redact: (value, name, kind) =>
|
|
428
|
-
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
429
|
-
maxEntries: 500,
|
|
430
|
-
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
431
|
-
})
|
|
432
|
-
);
|
|
433
|
-
```
|
|
434
|
-
|
|
435
|
-
What the example does with each run:
|
|
436
|
-
|
|
437
|
-
- **Which runs it keeps.** `keep` decides. By default it keeps a run that ended in a `Failure`, and discards a successful one after holding it in memory.
|
|
438
|
-
- **When your own code throws.** A `TypeError` after an API changed the shape of its response, say, makes the run reject. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept, and replaying it reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded. That run did no I/O, so its input reproduces it.
|
|
439
|
-
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out and marked `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
440
|
-
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
441
|
-
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
442
|
-
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
443
|
-
|
|
444
|
-
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` or `effectGen` built, so build the outermost flow with one of them. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
445
|
-
|
|
446
|
-
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
447
|
-
|
|
448
|
-
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow. So neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes:
|
|
449
|
-
|
|
450
|
-
| Recorded | Replayed as |
|
|
451
|
-
| ----------------------------------- | --------------------------------------------------- |
|
|
452
|
-
| a money object or a database entity | a plain object, without its methods |
|
|
453
|
-
| a `Date` | a string |
|
|
454
|
-
| a `Map` or a `Set` | `{}` |
|
|
455
|
-
| a `Buffer` | `{ type: 'Buffer', data }` |
|
|
456
|
-
| `undefined` in an array | `null` |
|
|
457
|
-
| an error returned inside a result | its own fields, such as `code`, without its message |
|
|
458
|
-
|
|
459
|
-
A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it. So a context holding a lazy client is left out whole.
|
|
460
|
-
|
|
461
|
-
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`. Rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
462
|
-
|
|
463
|
-
### Keeping secrets out of a trace
|
|
464
|
-
|
|
465
|
-
`redact` sees everything a trace stores: each result, each thrown error, the flow's input and the context, told apart by `kind`. It gets a copy in the form the trace stores, so changing it in place is safe, and what it returns is stored in that form too. A thrown `Error` arrives as a plain object holding its `name`, `message` and own fields, and any object returned for it replays as an `Error`.
|
|
466
|
-
|
|
467
|
-
Check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
468
|
-
|
|
469
|
-
```js
|
|
470
|
-
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
471
|
-
const mask = (value, field) =>
|
|
472
|
-
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
473
|
-
|
|
474
|
-
const redact = (value, name, kind) => {
|
|
475
|
-
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
476
|
-
if (kind === 'context') return mask(value, 'authToken');
|
|
477
|
-
if (kind === 'error') return mask(value, 'attempted');
|
|
478
|
-
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
479
|
-
};
|
|
480
|
-
recorder({ redact });
|
|
481
|
-
|
|
482
|
-
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
483
|
-
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
484
|
-
```
|
|
485
|
-
|
|
486
|
-
A replay rebuilds the flow from the redacted input and context. So a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does. A context field an `Ask` reads should stay.
|
|
487
|
-
|
|
488
|
-
## Hooks
|
|
489
|
-
|
|
490
|
-
Hooks see every run, and recording, replay and the telemetry example are built on them. There are three: `onRun` wraps a whole run, `onStep` wraps each Command and each `Parallel`, and `onBeforeCommand` runs before each Command and can veto it by throwing. A hook that only watches must `await` what it wraps and return its result; [`configureEffect`](#configureeffectconfigs) lists each hook's arguments.
|
|
491
|
-
|
|
492
|
-
`configureEffect` adds a layer of hooks for the whole process and returns a function that removes it. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's:
|
|
493
|
-
|
|
494
|
-
```js
|
|
495
|
-
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
496
|
-
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
497
|
-
remove();
|
|
498
|
-
```
|
|
499
|
-
|
|
500
|
-
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
501
|
-
|
|
502
|
-
```
|
|
503
|
-
runEffect(flow(input))
|
|
504
|
-
│
|
|
505
|
-
├─ telemetry.onRun first configuration: outermost
|
|
506
|
-
│ └─ recording.onRun last configuration: innermost
|
|
507
|
-
│ │
|
|
508
|
-
│ │ for each Command:
|
|
509
|
-
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
510
|
-
│ ├─ recording.onBeforeCommand
|
|
511
|
-
│ ├─ telemetry.onStep
|
|
512
|
-
│ │ └─ recording.onStep closest to the Command
|
|
513
|
-
│ │ └─ cmd()
|
|
514
|
-
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
515
|
-
│ ├─ telemetry.onStep returns from the inside out
|
|
516
|
-
│ │
|
|
517
|
-
│ ┌─ recording.onRun returns
|
|
518
|
-
├─ telemetry.onRun returns
|
|
519
|
-
```
|
|
520
|
-
|
|
521
|
-
**Knowing which request a step belongs to.** `onStep` and `onRun` get no context, so a hook that logs per request finds the request the way the rest of your logging does. An `AsyncLocalStorage` scope entered around `runEffect` reaches every hook the run calls, and the recording example uses one to give each run its own recorder:
|
|
522
|
-
|
|
523
|
-
```js
|
|
524
|
-
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
525
|
-
|
|
526
|
-
const requestScope = new AsyncLocalStorage();
|
|
527
|
-
const stepLog = [];
|
|
528
|
-
|
|
529
|
-
// Logs each step with its request and duration, and returns what the step returned.
|
|
530
|
-
const removeStepLog = configureEffect({
|
|
531
|
-
onStep: async (name, type, op, path) => {
|
|
532
|
-
const started = performance.now();
|
|
533
|
-
try {
|
|
534
|
-
return await op();
|
|
535
|
-
} finally {
|
|
536
|
-
const { requestId } = requestScope.getStore() ?? {};
|
|
537
|
-
stepLog.push({ requestId, name, path, ms: performance.now() - started });
|
|
538
|
-
}
|
|
539
|
-
}
|
|
540
|
-
});
|
|
541
|
-
|
|
542
|
-
await requestScope.run({ requestId: 'req_1' }, () => runEffect(registerUserFlow(input)));
|
|
543
|
-
assert.deepEqual(
|
|
544
|
-
stepLog.map((line) => [line.requestId, line.name]),
|
|
545
|
-
[
|
|
546
|
-
['req_1', 'cmdFindUser'],
|
|
547
|
-
['req_1', 'cmdSaveUser']
|
|
548
|
-
]
|
|
549
|
-
);
|
|
550
|
-
removeStepLog();
|
|
551
|
-
```
|
|
552
|
-
|
|
553
|
-
A Command that `onBeforeCommand` vetoes never reaches `onStep`, so log a veto where you make it. In tests, call `configureEffect()` with no arguments in a `beforeEach`, so a test that fails before removing its hooks does not leave them installed for the next.
|
|
554
|
-
|
|
555
|
-
**Per-call hooks.** `runEffect`'s third argument takes hooks for that call only. They are merged with the configured ones by the same rules: the configured wrappers run outside them, and the configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
556
|
-
|
|
557
|
-
```
|
|
558
|
-
inherit: true (default) inherit: false
|
|
559
|
-
|
|
560
|
-
onRun C.onRun (none)
|
|
561
|
-
onBeforeCommand C.onBeforeCommand (none)
|
|
562
|
-
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
563
|
-
```
|
|
564
|
-
|
|
565
|
-
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
566
|
-
|
|
567
|
-
## Passing Runtime Context
|
|
568
|
-
|
|
569
|
-
Some values come from the framework (an authenticated tenant, a request ID, environment config) rather than from the data being processed. `Ask` lets a step read the `context` object passed to `runEffect` without passing it through every function:
|
|
570
|
-
|
|
571
|
-
```js
|
|
572
|
-
import { Success, Failure, Command, Ask, effectPipe, runEffect } from 'pure-effect';
|
|
318
|
+
import { Success, Failure, Command, Ask, effectPipe, runEffect } from 'pure-effect';
|
|
573
319
|
|
|
574
320
|
const findProduct = (productId) =>
|
|
575
321
|
Ask((ctx) => {
|
|
@@ -615,20 +361,21 @@ const flakyNetwork = { retries: 3, delay: 200, backoff: 2 };
|
|
|
615
361
|
Retry(fetchPrice(sku), flakyNetwork);
|
|
616
362
|
```
|
|
617
363
|
|
|
618
|
-
The settings are a plain object, so a test can check them without running anything
|
|
364
|
+
The settings are a plain object, so a test can check them without running anything. The `Retry` wraps only the Command, and the decision about its answer comes in the next step, outside the `Retry`:
|
|
619
365
|
|
|
620
366
|
```js
|
|
621
|
-
import { Success, Failure, Command, Retry,
|
|
367
|
+
import { Success, Failure, Command, Retry, effectPipe } from 'pure-effect';
|
|
622
368
|
|
|
623
369
|
const fetchWeather = (city) => {
|
|
624
370
|
const cmdFetchWeather = () =>
|
|
625
371
|
fetch(`https://example-weather-api.com/v1/current?city=${city}`).then((r) => r.json());
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
{ retries: 3, delay: 200, backoff: 2 } // 200ms, 400ms, 800ms
|
|
629
|
-
);
|
|
372
|
+
// Four calls in all, waiting 200ms, 400ms, then 800ms.
|
|
373
|
+
return Retry(Command(cmdFetchWeather), { retries: 3, delay: 200, backoff: 2 });
|
|
630
374
|
};
|
|
631
375
|
|
|
376
|
+
const requireWeather = (data) => (data.error ? Failure(data.error) : Success(data));
|
|
377
|
+
const weatherFlow = (city) => effectPipe(fetchWeather, requireWeather)(city);
|
|
378
|
+
|
|
632
379
|
const weatherFn = fetchWeather('Tokyo');
|
|
633
380
|
assert.equal(weatherFn.type, 'Retry');
|
|
634
381
|
assert.equal(weatherFn.options.retries, 3);
|
|
@@ -686,6 +433,10 @@ const fetchPrice = (sku) =>
|
|
|
686
433
|
|
|
687
434
|
Every attempt and the fallback are recorded, so a replay repeats the same failures and the same fallback, without the delays. A fallback never starts in a `Parallel` branch that has already been cancelled. With `onExhausted` set, the fallback's error type takes the place of `RetryExhaustedError` in the TypeScript error union.
|
|
688
435
|
|
|
436
|
+
### Waiting as long as the server asks
|
|
437
|
+
|
|
438
|
+
`Retry` waits a fixed `delay`, so it cannot wait as long as a server names, as a rate limit's `Retry-After` does. Catch that answer inside the Command's function and return it as data, as [Which Errors Are Data](#which-errors-are-data) shows, then have `next` return a Command that waits that long, followed by the same request again. A replay answers the wait from the trace like any other Command, so it does not wait.
|
|
439
|
+
|
|
689
440
|
## Running Effects in Parallel
|
|
690
441
|
|
|
691
442
|
`Parallel` runs several flows at the same time and passes their results to `next` as an array, in order. If a branch fails, the other branches are cancelled, `next` is not called, and that branch's `Failure` is returned.
|
|
@@ -829,35 +580,7 @@ assert.deepEqual(
|
|
|
829
580
|
|
|
830
581
|
A settled `Parallel` cancels nothing, so both steps always run to the end, and the undoing starts once both have finished. The undo steps are part of the flow, so a trace records them and a replay repeats them. An undo step that fails ends the flow with its own error rather than the original one; wrap it in `Retry` if it can fail for a moment.
|
|
831
582
|
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type. So give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)). Otherwise it falls into the last error it expects, and reports a vetoed charge as a declined card.
|
|
835
|
-
|
|
836
|
-
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)). To stop a batch, read the operator's switch in a Command whose `next` returns a `Failure`, rather than vetoing in `onBeforeCommand`:
|
|
837
|
-
|
|
838
|
-
```js
|
|
839
|
-
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
840
|
-
const cmdBillingHalted = () => flags.get('billing.halted');
|
|
841
|
-
const billUnlessHalted = (subscription) =>
|
|
842
|
-
Command(cmdBillingHalted, (halted) => (halted ? Failure('halted') : billOne(subscription)));
|
|
843
|
-
|
|
844
|
-
Parallel(subscriptions.map(billUnlessHalted), (outcomes) => Success(outcomes.map(summarize)), {
|
|
845
|
-
limit: 5,
|
|
846
|
-
settled: true
|
|
847
|
-
});
|
|
848
|
-
```
|
|
849
|
-
|
|
850
|
-
The halt is then one of the branch's declared errors, typed like a declined card, and the trace records each read of the switch, so the batch replays. A vetoed Command leaves no entry in the trace, so a replay of a vetoed batch fails at the first veto.
|
|
851
|
-
|
|
852
|
-
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
853
|
-
|
|
854
|
-
### Replaying a cancelled `Parallel`
|
|
855
|
-
|
|
856
|
-
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
857
|
-
|
|
858
|
-
If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
859
|
-
|
|
860
|
-
A generator flow in a cancelled branch can run its `catch` and `finally` one step later in a replay than it did in production. A trace marks where a branch stopped only by its Commands. So a branch cancelled during a `Retry`'s wait, just before an `Ask`, or at a `Retry` or `Parallel` with no Command in it replays as though it stopped at its next Command.
|
|
583
|
+
Under `settled`, a branch whose Command throws reaches `next` as a `Failure`, so the undo still runs. When one call follows another instead, as a shipment follows a charge, the later Command's function has to catch its own error and return it as data: a throw stops the flow before its `next` runs, and the charge stays. [Translating `try`/`catch`](#translating-trycatch) shows that shape.
|
|
861
584
|
|
|
862
585
|
## Composing Larger Flows
|
|
863
586
|
|
|
@@ -896,6 +619,40 @@ When the lookup is written in place rather than reused, the Command's own `next`
|
|
|
896
619
|
|
|
897
620
|
In a generator flow, most of this comes for free. A step that branches into another flow is yielded like any other, as in `yield* step(processOrder(order))`. Values that do not depend on each other are joined with `const [cart, user] = yield* step(Parallel([fetchCart(input.cartId), fetchUser(input.userId)]))`. Values that do are variables, with no step to pass them on together; see [Where a generator helps](#where-a-generator-helps). Keep each decision in a step function that takes only what it needs, as the advice above says for a pipeline's steps.
|
|
898
621
|
|
|
622
|
+
### Ending a flow early without an error
|
|
623
|
+
|
|
624
|
+
Not every early end is an error. A webhook that has already handled an event should answer `duplicate` and do nothing else. A pipeline stops only at a `Failure`, so the step that decides ends the flow by what it returns: a `Success` holding the result, or the rest of the flow. Make it the last step of its pipeline, since a `Success` anywhere else passes on to the next step:
|
|
625
|
+
|
|
626
|
+
```js
|
|
627
|
+
const requireOrderId = (event) => (event.orderId ? Success(event) : Failure('Missing order ID.'));
|
|
628
|
+
|
|
629
|
+
// The rest of the flow, for an event seen for the first time.
|
|
630
|
+
const markOrderPaid = (event) => {
|
|
631
|
+
const cmdMarkOrderPaid = () => orderStore.markPaid(event.orderId);
|
|
632
|
+
return Command(cmdMarkOrderPaid, () => Success({ status: 'processed' }));
|
|
633
|
+
};
|
|
634
|
+
|
|
635
|
+
// Claiming the event decides: a duplicate ends the flow here, with a result rather than an error.
|
|
636
|
+
const claimEvent = (event) => {
|
|
637
|
+
const cmdClaimEvent = () => eventLog.claim(event.eventId);
|
|
638
|
+
return Command(cmdClaimEvent, (firstTime) => (firstTime ? markOrderPaid(event) : Success({ status: 'duplicate' })));
|
|
639
|
+
};
|
|
640
|
+
|
|
641
|
+
const handleEvent = (event) => effectPipe(requireOrderId, claimEvent)(event);
|
|
642
|
+
|
|
643
|
+
assert.deepEqual(
|
|
644
|
+
await runEffect(handleEvent({ eventId: 'evt_1', orderId: 'order_1' })),
|
|
645
|
+
Success({ status: 'processed' })
|
|
646
|
+
);
|
|
647
|
+
assert.deepEqual(
|
|
648
|
+
await runEffect(handleEvent({ eventId: 'evt_seen', orderId: 'order_1' })),
|
|
649
|
+
Success({ status: 'duplicate' })
|
|
650
|
+
);
|
|
651
|
+
assert.deepEqual(await runEffect(handleEvent({ eventId: 'evt_2' })), Failure('Missing order ID.'));
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
When the rest of the flow is longer, the step returns it as a pipeline of its own, as `processOrder` does above. In a generator flow, `return` ends the flow anywhere, but keep the decision in a step all the same: `return yield* step(claimEvent(event))`.
|
|
655
|
+
|
|
899
656
|
## Writing Loops
|
|
900
657
|
|
|
901
658
|
A loop is written in two parts: a step that gets the list, and a `Parallel` over it, with `limit: 1` so the items run one after another, as in a `for` loop. The `async` version is usually split the same way. Paying a customer's unpaid invoices in order, and stopping at the first that fails:
|
|
@@ -995,7 +752,7 @@ const listAllOrders = () => {
|
|
|
995
752
|
};
|
|
996
753
|
```
|
|
997
754
|
|
|
998
|
-
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test.
|
|
755
|
+
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test. This is rule 1 applied to a loop: the Command's function is the one call, `fetchAllOrders()`, and the loop lives in a function of its own that an integration test covers.
|
|
999
756
|
|
|
1000
757
|
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
1001
758
|
|
|
@@ -1023,7 +780,7 @@ const fetchPrice = (sku) => {
|
|
|
1023
780
|
|
|
1024
781
|
The failed attempt is recorded as that Command's result, with the error's `code` and `message`, so a replay takes the same fallback branch. Copy the fields you need rather than returning the error itself: a trace keeps data, not objects, so an error stored inside a result loses its message (see [Recording in Production](#recording-in-production)).
|
|
1025
782
|
|
|
1026
|
-
**A `Failure`
|
|
783
|
+
**A `Failure` stops the flow.** It goes back to the code that called `runEffect`, which decides what it means: an HTTP status, a queue retry, an alert. Nothing inside the flow can catch a `Failure`, so a reader never has to look elsewhere for a handler.
|
|
1027
784
|
|
|
1028
785
|
Rule of thumb: if you would handle it, return it as a value; if you would only report it, return a `Failure`. The one exception is retry exhaustion, which happens inside `Retry`; handle it with the `onExhausted` option in [Retrying Transient Failures](#retrying-transient-failures).
|
|
1029
786
|
|
|
@@ -1050,16 +807,90 @@ Retry(
|
|
|
1050
807
|
);
|
|
1051
808
|
```
|
|
1052
809
|
|
|
810
|
+
To retry some errors and not others, do both in one function: return the errors the flow handles, and rethrow the rest so `Retry` tries again. Here a charge the gateway refuses is handled, and anything else, such as a timeout, is retried:
|
|
811
|
+
|
|
812
|
+
```js
|
|
813
|
+
const chargeAmount = (amount) => {
|
|
814
|
+
const cmdChargeAmount = () =>
|
|
815
|
+
gateway.charge(amount).catch((error) => {
|
|
816
|
+
if (error.code === 'invalid_amount') return { refused: true, code: error.code };
|
|
817
|
+
throw error; // may pass on a later try
|
|
818
|
+
});
|
|
819
|
+
return Retry(Command(cmdChargeAmount), { retries: 2 });
|
|
820
|
+
};
|
|
821
|
+
|
|
822
|
+
// A later step turns the handled error into the flow's outcome.
|
|
823
|
+
const requireCharge = (charge) => (charge.refused ? Failure(`Charge refused: ${charge.code}`) : Success(charge));
|
|
824
|
+
const pay = (amount) => effectPipe(chargeAmount, requireCharge)(amount);
|
|
825
|
+
|
|
826
|
+
assert.deepEqual(await runEffect(pay(0)), Failure('Charge refused: invalid_amount'));
|
|
827
|
+
assert.deepEqual(await runEffect(pay(25)), Success({ chargeId: 'ch_1', amount: 25 }));
|
|
828
|
+
```
|
|
829
|
+
|
|
1053
830
|
`runEffect` tells these cases apart:
|
|
1054
831
|
|
|
1055
|
-
| what happened | what it is | what `Retry` does | what `onExhausted` sees |
|
|
1056
|
-
| --- | --- | --- | --- |
|
|
1057
|
-
| a step returned `Failure(...)`, or an `onBeforeCommand` hook threw |
|
|
1058
|
-
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
1059
|
-
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
832
|
+
| what happened | what it is | what `Retry` does | what `onExhausted` sees | what the caller's `Failure` holds |
|
|
833
|
+
| --- | --- | --- | --- | --- |
|
|
834
|
+
| a step returned `Failure(...)`, or an `onBeforeCommand` hook threw | a stop | nothing; it propagates at once, unwrapped | nothing | the error returned or thrown |
|
|
835
|
+
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone | the error thrown, or, once a `Retry` gives up, `{ retryExhausted, lastError, retries }` |
|
|
836
|
+
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing | nothing; `runEffect` rejects |
|
|
1060
837
|
|
|
1061
838
|
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is a bug, and is thrown as it is rather than returned as a `Failure`. `runEffect` rejects with it, or, from a step before the first Command, the call that builds the flow throws it.
|
|
1062
839
|
|
|
840
|
+
### Translating `try`/`catch`
|
|
841
|
+
|
|
842
|
+
A `catch` around `yield*` can log and rethrow, but it cannot recover, and it sees a `FlowStopped` error rather than the original. So each `try`/`catch` in an `async` function moves to where its error happens. The four common shapes:
|
|
843
|
+
|
|
844
|
+
- **Carry on past a failed call.** Catch inside the Command's function and return the error as data, as below.
|
|
845
|
+
- **Undo the calls that succeeded when one running beside them fails.** Run the calls in a `Parallel` with `settled: true`, and have its `next` undo the ones that succeeded; see [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails).
|
|
846
|
+
- **Undo an earlier call when a later one fails.** Catch inside the later Command's function and return the error as data, and have its `next` return the Command that undoes the earlier one, as in the second example below. A later Command that throws instead stops the flow before its `next` runs, so nothing is undone.
|
|
847
|
+
- **Retry, then give up.** Wrap the Command in `Retry`, and put what the `catch` did after the last try in `onExhausted`; see [Retrying Transient Failures](#retrying-transient-failures).
|
|
848
|
+
|
|
849
|
+
A welcome email whose failure the flow reports but does not stop for:
|
|
850
|
+
|
|
851
|
+
```js
|
|
852
|
+
// Takes the mailer, so a test can hand it one that fails.
|
|
853
|
+
const makeSendWelcome = (mailer) => (user) => {
|
|
854
|
+
const cmdSendWelcome = () =>
|
|
855
|
+
mailer
|
|
856
|
+
.sendWelcome(user.email)
|
|
857
|
+
.then(() => ({ sent: true }))
|
|
858
|
+
.catch((error) => ({ sent: false, reason: error.message }));
|
|
859
|
+
return Command(cmdSendWelcome);
|
|
860
|
+
};
|
|
861
|
+
|
|
862
|
+
const failingMailer = {
|
|
863
|
+
sendWelcome: async () => {
|
|
864
|
+
throw new Error('smtp down');
|
|
865
|
+
}
|
|
866
|
+
};
|
|
867
|
+
const welcome = await runEffect(makeSendWelcome(failingMailer)({ email: 'test@test.com' }));
|
|
868
|
+
assert.deepEqual(welcome, Success({ sent: false, reason: 'smtp down' }));
|
|
869
|
+
```
|
|
870
|
+
|
|
871
|
+
A reservation made after a charge, which refunds the charge when the reservation fails:
|
|
872
|
+
|
|
873
|
+
```js
|
|
874
|
+
const refundCharge = (charge, reason) => {
|
|
875
|
+
const cmdRefundCharge = () => payments.refund(charge.chargeId);
|
|
876
|
+
return Command(cmdRefundCharge, () => Failure(`Reservation failed: ${reason}`));
|
|
877
|
+
};
|
|
878
|
+
|
|
879
|
+
const reserveStock = (charge) => {
|
|
880
|
+
const cmdReserveStock = () =>
|
|
881
|
+
warehouse
|
|
882
|
+
.reserve(charge.orderId)
|
|
883
|
+
.then(() => ({ reserved: true }))
|
|
884
|
+
.catch((error) => ({ reserved: false, reason: error.message }));
|
|
885
|
+
return Command(cmdReserveStock, (reservation) =>
|
|
886
|
+
reservation.reserved ? Success(charge) : refundCharge(charge, reservation.reason)
|
|
887
|
+
);
|
|
888
|
+
};
|
|
889
|
+
|
|
890
|
+
const reservation = await runEffect(reserveStock({ orderId: 'order_1', chargeId: 'ch_1' }));
|
|
891
|
+
assert.deepEqual(reservation, Failure('Reservation failed: out of stock'));
|
|
892
|
+
```
|
|
893
|
+
|
|
1063
894
|
### A malformed flow
|
|
1064
895
|
|
|
1065
896
|
A flow that is not made of Effects is a bug, so it throws an `EffectTypeError` that says where, rather than returning a `Failure`. The steps before a flow's first Command run as it is built. So a mistake in one of them is thrown by the call that builds the flow, as `registerUserFlow(input)` would throw if its first step returned a plain object:
|
|
@@ -1160,6 +991,40 @@ A step in a body's `finally` is not supported. It runs when the body ends normal
|
|
|
1160
991
|
|
|
1161
992
|
In TypeScript, most of these do not compile; see [TypeScript](#typescript-typed-errors-and-context).
|
|
1162
993
|
|
|
994
|
+
## Parallel in Depth
|
|
995
|
+
|
|
996
|
+
What [Running Effects in Parallel](#running-effects-in-parallel) leaves out: errors no branch returned, and how a cancelled `Parallel` replays.
|
|
997
|
+
|
|
998
|
+
### Errors no branch returned
|
|
999
|
+
|
|
1000
|
+
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type. So give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)). Otherwise it falls into the last error it expects, and reports a vetoed charge as a declined card.
|
|
1001
|
+
|
|
1002
|
+
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)). To stop a batch, read the operator's switch in a Command whose `next` returns a `Failure`, rather than vetoing in `onBeforeCommand`:
|
|
1003
|
+
|
|
1004
|
+
```js
|
|
1005
|
+
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
1006
|
+
const cmdBillingHalted = () => flags.get('billing.halted');
|
|
1007
|
+
const billUnlessHalted = (subscription) =>
|
|
1008
|
+
Command(cmdBillingHalted, (halted) => (halted ? Failure('halted') : billOne(subscription)));
|
|
1009
|
+
|
|
1010
|
+
Parallel(subscriptions.map(billUnlessHalted), (outcomes) => Success(outcomes.map(summarize)), {
|
|
1011
|
+
limit: 5,
|
|
1012
|
+
settled: true
|
|
1013
|
+
});
|
|
1014
|
+
```
|
|
1015
|
+
|
|
1016
|
+
The halt is then one of the branch's declared errors, typed like a declined card, and the trace records each read of the switch, so the batch replays. A vetoed Command leaves no entry in the trace, so a replay of a vetoed batch fails at the first veto.
|
|
1017
|
+
|
|
1018
|
+
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
1019
|
+
|
|
1020
|
+
### Replaying a cancelled `Parallel`
|
|
1021
|
+
|
|
1022
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
1023
|
+
|
|
1024
|
+
If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
1025
|
+
|
|
1026
|
+
A generator flow in a cancelled branch can run its `catch` and `finally` one step later in a replay than it did in production. A trace marks where a branch stopped only by its Commands. So a branch cancelled during a `Retry`'s wait, just before an `Ask`, or at a `Retry` or `Parallel` with no Command in it replays as though it stopped at its next Command.
|
|
1027
|
+
|
|
1163
1028
|
## TypeScript: Typed Errors and Context
|
|
1164
1029
|
|
|
1165
1030
|
The bundled declarations need TypeScript 5.1 or later.
|
|
@@ -1195,6 +1060,30 @@ Value types are checked through the pipeline too, so a step that reads a field t
|
|
|
1195
1060
|
|
|
1196
1061
|
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
1197
1062
|
|
|
1063
|
+
### Handling every error
|
|
1064
|
+
|
|
1065
|
+
A `switch` over a flow's error union can make TypeScript refuse a case you forgot. Check first for a thrown error, which reaches `result.error` without being in the union, and end the `switch` with a `never`, so a case added to the union later is a compile error until it is handled. TypeScript narrows a `switch` only over a union, so with a single error type, leave out the `default`:
|
|
1066
|
+
|
|
1067
|
+
```ts
|
|
1068
|
+
type BillingError = { type: 'NoSubscription' } | { type: 'CardDeclined'; code: string };
|
|
1069
|
+
|
|
1070
|
+
const result = await runEffect(billTenant(tenantId), context);
|
|
1071
|
+
if (result.type === 'Failure') {
|
|
1072
|
+
const thrown: unknown = result.error;
|
|
1073
|
+
if (thrown instanceof Error) return serverError(thrown); // a Command's function threw
|
|
1074
|
+
switch (result.error.type) {
|
|
1075
|
+
case 'NoSubscription':
|
|
1076
|
+
return skipTenant();
|
|
1077
|
+
case 'CardDeclined':
|
|
1078
|
+
return notifyDecline(result.error.code);
|
|
1079
|
+
default: {
|
|
1080
|
+
const unhandled: never = result.error;
|
|
1081
|
+
return serverError(unhandled);
|
|
1082
|
+
}
|
|
1083
|
+
}
|
|
1084
|
+
}
|
|
1085
|
+
```
|
|
1086
|
+
|
|
1198
1087
|
### Typed context with `Ask`
|
|
1199
1088
|
|
|
1200
1089
|
`Effect<T, E, Ctx>` carries a third type parameter for the context object:
|
|
@@ -1223,7 +1112,28 @@ const registerUser = effectGen(function* (input: { email: string }) {
|
|
|
1223
1112
|
// (input: { email: string }) => Effect<User, 'EMAIL_TAKEN' | the errors findUser and saveUser declare>
|
|
1224
1113
|
```
|
|
1225
1114
|
|
|
1226
|
-
`yield* x` without `step`, reading a property of a step, and an `async function*` do not compile, and a body that returns an untyped value keeps its steps' errors. Returning a step or a Promise compiles, so that a generic helper returning a value of its type parameter does too, and fails when it runs. `yield step(x)` without the star does, and fails when it runs. Name the context `Ask()` reads as `Ask<AppContext>()`.
|
|
1115
|
+
`yield* x` without `step`, reading a property of a step, and an `async function*` do not compile, and a body that returns an untyped value keeps its steps' errors. Returning a step or a Promise compiles, so that a generic helper returning a value of its type parameter does too, and fails when it runs. `yield step(x)` without the star does, and fails when it runs. Name the context `Ask()` reads as `Ask<AppContext>()`. Given a function, `Ask`'s first type argument is the value instead, so annotate the parameter: `Ask((ctx: AppContext) => ...)`.
|
|
1116
|
+
|
|
1117
|
+
### A flow with no input
|
|
1118
|
+
|
|
1119
|
+
A flow takes its input as one parameter. For one that needs none, such as a nightly job that gets everything from its Commands, type the parameter `void`, so the flow is called with no argument. In a pipeline, that is the first step's parameter. Without it, `nightlyReport()` fails to compile with "Expected 1 arguments, but got 0":
|
|
1120
|
+
|
|
1121
|
+
```ts
|
|
1122
|
+
const nightlyReport = effectGen(function* (_input: void) {
|
|
1123
|
+
const users = yield* step(listUsers());
|
|
1124
|
+
return users.length;
|
|
1125
|
+
});
|
|
1126
|
+
|
|
1127
|
+
const result = await runEffect(nightlyReport());
|
|
1128
|
+
|
|
1129
|
+
// The same as a pipeline.
|
|
1130
|
+
const nightlyCount = effectPipe(
|
|
1131
|
+
(_input: void) => listUsers(),
|
|
1132
|
+
(users) => Success(users.length)
|
|
1133
|
+
);
|
|
1134
|
+
|
|
1135
|
+
const counted = await runEffect(nightlyCount());
|
|
1136
|
+
```
|
|
1227
1137
|
|
|
1228
1138
|
### Testing a flow in TypeScript
|
|
1229
1139
|
|
|
@@ -1247,6 +1157,369 @@ assertCommand(step1.next(null), 'cmdSaveUser');
|
|
|
1247
1157
|
|
|
1248
1158
|
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns. So where [Testing Without Mocks](#testing-without-mocks) passes `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
1249
1159
|
|
|
1160
|
+
## Time-Travel Debugging
|
|
1161
|
+
|
|
1162
|
+
Record what each Command returned, then feed those results back into the flow to retrace the path a request took, with no database or network.
|
|
1163
|
+
|
|
1164
|
+
Record a run with `recordEffect`, and `replayEffect` hands each Command what it returned the first time. That is how a run recorded in production replays on your machine, with nothing connected:
|
|
1165
|
+
|
|
1166
|
+
```js
|
|
1167
|
+
import { recordEffect, replayEffect } from 'pure-effect';
|
|
1168
|
+
|
|
1169
|
+
// A real run returns its result, and a trace of what each Command returned.
|
|
1170
|
+
const { result: live, trace } = await recordEffect(registerUserFlow(input));
|
|
1171
|
+
|
|
1172
|
+
// Anywhere else, with nothing connected, the same flow gets the same answers.
|
|
1173
|
+
const { result: replayed } = await replayEffect(registerUserFlow(trace.initialInput), trace);
|
|
1174
|
+
assert.deepEqual(replayed, live);
|
|
1175
|
+
```
|
|
1176
|
+
|
|
1177
|
+
In production, record each run with a version and a flow name, and replay its trace anywhere with `timeTravel`, which narrates each step:
|
|
1178
|
+
|
|
1179
|
+
```js
|
|
1180
|
+
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
1181
|
+
|
|
1182
|
+
// Record a real run. The context's flowName names the flow in the trace.
|
|
1183
|
+
const { result, trace } = await recordEffect(checkoutFlow(input), {
|
|
1184
|
+
version: process.env.BUILD_ID,
|
|
1185
|
+
context: { flowName: 'checkout' }
|
|
1186
|
+
});
|
|
1187
|
+
|
|
1188
|
+
// Later, somewhere else, with nothing connected:
|
|
1189
|
+
await timeTravel(checkoutFlow, trace);
|
|
1190
|
+
```
|
|
1191
|
+
|
|
1192
|
+
```
|
|
1193
|
+
Replaying 'checkout' (3 recorded steps)
|
|
1194
|
+
Initial input: {
|
|
1195
|
+
"cartId": "cart_abc123",
|
|
1196
|
+
"promoCode": "FREE_YEAR_VIP"
|
|
1197
|
+
}
|
|
1198
|
+
Step 1: cmdFetchCart returned in 41.277ms {
|
|
1199
|
+
"totalAmount": "120.00"
|
|
1200
|
+
}
|
|
1201
|
+
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
1202
|
+
"isValid": true,
|
|
1203
|
+
"discountValue": 100
|
|
1204
|
+
}
|
|
1205
|
+
Step 3: cmdChargeCard threw in 181.507ms {
|
|
1206
|
+
"code": "invalid_amount",
|
|
1207
|
+
"name": "Error",
|
|
1208
|
+
"message": "Amount must be non-zero."
|
|
1209
|
+
}
|
|
1210
|
+
Replay finished with state: Failure
|
|
1211
|
+
Error: {
|
|
1212
|
+
"code": "invalid_amount",
|
|
1213
|
+
"name": "Error",
|
|
1214
|
+
"message": "Amount must be non-zero."
|
|
1215
|
+
}
|
|
1216
|
+
```
|
|
1217
|
+
|
|
1218
|
+
A production incident becomes a regression test, with no mocks:
|
|
1219
|
+
|
|
1220
|
+
```js
|
|
1221
|
+
it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
1222
|
+
const { result } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
1223
|
+
assert.equal(result.type, 'Failure');
|
|
1224
|
+
assert.equal(result.error.code, 'invalid_amount');
|
|
1225
|
+
});
|
|
1226
|
+
```
|
|
1227
|
+
|
|
1228
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
1229
|
+
|
|
1230
|
+
A step the trace lacks ends the replay in a `Failure` holding a `ReplayError`. A trace lacks a step that comes after the end of the recording, one a hook vetoed in production, and one the recorder dropped under `maxEntries`.
|
|
1231
|
+
|
|
1232
|
+
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
1233
|
+
|
|
1234
|
+
```js
|
|
1235
|
+
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
1236
|
+
const { result, unreached } = await replayEffect(checkoutFlow(trace.initialInput), trace);
|
|
1237
|
+
assert.equal(result.type, 'Success');
|
|
1238
|
+
assert.deepEqual(
|
|
1239
|
+
unreached.map((e) => e.command),
|
|
1240
|
+
['cmdChargeCard']
|
|
1241
|
+
);
|
|
1242
|
+
});
|
|
1243
|
+
```
|
|
1244
|
+
|
|
1245
|
+
For every other recording, `unreached` should be empty, so a fix that skips a step by accident fails the test. This works when the removed step is the last one the flow reaches. Remove a step from the middle and every later step moves to a different path, so the replay stops at a `TimeParadox`, and `unreached` lists the steps it never got to. [Proving a fix that changes the flow's shape](#proving-a-fix-that-changes-the-flows-shape) shows how to test such a fix.
|
|
1246
|
+
|
|
1247
|
+
A test can also run a whole flow with no recording at all, answering each Command itself; see [Testing a whole flow](#testing-a-whole-flow).
|
|
1248
|
+
|
|
1249
|
+
### Reading a trace
|
|
1250
|
+
|
|
1251
|
+
A trace is plain JSON. This one is from a checkout whose card charge timed out once and then went through:
|
|
1252
|
+
|
|
1253
|
+
```text
|
|
1254
|
+
{
|
|
1255
|
+
"flowName": "checkout",
|
|
1256
|
+
"version": "b42",
|
|
1257
|
+
"initialInput": { "cartId": "cart_1" },
|
|
1258
|
+
"context": { "flowName": "checkout" },
|
|
1259
|
+
"dropped": 0,
|
|
1260
|
+
"trace": [
|
|
1261
|
+
{ "command": "cmdFetchCart", "path": "0", "result": { "total": 120 }, "durationMs": 41.3 },
|
|
1262
|
+
{ "command": "cmdChargeCard", "path": "1r0/0", "threw": true,
|
|
1263
|
+
"error": { "__error": true, "name": "Error", "message": "gateway timeout", "code": "ETIMEDOUT" }, "durationMs": 5002.1 },
|
|
1264
|
+
{ "command": "cmdChargeCard", "path": "1r1/0", "result": { "chargeId": "ch_1" }, "durationMs": 181.5 }
|
|
1265
|
+
]
|
|
1266
|
+
}
|
|
1267
|
+
```
|
|
1268
|
+
|
|
1269
|
+
`trace` holds one entry per step, in the order the steps finished; [`recorder`](#recorderoptions) lists each kind of entry. A replay matches each step by its `path`, its position in the flow, and a `TimeParadox` names the path where the flow and the trace part ways:
|
|
1270
|
+
|
|
1271
|
+
- A number counts the Commands, `Retry`s and `Parallel`s the flow reached, from `0`. A `Success` or an `Ask` between them adds nothing.
|
|
1272
|
+
- `r` and a number is one attempt of the `Retry` at that position, `r0` being the first try, and `rf` is its `onExhausted` fallback.
|
|
1273
|
+
- `p` and a number is one branch of the `Parallel` at that position, from `p0`. The `Parallel` has an entry of its own, at a path such as `2p`, which records whether a branch cancelled the others.
|
|
1274
|
+
- `/` goes one level in. `1r1/0` is the first step of the second try of the `Retry` at position 1. `2p1/0` is the first step of the second branch of the `Parallel` at position 2.
|
|
1275
|
+
|
|
1276
|
+
### What a replay cannot check
|
|
1277
|
+
|
|
1278
|
+
A replay proves that the flow asks for the same Commands in the same order and handles the same answers. Every result comes from the recording, so it cannot check:
|
|
1279
|
+
|
|
1280
|
+
- **What the flow computes.** Fix how a refund amount is calculated and replay the bad run: the charge step still gets the old amount from the recording. The replay reports `Success`, returns the value from before the fix, and flags nothing. Test a value change against the pure step that computes it.
|
|
1281
|
+
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix. Catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
1282
|
+
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
1283
|
+
|
|
1284
|
+
When a Command's result changes shape, rename the Command. A trace matches it by name. So under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
1285
|
+
|
|
1286
|
+
## Recording in Production
|
|
1287
|
+
|
|
1288
|
+
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup. The two files imported below ship in the package's `examples` folder, `node_modules/pure-effect/examples`, and in the repository's [examples folder](https://github.com/aycangulez/pure-effect/blob/main/examples); copy them into your project.
|
|
1289
|
+
|
|
1290
|
+
```js
|
|
1291
|
+
import { randomUUID } from 'node:crypto';
|
|
1292
|
+
import { configureEffect } from 'pure-effect';
|
|
1293
|
+
import { recordingHooks } from './recording-example.js';
|
|
1294
|
+
import { telemetryHooks } from './opentelemetry-example.js';
|
|
1295
|
+
|
|
1296
|
+
configureEffect(
|
|
1297
|
+
telemetryHooks(),
|
|
1298
|
+
recordingHooks({
|
|
1299
|
+
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
1300
|
+
// Attached to each trace, so timeTravel warns when it replays one against another build.
|
|
1301
|
+
version: process.env.BUILD_ID,
|
|
1302
|
+
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
1303
|
+
redact: (value, name, kind) =>
|
|
1304
|
+
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
1305
|
+
maxEntries: 500,
|
|
1306
|
+
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
1307
|
+
})
|
|
1308
|
+
);
|
|
1309
|
+
```
|
|
1310
|
+
|
|
1311
|
+
What the example does with each run:
|
|
1312
|
+
|
|
1313
|
+
- **Which runs it keeps.** `keep` decides. By default it keeps a run that ended in a `Failure`, and discards a successful one after holding it in memory.
|
|
1314
|
+
- **When your own code throws.** A `TypeError` after an API changed the shape of its response, say, makes the run reject. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept, and replaying it reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded. That run did no I/O, so its input reproduces it.
|
|
1315
|
+
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out and marked `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
1316
|
+
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
1317
|
+
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
1318
|
+
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
1319
|
+
|
|
1320
|
+
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` or `effectGen` built, so build the outermost flow with one of them. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
1321
|
+
|
|
1322
|
+
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
1323
|
+
|
|
1324
|
+
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow. So neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes:
|
|
1325
|
+
|
|
1326
|
+
| Recorded | Replayed as |
|
|
1327
|
+
| ----------------------------------- | --------------------------------------------------- |
|
|
1328
|
+
| a money object or a database entity | a plain object, without its methods |
|
|
1329
|
+
| a `Date` | a string |
|
|
1330
|
+
| a `Map` or a `Set` | `{}` |
|
|
1331
|
+
| a `Buffer` | `{ type: 'Buffer', data }` |
|
|
1332
|
+
| `undefined` in an array | `null` |
|
|
1333
|
+
| an error returned inside a result | its own fields, such as `code`, without its message |
|
|
1334
|
+
|
|
1335
|
+
A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it. So a context holding a lazy client is left out whole.
|
|
1336
|
+
|
|
1337
|
+
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`. Rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
1338
|
+
|
|
1339
|
+
### Keeping secrets out of a trace
|
|
1340
|
+
|
|
1341
|
+
`redact` sees everything a trace stores: each result, each thrown error, the flow's input and the context, told apart by `kind`. It gets a copy in the form the trace stores, so changing it in place is safe, and what it returns is stored in that form too. A thrown `Error` arrives as a plain object holding its `name`, `message` and own fields, and any object returned for it replays as an `Error`.
|
|
1342
|
+
|
|
1343
|
+
Check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
1344
|
+
|
|
1345
|
+
```js
|
|
1346
|
+
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
1347
|
+
const mask = (value, field) =>
|
|
1348
|
+
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
1349
|
+
|
|
1350
|
+
const redact = (value, name, kind) => {
|
|
1351
|
+
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
1352
|
+
if (kind === 'context') return mask(value, 'authToken');
|
|
1353
|
+
if (kind === 'error') return mask(value, 'attempted');
|
|
1354
|
+
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
1355
|
+
};
|
|
1356
|
+
recorder({ redact });
|
|
1357
|
+
|
|
1358
|
+
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
1359
|
+
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
1360
|
+
```
|
|
1361
|
+
|
|
1362
|
+
A replay rebuilds the flow from the redacted input and context. So a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does. A context field an `Ask` reads should stay.
|
|
1363
|
+
|
|
1364
|
+
## Hooks
|
|
1365
|
+
|
|
1366
|
+
Hooks see every run, and recording, replay and the telemetry example are built on them. There are three: `onRun` wraps a whole run, `onStep` wraps each Command and each `Parallel`, and `onBeforeCommand` runs before each Command and can veto it by throwing. A hook that only watches must `await` what it wraps and return its result; [`configureEffect`](#configureeffectconfigs) lists each hook's arguments.
|
|
1367
|
+
|
|
1368
|
+
`configureEffect` adds a layer of hooks for the whole process and returns a function that removes it. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's:
|
|
1369
|
+
|
|
1370
|
+
```js
|
|
1371
|
+
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
1372
|
+
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
1373
|
+
remove();
|
|
1374
|
+
```
|
|
1375
|
+
|
|
1376
|
+
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
1377
|
+
|
|
1378
|
+
```
|
|
1379
|
+
runEffect(flow(input))
|
|
1380
|
+
│
|
|
1381
|
+
├─ telemetry.onRun first configuration: outermost
|
|
1382
|
+
│ └─ recording.onRun last configuration: innermost
|
|
1383
|
+
│ │
|
|
1384
|
+
│ │ for each Command:
|
|
1385
|
+
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
1386
|
+
│ ├─ recording.onBeforeCommand
|
|
1387
|
+
│ ├─ telemetry.onStep
|
|
1388
|
+
│ │ └─ recording.onStep closest to the Command
|
|
1389
|
+
│ │ └─ cmd()
|
|
1390
|
+
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
1391
|
+
│ ├─ telemetry.onStep returns from the inside out
|
|
1392
|
+
│ │
|
|
1393
|
+
│ ┌─ recording.onRun returns
|
|
1394
|
+
├─ telemetry.onRun returns
|
|
1395
|
+
```
|
|
1396
|
+
|
|
1397
|
+
**Knowing which request a step belongs to.** `onStep` and `onRun` get no context, so a hook that logs per request finds the request the way the rest of your logging does. An `AsyncLocalStorage` scope entered around `runEffect` reaches every hook the run calls, and the recording example uses one to give each run its own recorder:
|
|
1398
|
+
|
|
1399
|
+
```js
|
|
1400
|
+
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
1401
|
+
|
|
1402
|
+
const requestScope = new AsyncLocalStorage();
|
|
1403
|
+
const stepLog = [];
|
|
1404
|
+
|
|
1405
|
+
// Logs each step with its request and duration, and returns what the step returned.
|
|
1406
|
+
const removeStepLog = configureEffect({
|
|
1407
|
+
onStep: async (name, type, op, path) => {
|
|
1408
|
+
const started = performance.now();
|
|
1409
|
+
try {
|
|
1410
|
+
return await op();
|
|
1411
|
+
} finally {
|
|
1412
|
+
const { requestId } = requestScope.getStore() ?? {};
|
|
1413
|
+
stepLog.push({ requestId, name, path, ms: performance.now() - started });
|
|
1414
|
+
}
|
|
1415
|
+
}
|
|
1416
|
+
});
|
|
1417
|
+
|
|
1418
|
+
await requestScope.run({ requestId: 'req_1' }, () => runEffect(registerUserFlow(input)));
|
|
1419
|
+
assert.deepEqual(
|
|
1420
|
+
stepLog.map((line) => [line.requestId, line.name]),
|
|
1421
|
+
[
|
|
1422
|
+
['req_1', 'cmdFindUser'],
|
|
1423
|
+
['req_1', 'cmdSaveUser']
|
|
1424
|
+
]
|
|
1425
|
+
);
|
|
1426
|
+
removeStepLog();
|
|
1427
|
+
```
|
|
1428
|
+
|
|
1429
|
+
A Command that `onBeforeCommand` vetoes never reaches `onStep`, so log a veto where you make it. In tests, call `configureEffect()` with no arguments in a `beforeEach`, so a test that fails before removing its hooks does not leave them installed for the next.
|
|
1430
|
+
|
|
1431
|
+
**Per-call hooks.** `runEffect`'s third argument takes hooks for that call only. They are merged with the configured ones by the same rules: the configured wrappers run outside them, and the configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
1432
|
+
|
|
1433
|
+
```
|
|
1434
|
+
inherit: true (default) inherit: false
|
|
1435
|
+
|
|
1436
|
+
onRun C.onRun (none)
|
|
1437
|
+
onBeforeCommand C.onBeforeCommand (none)
|
|
1438
|
+
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
1439
|
+
```
|
|
1440
|
+
|
|
1441
|
+
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
1442
|
+
|
|
1443
|
+
## Replay in Depth
|
|
1444
|
+
|
|
1445
|
+
What [Time-Travel Debugging](#time-travel-debugging) leaves out: proving a fix that changes a flow's shape, keeping a replay faithful, and replaying from storage other than a trace.
|
|
1446
|
+
|
|
1447
|
+
### Proving a fix that changes the flow's shape
|
|
1448
|
+
|
|
1449
|
+
A fix that removes or moves a step in the middle of a flow changes the path of every step after it, so its old trace stops at a `TimeParadox`. Here the incident is a checkout that tried to charge a free cart, and the fix skips the charge, so the save moves to where the charge was:
|
|
1450
|
+
|
|
1451
|
+
```js
|
|
1452
|
+
const fetchCart = (cartId) => {
|
|
1453
|
+
const cmdFetchCart = () => carts.find(cartId);
|
|
1454
|
+
return Command(cmdFetchCart);
|
|
1455
|
+
};
|
|
1456
|
+
|
|
1457
|
+
const chargeCart = (cart) => {
|
|
1458
|
+
const cmdChargeCard = () => gateway.charge(cart.total);
|
|
1459
|
+
return Command(cmdChargeCard, () => Success(cart));
|
|
1460
|
+
};
|
|
1461
|
+
|
|
1462
|
+
const saveOrder = (cart) => {
|
|
1463
|
+
const cmdSaveOrder = () => orderStore.save(cart);
|
|
1464
|
+
return Command(cmdSaveOrder);
|
|
1465
|
+
};
|
|
1466
|
+
|
|
1467
|
+
// Production ran this, and recorded the charge failing.
|
|
1468
|
+
const checkoutBefore = (cartId) => effectPipe(fetchCart, chargeCart, saveOrder)(cartId);
|
|
1469
|
+
const { trace: incident } = await recordEffect(checkoutBefore('cart_free'));
|
|
1470
|
+
|
|
1471
|
+
// The fix: no charge when there is nothing to charge.
|
|
1472
|
+
const chargeIfDue = (cart) => (cart.total === 0 ? Success(cart) : chargeCart(cart));
|
|
1473
|
+
const checkoutFixed = (cartId) => effectPipe(fetchCart, chargeIfDue, saveOrder)(cartId);
|
|
1474
|
+
|
|
1475
|
+
// Against the old trace, the fixed flow stops where the save took the charge's place.
|
|
1476
|
+
const { result: paradox } = await replayEffect(checkoutFixed(incident.initialInput), incident);
|
|
1477
|
+
assert.equal(paradox.error.name, 'TimeParadox');
|
|
1478
|
+
```
|
|
1479
|
+
|
|
1480
|
+
Prove such a fix in three parts. Test the decision that changed, as a pure step. Then walk the fixed flow from the incident's input, handing each step the answer production got for it, for as long as the fix leaves the steps as they were:
|
|
1481
|
+
|
|
1482
|
+
```js
|
|
1483
|
+
import { commandName } from 'pure-effect';
|
|
1484
|
+
|
|
1485
|
+
// The decision that changed: a free cart is not charged.
|
|
1486
|
+
assert.deepEqual(chargeIfDue({ id: 'cart_free', total: 0 }), Success({ id: 'cart_free', total: 0 }));
|
|
1487
|
+
|
|
1488
|
+
// Given the cart production fetched, the fixed flow saves the order without charging it.
|
|
1489
|
+
const fetchedCart = incident.trace.find((entry) => entry.command === 'cmdFetchCart').result;
|
|
1490
|
+
const first = checkoutFixed(incident.initialInput);
|
|
1491
|
+
assert.equal(commandName(first), 'cmdFetchCart');
|
|
1492
|
+
assert.equal(commandName(first.next(fetchedCart)), 'cmdSaveOrder');
|
|
1493
|
+
```
|
|
1494
|
+
|
|
1495
|
+
The walk takes production's answers only up to the change. Past it, each recorded answer belongs to a step that has moved, so the trace has nothing more to say about the fixed flow. Last, once the fix ships, record the incident again, so its regression test is a plain replay once more.
|
|
1496
|
+
|
|
1497
|
+
### Keeping a replay faithful
|
|
1498
|
+
|
|
1499
|
+
Put anything that varies between runs inside a Command. For replay, the current time or a random ID counts as I/O. In a Command it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay, and drifts from the trace.
|
|
1500
|
+
|
|
1501
|
+
To check a flow, record it, replay the trace, and compare the outcomes. A step that computes a fresh value shows up as a different final value, or as a `TimeParadox` if it changes which Commands run. A trace holds each result in the form JSON stores it, so this also catches a result your storage cannot keep, such as a `Date` the flow calls a method on:
|
|
1502
|
+
|
|
1503
|
+
```js
|
|
1504
|
+
const { result, trace } = await recordEffect(registerUserFlow(input));
|
|
1505
|
+
const { result: replayed } = await replayEffect(registerUserFlow(input), trace);
|
|
1506
|
+
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive storage
|
|
1507
|
+
```
|
|
1508
|
+
|
|
1509
|
+
A `Failure` compares the same way when the Command threw a plain `Error`. It comes back with its message, name, cause, custom properties and an `AggregateError`'s list of errors, even a `name` or `cause` set after it was created. An error of any other class, built-in ones included, comes back as a plain `Error` carrying that class's name. So `deepEqual` from `node:assert/strict` refuses the replayed `TypeError` that `fetch` rejects with when the network fails. Where a Command can throw one, compare `error.name` and `error.message` instead.
|
|
1510
|
+
|
|
1511
|
+
### Replaying from your own storage
|
|
1512
|
+
|
|
1513
|
+
`replayEffect` also accepts a resolver function instead of a trace, so you can replay from OpenTelemetry spans, a log pipeline, or a database table, not only from the JSON that `recorder` produces. Whatever you store needs each step's `path`, which `onStep` receives, and its result; the telemetry example records neither, since it keeps values off spans.
|
|
1514
|
+
|
|
1515
|
+
```js
|
|
1516
|
+
// A resolver answers one question: what did production get back for this step? Look the step up by its path, its
|
|
1517
|
+
// position in the flow. `step.index` counts steps in the order they finished, which shifts around a Parallel.
|
|
1518
|
+
const outputs = new Map(mySpans.map((span) => [span.attributes.path, span.attributes.output]));
|
|
1519
|
+
const resolve = (step) => (outputs.has(step.path) ? { result: outputs.get(step.path) } : undefined);
|
|
1520
|
+
await replayEffect(checkoutFlow(input), resolve);
|
|
1521
|
+
```
|
|
1522
|
+
|
|
1250
1523
|
## Why Pure Effect
|
|
1251
1524
|
|
|
1252
1525
|
**vs. Temporal and durable execution (Restate, Inngest):** The closest relatives, built on the same idea: record what every step returned and replay it. Those engines store the history on a server and resume workflows automatically after a crash. Pure Effect leaves storage and restarts to your application, and there is no infrastructure to run.
|
|
@@ -1366,7 +1639,7 @@ Runs a flow and resolves with its `Success` or `Failure`. See [Which Errors Are
|
|
|
1366
1639
|
|
|
1367
1640
|
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
1368
1641
|
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the configured hooks, or used alone with `inherit: false`; see [Hooks](#hooks).
|
|
1369
|
-
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure`.
|
|
1642
|
+
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure` holding the error thrown; under a `Retry` that gave up, it holds `{ retryExhausted, lastError, retries }`.
|
|
1370
1643
|
- A malformed flow, or a throw from a `next` or a pure step, is a bug. The call that builds the flow throws it before the first Command, and `runEffect` rejects with it after.
|
|
1371
1644
|
|
|
1372
1645
|
#### `configureEffect(...configs)`
|
|
@@ -1418,7 +1691,7 @@ Runs a flow with each Command answered from a trace instead of run, and returns
|
|
|
1418
1691
|
|
|
1419
1692
|
- A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`.
|
|
1420
1693
|
- A malformed trace rejects with a `ReplayError`.
|
|
1421
|
-
- `traceOrResolver`: a trace, its bare entries, or a resolver: a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of. See [Replaying from your own storage](#replaying-from-your-own-storage).
|
|
1694
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver: a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of. See [Testing a whole flow](#testing-a-whole-flow) and [Replaying from your own storage](#replaying-from-your-own-storage).
|
|
1422
1695
|
- Any other answer from a resolver, `null` included, or a throw from it, rejects the replay. With a resolver, `unreached` is absent.
|
|
1423
1696
|
- A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`. Answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1424
1697
|
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`; a trace whose context could not be recorded refuses to replay without one.
|