pure-effect 0.18.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +678 -306
- package/index.d.ts +82 -22
- package/index.js +625 -48
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Pure Effect
|
|
2
2
|
|
|
3
|
-
[](https://www.npmjs.com/package/pure-effect) [](https://www.npmjs.com/package/pure-effect) [](https://github.com/aycangulez/pure-effect/actions/workflows/ci.yml) [](https://www.npmjs.com/package/pure-effect) [](https://github.com/aycangulez/pure-effect/blob/main/LICENSE)
|
|
4
4
|
|
|
5
5
|
**Pure Effect** records what your business logic did in production and replays it anywhere: time-travel debugging for JavaScript and TypeScript, with zero dependencies. Business logic is plain data you can test without mocks.
|
|
6
6
|
|
|
@@ -9,17 +9,17 @@
|
|
|
9
9
|
- Inject context without touching function signatures
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
11
|
- OpenTelemetry-ready via lifecycle hooks
|
|
12
|
-
- Zero dependencies, about
|
|
12
|
+
- Zero dependencies, about 11 KB minified and gzipped
|
|
13
13
|
- Works in JavaScript, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
16
16
|
|
|
17
|
-
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [How It Works](#how-it-works) · [Testing Without Mocks](#testing-without-mocks) · [Coming from async/await](#coming-from-asyncawait)
|
|
18
|
-
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production)
|
|
19
|
-
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Which Errors Are Data](#which-errors-are-data) · [TypeScript](#typescript-typed-errors-and-context)
|
|
20
|
-
- **Reference:** [Why Pure Effect](#why-pure-effect) · [
|
|
17
|
+
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [How It Works](#how-it-works) · [Testing Without Mocks](#testing-without-mocks) · [Coming from async/await](#coming-from-asyncawait) · [Writing Flows as Generators](#writing-flows-as-generators)
|
|
18
|
+
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production) · [Hooks](#hooks)
|
|
19
|
+
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Writing Loops](#writing-loops) · [Which Errors Are Data](#which-errors-are-data) · [Generator Flows in Depth](#generator-flows-in-depth) · [TypeScript](#typescript-typed-errors-and-context)
|
|
20
|
+
- **Reference:** [Why Pure Effect](#why-pure-effect) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
21
|
|
|
22
|
-
The
|
|
22
|
+
The six sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: recording, retries, loops, steps that run at the same time, or TypeScript.
|
|
23
23
|
|
|
24
24
|
## Installation
|
|
25
25
|
|
|
@@ -70,9 +70,21 @@ async function registerUser(input) {
|
|
|
70
70
|
}
|
|
71
71
|
```
|
|
72
72
|
|
|
73
|
+
The same flow can be written as a generator, with `yield* step(...)` where `async` code writes `await`. [Writing Flows as Generators](#writing-flows-as-generators) covers what to know.
|
|
74
|
+
|
|
75
|
+
```js
|
|
76
|
+
import { effectGen, step } from 'pure-effect';
|
|
77
|
+
|
|
78
|
+
const registerUserFlowGen = effectGen(function* (input) {
|
|
79
|
+
yield* step(validateRegistration(input));
|
|
80
|
+
yield* step(ensureEmailAvailable(input));
|
|
81
|
+
return yield* step(saveUser(input));
|
|
82
|
+
});
|
|
83
|
+
```
|
|
84
|
+
|
|
73
85
|
## How It Works
|
|
74
86
|
|
|
75
|
-
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop.
|
|
87
|
+
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop. A generator flow builds the same Commands: `effectGen` runs your function up to each `yield*` and continues it with the answer.
|
|
76
88
|
|
|
77
89
|
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
78
90
|
|
|
@@ -80,9 +92,7 @@ Building a flow runs the pure steps right away: `registerUserFlow(badInput)` ret
|
|
|
80
92
|
|
|
81
93
|
## Testing Without Mocks
|
|
82
94
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
Validation runs as soon as you build the flow, so testing it needs nothing else:
|
|
95
|
+
A flow is plain objects, so a test can check what it will do without running it. Validation runs as soon as the flow is built, so testing it needs nothing else:
|
|
86
96
|
|
|
87
97
|
```js
|
|
88
98
|
const badInput = { email: 'bad-email', password: '123' };
|
|
@@ -114,62 +124,63 @@ assert.equal(commandName(step2), 'cmdSaveUser');
|
|
|
114
124
|
// The database was never touched.
|
|
115
125
|
```
|
|
116
126
|
|
|
117
|
-
Write both kinds
|
|
127
|
+
Write both kinds: only a step test sees the value passed from one step to the next. Suppose the email guard returned `Success(true)` instead of `Success(input)`. The step test would fail. The flow test would still pass, since the calls and their order are unchanged, while the flow saved `true` instead of the user.
|
|
118
128
|
|
|
119
|
-
|
|
129
|
+
A walk meets a `Retry` or a `Parallel` as a step of its own. A `Retry` holds the step it repeats as `effect`, a `Parallel` holds its branches as `effects`, and the `next` of either is the rest of the flow. The walks in [Writing Loops](#writing-loops) and [Undoing what succeeded when a branch fails](#undoing-what-succeeded-when-a-branch-fails) go through each.
|
|
120
130
|
|
|
121
|
-
|
|
131
|
+
To run the whole flow instead of walking it, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command by name. Nothing is called and no retry waits; answering `{ error }` makes that Command throw, which is how a test reaches a `Retry`'s fallback:
|
|
122
132
|
|
|
123
133
|
```js
|
|
134
|
+
import { replayEffect } from 'pure-effect';
|
|
135
|
+
|
|
124
136
|
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
125
137
|
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
126
138
|
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
127
139
|
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
128
140
|
```
|
|
129
141
|
|
|
130
|
-
|
|
142
|
+
Hand it a recorded run instead, and each Command gets what it returned the first time. That is how a run recorded in production replays on your machine, with nothing connected:
|
|
131
143
|
|
|
132
|
-
|
|
144
|
+
```js
|
|
145
|
+
import { recordEffect } from 'pure-effect';
|
|
133
146
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
import { commandName } from 'pure-effect';
|
|
137
|
-
import type { Effect } from 'pure-effect';
|
|
147
|
+
// A real run returns its result, and a trace of what each Command returned.
|
|
148
|
+
const { result: live, trace } = await recordEffect(registerUserFlow(input));
|
|
138
149
|
|
|
139
|
-
//
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
assert.equal(commandName(step), name);
|
|
143
|
-
return step;
|
|
144
|
-
}
|
|
145
|
-
|
|
146
|
-
const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
147
|
-
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
150
|
+
// Anywhere else, with nothing connected, the same flow gets the same answers.
|
|
151
|
+
const { result: replayed } = await replayEffect(registerUserFlow(trace.initialInput), trace);
|
|
152
|
+
assert.deepEqual(replayed, live);
|
|
148
153
|
```
|
|
149
154
|
|
|
150
|
-
|
|
155
|
+
[Time-Travel Debugging](#time-travel-debugging) turns such a trace into a regression test, and [Recording in Production](#recording-in-production) records every run of an application.
|
|
156
|
+
|
|
157
|
+
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them, with the services passed in as [Passing Runtime Context](#passing-runtime-context) shows.
|
|
158
|
+
|
|
159
|
+
In TypeScript, a walk needs a check at each step that the step is a Command; [Testing a flow in TypeScript](#testing-a-flow-in-typescript) has a helper that makes that one line.
|
|
151
160
|
|
|
152
161
|
## Coming from async/await
|
|
153
162
|
|
|
154
|
-
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command
|
|
163
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command: which call to make, and what to do with the answer. Each thing you would write with `async`/`await` has an equivalent:
|
|
155
164
|
|
|
156
|
-
|
|
|
157
|
-
| --- | --- |
|
|
158
|
-
| `
|
|
159
|
-
| the code after the `await
|
|
160
|
-
| an `if` on `
|
|
161
|
-
| `return value` | `Success(value)` |
|
|
162
|
-
| `throw` to stop | `Failure(error)` |
|
|
163
|
-
|
|
|
164
|
-
| a loop that retries a call | `Retry(
|
|
165
|
-
| `await Promise.all([...])` | `Parallel([...])` |
|
|
166
|
-
| a `for` loop with an `await` inside |
|
|
167
|
-
| a
|
|
168
|
-
| a
|
|
169
|
-
| a value from the request
|
|
170
|
-
| `Date.now()` or a random ID | a Command
|
|
171
|
-
|
|
172
|
-
|
|
165
|
+
| `async`/`await` | Generator flow | Pipeline |
|
|
166
|
+
| --- | --- | --- |
|
|
167
|
+
| `await db.findUser(email)` | `yield* step(findUser(email))` | `Command(() => db.findUser(email), next)` |
|
|
168
|
+
| the code after the `await` | the next `yield*` | the Command's `next` |
|
|
169
|
+
| an `if` on a result | a step that returns `Success` or `Failure` | a branch in `next` |
|
|
170
|
+
| `return value` | `return value` | `Success(value)` |
|
|
171
|
+
| `throw` to stop | a step that returns `Failure(error)` | `Failure(error)` |
|
|
172
|
+
| `try`/`catch` around a call | a catch inside the Command's function | the same |
|
|
173
|
+
| a loop that retries a call | `yield* step(Retry(...))` | `Retry(...)` |
|
|
174
|
+
| `await Promise.all([...])` | `yield* step(Parallel([...]))` | `Parallel([...])` |
|
|
175
|
+
| a `for` loop with an `await` inside | the same loop, with `yield*` | `Parallel(list.map(...), { limit: 1 })` |
|
|
176
|
+
| a loop that keeps going past a failure | `Parallel(..., { settled: true })` | the same |
|
|
177
|
+
| a call that needs two earlier results | two variables | a named step that passes both on |
|
|
178
|
+
| a value from the request | `yield* step(Ask())` | `Ask` |
|
|
179
|
+
| `Date.now()` or a random ID | a Command | a Command |
|
|
180
|
+
|
|
181
|
+
`findUser` stands for a step that returns the Command, as `ensureEmailAvailable` does in the Quick Start. A `catch` around `yield*` can log and rethrow, but it cannot recover; see [Which Errors Are Data](#which-errors-are-data).
|
|
182
|
+
|
|
183
|
+
One rule is behind both columns: each decision lives in a small step function that receives the answer it depends on and returns `Success` or `Failure`, so a test can call it directly. A pipeline joins those steps with `effectPipe`; a generator flow puts them in order with `yield*`. An `if` around a `yield*` works, but it can be tested only by walking the whole flow, so keep the body to putting steps in order. A `for` loop that runs a step for each item decides nothing, so it is fine.
|
|
173
184
|
|
|
174
185
|
The Quick Start's `registerUserFlow` is this function, translated row by row: each `await` became a Command, the code after it became that Command's `next`, and each `throw` became a `Failure`.
|
|
175
186
|
|
|
@@ -182,7 +193,7 @@ async function registerUserAsync(input) {
|
|
|
182
193
|
}
|
|
183
194
|
```
|
|
184
195
|
|
|
185
|
-
The difference shows
|
|
196
|
+
The difference shows in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace the database's methods, run the function, and put them back, here with Node's built-in mocks:
|
|
186
197
|
|
|
187
198
|
```js
|
|
188
199
|
import { mock } from 'node:test';
|
|
@@ -203,9 +214,9 @@ The flow's test of the same branch hands one step the answer, with nothing to re
|
|
|
203
214
|
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
204
215
|
```
|
|
205
216
|
|
|
206
|
-
|
|
217
|
+
Each step is a place a test can start, so a test reaches a branch by calling the step it lives in. A test of an `async` function has to run everything before the branch, with every call on the way replaced. The mocks do check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
207
218
|
|
|
208
|
-
You can switch one function at a time.
|
|
219
|
+
You can switch one function at a time. Keep its name and parameters, run the flow in its body, and turn a `Failure` back into a throw, so the code that calls it does not change. The mocked test above passes against this version too:
|
|
209
220
|
|
|
210
221
|
```js
|
|
211
222
|
// Callers still await registerUserAsync and catch what it throws.
|
|
@@ -219,84 +230,49 @@ async function registerUserAsync(input) {
|
|
|
219
230
|
await assert.rejects(registerUserAsync({ email: 'bad-email', password: '123' }), { message: 'Invalid email.' });
|
|
220
231
|
```
|
|
221
232
|
|
|
222
|
-
A loop
|
|
223
|
-
|
|
224
|
-
```js
|
|
225
|
-
async function payUnpaidAsync(customerId) {
|
|
226
|
-
const invoices = await billing.findUnpaid(customerId);
|
|
227
|
-
for (const invoice of invoices) {
|
|
228
|
-
const receipt = await payments.charge(invoice);
|
|
229
|
-
await mailer.sendReceipt(receipt);
|
|
230
|
-
}
|
|
231
|
-
}
|
|
233
|
+
A loop becomes a step that gets the list and a `Parallel` over it, as the table shows; [Writing Loops](#writing-loops) translates one.
|
|
232
234
|
|
|
233
|
-
|
|
234
|
-
const findUnpaidInvoices = (customerId) => {
|
|
235
|
-
const cmdFindUnpaid = () => billing.findUnpaid(customerId);
|
|
236
|
-
return Command(cmdFindUnpaid);
|
|
237
|
-
};
|
|
235
|
+
## Writing Flows as Generators
|
|
238
236
|
|
|
239
|
-
|
|
240
|
-
const chargeInvoice = (invoice) => {
|
|
241
|
-
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
242
|
-
return Retry(Command(cmdChargeInvoice), { attempts: 3 });
|
|
243
|
-
};
|
|
237
|
+
A generator flow is written like `async` code, with `yield* step(...)` where `async` code writes `await`. `effectGen` turns it into the same Commands a pipeline builds, so recording, replay, `Retry`, `Parallel` and the tests above work the same for both. The Quick Start's `registerUserFlowGen` is one. The same steps in the same order record the same paths in either form, so a trace recorded from a pipeline replays against its generator version. A loop written as a `for` loop rather than a `Parallel` records differently; see [Writing Loops](#writing-loops).
|
|
244
238
|
|
|
245
|
-
|
|
246
|
-
const cmdSendReceipt = () => mailer.sendReceipt(receipt);
|
|
247
|
-
return Command(cmdSendReceipt);
|
|
248
|
-
};
|
|
239
|
+
### Where a generator helps
|
|
249
240
|
|
|
250
|
-
|
|
241
|
+
A generator earns its keep when a later step needs the results of more than one earlier step. Here the welcome email needs both the saved user and the workspace created for them, and each is a variable:
|
|
251
242
|
|
|
252
|
-
|
|
253
|
-
const
|
|
243
|
+
```js
|
|
244
|
+
const onboardUser = effectGen(function* (input) {
|
|
245
|
+
const user = yield* step(registerUserFlowGen(input));
|
|
254
246
|
|
|
255
|
-
const
|
|
247
|
+
const cmdCreateWorkspace = () => db.createWorkspace(user.id);
|
|
248
|
+
const workspace = yield* step(Command(cmdCreateWorkspace));
|
|
256
249
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
assert.equal(commandName(flow), 'cmdFindUnpaid');
|
|
260
|
-
assert.equal(flow.next([{ id: 1 }, { id: 2 }]).effects.length, 2);
|
|
250
|
+
const cmdSendWelcome = () => mailer.sendWelcome(user.email, workspace.name);
|
|
251
|
+
yield* step(Command(cmdSendWelcome));
|
|
261
252
|
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
assert.equal(commandName(body.effect), 'cmdChargeInvoice');
|
|
265
|
-
assert.equal(commandName(body.next({ id: 'receipt_1' })), 'cmdSendReceipt');
|
|
253
|
+
return { user, workspace };
|
|
254
|
+
});
|
|
266
255
|
|
|
267
|
-
|
|
268
|
-
assert.
|
|
256
|
+
const onboarded = await runEffect(onboardUser(input));
|
|
257
|
+
assert.equal(onboarded.value.workspace.ownerId, onboarded.value.user.id);
|
|
269
258
|
```
|
|
270
259
|
|
|
271
|
-
|
|
260
|
+
A pipeline would need a step that passes the user and the workspace on together; see [Composing Larger Flows](#composing-larger-flows). Keep decisions in step functions all the same, as `registerUserFlowGen` does, so each can be tested on its own.
|
|
272
261
|
|
|
273
|
-
The
|
|
262
|
+
The body can run more than once: for each `Retry` attempt and each run, as an `async` function does when called again, and when a test walks a second branch. So keep it to its own variables, and get `Date.now()` or a random ID from a Command, so a replay gets the recorded one. [Generator Flows in Depth](#generator-flows-in-depth) covers when it runs again, how a `catch` or `finally` in it behaves when a step stops the flow, and the mistakes `effectGen` catches.
|
|
274
263
|
|
|
275
|
-
|
|
264
|
+
### Testing a generator flow
|
|
276
265
|
|
|
277
|
-
|
|
266
|
+
Test each step function by handing it an answer, as [Testing Without Mocks](#testing-without-mocks) does, since the generator only puts the steps in order. Then run the whole flow with each Command answered by name, as for a pipeline:
|
|
278
267
|
|
|
279
268
|
```js
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
const page = await api.listOrders(cursor);
|
|
285
|
-
orders.push(...page.items);
|
|
286
|
-
cursor = page.nextCursor;
|
|
287
|
-
} while (cursor);
|
|
288
|
-
return orders;
|
|
289
|
-
}
|
|
290
|
-
|
|
291
|
-
const listAllOrders = () => {
|
|
292
|
-
const cmdListAllOrders = () => fetchAllOrders();
|
|
293
|
-
return Command(cmdListAllOrders);
|
|
294
|
-
};
|
|
269
|
+
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
270
|
+
const answerByName = (asked) => (asked.name in answers ? { result: answers[asked.name] } : undefined);
|
|
271
|
+
const { result } = await replayEffect(registerUserFlowGen(input), answerByName);
|
|
272
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
295
273
|
```
|
|
296
274
|
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
275
|
+
Walking the flow step by step works as it does for a pipeline: a walk straight through runs the body once, and walking a second branch runs it again.
|
|
300
276
|
|
|
301
277
|
## Time-Travel Debugging
|
|
302
278
|
|
|
@@ -306,7 +282,7 @@ Record what each Command returned, then feed those results back into the flow to
|
|
|
306
282
|
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
307
283
|
|
|
308
284
|
// Record a real run. The context's flowName names the flow in the trace.
|
|
309
|
-
const { result, trace } = await recordEffect(checkoutFlow
|
|
285
|
+
const { result, trace } = await recordEffect(checkoutFlow(input), {
|
|
310
286
|
version: process.env.BUILD_ID,
|
|
311
287
|
context: { flowName: 'checkout' }
|
|
312
288
|
});
|
|
@@ -351,7 +327,9 @@ it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
|
351
327
|
});
|
|
352
328
|
```
|
|
353
329
|
|
|
354
|
-
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
330
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. It does not check the values the flow computes, since every result comes from the recording; see [What a replay cannot check](#what-a-replay-cannot-check). If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`.
|
|
331
|
+
|
|
332
|
+
A step the trace lacks ends the replay in a `Failure` holding a `ReplayError`. A trace lacks a step that comes after the end of the recording, one a hook vetoed in production, and one the recorder dropped under `maxEntries`.
|
|
355
333
|
|
|
356
334
|
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
357
335
|
|
|
@@ -366,29 +344,62 @@ it('incident 8f3a fixed: a 100% promo checks out without a charge', async () =>
|
|
|
366
344
|
});
|
|
367
345
|
```
|
|
368
346
|
|
|
369
|
-
For every other recording, `unreached` should be empty, so a fix that skips a step by accident fails the test.
|
|
347
|
+
For every other recording, `unreached` should be empty, so a fix that skips a step by accident fails the test. This works when the removed step is the last one the flow reaches. Remove a step from the middle and every later step moves to a different path, so the replay stops at a `TimeParadox`, and `unreached` lists the steps it never got to.
|
|
348
|
+
|
|
349
|
+
### Reading a trace
|
|
350
|
+
|
|
351
|
+
A trace is plain JSON. This one is from a checkout whose card charge timed out once and then went through:
|
|
352
|
+
|
|
353
|
+
```text
|
|
354
|
+
{
|
|
355
|
+
"flowName": "checkout",
|
|
356
|
+
"version": "b42",
|
|
357
|
+
"initialInput": { "cartId": "cart_1" },
|
|
358
|
+
"context": { "flowName": "checkout" },
|
|
359
|
+
"dropped": 0,
|
|
360
|
+
"trace": [
|
|
361
|
+
{ "command": "cmdFetchCart", "path": "0", "result": { "total": 120 }, "durationMs": 41.3 },
|
|
362
|
+
{ "command": "cmdChargeCard", "path": "1r0/0", "threw": true,
|
|
363
|
+
"error": { "__error": true, "name": "Error", "message": "gateway timeout", "code": "ETIMEDOUT" }, "durationMs": 5002.1 },
|
|
364
|
+
{ "command": "cmdChargeCard", "path": "1r1/0", "result": { "chargeId": "ch_1" }, "durationMs": 181.5 }
|
|
365
|
+
]
|
|
366
|
+
}
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
`trace` holds one entry per step, in the order the steps finished; [`recorder`](#recorderoptions) lists each kind of entry. A replay matches each step by its `path`, its position in the flow, and a `TimeParadox` names the path where the flow and the trace part ways:
|
|
370
|
+
|
|
371
|
+
- A number counts the Commands, `Retry`s and `Parallel`s the flow reached, from `0`. A `Success` or an `Ask` between them adds nothing.
|
|
372
|
+
- `r` and a number is one attempt of the `Retry` at that position, `r0` being the first try, and `rf` is its `onExhausted` fallback.
|
|
373
|
+
- `p` and a number is one branch of the `Parallel` at that position, from `p0`. The `Parallel` has an entry of its own, at a path such as `2p`, which records whether a branch cancelled the others.
|
|
374
|
+
- `/` goes one level in. `1r1/0` is the first step of the second try of the `Retry` at position 1. `2p1/0` is the first step of the second branch of the `Parallel` at position 2.
|
|
370
375
|
|
|
371
|
-
|
|
376
|
+
### What a replay cannot check
|
|
372
377
|
|
|
373
|
-
|
|
378
|
+
A replay proves that the flow asks for the same Commands in the same order and handles the same answers. Every result comes from the recording, so it cannot check:
|
|
374
379
|
|
|
375
|
-
**
|
|
380
|
+
- **What the flow computes.** Fix how a refund amount is calculated and replay the bad run: the charge step still gets the old amount from the recording. The replay reports `Success`, returns the value from before the fix, and flags nothing. Test a value change against the pure step that computes it.
|
|
381
|
+
- **A fix inside a Command's function.** The function never runs, so an old trace hands the flow what it returned or threw before the fix. Catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test.
|
|
382
|
+
- **The arguments a Command was called with.** A trace records results only. Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds its arguments for a debugger, but a test cannot assert on them. Storing them would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
376
383
|
|
|
377
|
-
|
|
384
|
+
When a Command's result changes shape, rename the Command. A trace matches it by name. So under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
378
385
|
|
|
379
|
-
|
|
386
|
+
### Keeping a replay faithful
|
|
380
387
|
|
|
381
|
-
|
|
388
|
+
Put anything that varies between runs inside a Command. For replay, the current time or a random ID counts as I/O. In a Command it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay, and drifts from the trace.
|
|
389
|
+
|
|
390
|
+
To check a flow, record it, replay the trace, and compare the outcomes. A step that computes a fresh value shows up as a different final value, or as a `TimeParadox` if it changes which Commands run. A trace holds each result in the form JSON stores it, so this also catches a result your storage cannot keep, such as a `Date` the flow calls a method on:
|
|
382
391
|
|
|
383
392
|
```js
|
|
384
|
-
const { result, trace } = await recordEffect(registerUserFlow
|
|
393
|
+
const { result, trace } = await recordEffect(registerUserFlow(input));
|
|
385
394
|
const { result: replayed } = await replayEffect(registerUserFlow(input), trace);
|
|
386
395
|
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive storage
|
|
387
396
|
```
|
|
388
397
|
|
|
389
|
-
|
|
398
|
+
A `Failure` compares the same way when the Command threw a plain `Error`. It comes back with its message, name, cause, custom properties and an `AggregateError`'s list of errors, even a `name` or `cause` set after it was created. An error of any other class, built-in ones included, comes back as a plain `Error` carrying that class's name. So `deepEqual` from `node:assert/strict` refuses the replayed `TypeError` that `fetch` rejects with when the network fails. Where a Command can throw one, compare `error.name` and `error.message` instead.
|
|
399
|
+
|
|
400
|
+
### Replaying from your own storage
|
|
390
401
|
|
|
391
|
-
|
|
402
|
+
`replayEffect` also accepts a resolver function instead of a trace, so you can replay from OpenTelemetry spans, a log pipeline, or a database table, not only from the JSON that `recorder` produces. Whatever you store needs each step's `path`, which `onStep` receives, and its result; the telemetry example records neither, since it keeps values off spans.
|
|
392
403
|
|
|
393
404
|
```js
|
|
394
405
|
// A resolver answers one question: what did production get back for this step? Look the step up by its path, its
|
|
@@ -421,13 +432,137 @@ configureEffect(
|
|
|
421
432
|
);
|
|
422
433
|
```
|
|
423
434
|
|
|
424
|
-
|
|
435
|
+
What the example does with each run:
|
|
436
|
+
|
|
437
|
+
- **Which runs it keeps.** `keep` decides. By default it keeps a run that ended in a `Failure`, and discards a successful one after holding it in memory.
|
|
438
|
+
- **When your own code throws.** A `TypeError` after an API changed the shape of its response, say, makes the run reject. `keep` is offered a `Failure` carrying the thrown error, so by default its trace is kept, and replaying it reproduces the throw. A step before the flow's first Command runs when the flow is built, outside `runEffect`, so its throw reaches no hook and is not recorded. That run did no I/O, so its input reproduces it.
|
|
439
|
+
- **What a trace holds.** `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. A value it throws on, or one JSON cannot hold, such as a `BigInt`, is left out and marked `unrecorded`, and a replay stops there. Everything else is already in the form JSON holds, so the sink can write it to S3 or a database column with `JSON.stringify`.
|
|
440
|
+
- **How long a trace gets.** `maxEntries` caps it and reports the overflow as `dropped`, and a capped trace replays only up to the first step it lacks.
|
|
441
|
+
- **When storage fails.** If `keep` or the sink throws, for example because storage is unreachable, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`.
|
|
442
|
+
- **What it warns about.** The example warns through `onWarning`, once per flow, when it keeps a trace that was capped, one holding values it could not record, or one whose steps include any named `'anonymous'`.
|
|
443
|
+
|
|
444
|
+
The example takes the input from `onRun`, which is handed it only for a flow `effectPipe` or `effectGen` built, so build the outermost flow with one of them. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one.
|
|
445
|
+
|
|
446
|
+
The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
447
|
+
|
|
448
|
+
**A trace keeps data, not objects.** A value is copied into the form JSON stores when it is recorded, and copied again each time a replay hands it to the flow. So neither a later step nor a replay can change what the trace says, and a replay from memory gets what a replay from storage gets. The copy keeps values but not classes:
|
|
449
|
+
|
|
450
|
+
| Recorded | Replayed as |
|
|
451
|
+
| ----------------------------------- | --------------------------------------------------- |
|
|
452
|
+
| a money object or a database entity | a plain object, without its methods |
|
|
453
|
+
| a `Date` | a string |
|
|
454
|
+
| a `Map` or a `Set` | `{}` |
|
|
455
|
+
| a `Buffer` | `{ type: 'Buffer', data }` |
|
|
456
|
+
| `undefined` in an array | `null` |
|
|
457
|
+
| an error returned inside a result | its own fields, such as `code`, without its message |
|
|
458
|
+
|
|
459
|
+
A reference back to an object the value is inside of, as between an HTTP client's request and response, is cut. A value JSON cannot hold at all, such as a `BigInt` or an object whose getter throws, is left out of the trace and marked `unrecorded` before `redact` sees it. So a context holding a lazy client is left out whole.
|
|
460
|
+
|
|
461
|
+
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`. Rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
462
|
+
|
|
463
|
+
### Keeping secrets out of a trace
|
|
464
|
+
|
|
465
|
+
`redact` sees everything a trace stores: each result, each thrown error, the flow's input and the context, told apart by `kind`. It gets a copy in the form the trace stores, so changing it in place is safe, and what it returns is stored in that form too. A thrown `Error` arrives as a plain object holding its `name`, `message` and own fields, and any object returned for it replays as an `Error`.
|
|
466
|
+
|
|
467
|
+
Check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
468
|
+
|
|
469
|
+
```js
|
|
470
|
+
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
471
|
+
const mask = (value, field) =>
|
|
472
|
+
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
473
|
+
|
|
474
|
+
const redact = (value, name, kind) => {
|
|
475
|
+
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
476
|
+
if (kind === 'context') return mask(value, 'authToken');
|
|
477
|
+
if (kind === 'error') return mask(value, 'attempted');
|
|
478
|
+
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
479
|
+
};
|
|
480
|
+
recorder({ redact });
|
|
481
|
+
|
|
482
|
+
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
483
|
+
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
484
|
+
```
|
|
485
|
+
|
|
486
|
+
A replay rebuilds the flow from the redacted input and context. So a field a step checks, such as a password, needs a stand-in the check treats the same way, as the example at the top of this section does. A context field an `Ask` reads should stay.
|
|
487
|
+
|
|
488
|
+
## Hooks
|
|
489
|
+
|
|
490
|
+
Hooks see every run, and recording, replay and the telemetry example are built on them. There are three: `onRun` wraps a whole run, `onStep` wraps each Command and each `Parallel`, and `onBeforeCommand` runs before each Command and can veto it by throwing. A hook that only watches must `await` what it wraps and return its result; [`configureEffect`](#configureeffectconfigs) lists each hook's arguments.
|
|
491
|
+
|
|
492
|
+
`configureEffect` adds a layer of hooks for the whole process and returns a function that removes it. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's:
|
|
493
|
+
|
|
494
|
+
```js
|
|
495
|
+
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
496
|
+
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
497
|
+
remove();
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
501
|
+
|
|
502
|
+
```
|
|
503
|
+
runEffect(flow(input))
|
|
504
|
+
│
|
|
505
|
+
├─ telemetry.onRun first configuration: outermost
|
|
506
|
+
│ └─ recording.onRun last configuration: innermost
|
|
507
|
+
│ │
|
|
508
|
+
│ │ for each Command:
|
|
509
|
+
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
510
|
+
│ ├─ recording.onBeforeCommand
|
|
511
|
+
│ ├─ telemetry.onStep
|
|
512
|
+
│ │ └─ recording.onStep closest to the Command
|
|
513
|
+
│ │ └─ cmd()
|
|
514
|
+
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
515
|
+
│ ├─ telemetry.onStep returns from the inside out
|
|
516
|
+
│ │
|
|
517
|
+
│ ┌─ recording.onRun returns
|
|
518
|
+
├─ telemetry.onRun returns
|
|
519
|
+
```
|
|
520
|
+
|
|
521
|
+
**Knowing which request a step belongs to.** `onStep` and `onRun` get no context, so a hook that logs per request finds the request the way the rest of your logging does. An `AsyncLocalStorage` scope entered around `runEffect` reaches every hook the run calls, and the recording example uses one to give each run its own recorder:
|
|
522
|
+
|
|
523
|
+
```js
|
|
524
|
+
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
525
|
+
|
|
526
|
+
const requestScope = new AsyncLocalStorage();
|
|
527
|
+
const stepLog = [];
|
|
528
|
+
|
|
529
|
+
// Logs each step with its request and duration, and returns what the step returned.
|
|
530
|
+
const removeStepLog = configureEffect({
|
|
531
|
+
onStep: async (name, type, op, path) => {
|
|
532
|
+
const started = performance.now();
|
|
533
|
+
try {
|
|
534
|
+
return await op();
|
|
535
|
+
} finally {
|
|
536
|
+
const { requestId } = requestScope.getStore() ?? {};
|
|
537
|
+
stepLog.push({ requestId, name, path, ms: performance.now() - started });
|
|
538
|
+
}
|
|
539
|
+
}
|
|
540
|
+
});
|
|
541
|
+
|
|
542
|
+
await requestScope.run({ requestId: 'req_1' }, () => runEffect(registerUserFlow(input)));
|
|
543
|
+
assert.deepEqual(
|
|
544
|
+
stepLog.map((line) => [line.requestId, line.name]),
|
|
545
|
+
[
|
|
546
|
+
['req_1', 'cmdFindUser'],
|
|
547
|
+
['req_1', 'cmdSaveUser']
|
|
548
|
+
]
|
|
549
|
+
);
|
|
550
|
+
removeStepLog();
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
A Command that `onBeforeCommand` vetoes never reaches `onStep`, so log a veto where you make it. In tests, call `configureEffect()` with no arguments in a `beforeEach`, so a test that fails before removing its hooks does not leave them installed for the next.
|
|
425
554
|
|
|
426
|
-
|
|
555
|
+
**Per-call hooks.** `runEffect`'s third argument takes hooks for that call only. They are merged with the configured ones by the same rules: the configured wrappers run outside them, and the configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
427
556
|
|
|
428
|
-
|
|
557
|
+
```
|
|
558
|
+
inherit: true (default) inherit: false
|
|
429
559
|
|
|
430
|
-
|
|
560
|
+
onRun C.onRun (none)
|
|
561
|
+
onBeforeCommand C.onBeforeCommand (none)
|
|
562
|
+
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
431
566
|
|
|
432
567
|
## Passing Runtime Context
|
|
433
568
|
|
|
@@ -452,20 +587,35 @@ app.post('/checkout', async (req, res) => {
|
|
|
452
587
|
});
|
|
453
588
|
```
|
|
454
589
|
|
|
455
|
-
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run
|
|
590
|
+
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run. So a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, belongs in a Command, whose result the trace records each time.
|
|
591
|
+
|
|
592
|
+
Keep the context to plain values such as IDs, flags and settings. Recording copies it into every trace, so a database client or connection pool in it puts its connection string there. One whose properties throw when read, as a lazy client's can, leaves the whole context out, and a replay then needs it passed in. Reach a service from the Command's function, as `findProduct` reaches `db`. To run a flow against fakes or a test database, have the function that builds it take the services, and pass them in the test:
|
|
593
|
+
|
|
594
|
+
```js
|
|
595
|
+
// Takes its catalog, so a test can run it against a fake.
|
|
596
|
+
const makeFindProduct = (catalog) => (productId) => {
|
|
597
|
+
const cmdFindProduct = () => catalog.findProduct(productId);
|
|
598
|
+
return Command(cmdFindProduct, (product) => (product ? Success(product) : Failure('Product not found.')));
|
|
599
|
+
};
|
|
600
|
+
|
|
601
|
+
// The fake answers by the ID it receives, so the test also checks what the Command's function sends.
|
|
602
|
+
const findInFakeCatalog = makeFindProduct({ findProduct: async (id) => (id === 'abc' ? { id } : null) });
|
|
603
|
+
assert.deepEqual(await runEffect(findInFakeCatalog('abc')), Success({ id: 'abc' }));
|
|
604
|
+
assert.deepEqual(await runEffect(findInFakeCatalog('xyz')), Failure('Product not found.'));
|
|
605
|
+
```
|
|
456
606
|
|
|
457
|
-
|
|
607
|
+
A walk or a replay never runs a Command's function, so this is the test that catches one passing the wrong value.
|
|
458
608
|
|
|
459
609
|
## Retrying Transient Failures
|
|
460
610
|
|
|
461
|
-
`Retry` runs part of a flow again when
|
|
611
|
+
`Retry` runs part of a flow again when a Command's function throws. A `Failure` a step returned is not retried: the flow decided to stop, so the `Failure` passes on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency; to reuse settings, share an object:
|
|
462
612
|
|
|
463
613
|
```js
|
|
464
|
-
const flakyNetwork = {
|
|
614
|
+
const flakyNetwork = { retries: 3, delay: 200, backoff: 2 };
|
|
465
615
|
Retry(fetchPrice(sku), flakyNetwork);
|
|
466
616
|
```
|
|
467
617
|
|
|
468
|
-
The
|
|
618
|
+
The settings are a plain object, so a test can check them without running anything:
|
|
469
619
|
|
|
470
620
|
```js
|
|
471
621
|
import { Success, Failure, Command, Retry, runEffect } from 'pure-effect';
|
|
@@ -475,45 +625,43 @@ const fetchWeather = (city) => {
|
|
|
475
625
|
fetch(`https://example-weather-api.com/v1/current?city=${city}`).then((r) => r.json());
|
|
476
626
|
return Retry(
|
|
477
627
|
Command(cmdFetchWeather, (data) => (data.error ? Failure(data.error) : Success(data))),
|
|
478
|
-
{
|
|
628
|
+
{ retries: 3, delay: 200, backoff: 2 } // 200ms, 400ms, 800ms
|
|
479
629
|
);
|
|
480
630
|
};
|
|
481
631
|
|
|
482
632
|
const weatherFn = fetchWeather('Tokyo');
|
|
483
633
|
assert.equal(weatherFn.type, 'Retry');
|
|
484
|
-
assert.equal(weatherFn.options.
|
|
634
|
+
assert.equal(weatherFn.options.retries, 3);
|
|
485
635
|
```
|
|
486
636
|
|
|
487
|
-
**Wrap the Command that fails, not the pipeline.** Each attempt
|
|
637
|
+
**Wrap the Command that fails, not the pipeline.** Each attempt runs everything inside the `Retry` again, including Commands that already succeeded:
|
|
488
638
|
|
|
489
639
|
```js
|
|
490
640
|
// Dangerous: a flaky receipt step charges the customer again on every attempt.
|
|
491
|
-
Retry(effectPipe(chargeCard, sendReceipt)(order), {
|
|
641
|
+
Retry(effectPipe(chargeCard, sendReceipt)(order), { retries: 3 });
|
|
492
642
|
|
|
493
643
|
// Correct: only the step that fails transiently is retried.
|
|
494
|
-
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), {
|
|
644
|
+
effectPipe(chargeCard, (charge) => Retry(sendReceipt(charge), { retries: 3 }))(order);
|
|
495
645
|
```
|
|
496
646
|
|
|
497
647
|
Wrapping a pipeline is safe only when every Command in it is idempotent (safe to run more than once).
|
|
498
648
|
|
|
499
|
-
**Wrapping one Command is not always enough.** A Command's `next` is inside the `Retry` too, so
|
|
649
|
+
**Wrapping one Command is not always enough.** A Command's `next` is inside the `Retry` too, so a `next` that continues the flow puts everything after it inside the `Retry` as well:
|
|
500
650
|
|
|
501
651
|
```js
|
|
502
652
|
// Dangerous: `next` continues the booking, so every later step is inside the Retry.
|
|
503
653
|
Retry(
|
|
504
654
|
Command(cmdHoldSeat, (seat) => chargeCard(trip, seat)),
|
|
505
|
-
{
|
|
655
|
+
{ retries: 2 }
|
|
506
656
|
);
|
|
507
657
|
```
|
|
508
658
|
|
|
509
|
-
If a later Command's function throws, the `Retry`
|
|
510
|
-
|
|
511
|
-
Instead, leave the retried Command with its default `next` and continue in a later pipeline step, outside the `Retry`:
|
|
659
|
+
If a later Command's function throws, the `Retry` holds the seat and charges the card again. With two of these nested, a failure in the last step of a booking charges the card nine times. Instead, give the retried Command its default `next` and continue in a later step, outside the `Retry`:
|
|
512
660
|
|
|
513
661
|
```js
|
|
514
662
|
// Safe: the tree the Retry repeats is that one Command.
|
|
515
663
|
effectPipe(
|
|
516
|
-
() => Retry(Command(cmdHoldSeat), {
|
|
664
|
+
() => Retry(Command(cmdHoldSeat), { retries: 2 }),
|
|
517
665
|
(seat) => chargeCard(trip, seat)
|
|
518
666
|
);
|
|
519
667
|
```
|
|
@@ -521,24 +669,22 @@ effectPipe(
|
|
|
521
669
|
When every attempt fails, `runEffect` returns a `Failure` with this error:
|
|
522
670
|
|
|
523
671
|
```text
|
|
524
|
-
{ retryExhausted: true, lastError: <the last error>,
|
|
672
|
+
{ retryExhausted: true, lastError: <the last error>, retries: 3 }
|
|
525
673
|
```
|
|
526
674
|
|
|
527
|
-
In TypeScript, a `Retry` adds `RetryExhaustedError` to the
|
|
675
|
+
In TypeScript, a `Retry` adds `RetryExhaustedError` to the error union, beside the errors of the steps it wraps. `lastError` is typed `unknown`, since it is whatever the Command's function threw, so check it before using it, for example with `instanceof Error`.
|
|
528
676
|
|
|
529
|
-
**Use `onExhausted` to fall back when every attempt fails.** The fallback runs
|
|
677
|
+
**Use `onExhausted` to fall back when every attempt fails.** The fallback runs in place of the `Failure`. If it succeeds, its value continues down the pipeline; if it fails, its own `Failure` is returned as is. The fallback can be any Effect, such as a Command that reads a cache, or `Success(defaultValue)`:
|
|
530
678
|
|
|
531
679
|
```js
|
|
532
680
|
const fetchPrice = (sku) =>
|
|
533
681
|
Retry(fetchLivePrice(sku), {
|
|
534
|
-
|
|
535
|
-
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError,
|
|
682
|
+
retries: 3,
|
|
683
|
+
onExhausted: (err) => fetchCachedPrice(sku) // err = { retryExhausted, lastError, retries }
|
|
536
684
|
});
|
|
537
685
|
```
|
|
538
686
|
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
Every attempt is recorded, so a replay repeats the same sequence of failures, without the delays.
|
|
687
|
+
Every attempt and the fallback are recorded, so a replay repeats the same failures and the same fallback, without the delays. A fallback never starts in a `Parallel` branch that has already been cancelled. With `onExhausted` set, the fallback's error type takes the place of `RetryExhaustedError` in the TypeScript error union.
|
|
542
688
|
|
|
543
689
|
## Running Effects in Parallel
|
|
544
690
|
|
|
@@ -565,7 +711,7 @@ Parallel(subscriptions.map(billOne), { limit: 5 });
|
|
|
565
711
|
Parallel(subscriptions.map(billOne), { limit: 5, settled: true });
|
|
566
712
|
```
|
|
567
713
|
|
|
568
|
-
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end and `next` receives one `Success` or `Failure` per branch, in order
|
|
714
|
+
Without `settled`, one failing branch cancels the rest and becomes the result of the whole `Parallel`, so one bad record can stop a batch halfway through. With `settled: true`, every branch runs to the end, and `next` receives one `Success` or `Failure` per branch, in order. A failed record is then something you can count instead of the end of the job. In TypeScript, write `settled: true` where you call `Parallel`, or mark a shared options object `as const`. Since `settled` decides what `next` receives, a value the compiler only knows as `true` or `false` is refused.
|
|
569
715
|
|
|
570
716
|
```js
|
|
571
717
|
Parallel(subscriptions.map(billOne), (outcomes) => Success(outcomes.map(summarize)), { limit: 5, settled: true });
|
|
@@ -581,25 +727,27 @@ Parallel(work, (outcomes) => Success(outcomes.map((o) => (o.type === 'Success' ?
|
|
|
581
727
|
|
|
582
728
|
The outcomes are plain `Success` and `Failure` objects. Returning one of them from `next` is the same as returning any other `Failure`: the flow stops, and `Retry` does not run it again.
|
|
583
729
|
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)), and stop a batch by reading the operator's switch in a Command whose `next` returns a `Failure`, rather than by vetoing in `onBeforeCommand`:
|
|
730
|
+
**Collecting some failures and stopping on others.** `settled` decides for every failure at once. Sometimes a batch should note a missing record and go on, but stop everything when its credentials expire. Leave `settled` off and decide in each branch: return what should be collected as a value inside a `Success`, and what should stop the batch as a `Failure`, which cancels the other branches:
|
|
587
731
|
|
|
588
732
|
```js
|
|
589
|
-
//
|
|
590
|
-
const
|
|
591
|
-
const
|
|
592
|
-
Command(
|
|
733
|
+
// A missing user is counted; an expired token stops the batch.
|
|
734
|
+
const fetchProfile = (userId) => {
|
|
735
|
+
const cmdFetchProfile = () => api.getProfile(userId); // resolves to { status, body }
|
|
736
|
+
return Command(cmdFetchProfile, (response) => {
|
|
737
|
+
if (response.status === 401) return Failure('auth_expired');
|
|
738
|
+
if (response.status === 404) return Success({ userId, found: false });
|
|
739
|
+
return Success({ userId, found: true, profile: response.body });
|
|
740
|
+
});
|
|
741
|
+
};
|
|
593
742
|
|
|
594
|
-
|
|
595
|
-
limit: 5,
|
|
596
|
-
settled: true
|
|
597
|
-
});
|
|
598
|
-
```
|
|
743
|
+
const fetchProfiles = (userIds) => Parallel(userIds.map(fetchProfile), { limit: 4 });
|
|
599
744
|
|
|
600
|
-
|
|
745
|
+
const profiles = fetchProfiles(['u1', 'u2']);
|
|
746
|
+
assert.deepEqual(profiles.effects[0].next({ status: 404 }), Success({ userId: 'u1', found: false }));
|
|
747
|
+
assert.deepEqual(profiles.effects[1].next({ status: 401 }), Failure('auth_expired'));
|
|
748
|
+
```
|
|
601
749
|
|
|
602
|
-
|
|
750
|
+
A Command whose function throws, such as on a network error, also stops the batch. Wrap it in `Retry` if the error is likely to pass, or catch it in the function and return it as data if it should be counted; see [Which Errors Are Data](#which-errors-are-data).
|
|
603
751
|
|
|
604
752
|
`limit` caps how many branches run at once; the rest start as others finish. Results and recorded paths stay in array order, so a limit changes only the pacing, and a trace recorded with a limit replays the same without one. A `limit` that is not a positive integer throws an `EffectTypeError` where the `Parallel` is built.
|
|
605
753
|
|
|
@@ -615,9 +763,13 @@ const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${user
|
|
|
615
763
|
|
|
616
764
|
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
617
765
|
|
|
618
|
-
|
|
766
|
+
A function gets the signal only when it declares a parameter without a default value. One written as `(...args)` or `({ signal } = {})`, or wrapped by a helper that hides its parameters, gets none, so once started it runs to the end. A parameter with a default value does not count, so `Command(nanoid)`, whose size is written `size = 21`, works everywhere.
|
|
767
|
+
|
|
768
|
+
That rule catches a function passed by name whose first parameter is optional and has no default. `function listUsers(page) { page ??= 1; return db.users(page); }` gets the signal as its page inside a `Parallel`, and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
769
|
+
|
|
770
|
+
### Undoing what succeeded when a branch fails
|
|
619
771
|
|
|
620
|
-
|
|
772
|
+
Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped. Sometimes one of them did something that has to be undone, such as charging a card for an order whose stock then ran out. The code that called `runEffect` no longer has the charge to refund. Use `settled: true`, so `next` sees every outcome, and have it return the steps that undo whatever succeeded, followed by the failure:
|
|
621
773
|
|
|
622
774
|
```js
|
|
623
775
|
const reserveStock = (order) => {
|
|
@@ -677,6 +829,36 @@ assert.deepEqual(
|
|
|
677
829
|
|
|
678
830
|
A settled `Parallel` cancels nothing, so both steps always run to the end, and the undoing starts once both have finished. The undo steps are part of the flow, so a trace records them and a replay repeats them. An undo step that fails ends the flow with its own error rather than the original one; wrap it in `Retry` if it can fail for a moment.
|
|
679
831
|
|
|
832
|
+
### Errors no branch returned
|
|
833
|
+
|
|
834
|
+
A `Failure` among a settled `Parallel`'s outcomes can also hold an error the branch never returned: what one of its Commands threw, or what an `onBeforeCommand` hook vetoed it with. Neither is in the branch's declared error type. So give the code that reads the outcomes a case for an error it does not recognise, as for a run's result (see [TypeScript](#typescript-typed-errors-and-context)). Otherwise it falls into the last error it expects, and reports a vetoed charge as a declined card.
|
|
835
|
+
|
|
836
|
+
Fewer such errors reach the outcomes when a branch returns what it can as data. Catch an error the flow can handle inside the Command's function and return it (see [Which Errors Are Data](#which-errors-are-data)). To stop a batch, read the operator's switch in a Command whose `next` returns a `Failure`, rather than vetoing in `onBeforeCommand`:
|
|
837
|
+
|
|
838
|
+
```js
|
|
839
|
+
// Read before each charge, so a switch flipped mid-batch stops the charges that have not started.
|
|
840
|
+
const cmdBillingHalted = () => flags.get('billing.halted');
|
|
841
|
+
const billUnlessHalted = (subscription) =>
|
|
842
|
+
Command(cmdBillingHalted, (halted) => (halted ? Failure('halted') : billOne(subscription)));
|
|
843
|
+
|
|
844
|
+
Parallel(subscriptions.map(billUnlessHalted), (outcomes) => Success(outcomes.map(summarize)), {
|
|
845
|
+
limit: 5,
|
|
846
|
+
settled: true
|
|
847
|
+
});
|
|
848
|
+
```
|
|
849
|
+
|
|
850
|
+
The halt is then one of the branch's declared errors, typed like a declined card, and the trace records each read of the switch, so the batch replays. A vetoed Command leaves no entry in the trace, so a replay of a vetoed batch fails at the first veto.
|
|
851
|
+
|
|
852
|
+
Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed flow, and an error thrown by a `next` function or a pure step both still escape a settled `Parallel`, after the other branches are cancelled.
|
|
853
|
+
|
|
854
|
+
### Replaying a cancelled `Parallel`
|
|
855
|
+
|
|
856
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error.
|
|
857
|
+
|
|
858
|
+
If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it. The trace also records how many branches each `Parallel` had, so a replay of one that has gained or lost a branch ends in a `TimeParadox` before any of its branches run.
|
|
859
|
+
|
|
860
|
+
A generator flow in a cancelled branch can run its `catch` and `finally` one step later in a replay than it did in production. A trace marks where a branch stopped only by its Commands. So a branch cancelled during a `Retry`'s wait, just before an `Ask`, or at a `Retry` or `Parallel` with no Command in it replays as though it stopped at its next Command.
|
|
861
|
+
|
|
680
862
|
## Composing Larger Flows
|
|
681
863
|
|
|
682
864
|
`effectPipe` runs steps in a straight line. Branching and joining use the same pieces:
|
|
@@ -712,6 +894,111 @@ When the lookup is written in place rather than reused, the Command's own `next`
|
|
|
712
894
|
|
|
713
895
|
**Pass only what the next steps need.** Avoid one object that collects everything computed so far. When a step's input is all it gets, it cannot depend on a value from far upstream without showing it, stale fields do not linger, and two steps cannot clash over the same key. Data also lives only as long as the flow needs it, which matters for credentials and personal data. In TypeScript, it keeps the pipeline type-checked: if an upstream step changes what it returns, the step that reads the value fails to compile instead of getting `undefined` at runtime.
|
|
714
896
|
|
|
897
|
+
In a generator flow, most of this comes for free. A step that branches into another flow is yielded like any other, as in `yield* step(processOrder(order))`. Values that do not depend on each other are joined with `const [cart, user] = yield* step(Parallel([fetchCart(input.cartId), fetchUser(input.userId)]))`. Values that do are variables, with no step to pass them on together; see [Where a generator helps](#where-a-generator-helps). Keep each decision in a step function that takes only what it needs, as the advice above says for a pipeline's steps.
|
|
898
|
+
|
|
899
|
+
## Writing Loops
|
|
900
|
+
|
|
901
|
+
A loop is written in two parts: a step that gets the list, and a `Parallel` over it, with `limit: 1` so the items run one after another, as in a `for` loop. The `async` version is usually split the same way. Paying a customer's unpaid invoices in order, and stopping at the first that fails:
|
|
902
|
+
|
|
903
|
+
```js
|
|
904
|
+
async function payUnpaidAsync(customerId) {
|
|
905
|
+
const invoices = await billing.findUnpaid(customerId);
|
|
906
|
+
for (const invoice of invoices) {
|
|
907
|
+
const receipt = await payments.charge(invoice);
|
|
908
|
+
await mailer.sendReceipt(receipt);
|
|
909
|
+
}
|
|
910
|
+
}
|
|
911
|
+
|
|
912
|
+
// First part: get the list.
|
|
913
|
+
const findUnpaidInvoices = (customerId) => {
|
|
914
|
+
const cmdFindUnpaid = () => billing.findUnpaid(customerId);
|
|
915
|
+
return Command(cmdFindUnpaid);
|
|
916
|
+
};
|
|
917
|
+
|
|
918
|
+
// Second part: the loop's body, a flow of its own for one invoice.
|
|
919
|
+
const chargeInvoice = (invoice) => {
|
|
920
|
+
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
921
|
+
return Retry(Command(cmdChargeInvoice), { retries: 3 });
|
|
922
|
+
};
|
|
923
|
+
|
|
924
|
+
const sendReceipt = (receipt) => {
|
|
925
|
+
const cmdSendReceipt = () => mailer.sendReceipt(receipt);
|
|
926
|
+
return Command(cmdSendReceipt);
|
|
927
|
+
};
|
|
928
|
+
|
|
929
|
+
const payInvoice = (invoice) => effectPipe(chargeInvoice, sendReceipt)(invoice);
|
|
930
|
+
|
|
931
|
+
// The loop: one body per invoice, run one at a time.
|
|
932
|
+
const payAll = (invoices) => Parallel(invoices.map(payInvoice), { limit: 1 });
|
|
933
|
+
|
|
934
|
+
const payUnpaid = (customerId) => effectPipe(findUnpaidInvoices, payAll)(customerId);
|
|
935
|
+
|
|
936
|
+
// The flow starts by getting the list, and hands what it gets to the loop.
|
|
937
|
+
const flow = payUnpaid('cus_1');
|
|
938
|
+
assert.equal(commandName(flow), 'cmdFindUnpaid');
|
|
939
|
+
assert.equal(flow.next([{ id: 1 }, { id: 2 }]).effects.length, 2);
|
|
940
|
+
|
|
941
|
+
// The body is tested on its own: the Retry holds only the charge as its effect, and its next leads to the receipt.
|
|
942
|
+
const body = payInvoice({ id: 1 });
|
|
943
|
+
assert.equal(commandName(body.effect), 'cmdChargeInvoice');
|
|
944
|
+
assert.equal(commandName(body.next({ id: 'receipt_1' })), 'cmdSendReceipt');
|
|
945
|
+
|
|
946
|
+
// An empty list runs nothing.
|
|
947
|
+
assert.deepEqual(await runEffect(payAll([])), Success([]));
|
|
948
|
+
```
|
|
949
|
+
|
|
950
|
+
Each invoice is paid after the one before it has finished, and the first that fails stops the rest from starting, as in the `async` version. When the payments can overlap, a higher `limit` runs several at a time.
|
|
951
|
+
|
|
952
|
+
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one, since a `Retry` runs everything it wraps again (see [Retrying Transient Failures](#retrying-transient-failures)). Around the whole loop it would charge the invoices already paid. Around the whole body, a receipt that failed to send would charge the card again.
|
|
953
|
+
|
|
954
|
+
The `Parallel`'s `next` receives what each body returned, in order, so a total or a summary of the loop goes there: `Parallel(invoices.map(payInvoice), (receipts) => Success(receipts.length), { limit: 1 })`. To keep going past a failure, as a `try`/`catch` inside the loop would, add `settled: true`: every invoice is tried, and `next` receives one `Success` or `Failure` per invoice. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
955
|
+
|
|
956
|
+
In a generator flow, the same loop can be a `for` loop over the list, with the body's steps inside it:
|
|
957
|
+
|
|
958
|
+
```js
|
|
959
|
+
const payUnpaidGen = effectGen(function* (customerId) {
|
|
960
|
+
const invoices = yield* step(findUnpaidInvoices(customerId));
|
|
961
|
+
for (const invoice of invoices) yield* step(payInvoice(invoice));
|
|
962
|
+
});
|
|
963
|
+
|
|
964
|
+
// Every charge is declined, so both forms give up on the first invoice once its retries run out, with the same Failure.
|
|
965
|
+
const answers = {
|
|
966
|
+
cmdFindUnpaid: { result: [{ id: 1 }, { id: 2 }] },
|
|
967
|
+
cmdChargeInvoice: { error: new Error('declined') }
|
|
968
|
+
};
|
|
969
|
+
const answer = (asked) => answers[asked.name];
|
|
970
|
+
const fromLoop = await replayEffect(payUnpaidGen('cus_1'), answer);
|
|
971
|
+
const fromParallel = await replayEffect(payUnpaid('cus_1'), answer);
|
|
972
|
+
assert.equal(fromLoop.result.error.retryExhausted, true);
|
|
973
|
+
assert.deepEqual(fromLoop.result, fromParallel.result);
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
It stops at the first invoice that fails, as the `Parallel` with `limit: 1` does. Use the `Parallel` instead when payments can overlap, with a higher `limit`. Use it too when the loop has to keep going past a failure, with `settled: true`, since a `catch` in the body cannot recover. The two forms record differently, so a trace recorded from one replays against the other as a `TimeParadox`.
|
|
977
|
+
|
|
978
|
+
Sometimes getting the list is a loop of its own. Paging through an API with a cursor is one: each request needs the cursor from the page before, so the requests cannot be listed in advance. Write that loop as an ordinary `async` function, call it from one Command, and map over the list it returns, as above:
|
|
979
|
+
|
|
980
|
+
```js
|
|
981
|
+
async function fetchAllOrders() {
|
|
982
|
+
const orders = [];
|
|
983
|
+
let cursor = null;
|
|
984
|
+
do {
|
|
985
|
+
const page = await api.listOrders(cursor);
|
|
986
|
+
orders.push(...page.items);
|
|
987
|
+
cursor = page.nextCursor;
|
|
988
|
+
} while (cursor);
|
|
989
|
+
return orders;
|
|
990
|
+
}
|
|
991
|
+
|
|
992
|
+
const listAllOrders = () => {
|
|
993
|
+
const cmdListAllOrders = () => fetchAllOrders();
|
|
994
|
+
return Command(cmdListAllOrders);
|
|
995
|
+
};
|
|
996
|
+
```
|
|
997
|
+
|
|
998
|
+
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test.
|
|
999
|
+
|
|
1000
|
+
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
1001
|
+
|
|
715
1002
|
## Which Errors Are Data
|
|
716
1003
|
|
|
717
1004
|
Pure Effect has no catch, on purpose. A flow's outcomes are of two kinds, and each is written differently.
|
|
@@ -759,7 +1046,7 @@ Retry(
|
|
|
759
1046
|
Command(function cmdFetchPrice() {
|
|
760
1047
|
return pricing.get(sku);
|
|
761
1048
|
}),
|
|
762
|
-
{
|
|
1049
|
+
{ retries: 3 }
|
|
763
1050
|
);
|
|
764
1051
|
```
|
|
765
1052
|
|
|
@@ -771,7 +1058,107 @@ Retry(
|
|
|
771
1058
|
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
772
1059
|
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
773
1060
|
|
|
774
|
-
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is
|
|
1061
|
+
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is a bug, and is thrown as it is rather than returned as a `Failure`. `runEffect` rejects with it, or, from a step before the first Command, the call that builds the flow throws it.
|
|
1062
|
+
|
|
1063
|
+
### A malformed flow
|
|
1064
|
+
|
|
1065
|
+
A flow that is not made of Effects is a bug, so it throws an `EffectTypeError` that says where, rather than returning a `Failure`. The steps before a flow's first Command run as it is built. So a mistake in one of them is thrown by the call that builds the flow, as `registerUserFlow(input)` would throw if its first step returned a plain object:
|
|
1066
|
+
|
|
1067
|
+
```
|
|
1068
|
+
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
1069
|
+
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
1070
|
+
```
|
|
1071
|
+
|
|
1072
|
+
A mistake after the first Command makes `runEffect` reject. That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments where they are built, before their own I/O runs. Each of these throws an `EffectTypeError` there:
|
|
1073
|
+
|
|
1074
|
+
- `Command(db.findUser(email))`, which makes the call while the flow is built
|
|
1075
|
+
- `Command(fn, { name })`, with the metadata in `next`'s place
|
|
1076
|
+
- a `Parallel` or `effectPipe` given the wrong thing
|
|
1077
|
+
|
|
1078
|
+
One built inside a `next` is checked when that `next` runs.
|
|
1079
|
+
|
|
1080
|
+
## Generator Flows in Depth
|
|
1081
|
+
|
|
1082
|
+
What [Writing Flows as Generators](#writing-flows-as-generators) leaves out: when a body runs again, what a `catch` or `finally` in it can do, and the mistakes `effectGen` catches.
|
|
1083
|
+
|
|
1084
|
+
### What runs once, and what runs again
|
|
1085
|
+
|
|
1086
|
+
A run goes through the body once, top to bottom, as `async` code does. Running the flow again starts the body from the top, as calling an `async` function again does. Each `Retry` attempt, each run of one flow value, and each `Parallel` branch holding it gets a fresh build, so every Command in it sees the same values. A test that hands an earlier step a second answer, to walk another branch, runs the body again from the top too, feeding it the answers before that step:
|
|
1087
|
+
|
|
1088
|
+
```js
|
|
1089
|
+
const flow = registerUserFlowGen(input);
|
|
1090
|
+
assert.equal(commandName(flow), 'cmdFindUser');
|
|
1091
|
+
|
|
1092
|
+
// The first answer finds a user, so the flow stops there.
|
|
1093
|
+
assert.deepEqual(flow.next({ id: 1 }), Failure('Email already in use.'));
|
|
1094
|
+
|
|
1095
|
+
// A second answer to the same step runs the body again from the top, and this time it goes on to save.
|
|
1096
|
+
assert.equal(commandName(flow.next(null)), 'cmdSaveUser');
|
|
1097
|
+
```
|
|
1098
|
+
|
|
1099
|
+
So keep the body to its own variables. A body run again for a walk has to ask for the same steps in the same order, and one that asks for a different one stops with an `EffectTypeError` naming both. `Date.now()`, a random ID, or a variable outside the body can change between runs. Get the first two from a Command, as a pipeline does, so a replay gets the recorded value. Pass the third in the flow's input.
|
|
1100
|
+
|
|
1101
|
+
### How a generator flow stops
|
|
1102
|
+
|
|
1103
|
+
`return` ends the flow. A plain value is its `Success`, `return Failure(error)` stops it with that error, and returning a Command or any other Effect runs it, as `return fetch(url)` does in an `async` function. A `throw` in the body is a bug, as one in a pipeline step is, and `runEffect` rejects with it.
|
|
1104
|
+
|
|
1105
|
+
When a step stops the flow, because its Command threw or it returned a `Failure`, the body's `catch` and `finally` blocks run, as they would for an `await` that rejected. The `catch` gets an error saying where the flow stopped, not the original, which stays the flow's `Failure`. So it cannot branch on the original's fields: `e.code` and the like are `undefined`, and `e.name` is `FlowStopped`. It can log, but it cannot recover:
|
|
1106
|
+
|
|
1107
|
+
```js
|
|
1108
|
+
const unreliableDb = {
|
|
1109
|
+
saveUser: async () => {
|
|
1110
|
+
throw new Error('connection reset');
|
|
1111
|
+
}
|
|
1112
|
+
};
|
|
1113
|
+
|
|
1114
|
+
const saveUser = effectGen(function* (user) {
|
|
1115
|
+
try {
|
|
1116
|
+
const cmdSaveUser = () => unreliableDb.saveUser(user);
|
|
1117
|
+
return yield* step(Command(cmdSaveUser));
|
|
1118
|
+
} catch (error) {
|
|
1119
|
+
// Runs when the save throws. It can log, and it has to let the error pass.
|
|
1120
|
+
throw error;
|
|
1121
|
+
}
|
|
1122
|
+
});
|
|
1123
|
+
|
|
1124
|
+
const result = await runEffect(saveUser({ id: 1 }));
|
|
1125
|
+
assert.equal(result.error.message, 'connection reset');
|
|
1126
|
+
```
|
|
1127
|
+
|
|
1128
|
+
A `catch` that returns or carries on, one that throws a different error, and a `finally` that runs another step all stop the run with an `EffectTypeError` that says so:
|
|
1129
|
+
|
|
1130
|
+
```js
|
|
1131
|
+
const saveLater = effectGen(function* (user) {
|
|
1132
|
+
try {
|
|
1133
|
+
const cmdSaveUser = () => unreliableDb.saveUser(user);
|
|
1134
|
+
return yield* step(Command(cmdSaveUser));
|
|
1135
|
+
} catch {
|
|
1136
|
+
return 'saved later';
|
|
1137
|
+
}
|
|
1138
|
+
});
|
|
1139
|
+
|
|
1140
|
+
await assert.rejects(runEffect(saveLater({ id: 1 })), /kept going after Command 'cmdSaveUser' stopped it/);
|
|
1141
|
+
```
|
|
1142
|
+
|
|
1143
|
+
Handle an error the flow expects inside the Command's function, and return it as data for the body to branch on; recover from one that keeps happening with `Retry`'s `onExhausted`. See [Which Errors Are Data](#which-errors-are-data). Cleanup for the whole run goes in a `try`/`finally` around `runEffect`, as it does for a pipeline.
|
|
1144
|
+
|
|
1145
|
+
A body's `finally` does not always run. It does not run when a bug or a `TimeParadox` rejects the run. In a `Parallel` where one branch threw, another branch that stops for its own reason before the rejection reaches it may still run its `catch` and `finally`, depending on timing.
|
|
1146
|
+
|
|
1147
|
+
A step in a body's `finally` is not supported. It runs when the body ends normally, and is refused when a step stopped the body. But if an error is passing through the body when its `finally` reaches a step, and the flow is cancelled before that step starts, the error is lost.
|
|
1148
|
+
|
|
1149
|
+
### Mistakes it catches
|
|
1150
|
+
|
|
1151
|
+
| Written | What happens |
|
|
1152
|
+
| --- | --- |
|
|
1153
|
+
| `yield step(x)`, without the star | an `EffectTypeError`: write `yield*` |
|
|
1154
|
+
| `yield x` | an `EffectTypeError`: write `yield* step(x)` |
|
|
1155
|
+
| `yield* x`, without `step` | the engine's `TypeError`, which on V8 becomes an `EffectTypeError` saying to write `yield* step(x)` |
|
|
1156
|
+
| `const found = step(x)`, then `found.id` | an `EffectTypeError`: `step(...)` was used as a value |
|
|
1157
|
+
| `[...step(x)]` or `for (const v of step(x))` | an `EffectTypeError`: `step(...)` was iterated outside `yield*` |
|
|
1158
|
+
| `step(db.findUser(email))` | an `EffectTypeError`: put the call in a Command |
|
|
1159
|
+
| `effectGen(async function* () {})` | an `EffectTypeError`: write `function*` |
|
|
1160
|
+
|
|
1161
|
+
In TypeScript, most of these do not compile; see [TypeScript](#typescript-typed-errors-and-context).
|
|
775
1162
|
|
|
776
1163
|
## TypeScript: Typed Errors and Context
|
|
777
1164
|
|
|
@@ -794,13 +1181,17 @@ if (result.type === 'Failure') {
|
|
|
794
1181
|
}
|
|
795
1182
|
```
|
|
796
1183
|
|
|
797
|
-
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object
|
|
1184
|
+
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object. A step that cannot fail, such as a Command without a `next`, adds nothing.
|
|
1185
|
+
|
|
1186
|
+
**The union covers only the `Failure`s your steps return.** A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure`, and its error is not in the union. So where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error. The same goes for the outcomes a settled `Parallel` hands `next`. One fallback covers the whole flow: catch inside a Command's function only what the flow can handle, and let the rest throw, which is also what lets `Retry` act on it.
|
|
1187
|
+
|
|
1188
|
+
To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error`. TypeScript refuses `instanceof Error` on an error type made only of strings, or on a flow that declares none.
|
|
798
1189
|
|
|
799
1190
|
To type what a Command's function returns, such as a JSON response, write the function's result type, as in `Command((): Promise<User | null> => fetchJson(url), next)`. `Command<User | null>(…)` also compiles, but with some type arguments given TypeScript infers none of the others, so that Command's errors are typed `unknown`, which hides the rest of the union.
|
|
800
1191
|
|
|
801
1192
|
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
802
1193
|
|
|
803
|
-
`Failure` keeps a literal exactly as written, but `Success` widens it, as TypeScript does for any object: `Success({ status: 'queued' })` holds a `status` of type `string`. A later step that tells values apart by such a field then refuses it, often several steps from where the value was built
|
|
1194
|
+
`Failure` keeps a literal exactly as written, but `Success` widens it, as TypeScript does for any object: `Success({ status: 'queued' })` holds a `status` of type `string`. A later step that tells values apart by such a field then refuses it, often several steps from where the value was built. So give the type there, as in `Success<Queued>({ status: 'queued', jobId })`.
|
|
804
1195
|
|
|
805
1196
|
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
806
1197
|
|
|
@@ -817,36 +1208,54 @@ const findProduct = (productId: string): Effect<Product, 'not_found', AppContext
|
|
|
817
1208
|
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
818
1209
|
```
|
|
819
1210
|
|
|
820
|
-
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object
|
|
1211
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object. It checks one passed as a variable against what the flow reads, as it checks one written in place, so a variable whose type lacks a field the flow reads does not compile. `recordEffect`, `replayEffect` and `timeTravel` check a context the same way, and `replayEffect` given a resolver requires one, since a resolver holds none.
|
|
821
1212
|
|
|
822
|
-
|
|
1213
|
+
### Generator flows
|
|
823
1214
|
|
|
824
|
-
|
|
1215
|
+
`effectGen` infers everything from the body. Each `yield*` is typed by its own step, the error union collects every step's errors and every `Failure` the body returns, and the context collects what every step reads:
|
|
825
1216
|
|
|
826
|
-
|
|
1217
|
+
```ts
|
|
1218
|
+
const registerUser = effectGen(function* (input: { email: string }) {
|
|
1219
|
+
const found = yield* step(findUser(input.email)); // User | null
|
|
1220
|
+
if (found) return Failure('EMAIL_TAKEN');
|
|
1221
|
+
return yield* step(saveUser(input));
|
|
1222
|
+
});
|
|
1223
|
+
// (input: { email: string }) => Effect<User, 'EMAIL_TAKEN' | the errors findUser and saveUser declare>
|
|
1224
|
+
```
|
|
827
1225
|
|
|
828
|
-
|
|
1226
|
+
`yield* x` without `step`, reading a property of a step, and an `async function*` do not compile, and a body that returns an untyped value keeps its steps' errors. Returning a step or a Promise compiles, so that a generic helper returning a value of its type parameter does too, and fails when it runs. `yield step(x)` without the star does, and fails when it runs. Name the context `Ask()` reads as `Ask<AppContext>()`.
|
|
829
1227
|
|
|
830
|
-
|
|
1228
|
+
### Testing a flow in TypeScript
|
|
831
1229
|
|
|
832
|
-
|
|
1230
|
+
A flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure`. So `.cmd` and `.next` compile only after the test checks that the step is a Command. `node:assert`'s `assert(step.type === 'Command')` is such a check, and TypeScript follows it. Jest's and Vitest's `expect` is not, so a walk needs a check at every step, and a small helper makes that one line per step with any test framework:
|
|
833
1231
|
|
|
834
|
-
|
|
1232
|
+
```ts
|
|
1233
|
+
import assert from 'node:assert/strict';
|
|
1234
|
+
import { commandName } from 'pure-effect';
|
|
1235
|
+
import type { Effect } from 'pure-effect';
|
|
835
1236
|
|
|
836
|
-
|
|
1237
|
+
// Fails unless the step is the Command named, and returns it typed as one.
|
|
1238
|
+
function assertCommand<T, E, C>(step: Effect<T, E, C>, name: string) {
|
|
1239
|
+
assert(step.type === 'Command', `expected ${name}, got ${step.type}`);
|
|
1240
|
+
assert.equal(commandName(step), name);
|
|
1241
|
+
return step;
|
|
1242
|
+
}
|
|
837
1243
|
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
-
|
|
843
|
-
|
|
1244
|
+
const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
1245
|
+
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
1246
|
+
```
|
|
1247
|
+
|
|
1248
|
+
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns. So where [Testing Without Mocks](#testing-without-mocks) passes `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
1249
|
+
|
|
1250
|
+
## Why Pure Effect
|
|
1251
|
+
|
|
1252
|
+
**vs. Temporal and durable execution (Restate, Inngest):** The closest relatives, built on the same idea: record what every step returned and replay it. Those engines store the history on a server and resume workflows automatically after a crash. Pure Effect leaves storage and restarts to your application, and there is no infrastructure to run.
|
|
844
1253
|
|
|
845
|
-
**
|
|
1254
|
+
**vs. Effect-TS (and fp-ts):** A full ecosystem with fibers, streaming, schema validation, structured concurrency, and more, with a steep learning curve and its own vocabulary. Pure Effect takes only the idea of describing side effects as data, and does less: testable pipelines, context injection, retry, parallel execution, and replayable traces. If you need fibers or streaming, use Effect-TS.
|
|
1255
|
+
|
|
1256
|
+
**vs. plain async/await with mocks:** A mock can pass every test while behaving differently from the real driver. Here, business logic never does I/O, so there is nothing to mock.
|
|
846
1257
|
|
|
847
|
-
|
|
848
|
-
- 5,000 branches in one `Parallel` ran in ~25 ms.
|
|
849
|
-
- Running a Command through `runEffect` costs about 0.2 microseconds, against 0.06 for a plain `await`.
|
|
1258
|
+
**When to use something else:** If your code does little async I/O, or testing and debugging production are not problems for you, plain async/await is simpler.
|
|
850
1259
|
|
|
851
1260
|
## API Reference
|
|
852
1261
|
|
|
@@ -866,7 +1275,7 @@ Returns `{ type: 'Failure', error }`. Stops the pipeline immediately.
|
|
|
866
1275
|
|
|
867
1276
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
868
1277
|
|
|
869
|
-
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch,
|
|
1278
|
+
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch, one that declares a parameter without a default value gets an `AbortSignal` that fires when a sibling fails; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
870
1279
|
- `next`: receives `cmd`'s result and returns the next Effect; by default `(result) => Success(result)`.
|
|
871
1280
|
- `meta`: metadata passed to `onBeforeCommand`. A string `meta.name` names the Command.
|
|
872
1281
|
|
|
@@ -880,36 +1289,37 @@ Command(function cmdChargeCard() {
|
|
|
880
1289
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
881
1290
|
```
|
|
882
1291
|
|
|
883
|
-
Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1292
|
+
- Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace.
|
|
1293
|
+
- Otherwise naming the function is enough. An arrow assigned to a `const` first, as in `const cmdChargeCard = () => api.charge()`, takes the `const`'s name.
|
|
1294
|
+
- An inline arrow does not. A replay cannot tell two `'anonymous'` Commands apart, so a refactor that swaps them replays without complaint, each handed the other's recorded result.
|
|
884
1295
|
|
|
885
1296
|
#### `commandName(command)`
|
|
886
1297
|
|
|
887
1298
|
Returns a Command's name, chosen in the order above, so a test checks a step by the name its trace would record: `assert.equal(commandName(step), 'cmdFindUser')`. Anything but a Command throws an `EffectTypeError`.
|
|
888
1299
|
|
|
889
|
-
#### `Ask(nextFn)`
|
|
1300
|
+
#### `Ask(nextFn?)`
|
|
890
1301
|
|
|
891
|
-
Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nextFn`.
|
|
1302
|
+
Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nextFn`. With no argument, the flow gets the context itself, as `yield* step(Ask())` reads it.
|
|
892
1303
|
|
|
893
1304
|
#### `Retry(effect, options?)`
|
|
894
1305
|
|
|
895
|
-
Returns `{ type: 'Retry', effect, options, next }`.
|
|
1306
|
+
Returns `{ type: 'Retry', effect, options, next }`. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
896
1307
|
|
|
897
|
-
- `options.
|
|
898
|
-
- `options.delay`:
|
|
899
|
-
- `options.backoff`:
|
|
900
|
-
|
|
901
|
-
An option set to `undefined` keeps its default.
|
|
902
|
-
|
|
903
|
-
- `options.onExhausted(error)`: Runs a fallback Effect when every attempt has failed, receiving `{ retryExhausted, lastError, attempts }`. The fallback's success feeds `next`; its failure propagates unwrapped. Per-use only. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
1308
|
+
- `options.retries`: tries after the first, a positive integer (default `3`), so `2` makes at most three calls. Anything else throws, `0` included: to handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
1309
|
+
- `options.delay`: milliseconds before the first retry, a finite number of 0 or more (default `100`).
|
|
1310
|
+
- `options.backoff`: the multiplier applied to the delay on each attempt, a finite number of 0 or more (default `1`, flat).
|
|
1311
|
+
- `options.onExhausted(error)`: a fallback Effect to run when every attempt has failed, given `{ retryExhausted, lastError, retries }`. Its success feeds `next`; its failure is returned unwrapped. Per use only.
|
|
1312
|
+
- An option set to `undefined` keeps its default.
|
|
904
1313
|
|
|
905
1314
|
#### `Parallel(effects, next?, options?)`
|
|
906
1315
|
|
|
907
|
-
Returns `{ type: 'Parallel', effects, next, options }
|
|
908
|
-
|
|
909
|
-
The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
1316
|
+
Returns `{ type: 'Parallel', effects, next, options }`. Runs the branches at the same time and hands `next` their values in order; by default `(values) => Success(values)`. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
910
1317
|
|
|
1318
|
+
- The first branch to fail cancels the others, and its `Failure` is the result, with `next` skipped. Which branch that is depends on timing, so it is recorded, and a replay returns the same one.
|
|
1319
|
+
- Each branch's Commands get an `AbortSignal` to pass to their I/O.
|
|
1320
|
+
- The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
911
1321
|
- `limit`: the most branches running at once, a positive integer. Results and recorded paths stay in array order, so it changes only the pacing.
|
|
912
|
-
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order
|
|
1322
|
+
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order, so no branch cancels the others or fails the `Parallel`. A bug still rejects the run: a step or `next` that throws, or returns something other than an Effect, is not a branch outcome.
|
|
913
1323
|
|
|
914
1324
|
### Building pipelines
|
|
915
1325
|
|
|
@@ -922,7 +1332,7 @@ Composes steps into a pipeline: each step gets the previous step's `Success` val
|
|
|
922
1332
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
923
1333
|
```
|
|
924
1334
|
|
|
925
|
-
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it
|
|
1335
|
+
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it. Passing the value through every step, as the [Quick Start](#quick-start) does, keeps the whole pipeline checked; see [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context).
|
|
926
1336
|
|
|
927
1337
|
In either language, a Command a step builds and does not return never runs, and any `Failure` it would have produced is lost:
|
|
928
1338
|
|
|
@@ -935,73 +1345,43 @@ In either language, a Command a step builds and does not return never runs, and
|
|
|
935
1345
|
|
|
936
1346
|
Return the Command, and have its `next` return the value the rest of the pipeline needs.
|
|
937
1347
|
|
|
938
|
-
|
|
1348
|
+
#### `effectGen(body)`
|
|
939
1349
|
|
|
940
|
-
|
|
1350
|
+
Returns a flow function, `(input) => Effect`, from a generator function, `function* (input) { ... }`. See [Writing Flows as Generators](#writing-flows-as-generators).
|
|
941
1351
|
|
|
942
|
-
|
|
1352
|
+
- Calling it runs the body up to its first `yield*`, so a `Failure` before the first step comes back at once, as a pipeline's does.
|
|
1353
|
+
- `return value` is `Success(value)`, `return Failure(error)` stops the flow, and a returned Effect runs.
|
|
1354
|
+
- When a step stops the flow, the body's `catch` and `finally` blocks run; a `catch` that recovers or throws another error, or a `finally` that runs a step, rejects the run.
|
|
1355
|
+
- An `async function*`, and a body that yields anything but `yield* step(effect)`, throws an `EffectTypeError`.
|
|
943
1356
|
|
|
944
|
-
|
|
945
|
-
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the `configureEffect` wiring, or used alone with `inherit: false`; see [`configureEffect`](#configureeffectconfigs). `onRun` wraps the run once, retries included. A key it does not read or a hook that is not a function throws a `TypeError`, as in `configureEffect`, and so does a `retry` key, since retry options belong to each `Retry`.
|
|
1357
|
+
#### `step(effect)`
|
|
946
1358
|
|
|
947
|
-
|
|
1359
|
+
Wraps an Effect for `yield*` inside `effectGen`, which gives back its value: `const user = yield* step(findUser(email))`. Anything but an Effect throws an `EffectTypeError`, and so does using the result any other way than `yield*`.
|
|
948
1360
|
|
|
949
|
-
|
|
950
|
-
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
951
|
-
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
952
|
-
```
|
|
1361
|
+
### Running a flow
|
|
953
1362
|
|
|
954
|
-
|
|
1363
|
+
#### `runEffect(effect, context?, callConfig?)`
|
|
955
1364
|
|
|
956
|
-
|
|
1365
|
+
Runs a flow and resolves with its `Success` or `Failure`. See [Which Errors Are Data](#which-errors-are-data).
|
|
957
1366
|
|
|
958
|
-
|
|
1367
|
+
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
1368
|
+
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the configured hooks, or used alone with `inherit: false`; see [Hooks](#hooks).
|
|
1369
|
+
- A throw from a Command's function, or an `onBeforeCommand` veto, ends the run with a `Failure`.
|
|
1370
|
+
- A malformed flow, or a throw from a `next` or a pure step, is a bug. The call that builds the flow throws it before the first Command, and `runEffect` rejects with it after.
|
|
959
1371
|
|
|
960
|
-
|
|
961
|
-
|
|
962
|
-
|
|
963
|
-
remove();
|
|
964
|
-
```
|
|
1372
|
+
#### `configureEffect(...configs)`
|
|
1373
|
+
|
|
1374
|
+
Adds a layer of hooks for the whole process and returns a function that removes that layer only. See [Hooks](#hooks).
|
|
965
1375
|
|
|
966
|
-
-
|
|
1376
|
+
- Several configurations passed to one call form one layer.
|
|
1377
|
+
- `configureEffect()` with no arguments removes every layer, and a call whose arguments are all `undefined` changes nothing.
|
|
1378
|
+
- Throws a `TypeError`, before installing anything, for a configuration that is not an object, a key other than the three below (`retry` included), or a hook that is not a function (`null` included).
|
|
1379
|
+
- `onRun(effect, pipeline, flowName, initialInput)`: wraps the whole run once, retries included. It must `await pipeline()` and return its result. `initialInput` is what the flow was called with when `effectPipe` or `effectGen` built it, and `undefined` otherwise; pass it on to any hook it calls.
|
|
967
1380
|
- `onStep(name, type, op, path)`: wraps each Command, and each `Parallel` with `name` and `type` both `'Parallel'`. It must `await op()` and return its result, and pass `path` on to any hook it calls, since a replay matches steps on it.
|
|
968
1381
|
- For a Command, `op()` returns a promise, even for a synchronous function. Returning a value without calling `op()` answers for the Command, which is how replay works, and throwing without calling it counts as the Command failing.
|
|
969
|
-
- A throw after `op()` succeeded, or `undefined`
|
|
970
|
-
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }
|
|
971
|
-
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it. A vetoed Command leaves no entry in a trace, so a replay of that run fails where it was vetoed;
|
|
972
|
-
|
|
973
|
-
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
974
|
-
|
|
975
|
-
```
|
|
976
|
-
runEffect(flow(input))
|
|
977
|
-
│
|
|
978
|
-
├─ telemetry.onRun first configuration: outermost
|
|
979
|
-
│ └─ recording.onRun last configuration: innermost
|
|
980
|
-
│ │
|
|
981
|
-
│ │ for each Command:
|
|
982
|
-
│ ├─ telemetry.onBeforeCommand interceptors all run, in the order given
|
|
983
|
-
│ ├─ recording.onBeforeCommand
|
|
984
|
-
│ ├─ telemetry.onStep
|
|
985
|
-
│ │ └─ recording.onStep closest to the Command
|
|
986
|
-
│ │ └─ cmd()
|
|
987
|
-
│ │ ┌─ recording.onStep returns a result, or a thrown error, unwinds
|
|
988
|
-
│ ├─ telemetry.onStep returns from the inside out
|
|
989
|
-
│ │
|
|
990
|
-
│ ┌─ recording.onRun returns
|
|
991
|
-
├─ telemetry.onRun returns
|
|
992
|
-
```
|
|
993
|
-
|
|
994
|
-
**Per-call hooks.** The hooks in a `callConfig` are merged over the configured ones by the same rules, the configured wrappers outside and the configured `onBeforeCommand` hooks first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
995
|
-
|
|
996
|
-
```
|
|
997
|
-
inherit: true (default) inherit: false
|
|
998
|
-
|
|
999
|
-
onRun C.onRun (none)
|
|
1000
|
-
onBeforeCommand C.onBeforeCommand (none)
|
|
1001
|
-
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
1002
|
-
```
|
|
1003
|
-
|
|
1004
|
-
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
1382
|
+
- A throw after `op()` succeeded, or `undefined` in place of its value (a forgotten `return` or `await`), is a bug in the hook: the run rejects, and `Retry` does not run the Command again.
|
|
1383
|
+
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, and how many branches there were, as in `{ cancelled: true, branch: 0, branches: 2 }`. A branch that threw adds `threw: true`.
|
|
1384
|
+
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it. A vetoed Command leaves no entry in a trace, so a replay of that run fails where it was vetoed; see [Errors no branch returned](#errors-no-branch-returned).
|
|
1005
1385
|
|
|
1006
1386
|
### Recording and replay
|
|
1007
1387
|
|
|
@@ -1009,8 +1389,8 @@ onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
|
1009
1389
|
|
|
1010
1390
|
Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run, passed as `runEffect(flow, context, { onStep })`. Give each run its own recorder: one shared by a whole application mixes requests into one trace, which a replay refuses. `recordEffect` and the [recording example](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js) both do this for you.
|
|
1011
1391
|
|
|
1012
|
-
- `onStep`: the hook. Recording never changes a run's outcome.
|
|
1013
|
-
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. `path
|
|
1392
|
+
- `onStep`: the hook. Recording never changes a run's outcome.
|
|
1393
|
+
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. A replay matches each on its `path`, the step's position in the flow.
|
|
1014
1394
|
|
|
1015
1395
|
```text
|
|
1016
1396
|
{ command, path, result, durationMs } a Command that returned
|
|
@@ -1021,61 +1401,53 @@ Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run,
|
|
|
1021
1401
|
because 'redact' threw on it or the 'copy' failed
|
|
1022
1402
|
```
|
|
1023
1403
|
|
|
1024
|
-
-
|
|
1025
|
-
- `
|
|
1404
|
+
- An older entry with `error` and no `threw` still counts as a throw.
|
|
1405
|
+
- `toTrace(meta)`: the entries as a trace, with `meta`'s `initialInput`, `context`, `flowName` and `version`, copied into the form JSON stores when it is called. If a Command can change the input or the context, as an ORM save that adds an id does, call it before the run as well, as `recordEffect` does.
|
|
1406
|
+
- `options.redact(value, name, kind)`: returns what the trace stores in place of `value`. `kind` is `'result'`, `'error'`, `'initialInput'` or `'context'`, and `name` is the Command's name, or the kind. See [Keeping secrets out of a trace](#keeping-secrets-out-of-a-trace).
|
|
1026
1407
|
- `options.maxEntries`: the most entries a trace keeps; the rest are counted in `dropped`, and a replay stops at the first step the trace lacks.
|
|
1027
1408
|
- `options.stack`: record stack traces for thrown errors (off by default).
|
|
1409
|
+
- A value `redact` throws on, or one JSON cannot hold, is left out and marked `unrecorded`: on its entry, or for the input and context in the trace's `unrecorded`, as in `{ context: 'copy' }`. A replay stops there with a `ReplayError` rather than hand the flow a stand-in. See [Recording in Production](#recording-in-production) for what a copy keeps.
|
|
1028
1410
|
|
|
1029
|
-
|
|
1030
|
-
|
|
1031
|
-
```js
|
|
1032
|
-
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
1033
|
-
const mask = (value, field) =>
|
|
1034
|
-
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
1035
|
-
|
|
1036
|
-
const redact = (value, name, kind) => {
|
|
1037
|
-
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
1038
|
-
if (kind === 'context') return mask(value, 'authToken');
|
|
1039
|
-
if (kind === 'error') return mask(value, 'attempted');
|
|
1040
|
-
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
1041
|
-
};
|
|
1042
|
-
recorder({ redact });
|
|
1043
|
-
|
|
1044
|
-
assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing found
|
|
1045
|
-
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
1046
|
-
```
|
|
1047
|
-
|
|
1048
|
-
A replay rebuilds the flow from the redacted input and context, so a field a step checks, such as a password, needs a stand-in the check treats the same way, as the recording example in [Recording in Production](#recording-in-production) does, and a context field an `Ask` reads should stay.
|
|
1411
|
+
#### `recordEffect(effect, options?)`
|
|
1049
1412
|
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts. To record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
1413
|
+
Runs a flow for real while recording, returning `{ result, trace }`. The trace holds the flow's input when `effectPipe` or `effectGen` built it, so `timeTravel` can build the flow again. Accepts `recorder` options plus `context` and `version`. For tests and scripts; to record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
1053
1414
|
|
|
1054
1415
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
1055
1416
|
|
|
1056
|
-
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for
|
|
1417
|
+
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for. See [Time-Travel Debugging](#time-travel-debugging).
|
|
1057
1418
|
|
|
1058
|
-
-
|
|
1059
|
-
-
|
|
1060
|
-
- `
|
|
1419
|
+
- A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`.
|
|
1420
|
+
- A malformed trace rejects with a `ReplayError`.
|
|
1421
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver: a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of. See [Replaying from your own storage](#replaying-from-your-own-storage).
|
|
1422
|
+
- Any other answer from a resolver, `null` included, or a throw from it, rejects the replay. With a resolver, `unreached` is absent.
|
|
1423
|
+
- A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`. Answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1424
|
+
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`; a trace whose context could not be recorded refuses to replay without one.
|
|
1425
|
+
- `options.onMissing`: `'throw'` (default) stops at a step the trace lacks; `'execute'` runs its Command for real, so use it only where Commands reach test doubles or only read.
|
|
1426
|
+
- A trace that dropped entries refuses `'execute'`. A flow that changed shape, as when a Command is newly wrapped in `Retry` or moved into a new `Parallel` branch, stops at a `TimeParadox` even under `'execute'`.
|
|
1061
1427
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
1062
1428
|
- `options.hooks` (default `false`): run the replay inside the configured hooks, a configured recorder included, so they see the replayed steps. Off, a replay cannot reach a telemetry backend or a trace sink.
|
|
1063
1429
|
- `options.onResolved(step, outcome)`: observes each replayed step. A throw stops the replay, and `replayEffect` rejects with it; it never changes an outcome.
|
|
1064
|
-
|
|
1065
|
-
A trace whose entries have no `path`, written by hand or recorded before paths existed, is matched by position. That works for a sequential flow; a Command inside a `Parallel` is refused with a `ReplayError`, since the order branches finish in cannot tell them apart.
|
|
1430
|
+
- A trace whose entries have no `path`, written by hand or recorded before paths existed, is matched in order, and a Command inside a `Parallel` is refused with a `ReplayError`.
|
|
1066
1431
|
|
|
1067
1432
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
1068
1433
|
|
|
1069
|
-
Replays a trace and narrates each step with its recorded duration
|
|
1434
|
+
Replays a trace and narrates each step with its recorded duration. Returns `{ result, unreached }`, as `replayEffect` does for a trace.
|
|
1435
|
+
|
|
1436
|
+
- Names any recorded steps that were never reached.
|
|
1437
|
+
- Warns when the trace's `version` differs from `options.version`, and when steps are named `'anonymous'`.
|
|
1438
|
+
- Refuses a trace whose input could not be recorded, since the flow is rebuilt from it.
|
|
1439
|
+
- `options.context` overrides the trace's; `options.log` replaces `console.log`.
|
|
1070
1440
|
|
|
1071
1441
|
## Limitations
|
|
1072
1442
|
|
|
1073
|
-
- **`Retry` repeats everything it wraps, including a Command's `next`.**
|
|
1074
|
-
- **Cancelling a `Parallel` branch cannot stop
|
|
1075
|
-
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.**
|
|
1076
|
-
- **`redact` covers a trace, not a result.** A `Failure` holds the full error,
|
|
1077
|
-
- **Replay checks the path, not the values.**
|
|
1078
|
-
- **A trace keeps data, not objects.**
|
|
1079
|
-
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two
|
|
1080
|
-
- **Configuration belongs to one copy of the library.**
|
|
1081
|
-
- **A Command needs a name.** `meta.name`
|
|
1443
|
+
- **`Retry` repeats everything it wraps, including a Command's `next`.** Commands that already succeeded run again, and a retried Command whose `next` continues the flow repeats the rest of the flow with it. Wrap only the Command that fails, and continue in a later step; see [Retrying Transient Failures](#retrying-transient-failures). A function that catches its own error is never retried; see [Which Errors Are Data](#which-errors-are-data).
|
|
1444
|
+
- **Cancelling a `Parallel` branch cannot stop a call already running.** A cancelled branch starts no new Commands, but a function that ignores its `AbortSignal`, or is never given one, runs to the end. So a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch before it returns; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1445
|
+
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.** So `Command(listUsers)`, where `listUsers(page)` treats `page` as optional, gets the signal as its page inside a `Parallel` and nothing outside one. Wrap such a function, as in `Command(() => listUsers())`; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
1446
|
+
- **`redact` covers a trace, not a result.** A `Failure` holds the full error, and an error can quote what a Command sent, as a database's duplicate-key error quotes the key. Keeping it out of your logs is up to you.
|
|
1447
|
+
- **Replay checks the path, not the values.** Every result comes from the recording, so a replay cannot check a fix that changes what a flow computes. A step removed from the middle of a flow stops the replay at a `TimeParadox`; see [What a replay cannot check](#what-a-replay-cannot-check).
|
|
1448
|
+
- **A trace keeps data, not objects.** A replayed value is what JSON stores: a class instance loses its methods, a `Date` becomes a string, and a thrown error comes back as a plain `Error`, so check `error.name` rather than `instanceof`. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|
|
1449
|
+
- **Replay reproduces what one flow saw, not timing.** A stale read replays exactly. A race between two requests does not, because a trace records one flow. Nor does a race between branches that share state: branches replay with the results they saw, not the order they ran in. A cancelled generator branch can stop one step later in a replay; see [Replaying a cancelled `Parallel`](#replaying-a-cancelled-parallel).
|
|
1450
|
+
- **Configuration belongs to one copy of the library.** Hooks live in module state, so two copies in one process (a dual ESM and CJS setup, two versions in the dependency tree, or a worker thread) do not share them, and nothing warns you.
|
|
1451
|
+
- **A Command needs a name that survives minification.** `meta.name` does. `cmd.name` does not, unless the minifier keeps function names.
|
|
1452
|
+
- **A generator flow's body can run more than once,** and code in it that changes something outside it, such as a counter or a log line, does so again; see [What runs once, and what runs again](#what-runs-once-and-what-runs-again).
|
|
1453
|
+
- **A generator body's `finally` does not always run.** It does not run when a bug or a `TimeParadox` rejects the run. Put cleanup in a `try`/`finally` around `runEffect`; see [How a generator flow stops](#how-a-generator-flow-stops).
|