pure-effect 0.15.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -0
- package/README.md +352 -110
- package/examples/opentelemetry-example.js +173 -0
- package/examples/recording-example.js +170 -0
- package/index.d.ts +1067 -1089
- package/index.js +292 -130
- package/package.json +7 -4
package/README.md
CHANGED
|
@@ -9,27 +9,17 @@
|
|
|
9
9
|
- Inject context without touching function signatures
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
11
|
- OpenTelemetry-ready via lifecycle hooks
|
|
12
|
-
- Zero dependencies,
|
|
13
|
-
- Works in JavaScript and TypeScript (full generics, bundled `.d.ts`)
|
|
12
|
+
- Zero dependencies, under 8 KB minified and gzipped
|
|
13
|
+
- Works in JavaScript, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
16
16
|
|
|
17
|
-
- [Installation](#installation)
|
|
18
|
-
- [
|
|
19
|
-
- [
|
|
20
|
-
- [
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
- [Passing Runtime Context](#passing-runtime-context)
|
|
24
|
-
- [Retrying Transient Failures](#retrying-transient-failures)
|
|
25
|
-
- [Running Effects in Parallel](#running-effects-in-parallel)
|
|
26
|
-
- [Composing Larger Flows](#composing-larger-flows)
|
|
27
|
-
- [Which Errors Are Data](#which-errors-are-data)
|
|
28
|
-
- [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context)
|
|
29
|
-
- [Why Pure Effect](#why-pure-effect)
|
|
30
|
-
- [Load Tests](#load-tests)
|
|
31
|
-
- [API Reference](#api-reference)
|
|
32
|
-
- [Limitations](#limitations)
|
|
17
|
+
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [How It Works](#how-it-works) · [Testing Without Mocks](#testing-without-mocks) · [Coming from async/await](#coming-from-asyncawait)
|
|
18
|
+
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production)
|
|
19
|
+
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Which Errors Are Data](#which-errors-are-data) · [TypeScript](#typescript-typed-errors-and-context)
|
|
20
|
+
- **Reference:** [Why Pure Effect](#why-pure-effect) · [Load Tests](#load-tests) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
|
+
|
|
22
|
+
The five sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: recording, retries, steps that run at the same time, or TypeScript.
|
|
33
23
|
|
|
34
24
|
## Installation
|
|
35
25
|
|
|
@@ -80,6 +70,14 @@ async function registerUser(input) {
|
|
|
80
70
|
}
|
|
81
71
|
```
|
|
82
72
|
|
|
73
|
+
## How It Works
|
|
74
|
+
|
|
75
|
+
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop.
|
|
76
|
+
|
|
77
|
+
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
78
|
+
|
|
79
|
+
Building a flow runs the pure steps right away: `registerUserFlow(badInput)` returns the validation `Failure` synchronously, so the tests below do not need `runEffect`. Only Commands need it.
|
|
80
|
+
|
|
83
81
|
## Testing Without Mocks
|
|
84
82
|
|
|
85
83
|
Pipelines return plain objects, so you can check what the code will do without running it.
|
|
@@ -103,14 +101,16 @@ assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
|
103
101
|
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
104
102
|
```
|
|
105
103
|
|
|
106
|
-
Then test the flow: the right calls, in the right order.
|
|
104
|
+
Then test the flow: the right calls, in the right order. `commandName` gives the name a Command is recorded under:
|
|
107
105
|
|
|
108
106
|
```js
|
|
107
|
+
import { commandName } from 'pure-effect';
|
|
108
|
+
|
|
109
109
|
const step1 = registerUserFlow(input);
|
|
110
|
-
assert.equal(step1
|
|
110
|
+
assert.equal(commandName(step1), 'cmdFindUser');
|
|
111
111
|
|
|
112
112
|
const step2 = step1.next(null); // pretend no user was found
|
|
113
|
-
assert.equal(step2
|
|
113
|
+
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
114
114
|
// The database was never touched.
|
|
115
115
|
```
|
|
116
116
|
|
|
@@ -120,15 +120,185 @@ For example, if the email guard returned `Success(true)` instead of `Success(inp
|
|
|
120
120
|
|
|
121
121
|
A step tested on its own returns a `Failure` without `initialInput`. That is why the assertion above is `Failure('Email already in use.')`, while the validation one is `Failure('Invalid email.', badInput)`: `effectPipe` adds the input when it builds a flow.
|
|
122
122
|
|
|
123
|
+
A walk stops at each Command to be handed its answer. To run the whole flow instead, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command by name. Nothing is called and a retry does not wait, and answering `{ error }` makes that Command throw, which is how a test reaches a `Retry`'s fallback:
|
|
124
|
+
|
|
125
|
+
```js
|
|
126
|
+
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
127
|
+
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
128
|
+
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
129
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
130
|
+
```
|
|
131
|
+
|
|
123
132
|
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them.
|
|
124
133
|
|
|
125
|
-
|
|
134
|
+
**In TypeScript**, a flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure`, so `.cmd` and `.next` compile only after the test checks that the step is a Command. `node:assert`'s `assert(step.type === 'Command')` is such a check, and TypeScript follows it. Jest's and Vitest's `expect` is not, so a walk needs a check at every step, and a small helper makes that one line per step with any test framework:
|
|
126
135
|
|
|
127
|
-
|
|
136
|
+
```ts
|
|
137
|
+
import assert from 'node:assert/strict';
|
|
138
|
+
import { commandName } from 'pure-effect';
|
|
139
|
+
import type { Effect } from 'pure-effect';
|
|
140
|
+
|
|
141
|
+
// Fails unless the step is the Command named, and returns it typed as one.
|
|
142
|
+
function assertCommand<T, E, C>(step: Effect<T, E, C>, name: string) {
|
|
143
|
+
assert(step.type === 'Command', `expected ${name}, got ${step.type}`);
|
|
144
|
+
assert.equal(commandName(step), name);
|
|
145
|
+
return step;
|
|
146
|
+
}
|
|
128
147
|
|
|
129
|
-
|
|
148
|
+
const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
149
|
+
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns, so where the tests above pass `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
153
|
+
|
|
154
|
+
## Coming from async/await
|
|
155
|
+
|
|
156
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command that says which call to make and what to do with the answer. That is what lets a test hand a step an answer instead of mocking the database, and a replay hand it the recorded one. Each thing you would write with `async`/`await` has an equivalent:
|
|
157
|
+
|
|
158
|
+
| With `async`/`await` | In a flow |
|
|
159
|
+
| --- | --- |
|
|
160
|
+
| `const user = await db.findUser(email)` | a Command: `Command(() => db.findUser(email), next)` |
|
|
161
|
+
| the code after the `await`, which uses `user` | the Command's `next`, which receives `user` |
|
|
162
|
+
| an `if` on `user` after the `await` | a branch in `next` |
|
|
163
|
+
| `return value` | `Success(value)` |
|
|
164
|
+
| `throw` to stop | `Failure(error)` |
|
|
165
|
+
| a call inside `try`/`catch` | catch inside the Command's function and return a value for `next` to branch on; see [Which Errors Are Data](#which-errors-are-data) |
|
|
166
|
+
| a loop that retries a call | `Retry(Command(...), options)` |
|
|
167
|
+
| `await Promise.all([...])` | `Parallel([...])` |
|
|
168
|
+
| a `for` loop with an `await` inside | a step that gets the list, then `Parallel(list.map(...), { limit: 1 })` |
|
|
169
|
+
| a `try`/`catch` inside a loop, to keep going past a failure | the same `Parallel`, with `settled: true` |
|
|
170
|
+
| a later call that needs two earlier results | a named step that passes both on; see [Composing Larger Flows](#composing-larger-flows) |
|
|
171
|
+
| a value from the request, such as the tenant | `Ask`; see [Passing Runtime Context](#passing-runtime-context) |
|
|
172
|
+
| `Date.now()` or a random ID | a Command, so a replay gets the recorded value |
|
|
173
|
+
|
|
174
|
+
One rule is behind every row. In `async` code, an `if` or a loop can have an `await` inside it. In a flow it can't: an `if` goes in a step or a Command's `next`, where the answer has already arrived, and a loop that makes calls becomes a `Parallel` or a `Retry`. So each decision lives in a small function of its own, which a test can call directly.
|
|
175
|
+
|
|
176
|
+
The Quick Start's `registerUserFlow` is this function, translated row by row: each `await` became a Command, the code after it became that Command's `next`, and each `throw` became a `Failure`.
|
|
177
|
+
|
|
178
|
+
```js
|
|
179
|
+
async function registerUserAsync(input) {
|
|
180
|
+
if (!input.email.includes('@')) throw new Error('Invalid email.');
|
|
181
|
+
const found = await db.findUser(input.email);
|
|
182
|
+
if (found) throw new Error('Email already in use.');
|
|
183
|
+
return db.saveUser(input);
|
|
184
|
+
}
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
The difference shows most in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace the database's methods, run the function, and put the methods back. Here it is with Node's built-in mocks; Jest's `spyOn` works the same way:
|
|
188
|
+
|
|
189
|
+
```js
|
|
190
|
+
import { mock } from 'node:test';
|
|
191
|
+
|
|
192
|
+
const input = { email: 'test@test.com', password: 'password123' };
|
|
193
|
+
|
|
194
|
+
mock.method(db, 'findUser', async () => ({ id: 1 }));
|
|
195
|
+
const saveUser = mock.method(db, 'saveUser', async (user) => user);
|
|
196
|
+
|
|
197
|
+
await assert.rejects(registerUserAsync(input), { message: 'Email already in use.' });
|
|
198
|
+
assert.equal(saveUser.mock.callCount(), 0);
|
|
199
|
+
mock.restoreAll();
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
The flow's test of the same branch hands one step the answer, with nothing to replace, wait for, or put back:
|
|
203
|
+
|
|
204
|
+
```js
|
|
205
|
+
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
This is what the small functions are for. Each step is a place a test can start, so a branch is tested by calling the step it lives in. An `async` function has one way in, so a test of any branch runs everything before it, with every call on the way replaced. The mocks can check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
209
|
+
|
|
210
|
+
You can switch one function at a time. The function keeps its name and parameters, and its body runs the flow and turns a `Failure` back into a throw, so the code that calls it does not change. The mocked test above passes against this version too, so it checks the switch before the flow's own tests replace it:
|
|
211
|
+
|
|
212
|
+
```js
|
|
213
|
+
// Callers still await registerUserAsync and catch what it throws.
|
|
214
|
+
async function registerUserAsync(input) {
|
|
215
|
+
const result = await runEffect(registerUserFlow(input));
|
|
216
|
+
// A step's Failure holds the error as written, here a string; a Command's function threw an Error.
|
|
217
|
+
if (result.type === 'Failure') throw result.error instanceof Error ? result.error : new Error(result.error);
|
|
218
|
+
return result.value;
|
|
219
|
+
}
|
|
220
|
+
|
|
221
|
+
await assert.rejects(registerUserAsync({ email: 'bad-email', password: '123' }), { message: 'Invalid email.' });
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
A loop is written in two parts: a step that gets the list, and a `Parallel` over it. `Parallel` runs a list of flows, at the same time unless told otherwise, and `limit: 1` makes it run them one after another, like a `for` loop. The `async` version is usually split the same way. Paying a customer's unpaid invoices in order, and stopping at the first that fails:
|
|
225
|
+
|
|
226
|
+
```js
|
|
227
|
+
async function payUnpaidAsync(customerId) {
|
|
228
|
+
const invoices = await billing.findUnpaid(customerId);
|
|
229
|
+
for (const invoice of invoices) {
|
|
230
|
+
const receipt = await payments.charge(invoice);
|
|
231
|
+
await mailer.sendReceipt(receipt);
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
// First part: get the list.
|
|
236
|
+
const findUnpaidInvoices = (customerId) => {
|
|
237
|
+
const cmdFindUnpaid = () => billing.findUnpaid(customerId);
|
|
238
|
+
return Command(cmdFindUnpaid);
|
|
239
|
+
};
|
|
240
|
+
|
|
241
|
+
// Second part: the loop's body, a flow of its own for one invoice.
|
|
242
|
+
const chargeInvoice = (invoice) => {
|
|
243
|
+
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
244
|
+
return Retry(Command(cmdChargeInvoice), { attempts: 3 });
|
|
245
|
+
};
|
|
246
|
+
|
|
247
|
+
const sendReceipt = (receipt) => {
|
|
248
|
+
const cmdSendReceipt = () => mailer.sendReceipt(receipt);
|
|
249
|
+
return Command(cmdSendReceipt);
|
|
250
|
+
};
|
|
251
|
+
|
|
252
|
+
const payInvoice = (invoice) => effectPipe(chargeInvoice, sendReceipt)(invoice);
|
|
253
|
+
|
|
254
|
+
// The loop: one body per invoice, run one at a time.
|
|
255
|
+
const payAll = (invoices) => Parallel(invoices.map(payInvoice), { limit: 1 });
|
|
130
256
|
|
|
131
|
-
|
|
257
|
+
const payUnpaid = (customerId) => effectPipe(findUnpaidInvoices, payAll)(customerId);
|
|
258
|
+
|
|
259
|
+
// The flow starts by getting the list, and hands what it gets to the loop.
|
|
260
|
+
const flow = payUnpaid('cus_1');
|
|
261
|
+
assert.equal(commandName(flow), 'cmdFindUnpaid');
|
|
262
|
+
assert.equal(flow.next([{ id: 1 }, { id: 2 }]).effects.length, 2);
|
|
263
|
+
|
|
264
|
+
// The body is tested on its own: the Retry holds only the charge, and the receipt comes after it.
|
|
265
|
+
const body = payInvoice({ id: 1 });
|
|
266
|
+
assert.equal(commandName(body.effect), 'cmdChargeInvoice');
|
|
267
|
+
assert.equal(commandName(body.next({ id: 'receipt_1' })), 'cmdSendReceipt');
|
|
268
|
+
|
|
269
|
+
// An empty list runs nothing.
|
|
270
|
+
assert.deepEqual(await runEffect(payAll([])), Success([]));
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
Each invoice is paid after the one before it has finished, and the first that fails stops the rest from starting, as in the `async` version. When the payments can overlap, a higher `limit` runs several at a time.
|
|
274
|
+
|
|
275
|
+
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one. Around the whole loop, a `Retry` would pay every invoice again after one failed, charging the ones already paid; around the whole body, a receipt that failed to send would charge the card again. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
276
|
+
|
|
277
|
+
The `Parallel`'s `next` receives what each body returned, in order, so a total or a summary of the loop goes there: `Parallel(invoices.map(payInvoice), (receipts) => Success(receipts.length), { limit: 1 })`. To keep going past a failure, as a `try`/`catch` inside the loop would, add `settled: true`: every invoice is tried, and `next` receives one `Success` or `Failure` per invoice. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
278
|
+
|
|
279
|
+
Sometimes getting the list is a loop of its own. Paging through an API with a cursor is one: each request needs the cursor from the page before, so the requests cannot be listed in advance. Write that loop as an ordinary `async` function, call it from one Command, and map over the list it returns, as above:
|
|
280
|
+
|
|
281
|
+
```js
|
|
282
|
+
async function fetchAllOrders() {
|
|
283
|
+
const orders = [];
|
|
284
|
+
let cursor = null;
|
|
285
|
+
do {
|
|
286
|
+
const page = await api.listOrders(cursor);
|
|
287
|
+
orders.push(...page.items);
|
|
288
|
+
cursor = page.nextCursor;
|
|
289
|
+
} while (cursor);
|
|
290
|
+
return orders;
|
|
291
|
+
}
|
|
292
|
+
|
|
293
|
+
const listAllOrders = () => {
|
|
294
|
+
const cmdListAllOrders = () => fetchAllOrders();
|
|
295
|
+
return Command(cmdListAllOrders);
|
|
296
|
+
};
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test.
|
|
300
|
+
|
|
301
|
+
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
132
302
|
|
|
133
303
|
## Time-Travel Debugging
|
|
134
304
|
|
|
@@ -183,9 +353,9 @@ it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
|
183
353
|
});
|
|
184
354
|
```
|
|
185
355
|
|
|
186
|
-
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay
|
|
356
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`. If the error handling changes, the assertion fails.
|
|
187
357
|
|
|
188
|
-
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge)
|
|
358
|
+
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
189
359
|
|
|
190
360
|
```js
|
|
191
361
|
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
@@ -204,6 +374,8 @@ This works when the removed step is the last one the flow reaches. Remove a step
|
|
|
204
374
|
|
|
205
375
|
**Replay checks the path, not the values.** It shows that the flow asks for the same Commands in the same order and handles the same answers. It cannot check what the flow computes, because every Command's result comes from the recording, not from the new code. If you fix how a refund amount is calculated and replay the bad run, the charge step gets the old amount from the recording: the replay reports `Success`, returns the value from before the fix, and flags nothing. Test a change to a value against the pure step that computes it.
|
|
206
376
|
|
|
377
|
+
**A replay cannot check a fix inside a Command's function either.** The function never runs, so the replay hands the flow what it returned or threw before the fix: catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test. And when a Command's result changes shape, rename the Command, since a trace matches it by name: under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
378
|
+
|
|
207
379
|
**A trace records what each Command returned, not the arguments it was called with.** Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds the arguments that Command ran with. You can inspect them in a debugger, but you cannot assert on them. Storing arguments would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
208
380
|
|
|
209
381
|
**Put anything that varies between runs inside a Command.** For replay, the current time or a random ID counts as I/O. Wrapped in a Command, it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay and drifts from the trace.
|
|
@@ -229,9 +401,10 @@ await replayEffect(checkoutFlow(input), resolve);
|
|
|
229
401
|
|
|
230
402
|
## Recording in Production
|
|
231
403
|
|
|
232
|
-
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup
|
|
404
|
+
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup. The two files imported below ship in the package's `examples` folder, `node_modules/pure-effect/examples`, and in the repository's [examples folder](https://github.com/aycangulez/pure-effect/blob/main/examples); copy them into your project.
|
|
233
405
|
|
|
234
406
|
```js
|
|
407
|
+
import { randomUUID } from 'node:crypto';
|
|
235
408
|
import { configureEffect } from 'pure-effect';
|
|
236
409
|
import { recordingHooks } from './recording-example.js';
|
|
237
410
|
import { telemetryHooks } from './opentelemetry-example.js';
|
|
@@ -239,15 +412,17 @@ import { telemetryHooks } from './opentelemetry-example.js';
|
|
|
239
412
|
configureEffect(
|
|
240
413
|
telemetryHooks(),
|
|
241
414
|
recordingHooks({
|
|
242
|
-
sink: (trace) => putObject(`traces/${trace.flowName}/${
|
|
243
|
-
|
|
415
|
+
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
416
|
+
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
417
|
+
redact: (value, name, kind) =>
|
|
418
|
+
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
244
419
|
maxEntries: 500,
|
|
245
420
|
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
246
421
|
})
|
|
247
422
|
);
|
|
248
423
|
```
|
|
249
424
|
|
|
250
|
-
By default, successful runs are held in memory and then discarded. A run that rejects because your own code threw, such as a `TypeError` after an API changed the shape of its response, is offered to `keep` as a `Failure` carrying the thrown error, so by default its trace is kept, and the run still rejects. Replaying that trace reproduces the throw. If `keep` or the sink throws, for example because `JSON.stringify` meets a value it cannot serialize, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`. `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. `maxEntries` caps the length of a trace and reports the overflow as `dropped`. The sink can write a trace to S3 or a database column as JSON, as long as your Commands return plain data.
|
|
425
|
+
By default, successful runs are held in memory and then discarded. A run that rejects because your own code threw, such as a `TypeError` after an API changed the shape of its response, is offered to `keep` as a `Failure` carrying the thrown error, so by default its trace is kept, and the run still rejects. Replaying that trace reproduces the throw. If `keep` or the sink throws, for example because `JSON.stringify` meets a value it cannot serialize, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`. `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. `maxEntries` caps the length of a trace and reports the overflow as `dropped`; a capped trace replays only up to the first step it lacks, so the example warns through `onWarning` the first time it keeps one for a flow. It warns the same way about a flow whose steps include any named `'anonymous'`. The sink can write a trace to S3 or a database column as JSON, as long as your Commands return plain data.
|
|
251
426
|
|
|
252
427
|
The example reads the input from the flow itself, where `effectPipe` stores it, so build the outermost flow with `effectPipe`. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one. The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
253
428
|
|
|
@@ -272,11 +447,13 @@ app.post('/checkout', async (req, res) => {
|
|
|
272
447
|
const result = await runEffect(checkoutFlow(req.body.productId), { tenant: req.tenant });
|
|
273
448
|
if (result.type === 'Success') return res.json(result.value);
|
|
274
449
|
// A Failure also carries the flow's input, so send the client only an error meant for it.
|
|
275
|
-
|
|
450
|
+
// Answer the errors the flow returns by name, and anything else, such as a database outage, as a server error.
|
|
451
|
+
if (result.error === 'Product not found.') return res.status(400).json({ error: result.error });
|
|
452
|
+
res.status(500).json({ error: 'Checkout failed.' });
|
|
276
453
|
});
|
|
277
454
|
```
|
|
278
455
|
|
|
279
|
-
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw.
|
|
456
|
+
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run, so read a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, in a Command, whose result the trace records each time.
|
|
280
457
|
|
|
281
458
|
## Retrying Transient Failures
|
|
282
459
|
|
|
@@ -409,7 +586,7 @@ Bugs in the flow are not collected. An `EffectTypeError`, thrown for a malformed
|
|
|
409
586
|
|
|
410
587
|
`limit` caps how many branches run at once; the rest start as others finish. Results and recorded paths stay in array order, so a limit changes only the pacing, and a trace recorded with a limit replays the same without one. A `limit` that is not a positive integer throws a `TypeError`.
|
|
411
588
|
|
|
412
|
-
**Cancelling is a request, not a guarantee.** A cancelled branch starts no new Commands: a three-step branch whose first step is running when a sibling fails finishes that step and stops. To stop the running step itself, its function has to
|
|
589
|
+
**Cancelling is a request, not a guarantee.** A cancelled branch starts no new Commands: a three-step branch whose first step is running when a sibling fails finishes that step and stops. To stop the running step itself, its function has to take the `AbortSignal` as a parameter, written without a default value, and pass it on to the I/O:
|
|
413
590
|
|
|
414
591
|
```js
|
|
415
592
|
// Cancellable: the request is aborted the moment a sibling branch fails.
|
|
@@ -421,7 +598,67 @@ const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${user
|
|
|
421
598
|
|
|
422
599
|
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
423
600
|
|
|
424
|
-
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay
|
|
601
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it.
|
|
602
|
+
|
|
603
|
+
**Undoing what succeeded when a branch fails.** Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped. When one of them did something that has to be undone, such as charging a card for an order whose stock then ran out, the code that called `runEffect` no longer has the charge to refund. Use `settled: true`, so `next` sees every outcome, and have it return the steps that undo whatever succeeded, followed by the failure:
|
|
604
|
+
|
|
605
|
+
```js
|
|
606
|
+
const reserveStock = (order) => {
|
|
607
|
+
const cmdReserveStock = () => inventory.reserve(order.items);
|
|
608
|
+
return Command(cmdReserveStock, (r) => (r.ok ? Success(r.reservationId) : Failure('out_of_stock')));
|
|
609
|
+
};
|
|
610
|
+
|
|
611
|
+
const chargeOrder = (order) => {
|
|
612
|
+
const cmdChargeOrder = () => payments.charge(order.customerId, order.total);
|
|
613
|
+
return Command(cmdChargeOrder, (c) => (c.ok ? Success(c.chargeId) : Failure('card_declined')));
|
|
614
|
+
};
|
|
615
|
+
|
|
616
|
+
const releaseStock = (reservationId) => {
|
|
617
|
+
const cmdReleaseStock = () => inventory.release(reservationId);
|
|
618
|
+
return Command(cmdReleaseStock);
|
|
619
|
+
};
|
|
620
|
+
|
|
621
|
+
const refundCharge = (chargeId) => {
|
|
622
|
+
const cmdRefundCharge = () => payments.refund(chargeId);
|
|
623
|
+
return Command(cmdRefundCharge);
|
|
624
|
+
};
|
|
625
|
+
|
|
626
|
+
// Passes both results on, or undoes whichever step succeeded and then fails.
|
|
627
|
+
const reserveAndCharge = (order) =>
|
|
628
|
+
Parallel(
|
|
629
|
+
[reserveStock(order), chargeOrder(order)],
|
|
630
|
+
([reservation, charge]) => {
|
|
631
|
+
if (reservation.type === 'Failure') {
|
|
632
|
+
const undo = charge.type === 'Success' ? [refundCharge(charge.value)] : [];
|
|
633
|
+
return Parallel(undo, () => Failure(reservation.error));
|
|
634
|
+
}
|
|
635
|
+
if (charge.type === 'Failure') {
|
|
636
|
+
return Parallel([releaseStock(reservation.value)], () => Failure(charge.error));
|
|
637
|
+
}
|
|
638
|
+
return Success({ order, reservationId: reservation.value, chargeId: charge.value });
|
|
639
|
+
},
|
|
640
|
+
{ settled: true }
|
|
641
|
+
);
|
|
642
|
+
```
|
|
643
|
+
|
|
644
|
+
`next` is a plain function, so each case is tested by handing it the outcomes, with no I/O:
|
|
645
|
+
|
|
646
|
+
```js
|
|
647
|
+
const order = { id: 'order_1', customerId: 'cus_1', items: [], total: 120 };
|
|
648
|
+
|
|
649
|
+
// The card was charged, then the stock ran out: the flow refunds the charge and fails.
|
|
650
|
+
const undoing = reserveAndCharge(order).next([Failure('out_of_stock'), Success('ch_1')]);
|
|
651
|
+
assert.equal(commandName(undoing.effects[0]), 'cmdRefundCharge');
|
|
652
|
+
assert.deepEqual(undoing.next([null]), Failure('out_of_stock'));
|
|
653
|
+
|
|
654
|
+
// Both succeeded, so there is nothing to undo.
|
|
655
|
+
assert.deepEqual(
|
|
656
|
+
reserveAndCharge(order).next([Success('res_1'), Success('ch_1')]),
|
|
657
|
+
Success({ order, reservationId: 'res_1', chargeId: 'ch_1' })
|
|
658
|
+
);
|
|
659
|
+
```
|
|
660
|
+
|
|
661
|
+
A settled `Parallel` cancels nothing, so both steps always run to the end, and the undoing starts once both have finished. The undo steps are part of the flow, so a trace records them and a replay repeats them. An undo step that fails ends the flow with its own error rather than the original one; wrap it in `Retry` if it can fail for a moment.
|
|
425
662
|
|
|
426
663
|
## Composing Larger Flows
|
|
427
664
|
|
|
@@ -444,17 +681,18 @@ const loadCheckout = effectPipe(
|
|
|
444
681
|
);
|
|
445
682
|
```
|
|
446
683
|
|
|
447
|
-
**Join values that depend on each other
|
|
684
|
+
**Join values that depend on each other in a named step.** When step B needs step A's result and step C needs both, write a small step that runs B and passes both forward in one value. Give it a name, and the pipeline reads as a list of steps:
|
|
448
685
|
|
|
449
686
|
```js
|
|
687
|
+
// Fetches the order's customer and passes on both.
|
|
688
|
+
const withCustomer = (order) => effectPipe(fetchCustomer, (customer) => Success({ order, customer }))(order.customerId);
|
|
689
|
+
|
|
450
690
|
const applyLoyaltyDiscount = (orderId) =>
|
|
451
|
-
effectPipe(
|
|
452
|
-
fetchOrder,
|
|
453
|
-
(order) => effectPipe(fetchCustomer, (customer) => Success({ order, customer }))(order.customerId),
|
|
454
|
-
({ order, customer }) => Success(discountedTotal(order, customer))
|
|
455
|
-
)(orderId);
|
|
691
|
+
effectPipe(fetchOrder, withCustomer, ({ order, customer }) => Success(discountedTotal(order, customer)))(orderId);
|
|
456
692
|
```
|
|
457
693
|
|
|
694
|
+
When the lookup is written in place rather than reused, the Command's own `next` does the join, with no pipeline inside the step: `Command(() => db.findCustomer(order.customerId), (customer) => Success({ order, customer }))`.
|
|
695
|
+
|
|
458
696
|
**Pass only what the next steps need.** Avoid one object that collects everything computed so far. When a step's input is all it gets, it cannot depend on a value from far upstream without showing it, stale fields do not linger, and two steps cannot clash over the same key. Data also lives only as long as the flow needs it, which matters for credentials and personal data. In TypeScript, it keeps the pipeline type-checked: if an upstream step changes what it returns, the step that reads the value fails to compile instead of getting `undefined` at runtime.
|
|
459
697
|
|
|
460
698
|
## Which Errors Are Data
|
|
@@ -515,13 +753,13 @@ Retry(
|
|
|
515
753
|
| a step returned `Failure(...)`, or an `onBeforeCommand` hook threw | an abort | nothing; it propagates at once, unwrapped | nothing |
|
|
516
754
|
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
517
755
|
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
518
|
-
| an `onStep` hook threw after the Command's function succeeded | a bug | nothing; it is thrown, not returned | nothing |
|
|
519
|
-
| an `onStep` hook returned `undefined` after the Command's function returned a value | a bug | nothing; it is thrown, not returned | nothing |
|
|
520
756
|
|
|
521
757
|
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is treated as a bug: `runEffect` rejects with the thrown error instead of returning a `Failure`.
|
|
522
758
|
|
|
523
759
|
## TypeScript: Typed Errors and Context
|
|
524
760
|
|
|
761
|
+
The bundled declarations need TypeScript 5.1 or later.
|
|
762
|
+
|
|
525
763
|
### Error union across pipeline steps
|
|
526
764
|
|
|
527
765
|
Each step in `effectPipe` carries its own error type. The compiler collects them into a union automatically:
|
|
@@ -539,6 +777,10 @@ if (result.type === 'Failure') {
|
|
|
539
777
|
}
|
|
540
778
|
```
|
|
541
779
|
|
|
780
|
+
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object, and a step that cannot fail, such as a Command without a `next`, adds nothing. The union covers the `Failure`s your steps return. A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure` the union does not name, so where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error. One fallback covers the whole flow: catch inside a Command's function only what the flow can handle, and let the rest throw, which is also what lets `Retry` act on it. To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error`, since TypeScript refuses `instanceof Error` on an error type made only of strings, or on a flow that declares none.
|
|
781
|
+
|
|
782
|
+
To type what a Command's function returns, such as a JSON response, write the function's result type, as in `Command((): Promise<User | null> => fetchJson(url), next)`. `Command<User | null>(…)` also compiles, but with some type arguments given TypeScript infers none of the others, so that Command's errors are typed `unknown`, which hides the rest of the union.
|
|
783
|
+
|
|
542
784
|
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
543
785
|
|
|
544
786
|
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
@@ -556,7 +798,7 @@ const findProduct = (productId: string): Effect<Product, 'not_found', AppContext
|
|
|
556
798
|
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
557
799
|
```
|
|
558
800
|
|
|
559
|
-
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object.
|
|
801
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object.
|
|
560
802
|
|
|
561
803
|
## Why Pure Effect
|
|
562
804
|
|
|
@@ -589,6 +831,8 @@ Timings are from Node 22 on a MacBook Pro with M4 Pro CPU:
|
|
|
589
831
|
|
|
590
832
|
## API Reference
|
|
591
833
|
|
|
834
|
+
Every `options` argument below throws for a name it does not read, so a misspelt option fails instead of running with its default.
|
|
835
|
+
|
|
592
836
|
### Building blocks
|
|
593
837
|
|
|
594
838
|
#### `Success(value)`
|
|
@@ -603,11 +847,11 @@ Returns `{ type: 'Failure', error, initialInput }`. Stops the pipeline immediate
|
|
|
603
847
|
|
|
604
848
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
605
849
|
|
|
606
|
-
- `cmd`:
|
|
607
|
-
- `next`:
|
|
608
|
-
- `meta`:
|
|
850
|
+
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch, a function that takes a parameter gets an `AbortSignal` in it that fires when a sibling fails; one with no parameters, or only parameters with default values, gets none, and so does every function outside a `Parallel` (see [Limitations](#limitations)).
|
|
851
|
+
- `next`: receives `cmd`'s result and returns the next Effect; by default `(result) => Success(result)`.
|
|
852
|
+
- `meta`: metadata passed to `onBeforeCommand`. A string `meta.name` names the Command.
|
|
609
853
|
|
|
610
|
-
**Every Command needs a name
|
|
854
|
+
**Every Command needs a name**, which tests, traces, replay matching and telemetry all use. It is `meta.name`, else the function's name, else `'anonymous'`:
|
|
611
855
|
|
|
612
856
|
```js
|
|
613
857
|
Command(cmdFn, next, { name: 'chargeCard' }); // 1. meta.name, independent of how cmdFn was written
|
|
@@ -617,7 +861,11 @@ Command(function cmdChargeCard() {
|
|
|
617
861
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
618
862
|
```
|
|
619
863
|
|
|
620
|
-
Use `meta.name` in code that gets minified, since
|
|
864
|
+
Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace. Otherwise naming the function is enough, as the examples do: an arrow assigned to a `const` first, as in `const cmdChargeCard = () => api.charge()`, takes the `const`'s name. An inline arrow does not, and a replay cannot tell two `'anonymous'` Commands apart, so a refactor that swaps them replays without complaint, each handed the other's recorded result.
|
|
865
|
+
|
|
866
|
+
#### `commandName(command)`
|
|
867
|
+
|
|
868
|
+
Returns a Command's name, chosen in the order above, so a test checks a step by the name its trace would record: `assert.equal(commandName(step), 'cmdFindUser')`. Anything but a Command throws an `EffectTypeError`.
|
|
621
869
|
|
|
622
870
|
#### `Ask(nextFn)`
|
|
623
871
|
|
|
@@ -637,29 +885,27 @@ An option set to `undefined` keeps its default.
|
|
|
637
885
|
|
|
638
886
|
#### `Parallel(effects, next?, options?)`
|
|
639
887
|
|
|
640
|
-
Returns `{ type: 'Parallel', effects, next, options }
|
|
888
|
+
Returns `{ type: 'Parallel', effects, next, options }`, which runs the branches at the same time and hands `next` their values in order; by default `(values) => Success(values)`. The first branch to fail cancels the others and its `Failure` is the result, with `next` skipped. Which branch that is depends on timing, so it is recorded, and a replay returns the same one. Each branch's Commands get an `AbortSignal` to pass to their I/O; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
641
889
|
|
|
642
|
-
The second argument
|
|
890
|
+
The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
643
891
|
|
|
644
|
-
- `limit`: the most branches running at once. Results and recorded paths stay in array order, so
|
|
645
|
-
- `settled`: run every branch to the end and
|
|
892
|
+
- `limit`: the most branches running at once, a positive integer. Results and recorded paths stay in array order, so it changes only the pacing.
|
|
893
|
+
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order: no branch cancels the others or fails the `Parallel`. A bug still rejects the run, since a step or `next` that throws, or returns something other than an Effect, is not a branch outcome.
|
|
646
894
|
|
|
647
895
|
### Building pipelines
|
|
648
896
|
|
|
649
897
|
#### `effectPipe(...functions)`
|
|
650
898
|
|
|
651
|
-
Composes
|
|
652
|
-
|
|
653
|
-
A step does not have to use the value it receives. In JavaScript, closing over something from the enclosing scope is fine:
|
|
899
|
+
Composes steps into a pipeline: each step gets the previous step's `Success` value, and a `Failure` from any step stops it. A step may ignore the value it gets and close over the enclosing scope instead, which is fine in JavaScript:
|
|
654
900
|
|
|
655
901
|
```js
|
|
656
902
|
// The last step ignores what came before and uses the enclosing input instead.
|
|
657
903
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
658
904
|
```
|
|
659
905
|
|
|
660
|
-
In TypeScript,
|
|
906
|
+
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it; passing the value through every step, as the [Quick Start](#quick-start) does, keeps the whole pipeline checked. See [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context).
|
|
661
907
|
|
|
662
|
-
|
|
908
|
+
In either language, a Command a step builds and does not return never runs, and any `Failure` it would have produced is lost:
|
|
663
909
|
|
|
664
910
|
```js
|
|
665
911
|
(value) => {
|
|
@@ -668,47 +914,44 @@ One shape to avoid in either language:
|
|
|
668
914
|
};
|
|
669
915
|
```
|
|
670
916
|
|
|
671
|
-
|
|
917
|
+
Return the Command, and have its `next` return the value the rest of the pipeline needs.
|
|
672
918
|
|
|
673
919
|
### Running a flow
|
|
674
920
|
|
|
675
921
|
#### `runEffect(effect, context?, callConfig?)`
|
|
676
922
|
|
|
677
|
-
|
|
923
|
+
Runs a flow and resolves with its `Success` or `Failure`.
|
|
678
924
|
|
|
679
|
-
- `context`:
|
|
680
|
-
- `callConfig`:
|
|
681
|
-
- `onRun` fires once per `runEffect` call. Retry attempts run inside that one span.
|
|
925
|
+
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
926
|
+
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the `configureEffect` wiring, or used alone with `inherit: false`; see [`configureEffect`](#configureeffectconfigs). `onRun` wraps the run once, retries included. A `retry` key throws a `TypeError`, since retry options belong to each `Retry`.
|
|
682
927
|
|
|
683
|
-
A
|
|
928
|
+
A flow that is not made of Effects is a bug, so `runEffect` rejects with an `EffectTypeError` that says where, rather than returning a `Failure`:
|
|
684
929
|
|
|
685
930
|
```
|
|
686
931
|
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
687
932
|
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
688
933
|
```
|
|
689
934
|
|
|
690
|
-
|
|
935
|
+
That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments as the flow is built, before any of its I/O runs, so `Command(db.findUser(email))`, which makes the call while the flow is built, `Command(fn, { name })`, with the metadata in `next`'s place, and a `Parallel` or `effectPipe` given the wrong thing each throw an `EffectTypeError` there. A throw from a Command's function, or from an `onBeforeCommand` veto, is a `Failure`; a throw from a `next` or a pure step is a bug too, and `runEffect` rejects with it as thrown.
|
|
691
936
|
|
|
692
937
|
#### `configureEffect(...configs)`
|
|
693
938
|
|
|
694
|
-
|
|
695
|
-
- `onStep(name, type, op, path)` wraps each Command; must `await op()` and return its result. A hook that calls another hook passes `path` on, since a replay matches steps on it. `op()` returns a promise, even for a synchronous Command. Returning a value _without_ calling `op()` is how replay works. A throw after `op()` succeeded is a bug in the hook: the run rejects, and `Retry` does not run the Command again. So is returning `undefined` after `op()` returned a value, which is what a hook that forgot its `return` does: the run rejects with a `TypeError` instead of continuing as though the Command had returned nothing. A throw without calling `op()` counts as the Command failing.
|
|
696
|
-
- `onStep` also wraps each `Parallel`, with `name` and `type` both `'Parallel'`. Its `op()` runs the branches, so a hook must call it; a hook that returns without calling it makes the run reject with a `TypeError`. It returns which branch, if any, cancelled the others, such as `{ cancelled: true, branch: 0 }`, and it returns even when a branch threw, since the run rejects only after the hook has returned. Telemetry gets one span per `Parallel`, with the spans of its branches' Commands inside it, marked as an error when the `Parallel` was cancelled.
|
|
697
|
-
- `onBeforeCommand(command, context)` fires before each Command; throw to abort. The run returns a `Failure` carrying the thrown error, and `Retry` does not retry it.
|
|
698
|
-
|
|
699
|
-
It only configures hooks. Retry options are passed to `Retry(effect, options)`, and a `retry` key here throws a `TypeError`.
|
|
700
|
-
|
|
701
|
-
Each call adds a layer of hooks on top of those already installed and returns a function that removes that layer. Passing several configurations to one call is the same as calling it once for each, so these two forms are equivalent:
|
|
939
|
+
Adds a layer of hooks for the whole process and returns a function that removes it. Several configurations passed to one call form one layer, the same as installing each on its own. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's. `configureEffect()` with no arguments removes every layer, and a call whose arguments are all `undefined`, such as `configureEffect(flag ? hooks : undefined)`, changes nothing. A `retry` key throws a `TypeError`, since retry options belong to each `Retry`.
|
|
702
940
|
|
|
703
941
|
```js
|
|
704
|
-
configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
configureEffect(telemetryHooks());
|
|
708
|
-
configureEffect(recordingHooks({ sink }));
|
|
942
|
+
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
943
|
+
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
944
|
+
remove();
|
|
709
945
|
```
|
|
710
946
|
|
|
711
|
-
|
|
947
|
+
- `onRun(effect, pipeline, flowName)`: wraps the whole run. It must `await pipeline()` and return its result.
|
|
948
|
+
- `onStep(name, type, op, path)`: wraps each Command, and each `Parallel` with `name` and `type` both `'Parallel'`. It must `await op()` and return its result, and pass `path` on to any hook it calls, since a replay matches steps on it.
|
|
949
|
+
- For a Command, `op()` returns a promise, even for a synchronous function. Returning a value without calling `op()` answers for the Command, which is how replay works, and throwing without calling it counts as the Command failing.
|
|
950
|
+
- A throw after `op()` succeeded, or `undefined` returned in place of its value, as a hook that forgot its `return` or its `await` does, is a bug in the hook: the run rejects, and `Retry` does not run the Command again.
|
|
951
|
+
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, as in `{ cancelled: true, branch: 0 }`, even when a branch threw.
|
|
952
|
+
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it.
|
|
953
|
+
|
|
954
|
+
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
712
955
|
|
|
713
956
|
```
|
|
714
957
|
runEffect(flow(input))
|
|
@@ -729,15 +972,7 @@ runEffect(flow(input))
|
|
|
729
972
|
├─ telemetry.onRun returns
|
|
730
973
|
```
|
|
731
974
|
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
```js
|
|
735
|
-
const remove = configureEffect(telemetryHooks());
|
|
736
|
-
// ... later
|
|
737
|
-
remove();
|
|
738
|
-
```
|
|
739
|
-
|
|
740
|
-
**Per-call hooks.** By default, the hooks in a `callConfig` are merged with the configured ones by the same rules: configured wrappers go outside, and configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out of the run. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
975
|
+
**Per-call hooks.** The hooks in a `callConfig` are merged over the configured ones by the same rules, the configured wrappers outside and the configured `onBeforeCommand` hooks first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
741
976
|
|
|
742
977
|
```
|
|
743
978
|
inherit: true (default) inherit: false
|
|
@@ -747,17 +982,29 @@ onBeforeCommand C.onBeforeCommand (none)
|
|
|
747
982
|
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
748
983
|
```
|
|
749
984
|
|
|
750
|
-
`recordEffect` inherits, so recording inside an application that already
|
|
985
|
+
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
751
986
|
|
|
752
987
|
### Recording and replay
|
|
753
988
|
|
|
754
989
|
#### `recorder(options?)`
|
|
755
990
|
|
|
756
|
-
Returns `{ onStep, entries, toTrace }
|
|
991
|
+
Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run, passed as `runEffect(flow, context, { onStep })`. Give each run its own recorder: one shared by a whole application mixes requests into one trace, which a replay refuses. `recordEffect` and the [recording example](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js) both do this for you.
|
|
992
|
+
|
|
993
|
+
- `onStep`: the hook. Recording never changes a run's outcome; a value `redact` fails on is recorded as `'[redaction failed]'`.
|
|
994
|
+
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. `path` is the step's position in the flow, which a replay matches on; an older entry with `error` and no `threw` still counts as a throw.
|
|
757
995
|
|
|
758
|
-
|
|
996
|
+
```text
|
|
997
|
+
{ command, path, result, durationMs } a Command that returned
|
|
998
|
+
{ command, path, threw: true, error, durationMs } a Command that threw
|
|
999
|
+
{ command: 'Parallel', path, result: { cancelled, branch } } which branch, if any, cancelled the others
|
|
1000
|
+
```
|
|
759
1001
|
|
|
760
|
-
- `
|
|
1002
|
+
- `toTrace(meta)`: the entries as a trace, with `meta`'s `initialInput`, `context`, `flowName` and `version`, copied when it is called (a part that cannot be copied, such as a function, is kept as it is). If a Command can change the input or the context, as an ORM save that adds an id does, call it before the run for those and again afterwards for the entries, as `recordEffect` does.
|
|
1003
|
+
- `options.redact(value, name, kind)`: returns what the trace stores in place of `value`. `kind` is `'result'`, `'error'`, `'initialInput'` or `'context'`, and `name` is the Command's name, or the kind. It gets a copy, so changing it in place is safe.
|
|
1004
|
+
- `options.maxEntries`: the most entries a trace keeps; the rest are counted in `dropped`, and a replay stops at the first step the trace lacks.
|
|
1005
|
+
- `options.stack`: record stack traces for thrown errors (off by default).
|
|
1006
|
+
|
|
1007
|
+
Results are copied as they are recorded, so a later step cannot change them; see [Recording in Production](#recording-in-production) for what a copy keeps. When writing `redact`, check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
761
1008
|
|
|
762
1009
|
```js
|
|
763
1010
|
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
@@ -765,7 +1012,7 @@ const mask = (value, field) =>
|
|
|
765
1012
|
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
766
1013
|
|
|
767
1014
|
const redact = (value, name, kind) => {
|
|
768
|
-
if (kind === 'initialInput') return mask(value, '
|
|
1015
|
+
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
769
1016
|
if (kind === 'context') return mask(value, 'authToken');
|
|
770
1017
|
if (kind === 'error') return mask(value, 'attempted');
|
|
771
1018
|
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
@@ -776,39 +1023,34 @@ assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing fou
|
|
|
776
1023
|
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
777
1024
|
```
|
|
778
1025
|
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
For `'result'` and `'error'`, `name` is the Command's name; for `'initialInput'` and `'context'`, it is the kind. Redacting `initialInput` rarely breaks replay, since Commands are not run; it matters only if a step branches on the removed field. Redacting `context` breaks `Ask` replay if a step reads what you removed.
|
|
782
|
-
|
|
783
|
-
- `options.maxEntries`: Cap trace length; overflow is counted in `dropped`.
|
|
784
|
-
- `options.stack`: Record stack traces for thrown errors (off by default).
|
|
1026
|
+
A replay rebuilds the flow from the redacted input and context, so a field a step checks, such as a password, needs a stand-in the check treats the same way, as the recording example in [Recording in Production](#recording-in-production) does, and a context field an `Ask` reads should stay.
|
|
785
1027
|
|
|
786
1028
|
#### `recordEffect(flowFn, initialInput, options?)`
|
|
787
1029
|
|
|
788
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts. To record an application without changing call sites, use the wiring in `examples/recording-example.js
|
|
1030
|
+
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts. To record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
789
1031
|
|
|
790
1032
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
791
1033
|
|
|
792
|
-
|
|
1034
|
+
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for, which is where a flow that stops early shows up. A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`, which tells them apart, and not only `result.type`. A malformed trace rejects with a `ReplayError`.
|
|
793
1035
|
|
|
794
|
-
- `traceOrResolver`: a trace
|
|
795
|
-
- `options.context`: context for `Ask
|
|
796
|
-
- `options.onMissing`: `'throw'` (default)
|
|
1036
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver, a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of, so a trace stored in any shape can be replayed. With a resolver, `unreached` is absent, since a resolver cannot list what it holds. A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`: answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1037
|
+
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`.
|
|
1038
|
+
- `options.onMissing`: `'throw'` (default) stops at a step the trace lacks. `'execute'` runs its Command for real, so use it only where Commands reach test doubles or only read. A trace can lack a step because a hook vetoed it in production, because it was added to the flow since the recording, or because the recorder dropped it under `maxEntries`; a trace that dropped entries refuses `'execute'`, so record the flow again with a higher `maxEntries`. After a flow changes shape, as when a Command is newly wrapped in `Retry`, every step sits at a new position and `'execute'` would run them all live: record the flow again instead.
|
|
797
1039
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
798
|
-
- `options.hooks` (default `false`): run the replay inside the configured hooks,
|
|
799
|
-
- `options.onResolved(step, outcome)`:
|
|
1040
|
+
- `options.hooks` (default `false`): run the replay inside the configured hooks, a configured recorder included, so they see the replayed steps. Off, a replay cannot reach a telemetry backend or a trace sink.
|
|
1041
|
+
- `options.onResolved(step, outcome)`: observes each replayed step. A throw stops the replay, and `replayEffect` rejects with it; it never changes an outcome.
|
|
800
1042
|
|
|
801
|
-
A trace whose entries
|
|
1043
|
+
A trace whose entries have no `path`, written by hand or recorded before paths existed, is matched by position. That works for a sequential flow, and a `Parallel` step in such a trace is refused with a `ReplayError`, since the order branches finish in cannot tell them apart.
|
|
802
1044
|
|
|
803
1045
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
804
1046
|
|
|
805
|
-
Replays a trace and narrates each step with its recorded duration, naming any recorded steps that were never reached and warning when the trace's `version` differs from `options.version`. Returns the flow's outcome; for the unreached entries as data, use `replayEffect`. Takes `options.context` to override the trace's, and `options.log` in place of `console.log`.
|
|
1047
|
+
Replays a trace and narrates each step with its recorded duration, naming any recorded steps that were never reached, and warning when the trace's `version` differs from `options.version` or when steps are named `'anonymous'`. Returns the flow's outcome; for the unreached entries as data, use `replayEffect`. Takes `options.context` to override the trace's, and `options.log` in place of `console.log`.
|
|
806
1048
|
|
|
807
1049
|
## Limitations
|
|
808
1050
|
|
|
809
1051
|
- **`Retry` repeats everything it wraps, including a Command's `next`.** When a Command's function throws, every Command inside the `Retry` runs again, including the ones that already succeeded. If a retried Command's `next` continues the flow, everything after it is retried too. Give a retried Command the default `next` and continue in a later pipeline step, or make sure every Command it reaches is safe to run more than once. A `Failure` a step returned is not retried at all, so a function that catches its own error and returns it as a value is never retried: see [Which Errors Are Data](#which-errors-are-data).
|
|
810
|
-
- **Cancelling a `Parallel` branch cannot stop everything.** A cancelled branch starts no new Commands, and a function that uses the `AbortSignal` it is given can be stopped while running. A function that ignores the signal runs to completion, so a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch to finish before returning, so no cancelled work is still running after the `Failure` is returned.
|
|
811
|
-
- **Inside a `Parallel`, a Command's function gets the `AbortSignal`
|
|
1052
|
+
- **Cancelling a `Parallel` branch cannot stop everything.** A cancelled branch starts no new Commands, and a function that uses the `AbortSignal` it is given can be stopped while running. A function that ignores the signal runs to completion, so a branch whose _first_ Command is a write can still write after another branch has failed. So does a function that is never given the signal: one written as `(...args)` or `({ signal } = {})`, or wrapped by a helper that hides its parameters. `Parallel` waits for every branch to finish before returning, so no cancelled work is still running after the `Failure` is returned.
|
|
1053
|
+
- **Inside a `Parallel`, a Command's function that takes a parameter gets the `AbortSignal` in it.** A function passed by name whose first parameter is optional but written without a default value takes the signal for it, so the same Command behaves differently inside a `Parallel` than outside one: `function listUsers(page) { page ??= 1; return db.users(page); }` gets the signal as its page. A parameter with a default value does not count, so `Command(nanoid)`, whose size is written `size = 21`, works everywhere. Wrap such a function, as in `Command(() => listUsers())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
812
1054
|
- **A `Failure` carries everything.** It holds the full error and the `initialInput` that `effectPipe` attached, and neither is trimmed, because tests and debugging need both. `redact` keeps sensitive data out of a **trace**. Keeping it out of your **logs** is up to you: log `result.error` rather than the whole `Failure`. The same goes for the outcomes a settled `Parallel` passes to `next`, which can carry their branch's input; for a login or registration flow, that input holds credentials.
|
|
813
1055
|
- **Replay checks the path, not the values.** Replaying an old trace cannot check a fix that changes what a flow computes, because every Command's result comes from the recording: the replay hands the step the value from before the fix, reports `Success`, and flags nothing. It can check a fix that changes which Commands run. A removed step shows up in `unreached` only if it was the last step the flow reached; removed from the middle, it shifts the later paths and the replay stops at a `TimeParadox`.
|
|
814
1056
|
- **A trace keeps data, not objects.** Recorded results come back without their classes: a money object or a database entity as a plain object, a `Buffer` as a plain byte array, and an error inside a result without properties such as `code`. Through JSON, a `Date` also comes back as a string and a `Map` as `{}`. A flow whose `next` calls a method on a result, or branches on one of those properties, replays differently from production. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|