pure-effect 0.14.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -0
- package/README.md +372 -120
- package/examples/opentelemetry-example.js +173 -0
- package/examples/recording-example.js +170 -0
- package/index.d.ts +1067 -1089
- package/index.js +835 -698
- package/package.json +7 -4
package/README.md
CHANGED
|
@@ -9,27 +9,17 @@
|
|
|
9
9
|
- Inject context without touching function signatures
|
|
10
10
|
- Built-in retry, plus parallel execution that cancels sibling branches on the first failure
|
|
11
11
|
- OpenTelemetry-ready via lifecycle hooks
|
|
12
|
-
- Zero dependencies, about
|
|
13
|
-
- Works in JavaScript and TypeScript (full generics, bundled `.d.ts`)
|
|
12
|
+
- Zero dependencies, about 7 KB minified and gzipped
|
|
13
|
+
- Works in JavaScript, and in TypeScript 5.1 or later (full generics, bundled `.d.ts`)
|
|
14
14
|
|
|
15
15
|
## Table of Contents
|
|
16
16
|
|
|
17
|
-
- [Installation](#installation)
|
|
18
|
-
- [
|
|
19
|
-
- [
|
|
20
|
-
- [
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
- [Passing Runtime Context](#passing-runtime-context)
|
|
24
|
-
- [Retrying Transient Failures](#retrying-transient-failures)
|
|
25
|
-
- [Running Effects in Parallel](#running-effects-in-parallel)
|
|
26
|
-
- [Composing Larger Flows](#composing-larger-flows)
|
|
27
|
-
- [Which Errors Are Data](#which-errors-are-data)
|
|
28
|
-
- [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context)
|
|
29
|
-
- [Why Pure Effect](#why-pure-effect)
|
|
30
|
-
- [Load Tests](#load-tests)
|
|
31
|
-
- [API Reference](#api-reference)
|
|
32
|
-
- [Limitations](#limitations)
|
|
17
|
+
- **Getting started:** [Installation](#installation) · [Quick Start](#quick-start) · [How It Works](#how-it-works) · [Testing Without Mocks](#testing-without-mocks) · [Coming from async/await](#coming-from-asyncawait)
|
|
18
|
+
- **Recording and replay:** [Time-Travel Debugging](#time-travel-debugging) · [Recording in Production](#recording-in-production)
|
|
19
|
+
- **Building flows:** [Passing Runtime Context](#passing-runtime-context) · [Retrying Transient Failures](#retrying-transient-failures) · [Running Effects in Parallel](#running-effects-in-parallel) · [Composing Larger Flows](#composing-larger-flows) · [Which Errors Are Data](#which-errors-are-data) · [TypeScript](#typescript-typed-errors-and-context)
|
|
20
|
+
- **Reference:** [Why Pure Effect](#why-pure-effect) · [Load Tests](#load-tests) · [API Reference](#api-reference) · [Limitations](#limitations)
|
|
21
|
+
|
|
22
|
+
The five sections under Getting started are enough to write and test a flow. Come back to the rest when a flow needs it: recording, retries, steps that run at the same time, or TypeScript.
|
|
33
23
|
|
|
34
24
|
## Installation
|
|
35
25
|
|
|
@@ -80,6 +70,14 @@ async function registerUser(input) {
|
|
|
80
70
|
}
|
|
81
71
|
```
|
|
82
72
|
|
|
73
|
+
## How It Works
|
|
74
|
+
|
|
75
|
+
A flow is a chain of pairs: an I/O call, and a `next` function that receives its answer and returns the next Command, a Success, or a Failure. `runEffect` walks the chain in a loop.
|
|
76
|
+
|
|
77
|
+
Recording and replay hook into the one place where a Command's function is called. Recording writes down each answer. Replay supplies the recorded answer instead of making the call, so a replayed flow does no I/O.
|
|
78
|
+
|
|
79
|
+
Building a flow runs the pure steps right away: `registerUserFlow(badInput)` returns the validation `Failure` synchronously, so the tests below do not need `runEffect`. Only Commands need it.
|
|
80
|
+
|
|
83
81
|
## Testing Without Mocks
|
|
84
82
|
|
|
85
83
|
Pipelines return plain objects, so you can check what the code will do without running it.
|
|
@@ -103,14 +101,16 @@ assert.deepEqual(ensureEmailAvailable(input).next(null), Success(input));
|
|
|
103
101
|
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
104
102
|
```
|
|
105
103
|
|
|
106
|
-
Then test the flow: the right calls, in the right order.
|
|
104
|
+
Then test the flow: the right calls, in the right order. `commandName` gives the name a Command is recorded under:
|
|
107
105
|
|
|
108
106
|
```js
|
|
107
|
+
import { commandName } from 'pure-effect';
|
|
108
|
+
|
|
109
109
|
const step1 = registerUserFlow(input);
|
|
110
|
-
assert.equal(step1
|
|
110
|
+
assert.equal(commandName(step1), 'cmdFindUser');
|
|
111
111
|
|
|
112
112
|
const step2 = step1.next(null); // pretend no user was found
|
|
113
|
-
assert.equal(step2
|
|
113
|
+
assert.equal(commandName(step2), 'cmdSaveUser');
|
|
114
114
|
// The database was never touched.
|
|
115
115
|
```
|
|
116
116
|
|
|
@@ -120,15 +120,169 @@ For example, if the email guard returned `Success(true)` instead of `Success(inp
|
|
|
120
120
|
|
|
121
121
|
A step tested on its own returns a `Failure` without `initialInput`. That is why the assertion above is `Failure('Email already in use.')`, while the validation one is `Failure('Invalid email.', badInput)`: `effectPipe` adds the input when it builds a flow.
|
|
122
122
|
|
|
123
|
+
A walk stops at each Command to be handed its answer. To run the whole flow instead, `Retry` and `Parallel` included, hand `replayEffect` a function that answers each Command by name. Nothing is called and a retry does not wait, and answering `{ error }` makes that Command throw, which is how a test reaches a `Retry`'s fallback:
|
|
124
|
+
|
|
125
|
+
```js
|
|
126
|
+
const answers = { cmdFindUser: null, cmdSaveUser: { id: 1, ...input } };
|
|
127
|
+
const answer = (step) => (step.name in answers ? { result: answers[step.name] } : undefined);
|
|
128
|
+
const { result } = await replayEffect(registerUserFlow(input), answer);
|
|
129
|
+
assert.deepEqual(result, Success({ id: 1, ...input }));
|
|
130
|
+
```
|
|
131
|
+
|
|
123
132
|
Tests cannot see inside a Command's function without running it. If `cmdFindUser` said `db.findUser(input.name)` instead of `input.email`, every test on this page would still pass. Keep those functions to a single call, and let an integration test cover them.
|
|
124
133
|
|
|
125
|
-
|
|
134
|
+
**In TypeScript**, a flow is typed as any kind of step it could start with, since validation may already have returned a `Success` or a `Failure`, so `.cmd` and `.next` compile only after the test checks that the step is a Command. `node:assert`'s `assert(step.type === 'Command')` is such a check, and TypeScript follows it. Jest's and Vitest's `expect` is not, so a walk needs a check at every step, and a small helper makes that one line per step with any test framework:
|
|
126
135
|
|
|
127
|
-
|
|
136
|
+
```ts
|
|
137
|
+
import assert from 'node:assert/strict';
|
|
138
|
+
import { commandName } from 'pure-effect';
|
|
139
|
+
import type { Effect } from 'pure-effect';
|
|
140
|
+
|
|
141
|
+
// Fails unless the step is the Command named, and returns it typed as one.
|
|
142
|
+
function assertCommand<T, E, C>(step: Effect<T, E, C>, name: string) {
|
|
143
|
+
assert(step.type === 'Command', `expected ${name}, got ${step.type}`);
|
|
144
|
+
assert.equal(commandName(step), name);
|
|
145
|
+
return step;
|
|
146
|
+
}
|
|
128
147
|
|
|
129
|
-
|
|
148
|
+
const step1 = assertCommand(registerUserFlow(input), 'cmdFindUser');
|
|
149
|
+
assertCommand(step1.next(null), 'cmdSaveUser');
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
For a step that returns its Command directly, as in `ensureEmailAvailable(input).next(found)`, `next` is checked against what the Command's function returns, so where the tests above pass `{ id: 1 }` for a user that was found, a TypeScript test passes a whole user, of the type `db.findUser` returns. A Command reached any other way, by walking the whole flow or through an `Ask`'s `next`, accepts any value, since its type does not say which Command it is.
|
|
153
|
+
|
|
154
|
+
## Coming from async/await
|
|
155
|
+
|
|
156
|
+
A flow does what `async`/`await` code does, as data. Instead of making a call and waiting for it, a step returns a Command that says which call to make and what to do with the answer. That is what lets a test hand a step an answer instead of mocking the database, and a replay hand it the recorded one. Each thing you would write with `async`/`await` has an equivalent:
|
|
157
|
+
|
|
158
|
+
| With `async`/`await` | In a flow |
|
|
159
|
+
| --- | --- |
|
|
160
|
+
| `const user = await db.findUser(email)` | a Command: `Command(() => db.findUser(email), next)` |
|
|
161
|
+
| the code after the `await`, which uses `user` | the Command's `next`, which receives `user` |
|
|
162
|
+
| an `if` on `user` after the `await` | a branch in `next` |
|
|
163
|
+
| `return value` | `Success(value)` |
|
|
164
|
+
| `throw` to stop | `Failure(error)` |
|
|
165
|
+
| a call inside `try`/`catch` | catch inside the Command's function and return a value for `next` to branch on; see [Which Errors Are Data](#which-errors-are-data) |
|
|
166
|
+
| a loop that retries a call | `Retry(Command(...), options)` |
|
|
167
|
+
| `await Promise.all([...])` | `Parallel([...])` |
|
|
168
|
+
| a `for` loop with an `await` inside | a step that gets the list, then `Parallel(list.map(...), { limit: 1 })` |
|
|
169
|
+
| a `try`/`catch` inside a loop, to keep going past a failure | the same `Parallel`, with `settled: true` |
|
|
170
|
+
| a later call that needs two earlier results | a named step that passes both on; see [Composing Larger Flows](#composing-larger-flows) |
|
|
171
|
+
| a value from the request, such as the tenant | `Ask`; see [Passing Runtime Context](#passing-runtime-context) |
|
|
172
|
+
| `Date.now()` or a random ID | a Command, so a replay gets the recorded value |
|
|
173
|
+
|
|
174
|
+
The Quick Start's `registerUserFlow` is this function, translated row by row: each `await` became a Command, the code after it became that Command's `next`, and each `throw` became a `Failure`.
|
|
175
|
+
|
|
176
|
+
```js
|
|
177
|
+
async function registerUserAsync(input) {
|
|
178
|
+
if (!input.email.includes('@')) throw new Error('Invalid email.');
|
|
179
|
+
const found = await db.findUser(input.email);
|
|
180
|
+
if (found) throw new Error('Email already in use.');
|
|
181
|
+
return db.saveUser(input);
|
|
182
|
+
}
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
The difference shows most in the tests. To reach the branch where the email is taken, a test of `registerUserAsync` has to replace the database's methods, run the function, and put the methods back. Here it is with Node's built-in mocks; Jest's `spyOn` works the same way:
|
|
186
|
+
|
|
187
|
+
```js
|
|
188
|
+
import { mock } from 'node:test';
|
|
189
|
+
|
|
190
|
+
const input = { email: 'test@test.com', password: 'password123' };
|
|
191
|
+
|
|
192
|
+
mock.method(db, 'findUser', async () => ({ id: 1 }));
|
|
193
|
+
const saveUser = mock.method(db, 'saveUser', async (user) => user);
|
|
194
|
+
|
|
195
|
+
await assert.rejects(registerUserAsync(input), { message: 'Email already in use.' });
|
|
196
|
+
assert.equal(saveUser.mock.callCount(), 0);
|
|
197
|
+
mock.restoreAll();
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
The flow's test of the same branch hands one step the answer, with nothing to replace, wait for, or put back:
|
|
201
|
+
|
|
202
|
+
```js
|
|
203
|
+
assert.deepEqual(ensureEmailAvailable(input).next({ id: 1 }), Failure('Email already in use.'));
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
This is what the small functions are for. Each step is a place a test can start, so a branch is tested by calling the step it lives in. An `async` function has one way in, so a test of any branch runs everything before it, with every call on the way replaced. The mocks can check one thing these tests cannot: the arguments each call received, which [Testing Without Mocks](#testing-without-mocks) leaves to an integration test.
|
|
207
|
+
|
|
208
|
+
A loop is written in two parts: a step that gets the list, and a `Parallel` over it. `Parallel` runs a list of flows, at the same time unless told otherwise, and `limit: 1` makes it run them one after another, like a `for` loop. The `async` version is usually split the same way. Paying a customer's unpaid invoices in order, and stopping at the first that fails:
|
|
209
|
+
|
|
210
|
+
```js
|
|
211
|
+
async function payUnpaidAsync(customerId) {
|
|
212
|
+
const invoices = await billing.findUnpaid(customerId);
|
|
213
|
+
for (const invoice of invoices) {
|
|
214
|
+
const receipt = await payments.charge(invoice);
|
|
215
|
+
await mailer.sendReceipt(receipt);
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
|
|
219
|
+
// First part: get the list.
|
|
220
|
+
const findUnpaidInvoices = (customerId) => {
|
|
221
|
+
const cmdFindUnpaid = () => billing.findUnpaid(customerId);
|
|
222
|
+
return Command(cmdFindUnpaid);
|
|
223
|
+
};
|
|
224
|
+
|
|
225
|
+
// Second part: the loop's body, a flow of its own for one invoice.
|
|
226
|
+
const chargeInvoice = (invoice) => {
|
|
227
|
+
const cmdChargeInvoice = () => payments.charge(invoice);
|
|
228
|
+
return Retry(Command(cmdChargeInvoice), { attempts: 3 });
|
|
229
|
+
};
|
|
230
|
+
|
|
231
|
+
const sendReceipt = (receipt) => {
|
|
232
|
+
const cmdSendReceipt = () => mailer.sendReceipt(receipt);
|
|
233
|
+
return Command(cmdSendReceipt);
|
|
234
|
+
};
|
|
235
|
+
|
|
236
|
+
const payInvoice = (invoice) => effectPipe(chargeInvoice, sendReceipt)(invoice);
|
|
237
|
+
|
|
238
|
+
// The loop: one body per invoice, run one at a time.
|
|
239
|
+
const payAll = (invoices) => Parallel(invoices.map(payInvoice), { limit: 1 });
|
|
240
|
+
|
|
241
|
+
const payUnpaid = (customerId) => effectPipe(findUnpaidInvoices, payAll)(customerId);
|
|
242
|
+
|
|
243
|
+
// The flow starts by getting the list, and hands what it gets to the loop.
|
|
244
|
+
const flow = payUnpaid('cus_1');
|
|
245
|
+
assert.equal(commandName(flow), 'cmdFindUnpaid');
|
|
246
|
+
assert.equal(flow.next([{ id: 1 }, { id: 2 }]).effects.length, 2);
|
|
247
|
+
|
|
248
|
+
// The body is tested on its own: the Retry holds only the charge, and the receipt comes after it.
|
|
249
|
+
const body = payInvoice({ id: 1 });
|
|
250
|
+
assert.equal(commandName(body.effect), 'cmdChargeInvoice');
|
|
251
|
+
assert.equal(commandName(body.next({ id: 'receipt_1' })), 'cmdSendReceipt');
|
|
252
|
+
|
|
253
|
+
// An empty list runs nothing.
|
|
254
|
+
assert.deepEqual(await runEffect(payAll([])), Success([]));
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
Each invoice is paid after the one before it has finished, and the first that fails stops the rest from starting, as in the `async` version. When the payments can overlap, a higher `limit` runs several at a time.
|
|
258
|
+
|
|
259
|
+
The loop's body is a flow built for one item, so it can hold as many steps as the body of a `for` loop. Put a `Retry` inside it, on the step that needs one. Around the whole loop, a `Retry` would pay every invoice again after one failed, charging the ones already paid; around the whole body, a receipt that failed to send would charge the card again. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
260
|
+
|
|
261
|
+
The `Parallel`'s `next` receives what each body returned, in order, so a total or a summary of the loop goes there: `Parallel(invoices.map(payInvoice), (receipts) => Success(receipts.length), { limit: 1 })`. To keep going past a failure, as a `try`/`catch` inside the loop would, add `settled: true`: every invoice is tried, and `next` receives one `Success` or `Failure` per invoice. See [Running Effects in Parallel](#running-effects-in-parallel).
|
|
130
262
|
|
|
131
|
-
|
|
263
|
+
Sometimes getting the list is a loop of its own. Paging through an API with a cursor is one: each request needs the cursor from the page before, so the requests cannot be listed in advance. Write that loop as an ordinary `async` function, call it from one Command, and map over the list it returns, as above:
|
|
264
|
+
|
|
265
|
+
```js
|
|
266
|
+
async function fetchAllOrders() {
|
|
267
|
+
const orders = [];
|
|
268
|
+
let cursor = null;
|
|
269
|
+
do {
|
|
270
|
+
const page = await api.listOrders(cursor);
|
|
271
|
+
orders.push(...page.items);
|
|
272
|
+
cursor = page.nextCursor;
|
|
273
|
+
} while (cursor);
|
|
274
|
+
return orders;
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
const listAllOrders = () => {
|
|
278
|
+
const cmdListAllOrders = () => fetchAllOrders();
|
|
279
|
+
return Command(cmdListAllOrders);
|
|
280
|
+
};
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
To the flow, the whole loop is one step. A trace records the finished list as that step's result, and a replay hands the list back without replaying the pages. A `Retry` around the Command starts again from the first page. A flow test cannot see inside it, so, like any Command's function, it is left to an integration test.
|
|
284
|
+
|
|
285
|
+
The loop belongs in the flow only when the flow has to decide something on each pass, such as stopping at the first order that matches, or saving each page as it arrives. Then the Command's `next` returns the Command for the next page instead of a `Success`, until there are no more pages.
|
|
132
286
|
|
|
133
287
|
## Time-Travel Debugging
|
|
134
288
|
|
|
@@ -137,8 +291,11 @@ Record what each Command returned, then feed those results back into the flow to
|
|
|
137
291
|
```js
|
|
138
292
|
import { recordEffect, replayEffect, timeTravel } from 'pure-effect';
|
|
139
293
|
|
|
140
|
-
// Record a real run.
|
|
141
|
-
const { result, trace } = await recordEffect(checkoutFlow, input, {
|
|
294
|
+
// Record a real run. The context's flowName names the flow in the trace.
|
|
295
|
+
const { result, trace } = await recordEffect(checkoutFlow, input, {
|
|
296
|
+
version: process.env.BUILD_ID,
|
|
297
|
+
context: { flowName: 'checkout' }
|
|
298
|
+
});
|
|
142
299
|
|
|
143
300
|
// Later, somewhere else, with nothing connected:
|
|
144
301
|
await timeTravel(checkoutFlow, trace);
|
|
@@ -146,11 +303,28 @@ await timeTravel(checkoutFlow, trace);
|
|
|
146
303
|
|
|
147
304
|
```
|
|
148
305
|
Replaying 'checkout' (3 recorded steps)
|
|
149
|
-
Initial input: {
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
306
|
+
Initial input: {
|
|
307
|
+
"cartId": "cart_abc123",
|
|
308
|
+
"promoCode": "FREE_YEAR_VIP"
|
|
309
|
+
}
|
|
310
|
+
Step 1: cmdFetchCart returned in 41.277ms {
|
|
311
|
+
"totalAmount": "120.00"
|
|
312
|
+
}
|
|
313
|
+
Step 2: cmdValidatePromo returned in 26.814ms {
|
|
314
|
+
"isValid": true,
|
|
315
|
+
"discountValue": 100
|
|
316
|
+
}
|
|
317
|
+
Step 3: cmdChargeCard threw in 181.507ms {
|
|
318
|
+
"code": "invalid_amount",
|
|
319
|
+
"name": "Error",
|
|
320
|
+
"message": "Amount must be non-zero."
|
|
321
|
+
}
|
|
153
322
|
Replay finished with state: Failure
|
|
323
|
+
Error: {
|
|
324
|
+
"code": "invalid_amount",
|
|
325
|
+
"name": "Error",
|
|
326
|
+
"message": "Amount must be non-zero."
|
|
327
|
+
}
|
|
154
328
|
```
|
|
155
329
|
|
|
156
330
|
A production incident becomes a regression test, with no mocks or fixtures:
|
|
@@ -163,9 +337,9 @@ it('prod incident 8f3a: a 100% promo produces a $0 charge', async () => {
|
|
|
163
337
|
});
|
|
164
338
|
```
|
|
165
339
|
|
|
166
|
-
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay
|
|
340
|
+
The test checks that the flow still takes the recorded path and handles the recorded outcomes the same way. If a refactor reorders or replaces a step, the replay ends in a `Failure` whose error is a `TimeParadox` naming where it diverged. It is not thrown, so a test that expects a `Failure` should check the error too, as this one checks its `code`. If the error handling changes, the assertion fails.
|
|
167
341
|
|
|
168
|
-
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge)
|
|
342
|
+
A flow that stops issuing Commands before the recording ends (for example, a fix that skips the charge) produces no `TimeParadox`, and the replay can end in `Success` with recorded steps left over. `replayEffect` returns those steps as `unreached`, so a test can assert which ones it expects to skip:
|
|
169
343
|
|
|
170
344
|
```js
|
|
171
345
|
it('incident 8f3a fixed: a 100% promo checks out without a charge', async () => {
|
|
@@ -184,19 +358,22 @@ This works when the removed step is the last one the flow reaches. Remove a step
|
|
|
184
358
|
|
|
185
359
|
**Replay checks the path, not the values.** It shows that the flow asks for the same Commands in the same order and handles the same answers. It cannot check what the flow computes, because every Command's result comes from the recording, not from the new code. If you fix how a refund amount is calculated and replay the bad run, the charge step gets the old amount from the recording: the replay reports `Success`, returns the value from before the fix, and flags nothing. Test a change to a value against the pure step that computes it.
|
|
186
360
|
|
|
361
|
+
**A replay cannot check a fix inside a Command's function either.** The function never runs, so the replay hands the flow what it returned or threw before the fix: catch a duplicate-key error inside the function and return it as data, and the old trace still replays the throw. Cover such a fix with an integration test. And when a Command's result changes shape, rename the Command, since a trace matches it by name: under the old name, an old trace hands the new code a result in the old shape, which it can take for another answer without any `TimeParadox`.
|
|
362
|
+
|
|
187
363
|
**A trace records what each Command returned, not the arguments it was called with.** Given the recorded input and results, the flow does the same thing again, so replaying up to a step rebuilds the arguments that Command ran with. You can inspect them in a debugger, but you cannot assert on them. Storing arguments would also double what `redact` has to cover, since arguments are usually the sensitive part of a call.
|
|
188
364
|
|
|
189
365
|
**Put anything that varies between runs inside a Command.** For replay, the current time or a random ID counts as I/O. Wrapped in a Command, it is recorded and replayed like any other result. A step that calls `Date.now()` directly gets a new value on every replay and drifts from the trace.
|
|
190
366
|
|
|
191
|
-
**To check for this,** record a flow,
|
|
367
|
+
**To check for this,** record a flow, send the trace through JSON as your storage would, replay it, and compare the outcomes. A step that computes a new value on each run shows up as a `TimeParadox` if it changes which Commands run, or as a different final value. Going through JSON also catches a result that storage cannot keep, such as a `Date`, a class instance, or `undefined` in an array, which is where a replay from memory and a replay from storage part ways:
|
|
192
368
|
|
|
193
369
|
```js
|
|
194
370
|
const { result, trace } = await recordEffect(registerUserFlow, input);
|
|
195
|
-
const
|
|
196
|
-
|
|
371
|
+
const stored = JSON.parse(JSON.stringify(trace)); // what your storage gives back
|
|
372
|
+
const { result: replayed } = await replayEffect(registerUserFlow(input), stored);
|
|
373
|
+
assert.deepEqual(replayed, result); // fails if a step computed a fresh value, or a result did not survive JSON
|
|
197
374
|
```
|
|
198
375
|
|
|
199
|
-
This also works for a `Failure`: an error read back from a trace keeps its message, name, cause, custom properties, and an `AggregateError`'s list of errors, and is deep-equal to the one the Command threw. The exception is a custom error class, which comes back as a plain `Error` with that class's name; compare `error.name` and `error.message` instead.
|
|
376
|
+
This also works for a `Failure`: an error read back from a trace keeps its message, name, cause, custom properties, and an `AggregateError`'s list of errors, and is deep-equal to the one the Command threw, including one whose `name` or `cause` was set after it was created. The exception is a custom error class, which comes back as a plain `Error` with that class's name; compare `error.name` and `error.message` instead.
|
|
200
377
|
|
|
201
378
|
**You choose the trace format.** `replayEffect` also accepts a resolver function instead of a trace, so you can replay from OpenTelemetry spans, a log pipeline, or a database table, not only from the JSON that `recorder` produces.
|
|
202
379
|
|
|
@@ -208,9 +385,10 @@ await replayEffect(checkoutFlow(input), resolve);
|
|
|
208
385
|
|
|
209
386
|
## Recording in Production
|
|
210
387
|
|
|
211
|
-
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup
|
|
388
|
+
`recordEffect` suits tests and scripts, where one call covers the whole run. To record an application without changing any call site, install the hooks once at startup. The two files imported below ship in the package's `examples` folder, `node_modules/pure-effect/examples`, and in the repository's [examples folder](https://github.com/aycangulez/pure-effect/blob/main/examples); copy them into your project.
|
|
212
389
|
|
|
213
390
|
```js
|
|
391
|
+
import { randomUUID } from 'node:crypto';
|
|
214
392
|
import { configureEffect } from 'pure-effect';
|
|
215
393
|
import { recordingHooks } from './recording-example.js';
|
|
216
394
|
import { telemetryHooks } from './opentelemetry-example.js';
|
|
@@ -218,19 +396,21 @@ import { telemetryHooks } from './opentelemetry-example.js';
|
|
|
218
396
|
configureEffect(
|
|
219
397
|
telemetryHooks(),
|
|
220
398
|
recordingHooks({
|
|
221
|
-
sink: (trace) => putObject(`traces/${trace.flowName}/${
|
|
222
|
-
|
|
399
|
+
sink: (trace) => putObject(`traces/${trace.flowName}/${randomUUID()}.json`, JSON.stringify(trace)),
|
|
400
|
+
// A replay runs validation again on the stored input, so the stand-in keeps the password rule's verdict.
|
|
401
|
+
redact: (value, name, kind) =>
|
|
402
|
+
kind === 'initialInput' ? { ...value, password: value.password.length >= 8 ? '[redacted]' : '' } : value,
|
|
223
403
|
maxEntries: 500,
|
|
224
404
|
keep: (result) => result.type === 'Failure' // the default; keep everything, or sample
|
|
225
405
|
})
|
|
226
406
|
);
|
|
227
407
|
```
|
|
228
408
|
|
|
229
|
-
By default, successful runs are held in memory and then discarded. A run that rejects because your own code threw, such as a `TypeError` after an API changed the shape of its response, is offered to `keep` as a `Failure` carrying the thrown error, so by default its trace is kept, and the run still rejects. Replaying that trace reproduces the throw. If `keep` or the sink throws, for example because `JSON.stringify` meets a value it cannot serialize, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`. `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. `maxEntries` caps the length of a trace and reports the overflow as `dropped`. The sink can write a trace to S3 or a database column as JSON, as long as your Commands return plain data.
|
|
409
|
+
By default, successful runs are held in memory and then discarded. A run that rejects because your own code threw, such as a `TypeError` after an API changed the shape of its response, is offered to `keep` as a `Failure` carrying the thrown error, so by default its trace is kept, and the run still rejects. Replaying that trace reproduces the throw. If `keep` or the sink throws, for example because `JSON.stringify` meets a value it cannot serialize, the run keeps its own outcome and the error goes to `onSinkError`, which defaults to `console.error`. `redact` runs before anything enters the trace, including the stored `initialInput` and `context`. `maxEntries` caps the length of a trace and reports the overflow as `dropped`; a capped trace replays only up to the first step it lacks, so the example warns through `onWarning` the first time it keeps one for a flow. It warns the same way about a flow whose steps include any named `'anonymous'`. The sink can write a trace to S3 or a database column as JSON, as long as your Commands return plain data.
|
|
230
410
|
|
|
231
|
-
The example reads the input from the flow itself, where `effectPipe` stores it, so build the outermost flow with `effectPipe`. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
411
|
+
The example reads the input from the flow itself, where `effectPipe` stores it, so build the outermost flow with `effectPipe`. A flow that starts with a bare `Command`, `Ask`, `Retry`, or `Parallel` records no input, and `timeTravel` would rebuild it from `undefined`. A one-step pipeline is enough, as in `runEffect(effectPipe(loadProfile)(userId))`. The example warns through `onWarning`, which defaults to `console.warn`, the first time it keeps a trace of such a flow, and `timeTravel` warns when it replays one. The example also takes the context from the run's first Command, so a run that stops before any Command, for example at an `Ask` check, records no context. Such a run did no I/O, so running the flow again with its input and the context from your logs reproduces it.
|
|
232
412
|
|
|
233
|
-
**A trace keeps data, not objects.** A result is copied when it is recorded, and copied again each time a replay hands it to the flow, so neither a later step nor a replay can change what the trace says. The copy keeps values but not classes: a money object or a database entity comes back as a plain object without its methods, and a `Buffer` as a plain byte array. An error returned inside a result keeps its message but loses properties such as `code`. Writing the trace as JSON loses more: a `Date` becomes a string, a `Map` becomes `{}`, an error inside a result becomes `{}`, and a `BigInt` makes the sink throw.
|
|
413
|
+
**A trace keeps data, not objects.** A result is copied when it is recorded, and copied again each time a replay hands it to the flow, so neither a later step nor a replay can change what the trace says. The copy keeps values but not classes: a money object or a database entity comes back as a plain object without its methods, and a `Buffer` as a plain byte array. An error returned inside a result keeps its message but loses properties such as `code`. Writing the trace as JSON loses more: a `Date` becomes a string, a `Map` becomes `{}`, an error inside a result becomes `{}`, `undefined` in an array becomes `null`, and a `BigInt` makes the sink throw.
|
|
234
414
|
|
|
235
415
|
So return plain data from a Command's function. Turn a class instance into a plain object before returning it, with a small named function you can test on its own, as in `() => db.findCart(id).then(cartToData)`, and rebuild the object in `next` if the flow needs its methods. When you catch an error to return it as data, copy the fields the flow branches on, as in `.catch((error) => ({ ok: false, code: error.code }))`, rather than returning the error itself.
|
|
236
416
|
|
|
@@ -251,15 +431,17 @@ app.post('/checkout', async (req, res) => {
|
|
|
251
431
|
const result = await runEffect(checkoutFlow(req.body.productId), { tenant: req.tenant });
|
|
252
432
|
if (result.type === 'Success') return res.json(result.value);
|
|
253
433
|
// A Failure also carries the flow's input, so send the client only an error meant for it.
|
|
254
|
-
|
|
434
|
+
// Answer the errors the flow returns by name, and anything else, such as a database outage, as a server error.
|
|
435
|
+
if (result.error === 'Product not found.') return res.status(400).json({ error: result.error });
|
|
436
|
+
res.status(500).json({ error: 'Checkout failed.' });
|
|
255
437
|
});
|
|
256
438
|
```
|
|
257
439
|
|
|
258
|
-
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw.
|
|
440
|
+
Recording stores the context alongside the trace, so `Ask` replays with the values the original request saw. It is copied once per run, so read a value that can change while a run is under way, such as a switch an operator can flip to stop a batch, in a Command, whose result the trace records each time.
|
|
259
441
|
|
|
260
442
|
## Retrying Transient Failures
|
|
261
443
|
|
|
262
|
-
`Retry` runs part of a flow again when its I/O fails, meaning a Command's function throws. It does not retry a `Failure` that a step returned: that means the flow decided to stop, so the `Failure` is passed on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency. To reuse the same settings, share an object:
|
|
444
|
+
`Retry` runs part of a flow again when its I/O fails, meaning a Command's function throws. It does not retry a `Failure` that a step returned: that means the flow decided to stop, so the `Failure` is passed on at once, unchanged. Options are given where `Retry` is used, since they describe one dependency. An option set to `undefined` keeps its default, so settings read from configuration can leave a key out. To reuse the same settings, share an object:
|
|
263
445
|
|
|
264
446
|
```js
|
|
265
447
|
const flakyNetwork = { attempts: 3, delay: 200, backoff: 2 };
|
|
@@ -400,7 +582,67 @@ const fetchProfileUncancellable = (userId) => Command(() => fetch(`/users/${user
|
|
|
400
582
|
|
|
401
583
|
Outside a `Parallel`, the function is called with no arguments. A `Retry` inside a cancelled branch stops retrying.
|
|
402
584
|
|
|
403
|
-
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay
|
|
585
|
+
Which branch failed first and cancelled the others depends on timing, so it is recorded with the trace. A replay of a cancelled `Parallel` returns the same failure production did, and stops each other branch where production stopped it, rather than letting whichever branch the replay reaches first decide. The same holds when the branch that cancelled the others was a `next` function or a pure step that threw: the replay throws the same error. If the branch that cancelled the others no longer fails, or the `Parallel` no longer has that branch, the replay ends in a `TimeParadox` naming it.
|
|
586
|
+
|
|
587
|
+
**Undoing what succeeded when a branch fails.** Without `settled`, the failing branch's `Failure` is the whole result, and the values of the branches that succeeded are dropped. When one of them did something that has to be undone, such as charging a card for an order whose stock then ran out, the code that called `runEffect` no longer has the charge to refund. Use `settled: true`, so `next` sees every outcome, and have it return the steps that undo whatever succeeded, followed by the failure:
|
|
588
|
+
|
|
589
|
+
```js
|
|
590
|
+
const reserveStock = (order) => {
|
|
591
|
+
const cmdReserveStock = () => inventory.reserve(order.items);
|
|
592
|
+
return Command(cmdReserveStock, (r) => (r.ok ? Success(r.reservationId) : Failure('out_of_stock')));
|
|
593
|
+
};
|
|
594
|
+
|
|
595
|
+
const chargeOrder = (order) => {
|
|
596
|
+
const cmdChargeOrder = () => payments.charge(order.customerId, order.total);
|
|
597
|
+
return Command(cmdChargeOrder, (c) => (c.ok ? Success(c.chargeId) : Failure('card_declined')));
|
|
598
|
+
};
|
|
599
|
+
|
|
600
|
+
const releaseStock = (reservationId) => {
|
|
601
|
+
const cmdReleaseStock = () => inventory.release(reservationId);
|
|
602
|
+
return Command(cmdReleaseStock);
|
|
603
|
+
};
|
|
604
|
+
|
|
605
|
+
const refundCharge = (chargeId) => {
|
|
606
|
+
const cmdRefundCharge = () => payments.refund(chargeId);
|
|
607
|
+
return Command(cmdRefundCharge);
|
|
608
|
+
};
|
|
609
|
+
|
|
610
|
+
// Passes both results on, or undoes whichever step succeeded and then fails.
|
|
611
|
+
const reserveAndCharge = (order) =>
|
|
612
|
+
Parallel(
|
|
613
|
+
[reserveStock(order), chargeOrder(order)],
|
|
614
|
+
([reservation, charge]) => {
|
|
615
|
+
if (reservation.type === 'Failure') {
|
|
616
|
+
const undo = charge.type === 'Success' ? [refundCharge(charge.value)] : [];
|
|
617
|
+
return Parallel(undo, () => Failure(reservation.error));
|
|
618
|
+
}
|
|
619
|
+
if (charge.type === 'Failure') {
|
|
620
|
+
return Parallel([releaseStock(reservation.value)], () => Failure(charge.error));
|
|
621
|
+
}
|
|
622
|
+
return Success({ order, reservationId: reservation.value, chargeId: charge.value });
|
|
623
|
+
},
|
|
624
|
+
{ settled: true }
|
|
625
|
+
);
|
|
626
|
+
```
|
|
627
|
+
|
|
628
|
+
`next` is a plain function, so each case is tested by handing it the outcomes, with no I/O:
|
|
629
|
+
|
|
630
|
+
```js
|
|
631
|
+
const order = { id: 'order_1', customerId: 'cus_1', items: [], total: 120 };
|
|
632
|
+
|
|
633
|
+
// The card was charged, then the stock ran out: the flow refunds the charge and fails.
|
|
634
|
+
const undoing = reserveAndCharge(order).next([Failure('out_of_stock'), Success('ch_1')]);
|
|
635
|
+
assert.equal(commandName(undoing.effects[0]), 'cmdRefundCharge');
|
|
636
|
+
assert.deepEqual(undoing.next([null]), Failure('out_of_stock'));
|
|
637
|
+
|
|
638
|
+
// Both succeeded, so there is nothing to undo.
|
|
639
|
+
assert.deepEqual(
|
|
640
|
+
reserveAndCharge(order).next([Success('res_1'), Success('ch_1')]),
|
|
641
|
+
Success({ order, reservationId: 'res_1', chargeId: 'ch_1' })
|
|
642
|
+
);
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
A settled `Parallel` cancels nothing, so both steps always run to the end, and the undoing starts once both have finished. The undo steps are part of the flow, so a trace records them and a replay repeats them. An undo step that fails ends the flow with its own error rather than the original one; wrap it in `Retry` if it can fail for a moment.
|
|
404
646
|
|
|
405
647
|
## Composing Larger Flows
|
|
406
648
|
|
|
@@ -423,17 +665,18 @@ const loadCheckout = effectPipe(
|
|
|
423
665
|
);
|
|
424
666
|
```
|
|
425
667
|
|
|
426
|
-
**Join values that depend on each other
|
|
668
|
+
**Join values that depend on each other in a named step.** When step B needs step A's result and step C needs both, write a small step that runs B and passes both forward in one value. Give it a name, and the pipeline reads as a list of steps:
|
|
427
669
|
|
|
428
670
|
```js
|
|
671
|
+
// Fetches the order's customer and passes on both.
|
|
672
|
+
const withCustomer = (order) => effectPipe(fetchCustomer, (customer) => Success({ order, customer }))(order.customerId);
|
|
673
|
+
|
|
429
674
|
const applyLoyaltyDiscount = (orderId) =>
|
|
430
|
-
effectPipe(
|
|
431
|
-
fetchOrder,
|
|
432
|
-
(order) => effectPipe(fetchCustomer, (customer) => Success({ order, customer }))(order.customerId),
|
|
433
|
-
({ order, customer }) => Success(discountedTotal(order, customer))
|
|
434
|
-
)(orderId);
|
|
675
|
+
effectPipe(fetchOrder, withCustomer, ({ order, customer }) => Success(discountedTotal(order, customer)))(orderId);
|
|
435
676
|
```
|
|
436
677
|
|
|
678
|
+
When the lookup is written in place rather than reused, the Command's own `next` does the join, with no pipeline inside the step: `Command(() => db.findCustomer(order.customerId), (customer) => Success({ order, customer }))`.
|
|
679
|
+
|
|
437
680
|
**Pass only what the next steps need.** Avoid one object that collects everything computed so far. When a step's input is all it gets, it cannot depend on a value from far upstream without showing it, stale fields do not linger, and two steps cannot clash over the same key. Data also lives only as long as the flow needs it, which matters for credentials and personal data. In TypeScript, it keeps the pipeline type-checked: if an upstream step changes what it returns, the step that reads the value fails to compile instead of getting `undefined` at runtime.
|
|
438
681
|
|
|
439
682
|
## Which Errors Are Data
|
|
@@ -495,11 +738,14 @@ Retry(
|
|
|
495
738
|
| a Command's function threw | an I/O fault | retries it | the exhaustion, once the attempts are gone |
|
|
496
739
|
| a `next` function or a pure step threw, or the flow is malformed | a bug | nothing; it is thrown, not returned | nothing |
|
|
497
740
|
| an `onStep` hook threw after the Command's function succeeded | a bug | nothing; it is thrown, not returned | nothing |
|
|
741
|
+
| an `onStep` hook returned `undefined` after the Command's function returned a value | a bug | nothing; it is thrown, not returned | nothing |
|
|
498
742
|
|
|
499
743
|
In short, **you can recover from an error your I/O produced, but you cannot catch a `Failure` a step returned.** Only the code that called `runEffect` acts on it. For the same reason, do not throw business errors from a Command's function: a throw there tells `Retry` that the I/O broke, and `Retry` will try again. A throw anywhere else in the flow is treated as a bug: `runEffect` rejects with the thrown error instead of returning a `Failure`.
|
|
500
744
|
|
|
501
745
|
## TypeScript: Typed Errors and Context
|
|
502
746
|
|
|
747
|
+
The bundled declarations need TypeScript 5.1 or later.
|
|
748
|
+
|
|
503
749
|
### Error union across pipeline steps
|
|
504
750
|
|
|
505
751
|
Each step in `effectPipe` carries its own error type. The compiler collects them into a union automatically:
|
|
@@ -517,6 +763,10 @@ if (result.type === 'Failure') {
|
|
|
517
763
|
}
|
|
518
764
|
```
|
|
519
765
|
|
|
766
|
+
The annotations are optional. Without them, the union holds whatever each step can pass to `Failure`, each error kept exactly as written, whether a string or an object, and a step that cannot fail, such as a Command without a `next`, adds nothing. The union covers the `Failure`s your steps return. A Command whose function throws, or that an `onBeforeCommand` hook vetoes, also ends the run with a `Failure` the union does not name, so where the result is handled, give it one fallback, such as a `default` in a `switch` over `result.error`, and treat what reaches it as a server error. One fallback covers the whole flow: catch inside a Command's function only what the flow can handle, and let the rest throw, which is also what lets `Retry` act on it. To check for a thrown error, copy it into a variable typed `unknown` first, as in `const error: unknown = result.error`, since TypeScript refuses `instanceof Error` on an error type made only of strings, or on a flow that declares none.
|
|
767
|
+
|
|
768
|
+
To type what a Command's function returns, such as a JSON response, write the function's result type, as in `Command((): Promise<User | null> => fetchJson(url), next)`. `Command<User | null>(…)` also compiles, but with some type arguments given TypeScript infers none of the others, so that Command's errors are typed `unknown`, which hides the rest of the union.
|
|
769
|
+
|
|
520
770
|
Value types are checked through the pipeline too, so a step that reads a field the previous step does not return is a compile error. This only works while every step uses the value it receives. A step written as `() => doSomething(outer)` ignores it, and the types stop being checked at that point.
|
|
521
771
|
|
|
522
772
|
`effectPipe` is typed for up to 20 steps. A pipeline is itself a step, so a longer one nests, which is usually easier to read anyway: `effectPipe(effectPipe(s1, s2), effectPipe(s3, s4))`.
|
|
@@ -534,7 +784,7 @@ const findProduct = (productId: string): Effect<Product, 'not_found', AppContext
|
|
|
534
784
|
const result = await runEffect(findProduct('abc'), { tenant: 'acme', requestId: '123' });
|
|
535
785
|
```
|
|
536
786
|
|
|
537
|
-
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object.
|
|
787
|
+
Every step's context counts, so a pipeline needs all the contexts its steps read, even when the first step reads none, and a `Parallel` needs every context its branches read. `runEffect` requires a context whenever the flow reads one, since the flow would otherwise get an empty object.
|
|
538
788
|
|
|
539
789
|
## Why Pure Effect
|
|
540
790
|
|
|
@@ -581,11 +831,11 @@ Returns `{ type: 'Failure', error, initialInput }`. Stops the pipeline immediate
|
|
|
581
831
|
|
|
582
832
|
Returns `{ type: 'Command', cmd, next, meta }`.
|
|
583
833
|
|
|
584
|
-
- `cmd`:
|
|
585
|
-
- `next`:
|
|
586
|
-
- `meta`:
|
|
834
|
+
- `cmd`: the function, sync or async, that does the I/O. Inside a `Parallel` branch it gets an `AbortSignal` that fires when a sibling fails, and elsewhere no argument, so wrap a function that takes an optional first argument (see [Limitations](#limitations)).
|
|
835
|
+
- `next`: receives `cmd`'s result and returns the next Effect; by default `(result) => Success(result)`.
|
|
836
|
+
- `meta`: metadata passed to `onBeforeCommand`. A string `meta.name` names the Command.
|
|
587
837
|
|
|
588
|
-
**Every Command needs a name
|
|
838
|
+
**Every Command needs a name**, which tests, traces, replay matching and telemetry all use. It is `meta.name`, else the function's name, else `'anonymous'`:
|
|
589
839
|
|
|
590
840
|
```js
|
|
591
841
|
Command(cmdFn, next, { name: 'chargeCard' }); // 1. meta.name, independent of how cmdFn was written
|
|
@@ -595,7 +845,11 @@ Command(function cmdChargeCard() {
|
|
|
595
845
|
Command(() => api.charge(), next); // 3. neither, so 'anonymous'
|
|
596
846
|
```
|
|
597
847
|
|
|
598
|
-
Use `meta.name` in code that gets minified, since
|
|
848
|
+
Use `meta.name` in code that gets minified, since a minifier renames functions and with them every step in every trace. Otherwise naming the function is enough, as the examples do: an arrow assigned to a `const` first, as in `const cmdChargeCard = () => api.charge()`, takes the `const`'s name. An inline arrow does not, and a replay cannot tell two `'anonymous'` Commands apart, so a refactor that swaps them replays without complaint, each handed the other's recorded result.
|
|
849
|
+
|
|
850
|
+
#### `commandName(command)`
|
|
851
|
+
|
|
852
|
+
Returns a Command's name, chosen in the order above, so a test checks a step by the name its trace would record: `assert.equal(commandName(step), 'cmdFindUser')`. Anything but a Command throws an `EffectTypeError`.
|
|
599
853
|
|
|
600
854
|
#### `Ask(nextFn)`
|
|
601
855
|
|
|
@@ -606,35 +860,36 @@ Returns `{ type: 'Ask', next }`. Passes the `context` from `runEffect` into `nex
|
|
|
606
860
|
Returns `{ type: 'Retry', effect, options, next }`.
|
|
607
861
|
|
|
608
862
|
- `options.attempts`: Max retries, not counting the first try (default: `3`). Must be a positive integer; anything else, including `0`, throws a `TypeError`. To handle an outcome without retrying, branch on it in the Command's `next`, or isolate a failing branch with `Parallel`'s `settled`.
|
|
609
|
-
- `options.delay`: Ms before the first retry (default: `100`).
|
|
610
|
-
- `options.backoff`: Multiplier applied to the delay on each attempt (default: `1`, flat).
|
|
863
|
+
- `options.delay`: Ms before the first retry (default: `100`). Must be a finite number of 0 or more, or it throws a `TypeError`.
|
|
864
|
+
- `options.backoff`: Multiplier applied to the delay on each attempt (default: `1`, flat). Must be a finite number of 0 or more, or it throws a `TypeError`.
|
|
865
|
+
|
|
866
|
+
An option set to `undefined` keeps its default.
|
|
867
|
+
|
|
611
868
|
- `options.onExhausted(error)`: Runs a fallback Effect when every attempt has failed, receiving `{ retryExhausted, lastError, attempts }`. The fallback's success feeds `next`; its failure propagates unwrapped. Per-use only. See [Retrying Transient Failures](#retrying-transient-failures).
|
|
612
869
|
|
|
613
870
|
#### `Parallel(effects, next?, options?)`
|
|
614
871
|
|
|
615
|
-
Returns `{ type: 'Parallel', effects, next, options }
|
|
872
|
+
Returns `{ type: 'Parallel', effects, next, options }`, which runs the branches at the same time and hands `next` their values in order; by default `(values) => Success(values)`. The first branch to fail cancels the others and its `Failure` is the result, with `next` skipped. Which branch that is depends on timing, so it is recorded, and a replay returns the same one. Each branch's Commands get an `AbortSignal` to pass to their I/O; see [Running Effects in Parallel](#running-effects-in-parallel).
|
|
616
873
|
|
|
617
|
-
The second argument
|
|
874
|
+
The second argument is `next` or the options, so `Parallel(effects, { limit: 5 })` works; after the options, a third argument throws.
|
|
618
875
|
|
|
619
|
-
- `limit`: the most branches running at once. Results and recorded paths stay in array order, so
|
|
620
|
-
- `settled`: run every branch to the end and
|
|
876
|
+
- `limit`: the most branches running at once, a positive integer. Results and recorded paths stay in array order, so it changes only the pacing.
|
|
877
|
+
- `settled`: run every branch to the end and hand `next` one `Success` or `Failure` per branch, in array order: no branch cancels the others or fails the `Parallel`. A bug still rejects the run, since a step or `next` that throws, or returns something other than an Effect, is not a branch outcome.
|
|
621
878
|
|
|
622
879
|
### Building pipelines
|
|
623
880
|
|
|
624
881
|
#### `effectPipe(...functions)`
|
|
625
882
|
|
|
626
|
-
Composes
|
|
627
|
-
|
|
628
|
-
A step does not have to use the value it receives. In JavaScript, closing over something from the enclosing scope is fine:
|
|
883
|
+
Composes steps into a pipeline: each step gets the previous step's `Success` value, and a `Failure` from any step stops it. A step may ignore the value it gets and close over the enclosing scope instead, which is fine in JavaScript:
|
|
629
884
|
|
|
630
885
|
```js
|
|
631
886
|
// The last step ignores what came before and uses the enclosing input instead.
|
|
632
887
|
const registerUserFlow = (input) => effectPipe(validateRegistration, () => saveUser(input))(input);
|
|
633
888
|
```
|
|
634
889
|
|
|
635
|
-
In TypeScript,
|
|
890
|
+
In TypeScript, type checking stops at that step, since a function that ignores its parameter constrains nothing before it; passing the value through every step, as the [Quick Start](#quick-start) does, keeps the whole pipeline checked. See [TypeScript: Typed Errors and Context](#typescript-typed-errors-and-context).
|
|
636
891
|
|
|
637
|
-
|
|
892
|
+
In either language, a Command a step builds and does not return never runs, and any `Failure` it would have produced is lost:
|
|
638
893
|
|
|
639
894
|
```js
|
|
640
895
|
(value) => {
|
|
@@ -643,47 +898,44 @@ One shape to avoid in either language:
|
|
|
643
898
|
};
|
|
644
899
|
```
|
|
645
900
|
|
|
646
|
-
|
|
901
|
+
Return the Command, and have its `next` return the value the rest of the pipeline needs.
|
|
647
902
|
|
|
648
903
|
### Running a flow
|
|
649
904
|
|
|
650
905
|
#### `runEffect(effect, context?, callConfig?)`
|
|
651
906
|
|
|
652
|
-
|
|
907
|
+
Runs a flow and resolves with its `Success` or `Failure`.
|
|
653
908
|
|
|
654
|
-
- `context`:
|
|
655
|
-
- `callConfig`:
|
|
656
|
-
- `onRun` fires once per `runEffect` call. Retry attempts run inside that one span.
|
|
909
|
+
- `context`: what `Ask` reads, also passed to `onBeforeCommand`. `context.flowName` names the run in traces and telemetry.
|
|
910
|
+
- `callConfig`: `onStep`, `onRun` and `onBeforeCommand` for this call only, added to the `configureEffect` wiring, or used alone with `inherit: false`; see [`configureEffect`](#configureeffectconfigs). `onRun` wraps the run once, retries included. A `retry` key throws a `TypeError`, since retry options belong to each `Retry`.
|
|
657
911
|
|
|
658
|
-
A
|
|
912
|
+
A flow that is not made of Effects is a bug, so `runEffect` rejects with an `EffectTypeError` that says where, rather than returning a `Failure`:
|
|
659
913
|
|
|
660
914
|
```
|
|
661
915
|
Step 'validateRegistration' returned a plain object. Return Success, Failure, Command, Ask,
|
|
662
916
|
Retry, or Parallel: a plain value has to be wrapped, as in Success(value).
|
|
663
917
|
```
|
|
664
918
|
|
|
665
|
-
|
|
919
|
+
That covers a missing `return`, a `next` that returns a plain value, a step or `next` written as `async` (do the awaited work in a Command instead), and `runEffect(flow)` where `runEffect(flow(input))` was meant. The building blocks check their arguments as the flow is built, before any of its I/O runs, so `Command(db.findUser(email))`, which makes the call while the flow is built, `Command(fn, { name })`, with the metadata in `next`'s place, and a `Parallel` or `effectPipe` given the wrong thing each throw an `EffectTypeError` there. A throw from a Command's function, or from an `onBeforeCommand` veto, is a `Failure`; a throw from a `next` or a pure step is a bug too, and `runEffect` rejects with it as thrown.
|
|
666
920
|
|
|
667
921
|
#### `configureEffect(...configs)`
|
|
668
922
|
|
|
669
|
-
|
|
670
|
-
- `onStep(name, type, op, path)` wraps each Command; must `await op()` and return its result. A hook that calls another hook passes `path` on, since a replay matches steps on it. `op()` returns a promise, even for a synchronous Command. Returning a value _without_ calling `op()` is how replay works. A throw after `op()` succeeded is a bug in the hook: the run rejects, and `Retry` does not run the Command again. A throw without calling `op()` counts as the Command failing.
|
|
671
|
-
- `onStep` also wraps each `Parallel`, with `name` and `type` both `'Parallel'`. Its `op()` runs the branches, so a hook must call it; a hook that returns without calling it makes the run reject with a `TypeError`. It returns which branch, if any, cancelled the others, such as `{ cancelled: true, branch: 0 }`, and it returns even when a branch threw, since the run rejects only after the hook has returned. Telemetry gets one span per `Parallel`, with the spans of its branches' Commands inside it, marked as an error when the `Parallel` was cancelled.
|
|
672
|
-
- `onBeforeCommand(command, context)` fires before each Command; throw to abort. The run returns a `Failure` carrying the thrown error, and `Retry` does not retry it.
|
|
673
|
-
|
|
674
|
-
It only configures hooks. Retry options are passed to `Retry(effect, options)`, and a `retry` key here throws a `TypeError`.
|
|
675
|
-
|
|
676
|
-
Each call adds a layer of hooks on top of those already installed and returns a function that removes that layer. Passing several configurations to one call is the same as calling it once for each, so these two forms are equivalent:
|
|
923
|
+
Adds a layer of hooks for the whole process and returns a function that removes it. Several configurations passed to one call form one layer, the same as installing each on its own. Removing a layer removes only that layer, even when others were installed after it, so a library can add and remove its own hooks without touching the application's. `configureEffect()` with no arguments removes every layer, and a call whose arguments are all `undefined`, such as `configureEffect(flag ? hooks : undefined)`, changes nothing. A `retry` key throws a `TypeError`, since retry options belong to each `Retry`.
|
|
677
924
|
|
|
678
925
|
```js
|
|
679
|
-
configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
configureEffect(telemetryHooks());
|
|
683
|
-
configureEffect(recordingHooks({ sink }));
|
|
926
|
+
const remove = configureEffect(telemetryHooks(), recordingHooks({ sink }));
|
|
927
|
+
// the same as configureEffect(telemetryHooks()) followed by configureEffect(recordingHooks({ sink }))
|
|
928
|
+
remove();
|
|
684
929
|
```
|
|
685
930
|
|
|
686
|
-
|
|
931
|
+
- `onRun(effect, pipeline, flowName)`: wraps the whole run. It must `await pipeline()` and return its result.
|
|
932
|
+
- `onStep(name, type, op, path)`: wraps each Command, and each `Parallel` with `name` and `type` both `'Parallel'`. It must `await op()` and return its result, and pass `path` on to any hook it calls, since a replay matches steps on it.
|
|
933
|
+
- For a Command, `op()` returns a promise, even for a synchronous function. Returning a value without calling `op()` answers for the Command, which is how replay works, and throwing without calling it counts as the Command failing.
|
|
934
|
+
- A throw after `op()` succeeded, or `undefined` returned in place of its value, as a hook that forgot its `return` does, is a bug in the hook: the run rejects, and `Retry` does not run the Command again.
|
|
935
|
+
- For a `Parallel`, `op()` runs the branches, so a hook must call it or the run rejects with a `TypeError`. It returns which branch, if any, cancelled the others, as in `{ cancelled: true, branch: 0 }`, even when a branch threw.
|
|
936
|
+
- `onBeforeCommand(command, context)`: runs before each Command. A throw vetoes the Command: the run returns a `Failure` carrying the thrown error, and `Retry` does not retry it.
|
|
937
|
+
|
|
938
|
+
Layers run in the order they were installed, the first outermost, so a result or a thrown error unwinds from the innermost hook out:
|
|
687
939
|
|
|
688
940
|
```
|
|
689
941
|
runEffect(flow(input))
|
|
@@ -704,15 +956,7 @@ runEffect(flow(input))
|
|
|
704
956
|
├─ telemetry.onRun returns
|
|
705
957
|
```
|
|
706
958
|
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
```js
|
|
710
|
-
const remove = configureEffect(telemetryHooks());
|
|
711
|
-
// ... later
|
|
712
|
-
remove();
|
|
713
|
-
```
|
|
714
|
-
|
|
715
|
-
**Per-call hooks.** By default, the hooks in a `callConfig` are merged with the configured ones by the same rules: configured wrappers go outside, and configured `onBeforeCommand` hooks run first. `inherit: false` leaves the configured hooks out of the run. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
959
|
+
**Per-call hooks.** The hooks in a `callConfig` are merged over the configured ones by the same rules, the configured wrappers outside and the configured `onBeforeCommand` hooks first. `inherit: false` leaves the configured hooks out. With configured hooks `C` and a call `K` that supplies only `onStep`:
|
|
716
960
|
|
|
717
961
|
```
|
|
718
962
|
inherit: true (default) inherit: false
|
|
@@ -722,17 +966,29 @@ onBeforeCommand C.onBeforeCommand (none)
|
|
|
722
966
|
onStep C.onStep( K.onStep( cmd ) ) K.onStep
|
|
723
967
|
```
|
|
724
968
|
|
|
725
|
-
`recordEffect` inherits, so recording inside an application that already
|
|
969
|
+
`recordEffect` inherits, so recording inside an application that already traces still produces spans. `replayEffect` passes `inherit: false` unless `hooks: true`.
|
|
726
970
|
|
|
727
971
|
### Recording and replay
|
|
728
972
|
|
|
729
973
|
#### `recorder(options?)`
|
|
730
974
|
|
|
731
|
-
Returns `{ onStep, entries, toTrace }
|
|
975
|
+
Returns `{ onStep, entries, toTrace }`, a hook that records every step of a run, passed as `runEffect(flow, context, { onStep })`. Give each run its own recorder: one shared by a whole application mixes requests into one trace, which a replay refuses. `recordEffect` and the [recording example](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js) both do this for you.
|
|
976
|
+
|
|
977
|
+
- `onStep`: the hook. Recording never changes a run's outcome; a value `redact` fails on is recorded as `'[redaction failed]'`.
|
|
978
|
+
- `entries`: the steps recorded so far, one per Command and one per `Parallel`. `path` is the step's position in the flow, which a replay matches on; an older entry with `error` and no `threw` still counts as a throw.
|
|
732
979
|
|
|
733
|
-
|
|
980
|
+
```text
|
|
981
|
+
{ command, path, result, durationMs } a Command that returned
|
|
982
|
+
{ command, path, threw: true, error, durationMs } a Command that threw
|
|
983
|
+
{ command: 'Parallel', path, result: { cancelled, branch } } which branch, if any, cancelled the others
|
|
984
|
+
```
|
|
734
985
|
|
|
735
|
-
- `
|
|
986
|
+
- `toTrace(meta)`: the entries as a trace, with `meta`'s `initialInput`, `context`, `flowName` and `version`, copied when it is called (a part that cannot be copied, such as a function, is kept as it is). If a Command can change the input or the context, as an ORM save that adds an id does, call it before the run for those and again afterwards for the entries, as `recordEffect` does.
|
|
987
|
+
- `options.redact(value, name, kind)`: returns what the trace stores in place of `value`. `kind` is `'result'`, `'error'`, `'initialInput'` or `'context'`, and `name` is the Command's name, or the kind. It gets a copy, so changing it in place is safe.
|
|
988
|
+
- `options.maxEntries`: the most entries a trace keeps; the rest are counted in `dropped`, and a replay stops at the first step the trace lacks.
|
|
989
|
+
- `options.stack`: record stack traces for thrown errors (off by default).
|
|
990
|
+
|
|
991
|
+
Results are copied as they are recorded, so a later step cannot change them; see [Recording in Production](#recording-in-production) for what a copy keeps. When writing `redact`, check a value before replacing a field in it, since `{ ...value, email: '[redacted]' }` turns a `null` into an object and the replay takes the other branch:
|
|
736
992
|
|
|
737
993
|
```js
|
|
738
994
|
// Masks a field only where there is one, so a lookup that found nothing still records null.
|
|
@@ -740,7 +996,7 @@ const mask = (value, field) =>
|
|
|
740
996
|
value && typeof value === 'object' && field in value ? { ...value, [field]: '[redacted]' } : value;
|
|
741
997
|
|
|
742
998
|
const redact = (value, name, kind) => {
|
|
743
|
-
if (kind === 'initialInput') return mask(value, '
|
|
999
|
+
if (kind === 'initialInput') return mask(value, 'cardNumber');
|
|
744
1000
|
if (kind === 'context') return mask(value, 'authToken');
|
|
745
1001
|
if (kind === 'error') return mask(value, 'attempted');
|
|
746
1002
|
return name === 'cmdFetchUser' ? mask(value, 'email') : value;
|
|
@@ -751,38 +1007,34 @@ assert.equal(redact(null, 'cmdFetchUser', 'result'), null); // still nothing fou
|
|
|
751
1007
|
assert.equal(redact('card_declined', 'cmdCharge', 'error'), 'card_declined'); // still a string
|
|
752
1008
|
```
|
|
753
1009
|
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
For `'result'` and `'error'`, `name` is the Command's name; for `'initialInput'` and `'context'`, it is the kind. Redacting `initialInput` rarely breaks replay, since Commands are not run; it matters only if a step branches on the removed field. Redacting `context` breaks `Ask` replay if a step reads what you removed.
|
|
757
|
-
|
|
758
|
-
- `options.maxEntries`: Cap trace length; overflow is counted in `dropped`.
|
|
759
|
-
- `options.stack`: Record stack traces for thrown errors (off by default).
|
|
1010
|
+
A replay rebuilds the flow from the redacted input and context, so a field a step checks, such as a password, needs a stand-in the check treats the same way, as the recording example in [Recording in Production](#recording-in-production) does, and a context field an `Ask` reads should stay.
|
|
760
1011
|
|
|
761
1012
|
#### `recordEffect(flowFn, initialInput, options?)`
|
|
762
1013
|
|
|
763
|
-
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts
|
|
1014
|
+
Runs a flow for real while recording, returning `{ result, trace }`. Accepts `recorder` options plus `context` and `version`. For tests and scripts. To record an application without changing call sites, use the wiring in [`examples/recording-example.js`](https://github.com/aycangulez/pure-effect/blob/main/examples/recording-example.js), which gives each run its own recorder.
|
|
764
1015
|
|
|
765
1016
|
#### `replayEffect(effect, traceOrResolver, options?)`
|
|
766
1017
|
|
|
767
|
-
|
|
1018
|
+
Runs a flow with each Command answered from a trace instead of run, and returns `{ result, unreached }`: the flow's outcome, and the recorded steps it never asked for, which is where a flow that stops early shows up. A flow that no longer matches the trace ends in a `Failure` whose `error` is a `TimeParadox`, or a `ReplayError` for a step the trace lacks. Neither is thrown, so a test checks `error.name`, which tells them apart, and not only `result.type`. A malformed trace rejects with a `ReplayError`.
|
|
768
1019
|
|
|
769
|
-
- `traceOrResolver`: a trace
|
|
770
|
-
- `options.context`: context for `Ask
|
|
771
|
-
- `options.onMissing`: `'throw'` (default)
|
|
1020
|
+
- `traceOrResolver`: a trace, its bare entries, or a resolver, a function that answers each step with `{ result }`, `{ error }`, or `undefined` for a step it has no record of, so a trace stored in any shape can be replayed. With a resolver, `unreached` is absent, since a resolver cannot list what it holds. A resolver is also asked about each `Parallel`, with `step.type` set to `'Parallel'`: answering with the recorded cancellation as `{ result }` replays it as production decided, and anything else replays it by timing.
|
|
1021
|
+
- `options.context`: the context for `Ask`, by default the one the trace recorded. A resolver has none, so pass one if the flow reads `Ask`.
|
|
1022
|
+
- `options.onMissing`: `'throw'` (default) stops at a step the trace lacks. `'execute'` runs its Command for real, so use it only where Commands reach test doubles or only read. A trace can lack a step because a hook vetoed it in production, because it was added to the flow since the recording, or because the recorder dropped it under `maxEntries`; a trace that dropped entries refuses `'execute'`, so record the flow again with a higher `maxEntries`. After a flow changes shape, as when a Command is newly wrapped in `Retry`, every step sits at a new position and `'execute'` would run them all live: record the flow again instead.
|
|
772
1023
|
- `options.fastRetry` (default `true`): strip `Retry` delays.
|
|
773
|
-
- `options.hooks` (default `false`): run the replay inside the configured hooks,
|
|
774
|
-
- `options.onResolved(step, outcome)`:
|
|
1024
|
+
- `options.hooks` (default `false`): run the replay inside the configured hooks, a configured recorder included, so they see the replayed steps. Off, a replay cannot reach a telemetry backend or a trace sink.
|
|
1025
|
+
- `options.onResolved(step, outcome)`: observes each replayed step. A throw stops the replay, and `replayEffect` rejects with it; it never changes an outcome.
|
|
775
1026
|
|
|
776
|
-
A trace whose entries
|
|
1027
|
+
A trace whose entries have no `path`, written by hand or recorded before paths existed, is matched by position. That works for a sequential flow, and a `Parallel` step in such a trace is refused with a `ReplayError`, since the order branches finish in cannot tell them apart.
|
|
777
1028
|
|
|
778
1029
|
#### `timeTravel(flowFn, traceLog, options?)`
|
|
779
1030
|
|
|
780
|
-
Replays a trace and narrates each step with its recorded duration, naming any recorded steps that were never reached and warning when the trace's `version` differs from `options.version`. Returns the flow's outcome; for the unreached entries as data, use `replayEffect`. Takes `options.context` to override the trace's, and `options.log` in place of `console.log`.
|
|
1031
|
+
Replays a trace and narrates each step with its recorded duration, naming any recorded steps that were never reached, and warning when the trace's `version` differs from `options.version` or when steps are named `'anonymous'`. Returns the flow's outcome; for the unreached entries as data, use `replayEffect`. Takes `options.context` to override the trace's, and `options.log` in place of `console.log`.
|
|
781
1032
|
|
|
782
1033
|
## Limitations
|
|
783
1034
|
|
|
784
1035
|
- **`Retry` repeats everything it wraps, including a Command's `next`.** When a Command's function throws, every Command inside the `Retry` runs again, including the ones that already succeeded. If a retried Command's `next` continues the flow, everything after it is retried too. Give a retried Command the default `next` and continue in a later pipeline step, or make sure every Command it reaches is safe to run more than once. A `Failure` a step returned is not retried at all, so a function that catches its own error and returns it as a value is never retried: see [Which Errors Are Data](#which-errors-are-data).
|
|
785
1036
|
- **Cancelling a `Parallel` branch cannot stop everything.** A cancelled branch starts no new Commands, and a function that uses the `AbortSignal` it is given can be stopped while running. A function that ignores the signal runs to completion, so a branch whose _first_ Command is a write can still write after another branch has failed. `Parallel` waits for every branch to finish before returning, so no cancelled work is still running after the `Failure` is returned.
|
|
1037
|
+
- **Inside a `Parallel`, a Command's function gets the `AbortSignal` as its first argument.** A function passed by name that has an optional first argument takes the signal for it, so the same Command behaves differently inside a `Parallel` than outside one: `Command(nanoid)` returns an empty ID, since `nanoid(size = 21)` reads the signal as the size, and a repository helper with a default page size gets the signal as its page size. Wrap such a function, as in `Command(() => nanoid())`. In TypeScript, a function whose first parameter cannot be an `AbortSignal` does not compile as a Command.
|
|
786
1038
|
- **A `Failure` carries everything.** It holds the full error and the `initialInput` that `effectPipe` attached, and neither is trimmed, because tests and debugging need both. `redact` keeps sensitive data out of a **trace**. Keeping it out of your **logs** is up to you: log `result.error` rather than the whole `Failure`. The same goes for the outcomes a settled `Parallel` passes to `next`, which can carry their branch's input; for a login or registration flow, that input holds credentials.
|
|
787
1039
|
- **Replay checks the path, not the values.** Replaying an old trace cannot check a fix that changes what a flow computes, because every Command's result comes from the recording: the replay hands the step the value from before the fix, reports `Success`, and flags nothing. It can check a fix that changes which Commands run. A removed step shows up in `unreached` only if it was the last step the flow reached; removed from the middle, it shifts the later paths and the replay stops at a `TimeParadox`.
|
|
788
1040
|
- **A trace keeps data, not objects.** Recorded results come back without their classes: a money object or a database entity as a plain object, a `Buffer` as a plain byte array, and an error inside a result without properties such as `code`. Through JSON, a `Date` also comes back as a string and a `Map` as `{}`. A flow whose `next` calls a method on a result, or branches on one of those properties, replays differently from production. Return plain data from Commands; see [Recording in Production](#recording-in-production).
|