@peter.naydenov/walk-async 6.1.0 → 7.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/Changelog.md +35 -4
- package/Migration.guide.md +179 -25
- package/README.md +501 -311
- package/dist/walk-async.cjs +1 -1
- package/dist/walk-async.esm.mjs +1 -1
- package/dist/walk-async.umd.js +1 -1
- package/package.json +7 -6
- package/src/copyObject.js +145 -0
- package/src/createCallbackArgs.js +15 -0
- package/src/findType.js +28 -0
- package/src/finish.js +38 -0
- package/src/ignore.js +24 -0
- package/src/main.js +230 -285
- package/src/pass.js +36 -0
- package/src/runCallback.js +55 -0
- package/src/setKey.js +12 -0
- package/src/validateForInsertion.js +12 -0
- package/types/copyObject.d.ts +8 -0
- package/types/copyObject.d.ts.map +1 -0
- package/types/createCallbackArgs.d.ts +32 -0
- package/types/createCallbackArgs.d.ts.map +1 -0
- package/types/findType.d.ts +3 -0
- package/types/findType.d.ts.map +1 -0
- package/types/finish.d.ts +25 -0
- package/types/finish.d.ts.map +1 -0
- package/types/ignore.d.ts +10 -0
- package/types/ignore.d.ts.map +1 -0
- package/types/main.d.ts +147 -61
- package/types/main.d.ts.map +1 -1
- package/types/pass.d.ts +23 -0
- package/types/pass.d.ts.map +1 -0
- package/types/runCallback.d.ts +3 -0
- package/types/runCallback.d.ts.map +1 -0
- package/types/setKey.d.ts +3 -0
- package/types/setKey.d.ts.map +1 -0
- package/types/validateForInsertion.d.ts +3 -0
- package/types/validateForInsertion.d.ts.map +1 -0
- package/.agents/skills/git-walk-async/SKILL.md +0 -173
- package/types/src/main.d.ts +0 -35
- package/types/src/main.d.ts.map +0 -1
package/README.md
CHANGED
|
@@ -2,436 +2,626 @@
|
|
|
2
2
|
|
|
3
3
|

|
|
4
4
|

|
|
5
|
+

|
|
5
6
|

|
|
6
|
-

|
|
8
|
+

|
|
7
9
|
|
|
8
|
-
|
|
10
|
+
**Write the rules. Let `walk` handle the nesting.**
|
|
9
11
|
|
|
10
|
-
|
|
11
|
-
- `keyCallback` fires on every primitive property
|
|
12
|
+
Mask values, remove branches, and reshape objects while building a new result. Or inspect the data without creating a copy. `walk` travels through nested objects and arrays for you, so one or two callback functions can replace layers of loops and repeated transformations.
|
|
12
13
|
|
|
13
|
-
|
|
14
|
+
A deep copy is the starting point. What you do during the copy is where it gets interesting:
|
|
14
15
|
|
|
15
16
|
```js
|
|
17
|
+
import walk from '@peter.naydenov/walk-async'
|
|
16
18
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
19
|
+
let data = {
|
|
20
|
+
user : { name:'Peter', email:'peter@example.test', password:'secret' }
|
|
21
|
+
, history : [{ action:'login' }]
|
|
22
|
+
};
|
|
23
|
+
|
|
24
|
+
let result = await walk ({
|
|
25
|
+
data
|
|
26
|
+
, objectCallback : ({ value, key, IGNORE }) => key === 'history' ? IGNORE() : value
|
|
27
|
+
, keyCallback : ({ value, key, IGNORE }) => {
|
|
28
|
+
if ( key === 'password' ) return IGNORE()
|
|
29
|
+
if ( key === 'email' ) return 'hidden'
|
|
30
|
+
return value
|
|
31
|
+
}
|
|
32
|
+
})
|
|
33
|
+
// { user:{ name:'Peter', email:'hidden' } }
|
|
27
34
|
```
|
|
28
35
|
|
|
29
|
-
|
|
30
|
-
> `structuredClone` is built into modern browsers and Node 17+ and is the right tool when you just need a deep copy. Reach for `walk-async` when you want to **modify particular properties, set values conditionally, or skip whole branches** during the copy — all in a single pass over the data — and your callbacks need to do async work. If you don't need callbacks, or you don't need async, `walk-async` is overkill: use `structuredClone` for the simple case, or use the sibling [`@peter.naydenov/walk`](https://github.com/PeterNaydenov/walk) for sync callbacks.
|
|
36
|
+
The rules work wherever those keys appear. The `history` branch is removed before its contents are visited. The source remains unchanged because these callbacks return replacements rather than modifying it.
|
|
31
37
|
|
|
38
|
+
> This README describes the upcoming major release. Callbacks can now return values or promises, with callable `IGNORE()` and `PASS()` helpers shared with the synchronous library. Existing `resolve`/`reject` callbacks remain supported. See the [migration guide](Migration.guide.md) when upgrading.
|
|
32
39
|
|
|
33
|
-
##
|
|
40
|
+
## Choose synchronous or asynchronous Walk
|
|
34
41
|
|
|
35
|
-
|
|
42
|
+
Choose the library according to the work your callbacks do:
|
|
36
43
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
44
|
+
| Callback work | Choose |
|
|
45
|
+
| --- | --- |
|
|
46
|
+
| Immediate transformations, masking, filtering, or collecting values | [`@peter.naydenov/walk`](https://github.com/PeterNaydenov/walk) |
|
|
47
|
+
| Awaiting network requests, database queries, file reads, or other asynchronous operations | [`@peter.naydenov/walk-async`](https://github.com/PeterNaydenov/walk-async) |
|
|
41
48
|
|
|
42
|
-
|
|
49
|
+
The two libraries share traversal rules, settings, paths, `IGNORE()`, `PASS()` and `FINISH()`. Synchronous Walk returns the result directly and does not await callback promises. Walk Async awaits callback results and returns a Promise, so await the walk to obtain the result or finish a deep forEach. Walk Async processes callbacks in traversal order; sibling callbacks do not run concurrently.
|
|
43
50
|
|
|
51
|
+
### Immediate callbacks: measured comparison
|
|
44
52
|
|
|
45
|
-
|
|
46
|
-
`walk-async` visits each member of the data once. `keyCallback` fires on every primitive property — types: *string, number, bigint, boolean, symbol, null, undefined, function*. Object and array values are visited by `objectCallback` instead (see below). Built-in types like `Date`, `Map`, `Set`, and typed arrays arrive at `keyCallback` as primitives and are passed through by reference — see [Built-in types](#built-in-types-date-regexp-map-set-typed-arrays-etc) below.
|
|
53
|
+
An immediately completed callback still has an async cost. In this historical sample recorded on October 3, 2026, before FINISH was added to either implementation, synchronous Walk completed the workloads about **4–9 times faster** with default settings. Median times in milliseconds:
|
|
47
54
|
|
|
48
|
-
|
|
55
|
+
| Shape | Sync | Async: plain return | Async: async callback | Async: immediate resolve(value) |
|
|
56
|
+
| --- | --- | --- | --- | --- |
|
|
57
|
+
| 1,000,000-key flat object | 317.64 | 1359.22 | 1332.44 | 1497.33 |
|
|
58
|
+
| 100,000 records with nested metadata and tags | 136.58 | 1065.70 | 1194.75 | 1224.21 |
|
|
59
|
+
| 4,000-level child chain | 8.83 | 75.77 | 74.93 | 69.07 |
|
|
49
60
|
|
|
50
|
-
|
|
51
|
-
- **Resolve a plain object or array** → walk continues into it with `objectCallback` and `keyCallback` applied to its children (same as any original nested object/array). The new walk is deferred via the `extend` mechanism, so the iteration order of the current level is preserved;
|
|
52
|
-
- **`reject()`** → that key is dropped from the result.
|
|
61
|
+
Both callbacks simply passed through `value`. The async variants returned it directly, used an `async` callback returning it, or called `resolve(value)` immediately. No callback waited for I/O or a timer. Copying, breadcrumbs, parent paths, and circular detection were enabled. Measurements used Node v26.10.0 on macOS arm64, one warmup per combination, and five samples with rotating execution order on the same machine.
|
|
53
62
|
|
|
54
|
-
|
|
55
|
-
function keyCallbackFn ({ value, key, breadcrumbs, resolve, reject }) {
|
|
56
|
-
// value: value of the property;
|
|
57
|
-
// key: key of the property;
|
|
58
|
-
// breadcrumbs: location of the property;
|
|
59
|
-
// resolve: function. Call it with the new value to store at that key;
|
|
60
|
-
// reject: function. Call it to drop the key from the result;
|
|
61
|
-
// Important: key callback should be resolved or rejected on every code path,
|
|
62
|
-
// otherwise the walk promise never settles. See section "Timeout".
|
|
63
|
-
}
|
|
64
|
-
|
|
65
|
-
const result = await walk ({ data, keyCallback : keyCallbackFn }); // It's the short way to provide only key-callback. Callback functions are optional.
|
|
66
|
-
// walk ({ data, keyCallback, objectCallback }); // If both callbacks are available
|
|
67
|
-
```
|
|
63
|
+
The comparison also covered disabled paths and traversal without copying; synchronous Walk was faster in every tested combination. The exact difference depends on the data, settings, runtime, and callback work. The 4–9 times range describes these default-setting samples, rather than a guarantee for every application. Choose sync for work that completes synchronously, and async when callbacks need to await operations.
|
|
68
64
|
|
|
65
|
+
The [benchmark script](https://github.com/PeterNaydenov/walk-async/blob/main/benchmarks/compareSpeed.mjs) and [complete samples](https://github.com/PeterNaydenov/walk-async/blob/main/benchmarks/compareSpeed.results.json) are kept in the Walk Async repository. The saved samples do not contain source hashes; rerunning the script measures the current checked-out sources. To run the comparison, check out both repositories beside each other and run `node benchmarks/compareSpeed.mjs` from Walk Async.
|
|
69
66
|
|
|
70
|
-
##
|
|
67
|
+
## Installation
|
|
71
68
|
|
|
72
|
-
|
|
69
|
+
```sh
|
|
70
|
+
npm install @peter.naydenov/walk-async
|
|
71
|
+
```
|
|
73
72
|
|
|
74
|
-
|
|
75
|
-
- **Resolve a primitive** (string, number, `null`, etc.) → it is stored at that key as-is, and walk does not descend into it (primitives have no children to walk);
|
|
76
|
-
- **`reject()`** → that key is dropped from the result.
|
|
73
|
+
Use an ES module import:
|
|
77
74
|
|
|
78
75
|
```js
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
// otherwise the walk promise never settles. See section "Timeout".
|
|
87
|
-
}
|
|
88
|
-
|
|
89
|
-
walk ({
|
|
90
|
-
data
|
|
91
|
-
, keyCallback: keyCallbackFn
|
|
92
|
-
, objectCallback : objectCallbackFn
|
|
93
|
-
})
|
|
94
|
-
.then ( resultOfWalk => {
|
|
95
|
-
// do something with the result of walk
|
|
96
|
-
})
|
|
76
|
+
import walk from '@peter.naydenov/walk-async'
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Or CommonJS:
|
|
80
|
+
|
|
81
|
+
```js
|
|
82
|
+
const walk = require ( '@peter.naydenov/walk-async' )
|
|
97
83
|
```
|
|
98
84
|
|
|
99
|
-
|
|
85
|
+
## One walk, one or two callbacks
|
|
100
86
|
|
|
101
|
-
Skip key-callbacks by not providing a keyCallback function:
|
|
102
87
|
```js
|
|
103
|
-
|
|
88
|
+
let result = await walk ({
|
|
89
|
+
data
|
|
90
|
+
, keyCallback
|
|
91
|
+
, objectCallback
|
|
92
|
+
, settings
|
|
93
|
+
}, ...args )
|
|
104
94
|
```
|
|
105
95
|
|
|
96
|
+
Only `data` is required. The walk returns a Promise; await it to get the result or finish a deep forEach. Both callbacks may be async, and their returned results are awaited before the next callback starts. Choose the callbacks your task needs:
|
|
97
|
+
|
|
98
|
+
| Callback | Receives | Use it to |
|
|
99
|
+
| --- | --- | --- |
|
|
100
|
+
| `keyCallback` | Leaf values: primitives, functions, and supported built-in values | Change, mask, collect, or remove individual values |
|
|
101
|
+
| `objectCallback` | Objects and arrays, including the root | Reshape containers, remove branches, or skip their immediate key callbacks |
|
|
102
|
+
|
|
103
|
+
Both callbacks receive an arguments object. Async-only `resolve` and `reject` fields support existing explicit-settlement callbacks; return-based callbacks can leave them unused. Extra arguments passed after the options are forwarded to both callbacks in the same order.
|
|
104
|
+
|
|
105
|
+
| Argument | Meaning |
|
|
106
|
+
| --- | --- |
|
|
107
|
+
| `value` | The current value |
|
|
108
|
+
| `key` | The property's name as a string; `'root'` for the root object callback |
|
|
109
|
+
| `breadcrumbs` | A string such as `'root/profile/age'` |
|
|
110
|
+
| `parentPath` | A read-only array such as `['root','profile']`, excluding the current key |
|
|
111
|
+
| `IGNORE` | Call and return `IGNORE()` to remove the key or branch |
|
|
112
|
+
| `FINISH` | Available to both callbacks; return `FINISH()` or `FINISH(value)` to finish the walk |
|
|
113
|
+
| `isFinished` | Available to `keyCallback`; `true` while processing a final branch selected by `objectCallback` |
|
|
114
|
+
| `PASS` | Available to `objectCallback`; return `PASS()` or `PASS(value)` to skip immediate key callbacks |
|
|
106
115
|
|
|
107
|
-
|
|
116
|
+
When copying, return the value you want in the result. Return `value` to keep it unchanged. Returning `null` or `undefined` stores that value; it does not remove the property. `IGNORE()` removes it while continuing the walk; `FINISH()` omits it and stops. With `settings.copy:false`, returns still control which structures get visited, but no result is built.
|
|
108
117
|
|
|
109
|
-
|
|
118
|
+
## Agent skill
|
|
110
119
|
|
|
111
|
-
|
|
120
|
+
The [Walk Async skill](skills/walk-async/SKILL.md) helps coding agents use the
|
|
121
|
+
upcoming version 7 API. It provides a concise entrypoint and linked callback,
|
|
122
|
+
control-flow, settings, and runnable-example references, loaded as needed.
|
|
123
|
+
Install or read the complete `skills/walk-async/` folder with its references.
|
|
124
|
+
It is portable across agents; discovery depends on the agent's integration.
|
|
125
|
+
See [skill integration and maintenance](skills/README.md).
|
|
112
126
|
|
|
113
|
-
|
|
127
|
+
The repository includes the skill. The current npm `files` whitelist still
|
|
128
|
+
excludes its new folder; [the integration notes](skills/README.md#package-integration)
|
|
129
|
+
record the manual replacement of `".agents/skills"` with `"skills"` needed
|
|
130
|
+
before the skill and its integration README can ship in npm.
|
|
114
131
|
|
|
115
|
-
|
|
132
|
+
## Await work during the walk
|
|
133
|
+
|
|
134
|
+
Use the same rules as the synchronous library, with an async callback when a value needs I/O:
|
|
116
135
|
|
|
117
136
|
```js
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
data: user
|
|
129
|
-
, keyCallback: omitKeys ( 'password', 'token' )
|
|
130
|
-
})
|
|
137
|
+
let data = { first:'https://example.test/first', nested:{ second:'https://example.test/second' } };
|
|
138
|
+
|
|
139
|
+
let result = await walk ({
|
|
140
|
+
data
|
|
141
|
+
, keyCallback : async ({ value }) => {
|
|
142
|
+
if ( typeof value !== 'string' || !value.startsWith ( 'https://' ) ) return value
|
|
143
|
+
const response = await fetch ( value );
|
|
144
|
+
return await response.json ()
|
|
145
|
+
}
|
|
146
|
+
})
|
|
131
147
|
```
|
|
132
148
|
|
|
149
|
+
The parsed objects are visited like any other replacement containers. Thrown errors and rejected promises propagate to the caller. To remove a failed value instead, catch the error in the callback and return `IGNORE()`.
|
|
133
150
|
|
|
134
|
-
|
|
151
|
+
The synchronous sibling uses the same `settings`, paths, `IGNORE()` and `PASS()` rules. Moving a return-based call here means changing the import to `@peter.naydenov/walk-async` and awaiting the walk. Choose the synchronous [walk](https://github.com/PeterNaydenov/walk) when callbacks do not need async work.
|
|
135
152
|
|
|
136
|
-
|
|
153
|
+
### Existing resolve/reject callbacks
|
|
137
154
|
|
|
155
|
+
Callbacks from v6 can keep explicitly settling their value:
|
|
156
|
+
|
|
157
|
+
```js
|
|
158
|
+
let result = await walk ({
|
|
159
|
+
data : { name:'Peter', password:'secret' }
|
|
160
|
+
, keyCallback : ({ value, key, resolve, reject }) => {
|
|
161
|
+
if ( key === 'password' ) reject ()
|
|
162
|
+
else resolve ( value )
|
|
163
|
+
}
|
|
164
|
+
})
|
|
165
|
+
// { name:'Peter' }
|
|
138
166
|
```
|
|
139
|
-
|
|
167
|
+
|
|
168
|
+
Reading or destructuring `resolve` or `reject` selects explicit settlement for that callback invocation. Call one of them on every path; the returned value is then ignored, but an async callback's completion is still awaited. Destructure only the fields you use: spreading the entire arguments object also reads these fields and selects explicit settlement. Return-based callbacks may return `undefined` normally, including side-effect callbacks with `copy:false`.
|
|
169
|
+
|
|
170
|
+
`resolve(value)` also awaits a supplied promise. It accepts `IGNORE()` and `PASS(...)` instructions under the same rules as callback returns. `reject()` removes a key or branch, including the root; its optional reason is ignored. To fail the walk, throw or return a rejected promise instead.
|
|
171
|
+
|
|
172
|
+
### Timeout
|
|
173
|
+
|
|
174
|
+
The optional `timeout` is a duration in milliseconds for the whole walk. It catches unresolved returned promises and explicit callbacks that forget to settle:
|
|
175
|
+
|
|
176
|
+
```js
|
|
177
|
+
await walk ({
|
|
178
|
+
data : { number:1 }
|
|
179
|
+
, keyCallback : ({ resolve }) => { /* forgotten settlement */ }
|
|
180
|
+
, timeout : 1000
|
|
181
|
+
})
|
|
182
|
+
// Rejects: walk-async: timed out after 1000ms; callbacks still pending:
|
|
183
|
+
// - keyCallback at 'root/number'
|
|
140
184
|
```
|
|
141
185
|
|
|
142
|
-
|
|
186
|
+
When breadcrumbs are disabled, diagnostics name the pending key instead. A timeout rejects the caller's promise; it does not cancel callback I/O or undo side effects. The timer is cleared when the walk settles.
|
|
187
|
+
|
|
188
|
+
## Transform while copying
|
|
189
|
+
|
|
190
|
+
Change a leaf in `keyCallback`, or return a replacement object from `objectCallback`. The returned structure becomes the input for the next part of the walk.
|
|
191
|
+
|
|
143
192
|
```js
|
|
144
|
-
let
|
|
193
|
+
let data = {
|
|
194
|
+
profile : { name:'Peter', active:true, internalCode:42 }
|
|
195
|
+
, visits : 3
|
|
196
|
+
};
|
|
197
|
+
|
|
198
|
+
let result = await walk ({
|
|
199
|
+
data
|
|
200
|
+
, objectCallback : ({ value, key }) => {
|
|
201
|
+
if ( key === 'profile' ) return { name:value.name, active:value.active }
|
|
202
|
+
return value
|
|
203
|
+
}
|
|
204
|
+
, keyCallback : ({ value, key }) => key === 'name' ? value.toUpperCase() : value
|
|
205
|
+
})
|
|
206
|
+
// { profile:{ name:'PETER', active:true }, visits:3 }
|
|
145
207
|
```
|
|
146
208
|
|
|
147
|
-
or
|
|
209
|
+
A plain object or array returned by either callback is walked into and copied. When `keyCallback` returns a container, its children receive the normal callbacks; `objectCallback` is not called again for that replacement container itself.
|
|
210
|
+
|
|
211
|
+
A simple value returned by `objectCallback` is passed to `keyCallback` if key callbacks are active for its parent. A simple replacement returned from the root object callback is returned directly.
|
|
212
|
+
|
|
213
|
+
For an ordinary copy, leave out both callbacks:
|
|
148
214
|
|
|
149
215
|
```js
|
|
150
|
-
|
|
216
|
+
let data = { profile:{ name:'Peter' }, scores:[1,2,3] };
|
|
217
|
+
let result = await walk ({ data })
|
|
218
|
+
|
|
219
|
+
result !== data // true
|
|
220
|
+
result.profile !== data.profile // true
|
|
221
|
+
result.scores !== data.scores // true
|
|
151
222
|
```
|
|
152
223
|
|
|
153
|
-
|
|
224
|
+
Objects and arrays get new containers. Supported built-in values and functions keep their original references; see [What gets copied](#what-gets-copied).
|
|
154
225
|
|
|
155
|
-
|
|
156
|
-
Library is using 'generator functions'. If support for old browsers
|
|
157
|
-
is required, add a polyfill for 'generators'.
|
|
226
|
+
A leaf value passed as root `data` resolves directly when copying; neither callback runs for that root. With `settings.copy:false`, the walk resolves `undefined` for any root.
|
|
158
227
|
|
|
228
|
+
## Choose how much data to visit
|
|
159
229
|
|
|
230
|
+
You control which work happens during traversal:
|
|
160
231
|
|
|
232
|
+
| Return from `objectCallback` | Effect |
|
|
233
|
+
| --- | --- |
|
|
234
|
+
| `value` | Continue into this object or array normally |
|
|
235
|
+
| `IGNORE()` | Remove the entire branch and stop visiting its contents |
|
|
236
|
+
| `PASS()` | Keep the current value and skip key callbacks on its immediate properties |
|
|
237
|
+
| `PASS(replacement)` | Use a replacement with the same rule as `PASS()` |
|
|
161
238
|
|
|
162
|
-
|
|
239
|
+
### Skip a branch
|
|
240
|
+
|
|
241
|
+
Return `IGNORE()` before entering an object or array. None of its descendants will reach either callback.
|
|
163
242
|
|
|
164
|
-
### Deep copy
|
|
165
243
|
```js
|
|
166
|
-
|
|
244
|
+
let data = { name:'Peter', archive:{ events:[1,2,3], details:{ count:3 } } };
|
|
245
|
+
|
|
246
|
+
let result = await walk ({
|
|
247
|
+
data
|
|
248
|
+
, objectCallback : ({ value, key, IGNORE }) => key === 'archive' ? IGNORE() : value
|
|
249
|
+
})
|
|
250
|
+
// { name:'Peter' }
|
|
167
251
|
```
|
|
168
252
|
|
|
253
|
+
From `keyCallback`, `IGNORE()` removes just that leaf. From the root object callback, it resolves an empty object or array matching the original root type; with copying disabled it resolves `undefined`.
|
|
254
|
+
|
|
255
|
+
### Pass immediate values through
|
|
169
256
|
|
|
170
|
-
|
|
171
|
-
`walk-async` operates on the own-enumerable-string-key model. Values whose data lives outside that model are **preserved by reference** — the same instance appears in the result:
|
|
257
|
+
Return `PASS()` when an object's immediate values should avoid `keyCallback`. Nested objects and arrays still run `objectCallback`, and their own values run `keyCallback` normally.
|
|
172
258
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
259
|
+
```js
|
|
260
|
+
let data = { branch:{ count:2, nested:{ count:3 } }, outside:1 };
|
|
261
|
+
|
|
262
|
+
let result = await walk ({
|
|
263
|
+
data
|
|
264
|
+
, objectCallback : ({ value, key, PASS }) => key === 'branch' ? PASS() : value
|
|
265
|
+
, keyCallback : ({ value }) => value * 2
|
|
266
|
+
})
|
|
267
|
+
// { branch:{ count:2, nested:{ count:6 } }, outside:2 }
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
To change the object and skip its immediate key callbacks, provide a replacement:
|
|
180
271
|
|
|
181
272
|
```js
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
273
|
+
let result = await walk ({
|
|
274
|
+
data : { branch:{ count:2, nested:{ count:3 } } }
|
|
275
|
+
, objectCallback : ({ value, key, PASS }) => key === 'branch' ? PASS ({ ...value, count:100 }) : value
|
|
276
|
+
, keyCallback : ({ value }) => value * 2
|
|
277
|
+
})
|
|
278
|
+
// { branch:{ count:100, nested:{ count:6 } } }
|
|
186
279
|
```
|
|
187
280
|
|
|
188
|
-
|
|
281
|
+
`PASS()` keeps the current value; `PASS(undefined)` explicitly replaces it with `undefined`. Simple replacements, including supported built-in values, are stored directly without `keyCallback`. For arrays, the rule covers immediate leaf elements and additional leaf properties. Root `PASS()` follows the same rule.
|
|
282
|
+
|
|
283
|
+
Promises supplied to `PASS(replacement)` are awaited before classifying the replacement or checking circular references. A rejection propagates to the walk promise.
|
|
189
284
|
|
|
285
|
+
Call the helpers and return their result. Returning the `IGNORE` or `PASS` function itself stores a function value. The internal instructions returned by these helpers do not appear in the result.
|
|
190
286
|
|
|
191
|
-
|
|
192
|
-
|
|
287
|
+
## Deep forEach
|
|
288
|
+
|
|
289
|
+
Use a callback to collect values, count entries, validate data, or log locations throughout a nested structure. Set `settings.copy:false` to run the walk without building a copied structure. You write the operation once; `walk` finds the matching values at every depth.
|
|
193
290
|
|
|
194
291
|
```js
|
|
195
|
-
let
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
walk ({ data:x, keyCallback: keyFn })
|
|
214
|
-
.then ( result => {
|
|
215
|
-
// result is a deep copy of x
|
|
216
|
-
})
|
|
292
|
+
let data = {
|
|
293
|
+
scores : [10,20]
|
|
294
|
+
, profile : { score:30 }
|
|
295
|
+
, archive : { scores:[100,200] }
|
|
296
|
+
};
|
|
297
|
+
let total = 0;
|
|
298
|
+
|
|
299
|
+
await walk ({
|
|
300
|
+
data
|
|
301
|
+
, settings : { copy:false }
|
|
302
|
+
, objectCallback : ({ value, key, IGNORE }) => key === 'archive' ? IGNORE() : value
|
|
303
|
+
, keyCallback : ({ value }) => {
|
|
304
|
+
if ( typeof value === 'number' ) total += value
|
|
305
|
+
return value
|
|
306
|
+
}
|
|
307
|
+
})
|
|
308
|
+
// total === 60
|
|
217
309
|
```
|
|
218
310
|
|
|
219
|
-
|
|
311
|
+
This is a deep `forEach` pattern with control over traversal. `IGNORE()` avoids visiting unwanted branches; `PASS()` avoids immediate key callbacks while continuing into nested containers. That can reduce the work compared with processing every value.
|
|
220
312
|
|
|
221
|
-
|
|
313
|
+
With copying disabled, `walk` resolves `undefined`. It does not allocate result objects or arrays or insert copied properties. Paths and traversal bookkeeping are still prepared as needed.
|
|
222
314
|
|
|
315
|
+
Callback rules stay the same: `IGNORE()` stops a branch, `PASS()` skips immediate key callbacks, and returned containers are still visited. `objectCallback` should return `value` to continue into an unchanged container; omitting its return stops traversal at that location. A `keyCallback` used only for side effects may omit its return. Return `value` if the same callback will also be used while copying.
|
|
223
316
|
|
|
224
|
-
|
|
225
|
-
|
|
317
|
+
Returning a replacement changes what is visited without assigning that replacement to the source. Walk does not modify the input by itself; callbacks can still modify objects they receive. Omit `copy:false` to create a result as usual.
|
|
318
|
+
|
|
319
|
+
## Finish when the job is done
|
|
320
|
+
|
|
321
|
+
Both callbacks receive `FINISH`. Return its instruction or pass it to `resolve`, just like `IGNORE()` and `PASS()`. The two callbacks finish at different points:
|
|
322
|
+
|
|
323
|
+
| Callback and return | Effect |
|
|
324
|
+
| --- | --- |
|
|
325
|
+
| Either callback: `FINISH()` | Omit the current value or branch, then stop immediately |
|
|
326
|
+
| `keyCallback`: `FINISH(value)` | Include the supplied value directly, then stop immediately |
|
|
327
|
+
| `objectCallback`: `FINISH(value)` | Select this final branch, run its key callbacks with `isFinished:true`, then stop |
|
|
328
|
+
|
|
329
|
+
Later siblings and unrelated pending containers are not visited. In the selected final branch, nested objects and arrays are traversed for their leaf values, without further object callbacks. `isFinished` is `false` during ordinary key callbacks, and `true` for key callbacks processing that final branch, including replacement containers they return.
|
|
226
330
|
|
|
227
331
|
```js
|
|
228
|
-
let
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
// Drop a single primitive (use keyCallback)
|
|
239
|
-
const r1 = await walk ({
|
|
240
|
-
data: x
|
|
241
|
-
, keyCallback: ({ key, value, resolve, reject }) => {
|
|
242
|
-
if ( key === 'password' ) reject ()
|
|
243
|
-
else resolve ( value )
|
|
244
|
-
}
|
|
245
|
-
})
|
|
246
|
-
// r1.metadata is still fully present; only r1.password is gone.
|
|
247
|
-
|
|
248
|
-
// Drop an entire subtree (use objectCallback)
|
|
249
|
-
const r2 = await walk ({
|
|
250
|
-
data: x
|
|
251
|
-
, objectCallback: ({ key, value, resolve, reject }) => {
|
|
252
|
-
if ( key === 'metadata' ) reject ()
|
|
253
|
-
else resolve ( value )
|
|
254
|
-
}
|
|
255
|
-
})
|
|
256
|
-
// r2.password is still present; the whole r2.metadata subtree is gone.
|
|
332
|
+
let data = [12,22,33,44,55,66];
|
|
333
|
+
|
|
334
|
+
let result = await walk ({
|
|
335
|
+
data
|
|
336
|
+
, keyCallback : ({ value, FINISH }) => {
|
|
337
|
+
if ( value === 33 ) return FINISH ( value )
|
|
338
|
+
return value
|
|
339
|
+
}
|
|
340
|
+
})
|
|
341
|
+
// [12,22,33]
|
|
257
342
|
```
|
|
258
343
|
|
|
259
|
-
|
|
344
|
+
Change the stopping return to `FINISH()` to get `[12,22]`. An explicit argument counts even when its value is `undefined`: `FINISH(undefined)` includes that value, while `FINISH()` omits it.
|
|
260
345
|
|
|
346
|
+
An object callback can select a final branch:
|
|
347
|
+
|
|
348
|
+
```js
|
|
349
|
+
let data = { a:12, b:15, c:{ internal:44, in1:'bbb', in2:'ccc' }, after:55 };
|
|
350
|
+
|
|
351
|
+
let result = await walk ({
|
|
352
|
+
data
|
|
353
|
+
, objectCallback : ({ key, value, FINISH }) => key === 'c' ? FINISH ( value ) : value
|
|
354
|
+
})
|
|
355
|
+
// { a:12, b:15, c:{ internal:44, in1:'bbb', in2:'ccc' } }
|
|
356
|
+
```
|
|
261
357
|
|
|
262
|
-
|
|
358
|
+
This example has no key callback, so the supplied branch is included directly: `result.c === data.c`. With `FINISH()` instead, the result is `{ a:12, b:15 }`.
|
|
359
|
+
|
|
360
|
+
When a key callback exists, it controls what gets copied from that final branch:
|
|
263
361
|
|
|
264
362
|
```js
|
|
265
|
-
let
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
};
|
|
275
|
-
function keyFn ({value,key,resolve,reject}) {
|
|
276
|
-
if ( key === 'name' ) reject ()
|
|
277
|
-
else resolve ( value )
|
|
278
|
-
}
|
|
279
|
-
|
|
280
|
-
walk ({
|
|
281
|
-
data : x
|
|
282
|
-
, keyCallback : keyFn
|
|
283
|
-
})
|
|
284
|
-
.then ( result => {
|
|
285
|
-
// result will copy all properties from x without the property 'name'.
|
|
286
|
-
// result.name === undefined
|
|
287
|
-
})
|
|
363
|
+
let result = await walk ({
|
|
364
|
+
data : { a:12, b:15, c:{ internal:44, in1:'bbb', in2:'ccc' }, after:55 }
|
|
365
|
+
, objectCallback : ({ key, value, FINISH }) => key === 'c' ? FINISH ( value ) : value
|
|
366
|
+
, keyCallback : ({ value, isFinished, FINISH }) => {
|
|
367
|
+
if ( isFinished ) return FINISH ( value )
|
|
368
|
+
return value
|
|
369
|
+
}
|
|
370
|
+
})
|
|
371
|
+
// { a:12, b:15, c:{ internal:44 } }
|
|
288
372
|
```
|
|
289
373
|
|
|
374
|
+
Choose a different return while `isFinished` is true to change the final branch:
|
|
375
|
+
|
|
376
|
+
| Return from the final key callbacks | Result inside `c` in this example |
|
|
377
|
+
| --- | --- |
|
|
378
|
+
| `value` | `{ internal:44, in1:'bbb', in2:'ccc' }` |
|
|
379
|
+
| `IGNORE()` | `{}` |
|
|
380
|
+
| `FINISH(value)` on the first leaf | `{ internal:44 }` |
|
|
381
|
+
| `FINISH()` on the first leaf | `{}` |
|
|
382
|
+
|
|
383
|
+
These examples use a flat final object. For nested data, ignoring every leaf preserves its containers, possibly empty. Stopping at the first leaf can leave containers already allocated before it. Values are visited in `Object.keys` order; choosing the first property depends on that order.
|
|
384
|
+
|
|
385
|
+
`FINISH(value)` returned from a key callback includes the supplied value directly, including object or array references, without visiting its contents. From an object callback with no key callback, it also includes the value directly. With a key callback, the final branch is copied and transformed using the normal leaf return rules and cycle detection. `PASS()` on an earlier container does not suppress key callbacks in a newly selected final branch. Returning the `FINISH` function without calling it stores an ordinary function value, just like the other helpers.
|
|
290
386
|
|
|
291
|
-
|
|
387
|
+
The result contains work already completed, without completing unrelated pending containers. For example, returning `FINISH()` from the key callback at `stop` in `{ queued:{ number:1 }, stop:2 }` leaves `{ queued:{} }`: the container was allocated before its contents were visited. Callback order stays the same within the selected work. On the root object callback, `FINISH()` resolves an empty object or array matching the input root; `FINISH(value)` follows the same final-branch rule. A simple root replacement is resolved directly without a key callback, as with ordinary root replacements.
|
|
388
|
+
|
|
389
|
+
With `settings.copy:false`, either form stops traversal and walk still resolves `undefined`. Collect a match outside the callback when searching:
|
|
292
390
|
|
|
293
391
|
```js
|
|
294
|
-
let
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
, keyCallback : ({resolve}) => resolve('xxx')
|
|
307
|
-
})
|
|
308
|
-
.then ( result => {
|
|
309
|
-
// 'result' will have the same structure as 'x' but all values are 'xxx'
|
|
310
|
-
// {
|
|
311
|
-
// ls : [ 'xxx','xxx','xxx' ]
|
|
312
|
-
// , name : 'xxx'
|
|
313
|
-
// , props : {
|
|
314
|
-
// eyeColor: 'xxx'
|
|
315
|
-
// , age : 'xxx'
|
|
316
|
-
// , height : 'xxx'
|
|
317
|
-
// , sizes : ['xxx','xxx','xxx','xxx']
|
|
318
|
-
// }
|
|
319
|
-
// }
|
|
392
|
+
let found;
|
|
393
|
+
|
|
394
|
+
await walk ({
|
|
395
|
+
data : [{ id:1 },{ id:2 },{ id:3 }]
|
|
396
|
+
, settings : { copy:false }
|
|
397
|
+
, objectCallback : ({ value, FINISH }) => {
|
|
398
|
+
if ( value.id === 2 ) {
|
|
399
|
+
found = value
|
|
400
|
+
return FINISH()
|
|
401
|
+
}
|
|
402
|
+
return value
|
|
403
|
+
}
|
|
320
404
|
})
|
|
405
|
+
// found: { id:2 }
|
|
321
406
|
```
|
|
322
407
|
|
|
323
|
-
|
|
408
|
+
Walk Async awaits callback completion and promises supplied to `FINISH(value)` before interpreting the instruction. Rejected payloads reject the walk; unresolved payloads remain in timeout diagnostics. For explicit settlement, use `resolve ( FINISH ( value ) )` or `resolve ( FINISH() )`. Calling the helper without returning or resolving its instruction has no effect.
|
|
409
|
+
|
|
410
|
+
Stopping belongs to that walk only. Finishing a nested `walk` call does not finish the outer call.
|
|
411
|
+
|
|
412
|
+
## Paths when you need them
|
|
413
|
+
|
|
414
|
+
Both callbacks receive `breadcrumbs` and `parentPath` by default:
|
|
415
|
+
|
|
416
|
+
| Location | `key` | `parentPath` | `breadcrumbs` |
|
|
417
|
+
| --- | --- | --- | --- |
|
|
418
|
+
| Root object callback | `'root'` | `[]` | `'root'` |
|
|
419
|
+
| `data.profile.age` | `'age'` | `['root','profile']` | `'root/profile/age'` |
|
|
420
|
+
| `data.items[0].name` | `'name'` | `['root','items','0']` | `'root/items/0/name'` |
|
|
421
|
+
|
|
422
|
+
Build a full path with `[ ...parentPath, key ]`. Parent arrays are frozen and shared by siblings; create a new array when extending one.
|
|
423
|
+
|
|
424
|
+
Use the array when property names contain `/`: `data['a/b'].c` has parent path `['root','a/b']`, while `data.a.b.c` has `['root','a','b']`. Both locations have the breadcrumb string `'root/a/b/c'`, so splitting breadcrumbs cannot reliably recover a path.
|
|
425
|
+
|
|
426
|
+
Array keys refer to input locations, even when filtering changes the indexes in the result.
|
|
427
|
+
|
|
428
|
+
### Settings
|
|
429
|
+
|
|
430
|
+
Disable paths your callbacks do not use:
|
|
324
431
|
|
|
325
432
|
```js
|
|
326
|
-
let
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
, height : 176
|
|
333
|
-
, sizes : [12,33,12,21]
|
|
334
|
-
}
|
|
335
|
-
};
|
|
336
|
-
|
|
337
|
-
function objectCallback ({ value:obj, key, resolve }) {
|
|
338
|
-
const {age, height} = obj;
|
|
339
|
-
if ( age && age > 30 ) {
|
|
340
|
-
resolve ({ age, height })
|
|
341
|
-
return
|
|
342
|
-
}
|
|
343
|
-
resolve ( obj )
|
|
344
|
-
}
|
|
345
|
-
|
|
346
|
-
walk ({
|
|
347
|
-
data: x
|
|
348
|
-
, objectCallback
|
|
349
|
-
})
|
|
350
|
-
.then ( result => {
|
|
351
|
-
// 'result.props' will have only 'age' and 'height' properties.
|
|
352
|
-
// {
|
|
353
|
-
// ls : [ 1,2,3 ]
|
|
354
|
-
// , name : 'Peter'
|
|
355
|
-
// , props : {
|
|
356
|
-
// age : 48
|
|
357
|
-
// , height : 176
|
|
358
|
-
// }
|
|
359
|
-
// }
|
|
360
|
-
})
|
|
433
|
+
let result = await walk ({
|
|
434
|
+
data : { password:'secret', name:'Peter' }
|
|
435
|
+
, settings : { breadcrumbs:false, parentPath:false }
|
|
436
|
+
, keyCallback : ({ value, key, IGNORE }) => key === 'password' ? IGNORE() : value
|
|
437
|
+
})
|
|
438
|
+
// { name:'Peter' }
|
|
361
439
|
```
|
|
362
440
|
|
|
441
|
+
| Setting | Default | Effect of `false` |
|
|
442
|
+
| --- | --- | --- |
|
|
443
|
+
| `copy` | `true` | Walk without building a result; resolve `undefined` |
|
|
444
|
+
| `breadcrumbs` | `true` | Skip preparing breadcrumb strings |
|
|
445
|
+
| `parentPath` | `true` | Skip preparing parent-path arrays |
|
|
446
|
+
| `detectCycles` | `true` | Skip circular-reference checks and their bookkeeping |
|
|
363
447
|
|
|
364
|
-
|
|
448
|
+
Only literal `false` disables a setting. Omitted settings remain enabled. Disabled path fields are absent from callback arguments; destructuring them gives `undefined`. With no callbacks, neither path is prepared.
|
|
449
|
+
|
|
450
|
+
Path preparation takes time and memory. Parent arrays grow with depth, so disabling unused paths is especially useful for deeply nested data. Path settings change metadata preparation. The `copy` setting controls result creation independently: `settings:{ copy:false }` still provides both paths, while `settings:{ copy:false, breadcrumbs:false, parentPath:false }` skips all three kinds of work.
|
|
451
|
+
|
|
452
|
+
Set `detectCycles:false` when you know the visited data and callback replacements have no circular references. This skips ancestor records, depth tracking, parent checks, and lookup maintenance. It is independent of copying and paths:
|
|
453
|
+
|
|
454
|
+
```js
|
|
455
|
+
let data = JSON.parse ( '{"items":[1,2,3]}' );
|
|
456
|
+
let total = 0;
|
|
457
|
+
|
|
458
|
+
await walk ({
|
|
459
|
+
data
|
|
460
|
+
, settings : { copy:false, breadcrumbs:false, parentPath:false, detectCycles:false }
|
|
461
|
+
, keyCallback : ({ value }) => { total += value }
|
|
462
|
+
})
|
|
463
|
+
// total === 6
|
|
464
|
+
```
|
|
365
465
|
|
|
366
|
-
|
|
466
|
+
With detection disabled, callbacks must prune any cyclic branches before revisiting them; otherwise walk can continue indefinitely. Returned containers are subject to the same rule.
|
|
367
467
|
|
|
368
|
-
|
|
369
|
-
| -------------------------------- | ---------------------------- |
|
|
370
|
-
| `return value` | `resolve(value)` |
|
|
371
|
-
| `return newObject` | `resolve(newObject)` |
|
|
372
|
-
| `return IGNORE` | `reject()` |
|
|
468
|
+
## Measurements with different data structures
|
|
373
469
|
|
|
374
|
-
|
|
470
|
+
Run `node benchmarks/detectCycles.mjs` to compare cycle detection on and off with copying and both paths independently on and off. It uses pass-through callbacks, one warmup per setting, three samples, and rotating sample order. These historical timings were recorded before FINISH was added. The [recorded samples](benchmarks/detectCycles.results.json) use Node v26.10.0 on macOS arm64; times are medians in milliseconds, with copying enabled:
|
|
471
|
+
|
|
472
|
+
| Shape | Both paths | Detection on | Detection off |
|
|
473
|
+
| --- | --- | --- | --- |
|
|
474
|
+
| 1,000,000-key flat object | On | 1322.17 | 1346.28 |
|
|
475
|
+
| 1,000,000-key flat object | Off | 1203.09 | 1291.67 |
|
|
476
|
+
| 100,000 records with nested metadata and tags | On | 1082.30 | 1080.45 |
|
|
477
|
+
| 100,000 records with nested metadata and tags | Off | 800.08 | 766.23 |
|
|
478
|
+
| 4,000-level child chain | On | 66.27 | 63.77 |
|
|
479
|
+
| 4,000-level child chain | Off | 3.70 | 3.09 |
|
|
480
|
+
|
|
481
|
+
These samples measure traversal overhead without callback I/O. Disabling detection did not improve every case. Path preparation dominated the deep-chain run; callback I/O, garbage collection, and workload shape can change the comparison. The complete record also includes no-copy samples. These measurements are evidence for choosing settings, not unit-test thresholds or a universal speed claim.
|
|
482
|
+
|
|
483
|
+
The inputs used by the memory benchmark below are a 1,000,000-key flat object (`key0` through `key999999`), 100,000 records shaped as `{ id, name:'item', metadata:{ active:true, score:id, tags:['a','b'] } }`, and 4,000 nested `child` links ending at `{ leaf:1 }`. The older timing scripts use per-record names and a `number` leaf instead; their timings and these memory measurements come from separate runs.
|
|
484
|
+
|
|
485
|
+
### Memory use
|
|
486
|
+
|
|
487
|
+
`copy:false` avoids allocating the result, but traversal still needs temporary keys, queue entries, callback arguments, paths, promises, and cycle bookkeeping. These fresh async measurements use the three memory inputs described above, with both callbacks returning `value` and cycle detection enabled throughout.
|
|
488
|
+
|
|
489
|
+
The table shows the **largest observed extra JavaScript heap usage during one awaited walk**, in MiB (1 MiB = 1,048,576 bytes). The input is already allocated and garbage-collected before the baseline is recorded, so its memory is excluded:
|
|
490
|
+
|
|
491
|
+
| Operation and settings | Flat object | Nested records | Deep chain |
|
|
492
|
+
| --- | ---: | ---: | ---: |
|
|
493
|
+
| Copy, all settings enabled | 286.52 | 381.21 | 31.54 |
|
|
494
|
+
| Copy, both paths disabled | 285.49 | 279.53 | 4.66 |
|
|
495
|
+
| Walk without copying, both paths enabled | 284.60 | 322.35 | 29.99 |
|
|
496
|
+
| Walk without copying, both paths disabled | 283.76 | 209.53 | 3.05 |
|
|
497
|
+
|
|
498
|
+
Temporary allocations and garbage collection affect those figures. They describe sampled heap usage, rather than total bytes allocated or an exact peak. Disabling unused paths reduced observed heap usage substantially on the nested records and deep chain. For the flat object, observed heap usage changed little when paths or copying were disabled; no-copy mode still avoided retaining a result.
|
|
499
|
+
|
|
500
|
+
Memory retained after garbage collection answers a different question: how much remains while the result is kept? These figures use default path settings and keep the input reachable:
|
|
501
|
+
|
|
502
|
+
| Structure | Input heap | Extra heap with copy retained | Extra heap without copying |
|
|
503
|
+
| --- | ---: | ---: | ---: |
|
|
504
|
+
| Flat object | 77.76 | 48.09 | 0.08 |
|
|
505
|
+
| Nested records | 12.97 | 29.27 | 0.16 |
|
|
506
|
+
| Deep chain | 0.13 | 0.28 | 0.06 |
|
|
507
|
+
|
|
508
|
+
All values are MiB. Small residual increases include runtime and compiled-code overhead. Disabling paths left retained copy sizes essentially unchanged. After releasing results, allowing one event-loop turn for the async frame to release its completed await value, and collecting again, the median increase was below 0.13 MiB for every combination. This checks these workloads; it does not establish that every possible callback or input is free of memory leaks.
|
|
509
|
+
|
|
510
|
+
Measured on 2026-10-04, using Node v26.10.0 on darwin arm64, an Apple M3 Max, and 128 GiB of RAM. Each of the 12 combinations ran five times in fresh, sequential Node processes, with rotating settings order and no warmup walks. Heap usage was sampled every 1,024 callbacks for the flat object and nested records, every 64 callbacks for the deep chain, and immediately after awaited traversal. Explicit garbage collection ran before the input baseline, after traversal with the result retained, and after releasing the result; none was forced during traversal. The release checkpoint includes an awaited `setImmediate` to clear the prior await result from the async frame. Tables show medians of the five samples. Instrumented memory runs are separate from the timing measurements above.
|
|
511
|
+
|
|
512
|
+
The [benchmark script](benchmarks/memory.mjs) and [complete samples](benchmarks/memory.results.json) include ranges, a hash of the measured async source, verified callback counts, and the process RSS high-water mark through the walk. RSS includes Node, input construction, and memory outside the JavaScript heap; it is recorded separately from extra heap usage.
|
|
513
|
+
|
|
514
|
+
Reproduce the measurements from the repository root:
|
|
515
|
+
|
|
516
|
+
```sh
|
|
517
|
+
node benchmarks/memory.mjs
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
The script enables garbage collection in its worker processes, verifies expected callback counts and result samples, and writes fresh async measurements to `benchmarks/memory.results.json`.
|
|
521
|
+
|
|
522
|
+
### Generated regression tests
|
|
523
|
+
|
|
524
|
+
The [generated test suite](test/11-generated.test.js) uses 32 fixed seeds, so a failing structure can be reproduced. It checks 768 transformation cases across copying, path settings, and cycle detection. Inputs mix objects, sparse arrays, unusual property names, primitives, built-in leaves, shared containers, and circular references. Some branches exceed the depth at which ancestor lookup changes. Circular inputs are tested with detection enabled; acyclic inputs exercise both detection settings.
|
|
525
|
+
|
|
526
|
+
Transformations are compared with a separate recursive reference implementation. Tests check transformed values, key order, array compaction, independent copies of shared containers, ancestor links, callback metadata, `IGNORE()`, local `PASS()`, replacements, and unchanged source containers. Another 384 early-finish combinations check omitted or retained final values, stopping at the visited leaf prefix, discarded pending work, and final-branch key callbacks after `PASS()`.
|
|
527
|
+
|
|
528
|
+
Run all regression tests and enforce full source coverage with:
|
|
529
|
+
|
|
530
|
+
```sh
|
|
531
|
+
npm run cover
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
## What gets copied
|
|
535
|
+
|
|
536
|
+
`walk` creates new containers for objects and arrays. It works with their own enumerable string keys. Primitives retain their values, and the following values retain their references:
|
|
537
|
+
|
|
538
|
+
| Value | Behaviour |
|
|
539
|
+
| --- | --- |
|
|
540
|
+
| Functions and DOM nodes | Same reference in the result |
|
|
541
|
+
| `Date`, `RegExp` | Same reference in the result |
|
|
542
|
+
| `Map`, `Set`, `WeakMap`, `WeakSet` | Same reference in the result |
|
|
543
|
+
| `ArrayBuffer`, `DataView`, typed arrays | Same reference in the result |
|
|
544
|
+
|
|
545
|
+
These values reach `keyCallback` as leaves; their contents are not walked. The same rules apply to values returned by callbacks.
|
|
375
546
|
|
|
376
547
|
```js
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
// walk-async — same shape, just promise-based
|
|
384
|
-
const result = await walk ({
|
|
385
|
-
data: user,
|
|
386
|
-
keyCallback: ({ value, key, resolve, reject }) => {
|
|
387
|
-
if ( key === 'password' ) reject ()
|
|
388
|
-
else resolve ( value )
|
|
389
|
-
}
|
|
390
|
-
})
|
|
548
|
+
let data = { profile:{ name:'Peter' }, when:new Date ( '2026-01-15' ), greet:() => 'Hello' };
|
|
549
|
+
let result = await walk ({ data })
|
|
550
|
+
|
|
551
|
+
result.profile !== data.profile // true: a new object
|
|
552
|
+
result.when === data.when // true: the same Date
|
|
553
|
+
result.greet === data.greet // true: the same function
|
|
391
554
|
```
|
|
392
555
|
|
|
393
|
-
|
|
556
|
+
The result is editable; `walk` does not freeze it. Copying alone does not modify the source. Callback values refer to the source or a returned replacement, so modifying `value` inside a callback can modify the original data. Return a new object, as in `PASS ({ ...value, count:100 })`, when you want to avoid that.
|
|
394
557
|
|
|
558
|
+
Array elements are copied into consecutive indexes. Sparse holes and ignored elements are omitted. Additional enumerable properties such as `'-1'` and `'01'` keep their names. Arrays from another JavaScript context are recognized as arrays.
|
|
395
559
|
|
|
396
|
-
|
|
560
|
+
### Circular references
|
|
397
561
|
|
|
398
|
-
|
|
562
|
+
A reference back to a parent becomes a reference to that parent's copy. Walk closes the circle without visiting the parent's contents again:
|
|
399
563
|
|
|
400
564
|
```js
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
.catch ( err => {
|
|
408
|
-
console.error ( err.message )
|
|
409
|
-
// walk-async: timed out after 5000ms; callbacks still pending:
|
|
410
|
-
// - objectCallback at 'root/props'
|
|
411
|
-
})
|
|
565
|
+
let data = { number:1 };
|
|
566
|
+
data.self = data
|
|
567
|
+
|
|
568
|
+
let result = await walk ({ data })
|
|
569
|
+
result !== data // true: a new object
|
|
570
|
+
result.self === result // true: the circle stays inside the copy
|
|
412
571
|
```
|
|
413
572
|
|
|
414
|
-
|
|
573
|
+
Object callbacks still run at circular properties, so they can remove the reference with `IGNORE()` or replace its value. Detection checks the returned container against its ancestors; containers returned by `keyCallback` follow the same rule. With `settings.copy:false`, walk stops at the circular reference without building a result. Path settings do not affect detection.
|
|
415
574
|
|
|
416
|
-
|
|
417
|
-
- `timeout` is disabled by default. Without it the behavior is exactly as before;
|
|
418
|
-
- The limit applies to the whole walk, not to a single callback. Set it well above the worst-case duration of the legitimate async work inside the callbacks — it is a debugging safety net, not a scheduler;
|
|
419
|
-
- When callbacks finish in time, the timer is cleared and the result is delivered as usual.
|
|
575
|
+
Detection is enabled by default. Only literal `false` in `settings.detectCycles` disables it.
|
|
420
576
|
|
|
577
|
+
An object shared by separate branches is copied independently in each branch. Only a reference back to an ancestor closes a circle. Walk checks the short parent chain for shallow data and uses an internal ancestor lookup for deeper branches.
|
|
421
578
|
|
|
422
|
-
##
|
|
423
|
-
- `walk-async` does not descend into built-in types (`Date`, `RegExp`, `Map`, `Set`, `WeakMap`, `WeakSet`, `ArrayBuffer`, `DataView`, typed arrays, DOM nodes, functions) — they are passed by reference, whether they appear in the input or are returned by a callback;
|
|
424
|
-
- `walk-async` can not execute another `walk-async` from inside the callbacks;
|
|
425
|
-
- The walk promise does not settle until every callback has resolved or rejected. Use the `timeout` option to turn a forgotten `resolve`/`reject` into a rejection with diagnostics.
|
|
579
|
+
## Callback order
|
|
426
580
|
|
|
581
|
+
`objectCallback` sees the root first. At each container, keys are processed in `Object.keys` order. Nested object callbacks run when their property is encountered; processing those containers' contents is deferred until the current container's keys finish. Containers are then processed in the order they were scheduled. Each callback completes before the next one starts, including across awaited delays. This is breadth-first over container contents, with no exit or post-order callback. Sibling callbacks are no longer started concurrently.
|
|
427
582
|
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
583
|
+
Object callbacks see containers before their contents reach key callbacks. Replacement containers follow the same traversal rules. `IGNORE()` prevents further visits within a branch; `PASS()` affects only immediate key callbacks. `FINISH()` stops immediately. `FINISH(value)` from a key callback includes the value directly and stops; from an object callback, it selects the final branch for key callbacks with `isFinished:true`, without further object callbacks. Without a key callback, it includes the value directly.
|
|
584
|
+
|
|
585
|
+
Nested containers are processed iteratively through a work queue. Processed entries are cleared as the walk advances, releasing their references to containers and paths.
|
|
586
|
+
|
|
587
|
+
Callbacks can await another finite `walk` call. Each walk owns its queue and ancestor lookup. Returning a container normally is enough to let the current walk visit its contents; an additional walk performs additional work.
|
|
588
|
+
|
|
589
|
+
## Reuse your rules
|
|
590
|
+
|
|
591
|
+
Keep related transformations in one callback instead of making a separate copy for each rule. When a rule is useful in several places, create a callback factory:
|
|
592
|
+
|
|
593
|
+
```js
|
|
594
|
+
function omitKeys ( ...keys ) {
|
|
595
|
+
const omitted = new Set ( keys );
|
|
596
|
+
return ({ value, key, IGNORE }) => omitted.has ( key ) ? IGNORE() : value
|
|
597
|
+
} // omitKeys func.
|
|
598
|
+
|
|
599
|
+
let result = await walk ({
|
|
600
|
+
data : { name:'Peter', password:'secret', token:'abc' }
|
|
601
|
+
, keyCallback : omitKeys ( 'password', 'token' )
|
|
602
|
+
})
|
|
603
|
+
// { name:'Peter' }
|
|
604
|
+
```
|
|
431
605
|
|
|
606
|
+
The factory is your application code. `walk` stays focused on traversal, callbacks, and building the result.
|
|
607
|
+
|
|
608
|
+
## Scope
|
|
609
|
+
|
|
610
|
+
- With cycle detection enabled, circular references to ancestors are preserved inside the copy. Repeated references on separate branches are copied independently; their shared identity is not preserved.
|
|
611
|
+
- Non-enumerable and symbol keys, object prototypes, and property descriptors are not preserved. Enumerable getters contribute their returned values.
|
|
612
|
+
- Callbacks may return values or promises. Returned promises are awaited. Thrown errors and rejected promises reject the walk; use `IGNORE()` to remove a branch without failing the walk.
|
|
613
|
+
- Built-in values listed above are shared by reference. Use a suitable clone operation for those values if they need independent copies.
|
|
614
|
+
|
|
615
|
+
## Links
|
|
616
|
+
|
|
617
|
+
- [Release history](Changelog.md)
|
|
618
|
+
- [Migration guide](Migration.guide.md)
|
|
619
|
+
- [walk](https://github.com/PeterNaydenov/walk)
|
|
432
620
|
|
|
433
621
|
## Credits
|
|
434
|
-
|
|
622
|
+
|
|
623
|
+
Created and maintained by Peter Naydenov.
|
|
435
624
|
|
|
436
625
|
## License
|
|
437
|
-
|
|
626
|
+
|
|
627
|
+
Released under the MIT License.
|