@cad0p/pi-memory-budget 0.1.0-20261120.0 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -8
- package/package.json +1 -1
- package/src/budget.mjs +8 -65
- package/src/gc-harness.js +1 -3
- package/src/orchestrate.mjs +1 -5
- package/src/report.mjs +5 -25
- package/src/runner.mjs +1 -18
- package/src/sampler.mjs +7 -62
- package/src/stats.mjs +5 -34
package/README.md
CHANGED
|
@@ -13,11 +13,10 @@ A run first **waits for quiescence**, but never before `--settle-min` (default 3
|
|
|
13
13
|
- **Offline**: every pi run gets `PI_OFFLINE=1`, `PI_SKIP_VERSION_CHECK=1`, `PI_TELEMETRY=0` — pi's startup network work (model catalog refresh, version check, telemetry) allocates tens of MB at an unpredictable time and made the vanilla level bimodal (120–178 MB on a quiet box). With it off, vanilla settles within a few MB. `--env` overrides these if a different value is genuinely needed. Limitation: an extension whose startup itself needs the network is measured without that traffic — if its cost matters, measure it on a longer window and read the transient spike, not steady.
|
|
14
14
|
- **GC-normalized**: an idle pi allocates almost nothing, so left to itself V8 never runs a major GC and the settled level is whatever garbage was uncollected when allocation stopped — run-to-run noise of tens of MB. Every pi run (vanilla and candidate alike, so it cancels out) is therefore started with `--expose-gc` — passed through `NODE_OPTIONS`, which reaches node whether pi is a global launcher, a `node_modules/.bin` shim or a plain `node cli.js` — plus a tiny harness extension (`src/gc-harness.js`) that forces major GCs at start, whenever the heap has grown by ~4 MB, and periodically. *steady* measures retained memory, not GC timing. `--no-gc` disables it. Garbage only collects in the pi process itself; children are measured exactly as before. On glibc, freed V8 pages and large buffers go back to the OS promptly, so the post-GC level is observable directly; anything malloc keeps is stable across runs and simply part of the measured level.
|
|
15
15
|
|
|
16
|
-
- **total RSS** = the pi pid plus every descendant process (recursive) — an extension cannot hide memory in a spawned helper; **main** and **children** are still reported separately for diagnosis.
|
|
17
|
-
- **steady** = median total RSS of the settled 10 s window; **spike** = max; **released** = last sample.
|
|
18
|
-
- Linux reads `/proc/<pid>/status` (`VmRSS
|
|
16
|
+
- **total RSS** = the pi pid plus every descendant process (recursive) — an extension cannot hide memory in a spawned helper; **main** and **children** are still reported separately for diagnosis.
|
|
17
|
+
- **steady** = median total RSS of the settled 10 s window; **spike** = max; **released** = last sample.
|
|
18
|
+
- Linux reads `/proc/<pid>/status` (`VmRSS`), macOS uses `ps`. Descendant processes of pi are reported separately as *children*.
|
|
19
19
|
- It sends RPC `get_state` every 100 ms and reports the largest response gap, as a check on event-loop blocking (reported, not budgeted).
|
|
20
|
-
- **allocator behaviour pinned**: every pi run gets `MALLOC_MMAP_THRESHOLD_=65536` (a glibc `mallopt` tunable; ignored on macOS). glibc's default mmap threshold is dynamic and climbs toward 32 MB, so the many large transient buffers of a 100–200 MB session load are served from malloc arenas and stay resident after `free()`. Before this, vanilla's own long-session steady RSS varied by 5–52 MB between runs with an identical post-GC heap (issue #7); with a pinned 64 KiB threshold those buffers `mmap`/`munmap` per call and the kernel takes them back promptly (vanilla long/long2x spread 1–3 MB). This does change glibc's allocation behaviour compared to a plain `pi` launch, which is acceptable because it is applied identically to vanilla, candidate and baseline — it cancels out of every delta, and what it removes is allocator retention noise, not extension cost. `--env MALLOC_MMAP_THRESHOLD_=<n>` overrides it for a run that must reproduce a user's stock environment. The **spike** check is deliberately left on raw total RSS: it is what a user actually feels, allocator retention included.
|
|
21
20
|
- Each configuration runs N times (default 3) and the **median** is used. RSS is noisy by a few MB, so the vanilla baseline always runs in the same job, interleaved with the candidate. The report also shows the run-to-run **steady spread** (max−min) per configuration: when the spread of the reference configuration alone exceeds a check's budget, that check is **INCONCLUSIVE** — a distinct non-zero verdict (exit code 3 when nothing else failed) — because no delta can be trusted against a baseline noisier than the budget. Rerun on a quieter machine or with more `--runs`.
|
|
22
21
|
|
|
23
22
|
Cases:
|
|
@@ -65,7 +64,7 @@ pi is resolved from `--pi <cmd>`, then `node_modules/.bin/pi` of `--cwd`, then `
|
|
|
65
64
|
| `--baseline-extension <path>` | baseline build, e.g. checked out from `main`, repeatable |
|
|
66
65
|
| `--config <budget.json>` | budgets, waivers, env, setup hooks |
|
|
67
66
|
| `--cwd <dir>` | working directory of pi (a workspace or vault the extension needs) |
|
|
68
|
-
| `--env K=V` | environment variable for every pi process, repeatable
|
|
67
|
+
| `--env K=V` | environment variable for every pi process, repeatable |
|
|
69
68
|
| `--runs <n>` | runs per configuration and case (default 3) |
|
|
70
69
|
| `--settle-min <s>` | no settle before this time; the GC harness has collected by then (default 30) |
|
|
71
70
|
| `--window <s>` | minimum window per run (default 30), `--steady-seconds` (default 10), `--settle-max` (default 120) |
|
|
@@ -110,10 +109,9 @@ A single run needs about `runs x cases x configurations x window` seconds (defau
|
|
|
110
109
|
```
|
|
111
110
|
|
|
112
111
|
- *Vanilla* and *Candidate* are medians in MB (steady for the steady check, candidate max for the spike); *Delta* is what the extension adds.
|
|
113
|
-
- A second table per case lists steady, the run-to-run steady spread, spike, released, children steady
|
|
114
|
-
- A large vanilla *spread* explains an INCONCLUSIVE verdict, and the breakdown says where it lives. Spread in **RssAnon** with a flat heapTotal is allocator retention or a genuine allocation difference; spread in **RssFile** is this machine's page cache (the session file and shared libraries) moving under host I/O pressure, not the measured process's retention — the note on the affected check says which, by comparing the anon spread against the budget.
|
|
112
|
+
- A second table per case lists steady, the run-to-run steady spread, spike, released, children steady and the max ping gap of every configuration, plus post-GC heap/external/arrayBuffers numbers from the GC harness (diagnostic only — budgets stay on total RSS). A large *children* value means the extension spawns processes that the budget on the pi process does not see. A large vanilla *spread* explains an INCONCLUSIVE verdict.
|
|
115
113
|
- *Growth with session size* compares the long and long2x deltas. A FAIL means the extension's memory scales with the session: fix it, or justify it in the PR.
|
|
116
|
-
- JSON (`schema: pi-memory-budget/1`) has the same checks plus aggregates and the raw RSS samples of every run (`[t seconds, rss MB, children MB
|
|
114
|
+
- JSON (`schema: pi-memory-budget/1`) has the same checks plus aggregates and the raw RSS samples of every run (`[t seconds, rss MB, children MB]`).
|
|
117
115
|
|
|
118
116
|
## CI
|
|
119
117
|
|
package/package.json
CHANGED
package/src/budget.mjs
CHANGED
|
@@ -82,58 +82,20 @@ export function resolveBudgets(budgets, waivers) {
|
|
|
82
82
|
* @property {boolean} settled whether every run settled
|
|
83
83
|
* @property {number} maxGapMs median of the per-run max ping gap
|
|
84
84
|
* @property {number} worstGapMs worst ping gap over all runs
|
|
85
|
-
* @property {number | null}
|
|
86
|
-
* @property {number | null} anonSteadySpreadMb max-min of per-run steady RssAnon
|
|
87
|
-
* @property {number | null} fileSteadyMb median of per-run steady RssFile
|
|
88
|
-
* @property {number | null} fileSteadySpreadMb max-min of per-run steady RssFile
|
|
89
|
-
* @property {number | null} shmemSteadyMb median of per-run steady RssShmem
|
|
90
|
-
* @property {number | null} shmemSteadySpreadMb max-min of per-run steady RssShmem
|
|
91
|
-
* @property {{ heapUsedMb: number, heapTotalMb: number | null, externalMb: number, arrayBuffersMb: number } | null} gcDiag median post-GC heap numbers (diagnostic)
|
|
85
|
+
* @property {{ heapUsedMb: number, externalMb: number, arrayBuffersMb: number } | null} gcDiag median post-GC heap numbers (diagnostic)
|
|
92
86
|
*/
|
|
93
87
|
|
|
94
88
|
/** @param {RunResult[]} runs */
|
|
95
89
|
function aggDiag(runs) {
|
|
96
90
|
const d = runs.map((r) => r.gcDiag).filter((x) => x);
|
|
97
91
|
if (!d.length) return null;
|
|
98
|
-
/** @param {"heapUsedMb"|"
|
|
99
|
-
const m = (k) =>
|
|
100
|
-
|
|
101
|
-
.map((x) => /** @type {NonNullable<typeof x>} */ (x)[k])
|
|
102
|
-
.filter((x) => x !== null);
|
|
103
|
-
return v.length ? median(v) : null;
|
|
104
|
-
};
|
|
92
|
+
/** @param {"heapUsedMb"|"externalMb"|"arrayBuffersMb"} k */
|
|
93
|
+
const m = (k) =>
|
|
94
|
+
median(d.map((x) => /** @type {NonNullable<typeof x>} */ (x)[k]));
|
|
105
95
|
return {
|
|
106
|
-
heapUsedMb:
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
arrayBuffersMb: /** @type {number} */ (m("arrayBuffersMb")),
|
|
110
|
-
};
|
|
111
|
-
}
|
|
112
|
-
|
|
113
|
-
/**
|
|
114
|
-
* Per-run values of a nullable summary field, for a median or a spread.
|
|
115
|
-
* @param {RunResult[]} runs
|
|
116
|
-
* @param {"anonSteadyMb"|"fileSteadyMb"|"shmemSteadyMb"} k
|
|
117
|
-
* @returns {number[]}
|
|
118
|
-
*/
|
|
119
|
-
function fieldValues(runs, k) {
|
|
120
|
-
return /** @type {number[]} */ (
|
|
121
|
-
runs.map((r) => r[k]).filter((x) => x !== null && !Number.isNaN(x))
|
|
122
|
-
);
|
|
123
|
-
}
|
|
124
|
-
|
|
125
|
-
/**
|
|
126
|
-
* Median and max-min spread of a per-run breakdown field; both null when the
|
|
127
|
-
* platform does not report it, spread null with fewer than 2 values.
|
|
128
|
-
* @param {RunResult[]} runs
|
|
129
|
-
* @param {"anonSteadyMb"|"fileSteadyMb"|"shmemSteadyMb"} k
|
|
130
|
-
* @returns {{ med: number | null, spread: number | null }}
|
|
131
|
-
*/
|
|
132
|
-
function fieldStats(runs, k) {
|
|
133
|
-
const v = fieldValues(runs, k);
|
|
134
|
-
return {
|
|
135
|
-
med: v.length ? median(v) : null,
|
|
136
|
-
spread: v.length >= 2 ? Math.max(...v) - Math.min(...v) : null,
|
|
96
|
+
heapUsedMb: m("heapUsedMb"),
|
|
97
|
+
externalMb: m("externalMb"),
|
|
98
|
+
arrayBuffersMb: m("arrayBuffersMb"),
|
|
137
99
|
};
|
|
138
100
|
}
|
|
139
101
|
|
|
@@ -172,12 +134,6 @@ export function aggregate(runs) {
|
|
|
172
134
|
childrenSteadyMb: med(runs.map((r) => r.childrenSteadyMb)),
|
|
173
135
|
maxGapMs: median(runs.map((r) => r.maxGapMs)),
|
|
174
136
|
worstGapMs: max(runs.map((r) => r.maxGapMs)),
|
|
175
|
-
anonSteadyMb: fieldStats(runs, "anonSteadyMb").med,
|
|
176
|
-
anonSteadySpreadMb: fieldStats(runs, "anonSteadyMb").spread,
|
|
177
|
-
fileSteadyMb: fieldStats(runs, "fileSteadyMb").med,
|
|
178
|
-
fileSteadySpreadMb: fieldStats(runs, "fileSteadyMb").spread,
|
|
179
|
-
shmemSteadyMb: fieldStats(runs, "shmemSteadyMb").med,
|
|
180
|
-
shmemSteadySpreadMb: fieldStats(runs, "shmemSteadyMb").spread,
|
|
181
137
|
gcDiag: aggDiag(runs),
|
|
182
138
|
};
|
|
183
139
|
}
|
|
@@ -275,20 +231,7 @@ export function evaluate(m, opts) {
|
|
|
275
231
|
const spreadNote = (ref, limitMb, name) => {
|
|
276
232
|
const s = ref.steadySpreadMb;
|
|
277
233
|
if (s === null || s <= limitMb) return null;
|
|
278
|
-
|
|
279
|
-
// Issue #7: the residual total-RSS spread on long sessions is usually
|
|
280
|
-
// file-backed pages (page cache of the session file plus shared
|
|
281
|
-
// libraries) moving with host I/O pressure, not retention. Say so, and
|
|
282
|
-
// point at the column that is comparable run to run.
|
|
283
|
-
const anon = ref.anonSteadySpreadMb;
|
|
284
|
-
if (anon !== null && anon <= limitMb) {
|
|
285
|
-
const file =
|
|
286
|
-
ref.fileSteadySpreadMb === null
|
|
287
|
-
? ""
|
|
288
|
-
: ` and RssFile ${ref.fileSteadySpreadMb.toFixed(1)} MB`;
|
|
289
|
-
note += `; it sits outside the JS heap in file-backed pages (RssAnon spread ${anon.toFixed(1)} MB${file}), so this machine's page cache is moving, not the measured process's retention`;
|
|
290
|
-
}
|
|
291
|
-
return note;
|
|
234
|
+
return `${name} steady RSS spread across ${ref.runs} runs is ${s.toFixed(1)} MB (max-min), over the ${limitMb} MB budget of this check: too noisy for a verdict; rerun on a quieter machine or with more runs`;
|
|
292
235
|
};
|
|
293
236
|
|
|
294
237
|
const cases = CASE_ORDER.filter((c) => m[c]);
|
package/src/gc-harness.js
CHANGED
|
@@ -57,10 +57,8 @@ export default function pmbGcHarness() {
|
|
|
57
57
|
const m = process.memoryUsage();
|
|
58
58
|
base = m.heapUsed;
|
|
59
59
|
// Diagnostic only (post-GC numbers); budgets stay on total RSS.
|
|
60
|
-
// heapTotal next to heapUsed: the gap is V8-committed but unused pages,
|
|
61
|
-
// one of the long-session spread suspects (issue #7).
|
|
62
60
|
process.stderr.write(
|
|
63
|
-
`${MARK} diag ${JSON.stringify({ heapUsed: m.heapUsed,
|
|
61
|
+
`${MARK} diag ${JSON.stringify({ heapUsed: m.heapUsed, external: m.external, arrayBuffers: m.arrayBuffers })}\n`,
|
|
64
62
|
);
|
|
65
63
|
}
|
|
66
64
|
}, INTERVAL_MS);
|
package/src/orchestrate.mjs
CHANGED
|
@@ -121,12 +121,8 @@ export async function runBudget(o, hooks = {}) {
|
|
|
121
121
|
releaseSec: o.budgets.spikeReleaseSeconds,
|
|
122
122
|
extraArgs: o.piArgs,
|
|
123
123
|
});
|
|
124
|
-
const anon =
|
|
125
|
-
r.anonSteadyMb === null || r.anonSteadyMb === undefined
|
|
126
|
-
? ""
|
|
127
|
-
: `, anon ${r.anonSteadyMb.toFixed(1)} + file ${(r.fileSteadyMb ?? 0).toFixed(1)} + shmem ${(r.shmemSteadyMb ?? 0).toFixed(1)}`;
|
|
128
124
|
log(
|
|
129
|
-
` ${r.settled ? `steady ${/** @type {number} */ (r.steadyMb).toFixed(1)} MB (settled at ${/** @type {number} */ (r.settleT).toFixed(0)}s, main ${/** @type {number} */ (r.mainSteadyMb).toFixed(1)} + children ${/** @type {number} */ (r.childrenSteadyMb).toFixed(1)}
|
|
125
|
+
` ${r.settled ? `steady ${/** @type {number} */ (r.steadyMb).toFixed(1)} MB (settled at ${/** @type {number} */ (r.settleT).toFixed(0)}s, main ${/** @type {number} */ (r.mainSteadyMb).toFixed(1)} + children ${/** @type {number} */ (r.childrenSteadyMb).toFixed(1)}` : "never settled"}, spike ${r.spikeMb.toFixed(1)} MB, max ping gap ${r.maxGapMs.toFixed(0)} ms`,
|
|
130
126
|
);
|
|
131
127
|
measurements[c][name].push(r);
|
|
132
128
|
if (o.gc && r.gcReady === false)
|
package/src/report.mjs
CHANGED
|
@@ -26,21 +26,13 @@ function statusCell(c) {
|
|
|
26
26
|
return c.status === "FAIL" && c.note ? `FAIL: ${c.note}` : c.status;
|
|
27
27
|
}
|
|
28
28
|
|
|
29
|
-
/** @param {{ heapUsedMb: number,
|
|
29
|
+
/** @param {{ heapUsedMb: number, externalMb: number, arrayBuffersMb: number } | null | undefined} d */
|
|
30
30
|
function gcDiagCell(d) {
|
|
31
31
|
return d
|
|
32
|
-
? `${fmt(d.heapUsedMb)} / ${fmt(d.
|
|
32
|
+
? `${fmt(d.heapUsedMb)} / ${fmt(d.externalMb)} / ${fmt(d.arrayBuffersMb)}`
|
|
33
33
|
: "n/a";
|
|
34
34
|
}
|
|
35
35
|
|
|
36
|
-
/** Three nullable values as "a / b / c".
|
|
37
|
-
* @param {(number | null | undefined)[]} v */
|
|
38
|
-
function tripleCell(v) {
|
|
39
|
-
return v.every((x) => x === null || x === undefined || Number.isNaN(x))
|
|
40
|
-
? "n/a"
|
|
41
|
-
: v.map((x) => fmt(x)).join(" / ");
|
|
42
|
-
}
|
|
43
|
-
|
|
44
36
|
/**
|
|
45
37
|
* @typedef {object} ReportInput
|
|
46
38
|
* @property {ReturnType<typeof import("./budget.mjs").evaluate>} result
|
|
@@ -102,8 +94,8 @@ export function renderMarkdown(input) {
|
|
|
102
94
|
}
|
|
103
95
|
out.push("");
|
|
104
96
|
out.push(
|
|
105
|
-
"| Configuration | settle at | steady | steady spread | spike | released | main steady | children steady | max ping gap |
|
|
106
|
-
"
|
|
97
|
+
"| Configuration | settle at | steady | steady spread | spike | released | main steady | children steady | max ping gap | post-GC heap/external/arrayBuffers (diagnostic) |",
|
|
98
|
+
"|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|",
|
|
107
99
|
);
|
|
108
100
|
for (const [name, a] of Object.entries(agg)) {
|
|
109
101
|
const settle =
|
|
@@ -119,7 +111,7 @@ export function renderMarkdown(input) {
|
|
|
119
111
|
? "0.0"
|
|
120
112
|
: `${fmt(a.steadySpreadMb)} (${fmt(a.steadyMinMb)}-${fmt(a.steadyMaxMb)})`;
|
|
121
113
|
out.push(
|
|
122
|
-
`| ${name} | ${settle} | ${a.steadyMb === null ? "not settled" : fmt(a.steadyMb)} | ${spread} | ${fmt(a.spikeMb)} | ${fmt(a.releasedMb)} | ${fmt(a.mainSteadyMb)} | ${fmt(a.childrenSteadyMb)} | ${fmt(a.maxGapMs, 0)} ms | ${
|
|
114
|
+
`| ${name} | ${settle} | ${a.steadyMb === null ? "not settled" : fmt(a.steadyMb)} | ${spread} | ${fmt(a.spikeMb)} | ${fmt(a.releasedMb)} | ${fmt(a.mainSteadyMb)} | ${fmt(a.childrenSteadyMb)} | ${fmt(a.maxGapMs, 0)} ms | ${gcDiagCell(a.gcDiag)} |`,
|
|
123
115
|
);
|
|
124
116
|
}
|
|
125
117
|
out.push("");
|
|
@@ -170,9 +162,6 @@ export function renderJson(input) {
|
|
|
170
162
|
peakT: r.peakT,
|
|
171
163
|
childrenSteadyMb: r.childrenSteadyMb,
|
|
172
164
|
childrenSpikeMb: r.childrenSpikeMb,
|
|
173
|
-
anonSteadyMb: r.anonSteadyMb,
|
|
174
|
-
fileSteadyMb: r.fileSteadyMb,
|
|
175
|
-
shmemSteadyMb: r.shmemSteadyMb,
|
|
176
165
|
gcReady: r.gcReady,
|
|
177
166
|
gcDiag: r.gcDiag,
|
|
178
167
|
maxGapMs: r.maxGapMs,
|
|
@@ -182,15 +171,6 @@ export function renderJson(input) {
|
|
|
182
171
|
Number(s.t.toFixed(2)),
|
|
183
172
|
Number(s.rss.toFixed(1)),
|
|
184
173
|
Number(s.kids.toFixed(1)),
|
|
185
|
-
s.anon === null || s.anon === undefined
|
|
186
|
-
? null
|
|
187
|
-
: Number(s.anon.toFixed(1)),
|
|
188
|
-
s.file === null || s.file === undefined
|
|
189
|
-
? null
|
|
190
|
-
: Number(s.file.toFixed(1)),
|
|
191
|
-
s.shmem === null || s.shmem === undefined
|
|
192
|
-
? null
|
|
193
|
-
: Number(s.shmem.toFixed(1)),
|
|
194
174
|
]),
|
|
195
175
|
}));
|
|
196
176
|
}
|
package/src/runner.mjs
CHANGED
|
@@ -46,15 +46,6 @@ export const PI_QUIET_ENV = Object.freeze({
|
|
|
46
46
|
PI_OFFLINE: "1",
|
|
47
47
|
PI_SKIP_VERSION_CHECK: "1",
|
|
48
48
|
PI_TELEMETRY: "0",
|
|
49
|
-
// glibc mallopt (Linux only, ignored elsewhere): pin the mmap threshold at
|
|
50
|
-
// 64 KiB. The default is a dynamic threshold that climbs toward 32 MB, so
|
|
51
|
-
// the large transient buffers of a 100-200 MB session load end up served
|
|
52
|
-
// from malloc arenas and stay resident after free() — that retention was
|
|
53
|
-
// most of the run-to-run long-case steady spread (issue #7: 5-52 MB).
|
|
54
|
-
// Under a pinned 64 KiB threshold those buffers mmap/munmap per call and
|
|
55
|
-
// are returned to the kernel promptly. Set identically for every
|
|
56
|
-
// configuration, so the delta is unaffected; --env overrides it.
|
|
57
|
-
MALLOC_MMAP_THRESHOLD_: "65536",
|
|
58
49
|
});
|
|
59
50
|
|
|
60
51
|
/**
|
|
@@ -68,11 +59,6 @@ export function parseGcDiag(stderr) {
|
|
|
68
59
|
const d = JSON.parse(all[all.length - 1][1]);
|
|
69
60
|
return {
|
|
70
61
|
heapUsedMb: d.heapUsed / 1048576,
|
|
71
|
-
// heapTotal: V8-committed heap pages — retained-but-unused after a
|
|
72
|
-
// major GC is exactly what this diagnoses (issue #7). Older harness
|
|
73
|
-
// output may lack it.
|
|
74
|
-
heapTotalMb:
|
|
75
|
-
typeof d.heapTotal === "number" ? d.heapTotal / 1048576 : null,
|
|
76
62
|
externalMb: d.external / 1048576,
|
|
77
63
|
arrayBuffersMb: d.arrayBuffers / 1048576,
|
|
78
64
|
};
|
|
@@ -119,7 +105,7 @@ const SETTLE_RANGE_MB = 5;
|
|
|
119
105
|
* exitT: number | null,
|
|
120
106
|
* windowEndSec: number,
|
|
121
107
|
* gcReady: boolean | null,
|
|
122
|
-
* gcDiag: { heapUsedMb: number,
|
|
108
|
+
* gcDiag: { heapUsedMb: number, externalMb: number, arrayBuffersMb: number } | null,
|
|
123
109
|
* }} RunResult
|
|
124
110
|
*/
|
|
125
111
|
|
|
@@ -257,9 +243,6 @@ export async function measureRun(spec) {
|
|
|
257
243
|
rss: r.rssMb,
|
|
258
244
|
main: r.mainMb ?? 0,
|
|
259
245
|
kids: r.kidsMb,
|
|
260
|
-
anon: r.anonMb,
|
|
261
|
-
file: r.fileMb,
|
|
262
|
-
shmem: r.shmemMb,
|
|
263
246
|
});
|
|
264
247
|
}
|
|
265
248
|
const lastT = (performance.now() - t0) / 1000;
|
package/src/sampler.mjs
CHANGED
|
@@ -16,31 +16,6 @@ export function parseVmRss(text) {
|
|
|
16
16
|
return m ? Number(m[1]) : null;
|
|
17
17
|
}
|
|
18
18
|
|
|
19
|
-
/**
|
|
20
|
-
* RSS breakdown of /proc/<pid>/status: VmRSS = RssAnon + RssFile + RssShmem.
|
|
21
|
-
* RssAnon is heap, stacks and malloc arenas; RssFile is page-cache-backed
|
|
22
|
-
* (the session file, shared libraries); RssShmem is SysV/POSIX shared memory
|
|
23
|
-
* (absent before Linux 4.15). Every field is nullable: a field that is not in
|
|
24
|
-
* the file stays null so the caller can report "unavailable" instead of a
|
|
25
|
-
* wrong 0.
|
|
26
|
-
* @param {string} text contents of /proc/<pid>/status
|
|
27
|
-
* @returns {{ rssKb: number | null, anonKb: number | null, fileKb: number | null, shmemKb: number | null }}
|
|
28
|
-
*/
|
|
29
|
-
export function parseStatusRss(text) {
|
|
30
|
-
/** @param {string} name */
|
|
31
|
-
const field = (name) => {
|
|
32
|
-
const re = new RegExp(`^${name}:\\s+(\\d+)\\s*kB`, "m");
|
|
33
|
-
const m = re.exec(text);
|
|
34
|
-
return m ? Number(m[1]) : null;
|
|
35
|
-
};
|
|
36
|
-
return {
|
|
37
|
-
rssKb: field("VmRSS"),
|
|
38
|
-
anonKb: field("RssAnon"),
|
|
39
|
-
fileKb: field("RssFile"),
|
|
40
|
-
shmemKb: field("RssShmem"),
|
|
41
|
-
};
|
|
42
|
-
}
|
|
43
|
-
|
|
44
19
|
/**
|
|
45
20
|
* @param {string} text output of `ps -A -o pid=,ppid=,rss=`
|
|
46
21
|
* @returns {{ pid: number, ppid: number, rssKb: number }[]}
|
|
@@ -83,10 +58,10 @@ export function descendantsRssKb(rows, root) {
|
|
|
83
58
|
return total;
|
|
84
59
|
}
|
|
85
60
|
|
|
86
|
-
/** @param {number} pid @returns {
|
|
87
|
-
function
|
|
61
|
+
/** @param {number} pid @returns {number | null} VmRSS in KB */
|
|
62
|
+
function procRssKb(pid) {
|
|
88
63
|
try {
|
|
89
|
-
return
|
|
64
|
+
return parseVmRss(fs.readFileSync(`/proc/${pid}/status`, "utf8"));
|
|
90
65
|
} catch {
|
|
91
66
|
return null;
|
|
92
67
|
}
|
|
@@ -144,39 +119,19 @@ function run(args) {
|
|
|
144
119
|
|
|
145
120
|
/**
|
|
146
121
|
* Total RSS of pid plus all descendants, and the split for diagnosis.
|
|
147
|
-
* On Linux anonMb/fileMb/shmemMb are the same sum per /proc RssAnon/RssFile/
|
|
148
|
-
* RssShmem (null when the kernel does not report them); on other platforms
|
|
149
|
-
* they are null.
|
|
150
122
|
* @param {number} pid
|
|
151
123
|
* @param {{ withChildren?: boolean }} [opts]
|
|
152
|
-
* @returns {Promise<{ rssMb: number | null, mainMb: number | null, kidsMb: number
|
|
124
|
+
* @returns {Promise<{ rssMb: number | null, mainMb: number | null, kidsMb: number }>}
|
|
153
125
|
* rssMb null when the process is gone (kidsMb is still the last read: descendants may outlive it)
|
|
154
126
|
*/
|
|
155
127
|
export async function readRss(pid, opts = {}) {
|
|
156
128
|
const withChildren = opts.withChildren ?? true;
|
|
157
129
|
if (process.platform === "linux") {
|
|
158
|
-
|
|
159
|
-
|
|
130
|
+
const mainKb = procRssKb(pid);
|
|
131
|
+
let kidsKb = 0;
|
|
160
132
|
if (withChildren) {
|
|
161
|
-
for (const d of linuxDescendants(pid))
|
|
133
|
+
for (const d of linuxDescendants(pid)) kidsKb += procRssKb(d) ?? 0;
|
|
162
134
|
}
|
|
163
|
-
const mainKb = stats[0]?.rssKb ?? null;
|
|
164
|
-
let kidsKb = 0;
|
|
165
|
-
for (let i = 1; i < stats.length; i++) kidsKb += stats[i]?.rssKb ?? 0;
|
|
166
|
-
/** Sum of one /proc field over main + descendants; null when nobody
|
|
167
|
-
* reports it.
|
|
168
|
-
* @param {"anonKb" | "fileKb" | "shmemKb"} k */
|
|
169
|
-
const sumField = (k) => {
|
|
170
|
-
let t = 0;
|
|
171
|
-
let any = false;
|
|
172
|
-
for (const s of stats) {
|
|
173
|
-
if (s && s[k] !== null) {
|
|
174
|
-
t += /** @type {number} */ (s[k]);
|
|
175
|
-
any = true;
|
|
176
|
-
}
|
|
177
|
-
}
|
|
178
|
-
return any ? t / KB_PER_MB : null;
|
|
179
|
-
};
|
|
180
135
|
return {
|
|
181
136
|
rssMb:
|
|
182
137
|
mainKb === null
|
|
@@ -186,13 +141,9 @@ export async function readRss(pid, opts = {}) {
|
|
|
186
141
|
: (mainKb + kidsKb) / KB_PER_MB,
|
|
187
142
|
mainMb: mainKb === null ? null : mainKb / KB_PER_MB,
|
|
188
143
|
kidsMb: kidsKb / KB_PER_MB,
|
|
189
|
-
anonMb: sumField("anonKb"),
|
|
190
|
-
fileMb: sumField("fileKb"),
|
|
191
|
-
shmemMb: sumField("shmemKb"),
|
|
192
144
|
};
|
|
193
145
|
}
|
|
194
146
|
// macOS / other unix: one ps call for the whole table (main row included).
|
|
195
|
-
// No anon/file breakdown there: report nulls.
|
|
196
147
|
const table = parsePsTable(await run(["-A", "-o", "pid=,ppid=,rss="]));
|
|
197
148
|
const self = table.find((r) => r.pid === pid);
|
|
198
149
|
const kidsKb = withChildren ? descendantsRssKb(table, pid) : 0;
|
|
@@ -201,17 +152,11 @@ export async function readRss(pid, opts = {}) {
|
|
|
201
152
|
rssMb: kidsKb > 0 ? kidsKb / KB_PER_MB : null,
|
|
202
153
|
mainMb: null,
|
|
203
154
|
kidsMb: kidsKb / KB_PER_MB,
|
|
204
|
-
anonMb: null,
|
|
205
|
-
fileMb: null,
|
|
206
|
-
shmemMb: null,
|
|
207
155
|
};
|
|
208
156
|
const kids = withChildren ? kidsKb : 0;
|
|
209
157
|
return {
|
|
210
158
|
rssMb: (self.rssKb + kids) / KB_PER_MB,
|
|
211
159
|
mainMb: self.rssKb / KB_PER_MB,
|
|
212
160
|
kidsMb: kids / KB_PER_MB,
|
|
213
|
-
anonMb: null,
|
|
214
|
-
fileMb: null,
|
|
215
|
-
shmemMb: null,
|
|
216
161
|
};
|
|
217
162
|
}
|
package/src/stats.mjs
CHANGED
|
@@ -27,10 +27,8 @@ export function max(values) {
|
|
|
27
27
|
/**
|
|
28
28
|
* One RSS sample. t is seconds since spawn. rss is the TOTAL memory of the pi
|
|
29
29
|
* process: main pid plus every descendant, in MB. main and kids split it for
|
|
30
|
-
* diagnosis.
|
|
31
|
-
*
|
|
32
|
-
* null or absent where the platform does not report them.
|
|
33
|
-
* @typedef {{ t: number, rss: number, main: number, kids: number, anon?: number | null, file?: number | null, shmem?: number | null }} Sample
|
|
30
|
+
* diagnosis.
|
|
31
|
+
* @typedef {{ t: number, rss: number, main: number, kids: number }} Sample
|
|
34
32
|
*/
|
|
35
33
|
|
|
36
34
|
/**
|
|
@@ -92,9 +90,6 @@ export function findSettle(samples, o = {}) {
|
|
|
92
90
|
* @property {number | null} mainSteadyMb median main-pid RSS of the settled window
|
|
93
91
|
* @property {number | null} childrenSteadyMb median descendant RSS of the settled window
|
|
94
92
|
* @property {number} childrenSpikeMb max descendant RSS
|
|
95
|
-
* @property {number | null} anonSteadyMb median RssAnon of the settled window
|
|
96
|
-
* @property {number | null} fileSteadyMb median RssFile of the settled window
|
|
97
|
-
* @property {number | null} shmemSteadyMb median RssShmem of the settled window
|
|
98
93
|
*/
|
|
99
94
|
|
|
100
95
|
/**
|
|
@@ -128,28 +123,9 @@ export function summarize(samples, o = {}) {
|
|
|
128
123
|
mainSteadyMb: tail.length ? median(tail.map((s) => s.main)) : null,
|
|
129
124
|
childrenSteadyMb: tail.length ? median(tail.map((s) => s.kids)) : null,
|
|
130
125
|
childrenSpikeMb: max(samples.map((s) => s.kids)),
|
|
131
|
-
anonSteadyMb: medSampleField(tail, "anon"),
|
|
132
|
-
fileSteadyMb: medSampleField(tail, "file"),
|
|
133
|
-
shmemSteadyMb: medSampleField(tail, "shmem"),
|
|
134
126
|
};
|
|
135
127
|
}
|
|
136
128
|
|
|
137
|
-
/**
|
|
138
|
-
* Median of a nullable per-sample breakdown field over the settled window;
|
|
139
|
-
* null when the window is empty or the field is not reported (non-Linux).
|
|
140
|
-
* @param {Sample[]} tail
|
|
141
|
-
* @param {"anon" | "file" | "shmem"} k
|
|
142
|
-
* @returns {number | null}
|
|
143
|
-
*/
|
|
144
|
-
function medSampleField(tail, k) {
|
|
145
|
-
const v = /** @type {number[]} */ (
|
|
146
|
-
tail
|
|
147
|
-
.map((s) => s[k])
|
|
148
|
-
.filter((x) => typeof x === "number" && !Number.isNaN(x))
|
|
149
|
-
);
|
|
150
|
-
return v.length ? median(v) : null;
|
|
151
|
-
}
|
|
152
|
-
|
|
153
129
|
/**
|
|
154
130
|
* Seconds from the RSS peak until the delta over the vanilla steady RSS first
|
|
155
131
|
* holds at or below the steady budget for `hold` consecutive samples (one dip
|
|
@@ -159,8 +135,6 @@ function medSampleField(tail, k) {
|
|
|
159
135
|
* @param {number} vanillaSteadyMb
|
|
160
136
|
* @param {number} steadyBudgetMb
|
|
161
137
|
* @param {number} [hold]
|
|
162
|
-
* @param {"rss" | "anon"} [field] sample field the release is measured on
|
|
163
|
-
* ("anon" = RssAnon where reported, total RSS elsewhere)
|
|
164
138
|
* @returns {number | null}
|
|
165
139
|
*/
|
|
166
140
|
export function releaseSeconds(
|
|
@@ -168,20 +142,17 @@ export function releaseSeconds(
|
|
|
168
142
|
vanillaSteadyMb,
|
|
169
143
|
steadyBudgetMb,
|
|
170
144
|
hold = 3,
|
|
171
|
-
field = "rss",
|
|
172
145
|
) {
|
|
173
146
|
if (samples.length === 0) return null;
|
|
174
|
-
/** @param {Sample} s */
|
|
175
|
-
const value = (s) => (field === "anon" ? (s.anon ?? s.rss) : s.rss);
|
|
176
147
|
let peakIdx = 0;
|
|
177
148
|
for (let i = 0; i < samples.length; i++) {
|
|
178
|
-
if (
|
|
149
|
+
if (samples[i].rss > samples[peakIdx].rss) peakIdx = i;
|
|
179
150
|
}
|
|
180
151
|
const peak = samples[peakIdx];
|
|
181
|
-
if (
|
|
152
|
+
if (peak.rss - vanillaSteadyMb <= steadyBudgetMb) return 0;
|
|
182
153
|
let streak = 0;
|
|
183
154
|
for (let i = peakIdx + 1; i < samples.length; i++) {
|
|
184
|
-
if (
|
|
155
|
+
if (samples[i].rss - vanillaSteadyMb <= steadyBudgetMb) {
|
|
185
156
|
streak++;
|
|
186
157
|
if (streak >= hold) return samples[i - streak + 1].t - peak.t;
|
|
187
158
|
} else {
|