jspurefix 5.9.1 → 5.9.2

package/BACKPORT_PLAN.md

@@ -435,13 +435,26 @@ Note this supersedes phase D6 of `DEMO_PORT_PLAN.md`, which had placed the regis
 and wildcard handling in the demo. Issue #153 showed they belong in the engine - the
 reporter hit them without writing a demo.
 
-### Follow-up found while testing
-
-The demo's `server-bounce` scenario does not pass, on this branch or on published
-5.8.5. The acceptor's persisted sender sequence lags what it actually sent, because
-`AsciiSession.txOnEncoded` writes to the store fire-and-forget; if the process exits
-first the store is behind, and on restart the client sees a sequence below what it
-has already recorded and drops the session. Needs its own phase.
+### Follow-up found while testing — resolved, and the first diagnosis was wrong
+
+This originally recorded the demo's `server-bounce` scenario as failing because
+`AsciiSession.txOnEncoded` writes to the session store fire-and-forget, leaving the
+acceptor's persisted sender sequence behind what it had actually sent.
+
+**That was wrong.** Reproducing the run by hand: after phase 1 the two stores agree
+exactly (client `4 : 14`, server `14 : 4`), and a second session started against them
+recovers and trades normally, including under the scenario's tight 5-second client
+budget. There is no store flush bug.
+
+The real cause was in the scenario harness. `run_quiet_bg` published its pid via
+command substitution, so `$!` belonged to a subshell and `wait` on it failed
+instantly with "not a child of this shell" rather than blocking. "Wait for the server
+to time out" was therefore a no-op, and the first server was still bound to the port
+when the next step started a second one. A `2>/dev/null || true` kept it silent.
+
+Fixed in jspf-demo; all six scenarios pass. Kept here as a reminder that a failing
+end-to-end scenario is not evidence of an engine bug until the engine has been
+driven directly.
 
 ---
 

package/WORKLOG.md

@@ -0,0 +1,119 @@
+# Worklog
+
+Running notes on what has been done and what is worth doing next, so a session can
+be resumed cold. The long-form plans live in `BACKPORT_PLAN.md` (engine, cspurefix →
+jspurefix) and `DEMO_PORT_PLAN.md` (the jspf-demo reference app).
+
+---
+
+## 2026-08-01 — acceptor hardening, released as 5.9.1
+
+Started from [#153](https://github.com/TimelordUK/jspurefix/issues/153) and ended up
+rewriting how the acceptor handles connections.
+
+**Merged (PRs #154, #156, #157, #158):**
+
+- transport subscribe/unsubscribe symmetry — `subscribe()` attached closures,
+  `unsubscribe()` removed method references, so it removed nothing. A dead transport
+  could drive a session that had moved to a newer connection.
+- per-connection DI scopes — `ParseBuffer` / `TransmitBuffer` were process-wide
+  singletons and the description was shared, so concurrent clients interleaved bytes
+  in one buffer and computed one `SessionId`
+- `SessionRegistry` — one live session per `SessionId`, stopping the stale one on
+  reconnect
+- wildcard `TargetCompID` — bound after the Logon, so each counterparty gets its own
+  store (cspurefix binds in the constructor and keys a wildcard store on `*`)
+- transport lifecycle — keep-alive, `destroy()`, linger, harvest on `close`, census
+  logging
+- custom session message factories preserved into per-session scopes (`cloneFor`)
+- `@jest/globals` moved out of `dependencies` — it was never imported, and shipped
+  the Babel toolchain to every consumer. Production tree 303 → 147 packages.
+- `npm run pack:local` for consuming unpublished builds
+
+**Issues resolved:** #153, #87, #140 (verified and closed), #77 (awaiting reporter
+confirmation — see below).
+
+**Docs:** `docs/acceptor.md` describes the resulting model.
+
+---
+
+## Next up
+
+### 1. Toolchain major bumps
+
+Deliberately held back from 5.9.1 so a regression could not be confused with the
+acceptor work:
+
+- `typescript` 6 → 7
+- `eslint` 9 → 10
+- `eslint-config-love` 151 → 154
+- `@types/node` 25 → 26
+
+Expect lint churn. There are 17 pre-existing lint errors on master (`fix-clock`,
+`resend-request-manager`, `session-sequence-*`, `tcp-initiator`) — worth clearing in
+the same pass, since they make it hard to tell new problems from old. `npm audit` is
+currently clean.
+
+### 2. #77 — confirm or close
+
+[#77](https://github.com/TimelordUK/jspurefix/issues/77) is very likely fixed by the
+per-connection parse buffers in 5.9.1, not by the field-ordering work the issue is
+named after. Both orderings of the reporter's message parse correctly today, and
+they always did — a test on the `fix 52` branch in 2023 also parsed the swapped
+message, which is why the issue stalled.
+
+The better explanation: until 5.9.0 all accepted connections shared one `ParseBuffer`
+while each parser kept its own state, and `beginMessage()` resets the shared buffer.
+Sweeping every split point of the reporter's MarketDataRequest, a shared buffer
+corrupts it at 170 of 170 — the trigger is one connection being mid-message, which
+needs a message that straddles a TCP segment, which is why small session messages
+never expose it. The reporter's deployment (per #87) was a multi-client acceptor
+publishing market data.
+
+Pinned in `src/test/ascii/interleaved-parse-isolation.test.ts`. Awaiting the
+reporter running 5.9.1 before closing — the exact `unknown tag type 146` was never
+reproduced.
+
+### 3. Other open issues worth a look
+
+Several are old and may already be fixed, in the way #140 turned out to be — worth
+verifying against 5.9.1 rather than reading the thread:
+
+- #96 / #69 — `DefaultApplVerID` / `SenderSubID` missing during login
+- #93 — customising the Logon message
+- #86 — make `encoderStream` protected
+- #85 — `ASessionMsgFactory` types patch (may be superseded by `cloneFor`)
+- #72 — `session-launcher.run()` never resolves when connection established
+
+### 4. Small things noticed, none urgent
+
+- on an acceptor, the parent container's `ParseBuffer` / `TransmitBuffer` are still
+  allocated and now serve only as a size template for the per-connection clones —
+  ~170KB of dead allocation per listener
+- `script/certs/` is untracked and contains private keys; worth a `.gitignore` entry
+  so it cannot be committed by accident
+- per-connection buffers cost ~170KB per connection (160KB parse + 10KB transmit).
+  Verified released on disconnect via `WeakRef` — a tsyringe parent keeps no
+  reference to its children — but it is the number to know if someone runs hundreds
+  of counterparties
+
+---
+
+## Corrected earlier note
+
+An earlier version of this work recorded the demo's `server-bounce` scenario as
+failing because of a fire-and-forget store write in `AsciiSession.txOnEncoded`
+leaving the acceptor's persisted sender sequence behind what it had sent.
+
+**That was wrong.** Reproducing the run by hand: after phase 1 the two stores agree
+exactly (client `4 : 14`, server `14 : 4`), and a second session started against them
+recovers and trades normally, including with the scenario's tight 5-second client
+budget.
+
+The real cause was in the scenario harness — `run_quiet_bg` published its pid via
+command substitution, so `$!` belonged to a subshell and `wait` failed instantly
+instead of blocking. The first server was therefore still bound to the port when the
+second started. Fixed in jspf-demo; all six scenarios now pass.
+
+Worth remembering as a general lesson: a `|| true` on a `wait` hid this for months,
+and the failure looked exactly like an engine sequencing bug.

package/dist/benchmark/bench-report.d.ts

@@ -0,0 +1,5 @@
+import { IBenchResult, IBenchSuiteResult } from './bench-types';
+export declare function formatEnvironment(suite: IBenchSuiteResult): string;
+export declare function formatResult(result: IBenchResult): string;
+export declare function formatSuite(suite: IBenchSuiteResult): string;
+export declare function formatComparison(baseline: IBenchSuiteResult, candidate: IBenchSuiteResult): string;

package/dist/benchmark/bench-report.js

@@ -0,0 +1,91 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.formatEnvironment = formatEnvironment;
+exports.formatResult = formatResult;
+exports.formatSuite = formatSuite;
+exports.formatComparison = formatComparison;
+const bench_stats_1 = require("./bench-stats");
+const bytesPerKb = 1024;
+const bytesPerMb = bytesPerKb * 1024;
+const nanosPerMicro = 1e3;
+function micros(nanos) {
+    return (nanos / nanosPerMicro).toFixed(3);
+}
+function mb(bytes) {
+    return (bytes / bytesPerMb).toFixed(2);
+}
+function thousands(n) {
+    return Math.round(n).toLocaleString('en-US');
+}
+function signed(n, digits = 2) {
+    return `${n >= 0 ? '+' : ''}${n.toFixed(digits)}`;
+}
+function formatEnvironment(suite) {
+    const env = suite.environment;
+    const lines = [
+        `${suite.label}  ${suite.capturedAt}`,
+        `node ${env.node}  v8 ${env.v8}  ${env.platform} ${env.arch}`,
+        `${env.cpu} x${env.cpuCount}`
+    ];
+    if (!env.forcedGcAvailable) {
+        lines.push('WARNING: no --expose-gc, rounds are not isolated and retained heap is indicative only');
+    }
+    return lines.join('\n');
+}
+function formatResult(result) {
+    const t = result.timing;
+    const m = result.memory;
+    const noise = t.coefficientOfVariation * 100;
+    const lines = [
+        `${result.name}`,
+        result.description ? `  ${result.description}` : '',
+        `  rounds        ${t.samples} x ${result.options.operationsPerRound} msgs` +
+            ` (warmup ${result.options.warmupOperations})`,
+        `  per msg       median ${micros(t.median)}us   min ${micros(t.min)}us` +
+            `   p95 ${micros(t.p95)}us   max ${micros(t.max)}us`,
+        `  throughput    ${thousands(result.operationsPerSecond)} msg/sec`,
+        `  noise         ${noise.toFixed(2)}% of median (a change smaller than this means nothing)`,
+        `  gc pass       ${m.operations} msgs uninterrupted: ${m.gc.minor} minor  ${m.gc.major} major  ` +
+            `${m.gc.totalPauseMs.toFixed(1)}ms paused  longest ${m.gc.longestPauseMs.toFixed(2)}ms`,
+        `  gc rate       ${m.minorGcPerMillionOperations.toFixed(1)} minor/M msgs   ` +
+            `${m.majorGcPerMillionOperations.toFixed(1)} major/M msgs   ` +
+            `${m.gcPauseMsPerMillionOperations.toFixed(0)}ms paused/M msgs`,
+        `  retained      ${mb(m.retainedHeapBytes)}MB   ` +
+            `${m.retainedBytesPerOperation.toFixed(1)} bytes/msg   heap at end of pass ${mb(m.heapAtEndOfPassBytes)}MB`
+    ];
+    return lines.filter(l => l.length > 0).join('\n');
+}
+function formatSuite(suite) {
+    const blocks = suite.results.map(formatResult);
+    return [formatEnvironment(suite), '', ...blocks].join('\n\n');
+}
+function formatComparison(baseline, candidate) {
+    const lines = [`baseline ${baseline.label} (${baseline.capturedAt})`, ''];
+    if (baseline.environment.cpu !== candidate.environment.cpu ||
+        baseline.environment.node !== candidate.environment.node) {
+        lines.push('WARNING: baseline was taken on a different machine or node build, timings are not comparable');
+        lines.push('');
+    }
+    for (const result of candidate.results) {
+        const before = baseline.results.find(r => r.name === result.name);
+        if (!before) {
+            lines.push(`${result.name}  no baseline`);
+            continue;
+        }
+        const timing = (0, bench_stats_1.compare)(before.timing, result.timing);
+        const verdict = timing.significant
+            ? (timing.deltaPercent < 0 ? 'FASTER' : 'SLOWER')
+            : 'noise';
+        lines.push(`${result.name}`);
+        lines.push(`  per msg   ${micros(before.timing.median)}us -> ${micros(result.timing.median)}us` +
+            `   ${signed(timing.deltaPercent)}%   ${verdict}` +
+            `   (noise floor +/-${timing.noiseThresholdPercent.toFixed(2)}%)`);
+        const gcBefore = before.memory.minorGcPerMillionOperations;
+        const gcAfter = result.memory.minorGcPerMillionOperations;
+        const gcDelta = gcBefore === 0 ? 0 : ((gcAfter - gcBefore) / gcBefore) * 100;
+        lines.push(`  minor gc  ${gcBefore.toFixed(1)} -> ${gcAfter.toFixed(1)} per M msgs   ${signed(gcDelta)}%`);
+        lines.push('');
+    }
+    return lines.join('\n');
+}
+//# sourceMappingURL=bench-report.js.map

package/dist/benchmark/bench-report.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"bench-report.js","sourceRoot":"","sources":["../../src/benchmark/bench-report.ts"],"names":[],"mappings":";;AAuBA,8CAWC;AAED,oCAsBC;AAED,kCAGC;AASD,4CA4BC;AApGD,+CAAuC;AAGvC,MAAM,UAAU,GAAG,IAAI,CAAA;AACvB,MAAM,UAAU,GAAG,UAAU,GAAG,IAAI,CAAA;AACpC,MAAM,aAAa,GAAG,GAAG,CAAA;AAEzB,SAAS,MAAM,CAAE,KAAa;IAC5B,OAAO,CAAC,KAAK,GAAG,aAAa,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAA;AAC3C,CAAC;AAED,SAAS,EAAE,CAAE,KAAa;IACxB,OAAO,CAAC,KAAK,GAAG,UAAU,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAA;AACxC,CAAC;AAED,SAAS,SAAS,CAAE,CAAS;IAC3B,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,OAAO,CAAC,CAAA;AAC9C,CAAC;AAED,SAAS,MAAM,CAAE,CAAS,EAAE,SAAiB,CAAC;IAC5C,OAAO,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAA;AACnD,CAAC;AAED,SAAgB,iBAAiB,CAAE,KAAwB;IACzD,MAAM,GAAG,GAAG,KAAK,CAAC,WAAW,CAAA;IAC7B,MAAM,KAAK,GAAG;QACZ,GAAG,KAAK,CAAC,KAAK,KAAK,KAAK,CAAC,UAAU,EAAE;QACrC,QAAQ,GAAG,CAAC,IAAI,QAAQ,GAAG,CAAC,EAAE,KAAK,GAAG,CAAC,QAAQ,IAAI,GAAG,CAAC,IAAI,EAAE;QAC7D,GAAG,GAAG,CAAC,GAAG,KAAK,GAAG,CAAC,QAAQ,EAAE;KAC9B,CAAA;IACD,IAAI,CAAC,GAAG,CAAC,iBAAiB,EAAE,CAAC;QAC3B,KAAK,CAAC,IAAI,CAAC,uFAAuF,CAAC,CAAA;IACrG,CAAC;IACD,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;AACzB,CAAC;AAED,SAAgB,YAAY,CAAE,MAAoB;IAChD,MAAM,CAAC,GAAG,MAAM,CAAC,MAAM,CAAA;IACvB,MAAM,CAAC,GAAG,MAAM,CAAC,MAAM,CAAA;IACvB,MAAM,KAAK,GAAG,CAAC,CAAC,sBAAsB,GAAG,GAAG,CAAA;IAC5C,MAAM,KAAK,GAAG;QACZ,GAAG,MAAM,CAAC,IAAI,EAAE;QAChB,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,KAAK,MAAM,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE;QACnD,mBAAmB,CAAC,CAAC,OAAO,MAAM,MAAM,CAAC,OAAO,CAAC,kBAAkB,OAAO;YACxE,YAAY,MAAM,CAAC,OAAO,CAAC,gBAAgB,GAAG;QAChD,0BAA0B,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,YAAY,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI;YACrE,UAAU,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,YAAY,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI;QACtD,mBAAmB,SAAS,CAAC,MAAM,CAAC,mBAAmB,CAAC,UAAU;QAClE,mBAAmB,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,wDAAwD;QAC3F,mBAAmB,CAAC,CAAC,UAAU,wBAAwB,CAAC,CAAC,EAAE,CAAC,KAAK,WAAW,CAAC,CAAC,EAAE,CAAC,KAAK,UAAU;YAC9F,GAAG,CAAC,CAAC,EAAE,CAAC,YAAY,CAAC,OAAO,CAAC,CAAC,CAAC,sBAAsB,CAAC,CAAC,EAAE,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI;QACzF,mBAAmB,CAAC,CAAC,2BAA2B,CAAC,OAAO,CAAC,CAAC,CAAC,kBAAkB;YAC3E,GAAG,CAAC,CAAC,2BAA2B,CAAC,OAAO,CAAC,CAAC,CAAC,kBAAkB;YAC7D,GAAG,CAAC,CAAC,6BAA6B,CAAC,OAAO,CAAC,CAAC,CAAC,kBAAkB;QACjE,mBAAmB,EAAE,CAAC,CAAC,CAAC,iBAAiB,CAAC,OAAO;YAC/C,GAAG,CAAC,CAAC,yBAAyB,CAAC,OAAO,CAAC,CAAC,CAAC,oCAAoC,EAAE,CAAC,CAAC,CAAC,oBAAoB,CAAC,IAAI;KAC9G,CAAA;IACD,OAAO,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;AACnD,CAAC;AAED,SAAgB,WAAW,CAAE,KAAwB;IACnD,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC,CAAA;IAC9C,OAAO,CAAC,iBAAiB,CAAC,KAAK,CAAC,EAAE,EAAE,EAAE,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAA;AAC/D,CAAC;AASD,SAAgB,gBAAgB,CAAE,QAA2B,EAAE,SAA4B;IACzF,MAAM,KAAK,GAAa,CAAC,YAAY,QAAQ,CAAC,KAAK,KAAK,QAAQ,CAAC,UAAU,GAAG,EAAE,EAAE,CAAC,CAAA;IACnF,IAAI,QAAQ,CAAC,WAAW,CAAC,GAAG,KAAK,SAAS,CAAC,WAAW,CAAC,GAAG;QACtD,QAAQ,CAAC,WAAW,CAAC,IAAI,KAAK,SAAS,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC;QAC7D,KAAK,CAAC,IAAI,CAAC,8FAA8F,CAAC,CAAA;QAC1G,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAA;IAChB,CAAC;IACD,KAAK,MAAM,MAAM,IAAI,SAAS,CAAC,OAAO,EAAE,CAAC;QACvC,MAAM,MAAM,GAAG,QAAQ,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,IAAI,CAAC,CAAA;QACjE,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,KAAK,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,IAAI,eAAe,CAAC,CAAA;YACzC,SAAQ;QACV,CAAC;QACD,MAAM,MAAM,GAAG,IAAA,qBAAO,EAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,CAAA;QACpD,MAAM,OAAO,GAAG,MAAM,CAAC,WAAW;YAChC,CAAC,CAAC,CAAC,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC;YACjD,CAAC,CAAC,OAAO,CAAA;QACX,KAAK,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC,CAAA;QAC5B,KAAK,CAAC,IAAI,CAAC,eAAe,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,SAAS,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI;YAC7F,MAAM,MAAM,CAAC,MAAM,CAAC,YAAY,CAAC,OAAO,OAAO,EAAE;YACjD,sBAAsB,MAAM,CAAC,qBAAqB,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,CAAA;QACpE,MAAM,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,2BAA2B,CAAA;QAC1D,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,2BAA2B,CAAA;QACzD,MAAM,OAAO,GAAG,QAAQ,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,QAAQ,CAAC,GAAG,QAAQ,CAAC,GAAG,GAAG,CAAA;QAC5E,KAAK,CAAC,IAAI,CAAC,eAAe,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,iBAAiB,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,CAAA;QAC1G,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAA;IAChB,CAAC;IACD,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;AACzB,CAAC","sourcesContent":["import { compare } from './bench-stats'\nimport { IBenchResult, IBenchSuiteResult } from './bench-types'\n\nconst bytesPerKb = 1024\nconst bytesPerMb = bytesPerKb * 1024\nconst nanosPerMicro = 1e3\n\nfunction micros (nanos: number): string {\n  return (nanos / nanosPerMicro).toFixed(3)\n}\n\nfunction mb (bytes: number): string {\n  return (bytes / bytesPerMb).toFixed(2)\n}\n\nfunction thousands (n: number): string {\n  return Math.round(n).toLocaleString('en-US')\n}\n\nfunction signed (n: number, digits: number = 2): string {\n  return `${n >= 0 ? '+' : ''}${n.toFixed(digits)}`\n}\n\nexport function formatEnvironment (suite: IBenchSuiteResult): string {\n  const env = suite.environment\n  const lines = [\n    `${suite.label}  ${suite.capturedAt}`,\n    `node ${env.node}  v8 ${env.v8}  ${env.platform} ${env.arch}`,\n    `${env.cpu} x${env.cpuCount}`\n  ]\n  if (!env.forcedGcAvailable) {\n    lines.push('WARNING: no --expose-gc, rounds are not isolated and retained heap is indicative only')\n  }\n  return lines.join('\\n')\n}\n\nexport function formatResult (result: IBenchResult): string {\n  const t = result.timing\n  const m = result.memory\n  const noise = t.coefficientOfVariation * 100\n  const lines = [\n    `${result.name}`,\n    result.description ? `  ${result.description}` : '',\n    `  rounds        ${t.samples} x ${result.options.operationsPerRound} msgs` +\n      ` (warmup ${result.options.warmupOperations})`,\n    `  per msg       median ${micros(t.median)}us   min ${micros(t.min)}us` +\n      `   p95 ${micros(t.p95)}us   max ${micros(t.max)}us`,\n    `  throughput    ${thousands(result.operationsPerSecond)} msg/sec`,\n    `  noise         ${noise.toFixed(2)}% of median (a change smaller than this means nothing)`,\n    `  gc pass       ${m.operations} msgs uninterrupted: ${m.gc.minor} minor  ${m.gc.major} major  ` +\n      `${m.gc.totalPauseMs.toFixed(1)}ms paused  longest ${m.gc.longestPauseMs.toFixed(2)}ms`,\n    `  gc rate       ${m.minorGcPerMillionOperations.toFixed(1)} minor/M msgs   ` +\n      `${m.majorGcPerMillionOperations.toFixed(1)} major/M msgs   ` +\n      `${m.gcPauseMsPerMillionOperations.toFixed(0)}ms paused/M msgs`,\n    `  retained      ${mb(m.retainedHeapBytes)}MB   ` +\n      `${m.retainedBytesPerOperation.toFixed(1)} bytes/msg   heap at end of pass ${mb(m.heapAtEndOfPassBytes)}MB`\n  ]\n  return lines.filter(l => l.length > 0).join('\\n')\n}\n\nexport function formatSuite (suite: IBenchSuiteResult): string {\n  const blocks = suite.results.map(formatResult)\n  return [formatEnvironment(suite), '', ...blocks].join('\\n\\n')\n}\n\n/**\n * Compare a suite against a stored baseline, matching results by name.\n *\n * A difference is only called out as real when it exceeds the run to run\n * spread both suites displayed. Everything else is reported as noise, because\n * that is what it is.\n */\nexport function formatComparison (baseline: IBenchSuiteResult, candidate: IBenchSuiteResult): string {\n  const lines: string[] = [`baseline ${baseline.label} (${baseline.capturedAt})`, '']\n  if (baseline.environment.cpu !== candidate.environment.cpu ||\n      baseline.environment.node !== candidate.environment.node) {\n    lines.push('WARNING: baseline was taken on a different machine or node build, timings are not comparable')\n    lines.push('')\n  }\n  for (const result of candidate.results) {\n    const before = baseline.results.find(r => r.name === result.name)\n    if (!before) {\n      lines.push(`${result.name}  no baseline`)\n      continue\n    }\n    const timing = compare(before.timing, result.timing)\n    const verdict = timing.significant\n      ? (timing.deltaPercent < 0 ? 'FASTER' : 'SLOWER')\n      : 'noise'\n    lines.push(`${result.name}`)\n    lines.push(`  per msg   ${micros(before.timing.median)}us -> ${micros(result.timing.median)}us` +\n      `   ${signed(timing.deltaPercent)}%   ${verdict}` +\n      `   (noise floor +/-${timing.noiseThresholdPercent.toFixed(2)}%)`)\n    const gcBefore = before.memory.minorGcPerMillionOperations\n    const gcAfter = result.memory.minorGcPerMillionOperations\n    const gcDelta = gcBefore === 0 ? 0 : ((gcAfter - gcBefore) / gcBefore) * 100\n    lines.push(`  minor gc  ${gcBefore.toFixed(1)} -> ${gcAfter.toFixed(1)} per M msgs   ${signed(gcDelta)}%`)\n    lines.push('')\n  }\n  return lines.join('\\n')\n}\n"]}

package/dist/benchmark/bench-runner.d.ts

@@ -0,0 +1,12 @@
+import { IBenchCase, IBenchOptions, IBenchResult, IBenchSuiteResult, IEnvironment } from './bench-types';
+export declare const defaultBenchOptions: IBenchOptions;
+export declare function environment(): IEnvironment;
+export declare class BenchRunner {
+    private readonly options;
+    constructor(options?: IBenchOptions);
+    run(benchCase: IBenchCase): Promise<IBenchResult>;
+    runAll(label: string, cases: IBenchCase[]): Promise<IBenchSuiteResult>;
+    private measureMemory;
+    private warmup;
+    private measureRound;
+}

package/dist/benchmark/bench-runner.js

@@ -0,0 +1,193 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
+    function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
+    return new (P || (P = Promise))(function (resolve, reject) {
+        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
+        function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
+        function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
+        step((generator = generator.apply(thisArg, _arguments || [])).next());
+    });
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.BenchRunner = exports.defaultBenchOptions = void 0;
+exports.environment = environment;
+const os = __importStar(require("os"));
+const gc_probe_1 = require("./gc-probe");
+const bench_stats_1 = require("./bench-stats");
+const nanosPerSecond = 1e9;
+const operationsPerMillion = 1e6;
+exports.defaultBenchOptions = {
+    warmupOperations: 20000,
+    rounds: 10,
+    operationsPerRound: 20000,
+    gcBetweenRounds: true
+};
+function environment() {
+    const cpus = os.cpus();
+    return {
+        node: process.version,
+        v8: process.versions.v8,
+        platform: `${os.type()} ${os.release()}`,
+        arch: process.arch,
+        cpu: cpus.length > 0 ? cpus[0].model.trim() : 'unknown',
+        cpuCount: cpus.length,
+        forcedGcAvailable: gc_probe_1.GcProbe.forcedGcAvailable()
+    };
+}
+function tick() {
+    return __awaiter(this, void 0, void 0, function* () {
+        yield new Promise(resolve => { setImmediate(resolve); });
+    });
+}
+function drain(probe_1) {
+    return __awaiter(this, arguments, void 0, function* (probe, maxTurns = 50) {
+        let quietTurns = 0;
+        for (let turn = 0; turn < maxTurns && quietTurns < 3; ++turn) {
+            const before = probe.seen();
+            yield tick();
+            quietTurns = probe.seen() === before ? quietTurns + 1 : 0;
+        }
+    });
+}
+class BenchRunner {
+    constructor(options = exports.defaultBenchOptions) {
+        this.options = options;
+    }
+    run(benchCase) {
+        return __awaiter(this, void 0, void 0, function* () {
+            var _a, _b;
+            const options = this.options;
+            (_a = benchCase.setup) === null || _a === void 0 ? void 0 : _a.call(benchCase);
+            try {
+                this.warmup(benchCase);
+                const rounds = [];
+                for (let round = 0; round < options.rounds; ++round) {
+                    if (options.gcBetweenRounds) {
+                        gc_probe_1.GcProbe.forceGc();
+                    }
+                    rounds.push(this.measureRound(benchCase, round));
+                }
+                const timing = (0, bench_stats_1.summarise)(rounds.map(r => r.nanosPerOperation));
+                return {
+                    name: benchCase.name,
+                    description: benchCase.description,
+                    options,
+                    rounds,
+                    timing,
+                    operationsPerSecond: timing.median === 0 ? 0 : nanosPerSecond / timing.median,
+                    memory: yield this.measureMemory(benchCase)
+                };
+            }
+            finally {
+                (_b = benchCase.teardown) === null || _b === void 0 ? void 0 : _b.call(benchCase);
+            }
+        });
+    }
+    runAll(label, cases) {
+        return __awaiter(this, void 0, void 0, function* () {
+            const results = [];
+            for (const benchCase of cases) {
+                results.push(yield this.run(benchCase));
+            }
+            return {
+                label,
+                capturedAt: new Date().toISOString(),
+                environment: environment(),
+                results
+            };
+        });
+    }
+    measureMemory(benchCase) {
+        return __awaiter(this, void 0, void 0, function* () {
+            gc_probe_1.GcProbe.forceGc();
+            yield tick();
+            const heapBefore = process.memoryUsage().heapUsed;
+            const probe = new gc_probe_1.GcProbe();
+            probe.start();
+            const operations = benchCase.run(this.options.operationsPerRound);
+            const heapAtEndOfPassBytes = process.memoryUsage().heapUsed;
+            yield drain(probe);
+            const gc = probe.stop();
+            gc_probe_1.GcProbe.forceGc();
+            const heapAfter = process.memoryUsage().heapUsed;
+            const retainedHeapBytes = heapAfter - heapBefore;
+            const perMillion = operations === 0 ? 0 : operationsPerMillion / operations;
+            return {
+                operations,
+                retainedHeapBytes,
+                retainedBytesPerOperation: operations === 0 ? 0 : retainedHeapBytes / operations,
+                heapAtEndOfPassBytes,
+                gc,
+                minorGcPerMillionOperations: gc.minor * perMillion,
+                majorGcPerMillionOperations: gc.major * perMillion,
+                gcPauseMsPerMillionOperations: gc.totalPauseMs * perMillion,
+                forcedGcAvailable: gc_probe_1.GcProbe.forcedGcAvailable()
+            };
+        });
+    }
+    warmup(benchCase) {
+        const target = this.options.warmupOperations;
+        if (target <= 0)
+            return;
+        let done = 0;
+        const chunk = Math.max(1, Math.min(target, this.options.operationsPerRound));
+        while (done < target) {
+            const requested = Math.min(chunk, target - done);
+            const completed = benchCase.run(requested);
+            if (completed !== requested) {
+                throw new Error(`${benchCase.name}: asked for ${requested} operations during warmup but completed ${completed}`);
+            }
+            done += completed;
+        }
+    }
+    measureRound(benchCase, round) {
+        const requested = this.options.operationsPerRound;
+        const startedAt = process.hrtime.bigint();
+        const operations = benchCase.run(requested);
+        const elapsedNanos = Number(process.hrtime.bigint() - startedAt);
+        if (operations !== requested) {
+            throw new Error(`${benchCase.name}: asked for ${requested} operations in round ${round} but completed ${operations}`);
+        }
+        return {
+            round,
+            operations,
+            elapsedNanos,
+            nanosPerOperation: operations === 0 ? 0 : elapsedNanos / operations
+        };
+    }
+}
+exports.BenchRunner = BenchRunner;
+//# sourceMappingURL=bench-runner.js.map

package/dist/benchmark/bench-runner.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"bench-runner.js","sourceRoot":"","sources":["../../src/benchmark/bench-runner.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAyBA,kCAWC;AApCD,uCAAwB;AACxB,yCAAoC;AACpC,+CAAyC;AAWzC,MAAM,cAAc,GAAG,GAAG,CAAA;AAC1B,MAAM,oBAAoB,GAAG,GAAG,CAAA;AAEnB,QAAA,mBAAmB,GAAkB;IAGhD,gBAAgB,EAAE,KAAK;IACvB,MAAM,EAAE,EAAE;IACV,kBAAkB,EAAE,KAAK;IACzB,eAAe,EAAE,IAAI;CACtB,CAAA;AAED,SAAgB,WAAW;IACzB,MAAM,IAAI,GAAG,EAAE,CAAC,IAAI,EAAE,CAAA;IACtB,OAAO;QACL,IAAI,EAAE,OAAO,CAAC,OAAO;QACrB,EAAE,EAAE,OAAO,CAAC,QAAQ,CAAC,EAAE;QACvB,QAAQ,EAAE,GAAG,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,OAAO,EAAE,EAAE;QACxC,IAAI,EAAE,OAAO,CAAC,IAAI;QAClB,GAAG,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,SAAS;QACvD,QAAQ,EAAE,IAAI,CAAC,MAAM;QACrB,iBAAiB,EAAE,kBAAO,CAAC,iBAAiB,EAAE;KAC/C,CAAA;AACH,CAAC;AAGD,SAAe,IAAI;;QACjB,MAAM,IAAI,OAAO,CAAO,OAAO,CAAC,EAAE,GAAG,YAAY,CAAC,OAAO,CAAC,CAAA,CAAC,CAAC,CAAC,CAAA;IAC/D,CAAC;CAAA;AAQD,SAAe,KAAK;yDAAE,KAAc,EAAE,WAAmB,EAAE;QACzD,IAAI,UAAU,GAAG,CAAC,CAAA;QAClB,KAAK,IAAI,IAAI,GAAG,CAAC,EAAE,IAAI,GAAG,QAAQ,IAAI,UAAU,GAAG,CAAC,EAAE,EAAE,IAAI,EAAE,CAAC;YAC7D,MAAM,MAAM,GAAG,KAAK,CAAC,IAAI,EAAE,CAAA;YAC3B,MAAM,IAAI,EAAE,CAAA;YACZ,UAAU,GAAG,KAAK,CAAC,IAAI,EAAE,KAAK,MAAM,CAAC,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;QAC3D,CAAC;IACH,CAAC;CAAA;AAgBD,MAAa,WAAW;IACtB,YAA8B,UAAyB,2BAAmB;QAA5C,YAAO,GAAP,OAAO,CAAqC;IAC1E,CAAC;IAEY,GAAG,CAAE,SAAqB;;;YACrC,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,CAAA;YAC5B,MAAA,SAAS,CAAC,KAAK,yDAAI,CAAA;YACnB,IAAI,CAAC;gBACH,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,CAAA;gBACtB,MAAM,MAAM,GAAmB,EAAE,CAAA;gBACjC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,KAAK,EAAE,CAAC;oBACpD,IAAI,OAAO,CAAC,eAAe,EAAE,CAAC;wBAG5B,kBAAO,CAAC,OAAO,EAAE,CAAA;oBACnB,CAAC;oBACD,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,YAAY,CAAC,SAAS,EAAE,KAAK,CAAC,CAAC,CAAA;gBAClD,CAAC;gBACD,MAAM,MAAM,GAAG,IAAA,uBAAS,EAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC,CAAA;gBAC9D,OAAO;oBACL,IAAI,EAAE,SAAS,CAAC,IAAI;oBACpB,WAAW,EAAE,SAAS,CAAC,WAAW;oBAClC,OAAO;oBACP,MAAM;oBACN,MAAM;oBACN,mBAAmB,EAAE,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc,GAAG,MAAM,CAAC,MAAM;oBAC7E,MAAM,EAAE,MAAM,IAAI,CAAC,aAAa,CAAC,SAAS,CAAC;iBAC5C,CAAA;YACH,CAAC;oBAAS,CAAC;gBACT,MAAA,SAAS,CAAC,QAAQ,yDAAI,CAAA;YACxB,CAAC;QACH,CAAC;KAAA;IAEY,MAAM,CAAE,KAAa,EAAE,KAAmB;;YACrD,MAAM,OAAO,GAAmB,EAAE,CAAA;YAClC,KAAK,MAAM,SAAS,IAAI,KAAK,EAAE,CAAC;gBAC9B,OAAO,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,CAAA;YACzC,CAAC;YACD,OAAO;gBACL,KAAK;gBAGL,UAAU,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;gBACpC,WAAW,EAAE,WAAW,EAAE;gBAC1B,OAAO;aACR,CAAA;QACH,CAAC;KAAA;IAUa,aAAa,CAAE,SAAqB;;YAIhD,kBAAO,CAAC,OAAO,EAAE,CAAA;YACjB,MAAM,IAAI,EAAE,CAAA;YAEZ,MAAM,UAAU,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAA;YACjD,MAAM,KAAK,GAAG,IAAI,kBAAO,EAAE,CAAA;YAC3B,KAAK,CAAC,KAAK,EAAE,CAAA;YACb,MAAM,UAAU,GAAG,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAA;YACjE,MAAM,oBAAoB,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAA;YAC3D,MAAM,KAAK,CAAC,KAAK,CAAC,CAAA;YAClB,MAAM,EAAE,GAAG,KAAK,CAAC,IAAI,EAAE,CAAA;YAEvB,kBAAO,CAAC,OAAO,EAAE,CAAA;YACjB,MAAM,SAAS,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAA;YAEhD,MAAM,iBAAiB,GAAG,SAAS,GAAG,UAAU,CAAA;YAChD,MAAM,UAAU,GAAG,UAAU,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,oBAAoB,GAAG,UAAU,CAAA;YAC3E,OAAO;gBACL,UAAU;gBACV,iBAAiB;gBACjB,yBAAyB,EAAE,UAAU,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,iBAAiB,GAAG,UAAU;gBAChF,oBAAoB;gBACpB,EAAE;gBACF,2BAA2B,EAAE,EAAE,CAAC,KAAK,GAAG,UAAU;gBAClD,2BAA2B,EAAE,EAAE,CAAC,KAAK,GAAG,UAAU;gBAClD,6BAA6B,EAAE,EAAE,CAAC,YAAY,GAAG,UAAU;gBAC3D,iBAAiB,EAAE,kBAAO,CAAC,iBAAiB,EAAE;aAC/C,CAAA;QACH,CAAC;KAAA;IAEO,MAAM,CAAE,SAAqB;QACnC,MAAM,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,gBAAgB,CAAA;QAC5C,IAAI,MAAM,IAAI,CAAC;YAAE,OAAM;QACvB,IAAI,IAAI,GAAG,CAAC,CAAA;QAGZ,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,MAAM,EAAE,IAAI,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC,CAAA;QAC5E,OAAO,IAAI,GAAG,MAAM,EAAE,CAAC;YACrB,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC,CAAA;YAChD,MAAM,SAAS,GAAG,SAAS,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;YAG1C,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;gBAC5B,MAAM,IAAI,KAAK,CAAC,GAAG,SAAS,CAAC,IAAI,eAAe,SAAS,2CAA2C,SAAS,EAAE,CAAC,CAAA;YAClH,CAAC;YACD,IAAI,IAAI,SAAS,CAAA;QACnB,CAAC;IACH,CAAC;IAEO,YAAY,CAAE,SAAqB,EAAE,KAAa;QACxD,MAAM,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,kBAAkB,CAAA;QACjD,MAAM,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,CAAA;QACzC,MAAM,UAAU,GAAG,SAAS,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;QAC3C,MAAM,YAAY,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,GAAG,SAAS,CAAC,CAAA;QAChE,IAAI,UAAU,KAAK,SAAS,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CAAC,GAAG,SAAS,CAAC,IAAI,eAAe,SAAS,wBAAwB,KAAK,kBAAkB,UAAU,EAAE,CAAC,CAAA;QACvH,CAAC;QACD,OAAO;YACL,KAAK;YACL,UAAU;YACV,YAAY;YACZ,iBAAiB,EAAE,UAAU,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,GAAG,UAAU;SACpE,CAAA;IACH,CAAC;CACF;AA3HD,kCA2HC","sourcesContent":["import * as os from 'os'\nimport { GcProbe } from './gc-probe'\nimport { summarise } from './bench-stats'\nimport {\n  IBenchCase,\n  IBenchOptions,\n  IBenchResult,\n  IBenchSuiteResult,\n  IEnvironment,\n  IMemoryResult,\n  IRoundResult\n} from './bench-types'\n\nconst nanosPerSecond = 1e9\nconst operationsPerMillion = 1e6\n\nexport const defaultBenchOptions: IBenchOptions = {\n  // v8 needs to see a function run a few thousand times before it stops\n  // interpreting it, so an unwarmed first round measures the wrong code\n  warmupOperations: 20000,\n  rounds: 10,\n  operationsPerRound: 20000,\n  gcBetweenRounds: true\n}\n\nexport function environment (): IEnvironment {\n  const cpus = os.cpus()\n  return {\n    node: process.version,\n    v8: process.versions.v8,\n    platform: `${os.type()} ${os.release()}`,\n    arch: process.arch,\n    cpu: cpus.length > 0 ? cpus[0].model.trim() : 'unknown',\n    cpuCount: cpus.length,\n    forcedGcAvailable: GcProbe.forcedGcAvailable()\n  }\n}\n\n/** hand the event loop one turn, so queued observer callbacks can run */\nasync function tick (): Promise<void> {\n  await new Promise<void>(resolve => { setImmediate(resolve) })\n}\n\n/**\n * Yield until the probe stops seeing new entries. Node dispatches a gc entry on\n * the turn after the collection, and a burst takes several turns to come\n * through, so a single yield loses most of them. Bounded so a pathological case\n * cannot spin here.\n */\nasync function drain (probe: GcProbe, maxTurns: number = 50): Promise<void> {\n  let quietTurns = 0\n  for (let turn = 0; turn < maxTurns && quietTurns < 3; ++turn) {\n    const before = probe.seen()\n    await tick()\n    quietTurns = probe.seen() === before ? quietTurns + 1 : 0\n  }\n}\n\n/**\n * Drives a case through a warmup, a series of timed rounds and a separate\n * diagnostics pass.\n *\n * One round is one sample. A single long run cannot tell you whether a number\n * is repeatable, and a difference smaller than the spread between rounds is not\n * a difference at all.\n *\n * Timing and memory are deliberately not measured together. Isolating rounds\n * means forcing a collection between them, and a forced collection is itself a\n * gc event - counting those would report the harness rather than the work. So\n * the timed rounds collect between them and report no gc figures, and a final\n * pass runs uninterrupted with the probe attached and reports no timings.\n */\nexport class BenchRunner {\n  constructor (private readonly options: IBenchOptions = defaultBenchOptions) {\n  }\n\n  public async run (benchCase: IBenchCase): Promise<IBenchResult> {\n    const options = this.options\n    benchCase.setup?.()\n    try {\n      this.warmup(benchCase)\n      const rounds: IRoundResult[] = []\n      for (let round = 0; round < options.rounds; ++round) {\n        if (options.gcBetweenRounds) {\n          // a collection provoked by the previous round must not be charged\n          // to the next one\n          GcProbe.forceGc()\n        }\n        rounds.push(this.measureRound(benchCase, round))\n      }\n      const timing = summarise(rounds.map(r => r.nanosPerOperation))\n      return {\n        name: benchCase.name,\n        description: benchCase.description,\n        options,\n        rounds,\n        timing,\n        operationsPerSecond: timing.median === 0 ? 0 : nanosPerSecond / timing.median,\n        memory: await this.measureMemory(benchCase)\n      }\n    } finally {\n      benchCase.teardown?.()\n    }\n  }\n\n  public async runAll (label: string, cases: IBenchCase[]): Promise<IBenchSuiteResult> {\n    const results: IBenchResult[] = []\n    for (const benchCase of cases) {\n      results.push(await this.run(benchCase))\n    }\n    return {\n      label,\n      // stamped from the clock rather than passed in, a suite is a record of\n      // when it was taken\n      capturedAt: new Date().toISOString(),\n      environment: environment(),\n      results\n    }\n  }\n\n  /**\n   * One uninterrupted pass with the gc probe attached.\n   *\n   * Nothing is forced while it runs, so every collection counted is one the\n   * work itself provoked. The heap readings either side are taken after a\n   * forced collection, making them what the pass retained rather than what it\n   * churned - churn is what the collection counts are for.\n   */\n  private async measureMemory (benchCase: IBenchCase): Promise<IMemoryResult> {\n    // settle, then let any delivery from that settling drain before the probe\n    // exists - entries are dispatched a turn late and would otherwise land in\n    // the middle of the measured pass\n    GcProbe.forceGc()\n    await tick()\n\n    const heapBefore = process.memoryUsage().heapUsed\n    const probe = new GcProbe()\n    probe.start()\n    const operations = benchCase.run(this.options.operationsPerRound)\n    const heapAtEndOfPassBytes = process.memoryUsage().heapUsed\n    await drain(probe)\n    const gc = probe.stop()\n\n    GcProbe.forceGc()\n    const heapAfter = process.memoryUsage().heapUsed\n\n    const retainedHeapBytes = heapAfter - heapBefore\n    const perMillion = operations === 0 ? 0 : operationsPerMillion / operations\n    return {\n      operations,\n      retainedHeapBytes,\n      retainedBytesPerOperation: operations === 0 ? 0 : retainedHeapBytes / operations,\n      heapAtEndOfPassBytes,\n      gc,\n      minorGcPerMillionOperations: gc.minor * perMillion,\n      majorGcPerMillionOperations: gc.major * perMillion,\n      gcPauseMsPerMillionOperations: gc.totalPauseMs * perMillion,\n      forcedGcAvailable: GcProbe.forcedGcAvailable()\n    }\n  }\n\n  private warmup (benchCase: IBenchCase): void {\n    const target = this.options.warmupOperations\n    if (target <= 0) return\n    let done = 0\n    // in chunks the size of a real round, so warmup exercises the same shape of\n    // work rather than one long unbroken call\n    const chunk = Math.max(1, Math.min(target, this.options.operationsPerRound))\n    while (done < target) {\n      const requested = Math.min(chunk, target - done)\n      const completed = benchCase.run(requested)\n      // a case that reports less than it was asked for would never let this\n      // loop reach its target, so refuse it here rather than spin\n      if (completed !== requested) {\n        throw new Error(`${benchCase.name}: asked for ${requested} operations during warmup but completed ${completed}`)\n      }\n      done += completed\n    }\n  }\n\n  private measureRound (benchCase: IBenchCase, round: number): IRoundResult {\n    const requested = this.options.operationsPerRound\n    const startedAt = process.hrtime.bigint()\n    const operations = benchCase.run(requested)\n    const elapsedNanos = Number(process.hrtime.bigint() - startedAt)\n    if (operations !== requested) {\n      throw new Error(`${benchCase.name}: asked for ${requested} operations in round ${round} but completed ${operations}`)\n    }\n    return {\n      round,\n      operations,\n      elapsedNanos,\n      nanosPerOperation: operations === 0 ? 0 : elapsedNanos / operations\n    }\n  }\n}\n"]}

package/dist/benchmark/bench-stats.d.ts

@@ -0,0 +1,10 @@
+import { ISampleStats } from './bench-types';
+export declare function summarise(values: number[]): ISampleStats;
+export declare function median(sorted: number[]): number;
+export declare function percentile(sorted: number[], p: number): number;
+export interface IComparison {
+    readonly deltaPercent: number;
+    readonly significant: boolean;
+    readonly noiseThresholdPercent: number;
+}
+export declare function compare(baseline: ISampleStats, candidate: ISampleStats): IComparison;

package/dist/benchmark/bench-stats.js

@@ -0,0 +1,56 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.summarise = summarise;
+exports.median = median;
+exports.percentile = percentile;
+exports.compare = compare;
+function summarise(values) {
+    if (values.length === 0) {
+        throw new Error('cannot summarise an empty sample set');
+    }
+    const sorted = [...values].sort((a, b) => a - b);
+    const n = sorted.length;
+    const mean = sorted.reduce((a, v) => a + v, 0) / n;
+    const variance = sorted.reduce((a, v) => a + (v - mean) * (v - mean), 0) / n;
+    const stdDev = Math.sqrt(variance);
+    return {
+        samples: n,
+        min: sorted[0],
+        max: sorted[n - 1],
+        mean,
+        median: median(sorted),
+        p95: percentile(sorted, 95),
+        p99: percentile(sorted, 99),
+        stdDev,
+        coefficientOfVariation: mean === 0 ? 0 : stdDev / mean
+    };
+}
+function median(sorted) {
+    const n = sorted.length;
+    if (n === 0)
+        return 0;
+    const mid = n >> 1;
+    return n % 2 === 1
+        ? sorted[mid]
+        : (sorted[mid - 1] + sorted[mid]) / 2;
+}
+function percentile(sorted, p) {
+    const n = sorted.length;
+    if (n === 0)
+        return 0;
+    const rank = Math.ceil((p / 100) * n);
+    const index = Math.min(Math.max(rank - 1, 0), n - 1);
+    return sorted[index];
+}
+function compare(baseline, candidate) {
+    const deltaPercent = baseline.median === 0
+        ? 0
+        : ((candidate.median - baseline.median) / baseline.median) * 100;
+    const noiseThresholdPercent = (baseline.coefficientOfVariation + candidate.coefficientOfVariation) * 100;
+    return {
+        deltaPercent,
+        significant: Math.abs(deltaPercent) > noiseThresholdPercent,
+        noiseThresholdPercent
+    };
+}
+//# sourceMappingURL=bench-stats.js.map

package/dist/benchmark/bench-stats.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"bench-stats.js","sourceRoot":"","sources":["../../src/benchmark/bench-stats.ts"],"names":[],"mappings":";;AAUA,8BAoBC;AAKD,wBAOC;AASD,gCAMC;AAiBD,0BAWC;AA3ED,SAAgB,SAAS,CAAE,MAAgB;IACzC,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAA;IACzD,CAAC;IACD,MAAM,MAAM,GAAG,CAAC,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;IAChD,MAAM,CAAC,GAAG,MAAM,CAAC,MAAM,CAAA;IACvB,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAA;IAClD,MAAM,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAA;IAC5E,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;IAClC,OAAO;QACL,OAAO,EAAE,CAAC;QACV,GAAG,EAAE,MAAM,CAAC,CAAC,CAAC;QACd,GAAG,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC;QAClB,IAAI;QACJ,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC;QACtB,GAAG,EAAE,UAAU,CAAC,MAAM,EAAE,EAAE,CAAC;QAC3B,GAAG,EAAE,UAAU,CAAC,MAAM,EAAE,EAAE,CAAC;QAC3B,MAAM;QACN,sBAAsB,EAAE,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,GAAG,IAAI;KACvD,CAAA;AACH,CAAC;AAKD,SAAgB,MAAM,CAAE,MAAgB;IACtC,MAAM,CAAC,GAAG,MAAM,CAAC,MAAM,CAAA;IACvB,IAAI,CAAC,KAAK,CAAC;QAAE,OAAO,CAAC,CAAA;IACrB,MAAM,GAAG,GAAG,CAAC,IAAI,CAAC,CAAA;IAClB,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC;QAChB,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC;QACb,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAA;AACzC,CAAC;AASD,SAAgB,UAAU,CAAE,MAAgB,EAAE,CAAS;IACrD,MAAM,CAAC,GAAG,MAAM,CAAC,MAAM,CAAA;IACvB,IAAI,CAAC,KAAK,CAAC;QAAE,OAAO,CAAC,CAAA;IACrB,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,CAAC,CAAA;IACrC,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAA;IACpD,OAAO,MAAM,CAAC,KAAK,CAAC,CAAA;AACtB,CAAC;AAiBD,SAAgB,OAAO,CAAE,QAAsB,EAAE,SAAuB;IACtE,MAAM,YAAY,GAAG,QAAQ,CAAC,MAAM,KAAK,CAAC;QACxC,CAAC,CAAC,CAAC;QACH,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,MAAM,GAAG,QAAQ,CAAC,MAAM,CAAC,GAAG,QAAQ,CAAC,MAAM,CAAC,GAAG,GAAG,CAAA;IAElE,MAAM,qBAAqB,GAAG,CAAC,QAAQ,CAAC,sBAAsB,GAAG,SAAS,CAAC,sBAAsB,CAAC,GAAG,GAAG,CAAA;IACxG,OAAO;QACL,YAAY;QACZ,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,GAAG,qBAAqB;QAC3D,qBAAqB;KACtB,CAAA;AACH,CAAC","sourcesContent":["import { ISampleStats } from './bench-types'\n\n/**\n * Descriptive statistics over a set of measured rounds.\n *\n * min matters as much as median here. A round can only ever be made slower by\n * interference - a collection landing mid round, the scheduler taking the core\n * away - so the fastest round is the closest reading to the true cost of the\n * work, while the median tells you what to expect in practice.\n */\nexport function summarise (values: number[]): ISampleStats {\n  if (values.length === 0) {\n    throw new Error('cannot summarise an empty sample set')\n  }\n  const sorted = [...values].sort((a, b) => a - b)\n  const n = sorted.length\n  const mean = sorted.reduce((a, v) => a + v, 0) / n\n  const variance = sorted.reduce((a, v) => a + (v - mean) * (v - mean), 0) / n\n  const stdDev = Math.sqrt(variance)\n  return {\n    samples: n,\n    min: sorted[0],\n    max: sorted[n - 1],\n    mean,\n    median: median(sorted),\n    p95: percentile(sorted, 95),\n    p99: percentile(sorted, 99),\n    stdDev,\n    coefficientOfVariation: mean === 0 ? 0 : stdDev / mean\n  }\n}\n\n/**\n * @param sorted values in ascending order\n */\nexport function median (sorted: number[]): number {\n  const n = sorted.length\n  if (n === 0) return 0\n  const mid = n >> 1\n  return n % 2 === 1\n    ? sorted[mid]\n    : (sorted[mid - 1] + sorted[mid]) / 2\n}\n\n/**\n * Nearest rank percentile - no interpolation, so every value reported is a\n * reading that actually occurred.\n *\n * @param sorted values in ascending order\n * @param p percentile in the range 0 to 100\n */\nexport function percentile (sorted: number[], p: number): number {\n  const n = sorted.length\n  if (n === 0) return 0\n  const rank = Math.ceil((p / 100) * n)\n  const index = Math.min(Math.max(rank - 1, 0), n - 1)\n  return sorted[index]\n}\n\nexport interface IComparison {\n  /** positive means the candidate took longer than the baseline */\n  readonly deltaPercent: number\n  /**\n   * true when the change is larger than the noise both runs displayed. Below\n   * this threshold a difference is not evidence of anything.\n   */\n  readonly significant: boolean\n  readonly noiseThresholdPercent: number\n}\n\n/**\n * Compare a candidate timing against a baseline, using the run to run spread\n * each of them showed to decide whether the difference is real.\n */\nexport function compare (baseline: ISampleStats, candidate: ISampleStats): IComparison {\n  const deltaPercent = baseline.median === 0\n    ? 0\n    : ((candidate.median - baseline.median) / baseline.median) * 100\n  // both runs contribute their own noise to any difference between them\n  const noiseThresholdPercent = (baseline.coefficientOfVariation + candidate.coefficientOfVariation) * 100\n  return {\n    deltaPercent,\n    significant: Math.abs(deltaPercent) > noiseThresholdPercent,\n    noiseThresholdPercent\n  }\n}\n"]}