@remnic/bench 9.3.726 → 9.3.728

package/README.md

@@ -211,7 +211,8 @@ A dedicated benchmark suite for [`@remnic/coding-graph`](https://www.npmjs.com/p
 |---|---|
 | `fullIndexMs` | Wall time to index the entire fixture in one batch. |
 | `fullIndexLocsPerSecond` | Sustained LOC/s during full index (higher is better). |
-| `incrementalUpdateP50Ms` / `incrementalUpdateP95Ms` | Single-file re-ingest latency (p50/p95 over ≥20 iterations). |
+| `incrementalUpdateP50Ms` / `incrementalUpdateP95Ms` | Single-file re-ingest latency for UNCHANGED content (the common-case no-op path; p50/p95 over ≥20 iterations). |
+| `incrementalModifiedUpdateP50Ms` / `incrementalModifiedUpdateP95Ms` | Single-file re-ingest latency for MODIFIED content — the change-heavy path (edge deletion/creation, symbol re-resolution). Complementary to `incrementalUpdate` (#1688). |
 | `tracePathP95Ms` | `trace_path` (BFS, depth ≤ 5) p95. |
 | `searchGraphP95Ms` | `search_graph` name-pattern p95. |
 | `deadCodeMs` | Dead-code query wall time. |
@@ -225,18 +226,21 @@ The harness ships a **deterministic** synthetic repo generator (`generateSynthet
 
 The measured numbers live in [`baselines/coding-graph-baseline.json`](./baselines/coding-graph-baseline.json) — bench-owned, separate from the structural ratchets in `scripts/ratchet-baseline.json`. The regression gate (`checkCodingGraphRegression`) compares a report against the baseline with a generous tolerance (default 30%). It hard-fails on gross regression — a real failing step, not a warning (rule 50). Tightening the baseline is a deliberate PR act (mirrors `check-ratchets --update`).
 
+The gate carries a **machine-fingerprint guard** (`compareMachineFingerprints`): when the report's machine class differs from the baseline's (arch/platform/Node major/cpuModel/cores), the comparison is *skipped* (`passed: true`, `skipped: true`, `machineMismatch` populated) rather than failed, so a cross-machine CI run does not false-positive on legitimate hardware variance (#1688). Regenerate the baseline on the target machine class for a real comparison.
+
 ### Measured numbers (first baseline)
 
 > Numbers below are from the **baseline JSON file** — this section is checked against it so prose can't drift from measurement. Run `remnic bench coding-graph` to reproduce.
 
 | Metric | Value | Machine |
 |---|---|---|
-| Full index | ~17.6 ms, ~115k LOC/s | Apple M2 Max, Node v22 |
-| Incremental update p95 | ~0.26 ms | Apple M2 Max |
-| trace_path p95 | ~0.14 ms | Apple M2 Max |
-| search_graph p95 | ~0.29 ms | Apple M2 Max |
-| dead_code | ~0.67 ms | Apple M2 Max |
-| DB size | ~2 KB/KLOC | Apple M2 Max |
+| Full index | ~15 ms, ~131k LOC/s | Apple M2 Max, Node v22 |
+| Incremental update p95 (idempotent) | ~0.24 ms | Apple M2 Max |
+| Incremental modified update p95 (change-heavy) | ~0.96 ms | Apple M2 Max |
+| trace_path p95 | ~0.13 ms | Apple M2 Max |
+| search_graph p95 | ~0.18 ms | Apple M2 Max |
+| dead_code | ~0.53 ms | Apple M2 Max |
+| DB size | ~270 KB/KLOC | Apple M2 Max |
 
 These are **working targets on a small fixture (20 files, 200 symbols)**, NOT parity claims against codebase-memory-mcp's published numbers (28M LOC in 3 min, <1ms Cypher). Scale targets at 1M+ LOC are tracked as stretch goals — the harness will measure them when Tier-L fixtures are wired (issue #1557 PR2).
 
@@ -245,9 +249,14 @@ These are **working targets on a small fixture (20 files, 200 symbols)**, NOT pa
 ```typescript
 import { runCodingGraphBenchmark, checkCodingGraphRegression } from "@remnic/bench";
 
+// Use the SAME fixture as the bundled baseline (20×10, density 0.3, seed
+// 42) so the regression gate compares like-for-like. A custom fixture
+// (different fileCount/symbolsPerFile/callDensity/seed/language) needs its
+// OWN baseline — build one with buildBaselineFromReport(report, note) and
+// commit it, otherwise checkCodingGraphRegression reports a fixture mismatch.
 const report = await runCodingGraphBenchmark({
-  fixture: { fileCount: 1000, symbolsPerFile: 10, callDensity: 0.2 },
-  iterations: 25,
+  fixture: { seed: 42, fileCount: 20, symbolsPerFile: 10, callDensity: 0.3, language: "typescript" },
+  iterations: 20,
 });
 
 const baseline = require("@remnic/bench/baselines/coding-graph-baseline.json");
@@ -256,4 +265,7 @@ if (!gate.passed) {
   console.error(gate.summary);
   process.exit(1);
 }
+// gate.skipped === true when the report's machine fingerprint differs from
+// the baseline's (different CPU/arch/Node major) — the comparison is skipped
+// rather than failed so cross-machine CI does not false-positive (#1688).
 ```

package/baselines/coding-graph-baseline.json

@@ -1,5 +1,5 @@
 {
-  "schemaVersion": 1,
+  "schemaVersion": 2,
   "machine": {
     "arch": "arm64",
     "platform": "darwin",
@@ -16,15 +16,17 @@
     "language": "typescript"
   },
   "metrics": {
-    "fullIndexMs": 14.864042000000154,
-    "fullIndexLocsPerSecond": 136302,
-    "incrementalUpdateP50Ms": 0.19358299999998962,
-    "incrementalUpdateP95Ms": 0.26237500000002,
-    "tracePathP95Ms": 0.15750000000002728,
-    "searchGraphP95Ms": 0.19700000000011642,
-    "deadCodeMs": 2.29424999999992,
-    "dbBytesPerKloc": 280636
+    "fullIndexMs": 15.452540999999655,
+    "fullIndexLocsPerSecond": 131111,
+    "incrementalUpdateP50Ms": 0.173167000000376,
+    "incrementalUpdateP95Ms": 0.23908399999982066,
+    "incrementalModifiedUpdateP50Ms": 0.6047909999997501,
+    "incrementalModifiedUpdateP95Ms": 0.9565000000002328,
+    "tracePathP95Ms": 0.13425000000006548,
+    "searchGraphP95Ms": 0.1764170000001286,
+    "deadCodeMs": 0.5299169999998412,
+    "dbBytesPerKloc": 270468
   },
-  "createdAt": "2026-07-06T10:50:56.821Z",
-  "note": "Initial baseline — first harness run with WAL-aware DB sizing (issue #1557 PR1)."
+  "createdAt": "2026-07-07T12:35:48.738Z",
+  "note": "Regenerated baseline — adds incrementalModifiedUpdate metric (#1688 item 1) + machine-fingerprint guard (#1688 item 2). WAL-aware DB sizing retained (#1557 PR1)."
 }

package/dist/index.d.ts

@@ -3979,7 +3979,7 @@ interface WallMetric {
  * The regression gate compares only keys present in BOTH the report and
  * the baseline — new metrics are additive, never blocking.
  */
-type CodingGraphMetricKey = "fullIndexMs" | "fullIndexLocsPerSecond" | "incrementalUpdateP50Ms" | "incrementalUpdateP95Ms" | "tracePathP95Ms" | "searchGraphP95Ms" | "deadCodeMs" | "dbBytesPerKloc";
+type CodingGraphMetricKey = "fullIndexMs" | "fullIndexLocsPerSecond" | "incrementalUpdateP50Ms" | "incrementalUpdateP95Ms" | "incrementalModifiedUpdateP50Ms" | "incrementalModifiedUpdateP95Ms" | "tracePathP95Ms" | "searchGraphP95Ms" | "deadCodeMs" | "dbBytesPerKloc";
 interface CodingGraphBenchReport {
     /** Schema version — bump when the report shape changes. */
     readonly schemaVersion: number;
@@ -3996,8 +3996,14 @@ interface CodingGraphBenchReport {
     readonly fullIndexMs: WallMetric;
     /** LOC/s sustained during full index (higher is better). */
     readonly fullIndexLocsPerSecond: number;
-    /** Incremental single-file update latency distribution. */
+    /** Incremental single-file update latency distribution (idempotent
+     *  re-ingest of UNCHANGED content — the common-case no-op path). */
     readonly incrementalUpdate: MicroMetric;
+    /** Incremental MODIFIED-content re-ingest latency distribution. A
+     *  complementary metric to `incrementalUpdate`: measures the change-heavy
+     *  path (edge deletion/creation, symbol re-resolution) rather than the
+     *  cache-hit no-op (#1688 item 1). */
+    readonly incrementalModifiedUpdate: MicroMetric;
     /** trace_path (depth ≤ 5) latency distribution. */
     readonly tracePath: MicroMetric;
     /** search_graph name-pattern latency distribution. */
@@ -4046,6 +4052,24 @@ interface RegressionGateResult {
     readonly passed: boolean;
     readonly regressions: readonly RegressionMetricDetail[];
     readonly summary: string;
+    /**
+     * True when the gate SKIPPED the metric comparison because of a
+     * non-fatal precondition (machine-fingerprint mismatch). `passed` is
+     * also true in that case — a hardware variance must not fail CI (#1688
+     * item 2). Callers that want to treat a skip as informational can check
+     * this flag.
+     */
+    readonly skipped?: boolean;
+    /**
+     * Present when `skipped` is true due to a machine-fingerprint mismatch
+     * between the report and the baseline. Lists the fingerprint fields that
+     * differ so a human can decide whether the comparison is still meaningful.
+     */
+    readonly machineMismatch?: {
+        readonly report: MachineFingerprint;
+        readonly baseline: MachineFingerprint;
+        readonly differingFields: readonly string[];
+    };
 }
 interface CodingGraphBenchConfig {
     /** Fixture parameters. Defaults to a small, fast smoke fixture. */
@@ -4073,7 +4097,7 @@ declare const MIN_ITERATIONS = 20;
 /** Default regression tolerance (issue #1557: generous, e.g. 30%). */
 declare const DEFAULT_TOLERANCE_PERCENT = 30;
 /** Report schema version — bump when the shape changes. */
-declare const CODING_GRAPH_BENCH_SCHEMA_VERSION = 1;
+declare const CODING_GRAPH_BENCH_SCHEMA_VERSION = 2;
 
 /**
  * Deterministic synthetic repo generator (issue #1557 design (a)).
@@ -4169,6 +4193,8 @@ declare const METRIC_DIRECTION: {
     fullIndexLocsPerSecond: "higher-is-better";
     incrementalUpdateP95Ms: "lower-is-better";
     incrementalUpdateP50Ms: "lower-is-better";
+    incrementalModifiedUpdateP95Ms: "lower-is-better";
+    incrementalModifiedUpdateP50Ms: "lower-is-better";
     tracePathP95Ms: "lower-is-better";
     searchGraphP95Ms: "lower-is-better";
     deadCodeMs: "lower-is-better";