parseman 0.45.0 → 0.47.0

package/README.md

@@ -11,7 +11,8 @@ usually mean grammar files, generated code, and extra tooling. Parséman is a co
 library with an optional compiler, and it gives you both.
 
 **Parsing to JS values, the macro build is the fastest general-purpose JS parser in the
-suite** — ahead of every other library measured, at every grammar and every input size.
+suite** — ahead of every other library measured, at every grammar and every input size in
+that suite.
 Every parser in the suite builds real output: objects, row arrays, AST nodes. On a 7.7 kB
 GraphQL document Parséman takes **131 µs**; Peggy takes 328 µs. Only a purpose-built native
 edges it out, `JSON.parse` on JSON.
@@ -138,8 +139,9 @@ See **[The three modes](https://matthew-dean.github.io/parseman/guide/modes)**.
   coverage ("which rules and choice arms ran?") and trace ("what did it try, select, and
   backtrack through?").
 - 🚦 **[First-char gating diagnostics](https://matthew-dean.github.io/parseman/guide/first-char-gating)** —
-  the build tells you which `choice` lost its O(1) dispatch, names the overlapping arms,
-  and says how to fix it.
+  `diagnoseGrammar()` tells you which `choice` lost its O(1) dispatch, names the
+  overlapping arms, and says how to fix it. Run it in a test or a CI job — compiling
+  reports nothing.
 - 📈 **[Performance guide](https://matthew-dean.github.io/parseman/guide/performance)** —
   the levers that actually move a grammar, and [how regexes lower](https://matthew-dean.github.io/parseman/guide/regex-lowering).
 
@@ -161,6 +163,9 @@ GraphQL **131 µs** vs Peggy's 328 µs, JSON **133 µs** vs Chevrotain's 241 µs
 **75.3 µs** vs Peggy's 420 µs. Native `JSON.parse` does JSON large in 51.6 µs. On the CST
 chart, macro build runs 174 µs against Lezer's 594 µs parse-only.
 
+Those are the committed charts, regenerated at 0.29.0 (2026-07-22) on an M4 Pro, measuring
+the JS-codegen lowering that `compile()` and the macro build emit.
+
 ![JSON parsing benchmarks](https://raw.githubusercontent.com/matthew-dean/parseman/main/assets/bench-json.svg)
 
 ![CSV parsing benchmarks](https://raw.githubusercontent.com/matthew-dean/parseman/main/assets/bench-csv.svg)

package/dist/analysis/choice-cost-gate.d.ts

@@ -0,0 +1,158 @@
+/**
+ * CHOICE COST — the policy layer.
+ * ===============================
+ *
+ * Analysis says what the grammar costs. This says which of those numbers fail a build.
+ * They are separate because they have different consumers and different lifetimes: a
+ * developer wants the ranked list on every run, CI wants one bit, and the threshold
+ * that produces that bit is a decision the owner makes once and writes down.
+ *
+ * This module is PURE — it takes a report and a baseline and returns a verdict. It
+ * reads no files, spawns nothing, and prints nothing, so it is testable without a
+ * fixture checkout and cannot be the reason a gate is flaky.
+ *
+ * THE BASELINE IS ABSOLUTE, NOT DIFFERENTIAL
+ * ------------------------------------------
+ * Gating each commit against its parent lets +2% per commit land forever: every
+ * individual step is under tolerance and the total is unbounded. So the committed
+ * baseline records the ACTUAL byte counts, and drift is measured against those
+ * numbers, not against yesterday's.
+ *
+ * A baseline is committed DATA, reviewed like any other diff. Nothing here writes one.
+ * Rebaselining is a deliberate act with a reviewable artifact and owner sign-off; there
+ * is no automatic refresh, because a gate that rebaselines itself is a gate that
+ * records regressions rather than catching them.
+ *
+ * AND IT IS TWO-SIDED — A WIN MUST BE BANKED
+ * ------------------------------------------
+ * Growth past the committed number fails, and so does an unbanked IMPROVEMENT. This is
+ * the half usually left to a comment asking a human politely, and it is the half that
+ * rots: `bench/grammar-density/config.json` and `bench/workloads/config.json` each
+ * carried exactly such a comment and sat unbumped for TEN releases. If a reorder takes
+ * a site from 7,846 wasted bytes to 2,000 and the baseline does not move with it, 5,846
+ * bytes of fresh headroom silently become budget for the next regression — and the gate
+ * that was supposed to catch that regression reads green through the whole of it.
+ *
+ * So the committed number is a BAND, not a floor: leaving it in either direction fails,
+ * and only the remedy differs. Raising a number needs owner sign-off; lowering one is
+ * mandatory. Both are the same one-line rebaseline and the same reviewable diff.
+ *
+ * `bench/size-guard.ts` is the shape this follows — it earned the tightness in 0.45 by
+ * catching +0.14% moves a 1% tolerance would have waved through.
+ *
+ * A BASELINE CANNOT LAUNDER A CEILING
+ * -----------------------------------
+ * If a ceiling is configured, a baseline that records a number ABOVE it is rejected as
+ * invalid rather than honoured. Otherwise the ceiling would be waivable by rebaselining,
+ * which makes it a suggestion. (`bench/size-guard.ts` validates its own baseline for
+ * exactly this reason; this follows it.)
+ *
+ * FAILS CLOSED
+ * ------------
+ * Every way of not having measured is a failure, never a pass: a missing baseline, a
+ * baseline of the wrong shape, an empty baseline, a report over zero corpus files, a
+ * report where no site was instrumentable, a report whose grammar walk was incomplete
+ * (`unresolvedRoots` non-empty), a baselined key that no longer exists, and a measured
+ * key with no baseline entry. This repo already contains the alternative — `ratio:
+ * ordered.length === 0 ? 1` in src/coverage.ts reports 100% covered when nothing was
+ * analysable — and the whole point of a gate is to not be that.
+ */
+import type { WastedWorkReport } from './choice-cost.ts';
+export type WastedWorkBaseline = {
+    readonly schema: 'parseman.wasted-work-baseline/1';
+    /** Informational: which revision produced the numbers. Never compared. */
+    gitRev: string;
+    /** Informational, `YYYY-MM-DD`. Never compared — comparing it would make the
+     *  verdict depend on the clock. */
+    updatedAt: string;
+    /**
+     * Absolute per-corpus totals, keyed by corpus id.
+     *
+     * `totalWastedBytes` is the INTERPRETED column, and that is deliberate even though
+     * `WastedWorkReport` names `totalGatedWastedBytes` as the headline. For BYTES the two
+     * are the same number, structurally: the modelled first-char guard is derived from the
+     * arm's first SET, which over-approximates what the arm can start with, so wherever the
+     * guard rejects, the arm's own leading terminal would have rejected at the same position
+     * having consumed nothing. Measured over four dialect grammars, zero arms differ (see the
+     * header of `choice-cost.ts`). What the guard removes is ATTEMPTS, not rescanned bytes —
+     * so `gatedAttempts`, `gatedFailures` and the inversion ranking DO read the gated columns,
+     * and only the byte totals are recorded from the interpreted one, where it is additionally
+     * the conservative UPPER bound of the two.
+     *
+     * That identity is not enforced anywhere it could be relied on silently: `checkWastedWork`
+     * judges the compiled column against this same band wherever the two columns part, so a
+     * divergence goes red instead of unnoticed.
+     */
+    totals: Record<string, {
+        corpusBytes: number;
+        totalWastedBytes: number;
+        instrumentedSites: number;
+    }>;
+    /** Absolute per-site wasted bytes, keyed by `<corpus>::<siteKey>`. Interpreted column,
+     *  for the reason given on `totals` above. */
+    sites: Record<string, number>;
+};
+export type WastedWorkPolicy = {
+    /**
+     * Maximum wasted bytes per corpus byte. A hard ceiling no rebaseline can waive.
+     * Omit for drift-only gating; a project should set it once it knows its own number.
+     */
+    ceilingRatio?: number;
+    /**
+     * Half-width of the band around each committed number, as a percentage. Default 1.
+     *
+     * SYMMETRIC: a measurement more than this ABOVE its baseline is a regression, and a
+     * measurement more than this BELOW it is an unbanked win. Both fail.
+     *
+     * This is not a noise allowance — the metric is a deterministic count of input bytes,
+     * so its noise floor is exactly zero and any non-zero value here is pure headroom.
+     * It exists only so incidental one-byte churn does not thrash the baseline in both
+     * directions. Set it as near zero as the corpus permits; `bench/choice-cost-guard.ts`
+     * runs at 0.1, matching `bench/size-guard.ts`'s ratchet slack.
+     */
+    driftTolerancePct?: number;
+    /**
+     * Fail on an ordering inversion — an arm that failed every one of its attempts while
+     * a later arm at the same site matched. Off by default: on an existing grammar this
+     * fires immediately, and a gate that is red on arrival for a pre-existing condition
+     * gets disabled rather than fixed. Turn it on once the backlog is empty, to keep it
+     * empty.
+     */
+    failOnInversions?: boolean;
+};
+export type GateBreach = {
+    /** `drift` is growth past the band; `shrank` is an improvement that was not banked.
+     *  Both are failures, and the `detail` says which remedy applies. */
+    kind: 'ceiling' | 'drift' | 'shrank' | 'unbaselined' | 'stale' | 'inversion' | 'unmeasurable' | 'invalid-baseline';
+    key: string;
+    /** One line. Says what happened and what number to look at — never advice. */
+    detail: string;
+};
+export type GateVerdict = {
+    readonly schema: 'parseman.wasted-work-verdict/1';
+    ok: boolean;
+    /** Ascending by (kind, key). Deterministic, so a verdict is diffable. */
+    breaches: readonly GateBreach[];
+    checkedCorpora: number;
+    checkedSites: number;
+};
+/**
+ * Judge one or more measured corpora against a committed baseline.
+ *
+ * `reports` is keyed by corpus id — a grammar is usually gated over more than one body
+ * of input, and a single blended number would let a regression on one hide behind an
+ * improvement on another.
+ */
+export declare function checkWastedWork(reports: Readonly<Record<string, WastedWorkReport>>, baseline: unknown, policy?: WastedWorkPolicy): GateVerdict;
+/**
+ * Build the baseline a passing measurement would record.
+ *
+ * Deliberately NOT called by the gate. It exists so a rebaseline is one explicit
+ * command whose output is a file in the diff — the reviewable record — rather than
+ * something the gate can do to itself on a red run.
+ */
+export declare function buildWastedWorkBaseline(reports: Readonly<Record<string, WastedWorkReport>>, meta: {
+    gitRev: string;
+    updatedAt: string;
+}): WastedWorkBaseline;
+//# sourceMappingURL=choice-cost-gate.d.ts.map

package/dist/analysis/choice-cost-gate.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"choice-cost-gate.d.ts","sourceRoot":"","sources":["../../src/analysis/choice-cost-gate.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA0DG;AAEH,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAA;AAExD,MAAM,MAAM,kBAAkB,GAAG;IAC/B,QAAQ,CAAC,MAAM,EAAE,iCAAiC,CAAA;IAClD,0EAA0E;IAC1E,MAAM,EAAE,MAAM,CAAA;IACd;uCACmC;IACnC,SAAS,EAAE,MAAM,CAAA;IACjB;;;;;;;;;;;;;;;;;OAiBG;IACH,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE;QAAE,WAAW,EAAE,MAAM,CAAC;QAAC,gBAAgB,EAAE,MAAM,CAAC;QAAC,iBAAiB,EAAE,MAAM,CAAA;KAAE,CAAC,CAAA;IACpG;kDAC8C;IAC9C,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;CAC9B,CAAA;AAED,MAAM,MAAM,gBAAgB,GAAG;IAC7B;;;OAGG;IACH,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB;;;;;;;;;;;OAWG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAA;IAC1B;;;;;;OAMG;IACH,gBAAgB,CAAC,EAAE,OAAO,CAAA;CAC3B,CAAA;AAED,MAAM,MAAM,UAAU,GAAG;IACvB;yEACqE;IACrE,IAAI,EAAE,SAAS,GAAG,OAAO,GAAG,QAAQ,GAAG,aAAa,GAAG,OAAO,GAAG,WAAW,GAAG,cAAc,GAAG,kBAAkB,CAAA;IAClH,GAAG,EAAE,MAAM,CAAA;IACX,8EAA8E;IAC9E,MAAM,EAAE,MAAM,CAAA;CACf,CAAA;AAED,MAAM,MAAM,WAAW,GAAG;IACxB,QAAQ,CAAC,MAAM,EAAE,gCAAgC,CAAA;IACjD,EAAE,EAAE,OAAO,CAAA;IACX,yEAAyE;IACzE,QAAQ,EAAE,SAAS,UAAU,EAAE,CAAA;IAC/B,cAAc,EAAE,MAAM,CAAA;IACtB,YAAY,EAAE,MAAM,CAAA;CACrB,CAAA;AAQD;;;;;;GAMG;AACH,wBAAgB,eAAe,CAC7B,OAAO,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,gBAAgB,CAAC,CAAC,EACnD,QAAQ,EAAE,OAAO,EACjB,MAAM,GAAE,gBAAqB,GAC5B,WAAW,CAkLb;AAED;;;;;;GAMG;AACH,wBAAgB,uBAAuB,CACrC,OAAO,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,gBAAgB,CAAC,CAAC,EACnD,IAAI,EAAE;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAA;CAAE,GAC1C,kBAAkB,CAWpB"}

package/dist/analysis/choice-cost-render.d.ts

@@ -0,0 +1,96 @@
+/**
+ * CHOICE COST — the human layer.
+ * ==============================
+ *
+ * Three layers, not two:
+ *
+ *   ANALYSIS   src/analysis/choice-cost.ts  — what the sites are and what they cost.
+ *   POLICY     bench/choice-cost-guard.ts   — which of those numbers fail a build.
+ *   RENDERING  this file                    — how a person reads either of them.
+ *
+ * The structured report is the source of truth. Nothing here computes a finding, and
+ * nothing here decides pass or fail; a gate consumes the data and never this text. If
+ * a number appears below that is not in the report, that is a bug in this file.
+ *
+ * WHAT A PARSER GENERATOR CAN SHOW THAT A COMPILER CANNOT
+ * -------------------------------------------------------
+ * A type error has one source to point at. A wasted-work finding has TWO, and relating
+ * them is the whole diagnostic:
+ *
+ *   - the GRAMMAR site — the ordered choice whose arm ordering costs the time
+ *   - the COST — measured, over a real corpus, per alternative
+ *
+ * So the ordering is printed WITH its measured cost beside each arm, which makes the
+ * fix self-evident without a paragraph explaining it:
+ *
+ *   StylesheetAtRule › dispatch[0]              1.31 MB rescanned
+ *     0  RoutedAtRuleStatement    failed 4,182 / 4,271     1.31 MB
+ *     1  RoutedLayerBlock         matched 4,182
+ *
+ * Reading that, nobody needs to be told what to do.
+ *
+ * QUIET BY DEFAULT
+ * ----------------
+ * Nothing in this module prints. It returns strings. There is no console call and no
+ * process-wide default-on channel, deliberately: importing `examples/css/parser.ts`
+ * already emits roughly sixty lines of gating advice before any user code runs, and a
+ * beautiful diagnostic that nobody reads because it is buried in build noise is not
+ * beautiful. This one is available on request — from the CLI gate, or by calling it —
+ * and silent otherwise.
+ *
+ * DETERMINISM BINDS THE RENDERING TOO
+ * -----------------------------------
+ * The rendering is diffable: it preserves the report's ordering, contains no timings,
+ * no dates, no absolute paths, and colour is opt-in rather than auto-detected, so the
+ * same report always renders to the same bytes. A renderer that sniffed `isTTY` would
+ * produce two different outputs from one report and break exactly that.
+ */
+import type { ChoiceInventoryReport, ChoiceInventoryEntry, WastedWorkReport } from './choice-cost.ts';
+import { groupDigits } from './format-number.ts';
+export type RenderOptions = {
+    /** Rows to show. Default 20. The report always holds all of them. */
+    limit?: number;
+    /**
+     * ANSI colour. Default false — NOT auto-detected from `isTTY`, because a renderer
+     * whose bytes depend on where it is piped cannot be diffed or snapshotted.
+     */
+    color?: boolean;
+};
+/**
+ * Deterministic thousands grouping. `toLocaleString()` is locale-dependent and
+ * would make the output differ between machines — the one thing a gateable
+ * rendering cannot do.
+ *
+ * DEFINED IN `./format-number.ts` and re-exported here because this module's own
+ * callers import it from here. It was a byte-identical second copy: `./terminal.ts`
+ * declared one, this file declared another, and the two were one edit away from
+ * disagreeing about how a report reads on two different screens.
+ */
+export { groupDigits };
+/** Byte counts at a glance. Binary units, fixed to one decimal, so 1,376,256 renders
+ *  as `1.3 MB` on every machine. */
+export declare function bytes(n: number): string;
+/**
+ * The combinator source a left-factoring would produce, ready to paste.
+ *
+ * Offered ONLY for the shape where it is mechanical and total: every arm of the choice
+ * in one group. A partial group needs the author to decide which arms move and in what
+ * order — an ordering decision that changes what the grammar accepts, which no
+ * generated rewrite may make silently.
+ *
+ * This is a PREVIEW. It is not applied, and — until the digest-verification loop
+ * described in `docs/` is wired — it is not yet proven behaviour-preserving, so it is
+ * labelled a candidate rather than offered as a fix. An unverified rewrite presented
+ * as verified is worse than no suggestion at all.
+ */
+export declare function leftFactorPreview(e: ChoiceInventoryEntry): string | null;
+export declare function renderChoiceInventory(r: ChoiceInventoryReport, opts?: RenderOptions): string;
+/**
+ * The ranked list, with each top site expanded into its arm ordering.
+ *
+ * The expansion is the point. A ranked list of site keys tells you WHERE; the arm
+ * breakdown beside it tells you WHY, and "arm 0 failed 4,182 of 4,271 times" is an
+ * argument for moving arm 0 that needs no prose attached.
+ */
+export declare function renderWastedWork(r: WastedWorkReport, opts?: RenderOptions): string;
+//# sourceMappingURL=choice-cost-render.d.ts.map

package/dist/analysis/choice-cost-render.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"choice-cost-render.d.ts","sourceRoot":"","sources":["../../src/analysis/choice-cost-render.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8CG;AAEH,OAAO,KAAK,EACV,qBAAqB,EAAE,oBAAoB,EAAE,gBAAgB,EAC9D,MAAM,kBAAkB,CAAA;AACzB,OAAO,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAA;AAEhD,MAAM,MAAM,aAAa,GAAG;IAC1B,qEAAqE;IACrE,KAAK,CAAC,EAAE,MAAM,CAAA;IACd;;;OAGG;IACH,KAAK,CAAC,EAAE,OAAO,CAAA;CAChB,CAAA;AAID;;;;;;;;;GASG;AACH,OAAO,EAAE,WAAW,EAAE,CAAA;AAEtB;oCACoC;AACpC,wBAAgB,KAAK,CAAC,CAAC,EAAE,MAAM,GAAG,MAAM,CAIvC;AA8CD;;;;;;;;;;;;GAYG;AACH,wBAAgB,iBAAiB,CAAC,CAAC,EAAE,oBAAoB,GAAG,MAAM,GAAG,IAAI,CAOxE;AAID,wBAAgB,qBAAqB,CAAC,CAAC,EAAE,qBAAqB,EAAE,IAAI,GAAE,aAAkB,GAAG,MAAM,CA0ChG;AAID;;;;;;GAMG;AACH,wBAAgB,gBAAgB,CAAC,CAAC,EAAE,gBAAgB,EAAE,IAAI,GAAE,aAAkB,GAAG,MAAM,CAmEtF"}