@mark1russell7/rigor-bench 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mark Russell
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,69 @@
-# Temporary Holding Version
+# @mark1russell7/rigor-bench
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The paired-time records of a bench, and the time verdict against an A/A arm. This package is the first part of the shared bench harness of rigor.
+
+```sh
+npm install @mark1russell7/rigor-bench
+```
+
+## The method
+
+- Each round runs the two arms of a pair on one case, one after the other. Arm `a` is the baseline, and arm `b` is the candidate.
+- The order turns each round (`firstArm`). Thus a warm cache or a slow start does not always help the same arm.
+- An A/A pair runs the same code in its two arms. Its ratios show how far apart two equal arms come on the machine.
+- The ratio of a case is `b / a` over its rounds (`caseRatios`).
+- The default rule `difference` gives the bootstrap interval of the candidate median over the A/A median. The verdict compares it with 1.
+
+## Give a verdict
+
+```ts
+import { firstArm, timeVerdict, type PairedTime } from "@mark1russell7/rigor-bench";
+
+const first = firstArm(round, caseIndex);         // "a" or "b": run this arm first
+const records: PairedTime[] = [{ case: "board-1", round: 0, first: "a", a: 12.1, b: 10.4 }];
+const v = timeVerdict(records, aaRecords);
+
+v.outcome;                                        // "faster", "slower", "same" or "cannot decide"
+v.reason;                                         // one sentence with the interval and the band
+v.comparison;                                     // the cases that gain and drop beyond the band
+```
+
+## The functions
+
+| Function | What it gives |
+| --- | --- |
+| `firstArm(round, offset)` | The arm that runs first. The order turns each round, and an odd offset starts with arm `b`. |
+| `checkPairedTime(record)` | The record, or a `RangeError` for a time that is not a finite number above 0. |
+| `caseRatios(records)` | The ratio of each case. It refuses 2 records of one round, and an order that did not turn. |
+| `aaBand(aa, options)` | The band of the A/A arm, symmetric on a log scale. The kind `median` (the default) holds the A/A median, and the kind `case` holds one A/A case. It needs 5 cases or more. |
+| `timeVerdict(candidate, aa, options)` | The outcome of the rule `difference` (the default) or `band`. |
+| `verdictPower(options)` | The share of each outcome over seeded benches with a known ratio: the power test of a rule. |
+
+## The outcomes
+
+| Outcome | The rule `difference` (the default) | The rule `band` |
+| --- | --- | --- |
+| `faster` | The whole interval is below 1. | The interval of the median ratio is below the band. |
+| `slower` | The whole interval is above 1. | The interval is above the band. |
+| `same` | The interval is in `[exp(-margin), exp(margin)]`. The default margin is 0.05. | The interval is in the band. |
+| `cannot decide` | The interval holds 1 and crosses the margin. | The interval crosses an end of the band. |
+
+Both rules give `cannot decide` when the A/A arm has fewer than 5 cases, or when the candidate has no cases.
+
+## The power test
+
+`verdictPower` drew 200 seeded benches for each cell. Each case had 5 rounds. The log time of each arm in each round had a normal noise of 0.1. The bootstrap had 1000 runs.
+
+| Rule | Cases | Finds a true speedup of 7% | Calls that speedup `same` | False `faster` or `slower` on equal arms |
+| --- | --- | --- | --- | --- |
+| `band`, kind `case` | 5 | 0.025 | 0.185 | 0 |
+| `band`, kind `case` | 10 | 0 | 0.515 | 0 |
+| `band`, kind `case` | 20 | 0 | 0.755 | 0 |
+| `band`, kind `median` | 5 | 0.025 | 0.22 | 0 |
+| `band`, kind `median` | 10 | 0.045 | 0.1 | 0 |
+| `band`, kind `median` | 20 | 0.22 | 0.035 | 0 |
+| `difference` | 5 | 0.235 | 0 | 0.04 |
+| `difference` | 10 | 0.44 | 0 | 0.045 |
+| `difference` | 20 | 0.71 | 0 | 0.04 |
+
+The rule `band` compares an interval with a band of the same noise, so it needs a change of approximately twice that noise. With the kind `case`, it also calls a real speedup `same`. Thus `difference` is the default. Its false rate is near the 5% of a two-sided interval of level 0.95. A change of 7% needs approximately 20 cases for a power of 0.7.

package/build/index.d.ts

@@ -0,0 +1,148 @@
+import { BootstrapOptions, Comparison, Interval } from "@mark1russell7/rigor-statistics";
+/** The two arms of a pair. Arm `a` is the baseline, and arm `b` is the candidate. */
+type Arm = "a" | "b";
+/** One round of one pair on one case: the times of the two arms in milliseconds, and the arm that ran first. */
+interface PairedTime {
+  /** The name of the case: one input of the bench. */
+  readonly case: string;
+  /** The number of the round, a whole number of 0 or more. */
+  readonly round: number;
+  readonly first: Arm;
+  /** The time of the baseline arm, in milliseconds. */
+  readonly a: number;
+  /** The time of the candidate arm, in milliseconds. */
+  readonly b: number;
+}
+/**
+ * This function gives the arm that runs first in a round. The order turns each round. A case with an odd `offset`
+ * starts with arm `b`, so half of the cases start with each arm.
+ */
+export declare function firstArm(round: number, offset?: number): Arm;
+/** This function examines one record. A time that is not a finite number above 0 is an error. */
+export declare function checkPairedTime(record: PairedTime): PairedTime;
+/** The ratio of one case: the candidate time over the baseline time, over the rounds of the case. */
+interface CaseRatio {
+  readonly case: string;
+  /** The value `exp(median(log(b / a)))` over the rounds. For an odd count of rounds, it is the median of the ratios. */
+  readonly ratio: number;
+  readonly rounds: number;
+}
+/**
+ * This function gives the ratio of each case, in the order of the first record of each case.
+ * It examines each record. It refuses a case with 2 records of one round.
+ * It also refuses a case whose two orders differ in count by more than 1, because that order did not turn each round.
+ */
+export declare function caseRatios(records: readonly PairedTime[]): CaseRatio[];
+/**
+ * The two kinds of an A/A band.
+ * - The band `case` holds the ratio of one A/A case. It is wide, so it finds only a large change.
+ * - The band `median` holds the median ratio of the A/A cases. It has the scale of the median of the candidate.
+ */
+type BandKind = "case" | "median";
+/** The options of the A/A band. */
+interface BandOptions {
+  /** The kind of the band. The default is `median`, from the power test of `verdictPower`. */
+  readonly kind?: BandKind;
+  /** The quantile of the band `case`, or the level of the bootstrap of the band `median`. The default is 0.95. */
+  readonly level?: number;
+  /** The fewest A/A cases that give a band. The default is 5, the 5-seed rule. */
+  readonly minimum?: number;
+  /** The bootstrap of the band `median`. The level of the band replaces its level. */
+  readonly bootstrap?: BootstrapOptions;
+}
+/** The band of the A/A arm: two equal arms come out in this band of ratios. */
+interface Band {
+  /** The count of the A/A cases. */
+  readonly cases: number;
+  readonly kind: BandKind;
+  /**
+   * The half width of the band on a log scale.
+   * For the band `case`, it is the `level` quantile of `abs(log(ratio))` over the A/A cases.
+   * For the band `median`, it is the larger distance of an end of the bootstrap interval of the A/A median from 0.
+   */
+  readonly noise: number;
+  /** The ends of the band: `exp(-noise)` and `exp(noise)`. */
+  readonly low: number;
+  readonly high: number;
+  /** The median ratio of the A/A cases. A value far from 1 shows a bias of the order or of the arm slots. */
+  readonly median: number;
+}
+/** This function gives the band of the A/A arm. The band is symmetric on a log scale. It refuses fewer cases than `minimum`. */
+export declare function aaBand(aa: readonly PairedTime[], options?: BandOptions): Band;
+/** The four outcomes of a time verdict. The candidate is the arm `b`, so `faster` is a ratio below 1. */
+type TimeOutcome = "faster" | "slower" | "same" | "cannot decide";
+/** The options of a time verdict. The bootstrap gives the interval of the median ratio of the candidate, and the band `median`. */
+/**
+ * The two rules of a time verdict.
+ * - The rule `band` compares the interval of the median ratio of the candidate with the A/A band.
+ * - The rule `difference` compares the interval of the candidate median over the A/A median with 1 and with `margin`.
+ */
+type VerdictRule = "band" | "difference";
+/** The options of a time verdict. The bootstrap gives each interval of the verdict. */
+interface VerdictOptions extends BandOptions {
+  /** The rule of the verdict. The default is `difference`, from the power test of `verdictPower`. */
+  readonly rule?: VerdictRule;
+  /** The half width on a log scale of the ratios that the rule `difference` calls `same`. The default is 0.05. */
+  readonly margin?: number;
+}
+/** A time verdict, with the facts that it uses. */
+interface TimeVerdict {
+  readonly outcome: TimeOutcome;
+  /** The reason of the outcome, in one sentence. */
+  readonly reason: string;
+  readonly cases: readonly CaseRatio[];
+  readonly rule: VerdictRule;
+  /**
+   * The interval that the rule compares, or undefined when there are no candidate cases.
+   * With too few A/A cases, it is the interval of the median ratio of the candidate.
+   * For the rule `band`, it is the bootstrap interval of the median ratio of the candidate.
+   * For the rule `difference`, it is the interval of the candidate median ratio over the A/A median ratio.
+   */
+  readonly interval: Interval | undefined;
+  /** The band of the A/A arm, or undefined when the A/A arm has too few cases. */
+  readonly band: Band | undefined;
+  /**
+   * The counts of the cases out of the band, or undefined without a band.
+   * The comparison uses the value `1 + log(ratio)` of each case and the noise of the band, so its two sides are the ends of the band.
+   */
+  readonly comparison: Comparison | undefined;
+}
+/**
+ * This function gives the time verdict of the candidate arm `b` against the baseline arm `a`.
+ * - The rule `difference` gives `faster` when the whole interval is below 1, and `slower` when it is above 1.
+ *   It gives `same` when the interval is in `[exp(-margin), exp(margin)]`.
+ * - The rule `band` gives `faster` below the band, `slower` above the band, and `same` in the band.
+ * - Each other case is `cannot decide`, and so is an A/A arm with too few cases.
+ */
+export declare function timeVerdict(candidate: readonly PairedTime[], aa: readonly PairedTime[], options?: VerdictOptions): TimeVerdict;
+/** The options of a power test. */
+interface PowerOptions {
+  /** The count of the cases of the candidate and of the A/A arm. */
+  readonly cases: number;
+  /** The count of the rounds of each case. The default is 5. */
+  readonly rounds?: number;
+  /** The true ratio of the candidate time to the baseline time, for example 0.93 for a speedup of 7%. */
+  readonly ratio: number;
+  /** The standard deviation of the log time of one arm in one round. The default is 0.1. */
+  readonly sigma?: number;
+  /** The count of the seeded benches. The default is 200. */
+  readonly trials?: number;
+  /** The seed of the draws. The default is 1. */
+  readonly seed?: number | string;
+  /** The options of the verdict of each bench. */
+  readonly verdict?: VerdictOptions;
+}
+/** The result of a power test: the share of each outcome over the trials. */
+interface Power {
+  readonly trials: number;
+  readonly outcomes: Readonly<Record<TimeOutcome, number>>;
+  /** The share of the trials whose outcome is the true one: `faster` for a ratio below 1, `slower` above 1, `same` at 1. */
+  readonly power: number;
+}
+/**
+ * This function draws `trials` benches and gives the share of each verdict. Each case has its own log scale.
+ * Each arm of each round adds a normal noise of `sigma` to the log time.
+ */
+export declare function verdictPower(options: PowerOptions): Power;
+export type { Arm, Band, BandKind, BandOptions, CaseRatio, PairedTime, Power, PowerOptions, TimeOutcome, TimeVerdict, VerdictOptions, VerdictRule };
+//# sourceMappingURL=index.d.ts.map

package/build/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","names":[],"sources":["../src/paired.ts","../src/verdict.ts","../src/power.ts"],"mappings":";;KAOY;;UAGK;;WAEN;;WAEA;WACA,OAAO;;WAEP;;WAEA;;;;;;wBAOK,SAAS,eAAe,kBAAa;;wBAOrC,gBAAgB,QAAQ,aAAa;;UAapC;WACN;;WAEA;WACA;;;;;;;wBAQK,WAAW,kBAAkB,eAAe;;;;;;KC7ChD;;UAGK;;WAEN,OAAO;;WAEP;;WAEA;;WAEA,YAAY;;;UAIN;;WAEN;WACA,MAAM;;;;;;WAMN;;WAEA;WACA;;WAEA;;;wBAIK,OAAO,aAAa,cAAc,UAAS,cAAmB;;KAiBlE;;;;;;;KAQA;;UAGK,uBAAuB;;WAE7B,OAAO;;WAEP;;;UAIM;WACN,SAAS;;WAET;WACA,gBAAgB;WAChB,MAAM;;;;;;;WAON,UAAU;;WAEV,MAAM;;;;;WAKN,YAAY;;;;;;;;;wBAUP,YAAY,oBAAoB,cAAc,aAAa,cAAc,UAAS,iBAAsB;;UCtGvG;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,UAAU;;;UAIJ;WACN;WACA,UAAU,SAAS,OAAO;;WAE1B;;;;;;wBAOK,aAAa,SAAS,eAAe"}

package/build/index.js

@@ -0,0 +1,251 @@
+import { bootstrapMedian, bootstrapTwo, compareRatios, median, quantile } from "@mark1russell7/rigor-statistics";
+import { random } from "@mark1russell7/rigor-generators";
+// src/paired.ts
+/**
+* The paired-time record of a bench. Each round runs the two arms of a pair on one case, one after the other, in a
+* fresh process. The order turns each round, so a warm cache or a slow start does not always help the same arm.
+*/
+/**
+* This function gives the arm that runs first in a round. The order turns each round. A case with an odd `offset`
+* starts with arm `b`, so half of the cases start with each arm.
+*/
+function firstArm(round, offset = 0) {
+	if (!Number.isSafeInteger(round) || round < 0) throw new RangeError(`firstArm: the round is a whole number of 0 or more, not ${round}.`);
+	if (!Number.isSafeInteger(offset)) throw new RangeError(`firstArm: the offset is a whole number, not ${offset}.`);
+	return ((round + offset) % 2 + 2) % 2 === 0 ? "a" : "b";
+}
+/** This function examines one record. A time that is not a finite number above 0 is an error. */
+function checkPairedTime(record) {
+	const where = `the record of the case ${JSON.stringify(record.case)}, round ${record.round}`;
+	if (typeof record.case !== "string" || record.case === "") throw new RangeError("checkPairedTime: the case is a string that is not empty.");
+	if (!Number.isSafeInteger(record.round) || record.round < 0) throw new RangeError(`checkPairedTime: in ${where}, the round is a whole number of 0 or more.`);
+	if (record.first !== "a" && record.first !== "b") throw new RangeError(`checkPairedTime: in ${where}, the first arm is "a" or "b", not ${String(record.first)}.`);
+	for (const arm of ["a", "b"]) {
+		const ms = record[arm];
+		if (!(typeof ms === "number" && Number.isFinite(ms) && ms > 0)) throw new RangeError(`checkPairedTime: in ${where}, the time of arm ${arm} is a finite number above 0, not ${String(ms)}.`);
+	}
+	return record;
+}
+/**
+* This function gives the ratio of each case, in the order of the first record of each case.
+* It examines each record. It refuses a case with 2 records of one round.
+* It also refuses a case whose two orders differ in count by more than 1, because that order did not turn each round.
+*/
+function caseRatios(records) {
+	const cases = /* @__PURE__ */ new Map();
+	for (const record of records) {
+		checkPairedTime(record);
+		const list = cases.get(record.case) ?? [];
+		list.push(record);
+		cases.set(record.case, list);
+	}
+	return [...cases].map(([name, list]) => {
+		if (new Set(list.map((r) => r.round)).size !== list.length) throw new RangeError(`caseRatios: the case ${JSON.stringify(name)} has 2 records of one round.`);
+		const aFirst = list.filter((r) => r.first === "a").length;
+		if (Math.abs(2 * aFirst - list.length) > 1) throw new RangeError(`caseRatios: the case ${JSON.stringify(name)} starts with arm a in ${aFirst} of ${list.length} rounds. The order turns each round.`);
+		return {
+			case: name,
+			ratio: Math.exp(median(list.map((r) => Math.log(r.b / r.a)))),
+			rounds: list.length
+		};
+	});
+}
+// src/verdict.ts
+/**
+* The time verdict of a candidate against a baseline, with the noise of an A/A arm. An A/A pair runs the same code in
+* its two arms. Its ratios show how far apart two equal arms come on this machine. A candidate is faster or slower only
+* when its interval is out of that band.
+*/
+/** This function gives the band of the A/A arm. The band is symmetric on a log scale. It refuses fewer cases than `minimum`. */
+function aaBand(aa, options = {}) {
+	const ratios = caseRatios(aa);
+	const minimum = options.minimum ?? 5;
+	if (ratios.length < minimum) throw new RangeError(`aaBand: the A/A arm has ${ratios.length} cases, and a band needs ${minimum} or more.`);
+	const logs = ratios.map((r) => Math.log(r.ratio));
+	const kind = options.kind ?? "median";
+	const level = options.level ?? .95;
+	let noise;
+	if (kind === "case") noise = quantile(logs.map(Math.abs), level);
+	else {
+		const interval = bootstrapMedian(logs, {
+			...options.bootstrap,
+			level
+		});
+		noise = Math.max(Math.abs(interval.low), Math.abs(interval.high));
+	}
+	return {
+		cases: ratios.length,
+		kind,
+		noise,
+		low: Math.exp(-noise),
+		high: Math.exp(noise),
+		median: Math.exp(quantile(logs, .5))
+	};
+}
+/**
+* This function gives the time verdict of the candidate arm `b` against the baseline arm `a`.
+* - The rule `difference` gives `faster` when the whole interval is below 1, and `slower` when it is above 1.
+*   It gives `same` when the interval is in `[exp(-margin), exp(margin)]`.
+* - The rule `band` gives `faster` below the band, `slower` above the band, and `same` in the band.
+* - Each other case is `cannot decide`, and so is an A/A arm with too few cases.
+*/
+function timeVerdict(candidate, aa, options = {}) {
+	const rule = options.rule ?? "difference";
+	const cases = caseRatios(candidate);
+	const aaRatios = caseRatios(aa);
+	const minimum = options.minimum ?? 5;
+	const none = {
+		cases,
+		rule,
+		interval: void 0,
+		band: void 0,
+		comparison: void 0
+	};
+	if (cases.length === 0) return {
+		...none,
+		outcome: "cannot decide",
+		reason: "The candidate has no cases."
+	};
+	if (aaRatios.length < minimum) {
+		const interval = bootstrapMedian(cases.map((c) => c.ratio), options.bootstrap);
+		return {
+			...none,
+			interval,
+			outcome: "cannot decide",
+			reason: `The A/A arm has ${aaRatios.length} cases, and a band needs ${minimum} or more.`
+		};
+	}
+	const band = aaBand(aa, options);
+	const facts = {
+		cases,
+		rule,
+		band,
+		comparison: compareRatios(cases.map((c) => 1 + Math.log(c.ratio)), {
+			better: "lower",
+			noise: band.noise
+		})
+	};
+	if (rule === "difference") {
+		const margin = options.margin ?? .05;
+		if (!(margin >= 0)) throw new RangeError(`timeVerdict: the margin is 0 or more, not ${margin}.`);
+		const logs = cases.map((c) => Math.log(c.ratio));
+		const aaLogs = aaRatios.map((c) => Math.log(c.ratio));
+		const d = bootstrapTwo(logs, aaLogs, (x, y) => median(x) - median(y), options.bootstrap);
+		const interval = {
+			...d,
+			estimate: Math.exp(d.estimate),
+			low: Math.exp(d.low),
+			high: Math.exp(d.high)
+		};
+		const span = `The interval of the candidate median over the A/A median is [${round(interval.low)}, ${round(interval.high)}].`;
+		if (interval.high < 1) return {
+			...facts,
+			interval,
+			outcome: "faster",
+			reason: `${span} The interval is below 1.`
+		};
+		if (interval.low > 1) return {
+			...facts,
+			interval,
+			outcome: "slower",
+			reason: `${span} The interval is above 1.`
+		};
+		if (d.low >= -margin && d.high <= margin) return {
+			...facts,
+			interval,
+			outcome: "same",
+			reason: `${span} The interval is in the margin of ${margin}.`
+		};
+		return {
+			...facts,
+			interval,
+			outcome: "cannot decide",
+			reason: `${span} The interval holds 1 and crosses the margin of ${margin}.`
+		};
+	}
+	const interval = bootstrapMedian(cases.map((c) => c.ratio), options.bootstrap);
+	const span = `The interval of the median ratio is [${round(interval.low)}, ${round(interval.high)}], and the A/A band is [${round(band.low)}, ${round(band.high)}].`;
+	if (interval.high < band.low) return {
+		...facts,
+		interval,
+		outcome: "faster",
+		reason: `${span} The interval is below the band.`
+	};
+	if (interval.low > band.high) return {
+		...facts,
+		interval,
+		outcome: "slower",
+		reason: `${span} The interval is above the band.`
+	};
+	if (interval.low >= band.low && interval.high <= band.high) return {
+		...facts,
+		interval,
+		outcome: "same",
+		reason: `${span} The interval is in the band.`
+	};
+	return {
+		...facts,
+		interval,
+		outcome: "cannot decide",
+		reason: `${span} The interval crosses an end of the band.`
+	};
+}
+function round(x) {
+	return Math.round(x * 1e4) / 1e4;
+}
+// src/power.ts
+/**
+* The power test of a time verdict. It draws seeded benches with a known speedup and counts the verdicts that find it.
+* It also draws benches with equal arms and counts the false verdicts.
+*/
+/**
+* This function draws `trials` benches and gives the share of each verdict. Each case has its own log scale.
+* Each arm of each round adds a normal noise of `sigma` to the log time.
+*/
+function verdictPower(options) {
+	const rounds = options.rounds ?? 5;
+	const sigma = options.sigma ?? .1;
+	const trials = options.trials ?? 200;
+	if (!Number.isSafeInteger(options.cases) || options.cases < 1) throw new RangeError(`verdictPower: cases is a whole number of 1 or more, not ${options.cases}.`);
+	if (!(options.ratio > 0 && Number.isFinite(options.ratio))) throw new RangeError(`verdictPower: the ratio is a finite number above 0, not ${options.ratio}.`);
+	const r = random(options.seed ?? 1);
+	const bench = (ratio, prefix) => {
+		const out = [];
+		for (let c = 0; c < options.cases; c++) {
+			const scale = 10 * Math.exp(r.normal());
+			for (let round = 0; round < rounds; round++) {
+				const a = scale * Math.exp(r.normal(0, sigma));
+				const b = scale * ratio * Math.exp(r.normal(0, sigma));
+				out.push({
+					case: `${prefix}${c}`,
+					round,
+					first: firstArm(round, c),
+					a,
+					b
+				});
+			}
+		}
+		return out;
+	};
+	const truth = options.ratio < 1 ? "faster" : options.ratio > 1 ? "slower" : "same";
+	const counts = {
+		faster: 0,
+		slower: 0,
+		same: 0,
+		"cannot decide": 0
+	};
+	for (let t = 0; t < trials; t++) {
+		const aa = bench(1, "aa");
+		const candidate = bench(options.ratio, "c");
+		counts[timeVerdict(candidate, aa, options.verdict).outcome]++;
+	}
+	const outcomes = Object.fromEntries(Object.entries(counts).map(([k, n]) => [k, n / trials]));
+	return {
+		trials,
+		outcomes,
+		power: outcomes[truth]
+	};
+}
+export { aaBand, caseRatios, checkPairedTime, firstArm, timeVerdict, verdictPower };
+
+//# sourceMappingURL=index.js.map

package/build/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","names":[],"sources":["../src/paired.ts","../src/verdict.ts","../src/power.ts"],"sourcesContent":["/**\n * The paired-time record of a bench. Each round runs the two arms of a pair on one case, one after the other, in a\n * fresh process. The order turns each round, so a warm cache or a slow start does not always help the same arm.\n */\nimport { median } from \"@mark1russell7/rigor-statistics\";\n\n/** The two arms of a pair. Arm `a` is the baseline, and arm `b` is the candidate. */\nexport type Arm = \"a\" | \"b\";\n\n/** One round of one pair on one case: the times of the two arms in milliseconds, and the arm that ran first. */\nexport interface PairedTime {\n  /** The name of the case: one input of the bench. */\n  readonly case: string;\n  /** The number of the round, a whole number of 0 or more. */\n  readonly round: number;\n  readonly first: Arm;\n  /** The time of the baseline arm, in milliseconds. */\n  readonly a: number;\n  /** The time of the candidate arm, in milliseconds. */\n  readonly b: number;\n}\n\n/**\n * This function gives the arm that runs first in a round. The order turns each round. A case with an odd `offset`\n * starts with arm `b`, so half of the cases start with each arm.\n */\nexport function firstArm(round: number, offset = 0): Arm {\n  if (!Number.isSafeInteger(round) || round < 0) throw new RangeError(`firstArm: the round is a whole number of 0 or more, not ${round}.`);\n  if (!Number.isSafeInteger(offset)) throw new RangeError(`firstArm: the offset is a whole number, not ${offset}.`);\n  return (((round + offset) % 2) + 2) % 2 === 0 ? \"a\" : \"b\";\n}\n\n/** This function examines one record. A time that is not a finite number above 0 is an error. */\nexport function checkPairedTime(record: PairedTime): PairedTime {\n  const where = `the record of the case ${JSON.stringify(record.case)}, round ${record.round}`;\n  if (typeof record.case !== \"string\" || record.case === \"\") throw new RangeError(\"checkPairedTime: the case is a string that is not empty.\");\n  if (!Number.isSafeInteger(record.round) || record.round < 0) throw new RangeError(`checkPairedTime: in ${where}, the round is a whole number of 0 or more.`);\n  if (record.first !== \"a\" && record.first !== \"b\") throw new RangeError(`checkPairedTime: in ${where}, the first arm is \"a\" or \"b\", not ${String(record.first)}.`);\n  for (const arm of [\"a\", \"b\"] as const) {\n    const ms = record[arm];\n    if (!(typeof ms === \"number\" && Number.isFinite(ms) && ms > 0)) throw new RangeError(`checkPairedTime: in ${where}, the time of arm ${arm} is a finite number above 0, not ${String(ms)}.`);\n  }\n  return record;\n}\n\n/** The ratio of one case: the candidate time over the baseline time, over the rounds of the case. */\nexport interface CaseRatio {\n  readonly case: string;\n  /** The value `exp(median(log(b / a)))` over the rounds. For an odd count of rounds, it is the median of the ratios. */\n  readonly ratio: number;\n  readonly rounds: number;\n}\n\n/**\n * This function gives the ratio of each case, in the order of the first record of each case.\n * It examines each record. It refuses a case with 2 records of one round.\n * It also refuses a case whose two orders differ in count by more than 1, because that order did not turn each round.\n */\nexport function caseRatios(records: readonly PairedTime[]): CaseRatio[] {\n  const cases = new Map<string, PairedTime[]>();\n  for (const record of records) {\n    checkPairedTime(record);\n    const list = cases.get(record.case) ?? [];\n    list.push(record);\n    cases.set(record.case, list);\n  }\n  return [...cases].map(([name, list]) => {\n    const rounds = new Set(list.map((r) => r.round));\n    if (rounds.size !== list.length) throw new RangeError(`caseRatios: the case ${JSON.stringify(name)} has 2 records of one round.`);\n    const aFirst = list.filter((r) => r.first === \"a\").length;\n    if (Math.abs(2 * aFirst - list.length) > 1) {\n      throw new RangeError(`caseRatios: the case ${JSON.stringify(name)} starts with arm a in ${aFirst} of ${list.length} rounds. The order turns each round.`);\n    }\n    return { case: name, ratio: Math.exp(median(list.map((r) => Math.log(r.b / r.a)))), rounds: list.length };\n  });\n}\n","/**\n * The time verdict of a candidate against a baseline, with the noise of an A/A arm. An A/A pair runs the same code in\n * its two arms. Its ratios show how far apart two equal arms come on this machine. A candidate is faster or slower only\n * when its interval is out of that band.\n */\nimport { type BootstrapOptions, type Comparison, type Interval, bootstrapMedian, bootstrapTwo, compareRatios, median, quantile } from \"@mark1russell7/rigor-statistics\";\nimport { type CaseRatio, type PairedTime, caseRatios } from \"./paired.ts\";\n\n/**\n * The two kinds of an A/A band.\n * - The band `case` holds the ratio of one A/A case. It is wide, so it finds only a large change.\n * - The band `median` holds the median ratio of the A/A cases. It has the scale of the median of the candidate.\n */\nexport type BandKind = \"case\" | \"median\";\n\n/** The options of the A/A band. */\nexport interface BandOptions {\n  /** The kind of the band. The default is `median`, from the power test of `verdictPower`. */\n  readonly kind?: BandKind;\n  /** The quantile of the band `case`, or the level of the bootstrap of the band `median`. The default is 0.95. */\n  readonly level?: number;\n  /** The fewest A/A cases that give a band. The default is 5, the 5-seed rule. */\n  readonly minimum?: number;\n  /** The bootstrap of the band `median`. The level of the band replaces its level. */\n  readonly bootstrap?: BootstrapOptions;\n}\n\n/** The band of the A/A arm: two equal arms come out in this band of ratios. */\nexport interface Band {\n  /** The count of the A/A cases. */\n  readonly cases: number;\n  readonly kind: BandKind;\n  /**\n   * The half width of the band on a log scale.\n   * For the band `case`, it is the `level` quantile of `abs(log(ratio))` over the A/A cases.\n   * For the band `median`, it is the larger distance of an end of the bootstrap interval of the A/A median from 0.\n   */\n  readonly noise: number;\n  /** The ends of the band: `exp(-noise)` and `exp(noise)`. */\n  readonly low: number;\n  readonly high: number;\n  /** The median ratio of the A/A cases. A value far from 1 shows a bias of the order or of the arm slots. */\n  readonly median: number;\n}\n\n/** This function gives the band of the A/A arm. The band is symmetric on a log scale. It refuses fewer cases than `minimum`. */\nexport function aaBand(aa: readonly PairedTime[], options: BandOptions = {}): Band {\n  const ratios = caseRatios(aa);\n  const minimum = options.minimum ?? 5;\n  if (ratios.length < minimum) throw new RangeError(`aaBand: the A/A arm has ${ratios.length} cases, and a band needs ${minimum} or more.`);\n  const logs = ratios.map((r) => Math.log(r.ratio));\n  const kind = options.kind ?? \"median\";\n  const level = options.level ?? 0.95;\n  let noise: number;\n  if (kind === \"case\") noise = quantile(logs.map(Math.abs), level);\n  else {\n    const interval = bootstrapMedian(logs, { ...options.bootstrap, level });\n    noise = Math.max(Math.abs(interval.low), Math.abs(interval.high));\n  }\n  return { cases: ratios.length, kind, noise, low: Math.exp(-noise), high: Math.exp(noise), median: Math.exp(quantile(logs, 0.5)) };\n}\n\n/** The four outcomes of a time verdict. The candidate is the arm `b`, so `faster` is a ratio below 1. */\nexport type TimeOutcome = \"faster\" | \"slower\" | \"same\" | \"cannot decide\";\n\n/** The options of a time verdict. The bootstrap gives the interval of the median ratio of the candidate, and the band `median`. */\n/**\n * The two rules of a time verdict.\n * - The rule `band` compares the interval of the median ratio of the candidate with the A/A band.\n * - The rule `difference` compares the interval of the candidate median over the A/A median with 1 and with `margin`.\n */\nexport type VerdictRule = \"band\" | \"difference\";\n\n/** The options of a time verdict. The bootstrap gives each interval of the verdict. */\nexport interface VerdictOptions extends BandOptions {\n  /** The rule of the verdict. The default is `difference`, from the power test of `verdictPower`. */\n  readonly rule?: VerdictRule;\n  /** The half width on a log scale of the ratios that the rule `difference` calls `same`. The default is 0.05. */\n  readonly margin?: number;\n}\n\n/** A time verdict, with the facts that it uses. */\nexport interface TimeVerdict {\n  readonly outcome: TimeOutcome;\n  /** The reason of the outcome, in one sentence. */\n  readonly reason: string;\n  readonly cases: readonly CaseRatio[];\n  readonly rule: VerdictRule;\n  /**\n   * The interval that the rule compares, or undefined when there are no candidate cases.\n   * With too few A/A cases, it is the interval of the median ratio of the candidate.\n   * For the rule `band`, it is the bootstrap interval of the median ratio of the candidate.\n   * For the rule `difference`, it is the interval of the candidate median ratio over the A/A median ratio.\n   */\n  readonly interval: Interval | undefined;\n  /** The band of the A/A arm, or undefined when the A/A arm has too few cases. */\n  readonly band: Band | undefined;\n  /**\n   * The counts of the cases out of the band, or undefined without a band.\n   * The comparison uses the value `1 + log(ratio)` of each case and the noise of the band, so its two sides are the ends of the band.\n   */\n  readonly comparison: Comparison | undefined;\n}\n\n/**\n * This function gives the time verdict of the candidate arm `b` against the baseline arm `a`.\n * - The rule `difference` gives `faster` when the whole interval is below 1, and `slower` when it is above 1.\n *   It gives `same` when the interval is in `[exp(-margin), exp(margin)]`.\n * - The rule `band` gives `faster` below the band, `slower` above the band, and `same` in the band.\n * - Each other case is `cannot decide`, and so is an A/A arm with too few cases.\n */\nexport function timeVerdict(candidate: readonly PairedTime[], aa: readonly PairedTime[], options: VerdictOptions = {}): TimeVerdict {\n  const rule = options.rule ?? \"difference\";\n  const cases = caseRatios(candidate);\n  const aaRatios = caseRatios(aa);\n  const minimum = options.minimum ?? 5;\n  const none = { cases, rule, interval: undefined, band: undefined, comparison: undefined };\n  if (cases.length === 0) return { ...none, outcome: \"cannot decide\", reason: \"The candidate has no cases.\" };\n  if (aaRatios.length < minimum) {\n    const interval = bootstrapMedian(cases.map((c) => c.ratio), options.bootstrap);\n    return { ...none, interval, outcome: \"cannot decide\", reason: `The A/A arm has ${aaRatios.length} cases, and a band needs ${minimum} or more.` };\n  }\n  const band = aaBand(aa, options);\n  const comparison = compareRatios(cases.map((c) => 1 + Math.log(c.ratio)), { better: \"lower\", noise: band.noise });\n  const facts = { cases, rule, band, comparison };\n  if (rule === \"difference\") {\n    const margin = options.margin ?? 0.05;\n    if (!(margin >= 0)) throw new RangeError(`timeVerdict: the margin is 0 or more, not ${margin}.`);\n    const logs = cases.map((c) => Math.log(c.ratio));\n    const aaLogs = aaRatios.map((c) => Math.log(c.ratio));\n    const d = bootstrapTwo(logs, aaLogs, (x, y) => median(x) - median(y), options.bootstrap);\n    const interval: Interval = { ...d, estimate: Math.exp(d.estimate), low: Math.exp(d.low), high: Math.exp(d.high) };\n    const span = `The interval of the candidate median over the A/A median is [${round(interval.low)}, ${round(interval.high)}].`;\n    if (interval.high < 1) return { ...facts, interval, outcome: \"faster\", reason: `${span} The interval is below 1.` };\n    if (interval.low > 1) return { ...facts, interval, outcome: \"slower\", reason: `${span} The interval is above 1.` };\n    if (d.low >= -margin && d.high <= margin) return { ...facts, interval, outcome: \"same\", reason: `${span} The interval is in the margin of ${margin}.` };\n    return { ...facts, interval, outcome: \"cannot decide\", reason: `${span} The interval holds 1 and crosses the margin of ${margin}.` };\n  }\n  const interval = bootstrapMedian(cases.map((c) => c.ratio), options.bootstrap);\n  const span = `The interval of the median ratio is [${round(interval.low)}, ${round(interval.high)}], and the A/A band is [${round(band.low)}, ${round(band.high)}].`;\n  if (interval.high < band.low) return { ...facts, interval, outcome: \"faster\", reason: `${span} The interval is below the band.` };\n  if (interval.low > band.high) return { ...facts, interval, outcome: \"slower\", reason: `${span} The interval is above the band.` };\n  if (interval.low >= band.low && interval.high <= band.high) return { ...facts, interval, outcome: \"same\", reason: `${span} The interval is in the band.` };\n  return { ...facts, interval, outcome: \"cannot decide\", reason: `${span} The interval crosses an end of the band.` };\n}\n\nfunction round(x: number): number {\n  return Math.round(x * 10_000) / 10_000;\n}\n","/**\n * The power test of a time verdict. It draws seeded benches with a known speedup and counts the verdicts that find it.\n * It also draws benches with equal arms and counts the false verdicts.\n */\nimport { random } from \"@mark1russell7/rigor-generators\";\nimport { firstArm, type PairedTime } from \"./paired.ts\";\nimport { type TimeOutcome, type VerdictOptions, timeVerdict } from \"./verdict.ts\";\n\n/** The options of a power test. */\nexport interface PowerOptions {\n  /** The count of the cases of the candidate and of the A/A arm. */\n  readonly cases: number;\n  /** The count of the rounds of each case. The default is 5. */\n  readonly rounds?: number;\n  /** The true ratio of the candidate time to the baseline time, for example 0.93 for a speedup of 7%. */\n  readonly ratio: number;\n  /** The standard deviation of the log time of one arm in one round. The default is 0.1. */\n  readonly sigma?: number;\n  /** The count of the seeded benches. The default is 200. */\n  readonly trials?: number;\n  /** The seed of the draws. The default is 1. */\n  readonly seed?: number | string;\n  /** The options of the verdict of each bench. */\n  readonly verdict?: VerdictOptions;\n}\n\n/** The result of a power test: the share of each outcome over the trials. */\nexport interface Power {\n  readonly trials: number;\n  readonly outcomes: Readonly<Record<TimeOutcome, number>>;\n  /** The share of the trials whose outcome is the true one: `faster` for a ratio below 1, `slower` above 1, `same` at 1. */\n  readonly power: number;\n}\n\n/**\n * This function draws `trials` benches and gives the share of each verdict. Each case has its own log scale.\n * Each arm of each round adds a normal noise of `sigma` to the log time.\n */\nexport function verdictPower(options: PowerOptions): Power {\n  const rounds = options.rounds ?? 5;\n  const sigma = options.sigma ?? 0.1;\n  const trials = options.trials ?? 200;\n  if (!Number.isSafeInteger(options.cases) || options.cases < 1) throw new RangeError(`verdictPower: cases is a whole number of 1 or more, not ${options.cases}.`);\n  if (!(options.ratio > 0 && Number.isFinite(options.ratio))) throw new RangeError(`verdictPower: the ratio is a finite number above 0, not ${options.ratio}.`);\n  const r = random(options.seed ?? 1);\n  const bench = (ratio: number, prefix: string): PairedTime[] => {\n    const out: PairedTime[] = [];\n    for (let c = 0; c < options.cases; c++) {\n      const scale = 10 * Math.exp(r.normal());\n      for (let round = 0; round < rounds; round++) {\n        const a = scale * Math.exp(r.normal(0, sigma));\n        const b = scale * ratio * Math.exp(r.normal(0, sigma));\n        out.push({ case: `${prefix}${c}`, round, first: firstArm(round, c), a, b });\n      }\n    }\n    return out;\n  };\n  const truth: TimeOutcome = options.ratio < 1 ? \"faster\" : options.ratio > 1 ? \"slower\" : \"same\";\n  const counts: Record<TimeOutcome, number> = { faster: 0, slower: 0, same: 0, \"cannot decide\": 0 };\n  for (let t = 0; t < trials; t++) {\n    const aa = bench(1, \"aa\");\n    const candidate = bench(options.ratio, \"c\");\n    counts[timeVerdict(candidate, aa, options.verdict).outcome]++;\n  }\n  const outcomes = Object.fromEntries(Object.entries(counts).map(([k, n]) => [k, n / trials])) as Record<TimeOutcome, number>;\n  return { trials, outcomes, power: outcomes[truth] };\n}\n"],"mappings":";;;;;;;;;;;AA0BA,SAAgB,SAAS,OAAe,SAAS,GAAQ;CACvD,IAAI,CAAC,OAAO,cAAc,KAAK,KAAK,QAAQ,GAAG,MAAM,IAAI,WAAW,2DAA2D,MAAM,EAAE;CACvI,IAAI,CAAC,OAAO,cAAc,MAAM,GAAG,MAAM,IAAI,WAAW,+CAA+C,OAAO,EAAE;CAChH,SAAU,QAAQ,UAAU,IAAK,KAAK,MAAM,IAAI,MAAM;AACxD;;AAGA,SAAgB,gBAAgB,QAAgC;CAC9D,MAAM,QAAQ,0BAA0B,KAAK,UAAU,OAAO,IAAI,EAAE,UAAU,OAAO;CACrF,IAAI,OAAO,OAAO,SAAS,YAAY,OAAO,SAAS,IAAI,MAAM,IAAI,WAAW,0DAA0D;CAC1I,IAAI,CAAC,OAAO,cAAc,OAAO,KAAK,KAAK,OAAO,QAAQ,GAAG,MAAM,IAAI,WAAW,uBAAuB,MAAM,4CAA4C;CAC3J,IAAI,OAAO,UAAU,OAAO,OAAO,UAAU,KAAK,MAAM,IAAI,WAAW,uBAAuB,MAAM,qCAAqC,OAAO,OAAO,KAAK,EAAE,EAAE;CAChK,KAAK,MAAM,OAAO,CAAC,KAAK,GAAG,GAAY;EACrC,MAAM,KAAK,OAAO;EAClB,IAAI,EAAE,OAAO,OAAO,YAAY,OAAO,SAAS,EAAE,KAAK,KAAK,IAAI,MAAM,IAAI,WAAW,uBAAuB,MAAM,oBAAoB,IAAI,mCAAmC,OAAO,EAAE,EAAE,EAAE;CAC5L;CACA,OAAO;AACT;;;;;;AAeA,SAAgB,WAAW,SAA6C;CACtE,MAAM,wBAAQ,IAAI,IAA0B;CAC5C,KAAK,MAAM,UAAU,SAAS;EAC5B,gBAAgB,MAAM;EACtB,MAAM,OAAO,MAAM,IAAI,OAAO,IAAI,KAAK,CAAC;EACxC,KAAK,KAAK,MAAM;EAChB,MAAM,IAAI,OAAO,MAAM,IAAI;CAC7B;CACA,OAAO,CAAC,GAAG,KAAK,CAAC,CAAC,KAAK,CAAC,MAAM,UAAU;EAEtC,IAAI,IADe,IAAI,KAAK,KAAK,MAAM,EAAE,KAAK,CACrC,CAAC,CAAC,SAAS,KAAK,QAAQ,MAAM,IAAI,WAAW,wBAAwB,KAAK,UAAU,IAAI,EAAE,6BAA6B;EAChI,MAAM,SAAS,KAAK,QAAQ,MAAM,EAAE,UAAU,GAAG,CAAC,CAAC;EACnD,IAAI,KAAK,IAAI,IAAI,SAAS,KAAK,MAAM,IAAI,GACvC,MAAM,IAAI,WAAW,wBAAwB,KAAK,UAAU,IAAI,EAAE,wBAAwB,OAAO,MAAM,KAAK,OAAO,qCAAqC;EAE1J,OAAO;GAAE,MAAM;GAAM,OAAO,KAAK,IAAI,OAAO,KAAK,KAAK,MAAM,KAAK,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;GAAG,QAAQ,KAAK;EAAO;CAC1G,CAAC;AACH;;;;;;;;AC7BA,SAAgB,OAAO,IAA2B,UAAuB,CAAC,GAAS;CACjF,MAAM,SAAS,WAAW,EAAE;CAC5B,MAAM,UAAU,QAAQ,WAAW;CACnC,IAAI,OAAO,SAAS,SAAS,MAAM,IAAI,WAAW,2BAA2B,OAAO,OAAO,2BAA2B,QAAQ,UAAU;CACxI,MAAM,OAAO,OAAO,KAAK,MAAM,KAAK,IAAI,EAAE,KAAK,CAAC;CAChD,MAAM,OAAO,QAAQ,QAAQ;CAC7B,MAAM,QAAQ,QAAQ,SAAS;CAC/B,IAAI;CACJ,IAAI,SAAS,QAAQ,QAAQ,SAAS,KAAK,IAAI,KAAK,GAAG,GAAG,KAAK;MAC1D;EACH,MAAM,WAAW,gBAAgB,MAAM;GAAE,GAAG,QAAQ;GAAW;EAAM,CAAC;EACtE,QAAQ,KAAK,IAAI,KAAK,IAAI,SAAS,GAAG,GAAG,KAAK,IAAI,SAAS,IAAI,CAAC;CAClE;CACA,OAAO;EAAE,OAAO,OAAO;EAAQ;EAAM;EAAO,KAAK,KAAK,IAAI,CAAC,KAAK;EAAG,MAAM,KAAK,IAAI,KAAK;EAAG,QAAQ,KAAK,IAAI,SAAS,MAAM,EAAG,CAAC;CAAE;AAClI;;;;;;;;AAmDA,SAAgB,YAAY,WAAkC,IAA2B,UAA0B,CAAC,GAAgB;CAClI,MAAM,OAAO,QAAQ,QAAQ;CAC7B,MAAM,QAAQ,WAAW,SAAS;CAClC,MAAM,WAAW,WAAW,EAAE;CAC9B,MAAM,UAAU,QAAQ,WAAW;CACnC,MAAM,OAAO;EAAE;EAAO;EAAM,UAAU,KAAA;EAAW,MAAM,KAAA;EAAW,YAAY,KAAA;CAAU;CACxF,IAAI,MAAM,WAAW,GAAG,OAAO;EAAE,GAAG;EAAM,SAAS;EAAiB,QAAQ;CAA8B;CAC1G,IAAI,SAAS,SAAS,SAAS;EAC7B,MAAM,WAAW,gBAAgB,MAAM,KAAK,MAAM,EAAE,KAAK,GAAG,QAAQ,SAAS;EAC7E,OAAO;GAAE,GAAG;GAAM;GAAU,SAAS;GAAiB,QAAQ,mBAAmB,SAAS,OAAO,2BAA2B,QAAQ;EAAW;CACjJ;CACA,MAAM,OAAO,OAAO,IAAI,OAAO;CAE/B,MAAM,QAAQ;EAAE;EAAO;EAAM;EAAM,YADhB,cAAc,MAAM,KAAK,MAAM,IAAI,KAAK,IAAI,EAAE,KAAK,CAAC,GAAG;GAAE,QAAQ;GAAS,OAAO,KAAK;EAAM,CACnE;CAAE;CAC9C,IAAI,SAAS,cAAc;EACzB,MAAM,SAAS,QAAQ,UAAU;EACjC,IAAI,EAAE,UAAU,IAAI,MAAM,IAAI,WAAW,6CAA6C,OAAO,EAAE;EAC/F,MAAM,OAAO,MAAM,KAAK,MAAM,KAAK,IAAI,EAAE,KAAK,CAAC;EAC/C,MAAM,SAAS,SAAS,KAAK,MAAM,KAAK,IAAI,EAAE,KAAK,CAAC;EACpD,MAAM,IAAI,aAAa,MAAM,SAAS,GAAG,MAAM,OAAO,CAAC,IAAI,OAAO,CAAC,GAAG,QAAQ,SAAS;EACvF,MAAM,WAAqB;GAAE,GAAG;GAAG,UAAU,KAAK,IAAI,EAAE,QAAQ;GAAG,KAAK,KAAK,IAAI,EAAE,GAAG;GAAG,MAAM,KAAK,IAAI,EAAE,IAAI;EAAE;EAChH,MAAM,OAAO,gEAAgE,MAAM,SAAS,GAAG,EAAE,IAAI,MAAM,SAAS,IAAI,EAAE;EAC1H,IAAI,SAAS,OAAO,GAAG,OAAO;GAAE,GAAG;GAAO;GAAU,SAAS;GAAU,QAAQ,GAAG,KAAK;EAA2B;EAClH,IAAI,SAAS,MAAM,GAAG,OAAO;GAAE,GAAG;GAAO;GAAU,SAAS;GAAU,QAAQ,GAAG,KAAK;EAA2B;EACjH,IAAI,EAAE,OAAO,CAAC,UAAU,EAAE,QAAQ,QAAQ,OAAO;GAAE,GAAG;GAAO;GAAU,SAAS;GAAQ,QAAQ,GAAG,KAAK,oCAAoC,OAAO;EAAG;EACtJ,OAAO;GAAE,GAAG;GAAO;GAAU,SAAS;GAAiB,QAAQ,GAAG,KAAK,kDAAkD,OAAO;EAAG;CACrI;CACA,MAAM,WAAW,gBAAgB,MAAM,KAAK,MAAM,EAAE,KAAK,GAAG,QAAQ,SAAS;CAC7E,MAAM,OAAO,wCAAwC,MAAM,SAAS,GAAG,EAAE,IAAI,MAAM,SAAS,IAAI,EAAE,0BAA0B,MAAM,KAAK,GAAG,EAAE,IAAI,MAAM,KAAK,IAAI,EAAE;CACjK,IAAI,SAAS,OAAO,KAAK,KAAK,OAAO;EAAE,GAAG;EAAO;EAAU,SAAS;EAAU,QAAQ,GAAG,KAAK;CAAkC;CAChI,IAAI,SAAS,MAAM,KAAK,MAAM,OAAO;EAAE,GAAG;EAAO;EAAU,SAAS;EAAU,QAAQ,GAAG,KAAK;CAAkC;CAChI,IAAI,SAAS,OAAO,KAAK,OAAO,SAAS,QAAQ,KAAK,MAAM,OAAO;EAAE,GAAG;EAAO;EAAU,SAAS;EAAQ,QAAQ,GAAG,KAAK;CAA+B;CACzJ,OAAO;EAAE,GAAG;EAAO;EAAU,SAAS;EAAiB,QAAQ,GAAG,KAAK;CAA2C;AACpH;AAEA,SAAS,MAAM,GAAmB;CAChC,OAAO,KAAK,MAAM,IAAI,GAAM,IAAI;AAClC;;;;;;;;;;AC9GA,SAAgB,aAAa,SAA8B;CACzD,MAAM,SAAS,QAAQ,UAAU;CACjC,MAAM,QAAQ,QAAQ,SAAS;CAC/B,MAAM,SAAS,QAAQ,UAAU;CACjC,IAAI,CAAC,OAAO,cAAc,QAAQ,KAAK,KAAK,QAAQ,QAAQ,GAAG,MAAM,IAAI,WAAW,2DAA2D,QAAQ,MAAM,EAAE;CAC/J,IAAI,EAAE,QAAQ,QAAQ,KAAK,OAAO,SAAS,QAAQ,KAAK,IAAI,MAAM,IAAI,WAAW,2DAA2D,QAAQ,MAAM,EAAE;CAC5J,MAAM,IAAI,OAAO,QAAQ,QAAQ,CAAC;CAClC,MAAM,SAAS,OAAe,WAAiC;EAC7D,MAAM,MAAoB,CAAC;EAC3B,KAAK,IAAI,IAAI,GAAG,IAAI,QAAQ,OAAO,KAAK;GACtC,MAAM,QAAQ,KAAK,KAAK,IAAI,EAAE,OAAO,CAAC;GACtC,KAAK,IAAI,QAAQ,GAAG,QAAQ,QAAQ,SAAS;IAC3C,MAAM,IAAI,QAAQ,KAAK,IAAI,EAAE,OAAO,GAAG,KAAK,CAAC;IAC7C,MAAM,IAAI,QAAQ,QAAQ,KAAK,IAAI,EAAE,OAAO,GAAG,KAAK,CAAC;IACrD,IAAI,KAAK;KAAE,MAAM,GAAG,SAAS;KAAK;KAAO,OAAO,SAAS,OAAO,CAAC;KAAG;KAAG;IAAE,CAAC;GAC5E;EACF;EACA,OAAO;CACT;CACA,MAAM,QAAqB,QAAQ,QAAQ,IAAI,WAAW,QAAQ,QAAQ,IAAI,WAAW;CACzF,MAAM,SAAsC;EAAE,QAAQ;EAAG,QAAQ;EAAG,MAAM;EAAG,iBAAiB;CAAE;CAChG,KAAK,IAAI,IAAI,GAAG,IAAI,QAAQ,KAAK;EAC/B,MAAM,KAAK,MAAM,GAAG,IAAI;EACxB,MAAM,YAAY,MAAM,QAAQ,OAAO,GAAG;EAC1C,OAAO,YAAY,WAAW,IAAI,QAAQ,OAAO,CAAC,CAAC,QAAQ;CAC7D;CACA,MAAM,WAAW,OAAO,YAAY,OAAO,QAAQ,MAAM,CAAC,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC,GAAG,IAAI,MAAM,CAAC,CAAC;CAC3F,OAAO;EAAE;EAAQ;EAAU,OAAO,SAAS;CAAO;AACpD"}

package/package.json

@@ -1,6 +1,62 @@
 {
+  "$schema": "https://json.schemastore.org/package",
   "name": "@mark1russell7/rigor-bench",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
+  "version": "0.1.0",
+  "description": "The paired-time records of a bench: the order that turns each round, the ratio of each case, and the time verdict against the band of an A/A arm.",
+  "keywords": [
+    "benchmark",
+    "paired",
+    "a/a",
+    "noise",
+    "bootstrap",
+    "performance",
+    "timing"
+  ],
+  "homepage": "https://github.com/mark1russell7/rigor/tree/main/packages/bench#readme",
+  "bugs": {
+    "url": "https://github.com/mark1russell7/rigor/issues"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/mark1russell7/rigor.git",
+    "directory": "packages/bench"
+  },
+  "license": "MIT",
+  "author": "Mark Russell (https://github.com/mark1russell7)",
+  "type": "module",
+  "main": "./build/index.js",
+  "files": [
+    "build"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "engines": {
+    "node": ">=22"
+  },
+  "dependencies": {
+    "@mark1russell7/rigor-statistics": "^0.1.3"
+  },
+  "devDependencies": {
+    "@mark1russell7/rigor-generators": "^0.1.0",
+    "@mark1russell7/cue": "^1.0.2",
+    "@types/node": "^26.6.5",
+    "tsdown": "^0.23.1",
+    "typescript": "^7.0.2",
+    "vitest": "^5.0.3"
+  },
+  "scripts": {
+    "build": "tsdown",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc -p tsconfig.json && tsc -p tsconfig.test.json"
+  },
+  "types": "./build/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./build/index.d.ts",
+      "import": "./build/index.js"
+    },
+    "./package.json": "./package.json"
+  }
 }