@mark1russell7/rigor-bench 0.2.17 → 0.2.19

package/README.md

@@ -46,6 +46,11 @@ v.comparison;                                     // the cases that gain and dro
 | `limitVerdict(candidate, aa, options)` | The verdict of the claim "the ratio of each case is at most `limit`": `within`, `beyond` or `cannot decide`. |
 | `tailLimitVerdict(candidate, aa, options)` | The verdict of the claim "the median and a high value of each case are at most their limits", for a bench with few rounds. |
 | `caseTails(records, rank)` | The value at one rank of the ratios of each case, from the largest. |
+| `modelTest(reads, options)` | The read of each CPU model against the rest, for each case: `differs`, `no read`, `too-few` or `not asked`. |
+| `visibleShift(values, rest, level, direction, options)` | The smallest shift in one direction that a test can see at this count and level, and the smallest p of this count. |
+| `modelRowText(row, family)` | The words of one row. No word says that a model is the same as the rest. |
+| `limitOnModel(limits, model)` | The limit of a gain on one CPU model: the gain limit, or the ceiling with its words. |
+| `armModels(pairs)` | The CPU models of each arm of a set of pairs, and if all the reads come from one known model. |
 | `tailFalseFails(aa, options)` | The cases of an A/A arm that the rule of `tailLimitVerdict` fails, and apart from these the cases with a limit under 1. |
 | `gateOf(cases)` | The gate of a list of case outcomes: `pass`, `block` or `no read`. |
 | `notSlowerVerdict(jobs, options)` | The gate of the claim "not slower by more than the margin", with looks and a family level. |
@@ -114,6 +119,21 @@ The verdict has no interval, so a limit needs a margin from the spread of earlie
 
 A limit under 1 states a gain, and an A/A pair shows no gain. So the A/A arm has no read of that case, and the case goes in `gains` with that reason. The false fails of a gain limit come from repeat reads of an unchanged tree against its fixed base.
 
+## A read of each CPU model
+
+Runners of one label have different CPU models. A check of the pool passes when most reads come from the other models, so it can hide a model that reads apart. `modelTest` reads each model against the rest of its case. It uses a two-sample permutation test of the median, with one Holm family over all the rows.
+
+- `differs`: the adjusted p is under `alpha`, and the shift is at `resolution` or more, when the caller gives one.
+- `no read`: the test finds no difference. Few reads have little power, so this does not say that the model is the same. The row gives `floor`, the smallest p of that count, and `level`, the first level of Holm in the family. When `floor` is over `level`, the words of `modelRowText` say that this size cannot reach the level, with no flip. A row of that size reads `differs` only if other rows read `differs` before it, because each of those rows raises the level. When the level is in reach, `visible` gives the smallest shift that this test can see at that count and family. The option `searchVisible` set to false stops only that search.
+- `too-few`: the model has under `minReads` reads, or the case has no other model. The row keeps its facts, and it stays out of the family.
+- `not asked`: the option `asked` leaves the row out of the question. State the question before the reads, because a family that is chosen after a read has no valid p.
+
+A read of `differs` tells where to look. It does not name a cause, because the reads of one model can share a runner image and a date.
+
+`limitOnModel` gives the limit of a gain that reads on some models only. A model with no earlier reads gets the ceiling, and its read carries the words of `CEILING_ONLY`. A pass at the ceiling stops a loss, and it does not hold the gain.
+
+`armModels` reads the models of the two arms of a set of pairs. When the arms come from different computers, a ratio compares the computers. A record with no model reads as `unknown`, and `unknown` is not one known model.
+
 ## A gate that can fail
 
 A check that blocks a merge needs a result that can fail. `gateOf` reads the case outcomes of a verdict, and `notSlowerVerdict` gives the claim of a speed gate.
@@ -323,6 +343,22 @@ Each finding fails a gate, and each one carries its tests with `n`, `ties`, `p`
 
 `winClaim(check, measure, entry, stratum)` gives `alone`, `show`, `likeControl`, `censored` and `open`. The fields are about this entry. The field `likeControl` is true when this entry does not beat a control on the measure and beats it on no partner. A win by another entry clears nothing. The field `alone` is true only when this entry itself beats each control on the measure, so a measure with no control is not `alone`.
 
+`jointClaim(check, measures, entry, stratum)` reads a set of measures together, because one measure cannot call a win when each control wins a measure and loses a partner:
+
+```ts
+const joint = jointClaim(check, ["fill", "order"], "my-entry");
+joint.verdict; // "survives", "dominated", "not-ahead" or "no-read"
+```
+
+| Verdict | Meaning |
+| --- | --- |
+| `survives` | No control dominates the entry, and the entry beats at least one control on each measure. |
+| `dominated` | A control beats the entry on at least one measure, and the entry beats that control on none. The field `dominatedBy` names it. |
+| `not-ahead` | The entry beats no control on a measure with each read made. The field `notAhead` names it. |
+| `no-read` | A test with too few keys, or a measure with no control, decides the claim. This verdict is not a pass. |
+
+A tie does not clear a control, and the win of another entry gives this entry no credit. The function adds no test: it reads the tests of `checkMeasures`, with their Holm families.
+
 ## The fresh runner
 
 The entry `@mark1russell7/rigor-bench/fresh` starts child processes, so it is for Node only. The root entry stays free of `node:child_process`.

package/build/index.d.ts

@@ -1,5 +1,5 @@
 import { a as caseRatios, i as armOrder, n as CaseRatio, o as checkPairedTime, r as PairedTime, s as firstArm, t as Arm } from "./paired.js";
-import { BootstrapOptions, Comparison as Comparison$1, Interval } from "@mark1russell7/rigor-statistics";
+import { BootstrapOptions, Comparison as Comparison$1, Interval, TwoSamplePermutationOptions } from "@mark1russell7/rigor-statistics";
 /** One value of one build on one case. The value is finite and 0 or more. */
 interface BudgetRound {
   /** The name of the case: one input of the bench. */
@@ -790,6 +790,137 @@ export declare function rungVerdict(rung: Rung, record: RungRecord | undefined,
  * a `RangeError` for a ladder with problems, because a verdict on a bad ladder has no meaning. See `checkLadder`.
  */
 export declare function ladderVerdict(ladder: Ladder, records: readonly RungRecord[], options: LadderOptions): LadderVerdict;
+/** The name of a record with no CPU model. */
+export declare const UNKNOWN_MODEL = "unknown";
+/** One value of one case, from one read on one CPU model. */
+interface ModelRead {
+  readonly case: string;
+  readonly model: string;
+  /** The value of the read: a ratio or a time. All the reads of a case have one unit. */
+  readonly value: number;
+}
+/** The options of `modelTest`. */
+interface ModelTestOptions {
+  /** The smallest count of reads of a model that the test reads. The default is 5. */
+  readonly minReads?: number;
+  /** The level of the Holm family over all the rows that the test reads. The default is 0.05. */
+  readonly alpha?: number;
+  /** The smallest shift of a median that counts. Without it, only the test decides. It has no default. */
+  readonly resolution?: number;
+  /**
+   * The largest number of splits of each test. The default gives 100 counts at the level of the strictest step of Holm, and
+   * 10 000 or more. A test reads each split when their count is this number or less, so its p is exact.
+   */
+  readonly runs?: number;
+  /** The seed of the splits. The default is 1. */
+  readonly seed?: number | string;
+  /**
+   * The rows of the question, when a question has fewer rows than the data. Each other row reads `not asked`, and it stays
+   * out of the family. State the question before the reads: a family that is chosen after a read has no valid p.
+   */
+  readonly asked?: (row: {
+    readonly case: string;
+    readonly model: string;
+  }) => boolean;
+  /**
+   * With false, the test does not search for the smallest shift of a row of `no read`, and the words of the row say so.
+   * This search is most of the cost of the test. The default is true.
+   */
+  readonly searchVisible?: boolean;
+}
+/** The result of one model on one case. */
+type ModelStatus = "differs" | "no read" | "too-few" | "not asked";
+/** One model on one case, against all the other reads of that case. */
+interface ModelRow {
+  readonly case: string;
+  readonly model: string;
+  readonly reads: number;
+  readonly median: number;
+  readonly low: number;
+  readonly high: number;
+  /** The count of the reads of the other models. */
+  readonly restReads: number;
+  /** The median of the reads of the other models, or undefined when no other model has a read. */
+  readonly restMedian: number | undefined;
+  /** The median of this model minus the median of the rest, or undefined when no other model has a read. */
+  readonly shift: number | undefined;
+  /** The p after the Holm correction, or undefined for a row outside the family. */
+  readonly p: number | undefined;
+  readonly status: ModelStatus;
+  /** The size of the smallest shift that this test can see at this count and family, in the direction of the shift of this row. It applies to a row of `no read`. It is undefined on each other row, and also when no shift can reach the family. */
+  readonly visible: number | undefined;
+  /** The p of this row with its reads far from the rest, in the direction of its shift, for a row of `no read`. No shift gives a smaller p. */
+  readonly floor: number | undefined;
+  /**
+   * The first level of Holm in this family, alpha over the size of the family, for a row of `no read`. A row with a floor over this level reads `differs` only if other rows read `differs` before it.
+   * Each of those rows raises the level.
+   */
+  readonly level: number | undefined;
+}
+/** The result of `modelTest`. */
+interface ModelTest {
+  /** Each model on each case, sorted by case and by model. */
+  readonly rows: readonly ModelRow[];
+  /** The count of the rows in the Holm family. */
+  readonly family: number;
+  /** Each row that reads `differs`. */
+  readonly differs: readonly ModelRow[];
+}
+/** The result of `visibleShift`. */
+interface VisibleShift {
+  /** The size of the smallest shift that reaches the level, or undefined when no shift reaches it. */
+  readonly shift: number | undefined;
+  /** The p with the values far from the rest, in this direction. No shift in this direction gives a smaller p. */
+  readonly floor: number;
+}
+/**
+ * This function finds the smallest shift of the median of `values` from the median of `rest` that gives a p of `level` or less.
+ * It keeps the spread of the values, and it reads one direction: 1 for a shift up, and -1 for a shift down. The two
+ * directions can differ, because the spread of the values and of the rest need not be symmetric.
+ *
+ * The level of a family is its alpha over its size, the strictest step of Holm. So the result errs on the side of a shift
+ * that is too large. The cost is 9 permutation tests when the level is in reach, and 1 when it is not.
+ */
+export declare function visibleShift(values: readonly number[], rest: readonly number[], level: number, direction: 1 | -1, options?: TwoSamplePermutationOptions): VisibleShift;
+/** This function reads each CPU model against the rest, for each case. */
+export declare function modelTest(reads: readonly ModelRead[], options?: ModelTestOptions): ModelTest;
+/** This function gives the words of one row. No word says that a model is the same as the rest. */
+export declare function modelRowText(row: ModelRow, family: number): string;
+/** The limits of a gain that reads on some CPU models only. */
+interface ModelLimits {
+  /** The limit that holds the gain, on a model of `models`. */
+  readonly gain: number;
+  /** The limit on each other model. It stops a loss, and it does not hold the gain. */
+  readonly ceiling: number;
+  /** The models with a read of the gain. */
+  readonly models: readonly string[];
+}
+/** The limit of a gain on one CPU model. */
+interface LimitOnModel {
+  readonly limit: number;
+  /** This field is true when a pass at this limit holds the gain. */
+  readonly holdsGain: boolean;
+  /** The words for a read that does not hold the gain, or undefined when it holds the gain. */
+  readonly note: string | undefined;
+}
+/** The words of a read at the ceiling. A pass at the ceiling is not a gain held. */
+export declare const CEILING_ONLY = "gain not read on this model, ceiling only";
+/** This function gives the limit of a gain on one CPU model. A model with no earlier reads gets the ceiling. */
+export declare function limitOnModel(limits: ModelLimits, model: string): LimitOnModel;
+/** One pair of reads, with the CPU model of each arm. A record with no model has `undefined`. */
+interface ArmPair {
+  readonly a: string | undefined;
+  readonly b: string | undefined;
+}
+/** The CPU models of the two arms of a set of pairs. */
+interface ArmModels {
+  readonly a: readonly string[];
+  readonly b: readonly string[];
+  /** This field is true when all the reads of the two arms have one known model. Otherwise, a ratio can compare two computers. */
+  readonly one: boolean;
+}
+/** This function gives the sorted CPU models of each arm, and tells if all the reads come from one known model. */
+export declare function armModels(pairs: readonly ArmPair[]): ArmModels;
 /** The value at one rank of the ratios of one case, from the largest. */
 interface CaseTail {
   readonly case: string;
@@ -1087,6 +1218,49 @@ interface WinClaim {
   /** The reason and the date, for a measure with no control. */
   readonly open: NoControl | undefined;
 }
+/** The read of one control over the set of measures of a joint claim. */
+interface JointControl {
+  readonly control: string;
+  /** The measures where the entry beats this control. */
+  readonly beaten: readonly string[];
+  /** The measures where this control beats the entry. */
+  readonly lostTo: readonly string[];
+  /** The measures with no read of the entry and this control: no test, or too few keys. A tie on enough keys is a read and does not count here. */
+  readonly unread: readonly string[];
+  /** This field is true when no measure is unread and the entry beats this control on none. This control also beats the entry on at least one. */
+  readonly dominates: boolean;
+}
+/** The read of one measure of a joint claim. */
+interface JointMeasure {
+  readonly measure: string;
+  /** The controls that the entry beats on this measure, each by its own test. */
+  readonly beats: readonly string[];
+  /** The controls in `beats` that also pass the level divided by the count of controls on this measure. Only these count for `survives`. */
+  readonly clears: readonly string[];
+  /** The controls that tie the entry on this measure: enough shared keys, and too few keys that differ. */
+  readonly tied: readonly string[];
+  /** The controls of the set with no read on this measure: no test, or too few keys. A measure with no test lists each control of the set. */
+  readonly unread: readonly string[];
+}
+/**
+ * The verdict of a joint claim. `dominated`: a control dominates the entry. `not-ahead`: on a measure with each read made, the entry beats no control.
+ * `no-read`: a read that is not there decides the claim, and that is not a pass. `survives`: no control dominates the entry, and the entry clears a control on each measure.
+ */
+type JointVerdict = "survives" | "dominated" | "not-ahead" | "no-read";
+/** What one real entry earned on a set of measures taken together. */
+interface JointClaim {
+  readonly entry: string;
+  readonly stratum: string | undefined;
+  readonly verdict: JointVerdict;
+  /** The controls that dominate the entry. */
+  readonly dominatedBy: readonly string[];
+  /** The controls with each test read, where neither side beats the other on any measure. The entry earned nothing over them, so a page shows this list. */
+  readonly equalledBy: readonly string[];
+  /** The measures where the entry clears no control. */
+  readonly notAhead: readonly string[];
+  readonly controls: readonly JointControl[];
+  readonly measures: readonly JointMeasure[];
+}
 /**
  * This function reads the controls and the spread of each measure of a bench.
  *
@@ -1111,6 +1285,16 @@ export declare function checkMeasures(rows: readonly MeasureRow[], options: Meas
  * The function throws a `RangeError` for a measure with no read and for an entry that the rows do not hold.
  */
 export declare function winClaim(check: MeasureChecks, measure: string, entry: string, stratum?: string): WinClaim;
+/**
+ * This function tells what one real entry earned on a set of measures taken together. Each win of a control loses a partner, so one measure cannot call a win.
+ * A control dominates the entry when the entry beats it on no measure of the set, and it beats the entry on at least one. A tie does not clear a control.
+ * The entry survives when no control dominates it and it clears at least one control on each measure. The function adds no test: it reads the tests of `check`.
+ * One win over any control is enough on a measure, so a beat clears only at the level divided by the count of controls there.
+ * All measures must hold, so the measures add no division. Dominance reads the plain tests, because that is the cautious direction.
+ * With a stratum, the function reads that stratum. Without one, it reads the rows with no stratum.
+ * The function throws a `RangeError` for an empty set, a repeat, a measure with no rule and an entry that the rows do not hold.
+ */
+export declare function jointClaim(check: MeasureChecks, measures: readonly string[], entry: string, stratum?: string): JointClaim;
 /** The options of a power test. */
 interface PowerOptions {
   /** The count of the cases of the candidate and of the A/A arm. */
@@ -1173,5 +1357,5 @@ interface Rounds {
  * keeps the work.
  */
 export declare function timeRounds(arms: Readonly<Record<Arm, () => number>>, options: RoundsOptions): Rounds;
-export { type Arm, type Balanced, type Band, type BandKind, type BandOptions, type BudgetLook, type BudgetLooksOptions, type BudgetLooksVerdict, type BudgetMore, type BudgetOptions, type BudgetOutcome, type BudgetResult, type BudgetRound, type BudgetVerdict, type CaseBudget, type CaseGate, type CaseInterval, type CaseLimit, type CaseRatio, type CaseTail, type CaseTailLimit, type Comparison, type ControlFactors, type ControlOptions, type ControlRead, type ControlRow, type EntryCount, type Gate, type GateCaseOutcome, type GateJobRead, type GateOutcome, type GroupCounts, type GroupFactor, type GuardRead, type JobLimitVerdict, type JobRecords, type JobVerdict, type Ladder, type LadderDimension, type LadderOptions, type LadderVerdict, type LimitOptions, type LimitOutcome, type LimitVerdict, type MeasureCheck, type MeasureCheckOptions, type MeasureChecks, type MeasureFinding, type MeasureFindingKind, type MeasureRow, type MeasureRule, type MeasuredFactor, type MedianBudget, type NoControl, type NoFactor, type NoiseCheck, type NotSlowerOptions, type NotSlowerRead, type NotSlowerVerdict, type PairedTime, type PartnerRead, type PoolFacts, type PoolOptions, type PooledJob, type PooledLimitVerdict, type PooledRecords, type PooledVerdict, type Power, type PowerOptions, type QuantileBudget, type RateBudget, type Rounds, type RoundsOptions, type RunPairs, type Rung, type RungBudget, type RungEvidence, type RungRecord, type RungVerdict, type SpreadRead, type TailFalseFails, type TailGainCase, type TailLimit, type TailLimitOptions, type TailLimitVerdict, type TailRuleOptions, type TimeOutcome, type TimeVerdict, type VerdictOptions, type VerdictRule, type WinClaim, type WorstBudget, armOrder, caseRatios, checkPairedTime, firstArm };
+export { type Arm, type ArmModels, type ArmPair, type Balanced, type Band, type BandKind, type BandOptions, type BudgetLook, type BudgetLooksOptions, type BudgetLooksVerdict, type BudgetMore, type BudgetOptions, type BudgetOutcome, type BudgetResult, type BudgetRound, type BudgetVerdict, type CaseBudget, type CaseGate, type CaseInterval, type CaseLimit, type CaseRatio, type CaseTail, type CaseTailLimit, type Comparison, type ControlFactors, type ControlOptions, type ControlRead, type ControlRow, type EntryCount, type Gate, type GateCaseOutcome, type GateJobRead, type GateOutcome, type GroupCounts, type GroupFactor, type GuardRead, type JobLimitVerdict, type JobRecords, type JobVerdict, type JointClaim, type JointControl, type JointMeasure, type JointVerdict, type Ladder, type LadderDimension, type LadderOptions, type LadderVerdict, type LimitOnModel, type LimitOptions, type LimitOutcome, type LimitVerdict, type MeasureCheck, type MeasureCheckOptions, type MeasureChecks, type MeasureFinding, type MeasureFindingKind, type MeasureRow, type MeasureRule, type MeasuredFactor, type MedianBudget, type ModelLimits, type ModelRead, type ModelRow, type ModelStatus, type ModelTest, type ModelTestOptions, type NoControl, type NoFactor, type NoiseCheck, type NotSlowerOptions, type NotSlowerRead, type NotSlowerVerdict, type PairedTime, type PartnerRead, type PoolFacts, type PoolOptions, type PooledJob, type PooledLimitVerdict, type PooledRecords, type PooledVerdict, type Power, type PowerOptions, type QuantileBudget, type RateBudget, type Rounds, type RoundsOptions, type RunPairs, type Rung, type RungBudget, type RungEvidence, type RungRecord, type RungVerdict, type SpreadRead, type TailFalseFails, type TailGainCase, type TailLimit, type TailLimitOptions, type TailLimitVerdict, type TailRuleOptions, type TimeOutcome, type TimeVerdict, type VerdictOptions, type VerdictRule, type VisibleShift, type WinClaim, type WorstBudget, armOrder, caseRatios, checkPairedTime, firstArm };
 //# sourceMappingURL=index.d.ts.map

package/build/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","names":["Comparison"],"sources":["../src/budget.ts","../src/verdict.ts","../src/limit.ts","../src/merge.ts","../src/gate.ts","../src/control.ts","../src/ladder.ts","../src/tail.ts","../src/measures.ts","../src/power.ts","../src/rounds.ts"],"mappings":";;;UAQiB;;WAEN;;WAEA;WACA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;WACA;;WAEA;WACA,SAAS;;WAET;;;UAIM;WACN,SAAS;;WAET;WACA;WACA;WACA;WACA,gBAAgB;;;;;;;;;;;;wBAyDX,cAAc,iBAAiB,eAAe,aAAa,eAAe,SAAS,gBAAgB;;UAyBlG;WACN;WACA;;;;;;qBAOE,uBAAuB;;UAOnB;;WAEN;;WAEA;;WAEA,iBAAiB;;WAEjB;;;UAIM;WACN;WACA;;;UAIM,2BAA2B;;WAEjC;;WAEA,eAAe;;WAEf;;;;;;wBAOK,gBAAgB,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;wBAoDhC,mBAAmB,iBAAiB,eAAe,aAAa,eAAe,SAAS,qBAAqB;;;;;;KCnOjH;;UAGK;;WAEN,OAAO;;WAEP;;WAEA;;WAEA,YAAY;;;UAIN;;WAEN;WACA,MAAM;;;;;;WAMN;;WAEA;WACA;;WAEA;;;wBAIK,OAAO,aAAa,cAAc,UAAS,cAAmB;;;;;wBAoB9D,SAAS,aAAa,cAAc;;UAWnC;;WAEN,kBAAkB;;WAElB;;WAEA;;;;;WAKA,OAAO,QAAQ;;;UAIT;;WAEN;;WAEA;;WAEA;WACA;;WAEA;;;;;;;;wBASK,WAAW,MAAM,aAAa,iBAAc;;KAmBhD;;;;;;;KAQA;;UAGK,uBAAuB;;WAE7B,OAAO;;;;;WAKP;;;;;;WAMA;;WAEA;;;;;WAKA,OAAO;;;UAID;;WAEN;WACA;;WAEA;;WAEA;;;;;WAKA;;;UAIM;WACN,SAAS;;WAET;WACA,gBAAgB;WAChB,MAAM;;;;;;;WAON,UAAU;;WAEV,MAAM;;;;;WAKN,YAAYA;;WAEZ,OAAO;;WAEP,QAAQ;;;;;;;;;;;;;;wBAeH,YAAY,oBAAoB,cAAc,aAAa,cAAc,UAAS,iBAAsB;;;;;;;wBA+CxG,cAAc,UAAS;;UChQtB,qBAAqB;;WAE3B;WACA;;WAEA;;;;;;wBAOK,cAAc,kBAAkB,cAAc,iBAAe;;KAcjE;;UAGK,qBAAqB;;WAE3B;;WAEA;;;;;WAKA;;;UAIM,kBAAkB;;WAExB;;WAEA;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;WACA;WACA,gBAAgB;;WAEhB,MAAM;;WAEN;;WAEA;;;;;;;;;;wBAWK,aAAa,oBAAoB,cAAc,aAAa,cAAc,SAAS,eAAe;;UCzEjG,SAAS,UAAU;WACzB,SAAS;;WAET;;;;;;;wBAQK,cAAc,UAAU,YAAY,kBAAkB,MAAM,SAAS;;UAkBpE;;WAEN;WACA,oBAAoB;WACpB,aAAa;;WAEb;aAA8B;;;;UAIxB;WACN;WACA,WAAW;WACX,IAAI;;;UAIE;;WAEN,WAAW;;WAEX,IAAI;;WAEJ,eAAe;;WAEf;;WAEA;;;;;;;;wBASK,YAAY,eAAe,eAAe;;UAsEzC;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,cAAc,eAAe,cAAc,UAAS,iBAAsB;;UASzE;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,mBAAmB,eAAe,cAAc,SAAS,eAAe;;UASvE;WACN,OAAO,SAAS,wBAAwB;;;;;;wBAOnC,YAAY,KAAK,UAAU,gBAAgB,UAAS,iBAAsB,eAAe;;KC7N7F;;UAGK;WACN;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;;WAEA;;WAEA;;;;;;;;wBASK,OAAO,gBAAgB,oBAAoB;;KAW/C;;qBAKC;;UAGI;;WAEN,mBAAmB;;WAEnB;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM,iBAAiB;WACvB,MAAM;WACN;;WAEA;;WAEA,eAAe;;WAEf;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;WACA,gBAAgB;;WAEhB,eAAe;;;;;;;;;;;;;;;;;wBA8EV,iBAAiB,eAAe,cAAc,SAAS,mBAAmB;;UC7KzE;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;;UAIM,uBAAuB;;WAE7B;;WAEA;WACA;;;UAIM,iBAAiB;WACvB;;WAEA;;;KAIC,cAAc,iBAAiB;;UAG1B;;WAEN;WACA,kBAAkB;;;;;;;;;;;;wBAuFb,cAAc,eAAe,cAAc,UAAS,iBAAsB;;qBC3I7E;;UAGI;WACN;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;;;;;;;;;;;;UAaM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;KAIC,aAAa,eAAe,iBAAiB,aAAa;;UAGrD;;WAEN;;WAEA,IAAI,SAAS;WACb,kBAAkB;;WAElB;;WAEA;;;UAIM;WACN,eAAe;;WAEf;WACA,qBAAqB;WACrB,gBAAgB;;;KAIf;WACG;WAAyB;WAA0B,iBAAiB;WAAwB,aAAa;;WACzG;WAA2B;WAA0B;WAAoC;;WACzF;WAAuB;WAA0B;WAA2B;;WAC5E;WAAwB;WAA0B;;;UAGhD;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,mBAAmB;;;UAIb;WACN;WACA,SAAS;;WAET;;;UAIM;WACN;WACA;WACA;WACA,SAAS;;WAET;WACA,kBAAkB;;WAElB;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,gBAAgB;;;UAIV;;WAEN;;WAEA;;;;;;;wBAkDK,YAAY,QAAQ;;wBAyFpB,YAAY,MAAM,MAAM,QAAQ,wBAAwB,SAAS,gBAAgB;;;;;;wBA6BjF,cAAc,QAAQ,QAAQ,kBAAkB,cAAc,SAAS,gBAAgB;;UClUtF;WACN;;WAEA;WACA;;;UAIM;;WAEN;;WAEA;;;UAIM;;WAEN,OAAO,cAAc,qBAAqB;;WAE1C;;;UAIM,yBAAyB;;;;;WAK/B;;WAEA;;;;;WAKA;;;UAIM,sBAAsB;;WAE5B;WACA,OAAO;;WAEP;;WAEA;WACA,SAAS;;;UAIH;WACN;;WAEA;;;UAIM;;WAEN;;WAEA,gBAAgB;;;UAIV;WACN,SAAS;;WAET;WACA,gBAAgB;;WAEhB;WACA;WACA;;;;;;wBAqBK,UAAU,kBAAkB,cAAc,gBAAW;;;;;;;;;;;;;;;wBAoCrD,iBAAiB,oBAAoB,cAAc,aAAa,cAAc,SAAS,mBAAmB;;;;;;;;;;wBA6C1G,eAAe,aAAa,cAAc,SAAS,kBAAkB;;UC9KpE;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,YAAY;;;UAIN;WACN,gBAAgB;;WAEhB;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;;;;WAKA;;WAEA;;WAEA,sBAAsB;;WAEtB;;WAEA;;WAEA,kBAAkB;;;UAIZ;WACN;WACA;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;;;;;WAKA;;;UAIM;WACN;WACA;WACA,QAAQ;WACR,mBAAmB;;;UAIb;WACN;WACA;;WAEA;;WAEA;;WAEA;WACA,iBAAiB;WACjB,iBAAiB;WACjB,kBAAkB;;WAElB;;WAEA,MAAM;;;KAIL;;UAGK;WACN,MAAM;WACN;WACA;WACA;;WAEA;;WAEA;;WAEA,sBAAsB;;;UAIhB;;WAEN;WACA,mBAAmB;;WAEnB,gBAAgB;WAChB,mBAAmB;;;UAIb;;WAEN;;WAEA;;WAEA;;WAEA;aAAqB;aAAwB;eAA8B;eAA0B;;;;WAErG,MAAM;;;;;;;;;;;;;;;;;;wBAuUD,cAAc,eAAe,cAAc,SAAS,sBAAsB;;;;;;;wBAgC1E,SAAS,OAAO,eAAe,iBAAiB,eAAe,mBAAmB;;UCnjBjF;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,UAAU;;;UAIJ;WACN;WACA,UAAU,SAAS,OAAO;;;;;;WAM1B;;;;;;wBAOK,aAAa,SAAS,eAAe;;UClCpC;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN,SAAS;;WAET;;WAEA;;;;;;;wBAQK,WAAW,MAAM,SAAS,OAAO,qBAAqB,SAAS,gBAAgB"}
+{"version":3,"file":"index.d.ts","names":["Comparison"],"sources":["../src/budget.ts","../src/verdict.ts","../src/limit.ts","../src/merge.ts","../src/gate.ts","../src/control.ts","../src/ladder.ts","../src/model.ts","../src/tail.ts","../src/measures.ts","../src/power.ts","../src/rounds.ts"],"mappings":";;;UAQiB;;WAEN;;WAEA;WACA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;WACA;;WAEA;WACA,SAAS;;WAET;;;UAIM;WACN,SAAS;;WAET;WACA;WACA;WACA;WACA,gBAAgB;;;;;;;;;;;;wBAyDX,cAAc,iBAAiB,eAAe,aAAa,eAAe,SAAS,gBAAgB;;UAyBlG;WACN;WACA;;;;;;qBAOE,uBAAuB;;UAOnB;;WAEN;;WAEA;;WAEA,iBAAiB;;WAEjB;;;UAIM;WACN;WACA;;;UAIM,2BAA2B;;WAEjC;;WAEA,eAAe;;WAEf;;;;;;wBAOK,gBAAgB,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;wBAoDhC,mBAAmB,iBAAiB,eAAe,aAAa,eAAe,SAAS,qBAAqB;;;;;;KCnOjH;;UAGK;;WAEN,OAAO;;WAEP;;WAEA;;WAEA,YAAY;;;UAIN;;WAEN;WACA,MAAM;;;;;;WAMN;;WAEA;WACA;;WAEA;;;wBAIK,OAAO,aAAa,cAAc,UAAS,cAAmB;;;;;wBAoB9D,SAAS,aAAa,cAAc;;UAWnC;;WAEN,kBAAkB;;WAElB;;WAEA;;;;;WAKA,OAAO,QAAQ;;;UAIT;;WAEN;;WAEA;;WAEA;WACA;;WAEA;;;;;;;;wBASK,WAAW,MAAM,aAAa,iBAAc;;KAmBhD;;;;;;;KAQA;;UAGK,uBAAuB;;WAE7B,OAAO;;;;;WAKP;;;;;;WAMA;;WAEA;;;;;WAKA,OAAO;;;UAID;;WAEN;WACA;;WAEA;;WAEA;;;;;WAKA;;;UAIM;WACN,SAAS;;WAET;WACA,gBAAgB;WAChB,MAAM;;;;;;;WAON,UAAU;;WAEV,MAAM;;;;;WAKN,YAAYA;;WAEZ,OAAO;;WAEP,QAAQ;;;;;;;;;;;;;;wBAeH,YAAY,oBAAoB,cAAc,aAAa,cAAc,UAAS,iBAAsB;;;;;;;wBA+CxG,cAAc,UAAS;;UChQtB,qBAAqB;;WAE3B;WACA;;WAEA;;;;;;wBAOK,cAAc,kBAAkB,cAAc,iBAAe;;KAcjE;;UAGK,qBAAqB;;WAE3B;;WAEA;;;;;WAKA;;;UAIM,kBAAkB;;WAExB;;WAEA;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;WACA;WACA,gBAAgB;;WAEhB,MAAM;;WAEN;;WAEA;;;;;;;;;;wBAWK,aAAa,oBAAoB,cAAc,aAAa,cAAc,SAAS,eAAe;;UCzEjG,SAAS,UAAU;WACzB,SAAS;;WAET;;;;;;;wBAQK,cAAc,UAAU,YAAY,kBAAkB,MAAM,SAAS;;UAkBpE;;WAEN;WACA,oBAAoB;WACpB,aAAa;;WAEb;aAA8B;;;;UAIxB;WACN;WACA,WAAW;WACX,IAAI;;;UAIE;;WAEN,WAAW;;WAEX,IAAI;;WAEJ,eAAe;;WAEf;;WAEA;;;;;;;;wBASK,YAAY,eAAe,eAAe;;UAsEzC;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,cAAc,eAAe,cAAc,UAAS,iBAAsB;;UASzE;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,mBAAmB,eAAe,cAAc,SAAS,eAAe;;UASvE;WACN,OAAO,SAAS,wBAAwB;;;;;;wBAOnC,YAAY,KAAK,UAAU,gBAAgB,UAAS,iBAAsB,eAAe;;KC7N7F;;UAGK;WACN;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;;WAEA;;WAEA;;;;;;;;wBASK,OAAO,gBAAgB,oBAAoB;;KAW/C;;qBAKC;;UAGI;;WAEN,mBAAmB;;WAEnB;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM,iBAAiB;WACvB,MAAM;WACN;;WAEA;;WAEA,eAAe;;WAEf;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;WACA,gBAAgB;;WAEhB,eAAe;;;;;;;;;;;;;;;;;wBA8EV,iBAAiB,eAAe,cAAc,SAAS,mBAAmB;;UC7KzE;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;;UAIM,uBAAuB;;WAE7B;;WAEA;WACA;;;UAIM,iBAAiB;WACvB;;WAEA;;;KAIC,cAAc,iBAAiB;;UAG1B;;WAEN;WACA,kBAAkB;;;;;;;;;;;;wBAuFb,cAAc,eAAe,cAAc,UAAS,iBAAsB;;qBC3I7E;;UAGI;WACN;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;;;;;;;;;;;;UAaM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;KAIC,aAAa,eAAe,iBAAiB,aAAa;;UAGrD;;WAEN;;WAEA,IAAI,SAAS;WACb,kBAAkB;;WAElB;;WAEA;;;UAIM;WACN,eAAe;;WAEf;WACA,qBAAqB;WACrB,gBAAgB;;;KAIf;WACG;WAAyB;WAA0B,iBAAiB;WAAwB,aAAa;;WACzG;WAA2B;WAA0B;WAAoC;;WACzF;WAAuB;WAA0B;WAA2B;;WAC5E;WAAwB;WAA0B;;;UAGhD;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,mBAAmB;;;UAIb;WACN;WACA,SAAS;;WAET;;;UAIM;WACN;WACA;WACA;WACA,SAAS;;WAET;WACA,kBAAkB;;WAElB;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,gBAAgB;;;UAIV;;WAEN;;WAEA;;;;;;;wBAkDK,YAAY,QAAQ;;wBAyFpB,YAAY,MAAM,MAAM,QAAQ,wBAAwB,SAAS,gBAAgB;;;;;;wBA6BjF,cAAc,QAAQ,QAAQ,kBAAkB,cAAc,SAAS,gBAAgB;;qBC5T1F;;UAGI;WACN;WACA;;WAEA;;;UAIM;;WAEN;;WAEA;;WAEA;;;;;WAKA;;WAEA;;;;;WAKA,SAAS;aAAgB;aAAuB;;;;;;WAKhD;;;KAIC;;UAGK;WACN;WACA;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;WAEA;;WAEA;WACA,QAAQ;;WAER;;WAEA;;;;;WAKA;;;UAIM;;WAEN,eAAe;;WAEf;;WAEA,kBAAkB;;;UAoBZ;;WAEN;;WAEA;;;;;;;;;;wBAwBK,aAAa,2BAA2B,yBAAyB,eAAe,mBAAmB,UAAS,8BAAmC;;wBAe/I,UAAU,gBAAgB,aAAa,UAAS,mBAAwB;;wBAgDxE,aAAa,KAAK,UAAU;;UAiB3B;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;;qBAIE;;wBAGG,aAAa,QAAQ,aAAa,gBAAgB;;UAMjD;WACN;WACA;;;UAIM;WACN;WACA;;WAEA;;;wBAIK,UAAU,gBAAgB,YAAY;;UC3PrC;WACN;;WAEA;WACA;;;UAIM;;WAEN;;WAEA;;;UAIM;;WAEN,OAAO,cAAc,qBAAqB;;WAE1C;;;UAIM,yBAAyB;;;;;WAK/B;;WAEA;;;;;WAKA;;;UAIM,sBAAsB;;WAE5B;WACA,OAAO;;WAEP;;WAEA;WACA,SAAS;;;UAIH;WACN;;WAEA;;;UAIM;;WAEN;;WAEA,gBAAgB;;;UAIV;WACN,SAAS;;WAET;WACA,gBAAgB;;WAEhB;WACA;WACA;;;;;;wBAqBK,UAAU,kBAAkB,cAAc,gBAAW;;;;;;;;;;;;;;;wBAoCrD,iBAAiB,oBAAoB,cAAc,aAAa,cAAc,SAAS,mBAAmB;;;;;;;;;;wBA6C1G,eAAe,aAAa,cAAc,SAAS,kBAAkB;;UC9KpE;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,YAAY;;;UAIN;WACN,gBAAgB;;WAEhB;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;;;;WAKA;;WAEA;;WAEA,sBAAsB;;WAEtB;;WAEA;;WAEA,kBAAkB;;;UAIZ;WACN;WACA;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;;;;;WAKA;;;UAIM;WACN;WACA;WACA,QAAQ;WACR,mBAAmB;;;UAIb;WACN;WACA;;WAEA;;WAEA;;WAEA;WACA,iBAAiB;WACjB,iBAAiB;WACjB,kBAAkB;;WAElB;;WAEA,MAAM;;;KAIL;;UAGK;WACN,MAAM;WACN;WACA;WACA;;WAEA;;WAEA;;WAEA,sBAAsB;;;UAIhB;;WAEN;WACA,mBAAmB;;WAEnB,gBAAgB;WAChB,mBAAmB;;;UAIb;;WAEN;;WAEA;;WAEA;;WAEA;aAAqB;aAAwB;eAA8B;eAA0B;;;;WAErG,MAAM;;;UAIA;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;KAOC;;UAGK;WACN;WACA;WACA,SAAS;;WAET;;WAEA;;WAEA;WACA,mBAAmB;WACnB,mBAAmB;;;;;;;;;;;;;;;;;;wBAuUd,cAAc,eAAe,cAAc,SAAS,sBAAsB;;;;;;;wBAgC1E,SAAS,OAAO,eAAe,iBAAiB,eAAe,mBAAmB;;;;;;;;;;wBAiClF,WAAW,OAAO,eAAe,6BAA6B,eAAe,mBAAmB;;UCnoB/F;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,UAAU;;;UAIJ;WACN;WACA,UAAU,SAAS,OAAO;;;;;;WAM1B;;;;;;wBAOK,aAAa,SAAS,eAAe;;UClCpC;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN,SAAS;;WAET;;WAEA;;;;;;;wBAQK,WAAW,MAAM,SAAS,OAAO,qBAAqB,SAAS,gBAAgB"}

package/build/index.js

@@ -1,5 +1,5 @@
 import { i as firstArm, n as caseRatios, r as checkPairedTime, t as armOrder } from "./paired.js";
-import { bootstrapMedian, bootstrapTwo, compareRatios, holm, median, medianInterval, partialConjunction, quantile, quantileInterval, quantileSamples, signTest, signTestPower, wilson } from "@mark1russell7/rigor-statistics";
+import { bootstrapMedian, bootstrapTwo, compareRatios, holm, median, medianInterval, partialConjunction, quantile, quantileInterval, quantileSamples, signTest, signTestPower, twoSamplePermutation, wilson } from "@mark1russell7/rigor-statistics";
 import { random } from "@mark1russell7/rigor-generators";
 // src/budget.ts
 /**
@@ -1128,6 +1128,212 @@ function limitVerdict(candidate, aa, options) {
 function round$1(x) {
 	return Math.round(x * 1e4) / 1e4;
 }
+// src/model.ts
+/**
+* The CPU model test. A limit from reads of many runners can hide a model that reads apart. A check of the pool
+* passes when most reads come from the other models. This test reads each model against the rest, for each case.
+*
+* A read of `differs` is about these reads. A read of `no read` does not say that a model is the same, because few reads
+* have little power. Its row gives the smallest p that this count can reach, and the level. So a row at a size that cannot
+* reach the level says so with no flip. Its row also gives the smallest shift that this test can see at that count and
+* family. A model with too few reads gives `too-few`, and it stays out of the Holm family.
+*
+* Reads of one model are not independent of their runner image and their date. The p holds only if the reads of a case
+* can change places under the null. So a read of `differs` tells where to look, and it does not name a cause.
+*/
+/** The name of a record with no CPU model. */
+const UNKNOWN_MODEL = "unknown";
+/** This function groups a list of reads by a key, and sorts the groups by their key. */
+function groupsOf(reads, key) {
+	const groups = /* @__PURE__ */ new Map();
+	for (const read of reads) groups.set(key(read), [...groups.get(key(read)) ?? [], read]);
+	return [...groups].sort(([a], [b]) => a < b ? -1 : 1);
+}
+/** The count of the halvings of the search for a visible shift. */
+const VISIBLE_STEPS = 8;
+/** The count of the splits at the level of a test, for the default of its runs. The noise of a p at the level is then 10 %. */
+const COUNTS_AT_LEVEL = 100;
+/** The smallest default of the runs of a test. */
+const MIN_RUNS = 1e4;
+/** This function gives a shift that puts the moved values on one side of the rest: twice the span of the pool. */
+function farShift(values, rest) {
+	const all = [...values, ...rest];
+	const span = Math.max(...all) - Math.min(...all);
+	return span > 0 ? 2 * span : 1;
+}
+/** This function gives the p of the values with their median moved to `shift` from the median of the rest. */
+function pAtShift(values, rest, shift, options) {
+	const move = median(rest) + shift - median(values);
+	return twoSamplePermutation(values.map((value) => value + move), rest, options).p;
+}
+/**
+* This function finds the smallest shift of the median of `values` from the median of `rest` that gives a p of `level` or less.
+* It keeps the spread of the values, and it reads one direction: 1 for a shift up, and -1 for a shift down. The two
+* directions can differ, because the spread of the values and of the rest need not be symmetric.
+*
+* The level of a family is its alpha over its size, the strictest step of Holm. So the result errs on the side of a shift
+* that is too large. The cost is 9 permutation tests when the level is in reach, and 1 when it is not.
+*/
+function visibleShift(values, rest, level, direction, options = {}) {
+	const far = farShift(values, rest);
+	const floor = pAtShift(values, rest, direction * far, options);
+	if (floor > level) return {
+		shift: void 0,
+		floor
+	};
+	let low = 0;
+	let high = far;
+	for (let step = 0; step < VISIBLE_STEPS; step++) {
+		const middle = (low + high) / 2;
+		if (pAtShift(values, rest, direction * middle, options) <= level) high = middle;
+		else low = middle;
+	}
+	return {
+		shift: high,
+		floor
+	};
+}
+/** This function reads each CPU model against the rest, for each case. */
+function modelTest(reads, options = {}) {
+	const minReads = options.minReads ?? 5;
+	if (!Number.isSafeInteger(minReads) || minReads < 2) throw new RangeError(`modelTest: minReads is a whole number of 2 or more, not ${minReads}.`);
+	for (const read of reads) if (!Number.isFinite(read.value)) throw new RangeError(`modelTest: the read of the case "${read.case}" on "${read.model}" is ${read.value}, not a finite value.`);
+	const seed = String(options.seed ?? 1);
+	const alpha = options.alpha ?? .05;
+	const draft = groupsOf(reads, (r) => r.case).flatMap(([caseName, ofCase]) => groupsOf(ofCase, (r) => r.model).map(([model, ofModel]) => {
+		const values = ofModel.map((r) => r.value);
+		const rest = ofCase.filter((r) => r.model !== model).map((r) => r.value);
+		const restMedian = rest.length === 0 ? void 0 : median(rest);
+		const middle = median(values);
+		const row = {
+			case: caseName,
+			model,
+			reads: values.length,
+			median: middle,
+			low: Math.min(...values),
+			high: Math.max(...values),
+			restReads: rest.length,
+			restMedian,
+			shift: restMedian === void 0 ? void 0 : middle - restMedian
+		};
+		const asked = options.asked?.({
+			case: caseName,
+			model
+		}) ?? true;
+		return {
+			row,
+			asked,
+			inFamily: asked && values.length >= minReads && rest.length > 0,
+			values,
+			rest
+		};
+	}));
+	const family = draft.filter((d) => d.inFamily).length;
+	const level = alpha / Math.max(family, 1);
+	const runs = options.runs ?? Math.max(MIN_RUNS, Math.ceil(COUNTS_AT_LEVEL / level));
+	const read = draft.map((d) => {
+		const test = {
+			seed: `${seed} ${d.row.case} ${d.row.model}`,
+			runs
+		};
+		return {
+			...d,
+			test,
+			raw: d.inFamily ? twoSamplePermutation(d.values, d.rest, test).p : void 0
+		};
+	});
+	const tested = read.flatMap((d) => d.raw === void 0 ? [] : [d.raw]);
+	const corrected = holm(tested, alpha);
+	let next = 0;
+	const rows = read.map(({ row, asked, raw, values, rest, test }) => {
+		if (raw === void 0) return {
+			...row,
+			p: void 0,
+			status: asked ? "too-few" : "not asked",
+			visible: void 0,
+			floor: void 0,
+			level: void 0
+		};
+		const held = corrected[next++];
+		if (held === void 0) throw new Error("modelTest: the Holm family lost a row.");
+		const large = options.resolution === void 0 || Math.abs(row.shift ?? 0) >= options.resolution;
+		if (held.reject && large) return {
+			...row,
+			p: held.adjusted,
+			status: "differs",
+			visible: void 0,
+			floor: void 0,
+			level: void 0
+		};
+		const direction = (row.shift ?? 0) < 0 ? -1 : 1;
+		if (options.searchVisible === false) return {
+			...row,
+			p: held.adjusted,
+			status: "no read",
+			visible: void 0,
+			floor: pAtShift(values, rest, direction * farShift(values, rest), test),
+			level
+		};
+		const seen = visibleShift(values, rest, level, direction, test);
+		const visible = seen.shift === void 0 ? void 0 : Math.max(seen.shift, options.resolution ?? 0);
+		return {
+			...row,
+			p: held.adjusted,
+			status: "no read",
+			visible,
+			floor: seen.floor,
+			level
+		};
+	});
+	return {
+		rows,
+		family,
+		differs: rows.filter((r) => r.status === "differs")
+	};
+}
+/** This function gives the words of one row. No word says that a model is the same as the rest. */
+function modelRowText(row, family) {
+	const counts = `${row.reads} reads against ${row.restReads}, family ${family}`;
+	const name = `${row.case} on ${row.model}`;
+	if (row.status === "not asked") return `not asked: ${name} has ${row.reads} reads, and it is outside the family`;
+	if (row.status === "too-few") return `too few: ${name} has ${row.reads} reads against ${row.restReads}, and the test does not read it`;
+	const shift = (row.shift ?? 0).toFixed(3);
+	if (row.status === "differs") return `differs: ${name} has a shift of ${shift} at ${counts}, Holm p ${(row.p ?? 1).toFixed(4)}`;
+	const floor = row.floor ?? 1;
+	const level = row.level ?? 0;
+	const facts = `the smallest p is ${floor.toFixed(5)}, and the first level of Holm is ${level.toFixed(5)}`;
+	const head = `no read: ${name} has a shift of ${shift} at ${counts}`;
+	if (floor > level) return `${head}. This size cannot reach the level: ${facts}`;
+	if (row.visible === void 0) return `${head}. The test did not search for the shift that it can see: ${facts}`;
+	return `${head}. This test sees a shift of ${row.visible.toFixed(3)} or more: ${facts}`;
+}
+/** The words of a read at the ceiling. A pass at the ceiling is not a gain held. */
+const CEILING_ONLY = "gain not read on this model, ceiling only";
+/** This function gives the limit of a gain on one CPU model. A model with no earlier reads gets the ceiling. */
+function limitOnModel(limits, model) {
+	if (limits.ceiling < limits.gain) throw new RangeError(`limitOnModel: the ceiling ${limits.ceiling} is under the gain limit ${limits.gain}.`);
+	return limits.models.includes(model) ? {
+		limit: limits.gain,
+		holdsGain: true,
+		note: void 0
+	} : {
+		limit: limits.ceiling,
+		holdsGain: false,
+		note: CEILING_ONLY
+	};
+}
+/** This function gives the sorted CPU models of each arm, and tells if all the reads come from one known model. */
+function armModels(pairs) {
+	const distinct = (values) => [...new Set(values)].sort();
+	const a = distinct(pairs.map((p) => p.a ?? "unknown"));
+	const b = distinct(pairs.map((p) => p.b ?? "unknown"));
+	const all = distinct([...a, ...b]);
+	return {
+		a,
+		b,
+		one: all.length === 1 && all[0] !== "unknown"
+	};
+}
 // src/tail.ts
 /**
 * The tail limit verdict of a bench with few rounds. The median of each case has one limit, and a high value of its rounds
@@ -1947,6 +2153,76 @@ function winClaim(check, measure, entry, stratum) {
 		open: read.open
 	};
 }
+/**
+* This function tells what one real entry earned on a set of measures taken together. Each win of a control loses a partner, so one measure cannot call a win.
+* A control dominates the entry when the entry beats it on no measure of the set, and it beats the entry on at least one. A tie does not clear a control.
+* The entry survives when no control dominates it and it clears at least one control on each measure. The function adds no test: it reads the tests of `check`.
+* One win over any control is enough on a measure, so a beat clears only at the level divided by the count of controls there.
+* All measures must hold, so the measures add no division. Dominance reads the plain tests, because that is the cautious direction.
+* With a stratum, the function reads that stratum. Without one, it reads the rows with no stratum.
+* The function throws a `RangeError` for an empty set, a repeat, a measure with no rule and an entry that the rows do not hold.
+*/
+function jointClaim(check, measures, entry, stratum) {
+	if (measures.length === 0) throw new RangeError("jointClaim: the set of measures is empty.");
+	if (new Set(measures).size !== measures.length) throw new RangeError("jointClaim: the set of measures has a repeat.");
+	for (const measure of measures) {
+		const read = check.measures.find((m) => m.measure === measure);
+		if (read === void 0) throw new RangeError(`jointClaim: the measure ${JSON.stringify(measure)} has no rule.`);
+		if (!read.mixed && !read.entries.some((c) => c.group === entry && c.stratum === stratum)) throw new RangeError(`jointClaim: the entry ${JSON.stringify(entry)} has no rows on ${JSON.stringify(measure)}${stratumText(stratum)}.`);
+	}
+	const reads = check.reads.filter((r) => r.stratum === stratum && measures.includes(r.measure));
+	const names = [...new Set(reads.map((r) => r.control))];
+	const testOf = (measure, control) => {
+		const read = reads.find((r) => r.measure === measure && r.control === control);
+		return {
+			mine: read?.comparisons.find((c) => c.better === entry),
+			theirs: read?.against.find((c) => c.other === entry)
+		};
+	};
+	const noRead = (test) => test === void 0 || unread(test);
+	const testsOn = (measure) => names.flatMap((c) => {
+		const mine = testOf(measure, c).mine;
+		return mine === void 0 ? [] : [[c, mine]];
+	});
+	const hasPower = (test, tests) => test.shared >= pairsNeeded(check.alpha / (test.family * tests), 1);
+	const controls = names.map((control) => {
+		const beaten = measures.filter((m) => testOf(m, control).mine?.beats === true);
+		const lostTo = measures.filter((m) => testOf(m, control).theirs?.beats === true);
+		const missing = measures.filter((m) => noRead(testOf(m, control).mine));
+		return {
+			control,
+			beaten,
+			lostTo,
+			unread: missing,
+			dominates: beaten.length === 0 && lostTo.length > 0 && missing.length === 0
+		};
+	});
+	const measureReads = measures.map((measure) => ({
+		measure,
+		beats: names.filter((c) => testOf(measure, c).mine?.beats === true),
+		clears: testsOn(measure).filter(([, test]) => clearsPartner(test, testsOn(measure).length, check.alpha)).map(([c]) => c),
+		tied: names.filter((c) => testOf(measure, c).mine?.tied === true),
+		unread: names.filter((c) => noRead(testOf(measure, c).mine))
+	}));
+	const dominatedBy = controls.filter((c) => c.dominates).map((c) => c.control);
+	const equalledBy = controls.filter((c) => c.beaten.length === 0 && c.lostTo.length === 0 && c.unread.length === 0).map((c) => c.control);
+	const notAhead = measureReads.filter((m) => m.clears.length === 0).map((m) => m.measure);
+	const closed = measureReads.some((m) => {
+		const tests = testsOn(m.measure);
+		return m.clears.length === 0 && m.unread.length === 0 && tests.length > 0 && tests.every(([, test]) => hasPower(test, tests.length));
+	});
+	const undecided = controls.some((c) => c.beaten.length === 0 && c.lostTo.length > 0 && c.unread.length > 0);
+	return {
+		entry,
+		stratum,
+		verdict: dominatedBy.length > 0 ? "dominated" : closed ? "not-ahead" : notAhead.length > 0 || undecided ? "no-read" : "survives",
+		dominatedBy,
+		equalledBy,
+		notAhead,
+		controls,
+		measures: measureReads
+	};
+}
 // src/power.ts
 /**
 * The power test of a time verdict. It draws seeded benches with a known speedup and counts the verdicts that find it.
@@ -2061,6 +2337,6 @@ function timeRounds(arms, options) {
 		sink
 	};
 }
-export { BUDGET_LOOKS, GATE_LOOKS, LADDER_FORMAT, aaBand, aaSpread, armOrder, armVerdicts, balanceOrders, budgetLookSpend, budgetLooksVerdict, budgetVerdict, caseIntervals, caseRatios, caseTails, checkLadder, checkMeasures, checkPairedTime, controlFactor, firstArm, gateOf, ladderVerdict, limitVerdict, notSlowerVerdict, poolRecords, pooledBand, pooledLimitVerdict, pooledVerdict, rungVerdict, tailFalseFails, tailLimitVerdict, timeRounds, timeVerdict, verdictMargin, verdictPower, winClaim };
+export { BUDGET_LOOKS, CEILING_ONLY, GATE_LOOKS, LADDER_FORMAT, UNKNOWN_MODEL, aaBand, aaSpread, armModels, armOrder, armVerdicts, balanceOrders, budgetLookSpend, budgetLooksVerdict, budgetVerdict, caseIntervals, caseRatios, caseTails, checkLadder, checkMeasures, checkPairedTime, controlFactor, firstArm, gateOf, jointClaim, ladderVerdict, limitOnModel, limitVerdict, modelRowText, modelTest, notSlowerVerdict, poolRecords, pooledBand, pooledLimitVerdict, pooledVerdict, rungVerdict, tailFalseFails, tailLimitVerdict, timeRounds, timeVerdict, verdictMargin, verdictPower, visibleShift, winClaim };
 
 //# sourceMappingURL=index.js.map