@mark1russell7/rigor-bench 0.2.23 → 0.2.25
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +42 -0
- package/build/index.d.ts +60 -1
- package/build/index.d.ts.map +1 -1
- package/build/index.js +89 -2
- package/build/index.js.map +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -33,6 +33,7 @@ v.comparison; // the cases that gain and dro
|
|
|
33
33
|
|
|
34
34
|
| Function | What it gives |
|
|
35
35
|
| --- | --- |
|
|
36
|
+
| `growthVerdict(rounds, options)` | The growth of a cost for each step of size, from a ratio inside each job, against a limit, for example 15 times for each 10 times. |
|
|
36
37
|
| `firstArm(round, offset)` | The arm that runs first. The order turns each round, and an odd offset starts with arm `b`. |
|
|
37
38
|
| `armOrder(arms, round)` | The order of any count of arms in a round. Each pair of arms has each order within 1 after each count of rounds. A report that keeps no `first` for each pair states the order that it used, so that a reader can make `first` again. |
|
|
38
39
|
| `checkPairedTime(record)` | The record, or a `RangeError` for a time that is not a finite number above 0. |
|
|
@@ -257,6 +258,32 @@ There are four kinds of budget:
|
|
|
257
258
|
- A record states what was real and what was synthetic. A record with no entry in `real` gives `cannot decide`.
|
|
258
259
|
- `checkLadder` gives each problem of a ladder as data. `ladderVerdict` throws a `RangeError` for a ladder with problems.
|
|
259
260
|
|
|
261
|
+
## A claim on the growth of a cost
|
|
262
|
+
|
|
263
|
+
A ladder holds each size to a limit. A growth claim holds the step between sizes, for example "the time of one publish grows less than 15 times for each 10 times more subscribers". A cost that grows as n x n grows 100 times for each 10 times. A linear cost grows 10 times.
|
|
264
|
+
|
|
265
|
+
```ts
|
|
266
|
+
const v = growthVerdict(
|
|
267
|
+
[
|
|
268
|
+
{ job: "job-1", size: 1_000, value: 9.0 },
|
|
269
|
+
{ job: "job-1", size: 10_000, value: 51.8 },
|
|
270
|
+
{ job: "job-1", size: 100_000, value: 590 },
|
|
271
|
+
// ... the same sizes for each other job
|
|
272
|
+
],
|
|
273
|
+
{ maxGrowth: 15 },
|
|
274
|
+
);
|
|
275
|
+
|
|
276
|
+
v.outcome; // "within", "beyond" or "cannot decide"
|
|
277
|
+
v.steps[0].median; // the median growth of the jobs from 1,000 to 10,000, for a step of 10 times
|
|
278
|
+
v.steps[0].exponent; // the cost grows as the size to this power
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
- The verdict reads a ratio inside each job: the cost at the larger size over the cost at the smaller size. Each job runs on one runner, so a slow runner changes both values and the ratio keeps its value. The verdict needs no A/A arm.
|
|
282
|
+
- A step that is not `per` times in size is scaled to `per` times, with the exponent of the step. The default of `per` is 10.
|
|
283
|
+
- The interval of each step is the distribution-free interval of the median of the jobs. With 5 jobs it is the full range, with the coverage 0.94. A step is `within` when the high end is less than `maxGrowth`.
|
|
284
|
+
- A job that gives more than one value at a size gives their median.
|
|
285
|
+
- The flip test of a growth gate is the old code: it must give `beyond`.
|
|
286
|
+
|
|
260
287
|
## Time the rounds
|
|
261
288
|
|
|
262
289
|
`timeRounds` times one case in one process. Start a fresh process for each case and each pair, and give the index of the case as `offset`:
|
|
@@ -345,6 +372,21 @@ Each finding fails a gate, and each one carries its tests with `n`, `ties`, `p`
|
|
|
345
372
|
| `no-control` | A rule has no control and no `noControl`. |
|
|
346
373
|
| `control-late` | The date of `noControl` is before `asOf`. |
|
|
347
374
|
|
|
375
|
+
### A narrow spread
|
|
376
|
+
|
|
377
|
+
The finding `no-spread` reads "no pair separates". One entry far from the others stops that finding, but the measure does not separate the others. Each row of `spread` in a `MeasureCheck` thus has a report of how many entries carry the separated pairs.
|
|
378
|
+
|
|
379
|
+
| Field | Meaning |
|
|
380
|
+
| --- | --- |
|
|
381
|
+
| `entries` | The real entries with at least one pair that has a read. |
|
|
382
|
+
| `share` | The median, over these entries, of the share of the other entries that a test separates from the entry. A pair with no read does not count. |
|
|
383
|
+
| `apart` | The entries that a test separates from more than half of the others that they have a read with. |
|
|
384
|
+
| `narrow` | Some pair separates and `share` is under `NARROW_SPREAD_SHARE`. |
|
|
385
|
+
|
|
386
|
+
- `narrow` is a report. It gives no finding, and it fails no gate.
|
|
387
|
+
- `NARROW_SPREAD_SHARE` is 0.1. This value is a fitted default from a gap in the rows of one project, and it is not a gate limit.
|
|
388
|
+
- One Holm family holds all pairs of all strata. Thus more strata in one call give fewer separated pairs and a lower `share`. Compare a `share` only with a `share` from the same strata.
|
|
389
|
+
|
|
348
390
|
`winClaim(check, measure, entry, stratum)` gives `alone`, `show`, `likeControl`, `censored` and `open`. The fields are about this entry. The field `likeControl` is true when this entry does not beat a control on the measure and beats it on no partner. A win by another entry clears nothing. The field `alone` is true only when this entry itself beats each control on the measure, so a measure with no control is not `alone`.
|
|
349
391
|
|
|
350
392
|
`jointClaim(check, measures, entry, stratum)` reads a set of measures together, because one measure cannot call a win when each control wins a measure and loses a partner:
|
package/build/index.d.ts
CHANGED
|
@@ -124,6 +124,52 @@ export declare function budgetLookSpend(looks: readonly BudgetLook[]): number;
|
|
|
124
124
|
* - The field `fails` is true only for the outcome `beyond`. Tell a final `cannot decide` to the reader of the run.
|
|
125
125
|
*/
|
|
126
126
|
export declare function budgetLooksVerdict(rounds: readonly BudgetRound[], aa: readonly BudgetRound[], options: BudgetLooksOptions): BudgetLooksVerdict;
|
|
127
|
+
/** One value of one job at one size. A job that gives more than one value at a size gives the median of its values. */
|
|
128
|
+
interface GrowthRound {
|
|
129
|
+
/** The name of the job: one runner, which gives each size. */
|
|
130
|
+
readonly job: string;
|
|
131
|
+
/** The size, for example the count of subscribers. It is finite and more than 0. */
|
|
132
|
+
readonly size: number;
|
|
133
|
+
/** The cost at the size, for example a time in ms. It is finite and more than 0. */
|
|
134
|
+
readonly value: number;
|
|
135
|
+
}
|
|
136
|
+
/** The three outcomes of a growth verdict. */
|
|
137
|
+
type GrowthOutcome = "within" | "beyond" | "cannot decide";
|
|
138
|
+
/** The options of a growth verdict. */
|
|
139
|
+
interface GrowthOptions {
|
|
140
|
+
/** The largest growth of the cost that the claim permits for each step of `per` times in size. It is more than 1. */
|
|
141
|
+
readonly maxGrowth: number;
|
|
142
|
+
/** The factor of size that `maxGrowth` refers to. The default is 10. */
|
|
143
|
+
readonly per?: number;
|
|
144
|
+
/** The level of the interval of the median growth of each step. The default is 0.95. */
|
|
145
|
+
readonly level?: number;
|
|
146
|
+
}
|
|
147
|
+
/** The verdict of one step, from one size to the next larger size. */
|
|
148
|
+
interface GrowthStep {
|
|
149
|
+
readonly from: number;
|
|
150
|
+
readonly to: number;
|
|
151
|
+
/** The growth of each job that gives both sizes, for a step of `per` times in size, in the order of the job names. */
|
|
152
|
+
readonly growths: readonly number[];
|
|
153
|
+
/** The median of `growths`. */
|
|
154
|
+
readonly median: number;
|
|
155
|
+
readonly low: number;
|
|
156
|
+
readonly high: number;
|
|
157
|
+
/** The coverage of the interval from `low` to `high`. It is less than the level with 5 jobs or fewer. */
|
|
158
|
+
readonly coverage: number;
|
|
159
|
+
/** The exponent of the median growth: the cost grows as the size to this power. */
|
|
160
|
+
readonly exponent: number;
|
|
161
|
+
readonly outcome: GrowthOutcome;
|
|
162
|
+
}
|
|
163
|
+
/** The verdict of a cost over all its sizes. */
|
|
164
|
+
interface GrowthVerdict {
|
|
165
|
+
/** It is `beyond` when a step is beyond, `within` when each step is within, and `cannot decide` in the other cases. */
|
|
166
|
+
readonly outcome: GrowthOutcome;
|
|
167
|
+
readonly maxGrowth: number;
|
|
168
|
+
readonly per: number;
|
|
169
|
+
readonly steps: readonly GrowthStep[];
|
|
170
|
+
}
|
|
171
|
+
/** This function gives the growth verdict of a cost over its sizes, from the values of one or more jobs. */
|
|
172
|
+
export declare function growthVerdict(rounds: readonly GrowthRound[], options: GrowthOptions): GrowthVerdict;
|
|
127
173
|
/**
|
|
128
174
|
* The two kinds of an A/A band.
|
|
129
175
|
* - The band `case` holds the ratio of one A/A case. It is wide, so it finds only a large change.
|
|
@@ -1147,7 +1193,20 @@ interface SpreadRead {
|
|
|
1147
1193
|
readonly tooFew: number;
|
|
1148
1194
|
/** The largest absolute median paired difference of a pair. */
|
|
1149
1195
|
readonly widest: number | undefined;
|
|
1196
|
+
/** The real entries with at least one pair that has a read. */
|
|
1197
|
+
readonly entries: number;
|
|
1198
|
+
/** The median, over these entries, of the share of the other entries that a test separates from the entry. A pair with no read does not count in a share. It is undefined with no such entry. */
|
|
1199
|
+
readonly share: number | undefined;
|
|
1200
|
+
/** The entries that a test separates from more than half of the others that they have a read with. */
|
|
1201
|
+
readonly apart: readonly string[];
|
|
1202
|
+
/** Some pair separates and `share` is under `NARROW_SPREAD_SHARE`. A few entries then carry the separated pairs. This field is a report, and no finding reads it. */
|
|
1203
|
+
readonly narrow: boolean;
|
|
1150
1204
|
}
|
|
1205
|
+
/**
|
|
1206
|
+
* The limit of `SpreadRead.narrow`. This value is a fitted default and not a limit of a gate.
|
|
1207
|
+
* It comes from a gap in the rows of one project, and it has no test on rows that had no part in its choice.
|
|
1208
|
+
*/
|
|
1209
|
+
export declare const NARROW_SPREAD_SHARE = .1;
|
|
1151
1210
|
/** The result of one control and one partner on one stratum. */
|
|
1152
1211
|
interface PartnerRead {
|
|
1153
1212
|
readonly partner: string;
|
|
@@ -1429,5 +1488,5 @@ interface Rounds {
|
|
|
1429
1488
|
* keeps the work.
|
|
1430
1489
|
*/
|
|
1431
1490
|
export declare function timeRounds(arms: Readonly<Record<Arm, () => number>>, options: RoundsOptions): Rounds;
|
|
1432
|
-
export { type Arm, type ArmModels, type ArmPair, type Balanced, type Band, type BandKind, type BandOptions, type BudgetLook, type BudgetLooksOptions, type BudgetLooksVerdict, type BudgetMore, type BudgetOptions, type BudgetOutcome, type BudgetResult, type BudgetRound, type BudgetVerdict, type CaseBudget, type CaseGate, type CaseInterval, type CaseLimit, type CaseRatio, type CaseResults, type CaseTail, type CaseTailLimit, type Comparison, type ControlFactors, type ControlOptions, type ControlRead, type ControlRow, type EntryCount, type Gate, type GateCaseOutcome, type GateJobRead, type GateOutcome, type GroupCounts, type GroupFactor, type GuardRead, type JobLimitVerdict, type JobRecords, type JobVerdict, type JointClaim, type JointControl, type JointMeasure, type JointVerdict, type Ladder, type LadderDimension, type LadderOptions, type LadderVerdict, type LimitOnModel, type LimitOptions, type LimitOutcome, type LimitVerdict, type MeasureCheck, type MeasureCheckOptions, type MeasureChecks, type MeasureFinding, type MeasureFindingKind, type MeasureRow, type MeasureRule, type MeasuredFactor, type MedianBudget, type ModelLimits, type ModelRead, type ModelRow, type ModelStatus, type ModelTest, type ModelTestOptions, type NoControl, type NoFactor, type NoiseCheck, type NotSlowerOptions, type NotSlowerRead, type NotSlowerVerdict, type OrderSize, type PairedTime, type PartnerRead, type PoolFacts, type PoolOptions, type PooledJob, type PooledLimitVerdict, type PooledRecords, type PooledVerdict, type Power, type PowerOptions, type QuantileBudget, type RateBudget, type ResultRead, type ResultRow, type ResultRows, type ResultVerdict, type Rounds, type RoundsOptions, type RowRead, type RunPairs, type Rung, type RungBudget, type RungEvidence, type RungRecord, type RungVerdict, type SpreadRead, type TailFalseFails, type TailGainCase, type TailLimit, type TailLimitOptions, type TailLimitVerdict, type TailRuleOptions, type TimeOutcome, type TimeVerdict, type VerdictOptions, type VerdictRule, type VisibleShift, type WinClaim, type WorstBudget, armOrder, caseRatios, checkPairedTime, firstArm, orderSize };
|
|
1491
|
+
export { type Arm, type ArmModels, type ArmPair, type Balanced, type Band, type BandKind, type BandOptions, type BudgetLook, type BudgetLooksOptions, type BudgetLooksVerdict, type BudgetMore, type BudgetOptions, type BudgetOutcome, type BudgetResult, type BudgetRound, type BudgetVerdict, type CaseBudget, type CaseGate, type CaseInterval, type CaseLimit, type CaseRatio, type CaseResults, type CaseTail, type CaseTailLimit, type Comparison, type ControlFactors, type ControlOptions, type ControlRead, type ControlRow, type EntryCount, type Gate, type GateCaseOutcome, type GateJobRead, type GateOutcome, type GroupCounts, type GroupFactor, type GrowthOptions, type GrowthOutcome, type GrowthRound, type GrowthStep, type GrowthVerdict, type GuardRead, type JobLimitVerdict, type JobRecords, type JobVerdict, type JointClaim, type JointControl, type JointMeasure, type JointVerdict, type Ladder, type LadderDimension, type LadderOptions, type LadderVerdict, type LimitOnModel, type LimitOptions, type LimitOutcome, type LimitVerdict, type MeasureCheck, type MeasureCheckOptions, type MeasureChecks, type MeasureFinding, type MeasureFindingKind, type MeasureRow, type MeasureRule, type MeasuredFactor, type MedianBudget, type ModelLimits, type ModelRead, type ModelRow, type ModelStatus, type ModelTest, type ModelTestOptions, type NoControl, type NoFactor, type NoiseCheck, type NotSlowerOptions, type NotSlowerRead, type NotSlowerVerdict, type OrderSize, type PairedTime, type PartnerRead, type PoolFacts, type PoolOptions, type PooledJob, type PooledLimitVerdict, type PooledRecords, type PooledVerdict, type Power, type PowerOptions, type QuantileBudget, type RateBudget, type ResultRead, type ResultRow, type ResultRows, type ResultVerdict, type Rounds, type RoundsOptions, type RowRead, type RunPairs, type Rung, type RungBudget, type RungEvidence, type RungRecord, type RungVerdict, type SpreadRead, type TailFalseFails, type TailGainCase, type TailLimit, type TailLimitOptions, type TailLimitVerdict, type TailRuleOptions, type TimeOutcome, type TimeVerdict, type VerdictOptions, type VerdictRule, type VisibleShift, type WinClaim, type WorstBudget, armOrder, caseRatios, checkPairedTime, firstArm, orderSize };
|
|
1433
1492
|
//# sourceMappingURL=index.d.ts.map
|
package/build/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","names":["Comparison"],"sources":["../src/budget.ts","../src/verdict.ts","../src/limit.ts","../src/merge.ts","../src/gate.ts","../src/control.ts","../src/ladder.ts","../src/model.ts","../src/tail.ts","../src/measures.ts","../src/power.ts","../src/results.ts","../src/rounds.ts"],"mappings":";;;UAQiB;;WAEN;;WAEA;WACA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;WACA;;WAEA;WACA,SAAS;;WAET;;;UAIM;WACN,SAAS;;WAET;WACA;WACA;WACA;WACA,gBAAgB;;;;;;;;;;;;wBAyDX,cAAc,iBAAiB,eAAe,aAAa,eAAe,SAAS,gBAAgB;;UAyBlG;WACN;WACA;;;;;;qBAOE,uBAAuB;;UAOnB;;WAEN;;WAEA;;WAEA,iBAAiB;;WAEjB;;;UAIM;WACN;WACA;;;UAIM,2BAA2B;;WAEjC;;WAEA,eAAe;;WAEf;;;;;;wBAOK,gBAAgB,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;wBAoDhC,mBAAmB,iBAAiB,eAAe,aAAa,eAAe,SAAS,qBAAqB;;;;;;
|
|
1
|
+
{"version":3,"file":"index.d.ts","names":["Comparison"],"sources":["../src/budget.ts","../src/growth.ts","../src/verdict.ts","../src/limit.ts","../src/merge.ts","../src/gate.ts","../src/control.ts","../src/ladder.ts","../src/model.ts","../src/tail.ts","../src/measures.ts","../src/power.ts","../src/results.ts","../src/rounds.ts"],"mappings":";;;UAQiB;;WAEN;;WAEA;WACA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;WACA;;WAEA;WACA,SAAS;;WAET;;;UAIM;WACN,SAAS;;WAET;WACA;WACA;WACA;WACA,gBAAgB;;;;;;;;;;;;wBAyDX,cAAc,iBAAiB,eAAe,aAAa,eAAe,SAAS,gBAAgB;;UAyBlG;WACN;WACA;;;;;;qBAOE,uBAAuB;;UAOnB;;WAEN;;WAEA;;WAEA,iBAAiB;;WAEjB;;;UAIM;WACN;WACA;;;UAIM,2BAA2B;;WAEjC;;WAEA,eAAe;;WAEf;;;;;;wBAOK,gBAAgB,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;wBAoDhC,mBAAmB,iBAAiB,eAAe,aAAa,eAAe,SAAS,qBAAqB;;UCrO5G;;WAEN;;WAEA;;WAEA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;WACA;WACA;;WAEA;;WAEA;WACA,SAAS;;;UAIH;;WAEN,SAAS;WACT;WACA;WACA,gBAAgB;;;wBAIX,cAAc,iBAAiB,eAAe,SAAS,gBAAgB;;;;;;KC/C3E;;UAGK;;WAEN,OAAO;;WAEP;;WAEA;;WAEA,YAAY;;;UAIN;;WAEN;WACA,MAAM;;;;;;WAMN;;WAEA;WACA;;WAEA;;;wBAIK,OAAO,aAAa,cAAc,UAAS,cAAmB;;;;;wBAoB9D,SAAS,aAAa,cAAc;;UAWnC;;WAEN,kBAAkB;;WAElB;;WAEA;;;;;WAKA,OAAO,QAAQ;;;UAIT;;WAEN;;WAEA;;WAEA;WACA;;WAEA;;;;;;;;wBASK,WAAW,MAAM,aAAa,iBAAc;;KAmBhD;;;;;;;KAQA;;UAGK,uBAAuB;;WAE7B,OAAO;;;;;WAKP;;;;;;WAMA;;WAEA;;;;;WAKA,OAAO;;;UAID;;WAEN;WACA;;WAEA;;WAEA;;;;;WAKA;;;UAIM;WACN,SAAS;;WAET;WACA,gBAAgB;WAChB,MAAM;;;;;;;WAON,UAAU;;WAEV,MAAM;;;;;WAKN,YAAYA;;WAEZ,OAAO;;WAEP,QAAQ;;;;;;;;;;;;;;wBAeH,YAAY,oBAAoB,cAAc,aAAa,cAAc,UAAS,iBAAsB;;;;;;;wBA+CxG,cAAc,UAAS;;UChQtB,qBAAqB;;WAE3B;WACA;;WAEA;;;;;;wBAOK,cAAc,kBAAkB,cAAc,iBAAe;;KAcjE;;UAGK,qBAAqB;;WAE3B;;WAEA;;;;;WAKA;;;UAIM,kBAAkB;;WAExB;;WAEA;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;WACA;WACA,gBAAgB;;WAEhB,MAAM;;WAEN;;WAEA;;;;;;;;;;wBAWK,aAAa,oBAAoB,cAAc,aAAa,cAAc,SAAS,eAAe;;UCzEjG,SAAS,UAAU;WACzB,SAAS;;WAET;;;;;;;wBAQK,cAAc,UAAU,YAAY,kBAAkB,MAAM,SAAS;;UAkBpE;;WAEN;WACA,oBAAoB;WACpB,aAAa;;WAEb;aAA8B;;;;UAIxB;WACN;WACA,WAAW;WACX,IAAI;;;UAIE;;WAEN,WAAW;;WAEX,IAAI;;WAEJ,eAAe;;WAEf;;WAEA;;;;;;;;wBASK,YAAY,eAAe,eAAe;;UAsEzC;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,cAAc,eAAe,cAAc,UAAS,iBAAsB;;UASzE;WACN;WACA,SAAS;;;UAIH;;WAEN,SAAS;;WAET,QAAQ;;WAER,eAAe;;WAEf;;WAEA;;WAEA;;WAEA;;;;;;;;wBAWK,mBAAmB,eAAe,cAAc,SAAS,eAAe;;UASvE;WACN,OAAO,SAAS,wBAAwB;;;;;;wBAOnC,YAAY,KAAK,UAAU,gBAAgB,UAAS,iBAAsB,eAAe;;KC7N7F;;UAGK;WACN;WACA,SAAS;;;UAIH;WACN,SAAS;;WAET;;WAEA;;WAEA;;;;;;;;wBASK,OAAO,gBAAgB,oBAAoB;;KAW/C;;qBAKC;;UAGI;;WAEN,mBAAmB;;WAEnB;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM,iBAAiB;WACvB,MAAM;WACN;;WAEA;;WAEA,eAAe;;WAEf;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;WACA,gBAAgB;;WAEhB,eAAe;;;;;;;;;;;;;;;;;wBA8EV,iBAAiB,eAAe,cAAc,SAAS,mBAAmB;;UC7KzE;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;;UAIM,uBAAuB;;WAE7B;;WAEA;WACA;;;UAIM,iBAAiB;WACvB;;WAEA;;;KAIC,cAAc,iBAAiB;;UAG1B;;WAEN;WACA,kBAAkB;;;;;;;;;;;;wBAuFb,cAAc,eAAe,cAAc,UAAS,iBAAsB;;qBC3I7E;;UAGI;WACN;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;;;;;;;;;;;;UAaM;WACN;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;;KAIC,aAAa,eAAe,iBAAiB,aAAa;;UAGrD;;WAEN;;WAEA,IAAI,SAAS;WACb,kBAAkB;;WAElB;;WAEA;;;UAIM;WACN,eAAe;;WAEf;WACA,qBAAqB;WACrB,gBAAgB;;;KAIf;WACG;WAAyB;WAA0B,iBAAiB;WAAwB,aAAa;;WACzG;WAA2B;WAA0B;WAAoC;;WACzF;WAAuB;WAA0B;WAA2B;;WAC5E;WAAwB;WAA0B;;;UAGhD;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,mBAAmB;;;UAIb;WACN;WACA,SAAS;;WAET;;;UAIM;WACN;WACA;WACA;WACA,SAAS;;WAET;WACA,kBAAkB;;WAElB;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;WACA,gBAAgB;;;UAIV;;WAEN;;WAEA;;;;;;;wBAkDK,YAAY,QAAQ;;wBAyFpB,YAAY,MAAM,MAAM,QAAQ,wBAAwB,SAAS,gBAAgB;;;;;;wBA6BjF,cAAc,QAAQ,QAAQ,kBAAkB,cAAc,SAAS,gBAAgB;;qBC5T1F;;UAGI;WACN;WACA;;WAEA;;;UAIM;;WAEN;;WAEA;;WAEA;;;;;WAKA;;WAEA;;;;;WAKA,SAAS;aAAgB;aAAuB;;;;;;WAKhD;;;KAIC;;UAGK;WACN;WACA;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;WAEA;;WAEA;WACA,QAAQ;;WAER;;WAEA;;;;;WAKA;;;UAIM;;WAEN,eAAe;;WAEf;;WAEA,kBAAkB;;;UAoBZ;;WAEN;;WAEA;;;;;;;;;;wBAwBK,aAAa,2BAA2B,yBAAyB,eAAe,mBAAmB,UAAS,8BAAmC;;wBAe/I,UAAU,gBAAgB,aAAa,UAAS,mBAAwB;;wBAgDxE,aAAa,KAAK,UAAU;;UAiB3B;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;;qBAIE;;wBAGG,aAAa,QAAQ,aAAa,gBAAgB;;UAMjD;WACN;WACA;;;UAIM;WACN;WACA;;WAEA;;;wBAIK,UAAU,gBAAgB,YAAY;;UC3PrC;WACN;;WAEA;WACA;;;UAIM;;WAEN;;WAEA;;;UAIM;;WAEN,OAAO,cAAc,qBAAqB;;WAE1C;;;UAIM,yBAAyB;;;;;WAK/B;;WAEA;;;;;WAKA;;;UAIM,sBAAsB;;WAE5B;WACA,OAAO;;WAEP;;WAEA;WACA,SAAS;;;UAIH;WACN;;WAEA;;;UAIM;;WAEN;;WAEA,gBAAgB;;;UAIV;WACN,SAAS;;WAET;WACA,gBAAgB;;WAEhB;WACA;WACA;;;;;;wBAqBK,UAAU,kBAAkB,cAAc,gBAAW;;;;;;;;;;;;;;;wBAoCrD,iBAAiB,oBAAoB,cAAc,aAAa,cAAc,SAAS,mBAAmB;;;;;;;;;;wBA6C1G,eAAe,aAAa,cAAc,SAAS,kBAAkB;;UC9KpE;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;WAKA,SAAS,SAAS;;WAElB,YAAY;;;UAIN;WACN,gBAAgB;;WAEhB;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;WAEA;;WAEA;WACA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;;;;WAKA;;WAEA;;WAEA,sBAAsB;;WAEtB;;WAEA;;WAEA,kBAAkB;;;UAIZ;WACN;WACA;WACA;WACA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;qBAOE;;UAGI;WACN;;;;;WAKA;;;UAIM;WACN;WACA;WACA,QAAQ;WACR,mBAAmB;;;UAIb;WACN;WACA;;WAEA;;WAEA;;WAEA;WACA,iBAAiB;WACjB,iBAAiB;WACjB,kBAAkB;;WAElB;;WAEA,MAAM;;;KAIL;;UAGK;WACN,MAAM;WACN;WACA;WACA;;WAEA;;WAEA;;WAEA,sBAAsB;;;UAIhB;;WAEN;WACA,mBAAmB;;WAEnB,gBAAgB;WAChB,mBAAmB;;;UAIb;;WAEN;;WAEA;;WAEA;;WAEA;aAAqB;aAAwB;eAA8B;eAA0B;;;;WAErG,MAAM;;;UAIA;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;KAOC;;UAGK;WACN;WACA;WACA,SAAS;;WAET;;WAEA;;WAEA;WACA,mBAAmB;WACnB,mBAAmB;;;;;;;;;;;;;;;;;;wBAoWd,cAAc,eAAe,cAAc,SAAS,sBAAsB;;;;;;;wBAgC1E,SAAS,OAAO,eAAe,iBAAiB,eAAe,mBAAmB;;;;;;;;;;wBAiClF,WAAW,OAAO,eAAe,6BAA6B,eAAe,mBAAmB;;UCrrB/F;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,UAAU;;;UAIJ;WACN;WACA,UAAU,SAAS,OAAO;;;;;;WAM1B;;;;;;wBAOK,aAAa,SAAS,eAAe;;;;;;;;;;;;;UC7BpC;WACN;;WAEA;;WAEA;;WAEA;;;UAIM;WACN;WACA;WACA;;;UAIM;WACN,eAAe;WACf,eAAe;;;UAIT,mBAAmB;WACzB;;WAEA;;;UAIM;WACN,KAAK;WACL;;;UAIM;;WAEN;;WAEA,gBAAgB;;WAEhB,iBAAiB;;WAEjB,iBAAiB;;WAEjB,gBAAgB;;WAEhB;;;wBAMK,WAAW,mBAAmB;;;;;wBAuC9B,cAAc,kBAAkB,eAAe,MAAM,aAAa;;UCnGjE;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;WACN,SAAS;;WAET;;WAEA;;;;;;;wBAQK,WAAW,MAAM,SAAS,OAAO,qBAAqB,SAAS,gBAAgB"}
|
package/build/index.js
CHANGED
|
@@ -212,6 +212,70 @@ function budgetLooksVerdict(rounds, aa, options) {
|
|
|
212
212
|
function round$3(x) {
|
|
213
213
|
return Math.round(x * 1e4) / 1e4;
|
|
214
214
|
}
|
|
215
|
+
// src/growth.ts
|
|
216
|
+
/**
|
|
217
|
+
* The growth verdict of a cost over sizes: does the cost grow slower than a limit, for each step of size? An example is
|
|
218
|
+
* the claim "the time of one publish grows less than 15 times for each 10 times more subscribers". A cost that grows as
|
|
219
|
+
* n x n grows 100 times for each 10 times, and a linear cost grows 10 times.
|
|
220
|
+
*
|
|
221
|
+
* The verdict reads a ratio inside each job: the cost at the larger size over the cost at the smaller size, on the same
|
|
222
|
+
* runner. So a slow runner changes both sizes, and the ratio keeps its value. The verdict needs no A/A arm.
|
|
223
|
+
*/
|
|
224
|
+
/** This function gives the growth verdict of a cost over its sizes, from the values of one or more jobs. */
|
|
225
|
+
function growthVerdict(rounds, options) {
|
|
226
|
+
const per = options.per ?? 10;
|
|
227
|
+
const level = options.level ?? .95;
|
|
228
|
+
if (!(Number.isFinite(options.maxGrowth) && options.maxGrowth > 1)) throw new RangeError("growthVerdict: maxGrowth is not a finite number more than 1.");
|
|
229
|
+
if (!(Number.isFinite(per) && per > 1)) throw new RangeError("growthVerdict: per is not a finite number more than 1.");
|
|
230
|
+
const byJob = /* @__PURE__ */ new Map();
|
|
231
|
+
for (const round of rounds) {
|
|
232
|
+
if (!(Number.isFinite(round.size) && round.size > 0)) throw new RangeError(`growthVerdict: job ${round.job} has the size ${round.size}, which is not finite and more than 0.`);
|
|
233
|
+
if (!(Number.isFinite(round.value) && round.value > 0)) throw new RangeError(`growthVerdict: job ${round.job} has the value ${round.value} at the size ${round.size}, which is not finite and more than 0.`);
|
|
234
|
+
const sizes = byJob.get(round.job) ?? /* @__PURE__ */ new Map();
|
|
235
|
+
byJob.set(round.job, sizes);
|
|
236
|
+
const values = sizes.get(round.size) ?? [];
|
|
237
|
+
sizes.set(round.size, values);
|
|
238
|
+
values.push(round.value);
|
|
239
|
+
}
|
|
240
|
+
const sizes = [...new Set(rounds.map((round) => round.size))].sort((a, b) => a - b);
|
|
241
|
+
if (sizes.length < 2) throw new RangeError("growthVerdict: the rounds give fewer than 2 sizes.");
|
|
242
|
+
const jobs = [...byJob.keys()].sort();
|
|
243
|
+
const steps = [];
|
|
244
|
+
for (let i = 0; i + 1 < sizes.length; i++) {
|
|
245
|
+
const from = sizes[i];
|
|
246
|
+
const to = sizes[i + 1];
|
|
247
|
+
const growths = [];
|
|
248
|
+
for (const job of jobs) {
|
|
249
|
+
const values = byJob.get(job);
|
|
250
|
+
const small = values.get(from);
|
|
251
|
+
const large = values.get(to);
|
|
252
|
+
if (small === void 0 || large === void 0) continue;
|
|
253
|
+
const exponent = Math.log(median(large) / median(small)) / Math.log(to / from);
|
|
254
|
+
growths.push(Math.pow(per, exponent));
|
|
255
|
+
}
|
|
256
|
+
if (growths.length === 0) throw new RangeError(`growthVerdict: no job gives both the size ${from} and the size ${to}.`);
|
|
257
|
+
const interval = medianInterval(growths, level);
|
|
258
|
+
const middle = median(growths);
|
|
259
|
+
const outcome = interval.high < options.maxGrowth ? "within" : interval.low > options.maxGrowth ? "beyond" : "cannot decide";
|
|
260
|
+
steps.push({
|
|
261
|
+
from,
|
|
262
|
+
to,
|
|
263
|
+
growths,
|
|
264
|
+
median: middle,
|
|
265
|
+
low: interval.low,
|
|
266
|
+
high: interval.high,
|
|
267
|
+
coverage: interval.coverage,
|
|
268
|
+
exponent: Math.log(middle) / Math.log(per),
|
|
269
|
+
outcome
|
|
270
|
+
});
|
|
271
|
+
}
|
|
272
|
+
return {
|
|
273
|
+
outcome: steps.some((step) => step.outcome === "beyond") ? "beyond" : steps.every((step) => step.outcome === "within") ? "within" : "cannot decide",
|
|
274
|
+
maxGrowth: options.maxGrowth,
|
|
275
|
+
per,
|
|
276
|
+
steps
|
|
277
|
+
};
|
|
278
|
+
}
|
|
215
279
|
// src/gate.ts
|
|
216
280
|
/**
|
|
217
281
|
* The gate of a bench: the result that a check of a merge reads. A verdict gives one of the words of `LimitOutcome` for each
|
|
@@ -1706,6 +1770,11 @@ function armVerdicts(run, aaPair, options = {}) {
|
|
|
1706
1770
|
* from one family of inputs, so they are not independent draws. The p of a test holds only under the sign test's own null: each
|
|
1707
1771
|
* untied key is a fair coin. A result is about these keys, and this check claims nothing about other inputs.
|
|
1708
1772
|
*/
|
|
1773
|
+
/**
|
|
1774
|
+
* The limit of `SpreadRead.narrow`. This value is a fitted default and not a limit of a gate.
|
|
1775
|
+
* It comes from a gap in the rows of one project, and it has no test on rows that had no part in its choice.
|
|
1776
|
+
*/
|
|
1777
|
+
const NARROW_SPREAD_SHARE = .1;
|
|
1709
1778
|
const DEFAULT_ALPHA = .05;
|
|
1710
1779
|
const DATE = /^\d{4}-\d{2}-\d{2}$/;
|
|
1711
1780
|
function checkedDate(name, date) {
|
|
@@ -1850,6 +1919,23 @@ function controlReads(measure, control, data, rule, real, alpha) {
|
|
|
1850
1919
|
};
|
|
1851
1920
|
});
|
|
1852
1921
|
}
|
|
1922
|
+
/** This function reads how many entries carry the separated pairs of one stratum. */
|
|
1923
|
+
function shareRead(here, real) {
|
|
1924
|
+
const shares = real.flatMap((entry) => {
|
|
1925
|
+
const own = here.filter((c) => !unread(c) && (c.better === entry || c.other === entry));
|
|
1926
|
+
return own.length === 0 ? [] : [{
|
|
1927
|
+
entry,
|
|
1928
|
+
share: own.filter((c) => c.beats).length / own.length
|
|
1929
|
+
}];
|
|
1930
|
+
});
|
|
1931
|
+
const share = shares.length === 0 ? void 0 : median(shares.map((s) => s.share));
|
|
1932
|
+
return {
|
|
1933
|
+
entries: shares.length,
|
|
1934
|
+
share,
|
|
1935
|
+
apart: shares.filter((s) => s.share > .5).map((s) => s.entry),
|
|
1936
|
+
narrow: share !== void 0 && share < .1 && here.some((c) => c.beats)
|
|
1937
|
+
};
|
|
1938
|
+
}
|
|
1853
1939
|
/** The read of the spread of one measure: one family over all pairs of real entries and all strata. */
|
|
1854
1940
|
function spreadReads(data, rule, real, alpha) {
|
|
1855
1941
|
const comparisons = corrected([...data].flatMap(([stratum, groups]) => real.flatMap((entry, i) => real.slice(i + 1).flatMap((other) => rawTest(entry, other, stratum, groups, rule.sense) ?? []))), 2, alpha, rule);
|
|
@@ -1862,7 +1948,8 @@ function spreadReads(data, rule, real, alpha) {
|
|
|
1862
1948
|
pairs: here.length,
|
|
1863
1949
|
separated: here.filter((c) => c.beats).length,
|
|
1864
1950
|
tooFew: here.filter(unread).length,
|
|
1865
|
-
widest
|
|
1951
|
+
widest,
|
|
1952
|
+
...shareRead(here, real)
|
|
1866
1953
|
};
|
|
1867
1954
|
}),
|
|
1868
1955
|
comparisons
|
|
@@ -2455,6 +2542,6 @@ function timeRounds(arms, options) {
|
|
|
2455
2542
|
sink
|
|
2456
2543
|
};
|
|
2457
2544
|
}
|
|
2458
|
-
export { BUDGET_LOOKS, CEILING_ONLY, GATE_LOOKS, LADDER_FORMAT, UNKNOWN_MODEL, aaBand, aaSpread, armModels, armOrder, armVerdicts, balanceOrders, budgetLookSpend, budgetLooksVerdict, budgetVerdict, caseIntervals, caseRatios, caseTails, checkLadder, checkMeasures, checkPairedTime, controlFactor, firstArm, gateOf, jointClaim, ladderVerdict, limitOnModel, limitVerdict, modelRowText, modelTest, notSlowerVerdict, orderSize, poolRecords, pooledBand, pooledLimitVerdict, pooledVerdict, resultRows, resultVerdict, rungVerdict, tailFalseFails, tailLimitVerdict, timeRounds, timeVerdict, verdictMargin, verdictPower, visibleShift, winClaim };
|
|
2545
|
+
export { BUDGET_LOOKS, CEILING_ONLY, GATE_LOOKS, LADDER_FORMAT, NARROW_SPREAD_SHARE, UNKNOWN_MODEL, aaBand, aaSpread, armModels, armOrder, armVerdicts, balanceOrders, budgetLookSpend, budgetLooksVerdict, budgetVerdict, caseIntervals, caseRatios, caseTails, checkLadder, checkMeasures, checkPairedTime, controlFactor, firstArm, gateOf, growthVerdict, jointClaim, ladderVerdict, limitOnModel, limitVerdict, modelRowText, modelTest, notSlowerVerdict, orderSize, poolRecords, pooledBand, pooledLimitVerdict, pooledVerdict, resultRows, resultVerdict, rungVerdict, tailFalseFails, tailLimitVerdict, timeRounds, timeVerdict, verdictMargin, verdictPower, visibleShift, winClaim };
|
|
2459
2546
|
|
|
2460
2547
|
//# sourceMappingURL=index.js.map
|