@mark1russell7/rigor-statistics 0.1.21 → 0.1.23

package/README.md

@@ -1,6 +1,6 @@
 # @mark1russell7/rigor-statistics
 
-The statistics of measured work. It gives quantiles, bootstrap intervals, the sign test, the paired permutation test, Wilson intervals and the Holm correction. It gives the interval of a median from order statistics, and the central interval of a binomial count. It also gives least squares, rank correlation, the comparison with a baseline beyond the noise, and the Brier score and the log-loss of predictions. It rates items from pairwise results with Elo, Glicko-2 and a Bradley-Terry fit. It fits response records that a browser can evaluate from plain JSON.
+The statistics of measured work. It gives quantiles, bootstrap intervals, the sign test, the paired permutation test, Wilson intervals and the Holm correction. It gives the interval of a median from order statistics, and the central interval of a binomial count. It also gives least squares, rank correlation, the comparison with a baseline beyond the noise, and the Brier score and the log-loss of predictions. It rates items from pairwise results with Elo, Glicko-2 and a Bradley-Terry fit. It fits response records that a browser can evaluate from plain JSON, and it gives the Sobol indices of their models.
 
 ```sh
 npm install @mark1russell7/rigor-statistics
@@ -100,6 +100,28 @@ A measure with `null` has no value. To tell why, declare the reasons in the fiel
 
 The default expansion has one product for each pair of levels of two dimensions. Two dimensions with 76 and 33 levels thus give 2508 products, and the limit `RESPONSE_MAX_TERMS` refuses them. With the option `interactions: "seen"`, a ridge fit keeps only the terms that are not 0 at one point or more. This choice looks only at the inputs of the points. A term with no point has the weight 0, so the choice does not change a prediction. The limit applies after the choice.
 
+### The Sobol indices of a model
+
+`sobolIndices(record, measure, options)` tells how much of the variance of a fitted model comes from each dimension. The first-order index `first` is the share of one dimension alone. The total index `total` also includes its interactions.
+
+```ts
+import { sobolIndices } from "@mark1russell7/rigor-statistics";
+
+const sobol = sobolIndices(record, "ms", { n: 4096, seed: 1 });
+sobol.indices;     // [{ dimension: "size", first: 0.71, total: 0.74 }, ...]
+sobol.variance;    // the variance of the prediction over the input distribution
+sobol.evaluations; // n * (dimensions + 2) evaluations of the model
+```
+
+- Each dimension is independent and uniform over its range or its levels. A dimension with `log` is uniform in log space.
+- A conditional dimension gets an independent draw. The model gets `null` for it where `whenHolds` fails.
+- The method is Monte Carlo with the matrices A, B and AB_i of Saltelli (2010). The first-order index uses the estimator of Saltelli (2010), and the total index uses the estimator of Jansen (1999).
+- A constant model has the variance 0. Then each index is 0, not NaN.
+- The option `scale: "model"` gives the indices on the scale of the model, for example the logarithm of a measure with a log scale.
+- The option `interval` adds a percentile bootstrap interval over the N rows to each index.
+
+A test examines models with known indices. At N = 16384, the root mean square error of an index is approximately 0.01.
+
 ## Rate items from pairwise results
 
 A pairwise result is a comparison of two items: the first item wins, the second item wins, or a tie. Each function of this section is pure, so a log of results gives the same ratings each time.
@@ -142,6 +164,7 @@ logLoss([{ p: p.a, hit: true }]);                // the probability of the obser
 | `bootstrap(data, statistic, options)` | A percentile bootstrap interval of any statistic. The options are `runs`, `level` or `ends`, `seed`, `method` and `against`. |
 | The option `against` | The interval also gives `pAbove` and `pBelow`, the bootstrap p of each side of the value, from the same draws. The smallest p is 1 / (`finite` + 1). Give `holm` enough runs: with m tests at level a, `runs` is more than m / a. |
 | `bootstrapMedian`, `bootstrapClusters` | The interval of the median, and a bootstrap that draws whole clusters (boards, seeds, sites). |
+| `bootstrapStrata(strata, statistic, options)` | A stratified bootstrap. Each run draws inside each stratum, so each sample keeps the size of each stratum. |
 | `bootstrapTwo(a, b, statistic, options)` | A bootstrap interval of a statistic of two independent samples, for example two arms of a benchmark. Each run draws all of `a`, then all of `b`. |
 | `bootstrapMedianRatio(numerator, denominator, options)` | The interval of `median(numerator) / median(denominator)`. It is the uncertainty of the ratio, not the spread of the samples. |
 | `sampleTotalInterval(values, { population, scale })` | The interval of the total of a population from a uniform sample of `m` of its `n` items without replacement. The deviations get the finite population correction `sqrt(1 - m / n)`. `scale` is `n / m` by default. When `m` is `n`, the interval is the estimate. For a small `m`, the interval is too narrow: at `m = 8` of `n = 40`, the coverage is 0.865. For a total over pairs, use `uStatisticInterval`. |
@@ -151,6 +174,7 @@ logLoss([{ p: p.a, hit: true }]);                // the probability of the obser
 | `ridgeSolve(gram, moment, lambda, { free })` | The ridge coefficients from the normal equations `X'X` and `X'y`, with a Cholesky factorization. The `free` coefficients have no penalty. A matrix that is not positive definite gives NaN coefficients. |
 | `signTest(better, worse, sides)`, `signTestPower(n, level)` | The sign test, and the fewest wins out of `n` that the test calls significant. |
 | `pairedPermutation(candidate, baseline, options)` | The paired permutation test (random sign flips). It uses the size of each difference. |
+| `signFlipTest(terms, options)` | The sign-flip test of a sum of terms, for example a weighted mean of differences. Up to `exactMax` terms (16), it reads each sign set, so the p is exact. |
 | `twoSamplePermutation(a, b, options)` | The two-sample permutation test (a split of the pooled values). It reads each split when their count is `runs` or less, so the p is exact. It compares groups of different sizes, and its default statistic is the difference of the medians. |
 | `wilson(k, n, z)` | The Wilson interval of a proportion. |
 | `wilsonSampleSize(target, expected, z, max)` | The smallest number of trials at which the Wilson lower end reaches the target, at the expected proportion. |
@@ -179,6 +203,7 @@ logLoss([{ p: p.a, hit: true }]);                // the probability of the obser
 | `parseResponseRecord(value)` | The examination of unknown data. It gives the record, or one fault with a path for each error. |
 | `predictResponse(model, inputs)`, `heldOutError(record, measure)` | The value of a model at one set of inputs, and the error of the folds of a measure. |
 | `heldOutErrorAt(record, measure, ids)` | The held-out error of a chosen set of points, from the predictions of the option `outOfFold`. |
+| `sobolIndices(record, measure, options)` | The first-order and the total Sobol index of each dimension, for the model of a measure. The default N is 4096. |
 | `responseCells(record, vary, steps)` | The number of fitted places of a view, level axes included. It lets a caller bound the time of a view before the view starts. |
 | `whenHolds(dimension, values)` | True when a place has the condition of a dimension. A dimension with no `when` is always present. |
 | `sliceResponse(record, measure, options)`, `surfaceResponse(record, measure, options)` | A fitted curve over one dimension, or a fitted surface over two, with the matching measured points, the matching points with no value, a band and the distance to the nearest measurement. |
@@ -264,6 +289,8 @@ The tests prove that these functions give the same numbers as the copies in the
 | `bootstrapMedian` of packing | `bootstrapMedian(sortedValues, { runs, level, seed })`. Give the values sorted, because packing draws from the sorted list. |
 | `bootstrapMedian` of highways | `bootstrap(values, (s) => quantile(s, 0.5), { runs, level, seed })` |
 | `bootstrap` of layout-contract | `bootstrapClusters(rows, (r) => r.cluster, statistic, { runs, seed, ends: [0.025, 0.975] })` |
+| The stratified bootstrap of `verdict` of layout-contract | `bootstrapStrata(families, meanOfMeans, { runs, seed: deriveSeed(seed, "verdict.stratified"), ends })` |
+| The permutation test of `verdict` of layout-contract | `signFlipTest(terms, { exactMax, runs, seed: deriveSeed(seed, "verdict.permutation") })` |
 | `signTestP` of packing | `signTest(better, worse)` |
 | `signTestPower` of highways | `signTestPower(n, level)` |
 | `wilson` of packing | `wilson(k, n, z)`. packing rounds the ends to 4 places at the call. |

package/build/index.d.ts

@@ -1,3 +1,4 @@
+import "@mark1russell7/rigor-generators";
 /**
  * The descriptive statistics: the mean, the standard deviation, the median, the quantiles and a summary of a list.
  * Each function throws a RangeError for an empty list or for a value that is NaN. Thus a missing value cannot make a
@@ -154,6 +155,14 @@ export declare function bootstrapMedian(values: readonly number[], options?: Boo
  * are in the order of their first row.
  */
 export declare function bootstrapClusters<T>(rows: readonly T[], cluster: (row: T) => string, statistic: (rows: readonly T[]) => number, options?: BootstrapOptions): Interval;
+/**
+ * A stratified bootstrap: each run draws, inside each stratum, as many items as the stratum has, with replacement.
+ * So each sample keeps the size of each stratum. Use it when the strata are fixed by the design, for example the
+ * families of the inputs of a benchmark. The draws of one run go through the strata in the order of the list. With one
+ * stratum, the draws are equal to the draws of `bootstrap` with the same seed. The estimate is the statistic of all
+ * the strata. A run whose statistic is not finite does not count.
+ */
+export declare function bootstrapStrata<T>(strata: readonly (readonly T[])[], statistic: (samples: readonly (readonly T[])[]) => number, options?: BootstrapOptions): Interval;
 interface SampleTotalOptions extends BootstrapOptions {
   /** The number of items in the whole population, `n`. It is a whole number, and it is not less than the sample. */
   readonly population: number;
@@ -757,6 +766,86 @@ export declare const RESPONSE_MAX_TERMS = 1024;
  * or less on a log scale with no `logFloor`. It throws for more than `RESPONSE_MAX_TERMS` terms.
  */
 export declare function fitResponse(data: ResponseData, options?: ResponseFitOptions): ResponseRecord;
+/** The options of `sobolIndices`. */
+interface SobolOptions {
+  /** The number N of rows of each base matrix. The default is 4096. */
+  readonly n?: number;
+  /** The seed of the draws. The default is 1. */
+  readonly seed?: number | string;
+  /**
+   * The scale of the output. The value `measure` uses `predictResponse`. The value `model` uses the scale of the
+   * model, so a model with a log scale gives the indices of the logarithm. The default is `measure`.
+   */
+  readonly scale?: "measure" | "model";
+  /**
+   * With this option, each index also gets a percentile bootstrap interval over the N rows. Each run draws N rows
+   * with replacement, and it calculates all the indices from these rows. The default is no interval.
+   */
+  readonly interval?: BootstrapOptions;
+}
+/** The two Sobol indices of one dimension. */
+interface SobolIndex {
+  /** The id of the dimension. */
+  readonly dimension: string;
+  /** The first-order index S_i: the estimator of Saltelli (2010). */
+  readonly first: number;
+  /** The total index ST_i: the estimator of Jansen (1999). */
+  readonly total: number;
+  /** The bootstrap interval of `first`. Only the option `interval` writes this field. */
+  readonly firstInterval?: Interval;
+  /** The bootstrap interval of `total`. Only the option `interval` writes this field. */
+  readonly totalInterval?: Interval;
+}
+/** The result of `sobolIndices`. */
+interface SobolIndices {
+  /** The id of the measure. */
+  readonly measure: string;
+  /** The scale of the output. */
+  readonly scale: "measure" | "model";
+  /** The number N of rows of each base matrix. */
+  readonly n: number;
+  /** The number of model evaluations: N * (d + 2). */
+  readonly evaluations: number;
+  /** The seed as a 32-bit number. */
+  readonly seed: number;
+  /** The mean of the outputs of A and B. */
+  readonly mean: number;
+  /** The variance of the outputs of A and B, with the divisor 2N. */
+  readonly variance: number;
+  /** The indices of each dimension of the record, in the order of the record. */
+  readonly indices: readonly SobolIndex[];
+}
+/**
+ * This function gives the first-order and the total Sobol index of each dimension of a record, for the model of
+ * `measure`. The indices are of the prediction of the model, not of the measured points.
+ *
+ * The input distribution:
+ * - Each dimension is independent of the other dimensions.
+ * - A continuous dimension is uniform over its range. With `log`, the logarithm of the value is uniform.
+ * - An ordinal or a categorical dimension is uniform over its levels.
+ * - A conditional dimension gets an independent draw. The model gets `null` for it where `whenHolds` fails for the drawn parent.
+ *
+ * Thus the inputs stay independent, and the indices have their usual meaning. A parent gets the effect of the absence of its child.
+ *
+ * The estimators:
+ * - The first-order index is `mean(f(B) * (f(AB_i) - f(A))) / V` (Saltelli, 2010).
+ * - The total index is `mean((f(A) - f(AB_i)) ** 2) / 2 / V` (Jansen, 1999).
+ * - V is the variance of the outputs of A and B together.
+ * - The variance uses the outputs of A and B less their mean. The first-order sum uses `f(B)` less that mean.
+ * - The total sum uses differences of outputs, so it uses no mean.
+ * - This step keeps the estimators unbiased, and it decreases their error when the mean is large.
+ *
+ * A model with a constant output has the variance 0. Then each index is 0, not NaN. An estimate can be a little
+ * below 0 or above 1, from the Monte Carlo error. The function does not clip it.
+ *
+ * The draws fill A row by row, then B row by row. Each row has one draw for each dimension, in the order of the
+ * record. Thus the same seed gives the same indices. The model can ignore a dimension. Then that dimension gets 0 for both
+ * indices exactly, unless it is the parent of a dimension that the model reads.
+ *
+ * The function throws a RangeError when the record has no model of `measure`, and when N is not a whole number of 2 or more.
+ * `predictResponse` throws for a value that the model refuses, for example a level that the model does not know.
+ */
+export declare function sobolIndices(record: Pick<ResponseRecord, "dimensions" | "models">, measure: string, options?: SobolOptions): SobolIndices;
 /**
  * The sign test of paired results: `better` inputs where the candidate wins and `worse` where it loses. The ties
  * do not count. One-sided (the default), it is the p of `better` wins or more if the two are equal. Two-sided, it is
@@ -821,6 +910,40 @@ interface PermutationResult {
  * the size of each difference, which the sign test does not. A pair with a difference of 0 adds nothing.
  */
 export declare function pairedPermutation(candidate: readonly number[], baseline: readonly number[], options?: PermutationOptions): PermutationResult;
+/** The default of the option `exactMax` of `signFlipTest`. */
+export declare const SIGN_FLIP_EXACT_MAX = 16;
+/** The largest value of the option `exactMax` of `signFlipTest`: 2^24 sign sets. */
+export declare const SIGN_FLIP_EXACT_LIMIT = 24;
+/** The options of `signFlipTest`. */
+interface SignFlipOptions {
+  /** The largest count of terms for which the test reads each of the `2^n` sign sets. The default is 16. The most is 24. */
+  readonly exactMax?: number;
+  /** The count of random sign sets above `exactMax`. The default is 10 000. */
+  readonly runs?: number;
+  /** The default is 1. */
+  readonly seed?: number | string;
+}
+/** The result of `signFlipTest`. */
+interface SignFlipResult {
+  /** The absolute value of the sum of the terms. */
+  readonly observed: number;
+  /** The two-sided p. */
+  readonly p: number;
+  /** This field is true when the test read each sign set, so the p has no noise from the draws. */
+  readonly exact: boolean;
+  /** The count of the sign sets that the test read: `2^n` when it is exact, otherwise the count of the draws. */
+  readonly runs: number;
+}
+/**
+ * The sign-flip test of a sum of terms. Under the null, each term has the sign + or - with equal chance. The statistic
+ * is the absolute value of the sum. A weighted mean of paired differences is such a sum: give the terms `w[i] * d[i]`.
+ *
+ * Up to `exactMax` terms, the test reads each sign set. The p is the share of the sets that are as far from 0 or
+ * farther. The observed set is one of them. Above `exactMax`, the test draws `runs` sets, and the p is (1 + the extreme
+ * draws) / (1 + runs). A sum within `1e-12` of the sum of the absolute terms counts as extreme, so rounding does not
+ * hide a tie.
+ */
+export declare function signFlipTest(terms: readonly number[], options?: SignFlipOptions): SignFlipResult;
 /** The options of `twoSamplePermutation`. */
 interface TwoSamplePermutationOptions {
   /** The largest number of splits that the test reads. The default is 10 000. */
@@ -1473,5 +1596,5 @@ interface BradleyTerryFit {
  * outcome, a bad weight and a bad option.
  */
 export declare function bradleyTerry(items: number, comparisons: readonly PairComparison[], options?: BradleyTerryOptions): BradleyTerryFit;
-export type { BinomialSide, BootstrapOptions, BradleyTerryFit, BradleyTerryOptions, CalibrationLevel, CategoricalDimension, CompareOptions, Comparison, ContinuousDimension, Corrected, CountInterval, CoverageBar, CoverageCell, CoverageMinSample, CoverageReason, CoverageStep, Effect, EloOptions, EloUpdate, Forecast, Glicko2Game, Glicko2Options, Glicko2Rating, Interval, Kappa, LeastSquaresFit, LeastSquaresOptions, MedianInterval, ModeOptions, NoiseOptions, NoiseState, NoiseVerdict, OptionFloorOptions, OrdinalDimension, PairComparison, PairOutcome, PairProbabilities, ParsedResponseRecord, PermutationOptions, PermutationResult, Proportion, QuantileInterval, QuantileMethod, RatingScale, ResponseAbsentPoint, ResponseBand, ResponseCurvePoint, ResponseData, ResponseDimension, ResponseDimensionBase, ResponseFactor, ResponseFault, ResponseFitOptions, ResponseFold, ResponseFoldError, ResponseHeldOutError, ResponseInputValue, ResponseMeasure, ResponseModel, ResponseModelBase, ResponsePoint, ResponseReason, ResponseRecord, ResponseSlice, ResponseSliceOptions, ResponseSlicePoint, ResponseSurface, ResponseSurfaceAbsentPoint, ResponseSurfaceOptions, ResponseSurfacePoint, ResponseTableCell, ResponseValue, ResponseViewOptions, ResponseWhen, RidgeInput, RidgeInputBase, RidgeLevelInput, RidgeNumberInput, RidgeOptions, RidgeRankInput, RidgeResponseModel, RidgeTerm, SampleTotalOptions, SideP, SmoothLaw, SmoothLawScore, Summary, TableResponseModel, TieModel, TwoSamplePermutationOptions, TwoSamplePermutationResult, UStatisticInterval, UStatisticMean, UStatisticMethod, UStatisticOptions };
+export type { BinomialSide, BootstrapOptions, BradleyTerryFit, BradleyTerryOptions, CalibrationLevel, CategoricalDimension, CompareOptions, Comparison, ContinuousDimension, Corrected, CountInterval, CoverageBar, CoverageCell, CoverageMinSample, CoverageReason, CoverageStep, Effect, EloOptions, EloUpdate, Forecast, Glicko2Game, Glicko2Options, Glicko2Rating, Interval, Kappa, LeastSquaresFit, LeastSquaresOptions, MedianInterval, ModeOptions, NoiseOptions, NoiseState, NoiseVerdict, OptionFloorOptions, OrdinalDimension, PairComparison, PairOutcome, PairProbabilities, ParsedResponseRecord, PermutationOptions, PermutationResult, Proportion, QuantileInterval, QuantileMethod, RatingScale, ResponseAbsentPoint, ResponseBand, ResponseCurvePoint, ResponseData, ResponseDimension, ResponseDimensionBase, ResponseFactor, ResponseFault, ResponseFitOptions, ResponseFold, ResponseFoldError, ResponseHeldOutError, ResponseInputValue, ResponseMeasure, ResponseModel, ResponseModelBase, ResponsePoint, ResponseReason, ResponseRecord, ResponseSlice, ResponseSliceOptions, ResponseSlicePoint, ResponseSurface, ResponseSurfaceAbsentPoint, ResponseSurfaceOptions, ResponseSurfacePoint, ResponseTableCell, ResponseValue, ResponseViewOptions, ResponseWhen, RidgeInput, RidgeInputBase, RidgeLevelInput, RidgeNumberInput, RidgeOptions, RidgeRankInput, RidgeResponseModel, RidgeTerm, SampleTotalOptions, SideP, SignFlipOptions, SignFlipResult, SmoothLaw, SmoothLawScore, SobolIndex, SobolIndices, SobolOptions, Summary, TableResponseModel, TieModel, TwoSamplePermutationOptions, TwoSamplePermutationResult, UStatisticInterval, UStatisticMean, UStatisticMethod, UStatisticOptions };
 //# sourceMappingURL=index.d.ts.map

package/build/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","names":[],"sources":["../src/describe.ts","../src/regression.ts","../src/bootstrap.ts","../src/distribution.ts","../src/ridge.ts","../src/response.ts","../src/response-parse.ts","../src/response-fit.ts","../src/tests.ts","../src/agreement.ts","../src/rank.ts","../src/discrete.ts","../src/compare.ts","../src/power.ts","../src/coverage.ts","../src/u-statistic.ts","../src/rating.ts","../src/bradley-terry.ts"],"mappings":";;;;;;KAOY;;wBAmBI,OAAO;wBAIP,KAAK;;;;;wBAgBL,GAAG;;;;;;;wBAgBH,eAAe,iCAAiC,WAAW,SAAQ;;wBAkBnE,SAAS,2BAA2B,WAAW,SAAQ;;;;;wBASvD,OAAO;;UAQN;WACN;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;;wBAGK,UAAU,4BAA4B;;;;;;UC3GrC;;WAEN;;UAGM;;WAEN;;WAEA;;;;;WAKA;;WAEA;;WAEA;;;;;;;;;;;wBAkBK,aAAa,mCAAmC,sBAAsB,UAAS,sBAA2B;UC7BzG;;WAEN;;WAEA;;WAEA;;WAEA,SAAS;;;;;;WAMT;;;;;WAKA;;;UAIM;;WAEN;;WAEA;;WAEA;;UAGM,iBAAiB,QAAQ;;WAE/B;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;;;;;wBAOK,UAAU,GAAG,eAAe,KAAK,YAAY,iBAAiB,gBAAgB,UAAS,mBAAwB;;;;;;;;wBAsB/G,aAAa,GAAG,GAC9B,YAAY,KACZ,YAAY,KACZ,YAAY,YAAY,KAAK,YAAY,gBACzC,UAAS,mBACR;;;;;;wBAwBa,qBAAqB,8BAA8B,gCAAgC,UAAS,mBAAwB;;;;;;;wBAUpH,sBACd,mCACA,sBACA,UAAS,mBAAmB,sBAC3B;;wBAoGa,gBAAgB,2BAA2B,UAAS,mBAAwB;;;;;;wBAS5E,kBAAkB,GAChC,eAAe,KACf,UAAU,KAAK,cACf,YAAY,eAAe,gBAC3B,UAAS,mBACR;UAec,2BAA2B;;WAEjC;;WAEA;;;;;;;;;;;;;;;;wBAiBK,oBAAoB,2BAA2B,SAAS,qBAAqB;;KC9QjF;;UAGK;WACN,KAAK;;WAEL;;;UAIM;;;;;WAKN;;WAEA;;;;;;;;;;;;;;;;;;;wBA0MK,UAAU,2BAA2B,UAAS;;;;;;;;wBAqB9C,gBAAgB,qCAAqC;;;;;;wBAwBrD,cAAc,4BAA4B;;;;;;;UCxQzC;;WAEN;;;;;;;;;;;;wBAoBK,WAAW,MAAM,mBAAmB,QAAQ,mBAAmB,gBAAgB,UAAS;;qBCd3F;;KAGD;;KAGA,qBAAqB;;UAGhB;;WAEN;WACA,iBAAiB;;;UAIX;;WAEN;;WAEA;;WAEA;;WAEA,OAAO;;;UAID,4BAA4B;WAClC;;WAEA;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;;;UAIM,6BAA6B;WACnC;WACA;;;KAIC,oBAAoB,sBAAsB,mBAAmB;;UAGxD;;WAEN;WACA;;WAEA;;WAEA;;;;;WAKA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA,QAAQ,SAAS,eAAe;;WAEhC,UAAU,SAAS;;WAEnB;;WAEA,SAAS,SAAS;;;KAIjB;WAA4B;WAA4B;;WAA6B;WAA4B,OAAO;;;UAGnH;WACN;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;WACA;WACA;;;UAIM,uBAAuB;WAC7B;WACA;WACA;WACA;;;UAIM,wBAAwB;WAC9B;WACA;;;KAIC,aAAa,mBAAmB,iBAAiB;;UAG5C;WACN,kBAAkB;WAClB;;;UAIM;;WAEN;;WAEA;;WAEA;;;UAIM,2BAA2B;WACjC;;WAEA;WACA,iBAAiB;WACjB;WACA,gBAAgB;;;UAIV;;WAEN,aAAa;;WAEb;;WAEA;;;UAIM,2BAA2B;WACjC;WACA;;WAEA;WACA,gBAAgB;;WAEhB;;;KAIC,gBAAgB,qBAAqB;;UAGhC;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA,QAAQ,SAAS,eAAe;;;;;WAKhC,cAAc,SAAS,eAAe,SAAS;;;UAIzC;WACN,eAAe;WACf;WACA,qBAAqB;WACrB,mBAAmB;;WAEnB,mBAAmB;WACnB,iBAAiB;;;UAIX,uBAAuB;WAC7B,iBAAiB;WACjB,gBAAgB;;;UAIV;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN,OAAO,SAAS,eAAe;;WAE/B;;;;;WAKA;;WAEA;;;UAIM,6BAA6B;;WAEnC;;;UAIM,+BAA+B;;WAErC;;;;;;UAOM;WACN;WACA;WACA;;;;;;;WAOA;;WAEA;;;UAIM,2BAA2B;;WAEjC,IAAI;;;UAIE;WACN;;WAEA,IAAI;WACJ;WACA;;;UAIM;WACN;;WAEA,IAAI;;WAEJ;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,gBAAgB;WAChB,iBAAiB;;WAEjB,iBAAiB;WACjB,OAAO;;WAEP;;;UAIM;WACN;;WAEA,cAAc,eAAe;WAC7B;WACA;;;UAIM;WACN;;WAEA,cAAc,eAAe;;WAE7B;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,YAAY;WACZ,YAAY;;WAEZ,0BAA0B;WAC1B,iBAAiB;;WAEjB,iBAAiB;WACjB,OAAO;;WAEP;;;wBAuBK,UAAU,WAAW,mBAAmB,QAAQ,SAAS;;;;;;;;;;;wBA4FzD,gBAAgB,OAAO,eAAe,QAAQ,SAAS,eAAe;;;;;wBAQtE,aAAa,QAAQ,KAAK,0BAA0B,kBAAkB;;;;;;;;;;;;wBAqCtE,eAAe,QAAQ,gBAAgB,iBAAiB,yBAAyB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAgPjF,cAAc,QAAQ,gBAAgB,iBAAiB,SAAS,uBAAuB;;;;;;;wBAsBvF,gBAAgB,QAAQ,gBAAgB,iBAAiB,SAAS,yBAAyB;;;;;;;;wBA4B3F,cAAc,QAAQ,KAAK,+BAA+B,0CAA0C;;qBC3yBvG;;UAGI;;WAEN;;WAEA;;;KAIC;WAAkC;WAAmB,QAAQ;;WAA8B;WAAoB,iBAAiB;;;;;;;;;;;;;;;;;;;;;;;;;;wBAgf5H,oBAAoB,iBAAiB;;UC/dpC;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;;;;WASA;;WAEA,2BAA2B;;WAE3B;;WAEA;;WAEA;;;;;WAKA;;;qBAIE;;qBAGA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAoTG,YAAY,MAAM,cAAc,UAAS,qBAA0B;;;;;;wBCvVnE,SAAS,gBAAgB,eAAe;;;;;wBAaxC,cAAc,WAAW,gBAAc;;qBAO1C;;UAGI;WACN;WACA;;;;;;wBAOK,OAAO,WAAW,WAAW,aAAkB;;UAc9C;;WAEN;;WAEA;;WAEA;;;;;;;wBAQK,KAAK,4BAA4B,iBAAe;;;;;;wBAkBhD,mBAAmB,4BAA4B;UAS9C;;WAEN;;WAEA;;WAEA,aAAa;;UAGP;;WAEN;;WAEA;WACA;;;;;;wBAOK,kBAAkB,8BAA8B,6BAA6B,UAAS,qBAA0B;;UAmB/G;;WAEN;;WAEA;;WAEA,aAAa,sBAAsB;;;UAI7B,mCAAmC;;WAEzC;;;;;;;;;wBAuCK,qBAAqB,sBAAsB,sBAAsB,UAAS,8BAAmC;;;;;;wBA+B7G,iBAAiB,gBAAgB,kBAAkB,YAAiB;;UCrPnE;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;wBAOK,WAAW,GAAG,YAAY,KAAK,YAAY,MAAM;;wBCHjD,MAAM;;;;;wBAgBN,SAAS,sBAAsB;;qBAmBlC;;;;;;wBAQG,WAAW,sBAAsB,sBAAsB;;UC1BtD;WACN;WACA;;WAEA;;WAEA;;;;;;;;;;wBAWK,eAAe,2BAA2B,iBAAe;;UAqBxD;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;;;wBAUK,iBAAiB,2BAA2B,WAAW,iBAAe;;;;;;wBAsCtE,gBAAgB,WAAW;;UAW1B;WACN;WACA;;WAEA;;;;;;;wBAQK,iBAAiB,WAAW,WAAW,iBAAe;;KAuB1D;;;;;;;;;;;;;;;wBAgBI,aAAa,WAAW,WAAW,WAAW,MAAM;;;;;;wBA2BpD,yBAAyB,mBAAmB,WAAW,iBAAe;UC9MrE;;WAEN;;WAEA;;;;;;WAMA;;;;;;wBAgBK,WAAW,2BAA2B,UAAS;;;;;KASnD;;UAGK;;WAEN;WACA;;WAEA;;WAEA;WACA;;WAEA;WACA,OAAO;;;;;;;;wBASF,aAAa,2BAA2B,UAAS;WAA0B;IAAqC;UAW/G;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;UAGM;;WAEN;WACA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;;WAEA;;WAEA;WACA,cAAc;;WAEd;WACA,cAAc;;;;;;;;wBAST,cAAc,2BAA2B,UAAS,iBAAsB;;UA6CvE;WACN;WACA;;;wBASK,MAAM,oBAAoB;;;;;;wBAY1B,QAAQ,oBAAoB;;UAQ3B;WACN;WACA;WACA;;WAEA;;WAEA,UAAU;;;wBAIL,YAAY,oBAAoB,aAAa;;;;;;;wBC5J7C,UAAU,WAAW;;UAgBpB;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;;;;;;;;;;wBAiBK,YAAY,SAAS;;UC7FpB;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;WAEA;;WAEA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;WACA,QAAQ;;;UAIF;;;;;WAKN;;WAEA;;WAEA,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;;wBA6BX,kBAAkB,gBAAgB,gBAAgB,KAAK,cAAc;;KCvFzE;;;;;;;qBAQC;;UAGI;;WAEN;;WAEA;;WAEA,QAAQ;;WAER;;WAEA,kBAAkB,aAAa;;;UAIzB;;WAEN;WACA;WACA;;;UAIM;;WAEN;WACA;WACA;WACA;;WAEA;;WAEA,WAAW;;WAEX,MAAM;;WAEN;;WAEA;;WAEA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAmFK,mBAAmB,2BAA2B,SAAS,oBAAoB;;;;;;;UCrI1E;;WAEN;;WAEA;;WAEA;;;UAIM,mBAAmB;;WAEzB;;;UAIM;;WAEN;;WAEA;;WAEA;;;;;;wBAyBK,YAAY,YAAY,YAAY,UAAS;;;;;;wBAY7C,UAAU,YAAY,YAAY,eAAe,UAAS,aAAkB;;UAU3E;WACN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;;UAIM;;WAEN;;WAEA;;;;;;;wBAmBK,cAAc,QAAQ,eAAe,gBAAgB,eAAe,UAAS,iBAAsB;;;;;wBAuEnG,YAAY,QAAQ,gBAAgB;;UAQnC;;WAEN;;WAEA;;WAEA;;;;;;;wBAuBK,kBAAkB,YAAY,YAAY,eAAU;;;;;wBAWpD,iBAAiB,kBAAkB,UAAS;;wBAO5C,iBAAiB,gBAAgB,UAAS;;KCtO9C;;UAGK;WACN;WACA;WACA,SAAS;;WAET;;;KAIC;;UAGK;;WAEN;;WAEA;;;;;WAKA,OAAO;;;;;WAKP;;WAEA;;WAEA;;;UAIM;;WAEN;;;;;WAKA;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;wBA6CK,aAAa,eAAe,sBAAsB,kBAAkB,UAAS,sBAA2B"}
+{"version":3,"file":"index.d.ts","names":[],"sources":["../src/describe.ts","../src/regression.ts","../src/bootstrap.ts","../src/distribution.ts","../src/ridge.ts","../src/response.ts","../src/response-parse.ts","../src/response-fit.ts","../src/response-sobol.ts","../src/tests.ts","../src/agreement.ts","../src/rank.ts","../src/discrete.ts","../src/compare.ts","../src/power.ts","../src/coverage.ts","../src/u-statistic.ts","../src/rating.ts","../src/bradley-terry.ts"],"mappings":";;;;;;;KAOY;;wBAmBI,OAAO;wBAIP,KAAK;;;;;wBAgBL,GAAG;;;;;;;wBAgBH,eAAe,iCAAiC,WAAW,SAAQ;;wBAkBnE,SAAS,2BAA2B,WAAW,SAAQ;;;;;wBASvD,OAAO;;UAQN;WACN;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;;wBAGK,UAAU,4BAA4B;;;;;;UC3GrC;;WAEN;;UAGM;;WAEN;;WAEA;;;;;WAKA;;WAEA;;WAEA;;;;;;;;;;;wBAkBK,aAAa,mCAAmC,sBAAsB,UAAS,sBAA2B;UC7BzG;;WAEN;;WAEA;;WAEA;;WAEA,SAAS;;;;;;WAMT;;;;;WAKA;;;UAIM;;WAEN;;WAEA;;WAEA;;UAGM,iBAAiB,QAAQ;;WAE/B;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;;;;;wBAOK,UAAU,GAAG,eAAe,KAAK,YAAY,iBAAiB,gBAAgB,UAAS,mBAAwB;;;;;;;;wBAsB/G,aAAa,GAAG,GAC9B,YAAY,KACZ,YAAY,KACZ,YAAY,YAAY,KAAK,YAAY,gBACzC,UAAS,mBACR;;;;;;wBAwBa,qBAAqB,8BAA8B,gCAAgC,UAAS,mBAAwB;;;;;;;wBAUpH,sBACd,mCACA,sBACA,UAAS,mBAAmB,sBAC3B;;wBAoGa,gBAAgB,2BAA2B,UAAS,mBAAwB;;;;;;wBAS5E,kBAAkB,GAChC,eAAe,KACf,UAAU,KAAK,cACf,YAAY,eAAe,gBAC3B,UAAS,mBACR;;;;;;;;wBAsBa,gBAAgB,GAC9B,2BAA2B,QAC3B,YAAY,4BAA4B,mBACxC,UAAS,mBACR;UAmBc,2BAA2B;;WAEjC;;WAEA;;;;;;;;;;;;;;;;wBAiBK,oBAAoB,2BAA2B,SAAS,qBAAqB;;KC5SjF;;UAGK;WACN,KAAK;;WAEL;;;UAIM;;;;;WAKN;;WAEA;;;;;;;;;;;;;;;;;;;wBA0MK,UAAU,2BAA2B,UAAS;;;;;;;;wBAqB9C,gBAAgB,qCAAqC;;;;;;wBAwBrD,cAAc,4BAA4B;;;;;;;UCxQzC;;WAEN;;;;;;;;;;;;wBAoBK,WAAW,MAAM,mBAAmB,QAAQ,mBAAmB,gBAAgB,UAAS;;qBCd3F;;KAGD;;KAGA,qBAAqB;;UAGhB;;WAEN;WACA,iBAAiB;;;UAIX;;WAEN;;WAEA;;WAEA;;WAEA,OAAO;;;UAID,4BAA4B;WAClC;;WAEA;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;;;UAIM,6BAA6B;WACnC;WACA;;;KAIC,oBAAoB,sBAAsB,mBAAmB;;UAGxD;;WAEN;WACA;;WAEA;;WAEA;;;;;WAKA;;;UAIM;;WAEN;;WAEA;;;UAIM;WACN;;WAEA,QAAQ,SAAS,eAAe;;WAEhC,UAAU,SAAS;;WAEnB;;WAEA,SAAS,SAAS;;;KAIjB;WAA4B;WAA4B;;WAA6B;WAA4B,OAAO;;;UAGnH;WACN;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;WACA;WACA;;;UAIM,uBAAuB;WAC7B;WACA;WACA;WACA;;;UAIM,wBAAwB;WAC9B;WACA;;;KAIC,aAAa,mBAAmB,iBAAiB;;UAG5C;WACN,kBAAkB;WAClB;;;UAIM;;WAEN;;WAEA;;WAEA;;;UAIM,2BAA2B;WACjC;;WAEA;WACA,iBAAiB;WACjB;WACA,gBAAgB;;;UAIV;;WAEN,aAAa;;WAEb;;WAEA;;;UAIM,2BAA2B;WACjC;WACA;;WAEA;WACA,gBAAgB;;WAEhB;;;KAIC,gBAAgB,qBAAqB;;UAGhC;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA,QAAQ,SAAS,eAAe;;;;;WAKhC,cAAc,SAAS,eAAe,SAAS;;;UAIzC;WACN,eAAe;WACf;WACA,qBAAqB;WACrB,mBAAmB;;WAEnB,mBAAmB;WACnB,iBAAiB;;;UAIX,uBAAuB;WAC7B,iBAAiB;WACjB,gBAAgB;;;UAIV;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN,OAAO,SAAS,eAAe;;WAE/B;;;;;WAKA;;WAEA;;;UAIM,6BAA6B;;WAEnC;;;UAIM,+BAA+B;;WAErC;;;;;;UAOM;WACN;WACA;WACA;;;;;;;WAOA;;WAEA;;;UAIM,2BAA2B;;WAEjC,IAAI;;;UAIE;WACN;;WAEA,IAAI;WACJ;WACA;;;UAIM;WACN;;WAEA,IAAI;;WAEJ;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,gBAAgB;WAChB,iBAAiB;;WAEjB,iBAAiB;WACjB,OAAO;;WAEP;;;UAIM;WACN;;WAEA,cAAc,eAAe;WAC7B;WACA;;;UAIM;WACN;;WAEA,cAAc,eAAe;;WAE7B;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,YAAY;WACZ,YAAY;;WAEZ,0BAA0B;WAC1B,iBAAiB;;WAEjB,iBAAiB;WACjB,OAAO;;WAEP;;;wBAuBK,UAAU,WAAW,mBAAmB,QAAQ,SAAS;;;;;;;;;;;wBA4FzD,gBAAgB,OAAO,eAAe,QAAQ,SAAS,eAAe;;;;;wBAQtE,aAAa,QAAQ,KAAK,0BAA0B,kBAAkB;;;;;;;;;;;;wBAqCtE,eAAe,QAAQ,gBAAgB,iBAAiB,yBAAyB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAgPjF,cAAc,QAAQ,gBAAgB,iBAAiB,SAAS,uBAAuB;;;;;;;wBAsBvF,gBAAgB,QAAQ,gBAAgB,iBAAiB,SAAS,yBAAyB;;;;;;;;wBA4B3F,cAAc,QAAQ,KAAK,+BAA+B,0CAA0C;;qBC3yBvG;;UAGI;;WAEN;;WAEA;;;KAIC;WAAkC;WAAmB,QAAQ;;WAA8B;WAAoB,iBAAiB;;;;;;;;;;;;;;;;;;;;;;;;;;wBAgf5H,oBAAoB,iBAAiB;;UC/dpC;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;;;;WASA;;WAEA,2BAA2B;;WAE3B;;WAEA;;WAEA;;;;;WAKA;;;qBAIE;;qBAGA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAoTG,YAAY,MAAM,cAAc,UAAS,qBAA0B;;UC5WlE;;WAEN;;WAEA;;;;;WAKA;;;;;WAKA,WAAW;;;UAIL;;WAEN;;WAEA;;WAEA;;WAEA,gBAAgB;;WAEhB,gBAAgB;;;UAIV;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAsGb,aAAa,QAAQ,KAAK,0CAA0C,iBAAiB,UAAS,eAAoB;;;;;;wBCjIlH,SAAS,gBAAgB,eAAe;;;;;wBAaxC,cAAc,WAAW,gBAAc;;qBAO1C;;UAGI;WACN;WACA;;;;;;wBAOK,OAAO,WAAW,WAAW,aAAkB;;UAc9C;;WAEN;;WAEA;;WAEA;;;;;;;wBAQK,KAAK,4BAA4B,iBAAe;;;;;;wBAkBhD,mBAAmB,4BAA4B;UAS9C;;WAEN;;WAEA;;WAEA,aAAa;;UAGP;;WAEN;;WAEA;WACA;;;;;;wBAOK,kBAAkB,8BAA8B,6BAA6B,UAAS,qBAA0B;;qBAmBnH;;qBAGA;;UAGI;;WAEN;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;;;;;;wBAYK,aAAa,0BAA0B,UAAS,kBAAuB;;UAgCtE;;WAEN;;WAEA;;WAEA,aAAa,sBAAsB;;;UAI7B,mCAAmC;;WAEzC;;;;;;;;;wBAuCK,qBAAqB,sBAAsB,sBAAsB,UAAS,8BAAmC;;;;;;wBA+B7G,iBAAiB,gBAAgB,kBAAkB,YAAiB;;UCzTnE;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;wBAOK,WAAW,GAAG,YAAY,KAAK,YAAY,MAAM;;wBCHjD,MAAM;;;;;wBAgBN,SAAS,sBAAsB;;qBAmBlC;;;;;;wBAQG,WAAW,sBAAsB,sBAAsB;;UC1BtD;WACN;WACA;;WAEA;;WAEA;;;;;;;;;;wBAWK,eAAe,2BAA2B,iBAAe;;UAqBxD;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;;;wBAUK,iBAAiB,2BAA2B,WAAW,iBAAe;;;;;;wBAsCtE,gBAAgB,WAAW;;UAW1B;WACN;WACA;;WAEA;;;;;;;wBAQK,iBAAiB,WAAW,WAAW,iBAAe;;KAuB1D;;;;;;;;;;;;;;;wBAgBI,aAAa,WAAW,WAAW,WAAW,MAAM;;;;;;wBA2BpD,yBAAyB,mBAAmB,WAAW,iBAAe;UC9MrE;;WAEN;;WAEA;;;;;;WAMA;;;;;;wBAgBK,WAAW,2BAA2B,UAAS;;;;;KASnD;;UAGK;;WAEN;WACA;;WAEA;;WAEA;WACA;;WAEA;WACA,OAAO;;;;;;;;wBASF,aAAa,2BAA2B,UAAS;WAA0B;IAAqC;UAW/G;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;UAGM;;WAEN;WACA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;;WAEA;;WAEA;WACA,cAAc;;WAEd;WACA,cAAc;;;;;;;;wBAST,cAAc,2BAA2B,UAAS,iBAAsB;;UA6CvE;WACN;WACA;;;wBASK,MAAM,oBAAoB;;;;;;wBAY1B,QAAQ,oBAAoB;;UAQ3B;WACN;WACA;WACA;;WAEA;;WAEA,UAAU;;;wBAIL,YAAY,oBAAoB,aAAa;;;;;;;wBC5J7C,UAAU,WAAW;;UAgBpB;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;;;;;;;;;;wBAiBK,YAAY,SAAS;;UC7FpB;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA;;WAEA;;WAEA;;;KAIC;;UAGK;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;WACA,QAAQ;;;UAIF;;;;;WAKN;;WAEA;;WAEA,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;;wBA6BX,kBAAkB,gBAAgB,gBAAgB,KAAK,cAAc;;KCvFzE;;;;;;;qBAQC;;UAGI;;WAEN;;WAEA;;WAEA,QAAQ;;WAER;;WAEA,kBAAkB,aAAa;;;UAIzB;;WAEN;WACA;WACA;;;UAIM;;WAEN;WACA;WACA;WACA;;WAEA;;WAEA,WAAW;;WAEX,MAAM;;WAEN;;WAEA;;WAEA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAmFK,mBAAmB,2BAA2B,SAAS,oBAAoB;;;;;;;UCrI1E;;WAEN;;WAEA;;WAEA;;;UAIM,mBAAmB;;WAEzB;;;UAIM;;WAEN;;WAEA;;WAEA;;;;;;wBAyBK,YAAY,YAAY,YAAY,UAAS;;;;;;wBAY7C,UAAU,YAAY,YAAY,eAAe,UAAS,aAAkB;;UAU3E;WACN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;;UAIM;;WAEN;;WAEA;;;;;;;wBAmBK,cAAc,QAAQ,eAAe,gBAAgB,eAAe,UAAS,iBAAsB;;;;;wBAuEnG,YAAY,QAAQ,gBAAgB;;UAQnC;;WAEN;;WAEA;;WAEA;;;;;;;wBAuBK,kBAAkB,YAAY,YAAY,eAAU;;;;;wBAWpD,iBAAiB,kBAAkB,UAAS;;wBAO5C,iBAAiB,gBAAgB,UAAS;;KCtO9C;;UAGK;WACN;WACA;WACA,SAAS;;WAET;;;KAIC;;UAGK;;WAEN;;WAEA;;;;;WAKA,OAAO;;;;;WAKP;;WAEA;;WAEA;;;UAIM;;WAEN;;;;;WAKA;;WAEA;;WAEA;;WAEA;;WAEA;;;;;;;wBA6CK,aAAa,eAAe,sBAAsB,kBAAkB,UAAS,sBAA2B"}

package/build/index.js

@@ -388,6 +388,26 @@ function bootstrapClusters(rows, cluster, statistic, options = {}) {
 	};
 }
 /**
+* A stratified bootstrap: each run draws, inside each stratum, as many items as the stratum has, with replacement.
+* So each sample keeps the size of each stratum. Use it when the strata are fixed by the design, for example the
+* families of the inputs of a benchmark. The draws of one run go through the strata in the order of the list. With one
+* stratum, the draws are equal to the draws of `bootstrap` with the same seed. The estimate is the statistic of all
+* the strata. A run whose statistic is not finite does not count.
+*/
+function bootstrapStrata(strata, statistic, options = {}) {
+	if (strata.length === 0) throw new RangeError("bootstrapStrata: there is no stratum.");
+	for (const [i, stratum] of strata.entries()) if (stratum.length === 0) throw new RangeError(`bootstrapStrata: the stratum ${i} is empty.`);
+	const samples = strata.map((stratum) => new Array(stratum.length));
+	return percentile("bootstrapStrata", options, (r) => {
+		for (const [k, stratum] of strata.entries()) {
+			const sample = samples[k];
+			const n = stratum.length;
+			for (let i = 0; i < n; i++) sample[i] = stratum[r.below(n)];
+		}
+		return statistic(samples);
+	}, () => statistic(strata));
+}
+/**
 * A bootstrap interval of the total of a population from a uniform sample of `m` of its `n` items, drawn without
 * replacement. The estimate is `scale * sum(values)`. Each run draws `m` values with replacement, as `bootstrap`
 * does. The deviation of each run from the sum of the sample gets the factor `scale * sqrt(1 - m / n)`, the finite
@@ -2242,6 +2262,168 @@ function fitResponse(data, options = {}) {
 		folds: foldRecords
 	};
 }
+// src/response-sobol.ts
+/**
+* The Sobol indices of a fitted model of a record of the format `response-records/1`. The first-order index of a
+* dimension is the share of the variance of the prediction that the dimension causes alone. The total index also
+* includes each interaction of the dimension with the other dimensions.
+*
+* The method is Monte Carlo with two independent matrices A and B of N rows, and one matrix `AB_i` for each dimension `i`.
+* `AB_i` is A with the column `i` of B (Saltelli and others, 2010). The first-order index uses the estimator of Saltelli
+* (2010), and the total index uses the estimator of Jansen (1999). The model gets N * (d + 2) evaluations for d dimensions.
+*
+* This file uses no Node import, no clock and no `Math.random`, the same as `response.ts`.
+*/
+/** This function draws one value of a dimension. Each dimension uses the same number of draws: one. */
+function drawValue(dimension, r) {
+	if (dimension.kind !== "continuous") return dimension.levels[r.below(dimension.levels.length)];
+	const [low, high] = dimension.range;
+	if (dimension.log !== true) return r.range(low, high);
+	const value = 10 ** r.range(Math.log10(low), Math.log10(high));
+	return Math.min(high, Math.max(low, value));
+}
+/** This function draws a matrix of `n` rows, row by row, with one draw for each dimension in the order of the record. */
+function drawMatrix(dimensions, n, r) {
+	return Array.from({ length: n }, () => dimensions.map((d) => drawValue(d, r)));
+}
+/** This function gives the inputs of a row. A conditional dimension gets `null` where its condition fails. */
+function inputsOf(dimensions, row) {
+	const drawn = {};
+	dimensions.forEach((d, i) => {
+		drawn[d.id] = row[i];
+	});
+	const inputs = {};
+	for (const d of dimensions) inputs[d.id] = whenHolds(d, drawn) ? drawn[d.id] : null;
+	return inputs;
+}
+/**
+* This function gives the variance and the indices from the rows `rows`. A variance of 0 gives each index as 0.
+* The value `constant` is true when each output is equal to the first output.
+*/
+function estimate(outputs, rows, n, constant) {
+	const d = outputs.ab.length;
+	const at = (j) => rows === null ? j : rows[j];
+	let sum = 0;
+	for (let j = 0; j < n; j++) sum += outputs.a[at(j)] + outputs.b[at(j)];
+	const mean = sum / (2 * n);
+	let square = 0;
+	for (let j = 0; j < n; j++) {
+		const ea = outputs.a[at(j)] - mean;
+		const eb = outputs.b[at(j)] - mean;
+		square += ea * ea + eb * eb;
+	}
+	const variance = constant ? 0 : square / (2 * n);
+	const first = [];
+	const total = [];
+	for (let i = 0; i < d; i++) {
+		const abi = outputs.ab[i];
+		let s = 0;
+		let t = 0;
+		for (let j = 0; j < n; j++) {
+			const fa = outputs.a[at(j)];
+			const fb = outputs.b[at(j)];
+			const fab = abi[at(j)];
+			s += (fb - mean) * (fab - fa);
+			t += (fa - fab) * (fa - fab);
+		}
+		first.push(variance > 0 ? s / n / variance : 0);
+		total.push(variance > 0 ? t / (2 * n) / variance : 0);
+	}
+	return {
+		mean,
+		variance,
+		first,
+		total
+	};
+}
+/**
+* This function gives the first-order and the total Sobol index of each dimension of a record, for the model of
+* `measure`. The indices are of the prediction of the model, not of the measured points.
+*
+* The input distribution:
+* - Each dimension is independent of the other dimensions.
+* - A continuous dimension is uniform over its range. With `log`, the logarithm of the value is uniform.
+* - An ordinal or a categorical dimension is uniform over its levels.
+* - A conditional dimension gets an independent draw. The model gets `null` for it where `whenHolds` fails for the drawn parent.
+*
+* Thus the inputs stay independent, and the indices have their usual meaning. A parent gets the effect of the absence of its child.
+*
+* The estimators:
+* - The first-order index is `mean(f(B) * (f(AB_i) - f(A))) / V` (Saltelli, 2010).
+* - The total index is `mean((f(A) - f(AB_i)) ** 2) / 2 / V` (Jansen, 1999).
+* - V is the variance of the outputs of A and B together.
+* - The variance uses the outputs of A and B less their mean. The first-order sum uses `f(B)` less that mean.
+* - The total sum uses differences of outputs, so it uses no mean.
+* - This step keeps the estimators unbiased, and it decreases their error when the mean is large.
+*
+* A model with a constant output has the variance 0. Then each index is 0, not NaN. An estimate can be a little
+* below 0 or above 1, from the Monte Carlo error. The function does not clip it.
+*
+* The draws fill A row by row, then B row by row. Each row has one draw for each dimension, in the order of the
+* record. Thus the same seed gives the same indices. The model can ignore a dimension. Then that dimension gets 0 for both
+* indices exactly, unless it is the parent of a dimension that the model reads.
+*
+* The function throws a RangeError when the record has no model of `measure`, and when N is not a whole number of 2 or more.
+* `predictResponse` throws for a value that the model refuses, for example a level that the model does not know.
+*/
+function sobolIndices(record, measure, options = {}) {
+	const model = record.models.find((m) => m.measure === measure);
+	if (model === void 0) throw new RangeError(`sobolIndices: the record has no model of the measure "${measure}".`);
+	const n = options.n ?? 4096;
+	if (!Number.isSafeInteger(n) || n < 2) throw new RangeError(`sobolIndices: n is a whole number of 2 or more, not ${n}.`);
+	const scale = options.scale ?? "measure";
+	const dimensions = record.dimensions;
+	const d = dimensions.length;
+	const r = random(options.seed ?? 1);
+	const a = drawMatrix(dimensions, n, r);
+	const b = drawMatrix(dimensions, n, r);
+	const f = (row) => {
+		const inputs = inputsOf(dimensions, row);
+		return scale === "model" ? predictScaled(model, inputs) : predictResponse(model, inputs);
+	};
+	const fa = Float64Array.from(a, f);
+	const fb = Float64Array.from(b, f);
+	const outputs = {
+		a: fa,
+		b: fb,
+		ab: dimensions.map((_, i) => Float64Array.from(a, (row, j) => f(row.map((v, k) => k === i ? b[j][i] : v))))
+	};
+	const first = fa[0];
+	const constant = fa.every((v) => v === first) && fb.every((v) => v === first);
+	const all = estimate(outputs, null, n, constant);
+	let intervals;
+	if (options.interval !== void 0) {
+		const rows = new Array(n);
+		const flat = (e) => [...e.first, ...e.total];
+		intervals = percentiles("sobolIndices", options.interval, 2 * d, (rr) => {
+			for (let j = 0; j < n; j++) rows[j] = rr.below(n);
+			return flat(estimate(outputs, rows, n, constant));
+		}, () => flat(all));
+	}
+	const indices = dimensions.map((dimension, i) => {
+		const index = {
+			dimension: dimension.id,
+			first: all.first[i],
+			total: all.total[i]
+		};
+		if (intervals === void 0) return index;
+		return {
+			...index,
+			firstInterval: intervals[i],
+			totalInterval: intervals[d + i]
+		};
+	});
+	return {
+		measure,
+		scale,
+		n,
+		evaluations: n * (d + 2),
+		seed: r.seed,
+		mean: constant ? first : all.mean,
+		variance: all.variance,
+		indices
+	};
+}
 // src/tests.ts
 /**
 * The tests of a difference: the sign test, the paired permutation test, the Wilson interval of a proportion and
@@ -2375,6 +2557,57 @@ function pairedPermutation(candidate, baseline, options = {}) {
 		runs
 	};
 }
+/** The default of the option `exactMax` of `signFlipTest`. */
+const SIGN_FLIP_EXACT_MAX = 16;
+/** The largest value of the option `exactMax` of `signFlipTest`: 2^24 sign sets. */
+const SIGN_FLIP_EXACT_LIMIT = 24;
+/**
+* The sign-flip test of a sum of terms. Under the null, each term has the sign + or - with equal chance. The statistic
+* is the absolute value of the sum. A weighted mean of paired differences is such a sum: give the terms `w[i] * d[i]`.
+*
+* Up to `exactMax` terms, the test reads each sign set. The p is the share of the sets that are as far from 0 or
+* farther. The observed set is one of them. Above `exactMax`, the test draws `runs` sets, and the p is (1 + the extreme
+* draws) / (1 + runs). A sum within `1e-12` of the sum of the absolute terms counts as extreme, so rounding does not
+* hide a tie.
+*/
+function signFlipTest(terms, options = {}) {
+	const exactMax = options.exactMax ?? 16;
+	if (!Number.isInteger(exactMax) || exactMax < 0 || exactMax > 24) throw new RangeError(`signFlipTest: exactMax is a whole number from 0 to 24, not ${exactMax}.`);
+	for (const [i, t] of terms.entries()) if (!Number.isFinite(t)) throw new RangeError(`signFlipTest: the term ${i} is ${t}, not a finite number.`);
+	const n = terms.length;
+	const observed = Math.abs(terms.reduce((sum, t) => sum + t, 0));
+	const slack = 1e-12 * terms.reduce((sum, t) => sum + Math.abs(t), 0);
+	if (n <= exactMax) {
+		const total = 2 ** n;
+		let hits = 0;
+		for (let mask = 0; mask < total; mask++) {
+			let sum = 0;
+			for (let i = 0; i < n; i++) sum += mask & 1 << i ? -terms[i] : terms[i];
+			if (Math.abs(sum) >= observed - slack) hits++;
+		}
+		return {
+			observed,
+			p: hits / total,
+			exact: true,
+			runs: total
+		};
+	}
+	const runs = options.runs ?? 1e4;
+	if (!Number.isSafeInteger(runs) || runs < 1) throw new RangeError(`signFlipTest: runs is a whole number of 1 or more, not ${runs}.`);
+	const r = random(options.seed ?? 1);
+	let hits = 0;
+	for (let run = 0; run < runs; run++) {
+		let sum = 0;
+		for (let i = 0; i < n; i++) sum += r.next() < .5 ? -terms[i] : terms[i];
+		if (Math.abs(sum) >= observed - slack) hits++;
+	}
+	return {
+		observed,
+		p: (1 + hits) / (1 + runs),
+		exact: false,
+		runs
+	};
+}
 /** This function gives the count of the ways to choose `k` of `n`, or Infinity when the count is over `limit`. */
 function choices(n, k, limit) {
 	const small = Math.min(k, n - k);
@@ -3708,6 +3941,6 @@ function inverseDiagonal(hessian, k) {
 	}
 	return out;
 }
-export { KENDALL_EXACT, RESPONSE_LAMBDAS, RESPONSE_MAX_POWER, RESPONSE_MAX_TERMS, RESPONSE_RECORDS_SCHEMA, U_STATISTIC_TIGHT_MIN_SAMPLE, Z95, bestSmoothLaw, binomialInterval, binomialTest, bootstrap, bootstrapClusters, bootstrapLeastSquares, bootstrapMedian, bootstrapMedianRatio, bootstrapTwo, bradleyTerry, brier, calibration, cohenKappa, compareRatios, coverageMinSample, eloExpected, eloUpdate, fitResponse, glicko2Idle, glicko2Update, heldOutError, heldOutErrorAt, holm, kendallTau, leastSquares, logLoss, mean, median, medianInterval, modeCount, negativeBinomialInterval, noiseFloor, noiseVerdict, optionFloor, pairProbabilities, pairedPermutation, parseResponseRecord, partialConjunction, predictResponse, quantile, quantileInterval, quantileSamples, quantileSorted, ranks, ratingToStrength, responseCells, ridgeSolve, sampleTotalInterval, sd, signTest, signTestPower, sliceResponse, smoothLawScores, sorted, spearman, strengthToRating, summarize, surfaceResponse, tQuantile, twoSamplePermutation, uStatisticInterval, whenHolds, wilson, wilsonSampleSize };
+export { KENDALL_EXACT, RESPONSE_LAMBDAS, RESPONSE_MAX_POWER, RESPONSE_MAX_TERMS, RESPONSE_RECORDS_SCHEMA, SIGN_FLIP_EXACT_LIMIT, SIGN_FLIP_EXACT_MAX, U_STATISTIC_TIGHT_MIN_SAMPLE, Z95, bestSmoothLaw, binomialInterval, binomialTest, bootstrap, bootstrapClusters, bootstrapLeastSquares, bootstrapMedian, bootstrapMedianRatio, bootstrapStrata, bootstrapTwo, bradleyTerry, brier, calibration, cohenKappa, compareRatios, coverageMinSample, eloExpected, eloUpdate, fitResponse, glicko2Idle, glicko2Update, heldOutError, heldOutErrorAt, holm, kendallTau, leastSquares, logLoss, mean, median, medianInterval, modeCount, negativeBinomialInterval, noiseFloor, noiseVerdict, optionFloor, pairProbabilities, pairedPermutation, parseResponseRecord, partialConjunction, predictResponse, quantile, quantileInterval, quantileSamples, quantileSorted, ranks, ratingToStrength, responseCells, ridgeSolve, sampleTotalInterval, sd, signFlipTest, signTest, signTestPower, sliceResponse, smoothLawScores, sobolIndices, sorted, spearman, strengthToRating, summarize, surfaceResponse, tQuantile, twoSamplePermutation, uStatisticInterval, whenHolds, wilson, wilsonSampleSize };
 
 //# sourceMappingURL=index.js.map