@mark1russell7/rigor-statistics 0.1.5 → 0.1.7

package/README.md

@@ -1,6 +1,6 @@
 # @mark1russell7/rigor-statistics
 
-The statistics of measured work. It gives quantiles, bootstrap intervals, the sign test, the paired permutation test, Wilson intervals and the Holm correction. It gives the interval of a median from order statistics, and the central interval of a binomial count. It also gives least squares, rank correlation, the comparison with a baseline beyond the noise, and the Brier score of predictions.
+The statistics of measured work. It gives quantiles, bootstrap intervals, the sign test, the paired permutation test, Wilson intervals and the Holm correction. It gives the interval of a median from order statistics, and the central interval of a binomial count. It also gives least squares, rank correlation, the comparison with a baseline beyond the noise, and the Brier score of predictions. It fits response records that a browser can evaluate from plain JSON.
 
 ```sh
 npm install @mark1russell7/rigor-statistics
@@ -9,7 +9,7 @@ npm install @mark1russell7/rigor-statistics
 ## Compare a candidate with a baseline
 
 ```ts
-import { compareRatios, noiseFloor } from "@mark1russell7/rigor-statistics";
+import { compareRatios, noiseFloor, noiseVerdict } from "@mark1russell7/rigor-statistics";
 
 // The ratio candidate / baseline of each input, and the measure of 5 seeds or folds of the same run.
 const noise = noiseFloor(foldMedians);           // 2 standard deviations; it needs 5 values or more
@@ -18,10 +18,61 @@ const c = compareRatios(ratios, { noise });
 c.wins, c.losses, c.signP;                       // how often, by any amount
 c.effect;                                        // by how much: the median, p10 and p90 of the inputs that are not ties
 c.gains, c.drops;                                // how often beyond the noise, with Wilson intervals
+
+// A difference (candidate - baseline) of each seed: beyond max(2 SD, margin)? Equal values give "no-spread".
+const v = noiseVerdict(foldDiffs, { margin: 0.02 }); // give the reason of the margin from the problem of the user
+v.state;                                         // "gain", "loss", "noise" or "no-spread"
 ```
 
 A promotion reads `gains` against `drops` and `effect`. A sign test alone is not enough, because it counts a win of any size. Judge a change of a default against the current default, not against "no change".
 
+## Fit and show a response
+
+The format `response-records/1` keeps the measured points of a study, a model for each measure and the held-out error of each fold. A record is plain JSON, and the functions use no Node API. Thus a browser reads a record and evaluates its models.
+
+```ts
+import { fitResponse, parseResponseRecord, sliceResponse } from "@mark1russell7/rigor-statistics";
+
+const record = fitResponse(
+  {
+    schema: "response-records/1",
+    name: "time of a placement",
+    dimensions: [
+      { id: "boxes", label: "Boxes", kind: "continuous", range: [10, 10000], unit: "", log: true },
+      { id: "strategy", label: "Strategy", kind: "categorical", levels: ["shelf", "skyline"], unit: "" },
+      // Only the arm "skyline" has this parameter. Each other point has null.
+      { id: "depth", label: "Depth", kind: "continuous", range: [1, 8], unit: "", when: { dimension: "strategy", levels: ["skyline"] } },
+    ],
+    measures: [{ id: "ms", label: "Time", unit: "ms", direction: "lower" }],
+    points, // { id, values, measures, group }: repeats of one input have one group
+  },
+  { log: ["ms"], logFloor: 0.001 },
+);
+
+// A server writes the text. A browser reads it, and it asks for one dimension with the other dimensions held.
+const text = JSON.stringify(record);
+const parsed = parseResponseRecord(JSON.parse(text));
+if (!parsed.ok) throw new Error(JSON.stringify(parsed.faults));
+const slice = sliceResponse(parsed.record, "ms", { vary: "depth", hold: { boxes: 1000 } });
+slice.curve; // { at, value, low, high, nearest } along the dimension
+slice.points; // the measured points with the held values
+slice.error; // the held-out error of the folds
+```
+
+A test of this package compiles and starts this example, and it examines that the two texts are equal.
+
+`fitResponse` keeps all the points of one `group` in one fold, so repeats of a group are not on both sides of a split. Each fold chooses its penalty from the other folds only. The folds thus measure the prediction of an unseen group.
+
+There are two kinds of model. A `ridge` model is a sum of weighted terms. Each term is a product of powers of number inputs and one-hot columns of levels. A `table` model has one cell for each combination of levels. `fitResponse` gives a table when each dimension is categorical, and a ridge model in other cases.
+
+A dimension with `when` is conditional: only some levels of its parent have the dimension. This suits a benchmark in which each arm has its own parameters. A point at another level has `null`. In a ridge model, a term with an absent dimension is 0, so the columns of the parent carry the mean of that level. A table keeps `null` in its cells. For each kind of model, `predictResponse` refuses `null` at a dimension with no condition.
+
+To vary a conditional dimension, `sliceResponse` needs its parent at a level inside `when`. With no held value, the parent takes the most common level inside `when`. A held conditional dimension reads `null` when its parent is outside `when`.
+
+The band of a curve is `value ± width × RMSE`, from the held-out error of the folds. On a log scale, the band is a ratio. The band has one width along the curve. It is the mean error of the model.
+
+The field `nearest` of each place tells how far the place is from a measurement. It goes from 0 to 1, and 0 means that a point was measured there. A viewer can fade the band away from the data. A table cell with the `fallback` value has `nearest` above 0.
+
 ## The functions
 
 | Function | What it gives |
@@ -42,8 +93,12 @@ A promotion reads `gains` against `drops` and `effect`. A sign test alone is not
 | `binomialInterval(n, p, level)`, `negativeBinomialInterval(successes, p, level)` | The central interval of a binomial count, and of a count of failures before `successes` successes. Each tail outside holds `(1 - level) / 2` or less. |
 | `holm(pValues, level)` | The Holm correction of a family of tests. |
 | `spearman`, `kendallTau`, `ranks` | The rank correlations, with mean ranks for ties. |
-| `compareRatios`, `noiseFloor` | The comparison with a baseline, and the noise floor of the 5-seed rule. |
+| `compareRatios`, `noiseFloor`, `noiseVerdict` | The comparison with a baseline, the noise floor of the 5-seed rule with a margin of practical effect, and the state of a difference over the seeds. |
 | `brier`, `calibration` | The Brier score of predictions, and the hit rate of each stated level. |
+| `fitResponse(data, options)` | A record of the format `response-records/1`, with one model for each measure and held-out folds. |
+| `parseResponseRecord(value)` | The examination of unknown data. It gives the record, or one fault with a path for each error. |
+| `predictResponse(model, inputs)`, `heldOutError(record, measure)` | The value of a model at one set of inputs, and the error of the folds of a measure. |
+| `sliceResponse(record, measure, options)`, `surfaceResponse(record, measure, options)` | A fitted curve over one dimension, or a fitted surface over two, with the matching measured points, a band and the distance to the nearest measurement. |
 
 Each function throws a RangeError for an empty list, for a NaN and for a parameter out of its range. Thus a missing value cannot make a quiet NaN in a record.
 

package/build/index.d.ts

@@ -8,7 +8,10 @@ type QuantileMethod = "linear" | "nearest-rank" | "higher";
 /** A new list with the values from the smallest to the largest. */
 export declare function sorted(values: readonly number[]): number[];
 export declare function mean(values: readonly number[]): number;
-/** The sample standard deviation (the divisor is n - 1). It needs 2 values or more. */
+/**
+ * The sample standard deviation (the divisor is n - 1). It needs 2 values or more. Equal values give exactly 0. The
+ * rounding of the mean gives a small number that is not 0 for some equal values, so the function examines them first.
+ */
 export declare function sd(values: readonly number[]): number;
 /**
  * The `p` quantile of a list that is already sorted. Use it when you take several quantiles of one list.
@@ -158,6 +161,420 @@ interface RidgeOptions {
  * `k` is 0. It also throws for a value that is not finite, for a `lambda` below 0, and for a bad free index.
  */
 export declare function ridgeSolve(gram: ArrayLike<number>, moment: ArrayLike<number>, lambda: number, options?: RidgeOptions): number[];
+/** The value of the field `schema` of a record. */
+export declare const RESPONSE_RECORDS_SCHEMA = "response-records/1";
+/** The value of a dimension at a point: a number, or the text of a level. */
+type ResponseValue = number | string;
+/** The value of a dimension at a place. The value `null` means that a conditional dimension is absent there. */
+type ResponseInputValue = ResponseValue | null;
+/** The condition of a dimension: the dimension has a value only at these levels of its parent. */
+interface ResponseWhen {
+  /** The id of the parent: a categorical or ordinal dimension with no condition. */
+  readonly dimension: string;
+  readonly levels: readonly ResponseValue[];
+}
+/** The fields of each kind of dimension. */
+interface ResponseDimensionBase {
+  /** The key of the dimension in the values of a point. */
+  readonly id: string;
+  /** The name that a view shows. */
+  readonly label: string;
+  /** The unit. An empty string means that the dimension has no unit. */
+  readonly unit: string;
+  /** With this field, a point has a value only at the stated levels of the parent. Each other point has `null`. */
+  readonly when?: ResponseWhen;
+}
+/** A dimension with each number in a range. */
+interface ContinuousDimension extends ResponseDimensionBase {
+  readonly kind: "continuous";
+  /** The smallest and the largest value. The first is less than the second. */
+  readonly range: readonly [number, number];
+  /** When true, a model and an axis use the base-10 logarithm of the value. All of the range is above 0. */
+  readonly log?: boolean;
+}
+/** A dimension with levels in order. Number levels ascend. A model reads a text level as its position. */
+interface OrdinalDimension extends ResponseDimensionBase {
+  readonly kind: "ordinal";
+  readonly levels: readonly number[] | readonly string[];
+  /** When true, a model uses the base-10 logarithm of the level. The levels are numbers above 0. */
+  readonly log?: boolean;
+}
+/** A dimension with levels that have no order, for example the arms of a benchmark. */
+interface CategoricalDimension extends ResponseDimensionBase {
+  readonly kind: "categorical";
+  readonly levels: readonly string[];
+}
+/** A dimension of a record. */
+type ResponseDimension = ContinuousDimension | OrdinalDimension | CategoricalDimension;
+/** A measured value of each point. */
+interface ResponseMeasure {
+  /** The key of the measure in the measures of a point. */
+  readonly id: string;
+  readonly label: string;
+  /** The unit. An empty string means that the measure has no unit. */
+  readonly unit: string;
+  /** The better direction of the value. The value `none` means that no direction is better. */
+  readonly direction: "higher" | "lower" | "none";
+}
+/** One measured point. */
+interface ResponsePoint {
+  readonly id: string;
+  /** The value of each dimension. */
+  readonly values: Readonly<Record<string, ResponseInputValue>>;
+  /** The value of each measure. The value `null` means that the run did not give the measure at this point. */
+  readonly measures: Readonly<Record<string, number | null>>;
+  /** The points of one group stay on one side of each split, for example the repeats of one seed. */
+  readonly group?: string;
+}
+/** One factor of a term: a power of a number dimension, or 1 at a level and 0 at each other level. */
+type ResponseFactor = {
+  readonly dimension: string;
+  readonly power: number;
+} | {
+  readonly dimension: string;
+  readonly level: ResponseValue;
+};
+/** The fields of each kind of input of a ridge model. */
+interface RidgeInputBase {
+  readonly dimension: string;
+  /** When true, the dimension is conditional, so `null` is a good value. Each term of an absent dimension is 0. */
+  readonly conditional?: boolean;
+}
+/** The coordinate of a number dimension: the value (or its logarithm), less `center`, divided by `spread`. */
+interface RidgeNumberInput extends RidgeInputBase {
+  readonly kind: "number";
+  readonly center: number;
+  readonly spread: number;
+  readonly log?: boolean;
+}
+/** The coordinate of an ordinal dimension with text levels: the position of the level, as in `RidgeNumberInput`. */
+interface RidgeRankInput extends RidgeInputBase {
+  readonly kind: "rank";
+  readonly levels: readonly string[];
+  readonly center: number;
+  readonly spread: number;
+}
+/** A categorical dimension: the factors of its levels are the one-hot columns. */
+interface RidgeLevelInput extends RidgeInputBase {
+  readonly kind: "level";
+  readonly levels: readonly string[];
+}
+/** An input of a ridge model. The model holds it, so `predictResponse` needs no record. */
+type RidgeInput = RidgeNumberInput | RidgeRankInput | RidgeLevelInput;
+/** A term of a ridge model: the product of its factors, times the weight. */
+interface RidgeTerm {
+  readonly factors: readonly ResponseFactor[];
+  readonly weight: number;
+}
+/** The fields of each kind of model. */
+interface ResponseModelBase {
+  /** The id of the measure of the model. */
+  readonly measure: string;
+  /** With `log`, the model and the held-out errors use the base-10 logarithm of the measure. */
+  readonly scale: "linear" | "log";
+  /** With the scale `log`, the fit gave this value to each smaller measured value. */
+  readonly floor?: number;
+}
+/** A ridge fit over a stated expansion: the intercept, plus the sum of the terms. */
+interface RidgeResponseModel extends ResponseModelBase {
+  readonly kind: "ridge";
+  /** The penalty of the fit. */
+  readonly lambda: number;
+  readonly inputs: readonly RidgeInput[];
+  readonly intercept: number;
+  readonly terms: readonly RidgeTerm[];
+}
+/** One cell of a table model. */
+interface ResponseTableCell {
+  /** The value of each dimension of the table, in the order of its `dimensions`. An absent dimension has `null`. */
+  readonly at: readonly ResponseInputValue[];
+  /** The mean of the measured values of the cell, on the scale of the model. */
+  readonly value: number;
+  /** The number of points of the cell. */
+  readonly n: number;
+}
+/** The mean of each measured combination of levels. */
+interface TableResponseModel extends ResponseModelBase {
+  readonly kind: "table";
+  readonly dimensions: readonly string[];
+  /** The ids of the dimensions of the table with a condition. Only these dimensions take `null`. */
+  readonly conditional?: readonly string[];
+  readonly cells: readonly ResponseTableCell[];
+  /** The value of a combination with no cell: the mean of all the fitted points, on the scale of the model. */
+  readonly fallback: number;
+}
+/** A model that a browser evaluates from plain JSON. */
+type ResponseModel = RidgeResponseModel | TableResponseModel;
+/** The held-out error of one measure in one fold, on the scale of the model. */
+interface ResponseFoldError {
+  /** The number of held-out points with a value of the measure. */
+  readonly n: number;
+  /** The mean absolute error. */
+  readonly mae: number;
+  /** The root mean square error. */
+  readonly rmse: number;
+  /** The mean absolute error of the mean of the training points, used as the prediction. */
+  readonly baselineMae: number;
+}
+/** One split of the points. The fold held out `heldOut`, and it fit a model on the other points. */
+interface ResponseFold {
+  /** The ids of the held-out points. */
+  readonly heldOut: readonly string[];
+  /** The error of each measure. A measure with no held-out point or no training point has no entry. */
+  readonly errors: Readonly<Record<string, ResponseFoldError>>;
+}
+/** The data of a record, before a fit. */
+interface ResponseData {
+  readonly schema: typeof RESPONSE_RECORDS_SCHEMA;
+  readonly name: string;
+  readonly dimensions: readonly ResponseDimension[];
+  readonly measures: readonly ResponseMeasure[];
+  readonly points: readonly ResponsePoint[];
+}
+/** A record: the data, one model for each measure, and the folds. */
+interface ResponseRecord extends ResponseData {
+  readonly models: readonly ResponseModel[];
+  readonly folds: readonly ResponseFold[];
+}
+/** The held-out error of one measure over all the folds, on the scale of the model. */
+interface ResponseHeldOutError {
+  /** The number of held-out points. */
+  readonly n: number;
+  /** The number of folds with this measure. */
+  readonly folds: number;
+  /** The mean absolute error of all the held-out points. */
+  readonly mae: number;
+  /** The root mean square error of all the held-out points. */
+  readonly rmse: number;
+  /** The mean absolute error of the baseline. */
+  readonly baselineMae: number;
+  /** The value `1 - mae / baselineMae`. The value 0 is no better than the mean. A baseline of 0 gives `null`. */
+  readonly skill: number | null;
+}
+/** The options of `sliceResponse` and `surfaceResponse`. */
+interface ResponseViewOptions {
+  /** The value of each held dimension. Each other dimension takes its usual value, as `sliceResponse` tells. */
+  readonly hold?: Readonly<Record<string, ResponseInputValue>>;
+  /** The number of values on a continuous axis, 2 or more. The default is 33. */
+  readonly steps?: number;
+  /**
+   * The share of the range of a held number dimension in which a point matches. A log dimension uses the log scale.
+   * The default is 0: only an equal value matches.
+   */
+  readonly tolerance?: number;
+  /** The half width of the band, as a multiple of the held-out root mean square error. The default is 1. */
+  readonly width?: number;
+}
+/** The options of `sliceResponse`. */
+interface ResponseSliceOptions extends ResponseViewOptions {
+  /** The id of the dimension of the curve. */
+  readonly vary: string;
+}
+/** The options of `surfaceResponse`. */
+interface ResponseSurfaceOptions extends ResponseViewOptions {
+  /** The ids of the two dimensions of the surface: first the x axis, then the y axis. */
+  readonly vary: readonly [string, string];
+}
+/**
+ * The fitted value at one place, with its band. The band is the mean error of the model, and `nearest` tells how far
+ * the place is from a measurement.
+ */
+interface ResponseBand {
+  readonly value: number;
+  readonly low: number;
+  readonly high: number;
+  /**
+   * The distance from this place to the nearest measured point with a value of the measure, from 0 to 1. The value 0
+   * means that a point was measured here. A number or rank dimension counts as its share of the axis. A different
+   * categorical level counts 1, and an absent value against a present value counts 1. The distance is the largest
+   * count over the dimensions.
+   */
+  readonly nearest: number;
+}
+/** One place of a curve. */
+interface ResponseCurvePoint extends ResponseBand {
+  /** The value of the varied dimension. */
+  readonly at: ResponseValue;
+}
+/** A measured point of a slice. */
+interface ResponseSlicePoint {
+  readonly id: string;
+  /** The value of the varied dimension. */
+  readonly at: ResponseValue;
+  readonly value: number;
+  readonly group?: string;
+}
+/** The fitted curve over one dimension, and the measured points that match the held values. */
+interface ResponseSlice {
+  readonly measure: string;
+  readonly vary: string;
+  /** The value of each other dimension. */
+  readonly held: Readonly<Record<string, ResponseInputValue>>;
+  readonly curve: readonly ResponseCurvePoint[];
+  readonly points: readonly ResponseSlicePoint[];
+  readonly error: ResponseHeldOutError;
+}
+/** A measured point of a surface. */
+interface ResponseSurfacePoint {
+  readonly id: string;
+  /** The values of the two varied dimensions. */
+  readonly at: readonly [ResponseValue, ResponseValue];
+  readonly value: number;
+  readonly group?: string;
+}
+/** The fitted surface over two dimensions, and the measured points that match the held values. */
+interface ResponseSurface {
+  readonly measure: string;
+  readonly vary: readonly [string, string];
+  /** The value of each other dimension. */
+  readonly held: Readonly<Record<string, ResponseInputValue>>;
+  readonly x: readonly ResponseValue[];
+  readonly y: readonly ResponseValue[];
+  /** One row for each value of `y`. Each row has one cell for each value of `x`. */
+  readonly cells: readonly (readonly ResponseBand[])[];
+  readonly points: readonly ResponseSurfacePoint[];
+  readonly error: ResponseHeldOutError;
+}
+/**
+ * This function gives the value of a model at `inputs`, on the scale of the measure. The inputs have a value for each
+ * dimension that the model reads. An absent conditional dimension has `null`. A table model gives its `fallback` for a
+ * combination with no cell.
+ *
+ * The function throws a RangeError for a missing input and for a value that is not a number or a level of the model.
+ * It throws for `null` at a dimension with no condition, for each kind of model. It also throws for a value at or below
+ * 0 on a log scale. A number outside the range of the record is good, so the
+ * function extrapolates.
+ */
+export declare function predictResponse(model: ResponseModel, inputs: Readonly<Record<string, ResponseInputValue>>): number;
+/**
+ * This function gives the held-out error of a measure over all the folds. Each point has the same weight, so a small
+ * fold has a small effect. The function throws a RangeError when no fold has the measure.
+ */
+export declare function heldOutError(record: Pick<ResponseRecord, "folds">, measure: string): ResponseHeldOutError;
+/**
+ * This function gives a view of a measure over one dimension of a parsed record:
+ * - `curve`: the fitted value at each level, or at `steps` values of a continuous dimension, with its band.
+ * - `points`: the measured points that match the held values. This list is empty when no point matches them.
+ * - `held`: the value of each other dimension.
+ * - `error`: the held-out error.
+ *
+ * Each dimension with no value in `hold` takes its usual value. For a continuous dimension, this value is the median
+ * of its measured values. For the other kinds, it is the most common level, and a tie gives the first level.
+ *
+ * A conditional dimension follows its parent:
+ * - To vary a conditional dimension, the parent needs a level inside `when`. With no held value, the parent takes the
+ *   most common level inside `when`. A held level outside `when` is a RangeError.
+ * - A held conditional dimension reads `null` in `held` when its parent is outside `when`, also with a value in `hold`.
+ *   The points that match have `null` at this dimension.
+ * - When the parent varies, a held conditional dimension keeps its value in `held`. Each place outside `when` reads
+ *   `null` for the fit, and for the matching points.
+ *
+ * The band is the fitted value, plus and less `width` times the held-out root mean square error. On a log scale, this
+ * distance is on the log scale of the model, so the band is a ratio. The band has one width along all the curve. The
+ * band is the mean error of the model, and `nearest` tells how far the place is from a measurement.
+ *
+ * The function throws a RangeError for an unknown measure or dimension, and for a held value that is not good for its
+ * dimension. It also throws for a dimension that is both varied and held, and for a bad option.
+ */
+export declare function sliceResponse(record: ResponseRecord, measure: string, options: ResponseSliceOptions): ResponseSlice;
+/**
+ * This function gives the view of `sliceResponse` over two dimensions. The fields `x` and `y` are the two axes, and
+ * `cells` has one row for each value of `y`. The rules of the held values, of the matching points and of the band are
+ * the rules of `sliceResponse`. The function also throws a RangeError when the two varied dimensions are one dimension.
+ * It throws when a conditional dimension varies with its parent, and when two varied dimensions need no common level.
+ */
+export declare function surfaceResponse(record: ResponseRecord, measure: string, options: ResponseSurfaceOptions): ResponseSurface;
+/** The largest power of a factor. A larger power of a coordinate from -1 to 1 adds almost nothing to a model. */
+export declare const RESPONSE_MAX_POWER = 8;
+/** One fault of a record. */
+interface ResponseFault {
+  /** The place of the fault, for example `points[3].values.size`. An empty string is the record itself. */
+  readonly path: string;
+  /** What is wrong, and what the format needs. */
+  readonly message: string;
+}
+/** The result of `parseResponseRecord`: the record, or each fault that the validator found. */
+type ParsedResponseRecord = {
+  readonly ok: true;
+  readonly record: ResponseRecord;
+} | {
+  readonly ok: false;
+  readonly faults: readonly ResponseFault[];
+};
+/**
+ * This function examines unknown data, for example the result of `JSON.parse`, against `response-records/1`. It gives
+ * `{ ok: true, record }`, with the same object as the record, or `{ ok: false, faults }`. Each fault has the path of
+ * the bad field and a message. The function does not throw. Data that the checks cannot read, for example an object
+ * with a loop, gives one fault.
+ *
+ * A field that the format does not have is a fault. Thus a wrong spelling of an optional field cannot go unseen.
+ *
+ * The function examines the models and the folds only when the data has no fault, because they refer to the data.
+ * Thus a correction of the data can show more faults. These rules apply to the folds:
+ * - A point is in one fold or in no fold.
+ * - All the points of a group are in one fold, or all are in no fold. Thus no group is on both sides of a split.
+ * - The count `n` of an error is the number of held-out points with a value of the measure.
+ * - Each measure has an error in one fold or more.
+ */
+export declare function parseResponseRecord(value: unknown): ParsedResponseRecord;
+/** The options of `fitResponse`. */
+interface ResponseFitOptions {
+  /** The number of folds, 3 or more. With fewer groups, each group is one fold. The default is 5. */
+  readonly folds?: number;
+  /** The seed of the split. The default is 1. */
+  readonly seed?: number | string;
+  /** The kind of model. The default `auto` gives a table when each dimension is categorical, and a ridge fit otherwise. */
+  readonly model?: "auto" | "ridge" | "table";
+  /** The largest total power of a term of a ridge fit, from 1 to 8. The default is 2. */
+  readonly degree?: number;
+  /**
+   * When true, a ridge fit also has the products of two dimensions, up to `degree`. The default is true. The value
+   * false also removes the products of a conditional dimension with the levels of its parent. Thus this option is the
+   * way out of the limit `RESPONSE_MAX_TERMS`.
+   */
+  readonly interactions?: boolean;
+  /** The stated terms of a ridge fit, in place of `degree` and `interactions`. */
+  readonly terms?: readonly (readonly ResponseFactor[])[];
+  /** The penalties that a ridge fit chooses from, each above 0. The default is `RESPONSE_LAMBDAS`. */
+  readonly lambdas?: readonly number[];
+  /** The ids of the measures to fit on a log scale. */
+  readonly log?: readonly string[];
+  /** On a log scale, each smaller value takes this value. With no floor, a value of 0 or less is an error. */
+  readonly logFloor?: number;
+}
+/** The default penalties of a ridge fit. The coordinates are from -1 to 1, so one list suits each record. */
+export declare const RESPONSE_LAMBDAS: readonly number[];
+/** The largest number of coefficients of a ridge fit. The memory of each fold grows as the square of this number. */
+export declare const RESPONSE_MAX_TERMS = 1024;
+/**
+ * This function fits each measure of `data`, and it gives a record with the models and the folds.
+ *
+ * The split: the groups, sorted and then shuffled with `seed`, go to the folds in turn. A point with no group is a
+ * group of its own. Thus the repeats of one group are not on both sides of a split. The same split applies to each
+ * measure. A record needs 3 groups or more.
+ *
+ * The model: a ridge fit over a stated expansion, or the mean of each cell of a table. The default expansion has
+ * each power of each number dimension up to `degree`. It has one column for each level of a categorical dimension.
+ * It also has the products of two dimensions up to `degree`. Each coordinate goes from -1 to 1 over the declared
+ * range. The intercept has no penalty.
+ *
+ * A conditional dimension: a term with an absent dimension is 0. Thus the columns of the parent carry the mean of each
+ * level with no value. A dimension with two parent levels or more also has each power times each of these levels, if
+ * `interactions` is true. A table keeps `null` as a level of its cells.
+ *
+ * The penalty: each fold of a ridge fit chooses from `lambdas` with the other folds. Thus the held-out error of a
+ * fold is not optimistic. The model of the record chooses with all the folds.
+ *
+ * The errors: a fold scores the model of the other folds on its held-out points, on the scale of the model.
+ *
+ * The result does not depend on the order of the points, except for the field `points`, which keeps the order of
+ * the data. The same data and options give the same bytes in one JavaScript engine.
+ *
+ * The function throws a RangeError for data with a fault, for a bad option, and for fewer than 3 groups. It also
+ * throws for a measure with no value, and for a measure with all its values in one fold. It throws for a value of 0
+ * or less on a log scale with no `logFloor`. It throws for more than `RESPONSE_MAX_TERMS` terms.
+ */
+export declare function fitResponse(data: ResponseData, options?: ResponseFitOptions): ResponseRecord;
 /**
  * The sign test of paired results: `better` inputs where the candidate wins and `worse` where it loses. The ties
  * do not count. One-sided (the default), it is the p of `better` wins or more if the two are equal. Two-sided, it is
@@ -273,12 +690,46 @@ interface NoiseOptions {
   readonly k?: number;
   /** The fewest values (seeds or folds) that give a floor. The default is 5, the 5-seed rule. */
   readonly minimum?: number;
+  /**
+   * The margin of practical effect, in the units of the values. A difference that is real but smaller than the margin
+   * is too small to matter, so it is not a result. Give the reason of the margin from the problem of the user. The
+   * default is 0: no margin.
+   */
+  readonly margin?: number;
 }
 /**
- * The noise floor of a measure: k sample standard deviations of its value over independent seeds or folds. A
- * difference smaller than the floor is not a result.
+ * The noise floor of a measure: k sample standard deviations of its value over independent seeds or folds. When the
+ * margin of practical effect is larger, the floor is the margin. A difference smaller than the floor is not a result.
  */
 export declare function noiseFloor(values: readonly number[], options?: NoiseOptions): number;
+/**
+ * The state of a difference over the seeds. "gain" and "loss": the mean is beyond the threshold. "noise": it is not.
+ * "no-spread": each seed gives the same value, so the seeds do not sample the noise and the rule tests nothing.
+ */
+type NoiseState = "gain" | "loss" | "noise" | "no-spread";
+/** The noise rule of a difference over independent seeds or folds. */
+interface NoiseVerdict {
+  /** The number of values. */
+  readonly n: number;
+  readonly mean: number;
+  /** The sample standard deviation of the values. */
+  readonly sd: number;
+  /** k times the standard deviation. */
+  readonly spread: number;
+  readonly margin: number;
+  /** The larger of `spread` and `margin`. The absolute mean must be above it. */
+  readonly threshold: number;
+  readonly state: NoiseState;
+}
+/**
+ * The noise rule of a difference (candidate minus baseline) over independent seeds or folds: the mean against
+ * max(k standard deviations, margin). With `better: "lower"`, a negative mean is the gain. When each value is equal,
+ * the state is "no-spread" and not a result, also when the mean is beyond the margin. Find out why the seeds do not
+ * change the value before you read the difference.
+ */
+export declare function noiseVerdict(values: readonly number[], options?: NoiseOptions & {
+  readonly better?: "higher" | "lower";
+}): NoiseVerdict;
 interface CompareOptions {
   /** "higher" (the default): a ratio above 1 is a gain. "lower": a ratio below 1 is a gain, as for a time or a cost. */
   readonly better?: "higher" | "lower";
@@ -345,5 +796,5 @@ interface CalibrationLevel {
 }
 /** The hit rate of each stated level, from the lowest level. */
 export declare function calibration(forecasts: readonly Forecast[]): CalibrationLevel[];
-export type { BootstrapOptions, CalibrationLevel, CompareOptions, Comparison, Corrected, CountInterval, Effect, Forecast, Interval, LeastSquaresFit, LeastSquaresOptions, MedianInterval, NoiseOptions, PermutationOptions, PermutationResult, Proportion, QuantileMethod, RidgeOptions, Summary };
+export type { BootstrapOptions, CalibrationLevel, CategoricalDimension, CompareOptions, Comparison, ContinuousDimension, Corrected, CountInterval, Effect, Forecast, Interval, LeastSquaresFit, LeastSquaresOptions, MedianInterval, NoiseOptions, NoiseState, NoiseVerdict, OrdinalDimension, ParsedResponseRecord, PermutationOptions, PermutationResult, Proportion, QuantileMethod, ResponseBand, ResponseCurvePoint, ResponseData, ResponseDimension, ResponseDimensionBase, ResponseFactor, ResponseFault, ResponseFitOptions, ResponseFold, ResponseFoldError, ResponseHeldOutError, ResponseInputValue, ResponseMeasure, ResponseModel, ResponseModelBase, ResponsePoint, ResponseRecord, ResponseSlice, ResponseSliceOptions, ResponseSlicePoint, ResponseSurface, ResponseSurfaceOptions, ResponseSurfacePoint, ResponseTableCell, ResponseValue, ResponseViewOptions, ResponseWhen, RidgeInput, RidgeInputBase, RidgeLevelInput, RidgeNumberInput, RidgeOptions, RidgeRankInput, RidgeResponseModel, RidgeTerm, Summary, TableResponseModel };
 //# sourceMappingURL=index.d.ts.map

package/build/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","names":[],"sources":["../src/describe.ts","../src/regression.ts","../src/bootstrap.ts","../src/ridge.ts","../src/tests.ts","../src/rank.ts","../src/discrete.ts","../src/compare.ts"],"mappings":";;;;;;KAOY;;wBAmBI,OAAO;wBAIP,KAAK;;wBAQL,GAAG;;;;;;;wBAeH,eAAe,iCAAiC,WAAW,SAAQ;;wBAkBnE,SAAS,2BAA2B,WAAW,SAAQ;;;;;wBASvD,OAAO;;UAQN;WACN;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;;wBAGK,UAAU,4BAA4B;;;;;;UClGrC;;WAEN;;UAGM;;WAEN;;WAEA;;;;;WAKA;;WAEA;;WAEA;;;;;;;;;;;wBAkBK,aAAa,mCAAmC,sBAAsB,UAAS,sBAA2B;UC/BzG;;WAEN;;WAEA;;WAEA;;WAEA,SAAS;;;;;;WAMT;;UAGM;;WAEN;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;;;;;wBAOK,UAAU,GAAG,eAAe,KAAK,YAAY,iBAAiB,gBAAgB,UAAS,mBAAwB;;;;;;;;wBAsB/G,aAAa,GAAG,GAC9B,YAAY,KACZ,YAAY,KACZ,YAAY,YAAY,KAAK,YAAY,gBACzC,UAAS,mBACR;;;;;;wBAwBa,qBAAqB,8BAA8B,gCAAgC,UAAS,mBAAwB;;;;;;;wBAUpH,sBACd,mCACA,sBACA,UAAS,mBAAmB,sBAC3B;;wBAmFa,gBAAgB,2BAA2B,UAAS,mBAAwB;;;;;;wBAS5E,kBAAkB,GAChC,eAAe,KACf,UAAU,KAAK,cACf,YAAY,eAAe,gBAC3B,UAAS,mBACR;;;;;;;UCxMc;;WAEN;;;;;;;;;;;;wBAoBK,WAAW,MAAM,mBAAmB,QAAQ,mBAAmB,gBAAgB,UAAS;;;;;;wBCQxF,SAAS,gBAAgB,eAAe;;;;;wBAaxC,cAAc,WAAW,gBAAc;;qBAO1C;;UAGI;WACN;WACA;;;;;;wBAOK,OAAO,WAAW,WAAW,aAAkB;;UAc9C;;WAEN;;WAEA;;WAEA;;;;;;;wBAQK,KAAK,4BAA4B,iBAAe;UAa/C;;WAEN;;WAEA;;WAEA,aAAa;;UAGP;;WAEN;;WAEA;WACA;;;;;;wBAOK,kBAAkB,8BAA8B,6BAA6B,UAAS,qBAA0B;;wBCtHhH,MAAM;;;;;wBAgBN,SAAS,sBAAsB;;qBAmBlC;;;;;;wBAQG,WAAW,sBAAsB,sBAAsB;;UC1BtD;WACN;WACA;;WAEA;;WAEA;;;;;;;;;;wBAWK,eAAe,2BAA2B,iBAAe;;UAqBxD;WACN;WACA;;WAEA;;;;;;;wBAQK,iBAAiB,WAAW,WAAW,iBAAe;;;;;;wBA2BtD,yBAAyB,mBAAmB,WAAW,iBAAe;UClGrE;;WAEN;;WAEA;;;;;;wBAOK,WAAW,2BAA2B,UAAS;UAM9C;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;UAGM;;WAEN;WACA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;;WAEA;;WAEA;WACA,cAAc;;WAEd;WACA,cAAc;;;;;;;;wBAST,cAAc,2BAA2B,UAAS,iBAAsB;;UA6CvE;WACN;WACA;;;wBASK,MAAM,oBAAoB;;UAQzB;WACN;WACA;WACA;;WAEA;;WAEA,UAAU;;;wBAIL,YAAY,oBAAoB,aAAa"}
+{"version":3,"file":"index.d.ts","names":[],"sources":["../src/describe.ts","../src/regression.ts","../src/bootstrap.ts","../src/ridge.ts","../src/response.ts","../src/response-parse.ts","../src/response-fit.ts","../src/tests.ts","../src/rank.ts","../src/discrete.ts","../src/compare.ts"],"mappings":";;;;;;KAOY;;wBAmBI,OAAO;wBAIP,KAAK;;;;;wBAgBL,GAAG;;;;;;;wBAgBH,eAAe,iCAAiC,WAAW,SAAQ;;wBAkBnE,SAAS,2BAA2B,WAAW,SAAQ;;;;;wBASvD,OAAO;;UAQN;WACN;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;WACA;;WAEA;;wBAGK,UAAU,4BAA4B;;;;;;UC3GrC;;WAEN;;UAGM;;WAEN;;WAEA;;;;;WAKA;;WAEA;;WAEA;;;;;;;;;;;wBAkBK,aAAa,mCAAmC,sBAAsB,UAAS,sBAA2B;UC/BzG;;WAEN;;WAEA;;WAEA;;WAEA,SAAS;;;;;;WAMT;;UAGM;;WAEN;WACA;WACA;WACA;WACA;;WAEA;;WAEA;;;;;;wBAOK,UAAU,GAAG,eAAe,KAAK,YAAY,iBAAiB,gBAAgB,UAAS,mBAAwB;;;;;;;;wBAsB/G,aAAa,GAAG,GAC9B,YAAY,KACZ,YAAY,KACZ,YAAY,YAAY,KAAK,YAAY,gBACzC,UAAS,mBACR;;;;;;wBAwBa,qBAAqB,8BAA8B,gCAAgC,UAAS,mBAAwB;;;;;;;wBAUpH,sBACd,mCACA,sBACA,UAAS,mBAAmB,sBAC3B;;wBAmFa,gBAAgB,2BAA2B,UAAS,mBAAwB;;;;;;wBAS5E,kBAAkB,GAChC,eAAe,KACf,UAAU,KAAK,cACf,YAAY,eAAe,gBAC3B,UAAS,mBACR;;;;;;;UCxMc;;WAEN;;;;;;;;;;;;wBAoBK,WAAW,MAAM,mBAAmB,QAAQ,mBAAmB,gBAAgB,UAAS;;qBCd3F;;KAGD;;KAGA,qBAAqB;;UAGhB;;WAEN;WACA,iBAAiB;;;UAIX;;WAEN;;WAEA;;WAEA;;WAEA,OAAO;;;UAID,4BAA4B;WAClC;;WAEA;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;;WAEA;;;UAIM,6BAA6B;WACnC;WACA;;;KAIC,oBAAoB,sBAAsB,mBAAmB;;UAGxD;;WAEN;WACA;;WAEA;;WAEA;;;UAIM;WACN;;WAEA,QAAQ,SAAS,eAAe;;WAEhC,UAAU,SAAS;;WAEnB;;;KAIC;WAA4B;WAA4B;;WAA6B;WAA4B,OAAO;;;UAGnH;WACN;;WAEA;;;UAIM,yBAAyB;WAC/B;WACA;WACA;WACA;;;UAIM,uBAAuB;WAC7B;WACA;WACA;WACA;;;UAIM,wBAAwB;WAC9B;WACA;;;KAIC,aAAa,mBAAmB,iBAAiB;;UAG5C;WACN,kBAAkB;WAClB;;;UAIM;;WAEN;;WAEA;;WAEA;;;UAIM,2BAA2B;WACjC;;WAEA;WACA,iBAAiB;WACjB;WACA,gBAAgB;;;UAIV;;WAEN,aAAa;;WAEb;;WAEA;;;UAIM,2BAA2B;WACjC;WACA;;WAEA;WACA,gBAAgB;;WAEhB;;;KAIC,gBAAgB,qBAAqB;;UAGhC;;WAEN;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN;;WAEA,QAAQ,SAAS,eAAe;;;UAI1B;WACN,eAAe;WACf;WACA,qBAAqB;WACrB,mBAAmB;WACnB,iBAAiB;;;UAIX,uBAAuB;WAC7B,iBAAiB;WACjB,gBAAgB;;;UAIV;;WAEN;;WAEA;;WAEA;;WAEA;;WAEA;;WAEA;;;UAIM;;WAEN,OAAO,SAAS,eAAe;;WAE/B;;;;;WAKA;;WAEA;;;UAIM,6BAA6B;;WAEnC;;;UAIM,+BAA+B;;WAErC;;;;;;UAOM;WACN;WACA;WACA;;;;;;;WAOA;;;UAIM,2BAA2B;;WAEjC,IAAI;;;UAIE;WACN;;WAEA,IAAI;WACJ;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,gBAAgB;WAChB,iBAAiB;WACjB,OAAO;;;UAID;WACN;;WAEA,cAAc,eAAe;WAC7B;WACA;;;UAIM;WACN;WACA;;WAEA,MAAM,SAAS,eAAe;WAC9B,YAAY;WACZ,YAAY;;WAEZ,0BAA0B;WAC1B,iBAAiB;WACjB,OAAO;;;;;;;;;;;;wBA8GF,gBAAgB,OAAO,eAAe,QAAQ,SAAS,eAAe;;;;;wBAQtE,aAAa,QAAQ,KAAK,0BAA0B,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;;;;wBAuMtE,cAAc,QAAQ,gBAAgB,iBAAiB,SAAS,uBAAuB;;;;;;;wBAmBvF,gBAAgB,QAAQ,gBAAgB,iBAAiB,SAAS,yBAAyB;;qBCroB9F;;UAGI;;WAEN;;WAEA;;;KAIC;WAAkC;WAAmB,QAAQ;;WAA8B;WAAoB,iBAAiB;;;;;;;;;;;;;;;;;wBAya5H,oBAAoB,iBAAiB;;UCzZpC;;WAEN;;WAEA;;WAEA;;WAEA;;;;;;WAMA;;WAEA,2BAA2B;;WAE3B;;WAEA;;WAEA;;;qBAIE;;qBAGA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;wBAgSG,YAAY,MAAM,cAAc,UAAS,qBAA0B;;;;;;wBC1TnE,SAAS,gBAAgB,eAAe;;;;;wBAaxC,cAAc,WAAW,gBAAc;;qBAO1C;;UAGI;WACN;WACA;;;;;;wBAOK,OAAO,WAAW,WAAW,aAAkB;;UAc9C;;WAEN;;WAEA;;WAEA;;;;;;;wBAQK,KAAK,4BAA4B,iBAAe;UAa/C;;WAEN;;WAEA;;WAEA,aAAa;;UAGP;;WAEN;;WAEA;WACA;;;;;;wBAOK,kBAAkB,8BAA8B,6BAA6B,UAAS,qBAA0B;;wBCtHhH,MAAM;;;;;wBAgBN,SAAS,sBAAsB;;qBAmBlC;;;;;;wBAQG,WAAW,sBAAsB,sBAAsB;;UC1BtD;WACN;WACA;;WAEA;;WAEA;;;;;;;;;;wBAWK,eAAe,2BAA2B,iBAAe;;UAqBxD;WACN;WACA;;WAEA;;;;;;;wBAQK,iBAAiB,WAAW,WAAW,iBAAe;;;;;;wBA2BtD,yBAAyB,mBAAmB,WAAW,iBAAe;UClGrE;;WAEN;;WAEA;;;;;;WAMA;;;;;;wBAgBK,WAAW,2BAA2B,UAAS;;;;;KASnD;;UAGK;;WAEN;WACA;;WAEA;;WAEA;WACA;;WAEA;WACA,OAAO;;;;;;;;wBASF,aAAa,2BAA2B,UAAS;WAA0B;IAAqC;UAW/G;;WAEN;;WAEA;;WAEA;;;UAIM;WACN;WACA;;WAEA;;WAEA;;UAGM;;WAEN;WACA;;WAEA;;WAEA;;WAEA;;WAEA,QAAQ;;WAER;;WAEA;;WAEA;WACA,cAAc;;WAEd;WACA,cAAc;;;;;;;;wBAST,cAAc,2BAA2B,UAAS,iBAAsB;;UA6CvE;WACN;WACA;;;wBASK,MAAM,oBAAoB;;UAQzB;WACN;WACA;WACA;;WAEA;;WAEA,UAAU;;;wBAIL,YAAY,oBAAoB,aAAa"}