skiftet_statistical 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -11,7 +11,9 @@ Modules:
11
11
  t-test, exact normal CDF and two-tailed p-values, the normal quantile, the
12
12
  chi-squared survival and the sample-ratio check.
13
13
  - **`Gamma`** / **`BetaFunction`** — the regularized incomplete gamma and beta
14
- functions, and the Beta quantile.
14
+ functions and the Beta quantile; since 0.5.0 also the shortest
15
+ (highest-density) Beta interval, and intervals of means of non-negative
16
+ values that always hold the mean.
15
17
  - **`Sampler`** — Gamma/Beta/Gaussian random sampling, RNG-injectable.
16
18
  - **`JointBeta`** — joint Monte Carlo draws over Beta posteriors (P(best),
17
19
  expected loss).
@@ -36,7 +38,9 @@ Modules:
36
38
  normal) and their weights are uncertain.
37
39
  - **`SharedCredit`** — Hájek inverse-propensity means of several parts per arm,
38
40
  with one covariance over every (arm, part) pair, when one row may credit
39
- several arms; its sums fold once and store as JSON.
41
+ several arms; its sums fold once and store as JSON. Since 0.5.0 also the
42
+ estimate under a prior pooled from its own arms, for arms of a handful of
43
+ visits.
40
44
  - **`HierarchicalRates`** — Gamma-Poisson rates on a tree: a node's ratio to
41
45
  what its parent leads us to expect, the fit of a level's strength (and of a
42
46
  two-groups hot share), the leaf fit, and checks beside them.
@@ -53,12 +57,16 @@ Modules:
53
57
  - **`Cusum`** — a one-sided change test: its statistic, its run length, the
54
58
  threshold for a stated false-alarm rate and the smallest change it can see.
55
59
 
60
+ The methods an application calls, each with what it takes, answers, guarantees
61
+ and refuses, are under [Interface contract](#interface-contract); what a
62
+ version number promises about seeded answers is under [Stability](#stability).
63
+
56
64
  ## Install
57
65
 
58
66
  In a Gemfile (published on rubygems.org):
59
67
 
60
68
  ```ruby
61
- gem "skiftet_statistical", "~> 0.4.0"
69
+ gem "skiftet_statistical", "~> 0.5.0"
62
70
  ```
63
71
 
64
72
  Or build/install locally:
@@ -69,6 +77,24 @@ bundle install
69
77
  gem build skiftet_statistical.gemspec
70
78
  ```
71
79
 
80
+ ## Stability
81
+
82
+ The version follows [semantic versioning](https://semver.org), with one rule
83
+ made explicit for a statistics library: **a seeded answer is part of the
84
+ public API.** A release that moves an answer for the same inputs, the same
85
+ key and the same RNG seed (a draw, an order, a pick, a pinned figure, a
86
+ p-value beyond its stated tolerance) is a major or a minor version, as its
87
+ CHANGELOG entry states (before 1.0, as semver allows, a minor one), and that
88
+ entry says which answers move and why. It is never a patch: a patch release
89
+ fixes documentation, adds examples, or turns a crash into the refusal the
90
+ documentation states, and moves no answer. Adding a method, or a keyword whose
91
+ default keeps every old answer, is a minor version.
92
+
93
+ The pins in `spec/pins` are literal values recorded when each behaviour was
94
+ built and never edited: a change that fails one has moved a public answer, and
95
+ ships only as the version above. The methods an application calls, with what
96
+ each guarantees, are listed under [Interface contract](#interface-contract).
97
+
72
98
  ## Descriptive statistics
73
99
 
74
100
  ```ruby
@@ -134,6 +160,7 @@ G.regularized_upper(a, x) # Q(a, x) = 1 - P(a, x)
134
160
  B = SkiftetStatistical::BetaFunction
135
161
  B.regularized(x, alpha, beta) # I_x(alpha, beta), the Beta(alpha, beta) distribution function
136
162
  B.quantile(0.975, 41.0, 361.0) # its inverse: an exact 95 % credible bound
163
+ B.shortest_interval(0.95, 1.0, 2.0) # => [0.0, 0.7763...]: the shortest 95 % interval, holding the mode
137
164
  ```
138
165
 
139
166
  The arithmetic and its order are part of the API: an app that pins these
@@ -141,6 +168,42 @@ outputs (at 1e-12, say) compares them with this exact sequence of
141
168
  operations, so a reordered formula is a breaking change even when it is "the
142
169
  same" formula.
143
170
 
171
+ ### Intervals that hold their figure
172
+
173
+ An equal-tailed interval of a share's flat-prior posterior, Beta(1 + c, 1 + n - c),
174
+ leaves the observed share c / n out when c is 0 or n: 0 of 1 reads 1.3-84.2 %.
175
+ `shortest_interval` is the highest-density interval wherever the density has one
176
+ peak or is monotone, so it holds the mode, which under the flat prior is the share:
177
+
178
+ ```ruby
179
+ B.shortest_interval(0.95, 1.0, 2.0) # => [0.0, 0.776...] (0 of 1 visit: 0 % inside)
180
+ B.shortest_interval(0.95, 41.0, 261.0) # => [0.0981, 0.1748] (40 of 300; equal tails: 0.0996-0.1765)
181
+ ```
182
+
183
+ `mean_intervals` does the same for means of non-negative values (a value per
184
+ visit that is never below 0), each read as a share of its visits: with the
185
+ visits' second moment q = n * s^2 + m^2 (s the mean's standard error), one event
186
+ is worth q / m and the item holds n * m^2 / q events; its interval is that worth
187
+ times the shortest interval of the events' flat-prior Beta. The mean always lies
188
+ inside, the interval never goes below 0, a large item's interval is the normal
189
+ one, and a value of 0 or 1 per visit gives the share's own interval. An item
190
+ with a mean of 0 takes the set's value of one event, so it too has an interval:
191
+
192
+ ```ruby
193
+ B.mean_intervals({ "a" => { visits: 342, mean: 0.0058, standard_error: 0.0041 },
194
+ "b" => { visits: 300, mean: 0.0, standard_error: 0.0 } }, mass: 0.95)
195
+ # => { "a" => [0.0009, 0.0184], "b" => [0.0, 0.0099] }
196
+ ```
197
+
198
+ Both are exact (no draw) and work by bisection: an interval with one peak
199
+ inside cost 0.7 to 2.0 ms of CPU with YJIT (Beta(3, 341) to
200
+ Beta(40 001, 1 160 001)), a one-sided one 0.06 ms, and `mean_intervals` of ten
201
+ items 9.9 ms, measured on one busy development machine, so a caller with many
202
+ items works them out once a pass rather than once a visit. An item whose
203
+ figures leave a Float's range in this arithmetic is refused, naming the item,
204
+ rather than answered with an infinite or empty interval; what each method
205
+ takes, answers and refuses is under [Interface contract](#interface-contract).
206
+
144
207
  ## Joint Beta draws
145
208
 
146
209
  For Bayesian A/B readouts and Thompson-sampling allocators: draw every arm's
@@ -933,7 +996,9 @@ weight), a symmetric positive semidefinite covariance, finite numbers, a weight
933
996
  for every part. `draws:` is an Integer >= 0 (`summary` needs at least 1).
934
997
  `rounding:` (0.0 by default) is how far each covariance entry may be off: hand
935
998
  it a `SharedCredit` estimate's `#rounding`, so a covariance summed from rows is
936
- never refused for its own rounding.
999
+ never refused for its own rounding. An arm of one visit, or of visits that all
1000
+ hold one value, has no variance and is drawn as a point; where arms may hold a
1001
+ handful of visits, draw from `SharedCredit.with_prior`'s estimate (below).
937
1002
 
938
1003
  Measured (`value_allocation_simulation_spec.rb`, below): the 95 % intervals of
939
1004
  each arm's value and of each difference covered the truth in 377 to 385 of 400
@@ -1049,6 +1114,64 @@ credit, two options of identical value read a mean P(best) of 0.58 and 0.42
1049
1114
  0.37; in the pooled cells, `contrast_for_fit` left 1,135 of 24,000 cell
1050
1115
  contrasts out of the fit.
1051
1116
 
1117
+ ### An arm of a handful of visits (`with_prior`)
1118
+
1119
+ The sandwich of an arm of one visit, or of visits that all hold one value, is
1120
+ exactly 0, so `LinearValue` draws that arm as a point: one visit at 1 against
1121
+ three at 0, 1 and 0 reads P(best) 0.994, and one visit at 0 against them 0.114,
1122
+ by the one visit's value alone. `with_prior(estimate, strength:)` answers the
1123
+ estimate under a prior pooled from its own credited arms, every arm's evidence
1124
+ topped up to `strength` visits, for drawing:
1125
+
1126
+ ```ruby
1127
+ estimate = SC.estimate(
1128
+ [ { credit: { a: 1.0 }, values: [ 1.0 ] },
1129
+ { credit: { b: 1.0 }, values: [ 0.0 ] }, { credit: { b: 1.0 }, values: [ 1.0 ] },
1130
+ { credit: { b: 1.0 }, values: [ 0.0 ] } ],
1131
+ arms: %i[a b], parts: %i[converted],
1132
+ )
1133
+ topped = SC.with_prior(estimate, strength: 20)
1134
+ # => { means: { a: [0.525], b: [0.475] }, covariance: [[0.01246875..., 0.0], [0.0, 0.01246875]],
1135
+ # weight_sums: { a: 1.0, b: 3.0 }, effective: { a: 1.0, b: 3.0 } }
1136
+ LV.summary(arms: %i[a b], parts: %i[converted], means: topped[:means], covariance: topped[:covariance],
1137
+ parameters: { w: { fixed: 1.0 } }, weights: ->(drawn) { { converted: drawn.fetch(:w) } },
1138
+ draws: 20_000, rng: Random.new(7), rounding: topped.rounding)
1139
+ # => P(best) a 0.61965, b 0.38035 (0.994 and 0.006 on the estimate itself)
1140
+ ```
1141
+
1142
+ The prior's mean per part is the credited arms' mean weighted by their Kish's
1143
+ sizes n_a, m0 = sum_a n_a * r_a / N; its per-visit covariance is
1144
+ V0 = sum_a n_a * (n_a * cov(r_a) + g_a * g_a^T) / N with g_a = r_a - m0, each
1145
+ arm's per-visit covariance and its distance from m0, so a spread between the
1146
+ arms widens it. An arm of Kish's size n below the strength k takes e = k - n
1147
+ visits of it, the Normal mean's rule taken jointly over its parts: means
1148
+ (e * m0 + n * r_a) / k, covariance block (n / k)^2 * cov(r_a) +
1149
+ (e * V0 + (e * n / k) * g_a * g_a^T) / k^2, and every covariance with another
1150
+ arm scaled by n / k, as its means are.
1151
+
1152
+ **Where visits are plenty nothing moves.** An arm at or above the strength,
1153
+ and an arm with no credit, keeps its means and its own covariance block bit for
1154
+ bit; where every credited arm is at or above it, the answer holds the given
1155
+ means and covariance, so every seeded draw `LinearValue` takes from it is the
1156
+ draw it takes from the estimate. The answer keeps the arms, the parts, the
1157
+ weight sums and Kish's sizes (the evidence's own: hand `contrast_for_fit` the
1158
+ given estimate, not this one); its rounding is the given one's or, where
1159
+ larger, that of an entry it changed. A strength of 0 tops nothing up. Refused
1160
+ (`ArgumentError`): a strength below 0 or not a finite number, an estimate
1161
+ without its weight sums or Kish's sizes, an arm with means but a Kish's size
1162
+ of 0, and an estimate whose figures rise past a Float's largest value under the
1163
+ prior (figures so small that the prior's products fall below Float::MIN read
1164
+ as 0; no estimate `estimate` answers holds them).
1165
+
1166
+ Measured (`shared_credit_prior_spec.rb`), at a strength of 20: a version of
1167
+ one visit read P(best) between 0.41 and 0.77 against versions of 1 to 100
1168
+ visits, over values of one part, of two parts at uncertain prices and of
1169
+ amounts, where the estimate itself read 0.98 or more, or 0.02 or less, in 12 of
1170
+ the table's 16 rows; no version of three visits or fewer reached 0.95. Two
1171
+ versions of one true value, one of 1 to 3 visits and the other of 3 to 50, read
1172
+ decided (a P(best) at or above 0.95) in none of 250 seeded datasets, against
1173
+ 158 on the estimate itself.
1174
+
1052
1175
  ## Hierarchical Poisson rates (`HierarchicalRates`)
1053
1176
 
1054
1177
  How far a node's rate stands from its parent's, from its count s and the count
@@ -1327,6 +1450,544 @@ Measured (`change_test_simulation_spec.rb`, below): a stated one false alarm in
1327
1450
  detectable size was caught within the horizon in 62 to 66 % of runs (bound
1328
1451
  half).
1329
1452
 
1453
+ ## Interface contract
1454
+
1455
+ The methods an application calls, each with what it takes, what it answers,
1456
+ what holds of the answer and what it refuses, so a caller can rely on the
1457
+ method without pinning its outputs in its own suite. An app tests that it hands
1458
+ these methods the right inputs and uses their answers; the library tests that
1459
+ the answers are right. Every guarantee below is backed by a library example:
1460
+ `spec/interface_contract_spec.rb` indexes them, method by method, and fails
1461
+ when a cited example is gone or a method below is no longer public. If a
1462
+ guarantee you need is missing here, ask for it to be stated and tested in the
1463
+ library rather than pinning it in your app.
1464
+
1465
+ Conventions:
1466
+
1467
+ - **Refused** means an `ArgumentError` raised before anything is computed or
1468
+ taken from an RNG, unless the entry says otherwise.
1469
+ - **Numbers** are read by the library's one number reader where the entry says
1470
+ so: a finite real number of magnitude at most 1e100 (`Numbers::LARGEST`); a
1471
+ String, a numeric one included, is not a number.
1472
+ - **Determinism** comes in three kinds, and each entry says which:
1473
+ - *keyed*: SHA256 and Integer arithmetic only, so the same parts give the
1474
+ same answer in every process, on every machine and on every Ruby version;
1475
+ - *seeded*: the same RNG stream gives the same answer, the library taking
1476
+ values from it in a stated order (`Random.new(seed)` gives the same stream
1477
+ on every Ruby the CI runs, 3.1 to 3.4);
1478
+ - *exact*: no randomness at all; the same inputs give the same Float on one
1479
+ platform, and where the computation goes through `Math.lgamma` another math
1480
+ library can move the last bits (the pins compare such figures at 1e-9
1481
+ relative, and orders, heads, branches and counts exactly, on every Ruby the
1482
+ CI runs).
1483
+
1484
+ ### Posterior sets (`Bandit::Posteriors`)
1485
+
1486
+ #### `Bandit::Posteriors.yes_no(items, priors: nil)`
1487
+
1488
+ - Takes: `items`, a Hash of item name => tally, the names non-blank Strings in
1489
+ the tie order; a tally `{ views:, outcomes: }` (Symbol or String keys,
1490
+ numbers >= 0, fractional allowed; optionally `weighted_views`,
1491
+ `weighted_outcomes` and `weighted_views_squared`); `priors` nil (Beta(1, 1)),
1492
+ one pair `[alpha, beta]` (each >= 0) or `{ item => pair }` naming every item.
1493
+ - Answers: a set, `kind` `:beta`, `parameters` `{ item => [alpha + c, beta + n - c] }`
1494
+ with c the outcomes clamped to [0, views] (or the weighted outcomes rescaled
1495
+ to the Kish size, never above the views), a pair whose total is above
1496
+ `LARGEST_TOTAL` (1e6) scaled to it with its mean kept; `views` the tallies'
1497
+ views as Floats.
1498
+ - Guarantees: plain arithmetic, so the same tallies build the same pairs bit
1499
+ for bit everywhere; an item's pair is its own tally's and prior's alone; a
1500
+ set built from another set's `parameters` keeps them bit for bit; the same
1501
+ rule as `WeightedPosterior.beta`; any number of items, none included. A set
1502
+ is a value: `items`, `kind`, `parameters` and `views` are frozen and no
1503
+ reader (`rank`, `pick`, `Order#repin`, `summary`, `means`, `values`,
1504
+ `greedy`) writes to it, so one set may be built once and read from many
1505
+ requests and threads (the order and position odds keep a memo on a set its
1506
+ caller has not frozen; a frozen set answers alike).
1507
+ - Refuses: `items` that is not a Hash; a name that is not a String, is blank,
1508
+ is given twice, holds a NUL byte or an earlier name's bytes; a count below 0,
1509
+ not a number or beyond 1e100; a Hash of priors naming a name that is not an
1510
+ item or leaving one out; a pair not above 0 once the tally is added.
1511
+
1512
+ #### `Bandit::Posteriors.beta(pairs, views: nil)`
1513
+
1514
+ - Takes: `pairs`, a Hash of item name => `[alpha, beta]`, each a finite number
1515
+ above 0; `views` nil (every item 0) or `{ item => views >= 0 }`, an item it
1516
+ leaves out reading 0.
1517
+ - Answers: a `:beta` set holding the pairs as given (scaled in `parameters`
1518
+ only above `LARGEST_TOTAL`; `summary`'s means and intervals read them as
1519
+ given). `normal(pairs, views:)` is the same for `[mean, standard deviation]`.
1520
+ - Guarantees: as `yes_no`'s set: a value, frozen, read by no reader's writes.
1521
+ - Refuses: the names `yes_no` refuses; a parameter that is not a finite number
1522
+ above 0; views naming a name that is not an item, or below 0.
1523
+
1524
+ #### `Bandit::Posteriors.amounts(items, priors:)`
1525
+
1526
+ - Takes: tallies `{ views:, outcomes:, outcome_squares: }` (the outcomes the
1527
+ sum of amounts, any sign) under one `Prior::Amount` or one an item.
1528
+ - Answers: a `:normal` set of `[mean, standard deviation]` for the mean per
1529
+ view.
1530
+ - Refuses: as `yes_no`, and a prior that is not a `Prior::Amount` (there is no
1531
+ flat amounts prior).
1532
+
1533
+ #### `Bandit::Posteriors#parameters`
1534
+
1535
+ - Answers: `{ item => pair }`, frozen: the one pair every draw (`rank`, `pick`,
1536
+ `values`, from any source), every odds call, `summary`'s P(best) and loss and
1537
+ `greedy` read.
1538
+ - Guarantees: unchanged by every reader; a Beta pair above `LARGEST_TOTAL` is
1539
+ scaled to it with its mean kept, never summing above it.
1540
+
1541
+ #### `Bandit::Posteriors#rank(key: | uniforms: | rng:, shuffle: 0.0, explore: 0.0, may_lead: nil, min_views: 0.0, pinned: {}, top: nil)`
1542
+
1543
+ - Takes: exactly one source: `key:` (a String, or an Array of parts: nil, a
1544
+ String, or anything whose `to_s` answers a String, none holding a NUL byte),
1545
+ `uniforms:` or `rng:` (answering `rand` with Floats in [0, 1)); `shuffle` and
1546
+ `explore` in [0, 1] summing to at most 1; `may_lead` nil or a list of item
1547
+ names; `min_views` a number (one below 0 asks for none, as 0 does); `pinned`
1548
+ `{ Integer position => item }` (0-based); `top` nil or an Integer >= 1.
1549
+ - Answers: a frozen `Order`: `items` (the first `top`, pinned items at their
1550
+ positions), `leader` (the free head when it has `min_views` views, else nil),
1551
+ `branch` (`:shuffled`, `:explored` or `:thompson`) and `draws` (each free
1552
+ item's posterior value; none on the shuffled branch).
1553
+ - Guarantees:
1554
+ - A key draws the same Order from the same set in every process (*keyed*
1555
+ uniforms, `Draw.uniform_for(*key, "explore")`, `(*key, "priority", item)`
1556
+ and `(*key, "draw", item)`; the draws are their quantiles on `parameters`,
1557
+ *exact*).
1558
+ - An item's uniforms are the key's and its own name's alone: with no pin, an
1559
+ added item never changes the others' relative order, and an added item
1560
+ that `may_lead` does not name changes neither the others' order, the leader
1561
+ nor the branch. `may_lead`, `min_views`, `shuffle` and `explore` choose the
1562
+ head, the branch and the leader; the items after the head follow by draw
1563
+ (by priority on the shuffled branch).
1564
+ - From an RNG (*seeded*): exactly one branch value (when shuffle + explore is
1565
+ above 0), one priority per free item on the shuffled or explored branch,
1566
+ then one `Sampler` draw per free item where the branch draws, in item
1567
+ order, and nothing else from the stream.
1568
+ - Every source draws exactly from the posteriors, so the pick, order and
1569
+ position odds describe each of them. Ties go to the earlier item. Pinned
1570
+ items keep their positions and draw nothing. The set is not changed.
1571
+ - Refuses, before any draw and before anything is taken from an RNG: a rule
1572
+ outside its bounds, a pinned position or name that is not an item, a name
1573
+ pinned twice, a `may_lead` name that is not an item, `top` below 1, no source
1574
+ or two, a malformed `uniforms` Hash, a key part holding a NUL byte or whose
1575
+ `to_s` answers no String, and under a key or given uniforms a Beta parameter
1576
+ below 1e-6. An RNG answer that is not a Float in [0, 1), or a raise from its
1577
+ `rand`, is an `ArgumentError` at that answer.
1578
+
1579
+ #### `Bandit::Posteriors#pick(key: | uniforms: | rng:, shuffle: 0.0, explore: 0.0, may_lead: nil, min_views: 0.0)`
1580
+
1581
+ - Answers: a frozen `Choice`: `item` (the leader, nil when the head has fewer
1582
+ than `min_views` views), `head`, `branch` and `draws`; nil, nil and no draws
1583
+ for an empty set.
1584
+ - Guarantees: `rank`'s; under every source it is the head of the Order `rank`
1585
+ answers there, an RNG giving up exactly the values `rank` takes.
1586
+ - Refuses: what `rank` refuses.
1587
+
1588
+ #### `Bandit::Order#repin(pinned)`
1589
+
1590
+ - Takes: `{ Integer position => item }`.
1591
+ - Answers: the Order `rank` answers on the same set and the same key or given
1592
+ uniforms with these pins under the held rule (shuffle 0, explore 0, no
1593
+ `may_lead`, `min_views` 0, the order's own `top`), `==` and every draw.
1594
+ - Guarantees: an item the order drew keeps its draw and is not drawn again, so
1595
+ re-pinning an order's own head first draws nothing at all; the source is the
1596
+ one `rank` read, so a caller's object changed after `rank` moves no re-pin;
1597
+ neither the order nor its set changes.
1598
+ - Refuses: an order drawn from an RNG, one no posterior set made, and what
1599
+ `rank` refuses on that source.
1600
+
1601
+ #### `Bandit::Posteriors#summary(credible_mass: 0.95, among: nil)`
1602
+
1603
+ - Takes: `credible_mass` above 0 and below 1; `among` nil (every item), `[]`
1604
+ (none: means and intervals only, any number of items) or a list of at most
1605
+ `EXACT_PICK_MAX` (20) item names.
1606
+ - Answers: `{ item => { views:, mean:, lower:, upper:, p_best:, expected_loss: } }`
1607
+ in item order, `p_best` and `expected_loss` nil for an item `among` does not
1608
+ name; the shape `Allocator.from_summary` reads.
1609
+ - Guarantees (*exact*): no key, uniform or RNG is read, so the same set answers
1610
+ the same summary bit for bit on every call whatever the process's random
1611
+ state; an item's views, mean and interval are its own pair's alone (the pair
1612
+ as given, never the scaled one), so the item answers them alike in any set
1613
+ that holds it, a set of its own included; the interval's ends are
1614
+ `BetaFunction.quantile`'s at (1 - mass) / 2 and 1 less that (its accuracy
1615
+ below), or the Normal's mean -+ z sd; P(best) is within 1e-12 of
1616
+ `BetaArgmax.probabilities` (`NormalArgmax`'s for a Normal set) among the
1617
+ named items only, and the loss is never below 0.
1618
+ - Refuses: a `credible_mass` outside (0, 1); an `among` that is not a list,
1619
+ names a name that is not an item or names more than 20; a Beta parameter
1620
+ below 1e-6 (on the pair as given, on every path; on the scaled pair when
1621
+ `among` names an item).
1622
+
1623
+ #### `Bandit::Posteriors#means`
1624
+
1625
+ - Answers: `{ item => mean }`, frozen, in item order: `summary`'s `:mean`, bit
1626
+ for bit.
1627
+ - Guarantees (*exact*): no quantile, no draw and no randomness, for a set of
1628
+ any size.
1629
+ - Refuses: nothing a built set holds.
1630
+
1631
+ ### Priors (`Bandit::Prior`)
1632
+
1633
+ #### `Bandit::Prior.flat`
1634
+
1635
+ - Answers: `[1.0, 1.0]`, the same frozen Array on every call.
1636
+
1637
+ #### `Bandit::Prior.borrowed(group:, own: nil)`
1638
+
1639
+ - Takes: two yes/no tallies `{ views:, outcomes: }`; nil reads as no evidence.
1640
+ - Answers: `{ views:, outcomes: }`, the group less the item's own, views at
1641
+ least 0 and outcomes within [0, views].
1642
+ - Guarantees: the one leave-one-out rule `pooled` and `pooled_new` borrow by.
1643
+ - Refuses: a tally that is not a Hash or nil; a count below 0, not a number or
1644
+ beyond 1e100.
1645
+
1646
+ #### `Bandit::Prior.pooled(group:, own: nil, strength:, base: Prior::FLAT)`
1647
+
1648
+ - Takes: two tallies, a `strength` >= 0 in views, a `base` pair above 0.
1649
+ - Answers: `[alpha, beta]`: the borrowed mean (base_alpha + y) /
1650
+ (base_alpha + base_beta + n) held at base_alpha + base_beta +
1651
+ min(strength, n); the base itself when nothing is borrowed.
1652
+ - Guarantees: plain arithmetic, the mean formed exactly and rounded once, so
1653
+ the same inputs answer the same pair bit for bit everywhere.
1654
+ - Refuses: a `base` that is not a pair above 0, then a `strength` below 0 or
1655
+ not a number (`"strength must be >= 0 (got -1.0)"`), whatever the tallies
1656
+ hold, a nil group included; then what `borrowed` refuses.
1657
+
1658
+ #### `Bandit::Prior.pooled_new(others:, group: nil, own: nil, weight: 2.0)`
1659
+
1660
+ - Takes: `others`, a list of `{ group:, own: }`; the new item's own tallies; a
1661
+ `weight` >= 0.
1662
+ - Answers: `[alpha, beta]`: the others' borrowed evidence pooled into
1663
+ (1 + Y) / (2 + N) at `weight`, plus the new item's own borrowed evidence;
1664
+ Beta(1, 1) with no others and no group.
1665
+ - Refuses: a `weight` below 0, `others` that is not a list of Hashes, and what
1666
+ `borrowed` refuses of any tally.
1667
+
1668
+ ### Keyed randomness (`Bandit::Draw`, `Bandit::Decay`)
1669
+
1670
+ #### `Bandit::Draw.seed_for(*parts)`
1671
+
1672
+ - Takes: parts, each nil (read as ""), a String (its bytes) or anything whose
1673
+ `to_s` answers a String (so 7 and "7" are one part).
1674
+ - Answers: an Integer in [0, 2**64): the first 8 bytes of SHA256 of the parts'
1675
+ bytes joined by one NUL byte, big-endian, for `Random.new`.
1676
+ - Guarantees (*keyed*): the same Integer in every process, on every machine
1677
+ and every Ruby version (unlike `String#hash`, which Ruby seeds per process);
1678
+ two different part lists never join to the same text.
1679
+ - Refuses: a part holding a NUL byte, and a part whose `to_s` answers no
1680
+ String.
1681
+
1682
+ #### `Bandit::Draw.uniform_for(*parts)`
1683
+
1684
+ - Answers: `seed_for`'s top 53 bits over 2**53, a Float in [0, 1).
1685
+ - Guarantees and refusals: `seed_for`'s.
1686
+
1687
+ #### `Bandit::Draw.from_weights(weights, uniform)`
1688
+
1689
+ - Takes: a Hash of key => weight (keys any objects) and a uniform in [0, 1].
1690
+ - Answers: the first key, in input order, whose running sum of counted weights
1691
+ reaches uniform * total (the last counted key when rounding leaves the sum
1692
+ short); nil for an empty Hash.
1693
+ - Guarantees: pure; a weight that is not a finite number above 0 counts 0 and
1694
+ is never answered; with no positive weight every key counts 1; a uniform
1695
+ drawn uniformly answers a key with probability its counted weight over the
1696
+ total; keys are answered as given and never read.
1697
+ - Refuses: weights that are not a Hash, a uniform outside [0, 1], a weight
1698
+ beyond 1e100, and counted weights whose sum is not finite.
1699
+
1700
+ #### `Bandit::Decay.weight(age, half_life:, cap: nil)`
1701
+
1702
+ - Answers: 2 ** (-age / half_life); 0.0 above the cap (inclusive at it).
1703
+ - Guarantees (*exact*): no clock of its own; 1.0 at age 0, 0.5 at one half-life.
1704
+ - Refuses: a negative age, a half-life not above 0, a cap below 0.
1705
+
1706
+ #### `Bandit::Decay.fold(rows, half_life:, cap: nil)`
1707
+
1708
+ - Answers: `{ views:, outcomes:, outcome_squares: }` summed over rows, each
1709
+ row's sums at its age's weight; linear in the rows, so rows summed by age
1710
+ first fold alike.
1711
+ - Refuses: a row without an age or with a negative one, and what the tally
1712
+ reader refuses.
1713
+
1714
+ ### A/B allocation (`Bandit::Allocator`, `Bandit::WinHold`, `Bandit::WeightedPosterior`)
1715
+
1716
+ #### `Bandit::Allocator::Config.new(**settings)`
1717
+
1718
+ - Takes: any of `warmup_exposures`, `floor`, `max_step`, `win_p_best`,
1719
+ `win_loss`, `won_share`, `won_rest`, `result_changed_below`,
1720
+ `credible_mass`, `prior_alpha` and `prior_beta` by keyword; one left out is
1721
+ its `DEFAULTS` value.
1722
+ - Answers: a frozen value, `==` by its settings.
1723
+ - Guarantees: every setting is read by the decision it names, at every call
1724
+ handed this Config, and from nowhere else.
1725
+ - Refuses: a name that is not a setting, a value that is not a finite number, a
1726
+ floor outside (0, 1].
1727
+
1728
+ #### `Bandit::Allocator.call(arms:, tallies:, status:, winner_arm:, rng:, draws:, previous_weights: nil, hold_satisfied: true, config: Config.new, fixed_shares: nil, arm_floors: nil)`
1729
+
1730
+ - Takes: 1 to `config.max_arms` arms (Symbols or Strings); tallies
1731
+ `{ arm => { exposures:, conversions: } }` (Symbol or String keys, optionally
1732
+ the weighted sums and guardrails); status `"running"`, `"paused"` or `"won"`;
1733
+ an RNG; a whole number of joint draws.
1734
+ - Answers: `{ weights:, phase:, stats:, winner:, result_changed:, leader: }`,
1735
+ arms as Symbols.
1736
+ - Guarantees (*seeded*): pure, no clock and no state; the same inputs and the
1737
+ same RNG stream answer the same result, the joint draws taken arm by arm in
1738
+ the given order (`JointBeta`); the inputs are not changed; `hold_satisfied`
1739
+ moves only the crown (weights, phase, winner), never `stats`; every setting
1740
+ read is the `config`'s. The pinned fixtures answer alike on every Ruby the CI
1741
+ runs.
1742
+ - Refuses, before any draw: a config that is not a `Config`, no arms or more
1743
+ than `config.max_arms`, `draws` below 1 once read by `Integer()` (whose own
1744
+ error a value it cannot read raises), the per-arm share checks, and
1745
+ `allocation: :fixed` (`FixedAllocation`, an `ArgumentError`).
1746
+
1747
+ #### `Bandit::Allocator.won_weights(arms, winner, config: Config.new)`
1748
+
1749
+ - Answers: the winner `config.won_share`, every other arm
1750
+ `config.won_rest / (arms.size - 1)`, keyed by the arms as given; a single arm
1751
+ 1.0.
1752
+ - Guarantees: pure arithmetic; the config's two settings are the ones read.
1753
+ - Refuses: nothing of its own: a winner that is not one of the arms is not
1754
+ checked (every arm then gets its `won_rest` share), and a config that is not
1755
+ a `Config` raises `NoMethodError`.
1756
+
1757
+ #### `Bandit::Allocator.lift(weights, config: Config.new)`
1758
+
1759
+ - Answers: every arm at least `config.floor`, the others scaled in proportion
1760
+ into what the pinned floors leave, keys as given.
1761
+ - Guarantees: 0.2.0's private `Allocator#with_floor`, step for step, in public.
1762
+ `with_floor` itself stays private and carries no promise: call `lift`.
1763
+ - Refuses: weights that are not a Hash of 1 to `max_arms` arms, a config that
1764
+ is not a `Config`, a share that is not a finite number.
1765
+
1766
+ #### `Bandit::WinHold.new(leading_arm:, leading_since:, now:, last_pass_at:, hold_seconds: 86_400, max_gap_seconds: 1_200)`
1767
+
1768
+ - Takes: any arm (a blank one reads as none, any other is kept as a String);
1769
+ clock values that are Times, numbers of seconds or anything whose difference
1770
+ compares with the two lengths; `leading_since` and `last_pass_at` may be nil.
1771
+ - Guarantees: pure, no clock of its own; the two lengths given are the two
1772
+ read.
1773
+ - Refuses: nothing of its own: leaving out a required keyword (`last_pass_at`
1774
+ included) is Ruby's `ArgumentError`, and a clock value that cannot be
1775
+ subtracted raises its own error when the hold is first asked.
1776
+
1777
+ #### `Bandit::WinHold#satisfied?`
1778
+
1779
+ - Answers: true from exactly `hold_seconds` after `leading_since`; false with
1780
+ no leader, no `leading_since`, or a run interrupted (no previous pass, or one
1781
+ more than `max_gap_seconds` ago; a gap of exactly `max_gap_seconds` is
1782
+ unbroken).
1783
+
1784
+ #### `Bandit::WinHold#satisfied_for?(arm)`
1785
+
1786
+ - Answers: `satisfied?` and `arm`, by its String form, is the arm that held.
1787
+
1788
+ #### `Bandit::WinHold#after_pass(leader)`
1789
+
1790
+ - Answers: `{ leading_arm:, leading_since: }`: unchanged while the same arm
1791
+ leads an unbroken run; the leader from `now` for another arm, after a gap or
1792
+ without `leading_since`; both nil with no leader.
1793
+
1794
+ #### `Bandit::WeightedPosterior.count(tally)`
1795
+
1796
+ - Takes: a Hash with Symbol or String keys, or nil (nobody seen); anything
1797
+ else raises `NoMethodError`.
1798
+ - Answers: `{ exposures:, conversions:, weighted_exposures:,
1799
+ weighted_conversions:, weighted_exposures_squared: }`, the weighted sums kept
1800
+ only when both are given and the weighted exposures are above 0, the squared
1801
+ sum only beside them and above 0.
1802
+ - Guarantees: 0.2.0's reading, kept as it was: the counts by `Kernel#Integer`
1803
+ (a Float truncates toward 0, a numeric String reads as its number), the
1804
+ weighted sums by `to_f`.
1805
+ - Refuses: what `Integer()` cannot read raises `Integer()`'s own error: an
1806
+ `ArgumentError` for a String that is no number, a `TypeError` for an Array or
1807
+ another object, a `FloatDomainError` for NaN or an infinity.
1808
+
1809
+ #### `Bandit::WeightedPosterior.beta(count, prior_alpha: 1.0, prior_beta: 1.0)`
1810
+
1811
+ - Answers: `[prior_alpha + c, prior_beta + n - c]`, n `effective_exposures`, c
1812
+ the conversions (or the weighted conversions rescaled to n) clamped to
1813
+ [0, n].
1814
+ - Guarantees: plain arithmetic; `Posteriors.yes_no` reads a tally by the same
1815
+ rule.
1816
+ - Refuses: a negative exposure count, as an `ArgumentError`.
1817
+
1818
+ #### `Bandit::WeightedPosterior.effective_exposures(count)`
1819
+
1820
+ - Answers: the raw count (an Integer) without a squared sum, else the Kish n
1821
+ (sum w)^2 / sum w^2, never above the raw count.
1822
+
1823
+ ### Joint draws, distribution functions and tests
1824
+
1825
+ #### `JointBeta.each_draw(posteriors, draws:, rng:)`
1826
+
1827
+ - Takes: a list of `[alpha, beta]` pairs, each above 0; an Integer >= 0 of
1828
+ draws; an RNG.
1829
+ - Answers: one Array per draw (one Float per arm, in order), yielded, or an
1830
+ Enumerator of them without a block.
1831
+ - Guarantees (*seeded*): the same pairs, draws and stream give the same rows;
1832
+ per draw, arm by arm in the given order, one `Sampler#beta` each, so adding,
1833
+ dropping or reordering an arm moves every later draw.
1834
+ - Refuses: `draws` that is not an Integer >= 0 (before anything is drawn), and
1835
+ a parameter that is not above 0 at its first draw (`Sampler`'s
1836
+ `"shape must be > 0"`; with 0 draws nothing is read).
1837
+
1838
+ #### `JointBeta.summary(posteriors, draws:, rng:)`
1839
+
1840
+ - Answers: `{ samples:, wins:, losses: }` over the same rows as `each_draw`, a
1841
+ tie's win split equally between the tied arms.
1842
+ - Guarantees and refusals: `each_draw`'s.
1843
+
1844
+ #### `BetaFunction.regularized(x, alpha, beta)`
1845
+
1846
+ - Answers: I_x(alpha, beta), the Beta distribution function; 0.0 for x <= 0
1847
+ and 1.0 for x >= 1.
1848
+ - Guarantees (*exact*): for parameters up to a few hundred, within 5e-12
1849
+ relative of an exact binomial reference. For parameters from 1 000 to
1850
+ 50 000 each (n = alpha + beta), within 4 n ln n machine epsilons, relative
1851
+ (about 1e-9 at n = 100 000), where the identities I_x(a, b) + I_(1-x)(b, a) = 1
1852
+ and I_(1/2)(a, a) = 1/2 hold to the same bound. Beyond a total of about
1853
+ 3 * 10**8 the continued fraction's fixed bound binds (see `summary`).
1854
+ - Refuses: alpha or beta whose `to_f` is not a finite number above 0, and an x
1855
+ whose `to_f` is NaN (each read by `to_f`, as `Gamma` reads its arguments).
1856
+
1857
+ #### `BetaFunction.quantile(probability, alpha, beta)`
1858
+
1859
+ - Answers: the x where I_x(alpha, beta) = probability, by bisection to a
1860
+ Float's spacing; 0.0 at and below 0, 1.0 at and above 1.
1861
+ - Guarantees (*exact*): `regularized(quantile(p))` is p within `regularized`'s
1862
+ bound; at parameters from 1 000 to 50 000 the 2.5 % and 97.5 % quantiles lie
1863
+ within 0.001 sd of the normal approximation with its first skewness term.
1864
+ - Refuses: as `regularized`, for the probability.
1865
+
1866
+ #### `BetaFunction.shortest_interval(mass, alpha, beta)`
1867
+
1868
+ - Takes: `mass`, a finite real number strictly between 0 and 1 (a String is
1869
+ not one); `alpha` and `beta`, read by `to_f` as `regularized` reads them.
1870
+ - Answers: `[lower, upper]`, two Floats with 0 <= lower <= upper <= 1: with one
1871
+ peak inside (alpha, beta > 1) the highest-density interval, its two ends of
1872
+ one density; [0, quantile(mass)] for a density that falls from 0
1873
+ (alpha <= 1 <= beta, not both 1) and [quantile(1 - mass), 1] for one that
1874
+ rises to 1; [(1 - mass) / 2, (1 + mass) / 2] for the flat density; for a
1875
+ U-shaped one (alpha, beta < 1) the shorter of the two one-sided intervals,
1876
+ the lower on a tie.
1877
+ - Guarantees (*exact*, by bisection, no draw; the mass goes through
1878
+ `regularized`): the interval holds `mass` within `regularized`'s accuracy,
1879
+ short of it by a Float's last bits at most; with one peak it is no wider than
1880
+ the equal-tailed interval of the mass, or than any interval of a 400-point
1881
+ grid; for alpha, beta >= 1 (not both 1) it holds the mode, so under the flat
1882
+ prior, Beta(1 + c, 1 + n - c), it holds the observed share c / n, at c = 0 and
1883
+ c = n too, where the equal-tailed interval leaves it out, and for fractional
1884
+ counts.
1885
+ - Refuses: a mass that is not a finite real number strictly between 0 and 1,
1886
+ then an alpha or beta whose `to_f` is not a finite number above 0.
1887
+
1888
+ #### `BetaFunction.mean_intervals(items, mass:)`
1889
+
1890
+ - Takes: `items`, a Hash of key => `{ visits:, mean:, standard_error: }`
1891
+ (Symbol or String field names, the keys any objects): the visits a mean of
1892
+ non-negative values is over, its mean per visit and its standard error, each
1893
+ a finite real number >= 0 where given, visits fractional or whole; `mass` as
1894
+ `shortest_interval` takes it.
1895
+ - Answers: `{ key => [lower, upper] }` in the items' order. With n the visits,
1896
+ m the mean, s the standard error and q = n s^2 + m^2: q / m times the shortest
1897
+ interval of Beta(1 + k, 1 + n - k), k = n m^2 / q (never above n); for a mean
1898
+ of 0, the set's value of one event (the summed n q over the summed n m of the
1899
+ items above 0) times [0, quantile(mass) of Beta(1, 1 + n)]. nil for an item
1900
+ with a figure missing (nil: its other figures are then not read) or no
1901
+ visits, and for a mean of 0 in a set with no item above 0.
1902
+ - Guarantees (*exact*): every mean lies inside its interval and no end is below
1903
+ 0; an item with a mean above 0 answers from its own figures alone, the same
1904
+ in any set; a mean of 0's interval narrows as its visits grow; a value of 0
1905
+ or 1 per visit gives the share's own interval (`shortest_interval` of
1906
+ Beta(1 + c, 1 + n - c), within 1e-9); in the normal limit the interval is the
1907
+ normal one (its width within 1 % at about 400 events), and with a handful of
1908
+ events it is skewed above the mean.
1909
+ - Refuses: a mass `shortest_interval` refuses; then items that are not a Hash
1910
+ of Hashes, and a visit count, mean or standard error, given beside the other
1911
+ two, that is not a finite real number or is below 0, before any interval is
1912
+ worked out; and, as the item is worked out, an item whose arithmetic leaves a
1913
+ Float's range (a value of one event, its own or the set's, that is not a
1914
+ finite number above 0; visits past about 2**53 that all hold one value),
1915
+ naming the item.
1916
+
1917
+ #### `Gamma.regularized_lower(a, x)`
1918
+
1919
+ - Answers: P(a, x), the Gamma(a, 1) distribution function; 0.0 for x <= 0.
1920
+ - Guarantees (*exact*): within 5e-12 relative of the exact Poisson sums for
1921
+ whole a up to 200, on both sides of x = a + 1; P + Q = 1.
1922
+ - Refuses: an a whose `to_f` is not a finite number above 0, and an x whose
1923
+ `to_f` is NaN (both are read by `to_f`, so a numeric String reads as its
1924
+ number, a non-numeric one as 0.0, and an object without `to_f` raises
1925
+ `NoMethodError`).
1926
+
1927
+ #### `Gamma.regularized_upper(a, x)`
1928
+
1929
+ - Answers: Q(a, x) = 1 - P(a, x); 1.0 for x <= 0.
1930
+ - Guarantees and refusals: `regularized_lower`'s.
1931
+
1932
+ #### `Significance.normal_quantile(p)`
1933
+
1934
+ - Answers: the z with Phi(z) = p; -Infinity at 0 and Infinity at 1.
1935
+ - Guarantees (*exact*): z maps back to p within 1e-12 relative, from p = 1e-300
1936
+ to 1 - 1e-6.
1937
+ - Refuses: a p outside [0, 1], NaN included.
1938
+
1939
+ #### `Significance.chi_squared_survival(statistic, degrees_of_freedom)`
1940
+
1941
+ - Answers: P(X >= statistic) for X ~ chi-squared(degrees_of_freedom), Q(dof / 2,
1942
+ statistic / 2); 1.0 for a statistic of 0 or below.
1943
+ - Refuses: degrees of freedom not above 0 (for a statistic above 0).
1944
+
1945
+ #### `Significance.sample_ratio_p_value(observed:, expected:, minimum_total: 200)`
1946
+
1947
+ - Takes: Hashes of arm => count and arm => expected count (Symbol or String
1948
+ keys, as `observed`'s); `minimum_total`, read as given above or below its
1949
+ default.
1950
+ - Answers: Pearson's chi-squared p on k - 1 degrees of freedom over the arms
1951
+ expected to get traffic; nil under `minimum_total` or with fewer than two
1952
+ such arms; 0.0 when an arm expected to get nothing got some.
1953
+ - Guarantees (*exact*): `chi_squared_survival`'s accuracy.
1954
+ - Refuses: nothing of its own (a non-Hash raises `NoMethodError`).
1955
+
1956
+ ### Estimates of several parts (`SharedCredit`)
1957
+
1958
+ #### `SharedCredit.with_prior(estimate, strength:)`
1959
+
1960
+ - Takes: an estimate in `SharedCredit.estimate`'s shape (an `Estimate`, or a
1961
+ plain Hash such as one read back from JSON), read by Symbol or String, with
1962
+ its weight sums and Kish's sizes; `strength`, a finite real number >= 0, in
1963
+ visits (0 tops nothing up).
1964
+ - Answers: an `Estimate` with the same arms in the same order, the parts (none
1965
+ for a plain Hash), the weight sums and Kish's sizes (the evidence's own),
1966
+ the means and covariance under the prior pooled from its own credited arms
1967
+ (the rule under
1968
+ [An arm of a handful of visits](#an-arm-of-a-handful-of-visits-with_prior)),
1969
+ and its rounding: the given one's (0.0 for a plain Hash) or, where larger,
1970
+ that of an entry it changed.
1971
+ - Guarantees (*exact*: plain arithmetic in a fixed order, no draw and no `Math`
1972
+ function, so the same Floats everywhere): the rule, against a reference in
1973
+ Rationals that reads the rows, from rows and from per-arm sums alike; the
1974
+ covariance exactly symmetric wherever the given one is (as every estimate
1975
+ `estimate` answers is), and taken by `LinearValue` at the answer's rounding;
1976
+ an arm at or above the strength, and an arm with no credit, keeps its means
1977
+ and covariance block bit for bit; with every credited arm at or above it the
1978
+ answer holds the given means, covariance and rounding, so every seeded
1979
+ `LinearValue` draw and summary is the one the estimate gives; in an ordinary
1980
+ test (no row credits two arms) an arm at or above the strength draws the same
1981
+ values beside an arm topped up; a part every credited arm holds at one mean
1982
+ keeps that mean exactly; the given estimate is not changed.
1983
+ Measured at a strength of 20: a version of one visit reads P(best) between
1984
+ 0.41 and 0.77, and no version of three visits or fewer reaches 0.95.
1985
+ - Refuses: a strength below 0 or not a finite real number; an estimate that is
1986
+ not a Hash, or whose means, covariance, weight sums or Kish's sizes are
1987
+ missing or malformed; an arm with means but a Kish's size of 0, each before
1988
+ anything is computed; and, as it is computed, an estimate whose figures leave
1989
+ a Float's range under the prior ("the sums leave a Float's range").
1990
+
1330
1991
  ## Bandit simulations: measured results
1331
1992
 
1332
1993
  `spec/simulations` holds ten files, tagged `simulation: true` and run only