skiftet_statistical 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +125 -1
- data/README.md +665 -4
- data/lib/skiftet_statistical/bandit/allocator.rb +23 -0
- data/lib/skiftet_statistical/bandit/decay.rb +3 -0
- data/lib/skiftet_statistical/bandit/draw.rb +16 -4
- data/lib/skiftet_statistical/bandit/posteriors.rb +29 -0
- data/lib/skiftet_statistical/bandit/prior.rb +16 -1
- data/lib/skiftet_statistical/bandit/ranking.rb +32 -0
- data/lib/skiftet_statistical/bandit/summary.rb +11 -0
- data/lib/skiftet_statistical/bandit/weighted_posterior.rb +7 -0
- data/lib/skiftet_statistical/bandit/win_hold.rb +9 -0
- data/lib/skiftet_statistical/beta_function.rb +261 -3
- data/lib/skiftet_statistical/gamma.rb +7 -0
- data/lib/skiftet_statistical/joint_beta.rb +6 -0
- data/lib/skiftet_statistical/shared_credit.rb +191 -0
- data/lib/skiftet_statistical/significance.rb +12 -1
- data/lib/skiftet_statistical/version.rb +1 -1
- metadata +1 -1
data/README.md
CHANGED
|
@@ -11,7 +11,9 @@ Modules:
|
|
|
11
11
|
t-test, exact normal CDF and two-tailed p-values, the normal quantile, the
|
|
12
12
|
chi-squared survival and the sample-ratio check.
|
|
13
13
|
- **`Gamma`** / **`BetaFunction`** — the regularized incomplete gamma and beta
|
|
14
|
-
functions
|
|
14
|
+
functions and the Beta quantile; since 0.5.0 also the shortest
|
|
15
|
+
(highest-density) Beta interval, and intervals of means of non-negative
|
|
16
|
+
values that always hold the mean.
|
|
15
17
|
- **`Sampler`** — Gamma/Beta/Gaussian random sampling, RNG-injectable.
|
|
16
18
|
- **`JointBeta`** — joint Monte Carlo draws over Beta posteriors (P(best),
|
|
17
19
|
expected loss).
|
|
@@ -36,7 +38,9 @@ Modules:
|
|
|
36
38
|
normal) and their weights are uncertain.
|
|
37
39
|
- **`SharedCredit`** — Hájek inverse-propensity means of several parts per arm,
|
|
38
40
|
with one covariance over every (arm, part) pair, when one row may credit
|
|
39
|
-
several arms; its sums fold once and store as JSON.
|
|
41
|
+
several arms; its sums fold once and store as JSON. Since 0.5.0 also the
|
|
42
|
+
estimate under a prior pooled from its own arms, for arms of a handful of
|
|
43
|
+
visits.
|
|
40
44
|
- **`HierarchicalRates`** — Gamma-Poisson rates on a tree: a node's ratio to
|
|
41
45
|
what its parent leads us to expect, the fit of a level's strength (and of a
|
|
42
46
|
two-groups hot share), the leaf fit, and checks beside them.
|
|
@@ -53,12 +57,16 @@ Modules:
|
|
|
53
57
|
- **`Cusum`** — a one-sided change test: its statistic, its run length, the
|
|
54
58
|
threshold for a stated false-alarm rate and the smallest change it can see.
|
|
55
59
|
|
|
60
|
+
The methods an application calls, each with what it takes, answers, guarantees
|
|
61
|
+
and refuses, are under [Interface contract](#interface-contract); what a
|
|
62
|
+
version number promises about seeded answers is under [Stability](#stability).
|
|
63
|
+
|
|
56
64
|
## Install
|
|
57
65
|
|
|
58
66
|
In a Gemfile (published on rubygems.org):
|
|
59
67
|
|
|
60
68
|
```ruby
|
|
61
|
-
gem "skiftet_statistical", "~> 0.
|
|
69
|
+
gem "skiftet_statistical", "~> 0.5.0"
|
|
62
70
|
```
|
|
63
71
|
|
|
64
72
|
Or build/install locally:
|
|
@@ -69,6 +77,24 @@ bundle install
|
|
|
69
77
|
gem build skiftet_statistical.gemspec
|
|
70
78
|
```
|
|
71
79
|
|
|
80
|
+
## Stability
|
|
81
|
+
|
|
82
|
+
The version follows [semantic versioning](https://semver.org), with one rule
|
|
83
|
+
made explicit for a statistics library: **a seeded answer is part of the
|
|
84
|
+
public API.** A release that moves an answer for the same inputs, the same
|
|
85
|
+
key and the same RNG seed (a draw, an order, a pick, a pinned figure, a
|
|
86
|
+
p-value beyond its stated tolerance) is a major or a minor version, as its
|
|
87
|
+
CHANGELOG entry states (before 1.0, as semver allows, a minor one), and that
|
|
88
|
+
entry says which answers move and why. It is never a patch: a patch release
|
|
89
|
+
fixes documentation, adds examples, or turns a crash into the refusal the
|
|
90
|
+
documentation states, and moves no answer. Adding a method, or a keyword whose
|
|
91
|
+
default keeps every old answer, is a minor version.
|
|
92
|
+
|
|
93
|
+
The pins in `spec/pins` are literal values recorded when each behaviour was
|
|
94
|
+
built and never edited: a change that fails one has moved a public answer, and
|
|
95
|
+
ships only as the version above. The methods an application calls, with what
|
|
96
|
+
each guarantees, are listed under [Interface contract](#interface-contract).
|
|
97
|
+
|
|
72
98
|
## Descriptive statistics
|
|
73
99
|
|
|
74
100
|
```ruby
|
|
@@ -134,6 +160,7 @@ G.regularized_upper(a, x) # Q(a, x) = 1 - P(a, x)
|
|
|
134
160
|
B = SkiftetStatistical::BetaFunction
|
|
135
161
|
B.regularized(x, alpha, beta) # I_x(alpha, beta), the Beta(alpha, beta) distribution function
|
|
136
162
|
B.quantile(0.975, 41.0, 361.0) # its inverse: an exact 95 % credible bound
|
|
163
|
+
B.shortest_interval(0.95, 1.0, 2.0) # => [0.0, 0.7763...]: the shortest 95 % interval, holding the mode
|
|
137
164
|
```
|
|
138
165
|
|
|
139
166
|
The arithmetic and its order are part of the API: an app that pins these
|
|
@@ -141,6 +168,42 @@ outputs (at 1e-12, say) compares them with this exact sequence of
|
|
|
141
168
|
operations, so a reordered formula is a breaking change even when it is "the
|
|
142
169
|
same" formula.
|
|
143
170
|
|
|
171
|
+
### Intervals that hold their figure
|
|
172
|
+
|
|
173
|
+
An equal-tailed interval of a share's flat-prior posterior, Beta(1 + c, 1 + n - c),
|
|
174
|
+
leaves the observed share c / n out when c is 0 or n: 0 of 1 reads 1.3-84.2 %.
|
|
175
|
+
`shortest_interval` is the highest-density interval wherever the density has one
|
|
176
|
+
peak or is monotone, so it holds the mode, which under the flat prior is the share:
|
|
177
|
+
|
|
178
|
+
```ruby
|
|
179
|
+
B.shortest_interval(0.95, 1.0, 2.0) # => [0.0, 0.776...] (0 of 1 visit: 0 % inside)
|
|
180
|
+
B.shortest_interval(0.95, 41.0, 261.0) # => [0.0981, 0.1748] (40 of 300; equal tails: 0.0996-0.1765)
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
`mean_intervals` does the same for means of non-negative values (a value per
|
|
184
|
+
visit that is never below 0), each read as a share of its visits: with the
|
|
185
|
+
visits' second moment q = n * s^2 + m^2 (s the mean's standard error), one event
|
|
186
|
+
is worth q / m and the item holds n * m^2 / q events; its interval is that worth
|
|
187
|
+
times the shortest interval of the events' flat-prior Beta. The mean always lies
|
|
188
|
+
inside, the interval never goes below 0, a large item's interval is the normal
|
|
189
|
+
one, and a value of 0 or 1 per visit gives the share's own interval. An item
|
|
190
|
+
with a mean of 0 takes the set's value of one event, so it too has an interval:
|
|
191
|
+
|
|
192
|
+
```ruby
|
|
193
|
+
B.mean_intervals({ "a" => { visits: 342, mean: 0.0058, standard_error: 0.0041 },
|
|
194
|
+
"b" => { visits: 300, mean: 0.0, standard_error: 0.0 } }, mass: 0.95)
|
|
195
|
+
# => { "a" => [0.0009, 0.0184], "b" => [0.0, 0.0099] }
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Both are exact (no draw) and work by bisection: an interval with one peak
|
|
199
|
+
inside cost 0.7 to 2.0 ms of CPU with YJIT (Beta(3, 341) to
|
|
200
|
+
Beta(40 001, 1 160 001)), a one-sided one 0.06 ms, and `mean_intervals` of ten
|
|
201
|
+
items 9.9 ms, measured on one busy development machine, so a caller with many
|
|
202
|
+
items works them out once a pass rather than once a visit. An item whose
|
|
203
|
+
figures leave a Float's range in this arithmetic is refused, naming the item,
|
|
204
|
+
rather than answered with an infinite or empty interval; what each method
|
|
205
|
+
takes, answers and refuses is under [Interface contract](#interface-contract).
|
|
206
|
+
|
|
144
207
|
## Joint Beta draws
|
|
145
208
|
|
|
146
209
|
For Bayesian A/B readouts and Thompson-sampling allocators: draw every arm's
|
|
@@ -933,7 +996,9 @@ weight), a symmetric positive semidefinite covariance, finite numbers, a weight
|
|
|
933
996
|
for every part. `draws:` is an Integer >= 0 (`summary` needs at least 1).
|
|
934
997
|
`rounding:` (0.0 by default) is how far each covariance entry may be off: hand
|
|
935
998
|
it a `SharedCredit` estimate's `#rounding`, so a covariance summed from rows is
|
|
936
|
-
never refused for its own rounding.
|
|
999
|
+
never refused for its own rounding. An arm of one visit, or of visits that all
|
|
1000
|
+
hold one value, has no variance and is drawn as a point; where arms may hold a
|
|
1001
|
+
handful of visits, draw from `SharedCredit.with_prior`'s estimate (below).
|
|
937
1002
|
|
|
938
1003
|
Measured (`value_allocation_simulation_spec.rb`, below): the 95 % intervals of
|
|
939
1004
|
each arm's value and of each difference covered the truth in 377 to 385 of 400
|
|
@@ -1049,6 +1114,64 @@ credit, two options of identical value read a mean P(best) of 0.58 and 0.42
|
|
|
1049
1114
|
0.37; in the pooled cells, `contrast_for_fit` left 1,135 of 24,000 cell
|
|
1050
1115
|
contrasts out of the fit.
|
|
1051
1116
|
|
|
1117
|
+
### An arm of a handful of visits (`with_prior`)
|
|
1118
|
+
|
|
1119
|
+
The sandwich of an arm of one visit, or of visits that all hold one value, is
|
|
1120
|
+
exactly 0, so `LinearValue` draws that arm as a point: one visit at 1 against
|
|
1121
|
+
three at 0, 1 and 0 reads P(best) 0.994, and one visit at 0 against them 0.114,
|
|
1122
|
+
by the one visit's value alone. `with_prior(estimate, strength:)` answers the
|
|
1123
|
+
estimate under a prior pooled from its own credited arms, every arm's evidence
|
|
1124
|
+
topped up to `strength` visits, for drawing:
|
|
1125
|
+
|
|
1126
|
+
```ruby
|
|
1127
|
+
estimate = SC.estimate(
|
|
1128
|
+
[ { credit: { a: 1.0 }, values: [ 1.0 ] },
|
|
1129
|
+
{ credit: { b: 1.0 }, values: [ 0.0 ] }, { credit: { b: 1.0 }, values: [ 1.0 ] },
|
|
1130
|
+
{ credit: { b: 1.0 }, values: [ 0.0 ] } ],
|
|
1131
|
+
arms: %i[a b], parts: %i[converted],
|
|
1132
|
+
)
|
|
1133
|
+
topped = SC.with_prior(estimate, strength: 20)
|
|
1134
|
+
# => { means: { a: [0.525], b: [0.475] }, covariance: [[0.01246875..., 0.0], [0.0, 0.01246875]],
|
|
1135
|
+
# weight_sums: { a: 1.0, b: 3.0 }, effective: { a: 1.0, b: 3.0 } }
|
|
1136
|
+
LV.summary(arms: %i[a b], parts: %i[converted], means: topped[:means], covariance: topped[:covariance],
|
|
1137
|
+
parameters: { w: { fixed: 1.0 } }, weights: ->(drawn) { { converted: drawn.fetch(:w) } },
|
|
1138
|
+
draws: 20_000, rng: Random.new(7), rounding: topped.rounding)
|
|
1139
|
+
# => P(best) a 0.61965, b 0.38035 (0.994 and 0.006 on the estimate itself)
|
|
1140
|
+
```
|
|
1141
|
+
|
|
1142
|
+
The prior's mean per part is the credited arms' mean weighted by their Kish's
|
|
1143
|
+
sizes n_a, m0 = sum_a n_a * r_a / N; its per-visit covariance is
|
|
1144
|
+
V0 = sum_a n_a * (n_a * cov(r_a) + g_a * g_a^T) / N with g_a = r_a - m0, each
|
|
1145
|
+
arm's per-visit covariance and its distance from m0, so a spread between the
|
|
1146
|
+
arms widens it. An arm of Kish's size n below the strength k takes e = k - n
|
|
1147
|
+
visits of it, the Normal mean's rule taken jointly over its parts: means
|
|
1148
|
+
(e * m0 + n * r_a) / k, covariance block (n / k)^2 * cov(r_a) +
|
|
1149
|
+
(e * V0 + (e * n / k) * g_a * g_a^T) / k^2, and every covariance with another
|
|
1150
|
+
arm scaled by n / k, as its means are.
|
|
1151
|
+
|
|
1152
|
+
**Where visits are plenty nothing moves.** An arm at or above the strength,
|
|
1153
|
+
and an arm with no credit, keeps its means and its own covariance block bit for
|
|
1154
|
+
bit; where every credited arm is at or above it, the answer holds the given
|
|
1155
|
+
means and covariance, so every seeded draw `LinearValue` takes from it is the
|
|
1156
|
+
draw it takes from the estimate. The answer keeps the arms, the parts, the
|
|
1157
|
+
weight sums and Kish's sizes (the evidence's own: hand `contrast_for_fit` the
|
|
1158
|
+
given estimate, not this one); its rounding is the given one's or, where
|
|
1159
|
+
larger, that of an entry it changed. A strength of 0 tops nothing up. Refused
|
|
1160
|
+
(`ArgumentError`): a strength below 0 or not a finite number, an estimate
|
|
1161
|
+
without its weight sums or Kish's sizes, an arm with means but a Kish's size
|
|
1162
|
+
of 0, and an estimate whose figures rise past a Float's largest value under the
|
|
1163
|
+
prior (figures so small that the prior's products fall below Float::MIN read
|
|
1164
|
+
as 0; no estimate `estimate` answers holds them).
|
|
1165
|
+
|
|
1166
|
+
Measured (`shared_credit_prior_spec.rb`), at a strength of 20: a version of
|
|
1167
|
+
one visit read P(best) between 0.41 and 0.77 against versions of 1 to 100
|
|
1168
|
+
visits, over values of one part, of two parts at uncertain prices and of
|
|
1169
|
+
amounts, where the estimate itself read 0.98 or more, or 0.02 or less, in 12 of
|
|
1170
|
+
the table's 16 rows; no version of three visits or fewer reached 0.95. Two
|
|
1171
|
+
versions of one true value, one of 1 to 3 visits and the other of 3 to 50, read
|
|
1172
|
+
decided (a P(best) at or above 0.95) in none of 250 seeded datasets, against
|
|
1173
|
+
158 on the estimate itself.
|
|
1174
|
+
|
|
1052
1175
|
## Hierarchical Poisson rates (`HierarchicalRates`)
|
|
1053
1176
|
|
|
1054
1177
|
How far a node's rate stands from its parent's, from its count s and the count
|
|
@@ -1327,6 +1450,544 @@ Measured (`change_test_simulation_spec.rb`, below): a stated one false alarm in
|
|
|
1327
1450
|
detectable size was caught within the horizon in 62 to 66 % of runs (bound
|
|
1328
1451
|
half).
|
|
1329
1452
|
|
|
1453
|
+
## Interface contract
|
|
1454
|
+
|
|
1455
|
+
The methods an application calls, each with what it takes, what it answers,
|
|
1456
|
+
what holds of the answer and what it refuses, so a caller can rely on the
|
|
1457
|
+
method without pinning its outputs in its own suite. An app tests that it hands
|
|
1458
|
+
these methods the right inputs and uses their answers; the library tests that
|
|
1459
|
+
the answers are right. Every guarantee below is backed by a library example:
|
|
1460
|
+
`spec/interface_contract_spec.rb` indexes them, method by method, and fails
|
|
1461
|
+
when a cited example is gone or a method below is no longer public. If a
|
|
1462
|
+
guarantee you need is missing here, ask for it to be stated and tested in the
|
|
1463
|
+
library rather than pinning it in your app.
|
|
1464
|
+
|
|
1465
|
+
Conventions:
|
|
1466
|
+
|
|
1467
|
+
- **Refused** means an `ArgumentError` raised before anything is computed or
|
|
1468
|
+
taken from an RNG, unless the entry says otherwise.
|
|
1469
|
+
- **Numbers** are read by the library's one number reader where the entry says
|
|
1470
|
+
so: a finite real number of magnitude at most 1e100 (`Numbers::LARGEST`); a
|
|
1471
|
+
String, a numeric one included, is not a number.
|
|
1472
|
+
- **Determinism** comes in three kinds, and each entry says which:
|
|
1473
|
+
- *keyed*: SHA256 and Integer arithmetic only, so the same parts give the
|
|
1474
|
+
same answer in every process, on every machine and on every Ruby version;
|
|
1475
|
+
- *seeded*: the same RNG stream gives the same answer, the library taking
|
|
1476
|
+
values from it in a stated order (`Random.new(seed)` gives the same stream
|
|
1477
|
+
on every Ruby the CI runs, 3.1 to 3.4);
|
|
1478
|
+
- *exact*: no randomness at all; the same inputs give the same Float on one
|
|
1479
|
+
platform, and where the computation goes through `Math.lgamma` another math
|
|
1480
|
+
library can move the last bits (the pins compare such figures at 1e-9
|
|
1481
|
+
relative, and orders, heads, branches and counts exactly, on every Ruby the
|
|
1482
|
+
CI runs).
|
|
1483
|
+
|
|
1484
|
+
### Posterior sets (`Bandit::Posteriors`)
|
|
1485
|
+
|
|
1486
|
+
#### `Bandit::Posteriors.yes_no(items, priors: nil)`
|
|
1487
|
+
|
|
1488
|
+
- Takes: `items`, a Hash of item name => tally, the names non-blank Strings in
|
|
1489
|
+
the tie order; a tally `{ views:, outcomes: }` (Symbol or String keys,
|
|
1490
|
+
numbers >= 0, fractional allowed; optionally `weighted_views`,
|
|
1491
|
+
`weighted_outcomes` and `weighted_views_squared`); `priors` nil (Beta(1, 1)),
|
|
1492
|
+
one pair `[alpha, beta]` (each >= 0) or `{ item => pair }` naming every item.
|
|
1493
|
+
- Answers: a set, `kind` `:beta`, `parameters` `{ item => [alpha + c, beta + n - c] }`
|
|
1494
|
+
with c the outcomes clamped to [0, views] (or the weighted outcomes rescaled
|
|
1495
|
+
to the Kish size, never above the views), a pair whose total is above
|
|
1496
|
+
`LARGEST_TOTAL` (1e6) scaled to it with its mean kept; `views` the tallies'
|
|
1497
|
+
views as Floats.
|
|
1498
|
+
- Guarantees: plain arithmetic, so the same tallies build the same pairs bit
|
|
1499
|
+
for bit everywhere; an item's pair is its own tally's and prior's alone; a
|
|
1500
|
+
set built from another set's `parameters` keeps them bit for bit; the same
|
|
1501
|
+
rule as `WeightedPosterior.beta`; any number of items, none included. A set
|
|
1502
|
+
is a value: `items`, `kind`, `parameters` and `views` are frozen and no
|
|
1503
|
+
reader (`rank`, `pick`, `Order#repin`, `summary`, `means`, `values`,
|
|
1504
|
+
`greedy`) writes to it, so one set may be built once and read from many
|
|
1505
|
+
requests and threads (the order and position odds keep a memo on a set its
|
|
1506
|
+
caller has not frozen; a frozen set answers alike).
|
|
1507
|
+
- Refuses: `items` that is not a Hash; a name that is not a String, is blank,
|
|
1508
|
+
is given twice, holds a NUL byte or an earlier name's bytes; a count below 0,
|
|
1509
|
+
not a number or beyond 1e100; a Hash of priors naming a name that is not an
|
|
1510
|
+
item or leaving one out; a pair not above 0 once the tally is added.
|
|
1511
|
+
|
|
1512
|
+
#### `Bandit::Posteriors.beta(pairs, views: nil)`
|
|
1513
|
+
|
|
1514
|
+
- Takes: `pairs`, a Hash of item name => `[alpha, beta]`, each a finite number
|
|
1515
|
+
above 0; `views` nil (every item 0) or `{ item => views >= 0 }`, an item it
|
|
1516
|
+
leaves out reading 0.
|
|
1517
|
+
- Answers: a `:beta` set holding the pairs as given (scaled in `parameters`
|
|
1518
|
+
only above `LARGEST_TOTAL`; `summary`'s means and intervals read them as
|
|
1519
|
+
given). `normal(pairs, views:)` is the same for `[mean, standard deviation]`.
|
|
1520
|
+
- Guarantees: as `yes_no`'s set: a value, frozen, read by no reader's writes.
|
|
1521
|
+
- Refuses: the names `yes_no` refuses; a parameter that is not a finite number
|
|
1522
|
+
above 0; views naming a name that is not an item, or below 0.
|
|
1523
|
+
|
|
1524
|
+
#### `Bandit::Posteriors.amounts(items, priors:)`
|
|
1525
|
+
|
|
1526
|
+
- Takes: tallies `{ views:, outcomes:, outcome_squares: }` (the outcomes the
|
|
1527
|
+
sum of amounts, any sign) under one `Prior::Amount` or one an item.
|
|
1528
|
+
- Answers: a `:normal` set of `[mean, standard deviation]` for the mean per
|
|
1529
|
+
view.
|
|
1530
|
+
- Refuses: as `yes_no`, and a prior that is not a `Prior::Amount` (there is no
|
|
1531
|
+
flat amounts prior).
|
|
1532
|
+
|
|
1533
|
+
#### `Bandit::Posteriors#parameters`
|
|
1534
|
+
|
|
1535
|
+
- Answers: `{ item => pair }`, frozen: the one pair every draw (`rank`, `pick`,
|
|
1536
|
+
`values`, from any source), every odds call, `summary`'s P(best) and loss and
|
|
1537
|
+
`greedy` read.
|
|
1538
|
+
- Guarantees: unchanged by every reader; a Beta pair above `LARGEST_TOTAL` is
|
|
1539
|
+
scaled to it with its mean kept, never summing above it.
|
|
1540
|
+
|
|
1541
|
+
#### `Bandit::Posteriors#rank(key: | uniforms: | rng:, shuffle: 0.0, explore: 0.0, may_lead: nil, min_views: 0.0, pinned: {}, top: nil)`
|
|
1542
|
+
|
|
1543
|
+
- Takes: exactly one source: `key:` (a String, or an Array of parts: nil, a
|
|
1544
|
+
String, or anything whose `to_s` answers a String, none holding a NUL byte),
|
|
1545
|
+
`uniforms:` or `rng:` (answering `rand` with Floats in [0, 1)); `shuffle` and
|
|
1546
|
+
`explore` in [0, 1] summing to at most 1; `may_lead` nil or a list of item
|
|
1547
|
+
names; `min_views` a number (one below 0 asks for none, as 0 does); `pinned`
|
|
1548
|
+
`{ Integer position => item }` (0-based); `top` nil or an Integer >= 1.
|
|
1549
|
+
- Answers: a frozen `Order`: `items` (the first `top`, pinned items at their
|
|
1550
|
+
positions), `leader` (the free head when it has `min_views` views, else nil),
|
|
1551
|
+
`branch` (`:shuffled`, `:explored` or `:thompson`) and `draws` (each free
|
|
1552
|
+
item's posterior value; none on the shuffled branch).
|
|
1553
|
+
- Guarantees:
|
|
1554
|
+
- A key draws the same Order from the same set in every process (*keyed*
|
|
1555
|
+
uniforms, `Draw.uniform_for(*key, "explore")`, `(*key, "priority", item)`
|
|
1556
|
+
and `(*key, "draw", item)`; the draws are their quantiles on `parameters`,
|
|
1557
|
+
*exact*).
|
|
1558
|
+
- An item's uniforms are the key's and its own name's alone: with no pin, an
|
|
1559
|
+
added item never changes the others' relative order, and an added item
|
|
1560
|
+
that `may_lead` does not name changes neither the others' order, the leader
|
|
1561
|
+
nor the branch. `may_lead`, `min_views`, `shuffle` and `explore` choose the
|
|
1562
|
+
head, the branch and the leader; the items after the head follow by draw
|
|
1563
|
+
(by priority on the shuffled branch).
|
|
1564
|
+
- From an RNG (*seeded*): exactly one branch value (when shuffle + explore is
|
|
1565
|
+
above 0), one priority per free item on the shuffled or explored branch,
|
|
1566
|
+
then one `Sampler` draw per free item where the branch draws, in item
|
|
1567
|
+
order, and nothing else from the stream.
|
|
1568
|
+
- Every source draws exactly from the posteriors, so the pick, order and
|
|
1569
|
+
position odds describe each of them. Ties go to the earlier item. Pinned
|
|
1570
|
+
items keep their positions and draw nothing. The set is not changed.
|
|
1571
|
+
- Refuses, before any draw and before anything is taken from an RNG: a rule
|
|
1572
|
+
outside its bounds, a pinned position or name that is not an item, a name
|
|
1573
|
+
pinned twice, a `may_lead` name that is not an item, `top` below 1, no source
|
|
1574
|
+
or two, a malformed `uniforms` Hash, a key part holding a NUL byte or whose
|
|
1575
|
+
`to_s` answers no String, and under a key or given uniforms a Beta parameter
|
|
1576
|
+
below 1e-6. An RNG answer that is not a Float in [0, 1), or a raise from its
|
|
1577
|
+
`rand`, is an `ArgumentError` at that answer.
|
|
1578
|
+
|
|
1579
|
+
#### `Bandit::Posteriors#pick(key: | uniforms: | rng:, shuffle: 0.0, explore: 0.0, may_lead: nil, min_views: 0.0)`
|
|
1580
|
+
|
|
1581
|
+
- Answers: a frozen `Choice`: `item` (the leader, nil when the head has fewer
|
|
1582
|
+
than `min_views` views), `head`, `branch` and `draws`; nil, nil and no draws
|
|
1583
|
+
for an empty set.
|
|
1584
|
+
- Guarantees: `rank`'s; under every source it is the head of the Order `rank`
|
|
1585
|
+
answers there, an RNG giving up exactly the values `rank` takes.
|
|
1586
|
+
- Refuses: what `rank` refuses.
|
|
1587
|
+
|
|
1588
|
+
#### `Bandit::Order#repin(pinned)`
|
|
1589
|
+
|
|
1590
|
+
- Takes: `{ Integer position => item }`.
|
|
1591
|
+
- Answers: the Order `rank` answers on the same set and the same key or given
|
|
1592
|
+
uniforms with these pins under the held rule (shuffle 0, explore 0, no
|
|
1593
|
+
`may_lead`, `min_views` 0, the order's own `top`), `==` and every draw.
|
|
1594
|
+
- Guarantees: an item the order drew keeps its draw and is not drawn again, so
|
|
1595
|
+
re-pinning an order's own head first draws nothing at all; the source is the
|
|
1596
|
+
one `rank` read, so a caller's object changed after `rank` moves no re-pin;
|
|
1597
|
+
neither the order nor its set changes.
|
|
1598
|
+
- Refuses: an order drawn from an RNG, one no posterior set made, and what
|
|
1599
|
+
`rank` refuses on that source.
|
|
1600
|
+
|
|
1601
|
+
#### `Bandit::Posteriors#summary(credible_mass: 0.95, among: nil)`
|
|
1602
|
+
|
|
1603
|
+
- Takes: `credible_mass` above 0 and below 1; `among` nil (every item), `[]`
|
|
1604
|
+
(none: means and intervals only, any number of items) or a list of at most
|
|
1605
|
+
`EXACT_PICK_MAX` (20) item names.
|
|
1606
|
+
- Answers: `{ item => { views:, mean:, lower:, upper:, p_best:, expected_loss: } }`
|
|
1607
|
+
in item order, `p_best` and `expected_loss` nil for an item `among` does not
|
|
1608
|
+
name; the shape `Allocator.from_summary` reads.
|
|
1609
|
+
- Guarantees (*exact*): no key, uniform or RNG is read, so the same set answers
|
|
1610
|
+
the same summary bit for bit on every call whatever the process's random
|
|
1611
|
+
state; an item's views, mean and interval are its own pair's alone (the pair
|
|
1612
|
+
as given, never the scaled one), so the item answers them alike in any set
|
|
1613
|
+
that holds it, a set of its own included; the interval's ends are
|
|
1614
|
+
`BetaFunction.quantile`'s at (1 - mass) / 2 and 1 less that (its accuracy
|
|
1615
|
+
below), or the Normal's mean -+ z sd; P(best) is within 1e-12 of
|
|
1616
|
+
`BetaArgmax.probabilities` (`NormalArgmax`'s for a Normal set) among the
|
|
1617
|
+
named items only, and the loss is never below 0.
|
|
1618
|
+
- Refuses: a `credible_mass` outside (0, 1); an `among` that is not a list,
|
|
1619
|
+
names a name that is not an item or names more than 20; a Beta parameter
|
|
1620
|
+
below 1e-6 (on the pair as given, on every path; on the scaled pair when
|
|
1621
|
+
`among` names an item).
|
|
1622
|
+
|
|
1623
|
+
#### `Bandit::Posteriors#means`
|
|
1624
|
+
|
|
1625
|
+
- Answers: `{ item => mean }`, frozen, in item order: `summary`'s `:mean`, bit
|
|
1626
|
+
for bit.
|
|
1627
|
+
- Guarantees (*exact*): no quantile, no draw and no randomness, for a set of
|
|
1628
|
+
any size.
|
|
1629
|
+
- Refuses: nothing a built set holds.
|
|
1630
|
+
|
|
1631
|
+
### Priors (`Bandit::Prior`)
|
|
1632
|
+
|
|
1633
|
+
#### `Bandit::Prior.flat`
|
|
1634
|
+
|
|
1635
|
+
- Answers: `[1.0, 1.0]`, the same frozen Array on every call.
|
|
1636
|
+
|
|
1637
|
+
#### `Bandit::Prior.borrowed(group:, own: nil)`
|
|
1638
|
+
|
|
1639
|
+
- Takes: two yes/no tallies `{ views:, outcomes: }`; nil reads as no evidence.
|
|
1640
|
+
- Answers: `{ views:, outcomes: }`, the group less the item's own, views at
|
|
1641
|
+
least 0 and outcomes within [0, views].
|
|
1642
|
+
- Guarantees: the one leave-one-out rule `pooled` and `pooled_new` borrow by.
|
|
1643
|
+
- Refuses: a tally that is not a Hash or nil; a count below 0, not a number or
|
|
1644
|
+
beyond 1e100.
|
|
1645
|
+
|
|
1646
|
+
#### `Bandit::Prior.pooled(group:, own: nil, strength:, base: Prior::FLAT)`
|
|
1647
|
+
|
|
1648
|
+
- Takes: two tallies, a `strength` >= 0 in views, a `base` pair above 0.
|
|
1649
|
+
- Answers: `[alpha, beta]`: the borrowed mean (base_alpha + y) /
|
|
1650
|
+
(base_alpha + base_beta + n) held at base_alpha + base_beta +
|
|
1651
|
+
min(strength, n); the base itself when nothing is borrowed.
|
|
1652
|
+
- Guarantees: plain arithmetic, the mean formed exactly and rounded once, so
|
|
1653
|
+
the same inputs answer the same pair bit for bit everywhere.
|
|
1654
|
+
- Refuses: a `base` that is not a pair above 0, then a `strength` below 0 or
|
|
1655
|
+
not a number (`"strength must be >= 0 (got -1.0)"`), whatever the tallies
|
|
1656
|
+
hold, a nil group included; then what `borrowed` refuses.
|
|
1657
|
+
|
|
1658
|
+
#### `Bandit::Prior.pooled_new(others:, group: nil, own: nil, weight: 2.0)`
|
|
1659
|
+
|
|
1660
|
+
- Takes: `others`, a list of `{ group:, own: }`; the new item's own tallies; a
|
|
1661
|
+
`weight` >= 0.
|
|
1662
|
+
- Answers: `[alpha, beta]`: the others' borrowed evidence pooled into
|
|
1663
|
+
(1 + Y) / (2 + N) at `weight`, plus the new item's own borrowed evidence;
|
|
1664
|
+
Beta(1, 1) with no others and no group.
|
|
1665
|
+
- Refuses: a `weight` below 0, `others` that is not a list of Hashes, and what
|
|
1666
|
+
`borrowed` refuses of any tally.
|
|
1667
|
+
|
|
1668
|
+
### Keyed randomness (`Bandit::Draw`, `Bandit::Decay`)
|
|
1669
|
+
|
|
1670
|
+
#### `Bandit::Draw.seed_for(*parts)`
|
|
1671
|
+
|
|
1672
|
+
- Takes: parts, each nil (read as ""), a String (its bytes) or anything whose
|
|
1673
|
+
`to_s` answers a String (so 7 and "7" are one part).
|
|
1674
|
+
- Answers: an Integer in [0, 2**64): the first 8 bytes of SHA256 of the parts'
|
|
1675
|
+
bytes joined by one NUL byte, big-endian, for `Random.new`.
|
|
1676
|
+
- Guarantees (*keyed*): the same Integer in every process, on every machine
|
|
1677
|
+
and every Ruby version (unlike `String#hash`, which Ruby seeds per process);
|
|
1678
|
+
two different part lists never join to the same text.
|
|
1679
|
+
- Refuses: a part holding a NUL byte, and a part whose `to_s` answers no
|
|
1680
|
+
String.
|
|
1681
|
+
|
|
1682
|
+
#### `Bandit::Draw.uniform_for(*parts)`
|
|
1683
|
+
|
|
1684
|
+
- Answers: `seed_for`'s top 53 bits over 2**53, a Float in [0, 1).
|
|
1685
|
+
- Guarantees and refusals: `seed_for`'s.
|
|
1686
|
+
|
|
1687
|
+
#### `Bandit::Draw.from_weights(weights, uniform)`
|
|
1688
|
+
|
|
1689
|
+
- Takes: a Hash of key => weight (keys any objects) and a uniform in [0, 1].
|
|
1690
|
+
- Answers: the first key, in input order, whose running sum of counted weights
|
|
1691
|
+
reaches uniform * total (the last counted key when rounding leaves the sum
|
|
1692
|
+
short); nil for an empty Hash.
|
|
1693
|
+
- Guarantees: pure; a weight that is not a finite number above 0 counts 0 and
|
|
1694
|
+
is never answered; with no positive weight every key counts 1; a uniform
|
|
1695
|
+
drawn uniformly answers a key with probability its counted weight over the
|
|
1696
|
+
total; keys are answered as given and never read.
|
|
1697
|
+
- Refuses: weights that are not a Hash, a uniform outside [0, 1], a weight
|
|
1698
|
+
beyond 1e100, and counted weights whose sum is not finite.
|
|
1699
|
+
|
|
1700
|
+
#### `Bandit::Decay.weight(age, half_life:, cap: nil)`
|
|
1701
|
+
|
|
1702
|
+
- Answers: 2 ** (-age / half_life); 0.0 above the cap (inclusive at it).
|
|
1703
|
+
- Guarantees (*exact*): no clock of its own; 1.0 at age 0, 0.5 at one half-life.
|
|
1704
|
+
- Refuses: a negative age, a half-life not above 0, a cap below 0.
|
|
1705
|
+
|
|
1706
|
+
#### `Bandit::Decay.fold(rows, half_life:, cap: nil)`
|
|
1707
|
+
|
|
1708
|
+
- Answers: `{ views:, outcomes:, outcome_squares: }` summed over rows, each
|
|
1709
|
+
row's sums at its age's weight; linear in the rows, so rows summed by age
|
|
1710
|
+
first fold alike.
|
|
1711
|
+
- Refuses: a row without an age or with a negative one, and what the tally
|
|
1712
|
+
reader refuses.
|
|
1713
|
+
|
|
1714
|
+
### A/B allocation (`Bandit::Allocator`, `Bandit::WinHold`, `Bandit::WeightedPosterior`)
|
|
1715
|
+
|
|
1716
|
+
#### `Bandit::Allocator::Config.new(**settings)`
|
|
1717
|
+
|
|
1718
|
+
- Takes: any of `warmup_exposures`, `floor`, `max_step`, `win_p_best`,
|
|
1719
|
+
`win_loss`, `won_share`, `won_rest`, `result_changed_below`,
|
|
1720
|
+
`credible_mass`, `prior_alpha` and `prior_beta` by keyword; one left out is
|
|
1721
|
+
its `DEFAULTS` value.
|
|
1722
|
+
- Answers: a frozen value, `==` by its settings.
|
|
1723
|
+
- Guarantees: every setting is read by the decision it names, at every call
|
|
1724
|
+
handed this Config, and from nowhere else.
|
|
1725
|
+
- Refuses: a name that is not a setting, a value that is not a finite number, a
|
|
1726
|
+
floor outside (0, 1].
|
|
1727
|
+
|
|
1728
|
+
#### `Bandit::Allocator.call(arms:, tallies:, status:, winner_arm:, rng:, draws:, previous_weights: nil, hold_satisfied: true, config: Config.new, fixed_shares: nil, arm_floors: nil)`
|
|
1729
|
+
|
|
1730
|
+
- Takes: 1 to `config.max_arms` arms (Symbols or Strings); tallies
|
|
1731
|
+
`{ arm => { exposures:, conversions: } }` (Symbol or String keys, optionally
|
|
1732
|
+
the weighted sums and guardrails); status `"running"`, `"paused"` or `"won"`;
|
|
1733
|
+
an RNG; a whole number of joint draws.
|
|
1734
|
+
- Answers: `{ weights:, phase:, stats:, winner:, result_changed:, leader: }`,
|
|
1735
|
+
arms as Symbols.
|
|
1736
|
+
- Guarantees (*seeded*): pure, no clock and no state; the same inputs and the
|
|
1737
|
+
same RNG stream answer the same result, the joint draws taken arm by arm in
|
|
1738
|
+
the given order (`JointBeta`); the inputs are not changed; `hold_satisfied`
|
|
1739
|
+
moves only the crown (weights, phase, winner), never `stats`; every setting
|
|
1740
|
+
read is the `config`'s. The pinned fixtures answer alike on every Ruby the CI
|
|
1741
|
+
runs.
|
|
1742
|
+
- Refuses, before any draw: a config that is not a `Config`, no arms or more
|
|
1743
|
+
than `config.max_arms`, `draws` below 1 once read by `Integer()` (whose own
|
|
1744
|
+
error a value it cannot read raises), the per-arm share checks, and
|
|
1745
|
+
`allocation: :fixed` (`FixedAllocation`, an `ArgumentError`).
|
|
1746
|
+
|
|
1747
|
+
#### `Bandit::Allocator.won_weights(arms, winner, config: Config.new)`
|
|
1748
|
+
|
|
1749
|
+
- Answers: the winner `config.won_share`, every other arm
|
|
1750
|
+
`config.won_rest / (arms.size - 1)`, keyed by the arms as given; a single arm
|
|
1751
|
+
1.0.
|
|
1752
|
+
- Guarantees: pure arithmetic; the config's two settings are the ones read.
|
|
1753
|
+
- Refuses: nothing of its own: a winner that is not one of the arms is not
|
|
1754
|
+
checked (every arm then gets its `won_rest` share), and a config that is not
|
|
1755
|
+
a `Config` raises `NoMethodError`.
|
|
1756
|
+
|
|
1757
|
+
#### `Bandit::Allocator.lift(weights, config: Config.new)`
|
|
1758
|
+
|
|
1759
|
+
- Answers: every arm at least `config.floor`, the others scaled in proportion
|
|
1760
|
+
into what the pinned floors leave, keys as given.
|
|
1761
|
+
- Guarantees: 0.2.0's private `Allocator#with_floor`, step for step, in public.
|
|
1762
|
+
`with_floor` itself stays private and carries no promise: call `lift`.
|
|
1763
|
+
- Refuses: weights that are not a Hash of 1 to `max_arms` arms, a config that
|
|
1764
|
+
is not a `Config`, a share that is not a finite number.
|
|
1765
|
+
|
|
1766
|
+
#### `Bandit::WinHold.new(leading_arm:, leading_since:, now:, last_pass_at:, hold_seconds: 86_400, max_gap_seconds: 1_200)`
|
|
1767
|
+
|
|
1768
|
+
- Takes: any arm (a blank one reads as none, any other is kept as a String);
|
|
1769
|
+
clock values that are Times, numbers of seconds or anything whose difference
|
|
1770
|
+
compares with the two lengths; `leading_since` and `last_pass_at` may be nil.
|
|
1771
|
+
- Guarantees: pure, no clock of its own; the two lengths given are the two
|
|
1772
|
+
read.
|
|
1773
|
+
- Refuses: nothing of its own: leaving out a required keyword (`last_pass_at`
|
|
1774
|
+
included) is Ruby's `ArgumentError`, and a clock value that cannot be
|
|
1775
|
+
subtracted raises its own error when the hold is first asked.
|
|
1776
|
+
|
|
1777
|
+
#### `Bandit::WinHold#satisfied?`
|
|
1778
|
+
|
|
1779
|
+
- Answers: true from exactly `hold_seconds` after `leading_since`; false with
|
|
1780
|
+
no leader, no `leading_since`, or a run interrupted (no previous pass, or one
|
|
1781
|
+
more than `max_gap_seconds` ago; a gap of exactly `max_gap_seconds` is
|
|
1782
|
+
unbroken).
|
|
1783
|
+
|
|
1784
|
+
#### `Bandit::WinHold#satisfied_for?(arm)`
|
|
1785
|
+
|
|
1786
|
+
- Answers: `satisfied?` and `arm`, by its String form, is the arm that held.
|
|
1787
|
+
|
|
1788
|
+
#### `Bandit::WinHold#after_pass(leader)`
|
|
1789
|
+
|
|
1790
|
+
- Answers: `{ leading_arm:, leading_since: }`: unchanged while the same arm
|
|
1791
|
+
leads an unbroken run; the leader from `now` for another arm, after a gap or
|
|
1792
|
+
without `leading_since`; both nil with no leader.
|
|
1793
|
+
|
|
1794
|
+
#### `Bandit::WeightedPosterior.count(tally)`
|
|
1795
|
+
|
|
1796
|
+
- Takes: a Hash with Symbol or String keys, or nil (nobody seen); anything
|
|
1797
|
+
else raises `NoMethodError`.
|
|
1798
|
+
- Answers: `{ exposures:, conversions:, weighted_exposures:,
|
|
1799
|
+
weighted_conversions:, weighted_exposures_squared: }`, the weighted sums kept
|
|
1800
|
+
only when both are given and the weighted exposures are above 0, the squared
|
|
1801
|
+
sum only beside them and above 0.
|
|
1802
|
+
- Guarantees: 0.2.0's reading, kept as it was: the counts by `Kernel#Integer`
|
|
1803
|
+
(a Float truncates toward 0, a numeric String reads as its number), the
|
|
1804
|
+
weighted sums by `to_f`.
|
|
1805
|
+
- Refuses: what `Integer()` cannot read raises `Integer()`'s own error: an
|
|
1806
|
+
`ArgumentError` for a String that is no number, a `TypeError` for an Array or
|
|
1807
|
+
another object, a `FloatDomainError` for NaN or an infinity.
|
|
1808
|
+
|
|
1809
|
+
#### `Bandit::WeightedPosterior.beta(count, prior_alpha: 1.0, prior_beta: 1.0)`
|
|
1810
|
+
|
|
1811
|
+
- Answers: `[prior_alpha + c, prior_beta + n - c]`, n `effective_exposures`, c
|
|
1812
|
+
the conversions (or the weighted conversions rescaled to n) clamped to
|
|
1813
|
+
[0, n].
|
|
1814
|
+
- Guarantees: plain arithmetic; `Posteriors.yes_no` reads a tally by the same
|
|
1815
|
+
rule.
|
|
1816
|
+
- Refuses: a negative exposure count, as an `ArgumentError`.
|
|
1817
|
+
|
|
1818
|
+
#### `Bandit::WeightedPosterior.effective_exposures(count)`
|
|
1819
|
+
|
|
1820
|
+
- Answers: the raw count (an Integer) without a squared sum, else the Kish n
|
|
1821
|
+
(sum w)^2 / sum w^2, never above the raw count.
|
|
1822
|
+
|
|
1823
|
+
### Joint draws, distribution functions and tests
|
|
1824
|
+
|
|
1825
|
+
#### `JointBeta.each_draw(posteriors, draws:, rng:)`
|
|
1826
|
+
|
|
1827
|
+
- Takes: a list of `[alpha, beta]` pairs, each above 0; an Integer >= 0 of
|
|
1828
|
+
draws; an RNG.
|
|
1829
|
+
- Answers: one Array per draw (one Float per arm, in order), yielded, or an
|
|
1830
|
+
Enumerator of them without a block.
|
|
1831
|
+
- Guarantees (*seeded*): the same pairs, draws and stream give the same rows;
|
|
1832
|
+
per draw, arm by arm in the given order, one `Sampler#beta` each, so adding,
|
|
1833
|
+
dropping or reordering an arm moves every later draw.
|
|
1834
|
+
- Refuses: `draws` that is not an Integer >= 0 (before anything is drawn), and
|
|
1835
|
+
a parameter that is not above 0 at its first draw (`Sampler`'s
|
|
1836
|
+
`"shape must be > 0"`; with 0 draws nothing is read).
|
|
1837
|
+
|
|
1838
|
+
#### `JointBeta.summary(posteriors, draws:, rng:)`
|
|
1839
|
+
|
|
1840
|
+
- Answers: `{ samples:, wins:, losses: }` over the same rows as `each_draw`, a
|
|
1841
|
+
tie's win split equally between the tied arms.
|
|
1842
|
+
- Guarantees and refusals: `each_draw`'s.
|
|
1843
|
+
|
|
1844
|
+
#### `BetaFunction.regularized(x, alpha, beta)`
|
|
1845
|
+
|
|
1846
|
+
- Answers: I_x(alpha, beta), the Beta distribution function; 0.0 for x <= 0
|
|
1847
|
+
and 1.0 for x >= 1.
|
|
1848
|
+
- Guarantees (*exact*): for parameters up to a few hundred, within 5e-12
|
|
1849
|
+
relative of an exact binomial reference. For parameters from 1 000 to
|
|
1850
|
+
50 000 each (n = alpha + beta), within 4 n ln n machine epsilons, relative
|
|
1851
|
+
(about 1e-9 at n = 100 000), where the identities I_x(a, b) + I_(1-x)(b, a) = 1
|
|
1852
|
+
and I_(1/2)(a, a) = 1/2 hold to the same bound. Beyond a total of about
|
|
1853
|
+
3 * 10**8 the continued fraction's fixed bound binds (see `summary`).
|
|
1854
|
+
- Refuses: alpha or beta whose `to_f` is not a finite number above 0, and an x
|
|
1855
|
+
whose `to_f` is NaN (each read by `to_f`, as `Gamma` reads its arguments).
|
|
1856
|
+
|
|
1857
|
+
#### `BetaFunction.quantile(probability, alpha, beta)`
|
|
1858
|
+
|
|
1859
|
+
- Answers: the x where I_x(alpha, beta) = probability, by bisection to a
|
|
1860
|
+
Float's spacing; 0.0 at and below 0, 1.0 at and above 1.
|
|
1861
|
+
- Guarantees (*exact*): `regularized(quantile(p))` is p within `regularized`'s
|
|
1862
|
+
bound; at parameters from 1 000 to 50 000 the 2.5 % and 97.5 % quantiles lie
|
|
1863
|
+
within 0.001 sd of the normal approximation with its first skewness term.
|
|
1864
|
+
- Refuses: as `regularized`, for the probability.
|
|
1865
|
+
|
|
1866
|
+
#### `BetaFunction.shortest_interval(mass, alpha, beta)`
|
|
1867
|
+
|
|
1868
|
+
- Takes: `mass`, a finite real number strictly between 0 and 1 (a String is
|
|
1869
|
+
not one); `alpha` and `beta`, read by `to_f` as `regularized` reads them.
|
|
1870
|
+
- Answers: `[lower, upper]`, two Floats with 0 <= lower <= upper <= 1: with one
|
|
1871
|
+
peak inside (alpha, beta > 1) the highest-density interval, its two ends of
|
|
1872
|
+
one density; [0, quantile(mass)] for a density that falls from 0
|
|
1873
|
+
(alpha <= 1 <= beta, not both 1) and [quantile(1 - mass), 1] for one that
|
|
1874
|
+
rises to 1; [(1 - mass) / 2, (1 + mass) / 2] for the flat density; for a
|
|
1875
|
+
U-shaped one (alpha, beta < 1) the shorter of the two one-sided intervals,
|
|
1876
|
+
the lower on a tie.
|
|
1877
|
+
- Guarantees (*exact*, by bisection, no draw; the mass goes through
|
|
1878
|
+
`regularized`): the interval holds `mass` within `regularized`'s accuracy,
|
|
1879
|
+
short of it by a Float's last bits at most; with one peak it is no wider than
|
|
1880
|
+
the equal-tailed interval of the mass, or than any interval of a 400-point
|
|
1881
|
+
grid; for alpha, beta >= 1 (not both 1) it holds the mode, so under the flat
|
|
1882
|
+
prior, Beta(1 + c, 1 + n - c), it holds the observed share c / n, at c = 0 and
|
|
1883
|
+
c = n too, where the equal-tailed interval leaves it out, and for fractional
|
|
1884
|
+
counts.
|
|
1885
|
+
- Refuses: a mass that is not a finite real number strictly between 0 and 1,
|
|
1886
|
+
then an alpha or beta whose `to_f` is not a finite number above 0.
|
|
1887
|
+
|
|
1888
|
+
#### `BetaFunction.mean_intervals(items, mass:)`
|
|
1889
|
+
|
|
1890
|
+
- Takes: `items`, a Hash of key => `{ visits:, mean:, standard_error: }`
|
|
1891
|
+
(Symbol or String field names, the keys any objects): the visits a mean of
|
|
1892
|
+
non-negative values is over, its mean per visit and its standard error, each
|
|
1893
|
+
a finite real number >= 0 where given, visits fractional or whole; `mass` as
|
|
1894
|
+
`shortest_interval` takes it.
|
|
1895
|
+
- Answers: `{ key => [lower, upper] }` in the items' order. With n the visits,
|
|
1896
|
+
m the mean, s the standard error and q = n s^2 + m^2: q / m times the shortest
|
|
1897
|
+
interval of Beta(1 + k, 1 + n - k), k = n m^2 / q (never above n); for a mean
|
|
1898
|
+
of 0, the set's value of one event (the summed n q over the summed n m of the
|
|
1899
|
+
items above 0) times [0, quantile(mass) of Beta(1, 1 + n)]. nil for an item
|
|
1900
|
+
with a figure missing (nil: its other figures are then not read) or no
|
|
1901
|
+
visits, and for a mean of 0 in a set with no item above 0.
|
|
1902
|
+
- Guarantees (*exact*): every mean lies inside its interval and no end is below
|
|
1903
|
+
0; an item with a mean above 0 answers from its own figures alone, the same
|
|
1904
|
+
in any set; a mean of 0's interval narrows as its visits grow; a value of 0
|
|
1905
|
+
or 1 per visit gives the share's own interval (`shortest_interval` of
|
|
1906
|
+
Beta(1 + c, 1 + n - c), within 1e-9); in the normal limit the interval is the
|
|
1907
|
+
normal one (its width within 1 % at about 400 events), and with a handful of
|
|
1908
|
+
events it is skewed above the mean.
|
|
1909
|
+
- Refuses: a mass `shortest_interval` refuses; then items that are not a Hash
|
|
1910
|
+
of Hashes, and a visit count, mean or standard error, given beside the other
|
|
1911
|
+
two, that is not a finite real number or is below 0, before any interval is
|
|
1912
|
+
worked out; and, as the item is worked out, an item whose arithmetic leaves a
|
|
1913
|
+
Float's range (a value of one event, its own or the set's, that is not a
|
|
1914
|
+
finite number above 0; visits past about 2**53 that all hold one value),
|
|
1915
|
+
naming the item.
|
|
1916
|
+
|
|
1917
|
+
#### `Gamma.regularized_lower(a, x)`
|
|
1918
|
+
|
|
1919
|
+
- Answers: P(a, x), the Gamma(a, 1) distribution function; 0.0 for x <= 0.
|
|
1920
|
+
- Guarantees (*exact*): within 5e-12 relative of the exact Poisson sums for
|
|
1921
|
+
whole a up to 200, on both sides of x = a + 1; P + Q = 1.
|
|
1922
|
+
- Refuses: an a whose `to_f` is not a finite number above 0, and an x whose
|
|
1923
|
+
`to_f` is NaN (both are read by `to_f`, so a numeric String reads as its
|
|
1924
|
+
number, a non-numeric one as 0.0, and an object without `to_f` raises
|
|
1925
|
+
`NoMethodError`).
|
|
1926
|
+
|
|
1927
|
+
#### `Gamma.regularized_upper(a, x)`
|
|
1928
|
+
|
|
1929
|
+
- Answers: Q(a, x) = 1 - P(a, x); 1.0 for x <= 0.
|
|
1930
|
+
- Guarantees and refusals: `regularized_lower`'s.
|
|
1931
|
+
|
|
1932
|
+
#### `Significance.normal_quantile(p)`
|
|
1933
|
+
|
|
1934
|
+
- Answers: the z with Phi(z) = p; -Infinity at 0 and Infinity at 1.
|
|
1935
|
+
- Guarantees (*exact*): z maps back to p within 1e-12 relative, from p = 1e-300
|
|
1936
|
+
to 1 - 1e-6.
|
|
1937
|
+
- Refuses: a p outside [0, 1], NaN included.
|
|
1938
|
+
|
|
1939
|
+
#### `Significance.chi_squared_survival(statistic, degrees_of_freedom)`
|
|
1940
|
+
|
|
1941
|
+
- Answers: P(X >= statistic) for X ~ chi-squared(degrees_of_freedom), Q(dof / 2,
|
|
1942
|
+
statistic / 2); 1.0 for a statistic of 0 or below.
|
|
1943
|
+
- Refuses: degrees of freedom not above 0 (for a statistic above 0).
|
|
1944
|
+
|
|
1945
|
+
#### `Significance.sample_ratio_p_value(observed:, expected:, minimum_total: 200)`
|
|
1946
|
+
|
|
1947
|
+
- Takes: Hashes of arm => count and arm => expected count (Symbol or String
|
|
1948
|
+
keys, as `observed`'s); `minimum_total`, read as given above or below its
|
|
1949
|
+
default.
|
|
1950
|
+
- Answers: Pearson's chi-squared p on k - 1 degrees of freedom over the arms
|
|
1951
|
+
expected to get traffic; nil under `minimum_total` or with fewer than two
|
|
1952
|
+
such arms; 0.0 when an arm expected to get nothing got some.
|
|
1953
|
+
- Guarantees (*exact*): `chi_squared_survival`'s accuracy.
|
|
1954
|
+
- Refuses: nothing of its own (a non-Hash raises `NoMethodError`).
|
|
1955
|
+
|
|
1956
|
+
### Estimates of several parts (`SharedCredit`)
|
|
1957
|
+
|
|
1958
|
+
#### `SharedCredit.with_prior(estimate, strength:)`
|
|
1959
|
+
|
|
1960
|
+
- Takes: an estimate in `SharedCredit.estimate`'s shape (an `Estimate`, or a
|
|
1961
|
+
plain Hash such as one read back from JSON), read by Symbol or String, with
|
|
1962
|
+
its weight sums and Kish's sizes; `strength`, a finite real number >= 0, in
|
|
1963
|
+
visits (0 tops nothing up).
|
|
1964
|
+
- Answers: an `Estimate` with the same arms in the same order, the parts (none
|
|
1965
|
+
for a plain Hash), the weight sums and Kish's sizes (the evidence's own),
|
|
1966
|
+
the means and covariance under the prior pooled from its own credited arms
|
|
1967
|
+
(the rule under
|
|
1968
|
+
[An arm of a handful of visits](#an-arm-of-a-handful-of-visits-with_prior)),
|
|
1969
|
+
and its rounding: the given one's (0.0 for a plain Hash) or, where larger,
|
|
1970
|
+
that of an entry it changed.
|
|
1971
|
+
- Guarantees (*exact*: plain arithmetic in a fixed order, no draw and no `Math`
|
|
1972
|
+
function, so the same Floats everywhere): the rule, against a reference in
|
|
1973
|
+
Rationals that reads the rows, from rows and from per-arm sums alike; the
|
|
1974
|
+
covariance exactly symmetric wherever the given one is (as every estimate
|
|
1975
|
+
`estimate` answers is), and taken by `LinearValue` at the answer's rounding;
|
|
1976
|
+
an arm at or above the strength, and an arm with no credit, keeps its means
|
|
1977
|
+
and covariance block bit for bit; with every credited arm at or above it the
|
|
1978
|
+
answer holds the given means, covariance and rounding, so every seeded
|
|
1979
|
+
`LinearValue` draw and summary is the one the estimate gives; in an ordinary
|
|
1980
|
+
test (no row credits two arms) an arm at or above the strength draws the same
|
|
1981
|
+
values beside an arm topped up; a part every credited arm holds at one mean
|
|
1982
|
+
keeps that mean exactly; the given estimate is not changed.
|
|
1983
|
+
Measured at a strength of 20: a version of one visit reads P(best) between
|
|
1984
|
+
0.41 and 0.77, and no version of three visits or fewer reaches 0.95.
|
|
1985
|
+
- Refuses: a strength below 0 or not a finite real number; an estimate that is
|
|
1986
|
+
not a Hash, or whose means, covariance, weight sums or Kish's sizes are
|
|
1987
|
+
missing or malformed; an arm with means but a Kish's size of 0, each before
|
|
1988
|
+
anything is computed; and, as it is computed, an estimate whose figures leave
|
|
1989
|
+
a Float's range under the prior ("the sums leave a Float's range").
|
|
1990
|
+
|
|
1330
1991
|
## Bandit simulations: measured results
|
|
1331
1992
|
|
|
1332
1993
|
`spec/simulations` holds ten files, tagged `simulation: true` and run only
|