skiftet_statistical 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b72f7bccf09703650bb6232fef2bf588bc41564bf10ecc4a3802d9a432ecaee8
-  data.tar.gz: 9bb85de398e7bd00633db142d08122cdc3b55fda9320ef5035eab53b04051883
+  metadata.gz: f02da24bcb2a34afee81fe7e8f78bdbd2dd5abf98535dbfc8a5963e15bb638ef
+  data.tar.gz: 35134c7fe4639022d8fc2343541ca5ad0e6f565efa1282470ead22662f251ec2
 SHA512:
-  metadata.gz: 17c6847d2e87c057b4f1a22901f6b9d542a1576af7edf0cb261c7962542113359e63ebb69a1ec408d2f80937d3dfc13ecd3c11f12c24841d277f11b47487195a
-  data.tar.gz: 0e37add790389b5362df68bbf9c71e7e08b8ec967a0af6a615bb2958a4bbcde67ba1e0766cb52bb1a8560a4001b61cf9a7429e9d10578c613cd285e5e4a79107
+  metadata.gz: 56cae903729325b52f01f4e7da075f99730c80fcf339aecb1b4579e693f53386f35008df09ccadf316165f36700bd6030037c98e913820229b153f0730f836f1
+  data.tar.gz: d51df413daa913ae97fe272d00f83bdb349b55a93a2c9f57d02160ce3e40f48708aefc5d47a96f3e88b7c4aef669479987b307c6bc0b7d658c68b873a13d0cd4

data/CHANGELOG.md

@@ -1,5 +1,49 @@
 # Changelog
 
+## [0.2.0] - 2026-10-16
+
+The statistics f1 wrote more than once, and f1's A/B-test allocator, in one
+place. Pure Ruby, no new dependency; each function keeps f1's arithmetic in
+f1's order, so on the inputs f1 gives it (and the same seed) it answers as f1
+did, bit for bit.
+
+- `SkiftetStatistical::Gamma` — `regularized_lower(a, x)` and
+  `regularized_upper(a, x)`, the regularized incomplete gamma functions P and Q
+  (series below x = a + 1, Lentz continued fraction above).
+- `SkiftetStatistical::BetaFunction` — `regularized(x, alpha, beta)`, the
+  regularized incomplete beta function, and `quantile(p, alpha, beta)`, its
+  inverse by bisection.
+- `SkiftetStatistical::JointBeta` — `each_draw(posteriors, draws:, rng:)` (one
+  row per draw, arms in order, an Enumerator without a block) and
+  `summary(posteriors, draws:, rng:)` (`{ samples:, wins:, losses: }`, a tie
+  split equally).
+- `Significance.normal_quantile(p)`, `.chi_squared_survival(statistic, dof)`
+  and `.sample_ratio_p_value(observed:, expected:, minimum_total: 200)`.
+- `SkiftetStatistical::Bandit::Allocator` — f1's A/B-test allocator:
+  `call(arms:, tallies:, status:, winner_arm:, rng:, previous_weights: nil,
+  draws:, hold_satisfied: true, allocation: :adaptive, config: Config.new)`
+  returns `{ weights:, phase:, stats:, winner:, result_changed:, leader: }`
+  (Thompson sampling on joint Beta draws, a floor per arm, a step limit, an
+  automatic win and the won split); `won_weights(arms, winner, config:)`;
+  `Allocator::Config` (warmup_exposures, floor, max_step, win_p_best,
+  win_loss, won_share, won_rest, result_changed_below, credible_mass,
+  prior_alpha, prior_beta; f1's values by default); `Allocator::
+  FixedAllocation` (an `ArgumentError`), raised for a fixed split.
+- `SkiftetStatistical::Bandit::WinHold` — the hold on an automatic win:
+  `new(leading_arm:, leading_since:, now:, last_pass_at:, hold_seconds:
+  86_400, max_gap_seconds: 1_200)` with `satisfied?`, `interrupted?`,
+  `satisfied_for?(arm)` and `after_pass(leader)`.
+- `SkiftetStatistical::Bandit::WeightedPosterior` — an arm's Beta posterior
+  from raw or inverse-propensity counts: `count(tally)`, `beta(count,
+  prior_alpha: 1.0, prior_beta: 1.0)`, `effective_exposures(count)` (the Kish
+  n, never above the raw count).
+- The bandit simulations (`spec/simulations`, tagged `simulation: true`) run
+  only with `SIMULATIONS=1`; CI runs them nightly, by hand, and on a push or
+  pull request that changes the bandit code. Measured results: README,
+  "Bandit simulations: measured results".
+- Unchanged: the 0.1 API, and how `Sampler` uses its RNG (a seed gives the same
+  draws as in 0.1).
+
 ## [0.1.0] - 2026-06-23
 
 Initial release — Skiftet's shared statistics toolkit.

data/README.md

@@ -8,17 +8,25 @@ Modules:
 
 - **`Descriptive`** — mean, variance, standard deviation, percentiles/median.
 - **`Significance`** — A/B significance testing: two-proportion z-test, Welch's
-  t-test, exact normal CDF and two-tailed p-values.
+  t-test, exact normal CDF and two-tailed p-values, the normal quantile, the
+  chi-squared survival and the sample-ratio check.
+- **`Gamma`** / **`BetaFunction`** — the regularized incomplete gamma and beta
+  functions, and the Beta quantile.
 - **`Sampler`** — Gamma/Beta/Gaussian random sampling, RNG-injectable.
+- **`JointBeta`** — joint Monte Carlo draws over Beta posteriors (P(best),
+  expected loss).
 - **`Bandit`** (+ `Policies`) — multi-armed bandit (Thompson Sampling,
   epsilon-greedy, UCB1, Softmax) for online explore/exploit decisions.
+- **`Bandit::Allocator`**, **`Bandit::WinHold`**, **`Bandit::WeightedPosterior`**
+  — an A/B test's traffic allocation: Thompson sampling with a floor per arm, a
+  step limit, an automatic win and a hold on it (f1's rule).
 
 ## Install
 
-In a Gemfile (path dependency within the Skiftet workspace):
+In a Gemfile (published on rubygems.org):
 
 ```ruby
-gem "skiftet_statistical", path: "../skiftet_statistical"
+gem "skiftet_statistical", "~> 0.2.0"
 ```
 
 Or build/install locally:
@@ -63,6 +71,68 @@ S.two_tailed_p_value(1.96) # => ~0.05
 `two_proportion_z_test` / `welch_t_test` return `nil` when the test is undefined
 (an empty group or zero variance), matching the existing analyzers' behaviour.
 
+### Sample-ratio check
+
+Does each arm's traffic fit the split its weights promise? Pearson's chi-squared
+over the arms expected to get traffic, on k - 1 degrees of freedom. A small p
+means the split is not doing what it says (a broken arm, a cache, a bot).
+
+```ruby
+S.sample_ratio_p_value(observed: { control: 700, a: 300 },
+                       expected: { control: 500.0, a: 500.0 })   # => ~1.1e-36
+S.sample_ratio_p_value(observed: ..., expected: ..., minimum_total: 500)
+```
+
+`nil` (not tested) under `minimum_total` observations (default 200) or with
+fewer than two arms expected to get traffic; `0.0` when an arm expected to get
+nothing got some.
+
+## Distribution functions
+
+Pure Ruby, Float in and out. Every function takes Integers too.
+
+```ruby
+S.normal_quantile(0.975)                  # => 1.9599639845400532 (the probit; bisection on erfc)
+S.chi_squared_survival(3.841458820694124, 1) # => 0.05000000000000006: P(X >= statistic), X ~ chi-squared(1)
+
+G = SkiftetStatistical::Gamma
+G.regularized_lower(a, x)                 # P(a, x), the Gamma(a, 1) distribution function
+G.regularized_upper(a, x)                 # Q(a, x) = 1 - P(a, x)
+
+B = SkiftetStatistical::BetaFunction
+B.regularized(x, alpha, beta)             # I_x(alpha, beta), the Beta(alpha, beta) distribution function
+B.quantile(0.975, 41.0, 361.0)            # its inverse: an exact 95 % credible bound
+```
+
+The arithmetic and its order are part of the API: an app that pins these
+outputs (f1 does, at 1e-12) compares them with this exact sequence of
+operations, so a reordered formula is a breaking change even when it is "the
+same" formula.
+
+## Joint Beta draws
+
+For Bayesian A/B readouts and Thompson-sampling allocators: draw every arm's
+Beta posterior once per draw, jointly, so "P(arm is best)" and "expected loss"
+compare the same worlds.
+
+```ruby
+posteriors = [ [ 41.0, 361.0 ], [ 52.0, 350.0 ] ]   # [alpha, beta] per arm
+
+SkiftetStatistical::JointBeta.each_draw(posteriors, draws: 2_000, rng: Random.new(7)) do |row|
+  row  # => one Float per arm, in order
+end
+
+summary = SkiftetStatistical::JointBeta.summary(posteriors, draws: 2_000, rng: Random.new(7))
+summary[:samples]  # each arm's 2 000 draws
+summary[:wins]     # draws each arm won (a tie split equally); / draws = P(best)
+summary[:losses]   # summed (best - arm) per arm; / draws = expected loss
+```
+
+**The draw order is part of the API.** Per draw, arm by arm in the order
+given, one `Sampler#beta` each. `Sampler#gamma` takes a variable number of
+values from the RNG, so one arm's posterior moves every later draw: a seeded
+caller gets the same rows only while this order holds.
+
 ## Quick start (multi-armed bandit)
 
 ```ruby
@@ -150,6 +220,154 @@ bandit.record(channel, 1)
 Rails.cache.write("share_bandit_state", bandit.to_h)
 ```
 
+## A/B-test allocation (`Bandit::Allocator`)
+
+The decision an A/B test's allocation job makes on every pass: how much traffic
+each arm gets next, and whether an arm has won. It is f1's rule, moved here;
+pure (no clock, no state), with the randomness only from the injected `rng`.
+
+```ruby
+A = SkiftetStatistical::Bandit::Allocator
+
+result = A.call(
+  arms: %i[control a],
+  tallies: { control: { exposures: 1_000, conversions: 500 },
+             a: { exposures: 1_000, conversions: 533 } },
+  status: "running", winner_arm: nil,
+  previous_weights: { control: 0.5, a: 0.5 },   # the latest allocation, or nil
+  rng: Random.new(7), draws: 20_000,
+)
+result[:weights]   # => { control: 0.35, a: 0.65 }  (next pass's split)
+result[:phase]     # => :shifting  (:warmup, :shifting, :won or :paused)
+result[:stats][:a] # => { exposures: 1000, conversions: 533, rate: 0.533, weighted_rate: 0.533,
+                   #      lower: 0.502..., upper: 0.563..., p_best: 0.93175, expected_loss: 0.00066...,
+                   #      guardrails: {} }
+result[:winner]    # => nil  (the crowned arm)
+result[:leader]    # => nil  (the arm meeting the win rule at this pass)
+result[:result_changed] # => false  (a won test whose winner fell below P(best) 0.5)
+```
+
+The rule, in order:
+
+1. **Paused** (`status: "paused"`): the previous weights unchanged (equal
+   weights without them).
+2. **Won** (`status: "won"` and `winner_arm` one of the arms): the won split,
+   95 % to the winner and 5 % shared by the others. `result_changed` flags a
+   winner whose P(best) has fallen below 0.5; a won test never un-wins.
+3. **Warm-up**, while any arm has fewer than 300 exposures: equal weights.
+4. **The automatic win**: the first arm (in the order given) with P(best)
+   >= 0.95 and expected loss <= 0.0025 is the `leader`. It gets the won split
+   when `hold_satisfied:` is true (the default); with false (the hold below is
+   still running) it gets step 5's weights.
+5. **Shifting**: the target is each arm's P(best), every arm at least 10 %
+   (the others scaled down in proportion), reached by a convex step from the
+   previous weights in which no arm moves more than 0.15.
+
+P(best) and expected loss come from `draws` joint draws of the arms' Beta
+posteriors (`JointBeta`); the interval is the 95 % equal-tailed one of the
+arm's draws. **The draw order is part of the API**: with the same input and
+seed the result is the same only while no draw is added, dropped or reordered.
+
+A tally may carry inverse-propensity sums, `weighted_exposures` and
+`weighted_conversions` (each exposure counted as 1 / the arm's share when it
+was drawn) and `weighted_exposures_squared`: the posterior is then built on
+the weighted conversions rescaled to the Kish effective sample size, never
+above the raw count (`WeightedPosterior`), so an arm whose lead comes from a
+traffic surge it was favoured in is not crowned. A tally may also carry
+`guardrails: { goal => conversions }`, reported per arm with its raw rate and
+never decided on. Tallies and previous weights may use Symbol or String keys
+(rows read back from JSON); previous weights that do not name exactly the arms
+with finite numbers read as none.
+
+Every number above is a setting of `Allocator::Config`, with f1's values as
+defaults:
+
+```ruby
+config = A::Config.new(warmup_exposures: 100, floor: 0.05) # the rest stay
+config.max_arms                                             # => 20 (1 / floor)
+A.call(..., config: config)
+A.won_weights(%i[control a b], :a, config: config)          # => { control: 0.025, a: 0.95, b: 0.025 }
+```
+
+Settings: `warmup_exposures` (300), `floor` (0.10), `max_step` (0.15),
+`win_p_best` (0.95), `win_loss` (0.0025), `won_share` (0.95), `won_rest`
+(0.05), `result_changed_below` (0.5), `credible_mass` (0.95), `prior_alpha`
+and `prior_beta` (1.0). More arms than `max_arms` raise `ArgumentError`, and
+`allocation: :fixed` (a split that is measured, not allocated) raises
+`A::FixedAllocation`, an `ArgumentError`, before anything is computed.
+
+### The hold on a win (`Bandit::WinHold`)
+
+An automatic win counts only after the same arm met the win rule at every pass
+for 24 hours. The caller keeps the state (the leading arm, when it started
+leading, the time of the previous pass) and gives the clock:
+
+```ruby
+hold = SkiftetStatistical::Bandit::WinHold.new(
+  leading_arm: "a", leading_since: now - 90_000, now: now, last_pass_at: now - 900,
+  # hold_seconds: 86_400, max_gap_seconds: 1_200 (the defaults)
+)
+hold.satisfied?        # => true: the stored leader held for 24 hours or more
+hold.satisfied_for?(:a)
+result = A.call(..., hold_satisfied: hold.satisfied?)
+state = hold.after_pass(result[:leader]) # => the leading arm and since, to store
+```
+
+A run of passes with a gap longer than `max_gap_seconds` (or no previous pass)
+is interrupted: not satisfied, and a leader starts over. Call the allocator a
+second time with the same seed and `hold_satisfied: false` when the arm it
+crowns is not the arm that held (`satisfied_for?`), as f1's job does.
+
+## Bandit simulations: measured results
+
+`spec/simulations` replays A/B tests through `Bandit::Allocator` (and
+`Bandit::WinHold`) in memory: every pass, 40 visits are drawn by the current
+weights, each exposure counts 1 / the weight it was drawn under, and the
+allocator is called with the previous weights, as an allocation job does every
+15 minutes. Every run is seeded, so each count below is exact for this version
+of the code. The three examples (`SIMULATIONS=1 bundle exec rspec
+spec/simulations`) assert the bounds; the other figures were measured with the
+same code and seeds. Moved from f1 (its `allocator_simulation_oracle_spec.rb`
+and `win_hold_simulation_spec.rb`) with the same seeds, runs, passes, draws and
+bounds; measured again here on 2026-10-15 with skiftet_statistical 0.2.0 and
+Ruby 3.4.9, and every count f1 had recorded came out the same.
+
+### The automatic win (`allocator_simulation_spec.rb`)
+
+400 passes of 40 visits, 400 draws.
+
+| Runs | Arms (true conversion rate) | Bound (asserted) | Measured |
+|---|---|---|---|
+| 200 (seeds 20 261 003 + run) | four arms at 30 % | an arm crowned in fewer than 20 / 200 | 15 / 200 (7.5 %) |
+| 100 (seeds 30 261 003 + run) | three arms at 30 %, one at 25 % | the worse arm crowned in 0 / 100 | 0 / 100 (an equal arm in 21 / 100) |
+
+With equal arms a crowned arm costs nothing (it is as good as the rest, and its
+expected loss is at most `win_loss`), so the bound caps noise wins; a truly
+worse arm is never crowned.
+
+### The hold (`win_hold_simulation_spec.rb`)
+
+Two arms, 11 % against 10 % (the worse arm), 400 passes of 40 visits (100
+hours), 200 runs (seeds 40 261 003 + run). The decision step is f1's: the
+allocator with `hold_satisfied: hold.satisfied?`, and the same seed again with
+`hold_satisfied: false` when the arm it crowns is not the arm that held.
+
+- WITHOUT the hold (the allocator alone, `hold_satisfied: true` at every pass):
+  the worse arm crowned in 26 / 200 (13.0 %) at 400 draws, 24 / 200 (12.0 %) at
+  2 000 draws.
+- The hold's code with no wait (`hold_seconds: 0`), 1 000 draws: the worse arm
+  crowned in 13 / 200 (6.5 %).
+- WITH the hold (24 hours): the worse arm crowned in 0 / 200 at 400, 1 000,
+  2 000 and 20 000 draws; the better arm in 41-46 / 200 (43 at 400, 46 at
+  1 000, 41 at 2 000, 44 at 20 000), and the rest undecided after 100 hours.
+- The asserted example runs at 1 000 draws: the worse arm at most 6 / 200 and
+  the better arm at least 20 / 200 (a hold that never releases would crown
+  nothing and pass the first bound alone). Measured: worse 0 / 200, better
+  46 / 200, undecided 154 / 200.
+
+A change that moves any of these counts changes the rule's behaviour; measure
+again and update this section in the same change.
+
 ## Development
 
 ```sh
@@ -157,6 +375,7 @@ bundle install
 bundle exec rake spec      # run the specs
 bundle exec rake rubocop   # lint
 bundle exec rake           # both
+SIMULATIONS=1 bundle exec rspec spec/simulations   # the bandit simulations (minutes)
 ```
 
 ## License

data/lib/skiftet_statistical/bandit/allocator.rb

@@ -0,0 +1,274 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  class Bandit
+    # A Thompson-sampling allocator's decision for one experiment: how much traffic each arm gets next, and
+    # whether an arm has won. Pure: no clock, no state; the randomness comes only from the injected `rng`, so the
+    # same input and the same seed give the same output.
+    #
+    #   SkiftetStatistical::Bandit::Allocator.call(
+    #     arms: %i[control a],
+    #     tallies: { control: { exposures: 1_000, conversions: 500 }, a: { exposures: 1_000, conversions: 533 } },
+    #     status: "running", winner_arm: nil, previous_weights: { control: 0.5, a: 0.5 },
+    #     rng: Random.new(7), draws: 20_000)
+    #   # => { weights: { control: 0.35, a: 0.65 }, phase: :shifting,
+    #   #      stats: { control: { ... }, a: { ... } }, winner: nil, result_changed: false, leader: nil }
+    #
+    # The rule, with the settings of Config:
+    # - paused: the previous weights unchanged (equal weights without them);
+    # - won (status "won" and `winner_arm` one of the arms): the won split, `won_share` to the winner and
+    #   `won_rest` shared by the others, and `result_changed` when the winner's P(best) has fallen below
+    #   `result_changed_below`;
+    # - warm-up, while any arm has fewer than `warmup_exposures`: equal weights;
+    # - the automatic win: the first arm (in order) with P(best) >= `win_p_best` and expected loss <= `win_loss`
+    #   is the `leader`; it gets the won split when `hold_satisfied:` is true (the default). With false the
+    #   leader gets the step below and the caller keeps its hold (WinHold);
+    # - otherwise, shifting: the target is each arm's P(best), every arm at least `floor` (the others scaled
+    #   down in proportion), reached by a convex step from the previous weights in which no arm moves more than
+    #   `max_step`.
+    #
+    # The statistics (`stats`, per arm): exposures, conversions, the raw rate (nil at no exposures), the weighted
+    # rate, the `credible_mass` interval (lower, upper) of the arm's draws, P(best) (a tie split equally) and the
+    # expected loss (the mean of the draw's best minus the arm). The posteriors are WeightedPosterior's; the
+    # draws are JointBeta's (`draws` joint draws, arm by arm in order). A tally MAY carry `guardrails:
+    # { goal => conversions }` (or `{ goal => { conversions: n } }`): each goes into the stats with its raw rate,
+    # for display only.
+    #
+    # Arms are Symbols in the result (Symbols or Strings in); tallies and previous weights may use either. Previous
+    # weights count only when they name exactly the current arms, each with a finite number: anything else reads
+    # as none. A share below the floor (a won split read back under "running") is lifted onto it before the step.
+    #
+    # THE DRAW ORDER IS THE CONTRACT (JointBeta): a seeded caller gets the same result only while no draw is added,
+    # dropped or reordered.
+    class Allocator
+      # Asked to allocate a fixed split: its weights are its own, never the allocator's to move or crown.
+      class FixedAllocation < ArgumentError; end
+
+      # The allocator's settings, each a finite number. The defaults are the values f1's A/B tests run with.
+      class Config
+        DEFAULTS = {
+          warmup_exposures: 300, floor: 0.10, max_step: 0.15, win_p_best: 0.95, win_loss: 0.0025, won_share: 0.95,
+          won_rest: 0.05, result_changed_below: 0.5, credible_mass: 0.95, prior_alpha: 1.0, prior_beta: 1.0
+        }.freeze
+
+        attr_reader(*DEFAULTS.keys)
+
+        def initialize(**settings)
+          unknown = settings.keys - DEFAULTS.keys
+          raise ArgumentError, "unknown allocator setting(s): #{unknown.join(', ')}" unless unknown.empty?
+
+          DEFAULTS.merge(settings).each do |name, value|
+            unless value.is_a?(Numeric) && value.finite?
+              raise ArgumentError, "#{name} must be a finite number, got #{value.inspect}"
+            end
+
+            instance_variable_set(:"@#{name}", value)
+          end
+          raise ArgumentError, "floor must be above 0 and at most 1, got #{floor.inspect}" unless floor.positive? && floor <= 1
+
+          freeze
+        end
+
+        # More arms than this cannot all hold the floor.
+        def max_arms
+          (1 / floor).round
+        end
+
+        def to_h
+          DEFAULTS.keys.to_h { |name| [ name, public_send(name) ] }
+        end
+
+        def ==(other)
+          other.is_a?(Config) && to_h == other.to_h
+        end
+        alias_method :eql?, :==
+
+        def hash
+          to_h.hash
+        end
+      end
+
+      # The won split: the winner `won_share`, the other arms `won_rest` shared equally; a single arm gets
+      # everything. The keys are `arms` as given (Symbols or Strings).
+      def self.won_weights(arms, winner, config: Config.new)
+        return { winner => 1.0 } if arms.size == 1
+
+        rest = config.won_rest / (arms.size - 1)
+        arms.to_h { |arm| [ arm, arm == winner ? config.won_share : rest ] }
+      end
+
+      def self.call(arms:, tallies:, status:, winner_arm:, rng:, previous_weights: nil, draws:, hold_satisfied: true,
+                    allocation: :adaptive, config: Config.new)
+        new(arms: arms, tallies: tallies, status: status, winner_arm: winner_arm, rng: rng,
+            previous_weights: previous_weights, draws: draws, hold_satisfied: hold_satisfied, allocation: allocation,
+            config: config).call
+      end
+
+      def initialize(arms:, tallies:, status:, winner_arm:, rng:, draws:, previous_weights: nil, hold_satisfied: true,
+                     allocation: :adaptive, config: Config.new)
+        if allocation.to_s == "fixed"
+          raise FixedAllocation, "a fixed split's weights are its phase's: the allocator never moves or crowns them"
+        end
+        raise ArgumentError, "config must be a #{Config}, got #{config.inspect}" unless config.is_a?(Config)
+
+        @config = config
+        @arms = arms.map(&:to_sym)
+        raise ArgumentError, "an experiment needs at least one arm" if @arms.empty?
+        raise ArgumentError, "at most #{config.max_arms} arms keep every arm at the floor" if @arms.size > config.max_arms
+
+        @counts = @arms.map { |arm| read_tally(tallies, arm) }
+        @status = status.to_s
+        @winner_arm = winner_arm&.to_sym
+        @previous = read_weights(previous_weights)
+        @rng = rng
+        @draws = Integer(draws)
+        raise ArgumentError, "draws must be positive" unless @draws.positive?
+
+        @hold_satisfied = hold_satisfied ? true : false
+      end
+
+      def call
+        stats = build_stats
+
+        if @status == "paused"
+          result(@previous || equal_weights, :paused, stats)
+        elsif @status == "won" && @arms.include?(@winner_arm)
+          changed = stats.fetch(@winner_arm)[:p_best] < @config.result_changed_below
+          result(won_weights(@winner_arm), :won, stats, winner: @winner_arm, result_changed: changed)
+        elsif @counts.any? { |count| count[:exposures] < @config.warmup_exposures }
+          result(equal_weights, :warmup, stats)
+        elsif (leader = automatic_winner(stats)) && @hold_satisfied
+          result(won_weights(leader), :won, stats, winner: leader, leader: leader)
+        else
+          target = with_floor(@arms.to_h { |arm| [ arm, stats.fetch(arm)[:p_best] ] })
+          result(step_toward(target), :shifting, stats, leader: leader)
+        end
+      end
+
+      private
+
+      def result(weights, phase, stats, winner: nil, result_changed: false, leader: nil)
+        { weights: weights, phase: phase, stats: stats, winner: winner, result_changed: result_changed, leader: leader }
+      end
+
+      # The tally's numbers as WeightedPosterior reads them, plus the guardrails (display only).
+      def read_tally(tallies, arm)
+        tally = tallies[arm] || tallies[arm.to_s] || {}
+        WeightedPosterior.count(tally).merge(guardrails: fetch_key(tally, :guardrails) || {})
+      end
+
+      def fetch_key(hash, key)
+        hash.key?(key) ? hash[key] : hash[key.to_s]
+      end
+
+      # Previous weights count only when they are a Hash that names exactly the current arms, each with a finite
+      # number; a value that is not a number (a String, a Hash, nil) is ignored, so a corrupt row gives none. Zero
+      # and negative shares are kept: #step_toward lifts a share below the floor onto it first.
+      def read_weights(weights)
+        return nil unless weights.is_a?(Hash)
+
+        normalized = weights.each_with_object({}) do |(arm, share), out|
+          out[arm.to_sym] = share.to_f if share.is_a?(Numeric) && share.to_f.finite?
+        end
+        return nil unless normalized.keys.sort == @arms.sort
+
+        @arms.to_h { |arm| [ arm, normalized.fetch(arm) ] }
+      end
+
+      def equal_weights
+        share = 1.0 / @arms.size
+        @arms.to_h { |arm| [ arm, share ] }
+      end
+
+      def won_weights(winner)
+        self.class.won_weights(@arms, winner, config: @config)
+      end
+
+      # The arm that is both very likely best and cheap to be wrong about.
+      def automatic_winner(stats)
+        return nil if @arms.size < 2
+
+        @arms.find do |arm|
+          stats.fetch(arm)[:p_best] >= @config.win_p_best && stats.fetch(arm)[:expected_loss] <= @config.win_loss
+        end
+      end
+
+      # Every arm at least the floor, the others scaled down in proportion.
+      def with_floor(target)
+        floor = @config.floor
+        fixed = []
+        loop do
+          free = @arms - fixed
+          mass = 1.0 - (fixed.size * floor)
+          free_total = free.sum { |arm| target.fetch(arm) }
+          scaled = free.to_h do |arm|
+            [ arm, free_total.positive? ? mass * target.fetch(arm) / free_total : mass / free.size ]
+          end
+          below = scaled.select { |_arm, share| share < floor }.keys
+          return @arms.to_h { |arm| [ arm, fixed.include?(arm) ? floor : scaled.fetch(arm) ] } if below.empty?
+
+          fixed.concat(below)
+        end
+      end
+
+      # A convex step from the previous weights, no arm moving more than max_step. A previous share below the
+      # floor is lifted onto it first.
+      def step_toward(target)
+        previous = @previous || equal_weights
+        previous = with_floor(previous) if previous.values.any? { |share| share < @config.floor }
+        largest = @arms.map { |arm| (target.fetch(arm) - previous.fetch(arm)).abs }.max
+        lambda = largest > @config.max_step ? @config.max_step / largest : 1.0
+
+        @arms.to_h { |arm| [ arm, previous.fetch(arm) + (lambda * (target.fetch(arm) - previous.fetch(arm))) ] }
+      end
+
+      def build_stats
+        posteriors = @counts.map do |count|
+          WeightedPosterior.beta(count, prior_alpha: @config.prior_alpha, prior_beta: @config.prior_beta)
+        end
+        samples, wins, losses = JointBeta.summary(posteriors, draws: @draws, rng: @rng).values_at(:samples, :wins, :losses)
+        tail = (1.0 - @config.credible_mass) / 2.0
+
+        @arms.each_with_index.to_h do |arm, index|
+          count = @counts[index]
+          sorted = samples[index].sort
+          [ arm, {
+            exposures: count[:exposures],
+            conversions: count[:conversions],
+            rate: rate(count[:conversions], count[:exposures]),
+            weighted_rate: weighted_rate(count),
+            lower: quantile(sorted, tail),
+            upper: quantile(sorted, 1.0 - tail),
+            p_best: wins[index] / @draws,
+            expected_loss: losses[index] / @draws,
+            guardrails: guardrails(count)
+          } ]
+        end
+      end
+
+      def quantile(sorted, probability)
+        position = probability * (sorted.size - 1)
+        low = sorted[position.floor]
+        high = sorted[position.ceil]
+        low + ((high - low) * (position - position.floor))
+      end
+
+      def rate(conversions, exposures)
+        exposures.positive? ? conversions.fdiv(exposures) : nil
+      end
+
+      def weighted_rate(count)
+        return rate(count[:conversions], count[:exposures]) unless count[:weighted_exposures]
+
+        count[:weighted_conversions] / count[:weighted_exposures]
+      end
+
+      def guardrails(count)
+        count[:guardrails].to_h do |goal, value|
+          conversions = Integer((value.is_a?(Hash) ? fetch_key(value, :conversions) : value) || 0)
+          [ goal.to_s, { conversions: conversions, rate: rate(conversions, count[:exposures]) } ]
+        end
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/bandit/weighted_posterior.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  class Bandit
+    # One arm's Beta posterior as the allocator judges it, from a tally of exposures and conversions. Pure: no
+    # randomness, no state.
+    #
+    #   count = SkiftetStatistical::Bandit::WeightedPosterior.count(exposures: 400, conversions: 40)
+    #   SkiftetStatistical::Bandit::WeightedPosterior.beta(count) # => [41.0, 361.0]
+    #
+    # A tally MAY carry `weighted_exposures` and `weighted_conversions`, the sums of 1 / the arm's share at each
+    # draw (inverse-propensity counts), and `weighted_exposures_squared`, the sum of their squares. The weighted
+    # conversions are then rescaled to an exposure count n before the posterior is built, so an arm drawn mostly
+    # under a small share holds no more evidence than it has. n is the Kish effective sample size
+    # (sum w)^2 / sum w^2 when the squared sum is present, never more than the raw count; else the raw exposure
+    # count. Without the weighted sums the raw counts are judged.
+    module WeightedPosterior
+      module_function
+
+      # The numbers of a tally (Symbol or String keys; a nil tally is nobody seen).
+      def count(tally)
+        tally ||= {}
+        weighted_exposures = fetch_key(tally, :weighted_exposures)&.to_f
+        weighted_conversions = fetch_key(tally, :weighted_conversions)&.to_f
+        weighted_squared = fetch_key(tally, :weighted_exposures_squared)&.to_f
+        weighted = !weighted_exposures.nil? && !weighted_conversions.nil? && weighted_exposures.positive?
+
+        {
+          exposures: Integer(fetch_key(tally, :exposures) || 0),
+          conversions: Integer(fetch_key(tally, :conversions) || 0),
+          weighted_exposures: weighted ? weighted_exposures : nil,
+          weighted_conversions: weighted ? weighted_conversions : nil,
+          weighted_exposures_squared: weighted && weighted_squared&.positive? ? weighted_squared : nil
+        }
+      end
+
+      # [alpha, beta] = [prior_alpha + c, prior_beta + n - c], c clamped to [0, n] (n from #effective_exposures,
+      # c the weighted conversions rescaled to n when the count is weighted).
+      def beta(count, prior_alpha: 1.0, prior_beta: 1.0)
+        exposures = effective_exposures(count)
+        conversions =
+          if count[:weighted_exposures]
+            (count[:weighted_conversions] * exposures / count[:weighted_exposures]).clamp(0.0, exposures.to_f)
+          else
+            count[:conversions].to_f.clamp(0.0, exposures.to_f)
+          end
+
+        [ prior_alpha + conversions, prior_beta + exposures - conversions ]
+      end
+
+      # The exposure count the posterior is built on: the Kish n when the squared sum is present (never above the
+      # raw count), else the raw count.
+      def effective_exposures(count)
+        return count[:exposures] unless count[:weighted_exposures_squared]
+
+        kish = (count[:weighted_exposures]**2) / count[:weighted_exposures_squared]
+        [ kish, count[:exposures].to_f ].min
+      end
+
+      def fetch_key(hash, key)
+        hash.key?(key) ? hash[key] : hash[key.to_s]
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/bandit/win_hold.rb

@@ -0,0 +1,86 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  class Bandit
+    # The hold on an automatic win: the win rule must hold for the SAME arm at every allocation pass for
+    # `hold_seconds` before the win counts. Pure: the caller gives the clock (Time values, or any values whose
+    # difference is a number of seconds).
+    #
+    # The state is the leading arm, the time of the first pass in the current run of passes where that arm met the
+    # rule (`leading_since`), and the time of the previous pass (`last_pass_at`):
+    #
+    #   hold = SkiftetStatistical::Bandit::WinHold.new(leading_arm: "a", leading_since: now - 90_000, now: now,
+    #                                                  last_pass_at: now - 600)
+    #   hold.satisfied?            # => true (the stored leader met the rule for 24 hours or more)
+    #   hold.satisfied_for?(:a)    # => true
+    #   hold.after_pass(:a)        # => { leading_arm: "a", leading_since: <unchanged> }
+    #   hold.after_pass(:control)  # => { leading_arm: "control", leading_since: now }  (starts over)
+    #   hold.after_pass(nil)       # => { leading_arm: nil, leading_since: nil }         (reset)
+    #
+    # "Every pass" is checked, not assumed: `max_gap_seconds` is the longest time between two passes that still
+    # counts as an unbroken run. When `last_pass_at` is nil (no previous pass) or more than `max_gap_seconds` before
+    # `now`, the hold is interrupted: it is not satisfied, and a leader starts over at this pass. `last_pass_at` is
+    # required, so a call cannot leave it out by mistake and get a hold that never holds.
+    #
+    # A blank leading arm (nil, false, an empty or whitespace-only String, an empty Symbol) reads as none; any other
+    # arm is kept as a String. This is ActiveSupport's `blank?`, in every encoding: a String that is not
+    # ASCII-compatible (UTF-16, UTF-32) is matched with the pattern in its own encoding.
+    class WinHold
+      HOLD_SECONDS = 86_400
+      MAX_GAP_SECONDS = 1_200
+      BLANK = /\A[[:space:]]*\z/
+
+      attr_reader :leading_arm, :leading_since, :now, :last_pass_at, :hold_seconds, :max_gap_seconds
+
+      def self.blank?(value)
+        case value
+        when nil, false then true
+        when String then value.empty? || blank_string?(value)
+        else value.respond_to?(:empty?) ? (value.empty? ? true : false) : false
+        end
+      end
+
+      def self.blank_string?(value)
+        BLANK.match?(value)
+      rescue Encoding::CompatibilityError
+        Regexp.new(BLANK.source.encode(value.encoding), BLANK.options | Regexp::FIXEDENCODING).match?(value)
+      end
+      private_class_method :blank_string?
+
+      def initialize(leading_arm:, leading_since:, now:, last_pass_at:, hold_seconds: HOLD_SECONDS,
+                     max_gap_seconds: MAX_GAP_SECONDS)
+        @leading_arm = self.class.blank?(leading_arm) ? nil : leading_arm.to_s
+        @leading_since = leading_since
+        @now = now
+        @last_pass_at = last_pass_at
+        @hold_seconds = hold_seconds
+        @max_gap_seconds = max_gap_seconds
+      end
+
+      # The stored leader has met the rule at every pass since `hold_seconds` ago or longer.
+      def satisfied?
+        !leading_arm.nil? && !leading_since.nil? && !interrupted? && now - leading_since >= hold_seconds
+      end
+
+      # The run of passes since `leading_since` has a gap: no pass in the last `max_gap_seconds`.
+      def interrupted?
+        last_pass_at.nil? || now - last_pass_at > max_gap_seconds
+      end
+
+      # The hold is satisfied AND `arm` is the arm that held it.
+      def satisfied_for?(arm)
+        satisfied? && !arm.nil? && arm.to_s == leading_arm
+      end
+
+      # The state after a pass whose leader (the arm meeting the win rule this pass) is `leader`, or nil.
+      def after_pass(leader)
+        return { leading_arm: nil, leading_since: nil } if leader.nil?
+        if leader.to_s == leading_arm && leading_since && !interrupted?
+          return { leading_arm: leading_arm, leading_since: leading_since }
+        end
+
+        { leading_arm: leader.to_s, leading_since: now }
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/beta_function.rb

@@ -0,0 +1,126 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  # The regularized incomplete beta function I_x(alpha, beta), which is the Beta(alpha, beta) distribution
+  # function, and its inverse, the Beta quantile. An exact credible interval on a conversion rate is two quantiles
+  # of its Beta posterior; computed this way it does not move when a Monte Carlo seed does.
+  #
+  #   SkiftetStatistical::BetaFunction.regularized(0.5, 2.0, 2.0)   # => 0.5
+  #   SkiftetStatistical::BetaFunction.quantile(0.975, 41.0, 361.0) # => the 97.5 % bound of Beta(41, 361)
+  #
+  # The arithmetic and its order are part of the contract: with parameters in the hundreds or more the lgamma
+  # terms of the front factor reach 1e3-1e5, so a reordered sum can move the result by 1e-12 to 1e-10.
+  module BetaFunction
+    # Lentz's floor for a denominator that would otherwise be zero.
+    TINY = 1.0e-30
+    # A fraction step this close to 1 is the last one taken.
+    EPSILON = 1.0e-14
+    # Numerical Recipes' bound for betacf. Near the mean the fraction takes 116 steps for Beta(10 001, 40 001) and
+    # 244 for Beta(100 001, 400 001) (measured), so it binds only past a few hundred thousand observations; there a
+    # different bound gives different digits, which makes it part of the contract.
+    MAX_ITERATIONS = 300
+    # Halvings of [0, 1] in the quantile: 80 take the bracket below a Float's spacing anywhere in (0, 1).
+    QUANTILE_STEPS = 80
+
+    class << self
+      # I_x(alpha, beta) for alpha, beta > 0: 0.0 for x <= 0, 1.0 for x >= 1.
+      def regularized(x, alpha, beta)
+        x = number(x, "x")
+        incomplete(x, *parameters(alpha, beta))
+      end
+
+      # The x where I_x(alpha, beta) = probability, by bisection: 0.0 for probability <= 0, 1.0 for >= 1.
+      def quantile(probability, alpha, beta)
+        probability = number(probability, "probability")
+        alpha, beta = parameters(alpha, beta)
+        return 0.0 if probability <= 0.0
+        return 1.0 if probability >= 1.0
+
+        low = 0.0
+        high = 1.0
+        QUANTILE_STEPS.times do
+          mid = (low + high) / 2.0
+          if incomplete(mid, alpha, beta) < probability
+            low = mid
+          else
+            high = mid
+          end
+        end
+        (low + high) / 2.0
+      end
+
+      private
+
+      def number(value, name)
+        value = value.to_f
+        raise ArgumentError, "#{name} must be a number, got #{value}" if value.nan?
+
+        value
+      end
+
+      def parameters(alpha, beta)
+        alpha = alpha.to_f
+        beta = beta.to_f
+        unless alpha.finite? && alpha.positive? && beta.finite? && beta.positive?
+          raise ArgumentError, "alpha and beta must be finite numbers > 0, got #{alpha}, #{beta}"
+        end
+
+        [ alpha, beta ]
+      end
+
+      # The continued fraction converges fast below x = (alpha + 1) / (alpha + beta + 2); above it the function
+      # is read through the symmetry I_x(a, b) = 1 - I_(1-x)(b, a).
+      def incomplete(x, alpha, beta)
+        return 0.0 if x <= 0.0
+        return 1.0 if x >= 1.0
+
+        front = Math.exp(
+          Math.lgamma(alpha + beta)[0] - Math.lgamma(alpha)[0] - Math.lgamma(beta)[0] +
+          (alpha * Math.log(x)) + (beta * Math.log(1.0 - x))
+        )
+
+        if x < (alpha + 1.0) / (alpha + beta + 2.0)
+          front * continued_fraction(x, alpha, beta) / alpha
+        else
+          1.0 - (front * continued_fraction(1.0 - x, beta, alpha) / beta)
+        end
+      end
+
+      # Numerical Recipes' betacf, evaluated with Lentz's method.
+      def continued_fraction(x, alpha, beta)
+        qab = alpha + beta
+        qap = alpha + 1.0
+        qam = alpha - 1.0
+
+        c = 1.0
+        d = 1.0 - (qab * x / qap)
+        d = TINY if d.abs < TINY
+        d = 1.0 / d
+        h = d
+
+        (1..MAX_ITERATIONS).each do |m|
+          m2 = 2 * m
+
+          numerator = m * (beta - m) * x / ((qam + m2) * (alpha + m2))
+          d = 1.0 / guard(1.0 + (numerator * d))
+          c = guard(1.0 + (numerator / c))
+          h *= d * c
+
+          numerator = -(alpha + m) * (qab + m) * x / ((alpha + m2) * (qap + m2))
+          d = 1.0 / guard(1.0 + (numerator * d))
+          c = guard(1.0 + (numerator / c))
+          delta = d * c
+          h *= delta
+
+          break if (delta - 1.0).abs < EPSILON
+        end
+
+        h
+      end
+
+      def guard(value)
+        value.abs < TINY ? TINY : value
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/gamma.rb

@@ -0,0 +1,97 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  # The regularized incomplete gamma functions: P(a, x), the Gamma(a, 1) distribution function at x, and
+  # Q(a, x) = 1 - P(a, x). The chi-squared survival (Significance.chi_squared_survival) and an exact Poisson
+  # interval or p-value are both read off them.
+  #
+  # Below x = a + 1 the power series converges fast and gives P; at and above it the continued fraction
+  # (modified Lentz) converges fast and gives Q. Each function reads the other half as one minus it
+  # (Numerical Recipes, gammp / gammq).
+  #
+  #   SkiftetStatistical::Gamma.regularized_lower(1.0, 2.0) # => 0.8646647167633873 (1 - e**-2)
+  #   SkiftetStatistical::Gamma.regularized_upper(1.0, 2.0) # => 0.1353352832366127 (e**-2)
+  #
+  # The arithmetic and its order are part of the contract: an app that pins outputs at 1e-12 compares them with
+  # this exact sequence of operations, so a reordered formula is a breaking change even when it is "the same".
+  module Gamma
+    # A series term or a fraction step this small next to the total is the last one taken.
+    EPSILON = 1e-15
+    # A bound, not a tuning knob: near x = a the series takes 7 323 terms at a = 1e6 and 62 463 at a = 1e8 (the
+    # fraction fewer), measured. A lower bound gives different digits for large a.
+    MAX_TERMS = 100_000
+    # Lentz's floor for a denominator that would otherwise be zero.
+    TINY = 1e-300
+
+    class << self
+      # P(a, x) for a > 0. 0.0 for x <= 0.
+      def regularized_lower(a, x)
+        a, x = arguments(a, x)
+        return 0.0 if x <= 0.0
+        return 1.0 if x.infinite?
+
+        x < a + 1.0 ? series(a, x) : 1.0 - fraction(a, x)
+      end
+
+      # Q(a, x) = 1 - P(a, x) for a > 0. 1.0 for x <= 0.
+      def regularized_upper(a, x)
+        a, x = arguments(a, x)
+        return 1.0 if x <= 0.0
+        return 0.0 if x.infinite?
+
+        x < a + 1.0 ? 1.0 - series(a, x) : fraction(a, x)
+      end
+
+      private
+
+      def arguments(a, x)
+        a = a.to_f
+        x = x.to_f
+        raise ArgumentError, "a must be a finite number > 0, got #{a}" unless a.finite? && a.positive?
+        raise ArgumentError, "x must be a number, got #{x}" if x.nan?
+
+        [ a, x ]
+      end
+
+      # x**a * e**-x / Gamma(a), in logs so it neither overflows nor underflows on its way.
+      def prefix(a, x)
+        Math.exp((a * Math.log(x)) - x - Math.lgamma(a).first)
+      end
+
+      # P(a, x) by its power series; converges fast for x < a + 1.
+      def series(a, x)
+        term = 1.0 / a
+        sum = term
+        denominator = a
+        MAX_TERMS.times do
+          denominator += 1.0
+          term *= x / denominator
+          sum += term
+          break if term.abs < sum.abs * EPSILON
+        end
+        sum * prefix(a, x)
+      end
+
+      # Q(a, x) by its continued fraction, modified Lentz; converges fast for x >= a + 1.
+      def fraction(a, x)
+        b = x + 1.0 - a
+        c = 1.0 / TINY
+        d = 1.0 / b
+        h = d
+        (1..MAX_TERMS).each do |i|
+          an = -i * (i - a)
+          b += 2.0
+          d = (an * d) + b
+          d = TINY if d.abs < TINY
+          c = b + (an / c)
+          c = TINY if c.abs < TINY
+          d = 1.0 / d
+          delta = d * c
+          h *= delta
+          break if (delta - 1.0).abs < EPSILON
+        end
+        h * prefix(a, x)
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/joint_beta.rb

@@ -0,0 +1,63 @@
+# frozen_string_literal: true
+
+module SkiftetStatistical
+  # Joint Monte Carlo draws from several independent Beta posteriors, one row per draw. Arms must be drawn
+  # jointly (the same draw index across arms) or "P(arm is best)" and "expected loss" compare unrelated worlds.
+  #
+  #   posteriors = [ [ 41.0, 361.0 ], [ 52.0, 350.0 ] ]   # [alpha, beta] per arm
+  #   SkiftetStatistical::JointBeta.each_draw(posteriors, draws: 3, rng: Random.new(7)).to_a
+  #   # => [[0.1035..., 0.1491...], [0.1197..., 0.1343...], [0.1074..., 0.1065...]]
+  #   SkiftetStatistical::JointBeta.summary(posteriors, draws: 2_000, rng: Random.new(7))
+  #   # => { samples: [[2 000 Floats], [2 000 Floats]], wins: [219.0, 1781.0], losses: [57.83..., 2.58...] }
+  #
+  # THE DRAW ORDER IS THE CONTRACT: per draw, arm by arm in the order given, one Sampler#beta each. Sampler#gamma
+  # takes a variable number of values from the RNG, so one arm's posterior moves every later draw: a seeded
+  # caller gets the same rows only while this order holds, and never with an added, dropped or reordered draw.
+  module JointBeta
+    class << self
+      # Yields one Array per draw (one Float per arm, in order). Without a block, an Enumerator of the rows.
+      # Returns nil.
+      def each_draw(posteriors, draws:, rng:)
+        draws = count(draws)
+        return enum_for(:each_draw, posteriors, draws: draws, rng: rng) { draws } unless block_given?
+
+        sampler = Sampler.new(rng)
+        draws.times { yield posteriors.map { |alpha, beta| sampler.beta(alpha, beta) } }
+        nil
+      end
+
+      # Over the same rows as each_draw: every arm's draws (samples[arm][draw]), how many draws each arm won
+      # (a tie split equally between the tied arms) and its summed loss (the draw's best minus the arm's value).
+      # Divide wins and losses by draws for P(best) and expected loss.
+      def summary(posteriors, draws:, rng:)
+        draws = count(draws)
+        size = posteriors.size
+        samples = Array.new(size) { Array.new(draws) }
+        wins = Array.new(size, 0.0)
+        losses = Array.new(size, 0.0)
+
+        each_draw(posteriors, draws: draws, rng: rng).with_index do |row, draw|
+          best = row.max
+          leaders = []
+          row.each_with_index do |value, index|
+            samples[index][draw] = value
+            losses[index] += best - value
+            leaders << index if value == best
+          end
+          share = 1.0 / leaders.size
+          leaders.each { |index| wins[index] += share }
+        end
+
+        { samples: samples, wins: wins, losses: losses }
+      end
+
+      private
+
+      def count(draws)
+        raise ArgumentError, "draws must be an Integer >= 0, got #{draws.inspect}" unless draws.is_a?(Integer) && draws >= 0
+
+        draws
+      end
+    end
+  end
+end

data/lib/skiftet_statistical/significance.rb

@@ -5,8 +5,18 @@ module SkiftetStatistical
   # two-proportion z-test, Welch's t-test and the normal CDF that were previously
   # re-implemented (inconsistently) across mej.la's AbTestAnalyzer and skram.la's
   # CRM::AbTestAnalysis. One correct, exact (erf-based) normal CDF — no polynomial
-  # approximations.
+  # approximations. Since 0.2 also the normal quantile, the chi-squared survival
+  # and the sample-ratio check (f1's copies, one home).
   module Significance
+    # normal_quantile bisects [-40, 40]: 0.5 * erfc(40 / sqrt(2)) underflows to 0.0,
+    # so every p a Float can hold strictly inside (0, 1) has its z in there.
+    NORMAL_QUANTILE_LIMIT = 40.0
+    # 60 halvings leave a bracket of 80 / 2**60 (about 7e-17).
+    NORMAL_QUANTILE_STEPS = 60
+    # Under this many observations a sample-ratio check is not run: too few to tell
+    # a broken split from chance.
+    SAMPLE_RATIO_MINIMUM_TOTAL = 200
+
     module_function
 
     # Standard normal cumulative distribution Phi(z), exact via erf.
@@ -49,6 +59,57 @@ module SkiftetStatistical
       Result.new(statistic: t, p_value: two_tailed_p_value(t))
     end
 
+    # The z where the standard normal distribution function is p (the probit), by
+    # bisection on erfc. -Infinity at p = 0 and Infinity at p = 1.
+    def normal_quantile(p)
+      p = p.to_f
+      raise ArgumentError, "p must be between 0 and 1, got #{p}" unless p.between?(0.0, 1.0)
+      return -Float::INFINITY if p.zero?
+      return Float::INFINITY if p == 1.0
+
+      low = -NORMAL_QUANTILE_LIMIT
+      high = NORMAL_QUANTILE_LIMIT
+      NORMAL_QUANTILE_STEPS.times do
+        middle = (low + high) / 2
+        0.5 * Math.erfc(-middle / Math.sqrt(2.0)) < p ? low = middle : high = middle
+      end
+      (low + high) / 2
+    end
+
+    # P(X >= statistic) for X ~ chi-squared(degrees_of_freedom): the regularized
+    # upper incomplete gamma Q(dof / 2, statistic / 2). 1.0 for a statistic <= 0.
+    def chi_squared_survival(statistic, degrees_of_freedom)
+      return 1.0 if statistic <= 0.0
+
+      Gamma.regularized_upper(degrees_of_freedom / 2.0, statistic / 2.0)
+    end
+
+    # The sample-ratio mismatch check: does each arm's observed count fit the count
+    # its share of the traffic leads us to expect? Pearson's chi-squared over the
+    # arms expected to get traffic, on k - 1 degrees of freedom. A small p means the
+    # split does not do what its weights say (a broken arm, a cache, a bot).
+    #
+    #   Significance.sample_ratio_p_value(observed: { control: 700, a: 300 },
+    #                                     expected: { control: 500.0, a: 500.0 }) # => ~1.1e-36
+    #
+    # The arms are observed's keys (expected is read by the same keys, missing = 0).
+    # nil (not tested) under minimum_total observations or with fewer than two arms
+    # expected to get traffic; 0.0 when an arm expected to get nothing got some,
+    # a mismatch no count explains.
+    def sample_ratio_p_value(observed:, expected:, minimum_total: SAMPLE_RATIO_MINIMUM_TOTAL)
+      total = observed.values.sum
+      return nil if total < minimum_total
+
+      cells = observed.keys.map { |arm| [ observed.fetch(arm).to_f, expected.fetch(arm, 0.0).to_f ] }
+      return 0.0 if cells.any? { |seen, wanted| wanted <= 0.0 && seen.positive? }
+
+      cells = cells.select { |_seen, wanted| wanted.positive? }
+      return nil if cells.size < 2
+
+      statistic = cells.sum { |seen, wanted| ((seen - wanted)**2) / wanted }
+      chi_squared_survival(statistic, cells.size - 1)
+    end
+
     # The outcome of a significance test: the test statistic and its two-tailed
     # p-value, with convenience predicates for the usual confidence levels.
     Result = Struct.new(:statistic, :p_value, keyword_init: true) do

data/lib/skiftet_statistical/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module SkiftetStatistical
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/skiftet_statistical.rb

@@ -2,6 +2,9 @@
 
 require_relative "skiftet_statistical/version"
 require_relative "skiftet_statistical/sampler"
+require_relative "skiftet_statistical/gamma"
+require_relative "skiftet_statistical/beta_function"
+require_relative "skiftet_statistical/joint_beta"
 require_relative "skiftet_statistical/descriptive"
 require_relative "skiftet_statistical/significance"
 require_relative "skiftet_statistical/arm"
@@ -11,6 +14,9 @@ require_relative "skiftet_statistical/policies/thompson_sampling"
 require_relative "skiftet_statistical/policies/ucb1"
 require_relative "skiftet_statistical/policies/softmax"
 require_relative "skiftet_statistical/bandit"
+require_relative "skiftet_statistical/bandit/weighted_posterior"
+require_relative "skiftet_statistical/bandit/allocator"
+require_relative "skiftet_statistical/bandit/win_hold"
 
 # Skiftet's shared statistics toolkit — a home for reusable, app-independent
 # statistical analysis code across the workspace.
@@ -18,10 +24,17 @@ require_relative "skiftet_statistical/bandit"
 # Modules:
 # - {Descriptive} — mean, variance, standard deviation, percentiles/median.
 # - {Significance} — A/B significance testing (two-proportion z-test, Welch's
-#   t-test, normal CDF / two-tailed p-values).
+#   t-test, normal CDF / two-tailed p-values, normal quantile, chi-squared
+#   survival, the sample-ratio check).
+# - {Gamma} — the regularized incomplete gamma functions P(a, x) and Q(a, x).
+# - {BetaFunction} — the regularized incomplete beta function and its quantile.
 # - {Sampler} — Gamma/Beta/Gaussian random sampling (RNG-injectable).
+# - {JointBeta} — joint Monte Carlo draws over Beta posteriors (P(best), loss).
 # - {Bandit} + {Policies} — multi-armed bandit (Thompson Sampling, epsilon-greedy,
 #   UCB1, Softmax) for online explore/exploit decisions.
+# - {Bandit::Allocator}, {Bandit::WinHold}, {Bandit::WeightedPosterior} — an A/B
+#   test's Thompson-sampling allocation with a floor, a step limit, an automatic
+#   win and a hold on it.
 #
 #   bandit = SkiftetStatistical.bandit(arms: %w[facebook whatsapp x])
 #   choice = bandit.select          # which arm to play now

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: skiftet_statistical
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Skiftet
@@ -27,7 +27,13 @@ files:
 - lib/skiftet_statistical.rb
 - lib/skiftet_statistical/arm.rb
 - lib/skiftet_statistical/bandit.rb
+- lib/skiftet_statistical/bandit/allocator.rb
+- lib/skiftet_statistical/bandit/weighted_posterior.rb
+- lib/skiftet_statistical/bandit/win_hold.rb
+- lib/skiftet_statistical/beta_function.rb
 - lib/skiftet_statistical/descriptive.rb
+- lib/skiftet_statistical/gamma.rb
+- lib/skiftet_statistical/joint_beta.rb
 - lib/skiftet_statistical/policies/base.rb
 - lib/skiftet_statistical/policies/epsilon_greedy.rb
 - lib/skiftet_statistical/policies/softmax.rb