sqa-bi 0.1.0

data/docs/guide/predictor.md

@@ -0,0 +1,191 @@
+# The Predictor
+
+`SQA::BI::TimeSeriesPredictor` is the facade over
+[`Prior`](prior.md), [`Likelihood`](likelihood.md) and
+[`Posterior`](posterior.md). Most code should use it rather than the three
+directly.
+
+## Construction
+
+```ruby
+predictor = SQA::BI::TimeSeriesPredictor.new(
+  outcomes:            [-2, -1, 0, 1, 2],
+  bandwidth:           1.0,
+  prior_probabilities: nil,    # nil = uniform
+  update_prior:        true
+)
+```
+
+Or the shorthand, which takes the same keywords:
+
+```ruby
+predictor = SQA::BI.predictor(outcomes: [-2, -1, 0, 1, 2], bandwidth: 1.0)
+```
+
+| Parameter | Default | Effect |
+|---|---|---|
+| `outcomes:` | required | the discrete set; sorted on construction |
+| `bandwidth:` | `1.0` | KDE kernel width — see [Tuning](tuning.md) |
+| `prior_probabilities:` | `nil` | explicit prior; `nil` means uniform |
+| `update_prior:` | `true` | relearn the prior from observed frequencies as you train |
+
+## Training
+
+```ruby
+predictor.train([1.0, 2.0, 3.0], outcome: 1)
+```
+
+Returns `self`, so it chains:
+
+```ruby
+predictor
+  .train([1.0, 2.0, 3.0], outcome:  1)
+  .train([1.1, 2.1, 2.9], outcome:  1)
+  .train([-1.0, -2.0, 0.5], outcome: -2)
+```
+
+In bulk:
+
+```ruby
+predictor.train_batch([
+  { features: [1.0, 2.0, 3.0], outcome: 1 },
+  { features: [1.1, 2.1, 2.9], outcome: 1 }
+])
+```
+
+### Validation
+
+Two checks run on every `train`:
+
+**Features** must be an `Array` of `Numeric`, and must match the dimension
+established by the first call:
+
+```
+ArgumentError: Features must be array of numbers
+ArgumentError: Feature dimension mismatch: expected 3, got 4
+```
+
+**Outcome** must be in the declared set:
+
+```
+ArgumentError: Invalid outcome: 7. Must be one of [-2, -1, 0, 1, 2]
+```
+
+That second check is worth more than it looks — a typo'd outcome would
+otherwise create a silent phantom class that the model can never predict and you
+would never notice.
+
+## Predicting
+
+```ruby
+posterior = predictor.predict([1.05, 2.05, 3.0])   # => Posterior
+outcome   = predictor.predict_outcome([1.05, 2.05, 3.0])  # => 1  (MAP only)
+probs     = predictor.predict_proba([1.05, 2.05, 3.0])    # => Hash
+```
+
+`predict` is the one to reach for. The other two throw away the uncertainty,
+which is the main thing this library exists to give you.
+
+## Inspecting state
+
+```ruby
+predictor.trained?              # => true
+predictor.training_size         # => 5391
+predictor.training_distribution # => {-2 => 47, -1 => 1_305, 0 => 1_297, ...}
+predictor.outcomes              # => [-2, -1, 0, 1, 2]
+predictor.feature_dimension     # => 5
+puts predictor.summary
+```
+
+```
+TimeSeriesPredictor Summary:
+=============================
+Outcomes: [-2, -1, 0, 1, 2]
+Feature Dimension: 5
+Training Samples: 5391
+Bandwidth: 0.5
+Update Prior: true
+
+Prior Distribution:
+Prior(-2: 0.009, -1: 0.242, 0: 0.241, 1: 0.245, 2: 0.007)
+Prior Entropy: 1.893 bits
+
+Training Distribution:
+  -2: 47
+  -1: 1305
+  ...
+```
+
+!!! tip "Read `training_distribution` before trusting anything"
+    Badly imbalanced classes are the most common reason a model looks broken.
+    In the stock demo the extreme bands (`-2`, `+2`) hold under 1% of samples
+    each — the KDE simply has too few neighbours there to estimate a density,
+    and those outcomes are effectively never predicted. That is visible
+    immediately in this hash, and invisible in an accuracy score.
+
+## Resetting and retuning
+
+```ruby
+predictor.reset!                    # clears observations AND prior
+predictor.reset!(keep_prior: true)  # clears observations, keeps the prior
+predictor.bandwidth = 0.5           # rebuilds the KDE, keeps observations
+```
+
+`bandwidth=` is what makes a bandwidth sweep cheap — the observations survive,
+only the estimator is rebuilt. See [Tuning](tuning.md).
+
+## A realistic workflow
+
+```ruby
+require "sqa/bi"
+
+BANDS = [
+  [-Float::INFINITY, -0.05, -2],
+  [-0.05,            -0.005, -1],
+  [-0.005,            0.005,  0],
+  [ 0.005,            0.05,   1],
+  [ 0.05,   Float::INFINITY,  2]
+]
+
+def categorize(change)
+  BANDS.find { |low, high, _| change > low && change <= high }.last
+end
+
+predictor = SQA::BI.predictor(outcomes: [-2, -1, 0, 1, 2], bandwidth: 0.5)
+
+# Features are deltas; the target is the NEXT period's move.
+rows.each_cons(2) do |today, tomorrow|
+  predictor.train(
+    [today[:close_delta], today[:volume_delta], today[:rsi_delta]],
+    outcome: categorize(tomorrow[:close_change])
+  )
+end
+
+posterior = predictor.predict([latest[:close_delta],
+                               latest[:volume_delta],
+                               latest[:rsi_delta]])
+
+if posterior.confidence > 0.3
+  puts "Signal: #{posterior.max_outcome} at #{(posterior.probability(posterior.max_outcome) * 100).round(1)}%"
+else
+  puts "No signal — posterior is near uniform"
+end
+```
+
+!!! danger "Train on the past, test on the future"
+    Split chronologically, never randomly. A random split lets tomorrow's
+    observations inform today's prediction, and the resulting accuracy is
+    fiction. The stock demo splits 80/20 by date for exactly this reason.
+
+## Known limitation — one horizon per predictor
+
+A predictor answers one question at one horizon. Predicting k = 1…10 days ahead
+means ten predictors, and they will not share strength even though `P(up at day
+3)` and `P(up at day 4)` are strongly correlated. Extending
+`TimeSeriesPredictor` to handle multiple horizons jointly is
+[an open thread](../EXPLORATION.md).
+
+## API
+
+See [`SQA::BI::TimeSeriesPredictor`](../api/time-series-predictor.md) for the
+full method list.

data/docs/guide/prior.md

@@ -0,0 +1,141 @@
+# The Prior
+
+`SQA::BI::Prior` holds `P(outcome)` — what you believe before looking at the
+current feature vector.
+
+## Constructing one
+
+### Uniform
+
+```ruby
+prior = SQA::BI::Prior.new([-2, -1, 0, 1, 2])
+prior.probability(0)   # => 0.2
+prior.entropy          # => 2.322  (maximum for 5 outcomes)
+```
+
+Every outcome gets `1 / n`. This is the honest starting point when you know
+nothing, and it is what `TimeSeriesPredictor` uses by default.
+
+### Custom
+
+```ruby
+prior = SQA::BI::Prior.new(
+  [-2, -1, 0, 1, 2],
+  { -2 => 0.10, -1 => 0.15, 0 => 0.50, 1 => 0.15, 2 => 0.10 }
+)
+```
+
+Validation is strict and happens at construction:
+
+| Condition | Result |
+|---|---|
+| An outcome has no probability | `ArgumentError: Missing probabilities for outcomes: [...]` |
+| Probabilities don't sum to 1.0 (±1e-6) | `ArgumentError: Probabilities must sum to 1.0, got 0.97` |
+| Any probability is negative | `ArgumentError: Probabilities must be non-negative` |
+
+Outcomes are sorted on construction, so `to_s` and every downstream report read
+in order regardless of the hash's insertion order.
+
+## Learning from observations
+
+```ruby
+observations = { -2 => 5, -1 => 10, 0 => 20, 1 => 12, 2 => 3 }
+updated = prior.update_from_observations(observations)
+```
+
+This returns a **new** `Prior` — the original is untouched. Every method on this
+class is non-mutating, which makes priors safe to share and trivial to test.
+
+### Laplace smoothing
+
+The update is not a raw frequency count:
+
+$$
+P(\text{outcome}) = \frac{\text{count} + \alpha}{\text{total} + \alpha n}
+$$
+
+with `α = 1.0` by default, adjustable via `smoothing:`.
+
+!!! quote "Why add-one smoothing is not optional"
+    A raw frequency of zero produces a prior probability of zero — and a zero
+    prior can **never recover**, no matter how much contradicting evidence
+    arrives later. The posterior multiplies prior by likelihood, and anything
+    times zero is zero.
+
+    This is Cromwell's rule, and it is the same reason
+    [LLM likelihoods are clamped](../llm/likelihood-estimation.md) away from 0
+    and 1. Smoothing buys the model the ability to change its mind.
+
+Raising `smoothing:` pulls the prior toward uniform, which is what you want when
+counts are small and you don't trust them yet:
+
+```ruby
+prior.update_from_observations(observations, smoothing: 5.0)
+```
+
+## Combining priors
+
+```ruby
+market_view = SQA::BI::Prior.new(outcomes, { ... })
+analyst_view = SQA::BI::Prior.new(outcomes, { ... })
+
+blended = market_view.combine(analyst_view, weight: 0.7)
+```
+
+A weighted average: `weight` applies to the receiver, `1 - weight` to the
+argument. Both priors must span the same outcomes or it raises
+`ArgumentError: Cannot combine priors with different outcomes`.
+
+This is the natural place to mix a data-driven prior with an
+[LLM-elicited one](../llm/prior-elicitation.md) rather than choosing between
+them.
+
+## Entropy
+
+```ruby
+uniform = SQA::BI::Prior.new([-2, -1, 0, 1, 2])
+uniform.entropy   # => 2.322  maximally uncertain
+
+peaked = SQA::BI::Prior.new([-2, -1, 0, 1, 2],
+                            { -2 => 0.0, -1 => 0.0, 0 => 1.0, 1 => 0.0, 2 => 0.0 })
+peaked.entropy    # => 0.0    completely certain
+```
+
+Shannon entropy in bits. Useful as a sanity check on an elicited prior: if an
+LLM hands back something with entropy near zero, it has claimed certainty it
+cannot possibly have, and you should distrust it.
+
+## Automatic updating inside the predictor
+
+`TimeSeriesPredictor` updates its prior from observed frequencies as you train,
+but only once there is enough data to be worth it:
+
+```ruby
+if @update_prior && @likelihood.size >= @outcomes.size
+  @prior = @prior.update_from_observations(@likelihood.outcome_counts)
+end
+```
+
+The `size >= outcomes.size` guard prevents the first two or three observations
+from stamping a lopsided prior onto the model. Disable the behavior entirely
+when you want a fixed prior — which is exactly what you want after eliciting
+one:
+
+```ruby
+predictor = SQA::BI::TimeSeriesPredictor.new(
+  outcomes: OUTCOMES,
+  prior_probabilities: elicited.probabilities,
+  update_prior: false
+)
+```
+
+!!! note "The prior fades on its own"
+    With `update_prior: true` and growing data, the learned prior converges on
+    the empirical base rate. And regardless of the prior, as observations
+    accumulate the likelihood term increasingly dominates the product. A prior
+    matters most when data is scarce — which is precisely when you have the
+    least to go on and need it most.
+
+## API
+
+See [`SQA::BI::Prior`](../api/prior.md) for the full method list.

data/docs/guide/tuning.md

@@ -0,0 +1,156 @@
+# Tuning and Modeling Choices
+
+Four decisions determine whether this works: **bandwidth**, **outcome bands**,
+**feature design**, and **how you evaluate**. Only the first is a parameter; the
+other three are modeling, and they matter more.
+
+## Bandwidth
+
+The kernel width `h`. It is scale-dependent, so there is no universally good
+value — only a value appropriate to *your* features.
+
+### Sweeping it
+
+Because `bandwidth=` rebuilds the estimator from existing observations, a sweep
+is cheap:
+
+```ruby
+predictor = SQA::BI.predictor(outcomes: OUTCOMES, bandwidth: 1.0)
+predictor.train_batch(training_set)
+
+[0.1, 0.25, 0.5, 1.0, 2.0, 4.0].each do |h|
+  predictor.bandwidth = h
+
+  correct = test_set.count do |row|
+    predictor.predict_outcome(row[:features]) == row[:outcome]
+  end
+
+  mean_confidence = test_set.sum { predictor.predict(_1[:features]).confidence } / test_set.size
+
+  puts format("h=%-5s accuracy=%.3f  mean confidence=%.3f",
+              h, correct.to_f / test_set.size, mean_confidence)
+end
+```
+
+Watch both columns. Accuracy alone will happily pick a bandwidth that is
+confidently wrong.
+
+### Reading the failure modes
+
+| Symptom | Likely cause |
+|---|---|
+| Every posterior is near-uniform | `h` too large — outcomes have smeared together |
+| Confidence is high but accuracy is at chance | `h` too small — memorizing training points |
+| One outcome dominates every prediction | class imbalance, not bandwidth |
+| Predictions barely move from the prior | features carry no signal — check `kl_divergence_from_prior` |
+
+That last row is worth checking before touching `h` at all. If mean KL
+divergence across your test set is near zero, no bandwidth will save you; the
+features are the problem.
+
+### A starting heuristic
+
+Scale `h` to the typical spread of your features. If each feature is a
+percentage change mostly within ±2, start near `0.5`. If they are raw prices in
+the hundreds, either normalize them — strongly preferred — or start `h` in the
+tens.
+
+!!! tip "Normalize instead of compensating"
+    A single bandwidth applies to a Euclidean distance across *all* dimensions
+    at once. If one feature ranges over ±200 and another over ±0.5, the first
+    completely dominates the distance and the second may as well not exist.
+    Standardize features to comparable scales and the bandwidth becomes a single
+    meaningful knob instead of a compromise.
+
+## Outcome bands
+
+Where you cut the continuous target into discrete bands decides what the model
+can learn.
+
+```ruby
+BANDS = [
+  [-Float::INFINITY, -0.05,  -2],   # big down
+  [-0.05,            -0.005, -1],   # small down
+  [-0.005,            0.005,  0],   # sideways
+  [ 0.005,            0.05,   1],   # small up
+  [ 0.05,   Float::INFINITY,  2]    # big up
+]
+```
+
+Two competing pressures:
+
+**Wide bands** collect plenty of observations per class, so density estimation
+is stable — but the prediction says little ("it went up somewhat").
+
+**Narrow bands** carry more information per prediction — but starve. In the
+stock demo the extreme bands hold under 1% of samples each, and the model
+essentially never predicts them. Check `training_distribution` and treat any
+class under ~5% as decorative.
+
+!!! note "Asymmetric bands are legitimate"
+    Nothing requires symmetry. If a 2% drop costs you more than a 2% gain earns,
+    cut the downside more finely. The bands should encode the decisions you
+    actually face, not statistical tidiness.
+
+## Feature design
+
+This is where the leverage is.
+
+**Use deltas, not levels.** A price of 187.40 is not transferable; a change of
++0.8% is. Absolute levels make the KDE memorize a regime that will not recur.
+
+**Keep the dimension modest.** Every added dimension dilutes Euclidean distance
+— the curse of dimensionality bites KDE hard. Three to six informative features
+beat twenty mediocre ones.
+
+**Drop redundant features.** Two features that move together add distance
+without adding information, and effectively double-weight whatever they both
+measure.
+
+**Check that each feature moves the posterior.** Add one, measure mean KL
+divergence over a held-out set, keep it only if the number rises.
+
+```ruby
+def mean_information_gain(predictor, test_set)
+  test_set.sum { predictor.predict(_1[:features]).kl_divergence_from_prior } / test_set.size
+end
+```
+
+## Evaluation
+
+**Split chronologically.** Always. A random split leaks the future into the
+past and produces accuracy that does not survive contact with reality.
+
+**Compare against the base rate, not zero.** If 40% of days are "small up",
+40% accuracy is worthless. The honest baseline is always-predict-the-majority.
+
+**Look at the confusion matrix.** Accuracy hides the shape of the errors. The
+stock demo prints one, and it shows the model collapsing onto the middle three
+bands — visible instantly in the matrix, invisible in a single number.
+
+**Check calibration.** Bucket predictions by confidence and measure accuracy per
+bucket. If they do not rise together, the confidence number is noise.
+
+!!! danger "The failure mode this library makes easy"
+    A Bayesian posterior is internally coherent by construction. That coherence
+    is seductive: the numbers look principled, the entropy is precise, the
+    summary reads like a real report. None of it establishes that the model
+    predicts anything. Backtest first, believe second.
+
+## Prior selection
+
+With `update_prior: true` (the default) the prior converges on the empirical
+base rate as data accumulates, and you can mostly leave it alone.
+
+Set an explicit prior when:
+
+- **You are cold-starting.** Few observations, but you know something about the
+  domain. Consider [LLM elicitation](../llm/prior-elicitation.md).
+- **The regime just changed.** The historical base rate is actively misleading —
+  learning the prior from pre-change data will fight you.
+- **You are testing prior sensitivity.** Run the same features under several
+  priors; if the posterior swings wildly, your evidence is weak and you should
+  say so rather than picking the prior you like.
+
+When you supply an elicited prior, set `update_prior: false` — otherwise
+training will quietly overwrite the knowledge you just injected.

data/docs/guide/uncertainty.md

@@ -0,0 +1,148 @@
+# Quantifying Uncertainty
+
+The reason to compute a posterior instead of a label is that the posterior knows
+how sure it is. Three measures, each answering a different question.
+
+## Entropy — how spread out is the belief?
+
+```ruby
+posterior.entropy   # => 1.371 bits
+```
+
+$$
+H = -\sum_{i} p_i \log_2 p_i
+$$
+
+| Distribution over 5 outcomes | Entropy |
+|---|---|
+| Certain — `{1 => 1.0}` | 0.000 |
+| Confident — `{1 => 0.74, ...}` | ≈ 1.37 |
+| Uniform — all `0.2` | 2.322 |
+
+The maximum is `log₂(n)`: **2.322** for five outcomes, **1.585** for three.
+Entropy is absolute, in bits, and comparable only across distributions with the
+same number of outcomes.
+
+## Confidence — the normalized version
+
+```ruby
+posterior.confidence   # => 0.412
+```
+
+$$
+\text{confidence} = 1 - \frac{H}{\log_2 n}
+$$
+
+Always in `[0, 1]`, regardless of how many outcomes you have, which makes it the
+right thing to threshold on:
+
+```ruby
+if posterior.confidence < 0.25
+  :abstain
+else
+  posterior.max_outcome
+end
+```
+
+!!! warning "Confidence is not accuracy"
+    A confidence of 0.9 means the distribution is *sharp*, not that it is
+    *right*. A model with a badly wrong prior, or features that encode a
+    spurious correlation, will be sharply and consistently wrong. Confidence
+    measures internal coherence; only a backtest measures correctness.
+
+??? tip "Calibrating a threshold"
+    Don't pick 0.25 because it appears above. Bucket your backtest predictions
+    by confidence and measure accuracy per bucket:
+
+    ```ruby
+    results.group_by { |r| (r[:confidence] * 10).floor / 10.0 }
+           .transform_values { |rs| rs.count { _1[:correct] } / rs.size.to_f }
+    ```
+
+    If accuracy doesn't rise with confidence, the confidence number is not
+    carrying information and thresholding on it will not help.
+
+## KL divergence — how much did this evidence teach me?
+
+```ruby
+posterior.kl_divergence_from_prior   # => 0.943 bits
+```
+
+$$
+D_{KL}(\text{posterior} \parallel \text{prior}) =
+\sum_i p_i \log_2 \frac{p_i}{q_i}
+$$
+
+Always ≥ 0. Zero means the posterior equals the prior — the features moved your
+belief not at all.
+
+This is the measure with no equivalent in a plain classifier, and it answers a
+genuinely different question. A prediction can be *confident* while teaching you
+*nothing*, if the prior was already confident:
+
+| Confidence | KL divergence | Reading |
+|---|---|---|
+| High | High | Sharp belief, driven by the evidence — the good case |
+| High | Low | Sharp, but it is just your prior talking |
+| Low | High | Evidence moved you a lot, toward uncertainty — informative! |
+| Low | Low | Nothing happened |
+
+That third row is the underrated one. Evidence that *destroys* a confident prior
+is highly informative even though the result is uncertain.
+
+### Finding which evidence mattered
+
+When [chaining updates](posterior.md#chaining-updates-across-several-pieces-of-evidence),
+record the KL divergence at each step:
+
+```ruby
+evidence_list.each do |evidence|
+  likelihoods = estimator.likelihoods(evidence)
+  posterior   = SQA::BI::Posterior.new(prior, likelihoods)
+
+  puts "#{evidence}: #{posterior.kl_divergence_from_prior.round(3)} bits"
+
+  prior = SQA::BI::Prior.new(hypotheses.keys, posterior.to_h)
+end
+```
+
+In the [diagnosis demo](../llm/likelihood-estimation.md) this prints per-update
+gains from 0.025 to 0.959 bits — and the large values land precisely on the two
+facts that reversed the conclusion. The engine tells you not just what it
+believes but *why*.
+
+## Sampling — carrying uncertainty downstream
+
+```ruby
+posterior.samples(10_000, rng: Random.new(42))
+```
+
+Use this when the next stage of your pipeline cannot accept a distribution.
+Instead of collapsing to `max_outcome` and pretending, sample repeatedly and
+look at the spread of results:
+
+```ruby
+returns = posterior.samples(10_000).map { |outcome| simulate_pnl(outcome) }
+returns.sum / returns.size              # expected value
+returns.sort[(returns.size * 0.05).to_i] # 5th percentile — the downside
+```
+
+The point estimate would have given you the first number and hidden the second.
+
+## Reading all of it at once
+
+```ruby
+puts posterior.summary
+```
+
+```
+Most Likely: 1 (74.0%)
+Confidence: 41.2%
+Entropy: 1.371 bits
+KL Divergence from Prior: 0.943 bits
+```
+
+A confidence of 41% next to a 74% top outcome is not a contradiction — 74% of
+the mass is on one outcome, but the remaining 26% is spread widely enough to
+keep overall entropy substantial. Both numbers are telling the truth about
+different aspects of the same distribution.