sqa-bi 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.github/workflows/docs.yml +55 -0
- data/.quality/reek_baseline.txt +5 -0
- data/.rubocop.yml +224 -0
- data/CHANGELOG.md +33 -0
- data/CLAUDE.md +128 -0
- data/COMMITS.md +196 -0
- data/LICENSE.txt +21 -0
- data/README.md +229 -0
- data/Rakefile +170 -0
- data/decision_support_techniques.md +391 -0
- data/docs/EXPLORATION.md +128 -0
- data/docs/api/index.md +66 -0
- data/docs/api/likelihood.md +78 -0
- data/docs/api/llm-elicitors.md +119 -0
- data/docs/api/llm-support.md +136 -0
- data/docs/api/posterior.md +106 -0
- data/docs/api/prior.md +81 -0
- data/docs/api/time-series-predictor.md +96 -0
- data/docs/assets/css/custom.css +25 -0
- data/docs/assets/diagrams/architecture.svg +75 -0
- data/docs/assets/diagrams/bayes-pipeline.svg +48 -0
- data/docs/assets/diagrams/kde.svg +52 -0
- data/docs/assets/diagrams/llm-bayes-loop.svg +58 -0
- data/docs/assets/diagrams/provider-resolution.svg +77 -0
- data/docs/assets/js/mathjax.js +18 -0
- data/docs/development.md +164 -0
- data/docs/examples/index.md +198 -0
- data/docs/getting-started/core-concepts.md +119 -0
- data/docs/getting-started/installation.md +112 -0
- data/docs/getting-started/quick-start.md +143 -0
- data/docs/guide/likelihood.md +132 -0
- data/docs/guide/posterior.md +135 -0
- data/docs/guide/predictor.md +191 -0
- data/docs/guide/prior.md +141 -0
- data/docs/guide/tuning.md +156 -0
- data/docs/guide/uncertainty.md +148 -0
- data/docs/index.md +110 -0
- data/docs/llm/index.md +124 -0
- data/docs/llm/likelihood-estimation.md +161 -0
- data/docs/llm/prior-elicitation.md +172 -0
- data/docs/llm/providers.md +184 -0
- data/docs/requirements.txt +8 -0
- data/lib/sqa/bi/likelihood.rb +167 -0
- data/lib/sqa/bi/llm_likelihood_estimator.rb +110 -0
- data/lib/sqa/bi/llm_prior_elicitor.rb +110 -0
- data/lib/sqa/bi/llm_support.rb +299 -0
- data/lib/sqa/bi/posterior.rb +189 -0
- data/lib/sqa/bi/prior.rb +135 -0
- data/lib/sqa/bi/time_series_predictor.rb +219 -0
- data/lib/sqa/bi/version.rb +7 -0
- data/lib/sqa/bi.rb +57 -0
- data/mkdocs.yml +174 -0
- metadata +272 -0
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# LLM Elicitors
|
|
2
|
+
|
|
3
|
+
The two classes that turn language into numbers. Both accept an injectable
|
|
4
|
+
`chat:`, and both expose their prompt building and response parsing as public,
|
|
5
|
+
pure methods so they can be tested with no network.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## SQA::BI::LlmPriorElicitor
|
|
10
|
+
|
|
11
|
+
A [`Prior`](prior.md) from a natural-language description of a situation.
|
|
12
|
+
Narrative: [Prior Elicitation](../llm/prior-elicitation.md).
|
|
13
|
+
|
|
14
|
+
### `.new(outcomes:, outcome_descriptions: {}, chat: nil, model: nil, provider: nil)`
|
|
15
|
+
|
|
16
|
+
| Parameter | Type | Description |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| `outcomes:` | `Array` | the outcome set the prior ranges over |
|
|
19
|
+
| `outcome_descriptions:` | `Hash` | `{outcome => "human label"}` used in the prompt |
|
|
20
|
+
| `chat:` | `#ask, nil` | injectable chat; defaults to a lazily built `RubyLLM::Chat` |
|
|
21
|
+
| `model:` | `String, nil` | model id; `nil` resolves per provider |
|
|
22
|
+
| `provider:` | `Symbol, nil` | `:lms`, `:apfel`, `:cloud`; `nil` auto-detects |
|
|
23
|
+
|
|
24
|
+
### `#elicit(context)` → `Prior`
|
|
25
|
+
|
|
26
|
+
Asks the LLM for relative weights over the outcomes, given a prose description,
|
|
27
|
+
then floors, normalizes and validates them.
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
prior = elicitor.elicit("The Fed unexpectedly cut rates by 50bp...")
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### `#prior_from_response(content)` → `Prior`
|
|
34
|
+
|
|
35
|
+
Pure. `extract_json` → `rekey_to_outcomes` → `normalize_distribution` → `Prior`.
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
elicitor.prior_from_response('{"-1": 10, "0": 20, "1": 70}')
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### `#build_prompt(context)` → `String`
|
|
42
|
+
|
|
43
|
+
Pure. Public so the wording can be tested and iterated on.
|
|
44
|
+
|
|
45
|
+
### Attributes
|
|
46
|
+
|
|
47
|
+
`outcomes`, `outcome_descriptions`, `model`, `provider`
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## SQA::BI::LlmLikelihoodEstimator
|
|
52
|
+
|
|
53
|
+
`P(evidence | hypothesis)` for textual evidence. Narrative:
|
|
54
|
+
[Likelihood Estimation](../llm/likelihood-estimation.md).
|
|
55
|
+
|
|
56
|
+
### `.new(hypotheses:, chat: nil, model: nil, provider: nil)`
|
|
57
|
+
|
|
58
|
+
| Parameter | Type | Description |
|
|
59
|
+
|---|---|---|
|
|
60
|
+
| `hypotheses:` | `Hash` | `{id => description}`; ids become the outcome keys |
|
|
61
|
+
| `chat:` | `#ask, nil` | injectable chat |
|
|
62
|
+
| `model:` | `String, nil` | model id |
|
|
63
|
+
| `provider:` | `Symbol, nil` | `:lms`, `:apfel`, `:cloud` |
|
|
64
|
+
|
|
65
|
+
### `#likelihoods(evidence)` → `Hash`
|
|
66
|
+
|
|
67
|
+
`{hypothesis_id => Float}`, each clamped to `(0, 1)`.
|
|
68
|
+
|
|
69
|
+
```ruby
|
|
70
|
+
estimator.likelihoods("Error rate spiked 2 minutes after deploy")
|
|
71
|
+
# => { bad_deploy: 0.9, database: 0.2, network: 0.15 }
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
!!! important "These need not sum to 1"
|
|
75
|
+
Each is an independent judgment: *if this hypothesis were true, how
|
|
76
|
+
surprising would this evidence be?*
|
|
77
|
+
|
|
78
|
+
### `#likelihoods_from_response(content)` → `Hash`
|
|
79
|
+
|
|
80
|
+
Pure. `extract_json` → `rekey_to_outcomes` → `clamp_likelihood` on every value.
|
|
81
|
+
|
|
82
|
+
```ruby
|
|
83
|
+
estimator.likelihoods_from_response('{"a": 1.0, "b": 0.0}')
|
|
84
|
+
# => { a: 0.999, b: 0.001 }
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### `#build_prompt(evidence)` → `String`
|
|
88
|
+
|
|
89
|
+
Pure.
|
|
90
|
+
|
|
91
|
+
### Attributes
|
|
92
|
+
|
|
93
|
+
`hypotheses`, `model`, `provider`
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Testing both without a network
|
|
98
|
+
|
|
99
|
+
```ruby
|
|
100
|
+
class FakeChat
|
|
101
|
+
Response = Struct.new(:content)
|
|
102
|
+
attr_reader :last_prompt
|
|
103
|
+
|
|
104
|
+
def initialize(content) = @content = content
|
|
105
|
+
|
|
106
|
+
def ask(prompt)
|
|
107
|
+
@last_prompt = prompt
|
|
108
|
+
Response.new(@content)
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
chat = FakeChat.new('{"-1": 10, "0": 20, "1": 70}')
|
|
113
|
+
elicitor = SQA::BI::LlmPriorElicitor.new(outcomes: [-1, 0, 1], chat: chat)
|
|
114
|
+
|
|
115
|
+
elicitor.elicit("bullish setup").probability(1) # => 0.7
|
|
116
|
+
chat.last_prompt # assert on prompt wording
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
This is what the gem's own suite does — no test may reach a real provider.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# SQA::BI::LlmSupport
|
|
2
|
+
|
|
3
|
+
Shared plumbing for the classes that get numbers out of an LLM. Every method is
|
|
4
|
+
a `module_function`, so each is callable and testable in isolation.
|
|
5
|
+
|
|
6
|
+
```ruby
|
|
7
|
+
S = SQA::BI::LlmSupport
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
## Constants
|
|
11
|
+
|
|
12
|
+
| Constant | Value |
|
|
13
|
+
|---|---|
|
|
14
|
+
| `LMS_DEFAULT_BASE` | `http://localhost:1234/v1` |
|
|
15
|
+
| `APFEL_DEFAULT_BASE` | `http://127.0.0.1:11434/v1` |
|
|
16
|
+
| `CLOUD_DEFAULT_MODEL` | `claude-haiku-4-5` |
|
|
17
|
+
|
|
18
|
+
## Provider resolution
|
|
19
|
+
|
|
20
|
+
### `.resolve_provider` → `Symbol`
|
|
21
|
+
|
|
22
|
+
`:lms`, `:apfel`, or `:cloud`. Checks the environment override first, then
|
|
23
|
+
probes each local server in order. See [Providers](../llm/providers.md).
|
|
24
|
+
|
|
25
|
+
### `.server_alive?(base, timeout: 1)` → `Boolean`
|
|
26
|
+
|
|
27
|
+
`GET {base}/models`, true on a 2xx. Any exception is false.
|
|
28
|
+
|
|
29
|
+
### `.list_local_models(base)` → `Array<String>`
|
|
30
|
+
|
|
31
|
+
Model ids the server is serving right now. `[]` on any failure.
|
|
32
|
+
|
|
33
|
+
### `.choose_local_model(ids)` → `String, nil`
|
|
34
|
+
|
|
35
|
+
Rejects anything matching `/embed|ocr/i`, then prefers qwen, then gpt-oss, then
|
|
36
|
+
the first remaining id. `nil` when nothing is eligible.
|
|
37
|
+
|
|
38
|
+
### `.local_api_base(provider)` → `String, nil`
|
|
39
|
+
|
|
40
|
+
API base for `:lms` or `:apfel`; `nil` for `:cloud` or anything else.
|
|
41
|
+
|
|
42
|
+
### `.lms_api_base` / `.apfel_api_base` → `String`
|
|
43
|
+
|
|
44
|
+
Honor `LMS_API_BASE` / `APFEL_API_BASE`.
|
|
45
|
+
|
|
46
|
+
### `.env_value(*names)` → `String, nil`
|
|
47
|
+
|
|
48
|
+
First non-blank value among the named environment variables.
|
|
49
|
+
|
|
50
|
+
### `.env_model` → `String, nil`
|
|
51
|
+
|
|
52
|
+
`env_value('SQA_BI_LLM_MODEL', 'BI_LLM_MODEL')`.
|
|
53
|
+
|
|
54
|
+
## Visibility
|
|
55
|
+
|
|
56
|
+
### `.resolution_label(provider, model, base)` → `String`
|
|
57
|
+
|
|
58
|
+
Pure formatter. No I/O.
|
|
59
|
+
|
|
60
|
+
```ruby
|
|
61
|
+
S.resolution_label(:lms, "qwen/qwen3.8-27b", "http://localhost:1234/v1")
|
|
62
|
+
# => "lms — qwen/qwen3.8-27b at http://localhost:1234/v1"
|
|
63
|
+
|
|
64
|
+
S.resolution_label(:cloud, "claude-haiku-4-5", nil)
|
|
65
|
+
# => "cloud — claude-haiku-4-5"
|
|
66
|
+
|
|
67
|
+
S.resolution_label(:lms, nil, "http://localhost:1234/v1")
|
|
68
|
+
# => "lms — (no chat model available) at http://localhost:1234/v1"
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
### `.current_resolution(model = nil, provider: nil)` → `String`
|
|
72
|
+
|
|
73
|
+
Where the next chat would actually go. Probes the local servers, so it costs one
|
|
74
|
+
HTTP round trip. Pass `provider: :cloud` to skip the probe.
|
|
75
|
+
|
|
76
|
+
```ruby
|
|
77
|
+
S.current_resolution
|
|
78
|
+
# => "lms — qwen/qwen3.8-27b at http://localhost:1234/v1"
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Use it to surface the [silent cloud fallback](../llm/providers.md#making-resolution-visible).
|
|
82
|
+
|
|
83
|
+
## Chat construction
|
|
84
|
+
|
|
85
|
+
### `.build_chat(model = nil, provider: nil)` → `RubyLLM::Chat`
|
|
86
|
+
|
|
87
|
+
Requires `ruby_llm` lazily, configures it, resolves the provider, and returns a
|
|
88
|
+
chat. **Raises `SQA::BI::Error`** if `ruby_llm` cannot be loaded, or if a local
|
|
89
|
+
provider has no eligible chat model.
|
|
90
|
+
|
|
91
|
+
### `.local_chat(provider, model, base)` → `RubyLLM::Chat`
|
|
92
|
+
|
|
93
|
+
### `.require_ruby_llm` → `void`
|
|
94
|
+
|
|
95
|
+
Requires `ruby_llm`, converting both `LoadError` and `Gem::ConflictError` (which
|
|
96
|
+
is itself a `LoadError`) into one actionable `SQA::BI::Error`.
|
|
97
|
+
|
|
98
|
+
### `.configure_ruby_llm` → `void`
|
|
99
|
+
|
|
100
|
+
Registers the local provider gems and wires whichever cloud API keys are
|
|
101
|
+
present. Runs once; safe to call repeatedly.
|
|
102
|
+
|
|
103
|
+
### `.require_local_providers` → `void`
|
|
104
|
+
|
|
105
|
+
Requires both provider gems, swallowing `LoadError`. A missing gem disables only
|
|
106
|
+
that provider.
|
|
107
|
+
|
|
108
|
+
## Response plumbing
|
|
109
|
+
|
|
110
|
+
### `.extract_json(content)` → `Hash`
|
|
111
|
+
|
|
112
|
+
Pulls a Hash out of an already-parsed Hash, a raw JSON string, or prose
|
|
113
|
+
containing a JSON object (including inside a ```json fence).
|
|
114
|
+
|
|
115
|
+
**Raises `SQA::BI::Error`** when no JSON object is found or it is malformed.
|
|
116
|
+
|
|
117
|
+
### `.rekey_to_outcomes(raw, outcomes)` → `Hash`
|
|
118
|
+
|
|
119
|
+
JSON keys are always strings; outcomes may be integers or symbols. Matches on
|
|
120
|
+
`to_s`. Missing outcomes get `0.0`.
|
|
121
|
+
|
|
122
|
+
```ruby
|
|
123
|
+
S.rekey_to_outcomes({ "1" => 70, "0" => 20 }, [-1, 0, 1])
|
|
124
|
+
# => {-1 => 0.0, 0 => 20.0, 1 => 70.0}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
### `.normalize_distribution(weights, epsilon: 1e-6)` → `Hash`
|
|
128
|
+
|
|
129
|
+
Floors at zero, adds `epsilon`, normalizes to sum to 1. The epsilon is
|
|
130
|
+
[Cromwell's rule](../guide/prior.md#laplace-smoothing) — no outcome is ever
|
|
131
|
+
exactly zero.
|
|
132
|
+
|
|
133
|
+
### `.clamp_likelihood(value, floor: 0.001, ceiling: 0.999)` → `Float`
|
|
134
|
+
|
|
135
|
+
Keeps an overconfident `0.0` or `1.0` from zeroing out or saturating the
|
|
136
|
+
posterior in one update.
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# SQA::BI::Posterior
|
|
2
|
+
|
|
3
|
+
`P(outcome | features)` — the result of Bayes' theorem, plus the measures that
|
|
4
|
+
say how much to trust it.
|
|
5
|
+
|
|
6
|
+
Immutable; everything is computed at construction.
|
|
7
|
+
|
|
8
|
+
## Constructor
|
|
9
|
+
|
|
10
|
+
### `.new(prior, likelihoods)`
|
|
11
|
+
|
|
12
|
+
| Parameter | Type | Description |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| `prior` | `Prior` | the prior distribution |
|
|
15
|
+
| `likelihoods` | `Hash` | `{outcome => P(data \| outcome)}` |
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
SQA::BI::Posterior.new(prior, { -1 => 0.1, 0 => 0.3, 1 => 0.6 })
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Computes `likelihood × prior`, normalized. If the unnormalized total is `0.0`
|
|
22
|
+
it falls back to the prior rather than dividing by zero.
|
|
23
|
+
|
|
24
|
+
## Attributes
|
|
25
|
+
|
|
26
|
+
| Reader | Type | Description |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| `prior` | `Prior` | the prior it was built from |
|
|
29
|
+
| `likelihoods` | `Hash` | the likelihoods it was built from |
|
|
30
|
+
| `probabilities` | `Hash` | the normalized posterior |
|
|
31
|
+
|
|
32
|
+
## Reading the distribution
|
|
33
|
+
|
|
34
|
+
### `#probability(outcome)` → `Float`
|
|
35
|
+
|
|
36
|
+
**Alias:** `#[]`
|
|
37
|
+
|
|
38
|
+
### `#max_outcome` → outcome
|
|
39
|
+
|
|
40
|
+
The MAP estimate — highest posterior probability.
|
|
41
|
+
|
|
42
|
+
!!! tip "Read `confidence` alongside it"
|
|
43
|
+
`1` at 74% and `1` at 26% are different situations that `max_outcome`
|
|
44
|
+
reports identically.
|
|
45
|
+
|
|
46
|
+
### `#top_outcomes(n = 3)` → `Array<Array>`
|
|
47
|
+
|
|
48
|
+
`[[outcome, probability], ...]` sorted by probability **descending**.
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
posterior.top_outcomes(3) # => [[1, 0.74], [0, 0.15], [-1, 0.06]]
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
### `#to_a` → `Array<Array>`
|
|
55
|
+
|
|
56
|
+
`[[outcome, probability], ...]` sorted by **outcome** ascending — preserves the
|
|
57
|
+
ordinal scale, which is what you want for charting.
|
|
58
|
+
|
|
59
|
+
### `#to_h` → `Hash`
|
|
60
|
+
|
|
61
|
+
A copy of the probabilities.
|
|
62
|
+
|
|
63
|
+
## Uncertainty
|
|
64
|
+
|
|
65
|
+
### `#entropy` → `Float`
|
|
66
|
+
|
|
67
|
+
Shannon entropy in bits. `0.0` = certain; `log₂(n)` = uniform.
|
|
68
|
+
|
|
69
|
+
### `#confidence` → `Float`
|
|
70
|
+
|
|
71
|
+
`1 - entropy / log₂(n)`. Always in `[0, 1]`, so it is comparable across
|
|
72
|
+
different outcome-set sizes and is the right thing to threshold on.
|
|
73
|
+
|
|
74
|
+
### `#kl_divergence_from_prior` → `Float`
|
|
75
|
+
|
|
76
|
+
$$D_{KL} = \sum_i p_i \log_2 \frac{p_i}{q_i}$$
|
|
77
|
+
|
|
78
|
+
Bits of information this evidence added. Always ≥ 0; zero means the evidence
|
|
79
|
+
moved your belief not at all. See [Quantifying Uncertainty](../guide/uncertainty.md).
|
|
80
|
+
|
|
81
|
+
## Sampling
|
|
82
|
+
|
|
83
|
+
### `#sample(rng: Random.new)` → outcome
|
|
84
|
+
|
|
85
|
+
One draw by inverse-CDF sampling.
|
|
86
|
+
|
|
87
|
+
### `#samples(n, rng: Random.new)` → `Array`
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
posterior.samples(10_000, rng: Random.new(42))
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Pass `rng:` for reproducibility.
|
|
94
|
+
|
|
95
|
+
## Reporting
|
|
96
|
+
|
|
97
|
+
### `#to_s` → `String`
|
|
98
|
+
|
|
99
|
+
```
|
|
100
|
+
Posterior(-2: 0.010, -1: 0.060, 0: 0.150, 1: 0.740, 2: 0.040)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### `#summary` → `String`
|
|
104
|
+
|
|
105
|
+
Multi-line report: MAP outcome, confidence, entropy, KL divergence, and the full
|
|
106
|
+
distribution as both decimals and percentages.
|
data/docs/api/prior.md
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# SQA::BI::Prior
|
|
2
|
+
|
|
3
|
+
Prior probability distribution over discrete outcomes.
|
|
4
|
+
|
|
5
|
+
Immutable — every method that changes something returns a new instance.
|
|
6
|
+
|
|
7
|
+
## Constructor
|
|
8
|
+
|
|
9
|
+
### `.new(outcomes, probabilities = nil)`
|
|
10
|
+
|
|
11
|
+
| Parameter | Type | Description |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| `outcomes` | `Array` | discrete outcomes; sorted on construction |
|
|
14
|
+
| `probabilities` | `Hash, nil` | `{outcome => Float}`; `nil` gives a uniform prior |
|
|
15
|
+
|
|
16
|
+
```ruby
|
|
17
|
+
SQA::BI::Prior.new([-2, -1, 0, 1, 2])
|
|
18
|
+
SQA::BI::Prior.new([-1, 0, 1], { -1 => 0.2, 0 => 0.5, 1 => 0.3 })
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
**Raises `ArgumentError`** when probabilities are supplied and: an outcome is
|
|
22
|
+
missing, the values don't sum to 1.0 (tolerance 1e-6), or any value is negative.
|
|
23
|
+
|
|
24
|
+
## Attributes
|
|
25
|
+
|
|
26
|
+
| Reader | Type | Description |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| `outcomes` | `Array` | the sorted outcome set |
|
|
29
|
+
| `probabilities` | `Hash` | `{outcome => Float}` |
|
|
30
|
+
|
|
31
|
+
## Instance methods
|
|
32
|
+
|
|
33
|
+
### `#probability(outcome)` → `Float`
|
|
34
|
+
|
|
35
|
+
Probability of one outcome; `0.0` for an unknown one.
|
|
36
|
+
|
|
37
|
+
**Alias:** `#[]`
|
|
38
|
+
|
|
39
|
+
```ruby
|
|
40
|
+
prior.probability(0) # => 0.2
|
|
41
|
+
prior[0] # => 0.2
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### `#update_from_observations(observations, smoothing: 1.0)` → `Prior`
|
|
45
|
+
|
|
46
|
+
New prior from observed frequencies, with Laplace smoothing:
|
|
47
|
+
|
|
48
|
+
$$P(\text{outcome}) = \frac{\text{count} + \alpha}{\text{total} + \alpha n}$$
|
|
49
|
+
|
|
50
|
+
| Parameter | Type | Description |
|
|
51
|
+
|---|---|---|
|
|
52
|
+
| `observations` | `Hash` | `{outcome => Integer count}` |
|
|
53
|
+
| `smoothing:` | `Float` | α; default `1.0`. Higher pulls toward uniform |
|
|
54
|
+
|
|
55
|
+
```ruby
|
|
56
|
+
prior.update_from_observations({ -1 => 5, 0 => 20, 1 => 12 })
|
|
57
|
+
prior.update_from_observations(counts, smoothing: 5.0)
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Smoothing guarantees no outcome gets probability zero — see
|
|
61
|
+
[Cromwell's rule](../guide/prior.md#laplace-smoothing).
|
|
62
|
+
|
|
63
|
+
### `#combine(other_prior, weight: 0.5)` → `Prior`
|
|
64
|
+
|
|
65
|
+
Weighted average. `weight` applies to the receiver, `1 - weight` to the argument.
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
data_prior.combine(elicited_prior, weight: 0.7)
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
**Raises `ArgumentError`** if the two priors span different outcomes.
|
|
72
|
+
|
|
73
|
+
### `#entropy` → `Float`
|
|
74
|
+
|
|
75
|
+
Shannon entropy in bits. `0.0` when certain; `log₂(n)` when uniform.
|
|
76
|
+
|
|
77
|
+
### `#to_s` → `String`
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
Prior(-2: 0.100, -1: 0.150, 0: 0.500, 1: 0.150, 2: 0.100)
|
|
81
|
+
```
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# SQA::BI::TimeSeriesPredictor
|
|
2
|
+
|
|
3
|
+
The facade over [`Prior`](prior.md), [`Likelihood`](likelihood.md) and
|
|
4
|
+
[`Posterior`](posterior.md).
|
|
5
|
+
|
|
6
|
+
## Constructor
|
|
7
|
+
|
|
8
|
+
### `.new(outcomes:, bandwidth: 1.0, prior_probabilities: nil, update_prior: true)`
|
|
9
|
+
|
|
10
|
+
| Parameter | Type | Default | Description |
|
|
11
|
+
|---|---|---|---|
|
|
12
|
+
| `outcomes:` | `Array` | required | discrete outcome set; sorted |
|
|
13
|
+
| `bandwidth:` | `Float` | `1.0` | KDE kernel width |
|
|
14
|
+
| `prior_probabilities:` | `Hash, nil` | `nil` | explicit prior; `nil` is uniform |
|
|
15
|
+
| `update_prior:` | `Boolean` | `true` | relearn the prior from frequencies while training |
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
SQA::BI::TimeSeriesPredictor.new(
|
|
19
|
+
outcomes: [-2, -1, 0, 1, 2],
|
|
20
|
+
bandwidth: 0.5,
|
|
21
|
+
prior_probabilities: elicited.probabilities,
|
|
22
|
+
update_prior: false
|
|
23
|
+
)
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`SQA::BI.predictor(...)` is a shorthand for the same thing.
|
|
27
|
+
|
|
28
|
+
## Attributes
|
|
29
|
+
|
|
30
|
+
| Reader | Type | Description |
|
|
31
|
+
|---|---|---|
|
|
32
|
+
| `outcomes` | `Array` | the sorted outcome set |
|
|
33
|
+
| `prior` | `Prior` | current prior |
|
|
34
|
+
| `likelihood` | `Likelihood` | the KDE estimator |
|
|
35
|
+
| `feature_dimension` | `Integer, nil` | fixed by the first `train`; `nil` before |
|
|
36
|
+
| `bandwidth` | `Float` | current kernel width |
|
|
37
|
+
|
|
38
|
+
## Training
|
|
39
|
+
|
|
40
|
+
### `#train(features, outcome:)` → `self`
|
|
41
|
+
|
|
42
|
+
```ruby
|
|
43
|
+
predictor.train([1.0, 2.0, 3.0], outcome: 1)
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
**Raises `ArgumentError`** when features are not an array of numbers, the
|
|
47
|
+
dimension differs from the established one, or the outcome is not in the set.
|
|
48
|
+
|
|
49
|
+
With `update_prior: true`, the prior is relearned once
|
|
50
|
+
`likelihood.size >= outcomes.size` — the guard stops the first few observations
|
|
51
|
+
stamping a lopsided prior onto the model.
|
|
52
|
+
|
|
53
|
+
### `#train_batch(observations)` → `self`
|
|
54
|
+
|
|
55
|
+
```ruby
|
|
56
|
+
predictor.train_batch([{ features: [...], outcome: 1 }, ...])
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Prediction
|
|
60
|
+
|
|
61
|
+
### `#predict(features)` → `Posterior`
|
|
62
|
+
|
|
63
|
+
The full distribution. Prefer this.
|
|
64
|
+
|
|
65
|
+
### `#predict_outcome(features)` → outcome
|
|
66
|
+
|
|
67
|
+
`predict(features).max_outcome` — discards the uncertainty.
|
|
68
|
+
|
|
69
|
+
### `#predict_proba(features)` → `Hash`
|
|
70
|
+
|
|
71
|
+
`predict(features).to_h`.
|
|
72
|
+
|
|
73
|
+
## State
|
|
74
|
+
|
|
75
|
+
| Method | Returns | Description |
|
|
76
|
+
|---|---|---|
|
|
77
|
+
| `#trained?` | `Boolean` | any observations yet? |
|
|
78
|
+
| `#training_size` | `Integer` | observation count |
|
|
79
|
+
| `#training_distribution` | `Hash` | `{outcome => count}` — check this for class imbalance |
|
|
80
|
+
| `#summary` | `String` | outcomes, dimension, sample count, bandwidth, prior, distribution |
|
|
81
|
+
|
|
82
|
+
## Mutation
|
|
83
|
+
|
|
84
|
+
### `#reset!(keep_prior: false)` → `self`
|
|
85
|
+
|
|
86
|
+
Clears observations and the feature dimension. Also resets the prior to uniform
|
|
87
|
+
unless `keep_prior: true`.
|
|
88
|
+
|
|
89
|
+
### `#bandwidth=(new_bandwidth)` → `self`
|
|
90
|
+
|
|
91
|
+
Rebuilds the KDE from the existing observations — nothing is lost, which makes a
|
|
92
|
+
[bandwidth sweep](../guide/tuning.md#sweeping-it) cheap.
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
predictor.bandwidth = 0.5
|
|
96
|
+
```
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
/* Custom styles for SQA::BI documentation */
|
|
2
|
+
|
|
3
|
+
/* Diagrams are authored for a dark theme on a transparent background.
|
|
4
|
+
In light mode they need a dark plate behind them to stay legible. */
|
|
5
|
+
[data-md-color-scheme="default"] img[src*="/diagrams/"] {
|
|
6
|
+
background-color: #282a36;
|
|
7
|
+
border-radius: 8px;
|
|
8
|
+
padding: 0.75rem;
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
img[src*="/diagrams/"] {
|
|
12
|
+
width: 100%;
|
|
13
|
+
height: auto;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
/* Outcome scale used throughout the guide: negative red, zero amber,
|
|
17
|
+
positive green — matching the terminal output of the demos. */
|
|
18
|
+
.outcome-down { color: #ff5555; font-weight: 600; }
|
|
19
|
+
.outcome-flat { color: #ffb86c; font-weight: 600; }
|
|
20
|
+
.outcome-up { color: #50fa7b; font-weight: 600; }
|
|
21
|
+
|
|
22
|
+
/* Keep wide probability tables from wrapping mid-number. */
|
|
23
|
+
table td code {
|
|
24
|
+
white-space: nowrap;
|
|
25
|
+
}
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 520" font-family="Helvetica, Arial, sans-serif">
|
|
2
|
+
<!-- Dark theme, transparent background -->
|
|
3
|
+
<defs>
|
|
4
|
+
<marker id="archArrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
|
5
|
+
<path d="M 0 0 L 10 5 L 0 10 z" fill="#8be9fd"/>
|
|
6
|
+
</marker>
|
|
7
|
+
<marker id="archArrowAmber" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
|
8
|
+
<path d="M 0 0 L 10 5 L 0 10 z" fill="#ffb86c"/>
|
|
9
|
+
</marker>
|
|
10
|
+
</defs>
|
|
11
|
+
|
|
12
|
+
<text x="480" y="32" text-anchor="middle" fill="#f8f8f2" font-size="22" font-weight="bold">SQA::BI architecture</text>
|
|
13
|
+
<text x="480" y="54" text-anchor="middle" fill="#9aa0b0" font-size="13">a pure-Ruby core, with an optional LLM wing that only ever supplies numbers</text>
|
|
14
|
+
|
|
15
|
+
<!-- Core math region -->
|
|
16
|
+
<rect x="30" y="82" width="520" height="300" rx="14" fill="none" stroke="#6272a4" stroke-width="1.5" stroke-dasharray="6 4"/>
|
|
17
|
+
<text x="48" y="104" fill="#6272a4" font-size="12" font-weight="bold">CORE — no dependencies, always available</text>
|
|
18
|
+
|
|
19
|
+
<!-- TimeSeriesPredictor -->
|
|
20
|
+
<rect x="60" y="120" width="200" height="78" rx="10" fill="none" stroke="#8be9fd" stroke-width="2.5"/>
|
|
21
|
+
<text x="160" y="148" text-anchor="middle" fill="#8be9fd" font-size="14" font-weight="bold">TimeSeriesPredictor</text>
|
|
22
|
+
<text x="160" y="169" text-anchor="middle" fill="#d6d9e0" font-size="11">train / predict facade</text>
|
|
23
|
+
<text x="160" y="186" text-anchor="middle" fill="#9aa0b0" font-size="11">owns outcomes + bandwidth</text>
|
|
24
|
+
|
|
25
|
+
<!-- Prior -->
|
|
26
|
+
<rect x="60" y="240" width="145" height="76" rx="10" fill="none" stroke="#bd93f9" stroke-width="2"/>
|
|
27
|
+
<text x="132" y="266" text-anchor="middle" fill="#bd93f9" font-size="13" font-weight="bold">Prior</text>
|
|
28
|
+
<text x="132" y="286" text-anchor="middle" fill="#d6d9e0" font-size="11">P(outcome)</text>
|
|
29
|
+
<text x="132" y="303" text-anchor="middle" fill="#9aa0b0" font-size="10">Laplace smoothing</text>
|
|
30
|
+
|
|
31
|
+
<!-- Likelihood -->
|
|
32
|
+
<rect x="225" y="240" width="145" height="76" rx="10" fill="none" stroke="#ffb86c" stroke-width="2"/>
|
|
33
|
+
<text x="297" y="266" text-anchor="middle" fill="#ffb86c" font-size="13" font-weight="bold">Likelihood</text>
|
|
34
|
+
<text x="297" y="286" text-anchor="middle" fill="#d6d9e0" font-size="11">P(features | outcome)</text>
|
|
35
|
+
<text x="297" y="303" text-anchor="middle" fill="#9aa0b0" font-size="10">Gaussian KDE</text>
|
|
36
|
+
|
|
37
|
+
<!-- Posterior -->
|
|
38
|
+
<rect x="390" y="240" width="145" height="76" rx="10" fill="none" stroke="#50fa7b" stroke-width="2"/>
|
|
39
|
+
<text x="462" y="266" text-anchor="middle" fill="#50fa7b" font-size="13" font-weight="bold">Posterior</text>
|
|
40
|
+
<text x="462" y="286" text-anchor="middle" fill="#d6d9e0" font-size="11">P(outcome | features)</text>
|
|
41
|
+
<text x="462" y="303" text-anchor="middle" fill="#9aa0b0" font-size="10">entropy · KL · sampling</text>
|
|
42
|
+
|
|
43
|
+
<!-- Edges from predictor -->
|
|
44
|
+
<path d="M 130 198 L 130 240" fill="none" stroke="#8be9fd" stroke-width="1.6" marker-end="url(#archArrow)"/>
|
|
45
|
+
<path d="M 190 198 L 280 240" fill="none" stroke="#8be9fd" stroke-width="1.6" marker-end="url(#archArrow)"/>
|
|
46
|
+
<path d="M 240 198 L 440 240" fill="none" stroke="#8be9fd" stroke-width="1.6" marker-end="url(#archArrow)"/>
|
|
47
|
+
|
|
48
|
+
<text x="480" y="352" text-anchor="middle" fill="#9aa0b0" font-size="11" font-style="italic">Bayes' theorem is computed here, in Ruby, exactly — never delegated</text>
|
|
49
|
+
|
|
50
|
+
<!-- LLM wing -->
|
|
51
|
+
<rect x="580" y="82" width="350" height="300" rx="14" fill="none" stroke="#6272a4" stroke-width="1.5" stroke-dasharray="6 4"/>
|
|
52
|
+
<text x="598" y="104" fill="#6272a4" font-size="12" font-weight="bold">OPTIONAL — ruby_llm required lazily</text>
|
|
53
|
+
|
|
54
|
+
<rect x="610" y="120" width="290" height="66" rx="10" fill="none" stroke="#ff79c6" stroke-width="2"/>
|
|
55
|
+
<text x="755" y="145" text-anchor="middle" fill="#ff79c6" font-size="13" font-weight="bold">LlmPriorElicitor</text>
|
|
56
|
+
<text x="755" y="166" text-anchor="middle" fill="#9aa0b0" font-size="11">context in prose → relative weights → Prior</text>
|
|
57
|
+
|
|
58
|
+
<rect x="610" y="202" width="290" height="66" rx="10" fill="none" stroke="#ff79c6" stroke-width="2"/>
|
|
59
|
+
<text x="755" y="227" text-anchor="middle" fill="#ff79c6" font-size="13" font-weight="bold">LlmLikelihoodEstimator</text>
|
|
60
|
+
<text x="755" y="248" text-anchor="middle" fill="#9aa0b0" font-size="11">text evidence → P(e | H) per hypothesis</text>
|
|
61
|
+
|
|
62
|
+
<rect x="610" y="284" width="290" height="76" rx="10" fill="none" stroke="#f1fa8c" stroke-width="2"/>
|
|
63
|
+
<text x="755" y="309" text-anchor="middle" fill="#f1fa8c" font-size="13" font-weight="bold">LlmSupport</text>
|
|
64
|
+
<text x="755" y="329" text-anchor="middle" fill="#d6d9e0" font-size="11">provider resolution · JSON extraction</text>
|
|
65
|
+
<text x="755" y="346" text-anchor="middle" fill="#d6d9e0" font-size="11">re-keying · normalizing · clamping</text>
|
|
66
|
+
|
|
67
|
+
<!-- LLM wing feeds the core -->
|
|
68
|
+
<path d="M 610 153 L 205 258" fill="none" stroke="#ffb86c" stroke-width="1.6" stroke-dasharray="5 4" marker-end="url(#archArrowAmber)"/>
|
|
69
|
+
<path d="M 610 235 L 370 268" fill="none" stroke="#ffb86c" stroke-width="1.6" stroke-dasharray="5 4" marker-end="url(#archArrowAmber)"/>
|
|
70
|
+
|
|
71
|
+
<!-- Footer contract -->
|
|
72
|
+
<rect x="140" y="412" width="680" height="72" rx="12" fill="none" stroke="#50fa7b" stroke-width="2"/>
|
|
73
|
+
<text x="480" y="440" text-anchor="middle" fill="#50fa7b" font-size="15" font-weight="bold">The LLM judges. Ruby computes.</text>
|
|
74
|
+
<text x="480" y="463" text-anchor="middle" fill="#9aa0b0" font-size="11">the LLM answers one isolated question at a time and never accumulates, normalizes or updates a belief</text>
|
|
75
|
+
</svg>
|