sqa-bi 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. checksums.yaml +7 -0
  2. data/.github/workflows/docs.yml +55 -0
  3. data/.quality/reek_baseline.txt +5 -0
  4. data/.rubocop.yml +224 -0
  5. data/CHANGELOG.md +33 -0
  6. data/CLAUDE.md +128 -0
  7. data/COMMITS.md +196 -0
  8. data/LICENSE.txt +21 -0
  9. data/README.md +229 -0
  10. data/Rakefile +170 -0
  11. data/decision_support_techniques.md +391 -0
  12. data/docs/EXPLORATION.md +128 -0
  13. data/docs/api/index.md +66 -0
  14. data/docs/api/likelihood.md +78 -0
  15. data/docs/api/llm-elicitors.md +119 -0
  16. data/docs/api/llm-support.md +136 -0
  17. data/docs/api/posterior.md +106 -0
  18. data/docs/api/prior.md +81 -0
  19. data/docs/api/time-series-predictor.md +96 -0
  20. data/docs/assets/css/custom.css +25 -0
  21. data/docs/assets/diagrams/architecture.svg +75 -0
  22. data/docs/assets/diagrams/bayes-pipeline.svg +48 -0
  23. data/docs/assets/diagrams/kde.svg +52 -0
  24. data/docs/assets/diagrams/llm-bayes-loop.svg +58 -0
  25. data/docs/assets/diagrams/provider-resolution.svg +77 -0
  26. data/docs/assets/js/mathjax.js +18 -0
  27. data/docs/development.md +164 -0
  28. data/docs/examples/index.md +198 -0
  29. data/docs/getting-started/core-concepts.md +119 -0
  30. data/docs/getting-started/installation.md +112 -0
  31. data/docs/getting-started/quick-start.md +143 -0
  32. data/docs/guide/likelihood.md +132 -0
  33. data/docs/guide/posterior.md +135 -0
  34. data/docs/guide/predictor.md +191 -0
  35. data/docs/guide/prior.md +141 -0
  36. data/docs/guide/tuning.md +156 -0
  37. data/docs/guide/uncertainty.md +148 -0
  38. data/docs/index.md +110 -0
  39. data/docs/llm/index.md +124 -0
  40. data/docs/llm/likelihood-estimation.md +161 -0
  41. data/docs/llm/prior-elicitation.md +172 -0
  42. data/docs/llm/providers.md +184 -0
  43. data/docs/requirements.txt +8 -0
  44. data/lib/sqa/bi/likelihood.rb +167 -0
  45. data/lib/sqa/bi/llm_likelihood_estimator.rb +110 -0
  46. data/lib/sqa/bi/llm_prior_elicitor.rb +110 -0
  47. data/lib/sqa/bi/llm_support.rb +299 -0
  48. data/lib/sqa/bi/posterior.rb +189 -0
  49. data/lib/sqa/bi/prior.rb +135 -0
  50. data/lib/sqa/bi/time_series_predictor.rb +219 -0
  51. data/lib/sqa/bi/version.rb +7 -0
  52. data/lib/sqa/bi.rb +57 -0
  53. data/mkdocs.yml +174 -0
  54. metadata +272 -0
@@ -0,0 +1,48 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 940 420" font-family="Helvetica, Arial, sans-serif">
2
+ <!-- Dark theme, transparent background -->
3
+ <defs>
4
+ <marker id="arrowCyan" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
5
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#8be9fd"/>
6
+ </marker>
7
+ </defs>
8
+
9
+ <text x="470" y="34" text-anchor="middle" fill="#f8f8f2" font-size="22" font-weight="bold">Prior × Likelihood → Posterior</text>
10
+ <text x="470" y="56" text-anchor="middle" fill="#9aa0b0" font-size="13">one prediction, from belief and evidence to a distribution over outcomes</text>
11
+
12
+ <!-- Prior: purple = belief before data -->
13
+ <rect x="40" y="110" width="230" height="150" rx="12" fill="none" stroke="#bd93f9" stroke-width="2.5"/>
14
+ <text x="155" y="140" text-anchor="middle" fill="#bd93f9" font-size="16" font-weight="bold">Prior</text>
15
+ <text x="155" y="162" text-anchor="middle" fill="#9aa0b0" font-size="12" font-style="italic">P(outcome)</text>
16
+ <text x="155" y="190" text-anchor="middle" fill="#d6d9e0" font-size="12">belief before seeing</text>
17
+ <text x="155" y="208" text-anchor="middle" fill="#d6d9e0" font-size="12">this feature vector</text>
18
+ <text x="155" y="234" text-anchor="middle" fill="#9aa0b0" font-size="11">uniform, custom, learned from</text>
19
+ <text x="155" y="250" text-anchor="middle" fill="#9aa0b0" font-size="11">frequencies, or LLM-elicited</text>
20
+
21
+ <!-- Likelihood: amber = evidence -->
22
+ <rect x="355" y="110" width="230" height="150" rx="12" fill="none" stroke="#ffb86c" stroke-width="2.5"/>
23
+ <text x="470" y="140" text-anchor="middle" fill="#ffb86c" font-size="16" font-weight="bold">Likelihood</text>
24
+ <text x="470" y="162" text-anchor="middle" fill="#9aa0b0" font-size="12" font-style="italic">P(features | outcome)</text>
25
+ <text x="470" y="190" text-anchor="middle" fill="#d6d9e0" font-size="12">how typical these features</text>
26
+ <text x="470" y="208" text-anchor="middle" fill="#d6d9e0" font-size="12">are of each outcome</text>
27
+ <text x="470" y="234" text-anchor="middle" fill="#9aa0b0" font-size="11">Gaussian KDE over history,</text>
28
+ <text x="470" y="250" text-anchor="middle" fill="#9aa0b0" font-size="11">or an LLM over text evidence</text>
29
+
30
+ <!-- Posterior: green = result -->
31
+ <rect x="670" y="110" width="230" height="150" rx="12" fill="none" stroke="#50fa7b" stroke-width="2.5"/>
32
+ <text x="785" y="140" text-anchor="middle" fill="#50fa7b" font-size="16" font-weight="bold">Posterior</text>
33
+ <text x="785" y="162" text-anchor="middle" fill="#9aa0b0" font-size="12" font-style="italic">P(outcome | features)</text>
34
+ <text x="785" y="190" text-anchor="middle" fill="#d6d9e0" font-size="12">updated belief, as a full</text>
35
+ <text x="785" y="208" text-anchor="middle" fill="#d6d9e0" font-size="12">distribution not a point</text>
36
+ <text x="785" y="234" text-anchor="middle" fill="#9aa0b0" font-size="11">carries entropy, confidence,</text>
37
+ <text x="785" y="250" text-anchor="middle" fill="#9aa0b0" font-size="11">KL divergence, sampling</text>
38
+
39
+ <!-- Operators between boxes -->
40
+ <text x="312" y="195" text-anchor="middle" fill="#8be9fd" font-size="30" font-weight="bold">×</text>
41
+ <text x="627" y="195" text-anchor="middle" fill="#8be9fd" font-size="26" font-weight="bold">→</text>
42
+
43
+ <!-- Normalization note -->
44
+ <line x1="470" y1="285" x2="470" y2="315" stroke="#8be9fd" stroke-width="1.5" marker-end="url(#arrowCyan)" stroke-dasharray="4 3"/>
45
+ <rect x="245" y="318" width="450" height="60" rx="10" fill="none" stroke="#8be9fd" stroke-width="1.5"/>
46
+ <text x="470" y="342" text-anchor="middle" fill="#8be9fd" font-size="13" font-weight="bold">normalize by the evidence P(features)</text>
47
+ <text x="470" y="363" text-anchor="middle" fill="#9aa0b0" font-size="11">divide by the total so the posterior sums to 1 — Ruby does this, exactly, every time</text>
48
+ </svg>
@@ -0,0 +1,52 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 940 460" font-family="Helvetica, Arial, sans-serif">
2
+ <!-- Dark theme, transparent background -->
3
+ <defs>
4
+ <marker id="kdeArrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
5
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#8be9fd"/>
6
+ </marker>
7
+ </defs>
8
+
9
+ <text x="470" y="32" text-anchor="middle" fill="#f8f8f2" font-size="22" font-weight="bold">Gaussian kernel density estimation</text>
10
+ <text x="470" y="54" text-anchor="middle" fill="#9aa0b0" font-size="13">each stored observation of an outcome contributes a bump; the query point reads off their average</text>
11
+
12
+ <!-- Axis -->
13
+ <line x1="70" y1="330" x2="870" y2="330" stroke="#6272a4" stroke-width="1.5"/>
14
+ <text x="870" y="356" text-anchor="end" fill="#9aa0b0" font-size="12">distance from the query point, in feature space</text>
15
+
16
+ <!-- Individual kernels for outcome +1 (green) -->
17
+ <path d="M 150 330 Q 210 200 270 330" fill="none" stroke="#50fa7b" stroke-width="1.4" opacity="0.55"/>
18
+ <path d="M 200 330 Q 260 180 320 330" fill="none" stroke="#50fa7b" stroke-width="1.4" opacity="0.55"/>
19
+ <path d="M 250 330 Q 310 210 370 330" fill="none" stroke="#50fa7b" stroke-width="1.4" opacity="0.55"/>
20
+
21
+ <!-- Observation ticks for +1 -->
22
+ <circle cx="210" cy="330" r="4" fill="#50fa7b"/>
23
+ <circle cx="260" cy="330" r="4" fill="#50fa7b"/>
24
+ <circle cx="310" cy="330" r="4" fill="#50fa7b"/>
25
+ <text x="260" y="378" text-anchor="middle" fill="#50fa7b" font-size="12" font-weight="bold">observations with outcome +1</text>
26
+
27
+ <!-- Individual kernels for outcome -1 (red) -->
28
+ <path d="M 560 330 Q 620 235 680 330" fill="none" stroke="#ff5555" stroke-width="1.4" opacity="0.55"/>
29
+ <path d="M 620 330 Q 680 215 740 330" fill="none" stroke="#ff5555" stroke-width="1.4" opacity="0.55"/>
30
+
31
+ <circle cx="620" cy="330" r="4" fill="#ff5555"/>
32
+ <circle cx="680" cy="330" r="4" fill="#ff5555"/>
33
+ <text x="650" y="378" text-anchor="middle" fill="#ff5555" font-size="12" font-weight="bold">observations with outcome −1</text>
34
+
35
+ <!-- Query point -->
36
+ <line x1="280" y1="120" x2="280" y2="330" stroke="#8be9fd" stroke-width="2" stroke-dasharray="5 4"/>
37
+ <circle cx="280" cy="330" r="6" fill="#8be9fd"/>
38
+ <text x="280" y="110" text-anchor="middle" fill="#8be9fd" font-size="13" font-weight="bold">query features</text>
39
+
40
+ <!-- Density readings -->
41
+ <circle cx="280" cy="196" r="5" fill="none" stroke="#50fa7b" stroke-width="2"/>
42
+ <text x="300" y="192" fill="#50fa7b" font-size="12">high density → P(features | +1) is large</text>
43
+
44
+ <circle cx="280" cy="322" r="5" fill="none" stroke="#ff5555" stroke-width="2"/>
45
+ <line x1="286" y1="320" x2="430" y2="286" stroke="#ff5555" stroke-width="1" opacity="0.7"/>
46
+ <text x="436" y="282" fill="#ff5555" font-size="12">near zero → P(features | −1) is small</text>
47
+
48
+ <!-- Formula panel -->
49
+ <rect x="70" y="396" width="800" height="52" rx="10" fill="none" stroke="#bd93f9" stroke-width="1.5"/>
50
+ <text x="470" y="420" text-anchor="middle" fill="#bd93f9" font-size="14" font-weight="bold">P(x | outcome) = (1/n) · Σ K( ‖x − xᵢ‖ / h ) with K Gaussian</text>
51
+ <text x="470" y="439" text-anchor="middle" fill="#9aa0b0" font-size="11">bandwidth h sets the width of every bump: small h tracks local structure, large h smooths it away</text>
52
+ </svg>
@@ -0,0 +1,58 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 920 420" font-family="Helvetica, Arial, sans-serif">
2
+ <!-- Dark theme, transparent background -->
3
+ <defs>
4
+ <marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
5
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#8be9fd"/>
6
+ </marker>
7
+ <marker id="arrowAmber" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
8
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#ffb86c"/>
9
+ </marker>
10
+ <marker id="arrowGreen" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
11
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#50fa7b"/>
12
+ </marker>
13
+ </defs>
14
+
15
+ <text x="460" y="34" text-anchor="middle" fill="#f8f8f2" font-size="22" font-weight="bold">The LLM judges — Ruby computes</text>
16
+ <text x="460" y="56" text-anchor="middle" fill="#9aa0b0" font-size="13">division of labor between semantic judgment and probability arithmetic</text>
17
+
18
+ <!-- Evidence / context input -->
19
+ <rect x="30" y="120" width="200" height="90" rx="12" fill="none" stroke="#bd93f9" stroke-width="2"/>
20
+ <text x="130" y="150" text-anchor="middle" fill="#bd93f9" font-size="15" font-weight="bold">Natural language</text>
21
+ <text x="130" y="172" text-anchor="middle" fill="#d6d9e0" font-size="12">news, logs, reports,</text>
22
+ <text x="130" y="190" text-anchor="middle" fill="#d6d9e0" font-size="12">domain descriptions</text>
23
+
24
+ <!-- LLM box -->
25
+ <rect x="330" y="100" width="240" height="130" rx="12" fill="none" stroke="#ffb86c" stroke-width="2.5"/>
26
+ <text x="450" y="130" text-anchor="middle" fill="#ffb86c" font-size="16" font-weight="bold">LLM — the judge</text>
27
+ <text x="450" y="155" text-anchor="middle" fill="#d6d9e0" font-size="12">LlmPriorElicitor → P(H)</text>
28
+ <text x="450" y="175" text-anchor="middle" fill="#d6d9e0" font-size="12">LlmLikelihoodEstimator → P(e | H)</text>
29
+ <text x="450" y="203" text-anchor="middle" fill="#9aa0b0" font-size="11" font-style="italic">one independent judgment per call;</text>
30
+ <text x="450" y="219" text-anchor="middle" fill="#9aa0b0" font-size="11" font-style="italic">never asked to accumulate or normalize</text>
31
+
32
+ <!-- Bayes engine box -->
33
+ <rect x="660" y="100" width="230" height="130" rx="12" fill="none" stroke="#50fa7b" stroke-width="2.5"/>
34
+ <text x="775" y="130" text-anchor="middle" fill="#50fa7b" font-size="16" font-weight="bold">Ruby — the engine</text>
35
+ <text x="775" y="155" text-anchor="middle" fill="#d6d9e0" font-size="12">Prior × Likelihood → Posterior</text>
36
+ <text x="775" y="175" text-anchor="middle" fill="#d6d9e0" font-size="12">normalize · clamp · entropy · KL</text>
37
+ <text x="775" y="203" text-anchor="middle" fill="#9aa0b0" font-size="11" font-style="italic">exact arithmetic, no anchoring,</text>
38
+ <text x="775" y="219" text-anchor="middle" fill="#9aa0b0" font-size="11" font-style="italic">no double-counting</text>
39
+
40
+ <!-- Posterior output -->
41
+ <rect x="360" y="300" width="330" height="80" rx="12" fill="none" stroke="#8be9fd" stroke-width="2"/>
42
+ <text x="525" y="330" text-anchor="middle" fill="#8be9fd" font-size="15" font-weight="bold">Posterior belief</text>
43
+ <text x="525" y="352" text-anchor="middle" fill="#d6d9e0" font-size="12">MAP · confidence · information gain · samples</text>
44
+
45
+ <!-- Arrows -->
46
+ <line x1="230" y1="165" x2="322" y2="165" stroke="#ffb86c" stroke-width="2" marker-end="url(#arrowAmber)"/>
47
+ <text x="276" y="152" text-anchor="middle" fill="#9aa0b0" font-size="11">prompt</text>
48
+
49
+ <line x1="570" y1="165" x2="652" y2="165" stroke="#50fa7b" stroke-width="2" marker-end="url(#arrowGreen)"/>
50
+ <text x="611" y="152" text-anchor="middle" fill="#9aa0b0" font-size="11">numbers</text>
51
+
52
+ <line x1="775" y1="230" x2="700" y2="298" stroke="#8be9fd" stroke-width="2" marker-end="url(#arrow)"/>
53
+
54
+ <!-- Feedback loop: posterior becomes next prior -->
55
+ <path d="M 360 345 C 150 345, 130 300, 130 218" fill="none" stroke="#8be9fd" stroke-width="2" stroke-dasharray="6 5" marker-end="url(#arrow)"/>
56
+ <text x="180" y="330" text-anchor="middle" fill="#8be9fd" font-size="11" font-style="italic">posterior becomes the next prior</text>
57
+ <text x="180" y="346" text-anchor="middle" fill="#8be9fd" font-size="11" font-style="italic">as new evidence arrives</text>
58
+ </svg>
@@ -0,0 +1,77 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 520" font-family="Helvetica, Arial, sans-serif">
2
+ <!-- Dark theme, transparent background -->
3
+ <defs>
4
+ <marker id="prArrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
5
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#8be9fd"/>
6
+ </marker>
7
+ <marker id="prArrowGreen" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
8
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#50fa7b"/>
9
+ </marker>
10
+ <marker id="prArrowRed" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
11
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#ff5555"/>
12
+ </marker>
13
+ </defs>
14
+
15
+ <text x="480" y="32" text-anchor="middle" fill="#f8f8f2" font-size="22" font-weight="bold">Provider resolution — local first</text>
16
+ <text x="480" y="54" text-anchor="middle" fill="#9aa0b0" font-size="13">LlmSupport.resolve_provider, and the two ways a local run quietly becomes a cloud run</text>
17
+
18
+ <!-- Step 1: env override -->
19
+ <rect x="330" y="80" width="300" height="60" rx="10" fill="none" stroke="#f1fa8c" stroke-width="2"/>
20
+ <text x="480" y="105" text-anchor="middle" fill="#f1fa8c" font-size="13" font-weight="bold">SQA_BI_LLM_PROVIDER set?</text>
21
+ <text x="480" y="126" text-anchor="middle" fill="#9aa0b0" font-size="11">explicit wins — lms | apfel | cloud</text>
22
+
23
+ <path d="M 480 140 L 480 176" fill="none" stroke="#8be9fd" stroke-width="1.6" marker-end="url(#prArrow)"/>
24
+ <text x="500" y="163" fill="#9aa0b0" font-size="11">no</text>
25
+
26
+ <!-- Step 2: LM Studio probe -->
27
+ <rect x="330" y="178" width="300" height="72" rx="10" fill="none" stroke="#8be9fd" stroke-width="2"/>
28
+ <text x="480" y="203" text-anchor="middle" fill="#8be9fd" font-size="13" font-weight="bold">GET localhost:1234/v1/models</text>
29
+ <text x="480" y="223" text-anchor="middle" fill="#d6d9e0" font-size="11">LM Studio — ruby_llm-providers-lms</text>
30
+ <text x="480" y="240" text-anchor="middle" fill="#9aa0b0" font-size="10">1 second timeout</text>
31
+
32
+ <path d="M 480 250 L 480 286" fill="none" stroke="#8be9fd" stroke-width="1.6" marker-end="url(#prArrow)"/>
33
+ <text x="500" y="273" fill="#9aa0b0" font-size="11">no answer</text>
34
+
35
+ <!-- Step 3: apfel probe -->
36
+ <rect x="330" y="288" width="300" height="72" rx="10" fill="none" stroke="#8be9fd" stroke-width="2"/>
37
+ <text x="480" y="313" text-anchor="middle" fill="#8be9fd" font-size="13" font-weight="bold">GET 127.0.0.1:11434/v1/models</text>
38
+ <text x="480" y="333" text-anchor="middle" fill="#d6d9e0" font-size="11">Apfel — Apple Foundation Models</text>
39
+ <text x="480" y="350" text-anchor="middle" fill="#9aa0b0" font-size="10">1 second timeout</text>
40
+
41
+ <path d="M 480 360 L 480 396" fill="none" stroke="#ff5555" stroke-width="1.6" marker-end="url(#prArrowRed)"/>
42
+ <text x="500" y="383" fill="#9aa0b0" font-size="11">no answer</text>
43
+
44
+ <!-- Cloud fallback -->
45
+ <rect x="330" y="398" width="300" height="72" rx="10" fill="none" stroke="#ff5555" stroke-width="2.5"/>
46
+ <text x="480" y="423" text-anchor="middle" fill="#ff5555" font-size="13" font-weight="bold">cloud — CLOUD_DEFAULT_MODEL</text>
47
+ <text x="480" y="443" text-anchor="middle" fill="#d6d9e0" font-size="11">claude-haiku-4-5, needs an API key</text>
48
+ <text x="480" y="460" text-anchor="middle" fill="#9aa0b0" font-size="10">this fallback is silent — call current_resolution to see it</text>
49
+
50
+ <!-- Success branches to the right -->
51
+ <path d="M 630 214 L 740 214" fill="none" stroke="#50fa7b" stroke-width="1.6" marker-end="url(#prArrowGreen)"/>
52
+ <path d="M 630 324 L 740 324" fill="none" stroke="#50fa7b" stroke-width="1.6" marker-end="url(#prArrowGreen)"/>
53
+
54
+ <rect x="742" y="186" width="200" height="56" rx="10" fill="none" stroke="#50fa7b" stroke-width="2"/>
55
+ <text x="842" y="210" text-anchor="middle" fill="#50fa7b" font-size="13" font-weight="bold">:lms</text>
56
+ <text x="842" y="230" text-anchor="middle" fill="#9aa0b0" font-size="10">choose_local_model picks qwen</text>
57
+
58
+ <rect x="742" y="296" width="200" height="56" rx="10" fill="none" stroke="#50fa7b" stroke-width="2"/>
59
+ <text x="842" y="320" text-anchor="middle" fill="#50fa7b" font-size="13" font-weight="bold">:apfel</text>
60
+ <text x="842" y="340" text-anchor="middle" fill="#9aa0b0" font-size="10">on-device Apple model</text>
61
+
62
+ <!-- The two gotchas -->
63
+ <rect x="20" y="150" width="290" height="230" rx="12" fill="none" stroke="#ffb86c" stroke-width="2"/>
64
+ <text x="165" y="176" text-anchor="middle" fill="#ffb86c" font-size="13" font-weight="bold">Why the probe fails</text>
65
+
66
+ <text x="40" y="204" fill="#ffb86c" font-size="12" font-weight="bold">1 — the server is off</text>
67
+ <text x="40" y="224" fill="#d6d9e0" font-size="11">The LM Studio desktop app being</text>
68
+ <text x="40" y="240" fill="#d6d9e0" font-size="11">open is not the same as its HTTP</text>
69
+ <text x="40" y="256" fill="#d6d9e0" font-size="11">server running.</text>
70
+ <text x="40" y="276" fill="#8be9fd" font-size="11" font-family="monospace">lms status</text>
71
+ <text x="40" y="293" fill="#8be9fd" font-size="11" font-family="monospace">lms server start</text>
72
+
73
+ <text x="40" y="322" fill="#ffb86c" font-size="12" font-weight="bold">2 — no chat model loaded</text>
74
+ <text x="40" y="341" fill="#d6d9e0" font-size="11">Embedding and OCR models are</text>
75
+ <text x="40" y="357" fill="#d6d9e0" font-size="11">rejected, so a server offering only</text>
76
+ <text x="40" y="373" fill="#d6d9e0" font-size="11">those resolves to no model at all.</text>
77
+ </svg>
@@ -0,0 +1,18 @@
1
+ /* MathJax configuration for SQA::BI documentation */
2
+
3
+ window.MathJax = {
4
+ tex: {
5
+ inlineMath: [["\\(", "\\)"]],
6
+ displayMath: [["\\[", "\\]"]],
7
+ processEscapes: true,
8
+ processEnvironments: true
9
+ },
10
+ options: {
11
+ ignoreHtmlClass: ".*|",
12
+ processHtmlClass: "arithmatex"
13
+ }
14
+ };
15
+
16
+ document$.subscribe(() => {
17
+ MathJax.typesetPromise()
18
+ })
@@ -0,0 +1,164 @@
1
+ # Development
2
+
3
+ ## Setup
4
+
5
+ ```bash
6
+ cd ~/sandbox/git_repos/madbomber/projects/sqa_project/sqa-bi
7
+ bundle install
8
+ bundle exec rake test
9
+ ```
10
+
11
+ The workspace uses [asgard](https://github.com/MadBomber/asgard) `.loki` task
12
+ files. Run `asgard help` from any repo for the available commands rather than
13
+ memorizing raw `rake`/`bundle` invocations.
14
+
15
+ ## Testing
16
+
17
+ ```bash
18
+ asgard test # or: bundle exec rake test
19
+ ruby -Ilib:test test/llm_support_test.rb # a single file
20
+ ```
21
+
22
+ Minitest, with SimpleCov. Current state: **77 tests, 148 assertions, 0
23
+ failures, ~86% line coverage.**
24
+
25
+ ### Rules the suite enforces
26
+
27
+ **No test touches a real provider.** Both LLM classes accept an injectable
28
+ `chat:`, and `test/test_helper.rb` supplies a `FakeChat` that returns canned
29
+ content and records the prompt it was given. Any test that would make a network
30
+ call is a bug.
31
+
32
+ **Environment-dependent tests must be hermetic.** `LlmSupport.env_model` reads
33
+ `SQA_BI_LLM_MODEL` and `BI_LLM_MODEL`, so tests that exercise it use the
34
+ `without_env` helper to clear and restore them. A developer with either variable
35
+ exported must still get a green suite.
36
+
37
+ **Every method is testable in isolation.** `LlmSupport` is entirely
38
+ `module_function`. `build_prompt`, `prior_from_response` and
39
+ `likelihoods_from_response` are public and pure so prompt wording and parsing
40
+ can be tested directly. Even the examples follow this — `UI` methods return
41
+ Strings and print nothing; `ExampleBundle.gemfile_path` is pure and unit tested.
42
+
43
+ ## Quality gates
44
+
45
+ ```bash
46
+ asgard quality # tests + coverage, Flog, Flay, Reek
47
+ asgard rubocop
48
+ asgard rubocop_fix
49
+ ```
50
+
51
+ | Gate | Threshold |
52
+ |---|---|
53
+ | **Tests** | all pass, with SimpleCov coverage |
54
+ | **Flog** | warn ≥ 20, fail ≥ 50 (per method) |
55
+ | **Flay** | fail on structural duplication with mass ≥ 50 |
56
+ | **Reek** | fail only on files that are **new or worsened** vs the baseline |
57
+ | **RuboCop** | zero offenses |
58
+
59
+ ### RuboCop is shared, don't edit it here
60
+
61
+ `.rubocop.yml` is generated from the workspace's `.rubocop.yml.common`. Edit
62
+ that file and run `asgard sync_rubocop`; a local edit is overwritten at the next
63
+ sync.
64
+
65
+ ### Reek runs against a baseline
66
+
67
+ `.quality/reek_baseline.txt` grandfathers **11** smells across 5 files. All are
68
+ deliberate:
69
+
70
+ - **FeatureEnvy** in the `reduce`-based `Prior#entropy`, `Posterior#entropy`,
71
+ `#kl_divergence_from_prior`, and `Likelihood#euclidean_distance` — these are
72
+ numeric folds over a local, not misplaced methods.
73
+ - **TooManyStatements** in `LlmSupport.configure_ruby_llm` and
74
+ `Posterior#compute_posterior`.
75
+ - **ControlParameter** in `LlmSupport.local_api_base` and `.resolution_label` —
76
+ small lookup/format functions where branching on the argument *is* the
77
+ function.
78
+ - **BooleanParameter / ControlParameter** on `TimeSeriesPredictor#initialize`
79
+ (`update_prior:`) and `#reset!` (`keep_prior:`) — API shape, not accidents.
80
+
81
+ Regenerate with `asgard reek_baseline` after a genuine improvement. The gate is
82
+ baseline-aware by design: the point is to stop *regressions*, not to reach zero.
83
+
84
+ !!! warning "If you hand-edit the baseline"
85
+ Reek prints `1 warning:` — **singular** — for single-smell files. Grepping
86
+ for `warnings:` silently drops them and produces a baseline that fails the
87
+ gate. Derive counts from `Reek::Examiner`, the way `rake reek_check` does.
88
+
89
+ `Posterior#summary` sits at Flog 21.2 — above the warn threshold, below the fail
90
+ threshold. It is a formatting heredoc.
91
+
92
+ ## Architecture rules
93
+
94
+ **`sqa-bi` is a leaf.** No runtime dependencies, and deliberately no dependency
95
+ on `sqa`. It sits beside `sqa-tai` so that `sqa` can depend on *it*. Adding a
96
+ `sqa` dependency to the gemspec inverts the graph and creates a cycle the moment
97
+ `sqa` picks this gem up.
98
+
99
+ **The core math stays domain-agnostic.** It knows nothing about markets. Keep it
100
+ that way; market-facing code belongs in `sqa` or in the examples.
101
+
102
+ **`ruby_llm` stays lazily required** inside `LlmSupport.build_chat`, and stays a
103
+ development dependency. The library must load, and KDE must work, without it.
104
+
105
+ **The LLM judges, Ruby computes.** Any change to `LlmPriorElicitor` or
106
+ `LlmLikelihoodEstimator` must preserve the split: the model supplies isolated
107
+ judgments, Ruby does every piece of arithmetic.
108
+
109
+ ## Documentation
110
+
111
+ ```bash
112
+ mkdocs serve # live reload at http://127.0.0.1:8000
113
+ mkdocs build # static site into site/ (gitignored)
114
+ ```
115
+
116
+ MkDocs with the Material theme, dark by default.
117
+
118
+ Diagrams are hand-authored SVG in `docs/assets/diagrams/`, following the
119
+ project convention: **dark theme, transparent background, colour used to
120
+ distinguish function** — purple for prior/belief, amber for likelihood/evidence,
121
+ green for posterior/result, cyan for flow and structure, red for failure paths.
122
+ `docs/assets/css/custom.css` puts a dark plate behind them in light mode.
123
+
124
+ ## Bundle mode
125
+
126
+ The workspace root `.envrc` reads `.bundle_mode` and exports `BUNDLE_GEMFILE`:
127
+
128
+ | Mode | Gemfile | Resolves |
129
+ |---|---|---|
130
+ | `dev` (default) | `Gemfile.local` | sibling checkouts via `path:` |
131
+ | `prod` | `Gemfile` | released gems only |
132
+
133
+ Switch with `asgard dev` / `asgard prod` at the workspace root. A component's
134
+ `.envrc` must **not** re-export `BUNDLE_GEMFILE` — that would silently override
135
+ the workspace-wide mode.
136
+
137
+ !!! note "It is exported as a relative name"
138
+ Which means `bundle exec` only resolves correctly from a repo root. Running
139
+ from a subdirectory — `examples/`, say — fails to find the Gemfile and
140
+ silently falls back to the global gemset. `examples/example_bundle.rb`
141
+ works around it for the demos; be aware of it elsewhere.
142
+
143
+ ## Commit conventions
144
+
145
+ [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) — see
146
+ `COMMITS.md` in the repo.
147
+
148
+ ## Open threads
149
+
150
+ Tracked in `decision_support_techniques.md` and
151
+ [Exploration Notes](EXPLORATION.md):
152
+
153
+ - **Multi-horizon prediction** (k = 1…10) sharing strength across horizons
154
+ rather than fitting each independently. `TimeSeriesPredictor` is
155
+ single-horizon today, and adjacent horizons are strongly correlated.
156
+ - **A Laya sidecar** as an alternative likelihood backend behind `LlmSupport`'s
157
+ provider dispatch, benchmarked on latency and calibration (ECE).
158
+ - **`eps` / LightGBM as a benchmark** to score the posterior against — not as a
159
+ replacement engine.
160
+ - **Research JEV properly** before it influences any design decision.
161
+
162
+ Read the log before proposing a different inference engine; the tree-ensemble
163
+ and typed-decision routes were each evaluated there and set aside for recorded
164
+ reasons.
@@ -0,0 +1,198 @@
1
+ # Examples
2
+
3
+ Five runnable demos in [`examples/`](https://github.com/MadBomber/sqa-bi/tree/main/examples).
4
+ Each prints a narrated walkthrough rather than bare numbers.
5
+
6
+ ```bash
7
+ ruby examples/01_coin_flip.rb
8
+ ruby examples/02_time_series_prediction.rb
9
+ ruby examples/03_stock_market_prediction_v2.rb
10
+ ruby examples/04_llm_elicited_prior.rb
11
+ ruby examples/05_llm_likelihood_diagnosis.rb
12
+ ```
13
+
14
+ They also run directly by shebang (`./01_coin_flip.rb`) — `examples/common.rb`
15
+ activates the bundle first, which matters more than it sounds. See
16
+ [Running them](#running-them) below.
17
+
18
+ ---
19
+
20
+ ## 01 — Coin flip
21
+
22
+ **The mechanics, stripped to the minimum.** A one-dimensional feature (observed
23
+ heads-rate) mapped onto three outcomes: fair, leans heads, heavily heads.
24
+
25
+ Establishes the core loop — train, predict, read the distribution — with nothing
26
+ domain-specific in the way. Ends by sampling from the posterior to show what
27
+ residual uncertainty looks like as a tally:
28
+
29
+ ```
30
+ 0 38 38.0% ███████████████░░░░░░░░░ Fair coin
31
+ 1 60 60.0% ████████████████████████ Leans heads
32
+ 2 2 2.0% █░░░░░░░░░░░░░░░░░░░░░░░ Heavily heads
33
+ ```
34
+
35
+ **Read it for:** the shape of the API, and why confidence rises away from
36
+ category boundaries.
37
+
38
+ ---
39
+
40
+ ## 02 — Time series prediction
41
+
42
+ **Synthetic trends with known ground truth.** Generates trending, sideways and
43
+ reversing series, extracts features, trains, and predicts on held-out data.
44
+
45
+ Because the true outcome is known by construction, this is the one place you can
46
+ see the predictor being *right* and *wrong* without arguing about the data.
47
+
48
+ **Read it for:** feature extraction from a sequence, and how entropy behaves as
49
+ a series becomes more or less ambiguous.
50
+
51
+ ---
52
+
53
+ ## 03 — Stock market prediction
54
+
55
+ **Real OHLCV data through the `sqa` gem.** Predicts the next day's close band
56
+ from five delta features — close change, volume change, and changes in SMA(20),
57
+ RSI(14) and EMA(12).
58
+
59
+ ```
60
+ Loading T historical data...
61
+ Loaded 6761 days of T data
62
+ Date range: 1999-11-01 to 2026-09-18
63
+
64
+ Generated 6739 feature vectors
65
+ Training set: 5391 samples
66
+ Test set: 1348 samples
67
+ ```
68
+
69
+ It prints a confusion matrix, and the matrix is the point:
70
+
71
+ ```
72
+ Actual \ Predicted: 0 1 2 3 4
73
+ ----------------------------------------
74
+ 0 (big_down ): 0 2 5 3 0
75
+ 1 (small_down ): 0 27 343 69 0
76
+ 2 (sideways ): 0 28 353 62 0
77
+ 3 (small_up ): 0 51 327 70 0
78
+ 4 (big_up ): 0 0 7 1 0
79
+ ```
80
+
81
+ !!! warning "This demo is honest about failing"
82
+ Columns 0 and 4 are empty — the extreme bands hold under 1% of training
83
+ samples each, so the KDE has too few neighbours to ever favor them. The
84
+ model collapses onto the middle three bands and mostly predicts "sideways".
85
+
86
+ That is exactly what [Tuning](../guide/tuning.md) warns about, shown rather
87
+ than described. Next-day equity direction from five TA deltas is close to
88
+ unpredictable, and a method that reports its uncertainty honestly will tell
89
+ you so instead of manufacturing a signal.
90
+
91
+ **Requires the `sqa` gem**, which `sqa-bi` does not depend on. Run it in dev
92
+ bundle mode (`asgard dev` at the workspace root).
93
+
94
+ **Read it for:** delta-based feature engineering, chronological train/test
95
+ splitting, and what class imbalance looks like from the inside.
96
+
97
+ ---
98
+
99
+ ## 04 — LLM-elicited prior
100
+
101
+ **Domain knowledge as a prior.** Five training observations — far too few for
102
+ KDE — plus a paragraph of market context handed to a language model.
103
+
104
+ ```
105
+ LLM: lms — qwen/qwen3.8-27b at http://localhost:1234/v1
106
+
107
+ LLM-elicited prior: Prior(-2: 0.000, -1: 0.020, 0: 0.080, 1: 0.350, 2: 0.550)
108
+ Prior entropy: 1.409 bits (uniform would be 2.322)
109
+
110
+ With UNIFORM prior (data only)
111
+ Posterior(-2: 0.000, -1: 0.139, 0: 0.358, 1: 0.358, 2: 0.144)
112
+ Confidence: 19.9%
113
+
114
+ With LLM-ELICITED prior (knowledge + data)
115
+ Posterior(-2: 0.000, -1: 0.040, 0: 0.256, 1: 0.576, 2: 0.129)
116
+ Confidence: 34.2%
117
+ ```
118
+
119
+ The data-only posterior is a dead tie between "sideways" and "mild uptrend".
120
+ The elicited prior breaks it.
121
+
122
+ **Read it for:** [prior elicitation](../llm/prior-elicitation.md) in practice,
123
+ and the side-by-side that shows what the prior actually bought.
124
+
125
+ ---
126
+
127
+ ## 05 — LLM likelihood diagnosis
128
+
129
+ **Sequential Bayesian updating over textual evidence.** A production outage,
130
+ three competing hypotheses, five pieces of evidence arriving in order.
131
+
132
+ ```
133
+ LLM: lms — qwen/qwen3.8-27b at http://localhost:1234/v1
134
+
135
+ Initial belief: bad_deploy 33.3% database 33.3% network 33.3%
136
+
137
+ Evidence 1: HTTP 500 error rate jumped from 0.1% to 8% at 14:04, two
138
+ minutes after the deploy finished.
139
+ P(e|H): bad_deploy=0.95 database=0.60 network=0.10
140
+ gain: 0.351 bits bad_deploy 57.6% database 36.4% network 6.1%
141
+
142
+ ...
143
+
144
+ Final diagnosis: network at 99.7% confidence
145
+ ```
146
+
147
+ Evidence 1 looks damning for the deploy. Because likelihoods are clamped away
148
+ from 0 and 1, Bayes never fully commits — and evidence 3–5 cleanly reverse the
149
+ belief.
150
+
151
+ **Read it for:** [likelihood estimation](../llm/likelihood-estimation.md),
152
+ [chained updates](../guide/posterior.md#chaining-updates-across-several-pieces-of-evidence),
153
+ and per-update information gain identifying which facts mattered.
154
+
155
+ ---
156
+
157
+ ## Supporting files
158
+
159
+ **`common.rb`** — activates the bundle, loads the library, includes the
160
+ `SQA::BI` namespace, and provides the `UI` module of terminal helpers (banners,
161
+ bars, distribution charts). Every `UI` method returns a String and prints
162
+ nothing, so each is testable in isolation.
163
+
164
+ **`example_bundle.rb`** — resolves which Gemfile to activate. Pure, and unit
165
+ tested.
166
+
167
+ **`pure_ruby_indicators.rb`** — dependency-free SMA / EMA / RSI so the demos
168
+ need no `sqa-tai` build. Production code should use `sqa-tai`, which wraps the
169
+ real TA-Lib and provides 136+ indicators.
170
+
171
+ ## Running them
172
+
173
+ Prefer `bundle exec`, or run them directly — both work:
174
+
175
+ ```bash
176
+ bundle exec ruby examples/04_llm_elicited_prior.rb
177
+ cd examples && ./04_llm_elicited_prior.rb
178
+ ```
179
+
180
+ ??? info "Why the shebang path needs `example_bundle.rb`"
181
+ The workspace direnv exports `BUNDLE_GEMFILE` as a **bare relative name**
182
+ (`Gemfile` or `Gemfile.local`, per `asgard dev` / `asgard prod`) so each
183
+ repo picks up its own. Bundler resolves that against the current directory
184
+ — which, for a demo launched by its shebang, is `examples/`, not the repo
185
+ root. The lookup fails, the bundle is never activated, and RubyGems
186
+ resolves against the entire rbenv gemset instead.
187
+
188
+ That matters because `json 3.x` then wins, and `ruby_llm 2.0` requires
189
+ `json < 3`. The LLM demos die at `require 'ruby_llm'` with an opaque
190
+ `Gem::ConflictError` that says nothing about bundles.
191
+
192
+ `ExampleBundle.gemfile_path` anchors the relative name to the repo root
193
+ before `require 'bundler/setup'`, which fixes it.
194
+
195
+ The two LLM demos need a provider. With no local server running they fall
196
+ through to cloud silently — see
197
+ [Providers and Local Models](../llm/providers.md), and note that they print
198
+ their resolved provider on the first line for exactly this reason.