sqa-bi 0.1.0

data/docs/llm/providers.md

@@ -0,0 +1,184 @@
+# Providers and Local Models
+
+`SQA::BI::LlmSupport` decides where a chat request goes. The policy is
+**local first** — and the part that costs people time is that the fallback to
+cloud is silent.
+
+## Resolution order
+
+![Provider resolution — local first](../assets/diagrams/provider-resolution.svg)
+
+1. `SQA_BI_LLM_PROVIDER` if set — `lms`, `apfel`, or `cloud`
+2. **LM Studio** at `http://localhost:1234/v1` via `ruby_llm-providers-lms`
+3. **Apfel** (Apple Foundation Models) at `http://127.0.0.1:11434/v1` via
+   `ruby_llm-providers-apfel`
+4. **Cloud** through the plain `ruby_llm` registry
+
+Steps 2 and 3 are real HTTP probes against `/v1/models` with a one-second
+timeout. Any failure falls through.
+
+## Environment variables
+
+| Variable | Effect |
+|---|---|
+| `SQA_BI_LLM_PROVIDER` | force `lms`, `apfel`, or `cloud` |
+| `SQA_BI_LLM_MODEL` | force a model id |
+| `LMS_API_BASE` | override the LM Studio base URL |
+| `APFEL_API_BASE` | override the Apfel base URL |
+
+The unprefixed `BI_LLM_PROVIDER` / `BI_LLM_MODEL` names this library used before
+it moved into the SQA workspace are still honored as a fallback, via
+`LlmSupport.env_value`. The namespaced names win when both are set.
+
+## Setting up LM Studio
+
+Installing the provider gem is not enough. Two further things must be true, and
+neither happens just because the desktop app is open.
+
+### 1. The server must be running
+
+```bash
+lms status
+# Server:  OFF
+#
+# (i) To start the server, run the following command:
+#     lms server start
+
+lms server start
+# Success! Server is now running on port 1234
+```
+
+### 2. A chat model must be loaded
+
+```bash
+lms ps
+# No models are currently loaded.
+
+lms ls          # what you have downloaded
+lms load qwen/qwen3.8-27b
+# Model loaded successfully in 29.67s. (16.52 GiB)
+```
+
+!!! danger "Embedding models don't count"
+    `choose_local_model` rejects any id matching `/embed|ocr/i`. A server
+    offering only `text-embedding-nomic-embed-text-v1.5` resolves to **no chat
+    model at all** — `local_chat` then raises:
+
+    ```
+    SQA::BI::Error: No chat model available from lms server at http://localhost:1234/v1
+    ```
+
+### Model preference
+
+```ruby
+def choose_local_model(ids)
+  chat_ids = ids.reject { |id| id.match?(/embed|ocr/i) }
+  chat_ids.find { |id| id.match?(/qwen/i) } ||
+    chat_ids.find { |id| id.match?(/gpt-oss/i) } ||
+    chat_ids.first
+end
+```
+
+Qwen first — it honors JSON and structured prompts most reliably in LM Studio,
+which matters when every response must parse as a JSON object. Then gpt-oss,
+then whatever is left.
+
+## Making resolution visible
+
+This is the important part.
+
+```ruby
+SQA::BI::LlmSupport.current_resolution
+# => "lms — qwen/qwen3.8-27b at http://localhost:1234/v1"
+# => "cloud — claude-haiku-4-5"
+# => "lms — (no chat model available) at http://localhost:1234/v1"
+```
+
+Examples 04 and 05 print this before their first call:
+
+```
+LLM: lms — qwen/qwen3.8-27b at http://localhost:1234/v1
+```
+
+!!! warning "Why this exists"
+    With the server off, resolution drops to cloud and hits
+    `CLOUD_DEFAULT_MODEL` (`claude-haiku-4-5`). If that key is missing or stale,
+    the error you get is an Anthropic `401 — API key is invalid`, with a stack
+    trace through `RubyLLM::Protocols::Anthropic` and **no mention of LM Studio
+    anywhere**. Nothing in that error suggests the actual cause is a local
+    server that never started.
+
+    The silence is deliberate for a library — it should not print warnings — but
+    it means any application using local-first resolution should surface the
+    resolved provider itself.
+
+### Probing directly
+
+```ruby
+S = SQA::BI::LlmSupport
+
+S.resolve_provider                        # => :lms
+S.server_alive?(S.lms_api_base)           # => true
+S.list_local_models(S.lms_api_base)       # => ["qwen/qwen3.8-27b", ...]
+S.choose_local_model(S.list_local_models(S.lms_api_base))
+                                          # => "qwen/qwen3.8-27b"
+S.local_api_base(:cloud)                  # => nil
+```
+
+## Cloud providers
+
+`configure_ruby_llm` wires up whichever keys are present:
+
+```ruby
+config.anthropic_api_key = ENV['ANTHROPIC_API_KEY'] if ENV['ANTHROPIC_API_KEY']
+config.openai_api_key    = ENV['OPENAI_API_KEY']    if ENV['OPENAI_API_KEY']
+config.gemini_api_key    = ENV['GEMINI_API_KEY']    if ENV['GEMINI_API_KEY']
+```
+
+It runs once and is safe to call repeatedly. `CLOUD_DEFAULT_MODEL` is
+`claude-haiku-4-5`; override per-run:
+
+```bash
+SQA_BI_LLM_PROVIDER=cloud SQA_BI_LLM_MODEL=gemini-2.5-flash ruby examples/04_llm_elicited_prior.rb
+```
+
+!!! note "Free tiers will stop you mid-run"
+    Example 05 makes one call per piece of evidence. On the Gemini free tier
+    (20 requests/model) a few runs is enough to exhaust the quota and get
+    `RubyLLM::RateLimitError` partway through. Another argument for local.
+
+## Optional dependencies
+
+```ruby
+def require_local_providers
+  %w[ruby_llm/providers/lms ruby_llm/providers/apfel].each do |path|
+    require path
+  rescue LoadError
+    nil
+  end
+end
+```
+
+A missing provider gem disables only that provider. It never breaks the cloud
+path, and it certainly never breaks the KDE path — which needs no LLM at all.
+
+Similarly, `ruby_llm` itself is required lazily inside `build_chat`. If it
+cannot be loaded, `require_ruby_llm` raises a `SQA::BI::Error` explaining both
+likely causes — the gem is genuinely absent, or the process is running outside
+the bundle and RubyGems activated a conflicting dependency:
+
+```
+Could not load ruby_llm (Gem::ConflictError: Unable to activate ruby_llm-2.0.0,
+because json-3.0.2 conflicts with json (< 3)).
+
+ruby_llm is an optional dependency of sqa-bi — only the LLM-backed
+prior and likelihood paths need it; the KDE path does not.
+
+If it is installed, this usually means the process is running
+outside the bundle. Run it with `bundle exec`, or let direnv set
+BUNDLE_GEMFILE (`asgard dev` / `asgard prod` at the workspace root).
+```
+
+## API
+
+See [`SQA::BI::LlmSupport`](../api/llm-support.md) for the full method list.

data/docs/requirements.txt

@@ -0,0 +1,8 @@
+# Python dependencies for building this documentation site.
+#
+#   pip install -r docs/requirements.txt
+#   mkdocs serve
+#
+# MkDocs is Python-based and managed separately from the Ruby bundle.
+mkdocs>=1.6
+mkdocs-material>=9.5

data/lib/sqa/bi/likelihood.rb

@@ -0,0 +1,167 @@
+# frozen_string_literal: true
+
+module SQA
+  module BI
+    # Computes likelihood P(data | outcome) for Bayesian inference
+    #
+    # The Likelihood class estimates how probable our observed data is
+    # given each possible outcome. For time series data with features (x, y, z),
+    # it uses kernel density estimation with historical observations.
+    class Likelihood
+      attr_reader :observations, :bandwidth
+
+      # Initialize likelihood estimator
+      #
+      # @param observations [Array<Hash>] Historical observations
+      #   Each observation: {features: [x, y, z], outcome: result}
+      # @param bandwidth [Float] Kernel bandwidth for density estimation
+      #   Smaller = more sensitive to local patterns, Larger = smoother estimates
+      #
+      # @example
+      #   observations = [
+      #     {features: [1.0, 2.0, 3.0], outcome: 1},
+      #     {features: [1.1, 2.1, 2.9], outcome: 1},
+      #     {features: [-1.0, -2.0, 0.5], outcome: -2}
+      #   ]
+      #   likelihood = Likelihood.new(observations, bandwidth: 1.0)
+      def initialize(observations = [], bandwidth: 1.0)
+        @observations = observations
+        @bandwidth = bandwidth
+        @outcome_groups = group_by_outcome
+      end
+
+      # Add new observation to the likelihood model
+      #
+      # @param features [Array<Numeric>] Feature vector (e.g., [x, y, z])
+      # @param outcome [Numeric] Observed outcome
+      # @return [void]
+      def add_observation(features, outcome)
+        @observations << { features: features, outcome: outcome }
+        @outcome_groups = group_by_outcome
+      end
+
+      # Compute likelihood P(features | outcome) for each outcome
+      #
+      # Uses Gaussian kernel density estimation:
+      # For each outcome, compute average similarity to historical observations
+      # with that outcome using Gaussian kernel.
+      #
+      # @param features [Array<Numeric>] Query feature vector
+      # @param outcomes [Array] All possible outcomes
+      # @return [Hash] {outcome => likelihood_value}
+      #
+      # @example
+      #   likelihoods = likelihood.compute([1.0, 2.0, 3.0], [-2, -1, 0, 1, 2])
+      #   # => {-2 => 0.05, -1 => 0.1, 0 => 0.15, 1 => 0.6, 2 => 0.1}
+      def compute(features, outcomes)
+        if @observations.empty?
+          # No data yet, return uniform likelihoods
+          uniform = 1.0 / outcomes.size
+          return outcomes.to_h { |outcome| [outcome, uniform] }
+        end
+
+        likelihoods = outcomes.to_h do |outcome|
+          [outcome, estimate_density(features, outcome)]
+        end
+
+        # Normalize likelihoods to sum to 1
+        normalize_hash(likelihoods)
+      end
+
+      # Estimate kernel density for given features and outcome
+      #
+      # @param features [Array<Numeric>] Query features
+      # @param outcome [Numeric] Target outcome
+      # @return [Float] Density estimate
+      def estimate_density(features, outcome)
+        outcome_obs = @outcome_groups[outcome] || []
+
+        if outcome_obs.empty?
+          # No observations for this outcome, return small epsilon
+          return 1e-10
+        end
+
+        # Gaussian kernel density estimation
+        # K(u) = (1 / sqrt(2π)) * exp(-u² / 2)
+        # KDE(x) = (1 / n*h) * Σ K((x - xᵢ) / h)
+
+        total_density = outcome_obs.reduce(0.0) do |sum, obs|
+          distance = euclidean_distance(features, obs[:features])
+          kernel_value = gaussian_kernel(distance / @bandwidth)
+          sum + kernel_value
+        end
+
+        # Average density
+        density = total_density / outcome_obs.size
+
+        # Ensure minimum density to avoid zero probabilities
+        [density, 1e-10].max
+      end
+
+      # Get number of observations
+      #
+      # @return [Integer]
+      def size
+        @observations.size
+      end
+
+      # Check if likelihood model is empty
+      #
+      # @return [Boolean]
+      def empty?
+        @observations.empty?
+      end
+
+      # Get count of observations for each outcome
+      #
+      # @return [Hash] {outcome => count}
+      def outcome_counts
+        @outcome_groups.transform_values(&:size)
+      end
+
+      private
+
+      # Group observations by outcome
+      #
+      # @return [Hash] {outcome => [observations]}
+      def group_by_outcome
+        @observations.group_by { |obs| obs[:outcome] }
+      end
+
+      # Compute Euclidean distance between two feature vectors
+      #
+      # @param query [Array<Numeric>] First vector
+      # @param observed [Array<Numeric>] Second vector
+      # @return [Float] Euclidean distance
+      def euclidean_distance(query, observed)
+        raise ArgumentError, 'Feature vectors must have same dimension' unless query.size == observed.size
+
+        Math.sqrt(query.zip(observed).reduce(0.0) { |sum, (a, b)| sum + ((a - b)**2) })
+      end
+
+      # Gaussian kernel function
+      #
+      # @param normalized_distance [Float] distance divided by the bandwidth
+      # @return [Float] Kernel value
+      def gaussian_kernel(normalized_distance)
+        (1.0 / Math.sqrt(2 * Math::PI)) * Math.exp(-0.5 * (normalized_distance**2))
+      end
+
+      # Normalize hash values to sum to 1
+      #
+      # @param hash [Hash] Hash with numeric values
+      # @return [Hash] Normalized hash
+      def normalize_hash(hash)
+        total = hash.values.sum
+
+        if total.positive?
+          hash.transform_values { |v| v / total }
+        else
+          # All zeros, return uniform
+          uniform = 1.0 / hash.size
+          hash.transform_values { uniform }
+        end
+      end
+    end
+  end
+end

data/lib/sqa/bi/llm_likelihood_estimator.rb

@@ -0,0 +1,110 @@
+# frozen_string_literal: true
+
+require_relative 'llm_support'
+
+module SQA
+  module BI
+    # Uses an LLM as a likelihood function over soft, semantic evidence.
+    #
+    # Kernel density estimation (the Likelihood class) needs numeric
+    # feature vectors and historical data. But much real-world evidence is
+    # text: a log line, a witness statement, a news headline. An LLM can
+    # judge P(evidence | hypothesis) for that kind of evidence — and then
+    # this library does what LLMs are demonstrably bad at: combining those
+    # judgments coherently via Bayes' theorem, one update per piece of
+    # evidence, with no double-counting and no anchoring drift.
+    #
+    # @example Root-cause diagnosis from log evidence
+    #   estimator = LlmLikelihoodEstimator.new(
+    #     hypotheses: {
+    #       bad_deploy: 'The 14:02 deploy introduced a bug',
+    #       database:   'The primary database is degraded',
+    #       network:    'There is a network partition between AZs'
+    #     }
+    #   )
+    #   lh = estimator.likelihoods('Error rate spiked 2 minutes after deploy')
+    #   # => { bad_deploy: 0.9, database: 0.2, network: 0.15 }
+    class LlmLikelihoodEstimator
+      include LlmSupport
+
+      attr_reader :hypotheses, :model, :provider
+
+      # @param hypotheses [Hash] {id => description}; ids become the
+      #   outcome keys used with Prior/Posterior
+      # @param chat [#ask, nil] injectable chat object (tests pass a fake)
+      # @param model [String, nil] model id; nil resolves per provider
+      # @param provider [Symbol, nil] :lms, :apfel, :cloud; nil auto-detects
+      def initialize(hypotheses:, chat: nil, model: nil, provider: nil)
+        @hypotheses = hypotheses
+        @model = model
+        @provider = provider
+        @chat = chat
+      end
+
+      # Estimate P(evidence | hypothesis) for every hypothesis.
+      #
+      # Likelihoods are NOT probabilities of the hypotheses and need not
+      # sum to 1 — each answers "if this hypothesis were true, how
+      # surprising would this evidence be?" independently.
+      #
+      # @param evidence [String] one piece of evidence in natural language
+      # @return [Hash] {hypothesis_id => Float in (0, 1)}
+      def likelihoods(evidence)
+        response = chat.ask(build_prompt(evidence))
+        likelihoods_from_response(response.content)
+      end
+
+      # Convert raw LLM response content into a clamped likelihood hash.
+      # Public and pure so it can be tested without any network call.
+      #
+      # @param content [Hash, String] LLM response content
+      # @return [Hash] {hypothesis_id => Float}
+      def likelihoods_from_response(content)
+        raw = extract_json(content)
+        rekey_to_outcomes(raw, @hypotheses.keys)
+          .transform_values { |v| clamp_likelihood(v) }
+      end
+
+      # Build the likelihood-estimation prompt.
+      # Public and pure so prompt wording can be tested and iterated on.
+      #
+      # @param evidence [String]
+      # @return [String]
+      def build_prompt(evidence)
+        <<~PROMPT
+          You are a careful probabilistic reasoner. For EACH hypothesis below,
+          estimate the conditional probability of observing the given evidence
+          ASSUMING THAT HYPOTHESIS IS TRUE: P(evidence | hypothesis).
+
+          These are independent judgments — they do NOT need to sum to 1.
+          Use values between 0.01 and 0.99. Avoid 0 and 1; evidence is rarely
+          impossible or certain.
+
+          Evidence:
+          #{evidence}
+
+          Hypotheses:
+          #{hypothesis_lines}
+
+          Respond with ONLY a JSON object mapping each hypothesis id to its
+          conditional probability, for example: {#{example_json_keys}}.
+          No commentary.
+        PROMPT
+      end
+
+      private
+
+      def chat
+        @chat ||= LlmSupport.build_chat(@model, provider: @provider)
+      end
+
+      def hypothesis_lines
+        @hypotheses.map { |id, desc| "- #{id}: #{desc}" }.join("\n")
+      end
+
+      def example_json_keys
+        @hypotheses.keys.map { |id| "\"#{id}\": 0.5" }.join(', ')
+      end
+    end
+  end
+end

data/lib/sqa/bi/llm_prior_elicitor.rb

@@ -0,0 +1,110 @@
+# frozen_string_literal: true
+
+require_relative 'llm_support'
+
+module SQA
+  module BI
+    # Elicits a Prior distribution from an LLM given a natural-language
+    # description of the current situation.
+    #
+    # Classical Bayesian analysis stalls on the question "where does the
+    # prior come from?" — an LLM is a compressed archive of domain
+    # knowledge that can be queried for exactly that. The LLM never does
+    # probability arithmetic; it only supplies relative weights, which are
+    # floored, normalized, and validated by the existing Prior class.
+    #
+    # @example Elicit a market-trend prior from a news summary
+    #   elicitor = LlmPriorElicitor.new(
+    #     outcomes: [-2, -1, 0, 1, 2],
+    #     outcome_descriptions: {
+    #       -2 => 'strong downtrend', -1 => 'mild downtrend', 0 => 'sideways',
+    #        1 => 'mild uptrend',      2 => 'strong uptrend'
+    #     }
+    #   )
+    #   prior = elicitor.elicit('The Fed unexpectedly cut rates by 50bp...')
+    #   predictor = TimeSeriesPredictor.new(
+    #     outcomes: [-2, -1, 0, 1, 2],
+    #     prior_probabilities: prior.probabilities,
+    #     update_prior: false
+    #   )
+    class LlmPriorElicitor
+      include LlmSupport
+
+      attr_reader :outcomes, :outcome_descriptions, :model, :provider
+
+      # @param outcomes [Array] discrete outcomes the prior ranges over
+      # @param outcome_descriptions [Hash] optional {outcome => human label}
+      #   used in the prompt so the LLM knows what each outcome means
+      # @param chat [#ask, nil] injectable chat object (tests pass a fake);
+      #   defaults to a RubyLLM chat built on first use
+      # @param model [String, nil] model id; nil resolves per provider
+      # @param provider [Symbol, nil] :lms, :apfel, :cloud; nil auto-detects
+      def initialize(outcomes:, outcome_descriptions: {}, chat: nil, model: nil, provider: nil)
+        @outcomes = outcomes.sort
+        @outcome_descriptions = outcome_descriptions
+        @model = model
+        @provider = provider
+        @chat = chat
+      end
+
+      # Ask the LLM for a prior over the outcomes given the context.
+      #
+      # @param context [String] natural-language description of the situation
+      # @return [Prior] validated prior distribution
+      def elicit(context)
+        response = chat.ask(build_prompt(context))
+        prior_from_response(response.content)
+      end
+
+      # Convert raw LLM response content into a Prior.
+      # Public and pure so it can be tested without any network call.
+      #
+      # @param content [Hash, String] LLM response content
+      # @return [Prior]
+      def prior_from_response(content)
+        raw = extract_json(content)
+        weights = rekey_to_outcomes(raw, @outcomes)
+        Prior.new(@outcomes, normalize_distribution(weights))
+      end
+
+      # Build the elicitation prompt.
+      # Public and pure so prompt wording can be tested and iterated on.
+      #
+      # @param context [String]
+      # @return [String]
+      def build_prompt(context)
+        <<~PROMPT
+          You are a careful probabilistic forecaster. Based on the situation
+          described below, assign a relative weight between 0 and 100 to each
+          possible outcome, reflecting how probable each outcome is a priori.
+
+          Situation:
+          #{context}
+
+          Possible outcomes:
+          #{outcome_lines}
+
+          Respond with ONLY a JSON object mapping each outcome to its weight,
+          for example: {#{example_json_keys}}. No commentary.
+        PROMPT
+      end
+
+      private
+
+      def chat
+        @chat ||= LlmSupport.build_chat(@model, provider: @provider)
+      end
+
+      def outcome_lines
+        @outcomes.map do |outcome|
+          label = @outcome_descriptions[outcome]
+          label ? "- #{outcome}: #{label}" : "- #{outcome}"
+        end.join("\n")
+      end
+
+      def example_json_keys
+        @outcomes.map { |outcome| "\"#{outcome}\": 20" }.join(', ')
+      end
+    end
+  end
+end