irt_ruby 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7c8164ec4d62622374ff79fa5110baf742d68544719088c9dcae2e8851838895
-  data.tar.gz: 973555d52fd8bef2cd912d2890222d3089af65e127336c4332b2ff1f5afe9638
+  metadata.gz: 520defda852bedfe0f27ea300cb471b2f9afc6b8fbf32c16cb75fc3569cca5df
+  data.tar.gz: 16c98bd94b5d88e1c898be6aab91e987d1db3e957bb1dc921e73af02f61d9377
 SHA512:
-  metadata.gz: 4d193d94c2ee5b365337e26efe6efe7dc8c48b08e1d3e1f1043f0c7d360634f2ed79ec0a5c72f094482056adc15f2ec867f5798d8fdd84a3a39db348d8e98282
-  data.tar.gz: 6e8ef3b3c8d9d8e4e4de7783f0c39bada144a72e542de180a36f9b7f5a1f6d9d7860d4d2235a03e6936ed4776816fae5ed2144e261e415b9b033cd51ca3859f8
+  metadata.gz: 2c7a636273fbccda8bce8c96acb157fbb5a3f0d209cf275e4ddff845d1b5ea139f0389c6b5c9c296dfdf98c01ab7e89ab6080e8ed21940319e05845701853ef7
+  data.tar.gz: c8b2d1447884a130e1f25ac82a2cae0ec9198171372023ea48ff8aa57168ff78724becd51e1da040e7c1e7e9f822d9775541baf2bfe77f5d0d4af51d7dc761ff

data/CHANGELOG.md

@@ -2,7 +2,32 @@
 
 All notable changes to this project are documented in this file.
 
-## [0.3.0] - 2025-01-14
+## [0.4.0] - 2026-10-11
+
+### Fixed
+- Fixed 3PL gradients for ability, difficulty, discrimination, and guessing; they now match finite differences of the log-likelihood.
+- Only the largest strongly connected part of the response graph is estimated; other respondents and items are returned as `nil` instead of drifting with `max_iter` or keeping random initial values. This covers all-correct, all-incorrect, and unobserved rows or columns as well as perfectly separated or disconnected groups.
+- 3PL keeps the limiting `log(1 - c)` contribution of excluded respondents who answered every estimated item incorrectly, and keeps guessing gradients finite when the response function underflows.
+- Log-likelihoods are computed with a numerically stable log-sigmoid instead of adding `1e-15` inside `Math.log`, which flattened the objective for badly mispredicted responses.
+- Models copy response data on construction and return fresh arrays from `fit`, so later mutation of either does not change model state.
+- Model constructors validate optimization options and raise `ArgumentError` unless `max_iter` is a positive Integer, `tolerance`, `param_tolerance`, `gradient_tolerance`, and `learning_rate` are positive finite numbers, and `decay_factor` is strictly between 0 and 1.
+
+### Changed
+- `fit` reports results on a fixed latent scale: Rasch abilities are centered at 0; 2PL and 3PL abilities are standardized to mean 0 and SD 1 with likelihood-preserving item transformations.
+- `fit` results include `converged:` and `iterations:`. `converged` is true only at a stationary point on the reported scale, controlled by the new `gradient_tolerance:` option (default `1e-3`).
+- The learning rate grows by 10% after each accepted step, so it can recover from earlier decreases.
+- Clarified the gem summary and description to describe implemented capabilities rather than production-readiness claims.
+- Added CI RuboCop and gem package build/install verification, and documented the local quality and packaging commands for contributors. RuboCop is pinned to 1.92.0 for Ruby 2.7 and newer.
+- Clarified that response data must be a `Matrix` or array of arrays containing only integer `0`, integer `1`, or `nil`; floats, strings, booleans, and other values are rejected.
+
+### Added
+- Added a `seed:` keyword to all models for reproducible initialization without resetting or consuming Ruby's global random number generator.
+- Added finite-difference gradient, identification, extreme-pattern, and parameter-recovery specs.
+- Added executable specs for README usage examples to keep the documented Rasch, 2PL, 3PL, and missing-data flows aligned with the public API.
+
+---
+
+## [0.3.0] - 2025-06-13
 
 ### Changed
 - **Code Quality**
@@ -44,7 +69,3 @@ All notable changes to this project are documented in this file.
 - This release remains **backward-compatible** with `0.1.x` in terms of existing usage; the default `:ignore` missing-data approach matches prior behavior.
 - If upgrading, simply update your gem and enjoy the new features.
 - For more details, see the updated [README](./README.md) and expanded test suites.
-
----
-
-*(If you have older versions below `0.2.0`, you can keep them documented similarly, e.g., `## [0.1.x] ...`, under this new entry.)*

data/README.md

@@ -46,7 +46,17 @@ result = model.fit
 # Output the estimated abilities and difficulties
 puts "Abilities:    #{result[:abilities]}"
 puts "Difficulties: #{result[:difficulties]}"
+puts "Converged:    #{result[:converged]} after #{result[:iterations]} iterations"
 ```
+
+The third respondent answered every item correctly, so their ability has no
+finite estimate and is returned as `nil` (see [Estimation Details](#estimation-details)).
+
+Response data passed to model constructors must be either a `Matrix` or an
+array of arrays. Each response value must be the integer `0`, the integer `1`,
+or `nil` for missing data; floats such as `0.0`/`1.0`, strings, booleans, and
+other values are rejected.
+
 ### Using 2PL and 3PL Models
 ```ruby
 two_pl_model = IrtRuby::TwoParameterModel.new(data)
@@ -63,8 +73,12 @@ puts three_pl_result[:discriminations]
 puts three_pl_result[:guessings]
 ```
 
+Three respondents are far too few for 2PL or 3PL: on this matrix `fit` returns
+`converged: false`. Discrimination and guessing estimates need hundreds of
+respondents.
+
 ## Handling Missing Data
-Real-world data often has missing responses. Each model (Rasch, 2PL, 3PL) accepts a `missing_strategy: option` to handle nil entries:
+Real-world data often has missing responses. Each model (Rasch, 2PL, 3PL) accepts a `missing_strategy:` option to handle nil entries:
 
 - `:ignore` (default): Skip `nil` responses entirely in the log-likelihood and gradient calculations.
 - `:treat_as_incorrect`: Interpret `nil` as `0`.
@@ -96,16 +110,20 @@ This flexibility helps you handle datasets where missingness might signify a ski
 ### Adaptive Learning Rate & Convergence
 By default, each model uses a gradient ascent with:
 
-- An adaptive learning rate (if log-likelihood decreases, it reverts the step and reduces the rate).
-- Multiple convergence checks (change in log-likelihood and average parameter updates).
+- An adaptive learning rate: if log-likelihood decreases, the step is reverted and the rate is multiplied by `decay_factor`; after each accepted step the rate grows by 10%.
+- A stopping rule on the change in log-likelihood and the average parameter update.
+- A stationarity check that decides whether the stopping point is reported as converged.
 
 You can customize:
 
 - `max_iter`: The maximum number of iterations.
-- `tolerance` and `param_tolerance`: Convergence thresholds for log-likelihood change and parameter updates.
+- `tolerance` and `param_tolerance`: Stopping thresholds for log-likelihood change and parameter updates.
+- `gradient_tolerance` (default `1e-3`): `fit` reports `converged: true` only if, for every parameter, the gradient averaged over the responses it depends on is below this value.
 - `learning_rate`: Initial learning rate.
 - `decay_factor`: Factor by which the learning rate is reduced on a failed step.
 
+`fit` returns `converged: false` when `max_iter` runs out or when the stopping rule fires away from a stationary point, for example because the learning rate became too small to move. Check this flag before using the estimates; a larger `learning_rate` or `max_iter` usually helps.
+
 Example:
 ```ruby
 IrtRuby::TwoParameterModel.new(
@@ -117,13 +135,36 @@ IrtRuby::TwoParameterModel.new(
   decay_factor: 0.5
 )
 ```
+
+### Reproducible Initialization
+Each model initializes parameters randomly. By default, constructors use Ruby's global random number generator, preserving the historical behavior and honoring any external `srand` calls. For reproducible model initialization without resetting or consuming global RNG state, pass `seed:`:
+
+```ruby
+model_a = IrtRuby::ThreeParameterModel.new(data, seed: 1234)
+model_b = IrtRuby::ThreeParameterModel.new(data, seed: 1234)
+
+# Same data, options, and seed produce identical fitted results.
+model_a.fit == model_b.fit #=> true
+```
+
+The `seed:` keyword is available for `RaschModel`, `TwoParameterModel`, and `ThreeParameterModel`.
+
 ### Parameter Clamping
 For 2PL and 3PL:
 
-- **Discriminations** (`a`) are clamped between `0.01` and `5.0`.
+- **Discriminations** (`a`) are clamped between `0.01` and `5.0` during optimization.
 - **Guessings** (`c`, 3PL only) are clamped to `[0.0, 0.35]`.
 
-This prevents extreme or invalid parameter estimates.
+These bounds are model restrictions: a 3PL item with a true lower asymptote above `0.35` cannot be represented. The bounds apply to the internal optimization scale; returned discriminations are multiplied by the internal ability SD (see below), so they can lie outside `[0.01, 5.0]`.
+
+## Estimation Details
+
+All models estimate person and item parameters jointly by maximizing the likelihood (joint maximum likelihood, JMLE).
+
+- **Which parameters are estimated.** Think of the responses as a directed graph: a correct answer links the item to the respondent, an incorrect answer links the respondent to the item. Finite estimates on a common scale exist only within a strongly connected part of this graph, so IrtRuby estimates the largest such part and returns `nil` for every other respondent and item. This excludes respondents who answered every observed item correctly or every item incorrectly, items that everyone answered correctly or incorrectly, rows and columns with no observed responses, groups that are perfectly separated from the rest (for example, a group of respondents who answered every item of another group correctly), and groups that share no responses with the rest. Missing-data strategies are applied first, so `:treat_as_correct` and `:treat_as_incorrect` can create such patterns. For the Rasch model, and for 2PL with its bounded discriminations, this condition guarantees that finite estimates exist (the optimizer may still need more iterations to reach them). For 3PL it is necessary but not sufficient: fits can drift toward boundary solutions in which an item's difficulty keeps growing while its guessing parameter carries the item. Treat very large difficulties, or results with `converged: false`, as unreliable.
+- **3PL and all-incorrect respondents.** Under 3PL, an excluded respondent who answered every estimated item incorrectly still informs the guessing parameters: as their ability tends to minus infinity each response has probability `1 - c`, and that limiting term stays in the likelihood.
+- **Scale identification.** Results are reported on a fixed scale: Rasch abilities are centered to mean 0; 2PL and 3PL abilities are standardized to mean 0 and standard deviation 1, with difficulties and discriminations transformed so that response probabilities are unchanged. Only estimated respondents enter these statistics. The model keeps its internal parameters, so calling `fit` again continues from where it stopped; convergence is checked on the reported scale.
+- **Known limitations.** JMLE item estimates are biased when tests are short, and the bias does not vanish as the number of respondents grows. 2PL and 3PL JMLE is also slow to converge with plain gradient ascent. For high-stakes calibration, compare against marginal maximum likelihood software such as R's `mirt`.
 
 ## Performance Benchmarks
 
@@ -151,7 +192,7 @@ See `benchmarks/README.md` for detailed information about interpreting results.
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bundle exec rake spec` for the test suite or `bundle exec rake` for the default local quality gate: the RSpec suite and RuboCop. Before tagging a release, run `bundle exec rake build` to verify the gemspec packages cleanly from the current file list; CI also installs the built gem and smoke-tests `require 'irt_ruby'`. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 

data/benchmarks/convergence_benchmark.rb

@@ -5,46 +5,6 @@ require "bundler/setup"
 require "irt_ruby"
 require "benchmark"
 
-# Enhanced model classes that track iterations and convergence
-class TrackedRaschModel < IrtRuby::RaschModel
-  attr_reader :iterations, :final_log_likelihood, :convergence_reason
-
-  def fit
-    @iterations = 0
-    prev_ll = log_likelihood
-    @final_log_likelihood = prev_ll
-    @convergence_reason = :max_iterations
-
-    @max_iter.times do
-      @iterations += 1
-      grad_abilities, grad_difficulties = compute_gradient
-
-      old_a, old_d = apply_gradient_update(grad_abilities, grad_difficulties)
-
-      current_ll = log_likelihood
-      param_delta = average_param_update(old_a, old_d)
-
-      if current_ll < prev_ll
-        @abilities = old_a
-        @difficulties = old_d
-        @learning_rate *= @decay_factor
-      else
-        ll_diff = (current_ll - prev_ll).abs
-        @final_log_likelihood = current_ll
-
-        if ll_diff < @tolerance && param_delta < @param_tolerance
-          @convergence_reason = :tolerance_reached
-          break
-        end
-
-        prev_ll = current_ll
-      end
-    end
-
-    { abilities: @abilities, difficulties: @difficulties }
-  end
-end
-
 def generate_data(num_people, num_items, difficulty_range: (-2..2), ability_range: (-2..2))
   # Generate realistic IRT data based on known parameters
   true_abilities = Array.new(num_people) { rand(ability_range) }
@@ -84,27 +44,27 @@ tolerance_configs.each do |config|
 
   times = []
   iterations = []
-  convergence_reasons = []
+  convergences = []
 
   5.times do
     time = Benchmark.measure do
-      model = TrackedRaschModel.new(
+      model = IrtRuby::RaschModel.new(
         data,
         max_iter: 2000,
         tolerance: config[:tolerance],
         param_tolerance: config[:param_tolerance],
         learning_rate: 0.01
       )
-      model.fit
-      iterations << model.iterations
-      convergence_reasons << model.convergence_reason
+      result = model.fit
+      iterations << result[:iterations]
+      convergences << result[:converged]
     end.real
     times << time
   end
 
   avg_time = times.sum / times.size
   avg_iterations = iterations.sum.to_f / iterations.size
-  convergence_rate = convergence_reasons.count(:tolerance_reached) / 5.0
+  convergence_rate = convergences.count(true) / 5.0
 
   printf("  Time: %6.3fs  Iterations: %6.1f  Convergence Rate: %4.0f%%\n",
          avg_time, avg_iterations, convergence_rate * 100)
@@ -128,27 +88,27 @@ learning_rate_configs.each do |config|
 
   times = []
   iterations = []
-  convergence_reasons = []
+  convergences = []
 
   5.times do
     time = Benchmark.measure do
-      model = TrackedRaschModel.new(
+      model = IrtRuby::RaschModel.new(
         data,
         max_iter: 1000,
         tolerance: 1e-5,
         param_tolerance: 1e-5,
         learning_rate: config[:rate]
       )
-      model.fit
-      iterations << model.iterations
-      convergence_reasons << model.convergence_reason
+      result = model.fit
+      iterations << result[:iterations]
+      convergences << result[:converged]
     end.real
     times << time
   end
 
   avg_time = times.sum / times.size
   avg_iterations = iterations.sum.to_f / iterations.size
-  convergence_rate = convergence_reasons.count(:tolerance_reached) / 5.0
+  convergence_rate = convergences.count(true) / 5.0
 
   printf("  Time: %6.3fs  Iterations: %6.1f  Convergence Rate: %4.0f%%\n",
          avg_time, avg_iterations, convergence_rate * 100)
@@ -171,7 +131,7 @@ dataset_configs.each do |config|
 
   times = []
   iterations = []
-  convergence_reasons = []
+  convergences = []
 
   3.times do
     dataset = generate_data(
@@ -182,23 +142,23 @@ dataset_configs.each do |config|
     )
 
     time = Benchmark.measure do
-      model = TrackedRaschModel.new(
+      model = IrtRuby::RaschModel.new(
         dataset[:data],
         max_iter: 1000,
         tolerance: 1e-5,
         param_tolerance: 1e-5,
         learning_rate: 0.01
       )
-      model.fit
-      iterations << model.iterations
-      convergence_reasons << model.convergence_reason
+      result = model.fit
+      iterations << result[:iterations]
+      convergences << result[:converged]
     end.real
     times << time
   end
 
   avg_time = times.sum / times.size
   avg_iterations = iterations.sum.to_f / iterations.size
-  convergence_rate = convergence_reasons.count(:tolerance_reached) / 3.0
+  convergence_rate = convergences.count(true) / 3.0
 
   printf("  Time: %6.3fs  Iterations: %6.1f  Convergence Rate: %4.0f%%\n",
          avg_time, avg_iterations, convergence_rate * 100)
@@ -223,7 +183,7 @@ missing_configs.each do |config|
   # Generate data with missing values
   base_data = generate_data(100, 50)[:data]
 
-  data_with_missing = if (config[:rate]).positive?
+  data_with_missing = if config[:rate].positive?
                         base_data.map do |row|
                           row.map { |resp| rand < config[:rate] ? nil : resp }
                         end
@@ -233,11 +193,11 @@ missing_configs.each do |config|
 
   times = []
   iterations = []
-  convergence_reasons = []
+  convergences = []
 
   3.times do
     time = Benchmark.measure do
-      model = TrackedRaschModel.new(
+      model = IrtRuby::RaschModel.new(
         data_with_missing,
         max_iter: 1000,
         tolerance: 1e-5,
@@ -245,16 +205,16 @@ missing_configs.each do |config|
         learning_rate: 0.01,
         missing_strategy: config[:strategy]
       )
-      model.fit
-      iterations << model.iterations
-      convergence_reasons << model.convergence_reason
+      result = model.fit
+      iterations << result[:iterations]
+      convergences << result[:converged]
     end.real
     times << time
   end
 
   avg_time = times.sum / times.size
   avg_iterations = iterations.sum.to_f / iterations.size
-  convergence_rate = convergence_reasons.count(:tolerance_reached) / 3.0
+  convergence_rate = convergences.count(true) / 3.0
 
   printf("  Time: %6.3fs  Iterations: %6.1f  Convergence Rate: %4.0f%%\n",
          avg_time, avg_iterations, convergence_rate * 100)

data/lib/irt_ruby/estimation.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module IrtRuby
+  # Numerical helpers shared by IRT model implementations.
+  # @api private
+  module Estimation
+    # Learning-rate multiplier after an accepted step, so the rate can recover from earlier decays.
+    LEARNING_RATE_GROWTH = 1.1
+
+    module_function
+
+    def log_sigmoid(x)
+      x >= 0 ? -Math.log(1.0 + Math.exp(-x)) : x - Math.log(1.0 + Math.exp(x))
+    end
+
+    def each_response(responses, persons, items)
+      responses.each_with_index do |row, i|
+        next unless persons[i]
+
+        row.each_with_index do |value, j|
+          next if value.nil? || !items[j]
+
+          yield i, j, value
+        end
+      end
+    end
+
+    def response_counts(responses, persons, items)
+      person_counts = Array.new(responses.size, 0)
+      item_counts = Array.new(responses.first.size, 0)
+      each_response(responses, persons, items) do |i, j, _value|
+        person_counts[i] += 1
+        item_counts[j] += 1
+      end
+      [person_counts, item_counts]
+    end
+
+    # True when every gradient component, averaged over the responses it sums, is below tolerance.
+    def stationary?(person_gradient, item_gradients, person_counts, item_counts, tolerance)
+      small_residuals?(person_gradient, person_counts, tolerance) &&
+        item_gradients.all? { |gradient| small_residuals?(gradient, item_counts, tolerance) }
+    end
+
+    def small_residuals?(gradient, counts, tolerance)
+      gradient.each_index.all? { |k| counts[k].zero? || (gradient[k] / counts[k]).abs < tolerance }
+    end
+
+    # Drops gradient components that push a parameter further past the bound it sits on.
+    def project(gradient, values, min, max)
+      gradient.each_index.map do |k|
+        blocked = (values[k] <= min && gradient[k].negative?) || (values[k] >= max && gradient[k].positive?)
+        blocked ? 0.0 : gradient[k]
+      end
+    end
+
+    # Location and scale of the estimable abilities, used to report parameters on a fixed scale.
+    def scale(abilities, active)
+      values = abilities.select.with_index { |_, i| active[i] }
+      return [0.0, 1.0] if values.empty?
+
+      mean, sd = mean_and_sd(values)
+      [mean, sd.zero? ? 1.0 : sd]
+    end
+
+    def mean_and_sd(values)
+      mean = values.sum / values.size
+      [mean, Math.sqrt(values.sum { |v| (v - mean)**2 } / values.size)]
+    end
+
+    def changes(current, previous, active)
+      current.each_index.select { |k| active[k] }.map { |k| (current[k] - previous[k]).abs }
+    end
+
+    def estimates(values, active)
+      values.each_index.map { |k| active[k] ? values[k] : nil }
+    end
+  end
+end

data/lib/irt_ruby/model_options_validator.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+module IrtRuby
+  # Validates optimization hyperparameters shared by IRT model implementations.
+  module ModelOptionsValidator
+    module_function
+
+    def validate!(max_iter:, tolerance:, param_tolerance:, gradient_tolerance:, learning_rate:, decay_factor:)
+      validate_positive_integer!(:max_iter, max_iter)
+      validate_positive_finite_numeric!(:tolerance, tolerance)
+      validate_positive_finite_numeric!(:param_tolerance, param_tolerance)
+      validate_positive_finite_numeric!(:gradient_tolerance, gradient_tolerance)
+      validate_positive_finite_numeric!(:learning_rate, learning_rate)
+      validate_decay_factor!(decay_factor)
+    end
+
+    def validate_positive_integer!(name, value)
+      return if value.is_a?(Integer) && value.positive?
+
+      raise ArgumentError, "#{name} must be a positive Integer"
+    end
+
+    def validate_positive_finite_numeric!(name, value)
+      return if finite_numeric?(value) && value.positive?
+
+      raise ArgumentError, "#{name} must be a positive finite Numeric"
+    end
+
+    def validate_decay_factor!(value)
+      return if finite_numeric?(value) && value.positive? && value < 1
+
+      raise ArgumentError, "decay_factor must be a finite Numeric strictly between 0 and 1"
+    end
+
+    def finite_numeric?(value)
+      value.is_a?(Numeric) && !value.is_a?(Complex) && value.finite?
+    end
+  end
+end