lda-ruby 0.5.0-x64-mingw-ucrt

data/docs/precompiled-platform-policy.md

@@ -0,0 +1,81 @@
+# Precompiled Platform Gem Policy (Phase 5B)
+
+This document defines the publish strategy and compatibility policy for `lda-ruby` precompiled gems.
+
+## Artifact Strategy
+
+Each release version publishes a split package set:
+
+- Source gem: `lda-ruby-<version>.gem`
+- Precompiled platform gems:
+  - `lda-ruby-<version>-x86_64-linux.gem`
+  - `lda-ruby-<version>-x86_64-darwin.gem`
+  - `lda-ruby-<version>-arm64-darwin.gem`
+  - `lda-ruby-<version>-x64-mingw-ucrt.gem`
+  - `lda-ruby-<version>-x86_64-linux-musl.gem`
+
+The source gem remains the universal fallback. Platform gems are additive and are expected to install without local build tools.
+Precompiled artifacts are built on matching host runners (no cross-compilation in current workflow).
+
+## Compatibility Policy
+
+- Supported Ruby versions: 3.2 and 3.3 (plus future versions validated by CI).
+- Release-blocking precompiled targets:
+  - Linux `x86_64-linux`
+  - Linux musl `x86_64-linux-musl`
+  - macOS Intel `x86_64-darwin`
+  - macOS Apple Silicon `arm64-darwin`
+  - Windows `x64-mingw-ucrt`
+- Other platforms:
+  - Install from source gem.
+  - Runtime remains supported through native/pure fallback paths.
+
+Backend behavior expectations:
+
+- Platform gem install:
+  - `auto` backend resolves to `rust` by default.
+  - `native` and `pure` overrides continue to work.
+- Source gem install:
+  - Rust build policy is controlled by `LDA_RUBY_RUST_BUILD=auto|always|never`.
+  - If Rust build is skipped/unavailable, `auto` falls back to `native`, then `pure_ruby`.
+
+## Guardrails
+
+Validation must pass before publish:
+
+- `./bin/release-preflight` (source-gem checks).
+- `./bin/release-precompiled-artifacts --platform <target>` for each release-blocking platform.
+
+Release automation requirements:
+
+- `.github/workflows/release.yml` builds source + precompiled artifacts.
+- Release workflow matrix must include all release-blocking precompiled targets.
+- Publish jobs push all built gems and attach checksums to GitHub releases.
+- Post-publish verification job must validate RubyGems entries and GitHub release assets for the tagged version.
+
+Continuous integration guardrail:
+
+- `.github/workflows/ci.yml` runs `release-precompiled-artifacts` for the full release-blocking precompiled matrix (Linux, Linux musl, macOS Intel, macOS Apple Silicon, Windows) on every branch/PR.
+- macOS precompiled lanes pin Homebrew `llvm@18` (falling back to `llvm` if unavailable) and export `LIBCLANG_PATH` from the selected prefix to keep bindgen stable across Homebrew formula updates.
+- `.github/workflows/precompiled-candidate-evaluation.yml` is used for additional platform candidate checks.
+- `.github/workflows/release.yml` dry-run validates the full release-blocking matrix before publish.
+
+Latest release-matrix validation:
+
+- [release dry-run 22556487788](https://github.com/ealdent/lda-ruby/actions/runs/22556487788) succeeded for Linux, Linux musl, macOS Intel, macOS Apple Silicon, and Windows targets.
+
+## Rollout / Expansion Rules
+
+When adding a new precompiled platform:
+
+1. Add target to release workflow matrix.
+2. Add or update CI coverage for that platform family.
+3. Update this policy and the release runbook support matrix.
+4. Record feasibility evidence and rollout notes in `docs/precompiled-target-evaluation.md`.
+5. Validate a dry-run release with `workflow_dispatch` before shipping.
+
+When deprecating a precompiled platform:
+
+1. Remove platform from release matrix.
+2. Keep source-gem path available unless the overall platform support policy changes.
+3. Document deprecation in `CHANGELOG.md` and release notes.

data/docs/precompiled-target-evaluation.md

@@ -0,0 +1,67 @@
+# Precompiled Target Evaluation (Priority 2)
+
+This document tracks current feasibility for expanding precompiled gem targets beyond the Phase 5B baseline.
+
+Current release-blocking precompiled targets:
+
+- `x86_64-linux`
+- `x86_64-darwin`
+- `arm64-darwin`
+- `x64-mingw-ucrt`
+- `x86_64-linux-musl`
+
+Reference implementation constraints:
+
+- `bin/release-precompiled-artifacts` only supports host-matching platform builds (no cross-compilation).
+- Release workflow currently uses matching host runners for each precompiled target.
+
+## Candidate: Windows (`x64-mingw-ucrt`)
+
+Status: promoted to release-blocking after release dry-run matrix success.
+
+Feasibility notes:
+
+- GitHub Actions provides Windows runners, so host-matching builds are possible in principle.
+- Existing release tooling is bash-first and assumes POSIX shell ergonomics throughout.
+- Runtime smoke and packaged-gem checks were validated in candidate runs before promotion.
+- Candidate runs:
+  - [run 22555475302](https://github.com/ealdent/lda-ruby/actions/runs/22555475302): failed in native extension compile (`rake compile`) with `cokus.h` macro collision and `time_t` mismatch.
+  - [run 22555550326](https://github.com/ealdent/lda-ruby/actions/runs/22555550326): progressed further, failed on `utils.c` `mkdir(name, mode)` mismatch (Windows `_mkdir` required).
+  - [run 22556009214](https://github.com/ealdent/lda-ruby/actions/runs/22556009214): Rust bindgen/toolchain parsing fixed; build then failed on Windows DLL name staging expectation.
+  - [run 22556129503](https://github.com/ealdent/lda-ruby/actions/runs/22556129503): Windows candidate build + artifact upload succeeded after GNU toolchain alignment, bindgen header/sysroot setup, and dual DLL name staging support.
+  - [run 22556206925](https://github.com/ealdent/lda-ruby/actions/runs/22556206925): Windows candidate remained green with packaged-gem runtime smoke checks enabled.
+  - [run 22556487788](https://github.com/ealdent/lda-ruby/actions/runs/22556487788): release workflow dry-run succeeded with `windows-x64-mingw-ucrt` included in release matrix.
+
+Required validation to promote:
+
+1. Completed: release dry-run matrix validation passed.
+
+## Candidate: musl Linux (`x86_64-linux-musl`)
+
+Status: promoted to release-blocking after release dry-run matrix success.
+
+Feasibility notes:
+
+- Current workflow uses `ubuntu-latest` (glibc), not musl.
+- Current artifact script rejects cross-platform builds, so a musl artifact requires either:
+  - a musl-hosted builder, or
+  - a dedicated musl-native build container/workflow path treated as host-equivalent for packaging.
+- Local validation signal (2026-03-01): Alpine container dry-run succeeded for host-matching `aarch64-linux-musl` with:
+  - `./bin/release-precompiled-artifacts --platform <detected-musl-platform> --skip-preflight --skip-runtime-checks`
+- Candidate workflow runs (2026-03-01):
+  - [run 22555475302](https://github.com/ealdent/lda-ruby/actions/runs/22555475302): built `x86_64-linux-musl` successfully but artifact upload path was misconfigured.
+  - [run 22555550326](https://github.com/ealdent/lda-ruby/actions/runs/22555550326): musl candidate built and uploaded artifacts successfully with corrected glob path (`pkg/lda-ruby-*-linux-musl.gem*`).
+  - [run 22556129503](https://github.com/ealdent/lda-ruby/actions/runs/22556129503): musl candidate build + artifact upload remained green alongside the fixed Windows lane.
+  - [run 22556206925](https://github.com/ealdent/lda-ruby/actions/runs/22556206925): musl candidate remained green with packaged-gem runtime smoke checks enabled.
+  - [run 22556487788](https://github.com/ealdent/lda-ruby/actions/runs/22556487788): release workflow dry-run succeeded with `linux-musl-x86_64` included in release matrix.
+
+Required validation to promote:
+
+1. Completed: release dry-run matrix validation passed.
+
+## Recommendation
+
+Current expansion step is complete for Windows and musl. Any additional target should follow the same sequence:
+1. Add candidate workflow coverage.
+2. Verify candidate runtime checks.
+3. Validate one release dry-run with the new matrix lane before promotion.

data/docs/release-runbook.md

@@ -0,0 +1,192 @@
+# Release Runbook (Phase 5A + 5B)
+
+This runbook defines the maintainer workflow for shipping `lda-ruby` source and precompiled platform gem releases.
+
+Authoritative platform/support policy is maintained in `docs/precompiled-platform-policy.md`; expansion feasibility notes live in `docs/precompiled-target-evaluation.md`.
+
+## Scope
+
+- Release artifact types:
+  - source gem: `pkg/lda-ruby-<version>.gem`
+  - precompiled gems (current targets are defined in `docs/precompiled-platform-policy.md`)
+- Release trigger: git tag (`vX.Y.Z`) with matching version files
+- Publish targets:
+  - RubyGems (`gem push`)
+  - GitHub Releases (gem + checksum attachment)
+
+## Prerequisites
+
+1. Access:
+   - push/tag rights on `master`
+   - access to GitHub Actions environments for release approvals
+   - RubyGems owner access for `lda-ruby`
+2. Local tooling:
+   - Ruby 3.2+ with Bundler
+   - Rust toolchain (`cargo`) for local precompiled-gem build checks
+   - `libclang` available to Rust bindgen
+   - Docker (recommended for reproducible checks)
+3. Repository state:
+   - release commit merged to `master`
+   - clean working tree
+   - version files in sync
+
+## Required Secrets and Environments
+
+GitHub repository secret:
+
+- `RUBYGEMS_API_KEY`: API key with push rights for `lda-ruby` and non-interactive publish support (no OTP prompt during `gem push`).
+
+GitHub Actions environment:
+
+- `release`: protect this environment with required reviewer approval.
+- Both publish jobs in `.github/workflows/release.yml` are bound to `release`.
+
+## Release Preparation
+
+1. Prepare and update release files:
+
+   ```bash
+   ./bin/release-prepare 0.4.0
+   ```
+
+2. Review changes:
+   - `VERSION.yml`
+   - `lib/lda-ruby/version.rb`
+   - `CHANGELOG.md`
+
+3. Validate full release checks locally:
+
+   ```bash
+   SKIP_DOCKER=1 ./bin/release-preflight
+   ./bin/test-packaged-gem-manifest
+   ```
+
+4. Validate local precompiled gem flow for your current host platform:
+
+   ```bash
+   ./bin/release-precompiled-artifacts --tag v0.4.0 --skip-preflight
+   ```
+
+   Note: `release-precompiled-artifacts` only supports building for the current host platform (no cross-compilation).
+
+5. Verify RubyGems API key behavior before tagging:
+
+   ```bash
+   ./bin/verify-rubygems-api-key
+   ```
+
+   This check intentionally attempts a duplicate push of an existing gem version. A duplicate-rejected response is expected and confirms non-interactive auth works.
+
+6. Commit and merge to `master`.
+
+## Dry-Run Path (No Publish)
+
+Use `workflow_dispatch` with `publish=false`.
+
+Behavior:
+
+- runs release validation and artifact build
+- uploads source + precompiled `pkg/lda-ruby-*.gem` and checksum files as workflow artifacts
+- does not push to RubyGems
+- does not create a GitHub release
+
+Latest verified dry-run reference:
+
+- date: 2026-03-02
+- workflow run: `https://github.com/ealdent/lda-ruby/actions/runs/22556487788`
+- dispatch parameters: `release_tag=v0.4.0`, `publish=false`
+- result: success across `validate`, `build_artifacts`, and full `build_precompiled_artifacts` matrix
+  - verified precompiled lanes:
+    - `linux-x86_64`
+    - `linux-musl-x86_64`
+    - `macos-x86_64`
+    - `macos-arm64`
+    - `windows-x64-mingw-ucrt`
+
+Optional local dry-run equivalent:
+
+```bash
+./bin/release-artifacts --tag v0.4.0
+./bin/release-precompiled-artifacts --tag v0.4.0 --skip-preflight
+```
+
+Candidate expansion workflow:
+
+- For future platform evaluation beyond current release-blocking targets, run `.github/workflows/precompiled-candidate-evaluation.yml` via `workflow_dispatch`.
+- Record outcome artifacts/logs in `docs/precompiled-target-evaluation.md`.
+
+## Known Publish Incident (`v0.4.0`)
+
+- date: 2026-02-25
+- release runs:
+  - `https://github.com/ealdent/lda-ruby/actions/runs/22383716372`
+  - `https://github.com/ealdent/lda-ruby/actions/runs/22383849236` (attempt 1 + rerun attempt 2 + rerun attempt 3)
+- result: artifact build stages passed, `publish to RubyGems` failed with OTP-required auth (`You have enabled multifactor authentication but no OTP code provided.`)
+- recovery action: rotated `release` environment secret `RUBYGEMS_API_KEY` to a CI-safe key and reran run `22383849236`.
+- recovery result: rerun attempt 3 succeeded; RubyGems `0.4.0` and GitHub release `v0.4.0` published.
+
+## Publish Path (Tag-Driven)
+
+1. Ensure the release commit is on `master`.
+2. Create and push the release tag:
+
+   ```bash
+   git checkout master
+   git pull --ff-only
+   git tag -a v0.4.0 -m "Release v0.4.0"
+   git push origin v0.4.0
+   ```
+
+3. Monitor `.github/workflows/release.yml`:
+   - `validate`
+   - `build_artifacts`
+   - `build_precompiled_artifacts` (linux + linux-musl + macOS + windows matrix)
+   - environment-gated `publish_rubygems`
+   - environment-gated `publish_github_release`
+   - `verify_published_artifacts`
+   - on failed tag-triggered `release.yml` runs, `.github/workflows/release-failure-alert.yml` opens a triage issue with failed job links
+   - if the same release run later succeeds (for example via rerun), the alert issue is auto-closed by `.github/workflows/release-failure-alert.yml`
+4. Approve the protected `release` environment when prompted.
+5. Confirm published outputs:
+   - RubyGems shows `lda-ruby` `0.4.0` source gem and platform gems
+   - GitHub release `v0.4.0` exists with all gem and `.sha256` attachments
+   - workflow job `verify_published_artifacts` succeeds
+
+## Rollback and Recovery
+
+If publish fails before RubyGems push:
+
+1. Fix issue on `master`.
+2. Delete and recreate the tag only if the broken tag did not produce public artifacts:
+   - `git tag -d vX.Y.Z`
+   - `git push origin :refs/tags/vX.Y.Z`
+3. Re-tag and re-run release.
+
+If RubyGems push succeeds but GitHub release fails:
+
+1. Re-run only the GitHub release path by re-running the workflow job after fix.
+2. Do not re-push gem for the same version.
+
+If an incorrect gem is published:
+
+1. Yank from RubyGems:
+
+   ```bash
+   gem yank lda-ruby -v X.Y.Z
+   ```
+
+2. Publish a corrective version (for example `X.Y.(Z+1)`), do not re-use yanked version numbers.
+3. Update `CHANGELOG.md` and release notes to document the correction.
+
+## Troubleshooting
+
+- `Could not find 'bundler'`: install the Bundler version pinned in `Gemfile.lock`.
+- `cargo not found` in rust-enabled checks: ensure Rust toolchain is installed or run in Docker.
+- `libclang` not found while building precompiled gems: install LLVM/libclang and set `LIBCLANG_PATH` if needed.
+- Linux `Install Rust bindgen dependencies` can take several minutes on fresh runners due apt package index and package installs.
+- RubyGems publish asks for OTP (`You have enabled multi-factor authentication but no OTP code provided`): run `./bin/verify-rubygems-api-key`, then rotate `RUBYGEMS_API_KEY` to a CI-safe key if OTP is requested.
+- Post-publish verification fails: run `./bin/verify-release-publish --tag vX.Y.Z` and fix missing RubyGems entries or GitHub release assets before considering the release complete.
+- macOS Rust link errors (`symbol(s) not found` for Ruby APIs): ensure build path preserves `-C link-arg=-Wl,-undefined,dynamic_lookup` in `RUSTFLAGS`.
+- Tag/version mismatch: run `./bin/check-version-sync --tag vX.Y.Z`.
+- Artifact mismatch during release: rebuild with `./bin/release-artifacts --tag vX.Y.Z`.
+- Precompiled artifact mismatch: rebuild with `./bin/release-precompiled-artifacts --tag vX.Y.Z --skip-preflight`.

data/docs/rust-orchestration-guardrails.md

@@ -0,0 +1,50 @@
+# Rust Orchestration Guardrails
+
+This document defines the minimum parity and performance gates for deeper Rust orchestration refactors.
+
+## Numeric parity guardrails
+
+Required tests:
+
+- `bundle exec ruby -Ilib:test test/backend_compatibility_test.rb`
+- `bundle exec ruby -Ilib:test test/rust_orchestration_test.rb`
+
+Current parity expectations:
+
+- Rust vs pure backend fixture parity remains exact within existing tolerances used by tests.
+- Session-based orchestration paths (`run_em_on_session`, `run_em_on_session_with_start_seed`, `run_em_on_session_start`, `run_em_on_session_with_corpus`) must match direct non-session orchestration for equivalent settings/seeds.
+- `Lda::Backends::Rust` cached-corpus EM should prefer the managed Rust session entrypoint (`run_em_on_session_with_corpus`) even when no active session id is cached locally, rather than branching in Ruby between session-only, recovery, and direct paths.
+- `Lda::Backends::Rust` non-session fallback should prefer Rust start-aware orchestration (`run_em_with_start_seed`) before legacy beta-input orchestration (`run_em`).
+- Direct non-session fallback should reuse the backend's cached Rust corpus snapshot rather than rebuilding corpus arrays from `@corpus` for each invocation.
+- Legacy beta-input compatibility fallback should also reuse the backend's cached Rust corpus snapshot rather than rebuilding full EM corpus input in Ruby.
+- Rust backend corpus/session lifecycle must not leak session count across corpus replacement.
+- Missing-session recovery in managed session orchestration (`run_em_on_session_with_corpus`) must recreate a usable session and keep parity with direct orchestration.
+- Managed Rust corpus orchestration (`run_em_on_session_with_corpus`) must keep parity with direct orchestration even when it falls back internally from session-backed execution to start-seeded array execution.
+- Corpus reassignment through Rust session replacement lifecycle (`replace_corpus_session`) must preserve stable session count and route subsequent EM runs over updated corpus data.
+- Unknown start-mode handling in seed-aware Rust orchestration must match Ruby's non-seeded fallback behavior when given the same explicit seed.
+
+## Benchmark guardrail
+
+Run:
+
+- `./bin/check-rust-benchmark`
+
+Default benchmark policy:
+
+- `BENCH_RUST_TO_PURE_MAX_RATIO=0.045`
+  - i.e., Rust mean runtime must be no worse than 4.5% of pure mean runtime on the benchmark fixture/config.
+- CI benchmark guardrail job enforces the same ratio with `BENCH_RUNS=1` for runtime stability.
+- latest tightening evidence (2026-03-05): local Docker guardrail check with `BENCH_RUNS=3` observed Rust/Pure ratio `0.0368` (`rust=0.0758s`, `pure=2.0569s`), and prior CI streak data on `codex/rust-orchestration-phase8` (`22555725309` .. `22557953998`) observed `[0.0252, 0.0288]`, supporting a tighter `0.045` threshold with headroom.
+
+Configurable environment knobs:
+
+- `BENCH_RUNS` (default `5`)
+- `BENCH_START` (default `seeded`)
+- `BENCH_TOPICS` (default `8`)
+- `BENCH_MAX_ITER` (default `20`)
+- `BENCH_EM_MAX_ITER` (default `40`)
+- `BENCH_RUST_TO_PURE_MAX_RATIO` (default `0.045`)
+
+## When to tighten thresholds
+
+Tighten benchmark thresholds only after collecting multiple stable runs on the same host/environment and updating this document with the new target ratio.

data/ext/lda-ruby/cokus.c

@@ -0,0 +1,144 @@
+// This is the ``Mersenne Twister'' random number generator MT19937, which
+// generates pseudorandom integers uniformly distributed in 0..(2^32 - 1)
+// starting from any odd seed in 0..(2^32 - 1).  This version is a recode
+// by Shawn Cokus (Cokus@math.washington.edu) on March 8, 1998 of a version by
+// Takuji Nishimura (who had suggestions from Topher Cooper and Marc Rieffel in
+// July-August 1997).
+//
+// Effectiveness of the recoding (on Goedel2.math.washington.edu, a DEC Alpha
+// running OSF/1) using GCC -O3 as a compiler: before recoding: 51.6 sec. to
+// generate 300 million random numbers; after recoding: 24.0 sec. for the same
+// (i.e., 46.5% of original time), so speed is now about 12.5 million random
+// number generations per second on this machine.
+//
+// According to the URL <http://www.math.keio.ac.jp/~matumoto/emt.html>
+// (and paraphrasing a bit in places), the Mersenne Twister is ``designed
+// with consideration of the flaws of various existing generators,'' has
+// a period of 2^19937 - 1, gives a sequence that is 623-dimensionally
+// equidistributed, and ``has passed many stringent tests, including the
+// die-hard test of G. Marsaglia and the load test of P. Hellekalek and
+// S. Wegenkittl.''  It is efficient in memory usage (typically using 2506
+// to 5012 bytes of static data, depending on data type sizes, and the code
+// is quite short as well).  It generates random numbers in batches of 624
+// at a time, so the caching and pipelining of modern systems is exploited.
+// It is also divide- and mod-free.
+//
+// This library is free software; you can redistribute it and/or modify it
+// under the terms of the GNU Library General Public License as published by
+// the Free Software Foundation (either version 2 of the License or, at your
+// option, any later version).  This library is distributed in the hope that
+// it will be useful, but WITHOUT ANY WARRANTY, without even the implied
+// warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+// the GNU Library General Public License for more details.  You should have
+// received a copy of the GNU Library General Public License along with this
+// library; if not, write to the Free Software Foundation, Inc., 59 Temple
+// Place, Suite 330, Boston, MA 02111-1307, USA.
+//
+// The code as Shawn received it included the following notice:
+//
+//   Copyright (C) 1997 Makoto Matsumoto and Takuji Nishimura.  When
+//   you use this, send an e-mail to <matumoto@math.keio.ac.jp> with
+//   an appropriate reference to your work.
+//
+// It would be nice to CC: <Cokus@math.washington.edu> when you write.
+//
+
+#include "cokus.h"
+
+static uint32   state[COKUS_N+1];     // state vector + 1 extra to not violate ANSI C
+static uint32   *next;          // next random value is computed from here
+static int      left = -1;      // can *next++ this many times before reloading
+
+void seedMT(uint32 seed)
+ {
+    //
+    // We initialize state[0..(COKUS_N-1)] via the generator
+    //
+    //   x_new = (69069 * x_old) mod 2^32
+    //
+    // from Line 15 of Table 1, p. 106, Sec. 3.3.4 of Knuth's
+    // _The Art of Computer Programming_, Volume 2, 3rd ed.
+    //
+    // Notes (SJC): I do not know what the initial state requirements
+    // of the Mersenne Twister are, but it seems this seeding generator
+    // could be better.  It achieves the maximum period for its modulus
+    // (2^30) iff x_initial is odd (p. 20-21, Sec. 3.2.1.2, Knuth); if
+    // x_initial can be even, you have sequences like 0, 0, 0, ...;
+    // 2^31, 2^31, 2^31, ...; 2^30, 2^30, 2^30, ...; 2^29, 2^29 + 2^31,
+    // 2^29, 2^29 + 2^31, ..., etc. so I force seed to be odd below.
+    //
+    // Even if x_initial is odd, if x_initial is 1 mod 4 then
+    //
+    //   the          lowest bit of x is always 1,
+    //   the  next-to-lowest bit of x is always 0,
+    //   the 2nd-from-lowest bit of x alternates      ... 0 1 0 1 0 1 0 1 ... ,
+    //   the 3rd-from-lowest bit of x 4-cycles        ... 0 1 1 0 0 1 1 0 ... ,
+    //   the 4th-from-lowest bit of x has the 8-cycle ... 0 0 0 1 1 1 1 0 ... ,
+    //    ...
+    //
+    // and if x_initial is 3 mod 4 then
+    //
+    //   the          lowest bit of x is always 1,
+    //   the  next-to-lowest bit of x is always 1,
+    //   the 2nd-from-lowest bit of x alternates      ... 0 1 0 1 0 1 0 1 ... ,
+    //   the 3rd-from-lowest bit of x 4-cycles        ... 0 0 1 1 0 0 1 1 ... ,
+    //   the 4th-from-lowest bit of x has the 8-cycle ... 0 0 1 1 1 1 0 0 ... ,
+    //    ...
+    //
+    // The generator's potency (min. s>=0 with (69069-1)^s = 0 mod 2^32) is
+    // 16, which seems to be alright by p. 25, Sec. 3.2.1.3 of Knuth.  It
+    // also does well in the dimension 2..5 spectral tests, but it could be
+    // better in dimension 6 (Line 15, Table 1, p. 106, Sec. 3.3.4, Knuth).
+    //
+    // Note that the random number user does not see the values generated
+    // here directly since reloadMT() will always munge them first, so maybe
+    // none of all of this matters.  In fact, the seed values made here could
+    // even be extra-special desirable if the Mersenne Twister theory says
+    // so-- that's why the only change I made is to restrict to odd seeds.
+    //
+
+    register uint32 x = (seed | 1U) & 0xFFFFFFFFU, *s = state;
+    register int    j;
+
+    for(left=0, *s++=x, j=COKUS_N; --j;
+        *s++ = (x*=69069U) & 0xFFFFFFFFU);
+ }
+
+
+uint32 reloadMT(void)
+{
+    register uint32 *p0=state, *p2=state+2, *pM=state+COKUS_M, s0, s1;
+    register int    j;
+
+    if(left < -1)
+        seedMT(4357U);
+
+    left=COKUS_N-1, next=state+1;
+
+    for(s0=state[0], s1=state[1], j=COKUS_N-COKUS_M+1; --j; s0=s1, s1=*p2++)
+        *p0++ = *pM++ ^ (mixBits(s0, s1) >> 1) ^ (loBit(s1) ? COKUS_K : 0U);
+
+    for(pM=state, j=COKUS_M; --j; s0=s1, s1=*p2++)
+        *p0++ = *pM++ ^ (mixBits(s0, s1) >> 1) ^ (loBit(s1) ? COKUS_K : 0U);
+
+    s1=state[0], *p0 = *pM ^ (mixBits(s0, s1) >> 1) ^ (loBit(s1) ? COKUS_K : 0U);
+    s1 ^= (s1 >> 11);
+    s1 ^= (s1 <<  7) & 0x9D2C5680U;
+    s1 ^= (s1 << 15) & 0xEFC60000U;
+    return(s1 ^ (s1 >> 18));
+ }
+
+uint32 randomMT(void)
+ {
+    uint32 y;
+
+    if(--left < 0)
+        return(reloadMT());
+
+    y  = *next++;
+    y ^= (y >> 11);
+    y ^= (y <<  7) & 0x9D2C5680U;
+    y ^= (y << 15) & 0xEFC60000U;
+    y ^= (y >> 18);
+    return(y);
+ }

data/ext/lda-ruby/cokus.h

@@ -0,0 +1,27 @@
+#ifndef COKUS_H
+#define COKUS_H
+
+#include <stdio.h>
+#include <stdlib.h>
+
+//
+// uint32 must be an unsigned integer type capable of holding at least 32
+// bits; exactly 32 should be fastest, but 64 is better on an Alpha with
+// GCC at -O3 optimization so try your options and see what's best for you
+//
+
+typedef unsigned long uint32;
+
+#define COKUS_N        (624)                 // length of state vector
+#define COKUS_M        (397)                 // a period parameter
+#define COKUS_K        (0x9908B0DFU)         // a magic constant
+#define hiBit(u)       ((u) & 0x80000000U)   // mask all but highest   bit of u
+#define loBit(u)       ((u) & 0x00000001U)   // mask all but lowest    bit of u
+#define loBits(u)      ((u) & 0x7FFFFFFFU)   // mask     the highest   bit of u
+#define mixBits(u, v)  (hiBit(u)|loBits(v))  // move hi bit of u to hi bit of v
+
+void seedMT(uint32 seed);
+uint32 reloadMT(void);
+uint32 randomMT(void);
+
+#endif

data/ext/lda-ruby/extconf.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+require "mkmf"
+
+extension_name = "lda-ruby/lda"
+dir_config(extension_name)
+
+$defs << "-DUSE_RUBY"
+append_cflags("-Wall")
+append_cflags("-Wextra")
+append_cflags("-Wno-unused-parameter")
+
+create_makefile(extension_name)

data/ext/lda-ruby/lda-alpha.c

@@ -0,0 +1,96 @@
+// (C) Copyright 2004, David M. Blei (blei [at] cs [dot] cmu [dot] edu)
+
+// This file is part of LDA-C.
+
+// LDA-C is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 2 of the License, or (at your
+// option) any later version.
+
+// LDA-C is distributed in the hope that it will be useful, but WITHOUT
+// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
+// USA
+
+#include "lda-alpha.h"
+
+/*
+ * objective function and its derivatives
+ *
+ */
+
+double alhood(double a, double ss, int D, int K)
+{ return(D * (lgamma(K * a) - K * lgamma(a)) + (a - 1) * ss); }
+
+double d_alhood(double a, double ss, int D, int K)
+{ return(D * (K * digamma(K * a) - K * digamma(a)) + ss); }
+
+double d2_alhood(double a, int D, int K)
+{ return(D * (K * K * trigamma(K * a) - K * trigamma(a))); }
+
+
+/*
+ * newtons method
+ *
+ */
+
+double opt_alpha(double ss, int D, int K)
+{
+    double a, log_a, init_a = 100;
+    double f, df, d2f;
+    int iter = 0;
+
+    log_a = log(init_a);
+    do
+    {
+        iter++;
+        a = exp(log_a);
+        if (isnan(a))
+        {
+            init_a = init_a * 10;
+            printf("warning : alpha is nan; new init = %5.5f\n", init_a);
+            a = init_a;
+            log_a = log(a);
+        }
+        f = alhood(a, ss, D, K);
+        df = d_alhood(a, ss, D, K);
+        d2f = d2_alhood(a, D, K);
+        log_a = log_a - df/(d2f * a + df);
+        printf("alpha maximization : %5.5f   %5.5f\n", f, df);
+    }
+    while ((fabs(df) > NEWTON_THRESH) && (iter < MAX_ALPHA_ITER));
+    return(exp(log_a));
+}
+
+double quiet_opt_alpha(double ss, int D, int K)
+{
+    double a, log_a, init_a = 100;
+    double f, df, d2f;
+    int iter = 0;
+
+    log_a = log(init_a);
+    do
+    {
+        iter++;
+        a = exp(log_a);
+        if (isnan(a))
+        {
+            init_a = init_a * 10;
+            //printf("warning : alpha is nan; new init = %5.5f\n", init_a);
+            a = init_a;
+            log_a = log(a);
+        }
+        f = alhood(a, ss, D, K);
+        df = d_alhood(a, ss, D, K);
+        d2f = d2_alhood(a, D, K);
+        log_a = log_a - df/(d2f * a + df);
+        //printf("alpha maximization : %5.5f   %5.5f\n", f, df);
+    }
+    while ((fabs(df) > NEWTON_THRESH) && (iter < MAX_ALPHA_ITER));
+    return(exp(log_a));
+}