pystatistics 3.2.0__tar.gz → 3.4.0__tar.gz

pystatistics-3.4.0/.release/UNRELEASED.md

@@ -0,0 +1,29 @@
+# Unreleased Changes
+
+> This file tracks all changes since the last stable release.
+> Updated by whoever makes a change, on whatever machine.
+> Synced via git so all sessions (Mac, Linux, etc.) see the same state.
+>
+> When ready to release, run: `python .release/release.py --status`
+> and follow the manual release flow in the script docstring.
+
+## Changes
+
+- GPU backend for `mice` (CUDA). `mice(..., backend='gpu')` runs the `m`
+  imputation chains batched on the GPU as the leading tensor dimension: each
+  sweep step is a batched Bayesian linear solve (`torch.linalg.solve` + batched
+  Cholesky) and, for PMM, a batched nearest-neighbour donor search
+  (`(m, n_mis, n_obs)` distances → `topk`). `backend='auto'` selects CUDA when
+  available, else CPU (never MPS, matching the other modules). New `use_fp64`
+  argument runs the GPU path in double precision on CUDA (default FP32).
+- GPU results match the CPU reference distributionally at the GPU/FP32 tolerance
+  tier (PMM donor copies are exact in FP64, exact-to-FP32 otherwise). Validated
+  in `tests/mice/test_gpu.py` against the CPU path rather than R directly.
+- Measured PMM speedups vs the CPU backend on an RTX 5070 Ti: ~39× at n=1000
+  (p=8, m=20, maxit=8) and ~135× at n=3000 (p=10, m=20, maxit=8); the gain grows
+  with n because the donor search dominates and batches well on the GPU.
+- Internal: the MICE backend contract now takes a `seed` (each backend spawns
+  its own RNG streams) instead of pre-spawned streams, so the CPU and GPU
+  backends share one `run()` signature. No public API change.
+- MPS (Apple Silicon) is not yet validated for MICE; `backend='gpu'` raises a
+  clear error on MPS rather than running unverified. CUDA is supported.

{pystatistics-3.2.0 → pystatistics-3.4.0}/CHANGELOG.md

@@ -1,5 +1,47 @@
 # Changelog
 
+## 3.4.0
+
+GPU acceleration for MICE.
+
+- **`mice(..., backend='gpu')` runs on CUDA GPUs.** The `m` imputation chains
+  run together on the GPU, with the per-variable linear solves and the
+  predictive-mean-matching donor search batched across chains. `backend='auto'`
+  uses a CUDA GPU when one is available and falls back to the CPU otherwise.
+- **Large speedups on bigger problems.** Because the donor search dominates and
+  parallelises well, the GPU advantage grows with sample size — for example,
+  predictive mean matching was roughly 39× faster at n=1000 and 135× faster at
+  n=3000 than the CPU backend on an RTX 5070 Ti (m=20, maxit=8). GPU results
+  match the CPU backend at the standard GPU/FP32 tolerance.
+- **`use_fp64=True`** runs the GPU path in double precision on CUDA for closer
+  agreement with the CPU reference (default is FP32, the fast path).
+- GPU acceleration requires a CUDA GPU; Apple Silicon (MPS) is not yet supported
+  for MICE and is refused with a clear message rather than run unverified.
+
+## 3.3.0
+
+Multiple imputation by chained equations (MICE) for multivariate missing data.
+
+- **New `mice` module.** `pystatistics.mice.mice(data, m=5, maxit=5,
+  method='pmm', seed=...)` imputes a numeric data matrix that contains missing
+  values (NaN) by iteratively modelling each incomplete column from the others,
+  and returns `m` completed datasets. Two methods are available: `'pmm'`
+  (predictive mean matching — the default, which imputes by copying observed
+  donor values) and `'norm'` (Bayesian linear regression). Defaults follow R's
+  `mice` package (`m=5`, `maxit=5`, 5 PMM donors, left-to-right visit order).
+- **Reproducible by construction.** A `seed` is required; given the same seed,
+  imputations are identical across runs and machines. Each of the `m` chains
+  draws from its own independent random stream.
+- **Rubin's-rules pooling.** `pystatistics.mice.pool(estimates, variances)`
+  combines an analysis fitted on each completed dataset into a single inference,
+  with Barnard–Rubin degrees of freedom, confidence intervals, and the usual
+  diagnostics (within/between/total variance, relative increase in variance,
+  fraction of missing information).
+- **Validated against R.** Imputed-value distributions and pooled regression
+  output are checked against R's `mice` on a shared dataset.
+- This release covers numeric columns on the CPU. Categorical-column methods
+  and GPU acceleration are planned for future releases.
+
 ## 3.2.0
 
 Apple Silicon (MPS) GPU support for the FP32 backends, with honest,

{pystatistics-3.2.0 → pystatistics-3.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pystatistics
-Version: 3.2.0
+Version: 3.4.0
 Summary: GPU-accelerated statistical computing for Python
 Project-URL: Homepage, https://sgcx.org/technology/pystatistics/
 Project-URL: Documentation, https://sgcx.org/docs/pystatistics/
@@ -86,6 +86,7 @@ PyStatistics maintains two parallel computational paths with distinct goals:
 | `multivariate/` | Complete | PCA and maximum likelihood factor analysis with varimax/promax rotation |
 | `timeseries/` | Complete | ACF, PACF, ADF, KPSS, ETS, ARIMA, SARIMA, auto_arima, decompose, STL |
 | `gam/` | Complete | Generalized additive models with penalized regression splines matching R mgcv::gam |
+| `mice/` | Numeric | Multiple imputation by chained equations: PMM and Bayesian linear regression, Rubin's-rules pooling, validated against R mice; CUDA GPU backend |
 
 See [docs/ROADMAP.md](docs/ROADMAP.md) for detailed scope, GPU applicability, and implementation priority for each module.
 
@@ -373,6 +374,27 @@ y = np.sin(x) + np.random.randn(200) * 0.3
 result = gam(y, smooths=[s('x1')], smooth_data={'x1': x})
 print(result.edf, result.gcv)
 print(result.summary())
+
+# --- Multiple imputation (MICE) ---
+from pystatistics.mice import mice, pool
+
+# data is an (n, p) array with np.nan marking missing values.
+# Predictive mean matching (R default) for numeric columns; seed is required
+# so the imputation is fully reproducible.
+imp = mice(data, m=5, maxit=5, method='pmm', seed=0)
+completed = imp.completed_datasets()        # list of 5 completed (n, p) arrays
+
+# Fit your analysis on each completed dataset, then combine with Rubin's rules:
+estimates, variances = [], []
+for d in completed:
+    X = np.column_stack([np.ones(len(d)), d[:, 1]])
+    beta, *_ = np.linalg.lstsq(X, d[:, 0], rcond=None)
+    resid = d[:, 0] - X @ beta
+    cov = (resid @ resid / (len(d) - 2)) * np.linalg.inv(X.T @ X)
+    estimates.append(beta[1]); variances.append(cov[1, 1])
+
+pooled = pool(estimates, variances, dfcom=len(data) - 2)
+print(pooled.estimate, pooled.se, pooled.ci_low, pooled.ci_high, pooled.fmi)
 ```
 
 ## Installation
@@ -391,6 +413,30 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 3.4.0 — GPU acceleration for MICE
+
+- `mice(..., backend='gpu')` runs the imputation chains on a CUDA GPU, batching
+  the per-variable solves and the predictive-mean-matching donor search across
+  chains. `backend='auto'` uses a CUDA GPU when available, else the CPU.
+- The speedup grows with sample size (the donor search dominates and batches
+  well): predictive mean matching ran ~39× faster at n=1000 and ~135× faster at
+  n=3000 than the CPU backend on an RTX 5070 Ti. GPU results match the CPU
+  backend at the GPU/FP32 tolerance; pass `use_fp64=True` for double precision.
+- Requires a CUDA GPU; Apple Silicon (MPS) is not yet supported for MICE.
+
+### 3.3.0 — Multiple imputation (MICE)
+
+- New `mice` module: multiple imputation by chained equations for numeric data
+  with missing values. `mice(data, m=5, method='pmm', seed=...)` returns `m`
+  completed datasets, using predictive mean matching (the R default) or
+  Bayesian linear regression (`method='norm'`). Defaults follow R's `mice`.
+- Imputation is fully reproducible — `seed` is required, and each chain uses an
+  independent random stream.
+- `pool(estimates, variances)` combines per-dataset analyses with Rubin's rules
+  (Barnard–Rubin degrees of freedom, confidence intervals, fraction of missing
+  information).
+- Numeric columns on the CPU in this release; validated against R's `mice`.
+
 ### 3.2.0 — Apple Silicon (MPS) GPU support
 
 - `multinom`, `polr`, `gam`, and `arima` / `arima_batch` (Whittle) now run

{pystatistics-3.2.0 → pystatistics-3.4.0}/README.md

@@ -39,6 +39,7 @@ PyStatistics maintains two parallel computational paths with distinct goals:
 | `multivariate/` | Complete | PCA and maximum likelihood factor analysis with varimax/promax rotation |
 | `timeseries/` | Complete | ACF, PACF, ADF, KPSS, ETS, ARIMA, SARIMA, auto_arima, decompose, STL |
 | `gam/` | Complete | Generalized additive models with penalized regression splines matching R mgcv::gam |
+| `mice/` | Numeric | Multiple imputation by chained equations: PMM and Bayesian linear regression, Rubin's-rules pooling, validated against R mice; CUDA GPU backend |
 
 See [docs/ROADMAP.md](docs/ROADMAP.md) for detailed scope, GPU applicability, and implementation priority for each module.
 
@@ -326,6 +327,27 @@ y = np.sin(x) + np.random.randn(200) * 0.3
 result = gam(y, smooths=[s('x1')], smooth_data={'x1': x})
 print(result.edf, result.gcv)
 print(result.summary())
+
+# --- Multiple imputation (MICE) ---
+from pystatistics.mice import mice, pool
+
+# data is an (n, p) array with np.nan marking missing values.
+# Predictive mean matching (R default) for numeric columns; seed is required
+# so the imputation is fully reproducible.
+imp = mice(data, m=5, maxit=5, method='pmm', seed=0)
+completed = imp.completed_datasets()        # list of 5 completed (n, p) arrays
+
+# Fit your analysis on each completed dataset, then combine with Rubin's rules:
+estimates, variances = [], []
+for d in completed:
+    X = np.column_stack([np.ones(len(d)), d[:, 1]])
+    beta, *_ = np.linalg.lstsq(X, d[:, 0], rcond=None)
+    resid = d[:, 0] - X @ beta
+    cov = (resid @ resid / (len(d) - 2)) * np.linalg.inv(X.T @ X)
+    estimates.append(beta[1]); variances.append(cov[1, 1])
+
+pooled = pool(estimates, variances, dfcom=len(data) - 2)
+print(pooled.estimate, pooled.se, pooled.ci_low, pooled.ci_high, pooled.fmi)
 ```
 
 ## Installation
@@ -344,6 +366,30 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 3.4.0 — GPU acceleration for MICE
+
+- `mice(..., backend='gpu')` runs the imputation chains on a CUDA GPU, batching
+  the per-variable solves and the predictive-mean-matching donor search across
+  chains. `backend='auto'` uses a CUDA GPU when available, else the CPU.
+- The speedup grows with sample size (the donor search dominates and batches
+  well): predictive mean matching ran ~39× faster at n=1000 and ~135× faster at
+  n=3000 than the CPU backend on an RTX 5070 Ti. GPU results match the CPU
+  backend at the GPU/FP32 tolerance; pass `use_fp64=True` for double precision.
+- Requires a CUDA GPU; Apple Silicon (MPS) is not yet supported for MICE.
+
+### 3.3.0 — Multiple imputation (MICE)
+
+- New `mice` module: multiple imputation by chained equations for numeric data
+  with missing values. `mice(data, m=5, method='pmm', seed=...)` returns `m`
+  completed datasets, using predictive mean matching (the R default) or
+  Bayesian linear regression (`method='norm'`). Defaults follow R's `mice`.
+- Imputation is fully reproducible — `seed` is required, and each chain uses an
+  independent random stream.
+- `pool(estimates, variances)` combines per-dataset analyses with Rubin's rules
+  (Barnard–Rubin degrees of freedom, confidence intervals, fraction of missing
+  information).
+- Numeric columns on the CPU in this release; validated against R's `mice`.
+
 ### 3.2.0 — Apple Silicon (MPS) GPU support
 
 - `multinom`, `polr`, `gam`, and `arima` / `arima_batch` (Whittle) now run

{pystatistics-3.2.0 → pystatistics-3.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pystatistics"
-version = "3.2.0"
+version = "3.4.0"
 description = "GPU-accelerated statistical computing for Python"
 readme = "README.md"
 license = "MIT"

{pystatistics-3.2.0 → pystatistics-3.4.0}/pystatistics/__init__.py

@@ -16,7 +16,7 @@ Usage:
     result = fit(design)
 """
 
-__version__ = "3.2.0"
+__version__ = "3.4.0"
 __author__ = "Hai-Shuo"
 __email__ = "contact@sgcx.org"
 
@@ -34,6 +34,7 @@ from pystatistics import ordinal
 from pystatistics import multivariate
 from pystatistics import timeseries
 from pystatistics import gam
+from pystatistics import mice
 
 __all__ = [
     "__version__",
@@ -51,4 +52,5 @@ __all__ = [
     "multivariate",
     "timeseries",
     "gam",
+    "mice",
 ]

pystatistics-3.4.0/pystatistics/mice/__init__.py

@@ -0,0 +1,29 @@
+"""
+MICE — Multiple Imputation by Chained Equations.
+
+Iteratively impute each incomplete column from the others until the chain
+stabilises, producing ``m`` completed datasets, then combine analyses across
+them with Rubin's rules. R-pegged: numeric defaults follow R's ``mice`` package
+(predictive mean matching for numeric columns).
+
+Public API (Stage 1, CPU, numeric columns):
+
+    >>> from pystatistics.mice import mice, pool
+    >>> imp = mice(data, m=5, method='pmm', seed=0)
+    >>> completed = imp.completed_datasets()      # list of m arrays
+
+Stage 1 supports numeric columns only; categorical methods are planned.
+"""
+
+from pystatistics.mice.design import MICEDesign
+from pystatistics.mice.pooling import PooledResult, pool
+from pystatistics.mice.solution import MICESolution
+from pystatistics.mice.solvers import mice
+
+__all__ = [
+    "MICEDesign",
+    "MICESolution",
+    "PooledResult",
+    "mice",
+    "pool",
+]

pystatistics-3.4.0/pystatistics/mice/_chain.py

@@ -0,0 +1,113 @@
+"""
+The chained-equations sweep — one imputation chain.
+
+A single chain produces one completed dataset:
+
+  1. *Initialise* every missing cell by a random draw from the observed values of
+     its own column (R mice's default start), so all predictor columns are fully
+     populated from iteration 1.
+  2. *Sweep* ``maxit`` times. In each iteration, visit the incomplete columns in
+     the visit sequence; for each, fit the column's assigned method on the rows
+     where it is observed (using all other columns as predictors) and overwrite
+     its missing cells with fresh draws.
+  3. Record, per iteration and per incomplete column, the mean and variance of
+     the imputed cells — the trace MICE convergence diagnostics are read from.
+
+This module owns orchestration only. The per-variable statistical work lives
+behind the :class:`ImputationMethod` protocol, so the sweep never hard-codes a
+method and a GPU backend can swap in batched method implementations without
+touching this control flow.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+
+from pystatistics.mice.design import MICEDesign
+from pystatistics.mice.methods import get_method
+
+
+@dataclass(frozen=True)
+class ChainResult:
+    """Output of one chain: completed data plus convergence traces."""
+
+    completed: np.ndarray            # (n, p) — no NaN
+    chain_mean: np.ndarray           # (maxit, n_incomplete)
+    chain_var: np.ndarray            # (maxit, n_incomplete)
+    incomplete_columns: tuple[int, ...]
+
+
+def run_chain(
+    design: MICEDesign,
+    rng: np.random.Generator,
+    maxit: int,
+    visit_sequence: tuple[int, ...],
+) -> ChainResult:
+    """Run one chained-equations chain. See module docstring."""
+    data = design.data.copy()
+    mask = design.missing_mask
+    p = design.p
+    incomplete = design.incomplete_columns
+
+    _initialise(data, mask, incomplete, rng)
+
+    chain_mean = np.empty((maxit, len(incomplete)), dtype=np.float64)
+    chain_var = np.empty((maxit, len(incomplete)), dtype=np.float64)
+    incomplete_pos = {j: k for k, j in enumerate(incomplete)}
+
+    for it in range(maxit):
+        for j in visit_sequence:
+            mis_rows = mask[:, j]
+            obs_rows = ~mis_rows
+
+            predictors = _predictor_columns(data, j, p)
+            X_obs = predictors[obs_rows]
+            X_mis = predictors[mis_rows]
+            y_obs = data[obs_rows, j]
+
+            method = get_method(design.method_for(j))
+            imputed = method.impute(y_obs, X_obs, X_mis, rng)
+            data[mis_rows, j] = imputed
+
+        # Trace: summarise this iteration's imputed cells per incomplete column.
+        for j in incomplete:
+            cells = data[mask[:, j], j]
+            k = incomplete_pos[j]
+            chain_mean[it, k] = float(np.mean(cells))
+            chain_var[it, k] = float(np.var(cells))
+
+    return ChainResult(
+        completed=data,
+        chain_mean=chain_mean,
+        chain_var=chain_var,
+        incomplete_columns=incomplete,
+    )
+
+
+def _initialise(
+    data: np.ndarray,
+    mask: np.ndarray,
+    incomplete: tuple[int, ...],
+    rng: np.random.Generator,
+) -> None:
+    """Fill missing cells by sampling (with replacement) from observed values of
+    the same column. Mutates ``data`` in place."""
+    for j in incomplete:
+        mis_rows = mask[:, j]
+        observed = data[~mis_rows, j]
+        n_missing = int(np.count_nonzero(mis_rows))
+        data[mis_rows, j] = rng.choice(observed, size=n_missing, replace=True)
+
+
+def _predictor_columns(data: np.ndarray, j: int, p: int) -> np.ndarray:
+    """All columns except ``j`` (the current target), as a 2D predictor matrix.
+
+    Columns are fully populated at this point (initialised on entry, kept
+    complete by every visit), so the returned matrix is finite. Fancy-indexing
+    with a list always yields a 2D matrix, including the single-predictor
+    (p == 2) case.
+    """
+    cols = [c for c in range(p) if c != j]
+    return data[:, cols]

pystatistics-3.4.0/pystatistics/mice/_rng.py

@@ -0,0 +1,88 @@
+"""
+Deterministic random-number management for MICE.
+
+Multiple imputation is *intrinsically* stochastic: each completed dataset is a
+draw from the posterior predictive distribution of the missing data. That
+randomness is the method, not a bug. What we owe the user (CLAUDE.md Rule 6) is
+that the randomness be fully controlled by an explicit, injectable seed, and
+isolated behind this module so the rest of the system stays deterministic and
+testable.
+
+# NON-DETERMINISTIC: every function here produces or wraps a pseudo-random
+# stream. The non-determinism is real but is *seeded* — given the same seed,
+# every downstream draw is bit-for-bit reproducible across runs and machines.
+# No other module in `mice` is permitted to call into numpy.random directly;
+# all randomness must flow through the Generators handed out here.
+
+We deliberately do NOT try to reproduce R's Mersenne-Twister stream. MICE is
+validated against R *distributionally* (same defaults, same algorithm, matching
+statistical behaviour across seeds), not by RNG-stream parity — see
+``tests/mice/references/generate_mice_fixtures.R``.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+
+from pystatistics.core.exceptions import ValidationError
+
+
+def make_rng(seed: int) -> np.random.Generator:
+    """Construct a NumPy Generator from an integer seed.
+
+    Parameters
+    ----------
+    seed : int
+        Non-negative integer seed. Required (not optional): reproducibility is
+        a first-class guarantee of this module, so there is no "unseeded" path.
+
+    Returns
+    -------
+    numpy.random.Generator
+    """
+    _validate_seed(seed)
+    return np.random.default_rng(seed)
+
+
+def spawn_streams(seed: int, m: int) -> list[np.random.Generator]:
+    """Spawn ``m`` independent, reproducible Generators from one master seed.
+
+    Each of the ``m`` imputation chains needs its own random stream so the
+    chains are statistically independent, yet the whole set must be reproducible
+    from the single user-facing ``seed``. ``numpy.random.SeedSequence`` provides
+    exactly this: deterministic, high-quality spawning of independent substreams.
+
+    The substreams do not depend on the order in which they are *consumed*, only
+    on ``(seed, m)`` and their index — so a future GPU backend that runs the
+    chains concurrently gets identical draws to the sequential CPU backend.
+
+    Parameters
+    ----------
+    seed : int
+        Master seed.
+    m : int
+        Number of independent streams (one per imputation). Must be >= 1.
+
+    Returns
+    -------
+    list of numpy.random.Generator
+        Length ``m``.
+    """
+    _validate_seed(seed)
+    if not isinstance(m, (int, np.integer)) or isinstance(m, bool):
+        raise ValidationError(f"m must be an integer, got {type(m).__name__}")
+    if m < 1:
+        raise ValidationError(f"m must be >= 1, got {m}")
+
+    child_seeds = np.random.SeedSequence(int(seed)).spawn(int(m))
+    return [np.random.default_rng(cs) for cs in child_seeds]
+
+
+def _validate_seed(seed: int) -> None:
+    """Fail loud on an invalid seed (Rule 1)."""
+    if isinstance(seed, bool) or not isinstance(seed, (int, np.integer)):
+        raise ValidationError(
+            f"seed must be a non-negative integer, got {type(seed).__name__}"
+        )
+    if seed < 0:
+        raise ValidationError(f"seed must be non-negative, got {seed}")

pystatistics-3.4.0/pystatistics/mice/_visit.py

@@ -0,0 +1,58 @@
+"""
+Visit-sequence construction for the chained-equations sweep.
+
+The visit sequence is the order in which incomplete columns are imputed within
+each iteration. R mice's default ("roman") visits the columns that contain
+missing data in left-to-right column order, which is what Stage 1 implements.
+
+Isolated in its own module because R supports several sequence policies
+(monotone, reverse, custom orderings); keeping the policy here means adding one
+later does not touch the sweep loop.
+"""
+
+from __future__ import annotations
+
+from pystatistics.core.exceptions import ValidationError
+
+
+def default_visit_sequence(incomplete_columns: tuple[int, ...]) -> tuple[int, ...]:
+    """R-default ("roman"): incomplete columns in ascending index order."""
+    return tuple(sorted(incomplete_columns))
+
+
+def resolve_visit_sequence(
+    incomplete_columns: tuple[int, ...],
+    override,
+) -> tuple[int, ...]:
+    """Resolve the visit sequence, validating any user override.
+
+    Parameters
+    ----------
+    incomplete_columns : tuple of int
+        Column indices that actually contain missing values.
+    override : sequence of int or None
+        Explicit visit order. If None, uses the R default. If given, every
+        entry must be an incomplete column index; columns may be repeated
+        (legal in MICE — a variable can be visited more than once per
+        iteration) but every incomplete column must appear at least once so no
+        missing values are left un-imputed.
+    """
+    if override is None:
+        return default_visit_sequence(incomplete_columns)
+
+    incomplete_set = set(incomplete_columns)
+    seq = tuple(int(j) for j in override)
+
+    unknown = [j for j in seq if j not in incomplete_set]
+    if unknown:
+        raise ValidationError(
+            f"visit_sequence references columns {unknown} that have no missing "
+            f"values. Incomplete columns are {sorted(incomplete_set)}."
+        )
+    missing_from_seq = incomplete_set - set(seq)
+    if missing_from_seq:
+        raise ValidationError(
+            f"visit_sequence omits incomplete columns {sorted(missing_from_seq)}; "
+            f"their missing values would never be imputed."
+        )
+    return seq

pystatistics-3.4.0/pystatistics/mice/backends/__init__.py

@@ -0,0 +1 @@
+"""Compute backends for MICE (CPU in Stage 1; GPU planned for Stage 2)."""