pystatistics 3.2.0__tar.gz → 3.3.0__tar.gz

pystatistics-3.3.0/.release/UNRELEASED.md

@@ -0,0 +1,30 @@
+# Unreleased Changes
+
+> This file tracks all changes since the last stable release.
+> Updated by whoever makes a change, on whatever machine.
+> Synced via git so all sessions (Mac, Linux, etc.) see the same state.
+>
+> When ready to release, run: `python .release/release.py --status`
+> and follow the manual release flow in the script docstring.
+
+## Changes
+
+- New `mice` module: Multiple Imputation by Chained Equations for multivariate
+  missing data. `pystatistics.mice.mice(data, m=5, maxit=5, method='pmm',
+  seed=...)` iteratively imputes each incomplete numeric column from the others
+  and returns `m` completed datasets (`MICESolution`). Numeric methods `pmm`
+  (predictive mean matching, the R default) and `norm` (Bayesian linear
+  regression) are included, with R-faithful defaults (`m=5`, `maxit=5`, 5 PMM
+  donors, left-to-right visit sequence). Imputation is fully deterministic given
+  `seed` (required), with independent per-chain RNG streams.
+- New `pystatistics.mice.pool`: Rubin's-rules pooling of an analysis across the
+  `m` completed datasets, with Barnard-Rubin degrees of freedom and the standard
+  diagnostics (within/between/total variance, relative increase in variance,
+  lambda, fraction of missing information) and confidence intervals.
+- Validated distributionally against R's `mice` 3.19.0: imputed-value
+  distributions (mean, spread, quantiles) and Rubin's-rules pooled regression
+  output match R on a shared incomplete dataset. Reference fixtures and the
+  pinned generation script live in `tests/mice/references/`.
+- CPU implementation only in this release. The module is structured (method
+  registry, single backend entrypoint, per-column type metadata) so GPU
+  acceleration and categorical methods can be added without restructuring.

{pystatistics-3.2.0 → pystatistics-3.3.0}/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## 3.3.0
+
+Multiple imputation by chained equations (MICE) for multivariate missing data.
+
+- **New `mice` module.** `pystatistics.mice.mice(data, m=5, maxit=5,
+  method='pmm', seed=...)` imputes a numeric data matrix that contains missing
+  values (NaN) by iteratively modelling each incomplete column from the others,
+  and returns `m` completed datasets. Two methods are available: `'pmm'`
+  (predictive mean matching — the default, which imputes by copying observed
+  donor values) and `'norm'` (Bayesian linear regression). Defaults follow R's
+  `mice` package (`m=5`, `maxit=5`, 5 PMM donors, left-to-right visit order).
+- **Reproducible by construction.** A `seed` is required; given the same seed,
+  imputations are identical across runs and machines. Each of the `m` chains
+  draws from its own independent random stream.
+- **Rubin's-rules pooling.** `pystatistics.mice.pool(estimates, variances)`
+  combines an analysis fitted on each completed dataset into a single inference,
+  with Barnard–Rubin degrees of freedom, confidence intervals, and the usual
+  diagnostics (within/between/total variance, relative increase in variance,
+  fraction of missing information).
+- **Validated against R.** Imputed-value distributions and pooled regression
+  output are checked against R's `mice` on a shared dataset.
+- This release covers numeric columns on the CPU. Categorical-column methods
+  and GPU acceleration are planned for future releases.
+
 ## 3.2.0
 
 Apple Silicon (MPS) GPU support for the FP32 backends, with honest,

{pystatistics-3.2.0 → pystatistics-3.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pystatistics
-Version: 3.2.0
+Version: 3.3.0
 Summary: GPU-accelerated statistical computing for Python
 Project-URL: Homepage, https://sgcx.org/technology/pystatistics/
 Project-URL: Documentation, https://sgcx.org/docs/pystatistics/
@@ -86,6 +86,7 @@ PyStatistics maintains two parallel computational paths with distinct goals:
 | `multivariate/` | Complete | PCA and maximum likelihood factor analysis with varimax/promax rotation |
 | `timeseries/` | Complete | ACF, PACF, ADF, KPSS, ETS, ARIMA, SARIMA, auto_arima, decompose, STL |
 | `gam/` | Complete | Generalized additive models with penalized regression splines matching R mgcv::gam |
+| `mice/` | Numeric (CPU) | Multiple imputation by chained equations: PMM and Bayesian linear regression, Rubin's-rules pooling, validated against R mice |
 
 See [docs/ROADMAP.md](docs/ROADMAP.md) for detailed scope, GPU applicability, and implementation priority for each module.
 
@@ -373,6 +374,27 @@ y = np.sin(x) + np.random.randn(200) * 0.3
 result = gam(y, smooths=[s('x1')], smooth_data={'x1': x})
 print(result.edf, result.gcv)
 print(result.summary())
+
+# --- Multiple imputation (MICE) ---
+from pystatistics.mice import mice, pool
+
+# data is an (n, p) array with np.nan marking missing values.
+# Predictive mean matching (R default) for numeric columns; seed is required
+# so the imputation is fully reproducible.
+imp = mice(data, m=5, maxit=5, method='pmm', seed=0)
+completed = imp.completed_datasets()        # list of 5 completed (n, p) arrays
+
+# Fit your analysis on each completed dataset, then combine with Rubin's rules:
+estimates, variances = [], []
+for d in completed:
+    X = np.column_stack([np.ones(len(d)), d[:, 1]])
+    beta, *_ = np.linalg.lstsq(X, d[:, 0], rcond=None)
+    resid = d[:, 0] - X @ beta
+    cov = (resid @ resid / (len(d) - 2)) * np.linalg.inv(X.T @ X)
+    estimates.append(beta[1]); variances.append(cov[1, 1])
+
+pooled = pool(estimates, variances, dfcom=len(data) - 2)
+print(pooled.estimate, pooled.se, pooled.ci_low, pooled.ci_high, pooled.fmi)
 ```
 
 ## Installation
@@ -391,6 +413,19 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 3.3.0 — Multiple imputation (MICE)
+
+- New `mice` module: multiple imputation by chained equations for numeric data
+  with missing values. `mice(data, m=5, method='pmm', seed=...)` returns `m`
+  completed datasets, using predictive mean matching (the R default) or
+  Bayesian linear regression (`method='norm'`). Defaults follow R's `mice`.
+- Imputation is fully reproducible — `seed` is required, and each chain uses an
+  independent random stream.
+- `pool(estimates, variances)` combines per-dataset analyses with Rubin's rules
+  (Barnard–Rubin degrees of freedom, confidence intervals, fraction of missing
+  information).
+- Numeric columns on the CPU in this release; validated against R's `mice`.
+
 ### 3.2.0 — Apple Silicon (MPS) GPU support
 
 - `multinom`, `polr`, `gam`, and `arima` / `arima_batch` (Whittle) now run

{pystatistics-3.2.0 → pystatistics-3.3.0}/README.md

@@ -39,6 +39,7 @@ PyStatistics maintains two parallel computational paths with distinct goals:
 | `multivariate/` | Complete | PCA and maximum likelihood factor analysis with varimax/promax rotation |
 | `timeseries/` | Complete | ACF, PACF, ADF, KPSS, ETS, ARIMA, SARIMA, auto_arima, decompose, STL |
 | `gam/` | Complete | Generalized additive models with penalized regression splines matching R mgcv::gam |
+| `mice/` | Numeric (CPU) | Multiple imputation by chained equations: PMM and Bayesian linear regression, Rubin's-rules pooling, validated against R mice |
 
 See [docs/ROADMAP.md](docs/ROADMAP.md) for detailed scope, GPU applicability, and implementation priority for each module.
 
@@ -326,6 +327,27 @@ y = np.sin(x) + np.random.randn(200) * 0.3
 result = gam(y, smooths=[s('x1')], smooth_data={'x1': x})
 print(result.edf, result.gcv)
 print(result.summary())
+
+# --- Multiple imputation (MICE) ---
+from pystatistics.mice import mice, pool
+
+# data is an (n, p) array with np.nan marking missing values.
+# Predictive mean matching (R default) for numeric columns; seed is required
+# so the imputation is fully reproducible.
+imp = mice(data, m=5, maxit=5, method='pmm', seed=0)
+completed = imp.completed_datasets()        # list of 5 completed (n, p) arrays
+
+# Fit your analysis on each completed dataset, then combine with Rubin's rules:
+estimates, variances = [], []
+for d in completed:
+    X = np.column_stack([np.ones(len(d)), d[:, 1]])
+    beta, *_ = np.linalg.lstsq(X, d[:, 0], rcond=None)
+    resid = d[:, 0] - X @ beta
+    cov = (resid @ resid / (len(d) - 2)) * np.linalg.inv(X.T @ X)
+    estimates.append(beta[1]); variances.append(cov[1, 1])
+
+pooled = pool(estimates, variances, dfcom=len(data) - 2)
+print(pooled.estimate, pooled.se, pooled.ci_low, pooled.ci_high, pooled.fmi)
 ```
 
 ## Installation
@@ -344,6 +366,19 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 3.3.0 — Multiple imputation (MICE)
+
+- New `mice` module: multiple imputation by chained equations for numeric data
+  with missing values. `mice(data, m=5, method='pmm', seed=...)` returns `m`
+  completed datasets, using predictive mean matching (the R default) or
+  Bayesian linear regression (`method='norm'`). Defaults follow R's `mice`.
+- Imputation is fully reproducible — `seed` is required, and each chain uses an
+  independent random stream.
+- `pool(estimates, variances)` combines per-dataset analyses with Rubin's rules
+  (Barnard–Rubin degrees of freedom, confidence intervals, fraction of missing
+  information).
+- Numeric columns on the CPU in this release; validated against R's `mice`.
+
 ### 3.2.0 — Apple Silicon (MPS) GPU support
 
 - `multinom`, `polr`, `gam`, and `arima` / `arima_batch` (Whittle) now run

{pystatistics-3.2.0 → pystatistics-3.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pystatistics"
-version = "3.2.0"
+version = "3.3.0"
 description = "GPU-accelerated statistical computing for Python"
 readme = "README.md"
 license = "MIT"

{pystatistics-3.2.0 → pystatistics-3.3.0}/pystatistics/__init__.py

@@ -16,7 +16,7 @@ Usage:
     result = fit(design)
 """
 
-__version__ = "3.2.0"
+__version__ = "3.3.0"
 __author__ = "Hai-Shuo"
 __email__ = "contact@sgcx.org"
 
@@ -34,6 +34,7 @@ from pystatistics import ordinal
 from pystatistics import multivariate
 from pystatistics import timeseries
 from pystatistics import gam
+from pystatistics import mice
 
 __all__ = [
     "__version__",
@@ -51,4 +52,5 @@ __all__ = [
     "multivariate",
     "timeseries",
     "gam",
+    "mice",
 ]

pystatistics-3.3.0/pystatistics/mice/__init__.py

@@ -0,0 +1,29 @@
+"""
+MICE — Multiple Imputation by Chained Equations.
+
+Iteratively impute each incomplete column from the others until the chain
+stabilises, producing ``m`` completed datasets, then combine analyses across
+them with Rubin's rules. R-pegged: numeric defaults follow R's ``mice`` package
+(predictive mean matching for numeric columns).
+
+Public API (Stage 1, CPU, numeric columns):
+
+    >>> from pystatistics.mice import mice, pool
+    >>> imp = mice(data, m=5, method='pmm', seed=0)
+    >>> completed = imp.completed_datasets()      # list of m arrays
+
+Stage 1 supports numeric columns only; categorical methods are planned.
+"""
+
+from pystatistics.mice.design import MICEDesign
+from pystatistics.mice.pooling import PooledResult, pool
+from pystatistics.mice.solution import MICESolution
+from pystatistics.mice.solvers import mice
+
+__all__ = [
+    "MICEDesign",
+    "MICESolution",
+    "PooledResult",
+    "mice",
+    "pool",
+]

pystatistics-3.3.0/pystatistics/mice/_chain.py

@@ -0,0 +1,113 @@
+"""
+The chained-equations sweep — one imputation chain.
+
+A single chain produces one completed dataset:
+
+  1. *Initialise* every missing cell by a random draw from the observed values of
+     its own column (R mice's default start), so all predictor columns are fully
+     populated from iteration 1.
+  2. *Sweep* ``maxit`` times. In each iteration, visit the incomplete columns in
+     the visit sequence; for each, fit the column's assigned method on the rows
+     where it is observed (using all other columns as predictors) and overwrite
+     its missing cells with fresh draws.
+  3. Record, per iteration and per incomplete column, the mean and variance of
+     the imputed cells — the trace MICE convergence diagnostics are read from.
+
+This module owns orchestration only. The per-variable statistical work lives
+behind the :class:`ImputationMethod` protocol, so the sweep never hard-codes a
+method and a GPU backend can swap in batched method implementations without
+touching this control flow.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+
+from pystatistics.mice.design import MICEDesign
+from pystatistics.mice.methods import get_method
+
+
+@dataclass(frozen=True)
+class ChainResult:
+    """Output of one chain: completed data plus convergence traces."""
+
+    completed: np.ndarray            # (n, p) — no NaN
+    chain_mean: np.ndarray           # (maxit, n_incomplete)
+    chain_var: np.ndarray            # (maxit, n_incomplete)
+    incomplete_columns: tuple[int, ...]
+
+
+def run_chain(
+    design: MICEDesign,
+    rng: np.random.Generator,
+    maxit: int,
+    visit_sequence: tuple[int, ...],
+) -> ChainResult:
+    """Run one chained-equations chain. See module docstring."""
+    data = design.data.copy()
+    mask = design.missing_mask
+    p = design.p
+    incomplete = design.incomplete_columns
+
+    _initialise(data, mask, incomplete, rng)
+
+    chain_mean = np.empty((maxit, len(incomplete)), dtype=np.float64)
+    chain_var = np.empty((maxit, len(incomplete)), dtype=np.float64)
+    incomplete_pos = {j: k for k, j in enumerate(incomplete)}
+
+    for it in range(maxit):
+        for j in visit_sequence:
+            mis_rows = mask[:, j]
+            obs_rows = ~mis_rows
+
+            predictors = _predictor_columns(data, j, p)
+            X_obs = predictors[obs_rows]
+            X_mis = predictors[mis_rows]
+            y_obs = data[obs_rows, j]
+
+            method = get_method(design.method_for(j))
+            imputed = method.impute(y_obs, X_obs, X_mis, rng)
+            data[mis_rows, j] = imputed
+
+        # Trace: summarise this iteration's imputed cells per incomplete column.
+        for j in incomplete:
+            cells = data[mask[:, j], j]
+            k = incomplete_pos[j]
+            chain_mean[it, k] = float(np.mean(cells))
+            chain_var[it, k] = float(np.var(cells))
+
+    return ChainResult(
+        completed=data,
+        chain_mean=chain_mean,
+        chain_var=chain_var,
+        incomplete_columns=incomplete,
+    )
+
+
+def _initialise(
+    data: np.ndarray,
+    mask: np.ndarray,
+    incomplete: tuple[int, ...],
+    rng: np.random.Generator,
+) -> None:
+    """Fill missing cells by sampling (with replacement) from observed values of
+    the same column. Mutates ``data`` in place."""
+    for j in incomplete:
+        mis_rows = mask[:, j]
+        observed = data[~mis_rows, j]
+        n_missing = int(np.count_nonzero(mis_rows))
+        data[mis_rows, j] = rng.choice(observed, size=n_missing, replace=True)
+
+
+def _predictor_columns(data: np.ndarray, j: int, p: int) -> np.ndarray:
+    """All columns except ``j`` (the current target), as a 2D predictor matrix.
+
+    Columns are fully populated at this point (initialised on entry, kept
+    complete by every visit), so the returned matrix is finite. Fancy-indexing
+    with a list always yields a 2D matrix, including the single-predictor
+    (p == 2) case.
+    """
+    cols = [c for c in range(p) if c != j]
+    return data[:, cols]

pystatistics-3.3.0/pystatistics/mice/_rng.py

@@ -0,0 +1,88 @@
+"""
+Deterministic random-number management for MICE.
+
+Multiple imputation is *intrinsically* stochastic: each completed dataset is a
+draw from the posterior predictive distribution of the missing data. That
+randomness is the method, not a bug. What we owe the user (CLAUDE.md Rule 6) is
+that the randomness be fully controlled by an explicit, injectable seed, and
+isolated behind this module so the rest of the system stays deterministic and
+testable.
+
+# NON-DETERMINISTIC: every function here produces or wraps a pseudo-random
+# stream. The non-determinism is real but is *seeded* — given the same seed,
+# every downstream draw is bit-for-bit reproducible across runs and machines.
+# No other module in `mice` is permitted to call into numpy.random directly;
+# all randomness must flow through the Generators handed out here.
+
+We deliberately do NOT try to reproduce R's Mersenne-Twister stream. MICE is
+validated against R *distributionally* (same defaults, same algorithm, matching
+statistical behaviour across seeds), not by RNG-stream parity — see
+``tests/mice/references/generate_mice_fixtures.R``.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+
+from pystatistics.core.exceptions import ValidationError
+
+
+def make_rng(seed: int) -> np.random.Generator:
+    """Construct a NumPy Generator from an integer seed.
+
+    Parameters
+    ----------
+    seed : int
+        Non-negative integer seed. Required (not optional): reproducibility is
+        a first-class guarantee of this module, so there is no "unseeded" path.
+
+    Returns
+    -------
+    numpy.random.Generator
+    """
+    _validate_seed(seed)
+    return np.random.default_rng(seed)
+
+
+def spawn_streams(seed: int, m: int) -> list[np.random.Generator]:
+    """Spawn ``m`` independent, reproducible Generators from one master seed.
+
+    Each of the ``m`` imputation chains needs its own random stream so the
+    chains are statistically independent, yet the whole set must be reproducible
+    from the single user-facing ``seed``. ``numpy.random.SeedSequence`` provides
+    exactly this: deterministic, high-quality spawning of independent substreams.
+
+    The substreams do not depend on the order in which they are *consumed*, only
+    on ``(seed, m)`` and their index — so a future GPU backend that runs the
+    chains concurrently gets identical draws to the sequential CPU backend.
+
+    Parameters
+    ----------
+    seed : int
+        Master seed.
+    m : int
+        Number of independent streams (one per imputation). Must be >= 1.
+
+    Returns
+    -------
+    list of numpy.random.Generator
+        Length ``m``.
+    """
+    _validate_seed(seed)
+    if not isinstance(m, (int, np.integer)) or isinstance(m, bool):
+        raise ValidationError(f"m must be an integer, got {type(m).__name__}")
+    if m < 1:
+        raise ValidationError(f"m must be >= 1, got {m}")
+
+    child_seeds = np.random.SeedSequence(int(seed)).spawn(int(m))
+    return [np.random.default_rng(cs) for cs in child_seeds]
+
+
+def _validate_seed(seed: int) -> None:
+    """Fail loud on an invalid seed (Rule 1)."""
+    if isinstance(seed, bool) or not isinstance(seed, (int, np.integer)):
+        raise ValidationError(
+            f"seed must be a non-negative integer, got {type(seed).__name__}"
+        )
+    if seed < 0:
+        raise ValidationError(f"seed must be non-negative, got {seed}")

pystatistics-3.3.0/pystatistics/mice/_visit.py

@@ -0,0 +1,58 @@
+"""
+Visit-sequence construction for the chained-equations sweep.
+
+The visit sequence is the order in which incomplete columns are imputed within
+each iteration. R mice's default ("roman") visits the columns that contain
+missing data in left-to-right column order, which is what Stage 1 implements.
+
+Isolated in its own module because R supports several sequence policies
+(monotone, reverse, custom orderings); keeping the policy here means adding one
+later does not touch the sweep loop.
+"""
+
+from __future__ import annotations
+
+from pystatistics.core.exceptions import ValidationError
+
+
+def default_visit_sequence(incomplete_columns: tuple[int, ...]) -> tuple[int, ...]:
+    """R-default ("roman"): incomplete columns in ascending index order."""
+    return tuple(sorted(incomplete_columns))
+
+
+def resolve_visit_sequence(
+    incomplete_columns: tuple[int, ...],
+    override,
+) -> tuple[int, ...]:
+    """Resolve the visit sequence, validating any user override.
+
+    Parameters
+    ----------
+    incomplete_columns : tuple of int
+        Column indices that actually contain missing values.
+    override : sequence of int or None
+        Explicit visit order. If None, uses the R default. If given, every
+        entry must be an incomplete column index; columns may be repeated
+        (legal in MICE — a variable can be visited more than once per
+        iteration) but every incomplete column must appear at least once so no
+        missing values are left un-imputed.
+    """
+    if override is None:
+        return default_visit_sequence(incomplete_columns)
+
+    incomplete_set = set(incomplete_columns)
+    seq = tuple(int(j) for j in override)
+
+    unknown = [j for j in seq if j not in incomplete_set]
+    if unknown:
+        raise ValidationError(
+            f"visit_sequence references columns {unknown} that have no missing "
+            f"values. Incomplete columns are {sorted(incomplete_set)}."
+        )
+    missing_from_seq = incomplete_set - set(seq)
+    if missing_from_seq:
+        raise ValidationError(
+            f"visit_sequence omits incomplete columns {sorted(missing_from_seq)}; "
+            f"their missing values would never be imputed."
+        )
+    return seq

pystatistics-3.3.0/pystatistics/mice/backends/__init__.py

@@ -0,0 +1 @@
+"""Compute backends for MICE (CPU in Stage 1; GPU planned for Stage 2)."""

pystatistics-3.3.0/pystatistics/mice/backends/cpu.py

@@ -0,0 +1,92 @@
+"""
+CPU backend for MICE.
+
+Owns the single backend entrypoint ``run`` that the solver calls. It runs the
+``m`` imputation chains — each with its own pre-spawned, independent RNG stream
+— and assembles the completed datasets and convergence traces into a Result.
+
+This is the seam Stage 2 mirrors: a GPU backend implements the same ``run``
+signature and batches the chain loop, while the solver, design, methods, and
+pooling stay untouched.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+
+from pystatistics.core.compute.timing import Timer
+from pystatistics.core.result import Result
+from pystatistics.mice._chain import run_chain
+from pystatistics.mice.design import MICEDesign
+from pystatistics.mice.solution import MICEParams
+
+
+class CPUMiceBackend:
+    """Runs MICE chains on the CPU (reference path)."""
+
+    @property
+    def name(self) -> str:
+        return "cpu"
+
+    def run(
+        self,
+        design: MICEDesign,
+        *,
+        m: int,
+        maxit: int,
+        visit_sequence: tuple[int, ...],
+        streams: list[np.random.Generator],
+    ) -> Result[MICEParams]:
+        """Run ``m`` chains and package the imputations.
+
+        Parameters
+        ----------
+        design : MICEDesign
+        m : int
+            Number of imputations (chains).
+        maxit : int
+            Iterations per chain.
+        visit_sequence : tuple of int
+            Column visit order within each iteration.
+        streams : list of numpy.random.Generator
+            One independent RNG per chain; ``len(streams) == m`` (the solver
+            guarantees this).
+        """
+        timer = Timer()
+        timer.start()
+
+        completed = np.empty((m, design.n, design.p), dtype=np.float64)
+        chain_means = np.empty((m, maxit, len(design.incomplete_columns)))
+        chain_vars = np.empty((m, maxit, len(design.incomplete_columns)))
+
+        with timer.section("chains"):
+            for i in range(m):
+                result = run_chain(design, streams[i], maxit, visit_sequence)
+                completed[i] = result.completed
+                chain_means[i] = result.chain_mean
+                chain_vars[i] = result.chain_var
+
+        timer.stop()
+
+        params = MICEParams(
+            completed=completed,
+            chain_mean=chain_means,
+            chain_var=chain_vars,
+            incomplete_columns=design.incomplete_columns,
+            m=m,
+            maxit=maxit,
+            visit_sequence=visit_sequence,
+        )
+        return Result(
+            params=params,
+            info={
+                "method": "chained_equations",
+                "m": m,
+                "maxit": maxit,
+                "device": "cpu",
+                "visit_sequence": list(visit_sequence),
+            },
+            timing=timer.result(),
+            backend_name="cpu",
+            warnings=(),
+        )