pystatistics 1.0.2__tar.gz → 1.2.0__tar.gz

pystatistics-1.2.0/CHANGELOG.md

@@ -0,0 +1,92 @@
+# Changelog
+
+## 1.2.0
+
+### CLAUDE.md Compliance
+
+Full codebase audit and refactor to comply with all 7 rules in CLAUDE.md.
+
+### Breaking Changes (by design)
+
+- **Silent model switches are now errors.** Functions that previously fell back
+  silently to ridge regularization, LSTSQ, or CPU backends now raise
+  `NumericalError` or `RuntimeError` with descriptive messages suggesting
+  alternatives. This affects:
+  - `mvnmle.mcar_test.regularized_inverse()` — raises on ill-conditioned matrices
+  - `mvnmle.mlest()` — raises if EM encounters non-PD covariance
+  - `mvnmle` parameter extraction — raises instead of returning identity covariance
+  - `regression.fit(backend='gpu')` — raises on Cholesky failure (use `force=True`
+    to proceed with LSTSQ, or `backend='cpu'` for QR)
+  - `mixed.lmm()` / `mixed.glmm()` — raises on singular random effects covariance
+- **`backend='gpu'` now errors when GPU is unavailable.** Previously fell back
+  to CPU silently. `backend='auto'` still falls back silently (it means
+  "best available").
+- **GPU bootstrap/permutation raises `NotImplementedError`.** These were silently
+  running on CPU while reporting a GPU backend name. Now they honestly report
+  that GPU acceleration is not yet implemented for these operations.
+
+### New Features
+
+- **Reproducible Monte Carlo hypothesis tests**: `chisq_test()` and
+  `fisher_test()` now accept `seed` parameter for deterministic results
+  when `simulate_p_value=True`
+
+### Module Structure (Rule 3/4)
+
+- Split `regression/solution.py` into `_linear.py`, `_glm.py`, and
+  `_formatting.py` (backward-compatible re-export shim maintained)
+- Split `hypothesis/design.py` factory methods into `_design_factories.py`
+  (classmethods still work via thin wrappers)
+- All files now under 500 code lines (Rule 4 hard limit)
+
+### Code Quality
+
+- Added `# NUMERICAL GUARD:` comments to ~30 numerical stability operations
+  (clipping, clamping, floors) documenting why each exists
+- Removed dead code (`signaltonoise` try/except in Wilcoxon test)
+- Added per-module compliance tests:
+  - `tests/test_code_quality.py` — LOC limit enforcement
+  - `tests/regression/test_module_split.py` — split integrity + named coefficients
+  - `tests/hypothesis/test_design_split.py` — factory split + seed reproducibility
+  - `tests/mvnmle/test_no_silent_fallback.py` — hard stop verification
+
+## 1.1.0
+
+### New Features
+
+- **Named coefficients**: `fit()`, `coxph()`, and `discrete_time()` accept a
+  `names=` parameter for labeled output matching R's style
+- **`result.coef` dict property**: Access coefficients by variable name
+  (`result.coef["albumin"]`) on `LinearSolution`, `GLMSolution`,
+  `CoxSolution`, and `DiscreteTimeSolution`
+- **`result.hr` dict property**: Access hazard ratios by name on `CoxSolution`
+  and `DiscreteTimeSolution`
+- **Intercept auto-detection**: When `names` has one fewer element than columns
+  in X, `"(Intercept)"` is prepended automatically
+
+### Summary Output Improvements
+
+- **OLS**: Added residual quantiles (Min, 1Q, Median, 3Q, Max) and overall
+  F-statistic with p-value, matching R's `summary(lm())`
+- **Cox PH**: Added hazard ratio confidence intervals table (`exp(coef)`,
+  `exp(-coef)`, `lower .95`, `upper .95`), matching R's `summary(coxph())`
+
+### Bug Fixes
+
+- Fixed Kaplan-Meier `summary()` printing literal `{ci_pct}` instead of the
+  actual confidence level percentage (e.g., `95`)
+
+### Real-Data Validation
+
+- Added PBC clinical trial integration test suite (`tests/test_pbc_analysis.py`)
+  with 22 end-to-end analyses on the Mayo Clinic PBC dataset
+- Added R cross-validation test (`tests/test_pbc_vs_r.py`) confirming all 15
+  comparable analyses match R to `rtol=1e-10`
+
+## 1.0.2
+
+- Initial stable release
+- All 8 modules complete: regression, descriptive, hypothesis, montecarlo,
+  survival, anova, mixed, mvnmle
+- CPU backends validated against R to rtol=1e-10
+- GPU backends validated against CPU per documented tolerance tiers

pystatistics-1.2.0/CLAUDE.md

@@ -0,0 +1,190 @@
+# Coding Bible
+
+## Preamble
+
+This document is not a style guide. It is not a set of preferences. It is not a list of
+suggestions that can be weighed against convenience or time pressure.
+
+Every rule in this document exists because its absence has caused real, costly, documented
+failures in real systems — bugs that took days to trace, security holes that went undetected
+for months, codebases that became unmaintainable within a year of being written. The rules
+are the scar tissue. They are what you get when you ask experienced engineers what they wish
+someone had forced them to do the first time.
+
+When you are implementing something and a rule feels inconvenient, that feeling is a signal,
+not a reason to deviate. Inconvenience usually means the rule is doing its job — preventing
+the path of least resistance from becoming a future liability.
+
+**These rules are non-negotiable and not subject to contextual override.** The following
+exceptions are not acceptable justifications for deviation:
+
+- "This is a prototype / MVP / quick fix." Prototypes become production. Quick fixes become
+  permanent. Code written without discipline under time pressure is the most dangerous code
+  that exists.
+- "This is a small module that doesn't need it." Small modules grow. The time to enforce
+  structure is before the module is large enough to make restructuring painful.
+- "The architecture requires it." If the architecture requires violating these rules, the
+  architecture is wrong. Fix the architecture.
+- "It would take too long to do it correctly." It will take longer to fix it later. It always
+  does.
+
+If a genuine, documented edge case exists where a rule cannot be followed, that exception
+must be explicitly marked in a comment at the relevant location, stating which rule is being
+waived, why, and what mitigating measures are in place. Undocumented deviations are bugs.
+
+---
+
+## The Underlying Principle
+
+All seven rules are expressions of one idea:
+
+**Make the wrong thing hard to do accidentally.**
+
+Good architecture is not about making correct behavior possible. Correct behavior is always
+possible. Good architecture is about making incorrect behavior require deliberate, visible,
+documented effort — so that mistakes are loud, localized, and recoverable rather than silent,
+diffuse, and permanent.
+
+When in doubt, ask: *am I making the wrong thing easy to do accidentally?* If yes, stop and
+apply the relevant rule before continuing.
+
+---
+
+These rules are non-negotiable. They exist because every one of them corresponds to a
+real failure mode with real consequences. If you think a rule doesn't apply to a specific
+case, you are probably wrong. If you are certain it doesn't apply, document why explicitly
+in a comment before proceeding.
+
+---
+
+## 1. Fail Fast, Fail Loud, No Defaults
+
+Do not insert default behavior that fails silently. If something breaks, the system must
+break immediately, obviously, and traceably.
+
+**Rules:**
+- Raise explicit, descriptive errors. Never swallow exceptions.
+- Never return a default value to mask a missing or invalid state.
+- Log the failure with enough context to reproduce it.
+
+**Corollary — No Optimistic Assumptions:**
+Do not assume inputs are valid, dependencies are available, or external calls succeed.
+Assert and verify. If an assumption is required, document it and enforce it with a guard.
+
+---
+
+## 2. Trust Your Neighbors — Validate Input, Not Output
+
+You own your output contract. You do not trust anyone else's input contract.
+
+**Rules:**
+- All external inputs (API calls, user input, file reads, environment variables) must be
+  validated at the boundary before use.
+- Internal module output does not need re-validation by the caller — the module is
+  responsible for the correctness of what it emits.
+
+**Corollary — Define Your Contracts:**
+Every module must have an explicit, documented input/output contract. If a caller violates
+the input contract, the module must fail loudly (see Rule 1), not silently compensate.
+
+---
+
+## 3. UNIX Philosophy — One Module, One Job
+
+Each module does one thing only, and does that one thing well. If a module needs to do
+multiple things, it must be split into multiple files.
+
+**Rules:**
+- No file should be the sole owner of more than one domain concept.
+- If you find yourself writing "and" when describing what a file does, split it.
+- Tight coupling between modules is a design failure, not an implementation detail.
+
+**Corollary — No God Files:**
+A module that knows everything, owns everything, or touches everything is a liability.
+If removing a module would require changes across more than two other modules, it is
+probably doing too much.
+
+---
+
+## 4. LOC Limits — No Bloated Monoliths
+
+Files must not become monoliths. Complexity must be managed through decomposition,
+not consolidation.
+
+**Rules:**
+- **Hard limit: 500 lines of code** (comments and docstrings excluded). Do not exceed this
+  under any circumstances.
+- **Soft limit: 400 lines of code.** Above 400 lines, actively look for split opportunities.
+- If a file "requires" more than 500 lines, that is a signal the abstraction is wrong —
+  split it into multiple focused files.
+
+**Corollary — Splitting Is Not Optional:**
+Exceeding the hard limit is never acceptable, even temporarily. Split first, implement after.
+
+---
+
+## 5. No Hidden State — Explicit Data Flow
+
+All state must be explicitly passed or managed through clearly defined interfaces.
+Hidden state is a bug waiting to be discovered at the worst possible time.
+
+**Rules:**
+- No global variables. No module-level mutable state unless explicitly documented and
+  architecturally justified.
+- No implicit state — if a function's behavior depends on something, that something must
+  appear in its signature or be explicitly injected.
+- No hidden coupling between modules. If two modules share state, that relationship must
+  be visible and intentional.
+
+**Corollary — Spooky Action at a Distance Is a Bug:**
+If changing module A causes unexpected behavior in module B, and there is no explicit
+interface between them, the architecture is broken. Fix the architecture, not the symptom.
+
+---
+
+## 6. Deterministic Behavior by Default
+
+All functions must be deterministic unless explicitly documented otherwise.
+
+**Rules:**
+- No randomness, time-dependent logic, or non-deterministic behavior without explicit
+  documentation and an injectable seed or clock interface for testing.
+- Functions that depend on system time, random state, or external non-deterministic
+  sources must be flagged with a `# NON-DETERMINISTIC:` comment explaining why.
+- Non-deterministic behavior must be isolatable — wrap it so the rest of the system
+  can be tested deterministically.
+
+**Corollary — Reproducibility Is Not Optional:**
+If a bug cannot be reliably reproduced, it cannot be reliably fixed. Non-determinism
+is the enemy of reproducibility. Treat it like a dependency that must be explicitly
+managed, not a feature.
+
+---
+
+## 7. Tests Are First-Class Citizens
+
+Every module must have corresponding tests. Tests are not optional, not afterthoughts,
+and not someone else's job.
+
+**Rules:**
+- Every module must have a corresponding test file covering:
+  - **Normal cases** — expected inputs produce expected outputs.
+  - **Edge cases** — boundary conditions, empty inputs, maximum values, type boundaries.
+  - **Failure cases** — invalid inputs produce the correct explicit errors (see Rule 1).
+- No module ships without tests. No exceptions.
+- Tests must be runnable in isolation — no test should depend on the state left by
+  another test.
+
+**Corollary — Untested Code Is Unverified Assumptions:**
+If there is no test for a behavior, that behavior is assumed, not verified. Assumptions
+accumulate into system failures. If you cannot write a test for something, that is a signal
+the design is wrong.
+
+---
+
+## Meta-Rule
+
+If any of these rules feel inconvenient, that feeling is the point. These rules exist
+precisely because the convenient path leads to the failure modes each rule was written
+to prevent. When in doubt, ask: *am I making the wrong thing easy to do accidentally?*
+If yes, apply the relevant rule.

{pystatistics-1.0.2 → pystatistics-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pystatistics
-Version: 1.0.2
+Version: 1.2.0
 Summary: GPU-accelerated statistical computing for Python
 Project-URL: Homepage, https://sgcx.org/technology/pystatistics/
 Project-URL: Documentation, https://sgcx.org/docs/pystatistics/
@@ -48,6 +48,23 @@ Description-Content-Type: text/markdown
 
 GPU-accelerated statistical computing for Python.
 
+## What's New in 1.2
+
+- **No silent model switches.** PyStatistics now raises explicit errors instead of silently falling back to ridge regularization, different solvers, or CPU backends. You always know exactly what model you're running.
+- **`backend='gpu'` is honest.** Explicitly requesting GPU now errors if GPU is unavailable or the operation has no GPU implementation. `backend='auto'` still falls back silently.
+- **Reproducible Monte Carlo.** `chisq_test()` and `fisher_test()` accept `seed=` for deterministic Monte Carlo p-values.
+- **Module structure.** Large files split per CLAUDE.md Rule 4 (500 code-line limit). All backward-compatible.
+
+### What's New in 1.1
+
+- **Named coefficients**: Pass `names=` to `fit()`, `coxph()`, and `discrete_time()` to get labeled output matching R
+- **`result.coef` dict**: Access coefficients by name — `result.coef["albumin"]` instead of `result.coefficients[1]`
+- **`result.hr` dict** (Cox/discrete-time): Access hazard ratios by name
+- **Intercept auto-detection**: Pass p-1 names and `"(Intercept)"` is prepended automatically
+- **OLS summary improvements**: Now prints residual quantiles and F-statistic, matching R's `summary(lm())`
+- **Cox summary improvements**: Now prints hazard ratio confidence intervals, matching R's `summary(coxph())`
+- **Bug fix**: Kaplan-Meier summary now correctly displays confidence level percentage
+
 ---
 
 ## Design Philosophy
@@ -160,8 +177,10 @@ from pystatistics.regression import fit
 
 X = np.random.randn(1000, 5)
 y = X @ [1, 2, 3, -1, 0.5] + np.random.randn(1000) * 0.1
-result = fit(X, y)
-print(result.summary())
+result = fit(X, y, names=['x1', 'x2', 'x3', 'x4', 'x5'])
+print(result.summary())          # R-style output with variable names
+print(result.coef)                # {'x1': 1.00, 'x2': 2.00, ...}
+print(result.coef['x3'])          # 3.00
 
 # Logistic regression
 y_binary = (X @ [1, -1, 0.5, 0, 0] + np.random.randn(1000) > 0).astype(float)

{pystatistics-1.0.2 → pystatistics-1.2.0}/README.md

@@ -2,6 +2,23 @@
 
 GPU-accelerated statistical computing for Python.
 
+## What's New in 1.2
+
+- **No silent model switches.** PyStatistics now raises explicit errors instead of silently falling back to ridge regularization, different solvers, or CPU backends. You always know exactly what model you're running.
+- **`backend='gpu'` is honest.** Explicitly requesting GPU now errors if GPU is unavailable or the operation has no GPU implementation. `backend='auto'` still falls back silently.
+- **Reproducible Monte Carlo.** `chisq_test()` and `fisher_test()` accept `seed=` for deterministic Monte Carlo p-values.
+- **Module structure.** Large files split per CLAUDE.md Rule 4 (500 code-line limit). All backward-compatible.
+
+### What's New in 1.1
+
+- **Named coefficients**: Pass `names=` to `fit()`, `coxph()`, and `discrete_time()` to get labeled output matching R
+- **`result.coef` dict**: Access coefficients by name — `result.coef["albumin"]` instead of `result.coefficients[1]`
+- **`result.hr` dict** (Cox/discrete-time): Access hazard ratios by name
+- **Intercept auto-detection**: Pass p-1 names and `"(Intercept)"` is prepended automatically
+- **OLS summary improvements**: Now prints residual quantiles and F-statistic, matching R's `summary(lm())`
+- **Cox summary improvements**: Now prints hazard ratio confidence intervals, matching R's `summary(coxph())`
+- **Bug fix**: Kaplan-Meier summary now correctly displays confidence level percentage
+
 ---
 
 ## Design Philosophy
@@ -114,8 +131,10 @@ from pystatistics.regression import fit
 
 X = np.random.randn(1000, 5)
 y = X @ [1, 2, 3, -1, 0.5] + np.random.randn(1000) * 0.1
-result = fit(X, y)
-print(result.summary())
+result = fit(X, y, names=['x1', 'x2', 'x3', 'x4', 'x5'])
+print(result.summary())          # R-style output with variable names
+print(result.coef)                # {'x1': 1.00, 'x2': 2.00, ...}
+print(result.coef['x3'])          # 3.00
 
 # Logistic regression
 y_binary = (X @ [1, -1, 0.5, 0, 0] + np.random.randn(1000) > 0).astype(float)

{pystatistics-1.0.2 → pystatistics-1.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pystatistics"
-version = "1.0.2"
+version = "1.2.0"
 description = "GPU-accelerated statistical computing for Python"
 readme = "README.md"
 license = "MIT"

{pystatistics-1.0.2 → pystatistics-1.2.0}/pystatistics/__init__.py

@@ -16,7 +16,7 @@ Usage:
     result = fit(design)
 """
 
-__version__ = "0.1.0"
+__version__ = "1.2.0"
 __author__ = "Hai-Shuo"
 __email__ = "contact@sgcx.org"