multivariate-probit 0.1.0__tar.gz

multivariate_probit-0.1.0/.gitignore

@@ -0,0 +1,18 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+build/
+dist/
+.eggs/
+.pytest_cache/
+.coverage
+htmlcov/
+.venv/
+venv/
+.env
+.idea/
+.vscode/
+.DS_Store
+
+# Local data, not redistributed
+UCI/

multivariate_probit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mark Shipman
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

multivariate_probit-0.1.0/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.5
+Name: multivariate-probit
+Version: 0.1.0
+Summary: Multivariate probit models fitted by Inference Functions for Margins (IFM), with pluggable inner models.
+Project-URL: Homepage, https://github.com/sign-of-fourier/multivariate-probit
+Project-URL: Documentation, https://github.com/sign-of-fourier/multivariate-probit/blob/main/docs/ifm.md
+Project-URL: Issues, https://github.com/sign-of-fourier/multivariate-probit/issues
+Author: Mark Shipman
+License: MIT License
+        
+        Copyright (c) 2026 Mark Shipman
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: IFM,copula,correlated binary outcomes,multivariate probit,xgboost
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.22
+Requires-Dist: scipy>=1.8
+Provides-Extra: all
+Requires-Dist: scikit-learn>=1.1; extra == 'all'
+Requires-Dist: xgboost>=1.7; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: scikit-learn>=1.1; extra == 'dev'
+Requires-Dist: xgboost>=1.7; extra == 'dev'
+Provides-Extra: sklearn
+Requires-Dist: scikit-learn>=1.1; extra == 'sklearn'
+Provides-Extra: xgboost
+Requires-Dist: scikit-learn>=1.1; extra == 'xgboost'
+Requires-Dist: xgboost>=1.7; extra == 'xgboost'
+Description-Content-Type: text/markdown
+
+# multivariate-probit
+
+Multivariate probit models for correlated binary outcomes, with a pluggable
+inner model.
+
+## The model
+
+Each outcome is a threshold on a latent Gaussian variable, and the outcomes are
+tied together by the correlation of those latents:
+
+```
+Y_j = 1[ η_j(x) + e_j > 0 ],    e ~ N(0, Σ),    j = 1 … d
+```
+
+`η_j` is an arbitrary real-valued function of the features — linear by default,
+or XGBoost, a random forest, or anything else that fits `(X, y)`. Σ is a
+correlation matrix (unit diagonal) carrying the dependence between outcomes.
+Marginally, `P(Y_j = 1 | x) = Φ(η_j(x))`.
+
+Fitting is cross-fit two-stage IFM: every margin is fitted independently, then
+Σ is estimated by maximum likelihood with those margins held fixed. That
+separation is what lets the inner model be a black box. The algorithm, and the
+alternatives that were tested and rejected, are in
+**[docs/ifm.md](docs/ifm.md)**.
+
+## Install
+
+```bash
+pip install multivariate-probit           # core: numpy + scipy
+pip install multivariate-probit[xgboost]  # adds the "xgboost" preset
+pip install multivariate-probit[all]      # every preset
+```
+
+## Quickstart
+
+```python
+from multivariate_probit import MultivariateProbit
+
+model = MultivariateProbit(inner="linear").fit(X, Y)   # Y is (n, d), 0/1
+
+proba = model.predict_proba(X)
+proba.marginal                  # P(Y_j = 1 | x), shape (n, d)
+proba.joint([1, 0, 1])          # P(Y = pattern | x), shape (n,)
+proba.all()                     # P(every outcome = 1 | x)
+proba.any(outcomes=[0, 2])      # P(at least one of these | x)
+
+model.correlation_              # the fitted Σ, shape (d, d)
+model.transform(X)              # latent scores η, shape (n, d)
+model.sample(X, n_samples=100)  # simulated outcome patterns
+model.score(X, Y)               # mean joint log-likelihood
+```
+
+`predict_proba` returns an object that behaves like the marginal-probability
+array (`np.asarray(proba)`, indexing, `.shape`) and additionally answers the
+joint questions Σ was estimated for. Marginal predictions do not involve Σ at
+all; everything joint does.
+
+## Everything here is a squashing function over a latent index
+
+The inner model never sees a probability, and never sees another outcome's
+labels. It produces an unbounded score η_j(x) on (-∞, ∞); Φ is the only
+squashing function applied to it. Any estimator that emits a real-valued score,
+or a probability that can be pushed back through Φ⁻¹, is a legal margin.
+
+That is the whole abstraction, and it is why the inner model is swappable
+without touching the estimation code.
+
+## Inner models
+
+| Preset | Estimator | Requires |
+| --- | --- | --- |
+| `"linear"` (default), `"probit"` | native probit via IRLS | — |
+| `"xgboost"`, `"xgb"` | `XGBClassifier`, tuned for calibration | `xgboost` |
+| `"rf"`, `"random_forest"` | `RandomForestClassifier` | `scikit-learn` |
+
+```python
+MultivariateProbit(inner="xgboost")                        # a preset
+MultivariateProbit(inner=XGBClassifier(max_depth=4))       # any sklearn-shaped model
+MultivariateProbit(inner=["linear", "xgboost", "linear"])  # one per outcome
+```
+
+`available_inners()` lists the presets; `register_inner(name, factory)` adds
+your own. See **[docs/api.md](docs/api.md)** for the inner-model contract.
+
+## Two knobs that cost time
+
+- **`dependence`** — `"joint"` (default) maximises the full d-variate
+  likelihood for Σ. `"pairwise"` maximises each pair's bivariate likelihood
+  instead: consistent, orders of magnitude cheaper, and the right choice once
+  you have more than a handful of outcomes.
+- **`cv`** — `5` by default, cross-fitting the margins so Σ is never estimated
+  from in-sample predictions. This is not optional hygiene: in-sample margins
+  drive every fitted correlation to +1. `cv=None` skips it, which is defensible
+  for the linear default and reckless for anything that can overfit.
+
+## Status
+
+Alpha. The linear and XGBoost paths are covered by tests; the `rf` preset is
+wired but untested. Standard errors are not computed — `correlation_` is a
+point estimate. Known gaps are listed in
+**[docs/limitations.md](docs/limitations.md)**.
+
+## Documentation
+
+- **[docs/ifm.md](docs/ifm.md)** — the estimation algorithm, and why IFM over
+  the alternatives
+- **[docs/implementation.md](docs/implementation.md)** — what is hand-rolled,
+  what comes from SciPy, and why
+- **[docs/api.md](docs/api.md)** — parameters, attributes, methods, extension
+  points
+- **[docs/limitations.md](docs/limitations.md)** — known gaps and roadmap
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE).

multivariate_probit-0.1.0/README.md

@@ -0,0 +1,122 @@
+# multivariate-probit
+
+Multivariate probit models for correlated binary outcomes, with a pluggable
+inner model.
+
+## The model
+
+Each outcome is a threshold on a latent Gaussian variable, and the outcomes are
+tied together by the correlation of those latents:
+
+```
+Y_j = 1[ η_j(x) + e_j > 0 ],    e ~ N(0, Σ),    j = 1 … d
+```
+
+`η_j` is an arbitrary real-valued function of the features — linear by default,
+or XGBoost, a random forest, or anything else that fits `(X, y)`. Σ is a
+correlation matrix (unit diagonal) carrying the dependence between outcomes.
+Marginally, `P(Y_j = 1 | x) = Φ(η_j(x))`.
+
+Fitting is cross-fit two-stage IFM: every margin is fitted independently, then
+Σ is estimated by maximum likelihood with those margins held fixed. That
+separation is what lets the inner model be a black box. The algorithm, and the
+alternatives that were tested and rejected, are in
+**[docs/ifm.md](docs/ifm.md)**.
+
+## Install
+
+```bash
+pip install multivariate-probit           # core: numpy + scipy
+pip install multivariate-probit[xgboost]  # adds the "xgboost" preset
+pip install multivariate-probit[all]      # every preset
+```
+
+## Quickstart
+
+```python
+from multivariate_probit import MultivariateProbit
+
+model = MultivariateProbit(inner="linear").fit(X, Y)   # Y is (n, d), 0/1
+
+proba = model.predict_proba(X)
+proba.marginal                  # P(Y_j = 1 | x), shape (n, d)
+proba.joint([1, 0, 1])          # P(Y = pattern | x), shape (n,)
+proba.all()                     # P(every outcome = 1 | x)
+proba.any(outcomes=[0, 2])      # P(at least one of these | x)
+
+model.correlation_              # the fitted Σ, shape (d, d)
+model.transform(X)              # latent scores η, shape (n, d)
+model.sample(X, n_samples=100)  # simulated outcome patterns
+model.score(X, Y)               # mean joint log-likelihood
+```
+
+`predict_proba` returns an object that behaves like the marginal-probability
+array (`np.asarray(proba)`, indexing, `.shape`) and additionally answers the
+joint questions Σ was estimated for. Marginal predictions do not involve Σ at
+all; everything joint does.
+
+## Everything here is a squashing function over a latent index
+
+The inner model never sees a probability, and never sees another outcome's
+labels. It produces an unbounded score η_j(x) on (-∞, ∞); Φ is the only
+squashing function applied to it. Any estimator that emits a real-valued score,
+or a probability that can be pushed back through Φ⁻¹, is a legal margin.
+
+That is the whole abstraction, and it is why the inner model is swappable
+without touching the estimation code.
+
+## Inner models
+
+| Preset | Estimator | Requires |
+| --- | --- | --- |
+| `"linear"` (default), `"probit"` | native probit via IRLS | — |
+| `"xgboost"`, `"xgb"` | `XGBClassifier`, tuned for calibration | `xgboost` |
+| `"rf"`, `"random_forest"` | `RandomForestClassifier` | `scikit-learn` |
+
+```python
+MultivariateProbit(inner="xgboost")                        # a preset
+MultivariateProbit(inner=XGBClassifier(max_depth=4))       # any sklearn-shaped model
+MultivariateProbit(inner=["linear", "xgboost", "linear"])  # one per outcome
+```
+
+`available_inners()` lists the presets; `register_inner(name, factory)` adds
+your own. See **[docs/api.md](docs/api.md)** for the inner-model contract.
+
+## Two knobs that cost time
+
+- **`dependence`** — `"joint"` (default) maximises the full d-variate
+  likelihood for Σ. `"pairwise"` maximises each pair's bivariate likelihood
+  instead: consistent, orders of magnitude cheaper, and the right choice once
+  you have more than a handful of outcomes.
+- **`cv`** — `5` by default, cross-fitting the margins so Σ is never estimated
+  from in-sample predictions. This is not optional hygiene: in-sample margins
+  drive every fitted correlation to +1. `cv=None` skips it, which is defensible
+  for the linear default and reckless for anything that can overfit.
+
+## Status
+
+Alpha. The linear and XGBoost paths are covered by tests; the `rf` preset is
+wired but untested. Standard errors are not computed — `correlation_` is a
+point estimate. Known gaps are listed in
+**[docs/limitations.md](docs/limitations.md)**.
+
+## Documentation
+
+- **[docs/ifm.md](docs/ifm.md)** — the estimation algorithm, and why IFM over
+  the alternatives
+- **[docs/implementation.md](docs/implementation.md)** — what is hand-rolled,
+  what comes from SciPy, and why
+- **[docs/api.md](docs/api.md)** — parameters, attributes, methods, extension
+  points
+- **[docs/limitations.md](docs/limitations.md)** — known gaps and roadmap
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE).

multivariate_probit-0.1.0/docs/api.md

@@ -0,0 +1,154 @@
+# API reference
+
+For the model and a quickstart see [../README.md](../README.md); for the
+estimation algorithm see [ifm.md](ifm.md).
+
+## MultivariateProbit
+
+```python
+MultivariateProbit(
+    inner="linear",
+    inner_params=None,
+    dependence="joint",
+    cv=5,
+    n_quad=24,
+    optimizer="Nelder-Mead",
+    project_correlation=True,
+    random_state=None,
+)
+```
+
+### Parameters
+
+| Name | Default | Meaning |
+| --- | --- | --- |
+| `inner` | `"linear"` | Preset name, estimator instance, factory, or a list of length d giving one per outcome. Instances are deep-copied, so one instance can seed every margin. |
+| `inner_params` | `None` | Keyword arguments forwarded to the preset factory. Ignored when `inner` is already an instance. |
+| `dependence` | `"joint"` | `"joint"` maximises the full d-variate likelihood for Σ; `"pairwise"` maximises each pair's bivariate likelihood (composite likelihood, far cheaper). |
+| `cv` | `5` | Folds used to cross-fit the latent indices that stage two consumes. `None` skips cross-fitting — see the warning below. |
+| `n_quad` | `24` | Gauss-Legendre order for the orthant evaluator. Lower it if fitting with many outcomes gets slow. |
+| `optimizer` | `"Nelder-Mead"` | Passed to `scipy.optimize.minimize` for `dependence="joint"`. Derivative-free by design. |
+| `project_correlation` | `True` | Project a pairwise estimate onto the nearest positive-definite correlation matrix. No effect when `dependence="joint"`. |
+| `random_state` | `None` | Controls the cross-fitting split and `sample`. |
+
+> **`cv=None` is not a neutral speed-up.** In-sample margins drive every fitted
+> correlation toward +1. It is defensible for the linear default and reckless
+> for anything that can overfit. See
+> [ifm.md](ifm.md#why-cross-fitting-is-required).
+
+### Attributes
+
+| Name | Shape | Meaning |
+| --- | --- | --- |
+| `inner_models_` | list, length d | The fitted margins, one per outcome. |
+| `correlation_` | (d, d) | The fitted Σ. |
+| `eta_` | (n, d) | The (cross-fitted) latent indices stage two was fitted on. |
+| `nll_` | float | Negative log-likelihood at the end of the dependence fit. Joint and pairwise fits optimise different objectives, so the values are not comparable across settings. |
+| `optimize_result_` | OptimizeResult or None | The SciPy result for `dependence="joint"`. |
+| `n_outcomes_`, `n_features_in_` | int | |
+
+### Methods
+
+| Method | Returns | Notes |
+| --- | --- | --- |
+| `fit(X, Y, sample_weight=None)` | self | `Y` is (n, d) and strictly 0/1. Weights are forwarded to margins that accept them. |
+| `decision_function(X)` / `transform(X)` | (n, d) | Latent indices η on (-∞, ∞). |
+| `fit_transform(X, Y)` | (n, d) | |
+| `predict_proba(X)` | `MultivariateProbitProba` | See below. |
+| `predict_marginal_proba(X)` | (n, d) | Marginal probabilities as a plain array. |
+| `predict(X, threshold=0.5)` | (n, d) | Per-outcome 0/1 at a marginal threshold. For a joint decision use `predict_proba(X).all()`. |
+| `joint_proba(X, Y)` | (n,) | `P(Y = y | x)`. `Y` may be one pattern of length d, broadcast over rows, or an (n, d) array. |
+| `joint_log_proba(X, Y)` | (n,) | |
+| `score(X, Y, sample_weight=None)` | float | Mean joint log-likelihood. Higher is better. |
+| `sample(X, n_samples=1, random_state=None)` | (n, d) or (n_samples, n, d) | Draws patterns from the fitted model. |
+| `get_params` / `set_params` | | scikit-learn-style, without requiring scikit-learn. |
+
+## MultivariateProbitProba
+
+Returned by `predict_proba`. Behaves like the (n, d) array of marginal
+probabilities — `np.asarray(proba)`, `proba[i, j]`, `proba.shape`, `len(proba)`
+— and additionally answers the joint questions Σ was estimated for.
+
+| Member | Returns | Meaning |
+| --- | --- | --- |
+| `.marginal` | (n, d) | `P(Y_j = 1 | x)`. Does not involve Σ. |
+| `.eta` | (n, d) | The latent indices behind it. |
+| `.corr` | (d, d) | The Σ used for joint queries. |
+| `.joint(pattern)` | (n,) | `P(Y = pattern | x)` for a fully specified 0/1 pattern. |
+| `.all(outcomes=None)` | (n,) | `P(every selected outcome = 1 | x)`. |
+| `.any(outcomes=None)` | (n,) | `P(at least one selected outcome = 1 | x)`. |
+| `.none(outcomes=None)` | (n,) | `P(no selected outcome = 1 | x)`. |
+
+`outcomes` takes a list of column indices, so "do these three co-occur" is a
+one-liner.
+
+## Inner models
+
+### The contract
+
+An inner model is anything with `fit(X, y)` and `latent(X) -> (n,)`, where
+`latent` returns a real-valued index on the probit scale.
+
+Estimators that do not expose `latent` are wrapped automatically by
+`ProbitCalibrated`, which supplies it:
+
+- if the estimator has `predict_proba`, `latent(X)` is `Φ⁻¹(p̂)`, clipped away
+  from 0 and 1;
+- otherwise, if it has `decision_function`, that score is used as the index
+  as-is — which assumes it is already probit-scaled. Check that assumption
+  before relying on it.
+
+### Presets
+
+| Name | Aliases | Estimator | Requires |
+| --- | --- | --- | --- |
+| `linear` | `probit` | `ProbitRegressor` (native IRLS) | — |
+| `xgboost` | `xgb` | `XGBClassifier`, defaults tuned for calibration rather than ranking | `xgboost` |
+| `rf` | `random_forest` | `RandomForestClassifier` | `scikit-learn` |
+
+The registry is deliberately thin. Under IFM there is no per-family estimation
+logic — the presets differ only in which pre-wired instance stage one fits — so
+a preset is a factory function and nothing more.
+
+### Registry functions
+
+| Function | Purpose |
+| --- | --- |
+| `available_inners()` | Sorted preset names. |
+| `make_inner(name, **kwargs)` | Instantiate a preset, forwarding kwargs to the estimator. |
+| `register_inner(name, factory, overwrite=False)` | Add a preset. `factory(**kwargs)` returns a fresh estimator. |
+| `as_inner(spec, **kwargs)` | Coerce a name, factory, class, or instance into an unfitted inner model. |
+
+### Mixing models per outcome
+
+```python
+MultivariateProbit(inner=["linear", "xgboost", "linear"])
+MultivariateProbit(inner=["linear", make_inner("xgboost", max_depth=2)])
+```
+
+## ProbitRegressor
+
+The default margin, usable on its own as a binary classifier.
+
+```python
+ProbitRegressor(alpha=1e-6, fit_intercept=True, max_iter=100, tol=1e-8)
+```
+
+`alpha` is an L2 penalty on the slopes only, never the intercept. Exposes
+`coef_`, `intercept_`, `n_iter_`, `classes_`, and `fit` / `latent` /
+`decision_function` / `predict_proba` / `predict`.
+
+## Lower-level functions
+
+Exported for callers who want the pieces without the estimator.
+
+| Function | Purpose |
+| --- | --- |
+| `bvn_cdf(a, b, rho)` | Φ_2, vectorised, correlations may vary by row. |
+| `mvn_orthant(A, corr_stack)` | Φ_d with a per-row correlation stack. |
+| `orthant_prob(upper, corr)` | Φ_d with one shared correlation matrix. |
+| `pattern_prob(eta, Y, corr)` | `P(Y = y | x)` via the sign trick. |
+| `pairwise_correlation(eta, Y, ...)` | Stage two, composite likelihood. |
+| `joint_correlation(eta, Y, ...)` | Stage two, full likelihood. Returns `(corr, result)`. |
+| `pair_log_likelihood(rho, ...)` | Bivariate probit log-likelihood in ρ. |
+| `joint_log_likelihood(corr, eta, Y, ...)` | d-variate log-likelihood in Σ. |