stability-matched 1.0.0__py3-none-any.whl

stability_matched-1.0.0.dist-info/METADATA

@@ -0,0 +1,87 @@
+Metadata-Version: 2.4
+Name: stability-matched
+Version: 1.0.0
+Summary: Feature-selection stability (Nogueira's estimator) with its exact chance level for candidate pools chosen inside each resample
+Author: Hassan Nasiraee, Hajar Abbaszadeh, Parvin Farzanegi
+License: MIT
+Keywords: feature selection,stability,reproducibility,chance correction,bioinformatics
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.22
+Requires-Dist: scipy>=1.8
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Provides-Extra: examples
+Requires-Dist: scikit-learn>=1.2; extra == "examples"
+Dynamic: license-file
+
+# stability-matched
+
+Feature-selection stability with its **exact chance level when the candidate
+pool is chosen inside each resample** (for example a variance or univariate
+filter fitted on the training rows).
+
+Nogueira's estimator (Nogueira, Sechidis & Brown, JMLR 2018) corrects for
+chance as if every feature could be selected in every resample. When each
+resample first filters its own pool, features that pass every filter are
+favoured even by a selector that picks at random inside its pool, and part of
+the reported stability belongs to the filter. This package gives, in closed
+form and without simulation, the expected value of the estimator under random
+selection of the observed sizes from the observed pools, and the excess over
+it, together with a Monte Carlo test, a variant that keeps co-expression
+modules, and an exact chance level for the agreement between two selectors.
+
+## Install
+
+```bash
+pip install stability-matched
+```
+
+Only NumPy and SciPy are required.
+
+## Use
+
+```python
+from stability_matched import matrices_from_indices, phi_matched, report
+
+# supports[i]: indices selected in resample i; pools[i]: its candidate pool
+Z, P = matrices_from_indices(supports, pools, d)
+print(report(Z, P))                 # phi, interval, chance level, excess, p
+res = phi_matched(Z, P, B=999)      # the same as a dictionary
+```
+
+From the command line, with `Z` and `P` saved as `.npy`, `.csv`, or one
+`.npz` holding arrays `Z` and `P`:
+
+```bash
+stability-matched selections.npz
+```
+
+`examples/example_sklearn.py` runs the whole workflow for a cross-validated
+lasso with a variance filter fitted inside each resample.
+
+## Functions
+
+| Function | Returns |
+|---|---|
+| `phi_stability(Z)` | Nogueira's estimator, asymptotic variance, 95% interval |
+| `phi0_matched(Z, P)` | exact expected estimator under random selection within each resample's pool |
+| `phi0_stratified(Z, P, groups)` | the same, keeping how many features each resample takes from every group (e.g. co-expression module) |
+| `matched_null_test(Z, P, B)` | Monte Carlo p-value against B complete null matrices |
+| `agreement_exact(A, B, P)` | mean Jaccard between two selectors in the same resample and its exact hypergeometric chance level |
+| `phi_matched(Z, P, B)` | all of the above for one matrix |
+| `matrices_from_indices`, `report`, `main` | helpers and command line |
+
+Empty supports are kept (they lower stability); a matrix with no defined
+stability returns NaN. Tests: `pytest tests/`.
+
+## Cite
+
+H. Nasiraee, H. Abbaszadeh, P. Farzanegi. How much of feature-selection
+stability is chance? An exact correction, a re-evaluation of two published
+tools, and a multi-tissue audit. Code: DOI 10.5281/zenodo.22662385. MIT
+licence.

stability_matched-1.0.0.dist-info/RECORD

@@ -0,0 +1,7 @@
+stability_matched.py,sha256=Ql4nrLtr5EYCBPBnUyIuMv8mQD5lomUOiVNKOKKd77k,11189
+stability_matched-1.0.0.dist-info/licenses/LICENSE,sha256=yG5ruXPBTy8xMFSPdBVtLCYVmHsudlvdxtKY6Q0h12Y,1111
+stability_matched-1.0.0.dist-info/METADATA,sha256=uYSreZO7ECagU9zBgpbY_XBRiSDNI8vYHiYjzjhb1qQ,3535
+stability_matched-1.0.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
+stability_matched-1.0.0.dist-info/entry_points.txt,sha256=SxdBUn0lvxICJQvkPSTn1Qq868yLqqzTmsv07VAtdm8,61
+stability_matched-1.0.0.dist-info/top_level.txt,sha256=yqdWzJMMAx5A34IW4rK6XbD7gngz1inJP-2PYu8bN0w,18
+stability_matched-1.0.0.dist-info/RECORD,,

stability_matched-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.2.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

stability_matched-1.0.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+stability-matched = stability_matched:main

stability_matched-1.0.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Hassan Nasiraee, Hajar Abbaszadeh and Parvin Farzanegi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

stability_matched-1.0.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+stability_matched

stability_matched.py

@@ -0,0 +1,277 @@
+#!/usr/bin/env python3
+# =============================================================================
+#  stability_matched.py  --  stand-alone stability toolkit
+#
+#  For a binary selection matrix Z (M resamples x d features) and the
+#  candidate-pool matrix P (same shape; P[i, f] = 1 if feature f could be
+#  selected in resample i):
+#
+#    phi_stability(Z)          Nogueira, Sechidis & Brown (2018) estimator,
+#                              its asymptotic variance and 95% interval
+#    phi0_matched(Z, P)        exact E[phi] under the matched null: row i a
+#                              uniform random k_i-subset of pool P_i, rows
+#                              independent (Proposition 1 of the paper);
+#                              conditional on the observed k_i and P_i
+#    phi0_stratified(Z, P, g)  exact E[phi] when row i keeps how many genes
+#                              it selects from every group g (e.g. a
+#                              co-expression module) but chooses WHICH genes
+#                              of that group (within its pool) at random;
+#                              Proposition 1 with pi_if = c_ig / |g n P_i|
+#    matched_null_test(Z, P)   Monte Carlo p-value against B complete null
+#                              matrices, with null quantiles
+#    agreement_exact(A, B, P)  mean Jaccard between two selectors' supports
+#                              in the same resample, and its exact
+#                              hypergeometric chance level
+#    phi_matched(Z, P)         all of the above in one call
+#    matrices_from_indices(supports, pools, d)
+#                              build Z and P from per-resample index lists
+#    report(Z, P)              a plain-text report of the above
+#
+#  Command line:  stability-matched Z.npy P.npy   (or python -m
+#  stability_matched; also accepts one .npz with arrays Z and P, and .csv)
+#
+#  Empty supports are kept as empty rows (they lower p_f and kbar).  A matrix
+#  in which every row is empty, or every row selects every feature, has no
+#  defined stability (zero denominator): NaN is returned, never a number.
+#  Tests: test_stability_matched.py (pytest).
+# =============================================================================
+from __future__ import annotations
+
+import numpy as np
+from scipy import stats
+
+__version__ = "1.0.0"
+
+
+def phi_stability(Z):
+    """Nogueira's estimator with its asymptotic variance (Thm 9)."""
+    Z = np.asarray(Z, dtype=float)
+    M, d = Z.shape
+    k = Z.sum(axis=1)
+    kbar = k.mean()
+    p = Z.mean(axis=0)
+    s2 = M / (M - 1) * p * (1 - p)
+    den = (kbar / d) * (1 - kbar / d)
+    if den <= 0:
+        return {"phi": np.nan, "var": np.nan, "lo": np.nan, "hi": np.nan}
+    phi = 1 - s2.mean() / den
+    # asymptotic variance, Nogueira et al. (2018), Theorem 9
+    phis = np.empty(M)
+    for i in range(M):
+        phis[i] = (1 / den) * (np.mean(Z[i] * p) - k[i] * kbar / d ** 2
+                               + phi / 2 * (2 * k[i] * kbar / d ** 2
+                                            - k[i] / d - kbar / d + 1))
+    var = 4 / M ** 2 * np.sum((phis - phis.mean()) ** 2)
+    se = np.sqrt(var)
+    return {"phi": float(phi), "var": float(var),
+            "lo": float(phi - 1.96 * se), "hi": float(phi + 1.96 * se)}
+
+
+def phi0_matched(Z, P):
+    """Exact expectation of phi under the matched null (Proposition 1)."""
+    Z = np.asarray(Z, dtype=float)
+    P = np.asarray(P, dtype=bool)
+    M, d = Z.shape
+    k = Z.sum(axis=1)
+    m = P.sum(axis=1)
+    pi = np.where(P, (k / np.maximum(m, 1))[:, None], 0.0)
+    mu = pi.mean(axis=0)
+    v = (pi * (1 - pi)).sum(axis=0) / M ** 2
+    es2 = M / (M - 1) * (mu - mu ** 2 - v)
+    kb = k.mean()
+    den = (kb / d) * (1 - kb / d)
+    return float(1 - es2.mean() / den) if den > 0 else np.nan
+
+
+def _phi0_from_pi(pi, kbar):
+    M, d = pi.shape
+    mu = pi.mean(axis=0)
+    v = (pi * (1 - pi)).sum(axis=0) / M ** 2
+    es2 = M / (M - 1) * (mu - mu ** 2 - v)
+    den = (kbar / d) * (1 - kbar / d)
+    return float(1 - es2.mean() / den) if den > 0 else np.nan
+
+
+def phi0_stratified(Z, P, groups):
+    """Exact expectation of phi when, in every row, the number of selected
+    features in each group is kept and the features within the group (and
+    within the row's pool) are chosen uniformly at random.  groups: integer
+    label per feature (a feature alone in its group is kept as selected)."""
+    Z = np.asarray(Z, dtype=float)
+    P = np.asarray(P, dtype=bool)
+    g = np.asarray(groups, dtype=int)
+    if np.any((Z > 0) & ~P):
+        raise ValueError("a selected feature lies outside its pool")
+    M, d = Z.shape
+    G = g.max() + 1
+    pi = np.zeros((M, d))
+    for i in range(M):
+        a = np.flatnonzero(P[i])
+        n_g = np.bincount(g[a], minlength=G)
+        c_g = np.bincount(g[a], weights=Z[i, a], minlength=G)
+        pi[i, a] = c_g[g[a]] / n_g[g[a]]
+    return _phi0_from_pi(pi, Z.sum(axis=1).mean())
+
+
+def stratified_null_draws(Z, P, groups, B=999, rng=None):
+    rng = np.random.default_rng(rng)
+    Z = np.asarray(Z, dtype=bool)
+    P = np.asarray(P, dtype=bool)
+    g = np.asarray(groups, dtype=int)
+    M, d = Z.shape
+    out = np.empty(B)
+    for b in range(B):
+        R = np.zeros((M, d), dtype=np.uint8)
+        for i in range(M):
+            for grp in np.unique(g[Z[i]]):
+                cell = np.flatnonzero((g == grp) & P[i])
+                R[i, rng.choice(cell, int((Z[i] & (g == grp)).sum()),
+                                replace=False)] = 1
+        out[b] = phi_stability(R)["phi"]
+    return out
+
+
+def matched_null_draws(Z, P, B=999, rng=None):
+    rng = np.random.default_rng(rng)
+    Z = np.asarray(Z, dtype=bool)
+    P = np.asarray(P, dtype=bool)
+    M, d = Z.shape
+    k = Z.sum(axis=1)
+    pools = [np.flatnonzero(P[i]) for i in range(M)]
+    if np.any(Z & ~P):
+        raise ValueError("a selected feature lies outside its pool")
+    out = np.empty(B)
+    for b in range(B):
+        R = np.zeros((M, d), dtype=np.uint8)
+        for i in range(M):
+            if k[i]:
+                R[i, rng.choice(pools[i], k[i], replace=False)] = 1
+        out[b] = phi_stability(R)["phi"]
+    return out
+
+
+def matched_null_test(Z, P, B=999, rng=None):
+    obs = phi_stability(Z)["phi"]
+    nd = matched_null_draws(Z, P, B, rng)
+    return {"phi": obs, "p_mc": float((1 + np.sum(nd >= obs)) / (B + 1)),
+            "null_mean": float(nd.mean()), "null_sd": float(nd.std(ddof=1)),
+            "null_q95": float(np.quantile(nd, 0.95)),
+            "null_q99": float(np.quantile(nd, 0.99)), "B": B}
+
+
+def _hyper_jaccard(ka, kb, m):
+    x = np.arange(max(0, ka + kb - m), min(ka, kb) + 1)
+    p = stats.hypergeom.pmf(x, m, ka, kb)
+    return float(np.sum(p * x / (ka + kb - x)))
+
+
+def agreement_exact(A, B, P):
+    """Mean Jaccard of two supports per resample and its exact chance level
+    (resamples where either support is empty are skipped)."""
+    obs, ch = [], []
+    for a, b, p in zip(np.asarray(A, bool), np.asarray(B, bool),
+                       np.asarray(P, bool)):
+        ka, kb = int(a.sum()), int(b.sum())
+        if ka == 0 or kb == 0:
+            continue
+        obs.append((a & b).sum() / (a | b).sum())
+        ch.append(_hyper_jaccard(ka, kb, int(p.sum())))
+    if not obs:
+        return {"agree": np.nan, "agree0": np.nan, "n": 0}
+    return {"agree": float(np.mean(obs)), "agree0": float(np.mean(ch)),
+            "agree_x": float(np.mean(obs) - np.mean(ch)), "n": len(obs)}
+
+
+def phi_matched(Z, P, B=999, rng=None):
+    """Phi, its interval, the exact matched chance level, the excess and the
+    Monte Carlo test, in one call."""
+    st = phi_stability(Z)
+    p0 = phi0_matched(Z, P)
+    out = {"phi": st["phi"], "phi_lo": st["lo"], "phi_hi": st["hi"],
+           "phi0": p0, "phi_x": st["phi"] - p0,
+           "empty_share": float(np.mean(np.asarray(Z).sum(axis=1) == 0))}
+    if B:
+        t = matched_null_test(Z, P, B, rng)
+        out.update({"p_mc": t["p_mc"], "null_sd": t["null_sd"],
+                    "null_q99": t["null_q99"]})
+    return out
+
+
+def matrices_from_indices(supports, pools=None, d=None):
+    """Selection matrix Z and pool matrix P (M x d, 0/1) from per-resample
+    lists of selected feature indices and of candidate-pool indices.  With
+    pools=None every feature is a candidate in every resample."""
+    supports = [np.asarray(s, dtype=int) for s in supports]
+    if d is None:
+        alls = supports + ([np.asarray(p, dtype=int) for p in pools]
+                           if pools is not None else [])
+        d = int(max((a.max() for a in alls if a.size), default=-1)) + 1
+    M = len(supports)
+    Z = np.zeros((M, d), dtype=np.uint8)
+    P = np.ones((M, d), dtype=np.uint8) if pools is None else \
+        np.zeros((M, d), dtype=np.uint8)
+    for i, s in enumerate(supports):
+        Z[i, s] = 1
+        if pools is not None:
+            P[i, np.asarray(pools[i], dtype=int)] = 1
+    return Z, P
+
+
+def report(Z, P, B=999, rng=0):
+    """Plain-text summary: phi with interval, matched chance level, excess
+    and Monte Carlo test; empty supports and support sizes."""
+    r = phi_matched(Z, P, B=B, rng=rng)
+    k = np.asarray(Z).sum(axis=1)
+    m = np.asarray(P).sum(axis=1)
+    lines = [
+        f"resamples M = {len(k)}, features d = {np.asarray(Z).shape[1]}",
+        f"support size: mean {k.mean():.1f}, range {k.min()}-{k.max()}; "
+        f"empty supports {100 * r['empty_share']:.0f}%",
+        f"candidate pool size: mean {m.mean():.1f}, range {m.min()}-{m.max()}",
+        f"phi (Nogueira)          {r['phi']:.4f}  "
+        f"[{r['phi_lo']:.4f}, {r['phi_hi']:.4f}]",
+        f"matched chance level    {r['phi0']:.4f}",
+        f"excess over chance      {r['phi_x']:+.4f}  "
+        f"[{r['phi_lo'] - r['phi0']:+.4f}, {r['phi_hi'] - r['phi0']:+.4f}]"]
+    if B:
+        lines.append(f"Monte Carlo p (B = {B})  {r['p_mc']:.4f}   "
+                     f"(null SD {r['null_sd']:.4f})")
+    return "\n".join(lines)
+
+
+def _load(path, key=None):
+    path = str(path)
+    if path.endswith(".npz"):
+        z = np.load(path)
+        return z[key] if key else z
+    if path.endswith(".csv"):
+        return np.loadtxt(path, delimiter=",")
+    return np.load(path)
+
+
+def main(argv=None):
+    import argparse
+    ap = argparse.ArgumentParser(
+        prog="stability-matched",
+        description="Selection stability with its exact matched chance level")
+    ap.add_argument("Z", help="selection matrix (resamples x features, 0/1): "
+                    ".npy, .csv, or .npz holding arrays Z and P")
+    ap.add_argument("P", nargs="?", default=None,
+                    help="pool matrix of the same shape (omit if every "
+                    "feature is a candidate in every resample)")
+    ap.add_argument("-B", type=int, default=999,
+                    help="null matrices for the Monte Carlo test (0: none)")
+    ap.add_argument("--seed", type=int, default=0)
+    a = ap.parse_args(argv)
+    if a.Z.endswith(".npz") and a.P is None:
+        z = _load(a.Z)
+        Z = z["Z"]
+        P = z["P"] if "P" in z.files else np.ones_like(Z)
+    else:
+        Z = _load(a.Z)
+        P = _load(a.P) if a.P else np.ones_like(Z)
+    print(report(Z, P, B=a.B, rng=a.seed))
+
+
+if __name__ == "__main__":
+    main()