pyplasmode 1.0.0__tar.gz → 1.2.0__tar.gz

pyplasmode-1.2.0/CHANGELOG.md

@@ -0,0 +1,60 @@
+# Changelog
+
+## 1.2.0 — 2026-11-09
+
+### Added
+
+- `pyplasmode.metrics`: chance-corrected selection stability with unequal budgets
+  and ties, tie-aware RBO, inclusion-first consensus ranks, IPCW cumulative/dynamic
+  AUROC, paired participant bootstrap differences and oracle-minus-fitted
+  `discrimination_gap` (also available as `learnability_gap`).
+- `target_auc` for binary and survival outcomes, reporting expected oracle AUROC
+  and the solved odds/hazard ratio while retaining marginal risk.
+- Metrics and calibration guides, executable examples and a synthetic Cox,
+  ridge and XGBoost benchmark comparing held-out AUROC and chance-adjusted recovery.
+  Model dependencies are supplied by the optional `benchmark` extra.
+- Identity-aligned paired replicate comparisons with explicit failed/unmatched
+  pair counts and conditional Monte Carlo uncertainty.
+- `allow_subset=True` for fractional recovery, matched recovery and outcome
+  specificity, preserving generating denominators and the available-panel reference.
+- Python 3.15 support and NumPy string arrays as bootstrap participant IDs.
+
+### Changed
+
+- RBO defaults to Webber whole-tie prefixes; independent tie resolution remains
+  available through `tie_method="independent"`.
+- Censored AUROC defaults to exact score ties. Use `tied_tolerance=1e-8` to
+  reproduce scikit-survival's default score-tie convention.
+- Default survival AUROC horizons precede the administrative cutoff; an explicit
+  later administrative cutoff is supported.
+- Calibration parameters support direct lookup, and `metrics` is discoverable
+  through the root package. Unattainable target AUROCs raise explicitly.
+- The validation extra requires statsmodels 0.14.6 or later. Source distributions
+  omit the unreferenced concept PDFs, which remain in the repository.
+
+### Fixed
+
+- Never-selected consensus features sort below selected features even with wide ties.
+- AUROC-targeted calibration tolerates single extreme signal observations and
+  rejects requests above the supported effect bracket without clipping.
+- Edgeless graph regions report unavailable edge coverage and expose the target
+  edge count while retaining node recovery.
+- Panel restrictions retain module weights, group proxies and joint interaction
+  requirements rather than redefining the generating truth.
+
+## 1.0.0
+
+First public release. PyPlasmode provides outcome generation, feature-recovery evaluation,
+worked examples and reference validation for biomarker benchmarking.
+
+### Included in this release
+
+- Empirical complete-row resampling and source-partitioned generation.
+- Null, sparse, correlated-group, module, interaction and custom signals.
+- Five outcome families calibrated in observable units.
+- Recovery, prediction, panel, reconstruction and rank-stability evaluation.
+- API documentation, worked examples, validation figures and cross-platform CI.
+- A visual gallery of signal shapes, correlated-feature recovery and outcome families,
+  with editable vector figures and an executable source-data example.
+- Python 3.12-3.14 and scikit-learn 1.6 through the current 1.x series.
+- BSD 3-Clause licensing.

{pyplasmode-1.0.0 → pyplasmode-1.2.0}/CITATION.cff

@@ -2,7 +2,7 @@ cff-version: 1.2.0
 message: "If you use PyPlasmode in your research, please cite this software using the metadata below."
 title: "PyPlasmode"
 type: software
-version: 1.0.0
+version: 1.2.0
 repository-code: "https://github.com/BradSegal/PyPlasmode"
 license: BSD-3-Clause
 authors:

{pyplasmode-1.0.0 → pyplasmode-1.2.0}/CONTRIBUTING.md

@@ -4,7 +4,7 @@ Contributions to generators, evaluation functions, documentation and examples ar
 For a new signal or outcome family, describe the intended use and mathematical definition
 in an issue before implementing it.
 
-From a checkout, install a supported Python version (3.12-3.14) and uv, then run:
+From a checkout, install a supported Python version (3.12-3.15) and uv, then run:
 
 ```bash
 uv sync --extra validation --group docs
@@ -18,6 +18,10 @@ uv run twine check --strict dist/*
 uv run check-wheel-contents dist/*.whl
 ```
 
+Add `--extra benchmark` to `uv sync` to run the synthetic Cox/ridge/XGBoost
+example and its integration test. Without those optional dependencies the
+integration test is explicitly skipped.
+
 Run `python examples/validation_vignette.py --output validation-output` to regenerate the
 synthetic validation figures and tables. The `validation` extra is included in the setup above.
 
@@ -46,3 +50,9 @@ Record random seeds and dependency versions when sharing reproducible analyses.
 The CI matrix covers minimum numerical dependencies on Python 3.12 and current dependencies
 on supported Python versions, with Linux, Windows and macOS jobs. Test dependency changes
 against this matrix. Build release artifacts from the tagged version after CI passes.
+
+Attach the checked wheel and source distribution to a draft GitHub release, then publish
+the release. The `publish.yml` workflow validates those distributions and uploads them to
+PyPI through the `pypi` trusted-publishing environment. For an existing release, run
+`gh workflow run publish.yml --ref main -f tag=v1.0.0` with its tag. No PyPI API token is
+required, and already-published versions must not be replaced.

pyplasmode-1.2.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: pyplasmode
+Version: 1.2.0
+Summary: Simulate outcomes from feature data and evaluate prediction and feature selection.
+Project-URL: Repository, https://github.com/BradSegal/PyPlasmode
+Project-URL: Documentation, https://bradsegal.github.io/PyPlasmode/
+Project-URL: Issues, https://github.com/BradSegal/PyPlasmode/issues
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: <3.16,>=3.12
+Requires-Dist: numpy<3,>=2.0
+Requires-Dist: scikit-learn<2,>=1.6
+Requires-Dist: scipy<2,>=1.14
+Provides-Extra: benchmark
+Requires-Dist: scikit-survival<1,>=0.24; extra == 'benchmark'
+Requires-Dist: threadpoolctl<4,>=3.1; extra == 'benchmark'
+Requires-Dist: xgboost<4,>=2.1; extra == 'benchmark'
+Provides-Extra: validation
+Requires-Dist: matplotlib<4,>=3.10; extra == 'validation'
+Requires-Dist: statsmodels<0.16,>=0.14.6; extra == 'validation'
+Description-Content-Type: text/markdown
+
+# PyPlasmode
+
+PyPlasmode tests prediction and feature selection against a known signal. It combines your
+feature data with simulated outcomes, so you can measure both how accurately a model predicts
+and how well it identifies the features used to generate those outcomes.
+
+Built for biomarker benchmarking, the library works with numeric tabular data more generally.
+Its inputs are a matrix of observations and named features, not a particular assay or dataset.
+A plasmode uses observed measurements to retain realistic feature relationships while you
+control the outcome-generating mechanism. Synthetic matrices work too, as in the examples below.
+
+Use it with your existing modelling workflow: generate a dataset, fit and tune your models,
+then evaluate their predictions, feature rankings or selected panels.
+
+[![From observed measurements to simulated outcomes, model predictions and feature recovery](https://raw.githubusercontent.com/BradSegal/PyPlasmode/main/docs/assets/concepts/plasmode-workflow.png)](https://bradsegal.github.io/PyPlasmode/methods/)
+
+The same generated dataset supports two comparisons: predictions against outcomes, and
+selected features against the signal that generated them.
+
+Explore the [visual gallery](https://bradsegal.github.io/PyPlasmode/gallery/) for signal patterns, correlated-feature recovery
+and the five outcome families.
+
+The [feature-selection tutorial](https://bradsegal.github.io/PyPlasmode/tutorial/) follows a complete example from correlated
+measurements through model fitting, attribution and recovery above chance.
+
+## Installation
+
+Requires Python 3.12-3.15. Install PyPlasmode 1.2.0 from [PyPI](https://pypi.org/project/pyplasmode/):
+
+```bash
+python -m pip install pyplasmode
+```
+
+The runtime dependencies are NumPy, SciPy and scikit-learn. To include the
+reference validation and survival examples:
+
+```bash
+python -m pip install "pyplasmode[validation]"
+```
+
+## Ten-minute example
+
+```python
+import numpy as np
+from sklearn.linear_model import LogisticRegression
+import pyplasmode as ppm
+
+rng = np.random.default_rng(1)
+population = ppm.Population(
+    rng.normal(size=(2_000, 100)),
+    tuple(f"feature_{index}" for index in range(100)),
+)
+
+partition = ppm.partition_population(
+    population,
+    training_fraction=0.6,
+    validation_fraction=0.2,
+    seed=7,
+)
+samples = ppm.generate_partitioned(
+    partition,
+    truth=ppm.SparseTruth(features=("feature_3", "feature_17", "feature_42")),
+    outcome=ppm.BinaryOutcome(probability=0.15, odds_ratio=2.0),
+    sample_sizes=ppm.PartitionSampleSizes(1_200, 400, 400),
+    sampling_method="without_replacement",
+    seed=42,
+)
+
+train, test = samples.training, samples.evaluation
+model = LogisticRegression(max_iter=1_000).fit(train.X, train.outcome.values)
+ranking = tuple(train.feature_ids[index] for index in np.argsort(-np.abs(model.coef_[0])))
+
+recovery = ppm.evaluate_ranking(ranking, train.truth, depths=(10, 20, 50))
+prediction = ppm.evaluate_prediction(
+    test.outcome.values,
+    model.predict_proba(test.X)[:, 1],
+    metric="roc_auc",
+)
+print(recovery.at_depth[0].recall)
+print(prediction.value)
+```
+
+Recall measures how many of the three generating features appear in the model's top ten;
+AUC measures prediction on held-out observations. Repeat the comparison with different seeds
+to estimate average performance and its Monte Carlo uncertainty. In 1.2.0,
+`compare_paired_replicates` aligns procedures by world identity before computing
+paired effects and explicitly counts failed or unmatched pairs. See
+[paired evaluation](https://bradsegal.github.io/PyPlasmode/evaluation/#paired-procedure-comparisons).
+
+Source rows are split before resampling, keeping training, validation and test
+rows separate. This example uses a fixed model; the validation sample is available for
+hyperparameter tuning. See [Methods](https://bradsegal.github.io/PyPlasmode/methods/) for the generation process.
+
+## What can be varied
+
+- **Signal:** null, sparse features, correlated groups, distributed modules, pairwise
+  interactions or a custom function.
+- **Outcome:** binary probability, continuous mean and spread, count rate, ordinal category
+  probabilities or cumulative event risks at native-time horizons.
+- **Evaluation:** feature and group recovery, weighted-module and graph-region coverage,
+  signal reconstruction, prediction accuracy, panel performance and ranking stability.
+  Additional functions handle tied scores, incomplete selections and matched chance references.
+
+Start with [Methods](https://bradsegal.github.io/PyPlasmode/methods/), then choose a [signal](https://bradsegal.github.io/PyPlasmode/truths/) and
+[outcome](https://bradsegal.github.io/PyPlasmode/outcomes/). The [evaluation guide](https://bradsegal.github.io/PyPlasmode/evaluation/) defines each result.
+The [validation example](https://bradsegal.github.io/PyPlasmode/validation/) checks the generators and recovery metrics against
+known values and reference estimators.
+
+The executable [quickstart](https://github.com/BradSegal/PyPlasmode/blob/main/examples/quickstart.py),
+[survival example](https://github.com/BradSegal/PyPlasmode/blob/main/examples/survival.py)
+and [custom signal](https://github.com/BradSegal/PyPlasmode/blob/main/examples/custom_signal.py)
+use synthetic inputs. The [documentation](https://bradsegal.github.io/PyPlasmode/)
+includes the generation methods, evaluation definitions and API reference.
+
+## Working with empirical data
+
+Resampling preserves the source data's feature relationships and missingness patterns.
+It can reproduce individual records, so generated datasets require the same privacy protections
+as their source. See [Data handling](https://bradsegal.github.io/PyPlasmode/privacy-and-limitations/).
+
+## Licence
+
+PyPlasmode uses the [BSD 3-Clause License](https://github.com/BradSegal/PyPlasmode/blob/main/LICENSE).
+
+## New in 1.2.0
+
+Version 1.2.0 extends generation and evaluation. `pyplasmode.metrics` adds chance-corrected
+selection stability, tie-aware RBO, consensus ranks, censored cumulative/dynamic
+AUROC, paired participant bootstrap differences and discrimination gaps. Binary
+and survival specifications accept `target_auc` and report the solved effect
+and expected oracle AUROC. See [metrics](https://bradsegal.github.io/PyPlasmode/metrics/),
+[calibration](https://bradsegal.github.io/PyPlasmode/calibration/) and the [worked example](https://github.com/BradSegal/PyPlasmode/blob/main/examples/metrics_calibration.py).
+
+The [synthetic benchmark](https://bradsegal.github.io/PyPlasmode/benchmark-demo/) fits Cox, ridge and XGBoost
+across three signal mechanisms and three target oracle AUROCs, then compares
+held-out AUROC with chance-adjusted top-20 recovery. Install the optional
+`benchmark` extra to run `python examples/benchmark_recovery_demo.py`.
+Install it with `python -m pip install "pyplasmode[benchmark]"`.

pyplasmode-1.2.0/README.md

@@ -0,0 +1,138 @@
+# PyPlasmode
+
+PyPlasmode tests prediction and feature selection against a known signal. It combines your
+feature data with simulated outcomes, so you can measure both how accurately a model predicts
+and how well it identifies the features used to generate those outcomes.
+
+Built for biomarker benchmarking, the library works with numeric tabular data more generally.
+Its inputs are a matrix of observations and named features, not a particular assay or dataset.
+A plasmode uses observed measurements to retain realistic feature relationships while you
+control the outcome-generating mechanism. Synthetic matrices work too, as in the examples below.
+
+Use it with your existing modelling workflow: generate a dataset, fit and tune your models,
+then evaluate their predictions, feature rankings or selected panels.
+
+[![From observed measurements to simulated outcomes, model predictions and feature recovery](https://raw.githubusercontent.com/BradSegal/PyPlasmode/main/docs/assets/concepts/plasmode-workflow.png)](https://bradsegal.github.io/PyPlasmode/methods/)
+
+The same generated dataset supports two comparisons: predictions against outcomes, and
+selected features against the signal that generated them.
+
+Explore the [visual gallery](https://bradsegal.github.io/PyPlasmode/gallery/) for signal patterns, correlated-feature recovery
+and the five outcome families.
+
+The [feature-selection tutorial](https://bradsegal.github.io/PyPlasmode/tutorial/) follows a complete example from correlated
+measurements through model fitting, attribution and recovery above chance.
+
+## Installation
+
+Requires Python 3.12-3.15. Install PyPlasmode 1.2.0 from [PyPI](https://pypi.org/project/pyplasmode/):
+
+```bash
+python -m pip install pyplasmode
+```
+
+The runtime dependencies are NumPy, SciPy and scikit-learn. To include the
+reference validation and survival examples:
+
+```bash
+python -m pip install "pyplasmode[validation]"
+```
+
+## Ten-minute example
+
+```python
+import numpy as np
+from sklearn.linear_model import LogisticRegression
+import pyplasmode as ppm
+
+rng = np.random.default_rng(1)
+population = ppm.Population(
+    rng.normal(size=(2_000, 100)),
+    tuple(f"feature_{index}" for index in range(100)),
+)
+
+partition = ppm.partition_population(
+    population,
+    training_fraction=0.6,
+    validation_fraction=0.2,
+    seed=7,
+)
+samples = ppm.generate_partitioned(
+    partition,
+    truth=ppm.SparseTruth(features=("feature_3", "feature_17", "feature_42")),
+    outcome=ppm.BinaryOutcome(probability=0.15, odds_ratio=2.0),
+    sample_sizes=ppm.PartitionSampleSizes(1_200, 400, 400),
+    sampling_method="without_replacement",
+    seed=42,
+)
+
+train, test = samples.training, samples.evaluation
+model = LogisticRegression(max_iter=1_000).fit(train.X, train.outcome.values)
+ranking = tuple(train.feature_ids[index] for index in np.argsort(-np.abs(model.coef_[0])))
+
+recovery = ppm.evaluate_ranking(ranking, train.truth, depths=(10, 20, 50))
+prediction = ppm.evaluate_prediction(
+    test.outcome.values,
+    model.predict_proba(test.X)[:, 1],
+    metric="roc_auc",
+)
+print(recovery.at_depth[0].recall)
+print(prediction.value)
+```
+
+Recall measures how many of the three generating features appear in the model's top ten;
+AUC measures prediction on held-out observations. Repeat the comparison with different seeds
+to estimate average performance and its Monte Carlo uncertainty. In 1.2.0,
+`compare_paired_replicates` aligns procedures by world identity before computing
+paired effects and explicitly counts failed or unmatched pairs. See
+[paired evaluation](https://bradsegal.github.io/PyPlasmode/evaluation/#paired-procedure-comparisons).
+
+Source rows are split before resampling, keeping training, validation and test
+rows separate. This example uses a fixed model; the validation sample is available for
+hyperparameter tuning. See [Methods](https://bradsegal.github.io/PyPlasmode/methods/) for the generation process.
+
+## What can be varied
+
+- **Signal:** null, sparse features, correlated groups, distributed modules, pairwise
+  interactions or a custom function.
+- **Outcome:** binary probability, continuous mean and spread, count rate, ordinal category
+  probabilities or cumulative event risks at native-time horizons.
+- **Evaluation:** feature and group recovery, weighted-module and graph-region coverage,
+  signal reconstruction, prediction accuracy, panel performance and ranking stability.
+  Additional functions handle tied scores, incomplete selections and matched chance references.
+
+Start with [Methods](https://bradsegal.github.io/PyPlasmode/methods/), then choose a [signal](https://bradsegal.github.io/PyPlasmode/truths/) and
+[outcome](https://bradsegal.github.io/PyPlasmode/outcomes/). The [evaluation guide](https://bradsegal.github.io/PyPlasmode/evaluation/) defines each result.
+The [validation example](https://bradsegal.github.io/PyPlasmode/validation/) checks the generators and recovery metrics against
+known values and reference estimators.
+
+The executable [quickstart](https://github.com/BradSegal/PyPlasmode/blob/main/examples/quickstart.py),
+[survival example](https://github.com/BradSegal/PyPlasmode/blob/main/examples/survival.py)
+and [custom signal](https://github.com/BradSegal/PyPlasmode/blob/main/examples/custom_signal.py)
+use synthetic inputs. The [documentation](https://bradsegal.github.io/PyPlasmode/)
+includes the generation methods, evaluation definitions and API reference.
+
+## Working with empirical data
+
+Resampling preserves the source data's feature relationships and missingness patterns.
+It can reproduce individual records, so generated datasets require the same privacy protections
+as their source. See [Data handling](https://bradsegal.github.io/PyPlasmode/privacy-and-limitations/).
+
+## Licence
+
+PyPlasmode uses the [BSD 3-Clause License](https://github.com/BradSegal/PyPlasmode/blob/main/LICENSE).
+
+## New in 1.2.0
+
+Version 1.2.0 extends generation and evaluation. `pyplasmode.metrics` adds chance-corrected
+selection stability, tie-aware RBO, consensus ranks, censored cumulative/dynamic
+AUROC, paired participant bootstrap differences and discrimination gaps. Binary
+and survival specifications accept `target_auc` and report the solved effect
+and expected oracle AUROC. See [metrics](https://bradsegal.github.io/PyPlasmode/metrics/),
+[calibration](https://bradsegal.github.io/PyPlasmode/calibration/) and the [worked example](https://github.com/BradSegal/PyPlasmode/blob/main/examples/metrics_calibration.py).
+
+The [synthetic benchmark](https://bradsegal.github.io/PyPlasmode/benchmark-demo/) fits Cox, ridge and XGBoost
+across three signal mechanisms and three target oracle AUROCs, then compares
+held-out AUROC with chance-adjusted top-20 recovery. Install the optional
+`benchmark` extra to run `python examples/benchmark_recovery_demo.py`.
+Install it with `python -m pip install "pyplasmode[benchmark]"`.

{pyplasmode-1.0.0 → pyplasmode-1.2.0}/docs/api.md

@@ -1,19 +1,19 @@
 # API reference
 
 The common entry points are exported from `pyplasmode`. Matrices have shape
-`(participants, features)`; `feature_ids` supplies one unique string per column.
+`(observations, features)`; `feature_ids` supplies one unique string per column.
 Generated arrays are read-only. Convert result dataclasses to plain records with
 `dataclasses.asdict` for your reporting workflow.
 
 Use `generate` for one sample or `partition_population` followed by
-`generate_partitioned` for source-disjoint development and evaluation. Access
+`generate_partitioned` to keep source rows separate between training, validation and test. Access
 `outcome.values` for binary, continuous and count outcomes, `outcome.codes` for
 ordinal outcomes, and `outcome.time` plus `outcome.event` for survival outcomes.
 Survival times are in days; latent event and censoring times beyond administrative
 follow-up are represented by infinity.
 
-The rendered reference includes signatures, types and result fields. See the
-methods and evaluation guides for the scientific definitions.
+The reference below lists signatures, types and result fields. See
+[Methods](methods.md) for generation and [Evaluation](evaluation.md) for scoring.
 
 ## Populations and sampling
 
@@ -39,10 +39,17 @@ methods and evaluation guides for the scientific definitions.
 
 ::: pyplasmode.evaluate
 
-## Fractional nominations and matched references
+## Tied selections and matched references
+
+The `nominations` module represents selections with membership weights, so tied scores
+can share a place in a leading list. It also compares recovery with matched random selection.
 
 ::: pyplasmode.nominations
 
 ## Evaluation results
 
 ::: pyplasmode.results
+
+## Selection agreement and censored discrimination
+
+::: pyplasmode.metrics

pyplasmode-1.2.0/docs/assets/concepts/README.md

@@ -0,0 +1,8 @@
+# Concept diagrams
+
+These diagrams illustrate outcome generation and feature recovery in PyPlasmode.
+The repository retains the vector PDFs for editing and print use; the documentation
+uses PNG previews. The source distribution includes the previews.
+
+- `plasmode-workflow`: observed features, generating signal, outcomes and evaluation.
+- `signal-recovery`: generating contributors and features selected by a fitted model.

pyplasmode-1.2.0/docs/benchmark-demo.md

@@ -0,0 +1,80 @@
+# Prediction and biomarker recovery on synthetic data
+
+[`examples/benchmark_recovery_demo.py`](https://github.com/BradSegal/PyPlasmode/blob/main/examples/benchmark_recovery_demo.py)
+is a small, self-contained comparison of Cox, ridge and XGBoost. It generates
+all features and outcomes in memory; it requires no participant data or downloads.
+
+On macOS, install the OpenMP runtime required by XGBoost with
+`brew install libomp`. Then install the optional model dependencies and download
+the linked example:
+
+```bash
+python -m pip install "pyplasmode[benchmark]"
+python benchmark_recovery_demo.py --output benchmark-results.csv
+```
+
+The core package does not require scikit-survival or XGBoost. The extra is
+tested on Python 3.12–3.15 on Linux, with Windows and macOS covered by CI on
+Python 3.13; installation depends on the model libraries' platform support.
+
+The default is 3,000 rows and 120 features, with one numerical thread. Allow a
+few minutes on a laptop CPU. `--rows 1000` provides a smaller demonstration;
+`--seed` sets the random seed (default 42). The final console table and optional
+CSV contain all 27 comparisons: three mechanisms, three effect sizes and three
+methods. The CSV retains full precision, expected and realised oracle AUROC,
+the recovery definition and the achieved nomination size.
+
+## What is compared
+
+Features are Gaussian, arranged in 30 independent blocks of four. Features in
+the same block have population correlation 0.7. There are three signals:
+
+| Signal | Generating mechanism | Recovery at depth 20 |
+| --- | --- | --- |
+| Sparse | Eight direct features, one in each of eight blocks | Fraction of direct features selected |
+| Correlated | Eight direct sentinels, each with three admissible correlated substitutes | Fraction of groups represented by at least one selected member |
+| Distributed | Equal weights on the first 60 features | Fraction of absolute generating weight selected |
+
+The sparse and correlated signals use the same generating features. Their
+different recovery definitions show how allowing correlated substitutes changes
+the question being answered. A 20-feature panel cannot cover all 60 distributed
+features; its maximum unadjusted weight coverage is one third. Recovery columns
+therefore measure each stated mechanism and should be interpreted within it.
+
+`TimeToEventOutcome(target_auc=...)` sets expected ten-year oracle AUROC to
+0.60, 0.70 and 0.80 in the finite synthetic covariate population. Ten-year event
+risk is 0.20 and eleven-year risk is 0.22. Follow-up ends at eleven years,
+with no earlier censoring. Calibration precedes outcome sampling, so realised
+held-out oracle AUROC varies around the requested value. Common random draws
+are used across settings.
+
+A fixed 60:40 split separates training and evaluation. Scaling is learnt on
+training features only. Cox uses all observed event times with L2 penalty 1.
+Ridge uses ten-year event indicators with penalty 10. XGBoost uses the same
+indicators, 80 depth-two trees, learning rate 0.05 and one CPU thread. There is
+no hyperparameter search. For ridge and XGBoost the horizon labels are fully
+observed because there is no early censoring; this construction would need to
+change if early censoring were introduced.
+
+## Reading the output
+
+AUROC is the ten-year cumulative/dynamic estimate from
+`pyplasmode.metrics.cumulative_dynamic_auc`, using the training censoring
+distribution. Higher scores mean higher risk. With the chosen follow-up this
+equals ordinary binary AUROC at ten years.
+
+Absolute standardised coefficients rank Cox and ridge features; XGBoost uses
+gain importance. `nomination_from_scores` allocates boundary ties fractionally
+and does not fill unused slots with zero-importance features. `Top20` reports
+the achieved size. `evaluate_matched_recovery` compares each nomination to
+truth-blind random nominations with the same size, positive support and tie
+geometry, within one common feature stratum. All blocks have the same size and
+population correlation in this example.
+
+`Adjusted` is **observed minus expected chance recovery**, without rescaling
+or clipping. Negative values are possible. `Target` is the expected oracle AUROC requested and achieved by calibration.
+The `Oracle` column is the realised
+held-out AUROC of the generating signal. Compare fitted AUROC and recovery
+across the three targets: stronger discrimination need not yield an equally
+large change in recovery. This is one synthetic split with fixed model settings,
+so its table demonstrates the evaluation rather than establishing a method ranking.

pyplasmode-1.2.0/docs/calibration.md

@@ -0,0 +1,108 @@
+# Calibrating oracle AUROC
+
+Use `target_auc` when the intended simulation difficulty is easier to specify
+than an odds or hazard ratio. The generator solves a non-negative signal
+coefficient, while preserving the requested marginal probability or event risks.
+The resulting ratio is at least one per signal standard deviation.
+
+```python
+import numpy as np
+import pyplasmode as ppm
+
+signal = np.random.default_rng(12).normal(size=20_000)
+binary = ppm.generate_outcome(
+    ppm.BinaryOutcome(probability=0.1, target_auc=0.7),
+    signal,
+    np.random.default_rng(3),
+)
+assert abs(binary.calibration.oracle_auc - 0.7) < 1e-7
+odds_ratio = binary.calibration["odds_ratio"]
+```
+
+For survival, set `auc_horizon` strictly before the administrative cutoff.
+It defaults to the last event-risk point before the cutoff, or half the cutoff
+when there is only one point. Interior horizons interpolate the cumulative
+baseline hazard under the specified piecewise-constant rates. To target exactly
+ten years with one risk point, set `administrative_horizon=ppm.years(11)`.
+Survivors then remain under observation without additional events or censoring
+between the final risk point and that later cutoff. Independent censoring changes the precision of a
+censoring-aware estimate, but not the population oracle AUROC or solved hazard
+ratio. All event and censoring risk points are still calibrated. The example
+includes year 11 to retain follow-up beyond the ten-year AUROC horizon; unlike
+setting only a later administrative cutoff, this also specifies additional
+event and censoring risk between years 10 and 11.
+
+```python
+horizon = ppm.years(10)
+survival = ppm.generate_outcome(
+    ppm.TimeToEventOutcome(
+        event_risks={horizon: 0.005, ppm.years(11): 0.006},
+        target_auc=0.72,
+        auc_horizon=horizon,
+        censoring_risks={horizon: 0.05, ppm.years(11): 0.055},
+    ),
+    signal,
+    np.random.default_rng(3),
+)
+assert abs(survival.calibration.oracle_auc - 0.72) < 1e-7
+hazard_ratio = survival.calibration["hazard_ratio"]
+```
+
+## What is solved
+
+The signal is standardised on the supplied rows. Binary outcomes use logistic
+probabilities; survival outcomes use `1-exp(-H0(t)*exp(beta*z))`. For each trial
+coefficient, the intercept or baseline hazard is recalibrated to the requested
+marginal risk. Brent root-finding then matches the expected oracle AUROC.
+
+The expectation integrates case/control pairs over the empirical signal
+population, with half credit for ties. It uses independent draws from that
+population, including draws of the same source row. It does not optimise a noisy
+AUROC from one realised outcome sample. `CalibrationResult.oracle_auc` records
+this achieved expectation; `target_auc` records the request. Realised AUROC will
+vary with the seed, sample size, event count and censoring. The solved effect is
+deterministic for a given signal and specification; repeating the generator
+with the same seed also repeats the outcomes.
+
+`generate_partitioned` calibrates once on the combined partition covariates,
+so its returned samples share one solved effect. Calling `generate` separately
+on training and test cohorts solves `target_auc` independently on each cohort.
+Use `generate_partitioned` for a shared mechanism, or extract the solved ratio
+with `train.outcome.calibration["odds_ratio"]` and pass it explicitly for test
+outcomes. Their marginal baselines are still recalibrated separately.
+
+`CalibrationResult.parameter_dict` returns a fresh parameter mapping;
+`calibration["signal_coefficient"]` looks up one value. For protective effects
+(`odds_ratio < 1` or `hazard_ratio < 1`), use the solved coefficient times the
+standardised signal as the oracle risk score. Passing the unflipped signal
+would reverse the score direction. `target_auc` always solves a non-negative
+coefficient.
+
+## Invalid and unattainable targets
+
+Specify exactly one binary effect measure or `target_auc`, and exactly one
+survival `hazard_ratio` or `target_auc`. A target must lie in `[0.5, 1)`; invalid
+specifications raise `OutcomeSpecificationError`. A survival target requires
+positive event risk below one at its evaluation horizon.
+
+A target of 0.5 solves ratio one, including constant signals. A constant signal
+cannot attain a higher target. Ties and the signal distribution can also limit
+attainable discrimination. The solver supports log ratios from zero to eight,
+independently of the largest signal value. Logistic probabilities use `expit`
+and the intercept bracket expands to cover the supplied offsets. Survival
+multipliers saturate at `exp(-700)` and `exp(700)` to keep extreme tail hazards
+finite. No observations are dropped or winsorised. A request outside the
+supported effect range or the signal's attainable discrimination raises
+`CalibrationError`; it is never silently clipped. The target describes the
+true generating predictor: a model fitted to finite training data will usually
+have lower held-out AUROC.
+
+## Validation
+
+The statistical suite checks expected AUROC to absolute tolerance `1e-7`.
+Generated binary AUROCs are checked within 0.025 across probabilities 0.02, 0.2
+and 0.5 and targets 0.5, 0.6, 0.75 and 0.9, using 80,000 synthetic participants
+per cell. Survival checks use 120,000 participants, event risks 0.005, 0.05 and
+0.2, censoring risks 0, 0.05 and 0.3, and targets 0.5, 0.61, 0.72 and 0.79;
+the realised IPCW tolerance is 0.035. These are reproducible simulation checks,
+not a guarantee of this accuracy for a user's smaller sample.