pertrank 0.2.0rc2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. pertrank-0.2.0rc2/CHANGELOG.md +24 -0
  2. pertrank-0.2.0rc2/INTEGRATION.md +9 -0
  3. pertrank-0.2.0rc2/LICENSE +14 -0
  4. pertrank-0.2.0rc2/MANIFEST.in +10 -0
  5. pertrank-0.2.0rc2/PKG-INFO +156 -0
  6. pertrank-0.2.0rc2/README.md +132 -0
  7. pertrank-0.2.0rc2/READ_FIRST.zh.md +57 -0
  8. pertrank-0.2.0rc2/docs/api.md +165 -0
  9. pertrank-0.2.0rc2/docs/community_integration.md +56 -0
  10. pertrank-0.2.0rc2/docs/input_contracts.md +56 -0
  11. pertrank-0.2.0rc2/docs/release_process.md +35 -0
  12. pertrank-0.2.0rc2/docs/scientific_design.md +44 -0
  13. pertrank-0.2.0rc2/docs/validation.md +23 -0
  14. pertrank-0.2.0rc2/provenance/CURRENT_RELEASE.json +8 -0
  15. pertrank-0.2.0rc2/provenance/ORIGINAL_RELEASE.json +58 -0
  16. pertrank-0.2.0rc2/provenance/ORIGINAL_SHA256SUMS.txt +44 -0
  17. pertrank-0.2.0rc2/pyproject.toml +32 -0
  18. pertrank-0.2.0rc2/requirements-tested.txt +14 -0
  19. pertrank-0.2.0rc2/scripts/README.md +18 -0
  20. pertrank-0.2.0rc2/scripts/check_installation.py +41 -0
  21. pertrank-0.2.0rc2/scripts/check_tutorials.py +27 -0
  22. pertrank-0.2.0rc2/scripts/curate_replogle.py +96 -0
  23. pertrank-0.2.0rc2/scripts/curate_tcell.py +81 -0
  24. pertrank-0.2.0rc2/setup.cfg +4 -0
  25. pertrank-0.2.0rc2/src/pertrank/__init__.py +11 -0
  26. pertrank-0.2.0rc2/src/pertrank/_version.py +3 -0
  27. pertrank-0.2.0rc2/src/pertrank/api.py +457 -0
  28. pertrank-0.2.0rc2/src/pertrank/core.py +255 -0
  29. pertrank-0.2.0rc2/src/pertrank/data/replogle_k562.npz +0 -0
  30. pertrank-0.2.0rc2/src/pertrank/data/replogle_k562.provenance.json +62 -0
  31. pertrank-0.2.0rc2/src/pertrank/data/tcell_d3_rest.npz +0 -0
  32. pertrank-0.2.0rc2/src/pertrank/data/tcell_d3_rest.provenance.json +63 -0
  33. pertrank-0.2.0rc2/src/pertrank/datasets.py +52 -0
  34. pertrank-0.2.0rc2/src/pertrank/evaluation.py +107 -0
  35. pertrank-0.2.0rc2/src/pertrank/py.typed +0 -0
  36. pertrank-0.2.0rc2/src/pertrank.egg-info/PKG-INFO +156 -0
  37. pertrank-0.2.0rc2/src/pertrank.egg-info/SOURCES.txt +55 -0
  38. pertrank-0.2.0rc2/src/pertrank.egg-info/dependency_links.txt +1 -0
  39. pertrank-0.2.0rc2/src/pertrank.egg-info/requires.txt +12 -0
  40. pertrank-0.2.0rc2/src/pertrank.egg-info/top_level.txt +1 -0
  41. pertrank-0.2.0rc2/tests/REFERENCE_PROVENANCE.json +5 -0
  42. pertrank-0.2.0rc2/tests/conftest.py +41 -0
  43. pertrank-0.2.0rc2/tests/reference_core_v21_1.py +225 -0
  44. pertrank-0.2.0rc2/tests/test_anndata_api.py +283 -0
  45. pertrank-0.2.0rc2/tests/test_core_contract.py +124 -0
  46. pertrank-0.2.0rc2/tests/test_evaluation.py +122 -0
  47. pertrank-0.2.0rc2/tests/test_packaged_data.py +30 -0
  48. pertrank-0.2.0rc2/tutorials/00_anndata_quickstart.py +17 -0
  49. pertrank-0.2.0rc2/tutorials/01_replogle.html +8283 -0
  50. pertrank-0.2.0rc2/tutorials/01_replogle.ipynb +507 -0
  51. pertrank-0.2.0rc2/tutorials/02_tcell.html +8418 -0
  52. pertrank-0.2.0rc2/tutorials/02_tcell.ipynb +644 -0
  53. pertrank-0.2.0rc2/validation/RELEASE_VALIDATION.json +75 -0
  54. pertrank-0.2.0rc2/validation/installed_identity.json +18 -0
  55. pertrank-0.2.0rc2/validation/requirements-resolved.txt +78 -0
  56. pertrank-0.2.0rc2/validation/source_tests.log +3 -0
  57. pertrank-0.2.0rc2/validation/wheel_tests.log +3 -0
@@ -0,0 +1,24 @@
1
+ # Changelog
2
+
3
+ ## 0.2.0rc2 — manuscript-aligned private review update
4
+
5
+ ### Scientific identity and documentation
6
+
7
+ - Align the package overview with manuscript v25.2: control geometry enters target reliability, and gene readout follows the biological query.
8
+ - Explain complete weighting versus the added geometry through the three matched returned arms.
9
+ - Distinguish reuse with an external effect estimator from transforming readout coordinates.
10
+ - Document the separate decision evaluator, including exact expected overlap for independently resolved boundary ties and per-query output.
11
+ - Update tutorial explanations while retaining their archived real-data inputs and computational examples.
12
+ - Replace inherited release/test assertions with a version-specific validation record; distinguish current manifests from initial-release provenance.
13
+ - Keep private distribution, public package publication and ecosystem acceptance explicitly separate.
14
+
15
+ ### Correctness and reusable evaluation
16
+
17
+ - Keep source reliability statistics distinct from external readout effects; preserve the original gene axis for chained estimator reuse.
18
+ - Detach copied statistics and metadata so modifying an external result cannot mutate its source result.
19
+ - Record implementation version and effect origin with generated results.
20
+ - Validate integer counts and configuration, finite scores, identifier alignment and archived gene-axis provenance consistently.
21
+ - Export `evaluate_reference` for fixed-pool P@K with identity-based self exclusion and exact expected boundary-tie overlap.
22
+ - Verify actual installed-wheel code and data against the release source; configure CI to build and test wheels.
23
+
24
+ The test suite increases from 61 to 99 checks. Both real-data tutorial input files and the numerical regression oracle remain unchanged. Verified checks and their scope are recorded in `docs/validation.md`.
@@ -0,0 +1,9 @@
1
+ # Software integration
2
+
3
+ This tree supplies PertRank `0.2.0rc2`, aligned with manuscript v25.2. It replaces the previous `0.2.0rc1` software source in the scientific reproduction bundle. The software version and the reproduction-bundle version identify different deliverables.
4
+
5
+ The scientific equations, fixed five-direction geometry, attenuation 0.9 and archived tutorial data are retained. Interface validation, external-effect provenance and documentation have been updated. The result's source reliability statistics stay distinct from the effects currently used to rank candidates. See [CHANGELOG](CHANGELOG.md) and [current validation](docs/validation.md).
6
+
7
+ `tests/reference_core_v21_1.py` remains a frozen regression oracle: its historical name records where the unchanged numerical engine originated, rather than the version of the current manuscript. Similarly, `provenance/ORIGINAL_RELEASE.json` and `ORIGINAL_SHA256SUMS.txt` identify the initial separate software release; they are historical provenance, not manifests for the current tree. Current release checksums identify the delivered version.
8
+
9
+ The integrated scientific archive carries this source once. Standalone wheel and source distributions may be supplied separately for installation. Tutorial fixtures and numerical scientific results are shared with the existing reproduction chain; no scientific model is fitted as part of this software revision.
@@ -0,0 +1,14 @@
1
+ Private review distribution
2
+
3
+ This anonymous release is supplied for installation, examination and
4
+ reproduction of the accompanying method during private review. Copyright
5
+ and other rights remain with their respective owners. No new public
6
+ open-source license is granted by this packaging step.
7
+
8
+ Before a public PyPI release or scverse submission, the authors must select
9
+ and apply an OSI-approved license to their software. Third-party datasets
10
+ retain their original terms and attribution; the tutorial provenance files
11
+ identify the sources. Public dataset redistribution must respect those terms.
12
+
13
+ This notice describes the current distribution status and does not replace
14
+ any existing license or rights attached to a third-party component.
@@ -0,0 +1,10 @@
1
+ include README.md READ_FIRST.zh.md LICENSE CHANGELOG.md INTEGRATION.md requirements-tested.txt
2
+ recursive-include docs *.md
3
+ recursive-include tutorials *.ipynb *.py *.md *.html
4
+ recursive-include tests *.py *.json
5
+ recursive-include scripts *.py *.md
6
+ recursive-include src/pertrank/data *.npz *.json *.csv *.md
7
+ exclude tutorials/outputs/*
8
+ recursive-include validation *.json *.txt *.log
9
+ recursive-include provenance *.json *.txt
10
+ global-exclude __pycache__ *.py[cod]
@@ -0,0 +1,156 @@
1
+ Metadata-Version: 2.4
2
+ Name: pertrank
3
+ Version: 0.2.0rc2
4
+ Summary: Reusable control-geometry reliability for gene-specific perturbation prioritization
5
+ License: Private review distribution; public license pending author selection
6
+ Classifier: Development Status :: 3 - Alpha
7
+ Classifier: Intended Audience :: Science/Research
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
10
+ Requires-Python: >=3.10
11
+ Description-Content-Type: text/markdown
12
+ License-File: LICENSE
13
+ Requires-Dist: numpy>=1.24
14
+ Requires-Dist: scipy<1.17,>=1.10
15
+ Requires-Dist: pandas>=1.5
16
+ Requires-Dist: anndata>=0.9
17
+ Provides-Extra: test
18
+ Requires-Dist: pytest>=7; extra == "test"
19
+ Requires-Dist: h5py>=3.8; extra == "test"
20
+ Provides-Extra: tutorial
21
+ Requires-Dist: jupyterlab>=4; extra == "tutorial"
22
+ Requires-Dist: matplotlib>=3.6; extra == "tutorial"
23
+ Dynamic: license-file
24
+
25
+ # PertRank
26
+
27
+ **Use control geometry to improve candidate selection while preserving the biological readout.** PertRank combines profile-wide effect and variance information with leading covariance directions from matched controls to calculate one reliability weight per perturbation. It applies that weight to the queried gene effect, producing separate candidate lists for increasing and decreasing the response.
28
+
29
+ This **0.2.0rc2 private review release** aligns the reusable software with manuscript v25.2. It provides an AnnData interface, reuse of fitted reliability with labeled external estimates, two real-data statistics tutorials, and regression checks against the frozen research implementation. The release is distributed for private review; public PyPI publication, scverse listing and pertpy acceptance are separate steps. The intended distribution and import name is `pertrank`.
30
+
31
+ ## What the software separates
32
+
33
+ | Step | Information used | Output |
34
+ |---|---|---|
35
+ | Estimate reliability | Matched controls and each target's own effect and sampling variation across the expression profile | One bounded weight per target |
36
+ | Read the response | Original gene effects, or a labeled upstream effect estimate such as FR-Perturb | A response-specific up/down score |
37
+ | Allocate shortlist positions | The same eligible candidates and a fixed list length | Candidate lists and exact boundary-tie inclusion probabilities |
38
+
39
+ The three returned methods make the contribution inspectable: **Unweighted** uses the effect directly, **Euclidean** uses target-level reliability without directional geometry, and **PertRank** adds the fixed control geometry. A complete-workflow comparison against Unweighted and a geometry comparison against Euclidean answer different questions. See [scientific design](docs/scientific_design.md) for the equations, estimator reuse and connection to the manuscript evidence.
40
+
41
+ ## Install the review release
42
+
43
+ Use Python 3.10 or later. From this directory:
44
+
45
+ ```bash
46
+ python -m pip install .
47
+ ```
48
+
49
+ To create a wheel from the source tree:
50
+
51
+ ```bash
52
+ python -m pip wheel --no-deps . --wheel-dir dist
53
+ ```
54
+
55
+ Dependencies are declared in `pyproject.toml`. For development and checks:
56
+
57
+ ```bash
58
+ python -m pip install '.[test]'
59
+ python -m pytest
60
+ python scripts/check_tutorials.py
61
+ ```
62
+
63
+ ## Rank candidates from AnnData
64
+
65
+ Start with quality-controlled input cells and their matched non-targeting controls. Keep planning cells and reference cells separate. Gene and cell identifiers must be unique, and perturbed genes must use the same identifier namespace as query genes.
66
+
67
+ ```python
68
+ import anndata as ad
69
+ import pertrank as pr
70
+
71
+ adata = ad.read_h5ad("screen_inputs.h5ad", backed="r")
72
+ result = pr.rank(
73
+ adata,
74
+ queries=["CTLA4", "IL2"],
75
+ input_scale="counts",
76
+ layer=None, # raw counts in X
77
+ full_gene_axis=True, # assert the full measured axis is present
78
+ strata_keys=["donor", "condition", "batch"],
79
+ target_key="target_ids",
80
+ target_gene_key="target_gene_ids",
81
+ control_key="is_control",
82
+ top_k=25,
83
+ )
84
+ adata.file.close()
85
+
86
+ result.nominations.to_csv("nominations.csv", index=False)
87
+ result.weights.to_csv("reliability_weights.csv", index=False)
88
+ result.all_candidate_scores.to_csv("all_candidate_scores.csv", index=False)
89
+ ```
90
+
91
+ The count path accepts one prespecified matching stratum per call. The example's donor, condition and batch columns must therefore each have one value in the supplied object. Split the screen into its prespecified matching strata before calling this path. `strata_keys=[]` explicitly declares that the supplied cells belong to one stratum; use it only when the experimental design supports that declaration.
92
+
93
+ All supplied input cells contribute to the fit. The package neither selects queries nor chooses a favorable cell subset. The paper's shallow examples use 20 cells per target and 2,000 controls; the numerical minimum is a different concept. See [input contracts](docs/input_contracts.md) for scale, control matching and memory behavior.
94
+
95
+ ## Read the outputs
96
+
97
+ | Output | Meaning |
98
+ |---|---|
99
+ | `nominations` | Deterministic top-k lists for up/down responses and the Unweighted, Euclidean and PertRank methods |
100
+ | `all_candidate_scores` | Eligible query–target scores, ranks and boundary-tie inclusion probabilities |
101
+ | `weights` | Target-level reliability weights and input counts |
102
+ | `effects` | Target-by-gene original-coordinate effects |
103
+ | `controls`, `statistics`, `metadata` | Control models, sufficient statistics and computation provenance |
104
+
105
+ The up score is `effect × weight`; the down score reverses its sign. The perturbed gene is excluded from its own query. A zero weight retains the candidate at a zero score. Tie inclusion probabilities describe random selection among equally scored candidates at the cutoff; they are not probabilities of biological correctness.
106
+
107
+ ## Reuse reliability with an existing effect estimate
108
+
109
+ An external effect matrix must have unique target and gene labels, the same target population, and compatible effect orientation. Align it by labels rather than by row order.
110
+
111
+ ```python
112
+ import pandas as pd
113
+
114
+ external = pd.read_csv("external_effects.csv", index_col=0)
115
+ external_result = result.rank_effects(
116
+ external, queries=["CTLA4", "IL2"], top_k=25
117
+ )
118
+ external_result.nominations.to_csv("external_nominations.csv", index=False)
119
+ ```
120
+
121
+ This holds the fitted reliability calculation fixed while replacing the effect estimator. The returned `Unweighted`, `Euclidean` and `PertRank` arms then all use the supplied external effects, so the geometry comparison remains matched. The operation neither fits the upstream estimator nor re-estimates its uncertainty. Record that estimator's configuration and input design alongside the exported ranking.
122
+
123
+ ## Evaluate a fixed shortlist against a separate reference
124
+
125
+ ```python
126
+ reference = pd.read_csv("reference_effects.csv", index_col=0)
127
+ scores = pr.evaluate_reference(result, reference)
128
+ scores.to_csv("reference_precision.csv", index=False)
129
+ ```
130
+
131
+ Provide a target-by-gene reference effect table with the same target population and every queried gene. The evaluator returns precision and expected overlap for each method, query and direction, using the result's fixed list length and self-target exclusions. It accounts for boundary ties in both the predicted and reference lists. The reference is used only for scoring. Define its construction independently of the fitted ranking; biological interpretation depends on that design. See [API reference](docs/api.md#evaluate_reference) for the exact scoring rule. Query-level uncertainty and manuscript-specific aggregation remain in the full scientific reproduction bundle.
132
+
133
+ ## Try the interface immediately
134
+
135
+ ```bash
136
+ python tutorials/00_anndata_quickstart.py
137
+ ```
138
+
139
+ This quickstart creates a small synthetic AnnData solely for a software
140
+ check. The two atlas notebooks instead use authentic archived statistics.
141
+
142
+ ## Learn and reproduce
143
+
144
+ - [API reference](docs/api.md): public functions and result fields.
145
+ - [Scientific design](docs/scientific_design.md): reliability, readout and the interpretation of matched comparisons.
146
+ - [Input contracts](docs/input_contracts.md): counts, normalized inputs and matching strata.
147
+ - [Replogle tutorial](tutorials/01_replogle.ipynb) and [T-cell tutorial](tutorials/02_tcell.ipynb): worked examples using archived real-data sufficient statistics. The tutorials identify the source and processing level of each fixture. They replay candidate ranking from saved statistics; the AnnData example above is the cell-input route.
148
+ - `tests/`: checks of the public interface and numerical agreement with the frozen implementation.
149
+ - [Community integration proposal](docs/community_integration.md): prepared scverse registration and pertpy contribution scope.
150
+ - [Private release and public handoff](docs/release_process.md): installation, artifact verification and the remaining public-release metadata.
151
+
152
+ The scientific use is candidate prioritization for follow-up experiments. Weights describe estimated profile reliability, not calibrated probabilities of biological validation. The manuscript evaluates decision quality with fixed-length lists, matched exclusions and reference-supported precision; its independent RNA comparison supports the complete workflow, while matched Euclidean contrasts isolate the added geometry.
153
+
154
+ ## Attribution and release status
155
+
156
+ The computational equations and fixed parameters (`k=5`, `beta=0.9`) are inherited from the manuscript implementation. This release updates software contracts and documentation without fitting new scientific models or selecting new parameters. The accompanying manuscript and source manifests provide method and dataset attribution. Dataset rights remain with their original sources. Public redistribution and ecosystem submission require the rights holder's software-license decision; see [release process](docs/release_process.md).
@@ -0,0 +1,132 @@
1
+ # PertRank
2
+
3
+ **Use control geometry to improve candidate selection while preserving the biological readout.** PertRank combines profile-wide effect and variance information with leading covariance directions from matched controls to calculate one reliability weight per perturbation. It applies that weight to the queried gene effect, producing separate candidate lists for increasing and decreasing the response.
4
+
5
+ This **0.2.0rc2 private review release** aligns the reusable software with manuscript v25.2. It provides an AnnData interface, reuse of fitted reliability with labeled external estimates, two real-data statistics tutorials, and regression checks against the frozen research implementation. The release is distributed for private review; public PyPI publication, scverse listing and pertpy acceptance are separate steps. The intended distribution and import name is `pertrank`.
6
+
7
+ ## What the software separates
8
+
9
+ | Step | Information used | Output |
10
+ |---|---|---|
11
+ | Estimate reliability | Matched controls and each target's own effect and sampling variation across the expression profile | One bounded weight per target |
12
+ | Read the response | Original gene effects, or a labeled upstream effect estimate such as FR-Perturb | A response-specific up/down score |
13
+ | Allocate shortlist positions | The same eligible candidates and a fixed list length | Candidate lists and exact boundary-tie inclusion probabilities |
14
+
15
+ The three returned methods make the contribution inspectable: **Unweighted** uses the effect directly, **Euclidean** uses target-level reliability without directional geometry, and **PertRank** adds the fixed control geometry. A complete-workflow comparison against Unweighted and a geometry comparison against Euclidean answer different questions. See [scientific design](docs/scientific_design.md) for the equations, estimator reuse and connection to the manuscript evidence.
16
+
17
+ ## Install the review release
18
+
19
+ Use Python 3.10 or later. From this directory:
20
+
21
+ ```bash
22
+ python -m pip install .
23
+ ```
24
+
25
+ To create a wheel from the source tree:
26
+
27
+ ```bash
28
+ python -m pip wheel --no-deps . --wheel-dir dist
29
+ ```
30
+
31
+ Dependencies are declared in `pyproject.toml`. For development and checks:
32
+
33
+ ```bash
34
+ python -m pip install '.[test]'
35
+ python -m pytest
36
+ python scripts/check_tutorials.py
37
+ ```
38
+
39
+ ## Rank candidates from AnnData
40
+
41
+ Start with quality-controlled input cells and their matched non-targeting controls. Keep planning cells and reference cells separate. Gene and cell identifiers must be unique, and perturbed genes must use the same identifier namespace as query genes.
42
+
43
+ ```python
44
+ import anndata as ad
45
+ import pertrank as pr
46
+
47
+ adata = ad.read_h5ad("screen_inputs.h5ad", backed="r")
48
+ result = pr.rank(
49
+ adata,
50
+ queries=["CTLA4", "IL2"],
51
+ input_scale="counts",
52
+ layer=None, # raw counts in X
53
+ full_gene_axis=True, # assert the full measured axis is present
54
+ strata_keys=["donor", "condition", "batch"],
55
+ target_key="target_ids",
56
+ target_gene_key="target_gene_ids",
57
+ control_key="is_control",
58
+ top_k=25,
59
+ )
60
+ adata.file.close()
61
+
62
+ result.nominations.to_csv("nominations.csv", index=False)
63
+ result.weights.to_csv("reliability_weights.csv", index=False)
64
+ result.all_candidate_scores.to_csv("all_candidate_scores.csv", index=False)
65
+ ```
66
+
67
+ The count path accepts one prespecified matching stratum per call. The example's donor, condition and batch columns must therefore each have one value in the supplied object. Split the screen into its prespecified matching strata before calling this path. `strata_keys=[]` explicitly declares that the supplied cells belong to one stratum; use it only when the experimental design supports that declaration.
68
+
69
+ All supplied input cells contribute to the fit. The package neither selects queries nor chooses a favorable cell subset. The paper's shallow examples use 20 cells per target and 2,000 controls; the numerical minimum is a different concept. See [input contracts](docs/input_contracts.md) for scale, control matching and memory behavior.
70
+
71
+ ## Read the outputs
72
+
73
+ | Output | Meaning |
74
+ |---|---|
75
+ | `nominations` | Deterministic top-k lists for up/down responses and the Unweighted, Euclidean and PertRank methods |
76
+ | `all_candidate_scores` | Eligible query–target scores, ranks and boundary-tie inclusion probabilities |
77
+ | `weights` | Target-level reliability weights and input counts |
78
+ | `effects` | Target-by-gene original-coordinate effects |
79
+ | `controls`, `statistics`, `metadata` | Control models, sufficient statistics and computation provenance |
80
+
81
+ The up score is `effect × weight`; the down score reverses its sign. The perturbed gene is excluded from its own query. A zero weight retains the candidate at a zero score. Tie inclusion probabilities describe random selection among equally scored candidates at the cutoff; they are not probabilities of biological correctness.
82
+
83
+ ## Reuse reliability with an existing effect estimate
84
+
85
+ An external effect matrix must have unique target and gene labels, the same target population, and compatible effect orientation. Align it by labels rather than by row order.
86
+
87
+ ```python
88
+ import pandas as pd
89
+
90
+ external = pd.read_csv("external_effects.csv", index_col=0)
91
+ external_result = result.rank_effects(
92
+ external, queries=["CTLA4", "IL2"], top_k=25
93
+ )
94
+ external_result.nominations.to_csv("external_nominations.csv", index=False)
95
+ ```
96
+
97
+ This holds the fitted reliability calculation fixed while replacing the effect estimator. The returned `Unweighted`, `Euclidean` and `PertRank` arms then all use the supplied external effects, so the geometry comparison remains matched. The operation neither fits the upstream estimator nor re-estimates its uncertainty. Record that estimator's configuration and input design alongside the exported ranking.
98
+
99
+ ## Evaluate a fixed shortlist against a separate reference
100
+
101
+ ```python
102
+ reference = pd.read_csv("reference_effects.csv", index_col=0)
103
+ scores = pr.evaluate_reference(result, reference)
104
+ scores.to_csv("reference_precision.csv", index=False)
105
+ ```
106
+
107
+ Provide a target-by-gene reference effect table with the same target population and every queried gene. The evaluator returns precision and expected overlap for each method, query and direction, using the result's fixed list length and self-target exclusions. It accounts for boundary ties in both the predicted and reference lists. The reference is used only for scoring. Define its construction independently of the fitted ranking; biological interpretation depends on that design. See [API reference](docs/api.md#evaluate_reference) for the exact scoring rule. Query-level uncertainty and manuscript-specific aggregation remain in the full scientific reproduction bundle.
108
+
109
+ ## Try the interface immediately
110
+
111
+ ```bash
112
+ python tutorials/00_anndata_quickstart.py
113
+ ```
114
+
115
+ This quickstart creates a small synthetic AnnData solely for a software
116
+ check. The two atlas notebooks instead use authentic archived statistics.
117
+
118
+ ## Learn and reproduce
119
+
120
+ - [API reference](docs/api.md): public functions and result fields.
121
+ - [Scientific design](docs/scientific_design.md): reliability, readout and the interpretation of matched comparisons.
122
+ - [Input contracts](docs/input_contracts.md): counts, normalized inputs and matching strata.
123
+ - [Replogle tutorial](tutorials/01_replogle.ipynb) and [T-cell tutorial](tutorials/02_tcell.ipynb): worked examples using archived real-data sufficient statistics. The tutorials identify the source and processing level of each fixture. They replay candidate ranking from saved statistics; the AnnData example above is the cell-input route.
124
+ - `tests/`: checks of the public interface and numerical agreement with the frozen implementation.
125
+ - [Community integration proposal](docs/community_integration.md): prepared scverse registration and pertpy contribution scope.
126
+ - [Private release and public handoff](docs/release_process.md): installation, artifact verification and the remaining public-release metadata.
127
+
128
+ The scientific use is candidate prioritization for follow-up experiments. Weights describe estimated profile reliability, not calibrated probabilities of biological validation. The manuscript evaluates decision quality with fixed-length lists, matched exclusions and reference-supported precision; its independent RNA comparison supports the complete workflow, while matched Euclidean contrasts isolate the added geometry.
129
+
130
+ ## Attribution and release status
131
+
132
+ The computational equations and fixed parameters (`k=5`, `beta=0.9`) are inherited from the manuscript implementation. This release updates software contracts and documentation without fitting new scientific models or selecting new parameters. The accompanying manuscript and source manifests provide method and dataset attribution. Dataset rights remain with their original sources. Public redistribution and ecosystem submission require the rights holder's software-license decision; see [release process](docs/release_process.md).
@@ -0,0 +1,57 @@
1
+ # PertRank 0.2.0rc2 使用说明
2
+
3
+ 本版将可安装软件与 v25.2 论文的核心设计对齐:**把表达谱层面的对照信息用于扰动可靠性评估,保留与生物学问题一致的基因读出,并把同一可靠性计算复用到外部效应估计。** 软件仍使用固定的五个对照方向和 `beta=0.9`。
4
+
5
+ 当前为匿名私有评审版本。公开 PyPI 发布、scverse 登记和 pertpy 合并是独立的后续步骤;本地可安装包不代表已被这些平台收录。
6
+
7
+ ## 安装与运行
8
+
9
+ 在源代码目录运行:
10
+
11
+ ```bash
12
+ python -m pip install .
13
+ python tutorials/00_anndata_quickstart.py
14
+ ```
15
+
16
+ 如需从源代码构建 wheel:
17
+
18
+ ```bash
19
+ python -m pip wheel --no-deps . --wheel-dir dist
20
+ ```
21
+
22
+ `00_anndata_quickstart.py` 用合成数据演示 AnnData 到候选表的软件流程。自己的数据须明确输入尺度、完整基因轴、靶点身份和匹配对照,具体见 [README](README.md) 和 [输入要求](docs/input_contracts.md)。
23
+
24
+ ## 实际用途
25
+
26
+ 1. **从浅层细胞输入提名候选:** `pr.rank(...)` 计算可靠性、原始效应及每个查询的上调/下调名单。
27
+ 2. **为已有估计器复用可靠性:** `result.rank_effects(...)` 接收带靶点和基因标签的外部效应矩阵,保持原可靠性计算固定,重新生成候选名单。
28
+ 3. **对照独立参考评价名单:** `pr.evaluate_reference(...)` 在相同候选、查询、名单长度和自身靶点排除规则下,计算逐查询、逐方向的精确并列期望重合数及 precision。参考仅用于评分,区间和跨来源汇总由论文完整复现链处理。
29
+
30
+ 两个入口都返回 Unweighted、Euclidean 和 PertRank 三臂。PertRank 对 Unweighted 衡量完整加权流程;PertRank 对 Euclidean 才隔离新增控制几何。外部估计器复用后,这三臂均使用同一外部效应。权重是轮廓可靠性的估计,不是生物学验证概率。
31
+
32
+ 外部复用结果中的 `effects` 是当前排名所用效应,`statistics` 始终保留原始可靠性输入,避免把外部效应和原始能量混成一套统计量。结果元数据记录软件版本、效应来源、基因轴与计算配置。
33
+
34
+ ## 两篇真实数据教程
35
+
36
+ | 教程 | 归档内容 | 提供形式 |
37
+ |---|---|---|
38
+ | Replogle K562 | 全部 2,000 个候选、300 个查询;可靠性统计量来自 8,202 个基因 | `tutorials/01_replogle.ipynb` 及 HTML |
39
+ | T 细胞 D3 Rest | 全部 52 个候选、293 个查询;可靠性统计量来自 13,390 个基因,附已存 FR 输出 | `tutorials/02_tcell.ipynb` 及 HTML |
40
+
41
+ 教程离线读取真实的已存充分统计量,重新计算权重和候选表;保留完整候选池,演示查询按 ID 字典序选取。每套数据均附来源、哈希和处理层级。这是统计量到排名的复现,逐细胞数据入口由 AnnData 示例和软件测试单独覆盖。
42
+
43
+ ```bash
44
+ python scripts/check_tutorials.py
45
+ ```
46
+
47
+ 该命令运行两篇教程的所有代码单元。Jupyter 可交互查看 notebook,HTML 供直接阅读,候选表导出到 `outputs/`。
48
+
49
+ ## 验证与完整复现
50
+
51
+ 本版的实际检查范围、运行环境及结果见 [验证记录](docs/validation.md)。软件检查覆盖输入契约、存储格式、匹配对照、ID 对齐、并列排名、自身靶点排除、外部效应复用和冻结公式一致性。`requirements-tested.txt` 记录本版核验环境。
52
+
53
+ 软件教程负责使用与数值回归;论文总代码包另含参考构建、性能评分和配对区间等科学复现链。本轮软件修订不增加生物实验或科学性能比较。
54
+
55
+ ## 后续社区接入
56
+
57
+ [社区接入说明](docs/community_integration.md) 保留了 scverse 登记及 pertpy 贡献的准备材料。公开阶段须补齐实际维护者、仓库、作者选定的许可与引用信息,再完成发布和社区审核。
@@ -0,0 +1,165 @@
1
+ # API reference
2
+
3
+ Import the public interface with `import pertrank as pr`.
4
+
5
+ ## `rank`
6
+
7
+ ```python
8
+ pr.rank(
9
+ adata,
10
+ queries,
11
+ *,
12
+ input_scale,
13
+ strata_keys,
14
+ target_key="target_ids",
15
+ target_gene_key="target_gene_ids",
16
+ control_key="is_control",
17
+ layer=None,
18
+ top_k=25,
19
+ seed=2026,
20
+ batch_rows=512,
21
+ full_gene_axis=False,
22
+ library_totals_key=None,
23
+ )
24
+ ```
25
+
26
+ Compute reliability weights and query-specific candidate lists from input cells.
27
+
28
+ | Argument | Contract |
29
+ |---|---|
30
+ | `adata` | AnnData with unique observation and variable names; only prespecified input cells and matched input controls |
31
+ | `queries` | Unique response-gene identifiers on the input gene axis |
32
+ | `input_scale` | Required: `"counts"` for raw integer counts or `"log"` for the declared, already normalized log-expression coordinate system |
33
+ | `strata_keys` | Required observation column names defining matched controls; `[]` explicitly declares one common stratum |
34
+ | `target_key` | Observation column identifying perturbations; control labels and perturbation labels must be disjoint |
35
+ | `target_gene_key` | Observation column naming each perturbed gene in the `var_names` namespace; controls use a nonempty placeholder |
36
+ | `control_key` | Observation column containing Boolean or 0/1 input-control indicators |
37
+ | `layer` | Named matrix layer, or `None` to read `X` |
38
+ | `top_k` | Positive integer, no larger than each query's candidate pool after self-target exclusion |
39
+ | `seed` | Integer controlling reproducible hash-based tie selection; it does not select cells or fit parameters |
40
+ | `batch_rows` | Positive read-chunk size for the count-input path |
41
+ | `full_gene_axis` | Required `True` assertion for count input so library totals precede gene filtering |
42
+ | `library_totals_key` | Counts-only observation column containing retained full-library totals; if supplied, its values must exactly equal the supplied raw matrix row sums |
43
+
44
+ The geometry parameters are fixed at five directions and attenuation 0.9. Returns `PertRankResult`. See [input contracts](input_contracts.md) for single-stratum counts, multiple-stratum normalized inputs, control matching and memory behavior.
45
+
46
+ ## `PertRankResult`
47
+
48
+ | Attribute | Type | Contents |
49
+ |---|---|---|
50
+ | `nominations` | pandas DataFrame | Delivered query/method/direction top-k lists, scores and tie information |
51
+ | `all_candidate_scores` | pandas DataFrame | Eligible candidate scores and rank information |
52
+ | `weights` | pandas DataFrame | Target-level weights and input-cell counts |
53
+ | `effects` | pandas DataFrame | Effects currently used for ranking, indexed by target and gene identifiers |
54
+ | `controls` | mapping | Fitted control model for each matching stratum |
55
+ | `statistics` | mapping | Original source-cell effects, energies, variance contributions and labeled axes underlying the reliability fit |
56
+ | `metadata` | mapping | Software version, effect origin, scale, fixed parameters, axes and run configuration |
57
+
58
+ The result is separate from the input AnnData. Inspect or export the returned DataFrames; no persistent output file is written automatically by the ranking operation. `effect_origin` identifies source or external effects, and `pertrank_version` records the generating software. Following external-effect reuse, `effects` and `statistics["effects"]` serve different roles: the former is the current ranking readout, and the latter remains the original reliability input.
59
+
60
+ ## `PertRankResult.rank_effects`
61
+
62
+ ```python
63
+ external_result = result.rank_effects(
64
+ external_effects,
65
+ queries=None,
66
+ top_k=None,
67
+ seed=None,
68
+ )
69
+ ```
70
+
71
+ Apply the fitted target weights to a labeled external effect matrix. `external_effects` is a pandas DataFrame indexed by target identifiers with gene identifiers as columns. Label matching is checked, and row order is reconciled by label. `None` uses the original result's query list, shortlist size or tie seed. Returns a result with new effect-based rankings and the existing reliability weights.
72
+
73
+ The external matrix must contain every fitted target exactly once. Its columns may be a subset of the retained source-effect gene axis, provided they contain every requested query. The external estimator must use a compatible input population and effect orientation. The operation neither estimates external model parameters nor learns new weights. This method supplies a direct route for using PertRank reliability with another effect estimator. The equivalent top-level call is `pr.rank_effects(result, external_effects, queries=None, top_k=None, seed=None)`.
74
+
75
+ The external result preserves independent copies of the source `weights`, `controls` and `statistics`. Its `effects`, `nominations` and `all_candidate_scores` use the new estimator. Reuse can be chained: a query-only external matrix does not discard the original source axis for a later external estimate. Calling `from_statistics(external_result.statistics, ...)` replays the original source effects; use `external_result.effects` when exporting the external estimator's ranking inputs. For compact fixtures, the stored source-effect axis can be smaller than the original weight axis, whose size remains recorded separately.
76
+
77
+ ## `from_statistics`
78
+
79
+ ```python
80
+ result = pr.from_statistics(
81
+ statistics_mapping,
82
+ queries,
83
+ top_k=25,
84
+ seed=2026,
85
+ effect_gene_subset=False,
86
+ weight_gene_count=None,
87
+ )
88
+ ```
89
+
90
+ Replay reliability and ranking from an archived input-statistics mapping. Let `T` be the number of targets and `G` the number of genes on the full original weight axis.
91
+
92
+ | Key | Shape | Meaning |
93
+ |---|---|---|
94
+ | `gene_ids` | `(G,)` | Unique gene identifiers |
95
+ | `target_ids` | `(T,)` | Unique perturbation identifiers |
96
+ | `target_gene_ids` | `(T,)` | Perturbed-gene identifiers in the same gene namespace |
97
+ | `effects` | `(T, G)` | Original-coordinate target-minus-control effects |
98
+ | `energy` | `(T,)` | Sum of squared effects across the full gene axis |
99
+ | `noise_trace` | `(T,)` | Total variance contribution to the estimated effect |
100
+ | `direction_energy` | `(T, 5)` | Squared effect projections on the five fitted control directions |
101
+ | `direction_noise` | `(T, 5)` | Variance contributions on those directions |
102
+ | `n_cells` | `(T,)`, optional | Recorded integer target-cell counts, each at least two |
103
+ | `batch_ids` | `(T,)`, optional | Recorded matching-stratum identifiers |
104
+ | `control_models` | mapping, optional | Archived control model metadata |
105
+
106
+ By default, these arrays retain the full original gene axis, and `energy` is checked against the sum of squared `effects`. A compact fixture may instead save only query-gene effect columns while preserving the previously computed full-profile energy, noise and projected statistics. Declare that representation explicitly with `effect_gene_subset=True` and supply the original `weight_gene_count`. The implementation records the original weight-axis size separately from the retained effect-axis size and checks that the subset's squared-effect energy does not exceed the archived full-profile energy. It never reconstructs full-profile reliability statistics from the query subset.
107
+
108
+ Reliability weights are recomputed from those sufficient statistics rather than taken from a saved weight array. Tutorial fixture provenance identifies where the statistics were originally computed.
109
+
110
+ This entry point supports compact real-data demonstrations and numerical regression. It does not reconstruct missing cells or verify upstream normalization from statistics alone.
111
+
112
+ ## `evaluate_reference`
113
+
114
+ ```python
115
+ scores = pr.evaluate_reference(result, reference_effects)
116
+ ```
117
+
118
+ Evaluate a `PertRankResult` against a separate, labeled target-by-gene reference effect DataFrame. The reference must contain every fitted target exactly once and all requested query genes; additional reference genes are allowed. Values must be finite, identifiers unique, and response signs compatible with the ranking input. Rows are matched by identifier.
119
+
120
+ The operation holds the result's query set, top-k, target pool and own-target exclusion fixed. It returns one row per method, query and direction, with `precision_at_k`, `expected_overlap`, `top_k` and `n_candidates`.
121
+
122
+ For each eligible candidate `j`, let `p_j` and `q_j` denote expected inclusion in the predicted and reference top-k lists under uniform boundary-tie selection. The score is
123
+
124
+ ```text
125
+ expected_overlap = sum_j p_j q_j
126
+ precision_at_k = expected_overlap / top_k
127
+ ```
128
+
129
+ This expectation uses independent tie selection in the two lists. It does not compare one arbitrary deterministic realization of each list. A fully tied reference remains uninformative even if a deterministic sort would return the same candidate names in both lists.
130
+
131
+ The evaluator does not fit weights or alter nominations. Reference independence and its experimental interpretation must be established by the study design, rather than inferred from a DataFrame. No confidence intervals, source pooling or biological accuracy probabilities are produced. The method names identify weighting rules: for a result returned by `rank_effects`, `Unweighted` means the supplied external estimator without weights.
132
+
133
+ ## Advanced array interface
134
+
135
+ The original low-level functions remain available for compatibility with the frozen research implementation. The AnnData interface above is the recommended entry point for a new screen.
136
+
137
+ | Function | Inputs and return |
138
+ |---|---|
139
+ | `fit_cells(X, gene_ids, target_ids, batch_ids, is_control, target_gene_ids)` | Finite dense cell-by-gene normalized expression with aligned cell metadata; returns an effects/statistics/control-model dictionary and fixed-method weights |
140
+ | `normalize_counts(counts, library_totals, batch_ids, is_control)` | Nonnegative counts, externally supplied positive full-library totals and aligned normalization strata; returns `(log_expression, scales_by_stratum)` using median matched-control totals |
141
+ | `reliability_weights(energy, noise_trace, direction_energy, direction_noise, beta=0.9, k=5)` | Nonnegative target energies/noise and target-by-direction statistics; returns the three method weight arrays. Optional algebraic parameters support numerical verification; the high-level fitting interface fixes the manuscript values |
142
+ | `nominate(result, query_ids, top_k=25, seed=2026)` | A low-level fit dictionary and query IDs; returns `(records, inclusion_probabilities)`. Probability-array axes are method, query, direction and target |
143
+
144
+ The array normalization function can accept a selected gene axis when original full-library totals are supplied; the AnnData raw-count path deliberately requires the complete measured axis and computes those totals itself. The low-level functions do not infer cell roles, choose queries or access an evaluation reference.
145
+
146
+ ## Scores and ties
147
+
148
+ For target `t` and query `g`, the up score is `w[t] * effect[t, g]`; the down score is its negative. The Unweighted method sets all weights to one. Euclidean and PertRank use the same positive-part reliability rule with their respective energies and noise estimates. PertRank attenuates the fixed control directions in the reliability calculation while preserving the supplied gene-coordinate effect for ranking.
149
+
150
+ For a boundary tie of size `m` with `r` remaining slots, each tied candidate has expected inclusion `r / m`. The deterministic delivered list breaks that tie reproducibly. Neither number is a biological validation probability. A query's own gene perturbation is excluded consistently from every method.
151
+
152
+ ## Offline tutorial data
153
+
154
+ `from pertrank.datasets import load_example, software_example`
155
+
156
+ - `load_example("replogle")` returns `(statistics, provenance)` for the complete
157
+ archived 2,000-target task; 300 original-coordinate query columns are kept.
158
+ - `load_example("tcell")` returns the complete archived 52-target task with
159
+ 293 query columns and the saved FR effect matrix.
160
+ - `software_example(seed=2026)` creates a 280-cell, 60-gene synthetic AnnData
161
+ for interface checks only.
162
+
163
+ Both real fixtures verify their own content hash and work without network
164
+ access. Their full geometry axes contain 8,202 and 13,390 genes respectively.
165
+ Use the explicit `effect_gene_subset=True` route with those counts.
@@ -0,0 +1,56 @@
1
+ # Community integration proposal
2
+
3
+ Status: prepared for author review; no issue, pull request, registry entry or public release has been submitted.
4
+
5
+ ## Proposed place in the workflow
6
+
7
+ PertRank addresses candidate prioritization in shallow Perturb-seq: given a response gene, nominate perturbations for a fixed number of follow-up positions. It combines profile-wide effect and variance information with matched control directions to assess target reliability, retaining a gene-aligned readout. The same weights can be applied to an externally estimated effect matrix. This complements screen annotation, differential-expression analysis and global perturbation-distance tools.
8
+
9
+ The standalone package is the initial delivery. A future pertpy contribution can present the public operation through the namespace selected by its maintainers while retaining the same input checks and numerical tests. This package does not alter `pertpy.tl` or install a monkey patch.
10
+
11
+ ## Proposed upstream discussion text
12
+
13
+ **Title:** Add query-specific perturbation prioritization with PertRank
14
+
15
+ PertRank provides target-level reliability weighting for selecting perturbations associated with a specified response gene. The proposed feature accepts AnnData with explicit perturbation, perturbed-gene, matched-control and stratum metadata, and returns gene-specific up/down candidate tables. It also supports reusing fitted weights with external effect estimates.
16
+
17
+ The implementation uses the manuscript's fixed five-direction geometry and attenuation 0.9. It supplies numerical regression tests against the frozen research engine, checks count normalization before gene filtering, and makes control matching explicit. Replogle and T-cell examples replay the existing real-data analysis statistics, with the processing level of the compact fixtures disclosed.
18
+
19
+ The proposed contribution is an analysis operation and its documentation, rather than a new data-download framework. Maintainer guidance is requested on the preferred API shape, dependency boundary and return conventions before opening an implementation PR. The standalone package can remain independently maintained if that is the preferred integration model.
20
+
21
+ ## Deliverables to attach to an implementation PR
22
+
23
+ 1. A small API implementation using existing AnnData conventions and explicit scale/matching arguments.
24
+ 2. A documented result object and exportable candidate tables.
25
+ 3. Regression tests covering weights, scores, target alignment, self-target exclusion, ties and supported input storage types.
26
+ 4. Two reproducible tutorials and fixture provenance; tutorial data contain no evaluation inputs used to fit weights.
27
+ 5. API documentation and a concise explanation of the scientific task.
28
+ 6. A rights-holder-approved software license and completed attribution.
29
+
30
+ The pertpy contribution guide calls for corresponding tests and documentation and uses its own development, formatting and CI procedures. Before preparing the actual PR, apply the then-current repository checks. Source: [pertpy contributing guide](https://pertpy.scverse.org/en/latest/contributing.html).
31
+
32
+ ## scverse ecosystem registration specification
33
+
34
+ The public handoff should verify licensing, versioned registry installation, automated tests, API documentation, scverse data structures and maintainer agreement against the registry's current checklist. Ecosystem listing and scientific evaluation are separate matters. The proposed metadata below is a preparation checklist; the live schema and requirements must be checked when submitting. Source: [official ecosystem registry](https://github.com/scverse/ecosystem-packages).
35
+
36
+ Proposed descriptive content:
37
+
38
+ | Field | Proposed value or required evidence |
39
+ |---|---|
40
+ | Display name | PertRank |
41
+ | Description | Candidate prioritization using control geometry for reliability with gene-aligned effect readout |
42
+ | Primary category | `scRNA-seq` |
43
+ | Tags | `perturbation`, `denoising` |
44
+ | Language | `Python` |
45
+ | Data-structure use | AnnData inputs, aligned cell/gene metadata, labeled effect matrices and exportable ranking tables |
46
+ | Code, documentation and tutorial URLs | Actual public URLs after the approved release |
47
+ | Registry installation | Verified package name and published version |
48
+ | Maintainers | Actual responsible maintainer identity and contact |
49
+ | License | Rights-holder-approved OSI license |
50
+ | Inventory | `null` for README-only documentation; actual inventory URL if a compatible documentation site is built |
51
+
52
+ This is a submission specification, not a validated `meta.yaml`: unresolved identity and URL fields have deliberately not been replaced with invented values. At submission time, validate against the live registry schema and controlled vocabulary.
53
+
54
+ ## Public handoff sequence
55
+
56
+ Finalize rights, attribution and maintainership; release the tested standalone package; verify installation from the selected registry; submit the ecosystem registration; then discuss the scoped pertpy contribution. These are separate deliverables, each with its own acceptance decision. The current private review package establishes the installable and testable implementation before those public actions.