phenotopo 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- phenotopo-0.5.0/.github/workflows/tests.yml +21 -0
- phenotopo-0.5.0/.gitignore +23 -0
- phenotopo-0.5.0/CHANGELOG.md +99 -0
- phenotopo-0.5.0/CITATION.cff +21 -0
- phenotopo-0.5.0/LICENSE +21 -0
- phenotopo-0.5.0/PKG-INFO +416 -0
- phenotopo-0.5.0/README.md +356 -0
- phenotopo-0.5.0/examples/figures/connectivity.png +0 -0
- phenotopo-0.5.0/examples/figures/connectivity_heatmap.png +0 -0
- phenotopo-0.5.0/examples/figures/density.png +0 -0
- phenotopo-0.5.0/examples/figures/explain.png +0 -0
- phenotopo-0.5.0/examples/figures/hyperbolic.png +0 -0
- phenotopo-0.5.0/examples/figures/mapper.png +0 -0
- phenotopo-0.5.0/examples/figures/qc.png +0 -0
- phenotopo-0.5.0/examples/figures/small_multiples.png +0 -0
- phenotopo-0.5.0/examples/quickstart.py +124 -0
- phenotopo-0.5.0/pyproject.toml +49 -0
- phenotopo-0.5.0/src/phenotopo/__init__.py +61 -0
- phenotopo-0.5.0/src/phenotopo/cli.py +155 -0
- phenotopo-0.5.0/src/phenotopo/cohort.py +409 -0
- phenotopo-0.5.0/src/phenotopo/data.py +142 -0
- phenotopo-0.5.0/src/phenotopo/explain.py +231 -0
- phenotopo-0.5.0/src/phenotopo/graph.py +95 -0
- phenotopo-0.5.0/src/phenotopo/hyperbolic.py +132 -0
- phenotopo-0.5.0/src/phenotopo/layout.py +246 -0
- phenotopo-0.5.0/src/phenotopo/mapper.py +72 -0
- phenotopo-0.5.0/src/phenotopo/outliers.py +133 -0
- phenotopo-0.5.0/src/phenotopo/qc.py +254 -0
- phenotopo-0.5.0/src/phenotopo/report.py +198 -0
- phenotopo-0.5.0/src/phenotopo/robustness.py +167 -0
- phenotopo-0.5.0/src/phenotopo/stats.py +143 -0
- phenotopo-0.5.0/tests/test_cli.py +69 -0
- phenotopo-0.5.0/tests/test_cohort.py +101 -0
- phenotopo-0.5.0/tests/test_explain.py +98 -0
- phenotopo-0.5.0/tests/test_graph.py +49 -0
- phenotopo-0.5.0/tests/test_hyperbolic.py +41 -0
- phenotopo-0.5.0/tests/test_layout.py +15 -0
- phenotopo-0.5.0/tests/test_mapper.py +18 -0
- phenotopo-0.5.0/tests/test_outliers.py +56 -0
- phenotopo-0.5.0/tests/test_qc.py +58 -0
- phenotopo-0.5.0/tests/test_report.py +29 -0
- phenotopo-0.5.0/tests/test_robustness.py +38 -0
- phenotopo-0.5.0/tests/test_stats.py +52 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
name: tests
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
|
|
8
|
+
jobs:
|
|
9
|
+
test:
|
|
10
|
+
runs-on: ubuntu-latest
|
|
11
|
+
strategy:
|
|
12
|
+
matrix:
|
|
13
|
+
python-version: ["3.10", "3.12"]
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
- uses: actions/setup-python@v5
|
|
17
|
+
with:
|
|
18
|
+
python-version: ${{ matrix.python-version }}
|
|
19
|
+
- run: python -m pip install --upgrade pip
|
|
20
|
+
- run: pip install -e ".[dev]"
|
|
21
|
+
- run: pytest -q
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.py[cod]
|
|
3
|
+
.pytest_cache/
|
|
4
|
+
.ipynb_checkpoints/
|
|
5
|
+
build/
|
|
6
|
+
dist/
|
|
7
|
+
*.egg-info/
|
|
8
|
+
.venv/
|
|
9
|
+
venv/
|
|
10
|
+
.DS_Store
|
|
11
|
+
.vscode/
|
|
12
|
+
.idea/
|
|
13
|
+
|
|
14
|
+
# never commit cohort data, ontology dumps or analysis output
|
|
15
|
+
*.obo
|
|
16
|
+
*.hpoa
|
|
17
|
+
*.xlsx
|
|
18
|
+
*.csv
|
|
19
|
+
*.png
|
|
20
|
+
*.pdf
|
|
21
|
+
*.svg
|
|
22
|
+
# ... except the README figures produced by the synthetic example
|
|
23
|
+
!examples/figures/*.png
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
Format follows [Keep a Changelog](https://keepachangelog.com/); versioning is [SemVer](https://semver.org/).
|
|
4
|
+
|
|
5
|
+
## [0.5.0] — 2026-09-05
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
- **`Cohort.distance()` now defaults to SimGIC**, the IC-weighted ontology-aware
|
|
9
|
+
measure, with cosine kept as a sensitivity analysis; `Cohort.distances()` returns
|
|
10
|
+
both, ready for the robustness protocol. (Behaviour change: an explicit
|
|
11
|
+
`distance("cosine")` reproduces the old default.)
|
|
12
|
+
- QC flags are relative and descriptive — `LOW_ANNOTATION_DEPTH`,
|
|
13
|
+
`LOW_SPECIFICITY_RELATIVE_TO_COHORT`, `HIGH_REDUNDANCY`, resolving to
|
|
14
|
+
`review recommended` — replacing the absolute `under-phenotyped` / `non-specific`.
|
|
15
|
+
A short, well-chosen phenotype list is not a poor one. Summary keys renamed
|
|
16
|
+
accordingly (`pct_low_annotation_depth`, `pct_low_specificity`,
|
|
17
|
+
`pct_review_recommended`).
|
|
18
|
+
- `annotation_bias` reports `confound_risk` (LOW / MODERATE / HIGH) with an explicit
|
|
19
|
+
note that annotation counts predicting a group does **not** establish that the
|
|
20
|
+
separation is caused by them, replacing the causal verdict string.
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
- `distance(..., negatives="use")`: excluded phenotypes (propagated downwards, since
|
|
24
|
+
excluding a parent excludes its children) enter the similarity, so a shared
|
|
25
|
+
ruled-out phenotype counts. Refuses to run where absence was never recorded;
|
|
26
|
+
`onset=` is declared and raises rather than silently ignoring.
|
|
27
|
+
- `Ontology.descendants`, `Cohort.propagated_excluded`, `Cohort.has_negatives`,
|
|
28
|
+
`Cohort.excluded_terms`, `Cohort.distances`.
|
|
29
|
+
- `explain_groups` reports `excluded_a` / `excluded_b` — the fraction in whom a
|
|
30
|
+
phenotype was looked for and ruled out — alongside the test, never inside it.
|
|
31
|
+
- Command line (`phenotopo`): `demo` (synthetic cohort → report in a browser),
|
|
32
|
+
`report` (Phenopackets or a CSV/XLSX table → the same HTML, no Python needed),
|
|
33
|
+
`ontology install` / `path` (explicit local ontology cache; the only place the
|
|
34
|
+
package touches the network).
|
|
35
|
+
|
|
36
|
+
## [0.4.0] — 2026-09-05
|
|
37
|
+
|
|
38
|
+
### Added
|
|
39
|
+
- `cohort.Cohort` / `cohort.Ontology` with readers `from_hpo_table` and
|
|
40
|
+
`from_phenopackets` (GA4GH Phenopackets v2: excluded phenotypes, onset, disease and
|
|
41
|
+
gene), ancestor propagation, information content, and cosine / SimGIC distances —
|
|
42
|
+
so an analysis no longer starts with the user building a distance matrix.
|
|
43
|
+
- `qc.phenotype_qc`: per-patient annotation quality (terms, specificity, ontology
|
|
44
|
+
depth, redundant ancestors, recorded absence and onset) with flags, plus
|
|
45
|
+
`qc.plot_qc`.
|
|
46
|
+
- `qc.annotation_bias`: Kruskal–Wallis on annotation depth across groups and a
|
|
47
|
+
cross-validated k-NN comparison of group recovery from phenotype versus from
|
|
48
|
+
annotation counts alone — how much of the apparent structure is bookkeeping.
|
|
49
|
+
- `outliers.patient_outliers` (isolation and neighbourhood discordance reported
|
|
50
|
+
separately) and `outliers.explain_outlier` (terms that make a patient unusual, and
|
|
51
|
+
the terms its neighbourhood has that it lacks).
|
|
52
|
+
- `explain.explain_groups`: ontology-aware group comparison with a Westfall–Young
|
|
53
|
+
max-T permutation (FWER control that respects parent–child dependence), Newcombe
|
|
54
|
+
confidence intervals, an effect-size threshold and pruning of redundant ancestor
|
|
55
|
+
terms; `explain.plot_explain`.
|
|
56
|
+
- `report.cohort_report`: one self-contained local HTML file with QC, annotation
|
|
57
|
+
bias, outliers, robustness verdicts and group comparisons, figures embedded.
|
|
58
|
+
- `data.synthetic_hpo_cohort`: an annotated cohort with designed faults (a thinly
|
|
59
|
+
phenotyped site, discordant cases, redundant ancestor terms).
|
|
60
|
+
|
|
61
|
+
### Changed
|
|
62
|
+
- `phenotype_qc` returns its patient table in cohort order, so it lines up with
|
|
63
|
+
labels and distance matrices; the flagged subset is the sorted view.
|
|
64
|
+
|
|
65
|
+
## [0.3.1] — 2026-09-03
|
|
66
|
+
|
|
67
|
+
### Fixed
|
|
68
|
+
- Robustness tests assumed group sizes that the synthetic cohort does not have; simplified
|
|
69
|
+
pair-label formatting in `plot_forest`.
|
|
70
|
+
|
|
71
|
+
## [0.3.0] — 2026-09-03
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
- `robustness.connectivity_robustness`: the multi-configuration protocol (distances × k,
|
|
75
|
+
minimum group size, effect-size thresholds, all-configurations verdicts) and
|
|
76
|
+
`robustness.plot_forest`.
|
|
77
|
+
- Continuous integration (pytest on push / pull request), `CITATION.cff`.
|
|
78
|
+
|
|
79
|
+
## [0.2.0] — 2026-09-03
|
|
80
|
+
|
|
81
|
+
### Added
|
|
82
|
+
- `stats.permutation_test`: label-permutation null for every connectivity ratio
|
|
83
|
+
(p-values, BH q-values, z-scores, 95 % null interval), `stats.bootstrap_ratio`
|
|
84
|
+
(subsampling percentile CI), significance overlay on the connectivity heatmap.
|
|
85
|
+
- `plot_connectivity_heatmap`; optional adjustText label repulsion.
|
|
86
|
+
|
|
87
|
+
## [0.1.0] — 2026-09-03
|
|
88
|
+
|
|
89
|
+
### Added
|
|
90
|
+
- `knn_graph`, `group_connectivity`: PAGA-style connectivity between labelled groups
|
|
91
|
+
(observed k-NN edges relative to a configuration-model expectation).
|
|
92
|
+
- `plot_connectivity`: abstracted group graph drawn at group centroids.
|
|
93
|
+
- `plot_density`, `plot_small_multiples`: overlap-honest alternatives to a single
|
|
94
|
+
many-colour scatter.
|
|
95
|
+
- `hyperbolic`: Poincaré embedding of a term hierarchy, Einstein-midpoint placement
|
|
96
|
+
of patients in the disk, radial specificity axis.
|
|
97
|
+
- `mapper`: KeplerMapper wrapper producing a `networkx` graph coloured by any
|
|
98
|
+
per-patient value.
|
|
99
|
+
- `data.synthetic_cohort` / `data.synthetic_hierarchy` for examples and tests.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
cff-version: 1.2.0
|
|
2
|
+
message: "If you use phenotopo, please cite it as below."
|
|
3
|
+
title: "phenotopo: a QC and explainability toolkit for phenotype cohorts"
|
|
4
|
+
type: software
|
|
5
|
+
version: 0.5.0
|
|
6
|
+
date-released: "2026-09-05"
|
|
7
|
+
license: MIT
|
|
8
|
+
repository-code: "https://github.com/MargoSolo/phenotopo"
|
|
9
|
+
authors:
|
|
10
|
+
- family-names: Soloshenko
|
|
11
|
+
given-names: Margarita
|
|
12
|
+
email: margosoloshenko@mail.ru
|
|
13
|
+
keywords:
|
|
14
|
+
- HPO
|
|
15
|
+
- Phenopacket
|
|
16
|
+
- phenotype
|
|
17
|
+
- rare disease
|
|
18
|
+
- quality control
|
|
19
|
+
- outlier detection
|
|
20
|
+
- visualization
|
|
21
|
+
- connectivity
|
phenotopo-0.5.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Margarita Soloshenko
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
phenotopo-0.5.0/PKG-INFO
ADDED
|
@@ -0,0 +1,416 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: phenotopo
|
|
3
|
+
Version: 0.5.0
|
|
4
|
+
Summary: Phenotype QC, outlier detection and robust cohort structure for HPO / Phenopacket rare-disease cohorts.
|
|
5
|
+
Project-URL: Homepage, https://github.com/MargoSolo/phenotopo
|
|
6
|
+
Project-URL: Issues, https://github.com/MargoSolo/phenotopo/issues
|
|
7
|
+
Author-email: Margarita Soloshenko <margosoloshenko@mail.ru>
|
|
8
|
+
License: MIT License
|
|
9
|
+
|
|
10
|
+
Copyright (c) 2026 Margarita Soloshenko
|
|
11
|
+
|
|
12
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
13
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
14
|
+
in the Software without restriction, including without limitation the rights
|
|
15
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
16
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
17
|
+
furnished to do so, subject to the following conditions:
|
|
18
|
+
|
|
19
|
+
The above copyright notice and this permission notice shall be included in all
|
|
20
|
+
copies or substantial portions of the Software.
|
|
21
|
+
|
|
22
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
23
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
24
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
25
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
26
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
27
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
28
|
+
SOFTWARE.
|
|
29
|
+
License-File: LICENSE
|
|
30
|
+
Keywords: HPO,PAGA,Phenopacket,genotype-phenotype,outlier detection,phenotype,quality control,rare disease,visualization
|
|
31
|
+
Classifier: Development Status :: 4 - Beta
|
|
32
|
+
Classifier: Intended Audience :: Science/Research
|
|
33
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
34
|
+
Classifier: Operating System :: OS Independent
|
|
35
|
+
Classifier: Programming Language :: Python :: 3
|
|
36
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
37
|
+
Classifier: Topic :: Scientific/Engineering :: Visualization
|
|
38
|
+
Requires-Python: >=3.10
|
|
39
|
+
Requires-Dist: matplotlib>=3.7
|
|
40
|
+
Requires-Dist: networkx>=3.0
|
|
41
|
+
Requires-Dist: numpy>=1.24
|
|
42
|
+
Requires-Dist: pandas>=2.0
|
|
43
|
+
Requires-Dist: scikit-learn>=1.3
|
|
44
|
+
Requires-Dist: scipy>=1.10
|
|
45
|
+
Provides-Extra: all
|
|
46
|
+
Requires-Dist: adjusttext>=1.0; extra == 'all'
|
|
47
|
+
Requires-Dist: gensim>=4.3; extra == 'all'
|
|
48
|
+
Requires-Dist: kmapper>=2.0; extra == 'all'
|
|
49
|
+
Requires-Dist: umap-learn>=0.5; extra == 'all'
|
|
50
|
+
Provides-Extra: dev
|
|
51
|
+
Requires-Dist: adjusttext>=1.0; extra == 'dev'
|
|
52
|
+
Requires-Dist: gensim>=4.3; extra == 'dev'
|
|
53
|
+
Requires-Dist: kmapper>=2.0; extra == 'dev'
|
|
54
|
+
Requires-Dist: pytest>=7.0; extra == 'dev'
|
|
55
|
+
Provides-Extra: hyperbolic
|
|
56
|
+
Requires-Dist: gensim>=4.3; extra == 'hyperbolic'
|
|
57
|
+
Provides-Extra: tda
|
|
58
|
+
Requires-Dist: kmapper>=2.0; extra == 'tda'
|
|
59
|
+
Description-Content-Type: text/markdown
|
|
60
|
+
|
|
61
|
+
# phenotopo
|
|
62
|
+
|
|
63
|
+
**A QC and explainability toolkit for phenotype cohorts.**
|
|
64
|
+
|
|
65
|
+
[](https://github.com/MargoSolo/phenotopo/actions/workflows/tests.yml)
|
|
66
|
+
[](https://www.python.org/)
|
|
67
|
+
[](LICENSE)
|
|
68
|
+
[](https://pypi.org/project/phenotopo/)
|
|
69
|
+
[](CHANGELOG.md)
|
|
70
|
+
|
|
71
|
+
Two things go wrong when a rare-disease cohort is analysed by HPO terms.
|
|
72
|
+
|
|
73
|
+
The first is **bookkeeping mistaken for biology**: some patients were phenotyped in
|
|
74
|
+
two vague terms and others in twenty specific ones, the difference tracks a site or
|
|
75
|
+
a clinician, and every map then shows structure that is really annotation depth.
|
|
76
|
+
The second is **clusters that are not there**: a cohort described by phenotype is
|
|
77
|
+
usually a continuum with local structure, and one UMAP with twenty colours invites
|
|
78
|
+
everyone to read islands into it.
|
|
79
|
+
|
|
80
|
+
`phenotopo` is built around those two failures. It scores how well each patient was
|
|
81
|
+
phenotyped and flags when the group signal might be bookkeeping; it finds the
|
|
82
|
+
patients who sit away from their own group; it says which phenotypes actually
|
|
83
|
+
separate two groups, tested in a way that respects the ontology; and it reports
|
|
84
|
+
cohort structure only where the answer survives a change of distance, of *k* and of
|
|
85
|
+
resampling. Treating a cohort as a continuum rather than a set of clusters is the
|
|
86
|
+
methodology underneath; QC and explainability are what it is for.
|
|
87
|
+
|
|
88
|
+
Everything runs locally. No patient data leaves the machine, and the library makes
|
|
89
|
+
no network calls — the single exception is the explicit `phenotopo ontology install`
|
|
90
|
+
command, which prints the URL it fetches.
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
pip install phenotopo
|
|
94
|
+
phenotopo demo # a report from a synthetic cohort, in a browser, in 30 seconds
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Installation
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
pip install phenotopo # core: cohorts, QC, outliers, comparison, connectivity
|
|
103
|
+
pip install "phenotopo[hyperbolic]" # + Poincaré disk (gensim)
|
|
104
|
+
pip install "phenotopo[tda]" # + Mapper graph (kmapper)
|
|
105
|
+
pip install "phenotopo[all]" # everything, incl. umap-learn
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For development, clone the repository and `pip install -e ".[dev]"`.
|
|
109
|
+
|
|
110
|
+
If `adjustText` is installed, overlapping labels on the connectivity graph and the
|
|
111
|
+
Poincaré disk are pushed apart automatically; without it the plots still work.
|
|
112
|
+
|
|
113
|
+
## Without writing Python
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
phenotopo demo # see what a report looks like
|
|
117
|
+
phenotopo ontology install hp # cache the HPO release once
|
|
118
|
+
phenotopo report patients/ --labels diagnosis --compare GENE_A GENE_B -o report.html
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
`report` takes a directory of Phenopackets, a single `.json` packet, or a CSV/XLSX
|
|
122
|
+
table of HPO terms, and writes the same self-contained HTML file the Python API
|
|
123
|
+
produces. This is not a substitute for a real interface — it is what makes the tool
|
|
124
|
+
usable by someone who will not open a notebook.
|
|
125
|
+
|
|
126
|
+
## Quick start
|
|
127
|
+
|
|
128
|
+
```python
|
|
129
|
+
import phenotopo as pt
|
|
130
|
+
|
|
131
|
+
cohort = pt.from_phenopackets("patients/", ontology="hp.obo") # or from_hpo_table(df, ...)
|
|
132
|
+
D = cohort.distance() # SimGIC, the ontology-aware default
|
|
133
|
+
qc = pt.phenotype_qc(cohort) # how was each patient annotated?
|
|
134
|
+
bias = pt.annotation_bias(qc, cohort.labels("site"), # could the group signal be bookkeeping?
|
|
135
|
+
distance=D)
|
|
136
|
+
out = pt.patient_outliers(D, cohort.labels("diagnosis"), # who sits away from their group?
|
|
137
|
+
ids=cohort.ids)
|
|
138
|
+
diff = pt.explain_groups(cohort, cohort.labels("gene"), # which phenotypes separate them?
|
|
139
|
+
"GENE_A", "GENE_B")
|
|
140
|
+
pt.cohort_report(cohort, labels="diagnosis", distance=D, # one local HTML file
|
|
141
|
+
comparisons=diff, path="cohort_report.html")
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
A `Cohort` is optional throughout: every analysis function also takes a plain
|
|
145
|
+
distance matrix and an array of labels, so `phenotopo` sits on top of a pipeline
|
|
146
|
+
you already have rather than replacing it.
|
|
147
|
+
|
|
148
|
+
### Similarity, and which fields are actually used
|
|
149
|
+
|
|
150
|
+
`cohort.distance()` is **SimGIC** — the IC-weighted Jaccard of the propagated term
|
|
151
|
+
sets (Pesquita et al. 2008), the ontology-aware measure the HPO literature uses.
|
|
152
|
+
`distance("cosine")` (cosine between IC-weighted propagated vectors) is kept as a
|
|
153
|
+
sensitivity analysis, and `cohort.distances()` returns both, ready to hand to the
|
|
154
|
+
robustness protocol. Both are hierarchy-aware: two patients sharing no term but
|
|
155
|
+
sharing an ancestor are still similar.
|
|
156
|
+
|
|
157
|
+
`from_phenopackets` reads GA4GH Phenopackets v2 and keeps what a bare list of term
|
|
158
|
+
IDs throws away. What each field currently affects — stated plainly, because storing
|
|
159
|
+
a field is not the same as using it:
|
|
160
|
+
|
|
161
|
+
| Field | Read | Used by |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| present phenotypes | yes | everything |
|
|
164
|
+
| **excluded** phenotypes (looked for and absent) | yes | `phenotype_qc`; `distance(..., negatives="use")`, where a shared ruled-out phenotype makes two patients more similar; reported as a separate column by `explain_groups` and never mixed into the test |
|
|
165
|
+
| **onset** per observation | yes | `phenotype_qc` completeness only — `distance(..., onset=...)` is declared and raises, rather than pretending |
|
|
166
|
+
| disease, gene, sex | yes | metadata, so `labels("gene")` and `explain_groups` work straight after reading |
|
|
167
|
+
|
|
168
|
+
`not recorded` and `looked for and absent` are different observations, and only the
|
|
169
|
+
second is evidence of absence — so negatives are opt-in and refuse to run on a cohort
|
|
170
|
+
where absence was never recorded.
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
The figures below come from [`examples/quickstart.py`](examples/quickstart.py), run on
|
|
175
|
+
synthetic cohorts with *designed* faults and structure, so each can be checked against
|
|
176
|
+
ground truth.
|
|
177
|
+
|
|
178
|
+
## 1 · Phenotyping quality
|
|
179
|
+
|
|
180
|
+
```python
|
|
181
|
+
qc = pt.phenotype_qc(cohort)
|
|
182
|
+
qc["summary"] # median terms, % low depth, % low specificity, % redundant, ...
|
|
183
|
+
qc["flagged"] # worst-annotated patients first, with reasons
|
|
184
|
+
pt.plot_qc(qc, cohort.labels("site"))
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
<p align="center"><img src="examples/figures/qc.png" width="760"></p>
|
|
188
|
+
|
|
189
|
+
Per patient: terms asserted and after propagation, explicitly excluded phenotypes,
|
|
190
|
+
observations with an onset, mean and total information content, mean ontology depth,
|
|
191
|
+
**redundant ancestor terms** (a term recorded next to its own child — bookkeeping, not
|
|
192
|
+
information), the specificity percentile, and flags.
|
|
193
|
+
|
|
194
|
+
The flags are deliberately **relative to this cohort and descriptive**:
|
|
195
|
+
`LOW_ANNOTATION_DEPTH`, `LOW_SPECIFICITY_RELATIVE_TO_COHORT`, `HIGH_REDUNDANCY`,
|
|
196
|
+
resolving to `review recommended`. Six well-chosen terms can describe a skeletal
|
|
197
|
+
dysplasia completely while fifteen vague ones describe a neurodevelopmental case
|
|
198
|
+
badly, so the tool never declares a patient "under-phenotyped" in absolute terms.
|
|
199
|
+
|
|
200
|
+
Then the question that decides whether any of the rest can be believed:
|
|
201
|
+
|
|
202
|
+
```python
|
|
203
|
+
bias = pt.annotation_bias(qc, labels, distance=D)
|
|
204
|
+
bias["kruskal"] # do the groups differ in annotation depth? H, p, epsilon-squared
|
|
205
|
+
bias["recovery"] # {'phenotype': 0.92, 'annotation_only': 1.00, 'confound_risk': 'HIGH', ...}
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Group labels are predicted twice by cross-validated k-NN — from the phenotype
|
|
209
|
+
distance, and from **how much was written down alone** (`n_terms`, `n_propagated`).
|
|
210
|
+
Specificity measures are deliberately excluded from that second model: they depend on
|
|
211
|
+
*which* terms a patient has, so including them would report real biology as bias.
|
|
212
|
+
|
|
213
|
+
The result is an **annotation-confound risk** (LOW / MODERATE / HIGH), not a verdict.
|
|
214
|
+
That annotation counts predict the group does *not* establish that the separation is
|
|
215
|
+
caused by them: a group that genuinely differs in phenotype severity is usually also
|
|
216
|
+
annotated more thoroughly, and the chain runs group → severity → annotation depth.
|
|
217
|
+
HIGH means the analysis must address the confound — by matching, stratification or a
|
|
218
|
+
depth-controlled comparison — not that the finding is an artefact.
|
|
219
|
+
|
|
220
|
+
## 2 · Outliers — who sits away from their own group
|
|
221
|
+
|
|
222
|
+
```python
|
|
223
|
+
out = pt.patient_outliers(D, labels, ids=cohort.ids, k=15)
|
|
224
|
+
pt.explain_outlier(cohort, index, D, labels)
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Two things get called an outlier, and mixing them produces nonsense, so they are
|
|
228
|
+
reported separately: **isolation** (far from everybody — a sparse or unusual
|
|
229
|
+
phenotype) and **discordance** (plenty of close neighbours, and they belong to a
|
|
230
|
+
different group). `explain_outlier` names the terms that make a patient unusual for
|
|
231
|
+
its neighbourhood, and — often more useful — the terms its neighbourhood has that it
|
|
232
|
+
lacks, which is as likely to be a phenotyping gap as biology.
|
|
233
|
+
|
|
234
|
+
The output is never a claim that a diagnosis is wrong. It is *phenotypically
|
|
235
|
+
discordant with the assigned group*: a candidate for review in a diagnostic cohort,
|
|
236
|
+
a genotype–phenotype study or a reclassification project.
|
|
237
|
+
|
|
238
|
+
## 3 · What separates two groups
|
|
239
|
+
|
|
240
|
+
```python
|
|
241
|
+
diff = pt.explain_groups(cohort, labels, "GENE_A", "GENE_B", min_effect=0.15)
|
|
242
|
+
diff["top"] # term, name, prevalence in each group, effect in pp with CI, adjusted p
|
|
243
|
+
pt.plot_explain(diff)
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
<p align="center"><img src="examples/figures/explain.png" width="700"></p>
|
|
247
|
+
|
|
248
|
+
One test per HPO term, sorted by p-value, answers this badly, for two reasons
|
|
249
|
+
specific to ontologies:
|
|
250
|
+
|
|
251
|
+
- **The terms are not independent.** After propagation a patient annotated *Status
|
|
252
|
+
epilepticus* is also annotated *Seizure* and everything up to the root, so one
|
|
253
|
+
finding lights up a whole ancestor chain and Benjamini–Hochberg assumes far more
|
|
254
|
+
independence than exists. `explain_groups` therefore uses a **Westfall–Young max-T
|
|
255
|
+
permutation**: labels are shuffled and the *largest* prevalence difference over all
|
|
256
|
+
terms is recorded each time, so the null already contains the ontology's
|
|
257
|
+
correlation structure and the adjusted p-values control the family-wise error rate.
|
|
258
|
+
- **Significance is not the answer.** With a few thousand patients a 2-point
|
|
259
|
+
difference is significant and clinically empty. Terms are reported only above an
|
|
260
|
+
explicit **effect-size threshold** in percentage points, with a Newcombe confidence
|
|
261
|
+
interval, and redundant ancestors of an already-reported term are pruned — ties go
|
|
262
|
+
to the deeper term, so *Seizure* is dropped for *Status epilepticus* and never the
|
|
263
|
+
other way round.
|
|
264
|
+
|
|
265
|
+
## 4 · Cohort structure, only where it is robust
|
|
266
|
+
|
|
267
|
+
```python
|
|
268
|
+
res = pt.connectivity_robustness({"cosine": D_cos, "simgic": D_gic}, labels,
|
|
269
|
+
ks=(10, 15, 30), min_size=100)
|
|
270
|
+
res["summary"] # per pair: ratio, CI, range across configurations, verdict
|
|
271
|
+
pt.plot_forest(res)
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
The connectivity ratio compares the k-NN edges crossing between two groups with the
|
|
275
|
+
number expected under a degree-preserving null (the PAGA abstraction, Wolf et al.
|
|
276
|
+
2019): `1` is random mixing, `> 1` blending, `< 1` separation, and the diagonal is
|
|
277
|
+
within-group cohesion. A ratio from one distance, one *k* and one sample is a single
|
|
278
|
+
draw, so the protocol (i) drops groups below `min_size`, whose expected counts are
|
|
279
|
+
tiny and whose ratios explode; (ii) recomputes every ratio for every distance × *k*
|
|
280
|
+
with a permutation *q* and a bootstrap CI; (iii) applies an **effect-size threshold**,
|
|
281
|
+
not just significance; and (iv) issues a verdict only if the criterion holds in
|
|
282
|
+
**every** configuration. What survives is a sentence you can write down.
|
|
283
|
+
|
|
284
|
+
The underlying views, when you want to look rather than test:
|
|
285
|
+
|
|
286
|
+
```python
|
|
287
|
+
conn = pt.group_connectivity(pt.knn_graph(distance=D, k=15), labels)
|
|
288
|
+
pt.plot_connectivity(conn, embedding=emb, labels=labels) # node-link, few groups
|
|
289
|
+
pt.plot_connectivity_heatmap(conn, min_size=30, # every pair, many groups
|
|
290
|
+
significance=pt.permutation_test(pt.knn_graph(distance=D), labels))
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
<p align="center">
|
|
294
|
+
<img src="examples/figures/connectivity.png" width="420">
|
|
295
|
+
<img src="examples/figures/connectivity_heatmap.png" width="380">
|
|
296
|
+
</p>
|
|
297
|
+
|
|
298
|
+
Nodes sit at each group's median position, node area is group size, edge width ∝ log
|
|
299
|
+
ratio. Past about eight groups a node–link drawing stops being readable and the
|
|
300
|
+
heatmap is the honest form: every pair, the separated ones (`ratio < 1`) included,
|
|
301
|
+
`*` where the permutation q survives correction.
|
|
302
|
+
|
|
303
|
+
## 5 · Looking at the continuum
|
|
304
|
+
|
|
305
|
+
```python
|
|
306
|
+
pt.plot_density(emb, labels, levels=(0.5, 0.85)) # where groups overlap, drawn not hidden
|
|
307
|
+
pt.plot_small_multiples(emb, labels, ncols=5) # one panel per group, no 20-colour legend
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
<p align="center">
|
|
311
|
+
<img src="examples/figures/density.png" width="420">
|
|
312
|
+
<img src="examples/figures/small_multiples.png" width="420">
|
|
313
|
+
</p>
|
|
314
|
+
|
|
315
|
+
Two further views, for hierarchy and shape:
|
|
316
|
+
|
|
317
|
+
```python
|
|
318
|
+
from phenotopo.hyperbolic import poincare_terms, place_patients, plot_disk
|
|
319
|
+
from phenotopo.mapper import mapper_graph, node_values, plot_mapper
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
<p align="center">
|
|
323
|
+
<img src="examples/figures/hyperbolic.png" width="380">
|
|
324
|
+
<img src="examples/figures/mapper.png" width="440">
|
|
325
|
+
</p>
|
|
326
|
+
|
|
327
|
+
The **Poincaré disk** fits a hierarchy into two dimensions with the root at the centre
|
|
328
|
+
and specific leaves at the rim; patients sit at the Einstein midpoint of their terms,
|
|
329
|
+
so phenotyping specificity becomes a radial axis — the annotation-depth confound made
|
|
330
|
+
geometric instead of hidden. The **Mapper graph** covers the space with overlapping
|
|
331
|
+
bins and clusters locally inside each, showing branches, bridges and flares without
|
|
332
|
+
forcing every patient into a cluster; colour it by any per-patient outcome (here a
|
|
333
|
+
synthetic diagnostic yield) to see where that outcome concentrates.
|
|
334
|
+
|
|
335
|
+
## 6 · One file to hand over
|
|
336
|
+
|
|
337
|
+
```python
|
|
338
|
+
pt.cohort_report(cohort, labels="diagnosis", distance=D,
|
|
339
|
+
robustness=res, comparisons=[diff], path="cohort_report.html")
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
Cohort overview, QC and the annotation-bias verdict, the outlier list, the robustness
|
|
343
|
+
table with its forest plot and every group comparison — in a single self-contained
|
|
344
|
+
HTML file, figures embedded, no server and no network. It opens in a browser and
|
|
345
|
+
survives being emailed.
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
## Reading the connectivity ratio
|
|
350
|
+
|
|
351
|
+
| `ratio` | Meaning |
|
|
352
|
+
|---|---|
|
|
353
|
+
| ≫ 1 | groups blend — many more cross edges than chance |
|
|
354
|
+
| ≈ 1 | as connected as random mixing |
|
|
355
|
+
| ≪ 1 | separated — a real boundary in phenotype space |
|
|
356
|
+
|
|
357
|
+
The null preserves every group's total degree (the configuration model behind
|
|
358
|
+
modularity), so large groups are not rewarded merely for being large.
|
|
359
|
+
|
|
360
|
+
## API
|
|
361
|
+
|
|
362
|
+
| Module | Contents |
|
|
363
|
+
|---|---|
|
|
364
|
+
| `phenotopo.cohort` | `Cohort`, `Ontology`, `from_hpo_table`, `from_phenopackets` |
|
|
365
|
+
| `phenotopo.qc` | `phenotype_qc`, `annotation_bias`, `plot_qc` |
|
|
366
|
+
| `phenotopo.outliers` | `patient_outliers`, `explain_outlier` |
|
|
367
|
+
| `phenotopo.explain` | `explain_groups`, `plot_explain` |
|
|
368
|
+
| `phenotopo.report` | `cohort_report` |
|
|
369
|
+
| `phenotopo.graph` | `knn_graph`, `group_connectivity`, `group_centroids` |
|
|
370
|
+
| `phenotopo.stats` | `permutation_test`, `bootstrap_ratio`, `benjamini_hochberg` |
|
|
371
|
+
| `phenotopo.robustness` | `connectivity_robustness`, `plot_forest` |
|
|
372
|
+
| `phenotopo.layout` | `plot_connectivity`, `plot_connectivity_heatmap`, `plot_density`, `plot_small_multiples`, `default_palette` |
|
|
373
|
+
| `phenotopo.hyperbolic` | `poincare_terms`, `einstein_midpoint`, `place_patients`, `radial_specificity`, `plot_disk` |
|
|
374
|
+
| `phenotopo.mapper` | `mapper_graph`, `node_values`, `plot_mapper` |
|
|
375
|
+
| `phenotopo.data` | `synthetic_cohort`, `synthetic_hpo_cohort`, `synthetic_hierarchy`, `synthetic_term_lists` |
|
|
376
|
+
| `phenotopo.cli` | `phenotopo demo`, `phenotopo report`, `phenotopo ontology install/path` |
|
|
377
|
+
|
|
378
|
+
## Scope
|
|
379
|
+
|
|
380
|
+
`phenotopo` deliberately does **not** prioritise genes or diagnoses for a single
|
|
381
|
+
patient — Exomiser and LIRICAL do that — and does not capture phenotypes, pedigrees
|
|
382
|
+
or variants, which is PhenoTips' job. It sits after capture and beside
|
|
383
|
+
prioritisation: quality control, cohort structure, outliers and group comparison,
|
|
384
|
+
reading Phenopackets so it fits between the two.
|
|
385
|
+
|
|
386
|
+
## Citation
|
|
387
|
+
|
|
388
|
+
See [`CITATION.cff`](CITATION.cff).
|
|
389
|
+
|
|
390
|
+
## Tests
|
|
391
|
+
|
|
392
|
+
```bash
|
|
393
|
+
pytest -q
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
Runs on synthetic data only, by design: **no patient data belongs in this
|
|
397
|
+
repository**, and `.gitignore` blocks ontology dumps, cohort files and analysis
|
|
398
|
+
output. The figures in this README come from synthetic cohorts with designed
|
|
399
|
+
structure, which makes them checkable but not evidence about real cohorts; a
|
|
400
|
+
case study on public, published cases (Phenopacket Store) is the next release.
|
|
401
|
+
|
|
402
|
+
## References
|
|
403
|
+
|
|
404
|
+
- Wolf F.A. et al. *PAGA: graph abstraction reconciles clustering with trajectory inference through a topology preserving map of single cells.* Genome Biology, 2019.
|
|
405
|
+
- Westfall P.H., Young S.S. *Resampling-based multiple testing.* Wiley, 1993 — max-T procedure.
|
|
406
|
+
- Newcombe R.G. *Interval estimation for the difference between independent proportions.* Statistics in Medicine, 1998.
|
|
407
|
+
- Pesquita C. et al. *Metrics for GO based protein semantic similarity: a systematic evaluation.* BMC Bioinformatics, 2008 — SimGIC.
|
|
408
|
+
- Jacobsen J.O.B. et al. *The GA4GH Phenopacket schema defines a computable representation of clinical data.* Nature Biotechnology, 2022.
|
|
409
|
+
- Nickel M., Kiela D. *Poincaré embeddings for learning hierarchical representations.* NeurIPS, 2017.
|
|
410
|
+
- Ungar A.A. *Analytic hyperbolic geometry.* World Scientific, 2005 — Einstein midpoint.
|
|
411
|
+
- Singh G., Mémoli F., Carlsson G. *Topological methods for the analysis of high dimensional data sets.* Eurographics, 2007 — Mapper.
|
|
412
|
+
- van Veen H.J. et al. *Kepler Mapper.* Journal of Open Source Software, 2019.
|
|
413
|
+
|
|
414
|
+
## License
|
|
415
|
+
|
|
416
|
+
MIT — see [LICENSE](LICENSE).
|