hogprof 0.3.0__py3-none-any.whl

hogprof-0.3.0.dist-info/METADATA

@@ -0,0 +1,231 @@
+Metadata-Version: 2.4
+Name: hogprof
+Version: 0.3.0
+Summary: Phylogenetic Profiling with OMA and minhashing
+Author-email: Dave Moi <dmoi@unil.ch>, Athina Gavriilidou <athina.gavriilidou@unil.ch>, Nikolai Romashchenko <nikolai.romashchenko@unil.ch>
+Maintainer-email: Nikolai Romashchenko <nikolai.romashchenko@unil.ch>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/DessimozLab/HogProf
+Project-URL: Bug Tracker, https://github.com/DessimozLab/HogProf/issues
+Project-URL: Docs, https://dessimozlab.github.io/HogProf/
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Requires-Python: <3.13,>=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: biopython
+Requires-Dist: datasketch
+Requires-Dist: ete3
+Requires-Dist: goatools
+Requires-Dist: h5py
+Requires-Dist: legacy-cgi; python_version >= "3.13"
+Requires-Dist: lxml
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: polars
+Requires-Dist: pyham>=1.1.10
+Requires-Dist: pyoma>=0.14.2
+Requires-Dist: rich
+Requires-Dist: tables
+Provides-Extra: ml
+Requires-Dist: keras; extra == "ml"
+Dynamic: license-file
+
+# HogProf
+
+[![tests](https://github.com/DessimozLab/HogProf/actions/workflows/ci.yml/badge.svg)](https://github.com/DessimozLab/HogProf/actions/workflows/ci.yml)
+[![release](https://img.shields.io/pypi/v/HogProf.svg)](https://pypi.org/project/HogProf/)
+[![cite](https://img.shields.io/badge/DOI-10.1371%2Fjournal.pcbi.1007553-blue)](https://doi.org/10.1371/journal.pcbi.1007553)
+
+HogProf is an extensible and tunable approach to phylogenetic profiling using orthology data. It is powered by minhash-based data structures and computationally efficient.
+
+# Features
+  - Using orthoxoml files and a taxonomy calculated enhanced phylogenies of each family
+  - These are transformed into minhash signatures and a locally sensitive hashing forest object for search and comparison of profiles
+  - Taxonomic levels and evolutionary event types (presence, loss, duplication) can have custom weight in profile construction
+  - Optimization of weights using machine learning
+
+If you run into any problems, feel free to contact me at [dmoi@unil.ch](dmoi@unil.ch)
+
+# Quickstart
+## Install from PyPI (recommended)
+
+```bash
+pip install hogprof
+```
+
+## Install from sources
+```bash
+git clone https://github.com/DessimozLab/HogProf.git
+cd HogProf
+pip install .
+```
+
+
+# Usage: main mode
+
+## Example: using the OMA database
+Let's get a current version of the OMA hdf5 file and GAF. This will aloww us to use the HOGs and study the functional enrichment of our search results. **Careful, this download is heavy (hundreds of GB)**:
+
+```bash
+mkdir YourOmaDirectory
+cd YourOmaDirectory
+wget https://omabrowser.org/All/OmaServer.h5
+wget https://omabrowser.org/All/oma-go.txt.gz
+```
+
+The latest release (May.2026) is more than 300GB in size. Alternatively, you can use earlier releases, for example:
+```bash
+wget https://omabrowser.org/All.Jul2024/OmaServer.h5
+```
+which is 180GB in size. For other releases, check `https://omabrowser.org/oma/archives/`, select release and search for "OMA Browser database (as hdf5)". 
+
+
+We also need to make a location to store our pyprofiler databases
+
+```bash
+cd ..
+mkdir YourHogProfDirectory
+```
+
+Let's now compile a database containing all HOGs and our desired taxonomic levels using default settings. Launch the lshbuilder as shown below.
+`dbtypes` available on the command line are: `all`, `plants`, `archaea`, `bacteria`, `eukarya`, `protists`, `fungi`, `metazoa`, and `vertebrates`. These will use the NCBI taxonomy as a tree to annotate events in different gene family's histories.
+
+If you are using an OMA release before 2022 you will need to use the NCBI tree. This is the default tree used by HogProf.
+
+```
+lshbuilder --outpath YourHogProfDirectory --dbtype all --OMA YourOmaDirectory/OmaServer.h5 --nthreads numberOfCPUcores         
+
+```
+
+If you are using OMA releases after 2022 we will also be downloading the OMA taxonomic tree since it is more accurate than the NCBI tree. This will be used to annotate the events in the gene family histories.
+
+```bash
+wget https://omabrowser.org/All/speciestree.nwk
+```
+
+We are ready to use the OMA tree to build our database.
+
+```bash
+lshbuilder --outpath YourHogProfDirectory --dbtype all --OMA YourOmaDirectory/OmaServer.h5 --nthreads numberOfCPUcores --mastertree YourOmaDirectory/speciestree.nwk --reformat_names True
+
+```
+
+This should build a taxonomic tree for the genomes contained in the release and then calculate enhanced phylogenies for all HOGs in OMA.
+
+Once the database is completed it can be interogated using a profiler object. Construction and usage of this object should be done using a python script or notebook. This shown in the example notebook searchenrich.ipynb found in the examples. Please feel free to modify it to suit the needs of your own research.
+
+
+# Usage: levels mode
+
+The installation process is the same (github only for now).
+
+A fastOMA result folder is expected as input.
+
+**Step 1**: split the orthoxml into HOG-specific orthoxmls, using this script (requires a fastOMA installation):
+``` 
+python path/to/HogProf/src/HogProf/splitter.py \
+    /path/to/output_directory \
+    --fastoma-path /path/to/FastOMA
+```
+
+**Step 2**: build the HogProf database in **levels mode** by running `lshbuilder.py` with the `--slicesubhogs` option enabled. Point `--OrthoGlob` to the HOG-specific OrthoXML files generated in Step 1 and provide the species tree used by FastOMA.
+
+```bash
+python path/to/HogProf/src/HogProf/lshbuilder.py \
+    --OrthoGlob "/path/to/fastoma_results/splits/*.orthoxml" \
+    --outpath "/path/to/output_database/" \
+    --mastertree "/path/to/fastoma_results/species_tree_checked.nwk" \
+    --reformat_names True \
+    --slicesubhogs True \
+    --eventslim 1 \
+    --nthreads 20
+```
+
+Required parameters:
+- `--OrthoGlob`: glob matching the HOG-specific OrthoXML files created in Step 1.
+- `--outpath`: output directory for the HogProf database.
+- `--mastertree`: species tree corresponding to the FastOMA run.
+- `--slicesubhogs True`: enables levels mode by generating profiles for subHOGs.
+
+Common optional parameters:
+- `--eventslim`: minimum number of evolutionary events required to retain a subHOG (e.g. `1`).
+- `--specieslim`: minimum number of species required in a subHOG (default: `10`).
+- `--nthreads`: number of CPU threads to use.
+- `--reformat_names True`: recommended if the species tree labels need to be converted to numeric identifiers.
+
+**Step 3**: generate Jaccard similarity distributions and compute threshold tables from the HogProf database.
+
+This step analyzes the output of `lshbuilder` and produces the **bin-wise Jaccard threshold table** used for downstream filtering and scoring.
+
+```bash
+python path/to/HogProf/src/HogProf/generate_jaccard_distr.py \
+    --hogprof_folder "/path/to/2a_hogprof_output/" \
+    --output "/path/to/output_distributions/" \
+    --profiler-path "/path/to/profiler.py"
+```
+
+Required parameters:
+- `--hogprof_folder`: folder containing the HogProf LSHbuilder output (Step 2).
+- `--output`: directory where distribution plots and threshold tables will be written.
+
+Optional parameters:
+- `--profiler-path`: path to `profiler.py`.  
+  By default, this points to the `profiler.py` located in the same directory as the script.
+
+### Output
+
+This step produces a key file used for thresholding:
+
+- `bins_jaccard_thresholds.csv`  
+  → table of Jaccard similarity thresholds computed per bin, used for downstream scoring and filtering.
+
+The file is saved in the directory specified by `--output`.
+
+**Step 4**: extract level-specific hits using the HogProf profiler.
+
+This step runs `extract_levels_hits.py` to query the HogProf database built in Step 2 and generate a ranked list of hits per subHOG using the threshold table generated in Step 3.
+
+```bash
+python path/to/HogProf/src/HogProf/extract_levels_hits.py \
+    --input "/path/to/hogprof_output/" \
+    --outputfile "/path/to/output/extracted_hits.csv" \
+    --queries_file "/path/to/queries.tsv" \
+    --thresholds "/path/to/bins_jaccard_thresholds.csv" \
+    --k 1000 \
+    --allvsall True \
+    --profilerpath "profiler.py"
+```
+
+Required parameters:
+- `--input`: HogProf LSHbuilder output directory (Step 2).  
+  Must contain files such as:
+  `newlshforest.pkl`, `hashes.h5`, `reformatted_tree.nwk`, `fam2orthoxml.csv`, `profilersavingpath.csv`.
+- `--outputfile`: path to the CSV file where extracted hits will be saved.
+- `--queries_file`: table of query subHOGs (e.g. venom HOGs).  
+  Must contain a `hashid` column and a subHOG identifier column.
+- `--thresholds`: CSV file generated in Step 3 (`bins_jaccard_thresholds.csv`).
+
+Optional parameters:
+- `--k`: number of top hits to retain per query (default: `1000`).
+- `--allvsall`: also performs all-vs-all comparisons (default: `True`, may be computationally expensive).
+- `--profilerpath`: path to `profiler.py` from HogProf (default assumes local availability).
+
+### Output
+
+This step produces:
+
+- `extracted_hits.csv`  
+  → ranked hit table containing similarity matches between query subHOGs and the HogProf database, filtered using bin-specific Jaccard thresholds.
+
+**Step 5**: visualize coevolution networks in Cytoscape.
+
+This final step uses the Jupyter notebook `cytoscape_import.ipynb` to generate and export coevolution networks from the extracted HogProf hits.
+
+The notebook loads the output from Step 4 and prepares network files that can be visualized in Cytoscape.
+
+In addition to the extracted hits, the notebook requires a **nodes metadata table** corresponding to the query subHOGs used in Step 4.
+
+Once the database is completed it can be interogated using a profiler object. Construction and usage of this object should be done using a python script or notebook. This shown in the example notebook `searchenrich.ipynb` found in the examples. Please feel free to modify it to suit the needs of your own research.

hogprof-0.3.0.dist-info/RECORD

@@ -0,0 +1,21 @@
+hogprof/__init__.py,sha256=7rqlOVFXe5yUDqVX6h0Yem3IiY9cNQAVekevBdmyZ0o,116
+hogprof/cli.py,sha256=nTWDx4R_WG3X5q6WYQaw6I4iLWHXyz_6d1r_nfV754w,17707
+hogprof/extract_levels_hits.py,sha256=eftkHDYTG9H6zCD-zWnxLG6cvAgP9wtfb2KeAbI_Tyw,24079
+hogprof/generate_jaccard_distr.py,sha256=6hG_NV3fIjj6ctNK1khe-fmQK0kQAfca7nUESrRYhtc,23385
+hogprof/lshbuilder.py,sha256=4jw88UjF6CqTgO2Ie1QoZ0iPbQ2bd85i6z0r4D_aqp4,27643
+hogprof/metadata.py,sha256=D3KkjPcx55FtsM5cXoWzoK-Y5iZDE8p_9OaUTuf1txQ,3736
+hogprof/presets.py,sha256=OFrFbJLc2DJkvLuWKf8LLBISqeznCLqaPVa5UVwdzGc,886
+hogprof/profiler.py,sha256=Bb7G_gB2rL1cu5YcKcUz1jjaBnVhhBId-JbURdSxgUI,12065
+hogprof/splitter.py,sha256=MRrr1bafw-5k_-oAG56dBJgn-X4rYH8IXI5kKiAXKv8,1659
+hogprof/utils/__init__.py,sha256=ARYQ-WG3KfimoQSbXHHTLaIxt159Z8nou0U3BVB2wwk,15
+hogprof/utils/goautils.py,sha256=_7sGI8YWPUqfHfx-X4D5UcSUQlc8i38OmkCSSc6WLWg,7295
+hogprof/utils/hashutils.py,sha256=3gBYZChcXhw84ZpNu2K4KXBftGOnWBWesFsSkJeVkwY,7512
+hogprof/utils/orthoxml.py,sha256=9KJh7LqijvXz2yiDDlC30tAqkAgxYvfoZarjTn_hTw8,3132
+hogprof/utils/phylo.py,sha256=uPjdTSTVgekgUleQutK4Vgysj3xKArfjpnFGi4UEKgs,10338
+hogprof/utils/pyhamutils.py,sha256=RS9t1u29A1EjHBdqqB_A7zhjK9NlaXQwZcJPD2DVd6w,8610
+hogprof-0.3.0.dist-info/licenses/LICENSE,sha256=IK4QE3H9NvFJ0rBXXqRL7x_eASMoUdRsaWW_AZ5bbpo,1066
+hogprof-0.3.0.dist-info/METADATA,sha256=ZMBtkvbQJSbz-hzfEw5lrhEvVsoA2VvzzEvhhaXA2hw,10063
+hogprof-0.3.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+hogprof-0.3.0.dist-info/entry_points.txt,sha256=oWplV2sOD59S9onpyjsvifG437ttqS5GILBB6UOP_eY,67
+hogprof-0.3.0.dist-info/top_level.txt,sha256=vAhZifQO9UrL4XGD20aCcYKmleslCfSNsmSi8uSEOpE,8
+hogprof-0.3.0.dist-info/RECORD,,

hogprof-0.3.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

hogprof-0.3.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+hogprof = hogprof.cli:main
+hp = hogprof.cli:main

hogprof-0.3.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019 David Moi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

hogprof-0.3.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+hogprof