philm 0.0.1__tar.gz

philm-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Yiyan Yang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

philm-0.0.1/MANIFEST.in

@@ -0,0 +1,5 @@
+include README.md
+include LICENSE
+recursive-include src/philm/config *.yaml *.yml
+recursive-include img *.png *.jpg *.jpeg *.svg
+recursive-include env *.txt

philm-0.0.1/PKG-INFO

@@ -0,0 +1,386 @@
+Metadata-Version: 2.4
+Name: philm
+Version: 0.0.1
+Summary: PHILM: Phage-Host Interaction Learning from Metagenomic profiles
+Author: Yiyan Yang
+Project-URL: Homepage, https://github.com/YiyanYang0728/PHILM
+Project-URL: Repository, https://github.com/YiyanYang0728/PHILM
+Project-URL: Issues, https://github.com/YiyanYang0728/PHILM/issues
+Keywords: phage,host prediction,metagenomics,microbiome,virome,deep learning
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: scipy
+Requires-Dist: scikit-learn
+Requires-Dist: pyyaml
+Requires-Dist: tqdm
+Requires-Dist: matplotlib
+Provides-Extra: cpu
+Requires-Dist: torch; extra == "cpu"
+Provides-Extra: gpu
+Requires-Dist: torch; extra == "gpu"
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# PHILM: Phage-Host Interaction Learning from Metagenomic Profiles
+
+## Overview
+
+**PHILM** is a deep learning framework designed to predict phage-host interactions (PHIs) directly from metagenomic profiles and classify sample status using model-derived latent representations.
+
+**Workflow**
+
+![Workflow](img/Figure1.jpg)
+
+## Table of Contents
+
+- [PHILM: Phage-Host Interaction Learning from Metagenomic Profiles](#philm-phage-host-interaction-learning-from-metagenomic-profiles)
+  - [Overview](#overview)
+  - [Table of Contents](#table-of-contents)
+  - [Installation](#installation)
+    - [Clone the Repository](#clone-the-repository)
+    - [Create a Conda Environment](#create-a-conda-environment)
+      - [Option 1. Install using conda environment files](#option-1-install-using-conda-environment-files)
+      - [Option 2. Install with pip inside a conda environment](#option-2-install-with-pip-inside-a-conda-environment)
+  - [Usage](#usage)
+    - [1. Data Preparation](#1-data-preparation)
+    - [2. Model Training](#2-model-training)
+    - [3. Evaluation](#3-evaluation)
+    - [4. Interaction Inference](#4-interaction-inference)
+    - [5. Permutation-Based p-Value Estimation for Interactions (Optional)](#5-permutation-based-p-value-estimation-for-interactions-optional)
+    - [6. Latent Representation Extraction (Optional)](#6-latent-representation-extraction-optional)
+  - [Dependencies](#dependencies)
+  - [License](#license)
+  - [Citation](#citation)
+
+## Installation
+
+### Clone the Repository
+
+```bash
+git clone git@github.com:YiyanYang0728/PHILM.git
+cd PHILM
+```
+
+### Create a Conda Environment
+
+#### Option 1. Install using conda environment files
+
+For NVIDIA GPU systems with CUDA 12.1-compatible drivers:
+
+```bash
+conda env create -f env/environment.yml
+conda activate PHILM
+```
+
+For CPU-only systems:
+
+```bash
+conda env create -f env/environment.cpu.yml
+conda activate PHILM-cpu
+```
+If preferred, `mamba` can be used instead of `conda` in the commands above.
+
+#### Option 2. Install with pip inside a conda environment
+
+Create a clean conda environment:
+
+```bash
+conda create -n PHILM python=3.12
+conda activate PHILM
+pip install dist/philm-0.0.1-py3-none-any.whl # need to update
+```
+
+Install dependencies for NVIDIA GPU systems with CUDA 12.1-compatible drivers:
+
+```bash
+pip install -r env/requirements.gpu-cu121.txt
+```
+
+Install dependencies for CPU-only systems:
+
+```bash
+pip install -r env/requirements.cpu.txt
+```
+
+PHILM can run on CPU-only Linux systems and NVIDIA CUDA-enabled Linux systems. Separate requirement files are provided because PyTorch uses different installation packages for CUDA-enabled and CPU-only environments.
+
+## Usage
+
+### 1. Data Preparation
+
+Prepare three inputs for `scripts/split_data.py`: a prokaryotic relative abundance profile, a phage relative abundance profile, and an output directory.
+
+Cross-kingdom relative abundance profiles, including prokaryotic and phage profiles, can be generated using tools such as [sylph](https://github.com/bluenote-1577/sylph) and [phanta](https://github.com/bhattlab/phanta).
+
+```bash
+philm-split --bact-arc raw_data/Bact_arc_profile.tsv --phage raw_data/Phage_profile.tsv --outdir data
+```
+
+**Outputs:**
+
+* Raw split data without CLR transformation:
+  `data/<Phage/Bact_arc>_<train/validation/test>_no_clr.tsv`
+
+* CLR-transformed split data:
+  `data/<Phage/Bact_arc>_<train/validation/test>.tsv`
+
+* Feature names:
+  `data/<Phage/Bact_arc>_feature_names.txt`
+
+* Sample names:
+  `data/<train/validation/test>_samples.txt`
+
+### 2. Model Training
+
+Configure training parameters and file paths in a YAML file.
+
+```bash
+# Copy the template configuration file and customize it
+cp src/philm/config/config_train.yaml my_config_train.yaml
+
+# Run model training
+philm-train -c my_config_train.yaml
+```
+
+Real-time training progress, checkpoints, and logs are saved in `checkpoints/`.
+
+**Outputs:**
+
+* Best model:
+  `results/PHILM_best_model.pth`
+
+* Best parameters:
+  `results/PHILM_best_params.yaml`
+
+* Checkpoints and logs:
+  `checkpoints/`
+
+### 3. Evaluation
+
+Generate predictions for the test data.
+
+```bash
+# Prepare the evalutaion configuration file by merging the best parameters
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_test.yaml - \
+    > my_config_test.yaml
+# Run evalutaion on test dataset
+philm-evaluate -c my_config_test.yaml
+# Summarize metrics
+philm-summarize results/PHILM_predict_test &> results/PHILM_predict_test.ft.metrics
+```
+
+> **Tip:** Repeat the steps above for the validation and training sets by using `config_test.val.yaml` and `config_test.train.yaml`, respectively.
+
+```bash
+# Run evalutaion on validation dataset
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_test.val.yaml - \
+    > my_config_test.val.yaml
+philm-evaluate -c my_config_test.val.yaml
+philm-summarize results/PHILM_predict_val &> results/PHILM_predict_val.ft.metrics
+# Run evalutaion on training dataset
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_test.train.yaml - \
+    > my_config_test.train.yaml
+philm-evaluate -c my_config_test.train.yaml
+philm-summarize results/PHILM_predict_train &> results/PHILM_predict_train.ft.metrics
+```
+
+### 4. Interaction Inference
+
+Infer host-sensitivity scores for each phage-prokaryote feature pair.
+
+```bash
+# Infer interactions on the training dataset
+# Note: my_config_test.train.yaml must already exist
+philm-interaction -c my_config_test.train.yaml --data-dir data --split train \
+    --score-mode gradient --phage-normalize signed_zscore \
+    --out results/PHILM_interactions.tsv
+```
+
+Optionally, interactions can be inferred using the combined training, validation, and test datasets. This result is expected to be similar to the training-set-based result.
+
+```bash
+philm-interaction -c my_config_test.train.yaml --data-dir data --split all \
+    --score-mode gradient --phage-normalize signed_zscore \
+    --out results/PHILM_interactions.all.tsv
+```
+
+**Output:**
+
+* Predicted interaction scores:
+  `results/PHILM_interactions.tsv`
+
+### 5. Permutation-Based p-Value Estimation for Interactions (Optional)
+
+This optional step estimates empirical p-values and adjusted p-values for PHILM-inferred interactions using permutation-based null models.
+
+This step is computationally intensive and is recommended only when formal false discovery rate control is required. It is best suited for high-performance computing environments or systems with sufficient CPU resources. For exploratory analyses, users may first apply a heuristic cutoff, such as `normalized_score >= 3.5`, to prioritize candidate PHIs.
+
+```bash
+# Step 1: Retrain models using permuted data
+
+cat src/philm/config/config_train_perm.yaml \
+    <(awk '{print "  "$0}' results/PHILM_best_params.yaml) \
+    > train_perm.yaml
+N=99
+# Use N=999 for a larger permutation null when computational resources allow.
+d=$PWD
+seq 0 $N \
+    | awk -v d=$d '{print "philm-permutate -c train_perm.yaml --perm-id "$1}' \
+    > jobs.1.txt
+# Run the commands in jobs.1.txt in parallel or on an HPC cluster.
+```
+
+```bash
+# Step 2: Infer PHIs from permutation-trained models
+
+mkdir -p permutation_infer_configs
+
+for i in $(seq -f "%03g" 0 $N); do
+cat > permutation_infer_configs/config_infer_gradient.perm_${i}.yaml <<EOF
+data:
+  train_X: data/Phage_train.tsv
+  train_Y: data/Bact_arc_train.tsv
+  val_X: data/Phage_val.tsv
+  val_Y: data/Bact_arc_val.tsv
+  test_X: data/Phage_test.tsv
+  test_Y: data/Bact_arc_test.tsv
+
+model:
+  path: results/permutation_null/perm_${i}/PHILM_perm_${i}.pth
+  batch_size: 512
+EOF
+
+awk '$0~/hidden_size/{print "  "$0}' results/PHILM_best_params.yaml \
+    >> permutation_infer_configs/config_infer_gradient.perm_${i}.yaml
+done
+```
+
+```bash
+> jobs.2.txt
+
+d=$PWD
+
+for f in permutation_infer_configs/config_infer_gradient.perm_*.yaml; do
+    i=$(basename "$f" .yaml | cut -f2 -d. | cut -f2 -d_)
+
+    echo "philm-interaction \
+        -c permutation_infer_configs/config_infer_gradient.perm_${i}.yaml \
+        --data-dir data \
+        --split train \
+        --score-mode gradient \
+        --sort-by none \
+        --phage-normalize signed_zscore \
+        --out results/permutation_null/perm_${i}/PHILM_perm_${i}.raw_gradient.tsv" \
+        >> jobs.2.txt
+done
+
+# Run the commands in jobs.2.txt in parallel or on an HPC cluster.
+```
+
+```bash
+# Step 3: Calculate empirical p-values and adjusted p-values
+
+philm-pval --norm_mode off \
+    --observed results/PHILM_interactions.tsv \
+    --perm_glob "results/permutation_null/perm_*/PHILM_perm_*.raw_gradient.tsv" \
+    --out results/PHILM_interactions.pvalues.tsv
+```
+
+Filter significant interactions by adjusted p-value.
+
+```bash
+awk -F"\t" 'NR==1 || $5<0.05' \
+    results/PHILM_interactions.pvalues.tsv \
+    | cut -f1,2,3,5 \
+    > results/PHILM_interactions.pvalues.filtered.tsv
+```
+
+### 6. Latent Representation Extraction (Optional)
+
+PHILM can extract sample-level latent representations from a trained model. These representations can be used for downstream analyses such as sample classification, clustering, and visualization.
+
+```bash
+# Determine the output dimension
+outdim=$(wc -l < data/Bact_arc_feature_names.txt)
+
+# Prepare the representation extraction configuration file
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_repr.train.yaml - \
+    > my_config_repr.train.yaml
+sed -i "s|#OUTDIM#|$outdim|g" my_config_repr.train.yaml
+# Extract representations for the training data
+philm-repr -c my_config_repr.train.yaml
+```
+
+> **Tip:** Apply the same process for the validation and test datasets by using `config_repr.val.yaml` and `config_repr.test.yaml`.
+
+```bash
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_repr.val.yaml - \
+    > my_config_repr.val.yaml
+sed -i "s|#OUTDIM#|$outdim|g" my_config_repr.val.yaml
+philm-repr -c my_config_repr.val.yaml
+```
+
+```bash
+awk '{print "  "$0}' results/PHILM_best_params.yaml \
+    | cat src/philm/config/config_repr.test.yaml - \
+    > my_config_repr.test.yaml
+sed -i "s|#OUTDIM#|$outdim|g" my_config_repr.test.yaml
+philm-repr -c my_config_repr.test.yaml
+```
+
+Merge representations across splits.
+
+```bash
+cat results/PHILM_train_repr1.tsv \
+    results/PHILM_val_repr1.tsv \
+    results/PHILM_test_repr1.tsv \
+    > results/repr1_all.tsv
+
+cat results/PHILM_train_repr2.tsv \
+    results/PHILM_val_repr2.tsv \
+    results/PHILM_test_repr2.tsv \
+    > results/repr2_all.tsv
+```
+
+The `repr1_all.tsv` and `repr2_all.tsv` files are tab-separated files without row names or column names. The number of columns corresponds to the dimensionality of the PHILM-derived representations. Samples are organized as rows and are ordered consistently with `data/train_samples.txt`, `data/val_samples.txt`, and `data/test_samples.txt`.
+
+## Dependencies
+
+PHILM requires the following major Python packages:
+
+* PyTorch
+* torchdiffeq
+* NumPy
+* pandas
+* SciPy
+* scikit-learn
+* scikit-bio
+* PyYAML
+* Optuna
+
+Please see the requirement files in `env/` for the full dependency lists.
+
+## License
+
+This project is released under the [MIT License](./LICENSE).
+
+## Citation
+
+Please cite PHILM as follows:
+
+Yang, Y., Wang, T., Huang, D., Wang, X. W., Weiss, S. T., Korzenik, J., & Liu, Y. Y. (2025). *Deep Learning Transforms Phage-Host Interaction Discovery from Metagenomic Data*. bioRxiv.
+