phylotypy 0.2.0__tar.gz

phylotypy-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Chad Saltikov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

phylotypy-0.2.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.4
+Name: phylotypy
+Version: 0.2.0
+Summary: Naive Bayes Classifier with Rust-accelerated taxonomy functions
+Author-email: Chad Saltikov <saltikov@ucsc.edu>
+License: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: setuptools
+Requires-Dist: numpy~=2.1.3
+Requires-Dist: numba~=0.61.0
+Requires-Dist: pandas~=2.2.3
+Requires-Dist: requests
+Requires-Dist: pandarallel
+Requires-Dist: jax
+Requires-Dist: cython
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
+
+# phylotypy
+Naive Bayesian Classifier for 16S rRNA gene sequence data
+
+Porting Riffomonas's CodeClub R package, phylotypr to python: https://github.com/riffomonas/phylotypr
+
+It's been a great challenge learning how to interpret the R code into Python with minimal use of extra libraries.
+
+It's best to clone the repository.  Run vigentte.py to see if everything works.
+
+Training the model with the full reference database from RDP takes about 30 seconds on my MacBook Pro.
+
+You can modify the vignette at the end to classify your own sequences. I've done this using DADA2's output files.
+
+There's also [read_fasta.py](src/phylotypy/utilities/read_fasta.py) that lets you take a fasta file of DNA seqences and process them into a dataframe for runing this classifier.
+
+I made a separate vignette, [vignette.py](vignette.py) on how to do this and classify 16S sequence data from QIIME, DADA2, or text files.
+
+Thanks Riffomonas for the inspiration.  Check out the videos on his Youtube channel https://youtube.com/playlist?list=PLmNrK_nkqBpIZlWa3yGEc2-wX7An2kpCL&si=LmHDV02K5_wb6C0j
+
+## How to install
+```
+pip install git+https://github.com/csaltikov/phylotypy.git
+```
+or if using uv (recommended) (<a href="https://docs.astral.sh/uv/getting-started/installation/">how to install uv</a>)
+```
+uv pip install git+https://github.com/csaltikov/phylotypy.git
+```
+
+## How to get started:
+First download the training data, RDP's trainset19072023, either from https://mothur.org/wiki/rdp_reference_files/
+ You can use the one in the [data](data/) directory called [rdp_16S_v19.dada2.fasta](data/rdp_16S_v19.dada2.fasta)
+
+I processed the latest rdp reference data into a format that will work here and for DADA2.
+
+The taxonomy string looks like this semicolon separated:
+```
+Bacteria;Phylum;Class;Order;Family;Genus
+
+for example:
+>Bacteria;Pseudomonadota;Gammaproteobacteria;Enterobacterales;Enterobacteriaceae;Citrobacter
+TAGAGTTTGATCCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAACAC.....
+```
+
+1. Load the training data and sequences to be classified
+```
+from pathlib import Path
+from phylotypy import classifier, results, read_fasta
+
+rdp = read_fasta.read_taxa_fasta("data/rdp_16S_v19.dada2.fasta")
+moving_pics = read_fasta.read_taxa_fasta("data/dna_moving_pictures.fasta")
+```
+2. Create the classifier. We'll call it database
+```
+database = classifier.make_classifier(rdp)
+```
+3. Classify the sequences
+```
+classified = classifier.classify_sequences(moving_pics, database)
+```
+4. Format the output
+```
+classified = results.summarize_predictions(classified)
+print(classified.columns)
+```
+Output:
+```
+>>> Index(['id', 'sequence', 'classification', 'Kingdom', 'Phylum', 'Class',
+       'Order', 'Family', 'Genus', 'observed', 'lineage'],
+      dtype='object')
+```
+```
+print(classified["classification"].head())
+```
+Output:
+```
+   0    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   1    Bacteria(100);Pseudomonadota(100);Betaproteoba...
+   2    Bacteria(100);Bacillota(100);Bacilli(100);Lact...
+   3    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   4    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   Name: classification, dtype: object
+```
+Format the results using results.summarize_predictions() function.
+The output is a pandas dataframe and can be saved to csv.
+```
+from phylotypy import results
+classified = results.summarize_predictions(classified)
+print(classified.head())
+
+classified.to_csv("classified_results.csv")
+
+```
+## Example classification output:
+The taxonomic levels "Domain", "Phylum", "Class", "Order", "Family", "Genus" are separated by ";".  The numbers in the () represent the confidence in the classificaiton.  The default confidence is 80%.
+```
+>>> Bacteria(100);Pseudomonadota(99);Alphaproteobacteria(99);Rhodospirillales(99);Acetobacteraceae(99);Roseomonas(83)
+
+>>> Bacteria(99);Bacteroidota(97);Bacteroidia(93);Bacteroidales(93);Bacteroidales_unclassified(93);Bacteroidales_unclassified(93)
+
+```
+## Complete code block:
+```
+from pathlib import Path
+from phylotypy import classifier, results, read_fasta
+
+rdp = read_fasta.read_taxa_fasta("data/rdp_16S_v19.dada2.fasta")
+moving_pics = read_fasta.read_taxa_fasta("data/dna_moving_pictures.fasta")
+
+database = classifier.make_classifier(rdp)
+
+classified = classifier.classify_sequences(moving_pics, database)
+classified = results.summarize_predictions(classified)
+print(classified.head())
+```
+### Requirements
+setuptools
+numpy
+numba
+pandas
+requests
+pandarallel
+jax
+cython

phylotypy-0.2.0/README.md

@@ -0,0 +1,123 @@
+# phylotypy
+Naive Bayesian Classifier for 16S rRNA gene sequence data
+
+Porting Riffomonas's CodeClub R package, phylotypr to python: https://github.com/riffomonas/phylotypr
+
+It's been a great challenge learning how to interpret the R code into Python with minimal use of extra libraries.
+
+It's best to clone the repository.  Run vigentte.py to see if everything works.
+
+Training the model with the full reference database from RDP takes about 30 seconds on my MacBook Pro.
+
+You can modify the vignette at the end to classify your own sequences. I've done this using DADA2's output files.
+
+There's also [read_fasta.py](src/phylotypy/utilities/read_fasta.py) that lets you take a fasta file of DNA seqences and process them into a dataframe for runing this classifier.
+
+I made a separate vignette, [vignette.py](vignette.py) on how to do this and classify 16S sequence data from QIIME, DADA2, or text files.
+
+Thanks Riffomonas for the inspiration.  Check out the videos on his Youtube channel https://youtube.com/playlist?list=PLmNrK_nkqBpIZlWa3yGEc2-wX7An2kpCL&si=LmHDV02K5_wb6C0j
+
+## How to install
+```
+pip install git+https://github.com/csaltikov/phylotypy.git
+```
+or if using uv (recommended) (<a href="https://docs.astral.sh/uv/getting-started/installation/">how to install uv</a>)
+```
+uv pip install git+https://github.com/csaltikov/phylotypy.git
+```
+
+## How to get started:
+First download the training data, RDP's trainset19072023, either from https://mothur.org/wiki/rdp_reference_files/
+ You can use the one in the [data](data/) directory called [rdp_16S_v19.dada2.fasta](data/rdp_16S_v19.dada2.fasta)
+
+I processed the latest rdp reference data into a format that will work here and for DADA2.
+
+The taxonomy string looks like this semicolon separated:
+```
+Bacteria;Phylum;Class;Order;Family;Genus
+
+for example:
+>Bacteria;Pseudomonadota;Gammaproteobacteria;Enterobacterales;Enterobacteriaceae;Citrobacter
+TAGAGTTTGATCCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAACAC.....
+```
+
+1. Load the training data and sequences to be classified
+```
+from pathlib import Path
+from phylotypy import classifier, results, read_fasta
+
+rdp = read_fasta.read_taxa_fasta("data/rdp_16S_v19.dada2.fasta")
+moving_pics = read_fasta.read_taxa_fasta("data/dna_moving_pictures.fasta")
+```
+2. Create the classifier. We'll call it database
+```
+database = classifier.make_classifier(rdp)
+```
+3. Classify the sequences
+```
+classified = classifier.classify_sequences(moving_pics, database)
+```
+4. Format the output
+```
+classified = results.summarize_predictions(classified)
+print(classified.columns)
+```
+Output:
+```
+>>> Index(['id', 'sequence', 'classification', 'Kingdom', 'Phylum', 'Class',
+       'Order', 'Family', 'Genus', 'observed', 'lineage'],
+      dtype='object')
+```
+```
+print(classified["classification"].head())
+```
+Output:
+```
+   0    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   1    Bacteria(100);Pseudomonadota(100);Betaproteoba...
+   2    Bacteria(100);Bacillota(100);Bacilli(100);Lact...
+   3    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   4    Bacteria(100);Bacteroidota(100);Bacteroidia(10...
+   Name: classification, dtype: object
+```
+Format the results using results.summarize_predictions() function.
+The output is a pandas dataframe and can be saved to csv.
+```
+from phylotypy import results
+classified = results.summarize_predictions(classified)
+print(classified.head())
+
+classified.to_csv("classified_results.csv")
+
+```
+## Example classification output:
+The taxonomic levels "Domain", "Phylum", "Class", "Order", "Family", "Genus" are separated by ";".  The numbers in the () represent the confidence in the classificaiton.  The default confidence is 80%.
+```
+>>> Bacteria(100);Pseudomonadota(99);Alphaproteobacteria(99);Rhodospirillales(99);Acetobacteraceae(99);Roseomonas(83)
+
+>>> Bacteria(99);Bacteroidota(97);Bacteroidia(93);Bacteroidales(93);Bacteroidales_unclassified(93);Bacteroidales_unclassified(93)
+
+```
+## Complete code block:
+```
+from pathlib import Path
+from phylotypy import classifier, results, read_fasta
+
+rdp = read_fasta.read_taxa_fasta("data/rdp_16S_v19.dada2.fasta")
+moving_pics = read_fasta.read_taxa_fasta("data/dna_moving_pictures.fasta")
+
+database = classifier.make_classifier(rdp)
+
+classified = classifier.classify_sequences(moving_pics, database)
+classified = results.summarize_predictions(classified)
+print(classified.head())
+```
+### Requirements
+setuptools
+numpy
+numba
+pandas
+requests
+pandarallel
+jax
+cython

phylotypy-0.2.0/pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["setuptools>=45", "wheel", "Cython>=0.29.21", "numpy"]
+build-backend = "setuptools.build_meta"
+
+[tool.cython]
+language_level = "3"
+
+[tool.setuptools]
+include-package-data = true
+
+[tool.setuptools.package-data]
+"*" = ["*.pyx", "*.pxd"]
+
+[project]
+name = "phylotypy"
+version = "0.2.0"
+description = "Naive Bayes Classifier with Rust-accelerated taxonomy functions"
+authors = [
+    {name = "Chad Saltikov", email = "saltikov@ucsc.edu"}
+]
+license = {text = "MIT"}
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "setuptools",
+    "numpy~=2.1.3",
+    "numba~=0.61.0",
+    "pandas~=2.2.3",
+    "requests",
+    "pandarallel",
+    "jax",
+    "cython"
+]
+
+[project.optional-dependencies]
+test = [
+    "pytest",
+]
+

phylotypy-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

phylotypy-0.2.0/setup.py

@@ -0,0 +1,30 @@
+from setuptools import setup, Extension, find_packages
+from Cython.Build import cythonize
+import numpy as np
+
+
+extensions = [
+    Extension(
+        "phylotypy.classify_bootstraps.classify_bootstraps",
+        ["src/phylotypy/classify_bootstraps/classify_bootstraps.pyx"],
+        include_dirs=[np.get_include()],
+    ),
+        # New cond_prob_cython Cython extension
+    Extension(
+        "phylotypy.cond_prob_c.cond_prob_cython",
+        ["src/phylotypy/cond_prob_c/cond_prob_cython.pyx"],
+        include_dirs=[np.get_include()],
+        extra_compile_args=['-O3'],
+        define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")]
+    )
+]
+
+setup(
+    name="phylotypy",
+    package_dir={"": "src"},
+    packages=find_packages(where="src"),
+    ext_modules=cythonize(
+        extensions,
+        compiler_directives={'language_level': "3"},
+        annotate=True),
+)

phylotypy-0.2.0/src/phylotypy/__init__.py

@@ -0,0 +1,4 @@
+from .cond_prob_c import cond_prob_cython
+from .classify_bootstraps import classify_bootstraps_cython
+from .utilities import read_fasta
+from .utilities import utilities

phylotypy-0.2.0/src/phylotypy/bootstrap.py

@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+
+##
+from collections import Counter
+from functools import partial
+from itertools import repeat
+from pathlib import Path
+
+import numpy as np
+from numpy.typing import NDArray
+
+from phylotypy import kmers, classifier
+from phylotypy.utilities import read_fasta
+
+
+##
+def bootstrap(arr: list | np.ndarray, divider: int = 8, num_bs: int = 100) -> NDArray:
+    bootstrap_fn = partial(bootstrap_kmers, arr, divider)
+    return np.array(list(map(lambda _: bootstrap_fn(), repeat(1, num_bs))))
+
+
+def bootstrap_kmers(kmer_arr: np.array, kmer_size: int = 8):
+    """Performs a single bootstrap sampling on a kmers array"""
+    valid_kmers = kmer_arr[kmer_arr != -1]
+    num_removed_kmers = len(kmer_arr) - len(valid_kmers)
+    n_kmers = len(valid_kmers) + num_removed_kmers
+    return np.random.choice(valid_kmers, n_kmers // kmer_size, replace=True)
+
+
+def split_taxa_arr(taxa_arr: np.ndarray) -> np.ndarray:
+    """
+    Split array of taxonomy strings into a 2D array where each row is a
+    sample and each column is a taxonomic level.
+    """
+    return np.array([taxa_str.split(";") for taxa_str in taxa_arr])
+
+
+def bootstrap_consensus_helper(arr):
+    ids, scores = np.apply_along_axis(kmers.get_consensus, axis=0, arr=arr)
+    return ids, scores
+
+
+def sort_by_indices_helper(values_to_sort, indices_array):
+    sorted_indices = np.argsort(indices_array)
+    return values_to_sort[sorted_indices]
+
+
+##
+def bootstrap_consensus(classified_bs_kmers: np.ndarray, genera_names: np.ndarray):
+    """
+    Find consensus taxonomy from bootstrap samples.
+
+    Parameters:
+    classified_bs_kmers: 1D array of integers representing bootstrap samples
+    database: Database object containing genera_names mapping;
+                database = {conditional_prob:np.array
+                        genera_idx mapping}
+
+    Returns:
+    Dictionary with taxonomy consensus and confidence values
+    """
+    # Convert the 1D array of ints (classified_bs_kmers) to taxonomy strings
+    # using genera_names 1D array
+    taxa = genera_names[classified_bs_kmers]
+
+    # Split into a 2D array of (100,6)
+    res = split_taxa_arr(taxa)
+    n_levels = res.shape[1]
+
+    taxa_consensus = np.empty(n_levels, dtype=object)
+    confidence_consensus = np.zeros(n_levels, dtype=int)
+
+    # Use Counter directly instead of apply_along_axis with custom function
+    for i in range(n_levels):
+        counter = Counter(res[:, i])
+        most_common = counter.most_common(1)[0]  # Returns (taxon, count)
+        taxa_consensus[i] = most_common[0]
+        confidence_consensus[i] = most_common[1]
+
+    return dict(taxonomy=taxa_consensus, confidence=confidence_consensus)
+
+
+##
+if __name__ == "__main__":
+    from time import perf_counter
+
+    path = Path(__file__).parent
+    rdp_small_fasta =  path / "../../data/trainset19_072023_small_db.fasta"
+
+    moving_pics = read_fasta.read_taxa_fasta(path / "../../data/dna_moving_pictures.fasta")
+    rdp_df = read_fasta.read_taxa_fasta(rdp_small_fasta)
+
+    start = perf_counter()
+    database = classifier.make_classifier(rdp_df)
+    end = perf_counter()
+    print(f"Time taken: {(end - start):.2f}")
+
+    start = perf_counter()
+    database1 = kmers.build_kmer_database(rdp_df["sequence"], rdp_df["id"], verbose=True)
+    end = perf_counter()
+    print(f"Time taken: {(end - start):.2f}")
+
+    print(np.allclose(database.conditional_prob, database1.conditional_prob))
+    print(np.allclose(database.genera_idx, database1.genera_idx))
+    print(np.allclose(database.genera_names, database1.genera_names))