bscampp 1.0.1a0__tar.gz → 1.0.2__tar.gz

bscampp-1.0.2/CHANGELOG.md

@@ -0,0 +1,16 @@
+# BSCAMPP v1.0.2
+1. Added SCAMPP funtionality and its binary executables.
+
+# BSCAMPP v1.0.1
+1. Bumped version to full release.
+2. Completed examples for display in `bscampp --help`.
+
+# BSCAMPP v1.0.1b
+1. Removed redundant codes and fixed missing variables.
+2. Added badges for PyPI installation and current Python Build, etc.
+3. Added a single Pytest for dry-running BSCAMPP under `tests/`.
+4. For 3, added the `dry_run` parameter to function `pipeline.py:bscampp_pipeline`.
+
+# BSCAMPP v1.0.1a
+1. Completed features with both `epa-ng` and `pplacer` support.
+2. Refactorized all codes and worked out the PyPI installation for release.

{bscampp-1.0.1a0/bscampp.egg-info → bscampp-1.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: bscampp
-Version: 1.0.1a0
+Version: 1.0.2
 Summary: BSCAMPP - A Scalable Phylogenetic Placement Tool
 Author-email: Eleanor Wedell <ewedell2@illinois.edu>, Chengze Shen <chengze5@illinois.edu>
 License: MIT License
@@ -51,6 +51,11 @@ Requires-Dist: treeswift>=1.1.45
 Requires-Dist: taxtastic>=0.9.3
 
 # BSCAMPP - A Scalable Phylogenetic Placement Method and Framework
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/bscampp)](https://pypi.org/project/bscampp/)
+[![PyPI - Version](https://img.shields.io/pypi/v/bscampp?color=blue)](https://pypi.org/project/bscampp/#history)
+[![Build Status](https://img.shields.io/github/actions/workflow/status/ewedell/BSCAMPP/python-package.yml?branch=main&label=build)](https://github.com/ewedell/BSCAMPP/)
+[![PyPI - License](https://img.shields.io/pypi/l/bscampp?color=blue)](https://github.com/ewedell/BSCAMPP/blob/main/LICENSE)
+[![Changelog](https://img.shields.io/badge/CHANGELOG-grey)](https://github.com/ewedell/BSCAMPP/blob/main/CHANGELOG.md)
 
 **Table of Contents**
 1. [Overview](#overview)
@@ -83,8 +88,12 @@ It is recommended that BSCAMPP be used with subtrees of size 2000 and with 5 vot
 are fragmentary. Defaults for the subtree size and number of votes are set to 2,000 and 5 respectively (see [Usage](#usage) for more details
 on customizing BSCAMPP).
 
+#### SCAMPP
+SCAMPP is also implemented in BSCAMPP. The user can invoke SCAMPP by running
+`run_scampp.py` or `scampp` (if installed with PyPI) after installation.
+
 # Installation
-BSCAMPP was tested on **Python 3.7 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
+BSCAMPP was tested on **Python 3.8 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
 (2) from this GitHub repository. If you have any difficulties installing or running BSCAMPP, please contact Eleanor Wedell
 (ewedell@illinois.edu).
 
@@ -93,19 +102,25 @@ EPA-ng and/or pplacer are requirements to run BSCAMPP since BSCAMPP will use the
 By default, BSCAMPP will search for binary executables of `pplacer` and `epa-ng` in the user's environment when running for the first time.
 We also included a compiled version of `pplacer` for the Linux system under `bscampp/tools`.
 
-### (1) Install with `pip` (Coming soon)
+### (1) Install with `pip`
 The easiest way to install BSCAMPP is to use `pip install`. This will also install all required Python packages.
 
 ```bash
 # 1. install with pip (--user if no root access)
 pip install bscampp [--user]
 
-# 2. Two binary executables will be installed. The first time
+# 2. Four binary executables will be installed. The first time
 #    running any will create a config file at
 #    ~/.bscampp/main.config that resolves the links to all
 #    external software (e.g., epa-ng, pplacer)
+
+# ---- BSCAMPP functions
 bscampp [-h]    # or
 run_bscampp.py [-h]
+
+# ---- SCAMPP functions
+scampp  [-h]    # or
+run_scampp.py
 ```
 
 ### (2) Install from GitHub
@@ -155,7 +170,8 @@ run_bscampp.py -i [raxml best model] -t [reference tree] -a [reference alignment
 ### (3) Using `pplacer` as the base placement method
 ```bash
 run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
-    -a [reference alignment] -q [query sequence alignment]
+    -a [reference alignment] -q [query sequence alignment] \
+    --placement-method pplacer
 ```
 
 ### More comprehensive usage
@@ -216,14 +232,23 @@ run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
 >                         Temporary file indexing. Default: 0
 >   --fragmentflag FRAGMENTFLAG
 >                         If queries contains fragments. Default: True
+>  --subtreetype SUBTREETYPE
+>                         (SCAMPP only) Options for collecting nodes for the
+>                         subtree - d for edge weighted distances, n for node
+>                         distances, h for Hamming distances. Default: d
 >   --keeptemp KEEPTEMP   Boolean, True to keep all temporary files. Default:
                         False
 ```
 
 
 # Example Code and Data
-Example script and data are provided in this GitHub repository in `examples/`. The data is originally from the [RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
-* `examples/run.sh`: contains a simple script to test BSCAMPP with `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement tree.
-  The info file is from RAxML-ng when running `epa-ng`, and from FastTree-2 when running `pplacer`.
-  - `run.sh` will invoke BSCAMPP with `epa-ng`.
-  - `run.sh pplacer` will invoke BSCAMPP with `pplacer`.
+Example script and data are provided in this GitHub repository in `examples/`.
+The data is originally from the
+[RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
+* `examples/run_bscampp.sh`: contains a simple script to test BSCAMPP with
+  `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement
+  tree. The info file is from RAxML-ng when running `epa-ng`, and from
+  FastTree-2 when running `pplacer`.
+  - `run_bscampp.sh` will invoke BSCAMPP with `epa-ng`.
+  - `run_bscampp.sh pplacer` will invoke BSCAMPP with `pplacer`.
+* `examples/run_scampp.sh`: the same test script but running SCAMPP.

{bscampp-1.0.1a0 → bscampp-1.0.2}/README.md

@@ -1,4 +1,9 @@
 # BSCAMPP - A Scalable Phylogenetic Placement Method and Framework
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/bscampp)](https://pypi.org/project/bscampp/)
+[![PyPI - Version](https://img.shields.io/pypi/v/bscampp?color=blue)](https://pypi.org/project/bscampp/#history)
+[![Build Status](https://img.shields.io/github/actions/workflow/status/ewedell/BSCAMPP/python-package.yml?branch=main&label=build)](https://github.com/ewedell/BSCAMPP/)
+[![PyPI - License](https://img.shields.io/pypi/l/bscampp?color=blue)](https://github.com/ewedell/BSCAMPP/blob/main/LICENSE)
+[![Changelog](https://img.shields.io/badge/CHANGELOG-grey)](https://github.com/ewedell/BSCAMPP/blob/main/CHANGELOG.md)
 
 **Table of Contents**
 1. [Overview](#overview)
@@ -31,8 +36,12 @@ It is recommended that BSCAMPP be used with subtrees of size 2000 and with 5 vot
 are fragmentary. Defaults for the subtree size and number of votes are set to 2,000 and 5 respectively (see [Usage](#usage) for more details
 on customizing BSCAMPP).
 
+#### SCAMPP
+SCAMPP is also implemented in BSCAMPP. The user can invoke SCAMPP by running
+`run_scampp.py` or `scampp` (if installed with PyPI) after installation.
+
 # Installation
-BSCAMPP was tested on **Python 3.7 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
+BSCAMPP was tested on **Python 3.8 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
 (2) from this GitHub repository. If you have any difficulties installing or running BSCAMPP, please contact Eleanor Wedell
 (ewedell@illinois.edu).
 
@@ -41,19 +50,25 @@ EPA-ng and/or pplacer are requirements to run BSCAMPP since BSCAMPP will use the
 By default, BSCAMPP will search for binary executables of `pplacer` and `epa-ng` in the user's environment when running for the first time.
 We also included a compiled version of `pplacer` for the Linux system under `bscampp/tools`.
 
-### (1) Install with `pip` (Coming soon)
+### (1) Install with `pip`
 The easiest way to install BSCAMPP is to use `pip install`. This will also install all required Python packages.
 
 ```bash
 # 1. install with pip (--user if no root access)
 pip install bscampp [--user]
 
-# 2. Two binary executables will be installed. The first time
+# 2. Four binary executables will be installed. The first time
 #    running any will create a config file at
 #    ~/.bscampp/main.config that resolves the links to all
 #    external software (e.g., epa-ng, pplacer)
+
+# ---- BSCAMPP functions
 bscampp [-h]    # or
 run_bscampp.py [-h]
+
+# ---- SCAMPP functions
+scampp  [-h]    # or
+run_scampp.py
 ```
 
 ### (2) Install from GitHub
@@ -103,7 +118,8 @@ run_bscampp.py -i [raxml best model] -t [reference tree] -a [reference alignment
 ### (3) Using `pplacer` as the base placement method
 ```bash
 run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
-    -a [reference alignment] -q [query sequence alignment]
+    -a [reference alignment] -q [query sequence alignment] \
+    --placement-method pplacer
 ```
 
 ### More comprehensive usage
@@ -164,14 +180,23 @@ run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
 >                         Temporary file indexing. Default: 0
 >   --fragmentflag FRAGMENTFLAG
 >                         If queries contains fragments. Default: True
+>  --subtreetype SUBTREETYPE
+>                         (SCAMPP only) Options for collecting nodes for the
+>                         subtree - d for edge weighted distances, n for node
+>                         distances, h for Hamming distances. Default: d
 >   --keeptemp KEEPTEMP   Boolean, True to keep all temporary files. Default:
                         False
 ```
 
 
 # Example Code and Data
-Example script and data are provided in this GitHub repository in `examples/`. The data is originally from the [RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
-* `examples/run.sh`: contains a simple script to test BSCAMPP with `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement tree.
-  The info file is from RAxML-ng when running `epa-ng`, and from FastTree-2 when running `pplacer`.
-  - `run.sh` will invoke BSCAMPP with `epa-ng`.
-  - `run.sh pplacer` will invoke BSCAMPP with `pplacer`.
+Example script and data are provided in this GitHub repository in `examples/`.
+The data is originally from the
+[RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
+* `examples/run_bscampp.sh`: contains a simple script to test BSCAMPP with
+  `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement
+  tree. The info file is from RAxML-ng when running `epa-ng`, and from
+  FastTree-2 when running `pplacer`.
+  - `run_bscampp.sh` will invoke BSCAMPP with `epa-ng`.
+  - `run_bscampp.sh pplacer` will invoke BSCAMPP with `pplacer`.
+* `examples/run_scampp.sh`: the same test script but running SCAMPP.

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp/__init__.py

@@ -12,7 +12,7 @@ import logging, os
 # not really needed for BSCAMPP but safe to update here
 os.sys.setrecursionlimit(1000000)
 
-__version__ = "1.0.1a"
+__version__ = "1.0.2"
 _INSTALL_PATH = __path__[0]
 
 # global variables to store all loggers

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp/configs.py

@@ -50,6 +50,7 @@ class Configs:
     # miscellaneous
     tmpfilenbr = 0
     fragmentflag = True
+    subtreetype = 'd'
 
 # check if the given configuration is valid to add
 def set_valid_configuration(name, conf):

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp/functions.py

@@ -15,10 +15,13 @@ Function to read in the placement tree and alignment.
 If query alignment is provided, will use the provided query instead of
 the ones (potentially) included in the reference alignment
 '''
-def readData(workdir):
+def readData(workdir, dry_run=False):
     t0 = time.perf_counter()
     _LOG.info('Reading in input data...')
 
+    if dry_run:
+        return None, dict(), '', dict(), '', dict()
+
     # (1) load reference tree
     tree = treeswift.read_tree_newick(Configs.tree_path)
     tree.resolve_polytomies()
@@ -45,10 +48,10 @@ def readData(workdir):
         # after separating queries from the reference alignment, write
         # them to to TEMP/
         qaln_path = os.path.join(workdir, 'qaln.fa')
-        write_fasta(temp_qaln_path, q_dict)
+        write_fasta(qaln_path, q_dict)
         
         aln_path = os.path.join(workdir, 'aln.fa')
-        write_fasta(temp_aln_path, ref_dict)
+        write_fasta(aln_path, ref_dict)
 
     t1 = time.perf_counter()
     _LOG.info('Time to read in input data: {} seconds'.format(t1 - t0))
@@ -58,19 +61,25 @@ def readData(workdir):
 Function to get the closest leaf for each query sequence based on Hamming
 distance
 '''
-def getClosestLeaves(aln_path, qaln_path, aln, qaln, workdir):
+def getClosestLeaves(aln_path, qaln_path, aln, qaln, workdir, dry_run=False):
     t0 = time.perf_counter()
     _LOG.info('Computing closest leaves for query sequences...')
+
+    if dry_run:
+        return dict(), dict()
+
     query_votes_dict = dict()
     query_top_vote_dict = dict()
-    
     tmp_output = os.path.join(workdir, 'closest.txt') 
+    
+    if Configs.subtreetype == "h":
+        Configs.votes = Configs.subtreesize
 
     cmd = []
     if Configs.similarityflag:
         cmd.append(os.path.join(Configs.hamming_distance_dir, 'homology'))
     else:
-        if fragment_flag == False:
+        if Configs.fragmentflag == False:
             cmd.append(os.path.join(Configs.hamming_distance_dir, 'hamming'))
         else: 
             cmd.append(os.path.join(
@@ -115,10 +124,13 @@ def getClosestLeaves(aln_path, qaln_path, aln, qaln, workdir):
 Function to assign queries to subtrees based on their votes
 '''
 def assignQueriesToSubtrees(query_votes_dict, query_top_vote_dict,
-        tree, leaf_dict):
+        tree, leaf_dict, dry_run=False):
     t0 = time.perf_counter()
     _LOG.info('Adding query votes to the placement tree...')
 
+    if dry_run:
+        return dict(), []
+
     # (1) go over the query votes and add them to corresponding leaves
     lf_votes = Counter()
     leaf_queries = dict()
@@ -217,6 +229,51 @@ def assignQueriesToSubtrees(query_votes_dict, query_top_vote_dict,
     _LOG.info('Time to assign queries to subtrees: {} seconds'.format(t1 - t0))
     return new_subtree_dict, placed_query_list
 
+
+'''
+Function to assign queries to subtrees as used in SCAMPP 
+(subtrees are built using the nearest leaf as the seed sequence)
+'''
+def buildQuerySubtrees(query_votes_dict, query_top_vote_dict,
+        tree, leaf_dict, dry_run=False):
+    t0 = time.perf_counter()
+
+    if dry_run:
+        return dict(), []
+
+    # (1) go over the query seed sequences to see if any queries use 
+    # the same seed sequence (i.e. subtree)
+    seed_queries = dict()
+    for query, closest_leaf in query_top_vote_dict.items():
+        if closest_leaf not in seed_queries:           
+            seed_queries[closest_leaf] = [query]
+        else:
+            seed_queries[closest_leaf].append(query)
+
+    new_subtree_dict = dict()
+    # assign queries to subtrees, and remove them from the pool
+    # repeat until all queries are assigned
+    for seed_label, queries in seed_queries.items():
+        node_y = leaf_dict[seed_label]
+        # extract [subtreesize] leaves
+        if Configs.subtreetype == "h":
+            labels = query_votes_dict[queries[0]]
+        elif Configs.subtreetype == "n":
+            labels = utils.subtree_nodes(tree, node_y, Configs.subtreesize)
+        else:
+            labels = utils.subtree_nodes_with_edge_length(tree, node_y,
+                Configs.subtreesize)
+        subtree = tree.extract_tree_with(labels)
+        new_subtree_dict[subtree] = queries
+
+            
+    placed_query_list = []
+    
+    t1 = time.perf_counter()
+    _LOG.info('Time to assign queries to subtrees: {} seconds'.format(t1 - t0))
+    return new_subtree_dict, placed_query_list
+
+
 '''
 Helper function to run a single placement task. Designed to use with
 multiprocessing
@@ -229,10 +286,13 @@ def placeOneSubtree():
 Function to perform placement of queries for each subtree
 '''
 def placeQueriesToSubtrees(tree, leaf_dict, new_subtree_dict, placed_query_list,
-        aln, qaln, cmdline_args, workdir, pool, lock):
+        aln, qaln, cmdline_args, workdir, pool, lock, dry_run=False):
     t0 = time.perf_counter()
     _LOG.info('Performing placement on each subtree...')
 
+    if dry_run:
+        return dict()
+
     # prepare to write an aggregated results to local
     jplace = dict()
     utils.add_edge_nbrs(tree)
@@ -381,9 +441,12 @@ def placeQueriesToSubtrees(tree, leaf_dict, new_subtree_dict, placed_query_list,
 '''
 Function to write a given jplace object to local output
 '''
-def writeOutputJplace(output_jplace):
+def writeOutputJplace(output_jplace, dry_run=False):
     t0 = time.perf_counter()
     _LOG.info('Writing aggregated placements to local...')
+    
+    if dry_run:
+        return
 
     outpath = os.path.join(Configs.outdir, Configs.outname)
     outf = open(outpath, 'w')

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp/pipeline.py

@@ -22,8 +22,13 @@ def bscampp_pipeline(*args, **kwargs):
     t0 = time.perf_counter()
     m = Manager(); lock = m.Lock()
 
+    # set up a dry run if specified
+    dry_run = False
+    if 'dry_run' in kwargs and isinstance(kwargs['dry_run'], bool):
+        dry_run = kwargs['dry_run']
+
     # parse command line arguments and build configurations
-    parser, cmdline_args = parseArguments()
+    parser, cmdline_args = parseArguments(dry_run=dry_run)
 
     # initialize multiprocessing (if needed)
     _LOG.warning('Initializing ProcessPoolExecutor...')
@@ -31,33 +36,39 @@ def bscampp_pipeline(*args, **kwargs):
             initargs=(parser, cmdline_args,))
 
     # (0) temporary files wrote to here
-    workdir = os.path.join(Configs.outdir, f'tmp{Configs.tmpfilenbr}')
-    try:
-        if not os.path.isdir(workdir):
-            os.makedirs(workdir)
-    except OSError:
-        log_exception(_LOG)
+    if not dry_run:
+        workdir = os.path.join(Configs.outdir, f'tmp{Configs.tmpfilenbr}')
+        try:
+            if not os.path.isdir(workdir):
+                os.makedirs(workdir)
+        except OSError:
+            log_exception(_LOG)
+    else:
+        workdir = os.getcwd()
 
     # (1) read in tree, alignment, and separate reference sequences from
     # query sequences
-    tree, leaf_dict, aln_path, aln, qaln_path, qaln = readData(workdir)
+    tree, leaf_dict, aln_path, aln, qaln_path, qaln = readData(workdir,
+            dry_run=dry_run)
 
     # (2) compute closest leaves for all query sequences
     query_votes_dict, query_top_vote_dict = getClosestLeaves(
-            aln_path, qaln_path, aln, qaln, workdir)
+            aln_path, qaln_path, aln, qaln, workdir, dry_run=dry_run)
 
     # (3) first assign all queries to their closest-leaf subtrees,
     # then do reassignment to minimize distance between each's top vote
     # and the subtree's seed leaf
     new_subtree_dict, placed_query_list = assignQueriesToSubtrees(
-            query_votes_dict, query_top_vote_dict, tree, leaf_dict)
+            query_votes_dict, query_top_vote_dict, tree, leaf_dict,
+            dry_run=dry_run)
 
     # (4) perform placement for each subtree
     output_jplace = placeQueriesToSubtrees(tree, leaf_dict, new_subtree_dict,
-            placed_query_list, aln, qaln, cmdline_args, workdir, pool, lock)
+            placed_query_list, aln, qaln, cmdline_args, workdir, pool, lock,
+            dry_run=dry_run)
 
     # (5) write the output jplace to local
-    writeOutputJplace(output_jplace)
+    writeOutputJplace(output_jplace, dry_run=dry_run)
 
     # shutdown pool
     _LOG.warning('Shutting down ProcessPoolExecutor...')
@@ -73,6 +84,84 @@ def bscampp_pipeline(*args, **kwargs):
     send = time.perf_counter()
     _LOG.info('BSCAMPP completed in {} seconds...'.format(send - t0))
 
+    if dry_run:
+        return True
+    else:
+        return False
+
+
+# main pipeline for SCAMPP
+def scampp_pipeline(*args, **kwargs):
+    t0 = time.perf_counter()
+    m = Manager(); lock = m.Lock()
+
+    # set up a dry run if specified
+    dry_run = False
+    if 'dry_run' in kwargs and isinstance(kwargs['dry_run'], bool):
+        dry_run = kwargs['dry_run']
+
+    # parse command line arguments and build configurations
+    parser, cmdline_args = parseArguments(dry_run=dry_run, method="SCAMPP")
+
+    # initialize multiprocessing (if needed)
+    _LOG.warning('Initializing ProcessPoolExecutor...')
+    pool = ProcessPoolExecutor(Configs.num_cpus, initializer=initial_pool,
+            initargs=(parser, cmdline_args,))
+
+    # (0) temporary files wrote to here
+    if not dry_run:
+        workdir = os.path.join(Configs.outdir, f'tmp{Configs.tmpfilenbr}')
+        try:
+            if not os.path.isdir(workdir):
+                os.makedirs(workdir)
+        except OSError:
+            log_exception(_LOG)
+    else:
+        workdir = os.getcwd()
+
+    # (1) read in tree, alignment, and separate reference sequences from
+    # query sequences
+    tree, leaf_dict, aln_path, aln, qaln_path, qaln = readData(workdir,
+            dry_run=dry_run)
+
+    # (2) compute closest leaves for all query sequences
+    query_votes_dict, query_top_vote_dict = getClosestLeaves(
+            aln_path, qaln_path, aln, qaln, workdir, dry_run=dry_run)
+
+    # (3) first assign each query to the subtree built using the closest 
+    # leaf as the seed sequence 
+    new_subtree_dict, placed_query_list = buildQuerySubtrees(
+            query_votes_dict, query_top_vote_dict, tree, leaf_dict,
+            dry_run=dry_run)
+
+    # (4) perform placement for each subtree
+    output_jplace = placeQueriesToSubtrees(tree, leaf_dict, new_subtree_dict,
+            placed_query_list, aln, qaln, cmdline_args, workdir, pool, lock,
+            dry_run=dry_run)
+
+    # (5) write the output jplace to local
+    writeOutputJplace(output_jplace, dry_run=dry_run)
+
+    # shutdown pool
+    _LOG.warning('Shutting down ProcessPoolExecutor...')
+    pool.shutdown()
+    _LOG.warning('ProcessPoolExecutor shut down.')
+    
+    # clean up temp files if not keeping
+    if not Configs.keeptemp:
+        _LOG.info('Removing temporary files...')
+        clean_temp_files()
+
+    # stop SCAMPP 
+    send = time.perf_counter()
+    _LOG.info('SCAMPP completed in {} seconds...'.format(send - t0))
+
+    if dry_run:
+        return True
+    else:
+        return False
+
+
 def clean_temp_files():
     # all temporary files/directories to remove
     temp_items = [f'tmp{Configs.tmpfilenbr}']
@@ -86,29 +175,43 @@ def clean_temp_files():
             continue
         _LOG.info(f'- Removed {temp}')
 
-def parseArguments():
+def parseArguments(dry_run=False, method="BSCAMPP"):
     global _root_dir, main_config_path
-    parser = _init_parser()
+
+    default_outdir = f"{method.lower()}_output"
+    default_outname = f"{method.lower()}_result"
+
+    parser = _init_parser(default_outdir=default_outdir,
+            default_outname=default_outname)
     cmdline_args = sys.argv[1:]
+
+    if dry_run:
+        cmdline_args = ['-i', 'dummy.info', '-t', 'dummy.tre',
+                '-a', 'dummy.fa'] 
     
     # build config
     buildConfigs(parser, cmdline_args)
-    _LOG.info('BSCAMPP is running with: {}'.format(
+    _LOG.info('{} is running with: {}'.format(method,
         ' '.join(cmdline_args)))
     getConfigs()
 
     return parser, cmdline_args
 
-def _init_parser():
+def _init_parser(default_outdir="bscampp_output",
+        default_outname="bscampp_result"):
     # example usage
     example_usages = '''Example usages:
-> default
-    %(prog)s -i raxml.info 
+> (1) Default
+    %(prog)s -i raxml.bestModel -t reference.tre -a alignment.fa  
+> (2) Separate alignment file for query sequences
+    %(prog)s -i raxml.bestModel -t reference.tre -a reference.fa -q query.fa
+> (3) Use pplacer instead of EPA-ng as base method (need RAxML-ng info or FastTree log file)
+    %(prog)s -i fasttree.log -t reference.tre -a alignment.fa --placement-method pplacer
 '''
 
     parser = ArgumentParser(
             description=(
-                "This program runs BSCAMPP, a scalable phylogenetic "
+                "This program runs BSCAMPP/SCAMPP, a scalable phylogenetic "
                 "placement framework that scales EPA-ng/pplacer "
                 "to very large tree placement."
                 ),
@@ -135,7 +238,7 @@ def _init_parser():
     # basic group
     basic_group = parser.add_argument_group(
             "Basic parameters".upper(),
-            "These are the basic parameters for BSCAMPP.")
+            "These are the basic parameters for BSCAMPP/SCAMPP.")
     parser.groups['basic_group'] = basic_group
 
     basic_group.add_argument('--placement-method', type=str,
@@ -164,10 +267,10 @@ def _init_parser():
                   required=False, default=None)
     basic_group.add_argument("-d", "--outdir", type=str,
                   help="Directory path for output. Default: bscampp_output/",
-                  required=False, default="bscampp_output")
+                  required=False, default=default_outdir)
     basic_group.add_argument("-o", "--output", type=str, dest="outname",
                   help="Output file name. Default: bscampp_result.jplace",
-                  required=False, default="bscampp_result.jplace")
+                  required=False, default=f"{default_outname}.jplace")
     basic_group.add_argument("--threads", "--num-cpus", type=int,
                   dest="num_cpus",
                   help="Number of cores for parallelization, default: -1 (all)",
@@ -188,7 +291,8 @@ def _init_parser():
                   help="Integer size of the subtree. Default: 2000",
                   required=False, default=2000)
     advance_group.add_argument("-V", "--votes", type=int,
-                  help="Number of votes per query sequence. Default: 5",
+                  help="This is only used for BSCAMPP! Number of votes per "
+                  "query sequence. Default: 5",
                   required=False, default=5)
     advance_group.add_argument("--similarityflag", type=str2bool,
                   help="Boolean, True if maximizing sequence similarity "
@@ -207,6 +311,12 @@ def _init_parser():
     misc_group.add_argument("--fragmentflag", type=str2bool,
                   help="If queries contains fragments. Default: True",
                   required=False, default=True)
+    misc_group.add_argument("--subtreetype", type=str,
+                  help="(SCAMPP only) Options for collecting "
+                  "nodes for the subtree - d for edge weighted "
+                  "distances, n for node distances, h for Hamming "
+                  "distances. Default: d",
+                  required=False, default='d')
     misc_group.add_argument("--keeptemp", type=str2bool,
                   help="Boolean, True to keep all temporary files. "
                   "Default: False",

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp/utils.py

@@ -8,6 +8,7 @@ from os.path import expanduser,isfile
 import random
 import statistics
 import copy
+import gzip
 
 import argparse
 # reformat argparse help text formatting
@@ -813,25 +814,25 @@ def newick_edge_tokens_node(node):
             node_to_str[node] = ''.join(out)
     return node_to_str[node]
 
-def write_tree_newick_edge_tokens(tree, filename, hide_rooted_prefix=False):
-    '''
-    Modified from treeswift tree.write_tree_newick()
-    Write this ``Tree`` to a Newick file
-       Args:
-        ``filename`` (``str``): Path to desired output file (plain-text or gzipped)
-    '''
-    if not isinstance(filename, str):
-        raise TypeError("filename must be a str")
-    treestr = newick_edge_nbr_string(tree)
-    if hide_rooted_prefix:
-        if treestr.startswith('[&R]'):
-            treestr = treestr[4:].strip()
-    else:
-        warn("Specified hide_rooted_prefix, but tree was not rooted")
-    if filename.lower().endswith('.gz'): # gzipped file
-        f = gopen(expanduser(filename),'wb',9); f.write(treestr.encode()); f.close()
-    else: # plain-text file
-        f = open(expanduser(filename),'w'); f.write(treestr); f.close()
+#def write_tree_newick_edge_tokens(tree, filename, hide_rooted_prefix=False):
+#    '''
+#    Modified from treeswift tree.write_tree_newick()
+#    Write this ``Tree`` to a Newick file
+#       Args:
+#        ``filename`` (``str``): Path to desired output file (plain-text or gzipped)
+#    '''
+#    if not isinstance(filename, str):
+#        raise TypeError("filename must be a str")
+#    treestr = newick_edge_nbr_string(tree)
+#    if hide_rooted_prefix:
+#        if treestr.startswith('[&R]'):
+#            treestr = treestr[4:].strip()
+#    else:
+#        warn("Specified hide_rooted_prefix, but tree was not rooted")
+#    if filename.lower().endswith('.gz'): # gzipped file
+#        f = gopen(expanduser(filename),'wb',9); f.write(treestr.encode()); f.close()
+#    else: # plain-text file
+#        f = open(expanduser(filename),'w'); f.write(treestr); f.close()
 
 def read_tree_newick_edge_tokens(newick):
     '''
@@ -850,7 +851,7 @@ def read_tree_newick_edge_tokens(newick):
         except:
             raise TypeError("newick must be a str")
     if newick.lower().endswith('.gz'): # gzipped file
-        f = gopen(expanduser(newick)); ts = f.read().decode().strip(); f.close()
+        f = gzip.open(expanduser(newick)); ts = f.read().decode().strip(); f.close()
     elif isfile(expanduser(newick)): # plain-text file
         f = open(expanduser(newick)); ts = f.read().strip(); f.close()
     else:
@@ -867,7 +868,7 @@ def read_tree_newick_edge_tokens(newick):
             # end of Newick string
             if ts[i] == ';':
                 if i != len(ts)-1 or n != t.root:
-                    raise RuntimeError(INVALID_NEWICK)
+                    raise RuntimeError("INVALID NEWICK")
 
             # go to new child
             elif ts[i] == '(':

{bscampp-1.0.1a0 → bscampp-1.0.2/bscampp.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: bscampp
-Version: 1.0.1a0
+Version: 1.0.2
 Summary: BSCAMPP - A Scalable Phylogenetic Placement Tool
 Author-email: Eleanor Wedell <ewedell2@illinois.edu>, Chengze Shen <chengze5@illinois.edu>
 License: MIT License
@@ -51,6 +51,11 @@ Requires-Dist: treeswift>=1.1.45
 Requires-Dist: taxtastic>=0.9.3
 
 # BSCAMPP - A Scalable Phylogenetic Placement Method and Framework
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/bscampp)](https://pypi.org/project/bscampp/)
+[![PyPI - Version](https://img.shields.io/pypi/v/bscampp?color=blue)](https://pypi.org/project/bscampp/#history)
+[![Build Status](https://img.shields.io/github/actions/workflow/status/ewedell/BSCAMPP/python-package.yml?branch=main&label=build)](https://github.com/ewedell/BSCAMPP/)
+[![PyPI - License](https://img.shields.io/pypi/l/bscampp?color=blue)](https://github.com/ewedell/BSCAMPP/blob/main/LICENSE)
+[![Changelog](https://img.shields.io/badge/CHANGELOG-grey)](https://github.com/ewedell/BSCAMPP/blob/main/CHANGELOG.md)
 
 **Table of Contents**
 1. [Overview](#overview)
@@ -83,8 +88,12 @@ It is recommended that BSCAMPP be used with subtrees of size 2000 and with 5 vot
 are fragmentary. Defaults for the subtree size and number of votes are set to 2,000 and 5 respectively (see [Usage](#usage) for more details
 on customizing BSCAMPP).
 
+#### SCAMPP
+SCAMPP is also implemented in BSCAMPP. The user can invoke SCAMPP by running
+`run_scampp.py` or `scampp` (if installed with PyPI) after installation.
+
 # Installation
-BSCAMPP was tested on **Python 3.7 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
+BSCAMPP was tested on **Python 3.8 to 3.12**. There are two ways to install and use BSCAMPP: (1) with PyPI, or
 (2) from this GitHub repository. If you have any difficulties installing or running BSCAMPP, please contact Eleanor Wedell
 (ewedell@illinois.edu).
 
@@ -93,19 +102,25 @@ EPA-ng and/or pplacer are requirements to run BSCAMPP since BSCAMPP will use the
 By default, BSCAMPP will search for binary executables of `pplacer` and `epa-ng` in the user's environment when running for the first time.
 We also included a compiled version of `pplacer` for the Linux system under `bscampp/tools`.
 
-### (1) Install with `pip` (Coming soon)
+### (1) Install with `pip`
 The easiest way to install BSCAMPP is to use `pip install`. This will also install all required Python packages.
 
 ```bash
 # 1. install with pip (--user if no root access)
 pip install bscampp [--user]
 
-# 2. Two binary executables will be installed. The first time
+# 2. Four binary executables will be installed. The first time
 #    running any will create a config file at
 #    ~/.bscampp/main.config that resolves the links to all
 #    external software (e.g., epa-ng, pplacer)
+
+# ---- BSCAMPP functions
 bscampp [-h]    # or
 run_bscampp.py [-h]
+
+# ---- SCAMPP functions
+scampp  [-h]    # or
+run_scampp.py
 ```
 
 ### (2) Install from GitHub
@@ -155,7 +170,8 @@ run_bscampp.py -i [raxml best model] -t [reference tree] -a [reference alignment
 ### (3) Using `pplacer` as the base placement method
 ```bash
 run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
-    -a [reference alignment] -q [query sequence alignment]
+    -a [reference alignment] -q [query sequence alignment] \
+    --placement-method pplacer
 ```
 
 ### More comprehensive usage
@@ -216,14 +232,23 @@ run_bscampp.py -i [logfile from either RAxML/FastTree] -t [reference tree] \
 >                         Temporary file indexing. Default: 0
 >   --fragmentflag FRAGMENTFLAG
 >                         If queries contains fragments. Default: True
+>  --subtreetype SUBTREETYPE
+>                         (SCAMPP only) Options for collecting nodes for the
+>                         subtree - d for edge weighted distances, n for node
+>                         distances, h for Hamming distances. Default: d
 >   --keeptemp KEEPTEMP   Boolean, True to keep all temporary files. Default:
                         False
 ```
 
 
 # Example Code and Data
-Example script and data are provided in this GitHub repository in `examples/`. The data is originally from the [RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
-* `examples/run.sh`: contains a simple script to test BSCAMPP with `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement tree.
-  The info file is from RAxML-ng when running `epa-ng`, and from FastTree-2 when running `pplacer`.
-  - `run.sh` will invoke BSCAMPP with `epa-ng`.
-  - `run.sh pplacer` will invoke BSCAMPP with `pplacer`.
+Example script and data are provided in this GitHub repository in `examples/`.
+The data is originally from the
+[RNAsim-VS datasets](https://doi.org/10.1093/sysbio/syz063).
+* `examples/run_bscampp.sh`: contains a simple script to test BSCAMPP with
+  `epa-ng` or `pplacer`, placing 200 query sequences to a 10000-leaf placement
+  tree. The info file is from RAxML-ng when running `epa-ng`, and from
+  FastTree-2 when running `pplacer`.
+  - `run_bscampp.sh` will invoke BSCAMPP with `epa-ng`.
+  - `run_bscampp.sh pplacer` will invoke BSCAMPP with `pplacer`.
+* `examples/run_scampp.sh`: the same test script but running SCAMPP.

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp.egg-info/SOURCES.txt

@@ -29,4 +29,5 @@ bscampp/tools/hamming_distance/src/fragment_hamming.cpp
 bscampp/tools/hamming_distance/src/fragment_tree_hamming.cpp
 bscampp/tools/hamming_distance/src/fragment_tree_hamming_new.cpp
 bscampp/tools/hamming_distance/src/homology.cpp
-bscampp/tools/hamming_distance/src/new_hamming.cpp
+bscampp/tools/hamming_distance/src/new_hamming.cpp
+tests/test_dry_run.py

{bscampp-1.0.1a0 → bscampp-1.0.2}/bscampp.egg-info/entry_points.txt

@@ -1,3 +1,5 @@
 [console_scripts]
 bscampp = bscampp.pipeline:bscampp_pipeline
 run_bscampp.py = bscampp.pipeline:bscampp_pipeline
+run_scampp.py = bscampp.pipeline:scampp_pipeline
+scampp = bscampp.pipeline:scampp_pipeline

{bscampp-1.0.1a0 → bscampp-1.0.2}/pyproject.toml

@@ -37,6 +37,8 @@ classifiers = [
 [project.scripts]
 bscampp = "bscampp.pipeline:bscampp_pipeline"
 "run_bscampp.py" = "bscampp.pipeline:bscampp_pipeline"
+scampp = "bscampp.pipeline:scampp_pipeline"
+"run_scampp.py" = "bscampp.pipeline:scampp_pipeline"
 
 [project.urls]
 Homepage = "https://github.com/ewedell/BSCAMPP"

bscampp-1.0.2/tests/test_dry_run.py

@@ -0,0 +1,21 @@
+# tests/test_dry_run.py
+import pytest, os
+from bscampp.pipeline import bscampp_pipeline, scampp_pipeline
+
+# test BSCAMPP
+def test_bscampp_pipeline():
+    res = bscampp_pipeline(dry_run=True)
+    assert res == True
+
+    # remove bscampp_output that's created
+    if os.path.isdir('bscampp_output'):
+        os.rmdir('bscampp_output')
+
+# test SCAMPP (almost the same as BSCAMPP)
+def test_scampp_pipeline():
+    res = scampp_pipeline(dry_run=True)
+    assert res == True
+
+    # remove scampp_output that's created
+    if os.path.isdir('scampp_output'):
+        os.rmdir('scampp_output')

bscampp-1.0.1a0/CHANGELOG.md

@@ -1,3 +0,0 @@
-# BSCAMPP v1.0.1a
-1. Completed features with both `epa-ng` and `pplacer` support.
-2. Refactorized all codes and worked out the PyPI installation for release.