treesumstats 0.1__py3-none-any.whl

README.md

@@ -0,0 +1,122 @@
+# treesumstats
+
+Representing phylogenetic trees (or forests of such trees) as summary statistics.
+
+
+
+[//]: # ([![DOI:10.1093/sysbio/syad059](https://zenodo.org/badge/DOI/10.1093/sysbio/syad059.svg)](https://doi.org/10.1093/sysbio/syad059))
+[//]: # ([![GitHub release](https://img.shields.io/github/v/release/evolbioinfo/treesumstats.svg)](https://github.com/evolbioinfo/treesumstats/releases))
+[![PyPI version](https://badge.fury.io/py/treesumstats.svg)](https://pypi.org/project/treesumstats/)
+[![PyPI downloads](https://shields.io/pypi/dm/treesumstats)](https://pypi.org/project/treesumstats/)
+[![Docker pulls](https://img.shields.io/docker/pulls/evolbioinfo/treesumstats)](https://hub.docker.com/r/evolbioinfo/treesumstats/tags)
+
+## Implemented summary statistics
+
+
+
+## Input data
+One needs to supply a time-scaled phylogenetic tree in newick format. 
+If the file contains several trees (one tree per line), it will be considered as a forest.
+
+## Installation
+
+There are 4 alternative ways to run __treesumstats__ on your computer: 
+with [docker](https://www.docker.com/community-edition), 
+[apptainer](https://apptainer.org/),
+in Python3, or via command line (requires installation with Python3).
+
+
+
+### Run in python3 or command-line (for linux systems, recommended Ubuntu 21 or newer versions)
+
+You could either install python (version 3.9 or higher) system-wide and then install treesumstats via pip:
+```bash
+sudo apt install -y python3 python3-pip python3-setuptools python3-distutils
+pip3 install treesumstats
+```
+
+or alternatively, you could install python (version 3.9 or higher) and treesumstats via [conda](https://conda.io/docs/) (make sure that conda is installed first). 
+Here we will create a conda environment called _phyloenv_:
+```bash
+conda create --name phyloenv python=3.9
+conda activate phyloenv
+pip install treesumstats
+```
+
+
+#### Basic usage in a command line
+If you installed __treesumstats__ in a conda environment (here named _phyloenv_), do not forget to first activate it, e.g.
+
+```bash
+conda activate phyloenv
+```
+
+Run the following command to calculate the summary statistics for an input tree _tree.nwk_ and save them to an output tab-delimited file _stats.tab_. 
+(The non-mandatory option _--add_descriptions_ would add a column with statistics descriptions to the output table.)
+
+```bash
+treesumstats_encode --nwk tree.nwk --tab stats.tab --add_descriptions
+```
+
+#### Help
+
+To see detailed options, run:
+```bash
+treesumstats_encode --help
+```
+
+
+### Run with docker
+
+#### Basic usage
+Once [docker](https://www.docker.com/community-edition) is installed, 
+
+run the following command to calculate the summary statistics for an input tree _tree.nwk_ and save them to an output tab-delimited file _stats.tab_. 
+(The non-mandatory option _--add_descriptions_ would add a column with statistics descriptions to the output table.)
+
+
+```bash
+docker run -v <path_to_the_folder_containing_the_tree>:/data:rw -t evolbioinfo/treesumstats --nwk /data/tree.nwk --tab /data/stats.tab --add_descriptions
+```
+
+
+This will produce a tab-delimited file _stats.tab_ in the <path_to_the_folder_containing_the_tree> folder,
+
+ containing the summary statistic names in the _statistic_ column, their values in the _value_ column and their descriptions in the _description_ column. 
+
+#### Help
+
+To see advanced options, run
+```bash
+docker run -t evolbioinfo/treesumstats -h
+```
+
+
+
+### Run with apptainer
+
+#### Basic usage
+Once [apptainer](https://apptainer.org/docs/user/latest/quick_start.html#installation) is installed, 
+
+run the following command to calculate the summary statistics for an input tree _tree.nwk_ and save them to an output tab-delimited file _stats.tab_. 
+(The non-mandatory option _--add_descriptions_ would add a column with statistics descriptions to the output table.)
+
+
+```bash
+apptainer run docker://evolbioinfo/treesumstats --nwk tree.nwk --tab stats.tab --add_descriptions
+```
+
+
+This will produce a tab-delimited file _stats.tab_,
+
+ containing the summary statistic names in the _statistic_ column, their values in the _value_ column and their descriptions in the _description_ column. 
+
+
+#### Help
+
+To see advanced options, run
+```bash
+apptainer run docker://evolbioinfo/treesumstats -h
+```
+
+

treesumstats/__init__.py

@@ -0,0 +1,125 @@
+import warnings
+from abc import ABC, abstractmethod
+
+import numpy as np
+
+MED = 'median'
+
+VAR = 'var'
+
+MEAN = 'mean'
+
+MAX = 'max'
+
+MIN = 'min'
+
+EPSILON = 1e-6
+
+
+BASIC_ARRAY_METRICS = [MEAN, MIN, MAX, VAR, MED]
+
+BASIC_METRIC2HELP = {VAR: 'variance',
+                     MIN: 'minimum',
+                     MEAN: 'average',
+                     MED: 'median',
+                     MAX: 'maximum'}
+
+BASIC_METRIC2FUN = {VAR: np.var,
+                    MIN: np.min,
+                    MEAN: np.mean,
+                    MED: np.median,
+                    MAX: np.max}
+
+
+NORMALIZED = 'normalized'
+
+
+class FeatureCalculator(ABC):
+    """Abstract base class for feature calculators."""
+
+    @abstractmethod
+    def feature_names(self):
+        """Return a list of feature names this class can compute."""
+        pass
+
+    @abstractmethod
+    def set_forest(self, forest, **kwargs):
+        """Sets the forest on which the features are to be calculated"""
+        pass
+
+    @abstractmethod
+    def calculate(self, forest, feature_name, **kwargs):
+        """Compute a specific feature or features."""
+        pass
+
+    @abstractmethod
+    def help(self, feature_name, *args, **kwargs):
+        """Shows the definition of the specific feature."""
+        pass
+
+
+class FeatureRegistry:
+    """Registry for mapping feature names to their respective calculator classes."""
+
+    _registry = {}
+    _calculators = []
+
+    @classmethod
+    def register(cls, calculator):
+        """Register a feature calculator class and map its features."""
+        cls._calculators.append(calculator)
+        for feature in calculator.feature_names():
+            cls._registry[feature] = calculator
+
+    @classmethod
+    def get_calculator(cls, feature_name):
+        """Retrieve the appropriate calculator class for a given feature."""
+        return cls._registry.get(feature_name)
+
+    @classmethod
+    def list_features(cls):
+        return list(cls._registry.keys())
+
+    @classmethod
+    def list_calculators(cls):
+        return list(cls._calculators)
+
+
+class FeatureManager:
+    """Manages feature computation dynamically based on user input."""
+
+    @classmethod
+    def compute_features(cls, forest, *feature_list, **kwargs):
+        """Compute and return requested features."""
+
+        for calc in FeatureRegistry.list_calculators():
+            calc.set_forest(forest, **kwargs)
+
+        if not feature_list:
+            feature_list = FeatureManager.available_features()
+
+        for feature in feature_list:
+            calculator = FeatureRegistry.get_calculator(feature)
+            if calculator:
+                yield calculator.calculate(feature, **kwargs)
+            else:
+                warnings.warn(f'Did not find a Feature Calculator for feature {feature}')
+                yield None
+
+    @classmethod
+    def available_features(cls):
+        return FeatureRegistry.list_features()
+
+    @classmethod
+    def help(cls, feature_name, *args, **kwargs):
+        """Returns feature description"""
+
+        calculator = FeatureRegistry.get_calculator(feature_name)
+        if calculator:
+            return calculator.help(feature_name, **kwargs)
+        else:
+            warnings.warn(f'Did not find a Feature Calculator for feature {feature_name}')
+            return None
+
+
+

treesumstats/basic_sumstats.py

@@ -0,0 +1,36 @@
+from treesumstats import FeatureCalculator
+
+
+class BasicFeatureCalculator(FeatureCalculator):
+    """Computes tree height-related summary statistics."""
+
+    def __init__(self):
+        self._forest = None
+
+    def feature_names(self):
+        return ['n_trees', 'n_tips', 'n_inodes', 'len_forest']
+
+    def set_forest(self, forest, **kwargs):
+        self._forest = forest
+
+    def calculate(self, feature_name, **kwargs):
+        if 'n_trees' == feature_name:
+            return len(self._forest)
+        if 'n_tips' == feature_name:
+            return sum(len(tree) for tree in self._forest)
+        if 'n_inodes' == feature_name:
+            return sum(1 for tree in self._forest for n in tree.traverse() if not n.is_leaf())
+        if 'len_forest' == feature_name:
+            return sum(_.dist for tree in self._forest for _ in tree.traverse())
+        return None
+
+    def help(self, feature_name, *args, **kwargs):
+        if 'n_trees' == feature_name:
+            return 'number of trees in the forest.'
+        if 'n_tips' == feature_name:
+            return 'number of tips in the forest.'
+        if 'n_inodes' == feature_name:
+            return 'number of internal nodes in the forest.'
+        if 'len_forest' == feature_name:
+            return 'sum of forest branch lengths.'
+        return None

treesumstats/branch_sumstats.py

@@ -0,0 +1,180 @@
+from treesumstats import BASIC_METRIC2HELP, BASIC_METRIC2FUN, FeatureCalculator, MEAN, MED, VAR, EPSILON
+from treesumstats.tree_manager import annotate_forest_with_time, TIME
+
+BRLEN = 'brlen'
+
+class BranchFeatureCalculator(FeatureCalculator):
+    """Computes tree branch-related summary statistics."""
+
+    def __init__(self):
+        self._edists_top = None
+        self._idists_top = None
+        self._edists_bottom = None
+        self._idists_bottom = None
+        self._edists_middle = None
+        self._idists_middle = None
+        self._max_time = None
+        self._idists = None
+        self._edists = None
+        self._forest = None
+
+    def feature_names(self):
+        return [f'{BRLEN}_inode_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'{BRLEN}_tip_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'{BRLEN}_inode_top_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'{BRLEN}_tip_top_{flavour}' for flavour in[MEAN, MED, VAR]]\
+            + [f'{BRLEN}_inode_middle_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'{BRLEN}_tip_middle_{flavour}' for flavour in[MEAN, MED, VAR]] \
+            + [f'{BRLEN}_inode_bottom_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'{BRLEN}_tip_bottom_{flavour}' for flavour in[MEAN, MED, VAR]] \
+            + [f'frac_{BRLEN}_inode_{flavour}_by_{BRLEN}_tip_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'frac_{BRLEN}_inode_top_{flavour}_by_{BRLEN}_tip_top_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'frac_{BRLEN}_inode_middle_{flavour}_by_{BRLEN}_tip_middle_{flavour}' for flavour in [MEAN, MED, VAR]] \
+            + [f'frac_{BRLEN}_inode_bottom_{flavour}_by_{BRLEN}_tip_bottom_{flavour}' for flavour in [MEAN, MED, VAR]]
+
+
+    def set_forest(self, forest, **kwargs):
+        self._forest = forest
+        annotate_forest_with_time(self._forest)
+        self._idists = None
+        self._edists = None
+        self._edists_top = None
+        self._idists_top = None
+        self._edists_middle = None
+        self._idists_middle = None
+        self._edists_bottom = None
+        self._idists_bottom = None
+        self._max_time = None
+
+
+    @property
+    def max_time(self):
+        if self._max_time is None:
+            self._max_time = max(getattr(_, TIME) for tree in self._forest for _ in tree)
+        return self._max_time
+
+    @property
+    def idists(self):
+        if self._idists is None:
+            self._idists, self._edists = self._get_branch_lengths()
+        return self._idists
+
+    @property
+    def edists(self):
+        if self._edists is None:
+            self._idists, self._edists = self._get_branch_lengths()
+        return self._edists
+
+    @property
+    def idists_top(self):
+        if self._idists_top is None:
+
+            def time_filter(nd):
+                return getattr(nd, TIME) <= self.max_time / 3
+
+            self._idists_top, self._edists_top = self._get_branch_lengths(time_filter)
+        return self._idists_top
+
+    @property
+    def edists_top(self):
+        if self._edists_top is None:
+
+            def time_filter(nd):
+                return getattr(nd, TIME) <= self.max_time / 3
+
+            self._idists_top, self._edists_top = self._get_branch_lengths(time_filter)
+        return self._edists_top
+
+    @property
+    def idists_middle(self):
+        if self._idists_middle is None:
+
+            def time_filter(nd):
+                return (getattr(nd, TIME) <= 2 * self.max_time / 3) \
+                    and (self.max_time / 3 < (getattr(nd, TIME) - nd.dist))
+
+            self._idists_middle, self._edists_middle = self._get_branch_lengths(time_filter)
+        return self._idists_middle
+
+    @property
+    def edists_middle(self):
+        if self._edists_middle is None:
+
+            def time_filter(nd):
+                return (getattr(nd, TIME) <= 2 * self.max_time / 3) \
+                    and (self.max_time / 3 < (getattr(nd, TIME) - nd.dist))
+
+            self._idists_middle, self._edists_middle = self._get_branch_lengths(time_filter)
+        return self._edists_middle
+
+    @property
+    def idists_bottom(self):
+        if self._idists_bottom is None:
+
+            def time_filter(nd):
+                return 2 * self.max_time / 3 < (getattr(nd, TIME) - nd.dist)
+
+            self._idists_bottom, self._edists_bottom = self._get_branch_lengths(time_filter)
+        return self._idists_bottom
+
+    @property
+    def edists_bottom(self):
+        if self._edists is None:
+
+            def time_filter(nd):
+                return 2 * self.max_time / 3 < (getattr(nd, TIME) - nd.dist)
+
+            self._idists_bottom, self._edists_bottom = self._get_branch_lengths(time_filter)
+        return self._edists_bottom
+
+    def _get_branch_lengths(self, node_filter=lambda nd: True):
+        internal_dists, external_dists = [], []
+
+        for tree in self._forest:
+            for n in tree.traverse():
+                if not node_filter(n):
+                    continue
+                if n.is_leaf():
+                    external_dists.append(n.dist)
+                elif not n.is_root():
+                    internal_dists.append(n.dist)
+        return internal_dists, external_dists
+
+
+    def calculate(self, feature_name, **kwargs):
+        if feature_name.startswith(BRLEN):
+            is_internal = 'inode' in feature_name
+            flavour = feature_name.split('_')[-1]
+            if is_internal:
+                data = self.idists_top if 'top' in feature_name \
+                    else (self.idists_middle if 'middle' in feature_name \
+                              else (self.idists_bottom if 'bottom' in feature_name else self.idists))
+            else:
+                data = self.edists_top if 'top' in feature_name \
+                    else (self.edists_middle if 'middle' in feature_name \
+                              else (self.edists_bottom if 'bottom' in feature_name else self.edists))
+            return BASIC_METRIC2FUN[flavour](data) if len(data) else 0
+        if feature_name.startswith('frac'):
+            feature_name_1, feature_name_2 = feature_name[len('frac_'): ].split('_by_')
+            val1, val2 = self.calculate(feature_name_1), self.calculate(feature_name_2)
+            return val1 / max(val2, EPSILON)
+        return None
+
+    def help(self, feature_name, *args, **kwargs):
+        if feature_name.startswith(BRLEN):
+            is_internal = 'inode' in feature_name
+            flavour = feature_name.split('_')[-1]
+            res = f'{BASIC_METRIC2HELP[flavour]} ' \
+                  f'of the lengths of the forest {'non-root internal nodes' if is_internal else 'tips'}.'
+            part = 'top' if 'top' in feature_name \
+                else ('middle' if 'middle' in feature_name else ('bottom' if 'bottom' in feature_name else ''))
+            if part:
+                res += f' Only branches starting and finishing in the {part} third of the forest are considered. ' \
+                       f'(The total time between the earliest forest tree start and the last sampled tip '\
+                       'is split into three equal parts.)'
+            return res
+        if feature_name.startswith('frac'):
+            feature_name_1, feature_name_2 = feature_name[len('frac_'):].split('_by_')
+            return f'fraction of two values. The dividend is {self.help(feature_name_1).split('.')[0]}. '\
+                   f'The divisor is {self.help(feature_name_1)}'
+        return None

treesumstats/calculator.py

@@ -0,0 +1,62 @@
+from pandas.core.computation.ops import isnumeric
+
+from treesumstats.basic_sumstats import BasicFeatureCalculator
+from treesumstats.branch_sumstats import BranchFeatureCalculator
+from treesumstats.event_time_sumstats import EventTimeFeatureCalculator
+from treesumstats.ltt_sumstats import LTTFeatureCalculator
+from treesumstats.subtree_sumstats import SubtreeFeatureCalculator
+from treesumstats.topology_sumstats import TopologyFeatureCalculator
+from treesumstats import FeatureRegistry, FeatureManager
+from treesumstats.resolution_sumstats import ResolutionFeatureCalculator
+from treesumstats.transmission_chain_sumstats import TransmissionChainFeatureCalculator
+from treesumstats.tree_manager import read_forest
+
+FeatureRegistry.register(ResolutionFeatureCalculator())
+FeatureRegistry.register(TopologyFeatureCalculator())
+FeatureRegistry.register(EventTimeFeatureCalculator())
+FeatureRegistry.register(BasicFeatureCalculator())
+FeatureRegistry.register(BranchFeatureCalculator())
+FeatureRegistry.register(TransmissionChainFeatureCalculator())
+FeatureRegistry.register(LTTFeatureCalculator())
+FeatureRegistry.register(SubtreeFeatureCalculator())
+
+
+
+def main():
+    """
+    Entry point for tree encoding with command-line arguments.
+    :return: void
+    """
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Calculate summary statistics for an input tree/forest.")
+    parser.add_argument('--nwk', type=str, required=True, help="input tree/forest files in newick format")
+    parser.add_argument('--tab', type=str, required=True, help="output summary statistic table, in tab-delimited format: the first column will contain summary statistic name, and the second its value.")
+    parser.add_argument('--statistics', type=str, nargs='*', help='an optional list of summary statistics that should be calculated. If not given all the available statistics will be calculated.')
+    parser.add_argument('--add_descriptions', action='store_true', help='whether to add a column "description" with the description of each calculated statistic.')
+    params = parser.parse_args()
+
+    forest = read_forest(params.nwk)
+
+    features = params.statistics if params.statistics else FeatureManager.available_features()
+
+    def format_value(value):
+        try:
+            value = float(value)
+            if int(value) == value:
+                return f'{value:.0f}'
+            if value > 1e-6:
+                return f'{value:.6f}'
+            return f'{value:.g}'
+        except:
+            return value
+
+
+    with open(params.tab, 'w+') as f:
+        f.write(f'statistic\tvalue{'\tdescription' if params.add_definitions else ''}\n')
+        for feature_name, value in zip(features, FeatureManager.compute_features(forest)):
+            f.write(f'{feature_name}\t{format_value(value)}{f"\t{FeatureManager.help(feature_name)}" if params.add_definitions else ""}\n')
+
+
+if '__main__' == __name__:
+    main()

treesumstats/event_time_sumstats.py

@@ -0,0 +1,68 @@
+import numpy as np
+
+from treesumstats import BASIC_METRIC2HELP, BASIC_ARRAY_METRICS, BASIC_METRIC2FUN, FeatureCalculator, NORMALIZED
+from treesumstats.tree_manager import annotate_forest_with_time, TIME
+
+
+class EventTimeFeatureCalculator(FeatureCalculator):
+    """Computes tree event time-related summary statistics."""
+
+    def __init__(self):
+        self._max_time = None
+        self._tip_times = None
+        self._forest = None
+        self._inode_times = None
+
+    def feature_names(self):
+        return [f'{TIME}_tip_{flavour}' for flavour in BASIC_ARRAY_METRICS] \
+            +  [f'{TIME}_inode_{flavour}' for flavour in BASIC_ARRAY_METRICS] \
+            +  [f'{TIME}_tip_{NORMALIZED}_{flavour}' for flavour in BASIC_ARRAY_METRICS] \
+            + [f'{TIME}_inode_{NORMALIZED}_{flavour}' for flavour in BASIC_ARRAY_METRICS]
+
+    def set_forest(self, forest, **kwargs):
+        self._forest = forest
+        annotate_forest_with_time(self._forest)
+        self._tip_times = None
+        self._inode_times = None
+        self._max_time = None
+
+    @property
+    def tip_times(self):
+        if self._tip_times is None:
+            self._tip_times = np.array([getattr(t, TIME) for tree in self._forest for t in tree])
+        return self._tip_times
+
+    @property
+    def max_time(self):
+        if self._max_time is None:
+            self._max_time = np.max(self.tip_times)
+        return self._max_time
+
+    @property
+    def inode_times(self):
+        if self._inode_times is None:
+            self._inode_times \
+                = np.array([getattr(n, TIME) for tree in self._forest for n in tree.traverse() if not n.is_leaf()])
+        return self._inode_times
+
+    def calculate(self, feature_name, **kwargs):
+        if feature_name.startswith(TIME):
+            is_internal = 'inode' in feature_name
+            is_normalized = NORMALIZED in feature_name
+            flavour = feature_name.split('_')[-1]
+            return BASIC_METRIC2FUN[flavour]((self.inode_times if is_internal else self.tip_times)
+                                             / (self.max_time if is_normalized else 1))
+
+        return None
+
+    def help(self, feature_name, *args, **kwargs):
+        if feature_name.startswith(TIME):
+            is_internal = 'inode' in feature_name
+            node = 'internal node' if is_internal else 'tip'
+            is_normalized = NORMALIZED in feature_name
+            flavour = feature_name.split('_')[-1]
+            res = f'{BASIC_METRIC2HELP[flavour]} of the sampling times of forest {node}s.'
+            if is_normalized:
+                res += ' We normalize the sampling times by dividing them by the last tip\'s sampling time.'
+            return res
+        return None