cellmaps-utils 0.3.0__tar.gz → 0.4.0__tar.gz

cellmaps_utils-0.4.0/HISTORY.rst

@@ -0,0 +1,49 @@
+=======
+History
+=======
+
+0.4.0 (2024-07-02)
+-------------------
+
+* Updated provenance utils, added checks in for missing data in input RO-Crate,
+  and allowing to continue but logging errors in the process
+
+* Add ``HierarchyToHiDeFConverter``, a class to convert a hierarchy network
+  (in CX2 format) to a HiDeF format nodes and edges lists.
+
+* Add ``NDExHierarchyUploader``, a class for uploading hierarchy and
+  its parent network to NDEx.
+
+* Updated ``cellmaps_utilscmd.py`` ``apmsconverter``, ``ifconverter``,
+  ``crisprconverter`` to support tissue as well as output
+  ``data_info.json`` file to resulting RO-Crate so subsequent tools can
+  more easily get provenance information
+
+
+* Updated ``cellmaps_utilscmd.py crisprconverter`` to consume ``.h5ad``
+  files and updated readme.txt file
+
+0.3.0 (2024-04-15)
+-------------------
+
+* Bumped fairscape-cli dependency to ``0.2.0``
+
+
+0.2.0 (2024-02-20)
+------------------
+
+* Bumped fairscape-cli dependency to ``0.1.14`` to support schemas
+
+* Added support for ``schema`` to **data_dict** parameter in ``ProvenanceUtil.register_dataset()``
+
+* Added ``--release`` flag to ``cellmaps_utilscmd.py rocratetable`` and
+  in output table renamed "Name of Computation" to "Name" as well as
+  added "Type", "Cell Line", "Treatment", "Gene set", and "Version" to
+  table output
+
+* Set default logging level to ``ERROR`` for ``cellmaps_utilscmd.py`` command
+
+0.1.0 (2024-01-01)
+------------------
+
+* First release on PyPI.

cellmaps_utils-0.4.0/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.1
+Name: cellmaps_utils
+Version: 0.4.0
+Summary: Contains utilities needed by Cell Maps tools
+Home-page: https://github.com/idekerlab/cellmaps_utils
+Author: Cell Maps team
+Author-email: tools@cm4ai.org
+License: MIT license
+Keywords: cellmaps_utils
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Description-Content-Type: text/x-rst
+License-File: LICENSE
+License-File: AUTHORS.rst
+Requires-Dist: fairscape-cli==0.2.0
+Requires-Dist: ndex2<4.0.0,>=3.8.0
+Requires-Dist: scipy
+Requires-Dist: scikit-learn
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: dill
+Requires-Dist: tqdm
+Requires-Dist: requests
+
+=============================
+Cell Maps Pipeline Utilities
+=============================
+
+
+.. image:: https://img.shields.io/pypi/v/cellmaps_utils.svg
+        :target: https://pypi.python.org/pypi/cellmaps_utils
+
+.. image:: https://app.travis-ci.com/idekerlab/cellmaps_utils.svg?branch=main
+        :target: https://app.travis-ci.com/idekerlab/cellmaps_utils
+
+.. image:: https://readthedocs.org/projects/cellmaps-utils/badge/?version=latest
+        :target: https://cellmaps-utils.readthedocs.io/en/latest/?badge=latest
+        :alt: Documentation Status
+
+.. image:: https://zenodo.org/badge/599754117.svg
+        :target: https://zenodo.org/doi/10.5281/zenodo.10607322
+        :alt: Zenodo DOI badge
+
+Contains utilities needed by various tools in the cell maps toolkit. 
+
+
+* Free software: MIT license
+* Documentation: https://cellmaps-utils.readthedocs.io.
+* Source code: https://github.com/idekerlab/cellmaps_utils
+
+Dependencies
+------------
+
+* `fairscape-cli <https://pypi.org/project/fairscape-cli>`__
+* `scipy <https://pypi.org/project/scipy>`__
+* `scikit-learn <https://pypi.org/project/scikit-learn>`__
+* `pandas <https://pypi.org/project/pandas>`__
+* `numpy <https://pypi.org/project/numpy>`__
+* `dill <https://pypi.org/project/dill>`__
+
+Compatibility
+-------------
+
+* Python 3.8+
+
+Installation
+------------
+
+.. code-block::
+
+   git clone https://github.com/idekerlab/cellmaps_utils
+   cd cellmaps_utils
+   make dist
+   pip install dist/cellmaps_utils*whl
+
+
+Run **make** command with no arguments to see other build/deploy options including creation of Docker image 
+
+.. code-block::
+
+   make
+
+Output:
+
+.. code-block::
+
+   clean                remove all build, test, coverage and Python artifacts
+   clean-build          remove build artifacts
+   clean-pyc            remove Python file artifacts
+   clean-test           remove test and coverage artifacts
+   lint                 check style with flake8
+   test                 run tests quickly with the default Python
+   test-all             run tests on every Python version with tox
+   coverage             check code coverage quickly with the default Python
+   docs                 generate Sphinx HTML documentation, including API docs
+   servedocs            compile the docs watching for changes
+   testrelease          package and upload a TEST release
+   release              package and upload a release
+   dist                 builds source and wheel package
+   install              install the package to the active Python's site-packages
+
+For developers
+-------------------------------------------
+
+To deploy development versions of this package
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Below are steps to make changes to this code base, deploy, and then run
+against those changes.
+
+#. Make changes
+
+Modify code in this repo as desired
+
+#. Build and deploy
+
+.. code-block::
+
+    # From base directory of this repo cellmaps_hierarchyeval
+    pip uninstall cellmaps_utils -y ; make clean dist; pip install dist/cellmaps_utils*whl
+
+
+
+Credits
+-------
+
+This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template
+
+
+.. _Cookiecutter: https://github.com/audreyr/cookiecutter
+.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
+.. _NDEx: http://www.ndexbio.org
+
+
+
+=======
+History
+=======
+
+0.4.0 (2024-07-02)
+-------------------
+
+* Updated provenance utils, added checks in for missing data in input RO-Crate,
+  and allowing to continue but logging errors in the process
+
+* Add ``HierarchyToHiDeFConverter``, a class to convert a hierarchy network
+  (in CX2 format) to a HiDeF format nodes and edges lists.
+
+* Add ``NDExHierarchyUploader``, a class for uploading hierarchy and
+  its parent network to NDEx.
+
+* Updated ``cellmaps_utilscmd.py`` ``apmsconverter``, ``ifconverter``,
+  ``crisprconverter`` to support tissue as well as output
+  ``data_info.json`` file to resulting RO-Crate so subsequent tools can
+  more easily get provenance information
+
+
+* Updated ``cellmaps_utilscmd.py crisprconverter`` to consume ``.h5ad``
+  files and updated readme.txt file
+
+0.3.0 (2024-04-15)
+-------------------
+
+* Bumped fairscape-cli dependency to ``0.2.0``
+
+
+0.2.0 (2024-02-20)
+------------------
+
+* Bumped fairscape-cli dependency to ``0.1.14`` to support schemas
+
+* Added support for ``schema`` to **data_dict** parameter in ``ProvenanceUtil.register_dataset()``
+
+* Added ``--release`` flag to ``cellmaps_utilscmd.py rocratetable`` and
+  in output table renamed "Name of Computation" to "Name" as well as
+  added "Type", "Cell Line", "Treatment", "Gene set", and "Version" to
+  table output
+
+* Set default logging level to ``ERROR`` for ``cellmaps_utilscmd.py`` command
+
+0.1.0 (2024-01-01)
+------------------
+
+* First release on PyPI.

{cellmaps_utils-0.3.0 → cellmaps_utils-0.4.0}/cellmaps_utils/__init__.py

@@ -4,6 +4,6 @@
 
 __author__ = 'Cell Maps team'
 __email__ = 'tools@cm4ai.org'
-__version__ = '0.3.0'
+__version__ = '0.4.0'
 __repo_url__ = 'https://github.com/idekerlab/cellmaps_utils'
 __description__ = 'Contains utilities needed by Cell Maps tools'

cellmaps_utils-0.4.0/cellmaps_utils/apms_readme.txt

@@ -0,0 +1,85 @@
+Affinity Purification Mass spectrometry (AP-MS)
+-----------------------------------------------
+
+Affinity Purification-Mass Spectrometry (AP-MS) is a powerful technique used to uncover the intricate networks of
+protein interactions within cells. Different methods can be used but the most popular approach is to attach a purifiable
+tag to a protein of interest, the "bait". In the context of CM4AI the tag is directly attached to the endogenous protein
+of interest using gene editing to maintain physiological level of expression. The tagged bait is then purified from cell
+extract with its interacting partners. Once separated, the captured proteins are identified using a mass spectrometer
+allowing to infer protein-protein interactions. Understanding these interactions allows can unravel complex cellular
+processes, discover new roles for proteins, and gain insights into diseases caused by disrupted protein networks.
+
+
+Data processing
+---------------
+
+Raw data acquired on a Orbitrap Fusion™ Lumos™ Tribrid™ Mass Spectrometer (Thermo Scientific)
+Raw data are processed using Maxquant v1.6.12.0 using default parameter with LFQ quantification and the
+“202209_uniprot_human_reviewed.fasta” file as reference.
+Evidence file from maxquant are processed using ArtMS in R to generate SAINT input files using the
+following command lines. SAINTexpress was run locally using version SAINTexpress_v3.6.3
+SAINT is designed to assess the reliability and significance of protein-protein interactions identified
+through mass spectrometry experiments. It takes into account both the presence and abundance of proteins
+in different experimental conditions or replicates. The method uses a probabilistic model to estimate the
+probability that a given interaction is a true positive rather than a random or non-specific interaction.
+This approach helps to distinguish true interactions from noise in large-scale protein interaction datasets.
+
+apms.tsv column labels
+----------------------
+
+Bait: Name of the pull downed protein
+
+Prey: Uniprot ID number of identified proteins by MS in pull down (putative bait interactor).
+
+PreyGene.x: Uniprot protein name of identified protein by MS in pull down (putative bait interactor).
+
+Spec: Number of spectral count in each test biological replicates (separated by | ).
+
+SpecSum: Sum of Spectral counts in test samples.
+
+AvgSpec: Average Spectral counts across replicates in test samples.
+
+NumReplicates.x: Number of replicates in test samples.
+
+ctrlCounts: Number of spectral count in each control replicates (separated by | ).
+
+AvgP.x: Average probability that an interaction is true, measure of the likelihood that a given
+        interaction is a true positive rather than a random or non-specific interaction. A lower AvgP indicates
+        higher confidence in the interaction being genuine.
+
+MaxP.x: maximum probability associated with a protein interaction in the context of its prey-bait pair.
+        Similar to AvgP, a lower MaxP suggests a higher likelihood of the interaction being true.
+
+TopoAvgP.x: extension of the AvgP score that also takes into consideration the topology of the
+            interaction network. It incorporates information about the hierarchical structure of the interaction
+            data to provide a refined assessment of the interactions.
+
+TopoMaxP.x: topology-aware score that considers the maximum probability of an interaction in the
+            context of the interaction network's topology.
+
+SaintScore.x: composite score that integrates multiple aspects of the interaction data, including
+              spectral counts and probability estimates. It's designed to prioritize interactions based on their
+              strength and reliability. Higher SaintScores indicate interactions that are more likely to be true.
+
+logOddsScore: Logarithm of the odds ratio between test and control conditions for each prey as a
+              measure of interaction significance. The LogOddsScore is a statistical score that represents the
+              logarithm of the odds ratio for a protein-protein interaction. It's used to quantify the strength
+              and significance of the association between two proteins in an interaction network. The odds ratio
+              compares the likelihood of the interaction occurring to the likelihood of it not occurring. Taking
+              the logarithm of the odds ratio often helps to transform the score into a more symmetric and interpretable
+              form, making it easier to compare and analyze the interactions. Higher LogOddsScores typically indicate
+              stronger evidence for the interaction.
+
+FoldChange.x: represents the ratio of the abundance of a protein or interaction in one experimental
+              condition (Test) compared to another (control). It helps assess whether the abundance of a protein
+              changes significantly between different conditions.
+
+BFDR.x: Bayesian False Discovery Rate
+
+
+
+
+SAINT References:
+https://saint-apms.sourceforge.net/Main.html
+https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4102138/
+https://pubmed.ncbi.nlm.nih.gov/21131968/

{cellmaps_utils-0.3.0 → cellmaps_utils-0.4.0}/cellmaps_utils/apmstool.py

@@ -1,3 +1,4 @@
+import json
 import os
 import sys
 import uuid
@@ -39,6 +40,7 @@ class APMSDataLoader(BaseCommandLineTool):
         self._release = theargs.release
         self._cell_line = theargs.cell_line
         self._treatment = theargs.treatment
+        self._tissue = theargs.tissue
         self._author = theargs.author
         self._gene_set = theargs.gene_set
         self._provenance_utils = provenance_utils
@@ -47,6 +49,8 @@ class APMSDataLoader(BaseCommandLineTool):
 
     def run(self):
         """
+        Run method to create RO-Crate from AP-MS data tables. This process involves merging input tables,
+        registering the dataset and related software in the RO-Crate.
 
         :return:
         """
@@ -58,13 +62,28 @@ class APMSDataLoader(BaseCommandLineTool):
         os.makedirs(self._outdir, mode=0o755)
 
         keywords = [self._project_name, self._release,
-                    self._cell_line, self._treatment, 'AP-MS edgelist']
+                    self._cell_line, self._treatment, self._tissue,
+                    'AP-MS edgelist']
 
         if self._gene_set is not None:
             keywords.append(self._gene_set)
 
         description = ' '.join(keywords)
 
+        info_dict = {
+            constants.DATASET_NAME: self._name,
+            constants.DATASET_ORGANIZATION_NAME: self._organization_name,
+            constants.DATASET_PROJECT_NAME: self._project_name,
+            constants.DATASET_RELEASE: self._release,
+            constants.DATASET_CELL_LINE: self._cell_line,
+            constants.DATASET_TREATMENT: self._treatment,
+            constants.DATASET_TISSUE: self._tissue,
+            constants.DATASET_AUTHOR: self._author,
+            constants.DATASET_GENE_SET: self._gene_set
+        }
+
+        self.save_dataset_info_to_json(self._outdir, info_dict, constants.DATASET_INFO_FILE)
+
         self._provenance_utils.register_rocrate(self._outdir,
                                                 name=self._name,
                                                 organization_name=self._organization_name,
@@ -113,8 +132,8 @@ class APMSDataLoader(BaseCommandLineTool):
 
     def _generate_rocrate_dir_path(self):
         """
-
-        :return:
+        Generates a directory path for the RO-Crate based on provided metadata like project name, gene set,
+        cell line, treatment, and release version.
         """
         dir_name = self._project_name.lower() + '_'
         if self._gene_set is not None:
@@ -128,8 +147,10 @@ class APMSDataLoader(BaseCommandLineTool):
 
     def _merge_and_save_apms_data(self):
         """
+        Merges AP-MS data from input files into a single DataFrame and saves the combined data to a TSV file within
+        the RO-Crate directory.
 
-        :return:
+        :return: The file path to the saved AP-MS data TSV file.
         """
         df_list = []
         for input in self._inputs:
@@ -151,7 +172,7 @@ class APMSDataLoader(BaseCommandLineTool):
 
         df = pd.concat(df_list)
 
-        apms_path = os.path.join(self._outdir, 'apms.tsv')
+        apms_path = os.path.join(self._outdir, constants.APMS_TSV_FILE)
         df.to_csv(apms_path, sep='\t', index=False)
         return apms_path
 
@@ -159,6 +180,7 @@ class APMSDataLoader(BaseCommandLineTool):
                               description='',
                               keywords=[]):
         """
+        Registers the computation process in the RO-Crate
         # Todo: added in used dataset, software and what is being generated
         :return:
         """
@@ -199,6 +221,7 @@ class APMSDataLoader(BaseCommandLineTool):
 
     def add_subparser(subparsers):
         """
+        Adds a command-line subparser for the APMSDataLoader tool.
 
         :return:
         """
@@ -240,5 +263,11 @@ class APMSDataLoader(BaseCommandLineTool):
         parser.add_argument('--gene_set', choices=['chromatin', 'metabolic'],
                             default='chromatin',
                             help='Gene set for dataset')
+        parser.add_argument('--tissue', choices=['undifferentiated', 'neuron',
+                                                 'cardiomyocytes', ''],
+                            default='breast; mammary gland',
+                            help='Tissue for dataset. Since the default --cell_line '
+                                 'is MDA-MB-468, this value is set to the tissue '
+                                 'for that cell line')
         return parser
 

{cellmaps_utils-0.3.0 → cellmaps_utils-0.4.0}/cellmaps_utils/basecmdtool.py

@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
-
+import json
+import os
 import sys
 import argparse
 import logging
@@ -24,6 +25,22 @@ class BaseCommandLineTool(object):
         """
         pass
 
+    def save_dataset_info_to_json(self, outdir, info_dict, file_name):
+        """
+        Saves project information to a JSON file.
+
+        :param outdir: Output directory where the file will be saved
+        :param info_dict: Dictionary with dataset information
+        :param file_name: Name of the file to save the information.
+        """
+
+        json_str = json.dumps(info_dict, indent=4)
+
+        json_file_path = os.path.join(outdir, file_name)
+
+        with open(json_file_path, 'w') as json_file:
+            json_file.write(json_str)
+
     def run(self):
         """
         Should contain logic that will be run by command line tool.

{cellmaps_utils-0.3.0 → cellmaps_utils-0.4.0}/cellmaps_utils/constants.py

@@ -84,6 +84,16 @@ IMAGE_GENE_NODE_ERRORS_FILE = 'image_gene_node_attributes.errors'
 Image gene node attributes errors filename
 """
 
+PERTURBATION_FILE = 'perturbation.h5ad'
+"""
+Perturbation/CRISPRi h5ad file
+"""
+
+ANTIBODY_GENE_TABLE_FILE = 'antibody_gene_table.tsv'
+"""
+Antibody Gene Table file
+"""
+
 IMAGE_GENE_NODE_NAME_COL = 'name'
 """
 Gene Symbol name column
@@ -125,6 +135,11 @@ IMAGE_GENE_NODE_COLS = [IMAGE_GENE_NODE_NAME_COL,
 Columns in :py:const:`IMAGE_GENE_NODE_ATTR_FILE` file
 """
 
+APMS_TSV_FILE = 'apms.tsv'
+"""
+AP-MS tsv file
+"""
+
 PPI_EDGELIST_FILE = 'ppi_edgelist.tsv'
 """
 Protein to Protein interaction edgelist file name
@@ -174,7 +189,12 @@ PPI_NETWORK_PREFIX = 'ppi'
 
 HIERARCHY_NETWORK_PREFIX = 'hierarchy'
 """
-CX format hierarchy filename
+CX2 format hierarchy filename
+"""
+
+HIERARCHY_PARENT_NETWORK_PREFIX = 'hierarchy_parent'
+"""
+CX2 format hierarchy parent (interactome_ filename
 """
 
 HIERARCHY_NODES_FILE = 'hierarchy_node_attributes.tsv'
@@ -281,4 +301,62 @@ HIERARCHYEVAL_STEP_DIR = '5.hierarchyeval'
 Name of directory where hierarchy evaluations are stored
 """
 
+DATASET_INFO_FILE = 'dataset_info.json'
+"""
+Name of file where information about a dataset is stored
+"""
+
+DATASET_NAME = 'name'
+"""
+Name of the dataset
+"""
+
+DATASET_ORGANIZATION_NAME = 'organization_name'
+"""
+Name of the organization
+"""
+
+DATASET_PROJECT_NAME = 'project_name'
+"""
+Name of the project
+"""
+
+DATASET_RELEASE = 'release'
+"""
+Release of the dataset
+"""
+
+DATASET_CELL_LINE = 'cell_line'
+"""
+Cell line
+"""
+
+DATASET_TREATMENT = 'treatment'
+"""
+Treatment
+"""
+
+DATASET_TISSUE = 'tissue'
+"""
+Tissue
+"""
 
+DATASET_AUTHOR = 'author'
+"""
+Author of the dataset
+"""
+
+DATASET_SLICE = 'slice'
+"""
+Slice
+"""
+
+DATASET_GENE_SET = 'gene_set'
+"""
+Gene set
+"""
+
+DATASET_COLLECTION_SET = 'collection_set'
+"""
+Collection set
+"""

cellmaps_utils-0.4.0/cellmaps_utils/crispr_readme.txt

@@ -0,0 +1,19 @@
+@@H5AD@@ file
+-----------------------------------------------------------
+
+This file is a .h5ad file. Details about the data structure of this format
+can be found here: https://anndata.readthedocs.io/en/latest/. This data is
+a cell x gene matrix, with all preprocesing steps done in accordance to
+single cell best practices guide
+(https://www.sc-best-practices.org/conditions/perturbation_modeling.html#analysing-single-pooled-crispr-screens)
+up to section 19.4.5. The .X layer is set to the 'X_pert' output of the
+mixscape pipeline.
+
+cell line: @@CELL_LINE@@
+screening modality: perturb seq using CRISPR-dCas9-ZIM3
+screen targets: @@NUM_SCREEN_TARGETS_AND_GENE_SET@@ genes
+non-targeting controls: @@NUM_NON_TARGET_CTRLS@@
+number of guides per perturbation: @@NUM_PERTURB_GUIDES@@
+treatment: @@TREATMENT@@
+
+