smftools 0.1.0__tar.gz → 0.1.1__tar.gz

{smftools-0.1.0 → smftools-0.1.1}/.gitignore

@@ -1,10 +1,9 @@
 # Python
 __pycache__/
-/src/smftools/_version.py
 
 # Build files
 build/
-dist/
+/dist/
 /hatch.toml
 /Pipfile
 /Pipfile.lock
@@ -16,30 +15,32 @@ dist/
 /*-venv/
 /env-*/
 /venv-*/
+venv/
 /environment.yml
 
 # OS
-*.DS_Store
-*.LSOverride
-*Thumbs.db
+.DS_Store
+.LSOverride
+Thumbs.db
 *.ipynb_checkpoints/
 *.directory
 
 # IDEs and editors
-*.vscode/
-*.idea/
+.vscode/
+.idea/
 *.iml
 
 # Logs
 *.log
 
 # temp files
-temp/
-tmp/
+/temp*
+/tmp*
+*.temp
+*.tmp
 
 # Coverage reports
 .coverage
 htmlcov/
 
 # Docs
-/docs

smftools-0.1.1/.readthedocs.yaml

@@ -0,0 +1,17 @@
+version: 2
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.12"
+sphinx:
+  configuration: docs/source/conf.py
+  fail_on_warning: true
+python:
+  install:
+    - method: pip
+      path: .
+      extra_requirements:
+      - docs
+submodules:
+  include: all
+  recursive: true

smftools-0.1.1/CONTRIBUTING.md

@@ -0,0 +1,3 @@
+Contributing
+============
+Contributions to smftools are not currently being reviewed or accepted due to the pre-alpha phase status of the project. More mature versions of the project will have contribution guidelines added.

smftools-0.1.1/PKG-INFO

@@ -0,0 +1,88 @@
+Metadata-Version: 2.3
+Name: smftools
+Version: 0.1.1
+Summary: Single Molecule Footprinting Analysis in Python.
+Project-URL: Source, https://github.com/jkmckenna/smftools
+Author: Joseph McKenna
+Maintainer-email: Joseph McKenna <jkmckenna@berkeley.edu>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: anndata,chromatin-accessibility,machine-learning,nanopore,protein-dna-binding,single-locus,single-molecule-footprinting
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Natural Language :: English
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Topic :: Scientific/Engineering :: Visualization
+Requires-Python: >=3.9
+Requires-Dist: anndata>=0.10.0
+Requires-Dist: biopython>=1.79
+Requires-Dist: cython>=0.29.28
+Requires-Dist: networkx>=3.2
+Requires-Dist: numpy<2,>=1.22.0
+Requires-Dist: pandas>=1.4.2
+Requires-Dist: pod5>=0.1.21
+Requires-Dist: pomegranate>1.0.0
+Requires-Dist: pysam>=0.19.1
+Requires-Dist: scanpy>=1.9
+Requires-Dist: scikit-learn>=1.0.2
+Requires-Dist: scipy>=1.7.3
+Requires-Dist: seaborn>=0.11
+Requires-Dist: torch>=1.9.0
+Requires-Dist: tqdm
+Provides-Extra: base-tests
+Requires-Dist: pytest; extra == 'base-tests'
+Requires-Dist: pytest-cov; extra == 'base-tests'
+Provides-Extra: docs
+Requires-Dist: ipython>=7.20; extra == 'docs'
+Requires-Dist: matplotlib!=3.6.1; extra == 'docs'
+Requires-Dist: myst-nb>=1; extra == 'docs'
+Requires-Dist: myst-parser>=2; extra == 'docs'
+Requires-Dist: nbsphinx>=0.9; extra == 'docs'
+Requires-Dist: readthedocs-sphinx-search; extra == 'docs'
+Requires-Dist: setuptools; extra == 'docs'
+Requires-Dist: sphinx-autodoc-typehints>=1.25.2; extra == 'docs'
+Requires-Dist: sphinx-book-theme>=1.1.0; extra == 'docs'
+Requires-Dist: sphinx-copybutton; extra == 'docs'
+Requires-Dist: sphinx-design; extra == 'docs'
+Requires-Dist: sphinx>=7; extra == 'docs'
+Requires-Dist: sphinxcontrib-bibtex; extra == 'docs'
+Requires-Dist: sphinxext-opengraph; extra == 'docs'
+Description-Content-Type: text/markdown
+
+[![PyPI](https://img.shields.io/pypi/v/smftools.svg)](https://pypi.org/project/smftools)
+[![Docs](https://readthedocs.org/projects/smftools/badge/?version=latest)](https://smftools.readthedocs.io/en/latest/?badge=latest)
+
+# smftools
+A Python tool for processing raw sequencing data derived from single molecule footprinting experiments into [anndata](https://anndata.readthedocs.io/en/latest/) objects. Additional functionality for preprocessing, analysis, and visualization. Data structures are compatible with analyses developed within the [scverse](https://github.com/scverse) project, including [scanpy](https://github.com/scverse/scanpy) and [scvi-tools](https://github.com/scverse/scvi-tools).
+
+## Philosophy
+While most genomic data structures handle low-coverage data (<100X) along large references, smftools prioritizes high-coverage data (scalable to at least 1 million X coverage) of a few genomic loci at a time. This enables efficient data storage, rapid data operations, hierarchical metadata handling, seamless integration with various machine-learning packages, and ease of visualization. Furthermore, functionality is modularized, enabling analysis sessions to be saved, reloaded, and easily shared with collaborators. Analyses are centered around the [anndata](https://anndata.readthedocs.io/en/latest/) object, and are heavily inspired by the work conducted within the single-cell genomics community.
+
+## Dependencies
+The following CLI tools need to be installed and configured before using the informatics (smftools.inform) module of smftools:
+1) [Dorado](https://github.com/nanoporetech/dorado) -> For standard/modified basecalling and alignment. Can be attained by downloading and configuring nanopore MinKnow software.
+2) [Samtools](https://github.com/samtools/samtools) -> For working with SAM/BAM files
+3) [Minimap2](https://github.com/lh3/minimap2) -> The aligner used by Dorado
+4) [Modkit](https://github.com/nanoporetech/modkit) -> Extracting summary statistics and read level methylation calls from modified BAM files
+
+## Modules
+- Informatics: Processes raw SMF data coming from Nanopore POD5 files, BAM files, or FASTQ files and organizes it into an AnnData object.
+- Preprocessing: Filters the AnnData object on read length, total methylation, and a variety of QC metrics.
+- Tools: Appends various analyses to the AnnData object.
+- Plotting: Visualization of analyses stored within the AnnData object.
+
+## Announcements
+### 09/09/24 - The pre-alpha phase package ([smftools-0.1.1](https://pypi.org/project/smftools/))
+The informatics module has been bumped to alpha-phase status. This module can deal with POD5s and unaligned BAMS from nanopore conversion and direct SMF experiments, as well as FASTQs from Illumina conversion SMF experiments. Primary output from this module is an AnnData object containing all relevant SMF data, which is compatible with all downstream smftools modules. The other modules are still in pre-alpha phase. Preprocessing, Tools, and Plotting modules should be promoted to alpha-phase within the next month or so.
+
+### 08/30/24 - The pre-alpha phase package ([smftools-0.1.0](https://pypi.org/project/smftools/)) is installable through pypi!
+Currently, this package (smftools-0.1.0) is going through rapid improvement (dependency handling accross Linux and Mac OS, testing, documentation, debugging) and is still too early in development for standard use. The underlying functionality was originally developed as a collection of scripts for single molecule footprinting (SMF) experiments in our lab, but is being packaged/developed to facilitate the expansion of SMF to any lab that is interested in performing these styles of experiments/analyses. The alpha-phase package is expected to be available within a couple months, so stay tuned!

smftools-0.1.1/README.md

@@ -0,0 +1,28 @@
+[![PyPI](https://img.shields.io/pypi/v/smftools.svg)](https://pypi.org/project/smftools)
+[![Docs](https://readthedocs.org/projects/smftools/badge/?version=latest)](https://smftools.readthedocs.io/en/latest/?badge=latest)
+
+# smftools
+A Python tool for processing raw sequencing data derived from single molecule footprinting experiments into [anndata](https://anndata.readthedocs.io/en/latest/) objects. Additional functionality for preprocessing, analysis, and visualization. Data structures are compatible with analyses developed within the [scverse](https://github.com/scverse) project, including [scanpy](https://github.com/scverse/scanpy) and [scvi-tools](https://github.com/scverse/scvi-tools).
+
+## Philosophy
+While most genomic data structures handle low-coverage data (<100X) along large references, smftools prioritizes high-coverage data (scalable to at least 1 million X coverage) of a few genomic loci at a time. This enables efficient data storage, rapid data operations, hierarchical metadata handling, seamless integration with various machine-learning packages, and ease of visualization. Furthermore, functionality is modularized, enabling analysis sessions to be saved, reloaded, and easily shared with collaborators. Analyses are centered around the [anndata](https://anndata.readthedocs.io/en/latest/) object, and are heavily inspired by the work conducted within the single-cell genomics community.
+
+## Dependencies
+The following CLI tools need to be installed and configured before using the informatics (smftools.inform) module of smftools:
+1) [Dorado](https://github.com/nanoporetech/dorado) -> For standard/modified basecalling and alignment. Can be attained by downloading and configuring nanopore MinKnow software.
+2) [Samtools](https://github.com/samtools/samtools) -> For working with SAM/BAM files
+3) [Minimap2](https://github.com/lh3/minimap2) -> The aligner used by Dorado
+4) [Modkit](https://github.com/nanoporetech/modkit) -> Extracting summary statistics and read level methylation calls from modified BAM files
+
+## Modules
+- Informatics: Processes raw SMF data coming from Nanopore POD5 files, BAM files, or FASTQ files and organizes it into an AnnData object.
+- Preprocessing: Filters the AnnData object on read length, total methylation, and a variety of QC metrics.
+- Tools: Appends various analyses to the AnnData object.
+- Plotting: Visualization of analyses stored within the AnnData object.
+
+## Announcements
+### 09/09/24 - The pre-alpha phase package ([smftools-0.1.1](https://pypi.org/project/smftools/))
+The informatics module has been bumped to alpha-phase status. This module can deal with POD5s and unaligned BAMS from nanopore conversion and direct SMF experiments, as well as FASTQs from Illumina conversion SMF experiments. Primary output from this module is an AnnData object containing all relevant SMF data, which is compatible with all downstream smftools modules. The other modules are still in pre-alpha phase. Preprocessing, Tools, and Plotting modules should be promoted to alpha-phase within the next month or so.
+
+### 08/30/24 - The pre-alpha phase package ([smftools-0.1.0](https://pypi.org/project/smftools/)) is installable through pypi!
+Currently, this package (smftools-0.1.0) is going through rapid improvement (dependency handling accross Linux and Mac OS, testing, documentation, debugging) and is still too early in development for standard use. The underlying functionality was originally developed as a collection of scripts for single molecule footprinting (SMF) experiments in our lab, but is being packaged/developed to facilitate the expansion of SMF to any lab that is interested in performing these styles of experiments/analyses. The alpha-phase package is expected to be available within a couple months, so stay tuned!

smftools-0.1.1/docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

smftools-0.1.1/docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

smftools-0.1.1/docs/source/_static/tmp

@@ -0,0 +1 @@
+

smftools-0.1.1/docs/source/_templates/tmp

@@ -0,0 +1 @@
+

smftools-0.1.1/docs/source/api/datasets.md

@@ -0,0 +1,9 @@
+## Datasets:
+
+```{eval-rst}
+.. module:: smftools.datasets
+```
+
+```{eval-rst}
+.. currentmodule:: smftools
+```

smftools-0.1.1/docs/source/api/index.md

@@ -0,0 +1,16 @@
+# API
+
+Import smftools as:
+
+```
+import smftools as smf
+```
+
+```{toctree}
+:maxdepth: 2
+
+informatics
+preprocessing
+tools
+datasets
+```

smftools-0.1.1/docs/source/api/informatics.md

@@ -0,0 +1,11 @@
+## Informatics: `inform`
+
+```{eval-rst}
+.. module:: smftools.inform
+```
+
+```{eval-rst}
+.. currentmodule:: smftools
+```
+
+Processes raw sequencing data to load an adata object.

smftools-0.1.1/docs/source/api/preprocessing.md

@@ -0,0 +1,9 @@
+## Preprocessing: `pp`
+
+```{eval-rst}
+.. module:: smftools.pp
+```
+
+```{eval-rst}
+.. currentmodule:: smftools
+```

smftools-0.1.1/docs/source/api/tools.md

@@ -0,0 +1,9 @@
+## Tools: `tl`
+
+```{eval-rst}
+.. module:: smftools.tl
+```
+
+```{eval-rst}
+.. currentmodule:: smftools
+```

smftools-0.1.1/docs/source/conf.py

@@ -0,0 +1,117 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+import sys
+import os
+from pathlib import Path
+HERE = Path(__file__).parent
+PARENT_PARENT_HERE = HERE.parents[1]
+SRC_PATH = PARENT_PARENT_HERE / 'src'
+sys.path.insert(0, str(SRC_PATH))
+for x in os.walk(str(SRC_PATH)):
+  sys.path.insert(0, x[0])
+print(sys.path)
+try:
+    import smftools
+    print("smftools imported successfully.")
+except ImportError:
+    print("smftools is not imported.")
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'smftools'
+copyright = '2024, Joseph McKenna'
+author = 'Joseph McKenna'
+release = '0.1.0'
+repository_url = 'https://github.com/jkmckenna/smftools'
+
+# -- General configuration ---------------------------------------------------
+# Bibliography settings
+bibtex_bibfiles = ["references.bib"]
+bibtex_reference_style = "author_year"
+nitpicky = True
+needs_sphinx = "4.0"
+
+master_doc = "index"
+templates_path = ['_templates']
+exclude_patterns = [
+    "_build",
+    "Thumbs.db",
+    ".DS_Store",
+    # exclude version md files
+    "release-notes/[!i]*.md"
+]
+extensions = [
+    "myst_nb",
+    "sphinx_copybutton",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.doctest",
+    "sphinx.ext.coverage",
+    "sphinx.ext.mathjax",
+    "sphinx.ext.napoleon",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.extlinks",
+    "sphinxcontrib.bibtex",
+    "matplotlib.sphinxext.plot_directive",
+    "sphinx_autodoc_typehints",
+    "sphinx_design",
+    "sphinx_search.extension",
+    "sphinxext.opengraph",
+]
+
+# Generate the API documentation when building
+autosummary_generate = True
+autodoc_member_order = "bysource"
+napoleon_google_docstring = True
+napoleon_numpy_docstring = False
+napoleon_include_init_with_doc = False
+napoleon_use_rtype = True  # having a separate entry generally helps readability
+napoleon_use_param = True
+napoleon_custom_sections = [("Params", "Parameters")]
+todo_include_todos = False
+api_dir = HERE / "api" 
+myst_enable_extensions = [
+    "amsmath",
+    "colon_fence",
+    "deflist",
+    "dollarmath",
+    "html_image",
+    "html_admonition",
+]
+myst_url_schemes = ("http", "https", "mailto", "ftp")
+myst_heading_anchors = 3
+nb_output_stderr = "remove"
+nb_execution_mode = "off"
+nb_merge_streams = True
+
+suppress_warnings = [
+    "myst.header"
+]
+
+typehints_defaults = "braces"
+
+# html_context = {
+#     "display_github": True,
+#     "github_user": "jkmckenna",
+#     "github_repo": project,
+#     "github_version": "main",
+#     "conf_py_path": "/docs/source/",
+# }
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "sphinx_book_theme"
+html_title = project
+
+html_theme_options = {
+    "repository_url": repository_url,
+    "use_repository_button": True,
+    "show_toc_level": 1,
+    "path_to_docs": "docs/",
+    "repository_branch": release,
+}
+
+html_static_path = ['_static']

smftools-0.1.1/docs/source/contributors.md

@@ -0,0 +1,9 @@
+# Contributors
+
+## Current
+
+- Joseph Mckenna, lead developer (2024)
+
+## Acknowledgments
+
+- [Tjian/Darzacq Lab](https://www.tjian-darzacq.mcb.berkeley.edu/): Funding and a supportive lab environment!

smftools-0.1.1/docs/source/dev/index.md

@@ -0,0 +1,3 @@
+(contribution-guide)=
+
+# Contributing

smftools-0.1.1/docs/source/index.md

@@ -0,0 +1,53 @@
+smftools documentation
+======================
+
+```{include} ../../README.md
+:end-before: '## Dependencies'
+```
+
+::::{grid} 1 2 2 2
+:gutter: 2
+
+:::{grid-item-card} Installation {octicon}`plug;1em;`
+:link: installation
+:link-type: doc
+
+smftools installation instructions
+:::
+
+:::{grid-item-card} Tutorials {octicon}`play;1em;`
+:link: tutorials/index
+:link-type: doc
+
+Jupyter notebook tutorial of smftools usage.
+:::
+
+:::{grid-item-card} API reference {octicon}`book;1em;`
+:link: api/index
+:link-type: doc
+
+The API reference contains a detailed description of
+the smftools API.
+:::
+
+:::{grid-item-card} GitHub {octicon}`mark-github;1em;`
+:link: https://github.com/jkmckenna/smftools
+
+smftools GitHub link
+:::
+::::
+
+```{toctree}
+:hidden: true
+:maxdepth: 1
+
+installation
+tutorials/index
+api/index
+release-notes/index
+dev/index
+contributors
+references
+```
+
+[github]: https://github.com/jkmckenna/smftools

smftools-0.1.1/docs/source/installation.md

@@ -0,0 +1,20 @@
+# Installation
+
+## PyPi version
+
+Pull smftools from [PyPI](https://pypi.org/project/smftools):
+
+```shell
+pip install smftools
+```
+
+## Development Version
+
+```shell
+git clone https://github.com/jkmckenna/smftools.git
+cd smftools
+```
+
+```shell
+pip install .
+```