mirna-toolkit 0.1.0__tar.gz

mirna_toolkit-0.1.0/PKG-INFO

@@ -0,0 +1,147 @@
+Metadata-Version: 2.4
+Name: mirna-toolkit
+Version: 0.1.0
+Summary: Python-native end-to-end toolkit for miRNA analysis
+Author: miRNA-Toolkit Contributors
+License-Expression: MIT
+Keywords: mirna,bioinformatics,rna,genomics,small-rna
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.31
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2.0
+Requires-Dist: networkx>=3.2
+Requires-Dist: plotly>=5.20
+Requires-Dist: matplotlib>=3.8
+Requires-Dist: seaborn>=0.13
+Provides-Extra: full
+Requires-Dist: pysam>=0.22; extra == "full"
+Requires-Dist: scikit-learn>=1.4; extra == "full"
+Requires-Dist: shap>=0.45; extra == "full"
+Requires-Dist: lime>=0.2.0.1; extra == "full"
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+
+# miRNA-Toolkit
+
+miRNA-Toolkit is a Python-native library for end-to-end miRNA analysis, including preprocessing, alignment, quantification, target prediction, disease association, and visualization.
+
+Install name: `mirna-toolkit`
+Import name: `mirna_toolkit`
+
+## Features
+
+- I/O support for FASTQ, FASTA, BAM, and GTF/GFF-oriented workflows.
+- Wrappers for FastQC, fastp, Bowtie, HISAT2, and STAR (small-RNA mode).
+- Quantification and normalization (TPM, RPM, DESeq2-style size factors).
+- Target prediction (seed match + RNAhybrid-style scoring + external connectors).
+- Disease association integration (HMDD, miRCancer) and network export.
+- Visualization utilities for interaction networks, heatmaps, volcano plots, and pathway enrichment plots.
+- Optional ML utilities for classification, feature ranking, and explainability.
+- End-to-end reproducible workflow runner (`pipeline.run_end_to_end`) and CLI (`mirna-toolkit`).
+- Manifest-based batch workflow for multiple samples (`batch.run_batch_workflow`).
+- Retry-aware HTTP connectors with configurable strict mode for robust integrations.
+
+## Installation
+
+```bash
+pip install mirna-toolkit
+```
+
+For local development:
+
+```bash
+pip install -e .
+```
+
+Optional extras:
+
+```bash
+pip install -e .[full,dev]
+```
+
+## Quick Start
+
+```python
+from mirna_toolkit import alignment, quantification, prediction, disease, visualization
+
+bam_file = alignment.bowtie.align("sample.fastq", reference="mirbase.fa")
+counts = quantification.counts.from_bam(bam_file, annotation="mirbase.gtf")
+normalized = quantification.normalization.tpm(counts)
+
+targets = prediction.targetscan_api.get_targets("hsa-miR-21")
+assoc = disease.hmdd.get_associations("hsa-miR-21")
+
+fig = visualization.networks.plot_mirna_targets("hsa-miR-21", targets)
+fig.show()
+```
+
+## End-to-End Pipeline
+
+```python
+from mirna_toolkit.pipeline import PipelineConfig, run_end_to_end
+
+config = PipelineConfig(
+	fastq_path="sample.fastq",
+	reference="mirbase_index",
+	annotation="mirbase.gtf",
+	output_dir="run_001",
+	aligner="bowtie",
+	normalization_method="tpm",
+	run_qc=True,
+	trim=True,
+)
+
+result = run_end_to_end(config, database_versions={"miRBase": "22.1", "TargetScan": "8.0"})
+print(result.metadata_path)
+```
+
+## CLI
+
+```bash
+mirna-toolkit --fastq sample.fastq --reference mirbase_index --annotation mirbase.gtf --output-dir run_001 --aligner bowtie --normalize tpm --run-qc --trim
+```
+
+Helpful CLI topics:
+
+```bash
+mirna-toolkit help
+mirna-toolkit help run
+mirna-toolkit help batch
+```
+
+## Batch Workflow
+
+Manifest example (`samples.csv`):
+
+```csv
+sample_id,fastq_path,reference,annotation,run_qc,trim
+s1,s1.fastq,mirbase_index,mirbase.gtf,true,true
+s2,s2.fastq,mirbase_index,mirbase.gtf,false,false
+```
+
+Run it with:
+
+```bash
+mirna-toolkit batch --manifest samples.csv --output-dir batch_runs
+```
+
+## Reproducibility
+
+Use `mirna_toolkit.utils.versioning.DatabaseVersionTracker` to record exact versions/releases of external databases used in analyses.
+
+## Status
+
+This repository provides a production-ready scaffold with working core APIs and wrappers. External command wrappers require the corresponding tools (for example, `bowtie`, `fastqc`, `fastp`) to be installed and available on `PATH`.

mirna_toolkit-0.1.0/README.md

@@ -0,0 +1,111 @@
+# miRNA-Toolkit
+
+miRNA-Toolkit is a Python-native library for end-to-end miRNA analysis, including preprocessing, alignment, quantification, target prediction, disease association, and visualization.
+
+Install name: `mirna-toolkit`
+Import name: `mirna_toolkit`
+
+## Features
+
+- I/O support for FASTQ, FASTA, BAM, and GTF/GFF-oriented workflows.
+- Wrappers for FastQC, fastp, Bowtie, HISAT2, and STAR (small-RNA mode).
+- Quantification and normalization (TPM, RPM, DESeq2-style size factors).
+- Target prediction (seed match + RNAhybrid-style scoring + external connectors).
+- Disease association integration (HMDD, miRCancer) and network export.
+- Visualization utilities for interaction networks, heatmaps, volcano plots, and pathway enrichment plots.
+- Optional ML utilities for classification, feature ranking, and explainability.
+- End-to-end reproducible workflow runner (`pipeline.run_end_to_end`) and CLI (`mirna-toolkit`).
+- Manifest-based batch workflow for multiple samples (`batch.run_batch_workflow`).
+- Retry-aware HTTP connectors with configurable strict mode for robust integrations.
+
+## Installation
+
+```bash
+pip install mirna-toolkit
+```
+
+For local development:
+
+```bash
+pip install -e .
+```
+
+Optional extras:
+
+```bash
+pip install -e .[full,dev]
+```
+
+## Quick Start
+
+```python
+from mirna_toolkit import alignment, quantification, prediction, disease, visualization
+
+bam_file = alignment.bowtie.align("sample.fastq", reference="mirbase.fa")
+counts = quantification.counts.from_bam(bam_file, annotation="mirbase.gtf")
+normalized = quantification.normalization.tpm(counts)
+
+targets = prediction.targetscan_api.get_targets("hsa-miR-21")
+assoc = disease.hmdd.get_associations("hsa-miR-21")
+
+fig = visualization.networks.plot_mirna_targets("hsa-miR-21", targets)
+fig.show()
+```
+
+## End-to-End Pipeline
+
+```python
+from mirna_toolkit.pipeline import PipelineConfig, run_end_to_end
+
+config = PipelineConfig(
+	fastq_path="sample.fastq",
+	reference="mirbase_index",
+	annotation="mirbase.gtf",
+	output_dir="run_001",
+	aligner="bowtie",
+	normalization_method="tpm",
+	run_qc=True,
+	trim=True,
+)
+
+result = run_end_to_end(config, database_versions={"miRBase": "22.1", "TargetScan": "8.0"})
+print(result.metadata_path)
+```
+
+## CLI
+
+```bash
+mirna-toolkit --fastq sample.fastq --reference mirbase_index --annotation mirbase.gtf --output-dir run_001 --aligner bowtie --normalize tpm --run-qc --trim
+```
+
+Helpful CLI topics:
+
+```bash
+mirna-toolkit help
+mirna-toolkit help run
+mirna-toolkit help batch
+```
+
+## Batch Workflow
+
+Manifest example (`samples.csv`):
+
+```csv
+sample_id,fastq_path,reference,annotation,run_qc,trim
+s1,s1.fastq,mirbase_index,mirbase.gtf,true,true
+s2,s2.fastq,mirbase_index,mirbase.gtf,false,false
+```
+
+Run it with:
+
+```bash
+mirna-toolkit batch --manifest samples.csv --output-dir batch_runs
+```
+
+## Reproducibility
+
+Use `mirna_toolkit.utils.versioning.DatabaseVersionTracker` to record exact versions/releases of external databases used in analyses.
+
+## Status
+
+This repository provides a production-ready scaffold with working core APIs and wrappers. External command wrappers require the corresponding tools (for example, `bowtie`, `fastqc`, `fastp`) to be installed and available on `PATH`.

mirna_toolkit-0.1.0/pyproject.toml

@@ -0,0 +1,70 @@
+[build-system]
+requires = ["setuptools>=69", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mirna-toolkit"
+version = "0.1.0"
+description = "Python-native end-to-end toolkit for miRNA analysis"
+readme = "README.md"
+requires-python = ">=3.9"
+license = "MIT"
+authors = [
+  { name = "miRNA-Toolkit Contributors" }
+]
+keywords = ["mirna", "bioinformatics", "rna", "genomics", "small-rna"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Science/Research",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Topic :: Scientific/Engineering :: Bio-Informatics"
+]
+dependencies = [
+  "requests>=2.31",
+  "numpy>=1.24",
+  "pandas>=2.0",
+  "networkx>=3.2",
+  "plotly>=5.20",
+  "matplotlib>=3.8",
+  "seaborn>=0.13"
+]
+
+[project.optional-dependencies]
+full = [
+  "pysam>=0.22",
+  "scikit-learn>=1.4",
+  "shap>=0.45",
+  "lime>=0.2.0.1"
+]
+dev = [
+  "build>=1.2",
+  "pytest>=8.0",
+  "pytest-cov>=5.0",
+  "ruff>=0.5",
+  "mypy>=1.10"
+]
+
+[project.scripts]
+mirna-toolkit = "mirna_toolkit.cli:main"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+minversion = "8.0"
+addopts = "-ra -q"
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 150
+target-version = "py39"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP"]

mirna_toolkit-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mirna_toolkit-0.1.0/src/mirna_toolkit/__init__.py

@@ -0,0 +1,17 @@
+"""miRNA-Toolkit public package API."""
+
+from . import alignment, batch, disease, io, ml, pipeline, prediction, quantification, visualization
+
+__all__ = [
+    "alignment",
+    "batch",
+    "disease",
+    "io",
+    "ml",
+    "pipeline",
+    "prediction",
+    "quantification",
+    "visualization",
+]
+
+__version__ = "0.1.0"

mirna_toolkit-0.1.0/src/mirna_toolkit/alignment/__init__.py

@@ -0,0 +1,6 @@
+from . import bowtie, hisat2, star
+from .bowtie import align as bowtie_align
+from .hisat2 import align as hisat2_align
+from .star import align as star_align
+
+__all__ = ["bowtie", "bowtie_align", "hisat2", "hisat2_align", "star", "star_align"]

mirna_toolkit-0.1.0/src/mirna_toolkit/alignment/bowtie.py

@@ -0,0 +1,21 @@
+import subprocess
+from pathlib import Path
+
+
+def align(
+    fastq_path: str | Path,
+    reference: str | Path,
+    output_bam: str | Path | None = None,
+    threads: int = 4,
+) -> Path:
+    """Align reads with Bowtie and return BAM path."""
+    fastq = Path(fastq_path)
+    output = Path(output_bam) if output_bam else fastq.with_suffix(".bowtie.bam")
+    sam = output.with_suffix(".sam")
+
+    subprocess.run(
+        ["bowtie", "-p", str(threads), "-S", str(reference), str(fastq), str(sam)],
+        check=True,
+    )
+    subprocess.run(["samtools", "view", "-bS", str(sam), "-o", str(output)], check=True)
+    return output

mirna_toolkit-0.1.0/src/mirna_toolkit/alignment/hisat2.py

@@ -0,0 +1,21 @@
+import subprocess
+from pathlib import Path
+
+
+def align(
+    fastq_path: str | Path,
+    reference: str | Path,
+    output_bam: str | Path | None = None,
+    threads: int = 4,
+) -> Path:
+    """Align reads with HISAT2 and return BAM path."""
+    fastq = Path(fastq_path)
+    output = Path(output_bam) if output_bam else fastq.with_suffix(".hisat2.bam")
+    sam = output.with_suffix(".sam")
+
+    subprocess.run(
+        ["hisat2", "-p", str(threads), "-x", str(reference), "-U", str(fastq), "-S", str(sam)],
+        check=True,
+    )
+    subprocess.run(["samtools", "view", "-bS", str(sam), "-o", str(output)], check=True)
+    return output

mirna_toolkit-0.1.0/src/mirna_toolkit/alignment/star.py

@@ -0,0 +1,37 @@
+import subprocess
+from pathlib import Path
+
+
+def align(
+    fastq_path: str | Path,
+    reference: str | Path,
+    output_bam: str | Path | None = None,
+    threads: int = 4,
+    small_rna_mode: bool = True,
+) -> Path:
+    """Align reads with STAR in small-RNA-friendly settings and return BAM path."""
+    fastq = Path(fastq_path)
+    output = Path(output_bam) if output_bam else fastq.with_suffix(".star.bam")
+    prefix = output.with_suffix("")
+
+    cmd = [
+        "STAR",
+        "--genomeDir",
+        str(reference),
+        "--readFilesIn",
+        str(fastq),
+        "--runThreadN",
+        str(threads),
+        "--outSAMtype",
+        "BAM",
+        "Unsorted",
+        "--outFileNamePrefix",
+        f"{prefix}",
+    ]
+    if small_rna_mode:
+        cmd.extend(["--alignIntronMax", "1", "--outFilterMismatchNmax", "1"])
+
+    subprocess.run(cmd, check=True)
+    produced_bam = Path(f"{prefix}Aligned.out.bam")
+    produced_bam.replace(output)
+    return output

mirna_toolkit-0.1.0/src/mirna_toolkit/batch.py

@@ -0,0 +1,178 @@
+import csv
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from . import pipeline
+from .pipeline import PipelineConfig, PipelineResult
+
+
+@dataclass
+class BatchSampleConfig:
+    sample_id: str
+    fastq_path: str | Path
+    reference: str | Path
+    annotation: str | Path
+    output_dir: str | Path | None = None
+    aligner: str = "bowtie"
+    normalization_method: str = "tpm"
+    threads: int = 4
+    run_qc: bool = False
+    trim: bool = False
+
+
+@dataclass
+class BatchSampleResult:
+    sample_id: str
+    bam_path: Path
+    counts_path: Path
+    normalized_path: Path
+    metadata_path: Path
+    counts: dict[str, int]
+    normalized: dict[str, float]
+
+
+@dataclass
+class BatchRunResult:
+    manifest_path: Path
+    summary_json_path: Path
+    summary_csv_path: Path
+    samples: list[BatchSampleResult]
+
+
+def load_manifest(manifest_path: str | Path) -> list[BatchSampleConfig]:
+    path = Path(manifest_path)
+    suffix = path.suffix.lower()
+
+    if suffix == ".csv":
+        return _load_csv_manifest(path)
+    if suffix == ".json":
+        return _load_json_manifest(path)
+    raise ValueError("manifest_path must end with .csv or .json")
+
+
+def run_batch_workflow(
+    manifest_path: str | Path,
+    output_dir: str | Path | None = None,
+    database_versions: dict[str, str] | None = None,
+) -> BatchRunResult:
+    manifest_file = Path(manifest_path)
+    base_output_dir = Path(output_dir) if output_dir is not None else manifest_file.parent / "batch_outputs"
+    base_output_dir.mkdir(parents=True, exist_ok=True)
+
+    sample_configs = load_manifest(manifest_file)
+    sample_results: list[BatchSampleResult] = []
+
+    for sample in sample_configs:
+        sample_output_dir = Path(sample.output_dir) if sample.output_dir is not None else base_output_dir / sample.sample_id
+        config = PipelineConfig(
+            fastq_path=sample.fastq_path,
+            reference=sample.reference,
+            annotation=sample.annotation,
+            output_dir=sample_output_dir,
+            aligner=sample.aligner,
+            normalization_method=sample.normalization_method,
+            threads=sample.threads,
+            run_qc=sample.run_qc,
+            trim=sample.trim,
+        )
+
+        result = pipeline.run_end_to_end(config, database_versions=database_versions)
+        sample_results.append(_to_sample_result(sample.sample_id, result))
+
+    summary_json_path = base_output_dir / "batch_summary.json"
+    summary_csv_path = base_output_dir / "batch_summary.csv"
+    _write_summary(sample_results, summary_json_path, summary_csv_path)
+
+    return BatchRunResult(
+        manifest_path=manifest_file,
+        summary_json_path=summary_json_path,
+        summary_csv_path=summary_csv_path,
+        samples=sample_results,
+    )
+
+
+def _load_csv_manifest(path: Path) -> list[BatchSampleConfig]:
+    with path.open("r", encoding="utf-8", newline="") as handle:
+        reader = csv.DictReader(handle)
+        rows = list(reader)
+    return [_row_to_sample(row, index=index) for index, row in enumerate(rows, start=1)]
+
+
+def _load_json_manifest(path: Path) -> list[BatchSampleConfig]:
+    payload = json.loads(path.read_text(encoding="utf-8"))
+    if isinstance(payload, dict):
+        rows = payload.get("samples", [])
+    elif isinstance(payload, list):
+        rows = payload
+    else:
+        raise ValueError("JSON manifest must be a list or a dict with a samples key")
+    return [_row_to_sample(row, index=index) for index, row in enumerate(rows, start=1)]
+
+
+def _row_to_sample(row: dict[str, Any], index: int) -> BatchSampleConfig:
+    sample_id = str(row.get("sample_id") or row.get("sample") or row.get("id") or f"sample_{index}")
+    fastq_path = row.get("fastq_path") or row.get("fastq")
+    reference = row.get("reference")
+    annotation = row.get("annotation")
+
+    if not fastq_path or not reference or not annotation:
+        raise ValueError("Each manifest row must include fastq_path, reference, and annotation")
+
+    return BatchSampleConfig(
+        sample_id=sample_id,
+        fastq_path=fastq_path,
+        reference=reference,
+        annotation=annotation,
+        output_dir=row.get("output_dir") or None,
+        aligner=str(row.get("aligner") or "bowtie"),
+        normalization_method=str(row.get("normalization_method") or row.get("normalize") or "tpm"),
+        threads=int(row.get("threads") or 4),
+        run_qc=_parse_bool(row.get("run_qc") or row.get("qc") or False),
+        trim=_parse_bool(row.get("trim") or False),
+    )
+
+
+def _to_sample_result(sample_id: str, result: PipelineResult) -> BatchSampleResult:
+    return BatchSampleResult(
+        sample_id=sample_id,
+        bam_path=result.bam_path,
+        counts_path=result.counts_path,
+        normalized_path=result.normalized_path,
+        metadata_path=result.metadata_path,
+        counts=result.counts,
+        normalized=result.normalized,
+    )
+
+
+def _write_summary(samples: list[BatchSampleResult], json_path: Path, csv_path: Path) -> None:
+    summary_rows = [
+        {
+            "sample_id": sample.sample_id,
+            "bam_path": str(sample.bam_path),
+            "counts_path": str(sample.counts_path),
+            "normalized_path": str(sample.normalized_path),
+            "metadata_path": str(sample.metadata_path),
+        }
+        for sample in samples
+    ]
+
+    json_path.write_text(json.dumps(summary_rows, indent=2), encoding="utf-8")
+
+    with csv_path.open("w", encoding="utf-8", newline="") as handle:
+        writer = csv.DictWriter(handle, fieldnames=["sample_id", "bam_path", "counts_path", "normalized_path", "metadata_path"])
+        writer.writeheader()
+        writer.writerows(summary_rows)
+
+
+def _parse_bool(value: Any) -> bool:
+    if isinstance(value, bool):
+        return value
+    if value is None:
+        return False
+    if isinstance(value, (int, float)):
+        return value != 0
+
+    normalized = str(value).strip().lower()
+    return normalized in {"1", "true", "yes", "y", "on"}