singlet 0.4__tar.gz → 1.2.0__tar.gz

singlet-1.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024-2026 Singlet Bio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

singlet-1.2.0/MANIFEST.in

@@ -0,0 +1,31 @@
+include LICENSE
+include README.md
+include python/README.md
+include pyproject.toml
+include setup.py
+
+# C++ sources + headers needed to build the singlet._pz extension from an sdist
+recursive-include src/bindings *.cpp *.h *.hpp
+# only the .1pz codec headers: the extension includes nothing else (the rest of include/ is the C++ pipeline)
+include include/singlet/pileup/pz_reader.h
+include include/singlet/pileup/pz_writer.h
+# the pz2 matrix codec (header-only) and the binding helpers
+recursive-include include/singlet/pz2 *.h
+
+# Bundled package data
+recursive-include python/singlet/data *.parquet *.txt
+recursive-include python/singlet/celllabels/data *.gz
+recursive-include python/singlet/features/data *.json *.parquet
+# ...except the large 10x whitelists, which singlet.whitelists downloads on demand
+exclude python/singlet/data/whitelists/3M-february-2018.txt
+exclude python/singlet/data/whitelists/737K-august-2016.txt
+exclude python/singlet/data/whitelists/737K-cratac-v1.txt
+exclude python/singlet/data/whitelists/gex_737K-arc-v1.txt
+include python/singlet/py.typed
+
+# Keep test/build noise out of the sdist
+prune build
+prune dist
+global-exclude *.pyc
+global-exclude __pycache__
+global-exclude *.so

singlet-1.2.0/PKG-INFO

@@ -0,0 +1,277 @@
+Metadata-Version: 2.4
+Name: singlet
+Version: 1.2.0
+Summary: Open single-cell RNA-seq atlas: find, load, and analyze single-cell data at scale
+Author-email: Zach DeBruine <debruinz@gvsu.edu>
+License: MIT
+Project-URL: Homepage, https://singlet.bio
+Project-URL: Documentation, https://singlet.bio/docs
+Project-URL: Repository, https://github.com/Singlet-Bio/singlet
+Project-URL: Issues, https://github.com/Singlet-Bio/singlet/issues
+Project-URL: Changelog, https://github.com/Singlet-Bio/singlet/blob/main/CHANGELOG.md
+Keywords: single-cell,RNA-seq,bioinformatics,sparse-matrix,pytorch,anndata,GPU
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: C++
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.21
+Requires-Dist: scipy>=1.7
+Requires-Dist: pandas>=1.3
+Requires-Dist: anndata>=0.8
+Requires-Dist: requests>=2.25
+Requires-Dist: tqdm>=4.60
+Requires-Dist: scikit-learn>=1.0
+Requires-Dist: zstandard>=0.20
+Requires-Dist: pyarrow>=10.0
+Provides-Extra: torch
+Requires-Dist: torch>=2.0; extra == "torch"
+Provides-Extra: gpu
+Requires-Dist: cupy-cuda12x>=13.0; extra == "gpu"
+Provides-Extra: zarr
+Requires-Dist: zarr>=2.12; extra == "zarr"
+Provides-Extra: tiledb
+Requires-Dist: tiledbsoma>=1.4; extra == "tiledb"
+Provides-Extra: mcp
+Requires-Dist: mcp<2,>=1.0; extra == "mcp"
+Provides-Extra: migrate
+Requires-Dist: numba>=0.57; extra == "migrate"
+Provides-Extra: analysis
+Requires-Dist: matplotlib>=3.5; extra == "analysis"
+Requires-Dist: scanpy>=1.9; extra == "analysis"
+Requires-Dist: statsmodels>=0.13; extra == "analysis"
+Requires-Dist: igraph>=0.10; extra == "analysis"
+Requires-Dist: leidenalg>=0.9; extra == "analysis"
+Provides-Extra: all
+Requires-Dist: singlet[analysis,gpu,mcp,tiledb,torch,zarr]; extra == "all"
+Provides-Extra: test
+Requires-Dist: pytest>=7.0; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Requires-Dist: coverage>=7.0; extra == "test"
+Requires-Dist: hypothesis>=6.0; extra == "test"
+Requires-Dist: singlet[analysis]; extra == "test"
+Provides-Extra: dev
+Requires-Dist: singlet[test]; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: pyright; extra == "dev"
+Requires-Dist: pybind11>=2.11; extra == "dev"
+Dynamic: license-file
+
+# singlet
+
+**Find, load, and analyze re-processed public single-cell RNA-seq studies with one line of Python.**
+
+Singlet re-processes public GEO single-cell RNA-seq studies through one
+uniform pipeline and publishes each study as a `.singlet` file. You work with
+two simple things: GEO accession strings (`GSE…` / `GSM…`) and `.singlet`
+files. Everything loads as [AnnData](https://anndata.readthedocs.io/), ready
+for scanpy or PyTorch. Browse the catalog at [singlet.bio](https://singlet.bio).
+
+- **Data: CC0** (public domain) — **Code: MIT**.
+- Downloads need no login and no API key.
+
+## Install
+
+```bash
+pip install singlet
+```
+
+Wheels for Linux (x86_64, aarch64), macOS (Intel, Apple silicon) and Windows include the compiled `.1pz` / `.s1`
+codec, so no compiler is needed. The development version installs from GitHub:
+
+```bash
+pip install "singlet @ git+https://github.com/Singlet-Bio/singlet"
+```
+
+Building from source compiles the package's `.1pz` codec, so you need a **C++17 compiler**
+and **zstd** (headers + library): `apt install build-essential libzstd-dev`
+on Debian/Ubuntu, `brew install zstd` on macOS, or
+`conda install -c conda-forge zstd`. Linux, macOS and Windows are
+supported (Windows needs Visual Studio Build Tools and zstd to build from source; wheels are built in CI).
+
+Optional extras (same URL, extra in brackets):
+
+```bash
+pip install "singlet[analysis] @ git+https://github.com/Singlet-Bio/singlet"  # matplotlib, scanpy, statsmodels, igraph, leidenalg
+pip install "singlet[torch] @ git+https://github.com/Singlet-Bio/singlet"     # PyTorch DataLoaders
+pip install "singlet[zarr] @ git+https://github.com/Singlet-Bio/singlet"      # Zarr export/import
+pip install "singlet[mcp] @ git+https://github.com/Singlet-Bio/singlet"       # local MCP server (singlet-mcp)
+```
+
+## Quick start
+
+```python
+import singlet
+
+# Find studies in plain English → GSE accessions (live search)
+studies = singlet.find("human PBMC 10x")
+
+# Load a study → AnnData (free download, cached under ~/.singlet/cache)
+adata = singlet.load("GSE138867")
+
+# Load one sample: the parent study is downloaded, then filtered to it
+pbmc1 = singlet.load("GSM4120733")
+
+# Load and concatenate several studies into ONE AnnData
+adata = singlet.load(["GSE138867", "GSE146974"])
+
+# Per-cell labels come with every per-sample study (docs/cell_annotation.md section 8)
+adata.obs[
+    ["cell_type", "cell_type_ontology_term_id", "sex_chrom", "ancestry_hancestro", "called_ed"]
+]
+# ... and can select cells before anything else is downloaded
+heart = singlet.cells("GSE286650", "cell_type == 'cardiac muscle cell' and sex_chrom == 'male'")
+
+# Load a local .singlet file
+adata = singlet.load("path/to/GSE138867.singlet")
+
+# Search and load the top three matching studies in one step
+adata = singlet.find_load("human pancreas islet cells")
+```
+
+`load()` accepts a single accession, a single `.singlet` path, or a list/tuple
+mixing both. Lists are concatenated into one AnnData; `obs["source"]` and
+`obs["dataset"]` record where each cell came from.
+
+What you get back:
+
+- `X`: gene-level UMI counts (cells × genes, sparse), with `layers["spliced"]`
+  and `layers["unspliced"]` when both are available.
+- `var_names`: Ensembl gene ids; `var["gene_name"]`: gene symbols.
+- `obs`: `gsm_id`, organism, protocol and other per-sample metadata.
+- `uns["study_meta"]`, `uns["manifest"]`: study metadata and the file's
+  manifest (stored so that `write_h5ad` / `write_zarr` work).
+- `uns["skipped_samples"]`: present only when some samples had an empty count
+  matrix; those are skipped with a warning. A study with no usable sample
+  raises an error pointing to its page on singlet.bio.
+
+Pick genes by symbol or by Ensembl id:
+
+```python
+adata = singlet.load("GSE138867", genes=["CD3E", "MS4A1", "ENSG00000105374"])
+```
+
+Symbols are matched exactly first, then case-insensitively; anything
+unmatched is reported in a warning.
+
+## Natural-language search
+
+```python
+import singlet
+
+singlet.find("exhausted T cells in melanoma")  # study (GSE) accessions
+singlet.find("mouse brain 10x", level="gsm", limit=20)  # sample (GSM) accessions
+
+# find + load in one call (top 3 studies by default; raise limit deliberately)
+adata = singlet.find_load("human kidney organoids", limit=5)
+```
+
+Search runs against the hosted endpoint (`https://singlet.bio/api`; override
+with `$SINGLET_API_BASE`). It is rate-limited per client; for heavier use,
+create a key at https://singlet.bio/account and call `singlet.set_api_key()`
+(or set `$SINGLET_API_KEY`).
+
+## Browse the catalog snapshot (offline)
+
+```python
+import singlet
+
+singlet.summary()  # snapshot overview
+singlet.samples(organism="Homo sapiens", status="SUCCESS")
+singlet.tissues()  # tissue breakdown
+singlet.info("GSE138867")  # metadata
+```
+
+These functions read a catalog snapshot bundled with the package, so they
+work offline but lag the live catalog. `singlet.info()` falls back to the
+live API (`/api/gse/<GSE>`, `/api/gsm/<GSM>`) for accessions the snapshot
+does not know, and `python -m singlet` prints live totals from
+`https://singlet.bio/api/stats`. Set `SINGLET_OFFLINE=1` to disable those
+live lookups.
+
+## Export
+
+```python
+singlet.to_h5ad(adata, "study.h5ad")
+singlet.to_zarr(adata, "study.zarr")  # needs singlet[zarr]
+singlet.to_mtx(adata, "study_mtx/")  # 10x-style matrix.mtx.gz + barcodes + features
+```
+
+Loom export is not supported from Python.
+
+## Standard analysis (scanpy-compatible)
+
+```python
+import scanpy as sc
+
+adata = singlet.load("GSE138867")
+sc.pp.normalize_total(adata, target_sum=1e4)
+sc.pp.log1p(adata)
+sc.pp.highly_variable_genes(adata)
+sc.tl.pca(adata)
+sc.pp.neighbors(adata)
+sc.tl.umap(adata)
+sc.tl.leiden(adata)
+```
+
+The package also has built-in equivalents (`singlet.normalize`,
+`singlet.pca`, `singlet.neighbors`, `singlet.leiden`, `singlet.umap`, ...)
+that do not need scanpy.
+
+## PyTorch training
+
+```python
+from singlet.torch import SingletDataset, DataLoader
+
+# An array of .singlet files, concatenated into one training set
+loader = DataLoader(
+    ["a.singlet", "b.singlet", "c.singlet"],
+    batch_size=512,
+    device="cuda",
+    normalize=True,
+)
+
+# A single dataset / accession / AnnData also works
+ds = SingletDataset("GSE138867", normalize=True)
+```
+
+## MCP servers
+
+- **Hosted:** `https://singlet.bio/mcp` serves the live catalog to MCP clients;
+  nothing to install.
+- **Local:** `singlet-mcp` (install `singlet[mcp]`) runs 12 tools over the
+  offline catalog snapshot plus live search, over stdio. The old command name
+  `singlet` still works but is deprecated.
+
+## Configuration
+
+| Variable | Meaning |
+|---|---|
+| `SINGLET_DATA_BASE` | Bundle host: files live at `<base>/data/<GSE>/<GSE>.singlet` (default `https://data.singlet.bio`; a trailing `/data` is accepted, so the R client's value works too) |
+| `SINGLET_SAMPLE_MIRROR` | Self-hosted mirror of per-sample directories for `singlet.fetch()` (no public default) |
+| `SINGLET_API_BASE` | REST API base (default `https://singlet.bio/api`) |
+| `SINGLET_CACHE_DIR` | Download cache (default `~/.singlet/cache`) |
+| `SINGLET_API_KEY` | Optional key for natural-language search |
+| `SINGLET_OFFLINE` | `1` disables the live lookups in `info()` and `python -m singlet` |
+
+## Links
+
+- Homepage: https://singlet.bio
+- Documentation: https://singlet.bio/docs
+- Source: https://github.com/Singlet-Bio/singlet
+
+## License
+
+Code: MIT. Catalog data: CC0 (public domain).

singlet-1.2.0/README.md

@@ -0,0 +1,183 @@
+# singlet
+
+A single-cell genomics toolkit: process, load, and analyze public single-cell
+RNA-seq data. It is the client for the [singlet.bio](https://singlet.bio)
+catalog of re-processed GEO studies.
+
+**One repository. Three languages.**
+
+```bash
+pip install singlet                    # Python
+```
+```r
+remotes::install_github("Singlet-Bio/singlet", subdir = "r")  # R
+```
+```cmake
+find_package(Singlet REQUIRED)         # C++
+```
+
+Wheels for Linux (x86_64, aarch64), macOS (Intel, Apple silicon) and Windows include the compiled `.1pz` / `.s1`
+codec, so no compiler is needed.
+
+Installing the Python package from source (`pip install "singlet @ git+https://github.com/Singlet-Bio/singlet"`) compiles a small C++ extension (the
+`.1pz` codec), so you need a **C++17 compiler** and **zstd** headers + library
+(`apt install build-essential libzstd-dev`, `brew install zstd`, or
+`conda install -c conda-forge zstd`). Linux, macOS and Windows are
+supported (Windows needs Visual Studio Build Tools and zstd to build from source; wheels are built in CI).
+
+## What singlet does
+
+| Layer | Capability |
+|-------|-----------|
+| **Catalog** | Search and load re-processed GEO studies from singlet.bio as AnnData — see the site for current numbers |
+| **Format** | `.singlet` per-study bundles and the `.1pz` sparse-matrix codec |
+| **Pipeline** | Raw SRA reads → aligned, deduplicated, annotated count matrices (C++ binary) |
+| **Analysis** | PCA, neighbors, Leiden, UMAP, DE, batch correction — no scanpy required |
+| **PyTorch** | Sparse DataLoaders over `.singlet` files for ML training |
+
+## Quick Start
+
+```python
+import singlet
+
+# Find studies in plain English (live search) → GSE accessions
+singlet.find("human PBMC 10x")
+
+# Load a study → AnnData (free download from data.singlet.bio, cached)
+adata = singlet.load("GSE138867")
+
+# Load one sample of it (downloads the parent study, keeps this sample's cells)
+pbmc1 = singlet.load("GSM4120733")
+
+# Load several studies into ONE AnnData (obs["dataset"] records each study)
+adata = singlet.load(["GSE138867", "GSE146974"])
+
+# Subset genes by symbol or Ensembl id (var_names stay Ensembl ids)
+tcells = singlet.load("GSE138867", genes=["CD3E", "CD8A", "MS4A1"])
+
+# Per-cell labels come with every per-sample study (docs/cell_annotation.md section 8)
+adata.obs[["cell_type", "cell_type_ontology_term_id", "sex_chrom", "ancestry_hancestro", "called_ed"]]
+# ... and can select cells before anything else is downloaded
+heart = singlet.cells("GSE286650", "cell_type == 'cardiac muscle cell' and sex_chrom == 'male'")
+
+# Explore
+singlet.describe(adata)                     # quick summary stats
+
+# Preprocessing (no scanpy required)
+singlet.filter_cells(adata, min_genes=200, inplace=True)
+singlet.filter_genes(adata, min_cells=3, inplace=True)
+singlet.normalize(adata)                    # library-size + log1p
+singlet.highly_variable_genes(adata)        # feature selection
+
+# Analysis
+singlet.pca(adata)                          # dimensionality reduction
+singlet.harmony(adata, "dataset")           # batch-correct across the two studies
+singlet.neighbors(adata, use_rep="X_pca_harmony")
+singlet.leiden(adata)                       # clustering
+singlet.umap(adata)                         # 2D embedding
+
+# Differential expression & visualization (plotting needs matplotlib)
+singlet.rank_genes_groups(adata, "leiden")
+singlet.plot_umap(adata, color="leiden")
+
+# Export
+singlet.to_h5ad(adata, "pbmc.h5ad")         # also to_zarr, to_mtx
+```
+
+Samples whose count matrix is empty are skipped with a warning and listed in
+`adata.uns["skipped_samples"]`. Loom export is not supported from Python; use
+`.h5ad`, `.zarr` or 10x MTX.
+
+## Repository Layout
+
+```
+singlet/
+├── include/singlet/     C++ headers (libsinglet — header-only)
+│   ├── pz/              .1pz format codec
+│   ├── fq/              .1fq encoded FASTQ codec
+│   ├── pileup/          Streaming BAM pileup engine
+│   └── gpu/             CUDA analysis kernels (experimental)
+├── python/singlet/      Python package source
+│   ├── io/              Format I/O (.1pz, h5ad, zarr)
+│   ├── preprocessing/   Pipeline QC & preprocessing
+│   ├── torch/           PyTorch integration
+│   └── gpu/             GPU analysis wrappers (experimental)
+├── r/                   R package
+├── src/                 Compiled sources (pipeline binary, STAR aligner, GPU kernels)
+├── tests/               Test suites (Python + C++ + R)
+├── notebooks/           Jupyter tutorial notebooks
+├── docs/                User documentation
+└── papers/              Scientific manuscripts
+```
+
+## Installation
+
+See [docs/installation.md](docs/installation.md) for full details. The Python
+package is `pip install singlet`; the rows below install the development version from GitHub.
+
+| Install | Command |
+|---------|---------|
+| Python (core) | `pip install "singlet @ git+https://github.com/Singlet-Bio/singlet"` |
+| Python + analysis extras | `pip install "singlet[analysis] @ git+https://github.com/Singlet-Bio/singlet"` |
+| Python + PyTorch | `pip install "singlet[torch] @ git+https://github.com/Singlet-Bio/singlet"` |
+| Python + local MCP server | `pip install "singlet[mcp] @ git+https://github.com/Singlet-Bio/singlet"` |
+| R | `remotes::install_github("Singlet-Bio/singlet", subdir = "r")` |
+| C++ (CMake) | `find_package(Singlet COMPONENTS pz fq pileup)` |
+| Pipeline binary | `cmake -B build -DSINGLET_BUILD_PIPELINE=ON` |
+
+The GPU module (`singlet.gpu`) is experimental: `singlet[gpu]` only installs
+CuPy, and the CUDA extension it needs must be built from source on a CUDA 12
+machine.
+
+## MCP servers
+
+There are two, and they are different:
+
+- **Hosted:** `https://singlet.bio/mcp` — the live catalog, maintained by the
+  site. Point an MCP client at the URL; nothing to install.
+- **Local:** `singlet-mcp` (from `singlet[mcp]`) — 12 tools over the offline
+  catalog snapshot bundled with the package plus live search. The old command
+  name `singlet` still starts it but is deprecated, because `singlet` is also
+  the pipeline binary's name.
+
+## Notebooks
+
+| Notebook | Topic |
+|----------|-------|
+| [01_load_and_explore](notebooks/01_load_and_explore.ipynb) | Full analysis pipeline |
+| [quickstart](notebooks/quickstart.ipynb) | Catalog API |
+| [gene_counting](notebooks/gene_counting.ipynb) | STARsolo equivalence |
+| [1pz_format](notebooks/1pz_format.ipynb) | Format internals |
+| [doublet_detection](notebooks/doublet_detection.ipynb) | UMI-based doublet detection |
+| [rna_velocity](notebooks/rna_velocity.ipynb) | Spliced/unspliced for scVelo |
+| [cell_calling](notebooks/cell_calling.ipynb) | EmptyDrops deviance testing |
+| + more | QC, ancestry, sex, splicing, saturation, etc. |
+
+## Building from Source
+
+```bash
+# Pipeline binary (C++ — requires htslib, zstd, ncbi-vdb, OpenMP; HDF5 optional)
+source /opt/rh/gcc-toolset-13/enable  # RHEL/Rocky — need GCC 13+
+export CONDA_PREFIX=/path/to/conda/env  # must have htslib, ncbi-vdb
+export PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH
+cmake -B build -DSINGLET_BUILD_PIPELINE=ON -DCMAKE_BUILD_TYPE=Release
+cmake --build build -j$(nproc)
+# Binary: build/src/pipeline/singlet — set $SINGLET_BINARY to it (or put it on
+# $PATH as singlet-pipeline) for singlet.run_pipeline / singlet-process.
+
+# C++ tests
+cmake -B build -DSINGLET_BUILD_TESTS=ON
+cmake --build build -j$(nproc) && ctest --test-dir build -j$(nproc)
+
+# Python tests
+pip install -e ".[dev]"
+pytest tests/python/
+
+# Lint
+make lint
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE). Vendored STAR sources under
+`include/singlet/star/` keep their own licenses.