bsvae 0.3.0__tar.gz → 0.3.2__tar.gz

bsvae-0.3.2/PKG-INFO

@@ -0,0 +1,224 @@
+Metadata-Version: 2.4
+Name: bsvae
+Version: 0.3.2
+Summary: Gaussian Mixture VAE for multi-modal biological module discovery in omics data.
+License: GPL-3.0-only
+License-File: LICENSE
+Keywords: variational-autoencoder,gaussian-mixture-model,pytorch,bioinformatics,gene-expression,module-discovery,omics
+Author: Kynon J Benjamin
+Author-email: kj.benjamin90@gmail.com
+Requires-Python: >=3.11,<3.14
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Dist: faiss-cpu (>=1.7)
+Requires-Dist: h5py (>=3.0)
+Requires-Dist: numpy (<2.3)
+Requires-Dist: pandas (>=2.3.2,<3.0.0)
+Requires-Dist: scikit-learn (>=1.7.2,<2.0.0)
+Requires-Dist: scipy (>=1.16.2,<2.0.0)
+Requires-Dist: torch (>=2.8.0,<3.0.0)
+Requires-Dist: tqdm (>=4.67.1,<5.0.0)
+Project-URL: Bug Tracker, https://github.com/heart-gen/BSVAE/issues
+Project-URL: homepage, https://bsvae.readthedocs.io/en/latest/
+Project-URL: repository, https://github.com/heart-gen/BSVAE
+Description-Content-Type: text/markdown
+
+# BSVAE: Gaussian Mixture VAE for Module Discovery
+
+[![PyTorch](https://img.shields.io/badge/PyTorch-%3E%3D2.8-orange)](https://pytorch.org/)
+[![Python](https://img.shields.io/badge/Python-%3E%3D3.11-blue)](https://python.org/)
+[![License](https://img.shields.io/badge/license-GPLv3-blue.svg)](LICENSE)
+[![Documentation](https://readthedocs.org/projects/bsvae/badge/?version=latest)](https://bsvae.readthedocs.io/)
+
+BSVAE is a PyTorch package for feature-level module discovery in omics data. The main model, `GMMModuleVAE`, learns latent structure across features such as genes, transcripts, or proteins, then supports downstream network extraction, module assignment, latent analysis, and simulation-based benchmarking.
+
+## What The Package Provides
+
+- `bsvae-train` for training a two-phase GMM-VAE
+- `bsvae-sweep-k` for selecting the number of modules (`K`) with held-out validation and optional stability replicates
+- `bsvae-networks` for post-training network extraction, module extraction, latent export, and latent analysis
+- `bsvae-simulate` for synthetic data generation, scenario-grid simulation, and module-recovery benchmarking
+
+## Installation
+
+Install from PyPI:
+
+```bash
+pip install bsvae
+```
+
+Install from source:
+
+```bash
+git clone https://github.com/heart-gen/BSVAE.git
+cd BSVAE
+pip install -e .
+```
+
+Optional dependency:
+
+- `anndata` is only needed for `.h5ad` input or `.h5ad` latent export helpers
+
+## Input Data Contract
+
+Training and most downstream commands expect an expression matrix in `features x samples` orientation:
+
+- rows are features such as genes, transcripts, or proteins
+- columns are sample IDs
+- the first column is the feature index when using CSV/TSV
+
+Supported loaders:
+
+- `.csv` / `.csv.gz`
+- `.tsv` / `.tsv.gz`
+- `.h5` / `.hdf5`
+- `.h5ad` with optional `anndata`
+
+## CLI Entry Points
+
+- `bsvae-train`
+- `bsvae-sweep-k`
+- `bsvae-networks`
+- `bsvae-simulate`
+
+## Quickstart
+
+Train a small model:
+
+```bash
+bsvae-train pilot_run \
+  --dataset data/expression.csv \
+  --epochs 50 \
+  --n-modules 12 \
+  --latent-dim 16
+```
+
+Recommended production flow: select `K` first, then use the retrained final model.
+
+```bash
+bsvae-sweep-k sweep1 \
+  --dataset data/expression.csv \
+  --k-grid 8,12,16,24,32 \
+  --sweep-epochs 60 \
+  --stability-reps 5 \
+  --val-frac 0.1
+```
+
+This writes sweep outputs under `results/sweep1/sweep_k/` and retrains the selected model under `results/sweep1/final_k<K>/`.
+
+Extract module assignments from the final model:
+
+```bash
+bsvae-networks extract-modules \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/modules \
+  --expr data/expression.csv \
+  --soft-eigengenes
+```
+
+Export latents:
+
+```bash
+bsvae-networks export-latents \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output results/sweep1/final_k16/latents
+```
+
+This command writes `results/sweep1/final_k16/latents.npz` containing `mu`, `logvar`, `gamma`, and `feature_ids`.
+
+Extract feature-feature networks:
+
+```bash
+bsvae-networks extract-networks \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/networks \
+  --methods mu_cosine gamma_knn \
+  --top-k 50
+```
+
+Run latent-space analysis:
+
+```bash
+bsvae-networks latent-analysis \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/latent_analysis \
+  --kmeans-k 16 \
+  --umap
+```
+
+## Training Outputs
+
+`bsvae-train` writes to `results/<experiment>/` by default:
+
+- `model.pt`
+- `specs.json`
+- `train_losses.csv`
+- `model-<epoch>.pt` checkpoints when `--checkpoint-every` is greater than zero
+
+`bsvae-sweep-k` writes:
+
+- `results/<name>/sweep_k/sweep_results.csv`
+- `results/<name>/sweep_k/sweep_summary.json`
+- `results/<name>/sweep_k/k<K>/rep_<rep>/...` per-sweep run outputs
+- `results/<name>/final_k<K>/...` for the final retrained model when `--train-final` is enabled
+
+## Simulation and Benchmarking
+
+Generate one synthetic dataset:
+
+```bash
+bsvae-simulate generate \
+  --output data/sim_expr.csv \
+  --save-ground-truth data/sim_truth.csv
+```
+
+Generate a scenario grid:
+
+```bash
+bsvae-simulate init-config --output sim.yaml
+
+bsvae-simulate generate-grid \
+  --config sim.yaml \
+  --outdir results/sim_pub_v1 \
+  --reps 30 \
+  --base-seed 13
+```
+
+Validate the generated grid:
+
+```bash
+bsvae-simulate validate-grid --grid-dir results/sim_pub_v1
+```
+
+Each scenario replicate writes method-ready files under `results/sim_pub_v1/scenarios/<scenario_id>/rep_<rep>/`, including:
+
+- `expr/features_x_samples.tsv.gz`
+- `expr/samples_x_features.tsv.gz`
+- `truth/modules_hard.csv`
+- `method_inputs.json`
+
+## Documentation
+
+See the docs for the full workflow and command reference:
+
+- `docs/index.md`
+- `docs/quickstart.md`
+- `docs/tutorial.md`
+- `docs/cli.md`
+- `docs/networks.md`
+- `docs/hyperparameters.md`
+
+## License
+
+This project is licensed under the [GNU General Public License v3.0](LICENSE).
+

bsvae-0.3.2/README.md

@@ -0,0 +1,192 @@
+# BSVAE: Gaussian Mixture VAE for Module Discovery
+
+[![PyTorch](https://img.shields.io/badge/PyTorch-%3E%3D2.8-orange)](https://pytorch.org/)
+[![Python](https://img.shields.io/badge/Python-%3E%3D3.11-blue)](https://python.org/)
+[![License](https://img.shields.io/badge/license-GPLv3-blue.svg)](LICENSE)
+[![Documentation](https://readthedocs.org/projects/bsvae/badge/?version=latest)](https://bsvae.readthedocs.io/)
+
+BSVAE is a PyTorch package for feature-level module discovery in omics data. The main model, `GMMModuleVAE`, learns latent structure across features such as genes, transcripts, or proteins, then supports downstream network extraction, module assignment, latent analysis, and simulation-based benchmarking.
+
+## What The Package Provides
+
+- `bsvae-train` for training a two-phase GMM-VAE
+- `bsvae-sweep-k` for selecting the number of modules (`K`) with held-out validation and optional stability replicates
+- `bsvae-networks` for post-training network extraction, module extraction, latent export, and latent analysis
+- `bsvae-simulate` for synthetic data generation, scenario-grid simulation, and module-recovery benchmarking
+
+## Installation
+
+Install from PyPI:
+
+```bash
+pip install bsvae
+```
+
+Install from source:
+
+```bash
+git clone https://github.com/heart-gen/BSVAE.git
+cd BSVAE
+pip install -e .
+```
+
+Optional dependency:
+
+- `anndata` is only needed for `.h5ad` input or `.h5ad` latent export helpers
+
+## Input Data Contract
+
+Training and most downstream commands expect an expression matrix in `features x samples` orientation:
+
+- rows are features such as genes, transcripts, or proteins
+- columns are sample IDs
+- the first column is the feature index when using CSV/TSV
+
+Supported loaders:
+
+- `.csv` / `.csv.gz`
+- `.tsv` / `.tsv.gz`
+- `.h5` / `.hdf5`
+- `.h5ad` with optional `anndata`
+
+## CLI Entry Points
+
+- `bsvae-train`
+- `bsvae-sweep-k`
+- `bsvae-networks`
+- `bsvae-simulate`
+
+## Quickstart
+
+Train a small model:
+
+```bash
+bsvae-train pilot_run \
+  --dataset data/expression.csv \
+  --epochs 50 \
+  --n-modules 12 \
+  --latent-dim 16
+```
+
+Recommended production flow: select `K` first, then use the retrained final model.
+
+```bash
+bsvae-sweep-k sweep1 \
+  --dataset data/expression.csv \
+  --k-grid 8,12,16,24,32 \
+  --sweep-epochs 60 \
+  --stability-reps 5 \
+  --val-frac 0.1
+```
+
+This writes sweep outputs under `results/sweep1/sweep_k/` and retrains the selected model under `results/sweep1/final_k<K>/`.
+
+Extract module assignments from the final model:
+
+```bash
+bsvae-networks extract-modules \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/modules \
+  --expr data/expression.csv \
+  --soft-eigengenes
+```
+
+Export latents:
+
+```bash
+bsvae-networks export-latents \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output results/sweep1/final_k16/latents
+```
+
+This command writes `results/sweep1/final_k16/latents.npz` containing `mu`, `logvar`, `gamma`, and `feature_ids`.
+
+Extract feature-feature networks:
+
+```bash
+bsvae-networks extract-networks \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/networks \
+  --methods mu_cosine gamma_knn \
+  --top-k 50
+```
+
+Run latent-space analysis:
+
+```bash
+bsvae-networks latent-analysis \
+  --model-path results/sweep1/final_k16 \
+  --dataset data/expression.csv \
+  --output-dir results/sweep1/final_k16/latent_analysis \
+  --kmeans-k 16 \
+  --umap
+```
+
+## Training Outputs
+
+`bsvae-train` writes to `results/<experiment>/` by default:
+
+- `model.pt`
+- `specs.json`
+- `train_losses.csv`
+- `model-<epoch>.pt` checkpoints when `--checkpoint-every` is greater than zero
+
+`bsvae-sweep-k` writes:
+
+- `results/<name>/sweep_k/sweep_results.csv`
+- `results/<name>/sweep_k/sweep_summary.json`
+- `results/<name>/sweep_k/k<K>/rep_<rep>/...` per-sweep run outputs
+- `results/<name>/final_k<K>/...` for the final retrained model when `--train-final` is enabled
+
+## Simulation and Benchmarking
+
+Generate one synthetic dataset:
+
+```bash
+bsvae-simulate generate \
+  --output data/sim_expr.csv \
+  --save-ground-truth data/sim_truth.csv
+```
+
+Generate a scenario grid:
+
+```bash
+bsvae-simulate init-config --output sim.yaml
+
+bsvae-simulate generate-grid \
+  --config sim.yaml \
+  --outdir results/sim_pub_v1 \
+  --reps 30 \
+  --base-seed 13
+```
+
+Validate the generated grid:
+
+```bash
+bsvae-simulate validate-grid --grid-dir results/sim_pub_v1
+```
+
+Each scenario replicate writes method-ready files under `results/sim_pub_v1/scenarios/<scenario_id>/rep_<rep>/`, including:
+
+- `expr/features_x_samples.tsv.gz`
+- `expr/samples_x_features.tsv.gz`
+- `truth/modules_hard.csv`
+- `method_inputs.json`
+
+## Documentation
+
+See the docs for the full workflow and command reference:
+
+- `docs/index.md`
+- `docs/quickstart.md`
+- `docs/tutorial.md`
+- `docs/cli.md`
+- `docs/networks.md`
+- `docs/hyperparameters.md`
+
+## License
+
+This project is licensed under the [GNU General Public License v3.0](LICENSE).

{bsvae-0.3.0 → bsvae-0.3.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "bsvae"
-version = "0.3.0"  # dynamically resolved by poetry-dynamic-versioning
+version = "0.3.2"  # dynamically resolved by poetry-dynamic-versioning
 description = "Gaussian Mixture VAE for multi-modal biological module discovery in omics data."
 readme = "README.md"
 authors = ["Kynon J Benjamin <kj.benjamin90@gmail.com>"]
@@ -50,6 +50,7 @@ repository = "https://github.com/heart-gen/BSVAE"
 bsvae-train = "bsvae.cli.train:cli"
 bsvae-networks = "bsvae.cli.networks:cli"
 bsvae-simulate = "bsvae.cli.simulate:cli"
+bsvae-sweep-k = "bsvae.cli.sweep_k:cli"
 
 [tool.poetry.group.dev.dependencies]
 pytest = ">=8.4.2,<9.0.0"

{bsvae-0.3.0 → bsvae-0.3.2}/src/bsvae/cli/networks.py

@@ -64,6 +64,12 @@ def build_parser() -> argparse.ArgumentParser:
     mp.add_argument("--use-leiden", action="store_true",
                     help="Also run Leiden clustering as fallback comparison.")
     mp.add_argument("--leiden-resolution", type=float, default=1.0)
+    mp.add_argument("--tx2gene", type=str, default=None,
+                    help="TSV (transcript_id, gene_id) for isoform-to-gene aggregation.")
+    mp.add_argument("--aggregate-to-gene", action="store_true", default=False,
+                    help="Aggregate isoform-level γ to gene level by averaging across "
+                         "isoforms per gene. Saves gamma_gene.npz and "
+                         "hard_assignments_gene.npz. Requires --tx2gene.")
 
     # ------------------------------------------------------------------
     # export-latents
@@ -120,6 +126,35 @@ def handle_extract_networks(args, logger: logging.Logger) -> None:
     logger.info("Network extraction complete → %s", args.output_dir)
 
 
+def _aggregate_gamma_to_gene(
+    gamma: "np.ndarray",
+    feature_ids: List[str],
+    tx2gene_path: str,
+) -> "tuple[np.ndarray, list[str]]":
+    """Average isoform-level γ (F_tx, K) to gene level, renormalise rows to sum 1."""
+    import numpy as np
+    from collections import defaultdict
+    from bsvae.utils.hierarchy import load_tx2gene
+
+    tx2gene = load_tx2gene(path=tx2gene_path, feature_ids=feature_ids)
+    # Map every feature to its gene_id; fall back to itself for unmapped features
+    fid_to_gene = {fid: tx2gene.get(fid, fid) for fid in feature_ids}
+
+    gene_to_idxs: dict = defaultdict(list)
+    for i, fid in enumerate(feature_ids):
+        gene_to_idxs[fid_to_gene[fid]].append(i)
+
+    gene_ids = list(gene_to_idxs.keys())
+    gamma_gene = np.zeros((len(gene_ids), gamma.shape[1]), dtype=np.float32)
+    for j, gid in enumerate(gene_ids):
+        gamma_gene[j] = gamma[gene_to_idxs[gid]].mean(axis=0)
+
+    row_sums = gamma_gene.sum(axis=1, keepdims=True)
+    row_sums = np.where(row_sums > 0, row_sums, 1.0)
+    gamma_gene /= row_sums
+    return gamma_gene, gene_ids
+
+
 def handle_extract_modules(args, logger: logging.Logger) -> None:
     import numpy as np
     from bsvae.utils.modelIO import load_model
@@ -144,6 +179,30 @@ def handle_extract_modules(args, logger: logging.Logger) -> None:
         model.n_modules, len(feature_ids),
     )
 
+    if args.aggregate_to_gene:
+        if not args.tx2gene:
+            logger.warning("--aggregate-to-gene requires --tx2gene; skipping.")
+        else:
+            gamma_gene, gene_ids = _aggregate_gamma_to_gene(
+                result["gamma"], result["feature_ids"], args.tx2gene
+            )
+            hard_gene = gamma_gene.argmax(axis=1).astype(np.int32)
+            out = Path(args.output_dir)
+            np.savez_compressed(
+                out / "gamma_gene.npz",
+                gamma=gamma_gene,
+                feature_ids=np.array(gene_ids, dtype=object),
+            )
+            np.savez_compressed(
+                out / "hard_assignments_gene.npz",
+                hard_assignments=hard_gene,
+                feature_ids=np.array(gene_ids, dtype=object),
+            )
+            logger.info(
+                "Gene-level aggregation: %d transcripts → %d genes",
+                len(result["feature_ids"]), len(gene_ids),
+            )
+
     if args.soft_eigengenes and args.expr:
         import pandas as pd
         expr = pd.read_csv(args.expr, index_col=0, sep="\t" if args.expr.endswith(".tsv") else ",")

{bsvae-0.3.0 → bsvae-0.3.2}/src/bsvae/cli/simulate.py

@@ -3,8 +3,12 @@ bsvae-simulate — Generate synthetic omics data and benchmark module methods.
 
 Subcommands
 -----------
-  generate     Simulate omics data with known block-diagonal module structure.
-  benchmark    ARI/NMI/enrichment vs WGCNA/GNVAE on simulated data.
+  generate           Simulate omics data with known block-diagonal module structure.
+  benchmark          ARI/NMI/enrichment vs WGCNA/GNVAE on simulated data.
+  init-config        Write starter simulation grid config.
+  generate-grid      Generate all scenarios/replicates defined in config.
+  generate-scenario  Generate one scenario/replicate.
+  validate-grid      Validate generated grid output structure.
 """
 
 from __future__ import annotations
@@ -43,7 +47,7 @@ def build_parser() -> argparse.ArgumentParser:
     gp.add_argument("--between-corr", type=float, default=0.0,
                     help="Between-module correlation (default: 0.0).")
     gp.add_argument("--noise-std", type=float, default=0.2)
-    gp.add_argument("--seed", type=int, default=42)
+    gp.add_argument("--seed", type=int, default=13)
     gp.add_argument("--save-ground-truth", help="Path to save ground-truth module labels CSV.")
 
     # ------------------------------------------------------------------
@@ -57,6 +61,37 @@ def build_parser() -> argparse.ArgumentParser:
     bp.add_argument("--batch-size", type=int, default=128)
     bp.add_argument("--no-cuda", action="store_true", default=False)
 
+    # ------------------------------------------------------------------
+    # init-config
+    # ------------------------------------------------------------------
+    cp = subs.add_parser("init-config", help="Write a starter simulation config file.")
+    cp.add_argument("--output", default="sim.yaml", help="Destination config path.")
+
+    # ------------------------------------------------------------------
+    # generate-grid
+    # ------------------------------------------------------------------
+    gp2 = subs.add_parser("generate-grid", help="Generate a scenario grid from config.")
+    gp2.add_argument("--config", required=True, help="Simulation config path.")
+    gp2.add_argument("--outdir", required=True, help="Output directory for scenario grid.")
+    gp2.add_argument("--reps", type=int, default=None, help="Override replicates per scenario.")
+    gp2.add_argument("--base-seed", type=int, default=None, help="Override base seed for all runs.")
+
+    # ------------------------------------------------------------------
+    # generate-scenario
+    # ------------------------------------------------------------------
+    sp = subs.add_parser("generate-scenario", help="Generate a single scenario/replicate from config.")
+    sp.add_argument("--config", required=True, help="Simulation config path.")
+    sp.add_argument("--scenario-id", required=True, help="Scenario identifier from config expansion.")
+    sp.add_argument("--rep", type=int, default=0, help="Replicate index.")
+    sp.add_argument("--outdir", required=True, help="Output directory for scenario grid.")
+    sp.add_argument("--base-seed", type=int, default=None, help="Override base seed for this run.")
+
+    # ------------------------------------------------------------------
+    # validate-grid
+    # ------------------------------------------------------------------
+    vp = subs.add_parser("validate-grid", help="Validate scenario grid output files.")
+    vp.add_argument("--grid-dir", required=True, help="Scenario grid output directory.")
+
     return parser
 
 
@@ -144,6 +179,58 @@ def handle_benchmark(args, logger: logging.Logger) -> None:
     logger.info("Benchmark metrics saved to %s", out)
 
 
+def handle_init_config(args, logger: logging.Logger) -> None:
+    from bsvae.simulation.scenario import write_starter_config
+
+    write_starter_config(args.output)
+    logger.info("Starter simulation config written to %s", args.output)
+
+
+def handle_generate_grid(args, logger: logging.Logger) -> None:
+    from bsvae.simulation.scenario import load_config, generate_grid
+
+    config = load_config(args.config)
+    summary = generate_grid(
+        config=config,
+        outdir=args.outdir,
+        reps=args.reps,
+        base_seed=args.base_seed,
+    )
+    logger.info(
+        "Grid complete: %d scenarios × %d reps = %d runs",
+        summary["n_scenarios"],
+        summary["n_reps"],
+        summary["n_runs"],
+    )
+    logger.info("Manifest: %s", summary["manifest"])
+
+
+def handle_generate_scenario(args, logger: logging.Logger) -> None:
+    from bsvae.simulation.scenario import load_config, generate_scenario
+
+    config = load_config(args.config)
+    run_dir = generate_scenario(
+        config=config,
+        scenario_id=args.scenario_id,
+        rep=args.rep,
+        outdir=args.outdir,
+        base_seed=args.base_seed,
+    )
+    logger.info("Scenario generated: %s", run_dir)
+
+
+def handle_validate_grid(args, logger: logging.Logger) -> None:
+    from bsvae.simulation.scenario import validate_grid
+
+    result = validate_grid(args.grid_dir)
+    logger.info(
+        "Grid validation: checked %d/%d rows, all required files present=%s",
+        result["checked_rows"],
+        result["manifest_rows"],
+        result["all_required_files_present"],
+    )
+
+
 def cli(argv=None):
     parser = build_parser()
     args = parser.parse_args(argv)
@@ -153,6 +240,14 @@ def cli(argv=None):
         handle_generate(args, logger)
     elif args.command == "benchmark":
         handle_benchmark(args, logger)
+    elif args.command == "init-config":
+        handle_init_config(args, logger)
+    elif args.command == "generate-grid":
+        handle_generate_grid(args, logger)
+    elif args.command == "generate-scenario":
+        handle_generate_scenario(args, logger)
+    elif args.command == "validate-grid":
+        handle_validate_grid(args, logger)
     else:
         parser.error(f"Unknown command: {args.command}")