gengeneeval 0.3.0__tar.gz → 0.4.1__tar.gz

{gengeneeval-0.3.0 → gengeneeval-0.4.1}/PKG-INFO

@@ -1,10 +1,10 @@
 Metadata-Version: 2.4
 Name: gengeneeval
-Version: 0.3.0
-Summary: Comprehensive evaluation of generated gene expression data. Computes metrics between real and generated datasets with support for condition matching, train/test splits, memory-efficient lazy loading, CPU parallelization, GPU acceleration, and publication-quality visualizations.
+Version: 0.4.1
+Summary: Comprehensive evaluation of generated gene expression data. Computes metrics between real and generated datasets with support for condition matching, DEG-focused evaluation, per-context analysis, train/test splits, memory-efficient lazy loading, CPU parallelization, GPU acceleration, and publication-quality visualizations.
 License: MIT
 License-File: LICENSE
-Keywords: gene expression,evaluation,metrics,single-cell,generative models,benchmarking,memory-efficient
+Keywords: gene expression,evaluation,metrics,single-cell,generative models,benchmarking,memory-efficient,DEG,perturbation
 Author: GenEval Team
 Author-email: geneval@example.com
 Requires-Python: >=3.8,<4.0
@@ -78,6 +78,8 @@ All metrics are computed **per-gene** (returning a vector) and **aggregated**:
 - ✅ Condition-based matching (perturbation, cell type, etc.)
 - ✅ Train/test split support
 - ✅ Per-gene and aggregate metrics
+- ✅ **DEG-focused evaluation** with per-context (covariate × perturbation) support
+- ✅ **Fast DEG detection** via vectorized Welch's t-test, Student's t-test, Wilcoxon
 - ✅ **Memory-efficient lazy loading** for large datasets
 - ✅ **Batched evaluation** to avoid OOM errors
 - ✅ **CPU parallelization** via joblib (multi-core speedup)
@@ -247,6 +249,126 @@ print(f"MMD: {results['mmd'].aggregate_value:.4f}")
 | `device="mps"` | 5-20x | Apple Silicon Macs |
 | Vectorized NumPy | 2-5x | Automatic fallback |
 
+### DEG-Focused Evaluation
+
+GenEval supports **Differentially Expressed Genes (DEG)-focused evaluation**, computing metrics only on biologically relevant DEGs rather than all genes. This provides more meaningful evaluation for perturbation prediction tasks.
+
+#### Key Features
+
+- **Fast DEG detection**: Vectorized Welch's t-test, Student's t-test, or Wilcoxon rank-sum
+- **DEG vs all-genes comparison**: Compute metrics on both and compare
+- **Flexible DEG selection**: Top N by significance, or threshold-based filtering
+- **Per-context evaluation**: Automatically evaluates each (covariate × perturbation) combination
+- **GPU acceleration**: DEG detection and metrics on GPU for large datasets
+- **Comprehensive reporting**: Aggregated and expanded results with visualizations
+
+#### Quick Start
+
+```python
+from geneval import evaluate_degs
+import pandas as pd
+
+# Evaluate with DEG-focused metrics (computes both DEG and all-genes by default)
+results = evaluate_degs(
+    real_data=real_adata.X,           # (n_samples, n_genes)
+    generated_data=gen_adata.X,
+    real_obs=real_adata.obs,
+    generated_obs=gen_adata.obs,
+    condition_columns=["cell_type", "perturbation"],  # Context columns
+    control_key="control",            # Value indicating control samples
+    perturbation_column="perturbation",
+    deg_method="welch",               # or "student", "wilcoxon", "logfc"
+    pval_threshold=0.05,              # Significance threshold
+    lfc_threshold=0.5,                # log2 fold change threshold
+    compute_all_genes=True,           # Also compute metrics on all genes
+    device="cuda",                    # GPU acceleration
+)
+
+# Compare DEG-only vs all-genes metrics
+print(results.comparison_summary)
+#        metric  deg_mean  all_mean  difference  ratio
+# wasserstein_1     5.34      0.69        4.65   7.74
+#           mmd     1.14      0.13        1.02   9.00
+
+# Access per-context results
+print(results.expanded_metrics)       # Has deg_* and all_* columns
+print(results.deg_summary)            # DEG counts per context
+
+# Save results with plots
+results.save("deg_evaluation/")
+```
+
+#### DEG Selection Control
+
+```python
+# Option 1: Top N most significant DEGs
+results = evaluate_degs(
+    ...,
+    n_top_degs=50,      # Use only top 50 DEGs by adjusted p-value
+)
+
+# Option 2: Stricter thresholds
+results = evaluate_degs(
+    ...,
+    pval_threshold=0.01,    # More stringent p-value
+    lfc_threshold=1.0,      # 2-fold change minimum
+)
+
+# Option 3: DEGs only (skip all-genes metrics for speed)
+results = evaluate_degs(
+    ...,
+    compute_all_genes=False,
+)
+
+# Get DEG-only or all-genes metrics separately
+deg_only = results.get_deg_only_metrics()
+all_genes = results.get_all_genes_metrics()
+```
+
+#### Per-Context Evaluation
+
+When multiple condition columns are provided (e.g., `["cell_type", "perturbation"]`), GenEval evaluates **every combination** separately:
+
+| Context | n_DEGs | deg_W1 | all_W1 | deg_MMD | all_MMD |
+|---------|--------|--------|--------|---------|---------|
+| TypeA_drug1 | 234 | 5.42 | 0.69 | 1.03 | 0.13 |
+| TypeA_drug2 | 189 | 4.38 | 0.71 | 0.92 | 0.12 |
+| TypeB_drug1 | 312 | 6.51 | 0.68 | 1.21 | 0.14 |
+
+If only `perturbation` column is provided, evaluation is done per-perturbation.
+
+#### Available DEG Methods
+
+| Method | Description | Speed |
+|--------|-------------|-------|
+| `welch` | Welch's t-test (unequal variance) | ⚡ Fast |
+| `student` | Student's t-test (equal variance) | ⚡ Fast |
+| `wilcoxon` | Wilcoxon rank-sum (non-parametric) | 🐢 Slower |
+| `logfc` | Log fold change only (no p-value) | ⚡⚡ Fastest |
+
+#### Visualization
+
+```python
+from geneval.deg import (
+    plot_deg_distributions,
+    plot_context_heatmap,
+    plot_deg_counts,
+    create_deg_report,
+)
+
+# Distribution of metrics across contexts
+plot_deg_distributions(results, save_path="dist.png")
+
+# Heatmap: context × metric
+plot_context_heatmap(results, save_path="heatmap.png")
+
+# DEG counts per context (up/down regulated)
+plot_deg_counts(results, save_path="deg_counts.png")
+
+# Generate comprehensive report
+create_deg_report(results, "report/", include_plots=True)
+```
+
 ## Expected Data Format
 
 GenEval expects AnnData (h5ad) files with:

{gengeneeval-0.3.0 → gengeneeval-0.4.1}/README.md

@@ -38,6 +38,8 @@ All metrics are computed **per-gene** (returning a vector) and **aggregated**:
 - ✅ Condition-based matching (perturbation, cell type, etc.)
 - ✅ Train/test split support
 - ✅ Per-gene and aggregate metrics
+- ✅ **DEG-focused evaluation** with per-context (covariate × perturbation) support
+- ✅ **Fast DEG detection** via vectorized Welch's t-test, Student's t-test, Wilcoxon
 - ✅ **Memory-efficient lazy loading** for large datasets
 - ✅ **Batched evaluation** to avoid OOM errors
 - ✅ **CPU parallelization** via joblib (multi-core speedup)
@@ -207,6 +209,126 @@ print(f"MMD: {results['mmd'].aggregate_value:.4f}")
 | `device="mps"` | 5-20x | Apple Silicon Macs |
 | Vectorized NumPy | 2-5x | Automatic fallback |
 
+### DEG-Focused Evaluation
+
+GenEval supports **Differentially Expressed Genes (DEG)-focused evaluation**, computing metrics only on biologically relevant DEGs rather than all genes. This provides more meaningful evaluation for perturbation prediction tasks.
+
+#### Key Features
+
+- **Fast DEG detection**: Vectorized Welch's t-test, Student's t-test, or Wilcoxon rank-sum
+- **DEG vs all-genes comparison**: Compute metrics on both and compare
+- **Flexible DEG selection**: Top N by significance, or threshold-based filtering
+- **Per-context evaluation**: Automatically evaluates each (covariate × perturbation) combination
+- **GPU acceleration**: DEG detection and metrics on GPU for large datasets
+- **Comprehensive reporting**: Aggregated and expanded results with visualizations
+
+#### Quick Start
+
+```python
+from geneval import evaluate_degs
+import pandas as pd
+
+# Evaluate with DEG-focused metrics (computes both DEG and all-genes by default)
+results = evaluate_degs(
+    real_data=real_adata.X,           # (n_samples, n_genes)
+    generated_data=gen_adata.X,
+    real_obs=real_adata.obs,
+    generated_obs=gen_adata.obs,
+    condition_columns=["cell_type", "perturbation"],  # Context columns
+    control_key="control",            # Value indicating control samples
+    perturbation_column="perturbation",
+    deg_method="welch",               # or "student", "wilcoxon", "logfc"
+    pval_threshold=0.05,              # Significance threshold
+    lfc_threshold=0.5,                # log2 fold change threshold
+    compute_all_genes=True,           # Also compute metrics on all genes
+    device="cuda",                    # GPU acceleration
+)
+
+# Compare DEG-only vs all-genes metrics
+print(results.comparison_summary)
+#        metric  deg_mean  all_mean  difference  ratio
+# wasserstein_1     5.34      0.69        4.65   7.74
+#           mmd     1.14      0.13        1.02   9.00
+
+# Access per-context results
+print(results.expanded_metrics)       # Has deg_* and all_* columns
+print(results.deg_summary)            # DEG counts per context
+
+# Save results with plots
+results.save("deg_evaluation/")
+```
+
+#### DEG Selection Control
+
+```python
+# Option 1: Top N most significant DEGs
+results = evaluate_degs(
+    ...,
+    n_top_degs=50,      # Use only top 50 DEGs by adjusted p-value
+)
+
+# Option 2: Stricter thresholds
+results = evaluate_degs(
+    ...,
+    pval_threshold=0.01,    # More stringent p-value
+    lfc_threshold=1.0,      # 2-fold change minimum
+)
+
+# Option 3: DEGs only (skip all-genes metrics for speed)
+results = evaluate_degs(
+    ...,
+    compute_all_genes=False,
+)
+
+# Get DEG-only or all-genes metrics separately
+deg_only = results.get_deg_only_metrics()
+all_genes = results.get_all_genes_metrics()
+```
+
+#### Per-Context Evaluation
+
+When multiple condition columns are provided (e.g., `["cell_type", "perturbation"]`), GenEval evaluates **every combination** separately:
+
+| Context | n_DEGs | deg_W1 | all_W1 | deg_MMD | all_MMD |
+|---------|--------|--------|--------|---------|---------|
+| TypeA_drug1 | 234 | 5.42 | 0.69 | 1.03 | 0.13 |
+| TypeA_drug2 | 189 | 4.38 | 0.71 | 0.92 | 0.12 |
+| TypeB_drug1 | 312 | 6.51 | 0.68 | 1.21 | 0.14 |
+
+If only `perturbation` column is provided, evaluation is done per-perturbation.
+
+#### Available DEG Methods
+
+| Method | Description | Speed |
+|--------|-------------|-------|
+| `welch` | Welch's t-test (unequal variance) | ⚡ Fast |
+| `student` | Student's t-test (equal variance) | ⚡ Fast |
+| `wilcoxon` | Wilcoxon rank-sum (non-parametric) | 🐢 Slower |
+| `logfc` | Log fold change only (no p-value) | ⚡⚡ Fastest |
+
+#### Visualization
+
+```python
+from geneval.deg import (
+    plot_deg_distributions,
+    plot_context_heatmap,
+    plot_deg_counts,
+    create_deg_report,
+)
+
+# Distribution of metrics across contexts
+plot_deg_distributions(results, save_path="dist.png")
+
+# Heatmap: context × metric
+plot_context_heatmap(results, save_path="heatmap.png")
+
+# DEG counts per context (up/down regulated)
+plot_deg_counts(results, save_path="deg_counts.png")
+
+# Generate comprehensive report
+create_deg_report(results, "report/", include_plots=True)
+```
+
 ## Expected Data Format
 
 GenEval expects AnnData (h5ad) files with:

{gengeneeval-0.3.0 → gengeneeval-0.4.1}/pyproject.toml

@@ -1,13 +1,13 @@
 [tool.poetry]
 name = "gengeneeval"
-version = "0.3.0"
-description = "Comprehensive evaluation of generated gene expression data. Computes metrics between real and generated datasets with support for condition matching, train/test splits, memory-efficient lazy loading, CPU parallelization, GPU acceleration, and publication-quality visualizations."
+version = "0.4.1"
+description = "Comprehensive evaluation of generated gene expression data. Computes metrics between real and generated datasets with support for condition matching, DEG-focused evaluation, per-context analysis, train/test splits, memory-efficient lazy loading, CPU parallelization, GPU acceleration, and publication-quality visualizations."
 authors = ["GenEval Team <geneval@example.com>"]
 license = "MIT"
 readme = "README.md"
 homepage = "https://github.com/AndreaRubbi/GenGeneEval"
 repository = "https://github.com/AndreaRubbi/GenGeneEval"
-keywords = ["gene expression", "evaluation", "metrics", "single-cell", "generative models", "benchmarking", "memory-efficient"]
+keywords = ["gene expression", "evaluation", "metrics", "single-cell", "generative models", "benchmarking", "memory-efficient", "DEG", "perturbation"]
 classifiers = [
     "Development Status :: 4 - Beta",
     "Intended Audience :: Science/Research",

{gengeneeval-0.3.0 → gengeneeval-0.4.1}/src/geneval/__init__.py

@@ -7,8 +7,10 @@ and generated gene expression datasets stored in AnnData (h5ad) format.
 Features:
 - Multiple distance and correlation metrics (per-gene and aggregate)
 - Condition-based matching (perturbation, cell type, etc.)
+- DEG-focused evaluation with per-context (covariate × perturbation) support
 - Train/test split support
 - Memory-efficient lazy loading for large datasets
+- CPU parallelization and GPU acceleration
 - Publication-quality visualizations
 - Command-line interface
 
@@ -21,6 +23,17 @@ Quick Start:
     ...     output_dir="output/"
     ... )
 
+DEG-Focused Evaluation:
+    >>> from geneval import evaluate_degs
+    >>> results = evaluate_degs(
+    ...     real_data, generated_data,
+    ...     real_obs, generated_obs,
+    ...     condition_columns=["cell_type", "perturbation"],
+    ...     control_key="control",
+    ...     deg_method="welch",
+    ...     device="cuda",  # GPU acceleration
+    ... )
+
 Memory-Efficient Mode (for large datasets):
     >>> from geneval import evaluate_lazy
     >>> results = evaluate_lazy(
@@ -36,7 +49,7 @@ CLI Usage:
               --conditions perturbation cell_type --output results/
 """
 
-__version__ = "0.3.0"
+__version__ = "0.4.1"
 __author__ = "GenEval Team"
 
 # Main evaluation interface
@@ -109,6 +122,22 @@ from .metrics.accelerated import (
     compute_metrics_accelerated,
 )
 
+# DEG-focused evaluation
+from .deg import (
+    DEGEvaluator,
+    DEGResult,
+    DEGEvaluationResult,
+    ContextEvaluator,
+    ContextResult,
+    compute_degs_fast,
+    compute_degs_gpu,
+    get_contexts,
+    plot_deg_distributions,
+    plot_context_heatmap,
+    create_deg_report,
+)
+from .deg.evaluator import evaluate_degs
+
 # Visualization
 from .visualization.visualizer import (
     EvaluationVisualizer,
@@ -174,6 +203,19 @@ __all__ = [
     "ParallelMetricComputer",
     "get_available_backends",
     "compute_metrics_accelerated",
+    # DEG evaluation
+    "DEGEvaluator",
+    "DEGResult",
+    "DEGEvaluationResult",
+    "ContextEvaluator",
+    "ContextResult",
+    "compute_degs_fast",
+    "compute_degs_gpu",
+    "evaluate_degs",
+    "get_contexts",
+    "plot_deg_distributions",
+    "plot_context_heatmap",
+    "create_deg_report",
     # Visualization
     "EvaluationVisualizer",
     "visualize",

gengeneeval-0.4.1/src/geneval/deg/__init__.py

@@ -0,0 +1,69 @@
+"""
+Differentially Expressed Genes (DEG) module for GenGeneEval.
+
+This module provides:
+- Fast DEG detection using vectorized statistical tests
+- Per-context evaluation (covariates × perturbations)
+- DEG-focused metrics computation
+- Integration with GPU acceleration
+
+Example usage:
+    >>> from geneval.deg import DEGEvaluator, compute_degs_fast
+    >>> degs = compute_degs_fast(control_data, perturbed_data, method="welch")
+    >>> evaluator = DEGEvaluator(loader, deg_method="welch", pval_threshold=0.05)
+    >>> results = evaluator.evaluate()
+"""
+
+from .detection import (
+    compute_degs_fast,
+    compute_degs_gpu,
+    compute_degs_auto,
+    DEGResult,
+    DEGMethod,
+)
+from .context import (
+    ContextEvaluator,
+    ContextResult,
+    get_contexts,
+    get_context_id,
+    filter_by_context,
+)
+from .evaluator import (
+    DEGEvaluator,
+    DEGEvaluationResult,
+    DEGSettings,
+    ContextMetrics,
+    evaluate_degs,
+)
+from .visualization import (
+    plot_deg_distributions,
+    plot_context_heatmap,
+    plot_deg_counts,
+    create_deg_report,
+)
+
+__all__ = [
+    # Detection
+    "compute_degs_fast",
+    "compute_degs_gpu",
+    "compute_degs_auto",
+    "DEGResult",
+    "DEGMethod",
+    # Context
+    "ContextEvaluator",
+    "ContextResult",
+    "get_contexts",
+    "get_context_id",
+    "filter_by_context",
+    # Evaluator
+    "DEGEvaluator",
+    "DEGEvaluationResult",
+    "DEGSettings",
+    "ContextMetrics",
+    "evaluate_degs",
+    # Visualization
+    "plot_deg_distributions",
+    "plot_context_heatmap",
+    "plot_deg_counts",
+    "create_deg_report",
+]