scboa 0.1.0__tar.gz

scboa-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Qiang Su
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

scboa-0.1.0/PKG-INFO

@@ -0,0 +1,250 @@
+Metadata-Version: 2.4
+Name: scboa
+Version: 0.1.0
+Summary: Integrated Two-Stage Bayesian Optimization and Final Analysis Pipeline for scRNA-seq.
+Author-email: Your Name <your.email@example.com>
+License: MIT License
+        
+        Copyright (c) 2024 Qiang Su
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/your-username/scBOA-project
+Project-URL: Bug Tracker, https://github.com/your-username/scBOA-project/issues
+Keywords: scrna-seq,bioinformatics,bayesian optimization,scanpy,celltypist,single-cell,genomics
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: scanpy>=1.9.0
+Requires-Dist: anndata
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: scikit-learn
+Requires-Dist: matplotlib
+Requires-Dist: seaborn
+Requires-Dist: scikit-optimize
+Requires-Dist: celltypist>=1.5.0
+Requires-Dist: harmonypy
+Requires-Dist: leidenalg
+Requires-Dist: python-igraph
+Requires-Dist: openpyxl
+Dynamic: license-file
+
+# In scBOA/README.md
+
+# scBOA: scRNA-seq Bayesian Optimization and Analysis
+
+**scBOA** is an integrated, two-stage computational pipeline for single-cell RNA sequencing (scRNA-seq) analysis. It automates the discovery of optimal processing parameters using Bayesian Optimization (Stage 1) and then applies these parameters to a comprehensive downstream analysis workflow (Stage 2). The pipeline also features an optional multi-level refinement process (Stage 3/4) to iteratively re-analyze and improve annotations for low-confidence cell clusters.
+
+## Key Features
+
+-   **Automated Parameter Tuning**: Uses Bayesian Optimization to find the best parameters (`n_highly_variable_genes`, `n_pcs`, `n_neighbors`, `resolution`) for clustering and cell type annotation.
+-   **Multi-Metric Objective Function**: Optimizes for a balanced score that considers annotation accuracy (CAS), marker gene specificity (MCS), and cluster separation (Silhouette score).
+-   **Single & Multi-Sample Modes**: Natively supports analysis of a single dataset or the integration of two datasets (e.g., control vs. treated) using Harmony.
+-   **Iterative Refinement**: Automatically identifies low-confidence cell clusters and re-runs the entire optimization and analysis pipeline on them to improve annotation granularity and accuracy.
+-   **Comprehensive Outputs**: Generates publication-quality plots, detailed metric reports, annotated data objects (`.h5ad`), and summary tables for easy interpretation.
+
+---
+
+## Step-by-Step Workflow
+
+### 1. Prerequisites
+
+-   Python 3.9 or newer.
+-   Access to a Linux or macOS command line.
+
+### 2. Installation
+
+It is highly recommended to install `scboa` in a dedicated virtual environment to avoid conflicts with other Python projects.
+
+```bash
+# Create a virtual environment named 'scboa-env'
+python3 -m venv scboa-env
+
+# Activate the environment
+source scboa-env/bin/activate
+
+# Now, install scBOA from PyPI
+pip install scboa
+
+# To deactivate the environment later, simply run: deactivate
+```
+
+### 3. Prepare Your Data
+
+-   **scRNA-seq Data**: Ensure your Cell Ranger output (the folder containing `barcodes.tsv.gz`, `features.tsv.gz`, and `matrix.mtx.gz`) is accessible.
+-   **CellTypist Model**: Download a pre-trained CellTypist model (`.pkl` file). You can find available models on the official [CellTypist models website](https://www.celltypist.org/models).
+
+### 4. Run the Pipeline
+
+Here is an example command for a single-sample analysis:
+
+```bash
+scboa \
+  --data_dir /path/to/your/cellranger_output/ \
+  --output_dir ./my_analysis_output/ \
+  --model_path /path/to/your/celltypist_model.pkl \
+# --- Bayesian Optimization Parameters ---
+  --n_calls 50 \
+  --target all \
+  --model_type biological \
+  --seed 42 \
+# --- Analysis & Clustering Parameters ---
+  --hvg_min_mean 0.0125 \
+  --hvg_max_mean 3.0 \
+  --hvg_min_disp 0.3 \
+  --cas_aggregation_method leiden
+```
+
+---
+
+### For Developers (Contributing)
+
+# This clones only the latest commit, making it much faster and smaller
+git clone --depth 1 https://github.com/QiangSu/scBOA.git
+cd scBOA
+
+# Install in editable mode, which also installs development dependencies
+pip install -e .
+
+## Command-Line Arguments Explained
+
+#### `Stage 1 & 2: Main I/O and Mode`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--data_dir <path>` | Path to 10x Genomics data. | **(Single-Sample Mode)** Provide the path to the directory containing `matrix.mtx.gz`, etc. |
+| `--multi_sample <path1> <path2>` | Two paths for WT and Treated 10x data. | **(Multi-Sample Mode)** Provide two paths, first for control/WT, second for treated/perturbed. This mode enables Harmony integration. |
+| `--output_dir <path>` | Path for all output files. | The main directory where all results, plots, and logs will be saved. Subdirectories for each stage will be created here. |
+| `--model_path <path>` | Path to CellTypist model (`.pkl`). | **Required.** The pre-trained model used for cell type annotation. |
+| `--output_prefix <str>` | Base prefix for Stage 1 output files. | Default: `bayesian_opt`. Used for naming optimization reports and plots. |
+
+#### `Stage 1: Optimization Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--seed <int>` | Global random seed for reproducibility. | Default: `42`. Ensures that results are identical if run with the same data and parameters. |
+| `--n_calls <int>` | Number of trials for EACH optimization strategy. | Default: `50`. The script runs three strategies (Explore, Exploit, BO-EI), so `50` means a total of 150 optimization steps. |
+| `--model_type <choice>` | Optimization objective function type. | `biological`: Balances annotation agreement (CAS) and marker specificity (MCS). <br> `structural` (default): Adds cluster separation (Silhouette Score) to the biological metrics for more robust clusters. <br> `silhouette`: Optimizes solely for the best Silhouette Score. |
+| `--marker_gene_model <choice>` | Genes to use for MCS calculation. | `all`: All genes are considered. <br> `non-mitochondrial` (default): Excludes mitochondrial genes, which often act as non-specific markers of cell stress. |
+| `--target <choice>` | Optimization target metric. | `all` (default): Runs a single, balanced optimization (equivalent to `--model_type`). <br> `weighted_cas`, `simple_cas`, `mcs`: Runs optimization targeting only that specific metric. |
+| `--cas_aggregation_method <choice>` | Method for calculating Simple CAS. | `leiden` (default): Averages the purity score of each raw Leiden cluster. Best for assessing technical cluster quality. <br> `consensus`: Merges clusters with the same final cell type label before averaging purity. Best for assessing biological group quality. |
+
+#### `Stage 1 & 2: HVG Selection Method`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--hvg_min_mean <float>` | Min mean for two-step HVG selection. | If set, activates a pre-filtering step on genes based on expression and dispersion before selecting the top `n_hvg`. |
+| `--hvg_max_mean <float>` | Max mean for two-step HVG selection. | See above. |
+| `--hvg_min_disp <float>` | Min dispersion for two-step HVG selection. | See above. |
+
+#### `Stage 1 & 2: QC & Filtering Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--min_genes <int>` | Min genes per cell. | Default: `200`. Filters out low-quality cells/empty droplets. |
+| `--max_genes <int>` | Max genes per cell. | Default: `7000`. Filters out potential doublets. |
+| `--max_pct_mt <float>` | Max mitochondrial percentage. | Default: `10.0`. Filters out stressed or dying cells. |
+| `--min_cells <int>` | Min cells per gene. | Default: `3`. Filters out genes with negligible expression. |
+
+#### `Stage 2 & Optional Refinement: Final Run Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--final_run_prefix <str>` | Prefix for Stage 2 output files. | Default: `sc_analysis_repro`. |
+| `--fig_dpi <int>` | Resolution (DPI) for saved figures. | Default: `500`. |
+| `--n_pcs_compute <int>` | Number of principal components to compute. | Default: `105`. A higher number allows for a wider search space for the optimal `n_pcs`. |
+| `--n_top_genes <int>` | Number of top marker genes to show. | Default: `5`. Affects dot plots, heatmaps, and marker gene tables. |
+| `--cellmarker_db <path>` | Path to a cell marker database (.csv). | **(Optional)** If provided, performs a "manual-style" annotation based on cluster marker genes and calculates a Marker Capture Score. |
+| `--n_degs_for_capture <int>` | DEGs per cluster for Marker Capture Score. | Default: `50`. Number of top differentially expressed genes used to match against the marker DB. |
+| `--cas_refine_threshold <float>`| CAS threshold to trigger refinement. | **(Optional)** If a cluster's CAS score is below this value (e.g., `90`), its cells are pooled for a new round of optimization and analysis. |
+| `--refinement_depth <int>` | Maximum number of refinement iterations. | Default: `1`. If refinement is triggered, this controls how many times the process can repeat on the subsequently failing cells. |
+
+---
+
+## Output Directory Structure
+
+The script generates a structured output directory. Below is an example structure and an explanation of key files.
+
+```
+<output_dir>/
+├── stage_1_bayesian_optimization/
+│   ├── bayesian_opt_structural_balanced_FINAL_annotated.h5ad
+│   ├── bayesian_opt_structural_balanced_FINAL_best_params.txt
+│   ├── bayesian_opt_structural_balanced_yield_scores_report.csv
+│   ├── bayesian_opt_structural_balanced_optimizer_convergence.png
+│   ├── bayesian_opt_structural_balanced_BO-EI_opt_result.skopt
+│   ├── ... (other plots and strategy files) ...
+│   └── refinement_depth_1/
+│       ├── ... (mirrors the structure above, but for the refined subset of cells) ...
+│
+└── stage_2_final_analysis/
+    ├── sc_analysis_repro_final_processed.h5ad
+    ├── sc_analysis_repro_final_processed_with_refinement.h5ad
+    ├── sc_analysis_repro_all_annotations.csv
+    ├── sc_analysis_repro_all_annotations_with_refinement.csv
+    ├── sc_analysis_repro_leiden_cluster_annotation_scores.csv
+    ├── sc_analysis_repro_consensus_group_annotation_scores.csv
+    ├── sc_analysis_repro_combined_cluster_annotation_scores.csv
+    ├── sc_analysis_repro_cell_type_journey_summary.csv
+    ├── sc_analysis_repro_umap_leiden.png
+    ├── sc_analysis_repro_cluster_celltypist_umap.png
+    ├── sc_analysis_repro_umap_low_confidence_greyed.png
+    ├── ... (many other plots and result files) ...
+    └── refinement_depth_1/
+        ├── sc_analysis_repro_refinement_depth_1_final_processed.h5ad
+        ├── sc_analysis_repro_refinement_depth_1_umap_cumulative_result.png
+        └── ... (mirrors Stage 2 structure for the refined subset) ...
+```
+
+### Key File Explanations
+
+#### Stage 1: `stage_1_bayesian_optimization/`
+-   `*_FINAL_best_params.txt`: A summary of the optimal parameters found and the final performance metrics. **This is the most important summary file.**
+-   `*_FINAL_annotated.h5ad`: The AnnData object processed with the best parameters, containing all final annotations from the single best run.
+-   `*_yield_scores_report.csv`: A detailed log of every trial from every optimization strategy, including parameters tested and all resulting scores (CAS, MCS, Silhouette).
+-   `*_optimizer_convergence.png`: A plot showing how the best score improved over time for each strategy.
+-   `*_opt_result.skopt`: A saved state of the optimization process, which can be reloaded.
+
+#### Stage 2: `stage_2_final_analysis/`
+-   `*_final_processed.h5ad`: The final, fully annotated AnnData object from the initial Stage 2 run. Contains UMAP coordinates, clustering, and all annotations.
+-   `*_final_processed_with_refinement.h5ad`: **(If refinement runs)** The master AnnData object with the final, combined annotations after all refinement levels are complete.
+-   `*_all_annotations_with_refinement.csv`: A cell-by-cell table of all annotations, including the final `combined_annotation` column after refinement.
+-   `*_cluster_annotation_scores.csv`: Tables detailing the Cell Annotation Score (CAS) for each Leiden cluster and each consensus cell type group.
+-   `*_combined_cluster_annotation_scores.csv`: A concatenation of all CAS reports from the initial run and all refinement levels.
+-   `*_cell_type_journey_summary.csv`: A wide-format table showing how the cell count and CAS score for each cell type change across refinement stages.
+-   `*_umap_low_confidence_greyed.png`: A UMAP plot from the initial run where cells belonging to clusters that failed the CAS threshold are colored grey.
+-   `refinement_depth_1/*_umap_cumulative_result.png`: A UMAP plot showing the state of the data *after* a refinement level, with newly-annotated cells colored and any still-failing cells shown in grey.
+
+---
+
+## License
+
+This project is licensed under the MIT License. See the `LICENSE` file for details.
+

scboa-0.1.0/README.md

@@ -0,0 +1,192 @@
+# In scBOA/README.md
+
+# scBOA: scRNA-seq Bayesian Optimization and Analysis
+
+**scBOA** is an integrated, two-stage computational pipeline for single-cell RNA sequencing (scRNA-seq) analysis. It automates the discovery of optimal processing parameters using Bayesian Optimization (Stage 1) and then applies these parameters to a comprehensive downstream analysis workflow (Stage 2). The pipeline also features an optional multi-level refinement process (Stage 3/4) to iteratively re-analyze and improve annotations for low-confidence cell clusters.
+
+## Key Features
+
+-   **Automated Parameter Tuning**: Uses Bayesian Optimization to find the best parameters (`n_highly_variable_genes`, `n_pcs`, `n_neighbors`, `resolution`) for clustering and cell type annotation.
+-   **Multi-Metric Objective Function**: Optimizes for a balanced score that considers annotation accuracy (CAS), marker gene specificity (MCS), and cluster separation (Silhouette score).
+-   **Single & Multi-Sample Modes**: Natively supports analysis of a single dataset or the integration of two datasets (e.g., control vs. treated) using Harmony.
+-   **Iterative Refinement**: Automatically identifies low-confidence cell clusters and re-runs the entire optimization and analysis pipeline on them to improve annotation granularity and accuracy.
+-   **Comprehensive Outputs**: Generates publication-quality plots, detailed metric reports, annotated data objects (`.h5ad`), and summary tables for easy interpretation.
+
+---
+
+## Step-by-Step Workflow
+
+### 1. Prerequisites
+
+-   Python 3.9 or newer.
+-   Access to a Linux or macOS command line.
+
+### 2. Installation
+
+It is highly recommended to install `scboa` in a dedicated virtual environment to avoid conflicts with other Python projects.
+
+```bash
+# Create a virtual environment named 'scboa-env'
+python3 -m venv scboa-env
+
+# Activate the environment
+source scboa-env/bin/activate
+
+# Now, install scBOA from PyPI
+pip install scboa
+
+# To deactivate the environment later, simply run: deactivate
+```
+
+### 3. Prepare Your Data
+
+-   **scRNA-seq Data**: Ensure your Cell Ranger output (the folder containing `barcodes.tsv.gz`, `features.tsv.gz`, and `matrix.mtx.gz`) is accessible.
+-   **CellTypist Model**: Download a pre-trained CellTypist model (`.pkl` file). You can find available models on the official [CellTypist models website](https://www.celltypist.org/models).
+
+### 4. Run the Pipeline
+
+Here is an example command for a single-sample analysis:
+
+```bash
+scboa \
+  --data_dir /path/to/your/cellranger_output/ \
+  --output_dir ./my_analysis_output/ \
+  --model_path /path/to/your/celltypist_model.pkl \
+# --- Bayesian Optimization Parameters ---
+  --n_calls 50 \
+  --target all \
+  --model_type biological \
+  --seed 42 \
+# --- Analysis & Clustering Parameters ---
+  --hvg_min_mean 0.0125 \
+  --hvg_max_mean 3.0 \
+  --hvg_min_disp 0.3 \
+  --cas_aggregation_method leiden
+```
+
+---
+
+### For Developers (Contributing)
+
+# This clones only the latest commit, making it much faster and smaller
+git clone --depth 1 https://github.com/QiangSu/scBOA.git
+cd scBOA
+
+# Install in editable mode, which also installs development dependencies
+pip install -e .
+
+## Command-Line Arguments Explained
+
+#### `Stage 1 & 2: Main I/O and Mode`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--data_dir <path>` | Path to 10x Genomics data. | **(Single-Sample Mode)** Provide the path to the directory containing `matrix.mtx.gz`, etc. |
+| `--multi_sample <path1> <path2>` | Two paths for WT and Treated 10x data. | **(Multi-Sample Mode)** Provide two paths, first for control/WT, second for treated/perturbed. This mode enables Harmony integration. |
+| `--output_dir <path>` | Path for all output files. | The main directory where all results, plots, and logs will be saved. Subdirectories for each stage will be created here. |
+| `--model_path <path>` | Path to CellTypist model (`.pkl`). | **Required.** The pre-trained model used for cell type annotation. |
+| `--output_prefix <str>` | Base prefix for Stage 1 output files. | Default: `bayesian_opt`. Used for naming optimization reports and plots. |
+
+#### `Stage 1: Optimization Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--seed <int>` | Global random seed for reproducibility. | Default: `42`. Ensures that results are identical if run with the same data and parameters. |
+| `--n_calls <int>` | Number of trials for EACH optimization strategy. | Default: `50`. The script runs three strategies (Explore, Exploit, BO-EI), so `50` means a total of 150 optimization steps. |
+| `--model_type <choice>` | Optimization objective function type. | `biological`: Balances annotation agreement (CAS) and marker specificity (MCS). <br> `structural` (default): Adds cluster separation (Silhouette Score) to the biological metrics for more robust clusters. <br> `silhouette`: Optimizes solely for the best Silhouette Score. |
+| `--marker_gene_model <choice>` | Genes to use for MCS calculation. | `all`: All genes are considered. <br> `non-mitochondrial` (default): Excludes mitochondrial genes, which often act as non-specific markers of cell stress. |
+| `--target <choice>` | Optimization target metric. | `all` (default): Runs a single, balanced optimization (equivalent to `--model_type`). <br> `weighted_cas`, `simple_cas`, `mcs`: Runs optimization targeting only that specific metric. |
+| `--cas_aggregation_method <choice>` | Method for calculating Simple CAS. | `leiden` (default): Averages the purity score of each raw Leiden cluster. Best for assessing technical cluster quality. <br> `consensus`: Merges clusters with the same final cell type label before averaging purity. Best for assessing biological group quality. |
+
+#### `Stage 1 & 2: HVG Selection Method`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--hvg_min_mean <float>` | Min mean for two-step HVG selection. | If set, activates a pre-filtering step on genes based on expression and dispersion before selecting the top `n_hvg`. |
+| `--hvg_max_mean <float>` | Max mean for two-step HVG selection. | See above. |
+| `--hvg_min_disp <float>` | Min dispersion for two-step HVG selection. | See above. |
+
+#### `Stage 1 & 2: QC & Filtering Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--min_genes <int>` | Min genes per cell. | Default: `200`. Filters out low-quality cells/empty droplets. |
+| `--max_genes <int>` | Max genes per cell. | Default: `7000`. Filters out potential doublets. |
+| `--max_pct_mt <float>` | Max mitochondrial percentage. | Default: `10.0`. Filters out stressed or dying cells. |
+| `--min_cells <int>` | Min cells per gene. | Default: `3`. Filters out genes with negligible expression. |
+
+#### `Stage 2 & Optional Refinement: Final Run Parameters`
+
+| Argument | Description | Explanation/Usage |
+| :--- | :--- | :--- |
+| `--final_run_prefix <str>` | Prefix for Stage 2 output files. | Default: `sc_analysis_repro`. |
+| `--fig_dpi <int>` | Resolution (DPI) for saved figures. | Default: `500`. |
+| `--n_pcs_compute <int>` | Number of principal components to compute. | Default: `105`. A higher number allows for a wider search space for the optimal `n_pcs`. |
+| `--n_top_genes <int>` | Number of top marker genes to show. | Default: `5`. Affects dot plots, heatmaps, and marker gene tables. |
+| `--cellmarker_db <path>` | Path to a cell marker database (.csv). | **(Optional)** If provided, performs a "manual-style" annotation based on cluster marker genes and calculates a Marker Capture Score. |
+| `--n_degs_for_capture <int>` | DEGs per cluster for Marker Capture Score. | Default: `50`. Number of top differentially expressed genes used to match against the marker DB. |
+| `--cas_refine_threshold <float>`| CAS threshold to trigger refinement. | **(Optional)** If a cluster's CAS score is below this value (e.g., `90`), its cells are pooled for a new round of optimization and analysis. |
+| `--refinement_depth <int>` | Maximum number of refinement iterations. | Default: `1`. If refinement is triggered, this controls how many times the process can repeat on the subsequently failing cells. |
+
+---
+
+## Output Directory Structure
+
+The script generates a structured output directory. Below is an example structure and an explanation of key files.
+
+```
+<output_dir>/
+├── stage_1_bayesian_optimization/
+│   ├── bayesian_opt_structural_balanced_FINAL_annotated.h5ad
+│   ├── bayesian_opt_structural_balanced_FINAL_best_params.txt
+│   ├── bayesian_opt_structural_balanced_yield_scores_report.csv
+│   ├── bayesian_opt_structural_balanced_optimizer_convergence.png
+│   ├── bayesian_opt_structural_balanced_BO-EI_opt_result.skopt
+│   ├── ... (other plots and strategy files) ...
+│   └── refinement_depth_1/
+│       ├── ... (mirrors the structure above, but for the refined subset of cells) ...
+│
+└── stage_2_final_analysis/
+    ├── sc_analysis_repro_final_processed.h5ad
+    ├── sc_analysis_repro_final_processed_with_refinement.h5ad
+    ├── sc_analysis_repro_all_annotations.csv
+    ├── sc_analysis_repro_all_annotations_with_refinement.csv
+    ├── sc_analysis_repro_leiden_cluster_annotation_scores.csv
+    ├── sc_analysis_repro_consensus_group_annotation_scores.csv
+    ├── sc_analysis_repro_combined_cluster_annotation_scores.csv
+    ├── sc_analysis_repro_cell_type_journey_summary.csv
+    ├── sc_analysis_repro_umap_leiden.png
+    ├── sc_analysis_repro_cluster_celltypist_umap.png
+    ├── sc_analysis_repro_umap_low_confidence_greyed.png
+    ├── ... (many other plots and result files) ...
+    └── refinement_depth_1/
+        ├── sc_analysis_repro_refinement_depth_1_final_processed.h5ad
+        ├── sc_analysis_repro_refinement_depth_1_umap_cumulative_result.png
+        └── ... (mirrors Stage 2 structure for the refined subset) ...
+```
+
+### Key File Explanations
+
+#### Stage 1: `stage_1_bayesian_optimization/`
+-   `*_FINAL_best_params.txt`: A summary of the optimal parameters found and the final performance metrics. **This is the most important summary file.**
+-   `*_FINAL_annotated.h5ad`: The AnnData object processed with the best parameters, containing all final annotations from the single best run.
+-   `*_yield_scores_report.csv`: A detailed log of every trial from every optimization strategy, including parameters tested and all resulting scores (CAS, MCS, Silhouette).
+-   `*_optimizer_convergence.png`: A plot showing how the best score improved over time for each strategy.
+-   `*_opt_result.skopt`: A saved state of the optimization process, which can be reloaded.
+
+#### Stage 2: `stage_2_final_analysis/`
+-   `*_final_processed.h5ad`: The final, fully annotated AnnData object from the initial Stage 2 run. Contains UMAP coordinates, clustering, and all annotations.
+-   `*_final_processed_with_refinement.h5ad`: **(If refinement runs)** The master AnnData object with the final, combined annotations after all refinement levels are complete.
+-   `*_all_annotations_with_refinement.csv`: A cell-by-cell table of all annotations, including the final `combined_annotation` column after refinement.
+-   `*_cluster_annotation_scores.csv`: Tables detailing the Cell Annotation Score (CAS) for each Leiden cluster and each consensus cell type group.
+-   `*_combined_cluster_annotation_scores.csv`: A concatenation of all CAS reports from the initial run and all refinement levels.
+-   `*_cell_type_journey_summary.csv`: A wide-format table showing how the cell count and CAS score for each cell type change across refinement stages.
+-   `*_umap_low_confidence_greyed.png`: A UMAP plot from the initial run where cells belonging to clusters that failed the CAS threshold are colored grey.
+-   `refinement_depth_1/*_umap_cumulative_result.png`: A UMAP plot showing the state of the data *after* a refinement level, with newly-annotated cells colored and any still-failing cells shown in grey.
+
+---
+
+## License
+
+This project is licensed under the MIT License. See the `LICENSE` file for details.
+

scboa-0.1.0/pyproject.toml

@@ -0,0 +1,65 @@
+# pyproject.toml
+
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "scboa"
+version = "0.1.0"
+authors = [
+  { name="Your Name", email="your.email@example.com" },
+]
+description = "Integrated Two-Stage Bayesian Optimization and Final Analysis Pipeline for scRNA-seq."
+
+# Long description read from the README file
+readme = "README.md"
+
+# License information
+license = { file = "LICENSE" }
+
+# Python version requirement
+requires-python = ">=3.9"
+
+# Keywords for discoverability on PyPI
+keywords = ["scrna-seq", "bioinformatics", "bayesian optimization", "scanpy", "celltypist", "single-cell", "genomics"]
+
+# Trove classifiers for PyPI
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Bio-Informatics",
+]
+
+# Core dependencies required for the tool to run
+dependencies = [
+    "scanpy>=1.9.0",     # Core analysis framework
+    "anndata",           # Data structure for scanpy
+    "pandas",            # Data manipulation
+    "numpy",             # Numerical operations
+    "scipy",             # Scientific computing
+    "scikit-learn",      # For metrics like silhouette score
+    "matplotlib",        # Plotting
+    "seaborn",           # Enhanced plotting
+    "scikit-optimize",   # For Bayesian Optimization
+    "celltypist>=1.5.0", # For cell annotation
+    "harmonypy",         # For multi-sample integration
+    "leidenalg",         # Required by scanpy for Leiden clustering
+    "python-igraph",     # Required by scanpy for Leiden clustering
+    "openpyxl",          # For writing Excel files (e.g., marker genes)
+]
+
+[project.urls]
+"Homepage" = "https://github.com/your-username/scBOA-project"
+"Bug Tracker" = "https://github.com/your-username/scBOA-project/issues"
+
+# This section creates the command-line tool.
+# It links the command `scboa` to the `run` function in your `cli.py` module.
+[project.scripts]
+scboa = "scboa.cli:run"

scboa-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

scboa-0.1.0/src/scboa/cli.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+Command-Line Interface for the scBOA Pipeline.
+
+This script handles argument parsing and serves as the entry point for the
+scBOA pipeline when it's installed as a package. It imports and calls the 
+main orchestrator function from the `pipeline` module.
+"""
+
+import argparse
+
+# This is the crucial relative import. It looks for a file named 'pipeline.py'
+# within the same package directory ('src/scboa/') and imports the 'main' function.
+from .pipeline import main
+
+def run():
+    """
+    Parses all command-line arguments and executes the main scBOA pipeline.
+    
+    This function is the target for the [project.scripts] entry point specified
+    in the pyproject.toml file.
+    """
+    parser = argparse.ArgumentParser(
+        description="Integrated Two-Stage Bayesian Optimization and Final Analysis Pipeline for scRNA-seq.",
+        formatter_class=argparse.RawTextHelpFormatter
+    )
+
+    stage1_group = parser.add_argument_group('Stage 1 & 2: Main I/O and Mode')
+    mode_group = stage1_group.add_mutually_exclusive_group(required=True)
+    mode_group.add_argument('--data_dir', type=str, help='Path to 10x Genomics data for single-sample analysis.')
+    mode_group.add_argument('--multi_sample', nargs=2, metavar=('WT_DIR', 'TREATED_DIR'), help='Two paths for WT/Control and Treated/Perturbed 10x data for multi-sample integration.')
+    stage1_group.add_argument('--output_dir', type=str, required=True, help='Path for all output files.')
+    stage1_group.add_argument('--model_path', type=str, required=True, help='Path to CellTypist model (.pkl).')
+    stage1_group.add_argument('--output_prefix', type=str, default='bayesian_opt', help='Base prefix for Stage 1 output files.')
+
+    opt_group = parser.add_argument_group('Stage 1: Optimization Parameters')
+    opt_group.add_argument('--seed', type=int, default=42, help='Global random seed for reproducibility.')
+    opt_group.add_argument('--n_calls', type=int, default=50, help='Number of trials for EACH of the three optimization strategies.')
+    opt_group.add_argument(
+        '--model_type',
+        type=str,
+        default='structural',
+        choices=['biological', 'structural', 'silhouette'],
+        help= ("'biological': balances CAS & MCS.\n"
+               "'structural' (default): adds silhouette score to balance biological concordance with cluster quality.\n"
+               "'silhouette': optimizes solely to maximize the silhouette score.")
+    )
+    opt_group.add_argument('--marker_gene_model', type=str, default='non-mitochondrial', choices=['all', 'non-mitochondrial'], help="'all': use all genes. 'non-mitochondrial' (default): exclude mitochondrial genes from MCS markers.")
+    opt_group.add_argument('--target', type=str, default='all', choices=['all', 'weighted_cas', 'simple_cas', 'mcs'], help="'all' (default): runs a single, balanced optimization. Other options optimize for that specific metric.")
+    
+    opt_group.add_argument(
+        '--cas_aggregation_method',
+        type=str,
+        default='leiden',
+        choices=['leiden', 'consensus'],
+        help=("Method for calculating Simple Mean CAS and for determining refinement candidates.\n"
+              "'leiden' (default): Averages the purity of each individual Leiden cluster.\n"
+              "'consensus': Merges Leiden clusters with the same consensus label, then averages their purity.")
+    )
+
+    hvg_group = parser.add_argument_group('Stage 1 & 2: HVG Selection Method')
+    hvg_group.add_argument('--hvg_min_mean', type=float, default=None, help='(Optional) Activates two-step HVG selection. Min mean for initial filtering.')
+    hvg_group.add_argument('--hvg_max_mean', type=float, default=None, help='(Optional) Activates two-step HVG selection. Max mean for initial filtering.')
+    hvg_group.add_argument('--hvg_min_disp', type=float, default=None, help='(Optional) Activates two-step HVG selection. Min dispersion for initial filtering.')
+
+    qc_group = parser.add_argument_group('Stage 1 & 2: QC & Filtering Parameters')
+    qc_group.add_argument('--min_genes', type=int, default=200, help='Min genes per cell.')
+    qc_group.add_argument('--max_genes', type=int, default=7000, help='Max genes per cell.')
+    qc_group.add_argument('--max_pct_mt', type=float, default=10.0, help='Max mitochondrial percentage.')
+    qc_group.add_argument('--min_cells', type=int, default=3, help='Min cells per gene.')
+
+    stage2_group = parser.add_argument_group('Stage 2 & Optional Refinement: Final Run Parameters')
+    stage2_group.add_argument('--final_run_prefix', type=str, default='sc_analysis_repro', help='Prefix for all output files in the Stage 2 subdirectory.')
+    stage2_group.add_argument('--fig_dpi', default=500, type=int, help='Resolution (DPI) for saved figures in Stage 2.')
+    stage2_group.add_argument('--n_pcs_compute', type=int, default=105, help="Number of principal components to COMPUTE in Stage 1 and 2.")
+    stage2_group.add_argument('--n_top_genes', type=int, default=5, help="Number of top marker genes to show in plots/tables in Stage 1 and 2.")
+    stage2_group.add_argument('--cellmarker_db', type=str, default=None, help="(Optional) Path to a cell marker database (.csv) for manual annotation in Stage 2.")
+    stage2_group.add_argument('--n_degs_for_capture', type=int, default=50, help="Number of top DEGs per cluster to use for the Marker Capture Score calculation in Stage 2.")
+    stage2_group.add_argument('--cas_refine_threshold', type=float, default=None, help="(Optional) CAS percentage threshold (0-100). If a cluster's CAS is below this, its cells are pooled for a second, refined optimization run.")
+    stage2_group.add_argument('--refinement_depth', type=int, default=1, help="(Optional) Maximum number of times to repeat the refinement process on failing cells. Default is 1.")
+
+    # Parse the arguments provided by the user from the command line
+    parsed_args = parser.parse_args()
+
+    # Apply any pre-processing logic to the arguments before calling the main function
+    if parsed_args.multi_sample and "harmony" not in parsed_args.output_prefix:
+        parsed_args.output_prefix += "_harmony"
+    
+    # Call the main orchestrator function from pipeline.py and pass the parsed arguments
+    main(parsed_args)