bbbc-datasets 0.1.0__tar.gz

bbbc_datasets-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Mario Koddenbrock
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

bbbc_datasets-0.1.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include requirements.txt
+include assets/header.webp

bbbc_datasets-0.1.0/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.2
+Name: bbbc_datasets
+Version: 0.1.0
+Summary: PyTorch-compatible dataset manager for BBBC datasets
+Home-page: https://github.com/mario-koddenbrock/bbbc_datasets
+Author: Mario Koddenbrock
+Author-email: koddenbrock@gmail.com
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch
+Requires-Dist: torchvision
+Requires-Dist: numpy
+Requires-Dist: tifffile
+Requires-Dist: opencv-python
+Requires-Dist: requests
+Requires-Dist: matplotlib
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# BBBC Datasets - PyTorch Dataset Manager
+
+<img src="assets/header.webp" alt="Reposetory Header Image" width="400" height="200">
+
+## 📌 Overview
+
+This package provides a **PyTorch-compatible dataset manager** for
+[Broad Bioimage Benchmark Collection (BBBC) datasets](https://bbbc.broadinstitute.org/image_sets).
+
+It allows users to:
+
+- **Download and manage BBBC datasets automatically**
+- **Load datasets as PyTorch `Dataset` objects**
+- **Handle 2D & 3D images properly**
+- **Support image augmentations via `torchvision.transforms`** (#TODO)
+
+## 🚀 Installation
+
+### **Clone the Repository**
+
+```bash
+git clone https://github.com/mario-koddenbrock/bbbc_datasets.git
+cd bbbc_datasets
+```
+
+### **Install Dependencies**
+
+```bash
+pip install -r requirements.txt
+```
+
+#### Dependencies:
+
+- `torch`, `torchvision` (Deep Learning Framework)
+- `numpy`, `tifffile`, `opencv-python` (Image Handling)
+- `requests`, `matplotlib` (Downloading & Visualization)
+
+---
+
+## 📂 Available Datasets
+
+You can list all available datasets without downloading them:
+
+```bash
+python examples/load_dataset.py
+```
+
+**Output:**
+
+```
+📂 Available BBBC Datasets:
+- BBBC003: DIC microscopy images of mouse embryos for segmentation analysis.
+- BBBC004: Synthetic fluorescence microscopy images of nuclei with varying clustering probabilities.
+- BBBC005: Simulated fluorescence microscopy images with varying focus blur.
+- BBBC006: Z-stack fluorescence microscopy images of human U2OS cells.
+- BBBC008: Dataset: Fluorescence microscopy images of human HT29 colon cancer cells.
+- BBBC010: Live/dead assay of C. elegans exposed to pathogens.
+- BBBC024: 3D Synthetic HL60 Cell Line Images with Varying Clustering Probability and SNR.
+- BBBC027: 3D Synthetic Colon Tissue Images with Varying SNR.
+- BBBC028: Polymerized Structures in Differential Interference Contrast (DIC) Microscopy.
+- BBBC029: Synthetic Differential Interference Contrast (DIC) Images.
+- BBBC032: 3D Mouse Embryo Blastocyst Cells.
+- BBBC033: 3D Mouse Trophoblast Stem Cells.
+- BBBC034: 3D Induced Pluripotent Human Stem Cells (hiPSC).
+- BBBC035: Simulated HL60 Cells from the Cell Tracking Challenge.
+- BBBC038: Kaggle 2018 Data Science Bowl - Nuclei Segmentation.
+- BBBC039: Nuclei of U2OS Cells in a Chemical Screen.
+- BBBC046: FiloData3D - Synthetic 3D Time-Lapse Imaging of A549 Cells with Filopodia.
+- BBBC050: Nuclei of Mouse Embryonic Cells.
+```
+
+---
+
+## 🎨 Visualizing Dataset Samples
+
+### **Display First Image from Each Dataset**
+
+##### Requires downloading datasets first!
+
+```bash
+python examples/load_dataset.py
+```
+
+---
+
+## 📥 Download and Load a Dataset
+
+### **Load a Dataset as a PyTorch Dataset**
+
+```python
+from bbbc_datasets.datasets.bbbc004 import BBBC004
+from torch.utils.data import DataLoader
+
+# Load BBBC004 dataset
+dataset = BBBC004()
+
+# Create a DataLoader
+dataloader = DataLoader(dataset, batch_size=4, shuffle=True)
+
+# Iterate through data
+for images, labels in dataloader:
+    print(f"Batch shape: {images.shape}, Labels: {labels.shape if labels is not None else 'None'}")
+```
+
+---
+
+## 🛠 Running Tests
+
+### **Validate Dataset URLs**
+
+```bash
+python -m unittest tests/test_dataset_urls.py
+```
+
+- **Checks if all dataset URLs are reachable** 🌍
+
+### **Verify Dataset Loading**
+
+```bash
+python -m unittest tests/test_dataset_loading.py
+```
+
+- **Ensures datasets can be loaded and accessed correctly** 🛠
+
+---
+
+## 📝 Up-next
+
+- **Thinking of what is up next**
+- **Maybe integrating other datasets as cellpose, livecell, etc.**
+
+---
+
+## 🙌 Contributing
+
+Contributions are welcome! If you find issues or want to add new features, feel free to:
+
+- **Submit a Pull Request**
+- **Report Issues** in the GitHub Issues section
+
+---
+
+## 📜 License
+
+This project is licensed under the **MIT License**.

bbbc_datasets-0.1.0/README.md

@@ -0,0 +1,145 @@
+# BBBC Datasets - PyTorch Dataset Manager
+
+<img src="assets/header.webp" alt="Reposetory Header Image" width="400" height="200">
+
+## 📌 Overview
+
+This package provides a **PyTorch-compatible dataset manager** for
+[Broad Bioimage Benchmark Collection (BBBC) datasets](https://bbbc.broadinstitute.org/image_sets).
+
+It allows users to:
+
+- **Download and manage BBBC datasets automatically**
+- **Load datasets as PyTorch `Dataset` objects**
+- **Handle 2D & 3D images properly**
+- **Support image augmentations via `torchvision.transforms`** (#TODO)
+
+## 🚀 Installation
+
+### **Clone the Repository**
+
+```bash
+git clone https://github.com/mario-koddenbrock/bbbc_datasets.git
+cd bbbc_datasets
+```
+
+### **Install Dependencies**
+
+```bash
+pip install -r requirements.txt
+```
+
+#### Dependencies:
+
+- `torch`, `torchvision` (Deep Learning Framework)
+- `numpy`, `tifffile`, `opencv-python` (Image Handling)
+- `requests`, `matplotlib` (Downloading & Visualization)
+
+---
+
+## 📂 Available Datasets
+
+You can list all available datasets without downloading them:
+
+```bash
+python examples/load_dataset.py
+```
+
+**Output:**
+
+```
+📂 Available BBBC Datasets:
+- BBBC003: DIC microscopy images of mouse embryos for segmentation analysis.
+- BBBC004: Synthetic fluorescence microscopy images of nuclei with varying clustering probabilities.
+- BBBC005: Simulated fluorescence microscopy images with varying focus blur.
+- BBBC006: Z-stack fluorescence microscopy images of human U2OS cells.
+- BBBC008: Dataset: Fluorescence microscopy images of human HT29 colon cancer cells.
+- BBBC010: Live/dead assay of C. elegans exposed to pathogens.
+- BBBC024: 3D Synthetic HL60 Cell Line Images with Varying Clustering Probability and SNR.
+- BBBC027: 3D Synthetic Colon Tissue Images with Varying SNR.
+- BBBC028: Polymerized Structures in Differential Interference Contrast (DIC) Microscopy.
+- BBBC029: Synthetic Differential Interference Contrast (DIC) Images.
+- BBBC032: 3D Mouse Embryo Blastocyst Cells.
+- BBBC033: 3D Mouse Trophoblast Stem Cells.
+- BBBC034: 3D Induced Pluripotent Human Stem Cells (hiPSC).
+- BBBC035: Simulated HL60 Cells from the Cell Tracking Challenge.
+- BBBC038: Kaggle 2018 Data Science Bowl - Nuclei Segmentation.
+- BBBC039: Nuclei of U2OS Cells in a Chemical Screen.
+- BBBC046: FiloData3D - Synthetic 3D Time-Lapse Imaging of A549 Cells with Filopodia.
+- BBBC050: Nuclei of Mouse Embryonic Cells.
+```
+
+---
+
+## 🎨 Visualizing Dataset Samples
+
+### **Display First Image from Each Dataset**
+
+##### Requires downloading datasets first!
+
+```bash
+python examples/load_dataset.py
+```
+
+---
+
+## 📥 Download and Load a Dataset
+
+### **Load a Dataset as a PyTorch Dataset**
+
+```python
+from bbbc_datasets.datasets.bbbc004 import BBBC004
+from torch.utils.data import DataLoader
+
+# Load BBBC004 dataset
+dataset = BBBC004()
+
+# Create a DataLoader
+dataloader = DataLoader(dataset, batch_size=4, shuffle=True)
+
+# Iterate through data
+for images, labels in dataloader:
+    print(f"Batch shape: {images.shape}, Labels: {labels.shape if labels is not None else 'None'}")
+```
+
+---
+
+## 🛠 Running Tests
+
+### **Validate Dataset URLs**
+
+```bash
+python -m unittest tests/test_dataset_urls.py
+```
+
+- **Checks if all dataset URLs are reachable** 🌍
+
+### **Verify Dataset Loading**
+
+```bash
+python -m unittest tests/test_dataset_loading.py
+```
+
+- **Ensures datasets can be loaded and accessed correctly** 🛠
+
+---
+
+## 📝 Up-next
+
+- **Thinking of what is up next**
+- **Maybe integrating other datasets as cellpose, livecell, etc.**
+
+---
+
+## 🙌 Contributing
+
+Contributions are welcome! If you find issues or want to add new features, feel free to:
+
+- **Submit a Pull Request**
+- **Report Issues** in the GitHub Issues section
+
+---
+
+## 📜 License
+
+This project is licensed under the **MIT License**.

bbbc_datasets-0.1.0/bbbc_datasets/config.py

@@ -0,0 +1,15 @@
+BBBC_DATASETS = {
+    "BBBC001": {
+        "image_paths": [
+            "https://bbbc.broadinstitute.org/BBBC001/images_1",
+            "https://bbbc.broadinstitute.org/BBBC001/images_2",
+        ],
+        "label_path": "https://bbbc.broadinstitute.org/BBBC001/masks",
+        "local_path": "data/BBBC001",
+    },
+    "BBBC002": {
+        "image_paths": ["https://bbbc.broadinstitute.org/BBBC002/images"],
+        "label_path": None,  # No segmentation ground truth
+        "local_path": "data/BBBC002",
+    },
+}

bbbc_datasets-0.1.0/bbbc_datasets/dataset_manager.py

@@ -0,0 +1,123 @@
+import cv2
+import numpy as np
+import tifffile as tiff
+import torch
+from torch.utils.data import Dataset
+
+from tests import DATASETS  # Import shared dataset list
+
+
+class BBBCDataset(Dataset):
+    """
+    PyTorch-compatible dataset for loading BBBC datasets with full 3D support.
+
+    - Supports both 2D and 3D images.
+    - Optionally applies transformations (PyTorch `torchvision.transforms`).
+    - Returns (image, label) pairs where available.
+
+    Args:
+        dataset_cls: The BBBC dataset class to load.
+        transform (callable, optional): Optional transform to apply to images.
+        target_transform (callable, optional): Optional transform for labels.
+    """
+
+    def __init__(self, dataset_cls, transform=None, target_transform=None):
+        self.dataset = dataset_cls()
+        self.image_paths = self.dataset.get_image_paths()
+        self.label_path = self.dataset.get_label_paths()
+
+        if not self.image_paths:
+            raise RuntimeError(f"No images found in {dataset_cls.__name__}")
+
+        self.transform = transform
+        self.target_transform = target_transform
+
+    def __len__(self):
+        return len(self.image_paths)
+
+    def __getitem__(self, idx):
+        image_path = self.image_paths[idx]
+        image = self.load_image(image_path)
+
+        label = None
+        if self.label_path:
+            label = self.load_image(self.label_path)
+
+        # Apply transforms if provided
+        if self.transform:
+            image = self.transform(image)
+        if self.target_transform and label is not None:
+            label = self.target_transform(label)
+
+        return image, label
+
+    def load_image(self, image_path):
+        """
+        Loads an image (2D or full 3D) and converts it to a PyTorch tensor.
+        """
+        if image_path.endswith((".tif", ".tiff")):
+            img = tiff.imread(image_path)  # Load 3D TIFF or 2D image
+        else:
+            img = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)  # Load 2D image
+
+        if img is None:
+            raise ValueError(f"Error: Could not read image {image_path}")
+
+        # Normalize grayscale images
+        img = (img - np.min(img)) / (np.max(img) - np.min(img) + 1e-8)
+
+        # Convert to tensor
+        if len(img.shape) == 2:  # 2D grayscale image
+            img = torch.tensor(img, dtype=torch.float32).unsqueeze(0)  # (C, H, W)
+        elif len(img.shape) == 3:  # 3D volume (Z, H, W)
+            img = torch.tensor(img, dtype=torch.float32).unsqueeze(0)  # (C, Z, H, W)
+
+        return img
+
+
+class DatasetManager:
+    """
+    Manages all BBBC datasets and provides utilities.
+
+    - Lists available datasets.
+    - Loads datasets dynamically as PyTorch Datasets.
+    - Displays dataset information.
+    """
+
+    @staticmethod
+    def list_datasets():
+        """Prints a summary of available BBBC datasets."""
+        print("\n📂 Available BBBC Datasets:")
+        for dataset_cls in DATASETS:
+            dataset = dataset_cls()
+            print(f"- {dataset_cls.__name__}: {dataset.dataset_name}")
+
+    @staticmethod
+    def get_dataset(name, transform=None, target_transform=None):
+        """
+        Loads a dataset by name.
+
+        Args:
+            name (str): The dataset class name (e.g., "BBBC003").
+            transform: Optional image transformations.
+            target_transform: Optional label transformations.
+
+        Returns:
+            BBBCDataset instance.
+        """
+        for dataset_cls in DATASETS:
+            if dataset_cls.__name__ == name:
+                return BBBCDataset(dataset_cls, transform, target_transform)
+
+        raise ValueError(
+            f"Dataset {name} not found. Use DatasetManager.list_datasets() to see available datasets."
+        )
+
+
+if __name__ == "__main__":
+    # Example Usage
+    DatasetManager.list_datasets()
+
+    # Load a dataset
+    dataset = DatasetManager.get_dataset("BBBC003")
+    print(f"\nLoaded {len(dataset)} samples from BBBC003")