@bgicli/bgicli 2.1.1 → 2.2.0

package/data/skills/bio-imaging-mass-cytometry-cell-segmentation/SKILL.md

@@ -0,0 +1,241 @@
+---
+name: bio-imaging-mass-cytometry-cell-segmentation
+description: Cell segmentation from multiplexed tissue images. Covers deep learning (Cellpose, Mesmer) and classical approaches for nuclear and whole-cell segmentation. Use when extracting single-cell data from IMC or MIBI images after preprocessing.
+tool_type: python
+primary_tool: cellpose
+---
+
+## Version Compatibility
+
+Reference examples tested with: Cellpose 3.0+, anndata 0.10+, matplotlib 3.8+, numpy 1.26+, pandas 2.2+, scanpy 1.10+, steinbock 0.16+
+
+Before using code patterns, verify installed versions match. If versions differ:
+- Python: `pip show <package>` then `help(module.function)` to check signatures
+- CLI: `<tool> --version` then `<tool> --help` to confirm flags
+
+If code throws ImportError, AttributeError, or TypeError, introspect the installed
+package and adapt the example to match the actual API rather than retrying.
+
+# Cell Segmentation for IMC
+
+**"Segment cells from my IMC images"** → Identify individual cell boundaries in multiplexed imaging data using deep learning (Cellpose) or watershed-based approaches for single-cell extraction.
+- Python: `cellpose.models.Cellpose()` for deep learning segmentation
+- CLI: `steinbock segment` for pipeline-based segmentation
+
+## Cellpose Segmentation
+
+```python
+from cellpose import models, io
+import numpy as np
+import tifffile
+
+# Load image
+img = tifffile.imread('processed.tiff')
+
+# Extract nuclear channel (e.g., DNA1)
+nuclear_channel = img[0]  # Adjust index based on panel
+
+# Initialize Cellpose model
+model = models.Cellpose(model_type='nuclei', gpu=True)
+
+# Run segmentation
+masks, flows, styles, diams = model.eval(
+    nuclear_channel,
+    diameter=30,  # Average nucleus diameter in pixels
+    flow_threshold=0.4,
+    cellprob_threshold=0.0
+)
+
+# masks contains integer labels for each cell
+print(f'Cells segmented: {masks.max()}')
+```
+
+## Whole-Cell Segmentation with Cellpose
+
+```python
+# Use membrane marker for whole-cell
+membrane_channel = img[1]  # e.g., CD45
+
+# Combine nuclear and membrane for cyto model
+model = models.Cellpose(model_type='cyto2', gpu=True)
+
+# Create 2-channel input [membrane, nuclear]
+img_input = np.stack([membrane_channel, nuclear_channel])
+
+masks, flows, styles, diams = model.eval(
+    img_input,
+    channels=[1, 2],  # [membrane, nuclear]
+    diameter=50,
+    flow_threshold=0.4
+)
+```
+
+## Mesmer (DeepCell)
+
+```python
+from deepcell.applications import Mesmer
+
+# Initialize Mesmer
+app = Mesmer()
+
+# Prepare input: (batch, H, W, 2) - [nuclear, membrane]
+img_input = np.stack([nuclear_channel, membrane_channel], axis=-1)
+img_input = np.expand_dims(img_input, axis=0)
+
+# Segment
+predictions = app.predict(
+    img_input,
+    image_mpp=1.0,  # Microns per pixel
+    compartment='whole-cell'  # or 'nuclear'
+)
+
+masks = predictions[0, :, :, 0]
+```
+
+## steinbock Segmentation
+
+```bash
+# Using steinbock with Cellpose
+steinbock segment cellpose \
+    --img processed \
+    --model cyto2 \
+    --channelwise \
+    --nuclear-channel 0 \
+    --membrane-channel 1 \
+    -o masks
+
+# Using steinbock with DeepCell
+steinbock segment deepcell \
+    --img processed \
+    --nuclear-channel 0 \
+    --membrane-channel 1 \
+    -o masks
+```
+
+## Extract Single-Cell Data
+
+**Goal:** Convert a segmented cell mask and multi-channel image stack into a per-cell expression matrix suitable for downstream phenotyping and spatial analysis.
+
+**Approach:** Iterate over regionprops of the label mask, compute mean intensity per channel within each cell's pixels, and collect morphological features (area, centroid, eccentricity) into a structured DataFrame.
+
+```python
+from skimage import measure
+import pandas as pd
+
+def extract_single_cell_data(img, masks, channel_names):
+    '''Extract mean intensity per cell per channel'''
+
+    # Region properties
+    props = measure.regionprops(masks)
+
+    # Cell info
+    cell_data = []
+    intensities = []
+
+    for prop in props:
+        # Basic properties
+        cell_info = {
+            'cell_id': prop.label,
+            'area': prop.area,
+            'centroid_x': prop.centroid[1],
+            'centroid_y': prop.centroid[0],
+            'eccentricity': prop.eccentricity
+        }
+        cell_data.append(cell_info)
+
+        # Mean intensity per channel
+        cell_mask = masks == prop.label
+        cell_intensities = [img[c][cell_mask].mean() for c in range(len(channel_names))]
+        intensities.append(cell_intensities)
+
+    cell_df = pd.DataFrame(cell_data)
+    intensity_df = pd.DataFrame(intensities, columns=channel_names)
+
+    return cell_df, intensity_df
+
+cell_info, intensities = extract_single_cell_data(img, masks, channel_names)
+print(f'Extracted data for {len(cell_info)} cells')
+```
+
+## Quality Control
+
+```python
+import matplotlib.pyplot as plt
+
+def qc_segmentation(img, masks, nuclear_channel_idx=0):
+    '''Visualize segmentation quality'''
+
+    fig, axes = plt.subplots(1, 3, figsize=(15, 5))
+
+    # Nuclear channel
+    axes[0].imshow(img[nuclear_channel_idx], cmap='gray')
+    axes[0].set_title('Nuclear Channel')
+
+    # Segmentation masks
+    axes[1].imshow(masks, cmap='tab20')
+    axes[1].set_title(f'Segmentation ({masks.max()} cells)')
+
+    # Overlay
+    axes[2].imshow(img[nuclear_channel_idx], cmap='gray')
+    axes[2].contour(masks, colors='red', linewidths=0.5)
+    axes[2].set_title('Overlay')
+
+    for ax in axes:
+        ax.axis('off')
+
+    plt.tight_layout()
+    plt.savefig('segmentation_qc.png', dpi=150)
+    plt.close()
+
+    # Statistics
+    props = measure.regionprops(masks)
+    areas = [p.area for p in props]
+
+    print(f'Cells: {len(props)}')
+    print(f'Area: mean={np.mean(areas):.1f}, median={np.median(areas):.1f}')
+
+qc_segmentation(img, masks)
+```
+
+## Expand Nuclei to Cells
+
+```python
+from skimage.segmentation import expand_labels
+
+# If only nuclear segmentation available, expand to approximate cells
+nuclear_masks = masks  # From nuclear segmentation
+expanded_masks = expand_labels(nuclear_masks, distance=10)
+
+print(f'Expanded masks from nuclei')
+```
+
+## Save Results
+
+```python
+import tifffile
+
+# Save masks as labeled image
+tifffile.imwrite('cell_masks.tiff', masks.astype(np.uint16))
+
+# Save single-cell data
+cell_info.to_csv('cell_info.csv', index=False)
+intensities.to_csv('cell_intensities.csv', index=False)
+
+# Create combined AnnData
+import anndata as ad
+
+adata = ad.AnnData(X=intensities.values)
+adata.var_names = channel_names
+adata.obs = cell_info
+
+# Add spatial coordinates
+adata.obsm['spatial'] = cell_info[['centroid_x', 'centroid_y']].values
+
+adata.write('imc_segmented.h5ad')
+```
+
+## Related Skills
+
+- data-preprocessing - Prepare images before segmentation
+- phenotyping - Classify segmented cells
+- spatial-analysis - Analyze cell spatial relationships

package/data/skills/bio-imaging-mass-cytometry-data-preprocessing/SKILL.md

@@ -0,0 +1,279 @@
+---
+name: bio-imaging-mass-cytometry-data-preprocessing
+description: Load and preprocess imaging mass cytometry (IMC) and MIBI data. Covers MCD/TIFF handling, hot pixel removal, and image normalization. Use when starting IMC analysis from raw MCD files or preparing images for segmentation.
+tool_type: python
+primary_tool: steinbock
+---
+
+## Version Compatibility
+
+Reference examples tested with: anndata 0.10+, numpy 1.26+, pandas 2.2+, scanpy 1.10+, scipy 1.12+, steinbock 0.16+
+
+Before using code patterns, verify installed versions match. If versions differ:
+- Python: `pip show <package>` then `help(module.function)` to check signatures
+- CLI: `<tool> --version` then `<tool> --help` to confirm flags
+
+If code throws ImportError, AttributeError, or TypeError, introspect the installed
+package and adapt the example to match the actual API rather than retrying.
+
+# IMC Data Preprocessing
+
+**"Preprocess my imaging mass cytometry data"** → Load MCD files, apply hot pixel removal, channel cropping, and signal normalization to prepare multiplexed images for segmentation and analysis.
+- CLI: `steinbock preprocess` for automated IMC preprocessing pipeline
+
+## Load MCD Files with steinbock
+
+```bash
+# steinbock CLI workflow (Docker-based)
+# Convert MCD to TIFF
+steinbock preprocess imc \
+    --mcd raw/*.mcd \
+    --panel panel.csv \
+    -o img
+
+# Output: img/*.tiff (one per acquisition)
+```
+
+## Panel File Format
+
+```csv
+# panel.csv
+channel,name,keep,ilastik
+1,DNA1,1,1
+2,CD45,1,1
+3,CD3,1,0
+4,CD8,1,0
+5,CD4,1,0
+```
+
+## Python-Based Loading
+
+```python
+import readimc
+import numpy as np
+from pathlib import Path
+
+# Read MCD file
+mcd_file = Path('acquisition.mcd')
+with readimc.MCDFile(mcd_file) as mcd:
+    # List acquisitions
+    for acquisition in mcd.acquisitions:
+        print(f'Acquisition: {acquisition.id}')
+        print(f'  Channels: {len(acquisition.channel_metals)}')
+        print(f'  Size: {acquisition.width} x {acquisition.height}')
+
+    # Load specific acquisition
+    acq = mcd.acquisitions[0]
+    img = mcd.read_acquisition(acq)  # Returns (C, H, W) array
+
+    # Channel names
+    channel_names = acq.channel_names
+```
+
+## Hot Pixel Removal
+
+```python
+from scipy import ndimage
+import numpy as np
+
+def remove_hot_pixels(img, threshold=50):
+    '''Remove hot pixels using median filtering comparison'''
+    filtered = ndimage.median_filter(img, size=3)
+    diff = np.abs(img - filtered)
+    hot_pixels = diff > threshold
+
+    # Replace hot pixels with median
+    result = img.copy()
+    result[hot_pixels] = filtered[hot_pixels]
+
+    return result
+
+# Apply to each channel
+img_clean = np.stack([remove_hot_pixels(img[c]) for c in range(img.shape[0])])
+```
+
+## Spillover Correction
+
+**Goal:** Remove channel crosstalk caused by isotope impurities in IMC data so that each channel reflects only its intended metal target.
+
+**Approach:** Invert the measured spillover matrix (channels x channels) and multiply each pixel's channel vector by the inverse, clipping negative values to zero.
+
+```python
+import numpy as np
+import pandas as pd
+
+def apply_spillover_correction(img, spillover_matrix):
+    '''Apply spillover correction to IMC image
+
+    spillover_matrix: (n_channels, n_channels) DataFrame or array
+                      rows = measured, cols = emitting
+    '''
+    n_channels, height, width = img.shape
+
+    # Reshape to (pixels, channels)
+    pixels = img.reshape(n_channels, -1).T
+
+    # Invert spillover matrix
+    sm = np.array(spillover_matrix)
+    sm_inv = np.linalg.inv(sm)
+
+    # Apply correction
+    corrected = pixels @ sm_inv.T
+    corrected = np.clip(corrected, 0, None)  # No negative values
+
+    # Reshape back to image
+    return corrected.T.reshape(n_channels, height, width)
+
+# Load spillover matrix (from CATALYST or manual measurement)
+spillover = pd.read_csv('spillover_matrix.csv', index_col=0)
+img_corrected = apply_spillover_correction(img_clean, spillover)
+```
+
+## Estimate Spillover from Single-Stain Controls
+
+```python
+def estimate_spillover(single_stains, channel_names):
+    '''Estimate spillover matrix from single-stain controls'''
+    n_channels = len(channel_names)
+    spillover = np.eye(n_channels)
+
+    for i, (primary_channel, control_img) in enumerate(single_stains.items()):
+        primary_idx = channel_names.index(primary_channel)
+        primary_signal = control_img[primary_idx].flatten()
+        mask = primary_signal > np.percentile(primary_signal, 95)
+
+        for j, ch in enumerate(channel_names):
+            if i != j:
+                secondary_signal = control_img[j].flatten()[mask]
+                spillover[j, primary_idx] = np.median(secondary_signal / primary_signal[mask])
+
+    return pd.DataFrame(spillover, index=channel_names, columns=channel_names)
+```
+
+## Image Normalization
+
+```python
+def percentile_normalize(img, low=1, high=99):
+    '''Normalize to percentiles (per channel)'''
+    normalized = np.zeros_like(img, dtype=np.float32)
+
+    for c in range(img.shape[0]):
+        channel = img[c]
+        p_low = np.percentile(channel, low)
+        p_high = np.percentile(channel, high)
+
+        normalized[c] = np.clip((channel - p_low) / (p_high - p_low), 0, 1)
+
+    return normalized
+
+def arcsinh_transform(img, cofactor=5):
+    '''Arcsinh transformation (similar to flow cytometry)'''
+    return np.arcsinh(img / cofactor)
+
+# Apply transformations
+img_norm = percentile_normalize(img_clean)
+img_asinh = arcsinh_transform(img_clean)
+```
+
+## steinbock Preprocessing Pipeline
+
+```bash
+# Complete preprocessing with steinbock
+
+# 1. Extract images from MCD
+steinbock preprocess imc --mcd raw/*.mcd -o img
+
+# 2. Apply hot pixel removal
+steinbock preprocess filter --img img -o img_filtered
+
+# 3. Generate probability maps (for segmentation)
+# Requires trained Ilastik classifier
+steinbock classify ilastik \
+    --img img_filtered \
+    --ilastik-project pixel_classifier.ilp \
+    -o probabilities
+```
+
+## Visualize with napari
+
+```python
+import napari
+import tifffile
+
+# Load image
+img = tifffile.imread('acquisition.tiff')
+channel_names = ['DNA1', 'CD45', 'CD3', 'CD8', 'CD4']
+
+# Create viewer
+viewer = napari.Viewer()
+
+# Add channels
+for i, name in enumerate(channel_names):
+    viewer.add_image(img[i], name=name, colormap='gray', blending='additive')
+
+napari.run()
+```
+
+## Create AnnData Object
+
+```python
+import anndata as ad
+import pandas as pd
+
+# After segmentation, create AnnData from single-cell data
+def create_anndata(intensities, cell_info, channel_names):
+    '''Create AnnData from segmented single-cell data'''
+
+    # Intensities: cells x channels
+    adata = ad.AnnData(X=intensities)
+
+    # Channel names
+    adata.var_names = channel_names
+
+    # Cell metadata
+    adata.obs = cell_info  # DataFrame with area, centroid_x, centroid_y, etc.
+
+    return adata
+
+# Example usage
+adata = create_anndata(
+    intensities=cell_intensities,  # (n_cells, n_channels)
+    cell_info=cell_metadata,       # DataFrame
+    channel_names=channel_names
+)
+
+adata.write('imc_data.h5ad')
+```
+
+## Batch Processing
+
+```python
+from pathlib import Path
+import tifffile
+
+def process_batch(input_dir, output_dir):
+    '''Process all images in directory'''
+    input_dir = Path(input_dir)
+    output_dir = Path(output_dir)
+    output_dir.mkdir(exist_ok=True)
+
+    for img_path in input_dir.glob('*.tiff'):
+        img = tifffile.imread(img_path)
+
+        # Preprocessing
+        img = np.stack([remove_hot_pixels(img[c]) for c in range(img.shape[0])])
+        img = percentile_normalize(img)
+
+        # Save
+        output_path = output_dir / img_path.name
+        tifffile.imwrite(output_path, img.astype(np.float32))
+
+        print(f'Processed: {img_path.name}')
+
+process_batch('raw_images', 'processed_images')
+```
+
+## Related Skills
+
+- cell-segmentation - Segment preprocessed images
+- spatial-transcriptomics/spatial-data-io - Similar data loading concepts