isoview 0.1.0__tar.gz

isoview-0.1.0/.gitignore

@@ -0,0 +1,148 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+*.ipynb_checkpoints/
+*-checkpoint.ipynb
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+Pipfile.lock
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Virtual environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# PyCharm
+.idea/
+*.iml
+*.iws
+.idea/workspace.xml
+.idea/tasks.xml
+.idea/usage.statistics.xml
+.idea/dictionaries
+.idea/shelf
+
+# Sphinx documentation
+docs/_build/
+docs/_static/
+docs/_templates/
+.sphinx-cache/
+
+# Test data and outputs
+test/matlab_output/
+test/*.mat
+test/*.tif
+test/*.tiff
+test/*.klb
+test/*.zarr
+
+# OS-specific
+.DS_Store
+Thumbs.db
+
+# Temporary files
+*.log
+*.tmp
+nul
+
+# data files
+*.mat

isoview-0.1.0/PKG-INFO

@@ -0,0 +1,370 @@
+Metadata-Version: 2.4
+Name: isoview
+Version: 0.1.0
+Summary: Multi-view light sheet microscopy image processing pipeline
+Project-URL: Homepage, https://github.com/MillerBrainObservatory/isoview
+Project-URL: Repository, https://github.com/MillerBrainObservatory/isoview
+Author: Miller Brain Observatory
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Topic :: Scientific/Engineering :: Image Processing
+Requires-Python: ==3.12.9
+Requires-Dist: cellpose>=3.0.0
+Requires-Dist: dask[array]>=2024.1.0
+Requires-Dist: fsspec>=2023.12.0
+Requires-Dist: glfw>=2.8.0
+Requires-Dist: h5py>=3.10.0
+Requires-Dist: imageio>=2.31.0
+Requires-Dist: ipywidgets>=8.1.0
+Requires-Dist: jupyterlab
+Requires-Dist: jupyterlab-vim>=4.1.0
+Requires-Dist: matplotlib>=3.7.0
+Requires-Dist: mbo-fastplotlib[imgui,notebook]
+Requires-Dist: mbo-utilities
+Requires-Dist: numpy<3.0,>=1.26.0
+Requires-Dist: ome-zarr>=0.9.0
+Requires-Dist: opencv-python>=4.8.0
+Requires-Dist: pyklb>=0.3.0
+Requires-Dist: scikit-image>=0.22.0
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: scipy>=1.11.0
+Requires-Dist: tifffile>=2023.7.0
+Requires-Dist: xmltodict>=0.13.0
+Requires-Dist: zarr>=3.0.0
+Provides-Extra: dev
+Requires-Dist: mypy>=1.5.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=7.4.0; extra == 'dev'
+Requires-Dist: ruff>=0.0.280; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Isoview Pipeline: MATLAB -> Python
+
+*Conversion/Validation TODO*
+
+- [x] readKLB.m
+- [x] clusterPT_RC.m
+- [x] clusterPT with zarr benchmarks
+- [ ] clusterMF.m
+
+Note: Multiprocessing is not yet implemented, as it is in the MATLAB pipeline.
+
+---
+
+## `readKLB.m` -> `pyklb`
+
+Using our fork of [pyklb](https://github.com/MillerBrainObservatory/pyklb), on [pypi](https://pypi.org/project/pyklb/).
+
+**1. Generate MATLAB Reference**
+
+```bash
+matlab -batch \
+"run('C://Users//RBO//repos//isoview//excludes//save_matlab_klb.m')"
+```
+
+**2. Validate with Python**
+
+```bash
+uv run --no-sync python excludes/test_klb_matlab_python.py
+```
+
+| File | MATLAB Shape | Python Shape | Max Diff |
+|------|--------------|--------------|----------|
+| SPM00_TM000000_CM00_CHN00.klb | (2048, 2048, 543) | (543, 2048, 2048) -> (2048, 2048, 543) | 0.0 |
+| SPM00_TM000000_CM01_CHN00.klb | (2048, 2048, 543) | (543, 2048, 2048) -> (2048, 2048, 543) | 0.0 |
+| SPM00_TM000000_CM02_CHN01.klb | (2048, 2048, 592) | (592, 2048, 2048) -> (2048, 2048, 592) | 0.0 |
+| SPM00_TM000000_CM03_CHN01.klb | (2048, 2048, 592) | (592, 2048, 2048) -> (2048, 2048, 592) | 0.0 |
+
+**Total:** 4/4 files exactly equal (100%)
+
+- **MATLAB**: Stores volumes as `(Y, X, Z)` = `(2048, 2048, 543/592)`
+- **Python pyklb**: Returns volumes as `(Z, Y, X)` = `(543/592, 2048, 2048)`
+
+---
+
+## clusterPT_RC.m
+
+D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806.corrected\SPM00\TM000000
+
+**File structure**
+
+```
+project_root\
+  SPM00\
+    Background_0.tif         (one per camera)
+    Background_1.tif
+    Background_2.tif
+    Background_3.tif
+    TM00000\                 (one folder per timepoint)
+      ch0.xml              (rename from ch00_spec00.xml)
+      ch1.xml              (rename from ch01_spec00.xml)
+      ANG000\
+        SPC00_TM00000_ANG000_CM0_CHN00_PH0.stack
+        SPC00_TM00000_ANG000_CM1_CHN00_PH0.stack
+        SPC00_TM00000_ANG000_CM2_CHN01_PH0.stack
+        SPC00_TM00000_ANG000_CM3_CHN01_PH0.stack
+```
+
+```matlab
+inputFolder = 'D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806';
+inputType = 2;
+specimen = 0;
+
+% Run 1: Cameras 0,1 with Channel 0
+cameras = [0 1];
+channels = 0;
+
+% Run 2: Cameras 2,3 with Channel 1
+cameras = [2 3];
+channels = 1;
+```
+
+1. Background TIFFs: Place in SPM00 root (one per camera)
+2. XML files: Rename ch00_spec00.xml to ch0.xml, ch01_spec00.xml to ch1.xml
+3. XML files: Copy to each TM folder (TM00000, TM00001, etc.)
+
+### Comparison
+
+See the full notebook [here](https://github.com/MillerBrainObservatory/isoview/blob/master/clusterPT_comparison.ipynb)
+
+Most of the differences happen in the non-dense regions on the edge of the FOV:
+
+<img width="1097" height="508" alt="Image" src="https://github.com/user-attachments/assets/bf06d440-24f9-41d0-a5f7-c2f8737b7281" />
+
+---
+
+## clusterMF.m
+
+Multi-view fusion pipeline. Python implementation in `isoview/fusion.py`.
+
+**MATLAB equivalent**
+
+```matlab
+inputFolder = 'D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806.corrected_python_zarr';
+specimen = 0;
+timepoints = 0;
+cameras = [0 1 2 3];
+channels = [0 1];
+
+% Fusion parameters
+fusionType = 1;           % adaptive blending
+blendingRange = [20 4];   % [channel, camera]
+cameraPairs = [0 1; 2 3];
+flipH = true;
+flipV = false;
+```
+
+**Python equivalent**
+
+```python
+from isoview import ProcessingConfig, fuse
+
+config = ProcessingConfig(
+    input_dir=Path("D:/W2_DATA/.../mosquito-larva_20250930_165806.corrected_python_zarr"),
+    output_dir=Path("D:/W2_DATA/.../mosquito-larva_20250930_165806.corrected_python_zarr"),
+    specimen=0,
+    timepoints=[0],
+    cameras=[0, 1, 2, 3],
+    channels=[0, 1],
+    fusion_enable=True,
+    fusion_type="adaptive_blending",
+    fusion_blending_range=(20, 4),
+    fusion_camera_pairs=[(0, 1), (2, 3)],
+    fusion_flip_h=True,
+    fusion_flip_v=False,
+)
+
+fuse(config, estimate_params=True, apply_fusion=True)
+```
+
+### Feature Flags
+
+All MATLAB clusterMF features are available as config flags (disabled by default):
+
+| Feature | MATLAB | Python Flag |
+|---------|--------|-------------|
+| Temporal smoothing | rloess | `fusion_temporal_smoothing=True` |
+| Smoothing window | smoothingRange | `fusion_smoothing_window=100` |
+| Static parameters | staticFlag | `fusion_static=True` |
+| Mask fusion mode | maskFusionMode | `fusion_mask_fusion_mode=1` |
+| Mask padding | padding | `fusion_mask_padding=2` |
+| Small object removal | bwareaopen | `fusion_mask_min_object_size=1e-5` |
+| Slab processing | slabSize | `fusion_slab_size_cameras=3` |
+| Median filtering | medianFilterRange | `fusion_median_filter_range=100` |
+| Gaussian precision | preciseGauss | `fusion_precise_gauss=True` |
+| Lookup tables | generateLUT | `fusion_generate_lookup_table=True` |
+
+See [MATLAB_FEATURES.md](MATLAB_FEATURES.md) for complete documentation.
+
+### Diagnostic Output
+
+Pipeline generates diagnostic PNGs when `save_diagnostics=True` (default):
+
+```
+output_dir/
+  diagnostics/
+    TM000000/
+      cam0_1/
+        registration.png
+        intensity_correction.png
+        fusion_result.png
+        blending_weights.png
+        fused_overview.png
+```
+
+---
+
+## Zarr I/O Integration and Benchmarks
+
+**Dataset Parameters**
+
+- Data volume: (543, 2048, 2048) uint16
+- Uncompressed size: 4344.00 MB (4.242 GB)
+- Pixel spacing: [2.0, 0.406, 0.406] um
+- Chunk size: (10, 128, 128)
+
+### Quick Compression Performance
+
+Based on a single raw `.stack` from a mosquito recorded on isoview: `mosquito-larva_20250930_165806`
+
+| Method | Compression | Level | Write (s) | Read (s) | Size (MB) | Ratio | Write Speed (GB/s) | Read Speed (GB/s) |
+|--------|-------------|-------|-----------|----------|-----------|-------|-------------------|-------------------|
+| blosc-zstd-5-sharded | blosc-zstd | 5 | 42.71 | 2.55 | 1780.3 | 2.44x | 0.099 | 1.663 |
+| blosc-zstd-9-sharded | blosc-zstd | 9 | 52.96 | 2.52 | 1679.5 | 2.59x | 0.080 | 1.682 |
+| blosc-lz4-3-sharded | blosc-lz4 | 3 | 40.12 | 2.50 | 2205.0 | 1.97x | 0.106 | 1.698 |
+| blosc-zstd-5-no-shard | blosc-zstd | 5 | 44.99 | 2.25 | 1780.3 | 2.44x | 0.094 | 1.887 |
+| blosc-lz4-3-no-shard | blosc-lz4 | 3 | 44.01 | 2.50 | 2205.0 | 1.97x | 0.096 | 1.698 |
+| none-sharded | none | 0 | 45.34 | 2.28 | 4344.0 | 1.00x | 0.094 | 1.863 |
+| none-no-shard | none | 0 | 52.18 | 1.90 | 4344.0 | 1.00x | 0.081 | 2.237 |
+
+### Pipeline Compression Performance
+
+Now, instead of converting a `.stack`, we run the full `clusterPT` pipeline and compare `.tiff`, `.klb` and `.zarr`.
+
+From the KLB source code (`keller-lab-block-filetype/src/klb_imageIO.cpp`):
+
+```cpp
+int BWTblockSize = 9;  // maximum compression
+// compress the memory buffer (blocksize=9*100k, verbose=0, worklevel=30)
+int ret = BZ2_bzBuffToBuffCompress(bufferOutPtr, &sizeCompressed,
+                                    bufferIn, gcount, BWTblockSize, 0, 30);
+```
+
+**KLB uses bzip2 with level 9** (block size 9x100k = 900,000 bytes)
+
+```
+Testing 11 configurations:
+
+Baselines:
+  KLB_bzip2_9          - KLB format with bzip2, level 9 (block size 9x100k)
+  TIFF_uncompressed    - Uncompressed TIFF for size reference
+
+Zarr configurations (all with sharding: 10 frames/shard, 1 frame/chunk, full FOV):
+  Zarr_bzip2_9         - bzip2 level 9 (match KLB)
+  Zarr_zstd_6          - blosc-zstd at level 6
+  Zarr_zstd_9          - blosc-zstd at level 9
+  Zarr_lz4_6           - blosc-lz4 at level 6
+  Zarr_lz4_9           - blosc-lz4 at level 9
+  Zarr_lz4hc_6         - blosc-lz4hc at level 6
+  Zarr_lz4hc_9         - blosc-lz4hc at level 9
+  Zarr_gzip_6          - gzip at level 6
+  Zarr_gzip_9          - gzip at level 9
+```
+
+<img width="843" height="364" alt="Image" src="https://github.com/user-attachments/assets/0764e115-b731-4075-b1e9-43997a4c11d7" />
+
+## Compression Benchmark Results
+
+Tested on dataset: (543, 2048, 2048) uint16, ~4.24 GB uncompressed
+
+| Format | Compression | Level | Size (MB) | Ratio | vs Uncompressed |
+|--------|-------------|-------|-----------|-------|-----------------|
+| **KLB_bzip2_9** | bzip2 | 9 | 1107.99 | **3.92x** | **74.5% reduction** |
+| Zarr_bzip2_9 | bzip2 | 9 | 1127.56 | 3.85x | 74.0% reduction |
+| Zarr_zstd_9 | blosc-zstd | 9 | 1255.15 | 3.46x | 71.1% reduction |
+| Zarr_zstd_6 | blosc-zstd | 6 | 1297.95 | 3.35x | 70.1% reduction |
+| Zarr_lz4hc_9 | blosc-lz4hc | 9 | 1332.07 | 3.26x | 69.3% reduction |
+| Zarr_lz4hc_6 | blosc-lz4hc | 6 | 1337.21 | 3.25x | 69.2% reduction |
+| Zarr_gzip_9 | gzip | 9 | 1434.98 | 3.03x | 67.0% reduction |
+| Zarr_gzip_6 | gzip | 6 | 1457.96 | 2.98x | 66.4% reduction |
+| Zarr_lz4_9 | blosc-lz4 | 9 | 1517.57 | 2.86x | 65.1% reduction |
+| Zarr_lz4_6 | blosc-lz4 | 6 | 1519.42 | 2.86x | 65.0% reduction |
+| TIFF_uncompressed | none | - | 4344.13 | 1.00x | - |
+
+---
+
+## Module Overview
+
+### Core Pipeline
+
+| Module | Description | Key Functions |
+|--------|-------------|---------------|
+| `pipeline.py` | Main processing orchestrator | `IsoviewProcessor`, `process_dataset()` |
+| `config.py` | All configuration flags | `ProcessingConfig` dataclass |
+| `io.py` | File I/O (klb, tiff, zarr, stack) | `read_volume()`, `write_volume()`, `read_xml_metadata()` |
+| `array.py` | Lazy loader for processed data | `IsoviewArray` class |
+
+### Image Processing
+
+| Module | Description | Key Functions |
+|--------|-------------|---------------|
+| `corrections.py` | Dead pixel detection/correction | `correct_dead_pixels()`, `estimate_background()` |
+| `segmentation.py` | Foreground segmentation | `segment_foreground()`, `fuse_masks()`, `create_coordinate_masks()` |
+| `transforms.py` | Geometric transforms | `rotate_volume()`, `flip_volume()`, `crop_volume()`, `estimate_registration()`, `apply_registration()` |
+
+### Fusion Pipeline
+
+| Module | Description | Key Functions |
+|--------|-------------|---------------|
+| `fusion.py` | Multi-view fusion | `fuse()`, `blend_views()` |
+| `temporal.py` | Temporal parameter processing | `smooth_parameters_rloess()`, `apply_temporal_averaging()`, `create_lookup_table()` |
+| `masks.py` | Mask processing for fusion | `combine_masks()`, `pad_mask_to_center()`, `apply_bwareaopen()` |
+| `intensity.py` | Intensity correction/filtering | `apply_median_filter()`, `apply_gauss_filter()` |
+
+### Utilities
+
+| Module | Description | Key Functions |
+|--------|-------------|---------------|
+| `viz.py` | Diagnostic visualization | `plot_projections()`, `plot_registration_result()`, `plot_fusion_result()`, `plot_volume_overview()` |
+
+### Public API
+
+```python
+from isoview import (
+    ProcessingConfig,    # configuration dataclass
+    IsoviewProcessor,    # main processor class
+    process_dataset,     # process clusterPT pipeline
+    fuse,                # run clusterMF fusion
+    blend_views,         # blend two registered views
+    temporal,            # temporal processing submodule
+    viz,                 # visualization submodule
+)
+```
+
+### IsoviewArray
+
+Lazy loader for processed data with napari/visualization tool compatibility:
+
+```python
+from isoview.array import IsoviewArray
+
+arr = IsoviewArray("path/to/output/TM000000")
+
+# Shape: (Z, Views, Y, X) for single timepoint
+# Shape: (T, Z, Views, Y, X) for multi-timepoint
+print(arr.shape)       # (543, 4, 2048, 2048)
+print(arr.views)       # [(0, 0), (1, 0), (2, 1), (3, 1)]
+print(arr.metadata)    # microscope metadata dict
+
+# Lazy indexing
+frame = arr[100, 0]    # single Z-slice, first view
+volume = arr[:, 0]     # full Z-stack, first view
+
+# Access labels/projections (consolidated structure)
+mask = arr.get_labels(timepoint=0, camera=0, label_type='segmentation')
+proj = arr.get_projection(timepoint=0, camera=0, proj_type='xy')
+```