Rhapso 0.1.92__tar.gz → 0.1.94__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (109) hide show
  1. rhapso-0.1.94/PKG-INFO +410 -0
  2. rhapso-0.1.94/Rhapso.egg-info/PKG-INFO +410 -0
  3. {rhapso-0.1.92 → rhapso-0.1.94}/setup.py +18 -18
  4. rhapso-0.1.92/PKG-INFO +0 -39
  5. rhapso-0.1.92/Rhapso.egg-info/PKG-INFO +0 -39
  6. {rhapso-0.1.92 → rhapso-0.1.94}/LICENSE +0 -0
  7. {rhapso-0.1.92 → rhapso-0.1.94}/README.md +0 -0
  8. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/__init__.py +0 -0
  9. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/data_prep/__init__.py +0 -0
  10. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/data_prep/n5_reader.py +0 -0
  11. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/data_prep/s3_big_stitcher_reader.py +0 -0
  12. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/data_prep/xml_to_dataframe.py +0 -0
  13. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/__init__.py +0 -0
  14. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/advanced_refinement.py +0 -0
  15. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/difference_of_gaussian.py +0 -0
  16. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/image_reader.py +0 -0
  17. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/metadata_builder.py +0 -0
  18. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/overlap_detection.py +0 -0
  19. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/points_validation.py +0 -0
  20. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/save_interest_points.py +0 -0
  21. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/detection/view_transform_models.py +0 -0
  22. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/__init__.py +0 -0
  23. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/__init__.py +0 -0
  24. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/blend.py +0 -0
  25. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/fusion.py +0 -0
  26. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/geometry.py +0 -0
  27. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/io.py +0 -0
  28. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/script_utils.py +0 -0
  29. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion/setup.py +0 -0
  30. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/affine_fusion_worker.py +0 -0
  31. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/__init__.py +0 -0
  32. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/__init__.py +0 -0
  33. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/compress/__init__.py +0 -0
  34. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/compress/czi_to_zarr.py +0 -0
  35. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/compress/zarr_writer.py +0 -0
  36. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/models.py +0 -0
  37. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/utils/__init__.py +0 -0
  38. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/utils/utils.py +0 -0
  39. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_hcr_data_transformation/zeiss_job.py +0 -0
  40. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/__init__.py +0 -0
  41. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/array_to_zarr.py +0 -0
  42. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/radial_correction.py +0 -0
  43. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/run_capsule.py +0 -0
  44. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/utils/__init__.py +0 -0
  45. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/utils/utils.py +0 -0
  46. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale/aind_z1_radial_correction/worker.py +0 -0
  47. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/multiscale_worker.py +0 -0
  48. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/__init__.py +0 -0
  49. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/dispim_link.py +0 -0
  50. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/exaspim_link.py +0 -0
  51. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/hcr_link.py +0 -0
  52. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/iSPIM_top.py +0 -0
  53. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/link_utils.py +0 -0
  54. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/main.py +0 -0
  55. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/ng_layer.py +0 -0
  56. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/ng_state.py +0 -0
  57. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/parsers.py +0 -0
  58. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/raw_link.py +0 -0
  59. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/utils/__init__.py +0 -0
  60. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/utils/shader_utils.py +0 -0
  61. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/utils/transfer.py +0 -0
  62. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen/utils/utils.py +0 -0
  63. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/fusion/neuroglancer_link_gen_worker.py +0 -0
  64. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/matching/__init__.py +0 -0
  65. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/matching/load_and_transform_points.py +0 -0
  66. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/matching/ransac_matching.py +0 -0
  67. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/matching/save_matches.py +0 -0
  68. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/matching/xml_parser.py +0 -0
  69. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/__init__.py +0 -0
  70. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/__init__.py +0 -0
  71. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/aws/__init__.py +0 -0
  72. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/aws/alignment_pipeline.py +0 -0
  73. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/aws/config/__init__.py +0 -0
  74. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/evaluation.py +0 -0
  75. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/interest_point_detection.py +0 -0
  76. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/interest_point_matching.py +0 -0
  77. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/local/__init__.py +0 -0
  78. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/local/alignment_pipeline.py +0 -0
  79. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/matching_stats.py +0 -0
  80. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/param/__init__.py +0 -0
  81. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/solver.py +0 -0
  82. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/pipelines/ray/split_dataset.py +0 -0
  83. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/__init__.py +0 -0
  84. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/compute_tiles.py +0 -0
  85. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/concatenate_models.py +0 -0
  86. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/connected_graphs.py +0 -0
  87. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/data_prep.py +0 -0
  88. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/global_optimization.py +0 -0
  89. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/model_and_tile_setup.py +0 -0
  90. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/pre_align_tiles.py +0 -0
  91. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/save_results.py +0 -0
  92. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/view_transforms.py +0 -0
  93. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/solver/xml_to_dataframe_solver.py +0 -0
  94. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/__init__.py +0 -0
  95. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/compute_grid_rules.py +0 -0
  96. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/save_points.py +0 -0
  97. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/save_xml.py +0 -0
  98. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/split_images.py +0 -0
  99. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso/split_dataset/xml_to_dataframe_split.py +0 -0
  100. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso.egg-info/SOURCES.txt +0 -0
  101. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso.egg-info/dependency_links.txt +0 -0
  102. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso.egg-info/requires.txt +0 -0
  103. {rhapso-0.1.92 → rhapso-0.1.94}/Rhapso.egg-info/top_level.txt +0 -0
  104. {rhapso-0.1.92 → rhapso-0.1.94}/pyproject.toml +0 -0
  105. {rhapso-0.1.92 → rhapso-0.1.94}/setup.cfg +0 -0
  106. {rhapso-0.1.92 → rhapso-0.1.94}/tests/__init__.py +0 -0
  107. {rhapso-0.1.92 → rhapso-0.1.94}/tests/test_detection.py +0 -0
  108. {rhapso-0.1.92 → rhapso-0.1.94}/tests/test_matching.py +0 -0
  109. {rhapso-0.1.92 → rhapso-0.1.94}/tests/test_solving.py +0 -0
rhapso-0.1.94/PKG-INFO ADDED
@@ -0,0 +1,410 @@
1
+ Metadata-Version: 2.4
2
+ Name: Rhapso
3
+ Version: 0.1.94
4
+ Summary: A python package for aligning and stitching light sheet fluorescence microscopy images together
5
+ Home-page: https://github.com/AllenNeuralDynamics/Rhapso
6
+ Author: ND
7
+ Author-email: sean.fite@alleninstitute.org
8
+ Project-URL: Source, https://github.com/AllenNeuralDynamics/Rhapso
9
+ Project-URL: Bug Tracker, https://github.com/AllenNeuralDynamics/Rhapso/issues
10
+ Project-URL: Changelog, https://github.com/AllenNeuralDynamics/Rhapso/releases
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Natural Language :: English
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.7
16
+ Classifier: Programming Language :: Python :: 3.8
17
+ Classifier: Programming Language :: Python :: 3.9
18
+ Classifier: Programming Language :: Python :: 3.10
19
+ Classifier: Operating System :: OS Independent
20
+ Requires-Python: >=3.7
21
+ Description-Content-Type: text/markdown
22
+ License-File: LICENSE
23
+ Requires-Dist: pandas
24
+ Requires-Dist: dask[array]==2024.12.1
25
+ Requires-Dist: zarr==2.18.3
26
+ Requires-Dist: scipy==1.13.1
27
+ Requires-Dist: scikit-image
28
+ Requires-Dist: bioio==1.3.0
29
+ Requires-Dist: bioio-tifffile==1.0.0
30
+ Requires-Dist: tifffile==2025.1.10
31
+ Requires-Dist: dask-image==2024.5.3
32
+ Requires-Dist: boto3==1.35.92
33
+ Requires-Dist: numcodecs==0.13.1
34
+ Requires-Dist: matplotlib==3.10.0
35
+ Requires-Dist: memory-profiler==0.61.0
36
+ Requires-Dist: s3fs==2024.12.0
37
+ Requires-Dist: scikit-learn
38
+ Dynamic: author
39
+ Dynamic: author-email
40
+ Dynamic: classifier
41
+ Dynamic: description
42
+ Dynamic: description-content-type
43
+ Dynamic: home-page
44
+ Dynamic: license-file
45
+ Dynamic: project-url
46
+ Dynamic: requires-dist
47
+ Dynamic: requires-python
48
+ Dynamic: summary
49
+
50
+ # Rhapso
51
+
52
+ **Rhapso** is a modular Python toolkit for interest point based registration, alignment, and fusing of large-scale microscopy datasets.
53
+
54
+ [![License](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
55
+ [![Python Version](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/release/python-3110/)
56
+ [![Documentation](https://img.shields.io/badge/docs-wiki-blue)](https://github.com/AllenNeuralDynamics/Rhapso/wiki)
57
+
58
+ <!-- ## Example Usage Media Content Coming Soon....
59
+ -- -->
60
+
61
+ <br>
62
+
63
+ ## Table of Contents
64
+ - [Summary](#summary)
65
+ - [Contact](#contact)
66
+ - [Features](#features)
67
+ - [Performance](#performance)
68
+ - [Layout](#layout)
69
+ - [Installation](#installation)
70
+ - [Ray](#ray)
71
+ - [Run Locally w/ Ray](#run-locally-with-ray)
72
+ - [Run on AWS Cluster w/ Ray](#run-on-aws-cluster-with-ray)
73
+ - [Access Ray Dashboard](#access-ray-dashboard)
74
+ - [Parameters](#parameters)
75
+ - [Tuning Guide](#tuning-guide)
76
+ - [Build Package](#build-package)
77
+ - [Using the Built `.whl` File](#using-the-built-whl-file)
78
+
79
+ ---
80
+
81
+ <br>
82
+
83
+ **Update 11/26/25**
84
+ --------
85
+ Rhapso is still loading... and while we wrap up development, a couple things to know if you are outside the Allen Institute:
86
+ - This process requires a very specific XML structure to work.
87
+ - Fusion/Mutliscale is included but still under testing and development
88
+
89
+ <br>
90
+
91
+ ## Summary
92
+ Rhapso is a set of Python components for registration, alignment, and stitching of large-scale, 3D, overlapping tile-based, multiscale microscopy datasets.
93
+
94
+ Rhapso was developed by the Allen Institute for Neural Dynamics. Rhapso is comprised of stateless components. You can call these components using a pipeline script, with the option to run on a single machine or scale out with Ray to cloud based (currently only supporting AWS) clusters.
95
+
96
+ Current data loaders support Zarr and Tiff.
97
+
98
+ <br>
99
+
100
+ ## Contact
101
+ Questions or want to contribute? Please open an issue..
102
+
103
+ <br>
104
+
105
+ ## Features
106
+ - **Interest Point Detection** - using DOG based feature detection
107
+ - **Interest Point Matching** - using descriptor based RANSAC to match feature points
108
+ - **Global Optimization** - aligning matched features per tile, globally
109
+ - **Validation and Visualization Tools** - validate component specific results for the best output
110
+
111
+ ---
112
+
113
+ <br>
114
+
115
+ ## High Level Approach to Registration, Alignment, and Fusion
116
+
117
+ We first run **interest point detection** to capture feature points in the dataset, focusing on overlapping regions between tiles. These points drive all downstream alignment.
118
+
119
+ Next, we perform **alignment** in two-three stages, with regularized models:
120
+
121
+ 1. **Rigid matching + solver** – Match interest points with a rigid model and solve for globally consistent rigid transforms between all tiles.
122
+ 2. **Affine matching + solver** – Starting from the rigid solution, repeat matching with an affine model to recover more precise tile transforms.
123
+ 3. **Split affine matching + solver** – For very large z-stacks, we recommend first running the split dataset component to chunk tiles into smaller Z-bounds, then repeating affine matching and solving in “split affine” mode to refine local alignment.
124
+
125
+ All resulting transforms are written back into the input XML.
126
+
127
+ Whether you split or not, once the XML contains your final transforms, you are ready for **fusion**. We recommend viewing the aligned XML in FIJI/BDV to visually confirm alignment quality before running fusion.
128
+
129
+
130
+ ---
131
+
132
+ <br>
133
+
134
+ ## Performance
135
+
136
+ **Interest Point Detection Performance Example (130TB Zarr dataset)**
137
+
138
+ | Environment | Resources | Avg runtime |
139
+ |:----------------------|:---------------------|:-----------:|
140
+ | Local single machine | 10 CPU, 10 GB RAM | ~120 min |
141
+ | AWS Ray cluster | 560 CPU, 4.4 TB RAM | ~30 min |
142
+
143
+ <br>
144
+ *Actual times vary by pipeline components, dataset size, tiling, and parameter choices.*
145
+
146
+ ---
147
+
148
+ <br>
149
+
150
+ ## Layout
151
+
152
+ ```
153
+ Rhapso/
154
+ └── Rhapso/
155
+ ├── data_prep/ # Custom data loaders
156
+ ├── detection/
157
+ ├── evaluation/
158
+ ├── fusion/
159
+ ├── image_split/
160
+ ├── matching/
161
+ ├── pipelines/
162
+ │ └── ray/
163
+ │ ├── aws/
164
+ │ │ ├── config/ # Cluster templates (edit for your account)
165
+ │ │ └── alignment_pipeline.py # AWS Ray pipeline entry point
166
+ │ ├── local/
167
+ │ │ └── alignment_pipeline.py # Local Ray pipeline entry point
168
+ │ ├── param/ # Run parameter files (customize per run)
169
+ │ ├── interest_point_detection.py # Detection pipeline script
170
+ │ ├── interest_point_matching.py # Matching pipeline script
171
+ │ └── solver.py # Global solver script
172
+ ├── solver/
173
+ └── visualization/ # Validation tools
174
+ ```
175
+
176
+ ---
177
+
178
+ <br>
179
+
180
+
181
+ ## Installation
182
+
183
+ ```sh
184
+ # clone the repo
185
+ git clone https://github.com/AllenNeuralDynamics/Rhapso.git
186
+
187
+ # create and activate a virtual environment
188
+ python -m venv .venv && source .venv/bin/activate
189
+ # or: conda create -n rhapso python=3.11 && conda activate rhapso
190
+
191
+ # install deps
192
+ pip install -r requirements.txt
193
+ ```
194
+ ---
195
+
196
+ <br>
197
+
198
+ ## Ray
199
+
200
+ **Ray** is a Python framework for parallel and distributed computing. It lets you run regular Python functions in parallel on a single machine **or** scale them out to a cluster (e.g., AWS) with minimal code changes. In Rhapso, we use Ray to process large scale datasets.
201
+
202
+ - Convert a function into a distributed task with `@ray.remote`
203
+ - Control scheduling with resource hints (CPUs, memory)
204
+
205
+ <br>
206
+
207
+ > [!TIP]
208
+ > Ray schedules **greedily** by default and each task reserves **1 CPU**, so if you fire many tasks, Ray will try to run as many as your machine advertises—often too much for a laptop. Throttle concurrency explicitly so you don’t overload your system. Use your machine's activity monitor to track this or the Ray dashboard to monitor this on your cluster:
209
+ >
210
+ > - **Cap by CPUs**:
211
+ > ```python
212
+ > @ray.remote(num_cpus=3) # Ray will schedule each time 3 cpus are available
213
+ > ```
214
+ > - **Cap by Memory and CPU** if Tasks are RAM-Heavy (bytes):
215
+ > ```python
216
+ > @ray.remote(num_cpus=2, memory=4 * 1024**3) # 4 GiB and 2 CPU per task>
217
+ > ```
218
+ > - **No Cap** on Resources:
219
+ > ```python
220
+ > @ray.remote
221
+ > ```
222
+ > - **Good Local Default:**
223
+ > ```python
224
+ > @ray.remote(num_cpus=2)
225
+ > ```
226
+
227
+ ---
228
+
229
+ <br>
230
+
231
+
232
+ ## Run Locally with Ray
233
+
234
+ ### 1. Edit or create param file (templates in codebase)
235
+ ```python
236
+ Rhapso/Rhapso/pipelines/param/
237
+ ```
238
+
239
+ ### 2. Update alignment pipeline script to point to param file
240
+ ```python
241
+ with open("Rhapso/pipelines/ray/param/your_param_file.yml", "r") as file:
242
+ config = yaml.safe_load(file)
243
+ ```
244
+
245
+ ### 3. Run local alignment pipeline script
246
+ ```python
247
+ python Rhapso/pipelines/ray/local/alignment_pipeline.py
248
+
249
+ ```
250
+
251
+ ---
252
+
253
+ <br>
254
+
255
+
256
+ ## Run on AWS Cluster with Ray
257
+
258
+ ### 1. Edit/create param file (templates in codebase)
259
+ ```python
260
+ Rhapso/pipelines/ray/param/
261
+ ```
262
+
263
+ ### 2. Update alignment pipeline script to point to param file
264
+ ```python
265
+ with open("Rhapso/pipelines/ray/param/your_param_file.yml", "r") as file:
266
+ config = yaml.safe_load(file)
267
+ ```
268
+
269
+ ### 3. Edit/create config file (templates in codebase)
270
+ ```python
271
+ Rhapso/pipelines/ray/aws/config/
272
+ ```
273
+
274
+ ### 4. Update config file to point to whl location in setup_commands
275
+ ```python
276
+ - aws s3 cp s3://rhapso-whl-v2/Rhapso-0.1.8-py3-none-any.whl /tmp/Rhapso-0.1.8-py3-none-any.whl
277
+ ```
278
+
279
+ ### 5. Update alignment pipeline script to point to config file
280
+ ```python
281
+ unified_yml = "your_cluster_config_file_name.yml"
282
+ ```
283
+
284
+ ### 6. Create whl file and upload to s3
285
+ ```python
286
+ python setup.py sdist bdist_wheel
287
+ ```
288
+
289
+ ### 7. Run AWS alignment pipeline script
290
+ ```python
291
+ python Rhapso/pipelines/ray/aws/alignment_pipeline.py
292
+ ```
293
+
294
+ > [!TIP]
295
+ > - The pipeline script is set to always spin the cluster down, it is a good practice to double check in AWS.
296
+ > - If you experience a sticky cache on run params, you may have forgotten to spin your old cluster down.
297
+
298
+ <br>
299
+
300
+ ## Access Ray Dashboard
301
+
302
+ **This is a great place to tune your cluster's performance.**
303
+ 1. Find public IP of head node.
304
+ 2. Replace the ip address and PEM file location to ssh into head node.
305
+ ```
306
+ ssh -i /You/path/to/ssh/key.pem -L port:localhost:port ubuntu@public.ip.address
307
+ ```
308
+ 4. Go to dashboard.
309
+ ```
310
+ http://localhost:8265
311
+ ```
312
+
313
+ ---
314
+
315
+ <br>
316
+
317
+ ## Parameters
318
+
319
+ ### Detection
320
+ ```
321
+ | Parameter | Feature / step | What it does | Typical range\* |
322
+ | :----------------- | :--------------------- | :-------------------------------------------------------------------------------------------- | :-------------------------------- |
323
+ | `dsxy` | Downsampling (XY) | Reduces XY resolution before detection; speeds up & denoises, but raises minimum feature size | 16 |
324
+ | `dsz` | Downsampling (Z) | Reduces Z resolution; often lower than XY due to anisotropy | 16 |
325
+ | `min_intensity` | Normalization | Lower bound for intensity normalization prior to DoG | 1 |
326
+ | `max_intensity` | Normalization | Upper bound for intensity normalization prior to DoG | 5 |
327
+ | `sigma` | DoG blur | Gaussian blur scale (sets feature size); higher = smoother, fewer peaks | 1.5 - 2.5 |
328
+ | `threshold` | Peak detection (DoG) | Peak threshold (initial min peak ≈ `threshold / 3`); higher = fewer, stronger peaks | 0.0008 - .05 |
329
+ | `median_filter` | Pre-filter (XY) | Median filter size to suppress speckle/isolated noise before DoG | 1-10 |
330
+ | `combine_distance` | Post-merge (DoG peaks) | Merge radius (voxels) to de-duplicate nearby detections | 0.5 |
331
+ | `chunks_per_bound` | Tiling/parallelism | Sub-partitions per tile/bound; higher improves parallelism but adds overhead | 12-18 |
332
+ | `max_spots` | Post-cap | Maximum detections per bound to prevent domination by dense regions | 8,0000 - 10,000 |
333
+ ```
334
+ <br>
335
+
336
+ ### Matching
337
+ ```
338
+ # Candidate Selection
339
+ | Parameter | Feature / step | What it does | Typical range |
340
+ | :----------------------------- | :------------------ | :---------------------------------------------------------------- | :------------- |
341
+ | `num_neighbors` | Candidate search | Number of nearest neighbors to consider per point | 3 |
342
+ | `redundancy` | Candidate search | Extra neighbors added for robustness beyond `num_neighbors` | 0 - 1 |
343
+ | `significance` | Ratio test | Strictness of descriptor ratio test; larger = stricter acceptance | 3 |
344
+ | `search_radius` | Spatial gating | Max spatial distance for candidate matches (in downsampled units) | 100 - 300 |
345
+ | `num_required_neighbors` | Candidate filtering | Minimum neighbors required to keep a candidate point | 3 |
346
+
347
+ # Ransac
348
+ | Parameter | Feature / step | What it does | Typical range |
349
+ | :---------------------------- | :------------------- | :---------------------------------------------------------------- | :------------- |
350
+ | `model_min_matches` | RANSAC | Minimum correspondences to estimate a rigid transform | 18 – 32 |
351
+ | `inlier_factor` | RANSAC | Inlier tolerance scaling; larger = looser inlier threshold | 30 – 100 |
352
+ | `lambda_value` | RANSAC | Regularization strength during model fitting | 0.1 – 0.05 |
353
+ | `num_iterations` | RANSAC | Number of RANSAC trials; higher = more robust, slower | 10,0000 |
354
+ | `regularization_weight` | RANSAC | Weight applied to the regularization term | 1.0 |
355
+
356
+ ```
357
+ <br>
358
+
359
+ ### Solver
360
+ ```
361
+ | Parameter | Feature / step | What it does | Typical range |
362
+ | :------------------- | :------------- | :----------------------------------------------------------------- | :------------------ |
363
+ | `relative_threshold` | Graph pruning | Reject edges with residuals above dataset-relative cutoff | 3.5 |
364
+ | `absolute_threshold` | Graph pruning | Reject edges above an absolute error bound (detection-space units) | 7.0 |
365
+ | `min_matches` | Graph pruning | Minimum matches required to retain an edge between tiles | 3 |
366
+ | `damp` | Optimization | Damping for iterative solver; higher can stabilize tough cases | 1.0 |
367
+ | `max_iterations` | Optimization | Upper bound on solver iterations | 10,0000 |
368
+ | `max_allowed_error` | Optimization | Overall error cap; `inf` disables hard stop by error | `inf` |
369
+ | `max_plateauwidth` | Early stopping | Stagnation window before stopping on no improvement | 200 |
370
+
371
+ ```
372
+
373
+ ---
374
+
375
+ <br>
376
+
377
+ ## Tuning Guide
378
+
379
+ - **Start with Detection.** The quality and density of interest points strongly determine alignment outcomes.
380
+
381
+ - **Target Counts (exaSPIM):** ~25–35k points per tile in dense regions; ~10k for sparser tiles. Going much higher usually increases runtime without meaningful accuracy gains.
382
+
383
+ - **Inspect Early.** After detection, run the visualization script and verify that peaks form **clustered shapes/lines** with a **good spatial spread**—a good sign for robust rigid matches.
384
+
385
+ - **Rigid → Affine Dependency.** Weak rigid matches produce poor rigid transforms, which then degrade affine matching (points don’t land close enough). If tiles fail to align:
386
+ - Check **match counts** for the problem tile and its neighbors.
387
+ - Adjust high-impact detection knobs—`sigma`, `threshold`, and `median_filter`—within sensible ranges.
388
+ - Revisit `max_spots` and `combine_distance` to balance density vs. duplicate detections.
389
+
390
+ ---
391
+
392
+ <br>
393
+
394
+ ## Build Package
395
+
396
+ ### Using the Built `.whl` File
397
+
398
+ 1. **Build the `.whl` File in the root of this repo:**
399
+ ```sh
400
+ cd /path/to/Rhapso
401
+ pip install setuptools wheel
402
+ python setup.py sdist bdist_wheel
403
+ ```
404
+ The `.whl` file will appear in the `dist` directory. Do not rename it to ensure compatibility (e.g., `rhapso-0.1-py3-none-any.whl`).
405
+
406
+ ---
407
+
408
+ <br>
409
+ <br>
410
+ <br>