octopi 1.0__py3-none-any.whl → 1.2.0__py3-none-any.whl

octopi-1.2.0.dist-info/METADATA

@@ -0,0 +1,120 @@
+Metadata-Version: 2.4
+Name: octopi
+Version: 1.2.0
+Summary: Model architecture exploration for cryoET particle picking
+Project-URL: Homepage, https://github.com/chanzuckerberg/octopi
+Project-URL: Documentation, https://chanzuckerberg.github.io/octopi/
+Project-URL: Issues, https://github.com/chanzuckerberg/octopi/issues
+Author: Jonathan Schwartz, Kevin Zhao, Daniel Ji, Utz Ermel
+License: MIT
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Image Processing
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
+Requires-Python: >=3.9
+Requires-Dist: copick
+Requires-Dist: copick-utils
+Requires-Dist: ipywidgets
+Requires-Dist: kaleido
+Requires-Dist: matplotlib
+Requires-Dist: mlflow
+Requires-Dist: monai
+Requires-Dist: mrcfile
+Requires-Dist: multiprocess
+Requires-Dist: nibabel
+Requires-Dist: optuna
+Requires-Dist: optuna-integration[botorch,pytorch-lightning]
+Requires-Dist: pandas
+Requires-Dist: python-dotenv
+Requires-Dist: requests
+Requires-Dist: torch-ema
+Requires-Dist: tqdm
+Provides-Extra: dev
+Requires-Dist: black>=24.8.0; extra == 'dev'
+Requires-Dist: pre-commit>=3.8.0; extra == 'dev'
+Requires-Dist: pytest>=6.2.3; extra == 'dev'
+Requires-Dist: ruff>=0.6.4; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs; extra == 'docs'
+Requires-Dist: mkdocs-awesome-pages-plugin; extra == 'docs'
+Requires-Dist: mkdocs-git-authors-plugin; extra == 'docs'
+Requires-Dist: mkdocs-git-committers-plugin-2; extra == 'docs'
+Requires-Dist: mkdocs-git-revision-date-localized-plugin; extra == 'docs'
+Requires-Dist: mkdocs-material; extra == 'docs'
+Requires-Dist: mkdocs-minify-plugin; extra == 'docs'
+Requires-Dist: mkdocs-redirects; extra == 'docs'
+Description-Content-Type: text/markdown
+
+# OCTOPI 🐙🐙🐙
+
+[![License](https://img.shields.io/pypi/l/octopi.svg?color=green)](https://github.com/chanzuckerberg/octopi/raw/main/LICENSE)
+[![PyPI](https://img.shields.io/pypi/v/octopi.svg?color=green)](https://pypi.org/project/octopi)
+[![Python Version](https://img.shields.io/pypi/pyversions/octopi.svg?color=green)](https://www.python.org/)
+
+**O**bject dete**CT**ion **O**f **P**rote**I**ns. A deep learning framework for Cryo-ET 3D particle picking with autonomous model exploration capabilities.
+
+## 🚀 Introduction
+
+octopi addresses a critical bottleneck in cryo-electron tomography (cryo-ET) research: the efficient identification and extraction of proteins within complex cellular environments. As advances in cryo-ET enable the collection of thousands of tomograms, the need for automated, accurate particle picking has become increasingly urgent.
+
+Our deep learning-based pipeline streamlines the training and execution of 3D autoencoder models specifically designed for cryo-ET particle picking. Built on [copick](https://github.com/copick/copick), a storage-agnostic API, octopi seamlessly accesses tomograms and segmentations across local and remote environments. 
+
+## 🧩 Core Features
+
+- **3D U-Net Training**: Train and evaluate custom 3D U-Net models for particle segmentation
+- **Automatic Architecture Search**: Explore optimal model configurations using Bayesian optimization via Optuna
+- **Flexible Data Access**: Seamlessly work with tomograms from local storage or remote data portals
+- **HPC Ready**: Built-in support for SLURM-based clusters
+- **Experiment Tracking**: Integrated MLflow support for monitoring training and optimization
+- **Dual Interface**: Use via command-line or Python API
+
+## 🚀 Quick Start
+
+### Installation
+
+Octopi is availableon PyPI and can be installed using pip:
+```bash
+pip install octopi
+```
+
+⚠️ **Note**: One of the current dependencies is currently not working with pip 25. To temporarily reduce the pip version, run:
+```bash
+pip install --upgrade "pip<25"
+```
+
+### Basic Usage
+
+octopi provides two main command-line interfaces:
+
+```bash
+# Main CLI for training, inference, and data processing
+octopi --help
+
+# HPC-specific CLI for submitting jobs to SLURM clusters
+octopi-slurm --help
+```
+
+## 📚 Documentation
+
+For detailed documentation, tutorials, CLI and API reference, visit our [documentation](https://chanzuckerberg.github.io/octopi/).
+
+## 🤝 Contributing
+
+## Code of Conduct
+
+This project adheres to the Contributor Covenant [code of conduct](https://github.com/chanzuckerberg/.github/blob/master/CODE_OF_CONDUCT.md).
+By participating, you are expected to uphold this code. 
+Please report unacceptable behavior to [opensource@chanzuckerberg.com](mailto:opensource@chanzuckerberg.com).
+
+## 🔒 Security
+
+If you believe you have found a security issue, please responsibly disclose by contacting us at security@chanzuckerberg.com.
+
+

octopi-1.2.0.dist-info/RECORD

@@ -0,0 +1,62 @@
+octopi/__init__.py,sha256=Btl_98iBIXFtvGx47MqpfbaEVYoOMPBQn9bao7UASkQ,21
+octopi/main.py,sha256=ef_zpvopl6JiN4gOT_x2QghRJriqg4iwRzBnvuiBeTo,4864
+octopi/workflows.py,sha256=bRP5uPqu7PtUtQUOgtVjBZ6FulhG9uAdlcEH_kBYXPE,9235
+octopi/datasets/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/datasets/augment.py,sha256=k7UXQzidPANaPuLoBzer_ZHc4_vF-kKTOyZisEnAbNw,3203
+octopi/datasets/cached_datset.py,sha256=z20Ldfve1nNfLLPNx6nUE2p6i4IXx4V2SHmt99aVN_k,3918
+octopi/datasets/dataset.py,sha256=9C3AuD83FMswj4MYLFn5XPqvAP1a7MQSG7UNiStg090,511
+octopi/datasets/generators.py,sha256=qL43vd6gxpPrySZmt8RWujtJo3eO6sFcpS92CmNkqSA,18883
+octopi/datasets/io.py,sha256=COjGoCLVbPU_xQC2UCaHGPd24a8AXtDvUqyd8kOc-Eo,6978
+octopi/datasets/mixup.py,sha256=BJUAmM7ItZWFChs8glnd8RNSXR5qGW7DHscbcVc3TsU,1575
+octopi/datasets/multi_config_generator.py,sha256=_eAZOzWLo6bI_H8x0quoslyjfuERVRXEVHZ4aW7j_Ok,10814
+octopi/entry_points/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/entry_points/common.py,sha256=pHYQFQKdrtbYYU96klTQV4u1acUBfApkbEOc_TbuIzA,5367
+octopi/entry_points/create_slurm_submission.py,sha256=rW1hr59mLLvKzmHG-uozYvJvTcqnHNZdmy2nMr5EBDc,8914
+octopi/entry_points/run_create_targets.py,sha256=FPpsgVQHbepJvE2ZdC1VQKdT5RpALdhDXk7jU1RUXKo,11236
+octopi/entry_points/run_evaluate.py,sha256=ab_VEDlh3RxMHfSyoMCdFH6z9G3fpHIfAKvyoYmTP2M,2835
+octopi/entry_points/run_extract_mb_picks.py,sha256=u7mVqYKEvvkSN-L7v27UJVURxl74q7yiUZwTs-3zfYE,4866
+octopi/entry_points/run_extract_midpoint.py,sha256=O6GdkSD7oXIpSVqizOc6PHhv9nunz3j0RucmYQ2yryM,5742
+octopi/entry_points/run_localize.py,sha256=-CF9w3XKLjiGIAN-kWgeobpfFYfIvZxRpxR0D8fyb3M,8264
+octopi/entry_points/run_optuna.py,sha256=U1VYzwE0eYV8DNnms0gbjytqsDwSvHJ5BsMj48iZz8k,5784
+octopi/entry_points/run_segment_predict.py,sha256=5RvnxUDDNzBfRv4dOGJl6SieAlF5SR5stHL_Z6BgUh0,5439
+octopi/entry_points/run_train.py,sha256=O9uDunSyQGpEXqTTOYNSgO3LvTZt49K9HzcN0sGuD14,8095
+octopi/extract/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/extract/localize.py,sha256=SQY1p2Q9sYrB_yjOZySYqoWwwlfgcvBTR1f_Fpi2Wxc,9528
+octopi/extract/membranebound_extract.py,sha256=kTV_QFBbvzd3STJsC55QKoRTYcvg_raV3C757Y_9HBo,11087
+octopi/extract/midpoint_extract.py,sha256=iEUZJShlhBC5qAdP5w-zjjEOMz_nu9JZF6jhbJvcV94,7036
+octopi/models/AttentionUnet.py,sha256=r185aXRtfXhN-n8FxA-Sjz18nqpxHH_2t2uadrH-Mgs,1991
+octopi/models/MedNeXt.py,sha256=9q0FsyrqTx211hCbDv0Lm2XflzXL_bGA4-76BscziGk,4875
+octopi/models/ModelTemplate.py,sha256=X80EOXwSovCjmVb7x-0_JmRjHfDfLByDdd60MrgFTyw,1084
+octopi/models/SegResNet.py,sha256=1dK8dy_7hHHKYZLsTYafl__7MxQOlWGbBqQWPGxHSXg,3609
+octopi/models/Unet.py,sha256=7RGT_nl7ogsNlS3Y3Qexe305Ym9NlK9CV0y45g2DEU4,2171
+octopi/models/UnetPlusPlus.py,sha256=fnV-SvJV8B432KJXQAtdwLy8Va6DJ4fRB_7a1mZiqTU,1529
+octopi/models/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/models/common.py,sha256=qxx1ak2ayVC9fPbpjs6XtfUNPSX-PGywq3ySsXGdY74,2313
+octopi/processing/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/processing/create_targets_from_picks.py,sha256=5dbCCjFXq55A-zD9SX4mXUKRq_lPWriTlnRYb958hBc,4377
+octopi/processing/downsample.py,sha256=u3V2HULdKTLEthzI-MqJmI-NqpiyX0Vg61i77-j2MKQ,5392
+octopi/processing/evaluate.py,sha256=Xxx8hJrWzZ_pSxEs89CC_6AxGUch3_RSHpH7UtEBs1c,14139
+octopi/processing/importers.py,sha256=Rt-Y3QtFJz_ewCkdqtJpmiiI4xFSFeXaRcL1ZBn0bd0,8876
+octopi/processing/segmentation_from_picks.py,sha256=jah1gAXEn09LIok1Cb8IeVN-fT3jktcVPfjbOFHkgg0,7089
+octopi/pytorch/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/pytorch/hyper_search.py,sha256=ywwGEtTHJUkrBCOYSycrpWccUvxJlxny50q5GnWoWLA,9376
+octopi/pytorch/model_search_submitter.py,sha256=zEfvlcB9AthtanUaqV3a8f2HGG_eo5gvIOZDOvUziJo,11267
+octopi/pytorch/segmentation.py,sha256=nPW4tkQfgA4fC02j_bcLyKvS7SL9ghQnENKT7_YC_gc,11266
+octopi/pytorch/segmentation_multigpu.py,sha256=KK8BZIdB1PJBs_jCv7WBS0xORR_XlpcKmmPf3AgIfYY,6781
+octopi/pytorch/trainer.py,sha256=NAoOf00oh_-Sv5FZUaq_L5v7Y_VV_VtuExIaZCrvxfs,17760
+octopi/pytorch_lightning/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/pytorch_lightning/optuna_pl_ddp.py,sha256=ynD5i81IP-awr6OD9GDurjrQK-5Kc079qPaukphTHnA,11924
+octopi/pytorch_lightning/train_pl.py,sha256=igOHzU_mUdZRQGhoOGW5vmxJcHFcw5fAPHfVCIZ0eG4,10220
+octopi/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+octopi/utils/config.py,sha256=RLrHmyjnQ0tXX2_bls--ZecbgRTNlnGtkvLl7YWtYtc,1833
+octopi/utils/io.py,sha256=-PMzzBiIKbrUAIw7NcmeAj2V0yy9r4N2QRMIGlm2SII,4623
+octopi/utils/losses.py,sha256=fs9yR80Hs-hL07WgVMkRy5N81vzP5m9XBCvzO76bIPU,3097
+octopi/utils/parsers.py,sha256=InHLN-iYq0bWoKAHCX-hX82HO8b4pcZZ1on1LcwcsXo,5709
+octopi/utils/stopping_criteria.py,sha256=q-tkcqyZ3GfqR8BzWu1eKE8dt2pNZ3MOD9dcMe2T7fU,6112
+octopi/utils/submit_slurm.py,sha256=cRbJTESbPFCt6Cq4Hat2uPOQKFYMPcQxuNs0jc1ygUA,1945
+octopi/utils/visualization_tools.py,sha256=I-fmXKk9eyaLTbvsYxx8H6nlEvUOH94NJ_l2dayXXe4,7364
+octopi-1.2.0.dist-info/METADATA,sha256=UWZ2lfjUNDd34EDtL47LdpIp4p_TbkemNWhqjsehQ6U,5062
+octopi-1.2.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+octopi-1.2.0.dist-info/entry_points.txt,sha256=n4I7aA9RS52l1eYPidQx63VhxT9pPsMK4xitlBm7X7k,90
+octopi-1.2.0.dist-info/licenses/LICENSE,sha256=DRoLUEjEHgru_AwKHhii9UIOt6QA65am4WQd7UFzqnE,1822
+octopi-1.2.0.dist-info/RECORD,,

{octopi-1.0.dist-info → octopi-1.2.0.dist-info}/WHEEL

@@ -1,4 +1,4 @@
 Wheel-Version: 1.0
-Generator: poetry-core 2.1.3
+Generator: hatchling 1.27.0
 Root-Is-Purelib: true
 Tag: py3-none-any

octopi-1.2.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+octopi = octopi.main:cli_main
+octopi-slurm = octopi.main:cli_slurm_main

{octopi-1.0.dist-info → octopi-1.2.0.dist-info/licenses}/LICENSE

@@ -17,15 +17,15 @@ copies of the Software, and to permit persons to whom the Software is
 furnished to do so, subject to the following conditions:
 
 The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
+copies or substantial portions of the "Software".
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+OUT OF OR IN CONNECTION WITH THE "SOFTWARE" OR THE USE OR OTHER DEALINGS IN THE
+"SOFTWARE".
 ```
 
 ## License Notice for Dependencies

octopi/io.py

@@ -1,457 +0,0 @@
-from monai.data import DataLoader, CacheDataset, Dataset
-from monai.transforms import (
-    Compose, 
-    NormalizeIntensityd,
-    EnsureChannelFirstd,  
-)
-from sklearn.model_selection import train_test_split
-import copick, torch, os, json, random
-from collections import defaultdict
-from octopi import utils
-from typing import List
-from tqdm import tqdm
-import numpy as np
-
-##############################################################################################################################    
-
-def load_training_data(root, 
-                       runIDs: List[str],
-                       voxel_spacing: float, 
-                       tomo_algorithm: str, 
-                       segmenation_name: str,
-                       segmentation_session_id: str = None,
-                       segmentation_user_id: str = None,
-                       progress_update: bool = True):
-    
-    data_dicts = []
-    # Use tqdm for progress tracking only if progress_update is True
-    iterable = tqdm(runIDs, desc="Loading Training Data") if progress_update else runIDs
-    for runID in iterable:
-        run = root.get_run(str(runID))
-        tomogram = get_tomogram_array(run, voxel_spacing, tomo_algorithm)
-        segmentation = get_segmentation_array(run, 
-                                              voxel_spacing,
-                                              segmenation_name,
-                                              segmentation_session_id, 
-                                              segmentation_user_id)
-        data_dicts.append({"image": tomogram, "label": segmentation})
-
-    return data_dicts 
-
-##############################################################################################################################    
-
-def load_predict_data(root, 
-                      runIDs: List[str],
-                      voxel_spacing: float, 
-                      tomo_algorithm: str):
-    
-    data_dicts = []
-    for runID in tqdm(runIDs):
-        run = root.get_run(str(runID))
-        tomogram = get_tomogram_array(run, voxel_spacing, tomo_algorithm)
-        data_dicts.append({"image": tomogram})
-
-    return data_dicts 
-
-##############################################################################################################################    
-
-def create_predict_dataloader(
-    root,
-    voxel_spacing: float, 
-    tomo_algorithm: str,       
-    runIDs: str = None,       
-    ): 
-
-    # define pre transforms
-    pre_transforms = Compose(
-        [   EnsureChannelFirstd(keys=["image"], channel_dim="no_channel"),
-            NormalizeIntensityd(keys=["image"]),
-    ])
-
-    # Split trainRunIDs, validateRunIDs, testRunIDs
-    if runIDs is None:
-        runIDs = [run.name for run in root.runs]
-    test_files = load_predict_data(root, runIDs, voxel_spacing, tomo_algorithm) 
-
-    bs = min( len(test_files), 4)
-    test_ds = CacheDataset(data=test_files, transform=pre_transforms)
-    test_loader = DataLoader(test_ds, 
-                            batch_size=bs, 
-                            shuffle=False, 
-                            num_workers=4, 
-                            pin_memory=torch.cuda.is_available())
-    return test_loader, test_ds
-
-##############################################################################################################################
-
-def get_tomogram_array(run, 
-                       voxel_size: float = 10, 
-                       tomo_type: str = 'wbp',
-                       raise_error: bool = True):
-    
-    voxel_spacing_obj = run.get_voxel_spacing(voxel_size)
-
-    if voxel_spacing_obj is None:
-        # Query Avaiable Voxel Spacings
-        availableVoxelSpacings = [tomo.voxel_size for tomo in run.voxel_spacings]
-
-        # Report to the user which voxel spacings they can use 
-        message = (f"\n[Warning] No tomogram found for {run.name} with voxel size {voxel_size} and tomogram type {tomo_type}"
-                   f"\nAvailable spacings are: {', '.join(map(str, availableVoxelSpacings))}\n" ) 
-        if raise_error:
-            raise ValueError(message)
-        else:
-            print(message)
-            return None
-    
-    tomogram = voxel_spacing_obj.get_tomogram(tomo_type)
-    if tomogram is None:
-        # Get available algorithms
-        availableAlgorithms = [tomo.tomo_type for tomo in run.get_voxel_spacing(voxel_size).tomograms]
-        
-        # Report to the user which algorithms are available
-        message = (f"\n[Warning] No tomogram found for {run.name} with voxel size {voxel_size} and tomogram type {tomo_type}"
-                   f"\nAvailable algorithms are: {', '.join(availableAlgorithms)}\n")
-        if raise_error:
-            raise ValueError(message)
-        else:
-            print(message)
-            return None
-
-    return tomogram.numpy().astype(np.float32)
-
-##############################################################################################################################
-
-def get_segmentation_array(run, 
-                           voxel_spacing: float,
-                           segmentation_name: str, 
-                           session_id=None,
-                           user_id=None,
-                           raise_error: bool = True):
-
-    seg = run.get_segmentations(name=segmentation_name, 
-                                session_id = session_id,
-                                user_id = user_id,
-                                voxel_size=float(voxel_spacing))
-    
-    # No Segmentations Are Available, Result in Error
-    if len(seg) == 0:
-        # Get all available segmentations with their metadata
-        available_segs = run.get_segmentations(voxel_size=voxel_spacing)
-        seg_info = [(s.name, s.user_id, s.session_id) for s in available_segs]
-        
-        # Format the information for display
-        seg_details = [f"(name: {name}, user_id: {uid}, session_id: {sid})" 
-                      for name, uid, sid in seg_info]
-        
-        message = ( f'\nNo segmentation found matching:\n'
-                    f'  name: {segmentation_name}, user_id: {user_id}, session_id: {session_id}\n'
-                    f'Available segmentations in {run.name} are:\n  ' + 
-                    '\n  '.join(seg_details) )
-        if raise_error:
-            raise ValueError(message)
-        else:
-            print(message)
-            return None
-
-    # No Segmentations Are Available, Result in Error
-    if len(seg) > 1:
-        print(f'[Warning] More Than 1 Segmentation is Available for the Query Information. '
-              f'Available Segmentations are: {seg} '
-              f'Defaulting to Loading: {seg[0]}\n')
-    seg = seg[0]
-
-    return seg.numpy().astype(np.int8)
-
-##############################################################################################################################
-
-def get_copick_coordinates(run,                    # CoPick run object containing the segmentation data
-                           name: str,              # Name of the object or protein for which coordinates are being extracted
-                           user_id: str,           # Identifier of the user that generated the picks
-                           session_id: str = None, # Identifier of the session that generated the picks
-                           voxel_size: float = 10,  # Voxel size of the tomogram, used for scaling the coordinates
-                           raise_error: bool = True):
-                           
-    # Retrieve the pick points associated with the specified object and user ID
-    picks = run.get_picks(object_name=name, user_id=user_id, session_id=session_id)
-
-    if len(picks) == 0:
-        # Get all available segmentations with their metadata
-
-        available_picks = run.get_picks()
-        picks_info = [(s.pickable_object_name, s.user_id, s.session_id) for s in available_picks]
-        
-        # Format the information for display
-        picks_details = [f"(name: {name}, user_id: {uid}, session_id: {sid})" 
-                      for name, uid, sid in picks_info]
-        
-        message = ( f'\nNo picks found matching:\n'
-                    f'  name: {name}, user_id: {user_id}, session_id: {session_id}\n'
-                    f'Available picks are:\n  '  
-                    + '\n  '.join(picks_details) )
-        if raise_error:
-            raise ValueError(message)
-        else:
-            print(message)
-            return None
-    elif len(picks) > 1:
-        # Format pick information for display
-        picks_info = [(p.pickable_object_name, p.user_id, p.session_id) for p in picks]
-        picks_details = [f"(name: {name}, user_id: {uid}, session_id: {sid})" 
-                        for name, uid, sid in picks_info]
-
-        print(f'[Warning] More than 1 pick is available for the query information.'
-              f'\nAvailable picks are:\n  ' + 
-              '\n  '.join(picks_details) +
-              f'\nDefaulting to loading:\n {picks[0]}\n')
-    points = picks[0].points
-
-    # Initialize an array to store the coordinates
-    nPoints = len(picks[0].points)                      # Number of points retrieved
-    coordinates = np.zeros([len(picks[0].points), 3])   # Create an empty array to hold the (z, y, x) coordinates
-
-    # Iterate over all points and convert their locations to coordinates in voxel space
-    for ii in range(nPoints):
-        coordinates[ii,] = [points[ii].location.z / voxel_size,   # Scale z-coordinate by voxel size
-                            points[ii].location.y / voxel_size,   # Scale y-coordinate by voxel size
-                            points[ii].location.x / voxel_size]   # Scale x-coordinate by voxel size
-
-    # Return the array of coordinates
-    return coordinates
-    
-
-##############################################################################################################################
-
-def adjust_to_multiple(value, multiple = 16):
-    return int((value // multiple) * multiple)
-
-def get_input_dimensions(dataset, crop_size: int):
-    nx = dataset[0]['image'].shape[1]
-    if crop_size > nx:
-        first_dim = adjust_to_multiple(nx/2)
-        return first_dim, crop_size, crop_size
-    else:
-        return crop_size, crop_size, crop_size
-
-def get_num_classes(copick_config_path: str):
-
-    root = copick.from_file(copick_config_path)
-    return len(root.pickable_objects) + 1
-
-def split_multiclass_dataset(runIDs, 
-                             train_ratio: float = 0.7, 
-                             val_ratio: float = 0.15, 
-                             test_ratio: float = 0.15, 
-                             return_test_dataset: bool = True,
-                             random_state: int = 42):
-    """
-    Splits a given dataset into three subsets: training, validation, and testing. If the dataset
-    has categories (as tuples), splits are balanced across all categories. If the dataset is a 1D
-    list, it is split without categorization.
-
-    Parameters:
-    - runIDs: A list of items to split. It can be a 1D list or a list of tuples (category, value).
-    - train_ratio: Proportion of the dataset for training.
-    - val_ratio: Proportion of the dataset for validation.
-    - test_ratio: Proportion of the dataset for testing.
-    - return_test_dataset: Whether to return the test dataset.
-    - random_state: Random state for reproducibility.
-
-    Returns:
-    - trainRunIDs: Training subset.
-    - valRunIDs: Validation subset.
-    - testRunIDs: Testing subset (if return_test_dataset is True, otherwise None).
-    """
-
-    # Ensure the ratios add up to 1
-    assert train_ratio + val_ratio + test_ratio == 1.0, "Ratios must add up to 1.0"
-
-    # Check if the dataset has categories
-    if isinstance(runIDs[0], tuple) and len(runIDs[0]) == 2:
-        # Group by category
-        grouped = defaultdict(list)
-        for item in runIDs:
-            grouped[item[0]].append(item)
-
-        # Split each category
-        trainRunIDs, valRunIDs, testRunIDs = [], [], []
-        for category, items in grouped.items():
-            # Shuffle for randomness
-            random.shuffle(items)
-            # Split into train and remaining
-            train_items, remaining = train_test_split(items, test_size=(1 - train_ratio), random_state=random_state)
-            trainRunIDs.extend(train_items)
-
-            if return_test_dataset:
-                # Split remaining into validation and test
-                val_items, test_items = train_test_split(
-                    remaining,
-                    test_size=(test_ratio / (val_ratio + test_ratio)),
-                    random_state=random_state,
-                )
-                valRunIDs.extend(val_items)
-                testRunIDs.extend(test_items)
-            else:
-                valRunIDs.extend(remaining)
-                testRunIDs = []
-    else:
-        # If no categories, split as a 1D list
-        trainRunIDs, remaining = train_test_split(runIDs, test_size=(1 - train_ratio), random_state=random_state)
-        if return_test_dataset:
-            valRunIDs, testRunIDs = train_test_split(
-                remaining,
-                test_size=(test_ratio / (val_ratio + test_ratio)),
-                random_state=random_state,
-            )
-        else:
-            valRunIDs = remaining
-            testRunIDs = []
-
-    return trainRunIDs, valRunIDs, testRunIDs    
-
-##############################################################################################################################
-
-def load_copick_config(path: str):
-
-    if os.path.isfile(path):
-        root = copick.from_file(path)
-    else:
-        raise FileNotFoundError(f"Copick Config Path does not exist: {path}")
-    
-    return root
-
-##############################################################################################################################
-
-# Helper function to flatten and serialize nested parameters
-def flatten_params(params, parent_key=''):
-    flattened = {}
-    for key, value in params.items():
-        new_key = f"{parent_key}.{key}" if parent_key else key
-        if isinstance(value, dict):
-            flattened.update(flatten_params(value, new_key))
-        elif isinstance(value, list):
-            flattened[new_key] = ', '.join(map(str, value))  # Convert list to a comma-separated string
-        else:
-            flattened[new_key] = value
-    return flattened
-
-# Manually join specific lists into strings for inline display
-def prepare_for_inline_json(data):
-    for key in ["trainRunIDs", "valRunIDs", "testRunIDs"]:
-        if key in data['dataloader']:
-            data['dataloader'][key] = f"[{', '.join(map(repr, data['dataloader'][key]))}]"
-
-    for key in ['channels', 'strides']:
-        if key in data['model']:
-                data['model'][key] = f"[{', '.join(map(repr, data['model'][key]))}]"
-    return data
-
-def get_optimizer_parameters(trainer):
-
-    optimizer_parameters = {
-        'my_num_samples': trainer.num_samples,  
-        'val_interval': trainer.val_interval,
-        'lr': trainer.optimizer.param_groups[0]['lr'],
-        'optimizer': trainer.optimizer.__class__.__name__,
-        'metrics_function': trainer.metrics_function.__class__.__name__,
-        'loss_function': trainer.loss_function.__class__.__name__,
-    }
-
-    # Log Tversky Loss Parameters
-    if trainer.loss_function.__class__.__name__ == 'TverskyLoss':
-        optimizer_parameters['alpha'] = trainer.loss_function.alpha
-    elif trainer.loss_function.__class__.__name__ == 'FocalLoss':
-        optimizer_parameters['gamma'] = trainer.loss_function.gamma
-    elif trainer.loss_function.__class__.__name__ == 'WeightedFocalTverskyLoss':
-        optimizer_parameters['alpha'] = trainer.loss_function.alpha
-        optimizer_parameters['gamma'] = trainer.loss_function.gamma
-        optimizer_parameters['weight_tversky'] = trainer.loss_function.weight_tversky
-    elif trainer.loss_function.__class__.__name__ == 'FocalTverskyLoss':
-        optimizer_parameters['alpha'] = trainer.loss_function.alpha
-        optimizer_parameters['gamma'] = trainer.loss_function.gamma
-
-    return optimizer_parameters
-
-def save_parameters_to_yaml(model, trainer, dataloader, filename: str):
-
-    parameters = {
-        'model': model.get_model_parameters(),
-        'optimizer': get_optimizer_parameters(trainer),
-        'dataloader': dataloader.get_dataloader_parameters()
-    }
-
-    utils.save_parameters_yaml(parameters, filename)
-    print(f"Training Parameters saved to {filename}")
-
-def prepare_inline_results_json(results):
-    # Traverse the dictionary and format lists of lists as inline JSON
-    for key, value in results.items():
-        # Check if the value is a list of lists (like [[epoch, value], ...])
-        if isinstance(value, list) and all(isinstance(item, list) and len(item) == 2 for item in value):
-            # Format the list of lists as a single-line JSON string
-            results[key] = json.dumps(value)
-    return results    
-
-# Check to See if I'm Happy with This... Maybe Save as H5 File? 
-def save_results_to_json(results, filename: str):
-
-    results = prepare_inline_results_json(results)
-    with open(os.path.join(filename), "w") as json_file:
-        json.dump( results, json_file, indent=4 )
-    print(f"Training Results saved to {filename}")
-
-##############################################################################################################################
-
-# def save_parameters_to_json(model, trainer, dataloader, filename: str):
-
-#     parameters = {
-#         'model': model.get_model_parameters(),
-#         'optimizer': get_optimizer_parameters(trainer),
-#         'dataloader': dataloader.get_dataloader_parameters()
-#     }
-#     parameters = prepare_for_inline_json(parameters)
-
-#     with open(os.path.join(filename), "w") as json_file:
-#         json.dump( parameters, json_file, indent=4 )
-#     print(f"Training Parameters saved to {filename}")
-
-# def split_datasets(runIDs, 
-#                    train_ratio: float = 0.7, 
-#                    val_ratio: float = 0.15, 
-#                    test_ratio: float = 0.15, 
-#                    return_test_dataset: bool = True,
-#                    random_state: int = 42):
-#     """
-#     Splits a given dataset into three subsets: training, validation, and testing. The proportions
-#     of each subset are determined by the provided ratios, ensuring that they add up to 1. The
-#     function uses a fixed random state for reproducibility.
-
-#     Parameters:
-#     - runIDs: The complete dataset that needs to be split.
-#     - train_ratio: The proportion of the dataset to be used for training.
-#     - val_ratio: The proportion of the dataset to be used for validation.
-#     - test_ratio: The proportion of the dataset to be used for testing.
-
-#     Returns:
-#     - trainRunIDs: The subset of the dataset used for training.
-#     - valRunIDs: The subset of the dataset used for validation.
-#     - testRunIDs: The subset of the dataset used for testing.
-#     """
-
-#     # Ensure the ratios add up to 1
-#     assert train_ratio + val_ratio + test_ratio == 1.0, "Ratios must add up to 1.0"
-
-#     # First, split into train and remaining (30%)
-#     trainRunIDs, valRunIDs = train_test_split(runIDs, test_size=(1 - train_ratio), random_state=random_state)
-
-#     # (Optional) split the remaining into validation and test
-#     if return_test_dataset: 
-#         valRunIDs, testRunIDs = train_test_split(
-#             valRunIDs,
-#             test_size=(test_ratio / (val_ratio + test_ratio)),
-#             random_state=random_state,
-#         )
-#     else:
-#         testRunIDs = None
-
-#     return trainRunIDs, valRunIDs, testRunIDs