TwoPy 0.3.0__tar.gz → 0.3.2__tar.gz

{twopy-0.3.0 → twopy-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: twopy
-Version: 0.3.0
+Version: 0.3.2
 Summary: Two-photon imaging analysis tool with a napari interface.
 Author-email: Gustavo Madeira Santana <fromweb1@gumadeiras.com>
 License-Expression: MIT
@@ -30,7 +30,7 @@ Two-photon imaging analysis tool for the Clark Lab output format.
 
 twopy lets you open two-photon recordings, draw ROIs, plot responses in real time, process and analyze them, and save them.
 
-When you first load a recording, twopy converts it to a standardized HDF5 format. The converted format includes the aligned movie, mean image, stimulus tables, photodiode signals, and  recording metadata. Analysis and the GUI both work from the converted files, so the original source files remain separate from twopy's outputs. By default, converted recordings and interactive analysis outputs are cached locally under `~/.cache/twopy/recordings` for speed; saving ROIs and analysis publishes the changed output files back to the configured `analysis_output` location in the background.
+When you first load a recording, twopy converts it to HDF5 files that it owns. Those files include the aligned movie, mean image, stimulus tables, photodiode signals, and recording metadata. Analysis and the GUI both work from the converted files, so the original source files remain separate from twopy's outputs. By default, converted recordings and interactive analysis outputs are cached locally under `~/.cache/twopy/recordings` for speed; saving ROIs and analysis copies the changed output files back to the configured `analysis_output` location in the background.
 
 ## Install
 
@@ -48,7 +48,19 @@ Verify the install:
 micromamba run -n twopy python -c 'import twopy; print(twopy.__name__)'
 ```
 
-For a source checkout, copy `config.example.yml` to `config.yml` and edit the paths for your computer. `config.yml` stays local and is not tracked by git.
+Create the config template in your user folder:
+
+```sh
+twopy config setup
+```
+
+twopy prints the config path it wrote. On macOS and Linux the default is `~/.config/twopy/config.yml`; on Windows it is under `%APPDATA%\twopy\config.yml`. Edit that file with your database, data, and output paths before launching twopy. If you launch `twopy` before a config exists, twopy creates the same template, prints the path, and stops so you can edit it.
+
+Validate the active config and show its path and file contents:
+
+```sh
+twopy config
+```
 
 ## Start
 
@@ -57,6 +69,8 @@ micromamba activate twopy
 twopy
 ```
 
+`twopy` validates the config before opening napari. It does not require lab data folders or network drives to be mounted at launch.
+
 Open a source recording or converted recording directly:
 
 ```sh

{twopy-0.3.0 → twopy-0.3.2}/README.md

@@ -8,7 +8,7 @@ Two-photon imaging analysis tool for the Clark Lab output format.
 
 twopy lets you open two-photon recordings, draw ROIs, plot responses in real time, process and analyze them, and save them.
 
-When you first load a recording, twopy converts it to a standardized HDF5 format. The converted format includes the aligned movie, mean image, stimulus tables, photodiode signals, and  recording metadata. Analysis and the GUI both work from the converted files, so the original source files remain separate from twopy's outputs. By default, converted recordings and interactive analysis outputs are cached locally under `~/.cache/twopy/recordings` for speed; saving ROIs and analysis publishes the changed output files back to the configured `analysis_output` location in the background.
+When you first load a recording, twopy converts it to HDF5 files that it owns. Those files include the aligned movie, mean image, stimulus tables, photodiode signals, and recording metadata. Analysis and the GUI both work from the converted files, so the original source files remain separate from twopy's outputs. By default, converted recordings and interactive analysis outputs are cached locally under `~/.cache/twopy/recordings` for speed; saving ROIs and analysis copies the changed output files back to the configured `analysis_output` location in the background.
 
 ## Install
 
@@ -26,7 +26,19 @@ Verify the install:
 micromamba run -n twopy python -c 'import twopy; print(twopy.__name__)'
 ```
 
-For a source checkout, copy `config.example.yml` to `config.yml` and edit the paths for your computer. `config.yml` stays local and is not tracked by git.
+Create the config template in your user folder:
+
+```sh
+twopy config setup
+```
+
+twopy prints the config path it wrote. On macOS and Linux the default is `~/.config/twopy/config.yml`; on Windows it is under `%APPDATA%\twopy\config.yml`. Edit that file with your database, data, and output paths before launching twopy. If you launch `twopy` before a config exists, twopy creates the same template, prints the path, and stops so you can edit it.
+
+Validate the active config and show its path and file contents:
+
+```sh
+twopy config
+```
 
 ## Start
 
@@ -35,6 +47,8 @@ micromamba activate twopy
 twopy
 ```
 
+`twopy` validates the config before opening napari. It does not require lab data folders or network drives to be mounted at launch.
+
 Open a source recording or converted recording directly:
 
 ```sh

{twopy-0.3.0 → twopy-0.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "twopy"
-version = "0.3.0"
+version = "0.3.2"
 description = "Two-photon imaging analysis tool with a napari interface."
 readme = "README.md"
 requires-python = ">=3.13,<3.14"
@@ -37,7 +37,7 @@ Repository = "https://github.com/gumadeiras/twopy"
 where = ["src"]
 
 [tool.setuptools.package-data]
-twopy = ["calibrations/*.csv"]
+twopy = ["calibrations/*.csv", "config.example.yml"]
 
 [tool.ruff]
 line-length = 88

{twopy-0.3.0 → twopy-0.3.2}/src/twopy/__init__.py

@@ -94,15 +94,6 @@ from twopy.converted import (
     recording_frame_rate_hz,
     recording_hemisphere,
 )
-from twopy.napari import (
-    NapariRecordingView,
-    NapariSidebarWidgets,
-    add_twopy_magicgui_controls,
-    launch_napari,
-    open_recording_in_napari,
-    roi_label_image_from_layer,
-    save_napari_label_rois,
-)
 from twopy.photodiode_classification import (
     ClassifiedPhotodiodeEvent,
     ClassifiedStimulusTiming,
@@ -180,7 +171,6 @@ __all__ = [
     "AnalysisResponseComputation",
     "add_manual_roi_match_group",
     "analyze_recording_responses",
-    "add_twopy_magicgui_controls",
     "append_manual_roi_match_rows",
     "apply_motion_artifact_mask_to_delta_f_over_f",
     "BackgroundCorrectedRoiTraces",
@@ -227,7 +217,6 @@ __all__ = [
     "is_baseline_epoch_name",
     "KernelFitMethod",
     "KernelStimulusModality",
-    "launch_napari",
     "load_analysis_outputs",
     "load_response_map_data",
     "load_manual_fov_group_rows",
@@ -254,9 +243,6 @@ __all__ = [
     "remove_manual_roi_match_group",
     "replace_manual_roi_match_group",
     "map_stimulus_specific_column",
-    "NapariRecordingView",
-    "NapariSidebarWidgets",
-    "open_recording_in_napari",
     "pair_photodiode_events_to_imaging_frames",
     "PhotodiodeAlignment",
     "PhotodiodeDurationClass",
@@ -296,13 +282,11 @@ __all__ = [
     "RoiResponseTrial",
     "RoiSet",
     "RoiTraces",
-    "roi_label_image_from_layer",
     "roi_set_to_label_image",
     "save_analysis_outputs",
     "save_response_map_data",
     "save_manual_fov_group_rows",
     "save_manual_roi_match_rows",
-    "save_napari_label_rois",
     "save_roi_set",
     "select_baseline_frame_windows",
     "select_pixel_calibration_group",

{twopy-0.3.0 → twopy-0.3.2}/src/twopy/analysis/__init__.py

@@ -66,6 +66,13 @@ from twopy.analysis.response_maps import (
     load_response_map_data,
     save_response_map_data,
 )
+from twopy.analysis.response_plotting import (
+    EpochResponsePlotData,
+    ResponsePlotData,
+    filter_response_plot_data_rois,
+    response_plot_data_from_grouped,
+    response_plot_window_options_from_grouped,
+)
 from twopy.analysis.response_processing import (
     CorrelationFilterOptions,
     LowPassFilterOptions,
@@ -130,6 +137,7 @@ __all__ = [
     "DeltaFOverFOptions",
     "DeltaFOverFFitMode",
     "EpochResponseMap",
+    "EpochResponsePlotData",
     "EpochFrameWindow",
     "extract_background_corrected_roi_traces",
     "fit_recording_stimulus_kernels",
@@ -163,6 +171,7 @@ __all__ = [
     "ResponseMapData",
     "ResponseMapMode",
     "ResponseMapOptions",
+    "ResponsePlotData",
     "ResponseProcessingOptions",
     "RoiCorrelationScores",
     "RoiResponseSummary",
@@ -181,9 +190,12 @@ __all__ = [
     "save_manual_roi_match_rows",
     "select_baseline_frame_windows",
     "finite_mean_and_sem",
+    "filter_response_plot_data_rois",
     "split_traces_by_frame_windows",
     "StimulusKernelOptions",
     "SmoothingOptions",
+    "response_plot_data_from_grouped",
+    "response_plot_window_options_from_grouped",
     "summarize_epoch_roi_responses",
     "summarize_grouped_responses",
     "TimingSource",

{twopy-0.3.0 → twopy-0.3.2}/src/twopy/analysis/persistence.py

@@ -10,7 +10,9 @@ MATLAB/TIFF files or decide analysis parameters.
 
 import csv
 import json
-from collections.abc import Mapping, Sequence
+import os
+import tempfile
+from collections.abc import Callable, Iterator, Mapping, Sequence
 from dataclasses import dataclass
 from pathlib import Path
 
@@ -99,7 +101,8 @@ class LoadedAnalysisOutputs:
     """Analysis objects loaded from one twopy analysis HDF5 file.
 
     Inputs: one file written by ``save_analysis_outputs``.
-    Outputs: optional ROI, trace, dF/F, window, and response objects.
+    Outputs: optional ROI, trace, dF/F, window, response objects, and a warning
+        when an old file uses a setting twopy no longer uses.
 
     Groups are optional because scripts can save only the products they have
     computed. Missing groups load as ``None`` or an empty window tuple.
@@ -115,6 +118,7 @@ class LoadedAnalysisOutputs:
     response_processing_options: ResponseProcessingOptions | None
     normalization_factors: RoiNormalizationFactors | None
     correlation_scores: RoiCorrelationScores | None
+    load_warning: str | None = None
 
 
 def save_analysis_outputs(
@@ -144,7 +148,7 @@ def save_analysis_outputs(
         grouped_responses: Optional grouped trial responses.
         response_processing_options: Optional smoothing, low-pass, and
             correlation-QC settings used for these outputs.
-        normalization_factors: Optional per-ROI factors used for epoch-peak
+        normalization_factors: Optional per-ROI factors used for response-size
             normalization.
         correlation_scores: Optional ROI-level correlation QC scores.
         response_summary_trials_csv: Optional CSV path for one row per trial
@@ -157,8 +161,11 @@ def save_analysis_outputs(
         None.
 
     The HDF5 file stores arrays with gzip compression where arrays can be large.
-    The HDF5 file is the complete audit trail. CSV files duplicate response
-    time series in a spreadsheet-friendly layout for quick plotting and checks.
+    The HDF5 file is the complete audit trail. CSV files are derived exports for
+    quick spreadsheet plotting and checks, and are written after the HDF5 file.
+    Each output file is written through a temporary sibling path so a failed
+    write leaves the previous complete file in place; the HDF5 and CSV files are
+    not replaced as one multi-file transaction.
     """
     summary_outputs = (
         response_summary_trials_csv,
@@ -175,35 +182,40 @@ def save_analysis_outputs(
     if grouped_responses is not None:
         validate_grouped_roi_responses(grouped_responses)
 
-    output_path = path.expanduser()
-    output_path.parent.mkdir(parents=True, exist_ok=True)
-    with h5py.File(output_path, "w") as h5_file:
-        h5_file.attrs["twopy_format"] = ANALYSIS_OUTPUT_FILE_FORMAT
-        if roi_set is not None:
-            _write_roi_set(h5_file.create_group("roi_set"), roi_set)
-        if traces is not None:
-            _write_background_traces(h5_file.create_group("traces"), traces)
-        if dff is not None:
-            _write_dff(h5_file.create_group("dff"), dff)
-        if len(epoch_windows) > 0:
-            _write_epoch_windows(h5_file.create_group("epoch_windows"), epoch_windows)
-        if len(baseline_windows) > 0:
-            _write_frame_windows(
-                h5_file.create_group("baseline_windows"),
-                baseline_windows,
-            )
-        if grouped_responses is not None:
-            _write_grouped_responses(
-                h5_file.create_group("responses"),
-                grouped_responses,
-            )
-        if response_processing_options is not None:
-            write_response_processing_group(
-                h5_file.create_group("response_processing"),
-                options=response_processing_options,
-                normalization_factors=normalization_factors,
-                correlation_scores=correlation_scores,
-            )
+    def write_hdf5(output_path: Path) -> None:
+        """Write the analysis HDF5 contents to one already-created path."""
+        with h5py.File(output_path, "w") as h5_file:
+            h5_file.attrs["twopy_format"] = ANALYSIS_OUTPUT_FILE_FORMAT
+            if roi_set is not None:
+                _write_roi_set(h5_file.create_group("roi_set"), roi_set)
+            if traces is not None:
+                _write_background_traces(h5_file.create_group("traces"), traces)
+            if dff is not None:
+                _write_dff(h5_file.create_group("dff"), dff)
+            if len(epoch_windows) > 0:
+                _write_epoch_windows(
+                    h5_file.create_group("epoch_windows"),
+                    epoch_windows,
+                )
+            if len(baseline_windows) > 0:
+                _write_frame_windows(
+                    h5_file.create_group("baseline_windows"),
+                    baseline_windows,
+                )
+            if grouped_responses is not None:
+                _write_grouped_responses(
+                    h5_file.create_group("responses"),
+                    grouped_responses,
+                )
+            if response_processing_options is not None:
+                write_response_processing_group(
+                    h5_file.create_group("response_processing"),
+                    options=response_processing_options,
+                    normalization_factors=normalization_factors,
+                    correlation_scores=correlation_scores,
+                )
+
+    _write_file_atomically(path, write_hdf5)
 
     if grouped_responses is not None:
         if response_summary_trials_csv is not None:
@@ -218,6 +230,58 @@ def save_analysis_outputs(
             )
 
 
+def _write_file_atomically(
+    path: Path,
+    write_file: Callable[[Path], None],
+) -> None:
+    """Write a file beside its final path, then replace the old file.
+
+    Args:
+        path: Final destination path.
+        write_file: Callback that writes a complete file at the provided
+            temporary path.
+
+    Returns:
+        None.
+
+    Existing analysis files are useful audit records. Writing through a sibling
+    temporary path keeps a failed save from replacing the last complete output.
+    """
+    output_path = path.expanduser()
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    temp_descriptor, temp_name = tempfile.mkstemp(
+        prefix=f".{output_path.name}.",
+        suffix=".tmp",
+        dir=output_path.parent,
+    )
+    os.close(temp_descriptor)
+    temp_path = Path(temp_name)
+    try:
+        write_file(temp_path)
+        temp_path.chmod(_replacement_file_mode(output_path))
+        temp_path.replace(output_path)
+    except Exception:
+        temp_path.unlink(missing_ok=True)
+        raise
+
+
+def _replacement_file_mode(path: Path) -> int:
+    """Return the file mode an atomic replacement should keep.
+
+    Args:
+        path: Final destination path.
+
+    Returns:
+        Existing file permissions, or normal new-file permissions after the
+        process umask.
+    """
+    if path.exists():
+        return path.stat().st_mode & 0o777
+    current_umask = os.umask(0)
+    os.umask(current_umask)
+    return 0o666 & ~current_umask
+
+
 def load_analysis_outputs(path: Path) -> LoadedAnalysisOutputs:
     """Load twopy analysis outputs from HDF5.
 
@@ -282,6 +346,11 @@ def load_analysis_outputs(path: Path) -> LoadedAnalysisOutputs:
                 if response_processing is not None
                 else None
             ),
+            load_warning=(
+                response_processing.load_warning
+                if response_processing is not None
+                else None
+            ),
         )
 
 
@@ -297,25 +366,31 @@ def write_response_summary_trials_csv(
 
     Returns:
         None.
+
+    The CSV is written through a temporary sibling path so a failed write leaves
+    the previous complete file in place.
     """
-    output_path = path.expanduser()
-    output_path.parent.mkdir(parents=True, exist_ok=True)
     validate_grouped_roi_responses(grouped_responses)
     time_columns = _time_columns(grouped_responses.trials)
     fieldnames = (*_TRIAL_RESPONSE_CSV_BASE_COLUMNS, *time_columns)
-    with output_path.open("w", encoding="utf-8", newline="") as csv_file:
-        writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
-        writer.writeheader()
-        for trial in grouped_responses.trials:
-            for roi_index, roi_label in enumerate(grouped_responses.roi_labels):
-                writer.writerow(
-                    _trial_response_row(
-                        trial,
-                        roi_index=roi_index,
-                        roi_label=roi_label,
-                        time_columns=time_columns,
-                    ),
-                )
+
+    def write_csv(output_path: Path) -> None:
+        """Write trial-level rows to one complete CSV file."""
+        with output_path.open("w", encoding="utf-8", newline="") as csv_file:
+            writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
+            writer.writeheader()
+            for trial in grouped_responses.trials:
+                for roi_index, roi_label in enumerate(grouped_responses.roi_labels):
+                    writer.writerow(
+                        _trial_response_row(
+                            trial,
+                            roi_index=roi_index,
+                            roi_label=roi_label,
+                            time_columns=time_columns,
+                        ),
+                    )
+
+    _write_file_atomically(path, write_csv)
 
 
 def write_response_summary_grouped_csv(
@@ -330,17 +405,23 @@ def write_response_summary_grouped_csv(
 
     Returns:
         None.
+
+    The CSV is written through a temporary sibling path so a failed write leaves
+    the previous complete file in place.
     """
-    output_path = path.expanduser()
-    output_path.parent.mkdir(parents=True, exist_ok=True)
     validate_grouped_roi_responses(grouped_responses)
     time_columns = _time_columns(grouped_responses.trials)
     fieldnames = (*_GROUPED_RESPONSE_CSV_BASE_COLUMNS, *time_columns)
-    with output_path.open("w", encoding="utf-8", newline="") as csv_file:
-        writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
-        writer.writeheader()
-        for row in _grouped_response_rows(grouped_responses, time_columns):
-            writer.writerow(row)
+
+    def write_csv(output_path: Path) -> None:
+        """Write grouped rows to one complete CSV file."""
+        with output_path.open("w", encoding="utf-8", newline="") as csv_file:
+            writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
+            writer.writeheader()
+            for row in _iter_grouped_response_rows(grouped_responses, time_columns):
+                writer.writerow(row)
+
+    _write_file_atomically(path, write_csv)
 
 
 def _write_roi_set(group: h5py.Group, roi_set: RoiSet) -> None:
@@ -894,10 +975,10 @@ def _trial_response_row(
     return row
 
 
-def _grouped_response_rows(
+def _iter_grouped_response_rows(
     grouped: GroupedRoiResponses,
     time_columns: tuple[str, ...],
-) -> tuple[dict[str, str | int | float], ...]:
+) -> Iterator[dict[str, str | int | float]]:
     """Convert grouped trial responses to mean, SEM, and count CSV rows.
 
     Args:
@@ -905,12 +986,12 @@ def _grouped_response_rows(
         time_columns: Complete CSV time-column union.
 
     Returns:
-        Rows keyed by epoch, ROI, statistic, and relative timepoint.
+        Iterator over rows keyed by epoch, ROI, statistic, and relative
+        timepoint.
 
     Counts are per timepoint rather than one scalar because pre/post context or
     clipped recording edges can make some trials shorter than others.
     """
-    rows: list[dict[str, str | int | float]] = []
     for epoch_number, epoch_name, trials in _trials_by_epoch(grouped.trials):
         for roi_index, roi_label in enumerate(grouped.roi_labels):
             values_by_time = _values_by_time_column(
@@ -918,16 +999,13 @@ def _grouped_response_rows(
                 roi_index=roi_index,
                 time_columns=time_columns,
             )
-            rows.extend(
-                _epoch_roi_statistic_rows(
-                    epoch_number=epoch_number,
-                    epoch_name=epoch_name,
-                    roi_label=roi_label,
-                    values_by_time=values_by_time,
-                    time_columns=time_columns,
-                ),
+            yield from _epoch_roi_statistic_rows(
+                epoch_number=epoch_number,
+                epoch_name=epoch_name,
+                roi_label=roi_label,
+                values_by_time=values_by_time,
+                time_columns=time_columns,
             )
-    return tuple(rows)
 
 
 def _trials_by_epoch(