accusleepy 0.4.0__tar.gz → 0.6.0__tar.gz

{accusleepy-0.4.0 → accusleepy-0.6.0}/PKG-INFO

@@ -1,16 +1,16 @@
 Metadata-Version: 2.3
 Name: accusleepy
-Version: 0.4.0
+Version: 0.6.0
 Summary: Python implementation of AccuSleep
 License: GPL-3.0-only
 Author: Zeke Barger
 Author-email: zekebarger@gmail.com
-Requires-Python: >=3.10,<3.13
+Requires-Python: >=3.11,<3.14
 Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Requires-Dist: fastparquet (>=2024.11.0,<2025.0.0)
 Requires-Dist: joblib (>=1.4.2,<2.0.0)
 Requires-Dist: matplotlib (>=3.10.1,<4.0.0)
@@ -18,7 +18,7 @@ Requires-Dist: numpy (>=2.2.4,<3.0.0)
 Requires-Dist: pandas (>=2.2.3,<3.0.0)
 Requires-Dist: pillow (>=11.1.0,<12.0.0)
 Requires-Dist: pre-commit (>=4.2.0,<5.0.0)
-Requires-Dist: pyside6 (>=6.7.1,<6.8.0)
+Requires-Dist: pyside6 (>=6.9.0,<7.0.0)
 Requires-Dist: scipy (>=1.15.2,<2.0.0)
 Requires-Dist: toml (>=0.10.2,<0.11.0)
 Requires-Dist: torch (>=2.6.0,<3.0.0)
@@ -30,9 +30,17 @@ Description-Content-Type: text/markdown
 
 ## Description
 
-AccuSleePy is a python implementation of AccuSleep--a set of graphical user interfaces for scoring rodent
-sleep using EEG and EMG recordings. It offers several improvements over the original MATLAB version
-and is the only version that will be actively maintained.
+AccuSleePy is set of graphical user  interfaces for scoring rodent sleep
+using EEG and EMG recordings.
+It offers the following improvements over the MATLAB version (AccuSleep):
+
+- Up to 10 brain states can be configured through the user interface
+- Classification models can be trained through the user interface
+    - Model files contain useful metadata (brain state configuration,
+      epoch length, number of epochs)
+    - Models optimized for real-time scoring can be trained
+- Lists of recordings can be imported and exported for repeatable batch processing
+- Undo/redo functionality in the manual scoring interface
 
 If you use AccuSleep in your research, please cite our
 [publication](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0224642):
@@ -43,24 +51,56 @@ The data and models associated with AccuSleep are available at https://osf.io/py
 
 Please contact zekebarger (at) gmail (dot) com with any questions or comments about the software.
 
+
 ## Installation
 
 - (recommended) create a new virtual environment (using
 [venv](https://docs.python.org/3/library/venv.html),
 [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html),
-etc.) using python >=3.10,<3.13
+etc.) with python >=3.11,<3.14
 - (optional) if you have a CUDA device and want to speed up model training, [install PyTorch](https://pytorch.org/)
 - `pip install accusleepy`
 - (optional) download a classification model from https://osf.io/py5eb/ under /python_format/models/
 
+Note that upgrading or reinstalling the package will overwrite any changes
+to the [config file](accusleepy/config.json).
+
 ## Usage
 
 `python -m accusleepy` will open the primary interface.
 
+[Guide to the primary interface](accusleepy/gui/text/main_guide.md)
+
+[Guide to the manual scoring interface](accusleepy/gui/text/manual_scoring_guide.md)
+
+## Changelog
+
+- 0.6.0: Confidence scores can now be displayed and saved. Retraining your models is recommended
+    since the new calibration feature will make the confidence scores more accurate.
+- 0.5.0: Performance improvements
+- 0.4.5: Added support for python 3.13, **removed support for python 3.10.**
+- 0.4.4: Performance improvements
+- 0.4.3: Improved unit tests and user manuals
+- 0.4.0: Improved visuals and user manuals
+- 0.1.0-0.3.1: Early development versions
+
+## Screenshots
+
+Primary interface
+![AccuSleePy primary interface](accusleepy/gui/images/primary_window.png)
+
+Manual scoring interface
+![AccuSleePy manual scoring interface](accusleepy/gui/images/viewer_window.png)
+
 ## Acknowledgements
 
 We would like to thank [Franz Weber](https://www.med.upenn.edu/weberlab/) for creating an
-early version of the manual labeling interface.
+early version of the manual labeling interface. The code that
+creates spectrograms comes from the
+[Prerau lab](https://github.com/preraulab/multitaper_toolbox/blob/master/python/multitaper_spectrogram_python.py)
+with only minor modifications.
 Jim Bohnslav's [deepethogram](https://github.com/jbohnslav/deepethogram) served as an
 incredibly useful reference when reimplementing this project in python.
+The model calibration code added in version 0.6.0 comes from Geoff Pleiss'
+[temperature scaling repo](https://github.com/gpleiss/temperature_scaling).
 

accusleepy-0.6.0/README.md

@@ -0,0 +1,77 @@
+# AccuSleePy
+
+## Description
+
+AccuSleePy is set of graphical user  interfaces for scoring rodent sleep
+using EEG and EMG recordings.
+It offers the following improvements over the MATLAB version (AccuSleep):
+
+- Up to 10 brain states can be configured through the user interface
+- Classification models can be trained through the user interface
+    - Model files contain useful metadata (brain state configuration,
+      epoch length, number of epochs)
+    - Models optimized for real-time scoring can be trained
+- Lists of recordings can be imported and exported for repeatable batch processing
+- Undo/redo functionality in the manual scoring interface
+
+If you use AccuSleep in your research, please cite our
+[publication](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0224642):
+
+Barger, Z., Frye, C. G., Liu, D., Dan, Y., & Bouchard, K. E. (2019). Robust, automated sleep scoring by a compact neural network with distributional shift correction. *PLOS ONE, 14*(12), 1–18.
+
+The data and models associated with AccuSleep are available at https://osf.io/py5eb/
+
+Please contact zekebarger (at) gmail (dot) com with any questions or comments about the software.
+
+
+## Installation
+
+- (recommended) create a new virtual environment (using
+[venv](https://docs.python.org/3/library/venv.html),
+[conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html),
+etc.) with python >=3.11,<3.14
+- (optional) if you have a CUDA device and want to speed up model training, [install PyTorch](https://pytorch.org/)
+- `pip install accusleepy`
+- (optional) download a classification model from https://osf.io/py5eb/ under /python_format/models/
+
+Note that upgrading or reinstalling the package will overwrite any changes
+to the [config file](accusleepy/config.json).
+
+## Usage
+
+`python -m accusleepy` will open the primary interface.
+
+[Guide to the primary interface](accusleepy/gui/text/main_guide.md)
+
+[Guide to the manual scoring interface](accusleepy/gui/text/manual_scoring_guide.md)
+
+## Changelog
+
+- 0.6.0: Confidence scores can now be displayed and saved. Retraining your models is recommended
+    since the new calibration feature will make the confidence scores more accurate.
+- 0.5.0: Performance improvements
+- 0.4.5: Added support for python 3.13, **removed support for python 3.10.**
+- 0.4.4: Performance improvements
+- 0.4.3: Improved unit tests and user manuals
+- 0.4.0: Improved visuals and user manuals
+- 0.1.0-0.3.1: Early development versions
+
+## Screenshots
+
+Primary interface
+![AccuSleePy primary interface](accusleepy/gui/images/primary_window.png)
+
+Manual scoring interface
+![AccuSleePy manual scoring interface](accusleepy/gui/images/viewer_window.png)
+
+## Acknowledgements
+
+We would like to thank [Franz Weber](https://www.med.upenn.edu/weberlab/) for creating an
+early version of the manual labeling interface. The code that
+creates spectrograms comes from the
+[Prerau lab](https://github.com/preraulab/multitaper_toolbox/blob/master/python/multitaper_spectrogram_python.py)
+with only minor modifications.
+Jim Bohnslav's [deepethogram](https://github.com/jbohnslav/deepethogram) served as an
+incredibly useful reference when reimplementing this project in python.
+The model calibration code added in version 0.6.0 comes from Geoff Pleiss'
+[temperature scaling repo](https://github.com/gpleiss/temperature_scaling).

accusleepy-0.6.0/accusleepy/bouts.py

@@ -0,0 +1,142 @@
+import re
+from dataclasses import dataclass
+from operator import attrgetter
+
+import numpy as np
+
+
+@dataclass
+class Bout:
+    """Stores information about a brain state bout"""
+
+    length: int  # length, in number of epochs
+    start_index: int  # index where bout starts
+    end_index: int  # index where bout ends
+    surrounding_state: int  # brain state on both sides of the bout
+
+
+def find_last_adjacent_bout(sorted_bouts: list[Bout], bout_index: int) -> int:
+    """Find index of last consecutive same-length bout
+
+     When running the post-processing step that enforces a minimum duration
+     for brain state bouts, there is a special case when bouts below the
+     duration threshold occur consecutively. This function performs a
+     recursive search for the index of a bout at the end of such a sequence.
+     When initially called, bout_index will always be 0. If, for example, the
+     first three bouts in the list are consecutive, the function will return 2.
+
+    :param sorted_bouts: list of brain state bouts, sorted by start time
+    :param bout_index: index of the bout in question
+    :return: index of the last consecutive same-length bout
+    """
+    # if we're at the end of the bout list, stop
+    if bout_index == len(sorted_bouts) - 1:
+        return bout_index
+
+    # if there is an adjacent bout
+    if sorted_bouts[bout_index].end_index == sorted_bouts[bout_index + 1].start_index:
+        # look for more adjacent bouts using that one as a starting point
+        return find_last_adjacent_bout(sorted_bouts, bout_index + 1)
+    else:
+        return bout_index
+
+
+def enforce_min_bout_length(
+    labels: np.array, epoch_length: int | float, min_bout_length: int | float
+) -> np.array:
+    """Ensure brain state bouts meet the min length requirement
+
+    As a post-processing step for sleep scoring, we can require that any
+    bout (continuous period) of a brain state have a minimum duration.
+    This function sets any bout shorter than the minimum duration to the
+    surrounding brain state (if the states on the left and right sides
+    are the same). In the case where there are consecutive short bouts,
+    it either creates a transition at the midpoint or removes all short
+    bouts, depending on whether the number is even or odd. For example:
+    ...AAABABAAA...  -> ...AAAAAAAAA...
+    ...AAABABABBB... -> ...AAAAABBBBB...
+
+    :param labels: brain state labels (digits in the 0-9 range)
+    :param epoch_length: epoch length, in seconds
+    :param min_bout_length: minimum bout length, in seconds
+    :return: updated brain state labels
+    """
+    # if recording is very short, don't change anything
+    if labels.size < 3:
+        return labels
+
+    if epoch_length == min_bout_length:
+        return labels
+
+    # get minimum number of epochs in a bout
+    min_epochs = int(np.ceil(min_bout_length / epoch_length))
+    # get set of states in the labels
+    brain_states = set(labels.tolist())
+
+    while True:  # so true
+        # convert labels to a string for regex search
+        # There is probably a regex that can find all patterns like ab+a
+        # without consuming each "a" but I haven't found it :(
+        label_string = "".join(labels.astype(str))
+
+        bouts = list()
+
+        for state in brain_states:
+            for other_state in brain_states:
+                if state == other_state:
+                    continue
+                # get start and end indices of each bout
+                expression = (
+                    f"(?<={other_state}){state}{{1,{min_epochs - 1}}}(?={other_state})"
+                )
+                matches = re.finditer(expression, label_string)
+                spans = [match.span() for match in matches]
+
+                # if some bouts were found
+                for span in spans:
+                    bouts.append(
+                        Bout(
+                            length=span[1] - span[0],
+                            start_index=span[0],
+                            end_index=span[1],
+                            surrounding_state=other_state,
+                        )
+                    )
+
+        if len(bouts) == 0:
+            break
+
+        # only keep the shortest bouts
+        min_length_in_list = np.min([bout.length for bout in bouts])
+        bouts = [i for i in bouts if i.length == min_length_in_list]
+        # sort by start index
+        sorted_bouts = sorted(bouts, key=attrgetter("start_index"))
+
+        while len(sorted_bouts) > 0:
+            # get row index of latest adjacent bout (of same length)
+            last_adjacent_bout_index = find_last_adjacent_bout(sorted_bouts, 0)
+            # if there's an even number of adjacent bouts
+            if (last_adjacent_bout_index + 1) % 2 == 0:
+                midpoint = sorted_bouts[
+                    round((last_adjacent_bout_index + 1) / 2)
+                ].start_index
+                labels[sorted_bouts[0].start_index : midpoint] = sorted_bouts[
+                    0
+                ].surrounding_state
+                labels[midpoint : sorted_bouts[last_adjacent_bout_index].end_index] = (
+                    sorted_bouts[last_adjacent_bout_index].surrounding_state
+                )
+            else:
+                labels[
+                    sorted_bouts[0].start_index : sorted_bouts[
+                        last_adjacent_bout_index
+                    ].end_index
+                ] = sorted_bouts[0].surrounding_state
+
+            # delete the bouts we just fixed
+            if last_adjacent_bout_index == len(sorted_bouts) - 1:
+                sorted_bouts = []
+            else:
+                sorted_bouts = sorted_bouts[(last_adjacent_bout_index + 1) :]
+
+    return labels

{accusleepy-0.4.0 → accusleepy-0.6.0}/accusleepy/classification.py

@@ -61,13 +61,31 @@ def get_device():
     )
 
 
-def train_model(
+def create_dataloader(
+    annotations_file: str, img_dir: str, shuffle: bool = True
+) -> DataLoader:
+    """Create DataLoader for a dataset of training or calibration images
+
+    :param annotations_file: file with information on each training image
+    :param img_dir: training image location
+    :param shuffle: reshuffle data for every epoch
+    :return: DataLoader for the data
+
+    """
+    image_dataset = AccuSleepImageDataset(
+        annotations_file=annotations_file,
+        img_dir=img_dir,
+    )
+    return DataLoader(image_dataset, batch_size=BATCH_SIZE, shuffle=shuffle)
+
+
+def train_ssann(
     annotations_file: str,
     img_dir: str,
     mixture_weights: np.array,
     n_classes: int,
 ) -> SSANN:
-    """Train a classification model for sleep scoring
+    """Train a SSANN classification model for sleep scoring
 
     :param annotations_file: file with information on each training image
     :param img_dir: training image location
@@ -75,11 +93,9 @@ def train_model(
     :param n_classes: number of classes the model will learn
     :return: trained Sleep Scoring Artificial Neural Network model
     """
-    training_data = AccuSleepImageDataset(
-        annotations_file=annotations_file,
-        img_dir=img_dir,
+    train_dataloader = create_dataloader(
+        annotations_file=annotations_file, img_dir=img_dir
     )
-    train_dataloader = DataLoader(training_data, batch_size=BATCH_SIZE, shuffle=True)
 
     device = get_device()
     model = SSANN(n_classes=n_classes)
@@ -130,7 +146,7 @@ def score_recording(
     :param epoch_length: epoch length, in seconds
     :param epochs_per_img: number of epochs for the model to consider
     :param brain_state_set: set of brain state options
-    :return: brain state labels
+    :return: brain state labels, confidence scores
     """
     # prepare model
     device = get_device()
@@ -158,10 +174,12 @@ def score_recording(
     # perform classification
     with torch.no_grad():
         outputs = model(images)
-        _, predicted = torch.max(outputs, 1)
+        logits, predicted = torch.max(outputs, 1)
 
     labels = brain_state_set.convert_class_to_digit(predicted.cpu().numpy())
-    return labels
+    confidence_scores = 1 / (1 + np.e ** (-logits.cpu().numpy()))
+
+    return labels, confidence_scores
 
 
 def example_real_time_scoring_function(

{accusleepy-0.4.0 → accusleepy-0.6.0}/accusleepy/config.json

@@ -19,5 +19,6 @@
             "frequency": 0.55
         }
     ],
-    "default_epoch_length": 2.5
+    "default_epoch_length": 2.5,
+    "save_confidence_setting": true
 }

{accusleepy-0.4.0 → accusleepy-0.6.0}/accusleepy/constants.py

@@ -8,6 +8,7 @@ EEG_COL = "eeg"
 EMG_COL = "emg"
 # label file columns
 BRAIN_STATE_COL = "brain_state"
+CONFIDENCE_SCORE_COL = "confidence_score"
 
 
 # really don't change these
@@ -37,3 +38,9 @@ RECORDING_LIST_NAME = "recording_list"
 RECORDING_LIST_FILE_TYPE = ".json"
 # key for default epoch length in config
 DEFAULT_EPOCH_LENGTH_KEY = "default_epoch_length"
+# key used for default confidence score behavior in config
+DEFAULT_CONFIDENCE_SETTING_KEY = "save_confidence_setting"
+# filename used to store info about training image datasets
+ANNOTATIONS_FILENAME = "annotations.csv"
+# filename for annotation file for the calibration set
+CALIBRATION_ANNOTATION_FILENAME = "calibration_set.csv"

{accusleepy-0.4.0 → accusleepy-0.6.0}/accusleepy/fileio.py

@@ -4,13 +4,14 @@ from dataclasses import dataclass
 
 import numpy as np
 import pandas as pd
-import torch
 from PySide6.QtWidgets import QListWidgetItem
 
 from accusleepy.brain_state_set import BRAIN_STATES_KEY, BrainState, BrainStateSet
 from accusleepy.constants import (
     BRAIN_STATE_COL,
+    CONFIDENCE_SCORE_COL,
     CONFIG_FILE,
+    DEFAULT_CONFIDENCE_SETTING_KEY,
     DEFAULT_EPOCH_LENGTH_KEY,
     EEG_COL,
     EMG_COL,
@@ -19,7 +20,6 @@ from accusleepy.constants import (
     RECORDING_LIST_NAME,
     UNDEFINED_LABEL,
 )
-from accusleepy.models import SSANN
 
 
 @dataclass
@@ -46,57 +46,6 @@ def load_calibration_file(filename: str) -> (np.array, np.array):
     return mixture_means, mixture_sds
 
 
-def save_model(
-    model: SSANN,
-    filename: str,
-    epoch_length: int | float,
-    epochs_per_img: int,
-    model_type: str,
-    brain_state_set: BrainStateSet,
-) -> None:
-    """Save classification model and its metadata
-
-    :param model: classification model
-    :param epoch_length: epoch length used when training the model
-    :param epochs_per_img: number of epochs in each model input
-    :param model_type: default or real-time
-    :param brain_state_set: set of brain state options
-    :param filename: filename
-    """
-    state_dict = model.state_dict()
-    state_dict.update({"epoch_length": epoch_length})
-    state_dict.update({"epochs_per_img": epochs_per_img})
-    state_dict.update({"model_type": model_type})
-    state_dict.update(
-        {BRAIN_STATES_KEY: brain_state_set.to_output_dict()[BRAIN_STATES_KEY]}
-    )
-
-    torch.save(state_dict, filename)
-
-
-def load_model(filename: str) -> tuple[SSANN, int | float, int, str, dict]:
-    """Load classification model and its metadata
-
-    :param filename: filename
-    :return: model, epoch length used when training the model,
-        number of epochs in each model input, model type
-        (default or real-time), set of brain state options
-        used when training the model
-    """
-    state_dict = torch.load(
-        filename, weights_only=True, map_location=torch.device("cpu")
-    )
-    epoch_length = state_dict.pop("epoch_length")
-    epochs_per_img = state_dict.pop("epochs_per_img")
-    model_type = state_dict.pop("model_type")
-    brain_states = state_dict.pop(BRAIN_STATES_KEY)
-    n_classes = len([b for b in brain_states if b["is_scored"]])
-
-    model = SSANN(n_classes=n_classes)
-    model.load_state_dict(state_dict)
-    return model, epoch_length, epochs_per_img, model_type, brain_states
-
-
 def load_csv_or_parquet(filename: str) -> pd.DataFrame:
     """Load a csv or parquet file as a dataframe
 
@@ -125,49 +74,70 @@ def load_recording(filename: str) -> (np.array, np.array):
     return eeg, emg
 
 
-def load_labels(filename: str) -> np.array:
-    """Load file of brain state labels
+def load_labels(filename: str) -> (np.array, np.array):
+    """Load file of brain state labels and confidence scores
 
     :param filename: filename
-    :return: array of brain state labels
+    :return: array of brain state labels and, optionally, array of confidence scores
     """
     df = load_csv_or_parquet(filename)
-    return df[BRAIN_STATE_COL].values
+    if CONFIDENCE_SCORE_COL in df.columns:
+        return df[BRAIN_STATE_COL].values, df[CONFIDENCE_SCORE_COL].values
+    else:
+        return df[BRAIN_STATE_COL].values, None
 
 
-def save_labels(labels: np.array, filename: str) -> None:
+def save_labels(
+    labels: np.array, filename: str, confidence_scores: np.array = None
+) -> None:
     """Save brain state labels to file
 
     :param labels: brain state labels
     :param filename: filename
+    :param confidence_scores: optional confidence scores
     """
-    pd.DataFrame({BRAIN_STATE_COL: labels}).to_csv(filename, index=False)
+    if confidence_scores is not None:
+        pd.DataFrame(
+            {BRAIN_STATE_COL: labels, CONFIDENCE_SCORE_COL: confidence_scores}
+        ).to_csv(filename, index=False)
+    else:
+        pd.DataFrame({BRAIN_STATE_COL: labels}).to_csv(filename, index=False)
 
 
-def load_config() -> tuple[BrainStateSet, int | float]:
+def load_config() -> tuple[BrainStateSet, int | float, bool]:
     """Load configuration file with brain state options
 
-    :return: set of brain state options and default epoch length
+    :return: set of brain state options, other settings
     """
     with open(
         os.path.join(os.path.dirname(os.path.abspath(__file__)), CONFIG_FILE), "r"
     ) as f:
         data = json.load(f)
-    return BrainStateSet(
-        [BrainState(**b) for b in data[BRAIN_STATES_KEY]], UNDEFINED_LABEL
-    ), data[DEFAULT_EPOCH_LENGTH_KEY]
+
+    return (
+        BrainStateSet(
+            [BrainState(**b) for b in data[BRAIN_STATES_KEY]], UNDEFINED_LABEL
+        ),
+        data[DEFAULT_EPOCH_LENGTH_KEY],
+        data.get(DEFAULT_CONFIDENCE_SETTING_KEY, True),
+    )
 
 
 def save_config(
-    brain_state_set: BrainStateSet, default_epoch_length: int | float
+    brain_state_set: BrainStateSet,
+    default_epoch_length: int | float,
+    save_confidence_setting: bool,
 ) -> None:
     """Save configuration of brain state options to json file
 
     :param brain_state_set: set of brain state options
     :param default_epoch_length: epoch length to use when the GUI starts
+    :param save_confidence_setting: whether the option to save confidence
+        scores should be True by default
     """
     output_dict = brain_state_set.to_output_dict()
     output_dict.update({DEFAULT_EPOCH_LENGTH_KEY: default_epoch_length})
+    output_dict.update({DEFAULT_CONFIDENCE_SETTING_KEY: save_confidence_setting})
     with open(
         os.path.join(os.path.dirname(os.path.abspath(__file__)), CONFIG_FILE), "w"
     ) as f: