physioex 1.0.0.dev4__tar.gz → 1.0.3__tar.gz

{physioex-1.0.0.dev4 → physioex-1.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: physioex
-Version: 1.0.0.dev4
+Version: 1.0.3
 Summary: A python package for explainable sleep staging via deep learning
 Author-email: Guido Gagliardi <guido.gagliardi@phd.unipi.it>
 License: MIT License
@@ -31,6 +31,7 @@ Project-URL: Repository, https://github.com/guidogagl/physioex
 Project-URL: Issues, https://github.com/guidogagl/physioex/issues
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: tensorboard
 Requires-Dist: boto3
 Requires-Dist: botocore
 Requires-Dist: braindecode
@@ -61,6 +62,7 @@ Requires-Dist: torch
 Requires-Dist: torchmetrics
 Requires-Dist: tqdm
 Requires-Dist: wfdb
+Requires-Dist: gdown
 
 <div style = "text-align: center;">
 <img src="https://raw.githubusercontent.com/guidogagl/physioex/refs/heads/main/docs/assets/images/logo.svg" width = "250px", alt="PhysioEx Logo">
@@ -82,17 +84,32 @@ With PhysioEx you can simulate a state-of-the-art experiment just running the `t
 - [Chambon2018](https://ieeexplore.ieee.org/document/8307462) model for sleep stage classification ( raw time series as input).
 - [TinySleepNet](https://github.com/akaraspt/tinysleepnet) model for sleep stage classification (raw time series as input).
 - [SeqSleepNet](https://arxiv.org/pdf/1809.10932.pdf) model for sleep stage classification (time-frequency images as input).
+- [SleepTransformer](https://arxiv.org/pdf/2105.11043) model for sleep stage classification (time-frequency images as input).
 
 ## Supported datasets
 
-- [SHHS (Sleep Heart Health Study)](https://sleepdata.org/datasets/shhs): A multi-center cohort study designed to investigate the cardiovascular consequences of sleep-disordered breathing.
-- [MROS (MrOS Sleep Study)](https://sleepdata.org/datasets/mros): A study focusing on the outcomes of sleep disorders in older men.
-- [MESA (Multi-Ethnic Study of Atherosclerosis)](https://sleepdata.org/datasets/mesa): A study examining the prevalence, correlates, and progression of subclinical cardiovascular disease.
-- [DCSM (Dreem Challenge Sleep Monitoring)](https://physionet.org/content/dreem/1.0.0/): A dataset from the Dreem Challenge for automatic sleep staging.
-- [MASS (Montreal Archive of Sleep Studies)](https://massdb.herokuapp.com/en/): A comprehensive collection of polysomnographic sleep recordings.
-- [HMC (Home Monitoring of Cardiorespiratory Health)](https://physionet.org/content/hmc-kinematics/1.0.0/): A dataset for the study of cardiorespiratory health using home monitoring devices.
+### Publicly Available:
+
+For these datasets PhysioEx will take also to download the data with the `preprocess` command.
+
+- [Sleep-EDF(78)](https://physionet.org/content/sleep-edfx/1.0.0/), The sleep-edf database contains 197 whole-night PolySomnoGraphic sleep recordings, containing EEG, EOG, chin EMG, and event markers.
+- [HMC (Haaglanden Medisch Centrum)](https://physionet.org/content/hmc-sleep-staging/1.1/), is a collection of 151 whole-night PSG recordings from 85 men and 66 women, gathered at the Haaglanden Medisch Centrum sleep center. The PSG data includes 4 EEG channels (F4/M1, C4/M1, O2/M1, and C3/M2), two EOG channels (E1/M2 and E2/M2), and one bipolar chin EMG, with all signals sampled at 256 Hz.
+- [DCSM (Danish Center for Sleep Medicine)](https://erda.ku.dk/public/archives/db553715ecbe1f3ac66c1dc569826eef/published-archive.html), is a collection of 255 randomly selected and fully anonymized overnight lab-based PSG recordings from patients seeking diagnosis for non-specific sleep-related disorders at the DCSM. The PSG setup included EEG, EOG, and EMG channels, all sampled at 256 Hz.
+
+
+### [NSSR](https://sleepdata.org) Datasets 
+
+These datasets can be easily get from the NSSR archive. Once downloaded in your `data_folder`, place them into the folder `data_folder/dataset_name/` directory with name `dataset_raw`. Then you can run the `preprocess -d dataset_name -df data_folder` command to make the data readable by PhysioEx. 
+
+- [SHHS (Sleep Heart Health Study)](https://sleepdata.org/datasets/shhs), is a multi-center cohort study designed to investigate the cardiovascular and other consequences of sleep-disordered breathing. At visit 1, it included 5,793 participants aged 40 years or older. PSG recordings were typically conducted in the subjects' homes by trained and certified technicians. The recording montage included C3/A2 and C4/A1 EEGs sampled at 125 Hz, right and left EOGs sampled at 50 Hz, and a bipolar submental EMG sampled at 125 Hz.
+- [MESA (Multi-Ethnic Study of Atherosclerosis)](https://sleepdata.org/datasets/mesa), is a multi-center prospective study of 2.237 ethnically diverse men and women aged 45-84 from six communities in the United States. PSGs recordings were obtained using in-home settings including central C4-M1 EEG, bilateral EOG and chin EMG sampled at 256Hz. PSGs were scored by one of 3 MESA certified, registered polysomnologists.
+- [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
+- [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
+
+### Others
+
+- [MASS (Montreal Archive of Sleep Studies)](http://ceams-carsm.ca/mass/), is an open-access collaborative database containing laboratory-based PSG recordings. It includes 200 complete nights recorded from 97 men and 103 women, aged 18 to 76 years. All recordings have a sampling frequency of 256 Hz and feature an EEG montage of 4–20 channels, along with standard EOG and EMG
 
-For the public available datasets ( DCSM, HMC ) PhysioEx takes care of automatically download the data thanks to the `preprocess` command. The other datasets needs to be acquired first ( mostly on [NSSR](https://sleepdata.org) ) and then fetched by PhysioEx via the `preprocess` command.
 
 ## Installation guidelines
 
@@ -104,14 +121,8 @@ For the public available datasets ( DCSM, HMC ) PhysioEx takes care of automatic
     conda install pip
     pip install --upgrade pip  # On Windows, use `venv\Scripts\activate`
 ```
-### Install via pip
 
-1. **Install PhysioEx from PyPI:**
-```bash
-pip install physioex
-```
-
-### Install from source
+### Install from source ( Recommended )
 1. **Clone the Repository:**
 ```bash
    git clone https://github.com/guidogagl/physioex.git
@@ -122,3 +133,23 @@ pip install physioex
 ```bash
     pip install -e .
 ```
+
+### Install via pip
+
+1. **Install PhysioEx from PyPI:**
+```bash
+pip install physioex
+```
+
+Note: the github version of the library is kept updated weekly, the PiPy version may be outdated depending on the last commit of the github version. We recommend to use the github version if possible.  
+
+## Cite Us!
+```bib
+@article{10.1088/1361-6579/adaf73,
+	author={Gagliardi, Guido and Alfeo, Luca and Cimino, Mario G C A and Valenza, Gaetano and De Vos, Maarten},
+	title={PhysioEx, a new Python library for explainable sleep staging through deep learning},
+	journal={Physiological Measurement},
+	url={http://iopscience.iop.org/article/10.1088/1361-6579/adaf73},
+	year={2025},
+}
+```

physioex-1.0.3/README.md

@@ -0,0 +1,89 @@
+<div style = "text-align: center;">
+<img src="https://raw.githubusercontent.com/guidogagl/physioex/refs/heads/main/docs/assets/images/logo.svg" width = "250px", alt="PhysioEx Logo">
+
+<h1> PhysioEx </h1>
+</div>
+
+![Python Version](https://img.shields.io/badge/python-3.7%2B-blue)
+![PyPI Version](https://badge.fury.io/py/physioex.svg)
+
+**PhysioEx ( Physiological Signal Explainer )** is a versatile python library tailored for building, training, and explaining deep learning models for physiological signal analysis. 
+
+The main purpose of the library is to propose a standard and fast methodology to train and evalutate state-of-the-art deep learning architectures for physiological signal analysis, to shift the attention from the architecture building task to the explainability task. 
+
+With PhysioEx you can simulate a state-of-the-art experiment just running the `train`, `test_model`  and `finetune` commands; evaluating and saving the trained model; and start focusing on the explainability task! 
+
+## Supported deep learning architectures
+
+- [Chambon2018](https://ieeexplore.ieee.org/document/8307462) model for sleep stage classification ( raw time series as input).
+- [TinySleepNet](https://github.com/akaraspt/tinysleepnet) model for sleep stage classification (raw time series as input).
+- [SeqSleepNet](https://arxiv.org/pdf/1809.10932.pdf) model for sleep stage classification (time-frequency images as input).
+- [SleepTransformer](https://arxiv.org/pdf/2105.11043) model for sleep stage classification (time-frequency images as input).
+
+## Supported datasets
+
+### Publicly Available:
+
+For these datasets PhysioEx will take also to download the data with the `preprocess` command.
+
+- [Sleep-EDF(78)](https://physionet.org/content/sleep-edfx/1.0.0/), The sleep-edf database contains 197 whole-night PolySomnoGraphic sleep recordings, containing EEG, EOG, chin EMG, and event markers.
+- [HMC (Haaglanden Medisch Centrum)](https://physionet.org/content/hmc-sleep-staging/1.1/), is a collection of 151 whole-night PSG recordings from 85 men and 66 women, gathered at the Haaglanden Medisch Centrum sleep center. The PSG data includes 4 EEG channels (F4/M1, C4/M1, O2/M1, and C3/M2), two EOG channels (E1/M2 and E2/M2), and one bipolar chin EMG, with all signals sampled at 256 Hz.
+- [DCSM (Danish Center for Sleep Medicine)](https://erda.ku.dk/public/archives/db553715ecbe1f3ac66c1dc569826eef/published-archive.html), is a collection of 255 randomly selected and fully anonymized overnight lab-based PSG recordings from patients seeking diagnosis for non-specific sleep-related disorders at the DCSM. The PSG setup included EEG, EOG, and EMG channels, all sampled at 256 Hz.
+
+
+### [NSSR](https://sleepdata.org) Datasets 
+
+These datasets can be easily get from the NSSR archive. Once downloaded in your `data_folder`, place them into the folder `data_folder/dataset_name/` directory with name `dataset_raw`. Then you can run the `preprocess -d dataset_name -df data_folder` command to make the data readable by PhysioEx. 
+
+- [SHHS (Sleep Heart Health Study)](https://sleepdata.org/datasets/shhs), is a multi-center cohort study designed to investigate the cardiovascular and other consequences of sleep-disordered breathing. At visit 1, it included 5,793 participants aged 40 years or older. PSG recordings were typically conducted in the subjects' homes by trained and certified technicians. The recording montage included C3/A2 and C4/A1 EEGs sampled at 125 Hz, right and left EOGs sampled at 50 Hz, and a bipolar submental EMG sampled at 125 Hz.
+- [MESA (Multi-Ethnic Study of Atherosclerosis)](https://sleepdata.org/datasets/mesa), is a multi-center prospective study of 2.237 ethnically diverse men and women aged 45-84 from six communities in the United States. PSGs recordings were obtained using in-home settings including central C4-M1 EEG, bilateral EOG and chin EMG sampled at 256Hz. PSGs were scored by one of 3 MESA certified, registered polysomnologists.
+- [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
+- [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
+
+### Others
+
+- [MASS (Montreal Archive of Sleep Studies)](http://ceams-carsm.ca/mass/), is an open-access collaborative database containing laboratory-based PSG recordings. It includes 200 complete nights recorded from 97 men and 103 women, aged 18 to 76 years. All recordings have a sampling frequency of 256 Hz and feature an EEG montage of 4–20 channels, along with standard EOG and EMG
+
+
+## Installation guidelines
+
+### Create a Virtual Environment (Optional but Recommended)
+
+```bash
+    conda create -n physioex python==3.10
+    conda activate physioex
+    conda install pip
+    pip install --upgrade pip  # On Windows, use `venv\Scripts\activate`
+```
+
+### Install from source ( Recommended )
+1. **Clone the Repository:**
+```bash
+   git clone https://github.com/guidogagl/physioex.git
+   cd physioex
+```
+
+2. **Install Dependencies and Package in Development Mode**
+```bash
+    pip install -e .
+```
+
+### Install via pip
+
+1. **Install PhysioEx from PyPI:**
+```bash
+pip install physioex
+```
+
+Note: the github version of the library is kept updated weekly, the PiPy version may be outdated depending on the last commit of the github version. We recommend to use the github version if possible.  
+
+## Cite Us!
+```bib
+@article{10.1088/1361-6579/adaf73,
+	author={Gagliardi, Guido and Alfeo, Luca and Cimino, Mario G C A and Valenza, Gaetano and De Vos, Maarten},
+	title={PhysioEx, a new Python library for explainable sleep staging through deep learning},
+	journal={Physiological Measurement},
+	url={http://iopscience.iop.org/article/10.1088/1361-6579/adaf73},
+	year={2025},
+}
+```

physioex-1.0.3/physioex/data/datamodule.py

@@ -0,0 +1,109 @@
+import os
+from typing import Callable, List, Union
+
+import pytorch_lightning as pl
+from torch.utils.data import DataLoader, DistributedSampler, Subset, SubsetRandomSampler
+
+from physioex.data.dataset import PhysioExDataset
+
+
+class PhysioExDataModule(pl.LightningDataModule):
+    def __init__(
+        self,
+        datasets: Union[List[str], PhysioExDataset],
+        batch_size: int = 32,
+        preprocessing: str = "raw",
+        selected_channels: List[int] = ["EEG"],
+        sequence_length: int = 21,
+        target_transform: Callable = None,
+        task: str = "sleep",
+        folds: Union[int, List[int]] = -1,
+        data_folder: str = None,
+        evaluate_on_whole_night: bool = False,
+        num_nodes : int = 1,
+        num_workers: int = os.cpu_count(),
+    ):
+        super().__init__()
+
+        self.datasets_id = datasets
+        self.num_workers = num_workers
+
+        if isinstance(datasets, list):
+            self.dataset = PhysioExDataset(
+                datasets=datasets,
+                preprocessing=preprocessing,
+                selected_channels=selected_channels,
+                sequence_length=sequence_length,
+                target_transform=target_transform,
+                data_folder=data_folder,
+                task=task,
+            )
+            
+            if evaluate_on_whole_night:
+                self.eval_dataset = PhysioExDataset(
+                datasets=datasets,
+                preprocessing=preprocessing,
+                selected_channels=selected_channels,
+                sequence_length=-1,
+                target_transform=target_transform,
+                data_folder=data_folder,
+                task=task,
+            )
+            else:
+                self.eval_dataset = self.dataset
+            
+        elif isinstance(datasets, PhysioExDataset):
+            self.dataset = datasets
+            self.eval_dataset = datasets
+        else:
+            raise ValueError("ERR: datasets should be a list or a PhysioExDataset")
+
+        self.batch_size = batch_size
+
+        if isinstance(folds, int):
+            self.dataset.split(folds)
+        else:
+            assert len(folds) == len(
+                datasets
+            ), "ERR: folds and datasets should have the same length"
+            for i, fold in enumerate(folds):
+                self.dataset.split(fold, i)
+
+        train_idx, _, _ = self.dataset.get_sets()
+        _, valid_idx, test_idx = self.eval_dataset.get_sets()
+
+        self.train_dataset = Subset(self.dataset, train_idx)
+        self.valid_dataset = Subset(self.eval_dataset, valid_idx)
+        self.test_dataset = Subset(self.eval_dataset, test_idx)
+        
+        self.eown = evaluate_on_whole_night
+
+    def train_dataloader(self):
+        """
+        Returns the DataLoader for the training dataset.
+
+        Returns:
+            DataLoader: DataLoader for the training dataset.
+        """
+        return DataLoader(
+            self.train_dataset,
+            batch_size=self.batch_size,
+            shuffle=True,
+            num_workers=self.num_workers,
+        )
+
+    def val_dataloader(self):
+        return DataLoader(
+            self.valid_dataset,
+            batch_size=self.batch_size if not self.eown else 1,
+            shuffle=False,
+            num_workers=self.num_workers,
+        )
+
+    def test_dataloader(self):
+        return DataLoader(
+            self.test_dataset,
+            batch_size=self.batch_size if not self.eown else 1,
+            shuffle=False,
+            num_workers=self.num_workers,
+        )

{physioex-1.0.0.dev4 → physioex-1.0.3}/physioex/data/datareader.py

@@ -24,6 +24,8 @@ class Reader(ABC):
         pass
 
 
+
+
 class MemmapReader(Reader):
     # this object abstracts the reading of subject data from memmap files for a specific dataset
     def __init__(
@@ -59,7 +61,19 @@ class MemmapReader(Reader):
         num_windows = self.table["num_windows"].values
         subjects_id = self.table["subject_id"].values
 
-        self.len = int(np.sum(self.table["num_windows"].values - self.L + 1))
+        if self.L > np.max(num_windows):
+            logger.warning(
+                f"Sequence length {self.L} is greater than the max number of windows {np.max(num_windows)} for dataset {dataset}."
+            )
+
+        self.len = num_windows - self.L
+
+        neg = np.where(self.len < 0)[0]
+        if len(neg) > 0:
+            self.len[neg] = 0
+
+        self.len = int(np.sum(self.len + 1))
+
         self.subject_idx, self.relative_idx, self.windows_index = build_index(
             num_windows, subjects_id, self.L
         )
@@ -71,7 +85,7 @@ class MemmapReader(Reader):
     def __len__(self):
         return self.len
 
-    def __getitem__(self, idx):
+    def get_signal(self, idx):
         idx = idx - self.offset
 
         relative_id = self.relative_idx[idx]
@@ -79,20 +93,136 @@ class MemmapReader(Reader):
         num_windows = self.windows_index[subject_id]
 
         input_shape = tuple([num_windows] + self.input_shape)
-        labels_shape = (num_windows,)
 
         data_path = os.path.join(self.data_path, str(subject_id) + ".npy")
-        labels_path = os.path.join(self.labels_path, str(subject_id) + ".npy")
 
         X = np.memmap(data_path, dtype="float32", mode="r", shape=input_shape)
+
+        if relative_id + self.L > num_windows:
+            X = X[relative_id:, self.channels_index]
+
+            remainer = self.L - X.shape[0]
+            # add zeros to the end of the array
+            X = np.concatenate([X, np.zeros((remainer, *X.shape[1:]))], axis=0)
+
+        else:
+            X = X[relative_id : relative_id + self.L, self.channels_index]
+
+        X = (torch.tensor(X).float() - self.mean) / self.std
+
+        return X
+
+    def get_stages(self, idx):
+        idx = idx - self.offset
+
+        relative_id = self.relative_idx[idx]
+        subject_id = self.subject_idx[idx]
+        num_windows = self.windows_index[subject_id]
+
+        labels_shape = (num_windows,)
+
+        labels_path = os.path.join(self.labels_path, str(subject_id) + ".npy")
+
         y = np.memmap(labels_path, dtype="int16", mode="r", shape=labels_shape)
 
-        X = X[relative_id : relative_id + self.L, self.channels_index]
-        y = y[relative_id : relative_id + self.L]
+        if relative_id + self.L > num_windows:
+            y = y[relative_id:]
+
+            remainer = self.L - len(y)
+            # associate the padded values to the class 6
+            y = np.concatenate([y, np.ones(remainer) * 5], axis=0)
+        else:
+            y = y[relative_id : relative_id + self.L]
+
+        y = torch.tensor(y).long()
+
+        return y
+
+    def __getitem__(self, idx):
+
+        X = self.get_signal(idx)
+        y = self.get_stages(idx)
+
+        return X, y
+
+class WholeNightReader(MemmapReader):
+    # suppose the batch_size to be 1
+    # reads every time one night
+    def __init__(
+        self,
+        data_folder: str,
+        dataset: str,
+        preprocessing: str,
+        #sequence_length: int,
+        channels_index: List[int],
+        offset: int,
+    ):
+        super().__init__(
+            data_folder = data_folder,
+            dataset = dataset,
+            preprocessing = preprocessing,
+            sequence_length = 30 * 2 * 60 * 24, # 24 hours,  
+            channels_index = channels_index,
+            offset = offset            
+        )
+        
+    def get_signal(self, idx):
+        idx = idx - self.offset
 
+        subject_id = self.subject_idx[idx]
+        num_windows = self.windows_index[subject_id]
+
+        input_shape = tuple([num_windows] + self.input_shape)
+
+        data_path = os.path.join(self.data_path, str(subject_id) + ".npy")
+
+        X = np.memmap(data_path, dtype="float32", mode="r", shape=input_shape)
+        X = X[:, self.channels_index]
         X = (torch.tensor(X).float() - self.mean) / self.std
+
+        return X
+
+    def get_stages(self, idx):
+        idx = idx - self.offset
+
+        subject_id = self.subject_idx[idx]
+        num_windows = self.windows_index[subject_id]
+
+        labels_shape = (num_windows,)
+
+        labels_path = os.path.join(self.labels_path, str(subject_id) + ".npy")
+
+        y = np.memmap(labels_path, dtype="int16", mode="r", shape=labels_shape)
+
         y = torch.tensor(y).long()
 
+        return y
+
+class AgeMemmapReader(MemmapReader):
+    def __init__(
+        self,
+        data_folder: str,
+        dataset: str,
+        preprocessing: str,
+        sequence_length: int,
+        channels_index: List[int],
+        offset: int,
+    ):
+        super().__init__(
+            data_folder, dataset, preprocessing, sequence_length, channels_index, offset
+        )
+
+    def __getitem__(self, idx):
+        idx = idx - self.offset
+        subject_id = self.subject_idx[idx]
+        age = self.table.loc[
+            self.table["subject_id"] == subject_id, "nsrr_age"
+        ].values.astype(float)[0]
+
+        y = torch.tensor([age], dtype=torch.float32)
+
+        X = self.get_signal(idx)
+
         return X, y
 
 
@@ -180,16 +310,36 @@ class DataReader(Reader):
         channels_index: List[int],
         offset: int,
         hpc: bool,
+        task: str = "sleep",
     ):
-
-        self.reader = MemmapReader(
-            data_folder=data_folder,
-            dataset=dataset,
-            preprocessing=preprocessing,
-            sequence_length=sequence_length,
-            channels_index=channels_index,
-            offset=offset,
-        )
+        if task == "sleep" and sequence_length > 0:            
+            self.reader = MemmapReader(
+                data_folder=data_folder,
+                dataset=dataset,
+                preprocessing=preprocessing,
+                sequence_length=sequence_length,
+                channels_index=channels_index,
+                offset=offset,
+            )
+        elif task == "sleep" and sequence_length == -1:
+            self.reader = WholeNightReader(
+                data_folder=data_folder,
+                dataset=dataset,
+                preprocessing=preprocessing,
+                channels_index=channels_index,
+                offset=offset,
+            )
+        elif task == "age":
+            self.reader = AgeMemmapReader(
+                data_folder=data_folder,
+                dataset=dataset,
+                preprocessing=preprocessing,
+                sequence_length=sequence_length,
+                channels_index=channels_index,
+                offset=offset,
+            )
+        else:
+            raise ValueError("task must be either sleep or age")
 
     def __len__(self):
         return self.reader.__len__()
@@ -204,9 +354,19 @@ class DataReader(Reader):
     def get_table(self):
         return self.reader.get_table()
 
+    def get_sequence_length(self):
+        return self.reader.L
+
 
 def build_index(nums_windows, subjects_ids, sequence_length):
-    data_len = int(np.sum(nums_windows - sequence_length + 1))
+    nums_windows = nums_windows.astype(int)
+
+    data_len = nums_windows - sequence_length
+    neg = np.where(data_len < 0)[0]
+    if len(neg) > 0:
+        data_len[neg] = 0
+
+    data_len = np.sum(data_len + 1)
 
     subject_idx = np.zeros(data_len, dtype=np.uint16)
     relative_idx = np.zeros(data_len, dtype=np.uint16)
@@ -221,19 +381,19 @@ def build_index(nums_windows, subjects_ids, sequence_length):
                 f"subject_id {subject_id} exceeds the maximum value for np.uint16"
             )
 
-        # Check if the relative_id can be stored in a uint16
-        if num_windows - sequence_length + 1 > np.iinfo(np.uint16).max:
+        num_sequences = max((num_windows - sequence_length, 0)) + 1
+
+        if num_sequences > np.iinfo(np.uint16).max:
             raise ValueError(
-                f"Relative index {num_windows - sequence_length + 1} exceeds the maximum value for np.uint16"
+                f"Relative index {num_sequences} exceeds the maximum value for np.uint16"
             )
 
-        subject_idx[start_index : start_index + num_windows - sequence_length + 1] = (
-            subject_id
-        )
-        relative_idx[start_index : start_index + num_windows - sequence_length + 1] = (
-            np.arange(num_windows - sequence_length + 1)
+        subject_idx[start_index : start_index + num_sequences] = subject_id
+        relative_idx[start_index : start_index + num_sequences] = np.arange(
+            num_sequences
         )
-        start_index += num_windows - sequence_length + 1
+
+        start_index += num_sequences
 
     windows_index = np.zeros(np.max(subjects_ids) + 1, dtype=np.uint16)
     for i, num_windows in enumerate(nums_windows):

{physioex-1.0.0.dev4 → physioex-1.0.3}/physioex/data/dataset.py

@@ -22,9 +22,9 @@ class PhysioExDataset(torch.utils.data.Dataset):
         target_transform: Callable = None,
         hpc: bool = False,
         indexed_channels: List[int] = ["EEG", "EOG", "EMG", "ECG"],
+        task: str = "sleep",
     ):
         self.datasets = datasets
-        self.L = sequence_length
         self.channels_index = [indexed_channels.index(ch) for ch in selected_channels]
 
         self.readers = []
@@ -41,6 +41,7 @@ class PhysioExDataset(torch.utils.data.Dataset):
                 channels_index=self.channels_index,
                 offset=offset,
                 hpc=hpc,
+                task=task,
             )
             offset += len(reader)
 
@@ -55,6 +56,7 @@ class PhysioExDataset(torch.utils.data.Dataset):
         self.target_transform = target_transform
 
         self.len = offset
+        self.L = sequence_length if sequence_length != -1 else 30 * 2 * 60 * 24
 
     def __len__(self):
         return self.len
@@ -122,7 +124,7 @@ class PhysioExDataset(torch.utils.data.Dataset):
         for table in self.tables:
             for _, row in table.iterrows():
 
-                num_windows = row["num_windows"] - self.L + 1
+                num_windows = max(row["num_windows"] - self.L, 0) + 1
 
                 indices = np.arange(
                     start=start_index, stop=start_index + num_windows