physioex 2.0.2__tar.gz → 2.0.3__tar.gz

{physioex-2.0.2 → physioex-2.0.3}/CHANGELOG.md

@@ -4,6 +4,57 @@ All notable changes to PhysioEx are documented here. Versions follow the
 `physioex.__version__` string (read dynamically by the build) and the `v*` git tags
 that publish to PyPI.
 
+## 2.0.3 — 2026-10-15
+
+CCSHS dataset, opt-in utilities for multi-dataset training, and three data-layer fixes. STAGES and Alzheimers label
+caches are recomputed on first use, and event counts per type change (see "`artifact` event type").
+
+### Added — CCSHS dataset (`physioex.data.datasets.ccshs`)
+
+`get_dataset("ccshs")`: Cleveland Children's Sleep and Health Study (NSRR `ccshs-trec`, 515 recordings). EEG
+`(C3, A2)` / `(C4, A1)` and EOG `(LOC, ROC)` at 128 Hz, chin EMG at 256 Hz; stage 3 and 4 merged to N3,
+Movement/unscored to -1, labels indexed by event onset, unknown stage vocabulary raises. Eleven recordings are
+excluded by default (`CCSHSDataset.EXCLUDED_SUBJECTS`, `exclude_subjects=[]` keeps them): nine whose stage minutes
+disagree with the NSRR summary table and two without a row in it. Metadata is read from
+`datasets/ccshs-trec-dataset-*.csv` whatever its version. Tests in `tests/test_ccshs.py`.
+
+### Added — site-balanced sampling (`physioex.data.sampling`)
+
+`site_balanced_weights` / `make_site_sampler` (also exported from `physioex.data`): sampling weights that give every
+site the same mass and, inside a site, every subject the same mass, for training on collections of datasets of very
+different sizes with `MultiDataset`. Tests in `tests/test_sampling.py`.
+
+### Added — opt-in channel-accuracy tracking during training (`ChannelMixer`)
+
+`ChannelMixer.track_train_acc` and `acc_momentum` update the per-channel accuracy that drives the accuracy-weighted
+modality dropout from the training step too (EMA of the clean per-channel logits). Off by default, so the released
+models and existing training runs behave exactly as before. Tests in `tests/test_channel_acc_tracking.py`.
+
+### Fixed — subjects without any requested channel (`BasePhysioDataset`)
+
+A subject for which none of the requested channels can be resolved was returned as a block of zeros with the wrong
+shape (`(L,)` instead of `(L, F, T)`), which made `dict_collate_fn` fail on any batch containing it. Its zeros now have
+the per-epoch shape of the pipeline output, like the zero-filled channels of subjects that miss only some channels, and
+a warning names the subject. Subjects are still never dropped.
+
+### Fixed — stale cached labels (`BasePhysioDataset.LABELS_REVISION`)
+
+Cached labels had no version, so after a fix of a label reader the old caches kept serving the wrong labels without any
+warning. Datasets now declare `LABELS_REVISION` (default 1); the revision is stored with the cached labels and a cache
+with a lower revision is recomputed (with a warning), or used with a warning when the source files are not available.
+Caches without the key count as revision 1, so existing caches of unchanged readers stay valid. `STAGESDataset` and
+`AlzheimersDataset` are at revision 2 (timestamp-based alignment of STAGES labels, onset-indexed Alzheimer labels):
+caches written before those fixes are recomputed on first use. Cached EDF headers that predate the `start_sec` field
+(STAGES labels are aligned to the EDF start time with it) are probed again, or used with a warning when the EDF is not
+available.
+
+### Changed — `artifact` event type
+
+"SpO2 artifact" intervals (an unreliable oximeter signal, not a physiological event) were counted as `respiratory` by
+the NSRR parser and as `desaturation` by the STAGES parser. They now have their own type `artifact`; the concept text is
+unchanged. Event counts per type change accordingly, for example in CCSHS `respiratory` drops from 51 915 to 15 040
+and `artifact` has 36 875.
+
 ## 2.0.2 — 2026-10-10
 
 ### Fixed — accuracy-weighted modality dropout (`physioex.models.protosleepnet`)

{physioex-2.0.2 → physioex-2.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: physioex
-Version: 2.0.2
+Version: 2.0.3
 Summary: A python package for explainable sleep staging via deep learning
 Project-URL: Homepage, https://github.com/guidogagl/physioex
 Project-URL: Documentation, https://guidogagl.github.io/physioex
@@ -105,7 +105,7 @@ linear probing): `CBraMod`, `BENDR`, `LaBraM`, `BIOT`, `SleepFM`, `TFC`, `REVE`,
 ## Supported datasets
 
 All datasets are resolved by name via `get_dataset(name)`; `available_datasets()` returns:
-`alzheimers`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
+`alzheimers`, `ccshs`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
 
 ### Publicly Available:
 
@@ -125,6 +125,7 @@ These datasets can be obtained from the NSRR archive. Once downloaded, point Phy
 - [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
 - [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
 - [STAGES (Stanford Technology Analytics and Genomics in Sleep)](https://sleepdata.org/datasets/stages), 1,914 recordings across 13 clinical sites with a bipolar EEG montage (C4-M1 / C3-M2 equivalent), EOG and chin EMG, plus rich respiratory / arousal / PLM event annotations. Selectable by site and by recording (`first` / `second` / `all`).
+- [CCSHS (Cleveland Children's Sleep and Health Study)](https://sleepdata.org/datasets/ccshs) (`ccshs`), 515 children and adolescents with one in-home PSG each (NSRR `ccshs-trec`): C3/A2 and C4/A1 EEG (128 Hz), LOC/ROC EOG (128 Hz) and chin EMG (256 Hz), stages scored by Rechtschaffen & Kales and merged to N3. Eleven recordings whose hypnogram cannot be reconciled with the NSRR summary table are excluded by default (`exclude_subjects=[]` keeps them).
 - [HomePAP](https://sleepdata.org/datasets/homepap) (`hpap`), a positive-airway-pressure study in NSRR format, with three subsets — `lab-full`, `lab-split` and `home` — selectable via the `subset` parameter.
 
 ### Others

{physioex-2.0.2 → physioex-2.0.3}/README.md

@@ -46,7 +46,7 @@ linear probing): `CBraMod`, `BENDR`, `LaBraM`, `BIOT`, `SleepFM`, `TFC`, `REVE`,
 ## Supported datasets
 
 All datasets are resolved by name via `get_dataset(name)`; `available_datasets()` returns:
-`alzheimers`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
+`alzheimers`, `ccshs`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
 
 ### Publicly Available:
 
@@ -66,6 +66,7 @@ These datasets can be obtained from the NSRR archive. Once downloaded, point Phy
 - [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
 - [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
 - [STAGES (Stanford Technology Analytics and Genomics in Sleep)](https://sleepdata.org/datasets/stages), 1,914 recordings across 13 clinical sites with a bipolar EEG montage (C4-M1 / C3-M2 equivalent), EOG and chin EMG, plus rich respiratory / arousal / PLM event annotations. Selectable by site and by recording (`first` / `second` / `all`).
+- [CCSHS (Cleveland Children's Sleep and Health Study)](https://sleepdata.org/datasets/ccshs) (`ccshs`), 515 children and adolescents with one in-home PSG each (NSRR `ccshs-trec`): C3/A2 and C4/A1 EEG (128 Hz), LOC/ROC EOG (128 Hz) and chin EMG (256 Hz), stages scored by Rechtschaffen & Kales and merged to N3. Eleven recordings whose hypnogram cannot be reconciled with the NSRR summary table are excluded by default (`exclude_subjects=[]` keeps them).
 - [HomePAP](https://sleepdata.org/datasets/homepap) (`hpap`), a positive-airway-pressure study in NSRR format, with three subsets — `lab-full`, `lab-split` and `home` — selectable via the `subset` parameter.
 
 ### Others

{physioex-2.0.2 → physioex-2.0.3}/docs/pages/architecture/library/data.md

@@ -67,13 +67,14 @@ classDiagram
     _NSRRBaseDataset <|-- MrOSDataset
     _NSRRBaseDataset <|-- SHHSDataset
     _NSRRBaseDataset <|-- HPAPDataset
+    _NSRRBaseDataset <|-- CCSHSDataset
     MultiDataset o-- "N" BasePhysioDataset
     BasePhysioDataset ..> SubjectSpec : yields
 ```
 
-The 12 concrete datasets are resolved by name from a **string registry** in
+The 13 concrete datasets are resolved by name from a **string registry** in
 `physioex/data/datasets/__init__.py` — `get_dataset(name)` / `available_datasets()`;
-keys: `hmc, sleepedf, dcsm, mesa, mros, hpap, wsc, mass, alzheimers, parkinsons, shhs, stages`.
+keys: `hmc, sleepedf, dcsm, mesa, mros, hpap, ccshs, wsc, mass, alzheimers, parkinsons, shhs, stages`.
 
 ### Legacy (deprecated)
 

{physioex-2.0.2 → physioex-2.0.3}/docs/pages/data.md

@@ -19,13 +19,13 @@ see the [data-layer architecture](architecture/library/data.md) page.
 
 ## Resolving a dataset by name
 
-`physioex.data.datasets` exposes a registry of the 12 concrete datasets:
+`physioex.data.datasets` exposes a registry of the 13 concrete datasets:
 
 ```python
 from physioex.data.datasets import get_dataset, available_datasets
 
 available_datasets()
-# ['alzheimers', 'dcsm', 'hmc', 'hpap', 'mass', 'mesa', 'mros',
+# ['alzheimers', 'ccshs', 'dcsm', 'hmc', 'hpap', 'mass', 'mesa', 'mros',
 #  'parkinsons', 'shhs', 'sleepedf', 'stages', 'wsc']
 
 HMCDataset = get_dataset("hmc")   # the class, not an instance
@@ -98,6 +98,29 @@ dcsm = get_dataset("dcsm")(channels=["EEG"], pipelines="raw", sequence_length=21
 merged = MultiDataset([hmc, dcsm])
 ```
 
+### Balancing sites with very different sizes
+
+When the merged datasets differ a lot in size, uniform sampling lets the largest one dominate. `make_site_sampler`
+gives every *site* the same total probability mass and, inside a site, every *subject* the same mass:
+`p(window) ∝ 1 / (N_s · W_u)` with `N_s` the number of training subjects of the site and `W_u` the number of windows of
+the subject. Several wrapped datasets can form one site (for example the visits of one cohort): give them the same
+name in `site_of_dataset`.
+
+```python
+from torch.utils.data import DataLoader, Subset
+from physioex.data import dict_collate_fn, make_site_sampler
+
+train_idx = ...                                    # flat MultiDataset indices of the training windows
+sampler, report = make_site_sampler(
+    merged, train_idx, site_of_dataset=["hmc", "dcsm"], num_samples=100_000, seed=0
+)
+loader = DataLoader(Subset(merged, train_idx), batch_size=32, sampler=sampler, collate_fn=dict_collate_fn)
+report["hmc"]   # {'subjects': ..., 'windows': ..., 'mass': 0.5}
+```
+
+`site_balanced_weights(merged, train_idx, site_of_dataset)` returns the weights (summing to 1) if you want to build
+your own sampler.
+
 ## Building a DataLoader
 
 Because items are dicts, use `dict_collate_fn` as the loader's `collate_fn`:

physioex-2.0.3/physioex/__init__.py

@@ -0,0 +1 @@
+__version__ = "2.0.3"

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/__init__.py

@@ -29,6 +29,7 @@ from physioex.data.steps import (
 from physioex.data.presets import get_preset, available_presets, PRESETS
 from physioex.data.base import BasePhysioDataset, SubjectSpec
 from physioex.data.multi import MultiDataset
+from physioex.data.sampling import site_balanced_weights, make_site_sampler
 from physioex.data.cache import ChannelCache, SCHEMA_VERSION
 from physioex.data.collate import dict_collate_fn, stack_channels, is_dict_batch
 from physioex.data.readers import (
@@ -67,6 +68,9 @@ __all__ = [
     "BasePhysioDataset",
     "SubjectSpec",
     "MultiDataset",
+    # Sampling
+    "site_balanced_weights",
+    "make_site_sampler",
     # Cache
     "ChannelCache",
     "SCHEMA_VERSION",

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/base.py

@@ -105,6 +105,11 @@ class BasePhysioDataset(Dataset):
     # Epoch length in seconds (30 for humans, 4 for mouse)
     DEFAULT_EPOCH_LENGTH_SEC: float = 30.0
 
+    #: Revision of the label reader. Cached labels store the revision they were computed with; a cache with a
+    #: LOWER revision is stale (the reader changed how labels are built) and is recomputed. Caches without the
+    #: key count as revision 1. Bump it in a subclass whenever a fix changes the labels it returns.
+    LABELS_REVISION: int = 1
+
     def __init__(
         self,
         root: Optional[str] = None,
@@ -594,6 +599,7 @@ class BasePhysioDataset(Dataset):
         """Return the EDF header for a subject, using on-disk cache when
         valid.  When ``self.cache_enabled`` is False, the header is always
         probed from the source file and no disk I/O is performed."""
+        stale_header = None
         if self.cache_enabled:
             hpath = self.cache.header_path(self.DATASET_NAME, spec.subject_id)
             src_mtime = None
@@ -611,11 +617,24 @@ class BasePhysioDataset(Dataset):
                             or abs(float(cached.get("source_mtime", 0)) - src_mtime)
                             < 1e-3
                         ):
-                            return EDFHeader.from_dict(cached)
+                            if "start_sec" in cached:
+                                return EDFHeader.from_dict(cached)
+                            # Written before ``start_sec`` existed (label readers such as STAGES need it
+                            # to align the hypnogram to the EDF start): probe the header again.
+                            stale_header = cached
                     except Exception:
                         pass
 
-        header = self._read_edf_header(spec)
+        try:
+            header = self._read_edf_header(spec)
+        except Exception as exc:
+            if stale_header is None:
+                raise
+            logger.warning(
+                f"[{self.DATASET_NAME}] {spec.subject_id}: cached header predates the 'start_sec' field and "
+                f"cannot be refreshed ({type(exc).__name__}: {exc}); using the cached header"
+            )
+            return EDFHeader.from_dict(stale_header)
 
         if self.cache_enabled:
             try:
@@ -634,8 +653,10 @@ class BasePhysioDataset(Dataset):
         unconditionally available; subjects missing a channel get it
         zero-filled at access time.
 
-        Subjects are **never dropped** due to missing channels.
-        Only genuinely corrupt EDFs (header unreadable) are skipped
+        Subjects are **never dropped** due to missing channels (missing
+        channels are zero-filled with the shape the pipeline produces; a
+        subject with none of the requested channels gets a warning and all
+        zeros). Only genuinely corrupt EDFs (header unreadable) are skipped
         when ``skip_corrupt=True``.
         """
         # Phase 1: probe all headers
@@ -692,6 +713,13 @@ class BasePhysioDataset(Dataset):
                     f"{missing} not available (will be zero-filled)"
                 )
 
+            if self.channels and all(rc is None for rc in resolved):
+                warnings.warn(
+                    f"[{self.DATASET_NAME}] Subject {spec.subject_id}: none of the requested channels "
+                    f"{list(self.channels)} is available (EDF channels: {list(header.available_channels)}); "
+                    f"its signals will be all zeros"
+                )
+
             self._resolved[spec.subject_id] = resolved
 
     # ------------------------------------------------------------------
@@ -711,30 +739,93 @@ class BasePhysioDataset(Dataset):
             if lp.exists() and lmeta.exists():
                 try:
                     memmap, meta = self.cache.load_memmap(lp)
-                    return np.asarray(memmap).astype(np.int64, copy=False)
+                    cached = np.asarray(memmap).astype(np.int64, copy=False)
+                    cached_rev = int((meta or {}).get("labels_revision", 1))
+                    if cached_rev >= self.LABELS_REVISION:
+                        return cached
+                    stale_rev = cached_rev
                 except Exception as exc:
                     logger.warning(
                         f"Corrupt label cache for {spec.subject_id}: {exc}; "
                         f"rebuilding"
                     )
+                    stale_rev = None
+                if stale_rev is not None:
+                    # cache written by an older label reader: recompute if the source is available
+                    try:
+                        fresh = self._sanitize_labels(
+                            np.asarray(self._read_subject_labels(spec), dtype=np.int16)
+                        )
+                    except Exception as exc:
+                        logger.warning(
+                            f"[{self.DATASET_NAME}] {spec.subject_id}: cached labels have revision "
+                            f"{stale_rev} < {self.LABELS_REVISION} (stale) and cannot be recomputed "
+                            f"({type(exc).__name__}: {exc}); using the cached labels"
+                        )
+                        return cached
+                    if not getattr(self, "_warned_stale_labels", False):
+                        logger.warning(
+                            f"[{self.DATASET_NAME}] cached labels have revision {stale_rev} < "
+                            f"{self.LABELS_REVISION} (the label reader changed): recomputing them"
+                        )
+                        self._warned_stale_labels = True
+                    self._save_labels_cache(lp, spec, fresh)
+                    return fresh.astype(np.int64, copy=False)
 
         labels = self._read_subject_labels(spec)
         labels = self._sanitize_labels(np.asarray(labels, dtype=np.int16))
 
         if self.cache_enabled:
-            meta = {
-                "dataset": self.DATASET_NAME,
-                "subject_id": spec.subject_id,
-                "n_epochs": int(labels.shape[0]),
-                "sanitized_to": "AASM_5_class",
-            }
-            try:
-                self.cache.atomic_save_array(lp, labels, meta)
-            except Exception as exc:
-                logger.warning(f"Could not cache labels for {spec.subject_id}: {exc}")
+            self._save_labels_cache(lp, spec, labels)
 
         return labels.astype(np.int64, copy=False)
 
+    def _pipeline_output_shape(self) -> Tuple[int, ...]:
+        """Per-epoch output shape of the pipeline (e.g. ``(29, 129)`` for the spectrogram presets).
+
+        Used to zero-fill a subject for which no requested channel could be resolved: its zeros must have the
+        same trailing shape as the signals of the other subjects, or batches mixing them cannot be stacked.
+        """
+        cached = getattr(self, "_fallback_out_shape", None)
+        if cached is not None:
+            return cached
+        shape: Tuple[int, ...] = ()
+        try:
+            fs = 100.0
+            req = self.channels[0] if self.channels else "EEG"
+            stub = ResolvedChannel(
+                request=req,
+                physical=req,
+                modality=infer_channel_modality(str(req)).name,
+                fs_in=fs,
+                is_differential=False,
+            )
+            compiled = self._pipeline_for_channel(stub).compile(fs)
+            n = int(round(self.epoch_length_sec * fs))
+            out = self._epoch_and_run_pipeline(
+                np.zeros(n, dtype=np.float32), fs, compiled, stub
+            )
+            shape = tuple(out.shape[1:])
+        except Exception as exc:  # fall back to the historical 1-D zeros
+            logger.warning(
+                f"[{self.DATASET_NAME}] could not infer the pipeline output shape: {exc}"
+            )
+        self._fallback_out_shape = shape
+        return shape
+
+    def _save_labels_cache(self, lp, spec: SubjectSpec, labels: np.ndarray) -> None:
+        meta = {
+            "dataset": self.DATASET_NAME,
+            "subject_id": spec.subject_id,
+            "n_epochs": int(labels.shape[0]),
+            "sanitized_to": "AASM_5_class",
+            "labels_revision": int(self.LABELS_REVISION),
+        }
+        try:
+            self.cache.atomic_save_array(lp, labels, meta)
+        except Exception as exc:
+            logger.warning(f"Could not cache labels for {spec.subject_id}: {exc}")
+
     def _load_or_compute_channel(
         self, spec: SubjectSpec, resolved: ResolvedChannel
     ) -> np.ndarray:
@@ -1190,7 +1281,7 @@ class BasePhysioDataset(Dataset):
 
         # Fill in missing channels with zeros (same shape as resolved ones)
         if _ref_shape is None:
-            _ref_shape = (requested_len,)
+            _ref_shape = (requested_len,) + self._pipeline_output_shape()
         for key in channel_order:
             if key not in signals_raw:
                 signals_raw[key] = np.zeros(_ref_shape, dtype=np.float32)

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/datasets/__init__.py

@@ -15,6 +15,7 @@ from physioex.data.datasets.alzheimers import AlzheimersDataset
 from physioex.data.datasets.parkinsons import ParkinsonsDataset
 from physioex.data.datasets.shhs import SHHSDataset
 from physioex.data.datasets.stages import STAGESDataset
+from physioex.data.datasets.ccshs import CCSHSDataset
 
 REGISTRY = {
     "hmc": HMCDataset,
@@ -29,6 +30,7 @@ REGISTRY = {
     "parkinsons": ParkinsonsDataset,
     "shhs": SHHSDataset,
     "stages": STAGESDataset,
+    "ccshs": CCSHSDataset,
 }
 
 
@@ -55,6 +57,7 @@ __all__ = [
     "ParkinsonsDataset",
     "SHHSDataset",
     "STAGESDataset",
+    "CCSHSDataset",
     "REGISTRY",
     "get_dataset",
     "available_datasets",

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/datasets/alzheimers.py

@@ -41,6 +41,8 @@ class AlzheimersDataset(BasePhysioDataset):
 
     DATASET_NAME = "alzheimers"
     DATASET_SUBDIR = "AlzheimerData"
+    #: 2: labels are indexed by event onset (gaps stay -1); revision-1 caches are compacted and misaligned.
+    LABELS_REVISION = 2
     DEFAULT_EPOCH_LENGTH_SEC = 30.0
 
     # Channel preference lists.

physioex-2.0.3/physioex/data/datasets/ccshs.py

@@ -0,0 +1,200 @@
+"""CCSHS (Cleveland Children's Sleep and Health Study, NSRR ``ccshs-trec``).
+
+Layout on disk::
+
+    <root>/
+        polysomnography/
+            edfs/ccshs-trec-1800001.edf
+            annotations-events-nsrr/ccshs-trec-1800001-nsrr.xml
+        datasets/ccshs-trec-dataset-0.8.0.csv
+
+One recording per subject; subject ID = EDF stem (``ccshs-trec-<nsrrid>``).
+EEG C3/C4 are recorded against the contralateral mastoid (A2/A1) and are
+re-referenced by the differential-pair resolution (``("C3", "A2")``).
+EDF is mixed-rate (EEG/EOG 128 Hz, chin EMG 256 Hz).
+
+Eleven recordings are excluded by default (``CCSHSDataset.EXCLUDED_SUBJECTS``): nine whose
+per-stage minutes in the NSRR XML disagree with the stage minutes of the NSRR summary
+dataset CSV, and two that have no row in that CSV. Pass ``exclude_subjects=[]`` to keep them.
+
+Stage vocabulary in the NSRR XML (``EventType`` ``Stages|Stages``): ``Wake|0``,
+``Stage 1..4 sleep|1..4``, ``REM sleep|5``, ``Movement|6``. Stage 3 and 4 are
+merged to N3. Movement/unscored -> -1.  Any *other* stage concept raises, so
+a vocabulary change cannot silently become "unscored".
+"""
+from __future__ import annotations
+
+import logging
+import xml.etree.ElementTree as ET
+from pathlib import Path
+from typing import Any, Dict, Iterable, List, Optional
+
+import numpy as np
+
+from physioex.data.base import SubjectSpec, get_data_root
+from physioex.data.datasets._nsrr import _NSRRBaseDataset
+
+logger = logging.getLogger("physioex.data")
+
+CCSHS_STAGE_MAP: Dict[str, int] = {
+    "Wake|0": 0,
+    "Stage 1 sleep|1": 1,
+    "Stage 2 sleep|2": 2,
+    "Stage 3 sleep|3": 3,
+    "Stage 4 sleep|4": 3,  # S3 + S4 -> N3
+    "REM sleep|5": 4,
+    "Movement|6": -1,
+    "Unscored|9": -1,
+}
+
+
+def parse_ccshs_xml(xml_path, epoch_length_sec: float = 30.0) -> Dict[str, Any]:
+    """Parse a CCSHS NSRR XML.
+
+    Returns dict with ``labels`` (int16, indexed by event ONSET, length
+    ``floor(recording_duration / epoch)``), ``duration_sec`` (from the
+    "Recording Start Time" event, else the end of the last stage event) and
+    ``clock_time`` (string or None).
+    """
+    root = ET.parse(str(xml_path)).getroot()
+    events = []
+    rec_dur: Optional[float] = None
+    clock: Optional[str] = None
+    for ev in root.iter("ScoredEvent"):
+        etype = (ev.findtext("EventType") or "").strip()
+        concept = (ev.findtext("EventConcept") or "").strip()
+        if concept == "Recording Start Time":
+            rec_dur = float(ev.findtext("Duration"))
+            clock = ev.findtext("ClockTime")
+            continue
+        if not etype.startswith("Stages"):
+            continue
+        if concept not in CCSHS_STAGE_MAP:
+            raise ValueError(f"{xml_path}: unknown stage concept {concept!r}")
+        events.append(
+            (float(ev.findtext("Start")), float(ev.findtext("Duration")), concept)
+        )
+    if not events:
+        return {
+            "labels": np.array([], dtype=np.int16),
+            "duration_sec": rec_dur,
+            "clock_time": clock,
+        }
+    end = max(s + d for s, d, _ in events)
+    total = rec_dur if rec_dur is not None else end
+    n = int(total // epoch_length_sec)
+    labels = np.full(n, -1, dtype=np.int16)
+    for s, d, c in events:
+        i0 = int(round(s / epoch_length_sec))
+        i1 = min(int(round((s + d) / epoch_length_sec)), n)
+        if i0 < n:
+            labels[i0:i1] = CCSHS_STAGE_MAP[c]
+    return {"labels": labels, "duration_sec": total, "clock_time": clock}
+
+
+class CCSHSDataset(_NSRRBaseDataset):
+    """Cleveland Children's Sleep and Health Study (~515 children/adolescents)."""
+
+    DATASET_NAME = "ccshs"
+    DATASET_SUBDIR = "ccshs"
+
+    #: Recordings left out by default. Nine: stage minutes in the XML differ from the NSRR
+    #: summary CSV (``ccshs-trec-dataset-*.csv``); two: no CSV row, so the hypnogram cannot be
+    #: cross-checked. Subject ids are EDF stems.
+    EXCLUDED_SUBJECTS = frozenset(
+        f"ccshs-trec-{n}"
+        for n in (
+            1800064,
+            1800224,
+            1800306,
+            1800384,
+            1800432,
+            1800458,
+            1800620,
+            1800640,
+            1800873,  # CSV-discordant
+            1800109,
+            1800489,  # no CSV row
+        )
+    )
+
+    CHANNEL_PREFERENCES: Dict[str, List] = {
+        "EEG": [
+            ("C3", "A2"),
+            ("C4", "A1"),
+            "C3",
+            "C4",
+        ],
+        "EOG": [
+            ("LOC", "ROC"),
+            "LOC",
+            "ROC",
+        ],
+        "EMG": [
+            ("EMG1", "EMG2"),
+            ("EMG1", "EMG3"),
+            "EMG1",
+        ],
+    }
+
+    def __init__(
+        self,
+        root: Optional[str] = None,
+        exclude_subjects: Optional[Iterable[str]] = None,
+        **kwargs,
+    ):
+        """
+        Args:
+            root: dataset root (default ``<data_root>/ccshs``).
+            exclude_subjects: subject ids (``ccshs-trec-<nsrrid>``) to leave out. ``None`` uses
+                :attr:`EXCLUDED_SUBJECTS`; pass ``[]`` to keep every recording.
+        """
+        if root is None:
+            root = str(get_data_root() / self.DATASET_SUBDIR)
+        self._exclude = frozenset(
+            self.EXCLUDED_SUBJECTS if exclude_subjects is None else exclude_subjects
+        )
+        self._metadata = self._load_metadata(root)
+        super().__init__(root=root, **kwargs)
+
+    @staticmethod
+    def _load_metadata(root: str) -> Dict[str, Dict[str, Any]]:
+        # version-independent: ccshs-trec-dataset-<version>.csv (newest version wins)
+        found = sorted((Path(root) / "datasets").glob("ccshs-trec-dataset-*.csv"))
+        if not found:
+            logger.warning(
+                "CCSHS metadata CSV not found under %s", Path(root) / "datasets"
+            )
+            return {}
+        csv_path = found[-1]
+        import pandas as pd
+
+        df = pd.read_csv(csv_path, low_memory=False)
+        meta: Dict[str, Dict[str, Any]] = {}
+        for _, row in df.iterrows():
+            key = str(int(row["nsrrid"]))
+            meta[key] = {c: (None if pd.isna(v) else v) for c, v in row.items()}
+        return meta
+
+    def _list_subjects(self) -> List[SubjectSpec]:
+        root = Path(self.root)
+        specs = self._pair_edfs_with_xml(
+            root / "polysomnography" / "edfs",
+            root / "polysomnography" / "annotations-events-nsrr",
+        )
+        if self._exclude:
+            kept = [s for s in specs if s.subject_id not in self._exclude]
+            if len(kept) != len(specs):
+                logger.info(
+                    "CCSHS: %d recordings excluded by default list",
+                    len(specs) - len(kept),
+                )
+            specs = kept
+        for spec in specs:
+            nid = spec.subject_id.replace("ccshs-trec-", "")
+            if nid in self._metadata:
+                spec.external_meta.update(self._metadata[nid])
+        return specs
+
+    def _read_subject_labels(self, spec: SubjectSpec) -> np.ndarray:
+        return parse_ccshs_xml(spec.label_path, self.epoch_length_sec)["labels"]

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/datasets/channels_table.md

@@ -426,6 +426,30 @@ Comprehensive reference of available channels, modality types, and sampling freq
 
 ---
 
+## 13. CCSHS (Cleveland Children's Sleep and Health Study)
+
+**Source:** NSRR `ccshs-trec` - 515 children and adolescents, one in-home PSG each
+**Native Sampling Rate:** 128 Hz (EEG, EOG), 256 Hz (chin EMG)
+**Epoch Length:** 30 seconds
+**Stage vocabulary:** Wake, Stage 1-4, REM, Movement (NSRR XML, `Stages|Stages`). Stage 3 and 4 are merged to N3; Movement and unscored epochs are -1.
+
+| Channel Name | Modality Type | FS (Hz) | Usability | Notes |
+|--------------|---------------|---------|-----------|-------|
+| C3, C4 | EEG (0) | 128 | ✅ Usable | Recorded against the contralateral mastoid; use the pairs `(C3, A2)` and `(C4, A1)` |
+| A1, A2 | - | 128 | ➖ Reference | Mastoid references, used only to build the pairs |
+| LOC, ROC | EOG (1) | 128 | ✅ Usable | Pair `(LOC, ROC)` |
+| EMG1, EMG2, EMG3 | EMG (2) | 256 | ✅ Usable | Chin EMG, pair `(EMG1, EMG2)` preferred |
+
+**Modality Assessment:** ✅ Pair-resolved channels are classified correctly.
+
+**Exclusions:** eleven recordings are left out by default (`CCSHSDataset.EXCLUDED_SUBJECTS`): nine whose per-stage minutes in the NSRR XML disagree with the stage minutes in the NSRR summary CSV, and two with no CSV row. Pass `exclude_subjects=[]` to keep them.
+
+**Available Modalities for Sleep Staging:** EEG, EOG, EMG
+
+**Summary:** Paediatric community cohort with a standard AASM-like montage at a lower sampling rate than the clinical datasets; mixed rates inside one EDF are handled by the per-channel pipeline.
+
+---
+
 ## Cross-Dataset Summary
 
 ### Sampling Rate Summary
@@ -444,6 +468,7 @@ Comprehensive reference of available channels, modality types, and sampling freq
 | WSC | 512 | Uniform high frequency |
 | Alzheimer's | 200 | Uniform |
 | Parkinson's | 500 | Highest uniform frequency |
+| CCSHS | 128 (EEG, EOG), 256 (EMG) | Mixed inside one EDF |
 
 ### Modality Type Coverage
 

{physioex-2.0.2 → physioex-2.0.3}/physioex/data/datasets/stages.py

@@ -148,6 +148,8 @@ class STAGESDataset(BasePhysioDataset):
 
     DATASET_NAME = "stages"
     DATASET_SUBDIR = "stages"
+    #: 2: labels are placed by timestamp relative to the EDF start (fix 7b96d20); revision-1 caches are misaligned.
+    LABELS_REVISION = 2
     DEFAULT_EPOCH_LENGTH_SEC = 30.0
     CHANNEL_PREFERENCES = CHANNEL_PREFERENCES