physioex 2.0.2__tar.gz → 2.0.4__tar.gz

physioex-2.0.4/CHANGELOG.md

@@ -0,0 +1,145 @@
+# Changelog
+
+All notable changes to PhysioEx are documented here. Versions follow the
+`physioex.__version__` string (read dynamically by the build) and the `v*` git tags
+that publish to PyPI.
+
+## 2.0.4 — 2026-10-16
+
+Data-layer fixes found by an audit of what physioex reads against the raw files of every dataset. Label caches of
+Sleep-EDF, WSC and Alzheimers are recomputed on first use (`LABELS_REVISION` 2, 2 and 3). Cached WSC events are not
+versioned: delete `events/<subject>.json` of the 26 allscore nights whose events changed.
+
+### Fixed
+- **WSC `.allscore.txt`:** labels and events are anchored at `START RECORDING`. A `STAGE - NO STAGE` line timed before it
+  moved the time origin back by hours and shifted every later label (45 of 743 files). Midnight is a backward jump of
+  the clock, `PAUSED` to `RECORDING RESUMED` stretches are removed (the EDF has no samples there), and labels or events
+  past the end of the EDF are dropped. A file without `START RECORDING` yields no labels, as before.
+- **Sleep-EDF:** the hypnogram always runs to 24 h; its tail is cut at the end of the PSG instead of being paired with
+  zero padding (in one night 614 epochs of `Wake`).
+- **Alzheimers:** labels past the end of a truncated EDF are dropped.
+- **MESA:** `*_Off` electrode-status flags (1 Hz) are classified `OTHER`, not EEG/EOG/EMG.
+
+### Added
+- **WSC events from `.sco.txt`** (the 1827 Gamma-system nights; only the 743 Twin-system `.allscore.txt` nights had
+  events). Same event types as the allscore parser. `Scan # x2` counts samples at twice the EDF rate; a few files write
+  the scan with two extra digits (the clock column is used) and 548 write the length ×100 with a negative sign.
+
+## 2.0.3 — 2026-10-15
+
+CCSHS dataset, opt-in utilities for multi-dataset training, and three data-layer fixes. STAGES and Alzheimers label
+caches are recomputed on first use, and event counts per type change (see "`artifact` event type").
+
+### Added — CCSHS dataset (`physioex.data.datasets.ccshs`)
+
+`get_dataset("ccshs")`: Cleveland Children's Sleep and Health Study (NSRR `ccshs-trec`, 515 recordings). EEG
+`(C3, A2)` / `(C4, A1)` and EOG `(LOC, ROC)` at 128 Hz, chin EMG at 256 Hz; stage 3 and 4 merged to N3,
+Movement/unscored to -1, labels indexed by event onset, unknown stage vocabulary raises. Eleven recordings are
+excluded by default (`CCSHSDataset.EXCLUDED_SUBJECTS`, `exclude_subjects=[]` keeps them): nine whose stage minutes
+disagree with the NSRR summary table and two without a row in it. Metadata is read from
+`datasets/ccshs-trec-dataset-*.csv` whatever its version. Tests in `tests/test_ccshs.py`.
+
+### Added — site-balanced sampling (`physioex.data.sampling`)
+
+`site_balanced_weights` / `make_site_sampler` (also exported from `physioex.data`): sampling weights that give every
+site the same mass and, inside a site, every subject the same mass, for training on collections of datasets of very
+different sizes with `MultiDataset`. Tests in `tests/test_sampling.py`.
+
+### Added — opt-in channel-accuracy tracking during training (`ChannelMixer`)
+
+`ChannelMixer.track_train_acc` and `acc_momentum` update the per-channel accuracy that drives the accuracy-weighted
+modality dropout from the training step too (EMA of the clean per-channel logits). Off by default, so the released
+models and existing training runs behave exactly as before. Tests in `tests/test_channel_acc_tracking.py`.
+
+### Fixed — subjects without any requested channel (`BasePhysioDataset`)
+
+A subject for which none of the requested channels can be resolved was returned as a block of zeros with the wrong
+shape (`(L,)` instead of `(L, F, T)`), which made `dict_collate_fn` fail on any batch containing it. Its zeros now have
+the per-epoch shape of the pipeline output, like the zero-filled channels of subjects that miss only some channels, and
+a warning names the subject. Subjects are still never dropped.
+
+### Fixed — stale cached labels (`BasePhysioDataset.LABELS_REVISION`)
+
+Cached labels had no version, so after a fix of a label reader the old caches kept serving the wrong labels without any
+warning. Datasets now declare `LABELS_REVISION` (default 1); the revision is stored with the cached labels and a cache
+with a lower revision is recomputed (with a warning), or used with a warning when the source files are not available.
+Caches without the key count as revision 1, so existing caches of unchanged readers stay valid. `STAGESDataset` and
+`AlzheimersDataset` are at revision 2 (timestamp-based alignment of STAGES labels, onset-indexed Alzheimer labels):
+caches written before those fixes are recomputed on first use. Cached EDF headers that predate the `start_sec` field
+(STAGES labels are aligned to the EDF start time with it) are probed again, or used with a warning when the EDF is not
+available.
+
+### Changed — `artifact` event type
+
+"SpO2 artifact" intervals (an unreliable oximeter signal, not a physiological event) were counted as `respiratory` by
+the NSRR parser and as `desaturation` by the STAGES parser. They now have their own type `artifact`; the concept text is
+unchanged. Event counts per type change accordingly, for example in CCSHS `respiratory` drops from 51 915 to 15 040
+and `artifact` has 36 875.
+
+## 2.0.2 — 2026-10-10
+
+### Fixed — accuracy-weighted modality dropout (`physioex.models.protosleepnet`)
+
+`ZeroEmbeddingDropout` now samples the channels to occlude with probability
+proportional to their accuracy, `p_c = a_c / sum_j a_j`, so the channels the
+model relies on most are dropped most often. This is the rule the released
+ProtoSleepNet and ProtoSleepTransformer models were trained with; a later
+refactoring had inverted the sampling weights to `1 - a_c`. When no accuracy
+estimate is available yet the sampling is uniform. Unit tests in
+`tests/test_modality_dropout.py`.
+
+## 2.0.1 — 2026-09-13
+
+### Added — Layer-wise Relevance Propagation (`physioex.explain.lrp`)
+
+A conservation-based attribution family (Bach et al. 2015) covering the whole
+model zoo. Install with the new optional extra `pip install "physioex[explain]"`
+(Zennit only; no LXT dependency).
+
+- **Two entry points**, both seeding the target neuron with its logit so that
+  `Σ R ≈ f_c(x)`, both returning a per-sample `ConservationReport` with
+  `return_report=True`:
+  - `LRP` — Zennit composites for feed-forward / CNN models (Tsinalis,
+    Chambon2018): ε on dense, γ on convolutions, **w²** on the first layer
+    (unbounded EEG, not the pixel z-box), BatchNorm canonized; αβ / flat / z-box
+    configurable via `physioex_composite`.
+  - `ModelLRP` — whole-model LRP for recurrent, attention and transformer
+    architectures (TinySleepNet, SeqSleepNet, L-SeqSleepNet, SleepTransformer,
+    CoReSleep, ProtoSleepNet): **Arras signal-take** rule for LSTM/GRU at cell
+    level (`LRPLSTM`/`LRPGRU`, drop-in subclasses with a bit-identical forward,
+    `gate_rule="signal_take" | "uniform"`), **CP-LRP** for attention (value
+    path, exact conservation; Ali et al. 2022), identity rule for
+    LayerNorm/activations, proportional rule for residual sums, ε for linear
+    leaves and PhysioEx's learnable filterbank, CP-LRP adapters for the softmax
+    poolings (`AttentionPooling`, `AttentionLayer`, `ChannelMixer`).
+- `prepare_model_for_lrp` rewrites a deep copy of the model; plain `+` residuals
+  written in a model's own `forward` are redirected to the proportional rule at
+  runtime (`patch_residuals`); `audit_lrp_coverage` / `strict=True` report any
+  parametric leaf left on plain autograd; `register_lrp_adapter` adds custom
+  blocks. Signed, dtype-safe stabiliser `z + ε·sign z` everywhere (fp16-safe).
+- Docs: entry-point/rule matrix, how to read conservation and choose ε (ε must be
+  scaled to the activations: measured on the pretrained SeqSleepNet, 1e-6 inflates
+  relevance 2–3×, 1e-2 conserves), unsupported cases, and a literature note on
+  LRP vs path-based attribution (Deep Taylor, Integrated Gradients,
+  DeepLIFT/DeepSHAP). `physioex.explain.lrp` is in the API reference.
+- Example: `examples/explain/lrp_seqsleepnet_n3.py` explains the N3 logit of the
+  pretrained `seqsleepnet-phan` on MASS training sequences with Saliency,
+  Input×Gradient, Integrated Gradients and LRP under different rule assignments.
+- Tests: 91 new tests (`tests/explain/lrp/`), including forward-equivalence to
+  the fused modules, exact conservation on bias-free blocks and models, and all
+  six real architectures end-to-end with full rule coverage.
+
+### Known limitations
+
+CP-LRP assigns zero relevance to the attention query/key path by design (full
+AttnLRP is not implemented); bias relevance is absorbed (LRP-ε convention);
+attention masks, `batch_first=False` transformers, explicit RNN initial states
+and `proj_size` raise; ProtoSleepNet's VQ path runs under `no_grad`. Foundation
+encoders and frequency-domain (DFT/STFT) LRP are planned follow-ups.
+
+## 2.0.0 — 2026-07-15
+
+Ground-up release: raw-EDF lazy data layer with 12 dataset loaders, 9 pretrained
+foundation encoders behind a uniform `encode()` interface, hand-written `Trainer`,
+pytest suite and Sphinx documentation. Full notes in the
+[v2.0.0 GitHub Release](https://github.com/guidogagl/physioex/releases/tag/v2.0.0).

{physioex-2.0.2 → physioex-2.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: physioex
-Version: 2.0.2
+Version: 2.0.4
 Summary: A python package for explainable sleep staging via deep learning
 Project-URL: Homepage, https://github.com/guidogagl/physioex
 Project-URL: Documentation, https://guidogagl.github.io/physioex
@@ -105,7 +105,7 @@ linear probing): `CBraMod`, `BENDR`, `LaBraM`, `BIOT`, `SleepFM`, `TFC`, `REVE`,
 ## Supported datasets
 
 All datasets are resolved by name via `get_dataset(name)`; `available_datasets()` returns:
-`alzheimers`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
+`alzheimers`, `ccshs`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
 
 ### Publicly Available:
 
@@ -125,6 +125,7 @@ These datasets can be obtained from the NSRR archive. Once downloaded, point Phy
 - [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
 - [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
 - [STAGES (Stanford Technology Analytics and Genomics in Sleep)](https://sleepdata.org/datasets/stages), 1,914 recordings across 13 clinical sites with a bipolar EEG montage (C4-M1 / C3-M2 equivalent), EOG and chin EMG, plus rich respiratory / arousal / PLM event annotations. Selectable by site and by recording (`first` / `second` / `all`).
+- [CCSHS (Cleveland Children's Sleep and Health Study)](https://sleepdata.org/datasets/ccshs) (`ccshs`), 515 children and adolescents with one in-home PSG each (NSRR `ccshs-trec`): C3/A2 and C4/A1 EEG (128 Hz), LOC/ROC EOG (128 Hz) and chin EMG (256 Hz), stages scored by Rechtschaffen & Kales and merged to N3. Eleven recordings whose hypnogram cannot be reconciled with the NSRR summary table are excluded by default (`exclude_subjects=[]` keeps them).
 - [HomePAP](https://sleepdata.org/datasets/homepap) (`hpap`), a positive-airway-pressure study in NSRR format, with three subsets — `lab-full`, `lab-split` and `home` — selectable via the `subset` parameter.
 
 ### Others

{physioex-2.0.2 → physioex-2.0.4}/README.md

@@ -46,7 +46,7 @@ linear probing): `CBraMod`, `BENDR`, `LaBraM`, `BIOT`, `SleepFM`, `TFC`, `REVE`,
 ## Supported datasets
 
 All datasets are resolved by name via `get_dataset(name)`; `available_datasets()` returns:
-`alzheimers`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
+`alzheimers`, `ccshs`, `dcsm`, `hmc`, `hpap`, `mass`, `mesa`, `mros`, `parkinsons`, `shhs`, `sleepedf`, `stages`, `wsc`.
 
 ### Publicly Available:
 
@@ -66,6 +66,7 @@ These datasets can be obtained from the NSRR archive. Once downloaded, point Phy
 - [MrOS (The Osteoporotic Fractures in Men Study)](https://sleepdata.org/datasets/mros), is a multicenter study comprising 2,911 PSG recordings from men aged 65 years or older, enrolled at six clinical centers. PSG recordings were conducted in home settings and included C3/A2 and C4/A1 EEGs, chin EMG, and left-right EOG, all sampled at 256 Hz.
 - [WSC (The Wisconsin Sleep Cohort)](https://sleepdata.org/datasets/wsc) A longitudinal study of the causes, consequences, and natural history of sleep disorders using overnight in-laboratory sleep recordings gathered at the University of Wisconsin, United States, with a baseline sample of 1,500 subjects assessed at four-year intervals. The study consists of multiple visits with overnight PSG data acquisition. PSG recordings included C3/M2 EEG, EMG, and left-right EOG, all sampled at 200 Hz.
 - [STAGES (Stanford Technology Analytics and Genomics in Sleep)](https://sleepdata.org/datasets/stages), 1,914 recordings across 13 clinical sites with a bipolar EEG montage (C4-M1 / C3-M2 equivalent), EOG and chin EMG, plus rich respiratory / arousal / PLM event annotations. Selectable by site and by recording (`first` / `second` / `all`).
+- [CCSHS (Cleveland Children's Sleep and Health Study)](https://sleepdata.org/datasets/ccshs) (`ccshs`), 515 children and adolescents with one in-home PSG each (NSRR `ccshs-trec`): C3/A2 and C4/A1 EEG (128 Hz), LOC/ROC EOG (128 Hz) and chin EMG (256 Hz), stages scored by Rechtschaffen & Kales and merged to N3. Eleven recordings whose hypnogram cannot be reconciled with the NSRR summary table are excluded by default (`exclude_subjects=[]` keeps them).
 - [HomePAP](https://sleepdata.org/datasets/homepap) (`hpap`), a positive-airway-pressure study in NSRR format, with three subsets — `lab-full`, `lab-split` and `home` — selectable via the `subset` parameter.
 
 ### Others

{physioex-2.0.2 → physioex-2.0.4}/docs/pages/architecture/library/data.md

@@ -67,13 +67,14 @@ classDiagram
     _NSRRBaseDataset <|-- MrOSDataset
     _NSRRBaseDataset <|-- SHHSDataset
     _NSRRBaseDataset <|-- HPAPDataset
+    _NSRRBaseDataset <|-- CCSHSDataset
     MultiDataset o-- "N" BasePhysioDataset
     BasePhysioDataset ..> SubjectSpec : yields
 ```
 
-The 12 concrete datasets are resolved by name from a **string registry** in
+The 13 concrete datasets are resolved by name from a **string registry** in
 `physioex/data/datasets/__init__.py` — `get_dataset(name)` / `available_datasets()`;
-keys: `hmc, sleepedf, dcsm, mesa, mros, hpap, wsc, mass, alzheimers, parkinsons, shhs, stages`.
+keys: `hmc, sleepedf, dcsm, mesa, mros, hpap, ccshs, wsc, mass, alzheimers, parkinsons, shhs, stages`.
 
 ### Legacy (deprecated)
 

{physioex-2.0.2 → physioex-2.0.4}/docs/pages/data.md

@@ -19,13 +19,13 @@ see the [data-layer architecture](architecture/library/data.md) page.
 
 ## Resolving a dataset by name
 
-`physioex.data.datasets` exposes a registry of the 12 concrete datasets:
+`physioex.data.datasets` exposes a registry of the 13 concrete datasets:
 
 ```python
 from physioex.data.datasets import get_dataset, available_datasets
 
 available_datasets()
-# ['alzheimers', 'dcsm', 'hmc', 'hpap', 'mass', 'mesa', 'mros',
+# ['alzheimers', 'ccshs', 'dcsm', 'hmc', 'hpap', 'mass', 'mesa', 'mros',
 #  'parkinsons', 'shhs', 'sleepedf', 'stages', 'wsc']
 
 HMCDataset = get_dataset("hmc")   # the class, not an instance
@@ -98,6 +98,29 @@ dcsm = get_dataset("dcsm")(channels=["EEG"], pipelines="raw", sequence_length=21
 merged = MultiDataset([hmc, dcsm])
 ```
 
+### Balancing sites with very different sizes
+
+When the merged datasets differ a lot in size, uniform sampling lets the largest one dominate. `make_site_sampler`
+gives every *site* the same total probability mass and, inside a site, every *subject* the same mass:
+`p(window) ∝ 1 / (N_s · W_u)` with `N_s` the number of training subjects of the site and `W_u` the number of windows of
+the subject. Several wrapped datasets can form one site (for example the visits of one cohort): give them the same
+name in `site_of_dataset`.
+
+```python
+from torch.utils.data import DataLoader, Subset
+from physioex.data import dict_collate_fn, make_site_sampler
+
+train_idx = ...                                    # flat MultiDataset indices of the training windows
+sampler, report = make_site_sampler(
+    merged, train_idx, site_of_dataset=["hmc", "dcsm"], num_samples=100_000, seed=0
+)
+loader = DataLoader(Subset(merged, train_idx), batch_size=32, sampler=sampler, collate_fn=dict_collate_fn)
+report["hmc"]   # {'subjects': ..., 'windows': ..., 'mass': 0.5}
+```
+
+`site_balanced_weights(merged, train_idx, site_of_dataset)` returns the weights (summing to 1) if you want to build
+your own sampler.
+
 ## Building a DataLoader
 
 Because items are dicts, use `dict_collate_fn` as the loader's `collate_fn`:

physioex-2.0.4/physioex/__init__.py

@@ -0,0 +1 @@
+__version__ = "2.0.4"

{physioex-2.0.2 → physioex-2.0.4}/physioex/data/__init__.py

@@ -29,6 +29,7 @@ from physioex.data.steps import (
 from physioex.data.presets import get_preset, available_presets, PRESETS
 from physioex.data.base import BasePhysioDataset, SubjectSpec
 from physioex.data.multi import MultiDataset
+from physioex.data.sampling import site_balanced_weights, make_site_sampler
 from physioex.data.cache import ChannelCache, SCHEMA_VERSION
 from physioex.data.collate import dict_collate_fn, stack_channels, is_dict_batch
 from physioex.data.readers import (
@@ -67,6 +68,9 @@ __all__ = [
     "BasePhysioDataset",
     "SubjectSpec",
     "MultiDataset",
+    # Sampling
+    "site_balanced_weights",
+    "make_site_sampler",
     # Cache
     "ChannelCache",
     "SCHEMA_VERSION",

{physioex-2.0.2 → physioex-2.0.4}/physioex/data/base.py

@@ -105,6 +105,11 @@ class BasePhysioDataset(Dataset):
     # Epoch length in seconds (30 for humans, 4 for mouse)
     DEFAULT_EPOCH_LENGTH_SEC: float = 30.0
 
+    #: Revision of the label reader. Cached labels store the revision they were computed with; a cache with a
+    #: LOWER revision is stale (the reader changed how labels are built) and is recomputed. Caches without the
+    #: key count as revision 1. Bump it in a subclass whenever a fix changes the labels it returns.
+    LABELS_REVISION: int = 1
+
     def __init__(
         self,
         root: Optional[str] = None,
@@ -594,6 +599,7 @@ class BasePhysioDataset(Dataset):
         """Return the EDF header for a subject, using on-disk cache when
         valid.  When ``self.cache_enabled`` is False, the header is always
         probed from the source file and no disk I/O is performed."""
+        stale_header = None
         if self.cache_enabled:
             hpath = self.cache.header_path(self.DATASET_NAME, spec.subject_id)
             src_mtime = None
@@ -611,11 +617,24 @@ class BasePhysioDataset(Dataset):
                             or abs(float(cached.get("source_mtime", 0)) - src_mtime)
                             < 1e-3
                         ):
-                            return EDFHeader.from_dict(cached)
+                            if "start_sec" in cached:
+                                return EDFHeader.from_dict(cached)
+                            # Written before ``start_sec`` existed (label readers such as STAGES need it
+                            # to align the hypnogram to the EDF start): probe the header again.
+                            stale_header = cached
                     except Exception:
                         pass
 
-        header = self._read_edf_header(spec)
+        try:
+            header = self._read_edf_header(spec)
+        except Exception as exc:
+            if stale_header is None:
+                raise
+            logger.warning(
+                f"[{self.DATASET_NAME}] {spec.subject_id}: cached header predates the 'start_sec' field and "
+                f"cannot be refreshed ({type(exc).__name__}: {exc}); using the cached header"
+            )
+            return EDFHeader.from_dict(stale_header)
 
         if self.cache_enabled:
             try:
@@ -634,8 +653,10 @@ class BasePhysioDataset(Dataset):
         unconditionally available; subjects missing a channel get it
         zero-filled at access time.
 
-        Subjects are **never dropped** due to missing channels.
-        Only genuinely corrupt EDFs (header unreadable) are skipped
+        Subjects are **never dropped** due to missing channels (missing
+        channels are zero-filled with the shape the pipeline produces; a
+        subject with none of the requested channels gets a warning and all
+        zeros). Only genuinely corrupt EDFs (header unreadable) are skipped
         when ``skip_corrupt=True``.
         """
         # Phase 1: probe all headers
@@ -692,6 +713,13 @@ class BasePhysioDataset(Dataset):
                     f"{missing} not available (will be zero-filled)"
                 )
 
+            if self.channels and all(rc is None for rc in resolved):
+                warnings.warn(
+                    f"[{self.DATASET_NAME}] Subject {spec.subject_id}: none of the requested channels "
+                    f"{list(self.channels)} is available (EDF channels: {list(header.available_channels)}); "
+                    f"its signals will be all zeros"
+                )
+
             self._resolved[spec.subject_id] = resolved
 
     # ------------------------------------------------------------------
@@ -711,30 +739,93 @@ class BasePhysioDataset(Dataset):
             if lp.exists() and lmeta.exists():
                 try:
                     memmap, meta = self.cache.load_memmap(lp)
-                    return np.asarray(memmap).astype(np.int64, copy=False)
+                    cached = np.asarray(memmap).astype(np.int64, copy=False)
+                    cached_rev = int((meta or {}).get("labels_revision", 1))
+                    if cached_rev >= self.LABELS_REVISION:
+                        return cached
+                    stale_rev = cached_rev
                 except Exception as exc:
                     logger.warning(
                         f"Corrupt label cache for {spec.subject_id}: {exc}; "
                         f"rebuilding"
                     )
+                    stale_rev = None
+                if stale_rev is not None:
+                    # cache written by an older label reader: recompute if the source is available
+                    try:
+                        fresh = self._sanitize_labels(
+                            np.asarray(self._read_subject_labels(spec), dtype=np.int16)
+                        )
+                    except Exception as exc:
+                        logger.warning(
+                            f"[{self.DATASET_NAME}] {spec.subject_id}: cached labels have revision "
+                            f"{stale_rev} < {self.LABELS_REVISION} (stale) and cannot be recomputed "
+                            f"({type(exc).__name__}: {exc}); using the cached labels"
+                        )
+                        return cached
+                    if not getattr(self, "_warned_stale_labels", False):
+                        logger.warning(
+                            f"[{self.DATASET_NAME}] cached labels have revision {stale_rev} < "
+                            f"{self.LABELS_REVISION} (the label reader changed): recomputing them"
+                        )
+                        self._warned_stale_labels = True
+                    self._save_labels_cache(lp, spec, fresh)
+                    return fresh.astype(np.int64, copy=False)
 
         labels = self._read_subject_labels(spec)
         labels = self._sanitize_labels(np.asarray(labels, dtype=np.int16))
 
         if self.cache_enabled:
-            meta = {
-                "dataset": self.DATASET_NAME,
-                "subject_id": spec.subject_id,
-                "n_epochs": int(labels.shape[0]),
-                "sanitized_to": "AASM_5_class",
-            }
-            try:
-                self.cache.atomic_save_array(lp, labels, meta)
-            except Exception as exc:
-                logger.warning(f"Could not cache labels for {spec.subject_id}: {exc}")
+            self._save_labels_cache(lp, spec, labels)
 
         return labels.astype(np.int64, copy=False)
 
+    def _pipeline_output_shape(self) -> Tuple[int, ...]:
+        """Per-epoch output shape of the pipeline (e.g. ``(29, 129)`` for the spectrogram presets).
+
+        Used to zero-fill a subject for which no requested channel could be resolved: its zeros must have the
+        same trailing shape as the signals of the other subjects, or batches mixing them cannot be stacked.
+        """
+        cached = getattr(self, "_fallback_out_shape", None)
+        if cached is not None:
+            return cached
+        shape: Tuple[int, ...] = ()
+        try:
+            fs = 100.0
+            req = self.channels[0] if self.channels else "EEG"
+            stub = ResolvedChannel(
+                request=req,
+                physical=req,
+                modality=infer_channel_modality(str(req)).name,
+                fs_in=fs,
+                is_differential=False,
+            )
+            compiled = self._pipeline_for_channel(stub).compile(fs)
+            n = int(round(self.epoch_length_sec * fs))
+            out = self._epoch_and_run_pipeline(
+                np.zeros(n, dtype=np.float32), fs, compiled, stub
+            )
+            shape = tuple(out.shape[1:])
+        except Exception as exc:  # fall back to the historical 1-D zeros
+            logger.warning(
+                f"[{self.DATASET_NAME}] could not infer the pipeline output shape: {exc}"
+            )
+        self._fallback_out_shape = shape
+        return shape
+
+    def _save_labels_cache(self, lp, spec: SubjectSpec, labels: np.ndarray) -> None:
+        meta = {
+            "dataset": self.DATASET_NAME,
+            "subject_id": spec.subject_id,
+            "n_epochs": int(labels.shape[0]),
+            "sanitized_to": "AASM_5_class",
+            "labels_revision": int(self.LABELS_REVISION),
+        }
+        try:
+            self.cache.atomic_save_array(lp, labels, meta)
+        except Exception as exc:
+            logger.warning(f"Could not cache labels for {spec.subject_id}: {exc}")
+
     def _load_or_compute_channel(
         self, spec: SubjectSpec, resolved: ResolvedChannel
     ) -> np.ndarray:
@@ -1190,7 +1281,7 @@ class BasePhysioDataset(Dataset):
 
         # Fill in missing channels with zeros (same shape as resolved ones)
         if _ref_shape is None:
-            _ref_shape = (requested_len,)
+            _ref_shape = (requested_len,) + self._pipeline_output_shape()
         for key in channel_order:
             if key not in signals_raw:
                 signals_raw[key] = np.zeros(_ref_shape, dtype=np.float32)

{physioex-2.0.2 → physioex-2.0.4}/physioex/data/datasets/__init__.py

@@ -15,6 +15,7 @@ from physioex.data.datasets.alzheimers import AlzheimersDataset
 from physioex.data.datasets.parkinsons import ParkinsonsDataset
 from physioex.data.datasets.shhs import SHHSDataset
 from physioex.data.datasets.stages import STAGESDataset
+from physioex.data.datasets.ccshs import CCSHSDataset
 
 REGISTRY = {
     "hmc": HMCDataset,
@@ -29,6 +30,7 @@ REGISTRY = {
     "parkinsons": ParkinsonsDataset,
     "shhs": SHHSDataset,
     "stages": STAGESDataset,
+    "ccshs": CCSHSDataset,
 }
 
 
@@ -55,6 +57,7 @@ __all__ = [
     "ParkinsonsDataset",
     "SHHSDataset",
     "STAGESDataset",
+    "CCSHSDataset",
     "REGISTRY",
     "get_dataset",
     "available_datasets",

{physioex-2.0.2 → physioex-2.0.4}/physioex/data/datasets/alzheimers.py

@@ -41,6 +41,9 @@ class AlzheimersDataset(BasePhysioDataset):
 
     DATASET_NAME = "alzheimers"
     DATASET_SUBDIR = "AlzheimerData"
+    #: 2: labels are indexed by event onset (gaps stay -1); revision-1 caches are compacted and misaligned.
+    #: 3: labels past the end of the EDF are dropped (truncated recordings).
+    LABELS_REVISION = 3
     DEFAULT_EPOCH_LENGTH_SEC = 30.0
 
     # Channel preference lists.
@@ -206,4 +209,10 @@ class AlzheimersDataset(BasePhysioDataset):
         for epoch_idx, stage in rows:
             labels[epoch_idx] = stage
 
-        return labels
+        # Some EDFs are shorter than the annotated night (truncated recordings): labels
+        # past the end of the signal are dropped instead of being paired with zero padding.
+        import pyedflib
+
+        with pyedflib.EdfReader(str(spec.edf_path)) as f:
+            n_signal = int(f.getFileDuration() // self.epoch_length_sec)
+        return labels[:n_signal]