eegdash 0.3.9.dev170082126__tar.gz → 0.4.0__tar.gz

{eegdash-0.3.9.dev170082126/eegdash.egg-info → eegdash-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eegdash
-Version: 0.3.9.dev170082126
+Version: 0.4.0
 Summary: EEG data for machine learning
 Author-email: Young Truong <dt.young112@gmail.com>, Arnaud Delorme <adelorme@gmail.com>, Aviv Dotan <avivd220@gmail.com>, Oren Shriki <oren70@gmail.com>, Bruno Aristimunha <b.aristimunha@gmail.com>
 License-Expression: GPL-3.0-only
@@ -25,25 +25,24 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: braindecode>=1.0
-Requires-Dist: mne_bids>=0.16.0
+Requires-Dist: mne_bids>=0.17.0
 Requires-Dist: numba
 Requires-Dist: numpy
 Requires-Dist: pandas
-Requires-Dist: pybids
 Requires-Dist: pymongo
 Requires-Dist: python-dotenv
 Requires-Dist: s3fs
-Requires-Dist: scipy
 Requires-Dist: tqdm
-Requires-Dist: xarray
 Requires-Dist: h5io>=0.2.4
 Requires-Dist: pymatreader
 Requires-Dist: eeglabio
 Requires-Dist: tabulate
 Requires-Dist: docstring_inheritance
+Requires-Dist: rich
 Provides-Extra: tests
 Requires-Dist: pytest; extra == "tests"
 Requires-Dist: pytest-cov; extra == "tests"
+Requires-Dist: pytest-sugar; extra == "tests"
 Requires-Dist: codecov; extra == "tests"
 Requires-Dist: pytest_cases; extra == "tests"
 Requires-Dist: pytest-benchmark; extra == "tests"
@@ -57,6 +56,7 @@ Requires-Dist: sphinx_gallery; extra == "docs"
 Requires-Dist: sphinx_rtd_theme; extra == "docs"
 Requires-Dist: pydata-sphinx-theme; extra == "docs"
 Requires-Dist: sphinx-autobuild; extra == "docs"
+Requires-Dist: sphinx-copybutton; extra == "docs"
 Requires-Dist: sphinx-sitemap; extra == "docs"
 Requires-Dist: numpydoc; extra == "docs"
 Requires-Dist: memory_profiler; extra == "docs"
@@ -64,10 +64,14 @@ Requires-Dist: ipython; extra == "docs"
 Requires-Dist: lightgbm; extra == "docs"
 Requires-Dist: plotly; extra == "docs"
 Requires-Dist: nbformat; extra == "docs"
+Requires-Dist: graphviz; extra == "docs"
+Provides-Extra: digestion
+Requires-Dist: pybids; extra == "digestion"
 Provides-Extra: all
 Requires-Dist: eegdash[docs]; extra == "all"
 Requires-Dist: eegdash[dev]; extra == "all"
 Requires-Dist: eegdash[tests]; extra == "all"
+Requires-Dist: eegdash[digestion]; extra == "all"
 Dynamic: license-file
 
 # EEG-Dash
@@ -86,22 +90,6 @@ To leverage recent and ongoing advancements in large-scale computational methods
 
 The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will incorporate a subset of the data converted from NEMAR, which includes 330 MEEG BIDS-formatted datasets, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
 
-## Featured data
-
-The following HBN datasets are currently featured on EEGDash. Documentation about these datasets is available [here](https://neuromechanist.github.io/data/hbn/).
-
-| DatasetID | Participants | Files | Sessions | Population | Channels | Is 10-20? | Modality | Size |
-|---|---|---|---|---|---|---|---|---|
-| [ds005505](https://nemar.org/dataexplorer/detail?dataset_id=ds005505) | 136 | 5393 | 1 | Healthy | 129 | other | Visual | 103 GB |
-| [ds005506](https://nemar.org/dataexplorer/detail?dataset_id=ds005506) | 150 | 5645 | 1 | Healthy | 129 | other | Visual | 112 GB |
-| [ds005507](https://nemar.org/dataexplorer/detail?dataset_id=ds005507) | 184 | 7273 | 1 | Healthy | 129 | other | Visual | 140 GB |
-| [ds005508](https://nemar.org/dataexplorer/detail?dataset_id=ds005508) | 324 | 13393 | 1 | Healthy | 129 | other | Visual | 230 GB |
-| [ds005510](https://nemar.org/dataexplorer/detail?dataset_id=ds005510) | 135 | 4933 | 1 | Healthy | 129 | other | Visual | 91 GB |
-| [ds005512](https://nemar.org/dataexplorer/detail?dataset_id=ds005512) | 257 | 9305 | 1 | Healthy | 129 | other | Visual | 157 GB |
-| [ds005514](https://nemar.org/dataexplorer/detail?dataset_id=ds005514) | 295 | 11565 | 1 | Healthy | 129 | other | Visual | 185 GB |
-
-A total of [246 other datasets](datasets.md) are also available through EEGDash. 
-
 ## Data format
 
 EEGDash queries return a **Pytorch Dataset** formatted to facilitate machine learning (ML) and deep learning (DL) applications. PyTorch Datasets are the best format for EEGDash queries because they provide an efficient, scalable, and flexible structure for machine learning (ML) and deep learning (DL) applications. They allow seamless integration with PyTorch’s DataLoader, enabling efficient batching, shuffling, and parallel data loading, which is essential for training deep learning models on large EEG datasets.
@@ -113,47 +101,11 @@ EEGDash datasets are processed using the popular [braindecode](https://braindeco
 ## EEG-Dash usage
 
 ### Install
-Use your preferred Python environment manager with Python > 3.9 to install the package.
+Use your preferred Python environment manager with Python > 3.10 to install the package.
 * To install the eegdash package, use the following command: `pip install eegdash`
 * To verify the installation, start a Python session and type: `from eegdash import EEGDash`
 
-### Data access
-
-To use the data from a single subject, enter:
-
-```python
-from eegdash import EEGDashDataset
-
-ds_NDARDB033FW5 = EEGDashDataset(
-    {"dataset": "ds005514", "task":
-     "RestingState", "subject": "NDARDB033FW5"}, 
-     cache_dir="."
-)
-```
-
-This will search and download the metadata for the task **RestingState** for subject **NDARDB033FW5** in BIDS dataset **ds005514**. The actual data will not be downloaded at this stage. Following standard practice, data is only downloaded once it is processed. The **ds_NDARDB033FW5** object is a fully functional braindecode dataset, which is itself a PyTorch dataset. This [tutorial](https://github.com/sccn/EEGDash/blob/develop/notebooks/tutorial_eoec.ipynb) shows how to preprocess the EEG data, extracting portions of the data containing eyes-open and eyes-closed segments, then perform eyes-open vs. eyes-closed classification using a (shallow) deep-learning model. 
-
-To use the data from multiple subjects, enter:
-
-```python
-from eegdash import EEGDashDataset
-
-ds_ds005505rest = EEGDashDataset(
-    {"dataset": "ds005505", "task": "RestingState"}, target_name="sex", cache_dir=".
-)
-```
-
-This will search and download the metadata for the task 'RestingState' for all subjects in BIDS dataset 'ds005505' (a total of 136). As above, the actual data will not be downloaded at this stage so this command is quick to execute. Also, the target class for each subject is assigned using the target_name parameter. This means that this object is ready to be directly fed to a deep learning model, although the [tutorial script](https://github.com/sccn/EEGDash/blob/develop/notebooks/tutorial_sex_classification.ipynb) performs minimal processing on it, prior to training a deep-learning model. Because 14 gigabytes of data are downloaded, this tutorial takes about 10 minutes to execute.
-
-### Automatic caching
-
-By default, EEGDash caches downloaded data under a single, consistent folder:
-
-- If ``EEGDASH_CACHE_DIR`` is set in your environment, that path is used.
-- Else, if MNE’s ``MNE_DATA`` config is set, that path is used to align with other EEG tooling.
-- Otherwise, ``.eegdash_cache`` in the current working directory is used.
-
-This means that if you run the tutorial [scripts](https://github.com/sccn/EEGDash/tree/develop/notebooks), the data will only be downloaded the first time the script is executed and reused thereafter.
+Please check our tutorial webpages to explore what you can do with [eegdash](https://eegdash.org/)! 
 
 ## Education -- Coming soon...
 

eegdash-0.4.0/README.md

@@ -0,0 +1,45 @@
+# EEG-Dash
+
+[![PyPI version](https://img.shields.io/pypi/v/eegdash)](https://pypi.org/project/eegdash/)
+[![Docs](https://img.shields.io/badge/docs-stable-brightgreen.svg)](https://sccn.github.io/eegdash)
+
+[![License: GPL-2.0-or-later](https://img.shields.io/badge/License-GPL--2.0--or--later-blue.svg)](LICENSE)
+[![Python versions](https://img.shields.io/pypi/pyversions/eegdash.svg)](https://pypi.org/project/eegdash/)
+[![Downloads](https://pepy.tech/badge/eegdash)](https://pepy.tech/project/eegdash)
+<!-- [![Coverage](https://img.shields.io/codecov/c/github/sccn/eegdash)](https://codecov.io/gh/sccn/eegdash) -->
+
+To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
+
+## Data source
+
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will incorporate a subset of the data converted from NEMAR, which includes 330 MEEG BIDS-formatted datasets, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Data format
+
+EEGDash queries return a **Pytorch Dataset** formatted to facilitate machine learning (ML) and deep learning (DL) applications. PyTorch Datasets are the best format for EEGDash queries because they provide an efficient, scalable, and flexible structure for machine learning (ML) and deep learning (DL) applications. They allow seamless integration with PyTorch’s DataLoader, enabling efficient batching, shuffling, and parallel data loading, which is essential for training deep learning models on large EEG datasets.
+
+## Data preprocessing
+
+EEGDash datasets are processed using the popular [braindecode](https://braindecode.org/stable/index.html) library. In fact, EEGDash datasets are braindecode datasets, which are themselves PyTorch datasets. This means that any preprocessing possible on braindecode datasets is also possible on EEGDash datasets. Refer to [braindecode](https://braindecode.org/stable/index.html) tutorials for guidance on preprocessing EEG data.
+
+## EEG-Dash usage
+
+### Install
+Use your preferred Python environment manager with Python > 3.10 to install the package.
+* To install the eegdash package, use the following command: `pip install eegdash`
+* To verify the installation, start a Python session and type: `from eegdash import EEGDash`
+
+Please check our tutorial webpages to explore what you can do with [eegdash](https://eegdash.org/)! 
+
+## Education -- Coming soon...
+
+We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. Events for 2025 will be announced via the EEGLABNEWS mailing list. Be sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
+
+## About EEG-DaSh
+
+EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego (UCSD) and Ben-Gurion University (BGU) in Israel. 
+
+![Screenshot 2024-10-03 at 09 14 06](https://github.com/user-attachments/assets/327639d3-c3b4-46b1-9335-37803209b0d3)
+
+
+

eegdash-0.4.0/docs/Makefile

@@ -0,0 +1,51 @@
+# Minimal makefile for Sphinx documentation
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+PKG ?= eegdash
+APIDIR := $(SOURCEDIR)/api
+
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: apidoc
+apidoc:
+	# Generate full API docs, then prune duplicates covered by autosummary
+	@SPHINX_APIDOC_OPTIONS=members,undoc-members,show-inheritance,noindex \
+		python -m sphinx.ext.apidoc -f -e -M -o "$(APIDIR)/dataset" "../$(PKG)"
+	# Remove top-level package page and modules covered elsewhere
+
+
+# Standard build runs examples
+html: apidoc 
+
+# Fast build: do NOT execute examples (sphinx-gallery)
+.PHONY: html-noplot
+html-noplot: apidoc 
+	@python prepare_summary_tables.py ../eegdash/ $(BUILDDIR)
+	@$(SPHINXBUILD) -M html "$(SOURCEDIR)" "$(BUILDDIR)" \
+		$(SPHINXOPTS) -D sphinx_gallery_conf.plot_gallery=0 $(O)
+
+# Custom clean target to remove generated API docs and build files
+.PHONY: clean
+clean:
+	@echo "Removing generated API documentation..."
+	@rm -rf "$(APIDIR)/dataset"
+	@rm -rf "$(APIDIR)/generated"
+	@echo "Removing generated dataset pages..."
+	@rm -rf "$(APIDIR)/datasets"
+	@rm -f "$(APIDIR)/api_dataset.rst"
+	@echo "Removing other generated directories..."
+	@rm -rf "$(SOURCEDIR)/generated"
+	@rm -rf "$(SOURCEDIR)/gen_modules"
+	@echo "Removing build directory..."
+	@rm -rf "$(BUILDDIR)"
+	@echo "Clean completed."
+
+.PHONY: help apidoc
+Makefile: ;          
+
+%: Makefile
+	@python prepare_summary_tables.py ../eegdash/ $(BUILDDIR)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

eegdash-0.4.0/docs/build/html/_downloads/06c8d94b7e0b8be2de39fdc122dd12bb/tutorial_challenge_2.ipynb

@@ -0,0 +1,261 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "# For tips on running notebooks in Google Colab:\n# `pip install eegdash`\n%matplotlib inline"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n.. meta::\n   :html_theme.sidebar_secondary.remove: true\n\n# Challenge 2: Predicting the p-factor from EEG\n   :depth: 2\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" target=\"https://colab.research.google.com/github/eeg2025/startkit/blob/main/challenge_2.ipynb\" alt=\"Open In Colab\">\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Preliminary notes\nBefore we begin, I just want to make a deal with you, ok?\nThis is a community competition with a strong open-source foundation.\nWhen I say open-source, I mean volunteer work.\n\nSo, if you see something that does not work or could be improved, first, **please be kind**, and\nwe will fix it together on GitHub, okay?\n\nThe entire decoding community will only go further when we stop\nsolving the same problems over and over again, and it starts working together.\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Overview\nThe psychopathology factor (P-factor) is a widely recognized construct in mental health research, representing a common underlying dimension of psychopathology across various disorders.\nCurrently, the P-factor is often assessed using self-report questionnaires or clinician ratings, which can be subjective, prone to bias, and time-consuming.\n**The Challenge 2** consists of developing a model to predict the P-factor from EEG recordings.\n\nThe challenge encourages learning physiologically meaningful signal representations and discovery of reproducible biomarkers.\nModels of any size should emphasize robust, interpretable features that generalize across subjects,\nsessions, and acquisition sites.\n\nUnlike a standard in-distribution classification task, this regression problem stresses out-of-distribution robustness\nand extrapolation. The goal is not only to minimize error on seen subjects, but also to transfer effectively to unseen data.\nEnsure the dataset is available locally. If not, see the\n[dataset download guide](https://eeg2025.github.io/data/#downloading-the-data)_.\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Contents of this start kit\n<div class=\"alert alert-info\"><h4>Note</h4><p>If you need additional explanations on the\n   :doc:`EEGChallengeDataset\n   </api/dataset/eegdash.dataset.EEGChallengeDataset>` class, dataloading,\n   [braindecode](https://braindecode.org/stable/models/models_table.html)_'s\n   deep learning models, or brain decoding in general, please refer to the\n   start-kit of challenge 1 which delves deeper into these topics.</p></div>\n\nMore contents will be released during the competition inside the\n:mod:`eegdash` [examples webpage](https://eeglab.org/EEGDash/generated/auto_examples/index.html)_.\n\n.. admonition:: Prerequisites\n   :class: important\n\n   The tutorial assumes prior knowledge of:\n\n   - Standard neural network architectures (e.g., CNNs)\n   - Optimization by batch gradient descent and backpropagation\n   - Overfitting, early stopping, and regularization\n   - Some knowledge of PyTorch\n   - Basic familiarity with EEG and preprocessing\n   - An appreciation for open-source work :)\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Install dependencies on Colab\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>These installs are optional; skip on local environments\n   where you already have the dependencies installed.</p></div>\n\n```bash\npip install eegdash\n```\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Imports\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from pathlib import Path\nimport math\nimport os\nimport random\nfrom joblib import Parallel, delayed\n\nimport torch\nfrom torch.utils.data import DataLoader\nfrom torch import optim\nfrom torch.nn.functional import l1_loss\nfrom braindecode.preprocessing import create_fixed_length_windows\nfrom braindecode.datasets.base import EEGWindowsDataset, BaseConcatDataset, BaseDataset\nfrom braindecode.models import EEGNeX\nfrom eegdash import EEGChallengeDataset"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "<div class=\"alert alert-danger\"><h4>Warning</h4><p>In case of Colab, before starting, make sure you're on a GPU instance\n   for faster training! If running on Google Colab, please request a GPU runtime\n   by clicking `Runtime/Change runtime type` in the top bar menu, then selecting\n   'T4 GPU' under 'Hardware accelerator'.</p></div>\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Identify whether a CUDA-enabled GPU is available\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nif device == \"cuda\":\n    msg = \"CUDA-enabled GPU found. Training should be faster.\"\nelse:\n    msg = (\n        \"No GPU found. Training will be carried out on CPU, which might be \"\n        \"slower.\\n\\nIf running on Google Colab, you can request a GPU runtime by\"\n        \" clicking\\n`Runtime/Change runtime type` in the top bar menu, then \"\n        \"selecting 'T4 GPU'\\nunder 'Hardware accelerator'.\"\n    )\nprint(msg)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Understanding the P-factor regression task.\n\nThe psychopathology factor (P-factor) is a widely recognized construct in mental health research, representing a common underlying dimension of psychopathology across various disorders.\nThe P-factor is thought to reflect the shared variance among different psychiatric conditions, suggesting that individuals with higher P-factor scores may be more vulnerable to a range of mental health issues.\nCurrently, the P-factor is often assessed using self-report questionnaires or clinician ratings, which can be subjective, prone to bias, and time-consuming.\nIn the dataset of this challenge, the P-factor was assessed using the Child\nBehavior Checklist (CBCL) [McElroy et al., (2017)](https://doi.org/10.1111/jcpp.12849)_.\n\nThe goal of Challenge 2 is to develop a model to predict the P-factor from EEG recordings.\n**The feasibility of using EEG data for this purpose is still an open question**.\nThe solution may involve finding meaningful representations of the EEG data that correlate with the P-factor scores.\nThe challenge encourages learning physiologically meaningful signal representations and discovery of reproducible biomarkers.\nIf contestants are successful in this task, it could pave the way for more objective and efficient assessments of the P-factor in clinical settings.\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Define local path and (down)load the data\nIn this challenge 2 example, we load the EEG 2025 release using\n:doc:`EEGChallengeDataset </api/dataset/eegdash.dataset.EEGChallengeDataset>`.\n**Note:** in this example notebook, we load the contrast change detection task from one mini release only as an example. Naturally, you are encouraged to train your models on all complete releases, using data from all the tasks you deem relevant.\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "The first step is to define the cache folder!\nMatch tests' cache layout under ~/eegdash_cache/eeg_challenge_cache\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "DATA_DIR = (Path.home() / \"eegdash_cache\" / \"eeg_challenge_cache\").resolve()\n\n# Creating the path if it does not exist\nDATA_DIR.mkdir(parents=True, exist_ok=True)\n\n# We define the list of releases to load.\n# Here, only release 5 is loaded.\nrelease_list = [\"R5\"]\n\nall_datasets_list = [\n    EEGChallengeDataset(\n        release=release,\n        task=\"contrastChangeDetection\",\n        mini=True,\n        description_fields=[\n            \"subject\",\n            \"session\",\n            \"run\",\n            \"task\",\n            \"age\",\n            \"gender\",\n            \"sex\",\n            \"p_factor\",\n        ],\n        cache_dir=DATA_DIR,\n    )\n    for release in release_list\n]\nprint(\"Datasets loaded\")\nsub_rm = [\"NDARWV769JM7\"]"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Combine the datasets into a single one\nHere, we combine the datasets from the different releases into a single\n``BaseConcatDataset`` object.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "all_datasets = BaseConcatDataset(all_datasets_list)\nprint(all_datasets.description)\n\nraws = Parallel(n_jobs=os.cpu_count())(\n    delayed(lambda d: d.raw)(d) for d in all_datasets.datasets\n)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Inspect your data\nWe can check what is inside the dataset consuming the\nMNE-object inside the Braindecode dataset.\n\nThe following snippet, if uncommented, will show the first 10 seconds of the raw EEG signal.\nWe can also inspect the data further by looking at the events and annotations.\nWe strongly recommend you to take a look into the details and check how the events are structured.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "raw = all_datasets.datasets[0].raw  # mne.io.Raw object\n\nprint(raw.info)\n\nraw.plot(duration=10, scalings=\"auto\", show=True)\n\nprint(raw.annotations)\n\nSFREQ = 100"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Wrap the data into a PyTorch-compatible dataset\nThe class below defines a dataset wrapper that will extract 2-second windows,\nuniformly sampled over the whole signal. In addition, it will add useful information\nabout the extracted windows, such as the p-factor, the subject or the task.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "class DatasetWrapper(BaseDataset):\n    def __init__(self, dataset: EEGWindowsDataset, crop_size_samples: int, seed=None):\n        self.dataset = dataset\n        self.crop_size_samples = crop_size_samples\n        self.rng = random.Random(seed)\n\n    def __len__(self):\n        return len(self.dataset)\n\n    def __getitem__(self, index):\n        X, _, crop_inds = self.dataset[index]\n\n        # P-factor label:\n        p_factor = self.dataset.description[\"p_factor\"]\n        p_factor = float(p_factor)\n\n        # Additional information:\n        infos = {\n            \"subject\": self.dataset.description[\"subject\"],\n            \"sex\": self.dataset.description[\"sex\"],\n            \"age\": float(self.dataset.description[\"age\"]),\n            \"task\": self.dataset.description[\"task\"],\n            \"session\": self.dataset.description.get(\"session\", None) or \"\",\n            \"run\": self.dataset.description.get(\"run\", None) or \"\",\n        }\n\n        # Randomly crop the signal to the desired length:\n        i_window_in_trial, i_start, i_stop = crop_inds\n        assert i_stop - i_start >= self.crop_size_samples, f\"{i_stop=} {i_start=}\"\n        start_offset = self.rng.randint(0, i_stop - i_start - self.crop_size_samples)\n        i_start = i_start + start_offset\n        i_stop = i_start + self.crop_size_samples\n        X = X[:, start_offset : start_offset + self.crop_size_samples]\n\n        return X, p_factor, (i_window_in_trial, i_start, i_stop), infos\n\n\n# We filter out certain recordings, create fixed length windows and finally make use of our `DatasetWrapper`."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Filter out recordings that are too short\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "all_datasets = BaseConcatDataset(\n    [\n        ds\n        for ds in all_datasets.datasets\n        if ds.description.subject not in sub_rm\n        and ds.raw.n_times >= 4 * SFREQ\n        and len(ds.raw.ch_names) == 129\n        and not math.isnan(ds.description[\"p_factor\"])\n    ]\n)\n\n# Create 4-seconds windows with 2-seconds stride\nwindows_ds = create_fixed_length_windows(\n    all_datasets,\n    window_size_samples=4 * SFREQ,\n    window_stride_samples=2 * SFREQ,\n    drop_last_window=True,\n)\n\n# Wrap each sub-dataset in the windows_ds\nwindows_ds = BaseConcatDataset(\n    [DatasetWrapper(ds, crop_size_samples=2 * SFREQ) for ds in windows_ds.datasets]\n)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Inspect the label distribution\n\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "import numpy as np\nfrom skorch.helper import SliceDataset\n\ny_label = np.array(list(SliceDataset(windows_ds, 1)))\n\n# Plot histogram of the response times with matplotlib\nimport matplotlib.pyplot as plt\n\nfig, ax = plt.subplots(figsize=(10, 5))\nax.hist(y_label)\nax.set_title(\"Response Time Distribution\")\nax.set_xlabel(\"Response Time (s)\")\nax.set_ylabel(\"Count\")\nplt.tight_layout()\nplt.show()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Define, train and save a model\n ---------------------------------\n Now we have our pytorch dataset necessary for the training!\n\n Below, we define a simple EEGNeX model from Braindecode.\n All the braindecode models expect the input to be of shape (batch_size, n_channels, n_times)\n and have a test coverage about the behavior of the model.\n However, you can use any pytorch model you want.\n#######################################################################\n Initialize model\n ----------------\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "model = EEGNeX(n_chans=129, n_outputs=1, n_times=2 * SFREQ).to(device)\n\n# Specify optimizer\noptimizer = optim.Adamax(params=model.parameters(), lr=0.002)\n\nprint(model)\n\n\n# Finally, we can train our model. Here we define a simple training loop using pure PyTorch.\n# In this example, we only train for a single epoch. Feel free to increase the number of epochs.\n# Create PyTorch Dataloader\n\nnum_workers = (\n    0  # Set num_workers to 0 to avoid multiprocessing issues in notebooks/tutorials.\n)\ndataloader = DataLoader(\n    windows_ds, batch_size=128, shuffle=True, num_workers=num_workers\n)\n\nn_epochs = 1\n\n# Train model for 1 epoch\nfor epoch in range(n_epochs):\n    for idx, batch in enumerate(dataloader):\n        # Reset gradients\n        optimizer.zero_grad()\n\n        # Unpack the batch\n        X, y, crop_inds, infos = batch\n        X = X.to(dtype=torch.float32, device=device)\n        y = y.to(dtype=torch.float32, device=device).unsqueeze(1)\n\n        # Forward pass\n        y_pred = model(X)\n\n        # Compute loss\n        loss = l1_loss(y_pred, y)\n        print(f\"Epoch {0} - step {idx}, loss: {loss.item()}\")\n\n        # Gradient backpropagation\n        loss.backward()\n        optimizer.step()\n\n# Finally, we can save the model for later use\ntorch.save(model.state_dict(), \"weights_challenge_2.pt\")\nprint(\"Model saved as 'weights_challenge_2.pt'\")"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.12.11"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}