eegdash 0.0.1__tar.gz → 0.0.3__tar.gz

{eegdash-0.0.1/EEGDash.egg-info → eegdash-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: eegdash
-Version: 0.0.1
+Version: 0.0.3
 Summary: EEG data for machine learning
 Author-email: Young Truong <dt.young112@gmail.com>, Arnaud Delorme <adelorme@gmail.com>
 License: GNU General Public License
@@ -32,12 +32,24 @@ Classifier: Operating System :: OS Independent
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: signalstore
 
 # EEG-Dash
 To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
 
 ## Data source
-The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate a subset of the data converted from NEMAR, which includes 330 MEEG BIDS-formatted datasets, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Datasets available
+
+There are currently only two datasets made available for testing purposes. 
+
+| Dataset ID | Description                                                                                 | Participants | Channels        | Task            | NEMAR Link                                                                                      |
+|------------|---------------------------------------------------------------------------------------------|--------------|-----------------|-----------------|------------------------------------------------------------------------------------------------|
+| ds002718   | EEG dataset focused on face processing with MRI for source localization                     | 18           | 70 EEG, 2 EOG   | FaceRecognition | [NEMAR ds002718](https://nemar.org/dataexplorer/detail?dataset_id=ds002718)                    |
+| ds004745   | 8-Channel SSVEP EEG dataset with trials including voluntary movements to introduce artifacts | 6            | 8 EEG           | SSVEP tasks     | [NEMAR ds004745](https://nemar.org/dataexplorer/detail?dataset_id=ds004745)                    |
+
+
 
 ## Data formatting
 The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
@@ -49,15 +61,70 @@ The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep l
 
 The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
 
-### AWS S3
+### Install
+Use your preferred Python environment manager with Python > 3.9 to install the package. Here we show example using Conda environment with Python 3.11.5:
+* Create a new environment Python 3.11.5 -> `conda create --name eegdash python=3.11.5`
+* Switch to the right environment -> `conda activate eegdash`
+* Install dependencies (this is a temporary link that will be updated soon) -> `pip install -r https://raw.githubusercontent.com/sccn/EEG-Dash-Data/refs/heads/develop/requirements.txt`
+* Install _eegdash_ package (this is a temporary link that will be updated soon) -> `pip install -i https://test.pypi.org/simple/ eegdash`
+* Check installation. Start a Python session and type `from eegdash import EEGDash`
+
+### Python data access
+
+To create a local object for accessing the database, use the following code:
+
+```python
+from eegdash import EEGDash
+EEGDashInstance = EEGDash()
+```
+
+Once the object is instantiated, it can be utilized to search datasets. Providing an empty parameter will search the entire database and return all available datasets.
+
+```python
+EEGDashInstance.find({})
+```
+A list of dataset is returned.
+
+```python
+[{'schema_ref': 'eeg_signal',
+  'data_name': 'ds004745_sub-001_task-unnamed_eeg.set',
+  'dataset': 'ds004745',
+  'subject': '001',
+  'task': 'unnamed',
+  'session': '',
+  'run': '',
+  'modality': 'EEG',
+  'sampling_frequency': 1000,
+  'version_timestamp': 0,
+  'has_file': True,
+  'time_of_save': datetime.datetime(2024, 10, 25, 14, 11, 48, 843593, tzinfo=datetime.timezone.utc),
+  'time_of_removal': None}, ...
+
+```
+
+Additionally, users can search for a specific dataset by specifying criteria.
+
+```python
+EEGDashInstance.find({'task': 'FaceRecognition'})
+```
+
+After locating the desired dataset or data record, users can download it locally by executing the following command. This will return an xArray Python object.
+
+```python
+XArrayData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})
+```
+
+Optionally, this is how you may access the raw data for the first record. This will return an numpy array.
 
-Coming soon...
+```python
+npData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})[0].values
+```
 
-### EEG-Dash API
+## Example use
 
-Coming soon...
+This [example](tests/eegdash.ipynb) demonstrates the full workflow from data retrieval with `EEGDash` to model definition, data handling, and training in PyTorch.
 
-## Education
+## Education - Coming soon...
 
 We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
 

{eegdash-0.0.1 → eegdash-0.0.3}/README.md

@@ -2,7 +2,18 @@
 To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
 
 ## Data source
-The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate a subset of the data converted from NEMAR, which includes 330 MEEG BIDS-formatted datasets, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Datasets available
+
+There are currently only two datasets made available for testing purposes. 
+
+| Dataset ID | Description                                                                                 | Participants | Channels        | Task            | NEMAR Link                                                                                      |
+|------------|---------------------------------------------------------------------------------------------|--------------|-----------------|-----------------|------------------------------------------------------------------------------------------------|
+| ds002718   | EEG dataset focused on face processing with MRI for source localization                     | 18           | 70 EEG, 2 EOG   | FaceRecognition | [NEMAR ds002718](https://nemar.org/dataexplorer/detail?dataset_id=ds002718)                    |
+| ds004745   | 8-Channel SSVEP EEG dataset with trials including voluntary movements to introduce artifacts | 6            | 8 EEG           | SSVEP tasks     | [NEMAR ds004745](https://nemar.org/dataexplorer/detail?dataset_id=ds004745)                    |
+
+
 
 ## Data formatting
 The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
@@ -14,15 +25,70 @@ The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep l
 
 The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
 
-### AWS S3
+### Install
+Use your preferred Python environment manager with Python > 3.9 to install the package. Here we show example using Conda environment with Python 3.11.5:
+* Create a new environment Python 3.11.5 -> `conda create --name eegdash python=3.11.5`
+* Switch to the right environment -> `conda activate eegdash`
+* Install dependencies (this is a temporary link that will be updated soon) -> `pip install -r https://raw.githubusercontent.com/sccn/EEG-Dash-Data/refs/heads/develop/requirements.txt`
+* Install _eegdash_ package (this is a temporary link that will be updated soon) -> `pip install -i https://test.pypi.org/simple/ eegdash`
+* Check installation. Start a Python session and type `from eegdash import EEGDash`
+
+### Python data access
+
+To create a local object for accessing the database, use the following code:
+
+```python
+from eegdash import EEGDash
+EEGDashInstance = EEGDash()
+```
+
+Once the object is instantiated, it can be utilized to search datasets. Providing an empty parameter will search the entire database and return all available datasets.
+
+```python
+EEGDashInstance.find({})
+```
+A list of dataset is returned.
+
+```python
+[{'schema_ref': 'eeg_signal',
+  'data_name': 'ds004745_sub-001_task-unnamed_eeg.set',
+  'dataset': 'ds004745',
+  'subject': '001',
+  'task': 'unnamed',
+  'session': '',
+  'run': '',
+  'modality': 'EEG',
+  'sampling_frequency': 1000,
+  'version_timestamp': 0,
+  'has_file': True,
+  'time_of_save': datetime.datetime(2024, 10, 25, 14, 11, 48, 843593, tzinfo=datetime.timezone.utc),
+  'time_of_removal': None}, ...
+
+```
+
+Additionally, users can search for a specific dataset by specifying criteria.
+
+```python
+EEGDashInstance.find({'task': 'FaceRecognition'})
+```
+
+After locating the desired dataset or data record, users can download it locally by executing the following command. This will return an xArray Python object.
+
+```python
+XArrayData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})
+```
+
+Optionally, this is how you may access the raw data for the first record. This will return an numpy array.
 
-Coming soon...
+```python
+npData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})[0].values
+```
 
-### EEG-Dash API
+## Example use
 
-Coming soon...
+This [example](tests/eegdash.ipynb) demonstrates the full workflow from data retrieval with `EEGDash` to model definition, data handling, and training in PyTorch.
 
-## Education
+## Education - Coming soon...
 
 We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
 

{eegdash-0.0.1 → eegdash-0.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "eegdash"
-version = "0.0.1"
+version = "0.0.3"
 authors = [
   { name="Young Truong", email="dt.young112@gmail.com" },
   { name="Arnaud Delorme", email="adelorme@gmail.com" },
@@ -18,7 +18,10 @@ classifiers = [
     "License :: OSI Approved :: MIT License",
     "Operating System :: OS Independent",
 ]
-
+dependencies = [
+    "signalstore"
+]
 [project.urls]
 Homepage = "https://github.com/sccn/EEG-Dash-Data"
-Issues = "https://github.com/sccn/EEG-Dash-Data/issues"
+Issues = "https://github.com/sccn/EEG-Dash-Data/issues"
+

{eegdash-0.0.1/eegdash → eegdash-0.0.3/src}/data_utils.py

@@ -3,11 +3,104 @@ import sys
 from joblib import Parallel, delayed
 import mne
 import numpy as np
+import pandas as pd
 from pathlib import Path
 import re
 import json
+from mne.io import BaseRaw
+from mne._fiff.utils import _find_channels, _read_segments_file
+import s3fs
+import tempfile
+from mne._fiff.utils import _read_segments_file
 
-verbose = False
+class RawEEGDash(BaseRaw):
+    r"""Raw object from EEG-Dash connection with Openneuro S3 file.
+
+    Parameters
+    ----------
+    input_fname : path-like
+        Path to the S3 file
+    eog : list | tuple | 'auto'
+        Names or indices of channels that should be designated EOG channels.
+        If 'auto', the channel names containing ``EOG`` or ``EYE`` are used.
+        Defaults to empty tuple.
+    %(preload)s
+        Note that preload=False will be effective only if the data is stored
+        in a separate binary file.
+    %(uint16_codec)s
+    %(montage_units)s
+    %(verbose)s
+
+    See Also
+    --------
+    mne.io.Raw : Documentation of attributes and methods.
+
+    Notes
+    -----
+    .. versionadded:: 0.11.0
+    """
+
+    def __init__(
+        self,
+        input_fname,
+        metadata,
+        eog=(),
+        preload=False,
+        *,
+        cache_dir='./.eegdash_cache',
+        uint16_codec=None,
+        montage_units="auto",
+        verbose=None,
+    ):
+        '''
+        Get to work with S3 endpoint first, no caching
+        '''
+        # Create a simple RawArray
+        sfreq = metadata['sfreq']  # Sampling frequency
+        n_times = metadata['n_times']
+        ch_names = metadata['ch_names']
+        ch_types = []
+        for ch in metadata['ch_types']:
+            chtype = ch.lower()
+            if chtype == 'heog' or chtype == 'veog':
+                chtype = 'eog'
+            ch_types.append(chtype)
+        info = mne.create_info(ch_names=ch_names, sfreq=sfreq, ch_types=ch_types)
+        self.s3file = input_fname
+        os.makedirs(cache_dir, exist_ok=True)
+        self.filecache = os.path.join(cache_dir, os.path.basename(self.s3file))
+
+        if preload and not os.path.exists(self.filecache):
+            self._download_s3()
+            preload = self.filecache
+
+        super().__init__(
+            info,
+            preload,
+            last_samps=[n_times-1],
+            orig_format="single",
+            verbose=verbose,
+        )
+
+    def _download_s3(self):
+        filesystem = s3fs.S3FileSystem(anon=True, client_kwargs={'region_name': 'us-east-2'})
+        print('s3file', self.s3file)
+        print('filecache', self.filecache)
+        filesystem.download(self.s3file, self.filecache)
+        self.filenames = [self.filecache]
+
+    def _read_segment(
+        self, start=0, stop=None, sel=None, data_buffer=None, *, verbose=None
+    ):
+        if not os.path.exists(self.filecache): # not preload
+            self._download_s3()
+        else: # not preload and file is not cached
+            self.filenames = [self.filecache]
+        return super()._read_segment(start, stop, sel, data_buffer, verbose=verbose)
+    
+    def _read_segment_file(self, data, idx, fi, start, stop, cals, mult):
+        """Read a chunk of data from the file."""
+        _read_segments_file(self, data, idx, fi, start, stop, cals, mult, dtype="<f4")
 
 
 class BIDSDataset():
@@ -20,7 +113,7 @@ class BIDSDataset():
     }
     METADATA_FILE_EXTENSIONS = ['eeg.json', 'channels.tsv', 'electrodes.tsv', 'events.tsv', 'events.json']
     def __init__(self,
-            data_dir=None,                            # location of asr cleaned data 
+            data_dir=None,                            # location of bids dataset 
             dataset='',                               # dataset name
             raw_format='eeglab',                      # format of raw data
         ):                            
@@ -44,9 +137,25 @@ class BIDSDataset():
             self.files = np.load(temp_dir / f'{dataset}_files.npy', allow_pickle=True)
 
     def get_property_from_filename(self, property, filename):
-        lookup = re.search(rf'{property}-(.*?)[_\/]', filename)
+        import platform
+        if platform.system() == "Windows":
+            lookup = re.search(rf'{property}-(.*?)[_\\]', filename)
+        else:
+            lookup = re.search(rf'{property}-(.*?)[_\/]', filename)
         return lookup.group(1) if lookup else ''
 
+    def merge_json_inheritance(self, json_files):
+        '''
+        Merge list of json files found by get_bids_file_inheritance,
+        expecting the order (from left to right) is from lowest level to highest level,
+        and return a merged dictionary
+        '''
+        json_files.reverse()
+        json_dict = {}
+        for f in json_files:
+            json_dict.update(json.load(open(f)))
+        return json_dict
+
     def get_bids_file_inheritance(self, path, basename, extension):
         '''
         Get all files with given extension that applies to the basename file 
@@ -68,7 +177,7 @@ class BIDSDataset():
         for file in os.listdir(path):
             # target_file = path / f"{cur_file_basename}_{extension}"
             if os.path.isfile(path/file):
-                cur_file_basename = file[:file.rfind('_')]
+                cur_file_basename = file[:file.rfind('_')] # TODO: change to just search for any file with extension
                 if file.endswith(extension) and cur_file_basename in basename:
                     filepath = path / file
                     bids_files.append(filepath)
@@ -210,4 +319,21 @@ class BIDSDataset():
         return self.get_property_from_filename('run', data_filepath)
 
     def subject(self, data_filepath):
-        return self.get_property_from_filename('sub', data_filepath)
+        return self.get_property_from_filename('sub', data_filepath)
+
+    def num_channels(self, data_filepath):
+        channels_tsv = pd.read_csv(self.get_bids_metadata_files(data_filepath, 'channels.tsv')[0], sep='\t')
+        return len(channels_tsv)
+
+    def channel_labels(self, data_filepath):
+        channels_tsv = pd.read_csv(self.get_bids_metadata_files(data_filepath, 'channels.tsv')[0], sep='\t')
+        return channels_tsv['name'].tolist()
+    
+    def channel_types(self, data_filepath):
+        channels_tsv = pd.read_csv(self.get_bids_metadata_files(data_filepath, 'channels.tsv')[0], sep='\t')
+        return channels_tsv['type'].tolist()
+            
+    def num_times(self, data_filepath):
+        eeg_jsons = self.get_bids_metadata_files(data_filepath, 'eeg.json')
+        eeg_json_dict = self.merge_json_inheritance(eeg_jsons)
+        return int(eeg_json_dict['SamplingFrequency'] * eeg_json_dict['RecordingDuration'])

{eegdash-0.0.1 → eegdash-0.0.3/src/eegdash.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: eegdash
-Version: 0.0.1
+Version: 0.0.3
 Summary: EEG data for machine learning
 Author-email: Young Truong <dt.young112@gmail.com>, Arnaud Delorme <adelorme@gmail.com>
 License: GNU General Public License
@@ -32,12 +32,24 @@ Classifier: Operating System :: OS Independent
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: signalstore
 
 # EEG-Dash
 To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
 
 ## Data source
-The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate a subset of the data converted from NEMAR, which includes 330 MEEG BIDS-formatted datasets, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Datasets available
+
+There are currently only two datasets made available for testing purposes. 
+
+| Dataset ID | Description                                                                                 | Participants | Channels        | Task            | NEMAR Link                                                                                      |
+|------------|---------------------------------------------------------------------------------------------|--------------|-----------------|-----------------|------------------------------------------------------------------------------------------------|
+| ds002718   | EEG dataset focused on face processing with MRI for source localization                     | 18           | 70 EEG, 2 EOG   | FaceRecognition | [NEMAR ds002718](https://nemar.org/dataexplorer/detail?dataset_id=ds002718)                    |
+| ds004745   | 8-Channel SSVEP EEG dataset with trials including voluntary movements to introduce artifacts | 6            | 8 EEG           | SSVEP tasks     | [NEMAR ds004745](https://nemar.org/dataexplorer/detail?dataset_id=ds004745)                    |
+
+
 
 ## Data formatting
 The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
@@ -49,15 +61,70 @@ The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep l
 
 The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
 
-### AWS S3
+### Install
+Use your preferred Python environment manager with Python > 3.9 to install the package. Here we show example using Conda environment with Python 3.11.5:
+* Create a new environment Python 3.11.5 -> `conda create --name eegdash python=3.11.5`
+* Switch to the right environment -> `conda activate eegdash`
+* Install dependencies (this is a temporary link that will be updated soon) -> `pip install -r https://raw.githubusercontent.com/sccn/EEG-Dash-Data/refs/heads/develop/requirements.txt`
+* Install _eegdash_ package (this is a temporary link that will be updated soon) -> `pip install -i https://test.pypi.org/simple/ eegdash`
+* Check installation. Start a Python session and type `from eegdash import EEGDash`
+
+### Python data access
+
+To create a local object for accessing the database, use the following code:
+
+```python
+from eegdash import EEGDash
+EEGDashInstance = EEGDash()
+```
+
+Once the object is instantiated, it can be utilized to search datasets. Providing an empty parameter will search the entire database and return all available datasets.
+
+```python
+EEGDashInstance.find({})
+```
+A list of dataset is returned.
+
+```python
+[{'schema_ref': 'eeg_signal',
+  'data_name': 'ds004745_sub-001_task-unnamed_eeg.set',
+  'dataset': 'ds004745',
+  'subject': '001',
+  'task': 'unnamed',
+  'session': '',
+  'run': '',
+  'modality': 'EEG',
+  'sampling_frequency': 1000,
+  'version_timestamp': 0,
+  'has_file': True,
+  'time_of_save': datetime.datetime(2024, 10, 25, 14, 11, 48, 843593, tzinfo=datetime.timezone.utc),
+  'time_of_removal': None}, ...
+
+```
+
+Additionally, users can search for a specific dataset by specifying criteria.
+
+```python
+EEGDashInstance.find({'task': 'FaceRecognition'})
+```
+
+After locating the desired dataset or data record, users can download it locally by executing the following command. This will return an xArray Python object.
+
+```python
+XArrayData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})
+```
+
+Optionally, this is how you may access the raw data for the first record. This will return an numpy array.
 
-Coming soon...
+```python
+npData = EEGDashInstance.get({'task': 'FaceRecognition', 'subject': '019'})[0].values
+```
 
-### EEG-Dash API
+## Example use
 
-Coming soon...
+This [example](tests/eegdash.ipynb) demonstrates the full workflow from data retrieval with `EEGDash` to model definition, data handling, and training in PyTorch.
 
-## Education
+## Education - Coming soon...
 
 We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
 

eegdash-0.0.3/src/eegdash.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+LICENSE
+README.md
+pyproject.toml
+src/__init__.py
+src/data_utils.py
+src/main.py
+src/eegdash.egg-info/PKG-INFO
+src/eegdash.egg-info/SOURCES.txt
+src/eegdash.egg-info/dependency_links.txt
+src/eegdash.egg-info/requires.txt
+src/eegdash.egg-info/top_level.txt

eegdash-0.0.3/src/eegdash.egg-info/requires.txt

@@ -0,0 +1 @@
+signalstore

eegdash-0.0.3/src/eegdash.egg-info/top_level.txt

@@ -0,0 +1,4 @@
+DomainModels
+__init__
+data_utils
+main