eegdash 0.0.1__tar.gz

eegdash-0.0.1/EEGDash.egg-info/PKG-INFO

@@ -0,0 +1,72 @@
+Metadata-Version: 2.1
+Name: eegdash
+Version: 0.0.1
+Summary: EEG data for machine learning
+Author-email: Young Truong <dt.young112@gmail.com>, Arnaud Delorme <adelorme@gmail.com>
+License: GNU General Public License
+        
+        Copyright (C) 2024-2025
+        
+        Young Truong, UCSD, dt.young112@gmail.com
+        Arnaud Delorme, UCSD, adelorme@ucsd.edu
+        
+        This program is free software; you can redistribute it and/or modify
+        it under the terms of the GNU General Public License as published by
+        the Free Software Foundation; either version 2 of the License, or
+        (at your option) any later version.
+        
+        This program is distributed in the hope that it will be useful,
+        but WITHOUT ANY WARRANTY; without even the implied warranty of
+        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+        GNU General Public License for more details.
+        
+        You should have received a copy of the GNU General Public License
+        along with this program; if not, write to the Free Software
+        Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1.07  USA
+        
+Project-URL: Homepage, https://github.com/sccn/EEG-Dash-Data
+Project-URL: Issues, https://github.com/sccn/EEG-Dash-Data/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# EEG-Dash
+To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
+
+## Data source
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Data formatting
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+![Screenshot 2024-10-03 at 09 07 28](https://github.com/user-attachments/assets/b30a79bb-0d94-410a-843c-44c3fcea01fc)
+
+## Data access
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
+
+### AWS S3
+
+Coming soon...
+
+### EEG-Dash API
+
+Coming soon...
+
+## Education
+
+We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
+
+## About EEG-DaSh
+
+EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego (UCSD) and Ben-Gurion University (BGU) in Israel. 
+
+![Screenshot 2024-10-03 at 09 14 06](https://github.com/user-attachments/assets/327639d3-c3b4-46b1-9335-37803209b0d3)
+
+
+
+

eegdash-0.0.1/EEGDash.egg-info/SOURCES.txt

@@ -0,0 +1,79 @@
+LICENSE
+README.md
+pyproject.toml
+EEGDash.egg-info/PKG-INFO
+EEGDash.egg-info/SOURCES.txt
+EEGDash.egg-info/dependency_links.txt
+EEGDash.egg-info/top_level.txt
+eegdash/__init__.py
+eegdash/aws_ingest.py
+eegdash/data_utils.py
+eegdash/main.py
+eegdash/signalstore_data_utils.py
+eegdash.egg-info/PKG-INFO
+eegdash.egg-info/SOURCES.txt
+eegdash.egg-info/dependency_links.txt
+eegdash.egg-info/top_level.txt
+eegdash/SignalStore/__init__.py
+eegdash/SignalStore/signalstore/__init__.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/abstract_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/domain_modeling/schema_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/domain_modeling/vocabulary_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/handmade_records/excel_study_organizer_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/axona/axona_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/ReadIntanSpikeFile.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/intan_read_adapter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/load_intan_rhd_format.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/__init__.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/data_to_result.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/get_bytes_per_data_block.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/notch_filter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/qstring.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/read_header.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhd_format/intanutil/read_one_data_block.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/load_intan_rhs_format.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/__init__.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/data_to_result.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/get_bytes_per_data_block.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/notch_filter.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/qstring.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/read_header.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/intan/load_intan_rhs_format/intanutil/read_one_data_block.py
+eegdash/SignalStore/signalstore/adapters/read_adapters/recording_acquisitions/neurodata_without_borders/neurodata_without_borders_read_adapter.py
+eegdash/SignalStore/signalstore/operations/__init__.py
+eegdash/SignalStore/signalstore/operations/handler_executor.py
+eegdash/SignalStore/signalstore/operations/handler_factory.py
+eegdash/SignalStore/signalstore/operations/handlers/base_handler.py
+eegdash/SignalStore/signalstore/operations/handlers/domain/property_model_handlers.py
+eegdash/SignalStore/signalstore/operations/handlers/domain/schema_handlers.py
+eegdash/SignalStore/signalstore/operations/helpers/abstract_helper.py
+eegdash/SignalStore/signalstore/operations/helpers/neuroscikit_extractor.py
+eegdash/SignalStore/signalstore/operations/helpers/neuroscikit_rawio.py
+eegdash/SignalStore/signalstore/operations/helpers/spikeinterface_helper.py
+eegdash/SignalStore/signalstore/operations/helpers/wrappers/neo_wrappers.py
+eegdash/SignalStore/signalstore/operations/helpers/wrappers/nwb_wrappers.py
+eegdash/SignalStore/signalstore/store/__init__.py
+eegdash/SignalStore/signalstore/store/data_access_objects.py
+eegdash/SignalStore/signalstore/store/datafile_adapters.py
+eegdash/SignalStore/signalstore/store/repositories.py
+eegdash/SignalStore/signalstore/store/store_errors.py
+eegdash/SignalStore/signalstore/store/unit_of_work.py
+eegdash/SignalStore/signalstore/store/unit_of_work_provider.py
+eegdash/SignalStore/signalstore/utilities/data_adapters/spike_interface_adapters/si_recording.py
+eegdash/SignalStore/signalstore/utilities/data_adapters/spike_interface_adapters/si_sorter.py
+eegdash/SignalStore/signalstore/utilities/testing/data_mocks.py
+eegdash/SignalStore/signalstore/utilities/tools/dataarrays.py
+eegdash/SignalStore/signalstore/utilities/tools/mongo_records.py
+eegdash/SignalStore/signalstore/utilities/tools/operation_response.py
+eegdash/SignalStore/signalstore/utilities/tools/purge_orchestration_response.py
+eegdash/SignalStore/signalstore/utilities/tools/quantities.py
+eegdash/SignalStore/signalstore/utilities/tools/strings.py
+eegdash/SignalStore/signalstore/utilities/tools/time.py
+eegdash/SignalStore/tests/conftest.py
+eegdash/SignalStore/tests/data/valid_data/data_arrays/make_fake_data.py
+eegdash/SignalStore/tests/unit/test_ci_cd.py
+eegdash/SignalStore/tests/unit/store/conftest.py
+eegdash/SignalStore/tests/unit/store/test_data_access_objects.py
+eegdash/SignalStore/tests/unit/store/test_repositories.py
+eegdash/SignalStore/tests/unit/store/test_unit_of_work.py
+tests/__init__.py

eegdash-0.0.1/EEGDash.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

eegdash-0.0.1/EEGDash.egg-info/top_level.txt

@@ -0,0 +1 @@
+eegdash

eegdash-0.0.1/LICENSE

@@ -0,0 +1,20 @@
+GNU General Public License
+
+Copyright (C) 2024-2025
+
+Young Truong, UCSD, dt.young112@gmail.com
+Arnaud Delorme, UCSD, adelorme@ucsd.edu
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1.07  USA

eegdash-0.0.1/PKG-INFO

@@ -0,0 +1,72 @@
+Metadata-Version: 2.1
+Name: eegdash
+Version: 0.0.1
+Summary: EEG data for machine learning
+Author-email: Young Truong <dt.young112@gmail.com>, Arnaud Delorme <adelorme@gmail.com>
+License: GNU General Public License
+        
+        Copyright (C) 2024-2025
+        
+        Young Truong, UCSD, dt.young112@gmail.com
+        Arnaud Delorme, UCSD, adelorme@ucsd.edu
+        
+        This program is free software; you can redistribute it and/or modify
+        it under the terms of the GNU General Public License as published by
+        the Free Software Foundation; either version 2 of the License, or
+        (at your option) any later version.
+        
+        This program is distributed in the hope that it will be useful,
+        but WITHOUT ANY WARRANTY; without even the implied warranty of
+        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+        GNU General Public License for more details.
+        
+        You should have received a copy of the GNU General Public License
+        along with this program; if not, write to the Free Software
+        Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1.07  USA
+        
+Project-URL: Homepage, https://github.com/sccn/EEG-Dash-Data
+Project-URL: Issues, https://github.com/sccn/EEG-Dash-Data/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# EEG-Dash
+To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
+
+## Data source
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Data formatting
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+![Screenshot 2024-10-03 at 09 07 28](https://github.com/user-attachments/assets/b30a79bb-0d94-410a-843c-44c3fcea01fc)
+
+## Data access
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
+
+### AWS S3
+
+Coming soon...
+
+### EEG-Dash API
+
+Coming soon...
+
+## Education
+
+We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
+
+## About EEG-DaSh
+
+EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego (UCSD) and Ben-Gurion University (BGU) in Israel. 
+
+![Screenshot 2024-10-03 at 09 14 06](https://github.com/user-attachments/assets/327639d3-c3b4-46b1-9335-37803209b0d3)
+
+
+
+

eegdash-0.0.1/README.md

@@ -0,0 +1,37 @@
+# EEG-Dash
+To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications. 
+
+## Data source
+The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
+
+## Data formatting
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+![Screenshot 2024-10-03 at 09 07 28](https://github.com/user-attachments/assets/b30a79bb-0d94-410a-843c-44c3fcea01fc)
+
+## Data access
+The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
+
+The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally. 
+
+### AWS S3
+
+Coming soon...
+
+### EEG-Dash API
+
+Coming soon...
+
+## Education
+
+We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
+
+## About EEG-DaSh
+
+EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego (UCSD) and Ben-Gurion University (BGU) in Israel. 
+
+![Screenshot 2024-10-03 at 09 14 06](https://github.com/user-attachments/assets/327639d3-c3b4-46b1-9335-37803209b0d3)
+
+
+
+

eegdash-0.0.1/eegdash/SignalStore/signalstore/__init__.py

@@ -0,0 +1,3 @@
+from eegdash.SignalStore.signalstore.store.unit_of_work_provider import UnitOfWorkProvider
+
+__all__ = ['UnitOfWorkProvider']

eegdash-0.0.1/eegdash/SignalStore/signalstore/adapters/read_adapters/abstract_read_adapter.py

@@ -0,0 +1,13 @@
+from abc import ABC, abstractmethod
+
+class AbstractReadAdapter(ABC):
+
+    def __iter__(self):
+        return self.read().__iter__()
+
+    def __next__(self):
+        return self.read().__next__()
+
+    @abstractmethod
+    def read(self):
+        raise NotImplementedError('AbstractReadAdapter.read() not implemented.')

eegdash-0.0.1/eegdash/SignalStore/signalstore/adapters/read_adapters/domain_modeling/schema_read_adapter.py

@@ -0,0 +1,16 @@
+from signalstore.adapters.read_adapters.abstract_read_adapter import AbstractReadAdapter
+import json
+from upath import UPath
+
+class SchemaReadAdapter(AbstractReadAdapter):
+    def __init__(self, directory):
+        self.dir = UPath(directory)
+
+    def read(self):
+        """Reads JSON files that conform to the Neuroscikit data model schemata.
+        """
+        for json_filepath in self.dir.glob('*.json'):
+            with open(json_filepath) as f:
+                yield dict(json.load(f))
+
+

eegdash-0.0.1/eegdash/SignalStore/signalstore/adapters/read_adapters/domain_modeling/vocabulary_read_adapter.py

@@ -0,0 +1,19 @@
+from signalstore.adapters.read_adapters.abstract_read_adapter import AbstractReadAdapter
+
+import yaml
+
+class VocabularyReadAdapter(AbstractReadAdapter):
+    def __init__(self, filepath):
+        self.filepath = filepath
+
+    def read(self):
+        """Reads a YAML file and converts each data object into an xarray.DataArray with
+        the appropriate dimensions, coordinates and metadata attributes for the
+        Neuroscikit data model.
+        """
+        with open(self.filepath) as f:
+            yaml_dict = yaml.load(f, Loader=yaml.FullLoader)
+        for key, value in yaml_dict.items():
+            record = {"name": key}
+            record.update(value)
+            yield record

eegdash-0.0.1/eegdash/SignalStore/signalstore/adapters/read_adapters/handmade_records/excel_study_organizer_read_adapter.py

@@ -0,0 +1,114 @@
+from signalstore.operations.importers.adapters.abstract_read_adapter import AbstractReadAdapter
+
+import openpyxl as xl
+
+class ExcelStudyOrganizerReadAdapter(AbstractReadAdapter):
+    def __init__(self, path):
+        self.path = path
+        self.wb = xl.load_workbook(path)
+        self.ws = self.wb.active
+        self.tables = [str(table) for table in self.wb.sheetnames]
+
+    def read(self):
+        for table in self.tables:
+            for record in self._get_table_records(table):
+                yield record
+
+    def read_records(self):
+        records = []
+        for table in self.tables:
+            records.extend(list(self._get_table_records(table)))
+        return records
+
+    def read_records_by_table(self):
+        records = {}
+        for table in self.tables:
+            records[str(table).lower()] = list(self._get_table_records(table))
+        return records
+
+    def _classify_table(self, table):
+        # check if name column is unique
+        has_unique_keys = self._has_unique_keys(table)
+
+        # check if the table columns include only the name column,
+        # an attribute column and a value column
+        is_attr_value_format = self._is_attr_value_format(table)
+
+        if not has_unique_keys and is_attr_value_format:
+            return 'attribute'
+        elif has_unique_keys and not is_attr_value_format:
+            return 'record'
+        else:
+            error_string = f'Could not classify table {table}.'
+            if not has_unique_keys:
+                error_string += '\nTable does not have unique keys.'
+            if not is_attr_value_format:
+                error_string += '\nTable is not in attribute-value format.'
+            raise StudyOrganizerKeyError(error_string)
+
+    def _has_unique_keys(self, table):
+        ws = self.wb[table]
+        keys = [str(cell.value).lower() for cell in ws['A'] if cell.value is not None]
+        return len(keys) == len(set(keys))
+
+    def _is_attr_value_format(self, table):
+        ws = self.wb[table]
+        columns = [str(cell.value).lower() for cell in ws[1]]
+        if columns[0] == f'name' and columns[1] == 'attribute' and columns[2] == 'value' and len(columns) == 3:
+            return True
+        else:
+            return False
+
+    def _get_table_records(self, table):
+        table_type = self._classify_table(table)
+        readers = {'record': self._get_simple_table_records,
+                   'attribute': self._get_attribute_table_records}
+        records = readers[table_type](table)
+        for record in records:
+            yield record
+
+    def _get_simple_table_records(self, table):
+        self.ws = self.wb[table]
+        columns = [str(cell.value).lower() for cell in self.ws[1]]
+        self._validate_columns(columns, table)
+        for row in self.ws.iter_rows(min_row=2):
+            record = {}
+            for i, column in enumerate(columns):
+                value = row[i].value
+                if value is not None:
+                    record[column] = value
+            if record != {}:
+                record['type'] = table
+                yield record
+
+    def _get_attribute_table_records(self, table):
+        self.ws = self.wb[table]
+        columns = [str(cell.value).lower() for cell in self.ws[1]]
+        self._validate_columns(columns, table)
+        attr_records = []
+        for row in self.ws.iter_rows(min_row=2):
+            record = {}
+            for i, column in enumerate(columns):
+                record[column] = row[i].value
+            attr_records.append(record)
+        records = {}
+        for attr_record in attr_records:
+            rkey = attr_record['name']
+            if rkey not in records.keys() and rkey is not None:
+                records[rkey] = {'name': rkey, 'type': table}
+            if attr_record['value'] is not None:
+                records[rkey][attr_record['attribute']] = attr_record['value']
+        for record in records.values():
+            if record != {}:
+                yield record
+
+    def _validate_columns(self, columns, table_name):
+        if not self._first_column_is_key(columns, table_name):
+            raise StudyOrganizerKeyError(f'First column must be a "name" column, but is {columns[0]}.')
+
+    def _first_column_is_key(self, columns, table_name):
+        return str(columns[0]) == f'name'
+
+
+class StudyOrganizerKeyError(KeyError):
+    pass