brainscore-vision 2.2.4__tar.gz → 2.2.6__tar.gz

brainscore_vision-2.2.6/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.2
+Name: brainscore_vision
+Version: 2.2.6
+Summary: The Brain-Score library enables model comparisons to behavioral and neural experiments
+License: 
+        MIT License
+        
+        Copyright (c) 2017, Martin Schrimpf and Jon Prescott-Roy
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+        
+        
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy<2
+Requires-Dist: brainscore-core
+Requires-Dist: result-caching
+Requires-Dist: importlib-metadata<5
+Requires-Dist: scikit-learn
+Requires-Dist: scipy
+Requires-Dist: opencv-python
+Requires-Dist: h5py
+Requires-Dist: tqdm
+Requires-Dist: gitpython
+Requires-Dist: fire
+Requires-Dist: jupyter
+Requires-Dist: pybtex
+Requires-Dist: peewee
+Requires-Dist: psycopg2-binary
+Requires-Dist: networkx
+Requires-Dist: eva-decord
+Requires-Dist: psutil
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest_check; extra == "test"
+Requires-Dist: pytest-mock; extra == "test"
+Requires-Dist: pytest-timeout; extra == "test"
+Requires-Dist: torch; extra == "test"
+Requires-Dist: torchvision; extra == "test"
+Requires-Dist: matplotlib; extra == "test"
+Requires-Dist: pytest-mock; extra == "test"
+
+[![Build Status](https://app.travis-ci.com/brain-score/vision.svg?token=vqt7d2yhhpLGwHsiTZvT&branch=master)](https://app.travis-ci.com/brain-score/vision)
+[![Documentation Status](https://readthedocs.org/projects/brain-score/badge/?version=latest)](https://brain-score.readthedocs.io/en/latest/?badge=latest)
+[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](code_of_conduct.md) 
+
+Brain-Score is a platform to evaluate computational models of brain function 
+on their match to brain measurements in primate vision. 
+The intent of Brain-Score is to adopt many (ideally all) the experimental benchmarks in the field
+for the purpose of model testing, falsification, and comparison.
+To that end, Brain-Score operationalizes experimental data into quantitative benchmarks 
+that any model candidate following the [`BrainModel`](brainscore_vision/model_interface.py) interface can be scored on.
+
+Note that you can only access a limited set of public benchmarks when running locally. To score a model on all benchmarks, submit it via the [brain-score.org website](http://www.brain-score.org).
+
+See the [documentation](https://brain-score.readthedocs.io) for more details, e.g. for submitting a [model](https://brain-score.readthedocs.io/en/latest/modules/model_tutorial.html) or [benchmark](https://brain-score.readthedocs.io/en/latest/modules/benchmark_tutorial.html) to Brain-Score. For a step-by-step walkthrough on submitting models to the Brain-Score website, see these [web tutorials](https://www.brain-score.org/tutorial/).
+
+See these [code examples](https://github.com/brain-score/vision/blob/master/examples) on scoring models, retrieving data, using and defining benchmarks and metrics.
+These [previous examples](https://github.com/brain-score/candidate_models/blob/master/examples/score-model.ipynb) might be helpful, but their usage has been deprecated after the 2.0 update.
+
+Brain-Score is made by and for the community. 
+To contribute, please [send in a pull request](https://github.com/brain-score/vision/pulls).
+
+
+## Local installation
+
+You will need Python = 3.11 and pip >= 18.1.
+
+`pip install brainscore-vision`
+
+Test if the installation is successful by scoring a model on a public benchmark:
+
+```python
+import brainscore_vision
+
+benchmark = brainscore_vision.load_benchmark('MajajHong2015public.IT-pls')
+model = my_model()
+score = benchmark(model)
+
+# >  <xarray.Score ()>
+# >  array(0.07637264)
+# >  Attributes:
+# >  Attributes:
+# >      error:                 <xarray.Score ()>\narray(0.00548197)
+# >      raw:                   <xarray.Score ()>\narray(0.22545106)\nAttributes:\...
+# >      ceiling:               <xarray.DataArray ()>\narray(0.81579938)\nAttribut...
+# >      model_identifier:      my-model
+# >      benchmark_identifier:  MajajHong2015public.IT-pls
+```
+
+Some steps may take minutes because data has to be downloaded during first-time use.
+
+## Environment Variables
+
+| Variable               | Description                                                                                                                           |
+|------------------------|---------------------------------------------------------------------------------------------------------------------------------------|
+| RESULTCACHING_HOME     | directory to cache results (benchmark ceilings) in, `~/.result_caching` by default (see https://github.com/brain-score/result_caching) |
+
+
+## License
+
+MIT license
+
+
+## Troubleshooting
+
+<details>
+<summary>`ValueError: did not find HDF5 headers` during netcdf4 installation</summary>
+pip seems to fail properly setting up the HDF5_DIR required by netcdf4.
+Use conda: `conda install netcdf4`
+</details>
+
+<details>
+<summary>repeated runs of a benchmark / model do not change the outcome even though code was changed</summary>
+results (scores, activations) are cached on disk using https://github.com/mschrimpf/result_caching.
+Delete the corresponding file or directory to clear the cache.
+</details>
+
+
+## CI environment
+
+Add CI related build commands to `test_setup.sh`. The script is executed in CI environment for unittests.
+
+
+## References
+
+If you use Brain-Score in your work, please cite 
+["Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?"](https://www.biorxiv.org/content/10.1101/407007v2) (technical) and 
+["Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence"](https://www.cell.com/neuron/fulltext/S0896-6273(20)30605-X) (perspective) 
+as well as the respective benchmark sources.
+
+```bibtex
+@article{SchrimpfKubilius2018BrainScore,
+  title={Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?},
+  author={Martin Schrimpf and Jonas Kubilius and Ha Hong and Najib J. Majaj and Rishi Rajalingham and Elias B. Issa and Kohitij Kar and Pouya Bashivan and Jonathan Prescott-Roy and Franziska Geiger and Kailyn Schmidt and Daniel L. K. Yamins and James J. DiCarlo},
+  journal={bioRxiv preprint},
+  year={2018},
+  url={https://www.biorxiv.org/content/10.1101/407007v2}
+}
+
+@article{Schrimpf2020integrative,
+  title={Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence},
+  author={Schrimpf, Martin and Kubilius, Jonas and Lee, Michael J and Murty, N Apurva Ratan and Ajemian, Robert and DiCarlo, James J},
+  journal={Neuron},
+  year={2020},
+  url={https://www.cell.com/neuron/fulltext/S0896-6273(20)30605-X}
+}
+```

brainscore_vision-2.2.6/README.md

@@ -0,0 +1,106 @@
+[![Build Status](https://app.travis-ci.com/brain-score/vision.svg?token=vqt7d2yhhpLGwHsiTZvT&branch=master)](https://app.travis-ci.com/brain-score/vision)
+[![Documentation Status](https://readthedocs.org/projects/brain-score/badge/?version=latest)](https://brain-score.readthedocs.io/en/latest/?badge=latest)
+[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](code_of_conduct.md) 
+
+Brain-Score is a platform to evaluate computational models of brain function 
+on their match to brain measurements in primate vision. 
+The intent of Brain-Score is to adopt many (ideally all) the experimental benchmarks in the field
+for the purpose of model testing, falsification, and comparison.
+To that end, Brain-Score operationalizes experimental data into quantitative benchmarks 
+that any model candidate following the [`BrainModel`](brainscore_vision/model_interface.py) interface can be scored on.
+
+Note that you can only access a limited set of public benchmarks when running locally. To score a model on all benchmarks, submit it via the [brain-score.org website](http://www.brain-score.org).
+
+See the [documentation](https://brain-score.readthedocs.io) for more details, e.g. for submitting a [model](https://brain-score.readthedocs.io/en/latest/modules/model_tutorial.html) or [benchmark](https://brain-score.readthedocs.io/en/latest/modules/benchmark_tutorial.html) to Brain-Score. For a step-by-step walkthrough on submitting models to the Brain-Score website, see these [web tutorials](https://www.brain-score.org/tutorial/).
+
+See these [code examples](https://github.com/brain-score/vision/blob/master/examples) on scoring models, retrieving data, using and defining benchmarks and metrics.
+These [previous examples](https://github.com/brain-score/candidate_models/blob/master/examples/score-model.ipynb) might be helpful, but their usage has been deprecated after the 2.0 update.
+
+Brain-Score is made by and for the community. 
+To contribute, please [send in a pull request](https://github.com/brain-score/vision/pulls).
+
+
+## Local installation
+
+You will need Python = 3.11 and pip >= 18.1.
+
+`pip install brainscore-vision`
+
+Test if the installation is successful by scoring a model on a public benchmark:
+
+```python
+import brainscore_vision
+
+benchmark = brainscore_vision.load_benchmark('MajajHong2015public.IT-pls')
+model = my_model()
+score = benchmark(model)
+
+# >  <xarray.Score ()>
+# >  array(0.07637264)
+# >  Attributes:
+# >  Attributes:
+# >      error:                 <xarray.Score ()>\narray(0.00548197)
+# >      raw:                   <xarray.Score ()>\narray(0.22545106)\nAttributes:\...
+# >      ceiling:               <xarray.DataArray ()>\narray(0.81579938)\nAttribut...
+# >      model_identifier:      my-model
+# >      benchmark_identifier:  MajajHong2015public.IT-pls
+```
+
+Some steps may take minutes because data has to be downloaded during first-time use.
+
+## Environment Variables
+
+| Variable               | Description                                                                                                                           |
+|------------------------|---------------------------------------------------------------------------------------------------------------------------------------|
+| RESULTCACHING_HOME     | directory to cache results (benchmark ceilings) in, `~/.result_caching` by default (see https://github.com/brain-score/result_caching) |
+
+
+## License
+
+MIT license
+
+
+## Troubleshooting
+
+<details>
+<summary>`ValueError: did not find HDF5 headers` during netcdf4 installation</summary>
+pip seems to fail properly setting up the HDF5_DIR required by netcdf4.
+Use conda: `conda install netcdf4`
+</details>
+
+<details>
+<summary>repeated runs of a benchmark / model do not change the outcome even though code was changed</summary>
+results (scores, activations) are cached on disk using https://github.com/mschrimpf/result_caching.
+Delete the corresponding file or directory to clear the cache.
+</details>
+
+
+## CI environment
+
+Add CI related build commands to `test_setup.sh`. The script is executed in CI environment for unittests.
+
+
+## References
+
+If you use Brain-Score in your work, please cite 
+["Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?"](https://www.biorxiv.org/content/10.1101/407007v2) (technical) and 
+["Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence"](https://www.cell.com/neuron/fulltext/S0896-6273(20)30605-X) (perspective) 
+as well as the respective benchmark sources.
+
+```bibtex
+@article{SchrimpfKubilius2018BrainScore,
+  title={Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?},
+  author={Martin Schrimpf and Jonas Kubilius and Ha Hong and Najib J. Majaj and Rishi Rajalingham and Elias B. Issa and Kohitij Kar and Pouya Bashivan and Jonathan Prescott-Roy and Franziska Geiger and Kailyn Schmidt and Daniel L. K. Yamins and James J. DiCarlo},
+  journal={bioRxiv preprint},
+  year={2018},
+  url={https://www.biorxiv.org/content/10.1101/407007v2}
+}
+
+@article{Schrimpf2020integrative,
+  title={Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence},
+  author={Schrimpf, Martin and Kubilius, Jonas and Lee, Michael J and Murty, N Apurva Ratan and Ajemian, Robert and DiCarlo, James J},
+  journal={Neuron},
+  year={2020},
+  url={https://www.cell.com/neuron/fulltext/S0896-6273(20)30605-X}
+}
+```

brainscore_vision-2.2.6/brainscore_vision/data/baker2022/__init__.py

@@ -0,0 +1,40 @@
+from brainio.assemblies import BehavioralAssembly
+from brainscore_vision import data_registry, stimulus_set_registry, load_stimulus_set
+from brainscore_vision.data_helpers.s3 import load_assembly_from_s3, load_stimulus_set_from_s3
+
+
+# normal distortion:
+stimulus_set_registry['Baker2022_normal_distortion'] = lambda: load_stimulus_set_from_s3(
+    identifier='Baker2022_normal_distortion',
+    bucket="brainscore-storage/brainio-brainscore",
+    csv_sha1="17d4db7458a29a787d12bb29c34e91daef1872bf",
+    zip_sha1="2c726abaf081c8a9828269a559222f8c6eea0e4f",
+    csv_version_id="null",
+    zip_version_id="null")
+
+data_registry['Baker2022_normal_distortion'] = lambda: load_assembly_from_s3(
+    identifier='Baker2022_normal_distortion',
+    version_id="null",
+    sha1="46c79a48bf2dbd995a9305d8fbc03a134a852e17",
+    bucket="brainscore-storage/brainio-brainscore",
+    cls=BehavioralAssembly,
+    stimulus_set_loader=lambda: load_stimulus_set('Baker2022_normal_distortion'),
+)
+
+# inverted distortion:
+stimulus_set_registry['Baker2022_inverted_distortion'] = lambda: load_stimulus_set_from_s3(
+    identifier='Baker2022_inverted_distortion',
+    bucket="brainscore-storage/brainio-brainscore",
+    csv_sha1="91e452e4651024c2b382694edfcbc7bdc6c3189b",
+    zip_sha1="4740a096af994c2232350469c664e53796f17a05",
+    csv_version_id="null",
+    zip_version_id="null")
+
+data_registry['Baker2022_inverted_distortion'] = lambda: load_assembly_from_s3(
+    identifier='Baker2022_inverted_distortion',
+    version_id="null",
+    sha1="b76fb57b25a58ca68db78d188fd0a783e1dcaf73",
+    bucket="brainscore-storage/brainio-brainscore",
+    cls=BehavioralAssembly,
+    stimulus_set_loader=lambda: load_stimulus_set('Baker2022_inverted_distortion'),
+)

brainscore_vision-2.2.6/brainscore_vision/data/baker2022/data_packaging/inverted_distortion_data_assembly.py

@@ -0,0 +1,43 @@
+import numpy as np
+
+from brainio.assemblies import BehavioralAssembly
+from brainio.packaging import package_data_assembly
+import pandas as pd
+
+'''
+Experiment Information:
+https://www.sciencedirect.com/science/article/pii/S2589004222011853#sec9
+ - (Data/subjects are from experiment 1 in paper above)
+
+ - 32 subjects 
+ - 40 images for each condition (whole, fragmented, frankenstein) = 120 images/subject shown
+ - 120 * 32 = 3840 total images shown 
+ - However there are only 3706 trials total, for data issues (according to Baker)
+ - 9-way AFC, from set: {bear, bunny, cat, elephant, frog, lizard, tiger, turtle, wolf}
+ 
+'''
+
+# initial csv to dataframe processing:
+all_subjects = pd.read_csv('human_data/inverted.csv')
+
+
+# construct the assembly
+assembly = BehavioralAssembly(all_subjects['RSP'],
+                              coords={
+                                  'stimulus_id': ('presentation', all_subjects['FileName']),
+                                  'subject': ('presentation', all_subjects['Subj']),
+                                  'orientation': ('presentation', all_subjects['Inv']),
+                                  'condition': ('presentation', all_subjects['Config']),
+                                  'truth': ('presentation', all_subjects['Animal']),
+                                  'correct': ('presentation', all_subjects['RSP']),
+                              },
+                              dims=['presentation']
+                              )
+
+# give the assembly an identifier name
+assembly.name = 'Baker2022_inverted_distortion'
+
+# upload to S3
+package_data_assembly('brainscore-storage/brainio_brainscore', assembly, assembly_identifier=assembly.name,
+                      stimulus_set_identifier='Baker2022_inverted_distortion',
+                      assembly_class_name="BehavioralAssembly", bucket_name="brainscore-storage/brainio_brainscore")

brainscore_vision-2.2.6/brainscore_vision/data/baker2022/data_packaging/inverted_distortion_stimulus_set.py

@@ -0,0 +1,81 @@
+from pathlib import Path
+from brainio.stimuli import StimulusSet
+from brainio.packaging import package_stimulus_set
+import pandas as pd
+
+stimuli = []
+image_paths = {}
+stimuli_directory = 'images'
+
+
+'''
+Dataset Information:
+- From Baker 2022: https://www.sciencedirect.com/science/article/pii/S2589004222011853#sec9
+- 4320 trials total, across 12 subjects. Each subject saw the same 360 images, composed of:
+    - 90 inverted-whole, 90 inverted-frankenstein, 90-normal frankenstein, 90-normal whole.
+    - of those 4 subtypes, they are further broken into the same 10 images of the set of 9 categories below.
+
+Fields:
+1) ground_truth: the base object, in set above
+2) image_type:  a string in the set {w, f} for {whole, frankenstein} respectively.
+    Note: inverted only hase whole and frankenstein images, no fragmented 
+3) image_number: a number {1,2...40} indicates the image variation
+
+'''
+categories = ['bear', 'bunny', 'cat', 'elephant', 'frog', 'lizard', 'tiger', 'turtle', 'wolf']
+images_actually_shown = pd.read_csv('human_data/inverted.csv')
+images_actually_shown = set(images_actually_shown["FileName"].values)
+
+for filepath in Path(stimuli_directory).glob('*.jpg'):
+
+    # entire name of image file:
+    image_id = filepath.stem
+
+    if image_id in images_actually_shown:
+        import re
+        match = re.match(r"([a-z]+)([0-9]+)", image_id, re.I)
+        if match:
+            items = match.groups()
+        else:
+            items = ["", ""]
+
+        # ground truth
+        ground_truth = items[0]
+
+        # image_number:
+        image_number = items[1]
+
+        # parse the needed image type letter
+        if ground_truth in categories:
+            image_type = "w"
+        else:
+            image_type = ground_truth[0]
+            ground_truth = ground_truth[1:]
+
+        if "inv" in ground_truth:
+            ground_truth = ground_truth.replace("inv", "")
+        elif "nv" in ground_truth:
+            ground_truth = ground_truth.replace("nv", "")
+
+        image_paths[image_id] = filepath
+        stimuli.append({
+            'stimulus_id': image_id,
+            'animal': ground_truth,
+            'image_type': "w" if image_type is "i" else image_type,
+            'image_type_entire': "whole" if image_type is "w" else "frankenstein",
+            'image_number': image_number,
+            "orientation": "normal" if "inv" not in image_id else "inverted",
+        })
+    else:
+        pass
+
+stimuli = StimulusSet(stimuli)
+stimuli.stimulus_paths = image_paths
+
+
+# give the StimulusSet an identifier name
+stimuli.name = 'Baker2022_inverted_distortion'
+
+# upload to S3
+package_stimulus_set("brainscore-storage/brainio_brainscore", stimuli, stimulus_set_identifier=stimuli.name,
+                     bucket_name="brainscore-storage/brainio_brainscore")

brainscore_vision-2.2.6/brainscore_vision/data/baker2022/data_packaging/normal_distortion_data_assembly.py

@@ -0,0 +1,46 @@
+import numpy as np
+
+from brainio.assemblies import BehavioralAssembly
+from brainio.packaging import package_data_assembly
+import pandas as pd
+
+'''
+Experiment Information:
+ - https://www.sciencedirect.com/science/article/pii/S2589004222011853#sec9
+ - Data/subjects are from experiment 1 in paper above
+
+ - 32 subjects 
+ - 40 images for each condition (whole, fragmented, frankenstein) = 120 images/subject shown
+ - 120 * 32 = 3840 total images shown 
+ - However there are only 3706 trials total, for data issues (according to Baker)
+ - 9-way AFC, from set: {bear, bunny, cat, elephant, frog, lizard, tiger, turtle, wolf}
+ 
+'''
+
+# initial csv to dataframe processing:
+all_subjects = pd.read_csv('human_data/human_data.csv')
+
+# all images presented in this assembly are normal (non-inverted)
+all_subjects["orientation"] = "normal"
+
+# construct the assembly
+assembly = BehavioralAssembly(all_subjects['Correct?'],
+                              coords={
+                                  'stimulus_id': ('presentation', all_subjects['image_shown']),
+                                  'subject': ('presentation', all_subjects['Subj']),
+                                  'condition': ('presentation', all_subjects['Frankensteinonfig']),
+                                  'truth': ('presentation', all_subjects['Animal']),
+                                  'correct': ('presentation', all_subjects['Correct?']),
+                                  'orientation': ('presentation', all_subjects['orientation']),
+                              },
+                              dims=['presentation']
+                              )
+
+# give the assembly an identifier name
+assembly.name = 'Baker2022_normal_distortion'
+
+
+# upload to S3
+package_data_assembly('brainscore-storage/brainio_brainscore', assembly, assembly_identifier=assembly.name,
+                      stimulus_set_identifier='Baker2022_normal_distortion',
+                      assembly_class_name="BehavioralAssembly", bucket_name="brainscore-storage/brainio_brainscore")

brainscore_vision-2.2.6/brainscore_vision/data/baker2022/data_packaging/normal_distortion_stimulus_set.py

@@ -0,0 +1,94 @@
+from pathlib import Path
+from brainio.stimuli import StimulusSet
+from brainio.packaging import package_stimulus_set
+import pandas as pd
+import re
+
+stimuli = []
+image_paths = {}
+stimuli_directory = 'images'
+
+
+'''
+Dataset Information:
+
+- From Baker 2022: https://www.sciencedirect.com/science/article/pii/S2589004222011853#sec9
+- 1800 images total: 1080 normal distortion (** this stimulus set **), 720 inverted distortion
+- normal/inverted distortion -> contains the three classes of distortion:
+    1) normal image
+    2) fragmented image
+    3) frankenstein image
+
+- For the normal distortion, there are 40 images/condition, with 9 categories: 
+    categories are: {bear, bunny, cat, elephant, frog, lizard, tiger, turtle, wolf}
+- For inverted distortion, there are a variable number of images/condition, but 720 total unique images
+
+Fields:
+
+1) ground_truth: the base object, in set above
+2) image_type:  a string in the set {w, f, o} for {whole, frankenstein, fragmented} respectively. 
+3) image_number: a number {1,2...40} indicates the image variation
+
+'''
+categories = ['bear', 'bunny', 'cat', 'elephant', 'frog', 'lizard', 'tiger', 'turtle', 'wolf']
+images_actually_shown = pd.read_csv('human_data/images_shown.csv')
+images_actually_shown = pd.concat([images_actually_shown[col] for col in images_actually_shown]).values
+
+for filepath in Path(stimuli_directory).glob('*.jpg'):
+
+    # entire name of image file:
+    image_id = filepath.stem
+
+    if f"{image_id}.jpg" in images_actually_shown:
+        match = re.match(r"([a-z]+)([0-9]+)", image_id, re.I)
+        if match:
+            items = match.groups()
+        else:
+            items = ["", ""]
+
+        # ground truth
+        ground_truth = items[0]
+
+        # image_number:
+        image_number = items[1]
+
+        # parse the needed image type letter
+        if ground_truth in categories:
+            image_type = "w"
+        else:
+            image_type = ground_truth[0]
+            ground_truth = ground_truth[1:]
+
+        if "inv" in ground_truth:
+            ground_truth = ground_truth.replace("inv", "")
+        elif "nv" in ground_truth:
+            ground_truth = ground_truth.replace("nv", "")
+
+        if image_type is "w":
+            image_type_entire = "whole"
+        elif image_type is "o":
+            image_type_entire = "fragmented"
+        else:
+            image_type_entire = "frankenstein"
+
+        image_paths[image_id] = filepath
+        stimuli.append({
+            'stimulus_id': image_id,
+            'animal': ground_truth,
+            'image_type': "w" if image_type is "i" else image_type,
+            'image_type_entire': image_type_entire,
+            'image_number': image_number,
+            "orientation": "normal" if "inv" not in image_id else "inverted",
+        })
+
+stimuli = StimulusSet(stimuli)
+
+stimuli.stimulus_paths = image_paths
+
+# remove all inverted stimuli
+stimuli = stimuli[stimuli["orientation"] == "normal"]
+stimuli.name = 'Baker2022_normal_distortion'  # give the StimulusSet an identifier name
+
+# upload to S3
+package_stimulus_set("brainscore-storage/brainio_brainscore", stimuli, stimulus_set_identifier=stimuli.name,
+                     bucket_name="brainscore-storage/brainio_brainscore")

brainscore_vision-2.2.6/brainscore_vision/data/barbumayo2019/__init__.py

@@ -0,0 +1,23 @@
+from brainscore_vision import stimulus_set_registry
+from brainscore_vision.data_helpers.s3 import load_stimulus_set_from_s3
+
+BIBTEX = """@inproceedings{NEURIPS2019_97af07a1,
+ author = {Barbu, Andrei and Mayo, David and Alverio, Julian and Luo, William and Wang, Christopher and Gutfreund, Dan and Tenenbaum, Josh and Katz, Boris},
+ booktitle = {Advances in Neural Information Processing Systems},
+ editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
+ pages = {},
+ publisher = {Curran Associates, Inc.},
+ title = {ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models},
+ url = {https://proceedings.neurips.cc/paper/2019/file/97af07a14cacba681feacf3012730892-Paper.pdf},
+ volume = {32},
+ year = {2019}
+}"""
+
+# stimulus set
+stimulus_set_registry['BarbuMayo2019'] = lambda: load_stimulus_set_from_s3(
+    identifier="BarbuMayo2019",
+    bucket="brainscore-storage/brainio-brainscore",
+    csv_sha1="e4d8888ccb6beca28636e6698e7beb130e278e12",
+    zip_sha1="1365eb2a7231516806127a7d2a908343a7ac9464",
+    csv_version_id="null",
+    zip_version_id="null")

brainscore_vision-2.2.6/brainscore_vision/data/bashivankar2019/__init__.py

@@ -0,0 +1,52 @@
+from brainio.assemblies import NeuronRecordingAssembly
+from brainscore_vision import data_registry, stimulus_set_registry, load_stimulus_set
+from brainscore_vision.data_helpers.s3 import load_assembly_from_s3, load_stimulus_set_from_s3
+
+BIBTEX = """@article{bashivan2019neural,
+  title={Neural population control via deep image synthesis},
+  author={Bashivan, Pouya and Kar, Kohitij and DiCarlo, James J},
+  journal={Science},
+  volume={364},
+  number={6439},
+  pages={eaav9436},
+  year={2019},
+  publisher={American Association for the Advancement of Science}
+}"""
+
+# assemblies
+data_registry['BashivanKar2019.naturalistic'] = lambda: load_assembly_from_s3(
+    identifier="dicarlo.BashivanKar2019.naturalistic",
+    version_id="null",
+    sha1="1ec2f32ef800f0c6e15879d883be1d55b51b8b67",
+    bucket="brainscore-storage/brainio-brainscore",
+    cls=NeuronRecordingAssembly,
+    stimulus_set_loader=lambda: load_stimulus_set('BashivanKar2019.naturalistic'),
+)
+
+data_registry['BashivanKar2019.synthetic'] = lambda: load_assembly_from_s3(
+    identifier="dicarlo.BashivanKar2019.synthetic",
+    version_id="null",
+    sha1="f687c8d26f8943dc379dbcbe94d3feb148400c6b",
+    bucket="brainscore-storage/brainio-brainscore",
+    cls=NeuronRecordingAssembly,
+    stimulus_set_loader=lambda: load_stimulus_set('BashivanKar2019.synthetic'),
+)
+
+# stimulus sets
+# naturalistic
+stimulus_set_registry['BashivanKar2019.naturalistic'] = lambda: load_stimulus_set_from_s3(
+    identifier="BashivanKar2019.naturalistic",
+    bucket="brainscore-storage/brainio-brainscore",
+    csv_sha1="48ef84282552b8796142ffe7d0d2c632f8ef061a",
+    zip_sha1="d7b71b431cf23d435395205f1e38036a9e10acca",
+    csv_version_id="null",
+    zip_version_id="null")
+
+# synthetic
+stimulus_set_registry['BashivanKar2019.synthetic'] = lambda: load_stimulus_set_from_s3(
+    identifier="BashivanKar2019.synthetic",
+    bucket="brainscore-storage/brainio-brainscore",
+    csv_sha1="81da195e9b2a128b228fc4867e23ae6b21bd7abd",
+    zip_sha1="e2de33f25c5c19bcfb400055c1db399d553487e5",
+    csv_version_id="null",
+    zip_version_id="null")