aind-data-schema 0.0.1__tar.gz

aind-data-schema-0.0.1/.flake8

@@ -0,0 +1,7 @@
+[flake8]
+exclude =
+    .git,
+    __pycache__,
+    build
+max-complexity = 10
+max-line-length = 120

aind-data-schema-0.0.1/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

aind-data-schema-0.0.1/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

aind-data-schema-0.0.1/.github/ISSUE_TEMPLATE/user-story.md

@@ -0,0 +1,27 @@
+---
+name: User story
+about: This template provides a basic structure for user story issues.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+# User story
+As a ..., I want to ..., so I can ...
+
+*Ideally, this is in the issue title, but if not, you can put it here. If so, delete this section.*
+
+# Acceptance criteria
+- [ ] This is something that can be verified to show that this user story is satisfied.
+
+# Sprint Ready Checklist 
+- [ ] 1. Acceptance criteria defined 
+- [ ] 2. Team understands acceptance criteria 
+- [ ] 3. Team has defined solution / steps to satisfy acceptance criteria 
+- [ ] 4. Acceptance criteria is verifiable / testable 
+- [ ] 5. External / 3rd Party dependencies identified
+- [ ] 6. Ticket is prioritized and sized
+
+# Notes
+*Add any helpful notes here.*

aind-data-schema-0.0.1/.github/workflows/tag_and_publish.yml

@@ -0,0 +1,166 @@
+name: Tag and Publish
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  bump_versions_and_update_schemas:
+    runs-on: ubuntu-latest
+    env:
+      OUTPUT_DIR: 'schemas'
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          ref: ${{ env.DEFAULT_BRANCH }}
+          fetch-depth: 0
+          token: ${{ secrets.SERVICE_TOKEN }}
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v3
+        with:
+          python-version: 3.8
+      - name: Install dependencies
+        run: python -m pip install -e .[dev] --no-cache-dir
+      - name: Bump schema versions
+        run: |
+          python -m aind_data_schema.utils.schema_version_bump
+          echo "Finished schema version bump"
+      - name: Update schemas
+        run: |
+          rm -rf $OUTPUT_DIR
+          python -m pip install -e .
+          python -m aind_data_schema.utils.json_writer --output $OUTPUT_DIR
+      - name: Create example schemas
+        run: |
+          cd examples
+          for script in ./*.py; do
+              python "$script"
+          done
+      - name: Commit changes
+        uses: EndBug/add-and-commit@v9
+        with:
+          default_author: github_actions
+          message: "bump schema version [skip actions]"
+          add: '["src", "schemas", "examples"]'
+
+  update_docs:
+    needs: bump_versions_and_update_schemas
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    env:
+      DIAGRAM_DIRECTORY: ./docs/source/_static/diagrams
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          ref: ${{ env.DEFAULT_BRANCH }}
+          fetch-depth: 0
+          token: ${{ secrets.SERVICE_TOKEN }}
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.8
+      - name: Install dependencies
+        run: python -m pip install -e .[dev] --no-cache-dir
+      - name: Generate new rst files
+        run: |
+          sphinx-apidoc -o docs/source/ src
+          sphinx-build -b html docs/source/ docs/build/html
+      - name: Commit changes
+        uses: EndBug/add-and-commit@v9
+        with:
+          default_author: github_actions
+          message: "updates diagrams [skip actions]"
+          add: '["*.png", "*.rst"]'
+
+  tag:
+    needs: update_docs
+    uses: AllenNeuralDynamics/aind-github-actions/.github/workflows/tag.yml@main
+    secrets:
+      SERVICE_TOKEN: ${{ secrets.SERVICE_TOKEN }} # required
+
+  publish:
+    needs: tag
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Pull latest changes
+        run: git pull origin main
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.8
+      - name: Install dependencies
+        run: |
+          pip install --upgrade setuptools wheel twine build
+          python -m build
+          twine check dist/*
+      - name: Publish on PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1.5
+        with:
+          password: ${{ secrets.AIND_PYPI_TOKEN }}
+
+  upload_schemas_prod:
+    needs: publish
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      contents: read
+    env:
+      AWS_DATA_SCHEMA_ROLE: ${{ secrets.AWS_DATA_SCHEMA_ROLE_PROD}}
+      AWS_DATA_SCHEMA_BUCKET: ${{ vars.AWS_DATA_SCHEMA_BUCKET_PROD }}
+      AWS_REGION: ${{ vars.AWS_REGION_PROD }}
+      TEMP_DIR: 'temp_schemas'
+      S3_PREFIX: 'schemas'
+    steps:
+      - uses: actions/checkout@master
+      - name: Pull latest changes
+        run: git pull origin main
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v3
+        with:
+          python-version: 3.8
+      - name: Configure aws credentials
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          role-to-assume: ${{ env.AWS_DATA_SCHEMA_ROLE }}
+          role-session-name: github-schema-upload-session
+          aws-region: ${{ env.AWS_REGION }}
+      - name: Create and upload schemas
+        run: |
+          python -m pip install -e .
+          python -m aind_data_schema.utils.json_writer --output $TEMP_DIR --attach-version
+          python -m pip install awscli
+          aws s3 sync $TEMP_DIR s3://${AWS_DATA_SCHEMA_BUCKET}/$S3_PREFIX
+
+  upload_schemas_dev:
+    needs: upload_schemas_prod
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      contents: read
+    env:
+      AWS_DATA_SCHEMA_ROLE: ${{ secrets.AWS_DATA_SCHEMA_ROLE_DEV }}
+      AWS_DATA_SCHEMA_BUCKET: ${{ vars.AWS_DATA_SCHEMA_BUCKET_DEV }}
+      AWS_REGION : ${{ vars.AWS_REGION_DEV }}
+      TEMP_DIR: 'temp_schemas'
+      S3_PREFIX: 'schemas'
+    steps:
+      - uses: actions/checkout@master
+      - name: Pull latest changes
+        run: git pull origin main
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v3
+        with:
+          python-version: 3.8
+      - name: Configure aws credentials
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          role-to-assume: ${{ env.AWS_DATA_SCHEMA_ROLE }}
+          role-session-name: github-schema-upload-session
+          aws-region: ${{ env.AWS_REGION }}
+      - name: Create and upload schemas
+        run: | 
+          python -m pip install -e .
+          python -m aind_data_schema.utils.json_writer --output $TEMP_DIR --attach-version
+          python -m pip install awscli
+          aws s3 sync $TEMP_DIR s3://${AWS_DATA_SCHEMA_BUCKET}/$S3_PREFIX

aind-data-schema-0.0.1/.gitignore

@@ -0,0 +1,142 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/
+
+# MacOs
+**/.DS_Store
+
+# Jupyter
+notebook/

aind-data-schema-0.0.1/.readthedocs.yaml

@@ -0,0 +1,8 @@
+version: 2
+
+python:
+  install:
+    - method: pip
+      path: .
+      extra_requirements:
+        - dev

aind-data-schema-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Allen Institute for Neural Dynamics
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

aind-data-schema-0.0.1/PKG-INFO

@@ -0,0 +1,200 @@
+Metadata-Version: 2.1
+Name: aind-data-schema
+Version: 0.0.1
+Summary: A library that defines AIND data schema and validates JSON files.
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: dictdiffer
+Requires-Dist: pydantic>2.0
+Requires-Dist: inflection
+Requires-Dist: jsonschema
+Requires-Dist: semver
+Provides-Extra: dev
+Requires-Dist: aind_data_schema[docs]; extra == "dev"
+Requires-Dist: aind_data_schema[linters]; extra == "dev"
+Provides-Extra: linters
+Requires-Dist: codespell; extra == "linters"
+Requires-Dist: black; extra == "linters"
+Requires-Dist: coverage; extra == "linters"
+Requires-Dist: flake8; extra == "linters"
+Requires-Dist: interrogate; extra == "linters"
+Requires-Dist: isort; extra == "linters"
+Provides-Extra: docs
+Requires-Dist: Sphinx; extra == "docs"
+Requires-Dist: sphinx-jinja; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+
+# aind-data-schema
+
+[![License](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
+![Code Style](https://img.shields.io/badge/code%20style-black-black)
+[![Documentation Status](https://readthedocs.org/projects/aind-data-schema/badge/?version=latest)](https://aind-data-schema.readthedocs.io/en/latest/?badge=latest)
+
+A library that defines [AIND](https://alleninstitute.org/what-we-do/brain-science/research/allen-institute-neural-dynamics/) data schema and validates JSON files. 
+
+User documentation available on [readthedocs](https://aind-data-schema.readthedocs.io/en/latest/).
+
+## Overview
+
+This repository contains the schemas needed to ingest and validate metadata that are essential to ensuring [AIND](https://alleninstitute.org/what-we-do/brain-science/research/allen-institute-neural-dynamics/) data collection is completely reproducible. Our general approach is to semantically version core schema classes and include those version numbers in serialized metadata so that we can flexibly evolve the schemas over time without requiring difficult data migrations. In the future, we will provide a browsable list of these classes rendered to [JSONschema](https://json-schema.org/), including all historic versions.
+
+Be aware that this package is still under heavy preliminary development. Expect breaking changes regularly, although we will communicate these through semantic versioning.
+
+A simple example:
+
+```python
+import datetime
+
+from aind_data_schema.subject import Housing, Subject
+
+t = datetime.datetime(2022, 11, 22, 8, 43, 00)
+
+s = Subject(
+    species="Mus musculus",
+    subject_id="12345",
+    sex="Male",
+    date_of_birth=t.date(),
+    genotype="Emx1-IRES-Cre;Camk2a-tTA;Ai93(TITL-GCaMP6f)",
+    housing=Housing(home_cage_enrichment=["Running wheel"], cage_id="123"),
+    background_strain="C57BL/6J",
+)
+
+s.write_standard_file() # writes subject.json
+```
+
+```json
+{
+   "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema-test/main/src/aind_data_schema/subject.py",
+   "schema_version": "0.3.0",
+   "species": "Mus musculus",
+   "subject_id": "12345",
+   "sex": "Male",
+   "date_of_birth": "2022-11-22",
+   "genotype": "Emx1-IRES-Cre;Camk2a-tTA;Ai93(TITL-GCaMP6f)",
+   "mgi_allele_ids": null,
+   "background_strain": "C57BL/6J",
+   "source": null,
+   "rrid": null,
+   "restrictions": null,
+   "breeding_group": null,
+   "maternal_id": null,
+   "maternal_genotype": null,
+   "paternal_id": null,
+   "paternal_genotype": null,
+   "wellness_reports": null,
+   "housing": {
+      "cage_id": "123",
+      "room_id": null,
+      "light_cycle": null,
+      "home_cage_enrichment": [
+         "Running wheel"
+      ],
+      "cohoused_subjects": null
+   },
+   "notes": null
+}
+```
+
+## Installing and Upgrading
+
+To install the latest version:
+```
+pip install aind-data-schema
+```
+
+Every merge to the `main` branch is automatically tagged with a new major/minor/patch version and uploaded to PyPI. To upgrade to the latest version:
+```
+pip install aind-data-schema --upgrade
+```
+
+To develop the code, check out this repo and run the following in the cloned directory: 
+```
+pip install -e .[dev]
+```
+
+## Contributing
+
+If you've found a bug in the schemas or would like to make a minor change, open an [Issue](https://github.com/AllenNeuralDynamics/aind-data-schema-test/issues) on this repository. If you'd like to propose a large change or addition, or generally have a question about how things work, head start a new [Discussion](https://github.com/AllenNeuralDynamics/aind-data-schema-test/discussions)!
+
+
+### Linters and testing
+
+There are several libraries used to run linters, check documentation, and run tests.
+
+- To run tests locally, navigate to AIND-DATA-SCHEMA directory in terminal and run (this will not run any on-line only tests):
+
+```
+python -m unittest
+```
+
+- Please test your changes using the **coverage** library, which will run the tests and log a coverage report:
+
+```
+coverage run -m unittest discover && coverage report
+```
+
+- To test any of the following modules, conda/pip install the relevant package (interrogate, flake8, black, isort), navigate to relevant directory, and run any of the following commands in place of [command]:
+
+```
+[command] -v . 
+```
+
+- Use **interrogate** to check that modules, methods, etc. have been documented thoroughly:
+
+```
+interrogate .
+```
+
+- Use **flake8** to check that code is up to standards (no unused imports, etc.):
+
+```
+flake8 .
+```
+
+- Use **black** to automatically format the code into PEP standards:
+
+```
+black .
+```
+
+- Use **isort** to automatically sort import statements:
+
+```
+isort .
+```
+
+### Pull requests
+
+For internal members, please create a branch. For external members, please fork the repo and open a pull request from the fork. We'll primarily use [Angular](https://github.com/angular/angular/blob/main/CONTRIBUTING.md#commit) style for commit messages. Roughly, they should follow the pattern:
+```
+<type>(<scope>): <short summary>
+```
+
+where scope (optional) describes the packages affected by the code changes and type (mandatory) is one of:
+
+- **build**: Changes that affect the build system or external dependencies (example scopes: pyproject.toml, setup.py)
+- **ci**: Changes to our CI configuration files and scripts (examples: .github/workflows/ci.yml)
+- **docs**: Documentation only changes
+- **feat**: A new feature
+- **fix**: A bug fix
+- **perf**: A code change that improves performance
+- **refactor**: A code change that neither fixes a bug nor adds a feature
+- **test**: Adding missing tests or correcting existing tests
+
+### Documentation
+
+To generate the rst files source files for documentation, run:
+
+```
+sphinx-apidoc -o docs/source/ src
+```
+
+Then to create the documentation html files, run:
+```
+sphinx-build -b html docs/source/ docs/build/html
+```
+
+More info on sphinx installation can be found here: https://www.sphinx-doc.org/en/master/usage/installation.html