aind-data-transfer-lite 0.0.1__tar.gz

aind_data_transfer_lite-0.0.1/.flake8

@@ -0,0 +1,8 @@
+[flake8]
+exclude =
+    .git,
+    __pycache__,
+    build,
+    .venv,
+    venv
+max-complexity = 10

aind_data_transfer_lite-0.0.1/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

aind_data_transfer_lite-0.0.1/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

aind_data_transfer_lite-0.0.1/.github/ISSUE_TEMPLATE/user-story.md

@@ -0,0 +1,27 @@
+---
+name: User story
+about: This template provides a basic structure for user story issues.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+# User story
+As a ..., I want to ..., so I can ...
+
+*Ideally, this is in the issue title, but if not, you can put it here. If so, delete this section.*
+
+# Acceptance criteria
+- [ ] This is something that can be verified to show that this user story is satisfied.
+
+# Sprint Ready Checklist 
+- [ ] 1. Acceptance criteria defined 
+- [ ] 2. Team understands acceptance criteria 
+- [ ] 3. Team has defined solution / steps to satisfy acceptance criteria 
+- [ ] 4. Acceptance criteria is verifiable / testable 
+- [ ] 5. External / 3rd Party dependencies identified
+- [ ] 6. Ticket is prioritized and sized
+
+# Notes
+*Add any helpful notes here.*

aind_data_transfer_lite-0.0.1/.github/workflows/add_issue_to_project_board.yml

@@ -0,0 +1,15 @@
+name: Auto add issues to project board
+on:
+  issues:
+    types:
+      - opened
+
+jobs:
+  add-to-project:
+    name: Add issue to project
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/add-to-project@v1.0.2
+        with:
+          project-url: https://github.com/orgs/AllenNeuralDynamics/projects/9
+          github-token: ${{ secrets.SERVICE_TOKEN }}

aind_data_transfer_lite-0.0.1/.github/workflows/tag_and_publish.yml

@@ -0,0 +1,85 @@
+name: Tag and publish
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  update_badges:
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        ref: ${{ env.DEFAULT_BRANCH }}
+        fetch-depth: 0
+        token: ${{ secrets.SERVICE_TOKEN }}
+    - name: Set up Python 3.10
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.10'
+    - name: Install dependencies
+      run: |
+        python -m pip install -e .[dev] --no-cache-dir
+    - name: Get Python version and Update README.md
+      run: |
+        python_version=$(grep "requires-python" pyproject.toml | grep -o ">=[^\"]*")
+        python_badge=$(grep -o 'python-[^)]*' README.md)
+        new_python_badge="python-$python_version-blue?logo=python"
+        sed -i "s/$python_badge/$new_python_badge/g" README.md
+    - name: Get interrogate values and Update README.md
+      run: |
+        interrogate_val=$(interrogate . | grep -o 'actual: [0-9]*\.[0-9]*' | awk '{print $2}')
+        interrogate_badge=$(grep -o 'interrogate-[^)]*' README.md)
+        if (( $(echo "$interrogate_val >= 90.00" | bc -l) )); then
+            new_interrogate_badge="interrogate-$interrogate_val%25-brightgreen"
+        elif (( $(echo "$interrogate_val < 80.00" | bc -l) )); then
+            new_interrogate_badge="interrogate-$interrogate_val%25-red"
+        else
+            new_interrogate_badge="interrogate-$interrogate_val%25-yellow"
+        fi
+        sed -i "s/$interrogate_badge/$new_interrogate_badge/g" README.md
+    - name: Get Coverage values and Update README.md
+      run: |
+        coverage run -m unittest discover
+        coverage_val=$(coverage report | grep "^TOTAL" | grep -o '[0-9]\+%' | grep -o '[0-9]\+')
+        coverage_badge=$(grep -o "coverage-[^?]*" README.md)
+        if (( $(echo "$coverage_val >= 90.00" | bc -l) )); then
+            new_coverage_badge="coverage-$coverage_val%25-brightgreen"
+        elif (( $(echo "$coverage_val < 80.00" | bc -l) )); then
+            new_coverage_badge="coverage-$coverage_val%25-red"
+        else
+            new_coverage_badge="coverage-$coverage_val%25-yellow"
+        fi
+        sed -i "s/$coverage_badge/$new_coverage_badge/g" README.md
+    - name: Commit changes
+      uses: EndBug/add-and-commit@v9
+      with:
+        default_author: github_actions
+        message: "ci: update badges [skip actions]"
+        add: '["README.md"]'
+  tag:
+    needs: update_badges
+    uses: AllenNeuralDynamics/aind-github-actions/.github/workflows/tag.yml@main
+    secrets:
+      SERVICE_TOKEN: ${{ secrets.SERVICE_TOKEN }}
+  publish:
+    needs: tag
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Pull latest changes
+        run: git pull origin main
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+      - name: Install dependencies
+        run: |
+          pip install --upgrade setuptools wheel twine build
+          python -m build
+          twine check dist/*
+      - name: Publish on PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.AIND_PYPI_TOKEN }}

aind_data_transfer_lite-0.0.1/.github/workflows/test_and_lint.yml

@@ -0,0 +1,26 @@
+name: Lint and run tests
+
+on:
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  ci:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [ '3.10', '3.11', '3.12', '3.13' ]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: | 
+        python -m pip install -e .[dev]
+    - name: Run linter checks
+      run: flake8 . && interrogate --verbose .
+    - name: Run tests and coverage
+      run: coverage run -m unittest discover && coverage report

aind_data_transfer_lite-0.0.1/.gitignore

@@ -0,0 +1,139 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/
+
+# MacOs
+**/.DS_Store

aind_data_transfer_lite-0.0.1/CODE_OF_CONDUCT.md

@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official email address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+ScientificComputing@AllenInstitute.onmicrosoft.com.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

aind_data_transfer_lite-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Allen Institute for Neural Dynamics
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

aind_data_transfer_lite-0.0.1/PKG-INFO

@@ -0,0 +1,166 @@
+Metadata-Version: 2.4
+Name: aind-data-transfer-lite
+Version: 0.0.1
+Summary: Generated from aind-library-template
+Author: Allen Institute for Neural Dynamics
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic<2.12,>=2.0
+Requires-Dist: pydantic-settings<3.0,>=2.8.0
+Requires-Dist: aind-data-schema==2.0.5
+Requires-Dist: aind-data-schema-models<5.0
+Requires-Dist: boto3
+Provides-Extra: dev
+Requires-Dist: black; extra == "dev"
+Requires-Dist: coverage; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Requires-Dist: interrogate; extra == "dev"
+Requires-Dist: isort; extra == "dev"
+Requires-Dist: Sphinx; extra == "dev"
+Requires-Dist: furo; extra == "dev"
+Dynamic: license-file
+
+# aind-data-transfer-lite
+
+[![License](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
+![Code Style](https://img.shields.io/badge/code%20style-black-black)
+[![semantic-release: angular](https://img.shields.io/badge/semantic--release-angular-e10079?logo=semantic-release)](https://github.com/semantic-release/semantic-release)
+![Interrogate](https://img.shields.io/badge/interrogate-100.0%25-brightgreen)
+![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen?logo=codecov)
+![Python](https://img.shields.io/badge/python->=3.10-blue?logo=python)
+
+## Getting Started
+
+### Who is this for?
+
+You want to upload data to AIND's Cloud Storage platform on AWS.
+
+### Prerequisites
+
+- Install [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
+- Authentication for write permissions to aind-open-data bucket. Please reach to AIND Scientific Computing for access.
+
+## Installation
+Install directly from PyPI. We recommend installing into a virtual environment or conda environment.
+```bash
+pip install aind-data-transfer-lite
+```
+
+## Usage
+
+### Example Python Script
+
+```python
+from pathlib import Path
+import os
+from aind_data_transfer_lite.models import JobSettings
+from aind_data_transfer_lite.upload_data import UploadDataJob
+
+# Assuming running from same directory as this README file
+cwd = os.getcwd()
+behavior_path = Path(cwd) / "tests" / "resources" / "behavior_data"
+ecephys_path = Path(cwd) / "tests" / "resources" / "ecephys_data"
+metadata_path = Path(cwd) / "tests" / "resources" / "metadata_dir"
+
+modality_directories = {
+  "behavior": behavior_path,
+  "ecephys": ecephys_path
+}
+
+metadata_directory = metadata_path
+
+job_settings = JobSettings(
+  dry_run=True,
+  modality_directories=modality_directories,
+  metadata_directory=metadata_directory,
+  s3_bucket="aind-open-data-dev"
+)
+
+job = UploadDataJob(job_settings=job_settings)
+job.run_job()
+```
+
+### Example Command Line (Linux and MacOs)
+```bash
+python -m aind_data_transfer_lite.upload_data \
+--metadata_directory "./tests/resources/metadata_dir" \
+--modality_directories '{"behavior": "./tests/resources/behavior_data", "ecephys": "./tests/resources/ecephys_data"}' \
+--dry_run "True"
+```
+
+### Example Command Line (PowerShell)
+```bash
+python -m aind_data_transfer_lite.upload_data `
+--metadata_directory "./tests/resources/metadata_dir" `
+--modality_directories '{\"behavior\": \"./tests/resources/behavior_data\", \"ecephys\": \"./tests/resources/ecephys_data\"}' `
+--dry_run "True"
+```
+
+## Contributing
+
+For code development, clone the repo and install as
+```bash
+pip install -e ".[dev]"
+```
+
+### Linters and testing
+
+There are several libraries used to run linters, check documentation, and run tests.
+
+- Please test your changes using the **coverage** library, which will run the tests and log a coverage report:
+
+```bash
+coverage run -m unittest discover && coverage report
+```
+
+- Use **interrogate** to check that modules, methods, etc. have been documented thoroughly:
+
+```bash
+interrogate .
+```
+
+- Use **flake8** to check that code is up to standards (no unused imports, etc.):
+```bash
+flake8 .
+```
+
+- Use **black** to automatically format the code into PEP standards:
+```bash
+black .
+```
+
+- Use **isort** to automatically sort import statements:
+```bash
+isort .
+```
+
+### Pull requests
+
+For internal members, please create a branch. For external members, please fork the repository and open a pull request from the fork. We'll primarily use [Angular](https://github.com/angular/angular/blob/main/CONTRIBUTING.md#commit) style for commit messages. Roughly, they should follow the pattern:
+```text
+<type>(<scope>): <short summary>
+```
+
+where scope (optional) describes the packages affected by the code changes and type (mandatory) is one of:
+
+- **build**: Changes that affect build tools or external dependencies (example scopes: pyproject.toml, setup.py)
+- **ci**: Changes to our CI configuration files and scripts (examples: .github/workflows/ci.yml)
+- **docs**: Documentation only changes
+- **feat**: A new feature
+- **fix**: A bugfix
+- **perf**: A code change that improves performance
+- **refactor**: A code change that neither fixes a bug nor adds a feature
+- **test**: Adding missing tests or correcting existing tests
+
+### Semantic Release
+
+The table below, from [semantic release](https://github.com/semantic-release/semantic-release), shows which commit message gets you which release type when `semantic-release` runs (using the default configuration):
+
+| Commit message                                                                                                                                                                                   | Release type                                                                                                    |
+| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
+| `fix(pencil): stop graphite breaking when too much pressure applied`                                                                                                                             | ~~Patch~~ Fix Release, Default release                                                                          |
+| `feat(pencil): add 'graphiteWidth' option`                                                                                                                                                       | ~~Minor~~ Feature Release                                                                                       |
+| `perf(pencil): remove graphiteWidth option`<br><br>`BREAKING CHANGE: The graphiteWidth option has been removed.`<br>`The default graphite width of 10mm is always used for performance reasons.` | ~~Major~~ Breaking Release <br /> (Note that the `BREAKING CHANGE: ` token must be in the footer of the commit) |