nmdc-runtime 1.7.0__tar.gz → 1.9.0__tar.gz

nmdc_runtime-1.9.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,45 @@
+---
+name: Bug report
+about: Create a report to help us debug
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+Example user story template:
+AS A {user|persona|system},
+[INSTEAD OF {current condition}]
+I EXPECTED {result}
+[SO THAT {value or justification}]
+[NO LATER THAN {best by date}]
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Acceptance Criteria**
+- Clear to everyone involved
+- Can be tested or verified 
+- Either passes or fails (cannot be 50% completed, for example)
+- Focus on the outcome, not how the outcome is achieved
+- As specific as possible (fast page load speed vs. 3-second page load speed)
+- if multiple criteria, present as a bulleted list of short scenarios (see template below)
+
+Example scenario-based template:
+Given (some given context or precondition), when (I take this action), then (this will be the specific outcome).
+
+**Additional context**
+Add any other context about the problem here.

nmdc_runtime-1.9.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,41 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+Example user story template:
+AS A {user|persona|system},
+[INSTEAD OF {current condition}]
+I WANT TO {action} [IN {mode} TIME | IN {differentiating performance units} TO {utility performance units}]
+[SO THAT {value or justification}]
+[NO LATER THAN {best by date}]
+
+**Acceptance Criteria**
+- Clear to everyone involved
+- Can be tested or verified 
+- Either passes or fails (cannot be 50% completed, for example)
+- Focus on the outcome, not how the outcome is achieved
+- As specific as possible (fast page load speed vs. 3-second page load speed)
+- if multiple criteria, present as a bulleted list of short scenarios (see template below)
+
+Example scenario-based template:
+Given (some given context or precondition), when (I take this action), then (this will be the specific outcome).
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

{nmdc_runtime-1.7.0 → nmdc_runtime-1.9.0}/.github/pull_request_template.md

@@ -24,7 +24,8 @@ Please describe the tests that you ran to verify your changes. Provide instructi
 
 **Configuration Details**: none
 
-# Checklist:
+# Definition of Done (DoD) Checklist:
+A simple checklist of necessary activities that add verifiable/demonstrable value to the product by asserting the quality of a feature, not the functionality of that feature; the latter should be stated as acceptance criteria in the issue this PR closes. Not all activities in the PR template will be applicable to each feature since the definition of done is intended to be a comprehensive checklist. Consciously decide the applicability of value-added activities on a feature-by-feature basis.
 
 - [ ] My code follows the style guidelines of this project (have you run `black nmdc_runtime/`?)
 - [ ] I have performed a self-review of my code

nmdc_runtime-1.9.0/.github/workflows/build-and-release-to-spin-berkeley.yml

@@ -0,0 +1,101 @@
+# Note: This GitHub Actions workflow was initialized by copy/pasting the contents of `build-and-release-to-spin.yml`.
+#       Changes made here since then include:
+#       - Changed the triggering branch to `berkeley` (was `main`)
+#       - Excluded Git tag creation from triggering criteria
+#       - Hard-coded the Spin namespace as `nmdc-berkeley` for deployment
+#       - Disabled pushing to Docker Hub (only push to GHCR)
+#       - Changed tagging rules to, effectively, "always tag as :berkeley"
+
+name: Build Docker images and release to Spin (nmdc-berkeley)
+
+on:
+  push:
+    branches:
+      - berkeley  # the `berkeley` branch, not the `main` branch
+    paths:
+      - '.github/workflows/build-and-release-to-spin-berkeley.yml'
+      - 'Makefile'
+      - '**.Dockerfile'
+      - '**.py'
+      - 'requirements/main.txt'
+
+env:
+  # We don't want to do certain steps if this is running in a fork
+  IS_ORIGINAL_REPO: ${{ github.repository == 'microbiomedata/nmdc-runtime' }}
+
+  # Used when sending redeploy action requests to Rancher
+  RANCHER_NAMESPACE: 'nmdc-berkeley'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        image: [ fastapi, dagster ]
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          # history for all branches and tags is needed for setuptools-scm (part of build and push step)
+          fetch-depth: 0
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: |
+            ghcr.io/microbiomedata/nmdc-runtime-${{ matrix.image }}
+          flavor: |
+            latest=false
+          tags: |
+            type=raw,value=berkeley
+
+      - name: Login to DockerHub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      # Reference: https://docs.docker.com/build/ci/github-actions/push-multi-registries/
+      # Reference: https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build and push
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          push: ${{ env.IS_ORIGINAL_REPO }}
+          file: nmdc_runtime/${{ matrix.image }}.Dockerfile
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+
+  release:
+    needs: build
+
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        deployment: [ runtime-api, dagster-dagit, dagster-daemon ]
+
+    steps:
+      - name: Redeploy ${{ env.RANCHER_NAMESPACE }}:${{ matrix.deployment }}
+        if: ${{ env.IS_ORIGINAL_REPO }}
+        uses: fjogeleit/http-request-action@v1
+        with:
+          url: ${{ secrets.RANCHER_URL }}/v3/project/${{ secrets.RANCHER_CONTEXT }}/workloads/deployment:${{ env.RANCHER_NAMESPACE }}:${{ matrix.deployment }}?action=redeploy
+          method: POST
+          bearerToken: ${{ secrets.RANCHER_TOKEN }}

{nmdc_runtime-1.7.0 → nmdc_runtime-1.9.0}/Makefile

@@ -69,7 +69,7 @@ down-dev:
 	docker compose down
 
 down-test:
-	docker compose --file docker-compose.test.yml down
+	docker compose --file docker-compose.test.yml down --volumes
 
 follow-fastapi:
 	docker compose logs fastapi -f
@@ -97,14 +97,14 @@ nersc-mongo-tunnels:
 		-o ServerAliveInterval=60 \
 		${NERSC_USERNAME}@dtn02.nersc.gov
 
-mongorestore-nmdc-dev:
-	wget https://portal.nersc.gov/cfs/m3408/meta/mongodumps/mdb-nmdc-dev.tar.gz
-	tar zxvf mdb-nmdc-dev.tar.gz
-	rm -rf nmdc/._* # Not sure why ._* files are extracted.
-	mongorestore -h localhost:27018 -u admin -p root --authenticationDatabase=admin \
-		--drop --gzip -d nmdc nmdc
-	rm -rf nmdc/
-	rm mdb-nmdc-dev.tar.gz
+mongorestore-nmdc-db:
+	mkdir -p /tmp/remote-mongodump/nmdc
+	# SSH into the remote server, stream the dump directory as a gzipped tar archive, and extract it locally.
+	ssh -i ~/.ssh/nersc ${NERSC_USERNAME}@dtn01.nersc.gov \
+		'tar -czf - -C /global/cfs/projectdirs/m3408/nmdc-mongodumps/dump_nmdc-prod_2024-07-29_20-12-07/nmdc .' \
+		| tar -xzv -C /tmp/remote-mongodump/nmdc
+	mongorestore -v -h localhost:27018 -u admin -p root --authenticationDatabase=admin \
+		--drop --nsInclude='nmdc.*' --dir /tmp/remote-mongodump
 
 quick-blade:
 	python -c "from nmdc_runtime.api.core.idgen import generate_id; print(f'nmdc:nt-11-{generate_id(length=8, split_every=0)}')"

{nmdc_runtime-1.7.0/nmdc_runtime.egg-info → nmdc_runtime-1.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: nmdc_runtime
-Version: 1.7.0
+Version: 1.9.0
 Summary: A runtime system for NMDC data management and orchestration
 Home-page: https://github.com/microbiomedata/nmdc-runtime
 Author: Donny Winston

{nmdc_runtime-1.7.0 → nmdc_runtime-1.9.0}/docker-compose.test.yml

@@ -9,7 +9,7 @@ services:
     image: postgres:11
     container_name: dagster-postgresql
     volumes:
-      - nmdc_runtime_postgres_data:/var/lib/postgresql/data
+      - nmdc_runtime_test_postgres_data:/var/lib/postgresql/data
     environment:
       POSTGRES_USER: "postgres_user"
       POSTGRES_PASSWORD: "postgres_password"
@@ -80,7 +80,7 @@ services:
     ports:
       - "27018:27017"
     volumes:
-      - nmdc_runtime_mongo_data:/data/db
+      - nmdc_runtime_test_mongo_data:/data/db
       - ~/nmdcdb-mongodump:/nmdc_dump:ro
       - ./tests/mongorestore-nmdc-testdb.sh:/mongorestore-nmdc-testdb.sh:ro
     restart: unless-stopped
@@ -106,7 +106,7 @@ services:
       - .:/code
 
 volumes:
-  nmdc_runtime_postgres_data:
+  nmdc_runtime_test_postgres_data:
     driver: local
-  nmdc_runtime_mongo_data:
+  nmdc_runtime_test_mongo_data:
     driver: local