PyPI - nmdc-runtime - Versions diffs - 2.2.0__tar.gz → 2.3.0__tar.gz - Mend

nmdc-runtime 2.2.0tar.gz → 2.3.0tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.

This version of nmdc-runtime might be problematic. Click here for more details.

Files changed (541) hide show

{nmdc_runtime-2.2.0 → nmdc_runtime-2.3.0}/.github/pull_request_template.md RENAMED Viewed

@@ -4,12 +4,12 @@
     │ Summarize the changes you made on this branch. This is typically a more │
     │ detailed restatement of the PR title.                                   │
     │                                                                         │
-    │ Example: "In this branch, I updated the `/studies/{study_id}` endpoint  │
+    │ Example: "On this branch, I updated the `/studies/{study_id}` endpoint  │
     │           so it returns an HTTP 404 response when the specified study   │
     │           does not exist."                                              │
     └─────────────────────────────────────────────────────────────────────────┘-->
-In this branch, I...
+On this branch, I...
 ### Details

nmdc_runtime-2.3.0/.github/workflows/deploy-redirects.yml ADDED Viewed

@@ -0,0 +1,93 @@
+# This GitHub Actions workflow builds a website file tree and deploys it to GitHub Pages.
+# Reference: https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions
+name: Deploy redirects to GitHub Pages
+on:
+  push: { branches: [ main ] }
+  workflow_dispatch: { }
+# Reference: https://docs.github.com/en/actions/using-jobs/using-concurrency
+concurrency:
+  group: github-pages
+  cancel-in-progress: true
+jobs:
+  build:
+    name: Build website
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - name: Check out commit
+        uses: actions/checkout@v4
+      - name: Create website file tree
+        run: |
+          # Define helper function.
+          make_redirect() {
+            # This function prints the HTML markup for a web page that redirects the client.
+            local url="https://docs.microbiomedata.org/runtime/${1}"
+            echo "<html><head><meta http-equiv=\"refresh\" content=\"0; url=${url}\" /></head><body>Redirecting to <a href=\"${url}\">${url}</a>...</body></html>"
+          }
+          # Create directories.
+          mkdir -p \
+            _build/html \
+            _build/html/explanation \
+            _build/html/howto-guides \
+            _build/html/howto-guides/jobs \
+            _build/html/tutorials \
+            _build/html/nb
+          # Create HTML files containing redirects.
+          #
+          # Note: These HTML files will be accessible at the same URLs at which _original_ Runtime documentation pages
+          #       had been accessible (on the https://microbiomedata.github.io/nmdc-runtime/ website) prior to the
+          #       launch of the new, NMDC-wide documentation website (i.e., https://docs.microbiomedata.org).
+          #
+          #       For documents added to the Runtime repo's `docs/` directory _after_ the launch of the latter website,
+          #       creating redirects for those documents' URLs is unnecessary, since there was nothing at those URLs
+          #       before (presumably, nobody has expects anything other than an "HTTP 404 Not Found" error at them).
+          #
+          cd _build/html
+          make_redirect index.html                                          > index.html
+          make_redirect admin.html                                          > admin.html
+          make_redirect draft.html                                          > draft.html
+          make_redirect contributing-docs.html                              > contributing-docs.html
+          make_redirect explanation/domain-vision-statement.html            > explanation/domain-vision-statement.html
+          make_redirect explanation/identifiers.html                        > explanation/identifiers.html
+          make_redirect explanation/journeys.html                           > explanation/journeys.html
+          make_redirect howto-guides/update-sensors-ops.html                > howto-guides/update-sensors-ops.html
+          make_redirect howto-guides/create-triggers.html                   > howto-guides/create-triggers.html
+          make_redirect howto-guides/improving-search-api.html              > howto-guides/improving-search-api.html
+          make_redirect howto-guides/release-process.html                   > howto-guides/release-process.html
+          make_redirect howto-guides/author-changesheets.html               > howto-guides/author-changesheets.html
+          make_redirect howto-guides/claim-and-run-jobs.html                > howto-guides/claim-and-run-jobs.html
+          make_redirect howto-guides/jobs/gold-translation-etl.html         > howto-guides/jobs/gold-translation-etl.html
+          make_redirect tutorials/json.html                                 > tutorials/json.html
+          make_redirect tutorials/exporters.html                            > tutorials/exporters.html
+          make_redirect tutorials/metadata-in.html                          > tutorials/metadata-in.html
+          make_redirect tutorials/auth.html                                 > tutorials/auth.html
+          make_redirect tutorials/translators.html                          > tutorials/translators.html
+          make_redirect nb/bulk_validation_referential_integrity_check.html > nb/bulk_validation_referential_integrity_check.html
+          make_redirect nb/get_data.html                                    > nb/get_data.html
+          make_redirect nb/queue_and_trigger_data_jobs.html                 > nb/queue_and_trigger_data_jobs.html
+          make_redirect nb/wf_automation.html                               > nb/wf_automation.html
+      - name: Save the result for publishing to GitHub Pages
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: _build/html
+  deploy:
+    name: Deploy website
+    needs:
+      - build
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

nmdc_runtime-2.3.0/.github/workflows/mkdocs.yml ADDED Viewed

@@ -0,0 +1,45 @@
+name: mkdocs
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'docs/**'
+      - 'mkdocs_overrides/**'
+      - 'mkdocs.yml'
+      - '.github/workflows/mkdocs.yml'
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    if: github.event.repository.fork == false
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.x
+      - run: pip install git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git
+      - run: pip install mkdocs-mermaid2-plugin
+      - run: pip install mkdocs-jupyter
+      # Build the MkDocs site (but don't deploy it).
+      #
+      # Note: Until January 17, 2025, we would build and deploy the MkDocs website to GitHub Pages.
+      #       As of January 17, 2025, we just _build_ it — we don't deploy it.
+      #
+      #       The reason is that the NMDC has a new, centralized documentation website: https://docs.microbiomedata.org.
+      #       That website includes a copy of the Runtime documentation (it's at the path, `/runtime`). Instead of
+      #       deploying this redundant copy (being built here) to `https://microbiomedata.github.io/nmdc-runtime`,
+      #       we will (via the new `deploy-redirects.yml` workflow) deploy a bunch of HTML files that redirect visitors
+      #       from pages on `https://microbiomedata.github.io/nmdc-runtime` to their counterparts on
+      #       `https://docs.microbiomedata.org/runtime`.
+      #
+      #       This redirection is not implemented in the MkDocs site, _itself_, because those MkDocs site source files
+      #       are used to build the website hosted at `https://docs.microbiomedata.org/runtime`; and, if a page were
+      #       to redirect to itself, the result would be a circular redirect.
+      #
+      #       The reason we still bother to build the MkDocs site here is so we get an error (a failed GHA workflow run)
+      #       when the site is no longer buildable (e.g. due to someone inadvertently introducing an invalid source
+      #       file, or due to one of the dependencies becoming unavailable), which can prompt remedial action.
+      #
+      - run: mkdocs build
+env:
+  GH_TOKEN: ${{ secrets.GH_TOKEN_MKDOCS_MATERIAL_INSIDERS }}

{nmdc_runtime-2.2.0 → nmdc_runtime-2.3.0}/.github/workflows/python-app.yml RENAMED Viewed

@@ -3,6 +3,15 @@
 name: Python application
+# Note: The following events will trigger this workflow:
+#       1. Someone pushes a commit to `main` that includes changes to any of the listed files.
+#       2. Someone opens a pull request that includes changes to any of the listed files.
+#       3. Someone clicks the "Run workflow" button on the "Actions" tab on GitHub.
+#
+#  References:
+#  - https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow#example-including-paths
+#  - https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
+#
 on:
   push:
     branches: [ main ]
@@ -11,7 +20,9 @@ on:
       - 'Makefile'
       - '**.Dockerfile'
       - '**.py'
-      - 'requirements/main.txt'
+      - 'requirements/main.txt'
+      # Every file in the `data` directory or in any of its subdirectories:
+      - 'metadata-translation/notebooks/data/**'
   pull_request:
     paths:
       - '.github/workflows/python-app.yml'
@@ -19,6 +30,7 @@ on:
       - '**.Dockerfile'
       - '**.py'
       - 'requirements/main.txt'
+      - 'metadata-translation/notebooks/data/**'
   # Allow developers to trigger this workflow manually via the "Actions" page on GitHub.
   # Reference: https://docs.github.com/en/actions/managing-workflow-runs-and-deployments/managing-workflow-runs/manually-running-a-workflow
   workflow_dispatch: { }

{nmdc_runtime-2.2.0 → nmdc_runtime-2.3.0}/Makefile RENAMED Viewed

@@ -5,10 +5,16 @@ init:
 # Updates Python dependencies based upon the contents of the `requirements/main.in` and `requirements/dev.in` files.
 #
-# To omit the `--upgrade` option (included by default) when running `pip-compile`:
-# ```
-# $ make update-deps UPDATE_DEPS_MAIN_UPGRADE_OPT='' UPDATE_DEPS_DEV_UPGRADE_OPT=''
-# ```
+# Note: To omit the `--upgrade` option (included by default) when running `pip-compile`:
+#       ```
+#       $ make update-deps UPDATE_DEPS_MAIN_UPGRADE_OPT='' UPDATE_DEPS_DEV_UPGRADE_OPT=''
+#       ```
+#
+# Note: You can run the following command on your host machine to have `$ make update-deps` run within a container:
+#       ```sh
+#       $ docker compose run --rm --no-deps fastapi sh -c 'make update-deps'
+#       ```
+#
 UPDATE_DEPS_MAIN_UPGRADE_OPT ?= --upgrade
 UPDATE_DEPS_DEV_UPGRADE_OPT ?= --upgrade
 update-deps:
@@ -43,6 +49,8 @@ test-dbinit:
 	docker compose --file docker-compose.test.yml \
 		exec mongo /bin/bash -c "/mongorestore-nmdc-testdb.sh"
+# Tip: If you append a file path to this "recipe", pytest will run only the tests defined in that file.
+#      For example, append `tests/test_api/test_endpoints.py` to have pytest only run the endpoint tests.
 test-run:
 	docker compose --file docker-compose.test.yml run test
@@ -101,7 +109,7 @@ mongorestore-nmdc-db:
 	mkdir -p /tmp/remote-mongodump/nmdc
 	# Optionally, manually update MONGO_REMOTE_DUMP_DIR env var:
 	# ```bash
-	# export MONGO_REMOTE_DUMP_DIR=$(ssh -i ~/.ssh/nersc -q ${NERSC_USERNAME}@dtn01.nersc.gov 'bash -s ' < get_latest_nmdc_prod_dump_dir.sh 2>/dev/null)
+	# export MONGO_REMOTE_DUMP_DIR=$(ssh -i ~/.ssh/nersc -q ${NERSC_USERNAME}@dtn01.nersc.gov 'bash -s ' < util/get_latest_nmdc_prod_dump_dir.sh 2>/dev/null)
 	# ```
 	# Rsync the remote dump directory items of interest:
 	rsync -av --exclude='_*' --exclude='fs\.*' \

{nmdc_runtime-2.2.0/nmdc_runtime.egg-info → nmdc_runtime-2.3.0}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: nmdc_runtime
-Version: 2.2.0
+Version: 2.3.0
 Summary: A runtime system for NMDC data management and orchestration
 Home-page: https://github.com/microbiomedata/nmdc-runtime
 Author: Donny Winston
@@ -11,6 +11,14 @@ Classifier: License :: OSI Approved :: Apache Software License
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: requires-python
+Dynamic: summary
 A runtime system for NMDC data management and orchestration.

{nmdc_runtime-2.2.0 → nmdc_runtime-2.3.0}/docs/howto-guides/claim-and-run-jobs.md RENAMED Viewed

@@ -3,7 +3,7 @@
 The Runtime advertises jobs to be done, where a [Job](https://api.microbiomedata.org/docs#/jobs)
 is a [Workflow](https://api.microbiomedata.org/docs#/workflows) paired with a chosen input
 [Object](https://api.microbiomedata.org/docs#/objects). See [Guide - Create Workflow Triggers To
-Spawn Jobs](guide-create-triggers.md) to learn how to arrange for jobs of interest to be
+Spawn Jobs](create-triggers.md) to learn how to arrange for jobs of interest to be
 automatically available when relevant new workflow inputs are available.
 You can list open jobs via [`GET /jobs`](https://api.microbiomedata.org/docs#/jobs/list_jobs_jobs_get). To

{nmdc_runtime-2.2.0 → nmdc_runtime-2.3.0}/docs/howto-guides/jobs/gold-translation-etl.md RENAMED Viewed

@@ -1,6 +1,5 @@
 # GOLD Translation ETL
 This job takes a `nmdc_merged_data.tsv.zip` file as a data object input.
 First, start your local runtime stack:
@@ -13,7 +12,6 @@ Load <http://localhost:3000/workspace> and ensure the `translation` repository i
 ![dagster-workspace-repo-loaded](../../img/dagster-workspace-repo-loaded.png)
 Next, navigate to the Jobs page for the `translation` repository:
 ![dagster-translation-repo-jobs](../../img/dagster-translation-repo-jobs.png)
@@ -53,12 +51,13 @@ mongoexport --host="localhost:27018" \
     -d nmdc_etl_staging -c gold.biosample_set \
     --jsonArray -o gold.biosample_set.json
 ```
 ```
 jq '{biosample_set:.}' gold.biosample_set.json \
     > gold.biosample_set.as_nmdcdb.2021-11-18.json
 ```
-Now, you may [submit this metadata as JSON through the API](../tutorials/tutorial-metadata-in.md).
+Now, you may [submit this metadata as JSON through the API](../../tutorials/metadata-in.md)
 In the tutorial linked to above, GitHub's Gist service is used to host and obtain a URL for the
 JSON. Here's let's walk through using the NMDC's `www` directory on NERSC to host the file and
@@ -70,6 +69,7 @@ a DRS object in the API:
 scp  gold.biosample_set.as_nmdcdb.2021-11-18.json \
     dtn01.nersc.gov:/global/cfs/cdirs/m3408/www/meta/
 ```
 ```
 ssh dtn01.nersc.gov

nmdc-runtime 2.2.0__tar.gz → 2.3.0__tar.gz

Potentially problematic release.

nmdc-runtime 2.2.0tar.gz → 2.3.0tar.gz