groundworkers 0.3.3__tar.gz → 0.5.0__tar.gz

groundworkers-0.5.0/.githooks/pre-commit

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+# Canonical CAVA pre-commit hook: keeps uv.lock in sync with pyproject.toml.
+#
+# If pyproject.toml is staged, finds the specific dependency line(s) that
+# changed (added, removed, or had their version spec edited) and refreshes
+# only those packages in uv.lock via `uv lock --upgrade-package`. Does not
+# touch any other package -- this is a targeted sync, not a blanket
+# `uv lock`, so it never pulls in an unrelated upstream release.
+set -euo pipefail
+
+if ! git diff --cached --name-only | grep -q '^pyproject\.toml$'; then
+    exit 0
+fi
+
+if ! command -v uv >/dev/null 2>&1; then
+    echo "pre-commit: uv not found on PATH, skipping uv.lock sync" >&2
+    exit 0
+fi
+
+changed_names=$(git diff --cached -- pyproject.toml \
+    | sed -nE 's/^[+-][[:space:]]*"([A-Za-z0-9][A-Za-z0-9._-]*).*/\1/p' \
+    | sort -u)
+
+if [ -z "$changed_names" ]; then
+    exit 0
+fi
+
+upgrade_args=()
+while IFS= read -r name; do
+    upgrade_args+=(--upgrade-package "$name")
+done <<< "$changed_names"
+
+echo "pre-commit: syncing uv.lock for changed dependencies:"
+echo "$changed_names" | sed 's/^/  - /'
+uv lock "${upgrade_args[@]}"
+git add uv.lock

groundworkers-0.5.0/.github/CONTRIBUTING.md

@@ -0,0 +1,27 @@
+# Contributing
+
+## Development setup
+
+```bash
+uv sync --all-extras --dev
+uv run pytest -q
+uv run ruff check .
+```
+
+## Opening a pull request
+
+1. Apply **exactly one** label before merging:
+
+   | Label | When to use |
+   |---|---|
+   | `breaking` | Public API change, backward-incompatible |
+   | `feature` | New functionality, backward-compatible |
+   | `fix` | Bug fix |
+   | `dependencies` | Dependency version update |
+   | `chore` | CI changes, refactoring, test additions, docs — anything that does not affect the public-facing package. Bypasses the label gate; excluded from the changelog and does not bump the version. |
+
+2. When merging (squash), write a clear extended description in the merge dialog. That text — not the PR's opening description — becomes the changelog entry for this change. Leave it blank for `chore` PRs.
+
+## Versioning and releases
+
+Versions are derived from git tags; there is no version string in any source file. Releases are triggered by a maintainer publishing the standing draft release on the repository's Releases page. There is no automated commit-back to `main`.

groundworkers-0.5.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,66 @@
+name: "🐞 Bug Report"
+description: Report a reproducible bug.
+type: "Bug"
+body:
+  - type: checkboxes
+    attributes:
+      label: Pre-flight
+      options:
+        - label: I searched existing issues and this is not a duplicate.
+          required: true
+
+  - type: textarea
+    id: summary
+    attributes:
+      label: Summary
+      description: One or two sentences describing the bug.
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Reproduction
+      description: Steps and/or a minimal code example that reproduces the issue. Wrap code in triple backticks.
+      placeholder: |
+        1. Import X and call Y with Z
+        2. Observe error
+
+        ```python
+        # minimal example
+        ```
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual
+    attributes:
+      label: Actual behaviour
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behaviour
+    validations:
+      required: true
+
+  - type: textarea
+    id: logs
+    attributes:
+      label: Error output
+      description: Full traceback if applicable. Formatted automatically.
+      render: python-traceback
+
+  - type: textarea
+    id: system
+    attributes:
+      label: System info
+      description: |
+        Run this in your environment and paste the output:
+
+        ```shell
+        python <(curl -s https://raw.githubusercontent.com/AustralianCancerDataNetwork/cava-devops/main/scripts/cava_system_info.py)
+        ```
+      render: shell

groundworkers-0.5.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Discussions
+    url: https://github.com/orgs/AustralianCancerDataNetwork/discussions
+    about: Questions and general discussion about the CAVA stack.

groundworkers-0.5.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,32 @@
+name: "🚀 Feature Request"
+description: Propose a new feature or enhancement.
+type: "Feature"
+body:
+  - type: checkboxes
+    attributes:
+      label: Pre-flight
+      options:
+        - label: I searched existing issues and this has not been requested before.
+          required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem or motivation
+      description: What are you trying to do that you currently cannot?
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposed solution
+      description: How would you like this to work?
+    validations:
+      required: true
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives considered
+      description: Other approaches you considered and why you ruled them out.

groundworkers-0.5.0/.github/ISSUE_TEMPLATE/not_working.yml

@@ -0,0 +1,65 @@
+name: "❗ Something is not working"
+description: Something behaves unexpectedly but you are not sure if it is a bug.
+body:
+  - type: checkboxes
+    attributes:
+      label: Pre-flight
+      options:
+        - label: I searched existing issues for this problem.
+          required: true
+
+  - type: textarea
+    id: summary
+    attributes:
+      label: Problem summary
+      description: 1-2 sentences describing what is not working.
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Reproduction
+      description: Steps and/or a minimal code example. Wrap code in triple backticks.
+      placeholder: |
+        1. Import X and call Y with Z
+        2. Observe unexpected behaviour
+
+        ```python
+        # minimal example
+        ```
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual
+    attributes:
+      label: Actual outcome
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected outcome
+    validations:
+      required: true
+
+  - type: textarea
+    id: logs
+    attributes:
+      label: Error messages
+      description: Full traceback if applicable. Formatted automatically.
+      render: python-traceback
+
+  - type: textarea
+    id: system
+    attributes:
+      label: System info
+      description: |
+        Run this in your environment and paste the output:
+
+        ```shell
+        python <(curl -s https://raw.githubusercontent.com/AustralianCancerDataNetwork/cava-devops/main/scripts/cava_system_info.py)
+        ```
+      render: shell

groundworkers-0.5.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,9 @@
+## Summary
+
+<!-- What does this PR do? This text becomes the changelog entry — write it for a reader who will see it months from now, not for the reviewer reading the diff. Leave blank for chore PRs. -->
+
+## Checklist
+
+- [ ] Applied exactly one label (`breaking`, `feature`, `fix`, `dependencies`, or `chore`)
+- [ ] Tests pass locally (`uv run pytest -q`)
+- [ ] Lint passes (`uv run ruff check .`)

groundworkers-0.5.0/.github/release-drafter.yml

@@ -0,0 +1,31 @@
+name-template: 'v$RESOLVED_VERSION'
+tag-template: 'v$RESOLVED_VERSION'
+commitish: main
+
+categories:
+  - title: Breaking Changes
+    labels: ['breaking']
+  - title: Features
+    labels: ['feature']
+  - title: Fixes
+    labels: ['fix']
+  - title: Dependencies
+    labels: ['dependencies']
+
+template: |
+  $CHANGES
+
+change-template: '- **$TITLE** (#$NUMBER) @$AUTHOR'
+
+version-resolver:
+  major:
+    labels: ['breaking']
+  minor:
+    labels: ['feature']
+  patch:
+    labels: ['fix', 'dependencies']
+  default: patch
+
+exclude-labels: ['chore']
+
+autolabeler: []

groundworkers-0.5.0/.github/workflows/ci.yml

@@ -0,0 +1,52 @@
+name: CI
+on:
+  pull_request:
+    branches: [main]
+    types: [opened, synchronize, reopened, labeled, unlabeled]
+jobs:
+  label-gate:
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/label-gate.yml@main
+  build-test:
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/build-test.yml@main
+  build-test-postgres:
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/build-test-postgres.yml@main
+    with:
+      ty-src: src
+      postgres-image: pgvector/pgvector:pg18
+      postgres-db: test_db
+      postgres-user: postgres
+      postgres-password: postgres
+      setup-commands: |
+        mkdir -p "$RUNNER_TEMP/omop"
+        cp tests/support/postgres_ci_config.toml "$RUNNER_TEMP/omop/config.toml"
+        export OA_CONFIG_PATH="$RUNNER_TEMP/omop/config.toml"
+        export GROUNDWORKERS_CONFIG_PATH="$RUNNER_TEMP/omop/config.toml"
+        export GROUNDWORKERS_REQUIRE_INTEGRATION=1
+        export GROUNDWORKERS_TEST_DATABASE_URL="postgresql+psycopg://postgres:postgres@localhost:5432/test_db"
+        echo "OA_CONFIG_PATH=$RUNNER_TEMP/omop/config.toml" >> "$GITHUB_ENV"
+        echo "GROUNDWORKERS_CONFIG_PATH=$RUNNER_TEMP/omop/config.toml" >> "$GITHUB_ENV"
+        echo "GROUNDWORKERS_REQUIRE_INTEGRATION=1" >> "$GITHUB_ENV"
+        echo "GROUNDWORKERS_TEST_DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/test_db" >> "$GITHUB_ENV"
+        uv run omop-config connections add pg_test \
+          --dialect postgresql+psycopg \
+          --host localhost \
+          --port 5432 \
+          --user postgres \
+          --password postgres \
+          --database-name test_db \
+          --test-only false
+        uv run omop-config databases add test_cdm_db \
+          --kind cdm \
+          --connection pg_test \
+          --schema-name public
+        uv run omop-config databases add test_vector_db \
+          --kind generic \
+          --connection pg_test \
+          --schema-name public
+        uv run omop-config vector-stores add test_embeddings \
+          --backend-type pgvector \
+          --database test_vector_db
+        uv run omop-config configure groundworkers \
+          --cdm-db test_cdm_db \
+          --vector-store-name test_embeddings
+        uv run python -m tests.support.seed_postgres_fixture

groundworkers-0.5.0/.github/workflows/docs.yml

@@ -0,0 +1,10 @@
+name: Deploy Docs
+on:
+  push:
+    tags: ['v*']
+  workflow_dispatch:
+permissions:
+  contents: write
+jobs:
+  deploy:
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/deploy-docs.yml@main

groundworkers-0.5.0/.github/workflows/merge.yml

@@ -0,0 +1,13 @@
+name: Release Update
+on:
+  pull_request:
+    types: [closed]
+    branches: [main]
+permissions:
+  contents: write
+  pull-requests: read
+jobs:
+  draft:
+    if: github.event.pull_request.merged == true
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/release-drafter.yml@main
+    secrets: inherit

groundworkers-0.5.0/.github/workflows/publish.yml

@@ -0,0 +1,21 @@
+name: Publish
+on:
+  push:
+    tags: ['v*']
+jobs:
+  build:
+    uses: AustralianCancerDataNetwork/cava-devops/.github/workflows/publish.yml@main
+  publish:
+    # Inline: PyPI OIDC checks job_workflow_ref, which must point to this file.
+    # Moving pypa/gh-action-pypi-publish into cava-devops would break the trusted publisher.
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+    environment:
+      name: pypi
+      url: https://pypi.org/p/groundworkers
+    steps:
+      - uses: actions/download-artifact@v4
+        with: { name: dist, path: dist/ }
+      - uses: pypa/gh-action-pypi-publish@release/v1

groundworkers-0.5.0/.gitignore

@@ -0,0 +1,22 @@
+__pycache__/
+*.py[cod]
+.venv/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+dist/
+build/
+*.egg-info/
+.DS_Store
+_design/
+*.log
+*.bak
+*.tmp
+.vscode/
+settings.json
+*.local.yaml
+*.local.toml
+site/
+_knowledge/
+data/
+.env

groundworkers-0.5.0/.python-version

@@ -0,0 +1 @@
+3.12

groundworkers-0.5.0/CHANGELOG.md

@@ -0,0 +1,58 @@
+### CHANGELOG NO LONGER MAINTAINED
+now handled in ci/cd automation consistent the rest of the cava-devops stack
+
+## 0.3.4
+- started building out the configuration TUI 
+- lock upper boundary for oa_configurator compatibility
+
+## 0.3.3
+
+feature: changed the way that knowledge packs are registered, updated baseline knowledge packs (status is very draft for this part - ymmv)
+
+## 0.3.2
+
+fix: repaired integration tests
+
+## 0.3.1
+
+chore: version bump
+
+## 0.3.0
+
+feat: knowledge-base integration, layer cleanup, oa-configurator integration, graph and embedding services
+fix: candidate-bundle metadata, knowledge content-serving, embedding-backend honesty, source-profile signals
+
+- populate identity metadata (`concept_code`, `vocabulary_id`, `domain_id`, `concept_class_id`) on embedding-only candidates in `concept_candidate_bundle` via a single batch `concept_views` backfill (omop-emb never carries these); warn when the graph service is unavailable to enrich them
+- finish the knowledge catalogue: add `KnowledgeCatalogue.get_pack()` and a `knowledge_pack` MCP tool that serve pack `guidance`/`rules`/`examples` content — previously only manifest metadata and filenames were exposed; also collapse the dead tail branch in `PackApplicability.matches`
+- reject an unsupported embedding backend eagerly at build time with an actionable message, clarifying that FAISS is a query-time cache accelerator (`faiss_cache_dir` + the `embedding-faiss` extra), not a standalone backend value
+- record `inferred_vocab` on source-vocabulary columns so the router derives a table-level domain hint from them, and surface the matched source profile's `structural_skip_field_types` and `packed_value_column_hint` on `PreIngestBundle` / the assisted-plan REST response instead of discarding them
+- require `omop-emb>=1.1.1`
+- add knowledge-base catalogue integration and expand the knowledge-facing tool and service surface
+- introduce dedicated graph and grounding services so concept and resolver tools delegate through a cleaner service-layer split
+- integrate `oa-configurator` into application/bootstrap wiring and refresh the example configuration path
+- expand the `omop-graph` and `omop-emb` adapter layer, including parentless domain-constrained grounding support via `omop-graph>=1.3.0`
+- refresh architecture and service documentation, including new graph, grounding, resolver, and source-planning pages
+
+## 0.2.0
+
+feat: add direct Python service layer, mapping workflows, and docs refresh
+
+- add `build_application()` and a shared application container for direct Python consumers
+- add `MappingService` as a reusable service-layer API for mapping-oriented workflows
+- add mapping tools for normalized search, candidate bundles, parent backoff, mapping context, `Maps to value`, mapping-expression resolution, and candidate evaluation
+- keep mapping MCP tools thin by delegating orchestration to the service layer
+- tighten `omop-emb` typing through the adapter and server composition path
+- extend docs to cover MCP and direct-Python integration, layer boundaries, and mapping workflows
+
+## 0.1.1
+
+feat: add chunk coherence pass and review-state transitions
+
+- add coherence reranking based on approved-set distribution
+- infer provisional sets for inferred chunks
+- transition processed chunks to REVIEW and skip them on resume
+- add unit coverage for coherence behavior
+
+## 0.1.0
+
+- alpha release for review

groundworkers-0.5.0/Makefile

@@ -0,0 +1,23 @@
+.PHONY: setup test lint typecheck check describe
+
+PYTHON := .venv/bin/python3
+CONFIG ?= config/groundworkers.example.toml
+
+# Mirrors the install CI performs, so a local run reproduces the pipeline exactly.
+setup:
+	uv sync --all-extras --dev
+
+describe:
+	$(PYTHON) -m groundworkers.server --config $(CONFIG) --describe
+
+lint:
+	uv run ruff check .
+
+typecheck:
+	uv run ty check src/
+
+test:
+	uv run pytest -q
+
+# The three gates build-test.yml enforces on every pull request.
+check: typecheck lint test

groundworkers-0.5.0/PKG-INFO

@@ -0,0 +1,133 @@
+Metadata-Version: 2.5
+Name: groundworkers
+Version: 0.5.0
+Summary: Groundworkers MCP server — read-only agentive access to OMOP vocabularies, concept graphs, and embeddings.
+Requires-Python: >=3.12
+Requires-Dist: fastapi<1.0,>=0.110
+Requires-Dist: mcp[cli]<2,>=1
+Requires-Dist: numpy<3,>=1.26
+Requires-Dist: oa-configurator<2,>=1.3.0
+Requires-Dist: omop-alchemy<2,>=1.1
+Requires-Dist: omop-emb<3,>=2.1
+Requires-Dist: omop-graph<3,>=2.1
+Requires-Dist: omop-llm<2,>=1
+Requires-Dist: omop-semantics<1,>=0.6
+Requires-Dist: psutil<8,>=5.9
+Requires-Dist: psycopg[binary]<4,>=3.1
+Requires-Dist: pydantic<3,>=2
+Requires-Dist: python-dotenv<2,>=1
+Requires-Dist: pyyaml<7,>=6
+Requires-Dist: sqlalchemy<3,>=2
+Requires-Dist: uvicorn[standard]<1.0,>=0.29
+Provides-Extra: all-source
+Requires-Dist: openpyxl<4,>=3.1; extra == 'all-source'
+Requires-Dist: pdfplumber>=0.10; extra == 'all-source'
+Requires-Dist: python-docx>=1.0; extra == 'all-source'
+Provides-Extra: dev
+Requires-Dist: httpx<1,>=0.27; extra == 'dev'
+Requires-Dist: ipython>=8.0; extra == 'dev'
+Requires-Dist: mkdocs-material>=9.7.1; extra == 'dev'
+Requires-Dist: mkdocs-mermaid2-plugin; extra == 'dev'
+Requires-Dist: mkdocs>=1.6.1; extra == 'dev'
+Requires-Dist: mkdocstrings-python>=2.0.1; extra == 'dev'
+Requires-Dist: mypy>=1.8; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=9.0.3; extra == 'dev'
+Requires-Dist: requests>=2.33.0; extra == 'dev'
+Requires-Dist: ruff>=0.4; extra == 'dev'
+Requires-Dist: textual>=0.60; extra == 'dev'
+Requires-Dist: tornado>=6.5.5; extra == 'dev'
+Requires-Dist: ty>=0.0.59; extra == 'dev'
+Provides-Extra: docx
+Requires-Dist: python-docx>=1.0; extra == 'docx'
+Provides-Extra: embedding-artifacts
+Requires-Dist: h5py<4,>=3.10; extra == 'embedding-artifacts'
+Provides-Extra: embedding-faiss
+Requires-Dist: omop-emb[faiss-cpu]<3,>=2.1; extra == 'embedding-faiss'
+Provides-Extra: embedding-pgvector
+Requires-Dist: omop-emb[pgvector]<3,>=2.1; extra == 'embedding-pgvector'
+Provides-Extra: pdf
+Requires-Dist: pdfplumber>=0.10; extra == 'pdf'
+Provides-Extra: tui
+Requires-Dist: groundskeeping<1,>=0.8.1; extra == 'tui'
+Requires-Dist: rich<15,>=13; extra == 'tui'
+Requires-Dist: textual>=0.60; extra == 'tui'
+Provides-Extra: xlsx
+Requires-Dist: openpyxl<4,>=3.1; extra == 'xlsx'
+Description-Content-Type: text/markdown
+
+# groundworkers
+
+Groundworkers is the reusable, read-only capability layer for OMOP vocabulary lookup, concept mapping, source planning, and related analysis. It combines deterministic services with optional embeddings and model-assisted operations.
+
+It does not own patient-level writes, session state, or job lifecycle. Those concerns belong to the calling application; `groundcrew` is the usual stateful orchestrator.
+
+## Choose an interface
+
+| Interface | Use it when |
+| --- | --- |
+| MCP | An agent needs discoverable tools, prompts, resources, and a shared remote runtime. |
+| REST | An application needs the curated HTTP workflows for candidate bundles or assisted source planning. |
+| Python | A process owns batching, evaluation, or orchestration and can run in-process. |
+
+Start with the [documentation home](docs/index.md), or choose a path directly:
+
+- [Concepts and capability choices](docs/concepts.md) explains retrieval, grounding, mapping, source planning, and projection.
+- [Installation](docs/usage/installation.md) covers package extras and prerequisites.
+- [Initial local setup](docs/from-scratch.md) walks through a fresh CDM-backed deployment.
+- [Integrations](docs/usage/integrations.md) shows MCP, REST, and Python calls.
+- [Configuration](docs/usage/configuration.md) explains shared-stack references and optional capabilities.
+- [Tools overview](docs/tools/overview.md) maps jobs to the discoverable MCP surface.
+
+## Quick start
+
+Install the package and configure the shared OMOP stack:
+
+```bash
+pip install groundworkers
+omop-config configure groundworkers
+```
+
+Inspect the active configuration and MCP surface before starting a service:
+
+```bash
+groundworkers --describe
+```
+
+Run MCP over stdio for a client that starts the process, or over Streamable HTTP for a shared service:
+
+```bash
+groundworkers
+groundworkers --transport streamable-http --host 127.0.0.1 --port 8000
+```
+
+REST is a separate, curated interface:
+
+```bash
+groundworkers --transport rest --host 127.0.0.1 --port 8080
+```
+
+## Direct Python
+
+Build the application once and reuse it. Services are optional when their prerequisites are not configured:
+
+```python
+from groundworkers.app import build_application
+from groundworkers.bootstrap import build_app_config
+
+config = build_app_config()
+app = build_application(config)
+
+bundle = app.services.mapping.concept_candidate_bundle(
+    "type 2 diabetes",
+    domain="Condition",
+)
+```
+
+The `mapping` service in this example requires a configured CDM database. Embedding-backed channels require an embedding model and vector store; text, domain classification, and assisted source planning require a configured chat model. Use `groundworkers --describe` to see what the current deployment exposes.
+
+## Companion repositories
+
+- [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew) owns session state and orchestration.
+- [omop-graph](https://australiancancerdatanetwork.github.io/omop-graph/) provides graph-backed OMOP traversal.
+- [omop-emb](https://australiancancerdatanetwork.github.io/omop-emb/) provides embedding stores and indexes.