groundworkers 0.1.0__tar.gz

groundworkers-0.1.0/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: groundworkers
+Version: 0.1.0
+Summary: Groundworkers MCP server — read-only agentive access to OMOP vocabularies, concept graphs, and embeddings.
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: mcp[cli]<2,>=1
+Requires-Dist: pydantic<3,>=2
+Requires-Dist: pyyaml<7,>=6
+Requires-Dist: SQLAlchemy<3,>=2
+Requires-Dist: psycopg[binary]<4,>=3.1
+Requires-Dist: omop-graph>=1.1.0
+Requires-Dist: omop-emb>=1.0.0
+Provides-Extra: embedding-pgvector
+Requires-Dist: omop-emb[pgvector]>=1.0.0; extra == "embedding-pgvector"
+Provides-Extra: embedding-faiss
+Requires-Dist: omop-emb[faiss-cpu]>=1.0.0; extra == "embedding-faiss"
+Provides-Extra: dev
+Requires-Dist: pytest<9,>=8; extra == "dev"
+
+# groundworkers
+
+**groundworkers** is an atomic, read-only MCP (Model Context Protocol) tool library for
+navigating the OMOP vocabularies.  It exposes OMOP vocabulary lookups, embedding similarity search, 
+cohort concept references, and system status as typed MCP tools that any MCP client can call —
+including [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew),
+Claude Code, and autonomous agents.
+
+Read-only.  No patient-level data.  No write operations.
+
+## What it exposes
+
+| Group | Tools |
+|---|---|
+| **Concept** | `concept_get`, `concept_by_code`, `concept_ancestors`, `concept_descendants`, `concept_relationships`, `concept_equivalency_path`, `concept_path`, `concept_map_to_standard`, `concept_neighbors` |
+| **Resolver** | `concept_ground` (with `parent_ids`, scoring fields, and `grounding_explanation`) |
+| **Search** | `concept_search_exact`, `concept_search_fulltext`, `concept_navigate_to_standard` |
+| **Embedding** | `embedding_index_status`, `embedding_neighbours`, `embedding_search`, `embedding_encode` |
+| **Cohort** | `cohort_find_concept_references` |
+| **System** | `system_status`, `system_vocabulary_catalogue` |
+
+Tools are registered conditionally — if an adapter is not configured, its tools are
+simply not registered.  `system_status` and `system_vocabulary_catalogue` are always
+registered so clients can always query adapter availability.
+
+## Quick start
+
+```bash
+uv venv
+uv sync --extra dev --extra embedding-tools
+uv run groundworkers --config config/groundworkers.example.yaml --describe
+```
+
+Start the server:
+
+```bash
+uv run groundworkers --config config/groundworkers.example.yaml
+```
+
+## Example config
+
+```yaml
+omop_graph:
+  db_url: "postgresql+psycopg://user:pass@localhost:5432/omop"
+  vocab_schema: omop_vocab
+
+omop_emb:
+  enabled: true
+  backend_type: pgvector
+  db_url: "postgresql+psycopg://user:pass@localhost:5432/omop"
+  default_model_name: qwen3-embedding:0.6b
+  api_base: "http://localhost:11434/v1"
+  api_key: "ollama"
+```
+
+## Install matrix
+
+| Use case | Extras |
+|---|---|
+| Core server only | none |
+| Concept tools | `concept-tools` |
+| Cohort tools | `cohort-tools` |
+| Embedding tools (sqlite-vec) | `embedding-tools` |
+| Embedding tools (pgvector) | `embedding-pgvector` |
+| Embedding tools (FAISS sidecar) | `embedding-faiss` |
+| All tool families | `all-tools` |
+| All + pgvector embeddings | `all-tools-pgvector` |
+| All + FAISS embeddings | `all-tools-faiss` |
+| Development | `dev` |
+| Development + all tools | `dev-all` |
+
+## Layout
+
+```
+src/groundworkers/
+  adapters/          — omop_graph, omop_emb, oa_cohorts adapter classes
+  base/              — GroundcrewServer, errors, results, SQL helpers
+  tools/             — MCP tool registrations by domain
+  config.py          — Pydantic config models (AppConfig, OmopGraphConfig, etc.)
+  server.py          — Server factory and CLI entry point
+config/              — Example YAML configs
+_design/             — Architecture notes and spec documents
+tests/               — Unit and integration tests
+```
+
+## Adapter backends
+
+- **omop-graph** — concept lookup, hierarchy traversal, full-text search
+- **omop-emb** — embedding index (sqlite-vec, pgvector, or FAISS sidecar)
+- **OpenAnalytics cohorts** — cohort concept reference queries (Phase N)
+
+## Companion repos
+
+- [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew) — ACP orchestration layer that drives this tool substrate
+- [omop-graph](https://australiancancerdatanetwork.github.io/omop-graph/) — OMOP virtual knowledge graph library
+- [omop-emb](https://australiancancerdatanetwork.github.io/omop-emb/) — OMOP embedding index library

groundworkers-0.1.0/README.md

@@ -0,0 +1,96 @@
+# groundworkers
+
+**groundworkers** is an atomic, read-only MCP (Model Context Protocol) tool library for
+navigating the OMOP vocabularies.  It exposes OMOP vocabulary lookups, embedding similarity search, 
+cohort concept references, and system status as typed MCP tools that any MCP client can call —
+including [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew),
+Claude Code, and autonomous agents.
+
+Read-only.  No patient-level data.  No write operations.
+
+## What it exposes
+
+| Group | Tools |
+|---|---|
+| **Concept** | `concept_get`, `concept_by_code`, `concept_ancestors`, `concept_descendants`, `concept_relationships`, `concept_equivalency_path`, `concept_path`, `concept_map_to_standard`, `concept_neighbors` |
+| **Resolver** | `concept_ground` (with `parent_ids`, scoring fields, and `grounding_explanation`) |
+| **Search** | `concept_search_exact`, `concept_search_fulltext`, `concept_navigate_to_standard` |
+| **Embedding** | `embedding_index_status`, `embedding_neighbours`, `embedding_search`, `embedding_encode` |
+| **Cohort** | `cohort_find_concept_references` |
+| **System** | `system_status`, `system_vocabulary_catalogue` |
+
+Tools are registered conditionally — if an adapter is not configured, its tools are
+simply not registered.  `system_status` and `system_vocabulary_catalogue` are always
+registered so clients can always query adapter availability.
+
+## Quick start
+
+```bash
+uv venv
+uv sync --extra dev --extra embedding-tools
+uv run groundworkers --config config/groundworkers.example.yaml --describe
+```
+
+Start the server:
+
+```bash
+uv run groundworkers --config config/groundworkers.example.yaml
+```
+
+## Example config
+
+```yaml
+omop_graph:
+  db_url: "postgresql+psycopg://user:pass@localhost:5432/omop"
+  vocab_schema: omop_vocab
+
+omop_emb:
+  enabled: true
+  backend_type: pgvector
+  db_url: "postgresql+psycopg://user:pass@localhost:5432/omop"
+  default_model_name: qwen3-embedding:0.6b
+  api_base: "http://localhost:11434/v1"
+  api_key: "ollama"
+```
+
+## Install matrix
+
+| Use case | Extras |
+|---|---|
+| Core server only | none |
+| Concept tools | `concept-tools` |
+| Cohort tools | `cohort-tools` |
+| Embedding tools (sqlite-vec) | `embedding-tools` |
+| Embedding tools (pgvector) | `embedding-pgvector` |
+| Embedding tools (FAISS sidecar) | `embedding-faiss` |
+| All tool families | `all-tools` |
+| All + pgvector embeddings | `all-tools-pgvector` |
+| All + FAISS embeddings | `all-tools-faiss` |
+| Development | `dev` |
+| Development + all tools | `dev-all` |
+
+## Layout
+
+```
+src/groundworkers/
+  adapters/          — omop_graph, omop_emb, oa_cohorts adapter classes
+  base/              — GroundcrewServer, errors, results, SQL helpers
+  tools/             — MCP tool registrations by domain
+  config.py          — Pydantic config models (AppConfig, OmopGraphConfig, etc.)
+  server.py          — Server factory and CLI entry point
+config/              — Example YAML configs
+_design/             — Architecture notes and spec documents
+tests/               — Unit and integration tests
+```
+
+## Adapter backends
+
+- **omop-graph** — concept lookup, hierarchy traversal, full-text search
+- **omop-emb** — embedding index (sqlite-vec, pgvector, or FAISS sidecar)
+- **OpenAnalytics cohorts** — cohort concept reference queries (Phase N)
+
+## Companion repos
+
+- [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew) — ACP orchestration layer that drives this tool substrate
+- [omop-graph](https://australiancancerdatanetwork.github.io/omop-graph/) — OMOP virtual knowledge graph library
+- [omop-emb](https://australiancancerdatanetwork.github.io/omop-emb/) — OMOP embedding index library

groundworkers-0.1.0/pyproject.toml

@@ -0,0 +1,46 @@
+[project]
+name = "groundworkers"
+version = "0.1.0"
+description = "Groundworkers MCP server — read-only agentive access to OMOP vocabularies, concept graphs, and embeddings."
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+  "mcp[cli]>=1,<2",
+  "pydantic>=2,<3",
+  "pyyaml>=6,<7",
+  "SQLAlchemy>=2,<3",
+  "psycopg[binary]>=3.1,<4",
+  "omop-graph>=1.1.0",
+  "omop-emb>=1.0.0",
+]
+
+[project.optional-dependencies]
+embedding-pgvector = [
+  "omop-emb[pgvector]>=1.0.0",
+]
+embedding-faiss = [
+  "omop-emb[faiss-cpu]>=1.0.0",
+]
+dev = [
+  "pytest>=8,<9",
+]
+
+[project.scripts]
+groundworkers = "groundworkers.server:main"
+
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = [
+  "integration: requires a live backend or database",
+  "requires_client: requires an embedding API client",
+]

groundworkers-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

groundworkers-0.1.0/src/groundworkers/__init__.py

@@ -0,0 +1,3 @@
+__all__ = ["__version__"]
+
+__version__ = "0.1.0"

groundworkers-0.1.0/src/groundworkers/adapters/__init__.py

@@ -0,0 +1 @@
+

groundworkers-0.1.0/src/groundworkers/adapters/omop_emb.py

@@ -0,0 +1,251 @@
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Any
+
+import numpy as np
+from omop_emb import EmbeddingConceptFilter, EmbeddingReaderInterface
+from omop_emb.config import MetricType
+from omop_emb.embeddings.embedding_client import EmbeddingRole
+from omop_emb.interface import list_registered_models
+
+from groundworkers.base.errors import GroundworkersError
+
+
+class OmopEmbAdapter:
+    def __init__(
+        self,
+        *,
+        backend_factory: Callable[[], object],
+        backend_type: str | None,
+        default_model_name: str | None = None,
+        client_factory: Callable[[str], object] | None = None,
+        cdm_engine: object | None = None,
+        faiss_cache_dir: str | None = None,
+    ) -> None:
+        self._backend_factory = backend_factory
+        self._backend_type = backend_type
+        self._default_model_name = default_model_name
+        self._client_factory = client_factory
+        self._cdm_engine = cdm_engine
+        self._faiss_cache_dir = faiss_cache_dir
+        self._backend: object | None = None
+        self._clients: dict[str, object] = {}
+
+    def is_available(self) -> bool:
+        return self.index_status()["available"]
+
+    def has_client(self) -> bool:
+        return self._client_factory is not None
+
+    def close(self) -> None:
+        backend = self._backend
+        if backend is not None and hasattr(backend, "close"):
+            backend.close()
+        self._backend = None
+        self._clients.clear()
+
+    def index_status(self) -> dict[str, Any]:
+        try:
+            backend = self._get_backend()
+            records = list_registered_models(backend=backend)
+            models: list[dict[str, Any]] = []
+            for record in records:
+                metric_type = record.metric_type or MetricType.COSINE
+                concept_count = backend.get_embedding_count(
+                    model_name=record.model_name,
+                    metric_type=metric_type,
+                )
+                models.append(
+                    {
+                        "model_name": record.model_name,
+                        "provider": self._enum_value(record.provider_type),
+                        "dimensions": int(record.dimensions),
+                        "index_type": self._enum_value(record.index_type),
+                        "concept_count": int(concept_count),
+                    }
+                )
+            return {
+                "available": bool(models),
+                "backend_type": self._backend_type or self._backend_type_from_backend(backend),
+                "models": models,
+            }
+        except Exception:
+            return {
+                "available": False,
+                "backend_type": self._backend_type,
+                "models": [],
+            }
+
+    def get_neighbours(
+        self,
+        concept_id: int,
+        limit: int,
+        model_name: str | None,
+    ) -> dict[str, Any]:
+        record = self._resolve_model_record(model_name)
+        reader = self._build_reader(record)
+        vectors = reader.get_embeddings_by_concept_ids((concept_id,))
+        if concept_id not in vectors:
+            raise GroundworkersError("NOT_FOUND", f"Concept {concept_id} is not present in the embedding index")
+
+        vector = np.asarray(vectors[concept_id], dtype=float).reshape(1, -1)
+        # Request limit+1 so that self-exclusion below still yields `limit` results.
+        concept_filter = self._build_concept_filter(limit=limit + 1)
+        raw = reader.get_nearest_concepts(
+            query_embedding=vector,
+            concept_filter=concept_filter,
+            k=limit + 1,
+        )
+        matches = raw[0] if raw else ()
+        results = [
+            self._serialise_nearest_match(match)
+            for match in matches
+            if getattr(match, "concept_id", None) != concept_id
+        ][:limit]
+        return {
+            "query_concept_id": concept_id,
+            "model_name": record.model_name,
+            "results": results,
+        }
+
+    def search(
+        self,
+        query: str,
+        limit: int,
+        domain: str | None,
+        vocabulary: str | None,
+        standard_only: bool,
+        active_only: bool,
+        model_name: str | None,
+    ) -> dict[str, Any]:
+        if not self.has_client():
+            raise GroundworkersError(
+                "BACKEND_UNAVAIL",
+                "on-the-fly embedding requires a configured model client",
+            )
+        record = self._resolve_model_record(model_name)
+        reader = self._build_reader(record)
+        client = self._get_client(record.model_name)
+        concept_filter = self._build_concept_filter(
+            limit=limit,
+            domain=domain,
+            vocabulary=vocabulary,
+            standard_only=standard_only,
+            active_only=active_only,
+        )
+        raw = reader.get_nearest_concepts_from_query_texts(
+            query_texts=(query,),
+            embedding_client=client,
+            concept_filter=concept_filter,
+            k=limit,
+        )
+        matches = raw[0] if raw else ()
+        return {
+            "query_text": query,
+            "model_name": record.model_name,
+            "results": [self._serialise_nearest_match(match) for match in matches],
+        }
+
+    def encode(self, text: str, model_name: str | None) -> dict[str, Any]:
+        if not self.has_client():
+            raise GroundworkersError("BACKEND_UNAVAIL", "embedding client is not configured")
+        record = self._resolve_model_record(model_name)
+        client = self._get_client(record.model_name)
+        vector = client.embeddings(text, embedding_role=EmbeddingRole.QUERY)
+        array = np.asarray(vector, dtype=float)
+        if array.ndim != 2 or array.shape[0] != 1:
+            raise GroundworkersError("QUERY_ERROR", f"Expected one embedding vector, got shape {array.shape}")
+        row = array[0]
+        return {
+            "text": text,
+            "model_name": record.model_name,
+            "dimensions": int(row.shape[0]),
+            "vector": row.tolist(),
+        }
+
+    def _get_backend(self) -> object:
+        if self._backend is None:
+            try:
+                self._backend = self._backend_factory()
+            except Exception as exc:
+                raise GroundworkersError("BACKEND_UNAVAIL", f"Embedding backend is unavailable: {exc}") from exc
+        return self._backend
+
+    def _resolve_model_record(self, model_name: str | None) -> object:
+        backend = self._get_backend()
+        requested_name = model_name or self._default_model_name
+        records = list_registered_models(backend=backend, model_name=requested_name)
+        if requested_name is not None:
+            if not records:
+                raise GroundworkersError("NOT_FOUND", f"Embedding model {requested_name!r} is not registered")
+            return records[0]
+        # No specific model requested — records contains all registered models.
+        if len(records) == 1:
+            return records[0]
+        if not records:
+            raise GroundworkersError("BACKEND_UNAVAIL", "No embedding models are registered in the backend")
+        raise GroundworkersError(
+            "BACKEND_UNAVAIL",
+            "No default embedding model is configured and multiple registered models are available",
+        )
+
+    def _build_reader(self, record: object) -> EmbeddingReaderInterface:
+        return EmbeddingReaderInterface(
+            model=record.model_name,
+            backend=self._get_backend(),
+            metric_type=record.metric_type or MetricType.COSINE,
+            omop_cdm_engine=self._cdm_engine,
+            provider_name_or_type=record.provider_type,
+            faiss_cache_dir=self._faiss_cache_dir,
+        )
+
+    def _get_client(self, model_name: str) -> object:
+        if self._client_factory is None:
+            raise GroundworkersError("BACKEND_UNAVAIL", "embedding client is not configured")
+        client = self._clients.get(model_name)
+        if client is None:
+            try:
+                client = self._client_factory(model_name)
+            except Exception as exc:
+                raise GroundworkersError("BACKEND_UNAVAIL", f"Embedding client is unavailable: {exc}") from exc
+            self._clients[model_name] = client
+        return client
+
+    def _build_concept_filter(
+        self,
+        *,
+        limit: int,
+        domain: str | None = None,
+        vocabulary: str | None = None,
+        standard_only: bool = False,
+        active_only: bool = False,
+    ) -> EmbeddingConceptFilter:
+        domains = (domain,) if domain else None
+        vocabularies = (vocabulary,) if vocabulary else None
+        return EmbeddingConceptFilter(
+            domains=domains,
+            vocabularies=vocabularies,
+            require_standard=standard_only,
+            require_active=active_only,
+            limit=limit,
+        )
+
+    def _backend_type_from_backend(self, backend: object) -> str | None:
+        backend_type = getattr(backend, "backend_type", None)
+        return self._enum_value(backend_type)
+
+    def _serialise_nearest_match(self, match: object) -> dict[str, Any]:
+        return {
+            "concept_id": int(getattr(match, "concept_id")),
+            "concept_name": getattr(match, "concept_name", None),
+            "similarity": round(float(getattr(match, "similarity")), 6),
+            "is_standard": getattr(match, "is_standard", None),
+            "is_active": getattr(match, "is_active", None),
+        }
+
+    @staticmethod
+    def _enum_value(value: object) -> str | None:
+        if value is None:
+            return None
+        return getattr(value, "value", str(value))