groundworkers 0.2.0__tar.gz → 0.3.3__tar.gz

groundworkers-0.3.3/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: groundworkers
+Version: 0.3.3
+Summary: Groundworkers MCP server — read-only agentive access to OMOP vocabularies, concept graphs, and embeddings.
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: fastapi<1.0,>=0.110
+Requires-Dist: mcp[cli]<2,>=1
+Requires-Dist: pydantic<3,>=2
+Requires-Dist: pyyaml<7,>=6
+Requires-Dist: SQLAlchemy<3,>=2
+Requires-Dist: psycopg[binary]<4,>=3.1
+Requires-Dist: oa-configurator>=0.1.2
+Requires-Dist: omop-graph>=1.3.0
+Requires-Dist: omop-emb>=1.1.1
+Requires-Dist: uvicorn[standard]<1.0,>=0.29
+Provides-Extra: llm
+Requires-Dist: openai>=1.0; extra == "llm"
+Provides-Extra: xlsx
+Requires-Dist: openpyxl<4,>=3.1; extra == "xlsx"
+Provides-Extra: pdf
+Requires-Dist: pdfplumber>=0.10; extra == "pdf"
+Provides-Extra: docx
+Requires-Dist: python-docx>=1.0; extra == "docx"
+Provides-Extra: all-source
+Requires-Dist: openpyxl<4,>=3.1; extra == "all-source"
+Requires-Dist: pdfplumber>=0.10; extra == "all-source"
+Requires-Dist: python-docx>=1.0; extra == "all-source"
+Provides-Extra: embedding-pgvector
+Requires-Dist: omop-emb[pgvector]>=1.1.1; extra == "embedding-pgvector"
+Provides-Extra: embedding-faiss
+Requires-Dist: omop-emb[faiss-cpu]>=1.1.1; extra == "embedding-faiss"
+Provides-Extra: dev
+Requires-Dist: httpx<1,>=0.27; extra == "dev"
+Requires-Dist: ipython>=8.0; extra == "dev"
+Requires-Dist: tornado>=6.5.5; extra == "dev"
+Requires-Dist: pytest>=9.0.3; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: mypy>=1.8; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: mkdocs-material>=9.7.1; extra == "dev"
+Requires-Dist: mkdocstrings-python>=2.0.1; extra == "dev"
+Requires-Dist: mkdocs>=1.6.1; extra == "dev"
+Requires-Dist: requests>=2.33.0; extra == "dev"
+Requires-Dist: mkdocs-mermaid2-plugin; extra == "dev"
+
+# groundworkers
+
+`groundworkers` is the reusable capability layer for OMOP-grounded lookup,
+mapping, source planning, and knowledge-pack discovery.
+
+You can use it in three ways:
+
+- as an **MCP service** for agentic clients and tool discovery
+- as a **REST service** for fixed workflow applications
+- as a **direct Python library** for in-process orchestration
+
+No patient-level writes. No session state. No transport-specific business logic.
+
+## What it provides
+
+- OMOP concept lookup and hierarchy navigation
+- exact, normalized, full-text, and embedding-backed retrieval
+- mapping-oriented candidate bundles and context assembly
+- stateless source-planning workflows
+- bundled baseline knowledge packs for reusable mapping and planning context
+- LLM-backed text normalization and domain classification
+
+## Knowledge Packs
+
+`groundworkers` ships with bundled baseline knowledge packs inside the package.
+These packs are available by default and provide reusable guidance and rules
+that apply broadly across deployments.
+
+Site-specific or localisation packs are added through the shared stack config
+via `tools.groundworkers.knowledge.packs_root`. When a configured pack has the
+same `layer` and `name` as a bundled baseline pack, the configured copy wins.
+
+## Runtime model
+
+```mermaid
+flowchart TD
+    STACK[shared stack config] --> BOOT[build_app_config]
+    BOOT --> CFG[AppConfig]
+    CFG --> APP[build_application]
+    APP --> GW[GroundworkersApp]
+    GW --> SVC[services]
+    GW --> ADP[adapters]
+    MCP[MCP client] --> TOOLS[MCP tools]
+    REST[REST client] --> API[REST transport]
+    PY[Python caller] --> SVC
+    TOOLS --> SVC
+    TOOLS -. adapter-backed primitives .-> ADP
+    API --> SVC
+```
+
+`build_application(...)` is the composition root. It builds one reusable
+runtime container with transport-agnostic services plus dependency-facing
+adapters. Most caller-facing workflows go through services; some MCP tools are
+intentionally adapter-backed when the capability is closer to a backend
+primitive than a domain service.
+
+## Quick start
+
+### Install
+
+```bash
+pip install groundworkers
+```
+
+Optional extras:
+
+```bash
+pip install "groundworkers[llm,embedding-pgvector]"
+```
+
+### Configure the shared stack
+
+```bash
+omop-config configure omop_alchemy
+omop-config configure omop_graph
+omop-config configure groundworkers
+# optional if you want embedding-backed capabilities
+omop-config configure omop_emb
+```
+
+### Start MCP
+
+```bash
+groundworkers --describe
+groundworkers --transport streamable-http --host 0.0.0.0 --port 8000
+```
+
+### Start REST
+
+```bash
+groundworkers --transport rest --host 0.0.0.0 --port 8080
+```
+
+### Use from Python
+
+```python
+from groundworkers.app import build_application
+from groundworkers.bootstrap import build_app_config
+
+config = build_app_config()
+app = build_application(config)
+
+mapping = app.services.mapping
+bundle = mapping.concept_candidate_bundle(
+    "type 2 diabetes",
+    domain="Condition",
+    include_normalized=True,
+    include_fulltext=True,
+    include_embedding=True,
+)
+```
+
+## Main surfaces
+
+| Surface | Best for |
+|---|---|
+| MCP tools | Tool discovery, agent interoperability, shared capability services |
+| REST routes | Typed HTTP workflows such as candidate bundles and assisted source planning |
+| `app.services.*` | In-process Python applications and batch workflows |
+| `app.adapters.*` | Backend wrappers used when you intentionally need dependency-shaped primitives |
+
+## Learn more
+
+- Docs home: `docs/index.md`
+- Configuration: `docs/usage/configuration.md`
+- Integrations: `docs/usage/integrations.md`
+- Architecture: `docs/architecture.md`
+
+## Companion repos
+
+- [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew)
+- [omop-graph](https://australiancancerdatanetwork.github.io/omop-graph/)
+- [omop-emb](https://australiancancerdatanetwork.github.io/omop-emb/)

groundworkers-0.3.3/README.md

@@ -0,0 +1,133 @@
+# groundworkers
+
+`groundworkers` is the reusable capability layer for OMOP-grounded lookup,
+mapping, source planning, and knowledge-pack discovery.
+
+You can use it in three ways:
+
+- as an **MCP service** for agentic clients and tool discovery
+- as a **REST service** for fixed workflow applications
+- as a **direct Python library** for in-process orchestration
+
+No patient-level writes. No session state. No transport-specific business logic.
+
+## What it provides
+
+- OMOP concept lookup and hierarchy navigation
+- exact, normalized, full-text, and embedding-backed retrieval
+- mapping-oriented candidate bundles and context assembly
+- stateless source-planning workflows
+- bundled baseline knowledge packs for reusable mapping and planning context
+- LLM-backed text normalization and domain classification
+
+## Knowledge Packs
+
+`groundworkers` ships with bundled baseline knowledge packs inside the package.
+These packs are available by default and provide reusable guidance and rules
+that apply broadly across deployments.
+
+Site-specific or localisation packs are added through the shared stack config
+via `tools.groundworkers.knowledge.packs_root`. When a configured pack has the
+same `layer` and `name` as a bundled baseline pack, the configured copy wins.
+
+## Runtime model
+
+```mermaid
+flowchart TD
+    STACK[shared stack config] --> BOOT[build_app_config]
+    BOOT --> CFG[AppConfig]
+    CFG --> APP[build_application]
+    APP --> GW[GroundworkersApp]
+    GW --> SVC[services]
+    GW --> ADP[adapters]
+    MCP[MCP client] --> TOOLS[MCP tools]
+    REST[REST client] --> API[REST transport]
+    PY[Python caller] --> SVC
+    TOOLS --> SVC
+    TOOLS -. adapter-backed primitives .-> ADP
+    API --> SVC
+```
+
+`build_application(...)` is the composition root. It builds one reusable
+runtime container with transport-agnostic services plus dependency-facing
+adapters. Most caller-facing workflows go through services; some MCP tools are
+intentionally adapter-backed when the capability is closer to a backend
+primitive than a domain service.
+
+## Quick start
+
+### Install
+
+```bash
+pip install groundworkers
+```
+
+Optional extras:
+
+```bash
+pip install "groundworkers[llm,embedding-pgvector]"
+```
+
+### Configure the shared stack
+
+```bash
+omop-config configure omop_alchemy
+omop-config configure omop_graph
+omop-config configure groundworkers
+# optional if you want embedding-backed capabilities
+omop-config configure omop_emb
+```
+
+### Start MCP
+
+```bash
+groundworkers --describe
+groundworkers --transport streamable-http --host 0.0.0.0 --port 8000
+```
+
+### Start REST
+
+```bash
+groundworkers --transport rest --host 0.0.0.0 --port 8080
+```
+
+### Use from Python
+
+```python
+from groundworkers.app import build_application
+from groundworkers.bootstrap import build_app_config
+
+config = build_app_config()
+app = build_application(config)
+
+mapping = app.services.mapping
+bundle = mapping.concept_candidate_bundle(
+    "type 2 diabetes",
+    domain="Condition",
+    include_normalized=True,
+    include_fulltext=True,
+    include_embedding=True,
+)
+```
+
+## Main surfaces
+
+| Surface | Best for |
+|---|---|
+| MCP tools | Tool discovery, agent interoperability, shared capability services |
+| REST routes | Typed HTTP workflows such as candidate bundles and assisted source planning |
+| `app.services.*` | In-process Python applications and batch workflows |
+| `app.adapters.*` | Backend wrappers used when you intentionally need dependency-shaped primitives |
+
+## Learn more
+
+- Docs home: `docs/index.md`
+- Configuration: `docs/usage/configuration.md`
+- Integrations: `docs/usage/integrations.md`
+- Architecture: `docs/architecture.md`
+
+## Companion repos
+
+- [groundcrew](https://github.com/AustralianCancerDataNetwork/groundcrew)
+- [omop-graph](https://australiancancerdatanetwork.github.io/omop-graph/)
+- [omop-emb](https://australiancancerdatanetwork.github.io/omop-emb/)

{groundworkers-0.2.0 → groundworkers-0.3.3}/pyproject.toml

@@ -1,27 +1,48 @@
 [project]
 name = "groundworkers"
-version = "0.2.0"
+version = "0.3.3"
 description = "Groundworkers MCP server — read-only agentive access to OMOP vocabularies, concept graphs, and embeddings."
 readme = "README.md"
 requires-python = ">=3.12"
 dependencies = [
+  "fastapi>=0.110,<1.0",
   "mcp[cli]>=1,<2",
   "pydantic>=2,<3",
   "pyyaml>=6,<7",
   "SQLAlchemy>=2,<3",
   "psycopg[binary]>=3.1,<4",
-  "omop-graph>=1.1.0",
-  "omop-emb>=1.0.0",
+  "oa-configurator>=0.1.2",
+  "omop-graph>=1.3.0",
+  "omop-emb>=1.1.1",
+  "uvicorn[standard]>=0.29,<1.0",
 ]
 
 [project.optional-dependencies]
+llm = [
+  "openai>=1.0",
+]
+xlsx = [
+  "openpyxl>=3.1,<4",
+]
+pdf = [
+  "pdfplumber>=0.10",
+]
+docx = [
+  "python-docx>=1.0",
+]
+all_source = [
+  "openpyxl>=3.1,<4",
+  "pdfplumber>=0.10",
+  "python-docx>=1.0",
+]
 embedding-pgvector = [
-  "omop-emb[pgvector]>=1.0.0",
+  "omop-emb[pgvector]>=1.1.1",
 ]
 embedding-faiss = [
-  "omop-emb[faiss-cpu]>=1.0.0",
+  "omop-emb[faiss-cpu]>=1.1.1",
 ]
 dev = [
+  "httpx>=0.27,<1",
   "ipython>=8.0",
   "tornado>=6.5.5",
   "pytest>=9.0.3",
@@ -38,6 +59,9 @@ dev = [
 [project.scripts]
 groundworkers = "groundworkers.server:main"
 
+[project.entry-points."omop.config"]
+groundworkers = "groundworkers.config:GroundworkersConfig"
+
 [build-system]
 requires = ["setuptools>=68", "wheel"]
 build-backend = "setuptools.build_meta"
@@ -48,6 +72,12 @@ package-dir = {"" = "src"}
 [tool.setuptools.packages.find]
 where = ["src"]
 
+[tool.setuptools.package-data]
+groundworkers = [
+  "knowledge/packs/*/*/*.md",
+  "knowledge/packs/*/*/*.yaml",
+]
+
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 markers = [

groundworkers-0.3.3/src/groundworkers/adapters/cdm.py

@@ -0,0 +1,43 @@
+from __future__ import annotations
+
+from sqlalchemy import text
+from sqlalchemy.engine import Engine
+from sqlalchemy.orm import sessionmaker
+
+
+class CDMAdapter:
+    """Adapter for a CDM (Common Data Model) database connection.
+
+    Holds the SQLAlchemy engine and session factory for an OMOP CDM database.
+    Shared by services that need to query the CDM directly (VocabService,
+    OmopGraphAdapter).
+
+    Pass engine to adapters that wrap their own session management.
+    Use session() for services that need a scoped session context manager.
+    """
+
+    def __init__(self, engine: Engine) -> None:
+        self._engine = engine
+        self._session_factory = sessionmaker(engine)
+
+    @property
+    def engine(self) -> Engine:
+        """The underlying SQLAlchemy engine."""
+        return self._engine
+
+    def session(self):
+        """Return a session context manager."""
+        return self._session_factory()
+
+    def is_available(self) -> bool:
+        """Return True if the CDM database is reachable (SELECT 1 probe)."""
+        try:
+            with self._engine.connect() as conn:
+                conn.execute(text("SELECT 1"))
+            return True
+        except Exception:
+            return False
+
+    def close(self) -> None:
+        """Dispose the engine and release the connection pool."""
+        self._engine.dispose()

groundworkers-0.3.3/src/groundworkers/adapters/llm.py

@@ -0,0 +1,177 @@
+from __future__ import annotations
+
+import json
+from collections.abc import Callable
+from typing import Any
+
+from groundworkers.base.errors import GroundworkersError
+
+_STATUS_TIMEOUT_SECONDS = 2.0
+_COMPLETION_TIMEOUT_SECONDS = 180.0
+
+
+class LLMAdapter:
+    """Adapter for OpenAI-compatible LLM chat completion APIs.
+
+    Works with any provider that implements the OpenAI chat completions API:
+    local deployments (Ollama, vLLM, LM Studio) and remote services (OpenAI,
+    Azure OpenAI, and compatible cloud APIs). Configure ``api_base`` to point
+    at the correct endpoint.
+
+    Two completion modes are available:
+
+    - **Text completion** (``complete_text``): returns a raw text response.
+    - **Structured completion** (``complete_structured``): requests a JSON
+      response matching a caller-supplied schema. Preferred for MCP-facing
+      tools where downstream agents need to parse the output reliably.
+    """
+
+    def __init__(
+        self,
+        *,
+        provider: str,
+        default_model_name: str | None = None,
+        client_factory: Callable[[], Any],
+    ) -> None:
+        self._provider = provider
+        self._default_model_name = default_model_name
+        self._client_factory = client_factory
+        self._client: Any = None
+
+    def is_available(self) -> bool:
+        """Return True if the LLM API is reachable."""
+        return self.status()["available"]
+
+    def close(self) -> None:
+        """Release the cached client."""
+        self._client = None
+
+    def status(self) -> dict[str, Any]:
+        """Return availability and configuration details. Never raises.
+
+        Probes the API with a short timeout. On failure returns
+        ``{"available": False, ..., "detail": "<reason>"}``.
+        """
+        try:
+            client = self._get_client()
+            client.models.list(timeout=_STATUS_TIMEOUT_SECONDS)
+            return {
+                "available": True,
+                "provider": self._provider,
+                "default_model": self._default_model_name,
+                "structured_output_supported": True,
+                "detail": None,
+            }
+        except Exception as exc:
+            return {
+                "available": False,
+                "provider": self._provider,
+                "default_model": self._default_model_name,
+                "structured_output_supported": None,
+                "detail": repr(exc),
+            }
+
+    def complete_text(
+        self,
+        prompt: str,
+        *,
+        system_prompt: str | None = None,
+        model_name: str | None = None,
+        temperature: float = 0.0,
+    ) -> dict[str, Any]:
+        """Complete a prompt and return the response text.
+
+        Raises ``INVALID_INPUT`` if no model is resolvable.
+        Raises ``BACKEND_UNAVAIL`` if the API call fails.
+        """
+        client = self._get_client()
+        resolved_model = self._resolve_model(model_name)
+        messages = _build_messages(prompt, system_prompt)
+        try:
+            response = client.chat.completions.create(
+                model=resolved_model,
+                messages=messages,
+                temperature=temperature,
+                timeout=_COMPLETION_TIMEOUT_SECONDS,
+            )
+        except Exception as exc:
+            raise GroundworkersError("BACKEND_UNAVAIL", f"LLM call failed: {exc}") from exc
+        return {
+            "text": response.choices[0].message.content,
+            "model": response.model,
+            "provider": self._provider,
+        }
+
+    def complete_structured(
+        self,
+        prompt: str,
+        response_schema: dict[str, Any],
+        *,
+        system_prompt: str | None = None,
+        model_name: str | None = None,
+        temperature: float = 0.0,
+    ) -> dict[str, Any]:
+        """Complete a prompt and return a parsed JSON dict guided by response_schema.
+
+        The schema is injected into the system prompt and JSON mode is requested
+        from the API. This is compatible with Ollama, vLLM, and OpenAI endpoints.
+
+        The response is parsed but not validated against the schema — callers are
+        responsible for validating the returned dict (e.g. with Pydantic).
+
+        Raises ``INVALID_INPUT`` if no model is resolvable or if response_schema
+        is not JSON-serializable.
+        Raises ``BACKEND_UNAVAIL`` if the API call fails.
+        Raises ``QUERY_ERROR`` if the response is not valid JSON.
+        """
+        client = self._get_client()
+        resolved_model = self._resolve_model(model_name)
+        try:
+            schema_json = json.dumps(response_schema, indent=2)
+        except (TypeError, ValueError) as exc:
+            raise GroundworkersError(
+                "INVALID_INPUT", f"response_schema is not JSON-serializable: {exc}"
+            ) from exc
+        schema_directive = f"Respond with a JSON object matching this schema:\n{schema_json}"
+        augmented_system = f"{system_prompt}\n\n{schema_directive}" if system_prompt else schema_directive
+        messages = _build_messages(prompt, augmented_system)
+        try:
+            response = client.chat.completions.create(
+                model=resolved_model,
+                messages=messages,
+                temperature=temperature,
+                response_format={"type": "json_object"},
+                timeout=_COMPLETION_TIMEOUT_SECONDS,
+            )
+        except Exception as exc:
+            raise GroundworkersError("BACKEND_UNAVAIL", f"LLM call failed: {exc}") from exc
+        content = response.choices[0].message.content or ""
+        try:
+            return json.loads(content)
+        except json.JSONDecodeError as exc:
+            raise GroundworkersError("QUERY_ERROR", f"LLM response was not valid JSON: {exc}") from exc
+
+    def _get_client(self) -> Any:
+        if self._client is None:
+            try:
+                self._client = self._client_factory()
+            except Exception as exc:
+                raise GroundworkersError("BACKEND_UNAVAIL", f"LLM client could not be initialised: {exc}") from exc
+        return self._client
+
+    def _resolve_model(self, model_name: str | None) -> str:
+        resolved = model_name or self._default_model_name
+        if resolved is None:
+            raise GroundworkersError(
+                "INVALID_INPUT",
+                "No model specified and no default model is configured",
+            )
+        return resolved
+
+
+def _build_messages(prompt: str, system_prompt: str | None) -> list[dict[str, str]]:
+    messages: list[dict[str, str]] = []
+    if system_prompt:
+        messages.append({"role": "system", "content": system_prompt})
+    messages.append({"role": "user", "content": prompt})
+    return messages