pennsieve-ai-utils 0.2.0__tar.gz

pennsieve_ai_utils-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pennsieve
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pennsieve_ai_utils-0.2.0/PKG-INFO

@@ -0,0 +1,166 @@
+Metadata-Version: 2.4
+Name: pennsieve-ai-utils
+Version: 0.2.0
+Summary: Shared blackboard contract, governor LLM client, and literature tools for the Pennsieve AI Co-Scientist stages.
+License: MIT License
+        
+        Copyright (c) 2026 Pennsieve
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Repository, https://github.com/Pennsieve/pennsieve-ai-utils
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pennsieve-llm<0.7,>=0.6.1
+Requires-Dist: httpx>=0.27
+Requires-Dist: json-repair>=0.30
+Requires-Dist: numpy>=1.26
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Dynamic: license-file
+
+# pennsieve-ai-utils
+
+Shared utilities for the Pennsieve AI Co-Scientist stages:
+
+1. `pennsieve-ai-scout`
+2. `pennsieve-ai-hypothesize`
+3. `pennsieve-ai-analyze`
+4. `pennsieve-ai-write`
+
+Each stage used to carry its own copy of the blackboard schema, the governor
+LLM client, the model-role resolver, the agent helpers and the literature
+tools. They now import them from here. The package is published to PyPI as
+`pennsieve-ai-utils` (same release flow as `pennsieve-llm`: push a `vX.Y.Z`
+tag), so the Pennsieve App Store build of each stage can `pip install` it
+without GitHub credentials.
+
+See [RUNBOOK.md](RUNBOOK.md) for local stage supervision, stopping, review
+bundles and the remaining Pennsieve Test deployment work.
+
+```bash
+pip install pennsieve-ai-utils
+```
+
+## What's here
+
+| Module | Owns | Used by |
+|---|---|---|
+| `pennsieve_ai_utils.blackboard` | Canonical `Blackboard` JSON contract, `schema_version` migrations, unknown-field preservation | all stages |
+| `pennsieve_ai_utils.llm` | `LLMClient`: governor transport via `pennsieve-llm`, role-based model selection, preflight, self-healing on 403/404, `usage.jsonl` logging | all stages |
+| `pennsieve_ai_utils.models` | Role table (`reasoning`, `synthesis`, `bulk`), capability ranking, allow-list discovery, operator pins | `llm` |
+| `pennsieve_ai_utils.agents` | `call_json`, `extract_json`, `first_text`, `format_datasets`, attack-tag helpers, `load_prompt` | every agent |
+| `pennsieve_ai_utils.processor` | Pennsieve processor env contract (`INPUT_DIR`/`OUTPUT_DIR`/run IDs), SIGTERM handling, upstream blackboard loading, output persistence | every `main.py` |
+| `pennsieve_ai_utils.tools` | CrossRef, DataCite, DOI lookup, JATS/Markdown chunkers, Bedrock embeddings + reranker, RAG store reader | scout, hypothesize, write |
+| `pennsieve_ai_utils.workflow` | Local four-stage supervisor, handoff gates, review bundles (`python -m pennsieve_ai_utils.workflow`) | operators |
+
+Stage-specific science — pipelines, prompts, Discover/DANDI/OpenNeuro
+inspection, notebook execution, manuscript rendering — stays in each app.
+
+## Blackboard contract
+
+```python
+from pennsieve_ai_utils import Blackboard
+
+bb = Blackboard.load("blackboard.json")
+bb.record_model_assignment(
+    stage="hypothesize",
+    role="reasoning",
+    assignment={"model_id": "us.anthropic.claude-sonnet-4-6"},
+)
+bb.save("blackboard.json")
+```
+
+- One superset schema carries fields produced by every stage.
+- `schema_version` enables explicit migrations; documents without a version
+  are treated as legacy version 0.
+- Unknown fields are preserved at their original object level on a
+  load/save round trip.
+- Model provenance is cumulative by stage (`model_assignments[stage][role]`)
+  rather than overwritten by the next application.
+
+## LLM client
+
+```python
+from pennsieve_ai_utils.llm import LLMClient
+from pennsieve_ai_utils.agents import call_json
+
+client = LLMClient(stage="hypothesize", roles=("reasoning", "synthesis"))
+assignments = client.preflight()          # raises ModelNotAvailable if a floor can't be met
+for role, assignment in assignments.items():
+    bb.record_model_assignment(stage="hypothesize", role=role, assignment=assignment)
+
+verdict = call_json(client, agent="critic", system=..., user=..., run_id=bb.run_id,
+                    model="reasoning")
+```
+
+Agents address models by **role**, never by vendor tier. The legacy names
+`opus` / `sonnet` / `haiku` still resolve as aliases of
+`reasoning` / `synthesis` / `bulk`, and `VP_MODEL_<ROLE>` pins keep working.
+
+Preflight asks the governor's `GET /v1/models` for the allow-list first and
+falls back to tripping a `model_not_allowed` 403 on older governors. Each
+assigned model is then verified with a one-token call so an IAM block or a
+retired Bedrock model fails at startup rather than twenty minutes in.
+
+The client needs `LLM_GOVERNOR_FUNCTION_NAME` (platform-injected) or
+`PENNSIEVE_LLM_MOCK=1` for offline tests — see
+[LLM access on compute nodes](https://docs.pennsieve.io/docs/llm-access-on-pennsieve-compute-nodes).
+
+## Processor helpers
+
+```python
+from pennsieve_ai_utils.processor import (
+    ProcessorEnv, install_sigterm_handler, load_upstream_blackboard, write_outputs,
+)
+
+install_sigterm_handler()
+env = ProcessorEnv.from_environ()
+bb = load_upstream_blackboard(env.input_dir / "blackboard.json",
+                              stage="analyze", upstream="Hypothesize",
+                              run_id=env.execution_run_id)
+...
+write_outputs(bb, env.output_dir, client.usage_log)
+```
+
+These follow the
+[Pennsieve processor contract](https://docs.pennsieve.io/docs/pennsieve-processors):
+read from `INPUT_DIR`, write to `OUTPUT_DIR`, exit non-zero on failure.
+
+## Releasing
+
+```bash
+# bump version in pyproject.toml, commit, then:
+git tag v0.2.0 && git push origin v0.2.0
+```
+
+`.github/workflows/publish-pypi.yml` builds and publishes via PyPI Trusted
+Publishing. One-time setup on pypi.org: add this repository as a trusted
+publisher for the `pennsieve-ai-utils` project (GitHub environment `pypi`).
+Apps pin `pennsieve-ai-utils>=0.2,<0.3` and pick up patch releases on rebuild.
+
+## Development
+
+```bash
+python -m pip install -e ".[dev]"
+python -m ruff check src tests
+python -m pytest
+```

pennsieve_ai_utils-0.2.0/README.md

@@ -0,0 +1,127 @@
+# pennsieve-ai-utils
+
+Shared utilities for the Pennsieve AI Co-Scientist stages:
+
+1. `pennsieve-ai-scout`
+2. `pennsieve-ai-hypothesize`
+3. `pennsieve-ai-analyze`
+4. `pennsieve-ai-write`
+
+Each stage used to carry its own copy of the blackboard schema, the governor
+LLM client, the model-role resolver, the agent helpers and the literature
+tools. They now import them from here. The package is published to PyPI as
+`pennsieve-ai-utils` (same release flow as `pennsieve-llm`: push a `vX.Y.Z`
+tag), so the Pennsieve App Store build of each stage can `pip install` it
+without GitHub credentials.
+
+See [RUNBOOK.md](RUNBOOK.md) for local stage supervision, stopping, review
+bundles and the remaining Pennsieve Test deployment work.
+
+```bash
+pip install pennsieve-ai-utils
+```
+
+## What's here
+
+| Module | Owns | Used by |
+|---|---|---|
+| `pennsieve_ai_utils.blackboard` | Canonical `Blackboard` JSON contract, `schema_version` migrations, unknown-field preservation | all stages |
+| `pennsieve_ai_utils.llm` | `LLMClient`: governor transport via `pennsieve-llm`, role-based model selection, preflight, self-healing on 403/404, `usage.jsonl` logging | all stages |
+| `pennsieve_ai_utils.models` | Role table (`reasoning`, `synthesis`, `bulk`), capability ranking, allow-list discovery, operator pins | `llm` |
+| `pennsieve_ai_utils.agents` | `call_json`, `extract_json`, `first_text`, `format_datasets`, attack-tag helpers, `load_prompt` | every agent |
+| `pennsieve_ai_utils.processor` | Pennsieve processor env contract (`INPUT_DIR`/`OUTPUT_DIR`/run IDs), SIGTERM handling, upstream blackboard loading, output persistence | every `main.py` |
+| `pennsieve_ai_utils.tools` | CrossRef, DataCite, DOI lookup, JATS/Markdown chunkers, Bedrock embeddings + reranker, RAG store reader | scout, hypothesize, write |
+| `pennsieve_ai_utils.workflow` | Local four-stage supervisor, handoff gates, review bundles (`python -m pennsieve_ai_utils.workflow`) | operators |
+
+Stage-specific science — pipelines, prompts, Discover/DANDI/OpenNeuro
+inspection, notebook execution, manuscript rendering — stays in each app.
+
+## Blackboard contract
+
+```python
+from pennsieve_ai_utils import Blackboard
+
+bb = Blackboard.load("blackboard.json")
+bb.record_model_assignment(
+    stage="hypothesize",
+    role="reasoning",
+    assignment={"model_id": "us.anthropic.claude-sonnet-4-6"},
+)
+bb.save("blackboard.json")
+```
+
+- One superset schema carries fields produced by every stage.
+- `schema_version` enables explicit migrations; documents without a version
+  are treated as legacy version 0.
+- Unknown fields are preserved at their original object level on a
+  load/save round trip.
+- Model provenance is cumulative by stage (`model_assignments[stage][role]`)
+  rather than overwritten by the next application.
+
+## LLM client
+
+```python
+from pennsieve_ai_utils.llm import LLMClient
+from pennsieve_ai_utils.agents import call_json
+
+client = LLMClient(stage="hypothesize", roles=("reasoning", "synthesis"))
+assignments = client.preflight()          # raises ModelNotAvailable if a floor can't be met
+for role, assignment in assignments.items():
+    bb.record_model_assignment(stage="hypothesize", role=role, assignment=assignment)
+
+verdict = call_json(client, agent="critic", system=..., user=..., run_id=bb.run_id,
+                    model="reasoning")
+```
+
+Agents address models by **role**, never by vendor tier. The legacy names
+`opus` / `sonnet` / `haiku` still resolve as aliases of
+`reasoning` / `synthesis` / `bulk`, and `VP_MODEL_<ROLE>` pins keep working.
+
+Preflight asks the governor's `GET /v1/models` for the allow-list first and
+falls back to tripping a `model_not_allowed` 403 on older governors. Each
+assigned model is then verified with a one-token call so an IAM block or a
+retired Bedrock model fails at startup rather than twenty minutes in.
+
+The client needs `LLM_GOVERNOR_FUNCTION_NAME` (platform-injected) or
+`PENNSIEVE_LLM_MOCK=1` for offline tests — see
+[LLM access on compute nodes](https://docs.pennsieve.io/docs/llm-access-on-pennsieve-compute-nodes).
+
+## Processor helpers
+
+```python
+from pennsieve_ai_utils.processor import (
+    ProcessorEnv, install_sigterm_handler, load_upstream_blackboard, write_outputs,
+)
+
+install_sigterm_handler()
+env = ProcessorEnv.from_environ()
+bb = load_upstream_blackboard(env.input_dir / "blackboard.json",
+                              stage="analyze", upstream="Hypothesize",
+                              run_id=env.execution_run_id)
+...
+write_outputs(bb, env.output_dir, client.usage_log)
+```
+
+These follow the
+[Pennsieve processor contract](https://docs.pennsieve.io/docs/pennsieve-processors):
+read from `INPUT_DIR`, write to `OUTPUT_DIR`, exit non-zero on failure.
+
+## Releasing
+
+```bash
+# bump version in pyproject.toml, commit, then:
+git tag v0.2.0 && git push origin v0.2.0
+```
+
+`.github/workflows/publish-pypi.yml` builds and publishes via PyPI Trusted
+Publishing. One-time setup on pypi.org: add this repository as a trusted
+publisher for the `pennsieve-ai-utils` project (GitHub environment `pypi`).
+Apps pin `pennsieve-ai-utils>=0.2,<0.3` and pick up patch releases on rebuild.
+
+## Development
+
+```bash
+python -m pip install -e ".[dev]"
+python -m ruff check src tests
+python -m pytest
+```

pennsieve_ai_utils-0.2.0/pyproject.toml

@@ -0,0 +1,46 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pennsieve-ai-utils"
+version = "0.2.0"
+description = "Shared blackboard contract, governor LLM client, and literature tools for the Pennsieve AI Co-Scientist stages."
+readme = "README.md"
+requires-python = ">=3.10"
+license = {file = "LICENSE"}
+dependencies = [
+    # Governor transport: returns a configured anthropic.Anthropic (pulls in
+    # anthropic, httpx2 and boto3).
+    "pennsieve-llm>=0.6.1,<0.7",
+    # Direct HTTP for CrossRef / DataCite. Declared explicitly because
+    # pennsieve-llm depends on httpx2, not httpx.
+    "httpx>=0.27",
+    # Second-chance parsing of LLM JSON output.
+    "json-repair>=0.30",
+    # Vector math for the RAG store reader.
+    "numpy>=1.26",
+]
+
+[project.urls]
+Repository = "https://github.com/Pennsieve/pennsieve-ai-utils"
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "ruff>=0.5",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint]
+# Classic ruff defaults; newer ruff releases widen the default set.
+select = ["E4", "E7", "E9", "F"]

pennsieve_ai_utils-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

pennsieve_ai_utils-0.2.0/src/pennsieve_ai_utils/__init__.py

@@ -0,0 +1,27 @@
+"""Shared utilities for the Pennsieve AI Co-Scientist applications."""
+
+from .blackboard import (
+    CURRENT_SCHEMA_VERSION,
+    Blackboard,
+    Claim,
+    CriticReview,
+    DatasetSummary,
+    DirectorBrief,
+    Hypothesis,
+)
+from .models import LEGACY_ALIASES, ROLES, ModelNotAvailable, Role, degraded_ok
+
+__all__ = [
+    "CURRENT_SCHEMA_VERSION",
+    "Blackboard",
+    "Claim",
+    "CriticReview",
+    "DatasetSummary",
+    "DirectorBrief",
+    "Hypothesis",
+    "LEGACY_ALIASES",
+    "ROLES",
+    "ModelNotAvailable",
+    "Role",
+    "degraded_ok",
+]

pennsieve_ai_utils-0.2.0/src/pennsieve_ai_utils/agents.py

@@ -0,0 +1,167 @@
+"""Helpers every LLM agent in the Co-Scientist stages uses: prompt loading,
+JSON extraction, prompt formatting of shared blackboard state, and the
+role-addressed `call_json` round trip."""
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from pennsieve_ai_utils.blackboard import Blackboard
+    from pennsieve_ai_utils.llm import LLMClient
+
+
+def load_prompt(prompts_dir: Path, name: str) -> str:
+    """Read `<prompts_dir>/<name>.md`. Each stage keeps its own prompts."""
+    return (Path(prompts_dir) / f"{name}.md").read_text()
+
+
+def extract_json(text: str) -> dict:
+    """Extract a JSON object from LLM output that may have surrounding prose or fences.
+
+    Two-stage parsing:
+      1. `json.loads(strict=False)` — handles literal newlines/tabs inside strings
+         (which LLMs commonly emit in free-text fields).
+      2. Fall back to `json_repair` for unescaped quotes, trailing commas, and
+         other common LLM JSON foibles.
+    """
+    fence = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", text, re.DOTALL)
+    if fence:
+        candidate = fence.group(1)
+    else:
+        brace = re.search(r"\{.*\}", text, re.DOTALL)
+        if not brace:
+            raise ValueError(f"No JSON object found in model output:\n{text[:500]}")
+        candidate = brace.group(0)
+
+    try:
+        return json.loads(candidate, strict=False)
+    except json.JSONDecodeError as strict_err:
+        try:
+            from json_repair import repair_json
+        except ImportError:
+            raise strict_err
+        try:
+            repaired = repair_json(candidate, return_objects=True)
+        except Exception:
+            raise strict_err
+        if isinstance(repaired, dict):
+            return repaired
+        raise ValueError(f"json_repair returned non-dict ({type(repaired).__name__})") from strict_err
+
+
+_ATTACK_TAGS = ("[CONSTRAINT]", "[REASONING]", "[ALIGNMENT]")
+
+
+def attack_kind(attack: str) -> str:
+    """Return 'constraint' | 'reasoning' | 'alignment' | 'untagged' for a Critic attack string."""
+    s = (attack or "").strip()
+    for tag in _ATTACK_TAGS:
+        if s.startswith(tag):
+            return tag[1:-1].lower()
+    return "untagged"
+
+
+def strip_attack_tag(attack: str) -> str:
+    """Remove the leading [TAG] prefix if present, for readable display."""
+    s = (attack or "").strip()
+    for tag in _ATTACK_TAGS:
+        if s.startswith(tag):
+            return s[len(tag):].strip()
+    return s
+
+
+def format_datasets(bb: "Blackboard") -> str:
+    """Render dataset summaries with depositor README, file-type histogram,
+    metadata-record previews, and linked publications. Used by every stage so
+    all agents reason from the same picture of what the data is."""
+    chunks = []
+    for d in bb.dataset_summaries:
+        head = (
+            f"[{d.id}] {d.title}\n"
+            f"    Species: {d.species}, n={d.n_subjects}\n"
+            f"    Measurements: {', '.join(str(m) for m in d.measurements)}\n"
+            f"    {d.description}"
+        )
+        extras = []
+        if d.bucket_uri:
+            extras.append(f"    Bucket URI: {d.bucket_uri}  (access: {d.access_type or 'unknown'})")
+        if d.readme:
+            extras.append(f"    README: {d.readme[:1500].strip()}")
+        if d.file_types:
+            ft = ", ".join(f"{k}={v}" for k, v in list(d.file_types.items())[:5])
+            extras.append(f"    File types: {ft}")
+        if d.file_tree:
+            tree_block = "\n".join("    " + line for line in d.file_tree.split("\n"))
+            extras.append(f"    File tree (from manifest.json — these are the ONLY paths that exist):\n{tree_block}")
+        if d.record_tables:
+            for fn, info in d.record_tables.items():
+                cols = ", ".join(info.get("columns", [])[:8])
+                more = " ..." if len(info.get("columns", [])) > 8 else ""
+                preview = info.get("head", [])[:3]
+                preview_str = "; ".join(
+                    "{" + ", ".join(f"{k}={v}" for k, v in row.items()) + "}"
+                    for row in preview
+                )
+                extras.append(
+                    f"    Record table `{fn}` ({info.get('n_rows', 0)} rows): "
+                    f"columns=[{cols}{more}]; first rows: {preview_str[:400]}"
+                )
+        if d.external_publications:
+            pubs = ", ".join(p.get("doi", "?") for p in d.external_publications[:3])
+            extras.append(f"    Linked publications: {pubs}")
+        chunks.append(head + ("\n" + "\n".join(extras) if extras else ""))
+    return "\n\n".join(chunks)
+
+
+def call_json(
+    client: "LLMClient",
+    *,
+    agent: str,
+    system: str,
+    user: str,
+    run_id: str,
+    model: str = "reasoning",
+    max_tokens: int = 16384,
+) -> dict:
+    """Call the model serving `model` (a pipeline role) and parse JSON from
+    its response.
+
+    Where the target model supports it we use the canonical "prefill `{`"
+    trick to force JSON output. Claude 4.6 and newer reject an
+    assistant-message prefill, so for those we skip it and rely on
+    `extract_json` to recover JSON from whatever comes back.
+    """
+    # Ask the client rather than keying off the role name: a role can be
+    # served by a different concrete model than its first choice, and prefill
+    # rules differ by model.
+    prefill_ok = client.supports_prefill(model)
+    msg = client.messages_create(
+        model=model,
+        system=system,
+        messages=[{"role": "user", "content": user}],
+        prefill="{" if prefill_ok else None,
+        max_tokens=max_tokens,
+        run_id=run_id,
+        agent=agent,
+    )
+    # Re-ask after the call: a role rejected mid-call is re-resolved onto
+    # another model, which may have different prefill rules.
+    prefilled = prefill_ok and client.supports_prefill(model)
+    return extract_json(("{" if prefilled else "") + first_text(msg))
+
+
+def first_text(msg) -> str:
+    """The first text block's text.
+
+    Never index `content[0]` directly: models with thinking enabled put a
+    `thinking` block first, so a role pinned or degraded onto one would
+    otherwise raise AttributeError on a `ThinkingBlock`.
+    """
+    for block in msg.content:
+        if getattr(block, "type", None) == "text":
+            return block.text
+    kinds = ", ".join(getattr(b, "type", "?") for b in msg.content) or "(empty)"
+    raise ValueError(f"No text block in model response; got: {kinds}")

pennsieve_ai_utils-0.2.0/src/pennsieve_ai_utils/blackboard/__init__.py

@@ -0,0 +1,22 @@
+"""Canonical blackboard schema and compatibility helpers."""
+
+from .migrations import CURRENT_SCHEMA_VERSION
+from .schema import (
+    Blackboard,
+    Claim,
+    CriticReview,
+    DatasetSummary,
+    DirectorBrief,
+    Hypothesis,
+)
+
+__all__ = [
+    "CURRENT_SCHEMA_VERSION",
+    "Blackboard",
+    "Claim",
+    "CriticReview",
+    "DatasetSummary",
+    "DirectorBrief",
+    "Hypothesis",
+]
+

pennsieve_ai_utils-0.2.0/src/pennsieve_ai_utils/blackboard/migrations.py

@@ -0,0 +1,61 @@
+"""Pure JSON-document migrations for the shared blackboard."""
+from __future__ import annotations
+
+from copy import deepcopy
+from typing import Any, Mapping
+
+
+CURRENT_SCHEMA_VERSION = 1
+
+
+class InvalidBlackboard(ValueError):
+    """The serialized blackboard is not a JSON object or has an invalid version."""
+
+
+def migrate_document(document: Mapping[str, Any]) -> dict[str, Any]:
+    """Return a migrated copy without mutating the caller's document.
+
+    Future versions are retained as-is. The serializer preserves unknown
+    fields, allowing an older reader to inspect and re-emit newer documents
+    without silently deleting data it does not yet understand.
+    """
+    if not isinstance(document, Mapping):
+        raise InvalidBlackboard("blackboard JSON must contain an object at the top level")
+
+    migrated = deepcopy(dict(document))
+    raw_version = migrated.get("schema_version", 0)
+    if isinstance(raw_version, bool) or not isinstance(raw_version, int) or raw_version < 0:
+        raise InvalidBlackboard(f"invalid blackboard schema_version: {raw_version!r}")
+
+    version = raw_version
+    while version < CURRENT_SCHEMA_VERSION:
+        if version == 0:
+            migrated = _migrate_v0_to_v1(migrated)
+        else:  # pragma: no cover - every supported version has a branch
+            raise InvalidBlackboard(f"no migration registered for schema version {version}")
+        version += 1
+
+    return migrated
+
+
+def _migrate_v0_to_v1(document: dict[str, Any]) -> dict[str, Any]:
+    document["schema_version"] = 1
+    document.setdefault("degraded", [])
+
+    assignments = document.get("model_assignments")
+    if not isinstance(assignments, dict):
+        document["model_assignments"] = {}
+    elif assignments and _looks_like_flat_assignments(assignments):
+        # Legacy apps keyed assignments directly by role and overwrote the
+        # prior stage. Preserve that provenance under an explicit namespace.
+        document["model_assignments"] = {"legacy": assignments}
+
+    return document
+
+
+def _looks_like_flat_assignments(assignments: dict[str, Any]) -> bool:
+    values = list(assignments.values())
+    return bool(values) and all(
+        isinstance(value, dict) and "model_id" in value for value in values
+    )
+