ba-agent-mcp 0.1.0__tar.gz

ba_agent_mcp-0.1.0/MANIFEST.in

@@ -0,0 +1,28 @@
+# Keep the published sdist to the runtime package + the PyPI readme only.
+# (The wheel already contains just the lib/kb/mcp_server/smoke packages.)
+# The git-based sdist finder would otherwise sweep in tests/, the internal
+# README, and other repo files — exclude them so nothing internal ships.
+include README_PYPI.md
+include pyproject.toml
+
+exclude README.md
+exclude RUNBOOK.md
+exclude Makefile
+exclude vendor.lock
+exclude CLAUDE.md
+exclude .env
+exclude .env.example
+
+prune tests
+prune docs
+prune evals
+prune logs
+prune skill
+prune .codegraph
+prune "info to ingest"
+
+global-exclude *.sql
+global-exclude .env
+global-exclude *.pyc
+global-exclude *.docx
+global-exclude *.pdf

ba_agent_mcp-0.1.0/PKG-INFO

@@ -0,0 +1,96 @@
+Metadata-Version: 2.4
+Name: ba-agent-mcp
+Version: 0.1.0
+Summary: MCP server exposing the Modern-Expo virtual Business Analyst Knowledge Base (read/derive tools).
+Author: Modern-Expo
+Keywords: mcp,model-context-protocol,business-analyst,knowledge-base,rag
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: Other/Proprietary License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Documentation
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: mcp>=1.2
+Requires-Dist: pypdf>=4
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+
+# ba-agent-mcp
+
+An [MCP](https://modelcontextprotocol.io) server that exposes the Modern-Expo
+virtual **Business Analyst Knowledge Base** as a small set of **read / derive**
+tools. It never writes to SUPPA Docs or the Knowledge Base and is deliberately
+kept off the governed write path — authoring documents goes through the
+human-gated `business-analyst` skill, not this server.
+
+## Install
+
+```bash
+uvx ba-agent-mcp          # run on demand in an isolated environment (recommended)
+# or
+pip install ba-agent-mcp  # then run: ba-agent-mcp
+```
+
+## Credentials — you supply your own
+
+**This package ships no credentials.** Each user provides their **own** keys and
+their **own** SUPPA identity. Nothing is ever read from the MCP client JSON
+config; secrets are resolved from the environment, in this order:
+
+1. real environment variables in the launching shell, then
+2. an explicit env file at `$BA_AGENT_ENV_FILE`, then
+3. `~/.ba-agent/.env`
+
+```bash
+# ~/.ba-agent/.env   (or export these in your shell)
+SUPABASE_URL=https://<your-project>.supabase.co
+SUPABASE_SERVICE_ROLE_KEY=...        # or SUPABASE_ANON_KEY for read-only
+SUPPA_API_KEY=...                    # your own SUPPA identity (or SUPPA_TOKEN)
+GEMINI_API_KEY=...                   # optional — enables hybrid (vector) search
+FIGMA_TOKEN=...                      # optional — enables analyze_figma
+```
+
+> Never put any secret in the MCP client JSON. A tool whose credentials are
+> absent returns a clear, structured error instead of failing the server, so you
+> can add capabilities incrementally.
+
+## Connect (any MCP client)
+
+```json
+{
+  "mcpServers": {
+    "ba-agent": {
+      "command": "uvx",
+      "args": ["ba-agent-mcp"],
+      "env": { "BA_AGENT_ENV_FILE": "/absolute/path/to/your/.env" }
+    }
+  }
+}
+```
+
+## Tools
+
+| Tool | Returns |
+| --- | --- |
+| `kb_search` | Ranked KB hits with provenance (title, breadcrumb, SUPPA anchor, `data_class`). |
+| `kb_assemble_context` | A cited grounding block for a question + coverage + strictest `data_class`. |
+| `kb_assemble_corpus` | The whole project corpus grouped by document, for cross-corpus analysis. |
+| `kb_list_docs` | Inventory of governed documents in a project (reads SUPPA under your own identity). |
+| `analyze_figma` | A structured design profile of a Figma file (requires `FIGMA_TOKEN`). |
+
+## Data governance
+
+The KB read tools connect with the Supabase service-role key, which bypasses
+row-level security, so they **fail closed at the application layer**: any chunk
+that is not explicitly `data_class = 'internal'` is excluded from results.
+Consequence — **`client`-classed content is not readable over MCP**. The
+append-only logs never contain document bodies, only ids, hashes, identity, and
+HTTP status.
+
+---
+
+Proprietary — Modern-Expo internal tooling. All rights reserved.

ba_agent_mcp-0.1.0/README_PYPI.md

@@ -0,0 +1,75 @@
+# ba-agent-mcp
+
+An [MCP](https://modelcontextprotocol.io) server that exposes the Modern-Expo
+virtual **Business Analyst Knowledge Base** as a small set of **read / derive**
+tools. It never writes to SUPPA Docs or the Knowledge Base and is deliberately
+kept off the governed write path — authoring documents goes through the
+human-gated `business-analyst` skill, not this server.
+
+## Install
+
+```bash
+uvx ba-agent-mcp          # run on demand in an isolated environment (recommended)
+# or
+pip install ba-agent-mcp  # then run: ba-agent-mcp
+```
+
+## Credentials — you supply your own
+
+**This package ships no credentials.** Each user provides their **own** keys and
+their **own** SUPPA identity. Nothing is ever read from the MCP client JSON
+config; secrets are resolved from the environment, in this order:
+
+1. real environment variables in the launching shell, then
+2. an explicit env file at `$BA_AGENT_ENV_FILE`, then
+3. `~/.ba-agent/.env`
+
+```bash
+# ~/.ba-agent/.env   (or export these in your shell)
+SUPABASE_URL=https://<your-project>.supabase.co
+SUPABASE_SERVICE_ROLE_KEY=...        # or SUPABASE_ANON_KEY for read-only
+SUPPA_API_KEY=...                    # your own SUPPA identity (or SUPPA_TOKEN)
+GEMINI_API_KEY=...                   # optional — enables hybrid (vector) search
+FIGMA_TOKEN=...                      # optional — enables analyze_figma
+```
+
+> Never put any secret in the MCP client JSON. A tool whose credentials are
+> absent returns a clear, structured error instead of failing the server, so you
+> can add capabilities incrementally.
+
+## Connect (any MCP client)
+
+```json
+{
+  "mcpServers": {
+    "ba-agent": {
+      "command": "uvx",
+      "args": ["ba-agent-mcp"],
+      "env": { "BA_AGENT_ENV_FILE": "/absolute/path/to/your/.env" }
+    }
+  }
+}
+```
+
+## Tools
+
+| Tool | Returns |
+| --- | --- |
+| `kb_search` | Ranked KB hits with provenance (title, breadcrumb, SUPPA anchor, `data_class`). |
+| `kb_assemble_context` | A cited grounding block for a question + coverage + strictest `data_class`. |
+| `kb_assemble_corpus` | The whole project corpus grouped by document, for cross-corpus analysis. |
+| `kb_list_docs` | Inventory of governed documents in a project (reads SUPPA under your own identity). |
+| `analyze_figma` | A structured design profile of a Figma file (requires `FIGMA_TOKEN`). |
+
+## Data governance
+
+The KB read tools connect with the Supabase service-role key, which bypasses
+row-level security, so they **fail closed at the application layer**: any chunk
+that is not explicitly `data_class = 'internal'` is excluded from results.
+Consequence — **`client`-classed content is not readable over MCP**. The
+append-only logs never contain document bodies, only ids, hashes, identity, and
+HTTP status.
+
+---
+
+Proprietary — Modern-Expo internal tooling. All rights reserved.

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/PKG-INFO

@@ -0,0 +1,96 @@
+Metadata-Version: 2.4
+Name: ba-agent-mcp
+Version: 0.1.0
+Summary: MCP server exposing the Modern-Expo virtual Business Analyst Knowledge Base (read/derive tools).
+Author: Modern-Expo
+Keywords: mcp,model-context-protocol,business-analyst,knowledge-base,rag
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: Other/Proprietary License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Documentation
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: mcp>=1.2
+Requires-Dist: pypdf>=4
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+
+# ba-agent-mcp
+
+An [MCP](https://modelcontextprotocol.io) server that exposes the Modern-Expo
+virtual **Business Analyst Knowledge Base** as a small set of **read / derive**
+tools. It never writes to SUPPA Docs or the Knowledge Base and is deliberately
+kept off the governed write path — authoring documents goes through the
+human-gated `business-analyst` skill, not this server.
+
+## Install
+
+```bash
+uvx ba-agent-mcp          # run on demand in an isolated environment (recommended)
+# or
+pip install ba-agent-mcp  # then run: ba-agent-mcp
+```
+
+## Credentials — you supply your own
+
+**This package ships no credentials.** Each user provides their **own** keys and
+their **own** SUPPA identity. Nothing is ever read from the MCP client JSON
+config; secrets are resolved from the environment, in this order:
+
+1. real environment variables in the launching shell, then
+2. an explicit env file at `$BA_AGENT_ENV_FILE`, then
+3. `~/.ba-agent/.env`
+
+```bash
+# ~/.ba-agent/.env   (or export these in your shell)
+SUPABASE_URL=https://<your-project>.supabase.co
+SUPABASE_SERVICE_ROLE_KEY=...        # or SUPABASE_ANON_KEY for read-only
+SUPPA_API_KEY=...                    # your own SUPPA identity (or SUPPA_TOKEN)
+GEMINI_API_KEY=...                   # optional — enables hybrid (vector) search
+FIGMA_TOKEN=...                      # optional — enables analyze_figma
+```
+
+> Never put any secret in the MCP client JSON. A tool whose credentials are
+> absent returns a clear, structured error instead of failing the server, so you
+> can add capabilities incrementally.
+
+## Connect (any MCP client)
+
+```json
+{
+  "mcpServers": {
+    "ba-agent": {
+      "command": "uvx",
+      "args": ["ba-agent-mcp"],
+      "env": { "BA_AGENT_ENV_FILE": "/absolute/path/to/your/.env" }
+    }
+  }
+}
+```
+
+## Tools
+
+| Tool | Returns |
+| --- | --- |
+| `kb_search` | Ranked KB hits with provenance (title, breadcrumb, SUPPA anchor, `data_class`). |
+| `kb_assemble_context` | A cited grounding block for a question + coverage + strictest `data_class`. |
+| `kb_assemble_corpus` | The whole project corpus grouped by document, for cross-corpus analysis. |
+| `kb_list_docs` | Inventory of governed documents in a project (reads SUPPA under your own identity). |
+| `analyze_figma` | A structured design profile of a Figma file (requires `FIGMA_TOKEN`). |
+
+## Data governance
+
+The KB read tools connect with the Supabase service-role key, which bypasses
+row-level security, so they **fail closed at the application layer**: any chunk
+that is not explicitly `data_class = 'internal'` is excluded from results.
+Consequence — **`client`-classed content is not readable over MCP**. The
+append-only logs never contain document bodies, only ids, hashes, identity, and
+HTTP status.
+
+---
+
+Proprietary — Modern-Expo internal tooling. All rights reserved.

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/SOURCES.txt

@@ -0,0 +1,71 @@
+MANIFEST.in
+README_PYPI.md
+pyproject.toml
+ba_agent_mcp.egg-info/PKG-INFO
+ba_agent_mcp.egg-info/SOURCES.txt
+ba_agent_mcp.egg-info/dependency_links.txt
+ba_agent_mcp.egg-info/entry_points.txt
+ba_agent_mcp.egg-info/requires.txt
+ba_agent_mcp.egg-info/top_level.txt
+kb/__init__.py
+kb/adopt_doc.py
+kb/alerts.py
+kb/backfill_embeddings.py
+kb/chunker.py
+kb/embeddings.py
+kb/eval_kb.py
+kb/extend_doc_types.py
+kb/fast_metadata.py
+kb/ingest.py
+kb/instant_ingest.py
+kb/kb_smoke.py
+kb/reconcile.py
+kb/reference_loader.py
+kb/rerank.py
+kb/retrieval_eval.py
+kb/retrieve.py
+kb/search_client.py
+kb/supabase_rest.py
+lib/__init__.py
+lib/analysis_session.py
+lib/badoc_schema.py
+lib/baseline.py
+lib/citations.py
+lib/doc_lint.py
+lib/docx_text.py
+lib/draft.py
+lib/e2e.py
+lib/egress_audit.py
+lib/figma_analyze.py
+lib/figma_client.py
+lib/figma_doc.py
+lib/figma_extract.py
+lib/figma_publish.py
+lib/health.py
+lib/ingest_hook.py
+lib/migrate_to_theme_docs.py
+lib/project_doc.py
+lib/publish_prep.py
+lib/readback.py
+lib/reconcile_e2e.py
+lib/reconcile_plan.py
+lib/req_matcher.py
+lib/requirement_id.py
+lib/restore_lister.py
+lib/serializer.py
+lib/source_resolver.py
+lib/suppa_adapter.py
+lib/suppa_client.py
+lib/token_gate.py
+lib/transcript_store.py
+lib/validate_doc.py
+lib/write_sequencer.py
+mcp_server/__init__.py
+mcp_server/config.py
+mcp_server/server.py
+mcp_server/tools.py
+smoke/__init__.py
+smoke/canonical.py
+smoke/check_supabase.py
+smoke/fixture_brd.py
+smoke/phase0_smoke.py

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+ba-agent-mcp = mcp_server.server:main

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/requires.txt

@@ -0,0 +1,5 @@
+mcp>=1.2
+pypdf>=4
+
+[dev]
+pytest>=8

ba_agent_mcp-0.1.0/ba_agent_mcp.egg-info/top_level.txt

@@ -0,0 +1,4 @@
+kb
+lib
+mcp_server
+smoke

ba_agent_mcp-0.1.0/kb/adopt_doc.py

@@ -0,0 +1,177 @@
+"""Backfill: adopt an EXISTING SUPPA Doc into the BA governance layer + KB
+(BUILD_PLAN 2.9). The inventory (docs/INVENTORY_2_9.md) found legacy Docs with
+no BADoc rows; this turns one such Doc into a governed, retrievable document.
+
+Distinct from write_sequencer (which authors NEW content): the content already
+lives in SUPPA, so adoption does NOT write blocks. It:
+  1. verifies the Doc has live pages;
+  2. computes the live-content hash (kb.ingest.live_content_hash — sentinels
+     excluded, identical to ingest/reconcile);
+  3. registers a BADoc row pointing at the EXISTING Doc — inverted-order-lite:
+     insert status='writing' (external_id=doc_key:1, content_hash=live_hash,
+     no content write) -> flip 'published' + content_ref=doc_id -> read-back.
+     Idempotent on external_id (re-read on unique conflict); refuses to repoint
+     a doc_key already published against a DIFFERENT doc_id;
+  4. best-effort KB ingest (chunk + Gemini-embed + index). Ingest failure does
+     NOT fail adoption — the BADoc row stands and `make kb-reconcile` /
+     `kb-embed-backfill` heal the index later (same contract as instant ingest).
+
+Metadata (project / doc_key / doc_type / data_class / feature) is supplied by
+the operator — NOTHING is invented here. The BA Lead provides real values per
+doc from the inventory verdicts.
+
+CLI:
+  python -m kb.adopt_doc <doc_id> --project P --doc-key P/tech_doc/api \\
+      --doc-type tech_doc --data-class internal [--feature F] [--no-ingest]
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from kb import ingest as kb_ingest
+from kb.supabase_rest import KBRestError, SupabaseREST
+from lib.draft import DATA_CLASSES
+from lib.suppa_client import SuppaError, SuppaHTTPError
+
+
+class AdoptError(SuppaError):
+    pass
+
+
+@dataclass
+class AdoptResult:
+    status: str            # adopted | noop-already-adopted
+    doc_id: int
+    badoc_id: int | None
+    external_id: str
+    content_hash: str
+    ingest_status: str = "skipped"
+    chunks: int = 0
+    detail: str = ""
+
+
+def adopt_doc(doc_id: int, *, project: str, doc_key: str, doc_type: str,
+              data_class: str, suppa, rest: SupabaseREST | None = None,
+              feature: str | None = None, source: str = "backfill",
+              ingest: bool = True, doc_title: str | None = None) -> AdoptResult:
+    if data_class not in DATA_CLASSES:
+        raise AdoptError(f"data_class {data_class!r} not in {sorted(DATA_CLASSES)}")
+    doc_id = int(doc_id)
+    ext = f"{doc_key}:1"          # adopted legacy docs are version 1 of their doc_key
+
+    pages = kb_ingest.fetch_pages(suppa, doc_id)
+    if not pages:
+        raise AdoptError(f"doc {doc_id} has no live pages — nothing to adopt")
+    live_hash = kb_ingest.live_content_hash(pages)
+
+    # idempotency / conflict guard on the doc_key
+    published = [r for r in suppa.search_badoc(doc_key=doc_key, status="published")]
+    for r in published:
+        if r.get("content_ref") and int(r["content_ref"]) != doc_id:
+            raise AdoptError(
+                f"doc_key {doc_key!r} already published against doc "
+                f"{r['content_ref']} (not {doc_id}) — refusing to repoint")
+
+    existing = suppa.search_badoc(external_id=ext)
+    if existing and existing[0].get("status") == "published":
+        row = existing[0]
+        badoc_id = row.get("id")
+        # refresh content_hash if the legacy page changed since last adoption
+        if row.get("content_hash") != live_hash:
+            suppa.update_badoc(badoc_id, {"content_hash": live_hash})
+        result = AdoptResult("noop-already-adopted", doc_id, badoc_id, ext, live_hash)
+    else:
+        badoc_id = _register_badoc(suppa, ext, doc_id, live_hash, project=project,
+                                   feature=feature, doc_type=doc_type,
+                                   data_class=data_class, doc_key=doc_key, source=source)
+        result = AdoptResult("adopted", doc_id, badoc_id, ext, live_hash)
+
+    if ingest:
+        _ingest_best_effort(result, suppa, rest, project=project, feature=feature,
+                            doc_type=doc_type, data_class=data_class, doc_id=doc_id,
+                            content_hash=live_hash, doc_title=doc_title)
+    return result
+
+
+def _register_badoc(suppa, ext, doc_id, live_hash, *, project, feature, doc_type,
+                    data_class, doc_key, source) -> int:
+    row = {
+        "project": project, "feature": feature, "doc_type": doc_type,
+        "status": "writing", "data_class": data_class, "source": source,
+        "doc_key": doc_key, "version": 1, "external_id": ext,
+        "content_hash": live_hash,
+    }
+    try:
+        badoc_id = suppa.insert_badoc(row)
+    except SuppaHTTPError:
+        # unique conflict on external_id -> re-read, never parse the body
+        rows = suppa.search_badoc(external_id=ext)
+        if len(rows) != 1:
+            raise
+        badoc_id = rows[0]["id"]
+        if rows[0].get("content_hash") != live_hash:
+            suppa.update_badoc(badoc_id, {"content_hash": live_hash})
+    # flip to published pointing at the EXISTING doc; read-back
+    suppa.update_badoc(badoc_id, {"status": "published", "content_ref": doc_id})
+    back = suppa.search_badoc(external_id=ext)
+    if not back or back[0].get("status") != "published":
+        raise AdoptError(f"publish read-back failed for {ext}")
+    return badoc_id
+
+
+def _ingest_best_effort(result: AdoptResult, suppa, rest, *, project, feature,
+                        doc_type, data_class, doc_id, content_hash,
+                        doc_title=None) -> None:
+    badoc_row = {
+        "id": result.badoc_id, "external_id": result.external_id,
+        "project": project, "feature": feature, "doc_type": doc_type,
+        "status": "published", "data_class": data_class,
+        "content_hash": content_hash, "content_ref": doc_id,
+    }
+    try:
+        rest = rest or SupabaseREST()
+        res = kb_ingest.ingest_doc(badoc_row, suppa, rest, doc_title=doc_title)
+        result.ingest_status = res.status
+        result.chunks = res.chunks
+        if res.status == "failed":
+            kb_ingest.log_ingest_error(rest, doc_id, res.detail)
+    except KBRestError as e:
+        result.ingest_status = "error"
+        result.detail = str(e)[:200]
+        try:
+            kb_ingest.log_ingest_error(rest, doc_id, str(e))
+        except KBRestError:
+            pass  # reconcile scans had_errors anyway
+
+
+def main(argv=None) -> int:
+    import argparse
+    import sys
+    from lib.suppa_adapter import SuppaAdapter
+
+    p = argparse.ArgumentParser(description="Adopt an existing SUPPA Doc (BUILD_PLAN 2.9)")
+    p.add_argument("doc_id", type=int)
+    p.add_argument("--project", required=True)
+    p.add_argument("--doc-key", required=True, help="unique doc_key, e.g. PROJ/tech_doc/api")
+    p.add_argument("--doc-type", required=True)
+    p.add_argument("--data-class", required=True, choices=sorted(DATA_CLASSES))
+    p.add_argument("--feature", default=None)
+    p.add_argument("--source", default="backfill")
+    p.add_argument("--no-ingest", action="store_true")
+    args = p.parse_args(argv)
+
+    res = adopt_doc(args.doc_id, project=args.project, doc_key=args.doc_key,
+                    doc_type=args.doc_type, data_class=args.data_class,
+                    suppa=SuppaAdapter(), feature=args.feature, source=args.source,
+                    ingest=not args.no_ingest)
+    print(f"{res.status}: badoc={res.badoc_id} doc={res.doc_id} ext={res.external_id} "
+          f"hash={res.content_hash[:16]}… ingest={res.ingest_status} chunks={res.chunks}"
+          + (f" detail={res.detail}" if res.detail else ""))
+    return 0
+
+
+if __name__ == "__main__":
+    import sys
+    sys.stdout.reconfigure(encoding="utf-8")
+    raise SystemExit(main())

ba_agent_mcp-0.1.0/kb/alerts.py

@@ -0,0 +1,42 @@
+"""One alert sink for ALL hard failures (BUILD_PLAN 2.4 DoD): ingest retry
+exhaustion, sequencer hard errors, reconcile manual-edit warnings.
+
+Pilot transport: append-only logs/alerts.jsonl (ids/hashes only — never
+bodies, PRR §3.8) + optional webhook POST when ALERT_WEBHOOK_URL is set
+(channel decision — SUPPA task vs chat webhook — pending with the customer;
+recorded in BUILD_PLAN). Alerting must never raise into the caller.
+"""
+
+from __future__ import annotations
+
+import datetime
+import json
+import os
+import urllib.request
+from pathlib import Path
+
+DEFAULT_DIR = Path(__file__).resolve().parent.parent / "logs"
+
+
+def alert(kind: str, *, run_dir=None, **fields) -> dict:
+    entry = {
+        "ts": datetime.datetime.now(datetime.timezone.utc).isoformat(),
+        "alert": kind, **fields,
+    }
+    try:
+        out_dir = Path(run_dir) if run_dir else DEFAULT_DIR
+        out_dir.mkdir(parents=True, exist_ok=True)
+        with open(out_dir / "alerts.jsonl", "a", encoding="utf-8") as f:
+            f.write(json.dumps(entry, ensure_ascii=False, default=str) + "\n")
+    except Exception:
+        pass
+    url = os.environ.get("ALERT_WEBHOOK_URL", "")
+    if url:
+        try:
+            req = urllib.request.Request(
+                url, data=json.dumps(entry, ensure_ascii=False).encode("utf-8"),
+                headers={"Content-Type": "application/json"}, method="POST")
+            urllib.request.urlopen(req, timeout=10).read()
+        except Exception:
+            pass  # the file entry above is the durable record
+    return entry

ba_agent_mcp-0.1.0/kb/backfill_embeddings.py

@@ -0,0 +1,67 @@
+"""Embedding backfill (`make kb-embed-backfill`) — fills NULL embeddings
+WITHOUT re-ingesting content (the cheap heal path after chunks were ingested
+while GEMINI_API_KEY was absent or the API was down).
+
+Loops kb_missing_embeddings -> Gemini embed_texts -> kb_set_embeddings until
+none remain. ingestion_log had_errors rows are left to `make kb-reconcile`,
+which force-reingests them once and then converges (embeddings now succeed).
+"""
+
+from __future__ import annotations
+
+import sys
+
+from kb import embeddings
+from kb.supabase_rest import SupabaseREST
+from lib import egress_audit
+
+
+def backfill(rest: SupabaseREST | None = None, *, project: str | None = None,
+             batch: int = 100, sink=None) -> int:
+    rest = rest or SupabaseREST()
+    total = 0
+    while True:
+        rows = rest.rpc("kb_missing_embeddings",
+                        {"p_project": project, "p_limit": batch})
+        if not rows:
+            return total
+        texts = [r["content"] for r in rows]
+        # Chunk CONTENT egresses to Gemini here — audit it (always) and honor the
+        # fail-closed BLOCK switch (GEMINI_EGRESS_CLIENT_BLOCK=1). A blocked batch
+        # raises EgressBlocked: skip it (chunks keep their NULL embeddings) and
+        # stop — kb_missing_embeddings would just re-return the same rows, so
+        # looping would spin forever. They re-embed once the block lifts
+        # (best-effort contract, mirrors ingest/reconcile).
+        data_class = next((r.get("data_class") for r in rows if r.get("data_class")),
+                          None) or "client"
+        try:
+            egress_audit.check_and_record(
+                data_class=data_class, path="embeddings",
+                byte_count=sum(len(t.encode("utf-8")) for t in texts), sink=sink)
+        except egress_audit.EgressBlocked:
+            return total
+        vectors = embeddings.embed_texts(texts)
+        # C3: carry content_hash so kb_set_embeddings only writes the vector when
+        # the chunk's content is still the one we embedded (refuses to overwrite a
+        # row a concurrent re-ingest has since replaced — no silent vector/content skew).
+        n = rest.rpc("kb_set_embeddings", {"p": {"items": [
+            {"id": r["id"], "embedding": embeddings.vector_literal(v),
+             "content_hash": r.get("content_hash")}
+            for r, v in zip(rows, vectors)]}})
+        total += int(n or 0)
+        if len(rows) < batch:
+            return total
+
+
+def main() -> int:
+    if not embeddings.available():
+        print("GEMINI_API_KEY is not set — nothing to backfill with")
+        return 1
+    n = backfill()
+    print(f"backfilled embeddings for {n} chunks")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.stdout.reconfigure(encoding="utf-8")
+    raise SystemExit(main())