loopctl-mcp-server 2.33.0 → 2.34.0

package/README.md

@@ -62,10 +62,56 @@ Or if installed locally:
 | `LOOPCTL_API_KEY` | Global API key override (if set, always used) | -- |
 | `LOOPCTL_ORCH_KEY` | Orchestrator role API key (verify, reject, review, import) | -- |
 | `LOOPCTL_AGENT_KEY` | Agent role API key (contract, claim, start, request-review) | -- |
-| `LOOPCTL_USER_KEY` | User role API key. Required ONLY for destructive admin tools like `knowledge_bulk_publish`. Leave unset if you don't use those tools. | -- |
+| `LOOPCTL_USER_KEY` | User role API key (minted at signup). Required for **first-time BYO LLM key provisioning** (`set_llm_config` / `llm_config` — see [First-time setup](#first-time-setup--provision-your-byo-llm-keys)) and for destructive admin tools like `knowledge_bulk_publish`. | -- |
+| `LOOPCTL_STH_STATE_PATH` | Absolute path for the witness-protocol STH cache file (see [Witness protocol](#witness-protocol-sth)). Optional. | per-(server + key) file under the OS temp dir |
 
 Key resolution priority: `LOOPCTL_API_KEY` > tool-specific key > `LOOPCTL_ORCH_KEY`.
 
+## First-time setup — provision your BYO LLM keys
+
+**loopctl is agent-native and strictly BYO (bring-your-own-keys).** loopctl fronts
+**no** LLM cost — the knowledge wiki runs entirely on *your* provider keys, which you
+provision once and which bill you directly. Two SEPARATE keys power two capabilities:
+
+| Key | Provider | Powers | Missing ⇒ |
+|---|---|---|---|
+| `api_key` | Anthropic | knowledge **ingest** — extraction, classification, merge synthesis | `knowledge_ingest` returns **422** (`code: no_api_key`) |
+| `embedding_api_key` | OpenAI-compatible | article **embeddings + semantic search** | `knowledge_search` silently degrades to **keyword-only** (`meta.fallback_reason: no_embedding_key`) |
+
+Both keys are stored **encrypted** and are **never returned** by any tool. You can set
+per-operation model overrides (`extraction_model`, `classification_model`,
+`merge_model`, `embedding_model`); each defaults server-side when omitted.
+
+### The smooth path (once, at onboarding)
+
+1. **Sign up** and obtain your keys. Tenant signup is anchored to a hardware
+   authenticator (WebAuthn) — the one human touch. Signup mints your **user-role**
+   API key. Also grab your **agent** and **orchestrator** keys for day-to-day work.
+2. **Set the env** in your `.mcp.json` (see [Configuration](#configuration)):
+   `LOOPCTL_SERVER`, `LOOPCTL_USER_KEY` (needed for this step), plus `LOOPCTL_AGENT_KEY`
+   / `LOOPCTL_ORCH_KEY`.
+3. **Provision both keys in one call** (uses `LOOPCTL_USER_KEY`):
+
+   ```
+   set_llm_config({ api_key: "sk-ant-...", embedding_api_key: "sk-..." })
+   ```
+
+   Partial-merge: you can set or rotate one key at a time; omitting a key leaves the
+   existing one untouched. Optionally pass model overrides in the same call.
+4. **You're live.** `knowledge_ingest` extracts articles and `knowledge_search`
+   (combined/semantic) ranks by meaning. Check status anytime with
+   `llm_config` (reports `has_api_key` / `has_embedding_key` + masked last-4 hints,
+   never the key).
+
+### Self-healing: you can't get stuck
+
+If you call `knowledge_ingest` or `knowledge_search` **before** provisioning, the
+tool result **leads with an `ACTION REQUIRED` notice** and a machine-readable
+`remediation` object — naming the `set_llm_config` tool, a copy-paste example, the
+REST endpoint (`PATCH /api/v1/tenants/me/llm-config`), and the docs — so you (or an
+autonomous agent) can self-remediate without a human. Full agent-tenant lifecycle:
+[`docs/onboarding-agent-tenant.md`](../docs/onboarding-agent-tenant.md).
+
 ## Tools (72)
 
 ### Project Tools
@@ -176,7 +222,7 @@ Key resolution priority: `LOOPCTL_API_KEY` > tool-specific key > `LOOPCTL_ORCH_K
 | `knowledge_drafts` | List draft (unpublished) knowledge articles with pagination. Optional: `limit` (default 20, max 1000 — over-max → 400, no silent clamp), `offset` (default 0), `project_id`. Returns `meta.total_count`. |
 | `knowledge_lint` | Run a lint check on the knowledge wiki to identify stale or low-coverage articles. Optional: `project_id`, `stale_days`, `min_coverage`, `max_per_category` (default 50, max 500). True totals returned in `summary.total_per_category`. |
 | `knowledge_export` | Export all knowledge articles as an OKF v0.1 bundle (gzipped tar archive, unbounded, bounded-memory streaming, fail-closed). Returns a curl command for direct download — **the download requires `LOOPCTL_USER_KEY`** (an orchestrator key would 403). Pass `format=json` for buffered in-memory JSON (convenience tool for file writers; capped at `export_max_buffered_export_articles` — returns 413 if over-cap). Optional: `project_id`, `format` (`tar.gz` default or `json`). |
-| `knowledge_ingest` | Submit a URL or raw content for knowledge extraction. Enqueues an Oban job. Extracted articles are **drafts by default** (lower-trust LLM output); pass `publish: true` to publish on extraction. Required: `source_type`. One of: `url` or `content`. Optional: `project_id`, `publish`. |
+| `knowledge_ingest` | Submit a URL or raw content for knowledge extraction. Enqueues an Oban job. Extracted articles are **drafts by default** (lower-trust LLM output); pass `publish: true` to publish on extraction. **BYO:** runs on the tenant's own Anthropic key — a keyless tenant gets a 422 whose result leads with an `ACTION REQUIRED` notice pointing at `set_llm_config` (see [First-time setup](#first-time-setup--provision-your-byo-llm-keys)). Required: `source_type`. One of: `url` or `content`. Optional: `project_id`, `publish`. |
 | `knowledge_ingest_batch` | Submit up to 50 ingestion items in a single request. Each item has the same shape as `knowledge_ingest` (incl. `publish`). Returns per-item results. Required: `items`. Optional: batch-level `project_id` / `publish` defaults. |
 | `knowledge_ingestion_jobs` | List recent content ingestion jobs (last 7 days, max 50). |
 
@@ -184,8 +230,8 @@ Key resolution priority: `LOOPCTL_API_KEY` > tool-specific key > `LOOPCTL_ORCH_K
 
 | Tool | Description |
 |---|---|
-| `llm_config` | Get the tenant's BYO Anthropic LLM config: per-operation models, `has_api_key`, and a masked last-4 hint. Never returns the key. Requires **user** key. |
-| `set_llm_config` | Set/rotate the tenant's OWN Anthropic API key (stored encrypted, never returned) + the three per-operation models (`extraction_model`/`classification_model`/`merge_model`). Any subset; omitting `api_key` leaves it untouched. Requires **user** key. |
+| `llm_config` | **Check your onboarding status.** Get the tenant's BYO LLM config: per-operation models and whether each key is set — `has_api_key` (Anthropic) / `has_embedding_key` (OpenAI embedding) — plus masked last-4 hints. Never returns a key. Requires **user** key. |
+| `set_llm_config` | **First-time setup (do once).** Set/rotate the tenant's OWN **Anthropic `api_key`** (powers ingest) AND **OpenAI `embedding_api_key`** (powers semantic search) — both stored encrypted, never returned — plus the per-operation models (`extraction_model`/`classification_model`/`merge_model`/`embedding_model`). Any subset; partial-merge (omitting a key leaves it untouched). See [First-time setup](#first-time-setup--provision-your-byo-llm-keys). Requires **user** key. |
 | `knowledge_llm_usage` | Per-tenant LLM token-usage summary, grouped by operation + model + source_type + day over an optional `from`/`to` range (defaults to a 90-day lookback; effective window echoed in `meta.from`/`meta.to`), with `limit`/`offset` pagination. Record-only (no budget enforcement). Requires orchestrator key. |
 
 ### Knowledge Analytics Tools (orchestrator key)
@@ -322,6 +368,46 @@ loopctl enforces that nobody marks their own work as done. The API returns `409`
 
 The implementer's final action is `request_review`. All subsequent steps (report, review, verify) must come from different agents.
 
+## Witness protocol (STH)
+
+Every authenticated request echoes the caller's last-known Signed Tree Head (STH)
+via the `X-Loopctl-Last-Known-STH` header — loopctl's tamper-evident audit chain
+(chain-of-custody v2, §4.4). A brand-new caller has no STH, so its first request
+opts in with `X-Loopctl-STH-Bootstrap: true` to receive the current STH in the
+`x-loopctl-current-sth` response header. **That bootstrap grace is one-time per
+API key** (a deliberate security gate): once consumed, a later request that still
+lacks the header gets `412 witness_bootstrap_already_consumed`.
+
+This MCP server handles that transparently, so you never see the 412:
+
+- **Retry-once contract (412 only).** Any `412 witness_bootstrap_already_consumed`
+  carrying an `x-loopctl-current-sth` header is caught, the STH is cached, and the
+  SAME request is retried exactly **once** with `X-Loopctl-Last-Known-STH`. Bounded
+  to a single retry (never a loop) and anchored to the response's `error.code`. It
+  is safe because the server's witness plug halts *before* the operation runs, so
+  the rejected request had no side effect. The 412 body also carries a
+  machine-readable `error.remediation.retry` contract describing exactly this.
+- **A `409 witness_divergence` is NOT auto-retried** — deliberately. It means the
+  cached STH prefix does not match the server's (the genuine-fork / resync signal,
+  custody-01). The client caches the server's STH from the 409 so the *next*
+  request self-heals, but the current 409 is surfaced rather than papered over.
+- **Cross-process persistence.** The learned STH is cached to a small state file so
+  a **fresh** MCP process (a new Claude session, a script, a CI run) loads it and
+  sends a real header on its first request — avoiding the 412 entirely. The file is
+  keyed by **(server URL + API key)** so distinct keys/tenants on one host never
+  share (or clobber) each other's cache; only a non-secret hash of the key appears
+  in the filename (`loopctl-mcp-sth-<hash>.json` under the OS temp dir), never the
+  key itself. Override the location with `LOOPCTL_STH_STATE_PATH`. Writes are
+  **atomic and symlink-safe** (write to a private `0600` temp file with `O_EXCL`,
+  then rename over the target — never write through a symlink), and loads refuse a
+  symlinked or foreign-owned file. All file I/O degrades gracefully: a missing,
+  corrupt, unwritable, or refused state file falls back to in-memory caching plus
+  the retry-once contract above.
+
+Dispatch-based (v2) clients that mint a fresh ephemeral key per dispatch are
+unaffected: because the cache is keyed per API key, each fresh key gets its own
+clean one-time bootstrap (no cross-key collision).
+
 ## Troubleshooting
 
 ### Connection errors

package/index.js

@@ -10,15 +10,20 @@ import {
   ListToolsRequestSchema,
   CallToolRequestSchema,
 } from "@modelcontextprotocol/sdk/types.js";
-import { readFileSync } from "node:fs";
+import { readFileSync, writeFileSync, renameSync, lstatSync, unlinkSync } from "node:fs";
+import os from "node:os";
 import { fileURLToPath } from "node:url";
-import { dirname, join } from "node:path";
+import path, { dirname, join } from "node:path";
 import {
   projectsPath,
   ingestionJobsPath,
   llmUsagePath,
   parseJsonResponseBody,
 } from "./lib/http-helpers.js";
+import {
+  createWitnessClient,
+  resolveSthStatePath,
+} from "./lib/witness-sth.js";
 
 // Single source of truth for the server version: the package.json this file
 // ships with (npm always includes package.json in the published tarball).
@@ -32,11 +37,51 @@ const SERVER_VERSION = JSON.parse(
 // HTTP helper — witness protocol state
 // ---------------------------------------------------------------------------
 
-// The witness protocol requires clients to echo back the last-known Signed
-// Tree Head (STH) on every authenticated request. On the very first request
-// we send X-Loopctl-STH-Bootstrap: true to receive the current STH without
-// needing one already. After that we cache and send X-Loopctl-Last-Known-STH.
-let lastKnownSTH = null;
+// The witness protocol requires clients to echo back the last-known Signed Tree
+// Head (STH) on every authenticated request. The mechanics live in
+// ./lib/witness-sth.js (shared with the test suite):
+//
+//   * On the very first request from a caller that has never seen an STH we send
+//     `X-Loopctl-STH-Bootstrap: true` to receive the current STH in the
+//     `x-loopctl-current-sth` response header.
+//   * That STH is cached AND persisted to a small per-server state file, so a
+//     FRESH MCP process (new Claude session) loads it and sends a real
+//     `X-Loopctl-Last-Known-STH` on its first request — never tripping the
+//     one-time bootstrap-grace 412 (`witness_bootstrap_already_consumed`, #298).
+//   * If a request still hits that bootstrap-grace 412 (state file missing or
+//     corrupt), the client caches the STH from the 412's `x-loopctl-current-sth`
+//     header and RETRIES the SAME request exactly ONCE — transparently — so the
+//     tool call succeeds. The witness plug halts before the operation runs, so
+//     retrying a witness 412 is side-effect-safe even for POSTs.
+//
+// The state file location defaults to a per-(server + key) file under the OS temp
+// dir and can be overridden with LOOPCTL_STH_STATE_PATH. All file I/O degrades
+// gracefully (missing/corrupt/unwritable/symlinked → in-memory cache + the
+// transparent retry above). The write is atomic + symlink-safe (temp + rename).
+const WITNESS_FS = { readFileSync, writeFileSync, renameSync, lstatSync, unlinkSync };
+
+// One witness client PER API KEY (#298 review HIGH-2): the STH is per-tenant and
+// each key resolves to a tenant server-side, so distinct keys must NOT share an
+// in-memory cache or a state file (a collision causes spurious 409s + false
+// divergence telemetry). The state file is keyed by sha256(serverUrl + ":" + key)
+// — a non-secret hash; the key never hits disk in plaintext.
+const witnessClients = new Map();
+
+function witnessClientFor(apiKey) {
+  let client = witnessClients.get(apiKey);
+  if (!client) {
+    const statePath = resolveSthStatePath({ env: process.env, os, path, apiKey });
+    client = createWitnessClient({
+      statePath,
+      fs: WITNESS_FS,
+      getuid: typeof process.getuid === "function" ? () => process.getuid() : undefined,
+      pid: process.pid,
+      timeoutMs: 30_000,
+    });
+    witnessClients.set(apiKey, client);
+  }
+  return client;
+}
 
 function getBaseUrl() {
   return (process.env.LOOPCTL_SERVER || "https://loopctl.com").replace(/\/$/, "");
@@ -82,26 +127,16 @@ async function apiCall(method, path, body, keyOverride, { exactKey = false } = {
     Accept: "application/json",
   };
 
-  // Witness protocol: send cached STH or request bootstrap
-  if (lastKnownSTH) {
-    headers["X-Loopctl-Last-Known-STH"] = lastKnownSTH;
-  } else {
-    headers["X-Loopctl-STH-Bootstrap"] = "true";
-  }
-
-  const options = {
-    method,
-    headers,
-    signal: AbortSignal.timeout(30_000),
-  };
-
-  if (body !== undefined && body !== null) {
-    options.body = JSON.stringify(body);
-  }
+  const serializedBody =
+    body !== undefined && body !== null ? JSON.stringify(body) : undefined;
 
+  // The witness client injects the STH header, caches + persists any STH the
+  // server returns, and transparently retries a bootstrap-grace 412 ONCE (see
+  // ./lib/witness-sth.js and the module comment above). It returns the final
+  // attempt's Response; we keep body parsing / error shaping here.
   let response;
   try {
-    response = await fetch(url, options);
+    response = await witnessClientFor(key).send({ url, method, headers, serializedBody });
   } catch (err) {
     if (err.name === "TimeoutError") {
       return { error: true, status: 0, body: "Request timed out after 30s" };
@@ -110,12 +145,6 @@ async function apiCall(method, path, body, keyOverride, { exactKey = false } = {
     return { error: true, status: 0, body: `Network error: ${err.message}${cause}` };
   }
 
-  // Witness protocol: cache the STH from response for subsequent requests
-  const sthHeader = response.headers.get("x-loopctl-current-sth");
-  if (sthHeader) {
-    lastKnownSTH = sthHeader;
-  }
-
   if (response.status === 204) {
     return { ok: true };
   }
@@ -204,6 +233,51 @@ function toContentCompact(result) {
   return toContent(result);
 }
 
+/**
+ * Surface a server-emitted BYO-LLM `remediation` PROMINENTLY as a leading text
+ * block. A stranger agent that calls knowledge_ingest / knowledge_search /
+ * knowledge_context BEFORE provisioning its keys then reads the exact next step
+ * in the tool result — call set_llm_config — instead of hunting through raw JSON.
+ *
+ * The remediation lives in one of two places depending on the failure shape:
+ *   - ingest no-key 422  → result.body.error.remediation (code "no_api_key")
+ *   - search/context degrade → result.meta.remediation (fallback_reason
+ *     "no_embedding_key"); the request still returns 200 keyword-only results.
+ *
+ * Returns a notice string, or null when there is no LLM remediation to surface.
+ */
+function llmRemediationNotice(result) {
+  const rem =
+    (result &&
+      result.error === true &&
+      result.body &&
+      result.body.error &&
+      result.body.error.remediation) ||
+    (result && result.meta && result.meta.remediation) ||
+    null;
+
+  if (!rem || rem.action !== "configure_llm") return null;
+
+  const missing = Array.isArray(rem.missing) ? rem.missing.join(", ") : rem.missing;
+  return (
+    `ACTION REQUIRED — BYO LLM key not configured (missing: ${missing}). ` +
+    `${rem.message || ""} Provision it ONCE with the ${rem.mcp_tool} MCP tool, e.g. ` +
+    `${rem.example} (REST: ${rem.api}). Requires your user-role key (LOOPCTL_USER_KEY). ` +
+    `Docs: ${rem.docs}`
+  ).replace(/\s+/g, " ").trim();
+}
+
+/**
+ * toContent, but with any BYO-LLM remediation notice prepended so it is the first
+ * thing the agent reads. Preserves isError from the underlying result.
+ */
+function withRemediationNotice(result) {
+  const base = toContent(result);
+  const notice = llmRemediationNotice(result);
+  if (!notice) return base;
+  return { ...base, content: [{ type: "text", text: notice }, ...base.content] };
+}
+
 // ---------------------------------------------------------------------------
 // Tool implementations
 // ---------------------------------------------------------------------------
@@ -802,7 +876,10 @@ async function knowledgeSearch({ q, project_id, story_id, category, tags, match,
   if (offset != null) params.set("offset", String(offset));
 
   const result = await apiCall("GET", `/api/v1/knowledge/search?${params}`, null, process.env.LOOPCTL_AGENT_KEY);
-  return toContent(result);
+  // If search silently degraded to keyword-only for a missing embedding key, the
+  // server attaches meta.remediation — surface it PROMINENTLY so the agent knows to
+  // call set_llm_config to enable semantic ranking.
+  return withRemediationNotice(result);
 }
 
 async function knowledgeList({
@@ -873,7 +950,8 @@ async function knowledgeContext({
   if (conversation_id) params.set("conversation_id", conversation_id);
 
   const result = await apiCall("GET", `/api/v1/knowledge/context?${params}`, null, process.env.LOOPCTL_AGENT_KEY);
-  return toContent(result);
+  // Same as knowledge_search: surface a missing-embedding-key remediation prominently.
+  return withRemediationNotice(result);
 }
 
 async function knowledgeCreate({
@@ -1129,7 +1207,9 @@ async function knowledgeIngest({ url, content, source_type, project_id, publish
   if (project_id) body.project_id = project_id;
   if (publish) body.publish = true;
   const result = await apiCall("POST", "/api/v1/knowledge/ingest", body, process.env.LOOPCTL_ORCH_KEY);
-  return toContent(result);
+  // A keyless tenant gets a 422 (code no_api_key) carrying a remediation — surface it
+  // prominently so a first-time agent knows to call set_llm_config before ingesting.
+  return withRemediationNotice(result);
 }
 
 async function knowledgeIngestBatch({ items, project_id, publish }) {
@@ -1152,7 +1232,9 @@ async function knowledgeIngestBatch({ items, project_id, publish }) {
     { items: resolvedItems },
     process.env.LOOPCTL_ORCH_KEY
   );
-  return toContent(result);
+  // Batch ingest gates on the Anthropic key up front too — a keyless tenant gets a
+  // 422 with the same remediation; surface it prominently.
+  return withRemediationNotice(result);
 }
 
 async function knowledgeIngestionJobs(args = {}) {
@@ -2589,7 +2671,9 @@ const TOOLS = [
       "If semantic ranking is unavailable the search transparently degrades to keyword-only " +
       "(meta.fallback: true, meta.search_mode: 'keyword_only') and now reports meta.fallback_reason " +
       "— a stable tag naming WHY (e.g. no_embedding_key, embedding_circuit_open, " +
-      "embedding_provider_error_<status>, embedding_timeout).",
+      "embedding_provider_error_<status>, embedding_timeout). When the reason is a MISSING " +
+      "embedding key (no_embedding_key), the result leads with an ACTION REQUIRED notice + " +
+      "meta.remediation telling you to provision it with set_llm_config (BYO — do it once).",
     inputSchema: {
       type: "object",
       properties: {
@@ -3267,7 +3351,10 @@ const TOOLS = [
       "Enqueues an Oban job that fetches the content (if URL), extracts knowledge articles via LLM, " +
       "and inserts them. Extracted articles are DRAFTS by default (lower-trust LLM output, staged " +
       "for review) — unlike knowledge_create which publishes by default. Pass publish:true to " +
-      "publish them on extraction. Requires orchestrator role.",
+      "publish them on extraction. Requires orchestrator role. BYO: extraction runs on the " +
+      "tenant's OWN Anthropic key — a keyless tenant gets a 422 (code no_api_key) and the " +
+      "result leads with an ACTION REQUIRED notice telling you to provision it once via " +
+      "set_llm_config.",
     inputSchema: {
       type: "object",
       properties: {
@@ -3380,29 +3467,44 @@ const TOOLS = [
   {
     name: "llm_config",
     description:
-      "Get the tenant's BYO Anthropic + embedding configuration: the per-operation " +
-      "model choices, whether each key is configured (has_api_key / has_embedding_key), " +
-      "and masked last-4 hints. NEVER returns a key itself. Requires user role " +
-      "(LOOPCTL_USER_KEY).",
+      "CHECK your BYO LLM onboarding status. Returns this tenant's per-operation model " +
+      "choices and whether each key is configured — `has_api_key` (Anthropic, powers " +
+      "ingest) and `has_embedding_key` (OpenAI embedding, powers semantic search) — plus " +
+      "masked last-4 hints. NEVER returns a key itself. Call this BEFORE ingest/search to " +
+      "confirm setup, or after set_llm_config to verify it took. Uses your user-role key " +
+      "(LOOPCTL_USER_KEY, obtained at signup); managing tenant secrets is user-only, so an " +
+      "agent/orchestrator key is rejected.",
     inputSchema: { type: "object", properties: {}, required: [] },
   },
   {
     name: "set_llm_config",
     description:
-      "Set/rotate the tenant's OWN Anthropic API key AND OpenAI embedding key (both " +
-      "stored encrypted, never returned) and the per-operation models " +
-      "(extraction/classification/merge/embedding). loopctl fronts no cost — the " +
-      "tenant's keys bill the tenant. Mandatory BYO for embeddings: without an " +
-      "embedding_api_key the tenant's articles are not vector-searchable. Any subset " +
-      "of fields may be sent; omitting a key leaves the existing key untouched. Model " +
-      "ids are free-form (any model the key permits). Requires user role " +
-      "(LOOPCTL_USER_KEY).",
+      "FIRST-TIME SETUP — do this ONCE, right after signup, to bring the wiki online. " +
+      "Provision your OWN Anthropic + OpenAI embedding keys so knowledge ingest AND " +
+      "semantic search work. loopctl is strictly BYO: it fronts NO LLM cost — your keys " +
+      "bill you directly and are stored ENCRYPTED, never returned. WHY it's required: with " +
+      "no `api_key`, knowledge_ingest returns 422 (code no_api_key); with no " +
+      "`embedding_api_key`, knowledge_search silently degrades to keyword-only " +
+      "(meta.fallback_reason: no_embedding_key) — both responses carry a `remediation` " +
+      "pointing back to THIS tool. WHICH key powers WHAT: `api_key` (Anthropic) → ingest " +
+      "extraction / classification / merge synthesis; `embedding_api_key` " +
+      "(OpenAI-compatible) → article embeddings + semantic ranking. All params are optional " +
+      "and partial-merge — omitting a key leaves the existing one untouched, so you can set " +
+      "or rotate one at a time. The per-operation model overrides " +
+      "(extraction_model / classification_model / merge_model / embedding_model) are " +
+      "free-form (any model the key permits) and default server-side when omitted. Typical " +
+      'onboarding call: set_llm_config({api_key: "sk-ant-...", embedding_api_key: "sk-..."}). ' +
+      "Verify anytime with llm_config (has_api_key / has_embedding_key). REQUIRES your " +
+      "user-role key LOOPCTL_USER_KEY (minted by the human-anchored signup ceremony); if it " +
+      "is unset the tool fails fast telling you to set it.",
     inputSchema: {
       type: "object",
       properties: {
         api_key: {
           type: "string",
-          description: "Anthropic API key (write-only; stored encrypted, never returned).",
+          description:
+            "Anthropic API key — powers knowledge ingest (extraction/classification/merge). " +
+            "Write-only; stored encrypted, never returned.",
         },
         extraction_model: {
           type: "string",
@@ -3419,7 +3521,9 @@ const TOOLS = [
         embedding_api_key: {
           type: "string",
           description:
-            "OpenAI-compatible embedding API key (write-only; stored encrypted, never returned).",
+            "OpenAI-compatible embedding API key — powers article embeddings + semantic " +
+            "search. Write-only; stored encrypted, never returned. Without it, articles are " +
+            "not vector-searchable and search degrades to keyword-only.",
         },
         embedding_model: {
           type: "string",

package/lib/witness-sth.js

@@ -0,0 +1,296 @@
+// Witness-protocol (Signed Tree Head) client state — shared by index.js and the
+// test suite so both exercise the SAME retry + persistence logic.
+//
+// Background (loopctl chain-of-custody v2, §4.4): every authenticated request
+// must echo the caller's last-known STH via `X-Loopctl-Last-Known-STH`. A brand
+// new caller has no STH, so its FIRST request opts in with
+// `X-Loopctl-STH-Bootstrap: true` to receive the current STH in the response's
+// `x-loopctl-current-sth` header. That bootstrap grace is ONE-TIME PER API KEY
+// (custody-03 / GHSA-36g5-mcrh-rcrm): once consumed, a later request that still
+// lacks the header gets `412 witness_bootstrap_already_consumed`.
+//
+// The MCP server is a short-lived process: a fresh Claude session spawns a fresh
+// process that historically started with no STH, so — once the key's one-time
+// grace was consumed by an earlier process — its very first tool call hit that
+// 412 (#298). This module fixes that on the CLIENT side, two ways:
+//
+//   1. PERSISTENCE — the learned STH is cached to a small state file keyed by
+//      (server URL + API key), so a fresh process loads it and sends a real
+//      header on its first request, never tripping the bootstrap 412 at all.
+//   2. TRANSPARENT RETRY — if a request still hits a witness-plug 412 that
+//      carries the current STH (e.g. the state file was missing/corrupt), the
+//      client caches that STH and retries the SAME request ONCE, so the tool
+//      call succeeds instead of surfacing the 412.
+//
+// SAFETY: the witness plug runs early and HALTS the request before the operation
+// executes, so a witness 412/409 means the original request had no side effect —
+// retrying it once is safe even for non-idempotent POSTs. The retry is bounded to
+// a single attempt (never a loop). None of this weakens the server-side one-time
+// grace: it only teaches legit clients to carry/relearn the STH they are entitled
+// to. See mcp-server/README.md → "Witness protocol (STH)".
+//
+// 409 IS NOT AUTO-RETRIED (deliberate): a `409 witness_divergence` means the
+// client's cached STH prefix does not match the server's at that position — the
+// genuine-fork / resync signal (custody-01). The client caches the server's STH
+// from the 409's `x-loopctl-current-sth` header so the NEXT request self-heals,
+// but the current request's 409 is surfaced (not silently retried) so a real
+// divergence is never papered over. Only the bootstrap-grace 412 is retried.
+
+import { createHash, randomBytes } from "node:crypto";
+
+// Explicit override for the STH state-file location (absolute path). When unset,
+// the file is derived under the OS temp dir, keyed by (server URL + API key).
+export const STH_STATE_PATH_ENV = "LOOPCTL_STH_STATE_PATH";
+
+// The `x-loopctl-current-sth` header the client caches / retries with. Shape:
+// `<position>:<22-char base64url signature prefix>`. Validating the shape guards
+// against a corrupt state file feeding a malformed header (which the server would
+// itself reject 412 witness_header_malformed).
+const STH_SHAPE = /^\d+:[A-Za-z0-9_-]{22}$/;
+
+// The error code of the ONE 412 that is safe to transparently retry.
+const BOOTSTRAP_CONSUMED_CODE = "witness_bootstrap_already_consumed";
+
+/**
+ * True when `sth` is a well-formed `<position>:<22-char base64url prefix>` value.
+ * @param {unknown} sth
+ * @returns {boolean}
+ */
+export function isValidSth(sth) {
+  return typeof sth === "string" && STH_SHAPE.test(sth);
+}
+
+/**
+ * Resolve the STH state-file path. Honors an explicit `LOOPCTL_STH_STATE_PATH`
+ * override; otherwise derives a file under the OS temp dir keyed by BOTH the
+ * server URL AND the API key, so two keys/tenants on the same host never share
+ * (and clobber / spuriously invalidate) each other's cached STH. Only a NON-SECRET
+ * sha256 hash of the key goes in the filename — the key itself never hits disk.
+ *
+ * @param {{ env: Record<string,string|undefined>, os: { tmpdir(): string }, path: { join(...p: string[]): string }, apiKey?: string }} deps
+ * @returns {string}
+ */
+export function resolveSthStatePath({ env, os, path, apiKey = "" }) {
+  const override = env[STH_STATE_PATH_ENV];
+  if (typeof override === "string" && override.trim() !== "") return override;
+
+  const baseUrl = (env.LOOPCTL_SERVER || "https://loopctl.com").replace(/\/$/, "");
+  // Scope by server URL AND api key (#298 review HIGH-2): distinct keys must not
+  // collide on one cache file. The key is hashed (never stored in plaintext).
+  const digest = createHash("sha256").update(`${baseUrl}:${apiKey}`).digest("hex").slice(0, 16);
+  return path.join(os.tmpdir(), `loopctl-mcp-sth-${digest}.json`);
+}
+
+/**
+ * Load a persisted STH string from `filePath`, or `null`. NEVER throws — a
+ * missing, unreadable, non-JSON, or shape-invalid file degrades to `null` so the
+ * caller falls back to the bootstrap+retry path.
+ *
+ * SYMLINK / OWNERSHIP GUARD (CWE-59, #298 review HIGH-1): on a shared `/tmp` the
+ * cache path is predictable, so a co-located user could pre-plant a file/symlink
+ * there. Before reading we `lstat` the path and refuse (→ null, in-memory only) if
+ * it is a symlink or is not owned by the current uid.
+ *
+ * @param {string} filePath
+ * @param {{ fs: { readFileSync: Function, lstatSync?: Function }, getuid?: () => number }} deps
+ * @returns {string|null}
+ */
+export function loadPersistedSth(filePath, { fs, getuid }) {
+  try {
+    if (typeof fs.lstatSync === "function") {
+      const st = fs.lstatSync(filePath);
+      if (st.isSymbolicLink()) return null;
+      if (typeof getuid === "function" && st.uid !== getuid()) return null;
+    }
+    const parsed = JSON.parse(fs.readFileSync(filePath, "utf8"));
+    const sth = parsed && typeof parsed.sth === "string" ? parsed.sth : null;
+    return isValidSth(sth) ? sth : null;
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Persist an STH string to `filePath`. NEVER throws — a write failure (read-only
+ * temp dir, full disk, symlink refusal) degrades to in-memory-only caching.
+ * Returns whether the write succeeded.
+ *
+ * ATOMIC + SYMLINK-SAFE (CWE-59, #298 review HIGH-1): writes to a fresh
+ * per-process temp file opened with `wx` (O_CREAT|O_EXCL — refuses a pre-planted
+ * symlink at the temp path) and mode 0o600, then `rename`s it over `filePath`.
+ * `rename` replaces the destination path ENTRY atomically and never writes THROUGH
+ * a symlink at `filePath`, so an attacker's symlink there is replaced, not
+ * followed — no arbitrary-file clobber. The rename-over also fixes the torn-file
+ * race from a killed mid-write.
+ *
+ * @param {string} filePath
+ * @param {string} sth
+ * @param {{ fs: { writeFileSync: Function, renameSync: Function, unlinkSync?: Function }, pid?: number }} deps
+ * @returns {boolean}
+ */
+export function persistSth(filePath, sth, { fs, pid = process.pid }) {
+  if (!isValidSth(sth)) return false;
+  const tmp = `${filePath}.${pid}.${randomBytes(6).toString("hex")}.tmp`;
+  try {
+    fs.writeFileSync(
+      tmp,
+      JSON.stringify({ sth, updated_at: new Date().toISOString() }),
+      { encoding: "utf8", flag: "wx", mode: 0o600 },
+    );
+    fs.renameSync(tmp, filePath);
+    return true;
+  } catch {
+    // Best-effort cleanup of the temp file; ignore any failure.
+    try {
+      if (typeof fs.unlinkSync === "function") fs.unlinkSync(tmp);
+    } catch {
+      /* ignore */
+    }
+    return false;
+  }
+}
+
+/**
+ * Decide whether a response should trigger a one-shot transparent retry, and with
+ * which STH. Returns the STH to retry with, or `null`.
+ *
+ * Gated on `412` AND a shape-valid `x-loopctl-current-sth` header — which the
+ * witness plug sets ONLY on the `witness_bootstrap_already_consumed` branch (not
+ * on the malformed/missing 412s, where a retry couldn't help). This precisely
+ * targets the fresh-process bootstrap-grace 412 (#298). The caller additionally
+ * confirms the body's `error.code` (see `createWitnessClient`).
+ *
+ * @param {number} status
+ * @param {string|null|undefined} currentSthHeader
+ * @returns {string|null}
+ */
+export function bootstrapRetrySth(status, currentSthHeader) {
+  return status === 412 && isValidSth(currentSthHeader) ? currentSthHeader : null;
+}
+
+/**
+ * Create a witness-aware request client that owns the shared STH state (loaded
+ * from `statePath` at construction), injects the witness header on every attempt,
+ * caches + persists any STH the server returns, coalesces concurrent cold-start
+ * bootstraps, and transparently retries a bootstrap-grace 412 ONCE.
+ *
+ * `send` returns the raw `Response` of the FINAL attempt so the caller keeps full
+ * control of body parsing / error shaping. Network errors from `fetchImpl`
+ * propagate (the caller's try/catch handles them); a network failure is not
+ * retried.
+ *
+ * @param {{
+ *   fetchImpl?: typeof fetch,
+ *   statePath?: string|null,
+ *   fs?: object,
+ *   getuid?: () => number,
+ *   pid?: number,
+ *   timeoutMs?: number,
+ * }} [deps]
+ */
+export function createWitnessClient({
+  fetchImpl = fetch,
+  statePath = null,
+  fs,
+  getuid,
+  pid,
+  timeoutMs = 30_000,
+} = {}) {
+  // Load any persisted STH so a FRESH process sends a real header on request #1
+  // and skips the bootstrap-grace 412 entirely.
+  let lastKnownSTH =
+    statePath && fs ? loadPersistedSth(statePath, { fs, getuid }) : null;
+
+  // Cold-start singleflight (#298 review MEDIUM-6): when several tool calls fire
+  // at process start they all observe lastKnownSTH === null; without coalescing
+  // each would send its own bootstrap. The FIRST becomes the leader; the rest
+  // await its result and then send with the learned header.
+  let bootstrapInFlight = null;
+
+  function remember(sth) {
+    if (!isValidSth(sth) || sth === lastKnownSTH) return;
+    lastKnownSTH = sth;
+    if (statePath && fs) persistSth(statePath, sth, { fs, pid });
+  }
+
+  async function attempt({ url, method, headers, serializedBody }) {
+    // Witness protocol: echo the cached STH, or (never seen one) request a
+    // one-time bootstrap. A fresh AbortSignal per attempt so the retry gets its
+    // own timeout budget.
+    const withWitness = { ...headers };
+    if (lastKnownSTH) withWitness["X-Loopctl-Last-Known-STH"] = lastKnownSTH;
+    else withWitness["X-Loopctl-STH-Bootstrap"] = "true";
+
+    const options = {
+      method,
+      headers: withWitness,
+      signal: AbortSignal.timeout(timeoutMs),
+    };
+    if (serializedBody !== undefined) options.body = serializedBody;
+
+    return fetchImpl(url, options);
+  }
+
+  // Confirm a bootstrap-grace 412 against the DOCUMENTED contract before retrying
+  // (#298 review LOW-8): status 412 + a valid STH header, AND — when the body is
+  // readable — error.code === "witness_bootstrap_already_consumed". A body-read
+  // failure falls back to the status+header decision so a transient parse issue
+  // never defeats the retry. Reads a CLONE so the caller can still consume the
+  // body of a response we end up NOT retrying.
+  async function retrySthFor(response) {
+    const sth = response.headers.get("x-loopctl-current-sth");
+    if (!bootstrapRetrySth(response.status, sth)) return null;
+    try {
+      const body = await response.clone().json();
+      const code = body && body.error && body.error.code;
+      if (code && code !== BOOTSTRAP_CONSUMED_CODE) return null;
+    } catch {
+      /* unreadable body → fall back to status+header */
+    }
+    return sth;
+  }
+
+  async function performSend(req) {
+    let response = await attempt(req);
+    remember(response.headers.get("x-loopctl-current-sth"));
+
+    // Bounded, single transparent retry for the bootstrap-grace 412. `remember`
+    // above already cached the STH, so this second attempt sends a real header.
+    if (await retrySthFor(response)) {
+      response = await attempt(req);
+      remember(response.headers.get("x-loopctl-current-sth"));
+    }
+
+    return response;
+  }
+
+  async function send(req) {
+    // Cold-start coalescing: only the leader bootstraps; followers await it and
+    // then send with the learned STH. On the (rare) leader failure, followers
+    // fall through and each bootstrap — acceptable degradation.
+    if (lastKnownSTH === null && bootstrapInFlight) {
+      await bootstrapInFlight;
+      return performSend(req);
+    }
+
+    if (lastKnownSTH === null) {
+      let done;
+      bootstrapInFlight = new Promise((resolve) => {
+        done = resolve;
+      });
+      try {
+        return await performSend(req);
+      } finally {
+        done();
+        bootstrapInFlight = null;
+      }
+    }
+
+    return performSend(req);
+  }
+
+  return {
+    send,
+    getSTH: () => lastKnownSTH,
+  };
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "loopctl-mcp-server",
-  "version": "2.33.0",
+  "version": "2.34.0",
   "description": "MCP server for loopctl — structural trust for AI development loops",
   "type": "module",
   "main": "index.js",