loopctl-mcp-server 2.87.0 → 2.88.0

package/README.md

@@ -384,7 +384,8 @@ it is enforced server-side and a no-op for a non-superadmin key — see below.)
 | `knowledge_agent_usage` | Per-agent knowledge usage: total reads, unique articles, top read articles. Required: exactly one of `api_key_id` (credential) or `agent_id` (logical identity). Optional: `limit`, `since_days`. See Wiki Attribution section. |
 | `knowledge_unused_articles` | Published articles with zero accesses in the window. Optional: `days_unused` (default 30), `limit` (default 50, max 200). |
 | `knowledge_curation_log` | Concise human-readable log of KB CURATION adjustments — novelty-gate decisions (`gate_duplicate`/`gate_draft`/`gate_skip`) and conflict resolutions (`supersede`/`merge`/`dismiss`) — for analyzing the agents'-KB rollout, distinct from the verbose audit log. Each entry: `{at, kind, summary, refs, actor, confidence}`. **RECORDED ONLY when `settings.kb_curation_log` is on** (PATCH `/api/v1/admin/tenants/:id` with `settings:{kb_curation_log:true}`); off by default = no rows. Most recent first. Requires orchestrator role. Optional: `kind`, `since` (ISO8601), `limit` (default 50, max 500), `offset`. |
-| `knowledge_retrieval_metrics` | Daily retrieval-PRECISION time series: for each day, the share of RECORDED surfaced search RESULTS the agent then opened (search → get/context within a window). A proxy for whether retrieval is improving as the corpus is de-duplicated, better navigated (MOCs), and conflict-resolved. **Denominators (#582):** `precision` = `followed_through`/`searched`, where `searched` counts RECORDED surfaced RESULTS — one row per result put in front of the agent, capped at the first 20 per call (cap enforced by `Loopctl.Knowledge.Analytics.max_recorded_search_results/0`) — not search calls (`results_recorded` is the same number named for its unit). That cap makes `precision` precision@20: a call returning more contributes only 20, and an open of a result ranked beyond the cap is in neither term. The per-CALL rate is the separate `search_follow_through` = `searches_with_follow_through`/`searches` (distinct QUERY-BEARING calls — query-less `list`/`list_keyset` enumeration pages are excluded). `results_returned` is the true un-truncated count for those same calls. The four call-level fields are filtered per ROW, not per day, so a day mixing pre-#582 or enumeration rows with real searches reports a PARTIAL figure rather than 0 — and `results_returned` is NOT comparable to `searched`. Zero-result and keyless searches are unrecordable, so every ratio here is an upper bound. **`precision` ALONE rises when a search returns FEWER results** — its denominator counts surfaced RESULTS, while the two call-level rates divide CALL counts a narrower page does not shrink — so never optimise `precision` without the absolute `followed_through`. Both follow-through rates are biased DOWN by the recording cap (opens beyond rank 20 are invisible) and UP by crediting every search in the window that surfaced the opened article. **WHICH FOLLOW-THROUGH RATE TO QUOTE:** two are published over DIFFERENT populations, and picking the wrong one misstates agent behaviour by roughly 3.4x. `search_follow_through` is over every query-bearing call that survives the infrastructure exclusion — which still INCLUDES the recall hook and the session-start auto-query, channels that cannot follow through by construction — so read it as BLENDED and use it for total traffic. `scored_follow_through` = `searches_scored_with_follow_through`/`searches_scored` is over the calls carrying a session identity AND coming from a channel that can react to a result, and **IT is the rate to quote when asking whether AGENTS are consuming the KB**. It is `null` when nothing was scoreable, never `0.0` — zero would assert agents searched and opened nothing when the truth is the instrument could not see. Measured live for 2026-08-19..29: 10.8% blended against 38.0% scored, because the recall hook alone was 1,234 of that window's 1,708 calls. Most recent day first. Requires orchestrator role. Optional: `limit` (default 30, max 365), `offset`. |
+| `knowledge_retrieval_metrics` | Daily retrieval-PRECISION time series: for each day, the share of RECORDED surfaced search RESULTS the agent then opened (search → get/context within a window). A proxy for whether retrieval is improving as the corpus is de-duplicated, better navigated (MOCs), and conflict-resolved. **Denominators (#582):** `precision` = `followed_through`/`searched`, where `searched` counts RECORDED surfaced RESULTS — one row per result put in front of the agent, capped at the first 20 per call (cap enforced by `Loopctl.Knowledge.Analytics.max_recorded_search_results/0`) — not search calls (`results_recorded` is the same number named for its unit). That cap makes `precision` precision@20: a call returning more contributes only 20, and an open of a result ranked beyond the cap is in neither term. The per-CALL rate is the separate `search_follow_through` = `searches_with_follow_through`/`searches` (distinct QUERY-BEARING calls — query-less `list`/`list_keyset` enumeration pages are excluded). `results_returned` is the true un-truncated count for those same calls. The four call-level fields are filtered per ROW, not per day, so a day mixing pre-#582 or enumeration rows with real searches reports a PARTIAL figure rather than 0 — and `results_returned` is NOT comparable to `searched`. Zero-result and keyless searches are unrecordable, so every ratio here is an upper bound. **`precision` ALONE rises when a search returns FEWER results** — its denominator counts surfaced RESULTS, while the two call-level rates divide CALL counts a narrower page does not shrink — so never optimise `precision` without the absolute `followed_through`. Both follow-through rates are biased DOWN by the recording cap (opens beyond rank 20 are invisible) and UP by crediting every search in the window that surfaced the opened article. **WHICH FOLLOW-THROUGH RATE TO QUOTE:** two are published over DIFFERENT populations, and picking the wrong one misstates agent behaviour by roughly 3.4x. `search_follow_through` is over every query-bearing call that survives the infrastructure exclusion — which still INCLUDES the recall hook and the session-start auto-query, channels that cannot follow through by construction — so read it as BLENDED and use it for total traffic. `scored_follow_through` = `searches_scored_with_follow_through`/`searches_scored` is over the calls carrying a session identity AND coming from a channel that can react to a result, and **IT is the rate to quote when asking whether AGENTS are consuming the KB**. It is `null` when nothing was scoreable, never `0.0` — zero would assert agents searched and opened nothing when the truth is the instrument could not see. Measured live for 2026-08-19..29: 10.8% blended against 38.0% scored, because the recall hook alone was 1,234 of that window's 1,708 calls. Most recent day first. Requires orchestrator role. Optional: `limit` (default 30, max 365), `offset`. Every ratio here divides `search_events` / `article_access_events` columns, so a NULL column shrinks a denominator instead of reporting itself — check `knowledge_search_coverage` before reading a figure here as agent behaviour rather than instrumentation. |
+| `knowledge_search_coverage` | Which DECLARED columns of `search_events` are actually being filled, per search surface, over a bounded window. `search_events` shipped correct and nearly blind — 2 of its first 133 rows carried any `client_*` context, discoverable only by an audit nobody was scheduled to run — so a coverage PROFILE per `tool` declares the columns a correctly-instrumented caller should supply, and a surface emitting NOTHING reads as `rows: 0` (which no audit over existing rows can produce). **It cannot prove a PRESENT column is a CORRECT one:** one MCP process serves a session and every agent it dispatches with an environment frozen at spawn, so it labels every search `main` — such a row is 100% covered here and still wrong about the only thing `client_kind` exists to say. **Populations, not one denominator:** each column names the rows that COULD have carried it — `all`, `ran` (excludes `outcome=rejected`, which has no `mode_used`/`duration_ms` by construction), `agent` (rows carrying `client_kind` or `client_session_id`, i.e. rows that really came through the MCP client; the recall hook and smoke tests call the API directly and can never supply `client_*`). `share_missing` is `null`, never `0.0`, on an empty population. **`client_context` is the one to read FIRST:** the `agent` denominator is built from two of the columns it scores, so a client that sends NOTHING empties it and every `client_*` line reads a clean `0/0` — that figure is the same count over `all`, and it is where a fleet gone blind reports itself (a high share on `memory_recall` is the recall hook and expected). **Required vs enrichable:** `required` is fillable at record time so a miss is a defect; `enrichable` (`client_model`, `client_effort`, `agent_id`) is what no client can send and `mix loopctl.enrich_search_events` fills offline on a schedule — read a recent enrichable share as a floor, since a window ending near now measures the enrichment's lag. **Unprofiled:** every `tool` with rows and no profile is listed (`null` included), and `rows_total` covers the whole window, so a new surface cannot be silently dropped. Requires orchestrator role. Optional: `days` (default 30, max 366), `to` (ISO8601 date or datetime, exclusive upper bound; a bare date is read at 00:00:00Z, and an unparseable value is a 400 rather than a silent fallback to now). |
 
 ### Egress / Privacy Tools (US-41.4)
 

package/index.js

@@ -2504,6 +2504,18 @@ async function knowledgeRetrievalMetrics({ limit, offset } = {}) {
   return toContent(result);
 }
 
+async function knowledgeSearchCoverage({ days, to } = {}) {
+  const params = new URLSearchParams();
+  if (days != null) params.set("days", String(days));
+  if (to) params.set("to", to);
+  const qs = params.toString();
+  const path = qs
+    ? `/api/v1/knowledge/analytics/search-coverage?${qs}`
+    : "/api/v1/knowledge/analytics/search-coverage";
+  const result = await apiCall("GET", path, null, process.env.LOOPCTL_ORCH_KEY);
+  return toContent(result);
+}
+
 async function knowledgeCurationLog({ kind, since, limit, offset } = {}) {
   const params = new URLSearchParams();
   if (kind) params.set("kind", kind);
@@ -6935,6 +6947,11 @@ const TOOLS = [
       "within a window). A proxy for whether retrieval is improving — watch it trend up as " +
       "the corpus is de-duplicated, better navigated (MOCs), and conflict-resolved. Most " +
       "recent day first. Requires orchestrator role.\n\n" +
+      "Every ratio here divides columns of search_events / article_access_events, so a " +
+      "NULL column silently shrinks a denominator rather than reporting itself. " +
+      "knowledge_search_coverage is the companion that says which of those columns are " +
+      "actually being filled, per surface — check it before treating a figure here as " +
+      "a measurement of agent behaviour rather than of instrumentation.\n\n" +
       // The cap is enforced by Loopctl.Knowledge.Analytics.max_recorded_search_results/0
       // (Elixir); this JS string cannot interpolate it, so change both together.
       "Denominators (#582): precision = followed_through / searched, and `searched` counts " +
@@ -7033,6 +7050,69 @@ const TOOLS = [
       required: [],
     },
   },
+  {
+    name: "knowledge_search_coverage",
+    description:
+      "Report which DECLARED columns of search_events are actually being filled, per search " +
+      "surface, over a bounded window. Requires orchestrator role.\n\n" +
+      "WHY IT EXISTS: search_events shipped correct and nearly blind — 2 of its first 133 " +
+      "rows carried any client_* context, and that was discoverable only by an audit nobody " +
+      "was scheduled to run. A declared coverage PROFILE per tool names the columns a " +
+      "correctly-instrumented caller is expected to supply, so a surface emitting NOTHING " +
+      "shows up as rows: 0, which no audit over existing rows can produce. Prior art: " +
+      "MemoRizz v0.8.0 declares the evidence stages a task type must emit and reports the " +
+      "missing stage instead of leaving it to an audit.\n\n" +
+      "WHAT IT CANNOT PROVE: that a PRESENT column is a CORRECT one. client_kind is the " +
+      "worked example — one MCP process serves a session and every agent it dispatches with " +
+      "an environment frozen at spawn, so it labels every search 'main'. Such a row is 100% " +
+      "covered here and still wrong about the only thing that column exists to say; the " +
+      "offline enrichment is what refines it. It also cannot see a search path that records " +
+      "NO row at all.\n\n" +
+      "POPULATIONS, NOT ONE DENOMINATOR: each column names the rows that COULD have carried " +
+      "it, reported as scope/population beside every count. 'all' is every row; 'ran' " +
+      "excludes outcome=rejected (a rejected call never ran, so it has no mode_used and no " +
+      "duration_ms by construction); 'agent' is rows carrying client_kind or " +
+      "client_session_id, i.e. rows that really came through the MCP client — the recall " +
+      "hook and smoke tests call the API directly and can never supply client_*, so scoring " +
+      "them would measure loopctl's own automation. share_missing is null, never 0.0, on an " +
+      "empty population.\n\n" +
+      "CLIENT_CONTEXT: the 'agent' denominator is built from two of the columns it scores, " +
+      "so a client that sends NOTHING empties it and every client_* line then reads a clean " +
+      "0/0. Each profile therefore also carries client_context, scored over 'all', whose " +
+      "missing is the rows that carried NO client context at all — that is where a fleet gone " +
+      "blind reports itself. A high share on memory_recall is the recall hook and expected; a " +
+      "high share on knowledge_search is not.\n\n" +
+      "REQUIRED vs ENRICHABLE: required is what a client or the server can fill at record " +
+      "time, so a miss is a defect. enrichable (client_model, client_effort, agent_id) is " +
+      "what no client can send — the first two do not exist in the MCP server's spawn " +
+      "environment and are filled offline by mix loopctl.enrich_search_events, and agent_id " +
+      "is server-derived from a key that may own no agent. That enrichment runs on a " +
+      "schedule, so a window ending near now measures its LAG: read a recent enrichable " +
+      "share as a floor.\n\n" +
+      "UNPROFILED: every tool value with rows and no declared profile is listed with its row " +
+      "count, null included. rows_total counts the whole window, so rows_total minus the sum " +
+      "of profile rows is exactly the unprofiled traffic — a new surface cannot be silently " +
+      "dropped from the accounting.",
+    inputSchema: {
+      type: "object",
+      properties: {
+        days: {
+          type: "integer",
+          description: "Window length in days back from `to` (default 30, max 366). Clamped, never rejected.",
+          minimum: 1,
+          maximum: 366,
+        },
+        to: {
+          type: "string",
+          description:
+            "ISO8601 date or datetime, exclusive upper bound. Default now. A bare date is " +
+            "read at 00:00:00Z. The window is [from, to). Rejected with 400 when it cannot " +
+            "be parsed — never silently replaced with now.",
+        },
+      },
+      required: [],
+    },
+  },
   {
     name: "knowledge_analytics_top",
     description:
@@ -8225,6 +8305,9 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
     case "knowledge_retrieval_metrics":
       return await knowledgeRetrievalMetrics(args);
 
+    case "knowledge_search_coverage":
+      return await knowledgeSearchCoverage(args);
+
     case "knowledge_analytics_top":
       return await knowledgeAnalyticsTop(args);
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "loopctl-mcp-server",
-  "version": "2.87.0",
+  "version": "2.88.0",
   "description": "MCP server for loopctl — structural trust for AI development loops",
   "type": "module",
   "main": "index.js",