@skyphusion/search-mcp 0.3.0 → 0.5.0

package/README.md

@@ -6,7 +6,8 @@
 
 Open-source toolkit for [Cloudflare AI Search](https://developers.cloudflare.com/ai-search/):
 
-1. **MCP Worker** (`src/mcp.ts`) -- bearer-gated Streamable-HTTP MCP with a `search` tool for agents.
+1. **MCP Worker** (`src/mcp.ts`) -- bearer-gated Streamable-HTTP MCP for agents:
+   `search`, `list_repos`, `get_file`, `ask`, `corpus_status`, plus `corpus://` resources.
 2. **Query Worker** (`src/index.ts`) -- CORS + Turnstile + rate-limited `POST /ask` that streams answers for a browser widget.
 3. **Corpus sync** (`scripts/sync.mjs`, `scripts/sync-runner.mjs`) -- git-tracked sources to R2, with extension remapping so TypeScript, Dockerfiles, and other text AI Search would otherwise skip get indexed.
 
@@ -48,7 +49,7 @@ cp wrangler.mcp.toml.example wrangler.mcp.toml
 cp scripts/targets.json.example scripts/targets.json
 # edit the three files for your account, instance, bucket, and repos
 
-npm run typecheck
+npm run typecheck   # src + index.test.ts (Workers types) AND scripts/**/*.test.ts (Node types)
 npm test
 ```
 
@@ -56,25 +57,34 @@ Provision R2 + AI Search, sync your corpus, deploy both Workers. Step-by-step: [
 
 ## Skyphusion production
 
-This repo is the production home for Skyphusion AI Search (`search.vivijure.com`,
-`search-internal.vivijure.com`). Config is materialized from GitHub Actions secrets at deploy/sync
-time; do not commit `wrangler.toml`, `wrangler.mcp.toml`, or `scripts/targets.json`.
+This repo is the production home for three AI Search surfaces:
 
-- [Operator runbook](docs/skyphusion/OPERATOR.md) -- secrets, topology, bootstrap
-- [Cutover record (2026-07-08)](docs/skyphusion/CUTOVER.md) -- migration history, failure log, archive steps
+| Surface | Host | Worker config |
+| --- | --- | --- |
+| Public docs / marketing | `search.vivijure.com` | secret `SKYPHUSION_WRANGLER_TOML` → `wrangler.toml` at CI |
+| Internal MCP | `search-internal.vivijure.com` | secret `SKYPHUSION_WRANGLER_MCP_TOML` |
+| Rockenhaus court records | `search.rockenhaus.net` | committed `wrangler.rockenhaus.toml` |
+
+Do not commit `wrangler.toml`, `wrangler.mcp.toml`, or `scripts/targets.json` (gitignored).
+Rockenhaus is the exception: its wrangler file is public and tracked.
+
+- [Operator runbook](docs/skyphusion/OPERATOR.md) -- secrets, topology shape, escrow, reindex
+- [Cutover record (2026-07-08)](docs/skyphusion/CUTOVER.md) -- migration history (historical)
 
 ## Workers
 
 | Worker | Entry | Endpoint | Auth |
 | --- | --- | --- | --- |
-| Query | `wrangler.toml` | `POST /ask`, `GET /health` | Turnstile (optional) + CORS allowlist |
-| MCP | `wrangler.mcp.toml` | `POST /mcp`, `GET /health` | `Authorization: Bearer` (fail closed) |
+| Query (public) | `wrangler.toml` | `POST /ask`, `GET /health` | Turnstile (optional) + CORS allowlist |
+| MCP (internal) | `wrangler.mcp.toml` | `POST /mcp`, `GET /health` | `Authorization: Bearer` (fail closed) |
+| Query (rockenhaus) | `wrangler.rockenhaus.toml` | `POST /ask`, `GET /health` | CORS allowlist (court-record origins) |
 
-Deploy separately so browser traffic and agent traffic can bind different AI Search instances if you want.
+Deploy separately so browser traffic and agent traffic can bind different AI Search instances.
 
 ```sh
-npm run deploy       # query Worker
-npm run deploy:mcp   # MCP Worker
+npm run deploy              # public query Worker
+npm run deploy:mcp          # MCP Worker
+npm run deploy:rockenhaus   # rockenhaus query Worker (uses committed wrangler file)
 wrangler secret put MCP_TOKEN -c wrangler.mcp.toml
 wrangler secret put TURNSTILE_SECRET   # optional; skips verification when unset
 ```
@@ -95,17 +105,44 @@ wrangler secret put TURNSTILE_SECRET   # optional; skips verification when unset
 
 `MCP_TOKEN` accepts a single token or comma-separated `name=token` pairs for per-consumer attribution in logs.
 
+### MCP tools (agents)
+
+| Tool | Role |
+| --- | --- |
+| `list_repos` | Exact repo names for filters (`CORPUS_REPOS` or R2 prefixes) |
+| `search` | Hybrid/keyword/vector chunks; optional `repos`, `path_prefix`, `min_score`, `rewrite` |
+| `get_file` | Capped R2 read by `repo` + `path` (needs `CORPUS` binding) |
+| `ask` | One grounded answer via chatCompletions + source list |
+| `corpus_status` | Last sync metadata (`_meta/corpus-status.json` from `sync.mjs`) |
+
+Resources: `corpus://catalog`, `corpus://skill`. Prefer **list_repos → search → get_file**; use **ask** when you want prose rather than raw chunks.
+
+Bind the same R2 bucket the sync uploads to as `CORPUS` on the MCP Worker so `get_file` and status work. Optional `CORPUS_REPOS` JSON pins the catalog without listing R2.
+
 ## Corpus sync
 
 ```sh
 export R2_ACCESS_KEY_ID=... R2_SECRET_ACCESS_KEY=... CLOUDFLARE_ACCOUNT_ID=...
 export CORPUS_GIT_ORG=your-org GITHUB_TOKEN=...   # for sync-runner clone auth
 
-npm run sync:dry     # plan upload for the default `corpus` target
-npm run sync         # upload + prune
-npm run sync:run     # isolated clone root, sync all targets, optional reindex
+npm run sync:dry          # plan upload for the default `corpus` target
+npm run sync              # upload + prune
+npm run sync:public       # skyphusion public target (when targets.json has it)
+npm run sync:internal     # skyphusion internal target
+npm run sync:rockenhaus   # rockenhaus target
+npm run sync:run          # isolated clone root, sync all targets, optional reindex
 ```
 
+Useful npm scripts (also `npx` CLIs for the first two after install):
+
+| Script | Role |
+| --- | --- |
+| `search-mcp-sync` / `npm run sync` | One-target R2 sync |
+| `search-mcp-sync-run` / `npm run sync:run` | Clone/fetch + multi-target + reindex |
+| `npm run guard:targets` | Additive-only targets.json check |
+| `npm run escrow` | Age-escrow + restore proof for targets secret |
+| `npm run materialize-config` | CI: write wrangler + targets from env secrets |
+
 The sync remaps non-native extensions (`.ts`, `.tsx`, extensionless `Dockerfile`, `.service`, etc.) to `.txt` keys so AI Search indexes them. See `scripts/sync-ingest.mjs`.
 
 ### Bounding a corpus: `includePaths` vs `excludePaths`
@@ -115,6 +152,11 @@ repo and it silently joins the corpus. That is fine for a docs site. It is wrong
 whenever the corpus boundary actually matters, because "we forgot to exclude it"
 becomes a real incident.
 
+`includePaths` / `excludePaths` may sit at the **top level** (same rule for every
+target that lists the repo) or **nested under a target** (search-mcp#62). When
+both set the same repo, the per-target map wins for that target only -- so a
+repo can be full-tree on internal and `_corpus/`-only on a public court surface.
+
 `includePaths` is an allowlist and is **fail-closed**: when a repo has an entry,
 only paths under those prefixes are eligible and everything else is refused.
 
@@ -201,13 +243,16 @@ before dispatching:
 2. **The post-job cooldown.** Even once a job ends, a new one is refused for a cooldown window
    with `sync_in_cooldown [code: 7020]`. Waiting for the job to end is necessary but not
    sufficient, so we retry until it clears.
+3. **Transient connect failures.** `unable_to_connect_to_ai_search [code: 7017]` on jobs create
+   is retried under the same budget as cooldown (search-mcp#73). Treating it as terminal made
+   the whole sync red while R2 already had the new objects.
 
 Waiting (rather than skipping) means the job we start always lands strictly after our own
 upload, so it sees every object this run wrote. Merge bursts still coalesce: a waiting run holds
 the workflow concurrency group, and GitHub keeps only the newest queued run, so the runs behind
 it collapse instead of each firing their own reindex.
 
-Each wait has its own budget (10 min in-flight, 10 min cooldown) rather than one shared
+Each wait has its own budget (10 min in-flight, 10 min cooldown/connect) rather than one shared
 deadline, since the two are additive on a perfectly healthy path: a run can wait minutes for an
 in-flight reindex and then still owe a cooldown wait.
 
@@ -258,8 +303,9 @@ Operators building documentation search, agent tooling, or internal knowledge ba
 ## Links
 
 - **Deploy guide:** [docs/DEPLOY.md](docs/DEPLOY.md)
-- **Skyphusion Labs:** https://skyphusion.org · **Org:** https://github.com/skyphusion-labs
-- **Related:** [vivijure](https://github.com/skyphusion-labs/vivijure), [postern](https://github.com/skyphusion-labs/postern)
+- **Live ask UIs:** [skyphusion.net/search](https://skyphusion.net/search/) · [vivijure.com](https://vivijure.com) (ask widget)
+- **Skyphusion Labs:** https://skyphusion.org · **Blog:** https://skyphusion.net · **Org:** https://github.com/skyphusion-labs
+- **Related:** [vivijure](https://github.com/skyphusion-labs/vivijure), [postern](https://github.com/skyphusion-labs/postern), [security-audit](https://github.com/skyphusion-labs/security-audit), [prism](https://github.com/skyphusion-labs/prism)
 
 ## License
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@skyphusion/search-mcp",
-  "version": "0.3.0",
+  "version": "0.5.0",
   "description": "Bearer-gated MCP server and streaming query Worker over Cloudflare AI Search, plus a git-to-R2 corpus sync toolkit.",
   "license": "AGPL-3.0-only",
   "author": "Conrad Rockenhaus <conrad@skyphusion.org>",
@@ -52,7 +52,7 @@
     "dev:mcp": "wrangler dev -c wrangler.mcp.toml",
     "deploy": "wrangler deploy",
     "deploy:mcp": "wrangler deploy -c wrangler.mcp.toml",
-    "typecheck": "tsc --noEmit",
+    "typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.scripts.json",
     "test": "vitest run",
     "prepack": "npm run typecheck && npm test",
     "sync": "node scripts/sync.mjs corpus",
@@ -62,16 +62,22 @@
     "sync:run": "node scripts/sync-runner.mjs",
     "materialize-config": "node scripts/materialize-config.mjs",
     "escrow": "node scripts/escrow-targets.mjs",
-    "guard:targets": "node scripts/guard-targets-additive.mjs"
+    "guard:targets": "node scripts/guard-targets-additive.mjs",
+    "deploy:rockenhaus": "wrangler deploy -c wrangler.rockenhaus.toml",
+    "sync:rockenhaus": "node scripts/sync.mjs rockenhaus"
   },
   "dependencies": {
     "@aws-sdk/client-s3": "^3.888.0"
   },
   "devDependencies": {
     "@cloudflare/workers-types": "^5.20260713.1",
+    "@types/node": "^22.15.0",
     "@vitest/coverage-istanbul": "^4.1.10",
     "typescript": "^7.0.2",
     "vitest": "^4.1.10",
     "wrangler": "^4.110.0"
+  },
+  "overrides": {
+    "undici": "7.29.0"
   }
 }

package/scripts/sync-ingest.mjs

@@ -207,6 +207,75 @@ export function knownTargetRepos(cfg) {
   return repos;
 }
 
+/**
+ * Resolve include/exclude path maps for ONE target (search-mcp#62).
+ *
+ * Precedence: per-target maps win per-repo over top-level maps. Top-level alone
+ * keeps the pre-#62 shape (the same rule for every target that lists the repo).
+ * A repo listed only under another target's nested map is not visible here.
+ *
+ * Why both layers: the original top-level key was fine while every exclusion
+ * applied to exactly one target; the moment a repo is deliberately indexed at
+ * two granularities (public vs internal), a single map cannot express it.
+ */
+export function pathMapsForTarget(cfg, targetName) {
+  const target = cfg?.targets?.[targetName] || {};
+  const topInclude =
+    cfg?.includePaths && typeof cfg.includePaths === "object" && !Array.isArray(cfg.includePaths)
+      ? cfg.includePaths
+      : {};
+  const topExclude =
+    cfg?.excludePaths && typeof cfg.excludePaths === "object" && !Array.isArray(cfg.excludePaths)
+      ? cfg.excludePaths
+      : {};
+  const tInclude =
+    target.includePaths && typeof target.includePaths === "object" && !Array.isArray(target.includePaths)
+      ? target.includePaths
+      : {};
+  const tExclude =
+    target.excludePaths && typeof target.excludePaths === "object" && !Array.isArray(target.excludePaths)
+      ? target.excludePaths
+      : {};
+  return {
+    includePaths: { ...topInclude, ...tInclude },
+    excludePaths: { ...topExclude, ...tExclude },
+  };
+}
+
+function isPathMapObject(v) {
+  return v !== undefined && v !== null && typeof v === "object" && !Array.isArray(v);
+}
+
+/** Validate one path-map object against a known-repo set. Returns error objects. */
+function validateIncludePathsMap(includePaths, known, label) {
+  const errors = [];
+  if (includePaths === undefined || includePaths === null) return errors;
+  if (!isPathMapObject(includePaths)) {
+    errors.push({
+      code: "include_paths_not_object",
+      message:
+        `${label} must be an object keyed by repo, got ${describeType(includePaths)}. ` +
+        `Shape: "includePaths": { "my-repo": ["_corpus/"] }.`,
+    });
+    return errors;
+  }
+  for (const [repo, prefixes] of Object.entries(includePaths)) {
+    if (repo.startsWith("_")) continue; // documentation keys in the example file
+    if (!known.has(repo)) {
+      errors.push({
+        code: "include_paths_unknown_repo",
+        message:
+          `${label} names repo ${q(repo)}, which is not in the applicable target repo list. ` +
+          `Nothing reads that rule, and a rule nothing reads is indistinguishable from a rule ` +
+          `that works. Add the repo to the target or delete the entry. Applicable repos: ` +
+          `${[...known].join(", ") || "(none)"}.`,
+      });
+    }
+    errors.push(...validateIncludePathsEntry(repo, prefixes));
+  }
+  return errors;
+}
+
 /** Shape check for one repo entry. Returns {code, message} objects, never throws. */
 export function validateIncludePathsEntry(repo, prefixes) {
   const errors = [];
@@ -263,38 +332,24 @@ export function validateIncludePathsEntry(repo, prefixes) {
 }
 
 /**
- * Whole-config check for targets.json `includePaths`. Returns {code, message}
- * objects, never throws, so a caller can report all of them at once.
+ * Whole-config check for targets.json `includePaths` (top-level AND per-target,
+ * search-mcp#62). Returns {code, message} objects, never throws, so a caller can
+ * report all of them at once.
  *
- * Rejects an entry naming a repo no target lists: a rule nothing reads is
- * indistinguishable from a rule that works, which is how an allowlist rots.
+ * Rejects an entry naming a repo no applicable target lists: a rule nothing
+ * reads is indistinguishable from a rule that works, which is how an allowlist
+ * rots. Per-target maps are checked against that target's repos only.
  */
 export function validateIncludePathsConfig(cfg) {
   const errors = [];
-  const includePaths = cfg?.includePaths;
-  if (includePaths === undefined || includePaths === null) return errors;
-  if (typeof includePaths !== "object" || Array.isArray(includePaths)) {
-    errors.push({
-      code: "include_paths_not_object",
-      message:
-        `targets.json includePaths must be an object keyed by repo, got ` +
-        `${describeType(includePaths)}. Shape: "includePaths": { "my-repo": ["_corpus/"] }.`,
-    });
-    return errors;
-  }
-  const known = knownTargetRepos(cfg);
-  for (const [repo, prefixes] of Object.entries(includePaths)) {
-    if (!known.has(repo)) {
-      errors.push({
-        code: "include_paths_unknown_repo",
-        message:
-          `includePaths names repo ${q(repo)}, which no target lists in its repos. Nothing reads ` +
-          `that rule, and a rule nothing reads is indistinguishable from a rule that works. Add ` +
-          `the repo to a target or delete the entry. Repos across all targets: ` +
-          `${[...known].join(", ") || "(none)"}.`,
-      });
-    }
-    errors.push(...validateIncludePathsEntry(repo, prefixes));
+  const knownAll = knownTargetRepos(cfg);
+  errors.push(...validateIncludePathsMap(cfg?.includePaths, knownAll, "targets.json includePaths"));
+  for (const [name, target] of Object.entries(cfg?.targets || {})) {
+    if (!target || typeof target !== "object") continue;
+    const known = new Set(Array.isArray(target.repos) ? target.repos : []);
+    errors.push(
+      ...validateIncludePathsMap(target.includePaths, known, `targets.${name}.includePaths`),
+    );
   }
   return errors;
 }
@@ -307,15 +362,30 @@ export function assertIncludePathsConfig(cfg) {
 }
 
 /**
- * excludePaths keys naming a repo no target lists. Denylist rot is quieter than
- * allowlist rot (the corpus grows rather than empties), so this is a warning at
- * the call site, not a refusal.
+ * excludePaths keys (top-level OR nested under a target) naming a repo no
+ * applicable target lists. Denylist rot is quieter than allowlist rot (the
+ * corpus grows rather than empties), so this is a warning at the call site,
+ * not a refusal. Returns unique repo names.
  */
 export function unknownExcludePathsRepos(cfg) {
-  const excludePaths = cfg?.excludePaths;
-  if (!excludePaths || typeof excludePaths !== "object" || Array.isArray(excludePaths)) return [];
-  const known = knownTargetRepos(cfg);
-  return Object.keys(excludePaths).filter((repo) => !known.has(repo));
+  const out = new Set();
+  const knownAll = knownTargetRepos(cfg);
+  const top = cfg?.excludePaths;
+  if (isPathMapObject(top)) {
+    for (const repo of Object.keys(top)) {
+      if (!repo.startsWith("_") && !knownAll.has(repo)) out.add(repo);
+    }
+  }
+  for (const target of Object.values(cfg?.targets || {})) {
+    if (!target || typeof target !== "object") continue;
+    const nested = target.excludePaths;
+    if (!isPathMapObject(nested)) continue;
+    const known = new Set(Array.isArray(target.repos) ? target.repos : []);
+    for (const repo of Object.keys(nested)) {
+      if (!repo.startsWith("_") && !known.has(repo)) out.add(repo);
+    }
+  }
+  return [...out];
 }
 
 /**

package/scripts/sync-runner.mjs

@@ -172,10 +172,23 @@ export const REINDEX_POLL_MS = 15 * 1000;
  * AI Search refuses a new job for a cooldown window after the previous one ENDS, distinct from
  * the job being in flight. Measured 2026-07-16: rejected 10s after a job ended, accepted at
  * 32s. Waiting for `ended_at` alone is necessary but not sufficient.
+ *
+ * Also treated as transient (2026-08-04 live corpus-sync): `unable_to_connect_to_ai_search
+ * [code: 7017]` on jobs create for skyphusion-public -- the instance was healthy (list jobs
+ * worked; a later create succeeded). Retrying under the same budget as cooldown is correct;
+ * treating 7017 as terminal made the whole sync red while R2 already had the new objects.
  */
-export function isCooldownError(err) {
+export function isTransientReindexError(err) {
   const text = `${err?.stdout || ""}${err?.stderr || ""}${err?.message || ""}`;
-  return /sync_in_cooldown|code:\s*7020/.test(text);
+  return (
+    /sync_in_cooldown|code:\s*7020/.test(text) ||
+    /unable_to_connect_to_ai_search|code:\s*7017/.test(text)
+  );
+}
+
+/** @deprecated prefer isTransientReindexError; kept so older call sites and tests keep working. */
+export function isCooldownError(err) {
+  return isTransientReindexError(err);
 }
 
 /**
@@ -218,13 +231,14 @@ export async function awaitReindexSlot(instance, deps) {
 }
 
 /**
- * Dispatch a reindex, retrying while AI Search reports its post-job cooldown.
+ * Dispatch a reindex, retrying while AI Search reports a TRANSIENT refuse
+ * (post-job cooldown 7020, or unable_to_connect_to_ai_search 7017).
  *
- * Cooldown is transient and short (measured well under the budget), so the routine burst case
- * clears here and never goes red. Exhausting the budget therefore means something genuinely
- * anomalous upstream, and we fail loud rather than exit green: a green run while indexing is
- * actually stalled is the work-blind failure mode this whole issue exists to remove. The
- * message carries the honest blast radius so the red is actionable, not alarming.
+ * These clears are short relative to the budget, so the routine burst case clears here and
+ * never goes red. Exhausting the budget means something genuinely anomalous upstream, and we
+ * fail loud rather than exit green: a green run while indexing is actually stalled is the
+ * work-blind failure mode this whole issue exists to remove. The message carries the honest
+ * blast radius so the red is actionable, not alarming.
  */
 export async function dispatchWithCooldownRetry(instance, deps) {
   const {
@@ -242,17 +256,21 @@ export async function dispatchWithCooldownRetry(instance, deps) {
       runReindexOnce(instance);
       return { retried: announced };
     } catch (err) {
-      if (!isCooldownError(err)) throw err;
+      if (!isTransientReindexError(err)) throw err;
       if (now() >= deadline) {
         throw new Error(
-          `reindex for ${instance} blocked by AI Search cooldown for longer than ` +
-            `${Math.round(timeoutMs / 1000)}s. The R2 corpus uploaded OK and no data is lost; ` +
-            "the index lags until the next sync or the daily backstop. A cooldown this " +
-            "persistent is upstream and anomalous, so this run fails loudly on purpose.",
+          `reindex for ${instance} blocked by transient AI Search errors for longer than ` +
+            `${Math.round(timeoutMs / 1000)}s (cooldown 7020 and/or connect 7017). ` +
+            "The R2 corpus uploaded OK and no data is lost; the index lags until the next " +
+            "sync or the daily backstop. Persistence this long is upstream and anomalous, " +
+            "so this run fails loudly on purpose.",
         );
       }
       if (!announced) {
-        log(`  ${instance} is in AI Search cooldown; retrying until it clears.`);
+        log(
+          `  ${instance} hit a transient AI Search refuse (cooldown or unable_to_connect); ` +
+            "retrying until it clears.",
+        );
         announced = true;
       }
       await sleep(pollMs);

package/scripts/sync.mjs

@@ -39,6 +39,7 @@ import {
   selectRepoPaths,
   assertIncludePathsConfig,
   unknownExcludePathsRepos,
+  pathMapsForTarget,
   IncludePathsError,
 } from "./sync-ingest.mjs";
 import {
@@ -246,13 +247,16 @@ async function main() {
   console.log(`Target '${targetName}' -> bucket ${target.bucket} (instance ${target.instance})`);
   console.log(`Repo root: ${REPO_ROOT}${DRY ? "  [DRY RUN]" : ""}`);
 
+  // #62: per-target path maps win over top-level for this target.
+  const pathMaps = pathMapsForTarget(cfg, targetName);
+
   const plan = [];
   const skipped = [];
   for (const repo of target.repos) {
     const items = planRepo(
       repo,
-      cfg.excludePaths?.[repo],
-      cfg.includePaths?.[repo],
+      pathMaps.excludePaths?.[repo],
+      pathMaps.includePaths?.[repo],
       skipped,
     );
     if (items.length) console.log(`  + ${repo}: ${items.length} files`);
@@ -346,13 +350,36 @@ async function main() {
     ContinuationToken = res.IsTruncated ? res.NextContinuationToken : undefined;
   } while (ContinuationToken);
 
-  for (let i = 0; i < stale.length; i += 1000) {
-    const batch = stale.slice(i, i + 1000);
+  // Never prune operator/meta keys written for MCP corpus_status.
+  const stalePrune = stale.filter((o) => o.Key && !o.Key.startsWith("_meta/"));
+  for (let i = 0; i < stalePrune.length; i += 1000) {
+    const batch = stalePrune.slice(i, i + 1000);
     await s3.send(
       new DeleteObjectsCommand({ Bucket, Delete: { Objects: batch } }),
     );
   }
-  console.log(`Pruned ${stale.length} stale objects.`);
+  console.log(`Pruned ${stalePrune.length} stale objects.`);
+
+  // MCP corpus_status + list_repos freshness (search-mcp agent tooling).
+  const status = {
+    ts: Date.now(),
+    ok: true,
+    target: targetName,
+    bucket: target.bucket,
+    instance: target.instance,
+    objectCount: plan.length,
+    repos: [...target.repos],
+    note: "Written by scripts/sync.mjs after successful upload+prune.",
+  };
+  await s3.send(
+    new PutObjectCommand({
+      Bucket,
+      Key: "_meta/corpus-status.json",
+      Body: Buffer.from(JSON.stringify(status, null, 2), "utf8"),
+      ContentType: "application/json",
+    }),
+  );
+  console.log("Wrote _meta/corpus-status.json for MCP corpus_status.");
   console.log("Done. AI Search will re-index on its next sync.");
 }
 

package/scripts/targets.json.example

@@ -1,11 +1,11 @@
 {
   "_comment": "Copy to targets.json (./targets.json for npm CLI, or scripts/targets.json in a git clone). repos are directory names under SYNC_REPO_ROOT (default: cwd for npm, parent of this repo for dev clone). Only git-tracked files are uploaded.",
-  "_includePaths_comment": "ALLOWLIST, optional, per repo. When present, ONLY paths under these prefixes are eligible for the corpus and everything else is refused. Use it when the corpus boundary matters: a denylist is fail-OPEN, so a new top-level file silently joins the corpus, whereas an allowlist is fail-CLOSED. Omit a repo to keep the old behaviour of admitting everything; an entry that is present but EMPTY is an error, not a way to index nothing.",
-  "_includePaths_refusals": "A present allowlist must earn its keep: an entry matching zero git-tracked files, an entry naming a repo no target lists, a repo whose clone is missing, or a selection that excludePaths or the ingest filters empty out all refuse the sync (exit 2). An allowlist that quietly matches nothing would prune a healthy corpus and reindex over the emptiness with every status light green.",
+  "_includePaths_comment": "ALLOWLIST, optional, per repo. When present, ONLY paths under these prefixes are eligible for the corpus and everything else is refused. Use it when the corpus boundary matters: a denylist is fail-OPEN, so a new top-level file silently joins the corpus, whereas an allowlist is fail-CLOSED. Omit a repo to keep the old behaviour of admitting everything; an entry that is present but EMPTY is an error, not a way to index nothing. May live at the top level (applies to every target that lists the repo) OR nested under a target (search-mcp#62); per-target wins per repo when both set the same key.",
+  "_includePaths_refusals": "A present allowlist must earn its keep: an entry matching zero git-tracked files, an entry naming a repo no applicable target lists, a repo whose clone is missing, or a selection that excludePaths or the ingest filters empty out all refuse the sync (exit 2). An allowlist that quietly matches nothing would prune a healthy corpus and reindex over the emptiness with every status light green.",
   "includePaths": {
     "example-repo": ["docs/"]
   },
-  "_excludePaths_comment": "DENYLIST, optional, per repo. Applied after includePaths.",
+  "_excludePaths_comment": "DENYLIST, optional, per repo. Applied after includePaths. Same top-level vs per-target nesting as includePaths.",
   "excludePaths": {
     "example-repo": ["vendor/private-notes/"]
   },
@@ -15,6 +15,15 @@
       "instance": "my-ai-search-instance",
       "bucket": "my-search-corpus",
       "repos": ["example-repo"]
+    },
+    "_per_target_paths_example": {
+      "_comment": "Optional nested includePaths/excludePaths apply only to this target (search-mcp#62). Use when the same repo is indexed at different granularities across targets.",
+      "instance": "other-instance",
+      "bucket": "other-bucket",
+      "repos": ["example-repo"],
+      "includePaths": {
+        "example-repo": ["_corpus/"]
+      }
     }
   }
 }