@rubytech/create-realagent 1.0.616 → 1.0.618

package/payload/platform/plugins/admin/PLUGIN.md

@@ -67,3 +67,4 @@ Tools are available via the `admin` MCP server.
 
 - `hooks/pre-tool-use.sh` — enforces admin agent write boundaries
 - `hooks/playwright-file-guard.sh` — intercepts file:// URLs with actionable guidance
+- `hooks/webfetch-preflight.mjs` — short-circuits WebFetch on JS-SPA shells with a structured `WEBFETCH_CANNOT_READ_JS_SPA` error so the agent surfaces a loud failure to the owner instead of paying the 60s extraction timeout. Fail-open on any internal error.

package/payload/platform/plugins/admin/hooks/webfetch-preflight.mjs

@@ -0,0 +1,363 @@
+#!/usr/bin/env node
+// PreToolUse hook for WebFetch — detects JS-SPA shells before Claude Code's
+// internal extraction pipeline burns its 60-second axios timeout on an HTML
+// shell that has no server-rendered content. WebFetch is HTTP-only — when it
+// gets a body like `<div id="root"></div>` plus a single `<script>`, there
+// is nothing to extract, and waiting 60s changes nothing.
+//
+// Wired in account_settings.json hooks.PreToolUse[matcher=WebFetch] (see
+// platform/scripts/seed-neo4j.sh). Reads Claude Code hook protocol JSON on
+// stdin: { tool_name, tool_input: { url, prompt } }.
+//
+// Outcomes:
+//   spa-shell    → exit 2, stderr = WEBFETCH_CANNOT_READ_JS_SPA: <directive>
+//                  Claude Code synthesises a tool_result with is_error=true
+//                  and stderr as content, the agent surfaces a loud failure
+//                  to the user (per IDENTITY.md "Tool Failure Discipline").
+//   ok           → exit 0, allow the real WebFetch to run.
+//   inconclusive → exit 0, allow the real WebFetch to run. Anything that
+//                  isn't a high-confidence positive shell detection passes
+//                  through — fail-open is the only safe posture for a
+//                  preflight gate. Blocking false positives would be worse
+//                  than the 60-second stall this hook exists to prevent.
+//
+// Every branch writes one verdict line to
+// ${ACCOUNT_DIR}/logs/webfetch-preflight.log so the decision trail is
+// retrievable per-account without correlating across conversation logs.
+//
+// Cache: ${ACCOUNT_DIR}/cache/webfetch-preflight/<sha256(url)>.json with
+// a 10-minute mtime TTL. The verdict is intrinsic to the URL's response
+// shape, not to any conversation's context — account-scoped sharing
+// eliminates duplicate probes when the same URL recurs across sessions.
+
+import { createHash } from "node:crypto";
+import { mkdirSync, readFileSync, writeFileSync, statSync, appendFileSync } from "node:fs";
+import { resolve, dirname } from "node:path";
+
+const PROBE_TIMEOUT_MS = 2000;
+const MAX_BODY_BYTES = 64 * 1024;
+const CACHE_TTL_MS = 10 * 60 * 1000;
+const SHELL_BYTE_THRESHOLD = 5 * 1024;
+const TEXT_BYTE_THRESHOLD = 20;
+
+const accountDir = process.env.ACCOUNT_DIR || "";
+const logPath = accountDir ? resolve(accountDir, "logs", "webfetch-preflight.log") : null;
+
+function ts() {
+  return new Date().toISOString();
+}
+
+function logLine(line) {
+  if (!logPath) return;
+  try {
+    mkdirSync(dirname(logPath), { recursive: true });
+    appendFileSync(logPath, `[${ts()}] ${line}\n`);
+  } catch {
+    // Logging failure must never block the tool. The verdict still emits to
+    // stderr where Claude Code's own hook-stderr capture will catch it.
+  }
+}
+
+function emitVerdict(fields, allowFlag) {
+  const line = `[webfetch-preflight] ${fields.join(" ")}`;
+  logLine(line);
+  // Stderr trail so a reader inspecting Claude Code's hook stderr capture
+  // sees the same verdict line (independent of the per-account log file).
+  process.stderr.write(line + "\n");
+  if (allowFlag === "block") {
+    // Caller adds the structured WEBFETCH_CANNOT_READ_JS_SPA message before
+    // exiting — this function only emits the diagnostic line.
+    return;
+  }
+  process.exit(0);
+}
+
+async function readStdin() {
+  return new Promise((resolvePromise, rejectPromise) => {
+    const chunks = [];
+    let total = 0;
+    const limit = 64 * 1024;
+    process.stdin.on("data", (c) => {
+      chunks.push(c);
+      total += c.length;
+      if (total > limit) {
+        // Destroy the stream before rejecting so the `end` listener can't fire
+        // a no-op resolve on the rejected Promise (and so we don't allocate
+        // a large Buffer.concat over already-discarded chunks).
+        process.stdin.destroy();
+        rejectPromise(new Error("stdin-too-large"));
+      }
+    });
+    process.stdin.on("end", () => resolvePromise(Buffer.concat(chunks).toString("utf8")));
+    process.stdin.on("error", rejectPromise);
+  });
+}
+
+function readBodyCapped(res, maxBytes) {
+  return new Promise(async (resolvePromise, rejectPromise) => {
+    try {
+      const reader = res.body.getReader();
+      const chunks = [];
+      let total = 0;
+      while (true) {
+        const { value, done } = await reader.read();
+        if (done) break;
+        chunks.push(Buffer.from(value));
+        total += value.byteLength;
+        if (total >= maxBytes) {
+          reader.cancel().catch(() => {});
+          break;
+        }
+      }
+      resolvePromise(Buffer.concat(chunks, Math.min(total, maxBytes)));
+    } catch (err) {
+      rejectPromise(err);
+    }
+  });
+}
+
+function detectShape(bodyStr) {
+  // Strip <script>, <style>, comments, then all tags. What remains is the
+  // user-visible text. <20 chars after this strip = the page renders no
+  // meaningful content server-side.
+  const stripped = bodyStr
+    .replace(/<script\b[^>]*>[\s\S]*?<\/script>/gi, "")
+    .replace(/<style\b[^>]*>[\s\S]*?<\/style>/gi, "")
+    .replace(/<noscript\b[^>]*>[\s\S]*?<\/noscript>/gi, "")
+    .replace(/<!--[\s\S]*?-->/g, "")
+    .replace(/<[^>]+>/g, "")
+    .replace(/\s+/g, " ")
+    .trim();
+
+  const hasRootDiv =
+    /<div\s+id=["']root["']\s*>\s*<\/div>/i.test(bodyStr) ||
+    /<div\s+id=["']app["']\s*>\s*<\/div>/i.test(bodyStr) ||
+    /<div\s+id=["']__next["']\s*>\s*<\/div>/i.test(bodyStr);
+
+  // <body> contains only <script>/<link>/<meta> + whitespace + comments.
+  // Common pattern for Vite/CRA shells without a labelled root div.
+  const bodyMatch = bodyStr.match(/<body\b[^>]*>([\s\S]*?)<\/body>/i);
+  let bodyOnlyScripts = false;
+  if (bodyMatch) {
+    const inner = bodyMatch[1].replace(/<!--[\s\S]*?-->/g, "");
+    const tagless = inner
+      .replace(/<(script|link|meta)\b[^>]*>[\s\S]*?<\/\1>/gi, "")
+      .replace(/<(script|link|meta)\b[^>]*\/?>/gi, "")
+      .trim();
+    bodyOnlyScripts = tagless.length === 0 && /<script\b/i.test(inner);
+  }
+
+  return { textBytes: stripped.length, hasRootDiv, bodyOnlyScripts };
+}
+
+function classifyFetchError(err) {
+  const name = err && err.name ? err.name : "";
+  const code = err && err.cause && err.cause.code ? err.cause.code : err && err.code ? err.code : "";
+  if (name === "AbortError" || name === "TimeoutError") return "timeout";
+  if (code === "ENOTFOUND" || code === "EAI_AGAIN") return "dns-fail";
+  if (code === "ECONNREFUSED" || code === "ECONNRESET" || code === "EHOSTUNREACH") return "tcp-fail";
+  if (code === "CERT_HAS_EXPIRED" || code === "DEPTH_ZERO_SELF_SIGNED_CERT") return "tls-fail";
+  return `fetch-err-${code || name || "unknown"}`;
+}
+
+async function main() {
+  let raw;
+  try {
+    raw = await readStdin();
+  } catch (err) {
+    emitVerdict([`verdict=inconclusive`, `reason=stdin-${err.message || "unknown"}`]);
+    return;
+  }
+
+  let payload;
+  try {
+    payload = JSON.parse(raw);
+  } catch {
+    emitVerdict([`verdict=inconclusive`, `reason=stdin-not-json`]);
+    return;
+  }
+
+  const toolInput = payload && typeof payload === "object" ? payload.tool_input : null;
+  const url = toolInput && typeof toolInput.url === "string" ? toolInput.url : "";
+  if (!url) {
+    emitVerdict([`verdict=inconclusive`, `reason=no-url-in-tool-input`]);
+    return;
+  }
+
+  let parsed;
+  try {
+    parsed = new URL(url);
+  } catch {
+    emitVerdict([`verdict=inconclusive`, `reason=unparseable-url`, `url=${JSON.stringify(url)}`]);
+    return;
+  }
+
+  if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
+    emitVerdict([
+      `verdict=inconclusive`,
+      `reason=disallowed-scheme`,
+      `scheme=${parsed.protocol.replace(":", "")}`,
+    ]);
+    return;
+  }
+
+  // Cache key — strip fragment, normalise. Same URL across conversations
+  // hits the same cache entry; verdict is intrinsic to the response shape.
+  parsed.hash = "";
+  const cacheKey = parsed.toString();
+  const cacheHash = createHash("sha256").update(cacheKey).digest("hex").slice(0, 32);
+  const cacheDir = accountDir ? resolve(accountDir, "cache", "webfetch-preflight") : null;
+  const cachePath = cacheDir ? resolve(cacheDir, `${cacheHash}.json`) : null;
+
+  if (cachePath) {
+    try {
+      const stat = statSync(cachePath);
+      const ageMs = Date.now() - stat.mtimeMs;
+      if (ageMs < CACHE_TTL_MS) {
+        const cached = JSON.parse(readFileSync(cachePath, "utf8"));
+        if (cached && typeof cached.verdict === "string") {
+          const ageSec = Math.round(ageMs / 1000);
+          const fields = [
+            `verdict=${cached.verdict}`,
+            `body_bytes=${cached.body_bytes ?? 0}`,
+            `text_bytes=${cached.text_bytes ?? 0}`,
+            `cache=hit`,
+            `cache_age_s=${ageSec}`,
+            `url=${JSON.stringify(cacheKey)}`,
+          ];
+          if (cached.reason) fields.push(`reason=${cached.reason}`);
+          if (cached.verdict === "spa-shell") {
+            emitVerdict(fields, "block");
+            const bytes = cached.body_bytes ?? 0;
+            process.stderr.write(buildShortCircuitMessage(cacheKey, bytes) + "\n");
+            process.exit(2);
+          }
+          emitVerdict(fields);
+          return;
+        }
+      }
+    } catch {
+      // Cache miss or unreadable — continue to live probe.
+    }
+  }
+
+  let res;
+  try {
+    res = await fetch(parsed.toString(), {
+      headers: { Accept: "text/html,application/xhtml+xml,*/*;q=0.5" },
+      signal: AbortSignal.timeout(PROBE_TIMEOUT_MS),
+      redirect: "follow",
+    });
+  } catch (err) {
+    emitVerdict([
+      `verdict=inconclusive`,
+      `reason=${classifyFetchError(err)}`,
+      `url=${JSON.stringify(cacheKey)}`,
+    ]);
+    return;
+  }
+
+  if (res.status >= 500) {
+    emitVerdict([
+      `verdict=inconclusive`,
+      `reason=http-${res.status}`,
+      `url=${JSON.stringify(cacheKey)}`,
+    ]);
+    return;
+  }
+
+  const contentType = (res.headers.get("content-type") || "").toLowerCase();
+  if (contentType && !contentType.includes("text/html") && !contentType.includes("application/xhtml")) {
+    emitVerdict([
+      `verdict=inconclusive`,
+      `reason=non-html-content-type`,
+      `ct=${JSON.stringify(contentType.slice(0, 60))}`,
+      `url=${JSON.stringify(cacheKey)}`,
+    ]);
+    return;
+  }
+
+  let body;
+  try {
+    body = await readBodyCapped(res, MAX_BODY_BYTES);
+  } catch (err) {
+    emitVerdict([
+      `verdict=inconclusive`,
+      `reason=body-read-${classifyFetchError(err)}`,
+      `url=${JSON.stringify(cacheKey)}`,
+    ]);
+    return;
+  }
+
+  const bodyBytes = body.length;
+  const bodyStr = body.toString("utf8");
+  const { textBytes, hasRootDiv, bodyOnlyScripts } = detectShape(bodyStr);
+
+  const isShell =
+    bodyBytes <= SHELL_BYTE_THRESHOLD &&
+    textBytes < TEXT_BYTE_THRESHOLD &&
+    (hasRootDiv || bodyOnlyScripts);
+
+  const verdict = isShell ? "spa-shell" : "ok";
+  const reason = isShell ? "js-spa-no-server-content" : undefined;
+
+  if (cachePath && cacheDir) {
+    try {
+      mkdirSync(cacheDir, { recursive: true });
+      writeFileSync(
+        cachePath,
+        JSON.stringify({
+          verdict,
+          body_bytes: bodyBytes,
+          text_bytes: textBytes,
+          reason,
+          url: cacheKey,
+          fetched_at: Date.now(),
+        }),
+      );
+    } catch {
+      // Best-effort. A failed cache write means the next probe re-runs;
+      // not a correctness issue.
+    }
+  }
+
+  const fields = [
+    `verdict=${verdict}`,
+    `body_bytes=${bodyBytes}`,
+    `text_bytes=${textBytes}`,
+    `cache=miss`,
+    `url=${JSON.stringify(cacheKey)}`,
+  ];
+  if (isShell) {
+    fields.push(`has_root_div=${hasRootDiv}`, `body_only_scripts=${bodyOnlyScripts}`, `reason=${reason}`);
+    emitVerdict(fields, "block");
+    process.stderr.write(buildShortCircuitMessage(cacheKey, bodyBytes) + "\n");
+    process.exit(2);
+  }
+  emitVerdict(fields);
+}
+
+function buildShortCircuitMessage(url, bodyBytes) {
+  // Wording is the actual content the LLM sees as tool_result. Names the
+  // failure class (machine-readable prefix), the reason in plain English,
+  // and the two concrete user actions that unblock the task. The directive
+  // not to silently retry is explicit because the model's instinct is to
+  // try Playwright next — that hides the failure from the user.
+  return (
+    `WEBFETCH_CANNOT_READ_JS_SPA: The HTML response at ${url} is a JavaScript ` +
+    `single-page-application shell (${bodyBytes} bytes, no server-rendered text). ` +
+    `WebFetch does not execute JavaScript, so it has no content to extract. ` +
+    `Do NOT retry WebFetch on this URL and do NOT silently dispatch another tool ` +
+    `(Playwright, research-assistant, memory-search) to substitute — that hides ` +
+    `the failure from the user. Stop, tell the user plainly that you cannot ` +
+    `read this URL because it renders content with JavaScript, and ask them to ` +
+    `paste the relevant text or send a screenshot.`
+  );
+}
+
+main().catch((err) => {
+  // Top-level safety net — fail-open. A crash in the preflight must never
+  // block a tool dispatch; the worst case is we lose the optimisation and
+  // pay the existing 60s timeout cost.
+  emitVerdict([`verdict=inconclusive`, `reason=preflight-crash`, `err=${JSON.stringify(String(err).slice(0, 80))}`]);
+});

package/payload/platform/plugins/admin/skills/stream-log-review/SKILL.md

@@ -52,9 +52,12 @@ The `conversationId` is visible in the admin UI and appears on every `[spawn]` /
 - `[tool-wait]` lines every 5 s between `[tool-use]` and `[tool-result]` — distinguishes API silence from tool-internal stall.
 - `[tool-wait-diag]` at 15 / 30 / 45 / 60 s — confirms DNS/TCP/HTTP health across a stall window (not just at the timeout moment).
 - `[tool-wait-proc]` at 30 s — subprocess open FDs, sockets by TCP state, RSS.
-- `[subproc-stderr]` — subprocess stderr teed in, includes `NODE_DEBUG=http,http2,net,tls,undici,dns` network traces. Missing UNDICI/HTTP write lines during a tool-wait window = tool never sent the request.
+- `[subproc-stderr]` — line from the main Claude Code subprocess's stderr. Today the CLI is a bundled Bun binary that ignores Node's `NODE_DEBUG`, so this channel is normally silent — see `[subproc-debug-unavailable]` below. MCP server stderr lines arrive separately as `[mcp:<server>]`.
+- `[subproc-stderr-tee-attached]` / `[subproc-stderr-tee-detached]` (Task 535) — lifecycle of the main-subprocess stderr tee. `bytes=0 lines=0` on detach means the tee worked but the subprocess emitted nothing. Absence of both markers next to a `[spawn]` means the tee infrastructure is broken — escalate.
+- `[subproc-debug-unavailable]` (Task 535) — one line per spawn, `reason=bundled-bun-binary-ignores-node-debug`. This is the documented reason `[subproc-stderr]` lines are normally absent for the main subprocess. Treat its absence as a regression, not its presence as a problem.
 - `[tool-failure-diag]` — one-shot probe on the failure path (Task 530); complements the mid-flight `[tool-wait-diag]` with end-of-wait network state.
 - `[mcp-tee-attach]` / `[mcp-tee-skip]` / `[mcp:<server>]` — MCP server stderr routed into the stream log; a missing attach marker explains missing server diagnostics.
+- `[tool-result] error=true output="WEBFETCH_CANNOT_READ_JS_SPA: …"` — a WebFetch dispatch was short-circuited by the SPA preflight hook (Task 536). The URL is a JS-SPA shell that WebFetch cannot extract content from. The agent should have responded to the user naming the failure and asking for a paste or screenshot; if instead an `[agent-dispatch]` (Playwright, research-assistant) follows immediately, the loud-failure guidance in IDENTITY.md did not land. The hook's per-invocation decision trail lives in `{accountDir}/logs/webfetch-preflight.log` (one `[webfetch-preflight] verdict=…` line per probe).
 
 ## Boundaries
 

package/payload/platform/plugins/cloudflare/PLUGIN.md

@@ -1,6 +1,6 @@
 ---
 name: cloudflare
-description: Cloudflare Tunnel setup and management — declarative brand-zone scope, deterministic recovery
+description: Cloudflare Tunnel setup and management — bound account is the universe, agent owns it absolutely
 tools:
   - cloudflare-setup
   - cf-add-zone
@@ -19,7 +19,7 @@ tools:
 
 # Cloudflare Tunnel Setup
 
-The brand declares its Cloudflare zones in `brand.json` (`cloudflare.zones`) at build time. Every tool refuses operations against hostnames whose registrable parent is not in that list — declared scope is authoritative. `cloudflare-setup` is the single onboarding orchestrator: a UI-driven state machine that prompts for tunnel/zone selections and admin/public addresses via rendered components (`single-select`, `tunnel-route-picker`). Tunnel names are unique per customer (derived from chosen admin label + domain) so multiple customers can coexist on a shared zone. The agent never synthesises subdomains from free text — every label comes from a component submission. Authentication is OAuth-only via `tunnel-login`; on first success the device records an `account-binding.json` so any later cert rotation under a different Cloudflare account is detected and refused. `cf-verify` is the non-mutating audit; `cf-rebuild` is the deterministic recovery.
+Each installation has its own Cloudflare account, owned by the user, accessed via `cert.pem` from `tunnel-login` (OAuth — the only auth path). The bound account is the entire universe of routable zones; the agent has absolute authority over it (add, delete, restructure). Anything on the account that doesn't belong to the user's current intended state is pollution — `cf-rebuild` deletes it, with `reason=no-intent` refusal when intent is unstated. `cloudflare-setup` is the onboarding orchestrator: a UI-driven state machine that surfaces what's on the account, asks the user to pick a domain, requires explicit cleanup confirmation for any pollution, then creates the tunnel. `cf-verify` is the non-mutating audit (account state + device state + default pollution view). The device records `account-binding.json` on first login so cert rotation under a different account is detected and refused — that's the only inherited identity check.
 
 ## When to activate
 

package/payload/platform/plugins/cloudflare/mcp/__tests__/auth-binding.test.ts

@@ -50,7 +50,6 @@ beforeEach(() => {
   writeBrand({
     productName: "Maxy",
     configDir: ".maxy",
-    cloudflare: { zones: ["maxy.bot"] },
   });
 });