mcp-scraper 0.84.0 → 0.85.0

package/CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to MCP Scraper are documented here. The format is based on [
 
 ## [Unreleased]
 
+## [0.85.0] - 2026-09-01
+
+### Changed
+
+- Corrected X-Ray's public contract to a provider-neutral measurement foundation: the Pixel, seven observed attribution models, structured post-purchase surveys, explicit Observed + Reported impact, privacy lifecycle, and truthful unavailable view-evidence states remain supported.
+- Advanced the X-Ray install protocol to 3.0.0 and removed provider authorization and revenue-source continuations from generated install guidance.
+
+### Removed
+
+- Removed the four revenue-source setup MCP tools, five REST operations, revenue-setup ResourceTemplate, Stripe dashboard workflow, and deployable Stripe, Salesforce, and Pipedrive Nango registrations until each provider has complete production receipts.
+
+### Fixed
+
+- Routed the public X-Ray product page and nested attribution and SaaS guides through the canonical `mcpscraper.dev` host instead of leaving the successfully deployed web routes unreachable behind the API project.
+
 ## [0.84.0] - 2026-09-01
 
 ### Added

package/README.md

@@ -175,7 +175,7 @@ Build the branded one-click bundle:
 npm run build:mcpb
 ```
 
-The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.84.0`, SHA-256 `488d73bb3e41439c6b3676a25850ab4e9b883c29986b59732474ac3bfd48f79b`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, API-key configuration field, and manually curated current-release message from the bundle manifest.
+The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.85.0`, SHA-256 `2410fb6216af649dba24e62df9fa438b85e02c5ed2d3610f9e630314c0a0c2fb`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, API-key configuration field, and manually curated current-release message from the bundle manifest.
 
 The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
 
@@ -337,7 +337,7 @@ The `mcp-scraper` server (and the MCPB bundle, which runs it) exposes both secti
 
 All MCP tools return `structuredContent` with the IDs, URLs, CSV paths, transcripts, browser session handles, replay paths, artifacts, recipe fields, or blueprint fields needed by the next step, plus readable text content for compatibility. Runtime `tools/list` omits output schemas so strict clients can register the complete catalog; the generated developer manifest retains every canonical output schema for validation and typed SDK generation. All tools carry MCP annotations; file-writing tools such as replay downloads and annotations state their filesystem side effects.
 
-The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 373 tools: 272 scraper, browser, workflow, billing, connected-service, and personal-assistant tools plus 101 durable-memory tools. The scraper-side inventory includes complete Gmail selection, message, attachment, export, bulk-action, and Memory-import workflows; governed personal-assistant commands, messaging, approvals, grants, number setup, and execution readback; durable PAA starts and status; rendered site-content similarity; governed Local Sourcebook and Transparent Commons workflows; direct site-export reads; Editorial Reading Room and News Publisher templates; and production X-Ray setup, analytics, attribution, authoritative SaaS revenue, post-purchase surveys, reported impact, view-evidence status, CRM policy and receipt, campaign, export, and scheduled-report tools. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
+The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 369 tools: 268 scraper, browser, workflow, billing, connected-service, and personal-assistant tools plus 101 durable-memory tools. The scraper-side inventory includes complete Gmail selection, message, attachment, export, bulk-action, and Memory-import workflows; governed personal-assistant commands, messaging, approvals, grants, number setup, and execution readback; durable PAA starts and status; rendered site-content similarity; governed Local Sourcebook and Transparent Commons workflows; direct site-export reads; Editorial Reading Room and News Publisher templates; and production X-Ray setup, analytics, seven-model attribution, structured post-purchase surveys, reported impact, truthful view-evidence status, CRM policy and receipt, campaign, export, and scheduled-report tools. Provider setup remains absent until its authorization, ingestion, reconciliation, privacy, canary, cleanup, and deployment receipts are complete. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
 
 For contract parity, stdio and MCPB memory calls invoke the matching public tool on the hosted MCP Scraper `/mcp` endpoint. The hosted aggregate runtime owns MCP Scraper-specific billing, scheduling, credential, and in-process cutover policy; its internal `/memory/mcp-call` bridge is a fallback to the standalone memory service, not the public stdio execution path. Direct `mcp-memory` OAuth and stdio clients continue to use `memory.mcpscraper.dev` and must be verified as a separate dependent release surface.
 

package/dist/bin/api-server.js

@@ -17,8 +17,8 @@ loadDotEnv();
 async function main() {
   const [{ serve }, { app, personalAssistantProductionStartup }, { startWorker }, { migrate }] = await Promise.all([
     import("@hono/node-server"),
-    import("../server-3F3JMHBF.js"),
-    import("../worker-VTFGHWZL.js"),
+    import("../server-EOPETL7Q.js"),
+    import("../worker-5A375BGN.js"),
     import("../db-Z34LPZNR.js")
   ]);
   const PORT = parseInt(process.env.PORT ?? "3001");

package/dist/bin/mcp-scraper-cli.js

@@ -6,7 +6,7 @@ import {
   openWorkflowReport,
   runWorkflow,
   workflowOutputBaseDir
-} from "../chunk-LHIWX7RM.js";
+} from "../chunk-6IOGTFNS.js";
 import "../chunk-RJ7JVYKU.js";
 import "../chunk-OWF2JJKN.js";
 import {
@@ -15,11 +15,11 @@ import {
   XRAY_INSTALL_PROTOCOL_VERSION,
   renderXraySetupPrompt,
   xrayHostRecipe
-} from "../chunk-4S6HPHPB.js";
+} from "../chunk-P7FWOMU7.js";
 import "../chunk-GGZEC22A.js";
 import {
   PACKAGE_VERSION
-} from "../chunk-OUTE3755.js";
+} from "../chunk-RHAOKXQK.js";
 
 // src/cli/human-cli.ts
 import { Command } from "commander";

package/dist/bin/mcp-scraper-core.js

@@ -1,19 +1,19 @@
 #!/usr/bin/env node
 import {
   runMcpScraperStdio
-} from "../chunk-WDWWCNMW.js";
-import "../chunk-U77X4NIR.js";
+} from "../chunk-2ZDYUTOZ.js";
+import "../chunk-W4IY6CB4.js";
 import "../chunk-PGJQDMC2.js";
 import "../chunk-DNM65UCK.js";
 import "../chunk-T3MZISOF.js";
 import "../chunk-6HAV7LCE.js";
-import "../chunk-UM7DCY6W.js";
+import "../chunk-X5HRAAQM.js";
 import "../chunk-OM7HVEJ3.js";
 import "../chunk-OPQIGAFB.js";
-import "../chunk-NLWBM55O.js";
-import "../chunk-4S6HPHPB.js";
+import "../chunk-WN7N4ESQ.js";
+import "../chunk-P7FWOMU7.js";
 import "../chunk-GGZEC22A.js";
-import "../chunk-OUTE3755.js";
+import "../chunk-RHAOKXQK.js";
 import "../chunk-YXNDOQXN.js";
 
 // src/mcp/thorbit-restricted-tool-surface.ts

package/dist/bin/mcp-scraper-install.js

@@ -1,10 +1,10 @@
 #!/usr/bin/env node
 import {
   renderInstallTerminal
-} from "../chunk-NLWBM55O.js";
+} from "../chunk-WN7N4ESQ.js";
 import {
   PACKAGE_VERSION
-} from "../chunk-OUTE3755.js";
+} from "../chunk-RHAOKXQK.js";
 
 // bin/mcp-scraper-install.ts
 var noColor = process.argv.includes("--no-color") || process.env.NO_COLOR !== void 0 || process.env.FORCE_COLOR === "0" || !process.stdout.isTTY;

package/dist/bin/mcp-stdio-server.js

@@ -1,19 +1,19 @@
 #!/usr/bin/env node
 import {
   runMcpScraperStdio
-} from "../chunk-WDWWCNMW.js";
-import "../chunk-U77X4NIR.js";
+} from "../chunk-2ZDYUTOZ.js";
+import "../chunk-W4IY6CB4.js";
 import "../chunk-PGJQDMC2.js";
 import "../chunk-DNM65UCK.js";
 import "../chunk-T3MZISOF.js";
 import "../chunk-6HAV7LCE.js";
-import "../chunk-UM7DCY6W.js";
+import "../chunk-X5HRAAQM.js";
 import "../chunk-OM7HVEJ3.js";
 import "../chunk-OPQIGAFB.js";
-import "../chunk-NLWBM55O.js";
-import "../chunk-4S6HPHPB.js";
+import "../chunk-WN7N4ESQ.js";
+import "../chunk-P7FWOMU7.js";
 import "../chunk-GGZEC22A.js";
-import "../chunk-OUTE3755.js";
+import "../chunk-RHAOKXQK.js";
 import "../chunk-YXNDOQXN.js";
 
 // bin/mcp-stdio-server.ts

package/dist/{chunk-WDWWCNMW.js → chunk-2ZDYUTOZ.js}

@@ -12,13 +12,13 @@ import {
   registerScheduledResultsMcpTools,
   registerSerpIntelligenceCaptureTools,
   resolveDeploymentProfile
-} from "./chunk-U77X4NIR.js";
+} from "./chunk-W4IY6CB4.js";
 import {
   renderInstallTerminal
-} from "./chunk-NLWBM55O.js";
+} from "./chunk-WN7N4ESQ.js";
 import {
   PACKAGE_VERSION
-} from "./chunk-OUTE3755.js";
+} from "./chunk-RHAOKXQK.js";
 
 // src/mcp/stdio-runtime.ts
 import { readFileSync } from "fs";

package/dist/{chunk-LHIWX7RM.js → chunk-6IOGTFNS.js}

@@ -11,7 +11,7 @@ import {
   extractContactEvidence,
   extractSerpEmailEvidence,
   selectPrimaryEmail
-} from "./chunk-4S6HPHPB.js";
+} from "./chunk-P7FWOMU7.js";
 
 // src/lib/slugify.ts
 function slugify(s) {

package/dist/{chunk-ZB55UMMF.js → chunk-C7OKENCR.js}

@@ -7,7 +7,7 @@ import {
 } from "./chunk-DNM65UCK.js";
 import {
   LedgerOperation
-} from "./chunk-UM7DCY6W.js";
+} from "./chunk-X5HRAAQM.js";
 import {
   CREDIT_LOT_TTL,
   getDb,

package/dist/{chunk-4S6HPHPB.js → chunk-P7FWOMU7.js}

@@ -147,9 +147,8 @@ function extractContactEvidence(html, pageUrl) {
 
 // src/mcp/xray-install-contract.ts
 import { createHash } from "crypto";
-var XRAY_INSTALL_PROTOCOL_VERSION = "2.1.0";
+var XRAY_INSTALL_PROTOCOL_VERSION = "3.0.0";
 var XRAY_SETUP_TOOL = "analytics_setup_site";
-var XRAY_REVENUE_SETUP_TOOL = "analytics_setup_revenue_source";
 var XRAY_TRIGGER_PHRASE = "Install X-Ray";
 var XRAY_CANONICAL_GOAL = "Install X-Ray and tell me what drives customers";
 var CONTINUATION = `Call ${XRAY_SETUP_TOOL} with { "startUrl": "<site URL>", "goal": "${XRAY_CANONICAL_GOAL}" }; on every later call, reuse the returned setupId and revision.`;
@@ -179,10 +178,8 @@ var XRAY_INSTALL_CONTRACT_PAYLOAD = {
   },
   facade: {
     tool: XRAY_SETUP_TOOL,
-    saasRevenueTool: XRAY_REVENUE_SETUP_TOOL,
     initialCall: { startUrl: "<site URL>", goal: XRAY_CANONICAL_GOAL },
-    continuationRule: "Treat setupId plus revision as the durable continuation handle. Reuse both after approval, runtime publication, verification, retry, or return.",
-    saasContinuationRule: `When the Site setup reaches outcome_connection_required for SaaS, continue through ${XRAY_REVENUE_SETUP_TOOL} with an already-authorized connection reference. Never request or accept provider credentials.`
+    continuationRule: "Treat setupId plus revision as the durable continuation handle. Reuse both after approval, runtime publication, verification, retry, or return."
   },
   topologies: {
     local_stdio: {
@@ -294,7 +291,7 @@ var XRAY_INSTALL_CONTRACT_PAYLOAD = {
     "Infer the business model and customer outcome, then propose one privacy-safe measurement plan.",
     "If the Pixel is absent, return the canonical one-time tag and exact uncovered surfaces; do not claim ongoing control there.",
     "Publish an approved measurement revision as a digest-bound runtime manifest; source or repository access is not required.",
-    `For SaaS, continue outcome readiness through ${XRAY_REVENUE_SETUP_TOOL}: prepare an authorized connection, apply one exact revision, and verify a non-customer provider receipt.`,
+    "For SaaS, verify the measurement foundation: global tag coverage, ad-source capture, lifecycle events, currency and timezone, consent, and observed outcome events.",
     "Optionally publish a structured post-purchase survey after approved disclosure evidence; reported influence remains separate from deterministic touches.",
     "Compare observed attribution with customer-reported influence only with explicit weights totaling 1,000,000 micros and a named methodology version.",
     "Keep view-through reporting unavailable until event-level provider evidence can join to an eligible X-Ray subject; aggregate impressions never qualify.",
@@ -310,10 +307,10 @@ var XRAY_INSTALL_CONTRACT_PAYLOAD = {
     "Never treat a CTA click as a customer, sale, or revenue event without verified outcome evidence.",
     "Never claim full-domain installation while an in-scope route or renderer is missing, excluded without approval, or unverifiable.",
     "Never treat event-name presence as verification; require persisted definition ID and version evidence.",
-    "Never accept payment-provider credentials, survey invitation tokens, or exchange codes through MCP inputs or prompts.",
+    "Never accept survey invitation tokens or exchange codes through MCP inputs or prompts.",
     "Never treat customer-reported influence as identity, CRM mutation authority, ad-activation evidence, or a deterministic touch."
   ],
-  terminalStates: ["complete", "tag_required", "input_required", "approval_required", "runtime_convergence_required", "outcome_connection_required", "not_ready", "error"]
+  terminalStates: ["complete", "tag_required", "input_required", "approval_required", "runtime_convergence_required", "not_ready", "error"]
 };
 function canonicalJson(value) {
   if (Array.isArray(value)) return `[${value.map(canonicalJson).join(",")}]`;
@@ -343,7 +340,7 @@ function renderXraySetupPrompt() {
     "",
     `Use ${XRAY_SETUP_TOOL} as the single conversational setup facade. Ask for the target site URL if it is missing, then call it with the goal \u201C${XRAY_CANONICAL_GOAL}\u201D.`,
     "",
-    `Continue the same setup with its setupId and revision. Confirm one global Pixel across all public rendering surfaces; present one measurement plan; publish it as a digest-bound runtime manifest after exact-revision approval; verify manifest and rule versions; then return first customer-journey answers or explicit readiness thresholds. For SaaS outcome readiness, continue through ${XRAY_REVENUE_SETUP_TOOL} with an already-authorized provider connection and verify a non-customer receipt. GitHub or repository access is not required after the tag is installed.`,
+    "Continue the same setup with its setupId and revision. Confirm one global Pixel across all public rendering surfaces; present one measurement plan; publish it as a digest-bound runtime manifest after exact-revision approval; verify manifest and rule versions; then return first customer-journey answers or explicit readiness thresholds. For SaaS, verify the provider-neutral measurement foundation. Browser events never prove paid revenue. GitHub or repository access is not required after the tag is installed.",
     "",
     "Do not request pasted credentials, invent tracking code, silently activate measurement, call ad platforms directly, or claim customers/revenue from CTA clicks."
   ].join("\n");

package/dist/{chunk-OUTE3755.js → chunk-RHAOKXQK.js}

@@ -1,5 +1,5 @@
 // src/version.ts
-var PACKAGE_VERSION = "0.84.0";
+var PACKAGE_VERSION = "0.85.0";
 
 export {
   PACKAGE_VERSION

package/dist/{chunk-4KFD2WXG.js → chunk-W3SGF7GX.js}

@@ -19,7 +19,7 @@ import {
   LedgerOperation,
   MC_PER_CREDIT,
   calculateConnectedUsageCharge
-} from "./chunk-UM7DCY6W.js";
+} from "./chunk-X5HRAAQM.js";
 import {
   recordVendorUsage,
   runWithCostContext
@@ -528,6 +528,11 @@ var NANGO_TOOL_CALL_TIMEOUT_MS = 24e4;
 var NANGO_CONNECTION_PAGE_SIZE = 100;
 var NANGO_CONNECTION_MAX_PAGES = 20;
 var PROVIDER_IDENTITY_RETRY_MS = 6 * 60 * 60 * 1e3;
+var WITHDRAWN_NANGO_PROVIDER_CONFIG_KEYS = /* @__PURE__ */ new Set([
+  "stripe_revenue",
+  "salesforce",
+  "pipedrive"
+]);
 var MAIN_INTEGRATION_CONTRACT_VERSION = "2026-07-16.1";
 var MAIN_INTEGRATION_CONTRACT_HASH = "f037859ec80c2ea2fac6e4a67c1a59f3200dd22d3642c97789b9bb3236723d3e";
 var MainNangoTransportError = class extends Error {
@@ -544,6 +549,18 @@ var MainNangoTransportError = class extends Error {
     this.retryAfterMs = retryAfterMs;
   }
 };
+function isWithdrawnNangoProvider(providerConfigKey) {
+  return typeof providerConfigKey === "string" && WITHDRAWN_NANGO_PROVIDER_CONFIG_KEYS.has(providerConfigKey.normalize("NFKC").trim().toLowerCase());
+}
+function assertNangoProviderAvailable(providerConfigKey) {
+  if (!isWithdrawnNangoProvider(providerConfigKey)) return;
+  throw new MainNangoTransportError(
+    "This provider integration is not currently available.",
+    409,
+    "provider_unavailable",
+    false
+  );
+}
 function nangoClientErrorText(error) {
   if (error instanceof Error) return error.message;
   if (typeof error === "string") return error;
@@ -907,12 +924,13 @@ async function discoverOwnedNangoConnections(identity, options = {}) {
   }
   const discovered = rows.filter((row) => connectionIdentityMatches(row, identity)).map(discoveredConnection).filter((connection) => connection !== null);
   const reconciled = await reconcileDiscoveredNangoConnections(identity, discovered);
-  if (options.resolveProviderIdentity === false) return reconciled;
+  const visibleConnections = reconciled.filter((connection) => !isWithdrawnNangoProvider(connection.providerConfigKey));
+  if (options.resolveProviderIdentity === false) return visibleConnections;
   const remoteByKey = new Map(discovered.map((connection) => [
     `${connection.providerConfigKey}\0${connection.upstreamConnectionId}`,
     connection
   ]));
-  await Promise.all(reconciled.map(async (connection) => {
+  await Promise.all(visibleConnections.map(async (connection) => {
     if (connection.lifecycleStatus !== "connected" || !connection.upstreamConnectionId) return;
     const remote = remoteByKey.get(`${connection.providerConfigKey}\0${connection.upstreamConnectionId}`);
     if (!remote) return;
@@ -935,7 +953,7 @@ async function discoverOwnedNangoConnections(identity, options = {}) {
       sourceUpdatedAt
     });
   }));
-  return reconcileDiscoveredNangoConnections(identity, discovered);
+  return (await reconcileDiscoveredNangoConnections(identity, discovered)).filter((connection) => !isWithdrawnNangoProvider(connection.providerConfigKey));
 }
 async function listNangoCatalogDirect() {
   const [integrationsPayload, providersPayload] = await Promise.all([
@@ -952,7 +970,7 @@ async function listNangoCatalogDirect() {
     const row = asRecord(value);
     const id = text(row?.unique_key) ?? text(row?.id);
     const provider = text(row?.provider);
-    if (!id || !provider) return [];
+    if (!id || !provider || isWithdrawnNangoProvider(id)) return [];
     const definition = providers.get(provider);
     return [{
       id,
@@ -986,6 +1004,7 @@ function sessionPayload(value) {
   return { connectLink, expiresAt: text(data?.expires_at) };
 }
 async function createNangoSessionDirect(identity, providerConfigKey) {
+  assertNangoProviderAvailable(providerConfigKey);
   const defaults = connectDefaults(providerConfigKey);
   return sessionPayload(await nangoRequest("/connect/sessions", {
     method: "POST",
@@ -1010,7 +1029,7 @@ async function createNangoReconnectSessionDirect(identity, connectionId) {
   }));
 }
 async function deleteNangoConnectionDirect(identity, connectionId) {
-  const connection = await requireOwnedActiveConnection(identity, connectionId, true);
+  const connection = await requireOwnedNangoConnectionForCleanup(identity, connectionId);
   try {
     await nangoRequest(`/connections/${encodeURIComponent(connection.upstreamConnectionId)}`, {
       method: "DELETE",
@@ -1020,11 +1039,16 @@ async function deleteNangoConnectionDirect(identity, connectionId) {
     if (!(error instanceof MainNangoTransportError) || error.status !== 404) throw error;
   }
 }
-async function requireOwnedActiveConnection(identity, connectionId, allowNeedsReauth = false) {
+async function requireOwnedNangoConnectionForCleanup(identity, connectionId) {
   const connection = await getOwnedServiceConnection(identity, connectionId);
   if (!connection || connection.transport !== "nango" || !connection.upstreamConnectionId) {
     throw new MainNangoTransportError("Service connection not found.", 404, "connection_not_found", false);
   }
+  return connection;
+}
+async function requireOwnedActiveConnection(identity, connectionId, allowNeedsReauth = false) {
+  const connection = await requireOwnedNangoConnectionForCleanup(identity, connectionId);
+  assertNangoProviderAvailable(connection.providerConfigKey);
   if (connection.lifecycleStatus !== "connected" && !(allowNeedsReauth && connection.lifecycleStatus === "needs_reauth")) {
     throw new MainNangoTransportError("Service connection requires reconnection.", 409, "connection_inactive", false);
   }
@@ -2028,6 +2052,8 @@ export {
   settleConnectedUsage,
   listConnectedUsageHistory,
   MainNangoTransportError,
+  isWithdrawnNangoProvider,
+  assertNangoProviderAvailable,
   discoverOwnedNangoConnections,
   listNangoCatalogDirect,
   createNangoSessionDirect,