mcp-scraper 0.72.6 → 0.73.0

package/CHANGELOG.md

@@ -4,6 +4,36 @@ All notable changes to MCP Scraper are documented here. The format is based on [
 
 ## [Unreleased]
 
+## [0.73.0] - 2026-08-27
+
+### Added
+
+- Added a backend-owned, 142-operation analytics REST contract with generated Web validators and exact MCP/SDK projections for enhanced confirmed journeys.
+- Added consent-governed identity promotion for verified forms and authenticated accounts, including same-Site anonymous-history backfill, encrypted masked identity profiles, audited single-person reads, and deletion tombstones that prevent silent recreation.
+- Added date-bounded Journey reporting for summaries, common pages, events, acquisition and referrers, sessions, and chronology, with confirmed and best-guess totals kept separate.
+- Added first-party MCP Scraper subscription revenue outcomes through the ordinary versioned conversion-rule pipeline without copying billing contact data into analytics.
+- Added real PostgreSQL integration coverage for tenant isolation, consent withdrawal, deletion, no-resurrection, migration replay, and provider activation eligibility.
+
+### Changed
+
+- Consolidated Scheduler, Web, and Memory source ownership into this product repository under isolated `apps/` project roots while retaining the public SDK as the only separate client repository.
+- Aligned the main server with `mcpscraper-memory-tools` 1.16.0 so nested Mastra builds cannot inherit the server's former package versions from the shared repository root.
+- Expanded the public MCP catalog to 331 tools, including distinct confirmed/best-guess Journey listing and complete enhanced Journey reads with bounded `from` and `to` windows.
+- Browser and embedded-form conversions now preserve the conversion record while allowing provider activation only for a confirmed person with current advertising-measurement consent.
+
+### Fixed
+
+- Kept nested `apps/` source available to Vercel so each consolidated project can build from its configured monorepo root.
+- Prevented cross-Site identity, privacy, and Journey references; removed subject-specific export/provider payload remnants on withdrawal or deletion; and normalized unknown/deleted references to zero-discovery responses.
+- Normalized PostgreSQL timestamp projections and reconciled consent-withdrawal responses with the strict public REST contract.
+- Made consented login and first-party form resolution self-initialize an approved Site-scoped primary identity graph, so older Sites persist stable person references and masked profiles without manual namespace setup.
+
+## [0.72.7] - 2026-08-27
+
+### Fixed
+
+- Completed Gmail Memory imports now report recovered historical errors as resolved and derive attachment indexing from the verified nested file-asset receipts, keeping counts and sample states consistent with durable Memory state.
+
 ## [0.72.6] - 2026-08-27
 
 ### Fixed
@@ -1278,7 +1308,9 @@ All notable changes to MCP Scraper are documented here. The format is based on [
 - Write actions remain unavailable until the account owner explicitly enables them.
 - Provider-specific connection data is normalized into one agent-facing contract.
 
-[Unreleased]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.6...HEAD
+[Unreleased]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.73.0...HEAD
+[0.73.0]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.7...v0.73.0
+[0.72.7]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.6...v0.72.7
 [0.72.6]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.5...v0.72.6
 [0.72.5]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.4...v0.72.5
 [0.72.4]: https://github.com/VilovietaSEO/mcp-scraper/compare/v0.72.3...v0.72.4

package/README.md

@@ -2,6 +2,20 @@
 
 MCP Scraper is an MCP server for live web intelligence tools backed by `https://mcpscraper.dev`.
 
+## Product repository
+
+This repository owns the complete private product runtime while preserving independent service and
+deployment boundaries:
+
+- `/` — primary MCP/API server and public `mcp-scraper` package;
+- `apps/scheduler` — private scheduled execution and orchestration service;
+- `apps/web` — private Next.js product and marketing application;
+- `apps/memory` — hosted Memory service plus its independently publishable DB, Tools, and stdio packages.
+
+`mcpscraper-sdk` remains the only separate public-client repository. See [`apps/README.md`](apps/README.md)
+and [`product-topology.json`](product-topology.json) for exact ownership, source receipts, release units,
+deployment roots, and verification commands.
+
 ## Install
 
 Use the MCPB Desktop Extension for the branded Claude Desktop install, or use the npm package from any MCP client that can run stdio commands.
@@ -159,7 +173,7 @@ Build the branded one-click bundle:
 npm run build:mcpb
 ```
 
-The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.72.6`, SHA-256 `dfcf80fa18e14a314d19d459f6c3e1fa0ba0f03eb8eca6f35ca0b0fae8e267ec`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, API-key configuration field, and manually curated current-release message from the bundle manifest.
+The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.73.0`, SHA-256 `570d31a9b16833b89de8ea5296e5d97b3d4696336c94651739b716a0bda81ce6`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, API-key configuration field, and manually curated current-release message from the bundle manifest.
 
 The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
 
@@ -321,7 +335,7 @@ The `mcp-scraper` server (and the MCPB bundle, which runs it) exposes both secti
 
 All MCP tools return `structuredContent` with the IDs, URLs, CSV paths, transcripts, browser session handles, replay paths, artifacts, recipe fields, or blueprint fields needed by the next step, plus readable text content for compatibility. Runtime `tools/list` omits output schemas so strict clients can register the complete catalog; the generated developer manifest retains every canonical output schema for validation and typed SDK generation. All tools carry MCP annotations; file-writing tools such as replay downloads and annotations state their filesystem side effects.
 
-The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 330 tools: 229 scraper, browser, workflow, billing, and connected-service tools plus 101 durable-memory tools. The scraper-side inventory includes complete Gmail selection, message, attachment, export, bulk-action, and Memory-import workflows; durable PAA starts and status; rendered site-content similarity; governed Local Sourcebook and Transparent Commons workflows; direct site-export reads; Editorial Reading Room and News Publisher templates; and production X-Ray analytics, attribution, CRM policy and receipt, campaign, export, and scheduled-report tools. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
+The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 331 tools: 230 scraper, browser, workflow, billing, and connected-service tools plus 101 durable-memory tools. The scraper-side inventory includes complete Gmail selection, message, attachment, export, bulk-action, and Memory-import workflows; durable PAA starts and status; rendered site-content similarity; governed Local Sourcebook and Transparent Commons workflows; direct site-export reads; Editorial Reading Room and News Publisher templates; and production X-Ray analytics, attribution, CRM policy and receipt, campaign, export, and scheduled-report tools. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
 
 For contract parity, stdio and MCPB memory calls invoke the matching public tool on the hosted MCP Scraper `/mcp` endpoint. The hosted aggregate runtime owns MCP Scraper-specific billing, scheduling, credential, and in-process cutover policy; its internal `/memory/mcp-call` bridge is a fallback to the standalone memory service, not the public stdio execution path. Direct `mcp-memory` OAuth and stdio clients continue to use `memory.mcpscraper.dev` and must be verified as a separate dependent release surface.
 

package/dist/{analytics-repository-6YLRSWYO.js → analytics-repository-MSLJBHLU.js}

@@ -12,11 +12,14 @@ import {
   analyticsEventCounts,
   analyticsForecast,
   analyticsHealth,
+  analyticsIdentityPromotionAllowed,
+  analyticsIdentityResolutionAllowed,
   analyticsOverview,
   analyticsPaths,
   analyticsTimeseries,
   archiveAnalyticsActivationDestination,
   archiveAnalyticsCampaignLink,
+  backfillAnalyticsConfirmedHistory,
   claimAnalyticsCrmImportRows,
   claimAnalyticsFormDeliveryJobs,
   closeAnalyticsPool,
@@ -33,6 +36,7 @@ import {
   deferAnalyticsCrmImportRow,
   deferAnalyticsFormDelivery,
   deleteAnalyticsSite,
+  enrichAnalyticsExistingCrmIdentity,
   getAnalyticsActivationDestinationConnectionRef,
   getAnalyticsPersonJourney,
   getAnalyticsPool,
@@ -41,6 +45,7 @@ import {
   ingestAnalyticsEvents,
   isAnalyticsFormPlacementApproved,
   linkAnalyticsFormIdentity,
+  linkAnalyticsIdentityInTransaction,
   listAnalyticsActivationDestinations,
   listAnalyticsActivationReceipts,
   listAnalyticsCampaignLinks,
@@ -57,6 +62,7 @@ import {
   normalizeObservedHostname,
   pollAnalyticsActivationDiagnostics,
   prepareAnalyticsLinkerIssue,
+  projectAnalyticsPixelEventConsent,
   queueAnalyticsActivation,
   queueAnalyticsFormDelivery,
   recordAnalyticsFormSubmission,
@@ -66,6 +72,7 @@ import {
   requireAnalyticsAccess,
   requireAnalyticsEditor,
   resolveAnalyticsCampaignLink,
+  resolveAnalyticsConfirmedActivationIdentity,
   retryAnalyticsActivationJob,
   sanitizeAnalyticsProperties,
   sanitizeClickIds,
@@ -77,7 +84,7 @@ import {
   updateAnalyticsPixel,
   upsertAnalyticsAdSpend,
   upsertAnalyticsHostGroup
-} from "./chunk-SR7GSLEA.js";
+} from "./chunk-5EWXRBEH.js";
 import "./chunk-PJEEKOUM.js";
 import "./chunk-WEFPBAAG.js";
 export {
@@ -94,11 +101,14 @@ export {
   analyticsEventCounts,
   analyticsForecast,
   analyticsHealth,
+  analyticsIdentityPromotionAllowed,
+  analyticsIdentityResolutionAllowed,
   analyticsOverview,
   analyticsPaths,
   analyticsTimeseries,
   archiveAnalyticsActivationDestination,
   archiveAnalyticsCampaignLink,
+  backfillAnalyticsConfirmedHistory,
   claimAnalyticsCrmImportRows,
   claimAnalyticsFormDeliveryJobs,
   closeAnalyticsPool,
@@ -115,6 +125,7 @@ export {
   deferAnalyticsCrmImportRow,
   deferAnalyticsFormDelivery,
   deleteAnalyticsSite,
+  enrichAnalyticsExistingCrmIdentity,
   getAnalyticsActivationDestinationConnectionRef,
   getAnalyticsPersonJourney,
   getAnalyticsPool,
@@ -123,6 +134,7 @@ export {
   ingestAnalyticsEvents,
   isAnalyticsFormPlacementApproved,
   linkAnalyticsFormIdentity,
+  linkAnalyticsIdentityInTransaction,
   listAnalyticsActivationDestinations,
   listAnalyticsActivationReceipts,
   listAnalyticsCampaignLinks,
@@ -139,6 +151,7 @@ export {
   normalizeObservedHostname,
   pollAnalyticsActivationDiagnostics,
   prepareAnalyticsLinkerIssue,
+  projectAnalyticsPixelEventConsent,
   queueAnalyticsActivation,
   queueAnalyticsFormDelivery,
   recordAnalyticsFormSubmission,
@@ -148,6 +161,7 @@ export {
   requireAnalyticsAccess,
   requireAnalyticsEditor,
   resolveAnalyticsCampaignLink,
+  resolveAnalyticsConfirmedActivationIdentity,
   retryAnalyticsActivationJob,
   sanitizeAnalyticsProperties,
   sanitizeClickIds,

package/dist/bin/api-server.js

@@ -17,15 +17,15 @@ loadDotEnv();
 async function main() {
   const [{ serve }, { app }, { startWorker }, { migrate }] = await Promise.all([
     import("@hono/node-server"),
-    import("../server-LQ6TMI5W.js"),
-    import("../worker-JQORPCFQ.js"),
+    import("../server-N6MCRSQW.js"),
+    import("../worker-CBK5DWEE.js"),
     import("../db-566MOHHD.js")
   ]);
   const PORT = parseInt(process.env.PORT ?? "3001");
   try {
     await migrate();
     if (process.env.ANALYTICS_DATABASE_URL) {
-      const { migrateAnalytics } = await import("../analytics-repository-6YLRSWYO.js");
+      const { migrateAnalytics } = await import("../analytics-repository-MSLJBHLU.js");
       await migrateAnalytics();
     }
     startWorker();

package/dist/bin/mcp-scraper-cli.js

@@ -13,7 +13,7 @@ import "../chunk-VXLU74YZ.js";
 import "../chunk-GGZEC22A.js";
 import {
   PACKAGE_VERSION
-} from "../chunk-D5CX3WHG.js";
+} from "../chunk-JIWGG7Z4.js";
 
 // src/cli/human-cli.ts
 import { Command } from "commander";

package/dist/bin/mcp-scraper-install.js

@@ -1,10 +1,10 @@
 #!/usr/bin/env node
 import {
   renderInstallTerminal
-} from "../chunk-RAFQEPJ4.js";
+} from "../chunk-Q2J23DHY.js";
 import {
   PACKAGE_VERSION
-} from "../chunk-D5CX3WHG.js";
+} from "../chunk-JIWGG7Z4.js";
 
 // bin/mcp-scraper-install.ts
 var noColor = process.argv.includes("--no-color") || process.env.NO_COLOR !== void 0 || process.env.FORCE_COLOR === "0" || !process.stdout.isTTY;

package/dist/bin/mcp-stdio-server.js

@@ -13,7 +13,7 @@ import {
   registerScheduledResultsMcpTools,
   registerSerpIntelligenceCaptureTools,
   resolveDeploymentProfile
-} from "../chunk-XRH2DUAQ.js";
+} from "../chunk-KRIIVDV2.js";
 import "../chunk-PGJQDMC2.js";
 import "../chunk-T3MZISOF.js";
 import "../chunk-7RQULQF3.js";
@@ -21,14 +21,14 @@ import "../chunk-OM7HVEJ3.js";
 import "../chunk-CXY5WV45.js";
 import {
   renderInstallTerminal
-} from "../chunk-RAFQEPJ4.js";
+} from "../chunk-Q2J23DHY.js";
 import "../chunk-VXLU74YZ.js";
 import "../chunk-GGZEC22A.js";
 import {
   PACKAGE_VERSION
-} from "../chunk-D5CX3WHG.js";
+} from "../chunk-JIWGG7Z4.js";
 import "../chunk-DNM65UCK.js";
-import "../chunk-FVL4GUTP.js";
+import "../chunk-EQJRPDUS.js";
 import "../chunk-WEFPBAAG.js";
 
 // bin/mcp-stdio-server.ts