mcp-scraper 0.57.2 → 0.58.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/CHANGELOG.md +19 -0
  2. package/README.md +2 -2
  3. package/dist/bin/api-server.cjs +151 -12
  4. package/dist/bin/api-server.cjs.map +1 -1
  5. package/dist/bin/api-server.js +2 -2
  6. package/dist/bin/mcp-scraper-cli.cjs +1 -1
  7. package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
  8. package/dist/bin/mcp-scraper-cli.js +1 -1
  9. package/dist/bin/mcp-scraper-install.cjs +2 -2
  10. package/dist/bin/mcp-scraper-install.cjs.map +1 -1
  11. package/dist/bin/mcp-scraper-install.js +2 -2
  12. package/dist/bin/mcp-stdio-server.cjs +33 -6
  13. package/dist/bin/mcp-stdio-server.cjs.map +1 -1
  14. package/dist/bin/mcp-stdio-server.js +4 -4
  15. package/dist/{chunk-R6CO77BN.js → chunk-2U2ZDDSI.js} +3 -3
  16. package/dist/{chunk-R6CO77BN.js.map → chunk-2U2ZDDSI.js.map} +1 -1
  17. package/dist/chunk-EJ6KU4PC.js +7 -0
  18. package/dist/chunk-EJ6KU4PC.js.map +1 -0
  19. package/dist/{chunk-HU4R6BP2.js → chunk-IMBMDZUO.js} +2 -2
  20. package/dist/{chunk-HU4R6BP2.js.map → chunk-IMBMDZUO.js.map} +1 -1
  21. package/dist/{chunk-5XYBKE3X.js → chunk-KMGRM4FL.js} +2 -2
  22. package/dist/{chunk-67JUTUQD.js → chunk-P32KEY3D.js} +35 -7
  23. package/dist/chunk-P32KEY3D.js.map +1 -0
  24. package/dist/{extract-bundle-H7JEELY7.js → extract-bundle-XRRN6NAZ.js} +3 -3
  25. package/dist/{server-OZJXK7AB.js → server-J4YGBOTS.js} +126 -13
  26. package/dist/server-J4YGBOTS.js.map +1 -0
  27. package/dist/{site-extract-repository-CFMSWAVL.js → site-extract-repository-XAA2FVDN.js} +3 -3
  28. package/dist/{worker-S2WXELUC.js → worker-QFSOMF2N.js} +2 -2
  29. package/package.json +1 -1
  30. package/dist/chunk-67JUTUQD.js.map +0 -1
  31. package/dist/chunk-QN4W5SX6.js +0 -7
  32. package/dist/chunk-QN4W5SX6.js.map +0 -1
  33. package/dist/server-OZJXK7AB.js.map +0 -1
  34. /package/dist/{chunk-5XYBKE3X.js.map → chunk-KMGRM4FL.js.map} +0 -0
  35. /package/dist/{extract-bundle-H7JEELY7.js.map → extract-bundle-XRRN6NAZ.js.map} +0 -0
  36. /package/dist/{site-extract-repository-CFMSWAVL.js.map → site-extract-repository-XAA2FVDN.js.map} +0 -0
  37. /package/dist/{worker-S2WXELUC.js.map → worker-QFSOMF2N.js.map} +0 -0
package/CHANGELOG.md CHANGED
@@ -4,6 +4,25 @@ All notable changes to MCP Scraper are documented here. The format is based on [
4
4
 
5
5
  ## [Unreleased]
6
6
 
7
+ ## [0.58.0] - 2026-08-17
8
+
9
+ ### Added
10
+
11
+ - `commons_update_editorial_article` replaces the Markdown of one article in a published edition without resupplying the others. Every edition already stored each article's source; the edit reads them, swaps the named article, re-renders, and publishes the next revision with the rest byte-identical. Guarded by `baseRevision` and `idempotencyKey`.
12
+ - `commons_get_publication` accepts `includeArticles` to return stored article sources and slugs. Owner-only and off by default: reading a publication by name never returns sources, and the payload is large.
13
+
14
+ ### Fixed
15
+
16
+ - Published editions never returned their article sources. `articles_json` was written on every publish and omitted from the edition mapper, so no API response carried it — which made a single-article edit look impossible and forced a full 100-article resubmission to change one paragraph.
17
+
18
+ ## [0.57.3] - 2026-08-16
19
+
20
+ ### Fixed
21
+
22
+ - Local Sourcebook exact-place matching no longer rejects a Google listing that appends a service tagline to the business name. `nameOverlap` divided the intersection by the larger token set, so "Meljestic Spa" against "Meljestic Spa: Laser hair removal and Skincare" scored 0.33 against a 0.6 threshold and could never publish — even though the submitted name is fully contained in the Google name and Maps linked the same domain. Identity now also accepts containment of the smaller token set (which must carry two distinctive tokens) or a Maps listing that links the submitted domain. Wrong domain, wrong state, missing address, and single-generic-token still reject.
23
+ - `acquisition_error` is now returned as `acquisitionError` on the submission. It was written on every failure and never selected back, so no response explained why a listing failed.
24
+ - `local_sourcebook_submission_status` and `local_sourcebook_refresh` now state what a failed status means, that coverage counters are only written on success (so zeroed counters do not mean the crawl never ran), and that a replayed refresh still re-runs acquisition.
25
+
7
26
  ## [0.57.2] - 2026-08-16
8
27
 
9
28
  ### Fixed
package/README.md CHANGED
@@ -90,7 +90,7 @@ Build the branded one-click bundle:
90
90
  npm run build:mcpb
91
91
  ```
92
92
 
93
- The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.57.2`, SHA-256 `1c7ae1f1acf54fc91bc44a8dae29b1da20542a63b168f496ae91a67be9caaf33`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
93
+ The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.58.0`, SHA-256 `9d483b81b2e3c5825c2cee385d7e2d2220b6c56e1ed31b047be6b3dc03216894`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
94
94
 
95
95
  The MCPB install exposes every tool — web-intelligence plus all `browser_*` tools — through the one `mcp-scraper` server.
96
96
 
@@ -248,7 +248,7 @@ The `mcp-scraper` server (and the MCPB bundle, which runs it) exposes both secti
248
248
 
249
249
  All MCP tools expose output schemas and return `structuredContent` with the IDs, URLs, CSV paths, transcripts, browser session handles, replay paths, artifacts, recipe fields, or blueprint fields needed by the next step. Browser Agent tools keep a JSON text block for older clients, but structured data is the primary contract. All tools carry MCP annotations; file-writing tools such as replay downloads and annotations state their filesystem side effects.
250
250
 
251
- The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 251 tools: 150 scraper, browser, workflow, billing, and connected-service tools plus 101 durable-memory tools. The scraper-side inventory includes governed Local Sourcebook tools, Transparent Commons entity, linkset, and publication tools, the searchable News Publisher and Editorial Reading Room template workflow, and 24 X-Ray tools for pixel administration, reporting, attribution journeys, campaign links, exports, and scheduled reports. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
251
+ The canonical tool inventory is generated at `docs/mcp-tool-manifest.generated.json`. The unified server exposes 252 tools: 151 scraper, browser, workflow, billing, and connected-service tools plus 101 durable-memory tools. The scraper-side inventory includes governed Local Sourcebook tools, Transparent Commons entity, linkset, and publication tools, the searchable News Publisher and Editorial Reading Room template workflow, and 24 X-Ray tools for pixel administration, reporting, attribution journeys, campaign links, exports, and scheduled reports. Successful evidence-compiled Local Sourcebook revisions publish automatically to their canonical `localsourcebook.com` category profile and review URLs; administrator controls handle exceptional rejection or unpublishing. Release verification compares the exact local and hosted tool-name sets, not only the count.
252
252
 
253
253
  For contract parity, stdio and MCPB memory calls invoke the matching public tool on the hosted MCP Scraper `/mcp` endpoint. The hosted aggregate runtime owns MCP Scraper-specific billing, scheduling, credential, and in-process cutover policy; its internal `/memory/mcp-call` bridge is a fallback to the standalone memory service, not the public stdio execution path. Direct `mcp-memory` OAuth and stdio clients continue to use `memory.mcpscraper.dev` and must be verified as a separate dependent release surface.
254
254
 
@@ -12455,8 +12455,8 @@ var init_product_contract_generated = __esm({
12455
12455
  "freeSignupCredits": 0
12456
12456
  },
12457
12457
  "inventory": {
12458
- "totalTools": 251,
12459
- "scraperTools": 150,
12458
+ "totalTools": 252,
12459
+ "scraperTools": 151,
12460
12460
  "memoryTools": 101
12461
12461
  },
12462
12462
  "concurrencyPack": {
@@ -29050,6 +29050,7 @@ function mapSubmission(row) {
29050
29050
  businessName: String(rowValue(row, "business_name")),
29051
29051
  websiteUrl: String(rowValue(row, "website_url")),
29052
29052
  status: String(rowValue(row, "status")),
29053
+ acquisitionError: rowValue(row, "acquisition_error") == null ? null : String(rowValue(row, "acquisition_error")),
29053
29054
  draftRevision: Number(rowValue(row, "draft_revision")),
29054
29055
  publishedRevision: rowValue(row, "published_revision") == null ? null : Number(rowValue(row, "published_revision")),
29055
29056
  coverage: parseJson2(rowValue(row, "coverage_json")),
@@ -30915,6 +30916,15 @@ function nameOverlap(left, right) {
30915
30916
  const intersection = [...a].filter((token6) => b.has(token6)).length;
30916
30917
  return intersection / Math.max(a.size, b.size);
30917
30918
  }
30919
+ function nameContainment(left, right) {
30920
+ const a = tokens(left);
30921
+ const b = tokens(right);
30922
+ if (!a.size || !b.size) return 0;
30923
+ const smaller = a.size <= b.size ? a : b;
30924
+ if (smaller.size < 2) return 0;
30925
+ const intersection = [...a].filter((token6) => b.has(token6)).length;
30926
+ return intersection / smaller.size;
30927
+ }
30918
30928
  function validateLocalSourcebookMapsIdentity(submission, maps) {
30919
30929
  const expectedHost = host(submission.websiteUrl);
30920
30930
  const mapsHost = host(maps.website);
@@ -30924,7 +30934,9 @@ function validateLocalSourcebookMapsIdentity(submission, maps) {
30924
30934
  if (mapsHost && expectedHost && mapsHost !== expectedHost && !mapsHost.endsWith(`.${expectedHost}`) && !expectedHost.endsWith(`.${mapsHost}`)) {
30925
30935
  throw new Error(`Exact-place match rejected: Google Maps linked ${mapsHost}, not the submitted domain ${expectedHost}.`);
30926
30936
  }
30927
- if (overlap < 0.6) {
30937
+ const containment = nameContainment(submission.businessName, actualName);
30938
+ const domainConfirmsIdentity = Boolean(mapsHost && expectedHost);
30939
+ if (overlap < 0.6 && containment < 0.9 && !domainConfirmsIdentity) {
30928
30940
  throw new Error(`Exact-place match rejected: Google Maps returned \u201C${actualName},\u201D which does not sufficiently match \u201C${submission.businessName}.\u201D`);
30929
30941
  }
30930
30942
  if (addressState && addressState !== submission.state.toLowerCase()) {
@@ -44638,7 +44650,7 @@ var PACKAGE_VERSION;
44638
44650
  var init_version = __esm({
44639
44651
  "src/version.ts"() {
44640
44652
  "use strict";
44641
- PACKAGE_VERSION = "0.57.2";
44653
+ PACKAGE_VERSION = "0.58.0";
44642
44654
  }
44643
44655
  });
44644
44656
 
@@ -46467,7 +46479,7 @@ var init_contracts = __esm({
46467
46479
  });
46468
46480
 
46469
46481
  // src/mcp/mcp-tool-schemas.ts
46470
- var import_zod43, WEBSITE_URL_OR_DOMAIN_ERROR, WebsiteUrlOrDomainSchema, HarvestPaaInputSchema, ExtractUrlBaseInputSchema, ExtractUrlInputSchema, ExtractUrlLocalInputSchema, DiffPageBaseInputSchema, DiffPageInputSchema, DiffPageLocalInputSchema, MapSiteUrlsInputSchema, MapWaybackSnapshotsInputSchema, ExtractSiteInputSchema, AuditSiteInputSchema, CheckSiteExportInputSchema, ArchiveReadInputSchema, YoutubeHarvestInputSchema, YoutubeTranscribeInputSchema, FacebookPageIntelInputSchema, FacebookAdSearchInputSchema, RedditThreadInputSchema, RedditTrendingInputSchema, VideoFrameAnalysisInputSchema, VideoFrameAnalysisStatusInputSchema, FacebookAdTranscribeInputSchema, FacebookVideoTranscribeInputSchema, GoogleAdsSearchInputSchema, GoogleAdsPageIntelInputSchema, GoogleAdsTranscribeInputSchema, InstagramProfileContentInputSchema, InstagramMediaDownloadInputSchema, MapsPlaceIntelInputSchema, TrustpilotReviewsInputSchema, G2ReviewsInputSchema, ReviewCardSchema, MapsSearchInputSchema, DirectoryWorkflowInputSchema, LocationMarketsInputSchema, CommonsSearchEntitiesInputSchema, CommonsGetEntityInputSchema, CommonsGetEntityLinksetInputSchema, CommonsFeaturedImageInputSchema, CommonsMediaInputSchema, CommonsCitationInputSchema, CommonsSourceInputSchema, CommonsRelatedLinkInputSchema, CommonsClaimInputSchema, CommonsPrepareEntityInputSchema, CommonsSubmitEntityInputSchema, CommonsValidateEntityInputSchema, CommonsGetEntityLedgerInputSchema, CommonsHostImageInputSchema, CommonsGetProposalInputSchema, CommonsSaveFilterInputSchema, CommonsListFiltersInputSchema, CommonsListNeedsLinksInputSchema, CommonsGenericOutputSchema, DirectoryWorkflowStatusInputSchema, LocalSourcebookSubmitInputSchema, LocalSourcebookCategorySchema, LocalSourcebookSchemaTypeInputSchema, LocalSourcebookTagCandidateObjectSchema, LocalSourcebookTagDecisionObjectSchema, LocalSourcebookIdentityObjectSchema, GetLocalSourcebookContractInputSchema, ListLocalSourcebookTagsInputSchema, ResolveLocalSourcebookTagsInputSchema, PrepareLocalSourcebookWriteInputSchema, ValidateLocalSourcebookWriteInputSchema, LocalSourcebookCaptureInputSchema, LocalSourcebookSubmissionStatusInputSchema, LocalSourcebookRefreshInputSchema, LocalSourcebookOutputSchema, ArtifactPointerOutputSchema, EditorialReadingRoomSiteSchema, EditorialReadingRoomImageSchema, EditorialReadingRoomArticleSchema, EditorialReadingRoomGuideInputSchema, EditorialReadingRoomGuideOutputSchema, CreateEditorialReadingRoomInputSchema, EditorialReadingRoomArtifactSchema, CreateEditorialReadingRoomOutputSchema, RenewEditorialReadingRoomDownloadInputSchema, RenewEditorialReadingRoomDownloadOutputSchema, CommonsPublicationSubdomainSchema, CommonsPreparePublicationInputSchema, CommonsValidatePublicationInputSchema, CommonsClaimPublicationInputSchema, CommonsPublishEditorialInputSchema, CommonsGetPublicationInputSchema, RankTrackerModeSchema, RankTrackerBlueprintInputSchema, NullableString, MapsSearchAttemptOutput, MapsSearchOutputSchema, DirectoryMapsBusinessOutput, DirectoryCsvArtifactOutput, DirectoryWorkflowOutputSchema, LocationDatasetProvenanceOutput, LocationMarketsOutputSchema, RankTrackerToolPlanOutput, RankTrackerTableOutput, RankTrackerCronJobOutput, RankTrackerBlueprintOutputSchema, OrganicResultOutput, AiOverviewOutput, EntityIdsOutput, HarvestPaaOutputSchema, SearchSerpOutputSchema, PageMediaAssetOutput, PageMediaArtifactOutput, ExtractUrlOutputSchema, DiffPageOutputSchema, ExtractSiteOutputSchema, AuditSiteOutputSchema, CheckSiteExportOutputSchema, ArchiveEntryOutputSchema, ArchiveReadOutputSchema, MapsPlaceIntelOutputSchema, TrustpilotReviewsOutputSchema, G2ReviewsOutputSchema, CreditsInfoOutputSchema, MapSiteUrlsOutputSchema, WaybackCaptureOutputSchema, MapWaybackSnapshotsOutputSchema, YoutubeHarvestOutputSchema, FacebookAdSearchOutputSchema, VideoFrameAnalysisOutputSchema, VideoFrameAnalysisStatusOutputSchema, RedditThreadOutputSchema, RedditTrendingOutputSchema, FacebookPageIntelOutputSchema, GoogleAdsSearchOutputSchema, GoogleAdsPageIntelOutputSchema, TranscriptSignalOutput, FacebookVideoTranscribeOutputSchema, TranscriptChunkOutput, InstagramBrowserOutput, InstagramPaginationOutput, InstagramProfileContentOutputSchema, InstagramMediaTrackOutput, InstagramDownloadOutput, InstagramMediaDownloadOutputSchema, YoutubeTranscribeOutputSchema, FacebookAdTranscribeOutputSchema, GoogleAdsTranscribeOutputSchema, CaptureSerpSnapshotOutputSchema, CaptureSerpPageSnapshotsOutputSchema, CreditsInfoInputSchema, WorkflowIdSchema2, WorkflowListInputSchema, WorkflowSuggestInputSchema, WorkflowRunInputSchema, WorkflowStepInputSchema, WorkflowStatusInputSchema, WorkflowArtifactReadInputSchema, WorkflowRecipeOutput, WorkflowDefinitionOutput, WorkflowArtifactOutput, WorkflowListOutputSchema, WorkflowSuggestOutputSchema, WorkflowRunOutputSchema, WorkflowStepOutputSchema, WorkflowStatusOutputSchema, WorkflowArtifactReadOutputSchema, SearchSerpInputSchema, CaptureSerpSnapshotInputSchema, ScreenshotInputSchema, CaptureSerpPageSnapshotsInputSchema, ReportArtifactReadInputSchema, ReportArtifactReadOutputSchema, ListServiceConnectionsInputSchema, ListServiceConnectionsOutputSchema, TestServiceConnectionInputSchema, TestServiceConnectionOutputSchema, ReadServiceConnectionInputSchema, ReadServiceConnectionOutputSchema, MetaAdCreativeMediaInputSchema, MetaAdCreativeMediaOutputSchema, ImportServiceConnectionToMemoryInputSchema, ImportServiceConnectionToMemoryOutputSchema, DescribeServiceConnectionToolInputSchema, DescribeServiceConnectionToolOutputSchema, ConnectedDataContinuationSchema, ExportConnectedServiceDataInputSchema, ConnectedDataArtifactSchema, ExportConnectedServiceDataOutputSchema, SearchConsoleTableColumnSchema, SearchConsoleTableFilterSchema, ExportSearchConsoleTableDataInputSchema, ExportSearchConsoleTableDataOutputSchema, RenewConnectedDataExportDownloadInputSchema, RenewConnectedDataExportDownloadOutputSchema, CallServiceConnectionActionInputSchema, CallServiceConnectionActionOutputSchema, SetScheduledActionConnectionsInputSchema, SetScheduledActionConnectionsOutputSchema, SlackSendMessageInputSchema, SlackSendMessageOutputSchema, GmailSendMessageInputSchema, GmailSendMessageOutputSchema, GmailSearchContactsInputSchema, GmailSearchContactsOutputSchema, GoogleCalendarCreateEventInputSchema, GoogleCalendarCreateEventOutputSchema, ZoomCreateMeetingInputSchema, ZoomCreateMeetingOutputSchema;
46482
+ var import_zod43, WEBSITE_URL_OR_DOMAIN_ERROR, WebsiteUrlOrDomainSchema, HarvestPaaInputSchema, ExtractUrlBaseInputSchema, ExtractUrlInputSchema, ExtractUrlLocalInputSchema, DiffPageBaseInputSchema, DiffPageInputSchema, DiffPageLocalInputSchema, MapSiteUrlsInputSchema, MapWaybackSnapshotsInputSchema, ExtractSiteInputSchema, AuditSiteInputSchema, CheckSiteExportInputSchema, ArchiveReadInputSchema, YoutubeHarvestInputSchema, YoutubeTranscribeInputSchema, FacebookPageIntelInputSchema, FacebookAdSearchInputSchema, RedditThreadInputSchema, RedditTrendingInputSchema, VideoFrameAnalysisInputSchema, VideoFrameAnalysisStatusInputSchema, FacebookAdTranscribeInputSchema, FacebookVideoTranscribeInputSchema, GoogleAdsSearchInputSchema, GoogleAdsPageIntelInputSchema, GoogleAdsTranscribeInputSchema, InstagramProfileContentInputSchema, InstagramMediaDownloadInputSchema, MapsPlaceIntelInputSchema, TrustpilotReviewsInputSchema, G2ReviewsInputSchema, ReviewCardSchema, MapsSearchInputSchema, DirectoryWorkflowInputSchema, LocationMarketsInputSchema, CommonsSearchEntitiesInputSchema, CommonsGetEntityInputSchema, CommonsGetEntityLinksetInputSchema, CommonsFeaturedImageInputSchema, CommonsMediaInputSchema, CommonsCitationInputSchema, CommonsSourceInputSchema, CommonsRelatedLinkInputSchema, CommonsClaimInputSchema, CommonsPrepareEntityInputSchema, CommonsSubmitEntityInputSchema, CommonsValidateEntityInputSchema, CommonsGetEntityLedgerInputSchema, CommonsHostImageInputSchema, CommonsGetProposalInputSchema, CommonsSaveFilterInputSchema, CommonsListFiltersInputSchema, CommonsListNeedsLinksInputSchema, CommonsGenericOutputSchema, DirectoryWorkflowStatusInputSchema, LocalSourcebookSubmitInputSchema, LocalSourcebookCategorySchema, LocalSourcebookSchemaTypeInputSchema, LocalSourcebookTagCandidateObjectSchema, LocalSourcebookTagDecisionObjectSchema, LocalSourcebookIdentityObjectSchema, GetLocalSourcebookContractInputSchema, ListLocalSourcebookTagsInputSchema, ResolveLocalSourcebookTagsInputSchema, PrepareLocalSourcebookWriteInputSchema, ValidateLocalSourcebookWriteInputSchema, LocalSourcebookCaptureInputSchema, LocalSourcebookSubmissionStatusInputSchema, LocalSourcebookRefreshInputSchema, LocalSourcebookOutputSchema, ArtifactPointerOutputSchema, EditorialReadingRoomSiteSchema, EditorialReadingRoomImageSchema, EditorialReadingRoomArticleSchema, EditorialReadingRoomGuideInputSchema, EditorialReadingRoomGuideOutputSchema, CreateEditorialReadingRoomInputSchema, EditorialReadingRoomArtifactSchema, CreateEditorialReadingRoomOutputSchema, RenewEditorialReadingRoomDownloadInputSchema, RenewEditorialReadingRoomDownloadOutputSchema, CommonsPublicationSubdomainSchema, CommonsPreparePublicationInputSchema, CommonsValidatePublicationInputSchema, CommonsClaimPublicationInputSchema, CommonsPublishEditorialInputSchema, CommonsUpdateEditorialArticleInputSchema, CommonsGetPublicationInputSchema, RankTrackerModeSchema, RankTrackerBlueprintInputSchema, NullableString, MapsSearchAttemptOutput, MapsSearchOutputSchema, DirectoryMapsBusinessOutput, DirectoryCsvArtifactOutput, DirectoryWorkflowOutputSchema, LocationDatasetProvenanceOutput, LocationMarketsOutputSchema, RankTrackerToolPlanOutput, RankTrackerTableOutput, RankTrackerCronJobOutput, RankTrackerBlueprintOutputSchema, OrganicResultOutput, AiOverviewOutput, EntityIdsOutput, HarvestPaaOutputSchema, SearchSerpOutputSchema, PageMediaAssetOutput, PageMediaArtifactOutput, ExtractUrlOutputSchema, DiffPageOutputSchema, ExtractSiteOutputSchema, AuditSiteOutputSchema, CheckSiteExportOutputSchema, ArchiveEntryOutputSchema, ArchiveReadOutputSchema, MapsPlaceIntelOutputSchema, TrustpilotReviewsOutputSchema, G2ReviewsOutputSchema, CreditsInfoOutputSchema, MapSiteUrlsOutputSchema, WaybackCaptureOutputSchema, MapWaybackSnapshotsOutputSchema, YoutubeHarvestOutputSchema, FacebookAdSearchOutputSchema, VideoFrameAnalysisOutputSchema, VideoFrameAnalysisStatusOutputSchema, RedditThreadOutputSchema, RedditTrendingOutputSchema, FacebookPageIntelOutputSchema, GoogleAdsSearchOutputSchema, GoogleAdsPageIntelOutputSchema, TranscriptSignalOutput, FacebookVideoTranscribeOutputSchema, TranscriptChunkOutput, InstagramBrowserOutput, InstagramPaginationOutput, InstagramProfileContentOutputSchema, InstagramMediaTrackOutput, InstagramDownloadOutput, InstagramMediaDownloadOutputSchema, YoutubeTranscribeOutputSchema, FacebookAdTranscribeOutputSchema, GoogleAdsTranscribeOutputSchema, CaptureSerpSnapshotOutputSchema, CaptureSerpPageSnapshotsOutputSchema, CreditsInfoInputSchema, WorkflowIdSchema2, WorkflowListInputSchema, WorkflowSuggestInputSchema, WorkflowRunInputSchema, WorkflowStepInputSchema, WorkflowStatusInputSchema, WorkflowArtifactReadInputSchema, WorkflowRecipeOutput, WorkflowDefinitionOutput, WorkflowArtifactOutput, WorkflowListOutputSchema, WorkflowSuggestOutputSchema, WorkflowRunOutputSchema, WorkflowStepOutputSchema, WorkflowStatusOutputSchema, WorkflowArtifactReadOutputSchema, SearchSerpInputSchema, CaptureSerpSnapshotInputSchema, ScreenshotInputSchema, CaptureSerpPageSnapshotsInputSchema, ReportArtifactReadInputSchema, ReportArtifactReadOutputSchema, ListServiceConnectionsInputSchema, ListServiceConnectionsOutputSchema, TestServiceConnectionInputSchema, TestServiceConnectionOutputSchema, ReadServiceConnectionInputSchema, ReadServiceConnectionOutputSchema, MetaAdCreativeMediaInputSchema, MetaAdCreativeMediaOutputSchema, ImportServiceConnectionToMemoryInputSchema, ImportServiceConnectionToMemoryOutputSchema, DescribeServiceConnectionToolInputSchema, DescribeServiceConnectionToolOutputSchema, ConnectedDataContinuationSchema, ExportConnectedServiceDataInputSchema, ConnectedDataArtifactSchema, ExportConnectedServiceDataOutputSchema, SearchConsoleTableColumnSchema, SearchConsoleTableFilterSchema, ExportSearchConsoleTableDataInputSchema, ExportSearchConsoleTableDataOutputSchema, RenewConnectedDataExportDownloadInputSchema, RenewConnectedDataExportDownloadOutputSchema, CallServiceConnectionActionInputSchema, CallServiceConnectionActionOutputSchema, SetScheduledActionConnectionsInputSchema, SetScheduledActionConnectionsOutputSchema, SlackSendMessageInputSchema, SlackSendMessageOutputSchema, GmailSendMessageInputSchema, GmailSendMessageOutputSchema, GmailSearchContactsInputSchema, GmailSearchContactsOutputSchema, GoogleCalendarCreateEventInputSchema, GoogleCalendarCreateEventOutputSchema, ZoomCreateMeetingInputSchema, ZoomCreateMeetingOutputSchema;
46471
46483
  var init_mcp_tool_schemas = __esm({
46472
46484
  "src/mcp/mcp-tool-schemas.ts"() {
46473
46485
  "use strict";
@@ -47178,7 +47190,21 @@ var init_mcp_tool_schemas = __esm({
47178
47190
  baseRevision: import_zod43.z.number().int().positive().optional().describe("Required when revising an existing edition; use the current revision from commons_get_publication."),
47179
47191
  ...CreateEditorialReadingRoomInputSchema
47180
47192
  };
47193
+ CommonsUpdateEditorialArticleInputSchema = {
47194
+ publicationSubdomain: CommonsPublicationSubdomainSchema.describe("Publication owned by the authenticated account."),
47195
+ editionSlug: import_zod43.z.string().trim().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/).max(80).optional().describe("Edition holding the article. Defaults to the publication\u2019s latest edition."),
47196
+ articleSlug: import_zod43.z.string().trim().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/).max(80).describe("Slug of the one article to replace, from commons_get_publication with includeArticles."),
47197
+ idempotencyKey: import_zod43.z.string().trim().min(8).max(200).describe("Unique key for this intended edit. Reuse it only when retrying the same edit."),
47198
+ baseRevision: import_zod43.z.number().int().positive().describe("Current edition revision from commons_get_publication. The edit is refused if the edition moved on."),
47199
+ markdown: import_zod43.z.string().trim().min(1).max(4e5).optional().describe("Replacement Markdown body for this article. Omit to change only the metadata fields below."),
47200
+ title: import_zod43.z.string().trim().min(1).max(180).optional().describe("Replacement article title."),
47201
+ summary: import_zod43.z.string().trim().min(1).max(500).optional().describe("Replacement one or two sentence summary."),
47202
+ category: import_zod43.z.string().trim().min(1).max(80).optional().describe("Replacement section label."),
47203
+ kicker: import_zod43.z.string().trim().min(1).max(140).optional().describe("Replacement framing line above the title."),
47204
+ order: import_zod43.z.number().int().min(1).max(1e3).optional().describe("Replacement reading-order position.")
47205
+ };
47181
47206
  CommonsGetPublicationInputSchema = {
47207
+ includeArticles: import_zod43.z.boolean().optional().describe("Return every stored article\u2019s Markdown source alongside each edition. Large payload; request it only when you intend to edit an article. Owner-only: a publication read by name never returns sources."),
47182
47208
  subdomain: CommonsPublicationSubdomainSchema.optional().describe("Public publication name to inspect. Omit to return the publication owned by the authenticated account."),
47183
47209
  includeEditions: import_zod43.z.boolean().default(true).describe("Include the latest revision of every published edition.")
47184
47210
  };
@@ -50578,9 +50604,16 @@ function registerPaaExtractorMcpTools(server, executor, options = {}) {
50578
50604
  outputSchema: recordOutputSchema("commons_publish_editorial", CommonsGenericOutputSchema),
50579
50605
  annotations: { title: "Publish Transparent Commons Editorial Edition", readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: true }
50580
50606
  }, async (input) => executor.commonsPublishEditorial(input));
50607
+ server.registerTool("commons_update_editorial_article", {
50608
+ title: "Edit One Transparent Commons Article",
50609
+ description: "Replace the Markdown of a single article in a published edition without resupplying the others. The edition already stores every article\u2019s source, so this reads them, swaps the one you name, re-renders, and publishes the next revision with the rest byte-identical. Use commons_get_publication with includeArticles to read current sources and the revision first; commons_publish_editorial remains the full-replace path for a wholesale rewrite. Requires baseRevision and idempotencyKey.",
50610
+ inputSchema: CommonsUpdateEditorialArticleInputSchema,
50611
+ outputSchema: recordOutputSchema("commons_update_editorial_article", CommonsGenericOutputSchema),
50612
+ annotations: { title: "Edit One Transparent Commons Article", readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: true }
50613
+ }, async (input) => executor.commonsUpdateEditorialArticle(input));
50581
50614
  server.registerTool("commons_get_publication", {
50582
50615
  title: "Get Transparent Commons Publication",
50583
- description: "Read a Transparent Commons publication and its latest edition revisions. Omit subdomain to inspect the caller-owned publication; pass a name to inspect a public publication. Returns the permanent public and archive URLs needed for sharing or later edits.",
50616
+ description: "Read a Transparent Commons publication and its latest edition revisions. Omit subdomain to inspect the caller-owned publication; pass a name to inspect a public publication. Returns the permanent public and archive URLs needed for sharing or later edits. Set includeArticles to also return every stored article\u2019s Markdown source and slug, which is what commons_update_editorial_article edits \u2014 the payload is large, so request it only when you intend to edit.",
50584
50617
  inputSchema: CommonsGetPublicationInputSchema,
50585
50618
  outputSchema: recordOutputSchema("commons_get_publication", CommonsGenericOutputSchema),
50586
50619
  annotations: { title: "Get Transparent Commons Publication", readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false }
@@ -50647,14 +50680,14 @@ function registerPaaExtractorMcpTools(server, executor, options = {}) {
50647
50680
  }, async (input) => executor.localSourcebookCapture(input));
50648
50681
  server.registerTool("local_sourcebook_submission_status", {
50649
50682
  title: "Local Sourcebook Submission Status",
50650
- description: "Read the authenticated caller\u2019s listing draft, enrichment coverage, immutable revision number, publication state, and exact live LocalSourcebook.com profile and reviews URLs.",
50683
+ description: "Read the authenticated caller\u2019s listing draft, enrichment coverage, immutable revision number, publication state, and exact live LocalSourcebook.com profile and reviews URLs. When status is failed, acquisitionError states why in plain language \u2014 read it before retrying, because a refresh repeats the same acquisition and will fail the same way until the cause is fixed. Coverage counters are written only when acquisition succeeds, so a failed listing still shows the pre-run crawl block; zeroed counters do not mean the crawl never ran.",
50651
50684
  inputSchema: LocalSourcebookSubmissionStatusInputSchema,
50652
50685
  outputSchema: recordOutputSchema("local_sourcebook_submission_status", LocalSourcebookOutputSchema),
50653
50686
  annotations: localPlanningToolAnnotations("Local Sourcebook Submission Status")
50654
50687
  }, async (input) => executor.localSourcebookSubmissionStatus(input));
50655
50688
  server.registerTool("local_sourcebook_refresh", {
50656
50689
  title: "Refresh a Local Sourcebook Listing",
50657
- description: "Queue a new broad crawl and review/media acquisition pass for a listing owned by the authenticated MCP Scraper account. A refresh costs 2 Credits total, including acquisition; idempotent retries are not charged twice. The last published revision remains public until the refreshed evidence revision completes and auto-publishes.",
50690
+ description: "Queue a new broad crawl and review/media acquisition pass for a listing owned by the authenticated MCP Scraper account. A refresh costs 2 Credits total, including acquisition; idempotent retries are not charged twice, and a replayed retry still re-runs the acquisition, so a failed listing is always recoverable. Read local_sourcebook_submission_status first: a refresh repeats the same acquisition, so fix whatever acquisitionError reports before spending another pass. The last published revision remains public until the refreshed evidence revision completes and auto-publishes.",
50658
50691
  inputSchema: LocalSourcebookRefreshInputSchema,
50659
50692
  outputSchema: recordOutputSchema("local_sourcebook_refresh", LocalSourcebookOutputSchema),
50660
50693
  annotations: liveWebToolAnnotations("Refresh a Local Sourcebook Listing")
@@ -51577,6 +51610,12 @@ var init_http_mcp_tool_executor = __esm({
51577
51610
  commonsHostImage(input) {
51578
51611
  return this.call("/commons/images", input);
51579
51612
  }
51613
+ commonsUpdateEditorialArticle(input) {
51614
+ const { idempotencyKey: idempotencyKey3, ...body } = input;
51615
+ return this.call("/commons/publications/articles", { ...body, idempotencyKey: idempotencyKey3 }, this.timeoutMs, "POST", {
51616
+ "Idempotency-Key": `commons-article-${(0, import_node_crypto28.createHash)("sha256").update(idempotencyKey3).digest("hex")}`
51617
+ });
51618
+ }
51580
51619
  commonsSaveFilter(input) {
51581
51620
  return this.call("/commons/filters", input);
51582
51621
  }
@@ -51605,7 +51644,7 @@ var init_http_mcp_tool_executor = __esm({
51605
51644
  });
51606
51645
  }
51607
51646
  commonsGetPublication(input) {
51608
- const query = new URLSearchParams({ includeEditions: String(input.includeEditions ?? true) });
51647
+ const query = new URLSearchParams({ includeEditions: String(input.includeEditions ?? true), includeArticles: String(input.includeArticles ?? false) });
51609
51648
  return this.getJson(input.subdomain ? `/commons/publications/${encodeURIComponent(input.subdomain)}?${query}` : `/commons/publications/me?${query}`);
51610
51649
  }
51611
51650
  async captureSerpSnapshot(input) {
@@ -67229,6 +67268,92 @@ async function publishCommonsEditorial(input, user) {
67229
67268
  if (!edition) throw new CommonsPublicationError("publication_publish_failed", "The published edition was not persisted.");
67230
67269
  return publicationResult({ ...publication, latestEditionId: editionId, updatedAt: now }, edition, false);
67231
67270
  }
67271
+ async function updateCommonsEditorialArticle(input, user) {
67272
+ await ensureCommonsSchema();
67273
+ const subdomain = normalizePublicationSubdomain(input.publicationSubdomain);
67274
+ const publication = await getCommonsPublicationBySubdomain(subdomain);
67275
+ if (!publication) throw new CommonsPublicationError("publication_not_found", "Claim this publication name before editing an edition.", 404);
67276
+ if (publication.ownerUserId !== Number(user.id)) {
67277
+ throw new CommonsPublicationError("publication_not_owned", "Only the account that claimed this publication can edit it.", 404);
67278
+ }
67279
+ const idempotencyKey3 = requiredIdempotencyKey(input.idempotencyKey);
67280
+ const replay = await getEditionByIdempotency(Number(user.id), idempotencyKey3);
67281
+ if (replay) return publicationResult(publication, replay, true);
67282
+ const latestOverall = publication.latestEditionId ? await getPublicationEditionById(publication.latestEditionId) : null;
67283
+ const editionSlug = normalizeEditionSlug(input.editionSlug || latestOverall?.editionSlug || "");
67284
+ if (!editionSlug) throw new CommonsPublicationError("publication_edition_required", "This publication has no edition yet; publish one before editing an article.", 409);
67285
+ const latest = await getLatestEdition(publication.id, editionSlug);
67286
+ if (!latest) throw new CommonsPublicationError("publication_edition_not_found", `Edition ${editionSlug} does not exist.`, 404);
67287
+ if (input.baseRevision !== latest.revision) {
67288
+ throw new CommonsPublicationError("publication_revision_conflict", `Edition ${editionSlug} is revision ${latest.revision}; read it again before editing.`, 409);
67289
+ }
67290
+ const articles = latest.articles;
67291
+ const index = articles.findIndex((article) => article.slug === input.articleSlug);
67292
+ if (index === -1) {
67293
+ throw new CommonsPublicationError("publication_article_not_found", `Edition ${editionSlug} has no article with slug ${input.articleSlug}. Known slugs: ${articles.map((article) => article.slug).join(", ")}.`, 404);
67294
+ }
67295
+ const current = articles[index];
67296
+ const nextArticles = [...articles];
67297
+ nextArticles[index] = {
67298
+ ...current,
67299
+ ...input.markdown === void 0 ? {} : { markdown: input.markdown },
67300
+ ...input.title === void 0 ? {} : { title: input.title },
67301
+ ...input.summary === void 0 ? {} : { summary: input.summary },
67302
+ ...input.category === void 0 ? {} : { category: input.category },
67303
+ ...input.kicker === void 0 ? {} : { kicker: input.kicker },
67304
+ ...input.order === void 0 ? {} : { order: input.order }
67305
+ };
67306
+ const editionInput = { site: latest.site, deck: latest.deck, articles: nextArticles };
67307
+ const canonicalUrl = editionPublicUrl(subdomain, editionSlug);
67308
+ const rendered = renderEditorialReadingRoom(editionInput);
67309
+ const html = addPublicMetadata(rendered.html, canonicalUrl, publication.title);
67310
+ const editionId = `tced_${(0, import_node_crypto40.randomUUID)()}`;
67311
+ const revision = latest.revision + 1;
67312
+ const now = (/* @__PURE__ */ new Date()).toISOString();
67313
+ await getDb().batch([
67314
+ {
67315
+ sql: `
67316
+ INSERT INTO commons_publication_editions (
67317
+ id, publication_id, owner_user_id, edition_slug, revision, title, site_json, deck,
67318
+ articles_json, html, filename, sha256, article_count, word_count, bytes, warnings_json,
67319
+ idempotency_key, created_at, published_at
67320
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
67321
+ `,
67322
+ args: [
67323
+ editionId,
67324
+ publication.id,
67325
+ Number(user.id),
67326
+ editionSlug,
67327
+ revision,
67328
+ latest.site.title,
67329
+ JSON.stringify(latest.site),
67330
+ latest.deck,
67331
+ JSON.stringify(nextArticles),
67332
+ html,
67333
+ rendered.filename,
67334
+ (0, import_node_crypto40.createHash)("sha256").update(html).digest("hex"),
67335
+ rendered.articleCount,
67336
+ rendered.wordCount,
67337
+ Buffer.byteLength(html),
67338
+ JSON.stringify(rendered.warnings),
67339
+ idempotencyKey3,
67340
+ now,
67341
+ now
67342
+ ]
67343
+ },
67344
+ {
67345
+ sql: "UPDATE commons_publications SET latest_edition_id = ?, updated_at = ? WHERE id = ? AND owner_user_id = ?",
67346
+ args: [editionId, now, publication.id, Number(user.id)]
67347
+ }
67348
+ ], "write");
67349
+ const edition = await getPublicationEditionById(editionId);
67350
+ if (!edition) throw new CommonsPublicationError("publication_publish_failed", "The edited edition was not persisted.");
67351
+ return {
67352
+ ...publicationResult({ ...publication, latestEditionId: editionId, updatedAt: now }, edition, false),
67353
+ updatedArticle: { slug: input.articleSlug, previousRevision: latest.revision, revision },
67354
+ unchangedArticles: articles.length - 1
67355
+ };
67356
+ }
67232
67357
  async function getCommonsPublicationBySubdomain(subdomainInput) {
67233
67358
  await ensureCommonsSchema();
67234
67359
  const subdomain = normalizePublicationSubdomain(subdomainInput);
@@ -67326,6 +67451,7 @@ function rowToEdition(row, subdomain) {
67326
67451
  revision: Number(row.revision),
67327
67452
  title: String(row.title),
67328
67453
  site: parseJson4(row.site_json, {}),
67454
+ articles: parseJson4(row.articles_json, []),
67329
67455
  deck: String(row.deck),
67330
67456
  filename: String(row.filename),
67331
67457
  sha256: String(row.sha256),
@@ -67498,10 +67624,17 @@ function publicationError(c, error) {
67498
67624
  }
67499
67625
  throw error;
67500
67626
  }
67627
+ function withArticleSources(editions, include) {
67628
+ if (include) return editions;
67629
+ return editions.map((edition) => {
67630
+ const { articles: _articles, ...rest } = edition;
67631
+ return rest;
67632
+ });
67633
+ }
67501
67634
  function xmlEscape(value) {
67502
67635
  return value.replaceAll("&", "&amp;").replaceAll("<", "&lt;").replaceAll(">", "&gt;").replaceAll('"', "&quot;").replaceAll("'", "&apos;");
67503
67636
  }
67504
- var import_hono31, import_zod52, commonsApp, auth2, SearchBodySchema, FeaturedImageSchema, MediaSchema, CitationSchema, RelatedLinkSchema, SubmitEntitySchema, PrepareEntitySchema, ValidateEntitySchema, SaveFilterSchema, NeedsLinkBodySchema, PreparePublicationSchema, ValidatePublicationSchema, ClaimPublicationSchema, PublishEditorialSchema, HostImageSchema;
67637
+ var import_hono31, import_zod52, commonsApp, auth2, SearchBodySchema, FeaturedImageSchema, MediaSchema, CitationSchema, RelatedLinkSchema, SubmitEntitySchema, PrepareEntitySchema, ValidateEntitySchema, SaveFilterSchema, NeedsLinkBodySchema, PreparePublicationSchema, ValidatePublicationSchema, ClaimPublicationSchema, PublishEditorialSchema, UpdateEditorialArticleSchema, HostImageSchema;
67505
67638
  var init_commons_routes = __esm({
67506
67639
  "src/api/commons-routes.ts"() {
67507
67640
  "use strict";
@@ -67679,6 +67812,7 @@ var init_commons_routes = __esm({
67679
67812
  ValidatePublicationSchema = import_zod52.z.object(CommonsValidatePublicationInputSchema).strict();
67680
67813
  ClaimPublicationSchema = import_zod52.z.object(CommonsClaimPublicationInputSchema).strict();
67681
67814
  PublishEditorialSchema = import_zod52.z.object(CommonsPublishEditorialInputSchema).strict();
67815
+ UpdateEditorialArticleSchema = import_zod52.z.object(CommonsUpdateEditorialArticleInputSchema).strict();
67682
67816
  commonsApp.get("/health", async (c) => {
67683
67817
  await ensureCommonsSchema();
67684
67818
  return c.json({ ok: true, data: commonsDatabaseReport() });
@@ -67714,11 +67848,16 @@ var init_commons_routes = __esm({
67714
67848
  if (!parsed.success) return validationError(c, parsed.error);
67715
67849
  return publicationOperation(c, () => publishCommonsEditorial(parsed.data, c.get("user")), 201);
67716
67850
  });
67851
+ commonsApp.post("/publications/articles", auth2, async (c) => {
67852
+ const parsed = UpdateEditorialArticleSchema.safeParse(await c.req.json().catch(() => ({})));
67853
+ if (!parsed.success) return validationError(c, parsed.error);
67854
+ return publicationOperation(c, () => updateCommonsEditorialArticle(parsed.data, c.get("user")), 201);
67855
+ });
67717
67856
  commonsApp.get("/publications/me", auth2, async (c) => {
67718
67857
  const publication = await getCommonsPublicationForOwner(Number(c.get("user").id));
67719
67858
  if (!publication) return c.json({ ok: false, error: "publication_not_found", message: "This account has not claimed a Commons publication yet." }, 404);
67720
67859
  const editions = c.req.query("includeEditions") === "false" ? [] : await listCommonsPublicationEditions(publication.id);
67721
- return c.json({ ok: true, data: { publication, editions } });
67860
+ return c.json({ ok: true, data: { publication, editions: withArticleSources(editions, c.req.query("includeArticles") === "true") } });
67722
67861
  });
67723
67862
  commonsApp.get("/publications/:subdomain/site", async (c) => {
67724
67863
  try {
@@ -67758,7 +67897,7 @@ var init_commons_routes = __esm({
67758
67897
  const publication = await getCommonsPublicationBySubdomain(c.req.param("subdomain"));
67759
67898
  if (!publication) return c.json({ ok: false, error: "publication_not_found", message: "No Commons publication matched that name." }, 404);
67760
67899
  const editions = c.req.query("includeEditions") === "false" ? [] : await listCommonsPublicationEditions(publication.id);
67761
- return c.json({ ok: true, data: { publication, editions } });
67900
+ return c.json({ ok: true, data: { publication, editions: withArticleSources(editions, false) } });
67762
67901
  } catch (error) {
67763
67902
  return publicationError(c, error);
67764
67903
  }