mcpscraper-sdk 0.51.0 → 0.53.0

package/README.md

@@ -4,7 +4,7 @@ Official TypeScript/JavaScript client for the [mcpscraper.dev](https://mcpscrape
 
 [Release history](https://github.com/VilovietaSEO/mcpscraper-sdk/blob/main/CHANGELOG.md)
 
-This is a thin HTTP client generated against [`../../contracts/scraper.openapi.yaml`](../../contracts/scraper.openapi.yaml), the public contract for the hosted API. It contains no scraping, proxy, or billing logic — only typed request/response plumbing.
+This is a thin HTTP client generated against [`../../contracts/scraper.openapi.generated.json`](../../contracts/scraper.openapi.generated.json), the complete public REST contract for the hosted API. It contains no scraping, proxy, or billing logic — only typed request/response plumbing.
 
 ## Install
 
@@ -41,11 +41,11 @@ Every non-2xx response throws a `ScraperApiError` with `status`, `code`, and the
 - `err.isConcurrencyLimitExceeded()` — narrows `err.body` to `{ active, limit, retryable, upgrade, ... }`.
 - `err.isVerificationChallenge()` / `err.isTimeout()` — preserve actionable, vendor-neutral retry and charge details returned by the service.
 
-Google Search light mode returns organic positions, URLs, titles, and descriptions for 20 Credits per delivered SERP page, or 35 Credits when the backup supplies the result. One-page full mode adds available same-page SERP features for 35 Credits. Both modes default to one page; an unfiltered two-page search costs 40 Credits when the normal provider returns both pages. Past-week and past-month searches use a managed browser for organic results at 35 Credits per delivered page, and unfiltered searches use it only as the final backup. If page 2 cannot be delivered, an available page 1 is returned as partial and billed at the one-page rate. A 35-Credit-per-requested-page hold is settled after delivery. PAA costs 400 Credits plus 10 Credits per returned question. One optional concurrency pack adds two browser slots for $5/month; quantity `n` adds `2n` slots for `$5n` without changing the base plan.
+Google Search light mode returns organic positions, URLs, titles, and descriptions for 20 Credits per delivered SERP page, or 35 Credits when the backup supplies the result. One-page full mode adds available same-page SERP features for 35 Credits. Both modes default to one page; an unfiltered two-page search costs 40 Credits when the normal provider returns both pages. Past-week and past-month searches use a managed browser for organic results at 35 Credits per delivered page. Unfiltered light searches try the primary path twice, then one API fallback; an unfiltered one-page full search uses the feature-capable API directly. If page 2 cannot be delivered, an available page 1 is returned as partial and billed at the one-page rate. A 35-Credit-per-requested-page hold is settled after delivery. PAA costs 400 Credits plus 10 Credits per returned question. One optional concurrency pack adds two browser slots for $5/month; quantity `n` adds `2n` slots for `$5n` without changing the base plan.
 
 ## API surface
 
-`client.tools` is the generated, typed 364-tool MCP surface from `contracts/mcp.tools.json`, including provider-neutral X-Ray surveys, attribution-impact reporting, and truthful evidence-status operations.
+`client.tools` is the generated, typed 367-tool MCP surface from `contracts/mcp.tools.json`, including provider-neutral X-Ray surveys, attribution-impact reporting, and truthful evidence-status operations.
 
 For multimodal results such as `meta_ad_creative_media`, call `client.tools.callToolResult(...)` to preserve native MCP image/audio/resource blocks. `callTool(...)` remains backward-compatible and returns the parsed structured or text value.
 
@@ -181,14 +181,21 @@ Use `mcpscraper-memory-sdk`'s `MemoryClient` when you prefer its Memory-first na
 
 ## Regenerating types
 
-`src/schema.ts` is generated from the OpenAPI spec and checked in. After editing `../../contracts/scraper.openapi.yaml`, regenerate with:
+`src/schema.ts` is generated from the complete OpenAPI projection and checked in. For server-owned Maps search and directory routes, update `contracts/server-rest.v1.json` from the server artifact. Other public routes remain in the curated `contracts/scraper.openapi.yaml` base. Regenerate both language models from the repository root:
 
 ```bash
-npm run generate
+npm run generate:rest
+npm run verify:rest-contract
 ```
 
 ## See also
 
 [Repo README](../../README.md) (multi-language examples with real sample output) · [`mcpscraper-memory-sdk`](../memory) (Node, full 121-tool direct-memory surface) · [`mcpscraper-sdk` on PyPI](https://pypi.org/project/mcpscraper-sdk/) · [`mcpscraper-cli`](../cli)
 
-Release changes: [SDK 0.51.0 release notes](https://github.com/VilovietaSEO/mcpscraper-sdk/releases/tag/v0.51.0).
+Release changes: [SDK 0.53.0 release notes](https://github.com/VilovietaSEO/mcpscraper-sdk/releases/tag/v0.53.0).
+
+## Client attribution and full results
+
+Clients accept `clientAttribution: { installChannel?, aiClient?, aiModel? }` and default to calling surface `sdk`. Set only known caller labels; unspecified labels stay unknown. `onResponse({ requestId, status })` receives per-response correlation without changing results, and errors keep their own `requestId`. These options also apply to `McpToolsClient` and `MemoryClient`; labels are reporting declarations, not authorization.
+
+Large tool outputs may be delivery envelopes instead of inline domain values. Preserve `delivery`, `scope` and `recovery`, then use `tools.artifacts.describe`, `readRetained` (follow `nextOffset` to null), or `download` with the returned artifact ID. The legacy `artifacts.read` method remains the report reader. `callToolResult` preserves native content/resource links. Download URL expiry differs from artifact retention; renew a URL through `download` without rerunning acquisition. Use the returned capability for binary files.

package/dist/index.cjs

@@ -72,7 +72,29 @@ var PUBLIC_ERROR_CODES = /* @__PURE__ */ new Set([
   "site_export_not_found",
   "site_export_format_unavailable",
   "site_export_read_failed",
-  "site_export_image_not_found"
+  "site_export_image_not_found",
+  "directory_job_not_found",
+  "idempotency_in_progress",
+  "idempotency_conflict",
+  "image_url_invalid",
+  "image_url_not_https",
+  "image_host_unresolvable",
+  "image_host_blocked",
+  "image_redirect_invalid",
+  "image_source_unavailable",
+  "image_too_large",
+  "image_redirect_limit",
+  "image_base64_invalid",
+  "image_source_missing",
+  "image_source_ambiguous",
+  "image_source_type_invalid",
+  "image_artifact_not_found",
+  "image_page_too_large",
+  "page_image_not_found",
+  "page_image_candidates_unusable",
+  "image_format_unsupported",
+  "image_storage_unconfigured",
+  "featured_image_unhostable"
 ]);
 var PUBLIC_ERROR_FIELDS = /* @__PURE__ */ new Set([
   "error",
@@ -103,13 +125,51 @@ var PUBLIC_DETAIL_FIELDS = /* @__PURE__ */ new Set([
   "slots_per_pack",
   "pack_price_usd",
   "balance_credits",
-  "required_credits"
+  "required_credits",
+  "retry_guidance",
+  "http_status",
+  "job_id"
 ]);
+function boundedString(value, limit) {
+  return typeof value === "string" && value.length > 0 && value.length <= limit;
+}
+function publicRecoveryFields(body) {
+  if (!body || typeof body !== "object" || Array.isArray(body)) return {};
+  const row = body;
+  const safe = {};
+  if (boundedString(row.idempotencyKey, 512)) safe.idempotencyKey = row.idempotencyKey;
+  if (typeof row.idempotencyFingerprint === "string" && /^[a-f0-9]{16}$/.test(row.idempotencyFingerprint)) safe.idempotencyFingerprint = row.idempotencyFingerprint;
+  const operation = row.operation;
+  if (operation && typeof operation === "object" && !Array.isArray(operation)) {
+    const op = operation;
+    if ((op.runId === null || boundedString(op.runId, 200)) && boundedString(op.idempotencyKey, 512) && boundedString(op.state, 100)) {
+      safe.operation = { runId: op.runId, idempotencyKey: op.idempotencyKey, state: op.state };
+    }
+  }
+  const recovery = row.recovery;
+  if (recovery && typeof recovery === "object" && !Array.isArray(recovery)) {
+    const receipt = recovery;
+    if (typeof receipt.jobId === "string" && /^[a-zA-Z0-9-]{1,100}$/.test(receipt.jobId) && typeof receipt.statusTool === "string" && /^[a-z][a-z0-9_]{0,99}$/.test(receipt.statusTool) && receipt.executionState === "unknown" && ["held", "unconfirmed"].includes(String(receipt.holdState))) {
+      safe.recovery = { jobId: receipt.jobId, statusTool: receipt.statusTool, executionState: receipt.executionState, holdState: receipt.holdState };
+    }
+  }
+  if (typeof row.job_id === "string" && /^[a-zA-Z0-9-]{1,100}$/.test(row.job_id)) safe.job_id = row.job_id;
+  if (typeof row.status_tool === "string" && /^[a-z][a-z0-9_]{0,99}$/.test(row.status_tool)) safe.status_tool = row.status_tool;
+  if (row.executionState === "unknown") safe.executionState = row.executionState;
+  if (row.hold_state === "held" || row.hold_state === "unconfirmed") safe.hold_state = row.hold_state;
+  if (typeof row.runId === "string" && /^[a-zA-Z0-9_-]{1,200}$/.test(row.runId)) {
+    safe.runId = row.runId;
+    if (["queued", "running", "interrupted", "complete", "partial", "failed", "cancelled"].includes(String(row.runStatus))) safe.runStatus = row.runStatus;
+    if (["unreserved", "held", "settling", "settled", "refunded", "pending_reconciliation"].includes(String(row.billingState))) safe.billingState = row.billingState;
+    if (row.statusPath === `/maps/place/runs/${row.runId}`) safe.statusPath = row.statusPath;
+  }
+  return safe;
+}
 function recognizedPublicDetails(value) {
   if (!value || typeof value !== "object" || Array.isArray(value)) return void 0;
   const details = Object.fromEntries(
     Object.entries(value).filter(
-      ([key, child]) => PUBLIC_DETAIL_FIELDS.has(key) && (child === null || ["string", "number", "boolean"].includes(typeof child))
+      ([key, child]) => PUBLIC_DETAIL_FIELDS.has(key) && (child === null || typeof child === "boolean" || typeof child === "number" && Number.isFinite(child) || typeof child === "string" && child.length <= (key === "job_id" ? 200 : 500))
     )
   );
   return Object.keys(details).length > 0 ? details : void 0;
@@ -119,10 +179,13 @@ function recognizedPublicErrorBody(body) {
   const record = body;
   const code = readString(record, "error_code");
   if (!code || !PUBLIC_ERROR_CODES.has(code)) return void 0;
-  const safe = Object.fromEntries(Object.entries(record).filter(([key]) => PUBLIC_ERROR_FIELDS.has(key)));
+  const safe = Object.fromEntries(Object.entries(record).filter(([key, value]) => PUBLIC_ERROR_FIELDS.has(key) && (value === null || typeof value === "boolean" || typeof value === "number" && Number.isFinite(value) || typeof value === "string" && value.length <= 1e3)));
+  Object.assign(safe, publicRecoveryFields(record));
   const details = recognizedPublicDetails(record.details);
   if (details) safe.details = details;
   else delete safe.details;
+  const upgrade = recognizedPublicDetails(record.upgrade);
+  if (upgrade) safe.upgrade = upgrade;
   return safe;
 }
 function publicErrorBody(status, body) {
@@ -135,20 +198,23 @@ function publicErrorBody(status, body) {
     error_type: "service_unavailable",
     retryable: true,
     status,
-    message: PUBLIC_SERVICE_UNAVAILABLE_MESSAGE
+    message: PUBLIC_SERVICE_UNAVAILABLE_MESSAGE,
+    ...publicRecoveryFields(body)
   };
 }
 var ScraperApiError = class extends Error {
   status;
   code;
   body;
-  constructor(status, body) {
+  requestId;
+  constructor(status, body, details = {}) {
     const safeBody = publicErrorBody(status, body);
     super(readString(safeBody, "message") ?? `mcpscraper.dev API request failed with status ${status}`);
     this.name = "ScraperApiError";
     this.status = status;
     this.code = readString(safeBody, "error_code") ?? readString(safeBody, "error");
     this.body = safeBody;
+    this.requestId = details.requestId;
   }
   isInsufficientBalance() {
     return this.code === "insufficient_balance";
@@ -170,19 +236,29 @@ var ScraperApiError = class extends Error {
 // src/client.ts
 var import_mcpscraper_memory_sdk = require("mcpscraper-memory-sdk");
 var Requester = class {
-  constructor(apiKey, baseUrl, fetchImpl, timeoutMs) {
+  constructor(apiKey, baseUrl, fetchImpl, timeoutMs, metadataOptions = {}) {
     this.apiKey = apiKey;
     this.baseUrl = baseUrl;
     this.fetchImpl = fetchImpl;
     this.timeoutMs = timeoutMs;
+    this.metadataOptions = metadataOptions;
   }
   apiKey;
   baseUrl;
   fetchImpl;
   timeoutMs;
+  metadataOptions;
+  headers(extra = {}) {
+    return { ...(0, import_mcpscraper_memory_sdk.clientAttributionHeaders)(this.metadataOptions.clientAttribution), ...extra, "x-api-key": this.apiKey };
+  }
+  received(res) {
+    (0, import_mcpscraper_memory_sdk.notifyResponse)(this.metadataOptions.onResponse, (0, import_mcpscraper_memory_sdk.responseMetadata)(res));
+  }
   async request(url, init) {
     try {
-      return await this.fetchImpl(url, { ...init, signal: AbortSignal.timeout(this.timeoutMs) });
+      const response = await this.fetchImpl(url, { ...init, signal: AbortSignal.timeout(this.timeoutMs) });
+      this.received(response);
+      return response;
     } catch (error) {
       if (error instanceof DOMException && (error.name === "TimeoutError" || error.name === "AbortError")) {
         throw new ScraperApiError(408, {
@@ -207,14 +283,14 @@ var Requester = class {
     try {
       res = await this.fetchImpl(`${this.baseUrl}${path}`, {
         method,
-        headers: {
-          "x-api-key": this.apiKey,
+        headers: this.headers({
           ...body !== void 0 ? { "content-type": "application/json" } : {},
           ...extraHeaders
-        },
+        }),
         body: body !== void 0 ? JSON.stringify(body) : void 0,
         signal
       });
+      this.received(res);
     } catch (error) {
       if (error instanceof DOMException && (error.name === "TimeoutError" || error.name === "AbortError")) {
         if (options.signal?.aborted && !timeoutSignal.aborted) throw error;
@@ -229,14 +305,14 @@ var Requester = class {
       throw error;
     }
     const data = await res.json().catch(() => void 0);
-    if (!res.ok) throw new ScraperApiError(res.status, data);
+    if (!res.ok) throw new ScraperApiError(res.status, data, { requestId: (0, import_mcpscraper_memory_sdk.responseMetadata)(res).requestId });
     return { data, headers: res.headers };
   }
   async callRaw(method, path) {
-    const res = await this.request(`${this.baseUrl}${path}`, { method, headers: { "x-api-key": this.apiKey } });
+    const res = await this.request(`${this.baseUrl}${path}`, { method, headers: this.headers() });
     if (!res.ok) {
       const data = await res.json().catch(() => void 0);
-      throw new ScraperApiError(res.status, data);
+      throw new ScraperApiError(res.status, data, { requestId: (0, import_mcpscraper_memory_sdk.responseMetadata)(res).requestId });
     }
     return res.arrayBuffer();
   }
@@ -318,8 +394,14 @@ var RedditNamespace = class {
     this.r = r;
   }
   r;
-  thread(params) {
-    return this.r.call("POST", "/reddit/thread", params);
+  thread(params, options = {}) {
+    return this.r.call(
+      "POST",
+      "/reddit/thread",
+      params,
+      options.idempotencyKey === void 0 ? {} : { "Idempotency-Key": options.idempotencyKey },
+      options
+    );
   }
 };
 var VideoNamespace = class {
@@ -340,7 +422,8 @@ var MapsNamespace = class {
   }
   r;
   search(params, options = {}) {
-    return this.r.call("POST", "/maps/search", params, {}, options);
+    const headers = options.idempotencyKey ? { "Idempotency-Key": options.idempotencyKey } : {};
+    return this.r.call("POST", "/maps/search", params, headers, options);
   }
   place(params, options = {}) {
     const headers = {
@@ -386,8 +469,17 @@ var DirectoryNamespace = class {
     this.r = r;
   }
   r;
-  run(params) {
-    return this.r.call("POST", "/directory/run", params);
+  run(params, options = {}) {
+    return this.r.call(
+      "POST",
+      "/directory/run",
+      params,
+      options.idempotencyKey ? { "Idempotency-Key": options.idempotencyKey } : {},
+      options
+    );
+  }
+  status(jobId, options = {}) {
+    return this.r.call("GET", `/directory/jobs/${encodeURIComponent(jobId)}`, void 0, {}, options);
   }
 };
 var SerpIntelligenceNamespace = class {
@@ -412,7 +504,8 @@ var SerpIntelligenceNamespace = class {
     );
     return {
       data: result.data,
-      idempotencyKey: result.headers.get("Idempotency-Key")
+      idempotencyKey: result.headers.get("Idempotency-Key"),
+      ...result.headers.get("x-mcp-request-id") ? { requestId: result.headers.get("x-mcp-request-id") } : {}
     };
   }
   pageSnapshots(params) {
@@ -552,7 +645,7 @@ var ScraperClient = class {
   constructor(options) {
     const requestTimeoutMs = options.requestTimeoutMs ?? 59e4;
     if (!Number.isFinite(requestTimeoutMs) || requestTimeoutMs <= 0) throw new TypeError("requestTimeoutMs must be a positive number");
-    this.r = new Requester(options.apiKey, options.baseUrl ?? "https://mcpscraper.dev", options.fetch ?? globalThis.fetch, requestTimeoutMs);
+    this.r = new Requester(options.apiKey, options.baseUrl ?? "https://mcpscraper.dev", options.fetch ?? globalThis.fetch, requestTimeoutMs, options);
     this.youtube = new YoutubeNamespace(this.r);
     this.screenshot = new ScreenshotNamespace(this.r);
     this.facebook = new FacebookNamespace(this.r);
@@ -570,17 +663,19 @@ var ScraperClient = class {
       apiKey: options.apiKey,
       baseUrl: options.baseUrl ?? "https://mcpscraper.dev",
       fetch: options.fetch,
-      requestTimeoutMs
+      requestTimeoutMs,
+      clientAttribution: options.clientAttribution,
+      onResponse: options.onResponse
     });
   }
   async callMemoryTool(toolName, args) {
-    const result = await this.r.call("POST", "/memory/mcp-call", {
+    const { data: result, headers } = await this.r.callWithReceipt("POST", "/memory/mcp-call", {
       toolName,
       args: args ?? {}
     });
     if (result && typeof result === "object" && result.ok === false) {
       const failure = result;
-      throw new ScraperApiError(200, { message: failure.error, ...failure });
+      throw new ScraperApiError(200, { message: failure.error, ...failure }, { requestId: headers.get("x-mcp-request-id") ?? void 0 });
     }
     return result;
   }