mcp-scraper 0.2.14 → 0.2.16

package/README.md

@@ -8,7 +8,7 @@ Use the MCPB Desktop Extension for the branded Claude Desktop install, or use th
 
 MCP Scraper ships three local stdio entrypoints plus human-facing helper CLIs:
 
-- `mcp-scraper` — live web intelligence, SERP, PAA, site extraction, YouTube, Facebook, Maps, directory, rank tracker blueprint, and credit tools.
+- `mcp-scraper` — live web intelligence, SERP, PAA, site extraction, YouTube, Facebook ads and organic video transcripts, Maps, directory, rank tracker blueprint, and credit tools.
 - `browser-agent` — an agent-controlled live cloud browser with screenshots, clicks, typing, scrolling, live watch URLs, replay links, and MP4 replay download.
 - `mcp-scraper-combined` — one local MCP server that exposes both tool sets. This is the entrypoint used by the MCPB Desktop Extension.
 - `mcp-scraper-install` — a human-facing terminal installer card with the branded ASCII intro and copyable install commands. This command is safe to print because it is not an MCP stdio server.
@@ -81,7 +81,7 @@ Build the branded one-click bundle:
 npm run build:mcpb
 ```
 
-The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.2.13`, SHA-256 `93c3c35b6a3315a1b9319761891cf0e805465b1ab19efa0252c7b8f2d9383030`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
+The generated bundle is written to `build/mcpb/mcp-scraper-<version>.mcpb` and copied to `public/downloads/` for the hosted download. The current public bundle is `https://mcpscraper.dev/downloads/mcp-scraper.mcpb` (`0.2.16`, SHA-256 `a581f0ddb12d3cebd03d90dfb657cc9127a1f7ef995fb0934d5428faf8aec58b`). Install it by opening or dragging it into Claude Desktop. Claude displays the `MCP Scraper` install card, icon, and API-key configuration field from the bundle manifest.
 
 The MCPB install exposes the same web-intelligence tools as `mcp-scraper` plus all `browser_*` tools from `browser-agent` through one server.
 
@@ -154,7 +154,8 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
 - `youtube_transcribe`
 - `facebook_ad_search`
 - `facebook_page_intel`
-- `facebook_ad_transcribe`
+- `facebook_ad_transcribe` — transcribe a direct Facebook ad video URL returned by `facebook_page_intel`.
+- `facebook_video_transcribe` — transcribe an organic Facebook reel, video, watch, post, or share URL, including `fb.watch` links. The tool renders the page, extracts the best matching public Facebook CDN MP4 URL, then returns transcript text, timestamped chunks, selected quality, video metadata, and the extracted MP4 URL for follow-up download.
 - `maps_search` — search Google Maps for multiple business/profile candidates. Use for GMB/GBP prospect lists, competitors, categories, and anything needing more than the Google 3-pack. In default `proxyMode: "location"`, retryable failures rotate to a new residential proxy and new browser session for up to 5 attempts. `maxResults` defaults to 10 and is capped at 50.
 - `maps_place_intel` — hydrate one known/named Google Maps business with profile details and optional reviews. Use after `maps_search` when a selected candidate needs full details.
 - `directory_workflow` — build city-by-city directory/prospecting datasets from Census place selection plus Google Maps searches. Use it for requests like "all cities over 100k population in Tennessee, then get 20 roofers from Maps." In default `proxyMode: "location"`, each city search rotates retryable failures to a new residential proxy and new browser session for up to 5 attempts. The saved CSV includes `source_location`, `result_position`, `business_name`, `review_stars`, `review_count`, `category`, `address`, `phone`, `hours_status`, `website_url`, `directions_url`, `place_url`, `cid`, `cid_decimal`, Census population, and ZIP groups.
@@ -189,9 +190,9 @@ For Google Maps tools (`maps_search` and `directory_workflow`), keep `proxyMode`
 
 The MCPB bundle and `mcp-scraper-combined` expose both sections through one local MCP server. The split `mcp-scraper` entrypoint exposes only the web-intelligence tools, and the split `browser-agent` entrypoint exposes only the browser-agent tools.
 
-Chaining and planning tools (`maps_search`, `directory_workflow`, `rank_tracker_blueprint`, `map_site_urls`, `youtube_harvest`, `facebook_ad_search`, `facebook_page_intel`) advertise an `outputSchema` and return `structuredContent` with the IDs, URLs, CSV paths, or blueprint fields needed by the next step. All tools carry MCP annotations (`readOnlyHint: true`; live-web tools use `openWorldHint: true`, while local planning and credit tools are closed-world/idempotent).
+Chaining and planning tools (`maps_search`, `directory_workflow`, `rank_tracker_blueprint`, `map_site_urls`, `youtube_harvest`, `facebook_ad_search`, `facebook_page_intel`, `facebook_video_transcribe`) advertise an `outputSchema` and return `structuredContent` with the IDs, URLs, CSV paths, transcripts, or blueprint fields needed by the next step. All tools carry MCP annotations (`readOnlyHint: true`; live-web tools use `openWorldHint: true`, while local planning and credit tools are closed-world/idempotent).
 
-The hosted MCP endpoint at `https://mcpscraper.dev/mcp` exposes the 15 web-intelligence tools plus `capture_serp_snapshot` and `capture_serp_page_snapshots` (17 total). Browser-agent tools are local stdio tools backed by the REST API under `https://mcpscraper.dev/agent/*`.
+The hosted MCP endpoint at `https://mcpscraper.dev/mcp` exposes the 16 web-intelligence tools plus `capture_serp_snapshot` and `capture_serp_page_snapshots` (18 total). Browser-agent tools are local stdio tools backed by the REST API under `https://mcpscraper.dev/agent/*`.
 
 ## Resources
 

package/dist/bin/api-server.cjs

@@ -11618,6 +11618,132 @@ var init_FacebookAdGraphql = __esm({
   }
 });
 
+// src/extractor/FacebookOrganicVideoExtractor.ts
+function htmlDecode(value) {
+  return value.replace(/&amp;/g, "&").replace(/&quot;/g, '"').replace(/&#039;/g, "'").replace(/&lt;/g, "<").replace(/&gt;/g, ">");
+}
+function decodeEscapedValue(raw) {
+  let out = raw;
+  try {
+    out = JSON.parse(`"${raw.replace(/"/g, '\\"')}"`);
+  } catch {
+    out = raw.replace(/\\u0025/g, "%").replace(/\\u0026/g, "&").replace(/\\u003D/g, "=").replace(/\\u003d/g, "=").replace(/\\"/g, '"').replace(/\\\//g, "/");
+  }
+  return htmlDecode(out);
+}
+function facebookVideoIdFromUrl(url) {
+  try {
+    const parsed = new URL(url);
+    const path6 = parsed.pathname;
+    const direct = path6.match(/\/(?:reel|videos|watch)\/(?:[^/]+\/)?(\d{8,})/i);
+    if (direct) return direct[1];
+    const queryVideoId = parsed.searchParams.get("v") ?? parsed.searchParams.get("video_id");
+    if (queryVideoId && /^\d{8,}$/.test(queryVideoId)) return queryVideoId;
+    const numericTail = path6.match(/\/(\d{8,})\/?$/);
+    if (numericTail) return numericTail[1];
+  } catch {
+  }
+  return null;
+}
+function parseEfg(url) {
+  try {
+    const efg = new URL(url).searchParams.get("efg");
+    if (!efg) return null;
+    return JSON.parse(Buffer.from(efg, "base64").toString("utf8"));
+  } catch {
+    return null;
+  }
+}
+function manifestDurationSec(htmlWindow) {
+  const decoded = decodeEscapedValue(htmlWindow);
+  const match = decoded.match(/mediaPresentationDuration="PT([\d.]+)S"/);
+  return match ? Number(match[1]) : null;
+}
+function metadataContent(html, key) {
+  const escaped = key.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+  const re = new RegExp(`<meta\\s+[^>]*(?:property|name)=["']${escaped}["'][^>]*content=["']([^"']+)["'][^>]*>`, "i");
+  const m = html.match(re);
+  return m ? htmlDecode(m[1]).trim() : null;
+}
+function ownerNameFromHtml(html) {
+  const ogTitle = metadataContent(html, "og:title");
+  const pipe = ogTitle?.split("|").map((s) => s.trim()).filter(Boolean);
+  if (pipe && pipe.length > 1) return pipe[pipe.length - 1] ?? null;
+  return null;
+}
+function extractFacebookOrganicVideo(html, pageUrl, sourceUrl = pageUrl, quality = "best") {
+  const videoId = facebookVideoIdFromUrl(pageUrl) ?? facebookVideoIdFromUrl(sourceUrl);
+  const manifestIndex = videoId ? html.indexOf(`dash_mpd_debug.mpd?v=${videoId}`) : -1;
+  const searchWindow = manifestIndex >= 0 ? html.slice(Math.max(0, manifestIndex - 5e4), Math.min(html.length, manifestIndex + 5e4)) : html;
+  const targetDuration = manifestDurationSec(searchWindow);
+  const candidates = [];
+  const seen = /* @__PURE__ */ new Set();
+  const re = /browser_native_(sd|hd)_url\\?"\s*:\s*\\?"([^"<]+?)\\?"/g;
+  let match;
+  while ((match = re.exec(searchWindow)) !== null) {
+    const url = decodeEscapedValue(match[2]);
+    if (!url || seen.has(url)) continue;
+    seen.add(url);
+    const parsed = parseEfg(url);
+    let bitrate = null;
+    try {
+      const value = new URL(url).searchParams.get("bitrate");
+      bitrate = value ? Number(value) : null;
+    } catch {
+    }
+    candidates.push({
+      url,
+      quality: match[1],
+      bitrate: Number.isFinite(bitrate) ? bitrate : null,
+      durationSec: typeof parsed?.duration_s === "number" ? parsed.duration_s : null,
+      assetId: typeof parsed?.xpv_asset_id === "number" ? parsed.xpv_asset_id : null,
+      vencodeTag: typeof parsed?.vencode_tag === "string" ? parsed.vencode_tag : null
+    });
+  }
+  const targetCandidates = candidates.filter((candidate) => {
+    if (!targetDuration || candidate.durationSec == null) return true;
+    return Math.abs(candidate.durationSec - targetDuration) <= 2;
+  });
+  const selectable = targetCandidates.length ? targetCandidates : candidates;
+  const selected = [...selectable].filter((candidate) => quality === "best" || candidate.quality === quality).sort((a, b) => {
+    const aq = a.quality === "hd" ? 1 : 0;
+    const bq = b.quality === "hd" ? 1 : 0;
+    if (quality === "best" && aq !== bq) return bq - aq;
+    return (b.bitrate ?? 0) - (a.bitrate ?? 0);
+  })[0];
+  if (!selected) {
+    throw new Error("No progressive Facebook video URL was found in the rendered page state");
+  }
+  return {
+    sourceUrl,
+    pageUrl,
+    videoId,
+    title: metadataContent(html, "og:title") ?? metadataContent(html, "twitter:title"),
+    description: metadataContent(html, "og:description") ?? metadataContent(html, "description") ?? metadataContent(html, "twitter:description"),
+    ownerName: ownerNameFromHtml(html),
+    videoUrl: selected.url,
+    selectedQuality: selected.quality,
+    bitrate: selected.bitrate,
+    durationSec: targetDuration ?? selected.durationSec,
+    assetId: selected.assetId,
+    vencodeTag: selected.vencodeTag,
+    candidates: selectable,
+    extractedAt: (/* @__PURE__ */ new Date()).toISOString()
+  };
+}
+async function extractFacebookOrganicVideoFromPage(page, sourceUrl, quality = "best") {
+  await page.waitForLoadState("domcontentloaded", { timeout: 15e3 }).catch(() => {
+  });
+  await page.waitForTimeout(1500);
+  const html = await page.content();
+  return extractFacebookOrganicVideo(html, page.url(), sourceUrl, quality);
+}
+var init_FacebookOrganicVideoExtractor = __esm({
+  "src/extractor/FacebookOrganicVideoExtractor.ts"() {
+    "use strict";
+  }
+});
+
 // src/locations.ts
 var LOCATIONS;
 var init_locations = __esm({
@@ -12161,6 +12287,39 @@ var init_kernel_proxy_resolver = __esm({
 function invalidRequest(message) {
   return { error_code: "invalid_request", message };
 }
+function isAllowedFacebookOrganicUrl(url) {
+  const hostname = url.hostname.toLowerCase().replace(/^www\./, "").replace(/^m\./, "");
+  return hostname === "facebook.com" || hostname === "fb.watch";
+}
+function facebookTranscriptMarkdown(title, text, chunks, durationMs) {
+  const fmtTs2 = (s) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, "0")}`;
+  const lines = [title, "", `*Transcribed in ${(durationMs / 1e3).toFixed(1)}s*`, "", "## Full Text", "", text, ""];
+  if (chunks.length) {
+    lines.push("## Timestamped Segments", "");
+    for (const ch of chunks) {
+      lines.push(`**[${fmtTs2(ch.timestamp[0])} -> ${fmtTs2(ch.timestamp[1])}]** ${ch.text.trim()}`, "");
+    }
+  }
+  return lines.join("\n");
+}
+async function transcribeFacebookVideoUrl(videoUrl, markdownTitle = "# Facebook Video Transcript") {
+  const startMs = Date.now();
+  const result = await import_client3.fal.subscribe("fal-ai/wizper", {
+    input: { audio_url: videoUrl, task: "transcribe", language: "en" },
+    logs: false,
+    pollInterval: 3e3
+  });
+  const data = result.data;
+  const text = data.text ?? "";
+  const chunks = data.chunks ?? [];
+  const durationMs = Date.now() - startMs;
+  return {
+    text,
+    chunks,
+    durationMs,
+    markdown: facebookTranscriptMarkdown(markdownTitle, text, chunks, durationMs)
+  };
+}
 async function detectSoftBlock(driver) {
   const page = driver.getPage();
   const bodyText = await page.evaluate(() => document.body?.innerText ?? "").catch(() => "");
@@ -12190,7 +12349,7 @@ async function kernelLaunchOptsResidential() {
   }
   return { headless: true, kernelApiKey: browserServiceApiKey(), kernelProxyId: proxyId, viewport: { width: 1280, height: 900 }, locale: "en-US" };
 }
-var import_hono4, import_zod15, import_client3, FacebookAdBodySchema, FacebookPageIntelBodySchema, FacebookTranscribeBodySchema, FacebookSearchBodySchema, FacebookMediaBodySchema, facebookAdApp, ALLOWED_MEDIA_HOSTS;
+var import_hono4, import_zod15, import_client3, FacebookAdBodySchema, FacebookPageIntelBodySchema, FacebookTranscribeBodySchema, FacebookVideoTranscribeBodySchema, FacebookSearchBodySchema, FacebookMediaBodySchema, facebookAdApp, ALLOWED_MEDIA_HOSTS;
 var init_facebook_ad_routes = __esm({
   "src/api/facebook-ad-routes.ts"() {
     "use strict";
@@ -12202,6 +12361,7 @@ var init_facebook_ad_routes = __esm({
     init_BrowserDriver();
     init_FacebookAdExtractor();
     init_FacebookAdGraphql();
+    init_FacebookOrganicVideoExtractor();
     init_kernel_proxy_resolver();
     import_client3 = require("@fal-ai/client");
     init_api_auth();
@@ -12224,6 +12384,10 @@ var init_facebook_ad_routes = __esm({
     FacebookTranscribeBodySchema = import_zod15.z.object({
       videoUrl: import_zod15.z.string().trim().min(1, "videoUrl is required")
     });
+    FacebookVideoTranscribeBodySchema = import_zod15.z.object({
+      url: import_zod15.z.string().trim().min(1, "url is required"),
+      quality: import_zod15.z.enum(["best", "hd", "sd"]).default("best")
+    });
     FacebookSearchBodySchema = import_zod15.z.object({
       query: import_zod15.z.string().trim().min(1, "query is required"),
       country: import_zod15.z.string().trim().toUpperCase().optional(),
@@ -12347,26 +12511,9 @@ var init_facebook_ad_routes = __esm({
         const { ok, balance_mc } = await debitMc(fbUser.id, MC_COSTS.fb_transcribe, LedgerOperation.FB_TRANSCRIBE, videoUrl);
         if (!ok) return c.json(insufficientBalanceResponse(balance_mc, MC_COSTS.fb_transcribe), 402);
         debited = true;
-        const startMs = Date.now();
-        const result = await import_client3.fal.subscribe("fal-ai/wizper", {
-          input: { audio_url: videoUrl, task: "transcribe", language: "en" },
-          logs: false,
-          pollInterval: 3e3
-        });
-        const data = result.data;
-        const text = data.text ?? "";
-        const chunks = data.chunks ?? [];
-        const durationMs = Date.now() - startMs;
-        const fmtTs2 = (s) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, "0")}`;
-        const lines = ["# Facebook Ad Transcript", "", `*Transcribed in ${(durationMs / 1e3).toFixed(1)}s*`, "", "## Full Text", "", text, ""];
-        if (chunks.length) {
-          lines.push("## Timestamped Segments", "");
-          for (const ch of chunks) {
-            lines.push(`**[${fmtTs2(ch.timestamp[0])} \u2192 ${fmtTs2(ch.timestamp[1])}]** ${ch.text.trim()}`, "");
-          }
-        }
-        await logRequestEvent({ userId: fbUser.id, source: "facebook_transcribe", status: "done", query: videoUrl, resultCount: chunks.length, result: { text, chunks, durationMs } });
-        return c.json({ text, chunks, durationMs, markdown: lines.join("\n") });
+        const transcript = await transcribeFacebookVideoUrl(videoUrl, "# Facebook Ad Transcript");
+        await logRequestEvent({ userId: fbUser.id, source: "facebook_transcribe", status: "done", query: videoUrl, resultCount: transcript.chunks.length, result: { text: transcript.text, chunks: transcript.chunks, durationMs: transcript.durationMs } });
+        return c.json(transcript);
       } catch (err) {
         const msg = err instanceof Error ? err.message : String(err);
         if (debited) await creditMc(fbUser.id, MC_COSTS.fb_transcribe, LedgerOperation.FB_TRANSCRIBE_REFUND, "failed call");
@@ -12376,6 +12523,62 @@ var init_facebook_ad_routes = __esm({
         await releaseConcurrencyGate(gate.lockId);
       }
     });
+    facebookAdApp.post("/video-transcribe", createApiKeyAuth(), async (c) => {
+      const raw = await c.req.json().catch(() => ({}));
+      const parsed = FacebookVideoTranscribeBodySchema.safeParse(raw);
+      if (!parsed.success) {
+        return c.json(invalidRequest(parsed.error.issues[0]?.message ?? "Invalid request"), 400);
+      }
+      const body = parsed.data;
+      const urlCheck = await validatePublicHttpUrl(body.url, { field: "url", requireHttps: false });
+      if (urlCheck.error) {
+        return c.json(invalidRequest(urlCheck.error), 400);
+      }
+      const sourceUrl = urlCheck.parsed;
+      if (!isAllowedFacebookOrganicUrl(sourceUrl)) {
+        return c.json(invalidRequest("url must be a Facebook organic video, reel, or share URL"), 400);
+      }
+      const fbUser = c.get("user");
+      const gate = await acquireConcurrencyGate(fbUser, "facebook_video_transcribe", {
+        reuseLockId: c.req.header("x-mcp-scraper-concurrency-lock"),
+        metadata: { url: sourceUrl.href }
+      });
+      if (!gate.ok) return c.json(concurrencyLimitExceededResponse(gate), 429, { "Retry-After": String(gate.retryAfterSeconds) });
+      import_client3.fal.config({ credentials: process.env.FAL_KEY });
+      const driver = new BrowserDriver();
+      let debited = false;
+      try {
+        const { ok, balance_mc } = await debitMc(fbUser.id, MC_COSTS.fb_transcribe, LedgerOperation.FB_TRANSCRIBE, sourceUrl.href);
+        if (!ok) return c.json(insufficientBalanceResponse(balance_mc, MC_COSTS.fb_transcribe), 402);
+        debited = true;
+        await driver.launch(await kernelLaunchOptsResidential());
+        await driver.navigateTo(sourceUrl.href);
+        const page = driver.getPage();
+        const video = await extractFacebookOrganicVideoFromPage(page, sourceUrl.href, body.quality);
+        const transcript = await transcribeFacebookVideoUrl(video.videoUrl, "# Facebook Organic Video Transcript");
+        const result = {
+          ...video,
+          videoDurationSec: video.durationSec,
+          text: transcript.text,
+          chunks: transcript.chunks,
+          durationMs: transcript.durationMs,
+          markdown: transcript.markdown
+        };
+        await logRequestEvent({ userId: fbUser.id, source: "facebook_video_transcribe", status: "done", query: sourceUrl.href, resultCount: transcript.chunks.length, result });
+        return c.json(result);
+      } catch (err) {
+        const msg = err instanceof Error ? err.message : String(err);
+        if (debited) await creditMc(fbUser.id, MC_COSTS.fb_transcribe, LedgerOperation.FB_TRANSCRIBE_REFUND, "failed call");
+        await logRequestEvent({ userId: fbUser.id, source: "facebook_video_transcribe", status: "failed", query: sourceUrl.href, error: msg });
+        if (msg.toLowerCase().includes("blocked") || msg.toLowerCase().includes("captcha")) {
+          return c.json({ error: msg }, 503);
+        }
+        return c.json({ error: msg }, 500);
+      } finally {
+        await driver.close();
+        await releaseConcurrencyGate(gate.lockId);
+      }
+    });
     facebookAdApp.post("/search", createApiKeyAuth(), async (c) => {
       const raw = await c.req.json().catch(() => ({}));
       const parsed = FacebookSearchBodySchema.safeParse(raw);
@@ -14153,7 +14356,8 @@ ${adBlocks}`,
     `
 ---
 \u{1F4A1} **Tips**
-- Transcribe video ads: use \`facebook_ad_transcribe\` with the \`videoUrl\` above
+- Transcribe video ads: use \`facebook_ad_transcribe\` with the direct \`videoUrl\` above
+- Transcribe organic Facebook reels/posts: use \`facebook_video_transcribe\` with the public Facebook URL
 - Find other advertisers: use \`facebook_ad_search\``
   ].filter(Boolean).join("\n");
   return {
@@ -14599,10 +14803,66 @@ ${text}`,
 ${chunkRows}` : "",
     `
 ---
-\u{1F4A1} Get more ads from this advertiser: use \`facebook_page_intel\``
+\u{1F4A1} Get more ads from this advertiser: use \`facebook_page_intel\`. For public Facebook reel/post URLs, use \`facebook_video_transcribe\`.`
   ].filter(Boolean).join("\n");
   return oneBlock(full);
 }
+function formatFacebookVideoTranscribe(raw, input) {
+  const parsed = parseData(raw);
+  if ("error" in parsed) return { content: [{ type: "text", text: parsed.error }], isError: true };
+  const d = parsed.data;
+  const text = d.text ?? "";
+  const chunks = d.chunks ?? [];
+  const wordCount = text.trim() ? text.trim().split(/\s+/).length : 0;
+  const durSec = d.durationMs ? (d.durationMs / 1e3).toFixed(0) : "\u2014";
+  const videoDuration = typeof d.videoDurationSec === "number" ? `${Math.round(d.videoDurationSec)}s` : "\u2014";
+  const label = d.videoId ? `Facebook Organic Video \`${d.videoId}\`` : "Facebook Organic Video";
+  const chunkRows = chunks.slice(0, 50).map((c) => {
+    const sec = Number.isFinite(c.timestamp[0]) ? Math.floor(c.timestamp[0]) : 0;
+    const mm = String(Math.floor(sec / 60)).padStart(2, "0");
+    const ss = String(sec % 60).padStart(2, "0");
+    return `| ${mm}:${ss} | ${cell(truncate(c.text, 120))} |`;
+  }).join("\n");
+  const full = [
+    `# ${label} Transcript`,
+    d.ownerName ? `**Owner:** ${d.ownerName}` : "",
+    `**Video duration:** ${videoDuration} \xB7 **Transcribed in:** ${durSec}s \xB7 **${wordCount} words**`,
+    d.pageUrl ? `**Page URL:** ${d.pageUrl}` : `**Page URL:** ${input.url}`,
+    d.videoUrl ? `**Extracted MP4:** \`${d.videoUrl}\`` : "",
+    `
+## Full Transcript
+${text}`,
+    chunks.length ? `
+## Timestamped Chunks
+| Time | Text |
+|------|------|
+${chunkRows}` : "",
+    `
+---
+\u{1F4A1} Use \`videoUrl\` as the extracted MP4 for download or follow-up processing. Use \`facebook_ad_transcribe\` only for Ad Library videoUrl values.`
+  ].filter(Boolean).join("\n");
+  return {
+    ...oneBlock(full),
+    structuredContent: {
+      sourceUrl: d.sourceUrl ?? input.url,
+      pageUrl: d.pageUrl ?? input.url,
+      videoId: d.videoId ?? null,
+      ownerName: d.ownerName ?? null,
+      selectedQuality: d.selectedQuality ?? input.quality ?? "best",
+      bitrate: typeof d.bitrate === "number" ? d.bitrate : null,
+      videoDurationSec: typeof d.videoDurationSec === "number" ? d.videoDurationSec : null,
+      videoUrl: d.videoUrl ?? "",
+      wordCount,
+      chunkCount: chunks.length,
+      transcriptText: text,
+      chunks: chunks.map((c) => ({
+        startSec: Number.isFinite(c.timestamp[0]) ? c.timestamp[0] : 0,
+        endSec: Number.isFinite(c.timestamp[1]) ? c.timestamp[1] : 0,
+        text: c.text
+      }))
+    }
+  };
+}
 var import_node_fs3, import_node_os3, import_node_path5, reportSavingEnabled;
 var init_mcp_response_formatter = __esm({
   "src/mcp/mcp-response-formatter.ts"() {
@@ -19886,12 +20146,12 @@ var PACKAGE_VERSION;
 var init_version = __esm({
   "src/version.ts"() {
     "use strict";
-    PACKAGE_VERSION = "0.2.14";
+    PACKAGE_VERSION = "0.2.16";
   }
 });
 
 // src/mcp/mcp-tool-schemas.ts
-var import_zod26, HarvestPaaInputSchema, ExtractUrlInputSchema, MapSiteUrlsInputSchema, ExtractSiteInputSchema, YoutubeHarvestInputSchema, YoutubeTranscribeInputSchema, FacebookPageIntelInputSchema, FacebookAdSearchInputSchema, FacebookAdTranscribeInputSchema, MapsPlaceIntelInputSchema, MapsSearchInputSchema, DirectoryWorkflowInputSchema, RankTrackerModeSchema, RankTrackerBlueprintInputSchema, NullableString, MapsSearchAttemptOutput, MapsSearchOutputSchema, DirectoryMapsBusinessOutput, DirectoryWorkflowOutputSchema, RankTrackerToolPlanOutput, RankTrackerTableOutput, RankTrackerCronJobOutput, RankTrackerBlueprintOutputSchema, OrganicResultOutput, AiOverviewOutput, EntityIdsOutput, HarvestPaaOutputSchema, SearchSerpOutputSchema, ExtractUrlOutputSchema, ExtractSiteOutputSchema, MapsPlaceIntelOutputSchema, CreditsInfoOutputSchema, MapSiteUrlsOutputSchema, YoutubeHarvestOutputSchema, FacebookAdSearchOutputSchema, FacebookPageIntelOutputSchema, CreditsInfoInputSchema, SearchSerpInputSchema, CaptureSerpSnapshotInputSchema, ScreenshotInputSchema, CaptureSerpPageSnapshotsInputSchema;
+var import_zod26, HarvestPaaInputSchema, ExtractUrlInputSchema, MapSiteUrlsInputSchema, ExtractSiteInputSchema, YoutubeHarvestInputSchema, YoutubeTranscribeInputSchema, FacebookPageIntelInputSchema, FacebookAdSearchInputSchema, FacebookAdTranscribeInputSchema, FacebookVideoTranscribeInputSchema, MapsPlaceIntelInputSchema, MapsSearchInputSchema, DirectoryWorkflowInputSchema, RankTrackerModeSchema, RankTrackerBlueprintInputSchema, NullableString, MapsSearchAttemptOutput, MapsSearchOutputSchema, DirectoryMapsBusinessOutput, DirectoryWorkflowOutputSchema, RankTrackerToolPlanOutput, RankTrackerTableOutput, RankTrackerCronJobOutput, RankTrackerBlueprintOutputSchema, OrganicResultOutput, AiOverviewOutput, EntityIdsOutput, HarvestPaaOutputSchema, SearchSerpOutputSchema, ExtractUrlOutputSchema, ExtractSiteOutputSchema, MapsPlaceIntelOutputSchema, CreditsInfoOutputSchema, MapSiteUrlsOutputSchema, YoutubeHarvestOutputSchema, FacebookAdSearchOutputSchema, FacebookPageIntelOutputSchema, FacebookVideoTranscribeOutputSchema, CreditsInfoInputSchema, SearchSerpInputSchema, CaptureSerpSnapshotInputSchema, ScreenshotInputSchema, CaptureSerpPageSnapshotsInputSchema;
 var init_mcp_tool_schemas = __esm({
   "src/mcp/mcp-tool-schemas.ts"() {
     "use strict";
@@ -19946,7 +20206,11 @@ var init_mcp_tool_schemas = __esm({
       maxResults: import_zod26.z.number().int().min(1).max(20).default(10)
     };
     FacebookAdTranscribeInputSchema = {
-      videoUrl: import_zod26.z.string().url().describe("Facebook CDN video URL from a facebook_page_intel result")
+      videoUrl: import_zod26.z.string().url().describe("Direct Facebook CDN video URL from a facebook_page_intel ad result. Do not pass a public Facebook reel/post/share URL here; use facebook_video_transcribe for organic Facebook URLs.")
+    };
+    FacebookVideoTranscribeInputSchema = {
+      url: import_zod26.z.string().url().describe("Organic Facebook reel, video, watch, post, or share URL from facebook.com, m.facebook.com, or fb.watch. The tool renders the page, extracts the best matching public Facebook CDN MP4 URL, then transcribes it. Use this when the user pastes a normal Facebook video page URL and asks for the transcript or downloadable MP4."),
+      quality: import_zod26.z.enum(["best", "hd", "sd"]).default("best").describe("Preferred progressive MP4 quality. Use best by default; hd prefers the highest HD progressive URL; sd forces the SD URL.")
     };
     MapsPlaceIntelInputSchema = {
       businessName: import_zod26.z.string().min(1).describe('Business name only. If user says "Elite Roofing Denver CO", use businessName="Elite Roofing" and location="Denver, CO".'),
@@ -20313,6 +20577,24 @@ var init_mcp_tool_schemas = __esm({
         variations: import_zod26.z.number().int().nullable()
       }))
     };
+    FacebookVideoTranscribeOutputSchema = {
+      sourceUrl: import_zod26.z.string().url(),
+      pageUrl: import_zod26.z.string().url(),
+      videoId: NullableString,
+      ownerName: NullableString,
+      selectedQuality: import_zod26.z.string(),
+      bitrate: import_zod26.z.number().int().nullable(),
+      videoDurationSec: import_zod26.z.number().nullable(),
+      videoUrl: import_zod26.z.string().url(),
+      wordCount: import_zod26.z.number().int().min(0),
+      chunkCount: import_zod26.z.number().int().min(0),
+      transcriptText: import_zod26.z.string(),
+      chunks: import_zod26.z.array(import_zod26.z.object({
+        startSec: import_zod26.z.number(),
+        endSec: import_zod26.z.number(),
+        text: import_zod26.z.string()
+      }))
+    };
     CreditsInfoInputSchema = {
       item: import_zod26.z.string().optional().describe('Optional tool, action, or feature to look up, e.g. "maps reviews", "extract_url", "YouTube transcription", or "concurrency"'),
       includeLedger: import_zod26.z.boolean().default(false).describe("Whether to include recent credit ledger entries")
@@ -20802,7 +21084,7 @@ function registerPaaExtractorMcpTools(server, executor, options = {}) {
   }, async (input) => formatYoutubeTranscribe(await executor.youtubeTranscribe(input), input));
   server.registerTool("facebook_page_intel", {
     title: "Facebook Advertiser Ad Intel",
-    description: withReportNote("Harvest ads from a Facebook advertiser. Returns ad copy, headlines, CTAs, creative type, status, landing URLs, and video URLs ready for transcription. Accepts pageId, libraryId, or a brand/advertiser name as query. Use after facebook_ad_search when possible."),
+    description: withReportNote("Harvest ads from a Facebook advertiser. Returns ad copy, headlines, CTAs, creative type, status, landing URLs, and direct ad video URLs ready for facebook_ad_transcribe. Accepts pageId, libraryId, or a brand/advertiser name as query. Use after facebook_ad_search when possible. For normal public Facebook reels/posts/watch/share URLs, use facebook_video_transcribe instead."),
     inputSchema: FacebookPageIntelInputSchema,
     outputSchema: FacebookPageIntelOutputSchema,
     annotations: liveWebToolAnnotations("Facebook Advertiser Ad Intel")
@@ -20816,10 +21098,17 @@ function registerPaaExtractorMcpTools(server, executor, options = {}) {
   }, async (input) => formatFacebookAdSearch(await executor.facebookAdSearch(input), input));
   server.registerTool("facebook_ad_transcribe", {
     title: "Facebook Ad Transcription",
-    description: "Transcribe audio from a Facebook ad video. Returns full transcript and timestamped chunks. Use the videoUrl value from facebook_page_intel results.",
+    description: "Transcribe audio from a Facebook ad video CDN URL. Returns full transcript and timestamped chunks. Use only with the direct videoUrl value from facebook_page_intel results, not public Facebook post/reel/share URLs.",
     inputSchema: FacebookAdTranscribeInputSchema,
     annotations: liveWebToolAnnotations("Facebook Ad Transcription")
   }, async (input) => formatFacebookAdTranscribe(await executor.facebookAdTranscribe(input), input));
+  server.registerTool("facebook_video_transcribe", {
+    title: "Facebook Organic Video Transcription",
+    description: withReportNote("Transcribe audio from an organic Facebook reel, video, watch, post, or share URL, including fb.watch links. Use this when the user pastes a normal Facebook video page URL and wants the transcript or downloadable MP4. Renders the Facebook page in a browser, selects the best matching public Facebook CDN MP4 URL from page state, then returns sourceUrl, resolved pageUrl, videoId, ownerName, selectedQuality, bitrate, videoDurationSec, extracted MP4 URL, full transcript, and timestamped chunks."),
+    inputSchema: FacebookVideoTranscribeInputSchema,
+    outputSchema: FacebookVideoTranscribeOutputSchema,
+    annotations: liveWebToolAnnotations("Facebook Organic Video Transcription")
+  }, async (input) => formatFacebookVideoTranscribe(await executor.facebookVideoTranscribe(input), input));
   server.registerTool("maps_place_intel", {
     title: "Google Maps Business Profile Details",
     description: withReportNote('Extract Google Maps business intelligence for one known/named business: rating, review count, category, address, phone, website, hours, booking URL, review histogram, review topics, about attributes, entity IDs, and optional review cards. Do not use this for category searches, local market prospect lists, or requests for multiple GMB/GBP profiles; use maps_search first for those. Split business name from location (e.g. "Elite Roofing Denver CO" => businessName "Elite Roofing", location "Denver, CO"). Pass includeReviews true when the user asks for reviews/customer pain.'),
@@ -20968,6 +21257,9 @@ var init_http_mcp_tool_executor = __esm({
       facebookAdTranscribe(input) {
         return this.call("/facebook/transcribe", input);
       }
+      facebookVideoTranscribe(input) {
+        return this.call("/facebook/video-transcribe", input, this.httpTimeoutOverrideMs ?? 24e4);
+      }
       mapsPlaceIntel(input) {
         return this.call("/maps/place", input);
       }
@@ -22623,7 +22915,7 @@ __export(server_exports, {
   app: () => app
 });
 function configuredOrigins() {
-  const origins = /* @__PURE__ */ new Set(["https://mcpscraper.dev"]);
+  const origins = /* @__PURE__ */ new Set(["https://mcpscraper.dev", "https://www.mcpscraper.dev"]);
   for (const raw of (process.env.ALLOWED_ORIGINS ?? process.env.APP_ORIGIN ?? "").split(",")) {
     const trimmed = raw.trim();
     if (trimmed) origins.add(trimmed);