mastra-scavio 0.6.0 → 0.7.0

package/README.md

@@ -1,6 +1,6 @@
 # mastra-scavio
 
-[Scavio](https://scavio.dev) real-time data tools for [Mastra](https://mastra.ai) agents — **188 tools across 31 platforms** on one API key. Retail (Amazon, Walmart, eBay, Target, Home Depot), real estate (Zillow, Redfin), travel (Booking.com, Airbnb, Tripadvisor), local (Yelp, Google Maps), jobs (Indeed, Glassdoor, LinkedIn), software reviews (G2, Capterra), app stores (App Store, Google Play), company filings (SEC EDGAR, Companies House), ad libraries (Google Ads Transparency, Meta Ad Library), search (Google) and social (YouTube, TikTok, Instagram, X, Reddit, Threads, Kuaishou) — plus `scavioExtract`, which reads any URL as clean Markdown.
+[Scavio](https://scavio.dev) real-time data tools for [Mastra](https://mastra.ai) agents — **188 tools across 31 platforms** on one API key. Retail (Amazon, Walmart, eBay, Target, Home Depot), real estate (Zillow, Redfin), travel (Booking.com, Airbnb, Tripadvisor), local (Yelp, Google Maps), jobs (Indeed, Glassdoor, LinkedIn), software reviews (G2, Capterra), app stores (App Store, Google Play), company filings (SEC EDGAR, Companies House), ad libraries (Google Ads Transparency, Meta Ad Library), search (Google) and social (YouTube, TikTok, Instagram, X, Reddit, Threads, Kuaishou) — plus `scavioExtract`, which reads any URL as clean Markdown, and five opt-in tools that map a site's URLs, crawl a whole site or batch-extract a URL list.
 
 > **Amazon changed (breaking).** The upstream provider moved in 2026-07:
 > `domain` is replaced by `country`, a two-letter marketplace code (`us`, `gb`
@@ -53,11 +53,12 @@ const agent = new Agent({
 
 ## Tools
 
-`createScavioTools()` returns **188 tools** across 31 platforms plus `extract` — one per billable Scavio endpoint. Requires `scavio@^0.15.0`.
+`createScavioTools()` returns **188 tools** across 31 platforms plus `extract` — one per billable Scavio endpoint — and **193** with `{ enableCrawl: true }`, which adds the five website tools. Requires `scavio@^0.23.0`.
 
 | Platform | Tools | Names |
 |---|---|---|
 | [Extract (any URL)](https://scavio.dev/docs/extract) | 1 | `scavioExtract` |
+| Website map, crawl, batch (opt-in: `enableCrawl`) | 5 | `scavioMapWebsite`, `scavioCrawlWebsite`, `scavioCrawlStatus`, `scavioBatchExtract`, `scavioBatchStatus` |
 | Google | 14 | `scavioGoogleSearch`, `scavioGoogleAiMode`, `scavioGoogleMapsSearch`, `scavioGoogleMapsPlace`, `scavioGoogleMapsReviews`, `scavioGoogleShopping`, `scavioGoogleShoppingProduct`, `scavioGoogleShoppingStores`, `scavioGoogleFlights`, `scavioGoogleHotels`, `scavioGoogleHotelsDetail`, `scavioGoogleNews`, `scavioGoogleTrends`, `scavioGoogleTrending` |
 | Amazon | 3 | `scavioAmazonSearch`, `scavioAmazonProduct`, `scavioAmazonOffers` |
 | [Walmart](https://scavio.dev/docs/walmart-api) | 7 | `scavioWalmartSearch`, `scavioWalmartProduct`, `scavioWalmartReviews`, `scavioWalmartCategory`, `scavioWalmartOffers`, `scavioWalmartSeller`, `scavioWalmartSellerProducts` |
@@ -116,6 +117,10 @@ Reddit, TikTok and Instagram shipped two tools each before 0.5.0; the rest of th
 
 `scavioRedditSearch`, `scavioRedditPost`, `scavioTiktokProfile` and `scavioInstagramProfile` are unchanged.
 
+### New in 0.7.0
+
+Five website tools, opt-in with `createScavioTools({ enableCrawl: true })` so the default set is unchanged: `scavioMapWebsite`, `scavioCrawlWebsite`, `scavioCrawlStatus`, `scavioBatchExtract` and `scavioBatchStatus`. Requires `scavio@^0.23.0`, whose `extract` became a callable namespace (`scavio.extract({ url })` still reads one page). See [Map, crawl and batch](#map-crawl-and-batch).
+
 ### New in 0.6.0
 
 91 tools: 21 new platforms, five more Walmart endpoints, and `scavioExtract`. Nothing was renamed or removed. Three things worth knowing:
@@ -124,6 +129,60 @@ Reddit, TikTok and Instagram shipped two tools each before 0.5.0; the rest of th
 - **`scavioExtract` is the fallback for everything else.** It reads any URL and returns clean Markdown (or plain text, or raw HTML), so an agent that lands on a page no platform endpoint covers still has a move. It leads `createScavioTools()` for that reason, and it is a top-level SDK method — `scavio.extract({ url })`, never `scavio.extract.extract()`.
 - **Walmart moved from 2 tools to 7** and its two original tools were migrated to the spec pattern the rest of the package uses. `createScavioWalmartSearchTool` and `createScavioWalmartProductTool` keep their names and tool ids. One input tightened: `sort_by` on the search tool was an open string and is now the closed enum the API actually accepts (`best_match`, `price_low`, `price_high`, `best_seller`, `rating_high`, `new`) — a value outside it was never honoured upstream, it was just billed.
 
+## Map, crawl and batch
+
+Five opt-in tools for whole sites, behind `createScavioTools({ enableCrawl: true })` (off by default, because a
+crawl bills per page):
+
+| Tool | What it does | Credits |
+|---|---|---|
+| `scavio-map-website` | List a site's page URLs from its sitemaps and start page, deduplicated and filtered by path or search words | 1 (normal mode; advanced up to 2, ultra up to 10) |
+| `scavio-crawl-website` | Crawl a site from one URL; every same-site page as Markdown, text or HTML | per successful page |
+| `scavio-crawl-status` | A crawl's progress and page results, cursor-paged | free |
+| `scavio-batch-extract` | Extract a list of URLs (up to 1,000, plan-limited) in one job | per successful page |
+| `scavio-batch-status` | A batch's progress and page results, cursor-paged | free |
+
+Map answers in one call. Crawl and batch run as background jobs: the start tool
+posts the job and then **waits up to `wait_seconds` (default 45, max 300)** for
+it to finish, returning the job's status with its first page of results (10 by
+default, `limit` up to 100) either way. A job still `queued` or `running` when
+the wait ends keeps going server-side: read the rest later with the matching
+status tool, the returned `data.id` and `next_cursor`. That keeps an agent turn
+bounded while a large crawl (up to 6 hours) runs on.
+
+```typescript
+import { createScavioTools, createScavioCrawlWebsiteTool } from 'mastra-scavio';
+
+// Everything, plus the five website tools:
+const tools = createScavioTools({ enableCrawl: true });
+
+// Or just the crawler:
+const agent = new Agent({
+  id: 'docs-agent',
+  name: 'Docs Agent',
+  model: 'anthropic/claude-sonnet-4-6',
+  instructions: 'Crawl documentation sites with Scavio and answer from them.',
+  tools: { crawl: createScavioCrawlWebsiteTool() },
+});
+// scavio-crawl-website({ url: "https://hono.dev/docs", max_pages: 25, include_paths: ["/docs/**"] })
+//   -> { data: { id, status: "completed" | "running", pages, credits_used, results, next_cursor } }
+```
+
+**Pricing.** Crawl and batch bill **per successful page** at Extract's price for
+the mode: `normal` (the standard fetch) and `advanced` (renders JavaScript-built
+pages) 1 credit, `ultra` (the hardest pages) 2. Failed and skipped pages are
+free, and if your balance runs out the job stops cleanly with
+`stopped_reason: "insufficient_credits"`. Reading a job is free. Crawl and map
+obey the site's robots.txt, and results are kept for 3 days.
+
+**Plan limits** (pages per crawl / active jobs / URLs per batch): Free 10/1/10,
+PAYG 100/1/50, Project 500/2/100, Bootstrap 2,000/3/500, Startup 5,000/5/1,000,
+Growth 10,000/10/1,000.
+
+Cancellation, webhooks (signed `Scavio-Signature`) and idempotency keys exist in
+the `scavio` SDK (`extract.cancelCrawl`, `webhook_url`, `idempotencyKey`) and the
+[REST API](https://scavio.dev/docs/extract), but are deliberately not agent tools.
+
 ## Credits
 
 Most calls cost 1 credit. Every tool states its own cost in its description, so an agent can budget before it fans out.
@@ -151,7 +210,7 @@ Most calls cost 1 credit. Every tool states its own cost in its description, so
 
 Instagram is priced per endpoint rather than flat, on three tiers: 10 where both upstream legs are billed, 8 for the two calls with no fallback leg to hedge (`scavioInstagramPost`, `scavioInstagramCommentReplies`), and 2 for `scavioInstagramUserPosts`, whose cheap leg is the primary one. Prefer `scavioInstagramUserPosts` over `scavioInstagramUserReels` when the timeline will do — same shape of data, one fifth of the cost. Paginated tools bill per page. See [scavio.dev/docs](https://scavio.dev/docs).
 
-### Four surfaces are priced by the request body
+### These surfaces are priced by the request body
 
 No single number is true for these, so each tool states its own price and none of them says a flat cost:
 
@@ -161,6 +220,8 @@ No single number is true for these, so each tool states its own price and none o
 | `scavioThreadsProfile`, `scavioThreadsUserPosts`, `scavioThreadsUserReplies` | **2 credits addressed by `user_id`, 4 by `username`** — a handle buys a second upstream call, because the upstream handle lookup is dead. Pass `user_id` whenever you have it. The other three Threads tools have no username form and are always 2. |
 | Kuaishou | **Per endpoint, 1 to 40.** See the table above; one platform-wide figure would be wrong by up to 40x. |
 | `scavioExtract` | 1 credit on `mode` `normal` or `advanced`, **2 on `ultra`**. Only a successful extraction is billed — a dead link, bot wall or timeout costs nothing. |
+| `scavioMapWebsite` | 1 credit on `mode` `normal`; up to 2 on `advanced`, up to 10 on `ultra`, priced by what the call needed. |
+| `scavioCrawlWebsite`, `scavioBatchExtract` | **Per successful page**: 1 credit on `normal` or `advanced`, 2 on `ultra`. Failed and skipped pages are free; the two status tools are free. |
 
 ### Lookup-first platforms
 

package/dist/index.cjs

@@ -35,6 +35,8 @@ __export(index_exports, {
   createScavioAppStoreAppTool: () => createScavioAppStoreAppTool,
   createScavioAppStoreReviewsTool: () => createScavioAppStoreReviewsTool,
   createScavioAppStoreSearchTool: () => createScavioAppStoreSearchTool,
+  createScavioBatchExtractTool: () => createScavioBatchExtractTool,
+  createScavioBatchStatusTool: () => createScavioBatchStatusTool,
   createScavioBookingHotelTool: () => createScavioBookingHotelTool,
   createScavioBookingReviewsTool: () => createScavioBookingReviewsTool,
   createScavioBookingSearchTool: () => createScavioBookingSearchTool,
@@ -45,6 +47,8 @@ __export(index_exports, {
   createScavioCompaniesHouseFilingHistoryTool: () => createScavioCompaniesHouseFilingHistoryTool,
   createScavioCompaniesHouseOfficersTool: () => createScavioCompaniesHouseOfficersTool,
   createScavioCompaniesHouseSearchTool: () => createScavioCompaniesHouseSearchTool,
+  createScavioCrawlStatusTool: () => createScavioCrawlStatusTool,
+  createScavioCrawlWebsiteTool: () => createScavioCrawlWebsiteTool,
   createScavioEbayProductTool: () => createScavioEbayProductTool,
   createScavioEbaySearchTool: () => createScavioEbaySearchTool,
   createScavioEbaySellerTool: () => createScavioEbaySellerTool,
@@ -118,6 +122,7 @@ __export(index_exports, {
   createScavioLinkedinPostCommentsTool: () => createScavioLinkedinPostCommentsTool,
   createScavioLinkedinPostTool: () => createScavioLinkedinPostTool,
   createScavioLinkedinSearchJobsTool: () => createScavioLinkedinSearchJobsTool,
+  createScavioMapWebsiteTool: () => createScavioMapWebsiteTool,
   createScavioMetaAdsAdTool: () => createScavioMetaAdsAdTool,
   createScavioMetaAdsAdvertiserTool: () => createScavioMetaAdsAdvertiserTool,
   createScavioMetaAdsSearchTool: () => createScavioMetaAdsSearchTool,
@@ -218,6 +223,7 @@ __export(index_exports, {
   createScavioZillowSearchTool: () => createScavioZillowSearchTool,
   ebayToolSpecs: () => ebayToolSpecs,
   extractToolSpecs: () => extractToolSpecs,
+  extractWebToolSpecs: () => extractWebToolSpecs,
   g2ToolSpecs: () => g2ToolSpecs,
   getScavioClient: () => getScavioClient,
   glassdoorToolSpecs: () => glassdoorToolSpecs,
@@ -1932,7 +1938,7 @@ var extractToolSpecs = [
         "Output format: 'html' is the raw page, 'markdown' a readability extraction, 'text' that markdown flattened to plain text (server default 'markdown')."
       ),
       mode: import_zod11.z.enum(["normal", "advanced", "ultra"]).optional().describe(
-        "Fetch tier, and the price-bearing parameter: 'normal' plain datacenter fetch (1 credit), 'advanced' full browser render (1 credit), 'ultra' the hardest-target tier (2 credits). Server default 'normal'."
+        "Fetch tier, and the price-bearing parameter: 'normal' is the standard fetch (1 credit), 'advanced' renders JavaScript-built pages (1 credit), 'ultra' reaches the hardest pages (2 credits). Server default 'normal'."
       )
     }),
     call: (client, input) => client.extract(input)
@@ -4498,6 +4504,162 @@ var createScavioMetaAdsAdvertiserTool = toolFactory(
 );
 var createScavioMetaAdsAdTool = toolFactory(metaAdsToolSpecs, "scavioMetaAdsAd");
 
+// src/extract-web.ts
+var import_zod34 = require("zod");
+var JOB_POLL_INTERVAL_MS = 3e3;
+var JOB_WAIT_DEFAULT_SECONDS = 45;
+var JOB_WAIT_MAX_SECONDS = 300;
+var JOB_RESULTS_DEFAULT_LIMIT = 10;
+var JOB_TERMINAL = /* @__PURE__ */ new Set(["completed", "failed", "cancelled"]);
+var sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
+async function waitForJob(status, id, waitSeconds2, query) {
+  const deadline = Date.now() + Math.max(0, Math.min(waitSeconds2, JOB_WAIT_MAX_SECONDS)) * 1e3;
+  for (; ; ) {
+    const remaining = deadline - Date.now();
+    if (remaining <= 0) break;
+    await sleep(Math.min(JOB_POLL_INTERVAL_MS, remaining));
+    const probe = await status(id, { limit: 1, include_content: false });
+    if (JOB_TERMINAL.has(String(probe?.data?.status))) break;
+  }
+  return await status(id, query);
+}
+async function startAndWait(start, status, input) {
+  const { wait_seconds, limit: limit2, include_content, ...body } = input;
+  const created = await start(body);
+  const id = created?.data?.id;
+  const wait = wait_seconds ?? JOB_WAIT_DEFAULT_SECONDS;
+  if (!id || wait <= 0) return created;
+  const query = { limit: limit2 ?? JOB_RESULTS_DEFAULT_LIMIT };
+  if (include_content !== void 0) query.include_content = include_content;
+  return waitForJob(status, id, wait, query);
+}
+var MODE_DESCRIPTION = "Fetch tier, and the price-bearing parameter: 'normal' is the standard fetch, 'advanced' renders JavaScript-built pages, 'ultra' reaches the hardest pages. Server default 'normal'.";
+var PATHS_DESCRIPTION = "Path globs (max 20). '*' matches within one path segment, '**' across segments; a pattern without wildcards matches that path and everything under it. Example: ['/blog/**'].";
+var JOB_PRICE = "Billed per SUCCESSFUL page at Extract's price for the mode: normal and advanced 1 credit, ultra 2 credits. Failed and skipped pages are free, and if the balance runs out the job stops cleanly.";
+var mode = import_zod34.z.enum(["normal", "advanced", "ultra"]).optional().describe(MODE_DESCRIPTION);
+var format = import_zod34.z.enum(["html", "markdown", "text"]).optional().describe("Format for each page, as on extract: html, markdown or text (server default markdown).");
+var waitSeconds = import_zod34.z.number().int().min(0).max(JOB_WAIT_MAX_SECONDS).optional().describe(
+  "Seconds to wait for the job to finish before returning, 0-300 (default 45). 0 returns the job id at once. A job still running when the wait ends keeps going; read it later with its status tool."
+);
+var limit = import_zod34.z.number().int().min(1).max(100).optional().describe(
+  "Results per response, 1-100 (default 10, to keep page content inside the context window); page on with next_cursor."
+);
+var includeContent = import_zod34.z.boolean().optional().describe("Set false to list results without their page content (default true).");
+var statusSchema = import_zod34.z.object({
+  id: import_zod34.z.string().describe("Job id returned when the job was started (data.id)."),
+  cursor: import_zod34.z.string().optional().describe("next_cursor from the previous response, to read the next page."),
+  limit,
+  status: import_zod34.z.enum(["succeeded", "failed", "skipped"]).optional().describe("Only results with this status."),
+  include_content: includeContent
+});
+var readStatus = (fn) => (client, { id, limit: max, ...query }) => fn(client)(id, { ...query, limit: max ?? JOB_RESULTS_DEFAULT_LIMIT });
+var extractWebToolSpecs = [
+  defineScavioTool({
+    key: "scavioMapWebsite",
+    id: "scavio-map-website",
+    platform: "extract",
+    endpoint: "/api/v1/extract/map",
+    credits: 1,
+    description: "List a website's page URLs, gathered from its sitemaps and its start page, deduplicated, filtered by path or search words and limited: { url, links: [{ url, source, lastmod }], total_found, returned, truncated }. Pages the site's robots.txt disallows are left out. Use it to find the right pages before extracting or crawling. Costs 1 credit in normal mode; advanced costs up to 2 credits and ultra up to 10, priced by what the call needed.",
+    inputSchema: import_zod34.z.object({
+      url: import_zod34.z.string().describe("The site to map. A bare host is upgraded to https."),
+      search: import_zod34.z.string().optional().describe("Only return URLs containing every word of this text (case-insensitive)."),
+      limit: import_zod34.z.number().int().min(1).max(5e3).optional().describe(
+        "Maximum links to return, 1-5000. Default 1000 (or the plan's limit if lower: Free 500, PAYG 1,000, paid plans 5,000)."
+      ),
+      sitemap: import_zod34.z.enum(["include", "only", "skip"]).optional().describe(
+        "'include' combines the site's sitemaps with the start page's links (default); 'only' uses sitemaps alone; 'skip' uses page links alone."
+      ),
+      include_subdomains: import_zod34.z.boolean().optional().describe("Also return URLs on subdomains of the site (default false)."),
+      include_paths: import_zod34.z.array(import_zod34.z.string()).optional().describe(`Keep only URLs matching these. ${PATHS_DESCRIPTION}`),
+      exclude_paths: import_zod34.z.array(import_zod34.z.string()).optional().describe(`Drop URLs matching these. ${PATHS_DESCRIPTION}`),
+      ignore_query_parameters: import_zod34.z.boolean().optional().describe("Treat URLs that differ only by query string as one URL (default true)."),
+      mode
+    }),
+    call: async (client, input) => await client.extract.map(input)
+  }),
+  defineScavioTool({
+    key: "scavioCrawlWebsite",
+    id: "scavio-crawl-website",
+    platform: "extract",
+    endpoint: "/api/v1/extract/crawl",
+    credits: 1,
+    description: "Crawl a website from one start URL: same-site pages are discovered and extracted breadth-first within the page, depth and path limits, each returned as Markdown, text or HTML. Pages the site's robots.txt disallows are skipped. Waits up to wait_seconds (default 45) and returns { id, status, pages, credits_used, results, next_cursor }; if status is still queued or running, call scavio-crawl-status with the id later. " + JOB_PRICE,
+    inputSchema: import_zod34.z.object({
+      url: import_zod34.z.string().describe("Where the crawl starts. Only pages on the same site are crawled."),
+      max_pages: import_zod34.z.number().int().min(1).max(1e4).optional().describe(
+        "Maximum pages to fetch. Default 100. Plan limits: Free 10, PAYG 100, Project 500, Bootstrap 2,000, Startup 5,000, Growth 10,000."
+      ),
+      max_depth: import_zod34.z.number().int().min(0).max(10).optional().describe("How many links away from the start page to follow, 0-10 (default 3). 0 fetches only the start page."),
+      include_paths: import_zod34.z.array(import_zod34.z.string()).optional().describe(`Crawl only paths matching these. ${PATHS_DESCRIPTION}`),
+      exclude_paths: import_zod34.z.array(import_zod34.z.string()).optional().describe(`Skip paths matching these. ${PATHS_DESCRIPTION}`),
+      allow_subdomains: import_zod34.z.boolean().optional().describe("Also crawl subdomains of the site (default false)."),
+      ignore_query_parameters: import_zod34.z.boolean().optional().describe("Treat URLs that differ only by query string as one page (default true)."),
+      sitemap: import_zod34.z.enum(["include", "skip", "only"]).optional().describe(
+        "'include' also starts from the site's sitemap (default); 'skip' follows links only; 'only' fetches sitemap pages without following links."
+      ),
+      format,
+      mode,
+      wait_seconds: waitSeconds,
+      limit,
+      include_content: includeContent
+    }),
+    call: (client, input) => startAndWait(
+      (body) => client.extract.crawl(body),
+      (id, query) => client.extract.crawlStatus(id, query),
+      input
+    )
+  }),
+  defineScavioTool({
+    key: "scavioCrawlStatus",
+    id: "scavio-crawl-status",
+    platform: "extract",
+    endpoint: "/api/v1/extract/crawl/{id}",
+    credits: 0,
+    description: "Read a crawl's progress and its page results by job id, paged by cursor: { status, stopped_reason, pages, credits_used, results: [{ url, status, content }], next_cursor }. Results are kept for 3 days after the crawl finishes. Free: reading a job costs 0 credits.",
+    inputSchema: statusSchema,
+    call: readStatus((client) => (id, query) => client.extract.crawlStatus(id, query))
+  }),
+  defineScavioTool({
+    key: "scavioBatchExtract",
+    id: "scavio-batch-extract",
+    platform: "extract",
+    endpoint: "/api/v1/extract/batch",
+    credits: 1,
+    description: "Extract a list of URLs (up to 1,000, plan-limited) in one job, each returned as Markdown, text or HTML exactly like scavio-extract. Waits up to wait_seconds (default 45) and returns { id, status, pages, credits_used, results, next_cursor }; if status is still queued or running, call scavio-batch-status with the id later. " + JOB_PRICE,
+    inputSchema: import_zod34.z.object({
+      urls: import_zod34.z.array(import_zod34.z.string()).min(1).max(1e3).describe(
+        "The pages to extract (duplicates removed). Plan limits: Free 10, PAYG 50, Project 100, Bootstrap 500, Startup and Growth 1,000."
+      ),
+      format,
+      mode,
+      wait_seconds: waitSeconds,
+      limit,
+      include_content: includeContent
+    }),
+    call: (client, input) => startAndWait(
+      (body) => client.extract.batch(body),
+      (id, query) => client.extract.batchStatus(id, query),
+      input
+    )
+  }),
+  defineScavioTool({
+    key: "scavioBatchStatus",
+    id: "scavio-batch-status",
+    platform: "extract",
+    endpoint: "/api/v1/extract/batch/{id}",
+    credits: 0,
+    description: "Read a batch extract's progress and its page results by job id, paged by cursor: { status, pages, credits_used, results: [{ url, status, content }], next_cursor }. Results are kept for 3 days after the batch finishes. Free: reading a job costs 0 credits.",
+    inputSchema: statusSchema,
+    call: readStatus((client) => (id, query) => client.extract.batchStatus(id, query))
+  })
+];
+var createScavioMapWebsiteTool = toolFactory(extractWebToolSpecs, "scavioMapWebsite");
+var createScavioCrawlWebsiteTool = toolFactory(extractWebToolSpecs, "scavioCrawlWebsite");
+var createScavioCrawlStatusTool = toolFactory(extractWebToolSpecs, "scavioCrawlStatus");
+var createScavioBatchExtractTool = toolFactory(extractWebToolSpecs, "scavioBatchExtract");
+var createScavioBatchStatusTool = toolFactory(extractWebToolSpecs, "scavioBatchStatus");
+
 // src/tools.ts
 function createScavioTools(config) {
   return {
@@ -4688,7 +4850,14 @@ function createScavioTools(config) {
     scavioGoogleAdsCreative: createScavioGoogleAdsCreativeTool(config),
     scavioMetaAdsSearch: createScavioMetaAdsSearchTool(config),
     scavioMetaAdsAdvertiser: createScavioMetaAdsAdvertiserTool(config),
-    scavioMetaAdsAd: createScavioMetaAdsAdTool(config)
+    scavioMetaAdsAd: createScavioMetaAdsAdTool(config),
+    ...config?.enableCrawl ? {
+      scavioMapWebsite: createScavioMapWebsiteTool(config),
+      scavioCrawlWebsite: createScavioCrawlWebsiteTool(config),
+      scavioCrawlStatus: createScavioCrawlStatusTool(config),
+      scavioBatchExtract: createScavioBatchExtractTool(config),
+      scavioBatchStatus: createScavioBatchStatusTool(config)
+    } : {}
   };
 }
 // Annotate the CommonJS export names for ESM import in node:
@@ -4708,6 +4877,8 @@ function createScavioTools(config) {
   createScavioAppStoreAppTool,
   createScavioAppStoreReviewsTool,
   createScavioAppStoreSearchTool,
+  createScavioBatchExtractTool,
+  createScavioBatchStatusTool,
   createScavioBookingHotelTool,
   createScavioBookingReviewsTool,
   createScavioBookingSearchTool,
@@ -4718,6 +4889,8 @@ function createScavioTools(config) {
   createScavioCompaniesHouseFilingHistoryTool,
   createScavioCompaniesHouseOfficersTool,
   createScavioCompaniesHouseSearchTool,
+  createScavioCrawlStatusTool,
+  createScavioCrawlWebsiteTool,
   createScavioEbayProductTool,
   createScavioEbaySearchTool,
   createScavioEbaySellerTool,
@@ -4791,6 +4964,7 @@ function createScavioTools(config) {
   createScavioLinkedinPostCommentsTool,
   createScavioLinkedinPostTool,
   createScavioLinkedinSearchJobsTool,
+  createScavioMapWebsiteTool,
   createScavioMetaAdsAdTool,
   createScavioMetaAdsAdvertiserTool,
   createScavioMetaAdsSearchTool,
@@ -4891,6 +5065,7 @@ function createScavioTools(config) {
   createScavioZillowSearchTool,
   ebayToolSpecs,
   extractToolSpecs,
+  extractWebToolSpecs,
   g2ToolSpecs,
   getScavioClient,
   glassdoorToolSpecs,