webscraping-ai-cli 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -11,7 +11,7 @@
11
11
  "name": "webscraping-ai",
12
12
  "source": "./plugins/webscraping-ai",
13
13
  "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — parsed Google search results, and structured data for supported sites (YouTube, TikTok, X, LinkedIn, Instagram, Reddit, …), via the WebScraping.AI CLI.",
14
- "version": "1.2.1",
14
+ "version": "1.3.0",
15
15
  "category": "web-scraping",
16
16
  "keywords": [
17
17
  "web-scraping",
package/CHANGELOG.md CHANGED
@@ -5,6 +5,21 @@ All notable changes to `webscraping-ai-cli` are documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## 1.3.0 — 2026-10-24
9
+
10
+ ### Added
11
+
12
+ - `serp` flags for where the search runs from and how many pages it returns: `--location` (a name such as `Austin, TX` or `Austin,Texas,United States`, or a location id; sent as given), `--uule`, `--lat`/`--lon` (decimal degrees; negative values work as written, e.g. `--lat -33.86`), `--radius` (meters) and `--pages` (up to 5 consecutive pages as one de-duplicated list, billed per results page returned; see https://webscraping.ai/docs#serp). Every search runs from one fixed location (the location flags, otherwise the capital of the `--gl` country), so repeats and later pages are consistent, and `--gl` defaults to that location's country.
13
+ - `--pages` and `--radius` must be integers >= 1 and `--lat`/`--lon` plain decimal numbers; anything else exits 2 before stdin is read or the key is resolved. Combinations and upper limits are checked by the server (400, exit 3, not charged).
14
+ - `serp-locations [query]` subcommand wrapping the new `/serp/locations` endpoint: looks up names for `serp --location` and prints the JSON array (`id`, `canonical_name`, `country_code`, `target_type`, `reach`, `gps`, …), most populous first. `--limit <n>` (integer >= 1, exit 2 otherwise; at most 10) and the usual `-k`/`-o`/`--pretty`/`--no-pretty`. A lone `-` exits 2 (no stdin mode). Free on the server. `from_cli` isn't sent on it (the SDK method takes only `q` and `limit`), the same as `account`.
15
+ - Bundled agent skill documents the location flags, `--pages` and `serp-locations`.
16
+ - Smoke script adds a `serp --location 'Austin, TX' --pages 2` call (checks `search_parameters.location_used` contains `Austin` and non-empty `organic_results`) and a `serp-locations austin --limit 3` call (checks a non-empty list whose first `canonical_name` contains `Austin`).
17
+
18
+ ### Changed
19
+
20
+ - Requires `webscraping-ai` SDK `^4.3.0` (adds the `serp()` location/pages options and `serpLocations()`).
21
+ - Docs: stop stating credit prices (they're set server-side and change); link to https://webscraping.ai/docs pricing instead.
22
+
8
23
  ## 1.2.1 — 2026-09-26
9
24
 
10
25
  ### Security
package/README.md CHANGED
@@ -19,8 +19,8 @@ The package exposes two executables: `webscraping-ai` (primary) and `wsai` (shor
19
19
 
20
20
  ## Authenticate
21
21
 
22
- [Sign up](https://webscraping.ai/auth/sign_up) to get an API key — the free
23
- trial includes 2,000 credits, no credit card required. Your key lives in the
22
+ [Sign up](https://webscraping.ai/auth/sign_up) to get an API key — a free
23
+ trial, no credit card required. Your key lives in the
24
24
  [dashboard](https://webscraping.ai/dashboard).
25
25
 
26
26
  Pick one:
@@ -46,13 +46,26 @@ webscraping-ai selected-multiple <url> --selector a --selector b ...
46
46
  webscraping-ai ask <url> --question "..." # AI question about the page
47
47
  webscraping-ai extract <url> --fields '{"title":"...","price":"..."}'
48
48
  webscraping-ai serp <query> [--gl us] [--hl en] [--page 1] # Parsed Google results (alias: search)
49
+ webscraping-ai serp-locations [query] [--limit 10] # Names for serp --location
49
50
  webscraping-ai data <url> [--country us] [--transcript] # Structured JSON for a supported site
50
51
  webscraping-ai account # Remaining credits / quota
51
52
  ```
52
53
 
53
- `serp` (alias `search`) is query-shaped, not URL-shaped: it takes `--engine` (`google`, the default), `--gl` (country), `--hl` (language), `--page` (1-based, 10 results per page) plus `--api-key`/`--output`/`--pretty`, and none of the scrape flags below. It prints the JSON result (`search_parameters`, `search_information`, `organic_results`, `related_searches`, `pagination`). Flat 15 credits per search; failed searches are not charged. `--page` must be an integer >= 1 (pages are 1–100; the server rejects values above 100 with a 400); anything else is a usage error (exit 2) before any request. Multi-word queries don't need quoting; `-` reads one query per line from stdin, skipping only blank lines (a leading `#` is kept as part of the query, e.g. `#coffee`). The CLI's `from_cli` analytics flag is sent on `serp` through the SDK's `params` pass-through.
54
+ `serp` (alias `search`) is query-shaped, not URL-shaped: it takes `--engine` (`google`, the default), `--gl` (country), `--hl` (language), `--page` (1-based, 10 results per page), `--pages`, `--location`, `--uule`, `--lat`/`--lon`, `--radius` plus `--api-key`/`--output`/`--pretty`, and none of the scrape flags below. It prints the JSON result (`search_parameters`, `search_information`, `organic_results`, `related_searches`, `pagination`). Billed per results page returned (see https://webscraping.ai/docs#serp); failed searches are not charged. `--page` must be an integer >= 1 (pages are 1–100; the server rejects values above 100 with a 400); anything else is a usage error (exit 2) before any request. Multi-word queries don't need quoting; `-` reads one query per line from stdin, skipping only blank lines (a leading `#` is kept as part of the query, e.g. `#coffee`). The CLI's `from_cli` analytics flag is sent on `serp` through the SDK's `params` pass-through.
54
55
 
55
- `data <url>` returns structured JSON for a public page on a supported site — for example a YouTube video, TikTok profile, X post, LinkedIn company, Instagram reel or Reddit thread. Pass the page's normal URL; the site and page kind are detected server-side and echoed in `request_parameters` (`provider`, `type`), next to `parse_status` (`ok`, `parse_failed` or `not_found`) and `data` (shape depends on the site; `null` when nothing parsed). More sites are added on the server over time, so the CLI never checks the URL itself and sends it exactly as given. An unsupported URL or page type returns a 400 that is not charged (exit 3); its message lists what is supported. 15 credits per request, including `parse_failed`/`not_found` results; failed fetches are not charged. None of the scrape flags below apply; `data` has its own:
56
+ Every search runs from one fixed location: `--location`, `--lat` + `--lon` or `--uule`, otherwise the capital of the `--gl` country, so repeats and later pages are consistent. `--gl` defaults to that location's country.
57
+
58
+ - `--location <name>`: a canonical name (`Austin,Texas,United States`), a looser one (`Austin, TX`; large cities also match local names, e.g. `Москва`) or a location `id` from `serp-locations`. Sent as given. An unknown name returns a 400 (exit 3, not charged).
59
+ - `--uule <value>`: a Google-encoded location (`w+CAIQICI…` or `a+…`).
60
+ - `--lat <deg>` / `--lon <deg>`: the search origin in decimal degrees; use both. Negative values work as written (`--lat -33.86`). Anything but a plain decimal number is a usage error (exit 2).
61
+ - `--radius <m>`: meters around `--location` or `--lat`/`--lon` (1–1000).
62
+ - `--pages <n>`: fetch up to 5 consecutive pages, starting at `--page`, as one de-duplicated list. `position` then counts through the combined list, each result gets a `page` field and `pagination.pages_returned` says how many pages came back. Billed per page returned.
63
+
64
+ `--pages` and `--radius` must be integers >= 1 (exit 2 otherwise). Which location flags may be combined, and the upper limits, are checked by the server (400, not charged).
65
+
66
+ `serp-locations [query]` looks up names for `--location` and prints a JSON array of up to 10 locations, most populous first, each with `id`, `name`, `canonical_name`, `country_code`, `target_type`, `reach` and `gps` (`[longitude, latitude]`). Pass a result's `canonical_name` (or `id`) as `serp --location`. It matches English names (`petersburg`); without a query it returns the most populous locations. `--limit <n>` (integer >= 1, at most 10) caps the list. It is free and needs no API key on the server, but the CLI still sends the configured key like every command. Multi-word queries don't need quoting. There is no stdin mode: a lone `-` is a usage error (exit 2).
67
+
68
+ `data <url>` returns structured JSON for a public page on a supported site — for example a YouTube video, TikTok profile, X post, LinkedIn company, Instagram reel or Reddit thread. Pass the page's normal URL; the site and page kind are detected server-side and echoed in `request_parameters` (`provider`, `type`), next to `parse_status` (`ok`, `parse_failed` or `not_found`) and `data` (shape depends on the site; `null` when nothing parsed). More sites are added on the server over time, so the CLI never checks the URL itself and sends it exactly as given. An unsupported URL or page type returns a 400 that is not charged (exit 3); its message lists what is supported. Priced per site (see [pricing](https://webscraping.ai/docs#data)), including `parse_failed`/`not_found` results; unsupported URLs and failed fetches are not charged. None of the scrape flags below apply; `data` has its own:
56
69
 
57
70
  - `--country <code>`: two-letter country code of the proxy used to fetch the page, `us` by default.
58
71
  - `--transcript`: YouTube videos only. Also fetch the video's transcript into `data.transcript`. It's null when no matching captions are available. If the transcript fetch itself fails, the whole request fails with a 500 (exit 7) and is not charged.
@@ -135,6 +148,16 @@ cat urls.txt | webscraping-ai extract - \
135
148
  # Google results for a query, page 2, from Germany in German
136
149
  webscraping-ai serp coffee machines --gl de --hl de --page 2
137
150
 
151
+ # Two pages of results as searched from Austin, TX
152
+ webscraping-ai serp coffee machines --location 'Austin, TX' --pages 2
153
+
154
+ # Find the location name to use, then search from it
155
+ webscraping-ai serp-locations austin --limit 3 | jq -r '.[0].canonical_name'
156
+ webscraping-ai serp pizza --location 'Austin,Texas,United States'
157
+
158
+ # Search from coordinates (Sydney) within 1 km
159
+ webscraping-ai serp cafe --lat -33.8688 --lon 151.2093 --radius 1000
160
+
138
161
  # Just the organic result links
139
162
  webscraping-ai search "best espresso grinder" | jq -r '.organic_results[].link'
140
163
 
package/dist/cli.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  // src/cli.ts
4
- import { Command as Command12 } from "commander";
4
+ import { Command as Command13 } from "commander";
5
5
  import { readFileSync } from "fs";
6
6
  import { dirname as dirname3, resolve as resolve2 } from "path";
7
7
  import { fileURLToPath as fileURLToPath2 } from "url";
@@ -187,6 +187,8 @@ function createClient(options) {
187
187
  return (o) => target.serp({ ...o, params: { ...o.params, from_cli: true } });
188
188
  case "data":
189
189
  return (o) => target.data({ ...o, params: { ...o.params, from_cli: true } });
190
+ case "serpLocations":
191
+ return value.bind(target);
190
192
  case "account":
191
193
  return value.bind(target);
192
194
  default:
@@ -257,20 +259,43 @@ function buildQuestionOptions(url, question, flags, resolved) {
257
259
  function buildFieldsOptions(url, fields, flags, resolved) {
258
260
  return { url, fields, ...buildCommonOptions(flags, resolved) };
259
261
  }
262
+ function checkPositiveInteger(flag, value) {
263
+ if (value !== void 0 && (!Number.isSafeInteger(value) || value < 1)) {
264
+ throw new Error(`${flag} must be an integer >= 1, got: ${value}`);
265
+ }
266
+ }
267
+ function checkFinite(flag, value) {
268
+ if (value !== void 0 && !Number.isFinite(value)) {
269
+ throw new Error(`${flag} must be a number, got: ${value}`);
270
+ }
271
+ }
260
272
  function buildSerpOptions(query, flags = {}) {
261
273
  if (typeof query !== "string" || query.trim() === "") {
262
274
  throw new Error("serp requires a non-empty query");
263
275
  }
276
+ checkPositiveInteger("--page", flags.page);
277
+ checkPositiveInteger("--pages", flags.pages);
278
+ checkPositiveInteger("--radius", flags.radius);
279
+ checkFinite("--lat", flags.lat);
280
+ checkFinite("--lon", flags.lon);
264
281
  const out = { q: query };
265
282
  if (flags.engine !== void 0) out.engine = flags.engine;
266
283
  if (flags.gl) out.gl = flags.gl;
267
284
  if (flags.hl) out.hl = flags.hl;
268
- if (flags.page !== void 0) {
269
- if (!Number.isSafeInteger(flags.page) || flags.page < 1) {
270
- throw new Error(`--page must be an integer >= 1, got: ${flags.page}`);
271
- }
272
- out.page = flags.page;
273
- }
285
+ if (flags.page !== void 0) out.page = flags.page;
286
+ if (flags.pages !== void 0) out.pages = flags.pages;
287
+ if (flags.location !== void 0) out.location = flags.location;
288
+ if (flags.uule !== void 0) out.uule = flags.uule;
289
+ if (flags.lat !== void 0) out.lat = flags.lat;
290
+ if (flags.lon !== void 0) out.lon = flags.lon;
291
+ if (flags.radius !== void 0) out.radius = flags.radius;
292
+ return out;
293
+ }
294
+ function buildSerpLocationsOptions(query, limit) {
295
+ checkPositiveInteger("--limit", limit);
296
+ const out = {};
297
+ if (query !== void 0 && query.trim() !== "") out.q = query;
298
+ if (limit !== void 0) out.limit = limit;
274
299
  return out;
275
300
  }
276
301
  var RESERVED_DATA_PARAMS = /* @__PURE__ */ new Set(["api_key", "url", "from_cli", "__proto__"]);
@@ -327,6 +352,9 @@ function addCommonScrapeOptions(cmd) {
327
352
  function isPositiveIntegerString(value) {
328
353
  return /^[1-9]\d*$/.test(value) && Number.isSafeInteger(Number(value));
329
354
  }
355
+ function isDecimalString(value) {
356
+ return /^[+-]?(?:\d+\.?\d*|\.\d+)$/.test(value) && Number.isFinite(Number(value));
357
+ }
330
358
  function integerParser(value) {
331
359
  const n = Number.parseInt(value, 10);
332
360
  if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);
@@ -481,7 +509,7 @@ function collect(value, previous = []) {
481
509
  }
482
510
  function dataCommand() {
483
511
  const cmd = new Command4("data").description(
484
- "Structured JSON for a page on a supported site, e.g. YouTube, TikTok, X, LinkedIn, Instagram, Reddit (wraps /data, 15 credits/request; more sites are added server-side, an unsupported URL or page type gets a 400, not charged, listing what is supported)"
512
+ "Structured JSON for a page on a supported site, e.g. YouTube, TikTok, X, LinkedIn, Instagram, Reddit (wraps /data, priced per site; more sites are added server-side, an unsupported URL or page type gets a 400, not charged, listing what is supported)"
485
513
  ).argument("<url>", "page URL (or `-` to read URLs from stdin, one per line)").option(
486
514
  "--country <code>",
487
515
  "two-letter country code of the proxy used to fetch the page (default: us)"
@@ -633,29 +661,71 @@ function collectMulti(value, previous) {
633
661
  // src/commands/serp.ts
634
662
  import { Command as Command9, Option as Option5 } from "commander";
635
663
  function serpCommand() {
636
- const cmd = new Command9("serp").alias("search").description("Search Google and print parsed results as JSON (wraps /serp, 15 credits/search)").argument("<query...>", "search query (or `-` to read queries from stdin, one per line)").addOption(new Option5("--engine <name>", "search engine (default: google)").choices(["google"])).option("--gl <code>", "two-letter country code for search geolocation (default: us)").option("--hl <code>", "two-letter language code for the results (default: en)").option(
664
+ const cmd = new Command9("serp").alias("search").description("Search Google and print parsed results as JSON (wraps /serp)").argument("<query...>", "search query (or `-` to read queries from stdin, one per line)").addOption(new Option5("--engine <name>", "search engine (default: google)").choices(["google"])).option(
665
+ "--gl <code>",
666
+ "two-letter country code for the search (default: the country of --location, --lat/--lon or --uule, otherwise us)"
667
+ ).option("--hl <code>", "two-letter language code for the results (default: en)").option(
637
668
  "--page <n>",
638
669
  "results page number, integer >= 1, 10 results per page (default: 1; server rejects > 100 with a 400)"
639
- ).option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output");
670
+ ).option(
671
+ "--pages <n>",
672
+ "consecutive results pages to fetch in one call starting at --page, 1-5, returned as one de-duplicated list (default: 1; billed per page returned)"
673
+ ).option(
674
+ "--location <name>",
675
+ 'where the search runs from: a name like "Austin, TX" or "Austin,Texas,United States", or an id from `serp-locations`'
676
+ ).option("--uule <value>", "Google-encoded location (w+CAIQICI\u2026 or a+\u2026)").option(
677
+ "--lat <deg>",
678
+ "latitude of the search origin in decimal degrees (-90 to 90; needs --lon)"
679
+ ).option(
680
+ "--lon <deg>",
681
+ "longitude of the search origin in decimal degrees (-180 to 180; needs --lat)"
682
+ ).option("--radius <m>", "radius in meters around --location or --lat/--lon, integer 1-1000").option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output");
640
683
  cmd.action(async (words, opts) => {
641
684
  const query = words.join(" ");
642
- let page;
643
- if (opts.page !== void 0) {
644
- if (!isPositiveIntegerString(opts.page)) {
645
- cmd.error(`error: --page must be an integer >= 1, got: ${opts.page}`, {
646
- exitCode: EXIT_CODES.usage,
647
- code: "wsai.invalidPage"
648
- });
685
+ const usage = (message, code) => cmd.error(`error: ${message}`, { exitCode: EXIT_CODES.usage, code });
686
+ const integer = (name, value) => {
687
+ if (value === void 0) return void 0;
688
+ if (!isPositiveIntegerString(value)) {
689
+ usage(
690
+ `--${name} must be an integer >= 1, got: ${value}`,
691
+ `wsai.invalid${capitalize(name)}`
692
+ );
649
693
  }
650
- page = Number(opts.page);
651
- }
694
+ return Number(value);
695
+ };
696
+ const decimal = (name, value) => {
697
+ if (value === void 0) return void 0;
698
+ if (!isDecimalString(value)) {
699
+ usage(
700
+ `--${name} must be a decimal number, e.g. -33.86, got: ${value}`,
701
+ `wsai.invalid${capitalize(name)}`
702
+ );
703
+ }
704
+ return Number(value);
705
+ };
706
+ const page = integer("page", opts.page);
707
+ const pages = integer("pages", opts.pages);
708
+ const radius = integer("radius", opts.radius);
709
+ const lat = decimal("lat", opts.lat);
710
+ const lon = decimal("lon", opts.lon);
652
711
  if (!urlIsStdin(query) && query.trim() === "") {
653
712
  cmd.error("error: serp requires a non-empty query", {
654
713
  exitCode: EXIT_CODES.usage,
655
714
  code: "wsai.emptyQuery"
656
715
  });
657
716
  }
658
- const flags = { engine: opts.engine, gl: opts.gl, hl: opts.hl, page };
717
+ const flags = {
718
+ engine: opts.engine,
719
+ gl: opts.gl,
720
+ hl: opts.hl,
721
+ page,
722
+ pages,
723
+ location: opts.location,
724
+ uule: opts.uule,
725
+ lat,
726
+ lon,
727
+ radius
728
+ };
659
729
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
660
730
  const client = createClient({ apiKey });
661
731
  const write = await openEmitter(opts, query);
@@ -666,9 +736,45 @@ function serpCommand() {
666
736
  });
667
737
  return cmd;
668
738
  }
739
+ function capitalize(name) {
740
+ return name.charAt(0).toUpperCase() + name.slice(1);
741
+ }
669
742
 
670
- // src/commands/setupSkill.ts
743
+ // src/commands/serpLocations.ts
671
744
  import { Command as Command10 } from "commander";
745
+ function serpLocationsCommand() {
746
+ const cmd = new Command10("serp-locations").description(
747
+ "Look up names for `serp --location` and print them as a JSON array (wraps /serp/locations, free)"
748
+ ).argument("[query...]", "part of a location name, e.g. austin (default: most populous)").option("--limit <n>", "maximum number of locations, integer 1-10 (default: 10)").option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output");
749
+ cmd.action(async (words, opts) => {
750
+ let limit;
751
+ if (opts.limit !== void 0) {
752
+ if (!isPositiveIntegerString(opts.limit)) {
753
+ cmd.error(`error: --limit must be an integer >= 1, got: ${opts.limit}`, {
754
+ exitCode: EXIT_CODES.usage,
755
+ code: "wsai.invalidLimit"
756
+ });
757
+ }
758
+ limit = Number(opts.limit);
759
+ }
760
+ const query = words.join(" ");
761
+ if (query === "-") {
762
+ cmd.error("error: serp-locations has no stdin mode; pass the query as an argument", {
763
+ exitCode: EXIT_CODES.usage,
764
+ code: "wsai.stdinUnsupported"
765
+ });
766
+ }
767
+ const options = buildSerpLocationsOptions(query, limit);
768
+ const apiKey = await resolveApiKey({ flag: opts.apiKey });
769
+ const client = createClient({ apiKey });
770
+ const result = await client.serpLocations(options);
771
+ await emit(result, opts);
772
+ });
773
+ return cmd;
774
+ }
775
+
776
+ // src/commands/setupSkill.ts
777
+ import { Command as Command11 } from "commander";
672
778
 
673
779
  // src/lib/skill.ts
674
780
  import { promises as fs4 } from "fs";
@@ -785,7 +891,7 @@ async function copyDirectory(source, destination) {
785
891
 
786
892
  // src/commands/setupSkill.ts
787
893
  function setupCommand() {
788
- const cmd = new Command10("setup").description("Install the AI agent skill into your editor(s)");
894
+ const cmd = new Command11("setup").description("Install the AI agent skill into your editor(s)");
789
895
  cmd.command("skill").description("Install the WebScraping.AI skill into one or more AI coding editors").option(
790
896
  "-e, --editor <id>",
791
897
  `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(", ")}`,
@@ -828,9 +934,9 @@ function resolveSelection(opts) {
828
934
  }
829
935
 
830
936
  // src/commands/text.ts
831
- import { Command as Command11, Option as Option6 } from "commander";
937
+ import { Command as Command12, Option as Option6 } from "commander";
832
938
  function textCommand() {
833
- const cmd = new Command11("text").description("Fetch the visible text of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
939
+ const cmd = new Command12("text").description("Fetch the visible text of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
834
940
  new Option6("--text-format <fmt>", "text response format").choices(["plain", "json", "xml"])
835
941
  ).option("--return-links", "include page links (text-format=json only)");
836
942
  addCommonScrapeOptions(cmd);
@@ -864,7 +970,7 @@ function packageVersion() {
864
970
  return raw.version;
865
971
  }
866
972
  function buildProgram() {
867
- const program = new Command12();
973
+ const program = new Command13();
868
974
  program.name("webscraping-ai").description(
869
975
  "Official CLI for the WebScraping.AI API. Fetch pages, search Google, get structured site data, ask AI, extract fields.\nSet $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first."
870
976
  ).version(packageVersion(), "-v, --version", "output the version number").showHelpAfterError();
@@ -875,6 +981,7 @@ function buildProgram() {
875
981
  program.addCommand(askCommand());
876
982
  program.addCommand(extractCommand());
877
983
  program.addCommand(serpCommand());
984
+ program.addCommand(serpLocationsCommand());
878
985
  program.addCommand(dataCommand());
879
986
  program.addCommand(accountCommand());
880
987
  program.addCommand(authCommand());
package/dist/cli.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/cli.ts","../src/commands/account.ts","../src/lib/config.ts","../src/lib/output.ts","../src/lib/stdin.ts","../src/lib/sdk.ts","../src/lib/redact.ts","../src/commands/ask.ts","../src/lib/buildRequest.ts","../src/lib/options.ts","../src/commands/auth.ts","../src/commands/data.ts","../src/lib/errors.ts","../src/commands/extract.ts","../src/commands/html.ts","../src/commands/selected.ts","../src/commands/selectedMultiple.ts","../src/commands/serp.ts","../src/commands/setupSkill.ts","../src/lib/skill.ts","../src/commands/text.ts"],"sourcesContent":["/**\n * `webscraping-ai` / `wsai` CLI entry point.\n *\n * Wires every subcommand into a single commander root. Errors thrown from\n * subcommand actions land in the top-level `.catch`, which maps SDK\n * exception classes to stable exit codes (see `lib/errors.ts`).\n */\n\nimport { Command } from 'commander';\nimport { readFileSync } from 'node:fs';\nimport { dirname, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nimport { accountCommand } from './commands/account.js';\nimport { askCommand } from './commands/ask.js';\nimport { authCommand } from './commands/auth.js';\nimport { dataCommand } from './commands/data.js';\nimport { extractCommand } from './commands/extract.js';\nimport { htmlCommand } from './commands/html.js';\nimport { selectedCommand } from './commands/selected.js';\nimport { selectedMultipleCommand } from './commands/selectedMultiple.js';\nimport { serpCommand } from './commands/serp.js';\nimport { setupCommand } from './commands/setupSkill.js';\nimport { textCommand } from './commands/text.js';\nimport { exitCodeFor, formatError } from './lib/errors.js';\n\nfunction packageVersion(): string {\n // Read package.json at runtime so we don't need a code-gen step. The dist\n // bundle sits at `dist/cli.js`, so package.json is one directory up.\n const here = dirname(fileURLToPath(import.meta.url));\n const pkgPath = resolve(here, '..', 'package.json');\n const raw = JSON.parse(readFileSync(pkgPath, 'utf8')) as { version: string };\n return raw.version;\n}\n\nexport function buildProgram(): Command {\n const program = new Command();\n program\n .name('webscraping-ai')\n .description(\n 'Official CLI for the WebScraping.AI API. Fetch pages, search Google, get structured site data, ask AI, extract fields.\\n' +\n 'Set $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first.',\n )\n .version(packageVersion(), '-v, --version', 'output the version number')\n .showHelpAfterError();\n\n program.addCommand(htmlCommand());\n program.addCommand(textCommand());\n program.addCommand(selectedCommand());\n program.addCommand(selectedMultipleCommand());\n program.addCommand(askCommand());\n program.addCommand(extractCommand());\n program.addCommand(serpCommand());\n program.addCommand(dataCommand());\n program.addCommand(accountCommand());\n program.addCommand(authCommand());\n program.addCommand(setupCommand());\n\n return program;\n}\n\nasync function main(argv: string[]): Promise<void> {\n const program = buildProgram();\n try {\n await program.parseAsync(argv);\n } catch (err) {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n }\n}\n\n// Only execute when run as a script. When imported (e.g. from tests),\n// `buildProgram` is the public surface and `main` stays dormant.\nconst entryPath = process.argv[1] ? fileURLToPath(import.meta.url) : '';\nif (entryPath && process.argv[1] === entryPath) {\n main(process.argv).catch((err: unknown) => {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n });\n}\n","import { Command } from 'commander';\n\nimport { resolveApiKey } from '../lib/config.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface AccountFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nexport function accountCommand(): Command {\n return new Command('account')\n .description('Show account quota and usage information')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .action(async (opts: AccountFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const result = await client.account();\n await emit(result, opts);\n });\n}\n","/**\n * API key resolution and persisted config.\n *\n * Resolution order (first non-empty wins):\n * 1. `--api-key` flag (handled by the caller, passed in)\n * 2. `WEBSCRAPING_AI_API_KEY` env var\n * 3. `~/.config/webscraping-ai/config.json` (or `$XDG_CONFIG_HOME/...`)\n *\n * The config file is plain JSON with `{ \"apiKey\": \"...\" }`. We chmod it to\n * 0600 so it's not world-readable.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join } from 'node:path';\n\nconst CONFIG_FILENAME = 'config.json';\n\nexport function configDir(): string {\n const xdg = process.env.XDG_CONFIG_HOME;\n if (xdg && xdg.trim() !== '') return join(xdg, 'webscraping-ai');\n return join(homedir(), '.config', 'webscraping-ai');\n}\n\nexport function configPath(): string {\n return join(configDir(), CONFIG_FILENAME);\n}\n\nexport interface PersistedConfig {\n apiKey?: string;\n}\n\nexport async function readPersistedConfig(): Promise<PersistedConfig> {\n try {\n const text = await fs.readFile(configPath(), 'utf8');\n const parsed = JSON.parse(text) as unknown;\n if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {\n return parsed as PersistedConfig;\n }\n return {};\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return {};\n throw err;\n }\n}\n\nexport async function writePersistedConfig(config: PersistedConfig): Promise<void> {\n const path = configPath();\n await fs.mkdir(dirname(path), { recursive: true });\n await fs.writeFile(path, JSON.stringify(config, null, 2) + '\\n', { mode: 0o600 });\n // `mode` above only applies when the file is freshly created; an existing\n // file keeps its old (possibly world-readable) perms. Enforce 0600\n // unconditionally — chmod on a just-written file is cheap and idempotent.\n // No-op on Windows, but harmless.\n await fs.chmod(path, 0o600);\n}\n\nexport async function clearPersistedConfig(): Promise<boolean> {\n try {\n await fs.unlink(configPath());\n return true;\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return false;\n throw err;\n }\n}\n\nexport interface ResolveApiKeyOptions {\n /** `--api-key` flag value, if the user passed one. */\n flag?: string;\n}\n\nexport async function resolveApiKey(opts: ResolveApiKeyOptions = {}): Promise<string> {\n if (opts.flag && opts.flag.trim() !== '') return opts.flag.trim();\n\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') return env.trim();\n\n const persisted = await readPersistedConfig();\n if (persisted.apiKey && persisted.apiKey.trim() !== '') return persisted.apiKey.trim();\n\n throw new Error(\n 'No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`.',\n );\n}\n\nexport function maskApiKey(key: string): string {\n if (key.length <= 8) return '*'.repeat(key.length);\n return `${key.slice(0, 4)}${'*'.repeat(key.length - 8)}${key.slice(-4)}`;\n}\n","/**\n * Output handling for command results.\n *\n * All command handlers funnel through `emit()`. The result type drives format:\n * - strings → written verbatim\n * - objects/arrays → JSON.stringified (pretty by default; `--pretty` is the\n * default for terminals, single-line for pipes/redirects)\n *\n * `--output FILE` writes to disk instead of stdout. Commands use\n * `openEmitter()`: in stdin batch mode (`-`) it truncates the file up front\n * (so a first-item failure or empty stdin never leaves stale content) and\n * appends every result, so each item survives instead of overwriting the last.\n */\n\nimport { promises as fs } from 'node:fs';\n\nimport { urlIsStdin } from './stdin.js';\n\nexport interface EmitOptions {\n output?: string;\n pretty?: boolean;\n}\n\nexport interface EmitWriteOptions {\n /** Append to `--output` instead of truncating it (batch items after the first). */\n append?: boolean;\n}\n\nexport async function emit(\n value: unknown,\n opts: EmitOptions = {},\n write: EmitWriteOptions = {},\n): Promise<void> {\n const text = stringify(value, opts);\n\n if (opts.output && opts.output.trim() !== '') {\n const data = text + (text.endsWith('\\n') ? '' : '\\n');\n if (write.append) await fs.appendFile(opts.output, data);\n else await fs.writeFile(opts.output, data);\n return;\n }\n\n process.stdout.write(text);\n if (!text.endsWith('\\n')) process.stdout.write('\\n');\n}\n\n/**\n * Returns an `emit` bound to `opts` for a (possibly batched) loop: the first\n * call truncates `--output`, later calls append.\n */\nexport function createEmitter(opts: EmitOptions = {}): (value: unknown) => Promise<void> {\n let first = true;\n return async (value: unknown) => {\n const append = !first;\n first = false;\n await emit(value, opts, { append });\n };\n}\n\nexport function stringify(value: unknown, opts: EmitOptions = {}): string {\n if (typeof value === 'string') return value;\n\n const pretty = opts.pretty ?? process.stdout.isTTY ?? false;\n return JSON.stringify(value, null, pretty ? 2 : 0);\n}\n\n/**\n * Emitter for a command whose positional arg is `arg`. When `arg` is `-`\n * (stdin batch) and `--output` is set, the file is truncated now, before any\n * request, and every result is appended. Otherwise same as `createEmitter()`.\n */\nexport async function openEmitter(\n opts: EmitOptions,\n arg: string,\n): Promise<(value: unknown) => Promise<void>> {\n if (urlIsStdin(arg) && opts.output && opts.output.trim() !== '') {\n await fs.writeFile(opts.output, '');\n return (value: unknown) => emit(value, opts, { append: true });\n }\n return createEmitter(opts);\n}\n","/**\n * Stdin batching helpers.\n *\n * When a URL argument is `-`, the command reads one URL per line from stdin\n * and runs the operation for each. Empty lines and lines starting with `#`\n * are skipped so users can comment URL lists.\n *\n * `serp` reads queries instead (`queryIterator`): only blank lines are\n * skipped there, because `#hashtag` is a legitimate search query.\n */\n\nimport readline from 'node:readline';\n\nexport function urlIsStdin(url: string): boolean {\n return url === '-';\n}\n\nexport async function* readUrlsFromStdin(): AsyncIterable<string> {\n const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });\n for await (const raw of rl) {\n const line = raw.trim();\n if (line === '' || line.startsWith('#')) continue;\n yield line;\n }\n}\n\n/**\n * Read one search query per line. Blank / whitespace-only lines are skipped;\n * every other line is yielded verbatim (no `#` comment handling, no trim).\n */\nexport async function* readQueriesFromStdin(\n input: NodeJS.ReadableStream = process.stdin,\n): AsyncIterable<string> {\n const rl = readline.createInterface({ input, crlfDelay: Infinity });\n for await (const line of rl) {\n if (line.trim() === '') continue;\n yield line;\n }\n}\n\n/**\n * Iterate URLs from either a single CLI arg or from stdin (if the arg is `-`).\n */\nexport async function* urlIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readUrlsFromStdin();\n return;\n }\n yield arg;\n}\n\n/**\n * Iterate `serp` queries from a single CLI arg or from stdin (if it is `-`).\n */\nexport async function* queryIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readQueriesFromStdin();\n return;\n }\n yield arg;\n}\n","/**\n * SDK adapter.\n *\n * Wraps the published `webscraping-ai` Node SDK. The CLI sends a\n * `from_cli=true` analytics flag on every endpoint except `account` (mirrors\n * `from_n8n` and `from_mcp_server` used by the n8n node and MCP server). On\n * `serp` and `data` it rides in the SDK's `params` pass-through, since those\n * methods drop unknown top-level keys. `account()` takes no options, so it\n * isn't tagged. The SDK's typed\n * options don't accept arbitrary keys, so we go through a single cast point\n * here rather than scattering casts across each command.\n */\n\nimport { WebScrapingAI } from 'webscraping-ai';\n\nimport { registerSecret } from './redact.js';\nimport type {\n DataOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nexport interface CreateClientOptions {\n apiKey: string;\n /** Per-request timeout in ms. Maps to the SDK's `timeoutMs`. */\n requestTimeoutMs?: number;\n}\n\n/**\n * Returns a `WebScrapingAI` instance with the CLI's analytics tag baked into\n * every endpoint method. The methods preserve the SDK's public signatures.\n */\nexport function createClient(options: CreateClientOptions): WebScrapingAI {\n registerSecret(options.apiKey);\n const inner = new WebScrapingAI({\n apiKey: options.apiKey,\n timeoutMs: options.requestTimeoutMs,\n });\n\n // Wrap each endpoint method so it injects the CLI analytics tag. We splice\n // the tag in at this seam rather than at every callsite. The cast is local;\n // the SDK passes unknown query params through to the wire.\n const tag = { from_cli: true } as unknown as Record<string, never>;\n\n return new Proxy(inner, {\n get(target, prop, receiver) {\n const value = Reflect.get(target, prop, receiver);\n if (typeof value !== 'function') return value;\n switch (prop) {\n case 'html':\n return (o: HtmlOptions) => target.html({ ...o, ...tag });\n case 'text':\n return (o: TextOptions) => target.text({ ...o, ...tag });\n case 'selected':\n return (o: SelectedOptions) => target.selected({ ...o, ...tag });\n case 'selectedMultiple':\n return (o: SelectedMultipleOptions) => target.selectedMultiple({ ...o, ...tag });\n case 'question':\n return (o: QuestionOptions) => target.question({ ...o, ...tag });\n case 'fields':\n return (o: FieldsOptions) => target.fields({ ...o, ...tag });\n case 'serp':\n // `serp()` drops unknown top-level keys but sends its `params`\n // pass-through as-is, so the tag goes there (CLI tag wins).\n return (o: SerpOptions) => target.serp({ ...o, params: { ...o.params, from_cli: true } });\n case 'data':\n // `data()` drops unknown top-level keys but sends its `params`\n // pass-through as-is, so the tag goes there (CLI tag wins).\n return (o: DataOptions) => target.data({ ...o, params: { ...o.params, from_cli: true } });\n case 'account':\n // `account` takes no options; the SDK builds the request itself, so\n // there's no seam here for the analytics tag — accepted limitation.\n return value.bind(target);\n default:\n return typeof value === 'function' ? value.bind(target) : value;\n }\n },\n });\n}\n","/**\n * Secret redaction for anything the CLI prints to stderr.\n *\n * API errors can echo the request URL (e.g. an HTML error page that includes\n * `?api_key=...`), so every error message goes through `redactSecrets()`\n * before it is printed. The key in use is registered by `createClient()`.\n */\n\nconst secrets = new Set<string>();\n\n/** Remember a secret so later error output never prints it. */\nexport function registerSecret(value: string | undefined): void {\n if (value && value.trim() !== '') secrets.add(value);\n}\n\n/** Forget registered secrets (tests only). */\nexport function clearSecrets(): void {\n secrets.clear();\n}\n\nconst API_KEY_PARAM = /(api_key=)[^&\\s\"'<>]*/gi;\n\nexport function redactSecrets(text: string): string {\n let out = text.replace(API_KEY_PARAM, '$1[REDACTED]');\n const candidates = new Set(secrets);\n const env = process.env.WEBSCRAPING_AI_API_KEY?.trim();\n if (env) candidates.add(env);\n for (const secret of candidates) {\n for (const form of new Set([secret, encodeURIComponent(secret)])) {\n out = out.split(form).join('[REDACTED]');\n }\n }\n return out;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildQuestionOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface AskFlags extends CommonRawFlags {\n question?: string;\n}\n\nexport function askCommand(): Command {\n const cmd = new Command('ask')\n .description('Ask an AI question about a page (wraps /ai/question)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption('-q, --question <text>', 'the question / instructions for the model')\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: AskFlags) => {\n if (!opts.question) throw new Error('--question is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });\n const result = await client.question(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Pure builder that maps `--flag` values into SDK option objects.\n *\n * Extracted as a separate function so unit tests can pin the mapping without\n * spinning up commander or making HTTP calls. Each command handler reads its\n * own commander options, awaits `parseHeaders` / `parseFields`, then calls\n * the matching `build*` function here.\n */\n\nimport type {\n CommonRequestOptions,\n DataOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nimport type { CommonRawFlags } from './options.js';\n\nexport interface ResolvedCommonOptions {\n headers?: Record<string, string>;\n}\n\nexport function buildCommonOptions(\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions = {},\n): CommonRequestOptions {\n const out: CommonRequestOptions = {};\n // `flags.js` is `false` when `--no-js`, `true` when `--js`, `undefined`\n // when neither passed. Pass through only when explicit, so the API default\n // applies otherwise.\n if (flags.js !== undefined) out.js = flags.js;\n if (flags.proxy !== undefined) out.proxy = flags.proxy;\n if (flags.country !== undefined) out.country = flags.country;\n if (flags.device !== undefined) out.device = flags.device;\n if (flags.timeout !== undefined) out.timeout = flags.timeout;\n if (flags.jsTimeout !== undefined) out.js_timeout = flags.jsTimeout;\n if (flags.waitFor !== undefined) out.wait_for = flags.waitFor;\n if (flags.jsScript !== undefined) out.js_script = flags.jsScript;\n if (flags.customProxy !== undefined) out.custom_proxy = flags.customProxy;\n if (flags.errorOn404) out.error_on_404 = true;\n if (flags.errorOnRedirect) out.error_on_redirect = true;\n if (resolved.headers) out.headers = resolved.headers;\n return out;\n}\n\nexport interface HtmlExtras {\n returnScriptResult?: boolean;\n}\n\nexport function buildHtmlOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: HtmlExtras = {},\n): HtmlOptions {\n const out: HtmlOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n if (extras.returnScriptResult) out.return_script_result = true;\n return out;\n}\n\nexport interface TextExtras {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function buildTextOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: TextExtras = {},\n): TextOptions {\n const out: TextOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (extras.textFormat) out.text_format = extras.textFormat;\n if (extras.returnLinks) out.return_links = true;\n return out;\n}\n\nexport function buildSelectedOptions(\n url: string,\n selector: string | undefined,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedOptions {\n // `selector` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when provided. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selector !== undefined) out.selector = selector;\n if (flags.format) out.format = flags.format;\n return out as SelectedOptions;\n}\n\nexport function buildSelectedMultipleOptions(\n url: string,\n selectors: readonly string[],\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedMultipleOptions {\n // `selectors` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when non-empty. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedMultipleOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selectors.length > 0) out.selectors = selectors;\n return out as SelectedMultipleOptions;\n}\n\nexport function buildQuestionOptions(\n url: string,\n question: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): QuestionOptions {\n const out: QuestionOptions = { url, question, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n return out;\n}\n\nexport function buildFieldsOptions(\n url: string,\n fields: Record<string, string>,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): FieldsOptions {\n return { url, fields, ...buildCommonOptions(flags, resolved) };\n}\n\nexport interface SerpRawFlags {\n engine?: 'google';\n gl?: string;\n hl?: string;\n page?: number;\n}\n\n/**\n * `/serp` is query-shaped: it takes none of the common scrape options, so\n * this deliberately does not go through `buildCommonOptions`. `page` must be\n * an integer >= 1 (the server also rejects anything else with a 400, not\n * billed; checking client-side saves the round trip). Pages are 1-100: the\n * server rejects a page above 100 with a 400.\n */\nexport function buildSerpOptions(query: string, flags: SerpRawFlags = {}): SerpOptions {\n // Validate on the trimmed value but send the query as the user typed it.\n if (typeof query !== 'string' || query.trim() === '') {\n throw new Error('serp requires a non-empty query');\n }\n const out: SerpOptions = { q: query };\n if (flags.engine !== undefined) out.engine = flags.engine;\n if (flags.gl) out.gl = flags.gl;\n if (flags.hl) out.hl = flags.hl;\n if (flags.page !== undefined) {\n if (!Number.isSafeInteger(flags.page) || flags.page < 1) {\n throw new Error(`--page must be an integer >= 1, got: ${flags.page}`);\n }\n out.page = flags.page;\n }\n return out;\n}\n\nexport interface DataRawFlags {\n country?: string;\n transcript?: boolean;\n transcriptLanguage?: string;\n /** Raw repeatable `--param key=value` strings. */\n param?: readonly string[];\n}\n\n/** Keys `--param` must never set: owned by the SDK/CLI. */\nconst RESERVED_DATA_PARAMS = new Set(['api_key', 'url', 'from_cli', '__proto__']);\n\n/** `--param` keys that have their own flag: reject and point at the flag. */\nconst NAMED_DATA_PARAMS: Record<string, string> = {\n country: '--country',\n transcript: '--transcript',\n transcript_language: '--transcript-language',\n};\n\n/**\n * Parse repeatable `--param key=value` flags into a plain object. Splits on\n * the first `=`, so values may contain `=`. Throws on a missing `=`, an empty\n * or repeated key, a reserved key (`api_key`, `url`, `from_cli`,\n * `__proto__`), or a key that has its own flag (`country` → use `--country`).\n */\nexport function parseDataParams(raw: readonly string[] = []): Record<string, string> {\n const out: Record<string, string> = Object.create(null) as Record<string, string>;\n for (const entry of raw) {\n const eq = entry.indexOf('=');\n if (eq <= 0) throw new Error(`--param must be key=value, got: ${entry}`);\n const key = entry.slice(0, eq);\n if (RESERVED_DATA_PARAMS.has(key)) {\n throw new Error(`--param must not set \"${key}\"`);\n }\n if (Object.hasOwn(NAMED_DATA_PARAMS, key)) {\n throw new Error(`--param must not set \"${key}\"; use ${NAMED_DATA_PARAMS[key]}`);\n }\n if (Object.hasOwn(out, key)) {\n throw new Error(`--param \"${key}\" given more than once`);\n }\n out[key] = entry.slice(eq + 1);\n }\n return { ...out };\n}\n\n/**\n * `/data` is URL-shaped but takes none of the common scrape options, so this\n * deliberately does not go through `buildCommonOptions`. The URL is passed\n * through unmodified: there is no client-side site check (sites are added\n * server-side; the API answers an unsupported URL with a free 400).\n */\nexport function buildDataOptions(url: string, flags: DataRawFlags = {}): DataOptions {\n if (typeof url !== 'string' || url.trim() === '') {\n throw new Error('data requires a non-empty url');\n }\n const out: DataOptions = { url };\n if (flags.country) out.country = flags.country;\n if (flags.transcript !== undefined) out.transcript = flags.transcript;\n if (flags.transcriptLanguage) out.transcript_language = flags.transcriptLanguage;\n const params = parseDataParams(flags.param);\n if (Object.keys(params).length > 0) out.params = params;\n return out;\n}\n","/**\n * Shared commander options and the parsers/expanders behind them.\n *\n * All scrape endpoints share the same nine \"common\" flags. We define them in\n * one place and apply via `addCommonScrapeOptions(cmd)` so the surface stays\n * consistent and any change lands everywhere at once.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { Option } from 'commander';\nimport type { Command } from 'commander';\n\nexport interface CommonRawFlags {\n apiKey?: string;\n output?: string;\n format?: 'text' | 'json';\n pretty?: boolean;\n js?: boolean;\n proxy?: 'datacenter' | 'residential' | 'stealth';\n country?: string;\n device?: 'desktop' | 'mobile' | 'tablet';\n timeout?: number;\n jsTimeout?: number;\n waitFor?: string;\n headers?: string;\n jsScript?: string;\n customProxy?: string;\n errorOn404?: boolean;\n errorOnRedirect?: boolean;\n}\n\n/**\n * Add the flags every scrape-endpoint command needs. Adding here keeps the\n * surface aligned across `html`, `text`, `selected`, etc.\n */\nexport function addCommonScrapeOptions(cmd: Command): Command {\n return cmd\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .option('--js', 'execute JS via a headless browser (default: enabled)')\n .option('--no-js', 'disable JS execution (faster, cheaper)')\n .addOption(\n new Option('--proxy <type>', 'proxy pool').choices(['datacenter', 'residential', 'stealth']),\n )\n .option('--country <code>', 'two-letter proxy country code, e.g. us, gb, de')\n .addOption(\n new Option('--device <type>', 'device emulation').choices(['desktop', 'mobile', 'tablet']),\n )\n .option('--timeout <ms>', 'page-retrieval timeout in ms (max 30000)', integerParser)\n .option('--js-timeout <ms>', 'JS rendering time in ms after load (max 20000)', integerParser)\n .option('--wait-for <selector>', 'CSS selector to wait for before returning content')\n .option(\n '--headers <json>',\n 'HTTP headers to forward to the target page; JSON object or @file.json',\n )\n .option('--js-script <code>', 'custom JS to evaluate on the target page')\n .option('--custom-proxy <url>', 'custom proxy URL: http://user:pass@host:port')\n .option('--error-on-404', 'return error on 404')\n .option('--error-on-redirect', 'return error on redirect');\n}\n\n/**\n * Strict positive-integer check for flags like `serp --page`: digits only,\n * no leading zero, so `1.5`, `2abc`, `0` and `-1` are all rejected (unlike\n * `parseInt`, which would read `2abc` as 2). Values beyond\n * `Number.MAX_SAFE_INTEGER` are rejected too.\n */\nexport function isPositiveIntegerString(value: string): boolean {\n // isSafeInteger: a huge digit string becomes e.g. 1e+21 on the wire, which\n // the server rejects with a 400 (not billed); checking here saves the round trip.\n return /^[1-9]\\d*$/.test(value) && Number.isSafeInteger(Number(value));\n}\n\nexport function integerParser(value: string): number {\n const n = Number.parseInt(value, 10);\n if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);\n return n;\n}\n\n/**\n * Parse a `--headers` argument. Accepts a JSON object literal or `@path` to\n * load JSON from a file. Returns `undefined` if the input is empty.\n */\nexport async function parseHeaders(\n input: string | undefined,\n): Promise<Record<string, string> | undefined> {\n if (!input || input.trim() === '') return undefined;\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n return parseJsonObject(text, 'headers');\n}\n\n/**\n * Parse a `--fields` argument for `extract`. Accepts JSON object literal or\n * `@path` to a file.\n */\nexport async function parseFields(input: string): Promise<Record<string, string>> {\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n const parsed = parseJsonObject(text, 'fields');\n if (!parsed || Object.keys(parsed).length === 0) {\n throw new Error('fields must be a non-empty JSON object of { fieldName: description }');\n }\n return parsed;\n}\n\nfunction parseJsonObject(text: string, label: string): Record<string, string> {\n let parsed: unknown;\n try {\n parsed = JSON.parse(text);\n } catch (err) {\n throw new Error(`${label}: invalid JSON — ${(err as Error).message}`);\n }\n if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {\n throw new Error(`${label} must be a JSON object`);\n }\n const out: Record<string, string> = {};\n for (const [k, v] of Object.entries(parsed as Record<string, unknown>)) {\n if (typeof v !== 'string') throw new Error(`${label}.${k}: value must be a string`);\n out[k] = v;\n }\n return out;\n}\n","import { Command } from 'commander';\n\nimport {\n clearPersistedConfig,\n configPath,\n maskApiKey,\n readPersistedConfig,\n writePersistedConfig,\n} from '../lib/config.js';\n\nexport function authCommand(): Command {\n const cmd = new Command('auth').description('Manage your stored API key');\n\n cmd\n .command('set <key>')\n .description(`Save an API key to ${configPath()}`)\n .action(async (key: string) => {\n if (!key || key.trim() === '') throw new Error('API key is empty');\n await writePersistedConfig({ apiKey: key.trim() });\n process.stdout.write(`API key saved to ${configPath()}\\n`);\n });\n\n cmd\n .command('show')\n .description('Display the currently stored API key (masked)')\n .action(async () => {\n const persisted = await readPersistedConfig();\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') {\n process.stdout.write(\n `Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}\\n`,\n );\n }\n if (persisted.apiKey && persisted.apiKey.trim() !== '') {\n process.stdout.write(\n `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}\\n`,\n );\n }\n if (!env && !persisted.apiKey) {\n process.stdout.write('No API key configured. Run `webscraping-ai auth set <key>`.\\n');\n }\n });\n\n cmd\n .command('clear')\n .description('Remove the stored API key file')\n .action(async () => {\n const removed = await clearPersistedConfig();\n process.stdout.write(\n removed ? `Removed ${configPath()}\\n` : 'No stored API key to remove.\\n',\n );\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildDataOptions, parseDataParams } from '../lib/buildRequest.js';\nimport type { DataRawFlags } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIsStdin, urlIterator } from '../lib/stdin.js';\n\ninterface DataFlags extends DataRawFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nfunction collect(value: string, previous: string[] = []): string[] {\n return [...previous, value];\n}\n\n/**\n * `data <url>` — structured JSON for a page on a supported site (wraps /data).\n *\n * URL-shaped, but the server picks fetching/proxy/parsing per site, so the\n * common scrape flags (`--js`, `--proxy`, `--headers`, …) don't apply and are\n * deliberately not registered. The URL is never checked against a list of\n * sites: new ones are added server-side. An unsupported URL or page type\n * returns a 400 that is not charged (exit code 3); its message lists what is\n * supported. `-` reads one URL per line from stdin (blank and `#` lines\n * skipped, and the batch stops at the first failing item, like the other URL\n * commands). `--param key=value` passes site-specific parameters through.\n *\n * Usage errors (blank url, malformed/reserved `--param`) exit with code 2\n * before stdin is read or the API key is resolved.\n */\nexport function dataCommand(): Command {\n const cmd = new Command('data')\n .description(\n 'Structured JSON for a page on a supported site, e.g. YouTube, TikTok, X, LinkedIn, ' +\n 'Instagram, Reddit (wraps /data, 15 credits/request; more sites are added server-side, ' +\n 'an unsupported URL or page type gets a 400, not charged, listing what is supported)',\n )\n .argument('<url>', 'page URL (or `-` to read URLs from stdin, one per line)')\n .option(\n '--country <code>',\n 'two-letter country code of the proxy used to fetch the page (default: us)',\n )\n .option(\n '--transcript',\n 'YouTube videos only: also fetch the transcript into data.transcript (null without matching captions; a failed transcript fetch fails the request with a 500, not charged)',\n )\n .option(\n '--transcript-language <code>',\n 'caption language to pick, e.g. en or de (default: English, then the first track)',\n )\n .option(\n '--param <key=value>',\n 'extra site-specific query parameter sent as-is (repeatable)',\n collect,\n )\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (url: string, opts: DataFlags) => {\n if (!urlIsStdin(url) && url.trim() === '') {\n cmd.error('error: data requires a non-empty url', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.emptyUrl',\n });\n }\n try {\n parseDataParams(opts.param);\n } catch (err) {\n cmd.error(`error: ${(err as Error).message}`, {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.invalidParam',\n });\n }\n\n const flags: DataRawFlags = {\n country: opts.country,\n transcript: opts.transcript,\n transcriptLanguage: opts.transcriptLanguage,\n param: opts.param,\n };\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const result = await client.data(buildDataOptions(target, flags));\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Exit-code mapping for SDK errors.\n *\n * The SDK throws typed subclasses of `WebScrapingAIError`. Map each to a\n * stable exit code so shell scripts can branch on the failure mode without\n * parsing stderr.\n */\n\nimport {\n APIConnectionError,\n APIError,\n APITimeoutError,\n AuthenticationError,\n BadRequestError,\n GatewayTimeoutError,\n PaymentRequiredError,\n RateLimitError,\n ServerError,\n WebScrapingAIError,\n} from 'webscraping-ai';\n\nimport { redactSecrets } from './redact.js';\n\nexport const EXIT_CODES = {\n ok: 0,\n generic: 1,\n usage: 2,\n badRequest: 3,\n auth: 4,\n paymentRequired: 5,\n rateLimit: 6,\n serverError: 7,\n timeout: 8,\n connection: 9,\n} as const;\n\nexport function exitCodeFor(err: unknown): number {\n if (err instanceof BadRequestError) return EXIT_CODES.badRequest;\n if (err instanceof AuthenticationError) return EXIT_CODES.auth;\n if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;\n if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;\n if (err instanceof ServerError) return EXIT_CODES.serverError;\n if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APITimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APIConnectionError) return EXIT_CODES.connection;\n if (err instanceof APIError) return EXIT_CODES.serverError;\n if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;\n return EXIT_CODES.generic;\n}\n\n/**\n * Human-readable one-liner for stderr. Always redacted: the API key and any\n * `api_key=...` pattern are replaced, since error bodies can echo the URL.\n */\nexport function formatError(err: unknown): string {\n return redactSecrets(rawMessage(err));\n}\n\nfunction rawMessage(err: unknown): string {\n if (err instanceof APIError) {\n const parts = [err.message];\n if (err.status) parts.push(`(HTTP ${err.status})`);\n return parts.join(' ');\n }\n if (err instanceof Error) return err.message;\n return String(err);\n}\n","import { Command } from 'commander';\n\nimport { buildFieldsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseFields, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface ExtractFlags extends CommonRawFlags {\n fields?: string;\n}\n\nexport function extractCommand(): Command {\n const cmd = new Command('extract')\n .description('Extract structured fields from a page using AI (wraps /ai/fields)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption(\n '-f, --fields <json>',\n 'fields to extract — JSON object {name: description} or @file.json',\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: ExtractFlags) => {\n if (!opts.fields) throw new Error('--fields is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const fields = await parseFields(opts.fields);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });\n const result = await client.fields(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildHtmlOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface HtmlFlags extends CommonRawFlags {\n returnScriptResult?: boolean;\n}\n\nexport function htmlCommand(): Command {\n const cmd = new Command('html')\n .description('Fetch the full HTML of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--return-script-result', 'return the result of --js-script instead of HTML'),\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: HtmlFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildHtmlOptions(\n target,\n opts,\n { headers },\n {\n returnScriptResult: opts.returnScriptResult,\n },\n );\n const result = await client.html(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSelectedOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedFlags extends CommonRawFlags {\n selector?: string;\n}\n\nexport function selectedCommand(): Command {\n const cmd = new Command('selected')\n .description('Fetch HTML of one CSS-selected page area')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector for the target element(s); omit for whole-page HTML',\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });\n const result = await client.selected(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildSelectedMultipleOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedMultipleFlags extends CommonRawFlags {\n selector?: string[];\n}\n\nexport function selectedMultipleCommand(): Command {\n const cmd = new Command('selected-multiple')\n .description('Fetch HTML of multiple CSS-selected page areas')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector — pass multiple times for multiple selectors',\n collectMulti,\n [] as string[],\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedMultipleFlags) => {\n // `--selector` is optional: omitting it returns whole-page HTML.\n const selectors = opts.selector ?? [];\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });\n const result = await client.selectedMultiple(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n\nfunction collectMulti(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSerpOptions } from '../lib/buildRequest.js';\nimport type { SerpRawFlags } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { isPositiveIntegerString } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { queryIterator, urlIsStdin } from '../lib/stdin.js';\n\ninterface SerpFlags extends Omit<SerpRawFlags, 'page'> {\n /** Raw `--page` string; validated strictly in the action. */\n page?: string;\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\n/**\n * `serp <query...>` — parsed search engine results (wraps /serp).\n *\n * Query-shaped, not URL-shaped, so none of the common scrape flags\n * (`--js`, `--proxy`, `--country`, …) apply and they're deliberately not\n * registered here. Multiple words are joined with spaces, so quoting the\n * query is optional. A lone `-` reads one query per line from stdin; only\n * blank lines are skipped (a leading `#` is part of the query, not a comment).\n *\n * Usage errors (bad `--page`, empty query) exit with code 2 before stdin is\n * read or the API key is resolved.\n */\nexport function serpCommand(): Command {\n const cmd = new Command('serp')\n .alias('search')\n .description('Search Google and print parsed results as JSON (wraps /serp, 15 credits/search)')\n .argument('<query...>', 'search query (or `-` to read queries from stdin, one per line)')\n .addOption(new Option('--engine <name>', 'search engine (default: google)').choices(['google']))\n .option('--gl <code>', 'two-letter country code for search geolocation (default: us)')\n .option('--hl <code>', 'two-letter language code for the results (default: en)')\n .option(\n '--page <n>',\n 'results page number, integer >= 1, 10 results per page (default: 1; server rejects > 100 with a 400)',\n )\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (words: string[], opts: SerpFlags) => {\n const query = words.join(' ');\n\n let page: number | undefined;\n if (opts.page !== undefined) {\n if (!isPositiveIntegerString(opts.page)) {\n cmd.error(`error: --page must be an integer >= 1, got: ${opts.page}`, {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.invalidPage',\n });\n }\n page = Number(opts.page);\n }\n if (!urlIsStdin(query) && query.trim() === '') {\n cmd.error('error: serp requires a non-empty query', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.emptyQuery',\n });\n }\n\n const flags: SerpRawFlags = { engine: opts.engine, gl: opts.gl, hl: opts.hl, page };\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const write = await openEmitter(opts, query);\n\n for await (const q of queryIterator(query)) {\n const result = await client.serp(buildSerpOptions(q, flags));\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { EDITORS, editorById, installSkill, resolveTarget } from '../lib/skill.js';\nimport type { EditorAdapter } from '../lib/skill.js';\n\ninterface SetupSkillFlags {\n editor?: string[];\n all?: boolean;\n force?: boolean;\n}\n\nexport function setupCommand(): Command {\n const cmd = new Command('setup').description('Install the AI agent skill into your editor(s)');\n\n cmd\n .command('skill')\n .description('Install the WebScraping.AI skill into one or more AI coding editors')\n .option(\n '-e, --editor <id>',\n `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(', ')}`,\n collect,\n [] as string[],\n )\n .option('-a, --all', 'install into every supported editor')\n .option('-f, --force', 'overwrite existing skill files')\n .action(async (opts: SetupSkillFlags) => {\n const selected = resolveSelection(opts);\n if (selected.length === 0) {\n throw new Error(\n 'no editor selected — pass --all or --editor <id> (e.g. --editor claude-code)',\n );\n }\n\n for (const editor of selected) {\n const target = resolveTarget(editor);\n const result = await installSkill(target, { force: opts.force });\n process.stdout.write(`${editor.label}: ${result.status} → ${target.destination}\\n`);\n }\n });\n\n cmd\n .command('list', { hidden: false })\n .description('List supported editors for `setup skill`')\n .action(() => {\n for (const e of EDITORS) {\n const target = resolveTarget(e);\n process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}\\n`);\n }\n });\n\n return cmd;\n}\n\nfunction collect(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n\nfunction resolveSelection(opts: SetupSkillFlags): EditorAdapter[] {\n if (opts.all) return [...EDITORS];\n const ids = opts.editor ?? [];\n return ids.map((id) => {\n const editor = editorById(id);\n if (!editor) throw new Error(`unknown editor: ${id}`);\n return editor;\n });\n}\n","/**\n * Multi-editor skill installer.\n *\n * The canonical skill lives in this package at\n * `plugins/webscraping-ai/skills/webscraping-ai/SKILL.md`. Each supported\n * editor reads skill content from a different location and may want either\n * a single Markdown file or a whole skill directory. The `editors` registry\n * captures those differences so `setup skill` is a small driver over it.\n *\n * `mode: 'dir'` copies the entire skill directory (Claude Code-shaped\n * editors that scan a folder per skill). `mode: 'file'` writes the SKILL.md\n * contents to a single file (editors that read flat instructions, like\n * GitHub Copilot's `.github/copilot-instructions.md`).\n *\n * `kind: 'user'` writes to `$HOME`; `kind: 'project'` writes to the current\n * working directory. We prefer user-scope where the editor supports it so\n * the skill becomes globally available without polluting individual repos.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nexport type EditorMode = 'dir' | 'file';\nexport type EditorKind = 'user' | 'project';\n\nexport interface EditorAdapter {\n /** Stable identifier used by `--editor <name>`. */\n id: string;\n /** Human-readable label printed in summaries. */\n label: string;\n mode: EditorMode;\n kind: EditorKind;\n /**\n * Where the skill ends up. For `mode: 'dir'`, the skill directory copies\n * to `{root}/{path}`. For `mode: 'file'`, the SKILL.md contents write to\n * `{root}/{path}` (i.e. `path` ends in a filename).\n *\n * `root` is `$HOME` (user) or CWD (project).\n */\n path: string;\n}\n\n/**\n * Registry of supported editors. Order matters: it's also the order the\n * `setup skill` summary lists them in.\n */\nexport const EDITORS: readonly EditorAdapter[] = [\n // Claude Code: user-scope skills live under ~/.claude/skills/<name>/\n {\n id: 'claude-code',\n label: 'Claude Code',\n mode: 'dir',\n kind: 'user',\n path: '.claude/skills/webscraping-ai',\n },\n // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill\n // dir, so this is the editor's \"install\" surface.\n {\n id: 'cursor',\n label: 'Cursor',\n mode: 'file',\n kind: 'project',\n path: '.cursor/rules/webscraping-ai.mdc',\n },\n // Windsurf: project-scope rules. Same shape as Cursor.\n {\n id: 'windsurf',\n label: 'Windsurf',\n mode: 'file',\n kind: 'project',\n path: '.windsurf/rules/webscraping-ai.md',\n },\n // Kiro: user-scope steering directory.\n { id: 'kiro', label: 'Kiro', mode: 'dir', kind: 'user', path: '.kiro/steering/webscraping-ai' },\n // OpenCode: project-scope skills (mirrors Claude shape).\n {\n id: 'opencode',\n label: 'OpenCode',\n mode: 'dir',\n kind: 'project',\n path: '.opencode/skills/webscraping-ai',\n },\n // Gemini CLI: user-scope extension dir with a GEMINI.md.\n {\n id: 'gemini',\n label: 'Gemini CLI',\n mode: 'file',\n kind: 'user',\n path: '.gemini/extensions/webscraping-ai/GEMINI.md',\n },\n // GitHub Copilot: project-scope custom instructions file. Single file.\n {\n id: 'copilot',\n label: 'GitHub Copilot',\n mode: 'file',\n kind: 'project',\n path: '.github/copilot-instructions.md',\n },\n // Augment: project-scope rules file.\n {\n id: 'augment',\n label: 'Augment',\n mode: 'file',\n kind: 'project',\n path: '.augment/rules/webscraping-ai.md',\n },\n // Factory: project-scope rules file.\n {\n id: 'factory',\n label: 'Factory',\n mode: 'file',\n kind: 'project',\n path: '.factory/rules/webscraping-ai.md',\n },\n] as const;\n\nexport function editorById(id: string): EditorAdapter | undefined {\n return EDITORS.find((e) => e.id === id);\n}\n\n/**\n * Where this package keeps the canonical skill. Resolved relative to the\n * compiled `dist/cli.js` so it works after `npm install -g`.\n */\nexport function bundledSkillDir(): string {\n const here = dirname(fileURLToPath(import.meta.url));\n // dist/ sits at the package root, so the plugins dir is two levels up\n // from any compiled file: dist/cli.js → package root → plugins/...\n return resolve(here, '..', 'plugins', 'webscraping-ai', 'skills', 'webscraping-ai');\n}\n\nexport interface InstallTarget {\n editor: EditorAdapter;\n /** Absolute destination path (file or directory depending on `mode`). */\n destination: string;\n}\n\nexport interface ResolveOptions {\n cwd?: string;\n home?: string;\n}\n\nexport function resolveTarget(editor: EditorAdapter, opts: ResolveOptions = {}): InstallTarget {\n const root = editor.kind === 'user' ? (opts.home ?? homedir()) : (opts.cwd ?? process.cwd());\n return { editor, destination: join(root, editor.path) };\n}\n\nexport interface InstallOptions {\n /** If true, overwrite existing files/dirs. */\n force?: boolean;\n /** Override skill source dir (for tests). */\n sourceDir?: string;\n}\n\nexport type InstallStatus = 'installed' | 'skipped-exists' | 'overwritten';\n\nexport interface InstallResult {\n target: InstallTarget;\n status: InstallStatus;\n}\n\nexport async function installSkill(\n target: InstallTarget,\n opts: InstallOptions = {},\n): Promise<InstallResult> {\n const source = opts.sourceDir ?? bundledSkillDir();\n const exists = await pathExists(target.destination);\n\n if (exists && !opts.force) {\n return { target, status: 'skipped-exists' };\n }\n\n if (target.editor.mode === 'dir') {\n if (exists) await fs.rm(target.destination, { recursive: true, force: true });\n await copyDirectory(source, target.destination);\n } else {\n const skillBody = await fs.readFile(join(source, 'SKILL.md'), 'utf8');\n await fs.mkdir(dirname(target.destination), { recursive: true });\n await fs.writeFile(target.destination, skillBody);\n }\n\n return { target, status: exists ? 'overwritten' : 'installed' };\n}\n\nasync function pathExists(path: string): Promise<boolean> {\n try {\n await fs.access(path);\n return true;\n } catch {\n return false;\n }\n}\n\nasync function copyDirectory(source: string, destination: string): Promise<void> {\n // `fs.cp` with `recursive` exists on Node 16.7+; we target Node 20.\n await fs.cp(source, destination, { recursive: true, force: true });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildTextOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface TextFlags extends CommonRawFlags {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function textCommand(): Command {\n const cmd = new Command('text')\n .description('Fetch the visible text of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--text-format <fmt>', 'text response format').choices(['plain', 'json', 'xml']),\n )\n .option('--return-links', 'include page links (text-format=json only)');\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: TextFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildTextOptions(\n target,\n opts,\n { headers },\n {\n textFormat: opts.textFormat,\n returnLinks: opts.returnLinks,\n },\n );\n const result = await client.text(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n"],"mappings":";;;AAQA,SAAS,WAAAA,iBAAe;AACxB,SAAS,oBAAoB;AAC7B,SAAS,WAAAC,UAAS,WAAAC,gBAAe;AACjC,SAAS,iBAAAC,sBAAqB;;;ACX9B,SAAS,eAAe;;;ACYxB,SAAS,YAAY,UAAU;AAC/B,SAAS,eAAe;AACxB,SAAS,SAAS,YAAY;AAE9B,IAAM,kBAAkB;AAEjB,SAAS,YAAoB;AAClC,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,gBAAgB;AAC/D,SAAO,KAAK,QAAQ,GAAG,WAAW,gBAAgB;AACpD;AAEO,SAAS,aAAqB;AACnC,SAAO,KAAK,UAAU,GAAG,eAAe;AAC1C;AAMA,eAAsB,sBAAgD;AACpE,MAAI;AACF,UAAM,OAAO,MAAM,GAAG,SAAS,WAAW,GAAG,MAAM;AACnD,UAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,QAAI,UAAU,OAAO,WAAW,YAAY,CAAC,MAAM,QAAQ,MAAM,GAAG;AAClE,aAAO;AAAA,IACT;AACA,WAAO,CAAC;AAAA,EACV,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO,CAAC;AAC9D,UAAM;AAAA,EACR;AACF;AAEA,eAAsB,qBAAqB,QAAwC;AACjF,QAAM,OAAO,WAAW;AACxB,QAAM,GAAG,MAAM,QAAQ,IAAI,GAAG,EAAE,WAAW,KAAK,CAAC;AACjD,QAAM,GAAG,UAAU,MAAM,KAAK,UAAU,QAAQ,MAAM,CAAC,IAAI,MAAM,EAAE,MAAM,IAAM,CAAC;AAKhF,QAAM,GAAG,MAAM,MAAM,GAAK;AAC5B;AAEA,eAAsB,uBAAyC;AAC7D,MAAI;AACF,UAAM,GAAG,OAAO,WAAW,CAAC;AAC5B,WAAO;AAAA,EACT,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO;AAC7D,UAAM;AAAA,EACR;AACF;AAOA,eAAsB,cAAc,OAA6B,CAAC,GAAoB;AACpF,MAAI,KAAK,QAAQ,KAAK,KAAK,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,KAAK;AAEhE,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,IAAI,KAAK;AAE9C,QAAM,YAAY,MAAM,oBAAoB;AAC5C,MAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,GAAI,QAAO,UAAU,OAAO,KAAK;AAErF,QAAM,IAAI;AAAA,IACR;AAAA,EACF;AACF;AAEO,SAAS,WAAW,KAAqB;AAC9C,MAAI,IAAI,UAAU,EAAG,QAAO,IAAI,OAAO,IAAI,MAAM;AACjD,SAAO,GAAG,IAAI,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,OAAO,IAAI,SAAS,CAAC,CAAC,GAAG,IAAI,MAAM,EAAE,CAAC;AACxE;;;AC3EA,SAAS,YAAYC,WAAU;;;ACH/B,OAAO,cAAc;AAEd,SAAS,WAAW,KAAsB;AAC/C,SAAO,QAAQ;AACjB;AAEA,gBAAuB,oBAA2C;AAChE,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,QAAQ,OAAO,WAAW,SAAS,CAAC;AACjF,mBAAiB,OAAO,IAAI;AAC1B,UAAM,OAAO,IAAI,KAAK;AACtB,QAAI,SAAS,MAAM,KAAK,WAAW,GAAG,EAAG;AACzC,UAAM;AAAA,EACR;AACF;AAMA,gBAAuB,qBACrB,QAA+B,QAAQ,OAChB;AACvB,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,WAAW,SAAS,CAAC;AAClE,mBAAiB,QAAQ,IAAI;AAC3B,QAAI,KAAK,KAAK,MAAM,GAAI;AACxB,UAAM;AAAA,EACR;AACF;AAKA,gBAAuB,YAAY,KAAoC;AACrE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,kBAAkB;AACzB;AAAA,EACF;AACA,QAAM;AACR;AAKA,gBAAuB,cAAc,KAAoC;AACvE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,qBAAqB;AAC5B;AAAA,EACF;AACA,QAAM;AACR;;;ADhCA,eAAsB,KACpB,OACA,OAAoB,CAAC,GACrB,QAA0B,CAAC,GACZ;AACf,QAAM,OAAO,UAAU,OAAO,IAAI;AAElC,MAAI,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC5C,UAAM,OAAO,QAAQ,KAAK,SAAS,IAAI,IAAI,KAAK;AAChD,QAAI,MAAM,OAAQ,OAAMC,IAAG,WAAW,KAAK,QAAQ,IAAI;AAAA,QAClD,OAAMA,IAAG,UAAU,KAAK,QAAQ,IAAI;AACzC;AAAA,EACF;AAEA,UAAQ,OAAO,MAAM,IAAI;AACzB,MAAI,CAAC,KAAK,SAAS,IAAI,EAAG,SAAQ,OAAO,MAAM,IAAI;AACrD;AAMO,SAAS,cAAc,OAAoB,CAAC,GAAsC;AACvF,MAAI,QAAQ;AACZ,SAAO,OAAO,UAAmB;AAC/B,UAAM,SAAS,CAAC;AAChB,YAAQ;AACR,UAAM,KAAK,OAAO,MAAM,EAAE,OAAO,CAAC;AAAA,EACpC;AACF;AAEO,SAAS,UAAU,OAAgB,OAAoB,CAAC,GAAW;AACxE,MAAI,OAAO,UAAU,SAAU,QAAO;AAEtC,QAAM,SAAS,KAAK,UAAU,QAAQ,OAAO,SAAS;AACtD,SAAO,KAAK,UAAU,OAAO,MAAM,SAAS,IAAI,CAAC;AACnD;AAOA,eAAsB,YACpB,MACA,KAC4C;AAC5C,MAAI,WAAW,GAAG,KAAK,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC/D,UAAMA,IAAG,UAAU,KAAK,QAAQ,EAAE;AAClC,WAAO,CAAC,UAAmB,KAAK,OAAO,MAAM,EAAE,QAAQ,KAAK,CAAC;AAAA,EAC/D;AACA,SAAO,cAAc,IAAI;AAC3B;;;AEnEA,SAAS,qBAAqB;;;ACL9B,IAAM,UAAU,oBAAI,IAAY;AAGzB,SAAS,eAAe,OAAiC;AAC9D,MAAI,SAAS,MAAM,KAAK,MAAM,GAAI,SAAQ,IAAI,KAAK;AACrD;AAOA,IAAM,gBAAgB;AAEf,SAAS,cAAc,MAAsB;AAClD,MAAI,MAAM,KAAK,QAAQ,eAAe,cAAc;AACpD,QAAM,aAAa,IAAI,IAAI,OAAO;AAClC,QAAM,MAAM,QAAQ,IAAI,wBAAwB,KAAK;AACrD,MAAI,IAAK,YAAW,IAAI,GAAG;AAC3B,aAAW,UAAU,YAAY;AAC/B,eAAW,QAAQ,oBAAI,IAAI,CAAC,QAAQ,mBAAmB,MAAM,CAAC,CAAC,GAAG;AAChE,YAAM,IAAI,MAAM,IAAI,EAAE,KAAK,YAAY;AAAA,IACzC;AAAA,EACF;AACA,SAAO;AACT;;;ADIO,SAAS,aAAa,SAA6C;AACxE,iBAAe,QAAQ,MAAM;AAC7B,QAAM,QAAQ,IAAI,cAAc;AAAA,IAC9B,QAAQ,QAAQ;AAAA,IAChB,WAAW,QAAQ;AAAA,EACrB,CAAC;AAKD,QAAM,MAAM,EAAE,UAAU,KAAK;AAE7B,SAAO,IAAI,MAAM,OAAO;AAAA,IACtB,IAAI,QAAQ,MAAM,UAAU;AAC1B,YAAM,QAAQ,QAAQ,IAAI,QAAQ,MAAM,QAAQ;AAChD,UAAI,OAAO,UAAU,WAAY,QAAO;AACxC,cAAQ,MAAM;AAAA,QACZ,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAA+B,OAAO,iBAAiB,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjF,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAAqB,OAAO,OAAO,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QAC7D,KAAK;AAGH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,QAAQ,EAAE,GAAG,EAAE,QAAQ,UAAU,KAAK,EAAE,CAAC;AAAA,QAC1F,KAAK;AAGH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,QAAQ,EAAE,GAAG,EAAE,QAAQ,UAAU,KAAK,EAAE,CAAC;AAAA,QAC1F,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B;AACE,iBAAO,OAAO,UAAU,aAAa,MAAM,KAAK,MAAM,IAAI;AAAA,MAC9D;AAAA,IACF;AAAA,EACF,CAAC;AACH;;;AJvEO,SAAS,iBAA0B;AACxC,SAAO,IAAI,QAAQ,SAAS,EACzB,YAAY,0CAA0C,EACtD,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,OAAO,SAAuB;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,QAAQ;AACpC,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AACL;;;AMzBA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;AC2BzB,SAAS,mBACd,OACA,WAAkC,CAAC,GACb;AACtB,QAAM,MAA4B,CAAC;AAInC,MAAI,MAAM,OAAO,OAAW,KAAI,KAAK,MAAM;AAC3C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,cAAc,OAAW,KAAI,aAAa,MAAM;AAC1D,MAAI,MAAM,YAAY,OAAW,KAAI,WAAW,MAAM;AACtD,MAAI,MAAM,aAAa,OAAW,KAAI,YAAY,MAAM;AACxD,MAAI,MAAM,gBAAgB,OAAW,KAAI,eAAe,MAAM;AAC9D,MAAI,MAAM,WAAY,KAAI,eAAe;AACzC,MAAI,MAAM,gBAAiB,KAAI,oBAAoB;AACnD,MAAI,SAAS,QAAS,KAAI,UAAU,SAAS;AAC7C,SAAO;AACT;AAMO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,MAAI,OAAO,mBAAoB,KAAI,uBAAuB;AAC1D,SAAO;AACT;AAOO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,OAAO,WAAY,KAAI,cAAc,OAAO;AAChD,MAAI,OAAO,YAAa,KAAI,eAAe;AAC3C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AAIjB,QAAM,MAAgC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACpF,MAAI,aAAa,OAAW,KAAI,WAAW;AAC3C,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,6BACd,KACA,WACA,OACA,UACyB;AAIzB,QAAM,MAAwC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC5F,MAAI,UAAU,SAAS,EAAG,KAAI,YAAY;AAC1C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,mBACd,KACA,QACA,OACA,UACe;AACf,SAAO,EAAE,KAAK,QAAQ,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC/D;AAgBO,SAAS,iBAAiB,OAAe,QAAsB,CAAC,GAAgB;AAErF,MAAI,OAAO,UAAU,YAAY,MAAM,KAAK,MAAM,IAAI;AACpD,UAAM,IAAI,MAAM,iCAAiC;AAAA,EACnD;AACA,QAAM,MAAmB,EAAE,GAAG,MAAM;AACpC,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,SAAS,QAAW;AAC5B,QAAI,CAAC,OAAO,cAAc,MAAM,IAAI,KAAK,MAAM,OAAO,GAAG;AACvD,YAAM,IAAI,MAAM,wCAAwC,MAAM,IAAI,EAAE;AAAA,IACtE;AACA,QAAI,OAAO,MAAM;AAAA,EACnB;AACA,SAAO;AACT;AAWA,IAAM,uBAAuB,oBAAI,IAAI,CAAC,WAAW,OAAO,YAAY,WAAW,CAAC;AAGhF,IAAM,oBAA4C;AAAA,EAChD,SAAS;AAAA,EACT,YAAY;AAAA,EACZ,qBAAqB;AACvB;AAQO,SAAS,gBAAgB,MAAyB,CAAC,GAA2B;AACnF,QAAM,MAA8B,uBAAO,OAAO,IAAI;AACtD,aAAW,SAAS,KAAK;AACvB,UAAM,KAAK,MAAM,QAAQ,GAAG;AAC5B,QAAI,MAAM,EAAG,OAAM,IAAI,MAAM,mCAAmC,KAAK,EAAE;AACvE,UAAM,MAAM,MAAM,MAAM,GAAG,EAAE;AAC7B,QAAI,qBAAqB,IAAI,GAAG,GAAG;AACjC,YAAM,IAAI,MAAM,yBAAyB,GAAG,GAAG;AAAA,IACjD;AACA,QAAI,OAAO,OAAO,mBAAmB,GAAG,GAAG;AACzC,YAAM,IAAI,MAAM,yBAAyB,GAAG,UAAU,kBAAkB,GAAG,CAAC,EAAE;AAAA,IAChF;AACA,QAAI,OAAO,OAAO,KAAK,GAAG,GAAG;AAC3B,YAAM,IAAI,MAAM,YAAY,GAAG,wBAAwB;AAAA,IACzD;AACA,QAAI,GAAG,IAAI,MAAM,MAAM,KAAK,CAAC;AAAA,EAC/B;AACA,SAAO,EAAE,GAAG,IAAI;AAClB;AAQO,SAAS,iBAAiB,KAAa,QAAsB,CAAC,GAAgB;AACnF,MAAI,OAAO,QAAQ,YAAY,IAAI,KAAK,MAAM,IAAI;AAChD,UAAM,IAAI,MAAM,+BAA+B;AAAA,EACjD;AACA,QAAM,MAAmB,EAAE,IAAI;AAC/B,MAAI,MAAM,QAAS,KAAI,UAAU,MAAM;AACvC,MAAI,MAAM,eAAe,OAAW,KAAI,aAAa,MAAM;AAC3D,MAAI,MAAM,mBAAoB,KAAI,sBAAsB,MAAM;AAC9D,QAAM,SAAS,gBAAgB,MAAM,KAAK;AAC1C,MAAI,OAAO,KAAK,MAAM,EAAE,SAAS,EAAG,KAAI,SAAS;AACjD,SAAO;AACT;;;ACzNA,SAAS,YAAYC,WAAU;AAC/B,SAAS,cAAc;AA0BhB,SAAS,uBAAuB,KAAuB;AAC5D,SAAO,IACJ,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,QAAQ,sDAAsD,EACrE,OAAO,WAAW,wCAAwC,EAC1D;AAAA,IACC,IAAI,OAAO,kBAAkB,YAAY,EAAE,QAAQ,CAAC,cAAc,eAAe,SAAS,CAAC;AAAA,EAC7F,EACC,OAAO,oBAAoB,gDAAgD,EAC3E;AAAA,IACC,IAAI,OAAO,mBAAmB,kBAAkB,EAAE,QAAQ,CAAC,WAAW,UAAU,QAAQ,CAAC;AAAA,EAC3F,EACC,OAAO,kBAAkB,4CAA4C,aAAa,EAClF,OAAO,qBAAqB,kDAAkD,aAAa,EAC3F,OAAO,yBAAyB,mDAAmD,EACnF;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,sBAAsB,0CAA0C,EACvE,OAAO,wBAAwB,8CAA8C,EAC7E,OAAO,kBAAkB,qBAAqB,EAC9C,OAAO,uBAAuB,0BAA0B;AAC7D;AAQO,SAAS,wBAAwB,OAAwB;AAG9D,SAAO,aAAa,KAAK,KAAK,KAAK,OAAO,cAAc,OAAO,KAAK,CAAC;AACvE;AAEO,SAAS,cAAc,OAAuB;AACnD,QAAM,IAAI,OAAO,SAAS,OAAO,EAAE;AACnC,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,OAAM,IAAI,MAAM,0BAA0B,KAAK,EAAE;AAC1E,SAAO;AACT;AAMA,eAAsB,aACpB,OAC6C;AAC7C,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,GAAI,QAAO;AAC1C,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,SAAO,gBAAgB,MAAM,SAAS;AACxC;AAMA,eAAsB,YAAY,OAAgD;AAChF,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,QAAM,SAAS,gBAAgB,MAAM,QAAQ;AAC7C,MAAI,CAAC,UAAU,OAAO,KAAK,MAAM,EAAE,WAAW,GAAG;AAC/C,UAAM,IAAI,MAAM,sEAAsE;AAAA,EACxF;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,MAAc,OAAuC;AAC5E,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,SAAS,KAAK;AACZ,UAAM,IAAI,MAAM,GAAG,KAAK,yBAAqB,IAAc,OAAO,EAAE;AAAA,EACtE;AACA,MAAI,CAAC,UAAU,OAAO,WAAW,YAAY,MAAM,QAAQ,MAAM,GAAG;AAClE,UAAM,IAAI,MAAM,GAAG,KAAK,wBAAwB;AAAA,EAClD;AACA,QAAM,MAA8B,CAAC;AACrC,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,MAAiC,GAAG;AACtE,QAAI,OAAO,MAAM,SAAU,OAAM,IAAI,MAAM,GAAG,KAAK,IAAI,CAAC,0BAA0B;AAClF,QAAI,CAAC,IAAI;AAAA,EACX;AACA,SAAO;AACT;;;AF5GO,SAAS,aAAsB;AACpC,QAAM,MAAM,IAAIC,SAAQ,KAAK,EAC1B,YAAY,sDAAsD,EAClE,SAAS,SAAS,2CAA2C,EAC7D,eAAe,yBAAyB,2CAA2C,EACnF,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAmB;AAChD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AGvCA,SAAS,WAAAC,gBAAe;AAUjB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAAE,YAAY,4BAA4B;AAExE,MACG,QAAQ,WAAW,EACnB,YAAY,sBAAsB,WAAW,CAAC,EAAE,EAChD,OAAO,OAAO,QAAgB;AAC7B,QAAI,CAAC,OAAO,IAAI,KAAK,MAAM,GAAI,OAAM,IAAI,MAAM,kBAAkB;AACjE,UAAM,qBAAqB,EAAE,QAAQ,IAAI,KAAK,EAAE,CAAC;AACjD,YAAQ,OAAO,MAAM,oBAAoB,WAAW,CAAC;AAAA,CAAI;AAAA,EAC3D,CAAC;AAEH,MACG,QAAQ,MAAM,EACd,YAAY,+CAA+C,EAC3D,OAAO,YAAY;AAClB,UAAM,YAAY,MAAM,oBAAoB;AAC5C,UAAM,MAAM,QAAQ,IAAI;AACxB,QAAI,OAAO,IAAI,KAAK,MAAM,IAAI;AAC5B,cAAQ,OAAO;AAAA,QACb,4CAA4C,WAAW,IAAI,KAAK,CAAC,CAAC;AAAA;AAAA,MACpE;AAAA,IACF;AACA,QAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,IAAI;AACtD,cAAQ,OAAO;AAAA,QACb,eAAe,WAAW,CAAC,MAAM,WAAW,UAAU,OAAO,KAAK,CAAC,CAAC;AAAA;AAAA,MACtE;AAAA,IACF;AACA,QAAI,CAAC,OAAO,CAAC,UAAU,QAAQ;AAC7B,cAAQ,OAAO,MAAM,+DAA+D;AAAA,IACtF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,OAAO,EACf,YAAY,gCAAgC,EAC5C,OAAO,YAAY;AAClB,UAAM,UAAU,MAAM,qBAAqB;AAC3C,YAAQ,OAAO;AAAA,MACb,UAAU,WAAW,WAAW,CAAC;AAAA,IAAO;AAAA,IAC1C;AAAA,EACF,CAAC;AAEH,SAAO;AACT;;;ACtDA,SAAS,WAAAC,gBAAe;;;ACQxB;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAIA,IAAM,aAAa;AAAA,EACxB,IAAI;AAAA,EACJ,SAAS;AAAA,EACT,OAAO;AAAA,EACP,YAAY;AAAA,EACZ,MAAM;AAAA,EACN,iBAAiB;AAAA,EACjB,WAAW;AAAA,EACX,aAAa;AAAA,EACb,SAAS;AAAA,EACT,YAAY;AACd;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,qBAAsB,QAAO,WAAW;AAC3D,MAAI,eAAe,eAAgB,QAAO,WAAW;AACrD,MAAI,eAAe,YAAa,QAAO,WAAW;AAClD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,MAAI,eAAe,SAAU,QAAO,WAAW;AAC/C,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,SAAO,WAAW;AACpB;AAMO,SAAS,YAAY,KAAsB;AAChD,SAAO,cAAc,WAAW,GAAG,CAAC;AACtC;AAEA,SAAS,WAAW,KAAsB;AACxC,MAAI,eAAe,UAAU;AAC3B,UAAM,QAAQ,CAAC,IAAI,OAAO;AAC1B,QAAI,IAAI,OAAQ,OAAM,KAAK,SAAS,IAAI,MAAM,GAAG;AACjD,WAAO,MAAM,KAAK,GAAG;AAAA,EACvB;AACA,MAAI,eAAe,MAAO,QAAO,IAAI;AACrC,SAAO,OAAO,GAAG;AACnB;;;ADlDA,SAAS,QAAQ,OAAe,WAAqB,CAAC,GAAa;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAiBO,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B;AAAA,IACC;AAAA,EAGF,EACC,SAAS,SAAS,yDAAyD,EAC3E;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,EACF,EACC,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,QAAI,CAAC,WAAW,GAAG,KAAK,IAAI,KAAK,MAAM,IAAI;AACzC,UAAI,MAAM,wCAAwC;AAAA,QAChD,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AACA,QAAI;AACF,sBAAgB,KAAK,KAAK;AAAA,IAC5B,SAAS,KAAK;AACZ,UAAI,MAAM,UAAW,IAAc,OAAO,IAAI;AAAA,QAC5C,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,QAAsB;AAAA,MAC1B,SAAS,KAAK;AAAA,MACd,YAAY,KAAK;AAAA,MACjB,oBAAoB,KAAK;AAAA,MACzB,OAAO,KAAK;AAAA,IACd;AACA,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,SAAS,MAAM,OAAO,KAAK,iBAAiB,QAAQ,KAAK,CAAC;AAChE,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AElGA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,iBAA0B;AACxC,QAAM,MAAM,IAAIC,SAAQ,SAAS,EAC9B,YAAY,mEAAmE,EAC/E,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAuB;AACpD,QAAI,CAAC,KAAK,OAAQ,OAAM,IAAI,MAAM,sBAAsB;AACxD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,MAAM,YAAY,KAAK,MAAM;AAC5C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,mBAAmB,QAAQ,QAAQ,MAAM,EAAE,QAAQ,CAAC;AACvE,YAAM,SAAS,MAAM,OAAO,OAAO,UAAU;AAC7C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC1CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,+BAA+B,EAC3C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,0BAA0B,kDAAkD;AAAA,EACzF,EACC,UAAU,IAAIA,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,oBAAoB,KAAK;AAAA,QAC3B;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC/CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,kBAA2B;AACzC,QAAM,MAAM,IAAIC,SAAQ,UAAU,EAC/B,YAAY,0CAA0C,EACtD,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAwB;AACrD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACzCA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,0BAAmC;AACjD,QAAM,MAAM,IAAIC,SAAQ,mBAAmB,EACxC,YAAY,gDAAgD,EAC5D,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,IACA,CAAC;AAAA,EACH;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAgC;AAE7D,UAAM,YAAY,KAAK,YAAY,CAAC;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,6BAA6B,QAAQ,WAAW,MAAM,EAAE,QAAQ,CAAC;AACpF,YAAM,SAAS,MAAM,OAAO,iBAAiB,UAAU;AACvD,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,aAAa,OAAe,UAA8B;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;;;AChDA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AA+BzB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,MAAM,QAAQ,EACd,YAAY,iFAAiF,EAC7F,SAAS,cAAc,gEAAgE,EACvF,UAAU,IAAIC,QAAO,mBAAmB,iCAAiC,EAAE,QAAQ,CAAC,QAAQ,CAAC,CAAC,EAC9F,OAAO,eAAe,8DAA8D,EACpF,OAAO,eAAe,wDAAwD,EAC9E;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,OAAiB,SAAoB;AACrD,UAAM,QAAQ,MAAM,KAAK,GAAG;AAE5B,QAAI;AACJ,QAAI,KAAK,SAAS,QAAW;AAC3B,UAAI,CAAC,wBAAwB,KAAK,IAAI,GAAG;AACvC,YAAI,MAAM,+CAA+C,KAAK,IAAI,IAAI;AAAA,UACpE,UAAU,WAAW;AAAA,UACrB,MAAM;AAAA,QACR,CAAC;AAAA,MACH;AACA,aAAO,OAAO,KAAK,IAAI;AAAA,IACzB;AACA,QAAI,CAAC,WAAW,KAAK,KAAK,MAAM,KAAK,MAAM,IAAI;AAC7C,UAAI,MAAM,0CAA0C;AAAA,QAClD,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,QAAsB,EAAE,QAAQ,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,KAAK,IAAI,KAAK;AAClF,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,QAAQ,MAAM,YAAY,MAAM,KAAK;AAE3C,qBAAiB,KAAK,cAAc,KAAK,GAAG;AAC1C,YAAM,SAAS,MAAM,OAAO,KAAK,iBAAiB,GAAG,KAAK,CAAC;AAC3D,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AChFA,SAAS,WAAAC,iBAAe;;;ACmBxB,SAAS,YAAYC,WAAU;AAC/B,SAAS,WAAAC,gBAAe;AACxB,SAAS,WAAAC,UAAS,QAAAC,OAAM,eAAe;AACvC,SAAS,qBAAqB;AA0BvB,IAAM,UAAoC;AAAA;AAAA,EAE/C;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA;AAAA,EAGA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA,EAAE,IAAI,QAAQ,OAAO,QAAQ,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA,EAE9F;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AACF;AAEO,SAAS,WAAW,IAAuC;AAChE,SAAO,QAAQ,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE;AACxC;AAMO,SAAS,kBAA0B;AACxC,QAAM,OAAOD,SAAQ,cAAc,YAAY,GAAG,CAAC;AAGnD,SAAO,QAAQ,MAAM,MAAM,WAAW,kBAAkB,UAAU,gBAAgB;AACpF;AAaO,SAAS,cAAc,QAAuB,OAAuB,CAAC,GAAkB;AAC7F,QAAM,OAAO,OAAO,SAAS,SAAU,KAAK,QAAQD,SAAQ,IAAM,KAAK,OAAO,QAAQ,IAAI;AAC1F,SAAO,EAAE,QAAQ,aAAaE,MAAK,MAAM,OAAO,IAAI,EAAE;AACxD;AAgBA,eAAsB,aACpB,QACA,OAAuB,CAAC,GACA;AACxB,QAAM,SAAS,KAAK,aAAa,gBAAgB;AACjD,QAAM,SAAS,MAAM,WAAW,OAAO,WAAW;AAElD,MAAI,UAAU,CAAC,KAAK,OAAO;AACzB,WAAO,EAAE,QAAQ,QAAQ,iBAAiB;AAAA,EAC5C;AAEA,MAAI,OAAO,OAAO,SAAS,OAAO;AAChC,QAAI,OAAQ,OAAMH,IAAG,GAAG,OAAO,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AAC5E,UAAM,cAAc,QAAQ,OAAO,WAAW;AAAA,EAChD,OAAO;AACL,UAAM,YAAY,MAAMA,IAAG,SAASG,MAAK,QAAQ,UAAU,GAAG,MAAM;AACpE,UAAMH,IAAG,MAAME,SAAQ,OAAO,WAAW,GAAG,EAAE,WAAW,KAAK,CAAC;AAC/D,UAAMF,IAAG,UAAU,OAAO,aAAa,SAAS;AAAA,EAClD;AAEA,SAAO,EAAE,QAAQ,QAAQ,SAAS,gBAAgB,YAAY;AAChE;AAEA,eAAe,WAAW,MAAgC;AACxD,MAAI;AACF,UAAMA,IAAG,OAAO,IAAI;AACpB,WAAO;AAAA,EACT,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEA,eAAe,cAAc,QAAgB,aAAoC;AAE/E,QAAMA,IAAG,GAAG,QAAQ,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AACnE;;;AD3LO,SAAS,eAAwB;AACtC,QAAM,MAAM,IAAII,UAAQ,OAAO,EAAE,YAAY,gDAAgD;AAE7F,MACG,QAAQ,OAAO,EACf,YAAY,qEAAqE,EACjF;AAAA,IACC;AAAA,IACA,sCAAsC,QAAQ,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,KAAK,IAAI,CAAC;AAAA,IACzEC;AAAA,IACA,CAAC;AAAA,EACH,EACC,OAAO,aAAa,qCAAqC,EACzD,OAAO,eAAe,gCAAgC,EACtD,OAAO,OAAO,SAA0B;AACvC,UAAM,WAAW,iBAAiB,IAAI;AACtC,QAAI,SAAS,WAAW,GAAG;AACzB,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,eAAW,UAAU,UAAU;AAC7B,YAAM,SAAS,cAAc,MAAM;AACnC,YAAM,SAAS,MAAM,aAAa,QAAQ,EAAE,OAAO,KAAK,MAAM,CAAC;AAC/D,cAAQ,OAAO,MAAM,GAAG,OAAO,KAAK,KAAK,OAAO,MAAM,WAAM,OAAO,WAAW;AAAA,CAAI;AAAA,IACpF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,QAAQ,EAAE,QAAQ,MAAM,CAAC,EACjC,YAAY,0CAA0C,EACtD,OAAO,MAAM;AACZ,eAAW,KAAK,SAAS;AACvB,YAAM,SAAS,cAAc,CAAC;AAC9B,cAAQ,OAAO,MAAM,GAAG,EAAE,GAAG,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,OAAO,EAAE,CAAC,IAAI,OAAO,WAAW;AAAA,CAAI;AAAA,IACzF;AAAA,EACF,CAAC;AAEH,SAAO;AACT;AAEA,SAASA,SAAQ,OAAe,UAA8B;AAC5D,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAEA,SAAS,iBAAiB,MAAwC;AAChE,MAAI,KAAK,IAAK,QAAO,CAAC,GAAG,OAAO;AAChC,QAAM,MAAM,KAAK,UAAU,CAAC;AAC5B,SAAO,IAAI,IAAI,CAAC,OAAO;AACrB,UAAM,SAAS,WAAW,EAAE;AAC5B,QAAI,CAAC,OAAQ,OAAM,IAAI,MAAM,mBAAmB,EAAE,EAAE;AACpD,WAAO;AAAA,EACT,CAAC;AACH;;;AEjEA,SAAS,WAAAC,WAAS,UAAAC,eAAc;AAezB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,UAAQ,MAAM,EAC3B,YAAY,kCAAkC,EAC9C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,uBAAuB,sBAAsB,EAAE,QAAQ,CAAC,SAAS,QAAQ,KAAK,CAAC;AAAA,EAC5F,EACC,OAAO,kBAAkB,4CAA4C;AAExE,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,YAAY,KAAK;AAAA,UACjB,aAAa,KAAK;AAAA,QACpB;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ApBvBA,SAAS,iBAAyB;AAGhC,QAAM,OAAOC,SAAQC,eAAc,YAAY,GAAG,CAAC;AACnD,QAAM,UAAUC,SAAQ,MAAM,MAAM,cAAc;AAClD,QAAM,MAAM,KAAK,MAAM,aAAa,SAAS,MAAM,CAAC;AACpD,SAAO,IAAI;AACb;AAEO,SAAS,eAAwB;AACtC,QAAM,UAAU,IAAIC,UAAQ;AAC5B,UACG,KAAK,gBAAgB,EACrB;AAAA,IACC;AAAA,EAEF,EACC,QAAQ,eAAe,GAAG,iBAAiB,2BAA2B,EACtE,mBAAmB;AAEtB,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,gBAAgB,CAAC;AACpC,UAAQ,WAAW,wBAAwB,CAAC;AAC5C,UAAQ,WAAW,WAAW,CAAC;AAC/B,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,aAAa,CAAC;AAEjC,SAAO;AACT;AAEA,eAAe,KAAK,MAA+B;AACjD,QAAM,UAAU,aAAa;AAC7B,MAAI;AACF,UAAM,QAAQ,WAAW,IAAI;AAAA,EAC/B,SAAS,KAAK;AACZ,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B;AACF;AAIA,IAAM,YAAY,QAAQ,KAAK,CAAC,IAAIF,eAAc,YAAY,GAAG,IAAI;AACrE,IAAI,aAAa,QAAQ,KAAK,CAAC,MAAM,WAAW;AAC9C,OAAK,QAAQ,IAAI,EAAE,MAAM,CAAC,QAAiB;AACzC,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B,CAAC;AACH;","names":["Command","dirname","resolve","fileURLToPath","fs","fs","Command","Option","fs","Command","Option","Command","Command","Command","Command","Command","Command","Command","Option","Command","Option","Command","Option","Command","Option","Command","Command","Command","Option","Command","Option","Command","fs","homedir","dirname","join","Command","collect","Command","Option","Command","Option","dirname","fileURLToPath","resolve","Command"]}
1
+ {"version":3,"sources":["../src/cli.ts","../src/commands/account.ts","../src/lib/config.ts","../src/lib/output.ts","../src/lib/stdin.ts","../src/lib/sdk.ts","../src/lib/redact.ts","../src/commands/ask.ts","../src/lib/buildRequest.ts","../src/lib/options.ts","../src/commands/auth.ts","../src/commands/data.ts","../src/lib/errors.ts","../src/commands/extract.ts","../src/commands/html.ts","../src/commands/selected.ts","../src/commands/selectedMultiple.ts","../src/commands/serp.ts","../src/commands/serpLocations.ts","../src/commands/setupSkill.ts","../src/lib/skill.ts","../src/commands/text.ts"],"sourcesContent":["/**\n * `webscraping-ai` / `wsai` CLI entry point.\n *\n * Wires every subcommand into a single commander root. Errors thrown from\n * subcommand actions land in the top-level `.catch`, which maps SDK\n * exception classes to stable exit codes (see `lib/errors.ts`).\n */\n\nimport { Command } from 'commander';\nimport { readFileSync } from 'node:fs';\nimport { dirname, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nimport { accountCommand } from './commands/account.js';\nimport { askCommand } from './commands/ask.js';\nimport { authCommand } from './commands/auth.js';\nimport { dataCommand } from './commands/data.js';\nimport { extractCommand } from './commands/extract.js';\nimport { htmlCommand } from './commands/html.js';\nimport { selectedCommand } from './commands/selected.js';\nimport { selectedMultipleCommand } from './commands/selectedMultiple.js';\nimport { serpCommand } from './commands/serp.js';\nimport { serpLocationsCommand } from './commands/serpLocations.js';\nimport { setupCommand } from './commands/setupSkill.js';\nimport { textCommand } from './commands/text.js';\nimport { exitCodeFor, formatError } from './lib/errors.js';\n\nfunction packageVersion(): string {\n // Read package.json at runtime so we don't need a code-gen step. The dist\n // bundle sits at `dist/cli.js`, so package.json is one directory up.\n const here = dirname(fileURLToPath(import.meta.url));\n const pkgPath = resolve(here, '..', 'package.json');\n const raw = JSON.parse(readFileSync(pkgPath, 'utf8')) as { version: string };\n return raw.version;\n}\n\nexport function buildProgram(): Command {\n const program = new Command();\n program\n .name('webscraping-ai')\n .description(\n 'Official CLI for the WebScraping.AI API. Fetch pages, search Google, get structured site data, ask AI, extract fields.\\n' +\n 'Set $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first.',\n )\n .version(packageVersion(), '-v, --version', 'output the version number')\n .showHelpAfterError();\n\n program.addCommand(htmlCommand());\n program.addCommand(textCommand());\n program.addCommand(selectedCommand());\n program.addCommand(selectedMultipleCommand());\n program.addCommand(askCommand());\n program.addCommand(extractCommand());\n program.addCommand(serpCommand());\n program.addCommand(serpLocationsCommand());\n program.addCommand(dataCommand());\n program.addCommand(accountCommand());\n program.addCommand(authCommand());\n program.addCommand(setupCommand());\n\n return program;\n}\n\nasync function main(argv: string[]): Promise<void> {\n const program = buildProgram();\n try {\n await program.parseAsync(argv);\n } catch (err) {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n }\n}\n\n// Only execute when run as a script. When imported (e.g. from tests),\n// `buildProgram` is the public surface and `main` stays dormant.\nconst entryPath = process.argv[1] ? fileURLToPath(import.meta.url) : '';\nif (entryPath && process.argv[1] === entryPath) {\n main(process.argv).catch((err: unknown) => {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n });\n}\n","import { Command } from 'commander';\n\nimport { resolveApiKey } from '../lib/config.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface AccountFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nexport function accountCommand(): Command {\n return new Command('account')\n .description('Show account quota and usage information')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .action(async (opts: AccountFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const result = await client.account();\n await emit(result, opts);\n });\n}\n","/**\n * API key resolution and persisted config.\n *\n * Resolution order (first non-empty wins):\n * 1. `--api-key` flag (handled by the caller, passed in)\n * 2. `WEBSCRAPING_AI_API_KEY` env var\n * 3. `~/.config/webscraping-ai/config.json` (or `$XDG_CONFIG_HOME/...`)\n *\n * The config file is plain JSON with `{ \"apiKey\": \"...\" }`. We chmod it to\n * 0600 so it's not world-readable.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join } from 'node:path';\n\nconst CONFIG_FILENAME = 'config.json';\n\nexport function configDir(): string {\n const xdg = process.env.XDG_CONFIG_HOME;\n if (xdg && xdg.trim() !== '') return join(xdg, 'webscraping-ai');\n return join(homedir(), '.config', 'webscraping-ai');\n}\n\nexport function configPath(): string {\n return join(configDir(), CONFIG_FILENAME);\n}\n\nexport interface PersistedConfig {\n apiKey?: string;\n}\n\nexport async function readPersistedConfig(): Promise<PersistedConfig> {\n try {\n const text = await fs.readFile(configPath(), 'utf8');\n const parsed = JSON.parse(text) as unknown;\n if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {\n return parsed as PersistedConfig;\n }\n return {};\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return {};\n throw err;\n }\n}\n\nexport async function writePersistedConfig(config: PersistedConfig): Promise<void> {\n const path = configPath();\n await fs.mkdir(dirname(path), { recursive: true });\n await fs.writeFile(path, JSON.stringify(config, null, 2) + '\\n', { mode: 0o600 });\n // `mode` above only applies when the file is freshly created; an existing\n // file keeps its old (possibly world-readable) perms. Enforce 0600\n // unconditionally — chmod on a just-written file is cheap and idempotent.\n // No-op on Windows, but harmless.\n await fs.chmod(path, 0o600);\n}\n\nexport async function clearPersistedConfig(): Promise<boolean> {\n try {\n await fs.unlink(configPath());\n return true;\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return false;\n throw err;\n }\n}\n\nexport interface ResolveApiKeyOptions {\n /** `--api-key` flag value, if the user passed one. */\n flag?: string;\n}\n\nexport async function resolveApiKey(opts: ResolveApiKeyOptions = {}): Promise<string> {\n if (opts.flag && opts.flag.trim() !== '') return opts.flag.trim();\n\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') return env.trim();\n\n const persisted = await readPersistedConfig();\n if (persisted.apiKey && persisted.apiKey.trim() !== '') return persisted.apiKey.trim();\n\n throw new Error(\n 'No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`.',\n );\n}\n\nexport function maskApiKey(key: string): string {\n if (key.length <= 8) return '*'.repeat(key.length);\n return `${key.slice(0, 4)}${'*'.repeat(key.length - 8)}${key.slice(-4)}`;\n}\n","/**\n * Output handling for command results.\n *\n * All command handlers funnel through `emit()`. The result type drives format:\n * - strings → written verbatim\n * - objects/arrays → JSON.stringified (pretty by default; `--pretty` is the\n * default for terminals, single-line for pipes/redirects)\n *\n * `--output FILE` writes to disk instead of stdout. Commands use\n * `openEmitter()`: in stdin batch mode (`-`) it truncates the file up front\n * (so a first-item failure or empty stdin never leaves stale content) and\n * appends every result, so each item survives instead of overwriting the last.\n */\n\nimport { promises as fs } from 'node:fs';\n\nimport { urlIsStdin } from './stdin.js';\n\nexport interface EmitOptions {\n output?: string;\n pretty?: boolean;\n}\n\nexport interface EmitWriteOptions {\n /** Append to `--output` instead of truncating it (batch items after the first). */\n append?: boolean;\n}\n\nexport async function emit(\n value: unknown,\n opts: EmitOptions = {},\n write: EmitWriteOptions = {},\n): Promise<void> {\n const text = stringify(value, opts);\n\n if (opts.output && opts.output.trim() !== '') {\n const data = text + (text.endsWith('\\n') ? '' : '\\n');\n if (write.append) await fs.appendFile(opts.output, data);\n else await fs.writeFile(opts.output, data);\n return;\n }\n\n process.stdout.write(text);\n if (!text.endsWith('\\n')) process.stdout.write('\\n');\n}\n\n/**\n * Returns an `emit` bound to `opts` for a (possibly batched) loop: the first\n * call truncates `--output`, later calls append.\n */\nexport function createEmitter(opts: EmitOptions = {}): (value: unknown) => Promise<void> {\n let first = true;\n return async (value: unknown) => {\n const append = !first;\n first = false;\n await emit(value, opts, { append });\n };\n}\n\nexport function stringify(value: unknown, opts: EmitOptions = {}): string {\n if (typeof value === 'string') return value;\n\n const pretty = opts.pretty ?? process.stdout.isTTY ?? false;\n return JSON.stringify(value, null, pretty ? 2 : 0);\n}\n\n/**\n * Emitter for a command whose positional arg is `arg`. When `arg` is `-`\n * (stdin batch) and `--output` is set, the file is truncated now, before any\n * request, and every result is appended. Otherwise same as `createEmitter()`.\n */\nexport async function openEmitter(\n opts: EmitOptions,\n arg: string,\n): Promise<(value: unknown) => Promise<void>> {\n if (urlIsStdin(arg) && opts.output && opts.output.trim() !== '') {\n await fs.writeFile(opts.output, '');\n return (value: unknown) => emit(value, opts, { append: true });\n }\n return createEmitter(opts);\n}\n","/**\n * Stdin batching helpers.\n *\n * When a URL argument is `-`, the command reads one URL per line from stdin\n * and runs the operation for each. Empty lines and lines starting with `#`\n * are skipped so users can comment URL lists.\n *\n * `serp` reads queries instead (`queryIterator`): only blank lines are\n * skipped there, because `#hashtag` is a legitimate search query.\n */\n\nimport readline from 'node:readline';\n\nexport function urlIsStdin(url: string): boolean {\n return url === '-';\n}\n\nexport async function* readUrlsFromStdin(): AsyncIterable<string> {\n const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });\n for await (const raw of rl) {\n const line = raw.trim();\n if (line === '' || line.startsWith('#')) continue;\n yield line;\n }\n}\n\n/**\n * Read one search query per line. Blank / whitespace-only lines are skipped;\n * every other line is yielded verbatim (no `#` comment handling, no trim).\n */\nexport async function* readQueriesFromStdin(\n input: NodeJS.ReadableStream = process.stdin,\n): AsyncIterable<string> {\n const rl = readline.createInterface({ input, crlfDelay: Infinity });\n for await (const line of rl) {\n if (line.trim() === '') continue;\n yield line;\n }\n}\n\n/**\n * Iterate URLs from either a single CLI arg or from stdin (if the arg is `-`).\n */\nexport async function* urlIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readUrlsFromStdin();\n return;\n }\n yield arg;\n}\n\n/**\n * Iterate `serp` queries from a single CLI arg or from stdin (if it is `-`).\n */\nexport async function* queryIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readQueriesFromStdin();\n return;\n }\n yield arg;\n}\n","/**\n * SDK adapter.\n *\n * Wraps the published `webscraping-ai` Node SDK. The CLI sends a\n * `from_cli=true` analytics flag on every endpoint except `account` (mirrors\n * `from_n8n` and `from_mcp_server` used by the n8n node and MCP server). On\n * `serp` and `data` it rides in the SDK's `params` pass-through, since those\n * methods drop unknown top-level keys. `account()` takes no options and\n * `serpLocations()` only `q`/`limit`, so neither is tagged. The SDK's typed\n * options don't accept arbitrary keys, so we go through a single cast point\n * here rather than scattering casts across each command.\n */\n\nimport { WebScrapingAI } from 'webscraping-ai';\n\nimport { registerSecret } from './redact.js';\nimport type {\n DataOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nexport interface CreateClientOptions {\n apiKey: string;\n /** Per-request timeout in ms. Maps to the SDK's `timeoutMs`. */\n requestTimeoutMs?: number;\n}\n\n/**\n * Returns a `WebScrapingAI` instance with the CLI's analytics tag baked into\n * every endpoint method. The methods preserve the SDK's public signatures.\n */\nexport function createClient(options: CreateClientOptions): WebScrapingAI {\n registerSecret(options.apiKey);\n const inner = new WebScrapingAI({\n apiKey: options.apiKey,\n timeoutMs: options.requestTimeoutMs,\n });\n\n // Wrap each endpoint method so it injects the CLI analytics tag. We splice\n // the tag in at this seam rather than at every callsite. The cast is local;\n // the SDK passes unknown query params through to the wire.\n const tag = { from_cli: true } as unknown as Record<string, never>;\n\n return new Proxy(inner, {\n get(target, prop, receiver) {\n const value = Reflect.get(target, prop, receiver);\n if (typeof value !== 'function') return value;\n switch (prop) {\n case 'html':\n return (o: HtmlOptions) => target.html({ ...o, ...tag });\n case 'text':\n return (o: TextOptions) => target.text({ ...o, ...tag });\n case 'selected':\n return (o: SelectedOptions) => target.selected({ ...o, ...tag });\n case 'selectedMultiple':\n return (o: SelectedMultipleOptions) => target.selectedMultiple({ ...o, ...tag });\n case 'question':\n return (o: QuestionOptions) => target.question({ ...o, ...tag });\n case 'fields':\n return (o: FieldsOptions) => target.fields({ ...o, ...tag });\n case 'serp':\n // `serp()` drops unknown top-level keys but sends its `params`\n // pass-through as-is, so the tag goes there (CLI tag wins).\n return (o: SerpOptions) => target.serp({ ...o, params: { ...o.params, from_cli: true } });\n case 'data':\n // `data()` drops unknown top-level keys but sends its `params`\n // pass-through as-is, so the tag goes there (CLI tag wins).\n return (o: DataOptions) => target.data({ ...o, params: { ...o.params, from_cli: true } });\n case 'serpLocations':\n // `serpLocations()` sends only `q` and `limit` (no `params`\n // pass-through), so it isn't tagged — same accepted limitation as `account`.\n return value.bind(target);\n case 'account':\n // `account` takes no options; the SDK builds the request itself, so\n // there's no seam here for the analytics tag — accepted limitation.\n return value.bind(target);\n default:\n return typeof value === 'function' ? value.bind(target) : value;\n }\n },\n });\n}\n","/**\n * Secret redaction for anything the CLI prints to stderr.\n *\n * API errors can echo the request URL (e.g. an HTML error page that includes\n * `?api_key=...`), so every error message goes through `redactSecrets()`\n * before it is printed. The key in use is registered by `createClient()`.\n */\n\nconst secrets = new Set<string>();\n\n/** Remember a secret so later error output never prints it. */\nexport function registerSecret(value: string | undefined): void {\n if (value && value.trim() !== '') secrets.add(value);\n}\n\n/** Forget registered secrets (tests only). */\nexport function clearSecrets(): void {\n secrets.clear();\n}\n\nconst API_KEY_PARAM = /(api_key=)[^&\\s\"'<>]*/gi;\n\nexport function redactSecrets(text: string): string {\n let out = text.replace(API_KEY_PARAM, '$1[REDACTED]');\n const candidates = new Set(secrets);\n const env = process.env.WEBSCRAPING_AI_API_KEY?.trim();\n if (env) candidates.add(env);\n for (const secret of candidates) {\n for (const form of new Set([secret, encodeURIComponent(secret)])) {\n out = out.split(form).join('[REDACTED]');\n }\n }\n return out;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildQuestionOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface AskFlags extends CommonRawFlags {\n question?: string;\n}\n\nexport function askCommand(): Command {\n const cmd = new Command('ask')\n .description('Ask an AI question about a page (wraps /ai/question)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption('-q, --question <text>', 'the question / instructions for the model')\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: AskFlags) => {\n if (!opts.question) throw new Error('--question is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });\n const result = await client.question(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Pure builder that maps `--flag` values into SDK option objects.\n *\n * Extracted as a separate function so unit tests can pin the mapping without\n * spinning up commander or making HTTP calls. Each command handler reads its\n * own commander options, awaits `parseHeaders` / `parseFields`, then calls\n * the matching `build*` function here.\n */\n\nimport type {\n CommonRequestOptions,\n DataOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpLocationsOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nimport type { CommonRawFlags } from './options.js';\n\nexport interface ResolvedCommonOptions {\n headers?: Record<string, string>;\n}\n\nexport function buildCommonOptions(\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions = {},\n): CommonRequestOptions {\n const out: CommonRequestOptions = {};\n // `flags.js` is `false` when `--no-js`, `true` when `--js`, `undefined`\n // when neither passed. Pass through only when explicit, so the API default\n // applies otherwise.\n if (flags.js !== undefined) out.js = flags.js;\n if (flags.proxy !== undefined) out.proxy = flags.proxy;\n if (flags.country !== undefined) out.country = flags.country;\n if (flags.device !== undefined) out.device = flags.device;\n if (flags.timeout !== undefined) out.timeout = flags.timeout;\n if (flags.jsTimeout !== undefined) out.js_timeout = flags.jsTimeout;\n if (flags.waitFor !== undefined) out.wait_for = flags.waitFor;\n if (flags.jsScript !== undefined) out.js_script = flags.jsScript;\n if (flags.customProxy !== undefined) out.custom_proxy = flags.customProxy;\n if (flags.errorOn404) out.error_on_404 = true;\n if (flags.errorOnRedirect) out.error_on_redirect = true;\n if (resolved.headers) out.headers = resolved.headers;\n return out;\n}\n\nexport interface HtmlExtras {\n returnScriptResult?: boolean;\n}\n\nexport function buildHtmlOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: HtmlExtras = {},\n): HtmlOptions {\n const out: HtmlOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n if (extras.returnScriptResult) out.return_script_result = true;\n return out;\n}\n\nexport interface TextExtras {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function buildTextOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: TextExtras = {},\n): TextOptions {\n const out: TextOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (extras.textFormat) out.text_format = extras.textFormat;\n if (extras.returnLinks) out.return_links = true;\n return out;\n}\n\nexport function buildSelectedOptions(\n url: string,\n selector: string | undefined,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedOptions {\n // `selector` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when provided. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selector !== undefined) out.selector = selector;\n if (flags.format) out.format = flags.format;\n return out as SelectedOptions;\n}\n\nexport function buildSelectedMultipleOptions(\n url: string,\n selectors: readonly string[],\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedMultipleOptions {\n // `selectors` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when non-empty. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedMultipleOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selectors.length > 0) out.selectors = selectors;\n return out as SelectedMultipleOptions;\n}\n\nexport function buildQuestionOptions(\n url: string,\n question: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): QuestionOptions {\n const out: QuestionOptions = { url, question, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n return out;\n}\n\nexport function buildFieldsOptions(\n url: string,\n fields: Record<string, string>,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): FieldsOptions {\n return { url, fields, ...buildCommonOptions(flags, resolved) };\n}\n\nexport interface SerpRawFlags {\n engine?: 'google';\n gl?: string;\n hl?: string;\n page?: number;\n pages?: number;\n location?: string;\n uule?: string;\n lat?: number;\n lon?: number;\n radius?: number;\n}\n\nfunction checkPositiveInteger(flag: string, value: number | undefined): void {\n if (value !== undefined && (!Number.isSafeInteger(value) || value < 1)) {\n throw new Error(`${flag} must be an integer >= 1, got: ${value}`);\n }\n}\n\nfunction checkFinite(flag: string, value: number | undefined): void {\n if (value !== undefined && !Number.isFinite(value)) {\n throw new Error(`${flag} must be a number, got: ${value}`);\n }\n}\n\n/**\n * `/serp` is query-shaped: it takes none of the common scrape options, so\n * this deliberately does not go through `buildCommonOptions`. `page`,\n * `pages` and `radius` must be integers >= 1 and `lat`/`lon` finite numbers\n * (checking client-side saves a round trip). Ranges (page 1-100, pages 1-5,\n * radius 1-1000) and which location options combine are left to the server,\n * which rejects them with a 400 (not billed). `location` and `uule` are sent\n * as given.\n */\nexport function buildSerpOptions(query: string, flags: SerpRawFlags = {}): SerpOptions {\n // Validate on the trimmed value but send the query as the user typed it.\n if (typeof query !== 'string' || query.trim() === '') {\n throw new Error('serp requires a non-empty query');\n }\n checkPositiveInteger('--page', flags.page);\n checkPositiveInteger('--pages', flags.pages);\n checkPositiveInteger('--radius', flags.radius);\n checkFinite('--lat', flags.lat);\n checkFinite('--lon', flags.lon);\n\n const out: SerpOptions = { q: query };\n if (flags.engine !== undefined) out.engine = flags.engine;\n if (flags.gl) out.gl = flags.gl;\n if (flags.hl) out.hl = flags.hl;\n if (flags.page !== undefined) out.page = flags.page;\n if (flags.pages !== undefined) out.pages = flags.pages;\n if (flags.location !== undefined) out.location = flags.location;\n if (flags.uule !== undefined) out.uule = flags.uule;\n if (flags.lat !== undefined) out.lat = flags.lat;\n if (flags.lon !== undefined) out.lon = flags.lon;\n if (flags.radius !== undefined) out.radius = flags.radius;\n return out;\n}\n\n/**\n * Options for `/serp/locations`. A blank query is omitted, so the server\n * returns the most populous locations. `limit` must be an integer >= 1 (the\n * server rejects values above 10 with a 400).\n */\nexport function buildSerpLocationsOptions(\n query: string | undefined,\n limit?: number,\n): SerpLocationsOptions {\n checkPositiveInteger('--limit', limit);\n const out: SerpLocationsOptions = {};\n if (query !== undefined && query.trim() !== '') out.q = query;\n if (limit !== undefined) out.limit = limit;\n return out;\n}\n\nexport interface DataRawFlags {\n country?: string;\n transcript?: boolean;\n transcriptLanguage?: string;\n /** Raw repeatable `--param key=value` strings. */\n param?: readonly string[];\n}\n\n/** Keys `--param` must never set: owned by the SDK/CLI. */\nconst RESERVED_DATA_PARAMS = new Set(['api_key', 'url', 'from_cli', '__proto__']);\n\n/** `--param` keys that have their own flag: reject and point at the flag. */\nconst NAMED_DATA_PARAMS: Record<string, string> = {\n country: '--country',\n transcript: '--transcript',\n transcript_language: '--transcript-language',\n};\n\n/**\n * Parse repeatable `--param key=value` flags into a plain object. Splits on\n * the first `=`, so values may contain `=`. Throws on a missing `=`, an empty\n * or repeated key, a reserved key (`api_key`, `url`, `from_cli`,\n * `__proto__`), or a key that has its own flag (`country` → use `--country`).\n */\nexport function parseDataParams(raw: readonly string[] = []): Record<string, string> {\n const out: Record<string, string> = Object.create(null) as Record<string, string>;\n for (const entry of raw) {\n const eq = entry.indexOf('=');\n if (eq <= 0) throw new Error(`--param must be key=value, got: ${entry}`);\n const key = entry.slice(0, eq);\n if (RESERVED_DATA_PARAMS.has(key)) {\n throw new Error(`--param must not set \"${key}\"`);\n }\n if (Object.hasOwn(NAMED_DATA_PARAMS, key)) {\n throw new Error(`--param must not set \"${key}\"; use ${NAMED_DATA_PARAMS[key]}`);\n }\n if (Object.hasOwn(out, key)) {\n throw new Error(`--param \"${key}\" given more than once`);\n }\n out[key] = entry.slice(eq + 1);\n }\n return { ...out };\n}\n\n/**\n * `/data` is URL-shaped but takes none of the common scrape options, so this\n * deliberately does not go through `buildCommonOptions`. The URL is passed\n * through unmodified: there is no client-side site check (sites are added\n * server-side; the API answers an unsupported URL with a free 400).\n */\nexport function buildDataOptions(url: string, flags: DataRawFlags = {}): DataOptions {\n if (typeof url !== 'string' || url.trim() === '') {\n throw new Error('data requires a non-empty url');\n }\n const out: DataOptions = { url };\n if (flags.country) out.country = flags.country;\n if (flags.transcript !== undefined) out.transcript = flags.transcript;\n if (flags.transcriptLanguage) out.transcript_language = flags.transcriptLanguage;\n const params = parseDataParams(flags.param);\n if (Object.keys(params).length > 0) out.params = params;\n return out;\n}\n","/**\n * Shared commander options and the parsers/expanders behind them.\n *\n * All scrape endpoints share the same nine \"common\" flags. We define them in\n * one place and apply via `addCommonScrapeOptions(cmd)` so the surface stays\n * consistent and any change lands everywhere at once.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { Option } from 'commander';\nimport type { Command } from 'commander';\n\nexport interface CommonRawFlags {\n apiKey?: string;\n output?: string;\n format?: 'text' | 'json';\n pretty?: boolean;\n js?: boolean;\n proxy?: 'datacenter' | 'residential' | 'stealth';\n country?: string;\n device?: 'desktop' | 'mobile' | 'tablet';\n timeout?: number;\n jsTimeout?: number;\n waitFor?: string;\n headers?: string;\n jsScript?: string;\n customProxy?: string;\n errorOn404?: boolean;\n errorOnRedirect?: boolean;\n}\n\n/**\n * Add the flags every scrape-endpoint command needs. Adding here keeps the\n * surface aligned across `html`, `text`, `selected`, etc.\n */\nexport function addCommonScrapeOptions(cmd: Command): Command {\n return cmd\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .option('--js', 'execute JS via a headless browser (default: enabled)')\n .option('--no-js', 'disable JS execution (faster, cheaper)')\n .addOption(\n new Option('--proxy <type>', 'proxy pool').choices(['datacenter', 'residential', 'stealth']),\n )\n .option('--country <code>', 'two-letter proxy country code, e.g. us, gb, de')\n .addOption(\n new Option('--device <type>', 'device emulation').choices(['desktop', 'mobile', 'tablet']),\n )\n .option('--timeout <ms>', 'page-retrieval timeout in ms (max 30000)', integerParser)\n .option('--js-timeout <ms>', 'JS rendering time in ms after load (max 20000)', integerParser)\n .option('--wait-for <selector>', 'CSS selector to wait for before returning content')\n .option(\n '--headers <json>',\n 'HTTP headers to forward to the target page; JSON object or @file.json',\n )\n .option('--js-script <code>', 'custom JS to evaluate on the target page')\n .option('--custom-proxy <url>', 'custom proxy URL: http://user:pass@host:port')\n .option('--error-on-404', 'return error on 404')\n .option('--error-on-redirect', 'return error on redirect');\n}\n\n/**\n * Strict positive-integer check for flags like `serp --page`: digits only,\n * no leading zero, so `1.5`, `2abc`, `0` and `-1` are all rejected (unlike\n * `parseInt`, which would read `2abc` as 2). Values beyond\n * `Number.MAX_SAFE_INTEGER` are rejected too.\n */\nexport function isPositiveIntegerString(value: string): boolean {\n // isSafeInteger: a huge digit string becomes e.g. 1e+21 on the wire, which\n // the server rejects with a 400 (not billed); checking here saves the round trip.\n return /^[1-9]\\d*$/.test(value) && Number.isSafeInteger(Number(value));\n}\n\n/**\n * Strict decimal-number check for flags like `serp --lat`/`--lon`: an\n * optional sign, digits and an optional fraction (`-33.86`, `151`, `30.`, `.5`).\n * Exponents, hex, `Infinity`, `NaN`, blanks and trailing junk are rejected\n * (unlike `Number('')`, which is 0). The SDK writes the value as a plain\n * decimal with at most 7 fractional digits.\n */\nexport function isDecimalString(value: string): boolean {\n return /^[+-]?(?:\\d+\\.?\\d*|\\.\\d+)$/.test(value) && Number.isFinite(Number(value));\n}\n\nexport function integerParser(value: string): number {\n const n = Number.parseInt(value, 10);\n if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);\n return n;\n}\n\n/**\n * Parse a `--headers` argument. Accepts a JSON object literal or `@path` to\n * load JSON from a file. Returns `undefined` if the input is empty.\n */\nexport async function parseHeaders(\n input: string | undefined,\n): Promise<Record<string, string> | undefined> {\n if (!input || input.trim() === '') return undefined;\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n return parseJsonObject(text, 'headers');\n}\n\n/**\n * Parse a `--fields` argument for `extract`. Accepts JSON object literal or\n * `@path` to a file.\n */\nexport async function parseFields(input: string): Promise<Record<string, string>> {\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n const parsed = parseJsonObject(text, 'fields');\n if (!parsed || Object.keys(parsed).length === 0) {\n throw new Error('fields must be a non-empty JSON object of { fieldName: description }');\n }\n return parsed;\n}\n\nfunction parseJsonObject(text: string, label: string): Record<string, string> {\n let parsed: unknown;\n try {\n parsed = JSON.parse(text);\n } catch (err) {\n throw new Error(`${label}: invalid JSON — ${(err as Error).message}`);\n }\n if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {\n throw new Error(`${label} must be a JSON object`);\n }\n const out: Record<string, string> = {};\n for (const [k, v] of Object.entries(parsed as Record<string, unknown>)) {\n if (typeof v !== 'string') throw new Error(`${label}.${k}: value must be a string`);\n out[k] = v;\n }\n return out;\n}\n","import { Command } from 'commander';\n\nimport {\n clearPersistedConfig,\n configPath,\n maskApiKey,\n readPersistedConfig,\n writePersistedConfig,\n} from '../lib/config.js';\n\nexport function authCommand(): Command {\n const cmd = new Command('auth').description('Manage your stored API key');\n\n cmd\n .command('set <key>')\n .description(`Save an API key to ${configPath()}`)\n .action(async (key: string) => {\n if (!key || key.trim() === '') throw new Error('API key is empty');\n await writePersistedConfig({ apiKey: key.trim() });\n process.stdout.write(`API key saved to ${configPath()}\\n`);\n });\n\n cmd\n .command('show')\n .description('Display the currently stored API key (masked)')\n .action(async () => {\n const persisted = await readPersistedConfig();\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') {\n process.stdout.write(\n `Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}\\n`,\n );\n }\n if (persisted.apiKey && persisted.apiKey.trim() !== '') {\n process.stdout.write(\n `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}\\n`,\n );\n }\n if (!env && !persisted.apiKey) {\n process.stdout.write('No API key configured. Run `webscraping-ai auth set <key>`.\\n');\n }\n });\n\n cmd\n .command('clear')\n .description('Remove the stored API key file')\n .action(async () => {\n const removed = await clearPersistedConfig();\n process.stdout.write(\n removed ? `Removed ${configPath()}\\n` : 'No stored API key to remove.\\n',\n );\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildDataOptions, parseDataParams } from '../lib/buildRequest.js';\nimport type { DataRawFlags } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIsStdin, urlIterator } from '../lib/stdin.js';\n\ninterface DataFlags extends DataRawFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nfunction collect(value: string, previous: string[] = []): string[] {\n return [...previous, value];\n}\n\n/**\n * `data <url>` — structured JSON for a page on a supported site (wraps /data).\n *\n * URL-shaped, but the server picks fetching/proxy/parsing per site, so the\n * common scrape flags (`--js`, `--proxy`, `--headers`, …) don't apply and are\n * deliberately not registered. The URL is never checked against a list of\n * sites: new ones are added server-side. An unsupported URL or page type\n * returns a 400 that is not charged (exit code 3); its message lists what is\n * supported. `-` reads one URL per line from stdin (blank and `#` lines\n * skipped, and the batch stops at the first failing item, like the other URL\n * commands). `--param key=value` passes site-specific parameters through.\n *\n * Usage errors (blank url, malformed/reserved `--param`) exit with code 2\n * before stdin is read or the API key is resolved.\n */\nexport function dataCommand(): Command {\n const cmd = new Command('data')\n .description(\n 'Structured JSON for a page on a supported site, e.g. YouTube, TikTok, X, LinkedIn, ' +\n 'Instagram, Reddit (wraps /data, priced per site; more sites are added server-side, ' +\n 'an unsupported URL or page type gets a 400, not charged, listing what is supported)',\n )\n .argument('<url>', 'page URL (or `-` to read URLs from stdin, one per line)')\n .option(\n '--country <code>',\n 'two-letter country code of the proxy used to fetch the page (default: us)',\n )\n .option(\n '--transcript',\n 'YouTube videos only: also fetch the transcript into data.transcript (null without matching captions; a failed transcript fetch fails the request with a 500, not charged)',\n )\n .option(\n '--transcript-language <code>',\n 'caption language to pick, e.g. en or de (default: English, then the first track)',\n )\n .option(\n '--param <key=value>',\n 'extra site-specific query parameter sent as-is (repeatable)',\n collect,\n )\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (url: string, opts: DataFlags) => {\n if (!urlIsStdin(url) && url.trim() === '') {\n cmd.error('error: data requires a non-empty url', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.emptyUrl',\n });\n }\n try {\n parseDataParams(opts.param);\n } catch (err) {\n cmd.error(`error: ${(err as Error).message}`, {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.invalidParam',\n });\n }\n\n const flags: DataRawFlags = {\n country: opts.country,\n transcript: opts.transcript,\n transcriptLanguage: opts.transcriptLanguage,\n param: opts.param,\n };\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const result = await client.data(buildDataOptions(target, flags));\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Exit-code mapping for SDK errors.\n *\n * The SDK throws typed subclasses of `WebScrapingAIError`. Map each to a\n * stable exit code so shell scripts can branch on the failure mode without\n * parsing stderr.\n */\n\nimport {\n APIConnectionError,\n APIError,\n APITimeoutError,\n AuthenticationError,\n BadRequestError,\n GatewayTimeoutError,\n PaymentRequiredError,\n RateLimitError,\n ServerError,\n WebScrapingAIError,\n} from 'webscraping-ai';\n\nimport { redactSecrets } from './redact.js';\n\nexport const EXIT_CODES = {\n ok: 0,\n generic: 1,\n usage: 2,\n badRequest: 3,\n auth: 4,\n paymentRequired: 5,\n rateLimit: 6,\n serverError: 7,\n timeout: 8,\n connection: 9,\n} as const;\n\nexport function exitCodeFor(err: unknown): number {\n if (err instanceof BadRequestError) return EXIT_CODES.badRequest;\n if (err instanceof AuthenticationError) return EXIT_CODES.auth;\n if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;\n if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;\n if (err instanceof ServerError) return EXIT_CODES.serverError;\n if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APITimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APIConnectionError) return EXIT_CODES.connection;\n if (err instanceof APIError) return EXIT_CODES.serverError;\n if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;\n return EXIT_CODES.generic;\n}\n\n/**\n * Human-readable one-liner for stderr. Always redacted: the API key and any\n * `api_key=...` pattern are replaced, since error bodies can echo the URL.\n */\nexport function formatError(err: unknown): string {\n return redactSecrets(rawMessage(err));\n}\n\nfunction rawMessage(err: unknown): string {\n if (err instanceof APIError) {\n const parts = [err.message];\n if (err.status) parts.push(`(HTTP ${err.status})`);\n return parts.join(' ');\n }\n if (err instanceof Error) return err.message;\n return String(err);\n}\n","import { Command } from 'commander';\n\nimport { buildFieldsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseFields, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface ExtractFlags extends CommonRawFlags {\n fields?: string;\n}\n\nexport function extractCommand(): Command {\n const cmd = new Command('extract')\n .description('Extract structured fields from a page using AI (wraps /ai/fields)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption(\n '-f, --fields <json>',\n 'fields to extract — JSON object {name: description} or @file.json',\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: ExtractFlags) => {\n if (!opts.fields) throw new Error('--fields is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const fields = await parseFields(opts.fields);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });\n const result = await client.fields(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildHtmlOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface HtmlFlags extends CommonRawFlags {\n returnScriptResult?: boolean;\n}\n\nexport function htmlCommand(): Command {\n const cmd = new Command('html')\n .description('Fetch the full HTML of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--return-script-result', 'return the result of --js-script instead of HTML'),\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: HtmlFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildHtmlOptions(\n target,\n opts,\n { headers },\n {\n returnScriptResult: opts.returnScriptResult,\n },\n );\n const result = await client.html(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSelectedOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedFlags extends CommonRawFlags {\n selector?: string;\n}\n\nexport function selectedCommand(): Command {\n const cmd = new Command('selected')\n .description('Fetch HTML of one CSS-selected page area')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector for the target element(s); omit for whole-page HTML',\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });\n const result = await client.selected(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildSelectedMultipleOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedMultipleFlags extends CommonRawFlags {\n selector?: string[];\n}\n\nexport function selectedMultipleCommand(): Command {\n const cmd = new Command('selected-multiple')\n .description('Fetch HTML of multiple CSS-selected page areas')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector — pass multiple times for multiple selectors',\n collectMulti,\n [] as string[],\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedMultipleFlags) => {\n // `--selector` is optional: omitting it returns whole-page HTML.\n const selectors = opts.selector ?? [];\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });\n const result = await client.selectedMultiple(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n\nfunction collectMulti(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSerpOptions } from '../lib/buildRequest.js';\nimport type { SerpRawFlags } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { isDecimalString, isPositiveIntegerString } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { queryIterator, urlIsStdin } from '../lib/stdin.js';\n\ntype NumericFlag = 'page' | 'pages' | 'lat' | 'lon' | 'radius';\n\ninterface SerpFlags extends Omit<SerpRawFlags, NumericFlag> {\n /** Raw numeric flag strings; validated strictly in the action. */\n page?: string;\n pages?: string;\n lat?: string;\n lon?: string;\n radius?: string;\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\n/**\n * `serp <query...>` — parsed search engine results (wraps /serp).\n *\n * Query-shaped, not URL-shaped, so none of the common scrape flags\n * (`--js`, `--proxy`, `--country`, …) apply and they're deliberately not\n * registered here. Multiple words are joined with spaces, so quoting the\n * query is optional. A lone `-` reads one query per line from stdin; only\n * blank lines are skipped (a leading `#` is part of the query, not a comment).\n *\n * Every search runs from one fixed location (`--location`, `--lat` +\n * `--lon` or `--uule`, otherwise the capital of the `--gl` country). Which of\n * those combine, and their ranges, are checked server-side (400, not billed).\n * Negative coordinates work as `--lat -33.86`: an option's value is taken\n * even when it starts with `-`.\n *\n * Usage errors (bad `--page`/`--pages`/`--radius`/`--lat`/`--lon`, empty\n * query) exit with code 2 before stdin is read or the API key is resolved.\n */\nexport function serpCommand(): Command {\n const cmd = new Command('serp')\n .alias('search')\n .description('Search Google and print parsed results as JSON (wraps /serp)')\n .argument('<query...>', 'search query (or `-` to read queries from stdin, one per line)')\n .addOption(new Option('--engine <name>', 'search engine (default: google)').choices(['google']))\n .option(\n '--gl <code>',\n 'two-letter country code for the search (default: the country of --location, --lat/--lon or --uule, otherwise us)',\n )\n .option('--hl <code>', 'two-letter language code for the results (default: en)')\n .option(\n '--page <n>',\n 'results page number, integer >= 1, 10 results per page (default: 1; server rejects > 100 with a 400)',\n )\n .option(\n '--pages <n>',\n 'consecutive results pages to fetch in one call starting at --page, 1-5, returned as one de-duplicated list (default: 1; billed per page returned)',\n )\n .option(\n '--location <name>',\n 'where the search runs from: a name like \"Austin, TX\" or \"Austin,Texas,United States\", or an id from `serp-locations`',\n )\n .option('--uule <value>', 'Google-encoded location (w+CAIQICI… or a+…)')\n .option(\n '--lat <deg>',\n 'latitude of the search origin in decimal degrees (-90 to 90; needs --lon)',\n )\n .option(\n '--lon <deg>',\n 'longitude of the search origin in decimal degrees (-180 to 180; needs --lat)',\n )\n .option('--radius <m>', 'radius in meters around --location or --lat/--lon, integer 1-1000')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (words: string[], opts: SerpFlags) => {\n const query = words.join(' ');\n\n const usage = (message: string, code: string): never =>\n cmd.error(`error: ${message}`, { exitCode: EXIT_CODES.usage, code });\n const integer = (name: NumericFlag, value: string | undefined): number | undefined => {\n if (value === undefined) return undefined;\n if (!isPositiveIntegerString(value)) {\n usage(\n `--${name} must be an integer >= 1, got: ${value}`,\n `wsai.invalid${capitalize(name)}`,\n );\n }\n return Number(value);\n };\n const decimal = (name: NumericFlag, value: string | undefined): number | undefined => {\n if (value === undefined) return undefined;\n if (!isDecimalString(value)) {\n usage(\n `--${name} must be a decimal number, e.g. -33.86, got: ${value}`,\n `wsai.invalid${capitalize(name)}`,\n );\n }\n return Number(value);\n };\n\n const page = integer('page', opts.page);\n const pages = integer('pages', opts.pages);\n const radius = integer('radius', opts.radius);\n const lat = decimal('lat', opts.lat);\n const lon = decimal('lon', opts.lon);\n\n if (!urlIsStdin(query) && query.trim() === '') {\n cmd.error('error: serp requires a non-empty query', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.emptyQuery',\n });\n }\n\n const flags: SerpRawFlags = {\n engine: opts.engine,\n gl: opts.gl,\n hl: opts.hl,\n page,\n pages,\n location: opts.location,\n uule: opts.uule,\n lat,\n lon,\n radius,\n };\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const write = await openEmitter(opts, query);\n\n for await (const q of queryIterator(query)) {\n const result = await client.serp(buildSerpOptions(q, flags));\n await write(result);\n }\n });\n\n return cmd;\n}\n\nfunction capitalize(name: string): string {\n return name.charAt(0).toUpperCase() + name.slice(1);\n}\n","import { Command } from 'commander';\n\nimport { buildSerpLocationsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { isPositiveIntegerString } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface SerpLocationsFlags {\n /** Raw `--limit` string; validated strictly in the action. */\n limit?: string;\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\n/**\n * `serp-locations [query...]` — look up names for `serp --location` (wraps\n * /serp/locations). Prints a JSON array of up to 10 locations, most populous\n * first; pass a result's `canonical_name` (or `id`) as `--location`. Matches\n * English names. Multiple words are joined with spaces; without a query the\n * most populous locations are returned. The endpoint is free and needs no\n * API key, but the CLI sends the configured one like every other command.\n *\n * There is no stdin mode, so a lone `-` is a usage error. That and a bad\n * `--limit` exit with code 2 before the API key is resolved.\n */\nexport function serpLocationsCommand(): Command {\n const cmd = new Command('serp-locations')\n .description(\n 'Look up names for `serp --location` and print them as a JSON array (wraps /serp/locations, free)',\n )\n .argument('[query...]', 'part of a location name, e.g. austin (default: most populous)')\n .option('--limit <n>', 'maximum number of locations, integer 1-10 (default: 10)')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (words: string[], opts: SerpLocationsFlags) => {\n let limit: number | undefined;\n if (opts.limit !== undefined) {\n if (!isPositiveIntegerString(opts.limit)) {\n cmd.error(`error: --limit must be an integer >= 1, got: ${opts.limit}`, {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.invalidLimit',\n });\n }\n limit = Number(opts.limit);\n }\n\n const query = words.join(' ');\n if (query === '-') {\n cmd.error('error: serp-locations has no stdin mode; pass the query as an argument', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.stdinUnsupported',\n });\n }\n\n const options = buildSerpLocationsOptions(query, limit);\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const result = await client.serpLocations(options);\n await emit(result, opts);\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { EDITORS, editorById, installSkill, resolveTarget } from '../lib/skill.js';\nimport type { EditorAdapter } from '../lib/skill.js';\n\ninterface SetupSkillFlags {\n editor?: string[];\n all?: boolean;\n force?: boolean;\n}\n\nexport function setupCommand(): Command {\n const cmd = new Command('setup').description('Install the AI agent skill into your editor(s)');\n\n cmd\n .command('skill')\n .description('Install the WebScraping.AI skill into one or more AI coding editors')\n .option(\n '-e, --editor <id>',\n `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(', ')}`,\n collect,\n [] as string[],\n )\n .option('-a, --all', 'install into every supported editor')\n .option('-f, --force', 'overwrite existing skill files')\n .action(async (opts: SetupSkillFlags) => {\n const selected = resolveSelection(opts);\n if (selected.length === 0) {\n throw new Error(\n 'no editor selected — pass --all or --editor <id> (e.g. --editor claude-code)',\n );\n }\n\n for (const editor of selected) {\n const target = resolveTarget(editor);\n const result = await installSkill(target, { force: opts.force });\n process.stdout.write(`${editor.label}: ${result.status} → ${target.destination}\\n`);\n }\n });\n\n cmd\n .command('list', { hidden: false })\n .description('List supported editors for `setup skill`')\n .action(() => {\n for (const e of EDITORS) {\n const target = resolveTarget(e);\n process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}\\n`);\n }\n });\n\n return cmd;\n}\n\nfunction collect(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n\nfunction resolveSelection(opts: SetupSkillFlags): EditorAdapter[] {\n if (opts.all) return [...EDITORS];\n const ids = opts.editor ?? [];\n return ids.map((id) => {\n const editor = editorById(id);\n if (!editor) throw new Error(`unknown editor: ${id}`);\n return editor;\n });\n}\n","/**\n * Multi-editor skill installer.\n *\n * The canonical skill lives in this package at\n * `plugins/webscraping-ai/skills/webscraping-ai/SKILL.md`. Each supported\n * editor reads skill content from a different location and may want either\n * a single Markdown file or a whole skill directory. The `editors` registry\n * captures those differences so `setup skill` is a small driver over it.\n *\n * `mode: 'dir'` copies the entire skill directory (Claude Code-shaped\n * editors that scan a folder per skill). `mode: 'file'` writes the SKILL.md\n * contents to a single file (editors that read flat instructions, like\n * GitHub Copilot's `.github/copilot-instructions.md`).\n *\n * `kind: 'user'` writes to `$HOME`; `kind: 'project'` writes to the current\n * working directory. We prefer user-scope where the editor supports it so\n * the skill becomes globally available without polluting individual repos.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nexport type EditorMode = 'dir' | 'file';\nexport type EditorKind = 'user' | 'project';\n\nexport interface EditorAdapter {\n /** Stable identifier used by `--editor <name>`. */\n id: string;\n /** Human-readable label printed in summaries. */\n label: string;\n mode: EditorMode;\n kind: EditorKind;\n /**\n * Where the skill ends up. For `mode: 'dir'`, the skill directory copies\n * to `{root}/{path}`. For `mode: 'file'`, the SKILL.md contents write to\n * `{root}/{path}` (i.e. `path` ends in a filename).\n *\n * `root` is `$HOME` (user) or CWD (project).\n */\n path: string;\n}\n\n/**\n * Registry of supported editors. Order matters: it's also the order the\n * `setup skill` summary lists them in.\n */\nexport const EDITORS: readonly EditorAdapter[] = [\n // Claude Code: user-scope skills live under ~/.claude/skills/<name>/\n {\n id: 'claude-code',\n label: 'Claude Code',\n mode: 'dir',\n kind: 'user',\n path: '.claude/skills/webscraping-ai',\n },\n // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill\n // dir, so this is the editor's \"install\" surface.\n {\n id: 'cursor',\n label: 'Cursor',\n mode: 'file',\n kind: 'project',\n path: '.cursor/rules/webscraping-ai.mdc',\n },\n // Windsurf: project-scope rules. Same shape as Cursor.\n {\n id: 'windsurf',\n label: 'Windsurf',\n mode: 'file',\n kind: 'project',\n path: '.windsurf/rules/webscraping-ai.md',\n },\n // Kiro: user-scope steering directory.\n { id: 'kiro', label: 'Kiro', mode: 'dir', kind: 'user', path: '.kiro/steering/webscraping-ai' },\n // OpenCode: project-scope skills (mirrors Claude shape).\n {\n id: 'opencode',\n label: 'OpenCode',\n mode: 'dir',\n kind: 'project',\n path: '.opencode/skills/webscraping-ai',\n },\n // Gemini CLI: user-scope extension dir with a GEMINI.md.\n {\n id: 'gemini',\n label: 'Gemini CLI',\n mode: 'file',\n kind: 'user',\n path: '.gemini/extensions/webscraping-ai/GEMINI.md',\n },\n // GitHub Copilot: project-scope custom instructions file. Single file.\n {\n id: 'copilot',\n label: 'GitHub Copilot',\n mode: 'file',\n kind: 'project',\n path: '.github/copilot-instructions.md',\n },\n // Augment: project-scope rules file.\n {\n id: 'augment',\n label: 'Augment',\n mode: 'file',\n kind: 'project',\n path: '.augment/rules/webscraping-ai.md',\n },\n // Factory: project-scope rules file.\n {\n id: 'factory',\n label: 'Factory',\n mode: 'file',\n kind: 'project',\n path: '.factory/rules/webscraping-ai.md',\n },\n] as const;\n\nexport function editorById(id: string): EditorAdapter | undefined {\n return EDITORS.find((e) => e.id === id);\n}\n\n/**\n * Where this package keeps the canonical skill. Resolved relative to the\n * compiled `dist/cli.js` so it works after `npm install -g`.\n */\nexport function bundledSkillDir(): string {\n const here = dirname(fileURLToPath(import.meta.url));\n // dist/ sits at the package root, so the plugins dir is two levels up\n // from any compiled file: dist/cli.js → package root → plugins/...\n return resolve(here, '..', 'plugins', 'webscraping-ai', 'skills', 'webscraping-ai');\n}\n\nexport interface InstallTarget {\n editor: EditorAdapter;\n /** Absolute destination path (file or directory depending on `mode`). */\n destination: string;\n}\n\nexport interface ResolveOptions {\n cwd?: string;\n home?: string;\n}\n\nexport function resolveTarget(editor: EditorAdapter, opts: ResolveOptions = {}): InstallTarget {\n const root = editor.kind === 'user' ? (opts.home ?? homedir()) : (opts.cwd ?? process.cwd());\n return { editor, destination: join(root, editor.path) };\n}\n\nexport interface InstallOptions {\n /** If true, overwrite existing files/dirs. */\n force?: boolean;\n /** Override skill source dir (for tests). */\n sourceDir?: string;\n}\n\nexport type InstallStatus = 'installed' | 'skipped-exists' | 'overwritten';\n\nexport interface InstallResult {\n target: InstallTarget;\n status: InstallStatus;\n}\n\nexport async function installSkill(\n target: InstallTarget,\n opts: InstallOptions = {},\n): Promise<InstallResult> {\n const source = opts.sourceDir ?? bundledSkillDir();\n const exists = await pathExists(target.destination);\n\n if (exists && !opts.force) {\n return { target, status: 'skipped-exists' };\n }\n\n if (target.editor.mode === 'dir') {\n if (exists) await fs.rm(target.destination, { recursive: true, force: true });\n await copyDirectory(source, target.destination);\n } else {\n const skillBody = await fs.readFile(join(source, 'SKILL.md'), 'utf8');\n await fs.mkdir(dirname(target.destination), { recursive: true });\n await fs.writeFile(target.destination, skillBody);\n }\n\n return { target, status: exists ? 'overwritten' : 'installed' };\n}\n\nasync function pathExists(path: string): Promise<boolean> {\n try {\n await fs.access(path);\n return true;\n } catch {\n return false;\n }\n}\n\nasync function copyDirectory(source: string, destination: string): Promise<void> {\n // `fs.cp` with `recursive` exists on Node 16.7+; we target Node 20.\n await fs.cp(source, destination, { recursive: true, force: true });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildTextOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { openEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface TextFlags extends CommonRawFlags {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function textCommand(): Command {\n const cmd = new Command('text')\n .description('Fetch the visible text of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--text-format <fmt>', 'text response format').choices(['plain', 'json', 'xml']),\n )\n .option('--return-links', 'include page links (text-format=json only)');\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: TextFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = await openEmitter(opts, url);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildTextOptions(\n target,\n opts,\n { headers },\n {\n textFormat: opts.textFormat,\n returnLinks: opts.returnLinks,\n },\n );\n const result = await client.text(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n"],"mappings":";;;AAQA,SAAS,WAAAA,iBAAe;AACxB,SAAS,oBAAoB;AAC7B,SAAS,WAAAC,UAAS,WAAAC,gBAAe;AACjC,SAAS,iBAAAC,sBAAqB;;;ACX9B,SAAS,eAAe;;;ACYxB,SAAS,YAAY,UAAU;AAC/B,SAAS,eAAe;AACxB,SAAS,SAAS,YAAY;AAE9B,IAAM,kBAAkB;AAEjB,SAAS,YAAoB;AAClC,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,gBAAgB;AAC/D,SAAO,KAAK,QAAQ,GAAG,WAAW,gBAAgB;AACpD;AAEO,SAAS,aAAqB;AACnC,SAAO,KAAK,UAAU,GAAG,eAAe;AAC1C;AAMA,eAAsB,sBAAgD;AACpE,MAAI;AACF,UAAM,OAAO,MAAM,GAAG,SAAS,WAAW,GAAG,MAAM;AACnD,UAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,QAAI,UAAU,OAAO,WAAW,YAAY,CAAC,MAAM,QAAQ,MAAM,GAAG;AAClE,aAAO;AAAA,IACT;AACA,WAAO,CAAC;AAAA,EACV,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO,CAAC;AAC9D,UAAM;AAAA,EACR;AACF;AAEA,eAAsB,qBAAqB,QAAwC;AACjF,QAAM,OAAO,WAAW;AACxB,QAAM,GAAG,MAAM,QAAQ,IAAI,GAAG,EAAE,WAAW,KAAK,CAAC;AACjD,QAAM,GAAG,UAAU,MAAM,KAAK,UAAU,QAAQ,MAAM,CAAC,IAAI,MAAM,EAAE,MAAM,IAAM,CAAC;AAKhF,QAAM,GAAG,MAAM,MAAM,GAAK;AAC5B;AAEA,eAAsB,uBAAyC;AAC7D,MAAI;AACF,UAAM,GAAG,OAAO,WAAW,CAAC;AAC5B,WAAO;AAAA,EACT,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO;AAC7D,UAAM;AAAA,EACR;AACF;AAOA,eAAsB,cAAc,OAA6B,CAAC,GAAoB;AACpF,MAAI,KAAK,QAAQ,KAAK,KAAK,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,KAAK;AAEhE,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,IAAI,KAAK;AAE9C,QAAM,YAAY,MAAM,oBAAoB;AAC5C,MAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,GAAI,QAAO,UAAU,OAAO,KAAK;AAErF,QAAM,IAAI;AAAA,IACR;AAAA,EACF;AACF;AAEO,SAAS,WAAW,KAAqB;AAC9C,MAAI,IAAI,UAAU,EAAG,QAAO,IAAI,OAAO,IAAI,MAAM;AACjD,SAAO,GAAG,IAAI,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,OAAO,IAAI,SAAS,CAAC,CAAC,GAAG,IAAI,MAAM,EAAE,CAAC;AACxE;;;AC3EA,SAAS,YAAYC,WAAU;;;ACH/B,OAAO,cAAc;AAEd,SAAS,WAAW,KAAsB;AAC/C,SAAO,QAAQ;AACjB;AAEA,gBAAuB,oBAA2C;AAChE,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,QAAQ,OAAO,WAAW,SAAS,CAAC;AACjF,mBAAiB,OAAO,IAAI;AAC1B,UAAM,OAAO,IAAI,KAAK;AACtB,QAAI,SAAS,MAAM,KAAK,WAAW,GAAG,EAAG;AACzC,UAAM;AAAA,EACR;AACF;AAMA,gBAAuB,qBACrB,QAA+B,QAAQ,OAChB;AACvB,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,WAAW,SAAS,CAAC;AAClE,mBAAiB,QAAQ,IAAI;AAC3B,QAAI,KAAK,KAAK,MAAM,GAAI;AACxB,UAAM;AAAA,EACR;AACF;AAKA,gBAAuB,YAAY,KAAoC;AACrE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,kBAAkB;AACzB;AAAA,EACF;AACA,QAAM;AACR;AAKA,gBAAuB,cAAc,KAAoC;AACvE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,qBAAqB;AAC5B;AAAA,EACF;AACA,QAAM;AACR;;;ADhCA,eAAsB,KACpB,OACA,OAAoB,CAAC,GACrB,QAA0B,CAAC,GACZ;AACf,QAAM,OAAO,UAAU,OAAO,IAAI;AAElC,MAAI,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC5C,UAAM,OAAO,QAAQ,KAAK,SAAS,IAAI,IAAI,KAAK;AAChD,QAAI,MAAM,OAAQ,OAAMC,IAAG,WAAW,KAAK,QAAQ,IAAI;AAAA,QAClD,OAAMA,IAAG,UAAU,KAAK,QAAQ,IAAI;AACzC;AAAA,EACF;AAEA,UAAQ,OAAO,MAAM,IAAI;AACzB,MAAI,CAAC,KAAK,SAAS,IAAI,EAAG,SAAQ,OAAO,MAAM,IAAI;AACrD;AAMO,SAAS,cAAc,OAAoB,CAAC,GAAsC;AACvF,MAAI,QAAQ;AACZ,SAAO,OAAO,UAAmB;AAC/B,UAAM,SAAS,CAAC;AAChB,YAAQ;AACR,UAAM,KAAK,OAAO,MAAM,EAAE,OAAO,CAAC;AAAA,EACpC;AACF;AAEO,SAAS,UAAU,OAAgB,OAAoB,CAAC,GAAW;AACxE,MAAI,OAAO,UAAU,SAAU,QAAO;AAEtC,QAAM,SAAS,KAAK,UAAU,QAAQ,OAAO,SAAS;AACtD,SAAO,KAAK,UAAU,OAAO,MAAM,SAAS,IAAI,CAAC;AACnD;AAOA,eAAsB,YACpB,MACA,KAC4C;AAC5C,MAAI,WAAW,GAAG,KAAK,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC/D,UAAMA,IAAG,UAAU,KAAK,QAAQ,EAAE;AAClC,WAAO,CAAC,UAAmB,KAAK,OAAO,MAAM,EAAE,QAAQ,KAAK,CAAC;AAAA,EAC/D;AACA,SAAO,cAAc,IAAI;AAC3B;;;AEnEA,SAAS,qBAAqB;;;ACL9B,IAAM,UAAU,oBAAI,IAAY;AAGzB,SAAS,eAAe,OAAiC;AAC9D,MAAI,SAAS,MAAM,KAAK,MAAM,GAAI,SAAQ,IAAI,KAAK;AACrD;AAOA,IAAM,gBAAgB;AAEf,SAAS,cAAc,MAAsB;AAClD,MAAI,MAAM,KAAK,QAAQ,eAAe,cAAc;AACpD,QAAM,aAAa,IAAI,IAAI,OAAO;AAClC,QAAM,MAAM,QAAQ,IAAI,wBAAwB,KAAK;AACrD,MAAI,IAAK,YAAW,IAAI,GAAG;AAC3B,aAAW,UAAU,YAAY;AAC/B,eAAW,QAAQ,oBAAI,IAAI,CAAC,QAAQ,mBAAmB,MAAM,CAAC,CAAC,GAAG;AAChE,YAAM,IAAI,MAAM,IAAI,EAAE,KAAK,YAAY;AAAA,IACzC;AAAA,EACF;AACA,SAAO;AACT;;;ADIO,SAAS,aAAa,SAA6C;AACxE,iBAAe,QAAQ,MAAM;AAC7B,QAAM,QAAQ,IAAI,cAAc;AAAA,IAC9B,QAAQ,QAAQ;AAAA,IAChB,WAAW,QAAQ;AAAA,EACrB,CAAC;AAKD,QAAM,MAAM,EAAE,UAAU,KAAK;AAE7B,SAAO,IAAI,MAAM,OAAO;AAAA,IACtB,IAAI,QAAQ,MAAM,UAAU;AAC1B,YAAM,QAAQ,QAAQ,IAAI,QAAQ,MAAM,QAAQ;AAChD,UAAI,OAAO,UAAU,WAAY,QAAO;AACxC,cAAQ,MAAM;AAAA,QACZ,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAA+B,OAAO,iBAAiB,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjF,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAAqB,OAAO,OAAO,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QAC7D,KAAK;AAGH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,QAAQ,EAAE,GAAG,EAAE,QAAQ,UAAU,KAAK,EAAE,CAAC;AAAA,QAC1F,KAAK;AAGH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,QAAQ,EAAE,GAAG,EAAE,QAAQ,UAAU,KAAK,EAAE,CAAC;AAAA,QAC1F,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B;AACE,iBAAO,OAAO,UAAU,aAAa,MAAM,KAAK,MAAM,IAAI;AAAA,MAC9D;AAAA,IACF;AAAA,EACF,CAAC;AACH;;;AJ3EO,SAAS,iBAA0B;AACxC,SAAO,IAAI,QAAQ,SAAS,EACzB,YAAY,0CAA0C,EACtD,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,OAAO,SAAuB;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,QAAQ;AACpC,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AACL;;;AMzBA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;AC4BzB,SAAS,mBACd,OACA,WAAkC,CAAC,GACb;AACtB,QAAM,MAA4B,CAAC;AAInC,MAAI,MAAM,OAAO,OAAW,KAAI,KAAK,MAAM;AAC3C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,cAAc,OAAW,KAAI,aAAa,MAAM;AAC1D,MAAI,MAAM,YAAY,OAAW,KAAI,WAAW,MAAM;AACtD,MAAI,MAAM,aAAa,OAAW,KAAI,YAAY,MAAM;AACxD,MAAI,MAAM,gBAAgB,OAAW,KAAI,eAAe,MAAM;AAC9D,MAAI,MAAM,WAAY,KAAI,eAAe;AACzC,MAAI,MAAM,gBAAiB,KAAI,oBAAoB;AACnD,MAAI,SAAS,QAAS,KAAI,UAAU,SAAS;AAC7C,SAAO;AACT;AAMO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,MAAI,OAAO,mBAAoB,KAAI,uBAAuB;AAC1D,SAAO;AACT;AAOO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,OAAO,WAAY,KAAI,cAAc,OAAO;AAChD,MAAI,OAAO,YAAa,KAAI,eAAe;AAC3C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AAIjB,QAAM,MAAgC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACpF,MAAI,aAAa,OAAW,KAAI,WAAW;AAC3C,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,6BACd,KACA,WACA,OACA,UACyB;AAIzB,QAAM,MAAwC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC5F,MAAI,UAAU,SAAS,EAAG,KAAI,YAAY;AAC1C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,mBACd,KACA,QACA,OACA,UACe;AACf,SAAO,EAAE,KAAK,QAAQ,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC/D;AAeA,SAAS,qBAAqB,MAAc,OAAiC;AAC3E,MAAI,UAAU,WAAc,CAAC,OAAO,cAAc,KAAK,KAAK,QAAQ,IAAI;AACtE,UAAM,IAAI,MAAM,GAAG,IAAI,kCAAkC,KAAK,EAAE;AAAA,EAClE;AACF;AAEA,SAAS,YAAY,MAAc,OAAiC;AAClE,MAAI,UAAU,UAAa,CAAC,OAAO,SAAS,KAAK,GAAG;AAClD,UAAM,IAAI,MAAM,GAAG,IAAI,2BAA2B,KAAK,EAAE;AAAA,EAC3D;AACF;AAWO,SAAS,iBAAiB,OAAe,QAAsB,CAAC,GAAgB;AAErF,MAAI,OAAO,UAAU,YAAY,MAAM,KAAK,MAAM,IAAI;AACpD,UAAM,IAAI,MAAM,iCAAiC;AAAA,EACnD;AACA,uBAAqB,UAAU,MAAM,IAAI;AACzC,uBAAqB,WAAW,MAAM,KAAK;AAC3C,uBAAqB,YAAY,MAAM,MAAM;AAC7C,cAAY,SAAS,MAAM,GAAG;AAC9B,cAAY,SAAS,MAAM,GAAG;AAE9B,QAAM,MAAmB,EAAE,GAAG,MAAM;AACpC,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,SAAS,OAAW,KAAI,OAAO,MAAM;AAC/C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,aAAa,OAAW,KAAI,WAAW,MAAM;AACvD,MAAI,MAAM,SAAS,OAAW,KAAI,OAAO,MAAM;AAC/C,MAAI,MAAM,QAAQ,OAAW,KAAI,MAAM,MAAM;AAC7C,MAAI,MAAM,QAAQ,OAAW,KAAI,MAAM,MAAM;AAC7C,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,SAAO;AACT;AAOO,SAAS,0BACd,OACA,OACsB;AACtB,uBAAqB,WAAW,KAAK;AACrC,QAAM,MAA4B,CAAC;AACnC,MAAI,UAAU,UAAa,MAAM,KAAK,MAAM,GAAI,KAAI,IAAI;AACxD,MAAI,UAAU,OAAW,KAAI,QAAQ;AACrC,SAAO;AACT;AAWA,IAAM,uBAAuB,oBAAI,IAAI,CAAC,WAAW,OAAO,YAAY,WAAW,CAAC;AAGhF,IAAM,oBAA4C;AAAA,EAChD,SAAS;AAAA,EACT,YAAY;AAAA,EACZ,qBAAqB;AACvB;AAQO,SAAS,gBAAgB,MAAyB,CAAC,GAA2B;AACnF,QAAM,MAA8B,uBAAO,OAAO,IAAI;AACtD,aAAW,SAAS,KAAK;AACvB,UAAM,KAAK,MAAM,QAAQ,GAAG;AAC5B,QAAI,MAAM,EAAG,OAAM,IAAI,MAAM,mCAAmC,KAAK,EAAE;AACvE,UAAM,MAAM,MAAM,MAAM,GAAG,EAAE;AAC7B,QAAI,qBAAqB,IAAI,GAAG,GAAG;AACjC,YAAM,IAAI,MAAM,yBAAyB,GAAG,GAAG;AAAA,IACjD;AACA,QAAI,OAAO,OAAO,mBAAmB,GAAG,GAAG;AACzC,YAAM,IAAI,MAAM,yBAAyB,GAAG,UAAU,kBAAkB,GAAG,CAAC,EAAE;AAAA,IAChF;AACA,QAAI,OAAO,OAAO,KAAK,GAAG,GAAG;AAC3B,YAAM,IAAI,MAAM,YAAY,GAAG,wBAAwB;AAAA,IACzD;AACA,QAAI,GAAG,IAAI,MAAM,MAAM,KAAK,CAAC;AAAA,EAC/B;AACA,SAAO,EAAE,GAAG,IAAI;AAClB;AAQO,SAAS,iBAAiB,KAAa,QAAsB,CAAC,GAAgB;AACnF,MAAI,OAAO,QAAQ,YAAY,IAAI,KAAK,MAAM,IAAI;AAChD,UAAM,IAAI,MAAM,+BAA+B;AAAA,EACjD;AACA,QAAM,MAAmB,EAAE,IAAI;AAC/B,MAAI,MAAM,QAAS,KAAI,UAAU,MAAM;AACvC,MAAI,MAAM,eAAe,OAAW,KAAI,aAAa,MAAM;AAC3D,MAAI,MAAM,mBAAoB,KAAI,sBAAsB,MAAM;AAC9D,QAAM,SAAS,gBAAgB,MAAM,KAAK;AAC1C,MAAI,OAAO,KAAK,MAAM,EAAE,SAAS,EAAG,KAAI,SAAS;AACjD,SAAO;AACT;;;ACrQA,SAAS,YAAYC,WAAU;AAC/B,SAAS,cAAc;AA0BhB,SAAS,uBAAuB,KAAuB;AAC5D,SAAO,IACJ,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,QAAQ,sDAAsD,EACrE,OAAO,WAAW,wCAAwC,EAC1D;AAAA,IACC,IAAI,OAAO,kBAAkB,YAAY,EAAE,QAAQ,CAAC,cAAc,eAAe,SAAS,CAAC;AAAA,EAC7F,EACC,OAAO,oBAAoB,gDAAgD,EAC3E;AAAA,IACC,IAAI,OAAO,mBAAmB,kBAAkB,EAAE,QAAQ,CAAC,WAAW,UAAU,QAAQ,CAAC;AAAA,EAC3F,EACC,OAAO,kBAAkB,4CAA4C,aAAa,EAClF,OAAO,qBAAqB,kDAAkD,aAAa,EAC3F,OAAO,yBAAyB,mDAAmD,EACnF;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,sBAAsB,0CAA0C,EACvE,OAAO,wBAAwB,8CAA8C,EAC7E,OAAO,kBAAkB,qBAAqB,EAC9C,OAAO,uBAAuB,0BAA0B;AAC7D;AAQO,SAAS,wBAAwB,OAAwB;AAG9D,SAAO,aAAa,KAAK,KAAK,KAAK,OAAO,cAAc,OAAO,KAAK,CAAC;AACvE;AASO,SAAS,gBAAgB,OAAwB;AACtD,SAAO,6BAA6B,KAAK,KAAK,KAAK,OAAO,SAAS,OAAO,KAAK,CAAC;AAClF;AAEO,SAAS,cAAc,OAAuB;AACnD,QAAM,IAAI,OAAO,SAAS,OAAO,EAAE;AACnC,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,OAAM,IAAI,MAAM,0BAA0B,KAAK,EAAE;AAC1E,SAAO;AACT;AAMA,eAAsB,aACpB,OAC6C;AAC7C,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,GAAI,QAAO;AAC1C,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,SAAO,gBAAgB,MAAM,SAAS;AACxC;AAMA,eAAsB,YAAY,OAAgD;AAChF,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,QAAM,SAAS,gBAAgB,MAAM,QAAQ;AAC7C,MAAI,CAAC,UAAU,OAAO,KAAK,MAAM,EAAE,WAAW,GAAG;AAC/C,UAAM,IAAI,MAAM,sEAAsE;AAAA,EACxF;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,MAAc,OAAuC;AAC5E,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,SAAS,KAAK;AACZ,UAAM,IAAI,MAAM,GAAG,KAAK,yBAAqB,IAAc,OAAO,EAAE;AAAA,EACtE;AACA,MAAI,CAAC,UAAU,OAAO,WAAW,YAAY,MAAM,QAAQ,MAAM,GAAG;AAClE,UAAM,IAAI,MAAM,GAAG,KAAK,wBAAwB;AAAA,EAClD;AACA,QAAM,MAA8B,CAAC;AACrC,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,MAAiC,GAAG;AACtE,QAAI,OAAO,MAAM,SAAU,OAAM,IAAI,MAAM,GAAG,KAAK,IAAI,CAAC,0BAA0B;AAClF,QAAI,CAAC,IAAI;AAAA,EACX;AACA,SAAO;AACT;;;AFvHO,SAAS,aAAsB;AACpC,QAAM,MAAM,IAAIC,SAAQ,KAAK,EAC1B,YAAY,sDAAsD,EAClE,SAAS,SAAS,2CAA2C,EAC7D,eAAe,yBAAyB,2CAA2C,EACnF,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAmB;AAChD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AGvCA,SAAS,WAAAC,gBAAe;AAUjB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAAE,YAAY,4BAA4B;AAExE,MACG,QAAQ,WAAW,EACnB,YAAY,sBAAsB,WAAW,CAAC,EAAE,EAChD,OAAO,OAAO,QAAgB;AAC7B,QAAI,CAAC,OAAO,IAAI,KAAK,MAAM,GAAI,OAAM,IAAI,MAAM,kBAAkB;AACjE,UAAM,qBAAqB,EAAE,QAAQ,IAAI,KAAK,EAAE,CAAC;AACjD,YAAQ,OAAO,MAAM,oBAAoB,WAAW,CAAC;AAAA,CAAI;AAAA,EAC3D,CAAC;AAEH,MACG,QAAQ,MAAM,EACd,YAAY,+CAA+C,EAC3D,OAAO,YAAY;AAClB,UAAM,YAAY,MAAM,oBAAoB;AAC5C,UAAM,MAAM,QAAQ,IAAI;AACxB,QAAI,OAAO,IAAI,KAAK,MAAM,IAAI;AAC5B,cAAQ,OAAO;AAAA,QACb,4CAA4C,WAAW,IAAI,KAAK,CAAC,CAAC;AAAA;AAAA,MACpE;AAAA,IACF;AACA,QAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,IAAI;AACtD,cAAQ,OAAO;AAAA,QACb,eAAe,WAAW,CAAC,MAAM,WAAW,UAAU,OAAO,KAAK,CAAC,CAAC;AAAA;AAAA,MACtE;AAAA,IACF;AACA,QAAI,CAAC,OAAO,CAAC,UAAU,QAAQ;AAC7B,cAAQ,OAAO,MAAM,+DAA+D;AAAA,IACtF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,OAAO,EACf,YAAY,gCAAgC,EAC5C,OAAO,YAAY;AAClB,UAAM,UAAU,MAAM,qBAAqB;AAC3C,YAAQ,OAAO;AAAA,MACb,UAAU,WAAW,WAAW,CAAC;AAAA,IAAO;AAAA,IAC1C;AAAA,EACF,CAAC;AAEH,SAAO;AACT;;;ACtDA,SAAS,WAAAC,gBAAe;;;ACQxB;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAIA,IAAM,aAAa;AAAA,EACxB,IAAI;AAAA,EACJ,SAAS;AAAA,EACT,OAAO;AAAA,EACP,YAAY;AAAA,EACZ,MAAM;AAAA,EACN,iBAAiB;AAAA,EACjB,WAAW;AAAA,EACX,aAAa;AAAA,EACb,SAAS;AAAA,EACT,YAAY;AACd;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,qBAAsB,QAAO,WAAW;AAC3D,MAAI,eAAe,eAAgB,QAAO,WAAW;AACrD,MAAI,eAAe,YAAa,QAAO,WAAW;AAClD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,MAAI,eAAe,SAAU,QAAO,WAAW;AAC/C,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,SAAO,WAAW;AACpB;AAMO,SAAS,YAAY,KAAsB;AAChD,SAAO,cAAc,WAAW,GAAG,CAAC;AACtC;AAEA,SAAS,WAAW,KAAsB;AACxC,MAAI,eAAe,UAAU;AAC3B,UAAM,QAAQ,CAAC,IAAI,OAAO;AAC1B,QAAI,IAAI,OAAQ,OAAM,KAAK,SAAS,IAAI,MAAM,GAAG;AACjD,WAAO,MAAM,KAAK,GAAG;AAAA,EACvB;AACA,MAAI,eAAe,MAAO,QAAO,IAAI;AACrC,SAAO,OAAO,GAAG;AACnB;;;ADlDA,SAAS,QAAQ,OAAe,WAAqB,CAAC,GAAa;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAiBO,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B;AAAA,IACC;AAAA,EAGF,EACC,SAAS,SAAS,yDAAyD,EAC3E;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,EACF,EACC,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,QAAI,CAAC,WAAW,GAAG,KAAK,IAAI,KAAK,MAAM,IAAI;AACzC,UAAI,MAAM,wCAAwC;AAAA,QAChD,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AACA,QAAI;AACF,sBAAgB,KAAK,KAAK;AAAA,IAC5B,SAAS,KAAK;AACZ,UAAI,MAAM,UAAW,IAAc,OAAO,IAAI;AAAA,QAC5C,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,QAAsB;AAAA,MAC1B,SAAS,KAAK;AAAA,MACd,YAAY,KAAK;AAAA,MACjB,oBAAoB,KAAK;AAAA,MACzB,OAAO,KAAK;AAAA,IACd;AACA,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,SAAS,MAAM,OAAO,KAAK,iBAAiB,QAAQ,KAAK,CAAC;AAChE,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AElGA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,iBAA0B;AACxC,QAAM,MAAM,IAAIC,SAAQ,SAAS,EAC9B,YAAY,mEAAmE,EAC/E,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAuB;AACpD,QAAI,CAAC,KAAK,OAAQ,OAAM,IAAI,MAAM,sBAAsB;AACxD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,MAAM,YAAY,KAAK,MAAM;AAC5C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,mBAAmB,QAAQ,QAAQ,MAAM,EAAE,QAAQ,CAAC;AACvE,YAAM,SAAS,MAAM,OAAO,OAAO,UAAU;AAC7C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC1CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,+BAA+B,EAC3C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,0BAA0B,kDAAkD;AAAA,EACzF,EACC,UAAU,IAAIA,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,oBAAoB,KAAK;AAAA,QAC3B;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC/CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,kBAA2B;AACzC,QAAM,MAAM,IAAIC,SAAQ,UAAU,EAC/B,YAAY,0CAA0C,EACtD,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAwB;AACrD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACzCA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,0BAAmC;AACjD,QAAM,MAAM,IAAIC,SAAQ,mBAAmB,EACxC,YAAY,gDAAgD,EAC5D,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,IACA,CAAC;AAAA,EACH;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAgC;AAE7D,UAAM,YAAY,KAAK,YAAY,CAAC;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,6BAA6B,QAAQ,WAAW,MAAM,EAAE,QAAQ,CAAC;AACpF,YAAM,SAAS,MAAM,OAAO,iBAAiB,UAAU;AACvD,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,aAAa,OAAe,UAA8B;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;;;AChDA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AA2CzB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,MAAM,QAAQ,EACd,YAAY,8DAA8D,EAC1E,SAAS,cAAc,gEAAgE,EACvF,UAAU,IAAIC,QAAO,mBAAmB,iCAAiC,EAAE,QAAQ,CAAC,QAAQ,CAAC,CAAC,EAC9F;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,eAAe,wDAAwD,EAC9E;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,kBAAkB,uDAA6C,EACtE;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,gBAAgB,mEAAmE,EAC1F,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,OAAiB,SAAoB;AACrD,UAAM,QAAQ,MAAM,KAAK,GAAG;AAE5B,UAAM,QAAQ,CAAC,SAAiB,SAC9B,IAAI,MAAM,UAAU,OAAO,IAAI,EAAE,UAAU,WAAW,OAAO,KAAK,CAAC;AACrE,UAAM,UAAU,CAAC,MAAmB,UAAkD;AACpF,UAAI,UAAU,OAAW,QAAO;AAChC,UAAI,CAAC,wBAAwB,KAAK,GAAG;AACnC;AAAA,UACE,KAAK,IAAI,kCAAkC,KAAK;AAAA,UAChD,eAAe,WAAW,IAAI,CAAC;AAAA,QACjC;AAAA,MACF;AACA,aAAO,OAAO,KAAK;AAAA,IACrB;AACA,UAAM,UAAU,CAAC,MAAmB,UAAkD;AACpF,UAAI,UAAU,OAAW,QAAO;AAChC,UAAI,CAAC,gBAAgB,KAAK,GAAG;AAC3B;AAAA,UACE,KAAK,IAAI,gDAAgD,KAAK;AAAA,UAC9D,eAAe,WAAW,IAAI,CAAC;AAAA,QACjC;AAAA,MACF;AACA,aAAO,OAAO,KAAK;AAAA,IACrB;AAEA,UAAM,OAAO,QAAQ,QAAQ,KAAK,IAAI;AACtC,UAAM,QAAQ,QAAQ,SAAS,KAAK,KAAK;AACzC,UAAM,SAAS,QAAQ,UAAU,KAAK,MAAM;AAC5C,UAAM,MAAM,QAAQ,OAAO,KAAK,GAAG;AACnC,UAAM,MAAM,QAAQ,OAAO,KAAK,GAAG;AAEnC,QAAI,CAAC,WAAW,KAAK,KAAK,MAAM,KAAK,MAAM,IAAI;AAC7C,UAAI,MAAM,0CAA0C;AAAA,QAClD,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,QAAsB;AAAA,MAC1B,QAAQ,KAAK;AAAA,MACb,IAAI,KAAK;AAAA,MACT,IAAI,KAAK;AAAA,MACT;AAAA,MACA;AAAA,MACA,UAAU,KAAK;AAAA,MACf,MAAM,KAAK;AAAA,MACX;AAAA,MACA;AAAA,MACA;AAAA,IACF;AACA,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,QAAQ,MAAM,YAAY,MAAM,KAAK;AAE3C,qBAAiB,KAAK,cAAc,KAAK,GAAG;AAC1C,YAAM,SAAS,MAAM,OAAO,KAAK,iBAAiB,GAAG,KAAK,CAAC;AAC3D,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,WAAW,MAAsB;AACxC,SAAO,KAAK,OAAO,CAAC,EAAE,YAAY,IAAI,KAAK,MAAM,CAAC;AACpD;;;ACnJA,SAAS,WAAAC,iBAAe;AA4BjB,SAAS,uBAAgC;AAC9C,QAAM,MAAM,IAAIC,UAAQ,gBAAgB,EACrC;AAAA,IACC;AAAA,EACF,EACC,SAAS,cAAc,+DAA+D,EACtF,OAAO,eAAe,yDAAyD,EAC/E,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,OAAiB,SAA6B;AAC9D,QAAI;AACJ,QAAI,KAAK,UAAU,QAAW;AAC5B,UAAI,CAAC,wBAAwB,KAAK,KAAK,GAAG;AACxC,YAAI,MAAM,gDAAgD,KAAK,KAAK,IAAI;AAAA,UACtE,UAAU,WAAW;AAAA,UACrB,MAAM;AAAA,QACR,CAAC;AAAA,MACH;AACA,cAAQ,OAAO,KAAK,KAAK;AAAA,IAC3B;AAEA,UAAM,QAAQ,MAAM,KAAK,GAAG;AAC5B,QAAI,UAAU,KAAK;AACjB,UAAI,MAAM,0EAA0E;AAAA,QAClF,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,UAAU,0BAA0B,OAAO,KAAK;AACtD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,cAAc,OAAO;AACjD,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AAED,SAAO;AACT;;;ACpEA,SAAS,WAAAC,iBAAe;;;ACmBxB,SAAS,YAAYC,WAAU;AAC/B,SAAS,WAAAC,gBAAe;AACxB,SAAS,WAAAC,UAAS,QAAAC,OAAM,eAAe;AACvC,SAAS,qBAAqB;AA0BvB,IAAM,UAAoC;AAAA;AAAA,EAE/C;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA;AAAA,EAGA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA,EAAE,IAAI,QAAQ,OAAO,QAAQ,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA,EAE9F;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AACF;AAEO,SAAS,WAAW,IAAuC;AAChE,SAAO,QAAQ,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE;AACxC;AAMO,SAAS,kBAA0B;AACxC,QAAM,OAAOD,SAAQ,cAAc,YAAY,GAAG,CAAC;AAGnD,SAAO,QAAQ,MAAM,MAAM,WAAW,kBAAkB,UAAU,gBAAgB;AACpF;AAaO,SAAS,cAAc,QAAuB,OAAuB,CAAC,GAAkB;AAC7F,QAAM,OAAO,OAAO,SAAS,SAAU,KAAK,QAAQD,SAAQ,IAAM,KAAK,OAAO,QAAQ,IAAI;AAC1F,SAAO,EAAE,QAAQ,aAAaE,MAAK,MAAM,OAAO,IAAI,EAAE;AACxD;AAgBA,eAAsB,aACpB,QACA,OAAuB,CAAC,GACA;AACxB,QAAM,SAAS,KAAK,aAAa,gBAAgB;AACjD,QAAM,SAAS,MAAM,WAAW,OAAO,WAAW;AAElD,MAAI,UAAU,CAAC,KAAK,OAAO;AACzB,WAAO,EAAE,QAAQ,QAAQ,iBAAiB;AAAA,EAC5C;AAEA,MAAI,OAAO,OAAO,SAAS,OAAO;AAChC,QAAI,OAAQ,OAAMH,IAAG,GAAG,OAAO,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AAC5E,UAAM,cAAc,QAAQ,OAAO,WAAW;AAAA,EAChD,OAAO;AACL,UAAM,YAAY,MAAMA,IAAG,SAASG,MAAK,QAAQ,UAAU,GAAG,MAAM;AACpE,UAAMH,IAAG,MAAME,SAAQ,OAAO,WAAW,GAAG,EAAE,WAAW,KAAK,CAAC;AAC/D,UAAMF,IAAG,UAAU,OAAO,aAAa,SAAS;AAAA,EAClD;AAEA,SAAO,EAAE,QAAQ,QAAQ,SAAS,gBAAgB,YAAY;AAChE;AAEA,eAAe,WAAW,MAAgC;AACxD,MAAI;AACF,UAAMA,IAAG,OAAO,IAAI;AACpB,WAAO;AAAA,EACT,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEA,eAAe,cAAc,QAAgB,aAAoC;AAE/E,QAAMA,IAAG,GAAG,QAAQ,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AACnE;;;AD3LO,SAAS,eAAwB;AACtC,QAAM,MAAM,IAAII,UAAQ,OAAO,EAAE,YAAY,gDAAgD;AAE7F,MACG,QAAQ,OAAO,EACf,YAAY,qEAAqE,EACjF;AAAA,IACC;AAAA,IACA,sCAAsC,QAAQ,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,KAAK,IAAI,CAAC;AAAA,IACzEC;AAAA,IACA,CAAC;AAAA,EACH,EACC,OAAO,aAAa,qCAAqC,EACzD,OAAO,eAAe,gCAAgC,EACtD,OAAO,OAAO,SAA0B;AACvC,UAAM,WAAW,iBAAiB,IAAI;AACtC,QAAI,SAAS,WAAW,GAAG;AACzB,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,eAAW,UAAU,UAAU;AAC7B,YAAM,SAAS,cAAc,MAAM;AACnC,YAAM,SAAS,MAAM,aAAa,QAAQ,EAAE,OAAO,KAAK,MAAM,CAAC;AAC/D,cAAQ,OAAO,MAAM,GAAG,OAAO,KAAK,KAAK,OAAO,MAAM,WAAM,OAAO,WAAW;AAAA,CAAI;AAAA,IACpF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,QAAQ,EAAE,QAAQ,MAAM,CAAC,EACjC,YAAY,0CAA0C,EACtD,OAAO,MAAM;AACZ,eAAW,KAAK,SAAS;AACvB,YAAM,SAAS,cAAc,CAAC;AAC9B,cAAQ,OAAO,MAAM,GAAG,EAAE,GAAG,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,OAAO,EAAE,CAAC,IAAI,OAAO,WAAW;AAAA,CAAI;AAAA,IACzF;AAAA,EACF,CAAC;AAEH,SAAO;AACT;AAEA,SAASA,SAAQ,OAAe,UAA8B;AAC5D,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAEA,SAAS,iBAAiB,MAAwC;AAChE,MAAI,KAAK,IAAK,QAAO,CAAC,GAAG,OAAO;AAChC,QAAM,MAAM,KAAK,UAAU,CAAC;AAC5B,SAAO,IAAI,IAAI,CAAC,OAAO;AACrB,UAAM,SAAS,WAAW,EAAE;AAC5B,QAAI,CAAC,OAAQ,OAAM,IAAI,MAAM,mBAAmB,EAAE,EAAE;AACpD,WAAO;AAAA,EACT,CAAC;AACH;;;AEjEA,SAAS,WAAAC,WAAS,UAAAC,eAAc;AAezB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,UAAQ,MAAM,EAC3B,YAAY,kCAAkC,EAC9C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,uBAAuB,sBAAsB,EAAE,QAAQ,CAAC,SAAS,QAAQ,KAAK,CAAC;AAAA,EAC5F,EACC,OAAO,kBAAkB,4CAA4C;AAExE,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,MAAM,YAAY,MAAM,GAAG;AAEzC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,YAAY,KAAK;AAAA,UACjB,aAAa,KAAK;AAAA,QACpB;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ArBtBA,SAAS,iBAAyB;AAGhC,QAAM,OAAOC,SAAQC,eAAc,YAAY,GAAG,CAAC;AACnD,QAAM,UAAUC,SAAQ,MAAM,MAAM,cAAc;AAClD,QAAM,MAAM,KAAK,MAAM,aAAa,SAAS,MAAM,CAAC;AACpD,SAAO,IAAI;AACb;AAEO,SAAS,eAAwB;AACtC,QAAM,UAAU,IAAIC,UAAQ;AAC5B,UACG,KAAK,gBAAgB,EACrB;AAAA,IACC;AAAA,EAEF,EACC,QAAQ,eAAe,GAAG,iBAAiB,2BAA2B,EACtE,mBAAmB;AAEtB,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,gBAAgB,CAAC;AACpC,UAAQ,WAAW,wBAAwB,CAAC;AAC5C,UAAQ,WAAW,WAAW,CAAC;AAC/B,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,qBAAqB,CAAC;AACzC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,aAAa,CAAC;AAEjC,SAAO;AACT;AAEA,eAAe,KAAK,MAA+B;AACjD,QAAM,UAAU,aAAa;AAC7B,MAAI;AACF,UAAM,QAAQ,WAAW,IAAI;AAAA,EAC/B,SAAS,KAAK;AACZ,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B;AACF;AAIA,IAAM,YAAY,QAAQ,KAAK,CAAC,IAAIF,eAAc,YAAY,GAAG,IAAI;AACrE,IAAI,aAAa,QAAQ,KAAK,CAAC,MAAM,WAAW;AAC9C,OAAK,QAAQ,IAAI,EAAE,MAAM,CAAC,QAAiB;AACzC,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B,CAAC;AACH;","names":["Command","dirname","resolve","fileURLToPath","fs","fs","Command","Option","fs","Command","Option","Command","Command","Command","Command","Command","Command","Command","Option","Command","Option","Command","Option","Command","Option","Command","Command","Command","Option","Command","Option","Command","Command","Command","fs","homedir","dirname","join","Command","collect","Command","Option","Command","Option","dirname","fileURLToPath","resolve","Command"]}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "webscraping-ai-cli",
3
- "version": "1.2.1",
3
+ "version": "1.3.0",
4
4
  "description": "Official command-line client for the WebScraping.AI API. Includes a Claude Code skill so AI agents can use it too.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -53,7 +53,7 @@
53
53
  "homepage": "https://webscraping.ai",
54
54
  "dependencies": {
55
55
  "commander": "^12.1.0",
56
- "webscraping-ai": "^4.2.0"
56
+ "webscraping-ai": "^4.3.0"
57
57
  },
58
58
  "overrides": {
59
59
  "esbuild": "^0.28.1"
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "webscraping-ai",
3
3
  "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — parsed Google search results, and structured data for supported sites (YouTube, TikTok, X, LinkedIn, Instagram, Reddit, …), via the WebScraping.AI CLI.",
4
- "version": "1.2.1",
4
+ "version": "1.3.0",
5
5
  "author": {
6
6
  "name": "WebScraping.AI",
7
7
  "url": "https://webscraping.ai"
@@ -36,7 +36,7 @@ Don't use it for:
36
36
 
37
37
  ## Quick reference
38
38
 
39
- The CLI exposes 9 subcommands, one per API endpoint:
39
+ The CLI exposes 10 subcommands, one per API endpoint:
40
40
 
41
41
  | Command | What it does |
42
42
  | ------------------- | ------------------------------------------------- |
@@ -47,12 +47,13 @@ The CLI exposes 9 subcommands, one per API endpoint:
47
47
  | `ask` | Ask a natural-language question about a page (AI) |
48
48
  | `extract` | Extract structured fields with descriptions (AI) |
49
49
  | `serp` / `search` | Parsed Google search results for a query (JSON) |
50
+ | `serp-locations` | Look up location names for `serp --location` |
50
51
  | `data` | Structured JSON for a page on a supported site |
51
52
  | `account` | Show remaining API credits / quota |
52
53
 
53
54
  Authentication: set `WEBSCRAPING_AI_API_KEY`, pass `--api-key`, or run `webscraping-ai auth set <key>` once.
54
55
 
55
- Every page command supports common flags: `--js/--no-js`, `--proxy {datacenter|residential|stealth}`, `--country <cc>`, `--device {desktop|mobile|tablet}`, `--timeout <ms>`, `--js-timeout <ms>`, `--wait-for <selector>`, `--headers '{"Cookie":"..."}'`, `--output <file>`. `serp` is the exception: it takes a query, not a URL, and only `--engine`, `--gl <cc>`, `--hl <lang>`, `--page <n>`, `--output`. `data` takes a URL but its own flag set: `--country <cc>` (same meaning as on the page commands: the proxy country, `us` by default), `--transcript`, `--transcript-language <lang>`, `--param key=value`, `--output`. The other scrape flags (`--js`, `--proxy`, `--device`, `--timeout`, `--headers`, …) don't exist on `data`.
56
+ Every page command supports common flags: `--js/--no-js`, `--proxy {datacenter|residential|stealth}`, `--country <cc>`, `--device {desktop|mobile|tablet}`, `--timeout <ms>`, `--js-timeout <ms>`, `--wait-for <selector>`, `--headers '{"Cookie":"..."}'`, `--output <file>`. `serp` is the exception: it takes a query, not a URL, and only `--engine`, `--gl <cc>`, `--hl <lang>`, `--page <n>`, `--pages <n>`, `--location <name>`, `--uule <value>`, `--lat <deg>`, `--lon <deg>`, `--radius <m>`, `--output`. `serp-locations [query]` takes only `--limit <n>` and `--output`. `data` takes a URL but its own flag set: `--country <cc>` (same meaning as on the page commands: the proxy country, `us` by default), `--transcript`, `--transcript-language <lang>`, `--param key=value`, `--output`. The other scrape flags (`--js`, `--proxy`, `--device`, `--timeout`, `--headers`, …) don't exist on `data`.
56
57
 
57
58
  ## How to pick the right command
58
59
 
@@ -68,7 +69,7 @@ Every page command supports common flags: `--js/--no-js`, `--proxy {datacenter|r
68
69
 
69
70
  **Need a JSON object of typed fields**? → `extract --fields '{"title":"Main product title","price":"Current price including currency"}'`. Returns `{ result: { title: "...", price: "..." } }`.
70
71
 
71
- **Starting from a query, not a URL** (find pages about X, check who ranks for a term, see what Google shows in a given country)? → `serp "<query>"`. Returns JSON with `organic_results` (`position`, `title`, `link`, `domain`, `snippet`), `related_searches`, and `pagination`. Use `--gl`/`--hl` for country/language and `--page 2` for the next 10 results. Don't `html` a google.com search URL and parse it yourself — `serp` is cheaper, more reliable, and already parsed. Then feed the `link`s you care about into `text`/`ask`/`extract`.
72
+ **Starting from a query, not a URL** (find pages about X, check who ranks for a term, see what Google shows in a given country)? → `serp "<query>"`. Returns JSON with `organic_results` (`position`, `title`, `link`, `domain`, `snippet`), `related_searches`, and `pagination`. Use `--gl`/`--hl` for country/language and `--page 2` for the next 10 results, or `--pages 3` to get pages 1–3 in one call as one de-duplicated list (up to 5; billed per results page returned). For local results ("best pizza" as seen from a city), add `--location 'Austin, TX'` (or `--lat`/`--lon`, optionally `--radius` in meters). Every search runs from one fixed location (otherwise the capital of the `--gl` country), so repeats and later pages are consistent, and `--gl` defaults to that location's country. Not sure of the name? `serp-locations austin` lists matches (English names), most populous first; pass a result's `canonical_name` (or `id`) as `--location`. An unknown location is a 400 (exit 3, not charged). Don't `html` a google.com search URL and parse it yourself — `serp` is cheaper, more reliable, and already parsed. Then feed the `link`s you care about into `text`/`ask`/`extract`.
72
73
 
73
74
  **The URL is a YouTube video/channel/playlist, TikTok video/profile, X/Twitter post/profile, LinkedIn company/job/profile, Instagram post/reel/profile or Reddit post/subreddit/user**? → try `data <url>` first. It returns `{ request_parameters: { provider, type }, parse_status, data }` with the page's fields already parsed (snake_case; `null` for what the page doesn't show). Add `--transcript` on a YouTube video to also fetch its transcript into `data.transcript` (null when no matching captions are available; `--transcript-language de` picks the caption language, otherwise English, then the first track). If the transcript fetch itself fails, the whole request fails with a 500 (exit 7) and is not charged — retry without `--transcript` if you only need the metadata. Those sites are examples, not the full list: more are added server-side, so just try the URL. An unsupported URL or page type returns a 400 that is not charged (exit 3); its message lists what is supported. Then `extract`/`ask`/`text` are the fallback.
74
75
 
@@ -100,6 +101,12 @@ webscraping-ai extract https://shop.example.com/item \
100
101
  # Google results for a query (German results from Germany, page 2)
101
102
  webscraping-ai serp coffee machines --gl de --hl de --page 2
102
103
 
104
+ # Two pages of local results as searched from Austin, TX
105
+ webscraping-ai serp coffee machines --location 'Austin, TX' --pages 2
106
+
107
+ # Look up the exact location name first (free)
108
+ webscraping-ai serp-locations austin --limit 3 --no-pretty | jq -r '.[0].canonical_name'
109
+
103
110
  # Search, then read the top result
104
111
  webscraping-ai serp "rust async runtime comparison" --no-pretty | jq -r '.organic_results[0].link' \
105
112
  | webscraping-ai text -
@@ -121,8 +128,8 @@ webscraping-ai html https://geo.example.com \
121
128
  - **`--js` is on by default.** Pass `--no-js` for static pages — it's significantly faster and cheaper.
122
129
  - **`selected-multiple` returns nested arrays** (`Array<Array<string>>`) — known API response shape. The outer wrapper holds all matches concatenated; flatten in your script if needed.
123
130
  - **`extract` wraps its output** under a `result` key: the parsed shape is `{ "result": { ... } }`.
124
- - **`serp` costs a flat 15 credits per search** (failed searches aren't charged) and returns at most 10 organic results per page. Optional fields (`snippet`, `date`, `related_searches`, `pagination.next`, `search_information.total_results`) may be absent.
125
- - **`data` costs 15 credits per request**, including `parse_status` `parse_failed` (fetched but not parsed; `data` may be `null` or partial) and `not_found`. Unsupported URLs or page types (exit 3, HTTP 400) and failed fetches aren't charged. `--transcript` can turn an otherwise successful request into a refunded 500 (exit 7) when the transcript fetch fails. The shape of `data` differs per `provider`/`type`, and `provider`/`type` values grow over time.
131
+ - **`serp` is billed per results page returned** (see https://webscraping.ai/docs#serp; failed searches aren't charged) and returns at most 10 organic results per page. `position` restarts at 1 on every page, except with `--pages` > 1, where it counts through the combined list and each result has a `page` field. Use only one of `--location`, `--lat`+`--lon` or `--uule`; the server rejects other combinations with a 400 (not charged). Optional fields (`snippet`, `date`, `related_searches`, `pagination.next`, `search_information.total_results`) may be absent.
132
+ - **`data` is priced per site** (see https://webscraping.ai/docs), including `parse_status` `parse_failed` (fetched but not parsed; `data` may be `null` or partial) and `not_found`. Unsupported URLs or page types (exit 3, HTTP 400) and failed fetches aren't charged. `--transcript` can turn an otherwise successful request into a refunded 500 (exit 7) when the transcript fetch fails. The shape of `data` differs per `provider`/`type`, and `provider`/`type` values grow over time.
126
133
  - **API key in URL.** The CLI sends the key as a query-string param. Don't log full requests in shared logs.
127
134
  - **Exit codes** are stable per error class (auth → 4, payment → 5, rate-limit → 6, server → 7, timeout → 8, connection → 9). Script around them.
128
135
 
@@ -131,7 +138,7 @@ webscraping-ai html https://geo.example.com \
131
138
  - Text/HTML responses print verbatim. JSON responses pretty-print on a TTY and single-line on a pipe.
132
139
  - `--output FILE` writes the result to a file instead of stdout.
133
140
  - For batched stdin input, each URL's response is emitted in order, separated by newlines. `-o FILE` is truncated when the batch starts and collects all of them. A batch **stops at the first failing item** (same for every command, `data -` included): the results before it are already written, the rest of the list is not processed, and the exit code is that error's. Split the list or loop per URL in the shell if you need to keep going past failures. URL lists skip `#` comment lines; `serp -` skips only blank lines, so `#hashtag` queries are searched.
134
- - `serp --page` must be an integer >= 1 (server rejects > 100 with a 400); anything else exits with code 2 before any request.
141
+ - `serp --page`, `--pages` and `--radius` must be integers >= 1 (server rejects `--page` > 100, `--pages` > 5 and `--page` + `--pages` - 1 > 100 with a 400, not charged) and `--lat`/`--lon` plain decimals (`--lat -33.86` works); anything else exits with code 2 before any request. `serp-locations` prints a JSON array; `--limit` is 1–10, and it has no stdin mode (`serp-locations -` exits 2).
135
142
 
136
143
  ## See also
137
144