webscraping-ai-cli 1.0.2 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://json.schemastore.org/claude-code-plugin-marketplace.json",
3
3
  "name": "webscraping-ai",
4
- "description": "Official WebScraping.AI plugins for Claude Code — scrape and AI-extract data from any URL.",
4
+ "description": "Official WebScraping.AI plugins for Claude Code — scrape and AI-extract data from any URL, and get parsed Google search (SERP) results.",
5
5
  "owner": {
6
6
  "name": "WebScraping.AI",
7
7
  "url": "https://webscraping.ai"
@@ -10,10 +10,20 @@
10
10
  {
11
11
  "name": "webscraping-ai",
12
12
  "source": "./plugins/webscraping-ai",
13
- "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — via the WebScraping.AI CLI.",
14
- "version": "1.0.0",
13
+ "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — and parsed Google search results, via the WebScraping.AI CLI.",
14
+ "version": "1.1.0",
15
15
  "category": "web-scraping",
16
- "keywords": ["web-scraping", "html", "text-extraction", "ai-extraction", "scraping", "cli"]
16
+ "keywords": [
17
+ "web-scraping",
18
+ "html",
19
+ "text-extraction",
20
+ "ai-extraction",
21
+ "scraping",
22
+ "search",
23
+ "serp",
24
+ "google-search",
25
+ "cli"
26
+ ]
17
27
  }
18
28
  ]
19
29
  }
package/CHANGELOG.md CHANGED
@@ -5,6 +5,27 @@ All notable changes to `webscraping-ai-cli` are documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## 1.1.0 — 2026-09-25
9
+
10
+ ### Added
11
+
12
+ - `serp <query>` subcommand (alias `search`) wrapping the new `/serp` endpoint: parsed Google search results as JSON. Flags: `--engine` (`google`), `--gl`, `--hl`, `--page <n>`. Multi-word queries don't need quoting; `-` reads one query per line from stdin. The scrape flags (`--js`, `--proxy`, `--country`, …) don't apply and aren't registered on it. Flat 15 credits per search.
13
+ - Bundled agent skill documents when to reach for `serp` (query-first research) instead of scraping a search URL.
14
+
15
+ ### Changed
16
+
17
+ - Requires `webscraping-ai` SDK `^4.1.0` (adds `serp()`).
18
+ - `serp --page` is validated strictly (digits only, >= 1): `1.5`, `2abc` and `0` are rejected with exit code 2 before stdin is read or the key is resolved (the server also rejects them with a 400, not billed; checking client-side saves the round trip). Pages are 1–100: the server rejects a `page` above 100 with a 400.
19
+ - `serp` stdin batches skip only blank lines, so `#hashtag` queries are searched rather than dropped as comments. URL commands still treat `#` lines as comments.
20
+ - `serp` sends the query as typed (whitespace-only queries are still rejected). `from_cli` is not sent on `serp`: the SDK forwards only the SERP parameters.
21
+
22
+ ### Fixed
23
+
24
+ - `--pretty` / `--no-pretty` no longer fail with "option '--no-pretty' cannot be used with option '--no-pretty'" (a self-conflict on every scrape command and `serp`).
25
+ - `-o/--output` in stdin batch mode kept only the last result; the file is now truncated once and each result appended.
26
+ - Error messages printed to stderr redact the API key and any `api_key=...` pattern (an HTML error body echoing the request URL used to print the key).
27
+ - Smoke script (`npm run smoke`) now checks results, not just exit codes: `serp` needs non-empty `organic_results` and a matching `search_parameters.q`, `selected-multiple` needs a non-empty inner array; signal-killed runs fail; page commands run with `--no-js --proxy datacenter` (~31 credits per sweep); FAIL lines redact the key.
28
+
8
29
  ## 1.0.2 — 2026-07-17
9
30
 
10
31
  ### Changed
package/README.md CHANGED
@@ -45,29 +45,32 @@ webscraping-ai selected <url> --selector <css> # One CSS area
45
45
  webscraping-ai selected-multiple <url> --selector a --selector b ...
46
46
  webscraping-ai ask <url> --question "..." # AI question about the page
47
47
  webscraping-ai extract <url> --fields '{"title":"...","price":"..."}'
48
+ webscraping-ai serp <query> [--gl us] [--hl en] [--page 1] # Parsed Google results (alias: search)
48
49
  webscraping-ai account # Remaining credits / quota
49
50
  ```
50
51
 
52
+ `serp` (alias `search`) is query-shaped, not URL-shaped: it takes `--engine` (`google`, the default), `--gl` (country), `--hl` (language), `--page` (1-based, 10 results per page) plus `--api-key`/`--output`/`--pretty`, and none of the scrape flags below. It prints the JSON result (`search_parameters`, `search_information`, `organic_results`, `related_searches`, `pagination`). Flat 15 credits per search; failed searches are not charged. `--page` must be an integer >= 1 (pages are 1–100; the server rejects values above 100 with a 400); anything else is a usage error (exit 2) before any request. Multi-word queries don't need quoting; `-` reads one query per line from stdin, skipping only blank lines (a leading `#` is kept as part of the query, e.g. `#coffee`). The CLI's `from_cli` analytics flag is not sent on `serp` (the SDK forwards only the SERP parameters).
53
+
51
54
  Common flags shared by every scrape command:
52
55
 
53
- | Flag | Purpose |
54
- | --------------------- | -------------------------------------------------------- |
55
- | `--js` / `--no-js` | Toggle JavaScript rendering (default: on) |
56
- | `--proxy TYPE` | `datacenter` (default), `residential`, `stealth` |
57
- | `--country CC` | Two-letter proxy country code |
58
- | `--device TYPE` | `desktop`, `mobile`, `tablet` |
59
- | `--timeout MS` | Total page-retrieval timeout (max 30000) |
60
- | `--js-timeout MS` | JS rendering time after page load (max 20000) |
61
- | `--wait-for CSS` | CSS selector to wait for |
62
- | `--headers JSON` | HTTP headers to forward; inline JSON or `@file.json` |
63
- | `--custom-proxy URL` | `http://user:pass@host:port` |
64
- | `--js-script CODE` | Custom JS to evaluate on the target page |
65
- | `--error-on-404` | Treat 404 responses as errors |
66
- | `--error-on-redirect` | Treat redirects as errors |
67
- | `--output FILE` | Write to FILE instead of stdout |
68
- | `--pretty / --no-pretty` | Force pretty / single-line JSON |
69
-
70
- `<url>` may be `-` to read URLs from stdin (one per line; lines starting with `#` are ignored).
56
+ | Flag | Purpose |
57
+ | ------------------------ | ---------------------------------------------------- |
58
+ | `--js` / `--no-js` | Toggle JavaScript rendering (default: on) |
59
+ | `--proxy TYPE` | `datacenter` (default), `residential`, `stealth` |
60
+ | `--country CC` | Two-letter proxy country code |
61
+ | `--device TYPE` | `desktop`, `mobile`, `tablet` |
62
+ | `--timeout MS` | Total page-retrieval timeout (max 30000) |
63
+ | `--js-timeout MS` | JS rendering time after page load (max 20000) |
64
+ | `--wait-for CSS` | CSS selector to wait for |
65
+ | `--headers JSON` | HTTP headers to forward; inline JSON or `@file.json` |
66
+ | `--custom-proxy URL` | `http://user:pass@host:port` |
67
+ | `--js-script CODE` | Custom JS to evaluate on the target page |
68
+ | `--error-on-404` | Treat 404 responses as errors |
69
+ | `--error-on-redirect` | Treat redirects as errors |
70
+ | `--output FILE` | Write to FILE instead of stdout |
71
+ | `--pretty / --no-pretty` | Force pretty / single-line JSON |
72
+
73
+ `<url>` may be `-` to read URLs from stdin (one per line; lines starting with `#` are ignored). With `--output FILE`, a stdin batch writes every result to FILE, one after another (the file is truncated once at the start).
71
74
 
72
75
  ## Install the AI agent skill
73
76
 
@@ -88,18 +91,18 @@ You can also install the skill via the Claude Code [Plugin Marketplace](https://
88
91
 
89
92
  ## Exit codes
90
93
 
91
- | Code | Meaning |
92
- | ---- | ------------------------------------------------ |
93
- | 0 | Success |
94
- | 2 | Usage error (commander) |
95
- | 3 | 400 Bad Request from the API |
96
- | 4 | 403 Authentication error |
97
- | 5 | 402 Payment Required (out of credits) |
98
- | 6 | 429 Rate limit |
99
- | 7 | 500 Server error (or wrapped target-page error) |
100
- | 8 | Timeout (504 or local AbortController) |
101
- | 9 | Connection / DNS failure |
102
- | 1 | Anything else |
94
+ | Code | Meaning |
95
+ | ---- | ----------------------------------------------- |
96
+ | 0 | Success |
97
+ | 2 | Usage error (commander) |
98
+ | 3 | 400 Bad Request from the API |
99
+ | 4 | 403 Authentication error |
100
+ | 5 | 402 Payment Required (out of credits) |
101
+ | 6 | 429 Rate limit |
102
+ | 7 | 500 Server error (or wrapped target-page error) |
103
+ | 8 | Timeout (504 or local AbortController) |
104
+ | 9 | Connection / DNS failure |
105
+ | 1 | Anything else |
103
106
 
104
107
  Scripts can branch on these without parsing stderr.
105
108
 
@@ -118,6 +121,12 @@ cat urls.txt | webscraping-ai extract - \
118
121
  --fields '{"title":"Product title","price":"Current price"}' \
119
122
  --output products.ndjson
120
123
 
124
+ # Google results for a query, page 2, from Germany in German
125
+ webscraping-ai serp coffee machines --gl de --hl de --page 2
126
+
127
+ # Just the organic result links
128
+ webscraping-ai search "best espresso grinder" | jq -r '.organic_results[].link'
129
+
121
130
  # Headers from a file
122
131
  webscraping-ai html https://example.com --headers @./headers.json
123
132
  ```
package/dist/cli.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  // src/cli.ts
4
- import { Command as Command10 } from "commander";
4
+ import { Command as Command11 } from "commander";
5
5
  import { readFileSync } from "fs";
6
6
  import { dirname as dirname3, resolve as resolve2 } from "path";
7
7
  import { fileURLToPath as fileURLToPath2 } from "url";
@@ -67,15 +67,25 @@ function maskApiKey(key) {
67
67
 
68
68
  // src/lib/output.ts
69
69
  import { promises as fs2 } from "fs";
70
- async function emit(value, opts = {}) {
70
+ async function emit(value, opts = {}, write = {}) {
71
71
  const text = stringify(value, opts);
72
72
  if (opts.output && opts.output.trim() !== "") {
73
- await fs2.writeFile(opts.output, text + (text.endsWith("\n") ? "" : "\n"));
73
+ const data = text + (text.endsWith("\n") ? "" : "\n");
74
+ if (write.append) await fs2.appendFile(opts.output, data);
75
+ else await fs2.writeFile(opts.output, data);
74
76
  return;
75
77
  }
76
78
  process.stdout.write(text);
77
79
  if (!text.endsWith("\n")) process.stdout.write("\n");
78
80
  }
81
+ function createEmitter(opts = {}) {
82
+ let first = true;
83
+ return async (value) => {
84
+ const append = !first;
85
+ first = false;
86
+ await emit(value, opts, { append });
87
+ };
88
+ }
79
89
  function stringify(value, opts = {}) {
80
90
  if (typeof value === "string") return value;
81
91
  const pretty = opts.pretty ?? process.stdout.isTTY ?? false;
@@ -84,7 +94,29 @@ function stringify(value, opts = {}) {
84
94
 
85
95
  // src/lib/sdk.ts
86
96
  import { WebScrapingAI } from "webscraping-ai";
97
+
98
+ // src/lib/redact.ts
99
+ var secrets = /* @__PURE__ */ new Set();
100
+ function registerSecret(value) {
101
+ if (value && value.trim() !== "") secrets.add(value);
102
+ }
103
+ var API_KEY_PARAM = /(api_key=)[^&\s"'<>]*/gi;
104
+ function redactSecrets(text) {
105
+ let out = text.replace(API_KEY_PARAM, "$1[REDACTED]");
106
+ const candidates = new Set(secrets);
107
+ const env = process.env.WEBSCRAPING_AI_API_KEY?.trim();
108
+ if (env) candidates.add(env);
109
+ for (const secret of candidates) {
110
+ for (const form of /* @__PURE__ */ new Set([secret, encodeURIComponent(secret)])) {
111
+ out = out.split(form).join("[REDACTED]");
112
+ }
113
+ }
114
+ return out;
115
+ }
116
+
117
+ // src/lib/sdk.ts
87
118
  function createClient(options) {
119
+ registerSecret(options.apiKey);
88
120
  const inner = new WebScrapingAI({
89
121
  apiKey: options.apiKey,
90
122
  timeoutMs: options.requestTimeoutMs
@@ -107,6 +139,8 @@ function createClient(options) {
107
139
  return (o) => target.question({ ...o, ...tag });
108
140
  case "fields":
109
141
  return (o) => target.fields({ ...o, ...tag });
142
+ case "serp":
143
+ return (o) => target.serp({ ...o, ...tag });
110
144
  case "account":
111
145
  return value.bind(target);
112
146
  default:
@@ -177,19 +211,29 @@ function buildQuestionOptions(url, question, flags, resolved) {
177
211
  function buildFieldsOptions(url, fields, flags, resolved) {
178
212
  return { url, fields, ...buildCommonOptions(flags, resolved) };
179
213
  }
214
+ function buildSerpOptions(query, flags = {}) {
215
+ if (typeof query !== "string" || query.trim() === "") {
216
+ throw new Error("serp requires a non-empty query");
217
+ }
218
+ const out = { q: query };
219
+ if (flags.engine !== void 0) out.engine = flags.engine;
220
+ if (flags.gl) out.gl = flags.gl;
221
+ if (flags.hl) out.hl = flags.hl;
222
+ if (flags.page !== void 0) {
223
+ if (!Number.isSafeInteger(flags.page) || flags.page < 1) {
224
+ throw new Error(`--page must be an integer >= 1, got: ${flags.page}`);
225
+ }
226
+ out.page = flags.page;
227
+ }
228
+ return out;
229
+ }
180
230
 
181
231
  // src/lib/options.ts
182
232
  import { promises as fs3 } from "fs";
183
233
  import { Option } from "commander";
184
234
  function addCommonScrapeOptions(cmd) {
185
- return cmd.option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").addOption(
186
- new Option("--no-pretty", "force single-line JSON output").conflicts("pretty")
187
- ).option("--js", "execute JS via a headless browser (default: enabled)").option("--no-js", "disable JS execution (faster, cheaper)").addOption(
188
- new Option("--proxy <type>", "proxy pool").choices([
189
- "datacenter",
190
- "residential",
191
- "stealth"
192
- ])
235
+ return cmd.option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output").option("--js", "execute JS via a headless browser (default: enabled)").option("--no-js", "disable JS execution (faster, cheaper)").addOption(
236
+ new Option("--proxy <type>", "proxy pool").choices(["datacenter", "residential", "stealth"])
193
237
  ).option("--country <code>", "two-letter proxy country code, e.g. us, gb, de").addOption(
194
238
  new Option("--device <type>", "device emulation").choices(["desktop", "mobile", "tablet"])
195
239
  ).option("--timeout <ms>", "page-retrieval timeout in ms (max 30000)", integerParser).option("--js-timeout <ms>", "JS rendering time in ms after load (max 20000)", integerParser).option("--wait-for <selector>", "CSS selector to wait for before returning content").option(
@@ -197,6 +241,9 @@ function addCommonScrapeOptions(cmd) {
197
241
  "HTTP headers to forward to the target page; JSON object or @file.json"
198
242
  ).option("--js-script <code>", "custom JS to evaluate on the target page").option("--custom-proxy <url>", "custom proxy URL: http://user:pass@host:port").option("--error-on-404", "return error on 404").option("--error-on-redirect", "return error on redirect");
199
243
  }
244
+ function isPositiveIntegerString(value) {
245
+ return /^[1-9]\d*$/.test(value) && Number.isSafeInteger(Number(value));
246
+ }
200
247
  function integerParser(value) {
201
248
  const n = Number.parseInt(value, 10);
202
249
  if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);
@@ -246,6 +293,13 @@ async function* readUrlsFromStdin() {
246
293
  yield line;
247
294
  }
248
295
  }
296
+ async function* readQueriesFromStdin(input = process.stdin) {
297
+ const rl = readline.createInterface({ input, crlfDelay: Infinity });
298
+ for await (const line of rl) {
299
+ if (line.trim() === "") continue;
300
+ yield line;
301
+ }
302
+ }
249
303
  async function* urlIterator(arg) {
250
304
  if (urlIsStdin(arg)) {
251
305
  yield* readUrlsFromStdin();
@@ -253,6 +307,13 @@ async function* urlIterator(arg) {
253
307
  }
254
308
  yield arg;
255
309
  }
310
+ async function* queryIterator(arg) {
311
+ if (urlIsStdin(arg)) {
312
+ yield* readQueriesFromStdin();
313
+ return;
314
+ }
315
+ yield arg;
316
+ }
256
317
 
257
318
  // src/commands/ask.ts
258
319
  function askCommand() {
@@ -263,10 +324,11 @@ function askCommand() {
263
324
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
264
325
  const headers = await parseHeaders(opts.headers);
265
326
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
327
+ const write = createEmitter(opts);
266
328
  for await (const target of urlIterator(url)) {
267
329
  const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });
268
330
  const result = await client.question(sdkOptions);
269
- await emit(result, opts);
331
+ await write(result);
270
332
  }
271
333
  });
272
334
  return cmd;
@@ -286,8 +348,10 @@ function authCommand() {
286
348
  const persisted = await readPersistedConfig();
287
349
  const env = process.env.WEBSCRAPING_AI_API_KEY;
288
350
  if (env && env.trim() !== "") {
289
- process.stdout.write(`Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}
290
- `);
351
+ process.stdout.write(
352
+ `Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}
353
+ `
354
+ );
291
355
  }
292
356
  if (persisted.apiKey && persisted.apiKey.trim() !== "") {
293
357
  process.stdout.write(
@@ -301,8 +365,10 @@ function authCommand() {
301
365
  });
302
366
  cmd.command("clear").description("Remove the stored API key file").action(async () => {
303
367
  const removed = await clearPersistedConfig();
304
- process.stdout.write(removed ? `Removed ${configPath()}
305
- ` : "No stored API key to remove.\n");
368
+ process.stdout.write(
369
+ removed ? `Removed ${configPath()}
370
+ ` : "No stored API key to remove.\n"
371
+ );
306
372
  });
307
373
  return cmd;
308
374
  }
@@ -321,10 +387,11 @@ function extractCommand() {
321
387
  const headers = await parseHeaders(opts.headers);
322
388
  const fields = await parseFields(opts.fields);
323
389
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
390
+ const write = createEmitter(opts);
324
391
  for await (const target of urlIterator(url)) {
325
392
  const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });
326
393
  const result = await client.fields(sdkOptions);
327
- await emit(result, opts);
394
+ await write(result);
328
395
  }
329
396
  });
330
397
  return cmd;
@@ -333,18 +400,26 @@ function extractCommand() {
333
400
  // src/commands/html.ts
334
401
  import { Command as Command5, Option as Option3 } from "commander";
335
402
  function htmlCommand() {
336
- const cmd = new Command5("html").description("Fetch the full HTML of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(new Option3("--return-script-result", "return the result of --js-script instead of HTML")).addOption(new Option3("--format <fmt>", "response wrapping").choices(["text", "json"]));
403
+ const cmd = new Command5("html").description("Fetch the full HTML of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
404
+ new Option3("--return-script-result", "return the result of --js-script instead of HTML")
405
+ ).addOption(new Option3("--format <fmt>", "response wrapping").choices(["text", "json"]));
337
406
  addCommonScrapeOptions(cmd);
338
407
  cmd.action(async (url, opts) => {
339
408
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
340
409
  const headers = await parseHeaders(opts.headers);
341
410
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
411
+ const write = createEmitter(opts);
342
412
  for await (const target of urlIterator(url)) {
343
- const sdkOptions = buildHtmlOptions(target, opts, { headers }, {
344
- returnScriptResult: opts.returnScriptResult
345
- });
413
+ const sdkOptions = buildHtmlOptions(
414
+ target,
415
+ opts,
416
+ { headers },
417
+ {
418
+ returnScriptResult: opts.returnScriptResult
419
+ }
420
+ );
346
421
  const result = await client.html(sdkOptions);
347
- await emit(result, opts);
422
+ await write(result);
348
423
  }
349
424
  });
350
425
  return cmd;
@@ -362,10 +437,11 @@ function selectedCommand() {
362
437
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
363
438
  const headers = await parseHeaders(opts.headers);
364
439
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
440
+ const write = createEmitter(opts);
365
441
  for await (const target of urlIterator(url)) {
366
442
  const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });
367
443
  const result = await client.selected(sdkOptions);
368
- await emit(result, opts);
444
+ await write(result);
369
445
  }
370
446
  });
371
447
  return cmd;
@@ -386,10 +462,11 @@ function selectedMultipleCommand() {
386
462
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
387
463
  const headers = await parseHeaders(opts.headers);
388
464
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
465
+ const write = createEmitter(opts);
389
466
  for await (const target of urlIterator(url)) {
390
467
  const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });
391
468
  const result = await client.selectedMultiple(sdkOptions);
392
- await emit(result, opts);
469
+ await write(result);
393
470
  }
394
471
  });
395
472
  return cmd;
@@ -398,8 +475,98 @@ function collectMulti(value, previous) {
398
475
  return [...previous, value];
399
476
  }
400
477
 
478
+ // src/commands/serp.ts
479
+ import { Command as Command8, Option as Option5 } from "commander";
480
+
481
+ // src/lib/errors.ts
482
+ import {
483
+ APIConnectionError,
484
+ APIError,
485
+ APITimeoutError,
486
+ AuthenticationError,
487
+ BadRequestError,
488
+ GatewayTimeoutError,
489
+ PaymentRequiredError,
490
+ RateLimitError,
491
+ ServerError,
492
+ WebScrapingAIError
493
+ } from "webscraping-ai";
494
+ var EXIT_CODES = {
495
+ ok: 0,
496
+ generic: 1,
497
+ usage: 2,
498
+ badRequest: 3,
499
+ auth: 4,
500
+ paymentRequired: 5,
501
+ rateLimit: 6,
502
+ serverError: 7,
503
+ timeout: 8,
504
+ connection: 9
505
+ };
506
+ function exitCodeFor(err) {
507
+ if (err instanceof BadRequestError) return EXIT_CODES.badRequest;
508
+ if (err instanceof AuthenticationError) return EXIT_CODES.auth;
509
+ if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;
510
+ if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;
511
+ if (err instanceof ServerError) return EXIT_CODES.serverError;
512
+ if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;
513
+ if (err instanceof APITimeoutError) return EXIT_CODES.timeout;
514
+ if (err instanceof APIConnectionError) return EXIT_CODES.connection;
515
+ if (err instanceof APIError) return EXIT_CODES.serverError;
516
+ if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;
517
+ return EXIT_CODES.generic;
518
+ }
519
+ function formatError(err) {
520
+ return redactSecrets(rawMessage(err));
521
+ }
522
+ function rawMessage(err) {
523
+ if (err instanceof APIError) {
524
+ const parts = [err.message];
525
+ if (err.status) parts.push(`(HTTP ${err.status})`);
526
+ return parts.join(" ");
527
+ }
528
+ if (err instanceof Error) return err.message;
529
+ return String(err);
530
+ }
531
+
532
+ // src/commands/serp.ts
533
+ function serpCommand() {
534
+ const cmd = new Command8("serp").alias("search").description("Search Google and print parsed results as JSON (wraps /serp, 15 credits/search)").argument("<query...>", "search query (or `-` to read queries from stdin, one per line)").addOption(new Option5("--engine <name>", "search engine (default: google)").choices(["google"])).option("--gl <code>", "two-letter country code for search geolocation (default: us)").option("--hl <code>", "two-letter language code for the results (default: en)").option(
535
+ "--page <n>",
536
+ "results page number, integer >= 1, 10 results per page (default: 1; server rejects > 100 with a 400)"
537
+ ).option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output");
538
+ cmd.action(async (words, opts) => {
539
+ const query = words.join(" ");
540
+ let page;
541
+ if (opts.page !== void 0) {
542
+ if (!isPositiveIntegerString(opts.page)) {
543
+ cmd.error(`error: --page must be an integer >= 1, got: ${opts.page}`, {
544
+ exitCode: EXIT_CODES.usage,
545
+ code: "wsai.invalidPage"
546
+ });
547
+ }
548
+ page = Number(opts.page);
549
+ }
550
+ if (!urlIsStdin(query) && query.trim() === "") {
551
+ cmd.error("error: serp requires a non-empty query", {
552
+ exitCode: EXIT_CODES.usage,
553
+ code: "wsai.emptyQuery"
554
+ });
555
+ }
556
+ const flags = { engine: opts.engine, gl: opts.gl, hl: opts.hl, page };
557
+ const apiKey = await resolveApiKey({ flag: opts.apiKey });
558
+ const client = createClient({ apiKey });
559
+ const write = createEmitter(opts);
560
+ for await (const q of queryIterator(query)) {
561
+ const result = await client.serp(buildSerpOptions(q, flags));
562
+ await write(result);
563
+ }
564
+ });
565
+ return cmd;
566
+ }
567
+
401
568
  // src/commands/setupSkill.ts
402
- import { Command as Command8 } from "commander";
569
+ import { Command as Command9 } from "commander";
403
570
 
404
571
  // src/lib/skill.ts
405
572
  import { promises as fs4 } from "fs";
@@ -408,24 +575,72 @@ import { dirname as dirname2, join as join2, resolve } from "path";
408
575
  import { fileURLToPath } from "url";
409
576
  var EDITORS = [
410
577
  // Claude Code: user-scope skills live under ~/.claude/skills/<name>/
411
- { id: "claude-code", label: "Claude Code", mode: "dir", kind: "user", path: ".claude/skills/webscraping-ai" },
578
+ {
579
+ id: "claude-code",
580
+ label: "Claude Code",
581
+ mode: "dir",
582
+ kind: "user",
583
+ path: ".claude/skills/webscraping-ai"
584
+ },
412
585
  // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill
413
586
  // dir, so this is the editor's "install" surface.
414
- { id: "cursor", label: "Cursor", mode: "file", kind: "project", path: ".cursor/rules/webscraping-ai.mdc" },
587
+ {
588
+ id: "cursor",
589
+ label: "Cursor",
590
+ mode: "file",
591
+ kind: "project",
592
+ path: ".cursor/rules/webscraping-ai.mdc"
593
+ },
415
594
  // Windsurf: project-scope rules. Same shape as Cursor.
416
- { id: "windsurf", label: "Windsurf", mode: "file", kind: "project", path: ".windsurf/rules/webscraping-ai.md" },
595
+ {
596
+ id: "windsurf",
597
+ label: "Windsurf",
598
+ mode: "file",
599
+ kind: "project",
600
+ path: ".windsurf/rules/webscraping-ai.md"
601
+ },
417
602
  // Kiro: user-scope steering directory.
418
603
  { id: "kiro", label: "Kiro", mode: "dir", kind: "user", path: ".kiro/steering/webscraping-ai" },
419
604
  // OpenCode: project-scope skills (mirrors Claude shape).
420
- { id: "opencode", label: "OpenCode", mode: "dir", kind: "project", path: ".opencode/skills/webscraping-ai" },
605
+ {
606
+ id: "opencode",
607
+ label: "OpenCode",
608
+ mode: "dir",
609
+ kind: "project",
610
+ path: ".opencode/skills/webscraping-ai"
611
+ },
421
612
  // Gemini CLI: user-scope extension dir with a GEMINI.md.
422
- { id: "gemini", label: "Gemini CLI", mode: "file", kind: "user", path: ".gemini/extensions/webscraping-ai/GEMINI.md" },
613
+ {
614
+ id: "gemini",
615
+ label: "Gemini CLI",
616
+ mode: "file",
617
+ kind: "user",
618
+ path: ".gemini/extensions/webscraping-ai/GEMINI.md"
619
+ },
423
620
  // GitHub Copilot: project-scope custom instructions file. Single file.
424
- { id: "copilot", label: "GitHub Copilot", mode: "file", kind: "project", path: ".github/copilot-instructions.md" },
621
+ {
622
+ id: "copilot",
623
+ label: "GitHub Copilot",
624
+ mode: "file",
625
+ kind: "project",
626
+ path: ".github/copilot-instructions.md"
627
+ },
425
628
  // Augment: project-scope rules file.
426
- { id: "augment", label: "Augment", mode: "file", kind: "project", path: ".augment/rules/webscraping-ai.md" },
629
+ {
630
+ id: "augment",
631
+ label: "Augment",
632
+ mode: "file",
633
+ kind: "project",
634
+ path: ".augment/rules/webscraping-ai.md"
635
+ },
427
636
  // Factory: project-scope rules file.
428
- { id: "factory", label: "Factory", mode: "file", kind: "project", path: ".factory/rules/webscraping-ai.md" }
637
+ {
638
+ id: "factory",
639
+ label: "Factory",
640
+ mode: "file",
641
+ kind: "project",
642
+ path: ".factory/rules/webscraping-ai.md"
643
+ }
429
644
  ];
430
645
  function editorById(id) {
431
646
  return EDITORS.find((e) => e.id === id);
@@ -468,7 +683,7 @@ async function copyDirectory(source, destination) {
468
683
 
469
684
  // src/commands/setupSkill.ts
470
685
  function setupCommand() {
471
- const cmd = new Command8("setup").description("Install the AI agent skill into your editor(s)");
686
+ const cmd = new Command9("setup").description("Install the AI agent skill into your editor(s)");
472
687
  cmd.command("skill").description("Install the WebScraping.AI skill into one or more AI coding editors").option(
473
688
  "-e, --editor <id>",
474
689
  `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(", ")}`,
@@ -511,76 +726,34 @@ function resolveSelection(opts) {
511
726
  }
512
727
 
513
728
  // src/commands/text.ts
514
- import { Command as Command9, Option as Option5 } from "commander";
729
+ import { Command as Command10, Option as Option6 } from "commander";
515
730
  function textCommand() {
516
- const cmd = new Command9("text").description("Fetch the visible text of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
517
- new Option5("--text-format <fmt>", "text response format").choices(["plain", "json", "xml"])
731
+ const cmd = new Command10("text").description("Fetch the visible text of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
732
+ new Option6("--text-format <fmt>", "text response format").choices(["plain", "json", "xml"])
518
733
  ).option("--return-links", "include page links (text-format=json only)");
519
734
  addCommonScrapeOptions(cmd);
520
735
  cmd.action(async (url, opts) => {
521
736
  const apiKey = await resolveApiKey({ flag: opts.apiKey });
522
737
  const headers = await parseHeaders(opts.headers);
523
738
  const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
739
+ const write = createEmitter(opts);
524
740
  for await (const target of urlIterator(url)) {
525
- const sdkOptions = buildTextOptions(target, opts, { headers }, {
526
- textFormat: opts.textFormat,
527
- returnLinks: opts.returnLinks
528
- });
741
+ const sdkOptions = buildTextOptions(
742
+ target,
743
+ opts,
744
+ { headers },
745
+ {
746
+ textFormat: opts.textFormat,
747
+ returnLinks: opts.returnLinks
748
+ }
749
+ );
529
750
  const result = await client.text(sdkOptions);
530
- await emit(result, opts);
751
+ await write(result);
531
752
  }
532
753
  });
533
754
  return cmd;
534
755
  }
535
756
 
536
- // src/lib/errors.ts
537
- import {
538
- APIConnectionError,
539
- APIError,
540
- APITimeoutError,
541
- AuthenticationError,
542
- BadRequestError,
543
- GatewayTimeoutError,
544
- PaymentRequiredError,
545
- RateLimitError,
546
- ServerError,
547
- WebScrapingAIError
548
- } from "webscraping-ai";
549
- var EXIT_CODES = {
550
- ok: 0,
551
- generic: 1,
552
- usage: 2,
553
- badRequest: 3,
554
- auth: 4,
555
- paymentRequired: 5,
556
- rateLimit: 6,
557
- serverError: 7,
558
- timeout: 8,
559
- connection: 9
560
- };
561
- function exitCodeFor(err) {
562
- if (err instanceof BadRequestError) return EXIT_CODES.badRequest;
563
- if (err instanceof AuthenticationError) return EXIT_CODES.auth;
564
- if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;
565
- if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;
566
- if (err instanceof ServerError) return EXIT_CODES.serverError;
567
- if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;
568
- if (err instanceof APITimeoutError) return EXIT_CODES.timeout;
569
- if (err instanceof APIConnectionError) return EXIT_CODES.connection;
570
- if (err instanceof APIError) return EXIT_CODES.serverError;
571
- if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;
572
- return EXIT_CODES.generic;
573
- }
574
- function formatError(err) {
575
- if (err instanceof APIError) {
576
- const parts = [err.message];
577
- if (err.status) parts.push(`(HTTP ${err.status})`);
578
- return parts.join(" ");
579
- }
580
- if (err instanceof Error) return err.message;
581
- return String(err);
582
- }
583
-
584
757
  // src/cli.ts
585
758
  function packageVersion() {
586
759
  const here = dirname3(fileURLToPath2(import.meta.url));
@@ -589,9 +762,9 @@ function packageVersion() {
589
762
  return raw.version;
590
763
  }
591
764
  function buildProgram() {
592
- const program = new Command10();
765
+ const program = new Command11();
593
766
  program.name("webscraping-ai").description(
594
- "Official CLI for the WebScraping.AI API. Fetch pages, ask AI, extract structured data.\nSet $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first."
767
+ "Official CLI for the WebScraping.AI API. Fetch pages, search Google, ask AI, extract structured data.\nSet $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first."
595
768
  ).version(packageVersion(), "-v, --version", "output the version number").showHelpAfterError();
596
769
  program.addCommand(htmlCommand());
597
770
  program.addCommand(textCommand());
@@ -599,6 +772,7 @@ function buildProgram() {
599
772
  program.addCommand(selectedMultipleCommand());
600
773
  program.addCommand(askCommand());
601
774
  program.addCommand(extractCommand());
775
+ program.addCommand(serpCommand());
602
776
  program.addCommand(accountCommand());
603
777
  program.addCommand(authCommand());
604
778
  program.addCommand(setupCommand());
package/dist/cli.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/cli.ts","../src/commands/account.ts","../src/lib/config.ts","../src/lib/output.ts","../src/lib/sdk.ts","../src/commands/ask.ts","../src/lib/buildRequest.ts","../src/lib/options.ts","../src/lib/stdin.ts","../src/commands/auth.ts","../src/commands/extract.ts","../src/commands/html.ts","../src/commands/selected.ts","../src/commands/selectedMultiple.ts","../src/commands/setupSkill.ts","../src/lib/skill.ts","../src/commands/text.ts","../src/lib/errors.ts"],"sourcesContent":["/**\n * `webscraping-ai` / `wsai` CLI entry point.\n *\n * Wires every subcommand into a single commander root. Errors thrown from\n * subcommand actions land in the top-level `.catch`, which maps SDK\n * exception classes to stable exit codes (see `lib/errors.ts`).\n */\n\nimport { Command } from 'commander';\nimport { readFileSync } from 'node:fs';\nimport { dirname, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nimport { accountCommand } from './commands/account.js';\nimport { askCommand } from './commands/ask.js';\nimport { authCommand } from './commands/auth.js';\nimport { extractCommand } from './commands/extract.js';\nimport { htmlCommand } from './commands/html.js';\nimport { selectedCommand } from './commands/selected.js';\nimport { selectedMultipleCommand } from './commands/selectedMultiple.js';\nimport { setupCommand } from './commands/setupSkill.js';\nimport { textCommand } from './commands/text.js';\nimport { exitCodeFor, formatError } from './lib/errors.js';\n\nfunction packageVersion(): string {\n // Read package.json at runtime so we don't need a code-gen step. The dist\n // bundle sits at `dist/cli.js`, so package.json is one directory up.\n const here = dirname(fileURLToPath(import.meta.url));\n const pkgPath = resolve(here, '..', 'package.json');\n const raw = JSON.parse(readFileSync(pkgPath, 'utf8')) as { version: string };\n return raw.version;\n}\n\nexport function buildProgram(): Command {\n const program = new Command();\n program\n .name('webscraping-ai')\n .description(\n 'Official CLI for the WebScraping.AI API. Fetch pages, ask AI, extract structured data.\\n' +\n 'Set $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first.',\n )\n .version(packageVersion(), '-v, --version', 'output the version number')\n .showHelpAfterError();\n\n program.addCommand(htmlCommand());\n program.addCommand(textCommand());\n program.addCommand(selectedCommand());\n program.addCommand(selectedMultipleCommand());\n program.addCommand(askCommand());\n program.addCommand(extractCommand());\n program.addCommand(accountCommand());\n program.addCommand(authCommand());\n program.addCommand(setupCommand());\n\n return program;\n}\n\nasync function main(argv: string[]): Promise<void> {\n const program = buildProgram();\n try {\n await program.parseAsync(argv);\n } catch (err) {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n }\n}\n\n// Only execute when run as a script. When imported (e.g. from tests),\n// `buildProgram` is the public surface and `main` stays dormant.\nconst entryPath = process.argv[1] ? fileURLToPath(import.meta.url) : '';\nif (entryPath && process.argv[1] === entryPath) {\n main(process.argv).catch((err: unknown) => {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n });\n}\n","import { Command } from 'commander';\n\nimport { resolveApiKey } from '../lib/config.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface AccountFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nexport function accountCommand(): Command {\n return new Command('account')\n .description('Show account quota and usage information')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .action(async (opts: AccountFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const result = await client.account();\n await emit(result, opts);\n });\n}\n","/**\n * API key resolution and persisted config.\n *\n * Resolution order (first non-empty wins):\n * 1. `--api-key` flag (handled by the caller, passed in)\n * 2. `WEBSCRAPING_AI_API_KEY` env var\n * 3. `~/.config/webscraping-ai/config.json` (or `$XDG_CONFIG_HOME/...`)\n *\n * The config file is plain JSON with `{ \"apiKey\": \"...\" }`. We chmod it to\n * 0600 so it's not world-readable.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join } from 'node:path';\n\nconst CONFIG_FILENAME = 'config.json';\n\nexport function configDir(): string {\n const xdg = process.env.XDG_CONFIG_HOME;\n if (xdg && xdg.trim() !== '') return join(xdg, 'webscraping-ai');\n return join(homedir(), '.config', 'webscraping-ai');\n}\n\nexport function configPath(): string {\n return join(configDir(), CONFIG_FILENAME);\n}\n\nexport interface PersistedConfig {\n apiKey?: string;\n}\n\nexport async function readPersistedConfig(): Promise<PersistedConfig> {\n try {\n const text = await fs.readFile(configPath(), 'utf8');\n const parsed = JSON.parse(text) as unknown;\n if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {\n return parsed as PersistedConfig;\n }\n return {};\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return {};\n throw err;\n }\n}\n\nexport async function writePersistedConfig(config: PersistedConfig): Promise<void> {\n const path = configPath();\n await fs.mkdir(dirname(path), { recursive: true });\n await fs.writeFile(path, JSON.stringify(config, null, 2) + '\\n', { mode: 0o600 });\n // `mode` above only applies when the file is freshly created; an existing\n // file keeps its old (possibly world-readable) perms. Enforce 0600\n // unconditionally — chmod on a just-written file is cheap and idempotent.\n // No-op on Windows, but harmless.\n await fs.chmod(path, 0o600);\n}\n\nexport async function clearPersistedConfig(): Promise<boolean> {\n try {\n await fs.unlink(configPath());\n return true;\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return false;\n throw err;\n }\n}\n\nexport interface ResolveApiKeyOptions {\n /** `--api-key` flag value, if the user passed one. */\n flag?: string;\n}\n\nexport async function resolveApiKey(opts: ResolveApiKeyOptions = {}): Promise<string> {\n if (opts.flag && opts.flag.trim() !== '') return opts.flag.trim();\n\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') return env.trim();\n\n const persisted = await readPersistedConfig();\n if (persisted.apiKey && persisted.apiKey.trim() !== '') return persisted.apiKey.trim();\n\n throw new Error(\n 'No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`.',\n );\n}\n\nexport function maskApiKey(key: string): string {\n if (key.length <= 8) return '*'.repeat(key.length);\n return `${key.slice(0, 4)}${'*'.repeat(key.length - 8)}${key.slice(-4)}`;\n}\n","/**\n * Output handling for command results.\n *\n * All command handlers funnel through `emit()`. The result type drives format:\n * - strings → written verbatim\n * - objects/arrays → JSON.stringified (pretty by default; `--pretty` is the\n * default for terminals, single-line for pipes/redirects)\n *\n * `--output FILE` writes to disk instead of stdout.\n */\n\nimport { promises as fs } from 'node:fs';\n\nexport interface EmitOptions {\n output?: string;\n pretty?: boolean;\n}\n\nexport async function emit(value: unknown, opts: EmitOptions = {}): Promise<void> {\n const text = stringify(value, opts);\n\n if (opts.output && opts.output.trim() !== '') {\n await fs.writeFile(opts.output, text + (text.endsWith('\\n') ? '' : '\\n'));\n return;\n }\n\n process.stdout.write(text);\n if (!text.endsWith('\\n')) process.stdout.write('\\n');\n}\n\nexport function stringify(value: unknown, opts: EmitOptions = {}): string {\n if (typeof value === 'string') return value;\n\n const pretty = opts.pretty ?? process.stdout.isTTY ?? false;\n return JSON.stringify(value, null, pretty ? 2 : 0);\n}\n","/**\n * SDK adapter.\n *\n * Wraps the published `webscraping-ai` Node SDK. The CLI sends a\n * `from_cli=true` analytics flag on every request (mirrors `from_n8n` and\n * `from_mcp_server` used by the n8n node and MCP server). The SDK's typed\n * options don't accept arbitrary keys, so we go through a single cast point\n * here rather than scattering casts across each command.\n */\n\nimport { WebScrapingAI } from 'webscraping-ai';\nimport type {\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nexport interface CreateClientOptions {\n apiKey: string;\n /** Per-request timeout in ms. Maps to the SDK's `timeoutMs`. */\n requestTimeoutMs?: number;\n}\n\n/**\n * Returns a `WebScrapingAI` instance with the CLI's analytics tag baked into\n * every endpoint method. The methods preserve the SDK's public signatures.\n */\nexport function createClient(options: CreateClientOptions): WebScrapingAI {\n const inner = new WebScrapingAI({\n apiKey: options.apiKey,\n timeoutMs: options.requestTimeoutMs,\n });\n\n // Wrap each endpoint method so it injects the CLI analytics tag. We splice\n // the tag in at this seam rather than at every callsite. The cast is local;\n // the SDK passes unknown query params through to the wire.\n const tag = { from_cli: true } as unknown as Record<string, never>;\n\n return new Proxy(inner, {\n get(target, prop, receiver) {\n const value = Reflect.get(target, prop, receiver);\n if (typeof value !== 'function') return value;\n switch (prop) {\n case 'html':\n return (o: HtmlOptions) => target.html({ ...o, ...tag });\n case 'text':\n return (o: TextOptions) => target.text({ ...o, ...tag });\n case 'selected':\n return (o: SelectedOptions) => target.selected({ ...o, ...tag });\n case 'selectedMultiple':\n return (o: SelectedMultipleOptions) => target.selectedMultiple({ ...o, ...tag });\n case 'question':\n return (o: QuestionOptions) => target.question({ ...o, ...tag });\n case 'fields':\n return (o: FieldsOptions) => target.fields({ ...o, ...tag });\n case 'account':\n // `account` takes no options; the SDK builds the request itself, so\n // there's no seam here for the analytics tag — accepted limitation.\n return value.bind(target);\n default:\n return typeof value === 'function' ? value.bind(target) : value;\n }\n },\n });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildQuestionOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface AskFlags extends CommonRawFlags {\n question?: string;\n}\n\nexport function askCommand(): Command {\n const cmd = new Command('ask')\n .description('Ask an AI question about a page (wraps /ai/question)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption('-q, --question <text>', 'the question / instructions for the model')\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: AskFlags) => {\n if (!opts.question) throw new Error('--question is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });\n const result = await client.question(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n","/**\n * Pure builder that maps `--flag` values into SDK option objects.\n *\n * Extracted as a separate function so unit tests can pin the mapping without\n * spinning up commander or making HTTP calls. Each command handler reads its\n * own commander options, awaits `parseHeaders` / `parseFields`, then calls\n * the matching `build*` function here.\n */\n\nimport type {\n CommonRequestOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nimport type { CommonRawFlags } from './options.js';\n\nexport interface ResolvedCommonOptions {\n headers?: Record<string, string>;\n}\n\nexport function buildCommonOptions(\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions = {},\n): CommonRequestOptions {\n const out: CommonRequestOptions = {};\n // `flags.js` is `false` when `--no-js`, `true` when `--js`, `undefined`\n // when neither passed. Pass through only when explicit, so the API default\n // applies otherwise.\n if (flags.js !== undefined) out.js = flags.js;\n if (flags.proxy !== undefined) out.proxy = flags.proxy;\n if (flags.country !== undefined) out.country = flags.country;\n if (flags.device !== undefined) out.device = flags.device;\n if (flags.timeout !== undefined) out.timeout = flags.timeout;\n if (flags.jsTimeout !== undefined) out.js_timeout = flags.jsTimeout;\n if (flags.waitFor !== undefined) out.wait_for = flags.waitFor;\n if (flags.jsScript !== undefined) out.js_script = flags.jsScript;\n if (flags.customProxy !== undefined) out.custom_proxy = flags.customProxy;\n if (flags.errorOn404) out.error_on_404 = true;\n if (flags.errorOnRedirect) out.error_on_redirect = true;\n if (resolved.headers) out.headers = resolved.headers;\n return out;\n}\n\nexport interface HtmlExtras {\n returnScriptResult?: boolean;\n}\n\nexport function buildHtmlOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: HtmlExtras = {},\n): HtmlOptions {\n const out: HtmlOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n if (extras.returnScriptResult) out.return_script_result = true;\n return out;\n}\n\nexport interface TextExtras {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function buildTextOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: TextExtras = {},\n): TextOptions {\n const out: TextOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (extras.textFormat) out.text_format = extras.textFormat;\n if (extras.returnLinks) out.return_links = true;\n return out;\n}\n\nexport function buildSelectedOptions(\n url: string,\n selector: string | undefined,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedOptions {\n // `selector` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when provided. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selector !== undefined) out.selector = selector;\n if (flags.format) out.format = flags.format;\n return out as SelectedOptions;\n}\n\nexport function buildSelectedMultipleOptions(\n url: string,\n selectors: readonly string[],\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedMultipleOptions {\n // `selectors` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when non-empty. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedMultipleOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selectors.length > 0) out.selectors = selectors;\n return out as SelectedMultipleOptions;\n}\n\nexport function buildQuestionOptions(\n url: string,\n question: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): QuestionOptions {\n const out: QuestionOptions = { url, question, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n return out;\n}\n\nexport function buildFieldsOptions(\n url: string,\n fields: Record<string, string>,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): FieldsOptions {\n return { url, fields, ...buildCommonOptions(flags, resolved) };\n}\n","/**\n * Shared commander options and the parsers/expanders behind them.\n *\n * All scrape endpoints share the same nine \"common\" flags. We define them in\n * one place and apply via `addCommonScrapeOptions(cmd)` so the surface stays\n * consistent and any change lands everywhere at once.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { Option } from 'commander';\nimport type { Command } from 'commander';\n\nexport interface CommonRawFlags {\n apiKey?: string;\n output?: string;\n format?: 'text' | 'json';\n pretty?: boolean;\n js?: boolean;\n proxy?: 'datacenter' | 'residential' | 'stealth';\n country?: string;\n device?: 'desktop' | 'mobile' | 'tablet';\n timeout?: number;\n jsTimeout?: number;\n waitFor?: string;\n headers?: string;\n jsScript?: string;\n customProxy?: string;\n errorOn404?: boolean;\n errorOnRedirect?: boolean;\n}\n\n/**\n * Add the flags every scrape-endpoint command needs. Adding here keeps the\n * surface aligned across `html`, `text`, `selected`, etc.\n */\nexport function addCommonScrapeOptions(cmd: Command): Command {\n return cmd\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .addOption(\n new Option('--no-pretty', 'force single-line JSON output').conflicts('pretty'),\n )\n .option('--js', 'execute JS via a headless browser (default: enabled)')\n .option('--no-js', 'disable JS execution (faster, cheaper)')\n .addOption(\n new Option('--proxy <type>', 'proxy pool').choices([\n 'datacenter',\n 'residential',\n 'stealth',\n ]),\n )\n .option('--country <code>', 'two-letter proxy country code, e.g. us, gb, de')\n .addOption(\n new Option('--device <type>', 'device emulation').choices(['desktop', 'mobile', 'tablet']),\n )\n .option('--timeout <ms>', 'page-retrieval timeout in ms (max 30000)', integerParser)\n .option('--js-timeout <ms>', 'JS rendering time in ms after load (max 20000)', integerParser)\n .option('--wait-for <selector>', 'CSS selector to wait for before returning content')\n .option(\n '--headers <json>',\n 'HTTP headers to forward to the target page; JSON object or @file.json',\n )\n .option('--js-script <code>', 'custom JS to evaluate on the target page')\n .option('--custom-proxy <url>', 'custom proxy URL: http://user:pass@host:port')\n .option('--error-on-404', 'return error on 404')\n .option('--error-on-redirect', 'return error on redirect');\n}\n\nexport function integerParser(value: string): number {\n const n = Number.parseInt(value, 10);\n if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);\n return n;\n}\n\n/**\n * Parse a `--headers` argument. Accepts a JSON object literal or `@path` to\n * load JSON from a file. Returns `undefined` if the input is empty.\n */\nexport async function parseHeaders(input: string | undefined): Promise<Record<string, string> | undefined> {\n if (!input || input.trim() === '') return undefined;\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n return parseJsonObject(text, 'headers');\n}\n\n/**\n * Parse a `--fields` argument for `extract`. Accepts JSON object literal or\n * `@path` to a file.\n */\nexport async function parseFields(input: string): Promise<Record<string, string>> {\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n const parsed = parseJsonObject(text, 'fields');\n if (!parsed || Object.keys(parsed).length === 0) {\n throw new Error('fields must be a non-empty JSON object of { fieldName: description }');\n }\n return parsed;\n}\n\nfunction parseJsonObject(text: string, label: string): Record<string, string> {\n let parsed: unknown;\n try {\n parsed = JSON.parse(text);\n } catch (err) {\n throw new Error(`${label}: invalid JSON — ${(err as Error).message}`);\n }\n if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {\n throw new Error(`${label} must be a JSON object`);\n }\n const out: Record<string, string> = {};\n for (const [k, v] of Object.entries(parsed as Record<string, unknown>)) {\n if (typeof v !== 'string') throw new Error(`${label}.${k}: value must be a string`);\n out[k] = v;\n }\n return out;\n}\n","/**\n * Stdin batching helpers.\n *\n * When a URL argument is `-`, the command reads one URL per line from stdin\n * and runs the operation for each. Empty lines and lines starting with `#`\n * are skipped so users can comment URL lists.\n */\n\nimport readline from 'node:readline';\n\nexport function urlIsStdin(url: string): boolean {\n return url === '-';\n}\n\nexport async function* readUrlsFromStdin(): AsyncIterable<string> {\n const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });\n for await (const raw of rl) {\n const line = raw.trim();\n if (line === '' || line.startsWith('#')) continue;\n yield line;\n }\n}\n\n/**\n * Iterate URLs from either a single CLI arg or from stdin (if the arg is `-`).\n */\nexport async function* urlIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readUrlsFromStdin();\n return;\n }\n yield arg;\n}\n","import { Command } from 'commander';\n\nimport {\n clearPersistedConfig,\n configPath,\n maskApiKey,\n readPersistedConfig,\n writePersistedConfig,\n} from '../lib/config.js';\n\nexport function authCommand(): Command {\n const cmd = new Command('auth').description('Manage your stored API key');\n\n cmd\n .command('set <key>')\n .description(`Save an API key to ${configPath()}`)\n .action(async (key: string) => {\n if (!key || key.trim() === '') throw new Error('API key is empty');\n await writePersistedConfig({ apiKey: key.trim() });\n process.stdout.write(`API key saved to ${configPath()}\\n`);\n });\n\n cmd\n .command('show')\n .description('Display the currently stored API key (masked)')\n .action(async () => {\n const persisted = await readPersistedConfig();\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') {\n process.stdout.write(`Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}\\n`);\n }\n if (persisted.apiKey && persisted.apiKey.trim() !== '') {\n process.stdout.write(\n `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}\\n`,\n );\n }\n if (!env && !persisted.apiKey) {\n process.stdout.write('No API key configured. Run `webscraping-ai auth set <key>`.\\n');\n }\n });\n\n cmd\n .command('clear')\n .description('Remove the stored API key file')\n .action(async () => {\n const removed = await clearPersistedConfig();\n process.stdout.write(removed ? `Removed ${configPath()}\\n` : 'No stored API key to remove.\\n');\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildFieldsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseFields, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface ExtractFlags extends CommonRawFlags {\n fields?: string;\n}\n\nexport function extractCommand(): Command {\n const cmd = new Command('extract')\n .description('Extract structured fields from a page using AI (wraps /ai/fields)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption(\n '-f, --fields <json>',\n 'fields to extract — JSON object {name: description} or @file.json',\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: ExtractFlags) => {\n if (!opts.fields) throw new Error('--fields is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const fields = await parseFields(opts.fields);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });\n const result = await client.fields(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildHtmlOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface HtmlFlags extends CommonRawFlags {\n returnScriptResult?: boolean;\n}\n\nexport function htmlCommand(): Command {\n const cmd = new Command('html')\n .description('Fetch the full HTML of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(new Option('--return-script-result', 'return the result of --js-script instead of HTML'))\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: HtmlFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildHtmlOptions(target, opts, { headers }, {\n returnScriptResult: opts.returnScriptResult,\n });\n const result = await client.html(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSelectedOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedFlags extends CommonRawFlags {\n selector?: string;\n}\n\nexport function selectedCommand(): Command {\n const cmd = new Command('selected')\n .description('Fetch HTML of one CSS-selected page area')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector for the target element(s); omit for whole-page HTML',\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });\n const result = await client.selected(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildSelectedMultipleOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedMultipleFlags extends CommonRawFlags {\n selector?: string[];\n}\n\nexport function selectedMultipleCommand(): Command {\n const cmd = new Command('selected-multiple')\n .description('Fetch HTML of multiple CSS-selected page areas')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector — pass multiple times for multiple selectors',\n collectMulti,\n [] as string[],\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedMultipleFlags) => {\n // `--selector` is optional: omitting it returns whole-page HTML.\n const selectors = opts.selector ?? [];\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });\n const result = await client.selectedMultiple(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n\nfunction collectMulti(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n","import { Command } from 'commander';\n\nimport { EDITORS, editorById, installSkill, resolveTarget } from '../lib/skill.js';\nimport type { EditorAdapter } from '../lib/skill.js';\n\ninterface SetupSkillFlags {\n editor?: string[];\n all?: boolean;\n force?: boolean;\n}\n\nexport function setupCommand(): Command {\n const cmd = new Command('setup').description('Install the AI agent skill into your editor(s)');\n\n cmd\n .command('skill')\n .description('Install the WebScraping.AI skill into one or more AI coding editors')\n .option(\n '-e, --editor <id>',\n `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(', ')}`,\n collect,\n [] as string[],\n )\n .option('-a, --all', 'install into every supported editor')\n .option('-f, --force', 'overwrite existing skill files')\n .action(async (opts: SetupSkillFlags) => {\n const selected = resolveSelection(opts);\n if (selected.length === 0) {\n throw new Error(\n 'no editor selected — pass --all or --editor <id> (e.g. --editor claude-code)',\n );\n }\n\n for (const editor of selected) {\n const target = resolveTarget(editor);\n const result = await installSkill(target, { force: opts.force });\n process.stdout.write(`${editor.label}: ${result.status} → ${target.destination}\\n`);\n }\n });\n\n cmd\n .command('list', { hidden: false })\n .description('List supported editors for `setup skill`')\n .action(() => {\n for (const e of EDITORS) {\n const target = resolveTarget(e);\n process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}\\n`);\n }\n });\n\n return cmd;\n}\n\nfunction collect(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n\nfunction resolveSelection(opts: SetupSkillFlags): EditorAdapter[] {\n if (opts.all) return [...EDITORS];\n const ids = opts.editor ?? [];\n return ids.map((id) => {\n const editor = editorById(id);\n if (!editor) throw new Error(`unknown editor: ${id}`);\n return editor;\n });\n}\n","/**\n * Multi-editor skill installer.\n *\n * The canonical skill lives in this package at\n * `plugins/webscraping-ai/skills/webscraping-ai/SKILL.md`. Each supported\n * editor reads skill content from a different location and may want either\n * a single Markdown file or a whole skill directory. The `editors` registry\n * captures those differences so `setup skill` is a small driver over it.\n *\n * `mode: 'dir'` copies the entire skill directory (Claude Code-shaped\n * editors that scan a folder per skill). `mode: 'file'` writes the SKILL.md\n * contents to a single file (editors that read flat instructions, like\n * GitHub Copilot's `.github/copilot-instructions.md`).\n *\n * `kind: 'user'` writes to `$HOME`; `kind: 'project'` writes to the current\n * working directory. We prefer user-scope where the editor supports it so\n * the skill becomes globally available without polluting individual repos.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nexport type EditorMode = 'dir' | 'file';\nexport type EditorKind = 'user' | 'project';\n\nexport interface EditorAdapter {\n /** Stable identifier used by `--editor <name>`. */\n id: string;\n /** Human-readable label printed in summaries. */\n label: string;\n mode: EditorMode;\n kind: EditorKind;\n /**\n * Where the skill ends up. For `mode: 'dir'`, the skill directory copies\n * to `{root}/{path}`. For `mode: 'file'`, the SKILL.md contents write to\n * `{root}/{path}` (i.e. `path` ends in a filename).\n *\n * `root` is `$HOME` (user) or CWD (project).\n */\n path: string;\n}\n\n/**\n * Registry of supported editors. Order matters: it's also the order the\n * `setup skill` summary lists them in.\n */\nexport const EDITORS: readonly EditorAdapter[] = [\n // Claude Code: user-scope skills live under ~/.claude/skills/<name>/\n { id: 'claude-code', label: 'Claude Code', mode: 'dir', kind: 'user', path: '.claude/skills/webscraping-ai' },\n // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill\n // dir, so this is the editor's \"install\" surface.\n { id: 'cursor', label: 'Cursor', mode: 'file', kind: 'project', path: '.cursor/rules/webscraping-ai.mdc' },\n // Windsurf: project-scope rules. Same shape as Cursor.\n { id: 'windsurf', label: 'Windsurf', mode: 'file', kind: 'project', path: '.windsurf/rules/webscraping-ai.md' },\n // Kiro: user-scope steering directory.\n { id: 'kiro', label: 'Kiro', mode: 'dir', kind: 'user', path: '.kiro/steering/webscraping-ai' },\n // OpenCode: project-scope skills (mirrors Claude shape).\n { id: 'opencode', label: 'OpenCode', mode: 'dir', kind: 'project', path: '.opencode/skills/webscraping-ai' },\n // Gemini CLI: user-scope extension dir with a GEMINI.md.\n { id: 'gemini', label: 'Gemini CLI', mode: 'file', kind: 'user', path: '.gemini/extensions/webscraping-ai/GEMINI.md' },\n // GitHub Copilot: project-scope custom instructions file. Single file.\n { id: 'copilot', label: 'GitHub Copilot', mode: 'file', kind: 'project', path: '.github/copilot-instructions.md' },\n // Augment: project-scope rules file.\n { id: 'augment', label: 'Augment', mode: 'file', kind: 'project', path: '.augment/rules/webscraping-ai.md' },\n // Factory: project-scope rules file.\n { id: 'factory', label: 'Factory', mode: 'file', kind: 'project', path: '.factory/rules/webscraping-ai.md' },\n] as const;\n\nexport function editorById(id: string): EditorAdapter | undefined {\n return EDITORS.find((e) => e.id === id);\n}\n\n/**\n * Where this package keeps the canonical skill. Resolved relative to the\n * compiled `dist/cli.js` so it works after `npm install -g`.\n */\nexport function bundledSkillDir(): string {\n const here = dirname(fileURLToPath(import.meta.url));\n // dist/ sits at the package root, so the plugins dir is two levels up\n // from any compiled file: dist/cli.js → package root → plugins/...\n return resolve(here, '..', 'plugins', 'webscraping-ai', 'skills', 'webscraping-ai');\n}\n\nexport interface InstallTarget {\n editor: EditorAdapter;\n /** Absolute destination path (file or directory depending on `mode`). */\n destination: string;\n}\n\nexport interface ResolveOptions {\n cwd?: string;\n home?: string;\n}\n\nexport function resolveTarget(editor: EditorAdapter, opts: ResolveOptions = {}): InstallTarget {\n const root = editor.kind === 'user' ? (opts.home ?? homedir()) : (opts.cwd ?? process.cwd());\n return { editor, destination: join(root, editor.path) };\n}\n\nexport interface InstallOptions {\n /** If true, overwrite existing files/dirs. */\n force?: boolean;\n /** Override skill source dir (for tests). */\n sourceDir?: string;\n}\n\nexport type InstallStatus = 'installed' | 'skipped-exists' | 'overwritten';\n\nexport interface InstallResult {\n target: InstallTarget;\n status: InstallStatus;\n}\n\nexport async function installSkill(\n target: InstallTarget,\n opts: InstallOptions = {},\n): Promise<InstallResult> {\n const source = opts.sourceDir ?? bundledSkillDir();\n const exists = await pathExists(target.destination);\n\n if (exists && !opts.force) {\n return { target, status: 'skipped-exists' };\n }\n\n if (target.editor.mode === 'dir') {\n if (exists) await fs.rm(target.destination, { recursive: true, force: true });\n await copyDirectory(source, target.destination);\n } else {\n const skillBody = await fs.readFile(join(source, 'SKILL.md'), 'utf8');\n await fs.mkdir(dirname(target.destination), { recursive: true });\n await fs.writeFile(target.destination, skillBody);\n }\n\n return { target, status: exists ? 'overwritten' : 'installed' };\n}\n\nasync function pathExists(path: string): Promise<boolean> {\n try {\n await fs.access(path);\n return true;\n } catch {\n return false;\n }\n}\n\nasync function copyDirectory(source: string, destination: string): Promise<void> {\n // `fs.cp` with `recursive` exists on Node 16.7+; we target Node 20.\n await fs.cp(source, destination, { recursive: true, force: true });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildTextOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface TextFlags extends CommonRawFlags {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function textCommand(): Command {\n const cmd = new Command('text')\n .description('Fetch the visible text of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--text-format <fmt>', 'text response format').choices(['plain', 'json', 'xml']),\n )\n .option('--return-links', 'include page links (text-format=json only)');\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: TextFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildTextOptions(target, opts, { headers }, {\n textFormat: opts.textFormat,\n returnLinks: opts.returnLinks,\n });\n const result = await client.text(sdkOptions);\n await emit(result, opts);\n }\n });\n\n return cmd;\n}\n","/**\n * Exit-code mapping for SDK errors.\n *\n * The SDK throws typed subclasses of `WebScrapingAIError`. Map each to a\n * stable exit code so shell scripts can branch on the failure mode without\n * parsing stderr.\n */\n\nimport {\n APIConnectionError,\n APIError,\n APITimeoutError,\n AuthenticationError,\n BadRequestError,\n GatewayTimeoutError,\n PaymentRequiredError,\n RateLimitError,\n ServerError,\n WebScrapingAIError,\n} from 'webscraping-ai';\n\nexport const EXIT_CODES = {\n ok: 0,\n generic: 1,\n usage: 2,\n badRequest: 3,\n auth: 4,\n paymentRequired: 5,\n rateLimit: 6,\n serverError: 7,\n timeout: 8,\n connection: 9,\n} as const;\n\nexport function exitCodeFor(err: unknown): number {\n if (err instanceof BadRequestError) return EXIT_CODES.badRequest;\n if (err instanceof AuthenticationError) return EXIT_CODES.auth;\n if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;\n if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;\n if (err instanceof ServerError) return EXIT_CODES.serverError;\n if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APITimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APIConnectionError) return EXIT_CODES.connection;\n if (err instanceof APIError) return EXIT_CODES.serverError;\n if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;\n return EXIT_CODES.generic;\n}\n\nexport function formatError(err: unknown): string {\n if (err instanceof APIError) {\n const parts = [err.message];\n if (err.status) parts.push(`(HTTP ${err.status})`);\n return parts.join(' ');\n }\n if (err instanceof Error) return err.message;\n return String(err);\n}\n"],"mappings":";;;AAQA,SAAS,WAAAA,iBAAe;AACxB,SAAS,oBAAoB;AAC7B,SAAS,WAAAC,UAAS,WAAAC,gBAAe;AACjC,SAAS,iBAAAC,sBAAqB;;;ACX9B,SAAS,eAAe;;;ACYxB,SAAS,YAAY,UAAU;AAC/B,SAAS,eAAe;AACxB,SAAS,SAAS,YAAY;AAE9B,IAAM,kBAAkB;AAEjB,SAAS,YAAoB;AAClC,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,gBAAgB;AAC/D,SAAO,KAAK,QAAQ,GAAG,WAAW,gBAAgB;AACpD;AAEO,SAAS,aAAqB;AACnC,SAAO,KAAK,UAAU,GAAG,eAAe;AAC1C;AAMA,eAAsB,sBAAgD;AACpE,MAAI;AACF,UAAM,OAAO,MAAM,GAAG,SAAS,WAAW,GAAG,MAAM;AACnD,UAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,QAAI,UAAU,OAAO,WAAW,YAAY,CAAC,MAAM,QAAQ,MAAM,GAAG;AAClE,aAAO;AAAA,IACT;AACA,WAAO,CAAC;AAAA,EACV,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO,CAAC;AAC9D,UAAM;AAAA,EACR;AACF;AAEA,eAAsB,qBAAqB,QAAwC;AACjF,QAAM,OAAO,WAAW;AACxB,QAAM,GAAG,MAAM,QAAQ,IAAI,GAAG,EAAE,WAAW,KAAK,CAAC;AACjD,QAAM,GAAG,UAAU,MAAM,KAAK,UAAU,QAAQ,MAAM,CAAC,IAAI,MAAM,EAAE,MAAM,IAAM,CAAC;AAKhF,QAAM,GAAG,MAAM,MAAM,GAAK;AAC5B;AAEA,eAAsB,uBAAyC;AAC7D,MAAI;AACF,UAAM,GAAG,OAAO,WAAW,CAAC;AAC5B,WAAO;AAAA,EACT,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO;AAC7D,UAAM;AAAA,EACR;AACF;AAOA,eAAsB,cAAc,OAA6B,CAAC,GAAoB;AACpF,MAAI,KAAK,QAAQ,KAAK,KAAK,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,KAAK;AAEhE,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,IAAI,KAAK;AAE9C,QAAM,YAAY,MAAM,oBAAoB;AAC5C,MAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,GAAI,QAAO,UAAU,OAAO,KAAK;AAErF,QAAM,IAAI;AAAA,IACR;AAAA,EACF;AACF;AAEO,SAAS,WAAW,KAAqB;AAC9C,MAAI,IAAI,UAAU,EAAG,QAAO,IAAI,OAAO,IAAI,MAAM;AACjD,SAAO,GAAG,IAAI,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,OAAO,IAAI,SAAS,CAAC,CAAC,GAAG,IAAI,MAAM,EAAE,CAAC;AACxE;;;AC9EA,SAAS,YAAYC,WAAU;AAO/B,eAAsB,KAAK,OAAgB,OAAoB,CAAC,GAAkB;AAChF,QAAM,OAAO,UAAU,OAAO,IAAI;AAElC,MAAI,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC5C,UAAMA,IAAG,UAAU,KAAK,QAAQ,QAAQ,KAAK,SAAS,IAAI,IAAI,KAAK,KAAK;AACxE;AAAA,EACF;AAEA,UAAQ,OAAO,MAAM,IAAI;AACzB,MAAI,CAAC,KAAK,SAAS,IAAI,EAAG,SAAQ,OAAO,MAAM,IAAI;AACrD;AAEO,SAAS,UAAU,OAAgB,OAAoB,CAAC,GAAW;AACxE,MAAI,OAAO,UAAU,SAAU,QAAO;AAEtC,QAAM,SAAS,KAAK,UAAU,QAAQ,OAAO,SAAS;AACtD,SAAO,KAAK,UAAU,OAAO,MAAM,SAAS,IAAI,CAAC;AACnD;;;ACzBA,SAAS,qBAAqB;AAoBvB,SAAS,aAAa,SAA6C;AACxE,QAAM,QAAQ,IAAI,cAAc;AAAA,IAC9B,QAAQ,QAAQ;AAAA,IAChB,WAAW,QAAQ;AAAA,EACrB,CAAC;AAKD,QAAM,MAAM,EAAE,UAAU,KAAK;AAE7B,SAAO,IAAI,MAAM,OAAO;AAAA,IACtB,IAAI,QAAQ,MAAM,UAAU;AAC1B,YAAM,QAAQ,QAAQ,IAAI,QAAQ,MAAM,QAAQ;AAChD,UAAI,OAAO,UAAU,WAAY,QAAO;AACxC,cAAQ,MAAM;AAAA,QACZ,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAA+B,OAAO,iBAAiB,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjF,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAAqB,OAAO,OAAO,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QAC7D,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B;AACE,iBAAO,OAAO,UAAU,aAAa,MAAM,KAAK,MAAM,IAAI;AAAA,MAC9D;AAAA,IACF;AAAA,EACF,CAAC;AACH;;;AHvDO,SAAS,iBAA0B;AACxC,SAAO,IAAI,QAAQ,SAAS,EACzB,YAAY,0CAA0C,EACtD,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,OAAO,SAAuB;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,QAAQ;AACpC,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AACL;;;AIzBA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;ACyBzB,SAAS,mBACd,OACA,WAAkC,CAAC,GACb;AACtB,QAAM,MAA4B,CAAC;AAInC,MAAI,MAAM,OAAO,OAAW,KAAI,KAAK,MAAM;AAC3C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,cAAc,OAAW,KAAI,aAAa,MAAM;AAC1D,MAAI,MAAM,YAAY,OAAW,KAAI,WAAW,MAAM;AACtD,MAAI,MAAM,aAAa,OAAW,KAAI,YAAY,MAAM;AACxD,MAAI,MAAM,gBAAgB,OAAW,KAAI,eAAe,MAAM;AAC9D,MAAI,MAAM,WAAY,KAAI,eAAe;AACzC,MAAI,MAAM,gBAAiB,KAAI,oBAAoB;AACnD,MAAI,SAAS,QAAS,KAAI,UAAU,SAAS;AAC7C,SAAO;AACT;AAMO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,MAAI,OAAO,mBAAoB,KAAI,uBAAuB;AAC1D,SAAO;AACT;AAOO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,OAAO,WAAY,KAAI,cAAc,OAAO;AAChD,MAAI,OAAO,YAAa,KAAI,eAAe;AAC3C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AAIjB,QAAM,MAAgC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACpF,MAAI,aAAa,OAAW,KAAI,WAAW;AAC3C,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,6BACd,KACA,WACA,OACA,UACyB;AAIzB,QAAM,MAAwC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC5F,MAAI,UAAU,SAAS,EAAG,KAAI,YAAY;AAC1C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,mBACd,KACA,QACA,OACA,UACe;AACf,SAAO,EAAE,KAAK,QAAQ,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC/D;;;ACxHA,SAAS,YAAYC,WAAU;AAC/B,SAAS,cAAc;AA0BhB,SAAS,uBAAuB,KAAuB;AAC5D,SAAO,IACJ,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D;AAAA,IACC,IAAI,OAAO,eAAe,+BAA+B,EAAE,UAAU,QAAQ;AAAA,EAC/E,EACC,OAAO,QAAQ,sDAAsD,EACrE,OAAO,WAAW,wCAAwC,EAC1D;AAAA,IACC,IAAI,OAAO,kBAAkB,YAAY,EAAE,QAAQ;AAAA,MACjD;AAAA,MACA;AAAA,MACA;AAAA,IACF,CAAC;AAAA,EACH,EACC,OAAO,oBAAoB,gDAAgD,EAC3E;AAAA,IACC,IAAI,OAAO,mBAAmB,kBAAkB,EAAE,QAAQ,CAAC,WAAW,UAAU,QAAQ,CAAC;AAAA,EAC3F,EACC,OAAO,kBAAkB,4CAA4C,aAAa,EAClF,OAAO,qBAAqB,kDAAkD,aAAa,EAC3F,OAAO,yBAAyB,mDAAmD,EACnF;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,sBAAsB,0CAA0C,EACvE,OAAO,wBAAwB,8CAA8C,EAC7E,OAAO,kBAAkB,qBAAqB,EAC9C,OAAO,uBAAuB,0BAA0B;AAC7D;AAEO,SAAS,cAAc,OAAuB;AACnD,QAAM,IAAI,OAAO,SAAS,OAAO,EAAE;AACnC,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,OAAM,IAAI,MAAM,0BAA0B,KAAK,EAAE;AAC1E,SAAO;AACT;AAMA,eAAsB,aAAa,OAAwE;AACzG,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,GAAI,QAAO;AAC1C,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,SAAO,gBAAgB,MAAM,SAAS;AACxC;AAMA,eAAsB,YAAY,OAAgD;AAChF,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,QAAM,SAAS,gBAAgB,MAAM,QAAQ;AAC7C,MAAI,CAAC,UAAU,OAAO,KAAK,MAAM,EAAE,WAAW,GAAG;AAC/C,UAAM,IAAI,MAAM,sEAAsE;AAAA,EACxF;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,MAAc,OAAuC;AAC5E,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,SAAS,KAAK;AACZ,UAAM,IAAI,MAAM,GAAG,KAAK,yBAAqB,IAAc,OAAO,EAAE;AAAA,EACtE;AACA,MAAI,CAAC,UAAU,OAAO,WAAW,YAAY,MAAM,QAAQ,MAAM,GAAG;AAClE,UAAM,IAAI,MAAM,GAAG,KAAK,wBAAwB;AAAA,EAClD;AACA,QAAM,MAA8B,CAAC;AACrC,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,MAAiC,GAAG;AACtE,QAAI,OAAO,MAAM,SAAU,OAAM,IAAI,MAAM,GAAG,KAAK,IAAI,CAAC,0BAA0B;AAClF,QAAI,CAAC,IAAI;AAAA,EACX;AACA,SAAO;AACT;;;AC1GA,OAAO,cAAc;AAEd,SAAS,WAAW,KAAsB;AAC/C,SAAO,QAAQ;AACjB;AAEA,gBAAuB,oBAA2C;AAChE,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,QAAQ,OAAO,WAAW,SAAS,CAAC;AACjF,mBAAiB,OAAO,IAAI;AAC1B,UAAM,OAAO,IAAI,KAAK;AACtB,QAAI,SAAS,MAAM,KAAK,WAAW,GAAG,EAAG;AACzC,UAAM;AAAA,EACR;AACF;AAKA,gBAAuB,YAAY,KAAoC;AACrE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,kBAAkB;AACzB;AAAA,EACF;AACA,QAAM;AACR;;;AHlBO,SAAS,aAAsB;AACpC,QAAM,MAAM,IAAIC,SAAQ,KAAK,EAC1B,YAAY,sDAAsD,EAClE,SAAS,SAAS,2CAA2C,EAC7D,eAAe,yBAAyB,2CAA2C,EACnF,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAmB;AAChD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AIrCA,SAAS,WAAAC,gBAAe;AAUjB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAAE,YAAY,4BAA4B;AAExE,MACG,QAAQ,WAAW,EACnB,YAAY,sBAAsB,WAAW,CAAC,EAAE,EAChD,OAAO,OAAO,QAAgB;AAC7B,QAAI,CAAC,OAAO,IAAI,KAAK,MAAM,GAAI,OAAM,IAAI,MAAM,kBAAkB;AACjE,UAAM,qBAAqB,EAAE,QAAQ,IAAI,KAAK,EAAE,CAAC;AACjD,YAAQ,OAAO,MAAM,oBAAoB,WAAW,CAAC;AAAA,CAAI;AAAA,EAC3D,CAAC;AAEH,MACG,QAAQ,MAAM,EACd,YAAY,+CAA+C,EAC3D,OAAO,YAAY;AAClB,UAAM,YAAY,MAAM,oBAAoB;AAC5C,UAAM,MAAM,QAAQ,IAAI;AACxB,QAAI,OAAO,IAAI,KAAK,MAAM,IAAI;AAC5B,cAAQ,OAAO,MAAM,4CAA4C,WAAW,IAAI,KAAK,CAAC,CAAC;AAAA,CAAI;AAAA,IAC7F;AACA,QAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,IAAI;AACtD,cAAQ,OAAO;AAAA,QACb,eAAe,WAAW,CAAC,MAAM,WAAW,UAAU,OAAO,KAAK,CAAC,CAAC;AAAA;AAAA,MACtE;AAAA,IACF;AACA,QAAI,CAAC,OAAO,CAAC,UAAU,QAAQ;AAC7B,cAAQ,OAAO,MAAM,+DAA+D;AAAA,IACtF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,OAAO,EACf,YAAY,gCAAgC,EAC5C,OAAO,YAAY;AAClB,UAAM,UAAU,MAAM,qBAAqB;AAC3C,YAAQ,OAAO,MAAM,UAAU,WAAW,WAAW,CAAC;AAAA,IAAO,gCAAgC;AAAA,EAC/F,CAAC;AAEH,SAAO;AACT;;;AClDA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,iBAA0B;AACxC,QAAM,MAAM,IAAIC,SAAQ,SAAS,EAC9B,YAAY,mEAAmE,EAC/E,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAuB;AACpD,QAAI,CAAC,KAAK,OAAQ,OAAM,IAAI,MAAM,sBAAsB;AACxD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,MAAM,YAAY,KAAK,MAAM;AAC5C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,mBAAmB,QAAQ,QAAQ,MAAM,EAAE,QAAQ,CAAC;AACvE,YAAM,SAAS,MAAM,OAAO,OAAO,UAAU;AAC7C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACxCA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,+BAA+B,EAC3C,SAAS,SAAS,2CAA2C,EAC7D,UAAU,IAAIC,QAAO,0BAA0B,kDAAkD,CAAC,EAClG,UAAU,IAAIA,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,iBAAiB,QAAQ,MAAM,EAAE,QAAQ,GAAG;AAAA,QAC7D,oBAAoB,KAAK;AAAA,MAC3B,CAAC;AACD,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACtCA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,kBAA2B;AACzC,QAAM,MAAM,IAAIC,SAAQ,UAAU,EAC/B,YAAY,0CAA0C,EACtD,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAwB;AACrD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACvCA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,0BAAmC;AACjD,QAAM,MAAM,IAAIC,SAAQ,mBAAmB,EACxC,YAAY,gDAAgD,EAC5D,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,IACA,CAAC;AAAA,EACH;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAgC;AAE7D,UAAM,YAAY,KAAK,YAAY,CAAC;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,6BAA6B,QAAQ,WAAW,MAAM,EAAE,QAAQ,CAAC;AACpF,YAAM,SAAS,MAAM,OAAO,iBAAiB,UAAU;AACvD,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,aAAa,OAAe,UAA8B;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;;;AC9CA,SAAS,WAAAC,gBAAe;;;ACmBxB,SAAS,YAAYC,WAAU;AAC/B,SAAS,WAAAC,gBAAe;AACxB,SAAS,WAAAC,UAAS,QAAAC,OAAM,eAAe;AACvC,SAAS,qBAAqB;AA0BvB,IAAM,UAAoC;AAAA;AAAA,EAE/C,EAAE,IAAI,eAAe,OAAO,eAAe,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA;AAAA,EAG5G,EAAE,IAAI,UAAU,OAAO,UAAU,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAAA;AAAA,EAEzG,EAAE,IAAI,YAAY,OAAO,YAAY,MAAM,QAAQ,MAAM,WAAW,MAAM,oCAAoC;AAAA;AAAA,EAE9G,EAAE,IAAI,QAAQ,OAAO,QAAQ,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA,EAE9F,EAAE,IAAI,YAAY,OAAO,YAAY,MAAM,OAAO,MAAM,WAAW,MAAM,kCAAkC;AAAA;AAAA,EAE3G,EAAE,IAAI,UAAU,OAAO,cAAc,MAAM,QAAQ,MAAM,QAAQ,MAAM,8CAA8C;AAAA;AAAA,EAErH,EAAE,IAAI,WAAW,OAAO,kBAAkB,MAAM,QAAQ,MAAM,WAAW,MAAM,kCAAkC;AAAA;AAAA,EAEjH,EAAE,IAAI,WAAW,OAAO,WAAW,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAAA;AAAA,EAE3G,EAAE,IAAI,WAAW,OAAO,WAAW,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAC7G;AAEO,SAAS,WAAW,IAAuC;AAChE,SAAO,QAAQ,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE;AACxC;AAMO,SAAS,kBAA0B;AACxC,QAAM,OAAOD,SAAQ,cAAc,YAAY,GAAG,CAAC;AAGnD,SAAO,QAAQ,MAAM,MAAM,WAAW,kBAAkB,UAAU,gBAAgB;AACpF;AAaO,SAAS,cAAc,QAAuB,OAAuB,CAAC,GAAkB;AAC7F,QAAM,OAAO,OAAO,SAAS,SAAU,KAAK,QAAQD,SAAQ,IAAM,KAAK,OAAO,QAAQ,IAAI;AAC1F,SAAO,EAAE,QAAQ,aAAaE,MAAK,MAAM,OAAO,IAAI,EAAE;AACxD;AAgBA,eAAsB,aACpB,QACA,OAAuB,CAAC,GACA;AACxB,QAAM,SAAS,KAAK,aAAa,gBAAgB;AACjD,QAAM,SAAS,MAAM,WAAW,OAAO,WAAW;AAElD,MAAI,UAAU,CAAC,KAAK,OAAO;AACzB,WAAO,EAAE,QAAQ,QAAQ,iBAAiB;AAAA,EAC5C;AAEA,MAAI,OAAO,OAAO,SAAS,OAAO;AAChC,QAAI,OAAQ,OAAMH,IAAG,GAAG,OAAO,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AAC5E,UAAM,cAAc,QAAQ,OAAO,WAAW;AAAA,EAChD,OAAO;AACL,UAAM,YAAY,MAAMA,IAAG,SAASG,MAAK,QAAQ,UAAU,GAAG,MAAM;AACpE,UAAMH,IAAG,MAAME,SAAQ,OAAO,WAAW,GAAG,EAAE,WAAW,KAAK,CAAC;AAC/D,UAAMF,IAAG,UAAU,OAAO,aAAa,SAAS;AAAA,EAClD;AAEA,SAAO,EAAE,QAAQ,QAAQ,SAAS,gBAAgB,YAAY;AAChE;AAEA,eAAe,WAAW,MAAgC;AACxD,MAAI;AACF,UAAMA,IAAG,OAAO,IAAI;AACpB,WAAO;AAAA,EACT,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEA,eAAe,cAAc,QAAgB,aAAoC;AAE/E,QAAMA,IAAG,GAAG,QAAQ,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AACnE;;;AD3IO,SAAS,eAAwB;AACtC,QAAM,MAAM,IAAII,SAAQ,OAAO,EAAE,YAAY,gDAAgD;AAE7F,MACG,QAAQ,OAAO,EACf,YAAY,qEAAqE,EACjF;AAAA,IACC;AAAA,IACA,sCAAsC,QAAQ,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,KAAK,IAAI,CAAC;AAAA,IACzE;AAAA,IACA,CAAC;AAAA,EACH,EACC,OAAO,aAAa,qCAAqC,EACzD,OAAO,eAAe,gCAAgC,EACtD,OAAO,OAAO,SAA0B;AACvC,UAAM,WAAW,iBAAiB,IAAI;AACtC,QAAI,SAAS,WAAW,GAAG;AACzB,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,eAAW,UAAU,UAAU;AAC7B,YAAM,SAAS,cAAc,MAAM;AACnC,YAAM,SAAS,MAAM,aAAa,QAAQ,EAAE,OAAO,KAAK,MAAM,CAAC;AAC/D,cAAQ,OAAO,MAAM,GAAG,OAAO,KAAK,KAAK,OAAO,MAAM,WAAM,OAAO,WAAW;AAAA,CAAI;AAAA,IACpF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,QAAQ,EAAE,QAAQ,MAAM,CAAC,EACjC,YAAY,0CAA0C,EACtD,OAAO,MAAM;AACZ,eAAW,KAAK,SAAS;AACvB,YAAM,SAAS,cAAc,CAAC;AAC9B,cAAQ,OAAO,MAAM,GAAG,EAAE,GAAG,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,OAAO,EAAE,CAAC,IAAI,OAAO,WAAW;AAAA,CAAI;AAAA,IACzF;AAAA,EACF,CAAC;AAEH,SAAO;AACT;AAEA,SAAS,QAAQ,OAAe,UAA8B;AAC5D,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAEA,SAAS,iBAAiB,MAAwC;AAChE,MAAI,KAAK,IAAK,QAAO,CAAC,GAAG,OAAO;AAChC,QAAM,MAAM,KAAK,UAAU,CAAC;AAC5B,SAAO,IAAI,IAAI,CAAC,OAAO;AACrB,UAAM,SAAS,WAAW,EAAE;AAC5B,QAAI,CAAC,OAAQ,OAAM,IAAI,MAAM,mBAAmB,EAAE,EAAE;AACpD,WAAO;AAAA,EACT,CAAC;AACH;;;AEjEA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAezB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,kCAAkC,EAC9C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,uBAAuB,sBAAsB,EAAE,QAAQ,CAAC,SAAS,QAAQ,KAAK,CAAC;AAAA,EAC5F,EACC,OAAO,kBAAkB,4CAA4C;AAExE,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,iBAAiB,QAAQ,MAAM,EAAE,QAAQ,GAAG;AAAA,QAC7D,YAAY,KAAK;AAAA,QACjB,aAAa,KAAK;AAAA,MACpB,CAAC;AACD,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AClCA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEA,IAAM,aAAa;AAAA,EACxB,IAAI;AAAA,EACJ,SAAS;AAAA,EACT,OAAO;AAAA,EACP,YAAY;AAAA,EACZ,MAAM;AAAA,EACN,iBAAiB;AAAA,EACjB,WAAW;AAAA,EACX,aAAa;AAAA,EACb,SAAS;AAAA,EACT,YAAY;AACd;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,qBAAsB,QAAO,WAAW;AAC3D,MAAI,eAAe,eAAgB,QAAO,WAAW;AACrD,MAAI,eAAe,YAAa,QAAO,WAAW;AAClD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,MAAI,eAAe,SAAU,QAAO,WAAW;AAC/C,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,SAAO,WAAW;AACpB;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,UAAU;AAC3B,UAAM,QAAQ,CAAC,IAAI,OAAO;AAC1B,QAAI,IAAI,OAAQ,OAAM,KAAK,SAAS,IAAI,MAAM,GAAG;AACjD,WAAO,MAAM,KAAK,GAAG;AAAA,EACvB;AACA,MAAI,eAAe,MAAO,QAAO,IAAI;AACrC,SAAO,OAAO,GAAG;AACnB;;;AjBhCA,SAAS,iBAAyB;AAGhC,QAAM,OAAOC,SAAQC,eAAc,YAAY,GAAG,CAAC;AACnD,QAAM,UAAUC,SAAQ,MAAM,MAAM,cAAc;AAClD,QAAM,MAAM,KAAK,MAAM,aAAa,SAAS,MAAM,CAAC;AACpD,SAAO,IAAI;AACb;AAEO,SAAS,eAAwB;AACtC,QAAM,UAAU,IAAIC,UAAQ;AAC5B,UACG,KAAK,gBAAgB,EACrB;AAAA,IACC;AAAA,EAEF,EACC,QAAQ,eAAe,GAAG,iBAAiB,2BAA2B,EACtE,mBAAmB;AAEtB,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,gBAAgB,CAAC;AACpC,UAAQ,WAAW,wBAAwB,CAAC;AAC5C,UAAQ,WAAW,WAAW,CAAC;AAC/B,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,aAAa,CAAC;AAEjC,SAAO;AACT;AAEA,eAAe,KAAK,MAA+B;AACjD,QAAM,UAAU,aAAa;AAC7B,MAAI;AACF,UAAM,QAAQ,WAAW,IAAI;AAAA,EAC/B,SAAS,KAAK;AACZ,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B;AACF;AAIA,IAAM,YAAY,QAAQ,KAAK,CAAC,IAAIF,eAAc,YAAY,GAAG,IAAI;AACrE,IAAI,aAAa,QAAQ,KAAK,CAAC,MAAM,WAAW;AAC9C,OAAK,QAAQ,IAAI,EAAE,MAAM,CAAC,QAAiB;AACzC,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B,CAAC;AACH;","names":["Command","dirname","resolve","fileURLToPath","fs","Command","Option","fs","Command","Option","Command","Command","Command","Command","Command","Option","Command","Option","Command","Option","Command","Option","Command","Command","Command","fs","homedir","dirname","join","Command","Command","Option","Command","Option","dirname","fileURLToPath","resolve","Command"]}
1
+ {"version":3,"sources":["../src/cli.ts","../src/commands/account.ts","../src/lib/config.ts","../src/lib/output.ts","../src/lib/sdk.ts","../src/lib/redact.ts","../src/commands/ask.ts","../src/lib/buildRequest.ts","../src/lib/options.ts","../src/lib/stdin.ts","../src/commands/auth.ts","../src/commands/extract.ts","../src/commands/html.ts","../src/commands/selected.ts","../src/commands/selectedMultiple.ts","../src/commands/serp.ts","../src/lib/errors.ts","../src/commands/setupSkill.ts","../src/lib/skill.ts","../src/commands/text.ts"],"sourcesContent":["/**\n * `webscraping-ai` / `wsai` CLI entry point.\n *\n * Wires every subcommand into a single commander root. Errors thrown from\n * subcommand actions land in the top-level `.catch`, which maps SDK\n * exception classes to stable exit codes (see `lib/errors.ts`).\n */\n\nimport { Command } from 'commander';\nimport { readFileSync } from 'node:fs';\nimport { dirname, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nimport { accountCommand } from './commands/account.js';\nimport { askCommand } from './commands/ask.js';\nimport { authCommand } from './commands/auth.js';\nimport { extractCommand } from './commands/extract.js';\nimport { htmlCommand } from './commands/html.js';\nimport { selectedCommand } from './commands/selected.js';\nimport { selectedMultipleCommand } from './commands/selectedMultiple.js';\nimport { serpCommand } from './commands/serp.js';\nimport { setupCommand } from './commands/setupSkill.js';\nimport { textCommand } from './commands/text.js';\nimport { exitCodeFor, formatError } from './lib/errors.js';\n\nfunction packageVersion(): string {\n // Read package.json at runtime so we don't need a code-gen step. The dist\n // bundle sits at `dist/cli.js`, so package.json is one directory up.\n const here = dirname(fileURLToPath(import.meta.url));\n const pkgPath = resolve(here, '..', 'package.json');\n const raw = JSON.parse(readFileSync(pkgPath, 'utf8')) as { version: string };\n return raw.version;\n}\n\nexport function buildProgram(): Command {\n const program = new Command();\n program\n .name('webscraping-ai')\n .description(\n 'Official CLI for the WebScraping.AI API. Fetch pages, search Google, ask AI, extract structured data.\\n' +\n 'Set $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first.',\n )\n .version(packageVersion(), '-v, --version', 'output the version number')\n .showHelpAfterError();\n\n program.addCommand(htmlCommand());\n program.addCommand(textCommand());\n program.addCommand(selectedCommand());\n program.addCommand(selectedMultipleCommand());\n program.addCommand(askCommand());\n program.addCommand(extractCommand());\n program.addCommand(serpCommand());\n program.addCommand(accountCommand());\n program.addCommand(authCommand());\n program.addCommand(setupCommand());\n\n return program;\n}\n\nasync function main(argv: string[]): Promise<void> {\n const program = buildProgram();\n try {\n await program.parseAsync(argv);\n } catch (err) {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n }\n}\n\n// Only execute when run as a script. When imported (e.g. from tests),\n// `buildProgram` is the public surface and `main` stays dormant.\nconst entryPath = process.argv[1] ? fileURLToPath(import.meta.url) : '';\nif (entryPath && process.argv[1] === entryPath) {\n main(process.argv).catch((err: unknown) => {\n process.stderr.write(`Error: ${formatError(err)}\\n`);\n process.exit(exitCodeFor(err));\n });\n}\n","import { Command } from 'commander';\n\nimport { resolveApiKey } from '../lib/config.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface AccountFlags {\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\nexport function accountCommand(): Command {\n return new Command('account')\n .description('Show account quota and usage information')\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .action(async (opts: AccountFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const result = await client.account();\n await emit(result, opts);\n });\n}\n","/**\n * API key resolution and persisted config.\n *\n * Resolution order (first non-empty wins):\n * 1. `--api-key` flag (handled by the caller, passed in)\n * 2. `WEBSCRAPING_AI_API_KEY` env var\n * 3. `~/.config/webscraping-ai/config.json` (or `$XDG_CONFIG_HOME/...`)\n *\n * The config file is plain JSON with `{ \"apiKey\": \"...\" }`. We chmod it to\n * 0600 so it's not world-readable.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join } from 'node:path';\n\nconst CONFIG_FILENAME = 'config.json';\n\nexport function configDir(): string {\n const xdg = process.env.XDG_CONFIG_HOME;\n if (xdg && xdg.trim() !== '') return join(xdg, 'webscraping-ai');\n return join(homedir(), '.config', 'webscraping-ai');\n}\n\nexport function configPath(): string {\n return join(configDir(), CONFIG_FILENAME);\n}\n\nexport interface PersistedConfig {\n apiKey?: string;\n}\n\nexport async function readPersistedConfig(): Promise<PersistedConfig> {\n try {\n const text = await fs.readFile(configPath(), 'utf8');\n const parsed = JSON.parse(text) as unknown;\n if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {\n return parsed as PersistedConfig;\n }\n return {};\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return {};\n throw err;\n }\n}\n\nexport async function writePersistedConfig(config: PersistedConfig): Promise<void> {\n const path = configPath();\n await fs.mkdir(dirname(path), { recursive: true });\n await fs.writeFile(path, JSON.stringify(config, null, 2) + '\\n', { mode: 0o600 });\n // `mode` above only applies when the file is freshly created; an existing\n // file keeps its old (possibly world-readable) perms. Enforce 0600\n // unconditionally — chmod on a just-written file is cheap and idempotent.\n // No-op on Windows, but harmless.\n await fs.chmod(path, 0o600);\n}\n\nexport async function clearPersistedConfig(): Promise<boolean> {\n try {\n await fs.unlink(configPath());\n return true;\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === 'ENOENT') return false;\n throw err;\n }\n}\n\nexport interface ResolveApiKeyOptions {\n /** `--api-key` flag value, if the user passed one. */\n flag?: string;\n}\n\nexport async function resolveApiKey(opts: ResolveApiKeyOptions = {}): Promise<string> {\n if (opts.flag && opts.flag.trim() !== '') return opts.flag.trim();\n\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') return env.trim();\n\n const persisted = await readPersistedConfig();\n if (persisted.apiKey && persisted.apiKey.trim() !== '') return persisted.apiKey.trim();\n\n throw new Error(\n 'No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`.',\n );\n}\n\nexport function maskApiKey(key: string): string {\n if (key.length <= 8) return '*'.repeat(key.length);\n return `${key.slice(0, 4)}${'*'.repeat(key.length - 8)}${key.slice(-4)}`;\n}\n","/**\n * Output handling for command results.\n *\n * All command handlers funnel through `emit()`. The result type drives format:\n * - strings → written verbatim\n * - objects/arrays → JSON.stringified (pretty by default; `--pretty` is the\n * default for terminals, single-line for pipes/redirects)\n *\n * `--output FILE` writes to disk instead of stdout. In stdin batch mode use\n * `createEmitter()`: it truncates the file on the first result and appends\n * every later one, so each item survives instead of overwriting the last.\n */\n\nimport { promises as fs } from 'node:fs';\n\nexport interface EmitOptions {\n output?: string;\n pretty?: boolean;\n}\n\nexport interface EmitWriteOptions {\n /** Append to `--output` instead of truncating it (batch items after the first). */\n append?: boolean;\n}\n\nexport async function emit(\n value: unknown,\n opts: EmitOptions = {},\n write: EmitWriteOptions = {},\n): Promise<void> {\n const text = stringify(value, opts);\n\n if (opts.output && opts.output.trim() !== '') {\n const data = text + (text.endsWith('\\n') ? '' : '\\n');\n if (write.append) await fs.appendFile(opts.output, data);\n else await fs.writeFile(opts.output, data);\n return;\n }\n\n process.stdout.write(text);\n if (!text.endsWith('\\n')) process.stdout.write('\\n');\n}\n\n/**\n * Returns an `emit` bound to `opts` for a (possibly batched) loop: the first\n * call truncates `--output`, later calls append.\n */\nexport function createEmitter(opts: EmitOptions = {}): (value: unknown) => Promise<void> {\n let first = true;\n return async (value: unknown) => {\n const append = !first;\n first = false;\n await emit(value, opts, { append });\n };\n}\n\nexport function stringify(value: unknown, opts: EmitOptions = {}): string {\n if (typeof value === 'string') return value;\n\n const pretty = opts.pretty ?? process.stdout.isTTY ?? false;\n return JSON.stringify(value, null, pretty ? 2 : 0);\n}\n","/**\n * SDK adapter.\n *\n * Wraps the published `webscraping-ai` Node SDK. The CLI sends a\n * `from_cli=true` analytics flag on the page/AI endpoints (mirrors `from_n8n`\n * and `from_mcp_server` used by the n8n node and MCP server). It is NOT sent\n * on `serp` or `account`: the SDK builds those query strings from a fixed set\n * of keys and drops anything else. The SDK's typed\n * options don't accept arbitrary keys, so we go through a single cast point\n * here rather than scattering casts across each command.\n */\n\nimport { WebScrapingAI } from 'webscraping-ai';\n\nimport { registerSecret } from './redact.js';\nimport type {\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nexport interface CreateClientOptions {\n apiKey: string;\n /** Per-request timeout in ms. Maps to the SDK's `timeoutMs`. */\n requestTimeoutMs?: number;\n}\n\n/**\n * Returns a `WebScrapingAI` instance with the CLI's analytics tag baked into\n * every endpoint method. The methods preserve the SDK's public signatures.\n */\nexport function createClient(options: CreateClientOptions): WebScrapingAI {\n registerSecret(options.apiKey);\n const inner = new WebScrapingAI({\n apiKey: options.apiKey,\n timeoutMs: options.requestTimeoutMs,\n });\n\n // Wrap each endpoint method so it injects the CLI analytics tag. We splice\n // the tag in at this seam rather than at every callsite. The cast is local;\n // the SDK passes unknown query params through to the wire.\n const tag = { from_cli: true } as unknown as Record<string, never>;\n\n return new Proxy(inner, {\n get(target, prop, receiver) {\n const value = Reflect.get(target, prop, receiver);\n if (typeof value !== 'function') return value;\n switch (prop) {\n case 'html':\n return (o: HtmlOptions) => target.html({ ...o, ...tag });\n case 'text':\n return (o: TextOptions) => target.text({ ...o, ...tag });\n case 'selected':\n return (o: SelectedOptions) => target.selected({ ...o, ...tag });\n case 'selectedMultiple':\n return (o: SelectedMultipleOptions) => target.selectedMultiple({ ...o, ...tag });\n case 'question':\n return (o: QuestionOptions) => target.question({ ...o, ...tag });\n case 'fields':\n return (o: FieldsOptions) => target.fields({ ...o, ...tag });\n case 'serp':\n // Passed for parity only: SDK 4.1.0's `serp()` forwards just\n // q/engine/gl/hl/page and drops every other key, so `from_cli` is\n // not sent on /serp. Accepted limitation, like `account`.\n return (o: SerpOptions) => target.serp({ ...o, ...tag });\n case 'account':\n // `account` takes no options; the SDK builds the request itself, so\n // there's no seam here for the analytics tag — accepted limitation.\n return value.bind(target);\n default:\n return typeof value === 'function' ? value.bind(target) : value;\n }\n },\n });\n}\n","/**\n * Secret redaction for anything the CLI prints to stderr.\n *\n * API errors can echo the request URL (e.g. an HTML error page that includes\n * `?api_key=...`), so every error message goes through `redactSecrets()`\n * before it is printed. The key in use is registered by `createClient()`.\n */\n\nconst secrets = new Set<string>();\n\n/** Remember a secret so later error output never prints it. */\nexport function registerSecret(value: string | undefined): void {\n if (value && value.trim() !== '') secrets.add(value);\n}\n\n/** Forget registered secrets (tests only). */\nexport function clearSecrets(): void {\n secrets.clear();\n}\n\nconst API_KEY_PARAM = /(api_key=)[^&\\s\"'<>]*/gi;\n\nexport function redactSecrets(text: string): string {\n let out = text.replace(API_KEY_PARAM, '$1[REDACTED]');\n const candidates = new Set(secrets);\n const env = process.env.WEBSCRAPING_AI_API_KEY?.trim();\n if (env) candidates.add(env);\n for (const secret of candidates) {\n for (const form of new Set([secret, encodeURIComponent(secret)])) {\n out = out.split(form).join('[REDACTED]');\n }\n }\n return out;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildQuestionOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface AskFlags extends CommonRawFlags {\n question?: string;\n}\n\nexport function askCommand(): Command {\n const cmd = new Command('ask')\n .description('Ask an AI question about a page (wraps /ai/question)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption('-q, --question <text>', 'the question / instructions for the model')\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: AskFlags) => {\n if (!opts.question) throw new Error('--question is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });\n const result = await client.question(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Pure builder that maps `--flag` values into SDK option objects.\n *\n * Extracted as a separate function so unit tests can pin the mapping without\n * spinning up commander or making HTTP calls. Each command handler reads its\n * own commander options, awaits `parseHeaders` / `parseFields`, then calls\n * the matching `build*` function here.\n */\n\nimport type {\n CommonRequestOptions,\n FieldsOptions,\n HtmlOptions,\n QuestionOptions,\n SelectedMultipleOptions,\n SelectedOptions,\n SerpOptions,\n TextOptions,\n} from 'webscraping-ai';\n\nimport type { CommonRawFlags } from './options.js';\n\nexport interface ResolvedCommonOptions {\n headers?: Record<string, string>;\n}\n\nexport function buildCommonOptions(\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions = {},\n): CommonRequestOptions {\n const out: CommonRequestOptions = {};\n // `flags.js` is `false` when `--no-js`, `true` when `--js`, `undefined`\n // when neither passed. Pass through only when explicit, so the API default\n // applies otherwise.\n if (flags.js !== undefined) out.js = flags.js;\n if (flags.proxy !== undefined) out.proxy = flags.proxy;\n if (flags.country !== undefined) out.country = flags.country;\n if (flags.device !== undefined) out.device = flags.device;\n if (flags.timeout !== undefined) out.timeout = flags.timeout;\n if (flags.jsTimeout !== undefined) out.js_timeout = flags.jsTimeout;\n if (flags.waitFor !== undefined) out.wait_for = flags.waitFor;\n if (flags.jsScript !== undefined) out.js_script = flags.jsScript;\n if (flags.customProxy !== undefined) out.custom_proxy = flags.customProxy;\n if (flags.errorOn404) out.error_on_404 = true;\n if (flags.errorOnRedirect) out.error_on_redirect = true;\n if (resolved.headers) out.headers = resolved.headers;\n return out;\n}\n\nexport interface HtmlExtras {\n returnScriptResult?: boolean;\n}\n\nexport function buildHtmlOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: HtmlExtras = {},\n): HtmlOptions {\n const out: HtmlOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n if (extras.returnScriptResult) out.return_script_result = true;\n return out;\n}\n\nexport interface TextExtras {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function buildTextOptions(\n url: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n extras: TextExtras = {},\n): TextOptions {\n const out: TextOptions = { url, ...buildCommonOptions(flags, resolved) };\n if (extras.textFormat) out.text_format = extras.textFormat;\n if (extras.returnLinks) out.return_links = true;\n return out;\n}\n\nexport function buildSelectedOptions(\n url: string,\n selector: string | undefined,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedOptions {\n // `selector` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when provided. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selector !== undefined) out.selector = selector;\n if (flags.format) out.format = flags.format;\n return out as SelectedOptions;\n}\n\nexport function buildSelectedMultipleOptions(\n url: string,\n selectors: readonly string[],\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): SelectedMultipleOptions {\n // `selectors` is optional per the OpenAPI spec: omitting it returns the\n // whole-page HTML. Only attach it when non-empty. (The installed SDK types\n // still mark it required, so build the object as a partial and cast.)\n const out: Partial<SelectedMultipleOptions> = { url, ...buildCommonOptions(flags, resolved) };\n if (selectors.length > 0) out.selectors = selectors;\n return out as SelectedMultipleOptions;\n}\n\nexport function buildQuestionOptions(\n url: string,\n question: string,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): QuestionOptions {\n const out: QuestionOptions = { url, question, ...buildCommonOptions(flags, resolved) };\n if (flags.format) out.format = flags.format;\n return out;\n}\n\nexport function buildFieldsOptions(\n url: string,\n fields: Record<string, string>,\n flags: CommonRawFlags,\n resolved: ResolvedCommonOptions,\n): FieldsOptions {\n return { url, fields, ...buildCommonOptions(flags, resolved) };\n}\n\nexport interface SerpRawFlags {\n engine?: 'google';\n gl?: string;\n hl?: string;\n page?: number;\n}\n\n/**\n * `/serp` is query-shaped: it takes none of the common scrape options, so\n * this deliberately does not go through `buildCommonOptions`. `page` must be\n * an integer >= 1 (the server also rejects anything else with a 400, not\n * billed; checking client-side saves the round trip). Pages are 1-100: the\n * server rejects a page above 100 with a 400.\n */\nexport function buildSerpOptions(query: string, flags: SerpRawFlags = {}): SerpOptions {\n // Validate on the trimmed value but send the query as the user typed it.\n if (typeof query !== 'string' || query.trim() === '') {\n throw new Error('serp requires a non-empty query');\n }\n const out: SerpOptions = { q: query };\n if (flags.engine !== undefined) out.engine = flags.engine;\n if (flags.gl) out.gl = flags.gl;\n if (flags.hl) out.hl = flags.hl;\n if (flags.page !== undefined) {\n if (!Number.isSafeInteger(flags.page) || flags.page < 1) {\n throw new Error(`--page must be an integer >= 1, got: ${flags.page}`);\n }\n out.page = flags.page;\n }\n return out;\n}\n","/**\n * Shared commander options and the parsers/expanders behind them.\n *\n * All scrape endpoints share the same nine \"common\" flags. We define them in\n * one place and apply via `addCommonScrapeOptions(cmd)` so the surface stays\n * consistent and any change lands everywhere at once.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { Option } from 'commander';\nimport type { Command } from 'commander';\n\nexport interface CommonRawFlags {\n apiKey?: string;\n output?: string;\n format?: 'text' | 'json';\n pretty?: boolean;\n js?: boolean;\n proxy?: 'datacenter' | 'residential' | 'stealth';\n country?: string;\n device?: 'desktop' | 'mobile' | 'tablet';\n timeout?: number;\n jsTimeout?: number;\n waitFor?: string;\n headers?: string;\n jsScript?: string;\n customProxy?: string;\n errorOn404?: boolean;\n errorOnRedirect?: boolean;\n}\n\n/**\n * Add the flags every scrape-endpoint command needs. Adding here keeps the\n * surface aligned across `html`, `text`, `selected`, etc.\n */\nexport function addCommonScrapeOptions(cmd: Command): Command {\n return cmd\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output')\n .option('--js', 'execute JS via a headless browser (default: enabled)')\n .option('--no-js', 'disable JS execution (faster, cheaper)')\n .addOption(\n new Option('--proxy <type>', 'proxy pool').choices(['datacenter', 'residential', 'stealth']),\n )\n .option('--country <code>', 'two-letter proxy country code, e.g. us, gb, de')\n .addOption(\n new Option('--device <type>', 'device emulation').choices(['desktop', 'mobile', 'tablet']),\n )\n .option('--timeout <ms>', 'page-retrieval timeout in ms (max 30000)', integerParser)\n .option('--js-timeout <ms>', 'JS rendering time in ms after load (max 20000)', integerParser)\n .option('--wait-for <selector>', 'CSS selector to wait for before returning content')\n .option(\n '--headers <json>',\n 'HTTP headers to forward to the target page; JSON object or @file.json',\n )\n .option('--js-script <code>', 'custom JS to evaluate on the target page')\n .option('--custom-proxy <url>', 'custom proxy URL: http://user:pass@host:port')\n .option('--error-on-404', 'return error on 404')\n .option('--error-on-redirect', 'return error on redirect');\n}\n\n/**\n * Strict positive-integer check for flags like `serp --page`: digits only,\n * no leading zero, so `1.5`, `2abc`, `0` and `-1` are all rejected (unlike\n * `parseInt`, which would read `2abc` as 2). Values beyond\n * `Number.MAX_SAFE_INTEGER` are rejected too.\n */\nexport function isPositiveIntegerString(value: string): boolean {\n // isSafeInteger: a huge digit string becomes e.g. 1e+21 on the wire, which\n // the server rejects with a 400 (not billed); checking here saves the round trip.\n return /^[1-9]\\d*$/.test(value) && Number.isSafeInteger(Number(value));\n}\n\nexport function integerParser(value: string): number {\n const n = Number.parseInt(value, 10);\n if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);\n return n;\n}\n\n/**\n * Parse a `--headers` argument. Accepts a JSON object literal or `@path` to\n * load JSON from a file. Returns `undefined` if the input is empty.\n */\nexport async function parseHeaders(\n input: string | undefined,\n): Promise<Record<string, string> | undefined> {\n if (!input || input.trim() === '') return undefined;\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n return parseJsonObject(text, 'headers');\n}\n\n/**\n * Parse a `--fields` argument for `extract`. Accepts JSON object literal or\n * `@path` to a file.\n */\nexport async function parseFields(input: string): Promise<Record<string, string>> {\n const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n const parsed = parseJsonObject(text, 'fields');\n if (!parsed || Object.keys(parsed).length === 0) {\n throw new Error('fields must be a non-empty JSON object of { fieldName: description }');\n }\n return parsed;\n}\n\nfunction parseJsonObject(text: string, label: string): Record<string, string> {\n let parsed: unknown;\n try {\n parsed = JSON.parse(text);\n } catch (err) {\n throw new Error(`${label}: invalid JSON — ${(err as Error).message}`);\n }\n if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {\n throw new Error(`${label} must be a JSON object`);\n }\n const out: Record<string, string> = {};\n for (const [k, v] of Object.entries(parsed as Record<string, unknown>)) {\n if (typeof v !== 'string') throw new Error(`${label}.${k}: value must be a string`);\n out[k] = v;\n }\n return out;\n}\n","/**\n * Stdin batching helpers.\n *\n * When a URL argument is `-`, the command reads one URL per line from stdin\n * and runs the operation for each. Empty lines and lines starting with `#`\n * are skipped so users can comment URL lists.\n *\n * `serp` reads queries instead (`queryIterator`): only blank lines are\n * skipped there, because `#hashtag` is a legitimate search query.\n */\n\nimport readline from 'node:readline';\n\nexport function urlIsStdin(url: string): boolean {\n return url === '-';\n}\n\nexport async function* readUrlsFromStdin(): AsyncIterable<string> {\n const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });\n for await (const raw of rl) {\n const line = raw.trim();\n if (line === '' || line.startsWith('#')) continue;\n yield line;\n }\n}\n\n/**\n * Read one search query per line. Blank / whitespace-only lines are skipped;\n * every other line is yielded verbatim (no `#` comment handling, no trim).\n */\nexport async function* readQueriesFromStdin(\n input: NodeJS.ReadableStream = process.stdin,\n): AsyncIterable<string> {\n const rl = readline.createInterface({ input, crlfDelay: Infinity });\n for await (const line of rl) {\n if (line.trim() === '') continue;\n yield line;\n }\n}\n\n/**\n * Iterate URLs from either a single CLI arg or from stdin (if the arg is `-`).\n */\nexport async function* urlIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readUrlsFromStdin();\n return;\n }\n yield arg;\n}\n\n/**\n * Iterate `serp` queries from a single CLI arg or from stdin (if it is `-`).\n */\nexport async function* queryIterator(arg: string): AsyncIterable<string> {\n if (urlIsStdin(arg)) {\n yield* readQueriesFromStdin();\n return;\n }\n yield arg;\n}\n","import { Command } from 'commander';\n\nimport {\n clearPersistedConfig,\n configPath,\n maskApiKey,\n readPersistedConfig,\n writePersistedConfig,\n} from '../lib/config.js';\n\nexport function authCommand(): Command {\n const cmd = new Command('auth').description('Manage your stored API key');\n\n cmd\n .command('set <key>')\n .description(`Save an API key to ${configPath()}`)\n .action(async (key: string) => {\n if (!key || key.trim() === '') throw new Error('API key is empty');\n await writePersistedConfig({ apiKey: key.trim() });\n process.stdout.write(`API key saved to ${configPath()}\\n`);\n });\n\n cmd\n .command('show')\n .description('Display the currently stored API key (masked)')\n .action(async () => {\n const persisted = await readPersistedConfig();\n const env = process.env.WEBSCRAPING_AI_API_KEY;\n if (env && env.trim() !== '') {\n process.stdout.write(\n `Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}\\n`,\n );\n }\n if (persisted.apiKey && persisted.apiKey.trim() !== '') {\n process.stdout.write(\n `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}\\n`,\n );\n }\n if (!env && !persisted.apiKey) {\n process.stdout.write('No API key configured. Run `webscraping-ai auth set <key>`.\\n');\n }\n });\n\n cmd\n .command('clear')\n .description('Remove the stored API key file')\n .action(async () => {\n const removed = await clearPersistedConfig();\n process.stdout.write(\n removed ? `Removed ${configPath()}\\n` : 'No stored API key to remove.\\n',\n );\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildFieldsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseFields, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface ExtractFlags extends CommonRawFlags {\n fields?: string;\n}\n\nexport function extractCommand(): Command {\n const cmd = new Command('extract')\n .description('Extract structured fields from a page using AI (wraps /ai/fields)')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .requiredOption(\n '-f, --fields <json>',\n 'fields to extract — JSON object {name: description} or @file.json',\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: ExtractFlags) => {\n if (!opts.fields) throw new Error('--fields is required');\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const fields = await parseFields(opts.fields);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });\n const result = await client.fields(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildHtmlOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface HtmlFlags extends CommonRawFlags {\n returnScriptResult?: boolean;\n}\n\nexport function htmlCommand(): Command {\n const cmd = new Command('html')\n .description('Fetch the full HTML of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--return-script-result', 'return the result of --js-script instead of HTML'),\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: HtmlFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildHtmlOptions(\n target,\n opts,\n { headers },\n {\n returnScriptResult: opts.returnScriptResult,\n },\n );\n const result = await client.html(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSelectedOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedFlags extends CommonRawFlags {\n selector?: string;\n}\n\nexport function selectedCommand(): Command {\n const cmd = new Command('selected')\n .description('Fetch HTML of one CSS-selected page area')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector for the target element(s); omit for whole-page HTML',\n )\n .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });\n const result = await client.selected(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildSelectedMultipleOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedMultipleFlags extends CommonRawFlags {\n selector?: string[];\n}\n\nexport function selectedMultipleCommand(): Command {\n const cmd = new Command('selected-multiple')\n .description('Fetch HTML of multiple CSS-selected page areas')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .option(\n '-s, --selector <css>',\n 'CSS selector — pass multiple times for multiple selectors',\n collectMulti,\n [] as string[],\n );\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: SelectedMultipleFlags) => {\n // `--selector` is optional: omitting it returns whole-page HTML.\n const selectors = opts.selector ?? [];\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });\n const result = await client.selectedMultiple(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n\nfunction collectMulti(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSerpOptions } from '../lib/buildRequest.js';\nimport type { SerpRawFlags } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { EXIT_CODES } from '../lib/errors.js';\nimport { isPositiveIntegerString } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { queryIterator, urlIsStdin } from '../lib/stdin.js';\n\ninterface SerpFlags extends Omit<SerpRawFlags, 'page'> {\n /** Raw `--page` string; validated strictly in the action. */\n page?: string;\n apiKey?: string;\n output?: string;\n pretty?: boolean;\n}\n\n/**\n * `serp <query...>` — parsed search engine results (wraps /serp).\n *\n * Query-shaped, not URL-shaped, so none of the common scrape flags\n * (`--js`, `--proxy`, `--country`, …) apply and they're deliberately not\n * registered here. Multiple words are joined with spaces, so quoting the\n * query is optional. A lone `-` reads one query per line from stdin; only\n * blank lines are skipped (a leading `#` is part of the query, not a comment).\n *\n * Usage errors (bad `--page`, empty query) exit with code 2 before stdin is\n * read or the API key is resolved.\n */\nexport function serpCommand(): Command {\n const cmd = new Command('serp')\n .alias('search')\n .description('Search Google and print parsed results as JSON (wraps /serp, 15 credits/search)')\n .argument('<query...>', 'search query (or `-` to read queries from stdin, one per line)')\n .addOption(new Option('--engine <name>', 'search engine (default: google)').choices(['google']))\n .option('--gl <code>', 'two-letter country code for search geolocation (default: us)')\n .option('--hl <code>', 'two-letter language code for the results (default: en)')\n .option(\n '--page <n>',\n 'results page number, integer >= 1, 10 results per page (default: 1; server rejects > 100 with a 400)',\n )\n .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n .option('-o, --output <file>', 'write result to FILE instead of stdout')\n .option('--pretty', 'pretty-print JSON output (default on TTY)')\n .option('--no-pretty', 'force single-line JSON output');\n\n cmd.action(async (words: string[], opts: SerpFlags) => {\n const query = words.join(' ');\n\n let page: number | undefined;\n if (opts.page !== undefined) {\n if (!isPositiveIntegerString(opts.page)) {\n cmd.error(`error: --page must be an integer >= 1, got: ${opts.page}`, {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.invalidPage',\n });\n }\n page = Number(opts.page);\n }\n if (!urlIsStdin(query) && query.trim() === '') {\n cmd.error('error: serp requires a non-empty query', {\n exitCode: EXIT_CODES.usage,\n code: 'wsai.emptyQuery',\n });\n }\n\n const flags: SerpRawFlags = { engine: opts.engine, gl: opts.gl, hl: opts.hl, page };\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const client = createClient({ apiKey });\n const write = createEmitter(opts);\n\n for await (const q of queryIterator(query)) {\n const result = await client.serp(buildSerpOptions(q, flags));\n await write(result);\n }\n });\n\n return cmd;\n}\n","/**\n * Exit-code mapping for SDK errors.\n *\n * The SDK throws typed subclasses of `WebScrapingAIError`. Map each to a\n * stable exit code so shell scripts can branch on the failure mode without\n * parsing stderr.\n */\n\nimport {\n APIConnectionError,\n APIError,\n APITimeoutError,\n AuthenticationError,\n BadRequestError,\n GatewayTimeoutError,\n PaymentRequiredError,\n RateLimitError,\n ServerError,\n WebScrapingAIError,\n} from 'webscraping-ai';\n\nimport { redactSecrets } from './redact.js';\n\nexport const EXIT_CODES = {\n ok: 0,\n generic: 1,\n usage: 2,\n badRequest: 3,\n auth: 4,\n paymentRequired: 5,\n rateLimit: 6,\n serverError: 7,\n timeout: 8,\n connection: 9,\n} as const;\n\nexport function exitCodeFor(err: unknown): number {\n if (err instanceof BadRequestError) return EXIT_CODES.badRequest;\n if (err instanceof AuthenticationError) return EXIT_CODES.auth;\n if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;\n if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;\n if (err instanceof ServerError) return EXIT_CODES.serverError;\n if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APITimeoutError) return EXIT_CODES.timeout;\n if (err instanceof APIConnectionError) return EXIT_CODES.connection;\n if (err instanceof APIError) return EXIT_CODES.serverError;\n if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;\n return EXIT_CODES.generic;\n}\n\n/**\n * Human-readable one-liner for stderr. Always redacted: the API key and any\n * `api_key=...` pattern are replaced, since error bodies can echo the URL.\n */\nexport function formatError(err: unknown): string {\n return redactSecrets(rawMessage(err));\n}\n\nfunction rawMessage(err: unknown): string {\n if (err instanceof APIError) {\n const parts = [err.message];\n if (err.status) parts.push(`(HTTP ${err.status})`);\n return parts.join(' ');\n }\n if (err instanceof Error) return err.message;\n return String(err);\n}\n","import { Command } from 'commander';\n\nimport { EDITORS, editorById, installSkill, resolveTarget } from '../lib/skill.js';\nimport type { EditorAdapter } from '../lib/skill.js';\n\ninterface SetupSkillFlags {\n editor?: string[];\n all?: boolean;\n force?: boolean;\n}\n\nexport function setupCommand(): Command {\n const cmd = new Command('setup').description('Install the AI agent skill into your editor(s)');\n\n cmd\n .command('skill')\n .description('Install the WebScraping.AI skill into one or more AI coding editors')\n .option(\n '-e, --editor <id>',\n `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(', ')}`,\n collect,\n [] as string[],\n )\n .option('-a, --all', 'install into every supported editor')\n .option('-f, --force', 'overwrite existing skill files')\n .action(async (opts: SetupSkillFlags) => {\n const selected = resolveSelection(opts);\n if (selected.length === 0) {\n throw new Error(\n 'no editor selected — pass --all or --editor <id> (e.g. --editor claude-code)',\n );\n }\n\n for (const editor of selected) {\n const target = resolveTarget(editor);\n const result = await installSkill(target, { force: opts.force });\n process.stdout.write(`${editor.label}: ${result.status} → ${target.destination}\\n`);\n }\n });\n\n cmd\n .command('list', { hidden: false })\n .description('List supported editors for `setup skill`')\n .action(() => {\n for (const e of EDITORS) {\n const target = resolveTarget(e);\n process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}\\n`);\n }\n });\n\n return cmd;\n}\n\nfunction collect(value: string, previous: string[]): string[] {\n return [...previous, value];\n}\n\nfunction resolveSelection(opts: SetupSkillFlags): EditorAdapter[] {\n if (opts.all) return [...EDITORS];\n const ids = opts.editor ?? [];\n return ids.map((id) => {\n const editor = editorById(id);\n if (!editor) throw new Error(`unknown editor: ${id}`);\n return editor;\n });\n}\n","/**\n * Multi-editor skill installer.\n *\n * The canonical skill lives in this package at\n * `plugins/webscraping-ai/skills/webscraping-ai/SKILL.md`. Each supported\n * editor reads skill content from a different location and may want either\n * a single Markdown file or a whole skill directory. The `editors` registry\n * captures those differences so `setup skill` is a small driver over it.\n *\n * `mode: 'dir'` copies the entire skill directory (Claude Code-shaped\n * editors that scan a folder per skill). `mode: 'file'` writes the SKILL.md\n * contents to a single file (editors that read flat instructions, like\n * GitHub Copilot's `.github/copilot-instructions.md`).\n *\n * `kind: 'user'` writes to `$HOME`; `kind: 'project'` writes to the current\n * working directory. We prefer user-scope where the editor supports it so\n * the skill becomes globally available without polluting individual repos.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nexport type EditorMode = 'dir' | 'file';\nexport type EditorKind = 'user' | 'project';\n\nexport interface EditorAdapter {\n /** Stable identifier used by `--editor <name>`. */\n id: string;\n /** Human-readable label printed in summaries. */\n label: string;\n mode: EditorMode;\n kind: EditorKind;\n /**\n * Where the skill ends up. For `mode: 'dir'`, the skill directory copies\n * to `{root}/{path}`. For `mode: 'file'`, the SKILL.md contents write to\n * `{root}/{path}` (i.e. `path` ends in a filename).\n *\n * `root` is `$HOME` (user) or CWD (project).\n */\n path: string;\n}\n\n/**\n * Registry of supported editors. Order matters: it's also the order the\n * `setup skill` summary lists them in.\n */\nexport const EDITORS: readonly EditorAdapter[] = [\n // Claude Code: user-scope skills live under ~/.claude/skills/<name>/\n {\n id: 'claude-code',\n label: 'Claude Code',\n mode: 'dir',\n kind: 'user',\n path: '.claude/skills/webscraping-ai',\n },\n // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill\n // dir, so this is the editor's \"install\" surface.\n {\n id: 'cursor',\n label: 'Cursor',\n mode: 'file',\n kind: 'project',\n path: '.cursor/rules/webscraping-ai.mdc',\n },\n // Windsurf: project-scope rules. Same shape as Cursor.\n {\n id: 'windsurf',\n label: 'Windsurf',\n mode: 'file',\n kind: 'project',\n path: '.windsurf/rules/webscraping-ai.md',\n },\n // Kiro: user-scope steering directory.\n { id: 'kiro', label: 'Kiro', mode: 'dir', kind: 'user', path: '.kiro/steering/webscraping-ai' },\n // OpenCode: project-scope skills (mirrors Claude shape).\n {\n id: 'opencode',\n label: 'OpenCode',\n mode: 'dir',\n kind: 'project',\n path: '.opencode/skills/webscraping-ai',\n },\n // Gemini CLI: user-scope extension dir with a GEMINI.md.\n {\n id: 'gemini',\n label: 'Gemini CLI',\n mode: 'file',\n kind: 'user',\n path: '.gemini/extensions/webscraping-ai/GEMINI.md',\n },\n // GitHub Copilot: project-scope custom instructions file. Single file.\n {\n id: 'copilot',\n label: 'GitHub Copilot',\n mode: 'file',\n kind: 'project',\n path: '.github/copilot-instructions.md',\n },\n // Augment: project-scope rules file.\n {\n id: 'augment',\n label: 'Augment',\n mode: 'file',\n kind: 'project',\n path: '.augment/rules/webscraping-ai.md',\n },\n // Factory: project-scope rules file.\n {\n id: 'factory',\n label: 'Factory',\n mode: 'file',\n kind: 'project',\n path: '.factory/rules/webscraping-ai.md',\n },\n] as const;\n\nexport function editorById(id: string): EditorAdapter | undefined {\n return EDITORS.find((e) => e.id === id);\n}\n\n/**\n * Where this package keeps the canonical skill. Resolved relative to the\n * compiled `dist/cli.js` so it works after `npm install -g`.\n */\nexport function bundledSkillDir(): string {\n const here = dirname(fileURLToPath(import.meta.url));\n // dist/ sits at the package root, so the plugins dir is two levels up\n // from any compiled file: dist/cli.js → package root → plugins/...\n return resolve(here, '..', 'plugins', 'webscraping-ai', 'skills', 'webscraping-ai');\n}\n\nexport interface InstallTarget {\n editor: EditorAdapter;\n /** Absolute destination path (file or directory depending on `mode`). */\n destination: string;\n}\n\nexport interface ResolveOptions {\n cwd?: string;\n home?: string;\n}\n\nexport function resolveTarget(editor: EditorAdapter, opts: ResolveOptions = {}): InstallTarget {\n const root = editor.kind === 'user' ? (opts.home ?? homedir()) : (opts.cwd ?? process.cwd());\n return { editor, destination: join(root, editor.path) };\n}\n\nexport interface InstallOptions {\n /** If true, overwrite existing files/dirs. */\n force?: boolean;\n /** Override skill source dir (for tests). */\n sourceDir?: string;\n}\n\nexport type InstallStatus = 'installed' | 'skipped-exists' | 'overwritten';\n\nexport interface InstallResult {\n target: InstallTarget;\n status: InstallStatus;\n}\n\nexport async function installSkill(\n target: InstallTarget,\n opts: InstallOptions = {},\n): Promise<InstallResult> {\n const source = opts.sourceDir ?? bundledSkillDir();\n const exists = await pathExists(target.destination);\n\n if (exists && !opts.force) {\n return { target, status: 'skipped-exists' };\n }\n\n if (target.editor.mode === 'dir') {\n if (exists) await fs.rm(target.destination, { recursive: true, force: true });\n await copyDirectory(source, target.destination);\n } else {\n const skillBody = await fs.readFile(join(source, 'SKILL.md'), 'utf8');\n await fs.mkdir(dirname(target.destination), { recursive: true });\n await fs.writeFile(target.destination, skillBody);\n }\n\n return { target, status: exists ? 'overwritten' : 'installed' };\n}\n\nasync function pathExists(path: string): Promise<boolean> {\n try {\n await fs.access(path);\n return true;\n } catch {\n return false;\n }\n}\n\nasync function copyDirectory(source: string, destination: string): Promise<void> {\n // `fs.cp` with `recursive` exists on Node 16.7+; we target Node 20.\n await fs.cp(source, destination, { recursive: true, force: true });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildTextOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { createEmitter } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface TextFlags extends CommonRawFlags {\n textFormat?: 'plain' | 'json' | 'xml';\n returnLinks?: boolean;\n}\n\nexport function textCommand(): Command {\n const cmd = new Command('text')\n .description('Fetch the visible text of a page')\n .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n .addOption(\n new Option('--text-format <fmt>', 'text response format').choices(['plain', 'json', 'xml']),\n )\n .option('--return-links', 'include page links (text-format=json only)');\n\n addCommonScrapeOptions(cmd);\n\n cmd.action(async (url: string, opts: TextFlags) => {\n const apiKey = await resolveApiKey({ flag: opts.apiKey });\n const headers = await parseHeaders(opts.headers);\n const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n const write = createEmitter(opts);\n\n for await (const target of urlIterator(url)) {\n const sdkOptions = buildTextOptions(\n target,\n opts,\n { headers },\n {\n textFormat: opts.textFormat,\n returnLinks: opts.returnLinks,\n },\n );\n const result = await client.text(sdkOptions);\n await write(result);\n }\n });\n\n return cmd;\n}\n"],"mappings":";;;AAQA,SAAS,WAAAA,iBAAe;AACxB,SAAS,oBAAoB;AAC7B,SAAS,WAAAC,UAAS,WAAAC,gBAAe;AACjC,SAAS,iBAAAC,sBAAqB;;;ACX9B,SAAS,eAAe;;;ACYxB,SAAS,YAAY,UAAU;AAC/B,SAAS,eAAe;AACxB,SAAS,SAAS,YAAY;AAE9B,IAAM,kBAAkB;AAEjB,SAAS,YAAoB;AAClC,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,gBAAgB;AAC/D,SAAO,KAAK,QAAQ,GAAG,WAAW,gBAAgB;AACpD;AAEO,SAAS,aAAqB;AACnC,SAAO,KAAK,UAAU,GAAG,eAAe;AAC1C;AAMA,eAAsB,sBAAgD;AACpE,MAAI;AACF,UAAM,OAAO,MAAM,GAAG,SAAS,WAAW,GAAG,MAAM;AACnD,UAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,QAAI,UAAU,OAAO,WAAW,YAAY,CAAC,MAAM,QAAQ,MAAM,GAAG;AAClE,aAAO;AAAA,IACT;AACA,WAAO,CAAC;AAAA,EACV,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO,CAAC;AAC9D,UAAM;AAAA,EACR;AACF;AAEA,eAAsB,qBAAqB,QAAwC;AACjF,QAAM,OAAO,WAAW;AACxB,QAAM,GAAG,MAAM,QAAQ,IAAI,GAAG,EAAE,WAAW,KAAK,CAAC;AACjD,QAAM,GAAG,UAAU,MAAM,KAAK,UAAU,QAAQ,MAAM,CAAC,IAAI,MAAM,EAAE,MAAM,IAAM,CAAC;AAKhF,QAAM,GAAG,MAAM,MAAM,GAAK;AAC5B;AAEA,eAAsB,uBAAyC;AAC7D,MAAI;AACF,UAAM,GAAG,OAAO,WAAW,CAAC;AAC5B,WAAO;AAAA,EACT,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO;AAC7D,UAAM;AAAA,EACR;AACF;AAOA,eAAsB,cAAc,OAA6B,CAAC,GAAoB;AACpF,MAAI,KAAK,QAAQ,KAAK,KAAK,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,KAAK;AAEhE,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,IAAI,KAAK;AAE9C,QAAM,YAAY,MAAM,oBAAoB;AAC5C,MAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,GAAI,QAAO,UAAU,OAAO,KAAK;AAErF,QAAM,IAAI;AAAA,IACR;AAAA,EACF;AACF;AAEO,SAAS,WAAW,KAAqB;AAC9C,MAAI,IAAI,UAAU,EAAG,QAAO,IAAI,OAAO,IAAI,MAAM;AACjD,SAAO,GAAG,IAAI,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,OAAO,IAAI,SAAS,CAAC,CAAC,GAAG,IAAI,MAAM,EAAE,CAAC;AACxE;;;AC5EA,SAAS,YAAYC,WAAU;AAY/B,eAAsB,KACpB,OACA,OAAoB,CAAC,GACrB,QAA0B,CAAC,GACZ;AACf,QAAM,OAAO,UAAU,OAAO,IAAI;AAElC,MAAI,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC5C,UAAM,OAAO,QAAQ,KAAK,SAAS,IAAI,IAAI,KAAK;AAChD,QAAI,MAAM,OAAQ,OAAMA,IAAG,WAAW,KAAK,QAAQ,IAAI;AAAA,QAClD,OAAMA,IAAG,UAAU,KAAK,QAAQ,IAAI;AACzC;AAAA,EACF;AAEA,UAAQ,OAAO,MAAM,IAAI;AACzB,MAAI,CAAC,KAAK,SAAS,IAAI,EAAG,SAAQ,OAAO,MAAM,IAAI;AACrD;AAMO,SAAS,cAAc,OAAoB,CAAC,GAAsC;AACvF,MAAI,QAAQ;AACZ,SAAO,OAAO,UAAmB;AAC/B,UAAM,SAAS,CAAC;AAChB,YAAQ;AACR,UAAM,KAAK,OAAO,MAAM,EAAE,OAAO,CAAC;AAAA,EACpC;AACF;AAEO,SAAS,UAAU,OAAgB,OAAoB,CAAC,GAAW;AACxE,MAAI,OAAO,UAAU,SAAU,QAAO;AAEtC,QAAM,SAAS,KAAK,UAAU,QAAQ,OAAO,SAAS;AACtD,SAAO,KAAK,UAAU,OAAO,MAAM,SAAS,IAAI,CAAC;AACnD;;;ACjDA,SAAS,qBAAqB;;;ACJ9B,IAAM,UAAU,oBAAI,IAAY;AAGzB,SAAS,eAAe,OAAiC;AAC9D,MAAI,SAAS,MAAM,KAAK,MAAM,GAAI,SAAQ,IAAI,KAAK;AACrD;AAOA,IAAM,gBAAgB;AAEf,SAAS,cAAc,MAAsB;AAClD,MAAI,MAAM,KAAK,QAAQ,eAAe,cAAc;AACpD,QAAM,aAAa,IAAI,IAAI,OAAO;AAClC,QAAM,MAAM,QAAQ,IAAI,wBAAwB,KAAK;AACrD,MAAI,IAAK,YAAW,IAAI,GAAG;AAC3B,aAAW,UAAU,YAAY;AAC/B,eAAW,QAAQ,oBAAI,IAAI,CAAC,QAAQ,mBAAmB,MAAM,CAAC,CAAC,GAAG;AAChE,YAAM,IAAI,MAAM,IAAI,EAAE,KAAK,YAAY;AAAA,IACzC;AAAA,EACF;AACA,SAAO;AACT;;;ADEO,SAAS,aAAa,SAA6C;AACxE,iBAAe,QAAQ,MAAM;AAC7B,QAAM,QAAQ,IAAI,cAAc;AAAA,IAC9B,QAAQ,QAAQ;AAAA,IAChB,WAAW,QAAQ;AAAA,EACrB,CAAC;AAKD,QAAM,MAAM,EAAE,UAAU,KAAK;AAE7B,SAAO,IAAI,MAAM,OAAO;AAAA,IACtB,IAAI,QAAQ,MAAM,UAAU;AAC1B,YAAM,QAAQ,QAAQ,IAAI,QAAQ,MAAM,QAAQ;AAChD,UAAI,OAAO,UAAU,WAAY,QAAO;AACxC,cAAQ,MAAM;AAAA,QACZ,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAA+B,OAAO,iBAAiB,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjF,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAAqB,OAAO,OAAO,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QAC7D,KAAK;AAIH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B;AACE,iBAAO,OAAO,UAAU,aAAa,MAAM,KAAK,MAAM,IAAI;AAAA,MAC9D;AAAA,IACF;AAAA,EACF,CAAC;AACH;;;AHlEO,SAAS,iBAA0B;AACxC,SAAO,IAAI,QAAQ,SAAS,EACzB,YAAY,0CAA0C,EACtD,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,OAAO,SAAuB;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,QAAQ;AACpC,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AACL;;;AKzBA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;AC0BzB,SAAS,mBACd,OACA,WAAkC,CAAC,GACb;AACtB,QAAM,MAA4B,CAAC;AAInC,MAAI,MAAM,OAAO,OAAW,KAAI,KAAK,MAAM;AAC3C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,cAAc,OAAW,KAAI,aAAa,MAAM;AAC1D,MAAI,MAAM,YAAY,OAAW,KAAI,WAAW,MAAM;AACtD,MAAI,MAAM,aAAa,OAAW,KAAI,YAAY,MAAM;AACxD,MAAI,MAAM,gBAAgB,OAAW,KAAI,eAAe,MAAM;AAC9D,MAAI,MAAM,WAAY,KAAI,eAAe;AACzC,MAAI,MAAM,gBAAiB,KAAI,oBAAoB;AACnD,MAAI,SAAS,QAAS,KAAI,UAAU,SAAS;AAC7C,SAAO;AACT;AAMO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,MAAI,OAAO,mBAAoB,KAAI,uBAAuB;AAC1D,SAAO;AACT;AAOO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,OAAO,WAAY,KAAI,cAAc,OAAO;AAChD,MAAI,OAAO,YAAa,KAAI,eAAe;AAC3C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AAIjB,QAAM,MAAgC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACpF,MAAI,aAAa,OAAW,KAAI,WAAW;AAC3C,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,6BACd,KACA,WACA,OACA,UACyB;AAIzB,QAAM,MAAwC,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC5F,MAAI,UAAU,SAAS,EAAG,KAAI,YAAY;AAC1C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,mBACd,KACA,QACA,OACA,UACe;AACf,SAAO,EAAE,KAAK,QAAQ,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC/D;AAgBO,SAAS,iBAAiB,OAAe,QAAsB,CAAC,GAAgB;AAErF,MAAI,OAAO,UAAU,YAAY,MAAM,KAAK,MAAM,IAAI;AACpD,UAAM,IAAI,MAAM,iCAAiC;AAAA,EACnD;AACA,QAAM,MAAmB,EAAE,GAAG,MAAM;AACpC,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,GAAI,KAAI,KAAK,MAAM;AAC7B,MAAI,MAAM,SAAS,QAAW;AAC5B,QAAI,CAAC,OAAO,cAAc,MAAM,IAAI,KAAK,MAAM,OAAO,GAAG;AACvD,YAAM,IAAI,MAAM,wCAAwC,MAAM,IAAI,EAAE;AAAA,IACtE;AACA,QAAI,OAAO,MAAM;AAAA,EACnB;AACA,SAAO;AACT;;;ACzJA,SAAS,YAAYC,WAAU;AAC/B,SAAS,cAAc;AA0BhB,SAAS,uBAAuB,KAAuB;AAC5D,SAAO,IACJ,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,QAAQ,sDAAsD,EACrE,OAAO,WAAW,wCAAwC,EAC1D;AAAA,IACC,IAAI,OAAO,kBAAkB,YAAY,EAAE,QAAQ,CAAC,cAAc,eAAe,SAAS,CAAC;AAAA,EAC7F,EACC,OAAO,oBAAoB,gDAAgD,EAC3E;AAAA,IACC,IAAI,OAAO,mBAAmB,kBAAkB,EAAE,QAAQ,CAAC,WAAW,UAAU,QAAQ,CAAC;AAAA,EAC3F,EACC,OAAO,kBAAkB,4CAA4C,aAAa,EAClF,OAAO,qBAAqB,kDAAkD,aAAa,EAC3F,OAAO,yBAAyB,mDAAmD,EACnF;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,sBAAsB,0CAA0C,EACvE,OAAO,wBAAwB,8CAA8C,EAC7E,OAAO,kBAAkB,qBAAqB,EAC9C,OAAO,uBAAuB,0BAA0B;AAC7D;AAQO,SAAS,wBAAwB,OAAwB;AAG9D,SAAO,aAAa,KAAK,KAAK,KAAK,OAAO,cAAc,OAAO,KAAK,CAAC;AACvE;AAEO,SAAS,cAAc,OAAuB;AACnD,QAAM,IAAI,OAAO,SAAS,OAAO,EAAE;AACnC,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,OAAM,IAAI,MAAM,0BAA0B,KAAK,EAAE;AAC1E,SAAO;AACT;AAMA,eAAsB,aACpB,OAC6C;AAC7C,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,GAAI,QAAO;AAC1C,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,SAAO,gBAAgB,MAAM,SAAS;AACxC;AAMA,eAAsB,YAAY,OAAgD;AAChF,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,QAAM,SAAS,gBAAgB,MAAM,QAAQ;AAC7C,MAAI,CAAC,UAAU,OAAO,KAAK,MAAM,EAAE,WAAW,GAAG;AAC/C,UAAM,IAAI,MAAM,sEAAsE;AAAA,EACxF;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,MAAc,OAAuC;AAC5E,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,SAAS,KAAK;AACZ,UAAM,IAAI,MAAM,GAAG,KAAK,yBAAqB,IAAc,OAAO,EAAE;AAAA,EACtE;AACA,MAAI,CAAC,UAAU,OAAO,WAAW,YAAY,MAAM,QAAQ,MAAM,GAAG;AAClE,UAAM,IAAI,MAAM,GAAG,KAAK,wBAAwB;AAAA,EAClD;AACA,QAAM,MAA8B,CAAC;AACrC,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,MAAiC,GAAG;AACtE,QAAI,OAAO,MAAM,SAAU,OAAM,IAAI,MAAM,GAAG,KAAK,IAAI,CAAC,0BAA0B;AAClF,QAAI,CAAC,IAAI;AAAA,EACX;AACA,SAAO;AACT;;;AC/GA,OAAO,cAAc;AAEd,SAAS,WAAW,KAAsB;AAC/C,SAAO,QAAQ;AACjB;AAEA,gBAAuB,oBAA2C;AAChE,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,QAAQ,OAAO,WAAW,SAAS,CAAC;AACjF,mBAAiB,OAAO,IAAI;AAC1B,UAAM,OAAO,IAAI,KAAK;AACtB,QAAI,SAAS,MAAM,KAAK,WAAW,GAAG,EAAG;AACzC,UAAM;AAAA,EACR;AACF;AAMA,gBAAuB,qBACrB,QAA+B,QAAQ,OAChB;AACvB,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,WAAW,SAAS,CAAC;AAClE,mBAAiB,QAAQ,IAAI;AAC3B,QAAI,KAAK,KAAK,MAAM,GAAI;AACxB,UAAM;AAAA,EACR;AACF;AAKA,gBAAuB,YAAY,KAAoC;AACrE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,kBAAkB;AACzB;AAAA,EACF;AACA,QAAM;AACR;AAKA,gBAAuB,cAAc,KAAoC;AACvE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,qBAAqB;AAC5B;AAAA,EACF;AACA,QAAM;AACR;;;AH9CO,SAAS,aAAsB;AACpC,QAAM,MAAM,IAAIC,SAAQ,KAAK,EAC1B,YAAY,sDAAsD,EAClE,SAAS,SAAS,2CAA2C,EAC7D,eAAe,yBAAyB,2CAA2C,EACnF,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAmB;AAChD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AIvCA,SAAS,WAAAC,gBAAe;AAUjB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAAE,YAAY,4BAA4B;AAExE,MACG,QAAQ,WAAW,EACnB,YAAY,sBAAsB,WAAW,CAAC,EAAE,EAChD,OAAO,OAAO,QAAgB;AAC7B,QAAI,CAAC,OAAO,IAAI,KAAK,MAAM,GAAI,OAAM,IAAI,MAAM,kBAAkB;AACjE,UAAM,qBAAqB,EAAE,QAAQ,IAAI,KAAK,EAAE,CAAC;AACjD,YAAQ,OAAO,MAAM,oBAAoB,WAAW,CAAC;AAAA,CAAI;AAAA,EAC3D,CAAC;AAEH,MACG,QAAQ,MAAM,EACd,YAAY,+CAA+C,EAC3D,OAAO,YAAY;AAClB,UAAM,YAAY,MAAM,oBAAoB;AAC5C,UAAM,MAAM,QAAQ,IAAI;AACxB,QAAI,OAAO,IAAI,KAAK,MAAM,IAAI;AAC5B,cAAQ,OAAO;AAAA,QACb,4CAA4C,WAAW,IAAI,KAAK,CAAC,CAAC;AAAA;AAAA,MACpE;AAAA,IACF;AACA,QAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,IAAI;AACtD,cAAQ,OAAO;AAAA,QACb,eAAe,WAAW,CAAC,MAAM,WAAW,UAAU,OAAO,KAAK,CAAC,CAAC;AAAA;AAAA,MACtE;AAAA,IACF;AACA,QAAI,CAAC,OAAO,CAAC,UAAU,QAAQ;AAC7B,cAAQ,OAAO,MAAM,+DAA+D;AAAA,IACtF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,OAAO,EACf,YAAY,gCAAgC,EAC5C,OAAO,YAAY;AAClB,UAAM,UAAU,MAAM,qBAAqB;AAC3C,YAAQ,OAAO;AAAA,MACb,UAAU,WAAW,WAAW,CAAC;AAAA,IAAO;AAAA,IAC1C;AAAA,EACF,CAAC;AAEH,SAAO;AACT;;;ACtDA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,iBAA0B;AACxC,QAAM,MAAM,IAAIC,SAAQ,SAAS,EAC9B,YAAY,mEAAmE,EAC/E,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAuB;AACpD,QAAI,CAAC,KAAK,OAAQ,OAAM,IAAI,MAAM,sBAAsB;AACxD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,MAAM,YAAY,KAAK,MAAM;AAC5C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,mBAAmB,QAAQ,QAAQ,MAAM,EAAE,QAAQ,CAAC;AACvE,YAAM,SAAS,MAAM,OAAO,OAAO,UAAU;AAC7C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC1CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,+BAA+B,EAC3C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,0BAA0B,kDAAkD;AAAA,EACzF,EACC,UAAU,IAAIA,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,oBAAoB,KAAK;AAAA,QAC3B;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AC/CA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,kBAA2B;AACzC,QAAM,MAAM,IAAIC,SAAQ,UAAU,EAC/B,YAAY,0CAA0C,EACtD,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAwB;AACrD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACzCA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,0BAAmC;AACjD,QAAM,MAAM,IAAIC,SAAQ,mBAAmB,EACxC,YAAY,gDAAgD,EAC5D,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,IACA,CAAC;AAAA,EACH;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAgC;AAE7D,UAAM,YAAY,KAAK,YAAY,CAAC;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,6BAA6B,QAAQ,WAAW,MAAM,EAAE,QAAQ,CAAC;AACpF,YAAM,SAAS,MAAM,OAAO,iBAAiB,UAAU;AACvD,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,aAAa,OAAe,UAA8B;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;;;AChDA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;ACQhC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAIA,IAAM,aAAa;AAAA,EACxB,IAAI;AAAA,EACJ,SAAS;AAAA,EACT,OAAO;AAAA,EACP,YAAY;AAAA,EACZ,MAAM;AAAA,EACN,iBAAiB;AAAA,EACjB,WAAW;AAAA,EACX,aAAa;AAAA,EACb,SAAS;AAAA,EACT,YAAY;AACd;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,qBAAsB,QAAO,WAAW;AAC3D,MAAI,eAAe,eAAgB,QAAO,WAAW;AACrD,MAAI,eAAe,YAAa,QAAO,WAAW;AAClD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,MAAI,eAAe,SAAU,QAAO,WAAW;AAC/C,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,SAAO,WAAW;AACpB;AAMO,SAAS,YAAY,KAAsB;AAChD,SAAO,cAAc,WAAW,GAAG,CAAC;AACtC;AAEA,SAAS,WAAW,KAAsB;AACxC,MAAI,eAAe,UAAU;AAC3B,UAAM,QAAQ,CAAC,IAAI,OAAO;AAC1B,QAAI,IAAI,OAAQ,OAAM,KAAK,SAAS,IAAI,MAAM,GAAG;AACjD,WAAO,MAAM,KAAK,GAAG;AAAA,EACvB;AACA,MAAI,eAAe,MAAO,QAAO,IAAI;AACrC,SAAO,OAAO,GAAG;AACnB;;;ADnCO,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,MAAM,QAAQ,EACd,YAAY,iFAAiF,EAC7F,SAAS,cAAc,gEAAgE,EACvF,UAAU,IAAIC,QAAO,mBAAmB,iCAAiC,EAAE,QAAQ,CAAC,QAAQ,CAAC,CAAC,EAC9F,OAAO,eAAe,8DAA8D,EACpF,OAAO,eAAe,wDAAwD,EAC9E;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B;AAExD,MAAI,OAAO,OAAO,OAAiB,SAAoB;AACrD,UAAM,QAAQ,MAAM,KAAK,GAAG;AAE5B,QAAI;AACJ,QAAI,KAAK,SAAS,QAAW;AAC3B,UAAI,CAAC,wBAAwB,KAAK,IAAI,GAAG;AACvC,YAAI,MAAM,+CAA+C,KAAK,IAAI,IAAI;AAAA,UACpE,UAAU,WAAW;AAAA,UACrB,MAAM;AAAA,QACR,CAAC;AAAA,MACH;AACA,aAAO,OAAO,KAAK,IAAI;AAAA,IACzB;AACA,QAAI,CAAC,WAAW,KAAK,KAAK,MAAM,KAAK,MAAM,IAAI;AAC7C,UAAI,MAAM,0CAA0C;AAAA,QAClD,UAAU,WAAW;AAAA,QACrB,MAAM;AAAA,MACR,CAAC;AAAA,IACH;AAEA,UAAM,QAAsB,EAAE,QAAQ,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,KAAK,IAAI,KAAK;AAClF,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,KAAK,cAAc,KAAK,GAAG;AAC1C,YAAM,SAAS,MAAM,OAAO,KAAK,iBAAiB,GAAG,KAAK,CAAC;AAC3D,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AEhFA,SAAS,WAAAC,gBAAe;;;ACmBxB,SAAS,YAAYC,WAAU;AAC/B,SAAS,WAAAC,gBAAe;AACxB,SAAS,WAAAC,UAAS,QAAAC,OAAM,eAAe;AACvC,SAAS,qBAAqB;AA0BvB,IAAM,UAAoC;AAAA;AAAA,EAE/C;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA;AAAA,EAGA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA,EAAE,IAAI,QAAQ,OAAO,QAAQ,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA,EAE9F;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AAAA;AAAA,EAEA;AAAA,IACE,IAAI;AAAA,IACJ,OAAO;AAAA,IACP,MAAM;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,EACR;AACF;AAEO,SAAS,WAAW,IAAuC;AAChE,SAAO,QAAQ,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE;AACxC;AAMO,SAAS,kBAA0B;AACxC,QAAM,OAAOD,SAAQ,cAAc,YAAY,GAAG,CAAC;AAGnD,SAAO,QAAQ,MAAM,MAAM,WAAW,kBAAkB,UAAU,gBAAgB;AACpF;AAaO,SAAS,cAAc,QAAuB,OAAuB,CAAC,GAAkB;AAC7F,QAAM,OAAO,OAAO,SAAS,SAAU,KAAK,QAAQD,SAAQ,IAAM,KAAK,OAAO,QAAQ,IAAI;AAC1F,SAAO,EAAE,QAAQ,aAAaE,MAAK,MAAM,OAAO,IAAI,EAAE;AACxD;AAgBA,eAAsB,aACpB,QACA,OAAuB,CAAC,GACA;AACxB,QAAM,SAAS,KAAK,aAAa,gBAAgB;AACjD,QAAM,SAAS,MAAM,WAAW,OAAO,WAAW;AAElD,MAAI,UAAU,CAAC,KAAK,OAAO;AACzB,WAAO,EAAE,QAAQ,QAAQ,iBAAiB;AAAA,EAC5C;AAEA,MAAI,OAAO,OAAO,SAAS,OAAO;AAChC,QAAI,OAAQ,OAAMH,IAAG,GAAG,OAAO,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AAC5E,UAAM,cAAc,QAAQ,OAAO,WAAW;AAAA,EAChD,OAAO;AACL,UAAM,YAAY,MAAMA,IAAG,SAASG,MAAK,QAAQ,UAAU,GAAG,MAAM;AACpE,UAAMH,IAAG,MAAME,SAAQ,OAAO,WAAW,GAAG,EAAE,WAAW,KAAK,CAAC;AAC/D,UAAMF,IAAG,UAAU,OAAO,aAAa,SAAS;AAAA,EAClD;AAEA,SAAO,EAAE,QAAQ,QAAQ,SAAS,gBAAgB,YAAY;AAChE;AAEA,eAAe,WAAW,MAAgC;AACxD,MAAI;AACF,UAAMA,IAAG,OAAO,IAAI;AACpB,WAAO;AAAA,EACT,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEA,eAAe,cAAc,QAAgB,aAAoC;AAE/E,QAAMA,IAAG,GAAG,QAAQ,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AACnE;;;AD3LO,SAAS,eAAwB;AACtC,QAAM,MAAM,IAAII,SAAQ,OAAO,EAAE,YAAY,gDAAgD;AAE7F,MACG,QAAQ,OAAO,EACf,YAAY,qEAAqE,EACjF;AAAA,IACC;AAAA,IACA,sCAAsC,QAAQ,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,KAAK,IAAI,CAAC;AAAA,IACzE;AAAA,IACA,CAAC;AAAA,EACH,EACC,OAAO,aAAa,qCAAqC,EACzD,OAAO,eAAe,gCAAgC,EACtD,OAAO,OAAO,SAA0B;AACvC,UAAM,WAAW,iBAAiB,IAAI;AACtC,QAAI,SAAS,WAAW,GAAG;AACzB,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,eAAW,UAAU,UAAU;AAC7B,YAAM,SAAS,cAAc,MAAM;AACnC,YAAM,SAAS,MAAM,aAAa,QAAQ,EAAE,OAAO,KAAK,MAAM,CAAC;AAC/D,cAAQ,OAAO,MAAM,GAAG,OAAO,KAAK,KAAK,OAAO,MAAM,WAAM,OAAO,WAAW;AAAA,CAAI;AAAA,IACpF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,QAAQ,EAAE,QAAQ,MAAM,CAAC,EACjC,YAAY,0CAA0C,EACtD,OAAO,MAAM;AACZ,eAAW,KAAK,SAAS;AACvB,YAAM,SAAS,cAAc,CAAC;AAC9B,cAAQ,OAAO,MAAM,GAAG,EAAE,GAAG,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,OAAO,EAAE,CAAC,IAAI,OAAO,WAAW;AAAA,CAAI;AAAA,IACzF;AAAA,EACF,CAAC;AAEH,SAAO;AACT;AAEA,SAAS,QAAQ,OAAe,UAA8B;AAC5D,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAEA,SAAS,iBAAiB,MAAwC;AAChE,MAAI,KAAK,IAAK,QAAO,CAAC,GAAG,OAAO;AAChC,QAAM,MAAM,KAAK,UAAU,CAAC;AAC5B,SAAO,IAAI,IAAI,CAAC,OAAO;AACrB,UAAM,SAAS,WAAW,EAAE;AAC5B,QAAI,CAAC,OAAQ,OAAM,IAAI,MAAM,mBAAmB,EAAE,EAAE;AACpD,WAAO;AAAA,EACT,CAAC;AACH;;;AEjEA,SAAS,WAAAC,WAAS,UAAAC,eAAc;AAezB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,UAAQ,MAAM,EAC3B,YAAY,kCAAkC,EAC9C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,uBAAuB,sBAAsB,EAAE,QAAQ,CAAC,SAAS,QAAQ,KAAK,CAAC;AAAA,EAC5F,EACC,OAAO,kBAAkB,4CAA4C;AAExE,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,UAAM,QAAQ,cAAc,IAAI;AAEhC,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa;AAAA,QACjB;AAAA,QACA;AAAA,QACA,EAAE,QAAQ;AAAA,QACV;AAAA,UACE,YAAY,KAAK;AAAA,UACjB,aAAa,KAAK;AAAA,QACpB;AAAA,MACF;AACA,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,MAAM,MAAM;AAAA,IACpB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AnBxBA,SAAS,iBAAyB;AAGhC,QAAM,OAAOC,SAAQC,eAAc,YAAY,GAAG,CAAC;AACnD,QAAM,UAAUC,SAAQ,MAAM,MAAM,cAAc;AAClD,QAAM,MAAM,KAAK,MAAM,aAAa,SAAS,MAAM,CAAC;AACpD,SAAO,IAAI;AACb;AAEO,SAAS,eAAwB;AACtC,QAAM,UAAU,IAAIC,UAAQ;AAC5B,UACG,KAAK,gBAAgB,EACrB;AAAA,IACC;AAAA,EAEF,EACC,QAAQ,eAAe,GAAG,iBAAiB,2BAA2B,EACtE,mBAAmB;AAEtB,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,gBAAgB,CAAC;AACpC,UAAQ,WAAW,wBAAwB,CAAC;AAC5C,UAAQ,WAAW,WAAW,CAAC;AAC/B,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,aAAa,CAAC;AAEjC,SAAO;AACT;AAEA,eAAe,KAAK,MAA+B;AACjD,QAAM,UAAU,aAAa;AAC7B,MAAI;AACF,UAAM,QAAQ,WAAW,IAAI;AAAA,EAC/B,SAAS,KAAK;AACZ,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B;AACF;AAIA,IAAM,YAAY,QAAQ,KAAK,CAAC,IAAIF,eAAc,YAAY,GAAG,IAAI;AACrE,IAAI,aAAa,QAAQ,KAAK,CAAC,MAAM,WAAW;AAC9C,OAAK,QAAQ,IAAI,EAAE,MAAM,CAAC,QAAiB;AACzC,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B,CAAC;AACH;","names":["Command","dirname","resolve","fileURLToPath","fs","Command","Option","fs","Command","Option","Command","Command","Command","Command","Command","Option","Command","Option","Command","Option","Command","Option","Command","Command","Command","Option","Command","Option","Command","fs","homedir","dirname","join","Command","Command","Option","Command","Option","dirname","fileURLToPath","resolve","Command"]}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "webscraping-ai-cli",
3
- "version": "1.0.2",
3
+ "version": "1.1.0",
4
4
  "description": "Official command-line client for the WebScraping.AI API. Includes a Claude Code skill so AI agents can use it too.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -53,7 +53,7 @@
53
53
  "homepage": "https://webscraping.ai",
54
54
  "dependencies": {
55
55
  "commander": "^12.1.0",
56
- "webscraping-ai": "^4.0.0"
56
+ "webscraping-ai": "^4.1.0"
57
57
  },
58
58
  "overrides": {
59
59
  "esbuild": "^0.28.1"
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "webscraping-ai",
3
- "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — via the WebScraping.AI CLI.",
4
- "version": "1.0.0",
3
+ "description": "Fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages — and parsed Google search results, via the WebScraping.AI CLI.",
4
+ "version": "1.1.0",
5
5
  "author": {
6
6
  "name": "WebScraping.AI",
7
7
  "url": "https://webscraping.ai"
@@ -9,5 +9,15 @@
9
9
  "homepage": "https://github.com/webscraping-ai/webscraping-ai-cli#readme",
10
10
  "repository": "https://github.com/webscraping-ai/webscraping-ai-cli",
11
11
  "license": "MIT",
12
- "keywords": ["web-scraping", "html", "text-extraction", "ai-extraction", "scraping", "cli"]
12
+ "keywords": [
13
+ "web-scraping",
14
+ "html",
15
+ "text-extraction",
16
+ "ai-extraction",
17
+ "scraping",
18
+ "search",
19
+ "serp",
20
+ "google-search",
21
+ "cli"
22
+ ]
13
23
  }
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: webscraping-ai
3
- description: Use the WebScraping.AI CLI to fetch HTML, plain text, or AI-extracted data from any URL. Reach for this when you need real page content (including JS-rendered sites) instead of guessing or relying on stale knowledge.
3
+ description: Use the WebScraping.AI CLI to fetch HTML, plain text, or AI-extracted data from any URL, or to get parsed Google search results for a query. Reach for this when you need real page content (including JS-rendered sites) or current search results instead of guessing or relying on stale knowledge.
4
4
  ---
5
5
 
6
6
  # WebScraping.AI skill
@@ -25,6 +25,7 @@ Reach for `webscraping-ai` when you need to:
25
25
  - **Answer a question** about a specific page without scraping the whole thing first.
26
26
  - Render **JavaScript-heavy** pages (SPAs, lazy-loaded content) that a plain `curl` won't see.
27
27
  - Bypass **anti-bot protection** that's blocking `curl` / `fetch`.
28
+ - Get **current Google search results** for a query (to find candidate URLs, check rankings, or see what's out there) as clean JSON.
28
29
 
29
30
  Don't use it for:
30
31
 
@@ -34,21 +35,22 @@ Don't use it for:
34
35
 
35
36
  ## Quick reference
36
37
 
37
- The CLI exposes 7 subcommands, one per API endpoint:
38
+ The CLI exposes 8 subcommands, one per API endpoint:
38
39
 
39
- | Command | What it does |
40
- | ------------------ | ------------------------------------------------------- |
41
- | `html` | Fetch the full rendered HTML of a page |
42
- | `text` | Fetch only the visible text (plain / JSON / XML) |
43
- | `selected` | Fetch HTML of one CSS-selected area |
44
- | `selected-multiple`| Fetch HTML of multiple CSS-selected areas at once |
45
- | `ask` | Ask a natural-language question about a page (AI) |
46
- | `extract` | Extract structured fields with descriptions (AI) |
47
- | `account` | Show remaining API credits / quota |
40
+ | Command | What it does |
41
+ | ------------------- | ------------------------------------------------- |
42
+ | `html` | Fetch the full rendered HTML of a page |
43
+ | `text` | Fetch only the visible text (plain / JSON / XML) |
44
+ | `selected` | Fetch HTML of one CSS-selected area |
45
+ | `selected-multiple` | Fetch HTML of multiple CSS-selected areas at once |
46
+ | `ask` | Ask a natural-language question about a page (AI) |
47
+ | `extract` | Extract structured fields with descriptions (AI) |
48
+ | `serp` / `search` | Parsed Google search results for a query (JSON) |
49
+ | `account` | Show remaining API credits / quota |
48
50
 
49
51
  Authentication: set `WEBSCRAPING_AI_API_KEY`, pass `--api-key`, or run `webscraping-ai auth set <key>` once.
50
52
 
51
- Every command supports common flags: `--js/--no-js`, `--proxy {datacenter|residential|stealth}`, `--country <cc>`, `--device {desktop|mobile|tablet}`, `--timeout <ms>`, `--js-timeout <ms>`, `--wait-for <selector>`, `--headers '{"Cookie":"..."}'`, `--output <file>`.
53
+ Every page command supports common flags: `--js/--no-js`, `--proxy {datacenter|residential|stealth}`, `--country <cc>`, `--device {desktop|mobile|tablet}`, `--timeout <ms>`, `--js-timeout <ms>`, `--wait-for <selector>`, `--headers '{"Cookie":"..."}'`, `--output <file>`. `serp` is the exception: it takes a query, not a URL, and only `--engine`, `--gl <cc>`, `--hl <lang>`, `--page <n>`, `--output`.
52
54
 
53
55
  ## How to pick the right command
54
56
 
@@ -64,6 +66,8 @@ Every command supports common flags: `--js/--no-js`, `--proxy {datacenter|reside
64
66
 
65
67
  **Need a JSON object of typed fields**? → `extract --fields '{"title":"Main product title","price":"Current price including currency"}'`. Returns `{ result: { title: "...", price: "..." } }`.
66
68
 
69
+ **Starting from a query, not a URL** (find pages about X, check who ranks for a term, see what Google shows in a given country)? → `serp "<query>"`. Returns JSON with `organic_results` (`position`, `title`, `link`, `domain`, `snippet`), `related_searches`, and `pagination`. Use `--gl`/`--hl` for country/language and `--page 2` for the next 10 results. Don't `html` a google.com search URL and parse it yourself — `serp` is cheaper, more reliable, and already parsed. Then feed the `link`s you care about into `text`/`ask`/`extract`.
70
+
67
71
  **Need to check available credits**? → `account`.
68
72
 
69
73
  ## Example invocations
@@ -89,6 +93,13 @@ webscraping-ai ask https://shop.example.com/item -q "Is this product in stock?"
89
93
  webscraping-ai extract https://shop.example.com/item \
90
94
  --fields '{"title":"Product title","price":"Current price with currency","rating":"Average star rating as a number"}'
91
95
 
96
+ # Google results for a query (German results from Germany, page 2)
97
+ webscraping-ai serp coffee machines --gl de --hl de --page 2
98
+
99
+ # Search, then read the top result
100
+ webscraping-ai serp "rust async runtime comparison" --no-pretty | jq -r '.organic_results[0].link' \
101
+ | webscraping-ai text -
102
+
92
103
  # Pipe a list of URLs through the same command
93
104
  cat urls.txt | webscraping-ai text - --output ./articles.ndjson
94
105
 
@@ -103,6 +114,7 @@ webscraping-ai html https://geo.example.com \
103
114
  - **`--js` is on by default.** Pass `--no-js` for static pages — it's significantly faster and cheaper.
104
115
  - **`selected-multiple` returns nested arrays** (`Array<Array<string>>`) — known API response shape. The outer wrapper holds all matches concatenated; flatten in your script if needed.
105
116
  - **`extract` wraps its output** under a `result` key: the parsed shape is `{ "result": { ... } }`.
117
+ - **`serp` costs a flat 15 credits per search** (failed searches aren't charged) and returns at most 10 organic results per page. Optional fields (`snippet`, `date`, `related_searches`, `pagination.next`, `search_information.total_results`) may be absent.
106
118
  - **API key in URL.** The CLI sends the key as a query-string param. Don't log full requests in shared logs.
107
119
  - **Exit codes** are stable per error class (auth → 4, payment → 5, rate-limit → 6, server → 7, timeout → 8, connection → 9). Script around them.
108
120
 
@@ -110,7 +122,8 @@ webscraping-ai html https://geo.example.com \
110
122
 
111
123
  - Text/HTML responses print verbatim. JSON responses pretty-print on a TTY and single-line on a pipe.
112
124
  - `--output FILE` writes the result to a file instead of stdout.
113
- - For batched stdin input, each URL's response is emitted in order, separated by newlines.
125
+ - For batched stdin input, each URL's response is emitted in order, separated by newlines. `-o FILE` collects all of them in FILE. URL lists skip `#` comment lines; `serp -` skips only blank lines, so `#hashtag` queries are searched.
126
+ - `serp --page` must be an integer >= 1 (server rejects > 100 with a 400); anything else exits with code 2 before any request.
114
127
 
115
128
  ## See also
116
129