webscraping-ai-cli 1.0.0

package/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+All notable changes to `webscraping-ai-cli` are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] — unreleased
+
+Initial release.
+
+### Added
+
+- Seven endpoint subcommands wrapping the WebScraping.AI API:
+  `html`, `text`, `selected`, `selected-multiple`, `ask`, `extract`, `account`.
+- `auth set` / `auth show` / `auth clear` for managing the API key in
+  `~/.config/webscraping-ai/config.json` (alternative to the
+  `WEBSCRAPING_AI_API_KEY` environment variable).
+- `setup skill` installs a bundled AI agent skill into Claude Code, Cursor,
+  Windsurf, Kiro, OpenCode, Gemini CLI, GitHub Copilot, Augment, and Factory.
+- Two executable names: `webscraping-ai` (primary) and `wsai` (short alias).
+- Stdin batch mode: piping URLs to a command runs it per line.
+- Common scraping flags (`--js/--no-js`, `--proxy`, `--country`, `--device`,
+  `--timeout`, `--js-timeout`, `--wait-for`, `--headers`, `--js-script`,
+  `--custom-proxy`, `--error-on-404`, `--error-on-redirect`).
+- Output controls: `--output FILE`, `--format text|json`, `--pretty`.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) WebScraping.AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,120 @@
+# webscraping-ai-cli
+
+Official command-line client for the [WebScraping.AI](https://webscraping.ai) API. Fetch HTML, plain text, AI-extracted data, or single CSS selections from any URL — with JavaScript rendering, residential / stealth proxies, and per-country geolocation.
+
+Ships with an **AI agent skill** that you can install into Claude Code, Cursor, Windsurf, Kiro, OpenCode, Gemini CLI, GitHub Copilot, Augment, or Factory — your coding agent learns when and how to use the CLI for live page fetches.
+
+## Install
+
+```bash
+npm install -g webscraping-ai-cli
+# or run without installing
+npx -y webscraping-ai-cli --help
+```
+
+The package exposes two executables: `webscraping-ai` (primary) and `wsai` (short alias).
+
+## Authenticate
+
+Pick one:
+
+```bash
+# 1. Environment variable (simplest in CI)
+export WEBSCRAPING_AI_API_KEY=your-key-here
+
+# 2. Persist to ~/.config/webscraping-ai/config.json (chmod 0600)
+webscraping-ai auth set your-key-here
+
+# 3. Pass per command
+webscraping-ai html https://example.com --api-key your-key-here
+```
+
+## Commands
+
+```bash
+webscraping-ai html  <url> [flags]         # Full rendered HTML
+webscraping-ai text  <url> [flags]         # Visible text (plain | json | xml)
+webscraping-ai selected <url> --selector <css>           # One CSS area
+webscraping-ai selected-multiple <url> --selector a --selector b ...
+webscraping-ai ask <url> --question "..."  # AI question about the page
+webscraping-ai extract <url> --fields '{"title":"...","price":"..."}'
+webscraping-ai account                     # Remaining credits / quota
+```
+
+Common flags shared by every scrape command:
+
+| Flag                  | Purpose                                                  |
+| --------------------- | -------------------------------------------------------- |
+| `--js` / `--no-js`    | Toggle JavaScript rendering (default: on)                |
+| `--proxy TYPE`        | `datacenter` (default), `residential`, `stealth`         |
+| `--country CC`        | Two-letter proxy country code                            |
+| `--device TYPE`       | `desktop`, `mobile`, `tablet`                            |
+| `--timeout MS`        | Total page-retrieval timeout (max 30000)                 |
+| `--js-timeout MS`     | JS rendering time after page load (max 20000)            |
+| `--wait-for CSS`      | CSS selector to wait for                                 |
+| `--headers JSON`      | HTTP headers to forward; inline JSON or `@file.json`     |
+| `--custom-proxy URL`  | `http://user:pass@host:port`                             |
+| `--js-script CODE`    | Custom JS to evaluate on the target page                 |
+| `--error-on-404`      | Treat 404 responses as errors                            |
+| `--error-on-redirect` | Treat redirects as errors                                |
+| `--output FILE`       | Write to FILE instead of stdout                          |
+| `--pretty / --no-pretty` | Force pretty / single-line JSON                       |
+
+`<url>` may be `-` to read URLs from stdin (one per line; lines starting with `#` are ignored).
+
+## Install the AI agent skill
+
+```bash
+# Just for Claude Code
+webscraping-ai setup skill --editor claude-code
+
+# All supported editors
+webscraping-ai setup skill --all
+
+# List what's supported
+webscraping-ai setup list
+```
+
+Supported editor ids: `claude-code`, `cursor`, `windsurf`, `kiro`, `opencode`, `gemini`, `copilot`, `augment`, `factory`. Re-run with `--force` to overwrite existing installations.
+
+You can also install the skill via the Claude Code [Plugin Marketplace](https://docs.anthropic.com/en/docs/claude-code/plugins) using the bundled `plugins/webscraping-ai/.claude-plugin/marketplace.json`.
+
+## Exit codes
+
+| Code | Meaning                                          |
+| ---- | ------------------------------------------------ |
+| 0    | Success                                          |
+| 2    | Usage error (commander)                          |
+| 3    | 400 Bad Request from the API                     |
+| 4    | 403 Authentication error                         |
+| 5    | 402 Payment Required (out of credits)            |
+| 6    | 429 Rate limit                                   |
+| 7    | 500 Server error (or wrapped target-page error)  |
+| 8    | Timeout (504 or local AbortController)           |
+| 9    | Connection / DNS failure                         |
+| 1    | Anything else                                    |
+
+Scripts can branch on these without parsing stderr.
+
+## Examples
+
+```bash
+# Fast static fetch
+webscraping-ai html https://example.com --no-js
+
+# Residential proxy from Germany, wait for a selector
+webscraping-ai html https://geo.example.com \
+  --proxy residential --country de --wait-for '.product-card'
+
+# Pipe URLs through extract
+cat urls.txt | webscraping-ai extract - \
+  --fields '{"title":"Product title","price":"Current price"}' \
+  --output products.ndjson
+
+# Headers from a file
+webscraping-ai html https://example.com --headers @./headers.json
+```
+
+## License
+
+[MIT](./LICENSE) — © WebScraping.AI

package/dist/cli.js

@@ -0,0 +1,628 @@
+#!/usr/bin/env node
+
+// src/cli.ts
+import { Command as Command10 } from "commander";
+import { readFileSync } from "fs";
+import { dirname as dirname3, resolve as resolve2 } from "path";
+import { fileURLToPath as fileURLToPath2 } from "url";
+
+// src/commands/account.ts
+import { Command } from "commander";
+
+// src/lib/config.ts
+import { promises as fs } from "fs";
+import { homedir } from "os";
+import { dirname, join } from "path";
+var CONFIG_FILENAME = "config.json";
+function configDir() {
+  const xdg = process.env.XDG_CONFIG_HOME;
+  if (xdg && xdg.trim() !== "") return join(xdg, "webscraping-ai");
+  return join(homedir(), ".config", "webscraping-ai");
+}
+function configPath() {
+  return join(configDir(), CONFIG_FILENAME);
+}
+async function readPersistedConfig() {
+  try {
+    const text = await fs.readFile(configPath(), "utf8");
+    const parsed = JSON.parse(text);
+    if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
+      return parsed;
+    }
+    return {};
+  } catch (err) {
+    if (err.code === "ENOENT") return {};
+    throw err;
+  }
+}
+async function writePersistedConfig(config) {
+  const path = configPath();
+  await fs.mkdir(dirname(path), { recursive: true });
+  await fs.writeFile(path, JSON.stringify(config, null, 2) + "\n", { mode: 384 });
+}
+async function clearPersistedConfig() {
+  try {
+    await fs.unlink(configPath());
+    return true;
+  } catch (err) {
+    if (err.code === "ENOENT") return false;
+    throw err;
+  }
+}
+async function resolveApiKey(opts = {}) {
+  if (opts.flag && opts.flag.trim() !== "") return opts.flag.trim();
+  const env = process.env.WEBSCRAPING_AI_API_KEY;
+  if (env && env.trim() !== "") return env.trim();
+  const persisted = await readPersistedConfig();
+  if (persisted.apiKey && persisted.apiKey.trim() !== "") return persisted.apiKey.trim();
+  throw new Error(
+    "No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`."
+  );
+}
+function maskApiKey(key) {
+  if (key.length <= 8) return "*".repeat(key.length);
+  return `${key.slice(0, 4)}${"*".repeat(key.length - 8)}${key.slice(-4)}`;
+}
+
+// src/lib/output.ts
+import { promises as fs2 } from "fs";
+async function emit(value, opts = {}) {
+  const text = stringify(value, opts);
+  if (opts.output && opts.output.trim() !== "") {
+    await fs2.writeFile(opts.output, text + (text.endsWith("\n") ? "" : "\n"));
+    return;
+  }
+  process.stdout.write(text);
+  if (!text.endsWith("\n")) process.stdout.write("\n");
+}
+function stringify(value, opts = {}) {
+  if (typeof value === "string") return value;
+  const pretty = opts.pretty ?? process.stdout.isTTY ?? false;
+  return JSON.stringify(value, null, pretty ? 2 : 0);
+}
+
+// src/lib/sdk.ts
+import { WebScrapingAI } from "webscraping-ai";
+function createClient(options) {
+  const inner = new WebScrapingAI({
+    apiKey: options.apiKey,
+    timeoutMs: options.requestTimeoutMs
+  });
+  const tag = { from_cli: true };
+  return new Proxy(inner, {
+    get(target, prop, receiver) {
+      const value = Reflect.get(target, prop, receiver);
+      if (typeof value !== "function") return value;
+      switch (prop) {
+        case "html":
+          return (o) => target.html({ ...o, ...tag });
+        case "text":
+          return (o) => target.text({ ...o, ...tag });
+        case "selected":
+          return (o) => target.selected({ ...o, ...tag });
+        case "selectedMultiple":
+          return (o) => target.selectedMultiple({ ...o, ...tag });
+        case "question":
+          return (o) => target.question({ ...o, ...tag });
+        case "fields":
+          return (o) => target.fields({ ...o, ...tag });
+        case "account":
+          return value.bind(target);
+        default:
+          return typeof value === "function" ? value.bind(target) : value;
+      }
+    }
+  });
+}
+
+// src/commands/account.ts
+function accountCommand() {
+  return new Command("account").description("Show account quota and usage information").option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").option("--no-pretty", "force single-line JSON output").action(async (opts) => {
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const client = createClient({ apiKey });
+    const result = await client.account();
+    await emit(result, opts);
+  });
+}
+
+// src/commands/ask.ts
+import { Command as Command2, Option as Option2 } from "commander";
+
+// src/lib/buildRequest.ts
+function buildCommonOptions(flags, resolved = {}) {
+  const out = {};
+  if (flags.js !== void 0) out.js = flags.js;
+  if (flags.proxy !== void 0) out.proxy = flags.proxy;
+  if (flags.country !== void 0) out.country = flags.country;
+  if (flags.device !== void 0) out.device = flags.device;
+  if (flags.timeout !== void 0) out.timeout = flags.timeout;
+  if (flags.jsTimeout !== void 0) out.js_timeout = flags.jsTimeout;
+  if (flags.waitFor !== void 0) out.wait_for = flags.waitFor;
+  if (flags.jsScript !== void 0) out.js_script = flags.jsScript;
+  if (flags.customProxy !== void 0) out.custom_proxy = flags.customProxy;
+  if (flags.errorOn404) out.error_on_404 = true;
+  if (flags.errorOnRedirect) out.error_on_redirect = true;
+  if (resolved.headers) out.headers = resolved.headers;
+  return out;
+}
+function buildHtmlOptions(url, flags, resolved, extras = {}) {
+  const out = { url, ...buildCommonOptions(flags, resolved) };
+  if (flags.format) out.format = flags.format;
+  if (extras.returnScriptResult) out.return_script_result = true;
+  return out;
+}
+function buildTextOptions(url, flags, resolved, extras = {}) {
+  const out = { url, ...buildCommonOptions(flags, resolved) };
+  if (extras.textFormat) out.text_format = extras.textFormat;
+  if (extras.returnLinks) out.return_links = true;
+  return out;
+}
+function buildSelectedOptions(url, selector, flags, resolved) {
+  const out = { url, selector, ...buildCommonOptions(flags, resolved) };
+  if (flags.format) out.format = flags.format;
+  return out;
+}
+function buildSelectedMultipleOptions(url, selectors, flags, resolved) {
+  if (selectors.length === 0) {
+    throw new Error("selected-multiple requires at least one --selector");
+  }
+  return { url, selectors, ...buildCommonOptions(flags, resolved) };
+}
+function buildQuestionOptions(url, question, flags, resolved) {
+  const out = { url, question, ...buildCommonOptions(flags, resolved) };
+  if (flags.format) out.format = flags.format;
+  return out;
+}
+function buildFieldsOptions(url, fields, flags, resolved) {
+  return { url, fields, ...buildCommonOptions(flags, resolved) };
+}
+
+// src/lib/options.ts
+import { promises as fs3 } from "fs";
+import { Option } from "commander";
+function addCommonScrapeOptions(cmd) {
+  return cmd.option("-k, --api-key <key>", "API key (overrides $WEBSCRAPING_AI_API_KEY and config file)").option("-o, --output <file>", "write result to FILE instead of stdout").option("--pretty", "pretty-print JSON output (default on TTY)").addOption(
+    new Option("--no-pretty", "force single-line JSON output").conflicts("pretty")
+  ).option("--js", "execute JS via a headless browser (default: enabled)").option("--no-js", "disable JS execution (faster, cheaper)").addOption(
+    new Option("--proxy <type>", "proxy pool").choices([
+      "datacenter",
+      "residential",
+      "stealth"
+    ])
+  ).option("--country <code>", "two-letter proxy country code, e.g. us, gb, de").addOption(
+    new Option("--device <type>", "device emulation").choices(["desktop", "mobile", "tablet"])
+  ).option("--timeout <ms>", "page-retrieval timeout in ms (max 30000)", integerParser).option("--js-timeout <ms>", "JS rendering time in ms after load (max 20000)", integerParser).option("--wait-for <selector>", "CSS selector to wait for before returning content").option(
+    "--headers <json>",
+    "HTTP headers to forward to the target page; JSON object or @file.json"
+  ).option("--js-script <code>", "custom JS to evaluate on the target page").option("--custom-proxy <url>", "custom proxy URL: http://user:pass@host:port").option("--error-on-404", "return error on 404").option("--error-on-redirect", "return error on redirect");
+}
+function integerParser(value) {
+  const n = Number.parseInt(value, 10);
+  if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);
+  return n;
+}
+async function parseHeaders(input) {
+  if (!input || input.trim() === "") return void 0;
+  const text = input.startsWith("@") ? await fs3.readFile(input.slice(1), "utf8") : input;
+  return parseJsonObject(text, "headers");
+}
+async function parseFields(input) {
+  const text = input.startsWith("@") ? await fs3.readFile(input.slice(1), "utf8") : input;
+  const parsed = parseJsonObject(text, "fields");
+  if (!parsed || Object.keys(parsed).length === 0) {
+    throw new Error("fields must be a non-empty JSON object of { fieldName: description }");
+  }
+  return parsed;
+}
+function parseJsonObject(text, label) {
+  let parsed;
+  try {
+    parsed = JSON.parse(text);
+  } catch (err) {
+    throw new Error(`${label}: invalid JSON \u2014 ${err.message}`);
+  }
+  if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
+    throw new Error(`${label} must be a JSON object`);
+  }
+  const out = {};
+  for (const [k, v] of Object.entries(parsed)) {
+    if (typeof v !== "string") throw new Error(`${label}.${k}: value must be a string`);
+    out[k] = v;
+  }
+  return out;
+}
+
+// src/lib/stdin.ts
+import readline from "readline";
+function urlIsStdin(url) {
+  return url === "-";
+}
+async function* readUrlsFromStdin() {
+  const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });
+  for await (const raw of rl) {
+    const line = raw.trim();
+    if (line === "" || line.startsWith("#")) continue;
+    yield line;
+  }
+}
+async function* urlIterator(arg) {
+  if (urlIsStdin(arg)) {
+    yield* readUrlsFromStdin();
+    return;
+  }
+  yield arg;
+}
+
+// src/commands/ask.ts
+function askCommand() {
+  const cmd = new Command2("ask").description("Ask an AI question about a page (wraps /ai/question)").argument("<url>", "page URL (or `-` to read URLs from stdin)").requiredOption("-q, --question <text>", "the question / instructions for the model").addOption(new Option2("--format <fmt>", "response wrapping").choices(["text", "json"]));
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    if (!opts.question) throw new Error("--question is required");
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });
+      const result = await client.question(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+
+// src/commands/auth.ts
+import { Command as Command3 } from "commander";
+function authCommand() {
+  const cmd = new Command3("auth").description("Manage your stored API key");
+  cmd.command("set <key>").description(`Save an API key to ${configPath()}`).action(async (key) => {
+    if (!key || key.trim() === "") throw new Error("API key is empty");
+    await writePersistedConfig({ apiKey: key.trim() });
+    process.stdout.write(`API key saved to ${configPath()}
+`);
+  });
+  cmd.command("show").description("Display the currently stored API key (masked)").action(async () => {
+    const persisted = await readPersistedConfig();
+    const env = process.env.WEBSCRAPING_AI_API_KEY;
+    if (env && env.trim() !== "") {
+      process.stdout.write(`Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}
+`);
+    }
+    if (persisted.apiKey && persisted.apiKey.trim() !== "") {
+      process.stdout.write(
+        `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}
+`
+      );
+    }
+    if (!env && !persisted.apiKey) {
+      process.stdout.write("No API key configured. Run `webscraping-ai auth set <key>`.\n");
+    }
+  });
+  cmd.command("clear").description("Remove the stored API key file").action(async () => {
+    const removed = await clearPersistedConfig();
+    process.stdout.write(removed ? `Removed ${configPath()}
+` : "No stored API key to remove.\n");
+  });
+  return cmd;
+}
+
+// src/commands/extract.ts
+import { Command as Command4 } from "commander";
+function extractCommand() {
+  const cmd = new Command4("extract").description("Extract structured fields from a page using AI (wraps /ai/fields)").argument("<url>", "page URL (or `-` to read URLs from stdin)").requiredOption(
+    "-f, --fields <json>",
+    "fields to extract \u2014 JSON object {name: description} or @file.json"
+  );
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    if (!opts.fields) throw new Error("--fields is required");
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const fields = await parseFields(opts.fields);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });
+      const result = await client.fields(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+
+// src/commands/html.ts
+import { Command as Command5, Option as Option3 } from "commander";
+function htmlCommand() {
+  const cmd = new Command5("html").description("Fetch the full HTML of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(new Option3("--return-script-result", "return the result of --js-script instead of HTML")).addOption(new Option3("--format <fmt>", "response wrapping").choices(["text", "json"]));
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildHtmlOptions(target, opts, { headers }, {
+        returnScriptResult: opts.returnScriptResult
+      });
+      const result = await client.html(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+
+// src/commands/selected.ts
+import { Command as Command6, Option as Option4 } from "commander";
+function selectedCommand() {
+  const cmd = new Command6("selected").description("Fetch HTML of one CSS-selected page area").argument("<url>", "page URL (or `-` to read URLs from stdin)").requiredOption("-s, --selector <css>", "CSS selector for the target element(s)").addOption(new Option4("--format <fmt>", "response wrapping").choices(["text", "json"]));
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    if (!opts.selector) throw new Error("--selector is required");
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });
+      const result = await client.selected(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+
+// src/commands/selectedMultiple.ts
+import { Command as Command7 } from "commander";
+function selectedMultipleCommand() {
+  const cmd = new Command7("selected-multiple").description("Fetch HTML of multiple CSS-selected page areas").argument("<url>", "page URL (or `-` to read URLs from stdin)").option(
+    "-s, --selector <css>",
+    "CSS selector \u2014 pass multiple times for multiple selectors",
+    collectMulti,
+    []
+  );
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    const selectors = opts.selector ?? [];
+    if (selectors.length === 0) {
+      throw new Error("at least one --selector is required");
+    }
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });
+      const result = await client.selectedMultiple(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+function collectMulti(value, previous) {
+  return [...previous, value];
+}
+
+// src/commands/setupSkill.ts
+import { Command as Command8 } from "commander";
+
+// src/lib/skill.ts
+import { promises as fs4 } from "fs";
+import { homedir as homedir2 } from "os";
+import { dirname as dirname2, join as join2, resolve } from "path";
+import { fileURLToPath } from "url";
+var EDITORS = [
+  // Claude Code: user-scope skills live under ~/.claude/skills/<name>/
+  { id: "claude-code", label: "Claude Code", mode: "dir", kind: "user", path: ".claude/skills/webscraping-ai" },
+  // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill
+  // dir, so this is the editor's "install" surface.
+  { id: "cursor", label: "Cursor", mode: "file", kind: "project", path: ".cursor/rules/webscraping-ai.mdc" },
+  // Windsurf: project-scope rules. Same shape as Cursor.
+  { id: "windsurf", label: "Windsurf", mode: "file", kind: "project", path: ".windsurf/rules/webscraping-ai.md" },
+  // Kiro: user-scope steering directory.
+  { id: "kiro", label: "Kiro", mode: "dir", kind: "user", path: ".kiro/steering/webscraping-ai" },
+  // OpenCode: project-scope skills (mirrors Claude shape).
+  { id: "opencode", label: "OpenCode", mode: "dir", kind: "project", path: ".opencode/skills/webscraping-ai" },
+  // Gemini CLI: user-scope extension dir with a GEMINI.md.
+  { id: "gemini", label: "Gemini CLI", mode: "file", kind: "user", path: ".gemini/extensions/webscraping-ai/GEMINI.md" },
+  // GitHub Copilot: project-scope custom instructions file. Single file.
+  { id: "copilot", label: "GitHub Copilot", mode: "file", kind: "project", path: ".github/copilot-instructions.md" },
+  // Augment: project-scope rules file.
+  { id: "augment", label: "Augment", mode: "file", kind: "project", path: ".augment/rules/webscraping-ai.md" },
+  // Factory: project-scope rules file.
+  { id: "factory", label: "Factory", mode: "file", kind: "project", path: ".factory/rules/webscraping-ai.md" }
+];
+function editorById(id) {
+  return EDITORS.find((e) => e.id === id);
+}
+function bundledSkillDir() {
+  const here = dirname2(fileURLToPath(import.meta.url));
+  return resolve(here, "..", "plugins", "webscraping-ai", "skills", "webscraping-ai");
+}
+function resolveTarget(editor, opts = {}) {
+  const root = editor.kind === "user" ? opts.home ?? homedir2() : opts.cwd ?? process.cwd();
+  return { editor, destination: join2(root, editor.path) };
+}
+async function installSkill(target, opts = {}) {
+  const source = opts.sourceDir ?? bundledSkillDir();
+  const exists = await pathExists(target.destination);
+  if (exists && !opts.force) {
+    return { target, status: "skipped-exists" };
+  }
+  if (target.editor.mode === "dir") {
+    if (exists) await fs4.rm(target.destination, { recursive: true, force: true });
+    await copyDirectory(source, target.destination);
+  } else {
+    const skillBody = await fs4.readFile(join2(source, "SKILL.md"), "utf8");
+    await fs4.mkdir(dirname2(target.destination), { recursive: true });
+    await fs4.writeFile(target.destination, skillBody);
+  }
+  return { target, status: exists ? "overwritten" : "installed" };
+}
+async function pathExists(path) {
+  try {
+    await fs4.access(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
+async function copyDirectory(source, destination) {
+  await fs4.cp(source, destination, { recursive: true, force: true });
+}
+
+// src/commands/setupSkill.ts
+function setupCommand() {
+  const cmd = new Command8("setup").description("Install the AI agent skill into your editor(s)");
+  cmd.command("skill").description("Install the WebScraping.AI skill into one or more AI coding editors").option(
+    "-e, --editor <id>",
+    `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(", ")}`,
+    collect,
+    []
+  ).option("-a, --all", "install into every supported editor").option("-f, --force", "overwrite existing skill files").action(async (opts) => {
+    const selected = resolveSelection(opts);
+    if (selected.length === 0) {
+      throw new Error(
+        "no editor selected \u2014 pass --all or --editor <id> (e.g. --editor claude-code)"
+      );
+    }
+    for (const editor of selected) {
+      const target = resolveTarget(editor);
+      const result = await installSkill(target, { force: opts.force });
+      process.stdout.write(`${editor.label}: ${result.status} \u2192 ${target.destination}
+`);
+    }
+  });
+  cmd.command("list", { hidden: false }).description("List supported editors for `setup skill`").action(() => {
+    for (const e of EDITORS) {
+      const target = resolveTarget(e);
+      process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}
+`);
+    }
+  });
+  return cmd;
+}
+function collect(value, previous) {
+  return [...previous, value];
+}
+function resolveSelection(opts) {
+  if (opts.all) return [...EDITORS];
+  const ids = opts.editor ?? [];
+  return ids.map((id) => {
+    const editor = editorById(id);
+    if (!editor) throw new Error(`unknown editor: ${id}`);
+    return editor;
+  });
+}
+
+// src/commands/text.ts
+import { Command as Command9, Option as Option5 } from "commander";
+function textCommand() {
+  const cmd = new Command9("text").description("Fetch the visible text of a page").argument("<url>", "page URL (or `-` to read URLs from stdin)").addOption(
+    new Option5("--text-format <fmt>", "text response format").choices(["plain", "json", "xml"])
+  ).option("--return-links", "include page links (text-format=json only)");
+  addCommonScrapeOptions(cmd);
+  cmd.action(async (url, opts) => {
+    const apiKey = await resolveApiKey({ flag: opts.apiKey });
+    const headers = await parseHeaders(opts.headers);
+    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });
+    for await (const target of urlIterator(url)) {
+      const sdkOptions = buildTextOptions(target, opts, { headers }, {
+        textFormat: opts.textFormat,
+        returnLinks: opts.returnLinks
+      });
+      const result = await client.text(sdkOptions);
+      await emit(result, opts);
+    }
+  });
+  return cmd;
+}
+
+// src/lib/errors.ts
+import {
+  APIConnectionError,
+  APIError,
+  APITimeoutError,
+  AuthenticationError,
+  BadRequestError,
+  GatewayTimeoutError,
+  PaymentRequiredError,
+  RateLimitError,
+  ServerError,
+  WebScrapingAIError
+} from "webscraping-ai";
+var EXIT_CODES = {
+  ok: 0,
+  generic: 1,
+  usage: 2,
+  badRequest: 3,
+  auth: 4,
+  paymentRequired: 5,
+  rateLimit: 6,
+  serverError: 7,
+  timeout: 8,
+  connection: 9
+};
+function exitCodeFor(err) {
+  if (err instanceof BadRequestError) return EXIT_CODES.badRequest;
+  if (err instanceof AuthenticationError) return EXIT_CODES.auth;
+  if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;
+  if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;
+  if (err instanceof ServerError) return EXIT_CODES.serverError;
+  if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;
+  if (err instanceof APITimeoutError) return EXIT_CODES.timeout;
+  if (err instanceof APIConnectionError) return EXIT_CODES.connection;
+  if (err instanceof APIError) return EXIT_CODES.serverError;
+  if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;
+  return EXIT_CODES.generic;
+}
+function formatError(err) {
+  if (err instanceof APIError) {
+    const parts = [err.message];
+    if (err.status) parts.push(`(HTTP ${err.status})`);
+    return parts.join(" ");
+  }
+  if (err instanceof Error) return err.message;
+  return String(err);
+}
+
+// src/cli.ts
+function packageVersion() {
+  const here = dirname3(fileURLToPath2(import.meta.url));
+  const pkgPath = resolve2(here, "..", "package.json");
+  const raw = JSON.parse(readFileSync(pkgPath, "utf8"));
+  return raw.version;
+}
+function buildProgram() {
+  const program = new Command10();
+  program.name("webscraping-ai").description(
+    "Official CLI for the WebScraping.AI API. Fetch pages, ask AI, extract structured data.\nSet $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first."
+  ).version(packageVersion(), "-v, --version", "output the version number").showHelpAfterError();
+  program.addCommand(htmlCommand());
+  program.addCommand(textCommand());
+  program.addCommand(selectedCommand());
+  program.addCommand(selectedMultipleCommand());
+  program.addCommand(askCommand());
+  program.addCommand(extractCommand());
+  program.addCommand(accountCommand());
+  program.addCommand(authCommand());
+  program.addCommand(setupCommand());
+  return program;
+}
+async function main(argv) {
+  const program = buildProgram();
+  try {
+    await program.parseAsync(argv);
+  } catch (err) {
+    process.stderr.write(`Error: ${formatError(err)}
+`);
+    process.exit(exitCodeFor(err));
+  }
+}
+var entryPath = process.argv[1] ? fileURLToPath2(import.meta.url) : "";
+if (entryPath && process.argv[1] === entryPath) {
+  main(process.argv).catch((err) => {
+    process.stderr.write(`Error: ${formatError(err)}
+`);
+    process.exit(exitCodeFor(err));
+  });
+}
+export {
+  buildProgram
+};
+//# sourceMappingURL=cli.js.map

package/dist/cli.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/cli.ts","../src/commands/account.ts","../src/lib/config.ts","../src/lib/output.ts","../src/lib/sdk.ts","../src/commands/ask.ts","../src/lib/buildRequest.ts","../src/lib/options.ts","../src/lib/stdin.ts","../src/commands/auth.ts","../src/commands/extract.ts","../src/commands/html.ts","../src/commands/selected.ts","../src/commands/selectedMultiple.ts","../src/commands/setupSkill.ts","../src/lib/skill.ts","../src/commands/text.ts","../src/lib/errors.ts"],"sourcesContent":["/**\n * `webscraping-ai` / `wsai` CLI entry point.\n *\n * Wires every subcommand into a single commander root. Errors thrown from\n * subcommand actions land in the top-level `.catch`, which maps SDK\n * exception classes to stable exit codes (see `lib/errors.ts`).\n */\n\nimport { Command } from 'commander';\nimport { readFileSync } from 'node:fs';\nimport { dirname, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nimport { accountCommand } from './commands/account.js';\nimport { askCommand } from './commands/ask.js';\nimport { authCommand } from './commands/auth.js';\nimport { extractCommand } from './commands/extract.js';\nimport { htmlCommand } from './commands/html.js';\nimport { selectedCommand } from './commands/selected.js';\nimport { selectedMultipleCommand } from './commands/selectedMultiple.js';\nimport { setupCommand } from './commands/setupSkill.js';\nimport { textCommand } from './commands/text.js';\nimport { exitCodeFor, formatError } from './lib/errors.js';\n\nfunction packageVersion(): string {\n  // Read package.json at runtime so we don't need a code-gen step. The dist\n  // bundle sits at `dist/cli.js`, so package.json is one directory up.\n  const here = dirname(fileURLToPath(import.meta.url));\n  const pkgPath = resolve(here, '..', 'package.json');\n  const raw = JSON.parse(readFileSync(pkgPath, 'utf8')) as { version: string };\n  return raw.version;\n}\n\nexport function buildProgram(): Command {\n  const program = new Command();\n  program\n    .name('webscraping-ai')\n    .description(\n      'Official CLI for the WebScraping.AI API. Fetch pages, ask AI, extract structured data.\\n' +\n        'Set $WEBSCRAPING_AI_API_KEY or run `webscraping-ai auth set <key>` first.',\n    )\n    .version(packageVersion(), '-v, --version', 'output the version number')\n    .showHelpAfterError();\n\n  program.addCommand(htmlCommand());\n  program.addCommand(textCommand());\n  program.addCommand(selectedCommand());\n  program.addCommand(selectedMultipleCommand());\n  program.addCommand(askCommand());\n  program.addCommand(extractCommand());\n  program.addCommand(accountCommand());\n  program.addCommand(authCommand());\n  program.addCommand(setupCommand());\n\n  return program;\n}\n\nasync function main(argv: string[]): Promise<void> {\n  const program = buildProgram();\n  try {\n    await program.parseAsync(argv);\n  } catch (err) {\n    process.stderr.write(`Error: ${formatError(err)}\\n`);\n    process.exit(exitCodeFor(err));\n  }\n}\n\n// Only execute when run as a script. When imported (e.g. from tests),\n// `buildProgram` is the public surface and `main` stays dormant.\nconst entryPath = process.argv[1] ? fileURLToPath(import.meta.url) : '';\nif (entryPath && process.argv[1] === entryPath) {\n  main(process.argv).catch((err: unknown) => {\n    process.stderr.write(`Error: ${formatError(err)}\\n`);\n    process.exit(exitCodeFor(err));\n  });\n}\n","import { Command } from 'commander';\n\nimport { resolveApiKey } from '../lib/config.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\n\ninterface AccountFlags {\n  apiKey?: string;\n  output?: string;\n  pretty?: boolean;\n}\n\nexport function accountCommand(): Command {\n  return new Command('account')\n    .description('Show account quota and usage information')\n    .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n    .option('-o, --output <file>', 'write result to FILE instead of stdout')\n    .option('--pretty', 'pretty-print JSON output (default on TTY)')\n    .option('--no-pretty', 'force single-line JSON output')\n    .action(async (opts: AccountFlags) => {\n      const apiKey = await resolveApiKey({ flag: opts.apiKey });\n      const client = createClient({ apiKey });\n      const result = await client.account();\n      await emit(result, opts);\n    });\n}\n","/**\n * API key resolution and persisted config.\n *\n * Resolution order (first non-empty wins):\n *   1. `--api-key` flag                  (handled by the caller, passed in)\n *   2. `WEBSCRAPING_AI_API_KEY` env var\n *   3. `~/.config/webscraping-ai/config.json` (or `$XDG_CONFIG_HOME/...`)\n *\n * The config file is plain JSON with `{ \"apiKey\": \"...\" }`. We chmod it to\n * 0600 so it's not world-readable.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join } from 'node:path';\n\nconst CONFIG_FILENAME = 'config.json';\n\nexport function configDir(): string {\n  const xdg = process.env.XDG_CONFIG_HOME;\n  if (xdg && xdg.trim() !== '') return join(xdg, 'webscraping-ai');\n  return join(homedir(), '.config', 'webscraping-ai');\n}\n\nexport function configPath(): string {\n  return join(configDir(), CONFIG_FILENAME);\n}\n\nexport interface PersistedConfig {\n  apiKey?: string;\n}\n\nexport async function readPersistedConfig(): Promise<PersistedConfig> {\n  try {\n    const text = await fs.readFile(configPath(), 'utf8');\n    const parsed = JSON.parse(text) as unknown;\n    if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {\n      return parsed as PersistedConfig;\n    }\n    return {};\n  } catch (err) {\n    if ((err as NodeJS.ErrnoException).code === 'ENOENT') return {};\n    throw err;\n  }\n}\n\nexport async function writePersistedConfig(config: PersistedConfig): Promise<void> {\n  const path = configPath();\n  await fs.mkdir(dirname(path), { recursive: true });\n  await fs.writeFile(path, JSON.stringify(config, null, 2) + '\\n', { mode: 0o600 });\n}\n\nexport async function clearPersistedConfig(): Promise<boolean> {\n  try {\n    await fs.unlink(configPath());\n    return true;\n  } catch (err) {\n    if ((err as NodeJS.ErrnoException).code === 'ENOENT') return false;\n    throw err;\n  }\n}\n\nexport interface ResolveApiKeyOptions {\n  /** `--api-key` flag value, if the user passed one. */\n  flag?: string;\n}\n\nexport async function resolveApiKey(opts: ResolveApiKeyOptions = {}): Promise<string> {\n  if (opts.flag && opts.flag.trim() !== '') return opts.flag.trim();\n\n  const env = process.env.WEBSCRAPING_AI_API_KEY;\n  if (env && env.trim() !== '') return env.trim();\n\n  const persisted = await readPersistedConfig();\n  if (persisted.apiKey && persisted.apiKey.trim() !== '') return persisted.apiKey.trim();\n\n  throw new Error(\n    'No API key found. Set $WEBSCRAPING_AI_API_KEY, pass --api-key, or run `webscraping-ai auth set <key>`.',\n  );\n}\n\nexport function maskApiKey(key: string): string {\n  if (key.length <= 8) return '*'.repeat(key.length);\n  return `${key.slice(0, 4)}${'*'.repeat(key.length - 8)}${key.slice(-4)}`;\n}\n","/**\n * Output handling for command results.\n *\n * All command handlers funnel through `emit()`. The result type drives format:\n *   - strings → written verbatim\n *   - objects/arrays → JSON.stringified (pretty by default; `--pretty` is the\n *     default for terminals, single-line for pipes/redirects)\n *\n * `--output FILE` writes to disk instead of stdout.\n */\n\nimport { promises as fs } from 'node:fs';\n\nexport interface EmitOptions {\n  output?: string;\n  pretty?: boolean;\n}\n\nexport async function emit(value: unknown, opts: EmitOptions = {}): Promise<void> {\n  const text = stringify(value, opts);\n\n  if (opts.output && opts.output.trim() !== '') {\n    await fs.writeFile(opts.output, text + (text.endsWith('\\n') ? '' : '\\n'));\n    return;\n  }\n\n  process.stdout.write(text);\n  if (!text.endsWith('\\n')) process.stdout.write('\\n');\n}\n\nexport function stringify(value: unknown, opts: EmitOptions = {}): string {\n  if (typeof value === 'string') return value;\n\n  const pretty = opts.pretty ?? process.stdout.isTTY ?? false;\n  return JSON.stringify(value, null, pretty ? 2 : 0);\n}\n","/**\n * SDK adapter.\n *\n * Wraps the published `webscraping-ai` Node SDK. The CLI sends a\n * `from_cli=true` analytics flag on every request (mirrors `from_n8n` and\n * `from_mcp_server` used by the n8n node and MCP server). The SDK's typed\n * options don't accept arbitrary keys, so we go through a single cast point\n * here rather than scattering casts across each command.\n */\n\nimport { WebScrapingAI } from 'webscraping-ai';\nimport type {\n  FieldsOptions,\n  HtmlOptions,\n  QuestionOptions,\n  SelectedMultipleOptions,\n  SelectedOptions,\n  TextOptions,\n} from 'webscraping-ai';\n\nexport interface CreateClientOptions {\n  apiKey: string;\n  /** Per-request timeout in ms. Maps to the SDK's `timeoutMs`. */\n  requestTimeoutMs?: number;\n}\n\n/**\n * Returns a `WebScrapingAI` instance with the CLI's analytics tag baked into\n * every endpoint method. The methods preserve the SDK's public signatures.\n */\nexport function createClient(options: CreateClientOptions): WebScrapingAI {\n  const inner = new WebScrapingAI({\n    apiKey: options.apiKey,\n    timeoutMs: options.requestTimeoutMs,\n  });\n\n  // Wrap each endpoint method so it injects the CLI analytics tag. We splice\n  // the tag in at this seam rather than at every callsite. The cast is local;\n  // the SDK passes unknown query params through to the wire.\n  const tag = { from_cli: true } as unknown as Record<string, never>;\n\n  return new Proxy(inner, {\n    get(target, prop, receiver) {\n      const value = Reflect.get(target, prop, receiver);\n      if (typeof value !== 'function') return value;\n      switch (prop) {\n        case 'html':\n          return (o: HtmlOptions) => target.html({ ...o, ...tag });\n        case 'text':\n          return (o: TextOptions) => target.text({ ...o, ...tag });\n        case 'selected':\n          return (o: SelectedOptions) => target.selected({ ...o, ...tag });\n        case 'selectedMultiple':\n          return (o: SelectedMultipleOptions) => target.selectedMultiple({ ...o, ...tag });\n        case 'question':\n          return (o: QuestionOptions) => target.question({ ...o, ...tag });\n        case 'fields':\n          return (o: FieldsOptions) => target.fields({ ...o, ...tag });\n        case 'account':\n          // `account` takes no options; the SDK builds the request itself, so\n          // there's no seam here for the analytics tag — accepted limitation.\n          return value.bind(target);\n        default:\n          return typeof value === 'function' ? value.bind(target) : value;\n      }\n    },\n  });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildQuestionOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface AskFlags extends CommonRawFlags {\n  question?: string;\n}\n\nexport function askCommand(): Command {\n  const cmd = new Command('ask')\n    .description('Ask an AI question about a page (wraps /ai/question)')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .requiredOption('-q, --question <text>', 'the question / instructions for the model')\n    .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: AskFlags) => {\n    if (!opts.question) throw new Error('--question is required');\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildQuestionOptions(target, opts.question, opts, { headers });\n      const result = await client.question(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n","/**\n * Pure builder that maps `--flag` values into SDK option objects.\n *\n * Extracted as a separate function so unit tests can pin the mapping without\n * spinning up commander or making HTTP calls. Each command handler reads its\n * own commander options, awaits `parseHeaders` / `parseFields`, then calls\n * the matching `build*` function here.\n */\n\nimport type {\n  CommonRequestOptions,\n  FieldsOptions,\n  HtmlOptions,\n  QuestionOptions,\n  SelectedMultipleOptions,\n  SelectedOptions,\n  TextOptions,\n} from 'webscraping-ai';\n\nimport type { CommonRawFlags } from './options.js';\n\nexport interface ResolvedCommonOptions {\n  headers?: Record<string, string>;\n}\n\nexport function buildCommonOptions(\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions = {},\n): CommonRequestOptions {\n  const out: CommonRequestOptions = {};\n  // `flags.js` is `false` when `--no-js`, `true` when `--js`, `undefined`\n  // when neither passed. Pass through only when explicit, so the API default\n  // applies otherwise.\n  if (flags.js !== undefined) out.js = flags.js;\n  if (flags.proxy !== undefined) out.proxy = flags.proxy;\n  if (flags.country !== undefined) out.country = flags.country;\n  if (flags.device !== undefined) out.device = flags.device;\n  if (flags.timeout !== undefined) out.timeout = flags.timeout;\n  if (flags.jsTimeout !== undefined) out.js_timeout = flags.jsTimeout;\n  if (flags.waitFor !== undefined) out.wait_for = flags.waitFor;\n  if (flags.jsScript !== undefined) out.js_script = flags.jsScript;\n  if (flags.customProxy !== undefined) out.custom_proxy = flags.customProxy;\n  if (flags.errorOn404) out.error_on_404 = true;\n  if (flags.errorOnRedirect) out.error_on_redirect = true;\n  if (resolved.headers) out.headers = resolved.headers;\n  return out;\n}\n\nexport interface HtmlExtras {\n  returnScriptResult?: boolean;\n}\n\nexport function buildHtmlOptions(\n  url: string,\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n  extras: HtmlExtras = {},\n): HtmlOptions {\n  const out: HtmlOptions = { url, ...buildCommonOptions(flags, resolved) };\n  if (flags.format) out.format = flags.format;\n  if (extras.returnScriptResult) out.return_script_result = true;\n  return out;\n}\n\nexport interface TextExtras {\n  textFormat?: 'plain' | 'json' | 'xml';\n  returnLinks?: boolean;\n}\n\nexport function buildTextOptions(\n  url: string,\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n  extras: TextExtras = {},\n): TextOptions {\n  const out: TextOptions = { url, ...buildCommonOptions(flags, resolved) };\n  if (extras.textFormat) out.text_format = extras.textFormat;\n  if (extras.returnLinks) out.return_links = true;\n  return out;\n}\n\nexport function buildSelectedOptions(\n  url: string,\n  selector: string,\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n): SelectedOptions {\n  const out: SelectedOptions = { url, selector, ...buildCommonOptions(flags, resolved) };\n  if (flags.format) out.format = flags.format;\n  return out;\n}\n\nexport function buildSelectedMultipleOptions(\n  url: string,\n  selectors: readonly string[],\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n): SelectedMultipleOptions {\n  if (selectors.length === 0) {\n    throw new Error('selected-multiple requires at least one --selector');\n  }\n  return { url, selectors, ...buildCommonOptions(flags, resolved) };\n}\n\nexport function buildQuestionOptions(\n  url: string,\n  question: string,\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n): QuestionOptions {\n  const out: QuestionOptions = { url, question, ...buildCommonOptions(flags, resolved) };\n  if (flags.format) out.format = flags.format;\n  return out;\n}\n\nexport function buildFieldsOptions(\n  url: string,\n  fields: Record<string, string>,\n  flags: CommonRawFlags,\n  resolved: ResolvedCommonOptions,\n): FieldsOptions {\n  return { url, fields, ...buildCommonOptions(flags, resolved) };\n}\n","/**\n * Shared commander options and the parsers/expanders behind them.\n *\n * All scrape endpoints share the same nine \"common\" flags. We define them in\n * one place and apply via `addCommonScrapeOptions(cmd)` so the surface stays\n * consistent and any change lands everywhere at once.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { Option } from 'commander';\nimport type { Command } from 'commander';\n\nexport interface CommonRawFlags {\n  apiKey?: string;\n  output?: string;\n  format?: 'text' | 'json';\n  pretty?: boolean;\n  js?: boolean;\n  proxy?: 'datacenter' | 'residential' | 'stealth';\n  country?: string;\n  device?: 'desktop' | 'mobile' | 'tablet';\n  timeout?: number;\n  jsTimeout?: number;\n  waitFor?: string;\n  headers?: string;\n  jsScript?: string;\n  customProxy?: string;\n  errorOn404?: boolean;\n  errorOnRedirect?: boolean;\n}\n\n/**\n * Add the flags every scrape-endpoint command needs. Adding here keeps the\n * surface aligned across `html`, `text`, `selected`, etc.\n */\nexport function addCommonScrapeOptions(cmd: Command): Command {\n  return cmd\n    .option('-k, --api-key <key>', 'API key (overrides $WEBSCRAPING_AI_API_KEY and config file)')\n    .option('-o, --output <file>', 'write result to FILE instead of stdout')\n    .option('--pretty', 'pretty-print JSON output (default on TTY)')\n    .addOption(\n      new Option('--no-pretty', 'force single-line JSON output').conflicts('pretty'),\n    )\n    .option('--js', 'execute JS via a headless browser (default: enabled)')\n    .option('--no-js', 'disable JS execution (faster, cheaper)')\n    .addOption(\n      new Option('--proxy <type>', 'proxy pool').choices([\n        'datacenter',\n        'residential',\n        'stealth',\n      ]),\n    )\n    .option('--country <code>', 'two-letter proxy country code, e.g. us, gb, de')\n    .addOption(\n      new Option('--device <type>', 'device emulation').choices(['desktop', 'mobile', 'tablet']),\n    )\n    .option('--timeout <ms>', 'page-retrieval timeout in ms (max 30000)', integerParser)\n    .option('--js-timeout <ms>', 'JS rendering time in ms after load (max 20000)', integerParser)\n    .option('--wait-for <selector>', 'CSS selector to wait for before returning content')\n    .option(\n      '--headers <json>',\n      'HTTP headers to forward to the target page; JSON object or @file.json',\n    )\n    .option('--js-script <code>', 'custom JS to evaluate on the target page')\n    .option('--custom-proxy <url>', 'custom proxy URL: http://user:pass@host:port')\n    .option('--error-on-404', 'return error on 404')\n    .option('--error-on-redirect', 'return error on redirect');\n}\n\nexport function integerParser(value: string): number {\n  const n = Number.parseInt(value, 10);\n  if (!Number.isFinite(n)) throw new Error(`expected integer, got: ${value}`);\n  return n;\n}\n\n/**\n * Parse a `--headers` argument. Accepts a JSON object literal or `@path` to\n * load JSON from a file. Returns `undefined` if the input is empty.\n */\nexport async function parseHeaders(input: string | undefined): Promise<Record<string, string> | undefined> {\n  if (!input || input.trim() === '') return undefined;\n  const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n  return parseJsonObject(text, 'headers');\n}\n\n/**\n * Parse a `--fields` argument for `extract`. Accepts JSON object literal or\n * `@path` to a file.\n */\nexport async function parseFields(input: string): Promise<Record<string, string>> {\n  const text = input.startsWith('@') ? await fs.readFile(input.slice(1), 'utf8') : input;\n  const parsed = parseJsonObject(text, 'fields');\n  if (!parsed || Object.keys(parsed).length === 0) {\n    throw new Error('fields must be a non-empty JSON object of { fieldName: description }');\n  }\n  return parsed;\n}\n\nfunction parseJsonObject(text: string, label: string): Record<string, string> {\n  let parsed: unknown;\n  try {\n    parsed = JSON.parse(text);\n  } catch (err) {\n    throw new Error(`${label}: invalid JSON — ${(err as Error).message}`);\n  }\n  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {\n    throw new Error(`${label} must be a JSON object`);\n  }\n  const out: Record<string, string> = {};\n  for (const [k, v] of Object.entries(parsed as Record<string, unknown>)) {\n    if (typeof v !== 'string') throw new Error(`${label}.${k}: value must be a string`);\n    out[k] = v;\n  }\n  return out;\n}\n","/**\n * Stdin batching helpers.\n *\n * When a URL argument is `-`, the command reads one URL per line from stdin\n * and runs the operation for each. Empty lines and lines starting with `#`\n * are skipped so users can comment URL lists.\n */\n\nimport readline from 'node:readline';\n\nexport function urlIsStdin(url: string): boolean {\n  return url === '-';\n}\n\nexport async function* readUrlsFromStdin(): AsyncIterable<string> {\n  const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });\n  for await (const raw of rl) {\n    const line = raw.trim();\n    if (line === '' || line.startsWith('#')) continue;\n    yield line;\n  }\n}\n\n/**\n * Iterate URLs from either a single CLI arg or from stdin (if the arg is `-`).\n */\nexport async function* urlIterator(arg: string): AsyncIterable<string> {\n  if (urlIsStdin(arg)) {\n    yield* readUrlsFromStdin();\n    return;\n  }\n  yield arg;\n}\n","import { Command } from 'commander';\n\nimport {\n  clearPersistedConfig,\n  configPath,\n  maskApiKey,\n  readPersistedConfig,\n  writePersistedConfig,\n} from '../lib/config.js';\n\nexport function authCommand(): Command {\n  const cmd = new Command('auth').description('Manage your stored API key');\n\n  cmd\n    .command('set <key>')\n    .description(`Save an API key to ${configPath()}`)\n    .action(async (key: string) => {\n      if (!key || key.trim() === '') throw new Error('API key is empty');\n      await writePersistedConfig({ apiKey: key.trim() });\n      process.stdout.write(`API key saved to ${configPath()}\\n`);\n    });\n\n  cmd\n    .command('show')\n    .description('Display the currently stored API key (masked)')\n    .action(async () => {\n      const persisted = await readPersistedConfig();\n      const env = process.env.WEBSCRAPING_AI_API_KEY;\n      if (env && env.trim() !== '') {\n        process.stdout.write(`Active key (env WEBSCRAPING_AI_API_KEY): ${maskApiKey(env.trim())}\\n`);\n      }\n      if (persisted.apiKey && persisted.apiKey.trim() !== '') {\n        process.stdout.write(\n          `Stored key (${configPath()}): ${maskApiKey(persisted.apiKey.trim())}\\n`,\n        );\n      }\n      if (!env && !persisted.apiKey) {\n        process.stdout.write('No API key configured. Run `webscraping-ai auth set <key>`.\\n');\n      }\n    });\n\n  cmd\n    .command('clear')\n    .description('Remove the stored API key file')\n    .action(async () => {\n      const removed = await clearPersistedConfig();\n      process.stdout.write(removed ? `Removed ${configPath()}\\n` : 'No stored API key to remove.\\n');\n    });\n\n  return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildFieldsOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseFields, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface ExtractFlags extends CommonRawFlags {\n  fields?: string;\n}\n\nexport function extractCommand(): Command {\n  const cmd = new Command('extract')\n    .description('Extract structured fields from a page using AI (wraps /ai/fields)')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .requiredOption(\n      '-f, --fields <json>',\n      'fields to extract — JSON object {name: description} or @file.json',\n    );\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: ExtractFlags) => {\n    if (!opts.fields) throw new Error('--fields is required');\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const fields = await parseFields(opts.fields);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildFieldsOptions(target, fields, opts, { headers });\n      const result = await client.fields(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildHtmlOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface HtmlFlags extends CommonRawFlags {\n  returnScriptResult?: boolean;\n}\n\nexport function htmlCommand(): Command {\n  const cmd = new Command('html')\n    .description('Fetch the full HTML of a page')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .addOption(new Option('--return-script-result', 'return the result of --js-script instead of HTML'))\n    .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: HtmlFlags) => {\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildHtmlOptions(target, opts, { headers }, {\n        returnScriptResult: opts.returnScriptResult,\n      });\n      const result = await client.html(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n","import { Command, Option } from 'commander';\n\nimport { buildSelectedOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedFlags extends CommonRawFlags {\n  selector?: string;\n}\n\nexport function selectedCommand(): Command {\n  const cmd = new Command('selected')\n    .description('Fetch HTML of one CSS-selected page area')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .requiredOption('-s, --selector <css>', 'CSS selector for the target element(s)')\n    .addOption(new Option('--format <fmt>', 'response wrapping').choices(['text', 'json']));\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: SelectedFlags) => {\n    if (!opts.selector) throw new Error('--selector is required');\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildSelectedOptions(target, opts.selector, opts, { headers });\n      const result = await client.selected(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n","import { Command } from 'commander';\n\nimport { buildSelectedMultipleOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface SelectedMultipleFlags extends CommonRawFlags {\n  selector?: string[];\n}\n\nexport function selectedMultipleCommand(): Command {\n  const cmd = new Command('selected-multiple')\n    .description('Fetch HTML of multiple CSS-selected page areas')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .option(\n      '-s, --selector <css>',\n      'CSS selector — pass multiple times for multiple selectors',\n      collectMulti,\n      [] as string[],\n    );\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: SelectedMultipleFlags) => {\n    const selectors = opts.selector ?? [];\n    if (selectors.length === 0) {\n      throw new Error('at least one --selector is required');\n    }\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildSelectedMultipleOptions(target, selectors, opts, { headers });\n      const result = await client.selectedMultiple(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n\nfunction collectMulti(value: string, previous: string[]): string[] {\n  return [...previous, value];\n}\n","import { Command } from 'commander';\n\nimport { EDITORS, editorById, installSkill, resolveTarget } from '../lib/skill.js';\nimport type { EditorAdapter } from '../lib/skill.js';\n\ninterface SetupSkillFlags {\n  editor?: string[];\n  all?: boolean;\n  force?: boolean;\n}\n\nexport function setupCommand(): Command {\n  const cmd = new Command('setup').description('Install the AI agent skill into your editor(s)');\n\n  cmd\n    .command('skill')\n    .description('Install the WebScraping.AI skill into one or more AI coding editors')\n    .option(\n      '-e, --editor <id>',\n      `editor id (repeatable). Supported: ${EDITORS.map((e) => e.id).join(', ')}`,\n      collect,\n      [] as string[],\n    )\n    .option('-a, --all', 'install into every supported editor')\n    .option('-f, --force', 'overwrite existing skill files')\n    .action(async (opts: SetupSkillFlags) => {\n      const selected = resolveSelection(opts);\n      if (selected.length === 0) {\n        throw new Error(\n          'no editor selected — pass --all or --editor <id> (e.g. --editor claude-code)',\n        );\n      }\n\n      for (const editor of selected) {\n        const target = resolveTarget(editor);\n        const result = await installSkill(target, { force: opts.force });\n        process.stdout.write(`${editor.label}: ${result.status} → ${target.destination}\\n`);\n      }\n    });\n\n  cmd\n    .command('list', { hidden: false })\n    .description('List supported editors for `setup skill`')\n    .action(() => {\n      for (const e of EDITORS) {\n        const target = resolveTarget(e);\n        process.stdout.write(`${e.id.padEnd(14)} ${e.label.padEnd(18)} ${target.destination}\\n`);\n      }\n    });\n\n  return cmd;\n}\n\nfunction collect(value: string, previous: string[]): string[] {\n  return [...previous, value];\n}\n\nfunction resolveSelection(opts: SetupSkillFlags): EditorAdapter[] {\n  if (opts.all) return [...EDITORS];\n  const ids = opts.editor ?? [];\n  return ids.map((id) => {\n    const editor = editorById(id);\n    if (!editor) throw new Error(`unknown editor: ${id}`);\n    return editor;\n  });\n}\n","/**\n * Multi-editor skill installer.\n *\n * The canonical skill lives in this package at\n * `plugins/webscraping-ai/skills/webscraping-ai/SKILL.md`. Each supported\n * editor reads skill content from a different location and may want either\n * a single Markdown file or a whole skill directory. The `editors` registry\n * captures those differences so `setup skill` is a small driver over it.\n *\n * `mode: 'dir'` copies the entire skill directory (Claude Code-shaped\n * editors that scan a folder per skill). `mode: 'file'` writes the SKILL.md\n * contents to a single file (editors that read flat instructions, like\n * GitHub Copilot's `.github/copilot-instructions.md`).\n *\n * `kind: 'user'` writes to `$HOME`; `kind: 'project'` writes to the current\n * working directory. We prefer user-scope where the editor supports it so\n * the skill becomes globally available without polluting individual repos.\n */\n\nimport { promises as fs } from 'node:fs';\nimport { homedir } from 'node:os';\nimport { dirname, join, resolve } from 'node:path';\nimport { fileURLToPath } from 'node:url';\n\nexport type EditorMode = 'dir' | 'file';\nexport type EditorKind = 'user' | 'project';\n\nexport interface EditorAdapter {\n  /** Stable identifier used by `--editor <name>`. */\n  id: string;\n  /** Human-readable label printed in summaries. */\n  label: string;\n  mode: EditorMode;\n  kind: EditorKind;\n  /**\n   * Where the skill ends up. For `mode: 'dir'`, the skill directory copies\n   * to `{root}/{path}`. For `mode: 'file'`, the SKILL.md contents write to\n   * `{root}/{path}` (i.e. `path` ends in a filename).\n   *\n   * `root` is `$HOME` (user) or CWD (project).\n   */\n  path: string;\n}\n\n/**\n * Registry of supported editors. Order matters: it's also the order the\n * `setup skill` summary lists them in.\n */\nexport const EDITORS: readonly EditorAdapter[] = [\n  // Claude Code: user-scope skills live under ~/.claude/skills/<name>/\n  { id: 'claude-code', label: 'Claude Code', mode: 'dir', kind: 'user', path: '.claude/skills/webscraping-ai' },\n  // Cursor: project-scope rule file. Cursor doesn't have a user-wide skill\n  // dir, so this is the editor's \"install\" surface.\n  { id: 'cursor', label: 'Cursor', mode: 'file', kind: 'project', path: '.cursor/rules/webscraping-ai.mdc' },\n  // Windsurf: project-scope rules. Same shape as Cursor.\n  { id: 'windsurf', label: 'Windsurf', mode: 'file', kind: 'project', path: '.windsurf/rules/webscraping-ai.md' },\n  // Kiro: user-scope steering directory.\n  { id: 'kiro', label: 'Kiro', mode: 'dir', kind: 'user', path: '.kiro/steering/webscraping-ai' },\n  // OpenCode: project-scope skills (mirrors Claude shape).\n  { id: 'opencode', label: 'OpenCode', mode: 'dir', kind: 'project', path: '.opencode/skills/webscraping-ai' },\n  // Gemini CLI: user-scope extension dir with a GEMINI.md.\n  { id: 'gemini', label: 'Gemini CLI', mode: 'file', kind: 'user', path: '.gemini/extensions/webscraping-ai/GEMINI.md' },\n  // GitHub Copilot: project-scope custom instructions file. Single file.\n  { id: 'copilot', label: 'GitHub Copilot', mode: 'file', kind: 'project', path: '.github/copilot-instructions.md' },\n  // Augment: project-scope rules file.\n  { id: 'augment', label: 'Augment', mode: 'file', kind: 'project', path: '.augment/rules/webscraping-ai.md' },\n  // Factory: project-scope rules file.\n  { id: 'factory', label: 'Factory', mode: 'file', kind: 'project', path: '.factory/rules/webscraping-ai.md' },\n] as const;\n\nexport function editorById(id: string): EditorAdapter | undefined {\n  return EDITORS.find((e) => e.id === id);\n}\n\n/**\n * Where this package keeps the canonical skill. Resolved relative to the\n * compiled `dist/cli.js` so it works after `npm install -g`.\n */\nexport function bundledSkillDir(): string {\n  const here = dirname(fileURLToPath(import.meta.url));\n  // dist/ sits at the package root, so the plugins dir is two levels up\n  // from any compiled file: dist/cli.js → package root → plugins/...\n  return resolve(here, '..', 'plugins', 'webscraping-ai', 'skills', 'webscraping-ai');\n}\n\nexport interface InstallTarget {\n  editor: EditorAdapter;\n  /** Absolute destination path (file or directory depending on `mode`). */\n  destination: string;\n}\n\nexport interface ResolveOptions {\n  cwd?: string;\n  home?: string;\n}\n\nexport function resolveTarget(editor: EditorAdapter, opts: ResolveOptions = {}): InstallTarget {\n  const root = editor.kind === 'user' ? (opts.home ?? homedir()) : (opts.cwd ?? process.cwd());\n  return { editor, destination: join(root, editor.path) };\n}\n\nexport interface InstallOptions {\n  /** If true, overwrite existing files/dirs. */\n  force?: boolean;\n  /** Override skill source dir (for tests). */\n  sourceDir?: string;\n}\n\nexport type InstallStatus = 'installed' | 'skipped-exists' | 'overwritten';\n\nexport interface InstallResult {\n  target: InstallTarget;\n  status: InstallStatus;\n}\n\nexport async function installSkill(\n  target: InstallTarget,\n  opts: InstallOptions = {},\n): Promise<InstallResult> {\n  const source = opts.sourceDir ?? bundledSkillDir();\n  const exists = await pathExists(target.destination);\n\n  if (exists && !opts.force) {\n    return { target, status: 'skipped-exists' };\n  }\n\n  if (target.editor.mode === 'dir') {\n    if (exists) await fs.rm(target.destination, { recursive: true, force: true });\n    await copyDirectory(source, target.destination);\n  } else {\n    const skillBody = await fs.readFile(join(source, 'SKILL.md'), 'utf8');\n    await fs.mkdir(dirname(target.destination), { recursive: true });\n    await fs.writeFile(target.destination, skillBody);\n  }\n\n  return { target, status: exists ? 'overwritten' : 'installed' };\n}\n\nasync function pathExists(path: string): Promise<boolean> {\n  try {\n    await fs.access(path);\n    return true;\n  } catch {\n    return false;\n  }\n}\n\nasync function copyDirectory(source: string, destination: string): Promise<void> {\n  // `fs.cp` with `recursive` exists on Node 16.7+; we target Node 20.\n  await fs.cp(source, destination, { recursive: true, force: true });\n}\n","import { Command, Option } from 'commander';\n\nimport { buildTextOptions } from '../lib/buildRequest.js';\nimport { resolveApiKey } from '../lib/config.js';\nimport { addCommonScrapeOptions, parseHeaders } from '../lib/options.js';\nimport type { CommonRawFlags } from '../lib/options.js';\nimport { emit } from '../lib/output.js';\nimport { createClient } from '../lib/sdk.js';\nimport { urlIterator } from '../lib/stdin.js';\n\ninterface TextFlags extends CommonRawFlags {\n  textFormat?: 'plain' | 'json' | 'xml';\n  returnLinks?: boolean;\n}\n\nexport function textCommand(): Command {\n  const cmd = new Command('text')\n    .description('Fetch the visible text of a page')\n    .argument('<url>', 'page URL (or `-` to read URLs from stdin)')\n    .addOption(\n      new Option('--text-format <fmt>', 'text response format').choices(['plain', 'json', 'xml']),\n    )\n    .option('--return-links', 'include page links (text-format=json only)');\n\n  addCommonScrapeOptions(cmd);\n\n  cmd.action(async (url: string, opts: TextFlags) => {\n    const apiKey = await resolveApiKey({ flag: opts.apiKey });\n    const headers = await parseHeaders(opts.headers);\n    const client = createClient({ apiKey, requestTimeoutMs: opts.timeout });\n\n    for await (const target of urlIterator(url)) {\n      const sdkOptions = buildTextOptions(target, opts, { headers }, {\n        textFormat: opts.textFormat,\n        returnLinks: opts.returnLinks,\n      });\n      const result = await client.text(sdkOptions);\n      await emit(result, opts);\n    }\n  });\n\n  return cmd;\n}\n","/**\n * Exit-code mapping for SDK errors.\n *\n * The SDK throws typed subclasses of `WebScrapingAIError`. Map each to a\n * stable exit code so shell scripts can branch on the failure mode without\n * parsing stderr.\n */\n\nimport {\n  APIConnectionError,\n  APIError,\n  APITimeoutError,\n  AuthenticationError,\n  BadRequestError,\n  GatewayTimeoutError,\n  PaymentRequiredError,\n  RateLimitError,\n  ServerError,\n  WebScrapingAIError,\n} from 'webscraping-ai';\n\nexport const EXIT_CODES = {\n  ok: 0,\n  generic: 1,\n  usage: 2,\n  badRequest: 3,\n  auth: 4,\n  paymentRequired: 5,\n  rateLimit: 6,\n  serverError: 7,\n  timeout: 8,\n  connection: 9,\n} as const;\n\nexport function exitCodeFor(err: unknown): number {\n  if (err instanceof BadRequestError) return EXIT_CODES.badRequest;\n  if (err instanceof AuthenticationError) return EXIT_CODES.auth;\n  if (err instanceof PaymentRequiredError) return EXIT_CODES.paymentRequired;\n  if (err instanceof RateLimitError) return EXIT_CODES.rateLimit;\n  if (err instanceof ServerError) return EXIT_CODES.serverError;\n  if (err instanceof GatewayTimeoutError) return EXIT_CODES.timeout;\n  if (err instanceof APITimeoutError) return EXIT_CODES.timeout;\n  if (err instanceof APIConnectionError) return EXIT_CODES.connection;\n  if (err instanceof APIError) return EXIT_CODES.serverError;\n  if (err instanceof WebScrapingAIError) return EXIT_CODES.generic;\n  return EXIT_CODES.generic;\n}\n\nexport function formatError(err: unknown): string {\n  if (err instanceof APIError) {\n    const parts = [err.message];\n    if (err.status) parts.push(`(HTTP ${err.status})`);\n    return parts.join(' ');\n  }\n  if (err instanceof Error) return err.message;\n  return String(err);\n}\n"],"mappings":";;;AAQA,SAAS,WAAAA,iBAAe;AACxB,SAAS,oBAAoB;AAC7B,SAAS,WAAAC,UAAS,WAAAC,gBAAe;AACjC,SAAS,iBAAAC,sBAAqB;;;ACX9B,SAAS,eAAe;;;ACYxB,SAAS,YAAY,UAAU;AAC/B,SAAS,eAAe;AACxB,SAAS,SAAS,YAAY;AAE9B,IAAM,kBAAkB;AAEjB,SAAS,YAAoB;AAClC,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,gBAAgB;AAC/D,SAAO,KAAK,QAAQ,GAAG,WAAW,gBAAgB;AACpD;AAEO,SAAS,aAAqB;AACnC,SAAO,KAAK,UAAU,GAAG,eAAe;AAC1C;AAMA,eAAsB,sBAAgD;AACpE,MAAI;AACF,UAAM,OAAO,MAAM,GAAG,SAAS,WAAW,GAAG,MAAM;AACnD,UAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,QAAI,UAAU,OAAO,WAAW,YAAY,CAAC,MAAM,QAAQ,MAAM,GAAG;AAClE,aAAO;AAAA,IACT;AACA,WAAO,CAAC;AAAA,EACV,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO,CAAC;AAC9D,UAAM;AAAA,EACR;AACF;AAEA,eAAsB,qBAAqB,QAAwC;AACjF,QAAM,OAAO,WAAW;AACxB,QAAM,GAAG,MAAM,QAAQ,IAAI,GAAG,EAAE,WAAW,KAAK,CAAC;AACjD,QAAM,GAAG,UAAU,MAAM,KAAK,UAAU,QAAQ,MAAM,CAAC,IAAI,MAAM,EAAE,MAAM,IAAM,CAAC;AAClF;AAEA,eAAsB,uBAAyC;AAC7D,MAAI;AACF,UAAM,GAAG,OAAO,WAAW,CAAC;AAC5B,WAAO;AAAA,EACT,SAAS,KAAK;AACZ,QAAK,IAA8B,SAAS,SAAU,QAAO;AAC7D,UAAM;AAAA,EACR;AACF;AAOA,eAAsB,cAAc,OAA6B,CAAC,GAAoB;AACpF,MAAI,KAAK,QAAQ,KAAK,KAAK,KAAK,MAAM,GAAI,QAAO,KAAK,KAAK,KAAK;AAEhE,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,OAAO,IAAI,KAAK,MAAM,GAAI,QAAO,IAAI,KAAK;AAE9C,QAAM,YAAY,MAAM,oBAAoB;AAC5C,MAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,GAAI,QAAO,UAAU,OAAO,KAAK;AAErF,QAAM,IAAI;AAAA,IACR;AAAA,EACF;AACF;AAEO,SAAS,WAAW,KAAqB;AAC9C,MAAI,IAAI,UAAU,EAAG,QAAO,IAAI,OAAO,IAAI,MAAM;AACjD,SAAO,GAAG,IAAI,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,OAAO,IAAI,SAAS,CAAC,CAAC,GAAG,IAAI,MAAM,EAAE,CAAC;AACxE;;;ACzEA,SAAS,YAAYC,WAAU;AAO/B,eAAsB,KAAK,OAAgB,OAAoB,CAAC,GAAkB;AAChF,QAAM,OAAO,UAAU,OAAO,IAAI;AAElC,MAAI,KAAK,UAAU,KAAK,OAAO,KAAK,MAAM,IAAI;AAC5C,UAAMA,IAAG,UAAU,KAAK,QAAQ,QAAQ,KAAK,SAAS,IAAI,IAAI,KAAK,KAAK;AACxE;AAAA,EACF;AAEA,UAAQ,OAAO,MAAM,IAAI;AACzB,MAAI,CAAC,KAAK,SAAS,IAAI,EAAG,SAAQ,OAAO,MAAM,IAAI;AACrD;AAEO,SAAS,UAAU,OAAgB,OAAoB,CAAC,GAAW;AACxE,MAAI,OAAO,UAAU,SAAU,QAAO;AAEtC,QAAM,SAAS,KAAK,UAAU,QAAQ,OAAO,SAAS;AACtD,SAAO,KAAK,UAAU,OAAO,MAAM,SAAS,IAAI,CAAC;AACnD;;;ACzBA,SAAS,qBAAqB;AAoBvB,SAAS,aAAa,SAA6C;AACxE,QAAM,QAAQ,IAAI,cAAc;AAAA,IAC9B,QAAQ,QAAQ;AAAA,IAChB,WAAW,QAAQ;AAAA,EACrB,CAAC;AAKD,QAAM,MAAM,EAAE,UAAU,KAAK;AAE7B,SAAO,IAAI,MAAM,OAAO;AAAA,IACtB,IAAI,QAAQ,MAAM,UAAU;AAC1B,YAAM,QAAQ,QAAQ,IAAI,QAAQ,MAAM,QAAQ;AAChD,UAAI,OAAO,UAAU,WAAY,QAAO;AACxC,cAAQ,MAAM;AAAA,QACZ,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAmB,OAAO,KAAK,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACzD,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAA+B,OAAO,iBAAiB,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjF,KAAK;AACH,iBAAO,CAAC,MAAuB,OAAO,SAAS,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QACjE,KAAK;AACH,iBAAO,CAAC,MAAqB,OAAO,OAAO,EAAE,GAAG,GAAG,GAAG,IAAI,CAAC;AAAA,QAC7D,KAAK;AAGH,iBAAO,MAAM,KAAK,MAAM;AAAA,QAC1B;AACE,iBAAO,OAAO,UAAU,aAAa,MAAM,KAAK,MAAM,IAAI;AAAA,MAC9D;AAAA,IACF;AAAA,EACF,CAAC;AACH;;;AHvDO,SAAS,iBAA0B;AACxC,SAAO,IAAI,QAAQ,SAAS,EACzB,YAAY,0CAA0C,EACtD,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D,OAAO,eAAe,+BAA+B,EACrD,OAAO,OAAO,SAAuB;AACpC,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,SAAS,aAAa,EAAE,OAAO,CAAC;AACtC,UAAM,SAAS,MAAM,OAAO,QAAQ;AACpC,UAAM,KAAK,QAAQ,IAAI;AAAA,EACzB,CAAC;AACL;;;AIzBA,SAAS,WAAAC,UAAS,UAAAC,eAAc;;;ACyBzB,SAAS,mBACd,OACA,WAAkC,CAAC,GACb;AACtB,QAAM,MAA4B,CAAC;AAInC,MAAI,MAAM,OAAO,OAAW,KAAI,KAAK,MAAM;AAC3C,MAAI,MAAM,UAAU,OAAW,KAAI,QAAQ,MAAM;AACjD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,WAAW,OAAW,KAAI,SAAS,MAAM;AACnD,MAAI,MAAM,YAAY,OAAW,KAAI,UAAU,MAAM;AACrD,MAAI,MAAM,cAAc,OAAW,KAAI,aAAa,MAAM;AAC1D,MAAI,MAAM,YAAY,OAAW,KAAI,WAAW,MAAM;AACtD,MAAI,MAAM,aAAa,OAAW,KAAI,YAAY,MAAM;AACxD,MAAI,MAAM,gBAAgB,OAAW,KAAI,eAAe,MAAM;AAC9D,MAAI,MAAM,WAAY,KAAI,eAAe;AACzC,MAAI,MAAM,gBAAiB,KAAI,oBAAoB;AACnD,MAAI,SAAS,QAAS,KAAI,UAAU,SAAS;AAC7C,SAAO;AACT;AAMO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,MAAI,OAAO,mBAAoB,KAAI,uBAAuB;AAC1D,SAAO;AACT;AAOO,SAAS,iBACd,KACA,OACA,UACA,SAAqB,CAAC,GACT;AACb,QAAM,MAAmB,EAAE,KAAK,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACvE,MAAI,OAAO,WAAY,KAAI,cAAc,OAAO;AAChD,MAAI,OAAO,YAAa,KAAI,eAAe;AAC3C,SAAO;AACT;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,6BACd,KACA,WACA,OACA,UACyB;AACzB,MAAI,UAAU,WAAW,GAAG;AAC1B,UAAM,IAAI,MAAM,oDAAoD;AAAA,EACtE;AACA,SAAO,EAAE,KAAK,WAAW,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAClE;AAEO,SAAS,qBACd,KACA,UACA,OACA,UACiB;AACjB,QAAM,MAAuB,EAAE,KAAK,UAAU,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AACrF,MAAI,MAAM,OAAQ,KAAI,SAAS,MAAM;AACrC,SAAO;AACT;AAEO,SAAS,mBACd,KACA,QACA,OACA,UACe;AACf,SAAO,EAAE,KAAK,QAAQ,GAAG,mBAAmB,OAAO,QAAQ,EAAE;AAC/D;;;AClHA,SAAS,YAAYC,WAAU;AAC/B,SAAS,cAAc;AA0BhB,SAAS,uBAAuB,KAAuB;AAC5D,SAAO,IACJ,OAAO,uBAAuB,6DAA6D,EAC3F,OAAO,uBAAuB,wCAAwC,EACtE,OAAO,YAAY,2CAA2C,EAC9D;AAAA,IACC,IAAI,OAAO,eAAe,+BAA+B,EAAE,UAAU,QAAQ;AAAA,EAC/E,EACC,OAAO,QAAQ,sDAAsD,EACrE,OAAO,WAAW,wCAAwC,EAC1D;AAAA,IACC,IAAI,OAAO,kBAAkB,YAAY,EAAE,QAAQ;AAAA,MACjD;AAAA,MACA;AAAA,MACA;AAAA,IACF,CAAC;AAAA,EACH,EACC,OAAO,oBAAoB,gDAAgD,EAC3E;AAAA,IACC,IAAI,OAAO,mBAAmB,kBAAkB,EAAE,QAAQ,CAAC,WAAW,UAAU,QAAQ,CAAC;AAAA,EAC3F,EACC,OAAO,kBAAkB,4CAA4C,aAAa,EAClF,OAAO,qBAAqB,kDAAkD,aAAa,EAC3F,OAAO,yBAAyB,mDAAmD,EACnF;AAAA,IACC;AAAA,IACA;AAAA,EACF,EACC,OAAO,sBAAsB,0CAA0C,EACvE,OAAO,wBAAwB,8CAA8C,EAC7E,OAAO,kBAAkB,qBAAqB,EAC9C,OAAO,uBAAuB,0BAA0B;AAC7D;AAEO,SAAS,cAAc,OAAuB;AACnD,QAAM,IAAI,OAAO,SAAS,OAAO,EAAE;AACnC,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,OAAM,IAAI,MAAM,0BAA0B,KAAK,EAAE;AAC1E,SAAO;AACT;AAMA,eAAsB,aAAa,OAAwE;AACzG,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,GAAI,QAAO;AAC1C,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,SAAO,gBAAgB,MAAM,SAAS;AACxC;AAMA,eAAsB,YAAY,OAAgD;AAChF,QAAM,OAAO,MAAM,WAAW,GAAG,IAAI,MAAMA,IAAG,SAAS,MAAM,MAAM,CAAC,GAAG,MAAM,IAAI;AACjF,QAAM,SAAS,gBAAgB,MAAM,QAAQ;AAC7C,MAAI,CAAC,UAAU,OAAO,KAAK,MAAM,EAAE,WAAW,GAAG;AAC/C,UAAM,IAAI,MAAM,sEAAsE;AAAA,EACxF;AACA,SAAO;AACT;AAEA,SAAS,gBAAgB,MAAc,OAAuC;AAC5E,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,SAAS,KAAK;AACZ,UAAM,IAAI,MAAM,GAAG,KAAK,yBAAqB,IAAc,OAAO,EAAE;AAAA,EACtE;AACA,MAAI,CAAC,UAAU,OAAO,WAAW,YAAY,MAAM,QAAQ,MAAM,GAAG;AAClE,UAAM,IAAI,MAAM,GAAG,KAAK,wBAAwB;AAAA,EAClD;AACA,QAAM,MAA8B,CAAC;AACrC,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,MAAiC,GAAG;AACtE,QAAI,OAAO,MAAM,SAAU,OAAM,IAAI,MAAM,GAAG,KAAK,IAAI,CAAC,0BAA0B;AAClF,QAAI,CAAC,IAAI;AAAA,EACX;AACA,SAAO;AACT;;;AC1GA,OAAO,cAAc;AAEd,SAAS,WAAW,KAAsB;AAC/C,SAAO,QAAQ;AACjB;AAEA,gBAAuB,oBAA2C;AAChE,QAAM,KAAK,SAAS,gBAAgB,EAAE,OAAO,QAAQ,OAAO,WAAW,SAAS,CAAC;AACjF,mBAAiB,OAAO,IAAI;AAC1B,UAAM,OAAO,IAAI,KAAK;AACtB,QAAI,SAAS,MAAM,KAAK,WAAW,GAAG,EAAG;AACzC,UAAM;AAAA,EACR;AACF;AAKA,gBAAuB,YAAY,KAAoC;AACrE,MAAI,WAAW,GAAG,GAAG;AACnB,WAAO,kBAAkB;AACzB;AAAA,EACF;AACA,QAAM;AACR;;;AHlBO,SAAS,aAAsB;AACpC,QAAM,MAAM,IAAIC,SAAQ,KAAK,EAC1B,YAAY,sDAAsD,EAClE,SAAS,SAAS,2CAA2C,EAC7D,eAAe,yBAAyB,2CAA2C,EACnF,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAmB;AAChD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AIrCA,SAAS,WAAAC,gBAAe;AAUjB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAAE,YAAY,4BAA4B;AAExE,MACG,QAAQ,WAAW,EACnB,YAAY,sBAAsB,WAAW,CAAC,EAAE,EAChD,OAAO,OAAO,QAAgB;AAC7B,QAAI,CAAC,OAAO,IAAI,KAAK,MAAM,GAAI,OAAM,IAAI,MAAM,kBAAkB;AACjE,UAAM,qBAAqB,EAAE,QAAQ,IAAI,KAAK,EAAE,CAAC;AACjD,YAAQ,OAAO,MAAM,oBAAoB,WAAW,CAAC;AAAA,CAAI;AAAA,EAC3D,CAAC;AAEH,MACG,QAAQ,MAAM,EACd,YAAY,+CAA+C,EAC3D,OAAO,YAAY;AAClB,UAAM,YAAY,MAAM,oBAAoB;AAC5C,UAAM,MAAM,QAAQ,IAAI;AACxB,QAAI,OAAO,IAAI,KAAK,MAAM,IAAI;AAC5B,cAAQ,OAAO,MAAM,4CAA4C,WAAW,IAAI,KAAK,CAAC,CAAC;AAAA,CAAI;AAAA,IAC7F;AACA,QAAI,UAAU,UAAU,UAAU,OAAO,KAAK,MAAM,IAAI;AACtD,cAAQ,OAAO;AAAA,QACb,eAAe,WAAW,CAAC,MAAM,WAAW,UAAU,OAAO,KAAK,CAAC,CAAC;AAAA;AAAA,MACtE;AAAA,IACF;AACA,QAAI,CAAC,OAAO,CAAC,UAAU,QAAQ;AAC7B,cAAQ,OAAO,MAAM,+DAA+D;AAAA,IACtF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,OAAO,EACf,YAAY,gCAAgC,EAC5C,OAAO,YAAY;AAClB,UAAM,UAAU,MAAM,qBAAqB;AAC3C,YAAQ,OAAO,MAAM,UAAU,WAAW,WAAW,CAAC;AAAA,IAAO,gCAAgC;AAAA,EAC/F,CAAC;AAEH,SAAO;AACT;;;AClDA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,iBAA0B;AACxC,QAAM,MAAM,IAAIC,SAAQ,SAAS,EAC9B,YAAY,mEAAmE,EAC/E,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,EACF;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAuB;AACpD,QAAI,CAAC,KAAK,OAAQ,OAAM,IAAI,MAAM,sBAAsB;AACxD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,MAAM,YAAY,KAAK,MAAM;AAC5C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,mBAAmB,QAAQ,QAAQ,MAAM,EAAE,QAAQ,CAAC;AACvE,YAAM,SAAS,MAAM,OAAO,OAAO,UAAU;AAC7C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACxCA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,+BAA+B,EAC3C,SAAS,SAAS,2CAA2C,EAC7D,UAAU,IAAIC,QAAO,0BAA0B,kDAAkD,CAAC,EAClG,UAAU,IAAIA,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,iBAAiB,QAAQ,MAAM,EAAE,QAAQ,GAAG;AAAA,QAC7D,oBAAoB,KAAK;AAAA,MAC3B,CAAC;AACD,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACtCA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAczB,SAAS,kBAA2B;AACzC,QAAM,MAAM,IAAIC,SAAQ,UAAU,EAC/B,YAAY,0CAA0C,EACtD,SAAS,SAAS,2CAA2C,EAC7D,eAAe,wBAAwB,wCAAwC,EAC/E,UAAU,IAAIC,QAAO,kBAAkB,mBAAmB,EAAE,QAAQ,CAAC,QAAQ,MAAM,CAAC,CAAC;AAExF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAwB;AACrD,QAAI,CAAC,KAAK,SAAU,OAAM,IAAI,MAAM,wBAAwB;AAC5D,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,qBAAqB,QAAQ,KAAK,UAAU,MAAM,EAAE,QAAQ,CAAC;AAChF,YAAM,SAAS,MAAM,OAAO,SAAS,UAAU;AAC/C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;ACrCA,SAAS,WAAAC,gBAAe;AAcjB,SAAS,0BAAmC;AACjD,QAAM,MAAM,IAAIC,SAAQ,mBAAmB,EACxC,YAAY,gDAAgD,EAC5D,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC;AAAA,IACA;AAAA,IACA;AAAA,IACA,CAAC;AAAA,EACH;AAEF,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAgC;AAC7D,UAAM,YAAY,KAAK,YAAY,CAAC;AACpC,QAAI,UAAU,WAAW,GAAG;AAC1B,YAAM,IAAI,MAAM,qCAAqC;AAAA,IACvD;AACA,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,6BAA6B,QAAQ,WAAW,MAAM,EAAE,QAAQ,CAAC;AACpF,YAAM,SAAS,MAAM,OAAO,iBAAiB,UAAU;AACvD,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;AAEA,SAAS,aAAa,OAAe,UAA8B;AACjE,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;;;AChDA,SAAS,WAAAC,gBAAe;;;ACmBxB,SAAS,YAAYC,WAAU;AAC/B,SAAS,WAAAC,gBAAe;AACxB,SAAS,WAAAC,UAAS,QAAAC,OAAM,eAAe;AACvC,SAAS,qBAAqB;AA0BvB,IAAM,UAAoC;AAAA;AAAA,EAE/C,EAAE,IAAI,eAAe,OAAO,eAAe,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA;AAAA,EAG5G,EAAE,IAAI,UAAU,OAAO,UAAU,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAAA;AAAA,EAEzG,EAAE,IAAI,YAAY,OAAO,YAAY,MAAM,QAAQ,MAAM,WAAW,MAAM,oCAAoC;AAAA;AAAA,EAE9G,EAAE,IAAI,QAAQ,OAAO,QAAQ,MAAM,OAAO,MAAM,QAAQ,MAAM,gCAAgC;AAAA;AAAA,EAE9F,EAAE,IAAI,YAAY,OAAO,YAAY,MAAM,OAAO,MAAM,WAAW,MAAM,kCAAkC;AAAA;AAAA,EAE3G,EAAE,IAAI,UAAU,OAAO,cAAc,MAAM,QAAQ,MAAM,QAAQ,MAAM,8CAA8C;AAAA;AAAA,EAErH,EAAE,IAAI,WAAW,OAAO,kBAAkB,MAAM,QAAQ,MAAM,WAAW,MAAM,kCAAkC;AAAA;AAAA,EAEjH,EAAE,IAAI,WAAW,OAAO,WAAW,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAAA;AAAA,EAE3G,EAAE,IAAI,WAAW,OAAO,WAAW,MAAM,QAAQ,MAAM,WAAW,MAAM,mCAAmC;AAC7G;AAEO,SAAS,WAAW,IAAuC;AAChE,SAAO,QAAQ,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE;AACxC;AAMO,SAAS,kBAA0B;AACxC,QAAM,OAAOD,SAAQ,cAAc,YAAY,GAAG,CAAC;AAGnD,SAAO,QAAQ,MAAM,MAAM,WAAW,kBAAkB,UAAU,gBAAgB;AACpF;AAaO,SAAS,cAAc,QAAuB,OAAuB,CAAC,GAAkB;AAC7F,QAAM,OAAO,OAAO,SAAS,SAAU,KAAK,QAAQD,SAAQ,IAAM,KAAK,OAAO,QAAQ,IAAI;AAC1F,SAAO,EAAE,QAAQ,aAAaE,MAAK,MAAM,OAAO,IAAI,EAAE;AACxD;AAgBA,eAAsB,aACpB,QACA,OAAuB,CAAC,GACA;AACxB,QAAM,SAAS,KAAK,aAAa,gBAAgB;AACjD,QAAM,SAAS,MAAM,WAAW,OAAO,WAAW;AAElD,MAAI,UAAU,CAAC,KAAK,OAAO;AACzB,WAAO,EAAE,QAAQ,QAAQ,iBAAiB;AAAA,EAC5C;AAEA,MAAI,OAAO,OAAO,SAAS,OAAO;AAChC,QAAI,OAAQ,OAAMH,IAAG,GAAG,OAAO,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AAC5E,UAAM,cAAc,QAAQ,OAAO,WAAW;AAAA,EAChD,OAAO;AACL,UAAM,YAAY,MAAMA,IAAG,SAASG,MAAK,QAAQ,UAAU,GAAG,MAAM;AACpE,UAAMH,IAAG,MAAME,SAAQ,OAAO,WAAW,GAAG,EAAE,WAAW,KAAK,CAAC;AAC/D,UAAMF,IAAG,UAAU,OAAO,aAAa,SAAS;AAAA,EAClD;AAEA,SAAO,EAAE,QAAQ,QAAQ,SAAS,gBAAgB,YAAY;AAChE;AAEA,eAAe,WAAW,MAAgC;AACxD,MAAI;AACF,UAAMA,IAAG,OAAO,IAAI;AACpB,WAAO;AAAA,EACT,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEA,eAAe,cAAc,QAAgB,aAAoC;AAE/E,QAAMA,IAAG,GAAG,QAAQ,aAAa,EAAE,WAAW,MAAM,OAAO,KAAK,CAAC;AACnE;;;AD3IO,SAAS,eAAwB;AACtC,QAAM,MAAM,IAAII,SAAQ,OAAO,EAAE,YAAY,gDAAgD;AAE7F,MACG,QAAQ,OAAO,EACf,YAAY,qEAAqE,EACjF;AAAA,IACC;AAAA,IACA,sCAAsC,QAAQ,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,KAAK,IAAI,CAAC;AAAA,IACzE;AAAA,IACA,CAAC;AAAA,EACH,EACC,OAAO,aAAa,qCAAqC,EACzD,OAAO,eAAe,gCAAgC,EACtD,OAAO,OAAO,SAA0B;AACvC,UAAM,WAAW,iBAAiB,IAAI;AACtC,QAAI,SAAS,WAAW,GAAG;AACzB,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,eAAW,UAAU,UAAU;AAC7B,YAAM,SAAS,cAAc,MAAM;AACnC,YAAM,SAAS,MAAM,aAAa,QAAQ,EAAE,OAAO,KAAK,MAAM,CAAC;AAC/D,cAAQ,OAAO,MAAM,GAAG,OAAO,KAAK,KAAK,OAAO,MAAM,WAAM,OAAO,WAAW;AAAA,CAAI;AAAA,IACpF;AAAA,EACF,CAAC;AAEH,MACG,QAAQ,QAAQ,EAAE,QAAQ,MAAM,CAAC,EACjC,YAAY,0CAA0C,EACtD,OAAO,MAAM;AACZ,eAAW,KAAK,SAAS;AACvB,YAAM,SAAS,cAAc,CAAC;AAC9B,cAAQ,OAAO,MAAM,GAAG,EAAE,GAAG,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,OAAO,EAAE,CAAC,IAAI,OAAO,WAAW;AAAA,CAAI;AAAA,IACzF;AAAA,EACF,CAAC;AAEH,SAAO;AACT;AAEA,SAAS,QAAQ,OAAe,UAA8B;AAC5D,SAAO,CAAC,GAAG,UAAU,KAAK;AAC5B;AAEA,SAAS,iBAAiB,MAAwC;AAChE,MAAI,KAAK,IAAK,QAAO,CAAC,GAAG,OAAO;AAChC,QAAM,MAAM,KAAK,UAAU,CAAC;AAC5B,SAAO,IAAI,IAAI,CAAC,OAAO;AACrB,UAAM,SAAS,WAAW,EAAE;AAC5B,QAAI,CAAC,OAAQ,OAAM,IAAI,MAAM,mBAAmB,EAAE,EAAE;AACpD,WAAO;AAAA,EACT,CAAC;AACH;;;AEjEA,SAAS,WAAAC,UAAS,UAAAC,eAAc;AAezB,SAAS,cAAuB;AACrC,QAAM,MAAM,IAAIC,SAAQ,MAAM,EAC3B,YAAY,kCAAkC,EAC9C,SAAS,SAAS,2CAA2C,EAC7D;AAAA,IACC,IAAIC,QAAO,uBAAuB,sBAAsB,EAAE,QAAQ,CAAC,SAAS,QAAQ,KAAK,CAAC;AAAA,EAC5F,EACC,OAAO,kBAAkB,4CAA4C;AAExE,yBAAuB,GAAG;AAE1B,MAAI,OAAO,OAAO,KAAa,SAAoB;AACjD,UAAM,SAAS,MAAM,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC;AACxD,UAAM,UAAU,MAAM,aAAa,KAAK,OAAO;AAC/C,UAAM,SAAS,aAAa,EAAE,QAAQ,kBAAkB,KAAK,QAAQ,CAAC;AAEtE,qBAAiB,UAAU,YAAY,GAAG,GAAG;AAC3C,YAAM,aAAa,iBAAiB,QAAQ,MAAM,EAAE,QAAQ,GAAG;AAAA,QAC7D,YAAY,KAAK;AAAA,QACjB,aAAa,KAAK;AAAA,MACpB,CAAC;AACD,YAAM,SAAS,MAAM,OAAO,KAAK,UAAU;AAC3C,YAAM,KAAK,QAAQ,IAAI;AAAA,IACzB;AAAA,EACF,CAAC;AAED,SAAO;AACT;;;AClCA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEA,IAAM,aAAa;AAAA,EACxB,IAAI;AAAA,EACJ,SAAS;AAAA,EACT,OAAO;AAAA,EACP,YAAY;AAAA,EACZ,MAAM;AAAA,EACN,iBAAiB;AAAA,EACjB,WAAW;AAAA,EACX,aAAa;AAAA,EACb,SAAS;AAAA,EACT,YAAY;AACd;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,qBAAsB,QAAO,WAAW;AAC3D,MAAI,eAAe,eAAgB,QAAO,WAAW;AACrD,MAAI,eAAe,YAAa,QAAO,WAAW;AAClD,MAAI,eAAe,oBAAqB,QAAO,WAAW;AAC1D,MAAI,eAAe,gBAAiB,QAAO,WAAW;AACtD,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,MAAI,eAAe,SAAU,QAAO,WAAW;AAC/C,MAAI,eAAe,mBAAoB,QAAO,WAAW;AACzD,SAAO,WAAW;AACpB;AAEO,SAAS,YAAY,KAAsB;AAChD,MAAI,eAAe,UAAU;AAC3B,UAAM,QAAQ,CAAC,IAAI,OAAO;AAC1B,QAAI,IAAI,OAAQ,OAAM,KAAK,SAAS,IAAI,MAAM,GAAG;AACjD,WAAO,MAAM,KAAK,GAAG;AAAA,EACvB;AACA,MAAI,eAAe,MAAO,QAAO,IAAI;AACrC,SAAO,OAAO,GAAG;AACnB;;;AjBhCA,SAAS,iBAAyB;AAGhC,QAAM,OAAOC,SAAQC,eAAc,YAAY,GAAG,CAAC;AACnD,QAAM,UAAUC,SAAQ,MAAM,MAAM,cAAc;AAClD,QAAM,MAAM,KAAK,MAAM,aAAa,SAAS,MAAM,CAAC;AACpD,SAAO,IAAI;AACb;AAEO,SAAS,eAAwB;AACtC,QAAM,UAAU,IAAIC,UAAQ;AAC5B,UACG,KAAK,gBAAgB,EACrB;AAAA,IACC;AAAA,EAEF,EACC,QAAQ,eAAe,GAAG,iBAAiB,2BAA2B,EACtE,mBAAmB;AAEtB,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,gBAAgB,CAAC;AACpC,UAAQ,WAAW,wBAAwB,CAAC;AAC5C,UAAQ,WAAW,WAAW,CAAC;AAC/B,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,eAAe,CAAC;AACnC,UAAQ,WAAW,YAAY,CAAC;AAChC,UAAQ,WAAW,aAAa,CAAC;AAEjC,SAAO;AACT;AAEA,eAAe,KAAK,MAA+B;AACjD,QAAM,UAAU,aAAa;AAC7B,MAAI;AACF,UAAM,QAAQ,WAAW,IAAI;AAAA,EAC/B,SAAS,KAAK;AACZ,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B;AACF;AAIA,IAAM,YAAY,QAAQ,KAAK,CAAC,IAAIF,eAAc,YAAY,GAAG,IAAI;AACrE,IAAI,aAAa,QAAQ,KAAK,CAAC,MAAM,WAAW;AAC9C,OAAK,QAAQ,IAAI,EAAE,MAAM,CAAC,QAAiB;AACzC,YAAQ,OAAO,MAAM,UAAU,YAAY,GAAG,CAAC;AAAA,CAAI;AACnD,YAAQ,KAAK,YAAY,GAAG,CAAC;AAAA,EAC/B,CAAC;AACH;","names":["Command","dirname","resolve","fileURLToPath","fs","Command","Option","fs","Command","Option","Command","Command","Command","Command","Command","Option","Command","Option","Command","Option","Command","Option","Command","Command","Command","fs","homedir","dirname","join","Command","Command","Option","Command","Option","dirname","fileURLToPath","resolve","Command"]}

package/package.json

@@ -0,0 +1,71 @@
+{
+  "name": "webscraping-ai-cli",
+  "version": "1.0.0",
+  "description": "Official command-line client for the WebScraping.AI API. Includes a Claude Code skill so AI agents can use it too.",
+  "type": "module",
+  "bin": {
+    "webscraping-ai": "./dist/cli.js",
+    "wsai": "./dist/cli.js"
+  },
+  "files": [
+    "dist",
+    "plugins",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "build": "tsup",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "lint": "eslint .",
+    "lint:fix": "eslint . --fix",
+    "format": "prettier --write .",
+    "format:check": "prettier --check .",
+    "typecheck": "tsc --noEmit",
+    "smoke": "tsx bin/smoke.ts",
+    "prepublishOnly": "npm run build"
+  },
+  "engines": {
+    "node": ">=20"
+  },
+  "keywords": [
+    "webscraping",
+    "scraping",
+    "cli",
+    "api",
+    "llm",
+    "agent",
+    "claude",
+    "claude-code",
+    "skill"
+  ],
+  "license": "MIT",
+  "author": "WebScraping.AI",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/webscraping-ai/webscraping-ai-cli.git"
+  },
+  "bugs": {
+    "url": "https://github.com/webscraping-ai/webscraping-ai-cli/issues"
+  },
+  "homepage": "https://github.com/webscraping-ai/webscraping-ai-cli#readme",
+  "dependencies": {
+    "commander": "^12.1.0",
+    "webscraping-ai": "^4.0.0"
+  },
+  "devDependencies": {
+    "@eslint/js": "^9.17.0",
+    "@types/node": "^22.10.0",
+    "@typescript-eslint/eslint-plugin": "^8.18.0",
+    "@typescript-eslint/parser": "^8.18.0",
+    "eslint": "^9.17.0",
+    "eslint-config-prettier": "^9.1.0",
+    "globals": "^17.6.0",
+    "prettier": "^3.4.2",
+    "tsup": "^8.3.5",
+    "tsx": "^4.19.2",
+    "typescript": "^5.7.2",
+    "vitest": "^2.1.8"
+  }
+}

package/plugins/webscraping-ai/.claude-plugin/marketplace.json

@@ -0,0 +1,17 @@
+{
+  "$schema": "https://json.schemastore.org/claude-code-plugin-marketplace.json",
+  "name": "webscraping-ai",
+  "owner": {
+    "name": "WebScraping.AI",
+    "url": "https://webscraping.ai"
+  },
+  "plugins": [
+    {
+      "name": "webscraping-ai",
+      "description": "WebScraping.AI CLI skill: fetch HTML, text, selected elements, or AI-extracted data from any URL — including JS-rendered pages.",
+      "version": "1.0.0",
+      "source": ".",
+      "skills": ["./skills/webscraping-ai"]
+    }
+  ]
+}

package/plugins/webscraping-ai/skills/webscraping-ai/SKILL.md

@@ -0,0 +1,109 @@
+---
+name: webscraping-ai
+description: Use the WebScraping.AI CLI to fetch HTML, plain text, or AI-extracted data from any URL. Reach for this when you need real page content (including JS-rendered sites) instead of guessing or relying on stale knowledge.
+---
+
+# WebScraping.AI skill
+
+This skill teaches you to use the `webscraping-ai` CLI — the official command-line client for the WebScraping.AI API. It handles JS rendering, residential proxies, and per-country geolocation, and exposes two AI-powered endpoints (`ask`, `extract`) for one-shot question-answering and structured data extraction from any web page.
+
+## When to use this skill
+
+Reach for `webscraping-ai` when you need to:
+
+- See the **actual current content** of a page (the page may have changed since training, or you've never seen it).
+- Extract **structured data** (price, title, author, …) from a page without writing selectors.
+- **Answer a question** about a specific page without scraping the whole thing first.
+- Render **JavaScript-heavy** pages (SPAs, lazy-loaded content) that a plain `curl` won't see.
+- Bypass **anti-bot protection** that's blocking `curl` / `fetch`.
+
+Don't use it for:
+
+- Pages you already have the content of (you can grep / parse it directly).
+- Pages on the user's local filesystem.
+- Heavy bulk crawls (many thousand pages) — use the API directly with backoff for that.
+
+## Quick reference
+
+The CLI exposes 7 subcommands, one per API endpoint:
+
+| Command            | What it does                                            |
+| ------------------ | ------------------------------------------------------- |
+| `html`             | Fetch the full rendered HTML of a page                  |
+| `text`             | Fetch only the visible text (plain / JSON / XML)        |
+| `selected`         | Fetch HTML of one CSS-selected area                     |
+| `selected-multiple`| Fetch HTML of multiple CSS-selected areas at once       |
+| `ask`              | Ask a natural-language question about a page (AI)       |
+| `extract`          | Extract structured fields with descriptions (AI)        |
+| `account`          | Show remaining API credits / quota                      |
+
+Authentication: set `WEBSCRAPING_AI_API_KEY`, pass `--api-key`, or run `webscraping-ai auth set <key>` once.
+
+Every command supports common flags: `--js/--no-js`, `--proxy {datacenter|residential|stealth}`, `--country <cc>`, `--device {desktop|mobile|tablet}`, `--timeout <ms>`, `--js-timeout <ms>`, `--wait-for <selector>`, `--headers '{"Cookie":"..."}'`, `--output <file>`.
+
+## How to pick the right command
+
+**Need the whole page?** → `html`. Use `--no-js` first if the page is static — it's cheaper and faster. Add `--proxy residential` if the site blocks datacenter IPs; `--proxy stealth` for heavily protected sites.
+
+**Need just the readable content** (article body, listing text)? → `text`. Returns much less noise than HTML.
+
+**Need one specific element** (the price, the title)? → `selected --selector '.price'`. Combine with `--format text` to avoid HTML wrapping.
+
+**Need several elements at once**? → `selected-multiple --selector h1 --selector .price --selector '.author'`. One request, one cost.
+
+**Need a single answer to a question**? → `ask -q "What's the price of this product?"`. Cheaper than `html` + parsing yourself when you only need one thing.
+
+**Need a JSON object of typed fields**? → `extract --fields '{"title":"Main product title","price":"Current price including currency"}'`. Returns `{ result: { title: "...", price: "..." } }`.
+
+**Need to check available credits**? → `account`.
+
+## Example invocations
+
+```bash
+# Plain HTML, fast static fetch
+webscraping-ai html https://example.com --no-js
+
+# Article body as plain text
+webscraping-ai text https://news.example.com/article --text-format plain
+
+# Just the H1, no markup
+webscraping-ai selected https://example.com --selector h1 --format text
+
+# Three elements in one request
+webscraping-ai selected-multiple https://shop.example.com/item \
+  --selector h1 --selector '.price' --selector '.availability'
+
+# AI question — single answer
+webscraping-ai ask https://shop.example.com/item -q "Is this product in stock?"
+
+# AI structured extraction (writes JSON to stdout)
+webscraping-ai extract https://shop.example.com/item \
+  --fields '{"title":"Product title","price":"Current price with currency","rating":"Average star rating as a number"}'
+
+# Pipe a list of URLs through the same command
+cat urls.txt | webscraping-ai text - --output ./articles.ndjson
+
+# Residential proxy from Germany, wait for a selector before reading
+webscraping-ai html https://geo.example.com \
+  --proxy residential --country de --wait-for '.product-card'
+```
+
+## Gotchas worth knowing
+
+- **Default proxy is `datacenter`** (cheapest). Sites that work in a real browser but return 403 here usually need `--proxy residential` or `--proxy stealth`. Residential and stealth cost more credits per request.
+- **`--js` is on by default.** Pass `--no-js` for static pages — it's significantly faster and cheaper.
+- **`selected-multiple` returns nested arrays** (`Array<Array<string>>`) — known API response shape. The outer wrapper holds all matches concatenated; flatten in your script if needed.
+- **`extract` wraps its output** under a `result` key: the parsed shape is `{ "result": { ... } }`.
+- **API key in URL.** The CLI sends the key as a query-string param. Don't log full requests in shared logs.
+- **Exit codes** are stable per error class (auth → 4, payment → 5, rate-limit → 6, server → 7, timeout → 8, connection → 9). Script around them.
+
+## Output handling
+
+- Text/HTML responses print verbatim. JSON responses pretty-print on a TTY and single-line on a pipe.
+- `--output FILE` writes the result to a file instead of stdout.
+- For batched stdin input, each URL's response is emitted in order, separated by newlines.
+
+## See also
+
+- API documentation: <https://docs.webscraping.ai>
+- Source / issues: <https://github.com/webscraping-ai/webscraping-ai-cli>