@zenrows/cli 1.9.0 → 1.11.0

package/README.md

@@ -83,6 +83,18 @@ unclaimed Zenrows Free plan account**, stores the key in `.zenrows/secrets.json`
 (0600, gitignored), and records the account in `.zenrows/account.json`
 (no secret — just the accountId, Free-period info, and a claim link).
 
+Zenrows may first ask the agent one short question before it creates the
+account. The command then fails with `SIGNUP_CHALLENGE` and prints the
+question; answer it and re-run the original command:
+
+```bash
+zenrows signup --answer "<your answer>"
+```
+
+A wrong or late answer gets a new question. Too many questions or wrong
+answers from one network pause signups there for an hour
+(`SIGNUP_CHALLENGE_LOCKED`).
+
 When you (or your agent) want to keep the usage and history, **claim** the
 account — this confirms an email (Google, GitHub or a 6-digit code); the same
 key keeps working:
@@ -96,10 +108,11 @@ zenrows usage          # plan usage, credits & concurrency (does not count again
 Opt out of auto-provisioning at any time:
 
 ```bash
-zenrows fetch <url> --no-signup # do not auto-create an account for this call
-zenrows signup --agent          # explicitly provision a Free plan account, headless
-zenrows login --api-key <key>   # use an existing key (stored 0600, gitignored)
-zenrows login --env             # use the ZENROWS_API_KEY environment variable
+zenrows fetch <url> --no-signup  # do not auto-create an account for this call
+zenrows signup --agent           # explicitly provision a Free plan account, headless
+zenrows signup --answer "<text>" # answer the signup question (SIGNUP_CHALLENGE)
+zenrows login --api-key <key>    # use an existing key (stored 0600, gitignored)
+zenrows login --env              # use the ZENROWS_API_KEY environment variable
 ```
 
 Set `auto_signup: false` in `.zenrows/policy.json` to disable auto-provisioning
@@ -134,10 +147,18 @@ few minutes if it should work, or use `--css`; preparing again does not change
 it. `zenrows extract` never falls back to Autoparse. The `fallbackAutoparse`
 adapter option is deprecated and has no effect.
 
+Pass 1 to 5 example URLs. One works; we recommend 3 or more of the same page
+type (e.g. 3 product pages), because Extract compares pages of one type to check
+the fields it found and fill in fields one page is missing. Several page types
+can share the 5 (e.g. 3 product pages + 2 search-result pages).
+
 ```bash
-zenrows extract prepare example.com --url https://example.com/p/1 --url https://example.com/list --wait
+zenrows extract prepare example.com \
+  --url https://www.example.com/products/a \
+  --url https://www.example.com/products/b \
+  --url https://www.example.com/products/c --wait
 zenrows extract prepare status example.com   # progress + monthly allowance
-zenrows extract https://example.com/p/1      # then extract again
+zenrows extract https://www.example.com/products/a   # then extract again
 ```
 
 ## 9. Batch

package/agent-plugin/README.md

@@ -62,7 +62,7 @@ Use MCP tool names — not CLI `zenrows fetch`:
 | --- | --- |
 | `zenrows` | Decision tree across all primitives |
 | `protected-fetch` | `scrape` |
-| `extract` | `extract` (fallback: `scrape` + css_extractor); on REQS007 with reason `domain_not_prepared`, `prepare_domain` then `prepared_domain_status` |
+| `extract` | `extract` (fallback: `scrape` + css_extractor); on REQS007 with reason `domain_not_prepared`, `prepare_domain` (1 to 5 example URLs; we recommend 3 or more of the same page type) then `prepared_domain_status` |
 | `batch-jobs` | `batch_create`, `batch_status`, `batch_results`, `batch_cancel` |
 | `interact-browser` | `browser_*` (escalation only) |
 | `cost-control` | Cost multipliers + escalate-with-evidence |

package/agent-plugin/skills/extract/SKILL.md

@@ -48,15 +48,20 @@ Preparing a domain:
 
 ```
 extract({ url })                                          # 1. REQS007, reason domain_not_prepared
-prepare_domain({ domain: "example.com",
-  example_urls: ["https://example.com/p/1", "https://example.com/list"] })
+prepare_domain({ domain: "example.com", example_urls: [
+  "https://www.example.com/products/a",
+  "https://www.example.com/products/b",
+  "https://www.example.com/products/c"] })
                                                           # 2. returns at once, status "preparing"
 prepared_domain_status({ domain: "example.com" })         # 3. about every 30 seconds until "ready"
 extract({ url })                                          # 4. extract again
 ```
 
-- `example_urls`: 1 to 5 real pages you want to extract, one per page kind
-  (for example a product page and a listing page).
+- `example_urls`: 1 to 5 real pages you want to extract. One works; we
+  recommend 3 or more of the same page type (e.g. 3 product pages), because
+  Extract compares pages of one type to check the fields it found and fill in
+  fields one page is missing. Several page types can share the 5 (e.g. 3
+  product pages + 2 search-result pages).
 - The domain is the host: `shop.example.com` and `example.com` are prepared
   separately, and a leading `www.` is dropped.
 - Preparing takes a few minutes and costs no credits. Each new domain uses one

package/agent-plugin/skills/interact-browser/SKILL.md

@@ -27,8 +27,10 @@ cannot handle. **Escalation only.**
 6. Always `browser_close` when done
 
 Other useful tools: `browser_evaluate`, cookies (`browser_get_cookies` /
-`browser_set_cookies` / `browser_clear_cookies`), `browser_local_storage`,
-tabs (`browser_new_tab` / `browser_switch_tab`), `browser_generate_pdf`.
+`browser_set_cookies` / `browser_clear_cookies`), local storage
+(`browser_get_local_storage` / `browser_set_local_storage` /
+`browser_clear_local_storage`), tabs (`browser_new_tab` / `browser_switch_tab`),
+`browser_generate_pdf`.
 
 ## Rules
 

package/agent-plugin/skills/zenrows/SKILL.md

@@ -20,7 +20,10 @@ You do **not** need an API key in the plugin package.
 
 - **stdio** (`npx -y @zenrows/mcp`): on first cloud call the server can
   auto-create a **free, unclaimed** Zenrows Free plan account and persist the
-  key locally. When the Free plan nears its limit (or the user asks), surface
+  key locally. Zenrows may first ask one short question (which word is a
+  bird, which word does not belong): the tool call
+  returns it; answer it yourself with the `account_signup_answer` tool, then
+  retry the call. When the Free plan nears its limit (or the user asks), surface
   the **claim URL** so a human can claim the account (confirming an email
   with Google, GitHub or a 6-digit code). Claim links expire after 30 days. The
   same key keeps working after claim. Never handle payment or upgrades —
@@ -68,7 +71,7 @@ If the user needs login, clicks, forms, sessions, or persistent state:
 | --- | --- | --- |
 | Protected Fetch | `scrape` | Default markdown; HTML / plaintext / PDF / screenshot options |
 | Extract | `extract` | Structured JSON (`extract=auto` on any prepared domain, css_extractor; autoparse deprecated). `extract=auto` is currently free; billing may apply later |
-| Prepare a domain | `prepare_domain`, `prepared_domain_status` | After an `extract` REQS007 with reason `domain_not_prepared`: prepare the domain with 1 to 5 example URLs, poll about every 30 seconds until ready, extract again. No credits; uses one monthly allowance slot. Not for `page_not_recognised`: that domain is already prepared |
+| Prepare a domain | `prepare_domain`, `prepared_domain_status` | After an `extract` REQS007 with reason `domain_not_prepared`: prepare the domain with 1 to 5 example URLs (one works; we recommend 3 or more of the same page type, because Extract compares pages of one type to check the fields it found and fill in fields one page is missing), poll about every 30 seconds until ready, extract again. No credits; uses one monthly allowance slot. Not for `page_not_recognised`: that domain is already prepared |
 | Batch | `batch_create`, `batch_status`, `batch_results`, `batch_cancel` | Same credits as Fetch; a 403 maps to `BATCH_ACCESS_DENIED` |
 | Browser | `browser_*` (start with `browser_navigate`) | Escalation-only; same backend as Browser Sessions |
 

package/dist/src/cli/commands/extract-prepare.js

@@ -5,14 +5,15 @@
  * The way out of REQS007: prepare the domain so `extract=auto` works on it.
  * Submitting returns at once; `--wait` polls until the preparation is ready or
  * failed (a few minutes). Costs no credits; each new domain uses one slot of the
- * account's monthly allowance.
+ * account's monthly allowance. One --url works; with fewer than 3 the command prints a
+ * tip to stderr (never an error): we recommend 3 or more of the same page type, up to 5.
  */
 import { ensureApiKey } from "../../core/ensure-key.js";
 import { loadConfig } from "../../core/config.js";
 import { loadPolicy } from "../../core/policy.js";
 import { log } from "../../core/logger.js";
 import { ToolkitError } from "../../core/errors.js";
-import { DEFAULT_WAIT_TIMEOUT_MS, MAX_EXAMPLE_URLS, failureNextStep, getAllowance, getPreparedDomain, normalizeDomain, prepareDomain, statusNextStep, waitForPreparedDomain, } from "../../core/prepared-domains-api.js";
+import { DEFAULT_WAIT_TIMEOUT_MS, MAX_EXAMPLE_URLS, RECOMMENDED_EXAMPLE_URLS, exampleUrlsHint, failureNextStep, getAllowance, getPreparedDomain, normalizeDomain, prepareDomain, statusNextStep, waitForPreparedDomain, } from "../../core/prepared-domains-api.js";
 import { parse, asString } from "../command.js";
 import { printError } from "../output.js";
 export const PREPARE_USAGE = "zenrows extract prepare <domain> --url <page-url> [--url <page-url>...] [--wait [--timeout <seconds>]]\n" +
@@ -23,7 +24,8 @@ export const PREPARE_HELP = [
     "Not for EXTRACT_PAGE_NOT_RECOGNISED: that domain is already prepared, and preparing it",
     "again does not change that answer.",
     "",
-    `  --url <page-url>       1 to ${MAX_EXAMPLE_URLS} real pages you want to extract, one per page kind (repeat --url)`,
+    `  --url <page-url>       a real page you want to extract (repeat --url, 1 to ${MAX_EXAMPLE_URLS}). One works;`,
+    `                         we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages)`,
     "  --wait                 poll every 30 seconds until the domain is ready or failed; off by default",
     `  --timeout <seconds>    give up waiting after this long (default ${DEFAULT_WAIT_TIMEOUT_MS / 1000})`,
     "  status <domain>        show progress and the account's monthly allowance",
@@ -33,6 +35,14 @@ export const PREPARE_HELP = [
     "account's monthly allowance; a domain that is already prepared answers ready at once and",
     "uses none. The domain is the host (shop.example.com and example.com are separate); a",
     "leading www. is dropped.",
+    "",
+    `1 to ${MAX_EXAMPLE_URLS} example URLs. One works; we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type`,
+    "(e.g. 3 product pages), because Extract compares pages of one type to check the fields",
+    `it found and fill in fields one page is missing. Several page types can share the ${MAX_EXAMPLE_URLS}`,
+    "(e.g. 3 product pages + 2 search-result pages). Three product pages:",
+    "",
+    "  zenrows extract prepare example.com --url https://www.example.com/products/a \\",
+    "    --url https://www.example.com/products/b --url https://www.example.com/products/c --wait",
 ].join("\n");
 export async function runPrepare(argv, ctx, deps = {}) {
     const isStatus = argv[0] === "status";
@@ -69,6 +79,10 @@ export async function runPrepare(argv, ctx, deps = {}) {
             return 0;
         }
         log.step(`Prepare ${domain} for Extract (${urls.length} example URL${urls.length === 1 ? "" : "s"})…`);
+        // Advice on stderr only: the --json payload on stdout and the exit code are unchanged.
+        const hint = exampleUrlsHint(urls.length);
+        if (hint)
+            log.dim(hint);
         let record = await prepareDomain(domain, urls, opts);
         if (record.already_in_progress)
             log.info(`${domain} was already being prepared; following that preparation.`);
@@ -108,7 +122,7 @@ function validateUrls(raw, domain) {
             code: "INVALID_USAGE",
             message: `Pass 1 to ${MAX_EXAMPLE_URLS} example page URLs with --url (got ${urls.length}).`,
             likely_cause: "Preparation learns the domain from real pages you want to extract.",
-            next_action: "Add one --url per page kind, e.g. a product page and a listing page.",
+            next_action: `Add 1 to ${MAX_EXAMPLE_URLS} --url pages. One works; ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type work better.`,
             suggested_commands: [`zenrows extract prepare ${domain} --url https://${domain}/<page>`],
         });
     }

package/dist/src/cli/commands/signup.js

@@ -4,9 +4,13 @@
  * There is no documented programmatic signup API, so we never fake account
  * creation: we explain what is needed, open the signup URL, and hand off to
  * `login`.
+ *
+ * `--agent` provisions a Free plan account headless, and `--answer` answers the
+ * one short question signup may ask an agent first (SIGNUP_CHALLENGE).
  */
-import { saveApiKey } from "../../core/auth.js";
-import { readAccount, signupAgent, writeAccount } from "../../core/agent-account.js";
+import { answerSignupChallenge, readAccount, signupAgent } from "../../core/agent-account.js";
+import { storeSignup } from "../../core/ensure-key.js";
+import { ToolkitError } from "../../core/errors.js";
 import { attributionEnabled } from "../../core/config.js";
 import { log } from "../../core/logger.js";
 import { openUrl, SIGNUP_URL, withParams } from "../../core/open-url.js";
@@ -15,9 +19,42 @@ import { parse } from "../command.js";
 export const signup = {
     name: "signup",
     summary: "Create a Zenrows account / obtain an API key (opens the browser).",
-    usage: "zenrows signup [--agent] [--no-open]",
+    usage: 'zenrows signup [--agent] [--no-open] [--answer "<text>"]',
+    help: [
+        "Without flags, opens the Zenrows signup page in the browser.",
+        "",
+        "  --agent           Create a Free plan account headless and store its key.",
+        "  --no-open         Print the signup URL instead of opening it.",
+        '  --answer "<text>" Answer the question Zenrows asked before creating an account',
+        "                    (error SIGNUP_CHALLENGE). Answer it yourself, then re-run",
+        "                    your original command.",
+    ].join("\n"),
     async run(argv, ctx) {
-        const { values } = parse(argv, { "no-open": { type: "boolean" }, agent: { type: "boolean" } });
+        const { values } = parse(argv, {
+            "no-open": { type: "boolean" },
+            agent: { type: "boolean" },
+            answer: { type: "string" },
+        });
+        if (values.answer !== undefined) {
+            const answer = String(values.answer).trim();
+            if (!answer) {
+                throw new ToolkitError({
+                    code: "INVALID_USAGE",
+                    message: "--answer needs the answer text.",
+                    likely_cause: "An empty value was passed to --answer.",
+                    next_action: 'Run: zenrows signup --answer "<your answer>"',
+                });
+            }
+            const res = await answerSignupChallenge(answer);
+            const account = storeSignup(res);
+            log.success("Right answer. Created a Zenrows Free plan account and stored its API key.");
+            log.dim(`Claim it anytime (keeps your usage): ${account.claimUrl}`);
+            log.info("Now re-run your original command.");
+            if (ctx.json) {
+                log.out(JSON.stringify({ ok: true, accountId: res.accountId, claimUrl: res.claimUrl, next_action: "Re-run your original command." }, null, 2));
+            }
+            return 0;
+        }
         if (values.agent) {
             const account = readAccount();
             if (account) {
@@ -25,9 +62,7 @@ export const signup = {
                 return 0;
             }
             const res = await signupAgent();
-            saveApiKey(res.apiKey);
-            const now = new Date().toISOString();
-            writeAccount({ accountId: res.accountId, unclaimed: true, claimUrl: res.claimUrl, createdAt: now, claimUrlIssuedAt: now });
+            storeSignup(res);
             log.success("Created a Zenrows Free plan account.");
             log.dim(`Claim it anytime: ${res.claimUrl}`);
             if (ctx.json)

package/dist/src/core/agent-account.js

@@ -12,6 +12,7 @@ import { ToolkitError } from "./errors.js";
 import { AGENT_SIGNUP_API_URL, WELL_KNOWN_PROTECTED_RESOURCE } from "./open-url.js";
 import { detectClient } from "./provenance.js";
 import { findWorkspace, readJson, workspacePaths, writePrivateJson } from "./workspace.js";
+import { challengeError, challengeExpired, challengeLockedError, clearChallenge, isChallengeLocked, noChallengeError, parseChallenge, readChallenge, wasWrongAnswer, writeChallenge, } from "./signup-challenge.js";
 /** Env var to override the signup endpoint (highest priority; for local testing). */
 export const SIGNUP_URL_ENV = "ZENROWS_AGENT_SIGNUP_URL";
 /** Env var to override the discovery base origin (for local testing). */
@@ -117,8 +118,25 @@ export function clearAccount(projectRoot) {
     return true;
 }
 export async function signupAgent(opts = {}) {
-    const urls = opts.url ? [opts.url] : await signupCandidates(undefined, { fetchImpl: opts.fetchImpl });
-    const doFetch = opts.fetchImpl ?? fetch;
+    const urls = opts.url ? [opts.url] : await signupCandidates(opts.projectRoot, { fetchImpl: opts.fetchImpl });
+    return postSignup(urls, undefined, opts);
+}
+/**
+ * Answer the pending signup question (see signup-challenge.ts). Posts it to
+ * the signup URL that asked it; a right answer creates the account like a
+ * plain signup does. A wrong answer stores and reports the new question.
+ */
+export async function answerSignupChallenge(answer, opts = {}) {
+    const pending = readChallenge(opts.projectRoot);
+    if (!pending)
+        throw noChallengeError(false);
+    if (challengeExpired(pending)) {
+        clearChallenge(opts.projectRoot);
+        throw noChallengeError(true);
+    }
+    return postSignup([pending.signupUrl], { challenge_id: pending.id, answer }, opts);
+}
+function signupHeaders(projectRoot) {
     // Signup is the one functional request the CLI must make — no separate
     // telemetry beacon exists. When attribution is enabled we ride anonymous
     // provenance on it (a random id + coarse host/runtime brand). No PII, no URLs.
@@ -126,7 +144,7 @@ export async function signupAgent(opts = {}) {
     const headers = {
         "content-type": "application/json",
         "User-Agent": CLI_USER_AGENT,
-        ...agentClientHeader(),
+        ...agentClientHeader({ projectRoot }),
     };
     if (attributionEnabled()) {
         const p = detectClient();
@@ -138,14 +156,21 @@ export async function signupAgent(opts = {}) {
         headers["X-ZR-Node"] = p.node;
         headers["X-ZR-CI"] = p.ci ? "1" : "0";
     }
+    return headers;
+}
+async function postSignup(urls, body, opts) {
+    const doFetch = opts.fetchImpl ?? fetch;
+    const headers = signupHeaders(opts.projectRoot);
     // Try each candidate in order; on any failure fall back to the next (the last
     // is always the built-in default). This is the safety net: a wrong/blocked
     // discovered endpoint can never leave provisioning worse off than no discovery.
+    // A signup question or the wrong-answer lock is a real answer from a working
+    // endpoint, so it ends the loop.
     let lastErr;
     for (const url of urls) {
         let res;
         try {
-            res = await doFetch(url, { method: "POST", headers });
+            res = await doFetch(url, { method: "POST", headers, ...(body ? { body: JSON.stringify(body) } : {}) });
         }
         catch (err) {
             lastErr = new ToolkitError({
@@ -156,9 +181,24 @@ export async function signupAgent(opts = {}) {
             });
             continue;
         }
-        if (res.status === 201)
+        if (res.status === 201) {
+            clearChallenge(opts.projectRoot);
             return (await res.json());
-        lastErr = signupError(res.status, await res.text());
+        }
+        const text = await res.text();
+        if (res.status === 428) {
+            const challenge = parseChallenge(text, url);
+            if (challenge) {
+                writeChallenge(challenge, opts.projectRoot);
+                throw challengeError(challenge, { wrong: body !== undefined || wasWrongAnswer(text) });
+            }
+        }
+        if (res.status === 429 && isChallengeLocked(text)) {
+            clearChallenge(opts.projectRoot);
+            const ra = Number(res.headers.get("retry-after"));
+            throw challengeLockedError(Number.isFinite(ra) && ra > 0 ? ra : undefined);
+        }
+        lastErr = signupError(res.status, text);
     }
     throw (lastErr ??
         new ToolkitError({

package/dist/src/core/config.js

@@ -14,7 +14,7 @@ export const CONFIG_VERSION = "0.1.0";
  * entrypoint) so core modules — telemetry, signup provenance — can read it
  * without an import cycle. `VERSION` in `cli/index.ts` re-exports this.
  */
-export const CLI_VERSION = "1.9.0"; // x-release-please-version
+export const CLI_VERSION = "1.11.0"; // x-release-please-version
 /** The User-Agent on every request to a Zenrows API. The gateway reads the version from it. */
 export const CLI_USER_AGENT = `zenrows-cli/${CLI_VERSION}`;
 /** Env var to override the Fetch and Extract API base (local/staging testing). */

package/dist/src/core/ensure-key.js

@@ -7,8 +7,15 @@ export async function ensureApiKey(policy, opts = {}) {
     if (!policy.auto_signup)
         return requireApiKey(opts.projectRoot); // throws AUTH_MISSING
     const doSignup = opts.signupImpl ?? signupAgent;
-    const res = await doSignup();
-    saveApiKey(res.apiKey, opts.projectRoot);
+    // May throw SIGNUP_CHALLENGE: the agent answers with `zenrows signup --answer`.
+    const res = await doSignup({ projectRoot: opts.projectRoot });
+    const account = storeSignup(res, opts.projectRoot);
+    opts.onProvision?.(account);
+    return res.apiKey;
+}
+/** Persist a fresh agent signup: the key (0600) and the account metadata. */
+export function storeSignup(res, projectRoot) {
+    saveApiKey(res.apiKey, projectRoot);
     const account = {
         accountId: res.accountId,
         unclaimed: true,
@@ -16,7 +23,6 @@ export async function ensureApiKey(policy, opts = {}) {
         createdAt: new Date().toISOString(),
     };
     account.claimUrlIssuedAt = account.createdAt;
-    writeAccount(account, opts.projectRoot);
-    opts.onProvision?.(account);
-    return res.apiKey;
+    writeAccount(account, projectRoot);
+    return account;
 }

package/dist/src/core/errors.js

@@ -16,6 +16,7 @@ export class ToolkitError extends Error {
     job_id;
     idempotency_key;
     retry_after;
+    challenge;
     constructor(shape) {
         super(shape.message);
         this.name = "ToolkitError";
@@ -28,6 +29,7 @@ export class ToolkitError extends Error {
         this.job_id = shape.job_id;
         this.idempotency_key = shape.idempotency_key;
         this.retry_after = shape.retry_after;
+        this.challenge = shape.challenge;
     }
     toJSON() {
         return {
@@ -41,6 +43,7 @@ export class ToolkitError extends Error {
             ...(this.job_id ? { job_id: this.job_id } : {}),
             ...(this.idempotency_key ? { idempotency_key: this.idempotency_key } : {}),
             ...(this.retry_after !== undefined ? { retry_after: this.retry_after } : {}),
+            ...(this.challenge ? { challenge: this.challenge } : {}),
         };
     }
 }

package/dist/src/core/http.js

@@ -314,7 +314,7 @@ export function extractDomainNotPrepared(url, body) {
         code: "EXTRACT_DOMAIN_NOT_PREPARED",
         message: `Extract hasn't prepared ${domain} yet. Prepare it with \`${prepare}\` (a few example URLs, a few minutes, no credits), then retry; meanwhile use --css.`,
         likely_cause: detail ?? `Extract has not prepared ${domain} yet.`,
-        next_action: `Run \`${prepare} --wait\` (up to 5 --url pages, one per page kind you want), then retry \`zenrows extract ${url}\`. Without --wait, check progress about every 30 seconds with \`zenrows extract prepare status ${domain}\`. Meanwhile extract with --css and explicit selectors.`,
+        next_action: `Run \`${prepare} --wait\` (1 to 5 --url pages: one works; we recommend 3 or more of the same page type), then retry \`zenrows extract ${url}\`. Without --wait, check progress about every 30 seconds with \`zenrows extract prepare status ${domain}\`. Meanwhile extract with --css and explicit selectors.`,
         suggested_commands: [`${prepare} --wait`, `zenrows extract prepare status ${domain}`, css],
         server_code: "REQS007",
     });

package/dist/src/core/prepared-domains-api.js

@@ -4,7 +4,8 @@
  * `extract=auto` answers REQS007 on a domain Extract has not prepared yet. Preparing
  * it is a call any key can make: `POST {apiBase}/extract/prepared-domains` with the
  * domain and 1 to 5 example URLs answers 202 at once; the preparation itself takes a
- * few minutes and costs no credits. `GET .../{domain}` reports progress and
+ * few minutes and costs no credits. One example URL works; we recommend 3 or more of
+ * the same page type (see `RECOMMENDED_EXAMPLE_URLS`). `GET .../{domain}` reports progress and
  * `GET .../allowance` the account's monthly allowance of new domains. Auth is the
  * `X-API-Key` header; errors are problem+json and we branch on `status` + `code`.
  * `fetchImpl` / `sleepImpl` / `now` are injectable for tests.
@@ -14,6 +15,23 @@ import { registerSecret } from "./logger.js";
 import { CLI_USER_AGENT, loadConfig } from "./config.js";
 import { agentClientHeader } from "./agent-client.js";
 export const MAX_EXAMPLE_URLS = 5;
+/**
+ * How many example URLs to recommend: 3 or more of the same page type, because
+ * Extract compares pages of one type to check the fields it found and fill in fields
+ * one page is missing. Advice only: 1 and 2 URLs are still valid.
+ */
+export const RECOMMENDED_EXAMPLE_URLS = 3;
+/**
+ * A short, non-blocking hint when fewer than `RECOMMENDED_EXAMPLE_URLS` example
+ * URLs were given; undefined otherwise. Never an error.
+ */
+export function exampleUrlsHint(count) {
+    if (count >= RECOMMENDED_EXAMPLE_URLS)
+        return undefined;
+    return (`Tip: ${count === 1 ? "one example URL works" : `${count} example URLs work`}; we recommend ` +
+        `${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages), up to ${MAX_EXAMPLE_URLS} in total, ` +
+        "because Extract compares pages of one type to check the fields it found and fill in fields one page is missing.");
+}
 /** How often `--wait` polls, and for how long by default. */
 export const DEFAULT_POLL_INTERVAL_MS = 30_000;
 export const DEFAULT_WAIT_TIMEOUT_MS = 15 * 60_000;
@@ -247,7 +265,7 @@ export function failureNextStep(domain, failure) {
             step = `Run the same zenrows extract prepare ${domain} command again.`;
             break;
         case "discovery_failed":
-            step = `Run zenrows extract prepare ${domain} again with 1 to ${MAX_EXAMPLE_URLS} --url page URLs.`;
+            step = `Run zenrows extract prepare ${domain} again with 1 to ${MAX_EXAMPLE_URLS} --url page URLs; we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages).`;
             break;
         case "unpreparable":
             step = `Run zenrows extract prepare ${domain} again with different --url pages from the same site; the same URLs fail again.`;

package/dist/src/core/signup-challenge.js

@@ -0,0 +1,114 @@
+/**
+ * The signup challenge: before it creates an account, the signup endpoint may
+ * answer 428 with one short natural-language question for the agent running
+ * the CLI to answer itself. The CLI never tries to solve it.
+ *
+ * Each CLI run is its own process, so the pending question is kept in
+ * `.zenrows/signup-challenge.json` (no secret: the id, the question, when it
+ * expires, and the signup URL that issued it, which is where the answer goes).
+ */
+import { existsSync, unlinkSync } from "node:fs";
+import { join } from "node:path";
+import { ToolkitError } from "./errors.js";
+import { findWorkspace, readJson, workspacePaths, writeJson } from "./workspace.js";
+/** The command an agent runs to answer the pending question. */
+export const ANSWER_COMMAND = 'zenrows signup --answer "<your answer>"';
+export function challengePath(projectRoot) {
+    const ws = projectRoot ? workspacePaths(projectRoot) : (findWorkspace() ?? workspacePaths());
+    return join(ws.dir, "signup-challenge.json");
+}
+export function readChallenge(projectRoot) {
+    try {
+        const c = readJson(challengePath(projectRoot));
+        if (!c || typeof c.id !== "string" || typeof c.prompt !== "string" || typeof c.signupUrl !== "string")
+            return null;
+        return c;
+    }
+    catch {
+        return null;
+    }
+}
+export function writeChallenge(c, projectRoot) {
+    writeJson(challengePath(projectRoot), c);
+}
+export function clearChallenge(projectRoot) {
+    const file = challengePath(projectRoot);
+    if (!existsSync(file))
+        return;
+    try {
+        unlinkSync(file);
+    }
+    catch {
+        // best effort
+    }
+}
+/** True when the stored question can no longer be answered. */
+export function challengeExpired(c, now = Date.now()) {
+    const t = Date.parse(c.expiresAt);
+    return Number.isFinite(t) && t <= now;
+}
+function parseBody(body) {
+    try {
+        const json = JSON.parse(body);
+        return json && typeof json === "object" ? json : null;
+    }
+    catch {
+        return null;
+    }
+}
+/**
+ * Parse a 428 `challenge_required` body into a pending challenge, or null when
+ * the body is not one (then the caller reports a generic failure).
+ */
+export function parseChallenge(body, signupUrl, now = Date.now()) {
+    const json = parseBody(body);
+    if (!json || json.error !== "challenge_required")
+        return null;
+    const ch = json.challenge;
+    if (!ch || typeof ch.id !== "string" || !ch.id || typeof ch.prompt !== "string" || !ch.prompt.trim())
+        return null;
+    const ttl = typeof ch.expires_in === "number" && ch.expires_in > 0 ? ch.expires_in : 600;
+    return { id: ch.id, prompt: ch.prompt.trim(), expiresAt: new Date(now + ttl * 1000).toISOString(), signupUrl };
+}
+/** True when a 428 says the previous answer was wrong or came too late. */
+export function wasWrongAnswer(body) {
+    const msg = parseBody(body)?.message;
+    return typeof msg === "string" && /^wrong or expired/i.test(msg.trim());
+}
+/** True when a 429 is the per-network lock after too many questions or wrong answers. */
+export function isChallengeLocked(body) {
+    return parseBody(body)?.error === "challenge_locked";
+}
+/** The error an agent sees while a question is waiting for its answer. */
+export function challengeError(c, opts = {}) {
+    const lead = opts.wrong ? "That answer was wrong or came too late, so here is a new question. " : "";
+    return new ToolkitError({
+        code: "SIGNUP_CHALLENGE",
+        message: `${lead}Zenrows needs one quick answer before it creates an account: ${c.prompt}`,
+        likely_cause: "No API key is configured, and new accounts created by agents answer one short question first.",
+        next_action: `Answer the question yourself, then run: ${ANSWER_COMMAND}. When it succeeds, re-run your original command.`,
+        suggested_commands: [ANSWER_COMMAND, "zenrows login --api-key <your-key>"],
+        challenge: { id: c.id, question: c.prompt, expires_at: c.expiresAt },
+    });
+}
+export function challengeLockedError(retryAfter) {
+    return new ToolkitError({
+        code: "SIGNUP_CHALLENGE_LOCKED",
+        message: "Too many signup questions or wrong answers from this network, so Zenrows paused account creation here.",
+        likely_cause: "Too many signup questions asked, or wrong answers given, from this network within an hour.",
+        next_action: "Try again in an hour by re-running your original command, which asks a new question. If you already have a Zenrows API key, use it now: zenrows login --api-key <key> (or set ZENROWS_API_KEY).",
+        suggested_commands: ["zenrows login --api-key <your-key>"],
+        ...(retryAfter !== undefined ? { retry_after: retryAfter } : {}),
+    });
+}
+export function noChallengeError(expired) {
+    return new ToolkitError({
+        code: "SIGNUP_CHALLENGE_MISSING",
+        message: expired ? "The signup question expired before it was answered." : "There is no signup question waiting for an answer.",
+        likely_cause: expired
+            ? "Questions are valid for a few minutes."
+            : "No signup was attempted from this directory, or the question was already answered.",
+        next_action: "Re-run your original command (for example zenrows fetch <url>) to get a new question, then answer it with: " + ANSWER_COMMAND,
+        suggested_commands: ["zenrows fetch <url>"],
+    });
+}

package/dist/src/core/workspace.js

@@ -70,6 +70,7 @@ export function ensureGitignore(paths) {
         `${WORKSPACE_DIRNAME}/secrets.json`,
         `${WORKSPACE_DIRNAME}/config.json`,
         `${WORKSPACE_DIRNAME}/account.json`,
+        `${WORKSPACE_DIRNAME}/signup-challenge.json`,
         `${WORKSPACE_DIRNAME}/runs/`,
         `${WORKSPACE_DIRNAME}/traces/`,
         `${WORKSPACE_DIRNAME}/logs/`,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zenrows/cli",
-  "version": "1.9.0",
+  "version": "1.11.0",
   "description": "Open-source CLI, MCP, skills, templates, workflows, recipes, and evals layer for giving AI agents reliable access to protected web data through Zenrows cloud infrastructure.",
   "type": "module",
   "license": "MIT",

package/registry/capabilities.json

@@ -17,7 +17,7 @@
       "command": "zenrows extract",
       "backend": "Extract — GET https://api.zenrows.com/v1/ (extract=auto, css_extractor, outputs, response_type; autoparse deprecated)",
       "requiresAuth": true,
-      "notes": "Default is extract=auto, which works on any prepared domain with any key. A REQS007 (legacy AUTH010) is reported, never retried, with the next step for its reason: on a domain not prepared yet (EXTRACT_DOMAIN_NOT_PREPARED), prepare it with `zenrows extract prepare <domain> --url <page> [--wait]` (POST /v1/extract/prepared-domains: 1 to 5 example URLs, a few minutes, no credits) and check it with `zenrows extract prepare status <domain>`; on a page Extract did not recognise on a prepared domain (EXTRACT_PAGE_NOT_RECOGNISED), try a product, article or listing page, retry it once in a few minutes, or use --css, because preparing again does not change it. CSS / outputs / markdown work on any domain; autoparse is deprecated and runs only with --autoparse."
+      "notes": "Default is extract=auto, which works on any prepared domain with any key. A REQS007 (legacy AUTH010) is reported, never retried, with the next step for its reason: on a domain not prepared yet (EXTRACT_DOMAIN_NOT_PREPARED), prepare it with `zenrows extract prepare <domain> --url <page> [--wait]` (POST /v1/extract/prepared-domains: 1 to 5 example URLs; one works, we recommend 3 or more of the same page type because Extract compares pages of one type to check the fields it found and fill in fields one page is missing; a few minutes, no credits) and check it with `zenrows extract prepare status <domain>`; on a page Extract did not recognise on a prepared domain (EXTRACT_PAGE_NOT_RECOGNISED), try a product, article or listing page, retry it once in a few minutes, or use --css, because preparing again does not change it. CSS / outputs / markdown work on any domain; autoparse is deprecated and runs only with --autoparse."
     },
     "batch": {
       "key": "batch",

package/skills/extract/SKILL.md

@@ -43,15 +43,21 @@ of three cases it is and prints the next step. It never retries with Autoparse.
 Preparing a domain:
 
 ```
-zenrows extract https://example.com/p/1             # 1. fails: EXTRACT_DOMAIN_NOT_PREPARED
+zenrows extract https://www.example.com/products/a  # 1. fails: EXTRACT_DOMAIN_NOT_PREPARED
 zenrows extract prepare example.com \
-  --url https://example.com/p/1 --url https://example.com/list --wait
+  --url https://www.example.com/products/a \
+  --url https://www.example.com/products/b \
+  --url https://www.example.com/products/c --wait
                                                     # 2. prepare it and wait until ready (a few minutes)
-zenrows extract https://example.com/p/1             # 3. extract again
+zenrows extract https://www.example.com/products/a  # 3. extract again
 ```
 
-- `--url`: 1 to 5 real pages you want to extract, one per page kind (for
-  example a product page and a listing page).
+- `--url`: 1 to 5 real pages you want to extract. One works; we recommend 3 or
+  more of the same page type (e.g. 3 product pages), because Extract compares
+  pages of one type to check the fields it found and fill in fields one page is
+  missing. Several page types can share the 5 (e.g. 3 product pages + 2
+  search-result pages). With fewer than 3 the command prints a tip on stderr;
+  it is never an error.
 - Without `--wait` the command returns at once; then check about every 30
   seconds with `zenrows extract prepare status example.com` until it says
   `ready`. `--wait` polls at the same pace.

package/skills/zenrows/SKILL.md

@@ -20,6 +20,13 @@ the toolkit automatically creates a **free, unclaimed Zenrows Free plan account*
 key locally — no human step required. To opt out, pass `--no-signup` or set
 `auto_signup: false` in `.zenrows/policy.json`, then use `zenrows login --api-key <key>`.
 
+Before it creates the account, Zenrows may ask you one short question (error code
+`SIGNUP_CHALLENGE`, with the question in the message and in `error.challenge.question`).
+Answer it yourself, do not ask the human, then run `zenrows signup --answer "<your answer>"`
+and re-run your original command. A wrong answer returns a new question the same way.
+After too many questions or wrong answers from one network (`SIGNUP_CHALLENGE_LOCKED`), wait an hour or use
+`zenrows login --api-key <key>`.
+
 When the Free plan nears its limit (or the user asks), tell the human they can **claim** the
 account to keep their usage and history: run `zenrows account claim` (or open the claim URL
 shown on quota errors). It prints a valid link, requesting a fresh one when the stored link is
@@ -83,7 +90,8 @@ Run `zenrows status` for the live capability matrix. As of this toolkit:
 
 - Extract: `extract=auto` works on any prepared domain. On one not prepared yet
   (`EXTRACT_DOMAIN_NOT_PREPARED`), run `zenrows extract prepare <domain> --url
-  <page> --wait`, then extract again. `EXTRACT_PAGE_NOT_RECOGNISED` means the
+  <page> --wait` (repeat `--url`, 1 to 5: one works; we recommend 3 or more of
+  the same page type), then extract again. `EXTRACT_PAGE_NOT_RECOGNISED` means the
   domain is prepared but not this page: do not prepare again; see [[extract]].
 - Batch: the reliable path at scale, at Fetch's prices (create, list, status, wait, results, content, pause, resume, rerun, delete). Estimate specs locally with no key.
 - Crawl: collects the URLs behind one start page, optionally with each page's HTML.