@zenrows/cli 1.10.0 → 1.11.0

package/README.md

@@ -147,10 +147,18 @@ few minutes if it should work, or use `--css`; preparing again does not change
 it. `zenrows extract` never falls back to Autoparse. The `fallbackAutoparse`
 adapter option is deprecated and has no effect.
 
+Pass 1 to 5 example URLs. One works; we recommend 3 or more of the same page
+type (e.g. 3 product pages), because Extract compares pages of one type to check
+the fields it found and fill in fields one page is missing. Several page types
+can share the 5 (e.g. 3 product pages + 2 search-result pages).
+
 ```bash
-zenrows extract prepare example.com --url https://example.com/p/1 --url https://example.com/list --wait
+zenrows extract prepare example.com \
+  --url https://www.example.com/products/a \
+  --url https://www.example.com/products/b \
+  --url https://www.example.com/products/c --wait
 zenrows extract prepare status example.com   # progress + monthly allowance
-zenrows extract https://example.com/p/1      # then extract again
+zenrows extract https://www.example.com/products/a   # then extract again
 ```
 
 ## 9. Batch

package/agent-plugin/README.md

@@ -62,7 +62,7 @@ Use MCP tool names — not CLI `zenrows fetch`:
 | --- | --- |
 | `zenrows` | Decision tree across all primitives |
 | `protected-fetch` | `scrape` |
-| `extract` | `extract` (fallback: `scrape` + css_extractor); on REQS007 with reason `domain_not_prepared`, `prepare_domain` then `prepared_domain_status` |
+| `extract` | `extract` (fallback: `scrape` + css_extractor); on REQS007 with reason `domain_not_prepared`, `prepare_domain` (1 to 5 example URLs; we recommend 3 or more of the same page type) then `prepared_domain_status` |
 | `batch-jobs` | `batch_create`, `batch_status`, `batch_results`, `batch_cancel` |
 | `interact-browser` | `browser_*` (escalation only) |
 | `cost-control` | Cost multipliers + escalate-with-evidence |

package/agent-plugin/skills/extract/SKILL.md

@@ -48,15 +48,20 @@ Preparing a domain:
 
 ```
 extract({ url })                                          # 1. REQS007, reason domain_not_prepared
-prepare_domain({ domain: "example.com",
-  example_urls: ["https://example.com/p/1", "https://example.com/list"] })
+prepare_domain({ domain: "example.com", example_urls: [
+  "https://www.example.com/products/a",
+  "https://www.example.com/products/b",
+  "https://www.example.com/products/c"] })
                                                           # 2. returns at once, status "preparing"
 prepared_domain_status({ domain: "example.com" })         # 3. about every 30 seconds until "ready"
 extract({ url })                                          # 4. extract again
 ```
 
-- `example_urls`: 1 to 5 real pages you want to extract, one per page kind
-  (for example a product page and a listing page).
+- `example_urls`: 1 to 5 real pages you want to extract. One works; we
+  recommend 3 or more of the same page type (e.g. 3 product pages), because
+  Extract compares pages of one type to check the fields it found and fill in
+  fields one page is missing. Several page types can share the 5 (e.g. 3
+  product pages + 2 search-result pages).
 - The domain is the host: `shop.example.com` and `example.com` are prepared
   separately, and a leading `www.` is dropped.
 - Preparing takes a few minutes and costs no credits. Each new domain uses one

package/agent-plugin/skills/zenrows/SKILL.md

@@ -71,7 +71,7 @@ If the user needs login, clicks, forms, sessions, or persistent state:
 | --- | --- | --- |
 | Protected Fetch | `scrape` | Default markdown; HTML / plaintext / PDF / screenshot options |
 | Extract | `extract` | Structured JSON (`extract=auto` on any prepared domain, css_extractor; autoparse deprecated). `extract=auto` is currently free; billing may apply later |
-| Prepare a domain | `prepare_domain`, `prepared_domain_status` | After an `extract` REQS007 with reason `domain_not_prepared`: prepare the domain with 1 to 5 example URLs, poll about every 30 seconds until ready, extract again. No credits; uses one monthly allowance slot. Not for `page_not_recognised`: that domain is already prepared |
+| Prepare a domain | `prepare_domain`, `prepared_domain_status` | After an `extract` REQS007 with reason `domain_not_prepared`: prepare the domain with 1 to 5 example URLs (one works; we recommend 3 or more of the same page type, because Extract compares pages of one type to check the fields it found and fill in fields one page is missing), poll about every 30 seconds until ready, extract again. No credits; uses one monthly allowance slot. Not for `page_not_recognised`: that domain is already prepared |
 | Batch | `batch_create`, `batch_status`, `batch_results`, `batch_cancel` | Same credits as Fetch; a 403 maps to `BATCH_ACCESS_DENIED` |
 | Browser | `browser_*` (start with `browser_navigate`) | Escalation-only; same backend as Browser Sessions |
 

package/dist/src/cli/commands/extract-prepare.js

@@ -5,14 +5,15 @@
  * The way out of REQS007: prepare the domain so `extract=auto` works on it.
  * Submitting returns at once; `--wait` polls until the preparation is ready or
  * failed (a few minutes). Costs no credits; each new domain uses one slot of the
- * account's monthly allowance.
+ * account's monthly allowance. One --url works; with fewer than 3 the command prints a
+ * tip to stderr (never an error): we recommend 3 or more of the same page type, up to 5.
  */
 import { ensureApiKey } from "../../core/ensure-key.js";
 import { loadConfig } from "../../core/config.js";
 import { loadPolicy } from "../../core/policy.js";
 import { log } from "../../core/logger.js";
 import { ToolkitError } from "../../core/errors.js";
-import { DEFAULT_WAIT_TIMEOUT_MS, MAX_EXAMPLE_URLS, failureNextStep, getAllowance, getPreparedDomain, normalizeDomain, prepareDomain, statusNextStep, waitForPreparedDomain, } from "../../core/prepared-domains-api.js";
+import { DEFAULT_WAIT_TIMEOUT_MS, MAX_EXAMPLE_URLS, RECOMMENDED_EXAMPLE_URLS, exampleUrlsHint, failureNextStep, getAllowance, getPreparedDomain, normalizeDomain, prepareDomain, statusNextStep, waitForPreparedDomain, } from "../../core/prepared-domains-api.js";
 import { parse, asString } from "../command.js";
 import { printError } from "../output.js";
 export const PREPARE_USAGE = "zenrows extract prepare <domain> --url <page-url> [--url <page-url>...] [--wait [--timeout <seconds>]]\n" +
@@ -23,7 +24,8 @@ export const PREPARE_HELP = [
     "Not for EXTRACT_PAGE_NOT_RECOGNISED: that domain is already prepared, and preparing it",
     "again does not change that answer.",
     "",
-    `  --url <page-url>       1 to ${MAX_EXAMPLE_URLS} real pages you want to extract, one per page kind (repeat --url)`,
+    `  --url <page-url>       a real page you want to extract (repeat --url, 1 to ${MAX_EXAMPLE_URLS}). One works;`,
+    `                         we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages)`,
     "  --wait                 poll every 30 seconds until the domain is ready or failed; off by default",
     `  --timeout <seconds>    give up waiting after this long (default ${DEFAULT_WAIT_TIMEOUT_MS / 1000})`,
     "  status <domain>        show progress and the account's monthly allowance",
@@ -33,6 +35,14 @@ export const PREPARE_HELP = [
     "account's monthly allowance; a domain that is already prepared answers ready at once and",
     "uses none. The domain is the host (shop.example.com and example.com are separate); a",
     "leading www. is dropped.",
+    "",
+    `1 to ${MAX_EXAMPLE_URLS} example URLs. One works; we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type`,
+    "(e.g. 3 product pages), because Extract compares pages of one type to check the fields",
+    `it found and fill in fields one page is missing. Several page types can share the ${MAX_EXAMPLE_URLS}`,
+    "(e.g. 3 product pages + 2 search-result pages). Three product pages:",
+    "",
+    "  zenrows extract prepare example.com --url https://www.example.com/products/a \\",
+    "    --url https://www.example.com/products/b --url https://www.example.com/products/c --wait",
 ].join("\n");
 export async function runPrepare(argv, ctx, deps = {}) {
     const isStatus = argv[0] === "status";
@@ -69,6 +79,10 @@ export async function runPrepare(argv, ctx, deps = {}) {
             return 0;
         }
         log.step(`Prepare ${domain} for Extract (${urls.length} example URL${urls.length === 1 ? "" : "s"})…`);
+        // Advice on stderr only: the --json payload on stdout and the exit code are unchanged.
+        const hint = exampleUrlsHint(urls.length);
+        if (hint)
+            log.dim(hint);
         let record = await prepareDomain(domain, urls, opts);
         if (record.already_in_progress)
             log.info(`${domain} was already being prepared; following that preparation.`);
@@ -108,7 +122,7 @@ function validateUrls(raw, domain) {
             code: "INVALID_USAGE",
             message: `Pass 1 to ${MAX_EXAMPLE_URLS} example page URLs with --url (got ${urls.length}).`,
             likely_cause: "Preparation learns the domain from real pages you want to extract.",
-            next_action: "Add one --url per page kind, e.g. a product page and a listing page.",
+            next_action: `Add 1 to ${MAX_EXAMPLE_URLS} --url pages. One works; ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type work better.`,
             suggested_commands: [`zenrows extract prepare ${domain} --url https://${domain}/<page>`],
         });
     }

package/dist/src/core/config.js

@@ -14,7 +14,7 @@ export const CONFIG_VERSION = "0.1.0";
  * entrypoint) so core modules — telemetry, signup provenance — can read it
  * without an import cycle. `VERSION` in `cli/index.ts` re-exports this.
  */
-export const CLI_VERSION = "1.10.0"; // x-release-please-version
+export const CLI_VERSION = "1.11.0"; // x-release-please-version
 /** The User-Agent on every request to a Zenrows API. The gateway reads the version from it. */
 export const CLI_USER_AGENT = `zenrows-cli/${CLI_VERSION}`;
 /** Env var to override the Fetch and Extract API base (local/staging testing). */

package/dist/src/core/http.js

@@ -314,7 +314,7 @@ export function extractDomainNotPrepared(url, body) {
         code: "EXTRACT_DOMAIN_NOT_PREPARED",
         message: `Extract hasn't prepared ${domain} yet. Prepare it with \`${prepare}\` (a few example URLs, a few minutes, no credits), then retry; meanwhile use --css.`,
         likely_cause: detail ?? `Extract has not prepared ${domain} yet.`,
-        next_action: `Run \`${prepare} --wait\` (up to 5 --url pages, one per page kind you want), then retry \`zenrows extract ${url}\`. Without --wait, check progress about every 30 seconds with \`zenrows extract prepare status ${domain}\`. Meanwhile extract with --css and explicit selectors.`,
+        next_action: `Run \`${prepare} --wait\` (1 to 5 --url pages: one works; we recommend 3 or more of the same page type), then retry \`zenrows extract ${url}\`. Without --wait, check progress about every 30 seconds with \`zenrows extract prepare status ${domain}\`. Meanwhile extract with --css and explicit selectors.`,
         suggested_commands: [`${prepare} --wait`, `zenrows extract prepare status ${domain}`, css],
         server_code: "REQS007",
     });

package/dist/src/core/prepared-domains-api.js

@@ -4,7 +4,8 @@
  * `extract=auto` answers REQS007 on a domain Extract has not prepared yet. Preparing
  * it is a call any key can make: `POST {apiBase}/extract/prepared-domains` with the
  * domain and 1 to 5 example URLs answers 202 at once; the preparation itself takes a
- * few minutes and costs no credits. `GET .../{domain}` reports progress and
+ * few minutes and costs no credits. One example URL works; we recommend 3 or more of
+ * the same page type (see `RECOMMENDED_EXAMPLE_URLS`). `GET .../{domain}` reports progress and
  * `GET .../allowance` the account's monthly allowance of new domains. Auth is the
  * `X-API-Key` header; errors are problem+json and we branch on `status` + `code`.
  * `fetchImpl` / `sleepImpl` / `now` are injectable for tests.
@@ -14,6 +15,23 @@ import { registerSecret } from "./logger.js";
 import { CLI_USER_AGENT, loadConfig } from "./config.js";
 import { agentClientHeader } from "./agent-client.js";
 export const MAX_EXAMPLE_URLS = 5;
+/**
+ * How many example URLs to recommend: 3 or more of the same page type, because
+ * Extract compares pages of one type to check the fields it found and fill in fields
+ * one page is missing. Advice only: 1 and 2 URLs are still valid.
+ */
+export const RECOMMENDED_EXAMPLE_URLS = 3;
+/**
+ * A short, non-blocking hint when fewer than `RECOMMENDED_EXAMPLE_URLS` example
+ * URLs were given; undefined otherwise. Never an error.
+ */
+export function exampleUrlsHint(count) {
+    if (count >= RECOMMENDED_EXAMPLE_URLS)
+        return undefined;
+    return (`Tip: ${count === 1 ? "one example URL works" : `${count} example URLs work`}; we recommend ` +
+        `${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages), up to ${MAX_EXAMPLE_URLS} in total, ` +
+        "because Extract compares pages of one type to check the fields it found and fill in fields one page is missing.");
+}
 /** How often `--wait` polls, and for how long by default. */
 export const DEFAULT_POLL_INTERVAL_MS = 30_000;
 export const DEFAULT_WAIT_TIMEOUT_MS = 15 * 60_000;
@@ -247,7 +265,7 @@ export function failureNextStep(domain, failure) {
             step = `Run the same zenrows extract prepare ${domain} command again.`;
             break;
         case "discovery_failed":
-            step = `Run zenrows extract prepare ${domain} again with 1 to ${MAX_EXAMPLE_URLS} --url page URLs.`;
+            step = `Run zenrows extract prepare ${domain} again with 1 to ${MAX_EXAMPLE_URLS} --url page URLs; we recommend ${RECOMMENDED_EXAMPLE_URLS} or more of the same page type (e.g. 3 product pages).`;
             break;
         case "unpreparable":
             step = `Run zenrows extract prepare ${domain} again with different --url pages from the same site; the same URLs fail again.`;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zenrows/cli",
-  "version": "1.10.0",
+  "version": "1.11.0",
   "description": "Open-source CLI, MCP, skills, templates, workflows, recipes, and evals layer for giving AI agents reliable access to protected web data through Zenrows cloud infrastructure.",
   "type": "module",
   "license": "MIT",

package/registry/capabilities.json

@@ -17,7 +17,7 @@
       "command": "zenrows extract",
       "backend": "Extract — GET https://api.zenrows.com/v1/ (extract=auto, css_extractor, outputs, response_type; autoparse deprecated)",
       "requiresAuth": true,
-      "notes": "Default is extract=auto, which works on any prepared domain with any key. A REQS007 (legacy AUTH010) is reported, never retried, with the next step for its reason: on a domain not prepared yet (EXTRACT_DOMAIN_NOT_PREPARED), prepare it with `zenrows extract prepare <domain> --url <page> [--wait]` (POST /v1/extract/prepared-domains: 1 to 5 example URLs, a few minutes, no credits) and check it with `zenrows extract prepare status <domain>`; on a page Extract did not recognise on a prepared domain (EXTRACT_PAGE_NOT_RECOGNISED), try a product, article or listing page, retry it once in a few minutes, or use --css, because preparing again does not change it. CSS / outputs / markdown work on any domain; autoparse is deprecated and runs only with --autoparse."
+      "notes": "Default is extract=auto, which works on any prepared domain with any key. A REQS007 (legacy AUTH010) is reported, never retried, with the next step for its reason: on a domain not prepared yet (EXTRACT_DOMAIN_NOT_PREPARED), prepare it with `zenrows extract prepare <domain> --url <page> [--wait]` (POST /v1/extract/prepared-domains: 1 to 5 example URLs; one works, we recommend 3 or more of the same page type because Extract compares pages of one type to check the fields it found and fill in fields one page is missing; a few minutes, no credits) and check it with `zenrows extract prepare status <domain>`; on a page Extract did not recognise on a prepared domain (EXTRACT_PAGE_NOT_RECOGNISED), try a product, article or listing page, retry it once in a few minutes, or use --css, because preparing again does not change it. CSS / outputs / markdown work on any domain; autoparse is deprecated and runs only with --autoparse."
     },
     "batch": {
       "key": "batch",

package/skills/extract/SKILL.md

@@ -43,15 +43,21 @@ of three cases it is and prints the next step. It never retries with Autoparse.
 Preparing a domain:
 
 ```
-zenrows extract https://example.com/p/1             # 1. fails: EXTRACT_DOMAIN_NOT_PREPARED
+zenrows extract https://www.example.com/products/a  # 1. fails: EXTRACT_DOMAIN_NOT_PREPARED
 zenrows extract prepare example.com \
-  --url https://example.com/p/1 --url https://example.com/list --wait
+  --url https://www.example.com/products/a \
+  --url https://www.example.com/products/b \
+  --url https://www.example.com/products/c --wait
                                                     # 2. prepare it and wait until ready (a few minutes)
-zenrows extract https://example.com/p/1             # 3. extract again
+zenrows extract https://www.example.com/products/a  # 3. extract again
 ```
 
-- `--url`: 1 to 5 real pages you want to extract, one per page kind (for
-  example a product page and a listing page).
+- `--url`: 1 to 5 real pages you want to extract. One works; we recommend 3 or
+  more of the same page type (e.g. 3 product pages), because Extract compares
+  pages of one type to check the fields it found and fill in fields one page is
+  missing. Several page types can share the 5 (e.g. 3 product pages + 2
+  search-result pages). With fewer than 3 the command prints a tip on stderr;
+  it is never an error.
 - Without `--wait` the command returns at once; then check about every 30
   seconds with `zenrows extract prepare status example.com` until it says
   `ready`. `--wait` polls at the same pace.

package/skills/zenrows/SKILL.md

@@ -90,7 +90,8 @@ Run `zenrows status` for the live capability matrix. As of this toolkit:
 
 - Extract: `extract=auto` works on any prepared domain. On one not prepared yet
   (`EXTRACT_DOMAIN_NOT_PREPARED`), run `zenrows extract prepare <domain> --url
-  <page> --wait`, then extract again. `EXTRACT_PAGE_NOT_RECOGNISED` means the
+  <page> --wait` (repeat `--url`, 1 to 5: one works; we recommend 3 or more of
+  the same page type), then extract again. `EXTRACT_PAGE_NOT_RECOGNISED` means the
   domain is prepared but not this page: do not prepare again; see [[extract]].
 - Batch: the reliable path at scale, at Fetch's prices (create, list, status, wait, results, content, pause, resume, rerun, delete). Estimate specs locally with no key.
 - Crawl: collects the URLs behind one start page, optionally with each page's HTML.