@zenrows/cli 1.7.0 → 1.9.0

package/skills/batch-jobs/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: batch-jobs
-description: Scale protected fetch/extract over many URLs via Batch. Cloud create/status/results/cancel/wait/retry-failed work with an API key; estimate/validate run locally with no key.
+description: "Scale protected fetch/extract over many URLs via Batch. Cloud create/list/status/wait/results/content/cancel/pause/resume/rerun/delete work with an API key; estimate/validate run locally with no key."
 version: 0.1.0
 requires_backend_capabilities: [batch]
 ---
@@ -8,22 +8,41 @@ requires_backend_capabilities: [batch]
 # Batch Jobs
 
 Process large workloads reliably and asynchronously. Batch is where Zenrows'
-high-scale anti-bot advantage becomes obvious — Zenrows wins when the workflow
-runs over thousands, millions, or recurring sets of URLs.
+high-scale anti-bot advantage becomes obvious: thousands to 100,000 URLs per
+job, at Fetch's prices.
 
 > Zenrows **Batch** runs on a separate host (`async.api.zenrows.com/v1`) and
-> spends the same credits as Fetch. The cloud subcommands work with any active
-> API key; a 403 maps to `BATCH_ACCESS_DENIED`. The toolkit never fakes a
-> cloud run.
+> costs the same credits as Fetch (1 basic, 5 js_render, 10 premium_proxy, 25
+> both; Adaptive Stealth Mode 1 to 25). Failed tasks are not billed, except
+> target 404/410 answers, as on Fetch. The cloud subcommands work with any
+> active API key; a 403 maps to `BATCH_ACCESS_DENIED`. The toolkit never fakes
+> a cloud run.
+
+## Spec and estimate (local, no key)
 
-## Local (no key, always works)
 ```
-zenrows batch estimate jobs.jsonl     # validate the spec + estimate credits
+zenrows batch estimate jobs.jsonl [--premium-proxy] [--js-render] [--manual]
 ```
 
-`jobs.jsonl` is one JSON object per line, each with a `url` (plus optional
-per-task keys: `external_id`, `metadata`, and scrape params like `js_render`,
-`premium_proxy`, `proxy_country`, `mode`, `autoparse`, `extract`).
+`jobs.jsonl` is one JSON object per line, each with a `url` (http or https,
+up to 2048 bytes). Optional per-task keys:
+
+- `external_id`: up to 128 letters, digits, `.`, `_` or `-`; echoed on each
+  result row (it does not have to be unique).
+- `metadata`: up to 20 string-to-string entries; keys 1 to 64 of the same
+  characters, values up to 1024 bytes (UTF-8), 4 KiB in total.
+- `method` and `body`: `"method": "POST"` sends `body` (up to 16 KiB): a JSON
+  string goes as a form body, any other JSON value as JSON (`"body": null` means
+  no body). A POST task cannot
+  use `js_render`, `js_instructions` or `json_response`.
+- scrape params, flat (`js_render`, `premium_proxy`, `proxy_country`, `mode`,
+  `autoparse`, `extract`, ...) or nested under `zenrows_params`. Booleans are
+  `true`/`false` (or `"true"`/`"false"`).
+
+`estimate` checks every line against those limits and prices the run the way
+`create` would submit it, job-level flags included, as a range: Adaptive
+Stealth Mode tasks count 1 at the low end and 25 at the upper bound. It exits 1
+on any invalid line.
 
 ## Extract in Batch
 
@@ -39,42 +58,140 @@ correctly. The result carries two keys: `html` (the
 raw page) and `parsed` (the structured data). Validate on a single URL with
 `zenrows extract <url>` before running the full batch.
 
-## Cloud (needs a key)
+## Create, wait, read
+
 ```
-zenrows batch create jobs.jsonl [--proxy-country cc] [--wait]
-zenrows batch create jobs.jsonl --manual [--js-render] [--premium-proxy]
-zenrows batch status <id>             # run status + stats
-zenrows batch results <id> [--status successful|failed|all] [--out results.jsonl]
-zenrows batch wait <id> [--timeout ms]
-zenrows batch cancel <id>             # stop an in-flight run
-zenrows batch retry-failed <id>       # rerun only the failed tasks (new run)
+zenrows batch create jobs.jsonl --follow --json
+zenrows batch create jobs.jsonl --manual --premium-proxy --proxy-country us --output markdown
+zenrows batch create jobs.jsonl --name nightly --external-id run-42 --metadata team=growth
+zenrows batch results <id> --out results.jsonl   # every row, as JSONL
+zenrows batch results <id> --download out/       # each result body + _manifest.jsonl
+zenrows batch content <id> <task_id>             # one task's body
 ```
 
-Every task runs in Adaptive Stealth Mode (`mode=auto`) by default, like `fetch`
-and `extract`: Zenrows enables JS rendering and premium proxies only on the pages
-that need them, and bills only the configuration that succeeds. A task that sets
-`js_render`, `premium_proxy` or `mode` itself keeps its own settings, and
-`--js-render` / `--premium-proxy` force that configuration on every task instead.
-`--manual` turns Adaptive Stealth Mode off. `batch estimate` counts auto tasks at
-the 25x upper bound, so it's the most the run can cost, not the expected cost.
-
-`create` validates the spec first, then submits. Outside Adaptive Stealth Mode,
-`--proxy-country` needs `--premium-proxy` and is rejected before any request.
-Job-level flags apply to every task; per-task keys in the JSONL override them.
+- Every GET task runs in Adaptive Stealth Mode (`mode=auto`) by default, like
+  `fetch` and `extract`: Zenrows enables JS rendering and premium proxies only
+  on the pages that need them, and bills only the configuration that
+  succeeds. A task that turns on `js_render` or `premium_proxy`, or sets
+  `mode`, itself keeps its own settings; `--js-render` / `--premium-proxy` force that
+  configuration on every task; `--manual` turns Adaptive Stealth Mode off.
+  Outside Adaptive Stealth Mode, `--proxy-country` needs `--premium-proxy`.
+- Job-level flags apply to every task; per-task keys override them.
+- `--follow` (alias `--wait`) waits until the run ends, the run is paused, or
+  `--timeout` runs out (default 10 minutes; `30s`, `5m`, `1h`; a bare number
+  is milliseconds). A wait that runs out exits 0 and prints the run with a
+  `note` and `next: zenrows batch wait <id>`; the job keeps running. Ctrl-C
+  stops the wait, not the job, and exits 130.
+- A submit of 10,000 tasks or more answers 202 with `ingest_status: pending`
+  while the API still receives the tasks; results stay partial until `done`.
+- `create` and `rerun` send an `Idempotency-Key` (generated, or
+  `--idempotency-key`) and retry network errors and 502/503/504 under it.
+  When the outcome stays unknown, the error carries `idempotency_key` and the
+  exact command that repeats the request safely (the API keeps a key 24 hours
+  and answers a repeat with the original job). Never repeat with a new key.
+- Results are kept 14 days by default; after that a row reads `expired`, has no
+  `result_url`, and `content` answers `BATCH_RESULT_EXPIRED`. `--download`
+  skips expired rows and counts them. A row's `result_url` is valid 2 hours:
+  list the results again for a fresh one.
+- `results --status successful|failed|processing|all`, `--run <run_id>` for an
+  earlier run. On a run still in progress the output has `partial: true`.
+
+With `--json`, `create`, `status`, `wait`, `cancel`, `pause`, `resume` and
+`rerun` print `{ok, job_id, run_id, status, stats, job}` (plus `jobId`, kept
+for older scripts), with `paused`, `ingest_status`, `accepted_tasks`,
+`rerun_of`, `retried_tasks`, `inherited_tasks`, `failure_reason` and
+`failure_detail` when the API sends them, and `note` when a wait ends early.
+
+## While it runs, and after
 
-A run ends in one of `completed`, `failed`, `stopped`, or `deleted`. `status`,
-`wait`, and `create --wait` exit non-zero (`ok: false` under `--json`) only for
-`failed`, and print the run's `failure_reason` and `failure_detail`. When
-`failure_reason` is `api_key_cap_reached`, the key hit its own credit cap
-(`KEY_CREDIT_CAP_REACHED`): the account still has credits, other keys keep
-working, and the tasks not yet run stay pending. Wait for the cap to reset
-(`zenrows usage`) or raise the cap; do not resubmit the job with the same key.
-
-Deferred (documented, not yet in the CLI): CSV upload, open/queue jobs,
-scheduled jobs, webhooks/HMAC, and ZIP export.
+```
+zenrows batch list [--status open|closed] [--type regular|scheduled]
+zenrows batch status <id>
+zenrows batch wait <id> --timeout 30m
+zenrows batch pause <id>              # no new task is picked up
+zenrows batch resume <id>
+zenrows batch cancel <id>             # stop the run; an ended run answers as it ended
+zenrows batch retry-failed <id>       # rerun the failed tasks
+zenrows batch retry-failed <id> --include-pending   # also the unfinished ones
+zenrows batch rerun <id>              # failed + pending tasks; successful ones carry over
+zenrows batch rerun <id> --status failed,pending,processing
+zenrows batch rerun <id> --all        # every task (not with --status)
+zenrows batch delete <id> --yes       # job, runs and stored results; frees its slot
+```
 
-Validate locally and fan out with `zenrows fetch` per URL on a small sample
-before running a full batch.
+A run ends `completed`, `stopped`, `failed` or `deleted`. A paused run is not
+ended: `status` shows `paused: true`, and `wait` returns at once with a note
+to `zenrows batch resume <id>`.
+
+`status`, `wait`, `create --follow` and `rerun --follow` exit 1 (`ok: false`)
+only for `failed`. A run fails when an account problem stops it, and the
+error carries the same code and advice as the matching refused request, with
+the run's `failure_reason` as `server_code` and the `job_id`:
+
+- `insufficient_credits` → `POLICY_MAX_CREDITS_EXCEEDED`: add credits or claim
+  the account (`zenrows usage`).
+- `subscription_inactive` → `AUTH_INVALID`: fix the subscription in Billing.
+- `api_key_cap_reached` → `KEY_CREDIT_CAP_REACHED`: the account still has
+  credits and other keys keep working; wait for the cap to reset or raise it.
+- anything else → `BATCH_FAILED`.
+
+After any of these, and after a `cancel`, the tasks not yet run stay `pending`
+(some can stay `processing`): fix the cause, then run them with
+`zenrows batch rerun <id> --status failed,pending,processing`. Do not resubmit
+the job. `rerun` with no flag reruns `failed,pending`, never the successful
+tasks. A partial rerun (the default, or `--status`) works for 10 days after the
+run started; after that only a full rerun (`zenrows batch rerun <id> --all`)
+does. Every rerun is
+a new run that answers 202 with `ingest_status: pending`.
+
+## Limits
+
+- 100,000 tasks per job; 3 active jobs per account, shared with Crawl. A
+  closed job frees its slot when its run ends; an open or scheduled job only
+  when deleted.
+- The CLI's local policy caps a run at 10,000 URLs and 250,000 estimated
+  credits by default (an Adaptive Stealth Mode task counts 25), so a default
+  run of up to 10,000 URLs passes in any mode. For more, raise both:
+  `zenrows policy set max_pages_per_run 100000` and
+  `zenrows policy set max_credits_per_run 2500000`. A workspace created by an
+  earlier CLI version keeps the caps in its `policy.json` (1,000 and 10,000);
+  `POLICY_LIMIT_EXCEEDED` prints the exact command to raise them.
+- Results 14 days; `result_url` 2 hours; partial reruns 10 days; a task that
+  fails transiently is retried automatically up to 3 times.
+
+Schedules, webhooks, ZIP exports, CSV inputs, open jobs (add tasks, close),
+the runs list and task history are available through the Batch API and the
+SDKs (Python, Node.js, Go), not this CLI.
+
+## Errors
+
+Every Batch error carries the API's code as `server_code` (when it sent one)
+and `job_id` on any call about one job. Reads retry 429/502/503/504 and network
+errors up to 3 times; `cancel`, `pause` and `resume` retry 502/503/504; `create`
+and `rerun` retry the same under their Idempotency-Key. The "Retry?" column is
+for errors that remain after these retries.
+
+| Code | Meaning | Retry? |
+| --- | --- | --- |
+| `BATCH_ACCESS_DENIED` | 403: Batch is not enabled for this account or key | no |
+| `AUTH_INVALID` | 401: the key was rejected; or the subscription is not active (401 `subscription_inactive`, 402 "Subscription has expired."; fix it in Billing) | no |
+| `POLICY_MAX_CREDITS_EXCEEDED` | 402 `payment_required`: the account is out of credits | after credits renew or are added |
+| `KEY_CREDIT_CAP_REACHED` | 402 `api_key_cap_reached`: this key reached its credit cap | after the cap resets or is raised |
+| `BATCH_QUOTA_EXCEEDED` | 429 `quota_exceeded`: 3 active jobs (shared with Crawl), 5 scheduled jobs, or an open job's 100,000 tasks | after a slot frees (`zenrows batch list`) |
+| `BATCH_NOT_FOUND` | 404: wrong id, deleted, or another account's job | no |
+| `BATCH_RUN_IN_PROGRESS` | 409 `run_not_terminal`: the run is still going (or still receiving tasks) | after `wait` or `cancel` |
+| `BATCH_RUN_ENDED` | 409 `run_not_stoppable`: the run already ended (pause/resume) | no; `rerun` starts a new run |
+| `BATCH_NO_MATCHING_TASKS` | 409 `no_matching_tasks`: no task has the statuses asked for | with `--status failed,pending,processing`, or `rerun --all` |
+| `BATCH_RERUN_WINDOW_EXPIRED` | 409 `partial_rerun_window_expired`: a partial rerun is over 10 days late | with `rerun --all` |
+| `BATCH_REQUEST_IN_FLIGHT` | 409 `idempotency_key_conflict`: the key is in use or was used for another body | same command, same key, a moment later |
+| `BATCH_CONFLICT` | any other 409 (job state, key no longer valid, concurrent change) | read the cause |
+| `BATCH_TASK_NOT_READY` | 409 `task_not_ready` on `content`: the task has not finished | after `wait` |
+| `BATCH_TASK_FAILED` | 422 on `content`: the task failed | after `retry-failed` |
+| `BATCH_RESULT_EXPIRED` | 410 `result_expired`: past the 14-day visibility window | after `rerun --all` |
+| `BATCH_FAILED` | the run ended `failed` for another reason; a 400 (fix the flagged tasks) or 5xx | 5xx: yes |
+| `POLICY_LIMIT_EXCEEDED` | the local policy caps (see Limits) | after raising the cap |
+| `BACKEND_UNAVAILABLE` | the API could not be reached | yes |
 
 Rules: **never batch before validating** the workflow on a single page
-([[extract]]). Mind the cost multipliers ([[cost-control]]).
+([[extract]]). Mind the cost multipliers ([[cost-control]]). Read errors with
+[[trace-debug]].

package/skills/crawl/SKILL.md

@@ -34,7 +34,7 @@ zenrows crawl results <id> --out urls.jsonl  # same, as JSONL
   a depth has no `depth` field in its output.
 - `--max-items` (kept URLs) and `--max-pages` (fetched pages) cap the crawl;
   both default to 100. `--max-pages` bounds the cost: each page is one fetch.
-  The local policy `max_pages_per_run` caps `--max-pages` (default 1000);
+  The local policy `max_pages_per_run` caps `--max-pages` (default 10000);
   above it the CLI refuses with `POLICY_LIMIT_EXCEEDED` before any request.
 - `--include-patterns` / `--exclude-patterns` are substring matches on the
   normalized URL and repeat:

package/skills/extract/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: extract
-description: Turn protected pages into structured data with Extract / Autoparse / CSS / Markdown.
+description: Turn protected pages into structured data with Extract / CSS / Markdown (Autoparse is deprecated).
 version: 0.1.0
 requires_backend_capabilities: [extract]
 ---
@@ -12,8 +12,8 @@ access + extraction**, not generic LLM parsing.
 
 ## Methods
 ```
-zenrows extract <url>                              # extract=auto (default); falls back to Autoparse if the domain is not prepared yet
-zenrows extract <url> --autoparse                  # general-purpose Autoparse (any domain)
+zenrows extract <url>                              # extract=auto (default), any prepared domain
+zenrows extract <url> --autoparse                  # deprecated general-purpose Autoparse (any domain)
 zenrows extract <url> --css '{"title":"h1","price":".price"}'   # selector map
 zenrows extract <url> --outputs emails,links       # built-in output filters → JSON
 zenrows extract <url> --output markdown            # Markdown conversion
@@ -21,14 +21,58 @@ zenrows extract <url> --validate                   # fail if not valid JSON
 ```
 
 ## When to use which
-- **Extract** (`extract=auto`, default): richest fields; works on any domain Zenrows has prepared.
-- **Autoparse**: any website; also the automatic fallback when the domain is not prepared for Extract yet.
+- **Extract** (`extract=auto`, default): richest fields; works on any prepared
+  domain (you can prepare one yourself, see below).
+- **Autoparse** (deprecated): only with `--autoparse`. Never automatic; do not
+  build new work on it.
 - **CSS**: you know the exact fields/selectors and want determinism.
 - **Markdown/plaintext**: feed clean content to an LLM yourself.
 
+## REQS007: prepare the domain, then extract again
+
+`extract=auto` works on any prepared domain, and you can prepare one yourself.
+When Extract refuses a page with `403 REQS007`, `zenrows extract` says which
+of three cases it is and prints the next step. It never retries with Autoparse.
+
+| Error code | Meaning | Next step |
+| --- | --- | --- |
+| `EXTRACT_DOMAIN_NOT_PREPARED` | The domain is not prepared yet (also the legacy `402 AUTH010`). | Prepare it (below), then extract again. |
+| `EXTRACT_PAGE_NOT_RECOGNISED` | The domain is prepared, but Extract did not recognise this page as one it can extract (a page with nothing to extract, such as a home page, or a transient failure). | Try a product, article or listing page on the same site instead. If this page should work, retry it once in a few minutes; do not retry in a loop. Otherwise use `--css`. Preparing the domain again does not change this. |
+| `EXTRACT_DOMAIN_NOT_ENABLED` | Extract is not enabled for the site. | Use `--css`, or contact Zenrows support to have the site enabled. Preparing does not change this. |
+
+Preparing a domain:
+
+```
+zenrows extract https://example.com/p/1             # 1. fails: EXTRACT_DOMAIN_NOT_PREPARED
+zenrows extract prepare example.com \
+  --url https://example.com/p/1 --url https://example.com/list --wait
+                                                    # 2. prepare it and wait until ready (a few minutes)
+zenrows extract https://example.com/p/1             # 3. extract again
+```
+
+- `--url`: 1 to 5 real pages you want to extract, one per page kind (for
+  example a product page and a listing page).
+- Without `--wait` the command returns at once; then check about every 30
+  seconds with `zenrows extract prepare status example.com` until it says
+  `ready`. `--wait` polls at the same pace.
+- The domain is the host: `shop.example.com` and `example.com` are prepared
+  separately, and a leading `www.` is dropped.
+- Preparing takes a few minutes and costs no credits. Each new domain uses one
+  slot of the account's monthly allowance; `prepare status` shows `used` /
+  `limit` / `remaining`.
+- `EXTRACT_PREPARE_OUT_OF_ALLOWANCE`: no slots left this month; use `--css`.
+  `EXTRACT_PREPARE_UNAVAILABLE`: retry in a few minutes; do not loop.
+  A `failed` status prints the next step for its failure code and says whether
+  the attempt used an allowance slot.
+- A domain that is already prepared answers `ready` at once
+  (`already_prepared`) and uses no slot: run the extract. If it then fails with
+  `EXTRACT_PAGE_NOT_RECOGNISED`, follow that row of the table, not `prepare`.
+
 ## Rules
 - Validate on a single page before scaling across many URLs.
-- If Autoparse misses fields, switch to `--css` with explicit selectors. Auto
+- Check `empty` in `--json` output: true means every field came back null or
+  empty, so the result is not data.
+- If a field is missing, switch to `--css` with explicit selectors. Auto
   mode already renders JS when the target needs it, so reach for
   `--manual --js-render` only after auto mode has returned an incomplete page:
   it costs several times a basic call ([[cost-control]]) and it makes the

package/skills/trace-debug/SKILL.md

@@ -43,8 +43,23 @@ zenrows trace export <run-id>     # JSON for sharing
   local policy credit limit was hit). Do not retry-loop; `zenrows usage` shows
   when credits renew. Top up or upgrade, or claim the account if it is an
   auto-created Free plan.
-- `BATCH_FAILED` → the batch run ended `failed`; `failure_reason` and
-  `failure_detail` in `zenrows batch status <id> --json` say why.
+- `BATCH_FAILED` → the batch run ended `failed` for a reason other than
+  credits, subscription or key cap (those get `POLICY_MAX_CREDITS_EXCEEDED`,
+  `AUTH_INVALID` or `KEY_CREDIT_CAP_REACHED`, with the run's `failure_reason`
+  as `server_code` and its `job_id`); `failure_detail` in
+  `zenrows batch status <id> --json` says why. Once fixed, run the unfinished
+  tasks with `zenrows batch rerun <id> --status failed,pending,processing`;
+  do not resubmit. Also a 400 (fix the flagged tasks) or a 5xx (retry).
+- `BATCH_QUOTA_EXCEEDED` → 429: 3 active jobs (shared with Crawl), 5 scheduled
+  jobs, or an open job's task cap. Nothing was created; free a slot (see
+  `zenrows batch list` and `zenrows crawl list`).
+- `BATCH_ACCESS_DENIED` / `BATCH_NOT_FOUND` → 403 Batch not enabled, or 404
+  wrong or deleted job id. Do not retry.
+- `BATCH_RUN_IN_PROGRESS`, `BATCH_RUN_ENDED`, `BATCH_NO_MATCHING_TASKS`,
+  `BATCH_RERUN_WINDOW_EXPIRED`, `BATCH_REQUEST_IN_FLIGHT`, `BATCH_CONFLICT`,
+  `BATCH_TASK_NOT_READY`, `BATCH_TASK_FAILED`, `BATCH_RESULT_EXPIRED` → the
+  Batch API's 409/410/422 answers; `server_code` names the case and
+  `next_action` the fix. See [[batch-jobs]].
 - `CRAWL_NOT_ENABLED` → the API answered 403 `REQS008`: Crawl is not
   enabled for this account. Do not retry; ask Zenrows support for access.
 - `CRAWL_INSUFFICIENT_CREDITS` / `CRAWL_KEY_CAP_REACHED` → Crawl's 402: the

package/skills/zenrows/SKILL.md

@@ -34,7 +34,7 @@ If the user has a known URL and wants page content:
   → Use Protected Fetch.            (zenrows fetch <url>)
 
 If the user has a known URL and wants structured data:
-  → Use Extract.                    (zenrows extract <url> | --autoparse | --css)
+  → Use Extract.                    (zenrows extract <url> | --css)
 
 If the user has a start page (listing, index, section) but not the item URLs:
   → Use Crawl.                      (zenrows crawl create <url> --depth 1 --follow,
@@ -43,7 +43,8 @@ If the user has a start page (listing, index, section) but not the item URLs:
 If the user has many URLs (bulk):
   → Use Batch.                      (validate one page with fetch/extract first,
                                      then `zenrows batch create <jobs.jsonl>`)
-                                     Batch is the cheap path at scale — do NOT
+                                     Batch is the reliable path at scale, at
+                                     Fetch's prices. Do NOT
                                      fan out fetch/extract per URL for bulk work.
 
 If the user needs login, clicks, forms, sessions, or persistent state:
@@ -74,14 +75,17 @@ Run `zenrows status` for the live capability matrix. As of this toolkit:
 | Primitive | Command | Status |
 | --- | --- | --- |
 | Protected Fetch | `zenrows fetch` | **available**: `GET https://api.zenrows.com/v1/` |
-| Extract (`extract=auto`, Autoparse, CSS, Markdown) | `zenrows extract` | **available**: `GET https://api.zenrows.com/v1/?extract=auto` |
+| Extract (`extract=auto`, CSS, Markdown; Autoparse deprecated) | `zenrows extract` | **available**: `GET https://api.zenrows.com/v1/?extract=auto` |
 | Batch | `zenrows batch` | **available**: `POST https://async.api.zenrows.com/v1/jobs` |
 | Crawl (new) | `zenrows crawl` | **available**: `POST https://api.zenrows.com/v1/crawls` |
 | Browser | `zenrows browser` | **available**: `POST https://mcp.zenrows.com/browser/sessions` |
 | MCP | `zenrows mcp` | **available**: `https://mcp.zenrows.com/mcp` |
 
-- Extract: `extract=auto` falls back to Autoparse on a domain not prepared yet.
-- Batch: the cheapest path at scale (create, status, results). Estimate specs locally with no key.
+- Extract: `extract=auto` works on any prepared domain. On one not prepared yet
+  (`EXTRACT_DOMAIN_NOT_PREPARED`), run `zenrows extract prepare <domain> --url
+  <page> --wait`, then extract again. `EXTRACT_PAGE_NOT_RECOGNISED` means the
+  domain is prepared but not this page: do not prepare again; see [[extract]].
+- Batch: the reliable path at scale, at Fetch's prices (create, list, status, wait, results, content, pause, resume, rerun, delete). Estimate specs locally with no key.
 - Crawl: collects the URLs behind one start page, optionally with each page's HTML.
 - Browser: escalation-only, for pages Fetch can't handle; same backend as the MCP `browser_*` tools.
 

package/templates/batch-jsonl-pipeline/README.md

@@ -3,8 +3,9 @@
 Scaffold for a high-scale workload expressed as a JSONL job spec, run on the
 Zenrows **Batch**. One JSON object per line, each with a `url` (plus
 optional per-line overrides like `js_render`, `premium_proxy`, `proxy_country`,
-`mode`, `autoparse`, `extract`, and an `external_id` echoed back on each
-result).
+`mode`, `autoparse`, `extract`, an `external_id` echoed back on each result,
+string `metadata`, and `"method": "POST"` with a `body`). `batch estimate`
+checks every line against the API's limits.
 
 Set `"extract": "auto"` on a line to run that URL through Extract — structured
 data instead of raw HTML, at the same credit cost as a regular task. Extract
@@ -18,17 +19,21 @@ zenrows batch estimate jobs.example.jsonl    # validate the spec + estimate cred
 
 ## Cloud (needs a key)
 
-The cloud subcommands run with any active API key and spend the same credits
+The cloud subcommands run with any active API key and cost the same credits
 as Fetch.
 
 ```bash
-zenrows batch create jobs.example.jsonl --wait          # submit + wait for the run to finish
+zenrows batch create jobs.example.jsonl --follow        # submit + wait for the run to finish
 zenrows batch status <job-id>                           # run status + progress stats
 zenrows batch results <job-id> --out results.jsonl      # collect results (paginated)
-zenrows batch retry-failed <job-id>                     # rerun only the failed tasks
+zenrows batch retry-failed <job-id> --include-pending   # rerun what did not succeed
 zenrows batch cancel <job-id>                           # stop an in-flight run
 ```
 
+The local policy caps a run at 10,000 URLs and 250,000 estimated credits by
+default; raise them with `zenrows policy set max_pages_per_run <n>` and
+`zenrows policy set max_credits_per_run <n>`.
+
 Tasks use Adaptive Stealth Mode (`mode=auto`) by default. Job-level scrape
 options come from flags (`--js-render`, `--premium-proxy`, `--proxy-country`,
 `--output`, or `--manual` to turn Adaptive Stealth Mode off); per-line keys

package/templates/extract-pricing-pages/README.md

@@ -3,13 +3,11 @@
 Extract structured pricing fields from product pages with Zenrows Extract.
 
 ```bash
-# Default: extract=auto on an enabled domain (site-tailored JSON)
+# Default: extract=auto on a prepared domain (site-tailored JSON).
+# Not prepared yet (EXTRACT_DOMAIN_NOT_PREPARED)? zenrows extract prepare <domain> --url <product-url> --wait
 zenrows extract https://www.owler.com/company/meltwater
 
-# Autoparse — general-purpose JSON on any domain
-zenrows extract https://www.scrapingcourse.com/ecommerce/ --autoparse
-
-# Or a precise CSS selector map
+# A precise CSS selector map, on any domain
 zenrows extract <product-url> --css '{"title":"h1","price":".price"}' --validate
 ```
 

package/workflows/competitor-intelligence/WORKFLOW.md

@@ -9,7 +9,7 @@ status: available
 
 A higher-level process built on Zenrows **Fetch and Extract**: retrieve
 protected pages with `zenrows fetch`, then turn them into structured data with
-`zenrows extract` (the same API's Autoparse / CSS / Markdown — not a separate
+`zenrows extract` (the same API's `extract=auto` / CSS / Markdown — not a separate
 product). Scheduling and scaling use the tools you already have (cron/CI, a
 simple per-URL loop).
 
@@ -20,7 +20,7 @@ simple per-URL loop).
    ```
    zenrows fetch https://competitor.example/products
    ```
-2. **Validate extraction** on one page (extract=auto falls back to autoparse on a domain not prepared yet).
+2. **Validate extraction** on one page (on a domain not prepared yet, `EXTRACT_DOMAIN_NOT_PREPARED`, run `zenrows extract prepare <domain> --url <page> --wait` first).
    ```
    zenrows extract https://competitor.example/p/123 --validate
    ```