@trawlme/cli 3.8.2 → 3.9.1

package/README.md

@@ -36,7 +36,7 @@ Four methods:
 3. **Token flag** — `trawl login --token <jwt>` (CI/CD, direct JWT)
 4. **API key** — `TRAWL_API_KEY=trawl_xxx trawl list` (scoped, revocable one-by-one — the recommended credential for an agent driving this CLI; see [docs/agent-quickstart.md](docs/agent-quickstart.md))
 
-A `TRAWL_API_KEY` is a `trawl_*`-prefixed credential (create/revoke one in the Trawl dashboard) sent as `Authorization: Bearer` instead of the session `Cookie: TOKEN=` a JWT uses — `trawl` picks the right one automatically based on the credential's own shape, never a flag. `TRAWL_API_KEY` wins over `TRAWL_TOKEN` when both happen to be set. It is scoped server-side — in practice to a set of scraps, since the dashboard's key-create form sets only the scrap allow-list; narrowing which *actions* a key may perform needs an explicit `scopes` array at creation over the API, and a key without one has every action granted. It works on most of the Core tier — `create`/`list`/`get`/`data`/`history`/`run-info`/`run`/`trigger`/`ping`, including the `--watch` **polling flag** on `run`/`trigger` — plus, outside Core, `scraps account status`/`scraps doctor`/`scraps autofix` (all three only ever read routes trawl_node opened to keys). It does **not** work on `whoami`, `scraps update`/`delete`, `scraps account set`/`delete`/`clear-session`/`session set`, `scraps banner`, `scraps snapshot` (the scrap lookup it starts from is dual-auth, but the `html-snapshot` route it downloads from isn't), or the standalone SSE **command** `scraps watch` (do not conflate the two: `--watch` is a flag on `run`/`trigger` and works under a key; `scraps watch` is a separate command and is JWT-only) — those stay JWT-only and fail with a `kind:"auth"` envelope (exit `3`) pointing at `trawl login` under a key. This list mirrors trawl_node's route wiring as of this writing, not a frozen guarantee — for anything not named here, trust the real `--json` envelope over this paragraph. Never send both a key and a JWT on the same request — the CLI only ever attaches one.
+A `TRAWL_API_KEY` is a `trawl_*`-prefixed credential (create/revoke one in the Trawl dashboard) sent as `Authorization: Bearer` instead of the session `Cookie: TOKEN=` a JWT uses — `trawl` picks the right one automatically based on the credential's own shape, never a flag. `TRAWL_API_KEY` wins over `TRAWL_TOKEN` when both happen to be set. It is scoped server-side — in practice to a set of scraps, since the dashboard's key-create form sets only the scrap allow-list; narrowing which *actions* a key may perform needs an explicit `scopes` array at creation over the API, and a key without one has every action granted. It works on most of the Core tier — `create`/`list`/`get`/`data`/`history`/`run-info`/`run`/`trigger`/`ping`, including the `--watch` **polling flag** on `run`/`trigger` — plus, outside Core, `scraps account status`/`scraps doctor`/`scraps autofix` (all three only ever read routes trawl_node opened to keys). It does **not** work on `whoami`, `scraps update`/`delete`, `scraps account set`/`delete`/`clear-session`/`session set`, `scraps banner`, `scraps snapshot` (the scrap lookup it starts from is dual-auth, but the `html-snapshot` route it downloads from isn't), or the standalone SSE **command** `scraps watch` (do not conflate the two: `--watch` is a flag on `run`/`trigger` and works under a key; `scraps watch` is a separate command and is JWT-only) — those stay JWT-only and fail with a `kind:"auth"` envelope (exit `3`) pointing at `trawl login` under a key. This list mirrors trawl_node's route wiring as of this writing, not a frozen guarantee — for anything not named here, trust the real `--json` envelope over this paragraph. Never send both a key and a JWT on the same request — the CLI only ever attaches one. `list --unhealthy` (and the plain health badge on `list`/`get`) rides the same `GET /api/scraps`/`GET /api/scraps/:id` routes as the rest of the group — no separate auth path — so it works under a key exactly like plain `list`/`get`, and a scrap-scoped key still only ever sees its own allow-listed scraps through the filter.
 
 Custom API URL: `trawl login --url https://self-hosted.example.com`
 
@@ -51,7 +51,7 @@ All commands accept a global `--debug` flag to show full error stack traces on f
 ```
 trawl create <url> --prompt <goal> [--no-autofix] [--json]   # url also accepted as --url <url> (#116)
                                                               Create a persistent, self-healing scrap from a URL + a goal (AI-generated)
-trawl list|ls [--json] [--status <success|failure|never|running|regression>] [--limit <n>] [--page <n>]
+trawl list|ls [--json] [--status <success|failure|never|running|regression>] [--unhealthy] [--limit <n>] [--page <n>]
 trawl get <id> [--json]                        Get scrap details
 trawl run <id> [--watch] [--json]              Run a scrap (blocks until finished)
 trawl trigger <id> [--watch] [--wait] [--json] Launch a scrap as a background worker (returns immediately)
@@ -76,6 +76,7 @@ trawl spec [--json]                            Print a versioned, machine-readab
 - `history` lists past runs (newest first); `run-info <hid>` shows details of a single run from that history.
 - `data` returns the last persisted run payload (no execute quota); `--fresh` runs the scrap live instead (consumes execute quota); `--errors` shows the last run's error detail (`--json` on a never-run scrap returns `{"status":"no_runs"}`, exit 0, matching `scraps doctor --json`). `[]` on stdout means a genuine zero-item successful run — a scrap that has never run, whose last run failed, or whose payload aged out of retention returns a `--json` error envelope (exit 4/1/4 respectively) instead. Two more honest states: a run still **in flight** (`status: null` server-side) returns a `kind:"in_progress"` error envelope (exit 1, "retry shortly" — never suggests `--fresh`, which would just 429 against the run already holding the lock); a run whose item count **regressed** vs baseline (`statusDetail: "regression"`) still returns the real, non-empty items on stdout (exit 0) plus a stderr warning pointing at `scraps doctor <id>` — the data itself is genuine even though the run is flagged.
 - `get` (and anything reading through it, like `data`'s default path) embeds only the newest 100 history rows on the returned scrap object — `run-info` and `scraps doctor` fetch a single run directly and are unaffected by that cap. `list`/`get` show a distinct amber `▼` "regression" badge, never the red `✗` a genuine failure gets (matches `scraps doctor`'s own badge).
+- `list`/`get` also show a **health** badge next to the status badge. By default it reads the scrap's own `lastCronOutcome` field (always present, no extra cost): `⚠ cron paused` when the last scheduled tick was skipped for being unhealthy, `—` otherwise — a breadcrumb of the last tick, not a live read (never set on a no-cron scrap, can lag by one tick). `list --unhealthy` asks the API to filter to scraps whose *current* consecutive-failure streak is 3+ (trawl_node's own threshold) and annotates each with the exact `consecutiveFailedRuns`/`unhealthySince` — shown instead of the badge whenever present, and never re-derived client-side from history. `--json` carries whichever fields the request produced: `lastCronOutcome` always, `consecutiveFailedRuns`/`unhealthySince` only when `--unhealthy` was passed. **Old-server safety:** a server predating trawl_node#1952 silently ignores the unknown `--unhealthy` query param and returns everything, unfiltered — the CLI detects that (no returned item carries `consecutiveFailedRuns`) and prints a stderr warning instead of presenting the full list as "your unhealthy scraps".
 - **Long-running calls (`create`, `run`, `data --fresh`, `trigger --wait`):** these hit server-side paths that can legitimately take 30–250s+ (AI generation + scrap creation + a first run for `create`; proxy tier escalation + AI-fix retries for the other three) — the CLI arms a 300s timeout for exactly these four call sites instead of the generic 30s default. `TRAWL_TIMEOUT` (see below) still overrides ALL requests, including these — set it if you need a tighter or looser ceiling than 300s, but note a global override that tight also clamps `create`.
 - **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `run --watch` and `trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears. The watched run's outcome drives the exit code too, in BOTH human and `--json` mode: a genuinely failed terminal run, a poll timeout, or a persistently unreachable API all exit non-zero — a clean successful run is the only exit `0`. A run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>` (human mode) — see the `--json` shape below. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
 - `spec --json` prints the CLI's own command tree — `{specVersion, cliVersion, commands[], exitCodes, errorKinds, kindExitCodes}` — DERIVED at runtime by walking the live commander tree (never a hand-maintained file, which would silently drift from reality). Each entry in `commands[]` carries its full path (e.g. `"scraps account session set"`), description, `hidden` (the legacy `scraps <verb>` aliases above), `leaf` (false for a pure namespace/group node like `scraps`/`skills`/`telemetry` — invoking one directly is a guaranteed-failing tool, not a real command), `aliases`, `arguments`, and `options`; `exitCodes`/`errorKinds`/`kindExitCodes` are read from the exact same source `classifyError` uses (see [Exit codes](#exit-codes)) — never a second copy. `kindExitCodes` is the inverse of `exitCodes`: `kind -> exitCode`, since exit code `1` alone is a shared bucket (`api`/`refused`/`unknown`/`in_progress`/`run_failed`/`upgrade_failed`) that `exitCodes`' flat label can't disambiguate. Without `--json`, `spec` prints one short human line (version + visible command count) pointing at `--json`.
@@ -96,6 +97,8 @@ trawl scraps delete <id> [--force] [--json]         Alias: rm
 
 - `-u/--url` is **the site the scrap targets**, not necessarily the exact URL fetched at run time — a scrap whose real target is computed inside its `-r/--request` script (a templated search query, an interpolated item id, e.g. `https://www.ebay.com/sch/i.html?_nkw=${query}`) should still supply a base/site URL. Every domain-level policy (proxy tier ceiling, provider routing, per-domain memoize) keys off this field, so a blank value silently disables all of it. `create` requires it — validated client-side (must parse as an `http`/`https` URL) so a missing or malformed value fails fast with a readable usage error (exit 2) instead of a raw server 400/422; `update` keeps it optional (omitting it is still a no-op, matching the API) but a value that **is** passed is validated the same way.
 - `--tier` forces a proxy tier; `--force-tier` raises the proxy-tier ceiling past the auto-cap (history-gated: may be refused or cost more). `create --json`/`update --json` print the full scrap object (including the `_tierOverride` outcome) on stdout; a refused tier override exits 1 with a standard `--json` error envelope (`kind:"refused"` — distinct from `"unknown"`, so a script can branch on "the server said no"). When a tier was requested but the server's response carries no `_tierOverride` at all (an older server that can't confirm what actually got applied), a stderr warning is printed either way, and under `--json` the emitted object also carries `"_tierUnconfirmed": true` — the machine-readable counterpart to that warning, since a `--json` caller has no reliable reason to read stderr.
+
+`list --unhealthy` has the same shape of guard: against a server that predates the filter, the command warns on stderr and, under `--json`, wraps the payload as `{ "scraps": [...], "_healthFilterUnconfirmed": true }` instead of emitting a bare array — so a script can tell an unfiltered list from a genuinely-unhealthy one. A confirming server returns the bare array unchanged.
 - `scraps doctor` diagnoses the last run (error, failed selector, block status, page state, autofix outcome); `--autofix` includes the full autofix diff/dry-run/knowledge. A run that is still in flight (`status: null`, server-side) shows a `running` badge, never `failed`; a run whose item count regressed vs baseline (`statusDetail: "regression"`) shows its own amber `regression` badge, never `failed` either.
 - `scraps autofix` shows the last auto-fix attempt on its own (decision, diff, dry-run, knowledge). `--json` on a scrap that has **never run** returns `{"status":"no_runs"}` (exit 0) — distinct from `null`, which means a run exists but had no auto-fix attempt.
 - `scraps snapshot --error` fetches the error-path snapshot instead of the normal one; `-o <file>` writes to a file instead of stdout. On a scrap that has **never run**: `--json` returns `{"status":"no_runs"}` (exit 0), matching `doctor`/`autofix`; `-o <file>` (without `--json`) exits 4 (`not_found`) instead of silently exiting 0 with nothing written — a script checking the exit code alone must be able to tell "no file was produced" from success. `--json` takes priority when both are passed.

package/dist/commands/doctor.d.ts

@@ -25,19 +25,31 @@ export interface Run {
         selectors?: Record<string, number>;
     } | null;
     blocked?: boolean;
+    block?: {
+        kind?: string | null;
+    } | null;
+    /** @deprecated trawl_node#1950 renamed this to `block.kind`. Kept as a read
+     * fallback (see detectWallVendor) for the window where this CLI is
+     * published ahead of the trawl_node prod tag — a prod backend served from
+     * the pre-#1950 tag still returns this flat field, not `block.kind`. Drop
+     * once prod is confirmed on a tag containing #1950. */
     blockType?: string | null;
     proxyTier?: string | null;
-    regressionDetected?: boolean;
     baselineLength?: number | null;
     fixVersionId?: string | null;
     createdAt?: string;
     time?: number | null;
     triggeredBy?: string | null;
+    /** trawl_node#1975 — freeform failure classification (`'auth'` = login-wall
+     * empty run, cookies are the fix). Not a TS union — trawl_node's own set
+     * is additive/open, so this CLI must stay read-safe against a future
+     * value it doesn't know about yet, same posture as `block.kind` above. */
+    failureKind?: string | null;
 }
 /**
- * Resolve a known anti-bot vendor (or auth) name from a worker `blockType`
- * string. Returns null when the run succeeded, when there is no `blockType`
- * signal at all, or when `blockType` names something other than a known
+ * Resolve a known anti-bot vendor name from a worker `block.kind`
+ * string. Returns null when the run succeeded, when there is no `block.kind`
+ * signal at all, or when `block.kind` names something other than a known
  * vendor (e.g. `proxy-domain-gate`, `rate_limited_per_host`) — those stay on
  * the genuine-error path since we can't honestly attribute them to a specific
  * "no reliable bypass" wall.
@@ -45,12 +57,18 @@ export interface Run {
  * `blocked` is deliberately NOT consulted here (#177): it is a mid-run,
  * attempt-level signal the worker only stamps on one envelope shape, so it
  * reads `false` on the great majority of genuinely walled runs (the early-block
- * throw path sets `blockType` but never `blocked` — 63 of 64 walled runs
+ * throw path sets `block.kind` but never `blocked` — 63 of 64 walled runs
  * measured on prod). `status === true` wins instead: a run that ultimately
- * returned data was not walled, even if an earlier tier's `blockType` stamp
+ * returned data was not walled, even if an earlier tier's `block.kind` stamp
  * survived on the row.
+ *
+ * trawl_node#1950 renamed the flat `blockType` field to nested `block.kind`.
+ * Read `block?.kind` first, falling back to the deprecated flat `blockType` —
+ * this CLI can be published (and talk to prod) before the trawl_node prod tag
+ * containing #1950 is cut (see the `Run.blockType` doc comment), so a prod
+ * response can still be the pre-#1950 flat shape for a while.
  */
-export declare function detectWallVendor(run: Pick<Run, 'status' | 'statusDetail' | 'blockType'>): string | null;
+export declare function detectWallVendor(run: Pick<Run, 'status' | 'statusDetail' | 'block' | 'blockType'>): string | null;
 /**
  * Autofix activity metadata — from the persisted ai_fix_end activity.
  * aiUsage (cost) is stripped server-side; all diagnostics are kept.

package/dist/commands/doctor.js

@@ -2,23 +2,25 @@ import { api } from '../lib/api.js';
 import chalk from 'chalk';
 import { formatDate } from '../lib/format.js';
 /**
- * Known anti-bot vendors (+ auth) that the worker's `blockType` field may name.
- * Ordered by first-match; `blockType` is a freeform worker string, not an enum
- * (see the `Run.blockType` doc comment), so this is a best-effort substring
- * match against the real field — never an invented/mocked value.
+ * Known anti-bot vendors that the worker's `block.kind` field may name
+ * (ex-flat `blockType`, trawl_node#1950). Ordered by first-match;
+ * `block.kind` is a freeform worker string, not an enum (see the `Run.block`
+ * doc comment), so this is a best-effort substring match against the real
+ * field — never an invented/mocked value.
  *
  * `datadome`/`perimeterx`/`akamai`/`cloudflare` are literal substrings the
- * current worker emits (detect.js). `kasada` and `auth` are forward-compatible:
- * the issue (#62 / wall-registry WS-7) names them as genuine walls, but the
- * current worker vocabulary does not yet emit them — these branches light up
- * automatically if/when a future worker classifier does, without a CLI change.
+ * current worker emits (detect.js). `kasada` is forward-compatible: the issue
+ * (#62 / wall-registry WS-7) names it as a genuine wall, but the current
+ * worker vocabulary does not yet emit it — this branch lights up
+ * automatically if/when a future worker classifier does, without a CLI
+ * change.
  *
- * The `auth` pattern is delimiter-anchored (`^`/`$`/`-_:`) rather than a bare
- * substring so it matches worker-style tokens (`auth`, `auth-wall`, `auth_wall`,
- * `login:auth`) without false-matching `oauth`/`authorization`/`author`. It uses
- * `[-_:]` (not `\b`) because `_` is a JS `\w` char, so `\bauth\b` would miss the
- * underscore-delimited `auth_wall` form the worker's `rate_limited_per_host`-style
- * naming favors.
+ * trawl_cli#182 — an `auth` entry used to live here, rendering a login wall
+ * as "walled: auth — no reliable bypass". Removed: unlike a real anti-bot
+ * vendor, a login wall's own reliable bypass is the user's session cookies,
+ * so that verdict was wrong, not just early. `failureKind==='auth'`
+ * (trawl_node#1975) is the honest signal for a login wall — see the
+ * dedicated branch in `formatDoctor` below.
  */
 const WALL_VENDOR_PATTERNS = [
     [/datadome/i, 'DataDome'],
@@ -26,12 +28,11 @@ const WALL_VENDOR_PATTERNS = [
     [/perimeterx/i, 'PerimeterX'],
     [/akamai/i, 'Akamai'],
     [/cloudflare/i, 'Cloudflare'],
-    [/(?:^|[-_:])auth(?:$|[-_:])/i, 'auth'],
 ];
 /**
- * Resolve a known anti-bot vendor (or auth) name from a worker `blockType`
- * string. Returns null when the run succeeded, when there is no `blockType`
- * signal at all, or when `blockType` names something other than a known
+ * Resolve a known anti-bot vendor name from a worker `block.kind`
+ * string. Returns null when the run succeeded, when there is no `block.kind`
+ * signal at all, or when `block.kind` names something other than a known
  * vendor (e.g. `proxy-domain-gate`, `rate_limited_per_host`) — those stay on
  * the genuine-error path since we can't honestly attribute them to a specific
  * "no reliable bypass" wall.
@@ -39,24 +40,31 @@ const WALL_VENDOR_PATTERNS = [
  * `blocked` is deliberately NOT consulted here (#177): it is a mid-run,
  * attempt-level signal the worker only stamps on one envelope shape, so it
  * reads `false` on the great majority of genuinely walled runs (the early-block
- * throw path sets `blockType` but never `blocked` — 63 of 64 walled runs
+ * throw path sets `block.kind` but never `blocked` — 63 of 64 walled runs
  * measured on prod). `status === true` wins instead: a run that ultimately
- * returned data was not walled, even if an earlier tier's `blockType` stamp
+ * returned data was not walled, even if an earlier tier's `block.kind` stamp
  * survived on the row.
+ *
+ * trawl_node#1950 renamed the flat `blockType` field to nested `block.kind`.
+ * Read `block?.kind` first, falling back to the deprecated flat `blockType` —
+ * this CLI can be published (and talk to prod) before the trawl_node prod tag
+ * containing #1950 is cut (see the `Run.blockType` doc comment), so a prod
+ * response can still be the pre-#1950 flat shape for a while.
  */
 export function detectWallVendor(run) {
     // A run that returned data was not walled. `status === true` is not the whole
     // predicate: trawl_node flips a degraded-but-non-empty run to
     // `status:false, statusDetail:'regression'` (scraps.service.js, #1112) via a
-    // patch that never clears `blockType` — so keying on `status` alone would
+    // patch that never clears `block.kind` — so keying on `status` alone would
     // print "no reliable bypass" next to "Regression: length N vs baseline M",
     // which is the same contradiction this guard exists to remove.
     if (run.status === true || run.statusDetail === 'regression')
         return null;
-    if (!run.blockType)
+    const kind = run.block?.kind ?? run.blockType ?? null;
+    if (!kind)
         return null; // no signal at all
     for (const [pattern, label] of WALL_VENDOR_PATTERNS) {
-        if (pattern.test(run.blockType))
+        if (pattern.test(kind))
             return label;
     }
     return null;
@@ -70,8 +78,8 @@ const TIER_LABELS = {
 };
 const RUN_ALLOWLIST = [
     '_id', 'status', 'statusDetail', 'length', 'errorMessage', 'errorSnapshot',
-    'emptyContext', 'blocked', 'blockType', 'proxyTier', 'regressionDetected', 'baselineLength',
-    'fixVersionId', 'createdAt', 'time', 'triggeredBy',
+    'emptyContext', 'blocked', 'block', 'blockType', 'proxyTier', 'baselineLength',
+    'fixVersionId', 'createdAt', 'time', 'triggeredBy', 'failureKind',
 ];
 const FIX_ALLOWLIST = [
     'outcome', 'classification', 'reason', 'fixDiff', 'dryRunResults',
@@ -124,14 +132,81 @@ export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
     lines.push(`${chalk.bold(scrapTitle)}  ${badge}${run.statusDetail ? ` (${run.statusDetail})` : ''}`);
     lines.push(chalk.dim(`  Run ID:  ${run._id}`));
     // Error message — an honest accept-wall string for known-walled scraps
-    // (DataDome/Kasada/PerimeterX/Akamai/auth terminal verdict from the worker),
+    // (DataDome/Kasada/PerimeterX/Akamai terminal verdict from the worker),
     // otherwise the real error (genuine transient failure).
+    //
+    // trawl_cli#182 — the wall-vendor verdict (this CLI's own
+    // WALL_VENDOR_PATTERNS matched against block.kind/blockType) and the
+    // login-wall hint (node's failureKind==='auth', trawl_node#1975) are two
+    // INDEPENDENT server-derived signals with no shared contract: node's block
+    // classifier (modules/historys/helpers/failureKind.js BLOCK_TYPE_PATTERN)
+    // and this CLI's vendor table live in different repos and can legitimately
+    // disagree. Concretely, `kasada` matches this CLI's table but is absent
+    // from node's block pattern, and the deprecated flat `blockType` fallback
+    // is a field node's classifier no longer reads at all — so a run can come
+    // back `failureKind:'auth'` (the server says login wall) while also
+    // matching a vendor here (this CLI says Kasada/DataDome/etc), both true
+    // signals about the same run, neither one wrong.
+    //
+    // trawl_cli#169 precedent: never assert client-side what only the server
+    // knows. Silently picking a winner between two disagreeing signals is
+    // exactly that — printing "no reliable bypass" alone asserts the run is
+    // NOT an auth wall (may stop someone from trying a fixable cookie
+    // problem); printing the cookie hint alone asserts the run is NOT a
+    // vendor wall (may send someone chasing cookies against a wall no cookie
+    // fixes). This CLI cannot tell which is true without either importing
+    // node's BLOCK_TYPE_PATTERN (a second source of truth for the same list —
+    // how this defect got here) or re-detecting the login URL itself (the
+    // client-side re-derivation this fix deliberately rejects). So when both
+    // fire, render ONE hedged line naming both readings instead of picking:
+    // never emit the bare "no reliable bypass" verdict in that case — it is
+    // exactly the false precision this branch exists to avoid.
+    //
+    // `conflictingSignals` is deliberately NOT gated on `scrapId` the way the
+    // plain hint below is: the false "no reliable bypass" verdict is wrong
+    // regardless of whether we also have an id to build an actionable command
+    // for, so a scrapId-less conflicting run must still avoid it. Only the
+    // action line degrades (to a generic Settings pointer) when there's no id.
+    //
+    // Single-signal cases are untouched: a plain vendor match (no
+    // failureKind:'auth') still gets "no reliable bypass"; a plain
+    // failureKind:'auth' (no vendor match) still gets the plain hint block
+    // below. `errMsg` stays suppressed whenever any hint form (conflicting or
+    // plain) is about to render — node's own `errorMessage` for an auth run
+    // (historys.service.js ~line 774) IS the hint copy verbatim, literal
+    // `<id>` placeholder and all, so printing it here would just be a second
+    // rendering of the same information. If neither hint form renders (no
+    // vendor conflict and no scrapId for the plain hint), fall back to node's
+    // raw message rather than dropping the error entirely.
     const wallVendor = detectWallVendor(run);
     const errMsg = run.errorMessage ?? run.errorSnapshot?.errorMessage;
-    if (wallVendor) {
+    // trawl_cli#182 — same `status`/`statusDetail` guard as `detectWallVendor`
+    // above, and for the same reason (see its doc comment): `failureKind` is a
+    // terminal classification trawl_node stamps once
+    // (modules/historys/services/historys.service.js ~line 837), but
+    // `patchForRegression` (modules/scraps/services/scraps.service.js ~line
+    // 1972) can flip `status`/`statusDetail` to success/regression LATER,
+    // without ever clearing `failureKind`. Left unguarded, a run that
+    // ultimately succeeded or degraded could still carry a stale
+    // `failureKind:'auth'` and render the login-wall hint (or the
+    // conflicting-signals hedge) right next to a green "success" badge or a
+    // "Regression: length N vs baseline M" line — the exact contradiction
+    // `detectWallVendor`'s guard already exists to prevent for `block.kind`.
+    // One rule, not two coincidences: both checks guard the same two fields
+    // against the same after-the-fact patch.
+    const authWall = run.failureKind === 'auth' && run.status !== true && run.statusDetail !== 'regression';
+    const authHintWillRender = authWall && Boolean(scrapId);
+    const conflictingSignals = Boolean(wallVendor) && authWall;
+    if (conflictingSignals) {
+        lines.push(chalk.dim('  Error:   ') + chalk.yellow(`conflicting signals — the server reported ${wallVendor} on this run, but also classified it as a login wall; can't tell which is true here`));
+        lines.push(chalk.dim(scrapId
+            ? `  → cookies might help, but this may still be a genuine ${wallVendor} block: trawl scraps account session set ${scrapId} -c cookies.json`
+            : `  → cookies might help, but this may still be a genuine ${wallVendor} block (see app Settings → Account)`));
+    }
+    else if (wallVendor) {
         lines.push(chalk.dim('  Error:   ') + chalk.red(`walled: ${wallVendor} — no reliable bypass`));
     }
-    else if (errMsg) {
+    else if (errMsg && !authHintWillRender) {
         lines.push(chalk.dim('  Error:   ') + chalk.red(errMsg));
     }
     // Failed selector
@@ -149,8 +224,32 @@ export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
         const page = run.emptyContext.page;
         lines.push(chalk.dim('  Empty context:   ') + `url=${page.url ?? '?'}  anchors=${page.totalAnchors ?? '?'}`);
     }
-    // Regression detail
-    if (run.regressionDetected) {
+    // Login-wall hint (trawl_node#1975 `failureKind==='auth'`) — trust node's
+    // verdict, never re-detect the login URL client-side. Hypothesis-framed
+    // ("likely fix"), not a promise: measured cases (Reddit/X/Instagram) had
+    // valid cookies and stayed walled. Gated on `scrapId` alone (via
+    // `authHintWillRender` above, shared with the generic error line so the
+    // two can't drift apart), NOT the `scrapId ?? run._id` fallback used
+    // elsewhere here — `run._id` is a history id, and the session-set command
+    // on one silently targets the wrong document.
+    //
+    // trawl_cli#182 — excludes `conflictingSignals`: when a wall-vendor match
+    // also fired, the combined hedged line above already covers the cookie
+    // suggestion. Rendering this plain, unhedged block too would stack a
+    // second verdict under the first (and directly contradict it, since this
+    // block asserts the run IS a login wall with no caveat) — exactly the
+    // two-verdicts-for-one-run defect this fix removes. Exactly one verdict
+    // block renders per run.
+    if (authHintWillRender && !conflictingSignals) {
+        lines.push('');
+        lines.push(chalk.yellow('  Login wall (hypothesis): ') + 'this looks like a login redirect — your own session cookies are the likely fix.');
+        lines.push(chalk.dim(`  → trawl scraps account session set ${scrapId} -c cookies.json   (or app Settings → Account)`));
+    }
+    // Regression detail. trawl_node#1950 dropped the redundant `regressionDetected`
+    // boolean — it was `true` on every row iff `statusDetail === 'regression'`,
+    // so read that directly (works against a pre- or post-#1950 backend alike,
+    // no CLI/backend sequencing gap: `statusDetail` isn't part of this rename).
+    if (run.statusDetail === 'regression') {
         lines.push(chalk.dim('  Regression:      ') + `length ${run.length ?? '?'} vs baseline ${run.baselineLength ?? '?'}`);
     }
     // Autofix summary (when fix exists)

package/dist/commands/scraps.js

@@ -136,6 +136,67 @@ function statusIcon(status) {
         return chalk.hex('#FFA500')('▼');
     return chalk.dim('—');
 }
+/**
+ * #179 — health badge, next to the existing status icon. Two distinct
+ * sources, deliberately never blended into one derived verdict:
+ *   1. `consecutiveFailedRuns`/`unhealthySince` — present ONLY when `--unhealthy`
+ *      asked the API to attach them (see attachListCommand). This is the
+ *      live, exact server verdict — shown whenever available, since it is
+ *      strictly more informative than the breadcrumb below.
+ *   2. `lastCronOutcome === 'skipped_unhealthy'` — always present (a normal
+ *      scrap field, no extra query cost), but a STALE breadcrumb of the last
+ *      scheduled tick, not a live read: never set on a no-cron scrap, lags
+ *      by up to one tick, and flips back to 'ok'/'error' once the recovery
+ *      probe fires. Labeled as what it is ("cron paused"), never upgraded to
+ *      "unhealthy" — that word is reserved for the live verdict above.
+ * Never re-derives either signal from `history[]` — both come straight off
+ * the server (#1952 in trawl_node).
+ */
+function healthBadge(scrap) {
+    if (typeof scrap.consecutiveFailedRuns === 'number' && scrap.consecutiveFailedRuns > 0) {
+        const n = scrap.consecutiveFailedRuns;
+        return chalk.yellow(`⚠ failing ${n} ${n === 1 ? 'run' : 'runs'} (since ${formatDate(scrap.unhealthySince)})`);
+    }
+    if (scrap.lastCronOutcome === 'skipped_unhealthy') {
+        return chalk.yellow('⚠ cron paused');
+    }
+    return chalk.dim('—');
+}
+/**
+ * #179 — old-server fallback for `list --unhealthy`, same shape as
+ * `warnIfUnconfirmedTier` above (#86 finding 4b): an older trawl_node that
+ * predates #1952 ignores the unknown `minConsecutiveFailures` query param
+ * entirely and returns the FULL unfiltered list — silently presenting every
+ * scrap as "unhealthy" is exactly the kind of unconfirmed-outcome lie that
+ * precedent exists to prevent. A confirming server attaches
+ * `consecutiveFailedRuns` to EVERY item once the param is set (the
+ * controller writes `s.consecutiveFailedRuns || 0` unconditionally in that
+ * branch — see trawl_node scraps.controller.js), so its total absence across
+ * a non-empty response is the honest tell. Stderr only (safe under --json —
+ * stdout purity is untouched), never blocks the (unreliable) results below.
+ */
+function healthFilterUnconfirmed(data, unhealthyWasRequested) {
+    if (!unhealthyWasRequested || data.length === 0)
+        return false;
+    return !data.some((s) => s.consecutiveFailedRuns !== undefined);
+}
+function warnIfUnconfirmedHealthFilter(data, unhealthyWasRequested) {
+    if (!healthFilterUnconfirmed(data, unhealthyWasRequested))
+        return;
+    console.error(chalk.yellow('⚠ Server did not confirm the --unhealthy filter (older server, predates trawl_node#1952) — results below are UNFILTERED, not just the unhealthy scraps.'));
+}
+/**
+ * #179 — the machine-readable counterpart to the stderr warning above, in the
+ * exact shape `withTierUnconfirmed` already uses for the tier case. A --json
+ * caller has no reliable reason to read stderr, so without this a script sees
+ * a full unfiltered list and cannot tell it is not the unhealthy set.
+ * Emitted ONLY when the filter went unconfirmed — never fabricated otherwise.
+ */
+function withHealthFilterUnconfirmed(rows, data, unhealthyWasRequested) {
+    if (!healthFilterUnconfirmed(data, unhealthyWasRequested))
+        return rows;
+    return { scraps: rows, _healthFilterUnconfirmed: true };
+}
 export const scraps = new Command('scraps').description('Manage scraps');
 // shared SSE streaming helper. `asJson` (#107) emits one raw JSON object per
 // line (NDJSON) on stdout instead of the human-formatted timestamped text —
@@ -426,6 +487,15 @@ export async function pollRunProgress(id, before, opts = {}) {
 // keeps `--status`'s allowed values and its error message in one place,
 // mirrored from lastStatus()'s own return type so the two can never drift.
 const VALID_LAST_STATUSES = ['success', 'failure', 'never', 'running', 'regression'];
+// #179 — mirrors `UNHEALTHY_STREAK_LENGTH` in trawl_node
+// modules/scraps/helpers/scrapCronHealth.js. `--unhealthy` is a boolean
+// convenience flag for that ONE product-defined threshold (also what the
+// server's own cron-skip gate and email use), not a general numeric filter —
+// the API's `minConsecutiveFailures` param accepts any threshold, but this
+// CLI only ever asks for the one value the rest of the product means by
+// "unhealthy". If trawl_node's threshold ever changes, update this constant
+// to match.
+const UNHEALTHY_STREAK_LENGTH = 3;
 // list — promoted to a top-level verb (#108, see the AttachOptions comment above)
 export function attachListCommand(parent, attachOpts = {}) {
     return parent
@@ -443,6 +513,14 @@ export function attachListCommand(parent, attachOpts = {}) {
         // (~line 660) and report the actual bad input in the error message.
         .option('--limit <n>', 'Show only the first N results')
         .option('--page <n>', 'Fetch a specific page only (50 per page, no auto-pagination)')
+        // #179 — backed by the server's `minConsecutiveFailures` filter
+        // (trawl_node#1952): restricts the result set server-side to scraps
+        // whose CURRENT consecutive-failure streak is >= UNHEALTHY_STREAK_LENGTH,
+        // and annotates each returned scrap with the exact `consecutiveFailedRuns`/
+        // `unhealthySince` the aggregation computed. Deliberately never derived
+        // client-side from `history[]` — that would drift from what the web
+        // dashboard shows (comes-io/trawl_vue#1299 reads the identical filter).
+        .option('--unhealthy', `Only show scraps failing their last ${UNHEALTHY_STREAK_LENGTH}+ runs (server-computed)`)
         .action(async (opts, cmd) => {
         // #149 item 3 — validate --status against its enum the same way
         // --tier already validates (usageError + return, checked first, before
@@ -473,19 +551,23 @@ export function attachListCommand(parent, attachOpts = {}) {
                 return;
             }
         }
+        // #179 — same param on every request this action can issue (both
+        // branches below), so a filtered result stays filtered across the
+        // fetch-all pagination loop, not just its first page.
+        const healthFilter = opts.unhealthy ? `&minConsecutiveFailures=${UNHEALTHY_STREAK_LENGTH}` : '';
         let data;
         try {
             data = await spin(async () => {
                 if (page !== undefined) {
                     // Single-page mode: explicit page requested, no loop
-                    return api.get(`/api/scraps?perPage=50&page=${page}`);
+                    return api.get(`/api/scraps?perPage=50&page=${page}${healthFilter}`);
                 }
                 // Fetch-all mode: paginate until a page returns < 200 items
                 const perPage = 200;
                 let result = [];
                 let pageNum = 1;
                 while (true) {
-                    const batch = await api.get(`/api/scraps?perPage=${perPage}&page=${pageNum}`);
+                    const batch = await api.get(`/api/scraps?perPage=${perPage}&page=${pageNum}${healthFilter}`);
                     result = result.concat(batch);
                     if (batch.length < perPage)
                         break;
@@ -519,21 +601,28 @@ export function attachListCommand(parent, attachOpts = {}) {
             process.exitCode = exitCode;
             return;
         }
+        // #179 — check the RAW response, before --status narrows it further,
+        // so a genuinely empty (all-healthy) result is never mistaken for an
+        // unconfirmed filter.
+        warnIfUnconfirmedHealthFilter(data, opts.unhealthy);
         if (opts.status)
             data = data.filter((s) => lastStatus(s) === opts.status);
         const totalMatched = data.length;
         const rows = limit !== undefined ? data.slice(0, limit) : data;
         if (opts.json)
-            return json(rows);
+            return json(withHealthFilterUnconfirmed(rows, data, opts.unhealthy));
         const tableRows = rows.map((s) => ({
             id: s._id,
             title: s.title || '(untitled)',
             cron: s.cron || '—',
             status: statusIcon(lastStatus(s)),
+            // #179 — next to the existing status badge, per the issue's own
+            // wording (see healthBadge() above for the two source signals).
+            health: healthBadge(s),
             'last run': lastRun(s),
             updated: formatDate(s.updatedAt),
         }));
-        table(tableRows, ['id', 'title', 'cron', 'status', 'last run', 'updated']);
+        table(tableRows, ['id', 'title', 'cron', 'status', 'health', 'last run', 'updated']);
         // Print footer when --limit truncates
         if (limit !== undefined && rows.length < totalMatched) {
             console.log(chalk.dim(`Showing ${rows.length} of ${totalMatched} — omit --limit to see all`));
@@ -556,6 +645,10 @@ export function attachGetCommand(parent, attachOpts = {}) {
         console.log(chalk.dim(`  ID:       `) + data._id);
         console.log(chalk.dim(`  Cron:     `) + (data.cron || '—'));
         console.log(chalk.dim(`  Status:   `) + statusIcon(lastStatus(data)));
+        // #179 — same badge `list` renders (see healthBadge() above); `get`
+        // never sends `minConsecutiveFailures`, so this always reads the
+        // free `lastCronOutcome` breadcrumb, never the live per-run count.
+        console.log(chalk.dim(`  Health:   `) + healthBadge(data));
         console.log(chalk.dim(`  Last run: `) + lastRun(data));
         console.log(chalk.dim(`  Updated:  `) + formatDate(data.updatedAt));
     });
@@ -1141,7 +1234,9 @@ export function attachHistoryCommand(parent, attachOpts = {}) {
             time: h.time ?? null,
             tier: h.proxyTier ?? null,
             failureKind: h.failureKind ?? null,
-            blockType: h.blockType ?? null,
+            // trawl_node#1950 renamed blockType to block.kind; fall back to the
+            // deprecated flat field for a prod backend not yet on that tag.
+            blockType: h.block?.kind ?? h.blockType ?? null,
             createdAt: h.createdAt ?? null,
         }));
         if (opts.json) {
@@ -1167,7 +1262,9 @@ export function attachRunInfoCommand(parent, attachOpts = {}) {
             time: h.time ?? null,
             tier: h.proxyTier ?? null,
             failureKind: h.failureKind ?? null,
-            blockType: h.blockType ?? null,
+            // trawl_node#1950 renamed blockType to block.kind; fall back to the
+            // deprecated flat field for a prod backend not yet on that tag.
+            blockType: h.block?.kind ?? h.blockType ?? null,
             errorMessage: h.errorSnapshot?.errorMessage ?? null,
             selector: h.errorSnapshot?.selector ?? null,
             emptyContext: h.emptyContext ?? null,

package/docs/agent-quickstart.md

@@ -81,7 +81,7 @@ first-class on every one, and none of them ever blocks on a prompt (see
 ```
 trawl create <url> --prompt <goal> [--no-autofix] [--json]   # url also accepted as --url <url> (#116)
                                                               Create a persistent, self-healing scrap from a URL + a goal (AI-generated)
-trawl list|ls [--json] [--status <s>] [--limit <n>] [--page <n>]   List all scraps
+trawl list|ls [--json] [--status <s>] [--unhealthy] [--limit <n>] [--page <n>]   List all scraps
 trawl get <id> [--json]                        Get scrap details
 trawl run <id> [--watch] [--json]              Run a scrap (blocks until finished)
 trawl trigger <id> [--watch] [--wait] [--json] Launch a scrap as a background worker

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trawlme/cli",
-  "version": "3.8.2",
+  "version": "3.9.1",
   "description": "Trawl CLI — manage scraps from the terminal",
   "type": "module",
   "bin": {