@trawlme/cli 3.11.0 → 3.12.1

package/README.md

@@ -36,7 +36,7 @@ Four methods:
 3. **Token flag** — `trawl login --token <jwt>` (CI/CD, direct JWT)
 4. **API key** — `TRAWL_API_KEY=trawl_xxx trawl list` (scoped, revocable one-by-one — the recommended credential for an agent driving this CLI; see [docs/agent-quickstart.md](docs/agent-quickstart.md))
 
-A `TRAWL_API_KEY` is a `trawl_*`-prefixed credential (create/revoke one in the Trawl dashboard) sent as `Authorization: Bearer` instead of the session `Cookie: TOKEN=` a JWT uses — `trawl` picks the right one automatically based on the credential's own shape, never a flag. `TRAWL_API_KEY` wins over `TRAWL_TOKEN` when both happen to be set. It is scoped server-side — in practice to a set of scraps, since the dashboard's key-create form sets only the scrap allow-list; narrowing which *actions* a key may perform needs an explicit `scopes` array at creation over the API, and a key without one has every action granted. It works on most of the Core tier — `create`/`list`/`get`/`data`/`history`/`run-info`/`run`/`trigger`/`ping`, including the `--watch` **polling flag** on `run`/`trigger` — plus, outside Core, `scraps account status`/`scraps doctor`/`scraps autofix` (all three only ever read routes trawl_node opened to keys). It does **not** work on `whoami`, `scraps update`/`delete`, `scraps account set`/`delete`/`clear-session`/`session set`, `scraps banner`, `scraps snapshot` (the scrap lookup it starts from is dual-auth, but the `html-snapshot` route it downloads from isn't), or the standalone SSE **command** `scraps watch` (do not conflate the two: `--watch` is a flag on `run`/`trigger` and works under a key; `scraps watch` is a separate command and is JWT-only) — those stay JWT-only and fail with a `kind:"auth"` envelope (exit `3`) pointing at `trawl login` under a key. This list mirrors trawl_node's route wiring as of this writing, not a frozen guarantee — for anything not named here, trust the real `--json` envelope over this paragraph. Never send both a key and a JWT on the same request — the CLI only ever attaches one. `list --unhealthy` (and the plain health badge on `list`/`get`) rides the same `GET /api/scraps`/`GET /api/scraps/:id` routes as the rest of the group — no separate auth path — so it works under a key exactly like plain `list`/`get`, and a scrap-scoped key still only ever sees its own allow-listed scraps through the filter.
+A `TRAWL_API_KEY` is a `trawl_*`-prefixed credential (create/revoke one in the Trawl dashboard) sent as `Authorization: Bearer` instead of the session `Cookie: TOKEN=` a JWT uses — `trawl` picks the right one automatically based on the credential's own shape, never a flag. `TRAWL_API_KEY` wins over `TRAWL_TOKEN` when both happen to be set. It is scoped server-side — in practice to a set of scraps, since the dashboard's key-create form sets only the scrap allow-list; narrowing which *actions* a key may perform needs an explicit `scopes` array at creation over the API, and a key without one has every action granted. It works on most of the Core tier — `create`/`list`/`get`/`data`/`history`/`run-info`/`run`/`trigger`/`ping`, including the `--watch` **polling flag** on `run`/`trigger` — plus, outside Core, `scraps account status`/`scraps doctor`/`scraps autofix` (all three only ever read routes trawl_node opened to keys). It does **not** work on `whoami`, `scraps update`/`delete`, `scraps account set`/`delete`/`clear-session`/`session set`/`session capture`, `scraps banner`, `scraps snapshot` (the scrap lookup it starts from is dual-auth, but the `html-snapshot` route it downloads from isn't), or the standalone SSE **command** `scraps watch` (do not conflate the two: `--watch` is a flag on `run`/`trigger` and works under a key; `scraps watch` is a separate command and is JWT-only) — those stay JWT-only and fail with a `kind:"auth"` envelope (exit `3`) pointing at `trawl login` under a key. This list mirrors trawl_node's route wiring as of this writing, not a frozen guarantee — for anything not named here, trust the real `--json` envelope over this paragraph. Never send both a key and a JWT on the same request — the CLI only ever attaches one. `list --unhealthy` (and the plain health badge on `list`/`get`) rides the same `GET /api/scraps`/`GET /api/scraps/:id` routes as the rest of the group — no separate auth path — so it works under a key exactly like plain `list`/`get`, and a scrap-scoped key still only ever sees its own allow-listed scraps through the filter.
 
 Custom API URL: `trawl login --url https://self-hosted.example.com`
 
@@ -113,9 +113,12 @@ trawl scraps account delete <id> [--force] [--json]
 trawl scraps account clear-session <id> [--json]
 trawl scraps account status <id> [--json]
 trawl scraps account session set <id> -c <file> [--json]
+trawl scraps account session capture <id> [--chrome <path>] [--json]
 ```
 
-`account session set` uploads a Puppeteer cookie JSON array to bootstrap a logged-in session without storing credentials (BYO-cookies). A missing `-u/--username`/`-p/--password` on `account set` follows the same [non-interactive rule](#non-interactive-rule) as `login`.
+`account session set` uploads a cookie JSON array — or a `{ cookies, origins }` storageState file (see `session capture` below) — to bootstrap a logged-in session without storing credentials (BYO-cookies). A missing `-u/--username`/`-p/--password` on `account set` follows the same [non-interactive rule](#non-interactive-rule) as `login`.
+
+`account session capture <id>` is the one-command version of the same idea: it opens a real, visible Chrome window at the scrap's target URL, waits for **you** to log in there — 2FA included — then reads the resulting session over the Chrome DevTools Protocol (cookies **and** per-origin localStorage) and uploads it. You stay authenticated as yourself; Trawl never sees your credentials, and responsibility for lawful use of the captured session stays with you. The capture is scoped to the scrap's own target host by RFC 6265 domain-matching, the same rule a real browser uses to decide which cookies to send: a cookie explicitly scoped to a domain (e.g. `Domain=example.com`) is in scope for that host and its subdomains, while a host-only cookie (no `Domain` attribute) is in scope only for the exact host it was set on — never a sibling subdomain, even one sharing a parent domain, and never another tenant on the same multi-tenant hosting domain (e.g. a different `*.github.io` site). localStorage follows the same anchor: an origin is in scope only if it's the target host itself or a subdomain of it. The one deliberate gap: a host-only cookie set on a sibling host (e.g. `auth.example.com` while the scrap targets `app.example.com`) is excluded — a real browser would never send it to the target host either, so nothing replay-relevant is lost unless the scrap script itself later navigates to that sibling. Completion is explicit: press Enter in the terminal once you're signed in — the primary path, and the only one guaranteed cross-platform. Closing the Chrome window also completes the capture (cookies only, not localStorage) when Chrome itself stays running after its last window closes, which this CLI verified on macOS; on a platform where closing the last window quits Chrome entirely, the command reports that fact instead of a session, and pressing Enter is the reliable path there. This command needs a local interactive terminal with a real display; it does not work headless, in CI, or over a plain SSH session, and auto-detects a system Chrome/Chromium (override with `--chrome <path>` or `TRAWL_CHROME_PATH`). Never prints the captured session itself, in any mode — `--json` echoes only the server's response plus capture counts.
 
 ### Claude Code skills
 

package/dist/commands/create.d.ts

@@ -1,37 +1,9 @@
 import { Command } from 'commander';
-/**
- * `POST /api/ai/wizard` response contract (#114, trawl_node —
- * ai.wizard.service.js#runWizard, shipped S1, contract LOCKED). The wizard
- * chains, entirely server-side: AI code generation -> scrap creation ->
- * the FIRST run (ScrapsService.load) -> auto-fix on failure (autoFix
- * defaults true unless `--no-autofix` maps to `autoFix:false`).
- *
- *  - `success` is an HONEST outcome of that first run, not "did the HTTP
- *    call succeed" — a failed first run is still a 200 response (the scrap
- *    itself was created either way; auto-fix, when enabled, retries in the
- *    background). Callers must branch on `success`, never assume 2xx means
- *    "the scrap works".
- *  - `scrap` is the full created Scrap object (present whenever creation got
- *    far enough to persist it — a hard failure before that point surfaces as
- *    a real HTTP error instead, handled by the shared error path).
- *  - `historyId` is best-effort (a failed server-side lookup leaves it
- *    `null`, never breaks the response) — it points at the first run just
- *    executed.
- */
 export interface WizardResponse {
     success: boolean;
     scrap?: {
         _id: string;
         title: string;
-        /**
-         * Cron expression the wizard schedules this scrap on. As of #114/S1
-         * (trawl_node ai.wizard.service.js#runWizard's `scrapBody`) this is
-         * hardcoded server-side to a DAILY run — `'0 7 * * *'` / `cronTimezone:
-         * 'UTC'` — unconditionally, regardless of `--prompt`/`--no-autofix`.
-         * Present on the scrap object returned here (the wizard controller
-         * passes the created scrap straight through, no stripping) — change or
-         * disable it with `trawl scraps update <id> --cron <expr>` / `--no-cron`.
-         */
         cron?: string | null;
         cronTimezone?: string;
         [key: string]: unknown;

package/dist/commands/create.js

@@ -8,12 +8,6 @@ import { requireUrl, requireString } from '../lib/validate.js';
 import { UsageError } from '../lib/errors.js';
 import { renderPinch, pinchEnabled } from '../lib/pinch.js';
 import { startPinchAnimation } from '../lib/pinchAnimation.js';
-/** Best-effort, honest first-run summary — never claims a background retry
- * happened when auto-fix was disabled for this call, and never claims a
- * scrap was persisted when the response carries none (#114-F3 — a hard
- * failure before persistence still comes back as `success:false` with no
- * `scrap` at all; claiming "auto-fix retrying in the background" then would
- * be fabricated — there is nothing to retry). */
 function firstRunLabel(data, autoFixEnabled) {
     if (data.success)
         return 'succeeded';
@@ -23,10 +17,6 @@ function firstRunLabel(data, autoFixEnabled) {
         return 'failed (auto-fix retrying in the background)';
     return 'failed';
 }
-/** Best-effort human description of a daily cron (`M H * * *`) — the only
- * shape the wizard's server-side default currently produces. Falls back to
- * printing the raw expression for anything else rather than guessing at a
- * schedule the CLI can't actually parse. */
 function describeCron(cron) {
     const match = /^(\d{1,2})\s+(\d{1,2})\s+\*\s+\*\s+\*$/.exec(cron.trim());
     if (!match)
@@ -34,16 +24,6 @@ function describeCron(cron) {
     const [, min, hour] = match;
     return `daily ${hour.padStart(2, '0')}:${min.padStart(2, '0')}`;
 }
-/**
- * #114-F2 — a wizard-created scrap runs on a DAILY cron by default
- * server-side; surface that up front rather than leaving it to be
- * discovered later as an unexpected recurring quota charge. Reads the real
- * `cron`/`cronTimezone` field off the response scrap when present; falls
- * back to the known wizard default wording ONLY when the field is missing
- * from the response (an older server, or a future rename) — never invents a
- * schedule value that might not match what the server actually applied.
- * Returns null when there's no scrap to schedule at all.
- */
 function scheduleLabel(scrap) {
     if (!scrap)
         return null;
@@ -53,24 +33,11 @@ function scheduleLabel(scrap) {
     }
     return 'scheduled daily by default';
 }
-/**
- * #121 — onboarding principle: a successful `create` should SHOW the value,
- * not just an id. Best-effort fetch of the first run's persisted data (the
- * same read-only, no-quota path `trawl data <id>` uses:
- * `GET /api/historys/:historyId` → a JSON string `{ data: [...] }`) and print
- * a small proof-of-value sample (count + first-item keys + one truncated
- * value line — never a full dump; that's what `trawl data --json` is for).
- * ANY failure (network, parse, no data) is swallowed silently — the sample is
- * a bonus, it must never turn a successful create into a failure or noise.
- */
 async function printDataSample(historyId) {
     try {
         const detail = await api.get(`/api/historys/${historyId}`);
         if (typeof detail?.data !== 'string' || !detail.data)
             return;
-        // #159 — same JSON-string-within-JSON shape as scrap.history[0].data;
-        // tolerate the documented raw-control-char server quirk here too instead
-        // of silently losing the sample to the outer try/catch below.
         const items = parseServerJson(detail.data)?.data;
         if (!Array.isArray(items) || items.length === 0)
             return;
@@ -88,7 +55,6 @@ async function printDataSample(historyId) {
         }
     }
     catch {
-        // best-effort — a missing sample never fails or noises up a good create
     }
 }
 export const create = new Command('create')
@@ -99,15 +65,6 @@ export const create = new Command('create')
     .option('--no-autofix', 'Disable AI auto-fix on first-run failure (default: on)')
     .option('--json', 'Output the raw API payload')
     .action(async (rawUrl, opts) => {
-    // Fast, local usage-errors (exit 2) — never a round-trip to the server
-    // for something we can already tell is bad. Same fail-fast pattern the
-    // former `fetch` command used for its URL argument.
-    //
-    // #116 — the url is accepted BOTH as the positional argument (agent
-    // one-liner) and as `--url` (muscle memory from `scraps create` and the
-    // API body {url, goal}). Exactly one is required; both are fine only
-    // when identical — two DIFFERENT urls is ambiguous, refuse loudly
-    // rather than silently picking one.
     if (rawUrl === undefined && opts.url === undefined) {
         throw new UsageError("missing required argument 'url' (positional, or --url <url>)");
     }
@@ -121,58 +78,28 @@ export const create = new Command('create')
         goal,
         ...(opts.autofix === false && { autoFix: false }),
     };
-    // #91/#106-F1 long-run pattern — the wizard runs AI generation + scrap
-    // creation + a real FIRST run (+ autofix retries) entirely server-side,
-    // legitimately 30-250s+. The 30s DEFAULT_TIMEOUT_MS would abort it
-    // mid-flight and fabricate a NetworkError timeout for a request that was
-    // always going to succeed.
     const call = () => api.post('/api/ai/wizard', body, { timeoutMs: LONG_RUN_TIMEOUT_MS });
-    // #106-F2 pattern carried over from `fetch` — under --json stdout must
-    // be provably pure: no spinner channel at all. Only the human path gets
-    // the ora progress indicator; --json calls the API directly.
     let data;
     if (opts.json) {
         data = await call();
     }
     else {
-        // #122/#131 — while the server-side wizard runs (legitimately 30-250s+,
-        // see LONG_RUN_TIMEOUT_MS above) Pinch ANIMATES in place (claw-wiggle)
-        // on a color-capable TTY; otherwise the ora spinner (which itself
-        // no-ops under a non-TTY, so piped human output stays clean). Both
-        // write to stderr only — stdout purity under `--json` is guaranteed by
-        // the `opts.json` branch above, never by this code.
         const anim = pinchEnabled() ? startPinchAnimation(`Creating a scrap from ${url}…`) : null;
         try {
             data = anim
                 ? await call()
                 : await spin(call, {
-                    // No verdict symbol (#106-F3) — ora's success only means "the
-                    // HTTP call didn't throw", not "the first run succeeded".
                     text: `Creating a scrap from ${url}…`,
                     successText: 'Request complete',
                 });
         }
         catch (err) {
             anim?.stop();
-            // #114-F1 — a client-side timeout (NetworkError, "timed out after
-            // …ms" per api.ts's safeFetch) does NOT mean the wizard failed
-            // server-side: the scrap creation + first run keep going on the
-            // server after the CLI gives up waiting, so the scrap may already
-            // exist (or land moments later). Warn BEFORE rethrowing so a retry
-            // isn't the first instinct — a blind retry creates a duplicate scrap
-            // and burns AI-generation quota a second time for the same goal.
-            // Only a NetworkError whose message identifies it as the timeout
-            // branch qualifies — a DNS/connection-refused NetworkError never
-            // reached the server at all, so there's nothing to warn about here.
             if (err instanceof NetworkError && /timed out/i.test(err.message)) {
                 console.error(chalk.yellow('⚠ The request timed out client-side, but the scrap may STILL have been created server-side — run `trawl list` before retrying (a retry creates a DUPLICATE scrap + burns quota).'));
             }
-            // Rethrow unchanged so index.ts's classifyError/exit-code taxonomy
-            // stays intact (this stays a NetworkError -> exit 5, same as before).
             throw err;
         }
-        // Success — stop + erase the animation so the result prints on a clean
-        // line (the celebrating/confused frame below is the one-shot outcome).
         anim?.stop();
     }
     if (opts.json) {
@@ -190,19 +117,11 @@ export const create = new Command('create')
         console.log(chalk.dim(`  First run:  `) + firstRunLabel(data, autoFixEnabled));
         if (schedule)
             console.log(chalk.dim(`  Schedule:   `) + schedule);
-        // #121 — on a successful first run, prove the value: show a small data
-        // sample (best-effort, silent on failure) before the next-step hint.
         if (data.success && data.historyId)
             await printDataSample(data.historyId);
-        // #121 — the natural next command is the DATA, not the metadata: point
-        // at `trawl data` primarily, keep `trawl get` as the secondary detail view.
         if (scrapId) {
             console.log(chalk.dim(`  Next step:  `) + `trawl data ${scrapId}` + chalk.dim(`  (details: trawl get ${scrapId})`));
         }
-        // #114-F3 — only claim a background retry is happening when a scrap
-        // actually exists to retry (never fabricate progress that isn't real);
-        // `firstRunLabel` above already covers the !scrap / autofix-disabled
-        // wording, this adds the actionable poll target on top.
         if (!data.success && data.scrap && autoFixEnabled) {
             const pollTarget = data.historyId
                 ? `\`trawl data ${scrapId}\` or \`trawl run-info ${data.historyId}\``
@@ -210,18 +129,10 @@ export const create = new Command('create')
             console.log(chalk.yellow(`  Note:       `) +
                 `Auto-fix is retrying in the background — do NOT re-run create; poll ${pollTarget}.`);
         }
-        // #122 — Pinch reacts to the HONEST first-run outcome (same signal the
-        // ✓/✗ line above already renders): celebrates a real success, looks
-        // confused on a genuine failure. Belt-and-suspenders `!opts.json`
-        // alongside pinchEnabled() — see the 'thinking' print above for why.
         if (!opts.json && pinchEnabled()) {
             console.log(data.success ? renderPinch('celebrating') : renderPinch('confused'));
         }
     }
-    // Honest exit code alongside the honest payload — a --json caller gets
-    // the raw body regardless (never wrapped/altered), but a script checking
-    // the exit code alone must be able to tell "first run failed" from "ran
-    // fine" without parsing. Same contract the former `fetch` command used.
     if (!data.success)
         process.exitCode = 1;
 });

package/dist/commands/doctor.d.ts

@@ -1,9 +1,3 @@
-/**
- * Run diagnostics shape — mirrors the owner-safe REST shape returned by
- * GET /api/historys/:id (Phase 1 projection, 2026-05-27).
- * Cost and proxy-nature fields are intentionally absent by design.
- * Abstract proxyTier (Tier 0–4) is kept.
- */
 export interface Run {
     _id: string;
     status: boolean | null;
@@ -28,11 +22,6 @@ export interface Run {
     block?: {
         kind?: string | null;
     } | null;
-    /** @deprecated trawl_node#1950 renamed this to `block.kind`. Kept as a read
-     * fallback (see detectWallVendor) for the window where this CLI is
-     * published ahead of the trawl_node prod tag — a prod backend served from
-     * the pre-#1950 tag still returns this flat field, not `block.kind`. Drop
-     * once prod is confirmed on a tag containing #1950. */
     blockType?: string | null;
     proxyTier?: string | null;
     baselineLength?: number | null;
@@ -40,64 +29,14 @@ export interface Run {
     createdAt?: string;
     time?: number | null;
     triggeredBy?: string | null;
-    /** trawl_node#1975 — freeform failure classification (`'auth'` = login-wall
-     * empty run, cookies are the fix). Not a TS union — trawl_node's own set
-     * is additive/open, so this CLI must stay read-safe against a future
-     * value it doesn't know about yet, same posture as `block.kind` above. */
     failureKind?: string | null;
 }
-/**
- * Resolve a known anti-bot vendor name from a worker `block.kind`
- * string. Returns null when the run succeeded, when there is no `block.kind`
- * signal at all, or when `block.kind` names something other than a known
- * vendor (e.g. `proxy-domain-gate`, `rate_limited_per_host`) — those stay on
- * the genuine-error path since we can't honestly attribute them to a specific
- * "no reliable bypass" wall.
- *
- * `blocked` is deliberately NOT consulted here (#177): it is a mid-run,
- * attempt-level signal the worker only stamps on one envelope shape, so it
- * reads `false` on the great majority of genuinely walled runs (the early-block
- * throw path sets `block.kind` but never `blocked` — 63 of 64 walled runs
- * measured on prod). `status === true` wins instead: a run that ultimately
- * returned data was not walled, even if an earlier tier's `block.kind` stamp
- * survived on the row.
- *
- * trawl_node#1950 renamed the flat `blockType` field to nested `block.kind`.
- * Read `block?.kind` first, falling back to the deprecated flat `blockType` —
- * this CLI can be published (and talk to prod) before the trawl_node prod tag
- * containing #1950 is cut (see the `Run.blockType` doc comment), so a prod
- * response can still be the pre-#1950 flat shape for a while.
- */
 export declare function detectWallVendor(run: Pick<Run, 'status' | 'statusDetail' | 'block' | 'blockType'>): string | null;
-/**
- * #184 — true for a run currently carrying a live (non-stale) login-wall
- * verdict. Extracted out of `formatDoctor`'s own local `authWall` const so
- * `commands/scraps.ts`'s `doctor`/`run-info` actions can reuse the EXACT
- * same guard to decide whether to fire the skills-install safety-net nudge
- * (lib/skillsNudge.ts `maybeSuggestSkillsForAuthWall`) — one rule, not two
- * copies that could quietly drift apart.
- *
- * Same staleness guard as `detectWallVendor` above and for the same reason:
- * `failureKind` is a terminal classification trawl_node stamps once, but
- * `patchForRegression` can flip `status`/`statusDetail` to success/
- * regression LATER without ever clearing it — so a run that ultimately
- * succeeded or degraded must never still read as an active auth wall.
- *
- * Typed structurally loose (not `Pick<Run, ...>`) on purpose: `Run.status`/
- * `statusDetail` are required fields, but `scraps.ts`'s own `HistoryRun`
- * (the run-info command's shape) declares the same three fields OPTIONAL —
- * a `Pick<Run, ...>` parameter type would reject that caller at compile
- * time even though every field it actually reads is present at runtime.
- */
 export declare function isAuthWall(run: {
     failureKind?: string | null;
     status?: boolean | null;
     statusDetail?: string | null;
 }): boolean;
-/**
- * Autofix activity metadata — from the persisted ai_fix_end activity.
- * aiUsage (cost) is stripped server-side; all diagnostics are kept.
- */
 export interface FixActivity {
     outcome: 'applied' | 'failed' | 'skipped' | 'breaker_tripped' | 'timeout' | null;
     classification?: string | null;
@@ -127,26 +66,8 @@ interface ScrapHead {
 }
 export declare function pickRun(run: Run): Partial<Run>;
 export declare function pickFix(fix: FixActivity | null): Partial<FixActivity> | null;
-/**
- * Pure formatter — returns a human-readable diagnosis string for a run.
- * Used by `doctor`, `doctor --autofix`, and `data --errors`.
- *
- * @param scrapTitle - Display name of the scrap
- * @param run        - Owner-safe history run object
- * @param fix        - Optional autofix activity (null = no fix attempted)
- * @param scrapId    - Scrap document id (used in hint lines — autofix/snapshot take scrap id, not run id)
- * @returns Multi-line string ready for console.log
- */
 export declare function formatDoctor(scrapTitle: string, run: Run, fix?: FixActivity | null, scrapId?: string): string;
-/**
- * Pure formatter — returns a human-readable autofix detail block.
- * Shows diff, dry-run results, knowledge consulted.
- */
 export declare function formatAutofix(fix: FixActivity): string;
-/**
- * Fetch the latest run + its ai_fix_end activity for a scrap.
- * Returns null if the scrap has no runs yet.
- */
 export declare function fetchRunAndFix(scrapId: string): Promise<{
     scrap: ScrapHead;
     run: Run;

package/dist/commands/doctor.js

@@ -1,27 +1,6 @@
 import { api } from '../lib/api.js';
 import chalk from 'chalk';
 import { formatDate } from '../lib/format.js';
-/**
- * Known anti-bot vendors that the worker's `block.kind` field may name
- * (ex-flat `blockType`, trawl_node#1950). Ordered by first-match;
- * `block.kind` is a freeform worker string, not an enum (see the `Run.block`
- * doc comment), so this is a best-effort substring match against the real
- * field — never an invented/mocked value.
- *
- * `datadome`/`perimeterx`/`akamai`/`cloudflare` are literal substrings the
- * current worker emits (detect.js). `kasada` is forward-compatible: the issue
- * (#62 / wall-registry WS-7) names it as a genuine wall, but the current
- * worker vocabulary does not yet emit it — this branch lights up
- * automatically if/when a future worker classifier does, without a CLI
- * change.
- *
- * trawl_cli#182 — an `auth` entry used to live here, rendering a login wall
- * as "walled: auth — no reliable bypass". Removed: unlike a real anti-bot
- * vendor, a login wall's own reliable bypass is the user's session cookies,
- * so that verdict was wrong, not just early. `failureKind==='auth'`
- * (trawl_node#1975) is the honest signal for a login wall — see the
- * dedicated branch in `formatDoctor` below.
- */
 const WALL_VENDOR_PATTERNS = [
     [/datadome/i, 'DataDome'],
     [/kasada/i, 'Kasada'],
@@ -29,66 +8,18 @@ const WALL_VENDOR_PATTERNS = [
     [/akamai/i, 'Akamai'],
     [/cloudflare/i, 'Cloudflare'],
 ];
-/**
- * Resolve a known anti-bot vendor name from a worker `block.kind`
- * string. Returns null when the run succeeded, when there is no `block.kind`
- * signal at all, or when `block.kind` names something other than a known
- * vendor (e.g. `proxy-domain-gate`, `rate_limited_per_host`) — those stay on
- * the genuine-error path since we can't honestly attribute them to a specific
- * "no reliable bypass" wall.
- *
- * `blocked` is deliberately NOT consulted here (#177): it is a mid-run,
- * attempt-level signal the worker only stamps on one envelope shape, so it
- * reads `false` on the great majority of genuinely walled runs (the early-block
- * throw path sets `block.kind` but never `blocked` — 63 of 64 walled runs
- * measured on prod). `status === true` wins instead: a run that ultimately
- * returned data was not walled, even if an earlier tier's `block.kind` stamp
- * survived on the row.
- *
- * trawl_node#1950 renamed the flat `blockType` field to nested `block.kind`.
- * Read `block?.kind` first, falling back to the deprecated flat `blockType` —
- * this CLI can be published (and talk to prod) before the trawl_node prod tag
- * containing #1950 is cut (see the `Run.blockType` doc comment), so a prod
- * response can still be the pre-#1950 flat shape for a while.
- */
 export function detectWallVendor(run) {
-    // A run that returned data was not walled. `status === true` is not the whole
-    // predicate: trawl_node flips a degraded-but-non-empty run to
-    // `status:false, statusDetail:'regression'` (scraps.service.js, #1112) via a
-    // patch that never clears `block.kind` — so keying on `status` alone would
-    // print "no reliable bypass" next to "Regression: length N vs baseline M",
-    // which is the same contradiction this guard exists to remove.
     if (run.status === true || run.statusDetail === 'regression')
         return null;
     const kind = run.block?.kind ?? run.blockType ?? null;
     if (!kind)
-        return null; // no signal at all
+        return null;
     for (const [pattern, label] of WALL_VENDOR_PATTERNS) {
         if (pattern.test(kind))
             return label;
     }
     return null;
 }
-/**
- * #184 — true for a run currently carrying a live (non-stale) login-wall
- * verdict. Extracted out of `formatDoctor`'s own local `authWall` const so
- * `commands/scraps.ts`'s `doctor`/`run-info` actions can reuse the EXACT
- * same guard to decide whether to fire the skills-install safety-net nudge
- * (lib/skillsNudge.ts `maybeSuggestSkillsForAuthWall`) — one rule, not two
- * copies that could quietly drift apart.
- *
- * Same staleness guard as `detectWallVendor` above and for the same reason:
- * `failureKind` is a terminal classification trawl_node stamps once, but
- * `patchForRegression` can flip `status`/`statusDetail` to success/
- * regression LATER without ever clearing it — so a run that ultimately
- * succeeded or degraded must never still read as an active auth wall.
- *
- * Typed structurally loose (not `Pick<Run, ...>`) on purpose: `Run.status`/
- * `statusDetail` are required fields, but `scraps.ts`'s own `HistoryRun`
- * (the run-info command's shape) declares the same three fields OPTIONAL —
- * a `Pick<Run, ...>` parameter type would reject that caller at compile
- * time even though every field it actually reads is present at runtime.
- */
 export function isAuthWall(run) {
     return run.failureKind === 'auth' && run.status !== true && run.statusDetail !== 'regression';
 }
@@ -120,28 +51,8 @@ export function pickFix(fix) {
         .filter((k) => k in fix)
         .map((k) => [k, fix[k]]));
 }
-/**
- * Pure formatter — returns a human-readable diagnosis string for a run.
- * Used by `doctor`, `doctor --autofix`, and `data --errors`.
- *
- * @param scrapTitle - Display name of the scrap
- * @param run        - Owner-safe history run object
- * @param fix        - Optional autofix activity (null = no fix attempted)
- * @param scrapId    - Scrap document id (used in hint lines — autofix/snapshot take scrap id, not run id)
- * @returns Multi-line string ready for console.log
- */
 export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
     const lines = [];
-    // Header + status badge. status:null means a run is IN FLIGHT (node
-    // persists {status:null, statusDetail:null, inFlight:true} the moment a
-    // run starts) — that must never render as "failed". (#88 item 1)
-    //
-    // #91 LOW — a regression row (status:false, statusDetail:'regression') was
-    // falling into the same red "failed" bucket as a genuine failure, then
-    // showing a contradictory "Regression: X vs Y" detail line right below it.
-    // A regression run's write actually SUCCEEDED (the item count just dropped
-    // vs baseline, detected by an async patch afterward — see #88 item 2 in
-    // scraps.ts) — it needs its own distinct badge, not "failed".
     const ok = run.status === true;
     const badge = ok
         ? chalk.green('● success')
@@ -154,69 +65,8 @@ export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
                     : chalk.red('● failed');
     lines.push(`${chalk.bold(scrapTitle)}  ${badge}${run.statusDetail ? ` (${run.statusDetail})` : ''}`);
     lines.push(chalk.dim(`  Run ID:  ${run._id}`));
-    // Error message — an honest accept-wall string for known-walled scraps
-    // (DataDome/Kasada/PerimeterX/Akamai terminal verdict from the worker),
-    // otherwise the real error (genuine transient failure).
-    //
-    // trawl_cli#182 — the wall-vendor verdict (this CLI's own
-    // WALL_VENDOR_PATTERNS matched against block.kind/blockType) and the
-    // login-wall hint (node's failureKind==='auth', trawl_node#1975) are two
-    // INDEPENDENT server-derived signals with no shared contract: node's block
-    // classifier (modules/historys/helpers/failureKind.js BLOCK_TYPE_PATTERN)
-    // and this CLI's vendor table live in different repos and can legitimately
-    // disagree. Concretely, `kasada` matches this CLI's table but is absent
-    // from node's block pattern, and the deprecated flat `blockType` fallback
-    // is a field node's classifier no longer reads at all — so a run can come
-    // back `failureKind:'auth'` (the server says login wall) while also
-    // matching a vendor here (this CLI says Kasada/DataDome/etc), both true
-    // signals about the same run, neither one wrong.
-    //
-    // trawl_cli#169 precedent: never assert client-side what only the server
-    // knows. Silently picking a winner between two disagreeing signals is
-    // exactly that — printing "no reliable bypass" alone asserts the run is
-    // NOT an auth wall (may stop someone from trying a fixable cookie
-    // problem); printing the cookie hint alone asserts the run is NOT a
-    // vendor wall (may send someone chasing cookies against a wall no cookie
-    // fixes). This CLI cannot tell which is true without either importing
-    // node's BLOCK_TYPE_PATTERN (a second source of truth for the same list —
-    // how this defect got here) or re-detecting the login URL itself (the
-    // client-side re-derivation this fix deliberately rejects). So when both
-    // fire, render ONE hedged line naming both readings instead of picking:
-    // never emit the bare "no reliable bypass" verdict in that case — it is
-    // exactly the false precision this branch exists to avoid.
-    //
-    // `conflictingSignals` is deliberately NOT gated on `scrapId` the way the
-    // plain hint below is: the false "no reliable bypass" verdict is wrong
-    // regardless of whether we also have an id to build an actionable command
-    // for, so a scrapId-less conflicting run must still avoid it. Only the
-    // action line degrades (to a generic Settings pointer) when there's no id.
-    //
-    // Single-signal cases are untouched: a plain vendor match (no
-    // failureKind:'auth') still gets "no reliable bypass"; a plain
-    // failureKind:'auth' (no vendor match) still gets the plain hint block
-    // below. `errMsg` stays suppressed whenever any hint form (conflicting or
-    // plain) is about to render — node's own `errorMessage` for an auth run
-    // (historys.service.js ~line 774) IS the hint copy verbatim, literal
-    // `<id>` placeholder and all, so printing it here would just be a second
-    // rendering of the same information. If neither hint form renders (no
-    // vendor conflict and no scrapId for the plain hint), fall back to node's
-    // raw message rather than dropping the error entirely.
     const wallVendor = detectWallVendor(run);
     const errMsg = run.errorMessage ?? run.errorSnapshot?.errorMessage;
-    // trawl_cli#182 — same `status`/`statusDetail` guard as `detectWallVendor`
-    // above, and for the same reason (see its doc comment): `failureKind` is a
-    // terminal classification trawl_node stamps once
-    // (modules/historys/services/historys.service.js ~line 837), but
-    // `patchForRegression` (modules/scraps/services/scraps.service.js ~line
-    // 1972) can flip `status`/`statusDetail` to success/regression LATER,
-    // without ever clearing `failureKind`. Left unguarded, a run that
-    // ultimately succeeded or degraded could still carry a stale
-    // `failureKind:'auth'` and render the login-wall hint (or the
-    // conflicting-signals hedge) right next to a green "success" badge or a
-    // "Regression: length N vs baseline M" line — the exact contradiction
-    // `detectWallVendor`'s guard already exists to prevent for `block.kind`.
-    // One rule, not two coincidences: both checks guard the same two fields
-    // against the same after-the-fact patch.
     const authWall = isAuthWall(run);
     const authHintWillRender = authWall && Boolean(scrapId);
     const conflictingSignals = Boolean(wallVendor) && authWall;
@@ -232,69 +82,38 @@ export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
     else if (errMsg && !authHintWillRender) {
         lines.push(chalk.dim('  Error:   ') + chalk.red(errMsg));
     }
-    // Failed selector
     if (run.errorSnapshot?.selector) {
         lines.push(chalk.dim('  Failed selector: ') + chalk.cyan(run.errorSnapshot.selector));
     }
-    // Blocked
     lines.push(chalk.dim('  Blocked: ') + (run.blocked ? chalk.red('yes') : 'no'));
-    // Proxy tier (abstract, kept for owner)
     if (run.proxyTier && TIER_LABELS[run.proxyTier]) {
         lines.push(chalk.dim('  Proxy:   ') + TIER_LABELS[run.proxyTier]);
     }
-    // Empty context detail
     if (run.statusDetail === 'empty' && run.emptyContext?.page) {
         const page = run.emptyContext.page;
         lines.push(chalk.dim('  Empty context:   ') + `url=${page.url ?? '?'}  anchors=${page.totalAnchors ?? '?'}`);
     }
-    // Login-wall hint (trawl_node#1975 `failureKind==='auth'`) — trust node's
-    // verdict, never re-detect the login URL client-side. Hypothesis-framed
-    // ("likely fix"), not a promise: measured cases (Reddit/X/Instagram) had
-    // valid cookies and stayed walled. Gated on `scrapId` alone (via
-    // `authHintWillRender` above, shared with the generic error line so the
-    // two can't drift apart), NOT the `scrapId ?? run._id` fallback used
-    // elsewhere here — `run._id` is a history id, and the session-set command
-    // on one silently targets the wrong document.
-    //
-    // trawl_cli#182 — excludes `conflictingSignals`: when a wall-vendor match
-    // also fired, the combined hedged line above already covers the cookie
-    // suggestion. Rendering this plain, unhedged block too would stack a
-    // second verdict under the first (and directly contradict it, since this
-    // block asserts the run IS a login wall with no caveat) — exactly the
-    // two-verdicts-for-one-run defect this fix removes. Exactly one verdict
-    // block renders per run.
     if (authHintWillRender && !conflictingSignals) {
         lines.push('');
         lines.push(chalk.yellow('  Login wall (hypothesis): ') + 'this looks like a login redirect — your own session cookies are the likely fix.');
         lines.push(chalk.dim(`  → trawl scraps account session set ${scrapId} -c cookies.json   (or app Settings → Account)`));
     }
-    // Regression detail. trawl_node#1950 dropped the redundant `regressionDetected`
-    // boolean — it was `true` on every row iff `statusDetail === 'regression'`,
-    // so read that directly (works against a pre- or post-#1950 backend alike,
-    // no CLI/backend sequencing gap: `statusDetail` isn't part of this rename).
     if (run.statusDetail === 'regression') {
         lines.push(chalk.dim('  Regression:      ') + `length ${run.length ?? '?'} vs baseline ${run.baselineLength ?? '?'}`);
     }
-    // Autofix summary (when fix exists)
     if (fix) {
         lines.push(`  ${chalk.magenta('Autofix:')}        ${fix.outcome ?? '?'}${fix.classification ? ` — ${fix.classification}` : ''}${fix.reason ? ` (${fix.reason})` : ''}`);
         lines.push(chalk.dim(`  → full diff/dry-run/knowledge: trawl scraps autofix ${scrapId ?? run._id}  (or doctor --autofix)`));
     }
-    // Timestamp
     if (run.createdAt) {
         lines.push(chalk.dim('  Run at:  ') + formatDate(run.createdAt));
     }
-    // Snapshot hint
     if (run.errorSnapshot?.html || run.statusDetail === 'empty' || run.statusDetail === 'error') {
         lines.push('');
         lines.push(chalk.dim(`  → trawl scraps snapshot ${scrapId ?? run._id} --error   (download error-path HTML)`));
     }
     return lines.join('\n');
 }
-/**
- * Pure formatter — returns a human-readable autofix detail block.
- * Shows diff, dry-run results, knowledge consulted.
- */
 export function formatAutofix(fix) {
     const lines = [];
     lines.push(chalk.magenta.bold('Autofix attempt'));
@@ -324,17 +143,12 @@ export function formatAutofix(fix) {
     }
     return lines.join('\n');
 }
-/**
- * Fetch the latest run + its ai_fix_end activity for a scrap.
- * Returns null if the scrap has no runs yet.
- */
 export async function fetchRunAndFix(scrapId) {
     const scrap = await api.get(`/api/scraps/${scrapId}`);
     const hid = scrap.history?.[0]?._id;
     if (!hid)
         return null;
     const run = await api.get(`/api/historys/${hid}`);
-    // api.get unwraps the { data: T } envelope — the response is the array directly
     const acts = await api.get(`/api/scraps/${scrapId}/activities?history=${hid}&limit=5`);
     const fixAct = Array.isArray(acts) ? acts.find((a) => a.type === 'ai_fix_end') : undefined;
     const fix = fixAct ? { ...fixAct.metadata, createdAt: fixAct.createdAt } : null;