@trawlme/cli 1.19.1 → 1.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -28,16 +28,28 @@ Custom API URL: `trawl login --url https://self-hosted.example.com`
28
28
 
29
29
  ## Commands
30
30
 
31
- All commands accept a global `--debug` flag to show full error stack traces on failure.
31
+ All commands accept a global `--debug` flag to show full error stack traces on failure. Every command below supports `--json` (a raw payload / structured result on stdout, nothing else) and never blocks on an interactive prompt when `--json` is set or stdin/stdout isn't a real TTY — see [Non-interactive rule](#non-interactive-rule) and [Exit codes](#exit-codes). Two documented exceptions to "single payload": `scraps watch --json` is a live stream, so it emits NDJSON (one JSON object per line) instead; `scraps snapshot --json` only changes behavior on a never-run scrap (`{"status":"no_runs"}`) — when a run exists it still writes the raw HTML snapshot regardless of `--json` (there's no JSON-encoded form of an HTML page to emit).
32
32
 
33
33
  ### Auth
34
34
 
35
35
  ```
36
- trawl login [--url <url>] [--token <jwt>] [--email <email>] [--password <pass>]
37
- trawl logout
38
- trawl token Print the stored session JWT (for MCP Bearer auth)
36
+ trawl login [--url <url>] [--token <jwt>] [--email <email>] [--password <pass>] [--json]
37
+ trawl logout [--json]
38
+ trawl token [--json] Print the stored session JWT (for MCP Bearer auth)
39
39
  ```
40
40
 
41
+ `login --json` prints `{"ok":true,"apiUrl","config",email?}` on success — never the raw token (that's `token`'s job). `token --json` prints `{"token","exp","expiresAt"}` instead of the bare JWT + stderr advisories. See [Non-interactive rule](#non-interactive-rule) below: a missing `--email`/`--password` refuses immediately (usage error, exit 2) instead of prompting when `--json` is set or stdin/stdout isn't a real TTY.
42
+
43
+ ### Agent core verbs
44
+
45
+ ```
46
+ trawl fetch <url> [--json] [--reason <text>] One-shot fetch + extract readable content from a public URL (no scrap needed)
47
+ trawl whoami [--json] Show the authenticated user's identity
48
+ trawl ping [--json] Health/version handshake against the Trawl API
49
+ ```
50
+
51
+ Fully non-interactive — all three read auth only from `TRAWL_TOKEN`/the stored login token, never prompt. `trawl fetch` is the REST counterpart of the MCP `trawl_fetch_url` tool (same shared engine, `POST /api/scraps/fetch-url`): `status` is an honest outcome (`completed`/`failed`/`empty`/`blocked`), not "did the HTTP call succeed" — a failed fetch is still a 200 response with `status:'failed'` + `error`, and the CLI exits 1 in that case (both human and `--json` modes) even though `--json` always prints the raw payload verbatim. `truncated:true` means the 100KB response cap tripped; `result` is dropped and `url` repoints at the full history row instead. `trawl whoami`/`trawl ping` mirror the MCP `trawl_whoami`/`trawl_health_ping` tools as closely as the REST surface allows (`GET /api/users/me` / `GET /api/health`) — `ping`'s `--json` payload is admin-enriched (version/uptime/db) and just `{"status":"ok"}` for anyone else.
52
+
41
53
  ### Scraps
42
54
 
43
55
  ```
@@ -45,20 +57,20 @@ trawl scraps list [--json] [--status <success|failure|never|running|regression>]
45
57
  trawl scraps get <id> [--json]
46
58
  trawl scraps create -t <title> [-u <url>] [-r <request>] [-d <description>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--json]
47
59
  trawl scraps update <id> [-t <title>] [-u <url>] [-r <request>] [-d <description>] [--cron <expr>|--no-cron] [--alert <email>|--no-alert] [--autofix|--no-autofix] [-p <json>|--params-file <path>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--force-tier <tier0|tier1|tier2|tier3|tier4>] [--json]
48
- trawl scraps run <id> [--watch]
49
- trawl scraps trigger <id> [--watch] [--wait]
50
- trawl scraps watch <id>
60
+ trawl scraps run <id> [--watch] [--json]
61
+ trawl scraps trigger <id> [--watch] [--wait] [--json]
62
+ trawl scraps watch <id> [--json]
51
63
  trawl scraps data <id> [--json] [--fresh] [--errors]
52
64
  trawl scraps history <id> [--json] [-n <limit>]
53
65
  trawl scraps run-info <hid> [--json]
54
66
  trawl scraps doctor <id> [--json] [--autofix]
55
67
  trawl scraps autofix <id> [--json]
56
68
  trawl scraps snapshot <id> [--error] [-o <file>] [--json]
57
- trawl scraps banner <id> -f <file>
58
- trawl scraps delete <id> [--force] Alias: rm
69
+ trawl scraps banner <id> -f <file> [--json]
70
+ trawl scraps delete <id> [--force] [--json] Alias: rm
59
71
  ```
60
72
 
61
- - `--tier` forces a proxy tier; `--force-tier` raises the proxy-tier ceiling past the auto-cap (history-gated: may be refused or cost more). `create --json`/`update --json` print the full scrap object (including the `_tierOverride` outcome) on stdout; a refused tier override exits 1 with a standard `--json` error envelope. When a tier was requested but the server's response carries no `_tierOverride` at all (an older server that can't confirm what actually got applied), a stderr warning is printed either way, and under `--json` the emitted object also carries `"_tierUnconfirmed": true` — the machine-readable counterpart to that warning, since a `--json` caller has no reliable reason to read stderr.
73
+ - `--tier` forces a proxy tier; `--force-tier` raises the proxy-tier ceiling past the auto-cap (history-gated: may be refused or cost more). `create --json`/`update --json` print the full scrap object (including the `_tierOverride` outcome) on stdout; a refused tier override exits 1 with a standard `--json` error envelope (`kind:"refused"` — distinct from `"unknown"`, so a script can branch on "the server said no"). When a tier was requested but the server's response carries no `_tierOverride` at all (an older server that can't confirm what actually got applied), a stderr warning is printed either way, and under `--json` the emitted object also carries `"_tierUnconfirmed": true` — the machine-readable counterpart to that warning, since a `--json` caller has no reliable reason to read stderr.
62
74
  - `scraps doctor` diagnoses the last run (error, failed selector, block status, page state, autofix outcome); `--autofix` includes the full autofix diff/dry-run/knowledge. A run that is still in flight (`status: null`, server-side) shows a `running` badge, never `failed`; a run whose item count regressed vs baseline (`statusDetail: "regression"`) shows its own amber `regression` badge, never `failed` either (`scraps list`/`get` show the same distinction — a `▼` icon, not the red `✗` a genuine failure gets).
63
75
  - `scraps autofix` shows the last auto-fix attempt on its own (decision, diff, dry-run, knowledge). `--json` on a scrap that has **never run** returns `{"status":"no_runs"}` (exit 0) — distinct from `null`, which means a run exists but had no auto-fix attempt.
64
76
  - `scraps snapshot --error` fetches the error-path snapshot instead of the normal one; `-o <file>` writes to a file instead of stdout. On a scrap that has **never run**: `--json` returns `{"status":"no_runs"}` (exit 0), matching `doctor`/`autofix`; `-o <file>` (without `--json`) exits 4 (`not_found`) instead of silently exiting 0 with nothing written — a script checking the exit code alone must be able to tell "no file was produced" from success. `--json` takes priority when both are passed.
@@ -67,31 +79,34 @@ trawl scraps delete <id> [--force] Alias: rm
67
79
  - `scraps get` (and anything reading through it, like `scraps data`'s default path) embeds only the newest 100 history rows on the returned scrap object — `scraps run-info` and `scraps doctor` fetch a single run directly and are unaffected by that cap.
68
80
  - `scraps banner -f <file>` only accepts `png`/`jpg`/`jpeg`/`webp`; any other extension is a usage error (exit 2) instead of silently uploading the file under a fabricated `image/png` Content-Type.
69
81
  - **Long-running scrap execute (`scraps run`, `data --fresh`, `trigger --wait`):** these hit the same server-side scrap-execute path, which can legitimately take 30–250s (proxy tier escalation, AI-fix retries) — the CLI arms a 300s timeout for exactly these three call sites instead of the generic 30s default. `TRAWL_TIMEOUT` (see below) still overrides ALL requests, including these — set it if you need a tighter or looser ceiling than 300s for a long-running scrap.
70
- - **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `scraps run --watch` and `scraps trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears; a run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>`. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
82
+ - **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `scraps run --watch` and `scraps trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears. The watched run's outcome drives the exit code too, in BOTH human and `--json` mode: a genuinely failed terminal run, a poll timeout, or a persistently unreachable API all exit non-zero — a clean successful run is the only exit `0`. A run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>` (human mode) — see the `--json` shape below. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
83
+ - `scraps run --json`/`trigger --json` bypass the spinner and print the raw launch/trigger payload on stdout; combined with `--watch`, every intermediate progress line stays suppressed (stdout stays pure JSON) and, once the watch reaches its outcome, exactly ONE final NDJSON line is emitted: `{"runId","status"}` (the honest terminal status — `success`/`error`/`empty`/`regression`/…), `{"runId","status":"timeout"}` on a poll timeout, or `{"runId","status":"poll_error","error"}` if the API stays unreachable for several consecutive polls — `process.exitCode` is non-zero for all three except a genuine success. `scraps watch --json` emits one raw JSON object per activity line (NDJSON) instead of the formatted `[time] message` text — there's no single final payload to wait for on a live stream. `scraps delete`/`banner --json` print `{"deleted":true,"id"}` / the raw upload response.
71
84
 
72
85
  ### Scrap accounts
73
86
 
74
87
  ```
75
- trawl scraps account set <id> [-u <username>] [-p <password>]
76
- trawl scraps account delete <id> [--force]
77
- trawl scraps account clear-session <id>
88
+ trawl scraps account set <id> [-u <username>] [-p <password>] [--json]
89
+ trawl scraps account delete <id> [--force] [--json]
90
+ trawl scraps account clear-session <id> [--json]
78
91
  trawl scraps account status <id> [--json]
79
- trawl scraps account session set <id> -c <file>
92
+ trawl scraps account session set <id> -c <file> [--json]
80
93
  ```
81
94
 
82
- `account session set` uploads a Puppeteer cookie JSON array to bootstrap a logged-in session without storing credentials (BYO-cookies).
95
+ `account session set` uploads a Puppeteer cookie JSON array to bootstrap a logged-in session without storing credentials (BYO-cookies). A missing `-u/--username`/`-p/--password` on `account set` follows the same [non-interactive rule](#non-interactive-rule) as `login`.
83
96
 
84
97
  ### Claude Code skills
85
98
 
86
99
  The CLI bundles 5 Claude Code skills that teach Claude how to use `trawl`. Once installed, Claude can manage scraps for you via prompts.
87
100
 
88
101
  ```
89
- trawl skills list List bundled skills and install status
90
- trawl skills install [<skill>] [--local] [--force] Install all (or one). Default: ~/.claude/skills/
91
- trawl skills uninstall [<skill>] [--local] Remove
92
- trawl skills update [<skill>] [--local] [--force] Reinstall (force sync with CLI version)
102
+ trawl skills list [--json] List bundled skills and install status
103
+ trawl skills install [<skill>] [--local] [--force] [--json] Install all (or one). Default: ~/.claude/skills/
104
+ trawl skills uninstall [<skill>] [--local] [--json] Remove
105
+ trawl skills update [<skill>] [--local] [--force] [--json] Reinstall (force sync with CLI version)
93
106
  ```
94
107
 
108
+ `--json` prints a structured result (`{"version","skills":[...]}` for `list`; `{"installed"/"uninstalled"/"updated":[{"name","scope","dest"}]}` for the others) instead of the `✓`-prefixed lines — the orphan-sweep/re-sync stderr lines are unaffected either way (stdout stays pure).
109
+
95
110
  Skills auto-update when you upgrade the CLI — no need to re-install manually — but it is not silent: it prints an honest `trawl: re-synced skill "<name>" (<scope>) <old> → <new>` line to stderr whenever it rewrites a skill dir, so a rewrite is never invisible. If a bundled skill is renamed or dropped between CLI versions, the old install is swept too — a `trawl: removed orphaned skill "<name>" (<scope>) — no longer bundled with this CLI version` line to stderr, so a stale skill teaching outdated CLI usage never lingers silently. Only marker-owned dirs (installed by trawl itself) are ever touched by either line. Opt out with `TRAWL_SKILLS_SYNC=0`.
96
111
 
97
112
  A pre-existing skill directory that trawl did not install itself (no `.version` marker) is never touched — `install`/`update` refuse to overwrite it and require `--force` to proceed. This applies to the CLI-upgrade auto-sync too (it silently skips marker-less dirs rather than refusing, since there is no interactive user to show a refusal to).
@@ -101,9 +116,9 @@ You can also install skills standalone (without the CLI): `npx @trawlme/skills i
101
116
  ### Telemetry
102
117
 
103
118
  ```
104
- trawl telemetry on Enable usage telemetry (default)
105
- trawl telemetry off Disable usage telemetry
106
- trawl telemetry status Show current state, telemetry ID, and opt-out instructions
119
+ trawl telemetry on [--json] Enable usage telemetry (default)
120
+ trawl telemetry off [--json] Disable usage telemetry
121
+ trawl telemetry status [--json] Show current state, telemetry ID, and opt-out instructions
107
122
  ```
108
123
 
109
124
  ## Telemetry
@@ -139,6 +154,15 @@ TRAWL_TELEMETRY=0 trawl scraps list
139
154
 
140
155
  **Why:** usage data helps us prioritise CLI features and catch silent errors before users report them.
141
156
 
157
+ ## Non-interactive rule
158
+
159
+ Every command that would otherwise block on a `readline` prompt (a destructive-action `[y/N]` confirmation, or a required value like `login`'s email/password) follows the same rule: when `--json` is set, **or** stdin/stdout isn't a real TTY (a pipe, a CI runner, an agent driving this CLI as a subprocess), it **never prompts** — it reports a structured usage error (exit `2`) instead of hanging forever waiting for an answer that can't arrive.
160
+
161
+ - **Confirmations** (`scraps delete`/`rm`, `scraps account delete`): pass `-f`/`--force` to pre-confirm and skip the prompt entirely, interactive or not.
162
+ - **Required values with no flag equivalent for "skip"** (`login`'s email/password, `scraps account set`'s username/password): pass the flag (`-e`/`-p`, `-u`/`-p`) or set `TRAWL_TOKEN`, instead of relying on the prompt.
163
+ - The guard is a hard gate, not a timeout — an agent-spawned subprocess with an open-but-idle stdin (common when a harness doesn't explicitly close it) would otherwise hang indefinitely; this refuses immediately instead.
164
+ - **Behavior change:** a piped `y` (e.g. `echo y | trawl scraps delete <id>`) no longer confirms the deletion. Piped stdin isn't a real TTY, so this now hits the same non-interactive refusal as any other scripted invocation (exit `2`, fail-closed) instead of silently proceeding on whatever text happened to be piped in. Existing scripts that relied on `echo y | …` must pass `-f`/`--force` instead.
165
+
142
166
  ## Exit codes
143
167
 
144
168
  Every command exits with one of these codes — scripts and agents driving the CLI unattended can branch on the exact failure kind instead of a uniform pass/fail:
@@ -146,13 +170,13 @@ Every command exits with one of these codes — scripts and agents driving the C
146
170
  | Code | Meaning |
147
171
  |------|--------------------------------------------------------------------------|
148
172
  | `0` | Success |
149
- | `1` | Unknown/generic error (an unmapped failure — API errors other than 401/404, an unhandled bug — or a business-logic refusal like `scraps data`'s `run_failed`/`in_progress` states) |
150
- | `2` | Usage error (bad flag/value, invalid ID, missing required argument, unknown option/command) |
173
+ | `1` | Unknown/generic error (an unmapped failure — API errors other than 401/404, an unhandled bug — or a business-logic refusal like `scraps data`'s `run_failed`/`in_progress` states, or `trawl fetch`'s honest `status:'failed'`/`'blocked'` outcome). **Known overload:** `fetch`'s domain-level failure and an arbitrary unmapped bug both land on `1` — a script needs to read the `--json` payload's own `status`/`error` field (for `fetch`) to tell them apart; this is intentional (the REST contract's outcome states are a separate axis from the CLI's transport-error taxonomy) and documented here rather than "resolved" by inventing a new code that would only apply to one command. |
174
+ | `2` | Usage error (bad flag/value, invalid ID, missing required argument, unknown option/command — including the [non-interactive rule](#non-interactive-rule) refusing to prompt) |
151
175
  | `3` | Auth error (not logged in, or the session token is expired/invalid — run `trawl login`) |
152
176
  | `4` | Not found (no such resource, or — for `scraps data` — no persisted payload to read) |
153
177
  | `5` | Network error (the API host is unreachable, DNS/connection/TLS failure, or the request timed out) |
154
178
 
155
- Under `--json`, a failing command emits a single error envelope on stdout — `{"error":{"message","status?","kind"}}` — instead of prose; the human-readable line always goes to stderr, never stdout.
179
+ Under `--json`, a failing command emits a single error envelope on stdout — `{"error":{"message","status?","kind"}}` — instead of prose; the human-readable line always goes to stderr, never stdout. `kind` is the machine-readable discriminant (`"usage"`/`"auth"`/`"not_found"`/`"network"`/`"api"`/`"refused"`/`"unknown"`/…) — the short string a script should switch on (`"refused"` is a business-logic refusal the server explicitly reported back, e.g. `scraps create|update`'s tier-ceiling override rejection — distinct from `"unknown"`, which stays reserved for an unmapped bug); `status` is present only when a real HTTP response carried one (never fabricated for a local failure like an expired-locally JWT or a refused confirmation prompt).
156
180
 
157
181
  ## Environment variables
158
182
 
@@ -0,0 +1,31 @@
1
+ import { Command } from 'commander';
2
+ /**
3
+ * `POST /api/scraps/fetch-url` response contract (#1697, trawl_node —
4
+ * shipped, contract LOCKED). REST counterpart of the MCP `trawl_fetch_url`
5
+ * tool: both run through the same shared engine
6
+ * (scraps.fetchUrl.service.js#runEphemeralUrlFetch), so the outcome shape is
7
+ * identical regardless of transport.
8
+ *
9
+ * - `status`: 'completed' | 'failed' | 'empty' | 'blocked' — an HONEST
10
+ * outcome, not a bare "it ran". A failed/blocked/empty run is still a
11
+ * 200 response (this is not an HTTP error), so callers must branch on
12
+ * `status`, never assume 2xx means "got data".
13
+ * - `error` is present ONLY when `status === 'failed'`.
14
+ * - `truncated: true` when the 100KB response-payload cap tripped — the
15
+ * server drops `result` in that case and repoints `url` at the full
16
+ * history row (`/api/historys/:runId`) instead of echoing the fetched
17
+ * target.
18
+ */
19
+ export interface FetchUrlResponse {
20
+ url: string;
21
+ status: 'completed' | 'failed' | 'empty' | 'blocked';
22
+ runId?: string | null;
23
+ statusDetail?: string | null;
24
+ blocked?: boolean;
25
+ length?: number | null;
26
+ result?: unknown;
27
+ truncated?: boolean;
28
+ reason?: string;
29
+ error?: string;
30
+ }
31
+ export declare const fetchUrl: Command;
@@ -0,0 +1,95 @@
1
+ import { Command } from 'commander';
2
+ import chalk from 'chalk';
3
+ import { oraPromise } from 'ora';
4
+ import { api, LONG_RUN_TIMEOUT_MS } from '../lib/api.js';
5
+ import { json } from '../lib/format.js';
6
+ import { requireUrl } from '../lib/validate.js';
7
+ function statusIcon(status) {
8
+ if (status === 'completed')
9
+ return chalk.green('✓');
10
+ if (status === 'failed')
11
+ return chalk.red('✗');
12
+ if (status === 'blocked')
13
+ return chalk.yellow('⚠');
14
+ return chalk.dim('•'); // empty
15
+ }
16
+ /** Best-effort human summary of `result` — same "count + first-item keys"
17
+ * shape scraps.ts's renderScrapItems uses, never a full dump (that's what
18
+ * --json is for). */
19
+ function renderResultSummary(result) {
20
+ if (!Array.isArray(result))
21
+ return;
22
+ console.log(chalk.dim(` Items: `) + result.length);
23
+ const first = result[0];
24
+ if (result.length > 0 && first && typeof first === 'object') {
25
+ console.log(chalk.dim(` First keys: `) + Object.keys(first).join(', '));
26
+ }
27
+ }
28
+ export const fetchUrl = new Command('fetch')
29
+ .description('One-shot fetch + extract readable content from a public URL (no scrap needed)')
30
+ .argument('<url>', 'Target public HTTPS URL')
31
+ .option('--json', 'Output the raw API payload')
32
+ .option('--reason <reason>', 'Audit-trail reason for this fetch (logged server-side, max 500 chars)')
33
+ .action(async (rawUrl, opts) => {
34
+ // Fast, local usage-error (exit 2) on an obviously malformed URL — never
35
+ // a round-trip to the server for something we can already tell is bad.
36
+ // The server still re-validates (SSRF guard) — this is a UX fast-path,
37
+ // not a security boundary.
38
+ const url = requireUrl(rawUrl, 'url');
39
+ const body = {
40
+ url,
41
+ ...(opts.reason !== undefined && { reason: opts.reason }),
42
+ };
43
+ // #106 review F1 — this endpoint runs the FULL worker pipeline (browser
44
+ // `goto` networkidle2 + extraction + persistence), legitimately 30-250s
45
+ // server-side — same #91 P0 pattern as `scraps run` / `data --fresh` /
46
+ // `trigger --wait` (src/commands/scraps.ts). The 30s DEFAULT_TIMEOUT_MS
47
+ // was aborting it mid-flight and would have surfaced a fabricated
48
+ // NetworkError timeout for a request that was always going to succeed.
49
+ const call = () => api.post('/api/scraps/fetch-url', body, { timeoutMs: LONG_RUN_TIMEOUT_MS });
50
+ // #106 review F2 — under --json the stdout path must be provably pure:
51
+ // no spinner channel at all. Only the human path gets the ora progress
52
+ // indicator; --json calls the API directly.
53
+ const data = opts.json
54
+ ? await call()
55
+ : await oraPromise(call, {
56
+ text: `Fetching ${url}…`,
57
+ // #106 review F3 — no successText verdict here. ora's success
58
+ // symbol only means "the HTTP call didn't throw", not "the fetch
59
+ // succeeded" — a failed/blocked domain `status` is still a 200
60
+ // response. The real outcome is rendered below via statusIcon +
61
+ // the Status: line; a green check here would contradict a
62
+ // red/yellow icon printed right after it.
63
+ successText: 'Request complete',
64
+ });
65
+ if (opts.json) {
66
+ json(data);
67
+ }
68
+ else {
69
+ console.log(`${statusIcon(data.status)} ${chalk.bold(data.url)}`);
70
+ console.log(chalk.dim(` Status: `) + data.status);
71
+ if (data.runId)
72
+ console.log(chalk.dim(` Run ID: `) + data.runId);
73
+ if (data.statusDetail)
74
+ console.log(chalk.dim(` Detail: `) + data.statusDetail);
75
+ if (data.length != null)
76
+ console.log(chalk.dim(` Length: `) + data.length);
77
+ if (data.truncated) {
78
+ console.log(chalk.yellow(` ⚠ Truncated (payload too large) — full result: ${data.url}`));
79
+ }
80
+ if (data.status === 'failed' && data.error) {
81
+ console.log(chalk.red(` Error: `) + data.error);
82
+ }
83
+ renderResultSummary(data.result);
84
+ }
85
+ // Honest exit code alongside the honest payload — a --json caller gets
86
+ // the raw body regardless (never wrapped/altered), but a script checking
87
+ // the exit code alone must be able to tell "no usable data" from "ran
88
+ // fine" without parsing. `blocked` (#106 review F4 — an antibot wall) is
89
+ // a failure to get data exactly like `failed`: an agent scripting
90
+ // `trawl fetch ... || handle` must see non-zero for either. `empty`
91
+ // stays 0 on purpose — the fetch genuinely ran to completion, there was
92
+ // just nothing extractable at that URL; that's not an error.
93
+ if (data.status === 'failed' || data.status === 'blocked')
94
+ process.exitCode = 1;
95
+ });
@@ -4,6 +4,8 @@ import config, { getApiUrl } from '../lib/config.js';
4
4
  import { api } from '../lib/api.js';
5
5
  import { requireFreshJwt, requireUrl } from '../lib/validate.js';
6
6
  import { promptPassword } from '../lib/prompt.js';
7
+ import { requireInteractive } from '../lib/confirm.js';
8
+ import { json } from '../lib/format.js';
7
9
  async function promptEmail() {
8
10
  const { createInterface } = await import('readline');
9
11
  const rl = createInterface({ input: process.stdin, output: process.stdout });
@@ -25,12 +27,26 @@ async function promptEmail() {
25
27
  rl.close();
26
28
  }
27
29
  }
30
+ /** #107 — the shared success-reporting tail for all three login paths (env
31
+ * token, --token flag, email/password). Under --json, prints a single
32
+ * structured result on stdout instead of the chalk lines — never the raw
33
+ * token itself (that's what `trawl token` is for). */
34
+ function reportLoginSuccess(opts, email) {
35
+ if (opts.json) {
36
+ json({ ok: true, apiUrl: getApiUrl(), config: config.path, ...(email && { email }) });
37
+ return;
38
+ }
39
+ console.log(chalk.green(email ? `✓ Logged in as ${email}` : '✓ Logged in'));
40
+ console.log(chalk.dim(` API: ${getApiUrl()}`));
41
+ console.log(chalk.dim(` Config: ${config.path}`));
42
+ }
28
43
  export const login = new Command('login')
29
44
  .description('Authenticate with the Trawl API')
30
45
  .option('-u, --url <url>', 'API base URL')
31
46
  .option('-t, --token <token>', 'JWT token (for CI use only)')
32
47
  .option('-e, --email <email>', 'Email address')
33
48
  .option('-p, --password <password>', 'Password (CI only — visible in process list and shell history)')
49
+ .option('--json', 'Output as JSON')
34
50
  .action(async (opts) => {
35
51
  // Validate --url but do NOT persist it yet. A failed signin must not
36
52
  // brick the config by pointing it at an unreachable/wrong host while the
@@ -47,23 +63,28 @@ export const login = new Command('login')
47
63
  if (envToken) {
48
64
  config.set('token', requireFreshJwt(envToken, 'TRAWL_TOKEN'));
49
65
  persistUrlIfPending();
50
- console.log(chalk.green('✓ Logged in'));
51
- console.log(chalk.dim(` API: ${getApiUrl()}`));
52
- console.log(chalk.dim(` Config: ${config.path}`));
66
+ reportLoginSuccess(opts);
53
67
  return;
54
68
  }
55
69
  // --token flag: direct JWT (CI / retrocompat)
56
70
  if (opts.token) {
57
71
  config.set('token', requireFreshJwt(opts.token, '--token'));
58
72
  persistUrlIfPending();
59
- console.log(chalk.green('✓ Logged in'));
60
- console.log(chalk.dim(` API: ${getApiUrl()}`));
61
- console.log(chalk.dim(` Config: ${config.path}`));
73
+ reportLoginSuccess(opts);
62
74
  return;
63
75
  }
64
76
  // Email/password flow
65
77
  if (opts.password) {
66
- console.warn(chalk.yellow('Warning: passing --password on the command line is insecure.'));
78
+ console.error(chalk.yellow('Warning: passing --password on the command line is insecure.'));
79
+ }
80
+ // #107 — never blocks on a readline prompt under --json or a non-TTY
81
+ // invocation (agent/CI subprocess); refuses with a clear, structured
82
+ // usage error instead, naming the flag (or TRAWL_TOKEN) to pass.
83
+ if (!opts.email) {
84
+ requireInteractive('Email is required — pass -e/--email, --token, or set TRAWL_TOKEN (refusing to block on a prompt, non-interactive).', { json: opts.json });
85
+ }
86
+ if (!opts.password) {
87
+ requireInteractive('Password is required — pass -p/--password, --token, or set TRAWL_TOKEN (refusing to block on a prompt, non-interactive).', { json: opts.json });
67
88
  }
68
89
  const email = opts.email ?? (await promptEmail());
69
90
  const password = opts.password ?? (await promptPassword('Password: '));
@@ -82,13 +103,16 @@ export const login = new Command('login')
82
103
  const token = requireFreshJwt(raw, 'token');
83
104
  config.set('token', token);
84
105
  persistUrlIfPending();
85
- console.log(chalk.green(`✓ Logged in as ${email}`));
86
- console.log(chalk.dim(` API: ${getApiUrl()}`));
87
- console.log(chalk.dim(` Config: ${config.path}`));
106
+ reportLoginSuccess(opts, email);
88
107
  });
89
108
  export const logout = new Command('logout')
90
109
  .description('Clear stored credentials')
91
- .action(() => {
110
+ .option('--json', 'Output as JSON')
111
+ .action((opts) => {
92
112
  config.set('token', '');
113
+ if (opts.json) {
114
+ json({ ok: true });
115
+ return;
116
+ }
93
117
  console.log(chalk.green('✓ Logged out'));
94
118
  });
@@ -0,0 +1,24 @@
1
+ import { Command } from 'commander';
2
+ /**
3
+ * `GET /api/health` response shape (home.controller.js#health,
4
+ * home.service.js#getHealthStatus) — MCP `trawl_health_ping` parity
5
+ * (`{ok, version, uptime}`). The REST route enriches the payload with
6
+ * `version`/`uptime`/`db`/`memory` ONLY for an admin caller
7
+ * (home.controller.js's `isAdmin` gate); a non-admin JWT gets `{status:'ok'}`
8
+ * alone on a healthy server. A non-2xx response (degraded, `status:503`)
9
+ * throws an ApiError like any other failed request — but a 2xx response is
10
+ * NOT a guarantee that `status === 'ok'` (#106 review F5): a soft-degraded
11
+ * signal (e.g. a partial dependency outage) could still come back as a 200
12
+ * with a non-"ok" `status`. The HTTP call not throwing only means the
13
+ * transport succeeded, never that the reported health is good — so the
14
+ * human-mode `✓ OK` line is gated on the actual field, not assumed from a
15
+ * successful round-trip.
16
+ */
17
+ export interface PingResponse {
18
+ status: string;
19
+ db?: string;
20
+ uptime?: number;
21
+ version?: string;
22
+ memory?: unknown;
23
+ }
24
+ export declare const ping: Command;
@@ -0,0 +1,29 @@
1
+ import { Command } from 'commander';
2
+ import chalk from 'chalk';
3
+ import { api } from '../lib/api.js';
4
+ import { json } from '../lib/format.js';
5
+ export const ping = new Command('ping')
6
+ .description('Health/version handshake against the Trawl API')
7
+ .option('--json', 'Output as JSON')
8
+ .action(async (opts) => {
9
+ const data = await api.get('/api/health');
10
+ // #106 review (kimi) — honest exit code: a soft-degraded 200 (status !== 'ok')
11
+ // must exit non-zero so `trawl ping || handle_degraded` doesn't treat a
12
+ // degraded API as healthy. Applies in BOTH --json and human modes, alongside
13
+ // the raw/decorated output below.
14
+ if (data.status !== 'ok')
15
+ process.exitCode = 1;
16
+ if (opts.json) {
17
+ json(data);
18
+ return;
19
+ }
20
+ const versionSuffix = data.version ? ` (v${data.version})` : '';
21
+ if (data.status === 'ok') {
22
+ console.log(chalk.green('✓ OK') + versionSuffix);
23
+ }
24
+ else {
25
+ // #106 review F5 — never green-light a degraded API just because the
26
+ // HTTP call didn't throw; show the real reported status instead.
27
+ console.log(chalk.yellow(`⚠ ${data.status}`) + versionSuffix);
28
+ }
29
+ });
@@ -17,6 +17,17 @@ interface BeforeRunState {
17
17
  alreadyInFlight: boolean;
18
18
  captured: boolean;
19
19
  }
20
+ /** The machine-readable outcome `pollRunProgress` prints as the single final
21
+ * NDJSON line under `--json` (#107 review F1). `status` is the same honest
22
+ * terminal string the human-mode "Run finished: <status>" line already
23
+ * shows (`success`/`error`/`empty`/`regression`/…), or `'timeout'` /
24
+ * `'poll_error'` for the two non-terminal exits. */
25
+ export interface PollOutcome {
26
+ runId?: string;
27
+ status: string;
28
+ /** Present only for `status:'poll_error'` — the last poll failure's message. */
29
+ error?: string;
30
+ }
20
31
  /**
21
32
  * #91 P1 — replaces "await the run to completion, THEN open the activities
22
33
  * SSE stream" (which showed NOTHING: the activities SSE
@@ -78,9 +89,30 @@ interface BeforeRunState {
78
89
  * this function has, so it resolves to the same honest timeout rather than
79
90
  * risk reporting a possibly-wrong outcome (never a lie, at worst a timeout
80
91
  * telling the caller to check `doctor`).
92
+ *
93
+ * #107 review F1 — before this fix, `run|trigger --json --watch` was
94
+ * outcome-blind: `quiet` suppressed ALL output (including "Run finished:
95
+ * failure" and the timeout notice), a transient poll error was caught and
96
+ * silently retried FOREVER within the deadline, and the process always
97
+ * exited 0 after the poll loop regardless of what the watched run actually
98
+ * did — dead air, then a clean exit code, even for a failed or timed-out
99
+ * run. An agent scripting this CLI had no way to tell success from failure
100
+ * from "we gave up". Fixed by:
101
+ * - emitting exactly ONE final NDJSON line on stdout under `--json` once
102
+ * the watch reaches ANY of its three exits (terminal status, timeout, or
103
+ * a persistent poll error) — `{runId,status}` (+`error` for a poll
104
+ * error) — while every intermediate progress line stays suppressed
105
+ * (unchanged from before);
106
+ * - setting `process.exitCode` non-zero on a genuine run failure, a
107
+ * timeout, or a persistent poll error, and `0` on a real success — in
108
+ * BOTH `--json` and human `--watch` modes (human mode used to exit 0
109
+ * unconditionally, the same bug, just silent instead of dishonest);
110
+ * - giving up after `MAX_CONSECUTIVE_POLL_ERRORS` consecutive failed reads
111
+ * instead of retrying the same dead endpoint for the full 300s.
81
112
  */
82
113
  export declare function pollRunProgress(id: string, before: BeforeRunState | undefined, opts?: {
83
114
  intervalMs?: number;
84
115
  timeoutMs?: number;
116
+ json?: boolean;
85
117
  }): Promise<void>;
86
118
  export {};