@trawlme/cli 1.20.0 → 1.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +74 -51
- package/dist/commands/login.js +35 -11
- package/dist/commands/scraps.d.ts +61 -0
- package/dist/commands/scraps.js +635 -437
- package/dist/commands/skills.js +52 -6
- package/dist/commands/telemetry.js +23 -3
- package/dist/commands/token.js +9 -3
- package/dist/index.d.ts +8 -0
- package/dist/index.js +45 -7
- package/dist/lib/confirm.d.ts +70 -0
- package/dist/lib/confirm.js +79 -0
- package/dist/lib/errors.d.ts +13 -0
- package/dist/lib/errors.js +19 -0
- package/docs/agent-quickstart.md +103 -0
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -8,6 +8,8 @@ Command-line client for [Trawl](https://trawl.me) — manage your scraps from th
|
|
|
8
8
|
|
|
9
9
|
> ⚠️ **Early stage.** APIs may change before `1.0`. Pin a version if you depend on this in CI.
|
|
10
10
|
|
|
11
|
+
> 🤖 **Driving this CLI from an AI agent?** See [docs/agent-quickstart.md](docs/agent-quickstart.md) for the minimal surface (`fetch`, `--json`, `TRAWL_TOKEN`, exit codes) — this README is the full human/CI guide.
|
|
12
|
+
|
|
11
13
|
## Install
|
|
12
14
|
|
|
13
15
|
Requires Node.js >= 20.
|
|
@@ -21,99 +23,111 @@ npm install -g @trawlme/cli
|
|
|
21
23
|
Three methods:
|
|
22
24
|
|
|
23
25
|
1. **Interactive** — `trawl login` (prompts for email and password)
|
|
24
|
-
2. **Env var** — `TRAWL_TOKEN=<jwt> trawl
|
|
26
|
+
2. **Env var** — `TRAWL_TOKEN=<jwt> trawl list` (CI/CD, bypasses prompt)
|
|
25
27
|
3. **Token flag** — `trawl login --token <jwt>` (CI/CD, direct JWT)
|
|
26
28
|
|
|
27
29
|
Custom API URL: `trawl login --url https://self-hosted.example.com`
|
|
28
30
|
|
|
29
31
|
## Commands
|
|
30
32
|
|
|
31
|
-
|
|
33
|
+
`trawl --help` groups commands into two labeled tiers, in this exact wording: **Core commands (agent + human)** (listed first — `--json` first-class, never interactive) and **Management commands (human/CI)** (grouped below — the full scrap-admin/account/auth/skills/telemetry surface). Commander also prints a small ungrouped `help [command]` entry at the bottom, so "two tiers" describes the labeled sections, not literally every line of `--help` output. This section documents both tiers, in that order; [docs/agent-quickstart.md](docs/agent-quickstart.md) covers just the Core tier for an agent integration.
|
|
32
34
|
|
|
33
|
-
|
|
35
|
+
All commands accept a global `--debug` flag to show full error stack traces on failure. Every command below supports `--json` (a raw payload / structured result on stdout, nothing else) and never blocks on an interactive prompt when `--json` is set or stdin/stdout isn't a real TTY — see [Non-interactive rule](#non-interactive-rule) and [Exit codes](#exit-codes). Two documented exceptions to "single payload": `scraps watch --json` is a live stream, so it emits NDJSON (one JSON object per line) instead; `scraps snapshot --json` only changes behavior on a never-run scrap (`{"status":"no_runs"}`) — when a run exists it still writes the raw HTML snapshot regardless of `--json` (there's no JSON-encoded form of an HTML page to emit).
|
|
36
|
+
|
|
37
|
+
### Core commands (agent + human)
|
|
34
38
|
|
|
35
39
|
```
|
|
36
|
-
trawl
|
|
37
|
-
trawl
|
|
38
|
-
trawl
|
|
40
|
+
trawl fetch <url> [--json] [--reason <text>] One-shot fetch + extract readable content from a public URL (no scrap needed)
|
|
41
|
+
trawl run <id> [--watch] [--json] Run a scrap
|
|
42
|
+
trawl list|ls [--json] [--status <success|failure|never|running|regression>] [--limit <n>] [--page <n>]
|
|
43
|
+
trawl get <id> [--json] Get scrap details
|
|
44
|
+
trawl data <id> [--json] [--fresh] [--errors] Get scrap data (last persisted run — read-only, no quota; --fresh to launch one)
|
|
45
|
+
trawl history <id> [--json] [-n <limit>] List past runs for a scrap (newest first)
|
|
46
|
+
trawl run-info <hid> [--json] Show details of a single run
|
|
47
|
+
trawl trigger <id> [--watch] [--wait] [--json] Launch a scrap as a background worker (returns immediately)
|
|
48
|
+
trawl whoami [--json] Show the authenticated user's identity
|
|
49
|
+
trawl ping [--json] Health/version handshake against the Trawl API
|
|
39
50
|
```
|
|
40
51
|
|
|
41
|
-
|
|
52
|
+
> **No breaking change:** every verb above is also still reachable under its pre-reorg path, `trawl scraps <verb>` (e.g. `trawl scraps list`, `trawl scraps run <id>`) — kept as a hidden alias so scripts written before the surface reorg keep working. `trawl --help` only shows the top-level form above; `trawl scraps --help` only shows the remaining scrap-management commands below.
|
|
42
53
|
|
|
43
|
-
|
|
44
|
-
trawl fetch <url> [--json] [--reason <text>] One-shot fetch + extract readable content from a public URL (no scrap needed)
|
|
45
|
-
trawl whoami [--json] Show the authenticated user's identity
|
|
46
|
-
trawl ping [--json] Health/version handshake against the Trawl API
|
|
47
|
-
```
|
|
54
|
+
`fetch`/`whoami`/`ping` are fully non-interactive — all three read auth only from `TRAWL_TOKEN`/the stored login token, never prompt. `trawl fetch` is the REST counterpart of the MCP `trawl_fetch_url` tool (same shared engine, `POST /api/scraps/fetch-url`): `status` is an honest outcome (`completed`/`failed`/`empty`/`blocked`), not "did the HTTP call succeed" — a failed fetch is still a 200 response with `status:'failed'` + `error`, and the CLI exits 1 in that case (both human and `--json` modes) even though `--json` always prints the raw payload verbatim. `truncated:true` means the 100KB response cap tripped; `result` is dropped and `url` repoints at the full history row instead. `trawl whoami`/`trawl ping` mirror the MCP `trawl_whoami`/`trawl_health_ping` tools as closely as the REST surface allows (`GET /api/users/me` / `GET /api/health`) — `ping`'s `--json` payload is admin-enriched (version/uptime/db) and just `{"status":"ok"}` for anyone else.
|
|
48
55
|
|
|
49
|
-
|
|
56
|
+
- `list` has a short alias, `ls` (matches `trawl --help`'s `list|ls`).
|
|
57
|
+
- `history` lists past runs (newest first); `run-info <hid>` shows details of a single run from that history.
|
|
58
|
+
- `data` returns the last persisted run payload (no execute quota); `--fresh` runs the scrap live instead (consumes execute quota); `--errors` shows the last run's error detail (`--json` on a never-run scrap returns `{"status":"no_runs"}`, exit 0, matching `scraps doctor --json`). `[]` on stdout means a genuine zero-item successful run — a scrap that has never run, whose last run failed, or whose payload aged out of retention returns a `--json` error envelope (exit 4/1/4 respectively) instead. Two more honest states: a run still **in flight** (`status: null` server-side) returns a `kind:"in_progress"` error envelope (exit 1, "retry shortly" — never suggests `--fresh`, which would just 429 against the run already holding the lock); a run whose item count **regressed** vs baseline (`statusDetail: "regression"`) still returns the real, non-empty items on stdout (exit 0) plus a stderr warning pointing at `scraps doctor <id>` — the data itself is genuine even though the run is flagged.
|
|
59
|
+
- `get` (and anything reading through it, like `data`'s default path) embeds only the newest 100 history rows on the returned scrap object — `run-info` and `scraps doctor` fetch a single run directly and are unaffected by that cap. `list`/`get` show a distinct amber `▼` "regression" badge, never the red `✗` a genuine failure gets (matches `scraps doctor`'s own badge).
|
|
60
|
+
- **Long-running scrap execute (`run`, `data --fresh`, `trigger --wait`):** these hit the same server-side scrap-execute path, which can legitimately take 30–250s (proxy tier escalation, AI-fix retries) — the CLI arms a 300s timeout for exactly these three call sites instead of the generic 30s default. `TRAWL_TIMEOUT` (see below) still overrides ALL requests, including these — set it if you need a tighter or looser ceiling than 300s for a long-running scrap.
|
|
61
|
+
- **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `run --watch` and `trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears. The watched run's outcome drives the exit code too, in BOTH human and `--json` mode: a genuinely failed terminal run, a poll timeout, or a persistently unreachable API all exit non-zero — a clean successful run is the only exit `0`. A run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>` (human mode) — see the `--json` shape below. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
|
|
62
|
+
- `run --json`/`trigger --json` bypass the spinner and print the raw launch/trigger payload on stdout; combined with `--watch`, every intermediate progress line stays suppressed (stdout stays pure JSON) and, once the watch reaches its outcome, exactly ONE final NDJSON line is emitted: `{"runId","status"}` (the honest terminal status — `success`/`error`/`empty`/`regression`/…), `{"runId","status":"timeout"}` on a poll timeout, or `{"runId","status":"poll_error","error"}` if the API stays unreachable for several consecutive polls — `process.exitCode` is non-zero for all three except a genuine success. `scraps watch --json` emits one raw JSON object per activity line (NDJSON) instead of the formatted `[time] message` text — there's no single final payload to wait for on a live stream.
|
|
50
63
|
|
|
51
|
-
###
|
|
64
|
+
### Scrap management
|
|
52
65
|
|
|
53
66
|
```
|
|
54
|
-
trawl scraps list [--json] [--status <success|failure|never|running|regression>] [--limit <n>] [--page <n>]
|
|
55
|
-
trawl scraps get <id> [--json]
|
|
56
67
|
trawl scraps create -t <title> [-u <url>] [-r <request>] [-d <description>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--json]
|
|
57
68
|
trawl scraps update <id> [-t <title>] [-u <url>] [-r <request>] [-d <description>] [--cron <expr>|--no-cron] [--alert <email>|--no-alert] [--autofix|--no-autofix] [-p <json>|--params-file <path>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--force-tier <tier0|tier1|tier2|tier3|tier4>] [--json]
|
|
58
|
-
trawl scraps
|
|
59
|
-
trawl scraps trigger <id> [--watch] [--wait]
|
|
60
|
-
trawl scraps watch <id>
|
|
61
|
-
trawl scraps data <id> [--json] [--fresh] [--errors]
|
|
62
|
-
trawl scraps history <id> [--json] [-n <limit>]
|
|
63
|
-
trawl scraps run-info <hid> [--json]
|
|
69
|
+
trawl scraps watch <id> [--json]
|
|
64
70
|
trawl scraps doctor <id> [--json] [--autofix]
|
|
65
71
|
trawl scraps autofix <id> [--json]
|
|
66
72
|
trawl scraps snapshot <id> [--error] [-o <file>] [--json]
|
|
67
|
-
trawl scraps banner <id> -f <file>
|
|
68
|
-
trawl scraps delete <id> [--force] Alias: rm
|
|
73
|
+
trawl scraps banner <id> -f <file> [--json]
|
|
74
|
+
trawl scraps delete <id> [--force] [--json] Alias: rm
|
|
69
75
|
```
|
|
70
76
|
|
|
71
|
-
- `--tier` forces a proxy tier; `--force-tier` raises the proxy-tier ceiling past the auto-cap (history-gated: may be refused or cost more). `create --json`/`update --json` print the full scrap object (including the `_tierOverride` outcome) on stdout; a refused tier override exits 1 with a standard `--json` error envelope. When a tier was requested but the server's response carries no `_tierOverride` at all (an older server that can't confirm what actually got applied), a stderr warning is printed either way, and under `--json` the emitted object also carries `"_tierUnconfirmed": true` — the machine-readable counterpart to that warning, since a `--json` caller has no reliable reason to read stderr.
|
|
72
|
-
- `scraps doctor` diagnoses the last run (error, failed selector, block status, page state, autofix outcome); `--autofix` includes the full autofix diff/dry-run/knowledge. A run that is still in flight (`status: null`, server-side) shows a `running` badge, never `failed`; a run whose item count regressed vs baseline (`statusDetail: "regression"`) shows its own amber `regression` badge, never `failed` either
|
|
77
|
+
- `--tier` forces a proxy tier; `--force-tier` raises the proxy-tier ceiling past the auto-cap (history-gated: may be refused or cost more). `create --json`/`update --json` print the full scrap object (including the `_tierOverride` outcome) on stdout; a refused tier override exits 1 with a standard `--json` error envelope (`kind:"refused"` — distinct from `"unknown"`, so a script can branch on "the server said no"). When a tier was requested but the server's response carries no `_tierOverride` at all (an older server that can't confirm what actually got applied), a stderr warning is printed either way, and under `--json` the emitted object also carries `"_tierUnconfirmed": true` — the machine-readable counterpart to that warning, since a `--json` caller has no reliable reason to read stderr.
|
|
78
|
+
- `scraps doctor` diagnoses the last run (error, failed selector, block status, page state, autofix outcome); `--autofix` includes the full autofix diff/dry-run/knowledge. A run that is still in flight (`status: null`, server-side) shows a `running` badge, never `failed`; a run whose item count regressed vs baseline (`statusDetail: "regression"`) shows its own amber `regression` badge, never `failed` either.
|
|
73
79
|
- `scraps autofix` shows the last auto-fix attempt on its own (decision, diff, dry-run, knowledge). `--json` on a scrap that has **never run** returns `{"status":"no_runs"}` (exit 0) — distinct from `null`, which means a run exists but had no auto-fix attempt.
|
|
74
80
|
- `scraps snapshot --error` fetches the error-path snapshot instead of the normal one; `-o <file>` writes to a file instead of stdout. On a scrap that has **never run**: `--json` returns `{"status":"no_runs"}` (exit 0), matching `doctor`/`autofix`; `-o <file>` (without `--json`) exits 4 (`not_found`) instead of silently exiting 0 with nothing written — a script checking the exit code alone must be able to tell "no file was produced" from success. `--json` takes priority when both are passed.
|
|
75
|
-
- `scraps history` lists past runs (newest first); `scraps run-info <hid>` shows details of a single run from that history.
|
|
76
|
-
- `scraps data` returns the last persisted run payload (no execute quota); `--fresh` runs the scrap live instead (consumes execute quota); `--errors` shows the last run's error detail (`--json` on a never-run scrap returns `{"status":"no_runs"}`, exit 0, matching `doctor --json`). `[]` on stdout means a genuine zero-item successful run — a scrap that has never run, whose last run failed, or whose payload aged out of retention returns a `--json` error envelope (exit 4/1/4 respectively) instead. Two more honest states: a run still **in flight** (`status: null` server-side) returns a `kind:"in_progress"` error envelope (exit 1, "retry shortly" — never suggests `--fresh`, which would just 429 against the run already holding the lock); a run whose item count **regressed** vs baseline (`statusDetail: "regression"`) still returns the real, non-empty items on stdout (exit 0) plus a stderr warning pointing at `scraps doctor <id>` — the data itself is genuine even though the run is flagged.
|
|
77
|
-
- `scraps get` (and anything reading through it, like `scraps data`'s default path) embeds only the newest 100 history rows on the returned scrap object — `scraps run-info` and `scraps doctor` fetch a single run directly and are unaffected by that cap.
|
|
78
81
|
- `scraps banner -f <file>` only accepts `png`/`jpg`/`jpeg`/`webp`; any other extension is a usage error (exit 2) instead of silently uploading the file under a fabricated `image/png` Content-Type.
|
|
79
|
-
-
|
|
80
|
-
- **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `scraps run --watch` and `scraps trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears; a run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>`. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
|
|
82
|
+
- `scraps delete`/`banner --json` print `{"deleted":true,"id"}` / the raw upload response.
|
|
81
83
|
|
|
82
84
|
### Scrap accounts
|
|
83
85
|
|
|
84
86
|
```
|
|
85
|
-
trawl scraps account set <id> [-u <username>] [-p <password>]
|
|
86
|
-
trawl scraps account delete <id> [--force]
|
|
87
|
-
trawl scraps account clear-session <id>
|
|
87
|
+
trawl scraps account set <id> [-u <username>] [-p <password>] [--json]
|
|
88
|
+
trawl scraps account delete <id> [--force] [--json]
|
|
89
|
+
trawl scraps account clear-session <id> [--json]
|
|
88
90
|
trawl scraps account status <id> [--json]
|
|
89
|
-
trawl scraps account session set <id> -c <file>
|
|
91
|
+
trawl scraps account session set <id> -c <file> [--json]
|
|
90
92
|
```
|
|
91
93
|
|
|
92
|
-
`account session set` uploads a Puppeteer cookie JSON array to bootstrap a logged-in session without storing credentials (BYO-cookies).
|
|
94
|
+
`account session set` uploads a Puppeteer cookie JSON array to bootstrap a logged-in session without storing credentials (BYO-cookies). A missing `-u/--username`/`-p/--password` on `account set` follows the same [non-interactive rule](#non-interactive-rule) as `login`.
|
|
93
95
|
|
|
94
96
|
### Claude Code skills
|
|
95
97
|
|
|
96
98
|
The CLI bundles 5 Claude Code skills that teach Claude how to use `trawl`. Once installed, Claude can manage scraps for you via prompts.
|
|
97
99
|
|
|
98
100
|
```
|
|
99
|
-
trawl skills list List bundled skills and install status
|
|
100
|
-
trawl skills install [<skill>] [--local] [--force] Install all (or one). Default: ~/.claude/skills/
|
|
101
|
-
trawl skills uninstall [<skill>] [--local] Remove
|
|
102
|
-
trawl skills update [<skill>] [--local] [--force] Reinstall (force sync with CLI version)
|
|
101
|
+
trawl skills list [--json] List bundled skills and install status
|
|
102
|
+
trawl skills install [<skill>] [--local] [--force] [--json] Install all (or one). Default: ~/.claude/skills/
|
|
103
|
+
trawl skills uninstall [<skill>] [--local] [--json] Remove
|
|
104
|
+
trawl skills update [<skill>] [--local] [--force] [--json] Reinstall (force sync with CLI version)
|
|
103
105
|
```
|
|
104
106
|
|
|
107
|
+
`--json` prints a structured result (`{"version","skills":[...]}` for `list`; `{"installed"/"uninstalled"/"updated":[{"name","scope","dest"}]}` for the others) instead of the `✓`-prefixed lines — the orphan-sweep/re-sync stderr lines are unaffected either way (stdout stays pure).
|
|
108
|
+
|
|
105
109
|
Skills auto-update when you upgrade the CLI — no need to re-install manually — but it is not silent: it prints an honest `trawl: re-synced skill "<name>" (<scope>) <old> → <new>` line to stderr whenever it rewrites a skill dir, so a rewrite is never invisible. If a bundled skill is renamed or dropped between CLI versions, the old install is swept too — a `trawl: removed orphaned skill "<name>" (<scope>) — no longer bundled with this CLI version` line to stderr, so a stale skill teaching outdated CLI usage never lingers silently. Only marker-owned dirs (installed by trawl itself) are ever touched by either line. Opt out with `TRAWL_SKILLS_SYNC=0`.
|
|
106
110
|
|
|
107
111
|
A pre-existing skill directory that trawl did not install itself (no `.version` marker) is never touched — `install`/`update` refuse to overwrite it and require `--force` to proceed. This applies to the CLI-upgrade auto-sync too (it silently skips marker-less dirs rather than refusing, since there is no interactive user to show a refusal to).
|
|
108
112
|
|
|
109
113
|
You can also install skills standalone (without the CLI): `npx @trawlme/skills install`.
|
|
110
114
|
|
|
115
|
+
### Auth
|
|
116
|
+
|
|
117
|
+
```
|
|
118
|
+
trawl login [--url <url>] [--token <jwt>] [--email <email>] [--password <pass>] [--json]
|
|
119
|
+
trawl logout [--json]
|
|
120
|
+
trawl token [--json] Print the stored session JWT (for MCP Bearer auth)
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`login --json` prints `{"ok":true,"apiUrl","config",email?}` on success — never the raw token (that's `token`'s job). `token --json` prints `{"token","exp","expiresAt"}` instead of the bare JWT + stderr advisories. See [Non-interactive rule](#non-interactive-rule) below: a missing `--email`/`--password` refuses immediately (usage error, exit 2) instead of prompting when `--json` is set or stdin/stdout isn't a real TTY.
|
|
124
|
+
|
|
111
125
|
### Telemetry
|
|
112
126
|
|
|
113
127
|
```
|
|
114
|
-
trawl telemetry on Enable usage telemetry (default)
|
|
115
|
-
trawl telemetry off Disable usage telemetry
|
|
116
|
-
trawl telemetry status Show current state, telemetry ID, and opt-out instructions
|
|
128
|
+
trawl telemetry on [--json] Enable usage telemetry (default)
|
|
129
|
+
trawl telemetry off [--json] Disable usage telemetry
|
|
130
|
+
trawl telemetry status [--json] Show current state, telemetry ID, and opt-out instructions
|
|
117
131
|
```
|
|
118
132
|
|
|
119
133
|
## Telemetry
|
|
@@ -121,7 +135,7 @@ trawl telemetry status Show current state, telemetry ID, and opt-out instructi
|
|
|
121
135
|
`@trawlme/cli` collects anonymous usage data to help us improve the CLI based on real usage patterns. Telemetry is enabled by default.
|
|
122
136
|
|
|
123
137
|
**What we collect:**
|
|
124
|
-
- Command name (e.g. `scraps
|
|
138
|
+
- Command name (e.g. `list` — the #108 surface reorg renamed a promoted verb's telemetry event from `scraps <verb>` to `<verb>`; invoking the legacy `trawl scraps <verb>` path still emits the pre-reorg `scraps <verb>` name)
|
|
125
139
|
- CLI version
|
|
126
140
|
- Node.js version
|
|
127
141
|
- Platform (e.g. `darwin`, `linux`, `win32`)
|
|
@@ -144,11 +158,20 @@ trawl telemetry status Show current state, telemetry ID, and opt-out instructi
|
|
|
144
158
|
trawl telemetry off
|
|
145
159
|
|
|
146
160
|
# Session-level opt-out (env var, takes precedence)
|
|
147
|
-
TRAWL_TELEMETRY=0 trawl
|
|
161
|
+
TRAWL_TELEMETRY=0 trawl list
|
|
148
162
|
```
|
|
149
163
|
|
|
150
164
|
**Why:** usage data helps us prioritise CLI features and catch silent errors before users report them.
|
|
151
165
|
|
|
166
|
+
## Non-interactive rule
|
|
167
|
+
|
|
168
|
+
Every command that would otherwise block on a `readline` prompt (a destructive-action `[y/N]` confirmation, or a required value like `login`'s email/password) follows the same rule: when `--json` is set, **or** stdin/stdout isn't a real TTY (a pipe, a CI runner, an agent driving this CLI as a subprocess), it **never prompts** — it reports a structured usage error (exit `2`) instead of hanging forever waiting for an answer that can't arrive.
|
|
169
|
+
|
|
170
|
+
- **Confirmations** (`scraps delete`/`rm`, `scraps account delete`): pass `-f`/`--force` to pre-confirm and skip the prompt entirely, interactive or not.
|
|
171
|
+
- **Required values with no flag equivalent for "skip"** (`login`'s email/password, `scraps account set`'s username/password): pass the flag (`-e`/`-p`, `-u`/`-p`) or set `TRAWL_TOKEN`, instead of relying on the prompt.
|
|
172
|
+
- The guard is a hard gate, not a timeout — an agent-spawned subprocess with an open-but-idle stdin (common when a harness doesn't explicitly close it) would otherwise hang indefinitely; this refuses immediately instead.
|
|
173
|
+
- **Behavior change:** a piped `y` (e.g. `echo y | trawl scraps delete <id>`) no longer confirms the deletion. Piped stdin isn't a real TTY, so this now hits the same non-interactive refusal as any other scripted invocation (exit `2`, fail-closed) instead of silently proceeding on whatever text happened to be piped in. Existing scripts that relied on `echo y | …` must pass `-f`/`--force` instead.
|
|
174
|
+
|
|
152
175
|
## Exit codes
|
|
153
176
|
|
|
154
177
|
Every command exits with one of these codes — scripts and agents driving the CLI unattended can branch on the exact failure kind instead of a uniform pass/fail:
|
|
@@ -156,13 +179,13 @@ Every command exits with one of these codes — scripts and agents driving the C
|
|
|
156
179
|
| Code | Meaning |
|
|
157
180
|
|------|--------------------------------------------------------------------------|
|
|
158
181
|
| `0` | Success |
|
|
159
|
-
| `1` | Unknown/generic error (an unmapped failure — API errors other than 401/404, an unhandled bug — or a business-logic refusal like `
|
|
160
|
-
| `2` | Usage error (bad flag/value, invalid ID, missing required argument, unknown option/command) |
|
|
182
|
+
| `1` | Unknown/generic error (an unmapped failure — API errors other than 401/404, an unhandled bug — or a business-logic refusal like `data`'s `run_failed`/`in_progress` states, or `trawl fetch`'s honest `status:'failed'`/`'blocked'` outcome). **Known overload:** `fetch`'s domain-level failure and an arbitrary unmapped bug both land on `1` — a script needs to read the `--json` payload's own `status`/`error` field (for `fetch`) to tell them apart; this is intentional (the REST contract's outcome states are a separate axis from the CLI's transport-error taxonomy) and documented here rather than "resolved" by inventing a new code that would only apply to one command. |
|
|
183
|
+
| `2` | Usage error (bad flag/value, invalid ID, missing required argument, unknown option/command — including the [non-interactive rule](#non-interactive-rule) refusing to prompt) |
|
|
161
184
|
| `3` | Auth error (not logged in, or the session token is expired/invalid — run `trawl login`) |
|
|
162
|
-
| `4` | Not found (no such resource, or — for `
|
|
185
|
+
| `4` | Not found (no such resource, or — for `data` — no persisted payload to read) |
|
|
163
186
|
| `5` | Network error (the API host is unreachable, DNS/connection/TLS failure, or the request timed out) |
|
|
164
187
|
|
|
165
|
-
Under `--json`, a failing command emits a single error envelope on stdout — `{"error":{"message","status?","kind"}}` — instead of prose; the human-readable line always goes to stderr, never stdout.
|
|
188
|
+
Under `--json`, a failing command emits a single error envelope on stdout — `{"error":{"message","status?","kind"}}` — instead of prose; the human-readable line always goes to stderr, never stdout. `kind` is the machine-readable discriminant (`"usage"`/`"auth"`/`"not_found"`/`"network"`/`"api"`/`"refused"`/`"unknown"`/…) — the short string a script should switch on (`"refused"` is a business-logic refusal the server explicitly reported back, e.g. `scraps create|update`'s tier-ceiling override rejection — distinct from `"unknown"`, which stays reserved for an unmapped bug); `status` is present only when a real HTTP response carried one (never fabricated for a local failure like an expired-locally JWT or a refused confirmation prompt).
|
|
166
189
|
|
|
167
190
|
## Environment variables
|
|
168
191
|
|
|
@@ -173,13 +196,13 @@ Under `--json`, a failing command emits a single error envelope on stdout — `{
|
|
|
173
196
|
| `TRAWL_TELEMETRY` | Set to `0` to disable telemetry for the current session |
|
|
174
197
|
| `DO_NOT_TRACK` | Set to `1` to disable telemetry (cross-vendor convention, https://consoledonottrack.com) — same effect as `TRAWL_TELEMETRY=0` |
|
|
175
198
|
| `TRAWL_CONFIG_DIR` | Override where the config file (token, API URL, telemetry state) is stored — useful for hermetic CI runs or concurrent `trawl login`s that must not share one on-disk file |
|
|
176
|
-
| `TRAWL_TIMEOUT` | Override the per-request fetch timeout in milliseconds (default `30000`; `300000` for `
|
|
199
|
+
| `TRAWL_TIMEOUT` | Override the per-request fetch timeout in milliseconds (default `30000`; `300000` for `run`/`data --fresh`/`trigger --wait` — this env var always wins over that longer default too) |
|
|
177
200
|
| `TRAWL_SKILLS_SYNC` | Set to `0` to disable the startup skills auto-sync entirely |
|
|
178
201
|
|
|
179
202
|
Session override (no `trawl login` mutation, ideal for CI/QA against another env):
|
|
180
203
|
|
|
181
204
|
```bash
|
|
182
|
-
TRAWL_API_URL=https://dev.trawl.me TRAWL_TOKEN=<jwt> trawl
|
|
205
|
+
TRAWL_API_URL=https://dev.trawl.me TRAWL_TOKEN=<jwt> trawl list --json
|
|
183
206
|
```
|
|
184
207
|
|
|
185
208
|
## API documentation
|
package/dist/commands/login.js
CHANGED
|
@@ -4,6 +4,8 @@ import config, { getApiUrl } from '../lib/config.js';
|
|
|
4
4
|
import { api } from '../lib/api.js';
|
|
5
5
|
import { requireFreshJwt, requireUrl } from '../lib/validate.js';
|
|
6
6
|
import { promptPassword } from '../lib/prompt.js';
|
|
7
|
+
import { requireInteractive } from '../lib/confirm.js';
|
|
8
|
+
import { json } from '../lib/format.js';
|
|
7
9
|
async function promptEmail() {
|
|
8
10
|
const { createInterface } = await import('readline');
|
|
9
11
|
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
|
@@ -25,12 +27,26 @@ async function promptEmail() {
|
|
|
25
27
|
rl.close();
|
|
26
28
|
}
|
|
27
29
|
}
|
|
30
|
+
/** #107 — the shared success-reporting tail for all three login paths (env
|
|
31
|
+
* token, --token flag, email/password). Under --json, prints a single
|
|
32
|
+
* structured result on stdout instead of the chalk lines — never the raw
|
|
33
|
+
* token itself (that's what `trawl token` is for). */
|
|
34
|
+
function reportLoginSuccess(opts, email) {
|
|
35
|
+
if (opts.json) {
|
|
36
|
+
json({ ok: true, apiUrl: getApiUrl(), config: config.path, ...(email && { email }) });
|
|
37
|
+
return;
|
|
38
|
+
}
|
|
39
|
+
console.log(chalk.green(email ? `✓ Logged in as ${email}` : '✓ Logged in'));
|
|
40
|
+
console.log(chalk.dim(` API: ${getApiUrl()}`));
|
|
41
|
+
console.log(chalk.dim(` Config: ${config.path}`));
|
|
42
|
+
}
|
|
28
43
|
export const login = new Command('login')
|
|
29
44
|
.description('Authenticate with the Trawl API')
|
|
30
45
|
.option('-u, --url <url>', 'API base URL')
|
|
31
46
|
.option('-t, --token <token>', 'JWT token (for CI use only)')
|
|
32
47
|
.option('-e, --email <email>', 'Email address')
|
|
33
48
|
.option('-p, --password <password>', 'Password (CI only — visible in process list and shell history)')
|
|
49
|
+
.option('--json', 'Output as JSON')
|
|
34
50
|
.action(async (opts) => {
|
|
35
51
|
// Validate --url but do NOT persist it yet. A failed signin must not
|
|
36
52
|
// brick the config by pointing it at an unreachable/wrong host while the
|
|
@@ -47,23 +63,28 @@ export const login = new Command('login')
|
|
|
47
63
|
if (envToken) {
|
|
48
64
|
config.set('token', requireFreshJwt(envToken, 'TRAWL_TOKEN'));
|
|
49
65
|
persistUrlIfPending();
|
|
50
|
-
|
|
51
|
-
console.log(chalk.dim(` API: ${getApiUrl()}`));
|
|
52
|
-
console.log(chalk.dim(` Config: ${config.path}`));
|
|
66
|
+
reportLoginSuccess(opts);
|
|
53
67
|
return;
|
|
54
68
|
}
|
|
55
69
|
// --token flag: direct JWT (CI / retrocompat)
|
|
56
70
|
if (opts.token) {
|
|
57
71
|
config.set('token', requireFreshJwt(opts.token, '--token'));
|
|
58
72
|
persistUrlIfPending();
|
|
59
|
-
|
|
60
|
-
console.log(chalk.dim(` API: ${getApiUrl()}`));
|
|
61
|
-
console.log(chalk.dim(` Config: ${config.path}`));
|
|
73
|
+
reportLoginSuccess(opts);
|
|
62
74
|
return;
|
|
63
75
|
}
|
|
64
76
|
// Email/password flow
|
|
65
77
|
if (opts.password) {
|
|
66
|
-
console.
|
|
78
|
+
console.error(chalk.yellow('Warning: passing --password on the command line is insecure.'));
|
|
79
|
+
}
|
|
80
|
+
// #107 — never blocks on a readline prompt under --json or a non-TTY
|
|
81
|
+
// invocation (agent/CI subprocess); refuses with a clear, structured
|
|
82
|
+
// usage error instead, naming the flag (or TRAWL_TOKEN) to pass.
|
|
83
|
+
if (!opts.email) {
|
|
84
|
+
requireInteractive('Email is required — pass -e/--email, --token, or set TRAWL_TOKEN (refusing to block on a prompt, non-interactive).', { json: opts.json });
|
|
85
|
+
}
|
|
86
|
+
if (!opts.password) {
|
|
87
|
+
requireInteractive('Password is required — pass -p/--password, --token, or set TRAWL_TOKEN (refusing to block on a prompt, non-interactive).', { json: opts.json });
|
|
67
88
|
}
|
|
68
89
|
const email = opts.email ?? (await promptEmail());
|
|
69
90
|
const password = opts.password ?? (await promptPassword('Password: '));
|
|
@@ -82,13 +103,16 @@ export const login = new Command('login')
|
|
|
82
103
|
const token = requireFreshJwt(raw, 'token');
|
|
83
104
|
config.set('token', token);
|
|
84
105
|
persistUrlIfPending();
|
|
85
|
-
|
|
86
|
-
console.log(chalk.dim(` API: ${getApiUrl()}`));
|
|
87
|
-
console.log(chalk.dim(` Config: ${config.path}`));
|
|
106
|
+
reportLoginSuccess(opts, email);
|
|
88
107
|
});
|
|
89
108
|
export const logout = new Command('logout')
|
|
90
109
|
.description('Clear stored credentials')
|
|
91
|
-
.
|
|
110
|
+
.option('--json', 'Output as JSON')
|
|
111
|
+
.action((opts) => {
|
|
92
112
|
config.set('token', '');
|
|
113
|
+
if (opts.json) {
|
|
114
|
+
json({ ok: true });
|
|
115
|
+
return;
|
|
116
|
+
}
|
|
93
117
|
console.log(chalk.green('✓ Logged out'));
|
|
94
118
|
});
|
|
@@ -1,4 +1,26 @@
|
|
|
1
1
|
import { Command } from 'commander';
|
|
2
|
+
/**
|
|
3
|
+
* #108 — surface reorg. `run`/`list`/`get`/`data`/`history`/`run-info`/
|
|
4
|
+
* `trigger` are promoted to top-level verbs (see index.ts's `createProgram`)
|
|
5
|
+
* alongside `fetch`/`whoami`/`ping`. Each is built by an exported
|
|
6
|
+
* `attachXCommand(parent, attachOpts)` factory instead of a fixed
|
|
7
|
+
* `scraps.command(...)` chain, so it can be attached TWICE with a single
|
|
8
|
+
* source-of-truth definition: once to `program` (the new canonical
|
|
9
|
+
* top-level path) and once more, hidden, right back onto `scraps` — so every
|
|
10
|
+
* pre-#108 `trawl scraps <verb>` invocation keeps resolving unchanged
|
|
11
|
+
* (`{ hidden: true }` only affects help visibility, never resolution).
|
|
12
|
+
*
|
|
13
|
+
* Future-drift guard: ALL `.option()`/`.argument()`/`.action()` wiring for a
|
|
14
|
+
* promoted verb MUST live INSIDE its `attachXCommand` factory body, never
|
|
15
|
+
* bolted onto one of its two call sites (index.ts's `createProgram` for the
|
|
16
|
+
* top-level attach, this file's own `attachXCommand(scraps, { hidden: true
|
|
17
|
+
* })` call for the legacy one). That's the only thing keeping `trawl run
|
|
18
|
+
* <id>` and `trawl scraps run <id>` identical — a call-site-only tweak to
|
|
19
|
+
* one attachment would silently diverge the two.
|
|
20
|
+
*/
|
|
21
|
+
type AttachOptions = {
|
|
22
|
+
hidden?: boolean;
|
|
23
|
+
};
|
|
2
24
|
export declare const scraps: Command;
|
|
3
25
|
/** The top-of-history snapshot pollRunProgress needs to identify which run
|
|
4
26
|
* it's watching — see captureBeforeRunState.
|
|
@@ -17,6 +39,17 @@ interface BeforeRunState {
|
|
|
17
39
|
alreadyInFlight: boolean;
|
|
18
40
|
captured: boolean;
|
|
19
41
|
}
|
|
42
|
+
/** The machine-readable outcome `pollRunProgress` prints as the single final
|
|
43
|
+
* NDJSON line under `--json` (#107 review F1). `status` is the same honest
|
|
44
|
+
* terminal string the human-mode "Run finished: <status>" line already
|
|
45
|
+
* shows (`success`/`error`/`empty`/`regression`/…), or `'timeout'` /
|
|
46
|
+
* `'poll_error'` for the two non-terminal exits. */
|
|
47
|
+
export interface PollOutcome {
|
|
48
|
+
runId?: string;
|
|
49
|
+
status: string;
|
|
50
|
+
/** Present only for `status:'poll_error'` — the last poll failure's message. */
|
|
51
|
+
error?: string;
|
|
52
|
+
}
|
|
20
53
|
/**
|
|
21
54
|
* #91 P1 — replaces "await the run to completion, THEN open the activities
|
|
22
55
|
* SSE stream" (which showed NOTHING: the activities SSE
|
|
@@ -78,9 +111,37 @@ interface BeforeRunState {
|
|
|
78
111
|
* this function has, so it resolves to the same honest timeout rather than
|
|
79
112
|
* risk reporting a possibly-wrong outcome (never a lie, at worst a timeout
|
|
80
113
|
* telling the caller to check `doctor`).
|
|
114
|
+
*
|
|
115
|
+
* #107 review F1 — before this fix, `run|trigger --json --watch` was
|
|
116
|
+
* outcome-blind: `quiet` suppressed ALL output (including "Run finished:
|
|
117
|
+
* failure" and the timeout notice), a transient poll error was caught and
|
|
118
|
+
* silently retried FOREVER within the deadline, and the process always
|
|
119
|
+
* exited 0 after the poll loop regardless of what the watched run actually
|
|
120
|
+
* did — dead air, then a clean exit code, even for a failed or timed-out
|
|
121
|
+
* run. An agent scripting this CLI had no way to tell success from failure
|
|
122
|
+
* from "we gave up". Fixed by:
|
|
123
|
+
* - emitting exactly ONE final NDJSON line on stdout under `--json` once
|
|
124
|
+
* the watch reaches ANY of its three exits (terminal status, timeout, or
|
|
125
|
+
* a persistent poll error) — `{runId,status}` (+`error` for a poll
|
|
126
|
+
* error) — while every intermediate progress line stays suppressed
|
|
127
|
+
* (unchanged from before);
|
|
128
|
+
* - setting `process.exitCode` non-zero on a genuine run failure, a
|
|
129
|
+
* timeout, or a persistent poll error, and `0` on a real success — in
|
|
130
|
+
* BOTH `--json` and human `--watch` modes (human mode used to exit 0
|
|
131
|
+
* unconditionally, the same bug, just silent instead of dishonest);
|
|
132
|
+
* - giving up after `MAX_CONSECUTIVE_POLL_ERRORS` consecutive failed reads
|
|
133
|
+
* instead of retrying the same dead endpoint for the full 300s.
|
|
81
134
|
*/
|
|
82
135
|
export declare function pollRunProgress(id: string, before: BeforeRunState | undefined, opts?: {
|
|
83
136
|
intervalMs?: number;
|
|
84
137
|
timeoutMs?: number;
|
|
138
|
+
json?: boolean;
|
|
85
139
|
}): Promise<void>;
|
|
140
|
+
export declare function attachListCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
141
|
+
export declare function attachGetCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
142
|
+
export declare function attachRunCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
143
|
+
export declare function attachDataCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
144
|
+
export declare function attachHistoryCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
145
|
+
export declare function attachRunInfoCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
146
|
+
export declare function attachTriggerCommand(parent: Command, attachOpts?: AttachOptions): Command;
|
|
86
147
|
export {};
|