@trawlme/cli 3.9.1 → 3.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -79,7 +79,7 @@ trawl spec [--json] Print a versioned, machine-readab
79
79
  - `list`/`get` also show a **health** badge next to the status badge. By default it reads the scrap's own `lastCronOutcome` field (always present, no extra cost): `⚠ cron paused` when the last scheduled tick was skipped for being unhealthy, `—` otherwise — a breadcrumb of the last tick, not a live read (never set on a no-cron scrap, can lag by one tick). `list --unhealthy` asks the API to filter to scraps whose *current* consecutive-failure streak is 3+ (trawl_node's own threshold) and annotates each with the exact `consecutiveFailedRuns`/`unhealthySince` — shown instead of the badge whenever present, and never re-derived client-side from history. `--json` carries whichever fields the request produced: `lastCronOutcome` always, `consecutiveFailedRuns`/`unhealthySince` only when `--unhealthy` was passed. **Old-server safety:** a server predating trawl_node#1952 silently ignores the unknown `--unhealthy` query param and returns everything, unfiltered — the CLI detects that (no returned item carries `consecutiveFailedRuns`) and prints a stderr warning instead of presenting the full list as "your unhealthy scraps".
80
80
  - **Long-running calls (`create`, `run`, `data --fresh`, `trigger --wait`):** these hit server-side paths that can legitimately take 30–250s+ (AI generation + scrap creation + a first run for `create`; proxy tier escalation + AI-fix retries for the other three) — the CLI arms a 300s timeout for exactly these four call sites instead of the generic 30s default. `TRAWL_TIMEOUT` (see below) still overrides ALL requests, including these — set it if you need a tighter or looser ceiling than 300s, but note a global override that tight also clamps `create`.
81
81
  - **`--watch` is poll-based, not a live stream:** the activities SSE endpoint has no backlog and, for the default async `trigger` (no `--wait`), runs in a separate cron-consumer pod whose events never reach the API pod holding the SSE connection — a naive "await the run, then open SSE" shows nothing. `run --watch` and `trigger --watch` instead poll `GET /api/scraps/:id` (terminal status) and the activities REST list until the run finishes, printing each new activity line as it appears. The watched run's outcome drives the exit code too, in BOTH human and `--json` mode: a genuinely failed terminal run, a poll timeout, or a persistently unreachable API all exit non-zero — a clean successful run is the only exit `0`. A run that never reaches a terminal status within 300s prints an honest timeout notice pointing at `scraps doctor <id>` (human mode) — see the `--json` shape below. `scraps watch <id>` (the standalone command, no trigger) is unchanged — it still opens the live SSE stream directly.
82
- - `spec --json` prints the CLI's own command tree — `{specVersion, cliVersion, commands[], exitCodes, errorKinds, kindExitCodes}` — DERIVED at runtime by walking the live commander tree (never a hand-maintained file, which would silently drift from reality). Each entry in `commands[]` carries its full path (e.g. `"scraps account session set"`), description, `hidden` (the legacy `scraps <verb>` aliases above), `leaf` (false for a pure namespace/group node like `scraps`/`skills`/`telemetry` — invoking one directly is a guaranteed-failing tool, not a real command), `aliases`, `arguments`, and `options`; `exitCodes`/`errorKinds`/`kindExitCodes` are read from the exact same source `classifyError` uses (see [Exit codes](#exit-codes)) — never a second copy. `kindExitCodes` is the inverse of `exitCodes`: `kind -> exitCode`, since exit code `1` alone is a shared bucket (`api`/`refused`/`unknown`/`in_progress`/`run_failed`/`upgrade_failed`) that `exitCodes`' flat label can't disambiguate. Without `--json`, `spec` prints one short human line (version + visible command count) pointing at `--json`.
82
+ - `spec --json` prints the CLI's own command tree — `{specVersion, cliVersion, commands[], exitCodes, errorKinds, kindExitCodes, docsUrl?, llmsUrl?}` — DERIVED at runtime by walking the live commander tree (never a hand-maintained file, which would silently drift from reality). Each entry in `commands[]` carries its full path (e.g. `"scraps account session set"`), description, `hidden` (the legacy `scraps <verb>` aliases above), `leaf` (false for a pure namespace/group node like `scraps`/`skills`/`telemetry` — invoking one directly is a guaranteed-failing tool, not a real command), `aliases`, `arguments`, `options`, and an optional per-command `docs` deep link (e.g. every `scraps account *` command points at the account-sessions guide); `exitCodes`/`errorKinds`/`kindExitCodes` are read from the exact same source `classifyError` uses (see [Exit codes](#exit-codes)) — never a second copy. `kindExitCodes` is the inverse of `exitCodes`: `kind -> exitCode`, since exit code `1` alone is a shared bucket (`api`/`refused`/`unknown`/`in_progress`/`run_failed`/`upgrade_failed`) that `exitCodes`' flat label can't disambiguate. `docsUrl` is resolved server-first (`externalDocs.url` on the configured API base's own OpenAPI document, when declared), else derived from a KNOWN first-party `trawl.me` API host, else **omitted entirely** — never a guessed URL pointed at the wrong docs host for a self-hosted install. `llmsUrl` and every per-command `docs` link are narrower: they only ever come from that same known-host derivation, NEVER from a server-declared `docsUrl` — this CLI's own guide slugs have no reason to exist on a third party's own docs root, so a self-hosted server that declares `externalDocs.url` gets a correct top-level `docsUrl` but no `llmsUrl` and no per-command `docs` links. A `failureKind:"auth"` run's JSON payload (`doctor`/`data --errors`/`run-info`) carries the same `docs` field, by the same known-host-only rule. Without `--json`, `spec` prints one short human line (version + visible command count) pointing at `--json`.
83
83
  - `run --json`/`trigger --json` bypass the spinner and print the raw launch/trigger payload on stdout; combined with `--watch`, every intermediate progress line stays suppressed (stdout stays pure JSON) and, once the watch reaches its outcome, exactly ONE final NDJSON line is emitted: `{"runId","status"}` (the honest terminal status — `success`/`error`/`empty`/`regression`/…), `{"runId","status":"timeout"}` on a poll timeout, or `{"runId","status":"poll_error","error"}` if the API stays unreachable for several consecutive polls — `process.exitCode` is non-zero for all three except a genuine success. `scraps watch --json` emits one raw JSON object per activity line (NDJSON) instead of the formatted `[time] message` text — there's no single final payload to wait for on a live stream.
84
84
 
85
85
  ### Scrap management
@@ -134,6 +134,8 @@ Skills auto-update when you upgrade the CLI — no need to re-install manually
134
134
 
135
135
  A pre-existing skill directory that trawl did not install itself (no `.version` marker) is never touched — `install`/`update` refuse to overwrite it and require `--force` to proceed. This applies to the CLI-upgrade auto-sync too (it silently skips marker-less dirs rather than refusing, since there is no interactive user to show a refusal to).
136
136
 
137
+ **`trawl login` bootstraps any bundled skill you don't have yet** — the one command that does, since it's the one intentional human setup moment (interactive, a real TTY, not `--json`). It prints exactly what it installed and where, plus a fact you need to act on yourself: **Claude Code must be restarted to see them** — skills are loaded at session start, so anything installed mid-session stays invisible until then. Every other command only ever *suggests* running `trawl skills install` (throttled, never under `--json`/non-TTY/`TRAWL_SKILLS_SYNC=0`) — it never writes to `~/.claude/skills` as a side effect of something else you asked for. If you already have every bundled skill, or you're logging in non-interactively (CI, `--json`, `TRAWL_TOKEN=... trawl login`), nothing is written either way.
138
+
137
139
  You can also install skills standalone (without the CLI): `npx @trawlme/skills install`.
138
140
 
139
141
  ### Auth
@@ -69,6 +69,31 @@ export interface Run {
69
69
  * response can still be the pre-#1950 flat shape for a while.
70
70
  */
71
71
  export declare function detectWallVendor(run: Pick<Run, 'status' | 'statusDetail' | 'block' | 'blockType'>): string | null;
72
+ /**
73
+ * #184 — true for a run currently carrying a live (non-stale) login-wall
74
+ * verdict. Extracted out of `formatDoctor`'s own local `authWall` const so
75
+ * `commands/scraps.ts`'s `doctor`/`run-info` actions can reuse the EXACT
76
+ * same guard to decide whether to fire the skills-install safety-net nudge
77
+ * (lib/skillsNudge.ts `maybeSuggestSkillsForAuthWall`) — one rule, not two
78
+ * copies that could quietly drift apart.
79
+ *
80
+ * Same staleness guard as `detectWallVendor` above and for the same reason:
81
+ * `failureKind` is a terminal classification trawl_node stamps once, but
82
+ * `patchForRegression` can flip `status`/`statusDetail` to success/
83
+ * regression LATER without ever clearing it — so a run that ultimately
84
+ * succeeded or degraded must never still read as an active auth wall.
85
+ *
86
+ * Typed structurally loose (not `Pick<Run, ...>`) on purpose: `Run.status`/
87
+ * `statusDetail` are required fields, but `scraps.ts`'s own `HistoryRun`
88
+ * (the run-info command's shape) declares the same three fields OPTIONAL —
89
+ * a `Pick<Run, ...>` parameter type would reject that caller at compile
90
+ * time even though every field it actually reads is present at runtime.
91
+ */
92
+ export declare function isAuthWall(run: {
93
+ failureKind?: string | null;
94
+ status?: boolean | null;
95
+ statusDetail?: string | null;
96
+ }): boolean;
72
97
  /**
73
98
  * Autofix activity metadata — from the persisted ai_fix_end activity.
74
99
  * aiUsage (cost) is stripped server-side; all diagnostics are kept.
@@ -69,6 +69,29 @@ export function detectWallVendor(run) {
69
69
  }
70
70
  return null;
71
71
  }
72
+ /**
73
+ * #184 — true for a run currently carrying a live (non-stale) login-wall
74
+ * verdict. Extracted out of `formatDoctor`'s own local `authWall` const so
75
+ * `commands/scraps.ts`'s `doctor`/`run-info` actions can reuse the EXACT
76
+ * same guard to decide whether to fire the skills-install safety-net nudge
77
+ * (lib/skillsNudge.ts `maybeSuggestSkillsForAuthWall`) — one rule, not two
78
+ * copies that could quietly drift apart.
79
+ *
80
+ * Same staleness guard as `detectWallVendor` above and for the same reason:
81
+ * `failureKind` is a terminal classification trawl_node stamps once, but
82
+ * `patchForRegression` can flip `status`/`statusDetail` to success/
83
+ * regression LATER without ever clearing it — so a run that ultimately
84
+ * succeeded or degraded must never still read as an active auth wall.
85
+ *
86
+ * Typed structurally loose (not `Pick<Run, ...>`) on purpose: `Run.status`/
87
+ * `statusDetail` are required fields, but `scraps.ts`'s own `HistoryRun`
88
+ * (the run-info command's shape) declares the same three fields OPTIONAL —
89
+ * a `Pick<Run, ...>` parameter type would reject that caller at compile
90
+ * time even though every field it actually reads is present at runtime.
91
+ */
92
+ export function isAuthWall(run) {
93
+ return run.failureKind === 'auth' && run.status !== true && run.statusDetail !== 'regression';
94
+ }
72
95
  const TIER_LABELS = {
73
96
  tier0: 'Tier 0',
74
97
  tier1: 'Tier 1',
@@ -194,7 +217,7 @@ export function formatDoctor(scrapTitle, run, fix = null, scrapId) {
194
217
  // `detectWallVendor`'s guard already exists to prevent for `block.kind`.
195
218
  // One rule, not two coincidences: both checks guard the same two fields
196
219
  // against the same after-the-fact patch.
197
- const authWall = run.failureKind === 'auth' && run.status !== true && run.statusDetail !== 'regression';
220
+ const authWall = isAuthWall(run);
198
221
  const authHintWillRender = authWall && Boolean(scrapId);
199
222
  const conflictingSignals = Boolean(wallVendor) && authWall;
200
223
  if (conflictingSignals) {
@@ -4,8 +4,9 @@ import config, { getApiUrl, getLiveAuthEnvVar } from '../lib/config.js';
4
4
  import { api } from '../lib/api.js';
5
5
  import { requireFreshJwt, requireUrl } from '../lib/validate.js';
6
6
  import { promptPassword } from '../lib/prompt.js';
7
- import { requireInteractive } from '../lib/confirm.js';
7
+ import { requireInteractive, isInteractive } from '../lib/confirm.js';
8
8
  import { json } from '../lib/format.js';
9
+ import { bootstrapSkillsOnLogin, RESTART_CLAUDE_CODE_NOTE } from '../lib/skills.js';
9
10
  async function promptEmail() {
10
11
  const { createInterface } = await import('readline');
11
12
  const rl = createInterface({ input: process.stdin, output: process.stdout });
@@ -40,6 +41,62 @@ function reportLoginSuccess(opts, email) {
40
41
  if (overrideVar) {
41
42
  console.error(chalk.yellow(`⚠ ${overrideVar} is set in your environment — it overrides the token just stored here for every subsequent command until you unset it.`));
42
43
  }
44
+ // #184 — `login` is the ONE intentional human setup moment: bootstrap any
45
+ // bundled Claude skill the user doesn't have yet. Gated exactly like
46
+ // every other filesystem-mutating side effect in this CLI
47
+ // (json/TTY/opt-out — see lib/skills.ts's isSkillsActionAllowed), so a
48
+ // CI/agent `TRAWL_TOKEN=x trawl login --json` never writes into
49
+ // ~/.claude/skills on a machine with no Claude Code session to discover
50
+ // them. `isInteractive` is the exact TTY definition login already uses
51
+ // for its own email/password prompt gate (lib/confirm.ts) — stdin AND
52
+ // stdout, not just stdout. `bootstrapped` is `null` ONLY when gated out
53
+ // (json/non-TTY/opted-out) or when every bundled skill is already owned
54
+ // by trawl — the genuine "nothing to do" case, where the lines below
55
+ // naturally never print. It is non-null whenever anything was attempted,
56
+ // whether or not any of it actually landed — see the block below.
57
+ //
58
+ // stderr, same convention as the `overrideVar` warning right above (and
59
+ // for the same reason): this is orthogonal to the `--json` envelope's
60
+ // fixed shape below, so it must never risk landing on stdout — a defense
61
+ // that holds even if `bootstrapSkillsOnLogin`'s own json/TTY gate above it
62
+ // were ever wrong, not just a style match.
63
+ //
64
+ // #184 review (BLOCK + MAJOR) — `bootstrapped` is non-null whenever there
65
+ // was anything to attempt, whether or not any of it actually landed, so
66
+ // both halves below must be checked independently: `installed` prints the
67
+ // success line, `skipped` prints one honest line per skill that was
68
+ // requested but did NOT land (a permission error, or a pre-existing
69
+ // marker-less dir the ownership guard refused to overwrite) and WHY —
70
+ // `reason` is already a relayable, fact-only string (SkillOwnershipRefusalError's
71
+ // `.relayableReason`, or a raw fs error message) with no imperative, so it
72
+ // is safe to print verbatim on this channel. A total failure (installed
73
+ // empty, skipped non-empty) must read as "attempted and failed" — never
74
+ // fall through to silence, which is indistinguishable from "never
75
+ // attempted" (the exact false-success shape this issue exists to
76
+ // prevent). The restart note only applies to what actually landed, so it
77
+ // stays scoped to that branch.
78
+ //
79
+ // #184 defect 2 — `error` is the THIRD case: distinct from both "nothing
80
+ // to do" (bootstrapped is `null`, nothing prints) and "some/all skills
81
+ // failed" (`skipped`, above) — it means bootstrapSkillsOnLogin could not
82
+ // even determine which skills to install (the bundled skills package
83
+ // looks missing/corrupted). Stated as a fact, same convention as every
84
+ // other line here.
85
+ const bootstrapped = bootstrapSkillsOnLogin({ json: opts.json, isTTY: isInteractive(opts) });
86
+ if (bootstrapped) {
87
+ if (bootstrapped.error) {
88
+ console.error(chalk.yellow(`⚠ Claude skills bootstrap: ${bootstrapped.error}`));
89
+ }
90
+ if (bootstrapped.installed.length > 0) {
91
+ console.error(chalk.green(`✓ Installed Claude skill${bootstrapped.installed.length === 1 ? '' : 's'}: `) +
92
+ `${bootstrapped.installed.join(', ')}` +
93
+ chalk.dim(` at ${bootstrapped.dest}`));
94
+ console.error(chalk.yellow(RESTART_CLAUDE_CODE_NOTE));
95
+ }
96
+ for (const { name, reason } of bootstrapped.skipped) {
97
+ console.error(chalk.yellow(`⚠ Claude skill "${name}" install attempted but failed: ${reason}`));
98
+ }
99
+ }
43
100
  if (opts.json) {
44
101
  json({ ok: true, apiUrl: getApiUrl(), config: config.path, ...(email && { email }) });
45
102
  return;
@@ -8,9 +8,38 @@ import { promptPassword } from '../lib/prompt.js';
8
8
  import { validateObjectId, requireUrl } from '../lib/validate.js';
9
9
  import { classifyError, reportError, retryFieldsFor, UsageError, RefusalError } from '../lib/errors.js';
10
10
  import { confirmDestructive, isInteractive } from '../lib/confirm.js';
11
- import { formatDoctor, formatAutofix, fetchRunAndFix, pickRun, pickFix } from './doctor.js';
11
+ import { formatDoctor, formatAutofix, fetchRunAndFix, pickRun, pickFix, isAuthWall } from './doctor.js';
12
12
  import { renderPinch, pinchEnabled } from '../lib/pinch.js';
13
13
  import { maybeShowReferralTip } from '../lib/tips.js';
14
+ import { maybeSuggestSkillsForAuthWall } from '../lib/skillsNudge.js';
15
+ import { getApiUrl } from '../lib/config.js';
16
+ import { resolveDocsUrls, resolveFailureKindDocsUrl } from '../lib/docs.js';
17
+ /**
18
+ * #185 — `{ docs }` (spreadable, empty object when nothing to add) for a
19
+ * run's JSON payload (`doctor`/`data --errors`/`run-info`), keyed on
20
+ * trawl_node's `failureKind` (NOT errors.ts's unrelated `ErrorEnvelope.kind`
21
+ * — see docs.ts's `FAILURE_KIND_DOC_PATHS` doc comment for why those two
22
+ * `'auth'`s must never be conflated). Reuses `isAuthWall` — the SAME
23
+ * staleness guard `formatDoctor`'s human hint and the #184 skills nudge
24
+ * already share (a `failureKind:'auth'` stamp survives a later
25
+ * success/regression patch, see `isAuthWall`'s own doc comment) — so a run
26
+ * that ultimately succeeded or degraded never carries a stale `docs` link
27
+ * either, exactly the invariant `detectWallVendor`/`isAuthWall` already
28
+ * enforce for the human-facing surfaces. One call site, one rule, reused by
29
+ * all three JSON surfaces below instead of three copies that could drift.
30
+ */
31
+ function runDocsField(run) {
32
+ if (!isAuthWall(run))
33
+ return {};
34
+ // Whole resolved object (never just `.docsUrl`) so resolveFailureKindDocsUrl
35
+ // can gate on rung provenance — see docs.ts's DocsUrls.docsUrlIsDerived.
36
+ // This call never supplies `externalDocsUrl`, so `docsUrl` here can only
37
+ // ever be rung-2-derived or absent — never a rung-1 server-declared root —
38
+ // but the gate stays explicit rather than relying on that as an invariant.
39
+ const resolved = resolveDocsUrls({ apiBaseUrl: getApiUrl() });
40
+ const docs = resolveFailureKindDocsUrl(run.failureKind, resolved);
41
+ return docs ? { docs } : {};
42
+ }
14
43
  /**
15
44
  * Print a usage/validation error consistently: human text to stderr, or a
16
45
  * machine envelope on stdout under --json (never both — reportError is the
@@ -1064,8 +1093,12 @@ export function attachDataCommand(parent, attachOpts = {}) {
1064
1093
  // --json is honored for BOTH outcomes (success or failure) — an agent
1065
1094
  // parsing `data --errors --json` must always get the flat run object,
1066
1095
  // never prose gated behind a status check. (#71 finding 13)
1096
+ // #185 — `docs` rides alongside as a sibling field on the same flat
1097
+ // object (never nested under `run`, since there's no wrapper shape
1098
+ // here), present only for a live mapped failureKind (see
1099
+ // runDocsField's doc comment).
1067
1100
  if (opts.json)
1068
- return json(pickRun(result.run));
1101
+ return json({ ...pickRun(result.run), ...runDocsField(result.run) });
1069
1102
  if (result.run.status === true) {
1070
1103
  console.log(chalk.green('✓ Last run succeeded. No errors to show.'));
1071
1104
  return;
@@ -1269,6 +1302,11 @@ export function attachRunInfoCommand(parent, attachOpts = {}) {
1269
1302
  selector: h.errorSnapshot?.selector ?? null,
1270
1303
  emptyContext: h.emptyContext ?? null,
1271
1304
  createdAt: h.createdAt ?? null,
1305
+ // #185 — present only for a live mapped failureKind (see
1306
+ // runDocsField's doc comment); omitted, never `docs: undefined`, for
1307
+ // every other run so this object's own JSON.stringify never grows an
1308
+ // unexpected key on existing consumers.
1309
+ ...runDocsField(h),
1272
1310
  };
1273
1311
  if (opts.json) {
1274
1312
  json(info);
@@ -1276,6 +1314,13 @@ export function attachRunInfoCommand(parent, attachOpts = {}) {
1276
1314
  }
1277
1315
  // The blob is an object; the table needs one line. --json keeps it whole.
1278
1316
  table([{ ...info, emptyContext: summarizeEmptyContext(info.emptyContext) }], ['hid', 'status', 'time', 'tier', 'failureKind', 'blockType', 'errorMessage', 'selector', 'emptyContext', 'createdAt']);
1317
+ // #184 safety-net — the run just shown carries a live login-wall
1318
+ // verdict; if Claude's skills aren't installed either, point at the
1319
+ // command that installs them too (never under --json, see the early
1320
+ // return above).
1321
+ if (isAuthWall(h)) {
1322
+ maybeSuggestSkillsForAuthWall();
1323
+ }
1279
1324
  });
1280
1325
  }
1281
1326
  attachRunInfoCommand(scraps, { hidden: true });
@@ -1654,12 +1699,22 @@ scraps
1654
1699
  console.log(chalk.dim('No runs yet.'));
1655
1700
  return;
1656
1701
  }
1702
+ // #185 — `docs` sits alongside `run`/`fix` (present only for a live
1703
+ // mapped failureKind — see runDocsField's doc comment), never nested
1704
+ // inside the allowlisted `run` object.
1657
1705
  if (opts.json)
1658
- return json({ run: pickRun(result.run), fix: pickFix(result.fix) });
1706
+ return json({ run: pickRun(result.run), fix: pickFix(result.fix), ...runDocsField(result.run) });
1659
1707
  console.log(formatDoctor(result.scrap.title, result.run, result.fix, id));
1660
1708
  if (opts.autofix && result.fix) {
1661
1709
  console.log('\n' + formatAutofix(result.fix));
1662
1710
  }
1711
+ // #184 safety-net — formatDoctor already prints the login-wall hint
1712
+ // (#182) above when this is a live auth wall; if Claude's skills aren't
1713
+ // installed either, point at the command that installs them too (never
1714
+ // under --json, see the early return above).
1715
+ if (isAuthWall(result.run)) {
1716
+ maybeSuggestSkillsForAuthWall();
1717
+ }
1663
1718
  });
1664
1719
  // autofix — show full auto-fix attempt detail (diff, dry-run, knowledge)
1665
1720
  scraps
@@ -1,4 +1,5 @@
1
1
  import { Command } from 'commander';
2
+ import { type DocsUrls } from '../lib/docs.js';
2
3
  /**
3
4
  * `trawl spec --json` (#170) — a versioned, machine-readable description of
4
5
  * the command tree, so an agent can learn the CLI's surface without parsing
@@ -54,6 +55,18 @@ export interface CliSpecCommand {
54
55
  aliases: string[];
55
56
  arguments: CliSpecArgument[];
56
57
  options: CliSpecOption[];
58
+ /**
59
+ * #185 — a deep link into a specific guide when one is mapped for this
60
+ * command (docs.ts's `COMMAND_DOC_PATHS`, e.g. every `scraps account *`
61
+ * command -> the account-sessions guide). Omitted (never a guessed/generic
62
+ * link) whenever no guide is mapped, the top-level `docsUrl` itself
63
+ * couldn't be resolved, OR `docsUrl` resolved from rung 1 (a server-
64
+ * declared `externalDocs.url`) rather than rung 2 (this CLI's own known-host
65
+ * derivation) — appending our guide slug onto a third party's own docs root
66
+ * is exactly the confidently-wrong-404 defect this omission closes. See
67
+ * `resolveCommandDocsUrl` / `DocsUrls.docsUrlIsDerived`.
68
+ */
69
+ docs?: string;
57
70
  }
58
71
  export interface CliSpec {
59
72
  specVersion: 1;
@@ -81,12 +94,30 @@ export interface CliSpec {
81
94
  * (errors.ts's KIND_EXIT_CODES) — never a second copy.
82
95
  */
83
96
  kindExitCodes: Record<string, number>;
97
+ /**
98
+ * #185 — where the human-readable guides live, resolved via docs.ts's
99
+ * ladder (server `externalDocs.url` > derive from the API base's known
100
+ * `trawl.me` host > omit). Absent (never a guessed URL) when neither rung
101
+ * resolves — e.g. a self-hosted install with no `externalDocs` declared.
102
+ */
103
+ docsUrl?: string;
104
+ /** #185 — the llms.txt entry point alongside `docsUrl`, when resolvable.
105
+ * See docs.ts's module doc comment for why this never rides along with a
106
+ * server-declared `docsUrl` it wasn't itself derived from. */
107
+ llmsUrl?: string;
84
108
  }
85
109
  /**
86
110
  * Build the full spec from a live, already-constructed program (e.g.
87
111
  * `createProgram()`'s return value). Includes every node in the tree —
88
112
  * hidden legacy aliases (`scraps list`, …) included, flagged via `hidden`,
89
113
  * so a consumer that wants only the advertised surface can filter on it.
114
+ *
115
+ * #185 — `docs` is an already-RESOLVED `DocsUrls` (docs.ts's ladder run to
116
+ * completion), never computed in here: `buildSpec` stays pure/synchronous
117
+ * (no network, no config read) exactly as before, so every existing caller
118
+ * (the README-vs-spec contract test, the synthetic-tree unit tests) keeps
119
+ * working unchanged by simply omitting the parameter. Only `spec.ts`'s own
120
+ * action resolves rung 1 (a bounded fetch) before calling this.
90
121
  */
91
- export declare function buildSpec(program: Command): CliSpec;
122
+ export declare function buildSpec(program: Command, docs?: DocsUrls): CliSpec;
92
123
  export declare const spec: Command;
@@ -1,6 +1,9 @@
1
1
  import { Command, Help } from 'commander';
2
2
  import { json } from '../lib/format.js';
3
3
  import { EXIT_CODE_LABELS, ENVELOPE_KINDS, KIND_EXIT_CODES } from '../lib/errors.js';
4
+ import { getApiUrl } from '../lib/config.js';
5
+ import { api } from '../lib/api.js';
6
+ import { resolveDocsUrls, resolveCommandDocsUrl } from '../lib/docs.js';
4
7
  function buildOption(option) {
5
8
  const entry = {
6
9
  long: option.long ?? '',
@@ -15,8 +18,8 @@ function buildOption(option) {
15
18
  entry.default = option.defaultValue;
16
19
  return entry;
17
20
  }
18
- function buildCommandEntry(cmd, name, hidden) {
19
- return {
21
+ function buildCommandEntry(cmd, name, hidden, docs = {}) {
22
+ const entry = {
20
23
  name,
21
24
  description: cmd.description(),
22
25
  hidden,
@@ -29,20 +32,29 @@ function buildCommandEntry(cmd, name, hidden) {
29
32
  })),
30
33
  options: cmd.options.map(buildOption),
31
34
  };
35
+ const docsLink = resolveCommandDocsUrl(name, docs);
36
+ if (docsLink)
37
+ entry.docs = docsLink;
38
+ return entry;
32
39
  }
33
40
  /**
34
41
  * Walk `cmd.commands` recursively, collecting one entry per node under its
35
42
  * FULL path name. `hidden` is read via commander's own
36
43
  * `Help#visibleCommands()` — the same idiom this codebase already uses
37
44
  * (scraps.test.ts's #108 describe block) — rather than reaching for the
38
- * private, untyped `_hidden` field directly.
45
+ * private, untyped `_hidden` field directly. The resolved `docs` object,
46
+ * when present, threads through so every entry can carry its own per-command
47
+ * `docs` deep link (#185) off the SAME resolved root — never a second lookup
48
+ * per node, and never flattened to a bare string (see docs.ts's
49
+ * `DocsUrls.docsUrlIsDerived` — the rung provenance must survive the walk
50
+ * unchanged so `resolveCommandDocsUrl` can still gate on it at each leaf).
39
51
  */
40
- function walk(cmd, prefix, out) {
52
+ function walk(cmd, prefix, out, docs = {}) {
41
53
  const visible = new Set(new Help().visibleCommands(cmd));
42
54
  for (const sub of cmd.commands) {
43
55
  const name = prefix ? `${prefix} ${sub.name()}` : sub.name();
44
- out.push(buildCommandEntry(sub, name, !visible.has(sub)));
45
- walk(sub, name, out);
56
+ out.push(buildCommandEntry(sub, name, !visible.has(sub), docs));
57
+ walk(sub, name, out, docs);
46
58
  }
47
59
  }
48
60
  /**
@@ -50,11 +62,18 @@ function walk(cmd, prefix, out) {
50
62
  * `createProgram()`'s return value). Includes every node in the tree —
51
63
  * hidden legacy aliases (`scraps list`, …) included, flagged via `hidden`,
52
64
  * so a consumer that wants only the advertised surface can filter on it.
65
+ *
66
+ * #185 — `docs` is an already-RESOLVED `DocsUrls` (docs.ts's ladder run to
67
+ * completion), never computed in here: `buildSpec` stays pure/synchronous
68
+ * (no network, no config read) exactly as before, so every existing caller
69
+ * (the README-vs-spec contract test, the synthetic-tree unit tests) keeps
70
+ * working unchanged by simply omitting the parameter. Only `spec.ts`'s own
71
+ * action resolves rung 1 (a bounded fetch) before calling this.
53
72
  */
54
- export function buildSpec(program) {
73
+ export function buildSpec(program, docs = {}) {
55
74
  const commands = [];
56
- walk(program, '', commands);
57
- return {
75
+ walk(program, '', commands, docs);
76
+ const spec = {
58
77
  specVersion: 1,
59
78
  cliVersion: program.version() ?? 'unknown',
60
79
  commands,
@@ -62,18 +81,71 @@ export function buildSpec(program) {
62
81
  errorKinds: [...ENVELOPE_KINDS],
63
82
  kindExitCodes: { ...KIND_EXIT_CODES },
64
83
  };
84
+ if (docs.docsUrl)
85
+ spec.docsUrl = docs.docsUrl;
86
+ if (docs.llmsUrl)
87
+ spec.llmsUrl = docs.llmsUrl;
88
+ return spec;
89
+ }
90
+ /**
91
+ * #185 — bounds the SOCKET, not just the promise, so a slow/dead server can
92
+ * never perceptibly slow `spec --json` down. Mirrors lib/tips.ts's
93
+ * FLAG_CHECK_TIMEOUT_MS for the same reason: this is the one surface allowed
94
+ * a network call at all, and only because it's a deliberate, once-per-
95
+ * invocation agent probe.
96
+ *
97
+ * This is a REAL wall-clock ceiling, not merely a promise-level one: an
98
+ * earlier version of this fetch used `api.publicGet` (Node's global
99
+ * `fetch()` + `AbortSignal.timeout()`), which bounds the promise but not the
100
+ * OS-level connection — against a routable-but-silently-dropping host (a
101
+ * dropped SYN, no RST; a realistic firewalled/air-gapped shape, the very
102
+ * audience this feature cites) the promise rejected at ~2005ms while the
103
+ * PROCESS didn't exit until ~10.5s later, waiting out Node's own internal
104
+ * connect-timeout. `api.probeJson` (see its own doc comment in lib/api.ts)
105
+ * fixes this by managing the raw socket directly — an independent timer
106
+ * that `req.destroy()`s the connection, never relying on the idle-based
107
+ * `http.request` `timeout` option. Re-measured after the fix, same
108
+ * black-holed host, via the exact repro command (`time env
109
+ * TRAWL_API_URL=https://192.0.2.1 node dist/index.js spec --json`): total
110
+ * process time ~2.43s/~2.47s across two runs (the 2000ms bound plus Node
111
+ * startup and spec-tree construction), not ~10.68s.
112
+ */
113
+ const EXTERNAL_DOCS_FETCH_TIMEOUT_MS = 2_000;
114
+ /**
115
+ * Rung 1 — read `externalDocs.url` off the OpenAPI document the configured
116
+ * API base itself serves at `/api/spec.json`. Unset (null) server-side
117
+ * today, so this resolves to `null` in practice; forward-looking for when
118
+ * it isn't. ANY failure — network error, timeout, a malformed/unexpected
119
+ * response shape — is indistinguishable from "not declared" here: resolves
120
+ * to `null`, never throws (api.probeJson's own contract — see its doc
121
+ * comment). `spec --json` must stay a pure, reliable, side-effect-free probe
122
+ * even against an unreachable or ancient server.
123
+ */
124
+ async function fetchExternalDocsUrl() {
125
+ const data = await api.probeJson('/api/spec.json', {
126
+ timeoutMs: EXTERNAL_DOCS_FETCH_TIMEOUT_MS,
127
+ });
128
+ const url = data?.externalDocs?.url;
129
+ return typeof url === 'string' && url ? url : null;
65
130
  }
66
131
  export const spec = new Command('spec')
67
132
  .description('Print a versioned, machine-readable description of the command tree')
68
133
  .option('--json', 'Output as JSON')
69
- .action((opts) => {
134
+ .action(async (opts) => {
70
135
  // `spec` is registered as a direct child of the program root
71
136
  // (index.ts's createProgram), so `.parent` IS that root by the time this
72
137
  // action ever runs — commander sets it in `addCommand()`. Falling back
73
138
  // to `spec` itself only matters for an isolated unit invocation (e.g. a
74
139
  // test driving `spec.parseAsync()` directly, unattached to a program).
75
140
  const root = spec.parent ?? spec;
76
- const cliSpec = buildSpec(root);
141
+ const apiBaseUrl = getApiUrl();
142
+ // #185 — rung 1 (the server fetch) only runs under --json: the
143
+ // plain-text branch below never reads docsUrl at all, so paying a
144
+ // network round-trip for it would be pure waste. Rungs 2/3 (sync,
145
+ // no network) still apply either way via the fallback below.
146
+ const externalDocsUrl = opts.json ? await fetchExternalDocsUrl() : null;
147
+ const docs = resolveDocsUrls({ apiBaseUrl, externalDocsUrl });
148
+ const cliSpec = buildSpec(root, docs);
77
149
  if (opts.json) {
78
150
  json(cliSpec);
79
151
  return;
package/dist/index.d.ts CHANGED
@@ -32,6 +32,15 @@ export declare function resolveCommandName(actionCommand: Command | undefined):
32
32
  */
33
33
  export declare function collectCommandNames(root: Command): string[];
34
34
  export declare function createProgram(): Command;
35
+ /**
36
+ * #185 — see the `addHelpText('afterAll', …)` call above for why this is
37
+ * unconditional (no TTY/--json gate) and why a thrown error must never
38
+ * escape: commander calls this synchronously while already writing help
39
+ * output, so a throw here would crash a `--help` invocation, the one code
40
+ * path this whole feature is not allowed to touch (see docs.ts's own
41
+ * `docsFooterLine` for the "never a guessed URL" contract this composes).
42
+ */
43
+ export declare function docsFooterText(): string;
35
44
  /**
36
45
  * True when this module is the process entrypoint (not merely imported by a test).
37
46
  *
@@ -96,6 +105,14 @@ export declare function isBareInvocation(argv: string[]): boolean;
96
105
  * overstates its own invariant is worse than no comment, because the next
97
106
  * reader trusts it.
98
107
  *
108
+ * #185 — `spec --json` (not plain `spec`) now DOES make its own bounded
109
+ * (2s), swallowed-on-failure GET for `externalDocs.url` (see spec.ts's
110
+ * `fetchExternalDocsUrl`) — this is not a regression of the "no mutation"
111
+ * invariant above (a GET mutates nothing, and any failure — including no
112
+ * egress at all — degrades to the sync host-derivation fallback, never a
113
+ * thrown error), just a second, narrower kind of side effect this guard was
114
+ * never meant to suppress in the first place.
115
+ *
99
116
  * Before this, `trawl spec --json` silently re-synced
100
117
  * `~/.claude/skills` (and `./.claude/skills`) and printed `trawl: re-synced
101
118
  * skill …` lines to stderr BEFORE the JSON payload — an agent's very first
package/dist/index.js CHANGED
@@ -15,10 +15,13 @@ import { whoami } from './commands/whoami.js';
15
15
  import { ping } from './commands/ping.js';
16
16
  import { spec } from './commands/spec.js';
17
17
  import { autoUpdateInstalledSkills } from './lib/skills.js';
18
+ import { maybeSuggestSkillsInstall } from './lib/skillsNudge.js';
18
19
  import { initPostHog, captureCommand, shutdown, registerAllowedCommands } from './lib/posthog.js';
19
20
  import { classifyError, reportError, stripCommanderErrorPrefix, retryFieldsFor } from './lib/errors.js';
20
21
  import { renderPinch, pinchEnabled } from './lib/pinch.js';
21
22
  import { maybeNotifyUpdate, scheduleUpdateCheck } from './lib/updateNotifier.js';
23
+ import { getApiUrl } from './lib/config.js';
24
+ import { resolveDocsUrls, docsFooterLine } from './lib/docs.js';
22
25
  const __dirname = dirname(fileURLToPath(import.meta.url));
23
26
  const pkg = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf8'));
24
27
  /**
@@ -141,8 +144,37 @@ export function createProgram() {
141
144
  program.addCommand(logout);
142
145
  program.addCommand(telemetry);
143
146
  program.addCommand(upgrade);
147
+ // #185 — one dim line pointing at the docs, appended after EVERY `--help`
148
+ // in the tree (registering 'afterAll' on the root fires for a subcommand's
149
+ // own `--help` too — commander emits the afterAllHelp event on every
150
+ // ancestor of whichever command's outputHelp() ran, root included).
151
+ // Gated the way lib/tips.ts gates its own footer-shaped output: a pure,
152
+ // synchronous helper (docsFooterText, exported for unit testing) wrapped
153
+ // in try/catch so a broken footer can never turn a clean `--help` into a
154
+ // crash — but deliberately WITHOUT tips.ts's TTY/--json gate: `--help`
155
+ // itself never emits a --json payload (nothing here can pollute that
156
+ // channel), and a docs line inside `--help | less` is exactly the kind of
157
+ // thing worth keeping, unlike a promotional nudge.
158
+ program.addHelpText('afterAll', docsFooterText);
144
159
  return program;
145
160
  }
161
+ /**
162
+ * #185 — see the `addHelpText('afterAll', …)` call above for why this is
163
+ * unconditional (no TTY/--json gate) and why a thrown error must never
164
+ * escape: commander calls this synchronously while already writing help
165
+ * output, so a throw here would crash a `--help` invocation, the one code
166
+ * path this whole feature is not allowed to touch (see docs.ts's own
167
+ * `docsFooterLine` for the "never a guessed URL" contract this composes).
168
+ */
169
+ export function docsFooterText() {
170
+ try {
171
+ const line = docsFooterLine(resolveDocsUrls({ apiBaseUrl: getApiUrl() }).docsUrl);
172
+ return line ? chalk.dim(line) : '';
173
+ }
174
+ catch {
175
+ return '';
176
+ }
177
+ }
146
178
  /**
147
179
  * True when this module is the process entrypoint (not merely imported by a test).
148
180
  *
@@ -227,6 +259,14 @@ export function isBareInvocation(argv) {
227
259
  * overstates its own invariant is worse than no comment, because the next
228
260
  * reader trusts it.
229
261
  *
262
+ * #185 — `spec --json` (not plain `spec`) now DOES make its own bounded
263
+ * (2s), swallowed-on-failure GET for `externalDocs.url` (see spec.ts's
264
+ * `fetchExternalDocsUrl`) — this is not a regression of the "no mutation"
265
+ * invariant above (a GET mutates nothing, and any failure — including no
266
+ * egress at all — degrades to the sync host-derivation fallback, never a
267
+ * thrown error), just a second, narrower kind of side effect this guard was
268
+ * never meant to suppress in the first place.
269
+ *
230
270
  * Before this, `trawl spec --json` silently re-synced
231
271
  * `~/.claude/skills` (and `./.claude/skills`) and printed `trawl: re-synced
232
272
  * skill …` lines to stderr BEFORE the JSON payload — an agent's very first
@@ -537,6 +577,27 @@ export async function runCli(argv = process.argv) {
537
577
  catch {
538
578
  // swallow — background refresh must never affect this invocation.
539
579
  }
580
+ // #184 — "elsewhere: suggest, never install" half of the skills
581
+ // bootstrap story (lib/skills.ts's bootstrapSkillsOnLogin is the
582
+ // "install on login" half). Only on a clean exit — mirrors the
583
+ // referral tip's own success-only precedent (lib/tips.ts, called only
584
+ // after a run genuinely succeeds) and sidesteps a whole class of
585
+ // parse-error edge cases where `currentCommand` never resolved (so
586
+ // its own `--json` flag can't be read reliably here either). Running
587
+ // AFTER the command completed also means a `login` that just
588
+ // bootstrapped is already reflected in this nudge's own "any skill
589
+ // installed?" read — no separate command-name exclusion needed, and
590
+ // no double-message risk with the line `login` may have just printed.
591
+ if (process.exitCode === undefined || process.exitCode === 0) {
592
+ try {
593
+ maybeSuggestSkillsInstall({
594
+ json: Boolean(currentCommand?.opts()?.json),
595
+ });
596
+ }
597
+ catch {
598
+ // swallow — see skillsNudge.ts, this is already self-guarded too.
599
+ }
600
+ }
540
601
  }
541
602
  }
542
603
  }
package/dist/lib/api.d.ts CHANGED
@@ -95,6 +95,11 @@ export interface RequestOptions {
95
95
  export declare const api: {
96
96
  get: <T>(path: string, opts?: RequestOptions) => Promise<T>;
97
97
  publicGet: <T>(path: string, opts?: RequestOptions) => Promise<T>;
98
+ /** See `probeJson`'s own doc comment — hard-bounded, probe-only, never for
99
+ * a command's real work. Today's one caller: spec.ts's fetchExternalDocsUrl. */
100
+ probeJson: <T>(path: string, opts: {
101
+ timeoutMs: number;
102
+ }) => Promise<T | null>;
98
103
  getText: (path: string, opts?: RequestOptions) => Promise<string>;
99
104
  post: <T>(path: string, body?: unknown, opts?: RequestOptions) => Promise<T>;
100
105
  put: <T>(path: string, body?: unknown, opts?: RequestOptions) => Promise<T>;