@trawlme/cli 2.4.0 → 3.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -18,6 +18,15 @@ Requires Node.js >= 20.
18
18
  npm install -g @trawlme/cli
19
19
  ```
20
20
 
21
+ Already installed? Update the CLI itself with:
22
+
23
+ ```bash
24
+ trawl upgrade # install the latest version (old → new)
25
+ trawl upgrade --check # only report if an update is available (exit 1 if so)
26
+ ```
27
+
28
+ (`trawl upgrade` updates the CLI on your machine — distinct from `trawl scraps update <id>`, which updates a *scrap*.)
29
+
21
30
  ## Authentication
22
31
 
23
32
  Three methods:
@@ -46,8 +55,6 @@ trawl data <id> [--json] [--fresh] [--errors] Get scrap data (last persisted ru
46
55
  trawl history <id> [--json] [-n <limit>] List past runs for a scrap (newest first)
47
56
  trawl run-info <hid> [--json] Show details of a single run
48
57
  trawl trigger <id> [--watch] [--wait] [--json] Launch a scrap as a background worker (returns immediately)
49
- trawl update <id> [-t <title>] [-u <url>] [-r <request>] [-d <description>] [--cron <expr>|--no-cron] [--alert <email>|--no-alert] [--autofix|--no-autofix] [-p <json>|--params-file <path>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--force-tier <tier0|tier1|tier2|tier3|tier4>] [--json]
50
- Update an existing scrap
51
58
  trawl whoami [--json] Show the authenticated user's identity
52
59
  trawl ping [--json] Health/version handshake against the Trawl API
53
60
  ```
@@ -56,7 +63,7 @@ trawl ping [--json] Health/version handshake against
56
63
 
57
64
  `create`/`whoami`/`ping` are fully non-interactive — all three read auth only from `TRAWL_TOKEN`/the stored login token, never prompt. `trawl create` runs the AI wizard server-side (`POST /api/ai/wizard`): generate scrap code from `--prompt` via LLM, persist the scrap, trigger its FIRST run, and auto-fix on failure (default on — `--no-autofix` disables it, sending `autoFix:false`). On a successful first run (human mode) it prints a small **data sample** (item count + first-item fields + one truncated value) as proof of value — best-effort, silent if the sample can't be fetched — and points `Next step` at `trawl data <id>` (the data), with `trawl get <id>` as the secondary detail view. `--json` skips the sample fetch and prints the raw wizard payload verbatim. `success` is an honest outcome of that first run, not "did the HTTP call succeed" — a failed first run is still a 200 response (the scrap was still created; auto-fix, when enabled, retries in the background), and the CLI exits 1 in that case (both human and `--json` modes) even though `--json` always prints the raw payload verbatim. The call legitimately takes 30–250s+ server-side (AI generation + a real run), same long-run timeout as `run`/`data --fresh`/`trigger --wait` below. `trawl whoami`/`trawl ping` mirror the MCP `trawl_whoami`/`trawl_health_ping` tools as closely as the REST surface allows (`GET /api/users/me` / `GET /api/health`) — `ping`'s `--json` payload is admin-enriched (version/uptime/db) and just `{"status":"ok"}` for anyone else.
58
65
 
59
- > **`create` is NOT idempotent, and every wizard-created scrap runs on a DAILY cron by default.** A client-side timeout (exit `5`, a `NetworkError`) does not mean the wizard failed server-side — scrap creation + the first run keep going after the CLI gives up waiting, so the scrap may already exist. Run `trawl list` and look for a matching URL/title **before** retrying — a blind retry creates a DUPLICATE scrap and burns AI-generation quota a second time for the same goal. Separately, the scrap the wizard creates is scheduled to re-run every day at 07:00 UTC (`cron: "0 7 * * *"`, hardcoded server-side, unrelated to `--no-autofix`) — each of those recurring runs consumes execute quota like any other run. Review the generated scrap, then change or disable the schedule with `trawl update <id> --cron <expr>` (or `--no-cron` to disable it). Because the call can legitimately run 250s+, also confirm `TRAWL_TIMEOUT` isn't set to something tighter than `create` needs — the env var always wins over `create`'s own 300s default (see [Environment variables](#environment-variables)), so a value set for another purpose (e.g. a tight CI smoke-test budget) silently clamps `create` too; unset it or raise it before running `create`.
66
+ > **`create` is NOT idempotent, and every wizard-created scrap runs on a DAILY cron by default.** A client-side timeout (exit `5`, a `NetworkError`) does not mean the wizard failed server-side — scrap creation + the first run keep going after the CLI gives up waiting, so the scrap may already exist. Run `trawl list` and look for a matching URL/title **before** retrying — a blind retry creates a DUPLICATE scrap and burns AI-generation quota a second time for the same goal. Separately, the scrap the wizard creates is scheduled to re-run every day at 07:00 UTC (`cron: "0 7 * * *"`, hardcoded server-side, unrelated to `--no-autofix`) — each of those recurring runs consumes execute quota like any other run. Review the generated scrap, then change or disable the schedule with `trawl scraps update <id> --cron <expr>` (or `--no-cron` to disable it). Because the call can legitimately run 250s+, also confirm `TRAWL_TIMEOUT` isn't set to something tighter than `create` needs — the env var always wins over `create`'s own 300s default (see [Environment variables](#environment-variables)), so a value set for another purpose (e.g. a tight CI smoke-test budget) silently clamps `create` too; unset it or raise it before running `create`.
60
67
 
61
68
  - `list` has a short alias, `ls` (matches `trawl --help`'s `list|ls`).
62
69
  - `history` lists past runs (newest first); `run-info <hid>` shows details of a single run from that history.
@@ -70,6 +77,7 @@ trawl ping [--json] Health/version handshake against
70
77
 
71
78
  ```
72
79
  trawl scraps create -t <title> [-u <url>] [-r <request>] [-d <description>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--json]
80
+ trawl scraps update <id> [-t <title>] [-u <url>] [-r <request>] [-d <description>] [--cron <expr>|--no-cron] [--alert <email>|--no-alert] [--autofix|--no-autofix] [-p <json>|--params-file <path>] [--tier <tier0|tier1|tier2|tier3|tier4>] [--force-tier <tier0|tier1|tier2|tier3|tier4>] [--json]
73
81
  trawl scraps watch <id> [--json]
74
82
  trawl scraps doctor <id> [--json] [--autofix]
75
83
  trawl scraps autofix <id> [--json]
@@ -134,6 +142,14 @@ trawl telemetry off [--json] Disable usage telemetry
134
142
  trawl telemetry status [--json] Show current state, telemetry ID, and opt-out instructions
135
143
  ```
136
144
 
145
+ ### Upgrade the CLI
146
+
147
+ ```
148
+ trawl upgrade [--check] [--json] Update @trawlme/cli to the latest published version
149
+ ```
150
+
151
+ Compares the running version against the npm registry and runs `npm install -g @trawlme/cli@latest` when a newer one exists. `--check` reports without installing (exit `1` when an update is available). A permission error on a system-owned global prefix is surfaced with the manual command.
152
+
137
153
  ## Telemetry
138
154
 
139
155
  `@trawlme/cli` collects anonymous usage data to help us improve the CLI based on real usage patterns. Telemetry is enabled by default.
@@ -30,8 +30,7 @@ export interface WizardResponse {
30
30
  * 'UTC'` — unconditionally, regardless of `--prompt`/`--no-autofix`.
31
31
  * Present on the scrap object returned here (the wizard controller
32
32
  * passes the created scrap straight through, no stripping) — change or
33
- * disable it with `trawl update <id> --cron <expr>` / `--no-cron` (#120
34
- * — `update` is now a promoted top-level verb, same as `get`/`run`/…).
33
+ * disable it with `trawl scraps update <id> --cron <expr>` / `--no-cron`.
35
34
  */
36
35
  cron?: string | null;
37
36
  cronTimezone?: string;
@@ -140,7 +140,6 @@ export declare function pollRunProgress(id: string, before: BeforeRunState | und
140
140
  }): Promise<void>;
141
141
  export declare function attachListCommand(parent: Command, attachOpts?: AttachOptions): Command;
142
142
  export declare function attachGetCommand(parent: Command, attachOpts?: AttachOptions): Command;
143
- export declare function attachUpdateCommand(parent: Command, attachOpts?: AttachOptions): Command;
144
143
  export declare function attachRunCommand(parent: Command, attachOpts?: AttachOptions): Command;
145
144
  export declare function attachDataCommand(parent: Command, attachOpts?: AttachOptions): Command;
146
145
  export declare function attachHistoryCommand(parent: Command, attachOpts?: AttachOptions): Command;
@@ -583,132 +583,129 @@ scraps
583
583
  if (refused)
584
584
  process.exitCode = 1;
585
585
  });
586
- // update — promoted to a top-level verb (#108/#120)
587
- export function attachUpdateCommand(parent, attachOpts = {}) {
588
- return parent
589
- .command('update <id>', attachOpts)
590
- .description('Update an existing scrap')
591
- .option('-t, --title <title>', 'New title')
592
- .option('-u, --url <url>', 'New target URL')
593
- .option('-r, --request <request>', 'New request')
594
- .option('-d, --description <text>', 'New description')
595
- .option('--cron <expression>', 'Cron expression (empty string to disable)')
596
- .option('--no-cron', 'Disable cron (set to null)')
597
- .option('--alert <email>', 'Failure alert email (empty string to clear)')
598
- .option('--no-alert', 'Disable failure alert email (set to null)')
599
- .option('--autofix', 'Enable AI Fix (auto-recovery on selector breakage)')
600
- .option('--no-autofix', 'Disable AI Fix')
601
- .option('-p, --params <json>', 'Runtime params as JSON array of objects (e.g. \'[{"TRAWL.paramName":"value"}]\')')
602
- .option('--params-file <path>', 'Runtime params from a JSON file')
603
- .option('--tier <tier>', `Force proxy tier (${VALID_TIERS.join('|')})`)
604
- .option('--force-tier <tier>', `Raise the proxy-tier ceiling PAST the auto-cap (${VALID_TIERS.join('|')}) — history-gated: may be refused or cost more`)
605
- .option('--json', 'Output as JSON')
606
- .action(async (id, opts) => {
607
- validateObjectId(id);
608
- if (opts.tier !== undefined && !VALID_TIERS.includes(opts.tier)) {
609
- usageError(`Invalid --tier "${opts.tier}" (allowed: ${VALID_TIERS.join(', ')})`, { json: opts.json });
610
- return;
586
+ // update
587
+ scraps
588
+ .command('update <id>')
589
+ .description('Update an existing scrap')
590
+ .option('-t, --title <title>', 'New title')
591
+ .option('-u, --url <url>', 'New target URL')
592
+ .option('-r, --request <request>', 'New request')
593
+ .option('-d, --description <text>', 'New description')
594
+ .option('--cron <expression>', 'Cron expression (empty string to disable)')
595
+ .option('--no-cron', 'Disable cron (set to null)')
596
+ .option('--alert <email>', 'Failure alert email (empty string to clear)')
597
+ .option('--no-alert', 'Disable failure alert email (set to null)')
598
+ .option('--autofix', 'Enable AI Fix (auto-recovery on selector breakage)')
599
+ .option('--no-autofix', 'Disable AI Fix')
600
+ .option('-p, --params <json>', 'Runtime params as JSON array of objects (e.g. \'[{"TRAWL.paramName":"value"}]\')')
601
+ .option('--params-file <path>', 'Runtime params from a JSON file')
602
+ .option('--tier <tier>', `Force proxy tier (${VALID_TIERS.join('|')})`)
603
+ .option('--force-tier <tier>', `Raise the proxy-tier ceiling PAST the auto-cap (${VALID_TIERS.join('|')}) — history-gated: may be refused or cost more`)
604
+ .option('--json', 'Output as JSON')
605
+ .action(async (id, opts) => {
606
+ validateObjectId(id);
607
+ if (opts.tier !== undefined && !VALID_TIERS.includes(opts.tier)) {
608
+ usageError(`Invalid --tier "${opts.tier}" (allowed: ${VALID_TIERS.join(', ')})`, { json: opts.json });
609
+ return;
610
+ }
611
+ if (opts.forceTier !== undefined && !VALID_TIERS.includes(opts.forceTier)) {
612
+ usageError(`Invalid --force-tier "${opts.forceTier}" (allowed: ${VALID_TIERS.join(', ')})`, { json: opts.json });
613
+ return;
614
+ }
615
+ const body = {};
616
+ if (opts.title !== undefined)
617
+ body.title = opts.title;
618
+ if (opts.url !== undefined)
619
+ body.url = opts.url;
620
+ if (opts.request !== undefined)
621
+ body.request = opts.request;
622
+ if (opts.description !== undefined)
623
+ body.description = opts.description;
624
+ if (opts.cron === false)
625
+ body.cron = null;
626
+ else if (typeof opts.cron === 'string')
627
+ body.cron = opts.cron === '' ? null : opts.cron;
628
+ if (opts.alert === false)
629
+ body.alert = null;
630
+ else if (typeof opts.alert === 'string')
631
+ body.alert = opts.alert === '' ? null : opts.alert;
632
+ if (opts.autofix === true)
633
+ body.autoFix = true;
634
+ else if (opts.autofix === false)
635
+ body.autoFix = false;
636
+ if (opts.params !== undefined || opts.paramsFile !== undefined) {
637
+ let raw;
638
+ if (opts.paramsFile) {
639
+ const { readFileSync } = await import('fs');
640
+ raw = readFileSync(opts.paramsFile, 'utf8');
611
641
  }
612
- if (opts.forceTier !== undefined && !VALID_TIERS.includes(opts.forceTier)) {
613
- usageError(`Invalid --force-tier "${opts.forceTier}" (allowed: ${VALID_TIERS.join(', ')})`, { json: opts.json });
614
- return;
642
+ else {
643
+ raw = opts.params;
615
644
  }
616
- const body = {};
617
- if (opts.title !== undefined)
618
- body.title = opts.title;
619
- if (opts.url !== undefined)
620
- body.url = opts.url;
621
- if (opts.request !== undefined)
622
- body.request = opts.request;
623
- if (opts.description !== undefined)
624
- body.description = opts.description;
625
- if (opts.cron === false)
626
- body.cron = null;
627
- else if (typeof opts.cron === 'string')
628
- body.cron = opts.cron === '' ? null : opts.cron;
629
- if (opts.alert === false)
630
- body.alert = null;
631
- else if (typeof opts.alert === 'string')
632
- body.alert = opts.alert === '' ? null : opts.alert;
633
- if (opts.autofix === true)
634
- body.autoFix = true;
635
- else if (opts.autofix === false)
636
- body.autoFix = false;
637
- if (opts.params !== undefined || opts.paramsFile !== undefined) {
638
- let raw;
639
- if (opts.paramsFile) {
640
- const { readFileSync } = await import('fs');
641
- raw = readFileSync(opts.paramsFile, 'utf8');
642
- }
643
- else {
644
- raw = opts.params;
645
- }
646
- let parsed;
647
- try {
648
- parsed = JSON.parse(raw);
649
- }
650
- catch (e) {
651
- usageError(`Invalid JSON for --params: ${e.message}`, { json: opts.json });
652
- return;
653
- }
654
- if (!Array.isArray(parsed)) {
655
- usageError('--params must be a JSON array of objects', { json: opts.json });
656
- return;
657
- }
658
- body.params = parsed;
645
+ let parsed;
646
+ try {
647
+ parsed = JSON.parse(raw);
659
648
  }
660
- if (opts.tier !== undefined)
661
- body.proxyTier = opts.tier;
662
- if (opts.forceTier !== undefined) {
663
- // Raise the ceiling; also start the run at that tier unless --tier says otherwise.
664
- body.proxyMaxTier = opts.forceTier;
665
- if (opts.tier === undefined)
666
- body.proxyTier = opts.forceTier;
649
+ catch (e) {
650
+ usageError(`Invalid JSON for --params: ${e.message}`, { json: opts.json });
651
+ return;
667
652
  }
668
- if (Object.keys(body).length === 0) {
669
- if (opts.json) {
670
- process.exitCode = reportError(new UsageError('Nothing to update. Provide at least one option.'), { json: true });
671
- return;
672
- }
673
- console.log(chalk.yellow('Nothing to update. Provide at least one option.'));
653
+ if (!Array.isArray(parsed)) {
654
+ usageError('--params must be a JSON array of objects', { json: opts.json });
674
655
  return;
675
656
  }
676
- const data = await spin(() => api.put(`/api/scraps/${id}`, body), {
677
- text: 'Updating scrap…',
678
- successText: (d) => `Scrap updated: ${chalk.bold(d._id)}`,
679
- });
680
- // #1559 / #86 findings 4b/5 — surface the effective tier + clamp/refuse
681
- // reason (fixes the silent-clamp: the server may persist a lower tier
682
- // than requested), and NEVER echo the requested value as applied when
683
- // the server doesn't confirm it (old-server fallback below).
684
- const tierWasRequested = opts.tier !== undefined || opts.forceTier !== undefined;
685
- warnIfUnconfirmedTier(data, tierWasRequested, id);
686
- const refused = Boolean(data._tierOverride?.refused);
657
+ body.params = parsed;
658
+ }
659
+ if (opts.tier !== undefined)
660
+ body.proxyTier = opts.tier;
661
+ if (opts.forceTier !== undefined) {
662
+ // Raise the ceiling; also start the run at that tier unless --tier says otherwise.
663
+ body.proxyMaxTier = opts.forceTier;
664
+ if (opts.tier === undefined)
665
+ body.proxyTier = opts.forceTier;
666
+ }
667
+ if (Object.keys(body).length === 0) {
687
668
  if (opts.json) {
688
- if (refused) {
689
- process.exitCode = reportTierRefusal(data, true);
690
- return;
691
- }
692
- json(withTierUnconfirmed(data, tierWasRequested));
669
+ process.exitCode = reportError(new UsageError('Nothing to update. Provide at least one option.'), { json: true });
693
670
  return;
694
671
  }
695
- const shown = data;
696
- for (const key of Object.keys(body)) {
697
- // Tier keys are rendered exclusively by renderTierOverrideHuman /
698
- // warnIfUnconfirmedTier above never echo them here, whether or not
699
- // _tierOverride came back (an old-server echo of the REQUESTED value
700
- // is exactly the silent-clamp lie #1559 fixed).
701
- if (key === 'proxyTier' || key === 'proxyMaxTier')
702
- continue;
703
- const src = key in shown ? shown[key] : body[key];
704
- console.log(chalk.dim(` ${key}: `) + String(src ?? '—'));
705
- }
706
- renderTierOverrideHuman(data);
707
- if (refused)
708
- process.exitCode = 1;
672
+ console.log(chalk.yellow('Nothing to update. Provide at least one option.'));
673
+ return;
674
+ }
675
+ const data = await spin(() => api.put(`/api/scraps/${id}`, body), {
676
+ text: 'Updating scrap…',
677
+ successText: (d) => `Scrap updated: ${chalk.bold(d._id)}`,
709
678
  });
710
- }
711
- attachUpdateCommand(scraps, { hidden: true });
679
+ // #1559 / #86 findings 4b/5 — surface the effective tier + clamp/refuse
680
+ // reason (fixes the silent-clamp: the server may persist a lower tier
681
+ // than requested), and NEVER echo the requested value as applied when
682
+ // the server doesn't confirm it (old-server fallback below).
683
+ const tierWasRequested = opts.tier !== undefined || opts.forceTier !== undefined;
684
+ warnIfUnconfirmedTier(data, tierWasRequested, id);
685
+ const refused = Boolean(data._tierOverride?.refused);
686
+ if (opts.json) {
687
+ if (refused) {
688
+ process.exitCode = reportTierRefusal(data, true);
689
+ return;
690
+ }
691
+ json(withTierUnconfirmed(data, tierWasRequested));
692
+ return;
693
+ }
694
+ const shown = data;
695
+ for (const key of Object.keys(body)) {
696
+ // Tier keys are rendered exclusively by renderTierOverrideHuman /
697
+ // warnIfUnconfirmedTier above — never echo them here, whether or not
698
+ // _tierOverride came back (an old-server echo of the REQUESTED value
699
+ // is exactly the silent-clamp lie #1559 fixed).
700
+ if (key === 'proxyTier' || key === 'proxyMaxTier')
701
+ continue;
702
+ const src = key in shown ? shown[key] : body[key];
703
+ console.log(chalk.dim(` ${key}: `) + String(src ?? '—'));
704
+ }
705
+ renderTierOverrideHuman(data);
706
+ if (refused)
707
+ process.exitCode = 1;
708
+ });
712
709
  // run — promoted to a top-level verb (#108)
713
710
  export function attachRunCommand(parent, attachOpts = {}) {
714
711
  return parent
@@ -0,0 +1,2 @@
1
+ import { Command } from 'commander';
2
+ export declare const upgrade: Command;
@@ -0,0 +1,58 @@
1
+ import { Command } from 'commander';
2
+ import chalk from 'chalk';
3
+ import { execFile } from 'node:child_process';
4
+ import { promisify } from 'node:util';
5
+ import { json } from '../lib/format.js';
6
+ import { PKG_NAME, currentVersion, fetchLatestVersion } from '../lib/version.js';
7
+ const execFileP = promisify(execFile);
8
+ export const upgrade = new Command('upgrade')
9
+ .description('Update the trawl CLI itself to the latest published version')
10
+ .option('--check', 'Only report whether an update is available — do not install')
11
+ .option('--json', 'Output as JSON')
12
+ .action(async (opts) => {
13
+ const current = currentVersion();
14
+ const latest = await fetchLatestVersion();
15
+ const upToDate = current === latest;
16
+ if (upToDate) {
17
+ if (opts.json)
18
+ return json({ package: PKG_NAME, current, latest, upToDate: true, upgraded: false });
19
+ console.log(`${chalk.green('✓')} Already on the latest version (${chalk.bold(current)}).`);
20
+ return;
21
+ }
22
+ if (opts.check) {
23
+ if (opts.json)
24
+ return json({ package: PKG_NAME, current, latest, upToDate: false, upgraded: false });
25
+ console.log(`${chalk.yellow('↑')} Update available: ${chalk.bold(current)} → ${chalk.bold(latest)}. Run ${chalk.cyan('trawl upgrade')} to install.`);
26
+ // Honest exit code so a script can gate on "is an update available".
27
+ process.exitCode = 1;
28
+ return;
29
+ }
30
+ // Install the latest globally. This shells out to the same npm the user
31
+ // installed the CLI with; a permission error (EACCES on a system-owned
32
+ // global prefix) is surfaced with the manual command rather than swallowed.
33
+ if (!opts.json)
34
+ console.log(chalk.dim(`Upgrading ${PKG_NAME}: ${current} → ${latest}…`));
35
+ try {
36
+ await execFileP('npm', ['install', '-g', `${PKG_NAME}@latest`], { timeout: 180_000 });
37
+ }
38
+ catch (err) {
39
+ const e = err;
40
+ const hint = e.code === 'ENOENT'
41
+ ? 'npm was not found on PATH.'
42
+ : /EACCES|permission/i.test(String(e.stderr ?? e.message))
43
+ ? 'permission denied on the global npm prefix — retry with sudo, or use a Node version manager.'
44
+ : (e.stderr?.trim() || e.message);
45
+ if (opts.json) {
46
+ console.log(JSON.stringify({ error: { message: `upgrade failed: ${hint}`, kind: 'upgrade_failed' } }));
47
+ }
48
+ else {
49
+ console.error(chalk.red(`✗ Upgrade failed: ${hint}`));
50
+ console.error(chalk.dim(` Install manually: npm install -g ${PKG_NAME}@latest`));
51
+ }
52
+ process.exitCode = 1;
53
+ return;
54
+ }
55
+ if (opts.json)
56
+ return json({ package: PKG_NAME, current, latest, upToDate: false, upgraded: true });
57
+ console.log(`${chalk.green('✓')} Upgraded ${PKG_NAME}: ${chalk.bold(current)} → ${chalk.bold(latest)}.`);
58
+ });
package/dist/index.js CHANGED
@@ -4,10 +4,11 @@ import { readFileSync, realpathSync } from 'node:fs';
4
4
  import { fileURLToPath } from 'node:url';
5
5
  import { dirname, join } from 'node:path';
6
6
  import { login, logout } from './commands/login.js';
7
- import { scraps, attachListCommand, attachGetCommand, attachRunCommand, attachDataCommand, attachHistoryCommand, attachRunInfoCommand, attachTriggerCommand, attachUpdateCommand, } from './commands/scraps.js';
7
+ import { scraps, attachListCommand, attachGetCommand, attachRunCommand, attachDataCommand, attachHistoryCommand, attachRunInfoCommand, attachTriggerCommand, } from './commands/scraps.js';
8
8
  import { skills } from './commands/skills.js';
9
9
  import { telemetry } from './commands/telemetry.js';
10
10
  import { token } from './commands/token.js';
11
+ import { upgrade } from './commands/upgrade.js';
11
12
  import { create } from './commands/create.js';
12
13
  import { whoami } from './commands/whoami.js';
13
14
  import { ping } from './commands/ping.js';
@@ -15,6 +16,7 @@ import { autoUpdateInstalledSkills } from './lib/skills.js';
15
16
  import { initPostHog, captureCommand, shutdown, registerAllowedCommands } from './lib/posthog.js';
16
17
  import { classifyError, reportError } from './lib/errors.js';
17
18
  import { renderPinch, pinchEnabled } from './lib/pinch.js';
19
+ import { maybeNotifyUpdate, scheduleUpdateCheck } from './lib/updateNotifier.js';
18
20
  const __dirname = dirname(fileURLToPath(import.meta.url));
19
21
  const pkg = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf8'));
20
22
  /**
@@ -108,14 +110,11 @@ export function createProgram() {
108
110
  attachHistoryCommand(program);
109
111
  attachRunInfoCommand(program);
110
112
  attachTriggerCommand(program);
111
- attachUpdateCommand(program);
112
113
  program.addCommand(whoami);
113
114
  program.addCommand(ping);
114
115
  // Management (#108) — human/CI surface, grouped below. `scraps` still
115
- // holds every pre-#108 management command (create/delete/banner/
116
- // watch/account.*/session.*/doctor/autofix/snapshot) exactly as before
117
- // `update` is now a promoted core verb (#120), still reachable (hidden)
118
- // under `scraps` for backward compatibility.
116
+ // holds every pre-#108 management command (create/update/delete/banner/
117
+ // watch/account.*/session.*/doctor/autofix/snapshot) exactly as before.
119
118
  program.commandsGroup(MANAGEMENT_GROUP);
120
119
  program.addCommand(scraps);
121
120
  program.addCommand(skills);
@@ -123,6 +122,7 @@ export function createProgram() {
123
122
  program.addCommand(logout);
124
123
  program.addCommand(token);
125
124
  program.addCommand(telemetry);
125
+ program.addCommand(upgrade);
126
126
  return program;
127
127
  }
128
128
  /**
@@ -369,6 +369,23 @@ export async function runCli(argv = process.argv) {
369
369
  // Flush + close telemetry before the process exits. A `process.on('exit')`
370
370
  // handler cannot reliably run async work, so this must happen here.
371
371
  await shutdown();
372
+ // Passive "update available" notifier (#129) — never on a help/version
373
+ // query, and each half individually guarded so a notifier bug can never
374
+ // turn a successful command into a failure or change its exit code.
375
+ if (!isHelpOrVersion(argv)) {
376
+ try {
377
+ maybeNotifyUpdate();
378
+ }
379
+ catch {
380
+ // swallow — see updateNotifier.ts, this is already self-guarded too.
381
+ }
382
+ try {
383
+ scheduleUpdateCheck();
384
+ }
385
+ catch {
386
+ // swallow — background refresh must never affect this invocation.
387
+ }
388
+ }
372
389
  }
373
390
  }
374
391
  if (isEntryPoint(process.argv[1], import.meta.url)) {
@@ -4,6 +4,11 @@ interface TrawlConfig {
4
4
  token: string;
5
5
  telemetry: boolean;
6
6
  telemetryUserId: string;
7
+ /** Opt-out switch for the passive "update available" notifier
8
+ * (src/lib/updateNotifier.ts). Optional — absent/undefined means enabled;
9
+ * only an explicit `false` disables it. No default entry needed since it's
10
+ * optional. (#129) */
11
+ updateNotifier?: boolean;
7
12
  }
8
13
  declare const config: Conf<TrawlConfig>;
9
14
  /**
@@ -0,0 +1 @@
1
+ export {};
@@ -0,0 +1,30 @@
1
+ /**
2
+ * Standalone worker script spawned DETACHED by scheduleUpdateCheck()
3
+ * (updateNotifier.ts) to refresh the update-check cache in the background —
4
+ * this never runs on the CLI's hot path. Invoked as
5
+ * `node dist/lib/updateCheckWorker.js <cachePath>`, it queries the npm
6
+ * registry for the latest published version, writes
7
+ * `{ latest, checkedAt }` to `<cachePath>`, and exits. The NEXT `trawl`
8
+ * invocation picks up the refreshed cache — this run never waits on it.
9
+ *
10
+ * Any failure (npm missing, registry unreachable, cache dir unwritable) exits
11
+ * silently (code 0) — a failed background refresh must never surface to the
12
+ * user; the cache simply stays stale until the next stale-check fires.
13
+ */
14
+ import { execFile } from 'node:child_process';
15
+ import { writeFileSync } from 'node:fs';
16
+ import { PKG_NAME } from './version.js';
17
+ const cachePath = process.argv[2];
18
+ if (cachePath) {
19
+ execFile('npm', ['view', PKG_NAME, 'version'], { timeout: 30_000 }, (err, stdout) => {
20
+ if (!err) {
21
+ try {
22
+ writeFileSync(cachePath, JSON.stringify({ latest: String(stdout).trim(), checkedAt: Date.now() }));
23
+ }
24
+ catch {
25
+ // Best-effort — a write failure just means the next invocation retries.
26
+ }
27
+ }
28
+ process.exit(0);
29
+ });
30
+ }
@@ -0,0 +1,20 @@
1
+ /** The cache file lives beside the CLI's own `conf` config file — same
2
+ * directory, no extra config-dir resolution logic to duplicate. */
3
+ export declare function updateCachePath(): string;
4
+ /**
5
+ * Synchronous, instant — safe to call on every invocation's hot path. Prints
6
+ * ONE line to stderr when a cached "latest" version is a valid semver
7
+ * strictly greater than the running version, and every gate holds:
8
+ * an interactive stderr TTY, no `--json` flag, not a help/version query, and
9
+ * not opted out (env or config). Any error (bad cache, etc.) is swallowed —
10
+ * this must never throw.
11
+ */
12
+ export declare function maybeNotifyUpdate(argv?: string[]): void;
13
+ /**
14
+ * Non-blocking background refresh. When the cache is missing or older than
15
+ * CHECK_INTERVAL_MS, spawns a fully detached child (updateCheckWorker.ts)
16
+ * that queries npm and writes the refreshed cache, then exits — this
17
+ * invocation never waits on it (`.unref()`), so the refreshed cache is only
18
+ * ever used by the NEXT invocation. Never throws.
19
+ */
20
+ export declare function scheduleUpdateCheck(): void;
@@ -0,0 +1,150 @@
1
+ /**
2
+ * Passive "update available" notifier (#129) — follows the `update-notifier`
3
+ * model: notify from a CACHE, refresh the cache in the BACKGROUND, never
4
+ * block or slow down the command it hitches a ride on.
5
+ *
6
+ * - `maybeNotifyUpdate()` is synchronous and instant — it only ever reads a
7
+ * cache file that already exists on disk and prints at most one line to
8
+ * stderr. It never talks to the network and never throws.
9
+ * - `scheduleUpdateCheck()` refreshes that cache for the NEXT invocation by
10
+ * spawning a fully detached, unref'd child process
11
+ * (src/lib/updateCheckWorker.ts) — this invocation never awaits it.
12
+ *
13
+ * Both are called from src/index.ts's `runCli()` finally block, after
14
+ * `shutdown()`, wrapped so a notifier failure can never turn a successful
15
+ * command into a failed one.
16
+ */
17
+ import { readFileSync } from 'node:fs';
18
+ import { dirname, join } from 'node:path';
19
+ import { fileURLToPath } from 'node:url';
20
+ import { spawn } from 'node:child_process';
21
+ import chalk from 'chalk';
22
+ import config from './config.js';
23
+ import { currentVersion } from './version.js';
24
+ /** Refresh cadence for the background worker — a cache younger than this is
25
+ * left alone; older (or missing) triggers a detached refresh. */
26
+ const CHECK_INTERVAL_MS = 24 * 60 * 60 * 1000;
27
+ /** Sibling compiled worker script (dist/lib/updateCheckWorker.js next to this
28
+ * module's own dist/lib/updateNotifier.js) — resolved relative to THIS
29
+ * module so it works regardless of cwd or how the CLI was installed. */
30
+ const WORKER_PATH = fileURLToPath(new URL('./updateCheckWorker.js', import.meta.url));
31
+ /** The cache file lives beside the CLI's own `conf` config file — same
32
+ * directory, no extra config-dir resolution logic to duplicate. */
33
+ export function updateCachePath() {
34
+ return join(dirname(config.path), 'update-check.json');
35
+ }
36
+ function readCache() {
37
+ try {
38
+ const raw = readFileSync(updateCachePath(), 'utf8');
39
+ const parsed = JSON.parse(raw);
40
+ if (typeof parsed.latest !== 'string' || typeof parsed.checkedAt !== 'number')
41
+ return undefined;
42
+ return { latest: parsed.latest, checkedAt: parsed.checkedAt };
43
+ }
44
+ catch {
45
+ // Missing file, corrupt JSON, unreadable — treat all the same: no cache.
46
+ return undefined;
47
+ }
48
+ }
49
+ /** Opted out via env (presence-based, mirrors NO_COLOR) or an explicit
50
+ * `false` config value. Absent/undefined config key means enabled. */
51
+ function isOptedOut() {
52
+ if (process.env['TRAWL_NO_UPDATE_NOTIFIER'] !== undefined)
53
+ return true;
54
+ return config.get('updateNotifier') === false;
55
+ }
56
+ const SEMVER_RE = /^\d+\.\d+\.\d+$/;
57
+ function isValidSemver(v) {
58
+ return SEMVER_RE.test(v);
59
+ }
60
+ /** Plain MAJOR.MINOR.PATCH comparison — no prerelease/build-metadata support,
61
+ * consistent with this package's own version scheme (see version.ts). */
62
+ function isGreaterSemver(a, b) {
63
+ if (!isValidSemver(a) || !isValidSemver(b))
64
+ return false;
65
+ const pa = a.split('.').map(Number);
66
+ const pb = b.split('.').map(Number);
67
+ for (let i = 0; i < 3; i++) {
68
+ if (pa[i] > pb[i])
69
+ return true;
70
+ if (pa[i] < pb[i])
71
+ return false;
72
+ }
73
+ return false;
74
+ }
75
+ /** Best-effort `--json` argv scan — same "argv scan, never trust flag
76
+ * values" caveat as index.ts's hasJsonFlag: positional values never equal
77
+ * the literal `--json` string. Kept local (not imported from index.ts) so
78
+ * this module has no reverse dependency on the entrypoint. */
79
+ function hasJsonFlag(argv) {
80
+ return argv.includes('--json');
81
+ }
82
+ /** Same shape as index.ts's isHelpOrVersion (bare invocation, `help`,
83
+ * -h/--help/-V/--version) — kept local for the same reason as hasJsonFlag
84
+ * above. Belt-and-suspenders: src/index.ts already skips calling this
85
+ * function entirely on a help/version invocation. */
86
+ function isHelpOrVersionInvocation(argv) {
87
+ if (argv.some((a) => a === '-h' || a === '--help' || a === '-V' || a === '--version'))
88
+ return true;
89
+ const args = argv.slice(2);
90
+ if (args.length === 0)
91
+ return true;
92
+ if (args[0] === 'help')
93
+ return true;
94
+ return false;
95
+ }
96
+ /**
97
+ * Synchronous, instant — safe to call on every invocation's hot path. Prints
98
+ * ONE line to stderr when a cached "latest" version is a valid semver
99
+ * strictly greater than the running version, and every gate holds:
100
+ * an interactive stderr TTY, no `--json` flag, not a help/version query, and
101
+ * not opted out (env or config). Any error (bad cache, etc.) is swallowed —
102
+ * this must never throw.
103
+ */
104
+ export function maybeNotifyUpdate(argv = process.argv) {
105
+ try {
106
+ if (!process.stderr.isTTY)
107
+ return;
108
+ if (hasJsonFlag(argv))
109
+ return;
110
+ if (isHelpOrVersionInvocation(argv))
111
+ return;
112
+ if (isOptedOut())
113
+ return;
114
+ const cache = readCache();
115
+ if (!cache)
116
+ return;
117
+ const current = currentVersion();
118
+ if (!isGreaterSemver(cache.latest, current))
119
+ return;
120
+ console.error(chalk.dim(`↑ Update available: ${current} → ${cache.latest}. Run \`trawl upgrade\`.`));
121
+ }
122
+ catch {
123
+ // Never let the notifier turn a successful command into a failure.
124
+ }
125
+ }
126
+ /**
127
+ * Non-blocking background refresh. When the cache is missing or older than
128
+ * CHECK_INTERVAL_MS, spawns a fully detached child (updateCheckWorker.ts)
129
+ * that queries npm and writes the refreshed cache, then exits — this
130
+ * invocation never waits on it (`.unref()`), so the refreshed cache is only
131
+ * ever used by the NEXT invocation. Never throws.
132
+ */
133
+ export function scheduleUpdateCheck() {
134
+ try {
135
+ if (isOptedOut())
136
+ return;
137
+ const cache = readCache();
138
+ const isStale = !cache || Date.now() - cache.checkedAt > CHECK_INTERVAL_MS;
139
+ if (!isStale)
140
+ return;
141
+ const child = spawn(process.execPath, [WORKER_PATH, updateCachePath()], {
142
+ detached: true,
143
+ stdio: 'ignore',
144
+ });
145
+ child.unref();
146
+ }
147
+ catch {
148
+ // A failed background-refresh spawn must never affect the caller.
149
+ }
150
+ }
@@ -0,0 +1,15 @@
1
+ /**
2
+ * Shared version plumbing — the package name + "what version am I" + "what's
3
+ * published" primitives, used by BOTH `trawl upgrade` (src/commands/upgrade.ts)
4
+ * and the passive update notifier (src/lib/updateNotifier.ts). Extracted so
5
+ * neither duplicates the npm-registry lookup or the package.json read (#129).
6
+ */
7
+ export declare const PKG_NAME = "@trawlme/cli";
8
+ /** The version currently RUNNING — read from this package's own package.json.
9
+ * (At publish time semantic-release stamps the real version into the tarball's
10
+ * package.json, so at runtime this is the installed version; the repo's
11
+ * checked-in value is vestigial.) */
12
+ export declare function currentVersion(): string;
13
+ /** Latest published version from the npm registry. Throws with a friendly
14
+ * message when npm is missing or the registry is unreachable. */
15
+ export declare function fetchLatestVersion(): Promise<string>;
@@ -0,0 +1,36 @@
1
+ import { execFile } from 'node:child_process';
2
+ import { promisify } from 'node:util';
3
+ import { readFileSync } from 'node:fs';
4
+ import { fileURLToPath } from 'node:url';
5
+ import { dirname, join } from 'node:path';
6
+ const execFileP = promisify(execFile);
7
+ /**
8
+ * Shared version plumbing — the package name + "what version am I" + "what's
9
+ * published" primitives, used by BOTH `trawl upgrade` (src/commands/upgrade.ts)
10
+ * and the passive update notifier (src/lib/updateNotifier.ts). Extracted so
11
+ * neither duplicates the npm-registry lookup or the package.json read (#129).
12
+ */
13
+ export const PKG_NAME = '@trawlme/cli';
14
+ /** The version currently RUNNING — read from this package's own package.json.
15
+ * (At publish time semantic-release stamps the real version into the tarball's
16
+ * package.json, so at runtime this is the installed version; the repo's
17
+ * checked-in value is vestigial.) */
18
+ export function currentVersion() {
19
+ const here = dirname(fileURLToPath(import.meta.url)); // dist/lib
20
+ const pkg = JSON.parse(readFileSync(join(here, '..', '..', 'package.json'), 'utf8'));
21
+ return pkg.version;
22
+ }
23
+ /** Latest published version from the npm registry. Throws with a friendly
24
+ * message when npm is missing or the registry is unreachable. */
25
+ export async function fetchLatestVersion() {
26
+ try {
27
+ const { stdout } = await execFileP('npm', ['view', PKG_NAME, 'version'], { timeout: 30_000 });
28
+ return stdout.trim();
29
+ }
30
+ catch (err) {
31
+ const e = err;
32
+ if (e.code === 'ENOENT')
33
+ throw new Error('npm was not found on PATH — install Node.js/npm, then run: npm install -g @trawlme/cli@latest');
34
+ throw new Error(`could not reach the npm registry to check for updates (${e.message})`);
35
+ }
36
+ }
@@ -20,7 +20,7 @@ agent should never need them.)
20
20
 
21
21
  ## Core commands (agent + human)
22
22
 
23
- These eleven commands are the CLI's agent+human surface — `--json` is
23
+ These ten commands are the CLI's agent+human surface — `--json` is
24
24
  first-class on every one, and none of them ever blocks on a prompt (see
25
25
  [Non-interactive contract](#non-interactive-contract) below):
26
26
 
@@ -34,8 +34,6 @@ trawl data <id> [--json] [--fresh] [--errors] Get scrap data (last persisted ru
34
34
  trawl history <id> [--json] [-n <limit>] List past runs for a scrap
35
35
  trawl run-info <hid> [--json] Show details of a single run
36
36
  trawl trigger <id> [--watch] [--wait] [--json] Launch a scrap as a background worker
37
- trawl update <id> [-t <title>] [-u <url>] [-r <request>] [-d <description>] [--cron <expr>|--no-cron] [--alert <email>|--no-alert] [--autofix|--no-autofix] [-p <json>|--params-file <path>] [--tier <t>] [--force-tier <t>] [--json]
38
- Update an existing scrap
39
37
  trawl whoami [--json] Show the authenticated user's identity
40
38
  trawl ping [--json] Health/version handshake against the Trawl API
41
39
  ```
@@ -61,7 +59,7 @@ default.
61
59
  > scheduled to re-run every day at 07:00 UTC by default
62
60
  > (`cron: "0 7 * * *"`, unrelated to `--no-autofix`) — each recurring run
63
61
  > consumes execute quota. Disable or change it once you've reviewed the
64
- > scrap: `trawl update <id> --no-cron` (or `--cron <expr>`). Finally,
62
+ > scrap: `trawl scraps update <id> --no-cron` (or `--cron <expr>`). Finally,
65
63
  > if `TRAWL_TIMEOUT` is set globally for a tighter budget than 300s, it
66
64
  > clamps `create`'s ceiling too (env always wins) — unset it or raise it
67
65
  > before calling `create`.
@@ -71,10 +69,9 @@ default.
71
69
  > a hidden alias. Prefer the bare top-level form above; it's what
72
70
  > `trawl --help` now shows.
73
71
 
74
- For the full flag reference (tier overrides on `scraps create` (management)
75
- and the core `update` verb above note the core `create` verb has no
76
- `--tier` of its own, unlike `scraps create`, the `--watch` polling mechanics,
77
- retention/regression semantics on `data`, …)
72
+ For the full flag reference (tier overrides on `scraps create`/`scraps
73
+ update` the core `create` verb above has no `--tier` of its own, the
74
+ `--watch` polling mechanics, retention/regression semantics on `data`, …)
78
75
  see the README's [Core commands](../README.md#core-commands-agent--human) section
79
76
  — this doc intentionally stays minimal.
80
77
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@trawlme/cli",
3
- "version": "2.4.0",
3
+ "version": "3.1.0",
4
4
  "description": "Trawl CLI — manage scraps from the terminal",
5
5
  "type": "module",
6
6
  "bin": {