@hunt-labs/bytekit-cli 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -10,6 +10,100 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
10
10
 
11
11
  ## [Unreleased]
12
12
 
13
+ ## [0.9.0] - 2026-08-20
14
+
15
+ ### Removed
16
+
17
+ - **BREAKING: `--cookies` and `--headers` are gone from `scrape create` and `screenshots create`**
18
+ (#3242). The API no longer accepts caller-supplied cookies or request headers on any capture
19
+ endpoint, so passing either flag is now an unknown-option error before any request is issued,
20
+ and a body carrying those keys is rejected by the server with `400 validation_error`. There is
21
+ no deprecation window, feature flag, or compatibility header.
22
+
23
+ ```bash
24
+ # before
25
+ bytekit scrape create --url https://example.com --cookies '[{"name":"session","value":"abc"}]'
26
+ # after — the flag does not exist
27
+ bytekit scrape create --url https://example.com
28
+ ```
29
+
30
+ `--webhook-headers` on `monitors` is a different flag and is unaffected.
31
+
32
+ ## [0.8.1] - 2026-08-13
33
+
34
+ ### Fixed
35
+
36
+ - **`bytekit search` explains a malformed `200` instead of leaking a `TypeError`.** A server
37
+ answering `200 {"ok":true}` — a proxy, a stub, a future envelope change — drove the human-mode
38
+ renderer into `Error: response.results is not iterable`, which named an internal expression
39
+ rather than the field the body was missing. It now reports, on one line and with no stack trace,
40
+ `search: the response carries no "results" array — the server returned an unexpected 200 body. Re-run with --json to inspect it.`
41
+ and exits `1`. A `results` value that is present but not an array is caught by the same gate: a
42
+ **string** was previously iterated CHARACTER BY CHARACTER, printing one garbled row per character
43
+ and exiting `0`. Unchanged: an empty `results: []` is still a successful zero-hit search that
44
+ prints nothing and exits `0`; valid responses render byte-identically; and `--json` never reaches
45
+ the renderer, so it keeps dumping the raw body at exit `0` — which is how you inspect an
46
+ unexpected body.
47
+
48
+ ### Added
49
+
50
+ - **`screenshots … -o <file>` now says so when the filename contradicts the captured format.**
51
+ `--format` is omitted far more often than not and the server default is **jpeg**, so
52
+ `bytekit screenshots create --url … -o shot.png` wrote JPEG bytes into a `.png`-named file
53
+ silently, at exit `0`; `screenshots get` has no `--format` flag at all, so it could not even be
54
+ told what it was downloading. Both now print one advisory line to **stderr** —
55
+ `Note: the screenshot was captured as jpeg, but shot.png names .png. The file holds jpeg bytes — pass --format png to capture it as png.`
56
+ The captured format is read from the artifact URL's own path, which is the only place the CLI can
57
+ observe it. **Advisory, not enforcement:** the bytes written and the exit code are unchanged, and
58
+ the note goes to stderr, so `-o -` pipelines and stdout redirects stay byte-identical. It stays
59
+ silent unless both the destination and the artifact name a recognized image format and the two
60
+ differ — `.jpg` and `.jpeg` are the same format, and an extension-less destination, a non-image
61
+ extension, `-o -`, or an artifact URL with no usable extension all make no claim to contradict. A
62
+ not-ready poll is untouched: it still prints its bare id and poll note at exit `2`.
63
+
64
+ ### Changed
65
+
66
+ - **The README now documents the whole flag surface, and says which reference wins.** Every flag
67
+ worked and every flag appeared in `--help`; the README simply framed subsets as exhaustive, so
68
+ `bulk create`'s `--webhook-secret` / `--defaults` / `--metadata`, `scrape bulk create`'s
69
+ `--custom`, `sitemap create`'s `--webhook-url` / `--webhook-secret` / `--cache-ttl` /
70
+ `--compact` / `--process` / `--metadata`, `monitors create` and `monitors update`'s `--cron`
71
+ (required with `--interval-type cron`) and `--metadata` plus the rest of `update`'s surface,
72
+ `fetch create`'s ten markdown flags and `--custom`, and `screenshots create`'s `--headers` /
73
+ `--cookies` / `--metadata` were all undocumented. All of them are documented now, and a note at
74
+ the top of the README states that `--help` is the authoritative flag reference. **Required flags
75
+ are marked as such** — `--file` and `--webhook-url` on all three bulk creators, `--url` /
76
+ `--interval-type` / `--webhook-url` on `monitors create`, and `--url` on `scrape create`,
77
+ `screenshots create`, `sitemap create` and both `fetch` verbs — so no reader mistakes one for
78
+ optional. Note in particular that a **bulk job has no poll-only mode**: `--webhook-url` is
79
+ mandatory there, unlike on `sitemap create` where omitting it really does mean poll-only. A test
80
+ asserts the parity in the direction that matters — every flag the parser accepts must appear in
81
+ the README — so this cannot drift again silently. No command, flag or behavior changed.
82
+
83
+ - **Alignment-release note, carried forward from this version's original cut.** `0.8.0` published
84
+ to npm at `2026-08-11T21:18:01.304Z`. Doc comments in `src/client.ts` and
85
+ `src/package-version.ts` were reworded **after** the commit that introduced the `0.8.0` version
86
+ string — they had re-spelled the `X-ByteKit-Client` header name in prose, which issue #2906's
87
+ pre-merge gate 1 counts. No command, flag, exit code, or runtime behavior differs from `0.8.0`
88
+ on account of _those_ comments.
89
+
90
+ `0.8.1` was cut for that reason because `check-release-drift` (#2680) is version-gated and
91
+ anchors on the commit that introduced a version, not on the commit whose tree was actually
92
+ published. Once `0.8.0` reached npm, any shipped-file change after that anchor reads as
93
+ post-release drift. Bumping moves the anchor past those comments. `0.8.1` had not yet reached
94
+ npm when the entries above landed, so they ride this same pending release rather than stranding
95
+ it as a never-published version.
96
+
97
+ - **Corrected a false legacy-name claim carried since the rename (#2950).** The `0.8.0`-era note
98
+ below said the prior `rapidcrawl-cli` name "remains installable at its last version but is
99
+ deprecated." Verified 2026-08-13: `rapidcrawl-cli` returns npm `404` ("is not in this
100
+ registry"), and `npm deprecate rapidcrawl-cli@'*'` fails with the same `404` — there is no
101
+ published version left for a deprecation notice to attach to, so the claim never held. No
102
+ registry stub or deprecation was possible to publish (npm has nothing to act on); root
103
+ `CLAUDE.md` carried the same false claim and is corrected alongside this entry. Users of the old
104
+ name should install `@hunt-labs/bytekit-cli` directly. No command, flag, or runtime behavior
105
+ changes.
106
+
13
107
  ## [0.8.0] - 2026-08-11
14
108
 
15
109
  ### Added
@@ -446,14 +540,16 @@ Additive, lower-risk release — no breaking changes to existing commands.
446
540
  ### Changed
447
541
 
448
542
  - Package renamed to `@hunt-labs/bytekit-cli` (the `@bytekit` npm scope was
449
- unobtainable); the `bin` name stays `bytekit`. The prior `rapidcrawl-cli` name
450
- remains installable at its last version but is deprecated.
543
+ unobtainable); the `bin` name stays `bytekit`. ~~The prior `rapidcrawl-cli` name
544
+ remains installable at its last version but is deprecated.~~ **Corrected in
545
+ 0.8.1 (see below): that claim does not hold — `rapidcrawl-cli` 404s on npm and
546
+ there is no published version left to deprecate.**
451
547
  - Consumer call sites propagate the SDK's narrowed, spec-derived request types (#2426).
452
548
  - Publish-ready packaging metadata: dropped `main`, unpinned the SDK dependency to
453
549
  `workspace:^`, added `engines`, `keywords`, `homepage`, `bugs`, `README.md`,
454
550
  `LICENSE`, and an MIT `license` field.
455
551
 
456
- ## [0.1.0] - 2026
552
+ ## [0.1.0] - 2026-07-13
457
553
 
458
554
  ### Added
459
555
 
package/README.md CHANGED
@@ -4,6 +4,12 @@ Command-line interface for the [ByteKit](https://bytekit.com) API — scrape,
4
4
  fetch, screenshot, search, crawl sitemaps, and manage page monitors from your
5
5
  shell. Wraps [`@hunt-labs/bytekit-sdk`](https://www.npmjs.com/package/@hunt-labs/bytekit-sdk).
6
6
 
7
+ > **`--help` is the authoritative flag reference.** Every command and subcommand
8
+ > accepts it — `bytekit sitemap create --help`, `bytekit monitors update --help` —
9
+ > and the list it prints is generated from the code that parses your argv, so it
10
+ > is never out of date. This README documents the same surface in prose for
11
+ > reading; where the two ever disagree, `--help` is right.
12
+
7
13
  ## Install
8
14
 
9
15
  ```bash
@@ -86,9 +92,13 @@ bytekit scrape get sc_123abc
86
92
  bytekit scrape bulk create --file urls.txt --webhook-url https://your.app/hooks/bytekit
87
93
  ```
88
94
 
89
- `scrape create` also accepts the full `ScrapeRequest` surface, each flag omitted
90
- from the request when absent (a boolean flag never sends `false`, so a server
91
- default is never materialized):
95
+ `scrape bulk create` takes the shared bulk-job flags plus `--custom` — see
96
+ [`bulk`](#bulk--bulk-screenshot-jobs).
97
+
98
+ `scrape create` **requires** `--url <url>`. It also accepts the full
99
+ `ScrapeRequest` surface below, each of those flags omitted from the request when
100
+ absent (a boolean flag never sends `false`, so a server default is never
101
+ materialized):
92
102
 
93
103
  | Flag | Effect |
94
104
  | ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -161,7 +171,7 @@ exits `0`.
161
171
 
162
172
  ```bash
163
173
  bytekit fetch create --url https://example.com
164
- bytekit fetch get --url https://example.com --format markdown
174
+ bytekit fetch get --url https://example.com
165
175
  bytekit fetch get --url https://example.com/logo.png -o logo.png
166
176
  bytekit fetch bulk create --file urls.txt --webhook-url https://your.app/hooks/bytekit
167
177
  ```
@@ -183,9 +193,14 @@ Note the acronym casing — `X-Fetch-ID` and `X-Fetch-URL`, not `X-Fetch-Id` /
183
193
  `X-Fetch-Url`. (Versions up to 0.6.0 emitted the Title-Cased spellings; see the
184
194
  CHANGELOG.)
185
195
 
186
- Both verbs accept `--country <cc>`, `--timeout-ms <n>`, and `--cache-ttl <ttl>`
187
- (`0`, `Nh`, or `Nd`), each omitted when absent. `--markdown-mode`
188
- (`article`, `raw`, `llm`) is `fetch create` only.
196
+ Both verbs **require** `--url <url>`. They accept `--country <cc>`, `--timeout-ms <n>`, `--cache-ttl <ttl>`
197
+ (`0`, `Nh`, or `Nd`), and `-o, --output <file>` — each of those omitted from the
198
+ request when absent.
199
+
200
+ `fetch create` and `fetch get` send only the raw-fetch options listed above.
201
+
202
+ `fetch bulk create` takes the shared bulk-job flags — see
203
+ [`bulk`](#bulk--bulk-screenshot-jobs).
189
204
 
190
205
  ### `search` — ranked web search
191
206
 
@@ -206,7 +221,8 @@ bytekit screenshots create --url https://example.com --device mobile --format pn
206
221
  bytekit screenshots get ss_123abc -o shot.png
207
222
  ```
208
223
 
209
- `screenshots create` accepts `--device` (`desktop`, `mobile`), `--viewport WxH`
224
+ `screenshots create` **requires** `--url <url>`. It also accepts `--device`
225
+ (`desktop`, `mobile`), `--viewport WxH`
210
226
  (e.g. `1280x720`), `--format` (`jpeg`, `png`), `--quality <n>`, `--wait-until`
211
227
  (`load`, `domcontentloaded`, `networkidle`), `--clip x,y,w,h`, and `--async`
212
228
  (queue and return a job envelope). Each flag is omitted from the request when
@@ -214,7 +230,10 @@ absent.
214
230
 
215
231
  It also accepts `--wait-for-selector <sel>`, `--delay-ms <n>`,
216
232
  `--device-scale-factor` (`1` or `2`), `--dark-mode`, `--include-html`,
217
- `--country <cc>`, `--language <lang>`, and three **tri-state** blocker pairs:
233
+ `--country <cc>`, `--language <lang>`, three JSON flags — `--headers <json>`
234
+ (extra request headers as an object, e.g. `'{"X-A":"b"}'`), `--cookies <json>`
235
+ (an array of `{name,value,domain}`), and `--metadata <json>` (echoed back on the
236
+ screenshot envelope) — and three **tri-state** blocker pairs:
218
237
  `--block-ads` / `--no-block-ads`, `--block-cookie-banners` /
219
238
  `--no-block-cookie-banners`, and `--scroll` / `--no-scroll`. Each of these three
220
239
  defaults to `true` server-side, so passing neither half omits the key entirely
@@ -227,6 +246,23 @@ stdout). Because `-o` writes BINARY on `screenshots`, it cannot be combined with
227
246
  `--async` + `-o` prints the queued job envelope and skips the download (the
228
247
  image is not ready yet).
229
248
 
249
+ **`-o` never converts — it names the file, `--format` names the bytes.** The
250
+ server default format is **jpeg**, so `-o shot.png` on a capture that did not
251
+ pass `--format png` writes JPEG bytes into a `.png`-named file. When the
252
+ destination's extension and the captured format disagree, the CLI prints one
253
+ advisory line to stderr and downloads anyway:
254
+
255
+ ```text
256
+ Note: the screenshot was captured as jpeg, but shot.png names .png.
257
+ The file holds jpeg bytes — pass --format png to capture it as png.
258
+ ```
259
+
260
+ The bytes written and the exit code are unchanged — it is a note, not a refusal,
261
+ and it goes to stderr so `-o -` pipelines and stdout redirects stay
262
+ byte-identical. It stays silent when the two agree (`.jpg` and `.jpeg` are the
263
+ same format), and when the destination makes no format claim at all (`-o -`, no
264
+ extension, or a non-image extension).
265
+
230
266
  ### `bulk` — bulk screenshot jobs
231
267
 
232
268
  ```bash
@@ -237,8 +273,36 @@ bytekit bulk cancel "$bulk_id"
237
273
  bytekit bulk screenshots list "$bulk_id"
238
274
  ```
239
275
 
240
- `urls.txt` is newline-delimited: one plain URL (or `{"url":"https://..."}`) per line.
241
- `bulk cancel <id>` stops an in-flight job.
276
+ `urls.txt` is newline-delimited: one plain URL (or `{"url":"https://..."}`) per
277
+ line. A line whose only key is `url` is sent as a bare URL string; any line with
278
+ additional keys switches that entry to the per-item shape. `bulk cancel <id>`
279
+ stops an in-flight job; `bulk get <id>` and `bulk screenshots list <id>` take no
280
+ flags.
281
+
282
+ The three bulk-job creators — `bulk create`, `scrape bulk create` and
283
+ `fetch bulk create` — share one flag set:
284
+
285
+ | Flag | Effect |
286
+ | --------------------------- | ------------------------------------------------------------------------------------------------------ |
287
+ | `--file <path>` | **Required.** The newline-delimited input file described above. |
288
+ | `--webhook-url <url>` | **Required.** HTTPS webhook notified when the job completes. |
289
+ | `--webhook-secret <secret>` | HMAC secret used to sign that completion webhook. Omitted → unsigned. |
290
+ | `--defaults <json>` | Defaults applied to every entry, as a JSON object. Omitted → none. |
291
+ | `--metadata <json>` | Arbitrary metadata echoed on the job envelope, as a JSON object, e.g. `'{"run":"7"}'`. Omitted → none. |
292
+
293
+ A bulk job has **no poll-only mode**. `--file` and `--webhook-url` are both
294
+ required on all three creators — omitting either is a usage error raised before
295
+ any request is sent (`error: required option '--webhook-url <url>' not
296
+ specified`), and the API declares `webhook_url` required as well. The three
297
+ optional flags are omitted from the request when absent.
298
+
299
+ `--defaults` is validated differently per endpoint: `bulk` rejects unknown keys
300
+ with a `422`, while `scrape bulk` and `fetch bulk` ignore them — and `fetch bulk`
301
+ additionally rejects the browser-only keys (`cookies`, `wait_until`,
302
+ `wait_for_selector`, `delay_ms`) with a `400`.
303
+
304
+ `scrape bulk create` adds one flag the other two do not have: `--custom <json>`,
305
+ a user payload echoed on every scrape entry (a per-entry value in `--file` wins).
242
306
 
243
307
  ### `monitors` — watch a page for changes
244
308
 
@@ -260,14 +324,34 @@ each omitted when absent.
260
324
  page through a monitor's capture history — pass the response's `next_cursor`
261
325
  back as `--cursor` to fetch the next page. Both flags are omitted when absent.
262
326
 
263
- `monitors update` gains `--status` (`active` | `paused` — pause and resume from
264
- the shell), `--change-threshold <pct>` (accepts a float), `--notify-on`
265
- (`change`, `every`), `--webhook-secret <secret>`, and three JSON-object flags:
266
- `--webhook-headers`, `--options`, and `--scrape-options` (e.g.
267
- `--options '{"device":"mobile"}'`). `monitors create` accepts the same set plus
268
- `--type` (`screenshot` | `scrape`). `--type` is **create-only** — a monitor's
269
- type is immutable, so `monitors update` deliberately has no `--type`. Every flag
270
- is omitted from the request when absent.
327
+ `monitors create` and `monitors update` share almost all of their flags. On
328
+ `update` every one is optional and omitted from the request when absent, so
329
+ `monitors update` changes only what you name. Three are **required on `create`**
330
+ and marked below:
331
+
332
+ | Flag | Effect |
333
+ | --------------------------- | ------------------------------------------------------------------------------------------ |
334
+ | `--url <url>` | **Required on `create`.** URL to monitor. |
335
+ | `--interval-type <type>` | **Required on `create`.** Check interval: `hourly`, `daily`, `weekly`, or `cron`. |
336
+ | `--cron <expr>` | Cron expression — **required** when `--interval-type` is `cron`, e.g. `'0 * * * *'`. |
337
+ | `--webhook-url <url>` | **Required on `create`.** HTTPS webhook notified on change. |
338
+ | `--change-threshold <pct>` | Change percentage that triggers a notification (accepts a float). |
339
+ | `--notify-on <event>` | When to notify: `change`, `every`. |
340
+ | `--webhook-secret <secret>` | HMAC secret for webhook signature verification. |
341
+ | `--webhook-headers <json>` | Extra webhook headers as a JSON object, e.g. `'{"X-Sig":"abc"}'`. |
342
+ | `--options <json>` | Screenshot-monitor options as a JSON object, e.g. `--options '{"device":"mobile"}'`. |
343
+ | `--scrape-options <json>` | Scrape-monitor options as a JSON object (`--type scrape` only). |
344
+ | `--metadata <json>` | Arbitrary metadata echoed on the monitor envelope, as a JSON object, e.g. `'{"run":"7"}'`. |
345
+
346
+ Two flags are not shared. `--type` (`screenshot` | `scrape`) is **create-only** —
347
+ a monitor's type is immutable, so `monitors update` deliberately has none.
348
+ `--status` (`active` | `paused`, to pause and resume from the shell) is
349
+ **update-only**.
350
+
351
+ ```bash
352
+ bytekit monitors create --url https://example.com/pricing --interval-type cron \
353
+ --cron '0 * * * *' --webhook-url https://your.app/hooks/bytekit
354
+ ```
271
355
 
272
356
  ### `sitemap` — crawl a site's URL graph
273
357
 
@@ -276,8 +360,24 @@ bytekit sitemap create --url https://example.com --strategy deep --max-depth 3 -
276
360
  bytekit sitemap get sm_123abc
277
361
  ```
278
362
 
279
- `sitemap create` accepts `--strategy` (`quick`, `standard`, `deep`),
280
- `--max-depth <n>`, and `--max-urls <n>`, each omitted when absent.
363
+ `sitemap get <id>` takes no flags. `sitemap create` accepts:
364
+
365
+ | Flag | Effect |
366
+ | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
367
+ | `--url <url>` | **Required.** Root URL to crawl. |
368
+ | `--strategy <strategy>` | Crawl strategy: `quick`, `standard`, `deep`. Omitted → server default `standard`. |
369
+ | `--max-depth <n>` | Maximum crawl depth, 1–5. Omitted → server default `2`. |
370
+ | `--max-urls <n>` | Maximum URLs to collect. Omitted → server default `5000`. |
371
+ | `--webhook-url <url>` | HTTPS webhook notified when the crawl completes. Omitted → poll-only, no delivery. |
372
+ | `--webhook-secret <secret>` | HMAC secret used to sign that completion webhook. Omitted → unsigned. |
373
+ | `--cache-ttl <ttl>` | Cache freshness: `0`, `Nh` (hours), or `Nd` (days), e.g. `48h`, `30d`. Omitted → server default `30d`; max `90d`, enforced server-side. |
374
+ | `--compact` | Return a compact result payload. Omitted → server default off. |
375
+ | `--process <json>` | Post-crawl processing directive as a JSON object, e.g. `'{"action":"summarize"}'`. Omitted → none. |
376
+ | `--metadata <json>` | Arbitrary metadata echoed on the job envelope, as a JSON object, e.g. `'{"run":"7"}'`. Omitted → none. |
377
+
378
+ Only `--url` is required; every other flag is omitted from the request when
379
+ absent. Unlike a bulk job, a sitemap crawl really is poll-only without
380
+ `--webhook-url`.
281
381
 
282
382
  ### `account` — account details
283
383
 
@@ -1,19 +1,5 @@
1
1
  import { Command } from 'commander';
2
2
  import { type GlobalOptions } from '../client.js';
3
- /** The `/v1/fetch` spec union for `format` — mirrors `FetchOpts['format']` in the SDK. */
4
- type FetchFormat = 'markdown' | 'html';
5
- /**
6
- * Validate `--format` locally, before any HTTP request is issued.
7
- *
8
- * The SDK's `FetchOpts.format` is narrowed to the spec union `'markdown' | 'html'`
9
- * (#2475), so an off-spec value like `pdf` is a compile-time error at the call site.
10
- * Commander runs an option's argParser during parsing — ahead of the action callback —
11
- * so rejecting here means the client is never built and no request is issued, and the
12
- * caller gets an actionable message instead of a server 422 (same convention as
13
- * `parseLimit` in search.ts).
14
- */
15
- export declare function parseFormat(value: string): FetchFormat;
16
3
  export declare function makeFetchCommand(getGlobal: () => GlobalOptions & {
17
4
  json: boolean;
18
5
  }): Command;
19
- export {};
@@ -1,29 +1,8 @@
1
- import { Command, InvalidArgumentError } from 'commander';
1
+ import { Command } from 'commander';
2
2
  import { buildClient } from '../client.js';
3
3
  import { printResult, emitFetchResult, handleError } from '../output.js';
4
4
  import { parseBulkLines } from '../lib/url-lines.js';
5
- import { parseIntFlag, parseEnum, parseCacheTtl, parseJsonFlag } from '../lib/parse-flags.js';
6
- const FETCH_FORMATS = ['markdown', 'html'];
7
- /** The `FetchRequest.markdown_mode` spec enum (create/POST only — excluded from the GET query). */
8
- const MARKDOWN_MODES = ['article', 'raw', 'llm'];
9
- /** The `markdown_links` / `markdown_images` render-mode enum (shared with `scrape create`). */
10
- const MARKDOWN_RENDER_MODES = ['inline', 'references', 'none', 'text'];
11
- /**
12
- * Validate `--format` locally, before any HTTP request is issued.
13
- *
14
- * The SDK's `FetchOpts.format` is narrowed to the spec union `'markdown' | 'html'`
15
- * (#2475), so an off-spec value like `pdf` is a compile-time error at the call site.
16
- * Commander runs an option's argParser during parsing — ahead of the action callback —
17
- * so rejecting here means the client is never built and no request is issued, and the
18
- * caller gets an actionable message instead of a server 422 (same convention as
19
- * `parseLimit` in search.ts).
20
- */
21
- export function parseFormat(value) {
22
- if (!FETCH_FORMATS.includes(value)) {
23
- throw new InvalidArgumentError(`--format must be one of: ${FETCH_FORMATS.join(', ')}.`);
24
- }
25
- return value;
26
- }
5
+ import { parseIntFlag, parseCacheTtl, parseJsonFlag } from '../lib/parse-flags.js';
27
6
  export function makeFetchCommand(getGlobal) {
28
7
  const cmd = new Command('fetch').description('Fetch URL content operations');
29
8
  const bulkCmd = new Command('bulk').description('Bulk fetch operations');
@@ -82,23 +61,9 @@ export function makeFetchCommand(getGlobal) {
82
61
  .command('create')
83
62
  .description('Fetch URL content (POST)')
84
63
  .requiredOption('--url <url>', 'URL to fetch')
85
- .option('--format <format>', 'Response format (markdown or html)', parseFormat)
86
64
  .option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
87
65
  .option('--timeout-ms <n>', 'Per-request timeout in milliseconds', (v) => parseIntFlag('--timeout-ms', v))
88
66
  .option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
89
- // The markdown surface below is POST-only: FetchGetOpts excludes it, so adding any of it to
90
- // `get` would be a compile error — the create/get asymmetry is correct by construction.
91
- .option('--markdown-mode <mode>', `Markdown mode: ${MARKDOWN_MODES.join(', ')}`, (v) => parseEnum('--markdown-mode', MARKDOWN_MODES, v))
92
- .option('--markdown-query <q>', 'BM25 query string for relevance-ranked markdown filtering (omitted → disabled)')
93
- .option('--markdown-links <mode>', `Link rendering: ${MARKDOWN_RENDER_MODES.join(', ')} (omitted → server default)`, (v) => parseEnum('--markdown-links', MARKDOWN_RENDER_MODES, v))
94
- .option('--markdown-images <mode>', `Image retention: ${MARKDOWN_RENDER_MODES.join(', ')} (omitted → server default)`, (v) => parseEnum('--markdown-images', MARKDOWN_RENDER_MODES, v))
95
- .option('--with-links-summary', 'Append a Links footer to the markdown output (omitted → off)')
96
- .option('--with-images-summary', 'Append an Images footer to the markdown output (omitted → off)')
97
- .option('--markdown-compact', 'Collapse excessive whitespace in the markdown output (omitted → off)')
98
- .option('--markdown-filter-images', 'Filter low-signal images from the markdown output (omitted → off)')
99
- .option('--markdown-include-media', 'Return rich link/image objects and a tables array (omitted → off)')
100
- .option('--markdown-include-warnings', 'Include markdown-pipeline and tag-filter warnings (omitted → off)')
101
- .option('--markdown-include-stats', 'Include a top-level stats object (chars, tokens, blocks) (omitted → off)')
102
67
  .option('--custom <json>', `User payload echoed on the envelope as a JSON object, e.g. '{"job":"1"}' (omitted → none)`, (v) => parseJsonFlag('--custom', v, 'object'))
103
68
  .option('-o, --output <file>', 'Write the fetched body bytes to the given file path')
104
69
  .action(async (opts) => {
@@ -108,36 +73,12 @@ export function makeFetchCommand(getGlobal) {
108
73
  // exact same wire output they had as inline spreads; a boolean is written ONLY on
109
74
  // `=== true`, so it never serializes `false` (#2480 E1).
110
75
  const body = { url: opts.url };
111
- if (opts.format !== undefined)
112
- body['format'] = opts.format;
113
76
  if (opts.country !== undefined)
114
77
  body['country'] = opts.country;
115
78
  if (opts.timeoutMs !== undefined)
116
79
  body['timeout_ms'] = opts.timeoutMs;
117
80
  if (opts.cacheTtl !== undefined)
118
81
  body['cache_ttl'] = opts.cacheTtl;
119
- if (opts.markdownMode !== undefined)
120
- body['markdown_mode'] = opts.markdownMode;
121
- if (opts.markdownQuery !== undefined)
122
- body['markdown_query'] = opts.markdownQuery;
123
- if (opts.markdownLinks !== undefined)
124
- body['markdown_links'] = opts.markdownLinks;
125
- if (opts.markdownImages !== undefined)
126
- body['markdown_images'] = opts.markdownImages;
127
- if (opts.withLinksSummary === true)
128
- body['with_links_summary'] = true;
129
- if (opts.withImagesSummary === true)
130
- body['with_images_summary'] = true;
131
- if (opts.markdownCompact === true)
132
- body['markdown_compact'] = true;
133
- if (opts.markdownFilterImages === true)
134
- body['markdown_filter_images'] = true;
135
- if (opts.markdownIncludeMedia === true)
136
- body['markdown_include_media'] = true;
137
- if (opts.markdownIncludeWarnings === true)
138
- body['markdown_include_warnings'] = true;
139
- if (opts.markdownIncludeStats === true)
140
- body['markdown_include_stats'] = true;
141
82
  if (opts.custom !== undefined)
142
83
  body['custom'] = opts.custom;
143
84
  try {
@@ -154,7 +95,6 @@ export function makeFetchCommand(getGlobal) {
154
95
  .command('get')
155
96
  .description('Fetch URL content (GET via query params)')
156
97
  .requiredOption('--url <url>', 'URL to fetch')
157
- .option('--format <format>', 'Response format (markdown or html)', parseFormat)
158
98
  .option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
159
99
  .option('--timeout-ms <n>', 'Per-request timeout in milliseconds', (v) => parseIntFlag('--timeout-ms', v))
160
100
  .option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
@@ -166,7 +106,6 @@ export function makeFetchCommand(getGlobal) {
166
106
  const client = buildClient(g);
167
107
  const result = await client.fetch.get({
168
108
  url: opts.url,
169
- ...(opts.format !== undefined ? { format: opts.format } : {}),
170
109
  ...(opts.country !== undefined ? { country: opts.country } : {}),
171
110
  ...(opts.timeoutMs !== undefined ? { timeout_ms: opts.timeoutMs } : {}),
172
111
  ...(opts.cacheTtl !== undefined ? { cache_ttl: opts.cacheTtl } : {}),
@@ -48,7 +48,7 @@ export function makeScrapeCommand(getGlobal) {
48
48
  .requiredOption('--file <path>', 'Path to a newline-delimited file. Each line is a plain URL, or a JSON object carrying "url" plus per-item config. A line whose only key is "url" behaves exactly as before (sent as a bare URL string); any line with additional keys switches the payload to the per-item shape.')
49
49
  .requiredOption('--webhook-url <url>', 'HTTPS webhook URL notified when the job completes')
50
50
  .option('--webhook-secret <secret>', 'HMAC secret used to sign the completion webhook (omitted → unsigned)')
51
- .option('--defaults <json>', 'Defaults applied to every entry, as a JSON object (omitted → none). Unknown keys are rejected by /v1/bulk (422) and ignored by scrape bulk / fetch bulk; fetch bulk additionally rejects browser-only keys (cookies, wait_until, wait_for_selector, delay_ms) with a 400.', (v) => parseJsonFlag('--defaults', v, 'object'))
51
+ .option('--defaults <json>', 'Defaults applied to every entry, as a JSON object (omitted → none). Unknown keys are rejected by /v1/bulk (422) and ignored by scrape bulk / fetch bulk; fetch bulk additionally rejects browser-only keys (wait_until, wait_for_selector, delay_ms) with a 400. `cookies` and `headers` are no longer accepted by any capture endpoint and 400 (#3242).', (v) => parseJsonFlag('--defaults', v, 'object'))
52
52
  .option('--metadata <json>', `Arbitrary metadata echoed on the job envelope, as a JSON object, e.g. '{"run":"7"}' (omitted → none)`, (v) => parseJsonFlag('--metadata', v, 'object'))
53
53
  // /v1/scrape/bulk is the ONE bulk endpoint with a top-level `custom`
54
54
  // (validators/scrape-bulk.ts:88). Per-entry `custom` arrives through --file lines instead.
@@ -115,8 +115,6 @@ export function makeScrapeCommand(getGlobal) {
115
115
  .option('--markdown-mode <mode>', `Markdown extraction mode: ${MARKDOWN_MODES.join(', ')}`, (v) => parseEnum('--markdown-mode', MARKDOWN_MODES, v))
116
116
  .option('--markdown-query <q>', 'BM25 query string for relevance-ranked markdown filtering (omitted → disabled)')
117
117
  .option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
118
- .option('--cookies <json>', `Cookies as a JSON array, e.g. '[{"name":"a","value":"b"}]' (omitted → none)`, (v) => parseJsonFlag('--cookies', v, 'array'))
119
- .option('--headers <json>', `Extra request headers as a JSON object, e.g. '{"X-A":"b"}' (omitted → none)`, (v) => parseJsonFlag('--headers', v, 'object'))
120
118
  .option('--delay-ms <n>', 'Post-load delay in milliseconds before capture', (v) => parseIntFlag('--delay-ms', v))
121
119
  .option('--events <csv>', `Webhook events (comma-separated): ${SCRAPE_EVENTS.join(', ')} (omitted → server default)`, (v) => parseCsv(v).map((e) => parseEnum('--events', SCRAPE_EVENTS, e)))
122
120
  .option('--token-encoding <enc>', `Tokenizer for --token-budget: ${TOKEN_ENCODINGS.join(', ')} (omitted → server default)`, (v) => parseEnum('--token-encoding', TOKEN_ENCODINGS, v))
@@ -164,10 +162,6 @@ export function makeScrapeCommand(getGlobal) {
164
162
  body['markdown_query'] = opts.markdownQuery;
165
163
  if (opts.cacheTtl !== undefined)
166
164
  body['cache_ttl'] = opts.cacheTtl;
167
- if (opts.cookies !== undefined)
168
- body['cookies'] = opts.cookies;
169
- if (opts.headers !== undefined)
170
- body['headers'] = opts.headers;
171
165
  if (opts.delayMs !== undefined)
172
166
  body['delay_ms'] = opts.delayMs;
173
167
  if (opts.events !== undefined)
@@ -12,6 +12,32 @@ export declare function parseClip(value: string): {
12
12
  width: number;
13
13
  height: number;
14
14
  };
15
+ /**
16
+ * The advisory note for a `-o` destination whose extension contradicts the captured format, or
17
+ * undefined when there is nothing to say (#2956 item 3).
18
+ *
19
+ * THE DEFECT: `--format` is omitted far more often than not, and the server default is **jpeg** —
20
+ * so `screenshots create --url … -o shot.png` wrote JPEG bytes into a `.png`-named file, silently,
21
+ * at exit 0. The mislabelled file then propagated to whatever consumed it next. `screenshots get`
22
+ * has no `--format` flag at all, so it could not even be told what it was downloading.
23
+ *
24
+ * ADVISORY, NOT ENFORCEMENT — this returns a message; the caller writes it to STDERR and proceeds.
25
+ * The bytes on disk and the exit code are untouched, because the caller may well have meant it
26
+ * (`-o` is also how you rename an artifact), and because refusing the write would throw away a
27
+ * capture that was already taken and billed. It stays on the `-o` write path rather than moving
28
+ * into `downloadArtifact`: that helper is format-agnostic and has no business knowing about image
29
+ * extensions.
30
+ *
31
+ * Silent unless BOTH sides name a recognized image format AND they differ. Nothing to compare —
32
+ * `-o -` (stdout, no filename to mislabel), an extension-less or non-image destination, an
33
+ * artifact URL with no usable extension — means no note, never a guessed one.
34
+ *
35
+ * `-o -` needs no special case of its own: the literal `-` carries no extension, so
36
+ * {@link formatFromPath} already returns undefined for it. An explicit early return was written
37
+ * first and then removed — no mutation of it could turn a test red, which is the definition of a
38
+ * line that guards nothing. The stdout case stays PINNED by a test either way.
39
+ */
40
+ export declare function formatMismatchNote(imageUrl: string, dest: string): string | undefined;
15
41
  export declare function makeScreenshotsCommand(getGlobal: () => GlobalOptions & {
16
42
  json: boolean;
17
43
  }): Command;
@@ -99,6 +99,91 @@ function notReadyScreenshot(result) {
99
99
  return undefined;
100
100
  return { id: obj.id, status };
101
101
  }
102
+ /**
103
+ * Image extensions this CLI recognizes, mapped to their canonical format name.
104
+ *
105
+ * `jpg` and `jpeg` are the SAME format under two spellings — collapsing them here is what keeps
106
+ * the note off `-o shot.jpg`, which is the commonest spelling of a perfectly correct destination.
107
+ * The set is deliberately small: it exists to decide whether a name makes a FORMAT CLAIM at all,
108
+ * and an extension outside it (`.bin`, `.tmp`, none) makes none.
109
+ */
110
+ const IMAGE_EXTENSIONS = {
111
+ jpg: 'jpeg',
112
+ jpeg: 'jpeg',
113
+ png: 'png',
114
+ };
115
+ /**
116
+ * The canonical image format named by a path's extension, or undefined when it names none.
117
+ *
118
+ * Reads the extension off the last path SEGMENT so a directory containing a dot cannot supply one,
119
+ * and requires a non-empty stem so a dotfile (`.png`) is a name, not an extension.
120
+ */
121
+ function formatFromPath(path) {
122
+ const segment = path.slice(path.lastIndexOf('/') + 1);
123
+ const dot = segment.lastIndexOf('.');
124
+ if (dot <= 0)
125
+ return undefined;
126
+ return IMAGE_EXTENSIONS[segment.slice(dot + 1).toLowerCase()];
127
+ }
128
+ /**
129
+ * The format the artifact was actually captured in, read off its `image_url`, or undefined.
130
+ *
131
+ * `screenshotKey` (`packages/worker/src/storage/b2.ts:60`) builds the object key as
132
+ * `screenshots/<account>/<date>/<screenshot_id>.<format>`, so the URL's PATHNAME extension is the
133
+ * captured format — the only place the CLI can observe it. `ScreenshotResponse`
134
+ * (`docs/api/openapi.yaml`) carries no `format` field, and `screenshots get` has no `--format`
135
+ * flag, so neither the envelope nor the argv can answer this on the poll path.
136
+ *
137
+ * The pathname — never the raw string. A completed `image_url` is an AWS SigV4 PRESIGNED URL
138
+ * (`?X-Amz-Credential=…&X-Amz-Signature=…`, `gateway/src/lib/signed-url.ts`), and a
139
+ * `lastIndexOf('.')` over the whole string reads the signature instead of the key. An unparseable
140
+ * URL yields undefined rather than throwing: this is advisory, and it runs immediately before a
141
+ * download that must not be blocked by it.
142
+ */
143
+ function capturedFormat(imageUrl) {
144
+ try {
145
+ return formatFromPath(new URL(imageUrl).pathname);
146
+ }
147
+ catch {
148
+ return undefined;
149
+ }
150
+ }
151
+ /**
152
+ * The advisory note for a `-o` destination whose extension contradicts the captured format, or
153
+ * undefined when there is nothing to say (#2956 item 3).
154
+ *
155
+ * THE DEFECT: `--format` is omitted far more often than not, and the server default is **jpeg** —
156
+ * so `screenshots create --url … -o shot.png` wrote JPEG bytes into a `.png`-named file, silently,
157
+ * at exit 0. The mislabelled file then propagated to whatever consumed it next. `screenshots get`
158
+ * has no `--format` flag at all, so it could not even be told what it was downloading.
159
+ *
160
+ * ADVISORY, NOT ENFORCEMENT — this returns a message; the caller writes it to STDERR and proceeds.
161
+ * The bytes on disk and the exit code are untouched, because the caller may well have meant it
162
+ * (`-o` is also how you rename an artifact), and because refusing the write would throw away a
163
+ * capture that was already taken and billed. It stays on the `-o` write path rather than moving
164
+ * into `downloadArtifact`: that helper is format-agnostic and has no business knowing about image
165
+ * extensions.
166
+ *
167
+ * Silent unless BOTH sides name a recognized image format AND they differ. Nothing to compare —
168
+ * `-o -` (stdout, no filename to mislabel), an extension-less or non-image destination, an
169
+ * artifact URL with no usable extension — means no note, never a guessed one.
170
+ *
171
+ * `-o -` needs no special case of its own: the literal `-` carries no extension, so
172
+ * {@link formatFromPath} already returns undefined for it. An explicit early return was written
173
+ * first and then removed — no mutation of it could turn a test red, which is the definition of a
174
+ * line that guards nothing. The stdout case stays PINNED by a test either way.
175
+ */
176
+ export function formatMismatchNote(imageUrl, dest) {
177
+ const requested = formatFromPath(dest);
178
+ if (requested === undefined)
179
+ return undefined;
180
+ const actual = capturedFormat(imageUrl);
181
+ if (actual === undefined || actual === requested)
182
+ return undefined;
183
+ const ext = dest.slice(dest.lastIndexOf('.'));
184
+ return (`Note: the screenshot was captured as ${actual}, but ${dest} names ${ext}. ` +
185
+ `The file holds ${actual} bytes — pass --format ${requested} to capture it as ${requested}.`);
186
+ }
102
187
  /**
103
188
  * The failure carried by the FLAT `error_code`/`error_message` fields, or undefined (#2792).
104
189
  *
@@ -220,7 +305,15 @@ async function emitScreenshot(result, opts) {
220
305
  process.exitCode = QUEUED_EXIT_CODE;
221
306
  return;
222
307
  }
223
- await downloadArtifact(extractImageUrl(result), opts.output);
308
+ const imageUrl = extractImageUrl(result);
309
+ // Advisory only — see {@link formatMismatchNote}. Emitted BEFORE the download so the note is
310
+ // never lost to a failing fetch, and to STDERR so `-o -` pipelines and stdout redirects stay
311
+ // byte-identical. It is deliberately unreachable from the not-ready branch above (no artifact
312
+ // exists, so there is no format to compare) and from `extractImageUrl`'s throw paths.
313
+ const note = formatMismatchNote(imageUrl, opts.output);
314
+ if (note !== undefined)
315
+ console.error(note);
316
+ await downloadArtifact(imageUrl, opts.output);
224
317
  return;
225
318
  }
226
319
  printResult(result, opts.json);
@@ -253,8 +346,6 @@ export function makeScreenshotsCommand(getGlobal) {
253
346
  .option('--include-html', 'Include the rendered HTML alongside the screenshot')
254
347
  .option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
255
348
  .option('--language <lang>', 'Accept-Language override (e.g. en-US)')
256
- .option('--headers <json>', `Extra request headers as a JSON object, e.g. '{"X-A":"b"}' (omitted → none)`, (v) => parseJsonFlag('--headers', v, 'object'))
257
- .option('--cookies <json>', `Cookies as a JSON array of {name,value,domain}, e.g. '[{"name":"a","value":"b","domain":"example.com"}]' (omitted → none)`, (v) => parseJsonFlag('--cookies', v, 'array'))
258
349
  .option('--metadata <json>', `Arbitrary metadata echoed on the screenshot envelope, as a JSON object, e.g. '{"run":"7"}' (omitted → none)`, (v) => parseJsonFlag('--metadata', v, 'object'))
259
350
  .option('--async', 'Queue the screenshot and return a job envelope immediately (no wait)')
260
351
  .option('-o, --output <file>', 'Download the captured image to this file path (use - for stdout). Not valid with --json.')
@@ -301,12 +392,8 @@ export function makeScreenshotsCommand(getGlobal) {
301
392
  body['country'] = opts.country;
302
393
  if (opts.language !== undefined)
303
394
  body['language'] = opts.language;
304
- // The server defaults headers/cookies/metadata to {} / [] / {}; writing an empty
305
- // container on a bare create would materialize those defaults (#2480 E1).
306
- if (opts.headers !== undefined)
307
- body['headers'] = opts.headers;
308
- if (opts.cookies !== undefined)
309
- body['cookies'] = opts.cookies;
395
+ // The server defaults metadata to {}; writing an empty container on a bare
396
+ // create would materialize that default (#2480 E1).
310
397
  if (opts.metadata !== undefined)
311
398
  body['metadata'] = opts.metadata;
312
399
  try {
@@ -6,8 +6,33 @@ import { printResult, handleError } from '../output.js';
6
6
  *
7
7
  * The CLI never surfaces provider-specific fields — only the ranked result
8
8
  * columns the API returns.
9
+ *
10
+ * GUARD (#2956 item 1): `response.results` is typed as required, but the type describes what the
11
+ * API PROMISES, not what a given 200 actually carried — a proxy, a stub server, or a future
12
+ * envelope change can all answer `200 {"ok":true}`. Two distinct defects came out of trusting the
13
+ * type here:
14
+ *
15
+ * - a MISSING (or null) `results` drove the `for…of` below into
16
+ * `TypeError: response.results is not iterable`, which reached stderr as the CLI's whole
17
+ * diagnostic. It named an internal expression instead of the field, and left the caller with no
18
+ * next move;
19
+ * - a `results` that is a STRING is iterable, so it did not throw at all — it printed one garbled
20
+ * row per CHARACTER and exited 0, which is the quieter of the two failures.
21
+ *
22
+ * `Array.isArray` is therefore the gate (not a truthiness or length test), and it is checked
23
+ * BEFORE any row is written so a malformed body can never produce a half-rendered table. An EMPTY
24
+ * array stays a successful zero-hit search: it prints nothing and exits 0.
25
+ *
26
+ * The thrown error is a plain `Error` raised inside the action's `try`, so it reaches
27
+ * `handleError` — one `Error: `-prefixed stderr line, exit 1, no stack trace — exactly like every
28
+ * other command's unexpected-body diagnostic. It points at `--json`, which bypasses this renderer
29
+ * entirely and dumps the raw body, because inspecting that body is the caller's actual next step.
9
30
  */
10
31
  function renderTable(response) {
32
+ if (!Array.isArray(response.results)) {
33
+ throw new Error('search: the response carries no "results" array — the server returned an unexpected 200 body. ' +
34
+ 'Re-run with --json to inspect it.');
35
+ }
11
36
  for (const result of response.results) {
12
37
  console.log(`${result.position}. ${result.title}`);
13
38
  console.log(` ${result.url}`);
@@ -31,7 +31,7 @@ export declare function parseNumberFlag(flag: string, value: string): number;
31
31
  export declare function parseCacheTtl(flag: string, value: string): string;
32
32
  /**
33
33
  * Parse a JSON-object- or JSON-array-valued flag (#2545). The ONLY `JSON.parse` call site for CLI
34
- * flag values — every JSON-shaped flag (`--cookies`, `--headers`, `--custom`,
34
+ * flag values — every JSON-shaped flag (`--custom`, `--metadata`, `--defaults`,
35
35
  * `--webhook-headers`, `--options`, `--scrape-options`) routes through here, so command modules
36
36
  * carry no raw `JSON.parse` (grep-gated). Rejects malformed JSON AND a value whose top-level shape
37
37
  * does not match `expect`, with a copy-pasteable example in the message (E5).
@@ -57,7 +57,7 @@ export function parseCacheTtl(flag, value) {
57
57
  }
58
58
  /**
59
59
  * Parse a JSON-object- or JSON-array-valued flag (#2545). The ONLY `JSON.parse` call site for CLI
60
- * flag values — every JSON-shaped flag (`--cookies`, `--headers`, `--custom`,
60
+ * flag values — every JSON-shaped flag (`--custom`, `--metadata`, `--defaults`,
61
61
  * `--webhook-headers`, `--options`, `--scrape-options`) routes through here, so command modules
62
62
  * carry no raw `JSON.parse` (grep-gated). Rejects malformed JSON AND a value whose top-level shape
63
63
  * does not match `expect`, with a copy-pasteable example in the message (E5).
@@ -30,12 +30,12 @@ export type BulkLine = string | Record<string, unknown>;
30
30
  *
31
31
  * The rejection message for a malformed line (#2596 cr-1 / E12) names the 1-based PHYSICAL line
32
32
  * NUMBER (i.e. the line's own position in the file, counting blank lines) and the line's own KEY
33
- * NAMES only — never the raw line or any value. A `--file` line is where per-item `cookies`,
34
- * `headers`, and auth-bearing `metadata` now live (#2596), so echoing the raw line back would
35
- * leak session tokens and similar secrets to stderr, CI logs, and shell transcripts. Because the
36
- * raw line is deliberately never echoed, the line number is the ONLY locator left in the message —
37
- * it must be computed BEFORE blank lines are filtered out, or it silently mis-locates the bad
38
- * entry whenever the file contains blank lines above it (#2596 cr-2).
33
+ * NAMES only — never the raw line or any value. A `--file` line is where auth-bearing per-item
34
+ * `metadata` lives (#2596; per-item `cookies`/`headers` were removed in #3242), so echoing the
35
+ * raw line back would leak session tokens and similar secrets to stderr, CI logs, and shell
36
+ * transcripts. Because the raw line is deliberately never echoed, the line number is the ONLY
37
+ * locator left in the message — it must be computed BEFORE blank lines are filtered out, or it
38
+ * silently mis-locates the bad entry whenever the file contains blank lines above it (#2596 cr-2).
39
39
  */
40
40
  export declare function parseBulkLines(file: string): BulkLine[];
41
41
  /**
@@ -24,12 +24,12 @@ import { InvalidArgumentError } from 'commander';
24
24
  *
25
25
  * The rejection message for a malformed line (#2596 cr-1 / E12) names the 1-based PHYSICAL line
26
26
  * NUMBER (i.e. the line's own position in the file, counting blank lines) and the line's own KEY
27
- * NAMES only — never the raw line or any value. A `--file` line is where per-item `cookies`,
28
- * `headers`, and auth-bearing `metadata` now live (#2596), so echoing the raw line back would
29
- * leak session tokens and similar secrets to stderr, CI logs, and shell transcripts. Because the
30
- * raw line is deliberately never echoed, the line number is the ONLY locator left in the message —
31
- * it must be computed BEFORE blank lines are filtered out, or it silently mis-locates the bad
32
- * entry whenever the file contains blank lines above it (#2596 cr-2).
27
+ * NAMES only — never the raw line or any value. A `--file` line is where auth-bearing per-item
28
+ * `metadata` lives (#2596; per-item `cookies`/`headers` were removed in #3242), so echoing the
29
+ * raw line back would leak session tokens and similar secrets to stderr, CI logs, and shell
30
+ * transcripts. Because the raw line is deliberately never echoed, the line number is the ONLY
31
+ * locator left in the message — it must be computed BEFORE blank lines are filtered out, or it
32
+ * silently mis-locates the bad entry whenever the file contains blank lines above it (#2596 cr-2).
33
33
  */
34
34
  export function parseBulkLines(file) {
35
35
  return readFileSync(file, 'utf8')
@@ -0,0 +1,23 @@
1
+ /**
2
+ * Build a scratch COPY of packages/cli (issue #3270), mirroring
3
+ * `packages/sdk-typescript/src/test-helpers/build-scratch.ts` (issue #2423 cr-2).
4
+ *
5
+ * `entrypoint.test.ts` and `queued-raw-flush.test.ts` both need a built `dist/index.js` and both
6
+ * used to rebuild the shared, workspace `packages/cli/dist` in place whenever it was missing —
7
+ * racing each other's build under vitest's default parallel file execution on a checkout with no
8
+ * prior `pnpm build` (one file's `existsSync` check passes while the other's build is still
9
+ * writing `dist/`).
10
+ *
11
+ * So instead: copy the tracked, on-disk source (never `dist/`, `tsconfig.tsbuildinfo`, or
12
+ * `node_modules` — those are the mutable/heavy bits) into a scratch tree that mirrors the real
13
+ * repo's relative layout (`<scratch>/tsconfig.base.json`, `<scratch>/packages/cli/**`), symlink in
14
+ * the real `node_modules` (read-only from the child process's point of view — nothing here writes
15
+ * back into it), and let the caller run the package's own `npm run build` / `npm pack` scripts
16
+ * against that copy. The real `packages/cli/dist` is never read and never written by either suite.
17
+ *
18
+ * @param pkgRoot packages/cli (the real, workspace-shared package root)
19
+ * @param repoRoot the monorepo root (for tsconfig.base.json, which the scratch tsconfig extends)
20
+ * @param scratchRoot a fresh tmp dir (e.g. from mkdtempSync) to build the scratch tree under
21
+ * @returns the scratch package dir (`<scratchRoot>/packages/cli`)
22
+ */
23
+ export declare function createBuildScratch(pkgRoot: string, repoRoot: string, scratchRoot: string): string;
@@ -0,0 +1,41 @@
1
+ // Test-only fixture, compiled like any other `src` file (not excluded from the tsc build) — same
2
+ // tradeoff sdk-typescript's own copy of this helper documents: a small compiled fixture lands in
3
+ // `dist/test-helpers/`, but excluding it breaks ESLint's typed-lint project-service resolution.
4
+ import { mkdirSync, cpSync, copyFileSync, symlinkSync } from 'node:fs';
5
+ import { basename, join } from 'node:path';
6
+ /**
7
+ * Build a scratch COPY of packages/cli (issue #3270), mirroring
8
+ * `packages/sdk-typescript/src/test-helpers/build-scratch.ts` (issue #2423 cr-2).
9
+ *
10
+ * `entrypoint.test.ts` and `queued-raw-flush.test.ts` both need a built `dist/index.js` and both
11
+ * used to rebuild the shared, workspace `packages/cli/dist` in place whenever it was missing —
12
+ * racing each other's build under vitest's default parallel file execution on a checkout with no
13
+ * prior `pnpm build` (one file's `existsSync` check passes while the other's build is still
14
+ * writing `dist/`).
15
+ *
16
+ * So instead: copy the tracked, on-disk source (never `dist/`, `tsconfig.tsbuildinfo`, or
17
+ * `node_modules` — those are the mutable/heavy bits) into a scratch tree that mirrors the real
18
+ * repo's relative layout (`<scratch>/tsconfig.base.json`, `<scratch>/packages/cli/**`), symlink in
19
+ * the real `node_modules` (read-only from the child process's point of view — nothing here writes
20
+ * back into it), and let the caller run the package's own `npm run build` / `npm pack` scripts
21
+ * against that copy. The real `packages/cli/dist` is never read and never written by either suite.
22
+ *
23
+ * @param pkgRoot packages/cli (the real, workspace-shared package root)
24
+ * @param repoRoot the monorepo root (for tsconfig.base.json, which the scratch tsconfig extends)
25
+ * @param scratchRoot a fresh tmp dir (e.g. from mkdtempSync) to build the scratch tree under
26
+ * @returns the scratch package dir (`<scratchRoot>/packages/cli`)
27
+ */
28
+ export function createBuildScratch(pkgRoot, repoRoot, scratchRoot) {
29
+ const pkgDest = join(scratchRoot, 'packages/cli');
30
+ mkdirSync(join(scratchRoot, 'packages'), { recursive: true });
31
+ copyFileSync(join(repoRoot, 'tsconfig.base.json'), join(scratchRoot, 'tsconfig.base.json'));
32
+ cpSync(pkgRoot, pkgDest, {
33
+ recursive: true,
34
+ filter: (src) => {
35
+ const name = basename(src);
36
+ return name !== 'node_modules' && name !== 'dist' && name !== 'tsconfig.tsbuildinfo';
37
+ },
38
+ });
39
+ symlinkSync(join(pkgRoot, 'node_modules'), join(pkgDest, 'node_modules'), 'dir');
40
+ return pkgDest;
41
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@hunt-labs/bytekit-cli",
3
- "version": "0.8.0",
3
+ "version": "0.9.0",
4
4
  "description": "ByteKit CLI — wraps @hunt-labs/bytekit-sdk as shell subcommands",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -41,7 +41,7 @@
41
41
  ],
42
42
  "dependencies": {
43
43
  "commander": "^12.0.0",
44
- "@hunt-labs/bytekit-sdk": "^0.7.0"
44
+ "@hunt-labs/bytekit-sdk": "^0.9.0"
45
45
  },
46
46
  "scripts": {
47
47
  "build": "rm -rf dist tsconfig.tsbuildinfo && tsc",