@hunt-labs/bytekit-cli 0.5.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,8 +8,186 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
8
8
  This file records only the delta between versions. For how to use the package
9
9
  today, see [README.md](./README.md), which remains the authoritative usage guide.
10
10
 
11
+ ## [Unreleased]
12
+
13
+ ## [0.7.0] - 2026-08-07
14
+
15
+ ### Breaking
16
+
17
+ - **A not-ready `screenshots get -o` / `screenshots create -o` now exits `2` and prints the bare
18
+ `ss_` id, instead of exiting `1` with `not ready yet: screenshot is pending`.** The two poll
19
+ commands disagreed: `scrape get --raw` on a still-queued scrape already wrote the bare `sc_` id
20
+ to stdout and exited `2` — a distinct, scriptable "accepted but not ready" signal — while the
21
+ screenshots side reported the same state as a failure and threw the handle away, so a polling
22
+ script could branch on "queued" for one command and not the other. The screenshots branch now
23
+ mirrors scrape exactly: bare `ss_` id on stdout, a poll note on stderr, exit `2`, and (as before)
24
+ no file written. A script that treated any non-zero exit as "this screenshot failed" now sees `2`
25
+ for a job that is merely still running — branch on the code (`2` = poll again, `1` = failed)
26
+ rather than on non-zero. Three states deliberately do **not** move: a terminally `failed`
27
+ screenshot stays exit `1` with its error surfaced (it will never complete, so a "poll again"
28
+ signal would loop a script forever), an envelope carrying no `ss_` id stays exit `1` (there is no
29
+ handle to hand back), and `--json` keeps the envelope at exit `0`.
30
+ - **`--json` on a `fetch` command now emits `X-Fetch-ID` and `X-Fetch-URL`, not `X-Fetch-Id` and
31
+ `X-Fetch-Url`.** The metadata keys were derived by a generic segment-wise Title-Caser, which
32
+ lowercases every non-initial character of a segment — so the two acronym-bearing headers came out
33
+ in a spelling the API reference never uses, and a `jq '."X-Fetch-ID"'` filter written from the
34
+ docs returned `null`. All 16 `X-Fetch-*` keys now match the spec names byte-for-byte. The
35
+ **values** are unchanged, the non-`X-Fetch-*` headers stay filtered out, and every other key
36
+ (`X-Fetch-Waf-Vendor`, `X-Fetch-Duration-Ms`, `X-Fetch-Fast-Path`, …) is spelled exactly as
37
+ before. A consumer reading `.["X-Fetch-Id"]` must switch to `.["X-Fetch-ID"]`.
38
+
39
+ ### Fixed
40
+
41
+ - **A `screenshots get -o` / `screenshots create -o` poll only says "poll again" for a `pending` or
42
+ `processing` screenshot.** The not-ready predicate decided from the presence of an `image_url`,
43
+ an `id` and a nested error alone — `status` was read only to phrase the message — so a
44
+ `completed` screenshot whose `image_url` never materialized, a status outside the closed enum,
45
+ and the flat `error_code`/`error_message` failed shape all received exit `2` and "no image
46
+ artifact is available yet. Poll it with: …" for a job that would never complete. Those states now
47
+ exit `1`: a terminal failure prints `screenshot failed: …`, and any other artifact-less state
48
+ prints `no image artifact available: screenshot is <status>`. Genuine `pending` / `processing`
49
+ polls keep the bare `ss_` id, the poll note and exit `2` exactly as before — **including** a
50
+ poll whose envelope carries an `error_code` from an attempt that failed and is being retried,
51
+ since the status, not the error, decides whether polling can still help. `--json` still prints
52
+ the envelope at exit `0`.
53
+ - **A terminally failed screenshot reported through the spec's flat `error_code`/`error_message`
54
+ fields now reaches stderr.** Only the nested `{error:{code,message}}` envelope was read, so the
55
+ shape `ScreenshotResponse` actually declares surfaced neither code nor message. It now prints
56
+ `screenshot failed: <message> (<code>)` — the formatting `scrape get --raw` already uses for a
57
+ failed scrape. The nested envelope's own message is unchanged.
58
+ - **`scrape get --raw` on a terminally failed scrape now reports why it failed.** It printed
59
+ `--raw: response contains no formats` — technically true of a `ScrapeErrorEnvelope`, and the one
60
+ diagnostic that cannot explain anything — while the envelope's own `error.code` and
61
+ `error.message` never reached the caller. Both now go to stderr (exit `1`, unchanged). The
62
+ failure is reported ahead of the format diagnostics, so an envelope carrying both a failure and
63
+ an empty `formats: {}` still says why it failed rather than pointing at a `--format` choice that
64
+ cannot help.
65
+ - **An empty-formats success now names the warning that explains it.** A completed scrape whose
66
+ persisted artifact could not be rehydrated comes back with `formats: {}` and an
67
+ `artifact_unavailable` warning; `--raw` rendered that as `--raw requires --format when the
68
+ response contains multiple formats`, which is false on both counts. It now names the warning's
69
+ code and message, or says plainly that the formats map is empty when no warning explains it.
70
+ - **Error paths no longer risk losing their own diagnostic.** `handleError` and the missing-API-key
71
+ path each wrote to stderr and then immediately hard-`process.exit(1)`. Node's stderr is
72
+ asynchronous when it points at a pipe, and `process.exit` discards whatever is still queued — so
73
+ the one line a script greps was the line most at risk of being thrown away, which is exactly the
74
+ defect this package's flush policy exists to prevent. Both now set `process.exitCode` and let the
75
+ process end naturally once stderr has drained. The exit codes and the rendered messages are
76
+ byte-identical.
77
+ - `--headers`, `--custom` and `--metadata` said `must be valid JSON (a object)` on a malformed
78
+ value; the article now agrees — `an object` / `an array`.
79
+ - The `[0.6.0]` link reference, missing from the bottom of this file, is restored.
80
+
81
+ ## [0.6.0] - 2026-07-29
82
+
83
+ ### Breaking
84
+
85
+ - **`screenshots create --async` now exits `2` in human mode instead of `0`.** The queued 202 is
86
+ not a completed capture — there is no artifact yet — and every other queued path in the CLI
87
+ already signalled that with exit code `2` (`scrape create --async`). A script that relied on
88
+ exit `0` to mean "the screenshot is ready" was reading a false success; under `set -e` such a
89
+ script now stops at the queue step, which is the intended correction. The envelope itself is
90
+ unchanged and still printed. Two carve-outs: **`--json` keeps exit `0`** (the envelope is a
91
+ complete machine-readable payload — identical to `scrape create --async --json`), and
92
+ `--async -o <file>` also exits `2` while still writing no file (it prints the queued envelope
93
+ and skips the download, as before). Poll with `bytekit screenshots get <id>`.
94
+ - **A `--file` per-item key outside a bulk endpoint's own strict schema now fails the whole
95
+ create with a `422`** instead of silently succeeding with that key ignored. In 0.5.x, `--file`
96
+ never read anything but `url` from a JSON line — every other key was dropped client-side
97
+ before the request was ever built. As of 0.6.0 those keys ride to the server (`items` on
98
+ `bulk create` / `scrape bulk create`; inside `urls` on `fetch bulk create`), and `/v1/bulk` /
99
+ `/v1/scrape/bulk` reject any key their `.strict()` per-item schema doesn't recognize. A file
100
+ that worked in 0.5.x purely because its extra keys were quietly discarded may now `422` — see
101
+ each command's `--file` help for the keys actually honored on that endpoint.
102
+ - **`fetch bulk create --file` per-item keys outside `url`/`format`/`country`/`cache_ttl` now
103
+ fail the whole create with a `422`** instead of silently succeeding with that key ignored.
104
+ `/v1/fetch/bulk`'s per-item override schema (inside `urls[]`) is now `.strict()` on the
105
+ gateway (issue #2648), matching the `/v1/bulk` / `/v1/scrape/bulk` posture above. The
106
+ `--file` help text for `fetch bulk create` is now uniform with `bulk create` /
107
+ `scrape bulk create` and no longer enumerates keys — this entry is where the honored set
108
+ (`url`, `format`, `country`, `cache_ttl`) is documented instead.
109
+
110
+ ### Added
111
+
112
+ - **Per-item bulk configuration on all three bulk creates** (`bulk create`, `scrape bulk create`,
113
+ `fetch bulk create`). A `--file` line may now be a JSON object carrying `url` plus per-item
114
+ fields instead of only a bare URL; such a file is sent as `items` on `/v1/bulk` and
115
+ `/v1/scrape/bulk`, and as a heterogeneous `urls` array on `/v1/fetch/bulk` (which has no
116
+ `items` property). A line whose only key is `url` still normalizes to a bare URL string, so a
117
+ file of url-only lines produces a byte-identical request to before. A JSON line with a missing
118
+ or non-string `url` is now rejected locally with a usage error (exit `1`) instead of being sent
119
+ to the server as a URL. (See Breaking, above, for the per-item strict-schema behavior change
120
+ this enables.)
121
+ - `--defaults`, `--webhook-secret` and `--metadata` on all three bulk creates, plus `--custom` on
122
+ `scrape bulk create` only (`/v1/bulk` and `/v1/fetch/bulk` have no top-level `custom`, so the
123
+ flag would be rejected or stripped there). `--defaults` is forwarded verbatim; the server owns
124
+ precedence and unknown-key handling.
125
+ - The full markdown-tuning surface on `fetch create`: `--markdown-query`, `--markdown-links`,
126
+ `--markdown-images`, `--with-links-summary`, `--with-images-summary`, `--markdown-compact`,
127
+ `--markdown-filter-images`, `--markdown-include-media`, `--markdown-include-warnings`,
128
+ `--markdown-include-stats`, and `--custom`.
129
+ - Webhook delivery and processing flags on `sitemap create`: `--webhook-url`, `--webhook-secret`,
130
+ `--cache-ttl`, `--compact`, `--process`, `--metadata`. Webhook-driven sitemap crawls were
131
+ previously unreachable from the shell.
132
+ - `--headers`, `--cookies` and `--metadata` on `screenshots create`.
133
+ - `--metadata` on `monitors create`, and `--metadata` + `--url` on `monitors update`. `--type`
134
+ remains deliberately absent from `update` — it is immutable server-side.
135
+
136
+ ## [0.5.1] - 2026-07-28 [NEVER PUBLISHED]
137
+
138
+ > **Never published to npm.** This version was cut in-tree but no `@hunt-labs/bytekit-cli@0.5.1`
139
+ > tarball exists — `npm i @hunt-labs/bytekit-cli@0.5.1` fails. Everything below shipped to users in
140
+ > **0.6.0**, which is the first published release containing it. Reconciled by issue #2680; the
141
+ > `release-drift` CI gate now blocks a new phantom heading from being introduced.
142
+
143
+ ### Fixed
144
+
145
+ - **Corrects the scope of the 0.5.0 "stalled body" fix claim.** `screenshots create`/`get -o`
146
+ downloads the `image_url` artifact bytes with a bare `globalThis.fetch` call
147
+ (`lib/download-artifact.ts`) that sat entirely outside the transport seam `sendWithRetry` owns —
148
+ no abort signal, no connection-error taxonomy. The 0.5.0 entry below ("The per-attempt request
149
+ timeout now covers the response body read... a server that answered with headers and then
150
+ stalled its body... hung the CLI forever", #2546) reads as if it covered every response body; it
151
+ only ever covered API-call bodies read through the JSON/raw transports. A stalled artifact host
152
+ (headers, then a body that never completes) still hung the CLI indefinitely until now. The
153
+ download path now shares the same per-attempt abort budget and `ByteKitConnectionError` timeout
154
+ taxonomy as the API transport (`createAttemptBudget` / `toConnectionError`, exported from
155
+ `client.ts` — no second timeout implementation), with the same 120 s per-attempt default and no
156
+ retry (artifact GETs are not the API's 429 domain). Partial-file policy: the response is
157
+ buffered fully in memory before any write, so an aborted download never leaves a
158
+ partial/truncated file at the `-o` destination. **User-visible consequence:** a legitimately
159
+ slow (but not stalled) large artifact download now hard-fails after 120 s instead of waiting
160
+ indefinitely, and there is currently no CLI flag to raise that budget for a single command.
161
+ (#2587)
162
+ - While in the file: the 429-retry path left its (unread) response body undrained before sleeping
163
+ and retrying — undici cannot reclaim the underlying socket for connection-pool reuse until a body
164
+ is fully read. The body is now drained with `response.text()` on that path, bounded by its own
165
+ short (2 s) cap independent of the 120 s per-attempt budget, so a 429 whose body stalls can no
166
+ longer delay the retry by anywhere near the full attempt budget (or, with the budget disabled,
167
+ forever). **Correction to an intermediate `Unreleased` draft of this same fix:** that draft's cap
168
+ only bounded how long the _retry_ waited — it resolved on schedule but never touched the
169
+ abandoned `response.text()` call itself, which kept reading against a live socket indefinitely.
170
+ Because every CLI success path exits via a soft `process.exitCode` (#2544) rather than
171
+ `process.exit()`, that orphaned read kept the whole process alive forever — strictly worse than
172
+ having no drain at all, and on the exact class of hang this ticket exists to eliminate. The cap
173
+ now also **aborts** the request when it fires, so the abandoned read is actually torn down, not
174
+ merely ignored; this holds even when the per-attempt timeout itself is disabled
175
+ (`timeoutMs <= 0`). **User-visible consequence:** a 429 with a stalled body still costs the retry
176
+ at most ~2 s, exactly as before, but the CLI process now reliably exits afterward instead of
177
+ hanging forever regardless of that visible latency. The same discard-drain on the
178
+ artifact-download's non-2xx path is removed outright rather than capped: that path throws
179
+ straight into `process.exit(1)`, so there is no subsequent request in the process for a reclaimed
180
+ socket to serve, and the drain there bought nothing. (#2587)
181
+
11
182
  ## [0.5.0] - 2026-07-24
12
183
 
184
+ ### Note
185
+
186
+ - The `0.3.1` and `0.4.0` versions below document real, tagged changes but were
187
+ never published to npm — the registry's version history jumps straight from
188
+ `0.3.0` to `0.5.0`. Don't go looking for those tarballs; the changes they
189
+ describe ship for the first time in this release.
190
+
13
191
  ### Added
14
192
 
15
193
  - Surface parity round 2 (#2545): every `x-sdk-scope: v0.1` endpoint the SDK
@@ -83,7 +261,12 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
83
261
  `sc_` id** on stdout with a note on stderr (it does not print the queued job
84
262
  envelope). (#2544)
85
263
 
86
- ## [0.4.0] - 2026-07-23
264
+ ## [0.4.0] - 2026-07-23 [NEVER PUBLISHED]
265
+
266
+ > **Never published to npm.** This version was cut in-tree but no `@hunt-labs/bytekit-cli@0.4.0`
267
+ > tarball exists — `npm i @hunt-labs/bytekit-cli@0.4.0` fails. Everything below shipped to users in
268
+ > **0.5.0**, which is the first published release containing it. Reconciled by issue #2680; the
269
+ > `release-drift` CI gate now blocks a new phantom heading from being introduced.
87
270
 
88
271
  ### Removed (**BREAKING**)
89
272
 
@@ -104,7 +287,13 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
104
287
  **Replacement:** manage API keys from the dashboard at
105
288
  [app.bytekit.com](https://app.bytekit.com). (#2536)
106
289
 
107
- ## [0.3.1] - 2026-07-22
290
+ ## [0.3.1] - 2026-07-22 [NEVER PUBLISHED]
291
+
292
+ > **Never published to npm.** This version was cut in-tree but no `@hunt-labs/bytekit-cli@0.3.1`
293
+ > tarball exists — `npm i @hunt-labs/bytekit-cli@0.3.1` fails. Everything below shipped to users in
294
+ > **0.5.0**, which is the first published release containing it (0.4.0 was never published either —
295
+ > see its heading above). Reconciled by issue #2680; the `release-drift` CI gate now blocks a new
296
+ > phantom heading from being introduced.
108
297
 
109
298
  ### Fixed
110
299
 
@@ -187,7 +376,12 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
187
376
  rebuilding the shared `dist`), which is now fixed. No consumer-facing changes
188
377
  beyond 0.2.1.
189
378
 
190
- ## [0.2.1] - 2026-07-16
379
+ ## [0.2.1] - 2026-07-16 [NEVER PUBLISHED]
380
+
381
+ > **Never published to npm.** This version was cut in-tree but no `@hunt-labs/bytekit-cli@0.2.1`
382
+ > tarball exists — `npm i @hunt-labs/bytekit-cli@0.2.1` fails. Everything below shipped to users in
383
+ > **0.2.2**, which is the first published release containing it. Reconciled by issue #2680; the
384
+ > `release-drift` CI gate now blocks a new phantom heading from being introduced.
191
385
 
192
386
  ### Changed
193
387
 
@@ -236,7 +430,13 @@ Additive, lower-risk release — no breaking changes to existing commands.
236
430
  - Initial release of the ByteKit CLI, wrapping the ByteKit SDK as shell
237
431
  subcommands, published under the original `rapidcrawl-cli` name.
238
432
 
433
+ [Unreleased]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
434
+ [0.7.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
435
+ [0.6.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
436
+ [0.5.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
437
+ [0.5.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
239
438
  [0.4.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
439
+ [0.3.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
240
440
  [0.3.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
241
441
  [0.2.2]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
242
442
  [0.2.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/cli
package/README.md CHANGED
@@ -172,6 +172,17 @@ response metadata instead of the body. `-o <file>` writes the body bytes to a
172
172
  file byte-for-byte (with `--json`, the metadata object goes to the file
173
173
  instead). `fetch bulk` returns an ordinary JSON job envelope.
174
174
 
175
+ The `--json` metadata keys are the **spec header names exactly** as the API
176
+ reference declares them, so a filter written from the docs works unchanged:
177
+
178
+ ```bash
179
+ bytekit fetch get --url https://example.com --json | jq -r '."X-Fetch-ID"'
180
+ ```
181
+
182
+ Note the acronym casing — `X-Fetch-ID` and `X-Fetch-URL`, not `X-Fetch-Id` /
183
+ `X-Fetch-Url`. (Versions up to 0.6.0 emitted the Title-Cased spellings; see the
184
+ CHANGELOG.)
185
+
175
186
  Both verbs accept `--country <cc>`, `--timeout-ms <n>`, and `--cache-ttl <ttl>`
176
187
  (`0`, `Nh`, or `Nd`), each omitted when absent. `--markdown-mode`
177
188
  (`article`, `raw`, `llm`) is `fetch create` only.
@@ -182,6 +193,11 @@ Both verbs accept `--country <cc>`, `--timeout-ms <n>`, and `--cache-ttl <ttl>`
182
193
  bytekit search "ai web scraping" --limit 10 --country us
183
194
  ```
184
195
 
196
+ `search` also accepts `--type <type>` (`web` (default), `news`, or `images`),
197
+ `--language <code>` (two-letter language code, e.g. `en`, `fr`), and
198
+ `--date-range <window>` (recency filter: `any` (default), `hour`, `day`,
199
+ `week`, `month`, `year`). Each flag is omitted from the request when absent.
200
+
185
201
  ### `screenshots` — capture a page
186
202
 
187
203
  ```bash
@@ -232,7 +248,7 @@ bytekit monitors create --url https://example.com/pricing --interval-type daily
232
248
  bytekit monitors list --limit 25 --status active
233
249
  bytekit monitors update mon_123abc --status paused # pause
234
250
  bytekit monitors update mon_123abc --status active # resume
235
- bytekit monitors captures list mon_123abc
251
+ bytekit monitors captures list mon_123abc --limit 25
236
252
  bytekit monitors delete mon_123abc
237
253
  ```
238
254
 
@@ -240,6 +256,10 @@ bytekit monitors delete mon_123abc
240
256
  (`active`, `processing`, `paused`, `cancelled`, `suspended`) as query filters,
241
257
  each omitted when absent.
242
258
 
259
+ `monitors captures list <id>` accepts `--limit <n>` and `--cursor <cursor>` to
260
+ page through a monitor's capture history — pass the response's `next_cursor`
261
+ back as `--cursor` to fetch the next page. Both flags are omitted when absent.
262
+
243
263
  `monitors update` gains `--status` (`active` | `paused` — pause and resume from
244
264
  the shell), `--change-threshold <pct>` (accepts a float), `--notify-on`
245
265
  (`change`, `every`), `--webhook-secret <secret>`, and three JSON-object flags:
@@ -294,11 +314,47 @@ omitted when absent. `retry <id>` requeues a failed or exhausted delivery.
294
314
 
295
315
  ## Exit codes
296
316
 
297
- | Code | Meaning |
298
- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
299
- | `0` | Success. |
300
- | `1` | Usage error, or an API/connection error. The reason is on stderr, always prefixed `Error: `. |
301
- | `2` | The scrape was accepted but is still **queued**, so there is no content body to emit — `scrape create --async` with `--raw` or a bare `-o` (which writes the bare `sc_` id to stdout), and the same command in human mode. Not a failure: the job was accepted and billed. Poll it with `bytekit scrape get <id>`. |
317
+ | Code | Meaning |
318
+ | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
319
+ | `0` | Success. |
320
+ | `1` | Usage error, or an API/connection error. |
321
+ | `2` | The job was accepted but is **not ready**, so there is no content body or artifact to emit. Applies to `scrape create --async` with `--raw` or a bare `-o` (which writes the bare `sc_` id to stdout) and in human mode; to `scrape get --raw`/`-o` on a still-queued job (bare `sc_` id to stdout); to `screenshots create --async` in human mode (including with `-o`, which deliberately writes no file); and to `screenshots create -o` / `screenshots get -o` on a pending or processing screenshot (bare `ss_` id to stdout, no file written). Not a failure: the job was accepted and billed. Poll it with `bytekit scrape get <id>` / `bytekit screenshots get <id>`. **`--json` keeps exit `0` in every case** — the envelope is a complete machine-readable payload. |
322
+
323
+ A **terminally failed** job is NOT code `2` — it will never complete, so it is
324
+ exit `1` with the reason on stderr. `scrape get --raw` on a failed scrape prints
325
+ that scrape's own `error.code` and message, and `screenshots get -o` on a failed
326
+ screenshot prints `screenshot failed: …`.
327
+
328
+ Only `pending` and `processing` earn code `2`. Any other screenshot state that
329
+ still has no artifact — a `completed` screenshot whose `image_url` never
330
+ materialized, or a status this CLI build does not know — is exit `1` with
331
+ `no image artifact available: screenshot is <status>` on stderr, because
332
+ polling it again cannot change the outcome.
333
+
334
+ The status is what decides, not the presence of an error. A `pending` or
335
+ `processing` screenshot that reports an `error_code` from an attempt that failed
336
+ and is being **retried** is still code `2` with its `ss_` id — the job has not
337
+ finished, and abandoning it there would throw away a capture the server is still
338
+ working on.
339
+
340
+ A **human-mode poll without `-o`/`--raw`** stays exit `0` whatever the job's
341
+ state: `bytekit scrape get <id>` and `bytekit screenshots get <id>` were asked
342
+ for the envelope, and the envelope is exactly what they printed — the status
343
+ field in it is the answer. Code `2` is reserved for the case where you asked for
344
+ a **body or artifact** that does not exist yet.
345
+
346
+ The stderr message for exit code `1` has two shapes, depending on where the
347
+ error originates:
348
+
349
+ - **App-level errors** (a rejected API call, a connection failure, a missing
350
+ key) are always prefixed `Error: ` (uppercase) — e.g.
351
+ `Error: API key required. Pass --key or set BYTEKIT_API_KEY.`
352
+ - **Commander-level usage errors** (an unknown command, a missing required
353
+ argument, an unrecognized flag) come from the `commander` library itself and
354
+ use its own lowercase `error: ` prefix — e.g. `error: unknown command 'bogus'`.
355
+
356
+ Scripts that grep stderr for a failure reason should match case-insensitively,
357
+ or handle both prefixes.
302
358
 
303
359
  Code `2` is deliberately distinct from both `0` and `1` so a script can branch
304
360
  on "queued" without parsing stderr:
@@ -312,6 +368,18 @@ case $? in
312
368
  esac
313
369
  ```
314
370
 
371
+ The same branch works for a screenshot poll, because both commands now signal
372
+ "not ready" identically:
373
+
374
+ ```bash
375
+ ss_id=$(bytekit screenshots get "$ss_id" -o shot.png)
376
+ case $? in
377
+ 0) echo "downloaded shot.png" ;;
378
+ 2) echo "$ss_id not ready — poll again" ;;
379
+ *) exit 1 ;; # terminal failure, reason on stderr
380
+ esac
381
+ ```
382
+
315
383
  ## Documentation
316
384
 
317
385
  Full API reference: [bytekit.com/docs](https://bytekit.com/docs).
package/dist/client.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { ByteKit } from '@hunt-labs/bytekit-sdk';
1
+ import { ByteKit, ByteKitConnectionError } from '@hunt-labs/bytekit-sdk';
2
2
  export interface GlobalOptions {
3
3
  key?: string | undefined;
4
4
  baseUrl?: string | undefined;
@@ -22,6 +22,24 @@ export declare const MIN_RETRY_AFTER_MS = 1000;
22
22
  export declare const MAX_RETRY_AFTER_MS = 60000;
23
23
  /** Exactly one retry. A second 429 is terminal — no loop. */
24
24
  export declare const MAX_RETRIES = 1;
25
+ /**
26
+ * Cap on a best-effort, discard-only body drain (#2587 cr2-1). Independent of the per-attempt
27
+ * abort budget: the 429 drain below (`response.text().catch(() => {})`) exists ONLY so undici can
28
+ * return the socket to the connection pool — nothing downstream reads the drained bytes — but
29
+ * binding that read to the FULL per-attempt budget (120 s default, or, under `timeoutMs <= 0`,
30
+ * to no bound at all) let a stalled 429 body burn the whole budget, or hang forever, before a
31
+ * retry that needs nothing from that body. Short enough that a legitimately stalled 429 body costs
32
+ * the retry very little wall-clock time.
33
+ *
34
+ * #2587 cr3-2: past this cap, the read is not merely abandoned — {@link drainBestEffort} actively
35
+ * ABORTS its underlying request. Resolving the race without aborting anything (the earlier
36
+ * cr2-1 shape) let the retry proceed on schedule while the abandoned `response.text()` kept
37
+ * reading against a live socket forever, which — because the CLI exits via a soft
38
+ * `process.exitCode` (#2544) rather than `process.exit()` — kept the whole process alive
39
+ * indefinitely. `.catch`-guarding the read only made it safe to abandon the RESULT; it never made
40
+ * abandoning the underlying READ safe.
41
+ */
42
+ export declare const DRAIN_TIMEOUT_MS = 2000;
25
43
  /**
26
44
  * Default CLIENT-SIDE abort budget for a single fetch attempt, in ms.
27
45
  *
@@ -33,6 +51,49 @@ export declare const MAX_RETRIES = 1;
33
51
  * exists.
34
52
  */
35
53
  export declare const DEFAULT_TIMEOUT_MS = 120000;
54
+ /**
55
+ * A per-attempt abort budget: an `AbortController`, optionally fired by a `setTimeout`.
56
+ *
57
+ * The budget spans the WHOLE attempt — time-to-headers **and** the response body read (#2546).
58
+ * Arming it only until headers arrive left `response.text()` / `response.arrayBuffer()` unbounded,
59
+ * so a server that answered with headers and then stalled its body hung the CLI forever.
60
+ *
61
+ * The controller ALWAYS exists, even when `ms <= 0` — only its own timeout timer is conditional
62
+ * on `ms > 0`. `sendWithRetry`'s 429-drain cap ({@link DRAIN_TIMEOUT_MS}, via the returned
63
+ * `abort()`) needs a live controller to tear down an abandoned body read regardless of whether the
64
+ * OVERALL per-attempt budget is enabled — `ms <= 0` must still mean "no full-attempt timeout", not
65
+ * "no controller to abort a stalled drain with" (#2587 cr3-2: the earlier drain fix abandoned the
66
+ * read entirely rather than aborting it, so the socket — and the event loop — stayed alive
67
+ * indefinitely, worse than the pre-fix baseline it replaced).
68
+ *
69
+ * Deliberately NOT `AbortSignal.timeout` (which the SDK prefers): that schedules on a timer the
70
+ * test runner's fake timers cannot advance, so the transport's timeout would be untestable under
71
+ * `vi.useFakeTimers()`. CLI requests are short-lived and single-shot, so the GC advantage
72
+ * `AbortSignal.timeout` buys the long-running SDK does not apply here. The timer is `unref`'d so a
73
+ * pending budget never keeps the process alive, and `cleanup()` clears it on every exit path.
74
+ *
75
+ * Exported so `lib/download-artifact.ts` can reuse the same budget for the artifact-download path
76
+ * (#2587) rather than growing a second timeout implementation — that path sits entirely outside
77
+ * `sendWithRetry` (it is not a JSON/raw API call), so it would otherwise never inherit this bound.
78
+ */
79
+ export declare function createAttemptBudget(ms: number): {
80
+ signal: AbortSignal;
81
+ cleanup: () => void;
82
+ abort: (reason: unknown) => void;
83
+ };
84
+ /**
85
+ * Map a `fetch` or body-read rejection (no usable HTTP payload was produced) onto the SDK's
86
+ * `ByteKitConnectionError` taxonomy, so a CLI failure carries the same `{name, code, cause}` a
87
+ * direct SDK call would.
88
+ *
89
+ * `timedOut` is true when this attempt's own budget fired the abort, which is the reliable
90
+ * discriminator (undici rejects with the signal's reason — or a wrapper carrying it, see
91
+ * {@link abortNameOf} — whose `name` we also inspect as a fallback for a caller-style AbortError).
92
+ *
93
+ * Exported so `lib/download-artifact.ts` reuses the same taxonomy for the artifact-download path
94
+ * (#2587) instead of hand-rolling a second error mapping.
95
+ */
96
+ export declare function toConnectionError(err: unknown, method: string, path: string, timedOut: boolean): ByteKitConnectionError;
36
97
  /**
37
98
  * Parse a `Retry-After` header into a clamped millisecond delay.
38
99
  *
@@ -48,6 +109,18 @@ export declare function parseRetryAfterMs(raw: string | null): number;
48
109
  * property of the client (see {@link buildClient}) routes all JSON commands through it from one
49
110
  * seam. `/v1/fetch` does NOT ride this transport — it rides `requestRaw` (see
50
111
  * {@link createRetryingRequestRaw}); `/v1/fetch/bulk` is a JSON job envelope and does.
112
+ *
113
+ * DECISION (#2586 dod-4): the 4th `RequestOptions` parameter of the SDK's contract is accepted
114
+ * here — so the signature matches `ByteKit['request']` exactly, closing the gap where a caller-typo
115
+ * of a 5th positional argument would silently no-op instead of a compile error — but deliberately
116
+ * NOT forwarded into `sendWithRetry`. Two reasons: (1) no CLI command constructs a `RequestOptions`
117
+ * today (no `--timeout` flag exists yet, and `signal` has no CLI-level source), so the parameter is
118
+ * always `undefined` in practice; (2) the CLI already owns a per-attempt abort budget with its own
119
+ * semantics (`createAttemptBudget`, `timeoutMs` above) that would need explicit composition with a
120
+ * caller `signal` to honor `RequestOptions` correctly (the SDK's `composeSignals`) — inventing that
121
+ * unexercised composition here, with no caller able to reach it, would be undertested surface for no
122
+ * present behavior change. If a future CLI flag needs per-call overrides, wire it through
123
+ * `sendWithRetry`'s `timeoutMs` (already per-attempt) at that point, with its own tests.
51
124
  */
52
125
  export declare function createRetryingRequest(apiKey: string, baseUrl: string, timeoutMs?: number): ByteKit['request'];
53
126
  /**
@@ -62,19 +135,66 @@ export declare function createRetryingRequest(apiKey: string, baseUrl: string, t
62
135
  * so `FetchResource` builds an ordinary `FetchResult` from it and the CLI never forks fetch
63
136
  * handling away from the SDK. `bytes` is byte-exact (binary-safe); `body` is the best-effort UTF-8
64
137
  * decode, matching the SDK's own semantics.
138
+ *
139
+ * The 4th `RequestOptions` parameter is accepted but not forwarded — see the DECISION note on
140
+ * {@link createRetryingRequest}, which applies here verbatim. `FetchResource.create`/`get` DO thread
141
+ * a `requestOptions` argument through to this seam (unlike every other resource), but no CLI `fetch`
142
+ * command constructs one today (no `--timeout` flag), so it is still always `undefined` in practice.
65
143
  */
66
144
  export declare function createRetryingRequestRaw(apiKey: string, baseUrl: string, timeoutMs?: number): ByteKit['requestRaw'];
145
+ /**
146
+ * Build the CLI's header-carrying JSON transport: the SDK's `requestWithResponse` contract
147
+ * (`{data, headers, status}`, #2547), plus the same bounded 429 retry every other CLI transport
148
+ * gets.
149
+ *
150
+ * Unused by any CLI command today — `screenshots.createWithResponse` / `getWithResponse` and
151
+ * `sitemap.getWithResponse` all exist on the SDK surface but no command builds on them yet. It is
152
+ * still installed (see {@link buildClient}) rather than left un-overridden: the transport-coverage
153
+ * contract test (#2586 dod-3) derives its method list from the real SDK prototype, and leaving a
154
+ * transport method un-overridden is exactly how #2535 shipped broken — the first command that DOES
155
+ * call a `*WithResponse` resource method must not silently ride the bare SDK transport and skip the
156
+ * CLI's retry/timeout/error taxonomy.
157
+ *
158
+ * Same body-read shape as {@link createRetryingRequest} (204/empty → `undefined`, `JSON.parse`
159
+ * otherwise), with `headers` and `status` threaded through from the successful response alongside
160
+ * the parsed `data`. The 4th `RequestOptions` parameter is accepted but not forwarded — see the
161
+ * DECISION note on {@link createRetryingRequest}.
162
+ */
163
+ export declare function createRetryingRequestWithResponse(apiKey: string, baseUrl: string, timeoutMs?: number): ByteKit['requestWithResponse'];
67
164
  /**
68
165
  * Build an authenticated SDK client from global CLI options.
69
166
  *
70
167
  * Resolution order (explicit input beats environment):
71
168
  * apiKey: --key flag → BYTEKIT_API_KEY.
72
169
  * baseUrl: --base-url flag → BYTEKIT_BASE_URL → DEFAULT_BASE_URL.
73
- * Exits with a descriptive error when no key is available.
74
170
  *
75
- * The returned client's `request` AND `requestRaw` are both replaced by the retrying transports, so
76
- * every command inherits the 429 retry — JSON commands via `request`, the `/v1/fetch` content
77
- * commands via `requestRaw`. Overriding only `request` is the #2535 defect: the SDK's fetch
78
- * resource rides `requestRaw`, so fetch bypassed the retry and the CLI's error taxonomy entirely.
171
+ * THROWS a plain `Error` when no key is available (#2686 item 4). It used to `console.error` and
172
+ * then hard-exit — an async stderr write followed immediately by the one call that discards
173
+ * the userspace stdio queue, which is exactly the defect the package's #2544 flush policy exists to
174
+ * prevent. Throwing stops just as hard (there is no client to return, E4) while routing the message
175
+ * through `handleError`, whose soft `process.exitCode` lets stderr drain first. The rendered stderr
176
+ * line is byte-identical: `handleError` supplies the `Error: ` prefix, and a plain Error carries
177
+ * neither a status nor a ByteKit code, so no parenthesised suffix is added.
178
+ *
179
+ * Because it throws, EVERY caller invokes it from inside its action's `try` block. `index.ts` drives
180
+ * commander with `.parse()`, not `.parseAsync()`, so nothing awaits an action's promise: a throw
181
+ * raised above the try would escape as an unhandled rejection and print a raw stack trace instead of
182
+ * the clean one-liner — the same constraint `commands/scrape.ts` already documents for
183
+ * `resolveOutputMode`. `error-exit-contract.test.ts` pins that placement structurally, across every
184
+ * call site at once.
185
+ *
186
+ * The returned client's `request`, `requestRaw`, AND `requestWithResponse` are all replaced by the
187
+ * retrying transports, so every command inherits the 429 retry — JSON commands via `request`, the
188
+ * `/v1/fetch` content commands via `requestRaw`, and the header-carrying `*WithResponse` surface via
189
+ * `requestWithResponse`. Overriding only `request` is the #2535 defect: the SDK's fetch resource
190
+ * rides `requestRaw`, so fetch bypassed the retry and the CLI's error taxonomy entirely; leaving
191
+ * `requestWithResponse` un-overridden is the same class of defect (#2586) — it is unused by any CLI
192
+ * command TODAY, but a future command built on `screenshots.createWithResponse` /
193
+ * `sitemap.getWithResponse` would otherwise silently ride the bare SDK transport.
194
+ *
195
+ * `baseUrl` is normalized ONCE, here, before it reaches either the SDK constructor or the CLI's own
196
+ * transport builders below — the CLI's transports build their request URL from this local variable
197
+ * directly (they do not call the SDK's own normalized `send()`), so a trailing-slash flag would
198
+ * otherwise double-slash every request the CLI makes (#2586).
79
199
  */
80
200
  export declare function buildClient(opts: GlobalOptions): ByteKit;