@hunt-labs/bytekit-cli 0.8.1 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -2
- package/README.md +3 -22
- package/dist/commands/fetch.d.ts +0 -14
- package/dist/commands/fetch.js +2 -63
- package/dist/commands/scrape.js +1 -7
- package/dist/commands/screenshots.js +2 -8
- package/dist/lib/parse-flags.d.ts +1 -1
- package/dist/lib/parse-flags.js +1 -1
- package/dist/lib/url-lines.d.ts +6 -6
- package/dist/lib/url-lines.js +6 -6
- package/dist/test-helpers/build-scratch.d.ts +23 -0
- package/dist/test-helpers/build-scratch.js +41 -0
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -10,6 +10,25 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
|
|
|
10
10
|
|
|
11
11
|
## [Unreleased]
|
|
12
12
|
|
|
13
|
+
## [0.9.0] - 2026-08-20
|
|
14
|
+
|
|
15
|
+
### Removed
|
|
16
|
+
|
|
17
|
+
- **BREAKING: `--cookies` and `--headers` are gone from `scrape create` and `screenshots create`**
|
|
18
|
+
(#3242). The API no longer accepts caller-supplied cookies or request headers on any capture
|
|
19
|
+
endpoint, so passing either flag is now an unknown-option error before any request is issued,
|
|
20
|
+
and a body carrying those keys is rejected by the server with `400 validation_error`. There is
|
|
21
|
+
no deprecation window, feature flag, or compatibility header.
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
# before
|
|
25
|
+
bytekit scrape create --url https://example.com --cookies '[{"name":"session","value":"abc"}]'
|
|
26
|
+
# after — the flag does not exist
|
|
27
|
+
bytekit scrape create --url https://example.com
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`--webhook-headers` on `monitors` is a different flag and is unaffected.
|
|
31
|
+
|
|
13
32
|
## [0.8.1] - 2026-08-13
|
|
14
33
|
|
|
15
34
|
### Fixed
|
|
@@ -75,6 +94,16 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
|
|
|
75
94
|
npm when the entries above landed, so they ride this same pending release rather than stranding
|
|
76
95
|
it as a never-published version.
|
|
77
96
|
|
|
97
|
+
- **Corrected a false legacy-name claim carried since the rename (#2950).** The `0.8.0`-era note
|
|
98
|
+
below said the prior `rapidcrawl-cli` name "remains installable at its last version but is
|
|
99
|
+
deprecated." Verified 2026-08-13: `rapidcrawl-cli` returns npm `404` ("is not in this
|
|
100
|
+
registry"), and `npm deprecate rapidcrawl-cli@'*'` fails with the same `404` — there is no
|
|
101
|
+
published version left for a deprecation notice to attach to, so the claim never held. No
|
|
102
|
+
registry stub or deprecation was possible to publish (npm has nothing to act on); root
|
|
103
|
+
`CLAUDE.md` carried the same false claim and is corrected alongside this entry. Users of the old
|
|
104
|
+
name should install `@hunt-labs/bytekit-cli` directly. No command, flag, or runtime behavior
|
|
105
|
+
changes.
|
|
106
|
+
|
|
78
107
|
## [0.8.0] - 2026-08-11
|
|
79
108
|
|
|
80
109
|
### Added
|
|
@@ -511,8 +540,10 @@ Additive, lower-risk release — no breaking changes to existing commands.
|
|
|
511
540
|
### Changed
|
|
512
541
|
|
|
513
542
|
- Package renamed to `@hunt-labs/bytekit-cli` (the `@bytekit` npm scope was
|
|
514
|
-
unobtainable); the `bin` name stays `bytekit`. The prior `rapidcrawl-cli` name
|
|
515
|
-
remains installable at its last version but is deprecated
|
|
543
|
+
unobtainable); the `bin` name stays `bytekit`. ~~The prior `rapidcrawl-cli` name
|
|
544
|
+
remains installable at its last version but is deprecated.~~ **Corrected in
|
|
545
|
+
0.8.1 (see below): that claim does not hold — `rapidcrawl-cli` 404s on npm and
|
|
546
|
+
there is no published version left to deprecate.**
|
|
516
547
|
- Consumer call sites propagate the SDK's narrowed, spec-derived request types (#2426).
|
|
517
548
|
- Publish-ready packaging metadata: dropped `main`, unpinned the SDK dependency to
|
|
518
549
|
`workspace:^`, added `engines`, `keywords`, `homepage`, `bugs`, `README.md`,
|
package/README.md
CHANGED
|
@@ -171,7 +171,7 @@ exits `0`.
|
|
|
171
171
|
|
|
172
172
|
```bash
|
|
173
173
|
bytekit fetch create --url https://example.com
|
|
174
|
-
bytekit fetch get --url https://example.com
|
|
174
|
+
bytekit fetch get --url https://example.com
|
|
175
175
|
bytekit fetch get --url https://example.com/logo.png -o logo.png
|
|
176
176
|
bytekit fetch bulk create --file urls.txt --webhook-url https://your.app/hooks/bytekit
|
|
177
177
|
```
|
|
@@ -193,30 +193,11 @@ Note the acronym casing — `X-Fetch-ID` and `X-Fetch-URL`, not `X-Fetch-Id` /
|
|
|
193
193
|
`X-Fetch-Url`. (Versions up to 0.6.0 emitted the Title-Cased spellings; see the
|
|
194
194
|
CHANGELOG.)
|
|
195
195
|
|
|
196
|
-
Both verbs **require** `--url <url>`.
|
|
197
|
-
(`markdown` or `html`), `--country <cc>`, `--timeout-ms <n>`, `--cache-ttl <ttl>`
|
|
196
|
+
Both verbs **require** `--url <url>`. They accept `--country <cc>`, `--timeout-ms <n>`, `--cache-ttl <ttl>`
|
|
198
197
|
(`0`, `Nh`, or `Nd`), and `-o, --output <file>` — each of those omitted from the
|
|
199
198
|
request when absent.
|
|
200
199
|
|
|
201
|
-
`fetch create`
|
|
202
|
-
`fetch get` does not:
|
|
203
|
-
|
|
204
|
-
| Flag | Effect |
|
|
205
|
-
| ----------------------------- | ---------------------------------------------------------------------------- |
|
|
206
|
-
| `--markdown-mode <mode>` | Markdown extraction mode: `article`, `raw`, `llm`. |
|
|
207
|
-
| `--markdown-query <q>` | BM25 query for relevance-ranked markdown filtering. Omitted → disabled. |
|
|
208
|
-
| `--markdown-links <mode>` | Link rendering: `inline`, `references`, `none`, `text`. |
|
|
209
|
-
| `--markdown-images <mode>` | Image retention: `inline`, `references`, `none`, `text`. |
|
|
210
|
-
| `--with-links-summary` | Append a Links footer to the markdown. |
|
|
211
|
-
| `--with-images-summary` | Append an Images footer to the markdown. |
|
|
212
|
-
| `--markdown-compact` | Collapse excessive whitespace. |
|
|
213
|
-
| `--markdown-filter-images` | Filter low-signal images. |
|
|
214
|
-
| `--markdown-include-media` | Return rich link/image objects and a tables array. |
|
|
215
|
-
| `--markdown-include-warnings` | Include markdown-pipeline and tag-filter warnings. |
|
|
216
|
-
| `--markdown-include-stats` | Include a top-level stats object (chars, tokens, blocks). |
|
|
217
|
-
| `--custom <json>` | User payload echoed on the envelope, as a JSON object, e.g. `'{"job":"1"}'`. |
|
|
218
|
-
|
|
219
|
-
Each is omitted when absent, so a server default is never materialized.
|
|
200
|
+
`fetch create` and `fetch get` send only the raw-fetch options listed above.
|
|
220
201
|
|
|
221
202
|
`fetch bulk create` takes the shared bulk-job flags — see
|
|
222
203
|
[`bulk`](#bulk--bulk-screenshot-jobs).
|
package/dist/commands/fetch.d.ts
CHANGED
|
@@ -1,19 +1,5 @@
|
|
|
1
1
|
import { Command } from 'commander';
|
|
2
2
|
import { type GlobalOptions } from '../client.js';
|
|
3
|
-
/** The `/v1/fetch` spec union for `format` — mirrors `FetchOpts['format']` in the SDK. */
|
|
4
|
-
type FetchFormat = 'markdown' | 'html';
|
|
5
|
-
/**
|
|
6
|
-
* Validate `--format` locally, before any HTTP request is issued.
|
|
7
|
-
*
|
|
8
|
-
* The SDK's `FetchOpts.format` is narrowed to the spec union `'markdown' | 'html'`
|
|
9
|
-
* (#2475), so an off-spec value like `pdf` is a compile-time error at the call site.
|
|
10
|
-
* Commander runs an option's argParser during parsing — ahead of the action callback —
|
|
11
|
-
* so rejecting here means the client is never built and no request is issued, and the
|
|
12
|
-
* caller gets an actionable message instead of a server 422 (same convention as
|
|
13
|
-
* `parseLimit` in search.ts).
|
|
14
|
-
*/
|
|
15
|
-
export declare function parseFormat(value: string): FetchFormat;
|
|
16
3
|
export declare function makeFetchCommand(getGlobal: () => GlobalOptions & {
|
|
17
4
|
json: boolean;
|
|
18
5
|
}): Command;
|
|
19
|
-
export {};
|
package/dist/commands/fetch.js
CHANGED
|
@@ -1,29 +1,8 @@
|
|
|
1
|
-
import { Command
|
|
1
|
+
import { Command } from 'commander';
|
|
2
2
|
import { buildClient } from '../client.js';
|
|
3
3
|
import { printResult, emitFetchResult, handleError } from '../output.js';
|
|
4
4
|
import { parseBulkLines } from '../lib/url-lines.js';
|
|
5
|
-
import { parseIntFlag,
|
|
6
|
-
const FETCH_FORMATS = ['markdown', 'html'];
|
|
7
|
-
/** The `FetchRequest.markdown_mode` spec enum (create/POST only — excluded from the GET query). */
|
|
8
|
-
const MARKDOWN_MODES = ['article', 'raw', 'llm'];
|
|
9
|
-
/** The `markdown_links` / `markdown_images` render-mode enum (shared with `scrape create`). */
|
|
10
|
-
const MARKDOWN_RENDER_MODES = ['inline', 'references', 'none', 'text'];
|
|
11
|
-
/**
|
|
12
|
-
* Validate `--format` locally, before any HTTP request is issued.
|
|
13
|
-
*
|
|
14
|
-
* The SDK's `FetchOpts.format` is narrowed to the spec union `'markdown' | 'html'`
|
|
15
|
-
* (#2475), so an off-spec value like `pdf` is a compile-time error at the call site.
|
|
16
|
-
* Commander runs an option's argParser during parsing — ahead of the action callback —
|
|
17
|
-
* so rejecting here means the client is never built and no request is issued, and the
|
|
18
|
-
* caller gets an actionable message instead of a server 422 (same convention as
|
|
19
|
-
* `parseLimit` in search.ts).
|
|
20
|
-
*/
|
|
21
|
-
export function parseFormat(value) {
|
|
22
|
-
if (!FETCH_FORMATS.includes(value)) {
|
|
23
|
-
throw new InvalidArgumentError(`--format must be one of: ${FETCH_FORMATS.join(', ')}.`);
|
|
24
|
-
}
|
|
25
|
-
return value;
|
|
26
|
-
}
|
|
5
|
+
import { parseIntFlag, parseCacheTtl, parseJsonFlag } from '../lib/parse-flags.js';
|
|
27
6
|
export function makeFetchCommand(getGlobal) {
|
|
28
7
|
const cmd = new Command('fetch').description('Fetch URL content operations');
|
|
29
8
|
const bulkCmd = new Command('bulk').description('Bulk fetch operations');
|
|
@@ -82,23 +61,9 @@ export function makeFetchCommand(getGlobal) {
|
|
|
82
61
|
.command('create')
|
|
83
62
|
.description('Fetch URL content (POST)')
|
|
84
63
|
.requiredOption('--url <url>', 'URL to fetch')
|
|
85
|
-
.option('--format <format>', 'Response format (markdown or html)', parseFormat)
|
|
86
64
|
.option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
|
|
87
65
|
.option('--timeout-ms <n>', 'Per-request timeout in milliseconds', (v) => parseIntFlag('--timeout-ms', v))
|
|
88
66
|
.option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
|
|
89
|
-
// The markdown surface below is POST-only: FetchGetOpts excludes it, so adding any of it to
|
|
90
|
-
// `get` would be a compile error — the create/get asymmetry is correct by construction.
|
|
91
|
-
.option('--markdown-mode <mode>', `Markdown mode: ${MARKDOWN_MODES.join(', ')}`, (v) => parseEnum('--markdown-mode', MARKDOWN_MODES, v))
|
|
92
|
-
.option('--markdown-query <q>', 'BM25 query string for relevance-ranked markdown filtering (omitted → disabled)')
|
|
93
|
-
.option('--markdown-links <mode>', `Link rendering: ${MARKDOWN_RENDER_MODES.join(', ')} (omitted → server default)`, (v) => parseEnum('--markdown-links', MARKDOWN_RENDER_MODES, v))
|
|
94
|
-
.option('--markdown-images <mode>', `Image retention: ${MARKDOWN_RENDER_MODES.join(', ')} (omitted → server default)`, (v) => parseEnum('--markdown-images', MARKDOWN_RENDER_MODES, v))
|
|
95
|
-
.option('--with-links-summary', 'Append a Links footer to the markdown output (omitted → off)')
|
|
96
|
-
.option('--with-images-summary', 'Append an Images footer to the markdown output (omitted → off)')
|
|
97
|
-
.option('--markdown-compact', 'Collapse excessive whitespace in the markdown output (omitted → off)')
|
|
98
|
-
.option('--markdown-filter-images', 'Filter low-signal images from the markdown output (omitted → off)')
|
|
99
|
-
.option('--markdown-include-media', 'Return rich link/image objects and a tables array (omitted → off)')
|
|
100
|
-
.option('--markdown-include-warnings', 'Include markdown-pipeline and tag-filter warnings (omitted → off)')
|
|
101
|
-
.option('--markdown-include-stats', 'Include a top-level stats object (chars, tokens, blocks) (omitted → off)')
|
|
102
67
|
.option('--custom <json>', `User payload echoed on the envelope as a JSON object, e.g. '{"job":"1"}' (omitted → none)`, (v) => parseJsonFlag('--custom', v, 'object'))
|
|
103
68
|
.option('-o, --output <file>', 'Write the fetched body bytes to the given file path')
|
|
104
69
|
.action(async (opts) => {
|
|
@@ -108,36 +73,12 @@ export function makeFetchCommand(getGlobal) {
|
|
|
108
73
|
// exact same wire output they had as inline spreads; a boolean is written ONLY on
|
|
109
74
|
// `=== true`, so it never serializes `false` (#2480 E1).
|
|
110
75
|
const body = { url: opts.url };
|
|
111
|
-
if (opts.format !== undefined)
|
|
112
|
-
body['format'] = opts.format;
|
|
113
76
|
if (opts.country !== undefined)
|
|
114
77
|
body['country'] = opts.country;
|
|
115
78
|
if (opts.timeoutMs !== undefined)
|
|
116
79
|
body['timeout_ms'] = opts.timeoutMs;
|
|
117
80
|
if (opts.cacheTtl !== undefined)
|
|
118
81
|
body['cache_ttl'] = opts.cacheTtl;
|
|
119
|
-
if (opts.markdownMode !== undefined)
|
|
120
|
-
body['markdown_mode'] = opts.markdownMode;
|
|
121
|
-
if (opts.markdownQuery !== undefined)
|
|
122
|
-
body['markdown_query'] = opts.markdownQuery;
|
|
123
|
-
if (opts.markdownLinks !== undefined)
|
|
124
|
-
body['markdown_links'] = opts.markdownLinks;
|
|
125
|
-
if (opts.markdownImages !== undefined)
|
|
126
|
-
body['markdown_images'] = opts.markdownImages;
|
|
127
|
-
if (opts.withLinksSummary === true)
|
|
128
|
-
body['with_links_summary'] = true;
|
|
129
|
-
if (opts.withImagesSummary === true)
|
|
130
|
-
body['with_images_summary'] = true;
|
|
131
|
-
if (opts.markdownCompact === true)
|
|
132
|
-
body['markdown_compact'] = true;
|
|
133
|
-
if (opts.markdownFilterImages === true)
|
|
134
|
-
body['markdown_filter_images'] = true;
|
|
135
|
-
if (opts.markdownIncludeMedia === true)
|
|
136
|
-
body['markdown_include_media'] = true;
|
|
137
|
-
if (opts.markdownIncludeWarnings === true)
|
|
138
|
-
body['markdown_include_warnings'] = true;
|
|
139
|
-
if (opts.markdownIncludeStats === true)
|
|
140
|
-
body['markdown_include_stats'] = true;
|
|
141
82
|
if (opts.custom !== undefined)
|
|
142
83
|
body['custom'] = opts.custom;
|
|
143
84
|
try {
|
|
@@ -154,7 +95,6 @@ export function makeFetchCommand(getGlobal) {
|
|
|
154
95
|
.command('get')
|
|
155
96
|
.description('Fetch URL content (GET via query params)')
|
|
156
97
|
.requiredOption('--url <url>', 'URL to fetch')
|
|
157
|
-
.option('--format <format>', 'Response format (markdown or html)', parseFormat)
|
|
158
98
|
.option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
|
|
159
99
|
.option('--timeout-ms <n>', 'Per-request timeout in milliseconds', (v) => parseIntFlag('--timeout-ms', v))
|
|
160
100
|
.option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
|
|
@@ -166,7 +106,6 @@ export function makeFetchCommand(getGlobal) {
|
|
|
166
106
|
const client = buildClient(g);
|
|
167
107
|
const result = await client.fetch.get({
|
|
168
108
|
url: opts.url,
|
|
169
|
-
...(opts.format !== undefined ? { format: opts.format } : {}),
|
|
170
109
|
...(opts.country !== undefined ? { country: opts.country } : {}),
|
|
171
110
|
...(opts.timeoutMs !== undefined ? { timeout_ms: opts.timeoutMs } : {}),
|
|
172
111
|
...(opts.cacheTtl !== undefined ? { cache_ttl: opts.cacheTtl } : {}),
|
package/dist/commands/scrape.js
CHANGED
|
@@ -48,7 +48,7 @@ export function makeScrapeCommand(getGlobal) {
|
|
|
48
48
|
.requiredOption('--file <path>', 'Path to a newline-delimited file. Each line is a plain URL, or a JSON object carrying "url" plus per-item config. A line whose only key is "url" behaves exactly as before (sent as a bare URL string); any line with additional keys switches the payload to the per-item shape.')
|
|
49
49
|
.requiredOption('--webhook-url <url>', 'HTTPS webhook URL notified when the job completes')
|
|
50
50
|
.option('--webhook-secret <secret>', 'HMAC secret used to sign the completion webhook (omitted → unsigned)')
|
|
51
|
-
.option('--defaults <json>', 'Defaults applied to every entry, as a JSON object (omitted → none). Unknown keys are rejected by /v1/bulk (422) and ignored by scrape bulk / fetch bulk; fetch bulk additionally rejects browser-only keys (
|
|
51
|
+
.option('--defaults <json>', 'Defaults applied to every entry, as a JSON object (omitted → none). Unknown keys are rejected by /v1/bulk (422) and ignored by scrape bulk / fetch bulk; fetch bulk additionally rejects browser-only keys (wait_until, wait_for_selector, delay_ms) with a 400. `cookies` and `headers` are no longer accepted by any capture endpoint and 400 (#3242).', (v) => parseJsonFlag('--defaults', v, 'object'))
|
|
52
52
|
.option('--metadata <json>', `Arbitrary metadata echoed on the job envelope, as a JSON object, e.g. '{"run":"7"}' (omitted → none)`, (v) => parseJsonFlag('--metadata', v, 'object'))
|
|
53
53
|
// /v1/scrape/bulk is the ONE bulk endpoint with a top-level `custom`
|
|
54
54
|
// (validators/scrape-bulk.ts:88). Per-entry `custom` arrives through --file lines instead.
|
|
@@ -115,8 +115,6 @@ export function makeScrapeCommand(getGlobal) {
|
|
|
115
115
|
.option('--markdown-mode <mode>', `Markdown extraction mode: ${MARKDOWN_MODES.join(', ')}`, (v) => parseEnum('--markdown-mode', MARKDOWN_MODES, v))
|
|
116
116
|
.option('--markdown-query <q>', 'BM25 query string for relevance-ranked markdown filtering (omitted → disabled)')
|
|
117
117
|
.option('--cache-ttl <ttl>', 'Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default)', (v) => parseCacheTtl('--cache-ttl', v))
|
|
118
|
-
.option('--cookies <json>', `Cookies as a JSON array, e.g. '[{"name":"a","value":"b"}]' (omitted → none)`, (v) => parseJsonFlag('--cookies', v, 'array'))
|
|
119
|
-
.option('--headers <json>', `Extra request headers as a JSON object, e.g. '{"X-A":"b"}' (omitted → none)`, (v) => parseJsonFlag('--headers', v, 'object'))
|
|
120
118
|
.option('--delay-ms <n>', 'Post-load delay in milliseconds before capture', (v) => parseIntFlag('--delay-ms', v))
|
|
121
119
|
.option('--events <csv>', `Webhook events (comma-separated): ${SCRAPE_EVENTS.join(', ')} (omitted → server default)`, (v) => parseCsv(v).map((e) => parseEnum('--events', SCRAPE_EVENTS, e)))
|
|
122
120
|
.option('--token-encoding <enc>', `Tokenizer for --token-budget: ${TOKEN_ENCODINGS.join(', ')} (omitted → server default)`, (v) => parseEnum('--token-encoding', TOKEN_ENCODINGS, v))
|
|
@@ -164,10 +162,6 @@ export function makeScrapeCommand(getGlobal) {
|
|
|
164
162
|
body['markdown_query'] = opts.markdownQuery;
|
|
165
163
|
if (opts.cacheTtl !== undefined)
|
|
166
164
|
body['cache_ttl'] = opts.cacheTtl;
|
|
167
|
-
if (opts.cookies !== undefined)
|
|
168
|
-
body['cookies'] = opts.cookies;
|
|
169
|
-
if (opts.headers !== undefined)
|
|
170
|
-
body['headers'] = opts.headers;
|
|
171
165
|
if (opts.delayMs !== undefined)
|
|
172
166
|
body['delay_ms'] = opts.delayMs;
|
|
173
167
|
if (opts.events !== undefined)
|
|
@@ -346,8 +346,6 @@ export function makeScreenshotsCommand(getGlobal) {
|
|
|
346
346
|
.option('--include-html', 'Include the rendered HTML alongside the screenshot')
|
|
347
347
|
.option('--country <cc>', 'Two-letter country code to route the request through (e.g. us, de)')
|
|
348
348
|
.option('--language <lang>', 'Accept-Language override (e.g. en-US)')
|
|
349
|
-
.option('--headers <json>', `Extra request headers as a JSON object, e.g. '{"X-A":"b"}' (omitted → none)`, (v) => parseJsonFlag('--headers', v, 'object'))
|
|
350
|
-
.option('--cookies <json>', `Cookies as a JSON array of {name,value,domain}, e.g. '[{"name":"a","value":"b","domain":"example.com"}]' (omitted → none)`, (v) => parseJsonFlag('--cookies', v, 'array'))
|
|
351
349
|
.option('--metadata <json>', `Arbitrary metadata echoed on the screenshot envelope, as a JSON object, e.g. '{"run":"7"}' (omitted → none)`, (v) => parseJsonFlag('--metadata', v, 'object'))
|
|
352
350
|
.option('--async', 'Queue the screenshot and return a job envelope immediately (no wait)')
|
|
353
351
|
.option('-o, --output <file>', 'Download the captured image to this file path (use - for stdout). Not valid with --json.')
|
|
@@ -394,12 +392,8 @@ export function makeScreenshotsCommand(getGlobal) {
|
|
|
394
392
|
body['country'] = opts.country;
|
|
395
393
|
if (opts.language !== undefined)
|
|
396
394
|
body['language'] = opts.language;
|
|
397
|
-
// The server defaults
|
|
398
|
-
//
|
|
399
|
-
if (opts.headers !== undefined)
|
|
400
|
-
body['headers'] = opts.headers;
|
|
401
|
-
if (opts.cookies !== undefined)
|
|
402
|
-
body['cookies'] = opts.cookies;
|
|
395
|
+
// The server defaults metadata to {}; writing an empty container on a bare
|
|
396
|
+
// create would materialize that default (#2480 E1).
|
|
403
397
|
if (opts.metadata !== undefined)
|
|
404
398
|
body['metadata'] = opts.metadata;
|
|
405
399
|
try {
|
|
@@ -31,7 +31,7 @@ export declare function parseNumberFlag(flag: string, value: string): number;
|
|
|
31
31
|
export declare function parseCacheTtl(flag: string, value: string): string;
|
|
32
32
|
/**
|
|
33
33
|
* Parse a JSON-object- or JSON-array-valued flag (#2545). The ONLY `JSON.parse` call site for CLI
|
|
34
|
-
* flag values — every JSON-shaped flag (`--
|
|
34
|
+
* flag values — every JSON-shaped flag (`--custom`, `--metadata`, `--defaults`,
|
|
35
35
|
* `--webhook-headers`, `--options`, `--scrape-options`) routes through here, so command modules
|
|
36
36
|
* carry no raw `JSON.parse` (grep-gated). Rejects malformed JSON AND a value whose top-level shape
|
|
37
37
|
* does not match `expect`, with a copy-pasteable example in the message (E5).
|
package/dist/lib/parse-flags.js
CHANGED
|
@@ -57,7 +57,7 @@ export function parseCacheTtl(flag, value) {
|
|
|
57
57
|
}
|
|
58
58
|
/**
|
|
59
59
|
* Parse a JSON-object- or JSON-array-valued flag (#2545). The ONLY `JSON.parse` call site for CLI
|
|
60
|
-
* flag values — every JSON-shaped flag (`--
|
|
60
|
+
* flag values — every JSON-shaped flag (`--custom`, `--metadata`, `--defaults`,
|
|
61
61
|
* `--webhook-headers`, `--options`, `--scrape-options`) routes through here, so command modules
|
|
62
62
|
* carry no raw `JSON.parse` (grep-gated). Rejects malformed JSON AND a value whose top-level shape
|
|
63
63
|
* does not match `expect`, with a copy-pasteable example in the message (E5).
|
package/dist/lib/url-lines.d.ts
CHANGED
|
@@ -30,12 +30,12 @@ export type BulkLine = string | Record<string, unknown>;
|
|
|
30
30
|
*
|
|
31
31
|
* The rejection message for a malformed line (#2596 cr-1 / E12) names the 1-based PHYSICAL line
|
|
32
32
|
* NUMBER (i.e. the line's own position in the file, counting blank lines) and the line's own KEY
|
|
33
|
-
* NAMES only — never the raw line or any value. A `--file` line is where per-item
|
|
34
|
-
* `
|
|
35
|
-
* leak session tokens and similar secrets to stderr, CI logs, and shell
|
|
36
|
-
* raw line is deliberately never echoed, the line number is the ONLY
|
|
37
|
-
* it must be computed BEFORE blank lines are filtered out, or it
|
|
38
|
-
* entry whenever the file contains blank lines above it (#2596 cr-2).
|
|
33
|
+
* NAMES only — never the raw line or any value. A `--file` line is where auth-bearing per-item
|
|
34
|
+
* `metadata` lives (#2596; per-item `cookies`/`headers` were removed in #3242), so echoing the
|
|
35
|
+
* raw line back would leak session tokens and similar secrets to stderr, CI logs, and shell
|
|
36
|
+
* transcripts. Because the raw line is deliberately never echoed, the line number is the ONLY
|
|
37
|
+
* locator left in the message — it must be computed BEFORE blank lines are filtered out, or it
|
|
38
|
+
* silently mis-locates the bad entry whenever the file contains blank lines above it (#2596 cr-2).
|
|
39
39
|
*/
|
|
40
40
|
export declare function parseBulkLines(file: string): BulkLine[];
|
|
41
41
|
/**
|
package/dist/lib/url-lines.js
CHANGED
|
@@ -24,12 +24,12 @@ import { InvalidArgumentError } from 'commander';
|
|
|
24
24
|
*
|
|
25
25
|
* The rejection message for a malformed line (#2596 cr-1 / E12) names the 1-based PHYSICAL line
|
|
26
26
|
* NUMBER (i.e. the line's own position in the file, counting blank lines) and the line's own KEY
|
|
27
|
-
* NAMES only — never the raw line or any value. A `--file` line is where per-item
|
|
28
|
-
* `
|
|
29
|
-
* leak session tokens and similar secrets to stderr, CI logs, and shell
|
|
30
|
-
* raw line is deliberately never echoed, the line number is the ONLY
|
|
31
|
-
* it must be computed BEFORE blank lines are filtered out, or it
|
|
32
|
-
* entry whenever the file contains blank lines above it (#2596 cr-2).
|
|
27
|
+
* NAMES only — never the raw line or any value. A `--file` line is where auth-bearing per-item
|
|
28
|
+
* `metadata` lives (#2596; per-item `cookies`/`headers` were removed in #3242), so echoing the
|
|
29
|
+
* raw line back would leak session tokens and similar secrets to stderr, CI logs, and shell
|
|
30
|
+
* transcripts. Because the raw line is deliberately never echoed, the line number is the ONLY
|
|
31
|
+
* locator left in the message — it must be computed BEFORE blank lines are filtered out, or it
|
|
32
|
+
* silently mis-locates the bad entry whenever the file contains blank lines above it (#2596 cr-2).
|
|
33
33
|
*/
|
|
34
34
|
export function parseBulkLines(file) {
|
|
35
35
|
return readFileSync(file, 'utf8')
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Build a scratch COPY of packages/cli (issue #3270), mirroring
|
|
3
|
+
* `packages/sdk-typescript/src/test-helpers/build-scratch.ts` (issue #2423 cr-2).
|
|
4
|
+
*
|
|
5
|
+
* `entrypoint.test.ts` and `queued-raw-flush.test.ts` both need a built `dist/index.js` and both
|
|
6
|
+
* used to rebuild the shared, workspace `packages/cli/dist` in place whenever it was missing —
|
|
7
|
+
* racing each other's build under vitest's default parallel file execution on a checkout with no
|
|
8
|
+
* prior `pnpm build` (one file's `existsSync` check passes while the other's build is still
|
|
9
|
+
* writing `dist/`).
|
|
10
|
+
*
|
|
11
|
+
* So instead: copy the tracked, on-disk source (never `dist/`, `tsconfig.tsbuildinfo`, or
|
|
12
|
+
* `node_modules` — those are the mutable/heavy bits) into a scratch tree that mirrors the real
|
|
13
|
+
* repo's relative layout (`<scratch>/tsconfig.base.json`, `<scratch>/packages/cli/**`), symlink in
|
|
14
|
+
* the real `node_modules` (read-only from the child process's point of view — nothing here writes
|
|
15
|
+
* back into it), and let the caller run the package's own `npm run build` / `npm pack` scripts
|
|
16
|
+
* against that copy. The real `packages/cli/dist` is never read and never written by either suite.
|
|
17
|
+
*
|
|
18
|
+
* @param pkgRoot packages/cli (the real, workspace-shared package root)
|
|
19
|
+
* @param repoRoot the monorepo root (for tsconfig.base.json, which the scratch tsconfig extends)
|
|
20
|
+
* @param scratchRoot a fresh tmp dir (e.g. from mkdtempSync) to build the scratch tree under
|
|
21
|
+
* @returns the scratch package dir (`<scratchRoot>/packages/cli`)
|
|
22
|
+
*/
|
|
23
|
+
export declare function createBuildScratch(pkgRoot: string, repoRoot: string, scratchRoot: string): string;
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
// Test-only fixture, compiled like any other `src` file (not excluded from the tsc build) — same
|
|
2
|
+
// tradeoff sdk-typescript's own copy of this helper documents: a small compiled fixture lands in
|
|
3
|
+
// `dist/test-helpers/`, but excluding it breaks ESLint's typed-lint project-service resolution.
|
|
4
|
+
import { mkdirSync, cpSync, copyFileSync, symlinkSync } from 'node:fs';
|
|
5
|
+
import { basename, join } from 'node:path';
|
|
6
|
+
/**
|
|
7
|
+
* Build a scratch COPY of packages/cli (issue #3270), mirroring
|
|
8
|
+
* `packages/sdk-typescript/src/test-helpers/build-scratch.ts` (issue #2423 cr-2).
|
|
9
|
+
*
|
|
10
|
+
* `entrypoint.test.ts` and `queued-raw-flush.test.ts` both need a built `dist/index.js` and both
|
|
11
|
+
* used to rebuild the shared, workspace `packages/cli/dist` in place whenever it was missing —
|
|
12
|
+
* racing each other's build under vitest's default parallel file execution on a checkout with no
|
|
13
|
+
* prior `pnpm build` (one file's `existsSync` check passes while the other's build is still
|
|
14
|
+
* writing `dist/`).
|
|
15
|
+
*
|
|
16
|
+
* So instead: copy the tracked, on-disk source (never `dist/`, `tsconfig.tsbuildinfo`, or
|
|
17
|
+
* `node_modules` — those are the mutable/heavy bits) into a scratch tree that mirrors the real
|
|
18
|
+
* repo's relative layout (`<scratch>/tsconfig.base.json`, `<scratch>/packages/cli/**`), symlink in
|
|
19
|
+
* the real `node_modules` (read-only from the child process's point of view — nothing here writes
|
|
20
|
+
* back into it), and let the caller run the package's own `npm run build` / `npm pack` scripts
|
|
21
|
+
* against that copy. The real `packages/cli/dist` is never read and never written by either suite.
|
|
22
|
+
*
|
|
23
|
+
* @param pkgRoot packages/cli (the real, workspace-shared package root)
|
|
24
|
+
* @param repoRoot the monorepo root (for tsconfig.base.json, which the scratch tsconfig extends)
|
|
25
|
+
* @param scratchRoot a fresh tmp dir (e.g. from mkdtempSync) to build the scratch tree under
|
|
26
|
+
* @returns the scratch package dir (`<scratchRoot>/packages/cli`)
|
|
27
|
+
*/
|
|
28
|
+
export function createBuildScratch(pkgRoot, repoRoot, scratchRoot) {
|
|
29
|
+
const pkgDest = join(scratchRoot, 'packages/cli');
|
|
30
|
+
mkdirSync(join(scratchRoot, 'packages'), { recursive: true });
|
|
31
|
+
copyFileSync(join(repoRoot, 'tsconfig.base.json'), join(scratchRoot, 'tsconfig.base.json'));
|
|
32
|
+
cpSync(pkgRoot, pkgDest, {
|
|
33
|
+
recursive: true,
|
|
34
|
+
filter: (src) => {
|
|
35
|
+
const name = basename(src);
|
|
36
|
+
return name !== 'node_modules' && name !== 'dist' && name !== 'tsconfig.tsbuildinfo';
|
|
37
|
+
},
|
|
38
|
+
});
|
|
39
|
+
symlinkSync(join(pkgRoot, 'node_modules'), join(pkgDest, 'node_modules'), 'dir');
|
|
40
|
+
return pkgDest;
|
|
41
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@hunt-labs/bytekit-cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.9.0",
|
|
4
4
|
"description": "ByteKit CLI — wraps @hunt-labs/bytekit-sdk as shell subcommands",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -41,7 +41,7 @@
|
|
|
41
41
|
],
|
|
42
42
|
"dependencies": {
|
|
43
43
|
"commander": "^12.0.0",
|
|
44
|
-
"@hunt-labs/bytekit-sdk": "^0.
|
|
44
|
+
"@hunt-labs/bytekit-sdk": "^0.9.0"
|
|
45
45
|
},
|
|
46
46
|
"scripts": {
|
|
47
47
|
"build": "rm -rf dist tsconfig.tsbuildinfo && tsc",
|