@hunt-labs/bytekit-sdk 0.8.0 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +62 -0
- package/dist/generated/api.d.ts +244 -127
- package/dist/index.d.ts +1 -1
- package/dist/resources/fetch.d.ts +8 -36
- package/dist/resources/fetch.js +3 -5
- package/dist/resources/scrape.d.ts +1 -13
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -10,6 +10,68 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
|
|
|
10
10
|
|
|
11
11
|
## [Unreleased]
|
|
12
12
|
|
|
13
|
+
## [0.9.1] - 2026-08-25
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
- The generated `paths` type for the `200` of `GET /v1/scrape/bulk/{id}` now declares `items`
|
|
18
|
+
(#3632). The route builds one body and varies only the status code, so the terminal `200`
|
|
19
|
+
has always carried the per-URL scrape envelopes — but only the `202` documented them, so
|
|
20
|
+
`scrape.bulk.get()` typed the results away at exactly the poll that has them. Types only:
|
|
21
|
+
the field was already on the wire, and server behavior is unchanged.
|
|
22
|
+
|
|
23
|
+
- The generated `paths` types now describe the `202` that `GET /v1/scrape/bulk/{id}`,
|
|
24
|
+
`GET /v1/fetch/bulk/{id}` and `GET /v1/bulk/{id}` already serve while a bulk job is
|
|
25
|
+
non-terminal (#3533). Each of the three returns 202 with the job envelope until the job is
|
|
26
|
+
terminal and 200 afterwards, but only the 200 was declared, so anything typed off the spec
|
|
27
|
+
had no shape for the response the very first poll receives. Types only: `scrape.bulk.get()`,
|
|
28
|
+
`fetch.bulk.get()` and `bulk.get()` already resolved the body on any 2xx, and server behavior
|
|
29
|
+
is unchanged.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- The generated `topup_balance_bytes` description on `GET /v1/usage` now states that the value is
|
|
34
|
+
the remaining purchased top-up bytes, already net of any overage consumed against them (#3459).
|
|
35
|
+
Types only: the field's name, type and wire format are unchanged. The server previously
|
|
36
|
+
subtracted the consumed overage a second time when computing this value, so the number a client
|
|
37
|
+
reads is larger than before for accounts that ran past their included allowance.
|
|
38
|
+
|
|
39
|
+
## [0.9.0] - 2026-08-20
|
|
40
|
+
|
|
41
|
+
### Removed
|
|
42
|
+
|
|
43
|
+
- **BREAKING: the `cookies` and `headers` request-body fields are gone from every capture
|
|
44
|
+
endpoint** (#3242) — `/v1/scrape`, `/v1/scrape/bulk`, `/v1/screenshots`, `/v1/recordings` and
|
|
45
|
+
`/v1/schema`. A request that carries either field is rejected with `400 validation_error`; it
|
|
46
|
+
is not ignored, and there is no deprecation window, feature flag, or compatibility header.
|
|
47
|
+
`ScrapeOpts.cookies`, `ScrapeOpts.headers` and the exported `ScrapeCookie` type are removed
|
|
48
|
+
from the SDK surface, so a call that still passes them no longer compiles.
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
// before — compiled, and the values were forwarded to the target site
|
|
52
|
+
await client.scrape.create({ url, cookies: [{ name: 'session', value: 'abc' }] });
|
|
53
|
+
// after — does not compile; the same body over raw HTTP is a 400 validation_error
|
|
54
|
+
await client.scrape.create({ url });
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
ByteKit continues to manage its own cookie jar and request headers internally; what was removed
|
|
58
|
+
is the caller's ability to inject or override them.
|
|
59
|
+
|
|
60
|
+
### Changed
|
|
61
|
+
|
|
62
|
+
- **The generated types now declare the `503` response of `GET`/`POST /v1/fetch`** (#3372).
|
|
63
|
+
`/v1/fetch` answers `503` on two conditions and both carry the ordinary `Error` envelope, so
|
|
64
|
+
a typed consumer reading `operations['getFetch']['responses']` / `['postFetch']` can handle it
|
|
65
|
+
instead of meeting an undeclared status at runtime. Read `error.code` to tell the two apart,
|
|
66
|
+
because they disagree about `error.http_status`:
|
|
67
|
+
- `upstream_error` — the target returned HTTP `502`, relayed at `503` because Cloudflare
|
|
68
|
+
replaces a clean origin 502 with its own error page. `error.http_status` stays `502`, the
|
|
69
|
+
**true upstream status**, and is the only surviving record of what the target answered.
|
|
70
|
+
- `internal_error` (`Fetch service unavailable.`) — the fetch service was unreachable. There
|
|
71
|
+
is no upstream, so `error.http_status` reports `503`, matching what was served.
|
|
72
|
+
|
|
73
|
+
No runtime behaviour changes, and every other documented `/v1/fetch` status is unaffected.
|
|
74
|
+
|
|
13
75
|
## [0.8.0] - 2026-08-19
|
|
14
76
|
|
|
15
77
|
### Removed
|
package/dist/generated/api.d.ts
CHANGED
|
@@ -272,8 +272,9 @@ export interface paths {
|
|
|
272
272
|
};
|
|
273
273
|
/**
|
|
274
274
|
* Fetch a URL (GET)
|
|
275
|
-
* @description Direct HTTP fetch.
|
|
276
|
-
*
|
|
275
|
+
* @description Direct HTTP fetch. Successful responses return the upstream response bytes verbatim;
|
|
276
|
+
* metadata is returned in X-Fetch-* headers. Non-2xx upstream responses use the
|
|
277
|
+
* `upstream_error` JSON envelope with the upstream HTTP status.
|
|
277
278
|
*/
|
|
278
279
|
get: operations["getFetch"];
|
|
279
280
|
put?: never;
|
|
@@ -303,7 +304,7 @@ export interface paths {
|
|
|
303
304
|
/**
|
|
304
305
|
* Bulk fetch URLs
|
|
305
306
|
* @description Enqueues multiple URLs for HTTP fetching. Results delivered via webhook.
|
|
306
|
-
* Does not support options that require page rendering (
|
|
307
|
+
* Does not support options that require page rendering (wait_for_selector, etc.).
|
|
307
308
|
*
|
|
308
309
|
* **Limits:** request bodies are guarded in a two-tier shape. A streamed body
|
|
309
310
|
* with no `Content-Length` header is capped at 2 MB before your API key
|
|
@@ -588,8 +589,8 @@ export interface paths {
|
|
|
588
589
|
* Schema-driven structured extraction
|
|
589
590
|
* @description Fetches `url` and extracts structured JSON conforming to the supplied
|
|
590
591
|
* JSON Schema (draft 2020-12) using an LLM. Fast-path only — slow-path
|
|
591
|
-
* rendering options (`wait_until=networkidle`, `
|
|
592
|
-
*
|
|
592
|
+
* rendering options (`wait_until=networkidle`, `delay_ms`,
|
|
593
|
+
* `wait_for_selector`) cause a `400 unsupported_url`.
|
|
593
594
|
*
|
|
594
595
|
* Dual-axis billing: a fixed per-request credit charge **plus** the
|
|
595
596
|
* underlying page's bandwidth, billed exactly like `/v1/scrape`. The
|
|
@@ -1041,7 +1042,7 @@ export interface paths {
|
|
|
1041
1042
|
/**
|
|
1042
1043
|
* Query scrape by URL (deprecated)
|
|
1043
1044
|
* @deprecated
|
|
1044
|
-
* @description Use GET /v1/fetch?url
|
|
1045
|
+
* @description Use GET /v1/fetch?url=... instead.
|
|
1045
1046
|
*/
|
|
1046
1047
|
get: operations["queryScrapeLegacy"];
|
|
1047
1048
|
put?: never;
|
|
@@ -1404,14 +1405,6 @@ export interface components {
|
|
|
1404
1405
|
details?: unknown;
|
|
1405
1406
|
};
|
|
1406
1407
|
};
|
|
1407
|
-
Cookie: {
|
|
1408
|
-
name: string;
|
|
1409
|
-
value: string;
|
|
1410
|
-
domain?: string;
|
|
1411
|
-
path?: string;
|
|
1412
|
-
secure?: boolean;
|
|
1413
|
-
httpOnly?: boolean;
|
|
1414
|
-
};
|
|
1415
1408
|
Viewport: {
|
|
1416
1409
|
width: number;
|
|
1417
1410
|
height: number;
|
|
@@ -1534,12 +1527,6 @@ export interface components {
|
|
|
1534
1527
|
formats: ("raw_html" | "html" | "markdown" | "links" | "images")[];
|
|
1535
1528
|
/** @default US */
|
|
1536
1529
|
country: string;
|
|
1537
|
-
/** @default [] */
|
|
1538
|
-
cookies: components["schemas"]["Cookie"][];
|
|
1539
|
-
/** @default {} */
|
|
1540
|
-
headers: {
|
|
1541
|
-
[key: string]: string;
|
|
1542
|
-
};
|
|
1543
1530
|
/** @default 0 */
|
|
1544
1531
|
delay_ms: number;
|
|
1545
1532
|
/** @default 30000 */
|
|
@@ -1914,8 +1901,6 @@ export interface components {
|
|
|
1914
1901
|
FetchRequest: {
|
|
1915
1902
|
/** Format: uri */
|
|
1916
1903
|
url: string;
|
|
1917
|
-
/** @enum {string} */
|
|
1918
|
-
format?: "markdown" | "html";
|
|
1919
1904
|
/** @default US */
|
|
1920
1905
|
country: string;
|
|
1921
1906
|
/** @default 60000 */
|
|
@@ -1932,37 +1917,6 @@ export interface components {
|
|
|
1932
1917
|
custom: {
|
|
1933
1918
|
[key: string]: unknown;
|
|
1934
1919
|
};
|
|
1935
|
-
/**
|
|
1936
|
-
* @description Markdown processing mode. Only used when format=markdown.
|
|
1937
|
-
* @enum {string}
|
|
1938
|
-
*/
|
|
1939
|
-
markdown_mode?: "article" | "raw" | "llm";
|
|
1940
|
-
/** @description BM25 query string for relevance-ranked filtering. Only used when format=markdown. */
|
|
1941
|
-
markdown_query?: string;
|
|
1942
|
-
/**
|
|
1943
|
-
* @description Link rendering style. Only used when format=markdown.
|
|
1944
|
-
* @enum {string}
|
|
1945
|
-
*/
|
|
1946
|
-
markdown_links?: "inline" | "references" | "none" | "text";
|
|
1947
|
-
/**
|
|
1948
|
-
* @description Image retention mode. Only used when format=markdown.
|
|
1949
|
-
* @enum {string}
|
|
1950
|
-
*/
|
|
1951
|
-
markdown_images?: "inline" | "references" | "none" | "text";
|
|
1952
|
-
/** @description Append a Links footer to the markdown output. Only used when format=markdown. */
|
|
1953
|
-
with_links_summary?: boolean;
|
|
1954
|
-
/** @description Append an Images footer to the markdown output. Only used when format=markdown. */
|
|
1955
|
-
with_images_summary?: boolean;
|
|
1956
|
-
/** @description Compact whitespace output. Only used when format=markdown. */
|
|
1957
|
-
markdown_compact?: boolean;
|
|
1958
|
-
/** @description Filter low-signal images. Only used when format=markdown. */
|
|
1959
|
-
markdown_filter_images?: boolean;
|
|
1960
|
-
/** @description When true, `formats.links` and `formats.images` return `ScrapeScoredLink[]` / `ScrapeScoredImage[]` (rich objects) instead of `string[]`, and a top-level `tables` array is included. Only used when format=markdown. */
|
|
1961
|
-
markdown_include_media?: boolean;
|
|
1962
|
-
/** @description When true, enables markdown-pipeline and tag-filter warnings for affected html and markdown requests. Artifact availability warnings such as `artifact_unavailable` may appear independently of this flag. */
|
|
1963
|
-
markdown_include_warnings?: boolean;
|
|
1964
|
-
/** @description When true, includes a top-level `stats` object with `ScrapeStats` (chars, tokens, blocks). Only used when format=markdown. */
|
|
1965
|
-
markdown_include_stats?: boolean;
|
|
1966
1920
|
};
|
|
1967
1921
|
ScreenshotRequest: {
|
|
1968
1922
|
/** Format: uri */
|
|
@@ -2000,12 +1954,6 @@ export interface components {
|
|
|
2000
1954
|
scroll: boolean;
|
|
2001
1955
|
/** @default null */
|
|
2002
1956
|
clip: components["schemas"]["Clip"] | null;
|
|
2003
|
-
/** @default {} */
|
|
2004
|
-
headers: {
|
|
2005
|
-
[key: string]: string;
|
|
2006
|
-
};
|
|
2007
|
-
/** @default [] */
|
|
2008
|
-
cookies: components["schemas"]["Cookie"][];
|
|
2009
1957
|
/** @default false */
|
|
2010
1958
|
dark_mode: boolean;
|
|
2011
1959
|
/**
|
|
@@ -2125,12 +2073,6 @@ export interface components {
|
|
|
2125
2073
|
/** @default 0 */
|
|
2126
2074
|
delay_ms: number;
|
|
2127
2075
|
scroll?: components["schemas"]["ScrollOptions"];
|
|
2128
|
-
/** @default {} */
|
|
2129
|
-
headers: {
|
|
2130
|
-
[key: string]: string;
|
|
2131
|
-
};
|
|
2132
|
-
/** @default [] */
|
|
2133
|
-
cookies: components["schemas"]["Cookie"][];
|
|
2134
2076
|
/** @default false */
|
|
2135
2077
|
dark_mode: boolean;
|
|
2136
2078
|
/** @default US */
|
|
@@ -2205,6 +2147,7 @@ export interface components {
|
|
|
2205
2147
|
* @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
|
|
2206
2148
|
*/
|
|
2207
2149
|
webhook_url: string;
|
|
2150
|
+
/** @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code. */
|
|
2208
2151
|
webhook_secret?: string;
|
|
2209
2152
|
/** @description Max 20 headers. Keys must match /^[A-Za-z0-9-]+$/ and not be reserved. */
|
|
2210
2153
|
webhook_headers?: {
|
|
@@ -2234,6 +2177,7 @@ export interface components {
|
|
|
2234
2177
|
* @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
|
|
2235
2178
|
*/
|
|
2236
2179
|
webhook_url?: string;
|
|
2180
|
+
/** @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code. */
|
|
2237
2181
|
webhook_secret?: string;
|
|
2238
2182
|
webhook_headers?: {
|
|
2239
2183
|
[key: string]: string;
|
|
@@ -2294,7 +2238,7 @@ export interface components {
|
|
|
2294
2238
|
};
|
|
2295
2239
|
MonitorCreateResponse: components["schemas"]["MonitorResponse"] & {
|
|
2296
2240
|
/**
|
|
2297
|
-
* @description The webhook signing secret — returned **once** at creation time only. Subsequent GET requests omit this field. Always present on a create response (`webhook_url` is required on create). Format: 64 lowercase hex characters (32 bytes of entropy).
|
|
2241
|
+
* @description The webhook signing secret — returned **once** at creation time only. Subsequent GET requests omit this field. Always present on a create response (`webhook_url` is required on create). Format: 64 lowercase hex characters (32 bytes of entropy). Use it to verify the `X-Webhook-Signature` header on each delivery — see https://bytekit.com/docs/guides/webhooks.
|
|
2298
2242
|
* @example a3f1b2c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2
|
|
2299
2243
|
*/
|
|
2300
2244
|
webhook_secret?: string;
|
|
@@ -2343,7 +2287,10 @@ export interface components {
|
|
|
2343
2287
|
* @description Optional webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported). When omitted, no `sitemap.completed`/`sitemap.failed` webhook is delivered and the job is poll-only via `GET /v1/sitemap/{id}`. When present, delivery is unchanged. `webhook_secret` is ignored when `webhook_url` is absent.
|
|
2344
2288
|
*/
|
|
2345
2289
|
webhook_url?: string;
|
|
2346
|
-
/**
|
|
2290
|
+
/**
|
|
2291
|
+
* @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code.
|
|
2292
|
+
* @default null
|
|
2293
|
+
*/
|
|
2347
2294
|
webhook_secret: string | null;
|
|
2348
2295
|
/**
|
|
2349
2296
|
* @description Max 90 days.
|
|
@@ -2580,7 +2527,10 @@ export interface components {
|
|
|
2580
2527
|
};
|
|
2581
2528
|
/** Format: date-time */
|
|
2582
2529
|
cancelled_at: string | null;
|
|
2583
|
-
/**
|
|
2530
|
+
/**
|
|
2531
|
+
* Format: date-time
|
|
2532
|
+
* @description Reserved, and always `null` today: it reflects the account's cancellation grace deadline, which no current code path sets. It is not the payment-failure grace period — that deadline is tracked per invoice (7 days from the failed payment) in `billing_dunning_graces` and is not exposed on this response.
|
|
2533
|
+
*/
|
|
2584
2534
|
grace_period_expires_at: string | null;
|
|
2585
2535
|
/** Format: date-time */
|
|
2586
2536
|
created_at: string;
|
|
@@ -2665,6 +2615,7 @@ export interface components {
|
|
|
2665
2615
|
purchased_topup_bytes: number;
|
|
2666
2616
|
/** @description Bytes remaining in starter-grant lots, after active refund reservations. Purchased top-up bytes are excluded. */
|
|
2667
2617
|
granted_topup_bytes: number;
|
|
2618
|
+
/** @description Unspent subscription allowance carried over from earlier billing periods: bytes remaining in allowance lots granted before the current period started, after active refund reservations. Carried value is spent before the current period's allowance, so this falls to `0` before `limit_bytes` reaches the current allowance alone. */
|
|
2668
2619
|
rollover_bytes: number;
|
|
2669
2620
|
};
|
|
2670
2621
|
CreditBalance: {
|
|
@@ -2677,6 +2628,7 @@ export interface components {
|
|
|
2677
2628
|
purchased_topup_credits: number;
|
|
2678
2629
|
/** @description Credits remaining in starter-grant lots, after active refund reservations. Purchased top-up credits are excluded. */
|
|
2679
2630
|
granted_topup_credits: number;
|
|
2631
|
+
/** @description Always `0`. Search credits do not carry over between billing periods; the field is retained for contract stability and will not become non-zero without an announced plan change. */
|
|
2680
2632
|
rollover_credits: number;
|
|
2681
2633
|
};
|
|
2682
2634
|
};
|
|
@@ -2699,7 +2651,6 @@ export interface components {
|
|
|
2699
2651
|
*/
|
|
2700
2652
|
ClientSurfaceMarker: string;
|
|
2701
2653
|
FetchUrl: string;
|
|
2702
|
-
FetchFormat: "markdown" | "html";
|
|
2703
2654
|
FetchCountry: string;
|
|
2704
2655
|
FetchTimeoutMs: number;
|
|
2705
2656
|
/** @description How long a freshly fetched URL may be served from cache. `0` skips the cache read; cache-eligible fresh results are still written with the default 7-day TTL so later non-zero-TTL callers can HIT. Safety-classified WAF husks remain live-only and are not written. Accepts `Nh` (hours, up to 168h), `Nd` (days, up to 7d), or the integer `0`. Examples: `0`, `48h`, `2d`. Invalid values receive a 422 validation_error with an actionable message. */
|
|
@@ -2739,11 +2690,9 @@ export interface components {
|
|
|
2739
2690
|
XFetchURL: string;
|
|
2740
2691
|
/**
|
|
2741
2692
|
* @description Byte length of the response body served on this request (integer as string) — NOT the
|
|
2742
|
-
* billing basis.
|
|
2743
|
-
*
|
|
2744
|
-
* `
|
|
2745
|
-
* count and is typically far smaller than what is billed. Billing always reads the upstream
|
|
2746
|
-
* compressed wire bytes from `X-Raw-Bytes` / `X-Billable-Bytes`, on every path.
|
|
2693
|
+
* billing basis. The body is always the upstream response bytes, so this equals the upstream wire byte
|
|
2694
|
+
* count. Billing always reads the upstream compressed wire bytes from `X-Raw-Bytes` /
|
|
2695
|
+
* `X-Billable-Bytes`, on every path.
|
|
2747
2696
|
*/
|
|
2748
2697
|
XFetchContentLength: string;
|
|
2749
2698
|
XFetchCreditsCharged: string;
|
|
@@ -3625,10 +3574,6 @@ export interface operations {
|
|
|
3625
3574
|
url: string;
|
|
3626
3575
|
formats?: ("raw_html" | "html" | "markdown" | "links" | "images")[];
|
|
3627
3576
|
country?: string;
|
|
3628
|
-
cookies?: components["schemas"]["Cookie"][];
|
|
3629
|
-
headers?: {
|
|
3630
|
-
[key: string]: string;
|
|
3631
|
-
};
|
|
3632
3577
|
delay_ms?: number;
|
|
3633
3578
|
timeout_ms?: number;
|
|
3634
3579
|
metadata?: components["schemas"]["Metadata"];
|
|
@@ -3663,10 +3608,6 @@ export interface operations {
|
|
|
3663
3608
|
formats?: ("raw_html" | "html" | "markdown" | "links" | "images")[];
|
|
3664
3609
|
/** @default US */
|
|
3665
3610
|
country?: string;
|
|
3666
|
-
cookies?: components["schemas"]["Cookie"][];
|
|
3667
|
-
headers?: {
|
|
3668
|
-
[key: string]: string;
|
|
3669
|
-
};
|
|
3670
3611
|
/** @default 0 */
|
|
3671
3612
|
delay_ms?: number;
|
|
3672
3613
|
/** @default 30000 */
|
|
@@ -3688,6 +3629,7 @@ export interface operations {
|
|
|
3688
3629
|
* @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
|
|
3689
3630
|
*/
|
|
3690
3631
|
webhook_url: string;
|
|
3632
|
+
/** @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code. */
|
|
3691
3633
|
webhook_secret?: string;
|
|
3692
3634
|
metadata?: components["schemas"]["Metadata"];
|
|
3693
3635
|
/**
|
|
@@ -3726,7 +3668,7 @@ export interface operations {
|
|
|
3726
3668
|
completed_at: string | null;
|
|
3727
3669
|
/** @description Always null on the create response — the job has not been billed yet (routes/scrape-bulk.ts builds the 202 body from a partial job row). Fractional when populated (credits are NUMERIC(10,2)); the numeric branch is unreachable on this path and exists only so the 202 and 200 bodies share one shape. */
|
|
3728
3670
|
total_credits_charged: number | null;
|
|
3729
|
-
/** @description Estimated credits at submission time. Null in GET responses.
|
|
3671
|
+
/** @description Estimated credits at submission time. Null in GET responses. Fetch-bulk items use the raw fetch bandwidth billing factor. */
|
|
3730
3672
|
estimated_credits: number | null;
|
|
3731
3673
|
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
3732
3674
|
webhook_url: string;
|
|
@@ -3808,6 +3750,43 @@ export interface operations {
|
|
|
3808
3750
|
estimated_credits: number | null;
|
|
3809
3751
|
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
3810
3752
|
webhook_url: string;
|
|
3753
|
+
/** @description Rehydrated per-URL scrape envelopes — the terminal page of results (default 100 rows, `?limit=` up to 500). `routes/scrape-bulk.ts` spreads this key into the SAME body on both status branches, so it is present here exactly as it is on the 202; on the 200 the entries carry their final success or error envelopes (issue #3632). */
|
|
3754
|
+
items: components["schemas"]["ScrapeEnvelope"][];
|
|
3755
|
+
};
|
|
3756
|
+
};
|
|
3757
|
+
};
|
|
3758
|
+
/**
|
|
3759
|
+
* @description Bulk job still processing. `routes/scrape-bulk.ts` ends the poll handler
|
|
3760
|
+
* with `status: isTerminal ? 200 : 202` over an identical body, so this is
|
|
3761
|
+
* the SAME envelope the 200 documents — only the status code and the
|
|
3762
|
+
* reachable `status` values differ. Keep polling until you get a 200
|
|
3763
|
+
* (issue #3533).
|
|
3764
|
+
*/
|
|
3765
|
+
202: {
|
|
3766
|
+
headers: {
|
|
3767
|
+
[name: string]: unknown;
|
|
3768
|
+
};
|
|
3769
|
+
content: {
|
|
3770
|
+
"application/json": {
|
|
3771
|
+
/** @example bulk_01j9abc123 */
|
|
3772
|
+
id: string;
|
|
3773
|
+
/** @enum {string} */
|
|
3774
|
+
status: "processing";
|
|
3775
|
+
total: number;
|
|
3776
|
+
completed: number;
|
|
3777
|
+
failed: number;
|
|
3778
|
+
/** Format: date-time */
|
|
3779
|
+
created_at: string;
|
|
3780
|
+
/** Format: date-time */
|
|
3781
|
+
completed_at: string | null;
|
|
3782
|
+
/** @description Total credits billed for the job. Null until the job is billed. Fractional — credits are NUMERIC(10,2). */
|
|
3783
|
+
total_credits_charged: number | null;
|
|
3784
|
+
/** @description Null in GET responses (computed at submission time only). */
|
|
3785
|
+
estimated_credits: number | null;
|
|
3786
|
+
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
3787
|
+
webhook_url: string;
|
|
3788
|
+
/** @description Rehydrated per-URL scrape envelopes — the same page the 200 returns (default 100 rows, `?limit=` up to 500). While the job is non-terminal most entries carry the `queued` envelope. */
|
|
3789
|
+
items: components["schemas"]["ScrapeEnvelope"][];
|
|
3811
3790
|
};
|
|
3812
3791
|
};
|
|
3813
3792
|
};
|
|
@@ -3826,7 +3805,6 @@ export interface operations {
|
|
|
3826
3805
|
parameters: {
|
|
3827
3806
|
query: {
|
|
3828
3807
|
url: components["parameters"]["FetchUrl"];
|
|
3829
|
-
format?: components["parameters"]["FetchFormat"];
|
|
3830
3808
|
country?: components["parameters"]["FetchCountry"];
|
|
3831
3809
|
timeout_ms?: components["parameters"]["FetchTimeoutMs"];
|
|
3832
3810
|
/** @description How long a freshly fetched URL may be served from cache. `0` skips the cache read; cache-eligible fresh results are still written with the default 7-day TTL so later non-zero-TTL callers can HIT. Safety-classified WAF husks remain live-only and are not written. Accepts `Nh` (hours, up to 168h), `Nd` (days, up to 7d), or the integer `0`. Examples: `0`, `48h`, `2d`. Invalid values receive a 422 validation_error with an actionable message. */
|
|
@@ -3860,16 +3838,14 @@ export interface operations {
|
|
|
3860
3838
|
"X-Raw-Bytes"?: number;
|
|
3861
3839
|
/** @description Chargeable bytes after applying the billing multiplier. Equals round(X-Raw-Bytes × X-Billing-Multiplier). */
|
|
3862
3840
|
"X-Billable-Bytes"?: number;
|
|
3863
|
-
/** @description Bandwidth cost multiplier applied to X-Raw-Bytes.
|
|
3841
|
+
/** @description Bandwidth cost multiplier applied to X-Raw-Bytes. Single `/v1/fetch` returns the fetched bytes as the body, so it carries no `billing` node — this header trio is the ONLY billing surface here. Items under `GET /v1/fetch/bulk/{id}` do carry the node. */
|
|
3864
3842
|
"X-Billing-Multiplier"?: number;
|
|
3865
3843
|
/** @description Back-compat duplicate of `X-Billing-Multiplier` — identical value on every path. Prefer `X-Billing-Multiplier`; this alias is retained for clients that read the `X-Fetch-*` header family. */
|
|
3866
3844
|
"X-Fetch-Billing-Multiplier"?: number;
|
|
3867
3845
|
[name: string]: unknown;
|
|
3868
3846
|
};
|
|
3869
3847
|
content: {
|
|
3870
|
-
"
|
|
3871
|
-
"text/markdown": string;
|
|
3872
|
-
"text/html": string;
|
|
3848
|
+
"text/plain": string;
|
|
3873
3849
|
};
|
|
3874
3850
|
};
|
|
3875
3851
|
/** @description Invalid URL (`invalid_url`) or oversized request. */
|
|
@@ -3881,7 +3857,7 @@ export interface operations {
|
|
|
3881
3857
|
"application/json": components["schemas"]["Error"];
|
|
3882
3858
|
};
|
|
3883
3859
|
};
|
|
3884
|
-
/** @description Validation error (e.g.
|
|
3860
|
+
/** @description Validation error (e.g. slow-path-only field). */
|
|
3885
3861
|
422: {
|
|
3886
3862
|
headers: {
|
|
3887
3863
|
[name: string]: unknown;
|
|
@@ -3900,10 +3876,14 @@ export interface operations {
|
|
|
3900
3876
|
};
|
|
3901
3877
|
};
|
|
3902
3878
|
/**
|
|
3903
|
-
* @description
|
|
3904
|
-
*
|
|
3905
|
-
*
|
|
3906
|
-
*
|
|
3879
|
+
* @description Content transform failed, or a non-2xx upstream response relayed as the unified
|
|
3880
|
+
* Error envelope. On an upstream relay the HTTP status equals the upstream status
|
|
3881
|
+
* (code `upstream_error`), so any 4xx/5xx the target returns surfaces here as an
|
|
3882
|
+
* Error envelope rather than a raw body (#1462).
|
|
3883
|
+
*
|
|
3884
|
+
* **One divergence.** An upstream `502` is relayed at HTTP `503`, not `502` — see
|
|
3885
|
+
* the `503` response below. A fetch service that cannot be reached also answers
|
|
3886
|
+
* `503`, not `502` (#3373); this status no longer covers that condition.
|
|
3907
3887
|
*/
|
|
3908
3888
|
502: {
|
|
3909
3889
|
headers: {
|
|
@@ -3913,6 +3893,30 @@ export interface operations {
|
|
|
3913
3893
|
"application/json": components["schemas"]["Error"];
|
|
3914
3894
|
};
|
|
3915
3895
|
};
|
|
3896
|
+
/**
|
|
3897
|
+
* @description Two distinct conditions answer `503`, and they disagree about what the body's
|
|
3898
|
+
* `error.http_status` reports. Read `error.code` to tell them apart.
|
|
3899
|
+
*
|
|
3900
|
+
* - **`upstream_error`** — the target returned HTTP `502`, relayed here as `503`.
|
|
3901
|
+
* Cloudflare fronts this API and replaces a clean origin 502 with its own generic
|
|
3902
|
+
* `text/plain` error page, discarding the envelope before it reaches you, while a
|
|
3903
|
+
* 503 is passed through unchanged. The body's `error.http_status` still reports
|
|
3904
|
+
* the **true upstream status**, `502` — not `503` — and is the only surviving
|
|
3905
|
+
* record of what the target answered, so the body rather than the HTTP status is
|
|
3906
|
+
* authoritative here (#3355).
|
|
3907
|
+
* - **`internal_error`** (message `Fetch service unavailable.`) — the fetch service
|
|
3908
|
+
* could not be reached at all. There is no upstream status to preserve: the `503`
|
|
3909
|
+
* is this API's own choice of wire status, so the body's `error.http_status`
|
|
3910
|
+
* reports `503`, mirroring what was served (#3373).
|
|
3911
|
+
*/
|
|
3912
|
+
503: {
|
|
3913
|
+
headers: {
|
|
3914
|
+
[name: string]: unknown;
|
|
3915
|
+
};
|
|
3916
|
+
content: {
|
|
3917
|
+
"application/json": components["schemas"]["Error"];
|
|
3918
|
+
};
|
|
3919
|
+
};
|
|
3916
3920
|
};
|
|
3917
3921
|
};
|
|
3918
3922
|
postFetch: {
|
|
@@ -3926,10 +3930,7 @@ export interface operations {
|
|
|
3926
3930
|
content: {
|
|
3927
3931
|
/**
|
|
3928
3932
|
* @example {
|
|
3929
|
-
* "url": "https://example.com"
|
|
3930
|
-
* "format": "markdown",
|
|
3931
|
-
* "markdown_mode": "article",
|
|
3932
|
-
* "markdown_links": "inline"
|
|
3933
|
+
* "url": "https://example.com"
|
|
3933
3934
|
* }
|
|
3934
3935
|
*/
|
|
3935
3936
|
"application/json": components["schemas"]["FetchRequest"];
|
|
@@ -3958,16 +3959,14 @@ export interface operations {
|
|
|
3958
3959
|
"X-Raw-Bytes"?: number;
|
|
3959
3960
|
/** @description Chargeable bytes after applying the billing multiplier. Equals round(X-Raw-Bytes × X-Billing-Multiplier). */
|
|
3960
3961
|
"X-Billable-Bytes"?: number;
|
|
3961
|
-
/** @description Bandwidth cost multiplier applied to X-Raw-Bytes.
|
|
3962
|
+
/** @description Bandwidth cost multiplier applied to X-Raw-Bytes. Single `/v1/fetch` returns the fetched bytes as the body, so it carries no `billing` node — this header trio is the ONLY billing surface here. Items under `GET /v1/fetch/bulk/{id}` do carry the node. */
|
|
3962
3963
|
"X-Billing-Multiplier"?: number;
|
|
3963
3964
|
/** @description Back-compat duplicate of `X-Billing-Multiplier` — identical value on every path. Prefer `X-Billing-Multiplier`; this alias is retained for clients that read the `X-Fetch-*` header family. */
|
|
3964
3965
|
"X-Fetch-Billing-Multiplier"?: number;
|
|
3965
3966
|
[name: string]: unknown;
|
|
3966
3967
|
};
|
|
3967
3968
|
content: {
|
|
3968
|
-
"
|
|
3969
|
-
"text/markdown": string;
|
|
3970
|
-
"text/html": string;
|
|
3969
|
+
"text/plain": string;
|
|
3971
3970
|
};
|
|
3972
3971
|
};
|
|
3973
3972
|
/** @description Invalid URL (`invalid_url`) or oversized request body. */
|
|
@@ -3979,7 +3978,7 @@ export interface operations {
|
|
|
3979
3978
|
"application/json": components["schemas"]["Error"];
|
|
3980
3979
|
};
|
|
3981
3980
|
};
|
|
3982
|
-
/** @description Validation error (e.g. malformed JSON, unsupported
|
|
3981
|
+
/** @description Validation error (e.g. malformed JSON, unsupported fetch format, or slow-path-only field). */
|
|
3983
3982
|
422: {
|
|
3984
3983
|
headers: {
|
|
3985
3984
|
[name: string]: unknown;
|
|
@@ -3998,10 +3997,14 @@ export interface operations {
|
|
|
3998
3997
|
};
|
|
3999
3998
|
};
|
|
4000
3999
|
/**
|
|
4001
|
-
* @description
|
|
4002
|
-
*
|
|
4003
|
-
*
|
|
4004
|
-
*
|
|
4000
|
+
* @description Content transform failed, or a non-2xx upstream response relayed as the unified
|
|
4001
|
+
* Error envelope. On an upstream relay the HTTP status equals the upstream status
|
|
4002
|
+
* (code `upstream_error`), so any 4xx/5xx the target returns surfaces here as an
|
|
4003
|
+
* Error envelope rather than a raw body (#1462).
|
|
4004
|
+
*
|
|
4005
|
+
* **One divergence.** An upstream `502` is relayed at HTTP `503`, not `502` — see
|
|
4006
|
+
* the `503` response below. A fetch service that cannot be reached also answers
|
|
4007
|
+
* `503`, not `502` (#3373); this status no longer covers that condition.
|
|
4005
4008
|
*/
|
|
4006
4009
|
502: {
|
|
4007
4010
|
headers: {
|
|
@@ -4011,6 +4014,30 @@ export interface operations {
|
|
|
4011
4014
|
"application/json": components["schemas"]["Error"];
|
|
4012
4015
|
};
|
|
4013
4016
|
};
|
|
4017
|
+
/**
|
|
4018
|
+
* @description Two distinct conditions answer `503`, and they disagree about what the body's
|
|
4019
|
+
* `error.http_status` reports. Read `error.code` to tell them apart.
|
|
4020
|
+
*
|
|
4021
|
+
* - **`upstream_error`** — the target returned HTTP `502`, relayed here as `503`.
|
|
4022
|
+
* Cloudflare fronts this API and replaces a clean origin 502 with its own generic
|
|
4023
|
+
* `text/plain` error page, discarding the envelope before it reaches you, while a
|
|
4024
|
+
* 503 is passed through unchanged. The body's `error.http_status` still reports
|
|
4025
|
+
* the **true upstream status**, `502` — not `503` — and is the only surviving
|
|
4026
|
+
* record of what the target answered, so the body rather than the HTTP status is
|
|
4027
|
+
* authoritative here (#3355).
|
|
4028
|
+
* - **`internal_error`** (message `Fetch service unavailable.`) — the fetch service
|
|
4029
|
+
* could not be reached at all. There is no upstream status to preserve: the `503`
|
|
4030
|
+
* is this API's own choice of wire status, so the body's `error.http_status`
|
|
4031
|
+
* reports `503`, mirroring what was served (#3373).
|
|
4032
|
+
*/
|
|
4033
|
+
503: {
|
|
4034
|
+
headers: {
|
|
4035
|
+
[name: string]: unknown;
|
|
4036
|
+
};
|
|
4037
|
+
content: {
|
|
4038
|
+
"application/json": components["schemas"]["Error"];
|
|
4039
|
+
};
|
|
4040
|
+
};
|
|
4014
4041
|
};
|
|
4015
4042
|
};
|
|
4016
4043
|
createFetchBulk: {
|
|
@@ -4026,14 +4053,10 @@ export interface operations {
|
|
|
4026
4053
|
urls: (string | {
|
|
4027
4054
|
/** Format: uri */
|
|
4028
4055
|
url: string;
|
|
4029
|
-
/** @enum {string} */
|
|
4030
|
-
format?: "markdown" | "html";
|
|
4031
4056
|
country?: string;
|
|
4032
4057
|
cache_ttl?: string;
|
|
4033
4058
|
})[];
|
|
4034
4059
|
defaults?: {
|
|
4035
|
-
/** @enum {string} */
|
|
4036
|
-
format?: "markdown" | "html";
|
|
4037
4060
|
/** @default US */
|
|
4038
4061
|
country?: string;
|
|
4039
4062
|
/** @default 48h */
|
|
@@ -4044,6 +4067,7 @@ export interface operations {
|
|
|
4044
4067
|
* @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
|
|
4045
4068
|
*/
|
|
4046
4069
|
webhook_url: string;
|
|
4070
|
+
/** @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code. */
|
|
4047
4071
|
webhook_secret?: string;
|
|
4048
4072
|
metadata?: components["schemas"]["Metadata"];
|
|
4049
4073
|
};
|
|
@@ -4070,7 +4094,7 @@ export interface operations {
|
|
|
4070
4094
|
completed_at: string | null;
|
|
4071
4095
|
/** @description Always null on the create response — the job has not been billed yet (routes/fetch-bulk.ts builds the 202 body from a partial job row). Fractional when populated (credits are NUMERIC(10,2)); the numeric branch is unreachable on this path and exists only so the 202 and 200 bodies share one shape. */
|
|
4072
4096
|
total_credits_charged: number | null;
|
|
4073
|
-
/** @description Estimated credits at submission time. Null in GET responses.
|
|
4097
|
+
/** @description Estimated credits at submission time. Null in GET responses. Fetch items are raw passthrough requests and use the raw fetch bandwidth billing factor. */
|
|
4074
4098
|
estimated_credits: number | null;
|
|
4075
4099
|
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
4076
4100
|
webhook_url: string;
|
|
@@ -4179,6 +4203,72 @@ export interface operations {
|
|
|
4179
4203
|
};
|
|
4180
4204
|
};
|
|
4181
4205
|
};
|
|
4206
|
+
/**
|
|
4207
|
+
* @description Bulk job still processing. `routes/fetch-bulk.ts` ends the poll handler
|
|
4208
|
+
* with `status: isTerminal ? 200 : 202` over an identical body, so this is
|
|
4209
|
+
* the SAME envelope the 200 documents — only the status code and the
|
|
4210
|
+
* reachable `status` values differ. Keep polling until you get a 200; a
|
|
4211
|
+
* **cancelled** job stays on this branch forever (issue #3533).
|
|
4212
|
+
*
|
|
4213
|
+
* Item `content_url` values are pre-signed exactly as on the 200 — a fresh
|
|
4214
|
+
* 600-second signature on every poll.
|
|
4215
|
+
*/
|
|
4216
|
+
202: {
|
|
4217
|
+
headers: {
|
|
4218
|
+
[name: string]: unknown;
|
|
4219
|
+
};
|
|
4220
|
+
content: {
|
|
4221
|
+
"application/json": {
|
|
4222
|
+
/** @example bulk_01j9abc123 */
|
|
4223
|
+
id: string;
|
|
4224
|
+
/** @enum {string} */
|
|
4225
|
+
status: "processing" | "cancelled";
|
|
4226
|
+
total: number;
|
|
4227
|
+
completed: number;
|
|
4228
|
+
failed: number;
|
|
4229
|
+
/** Format: date-time */
|
|
4230
|
+
created_at: string;
|
|
4231
|
+
/** Format: date-time */
|
|
4232
|
+
completed_at: string | null;
|
|
4233
|
+
/** @description Total credits billed for the job. Null until the job is billed. Fractional — credits are NUMERIC(10,2). */
|
|
4234
|
+
total_credits_charged: number | null;
|
|
4235
|
+
/** @description Null in GET responses (computed at submission time only). */
|
|
4236
|
+
estimated_credits: number | null;
|
|
4237
|
+
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
4238
|
+
webhook_url: string;
|
|
4239
|
+
/** @description Per-URL fetch items, oldest first, capped at 500 (listBulkFetchItemsWithLatestAttempt). */
|
|
4240
|
+
items: {
|
|
4241
|
+
/**
|
|
4242
|
+
* @description Scrape-row id, `sc_`-prefixed. Fetch-bulk items are rows of the same resource sync `/v1/scrape` returns, so they share its prefix. Not to be confused with the `ft_`-prefixed `X-Fetch-ID` correlation header on sync `/v1/fetch`, which identifies a request rather than a stored item.
|
|
4243
|
+
* @example sc_01j9abc123
|
|
4244
|
+
*/
|
|
4245
|
+
id: string;
|
|
4246
|
+
/** Format: uri */
|
|
4247
|
+
url: string;
|
|
4248
|
+
/** @enum {string} */
|
|
4249
|
+
status: "pending" | "processing" | "completed" | "failed" | "cancelled";
|
|
4250
|
+
content_type: string | null;
|
|
4251
|
+
content_length: number | null;
|
|
4252
|
+
/** @description Pre-signed download URL for the fetched body, valid for 600 seconds (10 minutes) from the moment this response was generated — matching the signing TTL of `image_url` on `GET /v1/bulk/{id}/screenshots`. A new signature is minted on every poll. Null for items with no stored body; a completed item whose object cannot be signed falls back to the unsigned stored URL, which returns 401 on download. */
|
|
4253
|
+
content_url: string | null;
|
|
4254
|
+
waf_vendor: string | null;
|
|
4255
|
+
cache_hit: boolean;
|
|
4256
|
+
/**
|
|
4257
|
+
* @description Itemised account of this item's charge — the basis, the named factors
|
|
4258
|
+
* that composed the multiplier, and the resulting billed bytes. Omitted
|
|
4259
|
+
* for rows that completed before the billing node was persisted.
|
|
4260
|
+
*/
|
|
4261
|
+
billing?: components["schemas"]["BillingNode"];
|
|
4262
|
+
error_code: string | null;
|
|
4263
|
+
error_message: string | null;
|
|
4264
|
+
/** Format: date-time */
|
|
4265
|
+
created_at: string;
|
|
4266
|
+
/** Format: date-time */
|
|
4267
|
+
completed_at: string | null;
|
|
4268
|
+
}[];
|
|
4269
|
+
};
|
|
4270
|
+
};
|
|
4271
|
+
};
|
|
4182
4272
|
/** @description Resource not found. */
|
|
4183
4273
|
404: {
|
|
4184
4274
|
headers: {
|
|
@@ -4401,6 +4491,7 @@ export interface operations {
|
|
|
4401
4491
|
* @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
|
|
4402
4492
|
*/
|
|
4403
4493
|
webhook_url: string;
|
|
4494
|
+
/** @description HMAC secret used to sign webhook deliveries. Each delivery carries `X-Webhook-Timestamp` and `X-Webhook-Signature` (`sha256=` + the hex HMAC-SHA256 of `timestamp.body`). See https://bytekit.com/docs/guides/webhooks for verification code. */
|
|
4404
4495
|
webhook_secret?: string;
|
|
4405
4496
|
metadata?: components["schemas"]["Metadata"];
|
|
4406
4497
|
};
|
|
@@ -4427,7 +4518,7 @@ export interface operations {
|
|
|
4427
4518
|
completed_at: string | null;
|
|
4428
4519
|
/** @description Always null on the create response — the job has not been billed yet (routes/bulk.ts builds the 202 body from a partial job row). Fractional when populated (credits are NUMERIC(10,2)); the numeric branch is unreachable on this path and exists only so the 202 and 200 bodies share one shape. */
|
|
4429
4520
|
total_credits_charged: number | null;
|
|
4430
|
-
/** @description Estimated credits at submission time. Null in GET responses.
|
|
4521
|
+
/** @description Estimated credits at submission time. Null in GET responses. Fetch-bulk items use the raw fetch bandwidth billing factor. */
|
|
4431
4522
|
estimated_credits: number | null;
|
|
4432
4523
|
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
4433
4524
|
webhook_url: string;
|
|
@@ -4516,6 +4607,38 @@ export interface operations {
|
|
|
4516
4607
|
};
|
|
4517
4608
|
};
|
|
4518
4609
|
};
|
|
4610
|
+
/**
|
|
4611
|
+
* @description Bulk job still processing. `routes/bulk.ts` ends the poll handler with
|
|
4612
|
+
* `status: isTerminal ? 200 : 202` over an identical body, so this is the
|
|
4613
|
+
* SAME envelope the 200 documents — only the status code and the reachable
|
|
4614
|
+
* `status` values differ. Keep polling until you get a 200 (issue #3533).
|
|
4615
|
+
*/
|
|
4616
|
+
202: {
|
|
4617
|
+
headers: {
|
|
4618
|
+
[name: string]: unknown;
|
|
4619
|
+
};
|
|
4620
|
+
content: {
|
|
4621
|
+
"application/json": {
|
|
4622
|
+
/** @example bulk_01j9abc123 */
|
|
4623
|
+
id: string;
|
|
4624
|
+
/** @enum {string} */
|
|
4625
|
+
status: "processing";
|
|
4626
|
+
total: number;
|
|
4627
|
+
completed: number;
|
|
4628
|
+
failed: number;
|
|
4629
|
+
/** Format: date-time */
|
|
4630
|
+
created_at: string;
|
|
4631
|
+
/** Format: date-time */
|
|
4632
|
+
completed_at: string | null;
|
|
4633
|
+
/** @description Total credits billed for the job. Null until the job is billed. Fractional — credits are NUMERIC(10,2). */
|
|
4634
|
+
total_credits_charged: number | null;
|
|
4635
|
+
/** @description Null in GET responses (computed at submission time only). */
|
|
4636
|
+
estimated_credits: number | null;
|
|
4637
|
+
/** @description Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported) */
|
|
4638
|
+
webhook_url: string;
|
|
4639
|
+
};
|
|
4640
|
+
};
|
|
4641
|
+
};
|
|
4519
4642
|
/** @description Resource not found. */
|
|
4520
4643
|
404: {
|
|
4521
4644
|
headers: {
|
|
@@ -5144,7 +5267,7 @@ export interface operations {
|
|
|
5144
5267
|
"application/json": components["schemas"]["Error"];
|
|
5145
5268
|
};
|
|
5146
5269
|
};
|
|
5147
|
-
/** @description Payment required
|
|
5270
|
+
/** @description Payment required. Either the account is out of credits (`quota_exceeded`), or a custom overage-enabled plan would cross its configured overage spending cap (`spending_cap_reached`). */
|
|
5148
5271
|
402: {
|
|
5149
5272
|
headers: {
|
|
5150
5273
|
[name: string]: unknown;
|
|
@@ -5248,18 +5371,8 @@ export interface operations {
|
|
|
5248
5371
|
* @enum {string}
|
|
5249
5372
|
*/
|
|
5250
5373
|
wait_until?: "load" | "domcontentloaded" | "networkidle";
|
|
5251
|
-
/** @default [] */
|
|
5252
|
-
cookies?: {
|
|
5253
|
-
name: string;
|
|
5254
|
-
value: string;
|
|
5255
|
-
domain: string;
|
|
5256
|
-
}[];
|
|
5257
5374
|
/** @default 0 */
|
|
5258
5375
|
delay_ms?: number;
|
|
5259
|
-
/** @default {} */
|
|
5260
|
-
headers?: {
|
|
5261
|
-
[key: string]: string;
|
|
5262
|
-
};
|
|
5263
5376
|
/**
|
|
5264
5377
|
* @description Two-letter country code, upper-cased. `null` is coerced to `US`.
|
|
5265
5378
|
* @default US
|
|
@@ -5433,7 +5546,7 @@ export interface operations {
|
|
|
5433
5546
|
plan_type: "payg" | "subscription";
|
|
5434
5547
|
/** @description Total top-up spend in cents for this billing period. Present only in dual/bytes billing mode. */
|
|
5435
5548
|
topup_spent_cents?: number;
|
|
5436
|
-
/** @description Remaining top-up bytes
|
|
5549
|
+
/** @description Remaining purchased top-up bytes, already net of any overage consumed against them. Equal to topup_bytes. Present only in dual/bytes billing mode. */
|
|
5437
5550
|
topup_balance_bytes?: number;
|
|
5438
5551
|
/** @description Days elapsed since period_start. */
|
|
5439
5552
|
period_days_elapsed: number;
|
|
@@ -5443,9 +5556,13 @@ export interface operations {
|
|
|
5443
5556
|
bytes_bulk: number;
|
|
5444
5557
|
bytes_scrape: number;
|
|
5445
5558
|
bytes_monitor: number;
|
|
5559
|
+
/** @description The spendable byte budget for this period, summed over the account's value lots: the allowance granted for the current period, plus unspent allowance carried over from earlier periods, plus purchased top-up and starter-grant bytes. Carried-over value is spent first, so this figure falls as it is consumed. */
|
|
5446
5560
|
bytes_limit: number;
|
|
5561
|
+
/** @description Bytes still available across every active lot, after active refund reservations. */
|
|
5447
5562
|
bytes_remaining: number;
|
|
5563
|
+
/** @description Unspent allowance carried over from earlier billing periods. */
|
|
5448
5564
|
rollover_bytes: number;
|
|
5565
|
+
/** @description Bytes remaining in purchased top-up and starter-grant lots. Subscription allowance is excluded. */
|
|
5449
5566
|
topup_bytes: number;
|
|
5450
5567
|
/** @description Total API requests in the current billing period. */
|
|
5451
5568
|
request_count: number;
|
|
@@ -5624,9 +5741,9 @@ export interface operations {
|
|
|
5624
5741
|
status_code?: number | null;
|
|
5625
5742
|
/** @description Raw upstream bytes the request carried (the billing base before the multiplier). null while pending/failed. */
|
|
5626
5743
|
raw_bytes?: number | null;
|
|
5627
|
-
/** @description
|
|
5744
|
+
/** @description The bytes this request was BILLED — the same amount its own response reported in `X-Billable-Bytes`. For a scrape or fetch request this is the settled decomposition replayed as it was recorded at completion, so it already carries every factor that applied: the endpoint factor (`scrape` 1.0, `scrape_md` 1.5, `fetch` 1.0, `fetch_md` 1.5, a monitor run 1.0), the cache-hit ×0.5, and the clean-markdown ×3. Screenshots and recordings are `raw_bytes` × the fixed 1.5 Phase-1 render factor whatever capture options they carried — capture billing composes no discount factor, so that product is exact. It is NOT `raw_bytes` × `billing_multiplier`: that field reports a CREDIT count, a different quantity from the byte factors above, and on a recordings row their product is not bytes at all. One residual: a scrape or fetch row completed before the decomposition was recorded is reconstructed from its endpoint and cache-hit factors, which leaves a markdown `/v1/fetch` request of that vintage reported at 1.0 rather than 1.5 — nothing on such a row distinguishes it. That population ages out of this feed's retention window and is not backfilled. null while pending/failed (the row has no byte basis yet). */
|
|
5628
5745
|
billable_bytes?: number | null;
|
|
5629
|
-
/** @description BASE billing multiplier persisted for this request — the settled `credits_charged` factor
|
|
5746
|
+
/** @description BASE billing multiplier persisted for this request — the settled `credits_charged` factor. It is NOT what `billable_bytes` above is derived from: that field reports settled BYTES, and multiplying `raw_bytes` by this credit count does not give the amount billed. Its scale is not uniform and is not a rate you can carry from one row to another: each endpoint persists its own credit unit, and the single and bulk paths of the same endpoint may persist different factors for identical options. Read this as the factor this one row settled at, and see https://bytekit.com/docs/billing/credits for what each endpoint charges. This is a LOGS ROW field and is NOT the scrape response envelope field — the envelope carries no flat `billing_multiplier` at all; its billing surface is the `billing` node (see BillingNode). This row reports only the base settled factor, so it can differ from `billing.multiplier` and from the X-Billing-Multiplier response header, both of which report the composed real-endpoint value. null while pending/failed. */
|
|
5630
5747
|
billing_multiplier?: number | null;
|
|
5631
5748
|
duration_ms?: number | null;
|
|
5632
5749
|
/** @description How the request originated (single/bulk/monitor). */
|
|
@@ -6063,7 +6180,7 @@ export interface operations {
|
|
|
6063
6180
|
/** @description Whether the authenticated account's plan supports bandwidth top-up purchases. False for free and PAYG plans. The frontend should disable or hide the Buy Bandwidth affordance when this is false. */
|
|
6064
6181
|
available: boolean;
|
|
6065
6182
|
};
|
|
6066
|
-
/** @description The
|
|
6183
|
+
/** @description The bandwidth allotment configured on the account's current plan (before any rollover or top-up bytes). Use this value instead of computing the plan allotment client-side from `limit_bytes - rollover_bytes - topup_bytes`: the allowance actually granted is the lot written when the period's invoice was paid, so the two can differ before the first paid invoice of a plan. */
|
|
6067
6184
|
included_bytes?: number;
|
|
6068
6185
|
/** @description Number of automatic top-up transactions (bandwidth and credit) that have been triggered in the current calendar month. Includes both pending and completed transactions. */
|
|
6069
6186
|
triggers_this_month?: number;
|
package/dist/index.d.ts
CHANGED
|
@@ -3,7 +3,7 @@ export type { ByteKitOptions, RequestOptions, ApiResponse, ConnectionErrorCode,
|
|
|
3
3
|
export { V01_METHODS } from './allowlist.js';
|
|
4
4
|
export type { V01Method } from './allowlist.js';
|
|
5
5
|
export type { ScrapeId, ScreenshotId, BulkId, MonitorId, SitemapId, ApiKeyId, WebhookDeliveryId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
|
|
6
|
-
export type {
|
|
6
|
+
export type { ScrapeOpts, MarkdownMode, MarkdownLinks, MarkdownImages, TokenEncoding, ScrapeCreateResponse, ScrapeGetResponse, ScrapeBulkCreateRequest, ScrapeBulkCreateResponse, ScrapeBulkGetResponse, } from './resources/scrape.js';
|
|
7
7
|
export type { ScreenshotCreateRequest, ScreenshotCreateResponse, ScreenshotGetResponse, } from './resources/screenshots.js';
|
|
8
8
|
export type { FetchOpts, FetchGetOpts, FetchResult, FetchCacheStatus, FetchBulkCreateRequest, FetchBulkCreateResponse, FetchBulkGetResponse, } from './resources/fetch.js';
|
|
9
9
|
export type { MonitorCreateRequest, MonitorCreateResponse, ListMonitorsParams, MonitorStatus, ListMonitorsResponse, MonitorResponse, MonitorUpdateRequest, UpdateMonitorResponse, ListMonitorCapturesResponse, } from './resources/monitors.js';
|
|
@@ -15,32 +15,8 @@ export type FetchBulkGetResponse = operations['getFetchBulk']['responses'][200][
|
|
|
15
15
|
*/
|
|
16
16
|
export interface FetchOpts {
|
|
17
17
|
url: string;
|
|
18
|
-
/** Output format. `'markdown'` runs the HTML→markdown pipeline; `'html'` returns raw HTML. Narrowed to the spec union — off-spec values (e.g. `'pdf'`) are rejected at compile time. */
|
|
19
|
-
format?: 'markdown' | 'html';
|
|
20
18
|
country?: string;
|
|
21
19
|
timeout_ms?: number;
|
|
22
|
-
/** Markdown processing mode. Only used when `format=markdown`. article/raw/llm. */
|
|
23
|
-
markdown_mode?: 'article' | 'raw' | 'llm';
|
|
24
|
-
/** BM25 query string for relevance-ranked filtering. Only used when `format=markdown`. */
|
|
25
|
-
markdown_query?: string;
|
|
26
|
-
/** Link rendering style. Only used when `format=markdown`. */
|
|
27
|
-
markdown_links?: 'inline' | 'references' | 'none' | 'text';
|
|
28
|
-
/** Image retention mode. Only used when `format=markdown`. */
|
|
29
|
-
markdown_images?: 'inline' | 'references' | 'none' | 'text';
|
|
30
|
-
/** Append a `Links:` footer to the markdown output. Only used when `format=markdown`. */
|
|
31
|
-
with_links_summary?: boolean;
|
|
32
|
-
/** Append an `Images:` footer to the markdown output. Only used when `format=markdown`. */
|
|
33
|
-
with_images_summary?: boolean;
|
|
34
|
-
/** Compact whitespace output. Only used when `format=markdown`. */
|
|
35
|
-
markdown_compact?: boolean;
|
|
36
|
-
/** Filter low-signal images. Only used when `format=markdown`. */
|
|
37
|
-
markdown_filter_images?: boolean;
|
|
38
|
-
/** When true, `formats.links`/`formats.images` return rich objects and a top-level `tables` array is included. Only used when `format=markdown`. */
|
|
39
|
-
markdown_include_media?: boolean;
|
|
40
|
-
/** When true, enables markdown-pipeline and tag-filter warnings for affected html/markdown requests. */
|
|
41
|
-
markdown_include_warnings?: boolean;
|
|
42
|
-
/** When true, includes a top-level `stats` object (chars, tokens, blocks). Only used when `format=markdown`. */
|
|
43
|
-
markdown_include_stats?: boolean;
|
|
44
20
|
/**
|
|
45
21
|
* How long a freshly fetched URL may be served from cache. Matches the
|
|
46
22
|
* OpenAPI `FetchRequest.cache_ttl` schema: a duration string (`"48h"`,
|
|
@@ -66,17 +42,15 @@ export interface FetchOpts {
|
|
|
66
42
|
* the generated `getFetch` query type by `_AssertFetchGetQueryKeysCoverSpec`
|
|
67
43
|
* below, so it cannot drift away from the spec silently.
|
|
68
44
|
*/
|
|
69
|
-
declare const FETCH_GET_QUERY_KEYS: readonly ["url", "
|
|
45
|
+
declare const FETCH_GET_QUERY_KEYS: readonly ["url", "country", "timeout_ms", "cache_ttl"];
|
|
70
46
|
type FetchGetQueryKey = (typeof FETCH_GET_QUERY_KEYS)[number];
|
|
71
47
|
/**
|
|
72
48
|
* Options accepted by `fetch.get` (GET /v1/fetch).
|
|
73
49
|
*
|
|
74
|
-
* A strict subset of {@link FetchOpts}: only the
|
|
50
|
+
* A strict subset of {@link FetchOpts}: only the four params the spec's
|
|
75
51
|
* `GET /v1/fetch` actually declares. Everything else on the POST body —
|
|
76
|
-
* `custom`
|
|
77
|
-
*
|
|
78
|
-
* excluded here and passing it is a compile error. Use `create()` (POST) for
|
|
79
|
-
* those.
|
|
52
|
+
* `custom` has no GET query param and is excluded here; passing it is a compile
|
|
53
|
+
* error. Use `create()` (POST) for custom metadata.
|
|
80
54
|
*/
|
|
81
55
|
export type FetchGetOpts = Pick<FetchOpts, FetchGetQueryKey>;
|
|
82
56
|
/** Cache-lookup outcome reported in the `X-Fetch-Cache` response header. */
|
|
@@ -84,8 +58,8 @@ export type FetchCacheStatus = 'hit' | 'miss' | 'bypass';
|
|
|
84
58
|
/**
|
|
85
59
|
* The result of `fetch.create` / `fetch.get`.
|
|
86
60
|
*
|
|
87
|
-
* `/v1/fetch` returns the fetched content AS the response body
|
|
88
|
-
*
|
|
61
|
+
* `/v1/fetch` returns the fetched content AS the response body, including its
|
|
62
|
+
* original media type and raw bytes, with metadata carried in `X-Fetch-*`
|
|
89
63
|
* headers on the response — it is NOT a JSON envelope. `body` is therefore the raw
|
|
90
64
|
* content string, never a parsed object, and the named fields decode the
|
|
91
65
|
* headers the docs call out. Anything not surfaced here is available on
|
|
@@ -144,10 +118,8 @@ export declare class FetchResource {
|
|
|
144
118
|
*
|
|
145
119
|
* Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
|
|
146
120
|
*
|
|
147
|
-
* Accepts only the
|
|
148
|
-
*
|
|
149
|
-
* options with no GET query param — they were silently dropped by this builder,
|
|
150
|
-
* so they are now compile errors instead. Use `create()` (POST) for those.
|
|
121
|
+
* Accepts only the four params the spec declares for this endpoint. `custom`
|
|
122
|
+
* is a POST-body option with no GET query param; use `create()` for custom metadata.
|
|
151
123
|
*
|
|
152
124
|
* The query string is built by iterating `FETCH_GET_QUERY_KEYS`, the same list
|
|
153
125
|
* `FetchGetOpts` is derived from, so every accepted option is necessarily
|
package/dist/resources/fetch.js
CHANGED
|
@@ -9,7 +9,7 @@ import { buildPath } from '../client.js';
|
|
|
9
9
|
* the generated `getFetch` query type by `_AssertFetchGetQueryKeysCoverSpec`
|
|
10
10
|
* below, so it cannot drift away from the spec silently.
|
|
11
11
|
*/
|
|
12
|
-
const FETCH_GET_QUERY_KEYS = ['url', '
|
|
12
|
+
const FETCH_GET_QUERY_KEYS = ['url', 'country', 'timeout_ms', 'cache_ttl'];
|
|
13
13
|
const CACHE_STATUSES = ['hit', 'miss', 'bypass'];
|
|
14
14
|
/** Build a typed {@link FetchResult} from a raw body + its header set. */
|
|
15
15
|
function toFetchResult(raw) {
|
|
@@ -60,10 +60,8 @@ export class FetchResource {
|
|
|
60
60
|
*
|
|
61
61
|
* Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
|
|
62
62
|
*
|
|
63
|
-
* Accepts only the
|
|
64
|
-
*
|
|
65
|
-
* options with no GET query param — they were silently dropped by this builder,
|
|
66
|
-
* so they are now compile errors instead. Use `create()` (POST) for those.
|
|
63
|
+
* Accepts only the four params the spec declares for this endpoint. `custom`
|
|
64
|
+
* is a POST-body option with no GET query param; use `create()` for custom metadata.
|
|
67
65
|
*
|
|
68
66
|
* The query string is built by iterating `FETCH_GET_QUERY_KEYS`, the same list
|
|
69
67
|
* `FetchGetOpts` is derived from, so every accepted option is necessarily
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { ByteKit, RequestOptions } from '../client.js';
|
|
2
|
-
import type {
|
|
2
|
+
import type { operations } from '../generated/api.js';
|
|
3
3
|
import type { ScrapeId, BulkId } from '../types.js';
|
|
4
4
|
/**
|
|
5
5
|
* Response of POST /v1/scrape — a 200 sync success envelope or a 202 queued
|
|
@@ -26,23 +26,11 @@ export type MarkdownLinks = 'inline' | 'references' | 'none' | 'text';
|
|
|
26
26
|
export type MarkdownImages = 'inline' | 'references' | 'none' | 'text';
|
|
27
27
|
/** Tokenizer encoding hint for `token_budget`. */
|
|
28
28
|
export type TokenEncoding = 'cl100k_base' | 'o200k_base';
|
|
29
|
-
/**
|
|
30
|
-
* A cookie to set before loading the page — the spec's `Cookie` schema verbatim
|
|
31
|
-
* (`name` and `value` required; `domain`, `path`, `secure`, `httpOnly` optional).
|
|
32
|
-
*
|
|
33
|
-
* DERIVED from the generated schema (#2547), never hand-authored: `cookies` was
|
|
34
|
-
* previously an open bag of untyped records, so a nameless cookie compiled and only
|
|
35
|
-
* failed as a 422 on the wire.
|
|
36
|
-
*/
|
|
37
|
-
export type ScrapeCookie = components['schemas']['Cookie'];
|
|
38
29
|
/** Options for POST /v1/scrape. */
|
|
39
30
|
export interface ScrapeOpts {
|
|
40
31
|
url: string;
|
|
41
32
|
formats?: Array<'raw_html' | 'html' | 'markdown' | 'links' | 'images'>;
|
|
42
33
|
country?: string;
|
|
43
|
-
/** Cookies to set before loading the page. `name` and `value` are required (spec `Cookie` schema). */
|
|
44
|
-
cookies?: ScrapeCookie[];
|
|
45
|
-
headers?: Record<string, string>;
|
|
46
34
|
delay_ms?: number;
|
|
47
35
|
/** SERVER-side scrape budget in ms — how long the gateway may spend on the scrape itself. For the CLIENT-side abort budget (when this SDK gives up on the wire), see `RequestOptions.requestTimeoutMs`. */
|
|
48
36
|
timeout_ms?: number;
|
package/dist/version.d.ts
CHANGED
|
@@ -5,4 +5,4 @@
|
|
|
5
5
|
* outbound request carries — see the emit site in `client.ts`. The header name is spelled
|
|
6
6
|
* only there, so issue #2905's pre-merge gate 1 counts exactly one occurrence in `src/`.
|
|
7
7
|
*/
|
|
8
|
-
export declare const SDK_VERSION = "0.
|
|
8
|
+
export declare const SDK_VERSION = "0.9.1";
|
package/dist/version.js
CHANGED