@hunt-labs/bytekit-sdk 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +142 -0
- package/README.md +63 -17
- package/dist/allowlist.d.ts +1 -1
- package/dist/allowlist.js +7 -0
- package/dist/client.d.ts +49 -10
- package/dist/client.js +81 -21
- package/dist/index.d.ts +9 -3
- package/dist/index.js +1 -1
- package/dist/resources/account.d.ts +6 -7
- package/dist/resources/account.js +3 -4
- package/dist/resources/bulk.d.ts +18 -7
- package/dist/resources/fetch.d.ts +98 -10
- package/dist/resources/fetch.js +33 -6
- package/dist/resources/monitors.d.ts +50 -11
- package/dist/resources/monitors.js +17 -4
- package/dist/resources/query-string.d.ts +10 -0
- package/dist/resources/query-string.js +20 -0
- package/dist/resources/scrape.d.ts +28 -4
- package/dist/resources/screenshots.d.ts +13 -4
- package/dist/resources/screenshots.js +11 -3
- package/dist/resources/search.d.ts +2 -2
- package/dist/resources/sitemap.d.ts +11 -4
- package/dist/resources/usage.d.ts +31 -0
- package/dist/resources/usage.js +25 -0
- package/dist/resources/webhooks.d.ts +20 -0
- package/dist/resources/webhooks.js +15 -0
- package/dist/types.d.ts +7 -4
- package/dist/types.js +1 -1
- package/package.json +6 -10
- package/dist/resources/recordings.d.ts +0 -10
- package/dist/resources/recordings.js +0 -14
- package/dist/test-helpers/signal-aware-never-resolver.d.ts +0 -9
- package/dist/test-helpers/signal-aware-never-resolver.js +0 -25
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `@hunt-labs/bytekit-sdk` are documented here.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
This file records only the delta between versions. For how to use the package
|
|
9
|
+
today, see [README.md](./README.md), which remains the authoritative usage guide.
|
|
10
|
+
|
|
11
|
+
## [0.3.0] - 2026-07-21
|
|
12
|
+
|
|
13
|
+
### Removed (**BREAKING**)
|
|
14
|
+
|
|
15
|
+
- The deprecated `RapidCrawl`, `RapidCrawlError`, and `RapidCrawlConnectionError`
|
|
16
|
+
exports and the `RapidCrawlOptions` type alias (kept as a one-minor-version
|
|
17
|
+
bridge in #2421) are removed. Import the `ByteKit*` names instead. (#2473)
|
|
18
|
+
- The `RecordingId` type export is removed. Recordings are deactivated
|
|
19
|
+
(`/v1/recordings` returns 410), so the branded id had no reachable use.
|
|
20
|
+
Breaking in theory only — unused in practice. (#2481)
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- `fetch.create` / `fetch.get` no longer throw a bare `SyntaxError` on every
|
|
25
|
+
non-JSON body. `/v1/fetch` returns the fetched content AS the response body
|
|
26
|
+
(metadata in `X-Fetch-*` headers), but both methods routed through the
|
|
27
|
+
JSON-parsing transport. They now use a raw-capable transport and resolve to a
|
|
28
|
+
typed `FetchResult` — raw `body`, byte-exact `bytes`, plus `id`,
|
|
29
|
+
`creditsCharged`, `cache`, `custom`, `contentType`, and `headers` parsed from
|
|
30
|
+
the `X-Fetch-*` metadata. (#2466)
|
|
31
|
+
- A trailing slash on the constructor `baseUrl` (e.g. `"https://api.bytekit.com/"`)
|
|
32
|
+
no longer produces double-slash request URLs — it is normalized once, preserving
|
|
33
|
+
a path-carrying baseUrl. (#2481)
|
|
34
|
+
- `SearchResult.date` is now optional (`string | null | undefined`): `images`
|
|
35
|
+
search results omit the field entirely. (#2481)
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
|
|
39
|
+
- New `usage` resource (`get`, `daily`, `byEndpoint` — `GET /v1/usage`,
|
|
40
|
+
`/v1/usage/daily`, `/v1/usage/by-endpoint`) and `webhooks` resource (`list`,
|
|
41
|
+
`retry` — `GET /v1/webhook-deliveries`, `POST /v1/webhook-deliveries/{id}/retry`),
|
|
42
|
+
with their request/response types exported. (#2471)
|
|
43
|
+
- `ScrapeOpts` and `FetchOpts` now cover the full documented request field set
|
|
44
|
+
(10 previously missing `ScrapeRequest` fields; 11 markdown-tuning
|
|
45
|
+
`FetchRequest` fields), plus the `'text'` markdown-links mode. A compile-time
|
|
46
|
+
drift guard keeps them aligned with the generated OpenAPI schema. (#2475)
|
|
47
|
+
- `monitors`, `sitemap`, `bulk`, and `fetch.bulk` methods are now fully typed from
|
|
48
|
+
the generated OpenAPI `operations` (previously `Record<string, unknown>` in /
|
|
49
|
+
`Promise<unknown>` out); their request/response types are re-exported. (#2481)
|
|
50
|
+
- `monitors.list` accepts typed query params and returns a typed response
|
|
51
|
+
(`ListMonitorsParams` / `ListMonitorsResponse`, both exported). (#2480)
|
|
52
|
+
|
|
53
|
+
### Changed (**BREAKING**)
|
|
54
|
+
|
|
55
|
+
- `fetch.get` now rejects a `custom` option at compile time: `GET /v1/fetch` has
|
|
56
|
+
no `custom` query param, so the value was silently dropped before. Use
|
|
57
|
+
`fetch.create` (POST) to echo a `custom` payload. (#2481)
|
|
58
|
+
- `FetchOpts.format` is narrowed from `string` to the spec union
|
|
59
|
+
`'markdown' | 'html'`; off-spec values now fail to compile. (#2475)
|
|
60
|
+
|
|
61
|
+
## [0.2.2] - 2026-07-16
|
|
62
|
+
|
|
63
|
+
### Changed
|
|
64
|
+
|
|
65
|
+
- Re-release of 0.2.1. The 0.2.1 artifacts never reached npm — the prod publish's
|
|
66
|
+
`publish-sdk` step failed on a CI test-isolation flake (`cjs-require.test.ts`
|
|
67
|
+
rebuilding the shared `dist`), which is now fixed. No consumer-facing changes
|
|
68
|
+
beyond 0.2.1.
|
|
69
|
+
|
|
70
|
+
## [0.2.1] - 2026-07-16
|
|
71
|
+
|
|
72
|
+
### Changed
|
|
73
|
+
|
|
74
|
+
- Primary exports renamed `RapidCrawl`→`ByteKit`, `RapidCrawlError`→`ByteKitError`,
|
|
75
|
+
`RapidCrawlOptions`→`ByteKitOptions`. The old names remain as deprecated,
|
|
76
|
+
identity-preserving aliases (`RapidCrawl === ByteKit`), so existing imports keep
|
|
77
|
+
working. (#2421)
|
|
78
|
+
|
|
79
|
+
### Fixed
|
|
80
|
+
|
|
81
|
+
- `request()` now returns `undefined` on `204 No Content` / empty-body responses
|
|
82
|
+
instead of throwing a JSON-parse error — `account.apiKeys.revoke()` now resolves
|
|
83
|
+
as its `Promise<void>` contract declares. (#2450)
|
|
84
|
+
|
|
85
|
+
### Removed
|
|
86
|
+
|
|
87
|
+
- `client.recordings` (the `/v1/recordings` endpoint is permanently `410`
|
|
88
|
+
deactivated). (#2421)
|
|
89
|
+
|
|
90
|
+
## [0.2.0] - 2026-07-15
|
|
91
|
+
|
|
92
|
+
This is a breaking release. Four runtime behavior changes affect code written
|
|
93
|
+
against 0.1.0 — read each one before upgrading.
|
|
94
|
+
|
|
95
|
+
### Changed (breaking)
|
|
96
|
+
|
|
97
|
+
- **Default client-side timeout (120s).** Requests now abort after 120000 ms by
|
|
98
|
+
default (`DEFAULT_TIMEOUT_MS` in `src/client.ts`); 0.1.0 had no client-side
|
|
99
|
+
timeout and would hang indefinitely. A slow request now rejects with
|
|
100
|
+
`RapidCrawlConnectionError` carrying `code: 'timeout'`. Override per client via
|
|
101
|
+
`new RapidCrawl({ apiKey, timeoutMs })`.
|
|
102
|
+
- **Network failures throw `RapidCrawlConnectionError`.** Every `fetch()` rejection
|
|
103
|
+
is now wrapped by `toConnectionError()` and re-thrown as `RapidCrawlConnectionError`.
|
|
104
|
+
0.1.0 surfaced undici's bare `TypeError: fetch failed`. Code that matched on a bare
|
|
105
|
+
`TypeError` to detect a network failure will no longer match — catch
|
|
106
|
+
`RapidCrawlConnectionError` instead.
|
|
107
|
+
- **Constructor throws on an empty `apiKey`.** `new RapidCrawl({ apiKey: '' })` (or a
|
|
108
|
+
missing key) now throws a `TypeError` immediately instead of silently sending
|
|
109
|
+
`Authorization: Bearer undefined`. Ensure `apiKey` is populated (e.g. from
|
|
110
|
+
`process.env.BYTEKIT_API_KEY`) before constructing the client.
|
|
111
|
+
- **Request-body types narrowed to the OpenAPI schema.** Request payloads are now
|
|
112
|
+
typed from spec-derived types generated from the OpenAPI schema
|
|
113
|
+
(`src/generated/api.d.ts`) rather than `Record<string, unknown>`. Payloads that
|
|
114
|
+
previously compiled by virtue of the loose index type may now be rejected by the
|
|
115
|
+
type checker; align them with the documented request shape.
|
|
116
|
+
|
|
117
|
+
### Changed
|
|
118
|
+
|
|
119
|
+
- Package renamed to `@hunt-labs/bytekit-sdk` (the `@bytekit` npm scope was
|
|
120
|
+
unobtainable). The prior `rapidcrawl`/`@rapidcrawl/sdk` name remains installable at
|
|
121
|
+
its last version but is deprecated.
|
|
122
|
+
- Generated API types (`dist/generated/api.d.ts`) are now shipped in the tarball so
|
|
123
|
+
consumers get the typed request/response surface without regenerating from the spec.
|
|
124
|
+
|
|
125
|
+
### Added
|
|
126
|
+
|
|
127
|
+
- Publish-ready packaging metadata: types-first `exports`, `engines`, `keywords`,
|
|
128
|
+
`homepage`, `bugs`, `README.md`, `LICENSE`, and an MIT `license` field.
|
|
129
|
+
|
|
130
|
+
## [0.1.0] - 2026
|
|
131
|
+
|
|
132
|
+
### Added
|
|
133
|
+
|
|
134
|
+
- Initial release of the ByteKit TypeScript SDK — a typed client for the ByteKit API
|
|
135
|
+
(scrape, screenshot, search, and related endpoints), published under the original
|
|
136
|
+
`@rapidcrawl/sdk` name.
|
|
137
|
+
|
|
138
|
+
[0.3.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
|
|
139
|
+
[0.2.2]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
|
|
140
|
+
[0.2.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
|
|
141
|
+
[0.2.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
|
|
142
|
+
[0.1.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
|
package/README.md
CHANGED
|
@@ -10,11 +10,14 @@ screenshots, sitemaps, page monitors, and web search.
|
|
|
10
10
|
npm install @hunt-labs/bytekit-sdk
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
> **ESM-only.** This package ships
|
|
14
|
-
> CommonJS
|
|
15
|
-
> `package.json`, or a `.mjs` entrypoint)
|
|
16
|
-
> dynamic `await import('@hunt-labs/bytekit-sdk')
|
|
17
|
-
> `require(
|
|
13
|
+
> **ESM-only.** This package ships a single ESM build (`"type": "module"`) with
|
|
14
|
+
> no separate CommonJS bundle. Use it from an ESM project (`"type": "module"` in
|
|
15
|
+
> your `package.json`, or a `.mjs` entrypoint). From CommonJS you can either load
|
|
16
|
+
> it with a dynamic `await import('@hunt-labs/bytekit-sdk')`, or — on **Node
|
|
17
|
+
> ≥20.19 / ≥22.12**, which resolve `require()` of ESM — call a top-level
|
|
18
|
+
> `require('@hunt-labs/bytekit-sdk')` directly (the package's `exports` map
|
|
19
|
+
> carries a `default` condition for exactly this). On older Node, a top-level
|
|
20
|
+
> `require()` throws `ERR_REQUIRE_ESM`; use the dynamic `import()` there.
|
|
18
21
|
|
|
19
22
|
Requires Node.js 18+ (the client uses the global `fetch`).
|
|
20
23
|
|
|
@@ -24,16 +27,15 @@ Create an API key at [app.bytekit.com](https://app.bytekit.com) (keys are
|
|
|
24
27
|
prefixed `sk_live_`) and pass it to the client constructor:
|
|
25
28
|
|
|
26
29
|
```ts
|
|
27
|
-
import {
|
|
30
|
+
import { ByteKit } from '@hunt-labs/bytekit-sdk';
|
|
28
31
|
|
|
29
|
-
const client = new
|
|
32
|
+
const client = new ByteKit({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...
|
|
30
33
|
```
|
|
31
34
|
|
|
32
35
|
The key is sent as `Authorization: Bearer <apiKey>` on every request. Keep it
|
|
33
36
|
server-side — it is not safe to ship in a browser bundle.
|
|
34
37
|
|
|
35
|
-
>
|
|
36
|
-
> previous product name). It is the SDK's only entrypoint class.
|
|
38
|
+
> `ByteKit` is the SDK's only entrypoint class.
|
|
37
39
|
|
|
38
40
|
## Base-URL override
|
|
39
41
|
|
|
@@ -41,7 +43,7 @@ Requests go to `https://api.bytekit.com` by default. Point the client at
|
|
|
41
43
|
staging, a local gateway, or a proxy with the `baseUrl` option:
|
|
42
44
|
|
|
43
45
|
```ts
|
|
44
|
-
const client = new
|
|
46
|
+
const client = new ByteKit({
|
|
45
47
|
apiKey: process.env.BYTEKIT_API_KEY!,
|
|
46
48
|
baseUrl: 'https://api-stg.bytekit.com', // default: https://api.bytekit.com
|
|
47
49
|
});
|
|
@@ -66,8 +68,27 @@ const scrape = await client.scrape.get('sc_123abc');
|
|
|
66
68
|
|
|
67
69
|
### Raw fetch (unprocessed response body)
|
|
68
70
|
|
|
71
|
+
`/v1/fetch` returns the fetched content **as the body**, with metadata in
|
|
72
|
+
`X-Fetch-*` response headers. `fetch.create` / `fetch.get` resolve a
|
|
73
|
+
`FetchResult` — `body` is the raw content string (a JSON-serving upstream
|
|
74
|
+
comes back as the raw JSON string, never a parsed object), and the named
|
|
75
|
+
fields decode the metadata headers. `body` is UTF-8-decoded, so for true
|
|
76
|
+
binary content (images, PDFs, etc. — `/v1/fetch` supports raw passthrough)
|
|
77
|
+
use `bytes` instead, which is byte-for-byte identical to the wire content:
|
|
78
|
+
|
|
69
79
|
```ts
|
|
70
|
-
const
|
|
80
|
+
const res = await client.fetch.create({ url: 'https://example.com' });
|
|
81
|
+
|
|
82
|
+
console.log(res.body); // raw content (e.g. "<!doctype html>…") — UTF-8 decoded
|
|
83
|
+
console.log(res.bytes); // Uint8Array — byte-exact, safe for binary content
|
|
84
|
+
console.log(res.id); // X-Fetch-ID, e.g. "ft_0123abcd…"
|
|
85
|
+
console.log(res.creditsCharged); // X-Fetch-Credits-Charged, as a number
|
|
86
|
+
console.log(res.cache); // "hit" | "miss" | "bypass"
|
|
87
|
+
console.log(res.custom); // base64 X-Fetch-Custom echo, or null
|
|
88
|
+
console.log(res.contentType); // response Content-Type, e.g. "text/html"
|
|
89
|
+
|
|
90
|
+
// Anything not surfaced as a named field is on res.headers:
|
|
91
|
+
console.log(res.headers.get('X-Fetch-Waf-Vendor'));
|
|
71
92
|
```
|
|
72
93
|
|
|
73
94
|
### Web search
|
|
@@ -121,7 +142,10 @@ await client.bulk.screenshots.list('bulk_123abc');
|
|
|
121
142
|
```ts
|
|
122
143
|
const monitor = await client.monitors.create({
|
|
123
144
|
url: 'https://example.com/pricing',
|
|
145
|
+
type: 'screenshot',
|
|
124
146
|
interval_type: 'daily',
|
|
147
|
+
change_threshold: 5,
|
|
148
|
+
notify_on: 'change',
|
|
125
149
|
webhook_url: 'https://your.app/hooks/bytekit',
|
|
126
150
|
});
|
|
127
151
|
await client.monitors.list();
|
|
@@ -131,7 +155,16 @@ await client.monitors.delete('mon_123abc');
|
|
|
131
155
|
### Sitemap crawl
|
|
132
156
|
|
|
133
157
|
```ts
|
|
134
|
-
const crawl = await client.sitemap.create({
|
|
158
|
+
const crawl = await client.sitemap.create({
|
|
159
|
+
url: 'https://example.com',
|
|
160
|
+
strategy: 'standard',
|
|
161
|
+
max_depth: 2,
|
|
162
|
+
max_urls: 5000,
|
|
163
|
+
process: null,
|
|
164
|
+
webhook_secret: null,
|
|
165
|
+
cache_ttl: '30d',
|
|
166
|
+
compact: false,
|
|
167
|
+
});
|
|
135
168
|
const sitemap = await client.sitemap.get('sm_123abc');
|
|
136
169
|
```
|
|
137
170
|
|
|
@@ -140,23 +173,36 @@ const sitemap = await client.sitemap.get('sm_123abc');
|
|
|
140
173
|
```ts
|
|
141
174
|
const account = await client.account.get();
|
|
142
175
|
const keys = await client.account.apiKeys.list();
|
|
176
|
+
|
|
177
|
+
// Usage (Bearer API key auth)
|
|
178
|
+
const usage = await client.usage.get();
|
|
179
|
+
const daily = await client.usage.daily({ from: '2026-01-01', to: '2026-01-31' });
|
|
180
|
+
const byEndpoint = await client.usage.byEndpoint();
|
|
181
|
+
|
|
182
|
+
// Webhook deliveries
|
|
183
|
+
const deliveries = await client.webhooks.list();
|
|
184
|
+
await client.webhooks.retry('whd_123abc');
|
|
143
185
|
```
|
|
144
186
|
|
|
145
|
-
> `client.
|
|
146
|
-
>
|
|
187
|
+
> **Note:** `client.account.*` (including `client.account.apiKeys.*`) targets the
|
|
188
|
+
> `/v1/account/*` endpoints, which authenticate with a Clerk dashboard session —
|
|
189
|
+
> **not** a Bearer API key. Calling them with an API-key client will fail auth.
|
|
190
|
+
> They remain on the SDK surface for dashboard/session consumers and are marked
|
|
191
|
+
> `x-sdk-scope: internal` in the spec, so they are intentionally excluded from the
|
|
192
|
+
> v0.1 Bearer-key parity guarantee.
|
|
147
193
|
|
|
148
194
|
## Error handling
|
|
149
195
|
|
|
150
|
-
Any 4xx/5xx response throws a `
|
|
196
|
+
Any 4xx/5xx response throws a `ByteKitError` carrying the HTTP status and
|
|
151
197
|
the API's error code:
|
|
152
198
|
|
|
153
199
|
```ts
|
|
154
|
-
import {
|
|
200
|
+
import { ByteKit, ByteKitError } from '@hunt-labs/bytekit-sdk';
|
|
155
201
|
|
|
156
202
|
try {
|
|
157
203
|
await client.scrape.create({ url: 'https://example.com' });
|
|
158
204
|
} catch (err) {
|
|
159
|
-
if (err instanceof
|
|
205
|
+
if (err instanceof ByteKitError) {
|
|
160
206
|
console.error(err.status, err.code, err.message);
|
|
161
207
|
}
|
|
162
208
|
}
|
package/dist/allowlist.d.ts
CHANGED
|
@@ -8,5 +8,5 @@
|
|
|
8
8
|
*
|
|
9
9
|
* To add a method: annotate the operation in the spec first, then add it here.
|
|
10
10
|
*/
|
|
11
|
-
export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get"];
|
|
11
|
+
export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get", "usage.get", "usage.daily", "usage.byEndpoint", "webhooks.list", "webhooks.retry"];
|
|
12
12
|
export type V01Method = (typeof V01_METHODS)[number];
|
package/dist/allowlist.js
CHANGED
|
@@ -34,4 +34,11 @@ export const V01_METHODS = [
|
|
|
34
34
|
'sitemap.get',
|
|
35
35
|
'search.query',
|
|
36
36
|
'account.get',
|
|
37
|
+
// usage + webhook-delivery ops — resource surface shipped in #2471, brought
|
|
38
|
+
// under the parity guard in #2476.
|
|
39
|
+
'usage.get',
|
|
40
|
+
'usage.daily',
|
|
41
|
+
'usage.byEndpoint',
|
|
42
|
+
'webhooks.list',
|
|
43
|
+
'webhooks.retry',
|
|
37
44
|
];
|
package/dist/client.d.ts
CHANGED
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
import { ScrapeResource } from './resources/scrape.js';
|
|
2
2
|
import { ScreenshotsResource } from './resources/screenshots.js';
|
|
3
|
-
import { RecordingsResource } from './resources/recordings.js';
|
|
4
3
|
import { BulkResource } from './resources/bulk.js';
|
|
5
4
|
import { FetchResource } from './resources/fetch.js';
|
|
6
5
|
import { MonitorsResource } from './resources/monitors.js';
|
|
7
6
|
import { SitemapResource } from './resources/sitemap.js';
|
|
8
7
|
import { SearchResource } from './resources/search.js';
|
|
9
8
|
import { AccountResource } from './resources/account.js';
|
|
10
|
-
|
|
9
|
+
import { UsageResource } from './resources/usage.js';
|
|
10
|
+
import { WebhooksResource } from './resources/webhooks.js';
|
|
11
|
+
export interface ByteKitOptions {
|
|
11
12
|
apiKey: string;
|
|
12
13
|
baseUrl?: string;
|
|
13
14
|
/**
|
|
@@ -42,38 +43,39 @@ export interface RequestOptions {
|
|
|
42
43
|
/** Why a request never produced an HTTP response. */
|
|
43
44
|
export type ConnectionErrorCode = 'timeout' | 'aborted' | 'connection';
|
|
44
45
|
/** Thrown when the API responds with a 4xx or 5xx status code. */
|
|
45
|
-
export declare class
|
|
46
|
+
export declare class ByteKitError extends Error {
|
|
46
47
|
readonly status: number;
|
|
47
48
|
readonly code: string;
|
|
48
49
|
constructor(status: number, code: string, message: string);
|
|
49
50
|
}
|
|
50
51
|
/**
|
|
51
52
|
* Thrown when the request never produced an HTTP response at all — DNS, socket, TLS,
|
|
52
|
-
* a client-side timeout, or a caller-initiated abort. Distinct from `
|
|
53
|
+
* a client-side timeout, or a caller-initiated abort. Distinct from `ByteKitError`,
|
|
53
54
|
* which always carries a real HTTP status: catch both to cover every failure.
|
|
54
55
|
*
|
|
55
56
|
* The original failure is preserved on `.cause`.
|
|
56
57
|
*/
|
|
57
|
-
export declare class
|
|
58
|
+
export declare class ByteKitConnectionError extends Error {
|
|
58
59
|
readonly code: ConnectionErrorCode;
|
|
59
60
|
constructor(code: ConnectionErrorCode, message: string, options?: {
|
|
60
61
|
cause?: unknown;
|
|
61
62
|
});
|
|
62
63
|
}
|
|
63
|
-
export declare class
|
|
64
|
+
export declare class ByteKit {
|
|
64
65
|
private readonly apiKey;
|
|
65
66
|
private readonly baseUrl;
|
|
66
67
|
private readonly defaultTimeoutMs;
|
|
67
68
|
readonly scrape: ScrapeResource;
|
|
68
69
|
readonly screenshots: ScreenshotsResource;
|
|
69
|
-
readonly recordings: RecordingsResource;
|
|
70
70
|
readonly bulk: BulkResource;
|
|
71
71
|
readonly fetch: FetchResource;
|
|
72
72
|
readonly monitors: MonitorsResource;
|
|
73
73
|
readonly sitemap: SitemapResource;
|
|
74
74
|
readonly search: SearchResource;
|
|
75
75
|
readonly account: AccountResource;
|
|
76
|
-
|
|
76
|
+
readonly usage: UsageResource;
|
|
77
|
+
readonly webhooks: WebhooksResource;
|
|
78
|
+
constructor(options: ByteKitOptions);
|
|
77
79
|
/**
|
|
78
80
|
* Resolve the abort budget for one request from the caller's options and the
|
|
79
81
|
* constructor default. Precedence:
|
|
@@ -89,11 +91,48 @@ export declare class RapidCrawl {
|
|
|
89
91
|
*/
|
|
90
92
|
private resolveSignal;
|
|
91
93
|
/**
|
|
92
|
-
*
|
|
94
|
+
* Perform the authenticated round-trip and return the raw `Response`, having
|
|
95
|
+
* already thrown for the two non-body failure modes:
|
|
93
96
|
*
|
|
94
|
-
*
|
|
97
|
+
* - `ByteKitConnectionError` when no HTTP response was produced at all
|
|
98
|
+
* (DNS/socket/TLS failure, client-side timeout, or caller abort).
|
|
99
|
+
* - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
|
|
100
|
+
*
|
|
101
|
+
* The body is left unread so each caller can decode it as it needs — JSON for
|
|
102
|
+
* `request()`, raw text + headers for `requestRaw()`. This is the single place
|
|
103
|
+
* that owns auth, the abort budget, and the error taxonomy; the two public
|
|
104
|
+
* transports differ only in how they read a successful body.
|
|
105
|
+
*/
|
|
106
|
+
private send;
|
|
107
|
+
/**
|
|
108
|
+
* Make an authenticated JSON request to the ByteKit API.
|
|
109
|
+
*
|
|
110
|
+
* Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
|
|
95
111
|
* when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
|
|
96
112
|
* timeout, or caller abort).
|
|
97
113
|
*/
|
|
98
114
|
request<T>(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<T>;
|
|
115
|
+
/**
|
|
116
|
+
* Make an authenticated request whose response body IS the content, not a JSON
|
|
117
|
+
* envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
|
|
118
|
+
* response headers.
|
|
119
|
+
*
|
|
120
|
+
* Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
|
|
121
|
+
* response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
|
|
122
|
+
* deliberately a separate transport from `request()`: the generic JSON parse
|
|
123
|
+
* there is shared by every other resource and must not change (#2466).
|
|
124
|
+
*
|
|
125
|
+
* `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
|
|
126
|
+
* first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
|
|
127
|
+
* invalid byte sequence gets silently replaced with U+FFFD. Reading the body
|
|
128
|
+
* once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
|
|
129
|
+
* is always byte-exact, `body` is a best-effort string for callers who know the
|
|
130
|
+
* content is text.
|
|
131
|
+
*/
|
|
132
|
+
requestRaw(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<{
|
|
133
|
+
body: string;
|
|
134
|
+
bytes: Uint8Array;
|
|
135
|
+
headers: Headers;
|
|
136
|
+
status: number;
|
|
137
|
+
}>;
|
|
99
138
|
}
|
package/dist/client.js
CHANGED
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
import { ScrapeResource } from './resources/scrape.js';
|
|
2
2
|
import { ScreenshotsResource } from './resources/screenshots.js';
|
|
3
|
-
import { RecordingsResource } from './resources/recordings.js';
|
|
4
3
|
import { BulkResource } from './resources/bulk.js';
|
|
5
4
|
import { FetchResource } from './resources/fetch.js';
|
|
6
5
|
import { MonitorsResource } from './resources/monitors.js';
|
|
7
6
|
import { SitemapResource } from './resources/sitemap.js';
|
|
8
7
|
import { SearchResource } from './resources/search.js';
|
|
9
8
|
import { AccountResource } from './resources/account.js';
|
|
9
|
+
import { UsageResource } from './resources/usage.js';
|
|
10
|
+
import { WebhooksResource } from './resources/webhooks.js';
|
|
10
11
|
const DEFAULT_BASE_URL = 'https://api.bytekit.com';
|
|
11
12
|
/**
|
|
12
13
|
* Default client-side request timeout (2 minutes).
|
|
@@ -17,28 +18,28 @@ const DEFAULT_BASE_URL = 'https://api.bytekit.com';
|
|
|
17
18
|
*/
|
|
18
19
|
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
19
20
|
/** Thrown when the API responds with a 4xx or 5xx status code. */
|
|
20
|
-
export class
|
|
21
|
+
export class ByteKitError extends Error {
|
|
21
22
|
status;
|
|
22
23
|
code;
|
|
23
24
|
constructor(status, code, message) {
|
|
24
25
|
super(message);
|
|
25
|
-
this.name = '
|
|
26
|
+
this.name = 'ByteKitError';
|
|
26
27
|
this.status = status;
|
|
27
28
|
this.code = code;
|
|
28
29
|
}
|
|
29
30
|
}
|
|
30
31
|
/**
|
|
31
32
|
* Thrown when the request never produced an HTTP response at all — DNS, socket, TLS,
|
|
32
|
-
* a client-side timeout, or a caller-initiated abort. Distinct from `
|
|
33
|
+
* a client-side timeout, or a caller-initiated abort. Distinct from `ByteKitError`,
|
|
33
34
|
* which always carries a real HTTP status: catch both to cover every failure.
|
|
34
35
|
*
|
|
35
36
|
* The original failure is preserved on `.cause`.
|
|
36
37
|
*/
|
|
37
|
-
export class
|
|
38
|
+
export class ByteKitConnectionError extends Error {
|
|
38
39
|
code;
|
|
39
40
|
constructor(code, message, options) {
|
|
40
41
|
super(message, options);
|
|
41
|
-
this.name = '
|
|
42
|
+
this.name = 'ByteKitConnectionError';
|
|
42
43
|
this.code = code;
|
|
43
44
|
}
|
|
44
45
|
}
|
|
@@ -52,7 +53,7 @@ function validateTimeout(value, optionName) {
|
|
|
52
53
|
if (value === undefined)
|
|
53
54
|
return undefined;
|
|
54
55
|
if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) {
|
|
55
|
-
throw new TypeError(`
|
|
56
|
+
throw new TypeError(`ByteKit: ${optionName} must be a non-negative finite number (got ${describe(value)})`);
|
|
56
57
|
}
|
|
57
58
|
return value;
|
|
58
59
|
}
|
|
@@ -69,7 +70,7 @@ function describe(value) {
|
|
|
69
70
|
}
|
|
70
71
|
/** The abort reason used by the `setTimeout` fallback; named so it maps to `code: 'timeout'`. */
|
|
71
72
|
function timeoutReason(ms) {
|
|
72
|
-
const reason = new Error(`
|
|
73
|
+
const reason = new Error(`ByteKit: request timed out after ${ms}ms`);
|
|
73
74
|
reason.name = 'TimeoutError';
|
|
74
75
|
return reason;
|
|
75
76
|
}
|
|
@@ -147,39 +148,45 @@ function toConnectionError(err, method, url, timeoutSignal) {
|
|
|
147
148
|
: code === 'aborted'
|
|
148
149
|
? 'request aborted by the caller'
|
|
149
150
|
: 'connection failed before a response was received';
|
|
150
|
-
return new
|
|
151
|
+
return new ByteKitConnectionError(code, `ByteKit: ${method} ${url} — ${detail}`, {
|
|
151
152
|
cause: err,
|
|
152
153
|
});
|
|
153
154
|
}
|
|
154
|
-
export class
|
|
155
|
+
export class ByteKit {
|
|
155
156
|
apiKey;
|
|
156
157
|
baseUrl;
|
|
157
158
|
defaultTimeoutMs;
|
|
158
159
|
scrape;
|
|
159
160
|
screenshots;
|
|
160
|
-
recordings;
|
|
161
161
|
bulk;
|
|
162
162
|
fetch;
|
|
163
163
|
monitors;
|
|
164
164
|
sitemap;
|
|
165
165
|
search;
|
|
166
166
|
account;
|
|
167
|
+
usage;
|
|
168
|
+
webhooks;
|
|
167
169
|
constructor(options) {
|
|
168
170
|
if (!options?.apiKey) {
|
|
169
|
-
throw new TypeError('
|
|
171
|
+
throw new TypeError('ByteKit: `apiKey` is required — got an empty value. Pass options.apiKey (e.g. from process.env.BYTEKIT_API_KEY).');
|
|
170
172
|
}
|
|
171
173
|
this.apiKey = options.apiKey;
|
|
172
|
-
|
|
174
|
+
// Normalize once: strip any trailing slash so `baseUrl + path` (path always
|
|
175
|
+
// starts with `/`) never yields a double-slash URL (#2481). Only the trailing
|
|
176
|
+
// slash is removed, so a path-carrying baseUrl (e.g. a reverse-proxy prefix
|
|
177
|
+
// `…/bytekit`) is preserved (E2).
|
|
178
|
+
this.baseUrl = (options.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, '');
|
|
173
179
|
this.defaultTimeoutMs = validateTimeout(options.timeoutMs, '"timeoutMs"') ?? DEFAULT_TIMEOUT_MS;
|
|
174
180
|
this.scrape = new ScrapeResource(this);
|
|
175
181
|
this.screenshots = new ScreenshotsResource(this);
|
|
176
|
-
this.recordings = new RecordingsResource(this);
|
|
177
182
|
this.bulk = new BulkResource(this);
|
|
178
183
|
this.fetch = new FetchResource(this);
|
|
179
184
|
this.monitors = new MonitorsResource(this);
|
|
180
185
|
this.sitemap = new SitemapResource(this);
|
|
181
186
|
this.search = new SearchResource(this);
|
|
182
187
|
this.account = new AccountResource(this);
|
|
188
|
+
this.usage = new UsageResource(this);
|
|
189
|
+
this.webhooks = new WebhooksResource(this);
|
|
183
190
|
}
|
|
184
191
|
/**
|
|
185
192
|
* Resolve the abort budget for one request from the caller's options and the
|
|
@@ -218,13 +225,19 @@ export class RapidCrawl {
|
|
|
218
225
|
};
|
|
219
226
|
}
|
|
220
227
|
/**
|
|
221
|
-
*
|
|
228
|
+
* Perform the authenticated round-trip and return the raw `Response`, having
|
|
229
|
+
* already thrown for the two non-body failure modes:
|
|
222
230
|
*
|
|
223
|
-
*
|
|
224
|
-
*
|
|
225
|
-
*
|
|
231
|
+
* - `ByteKitConnectionError` when no HTTP response was produced at all
|
|
232
|
+
* (DNS/socket/TLS failure, client-side timeout, or caller abort).
|
|
233
|
+
* - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
|
|
234
|
+
*
|
|
235
|
+
* The body is left unread so each caller can decode it as it needs — JSON for
|
|
236
|
+
* `request()`, raw text + headers for `requestRaw()`. This is the single place
|
|
237
|
+
* that owns auth, the abort budget, and the error taxonomy; the two public
|
|
238
|
+
* transports differ only in how they read a successful body.
|
|
226
239
|
*/
|
|
227
|
-
async
|
|
240
|
+
async send(method, path, body, options) {
|
|
228
241
|
const url = `${this.baseUrl}${path}`;
|
|
229
242
|
const { signal, timeoutSignal, cleanup } = this.resolveSignal(options);
|
|
230
243
|
const init = {
|
|
@@ -264,8 +277,55 @@ export class RapidCrawl {
|
|
|
264
277
|
catch {
|
|
265
278
|
// leave defaults
|
|
266
279
|
}
|
|
267
|
-
throw new
|
|
280
|
+
throw new ByteKitError(response.status, code, message);
|
|
281
|
+
}
|
|
282
|
+
return response;
|
|
283
|
+
}
|
|
284
|
+
/**
|
|
285
|
+
* Make an authenticated JSON request to the ByteKit API.
|
|
286
|
+
*
|
|
287
|
+
* Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
|
|
288
|
+
* when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
|
|
289
|
+
* timeout, or caller abort).
|
|
290
|
+
*/
|
|
291
|
+
async request(method, path, body, options) {
|
|
292
|
+
const response = await this.send(method, path, body, options);
|
|
293
|
+
// A 204 No Content has an empty body, and a genuinely empty body can arrive on
|
|
294
|
+
// other success codes too (defensive). Calling response.json() on an empty body
|
|
295
|
+
// rejects with a SyntaxError — which is how a successful revoke() (DELETE → 204)
|
|
296
|
+
// used to throw instead of resolving. Read the body once as text and short-circuit
|
|
297
|
+
// to `undefined` when it is empty; a JSON-bearing response (scrape.create,
|
|
298
|
+
// apiKeys.create/get, …) still parses and returns unchanged.
|
|
299
|
+
if (response.status === 204) {
|
|
300
|
+
return undefined;
|
|
268
301
|
}
|
|
269
|
-
|
|
302
|
+
const text = await response.text();
|
|
303
|
+
if (text.trim().length === 0) {
|
|
304
|
+
return undefined;
|
|
305
|
+
}
|
|
306
|
+
return JSON.parse(text);
|
|
307
|
+
}
|
|
308
|
+
/**
|
|
309
|
+
* Make an authenticated request whose response body IS the content, not a JSON
|
|
310
|
+
* envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
|
|
311
|
+
* response headers.
|
|
312
|
+
*
|
|
313
|
+
* Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
|
|
314
|
+
* response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
|
|
315
|
+
* deliberately a separate transport from `request()`: the generic JSON parse
|
|
316
|
+
* there is shared by every other resource and must not change (#2466).
|
|
317
|
+
*
|
|
318
|
+
* `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
|
|
319
|
+
* first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
|
|
320
|
+
* invalid byte sequence gets silently replaced with U+FFFD. Reading the body
|
|
321
|
+
* once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
|
|
322
|
+
* is always byte-exact, `body` is a best-effort string for callers who know the
|
|
323
|
+
* content is text.
|
|
324
|
+
*/
|
|
325
|
+
async requestRaw(method, path, body, options) {
|
|
326
|
+
const response = await this.send(method, path, body, options);
|
|
327
|
+
const bytes = new Uint8Array(await response.arrayBuffer());
|
|
328
|
+
const text = new TextDecoder('utf-8').decode(bytes);
|
|
329
|
+
return { body: text, bytes, headers: response.headers, status: response.status };
|
|
270
330
|
}
|
|
271
331
|
}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,8 +1,14 @@
|
|
|
1
|
-
export {
|
|
2
|
-
export type {
|
|
1
|
+
export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
|
|
2
|
+
export type { ByteKitOptions, RequestOptions, ConnectionErrorCode } from './client.js';
|
|
3
3
|
export { V01_METHODS } from './allowlist.js';
|
|
4
4
|
export type { V01Method } from './allowlist.js';
|
|
5
|
-
export type { ScrapeId, ScreenshotId,
|
|
5
|
+
export type { ScrapeId, ScreenshotId, BulkId, MonitorId, SitemapId, ApiKeyId, WebhookDeliveryId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
|
|
6
6
|
export type { ScrapeCreateResponse, ScrapeGetResponse, ScrapeBulkCreateRequest, ScrapeBulkCreateResponse, ScrapeBulkGetResponse, } from './resources/scrape.js';
|
|
7
7
|
export type { ScreenshotCreateRequest, ScreenshotCreateResponse, ScreenshotGetResponse, } from './resources/screenshots.js';
|
|
8
|
+
export type { FetchOpts, FetchGetOpts, FetchResult, FetchCacheStatus, FetchBulkCreateRequest, FetchBulkCreateResponse, FetchBulkGetResponse, } from './resources/fetch.js';
|
|
9
|
+
export type { MonitorCreateRequest, MonitorCreateResponse, ListMonitorsParams, ListMonitorsResponse, MonitorResponse, MonitorUpdateRequest, UpdateMonitorResponse, ListMonitorCapturesResponse, } from './resources/monitors.js';
|
|
10
|
+
export type { SitemapCreateRequest, SitemapCreateResponse, SitemapGetResponse, } from './resources/sitemap.js';
|
|
11
|
+
export type { BulkCreateRequest, BulkCreateResponse, BulkGetResponse, BulkCancelResponse, ListBulkScreenshotsResponse, } from './resources/bulk.js';
|
|
8
12
|
export type { GetAccountResponse, ListApiKeysResponse, CreateApiKeyRequest, CreateApiKeyResponse, GetApiKeyResponse, UpdateApiKeyRequest, UpdateApiKeyResponse, RevealApiKeyResponse, } from './resources/account.js';
|
|
13
|
+
export type { GetUsageResponse, GetUsageDailyParams, GetUsageDailyResponse, GetUsageByEndpointParams, GetUsageByEndpointResponse, } from './resources/usage.js';
|
|
14
|
+
export type { ListWebhookDeliveriesParams, ListWebhookDeliveriesResponse, RetryWebhookDeliveryResponse, } from './resources/webhooks.js';
|
package/dist/index.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export {
|
|
1
|
+
export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
|
|
2
2
|
export { V01_METHODS } from './allowlist.js';
|