@hunt-labs/bytekit-sdk 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,142 @@
1
+ # Changelog
2
+
3
+ All notable changes to `@hunt-labs/bytekit-sdk` are documented here.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ This file records only the delta between versions. For how to use the package
9
+ today, see [README.md](./README.md), which remains the authoritative usage guide.
10
+
11
+ ## [0.3.0] - 2026-07-21
12
+
13
+ ### Removed (**BREAKING**)
14
+
15
+ - The deprecated `RapidCrawl`, `RapidCrawlError`, and `RapidCrawlConnectionError`
16
+ exports and the `RapidCrawlOptions` type alias (kept as a one-minor-version
17
+ bridge in #2421) are removed. Import the `ByteKit*` names instead. (#2473)
18
+ - The `RecordingId` type export is removed. Recordings are deactivated
19
+ (`/v1/recordings` returns 410), so the branded id had no reachable use.
20
+ Breaking in theory only — unused in practice. (#2481)
21
+
22
+ ### Fixed
23
+
24
+ - `fetch.create` / `fetch.get` no longer throw a bare `SyntaxError` on every
25
+ non-JSON body. `/v1/fetch` returns the fetched content AS the response body
26
+ (metadata in `X-Fetch-*` headers), but both methods routed through the
27
+ JSON-parsing transport. They now use a raw-capable transport and resolve to a
28
+ typed `FetchResult` — raw `body`, byte-exact `bytes`, plus `id`,
29
+ `creditsCharged`, `cache`, `custom`, `contentType`, and `headers` parsed from
30
+ the `X-Fetch-*` metadata. (#2466)
31
+ - A trailing slash on the constructor `baseUrl` (e.g. `"https://api.bytekit.com/"`)
32
+ no longer produces double-slash request URLs — it is normalized once, preserving
33
+ a path-carrying baseUrl. (#2481)
34
+ - `SearchResult.date` is now optional (`string | null | undefined`): `images`
35
+ search results omit the field entirely. (#2481)
36
+
37
+ ### Added
38
+
39
+ - New `usage` resource (`get`, `daily`, `byEndpoint` — `GET /v1/usage`,
40
+ `/v1/usage/daily`, `/v1/usage/by-endpoint`) and `webhooks` resource (`list`,
41
+ `retry` — `GET /v1/webhook-deliveries`, `POST /v1/webhook-deliveries/{id}/retry`),
42
+ with their request/response types exported. (#2471)
43
+ - `ScrapeOpts` and `FetchOpts` now cover the full documented request field set
44
+ (10 previously missing `ScrapeRequest` fields; 11 markdown-tuning
45
+ `FetchRequest` fields), plus the `'text'` markdown-links mode. A compile-time
46
+ drift guard keeps them aligned with the generated OpenAPI schema. (#2475)
47
+ - `monitors`, `sitemap`, `bulk`, and `fetch.bulk` methods are now fully typed from
48
+ the generated OpenAPI `operations` (previously `Record<string, unknown>` in /
49
+ `Promise<unknown>` out); their request/response types are re-exported. (#2481)
50
+ - `monitors.list` accepts typed query params and returns a typed response
51
+ (`ListMonitorsParams` / `ListMonitorsResponse`, both exported). (#2480)
52
+
53
+ ### Changed (**BREAKING**)
54
+
55
+ - `fetch.get` now rejects a `custom` option at compile time: `GET /v1/fetch` has
56
+ no `custom` query param, so the value was silently dropped before. Use
57
+ `fetch.create` (POST) to echo a `custom` payload. (#2481)
58
+ - `FetchOpts.format` is narrowed from `string` to the spec union
59
+ `'markdown' | 'html'`; off-spec values now fail to compile. (#2475)
60
+
61
+ ## [0.2.2] - 2026-07-16
62
+
63
+ ### Changed
64
+
65
+ - Re-release of 0.2.1. The 0.2.1 artifacts never reached npm — the prod publish's
66
+ `publish-sdk` step failed on a CI test-isolation flake (`cjs-require.test.ts`
67
+ rebuilding the shared `dist`), which is now fixed. No consumer-facing changes
68
+ beyond 0.2.1.
69
+
70
+ ## [0.2.1] - 2026-07-16
71
+
72
+ ### Changed
73
+
74
+ - Primary exports renamed `RapidCrawl`→`ByteKit`, `RapidCrawlError`→`ByteKitError`,
75
+ `RapidCrawlOptions`→`ByteKitOptions`. The old names remain as deprecated,
76
+ identity-preserving aliases (`RapidCrawl === ByteKit`), so existing imports keep
77
+ working. (#2421)
78
+
79
+ ### Fixed
80
+
81
+ - `request()` now returns `undefined` on `204 No Content` / empty-body responses
82
+ instead of throwing a JSON-parse error — `account.apiKeys.revoke()` now resolves
83
+ as its `Promise<void>` contract declares. (#2450)
84
+
85
+ ### Removed
86
+
87
+ - `client.recordings` (the `/v1/recordings` endpoint is permanently `410`
88
+ deactivated). (#2421)
89
+
90
+ ## [0.2.0] - 2026-07-15
91
+
92
+ This is a breaking release. Four runtime behavior changes affect code written
93
+ against 0.1.0 — read each one before upgrading.
94
+
95
+ ### Changed (breaking)
96
+
97
+ - **Default client-side timeout (120s).** Requests now abort after 120000 ms by
98
+ default (`DEFAULT_TIMEOUT_MS` in `src/client.ts`); 0.1.0 had no client-side
99
+ timeout and would hang indefinitely. A slow request now rejects with
100
+ `RapidCrawlConnectionError` carrying `code: 'timeout'`. Override per client via
101
+ `new RapidCrawl({ apiKey, timeoutMs })`.
102
+ - **Network failures throw `RapidCrawlConnectionError`.** Every `fetch()` rejection
103
+ is now wrapped by `toConnectionError()` and re-thrown as `RapidCrawlConnectionError`.
104
+ 0.1.0 surfaced undici's bare `TypeError: fetch failed`. Code that matched on a bare
105
+ `TypeError` to detect a network failure will no longer match — catch
106
+ `RapidCrawlConnectionError` instead.
107
+ - **Constructor throws on an empty `apiKey`.** `new RapidCrawl({ apiKey: '' })` (or a
108
+ missing key) now throws a `TypeError` immediately instead of silently sending
109
+ `Authorization: Bearer undefined`. Ensure `apiKey` is populated (e.g. from
110
+ `process.env.BYTEKIT_API_KEY`) before constructing the client.
111
+ - **Request-body types narrowed to the OpenAPI schema.** Request payloads are now
112
+ typed from spec-derived types generated from the OpenAPI schema
113
+ (`src/generated/api.d.ts`) rather than `Record<string, unknown>`. Payloads that
114
+ previously compiled by virtue of the loose index type may now be rejected by the
115
+ type checker; align them with the documented request shape.
116
+
117
+ ### Changed
118
+
119
+ - Package renamed to `@hunt-labs/bytekit-sdk` (the `@bytekit` npm scope was
120
+ unobtainable). The prior `rapidcrawl`/`@rapidcrawl/sdk` name remains installable at
121
+ its last version but is deprecated.
122
+ - Generated API types (`dist/generated/api.d.ts`) are now shipped in the tarball so
123
+ consumers get the typed request/response surface without regenerating from the spec.
124
+
125
+ ### Added
126
+
127
+ - Publish-ready packaging metadata: types-first `exports`, `engines`, `keywords`,
128
+ `homepage`, `bugs`, `README.md`, `LICENSE`, and an MIT `license` field.
129
+
130
+ ## [0.1.0] - 2026
131
+
132
+ ### Added
133
+
134
+ - Initial release of the ByteKit TypeScript SDK — a typed client for the ByteKit API
135
+ (scrape, screenshot, search, and related endpoints), published under the original
136
+ `@rapidcrawl/sdk` name.
137
+
138
+ [0.3.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
139
+ [0.2.2]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
140
+ [0.2.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
141
+ [0.2.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
142
+ [0.1.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
package/README.md CHANGED
@@ -10,11 +10,14 @@ screenshots, sitemaps, page monitors, and web search.
10
10
  npm install @hunt-labs/bytekit-sdk
11
11
  ```
12
12
 
13
- > **ESM-only.** This package ships as ESM (`"type": "module"`) and has no
14
- > CommonJS build. Use it from an ESM project (`"type": "module"` in your
15
- > `package.json`, or a `.mjs` entrypoint), or load it from CommonJS with a
16
- > dynamic `await import('@hunt-labs/bytekit-sdk')`. A top-level
17
- > `require('@hunt-labs/bytekit-sdk')` will not work.
13
+ > **ESM-only.** This package ships a single ESM build (`"type": "module"`) with
14
+ > no separate CommonJS bundle. Use it from an ESM project (`"type": "module"` in
15
+ > your `package.json`, or a `.mjs` entrypoint). From CommonJS you can either load
16
+ > it with a dynamic `await import('@hunt-labs/bytekit-sdk')`, or — on **Node
17
+ > ≥20.19 / ≥22.12**, which resolve `require()` of ESM — call a top-level
18
+ > `require('@hunt-labs/bytekit-sdk')` directly (the package's `exports` map
19
+ > carries a `default` condition for exactly this). On older Node, a top-level
20
+ > `require()` throws `ERR_REQUIRE_ESM`; use the dynamic `import()` there.
18
21
 
19
22
  Requires Node.js 18+ (the client uses the global `fetch`).
20
23
 
@@ -24,16 +27,15 @@ Create an API key at [app.bytekit.com](https://app.bytekit.com) (keys are
24
27
  prefixed `sk_live_`) and pass it to the client constructor:
25
28
 
26
29
  ```ts
27
- import { RapidCrawl } from '@hunt-labs/bytekit-sdk';
30
+ import { ByteKit } from '@hunt-labs/bytekit-sdk';
28
31
 
29
- const client = new RapidCrawl({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...
32
+ const client = new ByteKit({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...
30
33
  ```
31
34
 
32
35
  The key is sent as `Authorization: Bearer <apiKey>` on every request. Keep it
33
36
  server-side — it is not safe to ship in a browser bundle.
34
37
 
35
- > The client class is named `RapidCrawl` for historical reasons (ByteKit's
36
- > previous product name). It is the SDK's only entrypoint class.
38
+ > `ByteKit` is the SDK's only entrypoint class.
37
39
 
38
40
  ## Base-URL override
39
41
 
@@ -41,7 +43,7 @@ Requests go to `https://api.bytekit.com` by default. Point the client at
41
43
  staging, a local gateway, or a proxy with the `baseUrl` option:
42
44
 
43
45
  ```ts
44
- const client = new RapidCrawl({
46
+ const client = new ByteKit({
45
47
  apiKey: process.env.BYTEKIT_API_KEY!,
46
48
  baseUrl: 'https://api-stg.bytekit.com', // default: https://api.bytekit.com
47
49
  });
@@ -66,8 +68,27 @@ const scrape = await client.scrape.get('sc_123abc');
66
68
 
67
69
  ### Raw fetch (unprocessed response body)
68
70
 
71
+ `/v1/fetch` returns the fetched content **as the body**, with metadata in
72
+ `X-Fetch-*` response headers. `fetch.create` / `fetch.get` resolve a
73
+ `FetchResult` — `body` is the raw content string (a JSON-serving upstream
74
+ comes back as the raw JSON string, never a parsed object), and the named
75
+ fields decode the metadata headers. `body` is UTF-8-decoded, so for true
76
+ binary content (images, PDFs, etc. — `/v1/fetch` supports raw passthrough)
77
+ use `bytes` instead, which is byte-for-byte identical to the wire content:
78
+
69
79
  ```ts
70
- const raw = await client.fetch.create({ url: 'https://example.com' });
80
+ const res = await client.fetch.create({ url: 'https://example.com' });
81
+
82
+ console.log(res.body); // raw content (e.g. "<!doctype html>…") — UTF-8 decoded
83
+ console.log(res.bytes); // Uint8Array — byte-exact, safe for binary content
84
+ console.log(res.id); // X-Fetch-ID, e.g. "ft_0123abcd…"
85
+ console.log(res.creditsCharged); // X-Fetch-Credits-Charged, as a number
86
+ console.log(res.cache); // "hit" | "miss" | "bypass"
87
+ console.log(res.custom); // base64 X-Fetch-Custom echo, or null
88
+ console.log(res.contentType); // response Content-Type, e.g. "text/html"
89
+
90
+ // Anything not surfaced as a named field is on res.headers:
91
+ console.log(res.headers.get('X-Fetch-Waf-Vendor'));
71
92
  ```
72
93
 
73
94
  ### Web search
@@ -121,7 +142,10 @@ await client.bulk.screenshots.list('bulk_123abc');
121
142
  ```ts
122
143
  const monitor = await client.monitors.create({
123
144
  url: 'https://example.com/pricing',
145
+ type: 'screenshot',
124
146
  interval_type: 'daily',
147
+ change_threshold: 5,
148
+ notify_on: 'change',
125
149
  webhook_url: 'https://your.app/hooks/bytekit',
126
150
  });
127
151
  await client.monitors.list();
@@ -131,7 +155,16 @@ await client.monitors.delete('mon_123abc');
131
155
  ### Sitemap crawl
132
156
 
133
157
  ```ts
134
- const crawl = await client.sitemap.create({ url: 'https://example.com' });
158
+ const crawl = await client.sitemap.create({
159
+ url: 'https://example.com',
160
+ strategy: 'standard',
161
+ max_depth: 2,
162
+ max_urls: 5000,
163
+ process: null,
164
+ webhook_secret: null,
165
+ cache_ttl: '30d',
166
+ compact: false,
167
+ });
135
168
  const sitemap = await client.sitemap.get('sm_123abc');
136
169
  ```
137
170
 
@@ -140,23 +173,36 @@ const sitemap = await client.sitemap.get('sm_123abc');
140
173
  ```ts
141
174
  const account = await client.account.get();
142
175
  const keys = await client.account.apiKeys.list();
176
+
177
+ // Usage (Bearer API key auth)
178
+ const usage = await client.usage.get();
179
+ const daily = await client.usage.daily({ from: '2026-01-01', to: '2026-01-31' });
180
+ const byEndpoint = await client.usage.byEndpoint();
181
+
182
+ // Webhook deliveries
183
+ const deliveries = await client.webhooks.list();
184
+ await client.webhooks.retry('whd_123abc');
143
185
  ```
144
186
 
145
- > `client.recordings.*` targets the `/v1/recordings` endpoints, which are
146
- > deactivated server-side and return HTTP 410.
187
+ > **Note:** `client.account.*` (including `client.account.apiKeys.*`) targets the
188
+ > `/v1/account/*` endpoints, which authenticate with a Clerk dashboard session —
189
+ > **not** a Bearer API key. Calling them with an API-key client will fail auth.
190
+ > They remain on the SDK surface for dashboard/session consumers and are marked
191
+ > `x-sdk-scope: internal` in the spec, so they are intentionally excluded from the
192
+ > v0.1 Bearer-key parity guarantee.
147
193
 
148
194
  ## Error handling
149
195
 
150
- Any 4xx/5xx response throws a `RapidCrawlError` carrying the HTTP status and
196
+ Any 4xx/5xx response throws a `ByteKitError` carrying the HTTP status and
151
197
  the API's error code:
152
198
 
153
199
  ```ts
154
- import { RapidCrawl, RapidCrawlError } from '@hunt-labs/bytekit-sdk';
200
+ import { ByteKit, ByteKitError } from '@hunt-labs/bytekit-sdk';
155
201
 
156
202
  try {
157
203
  await client.scrape.create({ url: 'https://example.com' });
158
204
  } catch (err) {
159
- if (err instanceof RapidCrawlError) {
205
+ if (err instanceof ByteKitError) {
160
206
  console.error(err.status, err.code, err.message);
161
207
  }
162
208
  }
@@ -8,5 +8,5 @@
8
8
  *
9
9
  * To add a method: annotate the operation in the spec first, then add it here.
10
10
  */
11
- export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get"];
11
+ export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get", "usage.get", "usage.daily", "usage.byEndpoint", "webhooks.list", "webhooks.retry"];
12
12
  export type V01Method = (typeof V01_METHODS)[number];
package/dist/allowlist.js CHANGED
@@ -34,4 +34,11 @@ export const V01_METHODS = [
34
34
  'sitemap.get',
35
35
  'search.query',
36
36
  'account.get',
37
+ // usage + webhook-delivery ops — resource surface shipped in #2471, brought
38
+ // under the parity guard in #2476.
39
+ 'usage.get',
40
+ 'usage.daily',
41
+ 'usage.byEndpoint',
42
+ 'webhooks.list',
43
+ 'webhooks.retry',
37
44
  ];
package/dist/client.d.ts CHANGED
@@ -1,13 +1,14 @@
1
1
  import { ScrapeResource } from './resources/scrape.js';
2
2
  import { ScreenshotsResource } from './resources/screenshots.js';
3
- import { RecordingsResource } from './resources/recordings.js';
4
3
  import { BulkResource } from './resources/bulk.js';
5
4
  import { FetchResource } from './resources/fetch.js';
6
5
  import { MonitorsResource } from './resources/monitors.js';
7
6
  import { SitemapResource } from './resources/sitemap.js';
8
7
  import { SearchResource } from './resources/search.js';
9
8
  import { AccountResource } from './resources/account.js';
10
- export interface RapidCrawlOptions {
9
+ import { UsageResource } from './resources/usage.js';
10
+ import { WebhooksResource } from './resources/webhooks.js';
11
+ export interface ByteKitOptions {
11
12
  apiKey: string;
12
13
  baseUrl?: string;
13
14
  /**
@@ -42,38 +43,39 @@ export interface RequestOptions {
42
43
  /** Why a request never produced an HTTP response. */
43
44
  export type ConnectionErrorCode = 'timeout' | 'aborted' | 'connection';
44
45
  /** Thrown when the API responds with a 4xx or 5xx status code. */
45
- export declare class RapidCrawlError extends Error {
46
+ export declare class ByteKitError extends Error {
46
47
  readonly status: number;
47
48
  readonly code: string;
48
49
  constructor(status: number, code: string, message: string);
49
50
  }
50
51
  /**
51
52
  * Thrown when the request never produced an HTTP response at all — DNS, socket, TLS,
52
- * a client-side timeout, or a caller-initiated abort. Distinct from `RapidCrawlError`,
53
+ * a client-side timeout, or a caller-initiated abort. Distinct from `ByteKitError`,
53
54
  * which always carries a real HTTP status: catch both to cover every failure.
54
55
  *
55
56
  * The original failure is preserved on `.cause`.
56
57
  */
57
- export declare class RapidCrawlConnectionError extends Error {
58
+ export declare class ByteKitConnectionError extends Error {
58
59
  readonly code: ConnectionErrorCode;
59
60
  constructor(code: ConnectionErrorCode, message: string, options?: {
60
61
  cause?: unknown;
61
62
  });
62
63
  }
63
- export declare class RapidCrawl {
64
+ export declare class ByteKit {
64
65
  private readonly apiKey;
65
66
  private readonly baseUrl;
66
67
  private readonly defaultTimeoutMs;
67
68
  readonly scrape: ScrapeResource;
68
69
  readonly screenshots: ScreenshotsResource;
69
- readonly recordings: RecordingsResource;
70
70
  readonly bulk: BulkResource;
71
71
  readonly fetch: FetchResource;
72
72
  readonly monitors: MonitorsResource;
73
73
  readonly sitemap: SitemapResource;
74
74
  readonly search: SearchResource;
75
75
  readonly account: AccountResource;
76
- constructor(options: RapidCrawlOptions);
76
+ readonly usage: UsageResource;
77
+ readonly webhooks: WebhooksResource;
78
+ constructor(options: ByteKitOptions);
77
79
  /**
78
80
  * Resolve the abort budget for one request from the caller's options and the
79
81
  * constructor default. Precedence:
@@ -89,11 +91,48 @@ export declare class RapidCrawl {
89
91
  */
90
92
  private resolveSignal;
91
93
  /**
92
- * Make an authenticated JSON request to the RapidCrawl API.
94
+ * Perform the authenticated round-trip and return the raw `Response`, having
95
+ * already thrown for the two non-body failure modes:
93
96
  *
94
- * Throws `RapidCrawlError` on 4xx/5xx responses, and `RapidCrawlConnectionError`
97
+ * - `ByteKitConnectionError` when no HTTP response was produced at all
98
+ * (DNS/socket/TLS failure, client-side timeout, or caller abort).
99
+ * - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
100
+ *
101
+ * The body is left unread so each caller can decode it as it needs — JSON for
102
+ * `request()`, raw text + headers for `requestRaw()`. This is the single place
103
+ * that owns auth, the abort budget, and the error taxonomy; the two public
104
+ * transports differ only in how they read a successful body.
105
+ */
106
+ private send;
107
+ /**
108
+ * Make an authenticated JSON request to the ByteKit API.
109
+ *
110
+ * Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
95
111
  * when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
96
112
  * timeout, or caller abort).
97
113
  */
98
114
  request<T>(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<T>;
115
+ /**
116
+ * Make an authenticated request whose response body IS the content, not a JSON
117
+ * envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
118
+ * response headers.
119
+ *
120
+ * Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
121
+ * response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
122
+ * deliberately a separate transport from `request()`: the generic JSON parse
123
+ * there is shared by every other resource and must not change (#2466).
124
+ *
125
+ * `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
126
+ * first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
127
+ * invalid byte sequence gets silently replaced with U+FFFD. Reading the body
128
+ * once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
129
+ * is always byte-exact, `body` is a best-effort string for callers who know the
130
+ * content is text.
131
+ */
132
+ requestRaw(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<{
133
+ body: string;
134
+ bytes: Uint8Array;
135
+ headers: Headers;
136
+ status: number;
137
+ }>;
99
138
  }
package/dist/client.js CHANGED
@@ -1,12 +1,13 @@
1
1
  import { ScrapeResource } from './resources/scrape.js';
2
2
  import { ScreenshotsResource } from './resources/screenshots.js';
3
- import { RecordingsResource } from './resources/recordings.js';
4
3
  import { BulkResource } from './resources/bulk.js';
5
4
  import { FetchResource } from './resources/fetch.js';
6
5
  import { MonitorsResource } from './resources/monitors.js';
7
6
  import { SitemapResource } from './resources/sitemap.js';
8
7
  import { SearchResource } from './resources/search.js';
9
8
  import { AccountResource } from './resources/account.js';
9
+ import { UsageResource } from './resources/usage.js';
10
+ import { WebhooksResource } from './resources/webhooks.js';
10
11
  const DEFAULT_BASE_URL = 'https://api.bytekit.com';
11
12
  /**
12
13
  * Default client-side request timeout (2 minutes).
@@ -17,28 +18,28 @@ const DEFAULT_BASE_URL = 'https://api.bytekit.com';
17
18
  */
18
19
  const DEFAULT_TIMEOUT_MS = 120_000;
19
20
  /** Thrown when the API responds with a 4xx or 5xx status code. */
20
- export class RapidCrawlError extends Error {
21
+ export class ByteKitError extends Error {
21
22
  status;
22
23
  code;
23
24
  constructor(status, code, message) {
24
25
  super(message);
25
- this.name = 'RapidCrawlError';
26
+ this.name = 'ByteKitError';
26
27
  this.status = status;
27
28
  this.code = code;
28
29
  }
29
30
  }
30
31
  /**
31
32
  * Thrown when the request never produced an HTTP response at all — DNS, socket, TLS,
32
- * a client-side timeout, or a caller-initiated abort. Distinct from `RapidCrawlError`,
33
+ * a client-side timeout, or a caller-initiated abort. Distinct from `ByteKitError`,
33
34
  * which always carries a real HTTP status: catch both to cover every failure.
34
35
  *
35
36
  * The original failure is preserved on `.cause`.
36
37
  */
37
- export class RapidCrawlConnectionError extends Error {
38
+ export class ByteKitConnectionError extends Error {
38
39
  code;
39
40
  constructor(code, message, options) {
40
41
  super(message, options);
41
- this.name = 'RapidCrawlConnectionError';
42
+ this.name = 'ByteKitConnectionError';
42
43
  this.code = code;
43
44
  }
44
45
  }
@@ -52,7 +53,7 @@ function validateTimeout(value, optionName) {
52
53
  if (value === undefined)
53
54
  return undefined;
54
55
  if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) {
55
- throw new TypeError(`RapidCrawl: ${optionName} must be a non-negative finite number (got ${describe(value)})`);
56
+ throw new TypeError(`ByteKit: ${optionName} must be a non-negative finite number (got ${describe(value)})`);
56
57
  }
57
58
  return value;
58
59
  }
@@ -69,7 +70,7 @@ function describe(value) {
69
70
  }
70
71
  /** The abort reason used by the `setTimeout` fallback; named so it maps to `code: 'timeout'`. */
71
72
  function timeoutReason(ms) {
72
- const reason = new Error(`RapidCrawl: request timed out after ${ms}ms`);
73
+ const reason = new Error(`ByteKit: request timed out after ${ms}ms`);
73
74
  reason.name = 'TimeoutError';
74
75
  return reason;
75
76
  }
@@ -147,39 +148,45 @@ function toConnectionError(err, method, url, timeoutSignal) {
147
148
  : code === 'aborted'
148
149
  ? 'request aborted by the caller'
149
150
  : 'connection failed before a response was received';
150
- return new RapidCrawlConnectionError(code, `RapidCrawl: ${method} ${url} — ${detail}`, {
151
+ return new ByteKitConnectionError(code, `ByteKit: ${method} ${url} — ${detail}`, {
151
152
  cause: err,
152
153
  });
153
154
  }
154
- export class RapidCrawl {
155
+ export class ByteKit {
155
156
  apiKey;
156
157
  baseUrl;
157
158
  defaultTimeoutMs;
158
159
  scrape;
159
160
  screenshots;
160
- recordings;
161
161
  bulk;
162
162
  fetch;
163
163
  monitors;
164
164
  sitemap;
165
165
  search;
166
166
  account;
167
+ usage;
168
+ webhooks;
167
169
  constructor(options) {
168
170
  if (!options?.apiKey) {
169
- throw new TypeError('RapidCrawl: `apiKey` is required — got an empty value. Pass options.apiKey (e.g. from process.env.BYTEKIT_API_KEY).');
171
+ throw new TypeError('ByteKit: `apiKey` is required — got an empty value. Pass options.apiKey (e.g. from process.env.BYTEKIT_API_KEY).');
170
172
  }
171
173
  this.apiKey = options.apiKey;
172
- this.baseUrl = options.baseUrl ?? DEFAULT_BASE_URL;
174
+ // Normalize once: strip any trailing slash so `baseUrl + path` (path always
175
+ // starts with `/`) never yields a double-slash URL (#2481). Only the trailing
176
+ // slash is removed, so a path-carrying baseUrl (e.g. a reverse-proxy prefix
177
+ // `…/bytekit`) is preserved (E2).
178
+ this.baseUrl = (options.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, '');
173
179
  this.defaultTimeoutMs = validateTimeout(options.timeoutMs, '"timeoutMs"') ?? DEFAULT_TIMEOUT_MS;
174
180
  this.scrape = new ScrapeResource(this);
175
181
  this.screenshots = new ScreenshotsResource(this);
176
- this.recordings = new RecordingsResource(this);
177
182
  this.bulk = new BulkResource(this);
178
183
  this.fetch = new FetchResource(this);
179
184
  this.monitors = new MonitorsResource(this);
180
185
  this.sitemap = new SitemapResource(this);
181
186
  this.search = new SearchResource(this);
182
187
  this.account = new AccountResource(this);
188
+ this.usage = new UsageResource(this);
189
+ this.webhooks = new WebhooksResource(this);
183
190
  }
184
191
  /**
185
192
  * Resolve the abort budget for one request from the caller's options and the
@@ -218,13 +225,19 @@ export class RapidCrawl {
218
225
  };
219
226
  }
220
227
  /**
221
- * Make an authenticated JSON request to the RapidCrawl API.
228
+ * Perform the authenticated round-trip and return the raw `Response`, having
229
+ * already thrown for the two non-body failure modes:
222
230
  *
223
- * Throws `RapidCrawlError` on 4xx/5xx responses, and `RapidCrawlConnectionError`
224
- * when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
225
- * timeout, or caller abort).
231
+ * - `ByteKitConnectionError` when no HTTP response was produced at all
232
+ * (DNS/socket/TLS failure, client-side timeout, or caller abort).
233
+ * - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
234
+ *
235
+ * The body is left unread so each caller can decode it as it needs — JSON for
236
+ * `request()`, raw text + headers for `requestRaw()`. This is the single place
237
+ * that owns auth, the abort budget, and the error taxonomy; the two public
238
+ * transports differ only in how they read a successful body.
226
239
  */
227
- async request(method, path, body, options) {
240
+ async send(method, path, body, options) {
228
241
  const url = `${this.baseUrl}${path}`;
229
242
  const { signal, timeoutSignal, cleanup } = this.resolveSignal(options);
230
243
  const init = {
@@ -264,8 +277,55 @@ export class RapidCrawl {
264
277
  catch {
265
278
  // leave defaults
266
279
  }
267
- throw new RapidCrawlError(response.status, code, message);
280
+ throw new ByteKitError(response.status, code, message);
281
+ }
282
+ return response;
283
+ }
284
+ /**
285
+ * Make an authenticated JSON request to the ByteKit API.
286
+ *
287
+ * Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
288
+ * when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
289
+ * timeout, or caller abort).
290
+ */
291
+ async request(method, path, body, options) {
292
+ const response = await this.send(method, path, body, options);
293
+ // A 204 No Content has an empty body, and a genuinely empty body can arrive on
294
+ // other success codes too (defensive). Calling response.json() on an empty body
295
+ // rejects with a SyntaxError — which is how a successful revoke() (DELETE → 204)
296
+ // used to throw instead of resolving. Read the body once as text and short-circuit
297
+ // to `undefined` when it is empty; a JSON-bearing response (scrape.create,
298
+ // apiKeys.create/get, …) still parses and returns unchanged.
299
+ if (response.status === 204) {
300
+ return undefined;
268
301
  }
269
- return response.json();
302
+ const text = await response.text();
303
+ if (text.trim().length === 0) {
304
+ return undefined;
305
+ }
306
+ return JSON.parse(text);
307
+ }
308
+ /**
309
+ * Make an authenticated request whose response body IS the content, not a JSON
310
+ * envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
311
+ * response headers.
312
+ *
313
+ * Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
314
+ * response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
315
+ * deliberately a separate transport from `request()`: the generic JSON parse
316
+ * there is shared by every other resource and must not change (#2466).
317
+ *
318
+ * `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
319
+ * first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
320
+ * invalid byte sequence gets silently replaced with U+FFFD. Reading the body
321
+ * once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
322
+ * is always byte-exact, `body` is a best-effort string for callers who know the
323
+ * content is text.
324
+ */
325
+ async requestRaw(method, path, body, options) {
326
+ const response = await this.send(method, path, body, options);
327
+ const bytes = new Uint8Array(await response.arrayBuffer());
328
+ const text = new TextDecoder('utf-8').decode(bytes);
329
+ return { body: text, bytes, headers: response.headers, status: response.status };
270
330
  }
271
331
  }
package/dist/index.d.ts CHANGED
@@ -1,8 +1,14 @@
1
- export { RapidCrawl, RapidCrawlError, RapidCrawlConnectionError } from './client.js';
2
- export type { RapidCrawlOptions, RequestOptions, ConnectionErrorCode } from './client.js';
1
+ export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
2
+ export type { ByteKitOptions, RequestOptions, ConnectionErrorCode } from './client.js';
3
3
  export { V01_METHODS } from './allowlist.js';
4
4
  export type { V01Method } from './allowlist.js';
5
- export type { ScrapeId, ScreenshotId, RecordingId, BulkId, MonitorId, SitemapId, ApiKeyId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
5
+ export type { ScrapeId, ScreenshotId, BulkId, MonitorId, SitemapId, ApiKeyId, WebhookDeliveryId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
6
6
  export type { ScrapeCreateResponse, ScrapeGetResponse, ScrapeBulkCreateRequest, ScrapeBulkCreateResponse, ScrapeBulkGetResponse, } from './resources/scrape.js';
7
7
  export type { ScreenshotCreateRequest, ScreenshotCreateResponse, ScreenshotGetResponse, } from './resources/screenshots.js';
8
+ export type { FetchOpts, FetchGetOpts, FetchResult, FetchCacheStatus, FetchBulkCreateRequest, FetchBulkCreateResponse, FetchBulkGetResponse, } from './resources/fetch.js';
9
+ export type { MonitorCreateRequest, MonitorCreateResponse, ListMonitorsParams, ListMonitorsResponse, MonitorResponse, MonitorUpdateRequest, UpdateMonitorResponse, ListMonitorCapturesResponse, } from './resources/monitors.js';
10
+ export type { SitemapCreateRequest, SitemapCreateResponse, SitemapGetResponse, } from './resources/sitemap.js';
11
+ export type { BulkCreateRequest, BulkCreateResponse, BulkGetResponse, BulkCancelResponse, ListBulkScreenshotsResponse, } from './resources/bulk.js';
8
12
  export type { GetAccountResponse, ListApiKeysResponse, CreateApiKeyRequest, CreateApiKeyResponse, GetApiKeyResponse, UpdateApiKeyRequest, UpdateApiKeyResponse, RevealApiKeyResponse, } from './resources/account.js';
13
+ export type { GetUsageResponse, GetUsageDailyParams, GetUsageDailyResponse, GetUsageByEndpointParams, GetUsageByEndpointResponse, } from './resources/usage.js';
14
+ export type { ListWebhookDeliveriesParams, ListWebhookDeliveriesResponse, RetryWebhookDeliveryResponse, } from './resources/webhooks.js';
package/dist/index.js CHANGED
@@ -1,2 +1,2 @@
1
- export { RapidCrawl, RapidCrawlError, RapidCrawlConnectionError } from './client.js';
1
+ export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
2
2
  export { V01_METHODS } from './allowlist.js';