@hunt-labs/bytekit-sdk 0.2.2 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,56 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
8
8
  This file records only the delta between versions. For how to use the package
9
9
  today, see [README.md](./README.md), which remains the authoritative usage guide.
10
10
 
11
+ ## [0.3.0] - 2026-07-21
12
+
13
+ ### Removed (**BREAKING**)
14
+
15
+ - The deprecated `RapidCrawl`, `RapidCrawlError`, and `RapidCrawlConnectionError`
16
+ exports and the `RapidCrawlOptions` type alias (kept as a one-minor-version
17
+ bridge in #2421) are removed. Import the `ByteKit*` names instead. (#2473)
18
+ - The `RecordingId` type export is removed. Recordings are deactivated
19
+ (`/v1/recordings` returns 410), so the branded id had no reachable use.
20
+ Breaking in theory only — unused in practice. (#2481)
21
+
22
+ ### Fixed
23
+
24
+ - `fetch.create` / `fetch.get` no longer throw a bare `SyntaxError` on every
25
+ non-JSON body. `/v1/fetch` returns the fetched content AS the response body
26
+ (metadata in `X-Fetch-*` headers), but both methods routed through the
27
+ JSON-parsing transport. They now use a raw-capable transport and resolve to a
28
+ typed `FetchResult` — raw `body`, byte-exact `bytes`, plus `id`,
29
+ `creditsCharged`, `cache`, `custom`, `contentType`, and `headers` parsed from
30
+ the `X-Fetch-*` metadata. (#2466)
31
+ - A trailing slash on the constructor `baseUrl` (e.g. `"https://api.bytekit.com/"`)
32
+ no longer produces double-slash request URLs — it is normalized once, preserving
33
+ a path-carrying baseUrl. (#2481)
34
+ - `SearchResult.date` is now optional (`string | null | undefined`): `images`
35
+ search results omit the field entirely. (#2481)
36
+
37
+ ### Added
38
+
39
+ - New `usage` resource (`get`, `daily`, `byEndpoint` — `GET /v1/usage`,
40
+ `/v1/usage/daily`, `/v1/usage/by-endpoint`) and `webhooks` resource (`list`,
41
+ `retry` — `GET /v1/webhook-deliveries`, `POST /v1/webhook-deliveries/{id}/retry`),
42
+ with their request/response types exported. (#2471)
43
+ - `ScrapeOpts` and `FetchOpts` now cover the full documented request field set
44
+ (10 previously missing `ScrapeRequest` fields; 11 markdown-tuning
45
+ `FetchRequest` fields), plus the `'text'` markdown-links mode. A compile-time
46
+ drift guard keeps them aligned with the generated OpenAPI schema. (#2475)
47
+ - `monitors`, `sitemap`, `bulk`, and `fetch.bulk` methods are now fully typed from
48
+ the generated OpenAPI `operations` (previously `Record<string, unknown>` in /
49
+ `Promise<unknown>` out); their request/response types are re-exported. (#2481)
50
+ - `monitors.list` accepts typed query params and returns a typed response
51
+ (`ListMonitorsParams` / `ListMonitorsResponse`, both exported). (#2480)
52
+
53
+ ### Changed (**BREAKING**)
54
+
55
+ - `fetch.get` now rejects a `custom` option at compile time: `GET /v1/fetch` has
56
+ no `custom` query param, so the value was silently dropped before. Use
57
+ `fetch.create` (POST) to echo a `custom` payload. (#2481)
58
+ - `FetchOpts.format` is narrowed from `string` to the spec union
59
+ `'markdown' | 'html'`; off-spec values now fail to compile. (#2475)
60
+
11
61
  ## [0.2.2] - 2026-07-16
12
62
 
13
63
  ### Changed
@@ -85,6 +135,7 @@ against 0.1.0 — read each one before upgrading.
85
135
  (scrape, screenshot, search, and related endpoints), published under the original
86
136
  `@rapidcrawl/sdk` name.
87
137
 
138
+ [0.3.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
88
139
  [0.2.2]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
89
140
  [0.2.1]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
90
141
  [0.2.0]: https://github.com/Hunt-Labs-Inc/ByteKit/tree/main/packages/sdk-typescript
package/README.md CHANGED
@@ -27,16 +27,15 @@ Create an API key at [app.bytekit.com](https://app.bytekit.com) (keys are
27
27
  prefixed `sk_live_`) and pass it to the client constructor:
28
28
 
29
29
  ```ts
30
- import { RapidCrawl } from '@hunt-labs/bytekit-sdk';
30
+ import { ByteKit } from '@hunt-labs/bytekit-sdk';
31
31
 
32
- const client = new RapidCrawl({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...
32
+ const client = new ByteKit({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...
33
33
  ```
34
34
 
35
35
  The key is sent as `Authorization: Bearer <apiKey>` on every request. Keep it
36
36
  server-side — it is not safe to ship in a browser bundle.
37
37
 
38
- > The client class is named `RapidCrawl` for historical reasons (ByteKit's
39
- > previous product name). It is the SDK's only entrypoint class.
38
+ > `ByteKit` is the SDK's only entrypoint class.
40
39
 
41
40
  ## Base-URL override
42
41
 
@@ -44,7 +43,7 @@ Requests go to `https://api.bytekit.com` by default. Point the client at
44
43
  staging, a local gateway, or a proxy with the `baseUrl` option:
45
44
 
46
45
  ```ts
47
- const client = new RapidCrawl({
46
+ const client = new ByteKit({
48
47
  apiKey: process.env.BYTEKIT_API_KEY!,
49
48
  baseUrl: 'https://api-stg.bytekit.com', // default: https://api.bytekit.com
50
49
  });
@@ -69,8 +68,27 @@ const scrape = await client.scrape.get('sc_123abc');
69
68
 
70
69
  ### Raw fetch (unprocessed response body)
71
70
 
71
+ `/v1/fetch` returns the fetched content **as the body**, with metadata in
72
+ `X-Fetch-*` response headers. `fetch.create` / `fetch.get` resolve a
73
+ `FetchResult` — `body` is the raw content string (a JSON-serving upstream
74
+ comes back as the raw JSON string, never a parsed object), and the named
75
+ fields decode the metadata headers. `body` is UTF-8-decoded, so for true
76
+ binary content (images, PDFs, etc. — `/v1/fetch` supports raw passthrough)
77
+ use `bytes` instead, which is byte-for-byte identical to the wire content:
78
+
72
79
  ```ts
73
- const raw = await client.fetch.create({ url: 'https://example.com' });
80
+ const res = await client.fetch.create({ url: 'https://example.com' });
81
+
82
+ console.log(res.body); // raw content (e.g. "<!doctype html>…") — UTF-8 decoded
83
+ console.log(res.bytes); // Uint8Array — byte-exact, safe for binary content
84
+ console.log(res.id); // X-Fetch-ID, e.g. "ft_0123abcd…"
85
+ console.log(res.creditsCharged); // X-Fetch-Credits-Charged, as a number
86
+ console.log(res.cache); // "hit" | "miss" | "bypass"
87
+ console.log(res.custom); // base64 X-Fetch-Custom echo, or null
88
+ console.log(res.contentType); // response Content-Type, e.g. "text/html"
89
+
90
+ // Anything not surfaced as a named field is on res.headers:
91
+ console.log(res.headers.get('X-Fetch-Waf-Vendor'));
74
92
  ```
75
93
 
76
94
  ### Web search
@@ -124,7 +142,10 @@ await client.bulk.screenshots.list('bulk_123abc');
124
142
  ```ts
125
143
  const monitor = await client.monitors.create({
126
144
  url: 'https://example.com/pricing',
145
+ type: 'screenshot',
127
146
  interval_type: 'daily',
147
+ change_threshold: 5,
148
+ notify_on: 'change',
128
149
  webhook_url: 'https://your.app/hooks/bytekit',
129
150
  });
130
151
  await client.monitors.list();
@@ -134,7 +155,16 @@ await client.monitors.delete('mon_123abc');
134
155
  ### Sitemap crawl
135
156
 
136
157
  ```ts
137
- const crawl = await client.sitemap.create({ url: 'https://example.com' });
158
+ const crawl = await client.sitemap.create({
159
+ url: 'https://example.com',
160
+ strategy: 'standard',
161
+ max_depth: 2,
162
+ max_urls: 5000,
163
+ process: null,
164
+ webhook_secret: null,
165
+ cache_ttl: '30d',
166
+ compact: false,
167
+ });
138
168
  const sitemap = await client.sitemap.get('sm_123abc');
139
169
  ```
140
170
 
@@ -143,20 +173,36 @@ const sitemap = await client.sitemap.get('sm_123abc');
143
173
  ```ts
144
174
  const account = await client.account.get();
145
175
  const keys = await client.account.apiKeys.list();
176
+
177
+ // Usage (Bearer API key auth)
178
+ const usage = await client.usage.get();
179
+ const daily = await client.usage.daily({ from: '2026-01-01', to: '2026-01-31' });
180
+ const byEndpoint = await client.usage.byEndpoint();
181
+
182
+ // Webhook deliveries
183
+ const deliveries = await client.webhooks.list();
184
+ await client.webhooks.retry('whd_123abc');
146
185
  ```
147
186
 
187
+ > **Note:** `client.account.*` (including `client.account.apiKeys.*`) targets the
188
+ > `/v1/account/*` endpoints, which authenticate with a Clerk dashboard session —
189
+ > **not** a Bearer API key. Calling them with an API-key client will fail auth.
190
+ > They remain on the SDK surface for dashboard/session consumers and are marked
191
+ > `x-sdk-scope: internal` in the spec, so they are intentionally excluded from the
192
+ > v0.1 Bearer-key parity guarantee.
193
+
148
194
  ## Error handling
149
195
 
150
- Any 4xx/5xx response throws a `RapidCrawlError` carrying the HTTP status and
196
+ Any 4xx/5xx response throws a `ByteKitError` carrying the HTTP status and
151
197
  the API's error code:
152
198
 
153
199
  ```ts
154
- import { RapidCrawl, RapidCrawlError } from '@hunt-labs/bytekit-sdk';
200
+ import { ByteKit, ByteKitError } from '@hunt-labs/bytekit-sdk';
155
201
 
156
202
  try {
157
203
  await client.scrape.create({ url: 'https://example.com' });
158
204
  } catch (err) {
159
- if (err instanceof RapidCrawlError) {
205
+ if (err instanceof ByteKitError) {
160
206
  console.error(err.status, err.code, err.message);
161
207
  }
162
208
  }
@@ -8,5 +8,5 @@
8
8
  *
9
9
  * To add a method: annotate the operation in the spec first, then add it here.
10
10
  */
11
- export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get"];
11
+ export declare const V01_METHODS: readonly ["scrape.create", "scrape.get", "screenshots.create", "screenshots.get", "bulk.create", "bulk.get", "bulk.cancel", "bulk.screenshots.list", "scrape.bulk.create", "scrape.bulk.get", "fetch.create", "fetch.get", "fetch.bulk.create", "fetch.bulk.get", "monitors.create", "monitors.list", "monitors.get", "monitors.update", "monitors.delete", "monitors.captures.list", "sitemap.create", "sitemap.get", "search.query", "account.get", "usage.get", "usage.daily", "usage.byEndpoint", "webhooks.list", "webhooks.retry"];
12
12
  export type V01Method = (typeof V01_METHODS)[number];
package/dist/allowlist.js CHANGED
@@ -34,4 +34,11 @@ export const V01_METHODS = [
34
34
  'sitemap.get',
35
35
  'search.query',
36
36
  'account.get',
37
+ // usage + webhook-delivery ops — resource surface shipped in #2471, brought
38
+ // under the parity guard in #2476.
39
+ 'usage.get',
40
+ 'usage.daily',
41
+ 'usage.byEndpoint',
42
+ 'webhooks.list',
43
+ 'webhooks.retry',
37
44
  ];
package/dist/client.d.ts CHANGED
@@ -6,6 +6,8 @@ import { MonitorsResource } from './resources/monitors.js';
6
6
  import { SitemapResource } from './resources/sitemap.js';
7
7
  import { SearchResource } from './resources/search.js';
8
8
  import { AccountResource } from './resources/account.js';
9
+ import { UsageResource } from './resources/usage.js';
10
+ import { WebhooksResource } from './resources/webhooks.js';
9
11
  export interface ByteKitOptions {
10
12
  apiKey: string;
11
13
  baseUrl?: string;
@@ -71,6 +73,8 @@ export declare class ByteKit {
71
73
  readonly sitemap: SitemapResource;
72
74
  readonly search: SearchResource;
73
75
  readonly account: AccountResource;
76
+ readonly usage: UsageResource;
77
+ readonly webhooks: WebhooksResource;
74
78
  constructor(options: ByteKitOptions);
75
79
  /**
76
80
  * Resolve the abort budget for one request from the caller's options and the
@@ -86,6 +90,20 @@ export declare class ByteKit {
86
90
  * A timeout of `0` (either level) means "no SDK-side timeout".
87
91
  */
88
92
  private resolveSignal;
93
+ /**
94
+ * Perform the authenticated round-trip and return the raw `Response`, having
95
+ * already thrown for the two non-body failure modes:
96
+ *
97
+ * - `ByteKitConnectionError` when no HTTP response was produced at all
98
+ * (DNS/socket/TLS failure, client-side timeout, or caller abort).
99
+ * - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
100
+ *
101
+ * The body is left unread so each caller can decode it as it needs — JSON for
102
+ * `request()`, raw text + headers for `requestRaw()`. This is the single place
103
+ * that owns auth, the abort budget, and the error taxonomy; the two public
104
+ * transports differ only in how they read a successful body.
105
+ */
106
+ private send;
89
107
  /**
90
108
  * Make an authenticated JSON request to the ByteKit API.
91
109
  *
@@ -94,6 +112,27 @@ export declare class ByteKit {
94
112
  * timeout, or caller abort).
95
113
  */
96
114
  request<T>(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<T>;
115
+ /**
116
+ * Make an authenticated request whose response body IS the content, not a JSON
117
+ * envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
118
+ * response headers.
119
+ *
120
+ * Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
121
+ * response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
122
+ * deliberately a separate transport from `request()`: the generic JSON parse
123
+ * there is shared by every other resource and must not change (#2466).
124
+ *
125
+ * `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
126
+ * first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
127
+ * invalid byte sequence gets silently replaced with U+FFFD. Reading the body
128
+ * once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
129
+ * is always byte-exact, `body` is a best-effort string for callers who know the
130
+ * content is text.
131
+ */
132
+ requestRaw(method: string, path: string, body?: unknown, options?: RequestOptions): Promise<{
133
+ body: string;
134
+ bytes: Uint8Array;
135
+ headers: Headers;
136
+ status: number;
137
+ }>;
97
138
  }
98
- export { ByteKit as RapidCrawl, ByteKitError as RapidCrawlError, ByteKitConnectionError as RapidCrawlConnectionError, };
99
- export type { ByteKitOptions as RapidCrawlOptions };
package/dist/client.js CHANGED
@@ -6,6 +6,8 @@ import { MonitorsResource } from './resources/monitors.js';
6
6
  import { SitemapResource } from './resources/sitemap.js';
7
7
  import { SearchResource } from './resources/search.js';
8
8
  import { AccountResource } from './resources/account.js';
9
+ import { UsageResource } from './resources/usage.js';
10
+ import { WebhooksResource } from './resources/webhooks.js';
9
11
  const DEFAULT_BASE_URL = 'https://api.bytekit.com';
10
12
  /**
11
13
  * Default client-side request timeout (2 minutes).
@@ -162,12 +164,18 @@ export class ByteKit {
162
164
  sitemap;
163
165
  search;
164
166
  account;
167
+ usage;
168
+ webhooks;
165
169
  constructor(options) {
166
170
  if (!options?.apiKey) {
167
171
  throw new TypeError('ByteKit: `apiKey` is required — got an empty value. Pass options.apiKey (e.g. from process.env.BYTEKIT_API_KEY).');
168
172
  }
169
173
  this.apiKey = options.apiKey;
170
- this.baseUrl = options.baseUrl ?? DEFAULT_BASE_URL;
174
+ // Normalize once: strip any trailing slash so `baseUrl + path` (path always
175
+ // starts with `/`) never yields a double-slash URL (#2481). Only the trailing
176
+ // slash is removed, so a path-carrying baseUrl (e.g. a reverse-proxy prefix
177
+ // `…/bytekit`) is preserved (E2).
178
+ this.baseUrl = (options.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, '');
171
179
  this.defaultTimeoutMs = validateTimeout(options.timeoutMs, '"timeoutMs"') ?? DEFAULT_TIMEOUT_MS;
172
180
  this.scrape = new ScrapeResource(this);
173
181
  this.screenshots = new ScreenshotsResource(this);
@@ -177,6 +185,8 @@ export class ByteKit {
177
185
  this.sitemap = new SitemapResource(this);
178
186
  this.search = new SearchResource(this);
179
187
  this.account = new AccountResource(this);
188
+ this.usage = new UsageResource(this);
189
+ this.webhooks = new WebhooksResource(this);
180
190
  }
181
191
  /**
182
192
  * Resolve the abort budget for one request from the caller's options and the
@@ -215,13 +225,19 @@ export class ByteKit {
215
225
  };
216
226
  }
217
227
  /**
218
- * Make an authenticated JSON request to the ByteKit API.
228
+ * Perform the authenticated round-trip and return the raw `Response`, having
229
+ * already thrown for the two non-body failure modes:
219
230
  *
220
- * Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
221
- * when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
222
- * timeout, or caller abort).
231
+ * - `ByteKitConnectionError` when no HTTP response was produced at all
232
+ * (DNS/socket/TLS failure, client-side timeout, or caller abort).
233
+ * - `ByteKitError` on any 4xx/5xx, carrying the API's `code`/`message`.
234
+ *
235
+ * The body is left unread so each caller can decode it as it needs — JSON for
236
+ * `request()`, raw text + headers for `requestRaw()`. This is the single place
237
+ * that owns auth, the abort budget, and the error taxonomy; the two public
238
+ * transports differ only in how they read a successful body.
223
239
  */
224
- async request(method, path, body, options) {
240
+ async send(method, path, body, options) {
225
241
  const url = `${this.baseUrl}${path}`;
226
242
  const { signal, timeoutSignal, cleanup } = this.resolveSignal(options);
227
243
  const init = {
@@ -263,6 +279,17 @@ export class ByteKit {
263
279
  }
264
280
  throw new ByteKitError(response.status, code, message);
265
281
  }
282
+ return response;
283
+ }
284
+ /**
285
+ * Make an authenticated JSON request to the ByteKit API.
286
+ *
287
+ * Throws `ByteKitError` on 4xx/5xx responses, and `ByteKitConnectionError`
288
+ * when no HTTP response was produced at all (DNS/socket/TLS failure, client-side
289
+ * timeout, or caller abort).
290
+ */
291
+ async request(method, path, body, options) {
292
+ const response = await this.send(method, path, body, options);
266
293
  // A 204 No Content has an empty body, and a genuinely empty body can arrive on
267
294
  // other success codes too (defensive). Calling response.json() on an empty body
268
295
  // rejects with a SyntaxError — which is how a successful revoke() (DELETE → 204)
@@ -278,15 +305,27 @@ export class ByteKit {
278
305
  }
279
306
  return JSON.parse(text);
280
307
  }
308
+ /**
309
+ * Make an authenticated request whose response body IS the content, not a JSON
310
+ * envelope — the `/v1/fetch` contract, where metadata rides in `X-Fetch-*`
311
+ * response headers.
312
+ *
313
+ * Returns BOTH a UTF-8-decoded string AND the raw, byte-for-byte body, plus the
314
+ * response `Headers` and status, WITHOUT ever calling `JSON.parse`. This is
315
+ * deliberately a separate transport from `request()`: the generic JSON parse
316
+ * there is shared by every other resource and must not change (#2466).
317
+ *
318
+ * `/v1/fetch` supports true binary passthrough (images, PDFs, etc.) as a
319
+ * first-class mode, and UTF-8 decoding a non-UTF-8 byte stream is lossy — any
320
+ * invalid byte sequence gets silently replaced with U+FFFD. Reading the body
321
+ * once via `arrayBuffer()` lets both forms be derived from the same bytes: `bytes`
322
+ * is always byte-exact, `body` is a best-effort string for callers who know the
323
+ * content is text.
324
+ */
325
+ async requestRaw(method, path, body, options) {
326
+ const response = await this.send(method, path, body, options);
327
+ const bytes = new Uint8Array(await response.arrayBuffer());
328
+ const text = new TextDecoder('utf-8').decode(bytes);
329
+ return { body: text, bytes, headers: response.headers, status: response.status };
330
+ }
281
331
  }
282
- // ---------------------------------------------------------------------------
283
- // Deprecated RapidCrawl* aliases (#2421).
284
- //
285
- // The public surface renamed from RapidCrawl to ByteKit. These aliases point at
286
- // the SAME implementations (identity preserved: `RapidCrawl === ByteKit`), so an
287
- // existing consumer's `instanceof` / error-shape contract keeps holding for one
288
- // minor version. Exported here — not only from index.ts — so the SDK's own
289
- // resource files can keep importing the legacy type name from './client.js'
290
- // without churn. Re-exported (with @deprecated JSDoc) from index.ts.
291
- // ---------------------------------------------------------------------------
292
- export { ByteKit as RapidCrawl, ByteKitError as RapidCrawlError, ByteKitConnectionError as RapidCrawlConnectionError, };
package/dist/index.d.ts CHANGED
@@ -1,12 +1,14 @@
1
1
  export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
2
2
  export type { ByteKitOptions, RequestOptions, ConnectionErrorCode } from './client.js';
3
- /** @deprecated Renamed to ByteKit / ByteKitError / ByteKitConnectionError — kept as aliases for one minor version. */
4
- export { RapidCrawl, RapidCrawlError, RapidCrawlConnectionError } from './client.js';
5
- /** @deprecated Renamed to ByteKitOptions — kept as an alias for one minor version. */
6
- export type { RapidCrawlOptions } from './client.js';
7
3
  export { V01_METHODS } from './allowlist.js';
8
4
  export type { V01Method } from './allowlist.js';
9
- export type { ScrapeId, ScreenshotId, RecordingId, BulkId, MonitorId, SitemapId, ApiKeyId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
5
+ export type { ScrapeId, ScreenshotId, BulkId, MonitorId, SitemapId, ApiKeyId, WebhookDeliveryId, SearchType, SearchDateRange, SearchOpts, SearchResult, SearchResultImage, SearchResponse, } from './types.js';
10
6
  export type { ScrapeCreateResponse, ScrapeGetResponse, ScrapeBulkCreateRequest, ScrapeBulkCreateResponse, ScrapeBulkGetResponse, } from './resources/scrape.js';
11
7
  export type { ScreenshotCreateRequest, ScreenshotCreateResponse, ScreenshotGetResponse, } from './resources/screenshots.js';
8
+ export type { FetchOpts, FetchGetOpts, FetchResult, FetchCacheStatus, FetchBulkCreateRequest, FetchBulkCreateResponse, FetchBulkGetResponse, } from './resources/fetch.js';
9
+ export type { MonitorCreateRequest, MonitorCreateResponse, ListMonitorsParams, ListMonitorsResponse, MonitorResponse, MonitorUpdateRequest, UpdateMonitorResponse, ListMonitorCapturesResponse, } from './resources/monitors.js';
10
+ export type { SitemapCreateRequest, SitemapCreateResponse, SitemapGetResponse, } from './resources/sitemap.js';
11
+ export type { BulkCreateRequest, BulkCreateResponse, BulkGetResponse, BulkCancelResponse, ListBulkScreenshotsResponse, } from './resources/bulk.js';
12
12
  export type { GetAccountResponse, ListApiKeysResponse, CreateApiKeyRequest, CreateApiKeyResponse, GetApiKeyResponse, UpdateApiKeyRequest, UpdateApiKeyResponse, RevealApiKeyResponse, } from './resources/account.js';
13
+ export type { GetUsageResponse, GetUsageDailyParams, GetUsageDailyResponse, GetUsageByEndpointParams, GetUsageByEndpointResponse, } from './resources/usage.js';
14
+ export type { ListWebhookDeliveriesParams, ListWebhookDeliveriesResponse, RetryWebhookDeliveryResponse, } from './resources/webhooks.js';
package/dist/index.js CHANGED
@@ -1,4 +1,2 @@
1
1
  export { ByteKit, ByteKitError, ByteKitConnectionError } from './client.js';
2
- /** @deprecated Renamed to ByteKit / ByteKitError / ByteKitConnectionError — kept as aliases for one minor version. */
3
- export { RapidCrawl, RapidCrawlError, RapidCrawlConnectionError } from './client.js';
4
2
  export { V01_METHODS } from './allowlist.js';
@@ -1,4 +1,4 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
2
  import type { operations } from '../generated/api.js';
3
3
  import type { ApiKeyId } from '../types.js';
4
4
  /**
@@ -36,13 +36,13 @@ export type UpdateApiKeyResponse = operations['updateApiKey']['responses'][200][
36
36
  export declare class AccountResource {
37
37
  private readonly client;
38
38
  readonly apiKeys: ApiKeysResource;
39
- constructor(client: RapidCrawl);
39
+ constructor(client: ByteKit);
40
40
  /** GET /v1/account — get the authenticated account */
41
41
  get(): Promise<GetAccountResponse>;
42
42
  }
43
43
  declare class ApiKeysResource {
44
44
  private readonly client;
45
- constructor(client: RapidCrawl);
45
+ constructor(client: ByteKit);
46
46
  /**
47
47
  * POST /v1/account/api-keys — create a new API key.
48
48
  *
@@ -1,20 +1,31 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
2
3
  import type { BulkId } from '../types.js';
4
+ /** Body of POST /v1/bulk. */
5
+ export type BulkCreateRequest = operations['createBulk']['requestBody']['content']['application/json'];
6
+ /** 202 payload of POST /v1/bulk — bulk jobs are always async. */
7
+ export type BulkCreateResponse = operations['createBulk']['responses'][202]['content']['application/json'];
8
+ /** 200 payload of GET /v1/bulk/{id}. */
9
+ export type BulkGetResponse = operations['getBulk']['responses'][200]['content']['application/json'];
10
+ /** 200 payload of DELETE /v1/bulk/{id} — cancels an in-flight job. */
11
+ export type BulkCancelResponse = operations['deleteBulk']['responses'][200]['content']['application/json'];
12
+ /** 200 payload of GET /v1/bulk/{id}/screenshots. */
13
+ export type ListBulkScreenshotsResponse = operations['listBulkScreenshots']['responses'][200]['content']['application/json'];
3
14
  export declare class BulkResource {
4
15
  private readonly client;
5
16
  readonly screenshots: BulkScreenshotsResource;
6
- constructor(client: RapidCrawl);
17
+ constructor(client: ByteKit);
7
18
  /** POST /v1/bulk — create a bulk screenshots job */
8
- create(opts: Record<string, unknown>): Promise<unknown>;
19
+ create(opts: BulkCreateRequest): Promise<BulkCreateResponse>;
9
20
  /** GET /v1/bulk/{id} — get a bulk job status */
10
- get(id: BulkId): Promise<unknown>;
21
+ get(id: BulkId): Promise<BulkGetResponse>;
11
22
  /** DELETE /v1/bulk/{id} — cancel an in-flight bulk job */
12
- cancel(id: BulkId): Promise<unknown>;
23
+ cancel(id: BulkId): Promise<BulkCancelResponse>;
13
24
  }
14
25
  declare class BulkScreenshotsResource {
15
26
  private readonly client;
16
- constructor(client: RapidCrawl);
27
+ constructor(client: ByteKit);
17
28
  /** GET /v1/bulk/{id}/screenshots — list screenshots in a bulk job */
18
- list(id: BulkId): Promise<unknown>;
29
+ list(id: BulkId): Promise<ListBulkScreenshotsResponse>;
19
30
  }
20
31
  export {};
@@ -1,5 +1,12 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
2
3
  import type { BulkId } from '../types.js';
4
+ /** Body of POST /v1/fetch/bulk. */
5
+ export type FetchBulkCreateRequest = operations['createFetchBulk']['requestBody']['content']['application/json'];
6
+ /** 202 payload of POST /v1/fetch/bulk — bulk fetch jobs are always async. */
7
+ export type FetchBulkCreateResponse = operations['createFetchBulk']['responses'][202]['content']['application/json'];
8
+ /** 200 payload of GET /v1/fetch/bulk/{id}. */
9
+ export type FetchBulkGetResponse = operations['getFetchBulk']['responses'][200]['content']['application/json'];
3
10
  /**
4
11
  * Options for fetch endpoints.
5
12
  *
@@ -8,9 +15,32 @@ import type { BulkId } from '../types.js';
8
15
  */
9
16
  export interface FetchOpts {
10
17
  url: string;
11
- format?: string;
18
+ /** Output format. `'markdown'` runs the HTML→markdown pipeline; `'html'` returns raw HTML. Narrowed to the spec union — off-spec values (e.g. `'pdf'`) are rejected at compile time. */
19
+ format?: 'markdown' | 'html';
12
20
  country?: string;
13
21
  timeout_ms?: number;
22
+ /** Markdown processing mode. Only used when `format=markdown`. article/raw/llm. */
23
+ markdown_mode?: 'article' | 'raw' | 'llm';
24
+ /** BM25 query string for relevance-ranked filtering. Only used when `format=markdown`. */
25
+ markdown_query?: string;
26
+ /** Link rendering style. Only used when `format=markdown`. */
27
+ markdown_links?: 'inline' | 'references' | 'none' | 'text';
28
+ /** Image retention mode. Only used when `format=markdown`. */
29
+ markdown_images?: 'inline' | 'references' | 'none' | 'text';
30
+ /** Append a `Links:` footer to the markdown output. Only used when `format=markdown`. */
31
+ with_links_summary?: boolean;
32
+ /** Append an `Images:` footer to the markdown output. Only used when `format=markdown`. */
33
+ with_images_summary?: boolean;
34
+ /** Compact whitespace output. Only used when `format=markdown`. */
35
+ markdown_compact?: boolean;
36
+ /** Filter low-signal images. Only used when `format=markdown`. */
37
+ markdown_filter_images?: boolean;
38
+ /** When true, `formats.links`/`formats.images` return rich objects and a top-level `tables` array is included. Only used when `format=markdown`. */
39
+ markdown_include_media?: boolean;
40
+ /** When true, enables markdown-pipeline and tag-filter warnings for affected html/markdown requests. */
41
+ markdown_include_warnings?: boolean;
42
+ /** When true, includes a top-level `stats` object (chars, tokens, blocks). Only used when `format=markdown`. */
43
+ markdown_include_stats?: boolean;
14
44
  /**
15
45
  * How long a freshly fetched URL may be served from cache. Matches the
16
46
  * OpenAPI `FetchRequest.cache_ttl` schema: a duration string (`"48h"`,
@@ -26,29 +56,87 @@ export interface FetchOpts {
26
56
  */
27
57
  custom?: Record<string, unknown>;
28
58
  }
59
+ /**
60
+ * Options accepted by `fetch.get` (GET /v1/fetch).
61
+ *
62
+ * Identical to {@link FetchOpts} minus `custom`: the spec's `GET /v1/fetch` has
63
+ * no `custom` query param, so it is excluded here — passing it is a compile
64
+ * error rather than a silently-dropped option (#2481).
65
+ */
66
+ export type FetchGetOpts = Omit<FetchOpts, 'custom'>;
67
+ /** Cache-lookup outcome reported in the `X-Fetch-Cache` response header. */
68
+ export type FetchCacheStatus = 'hit' | 'miss' | 'bypass';
69
+ /**
70
+ * The result of `fetch.create` / `fetch.get`.
71
+ *
72
+ * `/v1/fetch` returns the fetched content AS the response body (text/html,
73
+ * text/markdown, or raw binary passthrough) with metadata carried in `X-Fetch-*`
74
+ * response headers — it is NOT a JSON envelope. `body` is therefore the raw
75
+ * content string, never a parsed object, and the named fields decode the
76
+ * headers the docs call out. Anything not surfaced here is available on
77
+ * `headers` (e.g. `X-Fetch-Duration-Ms`, `X-Fetch-Waf-Vendor`).
78
+ */
79
+ export interface FetchResult {
80
+ /**
81
+ * The raw response body exactly as returned — never `JSON.parse`d. Decoded as
82
+ * UTF-8: for text content (HTML, markdown, JSON-as-string) this is exact, but for
83
+ * true binary content (images, PDFs, etc. — `/v1/fetch` supports raw passthrough)
84
+ * any byte sequence that isn't valid UTF-8 is silently replaced with U+FFFD. Use
85
+ * {@link FetchResult.bytes} when the content may be binary.
86
+ */
87
+ body: string;
88
+ /**
89
+ * The raw response body as bytes, byte-for-byte identical to the wire content —
90
+ * lossless for both text and true binary content (images, PDFs, etc.). Prefer
91
+ * this over `body` whenever the fetched content might not be UTF-8 text.
92
+ */
93
+ bytes: Uint8Array;
94
+ /** `X-Fetch-ID` — unique fetch id (`ft_…`), or `null` if the header is absent. */
95
+ id: string | null;
96
+ /** `X-Fetch-Credits-Charged` parsed as a number, or `null` if absent/unparseable. */
97
+ creditsCharged: number | null;
98
+ /** `X-Fetch-Cache` — `hit` | `miss` | `bypass`, or `null` if absent. */
99
+ cache: FetchCacheStatus | null;
100
+ /**
101
+ * `X-Fetch-Custom` — base64-encoded echo of the request's `custom` payload,
102
+ * or `null` when the request sent no `custom`.
103
+ */
104
+ custom: string | null;
105
+ /** The response `Content-Type` (e.g. `text/html`), or `null` if absent. */
106
+ contentType: string | null;
107
+ /** The full response headers, for any `X-Fetch-*` value not surfaced above. */
108
+ headers: Headers;
109
+ }
29
110
  export declare class FetchResource {
30
111
  private readonly client;
31
112
  readonly bulk: FetchBulkResource;
32
- constructor(client: RapidCrawl);
113
+ constructor(client: ByteKit);
33
114
  /**
34
115
  * POST /v1/fetch — fetch URL content via POST body.
35
116
  *
36
- * Returns content directly (not ID-based).
117
+ * Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
118
+ * The body is returned verbatim — a JSON-serving upstream comes back as the
119
+ * raw JSON string, not a parsed object.
37
120
  */
38
- create(opts: FetchOpts): Promise<unknown>;
121
+ create(opts: FetchOpts): Promise<FetchResult>;
39
122
  /**
40
123
  * GET /v1/fetch — fetch URL content via query params.
41
124
  *
42
- * Returns content directly (not ID-based).
125
+ * Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
126
+ *
127
+ * `custom` is intentionally absent from the accepted opts: the spec's
128
+ * `GET /v1/fetch` has no `custom` query param and the value was never
129
+ * serialized, so passing it is a compile error rather than a silent drop
130
+ * (#2481). Use `create()` (POST) when you need to echo a `custom` payload.
43
131
  */
44
- get(opts: FetchOpts): Promise<unknown>;
132
+ get(opts: FetchGetOpts): Promise<FetchResult>;
45
133
  }
46
134
  declare class FetchBulkResource {
47
135
  private readonly client;
48
- constructor(client: RapidCrawl);
136
+ constructor(client: ByteKit);
49
137
  /** POST /v1/fetch/bulk — create a bulk fetch job */
50
- create(opts: Record<string, unknown>): Promise<unknown>;
138
+ create(opts: FetchBulkCreateRequest): Promise<FetchBulkCreateResponse>;
51
139
  /** GET /v1/fetch/bulk/{id} — get a bulk fetch job */
52
- get(id: BulkId): Promise<unknown>;
140
+ get(id: BulkId): Promise<FetchBulkGetResponse>;
53
141
  }
54
142
  export {};
@@ -1,3 +1,23 @@
1
+ const CACHE_STATUSES = ['hit', 'miss', 'bypass'];
2
+ /** Build a typed {@link FetchResult} from a raw body + response headers. */
3
+ function toFetchResult(raw) {
4
+ const { body, bytes, headers } = raw;
5
+ const creditsRaw = headers.get('X-Fetch-Credits-Charged');
6
+ const credits = creditsRaw === null ? NaN : Number(creditsRaw);
7
+ const cacheRaw = headers.get('X-Fetch-Cache');
8
+ return {
9
+ body,
10
+ bytes,
11
+ id: headers.get('X-Fetch-ID'),
12
+ creditsCharged: Number.isFinite(credits) ? credits : null,
13
+ cache: cacheRaw !== null && CACHE_STATUSES.includes(cacheRaw)
14
+ ? cacheRaw
15
+ : null,
16
+ custom: headers.get('X-Fetch-Custom'),
17
+ contentType: headers.get('Content-Type'),
18
+ headers,
19
+ };
20
+ }
1
21
  export class FetchResource {
2
22
  client;
3
23
  bulk;
@@ -8,17 +28,24 @@ export class FetchResource {
8
28
  /**
9
29
  * POST /v1/fetch — fetch URL content via POST body.
10
30
  *
11
- * Returns content directly (not ID-based).
31
+ * Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
32
+ * The body is returned verbatim — a JSON-serving upstream comes back as the
33
+ * raw JSON string, not a parsed object.
12
34
  */
13
- create(opts) {
14
- return this.client.request('POST', '/v1/fetch', opts);
35
+ async create(opts) {
36
+ return toFetchResult(await this.client.requestRaw('POST', '/v1/fetch', opts));
15
37
  }
16
38
  /**
17
39
  * GET /v1/fetch — fetch URL content via query params.
18
40
  *
19
- * Returns content directly (not ID-based).
41
+ * Resolves a `FetchResult` — the raw content body plus `X-Fetch-*` metadata.
42
+ *
43
+ * `custom` is intentionally absent from the accepted opts: the spec's
44
+ * `GET /v1/fetch` has no `custom` query param and the value was never
45
+ * serialized, so passing it is a compile error rather than a silent drop
46
+ * (#2481). Use `create()` (POST) when you need to echo a `custom` payload.
20
47
  */
21
- get(opts) {
48
+ async get(opts) {
22
49
  const params = new URLSearchParams();
23
50
  params.set('url', opts.url);
24
51
  if (opts.format !== undefined)
@@ -29,7 +56,7 @@ export class FetchResource {
29
56
  params.set('timeout_ms', String(opts.timeout_ms));
30
57
  if (opts.cache_ttl !== undefined)
31
58
  params.set('cache_ttl', String(opts.cache_ttl));
32
- return this.client.request('GET', `/v1/fetch?${params.toString()}`);
59
+ return toFetchResult(await this.client.requestRaw('GET', `/v1/fetch?${params.toString()}`));
33
60
  }
34
61
  }
35
62
  class FetchBulkResource {
@@ -1,27 +1,66 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
2
3
  import type { MonitorId } from '../types.js';
4
+ /**
5
+ * Query params of GET /v1/monitors — all optional, omitted when unset.
6
+ *
7
+ * A `type` alias (not an `interface`) so it carries an implicit index signature and stays
8
+ * assignable to {@link toQueryString}'s `Record<string, string | number | undefined>` param
9
+ * (interfaces do not get the implicit index signature; type aliases do).
10
+ */
11
+ export type ListMonitorsParams = {
12
+ limit?: number;
13
+ cursor?: string;
14
+ status?: string;
15
+ };
16
+ /** Body of POST /v1/monitors. */
17
+ export type MonitorCreateRequest = operations['createMonitor']['requestBody']['content']['application/json'];
18
+ /** 201 payload of POST /v1/monitors. */
19
+ export type MonitorCreateResponse = operations['createMonitor']['responses'][201]['content']['application/json'];
20
+ /** 200 payload of GET /v1/monitors — the paginated monitor list. */
21
+ export type ListMonitorsResponse = operations['listMonitors']['responses'][200]['content']['application/json'];
22
+ /** 200 payload of GET /v1/monitors/{id} — a single monitor. */
23
+ export type MonitorResponse = operations['getMonitor']['responses'][200]['content']['application/json'];
24
+ /** Body of PATCH /v1/monitors/{id}. */
25
+ export type MonitorUpdateRequest = operations['updateMonitor']['requestBody']['content']['application/json'];
26
+ /** 200 payload of PATCH /v1/monitors/{id}. */
27
+ export type UpdateMonitorResponse = operations['updateMonitor']['responses'][200]['content']['application/json'];
28
+ /** 200 payload of GET /v1/monitors/{id}/captures — the paginated capture list. */
29
+ export type ListMonitorCapturesResponse = operations['listMonitorCaptures']['responses'][200]['content']['application/json'];
3
30
  export declare class MonitorsResource {
4
31
  private readonly client;
5
32
  readonly captures: MonitorCapturesResource;
6
- constructor(client: RapidCrawl);
33
+ constructor(client: ByteKit);
7
34
  /**
8
35
  * POST /v1/monitors — create a page-change monitor (screenshot or scrape capture).
9
36
  * First-capture behavior is the same for both types: the first capture is a silent baseline (has_change=false, change_pct=null) with no change_detected webhook, whether the monitor is `scrape` or `screenshot`. From the second capture onward both types diff against the previous capture.
10
37
  */
11
- create(opts: Record<string, unknown>): Promise<unknown>;
12
- /** GET /v1/monitors — list all monitors for the account */
13
- list(): Promise<unknown>;
38
+ create(opts: MonitorCreateRequest): Promise<MonitorCreateResponse>;
39
+ /**
40
+ * GET /v1/monitors — list monitors for the account.
41
+ *
42
+ * `params` is optional and each key is omitted-when-absent, so `list()` with no argument
43
+ * serializes the bare `/v1/monitors` URL unchanged — existing zero-arg callers are unaffected.
44
+ */
45
+ list(params?: ListMonitorsParams): Promise<ListMonitorsResponse>;
14
46
  /** GET /v1/monitors/{id} — get a monitor by ID */
15
- get(id: MonitorId): Promise<unknown>;
47
+ get(id: MonitorId): Promise<MonitorResponse>;
16
48
  /** PATCH /v1/monitors/{id} — update a monitor */
17
- update(id: MonitorId, opts: Record<string, unknown>): Promise<unknown>;
18
- /** DELETE /v1/monitors/{id} — delete a monitor */
19
- delete(id: MonitorId): Promise<unknown>;
49
+ update(id: MonitorId, opts: MonitorUpdateRequest): Promise<UpdateMonitorResponse>;
50
+ /**
51
+ * DELETE /v1/monitors/{id} — delete a monitor.
52
+ *
53
+ * Returns 204 No Content (`operations['deleteMonitor']['responses'][204]`
54
+ * declares `content?: never`), so there is no JSON arm to derive a payload
55
+ * from — hence `Promise<void>`. `client.request()` short-circuits the empty
56
+ * body to `undefined` before it would parse as JSON (#2450).
57
+ */
58
+ delete(id: MonitorId): Promise<void>;
20
59
  }
21
60
  declare class MonitorCapturesResource {
22
61
  private readonly client;
23
- constructor(client: RapidCrawl);
62
+ constructor(client: ByteKit);
24
63
  /** GET /v1/monitors/{id}/captures — list captures for a monitor */
25
- list(id: MonitorId): Promise<unknown>;
64
+ list(id: MonitorId): Promise<ListMonitorCapturesResponse>;
26
65
  }
27
66
  export {};
@@ -1,3 +1,4 @@
1
+ import { toQueryString } from './query-string.js';
1
2
  export class MonitorsResource {
2
3
  client;
3
4
  captures;
@@ -12,9 +13,14 @@ export class MonitorsResource {
12
13
  create(opts) {
13
14
  return this.client.request('POST', '/v1/monitors', opts);
14
15
  }
15
- /** GET /v1/monitors — list all monitors for the account */
16
- list() {
17
- return this.client.request('GET', '/v1/monitors');
16
+ /**
17
+ * GET /v1/monitors — list monitors for the account.
18
+ *
19
+ * `params` is optional and each key is omitted-when-absent, so `list()` with no argument
20
+ * serializes the bare `/v1/monitors` URL unchanged — existing zero-arg callers are unaffected.
21
+ */
22
+ list(params) {
23
+ return this.client.request('GET', `/v1/monitors${toQueryString(params)}`);
18
24
  }
19
25
  /** GET /v1/monitors/{id} — get a monitor by ID */
20
26
  get(id) {
@@ -24,7 +30,14 @@ export class MonitorsResource {
24
30
  update(id, opts) {
25
31
  return this.client.request('PATCH', `/v1/monitors/${id}`, opts);
26
32
  }
27
- /** DELETE /v1/monitors/{id} — delete a monitor */
33
+ /**
34
+ * DELETE /v1/monitors/{id} — delete a monitor.
35
+ *
36
+ * Returns 204 No Content (`operations['deleteMonitor']['responses'][204]`
37
+ * declares `content?: never`), so there is no JSON arm to derive a payload
38
+ * from — hence `Promise<void>`. `client.request()` short-circuits the empty
39
+ * body to `undefined` before it would parse as JSON (#2450).
40
+ */
28
41
  delete(id) {
29
42
  return this.client.request('DELETE', `/v1/monitors/${id}`);
30
43
  }
@@ -0,0 +1,10 @@
1
+ /**
2
+ * Serialize a flat params object to a `?a=1&b=2` query string, omitting `undefined` values.
3
+ *
4
+ * Returns `''` (not `'?'`) for a `undefined` param bag or one whose every value is `undefined`,
5
+ * so a caller can unconditionally append the result to a path and get a bare URL when no params
6
+ * are set. This is the single query-string serializer shared across the query-param resources
7
+ * (`usage`, `webhooks`, `monitors`) — extracting it keeps their omit-when-absent semantics
8
+ * byte-identical rather than drifting across per-file copies.
9
+ */
10
+ export declare function toQueryString(params: Record<string, string | number | undefined> | undefined): string;
@@ -0,0 +1,20 @@
1
+ /**
2
+ * Serialize a flat params object to a `?a=1&b=2` query string, omitting `undefined` values.
3
+ *
4
+ * Returns `''` (not `'?'`) for a `undefined` param bag or one whose every value is `undefined`,
5
+ * so a caller can unconditionally append the result to a path and get a bare URL when no params
6
+ * are set. This is the single query-string serializer shared across the query-param resources
7
+ * (`usage`, `webhooks`, `monitors`) — extracting it keeps their omit-when-absent semantics
8
+ * byte-identical rather than drifting across per-file copies.
9
+ */
10
+ export function toQueryString(params) {
11
+ if (!params)
12
+ return '';
13
+ const search = new URLSearchParams();
14
+ for (const [key, value] of Object.entries(params)) {
15
+ if (value !== undefined)
16
+ search.set(key, String(value));
17
+ }
18
+ const qs = search.toString();
19
+ return qs ? `?${qs}` : '';
20
+ }
@@ -1,4 +1,4 @@
1
- import type { RapidCrawl, RequestOptions } from '../client.js';
1
+ import type { ByteKit, RequestOptions } from '../client.js';
2
2
  import type { operations } from '../generated/api.js';
3
3
  import type { ScrapeId, BulkId } from '../types.js';
4
4
  /**
@@ -21,7 +21,11 @@ export type ScrapeBulkGetResponse = operations['getScrapeBulk']['responses'][200
21
21
  /** Markdown processing mode for the pipeline. */
22
22
  export type MarkdownMode = 'article' | 'raw' | 'llm';
23
23
  /** Link rendering style in the markdown output. */
24
- export type MarkdownLinks = 'inline' | 'references' | 'none';
24
+ export type MarkdownLinks = 'inline' | 'references' | 'none' | 'text';
25
+ /** Image retention mode in the markdown output. */
26
+ export type MarkdownImages = 'inline' | 'references' | 'none' | 'text';
27
+ /** Tokenizer encoding hint for `token_budget`. */
28
+ export type TokenEncoding = 'cl100k_base' | 'o200k_base';
25
29
  /** Options for POST /v1/scrape. */
26
30
  export interface ScrapeOpts {
27
31
  url: string;
@@ -41,6 +45,12 @@ export interface ScrapeOpts {
41
45
  markdown_query?: string;
42
46
  /** Link rendering style in the markdown output. */
43
47
  markdown_links?: MarkdownLinks;
48
+ /** Image retention mode in the markdown output. Distinct from `markdown_filter_images` (which drops low-signal images before conversion). */
49
+ markdown_images?: MarkdownImages;
50
+ /** When true, appends a `Links:` footer to the markdown output. Automatically enabled when `markdown_links` is `'references'`. */
51
+ with_links_summary?: boolean;
52
+ /** When true, appends an `Images:` footer to the markdown output. */
53
+ with_images_summary?: boolean;
44
54
  /** Collapse excessive whitespace for a more compact output. */
45
55
  markdown_compact?: boolean;
46
56
  /** Filter low-signal images from the markdown output. */
@@ -51,6 +61,20 @@ export interface ScrapeOpts {
51
61
  markdown_include_warnings?: boolean;
52
62
  /** When true, the response includes a top-level `stats` object with `ScrapeStats` (chars, tokens, blocks). Only effective when `markdown` is in `formats`. */
53
63
  markdown_include_stats?: boolean;
64
+ /** Maximum token count for the markdown output. Truncates at the last paragraph → sentence → token boundary under the limit. Absent → no truncation. */
65
+ token_budget?: number;
66
+ /** Tokenizer encoding hint for `token_budget`. `cl100k_base` (default) matches GPT-4/4o; `o200k_base` matches GPT-4o-mini / o-series. */
67
+ token_encoding?: TokenEncoding;
68
+ /** When true, emulate a mobile device (mobile UA + viewport). Default false (desktop). Part of the cache key, so mobile and desktop responses cache independently. */
69
+ mobile?: boolean;
70
+ /** When true (default), base64-encoded `data:` image sources are stripped before markdown conversion and from `html` output. Set false to preserve them. Does not affect `raw_html`. */
71
+ remove_base64_images?: boolean;
72
+ /** HTML element names (not CSS selectors) to keep. When non-empty, only content within these elements is included in `html`/`markdown` output. Applied before `exclude_tags`. Does not affect `raw_html`. */
73
+ include_tags?: string[];
74
+ /** HTML element names (not CSS selectors) to strip from `html`/`markdown` output. Applied after `include_tags`. Does not affect `raw_html`. */
75
+ exclude_tags?: string[];
76
+ /** When true and `markdown` is in `formats`, the converted markdown is sent to Groq for cleaning before return. Graceful degradation on any Groq failure. Billing: 3× credits and bytes applied only when cleaning succeeds. */
77
+ clean_markdown?: boolean;
54
78
  /**
55
79
  * How long a freshly fetched URL may be served from cache. `'0'`/`0` skips the
56
80
  * cache read (the result is still written), `'Nh'`/`'Nd'` set a TTL (capped at
@@ -70,7 +94,7 @@ export interface ScrapeOpts {
70
94
  export declare class ScrapeResource {
71
95
  private readonly client;
72
96
  readonly bulk: ScrapeBulkResource;
73
- constructor(client: RapidCrawl);
97
+ constructor(client: ByteKit);
74
98
  /**
75
99
  * POST /v1/scrape — create a scrape job.
76
100
  *
@@ -102,7 +126,7 @@ export declare class ScrapeResource {
102
126
  }
103
127
  declare class ScrapeBulkResource {
104
128
  private readonly client;
105
- constructor(client: RapidCrawl);
129
+ constructor(client: ByteKit);
106
130
  /** POST /v1/scrape/bulk — create a bulk scrape job */
107
131
  create(opts: ScrapeBulkCreateRequest): Promise<ScrapeBulkCreateResponse>;
108
132
  /** GET /v1/scrape/bulk/{id} — get a bulk scrape job */
@@ -1,4 +1,4 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
2
  import type { operations } from '../generated/api.js';
3
3
  import type { ScreenshotId } from '../types.js';
4
4
  /** The `createScreenshot` request body exactly as the OpenAPI spec declares it. */
@@ -47,9 +47,18 @@ export type ScreenshotCreateResponse = operations['createScreenshot']['responses
47
47
  export type ScreenshotGetResponse = operations['getScreenshot']['responses'][200]['content']['application/json'] | operations['getScreenshot']['responses'][202]['content']['application/json'];
48
48
  export declare class ScreenshotsResource {
49
49
  private readonly client;
50
- constructor(client: RapidCrawl);
51
- /** POST /v1/screenshots — capture a screenshot */
52
- create(opts: ScreenshotCreateRequest): Promise<ScreenshotCreateResponse>;
50
+ constructor(client: ByteKit);
51
+ /**
52
+ * POST /v1/screenshots — capture a screenshot.
53
+ *
54
+ * `params.async` maps to the `?async=true` query param (queued 202 capture). It is a QUERY
55
+ * param on `createScreenshot`, distinct from scrape where `async` is a body field. `params` is
56
+ * optional and `async` omitted-when-absent: `create(opts)` with no second argument posts to the
57
+ * bare `/v1/screenshots` URL unchanged — existing single-arg callers are unaffected.
58
+ */
59
+ create(opts: ScreenshotCreateRequest, params?: {
60
+ async?: boolean;
61
+ }): Promise<ScreenshotCreateResponse>;
53
62
  /**
54
63
  * GET /v1/screenshots/{id} — poll a screenshot by ID.
55
64
  *
@@ -3,9 +3,17 @@ export class ScreenshotsResource {
3
3
  constructor(client) {
4
4
  this.client = client;
5
5
  }
6
- /** POST /v1/screenshots — capture a screenshot */
7
- create(opts) {
8
- return this.client.request('POST', '/v1/screenshots', opts);
6
+ /**
7
+ * POST /v1/screenshots — capture a screenshot.
8
+ *
9
+ * `params.async` maps to the `?async=true` query param (queued 202 capture). It is a QUERY
10
+ * param on `createScreenshot`, distinct from scrape where `async` is a body field. `params` is
11
+ * optional and `async` omitted-when-absent: `create(opts)` with no second argument posts to the
12
+ * bare `/v1/screenshots` URL unchanged — existing single-arg callers are unaffected.
13
+ */
14
+ create(opts, params) {
15
+ const suffix = params?.async === true ? '?async=true' : '';
16
+ return this.client.request('POST', `/v1/screenshots${suffix}`, opts);
9
17
  }
10
18
  /**
11
19
  * GET /v1/screenshots/{id} — poll a screenshot by ID.
@@ -1,8 +1,8 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
2
  import type { SearchOpts, SearchResponse } from '../types.js';
3
3
  export declare class SearchResource {
4
4
  private readonly client;
5
- constructor(client: RapidCrawl);
5
+ constructor(client: ByteKit);
6
6
  /** POST /v1/search — run a web search and return ranked organic results. */
7
7
  query(opts: SearchOpts): Promise<SearchResponse>;
8
8
  }
@@ -1,10 +1,17 @@
1
- import type { RapidCrawl } from '../client.js';
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
2
3
  import type { SitemapId } from '../types.js';
4
+ /** Body of POST /v1/sitemap. */
5
+ export type SitemapCreateRequest = operations['createSitemap']['requestBody']['content']['application/json'];
6
+ /** 202 payload of POST /v1/sitemap — sitemap crawls are always async. */
7
+ export type SitemapCreateResponse = operations['createSitemap']['responses'][202]['content']['application/json'];
8
+ /** 200 payload of GET /v1/sitemap/{id}. */
9
+ export type SitemapGetResponse = operations['getSitemap']['responses'][200]['content']['application/json'];
3
10
  export declare class SitemapResource {
4
11
  private readonly client;
5
- constructor(client: RapidCrawl);
12
+ constructor(client: ByteKit);
6
13
  /** POST /v1/sitemap — start a sitemap crawl */
7
- create(opts: Record<string, unknown>): Promise<unknown>;
14
+ create(opts: SitemapCreateRequest): Promise<SitemapCreateResponse>;
8
15
  /** GET /v1/sitemap/{id} — get a sitemap job by ID */
9
- get(id: SitemapId): Promise<unknown>;
16
+ get(id: SitemapId): Promise<SitemapGetResponse>;
10
17
  }
@@ -0,0 +1,31 @@
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
3
+ /** 200 payload of GET /v1/usage — current billing period usage summary. */
4
+ export type GetUsageResponse = operations['getUsage']['responses'][200]['content']['application/json'];
5
+ /**
6
+ * Query params of GET /v1/usage/daily.
7
+ * Derived from the generated OpenAPI types so it stays in sync with the spec.
8
+ */
9
+ export type GetUsageDailyParams = NonNullable<operations['getUsageDaily']['parameters']['query']>;
10
+ /** 200 payload of GET /v1/usage/daily — per-day usage breakdown. */
11
+ export type GetUsageDailyResponse = operations['getUsageDaily']['responses'][200]['content']['application/json'];
12
+ /** Query params of GET /v1/usage/by-endpoint. */
13
+ export type GetUsageByEndpointParams = NonNullable<operations['getUsageByEndpoint']['parameters']['query']>;
14
+ /** 200 payload of GET /v1/usage/by-endpoint — per-endpoint bandwidth breakdown. */
15
+ export type GetUsageByEndpointResponse = operations['getUsageByEndpoint']['responses'][200]['content']['application/json'];
16
+ export declare class UsageResource {
17
+ private readonly client;
18
+ constructor(client: ByteKit);
19
+ /** GET /v1/usage — get current billing period usage. */
20
+ get(): Promise<GetUsageResponse>;
21
+ /**
22
+ * GET /v1/usage/daily — get daily usage breakdown.
23
+ * Default range is the last 30 days; max range is 366 days.
24
+ */
25
+ daily(params?: GetUsageDailyParams): Promise<GetUsageDailyResponse>;
26
+ /**
27
+ * GET /v1/usage/by-endpoint — get per-endpoint bandwidth breakdown.
28
+ * Default range is the last 30 days; max range is 366 days.
29
+ */
30
+ byEndpoint(params?: GetUsageByEndpointParams): Promise<GetUsageByEndpointResponse>;
31
+ }
@@ -0,0 +1,25 @@
1
+ import { toQueryString } from './query-string.js';
2
+ export class UsageResource {
3
+ client;
4
+ constructor(client) {
5
+ this.client = client;
6
+ }
7
+ /** GET /v1/usage — get current billing period usage. */
8
+ get() {
9
+ return this.client.request('GET', '/v1/usage');
10
+ }
11
+ /**
12
+ * GET /v1/usage/daily — get daily usage breakdown.
13
+ * Default range is the last 30 days; max range is 366 days.
14
+ */
15
+ daily(params) {
16
+ return this.client.request('GET', `/v1/usage/daily${toQueryString(params)}`);
17
+ }
18
+ /**
19
+ * GET /v1/usage/by-endpoint — get per-endpoint bandwidth breakdown.
20
+ * Default range is the last 30 days; max range is 366 days.
21
+ */
22
+ byEndpoint(params) {
23
+ return this.client.request('GET', `/v1/usage/by-endpoint${toQueryString(params)}`);
24
+ }
25
+ }
@@ -0,0 +1,20 @@
1
+ import type { ByteKit } from '../client.js';
2
+ import type { operations } from '../generated/api.js';
3
+ import type { WebhookDeliveryId } from '../types.js';
4
+ /**
5
+ * Query params of GET /v1/webhook-deliveries.
6
+ * Derived from the generated OpenAPI types so it stays in sync with the spec.
7
+ */
8
+ export type ListWebhookDeliveriesParams = NonNullable<operations['listWebhookDeliveries']['parameters']['query']>;
9
+ /** 200 payload of GET /v1/webhook-deliveries. */
10
+ export type ListWebhookDeliveriesResponse = operations['listWebhookDeliveries']['responses'][200]['content']['application/json'];
11
+ /** 200 payload of POST /v1/webhook-deliveries/{id}/retry. */
12
+ export type RetryWebhookDeliveryResponse = operations['retryWebhookDelivery']['responses'][200]['content']['application/json'];
13
+ export declare class WebhooksResource {
14
+ private readonly client;
15
+ constructor(client: ByteKit);
16
+ /** GET /v1/webhook-deliveries — list webhook deliveries. */
17
+ list(params?: ListWebhookDeliveriesParams): Promise<ListWebhookDeliveriesResponse>;
18
+ /** POST /v1/webhook-deliveries/{id}/retry — requeue a failed or exhausted delivery. */
19
+ retry(id: WebhookDeliveryId): Promise<RetryWebhookDeliveryResponse>;
20
+ }
@@ -0,0 +1,15 @@
1
+ import { toQueryString } from './query-string.js';
2
+ export class WebhooksResource {
3
+ client;
4
+ constructor(client) {
5
+ this.client = client;
6
+ }
7
+ /** GET /v1/webhook-deliveries — list webhook deliveries. */
8
+ list(params) {
9
+ return this.client.request('GET', `/v1/webhook-deliveries${toQueryString(params)}`);
10
+ }
11
+ /** POST /v1/webhook-deliveries/{id}/retry — requeue a failed or exhausted delivery. */
12
+ retry(id) {
13
+ return this.client.request('POST', `/v1/webhook-deliveries/${id}/retry`);
14
+ }
15
+ }
package/dist/types.d.ts CHANGED
@@ -1,16 +1,16 @@
1
1
  /**
2
- * Prefixed-ID branded types for all RapidCrawl resource identifiers.
2
+ * Prefixed-ID branded types for all ByteKit resource identifiers.
3
3
  *
4
4
  * These mirror the `ss_`, `rec_`, `bulk_`, etc. prefixes used at the API
5
5
  * boundary (internal rows use raw UUIDs).
6
6
  */
7
7
  export type ScrapeId = `sc_${string}`;
8
8
  export type ScreenshotId = `ss_${string}`;
9
- export type RecordingId = `rec_${string}`;
10
9
  export type BulkId = `bulk_${string}`;
11
10
  export type MonitorId = `mon_${string}`;
12
11
  export type SitemapId = `sm_${string}`;
13
12
  export type ApiKeyId = `ak_${string}`;
13
+ export type WebhookDeliveryId = `wh_${string}`;
14
14
  /** Which search index to query. `web` (default), `news`, or `images`. */
15
15
  export type SearchType = 'web' | 'news' | 'images';
16
16
  /** Recency window for search results (maps to the upstream time filter). */
@@ -66,8 +66,11 @@ export interface SearchResult {
66
66
  url: string;
67
67
  /** Short text excerpt for the result. Omitted for `images` results. */
68
68
  snippet: string;
69
- /** Published date when the provider reports one, else `null`. */
70
- date: string | null;
69
+ /**
70
+ * Published date when the provider reports one, else `null`. Present for
71
+ * `web`/`news` results; omitted entirely (`undefined`) for `images` results.
72
+ */
73
+ date?: string | null;
71
74
  /** Publisher / source name. Present for `news` and `images` results. */
72
75
  source?: string;
73
76
  /** Hosting-page domain of the result. Present only for `images` results. */
package/dist/types.js CHANGED
@@ -1,5 +1,5 @@
1
1
  /**
2
- * Prefixed-ID branded types for all RapidCrawl resource identifiers.
2
+ * Prefixed-ID branded types for all ByteKit resource identifiers.
3
3
  *
4
4
  * These mirror the `ss_`, `rec_`, `bulk_`, etc. prefixes used at the API
5
5
  * boundary (internal rows use raw UUIDs).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@hunt-labs/bytekit-sdk",
3
- "version": "0.2.2",
3
+ "version": "0.3.0",
4
4
  "description": "Official TypeScript SDK for the ByteKit API",
5
5
  "license": "MIT",
6
6
  "type": "module",