@hunt-labs/bytekit-sdk 0.9.3 → 0.11.0

package/CHANGELOG.md

@@ -10,6 +10,176 @@ today, see [README.md](./README.md), which remains the authoritative usage guide
 
 ## [Unreleased]
 
+## [0.11.0] - 2026-09-14
+
+### Added
+
+- **`solve_challenge` on `scrape.create` (#4447).** `ScrapeOpts` now carries
+  `solve_challenge?: boolean`, and the generated `ScrapeRequest` schema type carries the field
+  with its server default `false`. When `true`,
+  the scrape attempts to clear a Cloudflare challenge; success is not guaranteed. It is billed
+  as a normal browser render. A scrape that omits the option sends no `solve_challenge` key,
+  so existing calls are unchanged on the wire.
+- **`credits_scrape` on the usage response types (#4452).** The generated `GET /v1/usage`
+  response type and each `GET /v1/usage/daily` data item now carry `credits_scrape: number`:
+  credits drawn under the scrape usage source, such as `/v1/schema` extractions. The gateway
+  always charged this bucket but never returned it, so the published per-source credit fields
+  could not add up to `credits_used`.
+
+### Changed
+
+- **Internal `GET /v1/bulk` is cursor-paginated and its 100-job cap is gone (#4508).** The
+  generated `listBulk` types record a changed response shape: the operation used to return a
+  bare array of at most the 100 newest jobs, and now returns the same page envelope
+  `GET /v1/monitors` does, `{ data, next_cursor, has_more }`, plus `limit` and `cursor` query
+  parameters (`limit` is 1 to 100, default 25). A `status` filter and a `cursor` combine in
+  one request: `status` narrows the set and the cursor addresses a position inside it. The
+  operation's summary also no longer says "in-flight", which was wrong — it has always
+  returned jobs of every status, `cancelled` included. This is a breaking change to the wire
+  shape of an `x-sdk-scope: internal` operation; no SDK method calls it, so no published
+  method signature moves.
+- **`GET /v1/logs` `billing_multiplier` is the bandwidth factor, not a credit count (#4469).**
+  The generated logs-row type records the changed meaning of a public field: the value is now
+  `billable_bytes` / `raw_bytes` rounded to 2 decimal places — the factor the request settled
+  at, carrying the endpoint, cache-hit and clean-markdown factors, and the flat 1.5 Phase-1
+  factor on screenshots and recordings. It used to be the row's persisted `credits_charged`,
+  an account credit count charged per capture option, so two screenshots billed at the same
+  1.5 factor reported `2` and `1`. The 2 decimal places are load-bearing rather than
+  cosmetic: `billable_bytes` is itself rounded, so an odd byte basis would otherwise surface
+  as `1.500001943158721` instead of `1.5` on roughly half of all such rows. Every factor the
+  field can report is a multiple of 0.25, so 2 places is exact for any basis of 100 bytes or
+  more, and `raw_bytes` × `billing_multiplier` rounded to the nearest byte still reproduces
+  `billable_bytes`. The `billable_bytes` doc comment therefore no longer says it is NOT
+  `raw_bytes` × `billing_multiplier`. The field is also null on more rows than before: a
+  search or webhook row has no byte basis, a capture whose Phase 1 was skipped has a zero one,
+  and a capture old enough to predate the Phase-1 basis being recorded has none at all. One
+  residual is documented rather than fixed: a markdown `/v1/fetch` row completed before the
+  billing decomposition was recorded reports 1.0 where settlement charged 1.5, because nothing
+  on such a row distinguishes it. The type shapes are unchanged (`number | null`), and no SDK
+  method calls this endpoint.
+- **An all-null `/v1/schema` extraction your schema allows is validated again (#4490).** The
+  generated `/v1/schema` 200 response type records the exception to #4455's rule: an output
+  whose every leaf is `null` still arrives `validated: true` with the object intact, and is
+  charged, when the schema you sent declares `"null"` in the `type` union of every leaf the
+  output returned as `null`. Only that form counts — `nullable: true`, an `enum` containing
+  `null` and `const: null` do not, and a response that is itself `null` rather than an object is
+  never accepted however the root is declared. Everything else stays the #4455 miss. The `data`
+  doc comment now also states the other half: a `validated: true` response always carries an
+  object, never `null`. The type shapes are unchanged; only their doc comments moved.
+- **An all-null `/v1/schema` extraction is reported, not validated (#4455).** The generated
+  `/v1/schema` 200 response type documents a new additive warning code, `empty_extraction`: a
+  schema-valid output whose every leaf is `null` now arrives with `data: null` and
+  `validated: false`, and is not charged the per-request credits. Previously it arrived as
+  `validated: true` with the all-null object and no warning. The `data` and `validated` doc
+  comments now also say that every `validated: false` response carries `data: null`; the old
+  comment claimed exhausted retries returned the last raw output, which the API never did. The
+  type shapes are unchanged; only their doc comments moved.
+- **An inline `POST /v1/scrape` failure is documented with a persisted `id` (#4454).** The
+  gateway now records a sync scrape that ends `upstream_error` or `blocked` as a failed scrape
+  job, so its `failed` envelope carries that job's `sc_` id instead of `null`, `scrape.get(id)`
+  returns the same envelope, and the request appears in the dashboard Logs. The generated
+  `createScrape` 200 and `ScrapeErrorEnvelope` doc comments and the `scrape.create` doc comment
+  no longer say `id: null`. The type shapes are unchanged (`id` stays `string | null`), and
+  `scrape.create` still throws `ByteKitError` on that envelope exactly as before.
+- **Internal `GET /v1/logs` types list search and webhook rows (#4460).** Both `endpoint` enums
+  add `search` and `webhook`; a row's `source` is now `string | null`. No SDK method calls it.
+
+## [0.10.0] - 2026-09-11
+
+### Changed
+
+- **A terminal `POST /v1/scrape` failure now throws instead of resolving (#1870).** The
+  gateway used to report an upstream fetch failure — unreachable host, refused connection,
+  or bot protection that defeated every proxy tier — as HTTP 503; it now answers **HTTP 200**
+  with the canonical `failed` envelope (`id: null`), because a definitive upstream outcome is
+  a result, not a transport error. `!response.ok` was this SDK's only failure seam, so without
+  this change that body would resolve like a success and silence every caller's `catch`.
+  `scrape.create` (and any `request` / `requestWithResponse` call to `POST /v1/scrape`) now
+  raises `ByteKitError` carrying the envelope's `code` and `message`, the **actual** status
+  `200`, the response headers, and `error.details` when one is supplied. A legacy 503 from a
+  not-yet-upgraded gateway still throws through the unchanged non-2xx decoder, so the SDK
+  works against both. **Upgrade note:** if you inspected `result.status === 'failed'` after
+  `scrape.create`, move that handling into a `catch`.
+- Everything else is untouched: `scrape.get(id)`, bulk item results, and every other endpoint
+  still deliver `failed` as data; `requestRaw` (`/v1/fetch`) never parses JSON and is
+  unaffected; 204 and empty-body handling are unchanged. `request()` now delegates to
+  `requestWithResponse()` so the guard has a single application site — identical in every
+  observable respect but the return value, as both were already documented.
+
+## [0.9.4] - 2026-09-06 [NEVER PUBLISHED]
+
+**This version was cut but NEVER PUBLISHED — npm `latest` goes from 0.9.3 straight to 0.10.0.**
+Publication is triggered by CI on `main` from the version the manifest declares, and 0.10.0
+(#1870) superseded 0.9.4 in the tree before any release reached `main`. Everything below
+reaches consumers in the **0.10.0** artifact, which contains all of it; nothing is installable
+at 0.9.4: `npm i @hunt-labs/bytekit-sdk@0.9.4` fails.
+
+### Added
+
+- Bulk fetch, scrape, and mixed-result reads now accept optional cursor, limit, and status
+  page parameters while preserving the existing `(id, requestOptions?)` call shape (#3717).
+  Generated response types publish stable item identities, page cursors and completeness,
+  whole-job counters, exact mixed artifact keys, artifact warnings, bounded webhook recovery
+  metadata, and the `artifact_unavailable` error contract.
+
+### Fixed
+
+- Corrected the artifact-URL fallback documentation on `GET /v1/bulk/{id}/screenshots`
+  (`image_url`, `content_url`, `markdown_url`) and `GET /v1/logs` (`artifact_url`) (#3717).
+  Those fields were documented as null-plus-`artifact_unavailable` when no signature can be
+  minted; the servers have always returned the row's stored value unchanged instead — `null`
+  while nothing is stored, and otherwise the private unsigned storage URL, which answers `401`
+  — and raise no warning. `GET /v1/fetch/bulk/{id}` does null the field and report
+  `artifact_unavailable`, and its documentation is unchanged. No server behaviour changed and
+  no type moved; the descriptions now match what the endpoints return.
+
+### Changed
+
+- Regenerated `src/generated/api.d.ts` against the corrected `topup_balance_bytes` description
+  on `GET /v1/usage` (#4251). The description read "Remaining purchased top-up bytes", which
+  contradicted its own "Equal to `topup_bytes`" clause: the value covers purchased top-up lots
+  and the one-time starter grant, which is what the endpoint has always returned. The clause
+  recording that the figure is already net of consumed overage, and the "Present only in
+  dual/bytes billing mode" clause, are both unchanged and still true. Types are unchanged and
+  no server value changed; only the documentation on them moved.
+- Regenerated `src/generated/api.d.ts` against the clarified `cache_age_s` description on
+  `ScrapeSuccessEnvelope` (#4237). The description now records that `GET /v1/scrape/{id}`
+  reports `cache: hit` without `cache_age_s`: the age is a property of the replay that happened
+  at completion time, is not persisted on the row, and cannot be derived honestly afterwards.
+  The one-directional "Present only on `cache: hit`" clause is unchanged and still true. Types
+  are unchanged; only the documentation on them moved.
+- Regenerated `src/generated/api.d.ts` against the completed `cache` presence list on
+  `ScrapeSuccessEnvelope` (#4203). The field's description now names `GET /v1/scrape/{id}` as a
+  third delivery path, alongside the synchronous response and webhook delivery: a polled scrape
+  now reports the cache disposition its row recorded, and omits the field — rather than
+  defaulting it to `miss` — for a row that recorded none. Types are unchanged; only the
+  documentation on them moved.
+- Regenerated `src/generated/api.d.ts` against the corrected `PUT /v1/billing/auto-topup`
+  request body (#3984). `monthly_cap_cents` is now typed `number | null`: an explicit `null` is
+  the wire representation of "no monthly cap", and it is what the endpoint has always accepted —
+  the request schema simply never documented it, while the 200 response schema already did.
+  Omitting the key while `enabled` is `true` is still rejected with `422`, so send `null`
+  rather than dropping the field. No server behaviour changes; the type widens to match what
+  the API already accepted.
+- Regenerated `src/generated/api.d.ts` against the corrected monitor `scrape_options`
+  documentation (#4024). The description no longer lists `headers` among the accepted
+  fields — `POST /v1/monitors` never honoured a caller-supplied `headers` option and now
+  rejects it with the same 400 every other capture endpoint returns for the field. Types are
+  unchanged; only the documentation on them moved.
+- Regenerated `src/generated/api.d.ts` against the completed `X-Scrape-Proxy-Tier-Label`
+  omission list (#4030). The header's description now names a third case in which no tier is
+  reported: a response replayed from stored bytes after this request's own cache lookup missed,
+  which arrives as `X-Scrape-Cache: miss` and likewise ran no proxy tier. The two cases already
+  documented — a cache hit, and responses the fast path did not serve — are unchanged. Types are
+  unchanged; only the documentation on them moved.
+- Regenerated `src/generated/api.d.ts` against the qualified `X-Scrape-Proxy-Tier-Label`
+  omission list (#4132). The third case that list records — a response replayed from stored bytes
+  after this request's own cache lookup missed — is not reachable through this API: every caller
+  of the internal fetch path closes the handler branch that produces it, and the branch survives
+  only as a deploy-skew safeguard. The description now says so rather than presenting the case as
+  ordinary behaviour. The two reachable cases — a cache hit, and responses the fast path did not
+  serve — are unchanged. Types are unchanged; only the documentation on them moved.
+
 ## [0.9.3] - 2026-09-04
 
 ### Changed

package/README.md

@@ -300,6 +300,32 @@ try {
 }
 ```
 
+### A failed scrape throws, even though it arrives on a 200
+
+`POST /v1/scrape` reports a terminal upstream fetch failure — an unreachable
+host, a refused connection, or a target whose bot protection defeats every proxy
+tier — as **HTTP 200** carrying the canonical `failed` envelope, because a
+definitive upstream outcome is a result rather than a transport error. The SDK
+turns that one body into the same `ByteKitError` you already catch, so a failed
+scrape can never be mistaken for a successful one:
+
+```ts
+import { ByteKit, ByteKitError } from '@hunt-labs/bytekit-sdk';
+
+try {
+  await client.scrape.create({ url: 'https://does-not-resolve.invalid' });
+} catch (err) {
+  if (err instanceof ByteKitError) {
+    // 200 'upstream_error' 'Failed to fetch URL.'  (or 'blocked')
+    console.error(err.status, err.code, err.message);
+  }
+}
+```
+
+`err.status` is the real transport status, `200` — nothing is synthesized. This
+applies to `scrape.create` only. `scrape.get(id)` and bulk item results deliver
+`failed` as **data**, exactly as before: narrow on `result.status` there.
+
 ### Reading which field was rejected
 
 `err.details` carries the API envelope's structured rejection reason when the

package/dist/client.d.ts

@@ -17,8 +17,8 @@ export interface ByteKitOptions {
      * read (so a large legitimate body counts against it too). `0` disables the SDK-side
      * timeout. Default 120000.
      *
-     * NOT the same thing as `ScrapeOpts.timeout_ms`, which is the SERVER-side budget the
-     * gateway applies to the scrape itself. Override per call with
+     * NOT the same thing as `ScrapeOpts.timeout_ms`, which is the SERVER-side budget
+     * ByteKit applies to the scrape itself. Override per call with
      * `RequestOptions.requestTimeoutMs`.
      */
     timeoutMs?: number;

package/dist/client.js

@@ -214,6 +214,53 @@ function parseJsonBody(text) {
     }
     return JSON.parse(text);
 }
+/**
+ * Throw a `ByteKitError` for the one 2xx body that is a FAILURE: the terminal
+ * `status: 'failed'` envelope `POST /v1/scrape` delivers on HTTP 200 (#1870).
+ *
+ * The gateway used to report a terminal upstream fetch failure as HTTP 503, and
+ * `!response.ok` in {@link ByteKit.send} was this SDK's only failure seam. Now that the
+ * same outcome arrives as an intentional 200 — a definitive upstream result is not a
+ * transport error, and `id` names the failed job the gateway recorded it as —
+ * that seam alone would let the failure resolve like a success and silence every caller's
+ * `catch`. This restores the exception integrators already write code for, with the
+ * envelope's own `code` and `message`.
+ *
+ * Scope is deliberately narrow — `POST` to exactly `/v1/scrape`:
+ *
+ *   - `GET /v1/scrape/{id}`, bulk item results, and the `scrape.failed` webhook payload
+ *     have always delivered `failed` as DATA, and a caller polling a job must keep
+ *     receiving it as a value;
+ *   - `requestRaw` (`/v1/fetch`) never parses JSON and is not routed through here at all.
+ *
+ * `status` is the ACTUAL transport status (200). The envelope's optional
+ * `error.http_status` is the TARGET's status when one was received, never ours, so it is
+ * not substituted here — and the old 503 is not synthesized either.
+ *
+ * Runs on the already-parsed body, outside `send`'s connection-error catch, so it can
+ * never be remapped into a `ByteKitConnectionError`.
+ */
+function throwIfScrapeFailureEnvelope(method, path, data, status, headers) {
+    if (method.toUpperCase() !== 'POST' || path !== '/v1/scrape')
+        return;
+    if (typeof data !== 'object' || data === null)
+        return;
+    const envelope = data;
+    if (envelope.status !== 'failed')
+        return;
+    const error = typeof envelope.error === 'object' && envelope.error !== null
+        ? envelope.error
+        : {};
+    const code = typeof error.code === 'string' ? error.code : 'internal_error';
+    const message = typeof error.message === 'string' ? error.message : 'Scrape failed.';
+    // Same discipline as the non-2xx decoder above: arrays and scalars are rejected,
+    // because `ByteKitErrorDetails` promises something indexable.
+    const rawDetails = error.details;
+    const details = typeof rawDetails === 'object' && rawDetails !== null && !Array.isArray(rawDetails)
+        ? rawDetails
+        : undefined;
+    throw new ByteKitError(status, code, message, details, headers);
+}
 /** The `name` of an error-shaped value, or `undefined` when it carries none. */
 const nameOf = (e) => typeof e === 'object' && e !== null && 'name' in e ? String(e.name) : undefined;
 /** The `cause` of an error-shaped value, or `undefined` when it carries none. */
@@ -477,8 +524,14 @@ export class ByteKit {
         // Only the read runs inside the transport's guarded region (#2546) — `JSON.parse`
         // stays out here (in `parseJsonBody`), so a malformed body still throws its own
         // SyntaxError rather than being remapped to a ByteKitConnectionError.
-        const text = await this.send(method, path, body, options, readSuccessText);
-        return parseJsonBody(text);
+        //
+        // #1870: delegated to `requestWithResponse` so the terminal-scrape-failure guard has
+        // exactly ONE application site across both public JSON transports, rather than a copy
+        // per entrypoint that could drift. The two were already documented as identical in
+        // every observable respect but the return value, and the header/status pair the
+        // delegate captures is what the thrown `ByteKitError` needs anyway.
+        const { data } = await this.requestWithResponse(method, path, body, options);
+        return data;
     }
     /**
      * Make an authenticated request whose response body IS the content, not a JSON
@@ -536,6 +589,11 @@ export class ByteKit {
             headers: response.headers,
             status: response.status,
         }));
-        return { data: parseJsonBody(text), headers, status };
+        const data = parseJsonBody(text);
+        // #1870: the one 2xx body that is a failure. Placed after the parse and outside
+        // `send`'s guarded region, so the throw is a ByteKitError and never a remapped
+        // connection error.
+        throwIfScrapeFailureEnvelope(method, path, data, status, headers);
+        return { data, headers, status };
     }
 }