@boy-offi9-inc/reqkit 0.2.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,6 +5,8 @@
5
5
  </p>
6
6
 
7
7
  <p align="center">
8
+ <a href="https://www.npmjs.com/package/@boy-offi9-inc/reqkit"><img src="https://img.shields.io/npm/v/@boy-offi9-inc/reqkit?color=CB3837&logo=npm&logoColor=white" alt="npm version"></a>
9
+ <a href="https://github.com/boy-offi9-inc/reqkit/actions/workflows/ci.yml"><img src="https://github.com/boy-offi9-inc/reqkit/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
8
10
  <a href="./LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
9
11
  <img src="https://img.shields.io/badge/node-%3E%3D18-339933?logo=node.js&logoColor=white" alt="Node >=18">
10
12
  <img src="https://img.shields.io/badge/npm-%40boy--offi9--inc%2Freqkit-CB3837?logo=npm&logoColor=white" alt="npm package">
@@ -15,27 +17,96 @@
15
17
 
16
18
  # reqkit
17
19
 
18
- Small, composable HTTP helper functions. Not a client, not an axios replacement — works alongside `fetch`, `axios`, or anything else you're already using. Pull in the one function you need; ignore the rest.
20
+ **Tiny, zero-dependency HTTP helpers for `fetch`, axios, or anything else — retries that respect `Retry-After`, timeouts that really cancel, and errors that make sense.**
21
+
22
+ Not a client, not an axios replacement. Pull in the one function you need, or let [`createFetch`](#createfetchdefaults) wire them together for you — you always get a plain `Response` back.
19
23
 
20
24
  ```js
21
- const { withRetry, normalizeError } = require("@boy-offi9-inc/reqkit");
25
+ import { createFetch } from "@boy-offi9-inc/reqkit";
26
+
27
+ const api = createFetch({
28
+ baseUrl: "https://api.example.com/v1",
29
+ headers: () => ({ authorization: `Bearer ${getToken()}` }),
30
+ timeout: 10_000, // per attempt
31
+ });
22
32
 
23
- const data = await withRetry(() => fetch(url).then(r => r.json()));
33
+ const user = await api.json("/users/1", { query: { expand: "team" } });
34
+ // retries network errors, timeouts and 429/5xx, waits for Retry-After,
35
+ // doesn't retry a POST unless you opt in, throws HTTPError on non-2xx
24
36
  ```
25
37
 
38
+ Or take just the pieces you want:
39
+
40
+ ```js
41
+ import { createLimiter, withRetry, withTimeout } from "@boy-offi9-inc/reqkit";
42
+
43
+ const limit = createLimiter(4); // 4 downloads at a time
44
+ const pages = await limit.map(urls, (url) =>
45
+ withRetry(
46
+ () => withTimeout((signal) => fetch(url, { signal }), 8000),
47
+ { retryStatusCodes: [429, 503] } // and it waits for Retry-After
48
+ )
49
+ );
50
+ ```
26
51
  ---
27
52
 
28
- ## Why this exists
53
+ ## Playground
54
+
55
+ Try everything in your browser — retries obeying `Retry-After`, rate limiting, the circuit breaker, caching, SSE streaming and a resumable download that survives a dropped connection. It runs against a fake in-memory server, so there's nothing to install and no network involved.
56
+
57
+ Run it locally: `npm run build && python3 -m http.server -d docs` → open `http://localhost:8000`. (ES modules don't load from `file://`, so it needs a server.) The repo's `pages.yml` workflow publishes the same folder to GitHub Pages — enable it once under Settings → Pages → Source: "GitHub Actions".
29
58
 
30
- Most HTTP retry/timeout wrappers bundle everything into a client you have to fully adopt. This is the opposite: standalone functions that compose with whatever you're already using, and a couple of small helpers that you can pick up independently.
59
+ ## Works with what you already use
31
60
 
32
- **Honest note:** `withRetry`, `withTimeout`, and `normalizeError` solve a well-covered problem — there are other solid packages doing similar things (`fetchy`, `fetchpilot`, `p-retry`, to name a few).
61
+ Recipes for **axios, ky, got, undici/node-fetch**, testing, and LLM streaming live in [RECIPES.md](./RECIPES.md): keep your client, bolt on just the piece you're missing (`Retry-After`-aware retries, a rate limiter, a circuit breaker).
33
62
 
34
- - **`retryAfterAware`** — actually reads `Retry-After`/`X-RateLimit-Reset` headers instead of blind backoff on 429s. Most retry libraries treat rate-limit responses like any other failure and ignore the server's suggested retry time.
35
- - **`withProgress`** — progress callback for any streamed response body (downloads, large payloads, anything), not tied to a specific HTTP client.
36
- - **`withDedupe`** — coalesces concurrent calls with the same key into one in-flight call, so simultaneous requests for the same resource don't each hit the network.
63
+ ## Runtime support
37
64
 
38
- Zero dependencies. CJS + ESM. TypeScript types included.
65
+ | Runtime | Status |
66
+ |---|---|
67
+ | **Node 18, 20, 22, 24** | Primary target. Full test suite runs in CI on all four. |
68
+ | **Deno, Bun, browsers, Cloudflare Workers** | Use the dependency-free ESM bundle (`@boy-offi9-inc/reqkit/browser`, or `import … from "npm:@boy-offi9-inc/reqkit"` in Deno). Only web-standard APIs are used. CI smoke-tests the bundle on Deno, Bun and headless Chromium — **experimental** until those jobs have a track record; please open an issue if one misbehaves. |
69
+
70
+ ```html
71
+ <script type="module">
72
+ import { createFetch } from "https://cdn.jsdelivr.net/npm/@boy-offi9-inc/reqkit@0.5.0/dist/reqkit.mjs";
73
+ </script>
74
+ ```
75
+
76
+ ---
77
+
78
+ ## Pick what you need
79
+
80
+ | You want to… | Use |
81
+ |---|---|
82
+ | Retry on errors *and* bad statuses, honoring `Retry-After` | [`withRetry`](#withretryfn-opts) / [`retryAfterAware`](#retryafterawarefn-opts) |
83
+ | Time out *and actually cancel* the request | [`withTimeout`](#withtimeoutfn-ms-opts) |
84
+ | One error shape from fetch, axios and node (incl. `fetch failed`) | [`normalizeError`](#normalizeerrorerr) |
85
+ | Parse JSON without try/catch | [`parseJsonSafe`](#parsejsonsafeinput) |
86
+ | Build a query string from an object | [`buildQuery`](#buildqueryparams) |
87
+ | Download progress and size caps for any stream | [`withProgress`](#withprogressresponse-onprogress-opts) |
88
+ | Merge identical concurrent calls into one | [`withDedupe`](#withdedupefn-opts) |
89
+ | Run N things at a time | [`createLimiter`](#createlimiterconcurrency) |
90
+ | Slow down *before* the server says 429 (obey `X-RateLimit-*`) | [`createRateLimiter`](#createratelimiteropts) |
91
+ | Stop hammering a service that keeps failing | [`createCircuitBreaker`](#createcircuitbreakeropts) |
92
+ | Cache GETs (TTL, stale-while-revalidate, ETag/304) | [`createFetch` → `cache`](#caching) |
93
+ | Resume a download after the connection drops | [`downloadResumable`](#downloadresumableurl-opts) |
94
+ | Read SSE / NDJSON / lines from a streamed body (LLM tokens!) | [`readSSE` / `readNDJSON` / `readLines`](#streaming-readsse-readndjson-readlines) |
95
+ | Test retry/timeout code without a network | [`createMockFetch`](#createmockfetch) |
96
+ | All of the above, wired up | [`createFetch`](#createfetchdefaults) |
97
+
98
+ ---
99
+
100
+ ## Why this exists
101
+
102
+ Most HTTP retry/timeout wrappers bundle everything into a client you have to fully adopt. This is the opposite: standalone functions that compose with whatever you're already using. Zero dependencies, CJS + ESM, TypeScript types included.
103
+
104
+ **Honest note:** retries, timeouts and error normalization are a well-covered problem — there are other solid packages doing similar things (`fetchy`, `fetchpilot`, `p-retry`, to name a few). What reqkit leans on:
105
+
106
+ - **It reads the server.** Retries wait for `Retry-After` / `X-RateLimit-Reset` instead of blind backoff on 429s — most retry libraries treat rate-limit responses like any other failure.
107
+ - **It's safe by default.** `createFetch` only retries idempotent methods, so a timed-out `POST` can't double-charge anyone.
108
+ - **It understands real-world errors.** Node's `TypeError: fetch failed` and the browser's `Failed to fetch` are classified as network errors, and a deliberate abort is never reported as a timeout.
109
+ - **It's not tied to a client.** `withProgress`, `withDedupe` and `createLimiter` work with any streamed body or async function.
39
110
 
40
111
  ---
41
112
 
@@ -51,23 +122,36 @@ npm install @boy-offi9-inc/reqkit
51
122
 
52
123
  ### `withRetry(fn, opts?)`
53
124
 
54
- Retries an async function with exponential backoff + jitter.
125
+ Retries an async function with exponential backoff + jitter. Retries on thrown errors, and — opt in — on *results* (HTTP status codes or any predicate), waiting for `Retry-After` / `X-RateLimit-Reset` when the result has headers.
55
126
 
56
127
  ```js
57
128
  const data = await withRetry(
58
129
  () => fetch(url).then(r => r.json()),
59
130
  { retries: 3, baseDelayMs: 300, onRetry: (err, attempt, delayMs) => console.log(`retry ${attempt} in ${delayMs}ms`) }
60
131
  );
132
+
133
+ // fetch doesn't throw on 4xx/5xx — retry on statuses by returning the Response:
134
+ const res = await withRetry(() => fetch(url), { retryStatusCodes: [429, 500, 502, 503, 504] });
61
135
  ```
62
136
 
63
137
  | Option | Default | Description |
64
138
  |---|---|---|
65
- | `retries` | `3` | max retry attempts after the first try |
139
+ | `retries` | `3` | max retry attempts after the first try (`Infinity` works with `maxRetryTime`) |
66
140
  | `baseDelayMs` | `300` | initial delay |
67
- | `maxDelayMs` | `10000` | delay ceiling |
68
- | `shouldRetry` | retries anything but AbortError | `(err, attempt) => boolean` |
69
- | `onRetry` | — | `(err, attempt, delayMs) => void` |
70
- | `signal` | — | `AbortSignal` — cancels the whole retry loop, including any pending backoff wait, throwing `RetryAbortedError` |
141
+ | `maxDelayMs` | `10000` | backoff ceiling |
142
+ | `factor` | `2` | backoff multiplier |
143
+ | `jitter` | `0.2` | extra random fraction on top of the delay; `0` disables; `"full"` picks uniformly in `[0, delay]` (AWS "full jitter" — best when many clients retry at once) |
144
+ | `maxRetryTime` | `Infinity` | total time budget across attempts and waits; once the next wait would blow it, the last error/result is surfaced |
145
+ | `shouldRetry` | retries anything but AbortError | `(err, attempt) => boolean` — for thrown errors |
146
+ | `retryStatusCodes` | `[]` | retry when `result.status` is in this list |
147
+ | `shouldRetryResult` | — | `(result, attempt) => boolean` — retry on any result predicate |
148
+ | `respectRetryAfter` | `true` | wait for `Retry-After` / `X-RateLimit-Reset` from `result.headers` |
149
+ | `maxRetryAfterMs` | `60000` | cap on a header-provided wait |
150
+ | `calculateDelay` | — | `({ attempt, error, result, delayMs }) => ms \| false` — override the wait, or `false` to stop |
151
+ | `onRetry` | — | `(err, attempt, delayMs, { reason, result, status }) => void` — `err` is `undefined` when a result triggered the retry |
152
+ | `signal` | — | `AbortSignal` — cancels the whole retry loop, including any pending backoff wait, throwing `RetryAbortedError` (its `cause` is the last failure) |
153
+
154
+ When retries run out on a *result* (e.g. a 503), the last result is returned rather than thrown — you decide what a bad status means. Bodies of retried responses are cancelled so connections are freed.
71
155
 
72
156
  ```js
73
157
  // Cancel a long retry/backoff sequence externally — e.g. the user navigated away
@@ -76,9 +160,19 @@ const promise = withRetry(() => fetch(url), { retries: 5, signal: controller.sig
76
160
  controller.abort(); // rejects with RetryAbortedError, even mid-backoff
77
161
  ```
78
162
 
79
- > **Heads up:** `fetch` doesn't throw on 4xx/5xx, so `withRetry` alone won't retry a `500`. Throw inside `fn` for the statuses you care about (`if (!r.ok) throw ...`), or use `retryAfterAware` below, which retries on status codes.
163
+ **Stop early with `StopRetry`** — throw it from inside `fn` when an error is not worth retrying (bad credentials, a 400...):
80
164
 
81
- ### `withTimeout(fn, ms)`
165
+ ```js
166
+ const { withRetry, StopRetry } = require("@boy-offi9-inc/reqkit");
167
+
168
+ await withRetry(async () => {
169
+ const res = await fetch(url);
170
+ if (res.status === 401) throw new StopRetry(new Error("bad token")); // rethrown as-is, no more attempts
171
+ return res;
172
+ });
173
+ ```
174
+
175
+ ### `withTimeout(fn, ms, opts?)`
82
176
 
83
177
  Races an async function against a timeout, passing it an `AbortSignal` so the underlying request can actually be cancelled.
84
178
 
@@ -90,6 +184,12 @@ const data = await withTimeout(
90
184
  // throws TimeoutError on timeout, regardless of whether fn cooperates with the signal
91
185
  ```
92
186
 
187
+ Pass your own `signal` to cancel from the outside too — it's linked into the one `fn` receives, and aborting it rejects with its reason even if `fn` ignores the signal. Use `ms: Infinity` to disable the timer and only link the signal. A negative or non-number `ms` throws a `TypeError`.
188
+
189
+ ```js
190
+ await withTimeout(signal => fetch(url, { signal }), 5000, { signal: userController.signal });
191
+ ```
192
+
93
193
  ### `normalizeError(err)`
94
194
 
95
195
  One consistent error shape regardless of source (fetch, axios, node http, generic):
@@ -126,7 +226,7 @@ For full parsing/nested-object support, use [`query-string`](https://www.npmjs.c
126
226
 
127
227
  ### `retryAfterAware(fn, opts?)`
128
228
 
129
- Retries a fetch-like call, honoring `Retry-After`/`X-RateLimit-Reset` headers on 429/503 instead of blind backoff.
229
+ A ready-made preset over `withRetry`: retries a fetch-like call on 429/503, honoring `Retry-After`/`X-RateLimit-Reset` headers instead of blind backoff (no jitter; thrown network errors and timeouts are retried too). Use `withRetry` directly for jitter, budgets, or custom statuses.
130
230
 
131
231
  ```js
132
232
  const response = await retryAfterAware(
@@ -148,7 +248,7 @@ const response = await retryAfterAware(
148
248
 
149
249
  The body of every response that gets retried is cancelled so the connection is freed.
150
250
 
151
- ### `withProgress(response, onProgress?)`
251
+ ### `withProgress(response, onProgress?, opts?)`
152
252
 
153
253
  Reads a streamed `Response` body while reporting progress, returns the collected bytes as a `Uint8Array`.
154
254
 
@@ -193,9 +293,193 @@ const fn = withDedupe((req) => doFetch(req), { keyFn: (req) => req.url });
193
293
 
194
294
  ---
195
295
 
296
+ ### `createLimiter(concurrency)`
297
+
298
+ At most `concurrency` calls run at once; the rest wait in a FIFO queue.
299
+
300
+ ```js
301
+ const limit = createLimiter(4);
302
+
303
+ const files = await limit.map(urls, (url) => download(url)); // results keep input order
304
+ // or one at a time:
305
+ const res = await limit(() => fetch(url));
306
+
307
+ limit.activeCount; // running now
308
+ limit.pendingCount; // waiting
309
+ ```
310
+
311
+ A task that throws or rejects only fails its own promise — the queue keeps going. `concurrency` must be an integer ≥ 1 (or `Infinity`).
312
+
313
+ ### `createRateLimiter(opts?)`
314
+
315
+ A *proactive* limiter: it slows you down before the server has to say 429. It's fed by your own budget (`requests` per `perMs`, sliding window) **and** by what the server tells you — `X-RateLimit-Remaining` / `-Reset`, `RateLimit-*`, and `Retry-After` on 429/503. Once the server says `remaining: 0`, everything pauses until the reset. Callers are served FIFO.
316
+
317
+ ```js
318
+ const rl = createRateLimiter({ requests: 10, perMs: 1000 });
319
+
320
+ const res = await rl.run(() => fetch(url)); // acquire → call → learn from the response headers
321
+ // or by hand:
322
+ await rl.acquire({ signal }); // rejects with the signal's reason if aborted while waiting
323
+ rl.update(response);
324
+ rl.blockedForMs; // time left on a server-imposed pause
325
+ ```
326
+
327
+ Reset headers are understood in both flavors: delta seconds (`30`) and absolute epoch (seconds or ms). In `createFetch`, waiting for the budget never counts against the per-attempt timeout.
328
+
329
+ ### `createCircuitBreaker(opts?)`
330
+
331
+ After `failureThreshold` consecutive failures the circuit **opens** and calls fail instantly with `CircuitOpenError` for `resetTimeoutMs` — so a dying service isn't hammered and your callers don't sit waiting on it. Then it goes **half-open**, lets a probe through, and closes again on success.
332
+
333
+ ```js
334
+ const breaker = createCircuitBreaker({ failureThreshold: 5, resetTimeoutMs: 30_000 });
335
+
336
+ try {
337
+ const res = await breaker.run(() => fetch(url));
338
+ } catch (err) {
339
+ if (err instanceof CircuitOpenError) console.log(`try again in ${err.retryAfterMs}ms`);
340
+ }
341
+ breaker.state; // "closed" | "open" | "half-open"
342
+ ```
343
+
344
+ By default a thrown error (except `AbortError`) or a result with `status >= 500` counts as a failure; override with `isFailure(valueOrError, isError)`. `CircuitOpenError` has `code: "CIRCUIT_OPEN"` and is never classified as a network error, so retries stop the moment the circuit opens.
345
+
346
+ ### Streaming: `readSSE`, `readNDJSON`, `readLines`
347
+
348
+ Async iterators over a streamed body — a `Response`, a web `ReadableStream`, or any async iterable (Node streams). Early `break` and `signal` abort both cancel the stream so the connection is released.
349
+
350
+ ```js
351
+ // LLM-style token stream (Server-Sent Events)
352
+ for await (const { event, data, id } of readSSE(response, { signal })) {
353
+ if (data === "[DONE]") break;
354
+ process.stdout.write(JSON.parse(data).token);
355
+ }
356
+
357
+ for await (const row of readNDJSON(response)) console.log(row); // newline-delimited JSON
358
+ for await (const line of readLines(response)) console.log(line);
359
+ ```
360
+
361
+ `readSSE` follows the spec: multi-line `data:` is joined with `\n`, comments are ignored, `event` defaults to `"message"`, `id` carries over, `retry` is parsed, and an event cut off by the end of the stream is dropped. `readNDJSON` skips blank lines and names the bad line on a parse error (or pass `skipInvalid: true`). Line splitting handles `\n`, `\r\n`, `\r`, a BOM, and multi-byte characters split across chunks.
362
+
363
+ ### `downloadResumable(url, opts?)`
364
+
365
+ Downloads into memory and, if the connection drops mid-body, **resumes with an HTTP `Range` request** instead of starting over. It sends `If-Range` with the first response's strong ETag (or `Last-Modified`), so a file that changed on the server is restarted rather than stitched together wrongly; if the server ignores `Range`, it restarts from zero.
366
+
367
+ ```js
368
+ const bytes = await downloadResumable(url, {
369
+ onProgress: ({ loaded, total, percent }) => console.log(percent),
370
+ maxBytes: 200 * 1024 * 1024, // BodyTooLargeError past this
371
+ retries: 5, // consecutive attempts *without progress*
372
+ signal,
373
+ onRetry: (err, attempt, delayMs, { resumeFrom }) => console.log(`resuming from byte ${resumeFrom}`),
374
+ });
375
+ ```
376
+
377
+ It honors `Retry-After` on 429/503, never retries a 4xx (other than 408/425/429) or a size-limit error, and asks for `Accept-Encoding: identity` so byte offsets line up. Pass `fetch: myCreateFetchInstance` to route it through your own client.
378
+
379
+ ### `createMockFetch()`
380
+
381
+ A scriptable, fetch-compatible test double — for testing code with retries, timeouts, and rate limits without a network.
382
+
383
+ ```js
384
+ const mock = createMockFetch();
385
+
386
+ mock.on("GET", "/users/1")
387
+ .replyOnce(503, "busy", { headers: { "retry-after": "0" } }) // consumed in order…
388
+ .reply(200, { id: 1 }); // …then this, forever
389
+ mock.post(/orders/).failOnce(mock.networkError("ECONNRESET")); // reject like a real network failure
390
+ mock.get("/slow").reply(200, "late", { delayMs: 500 }); // respects AbortSignal → test timeouts
391
+
392
+ const api = createFetch({ fetch: mock.fetch, baseUrl: "https://api.test" });
393
+ await api.json("/users/1"); // { id: 1 } — after one retry
394
+ mock.calls; // [{ url, method, headers, body, json, signal }, ...]
395
+ mock.reset();
396
+ ```
397
+
398
+ String matchers match the full URL, `pathname + search`, or the bare `pathname`; RegExp and `(url) => boolean` work too, and `"*"` matches any method. `replyWith(call => …)` returns a `Response` or `{ status, body, headers }`. Unmatched requests reject with a clear error (and are still recorded in `mock.calls`).
399
+
400
+ ### `createFetch(defaults)`
401
+
402
+ Opt-in assembly of the pieces above into a fetch-like function. It's still plain `fetch` underneath: you get a standard `Response`, and you can pass your own `fetch`.
403
+
404
+ ```js
405
+ const { createFetch } = require("@boy-offi9-inc/reqkit");
406
+
407
+ const api = createFetch({
408
+ baseUrl: "https://api.example.com/v1",
409
+ headers: () => ({ authorization: `Bearer ${getToken()}` }), // object, or a function for changing tokens
410
+ timeout: 10_000, // per attempt
411
+ retry: { retries: 3 }, // false to disable
412
+ dedupe: true, // merge identical concurrent GETs
413
+ });
414
+
415
+ const user = await api.json("/users/1", { query: { expand: ["team", "roles"] } });
416
+ const created = await api("/users", { method: "POST", json: { name: "Ada" } }); // Response
417
+
418
+ try {
419
+ await api("/missing");
420
+ } catch (err) {
421
+ if (err instanceof HTTPError) console.log(err.status, await err.response.json());
422
+ }
423
+
424
+ const admin = api.extend({ headers: { "x-role": "admin" } }); // merges over the parent
425
+ ```
426
+
427
+ | Default / per-request option | Default | Description |
428
+ |---|---|---|
429
+ | `baseUrl` | — | joined onto relative URLs; absolute URLs are left alone |
430
+ | `headers` | — | object, `Headers`, or `() => headers` (sync or async) |
431
+ | `query` *(per request)* | — | object, serialized with `buildQuery` |
432
+ | `json` *(per request)* | — | serialized as the body, `content-type: application/json` set for you |
433
+ | `timeout` | `30000` | per attempt, in ms; `false` disables |
434
+ | `signal` *(per request)* | — | external abort; surfaces as a plain `AbortError`, never retried |
435
+ | `retry` | see below | `false` disables; otherwise `{ retries, statusCodes, methods, baseDelayMs, maxDelayMs, factor, jitter, maxRetryTime, shouldRetry, calculateDelay, onRetry }` |
436
+ | `rateLimit` | — | `true` just learns from `X-RateLimit-*` / `Retry-After`; `{ requests, perMs }` also enforces a budget; or pass a shared [`createRateLimiter`](#createratelimiteropts) |
437
+ | `circuitBreaker` | — | `true` / options → one breaker **per origin**; or pass a shared [`createCircuitBreaker`](#createcircuitbreakeropts), or `(url) => breaker`. An open circuit stops retries immediately. |
438
+ | `cache` | — | in-memory GET cache — see [Caching](#caching). `true` = 60s TTL. Per request: `cache: false` bypasses, `"reload"` refreshes |
439
+ | `dedupe` | `false` | merge identical concurrent `GET`/`HEAD` calls; every caller gets its own readable clone. Requests with their own `signal` are never merged. |
440
+ | `throwHttpErrors` | `true` | throw `HTTPError` for non-2xx (`err.status`, `err.response`, `err.url`, `err.method`) |
441
+ | `fetch` | `globalThis.fetch` | swap in undici, node-fetch, a test double... |
442
+ | `onRequest(ctx)` | — | once per call; may mutate `ctx.url` / `ctx.init` (e.g. add a header) |
443
+ | `onResponse(ctx)` | — | once per call, with the final response, before `HTTPError` is thrown |
444
+
445
+ #### Caching
446
+
447
+ ```js
448
+ const api = createFetch({
449
+ baseUrl: "https://api.example.com",
450
+ cache: { ttl: 30_000, staleWhileRevalidate: 5 * 60_000 },
451
+ });
452
+
453
+ await api.json("/config"); // network
454
+ await api.json("/config"); // fresh → served from memory, no request
455
+ // after 30s: stale → returned instantly, refreshed in the background
456
+ // after 5m30s: revalidated with If-None-Match / If-Modified-Since — a 304 reuses the cached body
457
+ ```
458
+
459
+ | `cache` option | Default | Description |
460
+ |---|---|---|
461
+ | `ttl` | `60000` | serve without asking the server for this long |
462
+ | `staleWhileRevalidate` | `0` | after the TTL, keep serving the stale copy this long while refreshing in the background |
463
+ | `maxEntries` | `100` | size of the built-in LRU store |
464
+ | `maxBodyBytes` | `1 MiB` | bigger responses aren't cached |
465
+ | `store` | built-in LRU | bring your own (`get`/`set`/`delete`, sync or async) — Redis, IndexedDB, a `Map`… |
466
+ | `onEvent` | — | `({ type: "hit" \| "stale" \| "revalidated" \| "miss" \| "store", url })` for logging/metrics |
467
+
468
+ Only `GET` `200` responses are cached; `Cache-Control: no-store` and `Vary: *` are respected. **Every request header is part of the cache key**, so a response fetched with one user's `Authorization` can never be served to another. Each hit is a fresh `Response`, so bodies can be read as many times as you like. `extend()` children share the parent's cache, limiter and breakers.
469
+
470
+ **Retry defaults:** 2 retries on network errors, timeouts, and status `408, 425, 429, 500, 502, 503, 504`, honoring `Retry-After`. Only **idempotent methods** (`GET, HEAD, OPTIONS, PUT, DELETE`) are retried — `POST`/`PATCH` aren't, so a timeout can't double-charge anyone. Opt a request in by sending an `Idempotency-Key` header, or by listing the method in `retry.methods`. Stream bodies are never retried (they can't be replayed).
471
+
472
+ ---
473
+
196
474
  ## Design notes
197
475
 
198
476
  - **Zero dependencies.** Nothing to audit, nothing to break underneath you.
199
477
  - **CJS + ESM.** `require()` and `import` both work.
200
478
  - **Composable, not a client.** Every function takes and returns plain values (`Response`, plain objects, `Uint8Array`) — nothing proprietary to learn.
201
- - **TypeScript types included**, hand-written (no build step to go wrong).
479
+ - **TypeScript types included**, hand-written (no build step to go wrong), with separate entries for CJS and ESM.
480
+
481
+ ---
482
+
483
+ ## Prior art
484
+
485
+ reqkit's building blocks are small, from-scratch implementations of ideas that proved themselves in popular libraries — go look at them, they're great: [`p-retry`](https://github.com/sindresorhus/p-retry) (`StopRetry`, time budgets), [`ky`](https://github.com/sindresorhus/ky) and [`ofetch`](https://github.com/unjs/ofetch) (`HTTPError`, `json`/`query`/`baseUrl`, `extend`, retry statuses), [`got`](https://github.com/sindresorhus/got) and [`axios-retry`](https://github.com/softonic/axios-retry) (retry only idempotent methods, `calculateDelay`), [`p-limit`](https://github.com/sindresorhus/p-limit) (concurrency limiter), [`opossum`](https://github.com/nodeshift/opossum) (circuit breaker), [`bottleneck`](https://github.com/SGrondin/bottleneck) / [`p-throttle`](https://github.com/sindresorhus/p-throttle) (rate limiting), [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) (SSE), [`nock`](https://github.com/nock/nock) / undici's `MockAgent` (mocking), the HTTP caching RFCs (`stale-while-revalidate`, conditional requests), and the AWS Architecture Blog's "full jitter" backoff.
package/RECIPES.md ADDED
@@ -0,0 +1,243 @@
1
+ # Recipes
2
+
3
+ reqkit is a set of small functions, so it slots *around* the HTTP client you already use. Each recipe below keeps your client and adds just the piece you're missing.
4
+
5
+ > The snippets use only reqkit's documented API plus each library's documented hooks/interceptors. The reqkit side is covered by its test suite; the third-party libraries (axios, ky, got) aren't installed in reqkit's CI, so if a snippet drifts with a new major version of one of them, please open an issue.
6
+
7
+ - [axios](#axios) · [ky](#ky) · [got](#got) · [undici / node-fetch / any fetch](#undici--node-fetch--any-fetch)
8
+ - [Deno, Bun, Cloudflare Workers, browsers](#deno-bun-cloudflare-workers-browsers)
9
+ - [Testing your own code](#testing-your-own-code)
10
+ - [LLM streaming (SSE)](#llm-streaming-sse)
11
+
12
+ A tiny helper used in the axios and got recipes — reqkit reads headers through `.get(name)`, and those libraries hand you plain objects:
13
+
14
+ ```js
15
+ const asHeaders = (h) => ({
16
+ get: (name) => (typeof h?.get === "function" ? h.get(name) : h?.[name.toLowerCase()]) ?? null,
17
+ });
18
+ ```
19
+
20
+ ---
21
+
22
+ ## axios
23
+
24
+ axios throws on 4xx/5xx, so retry decisions go in `shouldRetry`. `normalizeError` already understands axios errors (`status`, `isNetworkError`, `isTimeout`, cancellation).
25
+
26
+ **Retries that honor `Retry-After`:**
27
+
28
+ ```js
29
+ import axios from "axios";
30
+ import { withRetry, normalizeError, getWaitMsFromHeaders } from "@boy-offi9-inc/reqkit";
31
+
32
+ const RETRY_STATUS = new Set([408, 425, 429, 500, 502, 503, 504]);
33
+
34
+ export function getWithRetry(url, config) {
35
+ return withRetry(() => axios.get(url, config), {
36
+ retries: 3,
37
+ shouldRetry: (err) => {
38
+ const n = normalizeError(err);
39
+ return !n.isAborted && (n.isNetworkError || n.isTimeout || RETRY_STATUS.has(n.status));
40
+ },
41
+ // use the server's Retry-After when it sent one, otherwise reqkit's backoff
42
+ calculateDelay: ({ error, delayMs }) => {
43
+ const wait = getWaitMsFromHeaders(asHeaders(error?.response?.headers));
44
+ return wait === null ? delayMs : Math.min(wait, 60_000);
45
+ },
46
+ });
47
+ }
48
+ ```
49
+
50
+ Only retry idempotent requests (GET/PUT/DELETE) this way — a retried POST can run twice.
51
+
52
+ **Proactive rate limiting** (slow down *before* the 429) with interceptors:
53
+
54
+ ```js
55
+ import { createRateLimiter } from "@boy-offi9-inc/reqkit";
56
+
57
+ const rl = createRateLimiter({ requests: 10, perMs: 1000 });
58
+ const api = axios.create({ baseURL: "https://api.example.com" });
59
+
60
+ api.interceptors.request.use(async (config) => {
61
+ await rl.acquire({ signal: config.signal });
62
+ return config;
63
+ });
64
+ api.interceptors.response.use(
65
+ (res) => {
66
+ rl.update({ status: res.status, headers: asHeaders(res.headers) }); // learns X-RateLimit-* / Retry-After
67
+ return res;
68
+ },
69
+ (err) => {
70
+ if (err.response) rl.update({ status: err.response.status, headers: asHeaders(err.response.headers) });
71
+ throw err;
72
+ }
73
+ );
74
+ ```
75
+
76
+ **Circuit breaker:**
77
+
78
+ ```js
79
+ import { createCircuitBreaker, CircuitOpenError } from "@boy-offi9-inc/reqkit";
80
+
81
+ const breaker = createCircuitBreaker({
82
+ failureThreshold: 5,
83
+ resetTimeoutMs: 30_000,
84
+ // count network errors and 5xx — not 404s or user cancellations
85
+ isFailure: (err, isError) => isError && !axios.isCancel(err) && (!err.response || err.response.status >= 500),
86
+ });
87
+
88
+ try {
89
+ const { data } = await breaker.run(() => api.get("/users"));
90
+ } catch (err) {
91
+ if (err instanceof CircuitOpenError) showOfflineBanner(err.retryAfterMs);
92
+ }
93
+ ```
94
+
95
+ ---
96
+
97
+ ## ky
98
+
99
+ ky already retries, so the best additions are the pieces it doesn't have: a **rate limiter** and a **circuit breaker**. ky's hooks make the limiter a few lines:
100
+
101
+ ```js
102
+ import ky from "ky";
103
+ import { createRateLimiter, createCircuitBreaker, readSSE } from "@boy-offi9-inc/reqkit";
104
+
105
+ const rl = createRateLimiter({ requests: 5, perMs: 1000 });
106
+
107
+ const api = ky.create({
108
+ prefixUrl: "https://api.example.com",
109
+ hooks: {
110
+ beforeRequest: [async (request) => { await rl.acquire({ signal: request.signal }); }],
111
+ afterResponse: [(request, options, response) => { rl.update(response); }], // ky gives you a real Response
112
+ },
113
+ });
114
+
115
+ const breaker = createCircuitBreaker({
116
+ // ky throws HTTPError for non-2xx with `err.response`
117
+ isFailure: (err, isError) => isError && err.name !== "AbortError" && (!err.response || err.response.status >= 500),
118
+ });
119
+
120
+ const users = await breaker.run(() => api.get("users").json());
121
+ ```
122
+
123
+ Prefer reqkit's retries over ky's? Set `retry: 0` on the ky instance and wrap the call in `withRetry` (with `shouldRetry` using `normalizeError`, as in the axios recipe).
124
+
125
+ ---
126
+
127
+ ## got
128
+
129
+ ```js
130
+ import got from "got";
131
+ import { createRateLimiter } from "@boy-offi9-inc/reqkit";
132
+
133
+ const rl = createRateLimiter({ requests: 10, perMs: 1000 });
134
+
135
+ const api = got.extend({
136
+ prefixUrl: "https://api.example.com",
137
+ hooks: {
138
+ beforeRequest: [async () => { await rl.acquire(); }],
139
+ afterResponse: [
140
+ (response) => {
141
+ rl.update({ status: response.statusCode, headers: asHeaders(response.headers) });
142
+ return response;
143
+ },
144
+ ],
145
+ },
146
+ });
147
+ ```
148
+
149
+ got has its own retry logic (it only retries idempotent methods by default, and honors `Retry-After`), so there's usually no need to add reqkit's.
150
+
151
+ ---
152
+
153
+ ## undici / node-fetch / any fetch
154
+
155
+ `createFetch` takes any fetch-compatible function:
156
+
157
+ ```js
158
+ import { fetch as undiciFetch, Agent } from "undici";
159
+ import { createFetch } from "@boy-offi9-inc/reqkit";
160
+
161
+ const dispatcher = new Agent({ connections: 20 });
162
+ const api = createFetch({
163
+ baseUrl: "https://api.example.com",
164
+ fetch: (url, init) => undiciFetch(url, { ...init, dispatcher }),
165
+ retry: { retries: 3 },
166
+ cache: { ttl: 30_000 },
167
+ });
168
+ ```
169
+
170
+ The standalone helpers (`withRetry`, `withTimeout`, `createLimiter`, `createRateLimiter`, `downloadResumable`, …) don't care which client sits underneath.
171
+
172
+ ---
173
+
174
+ ## Deno, Bun, Cloudflare Workers, browsers
175
+
176
+ reqkit only uses web-standard APIs, and the package ships a dependency-free ESM bundle at `@boy-offi9-inc/reqkit/browser`.
177
+
178
+ ```js
179
+ // Deno
180
+ import { createFetch } from "npm:@boy-offi9-inc/reqkit";
181
+
182
+ // Bun / Node / Workers (via wrangler's bundler)
183
+ import { createFetch } from "@boy-offi9-inc/reqkit";
184
+
185
+ // Browser, no build step — pin the version
186
+ import { createFetch } from "https://cdn.jsdelivr.net/npm/@boy-offi9-inc/reqkit@0.5.0/dist/reqkit.mjs";
187
+ ```
188
+
189
+ Notes: in Workers the in-memory cache and rate limiter live **per isolate**, so treat them as best-effort or pass a `store` backed by KV / Durable Objects. In browsers, `cache` is separate from the HTTP cache the browser already keeps. Node 18+ is the primary, fully tested target; CI also smoke-tests the bundle on Deno, Bun and headless Chromium (those jobs are new — if one is red, open an issue).
190
+
191
+ ---
192
+
193
+ ## Testing your own code
194
+
195
+ Take `fetch` as a parameter (or pass it to `createFetch`) and hand in `createMockFetch` during tests:
196
+
197
+ ```js
198
+ import { createMockFetch, createFetch } from "@boy-offi9-inc/reqkit";
199
+
200
+ test("recovers from a flaky upstream", async () => {
201
+ const mock = createMockFetch();
202
+ mock.get("/profile")
203
+ .failOnce(mock.networkError()) // connection refused
204
+ .replyOnce(503, "busy", { headers: { "retry-after": "0" } }) // then overloaded
205
+ .reply(200, { name: "Ada" }); // then fine
206
+
207
+ const api = createFetch({ fetch: mock.fetch, baseUrl: "https://api.test", retry: { baseDelayMs: 1 } });
208
+
209
+ expect(await api.json("/profile")).toEqual({ name: "Ada" });
210
+ expect(mock.calls).toHaveLength(3);
211
+ });
212
+
213
+ test("gives up politely on a hanging server", async () => {
214
+ const mock = createMockFetch();
215
+ mock.get("/slow").reply(200, "late", { delayMs: 5_000 }); // delayMs respects AbortSignal
216
+ const api = createFetch({ fetch: mock.fetch, baseUrl: "https://api.test", timeout: 50, retry: false });
217
+ await expect(api("/slow")).rejects.toMatchObject({ name: "TimeoutError" });
218
+ });
219
+ ```
220
+
221
+ ---
222
+
223
+ ## LLM streaming (SSE)
224
+
225
+ ```js
226
+ import { readSSE } from "@boy-offi9-inc/reqkit";
227
+
228
+ const res = await fetch("https://api.example.com/v1/chat", {
229
+ method: "POST",
230
+ headers: { "content-type": "application/json", accept: "text/event-stream" },
231
+ body: JSON.stringify({ prompt: "Hello", stream: true }),
232
+ signal, // abort → the stream is cancelled and the connection released
233
+ });
234
+
235
+ let text = "";
236
+ for await (const { data } of readSSE(res, { signal })) {
237
+ if (data === "[DONE]") break;
238
+ text += JSON.parse(data).token;
239
+ render(text);
240
+ }
241
+ ```
242
+
243
+ Want the request itself to be retried, timed out and rate limited? Make it through `createFetch` instead — it returns the same `Response`, so `readSSE(res)` works unchanged.