tunnelfetch 1.8.1 → 1.8.2

package/README.md

@@ -1047,6 +1047,75 @@ subject DN, and only the one anchor a chain lands on is ever decoded, so startup
 the 380 KB bundle (133 KB gzipped) and a request that imports but does not use the package costs
 0 ms.
 
+### Streaming APIs: turn HTTP/2 off
+
+An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
+arriving as hundreds of tiny events rather than a large one arriving in a few chunks. Over HTTP/2
+each of those events is a DATA frame with flow control and a `WINDOW_UPDATE` behind it; over
+HTTP/1.1 it is a chunked-encoding chunk and nothing else. These APIs do not multiplex, so the
+cheaper framing simply wins:
+
+```js
+new Client({ connect, proxy, http2: false });
+```
+
+Measured against a real streaming endpoint through a proxy, 20K tokens in and 8K tokens out:
+
+| | CPU/request | per 1M requests |
+| --- | --- | --- |
+| HTTP/2 | 32 ms | $0.94 |
+| **HTTP/1.1** | **28 ms** | **$0.86** |
+| platform `fetch` — reference; it cannot use a proxy | 5 ms | $0.40 |
+
+**13% for one option, and nothing is given up** — a request that never opens a second stream gains
+nothing from multiplexing.
+
+Two things about this shape are worth knowing because they are counter-intuitive:
+
+**The cost is flat in output length.** 512 tokens and 8000 tokens cost within 2 ms of each other.
+Events are batched by the server and V8 tiers up inside the request, and the two together flatten
+the curve completely. An earlier version of this section modelled it as linear from two nearby
+points and predicted a break-even 15x lower than the truth; do not extrapolate this from a slope.
+
+**The request body is cheap.** A 20K-token prompt is ~108 KB of JSON to serialise, buffer, encrypt
+and frame, and it costs 4 ms on h1 or 6 ms on h2 — 15–20% of the request, not the dominant term.
+
+For scale: at these sizes the model's own bill is about **$0.0088 per request**, so this package's
+CPU is **0.011%** of what you pay. Turn h2 off because it is free, not because it will show up on an
+invoice.
+
+### Streaming APIs, and what they cost
+
+An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
+arriving as one event per output token rather than a large one in a few chunks. The per-chunk costs
+that are a footnote for a 4 MB page are the whole bill here.
+
+Measured against a real streaming endpoint through a proxy, with the API's own `usage` block as the
+token count rather than an estimate from response size:
+
+| | per event | per 1M output tokens |
+| --- | --- | --- |
+| this package | **250–310 µs** | $5,000–6,200 |
+| platform `fetch` — reference; it cannot use a proxy | ~105 µs | $2,100 |
+
+Events map to output tokens roughly 1:1, so a 128K-token completion is about **35 s of CPU**. That
+is spread across the minutes the model takes to generate it — utilisation is 2–5%, so it is a
+billing question, not a capacity one.
+
+**As a share of what you pay, it is constant.** CPU and the model's output charge both scale with
+output tokens, so the ratio does not move with length: at $0.6/M output it is **~0.9% of the model
+bill**, at any size. Input tokens appear only in the denominator and push it lower.
+
+A range is given rather than a figure because this measurement repeats to about ±20%. Two things
+were tried and did not survive that: HTTP/1.1 came out ahead of HTTP/2 in one sweep and behind in
+the next, and turning decoding off measured both faster and slower. **Neither is a recommendation.**
+
+Splitting the gap to the platform's own `fetch`: decoding is a small part of it and the transport is
+~155 µs per event, spread across the record layer, framing, the byte reader, the body stream and the
+deadline wrapper — a few tens of microseconds each, with no single layer to remove. That is why the
+`IdentityTransformStream` win on large bodies has no equivalent here: there, one JavaScript layer
+could be deleted outright; here there are seven, each small.
+
 ### Plan limits
 
 On the paid plan the 30 s default CPU limit is not the binding constraint — that is roughly 3 000

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tunnelfetch",
-  "version": "1.8.1",
+  "version": "1.8.2",
   "description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
   "keywords": [
     "fetch",

package/src/util/deadline.js

@@ -178,22 +178,31 @@ export class DeadlineController {
    * Reject as soon as the signal aborts, resolve when `promise` settles first.
    * The abort reason is preserved so the caller sees the typed TimeoutError, not a generic abort.
    */
+  /**
+   * A promise that rejects when this controller aborts, created once and reused.
+   *
+   * `race` used to register and unregister an abort listener PER CALL, which is per chunk on a body
+   * stream. That is invisible on a 4 MB response — a few dozen chunks — and it is the dominant
+   * per-event cost on an SSE stream, where a single completion can be a hundred thousand chunks of
+   * a few hundred bytes each. The signal does not change over the life of the controller, so one
+   * registration is enough.
+   *
+   * Pre-observed, because a rejection nobody has awaited yet is an unhandled rejection.
+   */
+  get _abortPromise() {
+    if (!this.__abortP) {
+      this.__abortP = new Promise((_, reject) => {
+        if (this.aborted) reject(this.signal.reason);
+        else this.signal.addEventListener('abort', () => reject(this.signal.reason), { once: true });
+      });
+      this.__abortP.catch(() => {});
+    }
+    return this.__abortP;
+  }
+
   race(promise) {
     if (this.aborted) return Promise.reject(this.signal.reason);
-    return new Promise((resolve, reject) => {
-      const onAbort = () => reject(this.signal.reason);
-      this.signal.addEventListener('abort', onAbort, { once: true });
-      promise.then(
-        (v) => {
-          this.signal.removeEventListener('abort', onAbort);
-          resolve(v);
-        },
-        (e) => {
-          this.signal.removeEventListener('abort', onAbort);
-          reject(e);
-        },
-      );
-    });
+    return Promise.race([promise, this._abortPromise]);
   }
 }
 

package/types/util/deadline.d.ts

@@ -106,6 +106,19 @@ export class DeadlineController {
      * Reject as soon as the signal aborts, resolve when `promise` settles first.
      * The abort reason is preserved so the caller sees the typed TimeoutError, not a generic abort.
      */
+    /**
+     * A promise that rejects when this controller aborts, created once and reused.
+     *
+     * `race` used to register and unregister an abort listener PER CALL, which is per chunk on a body
+     * stream. That is invisible on a 4 MB response — a few dozen chunks — and it is the dominant
+     * per-event cost on an SSE stream, where a single completion can be a hundred thousand chunks of
+     * a few hundred bytes each. The signal does not change over the life of the controller, so one
+     * registration is enough.
+     *
+     * Pre-observed, because a rejection nobody has awaited yet is an unhandled rejection.
+     */
+    get _abortPromise(): Promise<any>;
+    __abortP: Promise<any> | undefined;
     race(promise: any): Promise<any>;
 }
 export type DeadlineOptions = {