tunnelfetch 1.8.0 → 1.8.2

package/README.md

@@ -819,16 +819,21 @@ Workers Standard bills $5/month including 10 million requests and 30 million CPU
 $0.30 per additional million requests and $0.02 per additional million CPU milliseconds. Applying
 the measurements above, with the charge split out so it is clear what is yours to change:
 
-| Workload | CPU/request | 10M/mo, cold | 10M/mo, warmed | 1B/mo, cold | 1B/mo, warmed |
-| --- | --- | --- | --- | --- | --- |
-| Platform `fetch`, 16 KB — reference; it cannot use a proxy | 0.3 ms | $5.00 | $5.00 | $307.40 | $307.40 |
-| Platform `fetch`, 4 MB — same reference, measured | 3.2 ms | $5.04 | $5.04 | $365.40 | $365.40 |
-| Pooled connection, 16 KB pages | 3.1 ms | $5.90 | $5.24 | $451.20 | $385.20 |
-| New connection per request, 16 KB | 10.6 ms | $7.41 | $6.75 | $602.20 | $536.20 |
-| Pooled connection, 1 MB pages | 53.3 ms | $15.94 | $15.28 | $1455.20 | $1389.20 |
-| New connection per request, 1 MB | 60.8 ms | $17.45 | $16.79 | $1606.20 | $1540.20 |
-| Pooled connection, 4 MB pages | 118.3 ms | $28.94 | $28.28 | $2755.20 | $2689.20 |
-| New connection per request, 4 MB | 125.8 ms | $30.45 | $29.79 | $2906.20 | $2840.20 |
+| Workload | CPU/request | 10M/mo | 1B/mo |
+| --- | --- | --- | --- |
+| Platform `fetch`, 16 KB — reference; it cannot use a proxy | 0.3 ms | $8.06 | $311.00 |
+| Platform `fetch`, 4 MB — same reference, measured | 3.2 ms | $8.64 | $369.00 |
+| Pooled connection, 16 KB pages | 3.5 ms | $8.70 | $375.00 |
+| New connection per request, 16 KB | 13.5 ms | $10.70 | $575.00 |
+| Pooled connection, 1 MB pages | 51.5 ms | $18.30 | $1,335.00 |
+| New connection per request, 1 MB | 61.5 ms | $20.30 | $1,535.00 |
+| Pooled connection, 4 MB pages | 104 ms | $28.80 | $2,385.00 |
+| New connection per request, 4 MB | 114 ms | $30.80 | $2,585.00 |
+
+These follow the CPU table above and nothing else. An earlier version of this section was computed
+from a superseded set of measurements and was left behind when that table was replaced, so the
+document quoted 118.3 ms and 104 ms for the same row in two places. Any figure here that does not
+fall out of the table above is a bug in this README.
 
 The reference row is given at two sizes because the platform's own `fetch` is **not flat** — it
 scales at about 0.82 ms per decompressed MB, measured on a size ladder from one CDN so that only the
@@ -851,14 +856,14 @@ The rows above are the default identity: gzip on the wire, AES-256-GCM, x25519.
 bundles every change together, which is not much use for deciding. Priced one at a time against a
 pooled 1 MB workload at a billion requests a month, warmed:
 
-| Change from the baseline | CPU/request | 1B/mo, warmed | Δ | Paid when |
+| Change from the baseline | CPU/request | 1B/mo | Δ | Paid when |
 | --- | --- | --- | --- | --- |
-| baseline — gzip, AES-256-GCM, x25519 | 53.3 ms | $1,389 | — | always |
-| origin serves `br` instead of gzip | 57.6 ms | $1,474 | **+$85** | the origin chooses `br` |
-| server selects ChaCha20-Poly1305 | 56.3 ms | $1,448 | **+$59** | the server picks it over AES |
-| origin serves `zstd` instead of gzip | 56.1 ms | $1,444 | **+$55** | the origin chooses `zstd` |
-| X25519MLKEM768, 1 request per connection | 61.0 ms | $1,542 | **+$153** | every handshake |
-| X25519MLKEM768, 20 requests per connection | 53.3 ms | $1,390 | **+$0.15** | the same handshake, amortised |
+| baseline — gzip, AES-256-GCM, x25519 | 51.5 ms | $1,335 | — | always |
+| origin serves `br` instead of gzip | 55.8 ms | $1,421 | **+$86** | the origin chooses `br` |
+| server selects ChaCha20-Poly1305 | 54.5 ms | $1,395 | **+$60** | the server picks it over AES |
+| origin serves `zstd` instead of gzip | 54.3 ms | $1,391 | **+$56** | the origin chooses `zstd` |
+| X25519MLKEM768, 1 request per connection | 59.2 ms | $1,489 | **+$154** | every handshake |
+| X25519MLKEM768, 20 requests per connection | 51.5 ms | $1,335 | **+$0.15** | the same handshake, amortised |
 
 The last two rows are the same 0.15 ms of ML-KEM, and the difference between them is entirely
 connection reuse — which is the point worth taking from this table. Post-quantum key exchange is
@@ -1042,6 +1047,75 @@ subject DN, and only the one anchor a chain lands on is ever decoded, so startup
 the 380 KB bundle (133 KB gzipped) and a request that imports but does not use the package costs
 0 ms.
 
+### Streaming APIs: turn HTTP/2 off
+
+An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
+arriving as hundreds of tiny events rather than a large one arriving in a few chunks. Over HTTP/2
+each of those events is a DATA frame with flow control and a `WINDOW_UPDATE` behind it; over
+HTTP/1.1 it is a chunked-encoding chunk and nothing else. These APIs do not multiplex, so the
+cheaper framing simply wins:
+
+```js
+new Client({ connect, proxy, http2: false });
+```
+
+Measured against a real streaming endpoint through a proxy, 20K tokens in and 8K tokens out:
+
+| | CPU/request | per 1M requests |
+| --- | --- | --- |
+| HTTP/2 | 32 ms | $0.94 |
+| **HTTP/1.1** | **28 ms** | **$0.86** |
+| platform `fetch` — reference; it cannot use a proxy | 5 ms | $0.40 |
+
+**13% for one option, and nothing is given up** — a request that never opens a second stream gains
+nothing from multiplexing.
+
+Two things about this shape are worth knowing because they are counter-intuitive:
+
+**The cost is flat in output length.** 512 tokens and 8000 tokens cost within 2 ms of each other.
+Events are batched by the server and V8 tiers up inside the request, and the two together flatten
+the curve completely. An earlier version of this section modelled it as linear from two nearby
+points and predicted a break-even 15x lower than the truth; do not extrapolate this from a slope.
+
+**The request body is cheap.** A 20K-token prompt is ~108 KB of JSON to serialise, buffer, encrypt
+and frame, and it costs 4 ms on h1 or 6 ms on h2 — 15–20% of the request, not the dominant term.
+
+For scale: at these sizes the model's own bill is about **$0.0088 per request**, so this package's
+CPU is **0.011%** of what you pay. Turn h2 off because it is free, not because it will show up on an
+invoice.
+
+### Streaming APIs, and what they cost
+
+An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
+arriving as one event per output token rather than a large one in a few chunks. The per-chunk costs
+that are a footnote for a 4 MB page are the whole bill here.
+
+Measured against a real streaming endpoint through a proxy, with the API's own `usage` block as the
+token count rather than an estimate from response size:
+
+| | per event | per 1M output tokens |
+| --- | --- | --- |
+| this package | **250–310 µs** | $5,000–6,200 |
+| platform `fetch` — reference; it cannot use a proxy | ~105 µs | $2,100 |
+
+Events map to output tokens roughly 1:1, so a 128K-token completion is about **35 s of CPU**. That
+is spread across the minutes the model takes to generate it — utilisation is 2–5%, so it is a
+billing question, not a capacity one.
+
+**As a share of what you pay, it is constant.** CPU and the model's output charge both scale with
+output tokens, so the ratio does not move with length: at $0.6/M output it is **~0.9% of the model
+bill**, at any size. Input tokens appear only in the denominator and push it lower.
+
+A range is given rather than a figure because this measurement repeats to about ±20%. Two things
+were tried and did not survive that: HTTP/1.1 came out ahead of HTTP/2 in one sweep and behind in
+the next, and turning decoding off measured both faster and slower. **Neither is a recommendation.**
+
+Splitting the gap to the platform's own `fetch`: decoding is a small part of it and the transport is
+~155 µs per event, spread across the record layer, framing, the byte reader, the body stream and the
+deadline wrapper — a few tens of microseconds each, with no single layer to remove. That is why the
+`IdentityTransformStream` win on large bodies has no equivalent here: there, one JavaScript layer
+could be deleted outright; here there are seven, each small.
+
 ### Plan limits
 
 On the paid plan the 30 s default CPU limit is not the binding constraint — that is roughly 3 000

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tunnelfetch",
-  "version": "1.8.0",
+  "version": "1.8.2",
   "description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
   "keywords": [
     "fetch",

package/src/util/deadline.js

@@ -178,22 +178,31 @@ export class DeadlineController {
    * Reject as soon as the signal aborts, resolve when `promise` settles first.
    * The abort reason is preserved so the caller sees the typed TimeoutError, not a generic abort.
    */
+  /**
+   * A promise that rejects when this controller aborts, created once and reused.
+   *
+   * `race` used to register and unregister an abort listener PER CALL, which is per chunk on a body
+   * stream. That is invisible on a 4 MB response — a few dozen chunks — and it is the dominant
+   * per-event cost on an SSE stream, where a single completion can be a hundred thousand chunks of
+   * a few hundred bytes each. The signal does not change over the life of the controller, so one
+   * registration is enough.
+   *
+   * Pre-observed, because a rejection nobody has awaited yet is an unhandled rejection.
+   */
+  get _abortPromise() {
+    if (!this.__abortP) {
+      this.__abortP = new Promise((_, reject) => {
+        if (this.aborted) reject(this.signal.reason);
+        else this.signal.addEventListener('abort', () => reject(this.signal.reason), { once: true });
+      });
+      this.__abortP.catch(() => {});
+    }
+    return this.__abortP;
+  }
+
   race(promise) {
     if (this.aborted) return Promise.reject(this.signal.reason);
-    return new Promise((resolve, reject) => {
-      const onAbort = () => reject(this.signal.reason);
-      this.signal.addEventListener('abort', onAbort, { once: true });
-      promise.then(
-        (v) => {
-          this.signal.removeEventListener('abort', onAbort);
-          resolve(v);
-        },
-        (e) => {
-          this.signal.removeEventListener('abort', onAbort);
-          reject(e);
-        },
-      );
-    });
+    return Promise.race([promise, this._abortPromise]);
   }
 }
 

package/types/util/deadline.d.ts

@@ -106,6 +106,19 @@ export class DeadlineController {
      * Reject as soon as the signal aborts, resolve when `promise` settles first.
      * The abort reason is preserved so the caller sees the typed TimeoutError, not a generic abort.
      */
+    /**
+     * A promise that rejects when this controller aborts, created once and reused.
+     *
+     * `race` used to register and unregister an abort listener PER CALL, which is per chunk on a body
+     * stream. That is invisible on a 4 MB response — a few dozen chunks — and it is the dominant
+     * per-event cost on an SSE stream, where a single completion can be a hundred thousand chunks of
+     * a few hundred bytes each. The signal does not change over the life of the controller, so one
+     * registration is enough.
+     *
+     * Pre-observed, because a rejection nobody has awaited yet is an unhandled rejection.
+     */
+    get _abortPromise(): Promise<any>;
+    __abortP: Promise<any> | undefined;
     race(promise: any): Promise<any>;
 }
 export type DeadlineOptions = {