tunnelfetch 1.8.3 → 1.9.0

package/README.md

@@ -453,6 +453,36 @@ Until 1.6.2 the profile carried curl **8.7.1's** window while presenting curl **
 ClientHello: one named client, two source versions, and a split identity that only a capture could
 find, because each half was individually true of some curl.
 
+**`extensionOrder` arranges extensions; it cannot add them.** The builder filters to the extensions
+it actually generated and sorts those, so an identity needing one this package does not build is not
+reachable by ordering. Chromium sends five that it does not:
+
+| | |
+| --- | --- |
+| `signed_certificate_timestamp` | 18 |
+| `compress_certificate` | 27 |
+| `session_ticket` | 35 |
+| `application_settings` | 17613 |
+| `encrypted_client_hello` | 65037 |
+
+So `profiles.chrome` presents a Chromium cipher list, group list and GREASE placement over an
+extension **set** that is curl's. A test reads that gap straight out of the committed Chromium
+capture and fails if it changes, so it cannot drift quietly — but it is a real difference and a
+JA3/JA4 hash sees it.
+
+`tls.extraExtensions` takes pre-encoded extensions and is the only way to close it. They are ordered
+like any other and reproduced on a HelloRetryRequest retry, because a second hello that changed its
+extension set would be both malformed (RFC 8446 §4.1.2) and a signal in itself. Encoding them
+correctly is the caller's job; this package does not parse what it did not build.
+
+**Every offered cipher must be one this package can perform.** `tls.ciphers` used to be taken
+verbatim, so a list containing a CBC or RSA-key-exchange suite put a number on the wire that a
+server could select — after which the AEAD layer had nothing to build and the connection died
+mid-handshake. Such a list is now refused at configuration time. An explicit `TLS_CHACHA20_POLY1305_SHA256`
+without an injected implementation is refused too, rather than silently dropped: quietly presenting
+a different fingerprint from the one you asked for is the worst outcome available to a package like
+this one.
+
 Extension order matters because JA3 and JA4 hash the extension list **in wire order**, so it is most
 of what a fingerprinter reads. `pre_shared_key` is forced last whatever you ask for: RFC 8446
 §4.2.11 defines the binder transcript as the hello truncated just before the binders, which is a
@@ -1047,43 +1077,6 @@ subject DN, and only the one anchor a chain lands on is ever decoded, so startup
 the 380 KB bundle (133 KB gzipped) and a request that imports but does not use the package costs
 0 ms.
 
-### Streaming APIs: turn HTTP/2 off
-
-An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
-arriving as hundreds of tiny events rather than a large one arriving in a few chunks. Over HTTP/2
-each of those events is a DATA frame with flow control and a `WINDOW_UPDATE` behind it; over
-HTTP/1.1 it is a chunked-encoding chunk and nothing else. These APIs do not multiplex, so the
-cheaper framing simply wins:
-
-```js
-new Client({ connect, proxy, http2: false });
-```
-
-Measured against a real streaming endpoint through a proxy, 20K tokens in and 8K tokens out:
-
-| | CPU/request | per 1M requests |
-| --- | --- | --- |
-| HTTP/2 | 32 ms | $0.94 |
-| **HTTP/1.1** | **28 ms** | **$0.86** |
-| platform `fetch` — reference; it cannot use a proxy | 5 ms | $0.40 |
-
-**13% for one option, and nothing is given up** — a request that never opens a second stream gains
-nothing from multiplexing.
-
-Two things about this shape are worth knowing because they are counter-intuitive:
-
-**The cost is flat in output length.** 512 tokens and 8000 tokens cost within 2 ms of each other.
-Events are batched by the server and V8 tiers up inside the request, and the two together flatten
-the curve completely. An earlier version of this section modelled it as linear from two nearby
-points and predicted a break-even 15x lower than the truth; do not extrapolate this from a slope.
-
-**The request body is cheap.** A 20K-token prompt is ~108 KB of JSON to serialise, buffer, encrypt
-and frame, and it costs 4 ms on h1 or 6 ms on h2 — 15–20% of the request, not the dominant term.
-
-For scale: at these sizes the model's own bill is about **$0.0088 per request**, so this package's
-CPU is **0.011%** of what you pay. Turn h2 off because it is free, not because it will show up on an
-invoice.
-
 ### Streaming APIs, and what they cost
 
 An SSE response from an LLM API is the opposite shape to everything else measured here: a small body

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tunnelfetch",
-  "version": "1.8.3",
+  "version": "1.9.0",
   "description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
   "keywords": [
     "fetch",

package/src/tls/connect.js

@@ -137,7 +137,16 @@ function expectServerHello(msg, offers12) {
  * @property {number[]} [groups] supported_groups, in preference order.
  * @property {number[]} [offerGroups] groups to send an actual key_share for. Default the first
  *   supported group; a HelloRetryRequest recovers any other choice at the cost of a round trip.
- * @property {number[]} [ciphers] cipher suites to offer, in preference order.
+ * @property {number[]} [ciphers] cipher suites to offer, in preference order. Every suite must be
+ *   one this package can perform; an offer it cannot honour is a dead connection the moment a
+ *   server selects it, so an unknown suite is refused here rather than on the wire.
+ * @property {Uint8Array[]} [extraExtensions] pre-encoded ClientHello extensions, appended before
+ *   ordering. `extensionOrder` can only arrange extensions that were BUILT — it filters to what
+ *   exists and sorts that — so ordering alone cannot produce an extension this package does not
+ *   generate. Chromium sends several that it does not: `signed_certificate_timestamp` (18),
+ *   `compress_certificate` (27), `session_ticket` (35), `application_settings` (17613) and ECH
+ *   (65037). Supplying them here is the only way to close that gap, and it is the caller's job to
+ *   encode them correctly — this package does not parse what it did not build.
  * @property {number[]} [sigSchemes] signature_algorithms to offer, in preference order.
  * @property {boolean | number} [grease] send GREASE (RFC 8701) reserved values in the cipher list,
  *   the extension list (one at each end), supported_groups, supported_versions and key_share.
@@ -328,6 +337,38 @@ async function drive({ record, hostname, verifyPeer, options, deps, versions })
   ];
   if (!chacha) ciphers = ciphers.filter((c) => c !== CIPHER.TLS_CHACHA20_POLY1305_SHA256);
 
+  // A caller-supplied list was previously taken verbatim. That let a hello offer suites this
+  // package cannot perform — the CBC and RSA-key-exchange suites, most obviously — and the failure
+  // did not surface at configuration time. It surfaced after the ClientHello was on the wire, when
+  // a server took the offer up and the AEAD layer had nothing to build: a dead connection rather
+  // than a rejected config.
+  //
+  // profiles.js already states the principle for the profile system: "a fingerprint field this
+  // package cannot perform is an offer a server may take and then find unhonoured, which fails the
+  // connection rather than merely looking wrong". It applies just as much to `tls.ciphers`, and
+  // now does. CIPHER_PARAMS is the set this package knows how to key and seal.
+  if (options.ciphers) {
+    const unperformable = ciphers.filter((c) => !CIPHER_PARAMS[c]);
+    if (unperformable.length) {
+      throw new TlsError(
+        codes.CONFIG_INVALID,
+        `tls.ciphers offers ${unperformable.map((c) => `0x${c.toString(16).padStart(4, '0')}`).join(', ')}, ` +
+          'which this package cannot perform. Offering a suite it cannot complete trades a ' +
+          'fingerprint match for a dead connection the moment a server selects one — see the ' +
+          'chrome profile, which drops the CBC and RSA-key-exchange suites for exactly this reason.',
+        { unperformable },
+      );
+    }
+    if (!chacha && options.ciphers.includes(CIPHER.TLS_CHACHA20_POLY1305_SHA256)) {
+      throw new TlsError(
+        codes.CONFIG_INVALID,
+        'tls.ciphers offers TLS_CHACHA20_POLY1305_SHA256 but no ChaCha20 implementation was ' +
+          'supplied. Pass one through `ciphers: { chacha20 }`, or drop the suite from the list — ' +
+          'silently removing it would change the fingerprint you asked for without saying so.',
+      );
+    }
+  }
+
   // --- resumption offer ----------------------------------------------------------------------
   // Everything about the offered PSK that later steps need is derived once, up front: the Early
   // Secret and binder key here (they depend only on the PSK), the binder itself per hello (it
@@ -372,6 +413,7 @@ async function drive({ record, hostname, verifyPeer, options, deps, versions })
     groups,
     alpn,
     ciphers,
+    extraExtensions: options.extraExtensions ?? [],
     versions,
     extensionOrder: options.extensionOrder,
     sigSchemes: options.sigSchemes,

package/src/tls/handshake.js

@@ -246,7 +246,14 @@ export async function continueTls13(ctx) {
       grease: options.grease ?? false,
       random: hello.clientRandom,
       legacySessionId: hello.legacySessionId,
-      extraExtensions: cookie ? [cookieExtension(cookie)] : [],
+      // The caller's extras must be reproduced too, for the same reason the order is: s4.1.2 does
+      // not permit a second hello to change its extension SET, and a retry that quietly dropped
+      // them would present one fingerprint on the first flight and a different one on the second —
+      // a difference that is itself a signal.
+      extraExtensions: [
+        ...(options.extraExtensions ?? []),
+        ...(cookie ? [cookieExtension(cookie)] : []),
+      ],
       psk: offeredPsk && {
         identity: offeredPsk.identity,
         obfuscatedTicketAge: offeredPsk.obfuscatedTicketAge(),

package/types/tls/connect.d.ts

@@ -13,7 +13,16 @@
  * @property {number[]} [groups] supported_groups, in preference order.
  * @property {number[]} [offerGroups] groups to send an actual key_share for. Default the first
  *   supported group; a HelloRetryRequest recovers any other choice at the cost of a round trip.
- * @property {number[]} [ciphers] cipher suites to offer, in preference order.
+ * @property {number[]} [ciphers] cipher suites to offer, in preference order. Every suite must be
+ *   one this package can perform; an offer it cannot honour is a dead connection the moment a
+ *   server selects it, so an unknown suite is refused here rather than on the wire.
+ * @property {Uint8Array[]} [extraExtensions] pre-encoded ClientHello extensions, appended before
+ *   ordering. `extensionOrder` can only arrange extensions that were BUILT — it filters to what
+ *   exists and sorts that — so ordering alone cannot produce an extension this package does not
+ *   generate. Chromium sends several that it does not: `signed_certificate_timestamp` (18),
+ *   `compress_certificate` (27), `session_ticket` (35), `application_settings` (17613) and ECH
+ *   (65037). Supplying them here is the only way to close that gap, and it is the caller's job to
+ *   encode them correctly — this package does not parse what it did not build.
  * @property {number[]} [sigSchemes] signature_algorithms to offer, in preference order.
  * @property {boolean | number} [grease] send GREASE (RFC 8701) reserved values in the cipher list,
  *   the extension list (one at each end), supported_groups, supported_versions and key_share.
@@ -160,9 +169,21 @@ export type TlsOptions = {
      */
     offerGroups?: number[] | undefined;
     /**
-     * cipher suites to offer, in preference order.
+     * cipher suites to offer, in preference order. Every suite must be
+     * one this package can perform; an offer it cannot honour is a dead connection the moment a
+     * server selects it, so an unknown suite is refused here rather than on the wire.
      */
     ciphers?: number[] | undefined;
+    /**
+     * pre-encoded ClientHello extensions, appended before
+     * ordering. `extensionOrder` can only arrange extensions that were BUILT — it filters to what
+     * exists and sorts that — so ordering alone cannot produce an extension this package does not
+     * generate. Chromium sends several that it does not: `signed_certificate_timestamp` (18),
+     * `compress_certificate` (27), `session_ticket` (35), `application_settings` (17613) and ECH
+     * (65037). Supplying them here is the only way to close that gap, and it is the caller's job to
+     * encode them correctly — this package does not parse what it did not build.
+     */
+    extraExtensions?: Uint8Array<ArrayBufferLike>[] | undefined;
     /**
      * signature_algorithms to offer, in preference order.
      */