tunnelfetch 1.9.0 → 1.12.0

package/README.zh-CN.md

@@ -517,17 +517,26 @@ CertificateError [CERT_PIN_MISMATCH]: no certificate in the chain matches any co
 
 ### 一个请求要多少
 
-通过代理抓取一个尺寸可控的源站，热态，同一 isolate 上 7 轮以上取中位数，传输走 gzip。最后一列是同样的数字
-换算成速率，那是更值得随身记住的形式：
+通过代理抓取 `sizeorigin/`，传输走 gzip，所有尺寸在**同一次扫描、同一个 isolate** 内跑完，n>=5 取中位数。
+方法写在这里，是因为上一版这张表没有把方法记清楚到能复现的程度：**一个热态页面 = `(reuse=4 - reuse=1) / 3`**，
+也就是连接已经打开之后第 2 到第 4 个页面的成本；最后一列是同一次扫描测出来的新建连接数字。
 
-| Body | 5 页均摊 | 复用连接 | 新建连接 |
+| Body | 热态页面 | 每解压 MB | 新连接上的第一个请求 |
 | --- | --- | --- | --- |
-| 1 KB | 3.2 ms | 1.7 ms | 9.2 ms |
-| 16 KB | 4.6 ms | 3.1 ms | 10.6 ms |
-| 64 KB | 8.2 ms | 6.7 ms | 14.2 ms |
-| 256 KB | 18.2 ms | 16.7 ms | 24.2 ms |
-| 1 MB | 54.8 ms | 53.3 ms | 60.8 ms |
-| 4 MB | 119.8 ms | 118.3 ms | 125.8 ms |
+| 1 KB | **1.3 ms** | — | 8 ms |
+| 16 KB | **2.3 ms** | 149 ms/MB | 9 ms |
+| 64 KB | **6.0 ms** | 96 ms/MB | 13 ms |
+| 256 KB | **14.3 ms** | 57 ms/MB | 28 ms |
+| 1 MB | **36.3 ms** | 36 ms/MB | 59 ms |
+| 4 MB | **102 ms** | 26 ms/MB | 135 ms |
+
+最后一列不是「热态页面 + 一次握手」。4 MB 那一行它比热态页面高 33 ms，而握手只值个位数——因为一条连接上的
+**第一个** body 还要在 V8 把解码循环编译优化之前跑一遍。评估一个新源站要看这一列，不要看第一列。
+
+**2026 年 8 月重测，中间几档动了。** 4 MB 那行几乎原样复现（102 对先前发布的 104）；64 KB 到 1 MB 比这张表
+原先的数字低 20–30%。那**不是**下面讲的 socket 视图大小改动——在同一个 isolate 里 A/B 新旧视图大小，1 MB 热态
+页面只差约 1 ms——所以它要么是单次扫描看不到的日间波动，要么是被取代的那张表取数方式不同。旧数字已经无从复查，
+这正是把方法写在这里的理由。
 
 冷启动成本是**总数**,不是往上面某一行加的增量:
 
@@ -554,35 +563,45 @@ CertificateError [CERT_PIN_MISMATCH]: no certificate in the chain matches any co
 ### 这些折算成多少钱
 
 Workers Standard 每月 $5，含 1000 万请求和 3000 万 CPU 毫秒，超出部分每百万请求 $0.30、每百万 CPU 毫秒
-$0.02。把上面的实测代入，并把冷启动的影响单独列成两组列，这样任一负载「预热与否」的差别是看得见的：
-
-| 工作负载 | CPU/请求 | 1000 万,冷 | 1000 万,预热 | 10 亿,冷 | 10 亿,预热 |
-| --- | --- | --- | --- | --- | --- |
-| 平台 `fetch`,16 KB —— 参照;它用不了代理 | 0.3 ms | $5.00 | $5.00 | $307.40 | $307.40 |
-| 平台 `fetch`,4 MB —— 同一参照,实测 | 3.2 ms | $5.04 | $5.04 | $365.40 | $365.40 |
-| 连接复用,16 KB 页面 | 3.1 ms | $5.90 | $5.24 | $451.20 | $385.20 |
-| 每请求新建连接,16 KB | 10.6 ms | $7.41 | $6.75 | $602.20 | $536.20 |
-| 连接复用,1 MB 页面 | 53.3 ms | $15.94 | $15.28 | $1455.20 | $1389.20 |
-| 每请求新建连接,1 MB | 60.8 ms | $17.45 | $16.79 | $1606.20 | $1540.20 |
-| 连接复用,4 MB 页面 | 118.3 ms | $28.94 | $28.28 | $2755.20 | $2689.20 |
-| 每请求新建连接,4 MB | 125.8 ms | $30.45 | $29.79 | $2906.20 | $2840.20 |
+$0.02。公式就是 `$5 + max(0, 请求数 - 1000万) x $0.30/百万 + max(0, CPU毫秒 - 3000万) x $0.02/百万`，
+没有别的：
+
+| 工作负载 | CPU/请求 | 1000 万/月 | 10 亿/月 |
+| --- | --- | --- | --- |
+| 平台 `fetch`,16 KB —— 参照;它用不了代理 | 0.3 ms | $5.00 | $307.40 |
+| 平台 `fetch`,4 MB —— 同一参照,实测 | 3.2 ms | $5.04 | $365.40 |
+| 连接复用,16 KB 页面 | 2.3 ms | $5.00 | $347.40 |
+| 每请求新建连接,16 KB | 9 ms | $6.20 | $481.40 |
+| 连接复用,1 MB 页面 | 36.3 ms | $11.66 | $1,027.40 |
+| 每请求新建连接,1 MB | 59 ms | $16.20 | $1,481.40 |
+| 连接复用,4 MB 页面 | 102 ms | $24.80 | $2,341.40 |
+| 每请求新建连接,4 MB | 135 ms | $31.40 | $3,001.40 |
+
+那两个 `max(0, ...)` 是新加的。上一版这张表把每一个请求、每一毫秒 CPU 都算了钱，忽略了它自己上面那句话
+描述的免费额度——于是 1000 万/月那一列最多高估了 74%（$8.70，而实际账单是 $5.00），而在 10 亿那一列误差在
+0.2% 以内，因为那时免费额度只是个舍入误差。高估的方向是对本包不利的，大概这就是它活了这么久的原因。
 
 参照那一行给了两个尺寸,因为平台自己的 `fetch` **不是平的**——它按每解压 MB 约 0.82 ms 增长,这是在同一个 CDN 的尺寸阶梯上测的,只有大小在变。把它写成单一的 0.3 ms 再拿去和 4 MB 那一行比,是拿不同的东西相比,而且是**抬高了对手而不是抬高本包**。
 
-这些美元数字跟随上面修正后的 CPU 测量,所以正文偏重的行是 1.4.0 及以前的**两到三倍**。那不是包变慢了,是移除了一个内容压缩比 220:1 的源站。
+这张表原先还按「冷 / 预热」分成两组列。那两组列现在去掉了：新的 CPU 数字全部是热态和新建连接的实测，冷 isolate
+下的同一批负载没有重测过，与其把上一版的冷启动增量套到新基线上凑出四列，不如只写实际测过的两列。冷启动的成本
+在上面那张 `warmup()` 表里，那是一个**总数**，不是往这里某一行上加的增量。
 
 #### Chrome 身份的每个选项各花多少
 
 上面那张表是默认身份:线上 gzip、AES-256-GCM、x25519。Chrome 那一行把所有变化捆在一起,对做决定没什么用。按"连接复用 + 1 MB 页面 + 十亿请求/月 + 已预热"逐项拆开:
 
-| 相对基线的变化 | CPU/请求 | 10 亿/月,已预热 | Δ | 何时才付 |
+| 相对基线的变化 | CPU/请求 | 10 亿/月 | Δ | 何时才付 |
 | --- | --- | --- | --- | --- |
-| 基线 —— gzip、AES-256-GCM、x25519 | 53.3 ms | $1,389 | — | 总是 |
-| 源站发 `br` 而不是 gzip | 57.6 ms | $1,474 | **+$85** | 源站选择发 `br` |
-| 服务器选中 ChaCha20-Poly1305 | 56.3 ms | $1,448 | **+$59** | 服务器优先选它而非 AES |
-| 源站发 `zstd` 而不是 gzip | 56.1 ms | $1,444 | **+$55** | 源站选择发 `zstd` |
-| 协商 X25519MLKEM768,每连接 1 个请求 | 61.0 ms | $1,542 | **+$153** | 每次握手 |
-| 协商 X25519MLKEM768,每连接 20 个请求 | 53.3 ms | $1,390 | **+$0.15** | 同一次握手,摊薄后 |
+| 基线 —— gzip、AES-256-GCM、x25519,热态 | 36.3 ms | $1,027 | — | 总是 |
+| 源站发 `br` 而不是 gzip | 40.5 ms | $1,111 | **+$84** | 源站选择发 `br` |
+| 服务器选中 ChaCha20-Poly1305 | 39.3 ms | $1,086 | **+$59** | 服务器优先选它而非 AES |
+| 源站发 `zstd` 而不是 gzip | 39.1 ms | $1,083 | **+$56** | 源站选择发 `zstd` |
+| 协商 X25519MLKEM768,每连接 1 个请求 | 59.2 ms | $1,484 | **+$3** | 每次握手 |
+| 协商 X25519MLKEM768,每连接 20 个请求 | 36.3 ms | $1,028 | **+$0.15** | 同一次握手,摊薄后 |
+
+ML-KEM 这两行是拿**新建连接**的 1 MB 基线 59 ms（$1,481）来比的，不是拿表头那个热态基线；Δ 列反映的是这一点，
+所以是 $3 而不是旧版写的 $153。那 $153 是「不复用连接」的成本，被算到了后量子密钥交换头上。
 
 **最后两行是同样的 0.15 ms ML-KEM,差别完全来自连接复用**——这才是这张表最值得带走的一条。后量子密钥交换在你让 `Client` 活着时是这里最便宜的东西,在你不这么做时是最贵的,因为它是**每握手**,而其余全是每字节。
 
@@ -655,8 +674,40 @@ CPU 计费，所以十亿请求下「预热」两列省下的每月 $65 是净
 大 body 的"每字节成本"其实不按字节计——按的是流边界穿越次数。这个运行时的 `DecompressionStream` 以
 4096 字节为块产出输出，套接字单次交付也至多 4096 字节，而每一块在运行时与 JS 之间穿越一次都要几十微秒，
 与块大小无关——实测约 **17 µs 一次穿越**，来自同一个 1 MB 分别按 4 KiB（6.0 ms/MB）到 256 KiB
-（1.67 ms/MB）收集的阶梯。因此两条热路径都改为用 BYOB 读来抽干来源：它把已经缓冲的数据一次交付，且只要
-有一个字节就立即以部分填充返回，流式延迟不变。
+（1.67 ms/MB）收集的阶梯。因此两条热路径都改为用 BYOB 读来抽干来源：它把好几块合并成一次穿越，且只要
+有一个字节就立即以部分填充返回，流式延迟不变。合并多少是**传输层**决定的，不是视图决定的——BYOB 读从不
+等着填满——所以视图开得比传输层单次交给你的多，买不到任何东西，只会白付一次分配。4 MB body 上实测的平均
+填充：直连 37 KB，走代理 8 KB。
+
+### 一个在本包赖以存在的那条路径上从未接通的旋钮
+
+`openTunnel` 做完 CONNECT（或 SOCKS5）握手时手里握着一个带缓冲的 reader——对端可能把隧道数据和应答塞在
+同一个 chunk 里——它把这个交出去时包成了 `new ReadableStream({ pull })`。逻辑正确，但那是**普通流，不是
+byte stream**。TLS 记录层要 BYOB reader，要不到就静默降级成 default reader，于是*每一条走代理的连接*都丢了
+BYOB，连带 `tls.pullBytes`（它唯一的职责就是决定 BYOB 视图多大）一起失效。
+
+同源、1 MB 过记录层、同一 isolate、n=15，修复前：
+
+| | `pullBytes: 16 KiB` | `pullBytes: 1 MiB` |
+|---|---|---|
+| 直连 | min 25, p50 31 | min 73, p50 90 |
+| **走代理** | min 95, p50 111 | min 101, p50 121 |
+
+**直连 2.9×，代理 6%**——那个旋钮在代理路径上根本没被读到。连带后果：当初用来定 64 KiB 默认值、标着「对着
+真实代理 socket 扫出来」的那条 U 曲线，是同一个配置的四个采样。
+
+`src/proxy/tunnel.js` 把隧道改成 byte stream，并且在握手剩余字节吐完之后**把调用方自己的视图直接递给
+socket**。旋钮接上之后重扫（1 MB 过记录层，p50）：
+
+| | 8 KiB | **16 KiB** | 32 KiB | 64 KiB | 256 KiB |
+|---|---|---|---|---|---|
+| 直连 | 21 | 20 | 18 | 22 | 40 |
+| 代理 A | 51 | 61 | 61 | 89 | 152 |
+| 代理 B | 68 | 74 | — | 102 | — |
+
+两个独立代理上都单调。默认值因此从 64 KiB 改为 **16 KiB**：三条路径上都在最优列 20% 以内，而 64 KiB 最差
+偏离 45%，分配量还是四分之一，并且正好是一个 TLS 记录。端到端走真 Client + 代理、n=13：1 MB **63 对 78 ms**、
+4 MB **145 对 164 ms**（中位数），4 MB 的最小值在噪声内。
 
 读进去的那个视图是 **16 KiB，而且这个尺寸是扫出来的，不是拍的**。它比看上去重要：输入由一个 JS 任务在与
 拉取方相同的事件循环上泵送，所以读到达时解压器手上通常只压着一两块，读回来就是部分填充——实测一个 1 MB

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tunnelfetch",
-  "version": "1.9.0",
+  "version": "1.12.0",
   "description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
   "keywords": [
     "fetch",

package/src/proxy/http-connect.js

@@ -13,6 +13,7 @@
 
 import { ProxyError, LimitError, codes } from '../errors.js';
 import { ByteReader, ByteWriter, latin1, utf8 } from '../util/bytes.js';
+import { tunnelReadable } from './tunnel.js';
 
 const CRLFCRLF = utf8('\r\n\r\n');
 const MAX_REPLY_HEADER = 32 * 1024;
@@ -122,9 +123,15 @@ function buildConnectRequest(proxy, target) {
   if (proxy.username) {
     lines.push(`Proxy-Authorization: Basic ${basicCredentials(proxy.username, proxy.password)}`);
   }
-  // Some proxies still key off the pre-standard hop header; sending it is harmless and avoids a
-  // class of proxy that closes the tunnel after one request without it.
-  lines.push('Proxy-Connection: keep-alive');
+  // `Proxy-Connection` is a pre-standard hop header that never made it into a spec, and clients
+  // disagree about it: some send `keep-alive`, some send `close`, some omit it. The proxy sees this
+  // even though the origin never does, so for anyone matching a client's behaviour AT THE PROXY it
+  // is part of the fingerprint. `keep-alive` stays the default — it avoids a class of proxy that
+  // closes the tunnel after one request — but it is no longer fixed.
+  //
+  // `null` omits the header entirely, which is not the same as sending `close`.
+  const pc = proxy.proxyConnection === undefined ? 'keep-alive' : proxy.proxyConnection;
+  if (pc !== null) lines.push(`Proxy-Connection: ${pc}`);
   return `${lines.join('\r\n')}\r\n\r\n`;
 }
 
@@ -188,16 +195,7 @@ function replyError(reply, proxy, target, where) {
  */
 function tunnelFrom(socket, reader) {
   return {
-    readable: new ReadableStream({
-      async pull(controller) {
-        const chunk = await reader.readSome();
-        if (chunk === null) controller.close();
-        else controller.enqueue(chunk);
-      },
-      cancel(reason) {
-        return reader.cancel(reason);
-      },
-    }),
+    readable: tunnelReadable(socket, reader),
     writable: socket.writable,
     opened: socket.opened,
     close: () => socket.close?.(),

package/src/proxy/index.js

@@ -19,7 +19,13 @@ import { openSocks5 } from './socks5.js';
  * @typedef {(addr: {hostname: string, port: number},
  *            opts?: {secureTransport?: 'off'|'on'|'starttls', allowHalfOpen?: boolean}) => Duplex} ConnectFn
  * @typedef {{ protocol: 'http'|'https'|'socks5'|'socks5h', hostname: string, port: number,
- *             username?: string, password?: string }} ProxyConfig
+ *             username?: string, password?: string,
+ *             proxyConnection?: string | null }} ProxyConfig
+ *
+ * `proxyConnection` sets the pre-standard `Proxy-Connection` header on a CONNECT request, or
+ * omits it entirely when null. Default 'keep-alive'. The origin never sees this header; the
+ * proxy does, so it belongs to whatever fingerprint the proxy is reading. Clients disagree —
+ * some send keep-alive, some close, some nothing — and omitting is not the same as 'close'.
  */
 
 const DEFAULT_PORTS = { http: 8080, https: 443, socks5: 1080, socks5h: 1080 };
@@ -114,6 +120,10 @@ function normalise(cfg) {
     port,
     username: cfg.username || undefined,
     password: cfg.password || undefined,
+    // Listed explicitly because this function REBUILDS the config rather than copying it, so a
+    // field not named here is dropped without a word. That is how `http2ConnectionWindow` came to
+    // be declared in a profile and read by nothing.
+    proxyConnection: cfg.proxyConnection,
   });
 }
 

package/src/proxy/socks5.js

@@ -14,6 +14,7 @@
 
 import { ProxyError, ConfigError, codes, hex8 } from '../errors.js';
 import { ByteReader, ByteWriter, concat, utf8 } from '../util/bytes.js';
+import { tunnelReadable } from './tunnel.js';
 
 const VERSION = 0x05;
 const AUTH_VERSION = 0x01;
@@ -326,16 +327,7 @@ async function readExactly(reader, n, what, where) {
 /** Bytes that arrived alongside the reply are tunnel payload; the buffered reader carries them. */
 function tunnelFrom(socket, reader) {
   return {
-    readable: new ReadableStream({
-      async pull(controller) {
-        const chunk = await reader.readSome();
-        if (chunk === null) controller.close();
-        else controller.enqueue(chunk);
-      },
-      cancel(reason) {
-        return reader.cancel(reason);
-      },
-    }),
+    readable: tunnelReadable(socket, reader),
     writable: socket.writable,
     opened: socket.opened,
     close: () => socket.close?.(),

package/src/proxy/tunnel.js

@@ -0,0 +1,110 @@
+// The tunnel's readable half, and why it has to be a byte stream.
+//
+// Both dialects finish their handshake holding a buffered reader that may already carry tunnel
+// payload: an HTTP CONNECT reply is read up to its blank line and a SOCKS5 reply to its fixed
+// length, and in both cases the peer is free to have sent application bytes immediately behind it.
+// Those bytes must be delivered first and in order, which is why the socket cannot simply be
+// handed onward.
+//
+// What this replaces was `new ReadableStream({ pull })` — correct, and a plain stream rather than a
+// byte stream. That distinction is invisible until you look at what reads it. The TLS record layer
+// asks for a BYOB reader and quietly falls back to a default one when it cannot have it, so every
+// connection through a proxy lost BYOB reads, and with them `tls.pullBytes`, whose entire job is to
+// decide how much of the socket arrives per boundary crossing.
+//
+// Measured on the edge against one origin, 4 MB, differenced at a single request:
+//
+//     direct     132 socket reads, 31.8 KB average fill
+//     proxied    484 socket reads,  8.7 KB average fill
+//
+// and the knob itself, 1 MB through the record layer, n=15 in one isolate:
+//
+//                       pullBytes 16 KiB     pullBytes 1 MiB
+//     direct            min 25  p50 31       min 73  p50 90     <- 2.9x, the knob works
+//     proxied           min 95  p50 111      min 101 p50 121    <- 6%, the knob is not read
+//
+// So the U-curve recorded against "a real proxied socket" in util/bytes.js was four samples of one
+// configuration. The knob was never reaching the code on that path.
+//
+// The fix is not to copy harder. Once the handshake's leftovers are drained this hands the caller's
+// own view straight to the socket, so a read through a tunnel costs exactly what a read without one
+// costs — no wrapper copy, and the same coalescing.
+
+import { ByteReader } from '../util/bytes.js';
+
+/**
+ * View size handed to a *default* reader of this stream. BYOB readers supply their own view and
+ * never see this; it exists so that a caller which does not do BYOB still gets socket-sized chunks
+ * rather than whatever the transport felt like emitting.
+ */
+const TUNNEL_CHUNK = 65536;
+
+/**
+ * Wrap a proxy handshake's buffered reader plus its socket as one byte stream.
+ *
+ * @param {{ readable: ReadableStream<Uint8Array> }} socket the raw transport
+ * @param {import('../util/bytes.js').ByteReader} reader the handshake's reader, possibly holding
+ *   bytes that belong to the tunnel
+ * @returns {ReadableStream<Uint8Array>}
+ */
+export function tunnelReadable(socket, reader) {
+  /** Taken once `reader` runs dry, after which every read goes straight to the socket. */
+  let direct = null;
+  /** Buffered stand-in for `reader` when the socket turns out not to be a byte stream. */
+  let buffered = reader;
+  let promoted = false;
+
+  // Whether a stream is BYOB-capable can only be discovered by asking it for a BYOB reader, and it
+  // cannot be asked while the handshake's reader holds the lock. So: release (safe only because
+  // the guard below proves nothing is buffered), ask, and on refusal re-take a buffered reader —
+  // which is exactly the shape that was here before, for transports that cannot do better. Every
+  // in-process stream in this package's tests is one of those, so this path is well covered.
+  const promote = () => {
+    if (promoted || buffered.buffered > 0 || buffered.atEof) return;
+    promoted = true;
+    buffered.releaseLock();
+    try {
+      direct = socket.readable.getReader({ mode: 'byob' });
+    } catch {
+      buffered = new ByteReader(socket.readable);
+    }
+  };
+
+  return new ReadableStream({
+    type: 'bytes',
+    autoAllocateChunkSize: TUNNEL_CHUNK,
+    async pull(controller) {
+      // autoAllocateChunkSize guarantees a byobRequest even for a default reader, so there is one
+      // path here rather than two.
+      const req = controller.byobRequest;
+      const view = req.view;
+      promote();
+      if (direct) {
+        const { value, done } = await direct.read(
+          new Uint8Array(view.buffer, view.byteOffset, view.byteLength),
+        );
+        if (done) {
+          // Close first: a BYOB request may only be answered with zero bytes once the stream is
+          // closed, and the read has detached the original view, so the answer has to be the
+          // zero-length view the reader handed back rather than a plain respond(0).
+          controller.close();
+          req.respondWithNewView(value ?? new Uint8Array(0));
+          return;
+        }
+        req.respondWithNewView(value);
+        return;
+      }
+      const chunk = await buffered.readSome(view.byteLength);
+      if (chunk === null) {
+        controller.close();
+        req.respond(0);
+        return;
+      }
+      new Uint8Array(view.buffer, view.byteOffset, view.byteLength).set(chunk);
+      req.respond(chunk.byteLength);
+    },
+    cancel(reason) {
+      return direct ? direct.cancel(reason) : buffered.cancel(reason);
+    },
+  });
+}

package/src/tls/connect.js

@@ -137,9 +137,23 @@ function expectServerHello(msg, offers12) {
  * @property {number[]} [groups] supported_groups, in preference order.
  * @property {number[]} [offerGroups] groups to send an actual key_share for. Default the first
  *   supported group; a HelloRetryRequest recovers any other choice at the cost of a round trip.
- * @property {number[]} [ciphers] cipher suites to offer, in preference order. Every suite must be
- *   one this package can perform; an offer it cannot honour is a dead connection the moment a
- *   server selects it, so an unknown suite is refused here rather than on the wire.
+ * @property {number[]} [ciphers] cipher suites to offer, in preference order. By default every
+ *   suite must be one this package can perform: an offer it cannot honour is a dead connection the
+ *   moment a server selects it, so an unknown suite is refused here rather than on the wire.
+ * @property {number[]} [omitExtensions] extension types to leave out of the ClientHello, the
+ *   subtractive counterpart to `extraExtensions`. `status_request` (5) is the one extension this
+ *   package sends that curl does not, so an identity matching a sample without it needs this.
+ *   Dropping it gives up OCSP stapling, which is the only revocation signal this package can
+ *   consume — pairing it with `trust.revocation: 'require-staple'` is refused rather than left to
+ *   fail every connection.
+ * @property {boolean} [allowUnperformableCiphers] offer suites this package cannot complete.
+ *   For fingerprint fidelity only. Real clients offer far more than this package implements — curl
+ *   8.21.0 offers thirty against seven performable here, Chromium fifteen against seven — so a
+ *   hello restricted to what it can honour carries a cipher list shorter than any real client's,
+ *   which is exactly what a JA3 hash reads. With this set, a server that selects an unperformable
+ *   suite fails the handshake; the first such suite sits behind the TLS 1.3 ones in both real
+ *   lists, so a 1.3-capable server does not reach it. Knowingly trading a rare failure for an
+ *   accurate fingerprint is a legitimate choice; making it silently is not.
  * @property {Uint8Array[]} [extraExtensions] pre-encoded ClientHello extensions, appended before
  *   ordering. `extensionOrder` can only arrange extensions that were BUILT — it filters to what
  *   exists and sorts that — so ordering alone cannot produce an extension this package does not
@@ -347,7 +361,16 @@ async function drive({ record, hostname, verifyPeer, options, deps, versions })
   // package cannot perform is an offer a server may take and then find unhonoured, which fails the
   // connection rather than merely looking wrong". It applies just as much to `tls.ciphers`, and
   // now does. CIPHER_PARAMS is the set this package knows how to key and seal.
-  if (options.ciphers) {
+  //
+  // `allowUnperformableCiphers` opts out, and it exists because refusing outright was the wrong
+  // default to have no escape from. Every real client offers far more suites than this package
+  // implements — curl 8.21.0 offers thirty and seven are performable here; Chromium offers fifteen
+  // and seven are — so a hello restricted to what can be honoured has a cipher list shorter than
+  // any real client's, and that is itself what a JA3 hash reads. A caller matching a fingerprint
+  // may rationally prefer the accurate list: the first unperformable suite sits at index 5 of
+  // curl's and 7 of Chromium's, behind the TLS 1.3 suites, so a server with 1.3 available never
+  // reaches it. The risk is real but narrow, and it is the caller's to take knowingly.
+  if (options.ciphers && !options.allowUnperformableCiphers) {
     const unperformable = ciphers.filter((c) => !CIPHER_PARAMS[c]);
     if (unperformable.length) {
       throw new TlsError(
@@ -414,6 +437,7 @@ async function drive({ record, hostname, verifyPeer, options, deps, versions })
     alpn,
     ciphers,
     extraExtensions: options.extraExtensions ?? [],
+    omitExtensions: options.omitExtensions ?? [],
     versions,
     extensionOrder: options.extensionOrder,
     sigSchemes: options.sigSchemes,

package/src/tls/handshake-messages.js

@@ -192,6 +192,7 @@ export async function deriveSharedSecret(group, privateKey, peerKey, deps = {})
  * @property {string[]} [alpn] default ['http/1.1']; empty array omits the extension
  * @property {number[]} [versions] default [TLS13, TLS12]
  * @property {Uint8Array[]} [extraExtensions] pre-encoded, sent verbatim (the HRR cookie)
+ * @property {number[]} [omitExtensions] extension types to leave out of the hello
  * @property {{ identity: Uint8Array, obfuscatedTicketAge: number, binderLen: number }} [psk]
  *   offer this resumption PSK. Encoded with a zeroed binder placeholder; the caller MUST derive
  *   the real binder over `message.subarray(0, truncatedLength)` and patch it in at
@@ -299,6 +300,7 @@ export function buildClientHello({
   versions = [TLS13, TLS12],
   extensionOrder = CURL_EXTENSION_ORDER,
   extraExtensions = [],
+  omitExtensions = [],
   psk = null,
   grease = false,
   randomBytes = defaultRandom,
@@ -341,11 +343,15 @@ export function buildClientHello({
       'a resumption PSK was supplied but TLS 1.3 is not among the offered versions');
   }
 
+  const omit = new Set(omitExtensions);
   const extensionParts = [
     encodeServerName(hostname),
-    // Always offered, for either version: without it a server may not staple (RFC 6066 s8), and
-    // a stapled OCSP response is the only revocation signal this package can consume.
-    encodeStatusRequest(),
+    // Offered by default for either version: without it a server may not staple (RFC 6066 s8), and
+    // a stapled OCSP response is the only revocation signal this package can consume. It is also
+    // the one extension this package sends that curl does not, so an identity matching a sample
+    // without it needs a way to drop it — `omitExtensions` is the subtractive counterpart to
+    // `extraExtensions`, and dropping this one gives up stapling.
+    omit.has(EXTENSION.status_request) ? null : encodeStatusRequest(),
     encodeSupportedGroups(g ? [g.take(), ...groups] : groups),
     encodeSignatureAlgorithms(sigSchemes),
     alpn.length ? encodeAlpn(alpn) : null,
@@ -589,7 +595,10 @@ export function negotiateCipher(serverHello, { offeredCiphers, version }) {
   if (!params) {
     throw new TlsUnsupportedError(
       codes.TLS_CIPHER_UNSUPPORTED,
-      `cipher suite ${hex16(suite)} has no parameters; this is a bug in the offer list`,
+      `cipher suite ${hex16(suite)} was offered but this package cannot perform it, so the ` +
+        'handshake cannot continue. Either the offer list has a bug, or `allowUnperformableCiphers` ' +
+        'was set to match a real client\'s cipher list and a server has selected one of the suites ' +
+        'that choice knowingly put on the wire.',
       { cipherSuite: suite },
     );
   }

package/src/tls/handshake.js

@@ -250,6 +250,7 @@ export async function continueTls13(ctx) {
       // not permit a second hello to change its extension SET, and a retry that quietly dropped
       // them would present one fingerprint on the first flight and a different one on the second —
       // a difference that is itself a signal.
+      omitExtensions: options.omitExtensions ?? [],
       extraExtensions: [
         ...(options.extraExtensions ?? []),
         ...(cookie ? [cookieExtension(cookie)] : []),

package/src/transport.js

@@ -124,6 +124,20 @@ export async function openConnection({
 }) {
   const target = targetFromUrl(url);
   const proxyConfig = parseProxy(proxy);
+
+  // Dropping status_request gives up OCSP stapling, and stapling is the only revocation signal this
+  // package consumes. Combined with `require-staple` that is not a weakened check, it is a policy
+  // that can never be satisfied: every connection would fail with OCSP_REQUIRED, on a certificate
+  // that was never asked to carry a staple. Refused here, where both settings are visible.
+  if ((tls?.omitExtensions ?? []).includes(5) && trust?.revocation === 'require-staple') {
+    throw new ConfigError(
+      codes.CONFIG_INVALID,
+      "tls.omitExtensions drops status_request (5) while trust.revocation is 'require-staple'. " +
+        'Without status_request a server is not asked to staple, so no staple can arrive and every ' +
+        'connection would fail. Keep the extension, or relax the revocation policy — the two are ' +
+        'a contradiction rather than a stricter setting.',
+    );
+  }
   const owns = !deadlines;
   const dl = deadlines ?? new DeadlineController({}, { signal });
 

package/src/util/bytes.js

@@ -15,16 +15,29 @@ const EMPTY = new Uint8Array(0);
  * so a large view never delays delivery; it only lets bytes the transport has already buffered
  * arrive in one crossing instead of many.
  */
-// Swept on the edge against a real proxied socket, ms of CPU per 4 MB body at the record layer:
+// Swept on the edge, ms of CPU for a 1 MB body at the record layer, p50 of n>=7 in one isolate,
+// each column an independent path against the same origin:
 //
-//     16 KiB  42.0     64 KiB  38.5     256 KiB  46.5     1 MiB  57.0
+//                    8 KiB   16 KiB   32 KiB   64 KiB   256 KiB
+//     direct           21      20       18       22        40
+//     proxy A          51      61       61       89       152
+//     proxy B          68      74        -      102         -
 //
-// A U with its floor on the current value, and going LARGER is worse — 1 MiB costs 48% more than
-// the default, because allocating the view outgrows the boundary crossings it saves. Recorded so
-// the next person to reach for this knob does not have to re-run the sweep to find there is nothing
-// in it: the 42 ms this layer costs on a 4 MB body is the price of moving bytes off a real socket,
-// of which the AEAD is under 2 ms. It is not a tuning problem.
-const BYOB_PULL_BYTES = 65536;
+// Monotonic on both proxies, and shallow on the direct path: too LARGE is what costs, because a
+// BYOB read resolves the instant any byte exists and never waits to fill, so a view bigger than
+// what the transport hands over per read is allocation that is never used. Average fill measured
+// over a 4 MB body: 37 KB direct, 8 KB through a proxy — which is why the two paths want different
+// numbers and why the proxied one wants a small one.
+//
+// 16 KiB is within ~20% of the best figure on all three paths; the 64 KiB that used to sit here is
+// up to 45% off. It is also exactly one TLS record, which is the unit the caller above asks for.
+//
+// The previous value came from a sweep captioned "against a real proxied socket" that reported a
+// clean U with its floor at 64 KiB. That sweep measured nothing: openTunnel wrapped the socket in a
+// plain ReadableStream, so the record layer could not take a BYOB reader on any proxied connection
+// and this constant was never read on that path. The four numbers were four samples of one
+// configuration. See proxy/tunnel.js, which is where that was fixed.
+const BYOB_PULL_BYTES = 16384;
 
 /** Raised when the peer stops sending in the middle of a structure we must read whole. */
 export class UnexpectedEofError extends TunnelFetchError {
@@ -50,21 +63,25 @@ export class UnexpectedEofError extends TunnelFetchError {
  * be retained beyond the caller's immediate use if memory matters.
  *
  * When the source is a byte stream — on the target runtime, a socket's readable is one — the
- * reader pulls with BYOB reads into large fresh views instead of taking the source's own
- * chunking. This is measured, not stylistic: the runtime delivers socket data in chunks of at
- * most 4096 bytes, ~1200 of them for a 4 MB body, and every chunk is a runtime/JS boundary
- * crossing; a BYOB read hands over everything the transport has buffered (up to the view size)
- * in one crossing, and resolves with a partial fill the instant anything at all is available,
- * so delivery latency is unchanged. Sources that are not byte streams (every in-process
+ * reader pulls with BYOB reads instead of taking the source's own chunking. This is measured, not
+ * stylistic: the runtime delivers socket data in chunks of at most 4096 bytes, ~1200 of them for a
+ * 4 MB body, and every chunk is a runtime/JS boundary crossing; a BYOB read collects several of
+ * them into one.
+ *
+ * How MANY it collects is the transport's decision, not the view's. A BYOB read resolves the
+ * instant any byte is available and never waits to fill, so the view is a ceiling that is normally
+ * not reached: measured over a 4 MB body, 37 KB average fill on a direct socket and 8 KB through a
+ * proxy, whatever the view size. Sizing the view far above that buys nothing and costs the
+ * allocation — see BYOB_PULL_BYTES. Sources that are not byte streams (every in-process
  * ReadableStream in this package and its tests) take the default-reader path unchanged.
  */
 export class ByteReader {
   /**
    * @param {ReadableStream<Uint8Array>} readable
-   * @param {number} [pullBytes] size of each BYOB view pulled from the source. Tunable because it
-   *   decides how many times a body crosses the runtime boundary on the way in, and that turned out
-   *   to be the largest single cost in a large response — 42 ms of a 106 ms 4 MB request is socket
-   *   reads and record decryption, of which the AEAD itself is under 2 ms.
+   * @param {number} [pullBytes] size of each BYOB view pulled from the source. Tunable because the
+   *   right value depends on how much the transport hands over per read, and that differs by a
+   *   factor of four between a direct socket and a proxied one — see BYOB_PULL_BYTES for the sweep.
+   *   Ignored on sources that are not byte streams, which take the default-reader path.
    */
   constructor(readable, pullBytes = BYOB_PULL_BYTES) {
     this._pullBytes = pullBytes > 0 ? pullBytes : BYOB_PULL_BYTES;

package/types/proxy/index.d.ts

@@ -49,12 +49,19 @@ export type ConnectFn = (addr: {
     secureTransport?: "off" | "on" | "starttls";
     allowHalfOpen?: boolean;
 }) => Duplex;
+/**
+ * `proxyConnection` sets the pre-standard `Proxy-Connection` header on a CONNECT request, or
+ * omits it entirely when null. Default 'keep-alive'. The origin never sees this header; the
+ * proxy does, so it belongs to whatever fingerprint the proxy is reading. Clients disagree —
+ * some send keep-alive, some close, some nothing — and omitting is not the same as 'close'.
+ */
 export type ProxyConfig = {
     protocol: "http" | "https" | "socks5" | "socks5h";
     hostname: string;
     port: number;
     username?: string;
     password?: string;
+    proxyConnection?: string | null;
 };
 import { openDirect } from './direct.js';
 import { openHttpConnect } from './http-connect.js';

package/types/proxy/tunnel.d.ts

@@ -0,0 +1,11 @@
+/**
+ * Wrap a proxy handshake's buffered reader plus its socket as one byte stream.
+ *
+ * @param {{ readable: ReadableStream<Uint8Array> }} socket the raw transport
+ * @param {import('../util/bytes.js').ByteReader} reader the handshake's reader, possibly holding
+ *   bytes that belong to the tunnel
+ * @returns {ReadableStream<Uint8Array>}
+ */
+export function tunnelReadable(socket: {
+    readable: ReadableStream<Uint8Array>;
+}, reader: import("../util/bytes.js").ByteReader): ReadableStream<Uint8Array>;