tunnelfetch 1.12.0 → 1.13.0

package/README.md

@@ -149,6 +149,42 @@ Two related rules of §5.7 are **not** implemented, and are worth knowing if you
 security: "Leave Secure Cookies Alone" (step 16), so a plain-named `Secure` cookie set over https
 can still be overwritten from http, and the 4096-octet name-plus-value cap (step 4).
 
+### Obtaining the socket factory
+
+`connect` is an argument rather than an import on purpose: it is the one piece this package cannot
+supply itself, and taking it from the caller is what keeps every layer above it testable over an
+in-memory pipe. Resolve one robustly:
+
+```js
+import { resolveConnect } from 'tunnelfetch';
+
+const connect = await resolveConnect({
+  specifiers: ['cloudflare:sockets'],
+});
+const client = new Client({ connect, proxy: env.PROXY_URL });
+```
+
+The first specifier that yields a callable export wins, and a total failure names every specifier
+tried and why it failed — the mistake surfaces at startup, not as "connect is not a function" from
+inside the TLS layer. Two traps are closed on the way:
+
+1. The import is dynamic and its failure is caught. A specifier that only resolves on one runtime,
+   imported statically, makes the whole package unloadable everywhere else; a portable list naming
+   several runtimes is safe.
+2. Bundlers cannot see through a variable specifier. On a bundled deploy pass `connect` straight
+   in, or hand `resolveConnect` an `importModule` that closes over a literal the bundler can see:
+
+   ```js
+   const connect = await resolveConnect({
+     importModule: () => import('cloudflare:sockets'),
+   });
+   ```
+
+`normaliseSocket` flattens one runtime socket into a plain duplex, and `normalisingConnect` wraps a
+factory so every socket it returns is flattened. They exist because on the edge runtime a socket's
+`readable` and `writable` are prototype accessors, so `{ ...socket }` copies neither and the first
+read fails far away, inside the TLS layer, with a complaint about `getReader`.
+
 ### Replacing the global
 
 For libraries that only ever call the bare global:

package/README.zh-CN.md

@@ -117,6 +117,38 @@ await client.close();          // 必须调用：释放池中的 socket
 （第 16 步），所以一个普通名字的 `Secure` cookie 即便是在 https 上设置的，仍可能被 http 覆盖;以及名字加值
 4096 字节的上限（第 4 步）。
 
+### 获取 socket 工厂
+
+`connect` 是参数而不是 import，这是刻意的：它是这个包唯一无法自带的部分，由调用方提供也正是上面每一层都能在
+内存管道上做字节级测试的原因。稳妥地获取一个：
+
+```js
+import { resolveConnect } from 'tunnelfetch';
+
+const connect = await resolveConnect({
+  specifiers: ['cloudflare:sockets'],
+});
+const client = new Client({ connect, proxy: env.PROXY_URL });
+```
+
+第一个能导出可调用函数的 specifier 胜出；全部失败时报错会列出试过的每一个 specifier 及其失败原因——错误在
+启动时就现身，而不是以 "connect is not a function" 的样子从 TLS 层深处冒出来。过程中关掉了两个坑：
+
+1. 导入是动态的，失败会被捕获。某个只在一个运行时上能解析的 specifier，一旦被静态导入，就会让整个包在其它
+   所有平台上无法加载；列着多个运行时的可移植列表则是安全的。
+2. 打包器看不穿变量形式的 specifier。打包部署时直接把 `connect` 传进来，或者给 `resolveConnect` 传一个闭包了
+   字面量的 `importModule`，让打包器看得到：
+
+   ```js
+   const connect = await resolveConnect({
+     importModule: () => import('cloudflare:sockets'),
+   });
+   ```
+
+`normaliseSocket` 把一个运行时 socket 拍平成普通双工对象，`normalisingConnect` 包一层工厂，让每个返回的 socket
+都被拍平。它们的存在是因为 edge 运行时上 socket 的 `readable` 和 `writable` 是原型上的访问器，`{ ...socket }`
+两个都拷不到，第一次读取就会在 TLS 层深处报 `getReader` 的错误。
+
 ### 替换全局 fetch
 
 有的库只会直接调用全局 `fetch`，为它们准备的是：

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tunnelfetch",
-  "version": "1.12.0",
+  "version": "1.13.0",
   "description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
   "keywords": [
     "fetch",

package/src/connect.js

@@ -0,0 +1,187 @@
+// Obtaining the host runtime's raw-socket factory.
+//
+// Every layer above this one takes `connect` as an argument, and that is deliberate: it is what
+// keeps src/ free of runtime-specific imports and what makes every protocol path testable over an
+// in-memory pipe. This module does not weaken that. It answers a narrower question that callers
+// kept answering badly by hand:
+//
+//   given a runtime whose socket factory lives behind a module specifier, produce a validated
+//   ConnectFn, or fail with an error that says what was tried.
+//
+// Two rules shape the design.
+//
+//  1. The specifier is DATA, supplied by the caller. No module name is baked in here, because a
+//     name baked in is behaviour conditioned on one vendor's runtime, and this package is not
+//     allowed to have any. The host application knows which runtime it is running on; this module
+//     does not need to guess, and a wrong guess is worse than an argument.
+//
+//  2. The import is dynamic and its failure is caught. A static import of a specifier that only
+//     resolves on one runtime makes this module unloadable everywhere else, which would take the
+//     whole package down with it on every other platform.
+//
+// A BUNDLER CAVEAT that matters more than it looks. The default importer calls `import(spec)` with
+// a variable. Bundlers cannot see through a variable specifier, so they will not include the
+// target module in the output, and on a bundled deployment the import fails at runtime even though
+// the module exists. Where a bundler is in play — which on edge runtimes is nearly always — pass
+// the socket factory straight in as `connect`, or pass an `importModule` that closes over a
+// LITERAL specifier the bundler can see:
+//
+//   resolveConnect({ importModule: () => import('some:sockets') })
+//
+// That is why `connect` is checked first and short-circuits without importing anything: direct
+// injection is the recommended path, and the specifier list is the convenience, not the contract.
+
+import { ConfigError, codes } from './errors.js';
+
+/** @typedef {import('./proxy/index.js').ConnectFn} ConnectFn */
+/** @typedef {import('./proxy/index.js').Duplex} Duplex */
+
+/**
+ * @typedef {object} ResolveConnectOptions
+ * @property {ConnectFn} [connect] an already-obtained socket factory. Checked first and returned
+ *   as-is, so the recommended path costs no import and cannot be defeated by a bundler.
+ * @property {string[]} [specifiers] module specifiers to try, in order. The first module whose
+ *   `exportName` is callable wins.
+ * @property {(specifier: string) => Promise<object>} [importModule] the importer. Defaults to a
+ *   dynamic `import()`; override it to hand the bundler a literal specifier.
+ * @property {string} [exportName] the export to read off each module. Default 'connect'.
+ */
+
+/**
+ * Is `value` shaped like a socket factory? Only callability can be checked without dialling, so
+ * that is what this checks — the duplex contract is enforced when a socket is actually opened.
+ * @param {unknown} value
+ * @returns {value is ConnectFn}
+ */
+export function isConnectFn(value) {
+  return typeof value === 'function';
+}
+
+/**
+ * Resolve a ConnectFn from an injected factory or from the first module specifier that yields one.
+ *
+ * Resolution order, and nothing else is consulted:
+ *   1. `options.connect`, when callable.
+ *   2. each entry of `options.specifiers`, in order.
+ *
+ * Throws ConfigError (CONFIG_UNSATISFIABLE) when nothing resolves, listing every specifier tried
+ * with the reason it failed. A socket factory that cannot be found is a deployment mistake, and a
+ * deployment mistake deserves to name the thing that was missing rather than surface later as
+ * "connect is not a function" from inside the TLS layer.
+ *
+ * @param {ResolveConnectOptions} [options]
+ * @returns {Promise<ConnectFn>}
+ */
+export async function resolveConnect(options = {}) {
+  const { connect, specifiers = [], importModule = defaultImport, exportName = 'connect' } = options;
+
+  if (connect !== undefined && connect !== null) {
+    if (!isConnectFn(connect)) {
+      throw new ConfigError(
+        codes.CONFIG_INVALID,
+        `connect must be a function returning { readable, writable, opened?, close? }, got ` +
+          `${describe(connect)}`,
+      );
+    }
+    return connect;
+  }
+
+  if (!Array.isArray(specifiers)) {
+    throw new ConfigError(codes.CONFIG_INVALID, 'specifiers must be an array of module specifiers');
+  }
+  if (typeof importModule !== 'function') {
+    throw new ConfigError(codes.CONFIG_INVALID, 'importModule must be a function');
+  }
+
+  /** @type {string[]} */
+  const tried = [];
+  for (const specifier of specifiers) {
+    if (typeof specifier !== 'string' || specifier === '') {
+      throw new ConfigError(
+        codes.CONFIG_INVALID,
+        `specifiers must be non-empty strings, got ${describe(specifier)}`,
+      );
+    }
+    let mod;
+    try {
+      mod = await importModule(specifier);
+    } catch (cause) {
+      // Not resolvable on this runtime, which for a portable specifier list is the expected
+      // outcome for every entry but one. Recorded, not thrown.
+      tried.push(`${specifier}: not importable (${cause?.message ?? cause})`);
+      continue;
+    }
+    const candidate = mod?.[exportName] ?? mod?.default?.[exportName];
+    if (isConnectFn(candidate)) return candidate;
+    tried.push(`${specifier}: imported, but has no callable "${exportName}" export`);
+  }
+
+  throw new ConfigError(
+    codes.CONFIG_UNSATISFIABLE,
+    'no socket factory could be resolved. Supply `connect` directly — the raw-TCP socket factory ' +
+      'of the host runtime, returning { readable, writable, opened?, close? } — or name a module ' +
+      'that exports one. ' +
+      (tried.length ? `Tried:\n  ${tried.join('\n  ')}` : 'No specifiers were given.'),
+    { tried },
+  );
+}
+
+/**
+ * Flatten a runtime socket into a plain duplex object.
+ *
+ * Worth its own function because of a trap that has already cost this package a bug: on the edge
+ * runtime a socket's `readable` and `writable` are ACCESSORS ON THE PROTOTYPE, so `{ ...socket }`
+ * copies neither and the first read fails far away, inside the TLS layer, with "Cannot read
+ * properties of undefined (reading 'getReader')". Reading the properties explicitly — which is
+ * what this does — is the only spread-safe way to hand a host socket to code that may copy it.
+ *
+ * @param {Duplex} socket
+ * @returns {Duplex}
+ */
+export function normaliseSocket(socket) {
+  if (!socket || typeof socket !== 'object') {
+    throw new ConfigError(
+      codes.CONFIG_INVALID,
+      `connect must return a socket object, got ${describe(socket)}`,
+    );
+  }
+  const { readable, writable } = socket;
+  if (!readable || typeof readable.getReader !== 'function') {
+    throw new ConfigError(codes.CONFIG_INVALID, 'socket has no ReadableStream `readable`');
+  }
+  if (!writable || typeof writable.getWriter !== 'function') {
+    throw new ConfigError(codes.CONFIG_INVALID, 'socket has no WritableStream `writable`');
+  }
+  return {
+    readable,
+    writable,
+    opened: socket.opened,
+    close: () => socket.close?.(),
+  };
+}
+
+/**
+ * Wrap a ConnectFn so every socket it returns is flattened by normaliseSocket. Useful when the
+ * factory comes from a runtime whose sockets are host objects rather than plain records.
+ * @param {ConnectFn} connect
+ * @returns {ConnectFn}
+ */
+export function normalisingConnect(connect) {
+  if (!isConnectFn(connect)) {
+    throw new ConfigError(codes.CONFIG_INVALID, `connect must be a function, got ${describe(connect)}`);
+  }
+  return (addr, opts) => normaliseSocket(connect(addr, opts));
+}
+
+/** The default importer: a dynamic import a bundler cannot see through. See the header caveat. */
+function defaultImport(specifier) {
+  return import(specifier);
+}
+
+/** A short, safe rendering of a bad argument for an error message. */
+function describe(value) {
+  if (value === null) return 'null';
+  if (Array.isArray(value)) return 'an array';
+  const t = typeof value;
+  return t === 'object' ? 'an object' : t === 'string' ? JSON.stringify(value) : t;
+}

package/src/index.js

@@ -16,6 +16,7 @@ export { warmup } from './warmup.js';
 export { ConnectionPool, poolKey } from './pool.js';
 export { openConnection, targetFromUrl, nativeFetchCanServe } from './transport.js';
 export { openTunnel, parseProxy } from './proxy/index.js';
+export { resolveConnect, isConnectFn, normaliseSocket, normalisingConnect } from './connect.js';
 export { CookieJar } from './client/cookies.js';
 export { verifyChain, rootStoreProvenance } from './trust/index.js';
 

package/types/connect.d.ts

@@ -0,0 +1,78 @@
+/** @typedef {import('./proxy/index.js').ConnectFn} ConnectFn */
+/** @typedef {import('./proxy/index.js').Duplex} Duplex */
+/**
+ * @typedef {object} ResolveConnectOptions
+ * @property {ConnectFn} [connect] an already-obtained socket factory. Checked first and returned
+ *   as-is, so the recommended path costs no import and cannot be defeated by a bundler.
+ * @property {string[]} [specifiers] module specifiers to try, in order. The first module whose
+ *   `exportName` is callable wins.
+ * @property {(specifier: string) => Promise<object>} [importModule] the importer. Defaults to a
+ *   dynamic `import()`; override it to hand the bundler a literal specifier.
+ * @property {string} [exportName] the export to read off each module. Default 'connect'.
+ */
+/**
+ * Is `value` shaped like a socket factory? Only callability can be checked without dialling, so
+ * that is what this checks — the duplex contract is enforced when a socket is actually opened.
+ * @param {unknown} value
+ * @returns {value is ConnectFn}
+ */
+export function isConnectFn(value: unknown): value is ConnectFn;
+/**
+ * Resolve a ConnectFn from an injected factory or from the first module specifier that yields one.
+ *
+ * Resolution order, and nothing else is consulted:
+ *   1. `options.connect`, when callable.
+ *   2. each entry of `options.specifiers`, in order.
+ *
+ * Throws ConfigError (CONFIG_UNSATISFIABLE) when nothing resolves, listing every specifier tried
+ * with the reason it failed. A socket factory that cannot be found is a deployment mistake, and a
+ * deployment mistake deserves to name the thing that was missing rather than surface later as
+ * "connect is not a function" from inside the TLS layer.
+ *
+ * @param {ResolveConnectOptions} [options]
+ * @returns {Promise<ConnectFn>}
+ */
+export function resolveConnect(options?: ResolveConnectOptions): Promise<ConnectFn>;
+/**
+ * Flatten a runtime socket into a plain duplex object.
+ *
+ * Worth its own function because of a trap that has already cost this package a bug: on the edge
+ * runtime a socket's `readable` and `writable` are ACCESSORS ON THE PROTOTYPE, so `{ ...socket }`
+ * copies neither and the first read fails far away, inside the TLS layer, with "Cannot read
+ * properties of undefined (reading 'getReader')". Reading the properties explicitly — which is
+ * what this does — is the only spread-safe way to hand a host socket to code that may copy it.
+ *
+ * @param {Duplex} socket
+ * @returns {Duplex}
+ */
+export function normaliseSocket(socket: Duplex): Duplex;
+/**
+ * Wrap a ConnectFn so every socket it returns is flattened by normaliseSocket. Useful when the
+ * factory comes from a runtime whose sockets are host objects rather than plain records.
+ * @param {ConnectFn} connect
+ * @returns {ConnectFn}
+ */
+export function normalisingConnect(connect: ConnectFn): ConnectFn;
+export type ConnectFn = import("./proxy/index.js").ConnectFn;
+export type Duplex = import("./proxy/index.js").Duplex;
+export type ResolveConnectOptions = {
+    /**
+     * an already-obtained socket factory. Checked first and returned
+     * as-is, so the recommended path costs no import and cannot be defeated by a bundler.
+     */
+    connect?: import("./proxy/index.js").ConnectFn | undefined;
+    /**
+     * module specifiers to try, in order. The first module whose
+     * `exportName` is callable wins.
+     */
+    specifiers?: string[] | undefined;
+    /**
+     * the importer. Defaults to a
+     * dynamic `import()`; override it to hand the bundler a literal specifier.
+     */
+    importModule?: ((specifier: string) => Promise<object>) | undefined;
+    /**
+     * the export to read off each module. Default 'connect'.
+     */
+    exportName?: string | undefined;
+};

package/types/index.d.ts

@@ -13,6 +13,7 @@ export { Client, createFetch, install } from "./client.js";
 export { ConnectionPool, poolKey } from "./pool.js";
 export { openConnection, targetFromUrl, nativeFetchCanServe } from "./transport.js";
 export { openTunnel, parseProxy } from "./proxy/index.js";
+export { resolveConnect, isConnectFn, normaliseSocket, normalisingConnect } from "./connect.js";
 export { verifyChain, rootStoreProvenance } from "./trust/index.js";
 export { TunnelFetchError, ProxyError, HttpError, TlsError, TlsUnsupportedError, Http2Error, CertificateError, TimeoutError, LimitError, ConfigError, codes } from "./errors.js";
 export { profiles, curl, chrome, applyProfile } from "./profiles.js";