undici 4.15.1 → 5.1.0

package/README.md

@@ -2,7 +2,7 @@
 
 [![Node CI](https://github.com/nodejs/undici/actions/workflows/nodejs.yml/badge.svg)](https://github.com/nodejs/undici/actions/workflows/nodejs.yml) [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat)](http://standardjs.com/) [![npm version](https://badge.fury.io/js/undici.svg)](https://badge.fury.io/js/undici) [![codecov](https://codecov.io/gh/nodejs/undici/branch/main/graph/badge.svg?token=yZL6LtXkOA)](https://codecov.io/gh/nodejs/undici)
 
-A HTTP/1.1 client, written from scratch for Node.js.
+An HTTP/1.1 client, written from scratch for Node.js.
 
 > Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici.
 It is also a Stranger Things reference.
@@ -65,7 +65,15 @@ for await (const data of body) {
 console.log('trailers', trailers)
 ```
 
-Using [the body mixin from the Fetch Standard](https://fetch.spec.whatwg.org/#body-mixin).
+## Body Mixins
+
+The `body` mixins are the most common way to format the request/response body. Mixins include:
+
+- [`.formData()`](https://fetch.spec.whatwg.org/#dom-body-formdata)
+- [`.json()`](https://fetch.spec.whatwg.org/#dom-body-json)
+- [`.text()`](https://fetch.spec.whatwg.org/#dom-body-text)
+
+Example usage:
 
 ```js
 import { request } from 'undici'
@@ -83,6 +91,12 @@ console.log('data', await body.json())
 console.log('trailers', trailers)
 ```
 
+_Note: Once a mixin has been called then the body cannot be reused, thus calling additional mixins on `.body`, e.g. `.body.json(); .body.text()` will result in an error `TypeError: unusable` being thrown and returned through the `Promise` rejection._
+
+Should you need to access the `body` in plain-text after using a mixin, the best practice is to use the `.text()` mixin first and then manually parse the text to the desired format.
+
+For more information about their behavior, please reference the body mixin from the [Fetch Standard](https://fetch.spec.whatwg.org/#body-mixin).
+
 ## Common API Methods
 
 This section documents our most commonly used API methods. Additional APIs are documented in their own files within the [docs](./docs/) folder and are accessible via the navigation list on the left side of the docs site.
@@ -213,7 +227,7 @@ const data = {
 
 #### `response.body`
 
-Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v16.x/docs/api/webstreams.html) which follow the API of the WHATWG web standard found in browsers, and an older Node-specific [streams API](https://nodejs.org/api/stream.html). `response.body` returns a readable web stream. If you would prefer to work with a Node stream you can convert a web stream using `.fromWeb()`.
+Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v16.x/docs/api/webstreams.html), which follow the API of the WHATWG web standard found in browsers, and an older Node-specific [streams API](https://nodejs.org/api/stream.html). `response.body` returns a readable web stream. If you would prefer to work with a Node stream you can convert a web stream using `.fromWeb()`.
 
 ```js
     import {fetch} from 'undici';
@@ -228,7 +242,7 @@ Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v1
 
 #### Specification Compliance
 
-This section documents parts of the [Fetch Standard](https://fetch.spec.whatwg.org) which Undici does
+This section documents parts of the [Fetch Standard](https://fetch.spec.whatwg.org) that Undici does
 not support or does not fully implement.
 
 ##### Garbage Collection
@@ -239,7 +253,7 @@ The [Fetch Standard](https://fetch.spec.whatwg.org) allows users to skip consumi
 [garbage collection](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_Management#garbage_collection) to release connection resources. Undici does not do the same. Therefore, it is important to always either consume or cancel the response body.
 
 Garbage collection in Node is less aggressive and deterministic
-(due to the lack of clear idle periods that browser have through the rendering refresh rate)
+(due to the lack of clear idle periods that browsers have through the rendering refresh rate)
 which means that leaving the release of connection resources to the garbage collector can lead
 to excessive connection usage, reduced performance (due to less connection re-use), and even
 stalls or deadlocks when running out of connections.
@@ -301,7 +315,7 @@ Returns: `Dispatcher`
 
 ## Specification Compliance
 
-This section documents parts of the HTTP/1.1 specification which Undici does
+This section documents parts of the HTTP/1.1 specification that Undici does
 not support or does not fully implement.
 
 ### Expect
@@ -334,7 +348,7 @@ aborted.
 
 ### Manual Redirect
 
-Since it is not possible to manually follow an HTTP redirect on server-side,
+Since it is not possible to manually follow an HTTP redirect on the server-side,
 Undici returns the actual response instead of an `opaqueredirect` filtered one
 when invoked with a `manual` redirect. This aligns `fetch()` with the other
 implementations in Deno and Cloudflare Workers.

package/docs/api/Dispatcher.md

@@ -193,7 +193,7 @@ Returns: `Boolean` - `false` if dispatcher is busy and further dispatch calls wo
 * **path** `string`
 * **method** `string`
 * **body** `string | Buffer | Uint8Array | stream.Readable | Iterable | AsyncIterable | null` (optional) - Default: `null`
-* **headers** `UndiciHeaders` (optional) - Default: `null`
+* **headers** `UndiciHeaders | string[]` (optional) - Default: `null`.
 * **idempotent** `boolean` (optional) - Default: `true` if `method` is `'HEAD'` or `'GET'` - Whether the requests can be safely retried or not. If `false` the request won't be sent until all preceding requests in the pipeline has completed.
 * **blocking** `boolean` (optional) - Default: `false` - Whether the response is expected to take a long time and would end up blocking the pipeline. When this is set to `true` further pipelining will be avoided on the same connection until headers have been received.
 * **upgrade** `string | null` (optional) - Default: `null` - Upgrade the request. Should be used to specify the kind of upgrade i.e. `'Websocket'`.
@@ -489,6 +489,8 @@ The `RequestOptions.method` property should not be value `'CONNECT'`.
 - `body`
 - `bodyUsed`
 
+`body` can not be consumed twice. For example, calling `text()` after `json()` throws `TypeError`.
+
 `body` contains the following additional extensions:
 
 - `dump({ limit: Integer })`, dump the response by reading up to `limit` bytes without killing the socket (optional) - Default: 262144.

package/docs/api/MockAgent.md

@@ -72,11 +72,7 @@ const mockAgent = new MockAgent()
 setGlobalDispatcher(mockAgent)
 
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const { statusCode, body } = await request('http://localhost:3000/foo')
 
@@ -95,11 +91,7 @@ import { MockAgent, request } from 'undici'
 const mockAgent = new MockAgent()
 
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -121,11 +113,7 @@ import { MockAgent, request } from 'undici'
 const mockAgent = new MockAgent()
 
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -147,11 +135,7 @@ import { MockAgent, request } from 'undici'
 const mockAgent = new MockAgent({ connections: 1 })
 
 const mockClient = mockAgent.get('http://localhost:3000')
-
-mockClient.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockClient.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -174,16 +158,8 @@ const mockAgent = new MockAgent()
 setGlobalDispatcher(mockAgent)
 
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
-
-mockPool.intercept({
-  path: '/hello',
-  method: 'GET'
-}).reply(200, 'hello')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
+mockPool.intercept({ path: '/hello'}).reply(200, 'hello')
 
 const result1 = await request('http://localhost:3000/foo')
 
@@ -250,11 +226,7 @@ const mockAgent = new MockAgent()
 setGlobalDispatcher(mockAgent)
 
 const mockPool = mockAgent.get(new RegExp('http://localhost:3000'))
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET',
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -277,11 +249,7 @@ const mockAgent = new MockAgent()
 setGlobalDispatcher(mockAgent)
 
 const mockPool = mockAgent.get((origin) => origin === 'http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -328,11 +296,7 @@ import { MockAgent } from 'undici'
 const mockAgent = new MockAgent()
 
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET'
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,
@@ -481,3 +445,79 @@ mockAgent.disableNetConnect()
 await request('http://example.com')
 // Will throw
 ```
+
+### `MockAgent.pendingInterceptors()`
+
+This method returns any pending interceptors registered on a mock agent. A pending interceptor meets one of the following criteria:
+
+- Is registered with neither `.times(<number>)` nor `.persist()`, and has not been invoked;
+- Is persistent (i.e., registered with `.persist()`) and has not been invoked;
+- Is registered with `.times(<number>)` and has not been invoked `<number>` of times.
+
+Returns: `PendingInterceptor[]` (where `PendingInterceptor` is a `MockDispatch` with an additional `origin: string`)
+
+#### Example - List all pending inteceptors
+
+```js
+const agent = new MockAgent()
+agent.disableNetConnect()
+
+agent
+  .get('https://example.com')
+  .intercept({ method: 'GET', path: '/' })
+  .reply(200, '')
+
+const pendingInterceptors = agent.pendingInterceptors()
+// Returns [
+//   {
+//     timesInvoked: 0,
+//     times: 1,
+//     persist: false,
+//     consumed: false,
+//     pending: true,
+//     path: '/',
+//     method: 'GET',
+//     body: undefined,
+//     headers: undefined,
+//     data: {
+//       error: null,
+//       statusCode: 200,
+//       data: '',
+//       headers: {},
+//       trailers: {}
+//     },
+//     origin: 'https://example.com'
+//   }
+// ]
+```
+
+### `MockAgent.assertNoPendingInterceptors([options])`
+
+This method throws if the mock agent has any pending interceptors. A pending interceptor meets one of the following criteria:
+
+- Is registered with neither `.times(<number>)` nor `.persist()`, and has not been invoked;
+- Is persistent (i.e., registered with `.persist()`) and has not been invoked;
+- Is registered with `.times(<number>)` and has not been invoked `<number>` of times.
+
+#### Example - Check that there are no pending interceptors
+
+```js
+const agent = new MockAgent()
+agent.disableNetConnect()
+
+agent
+  .get('https://example.com')
+  .intercept({ method: 'GET', path: '/' })
+  .reply(200, '')
+
+agent.assertNoPendingInterceptors()
+// Throws an UndiciError with the following message:
+//
+// 1 interceptor is pending:
+//
+// ┌─────────┬────────┬───────────────────────┬──────┬─────────────┬────────────┬─────────────┬───────────┐
+// │ (index) │ Method │        Origin         │ Path │ Status code │ Persistent │ Invocations │ Remaining │
+// ├─────────┼────────┼───────────────────────┼──────┼─────────────┼────────────┼─────────────┼───────────┤
+// │    0    │ 'GET'  │ 'https://example.com' │ '/'  │     200     │    '❌'    │      0      │     1     │
+// └─────────┴────────┴───────────────────────┴──────┴─────────────┴────────────┴─────────────┴───────────┘
+```

package/docs/api/MockClient.md

@@ -58,10 +58,7 @@ import { MockAgent } from 'undici'
 const mockAgent = new MockAgent({ connections: 1 })
 
 const mockClient = mockAgent.get('http://localhost:3000')
-mockClient.intercept({
-  path: '/foo',
-  method: 'GET',
-}).reply(200, 'foo')
+mockClient.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,

package/docs/api/MockPool.md

@@ -95,11 +95,7 @@ setGlobalDispatcher(mockAgent)
 
 // MockPool
 const mockPool = mockAgent.get('http://localhost:3000')
-
-mockPool.intercept({
-  path: '/foo',
-  method: 'GET',
-}).reply(200, 'foo')
+mockPool.intercept({ path: '/foo' }).reply(200, 'foo')
 
 const {
   statusCode,

package/docs/best-practices/mocking-request.md

@@ -5,17 +5,20 @@ Undici have its own mocking [utility](../api/MockAgent.md). It allow us to inter
 Example:
 
 ```js
-// index.mjs
+// bank.mjs
 import { request } from 'undici'
 
-export async function bankTransfer(recepient, ammount) {
-  const { body } = await request('http://localhost:3000/bank-transfer', 
+export async function bankTransfer(recepient, amount) {
+  const { body } = await request('http://localhost:3000/bank-transfer',
     {
       method: 'POST',
       headers: {
         'X-TOKEN-SECRET': 'SuperSecretToken',
       },
-      body: JSON.stringify({ recepient })
+      body: JSON.stringify({
+        recepient,
+        amount
+      })
     }
   )
   return await body.json()
@@ -28,7 +31,7 @@ And this is what the test file looks like:
 // index.test.mjs
 import { strict as assert } from 'assert'
 import { MockAgent, setGlobalDispatcher, } from 'undici'
-import { bankTransfer } from './undici.mjs'
+import { bankTransfer } from './bank.mjs'
 
 const mockAgent = new MockAgent();
 
@@ -46,7 +49,7 @@ mockPool.intercept({
   },
   body: JSON.stringify({
     recepient: '1234567890',
-    ammount: '100'
+    amount: '100'
   })
 }).reply(200, {
   message: 'transaction processed'
@@ -94,7 +97,7 @@ mockPool.intercept({
 
 const badRequest = await bankTransfer('1234567890', '100')
 // Will throw an error
-// MockNotMatchedError: Mock dispatch not matched for path '/bank-transfer': 
+// MockNotMatchedError: Mock dispatch not matched for path '/bank-transfer':
 // subsequent request to origin http://localhost:3000 was not allowed (net.connect disabled)
 ```
 

package/index.d.ts

@@ -16,6 +16,7 @@ import { request, pipeline, stream, connect, upgrade } from './types/api'
 export * from './types/fetch'
 export * from './types/file'
 export * from './types/formdata'
+export { Interceptable } from './types/mock-interceptor'
 
 export { Dispatcher, BalancedPool, Pool, Client, buildConnector, errors, Agent, request, stream, pipeline, connect, upgrade, setGlobalDispatcher, getGlobalDispatcher, MockClient, MockPool, MockAgent, mockErrors, ProxyAgent }
 export default Undici

package/lib/agent.js

@@ -1,20 +1,14 @@
 'use strict'
 
-const {
-  ClientClosedError,
-  InvalidArgumentError,
-  ClientDestroyedError
-} = require('./core/errors')
-const { kClients, kRunning } = require('./core/symbols')
-const Dispatcher = require('./dispatcher')
+const { InvalidArgumentError } = require('./core/errors')
+const { kClients, kRunning, kClose, kDestroy, kDispatch } = require('./core/symbols')
+const DispatcherBase = require('./dispatcher-base')
 const Pool = require('./pool')
 const Client = require('./client')
 const util = require('./core/util')
 const RedirectHandler = require('./handler/redirect')
 const { WeakRef, FinalizationRegistry } = require('./compat/dispatcher-weakref')()
 
-const kDestroyed = Symbol('destroyed')
-const kClosed = Symbol('closed')
 const kOnConnect = Symbol('onConnect')
 const kOnDisconnect = Symbol('onDisconnect')
 const kOnConnectionError = Symbol('onConnectionError')
@@ -30,7 +24,7 @@ function defaultFactory (origin, opts) {
     : new Pool(origin, opts)
 }
 
-class Agent extends Dispatcher {
+class Agent extends DispatcherBase {
   constructor ({ factory = defaultFactory, maxRedirections = 0, connect, ...options } = {}) {
     super()
 
@@ -60,8 +54,6 @@ class Agent extends Dispatcher {
         this[kClients].delete(key)
       }
     })
-    this[kClosed] = false
-    this[kDestroyed] = false
 
     const agent = this
 
@@ -94,76 +86,38 @@ class Agent extends Dispatcher {
     return ret
   }
 
-  dispatch (opts, handler) {
-    if (!handler || typeof handler !== 'object') {
-      throw new InvalidArgumentError('handler must be an object.')
+  [kDispatch] (opts, handler) {
+    let key
+    if (opts.origin && (typeof opts.origin === 'string' || opts.origin instanceof URL)) {
+      key = String(opts.origin)
+    } else {
+      throw new InvalidArgumentError('opts.origin must be a non-empty string or URL.')
     }
 
-    try {
-      if (!opts || typeof opts !== 'object') {
-        throw new InvalidArgumentError('opts must be an object.')
-      }
-
-      let key
-      if (opts.origin && (typeof opts.origin === 'string' || opts.origin instanceof URL)) {
-        key = String(opts.origin)
-      } else {
-        throw new InvalidArgumentError('opts.origin must be a non-empty string or URL.')
-      }
-
-      if (this[kDestroyed]) {
-        throw new ClientDestroyedError()
-      }
-
-      if (this[kClosed]) {
-        throw new ClientClosedError()
-      }
+    const ref = this[kClients].get(key)
 
-      const ref = this[kClients].get(key)
-
-      let dispatcher = ref ? ref.deref() : null
-      if (!dispatcher) {
-        dispatcher = this[kFactory](opts.origin, this[kOptions])
-          .on('drain', this[kOnDrain])
-          .on('connect', this[kOnConnect])
-          .on('disconnect', this[kOnDisconnect])
-          .on('connectionError', this[kOnConnectionError])
-
-        this[kClients].set(key, new WeakRef(dispatcher))
-        this[kFinalizer].register(dispatcher, key)
-      }
-
-      const { maxRedirections = this[kMaxRedirections] } = opts
-      if (maxRedirections != null && maxRedirections !== 0) {
-        opts = { ...opts, maxRedirections: 0 } // Stop sub dispatcher from also redirecting.
-        handler = new RedirectHandler(this, maxRedirections, opts, handler)
-      }
-
-      return dispatcher.dispatch(opts, handler)
-    } catch (err) {
-      if (typeof handler.onError !== 'function') {
-        throw new InvalidArgumentError('invalid onError method')
-      }
+    let dispatcher = ref ? ref.deref() : null
+    if (!dispatcher) {
+      dispatcher = this[kFactory](opts.origin, this[kOptions])
+        .on('drain', this[kOnDrain])
+        .on('connect', this[kOnConnect])
+        .on('disconnect', this[kOnDisconnect])
+        .on('connectionError', this[kOnConnectionError])
 
-      handler.onError(err)
+      this[kClients].set(key, new WeakRef(dispatcher))
+      this[kFinalizer].register(dispatcher, key)
     }
-  }
-
-  get closed () {
-    return this[kClosed]
-  }
-
-  get destroyed () {
-    return this[kDestroyed]
-  }
 
-  close (callback) {
-    if (callback != null && typeof callback !== 'function') {
-      throw new InvalidArgumentError('callback must be a function')
+    const { maxRedirections = this[kMaxRedirections] } = opts
+    if (maxRedirections != null && maxRedirections !== 0) {
+      opts = { ...opts, maxRedirections: 0 } // Stop sub dispatcher from also redirecting.
+      handler = new RedirectHandler(this, maxRedirections, opts, handler)
     }
 
-    this[kClosed] = true
+    return dispatcher.dispatch(opts, handler)
+  }
 
+  async [kClose] () {
     const closePromises = []
     for (const ref of this[kClients].values()) {
       const client = ref.deref()
@@ -173,27 +127,10 @@ class Agent extends Dispatcher {
       }
     }
 
-    if (!callback) {
-      return Promise.all(closePromises)
-    }
-
-    // Should never error.
-    Promise.all(closePromises).then(() => process.nextTick(callback))
+    await Promise.all(closePromises)
   }
 
-  destroy (err, callback) {
-    if (typeof err === 'function') {
-      callback = err
-      err = null
-    }
-
-    if (callback != null && typeof callback !== 'function') {
-      throw new InvalidArgumentError('callback must be a function')
-    }
-
-    this[kClosed] = true
-    this[kDestroyed] = true
-
+  async [kDestroy] (err) {
     const destroyPromises = []
     for (const ref of this[kClients].values()) {
       const client = ref.deref()
@@ -203,12 +140,7 @@ class Agent extends Dispatcher {
       }
     }
 
-    if (!callback) {
-      return Promise.all(destroyPromises)
-    }
-
-    // Should never error.
-    Promise.all(destroyPromises).then(() => process.nextTick(callback))
+    await Promise.all(destroyPromises)
   }
 }
 

package/lib/api/api-request.js

@@ -88,14 +88,16 @@ class RequestHandler extends AsyncResource {
     this.res = body
     const headers = this.responseHeaders === 'raw' ? util.parseRawHeaders(rawHeaders) : util.parseHeaders(rawHeaders)
 
-    this.runInAsyncScope(callback, null, null, {
-      statusCode,
-      headers,
-      trailers: this.trailers,
-      opaque,
-      body,
-      context
-    })
+    if (callback !== null) {
+      this.runInAsyncScope(callback, null, null, {
+        statusCode,
+        headers,
+        trailers: this.trailers,
+        opaque,
+        body,
+        context
+      })
+    }
   }
 
   onData (chunk) {

package/lib/balanced-pool.js

@@ -20,7 +20,7 @@ const kFactory = Symbol('factory')
 const kOptions = Symbol('options')
 
 function defaultFactory (origin, opts) {
-  return new Pool(origin, opts);
+  return new Pool(origin, opts)
 }
 
 class BalancedPool extends PoolBase {