undici 5.0.0 → 5.2.0

package/README.md

@@ -2,7 +2,7 @@
 
 [![Node CI](https://github.com/nodejs/undici/actions/workflows/nodejs.yml/badge.svg)](https://github.com/nodejs/undici/actions/workflows/nodejs.yml) [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat)](http://standardjs.com/) [![npm version](https://badge.fury.io/js/undici.svg)](https://badge.fury.io/js/undici) [![codecov](https://codecov.io/gh/nodejs/undici/branch/main/graph/badge.svg?token=yZL6LtXkOA)](https://codecov.io/gh/nodejs/undici)
 
-A HTTP/1.1 client, written from scratch for Node.js.
+An HTTP/1.1 client, written from scratch for Node.js.
 
 > Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici.
 It is also a Stranger Things reference.
@@ -65,7 +65,15 @@ for await (const data of body) {
 console.log('trailers', trailers)
 ```
 
-Using [the body mixin from the Fetch Standard](https://fetch.spec.whatwg.org/#body-mixin).
+## Body Mixins
+
+The `body` mixins are the most common way to format the request/response body. Mixins include:
+
+- [`.formData()`](https://fetch.spec.whatwg.org/#dom-body-formdata)
+- [`.json()`](https://fetch.spec.whatwg.org/#dom-body-json)
+- [`.text()`](https://fetch.spec.whatwg.org/#dom-body-text)
+
+Example usage:
 
 ```js
 import { request } from 'undici'
@@ -83,6 +91,12 @@ console.log('data', await body.json())
 console.log('trailers', trailers)
 ```
 
+_Note: Once a mixin has been called then the body cannot be reused, thus calling additional mixins on `.body`, e.g. `.body.json(); .body.text()` will result in an error `TypeError: unusable` being thrown and returned through the `Promise` rejection._
+
+Should you need to access the `body` in plain-text after using a mixin, the best practice is to use the `.text()` mixin first and then manually parse the text to the desired format.
+
+For more information about their behavior, please reference the body mixin from the [Fetch Standard](https://fetch.spec.whatwg.org/#body-mixin).
+
 ## Common API Methods
 
 This section documents our most commonly used API methods. Additional APIs are documented in their own files within the [docs](./docs/) folder and are accessible via the navigation list on the left side of the docs site.
@@ -180,6 +194,21 @@ Basic usage example:
     }
 ```
 
+You can pass an optional dispatcher to `fetch` as:
+
+```js
+import { fetch, Agent } from 'undici'
+  
+const res = await fetch('https://example.com', {
+  // Mocks are also supported
+  dispatcher: new Agent({
+    keepAliveTimeout: 10,
+    keepAliveMaxTimeout: 10
+  })
+})
+const json = await res.json()
+console.log(json)
+```
 
 #### `request.body`
 
@@ -213,7 +242,7 @@ const data = {
 
 #### `response.body`
 
-Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v16.x/docs/api/webstreams.html) which follow the API of the WHATWG web standard found in browsers, and an older Node-specific [streams API](https://nodejs.org/api/stream.html). `response.body` returns a readable web stream. If you would prefer to work with a Node stream you can convert a web stream using `.fromWeb()`.
+Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v16.x/docs/api/webstreams.html), which follow the API of the WHATWG web standard found in browsers, and an older Node-specific [streams API](https://nodejs.org/api/stream.html). `response.body` returns a readable web stream. If you would prefer to work with a Node stream you can convert a web stream using `.fromWeb()`.
 
 ```js
     import {fetch} from 'undici';
@@ -228,7 +257,7 @@ Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v1
 
 #### Specification Compliance
 
-This section documents parts of the [Fetch Standard](https://fetch.spec.whatwg.org) which Undici does
+This section documents parts of the [Fetch Standard](https://fetch.spec.whatwg.org) that Undici does
 not support or does not fully implement.
 
 ##### Garbage Collection
@@ -239,7 +268,7 @@ The [Fetch Standard](https://fetch.spec.whatwg.org) allows users to skip consumi
 [garbage collection](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_Management#garbage_collection) to release connection resources. Undici does not do the same. Therefore, it is important to always either consume or cancel the response body.
 
 Garbage collection in Node is less aggressive and deterministic
-(due to the lack of clear idle periods that browser have through the rendering refresh rate)
+(due to the lack of clear idle periods that browsers have through the rendering refresh rate)
 which means that leaving the release of connection resources to the garbage collector can lead
 to excessive connection usage, reduced performance (due to less connection re-use), and even
 stalls or deadlocks when running out of connections.
@@ -301,7 +330,7 @@ Returns: `Dispatcher`
 
 ## Specification Compliance
 
-This section documents parts of the HTTP/1.1 specification which Undici does
+This section documents parts of the HTTP/1.1 specification that Undici does
 not support or does not fully implement.
 
 ### Expect
@@ -334,7 +363,7 @@ aborted.
 
 ### Manual Redirect
 
-Since it is not possible to manually follow an HTTP redirect on server-side,
+Since it is not possible to manually follow an HTTP redirect on the server-side,
 Undici returns the actual response instead of an `opaqueredirect` filtered one
 when invoked with a `manual` redirect. This aligns `fetch()` with the other
 implementations in Deno and Cloudflare Workers.

package/docs/api/Dispatcher.md

@@ -193,7 +193,7 @@ Returns: `Boolean` - `false` if dispatcher is busy and further dispatch calls wo
 * **path** `string`
 * **method** `string`
 * **body** `string | Buffer | Uint8Array | stream.Readable | Iterable | AsyncIterable | null` (optional) - Default: `null`
-* **headers** `UndiciHeaders` (optional) - Default: `null`
+* **headers** `UndiciHeaders | string[]` (optional) - Default: `null`.
 * **idempotent** `boolean` (optional) - Default: `true` if `method` is `'HEAD'` or `'GET'` - Whether the requests can be safely retried or not. If `false` the request won't be sent until all preceding requests in the pipeline has completed.
 * **blocking** `boolean` (optional) - Default: `false` - Whether the response is expected to take a long time and would end up blocking the pipeline. When this is set to `true` further pipelining will be avoided on the same connection until headers have been received.
 * **upgrade** `string | null` (optional) - Default: `null` - Upgrade the request. Should be used to specify the kind of upgrade i.e. `'Websocket'`.

package/docs/api/Errors.md

@@ -19,7 +19,6 @@ import { errors } from 'undici'
 | `RequestContentLengthMismatchError`  | `UND_ERR_REQ_CONTENT_LENGTH_MISMATCH` | request body does not match content-length header  |
 | `ResponseContentLengthMismatchError` | `UND_ERR_RES_CONTENT_LENGTH_MISMATCH` | response body does not match content-length header |
 | `InformationalError`                 | `UND_ERR_INFO`                        | expected error with reason                         |
-| `TrailerMismatchError`               | `UND_ERR_TRAILER_MISMATCH`            | trailers did not match specification               |
 
 ### `SocketError`
 

package/docs/api/MockAgent.md

@@ -445,3 +445,79 @@ mockAgent.disableNetConnect()
 await request('http://example.com')
 // Will throw
 ```
+
+### `MockAgent.pendingInterceptors()`
+
+This method returns any pending interceptors registered on a mock agent. A pending interceptor meets one of the following criteria:
+
+- Is registered with neither `.times(<number>)` nor `.persist()`, and has not been invoked;
+- Is persistent (i.e., registered with `.persist()`) and has not been invoked;
+- Is registered with `.times(<number>)` and has not been invoked `<number>` of times.
+
+Returns: `PendingInterceptor[]` (where `PendingInterceptor` is a `MockDispatch` with an additional `origin: string`)
+
+#### Example - List all pending inteceptors
+
+```js
+const agent = new MockAgent()
+agent.disableNetConnect()
+
+agent
+  .get('https://example.com')
+  .intercept({ method: 'GET', path: '/' })
+  .reply(200, '')
+
+const pendingInterceptors = agent.pendingInterceptors()
+// Returns [
+//   {
+//     timesInvoked: 0,
+//     times: 1,
+//     persist: false,
+//     consumed: false,
+//     pending: true,
+//     path: '/',
+//     method: 'GET',
+//     body: undefined,
+//     headers: undefined,
+//     data: {
+//       error: null,
+//       statusCode: 200,
+//       data: '',
+//       headers: {},
+//       trailers: {}
+//     },
+//     origin: 'https://example.com'
+//   }
+// ]
+```
+
+### `MockAgent.assertNoPendingInterceptors([options])`
+
+This method throws if the mock agent has any pending interceptors. A pending interceptor meets one of the following criteria:
+
+- Is registered with neither `.times(<number>)` nor `.persist()`, and has not been invoked;
+- Is persistent (i.e., registered with `.persist()`) and has not been invoked;
+- Is registered with `.times(<number>)` and has not been invoked `<number>` of times.
+
+#### Example - Check that there are no pending interceptors
+
+```js
+const agent = new MockAgent()
+agent.disableNetConnect()
+
+agent
+  .get('https://example.com')
+  .intercept({ method: 'GET', path: '/' })
+  .reply(200, '')
+
+agent.assertNoPendingInterceptors()
+// Throws an UndiciError with the following message:
+//
+// 1 interceptor is pending:
+//
+// ┌─────────┬────────┬───────────────────────┬──────┬─────────────┬────────────┬─────────────┬───────────┐
+// │ (index) │ Method │        Origin         │ Path │ Status code │ Persistent │ Invocations │ Remaining │
+// ├─────────┼────────┼───────────────────────┼──────┼─────────────┼────────────┼─────────────┼───────────┤
+// │    0    │ 'GET'  │ 'https://example.com' │ '/'  │     200     │    '❌'    │      0      │     1     │
+// └─────────┴────────┴───────────────────────┴──────┴─────────────┴────────────┴─────────────┴───────────┘
+```

package/docs/best-practices/mocking-request.md

@@ -5,17 +5,20 @@ Undici have its own mocking [utility](../api/MockAgent.md). It allow us to inter
 Example:
 
 ```js
-// index.mjs
+// bank.mjs
 import { request } from 'undici'
 
-export async function bankTransfer(recepient, ammount) {
-  const { body } = await request('http://localhost:3000/bank-transfer', 
+export async function bankTransfer(recepient, amount) {
+  const { body } = await request('http://localhost:3000/bank-transfer',
     {
       method: 'POST',
       headers: {
         'X-TOKEN-SECRET': 'SuperSecretToken',
       },
-      body: JSON.stringify({ recepient })
+      body: JSON.stringify({
+        recepient,
+        amount
+      })
     }
   )
   return await body.json()
@@ -28,7 +31,7 @@ And this is what the test file looks like:
 // index.test.mjs
 import { strict as assert } from 'assert'
 import { MockAgent, setGlobalDispatcher, } from 'undici'
-import { bankTransfer } from './undici.mjs'
+import { bankTransfer } from './bank.mjs'
 
 const mockAgent = new MockAgent();
 
@@ -46,7 +49,7 @@ mockPool.intercept({
   },
   body: JSON.stringify({
     recepient: '1234567890',
-    ammount: '100'
+    amount: '100'
   })
 }).reply(200, {
   message: 'transaction processed'
@@ -94,7 +97,7 @@ mockPool.intercept({
 
 const badRequest = await bankTransfer('1234567890', '100')
 // Will throw an error
-// MockNotMatchedError: Mock dispatch not matched for path '/bank-transfer': 
+// MockNotMatchedError: Mock dispatch not matched for path '/bank-transfer':
 // subsequent request to origin http://localhost:3000 was not allowed (net.connect disabled)
 ```
 

package/index-fetch.js

@@ -0,0 +1,13 @@
+'use strict'
+
+const { getGlobalDispatcher } = require('./lib/global')
+const fetchImpl = require('./lib/fetch')
+
+module.exports.fetch = async function fetch (resource) {
+  const dispatcher = (arguments[1] && arguments[1].dispatcher) || getGlobalDispatcher()
+  return fetchImpl.apply(dispatcher, arguments)
+}
+module.exports.FormData = require('./lib/fetch/formdata').FormData
+module.exports.Headers = require('./lib/fetch/headers').Headers
+module.exports.Response = require('./lib/fetch/response').Response
+module.exports.Request = require('./lib/fetch/request').Request

package/index.d.ts

@@ -16,6 +16,7 @@ import { request, pipeline, stream, connect, upgrade } from './types/api'
 export * from './types/fetch'
 export * from './types/file'
 export * from './types/formdata'
+export { Interceptable } from './types/mock-interceptor'
 
 export { Dispatcher, BalancedPool, Pool, Client, buildConnector, errors, Agent, request, stream, pipeline, connect, upgrade, setGlobalDispatcher, getGlobalDispatcher, MockClient, MockPool, MockAgent, mockErrors, ProxyAgent }
 export default Undici

package/index.js

@@ -15,6 +15,7 @@ const MockAgent = require('./lib/mock/mock-agent')
 const MockPool = require('./lib/mock/mock-pool')
 const mockErrors = require('./lib/mock/mock-errors')
 const ProxyAgent = require('./lib/proxy-agent')
+const { getGlobalDispatcher, setGlobalDispatcher } = require('./lib/global')
 
 const nodeVersion = process.versions.node.split('.')
 const nodeMajor = Number(nodeVersion[0])
@@ -32,19 +33,6 @@ module.exports.ProxyAgent = ProxyAgent
 module.exports.buildConnector = buildConnector
 module.exports.errors = errors
 
-let globalDispatcher = new Agent()
-
-function setGlobalDispatcher (agent) {
-  if (!agent || typeof agent.dispatch !== 'function') {
-    throw new InvalidArgumentError('Argument agent must implement Agent')
-  }
-  globalDispatcher = agent
-}
-
-function getGlobalDispatcher () {
-  return globalDispatcher
-}
-
 function makeDispatcher (fn) {
   return (url, opts, handler) => {
     if (typeof opts === 'function') {
@@ -98,7 +86,7 @@ if (nodeMajor > 16 || (nodeMajor === 16 && nodeMinor >= 5)) {
     if (!fetchImpl) {
       fetchImpl = require('./lib/fetch')
     }
-    const dispatcher = getGlobalDispatcher()
+    const dispatcher = (arguments[1] && arguments[1].dispatcher) || getGlobalDispatcher()
     return fetchImpl.apply(dispatcher, arguments)
   }
   module.exports.Headers = require('./lib/fetch/headers').Headers

package/lib/api/api-request.js

@@ -88,14 +88,16 @@ class RequestHandler extends AsyncResource {
     this.res = body
     const headers = this.responseHeaders === 'raw' ? util.parseRawHeaders(rawHeaders) : util.parseHeaders(rawHeaders)
 
-    this.runInAsyncScope(callback, null, null, {
-      statusCode,
-      headers,
-      trailers: this.trailers,
-      opaque,
-      body,
-      context
-    })
+    if (callback !== null) {
+      this.runInAsyncScope(callback, null, null, {
+        statusCode,
+        headers,
+        trailers: this.trailers,
+        opaque,
+        body,
+        context
+      })
+    }
   }
 
   onData (chunk) {

package/lib/client.js

@@ -11,7 +11,6 @@ const RedirectHandler = require('./handler/redirect')
 const {
   RequestContentLengthMismatchError,
   ResponseContentLengthMismatchError,
-  TrailerMismatchError,
   InvalidArgumentError,
   RequestAbortedError,
   HeadersTimeoutError,
@@ -425,7 +424,6 @@ class Parser {
 
     this.bytesRead = 0
 
-    this.trailer = ''
     this.keepAlive = ''
     this.contentLength = ''
   }
@@ -615,8 +613,6 @@ class Parser {
     const key = this.headers[len - 2]
     if (key.length === 10 && key.toString().toLowerCase() === 'keep-alive') {
       this.keepAlive += buf.toString()
-    } else if (key.length === 7 && key.toString().toLowerCase() === 'trailer') {
-      this.trailer += buf.toString()
     } else if (key.length === 14 && key.toString().toLowerCase() === 'content-length') {
       this.contentLength += buf.toString()
     }
@@ -819,7 +815,7 @@ class Parser {
   }
 
   onMessageComplete () {
-    const { client, socket, statusCode, upgrade, trailer, headers, contentLength, bytesRead, shouldKeepAlive } = this
+    const { client, socket, statusCode, upgrade, headers, contentLength, bytesRead, shouldKeepAlive } = this
 
     if (socket.destroyed && (!statusCode || shouldKeepAlive)) {
       return -1
@@ -838,7 +834,6 @@ class Parser {
     this.statusText = ''
     this.bytesRead = 0
     this.contentLength = ''
-    this.trailer = ''
     this.keepAlive = ''
 
     assert(this.headers.length % 2 === 0)
@@ -849,23 +844,6 @@ class Parser {
       return
     }
 
-    const trailers = trailer ? trailer.split(/,\s*/) : []
-    for (let i = 0; i < trailers.length; i++) {
-      const trailer = trailers[i]
-      let found = false
-      for (let n = 0; n < headers.length; n += 2) {
-        const key = headers[n]
-        if (key.length === trailer.length && key.toString().toLowerCase() === trailer.toLowerCase()) {
-          found = true
-          break
-        }
-      }
-      if (!found) {
-        util.destroy(socket, new TrailerMismatchError())
-        return -1
-      }
-    }
-
     /* istanbul ignore next: should be handled by llhttp? */
     if (request.method !== 'HEAD' && contentLength && bytesRead !== parseInt(contentLength, 10)) {
       util.destroy(socket, new ResponseContentLengthMismatchError())

package/lib/core/errors.js

@@ -116,16 +116,6 @@ class ResponseContentLengthMismatchError extends UndiciError {
   }
 }
 
-class TrailerMismatchError extends UndiciError {
-  constructor (message) {
-    super(message)
-    Error.captureStackTrace(this, TrailerMismatchError)
-    this.name = 'TrailerMismatchError'
-    this.message = message || 'Trailers does not match trailer header'
-    this.code = 'UND_ERR_TRAILER_MISMATCH'
-  }
-}
-
 class ClientDestroyedError extends UndiciError {
   constructor (message) {
     super(message)
@@ -196,7 +186,6 @@ module.exports = {
   BodyTimeoutError,
   RequestContentLengthMismatchError,
   ConnectTimeoutError,
-  TrailerMismatchError,
   InvalidArgumentError,
   InvalidReturnValueError,
   RequestAbortedError,

package/lib/core/request.js

@@ -11,6 +11,12 @@ const kHandler = Symbol('handler')
 
 const channels = {}
 
+let extractBody
+
+const nodeVersion = process.versions.node.split('.')
+const nodeMajor = Number(nodeVersion[0])
+const nodeMinor = Number(nodeVersion[1])
+
 try {
   const diagnosticsChannel = require('diagnostics_channel')
   channels.create = diagnosticsChannel.channel('undici:request:create')
@@ -79,7 +85,7 @@ class Request {
       this.body = body.byteLength ? body : null
     } else if (typeof body === 'string') {
       this.body = body.length ? Buffer.from(body) : null
-    } else if (util.isIterable(body) || util.isBlobLike(body)) {
+    } else if (util.isFormDataLike(body) || util.isIterable(body) || util.isBlobLike(body)) {
       this.body = body
     } else {
       throw new InvalidArgumentError('body must be a string, a Buffer, a Readable stream, an iterable, or an async iterable')
@@ -126,7 +132,22 @@ class Request {
       throw new InvalidArgumentError('headers must be an object or an array')
     }
 
-    if (util.isBlobLike(body) && this.contentType == null && body.type) {
+    if (util.isFormDataLike(this.body)) {
+      if (nodeMajor < 16 || (nodeMajor === 16 && nodeMinor < 5)) {
+        throw new InvalidArgumentError('Form-Data bodies are only supported in node v16.5 and newer.')
+      }
+
+      if (!extractBody) {
+        extractBody = require('../fetch/body.js').extractBody
+      }
+
+      const [bodyStream, contentType] = extractBody(body)
+      if (this.contentType == null) {
+        this.contentType = contentType
+        this.headers += `content-type: ${contentType}\r\n`
+      }
+      this.body = bodyStream.stream
+    } else if (util.isBlobLike(body) && this.contentType == null && body.type) {
       this.contentType = body.type
       this.headers += `content-type: ${body.type}\r\n`
     }

package/lib/core/util.js

@@ -324,6 +324,10 @@ function ReadableStreamFrom (iterable) {
   )
 }
 
+function isFormDataLike (chunk) {
+  return chunk && chunk.constructor && chunk.constructor.name === 'FormData'
+}
+
 const kEnumerableProperty = Object.create(null)
 kEnumerableProperty.enumerable = true
 
@@ -352,5 +356,6 @@ module.exports = {
   ReadableStreamFrom,
   isBuffer,
   validateHandler,
-  getSocketInfo
+  getSocketInfo,
+  isFormDataLike
 }

package/lib/fetch/body.js

@@ -71,7 +71,7 @@ function extractBody (object, keepalive = false) {
 
     // Set source to a copy of the bytes held by object.
     source = new Uint8Array(object)
-  } else if (object instanceof FormData) {
+  } else if (util.isFormDataLike(object)) {
     const boundary = '----formdata-undici-' + Math.random()
     const prefix = `--${boundary}\r\nContent-Disposition: form-data`
 
@@ -348,7 +348,7 @@ const properties = {
   bodyUsed: {
     enumerable: true,
     get () {
-      return this[kState].body && util.isDisturbed(this[kState].body.stream)
+      return !!this[kState].body && util.isDisturbed(this[kState].body.stream)
     }
   }
 }