undici 5.9.1 → 5.11.0

package/README.md

@@ -185,12 +185,12 @@ Help us improve the test coverage by following instructions at [nodejs/undici/#9
 Basic usage example:
 
 ```js
-import { fetch } from 'undici';
+import { fetch } from 'undici'
 
 
 const res = await fetch('https://example.com')
 const json = await res.json()
-console.log(json);
+console.log(json)
 ```
 
 You can pass an optional dispatcher to `fetch` as:
@@ -225,16 +225,16 @@ A body can be of the following types:
 In this implementation of fetch, ```request.body``` now accepts ```Async Iterables```. It is not present in the [Fetch Standard.](https://fetch.spec.whatwg.org)
 
 ```js
-import { fetch } from "undici";
+import { fetch } from 'undici'
 
 const data = {
   async *[Symbol.asyncIterator]() {
-    yield "hello";
-    yield "world";
+    yield 'hello'
+    yield 'world'
   },
-};
+}
 
-await fetch("https://example.com", { body: data, method: 'POST' });
+await fetch('https://example.com', { body: data, method: 'POST' })
 ```
 
 #### `response.body`
@@ -242,12 +242,12 @@ await fetch("https://example.com", { body: data, method: 'POST' });
 Nodejs has two kinds of streams: [web streams](https://nodejs.org/dist/latest-v16.x/docs/api/webstreams.html), which follow the API of the WHATWG web standard found in browsers, and an older Node-specific [streams API](https://nodejs.org/api/stream.html). `response.body` returns a readable web stream. If you would prefer to work with a Node stream you can convert a web stream using `.fromWeb()`.
 
 ```js
-import { fetch } from 'undici';
-import { Readable } from 'node:stream';
+import { fetch } from 'undici'
+import { Readable } from 'node:stream'
 
 const response = await fetch('https://example.com')
-const readableWebStream = response.body;
-const readableNodeStream = Readable.fromWeb(readableWebStream);
+const readableWebStream = response.body
+const readableNodeStream = Readable.fromWeb(readableWebStream)
 ```
 
 #### Specification Compliance
@@ -329,6 +329,28 @@ Gets the global dispatcher used by Common API Methods.
 
 Returns: `Dispatcher`
 
+### `undici.setGlobalOrigin(origin)`
+
+* origin `string | URL | undefined`
+
+Sets the global origin used in `fetch`.
+
+If `undefined` is passed, the global origin will be reset. This will cause `Response.redirect`, `new Request()`, and `fetch` to throw an error when a relative path is passed.
+
+```js
+setGlobalOrigin('http://localhost:3000')
+
+const response = await fetch('/api/ping')
+
+console.log(response.url) // http://localhost:3000/api/ping
+```
+
+### `undici.getGlobalOrigin()`
+
+Gets the global origin used in `fetch`.
+
+Returns: `URL`
+
 ### `UrlObject`
 
 * **port** `string | number` (optional)

package/docs/api/Agent.md

@@ -16,10 +16,11 @@ Returns: `Agent`
 
 ### Parameter: `AgentOptions`
 
-Extends: [`ClientOptions`](Pool.md#parameter-pooloptions)
+Extends: [`PoolOptions`](Pool.md#parameter-pooloptions)
 
 * **factory** `(origin: URL, opts: Object) => Dispatcher` - Default: `(origin, opts) => new Pool(origin, opts)`
 * **maxRedirections** `Integer` - Default: `0`. The number of HTTP redirection to follow unless otherwise specified in `DispatchOptions`.
+* **interceptors** `{ Agent: DispatchInterceptor[] }` - Default: `[RedirectInterceptor]` - A list of interceptors that are applied to the dispatch method. Additional logic can be applied (such as, but not limited to: 302 status code handling, authentication, cookies, compression and caching). Note that the behavior of interceptors is Experimental and might change at any given time.
 
 ## Instance Properties
 

package/docs/api/Client.md

@@ -26,6 +26,7 @@ Returns: `Client`
 * **pipelining** `number | null` (optional) - Default: `1` - The amount of concurrent requests to be sent over the single TCP/TLS connection according to [RFC7230](https://tools.ietf.org/html/rfc7230#section-6.3.2). Carefully consider your workload and environment before enabling concurrent requests as pipelining may reduce performance if used incorrectly. Pipelining is sensitive to network stack settings as well as head of line blocking caused by e.g. long running requests. Set to `0` to disable keep-alive connections.
 * **connect** `ConnectOptions | Function | null` (optional) - Default: `null`.
 * **strictContentLength** `Boolean` (optional) - Default: `true` - Whether to treat request content length mismatches as errors. If true, an error is thrown when the request content-length header doesn't match the length of the request body.
+* **interceptors** `{ Client: DispatchInterceptor[] }` - Default: `[RedirectInterceptor]` - A list of interceptors that are applied to the dispatch method. Additional logic can be applied (such as, but not limited to: 302 status code handling, authentication, cookies, compression and caching). Note that the behavior of interceptors is Experimental and might change at any given time.
 
 #### Parameter: `ConnectOptions`
 

package/docs/api/DispatchInterceptor.md

@@ -0,0 +1,60 @@
+#Interface: DispatchInterceptor
+
+Extends: `Function`
+
+A function that can be applied to the `Dispatcher.Dispatch` function before it is invoked with a dispatch request.
+
+This allows one to write logic to intercept both the outgoing request, and the incoming response.
+
+### Parameter: `Dispatcher.Dispatch`
+
+The base dispatch function you are decorating.
+
+### ReturnType: `Dispatcher.Dispatch`
+
+A dispatch function that has been altered to provide additional logic
+
+### Basic Example
+
+Here is an example of an interceptor being used to provide a JWT bearer token
+
+```js
+'use strict'
+
+const insertHeaderInterceptor = dispatch => {
+  return function InterceptedDispatch(opts, handler){
+    opts.headers.push('Authorization', 'Bearer [Some token]')
+    return dispatch(opts, handler)
+  }
+}
+
+const client = new Client('https://localhost:3000', {
+  interceptors: { Client: [insertHeaderInterceptor] }
+})
+
+```
+
+### Basic Example 2
+
+Here is a contrived example of an interceptor stripping the headers from a response.
+
+```js
+'use strict'
+
+const clearHeadersInterceptor = dispatch => {
+  const { DecoratorHandler } = require('undici')
+  class ResultInterceptor extends DecoratorHandler {
+    onHeaders (statusCode, headers, resume) {
+      return super.onHeaders(statusCode, [], resume)
+    }
+  }
+  return function InterceptedDispatch(opts, handler){
+    return dispatch(opts, new ResultInterceptor(handler))
+  }
+}
+
+const client = new Client('https://localhost:3000', {
+  interceptors: { Client: [clearHeadersInterceptor] }
+})
+
+```

package/docs/api/MockPool.md

@@ -54,7 +54,7 @@ Returns: `MockInterceptor` corresponding to the input options.
 ### Parameter: `MockPoolInterceptOptions`
 
 * **path** `string | RegExp | (path: string) => boolean` - a matcher for the HTTP request path.
-* **method** `string | RegExp | (method: string) => boolean` - a matcher for the HTTP request method.
+* **method** `string | RegExp | (method: string) => boolean` - (optional) - a matcher for the HTTP request method. Defaults to `GET`.
 * **body** `string | RegExp | (body: string) => boolean` - (optional) - a matcher for the HTTP request body.
 * **headers** `Record<string, string | RegExp | (body: string) => boolean`> - (optional) - a matcher for the HTTP request headers. To be intercepted, a request must match all defined headers. Extra headers not defined here may (or may not) be included in the request and do not affect the interception in any way.
 * **query** `Record<string, any> | null` - (optional) - a matcher for the HTTP request query string params.

package/docs/api/Pool.md

@@ -19,6 +19,7 @@ Extends: [`ClientOptions`](Client.md#parameter-clientoptions)
 
 * **factory** `(origin: URL, opts: Object) => Dispatcher` - Default: `(origin, opts) => new Client(origin, opts)`
 * **connections** `number | null` (optional) - Default: `null` - The number of `Client` instances to create. When set to `null`, the `Pool` instance will create an unlimited amount of `Client` instances.
+* **interceptors** `{ Pool: DispatchInterceptor[] } }` - Default: `{ Pool: [] }` - A list of interceptors that are applied to the dispatch method. Additional logic can be applied (such as, but not limited to: 302 status code handling, authentication, cookies, compression and caching).
 
 ## Instance Properties
 

package/index-fetch.js

@@ -1,11 +1,16 @@
 'use strict'
 
 const { getGlobalDispatcher } = require('./lib/global')
-const fetchImpl = require('./lib/fetch')
+const fetchImpl = require('./lib/fetch').fetch
 
 module.exports.fetch = async function fetch (resource) {
   const dispatcher = (arguments[1] && arguments[1].dispatcher) || getGlobalDispatcher()
-  return fetchImpl.apply(dispatcher, arguments)
+  try {
+    return await fetchImpl.apply(dispatcher, arguments)
+  } catch (err) {
+    Error.captureStackTrace(err, this)
+    throw err
+  }
 }
 module.exports.FormData = require('./lib/fetch/formdata').FormData
 module.exports.Headers = require('./lib/fetch/headers').Headers

package/index.d.ts

@@ -1,6 +1,9 @@
 import Dispatcher = require('./types/dispatcher')
 import { setGlobalDispatcher, getGlobalDispatcher } from './types/global-dispatcher'
+import { setGlobalOrigin, getGlobalOrigin } from './types/global-origin'
 import Pool = require('./types/pool')
+import { RedirectHandler, DecoratorHandler } from './types/handlers'
+
 import BalancedPool = require('./types/balanced-pool')
 import Client = require('./types/client')
 import buildConnector = require('./types/connector')
@@ -19,14 +22,15 @@ export * from './types/formdata'
 export * from './types/diagnostics-channel'
 export { Interceptable } from './types/mock-interceptor'
 
-export { Dispatcher, BalancedPool, Pool, Client, buildConnector, errors, Agent, request, stream, pipeline, connect, upgrade, setGlobalDispatcher, getGlobalDispatcher, MockClient, MockPool, MockAgent, mockErrors, ProxyAgent }
+export { Dispatcher, BalancedPool, Pool, Client, buildConnector, errors, Agent, request, stream, pipeline, connect, upgrade, setGlobalDispatcher, getGlobalDispatcher, setGlobalOrigin, getGlobalOrigin, MockClient, MockPool, MockAgent, mockErrors, ProxyAgent, RedirectHandler, DecoratorHandler }
 export default Undici
 
-declare function Undici(url: string, opts: Pool.Options): Pool
-
 declare namespace Undici {
   var Dispatcher: typeof import('./types/dispatcher')
   var Pool: typeof import('./types/pool');
+  var RedirectHandler: typeof import ('./types/handlers').RedirectHandler
+  var DecoratorHandler: typeof import ('./types/handlers').DecoratorHandler
+  var createRedirectInterceptor: typeof import ('./types/interceptors').createRedirectInterceptor
   var BalancedPool: typeof import('./types/balanced-pool');
   var Client: typeof import('./types/client');
   var buildConnector: typeof import('./types/connector');

package/index.js

@@ -16,6 +16,9 @@ const MockPool = require('./lib/mock/mock-pool')
 const mockErrors = require('./lib/mock/mock-errors')
 const ProxyAgent = require('./lib/proxy-agent')
 const { getGlobalDispatcher, setGlobalDispatcher } = require('./lib/global')
+const DecoratorHandler = require('./lib/handler/DecoratorHandler')
+const RedirectHandler = require('./lib/handler/RedirectHandler')
+const createRedirectInterceptor = require('./lib/interceptor/redirectInterceptor')
 
 const nodeVersion = process.versions.node.split('.')
 const nodeMajor = Number(nodeVersion[0])
@@ -30,6 +33,10 @@ module.exports.BalancedPool = BalancedPool
 module.exports.Agent = Agent
 module.exports.ProxyAgent = ProxyAgent
 
+module.exports.DecoratorHandler = DecoratorHandler
+module.exports.RedirectHandler = RedirectHandler
+module.exports.createRedirectInterceptor = createRedirectInterceptor
+
 module.exports.buildConnector = buildConnector
 module.exports.errors = errors
 
@@ -89,16 +96,26 @@ if (nodeMajor > 16 || (nodeMajor === 16 && nodeMinor >= 8)) {
   let fetchImpl = null
   module.exports.fetch = async function fetch (resource) {
     if (!fetchImpl) {
-      fetchImpl = require('./lib/fetch')
+      fetchImpl = require('./lib/fetch').fetch
     }
     const dispatcher = (arguments[1] && arguments[1].dispatcher) || getGlobalDispatcher()
-    return fetchImpl.apply(dispatcher, arguments)
+    try {
+      return await fetchImpl.apply(dispatcher, arguments)
+    } catch (err) {
+      Error.captureStackTrace(err, this)
+      throw err
+    }
   }
   module.exports.Headers = require('./lib/fetch/headers').Headers
   module.exports.Response = require('./lib/fetch/response').Response
   module.exports.Request = require('./lib/fetch/request').Request
   module.exports.FormData = require('./lib/fetch/formdata').FormData
   module.exports.File = require('./lib/fetch/file').File
+
+  const { setGlobalOrigin, getGlobalOrigin } = require('./lib/fetch/global')
+
+  module.exports.setGlobalOrigin = setGlobalOrigin
+  module.exports.getGlobalOrigin = getGlobalOrigin
 }
 
 module.exports.request = makeDispatcher(api.request)

package/lib/agent.js

@@ -1,12 +1,12 @@
 'use strict'
 
 const { InvalidArgumentError } = require('./core/errors')
-const { kClients, kRunning, kClose, kDestroy, kDispatch } = require('./core/symbols')
+const { kClients, kRunning, kClose, kDestroy, kDispatch, kInterceptors } = require('./core/symbols')
 const DispatcherBase = require('./dispatcher-base')
 const Pool = require('./pool')
 const Client = require('./client')
 const util = require('./core/util')
-const RedirectHandler = require('./handler/redirect')
+const createRedirectInterceptor = require('./interceptor/redirectInterceptor')
 const { WeakRef, FinalizationRegistry } = require('./compat/dispatcher-weakref')()
 
 const kOnConnect = Symbol('onConnect')
@@ -44,7 +44,14 @@ class Agent extends DispatcherBase {
       connect = { ...connect }
     }
 
+    this[kInterceptors] = options.interceptors && options.interceptors.Agent && Array.isArray(options.interceptors.Agent)
+      ? options.interceptors.Agent
+      : [createRedirectInterceptor({ maxRedirections })]
+
     this[kOptions] = { ...util.deepClone(options), connect }
+    this[kOptions].interceptors = options.interceptors
+      ? { ...options.interceptors }
+      : undefined
     this[kMaxRedirections] = maxRedirections
     this[kFactory] = factory
     this[kClients] = new Map()
@@ -108,12 +115,6 @@ class Agent extends DispatcherBase {
       this[kFinalizer].register(dispatcher, key)
     }
 
-    const { maxRedirections = this[kMaxRedirections] } = opts
-    if (maxRedirections != null && maxRedirections !== 0) {
-      opts = { ...opts, maxRedirections: 0 } // Stop sub dispatcher from also redirecting.
-      handler = new RedirectHandler(this, maxRedirections, opts, handler)
-    }
-
     return dispatcher.dispatch(opts, handler)
   }
 

package/lib/api/readable.js

@@ -93,7 +93,7 @@ module.exports = class BodyReadable extends Readable {
   }
 
   push (chunk) {
-    if (this[kConsume] && chunk !== null) {
+    if (this[kConsume] && chunk !== null && this.readableLength === 0) {
       consumePush(this[kConsume], chunk)
       return this[kReading] ? super.push(chunk) : true
     }

package/lib/balanced-pool.js

@@ -13,7 +13,7 @@ const {
   kGetDispatcher
 } = require('./pool-base')
 const Pool = require('./pool')
-const { kUrl } = require('./core/symbols')
+const { kUrl, kInterceptors } = require('./core/symbols')
 const { parseOrigin } = require('./core/util')
 const kFactory = Symbol('factory')
 
@@ -53,6 +53,9 @@ class BalancedPool extends PoolBase {
       throw new InvalidArgumentError('factory must be a function.')
     }
 
+    this[kInterceptors] = opts.interceptors && opts.interceptors.BalancedPool && Array.isArray(opts.interceptors.BalancedPool)
+      ? opts.interceptors.BalancedPool
+      : []
     this[kFactory] = factory
 
     for (const upstream of upstreams) {

package/lib/client.js

@@ -7,7 +7,6 @@ const net = require('net')
 const util = require('./core/util')
 const Request = require('./core/request')
 const DispatcherBase = require('./dispatcher-base')
-const RedirectHandler = require('./handler/redirect')
 const {
   RequestContentLengthMismatchError,
   ResponseContentLengthMismatchError,
@@ -60,7 +59,8 @@ const {
   kCounter,
   kClose,
   kDestroy,
-  kDispatch
+  kDispatch,
+  kInterceptors
 } = require('./core/symbols')
 
 const kClosedResolve = Symbol('kClosedResolve')
@@ -82,6 +82,7 @@ try {
 
 class Client extends DispatcherBase {
   constructor (url, {
+    interceptors,
     maxHeaderSize,
     headersTimeout,
     socketTimeout,
@@ -179,6 +180,9 @@ class Client extends DispatcherBase {
       })
     }
 
+    this[kInterceptors] = interceptors && interceptors.Client && Array.isArray(interceptors.Client)
+      ? interceptors.Client
+      : [createRedirectInterceptor({ maxRedirections })]
     this[kUrl] = util.parseOrigin(url)
     this[kConnector] = connect
     this[kSocket] = null
@@ -254,11 +258,6 @@ class Client extends DispatcherBase {
   }
 
   [kDispatch] (opts, handler) {
-    const { maxRedirections = this[kMaxRedirections] } = opts
-    if (maxRedirections) {
-      handler = new RedirectHandler(this, maxRedirections, opts, handler)
-    }
-
     const origin = opts.origin || this[kUrl].origin
 
     const request = new Request(origin, opts, handler)
@@ -319,6 +318,7 @@ class Client extends DispatcherBase {
 }
 
 const constants = require('./llhttp/constants')
+const createRedirectInterceptor = require('./interceptor/redirectInterceptor')
 const EMPTY_BUF = Buffer.alloc(0)
 
 async function lazyllhttp () {

package/lib/core/symbols.js

@@ -48,5 +48,6 @@ module.exports = {
   kMaxRedirections: Symbol('maxRedirections'),
   kMaxRequests: Symbol('maxRequestsPerClient'),
   kProxy: Symbol('proxy agent options'),
-  kCounter: Symbol('socket request counter')
+  kCounter: Symbol('socket request counter'),
+  kInterceptors: Symbol('dispatch interceptors')
 }

package/lib/core/util.js

@@ -8,6 +8,7 @@ const net = require('net')
 const { InvalidArgumentError } = require('./errors')
 const { Blob } = require('buffer')
 const nodeUtil = require('util')
+const { stringify } = require('querystring')
 
 function nop () {}
 
@@ -26,46 +27,15 @@ function isBlobLike (object) {
   )
 }
 
-function isObject (val) {
-  return val !== null && typeof val === 'object'
-}
-
-// this escapes all non-uri friendly characters
-function encode (val) {
-  return encodeURIComponent(val)
-}
-
-// based on https://github.com/axios/axios/blob/63e559fa609c40a0a460ae5d5a18c3470ffc6c9e/lib/helpers/buildURL.js (MIT license)
 function buildURL (url, queryParams) {
   if (url.includes('?') || url.includes('#')) {
     throw new Error('Query params cannot be passed when url already contains "?" or "#".')
   }
-  if (!isObject(queryParams)) {
-    throw new Error('Query params must be an object')
-  }
-
-  const parts = []
-  for (let [key, val] of Object.entries(queryParams)) {
-    if (val === null || typeof val === 'undefined') {
-      continue
-    }
-
-    if (!Array.isArray(val)) {
-      val = [val]
-    }
-
-    for (const v of val) {
-      if (isObject(v)) {
-        throw new Error('Passing object as a query param is not supported, please serialize to string up-front')
-      }
-      parts.push(encode(key) + '=' + encode(v))
-    }
-  }
 
-  const serializedParams = parts.join('&')
+  const stringified = stringify(queryParams)
 
-  if (serializedParams) {
-    url += '?' + serializedParams
+  if (stringified) {
+    url += '?' + stringified
   }
 
   return url

package/lib/dispatcher-base.js

@@ -6,12 +6,13 @@ const {
   ClientClosedError,
   InvalidArgumentError
 } = require('./core/errors')
-const { kDestroy, kClose, kDispatch } = require('./core/symbols')
+const { kDestroy, kClose, kDispatch, kInterceptors } = require('./core/symbols')
 
 const kDestroyed = Symbol('destroyed')
 const kClosed = Symbol('closed')
 const kOnDestroyed = Symbol('onDestroyed')
 const kOnClosed = Symbol('onClosed')
+const kInterceptedDispatch = Symbol('Intercepted Dispatch')
 
 class DispatcherBase extends Dispatcher {
   constructor () {
@@ -31,6 +32,23 @@ class DispatcherBase extends Dispatcher {
     return this[kClosed]
   }
 
+  get interceptors () {
+    return this[kInterceptors]
+  }
+
+  set interceptors (newInterceptors) {
+    if (newInterceptors) {
+      for (let i = newInterceptors.length - 1; i >= 0; i--) {
+        const interceptor = this[kInterceptors][i]
+        if (typeof interceptor !== 'function') {
+          throw new InvalidArgumentError('interceptor must be an function')
+        }
+      }
+    }
+
+    this[kInterceptors] = newInterceptors
+  }
+
   close (callback) {
     if (callback === undefined) {
       return new Promise((resolve, reject) => {
@@ -125,6 +143,20 @@ class DispatcherBase extends Dispatcher {
     })
   }
 
+  [kInterceptedDispatch] (opts, handler) {
+    if (!this[kInterceptors] || this[kInterceptors].length === 0) {
+      this[kInterceptedDispatch] = this[kDispatch]
+      return this[kDispatch](opts, handler)
+    }
+
+    let dispatch = this[kDispatch].bind(this)
+    for (let i = this[kInterceptors].length - 1; i >= 0; i--) {
+      dispatch = this[kInterceptors][i](dispatch)
+    }
+    this[kInterceptedDispatch] = dispatch
+    return dispatch(opts, handler)
+  }
+
   dispatch (opts, handler) {
     if (!handler || typeof handler !== 'object') {
       throw new InvalidArgumentError('handler must be an object')
@@ -143,7 +175,7 @@ class DispatcherBase extends Dispatcher {
         throw new ClientClosedError()
       }
 
-      return this[kDispatch](opts, handler)
+      return this[kInterceptedDispatch](opts, handler)
     } catch (err) {
       if (typeof handler.onError !== 'function') {
         throw new InvalidArgumentError('invalid onError method')