tangerine 1.2.1 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +224 -218
- package/index.js +90 -65
- package/package.json +7 -4
package/README.md
CHANGED
|
@@ -11,11 +11,11 @@
|
|
|
11
11
|
</div>
|
|
12
12
|
<br />
|
|
13
13
|
<div align="center">
|
|
14
|
-
🍊 <a href="https://github.com/forwardemail/tangerine" target="_blank">Tangerine</a> is the best <a href="https://nodejs.org" target="_blank">Node.js</a> drop-in replacement for <a href="https://nodejs.org/api/dns.html#resolveroptions" target="_blank">dns.promises.Resolver</a> using <a href="https://en.wikipedia.org/wiki/DNS_over_HTTPS" target="_blank">DNS over HTTPS</a> ("DoH") via <a href="https://github.com/nodejs/undici" target="_blank">undici</a> with built-in retries, timeouts, smart server rotation, <a href="https://developer.mozilla.org/en-US/docs/Web/API/AbortController" target="_blank">AbortControllers</a>, and caching support for multiple backends
|
|
14
|
+
🍊 <a href="https://github.com/forwardemail/tangerine" target="_blank">Tangerine</a> is the best <a href="https://nodejs.org" target="_blank">Node.js</a> drop-in replacement for <a href="https://nodejs.org/api/dns.html#resolveroptions" target="_blank">dns.promises.Resolver</a> using <a href="https://en.wikipedia.org/wiki/DNS_over_HTTPS" target="_blank">DNS over HTTPS</a> ("DoH") via <a href="https://github.com/nodejs/undici" target="_blank">undici</a> with built-in retries, timeouts, smart server rotation, <a href="https://developer.mozilla.org/en-US/docs/Web/API/AbortController" target="_blank">AbortControllers</a>, and caching support for multiple backends (with TTL support).
|
|
15
15
|
</div>
|
|
16
16
|
<hr />
|
|
17
17
|
<div align="center">
|
|
18
|
-
⚡ <
|
|
18
|
+
⚡ <a href="#tangerine-benchmarks"><i><u><strong>AS FAST AS</strong></u></i></a> native <a href="https://nodejs.org/api/dns.html" target="_blank">Node.js <code>dns</code></a>! 🚀 • Supports Node v16+ with ESM/CJS • Made for <a href="https://forwardemail.net" target="_blank"><strong>Forward Email</strong></a>.
|
|
19
19
|
</div>
|
|
20
20
|
<hr />
|
|
21
21
|
|
|
@@ -55,6 +55,7 @@
|
|
|
55
55
|
* [`tangerine.setDefaultResultOrder(order)`](#tangerinesetdefaultresultorderorder)
|
|
56
56
|
* [`tangerine.setServers(servers)`](#tangerinesetserversservers)
|
|
57
57
|
* [Options](#options)
|
|
58
|
+
* [Cache](#cache)
|
|
58
59
|
* [Debugging](#debugging)
|
|
59
60
|
* [Benchmarks](#benchmarks)
|
|
60
61
|
* [Tangerine Benchmarks](#tangerine-benchmarks)
|
|
@@ -96,7 +97,7 @@ Our team at [Forward Email](https://forwardemail.net) (100% open-source and priv
|
|
|
96
97
|
* Once popular packages such as [native-dns](https://github.com/tjfontaine/node-dns/issues/111) and [dnscached](https://github.com/yahoo/dnscache/issues/28) are archived or deprecated.
|
|
97
98
|
* [Other packages](https://www.npmjs.com/search?q=dns%20cache) only provide `lookup` functions, have a limited sub-set of methods such as [@zeit/dns-cached-resolver](https://github.com/vercel/dns-cached-resolve), or are unmaintained.
|
|
98
99
|
* Act as a 1:1 drop-in replacement for `dns.promises.Resolver` with DNS over HTTPS ("DoH").
|
|
99
|
-
* Support caching for multiple backends
|
|
100
|
+
* Support caching for multiple backends (with TTL support), retries, smart server rotation, and [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) usage.
|
|
100
101
|
* Provide out of the box support for both ECMAScript modules (ESM) **and** CommonJS (CJS) (see discussions [for](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c) and [against](https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7dfcad3)).
|
|
101
102
|
* The native Node.js `dns` module does not support caching out of the box – which is a [highly requested feature](https://github.com/nodejs/node/issues/5893) (but belongs in userland).
|
|
102
103
|
* Writing tests against DNS-related infrastructure requires either hacky DNS mocking or a DNS server (manipulating cache is much easier).
|
|
@@ -211,8 +212,7 @@ tangerine.resolve('forwardemail.net').then(console.log);
|
|
|
211
212
|
retry: {
|
|
212
213
|
limit: 0
|
|
213
214
|
}
|
|
214
|
-
}
|
|
215
|
-
requestTimeout: (ms) => ({ timeout: { request: ms } })
|
|
215
|
+
}
|
|
216
216
|
},
|
|
217
217
|
got
|
|
218
218
|
);
|
|
@@ -223,8 +223,6 @@ tangerine.resolve('forwardemail.net').then(console.log);
|
|
|
223
223
|
* The `body` property returned should be either a `Buffer` or `Stream`.
|
|
224
224
|
|
|
225
225
|
* Specify default request options based off the library under `requestOptions` below
|
|
226
|
-
|
|
227
|
-
* See `requestTimeout` function below, as it is required to be set properly if you are using a custom HTTP library function.
|
|
228
226
|
* Instance methods of [dns.promises.Resolver](https://nodejs.org/api/dns.html) are mirrored to :tangerine: Tangerine.
|
|
229
227
|
* Resolver methods accept an optional `abortController` argument, which is an instance of [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController). Note that :tangerine: Tangerine manages `AbortController` usage internally – so you most likely won't need to pass your own (see [index.js](https://github.com/forwardemail/tangerine/blob/main/index.js) for more insight).
|
|
230
228
|
* See the complete list of [Options](#options) below.
|
|
@@ -279,28 +277,86 @@ Tangerine supports a new `ecsSubnet` property in the `options` Object argument.
|
|
|
279
277
|
|
|
280
278
|
Similar to the `options` argument from `new dns.promises.Resolver(options)` invocation – :tangerine: Tangerine also has its own options with default `dns` behavior mirrored. See [index.js](https://github.com/forwardemail/tangerine/blob/main/index.js) for more insight into how these options work.
|
|
281
279
|
|
|
282
|
-
| Property | Type
|
|
283
|
-
| ------------------------- |
|
|
284
|
-
| `timeout` | `Number`
|
|
285
|
-
| `tries` | `Number`
|
|
286
|
-
| `servers` | `Set` or `Array`
|
|
287
|
-
| `requestOptions` | `Object`
|
|
288
|
-
| `requestOptions.method` | `String`
|
|
289
|
-
| `requestOptions.headers` | `Object`
|
|
290
|
-
| `
|
|
291
|
-
| `
|
|
292
|
-
| `
|
|
293
|
-
| `
|
|
294
|
-
| `
|
|
295
|
-
| `
|
|
296
|
-
| `
|
|
297
|
-
| `
|
|
298
|
-
| `
|
|
299
|
-
| `
|
|
300
|
-
| `
|
|
301
|
-
| `
|
|
302
|
-
| `
|
|
303
|
-
| `
|
|
280
|
+
| Property | Type | Default Value | Description |
|
|
281
|
+
| ------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
282
|
+
| `timeout` | `Number` | `5000` | Number of milliseconds for requests to timeout. |
|
|
283
|
+
| `tries` | `Number` | `4` | Number of tries per `server` in `servers` to attempt. |
|
|
284
|
+
| `servers` | `Set` or `Array` | `new Set(['1.1.1.1', '1.0.0.1'])` | A Set or Array of [RFC 5952](https://tools.ietf.org/html/rfc5952#section-6) formatted addresses for DNS queries (matches default Node.js dns module behavior). Duplicates will be removed as this is converted to a `Set` internally. Defaults to Cloudflare's of `1.1.1.1` and `1.0.0.1`. If an `Array` is passed, then it will be converted to a `Set`. |
|
|
285
|
+
| `requestOptions` | `Object` | Defaults to an Object with `requestOptions.method` and `requestOptions.headers` properties and values below | Default options to pass to [undici](https://github.com/nodejs/undici) (or your custom HTTP library function passed as `request`). |
|
|
286
|
+
| `requestOptions.method` | `String` | Defaults to `"GET"` (must be either `"GET"` or `"POST"`, case-insensitive depending on library you use). | Default HTTP method to use for DNS over HTTP ("DoH") requests. |
|
|
287
|
+
| `requestOptions.headers` | `Object` | Defaults to `{ 'content-type': 'application/dns-message', 'user-agent': pkg.name + "/" + pkg.version, accept: 'application/dns-message' }`. | Default HTTP headers to use for DNS over HTTP ("DoH") requests. |
|
|
288
|
+
| `protocol` | `String` | Defaults to `"https"`. | Default HTTP protocol to use for DNS over HTTPS ("DoH") requests. |
|
|
289
|
+
| `dnsOrder` | `String` | Defaults to `"verbatim"` for Node.js v17.0.0+ and `"ipv4first"` for older versions. | Sets the default result order of `lookup` invocations (see [dns.setDefaultResultOrder](https://nodejs.org/api/dns.html#dnssetdefaultresultorderorder) for more insight). |
|
|
290
|
+
| `logger` | `Object` | `false` | This is the default logger. We recommend using [Cabin](https://github.com/cabinjs) instead of using `console` as your default logger. Set this value to `false` to disable logging entirely (uses noop function). |
|
|
291
|
+
| `id` | `Number` or `Function` | `0` | Default `id` to be passed for DNS packet creation. This could alternatively be a synchronous or asynchronous function that returns a `Number` (e.g. `id: () => Tangerine.getRandomInt(1, 65534)`). |
|
|
292
|
+
| `concurrency` | `Number` | `os.cpus().length` | Default concurrency to use for `resolveAny` lookup via [p-map](https://github.com/sindresorhus/p-map). The default value is the number of CPU's available to the system using the Node.js `os` module [os.cpus()](https://nodejs.org/api/os.html#oscpus) method. |
|
|
293
|
+
| `ipv4` | `String` | `"0.0.0.0"` | Default IPv4 address to use for HTTP agent `localAddress` if DNS `server` was an IPv4 address. |
|
|
294
|
+
| `ipv6` | `String` | `"::0"` | Default IPv6 address to use for HTTP agent `localAddress` if DNS `server` was an IPv6 address. |
|
|
295
|
+
| `ipv4Port` | `Number` | `undefined` | Default port to use for HTTP agent `localPort` if DNS `server` was an IPv4 address. |
|
|
296
|
+
| `ipv6Port` | `Number` | `undefined` | Default port to use for HTTP agent `localPort` if DNS `server` was an IPv6 address. |
|
|
297
|
+
| `cache` | `Map`, `Boolean`, or custom cache implementation with `get` and `set` methods | `new Map()` | Set this to `false` in order to disable caching. By default or if you pass `cache: true`, it will use a new `Map` instance for caching. See [Cache](#cache) documentation and the options `defaultTTLSeconds`, `maxTTLSeconds`, and `setCacheArgs` below. |
|
|
298
|
+
| `defaultTTLSeconds` | `Number` (seconds) | `300` | The default number of seconds to use for storing results in cache (defaults to [Cloudflare's recommendation](https://developers.cloudflare.com/dns/manage-dns-records/reference/ttl/) of 300 seconds – 5 minutes). |
|
|
299
|
+
| `maxTTLSeconds` | `Number` (seconds) | `86400` | The maximum number of seconds to use for storing results in cache (defaults to [Cloudflare's recommendation](https://developers.cloudflare.com/dns/manage-dns-records/reference/ttl/) of 86,400 seconds – 24 hours – 1 day). |
|
|
300
|
+
| `setCacheArgs` | `Function` | `(key, result) => []` | This is a helper function used for cache store providers such as [ioredis](https://github.com/luin/ioredis) or [lru-cache](https://github.com/isaacs/node-lru-cache) which support more than two arguments to `cache.set()` function. See [Cache](#cache) documentation below for more insight and examples into how this works. You may want to set this to something such as `(key, result) => [ 'PX', Math.round(result.ttl * 1000) ]` if you are using `ioredis`. |
|
|
301
|
+
| `returnHTTPErrors` | `Boolean` | `false` | Whether to return HTTP errors instead of mapping them to corresponding DNS errors. |
|
|
302
|
+
| `smartRotate` | `Boolean` | `true` | Whether to do smart server rotation if servers fail. |
|
|
303
|
+
| `defaultHTTPErrorMessage` | `String` | `"Unsuccessful HTTP response"` | Default fallback message if `statusCode` returned from HTTP request was not found in [http.STATUS_CODES](https://nodejs.org/api/http.html#httpstatus_codes). |
|
|
304
|
+
|
|
305
|
+
|
|
306
|
+
## Cache
|
|
307
|
+
|
|
308
|
+
:tangerine: Tangerine supports custom cache implementations, such as with [ioredis](https://github.com/luin/ioredis) or any other cache store that has a Map-like implementation with `set(key, value)` and `get(key)` methods. If your cache implementation allows a third argument to `set()`, such as `set(key, value, ttl)` or `set(key, value, { maxAge })`, then you must set the `setCacheArgs` option respectively (see below examples). A third argument with TTL argument support is optional as it is already built-in to :tangerine: Tangerine out of the box (cached results store their TTL and expiration time on the objects themselves – view source code for insight).
|
|
309
|
+
|
|
310
|
+
```sh
|
|
311
|
+
npm install tangerine undici ioredis
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
```js
|
|
315
|
+
// app.js
|
|
316
|
+
|
|
317
|
+
const Redis = require('ioredis');
|
|
318
|
+
const Tangerine = require('tangerine');
|
|
319
|
+
|
|
320
|
+
// <https://github.com/luin/ioredis/issues/1179>
|
|
321
|
+
Redis.Command.setArgumentTransformer('set', (args) => {
|
|
322
|
+
if (typeof args[1] === 'object') args[1] = JSON.stringify(args[1]);
|
|
323
|
+
return args;
|
|
324
|
+
});
|
|
325
|
+
|
|
326
|
+
Redis.Command.setReplyTransformer('get', (value) => {
|
|
327
|
+
if (value && typeof value === 'string') {
|
|
328
|
+
try {
|
|
329
|
+
value = JSON.parse(value);
|
|
330
|
+
} catch {}
|
|
331
|
+
}
|
|
332
|
+
|
|
333
|
+
return value;
|
|
334
|
+
});
|
|
335
|
+
|
|
336
|
+
const cache = new Redis();
|
|
337
|
+
const tangerine = new Tangerine({
|
|
338
|
+
cache,
|
|
339
|
+
setCacheArgs(key, result) {
|
|
340
|
+
return ['PX', Math.round(result.ttl * 1000)];
|
|
341
|
+
}
|
|
342
|
+
});
|
|
343
|
+
|
|
344
|
+
(async () => {
|
|
345
|
+
console.time('without cache');
|
|
346
|
+
await tangerine.resolve('forwardemail.net'); // <-- cached
|
|
347
|
+
console.timeEnd('without cache');
|
|
348
|
+
|
|
349
|
+
console.time('with cache');
|
|
350
|
+
await tangerine.resolve('forwardemail.net'); // <-- uses cached value
|
|
351
|
+
console.timeEnd('with cache');
|
|
352
|
+
})();
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
```sh
|
|
356
|
+
❯ node app
|
|
357
|
+
without cache: 98.25ms
|
|
358
|
+
with cache: 0.091ms
|
|
359
|
+
```
|
|
304
360
|
|
|
305
361
|
|
|
306
362
|
## Debugging
|
|
@@ -337,107 +393,83 @@ BENCHMARK_PROTOCOL="http" BENCHMARK_HOST="127.0.0.1" BENCHMARK_PORT="4000" BENCH
|
|
|
337
393
|
|
|
338
394
|
### Tangerine Benchmarks
|
|
339
395
|
|
|
340
|
-
We have written extensive benchmarks to show that :tangerine: Tangerine
|
|
396
|
+
We have written extensive benchmarks to show that :tangerine: Tangerine is as fast as the native Node.js DNS module (with the exception of the `lookup` command). Note that performance is opinionated – since rate limiting plays a factor dependent on the DNS servers you are using and since caching is most likely going to takeover.
|
|
341
397
|
|
|
342
|
-
The
|
|
398
|
+
The latest benchmark results are viewable on GitHub under this repository's [GitHub CI actions logs](https://github.com/forwardemail/tangerine/actions?query=event%3Apush):
|
|
343
399
|
|
|
344
|
-
> [Node 16 on ubuntu-latest](https://github.com/forwardemail/tangerine/actions/runs/
|
|
345
|
-
|
|
346
|
-
```sh
|
|
347
|
-
> node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse && node benchmarks/http
|
|
400
|
+
> [Node 16 on ubuntu-latest](https://github.com/forwardemail/tangerine/actions/runs/4297805550/jobs/7491228635#step:6:1)
|
|
348
401
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
402
|
+
```diff
|
|
403
|
+
> node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse
|
|
404
|
+
|
|
405
|
+
Started: lookup
|
|
406
|
+
tangerine.lookup POST with caching using Cloudflare x 735 ops/sec ±195.35% (88 runs sampled)
|
|
407
|
+
tangerine.lookup POST without caching using Cloudflare x 142 ops/sec ±0.58% (84 runs sampled)
|
|
408
|
+
tangerine.lookup GET with caching using Cloudflare x 222,397 ops/sec ±0.52% (88 runs sampled)
|
|
409
|
+
+tangerine.lookup GET without caching using Cloudflare x 142 ops/sec ±0.46% (83 runs sampled)
|
|
410
|
+
dns.promises.lookup with caching using Cloudflare x 6,169,417 ops/sec ±1.67% (84 runs sampled)
|
|
411
|
+
-dns.promises.lookup without caching using Cloudflare x 4,186 ops/sec ±0.58% (89 runs sampled)
|
|
355
412
|
Fastest without caching is: dns.promises.lookup without caching using Cloudflare
|
|
356
|
-
tangerine.resolve POST with caching using Cloudflare x 734 ops/sec ±190.07% (84 runs sampled)
|
|
357
|
-
tangerine.resolve POST without caching using Cloudflare x 234 ops/sec ±3.75% (82 runs sampled)
|
|
358
|
-
tangerine.resolve GET with caching using Cloudflare x 24,040 ops/sec ±1.93% (83 runs sampled)
|
|
359
|
-
tangerine.resolve GET without caching using Cloudflare x 215 ops/sec ±16.62% (75 runs sampled)
|
|
360
|
-
tangerine.resolve POST with caching using Google x 23,937 ops/sec ±2.04% (81 runs sampled)
|
|
361
|
-
tangerine.resolve POST without caching using Google x 213 ops/sec ±9.51% (71 runs sampled)
|
|
362
|
-
tangerine.resolve GET with caching using Google x 24,272 ops/sec ±1.74% (83 runs sampled)
|
|
363
|
-
tangerine.resolve GET without caching using Google x 257 ops/sec ±4.02% (80 runs sampled)
|
|
364
|
-
resolver.resolve with caching using Cloudflare x 158,842 ops/sec ±2.57% (84 runs sampled)
|
|
365
|
-
resolver.resolve without caching using Cloudflare x 8.02 ops/sec ±191.78% (41 runs sampled)
|
|
366
|
-
Fastest without caching is: tangerine.resolve GET without caching using Google <--------
|
|
367
|
-
tangerine.reverse GET with caching x 694 ops/sec ±189.48% (76 runs sampled)
|
|
368
|
-
tangerine.reverse GET without caching x 123 ops/sec ±90.74% (81 runs sampled)
|
|
369
|
-
resolver.reverse x 0.24 ops/sec ±86.12% (10 runs sampled)
|
|
370
|
-
dns.promises.reverse x 0.70 ops/sec ±164.50% (42 runs sampled)
|
|
371
|
-
Fastest without caching is: tangerine.reverse GET without caching <--------
|
|
372
|
-
http.request POST request x 384 ops/sec ±1.08% (84 runs sampled)
|
|
373
|
-
http.request GET request x 398 ops/sec ±0.83% (83 runs sampled)
|
|
374
|
-
undici GET request x 206 ops/sec ±5.59% (58 runs sampled)
|
|
375
|
-
undici POST request x 211 ops/sec ±4.44% (74 runs sampled)
|
|
376
|
-
axios GET request x 343 ops/sec ±1.97% (82 runs sampled)
|
|
377
|
-
axios POST request x 350 ops/sec ±3.35% (82 runs sampled)
|
|
378
|
-
got GET request x 325 ops/sec ±1.61% (81 runs sampled)
|
|
379
|
-
got POST request x 341 ops/sec ±2.86% (84 runs sampled)
|
|
380
|
-
fetch GET request x 657 ops/sec ±1.42% (82 runs sampled)
|
|
381
|
-
fetch POST request x 680 ops/sec ±1.21% (84 runs sampled)
|
|
382
|
-
request GET request x 370 ops/sec ±1.08% (85 runs sampled)
|
|
383
|
-
request POST request x 370 ops/sec ±0.88% (84 runs sampled)
|
|
384
|
-
superagent GET request x 380 ops/sec ±1.14% (83 runs sampled)
|
|
385
|
-
superagent POST request x 386 ops/sec ±1.04% (83 runs sampled)
|
|
386
|
-
phin GET request x 396 ops/sec ±0.86% (84 runs sampled)
|
|
387
|
-
phin POST request x 398 ops/sec ±0.83% (85 runs sampled)
|
|
388
|
-
Fastest is fetch POST request
|
|
389
|
-
```
|
|
390
413
|
|
|
391
|
-
|
|
414
|
+
Started: resolve
|
|
415
|
+
tangerine.resolve POST with caching using Cloudflare x 951 ops/sec ±195.84% (87 runs sampled)
|
|
416
|
+
tangerine.resolve POST without caching using Cloudflare x 135 ops/sec ±1.27% (79 runs sampled)
|
|
417
|
+
tangerine.resolve GET with caching using Cloudflare x 1,134,724 ops/sec ±0.27% (87 runs sampled)
|
|
418
|
+
+tangerine.resolve GET without caching using Cloudflare x 135 ops/sec ±1.34% (81 runs sampled)
|
|
419
|
+
tangerine.resolve POST with caching using Google x 1,103,189 ops/sec ±0.44% (86 runs sampled)
|
|
420
|
+
tangerine.resolve POST without caching using Google x 55.76 ops/sec ±3.57% (80 runs sampled)
|
|
421
|
+
tangerine.resolve GET with caching using Google x 1,140,499 ops/sec ±0.32% (87 runs sampled)
|
|
422
|
+
tangerine.resolve GET without caching using Google x 70.51 ops/sec ±0.93% (84 runs sampled)
|
|
423
|
+
resolver.resolve with caching using Cloudflare x 4,790,171 ops/sec ±0.43% (87 runs sampled)
|
|
424
|
+
-resolver.resolve without caching using Cloudflare x 158 ops/sec ±1.26% (83 runs sampled)
|
|
425
|
+
Fastest without caching is: resolver.resolve without caching using Cloudflare
|
|
426
|
+
|
|
427
|
+
Started: reverse
|
|
428
|
+
tangerine.reverse GET with caching x 771 ops/sec ±195.37% (85 runs sampled)
|
|
429
|
+
+tangerine.reverse GET without caching x 135 ops/sec ±0.74% (81 runs sampled)
|
|
430
|
+
resolver.reverse with caching x 5,353,130 ops/sec ±0.36% (89 runs sampled)
|
|
431
|
+
-resolver.reverse without caching x 1.90 ops/sec ±210.52% (16 runs sampled)
|
|
432
|
+
dns.promises.reverse with caching x 5,123,900 ops/sec ±0.96% (85 runs sampled)
|
|
433
|
+
-dns.promises.reverse without caching x 0.29 ops/sec ±171.85% (18 runs sampled)
|
|
434
|
+
+Fastest without caching is: tangerine.reverse GET without caching
|
|
435
|
+
```
|
|
392
436
|
|
|
393
|
-
> [Node 18 on ubuntu latest](https://github.com/forwardemail/tangerine/actions/runs/
|
|
437
|
+
> [Node 18 on ubuntu latest](https://github.com/forwardemail/tangerine/actions/runs/4297805550/jobs/7491228742#step:6:1)
|
|
394
438
|
|
|
395
|
-
```
|
|
439
|
+
```diff
|
|
396
440
|
> node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse && node benchmarks/http
|
|
397
441
|
|
|
398
|
-
|
|
399
|
-
tangerine.lookup POST
|
|
400
|
-
tangerine.lookup
|
|
401
|
-
tangerine.lookup GET
|
|
402
|
-
|
|
403
|
-
dns.promises.lookup
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
tangerine.resolve POST without caching using Cloudflare x 59.66 ops/sec ±5.69% (74 runs sampled)
|
|
407
|
-
tangerine.resolve GET with caching using Cloudflare x 33,813 ops/sec ±0.33% (90 runs sampled)
|
|
408
|
-
tangerine.resolve GET without caching using Cloudflare x 60.16 ops/sec ±4.03% (73 runs sampled)
|
|
409
|
-
tangerine.resolve POST with caching using Google x 1,184 ops/sec ±189.17% (90 runs sampled)
|
|
410
|
-
tangerine.resolve POST without caching using Google x 41.23 ops/sec ±7.30% (70 runs sampled)
|
|
411
|
-
tangerine.resolve GET with caching using Google x 33,811 ops/sec ±0.56% (91 runs sampled)
|
|
412
|
-
tangerine.resolve GET without caching using Google x 54.34 ops/sec ±5.71% (69 runs sampled)
|
|
413
|
-
resolver.resolve with caching using Cloudflare x 202,804 ops/sec ±0.39% (88 runs sampled)
|
|
414
|
-
resolver.resolve without caching using Cloudflare x 61.93 ops/sec ±5.76% (76 runs sampled)
|
|
415
|
-
Fastest without caching is: resolver.resolve without caching using Cloudflare <--------
|
|
416
|
-
tangerine.reverse GET with caching x 594 ops/sec ±192.60% (86 runs sampled)
|
|
417
|
-
tangerine.reverse GET without caching x 60.73 ops/sec ±3.06% (74 runs sampled)
|
|
418
|
-
resolver.reverse x 66.00 ops/sec ±0.91% (78 runs sampled)
|
|
419
|
-
dns.promises.reverse x 1.84 ops/sec ±190.54% (71 runs sampled)
|
|
420
|
-
Fastest without caching is: tangerine.reverse GET without caching <--------
|
|
421
|
-
http.request POST request x 438 ops/sec ±0.61% (86 runs sampled)
|
|
422
|
-
http.request GET request x 442 ops/sec ±0.64% (87 runs sampled)
|
|
423
|
-
undici GET request x 203 ops/sec ±3.67% (42 runs sampled)
|
|
424
|
-
undici POST request x 194 ops/sec ±3.77% (62 runs sampled)
|
|
425
|
-
axios GET request x 403 ops/sec ±1.67% (86 runs sampled)
|
|
426
|
-
axios POST request x 414 ops/sec ±0.65% (88 runs sampled)
|
|
427
|
-
got GET request x 391 ops/sec ±1.63% (85 runs sampled)
|
|
428
|
-
got POST request x 403 ops/sec ±0.90% (85 runs sampled)
|
|
429
|
-
fetch GET request x 794 ops/sec ±2.32% (84 runs sampled)
|
|
430
|
-
fetch POST request x 821 ops/sec ±0.89% (86 runs sampled)
|
|
431
|
-
request GET request x 423 ops/sec ±0.75% (86 runs sampled)
|
|
432
|
-
request POST request x 426 ops/sec ±0.78% (86 runs sampled)
|
|
433
|
-
superagent GET request x 435 ops/sec ±0.79% (87 runs sampled)
|
|
434
|
-
superagent POST request x 437 ops/sec ±0.82% (88 runs sampled)
|
|
435
|
-
phin GET request x 443 ops/sec ±0.64% (86 runs sampled)
|
|
436
|
-
phin POST request x 445 ops/sec ±0.60% (86 runs sampled)
|
|
437
|
-
Fastest is fetch POST request
|
|
438
|
-
```
|
|
442
|
+
Started: lookup
|
|
443
|
+
tangerine.lookup POST with caching using Cloudflare x 666 ops/sec ±195.48% (87 runs sampled)
|
|
444
|
+
tangerine.lookup POST without caching using Cloudflare x 90.81 ops/sec ±8.06% (89 runs sampled)
|
|
445
|
+
tangerine.lookup GET with caching using Cloudflare x 256,141 ops/sec ±1.72% (87 runs sampled)
|
|
446
|
+
+tangerine.lookup GET without caching using Cloudflare x 96.39 ops/sec ±0.31% (89 runs sampled)
|
|
447
|
+
dns.promises.lookup with caching using Cloudflare x 1,473 ops/sec ±195.95% (87 runs sampled)
|
|
448
|
+
-dns.promises.lookup without caching using Cloudflare x 4,191 ops/sec ±0.54% (85 runs sampled)
|
|
449
|
+
Fastest without caching is: dns.promises.lookup without caching using Cloudflare
|
|
439
450
|
|
|
440
|
-
|
|
451
|
+
Started: resolve
|
|
452
|
+
tangerine.resolve POST with caching using Cloudflare x 683 ops/sec ±195.88% (87 runs sampled)
|
|
453
|
+
tangerine.resolve POST without caching using Cloudflare x 93.37 ops/sec ±0.48% (87 runs sampled)
|
|
454
|
+
tangerine.resolve GET with caching using Cloudflare x 1,146,727 ops/sec ±0.58% (88 runs sampled)
|
|
455
|
+
+tangerine.resolve GET without caching using Cloudflare x 93.33 ops/sec ±0.51% (87 runs sampled)
|
|
456
|
+
tangerine.resolve POST with caching using Google x 1,133,683 ops/sec ±2.74% (89 runs sampled)
|
|
457
|
+
tangerine.resolve POST without caching using Google x 83.91 ops/sec ±6.32% (76 runs sampled)
|
|
458
|
+
tangerine.resolve GET with caching using Google x 1,147,212 ops/sec ±0.32% (90 runs sampled)
|
|
459
|
+
tangerine.resolve GET without caching using Google x 79.73 ops/sec ±4.02% (77 runs sampled)
|
|
460
|
+
resolver.resolve with caching using Cloudflare x 5,318,406 ops/sec ±0.67% (86 runs sampled)
|
|
461
|
+
-resolver.resolve without caching using Cloudflare x 100 ops/sec ±1.55% (79 runs sampled)
|
|
462
|
+
Fastest without caching is: resolver.resolve without caching using Cloudflare
|
|
463
|
+
|
|
464
|
+
Started: reverse
|
|
465
|
+
tangerine.reverse GET with caching x 722 ops/sec ±195.42% (88 runs sampled)
|
|
466
|
+
+tangerine.reverse GET without caching x 93.19 ops/sec ±0.74% (87 runs sampled)
|
|
467
|
+
resolver.reverse with caching x 5,520,569 ops/sec ±0.59% (85 runs sampled)
|
|
468
|
+
-resolver.reverse without caching x 17.42 ops/sec ±162.63% (70 runs sampled)
|
|
469
|
+
dns.promises.reverse with caching x 5,164,258 ops/sec ±0.96% (86 runs sampled)
|
|
470
|
+
-dns.promises.reverse without caching x 0.20 ops/sec ±184.87% (25 runs sampled)
|
|
471
|
+
+Fastest without caching is: tangerine.reverse GET without caching
|
|
472
|
+
```
|
|
441
473
|
|
|
442
474
|
---
|
|
443
475
|
|
|
@@ -447,84 +479,84 @@ You can also [run the benchmarks yourself](#benchmarks).
|
|
|
447
479
|
|
|
448
480
|
Provided below are additional benchmark tests we have run:
|
|
449
481
|
|
|
450
|
-
> Node
|
|
482
|
+
> Node v18.14.2 on MacBook Air M1 16GB (without VPN):
|
|
451
483
|
|
|
452
|
-
```
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
❯ node benchmarks/resolve
|
|
457
|
-
tangerine POST with caching using Cloudflare x 1,044 ops/sec ±193.21% (90 runs sampled)
|
|
458
|
-
tangerine POST without caching using Cloudflare x 40.93 ops/sec ±53.83% (50 runs sampled)
|
|
459
|
-
tangerine GET with caching using Cloudflare x 73,896 ops/sec ±0.27% (90 runs sampled)
|
|
460
|
-
tangerine GET without caching using Cloudflare x 38.66 ops/sec ±21.98% (55 runs sampled)
|
|
461
|
-
tangerine POST with caching using Google x 992 ops/sec ±193.33% (87 runs sampled)
|
|
462
|
-
tangerine POST without caching using Google x 31.98 ops/sec ±21.35% (58 runs sampled)
|
|
463
|
-
tangerine GET with caching using Google x 74,410 ops/sec ±0.22% (91 runs sampled)
|
|
464
|
-
tangerine GET without caching using Google x 41.52 ops/sec ±18.91% (56 runs sampled)
|
|
465
|
-
dns.promises.resolve without caching using Cloudflare x 25.46 ops/sec ±100.19% (50 runs sampled)
|
|
466
|
-
dns.promises.resolve with caching using Cloudflare x 505,956 ops/sec ±2.34% (89 runs sampled)
|
|
467
|
-
Fastest without caching is: tangerine GET without caching using Google, tangerine GET without caching using Cloudflare
|
|
468
|
-
```
|
|
484
|
+
```diff
|
|
485
|
+
> node --version
|
|
486
|
+
v18.14.2
|
|
469
487
|
|
|
470
|
-
>
|
|
488
|
+
> node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse
|
|
471
489
|
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
tangerine POST
|
|
483
|
-
tangerine
|
|
484
|
-
tangerine GET
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
490
|
+
Started: lookup
|
|
491
|
+
tangerine.lookup POST with caching using Cloudflare x 1,035 ops/sec ±195.73% (91 runs sampled)
|
|
492
|
+
tangerine.lookup POST without caching using Cloudflare x 52.76 ops/sec ±51.29% (53 runs sampled)
|
|
493
|
+
tangerine.lookup GET with caching using Cloudflare x 694,910 ops/sec ±1.54% (87 runs sampled)
|
|
494
|
+
+tangerine.lookup GET without caching using Cloudflare x 40.18 ops/sec ±60.19% (49 runs sampled)
|
|
495
|
+
dns.promises.lookup with caching using Cloudflare x 12,645,103 ops/sec ±0.26% (90 runs sampled)
|
|
496
|
+
-dns.promises.lookup without caching using Cloudflare x 2,664 ops/sec ±0.54% (88 runs sampled)
|
|
497
|
+
Fastest without caching is: dns.promises.lookup without caching using Cloudflare
|
|
498
|
+
|
|
499
|
+
Started: resolve
|
|
500
|
+
tangerine.resolve POST with caching using Cloudflare x 1,005 ops/sec ±195.93% (91 runs sampled)
|
|
501
|
+
tangerine.resolve POST without caching using Cloudflare x 55.52 ops/sec ±46.26% (57 runs sampled)
|
|
502
|
+
tangerine.resolve GET with caching using Cloudflare x 2,879,865 ops/sec ±0.35% (86 runs sampled)
|
|
503
|
+
+tangerine.resolve GET without caching using Cloudflare x 71.11 ops/sec ±2.94% (74 runs sampled)
|
|
504
|
+
tangerine.resolve POST with caching using Google x 1,292 ops/sec ±195.91% (88 runs sampled)
|
|
505
|
+
tangerine.resolve POST without caching using Google x 36.88 ops/sec ±41.76% (53 runs sampled)
|
|
506
|
+
tangerine.resolve GET with caching using Google x 2,885,428 ops/sec ±0.22% (88 runs sampled)
|
|
507
|
+
tangerine.resolve GET without caching using Google x 70.38 ops/sec ±3.72% (68 runs sampled)
|
|
508
|
+
resolver.resolve with caching using Cloudflare x 10,645,813 ops/sec ±0.23% (91 runs sampled)
|
|
509
|
+
-resolver.resolve without caching using Cloudflare x 71.80 ops/sec ±2.84% (67 runs sampled)
|
|
510
|
+
+Fastest without caching is: resolver.resolve without caching using Cloudflare, tangerine.resolve GET without caching using Cloudflare, tangerine.resolve GET without caching using Google, tangerine.resolve POST without caching using Cloudflare
|
|
511
|
+
|
|
512
|
+
Started: reverse
|
|
513
|
+
tangerine.reverse GET with caching x 917 ops/sec ±195.78% (88 runs sampled)
|
|
514
|
+
+tangerine.reverse GET without caching x 51.15 ops/sec ±51.92% (61 runs sampled)
|
|
515
|
+
resolver.reverse with caching x 11,058,579 ops/sec ±0.37% (88 runs sampled)
|
|
516
|
+
-resolver.reverse without caching x 62.30 ops/sec ±24.83% (64 runs sampled)
|
|
517
|
+
dns.promises.reverse with caching x 11,276,123 ops/sec ±0.17% (90 runs sampled)
|
|
518
|
+
-dns.promises.reverse without caching x 73.46 ops/sec ±1.99% (69 runs sampled)
|
|
519
|
+
Fastest without caching is: dns.promises.reverse without caching, resolver.reverse without caching
|
|
488
520
|
```
|
|
489
521
|
|
|
490
|
-
> Node v18.
|
|
522
|
+
> Node v18.14.2 on MacBook Air M1 16GB (with DNS blackholed VPN) – **this highlights the DNS blackhole problem**:
|
|
491
523
|
|
|
492
|
-
```
|
|
493
|
-
|
|
494
|
-
v18.
|
|
495
|
-
|
|
496
|
-
❯ node benchmarks/resolve
|
|
497
|
-
tangerine POST with caching using Cloudflare x 817 ops/sec ±193.86% (89 runs sampled)
|
|
498
|
-
tangerine POST without caching using Cloudflare x 42.57 ops/sec ±38.18% (62 runs sampled)
|
|
499
|
-
tangerine GET with caching using Cloudflare x 853 ops/sec ±193.79% (91 runs sampled)
|
|
500
|
-
tangerine GET without caching using Cloudflare x 41.13 ops/sec ±57.37% (48 runs sampled)
|
|
501
|
-
tangerine POST with caching using Google x 1,488 ops/sec ±192.10% (90 runs sampled)
|
|
502
|
-
tangerine POST without caching using Google x 38.46 ops/sec ±12.08% (59 runs sampled)
|
|
503
|
-
tangerine GET with caching using Google x 74,240 ops/sec ±0.31% (90 runs sampled)
|
|
504
|
-
tangerine GET without caching using Google x 39.20 ops/sec ±23.52% (58 runs sampled)
|
|
505
|
-
dns.promises.resolve without caching using Cloudflare x 59.11 ops/sec ±13.96% (63 runs sampled)
|
|
506
|
-
dns.promises.resolve with caching using Cloudflare x 529,961 ops/sec ±0.33% (91 runs sampled)
|
|
507
|
-
Fastest without caching is: dns.promises.resolve without caching using Cloudflare, tangerine GET without caching using Cloudflare
|
|
508
|
-
```
|
|
524
|
+
```diff
|
|
525
|
+
> node --version
|
|
526
|
+
v18.14.2
|
|
509
527
|
|
|
510
|
-
>
|
|
528
|
+
> node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse
|
|
511
529
|
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
tangerine POST
|
|
523
|
-
tangerine
|
|
524
|
-
tangerine GET
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
530
|
+
Started: lookup
|
|
531
|
+
tangerine.lookup POST with caching using Cloudflare x 1,327 ops/sec ±195.65% (89 runs sampled)
|
|
532
|
+
tangerine.lookup POST without caching using Cloudflare x 71.11 ops/sec ±8.24% (71 runs sampled)
|
|
533
|
+
tangerine.lookup GET with caching using Cloudflare x 759,816 ops/sec ±0.46% (90 runs sampled)
|
|
534
|
+
+tangerine.lookup GET without caching using Cloudflare x 73.98 ops/sec ±1.78% (69 runs sampled)
|
|
535
|
+
dns.promises.lookup with caching using Cloudflare x 1,744 ops/sec ±195.97% (88 runs sampled)
|
|
536
|
+
-dns.promises.lookup without caching using Cloudflare x 2,717 ops/sec ±0.82% (87 runs sampled)
|
|
537
|
+
Fastest without caching is: dns.promises.lookup without caching using Cloudflare
|
|
538
|
+
|
|
539
|
+
Started: resolve
|
|
540
|
+
tangerine.resolve POST with caching using Cloudflare x 947 ops/sec ±195.93% (91 runs sampled)
|
|
541
|
+
tangerine.resolve POST without caching using Cloudflare x 44.33 ops/sec ±73.30% (75 runs sampled)
|
|
542
|
+
tangerine.resolve GET with caching using Cloudflare x 2,814,737 ops/sec ±0.17% (91 runs sampled)
|
|
543
|
+
+tangerine.resolve GET without caching using Cloudflare x 57.25 ops/sec ±51.61% (73 runs sampled)
|
|
544
|
+
tangerine.resolve POST with caching using Google x 1,087 ops/sec ±195.92% (91 runs sampled)
|
|
545
|
+
tangerine.resolve POST without caching using Google x 36.84 ops/sec ±7.04% (62 runs sampled)
|
|
546
|
+
tangerine.resolve GET with caching using Google x 2,784,199 ops/sec ±0.15% (92 runs sampled)
|
|
547
|
+
tangerine.resolve GET without caching using Google x 47.55 ops/sec ±5.66% (76 runs sampled)
|
|
548
|
+
resolver.resolve with caching using Cloudflare x 0.09 ops/sec ±6.41% (5 runs sampled)
|
|
549
|
+
-resolver.resolve without caching using Cloudflare x 0.10 ops/sec ±6.52% (5 runs sampled)
|
|
550
|
+
+Fastest without caching is: tangerine.resolve GET without caching using Google
|
|
551
|
+
|
|
552
|
+
Started: reverse
|
|
553
|
+
tangerine.reverse GET with caching x 1,345 ops/sec ±195.66% (92 runs sampled)
|
|
554
|
+
+tangerine.reverse GET without caching x 71.73 ops/sec ±3.03% (73 runs sampled)
|
|
555
|
+
resolver.reverse with caching x 0.10 ops/sec ±6.54% (5 runs sampled)
|
|
556
|
+
-resolver.reverse without caching x 0.10 ops/sec ±0.01% (5 runs sampled)
|
|
557
|
+
dns.promises.reverse with caching x 0.10 ops/sec ±6.54% (5 runs sampled)
|
|
558
|
+
-dns.promises.reverse without caching x 0.10 ops/sec ±0.01% (5 runs sampled)
|
|
559
|
+
+Fastest without caching is: tangerine.reverse GET without caching
|
|
528
560
|
```
|
|
529
561
|
|
|
530
562
|
Also see this [write-up](https://samknows.com/blog/dns-over-https-performance) on UDP-based DNS versus DNS over HTTPS ("DoH") benchmarks.
|
|
@@ -535,39 +567,13 @@ Also see this [write-up](https://samknows.com/blog/dns-over-https-performance) o
|
|
|
535
567
|
|
|
536
568
|
Originally we wrote this library using [got](https://github.com/sindresorhus/got) – however after running benchmarks and learning of [how performant](https://github.com/sindresorhus/got/issues/1419) undici is, we weren't happy – and we rewrote it with [undici](https://github.com/nodejs/undici). Here are test results from the latest versions of all HTTP libraries against our real-world API (both client and server running locally):
|
|
537
569
|
|
|
538
|
-
> Node v16.18.1 on MacBook Air M1 16GB (using real-world API server):
|
|
539
|
-
|
|
540
|
-
```sh
|
|
541
|
-
❯ node --version
|
|
542
|
-
v16.18.1
|
|
543
|
-
|
|
544
|
-
❯ BENCHMARK_HOST="127.0.0.1" BENCHMARK_PORT="4000" BENCHMARK_PATH="/v1/test" node benchmarks/http
|
|
545
|
-
http.request POST request x 860 ops/sec ±6.33% (75 runs sampled)
|
|
546
|
-
http.request GET request x 978 ops/sec ±5.17% (83 runs sampled)
|
|
547
|
-
undici GET request x 2,732 ops/sec ±4.14% (83 runs sampled)
|
|
548
|
-
undici POST request x 1,204 ops/sec ±5.01% (81 runs sampled)
|
|
549
|
-
axios GET request x 855 ops/sec ±5.45% (81 runs sampled)
|
|
550
|
-
axios POST request x 723 ops/sec ±15.28% (71 runs sampled)
|
|
551
|
-
got GET request x 1,355 ops/sec ±16.60% (63 runs sampled)
|
|
552
|
-
got POST request x 93.65 ops/sec ±181.51% (29 runs sampled)
|
|
553
|
-
fetch GET request x 949 ops/sec ±40.26% (45 runs sampled)
|
|
554
|
-
fetch POST request x 672 ops/sec ±22.43% (67 runs sampled)
|
|
555
|
-
request GET request x 960 ops/sec ±50.90% (48 runs sampled)
|
|
556
|
-
request POST request x 612 ops/sec ±45.48% (57 runs sampled)
|
|
557
|
-
superagent GET request x 126 ops/sec ±188.34% (29 runs sampled)
|
|
558
|
-
superagent POST request x 747 ops/sec ±18.16% (67 runs sampled)
|
|
559
|
-
phin GET request x 374 ops/sec ±147.42% (57 runs sampled)
|
|
560
|
-
phin POST request x 566 ops/sec ±38.08% (51 runs sampled)
|
|
561
|
-
Fastest is undici GET request
|
|
562
|
-
```
|
|
563
|
-
|
|
564
570
|
> Node v18.14.2 on MacBook Air M1 16GB (using real-world API server):
|
|
565
571
|
|
|
566
572
|
```sh
|
|
567
|
-
|
|
573
|
+
> node --version
|
|
568
574
|
v18.14.2
|
|
569
575
|
|
|
570
|
-
|
|
576
|
+
> BENCHMARK_HOST="127.0.0.1" BENCHMARK_PORT="4000" BENCHMARK_PATH="/v1/test" node benchmarks/http
|
|
571
577
|
http.request POST request x 765 ops/sec ±9.83% (72 runs sampled)
|
|
572
578
|
http.request GET request x 1,000 ops/sec ±3.88% (85 runs sampled)
|
|
573
579
|
undici GET request x 2,740 ops/sec ±5.92% (78 runs sampled)
|
package/index.js
CHANGED
|
@@ -7,14 +7,15 @@ const { debuglog } = require('node:util');
|
|
|
7
7
|
const { getEventListeners, setMaxListeners } = require('node:events');
|
|
8
8
|
const { isIP, isIPv4, isIPv6 } = require('node:net');
|
|
9
9
|
|
|
10
|
-
const Keyv = require('keyv');
|
|
11
10
|
const getStream = require('get-stream');
|
|
12
11
|
const ipaddr = require('ipaddr.js');
|
|
13
12
|
const mergeOptions = require('merge-options');
|
|
14
13
|
const pMap = require('p-map');
|
|
14
|
+
const pTimeout = require('p-timeout');
|
|
15
15
|
const pWaitFor = require('p-wait-for');
|
|
16
16
|
const packet = require('dns-packet');
|
|
17
17
|
const semver = require('semver');
|
|
18
|
+
const structuredClone = require('@ungap/structured-clone').default;
|
|
18
19
|
const { getService } = require('port-numbers');
|
|
19
20
|
|
|
20
21
|
const pkg = require('./package.json');
|
|
@@ -69,6 +70,8 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
69
70
|
let err;
|
|
70
71
|
if (errors.length === 1) {
|
|
71
72
|
err = errors[0];
|
|
73
|
+
} else if (errors.every((e) => e instanceof pTimeout.TimeoutError)) {
|
|
74
|
+
err = errors[0];
|
|
72
75
|
} else {
|
|
73
76
|
err = new Error(
|
|
74
77
|
[...new Set(errors.map((e) => e.message).filter(Boolean))].join('; ')
|
|
@@ -76,6 +79,14 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
76
79
|
err.stack = [...new Set(errors.map((e) => e.stack).filter(Boolean))].join(
|
|
77
80
|
'\n\n'
|
|
78
81
|
);
|
|
82
|
+
|
|
83
|
+
// if all errors had `name` and they were all the same then preserve it
|
|
84
|
+
if (
|
|
85
|
+
typeof errors[0].name !== 'undefined' &&
|
|
86
|
+
errors.every((e) => e.name === errors[0].name)
|
|
87
|
+
)
|
|
88
|
+
err.name = errors[0].name;
|
|
89
|
+
|
|
79
90
|
// if all errors had `code` and they were all the same then preserve it
|
|
80
91
|
if (
|
|
81
92
|
typeof errors[0].code !== 'undefined' &&
|
|
@@ -251,7 +262,6 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
251
262
|
accept: 'application/dns-message'
|
|
252
263
|
}
|
|
253
264
|
},
|
|
254
|
-
requestTimeout: (ms) => ({ bodyTimeout: ms }),
|
|
255
265
|
//
|
|
256
266
|
// NOTE: we set the default to "get" since it is faster from `benchmark` results
|
|
257
267
|
//
|
|
@@ -277,8 +287,19 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
277
287
|
ipv6: '::0',
|
|
278
288
|
ipv4Port: undefined,
|
|
279
289
|
ipv6Port: undefined,
|
|
280
|
-
// cache mapping (e.g. txt -> keyv instance) - see below
|
|
290
|
+
// cache mapping (e.g. txt -> Map/keyv/redis instance) - see below
|
|
281
291
|
cache: new Map(),
|
|
292
|
+
// <https://developers.cloudflare.com/dns/manage-dns-records/reference/ttl/>
|
|
293
|
+
defaultTTLSeconds: 300,
|
|
294
|
+
maxTTLSeconds: 86400,
|
|
295
|
+
// default is to support ioredis
|
|
296
|
+
// setCacheArgs(key, result) {
|
|
297
|
+
setCacheArgs() {
|
|
298
|
+
// also you have access to `result.expires` which is is ms since epoch
|
|
299
|
+
// (can be converted to Date via `new Date(result.expires)`)
|
|
300
|
+
// return ['PX', Math.round(result.ttl * 1000)];
|
|
301
|
+
return [];
|
|
302
|
+
},
|
|
282
303
|
// whether to do 1:1 HTTP -> DNS error mapping
|
|
283
304
|
returnHTTPErrors: false,
|
|
284
305
|
// whether to smart rotate and bump-to-end servers that have issues
|
|
@@ -327,19 +348,6 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
327
348
|
// so to turn that off, you need to supply `dnsCache: undefined` in `got` object (?)
|
|
328
349
|
if (this.options.cache === true) this.options.cache = new Map();
|
|
329
350
|
|
|
330
|
-
if (this.options.cache instanceof Map) {
|
|
331
|
-
// each of the types have their own Keyv with prefix
|
|
332
|
-
for (const type of this.constructor.TYPES) {
|
|
333
|
-
if (!this.options.cache.get(type))
|
|
334
|
-
this.options.cache.set(
|
|
335
|
-
type,
|
|
336
|
-
new Keyv({
|
|
337
|
-
namespace: `dns:${type.toLowerCase()}`
|
|
338
|
-
})
|
|
339
|
-
);
|
|
340
|
-
}
|
|
341
|
-
}
|
|
342
|
-
|
|
343
351
|
// convert `false` logger option into noop
|
|
344
352
|
// <https://github.com/breejs/bree/issues/147>
|
|
345
353
|
if (this.options.logger === false)
|
|
@@ -756,7 +764,6 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
756
764
|
|
|
757
765
|
const options = {
|
|
758
766
|
...this.options.requestOptions,
|
|
759
|
-
...this.options.requestTimeout(timeout), // returns `{ bodyTimeout: requestTimeout }`
|
|
760
767
|
signal: abortController.signal
|
|
761
768
|
};
|
|
762
769
|
|
|
@@ -772,7 +779,9 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
772
779
|
}
|
|
773
780
|
|
|
774
781
|
debug('request', { url, options });
|
|
775
|
-
const response = await this.request(url, options)
|
|
782
|
+
const response = await pTimeout(this.request(url, options), timeout, {
|
|
783
|
+
signal: abortController.signal
|
|
784
|
+
});
|
|
776
785
|
return response;
|
|
777
786
|
}
|
|
778
787
|
|
|
@@ -901,7 +910,9 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
901
910
|
const err = this.constructor.createError(
|
|
902
911
|
name,
|
|
903
912
|
rrtype,
|
|
904
|
-
_err.
|
|
913
|
+
_err instanceof pTimeout.TimeoutError || _err.name === 'TimeoutError'
|
|
914
|
+
? dns.TIMEOUT
|
|
915
|
+
: _err.code,
|
|
905
916
|
_err.errno
|
|
906
917
|
);
|
|
907
918
|
// then map it to dns.CONNREFUSED
|
|
@@ -1073,7 +1084,7 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
1073
1084
|
}
|
|
1074
1085
|
|
|
1075
1086
|
//
|
|
1076
|
-
//
|
|
1087
|
+
// NOTE: every address must be ipv4 or ipv6 (use `new URL` to parse and check)
|
|
1077
1088
|
// servers [ string ] - array of RFC 5952 formatted addresses
|
|
1078
1089
|
//
|
|
1079
1090
|
|
|
@@ -1110,51 +1121,59 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
1110
1121
|
delete options.ecsSubnet;
|
|
1111
1122
|
}
|
|
1112
1123
|
|
|
1113
|
-
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
const key = ecsSubnet ? `${ecsSubnet}:${name}` : name;
|
|
1124
|
+
const key = (
|
|
1125
|
+
ecsSubnet ? `${rrtype}:${ecsSubnet}:${name}` : `${rrtype}:${name}`
|
|
1126
|
+
).toLowerCase();
|
|
1118
1127
|
|
|
1119
1128
|
let result;
|
|
1120
1129
|
let data;
|
|
1121
|
-
if (cache) {
|
|
1130
|
+
if (this.options.cache) {
|
|
1122
1131
|
//
|
|
1123
|
-
//
|
|
1124
|
-
//
|
|
1125
|
-
// NOTE: we store `result.lowest_answer_ttl` which was the lowest TTL determined
|
|
1132
|
+
// NOTE: we store `result.ttl` which was the lowest TTL determined
|
|
1126
1133
|
// (this saves us from duplicating the same `...sort().filter(Number.isFinite)` logic)
|
|
1127
1134
|
//
|
|
1128
|
-
data = await cache.get(key
|
|
1129
|
-
|
|
1130
|
-
|
|
1135
|
+
data = await this.options.cache.get(key);
|
|
1136
|
+
// safeguard in case cache pollution
|
|
1137
|
+
if (data && typeof data === 'object') {
|
|
1138
|
+
debug('cache retrieved', data);
|
|
1131
1139
|
const now = Date.now();
|
|
1140
|
+
// safeguard in case cache pollution
|
|
1132
1141
|
if (
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
data.
|
|
1137
|
-
now <= data.expires
|
|
1142
|
+
!Number.isFinite(data.expires) ||
|
|
1143
|
+
data.expires < now ||
|
|
1144
|
+
!Number.isFinite(data.ttl) ||
|
|
1145
|
+
data.ttl < 1
|
|
1138
1146
|
) {
|
|
1147
|
+
data = undefined;
|
|
1148
|
+
} else if (options?.ttl) {
|
|
1149
|
+
// clone the data so that we don't mutate cache (e.g. if it's in-memory)
|
|
1150
|
+
// <https://nodejs.org/api/globals.html#structuredclonevalue-options>
|
|
1151
|
+
// <https://github.com/ungap/structured-clone>
|
|
1152
|
+
data = structuredClone(data);
|
|
1153
|
+
|
|
1139
1154
|
// returns ms -> s conversion
|
|
1140
1155
|
const ttl = Math.round((data.expires - now) / 1000);
|
|
1141
|
-
const diff =
|
|
1156
|
+
const diff = data.ttl - ttl;
|
|
1142
1157
|
|
|
1143
|
-
for (let i = 0; i <
|
|
1158
|
+
for (let i = 0; i < data.answers.length; i++) {
|
|
1144
1159
|
// eslint-disable-next-line max-depth
|
|
1145
|
-
if (typeof
|
|
1160
|
+
if (typeof data.answers[i].ttl === 'number') {
|
|
1146
1161
|
// subtract ttl from answer
|
|
1147
|
-
|
|
1162
|
+
data.answers[i].ttl = Math.round(data.answers[i].ttl - diff);
|
|
1148
1163
|
|
|
1149
1164
|
// eslint-disable-next-line max-depth
|
|
1150
|
-
if (
|
|
1151
|
-
result = undefined;
|
|
1165
|
+
if (data.answers[i].ttl <= 0) {
|
|
1152
1166
|
data = undefined;
|
|
1153
1167
|
break;
|
|
1154
1168
|
}
|
|
1155
1169
|
}
|
|
1156
1170
|
}
|
|
1157
1171
|
}
|
|
1172
|
+
|
|
1173
|
+
// will only use cache if it's still set after parsing ttl
|
|
1174
|
+
result = data;
|
|
1175
|
+
} else {
|
|
1176
|
+
data = undefined;
|
|
1158
1177
|
}
|
|
1159
1178
|
}
|
|
1160
1179
|
|
|
@@ -1187,8 +1206,8 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
1187
1206
|
// - The DoH service could not contact Google Public DNS resolvers.
|
|
1188
1207
|
// - In the case of a 502 response, although retrying on an alternate Google Public DNS address might help, a more effective fallback response would be to try another DoH service, or to switch to traditional UDP or TCP DNS at 8.8.8.8.
|
|
1189
1208
|
//
|
|
1190
|
-
if (cache && result) {
|
|
1191
|
-
debug(`cached result found for "${
|
|
1209
|
+
if (this.options.cache && result) {
|
|
1210
|
+
debug(`cached result found for "${key}"`);
|
|
1192
1211
|
} else {
|
|
1193
1212
|
if (!abortController) {
|
|
1194
1213
|
abortController = new AbortController();
|
|
@@ -1228,31 +1247,37 @@ class Tangerine extends dns.promises.Resolver {
|
|
|
1228
1247
|
//
|
|
1229
1248
|
switch (result.rcode) {
|
|
1230
1249
|
case 'NOERROR': {
|
|
1231
|
-
//
|
|
1232
|
-
// NOTE: if the answer was truncated then unset results (?)
|
|
1233
|
-
// <https://github.com/EduardoRuizM/native-dnssec-dns/blob/fc27face6c64ab53675840bafc81f70bab48a743/lib/client.js#L354>
|
|
1234
|
-
// <https://github.com/hildjj/dohdec/issues/40>
|
|
1235
|
-
// if (result.flag_tc) throw createError(name, rrtype, dns.BADRESP);
|
|
1250
|
+
// <https://github.com/hildjj/dohdec/issues/40#issuecomment-1445554626>
|
|
1236
1251
|
if (result.flag_tc) {
|
|
1237
|
-
this.options.logger.error(
|
|
1238
|
-
|
|
1239
|
-
|
|
1240
|
-
|
|
1241
|
-
|
|
1242
|
-
|
|
1252
|
+
this.options.logger.error(
|
|
1253
|
+
new Error(
|
|
1254
|
+
'Truncated DNS response; Defer to https://github.com/hildjj/dohdec/issues/40#issuecomment-1445554626 for insight.'
|
|
1255
|
+
),
|
|
1256
|
+
{
|
|
1257
|
+
name,
|
|
1258
|
+
rrtype,
|
|
1259
|
+
result
|
|
1260
|
+
}
|
|
1261
|
+
);
|
|
1262
|
+
} else if (this.options.cache && !data) {
|
|
1243
1263
|
// store in cache based off lowest ttl
|
|
1244
|
-
|
|
1264
|
+
let ttl = result.answers
|
|
1245
1265
|
.map((answer) => answer.ttl)
|
|
1246
1266
|
.sort()
|
|
1247
1267
|
.find((ttl) => Number.isFinite(ttl));
|
|
1248
|
-
|
|
1249
|
-
|
|
1250
|
-
|
|
1251
|
-
|
|
1252
|
-
|
|
1253
|
-
|
|
1254
|
-
|
|
1255
|
-
|
|
1268
|
+
// if TTL is not a number or is < 1 or is > max then set to default
|
|
1269
|
+
if (
|
|
1270
|
+
!Number.isFinite(ttl) ||
|
|
1271
|
+
ttl < 1 ||
|
|
1272
|
+
ttl > this.options.maxTTLSeconds
|
|
1273
|
+
)
|
|
1274
|
+
ttl = this.options.defaultTTLSeconds;
|
|
1275
|
+
result.ttl = ttl;
|
|
1276
|
+
// this supports both redis-based key/value/ttl and simple key/value implementations
|
|
1277
|
+
result.expires = Date.now() + Math.round(result.ttl * 1000);
|
|
1278
|
+
const args = [key, result, ...this.options.setCacheArgs(key, result)];
|
|
1279
|
+
debug('setting cache', [key, result, ...args]);
|
|
1280
|
+
await this.options.cache.set(...args);
|
|
1256
1281
|
}
|
|
1257
1282
|
|
|
1258
1283
|
break;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tangerine",
|
|
3
|
-
"description": "Tangerine is the best Node.js drop-in replacement for dns.promises.Resolver using DNS over HTTPS (\"DoH\") via undici with built-in retries, timeouts, smart server rotation, AbortControllers, and caching support for multiple backends
|
|
4
|
-
"version": "1.
|
|
3
|
+
"description": "Tangerine is the best Node.js drop-in replacement for dns.promises.Resolver using DNS over HTTPS (\"DoH\") via undici with built-in retries, timeouts, smart server rotation, AbortControllers, and caching support for multiple backends (with TTL support).",
|
|
4
|
+
"version": "1.3.0",
|
|
5
5
|
"author": "Forward Email (https://forwardemail.net)",
|
|
6
6
|
"bugs": {
|
|
7
7
|
"url": "https://github.com/forwardemail/tangerine/issues"
|
|
@@ -10,13 +10,14 @@
|
|
|
10
10
|
"Forward Email (https://forwardemail.net)"
|
|
11
11
|
],
|
|
12
12
|
"dependencies": {
|
|
13
|
+
"@ungap/structured-clone": "^1.0.2",
|
|
13
14
|
"dns-packet": "^5.4.0",
|
|
14
15
|
"dohdec": "^5.0.3",
|
|
15
16
|
"get-stream": "6",
|
|
16
17
|
"ipaddr.js": "^2.0.1",
|
|
17
|
-
"keyv": "^4.5.2",
|
|
18
18
|
"merge-options": "3.0.4",
|
|
19
19
|
"p-map": "4",
|
|
20
|
+
"p-timeout": "4",
|
|
20
21
|
"p-wait-for": "3",
|
|
21
22
|
"port-numbers": "^6.0.1",
|
|
22
23
|
"semver": "^7.3.8"
|
|
@@ -34,6 +35,8 @@
|
|
|
34
35
|
"fixpack": "^4.0.0",
|
|
35
36
|
"got": "11",
|
|
36
37
|
"husky": "^8.0.3",
|
|
38
|
+
"ioredis": "^5.3.1",
|
|
39
|
+
"ioredis-mock": "^8.2.6",
|
|
37
40
|
"lint-staged": "^13.1.2",
|
|
38
41
|
"lodash": "^4.17.21",
|
|
39
42
|
"nock": "^13.3.0",
|
|
@@ -149,7 +152,7 @@
|
|
|
149
152
|
},
|
|
150
153
|
"scripts": {
|
|
151
154
|
"ava": "cross-env NODE_ENV=test ava",
|
|
152
|
-
"benchmarks": "node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse
|
|
155
|
+
"benchmarks": "node benchmarks/lookup && node benchmarks/resolve && node benchmarks/reverse",
|
|
153
156
|
"lint": "xo --fix && remark . -qfo && fixpack",
|
|
154
157
|
"nyc": "cross-env NODE_ENV=test nyc ava",
|
|
155
158
|
"prepare": "husky install",
|