newrelic 11.2.1 → 11.4.0

package/NEWS.md

@@ -1,3 +1,43 @@
+### v11.4.0 (2023-10-25)
+
+#### Features
+
+* Added support for parsing container ids from docker versions using cgroups v2. ([#1830](https://github.com/newrelic/node-newrelic/pull/1830)) ([9892901](https://github.com/newrelic/node-newrelic/commit/98929013da3e62e2223f94531b8d6f59eecfc35b))
+
+#### Miscellaneous chores
+
+* [Snyk] Upgraded @grpc/grpc-js from 1.9.2 to 1.9.4. ([#1823](https://github.com/newrelic/node-newrelic/pull/1823)) ([81f9450](https://github.com/newrelic/node-newrelic/commit/81f945033376e4d33651d1e42afc30aea19dbdeb))
+* **deps:** Updated aws-sdk, koa, superagent ([#1831](https://github.com/newrelic/node-newrelic/pull/1831)) ([8b4e963](https://github.com/newrelic/node-newrelic/commit/8b4e963e84d34e4727b9fda3aa630ef119aa3905))
+
+#### Tests
+
+* Increased timeout for integration tests to avoid random failures. ([#1827](https://github.com/newrelic/node-newrelic/pull/1827)) ([5202048](https://github.com/newrelic/node-newrelic/commit/52020485191868f479092ae4860019acf105b3eb))
+
+### v11.3.0 (2023-10-23)
+
+#### Features
+
+* Updated agent initialization to allow running in worker threads when config.worker_threads.enabled is true ([#1817](https://github.com/newrelic/node-newrelic/pull/1817)) ([a39f0ef](https://github.com/newrelic/node-newrelic/commit/a39f0ef5ac670d03ab407b24e5aeccd8d5e8c680))
+
+#### Bug fixes
+
+* Updated Elasticsearch instrumentation to register on v7.13.0+ only ([#1816](https://github.com/newrelic/node-newrelic/pull/1816)) ([6437671](https://github.com/newrelic/node-newrelic/commit/6437671b921cd6bd73ed31180b0d62f62cc229a2))
+
+#### Miscellaneous chores
+
+* **dev-deps:** Bumped @babel/traverse ([#1818](https://github.com/newrelic/node-newrelic/pull/1818)) ([d3c8d04](https://github.com/newrelic/node-newrelic/commit/d3c8d04b74b7a84846609b744e3b4922136dbdd6))
+* Updated release note headers to follow standard sentence caps ([#1806](https://github.com/newrelic/node-newrelic/pull/1806)) ([91d3600](https://github.com/newrelic/node-newrelic/commit/91d36009e0496af823cfbc3a4bdb2b32a97ba8c5))
+
+#### Tests
+
+* Updated the grpc versioned tests utils to dynamically bind ports to avoid conflicts between cjs and esm tests ([#1820](https://github.com/newrelic/node-newrelic/pull/1820)) ([95ac917](https://github.com/newrelic/node-newrelic/commit/95ac917da92575e178d8026bdc0badb08ba6fb83))
+
+#### Continuous integration
+
+* Disabled fail-fast on testing jobs to make sure all versions are run before getting canceled because of a flappy test ([#1819](https://github.com/newrelic/node-newrelic/pull/1819)) ([0928ee3](https://github.com/newrelic/node-newrelic/commit/0928ee3db82533e2386ad2bf4b87074b4f15f33b))
+* Updated CI workflow to use larger runners on versioned tests but only when running against the main branch ([#1812](https://github.com/newrelic/node-newrelic/pull/1812)) ([01eaa14](https://github.com/newrelic/node-newrelic/commit/01eaa14c76a32966f6af8475e6ba6d4e00b03513))
+* Updated post release script to update the nodejs_agent_version only on staging and us prod, eu will get it via replication now ([#1811](https://github.com/newrelic/node-newrelic/pull/1811)) ([317a00a](https://github.com/newrelic/node-newrelic/commit/317a00a9c160b52b053cb1f9f55292551c4c3428))
+
 ### v11.2.1 (2023-10-12)
 
 #### Bug fixes
@@ -59,6 +99,7 @@
 * Updated agent to use require-in-the-middle to register CommonJS instrumentation. You can no longer use an onResolved hook to register custom instrumentation.
 * Updated the default context manager to be AsyncLocalContextManager.
 * Renamed `shim.handleCATHeaders` to `shim.handleMqTracingHeaders`.
+* Updated agent to only run in the main thread. This is because running in a worker thread does not completely function out of the box. This will reduce the overhead for customers that are naively trying to load this into worker threads.
 
 #### Features
 

package/THIRD_PARTY_NOTICES.md

@@ -92,7 +92,7 @@ code, the source code can be found at [https://github.com/newrelic/node-newrelic
 
 ### @grpc/grpc-js
 
-This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.9.2](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.9.2)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.9.2/LICENSE):
+This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.9.4](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.9.4)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.9.4/LICENSE):
 
 ```
                                  Apache License
@@ -510,7 +510,7 @@ This product includes source derived from [@grpc/proto-loader](https://github.co
 
 ### @newrelic/aws-sdk
 
-This product includes source derived from [@newrelic/aws-sdk](https://github.com/newrelic/node-newrelic-aws-sdk) ([v7.0.0](https://github.com/newrelic/node-newrelic-aws-sdk/tree/v7.0.0)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-aws-sdk/blob/v7.0.0/LICENSE):
+This product includes source derived from [@newrelic/aws-sdk](https://github.com/newrelic/node-newrelic-aws-sdk) ([v7.0.2](https://github.com/newrelic/node-newrelic-aws-sdk/tree/v7.0.2)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-aws-sdk/blob/v7.0.2/LICENSE):
 
 ```
                                  Apache License
@@ -718,7 +718,7 @@ This product includes source derived from [@newrelic/aws-sdk](https://github.com
 
 ### @newrelic/koa
 
-This product includes source derived from [@newrelic/koa](https://github.com/newrelic/node-newrelic-koa) ([v8.0.0](https://github.com/newrelic/node-newrelic-koa/tree/v8.0.0)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-koa/blob/v8.0.0/LICENSE):
+This product includes source derived from [@newrelic/koa](https://github.com/newrelic/node-newrelic-koa) ([v8.0.1](https://github.com/newrelic/node-newrelic-koa/tree/v8.0.1)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-koa/blob/v8.0.1/LICENSE):
 
 ```
 Apache License
@@ -978,7 +978,7 @@ This license terminates when the Pre-Release Software stops being provided by Ne
 
 ### @newrelic/superagent
 
-This product includes source derived from [@newrelic/superagent](https://github.com/newrelic/node-newrelic-superagent) ([v7.0.0](https://github.com/newrelic/node-newrelic-superagent/tree/v7.0.0)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-superagent/blob/v7.0.0/LICENSE):
+This product includes source derived from [@newrelic/superagent](https://github.com/newrelic/node-newrelic-superagent) ([v7.0.1](https://github.com/newrelic/node-newrelic-superagent/tree/v7.0.1)), distributed under the [Apache-2.0 License](https://github.com/newrelic/node-newrelic-superagent/blob/v7.0.1/LICENSE):
 
 ```
                                  Apache License

package/index.js

@@ -20,28 +20,23 @@ const NAMES = require('./lib/metrics/names')
 const isESMSupported = psemver.satisfies('>=16.2.0')
 
 const pkgJSON = require('./package.json')
-if (isMainThread) {
-  logger.info(
-    'Using New Relic for Node.js. Agent version: %s; Node version: %s.',
-    pkgJSON.version,
-    process.version
+logger.info(
+  'Using New Relic for Node.js. Agent version: %s; Node version: %s.',
+  pkgJSON.version,
+  process.version
+)
+
+if (require.cache.__NR_cache) {
+  logger.warn(
+    'Attempting to load a second copy of newrelic from %s, using cache instead',
+    __dirname
   )
-
-  if (require.cache.__NR_cache) {
-    logger.warn(
-      'Attempting to load a second copy of newrelic from %s, using cache instead',
-      __dirname
-    )
-    if (require.cache.__NR_cache.agent) {
-      require.cache.__NR_cache.agent.recordSupportability('Agent/DoubleLoad')
-    }
-    module.exports = require.cache.__NR_cache
-  } else {
-    initialize()
+  if (require.cache.__NR_cache.agent) {
+    require.cache.__NR_cache.agent.recordSupportability('Agent/DoubleLoad')
   }
+  module.exports = require.cache.__NR_cache
 } else {
-  logger.warn('Using New Relic for Node.js in worker_threads is not supported. Not starting!')
-  initApi({ apiPath: './stub_api' })
+  initialize()
 }
 
 function initApi({ agent, apiPath }) {
@@ -97,7 +92,16 @@ function initialize() {
       logger.info('No configuration detected. Not starting.')
     } else if (!config.agent_enabled) {
       logger.info('Module disabled in configuration. Not starting.')
+    } else if (!config.worker_threads.enabled && !isMainThread) {
+      logger.warn(
+        'New Relic for Node.js in worker_threads is not officially supported. Not starting! To bypass this, set `config.worker_threads.enabled` to true in configuration.'
+      )
     } else {
+      if (!isMainThread && config.worker_threads.enabled) {
+        logger.warn(
+          'Attempting to load agent in worker thread. This is not officially supported. Use at your own risk.'
+        )
+      }
       agent = createAgent(config)
       addStartupSupportabilities(agent)
     }

package/lib/config/default.js

@@ -1320,11 +1320,31 @@ defaultConfig.definition = () => ({
     }
   },
 
+  /**
+   * When enabled, it will use `process.env.DYNO`
+   * to set the hostname of the running application
+   */
   heroku: {
     use_dyno_names: {
       default: true,
       formatter: boolean
     }
+  },
+
+  /**
+   * When enabled, it will allow loading of the agent
+   * in worker threads.
+   *
+   * In 11.0.0 we added code to prevent loading in worker threads
+   * to cut down on unnecessary overhead of the agent.  We have found
+   * in testing that traces and spans were useless unless work was
+   * completely self contained in the worker thread.
+   */
+  worker_threads: {
+    enabled: {
+      formatter: boolean,
+      default: false
+    }
   }
 })
 

package/lib/config/merge-server-config.js

@@ -5,6 +5,8 @@
 
 'use strict'
 
+const { isSimpleObject } = require('../util/objects')
+
 class MergeServerConfig {
   // eslint-disable-next-line max-params
   updateNestedIfChanged(config, remote, local, remoteKey, localKey, logger) {
@@ -21,7 +23,7 @@ class MergeServerConfig {
     // value is an array, a simple object, or anything else
     if (Array.isArray(value) && Array.isArray(local[localKey])) {
       this.updateArray(value, local, localKey)
-    } else if (this.isSimpleObject(value) && this.isSimpleObject(local[localKey])) {
+    } else if (isSimpleObject(value) && isSimpleObject(local[localKey])) {
       this.updateObject(value, local, localKey)
     } else {
       local[localKey] = value
@@ -54,10 +56,6 @@ class MergeServerConfig {
       }
     })
   }
-
-  isSimpleObject(thing) {
-    return 'object' === typeof thing && thing !== null
-  }
 }
 
 module.exports = MergeServerConfig

package/lib/instrumentation/@elastic/elasticsearch.js

@@ -4,7 +4,9 @@
  */
 
 'use strict'
+const semver = require('semver')
 const logger = require('../../logger').child({ component: 'ElasticSearch' })
+const { isNotEmpty } = require('../../util/objects')
 
 /**
  * Instruments the `@elastic/elasticsearch` module. This function is
@@ -17,6 +19,15 @@ const logger = require('../../logger').child({ component: 'ElasticSearch' })
  * @returns {void}
  */
 module.exports = function initialize(_agent, elastic, _moduleName, shim) {
+  const pkgVersion = shim.require('./package.json').version
+  if (semver.lt(pkgVersion, '7.13.0')) {
+    shim &&
+      shim.logger.debug(
+        `ElasticSearch support is for versions 7.13.0 and above. Not instrumenting ${pkgVersion}.`
+      )
+    return
+  }
+
   shim.setDatastore(shim.ELASTICSEARCH)
   shim.setParser(queryParser)
 
@@ -46,18 +57,17 @@ module.exports = function initialize(_agent, elastic, _moduleName, shim) {
  */
 function queryParser(params) {
   params = JSON.parse(params)
-
   const { collection, operation } = parsePath(params.path, params.method)
 
   // the substance of the query may be in querystring or in body.
   let queryParam = {}
-  if (typeof params.querystring === 'object' && Object.keys(params.querystring).length > 0) {
+  if (isNotEmpty(params.querystring)) {
     queryParam = params.querystring
   }
   // let body or bulkBody override querystring, as some requests have both
-  if (typeof params.body === 'object' && Object.keys(params.body).length > 0) {
+  if (isNotEmpty(params.body)) {
     queryParam = params.body
-  } else if (typeof params.bulkBody === 'object' && Object.keys(params.bulkBody).length > 0) {
+  } else if (isNotEmpty(params.bulkBody)) {
     queryParam = params.bulkBody
   }
 

package/lib/util/objects.js

@@ -0,0 +1,27 @@
+/*
+ * Copyright 2020 New Relic Corporation. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+'use strict'
+exports = module.exports = { isSimpleObject, isNotEmpty }
+
+/**
+ * Convenience function to test if a value is a non-null object
+ *
+ * @param {object} thing Value to be tested
+ * @returns {boolean} whether or not the value is an object and not null
+ */
+function isSimpleObject(thing) {
+  return 'object' === typeof thing && thing !== null
+}
+
+/**
+ * Convenience function to test if an object is not empty
+ *
+ * @param {object} thing Value to be tested
+ * @returns {boolean} true if the value is an object, not null, and has keys
+ */
+function isNotEmpty(thing) {
+  return isSimpleObject(thing) && Object.keys(thing).length > 0
+}

package/lib/utilization/docker-info.js

@@ -10,6 +10,8 @@ const common = require('./common')
 const NAMES = require('../metrics/names')
 const os = require('os')
 let vendorInfo = null
+const CGROUPS_V1_PATH = '/proc/self/cgroup'
+const CGROUPS_V2_PATH = '/proc/self/mountinfo'
 
 module.exports.getVendorInfo = fetchDockerVendorInfo
 module.exports.clearVendorCache = function clearDockerVendorCache() {
@@ -51,6 +53,12 @@ module.exports.getBootId = function getBootId(agent, callback) {
   }
 }
 
+/**
+ * Attempt to extract container id from either cgroups v1 or v2 file
+ *
+ * @param {object} agent NR instance
+ * @param {Function} callback function to call when done
+ */
 function fetchDockerVendorInfo(agent, callback) {
   if (!agent.config.utilization || !agent.config.utilization.detect_docker) {
     return callback(null, null)
@@ -62,16 +70,56 @@ function fetchDockerVendorInfo(agent, callback) {
 
   if (!os.platform().match(/linux/i)) {
     logger.debug('Platform is not a flavor of linux, omitting docker info')
-    return callback(null)
+    return callback(null, null)
   }
 
-  common.readProc('/proc/self/cgroup', function getCGroup(err, data) {
+  // try v2 path first and if null try parsing v1 path
+  common.readProc(CGROUPS_V2_PATH, function getV2CGroup(err, data) {
+    if (data === null) {
+      logger.debug(
+        `${CGROUPS_V2_PATH} not found, trying to parse container id from ${CGROUPS_V1_PATH}`
+      )
+      findCGroupsV1(callback)
+      return
+    }
+
+    parseCGroupsV2(data, callback)
+  })
+}
+
+/**
+ * Try extracting container id from a /proc/self/mountinfo
+ * e.g. - `528 519 254:1 /docker/containers/84cf3472a20d1bfb4b50e48b6ff50d96dfcd812652d76dd907951e6f98997bce/resolv.conf`
+ *
+ * @param {string} data file contents
+ * @param {Function} callback function to call when done
+ */
+function parseCGroupsV2(data, callback) {
+  const containerLine = new RegExp('/docker/containers/([0-9a-f]{64})/')
+  const line = containerLine.exec(data)
+  if (line) {
+    callback(null, { id: line[1] })
+  } else {
+    logger.debug(`Found ${CGROUPS_V2_PATH} but failed to parse Docker container id.`)
+    callback(null, null)
+  }
+}
+
+/**
+ * Read /proc/self/cgroup and try to extract the container id from a cpu line
+ * e.g. - `4:cpu:/docker/f37a7e4d17017e7bf774656b19ca4360c6cdc4951c86700a464101d0d9ce97ee`
+ *
+ * @param {Function} callback function to call when done
+ */
+function findCGroupsV1(callback) {
+  common.readProc(CGROUPS_V1_PATH, function getCGroup(err, data) {
     if (!data) {
+      logger.debug(`${CGROUPS_V1_PATH} not found, exiting parsing containerId.`)
       return callback(null)
     }
 
     let id = null
-    findCGroups(data, 'cpu', function forEachCpuGroup(cpuGroup) {
+    parseCGroupsV1(data, 'cpu', function forEachCpuGroup(cpuGroup) {
       const match = /(?:^|[^0-9a-f])([0-9a-f]{64})(?:[^0-9a-f]|$)/.exec(cpuGroup)
       if (match) {
         id = match[1]
@@ -91,7 +139,14 @@ function fetchDockerVendorInfo(agent, callback) {
   })
 }
 
-function findCGroups(info, cgroup, eachCb) {
+/**
+ * Iterate line by line to extract the container id from the cpu stanza
+ *
+ * @param {string} info contents of file
+ * @param {string} cgroup value is cpu
+ * @param {Function} eachCb funtion to test if the container id exists
+ */
+function parseCGroupsV1(info, cgroup, eachCb) {
   const target = new RegExp('^\\d+:[^:]*?\\b' + cgroup + '\\b[^:]*:')
   const lines = info.split('\n')
   for (let i = 0; i < lines.length; ++i) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "newrelic",
-  "version": "11.2.1",
+  "version": "11.4.0",
   "author": "New Relic Node.js agent team <nodejs@newrelic.com>",
   "license": "Apache-2.0",
   "contributors": [
@@ -146,7 +146,7 @@
     "bench": "node ./bin/run-bench.js",
     "docker-env": "./bin/docker-env-vars.sh",
     "docs": "npm ci && jsdoc -c ./jsdoc-conf.json --private -r .",
-    "integration": "npm run prepare-test && npm run sub-install && time c8 -o ./coverage/integration tap --test-regex='(\\/|^test\\/integration\\/.*\\.tap\\.js)$' --timeout=300 --no-coverage --reporter classic",
+    "integration": "npm run prepare-test && npm run sub-install && time c8 -o ./coverage/integration tap --test-regex='(\\/|^test\\/integration\\/.*\\.tap\\.js)$' --timeout=600 --no-coverage --reporter classic",
     "prepare-test": "npm run ssl && npm run docker-env",
     "lint": "eslint ./*.{js,mjs} lib test bin examples",
     "lint:fix": "eslint --fix, ./*.{js,mjs} lib test bin examples",
@@ -181,12 +181,12 @@
     "newrelic-naming-rules": "./bin/test-naming-rules.js"
   },
   "dependencies": {
-    "@grpc/grpc-js": "^1.8.10",
+    "@grpc/grpc-js": "^1.9.4",
     "@grpc/proto-loader": "^0.7.5",
-    "@newrelic/aws-sdk": "^7.0.0",
-    "@newrelic/koa": "^8.0.0",
+    "@newrelic/aws-sdk": "^7.0.2",
+    "@newrelic/koa": "^8.0.1",
     "@newrelic/security-agent": "0.3.0",
-    "@newrelic/superagent": "^7.0.0",
+    "@newrelic/superagent": "^7.0.1",
     "@tyriar/fibonacci-heap": "^2.0.7",
     "concat-stream": "^2.0.0",
     "https-proxy-agent": "^7.0.1",