newrelic 6.13.2 → 6.14.0

package/NEWS.md

@@ -1,4 +1,30 @@
-## 6.13.2 (2020-10-13):
+### 6.14.0 (2020-10-28):
+
+* Updated README for consistency.
+
+* Fixed issue where gRPC connection used for infinite tracing could throw if the server
+  shutdown during disconnect of an existing connection.
+
+* Bumped @grpc/grpc-js to 1.1.7.
+
+* Bumped @grpc/proto-loader to ^0.5.5.
+
+* Infinite tracing logging and support metric improvements.
+
+  * Increased logging level of certain infinite tracing / gRPC errors.
+  * Decreased logging interval of dropped span warning for infinite tracing.
+  * Added additional support metrics and logging for infinite tracing.
+
+* Fixed bug where errors would still be collected for transactions with ignored error
+  status codes in certain situations.
+
+* Converted errors ignore unit tests to tap API.
+
+* Added Node 14 to CI test coverage.
+
+  Many thanks to @jgeurts for the contribution.
+
+### 6.13.2 (2020-10-13):
 
 * Removed lodash as a development dependency
 
@@ -10,7 +36,7 @@
 
   Thank you to @ronen-e for the contribution!
 
-## 6.13.1 (2020-09-24):
+### 6.13.1 (2020-09-24):
 
 * Fixed named-pipe check for lambda invocations to avoid race-condition.
 

package/README.md

@@ -10,7 +10,7 @@ In order to take full advantage of this package, make sure you have a [New Relic
 
 As with any instrumentation tool, please test before using in production.
 
-## Installation and getting started
+## Installation
 
 To use New Relic's Node.js agent entails these three steps, which are described in detail below:
 
@@ -57,9 +57,13 @@ If you're compiling your JavaScript and can't control the final `require` order,
     $ node -r newrelic your-program.js
     $ node --require newrelic your-program.js
 
+## Getting Started
+
 For more information on getting started, [check the Node.js docs](https://docs.newrelic.com/docs/agents/nodejs-agent/getting-started/introduction-new-relic-nodejs).
 
-## Using the API
+## Usage
+
+### Using the API
 
 The `newrelic` module returns an object with the Node agent's API methods attached.
 
@@ -72,7 +76,7 @@ The `newrelic` module returns an object with the Node agent's API methods attach
 
 You can read more about using the API over on the [New Relic documentation](https://docs.newrelic.com/docs/agents/nodejs-agent/api-guides/guide-using-nodejs-agent-api) site.
 
-## Core agent development and tests
+## Testing
 
 These are the steps to work on core agent features, with more detail below:
 
@@ -149,17 +153,22 @@ Please review [New Relic’s General Data Privacy Notice](https://newrelic.com/t
 
 See our [roadmap](./ROADMAP_Node.md), to learn more about our product vision, understand our plans, and provide us valuable feedback.
 
-## Contributing
+## Contribute
 
 We encourage your contributions to improve the Node.js agent! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.
+
 If you have any questions, or to execute our corporate CLA, required if your contribution is on behalf of a company,  please drop us an email at opensource@newrelic.com.
 
 **A note about vulnerabilities**
 
-As noted in our [security policy](https://github.com/newrelic/node-newrelic/security/policy), New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.
+As noted in our [security policy](../../security/policy), New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.
 
 If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through [HackerOne](https://hackerone.com/newrelic).
 
+If you would like to contribute to this project, review [these guidelines](./CONTRIBUTING.md).
+
+To [all contributors](https://github.com/newrelic/node-newrelic/graphs/contributors), we thank you!  Without your contribution, this project would not be what it is today.  We also host a community project page dedicated to [New Relic Node Agent](https://opensource.newrelic.com/projects/newrelic/node-newrelic).
+
 ## License
 
 The Node.js agent is licensed under the [Apache 2.0](http://apache.org/licenses/LICENSE-2.0.txt) License.

package/THIRD_PARTY_NOTICES.md

@@ -76,7 +76,7 @@ code, the source code can be found at [https://github.com/newrelic/node-newrelic
 
 ### @grpc/grpc-js
 
-This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.0.5](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.0.5)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.0.5/LICENSE):
+This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.1.7](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.1.7)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.1.7/LICENSE):
 
 ```
                                  Apache License
@@ -285,7 +285,7 @@ This product includes source derived from [@grpc/grpc-js](https://github.com/grp
 
 ### @grpc/proto-loader
 
-This product includes source derived from [@grpc/proto-loader](https://github.com/grpc/grpc-node) ([v0.5.4](https://github.com/grpc/grpc-node/tree/v0.5.4)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/blob/v0.5.4/LICENSE):
+This product includes source derived from [@grpc/proto-loader](https://github.com/grpc/grpc-node) ([v0.5.5](https://github.com/grpc/grpc-node/tree/v0.5.5)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/blob/v0.5.5/LICENSE):
 
 ```
                                  Apache License

package/lib/agent.js

@@ -581,6 +581,10 @@ Agent.prototype._beforeMetricDataSend = function _beforeMetricDataSend() {
 
   // Send uninstrumented supportability metrics every metric harvest cycle
   uninstrumented.createMetrics(this.metrics)
+
+  if (this.spanEventAggregator.isStream) {
+    this.spanEventAggregator.createMetrics()
+  }
 }
 
 Agent.prototype._generateEntityStatsAndClear = function _generateHarvestMetrics() {

package/lib/collector/remote-method.js

@@ -107,17 +107,23 @@ RemoteMethod.prototype._post = function _post(data, nrHeaders, callback) {
   }
 
   if (options.compressed) {
-    var useGzip = this._config.compressed_content_encoding === 'gzip'
-    var compressor = useGzip ? zlib.gzip : zlib.deflate
-    compressor(data, function onCompress(err, compressed) {
-      if (err) {
-        logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
-        return callback(err)
-      }
-
-      options.body = compressed
-      makeRequest()
-    })
+    // NOTE: gzip and deflate throw immediately in Node 14+ with an invalid argument
+    try {
+      var useGzip = this._config.compressed_content_encoding === 'gzip'
+      var compressor = useGzip ? zlib.gzip : zlib.deflate
+      compressor(data, function onCompress(err, compressed) {
+        if (err) {
+          logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
+          return callback(err)
+        }
+
+        options.body = compressed
+        makeRequest()
+      })
+    } catch (err) {
+      logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
+      return callback(err)
+    }
   } else {
     options.body = data
     makeRequest()

package/lib/errors/error-collector.js

@@ -142,13 +142,18 @@ class ErrorCollector {
     }
 
     const isErroredTransaction = urltils.isError(this.config, transaction.statusCode)
+    const isIgnoredErrorStatusCode = urltils.isIgnoredError(
+      this.config,
+      transaction.statusCode
+    )
+
     const isExpectedErrorStatusCode = urltils.isExpectedError(
       this.config,
       transaction.statusCode
     )
 
     // collect other exceptions only if status code is not ignored
-    if (transaction.exceptions.length) {
+    if (transaction.exceptions.length && !isIgnoredErrorStatusCode) {
       for (let i = 0; i < transaction.exceptions.length; i++) {
         const exception = transaction.exceptions[i]
         if (this.collect(transaction, exception)) {

package/lib/errors/index.js

@@ -20,7 +20,20 @@ class Exception {
   }
 
   getErrorDetails(config) {
-    return errorHelper.extractErrorInformation(null, this.error, config)
+    const errorDetails = errorHelper.extractErrorInformation(null, this.error, config)
+    errorDetails.expected = this.isExpected(config, errorDetails)
+
+    return errorDetails
+  }
+
+  isExpected(config, {type, message}) {
+    if (!this._expected) {
+      this._expected =
+        errorHelper.isExpectedErrorClass(config, type) ||
+        errorHelper.isExpectedErrorMessage(config, type, message)
+    }
+
+    return this._expected
   }
 }
 

package/lib/grpc/connection.js

@@ -121,7 +121,7 @@ class GrpcConnection extends EventEmitter {
       // immediately.
       this._setState(connectionStates.connected, this.stream)
     } catch (err) {
-      logger.trace(
+      logger.warn(
         err,
         'Unexpected error establishing gRPC stream, will not attempt reconnect.'
       )
@@ -198,9 +198,26 @@ class GrpcConnection extends EventEmitter {
     if (this.stream) {
       this.stream.removeAllListeners()
 
+      const oldStream = this.stream
+      this.stream.on('status', function endStreamStatusHandler(grpcStatus) {
+        logger.trace('End stream status received [%s]: %s', grpcStatus.code, grpcStatus.details)
+
+        // Cleanup the final end stream listeners.
+        oldStream.removeAllListeners()
+      })
+
+      // Listen to any final errors to prevent throwing.
+      // This is unlikely but if the server closes post
+      // removing listeners and prior to response it could
+      // happen. We noticed this via tests on Node 14.
+      this.stream.on('error', function endStreamErrorHandler(err) {
+        logger.trace('End stream error received. Code: [%s]: %s', err.code, err.details)
+      })
+
       // Indicates to server we are done.
       // Server officially closes the stream.
       this.stream.end()
+
       this.stream = null
     }
 
@@ -238,6 +255,9 @@ class GrpcConnection extends EventEmitter {
         // per the spec, An UNIMPLEMENTED status code from gRPC indicates
         // that the versioned Trace Observer is no longer available. Agents
         // MUST NOT attempt to reconnect in this case
+        logger.info(
+          '[UNIMPLEMENTED]: Trace Obserserver is no longer available. Shutting down connection.'
+        )
         this._disconnect()
       } else if (grpc.status[grpc.status.OK] === grpcStatusName) {
         this._reconnect()
@@ -256,7 +276,11 @@ class GrpcConnection extends EventEmitter {
       this._metrics.getOrCreateMetric(NAMES.INFINITE_TRACING.SPAN_RESPONSE_ERROR)
         .incrementCallCount()
 
-      logger.trace('span stream error. Code: [%s]: %s',err.code, err.details)
+      // For errors, the status will either result in a disconnect or a reconnect
+      // delay that should prevent too frequent spamming. Unless the app is idle
+      // and regularly getting Status 13 reconnects from the server, in which case
+      // this will be almost the only logging.
+      logger.warn('Span stream error. Code: [%s]: %s', err.code, err.details)
     })
   }
 

package/lib/metrics/names.js

@@ -240,10 +240,13 @@ const SPAN_EVENTS = {
 const INFINITE_TRACING = {
   SEEN: SUPPORTABILITY.INFINITE_TRACING + '/Span/Seen',
   SENT: SUPPORTABILITY.INFINITE_TRACING + '/Span/Sent',
+  DROPPED: SUPPORTABILITY.INFINITE_TRACING + '/Span/Dropped',
   SPAN_RESPONSE_ERROR: SUPPORTABILITY.INFINITE_TRACING + '/Span/Response/Error',
   SPAN_RESPONSE_GRPC_UNIMPLEMENTED: SUPPORTABILITY.INFINITE_TRACING + '/Span/gRPC/UNIMPLEMENTED',
   SPAN_RESPONSE_GRPC_STATUS: SUPPORTABILITY.INFINITE_TRACING + '/Span/gRPC/%s',
-  DRAIN_DURATION: SUPPORTABILITY.INFINITE_TRACING + '/Drain/Duration'
+  QUEUE_CAPACITY: SUPPORTABILITY.INFINITE_TRACING + '/Span/QueueCapacity',
+  QUEUE_SIZE: SUPPORTABILITY.INFINITE_TRACING + '/Span/QueueSize',
+  DRAIN_DURATION: SUPPORTABILITY.INFINITE_TRACING + '/Drain/Duration',
 }
 
 module.exports = {

package/lib/spans/span-context.js

@@ -21,6 +21,9 @@ class SpanContext {
       customAttributes || new PrioritizedAttributes(ATTRIBUTE_SCOPE, MAXIMUM_CUSTOM_ATTRIBUTES)
 
     this.ATTRIBUTE_PRIORITY = ATTRIBUTE_PRIORITY
+
+    this.hasError = false
+    this.errorDetails = null
   }
 
   addIntrinsicAttribute(key, value) {
@@ -36,6 +39,21 @@ class SpanContext {
       priority
     )
   }
+
+  /**
+   * Set error details to be potentially be used to create span
+   * attributes. Attributes will be created unless the transaction
+   * ends with an ignored error status code.
+   *
+   * Last error wins.
+   */
+  setError(details) {
+    this.hasError = true
+
+    // Error details will be used to create attributes unless the transaction ends
+    // with an ignored status code.
+    this.errorDetails = details
+  }
 }
 
 module.exports = SpanContext

package/lib/spans/span-event.js

@@ -84,9 +84,21 @@ class SpanEvent {
    * @return {SpanEvent} The constructed event.
    */
   static fromSegment(segment, parentId = null, isRoot = false) {
+    const spanContext = segment.getSpanContext()
+
+    // Since segments already hold span agent attributes and we want to leverage
+    // filtering, we add to the segment attributes prior to processing.
+    if (spanContext.hasError && !segment.transaction.hasIgnoredErrorStatusCode()) {
+      const details = spanContext.errorDetails
+      segment.addSpanAttribute('error.message', details.message)
+      segment.addSpanAttribute('error.class', details.type)
+      if (details.expected) {
+        segment.addSpanAttribute('error.expected', details.expected)
+      }
+    }
+
     const attributes = segment.attributes.get(DESTINATIONS.SPAN_EVENT)
 
-    const spanContext = segment.getSpanContext()
     const customAttributes = spanContext.customAttributes.get(DESTINATIONS.SPAN_EVENT)
 
     let span = null
@@ -98,7 +110,6 @@ class SpanEvent {
       span = new SpanEvent(attributes, customAttributes)
     }
 
-
     for (const [key, value] of Object.entries(spanContext.intrinsicAttributes)) {
       span.intrinsics[key] = value
     }

package/lib/spans/span-streamer.js

@@ -8,10 +8,10 @@
 const logger = require('../logger').child({component: 'span-streamer'})
 const NAMES = require('../metrics/names').INFINITE_TRACING
 
+const SPAN_DROP_MSG_INTERVAL_MS = 30000
 const SPAN_DROP_MSG =
-  'Queue full, dropping spans.'
-const SPAN_DROP_INTERVAL = 5
-
+  'Queue full, dropping spans. ' +
+  `Will not warn again for ${SPAN_DROP_MSG_INTERVAL_MS / 1000} seconds.`
 
 class SpanStreamer {
   constructor(license_key, connection, metrics, queue_size) {
@@ -47,21 +47,20 @@ class SpanStreamer {
 
     // If not writeable (because of backpressure) queue the span
     if (!this._writable) {
-      if (this.spans.length <= this.queue_size) {
+      if (this.spans.length < this.queue_size) {
         this.spans.push(span)
         return
       }
 
+      // While this can be directionally calculated between seen/sent the
+      // queue makes that a bit more disconnected. This will be a bit more specific.
+      this._metrics.getOrCreateMetric(NAMES.DROPPED).incrementCallCount()
+
       // If the queue is full drop the span
       logger.infoOncePer(
-        // key for the OncePer
-        'SPAN_DROP_MSG',
-        // interval in ms
-        SPAN_DROP_INTERVAL * 1000,
-        // message
-        SPAN_DROP_MSG + ' Will not warn again for %s seconds.',
-        // variables to put in log message
-        SPAN_DROP_INTERVAL
+        'SPAN_DROP_MSG', // key for the OncePer
+        SPAN_DROP_MSG_INTERVAL_MS,
+        SPAN_DROP_MSG
       )
 
       return
@@ -121,6 +120,8 @@ class SpanStreamer {
   }
 
   sendQueue() {
+    logger.trace('Sending spans from queue.')
+
     // Continue sending the spans that were in the queue. _writable is checked
     // so that if a send fails while clearing the queue, this drain handler can
     // finish, and the drain handler setup on the failed send will then attempt
@@ -129,6 +130,11 @@ class SpanStreamer {
       const nextObject = this.spans.shift()
       this.send(nextObject)
     }
+
+    logger.trace(
+      'Finished sending spans from queue. Items left in queue: %s',
+      this.spans.length
+    )
   }
 
   connect(agent_run_id, requestHeadersMap) {
@@ -144,6 +150,11 @@ class SpanStreamer {
   disconnect() {
     this.connection.disconnect()
   }
+
+  createMetrics() {
+    this._metrics.getOrCreateMetric(NAMES.QUEUE_CAPACITY).recordValue(this.queue_size)
+    this._metrics.getOrCreateMetric(NAMES.QUEUE_SIZE).recordValue(this.spans.length)
+  }
 }
 
 module.exports = SpanStreamer

package/lib/spans/streaming-span-event-aggregator.js

@@ -98,6 +98,10 @@ class StreamingSpanEventAggregator extends Aggregator {
 
     this.requestHeadersMap = config.request_headers_map
   }
+
+  createMetrics() {
+    this.stream.createMetrics()
+  }
 }
 
 module.exports = StreamingSpanEventAggregator

package/lib/spans/streaming-span-event.js

@@ -102,9 +102,21 @@ class StreamingSpanEvent {
   }
 
   static fromSegment(segment, parentId = null, isRoot = false) {
+    const spanContext = segment.getSpanContext()
+
+    // Since segments already hold span agent attributes and we want to leverage
+    // filtering, we add to the segment attributes prior to processing.
+    if (spanContext.hasError && !segment.transaction.hasIgnoredErrorStatusCode()) {
+      const details = spanContext.errorDetails
+      segment.addSpanAttribute('error.message', details.message)
+      segment.addSpanAttribute('error.class', details.type)
+      if (details.expected) {
+        segment.addSpanAttribute('error.expected', details.expected)
+      }
+    }
+
     const agentAttributes = segment.attributes.get(DESTINATIONS.SPAN_EVENT)
 
-    const spanContext = segment.getSpanContext()
     const customAttributes = spanContext.customAttributes.get(DESTINATIONS.SPAN_EVENT)
 
     const transaction = segment.transaction

package/lib/transaction/index.js

@@ -679,7 +679,6 @@ Transaction.prototype.measure = function measure(name, scope, duration, exclusiv
 Transaction.prototype._setApdex = function _setApdex(name, duration, keyApdexInMillis) {
   var apdexStats = this.metrics.getOrCreateApdexMetric(name, null, keyApdexInMillis)
 
-
   // if we have an error-like status code, and all the errors are
   // expected, we know the status code was caused by an expected
   // error, so we will not report "frustrating".  Otherwise, we
@@ -747,19 +746,13 @@ function _linkExceptionToSegment(exception) {
     return
   }
 
-  const config = this.agent.config
-
-  // Add error attributes to the span
-  const details = exception.getErrorDetails(config)
-  segment.addSpanAttribute('error.message', details.message)
-  segment.addSpanAttribute('error.class', details.type)
-
-  const isExpected =
-    errorHelper.isExpectedErrorClass(config, details.type) ||
-    errorHelper.isExpectedErrorMessage(config, details.type, details.message)
-
-  if (isExpected) {
-    segment.addSpanAttribute('error.expected', isExpected)
+  const spanContext = segment.getSpanContext()
+  if (spanContext) {
+    // Exception attributes will be added to span unless transaction
+    // status code has been ignored. Last error wins.
+    const config = this.agent.config
+    const details = exception.getErrorDetails(config)
+    spanContext.setError(details)
   }
 
   // Add the span/segment ID to the exception as agent attributes
@@ -793,6 +786,14 @@ function _addUserError(exception) {
   this.userErrors.push(exception)
 }
 
+/**
+ * Returns if the transaction's current status code is errored
+ * but considered ignored via the config.
+ */
+Transaction.prototype.hasIgnoredErrorStatusCode = function _hasIgnoredErrorStatusCode() {
+  return urltils.isIgnoredError(this.agent.config, this.statusCode)
+}
+
 /**
  * Returns true if an error happened during the transaction or if the transaction itself is
  * considered to be an error.

package/newrelic.js

@@ -28,7 +28,7 @@ exports.config = {
      *
      * @env NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
      */
-    enabled: false,
+    enabled: false
   },
   logging: {
     /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "newrelic",
-  "version": "6.13.2",
+  "version": "6.14.0",
   "author": "New Relic Node.js agent team <nodejs@newrelic.com>",
   "license": "Apache-2.0",
   "contributors": [
@@ -142,8 +142,8 @@
     "newrelic-naming-rules": "./bin/test-naming-rules.js"
   },
   "dependencies": {
-    "@grpc/grpc-js": "1.0.5",
-    "@grpc/proto-loader": "^0.5.4",
+    "@grpc/grpc-js": "1.1.7",
+    "@grpc/proto-loader": "^0.5.5",
     "@newrelic/aws-sdk": "^2.0.0",
     "@newrelic/koa": "^4.0.0",
     "@newrelic/superagent": "^3.0.0",

package/third_party_manifest.json

@@ -1,30 +1,30 @@
 {
-  "lastUpdated": "Tue Oct 13 2020 14:24:25 GMT-0700 (Pacific Daylight Time)",
+  "lastUpdated": "Wed Oct 28 2020 11:06:05 GMT-0700 (Pacific Daylight Time)",
   "projectName": "New Relic Node Agent",
   "projectUrl": "https://github.com/newrelic/node-newrelic",
   "includeDev": true,
   "dependencies": {
-    "@grpc/grpc-js@1.0.5": {
+    "@grpc/grpc-js@1.1.7": {
       "name": "@grpc/grpc-js",
-      "version": "1.0.5",
-      "range": "1.0.5",
+      "version": "1.1.7",
+      "range": "1.1.7",
       "licenses": "Apache-2.0",
       "repoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js",
-      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.0.5",
+      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.1.7",
       "licenseFile": "node_modules/@grpc/grpc-js/LICENSE",
-      "licenseUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.0.5/LICENSE",
+      "licenseUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.1.7/LICENSE",
       "licenseTextSource": "file",
       "publisher": "Google Inc."
     },
-    "@grpc/proto-loader@0.5.4": {
+    "@grpc/proto-loader@0.5.5": {
       "name": "@grpc/proto-loader",
-      "version": "0.5.4",
-      "range": "^0.5.4",
+      "version": "0.5.5",
+      "range": "^0.5.5",
       "licenses": "Apache-2.0",
       "repoUrl": "https://github.com/grpc/grpc-node",
-      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/v0.5.4",
+      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/v0.5.5",
       "licenseFile": "node_modules/@grpc/proto-loader/LICENSE",
-      "licenseUrl": "https://github.com/grpc/grpc-node/blob/v0.5.4/LICENSE",
+      "licenseUrl": "https://github.com/grpc/grpc-node/blob/v0.5.5/LICENSE",
       "licenseTextSource": "file",
       "publisher": "Google Inc."
     },