newrelic 6.12.1 → 6.14.0

package/CONTRIBUTING.md

@@ -20,9 +20,9 @@ licensing restrictions.
 
 ### Code of Conduct
 
-Before contributing please read the [code of conduct](./CODE_OF_CONDUCT.md)
+Before contributing please read the [code of conduct](https://github.com/newrelic/.github/blob/main/CODE_OF_CONDUCT.md)
 
-Note that our [code of conduct](./CODE_OF_CONDUCT.md) applies to all platforms
+Note that our [code of conduct](https://github.com/newrelic/.github/blob/main/CODE_OF_CONDUCT.md) applies to all platforms
 and venues related to this project; please follow it in all your interactions
 with the project and its participants.
 
@@ -53,7 +53,7 @@ When contributing, keep in mind that New Relic customers (that's you!) are runni
 
 Be aware that the instrumentation needs to work with a wide range of versions of the instrumented modules, and that code that looks nonsensical or overcomplicated may be that way for compatibility-related reasons. Read all the comments and check the related tests before deciding whether existing code is incorrect.
 
-If you’re planning on contributing a new feature or an otherwise complex contribution, we kindly ask you to start a conversation with the maintainer team by opening up an Github issue first. 
+If you’re planning on contributing a new feature or an otherwise complex contribution, we kindly ask you to start a conversation with the maintainer team by opening up an Github issue first.
 
 ### General Guidelines
 

package/NEWS.md

@@ -1,3 +1,59 @@
+### 6.14.0 (2020-10-28):
+
+* Updated README for consistency.
+
+* Fixed issue where gRPC connection used for infinite tracing could throw if the server
+  shutdown during disconnect of an existing connection.
+
+* Bumped @grpc/grpc-js to 1.1.7.
+
+* Bumped @grpc/proto-loader to ^0.5.5.
+
+* Infinite tracing logging and support metric improvements.
+
+  * Increased logging level of certain infinite tracing / gRPC errors.
+  * Decreased logging interval of dropped span warning for infinite tracing.
+  * Added additional support metrics and logging for infinite tracing.
+
+* Fixed bug where errors would still be collected for transactions with ignored error
+  status codes in certain situations.
+
+* Converted errors ignore unit tests to tap API.
+
+* Added Node 14 to CI test coverage.
+
+  Many thanks to @jgeurts for the contribution.
+
+### 6.13.2 (2020-10-13):
+
+* Removed lodash as a development dependency
+
+* Check for named pipe existence before each flush
+
+  This removes the cached value used in 6.13.1
+
+* Update shim documentation
+
+  Thank you to @ronen-e for the contribution!
+
+### 6.13.1 (2020-09-24):
+
+* Fixed named-pipe check for lambda invocations to avoid race-condition.
+
+  Named-pipe existence will now be checked just prior to first write and then cached.
+
+* Updated README with community-plus header.
+
+* Updated README config copy example.
+
+* Added Open Source Policy workflow.
+
+* Removed repository CoC in favor of centralized CoC at org root.
+
+### 6.13.0 (2020-08-25):
+
+* Added ability for the agent to write to a named pipe, instead of stdout, when in serverless mode.
+
 ### 6.12.1 (2020-08-20):
 
 * **Security fix:** Resolves an issue where transaction traces will still capture the request URI when the Node.js agent is configured to exclude the 'request.uri' attribute. This can be problematic for certain customers in environments where sensitive information is included in the URI. See security bulletin [NR20-02](https://docs.newrelic.com/docs/security/new-relic-security/security-bulletins/security-bulletin-nr20-02).

package/README.md

@@ -1,4 +1,4 @@
-[![Community Project header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Project.png)](https://opensource.newrelic.com/oss-category/#community-project)
+[![Community Plus header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Plus.png)](https://opensource.newrelic.com/oss-category/#community-plus)
 
 # New Relic's Node.js agent [![Server Smoke Tests][3]][4] [![Node Agent CI][5]][6]
 
@@ -10,7 +10,7 @@ In order to take full advantage of this package, make sure you have a [New Relic
 
 As with any instrumentation tool, please test before using in production.
 
-## Installation and getting started
+## Installation
 
 To use New Relic's Node.js agent entails these three steps, which are described in detail below:
 
@@ -20,11 +20,11 @@ To use New Relic's Node.js agent entails these three steps, which are described
 
 1. To install the agent for performance monitoring, use your favorite npm-based package manager and install the `newrelic` package into your application:
 
-    $ npm install newrelic
+    `$ npm install newrelic`
 
 2. Then, copy the stock configuration file to your program's base folder:
 
-    $ cp node_modules/newrelic/newrelic.js
+    `$ cp ./node_modules/newrelic/newrelic.js ./<your destination>`
 
 3. Now, add your New Relic license key and application/service name to that file:
 
@@ -57,9 +57,13 @@ If you're compiling your JavaScript and can't control the final `require` order,
     $ node -r newrelic your-program.js
     $ node --require newrelic your-program.js
 
+## Getting Started
+
 For more information on getting started, [check the Node.js docs](https://docs.newrelic.com/docs/agents/nodejs-agent/getting-started/introduction-new-relic-nodejs).
 
-## Using the API
+## Usage
+
+### Using the API
 
 The `newrelic` module returns an object with the Node agent's API methods attached.
 
@@ -72,7 +76,7 @@ The `newrelic` module returns an object with the Node agent's API methods attach
 
 You can read more about using the API over on the [New Relic documentation](https://docs.newrelic.com/docs/agents/nodejs-agent/api-guides/guide-using-nodejs-agent-api) site.
 
-## Core agent development and tests
+## Testing
 
 These are the steps to work on core agent features, with more detail below:
 
@@ -149,17 +153,22 @@ Please review [New Relic’s General Data Privacy Notice](https://newrelic.com/t
 
 See our [roadmap](./ROADMAP_Node.md), to learn more about our product vision, understand our plans, and provide us valuable feedback.
 
-## Contributing
+## Contribute
 
 We encourage your contributions to improve the Node.js agent! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.
+
 If you have any questions, or to execute our corporate CLA, required if your contribution is on behalf of a company,  please drop us an email at opensource@newrelic.com.
 
 **A note about vulnerabilities**
 
-As noted in our [security policy](https://github.com/newrelic/node-newrelic/security/policy), New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.
+As noted in our [security policy](../../security/policy), New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.
 
 If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through [HackerOne](https://hackerone.com/newrelic).
 
+If you would like to contribute to this project, review [these guidelines](./CONTRIBUTING.md).
+
+To [all contributors](https://github.com/newrelic/node-newrelic/graphs/contributors), we thank you!  Without your contribution, this project would not be what it is today.  We also host a community project page dedicated to [New Relic Node Agent](https://opensource.newrelic.com/projects/newrelic/node-newrelic).
+
 ## License
 
 The Node.js agent is licensed under the [Apache 2.0](http://apache.org/licenses/LICENSE-2.0.txt) License.
@@ -172,4 +181,4 @@ The Node.js agent also uses source code from third-party libraries. You can find
 [3]: https://github.com/newrelic/node-newrelic/workflows/Server%20Smoke%20Tests/badge.svg
 [4]: https://github.com/newrelic/node-newrelic/actions?query=workflow%3A%22Server+Smoke+Tests%22
 [5]: https://github.com/newrelic/node-newrelic/workflows/Node%20Agent%20CI/badge.svg
-[6]: https://github.com/newrelic/node-newrelic/actions?query=workflow%3A%22Node+Agent+CI%22
+[6]: https://github.com/newrelic/node-newrelic/actions?query=workflow%3A%22Node+Agent+CI%22

package/THIRD_PARTY_NOTICES.md

@@ -48,7 +48,6 @@ code, the source code can be found at [https://github.com/newrelic/node-newrelic
 * [got](#got)
 * [http-errors](#http-errors)
 * [jsdoc](#jsdoc)
-* [lodash](#lodash)
 * [memcached](#memcached)
 * [minami](#minami)
 * [mongodb](#mongodb)
@@ -77,7 +76,7 @@ code, the source code can be found at [https://github.com/newrelic/node-newrelic
 
 ### @grpc/grpc-js
 
-This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.0.5](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.0.5)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.0.5/LICENSE):
+This product includes source derived from [@grpc/grpc-js](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js) ([v1.1.7](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.1.7)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.1.7/LICENSE):
 
 ```
                                  Apache License
@@ -286,7 +285,7 @@ This product includes source derived from [@grpc/grpc-js](https://github.com/grp
 
 ### @grpc/proto-loader
 
-This product includes source derived from [@grpc/proto-loader](https://github.com/grpc/grpc-node) ([v0.5.4](https://github.com/grpc/grpc-node/tree/v0.5.4)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/blob/v0.5.4/LICENSE):
+This product includes source derived from [@grpc/proto-loader](https://github.com/grpc/grpc-node) ([v0.5.5](https://github.com/grpc/grpc-node/tree/v0.5.5)), distributed under the [Apache-2.0 License](https://github.com/grpc/grpc-node/blob/v0.5.5/LICENSE):
 
 ```
                                  Apache License
@@ -2197,61 +2196,6 @@ https://github.com/jmblog/color-themes-for-google-code-prettify
 
 ```
 
-### lodash
-
-This product includes source derived from [lodash](https://github.com/lodash/lodash) ([v4.17.15](https://github.com/lodash/lodash/tree/v4.17.15)), distributed under the [MIT License](https://github.com/lodash/lodash/blob/v4.17.15/LICENSE):
-
-```
-Copyright OpenJS Foundation and other contributors <https://openjsf.org/>
-
-Based on Underscore.js, copyright Jeremy Ashkenas,
-DocumentCloud and Investigative Reporters & Editors <http://underscorejs.org/>
-
-This software consists of voluntary contributions made by many
-individuals. For exact contribution history, see the revision history
-available at https://github.com/lodash/lodash
-
-The following license applies to all parts of this software except as
-documented below:
-
-====
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-====
-
-Copyright and related rights for sample code are waived via CC0. Sample
-code is defined as all source code displayed within the prose of the
-documentation.
-
-CC0: http://creativecommons.org/publicdomain/zero/1.0/
-
-====
-
-Files located in the node_modules and vendor directories are externally
-maintained libraries used by this software which have their own
-licenses; we recommend you read them, as their terms may differ from the
-terms above.
-
-```
-
 ### memcached
 
 This product includes source derived from [memcached](https://github.com/3rd-Eden/node-memcached) ([v2.2.2](https://github.com/3rd-Eden/node-memcached/tree/v2.2.2)), distributed under the [MIT License](https://github.com/3rd-Eden/node-memcached/blob/v2.2.2/LICENSE):

package/lib/agent.js

@@ -581,6 +581,10 @@ Agent.prototype._beforeMetricDataSend = function _beforeMetricDataSend() {
 
   // Send uninstrumented supportability metrics every metric harvest cycle
   uninstrumented.createMetrics(this.metrics)
+
+  if (this.spanEventAggregator.isStream) {
+    this.spanEventAggregator.createMetrics()
+  }
 }
 
 Agent.prototype._generateEntityStatsAndClear = function _generateHarvestMetrics() {

package/lib/collector/remote-method.js

@@ -107,17 +107,23 @@ RemoteMethod.prototype._post = function _post(data, nrHeaders, callback) {
   }
 
   if (options.compressed) {
-    var useGzip = this._config.compressed_content_encoding === 'gzip'
-    var compressor = useGzip ? zlib.gzip : zlib.deflate
-    compressor(data, function onCompress(err, compressed) {
-      if (err) {
-        logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
-        return callback(err)
-      }
-
-      options.body = compressed
-      makeRequest()
-    })
+    // NOTE: gzip and deflate throw immediately in Node 14+ with an invalid argument
+    try {
+      var useGzip = this._config.compressed_content_encoding === 'gzip'
+      var compressor = useGzip ? zlib.gzip : zlib.deflate
+      compressor(data, function onCompress(err, compressed) {
+        if (err) {
+          logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
+          return callback(err)
+        }
+
+        options.body = compressed
+        makeRequest()
+      })
+    } catch (err) {
+      logger.warn(err, 'Error compressing JSON for delivery. Not sending.')
+      return callback(err)
+    }
   } else {
     options.body = data
     makeRequest()

package/lib/collector/serverless.js

@@ -14,6 +14,9 @@ const stringify = require('json-stringify-safe')
 const PAYLOAD_VERSION = 1
 const PAYLOAD_MARKER = 'NR_LAMBDA_MONITORING'
 
+const path = require('path')
+const defaultPipePath = path.resolve('/tmp', 'newrelic-telemetry')
+
 class ServerlessCollector {
   /**
    * Constructs a new serverless collector instance with the give agent.
@@ -24,7 +27,7 @@ class ServerlessCollector {
    *
    * @param {Agent} agent - The agent this collector will use
    */
-  constructor(agent) {
+  constructor(agent, pipePath) {
     this._agent = agent
     this.enabled = true
     this.metadata = {
@@ -35,6 +38,7 @@ class ServerlessCollector {
       agent_version: agent.version
     }
     this.payload = {}
+    this.pipePath = pipePath || process.env.NEWRELIC_PIPE_PATH || defaultPipePath
   }
 
   /**
@@ -243,6 +247,36 @@ class ServerlessCollector {
     }
   }
 
+  /**
+   * Writes payload to pipe
+   */
+  flushToPipeSync(payload) {
+    try {
+      fs.writeFileSync(this.pipePath, payload)
+      return true
+    } catch (e) {
+      logger.warn('Error attempting to write to pipe, falling back to stdout', e)
+      return false
+    }
+  }
+
+  flushToStdOut(serializedPayload, payloadLength, sync = false) {
+    if (sync) {
+      // Long log lines have been truncated at 65538
+      // Guarantees process.stdout will block, so long logs
+      // won't be truncated if process.exit() is called early.
+      const s = process.stdout
+      payloadLength > 65000 &&
+      s._handle &&
+      s._handle.setBlocking &&
+      s._handle.setBlocking(true)
+
+      fs.writeSync(process.stdout.fd, serializedPayload)
+    } else {
+      process.stdout.write(serializedPayload)
+    }
+  }
+
   /**
    * Internal method to handle flushing to stdout.
    *
@@ -258,19 +292,10 @@ class ServerlessCollector {
       payload
     ]) + '\n'
 
-    if (sync) {
-      // Long log lines have been truncated at 65538
-      // Guarantees process.stdout will block, so long logs
-      // won't be truncated if process.exit() is called early.
-      const s = process.stdout
-      payload.length > 65000 &&
-      s._handle &&
-      s._handle.setBlocking &&
-      s._handle.setBlocking(true)
+    const didUsePipe = fs.existsSync(this.pipePath) && this.flushToPipeSync(serializedPayload)
 
-      fs.writeSync(process.stdout.fd, serializedPayload)
-    } else {
-      process.stdout.write(serializedPayload)
+    if (!didUsePipe) {
+      this.flushToStdOut(serializedPayload, payload.length, sync)
     }
   }
 }

package/lib/errors/error-collector.js

@@ -142,13 +142,18 @@ class ErrorCollector {
     }
 
     const isErroredTransaction = urltils.isError(this.config, transaction.statusCode)
+    const isIgnoredErrorStatusCode = urltils.isIgnoredError(
+      this.config,
+      transaction.statusCode
+    )
+
     const isExpectedErrorStatusCode = urltils.isExpectedError(
       this.config,
       transaction.statusCode
     )
 
     // collect other exceptions only if status code is not ignored
-    if (transaction.exceptions.length) {
+    if (transaction.exceptions.length && !isIgnoredErrorStatusCode) {
       for (let i = 0; i < transaction.exceptions.length; i++) {
         const exception = transaction.exceptions[i]
         if (this.collect(transaction, exception)) {

package/lib/errors/index.js

@@ -20,7 +20,20 @@ class Exception {
   }
 
   getErrorDetails(config) {
-    return errorHelper.extractErrorInformation(null, this.error, config)
+    const errorDetails = errorHelper.extractErrorInformation(null, this.error, config)
+    errorDetails.expected = this.isExpected(config, errorDetails)
+
+    return errorDetails
+  }
+
+  isExpected(config, {type, message}) {
+    if (!this._expected) {
+      this._expected =
+        errorHelper.isExpectedErrorClass(config, type) ||
+        errorHelper.isExpectedErrorMessage(config, type, message)
+    }
+
+    return this._expected
   }
 }
 

package/lib/grpc/connection.js

@@ -121,7 +121,7 @@ class GrpcConnection extends EventEmitter {
       // immediately.
       this._setState(connectionStates.connected, this.stream)
     } catch (err) {
-      logger.trace(
+      logger.warn(
         err,
         'Unexpected error establishing gRPC stream, will not attempt reconnect.'
       )
@@ -198,9 +198,26 @@ class GrpcConnection extends EventEmitter {
     if (this.stream) {
       this.stream.removeAllListeners()
 
+      const oldStream = this.stream
+      this.stream.on('status', function endStreamStatusHandler(grpcStatus) {
+        logger.trace('End stream status received [%s]: %s', grpcStatus.code, grpcStatus.details)
+
+        // Cleanup the final end stream listeners.
+        oldStream.removeAllListeners()
+      })
+
+      // Listen to any final errors to prevent throwing.
+      // This is unlikely but if the server closes post
+      // removing listeners and prior to response it could
+      // happen. We noticed this via tests on Node 14.
+      this.stream.on('error', function endStreamErrorHandler(err) {
+        logger.trace('End stream error received. Code: [%s]: %s', err.code, err.details)
+      })
+
       // Indicates to server we are done.
       // Server officially closes the stream.
       this.stream.end()
+
       this.stream = null
     }
 
@@ -238,6 +255,9 @@ class GrpcConnection extends EventEmitter {
         // per the spec, An UNIMPLEMENTED status code from gRPC indicates
         // that the versioned Trace Observer is no longer available. Agents
         // MUST NOT attempt to reconnect in this case
+        logger.info(
+          '[UNIMPLEMENTED]: Trace Obserserver is no longer available. Shutting down connection.'
+        )
         this._disconnect()
       } else if (grpc.status[grpc.status.OK] === grpcStatusName) {
         this._reconnect()
@@ -256,7 +276,11 @@ class GrpcConnection extends EventEmitter {
       this._metrics.getOrCreateMetric(NAMES.INFINITE_TRACING.SPAN_RESPONSE_ERROR)
         .incrementCallCount()
 
-      logger.trace('span stream error. Code: [%s]: %s',err.code, err.details)
+      // For errors, the status will either result in a disconnect or a reconnect
+      // delay that should prevent too frequent spamming. Unless the app is idle
+      // and regularly getting Status 13 reconnects from the server, in which case
+      // this will be almost the only logging.
+      logger.warn('Span stream error. Code: [%s]: %s', err.code, err.details)
     })
   }
 

package/lib/metrics/names.js

@@ -240,10 +240,13 @@ const SPAN_EVENTS = {
 const INFINITE_TRACING = {
   SEEN: SUPPORTABILITY.INFINITE_TRACING + '/Span/Seen',
   SENT: SUPPORTABILITY.INFINITE_TRACING + '/Span/Sent',
+  DROPPED: SUPPORTABILITY.INFINITE_TRACING + '/Span/Dropped',
   SPAN_RESPONSE_ERROR: SUPPORTABILITY.INFINITE_TRACING + '/Span/Response/Error',
   SPAN_RESPONSE_GRPC_UNIMPLEMENTED: SUPPORTABILITY.INFINITE_TRACING + '/Span/gRPC/UNIMPLEMENTED',
   SPAN_RESPONSE_GRPC_STATUS: SUPPORTABILITY.INFINITE_TRACING + '/Span/gRPC/%s',
-  DRAIN_DURATION: SUPPORTABILITY.INFINITE_TRACING + '/Drain/Duration'
+  QUEUE_CAPACITY: SUPPORTABILITY.INFINITE_TRACING + '/Span/QueueCapacity',
+  QUEUE_SIZE: SUPPORTABILITY.INFINITE_TRACING + '/Span/QueueSize',
+  DRAIN_DURATION: SUPPORTABILITY.INFINITE_TRACING + '/Drain/Duration',
 }
 
 module.exports = {

package/lib/spans/span-context.js

@@ -21,6 +21,9 @@ class SpanContext {
       customAttributes || new PrioritizedAttributes(ATTRIBUTE_SCOPE, MAXIMUM_CUSTOM_ATTRIBUTES)
 
     this.ATTRIBUTE_PRIORITY = ATTRIBUTE_PRIORITY
+
+    this.hasError = false
+    this.errorDetails = null
   }
 
   addIntrinsicAttribute(key, value) {
@@ -36,6 +39,21 @@ class SpanContext {
       priority
     )
   }
+
+  /**
+   * Set error details to be potentially be used to create span
+   * attributes. Attributes will be created unless the transaction
+   * ends with an ignored error status code.
+   *
+   * Last error wins.
+   */
+  setError(details) {
+    this.hasError = true
+
+    // Error details will be used to create attributes unless the transaction ends
+    // with an ignored status code.
+    this.errorDetails = details
+  }
 }
 
 module.exports = SpanContext

package/lib/spans/span-event.js

@@ -84,9 +84,21 @@ class SpanEvent {
    * @return {SpanEvent} The constructed event.
    */
   static fromSegment(segment, parentId = null, isRoot = false) {
+    const spanContext = segment.getSpanContext()
+
+    // Since segments already hold span agent attributes and we want to leverage
+    // filtering, we add to the segment attributes prior to processing.
+    if (spanContext.hasError && !segment.transaction.hasIgnoredErrorStatusCode()) {
+      const details = spanContext.errorDetails
+      segment.addSpanAttribute('error.message', details.message)
+      segment.addSpanAttribute('error.class', details.type)
+      if (details.expected) {
+        segment.addSpanAttribute('error.expected', details.expected)
+      }
+    }
+
     const attributes = segment.attributes.get(DESTINATIONS.SPAN_EVENT)
 
-    const spanContext = segment.getSpanContext()
     const customAttributes = spanContext.customAttributes.get(DESTINATIONS.SPAN_EVENT)
 
     let span = null
@@ -98,7 +110,6 @@ class SpanEvent {
       span = new SpanEvent(attributes, customAttributes)
     }
 
-
     for (const [key, value] of Object.entries(spanContext.intrinsicAttributes)) {
       span.intrinsics[key] = value
     }

package/lib/spans/span-streamer.js

@@ -8,10 +8,10 @@
 const logger = require('../logger').child({component: 'span-streamer'})
 const NAMES = require('../metrics/names').INFINITE_TRACING
 
+const SPAN_DROP_MSG_INTERVAL_MS = 30000
 const SPAN_DROP_MSG =
-  'Queue full, dropping spans.'
-const SPAN_DROP_INTERVAL = 5
-
+  'Queue full, dropping spans. ' +
+  `Will not warn again for ${SPAN_DROP_MSG_INTERVAL_MS / 1000} seconds.`
 
 class SpanStreamer {
   constructor(license_key, connection, metrics, queue_size) {
@@ -47,21 +47,20 @@ class SpanStreamer {
 
     // If not writeable (because of backpressure) queue the span
     if (!this._writable) {
-      if (this.spans.length <= this.queue_size) {
+      if (this.spans.length < this.queue_size) {
         this.spans.push(span)
         return
       }
 
+      // While this can be directionally calculated between seen/sent the
+      // queue makes that a bit more disconnected. This will be a bit more specific.
+      this._metrics.getOrCreateMetric(NAMES.DROPPED).incrementCallCount()
+
       // If the queue is full drop the span
       logger.infoOncePer(
-        // key for the OncePer
-        'SPAN_DROP_MSG',
-        // interval in ms
-        SPAN_DROP_INTERVAL * 1000,
-        // message
-        SPAN_DROP_MSG + ' Will not warn again for %s seconds.',
-        // variables to put in log message
-        SPAN_DROP_INTERVAL
+        'SPAN_DROP_MSG', // key for the OncePer
+        SPAN_DROP_MSG_INTERVAL_MS,
+        SPAN_DROP_MSG
       )
 
       return
@@ -121,6 +120,8 @@ class SpanStreamer {
   }
 
   sendQueue() {
+    logger.trace('Sending spans from queue.')
+
     // Continue sending the spans that were in the queue. _writable is checked
     // so that if a send fails while clearing the queue, this drain handler can
     // finish, and the drain handler setup on the failed send will then attempt
@@ -129,6 +130,11 @@ class SpanStreamer {
       const nextObject = this.spans.shift()
       this.send(nextObject)
     }
+
+    logger.trace(
+      'Finished sending spans from queue. Items left in queue: %s',
+      this.spans.length
+    )
   }
 
   connect(agent_run_id, requestHeadersMap) {
@@ -144,6 +150,11 @@ class SpanStreamer {
   disconnect() {
     this.connection.disconnect()
   }
+
+  createMetrics() {
+    this._metrics.getOrCreateMetric(NAMES.QUEUE_CAPACITY).recordValue(this.queue_size)
+    this._metrics.getOrCreateMetric(NAMES.QUEUE_SIZE).recordValue(this.spans.length)
+  }
 }
 
 module.exports = SpanStreamer

package/lib/spans/streaming-span-event-aggregator.js

@@ -98,6 +98,10 @@ class StreamingSpanEventAggregator extends Aggregator {
 
     this.requestHeadersMap = config.request_headers_map
   }
+
+  createMetrics() {
+    this.stream.createMetrics()
+  }
 }
 
 module.exports = StreamingSpanEventAggregator

package/lib/spans/streaming-span-event.js

@@ -102,9 +102,21 @@ class StreamingSpanEvent {
   }
 
   static fromSegment(segment, parentId = null, isRoot = false) {
+    const spanContext = segment.getSpanContext()
+
+    // Since segments already hold span agent attributes and we want to leverage
+    // filtering, we add to the segment attributes prior to processing.
+    if (spanContext.hasError && !segment.transaction.hasIgnoredErrorStatusCode()) {
+      const details = spanContext.errorDetails
+      segment.addSpanAttribute('error.message', details.message)
+      segment.addSpanAttribute('error.class', details.type)
+      if (details.expected) {
+        segment.addSpanAttribute('error.expected', details.expected)
+      }
+    }
+
     const agentAttributes = segment.attributes.get(DESTINATIONS.SPAN_EVENT)
 
-    const spanContext = segment.getSpanContext()
     const customAttributes = spanContext.customAttributes.get(DESTINATIONS.SPAN_EVENT)
 
     const transaction = segment.transaction

package/lib/transaction/index.js

@@ -679,7 +679,6 @@ Transaction.prototype.measure = function measure(name, scope, duration, exclusiv
 Transaction.prototype._setApdex = function _setApdex(name, duration, keyApdexInMillis) {
   var apdexStats = this.metrics.getOrCreateApdexMetric(name, null, keyApdexInMillis)
 
-
   // if we have an error-like status code, and all the errors are
   // expected, we know the status code was caused by an expected
   // error, so we will not report "frustrating".  Otherwise, we
@@ -747,19 +746,13 @@ function _linkExceptionToSegment(exception) {
     return
   }
 
-  const config = this.agent.config
-
-  // Add error attributes to the span
-  const details = exception.getErrorDetails(config)
-  segment.addSpanAttribute('error.message', details.message)
-  segment.addSpanAttribute('error.class', details.type)
-
-  const isExpected =
-    errorHelper.isExpectedErrorClass(config, details.type) ||
-    errorHelper.isExpectedErrorMessage(config, details.type, details.message)
-
-  if (isExpected) {
-    segment.addSpanAttribute('error.expected', isExpected)
+  const spanContext = segment.getSpanContext()
+  if (spanContext) {
+    // Exception attributes will be added to span unless transaction
+    // status code has been ignored. Last error wins.
+    const config = this.agent.config
+    const details = exception.getErrorDetails(config)
+    spanContext.setError(details)
   }
 
   // Add the span/segment ID to the exception as agent attributes
@@ -793,6 +786,14 @@ function _addUserError(exception) {
   this.userErrors.push(exception)
 }
 
+/**
+ * Returns if the transaction's current status code is errored
+ * but considered ignored via the config.
+ */
+Transaction.prototype.hasIgnoredErrorStatusCode = function _hasIgnoredErrorStatusCode() {
+  return urltils.isIgnoredError(this.agent.config, this.statusCode)
+}
+
 /**
  * Returns true if an error happened during the transaction or if the transaction itself is
  * considered to be an error.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "newrelic",
-  "version": "6.12.1",
+  "version": "6.14.0",
   "author": "New Relic Node.js agent team <nodejs@newrelic.com>",
   "license": "Apache-2.0",
   "contributors": [
@@ -142,8 +142,8 @@
     "newrelic-naming-rules": "./bin/test-naming-rules.js"
   },
   "dependencies": {
-    "@grpc/grpc-js": "1.0.5",
-    "@grpc/proto-loader": "^0.5.4",
+    "@grpc/grpc-js": "1.1.7",
+    "@grpc/proto-loader": "^0.5.5",
     "@newrelic/aws-sdk": "^2.0.0",
     "@newrelic/koa": "^4.0.0",
     "@newrelic/superagent": "^3.0.0",
@@ -174,7 +174,6 @@
     "got": "^8.0.1",
     "http-errors": "^1.7.3",
     "jsdoc": "^3.6.3",
-    "lodash": "^4.17.14",
     "memcached": ">=0.2.8",
     "minami": "^1.1.1",
     "mongodb": "^3.3.3",

package/third_party_manifest.json

@@ -1,30 +1,30 @@
 {
-  "lastUpdated": "Mon Aug 03 2020 12:31:39 GMT-0700 (Pacific Daylight Time)",
+  "lastUpdated": "Wed Oct 28 2020 11:06:05 GMT-0700 (Pacific Daylight Time)",
   "projectName": "New Relic Node Agent",
   "projectUrl": "https://github.com/newrelic/node-newrelic",
   "includeDev": true,
   "dependencies": {
-    "@grpc/grpc-js@1.0.5": {
+    "@grpc/grpc-js@1.1.7": {
       "name": "@grpc/grpc-js",
-      "version": "1.0.5",
-      "range": "1.0.5",
+      "version": "1.1.7",
+      "range": "1.1.7",
       "licenses": "Apache-2.0",
       "repoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js",
-      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.0.5",
+      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/tree/v1.1.7",
       "licenseFile": "node_modules/@grpc/grpc-js/LICENSE",
-      "licenseUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.0.5/LICENSE",
+      "licenseUrl": "https://github.com/grpc/grpc-node/tree/master/packages/grpc-js/blob/v1.1.7/LICENSE",
       "licenseTextSource": "file",
       "publisher": "Google Inc."
     },
-    "@grpc/proto-loader@0.5.4": {
+    "@grpc/proto-loader@0.5.5": {
       "name": "@grpc/proto-loader",
-      "version": "0.5.4",
-      "range": "^0.5.4",
+      "version": "0.5.5",
+      "range": "^0.5.5",
       "licenses": "Apache-2.0",
       "repoUrl": "https://github.com/grpc/grpc-node",
-      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/v0.5.4",
+      "versionedRepoUrl": "https://github.com/grpc/grpc-node/tree/v0.5.5",
       "licenseFile": "node_modules/@grpc/proto-loader/LICENSE",
-      "licenseUrl": "https://github.com/grpc/grpc-node/blob/v0.5.4/LICENSE",
+      "licenseUrl": "https://github.com/grpc/grpc-node/blob/v0.5.5/LICENSE",
       "licenseTextSource": "file",
       "publisher": "Google Inc."
     },
@@ -358,19 +358,6 @@
       "publisher": "Michael Mathews",
       "email": "micmath@gmail.com"
     },
-    "lodash@4.17.15": {
-      "name": "lodash",
-      "version": "4.17.15",
-      "range": "^4.17.14",
-      "licenses": "MIT",
-      "repoUrl": "https://github.com/lodash/lodash",
-      "versionedRepoUrl": "https://github.com/lodash/lodash/tree/v4.17.15",
-      "licenseFile": "node_modules/lodash/LICENSE",
-      "licenseUrl": "https://github.com/lodash/lodash/blob/v4.17.15/LICENSE",
-      "licenseTextSource": "file",
-      "publisher": "John-David Dalton",
-      "email": "john.david.dalton@gmail.com"
-    },
     "memcached@2.2.2": {
       "name": "memcached",
       "version": "2.2.2",

package/CODE_OF_CONDUCT.md

@@ -1,5 +0,0 @@
-### Hello! We're glad you've joined us. 
-
-We believe participation in our community should be a harassment free experience for everyone. 
-
-Learn more about our guidelines and principles by reading our [Code of Conduct](https://opensource.newrelic.com/code-of-conduct/) on our Open Source Website.