newrelic_rpm 3.10.0.279 → 3.11.0.283

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 4556b4a30019cb741eba3e669f57af080c0189e3
+  data.tar.gz: cc3e6812a23b94c1e5ea69a6ccc442afb5c08ad7
+SHA512:
+  metadata.gz: d758439faefa78cb3673721f575f8cdf07caadcafd124e6d61f16676a3bd76874d8a92d0817e564bf8842ab19e1e494a0a80597a0872c6cc158b8db0dc7b1212
+  data.tar.gz: a14aa6eb45971e5b4f9ddc45376c3ef8d0b737cfb968bd07ffe48f77855abe80a9ce0091490f8927b37c988ad4df3b861257e46684b2c815e585f77207b1cbd6

data/CHANGELOG

@@ -1,5 +1,66 @@
 # New Relic Ruby Agent Release Notes #
 
+## v3.11.0 ##
+
+  * Unified view for SQL database and NoSQL datastore products.
+
+  The response time charts in the application overview page will now include
+  NoSQL datastores, such as MongoDB, and also the product name of existing SQL
+  databases such as MySQL, Postgres, etc.
+
+  The Databases page will now enable the filtering of metrics and operations
+  by product, and includes a table listing all operations.
+
+  For existing SQL databases, in addition to the existing breakdown of SQL
+  statements and operations, the queries are now also associated with the
+  database product being used.
+
+  For NoSQL datastores, such as MongoDB, we have now added information about
+  operations performed against those products, similar to what is being done
+  for SQL databases.
+
+  Because this introduces a notable change to how SQL database metrics are
+  collected, it is important that you upgrade the agent version on all hosts.
+  If you are unable to transition to the latest agent version on all hosts at
+  the same time, you can still access old and new metric data for SQL
+  databases, but the information will be split across two separate views.
+
+  For more information see https://docs.newrelic.com/docs/apm/applications-menu/monitoring/databases-slow-queries-dashboard
+
+  * Track background transactions as Key Transactions
+
+  In prior versions of the Ruby agent, only web transactions could be tracked
+  as Key Transactions. This functionality is now available to all
+  transactions, including custom Apdex values and X-Ray sessions.
+
+  For more information see https://docs.newrelic.com/docs/apm/selected-transactions/key-transactions/key-transactions-tracking-important-transactions-or-events
+
+  * More support and documentation for third-party extensions
+
+  It's always been possible to write extension gems for the Ruby agent, but
+  now there's one location with best practices and recommendations to guide
+  you in writing extensions. Check out
+  https://docs.newrelic.com/docs/agents/ruby-agent/frameworks/third-party-instrumentation
+
+  We've also added simpler APIs for tracing datastores and testing your
+  extensions. It's our way of giving back to everyone who's helped build on
+  the agent over the years. <3
+
+  * Fix for anonymous class middleware naming
+
+  Metric names based off anonymous middlewares lacked a class name in the UI.
+  The Ruby agent will now look for a superclass, or default to AnonymousClass
+  in those cases.
+
+  * Improved exit behavior in the presence of Sinatra
+
+  The agent uses an `at_exit` hook to ensure data from the last < 60s before a
+  process exits is sent to New Relic. Previously, this hook was skipped if
+  Sinatra::Application was defined. This unfortunately missed data for
+  short-lived background processes that required, but didn't run, Sinatra. Now
+  the agent only skips its `at_exit` hook if Sinatra actually runs from
+  `at_exit`.
+
 ## v3.10.0 ##
 
   * Support for the Grape framework

data/lib/new_relic/agent.rb

@@ -313,6 +313,14 @@ module NewRelic
       agent.drop_buffered_data
     end
 
+    # Require agent testing helper methods
+    #
+    # @api public
+    def require_test_helper
+      path = File.join(__FILE__, '..', '..', '..', 'test', 'agent_helper')
+      require File.expand_path(path)
+    end
+
     # Deprecated in favor of drop_buffered_data
     #
     # @api public
@@ -608,16 +616,14 @@ module NewRelic
       end
     end
 
-    # Yield to a block that is run with a database metric name
-    # context.  This means the Database instrumentation will use this
-    # for the metric name if it does not otherwise know about a model.
-    # This is re-entrant.
+    # Yield to a block that is run with a database metric name context.  This means
+    # the Database instrumentation will use this for the metric name if it does not
+    # otherwise know about a model.  This is re-entrant.
     #
-    # * <tt>model</tt> is the DB model class
-    # * <tt>method</tt> is the name of the finder method or other
-    #   method to identify the operation with.
+    # @param [String,Class,#to_s] model the DB model class
     #
-    # @api public
+    # @param [String] method the name of the finder method or other method to
+    # identify the operation with.
     #
     def with_database_metric_name(model, method, &block) #THREAD_LOCAL_ACCESS
       if txn = Transaction.tl_current

data/lib/new_relic/agent/agent.rb

@@ -337,37 +337,52 @@ module NewRelic
             NewRelic::Agent.disable_all_tracing { connect(:keep_retrying => false) }
           end
 
-          # If we're using sinatra, old versions run in an at_exit
-          # block so we should probably know that
-          def using_sinatra?
-            defined?(Sinatra::Application)
-          end
-
-          # we should not set an at_exit block if people are using
-          # these as they don't do standard at_exit behavior per MRI/YARV
-          def weird_ruby?
-            NewRelic::LanguageSupport.using_engine?('rbx') ||
-              NewRelic::LanguageSupport.using_engine?('jruby') ||
-              using_sinatra?
+          # This matters when the following three criteria are met:
+          #
+          # 1. A Sinatra 'classic' application is being run
+          # 2. The app is being run by executing the main file directly, rather
+          #    than via a config.ru file.
+          # 3. newrelic_rpm is required *after* sinatra
+          #
+          # In this case, the entire application runs from an at_exit handler in
+          # Sinatra, and if we were to install ours, it would be executed before
+          # the one in Sinatra, meaning that we'd shutdown the agent too early
+          # and never collect any data.
+          def sinatra_classic_app?
+            (
+              defined?(Sinatra::Application) &&
+              Sinatra::Application.respond_to?(:run) &&
+              Sinatra::Application.run?
+            )
+          end
+
+          def should_install_exit_handler?
+            (
+              Agent.config[:send_data_on_exit]                  &&
+              !NewRelic::LanguageSupport.using_engine?('rbx')   &&
+              !NewRelic::LanguageSupport.using_engine?('jruby') &&
+              !sinatra_classic_app?
+            )
+          end
+
+          # There's an MRI 1.9 bug that loses exit codes in at_exit blocks.
+          # A workaround is necessary to get correct exit codes for the agent's
+          # test suites.
+          # http://bugs.ruby-lang.org/issues/5218
+          def need_exit_code_workaround?
+            defined?(RUBY_ENGINE) && RUBY_ENGINE == "ruby" && RUBY_VERSION.match(/^1\.9/)
           end
 
-          # Installs our exit handler, which exploits the weird
-          # behavior of at_exit blocks to make sure it runs last, by
-          # doing an at_exit within an at_exit block.
           def install_exit_handler
-            if Agent.config[:send_data_on_exit] && !weird_ruby?
-              at_exit do
-                # Workaround for MRI 1.9 bug that loses exit codes in at_exit blocks.
-                # This is necessary to get correct exit codes for the agent's
-                # test suites.
-                # http://bugs.ruby-lang.org/issues/5218
-                if defined?(RUBY_ENGINE) && RUBY_ENGINE == "ruby" && RUBY_VERSION.match(/^1\.9/)
-                  exit_status = $!.status if $!.is_a?(SystemExit)
-                  shutdown
-                  exit exit_status if exit_status
-                else
-                  shutdown
-                end
+            return unless should_install_exit_handler?
+            NewRelic::Agent.logger.debug("Installing at_exit handler")
+            at_exit do
+              if need_exit_code_workaround?
+                exit_status = $!.status if $!.is_a?(SystemExit)
+                shutdown
+                exit exit_status if exit_status
+              else
+                shutdown
               end
             end
           end

data/lib/new_relic/agent/agent_logger.rb

@@ -2,6 +2,7 @@
 # This file is distributed under New Relic's license terms.
 # See https://github.com/newrelic/rpm/blob/master/LICENSE for complete details.
 
+require 'thread'
 require 'logger'
 require 'new_relic/agent/hostname'
 
@@ -9,9 +10,8 @@ module NewRelic
   module Agent
     class AgentLogger
 
-      attr_reader :already_logged
-
       def initialize(root = "", override_logger=nil)
+        @already_logged_lock = Mutex.new
         clear_already_logged
         create_log(root, override_logger)
         set_log_level!
@@ -43,25 +43,24 @@ module NewRelic
       NUM_LOG_ONCE_KEYS = 1000
 
       def log_once(level, key, *msgs)
-        # Since `already_logged` might change between calls, just grab it once
-        # and use it throughout this method.
-        logged = already_logged
-
-        return if logged.include?(key)
-
-        if logged.size >= NUM_LOG_ONCE_KEYS && key.kind_of?(String)
-          # The reason for preventing too many keys in `logged` is for
-          # memory concerns.
-          # The reason for checking the type of the key is that we always want
-          # to allow symbols to log, since there are very few of them.
-          # The assumption here is that you would NEVER pass dynamically-created
-          # symbols, because you would never create symbols dynamically in the
-          # first place, as that would already be a memory leak in most Rubies,
-          # even if we didn't hang on to them all here.
-          return
+        @already_logged_lock.synchronize do
+          return if @already_logged.include?(key)
+
+          if @already_logged.size >= NUM_LOG_ONCE_KEYS && key.kind_of?(String)
+            # The reason for preventing too many keys in `logged` is for
+            # memory concerns.
+            # The reason for checking the type of the key is that we always want
+            # to allow symbols to log, since there are very few of them.
+            # The assumption here is that you would NEVER pass dynamically-created
+            # symbols, because you would never create symbols dynamically in the
+            # first place, as that would already be a memory leak in most Rubies,
+            # even if we didn't hang on to them all here.
+            return
+          end
+
+          @already_logged[key] = true
         end
 
-        logged[key] = true
         self.send(level, *msgs)
       end
 
@@ -148,7 +147,9 @@ module NewRelic
       end
 
       def clear_already_logged
-        @already_logged = {}
+        @already_logged_lock.synchronize do
+          @already_logged = {}
+        end
       end
 
       def wants_stdout?

data/lib/new_relic/agent/configuration/default_source.rb

@@ -306,7 +306,7 @@ module NewRelic
           :description => 'Defines a host for communicating with New Relic via a proxy server.'
         },
         :proxy_port => {
-          :default => nil,
+          :default => 8080,
           :allow_nil => true,
           :public => true,
           :type => Fixnum,
@@ -595,6 +595,12 @@ module NewRelic
           :type => Boolean,
           :description => 'Enable or disable active record instrumentation.'
         },
+        :disable_data_mapper => {
+          :default => false,
+          :public => true,
+          :type => Boolean,
+          :description => 'Enable or disable DataMapper instrumentation.'
+        },
         :disable_activejob => {
           :default => false,
           :public => true,
@@ -602,6 +608,30 @@ module NewRelic
           :dynamic_name => true,
           :description => 'Enable or disable ActiveJob instrumentation.'
         },
+        :disable_memcached => {
+          :default => value_of(:disable_memcache_instrumentation),
+          :public => true,
+          :type => Boolean,
+          :description => 'Enable or disable instrumentation for the memcached gem.'
+        },
+        :disable_memcache_client => {
+          :default => value_of(:disable_memcache_instrumentation),
+          :public => true,
+          :type => Boolean,
+          :description => 'Enable or disable instrumenation for the memcache-client gem.'
+        },
+        :disable_dalli => {
+          :default => value_of(:disable_memcache_instrumentation),
+          :public => true,
+          :type => Boolean,
+          :description => 'Enable or disable instrumentation for the dalli gem.'
+        },
+        :disable_dalli_cas_client => {
+          :default => value_of(:disable_memcache_instrumentation),
+          :public => true,
+          :type => Boolean,
+          :description => "Enable or disable instrumentation for the dalli gem's additional CAS client support."
+        },
         :disable_memcache_instrumentation => {
           :default => false,
           :public => true,

data/lib/new_relic/agent/database.rb

@@ -250,9 +250,10 @@ module NewRelic
       ]
 
       SQL_COMMENT_REGEX = Regexp.new('/\*.*?\*/', Regexp::MULTILINE).freeze
+      EMPTY_STRING      = ''.freeze
 
       def parse_operation_from_query(sql)
-        sql = sql.gsub(SQL_COMMENT_REGEX, '')
+        sql = sql.gsub(SQL_COMMENT_REGEX, EMPTY_STRING)
         if sql =~ /(\w+)/
           op = $1.downcase
           return op if KNOWN_OPERATIONS.include?(op)

data/lib/new_relic/agent/datastores.rb

@@ -0,0 +1,177 @@
+# encoding: utf-8
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/rpm/blob/master/LICENSE for complete details.
+
+require 'new_relic/agent/datastores/metric_helper'
+
+module NewRelic
+  module Agent
+    module Datastores
+
+      # Add Datastore tracing to a method. This properly generates the metrics
+      # for New Relic's Datastore features. It does not capture the actual
+      # query content into Transaction Traces. Use wrap if you want to provide
+      # that functionality.
+      #
+      # @param [Class] clazz the class to instrument
+      #
+      # @param [String, Symbol] method_name the name of instance method to
+      # instrument
+      #
+      # @param [String] product name of your datastore for use in metric naming, e.g. "Redis"
+      #
+      # @param [optional,String] operation the name of operation if different
+      # than the instrumented method name
+      #
+      # @api public
+      #
+      def self.trace(clazz, method_name, product, operation = method_name)
+        clazz.class_eval do
+          method_name_without_newrelic = "#{method_name}_without_newrelic"
+
+          if NewRelic::Helper.instance_methods_include?(clazz, method_name) &&
+             !NewRelic::Helper.instance_methods_include?(clazz, method_name_without_newrelic)
+
+            visibility = NewRelic::Helper.instance_method_visibility(clazz, method_name)
+
+            alias_method method_name_without_newrelic, method_name
+
+            define_method(method_name) do |*args, &blk|
+              metrics = MetricHelper.metrics_for(product, operation)
+              NewRelic::Agent::MethodTracer.trace_execution_scoped(metrics) do
+                send(method_name_without_newrelic, *args, &blk)
+              end
+            end
+
+            send visibility, method_name
+            send visibility, method_name_without_newrelic
+          end
+        end
+      end
+
+      # Wrap a call to a datastore and record New Relic Datastore metrics. This
+      # method can be used when a collection (i.e. table or model name) is
+      # known at runtime to be included in the metric naming. It is intended
+      # for situations that the simpler NewRelic::Agent::Datastores.trace can't
+      # properly handle.
+      #
+      # To use this, wrap the datastore operation in the block passed to wrap.
+      #
+      #   NewRelic::Agent::Datastores.wrap("FauxDB", "find", "items") do
+      #     FauxDB.find(query)
+      #   end
+      #
+      # @param [String] product the datastore name for use in metric naming,
+      # e.g. "FauxDB"
+      #
+      # @param [String,Symbol] operation the name of operation (e.g. "select"),
+      # often named after the method that's being instrumented.
+      #
+      # @param [optional, String] collection the collection name for use in
+      # statement-level metrics (i.e. table or model name)
+      #
+      # @param [Proc,#call] callback proc or other callable to invoke after
+      # running the datastore block. Receives three arguments: result of the
+      # yield, the most specific (scoped) metric name, and elapsed time of the
+      # call. An example use is attaching SQL to Transaction Traces at the end
+      # of a wrapped datastore call.
+      #
+      #   callback = Proc.new do |result, metrics, elapsed|
+      #     NewRelic::Agent::Datastores.notice_sql(query, metrics, elapsed)
+      #   end
+      #
+      #   NewRelic::Agent::Datastores.wrap("FauxDB", "find", "items", callback) do
+      #     FauxDB.find(query)
+      #   end
+      #
+      # **NOTE: THERE ARE SECURITY CONCERNS WHEN CAPTURING SQL!**
+      # New Relic's Transaction Tracing and Slow SQL features will
+      # attempt to apply obfuscation to the passed queries, but it is possible
+      # for a query format to be unsupported and result in exposing user
+      # information embedded within captured queries.
+      #
+      # @api public
+      #
+      def self.wrap(product, operation, collection = nil, callback = nil)
+        return yield unless operation
+
+        metrics = MetricHelper.metrics_for(product, operation, collection)
+        scoped_metric = metrics.first
+        NewRelic::Agent::MethodTracer.trace_execution_scoped(metrics) do
+          t0 = Time.now
+          begin
+            result = yield
+          ensure
+            if callback
+              elapsed_time = (Time.now - t0).to_f
+              callback.call(result, scoped_metric, elapsed_time)
+            end
+          end
+        end
+      end
+
+      # Wrapper for simplifying attaching SQL queries during a transaction.
+      #
+      # If you are recording non-SQL data, please use the notice_statement
+      # method instead.
+      #
+      #   NewRelic::Agent::Datastores.notice_sql(query, metrics, elapsed)
+      #
+      # @param [String] query the SQL text to be captured. Note that depending
+      # on user settings, this string will be run through obfuscation, but
+      # some dialects of SQL (or non-SQL queries) are not guaranteed to be
+      # properly obfuscated by these routines!
+      #
+      # @param [String] scoped_metric The most specific metric relating to this
+      # query. Typically the result of
+      # NewRelic::Agent::Datastores::MetricHelper#metrics_for
+      #
+      # @param [Float] elapsed the elapsed time during query execution
+      #
+      # **NOTE: THERE ARE SECURITY CONCERNS WHEN CAPTURING SQL!**
+      # New Relic's Transaction Tracing and Slow SQL features will
+      # attempt to apply obfuscation to the passed queries, but it is possible
+      # for a query format to be unsupported and result in exposing user
+      # information embedded within captured queries.
+      #
+      def self.notice_sql(query, scoped_metric, elapsed)
+        agent = NewRelic::Agent.instance
+        agent.transaction_sampler.notice_sql(query, nil, elapsed)
+        agent.sql_sampler.notice_sql(query, scoped_metric, nil, elapsed)
+        nil
+      end
+
+      # Wrapper for simplifying attaching non-SQL data statements to a
+      # transaction. For instance, Mongo or CQL queries, Memcached or Redis
+      # keys would all be appropriate data to attach as statements.
+      #
+      # Data passed to this method is NOT obfuscated by New Relic, so please
+      # ensure that user information is obfuscated if the agent setting
+      # `transaction_tracer.record_sql` is set to `obfuscated`
+      #
+      #   NewRelic::Agent::Datastores.notice_statement("key", elapsed)
+      #
+      # @param [String] statement text of the statement to capture.
+      #
+      # @param [Float] elapsed the elapsed time during query execution
+      #
+      # **NOTE: THERE ARE SECURITY CONCERNS WHEN CAPTURING STATEMENTS!**
+      # This method will properly ignore statements when the user has turned
+      # off capturing queries, but it is not able to obfuscate arbitrary data!
+      # To prevent exposing user information embedded in captured queries,
+      # please ensure all data passed to this method is safe to transmit to
+      # New Relic.
+      #
+      def self.notice_statement(statement, elapsed)
+        # Settings may change eventually, but for now we follow the same
+        # capture rules as SQL for non-SQL statements.
+        return unless NewRelic::Agent::Database.should_record_sql?
+
+        agent = NewRelic::Agent.instance
+        agent.transaction_sampler.notice_nosql_statement(statement, elapsed)
+        nil
+      end
+
+    end
+  end
+end