newrelic_rpm 3.5.2.17 → 3.5.3.24

data/.travis.yml

@@ -4,4 +4,6 @@ rvm:
   - ree
   - 1.9.2
   - 1.9.3
-  - jruby
+  - jruby-19mode
+  - jruby-18mode
+

data/CHANGELOG

@@ -1,6 +1,22 @@
 
 # New Relic Ruby Agent Release Notes #
 
+## v3.5.3 ##
+
+  * Update the collector protocol to use JSON and Ruby primitives
+
+    The communication between the agent and the NewRelic will not longer be
+    marshalled Ruby objects, but rather JSON in the case of Ruby 1.9 and marshalled
+    Ruby primitives in the case of 1.8.  This results in greater harvest efficiency
+    as well as feature parity with other New Relic agents.
+
+  * Fix incorrect application of conditions in sinatra instrumentation
+
+    The agent's sinatra instrumentation was causing sinatra's condidtions to
+    be incorrectly applied in some obscure cases.  The bug was triggered
+    when a condition was present on a lower priority route that would match
+    the current request, except for the precense of a higher priority route.
+
 ## v3.5.2 ##
 
  * Simplified process of running agent test suite and documented code
@@ -19,13 +35,13 @@
 
  * Server-side configuration for ignoring errors was not being heeded by agent.
 
- * Better handling of a thread safety issue.  
+ * Better handling of a thread safety issue.
 
-   Some issues may remain, which we are working to address, but they should be gracefully handled 
+   Some issues may remain, which we are working to address, but they should be gracefully handled
    now, rather than crashing the running app.
 
  * Use "java_import" rather than "include_class" when require Java Jars into a JRuby app.
-   
+
    Thanks to Jan Habermann for the pull request
 
  * Replaced alias_method mechanism with super call in DataMapper instrumentation.
@@ -46,7 +62,7 @@
 
 ## v3.5.0.1 ##
 
- * (Fix) Due to a serious resource leak we have ended support for versions of Phusion Passenger 
+ * (Fix) Due to a serious resource leak we have ended support for versions of Phusion Passenger
    older than 2.1.1. Users of older versions are encouraged upgrade to a more recent version.
 
 ## v3.5.0 ##
@@ -59,19 +75,19 @@
  * When the Ruby Agent detects Unicorn as the dispatcher it creates an INFO level log message
    with additional information
 
-   To help customers using Unicorn, if the agent detects it (Unicorn) is being used as the 
+   To help customers using Unicorn, if the agent detects it (Unicorn) is being used as the
    dispatcher an INFO level log message it created that includes a link to New Relic
    online doc that has additional steps that may be required to get performance data reporting.
 
  * (Fix) In version 3.4.2 of the Ruby Agent the server side value for Apdex T was disgregarded
 
-   With version 3.4.2 of the agent, the value set in the newrelic.yml file took precedence over the 
+   With version 3.4.2 of the agent, the value set in the newrelic.yml file took precedence over the
    value set in the New Relic UI.  As of version 3.5.0 only the value for Apdex T set in the
    New Relic UI will be used. Any setting in the yaml file will be ignored.
 
  * Improved Error Detection/Reporting capabilities for Rails 3 apps
 
-   Some errors are missed by the agent's exception reporting handlers because they are 
+   Some errors are missed by the agent's exception reporting handlers because they are
    generated in the rails stack, outside of the instrumented controller action. A Rack
    middleware is now included that can detect these errors as they bubble out of the middleware stack.
    Note that this does not include Routing Errors.
@@ -87,11 +103,11 @@
 
  * Know issue with Ruby 1.8.7-p334, sqlite3-ruby 1.3.0 or older, and resque 1.23.0
 
-   The Ruby Agent will not work in conjunction with Ruby 1.8.7-p334, sqlite3-ruby 1.3.3 
+   The Ruby Agent will not work in conjunction with Ruby 1.8.7-p334, sqlite3-ruby 1.3.3
    or earlier, and resque 1.23.0. Your app will likely stop functioning. This is a known problem
-   with Ruby versions up to 1.8.7-p334. Upgrading to the last release of Ruby 1.8.7 
-   is recommended.  This issue has been present in every version of the agent we've tested 
-   going back for a year. 
+   with Ruby versions up to 1.8.7-p334. Upgrading to the last release of Ruby 1.8.7
+   is recommended.  This issue has been present in every version of the agent we've tested
+   going back for a year.
 
 
 ### previous versions ###
@@ -109,7 +125,7 @@
  * The RUM NRAGENT tk value gets more robustly sanitized to prevent potential XSS vulnerabilities
 
    The code that scrubes the token used in Real User Monitoring has been enhanced to be
-   more robust. 
+   more robust.
 
  * Support for Apdex T in server side configuration
 
@@ -123,26 +139,26 @@
 
 ## v3.4.1 ##
 #### Bug Fixes ####
- * Fix edge case in RUM auto instrumentation where X-UA-Compatible meta tag is 
+ * Fix edge case in RUM auto instrumentation where X-UA-Compatible meta tag is
    present but </head> tag is missing.
 
-   There is a somewhat obscure edge case where RUM auto instrumentation will 
-   crash a request. The issue seems to be triggered when the X-UA-Compatible 
+   There is a somewhat obscure edge case where RUM auto instrumentation will
+   crash a request. The issue seems to be triggered when the X-UA-Compatible
    meta tag is present and the </head> tag is missing.
 
- * Fixed reference to @service.request_timeout to @request_timeout in 
+ * Fixed reference to @service.request_timeout to @request_timeout in
    new_relic_service.rb. (Thanks to Matthew Savage)
 
-   When a timeout occurred during connection to the collector an "undefined 
+   When a timeout occurred during connection to the collector an "undefined
    method `request_timeout' for nil:NilClass'" would get raised.
 
- * preserve visibility on traced methods. 
+ * preserve visibility on traced methods.
 
-   Aliased methods now have the same visibility as the original traced method. 
-   A couple of the esoteric methods created in the process weren't getting the 
-   visibility  set properly. 
+   Aliased methods now have the same visibility as the original traced method.
+   A couple of the esoteric methods created in the process weren't getting the
+   visibility  set properly.
 
- * Agent service does not connect to directed shard collector after connecting 
+ * Agent service does not connect to directed shard collector after connecting
    to proxy
 
    After connecting to collector proxy name of real collector was updated, but
@@ -157,7 +173,7 @@
  * should reset RubyBench GC counter between polls
 
    On Ruby REE, the GC profiler does not reset the counter between polls. This
-   is only a problem if GC could happen *between* transactions, as in, for 
+   is only a problem if GC could happen *between* transactions, as in, for
    example, out-of-band GC in Unicorn. fixed.
 
 v3.4.0.1

data/GUIDELINES_FOR_CONTRIBUTING.md

@@ -16,8 +16,12 @@ specific libraries such as `ActiveSupport`.
 
 ## Testing
 
-The agent includes a suite of unit tests which should be used to verify your
-changes don't break existing functionality.
+The agent includes a suite of unit and functional tests which should be used to
+verify your changes don't break existing functionality.
+
+Unit tests are stored in the `test/new_relic` directory.
+
+Functional tests are stored in the `test/multiverse` directory.
 
 ### Running Tests
 
@@ -50,3 +54,20 @@ functionality (e.g. a performance optimization) and new tests are not required.
 In general, including tests with your pull request dramatically increases the
 chances it will be accepted.
 
+### Functional Testing
+
+For cases where the unit test environment is not sufficient for testing a
+change (e.g. instrumentation for a non-rails framework, not available in the
+unit test environment), we have a functional testing suite called multiverse.
+These tests can be run by invoking:
+
+    bundle
+    bundle exec rake test:multiverse
+
+### And Finally...
+
+You are welcome to send pull requests to us - however, by doing so you agree
+that you are granting New Relic a non-exclusive, non-revokable, no-cost license
+to use the code, algorithms, patents, and ideas in that code in our products if
+we so choose. You also agree the code is provided as-is and you provide no
+warranties as to its fitness or correctness for any purpose.

data/Rakefile

@@ -1,7 +1,39 @@
 require 'rubygems'
+require 'rake/testtask'
 require "#{File.dirname(__FILE__)}/lib/new_relic/version.rb"
 require "#{File.dirname(__FILE__)}/lib/tasks/all.rb"
 
+task :default => :test
+
+task :test => 'test:newrelic'
+
+namespace :test do
+  desc "Run all tests"
+  task :all => %w{newrelic multiverse}
+
+  agent_home = File.expand_path(File.dirname(__FILE__))
+
+  desc "Run functional test suite for newrelic"
+  task :multiverse, [:suite, :mode] => [:gemspec] do |t, args|
+    args.with_defaults(:suite => "", :mode => "")
+    if args.mode == "run_one"
+      puts `#{agent_home}/test/multiverse/script/run_one #{args.suite}`
+    else
+      ruby "#{agent_home}/test/multiverse/script/runner #{args.suite}"
+    end
+  end
+
+  Rake::TestTask.new(:intentional_fail) do |t|
+    t.libs << "#{agent_home}/test"
+    t.libs << "#{agent_home}/lib"
+    t.pattern = "#{agent_home}/test/intentional_fail.rb"
+    t.verbose = true
+  end
+
+  # Note unit testing task is defined in lib/tasks/tests.rake to facilitate
+  # running them in a rails application environment.
+
+end
 
 desc 'Generate gemspec [ build_number, stage ]'
 task :gemspec, [ :build_number, :stage ] do |t, args|

data/lib/new_relic/agent.rb

@@ -84,6 +84,7 @@ module NewRelic
     require 'new_relic/agent/stats_engine'
     require 'new_relic/agent/transaction_sampler'
     require 'new_relic/agent/sql_sampler'
+    require 'new_relic/agent/thread_profiler'
     require 'new_relic/agent/error_collector'
     require 'new_relic/agent/busy_calculator'
     require 'new_relic/agent/sampler'

data/lib/new_relic/agent/agent.rb

@@ -8,6 +8,7 @@ require 'new_relic/agent/new_relic_service'
 require 'new_relic/agent/pipe_service'
 require 'new_relic/agent/configuration/manager'
 require 'new_relic/agent/database'
+require 'new_relic/agent/thread_profiler'
 
 module NewRelic
   module Agent
@@ -26,6 +27,7 @@ module NewRelic
         @stats_engine = NewRelic::Agent::StatsEngine.new
         @transaction_sampler = NewRelic::Agent::TransactionSampler.new
         @sql_sampler = NewRelic::Agent::SqlSampler.new
+        @thread_profiler = NewRelic::Agent::ThreadProfiler.new
         @error_collector = NewRelic::Agent::ErrorCollector.new
         @connect_attempts = 0
 
@@ -70,6 +72,8 @@ module NewRelic
         # the transaction sampler that handles recording transactions
         attr_reader :transaction_sampler
         attr_reader :sql_sampler
+        # begins a thread profile session when instructed by agent commands
+        attr_reader :thread_profiler
         # error collector is a simple collection of recorded errors
         attr_reader :error_collector
         # whether we should record raw, obfuscated, or no sql
@@ -592,10 +596,9 @@ module NewRelic
         # See #connect for a description of connection_options.
         def start_worker_thread(connection_options = {})
           log.debug "Creating Ruby Agent worker thread."
-          @worker_thread = Thread.new do
+          @worker_thread = NewRelic::Agent::Thread.new('Worker Loop') do
             deferred_work!(connection_options)
-          end # thread new
-          @worker_thread['newrelic_label'] = 'Worker Loop'
+          end
         end
 
         # A shorthand for NewRelic::Control.instance
@@ -980,7 +983,7 @@ module NewRelic
               if Agent.config[:'transaction_tracer.explain_enabled']
                 options[:explain_sql] = Agent.config[:'transaction_tracer.explain_threshold']
               end
-              traces = @traces.collect {|trace| trace.prepare_to_send(options)}
+              traces = @traces.map {|trace| trace.prepare_to_send(options) }
               @service.transaction_sample_data(traces)
               log.debug "Sent slowest sample (#{@service.agent_id}) in #{Time.now - now} seconds"
             rescue UnrecoverableServerException => e
@@ -994,6 +997,17 @@ module NewRelic
           @traces = nil
         end
 
+        def harvest_and_send_thread_profile(disconnecting=false)
+          @thread_profiler.stop(true) if disconnecting
+
+          if @thread_profiler.finished?
+            profile = @thread_profiler.harvest
+
+            log.debug "Sending thread profile #{profile.profile_id}"
+            @service.profile_data(profile)
+          end
+        end
+
         # Gets the collection of unsent errors from the error
         # collector. We pass back in an existing array of errors that
         # may be left over from a previous send
@@ -1023,13 +1037,37 @@ module NewRelic
           end
         end
 
-        def transmit_data
+        # Only JSON marshalling appears to work with collector on
+        # get_agent_commands and agent_command_results. We only support
+        # these features on Ruby versions that can hack JSON out of the box
+        def agent_commands_supported?
+          RUBY_VERSION >= "1.9.2"
+        end
+
+        def check_for_agent_commands
+          if !agent_commands_supported?
+            log.debug("Skipping agent commands, as they aren't supported on this environment")
+            return
+          end
+
+          commands = @service.get_agent_commands
+          log.debug "Received get_agent_commands = #{commands}"
+
+          @thread_profiler.respond_to_commands(commands) do |command_id, error|
+            @service.agent_command_results(command_id, error)
+          end
+        end
+
+        def transmit_data(disconnecting=false)
           now = Time.now
           log.debug "Sending data to New Relic Service"
           harvest_and_send_errors
           harvest_and_send_slowest_sample
           harvest_and_send_slowest_sql
           harvest_and_send_timeslice_data
+          harvest_and_send_thread_profile(disconnecting)
+
+          check_for_agent_commands
         rescue => e
           retry_count ||= 0
           retry_count += 1
@@ -1055,7 +1093,8 @@ module NewRelic
           if @connected
             begin
               @service.request_timeout = 10
-              transmit_data
+              transmit_data(true)
+
               if @connected_pid == $$ && !@service.kind_of?(NewRelic::Agent::NewRelicService)
                 log.debug "Sending New Relic service agent run shutdown message"
                 @service.shutdown(Time.now.to_f)

data/lib/new_relic/agent/browser_monitoring.rb

@@ -31,14 +31,11 @@ module NewRelic
       # page as is reasonably possible - that is, before any style or
       # javascript inclusions, but after any header-related meta tags
       def browser_timing_header
-        if NewRelic::Agent.instance.beacon_configuration.nil? ||
-           !NewRelic::Agent.is_transaction_traced? ||
-           !NewRelic::Agent.is_execution_traced? ||
-           NewRelic::Agent::TransactionInfo.get.ignore_end_user?
-          return ""
+        if insert_js?
+          NewRelic::Agent.instance.beacon_configuration.browser_timing_header
+        else
+          ""
         end
-
-        NewRelic::Agent.instance.beacon_configuration.browser_timing_header
       end
 
       # This method returns a string suitable for inclusion in a page
@@ -50,19 +47,11 @@ module NewRelic
       # This is the footer string - it should be placed as low in the
       # page as is reasonably possible.
       def browser_timing_footer
-        config = NewRelic::Agent.instance.beacon_configuration
-
-        if config.nil? ||
-            !config.enabled? ||
-            Agent.config[:browser_key].nil? ||
-            Agent.config[:browser_key].empty? ||
-            !NewRelic::Agent.is_transaction_traced? ||
-            !NewRelic::Agent.is_execution_traced? ||
-            NewRelic::Agent::TransactionInfo.get.ignore_end_user?
-          return ""
-         end
-
-        generate_footer_js(config)
+        if insert_js?
+          generate_footer_js(NewRelic::Agent.instance.beacon_configuration)
+        else
+          ""
+        end
       end
 
       module_function
@@ -129,6 +118,33 @@ module NewRelic
 
       private
 
+      # Check whether RUM header and footer should be generated.  Log the
+      # reason if they shouldn't.
+      def insert_js?
+        if NewRelic::Agent.instance.beacon_configuration.nil?
+          NewRelic::Agent.logger.debug "Beacon configuration is nil. Skipping browser instrumentation."
+          false
+        elsif ! NewRelic::Agent.instance.beacon_configuration.enabled?
+          NewRelic::Agent.logger.debug "Beacon configuration is disabled. Skipping browser instrumentation."
+          NewRelic::Agent.logger.debug NewRelic::Agent.instance.beacon_configuration.inspect
+          false
+        elsif Agent.config[:browser_key].nil? || Agent.config[:browser_key].empty?
+          NewRelic::Agent.logger.debug "Browser key is not set. Skipping browser instrumentation."
+          false
+        elsif ! NewRelic::Agent.is_transaction_traced?
+          NewRelic::Agent.logger.debug "Transaction is not traced. Skipping browser instrumentation."
+          false
+        elsif ! NewRelic::Agent.is_execution_traced?
+          NewRelic::Agent.logger.debug "Execution is not traced. Skipping browser instrumentation."
+          false
+        elsif NewRelic::Agent::TransactionInfo.get.ignore_end_user?
+          NewRelic::Agent.logger.debug "Ignore end user for this transaction is set. Skipping browser instrumentation."
+          false
+        else
+          true
+        end
+      end
+
       def generate_footer_js(config)
         if browser_monitoring_start_time
           footer_js_string(config)

data/lib/new_relic/agent/busy_calculator.rb

@@ -30,7 +30,8 @@ module NewRelic
       # called when a transaction finishes, to add time to the
       # instance variable accumulator. this is harvested when we send
       # data to the server
-      def dispatcher_finish(end_time = Time.now)
+      def dispatcher_finish(end_time = nil)
+        end_time ||= time_now
         callers = Thread.current[:busy_entries] -= 1
         # Ignore nested calls
         return if callers > 0
@@ -56,15 +57,14 @@ module NewRelic
         Thread.current[:busy_entries] = 0
         @lock ||= Mutex.new
         @accumulator = 0
-        @harvest_start = Time.now
+        @harvest_start = time_now
       end
 
-      self.reset
 
       # Called before uploading to to the server to collect current busy stats.
       def harvest_busy
         busy = 0
-        t0 = Time.now
+        t0 = time_now
         @lock.synchronize do
           busy = accumulator
           @accumulator = 0
@@ -88,12 +88,20 @@ module NewRelic
         instance_busy_stats.record_data_point busy
         @harvest_start = t0
       end
+
       private
+
+      # so we can stub Time.now only for the BusyCalculator in tests
+      def time_now
+        Time.now
+      end
+
       def instance_busy_stats
         # Late binding on the Instance/busy stats
         NewRelic::Agent.agent.stats_engine.get_stats_no_scope 'Instance/Busy'
       end
 
+      self.reset
     end
   end
 end