elastic-apm 3.6.0 → 3.10.1

data/docs/performance-tuning.asciidoc

@@ -0,0 +1,106 @@
+[[tuning-and-overhead]]
+== Performance tuning
+
+Using any APM solution comes with trade-offs, and the Elastic APM Ruby Agent is no different.
+Instrumenting your code, using timers, recording context data, etc., uses resources—for example:
+
+* CPU time
+* Memory
+* Bandwidth
+* Elasticsearch storage
+
+We invest a lot of effort to ensure that the Ruby Agent is suitable for production code
+and that its overhead remains as low as possible.
+However, because every application is different, there are some knobs you can turn and tweak to adapt the Agent to your specific needs.
+
+[float]
+[[tuning-sample-rate]]
+=== Transaction sample rate
+
+By default, the Agent samples every transaction.
+The easiest way to reduce both the overhead of the agent and storage requirements,
+is to tell it to do less, i.e., sample fewer transactions.
+To do this, set the <<config-transaction-sample-rate,`transaction_sample_rate`>>
+to a value between `0.0` and `1.0`—the percentage of transactions you'd like to randomly sample.
+The Agent will still record the overall time and result of unsampled transactions,
+but not context information, tags, or spans.
+
+[float]
+[[tuning-frame-context]]
+=== Collecting frame context
+
+The Agent automatically captures several lines of source code around each frame location in the stack trace.
+This enables the APM app to provide greater insight into exactly where an error or span is occurring in your code.
+This insight does come at a cost—in terms of performance, stack trace collection is the most expensive thing the Agent does.
+
+There are settings you can modify to control this behavior:
+
+1. Disable stack trace frame collection for short-duration spans by setting
+<<config-span-frames-min-duration-ms,`span_frames_min_duration`>> to `0`.
+
+2. Modify the number of source code lines collected.
+These settings are divided between app frames, which represent your application code,
+and library frames, which represent the code of your dependencies.
+Each of these categories are further split into separate error and span settings.
++
+* <<config-source-lines-error-app-frames, `source_lines_error_app_frames`>>
+* <<config-source-lines-error-library-frames,`source_lines_error_library_frames`>>
+* <<config-source-lines-span-app-frames,`source_lines_span_app_frames`>>
+* <<config-source-lines-span-library-frames,`source_lines_span_library_frames`>>
+
+3. If you're using the API to create a custom span, you can disable stack trace collection with the
+<<api-agent-start_span,`include_stacktrace` argument>>.
+
+Reading source files inside a running application can cause a lot of disk I/O,
+and sending up source lines for each frame will have a network and storage cost that is quite high.
+Turning these limits down will prevent storing excessive amounts of data in Elasticsearch.
+
+[float]
+[[tuning-queue]]
+=== Transaction queue
+
+The Agent does not send every transaction as it happens.
+Instead, to reduce load on the APM Server, the Agent uses a queue.
+The queue is flushed periodically, or when it reaches a maximum size.
+
+While this reduces the load on the APM Server, holding on to transaction data in a queue uses memory.
+If you notice a large increase in memory use, try adjusting these settings:
+
+ * <<config-api-request-time,`api_request_time`>> to reduce the duration of a single streaming request.
+ This setting is helpful if you have a sustained high number of transactions.
+ * <<config-api-request-size,`api_request_size`>> to reduce the maximum size of one request.
+ This setting can help if you experience transaction peaks (a large number in a short period of time).
+
+Keep in mind that reducing the value of either setting will cause the agent to send more HTTP requests to the APM Server,
+potentially causing a higher load.
+
+[float]
+[[tuning-max-spans]]
+=== Spans per transaction
+
+The number of spans per transaction will influence both how much time the agent spends in each transaction collecting contextual data,
+and how much storage space is needed in Elasticsearch.
+In our experience, most _usual_ transactions should have well below 100 spans.
+In some cases, however, the number of spans can explode—for example:
+
+* Long-running transactions
+* Unoptimized code, e.g., doing hundreds of SQL queries in a loop
+
+To avoid these edge cases which overload both the Agent and the APM Server,
+the Agent will stop recording spans when a specified limit is reached.
+This limit is configurable with <<config-transaction-max-spans,`transaction_max_spans`>>.
+
+[float]
+[[tuning-body-headers]]
+=== Capturing headers and request body
+
+You can configure the Agent to capture headers and request bodies with
+<<config-capture-headers,`capture_headers`>> and <<config-capture-body,`capture_body`>>.
+By default, headers are captured and request bodies are not.
+
+Depending on the nature of your POST requests, capturing request bodies for transactions may introduce noticeable overhead,
+as well as increased storage use.
+In most scenarios, we advise against enabling request body capturing for transactions, and only enabling it if necessary for errors.
+
+Capturing request/response headers has less overhead on the agent than capturing request bodies,
+but can have an impact on storage use.

data/docs/set-up.asciidoc

@@ -9,8 +9,11 @@ endif::[]
 
 To get you off the ground, we’ve prepared guides for setting up the Agent with different frameworks:
 
+ * <<getting-started-rails,Rails>>
+ * <<getting-started-rack,Rack>>
+
+For custom instrumentation, see the <<api>>.
+
 include::./getting-started-rails.asciidoc[]
 
 include::./getting-started-rack.asciidoc[]
-
-For custom instrumentation, see the <<api>>.

data/docs/supported-technologies.asciidoc

@@ -60,6 +60,7 @@ See <<getting-started-grape>>.
 We automatically instrument database actions using:
 
 - ActiveRecord (v4.2+)
+- DynamoDB (v1.0+)
 - Elasticsearch (v0.9+)
 - Mongo (v2.1+)
 - Redis (v3.1+)
@@ -87,6 +88,7 @@ We automatically instrument background processing using:
 - Shoryuken
 - Sneakers (v2.12.0+) (Experimental, see https://github.com/elastic/apm-agent-ruby/pull/676[#676])
 - Resque (v2.0.0+)
+- SuckerPunch (v2.0.0+)
 
 [float]
 [[supported-technologies-resque]]
@@ -113,6 +115,21 @@ I, [XXX #81227]  INFO -- : Running before_first_fork hooks
 D, [XXX #81227] DEBUG -- : Starting ElasticAPM agent
 ----
 
+Also be sure to set the Resque environment variable `RUN_AT_EXIT_HOOKS` to `true`. Otherwise, the fork may be
+terminated before the agent has a chance to send all the fork's events to the APM server.
+
+[float]
+[[supported-technologies-sucker-punch]]
+=== SuckerPunch
+
+Asynchronously executed jobs in SuckerPunch are automatically instrumented.
+
+Note that errors raised in the user-defined `JobClass#perform` method will be first handled by the SuckerPunch exception
+handler before being handled by the agent. The handler is accessed/set via `SuckerPunch.exception_handler` in version
+2.0. The agent transaction will be marked as successful unless you re-raise the error in the exception handler.
+You can also explicitly report the error via <<api-agent-report,`ElasticAPM.report`>> in a custom SuckerPunch exception
+handler.
+
 [float]
 [[supported-technologies-grpc]]
 === gRPC
@@ -135,4 +152,4 @@ To instrument a server, add the `ElasticAPM::GRPC::ServerInterceptor`.
 [source,ruby]
 ----
 GRPC::RpcServer.new(interceptors: [ElasticAPM::GRPC::ServerInterceptor.new])
-----
+----

data/elastic-apm.gemspec

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'elastic_apm/version'

data/lib/elastic-apm.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 # Make bundler auto-require work

data/lib/elastic_apm.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'erb'
@@ -51,7 +68,7 @@ module ElasticAPM
     # one, if it is provided.
     # Starts the agent if it is not running.
     # Stops and starts the agent if it is running.
-    def restart(config = {})
+    def restart(config = nil)
       config ||= agent&.config
       stop if running?
       start(config)

data/lib/elastic_apm/agent.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'elastic_apm/error'
@@ -34,9 +51,9 @@ module ElasticAPM
       LOCK.synchronize do
         return @instance if @instance
 
-        unless config.active?
+        unless config.enabled?
           config.logger.debug format(
-            "%sAgent disabled with `active: false'",
+            "%sAgent disabled with `enabled: false'",
             Logging::PREFIX
           )
           return
@@ -72,6 +89,7 @@ module ElasticAPM
         metrics: metrics,
         stacktrace_builder: stacktrace_builder
       ) { |event| enqueue event }
+      @pid = Process.pid
     end
 
     attr_reader(
@@ -145,6 +163,9 @@ module ElasticAPM
       context: nil,
       trace_context: nil
     )
+      return unless config.recording?
+      detect_forking!
+
       instrumenter.start_transaction(
         name,
         type,
@@ -170,6 +191,12 @@ module ElasticAPM
       parent: nil,
       sync: nil
     )
+      detect_forking!
+
+      # We don't check config.recording? because the span
+      # will not be created if there's no transaction.
+      # We want to use the recording value from the config
+      # that existed when start_transaction was called. ~estolfo
       instrumenter.start_span(
         name,
         type,
@@ -207,6 +234,8 @@ module ElasticAPM
     # errors
 
     def report(exception, context: nil, handled: true)
+      return unless config.recording?
+      detect_forking!
       return if config.filter_exception_types.include?(exception.class.to_s)
 
       error = @error_builder.build_exception(
@@ -219,6 +248,9 @@ module ElasticAPM
     end
 
     def report_message(message, context: nil, backtrace: nil, **attrs)
+      return unless config.recording?
+      detect_forking!
+
       error = @error_builder.build_log(
         message,
         context: context,
@@ -240,5 +272,19 @@ module ElasticAPM
     def inspect
       super.split.first + '>'
     end
+
+    def detect_forking!
+      return if @pid == Process.pid
+
+      config.logger.debug "Detected forking,
+        restarting threads in process [PID:#{Process.pid}]"
+
+      central_config.handle_forking!
+      transport.handle_forking!
+      instrumenter.handle_forking!
+      metrics.handle_forking!
+
+      @pid = Process.pid
+    end
   end
 end

data/lib/elastic_apm/central_config.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'elastic_apm/central_config/cache_control'
@@ -81,15 +98,16 @@ module ElasticAPM
         update[key] = @modified_options.delete(key)
       end
 
-      update_config(update)
+      @config.replace_options(update)
     end
 
-    private
-
-    def update_config(new_options)
-      @config = config.dup.tap { |new_config| new_config.assign(new_options) }
+    def handle_forking!
+      stop
+      start
     end
 
+    private
+
     # rubocop:disable Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
     def handle_success(resp)
       if (etag = resp.headers['Etag'])
@@ -104,9 +122,10 @@ module ElasticAPM
           assign(update)
         end
 
-        if @modified_options.any?
+        if update && update.any?
           info 'Updated config from Kibana'
-          debug 'Modified: %s', @modified_options.inspect
+          debug 'Modified: %s', update.inspect
+          debug 'Modified original options: %s', @modified_options.inspect
         end
       end
 
@@ -151,7 +170,7 @@ module ElasticAPM
     def schedule_next_fetch(resp = nil)
       headers = resp&.headers
       seconds =
-        if headers['Cache-Control']
+        if headers && headers['Cache-Control']
           CacheControl.new(headers['Cache-Control']).max_age
         else
           DEFAULT_MAX_AGE

data/lib/elastic_apm/central_config/cache_control.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 module ElasticAPM

data/lib/elastic_apm/child_durations.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 module ElasticAPM

data/lib/elastic_apm/config.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'elastic_apm/config/options'
@@ -19,13 +36,13 @@ module ElasticAPM
     option :secret_token,                      type: :string
     option :api_key,                           type: :string
 
-    option :active,                            type: :bool,   default: true
     option :api_buffer_size,                   type: :int,    default: 256
     option :api_request_size,                  type: :bytes,  default: '750kb', converter: Bytes.new
     option :api_request_time,                  type: :float,  default: '10s',   converter: Duration.new
     option :breakdown_metrics,                 type: :bool,   default: true
     option :capture_body,                      type: :string, default: 'off'
     option :capture_headers,                   type: :bool,   default: true
+    option :capture_elasticsearch_queries,     type: :bool,   default: false
     option :capture_env,                       type: :bool,   default: true
     option :central_config,                    type: :bool,   default: true
     option :current_user_email_method,         type: :string, default: 'email'
@@ -38,6 +55,7 @@ module ElasticAPM
     option :disable_start_message,             type: :bool,   default: false
     option :disable_instrumentations,          type: :list,   default: %w[json]
     option :disabled_spies,                    type: :list,   default: []
+    option :enabled,                           type: :bool,   default: true
     option :environment,                       type: :string, default: ENV['RAILS_ENV'] || ENV['RACK_ENV']
     option :framework_name,                    type: :string
     option :framework_version,                 type: :string
@@ -57,9 +75,11 @@ module ElasticAPM
     option :proxy_password,                    type: :string
     option :proxy_port,                        type: :int
     option :proxy_username,                    type: :string
+    option :recording,                         type: :bool,   default: true
     option :sanitize_field_names,              type: :list,   default: [],      converter: WildcardPatternList.new
     option :server_ca_cert,                    type: :string
     option :service_name,                      type: :string
+    option :service_node_name,                 type: :string
     option :service_version,                   type: :string
     option :source_lines_error_app_frames,     type: :int,    default: 5
     option :source_lines_error_library_frames, type: :int,    default: 0
@@ -67,6 +87,7 @@ module ElasticAPM
     option :source_lines_span_library_frames,  type: :int,    default: 0
     option :span_frames_min_duration,          type: :float,  default: '5ms',   converter: Duration.new(default_unit: 'ms')
     option :stack_trace_limit,                 type: :int,    default: 999_999
+    option :transaction_ignore_urls,           type: :list,   default: [],      converter: WildcardPatternList.new
     option :transaction_max_spans,             type: :int,    default: 500
     option :transaction_sample_rate,           type: :float,  default: 1.0
     option :use_elastic_traceparent_header,    type: :bool,   default: true
@@ -111,6 +132,7 @@ module ElasticAPM
       %w[
         action_dispatch
         delayed_job
+        dynamo_db
         elasticsearch
         faraday
         http
@@ -125,6 +147,7 @@ module ElasticAPM
         sidekiq
         sinatra
         sneakers
+        sucker_punch
         tilt
       ]
     end
@@ -138,6 +161,15 @@ module ElasticAPM
       warn "The option `#{name}' has been removed."
     end
 
+    def replace_options(new_options)
+      return if new_options.nil? || new_options.empty?
+      options_copy = @options.dup
+      new_options.each do |key, value|
+        options_copy.fetch(key.to_sym).set(value)
+      end
+      @options = options_copy
+    end
+
     def app=(app)
       case app_type?(app)
       when :sinatra
@@ -203,6 +235,13 @@ module ElasticAPM
       self.default_labels = value
     end
 
+    def ignore_url_patterns=(value)
+      warn '[DEPRECATED] The option ignore_url_patterns is being removed. ' \
+        'Consider using transaction_ignore_urls instead.'
+
+      set(:ignore_url_patterns, value)
+    end
+
     def custom_key_filters=(value)
       unless value == self.class.schema[:custom_key_filters][:default]
         warn '[DEPRECATED] The option custom_key_filters is being removed. ' \
@@ -216,6 +255,11 @@ module ElasticAPM
       disable_instrumentations
     end
 
+    def active
+      enabled
+    end
+    alias active? active
+
     def disabled_instrumentations=(value)
       warn '[DEPRECATED] The option disabled_instrumentations has been ' \
         'renamed to disable_instrumentations to align with other agents.'
@@ -227,6 +271,12 @@ module ElasticAPM
         'use use_legacy_sql_parser and please report why you wish to do so.'
     end
 
+    def active=(value)
+      warn '[DEPRECATED] The option active has been renamed to enabled ' \
+        'to align with other agents and with the remote config.'
+      self.enabled = value
+    end
+
     private
 
     def load_config_file
@@ -276,7 +326,7 @@ module ElasticAPM
       self.logger ||= ::Rails.logger
 
       self.__root_path = ::Rails.root.to_s
-      self.__view_paths = app.config.paths['app/views'].existent
+      self.__view_paths = app.config.paths['app/views'].existent + [::Rails.root.to_s]
     end
 
     def rails_app_name(app)