elastic-apm 3.5.0 → 3.10.0

data/docs/debugging.asciidoc

@@ -28,3 +28,17 @@ Things to consider:
   - Experiencing high load? The agent can spawn multiple instances of its Workers that pick off the queue by changing the option `pool_size` (default is `1`).
   - If you have high load you may also consider setting `transaction_sample_rate` to something smaller than `1.0`. This determines whether to include _spans_ for every _transaction_. If you have enough traffic, skipping some (probably) identical spans won't have a noticeable effect on your data.
 
+[float]
+[[disable-agent]]
+=== Disable the Agent
+
+In the unlikely event the agent causes disruptions to a production application,
+you can disable the agent while you troubleshoot.
+
+If you have access to <<dynamic-configuration,dynamic configuration>>,
+you can disable the recording of events by setting <<config-recording,`recording`>> to `false`.
+When changed at runtime from a supported source, there's no need to restart your application.
+
+If that doesn't work, or you don't have access to dynamic configuration, you can disable the agent by setting
+<<config-enabled,`enabled`>> to `false`.
+You'll need to restart your application for the changes to apply.

data/docs/getting-started-rails.asciidoc

@@ -25,4 +25,6 @@ server_url: http://localhost:8200
 secret_token: ''
 ----
 
+Or if you prefer environment variables, skip the file and set `ELASTIC_APM_SERVER_URL` and `ELASTIC_APM_SECRET_TOKEN` in your local or server environment.
+
 This automatically sets up error logging and performance tracking but of course there are knobs to turn if you'd like to. See <<configuration>>.

data/docs/graphql.asciidoc

@@ -0,0 +1,23 @@
+ifdef::env-github[]
+NOTE: For the best reading experience,
+please view this documentation at https://www.elastic.co/guide/en/apm/agent/ruby[elastic.co]
+endif::[]
+
+[[graphql]]
+== GraphQL (experimental)
+
+The agent comes with support for GraphQL based APIs.
+
+This slightly alters how transactions are named when they relate to GraphQL
+queries, so they are easier to tell apart and debug.
+
+To enable GraphQL support, add the included Tracer to your schema:
+
+[source,ruby]
+----
+class MySchema < GraphQL::Schema
+  # ...
+
+  tracer ElasticAPM::GraphQL # <-- include this
+end
+----

data/docs/images/dynamic-config.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="59" height="20"><linearGradient id="b" x2="0" y2="100%"><stop offset="0" stop-color="#bbb" stop-opacity=".1"/><stop offset="1" stop-opacity=".1"/></linearGradient><clipPath id="a"><rect width="59" height="20" rx="3" fill="#fff"/></clipPath><g clip-path="url(#a)"><path fill="#9f9f9f" d="M0 0h0v20H0z"/><path fill="#9f9f9f" d="M0 0h59v20H0z"/><path fill="url(#b)" d="M0 0h59v20H0z"/></g><g fill="#fff" text-anchor="middle" font-family="DejaVu Sans,Verdana,Geneva,sans-serif" font-size="110"> <text x="295" y="150" fill="#010101" fill-opacity=".3" transform="scale(.1)" textLength="490">Dynamic</text><text x="295" y="140" transform="scale(.1)" textLength="490">Dynamic</text></g> </svg>

data/docs/index.asciidoc

@@ -25,9 +25,14 @@ include::./metrics.asciidoc[]
 
 include::./opentracing.asciidoc[]
 
+include::./graphql.asciidoc[]
+
 include::./log-correlation.asciidoc[]
 
+include::./performance-tuning.asciidoc[]
+
 include::./debugging.asciidoc[]
 
-include::./release-notes.asciidoc[]
+include::./upgrading.asciidoc[]
 
+include::./release-notes.asciidoc[]

data/docs/introduction.asciidoc

@@ -32,4 +32,5 @@ You can then use the APM app in Kibana to gain insight into latency issues and e
 === Additional Components
 
 APM Agents work in conjunction with the {apm-server-ref-v}/index.html[APM Server], {ref}/index.html[Elasticsearch], and {kibana-ref}/index.html[Kibana].
-Please view the {apm-overview-ref-v}/index.html[APM Overview] for details on how these components work together. 
+The {apm-overview-ref-v}/index.html[APM Overview] provides details on how these components work together,
+and provides a matrix outlining {apm-overview-ref-v}/agent-server-compatibility.html[Agent and Server compatibility].

data/docs/performance-tuning.asciidoc

@@ -0,0 +1,106 @@
+[[tuning-and-overhead]]
+== Performance tuning
+
+Using any APM solution comes with trade-offs, and the Elastic APM Ruby Agent is no different.
+Instrumenting your code, using timers, recording context data, etc., uses resources—for example:
+
+* CPU time
+* Memory
+* Bandwidth
+* Elasticsearch storage
+
+We invest a lot of effort to ensure that the Ruby Agent is suitable for production code
+and that its overhead remains as low as possible.
+However, because every application is different, there are some knobs you can turn and tweak to adapt the Agent to your specific needs.
+
+[float]
+[[tuning-sample-rate]]
+=== Transaction sample rate
+
+By default, the Agent samples every transaction.
+The easiest way to reduce both the overhead of the agent and storage requirements,
+is to tell it to do less, i.e., sample fewer transactions.
+To do this, set the <<config-transaction-sample-rate,`transaction_sample_rate`>>
+to a value between `0.0` and `1.0`—the percentage of transactions you'd like to randomly sample.
+The Agent will still record the overall time and result of unsampled transactions,
+but not context information, tags, or spans.
+
+[float]
+[[tuning-frame-context]]
+=== Collecting frame context
+
+The Agent automatically captures several lines of source code around each frame location in the stack trace.
+This enables the APM app to provide greater insight into exactly where an error or span is occurring in your code.
+This insight does come at a cost—in terms of performance, stack trace collection is the most expensive thing the Agent does.
+
+There are settings you can modify to control this behavior:
+
+1. Disable stack trace frame collection for short-duration spans by setting
+<<config-span-frames-min-duration-ms,`span_frames_min_duration`>> to `0`.
+
+2. Modify the number of source code lines collected.
+These settings are divided between app frames, which represent your application code,
+and library frames, which represent the code of your dependencies.
+Each of these categories are further split into separate error and span settings.
++
+* <<config-source-lines-error-app-frames, `source_lines_error_app_frames`>>
+* <<config-source-lines-error-library-frames,`source_lines_error_library_frames`>>
+* <<config-source-lines-span-app-frames,`source_lines_span_app_frames`>>
+* <<config-source-lines-span-library-frames,`source_lines_span_library_frames`>>
+
+3. If you're using the API to create a custom span, you can disable stack trace collection with the
+<<api-agent-start_span,`include_stacktrace` argument>>.
+
+Reading source files inside a running application can cause a lot of disk I/O,
+and sending up source lines for each frame will have a network and storage cost that is quite high.
+Turning these limits down will prevent storing excessive amounts of data in Elasticsearch.
+
+[float]
+[[tuning-queue]]
+=== Transaction queue
+
+The Agent does not send every transaction as it happens.
+Instead, to reduce load on the APM Server, the Agent uses a queue.
+The queue is flushed periodically, or when it reaches a maximum size.
+
+While this reduces the load on the APM Server, holding on to transaction data in a queue uses memory.
+If you notice a large increase in memory use, try adjusting these settings:
+
+ * <<config-api-request-time,`api_request_time`>> to reduce the duration of a single streaming request.
+ This setting is helpful if you have a sustained high number of transactions.
+ * <<config-api-request-size,`api_request_size`>> to reduce the maximum size of one request.
+ This setting can help if you experience transaction peaks (a large number in a short period of time).
+
+Keep in mind that reducing the value of either setting will cause the agent to send more HTTP requests to the APM Server,
+potentially causing a higher load.
+
+[float]
+[[tuning-max-spans]]
+=== Spans per transaction
+
+The number of spans per transaction will influence both how much time the agent spends in each transaction collecting contextual data,
+and how much storage space is needed in Elasticsearch.
+In our experience, most _usual_ transactions should have well below 100 spans.
+In some cases, however, the number of spans can explode—for example:
+
+* Long-running transactions
+* Unoptimized code, e.g., doing hundreds of SQL queries in a loop
+
+To avoid these edge cases which overload both the Agent and the APM Server,
+the Agent will stop recording spans when a specified limit is reached.
+This limit is configurable with <<config-transaction-max-spans,`transaction_max_spans`>>.
+
+[float]
+[[tuning-body-headers]]
+=== Capturing headers and request body
+
+You can configure the Agent to capture headers and request bodies with
+<<config-capture-headers,`capture_headers`>> and <<config-capture-body,`capture_body`>>.
+By default, headers are captured and request bodies are not.
+
+Depending on the nature of your POST requests, capturing request bodies for transactions may introduce noticeable overhead,
+as well as increased storage use.
+In most scenarios, we advise against enabling request body capturing for transactions, and only enabling it if necessary for errors.
+
+Capturing request/response headers has less overhead on the agent than capturing request bodies,
+but can have an impact on storage use.

data/docs/set-up.asciidoc

@@ -9,8 +9,11 @@ endif::[]
 
 To get you off the ground, we’ve prepared guides for setting up the Agent with different frameworks:
 
+ * <<getting-started-rails,Rails>>
+ * <<getting-started-rack,Rack>>
+
+For custom instrumentation, see the <<api>>.
+
 include::./getting-started-rails.asciidoc[]
 
 include::./getting-started-rack.asciidoc[]
-
-For custom instrumentation, see the <<api>>.

data/docs/supported-technologies.asciidoc

@@ -60,6 +60,7 @@ See <<getting-started-grape>>.
 We automatically instrument database actions using:
 
 - ActiveRecord (v4.2+)
+- DynamoDB (v1.0+)
 - Elasticsearch (v0.9+)
 - Mongo (v2.1+)
 - Redis (v3.1+)
@@ -85,4 +86,70 @@ We automatically instrument background processing using:
 - DelayedJob
 - Sidekiq
 - Shoryuken
-- Sneakers (v2.12.0+) (Experimental, see {pull}676[#676])
+- Sneakers (v2.12.0+) (Experimental, see https://github.com/elastic/apm-agent-ruby/pull/676[#676])
+- Resque (v2.0.0+)
+- SuckerPunch (v2.0.0+)
+
+[float]
+[[supported-technologies-resque]]
+=== Resque
+
+To make the agent work with Resque, you need to require `elastic_apm/resque` before you boot your Resque worker process.
+
+For example in your `Rakefile`:
+
+[source,ruby]
+----
+require 'resque'
+require 'elastic_apm'
+require 'elastic_apm/resque'
+----
+
+When you start Resque, you should see a series of messages like the following in the Resque logs:
+
+[source,ruby]
+----
+I, [XXX #81227]  INFO -- : Starting worker main
+D, [XXX #81227] DEBUG -- : Registered signals
+I, [XXX #81227]  INFO -- : Running before_first_fork hooks
+D, [XXX #81227] DEBUG -- : Starting ElasticAPM agent
+----
+
+Also be sure to set the Resque environment variable `RUN_AT_EXIT_HOOKS` to `true`. Otherwise, the fork may be
+terminated before the agent has a chance to send all the fork's events to the APM server.
+
+[float]
+[[supported-technologies-sucker-punch]]
+=== SuckerPunch
+
+Asynchronously executed jobs in SuckerPunch are automatically instrumented.
+
+Note that errors raised in the user-defined `JobClass#perform` method will be first handled by the SuckerPunch exception
+handler before being handled by the agent. The handler is accessed/set via `SuckerPunch.exception_handler` in version
+2.0. The agent transaction will be marked as successful unless you re-raise the error in the exception handler.
+You can also explicitly report the error via <<api-agent-report,`ElasticAPM.report`>> in a custom SuckerPunch exception
+handler.
+
+[float]
+[[supported-technologies-grpc]]
+=== gRPC
+
+We automatically instrument gRPC using the `grpc` gem. Note that this is experimental, as the `grpc` gem's
+support for `Interceptors` is experimental as of version 1.27.0.
+
+To instrument a client, add the `ElasticAPM::GRPC::ClientInterceptor` as an `interceptor` at Stub creation.
+
+[source,ruby]
+----
+Helloworld::Greeter::Stub.new(
+  'localhost:50051',
+  interceptors: [ElasticAPM::GRPC::ClientInterceptor.new]
+)
+----
+
+To instrument a server, add the `ElasticAPM::GRPC::ServerInterceptor`.
+
+[source,ruby]
+----
+GRPC::RpcServer.new(interceptors: [ElasticAPM::GRPC::ServerInterceptor.new])
+----

data/docs/upgrading.asciidoc

@@ -0,0 +1,45 @@
+[[upgrading]]
+== Upgrading
+Upgrades between minor versions of the agent, like from 2.1 to 2.2 are always backwards compatible.
+Upgrades that involve a major version bump often come with some backwards incompatible changes.
+
+Before upgrading the agent, be sure to review the:
+
+* <<release-notes,Agent release notes>>
+* {apm-overview-ref-v}/agent-server-compatibility.html[Agent and Server compatibility chart]
+
+[float]
+[[end-of-life-dates]]
+=== End of life dates
+
+We love all our products, but sometimes we must say goodbye to a release so that we can continue moving
+forward on future development and innovation.
+Our https://www.elastic.co/support/eol[End of life policy] defines how long a given release is considered supported,
+as well as how long a release is considered still in active development or maintenance.
+The table below is a simplified description of this policy.
+
+[options="header"]
+|====
+|Agent version |EOL Date |Maintained until
+|3.5.x |2021-07-17 | 3.6
+|3.4.x |2021-07-10 | 3.5
+|3.3.x |2021-06-04 | 3.4
+|3.2.x |2021-05-19 | 3.3
+|3.1.x |2021-04-21 | 3.2
+|3.0.x |2021-04-08 | 3.1
+|2.12.x |2021-04-01 |4.0
+|2.11.x |2021-03-23 |2.12
+|2.10.x |2021-03-03 |2.11
+|2.9.x |2020-12-25 |2.10
+|2.8.x |2020-11-20 |2.9
+|2.7.x |2020-11-07 |2.8
+|2.6.x |2020-09-19 |2.7
+|2.5.x |2020-09-01 |2.6
+|2.4.x |2020-08-27 |2.5
+|2.3.x |2020-07-29 |2.4
+|2.2.x |2020-07-22 |2.3
+|2.1.x |2020-06-04 |2.2
+|2.0.x |2020-05-14 |2.1
+|1.1.x |2020-03-07 |3.0
+|1.0.x |2019-12-29 |1.1
+|====

data/elastic-apm.gemspec

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'elastic_apm/version'

data/lib/elastic-apm.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 # Make bundler auto-require work

data/lib/elastic_apm.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'erb'
@@ -22,10 +39,12 @@ require 'elastic_apm/instrumenter'
 require 'elastic_apm/util'
 
 require 'elastic_apm/middleware'
+require 'elastic_apm/graphql'
 
 require 'elastic_apm/rails' if defined?(::Rails::Railtie)
 require 'elastic_apm/sinatra' if defined?(::Sinatra)
 require 'elastic_apm/grape' if defined?(::Grape)
+require 'elastic_apm/grpc' if defined?(::GRPC)
 
 # ElasticAPM
 module ElasticAPM
@@ -45,6 +64,16 @@ module ElasticAPM
       Agent.stop
     end
 
+    # Restarts the ElasticAPM Agent using the same config or a new
+    # one, if it is provided.
+    # Starts the agent if it is not running.
+    # Stops and starts the agent if it is running.
+    def restart(config = nil)
+      config ||= agent&.config
+      stop if running?
+      start(config)
+    end
+
     # @return [Boolean] Whether there's an [Agent] running
     def running?
       Agent.running?

data/lib/elastic_apm/agent.rb

@@ -1,3 +1,20 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 # frozen_string_literal: true
 
 require 'elastic_apm/error'
@@ -34,9 +51,9 @@ module ElasticAPM
       LOCK.synchronize do
         return @instance if @instance
 
-        unless config.active?
+        unless config.enabled?
           config.logger.debug format(
-            "%sAgent disabled with `active: false'",
+            "%sAgent disabled with `enabled: false'",
             Logging::PREFIX
           )
           return
@@ -72,6 +89,7 @@ module ElasticAPM
         metrics: metrics,
         stacktrace_builder: stacktrace_builder
       ) { |event| enqueue event }
+      @pid = Process.pid
     end
 
     attr_reader(
@@ -145,6 +163,9 @@ module ElasticAPM
       context: nil,
       trace_context: nil
     )
+      return unless config.recording?
+      detect_forking!
+
       instrumenter.start_transaction(
         name,
         type,
@@ -170,6 +191,12 @@ module ElasticAPM
       parent: nil,
       sync: nil
     )
+      detect_forking!
+
+      # We don't check config.recording? because the span
+      # will not be created if there's no transaction.
+      # We want to use the recording value from the config
+      # that existed when start_transaction was called. ~estolfo
       instrumenter.start_span(
         name,
         type,
@@ -207,6 +234,8 @@ module ElasticAPM
     # errors
 
     def report(exception, context: nil, handled: true)
+      return unless config.recording?
+      detect_forking!
       return if config.filter_exception_types.include?(exception.class.to_s)
 
       error = @error_builder.build_exception(
@@ -219,6 +248,9 @@ module ElasticAPM
     end
 
     def report_message(message, context: nil, backtrace: nil, **attrs)
+      return unless config.recording?
+      detect_forking!
+
       error = @error_builder.build_log(
         message,
         context: context,
@@ -240,5 +272,19 @@ module ElasticAPM
     def inspect
       super.split.first + '>'
     end
+
+    def detect_forking!
+      return if @pid == Process.pid
+
+      config.logger.debug "Detected forking,
+        restarting threads in process [PID:#{Process.pid}]"
+
+      central_config.handle_forking!
+      transport.handle_forking!
+      instrumenter.handle_forking!
+      metrics.handle_forking!
+
+      @pid = Process.pid
+    end
   end
 end