ddtrace 0.41.0 → 0.52.0

data/docs/ProfilingDevelopment.md

@@ -0,0 +1,107 @@
+# Profiling Development
+
+This file contains development notes specific to the profiling feature.
+
+For a more practical view of getting started with development of `ddtrace`, see <DevelopmentGuide.md>.
+
+## Profiling components high-level view
+
+Components below live inside <../lib/ddtrace/profiling>:
+
+* `Collectors::Stack`: Collects stack trace samples from Ruby threads for both CPU-time (if available) and wall-clock.
+  Runs on its own background thread.
+* `Encoding::Profile`: Encodes gathered data into the pprof format.
+* `Events::Stack`, `Events::StackSample`: Entity classes used to represent stacks.
+* `Ext::CPU`: Monkey patches Ruby's `Thread` with our `Ext::CThread` to enable CPU-time profiling.
+* `Ext::CThread`: Extension used to enable CPU-time profiling via use of Pthread's `getcpuclockid`.
+* `Ext::Forking`: Monkey patches `Kernel#fork`, adding a `Kernel#at_fork` callback mechanism which is used to restore
+  profiling abilities after the VM forks (such as re-instrumenting the main thread, and restarting profiler threads).
+* `Pprof::*` (in <../lib/ddtrace/profiling/pprof>): Converts samples captured in the `Recorder` into the pprof format.
+* `Tasks::Setup`: Takes care of loading our extensions/monkey patches to handle fork() and CPU profiling.
+* `Transport::*` (in <../lib/ddtrace/profiling/transport>): Implements transmission of profiling payloads to the Datadog agent
+  or backend.
+* `TraceIdentifiers::*`: Used to retrieve trace id and span id from tracers, to be used to connect traces to profiles.
+* `BacktraceLocation`: Entity class used to represent an entry in a stack trace.
+* `Buffer`: Bounded buffer used to store profiling events.
+* `Exporter`: Writes profiling data to a given transport.
+* `Flush`: Entity class used to represent metadata for a given profile.
+* `Profiler`: Profiling entry point, which coordinates collectors and a scheduler.
+* `Recorder`: Stores profiling events gathered by `Collector`s.
+* `Scheduler`: Periodically (every 1 minute) takes data from the `Recorder` and pushes them to all configured
+  `Exporter`s. Runs on its own background thread.
+
+## Initialization
+
+When started via `ddtracerb exec` (together with `DD_PROFILING_ENABLED=true`), initialization goes through the following
+flow:
+
+1. <../lib/ddtrace/profiling/preload.rb> triggers the creation of the `Datadog.profiler` instance by calling the method
+2. `Datadog.profiler` is handled by `Datadog::Configuration`, which triggers the configuration of `ddtrace` components
+   in `#build_components`
+3. Inside `Datadog::Components`, the `build_profiler` method triggers the execution of the `Tasks::Setup`
+4. The `Setup` task activates our extensions
+    * `Datadog::Profiling::Ext::Forking`
+    * `Datadog::Profiling::Ext::CPU`
+5. Still inside `Datadog::Components`, the `build_profiler` method then creates and wires up the Profiler as such:
+    ```asciiflow
+            +------------+
+            |  Profiler  |
+            +-+-------+--+
+              |       |
+              v       v
+    +---------+--+  +-+---------+
+    | Collectors |  | Scheduler |
+    +---------+--+  +-+-------+-+
+              |       |       |
+              v       |       v
+        +-----+-+     |  +----+------+
+        | Stack |     |  | Exporters |
+        +-----+-+     |  +-----------+
+              |       |
+              v       v
+            +-+-------+-+
+            | Recorder  |
+            +-----------+
+    ```
+6. The profiler gets started when `startup!` is called by `Datadog::Configuration` after component creation.
+
+## Run-time execution
+
+During run-time, the `Scheduler` and the `Collectors::Stack` each execute on their own background thread.
+
+The `Collectors::Stack` samples stack traces of threads, capturing both CPU-time (if available) and wall-clock, storing
+them in the `Recorder`.
+
+The `Scheduler` wakes up every 1 minute to flush the results of the `Recorder` into one or more `exporter`s.
+Usually only one exporter is in use. By default, the `Exporter` delegates to the default `Transport::HTTP` transport, which
+takes care of encoding the data and reporting it to the datadog agent (or to the API, when running without an agent).
+
+## How CPU-time profiling works
+
+**TODO**: Document our pthread-based approach to getting CPU-time for threads.
+
+## How linking of traces to profiles works
+
+The [code hotspots feature](https://docs.datadoghq.com/tracing/profiler/connect_traces_and_profiles) allows users to start
+from a trace and then to investigate the profile that corresponds to that trace.
+
+This works in two steps:
+1. Linking a trace to the profile that was gathered while it executed
+2. Enabling the filtering of a profile to contain only the samples relating to a given trace/span
+
+To link a trace to a profile, we must ensure that both have the same `runtime-id` tag.
+This tag is in `Datadog::Runtime::Identity.id` and is automatically added by both the tracer and the profiler to reported
+traces/profiles.
+
+The profiler backend links a trace covering a given time interval to the profiles covering the same time interval,
+whenever they share the same `runtime-id`.
+
+To further enable filtering of a profile to show only samples related to a given trace/span, each sample taken by the
+profiler is tagged with the trace_id and span_id for the given trace/span.
+
+This is done using the `Datadog::Profiling::TraceIdentifiers::Helper` that retrieves a trace_id and span_id, if
+available, from the supported tracers. This helper is called by the `Collectors::Stack` during sampling.
+
+Note that if a given trace executes too fast, it's possible that the profiler will not contain any samples for that
+specific trace. Nevertheless, the linking still works and is useful, as it allows users to explore what was going on their
+profile at that time, even if they can't filter down to the specific request.

data/ext/ddtrace_profiling_native_extension/extconf.rb

@@ -0,0 +1,28 @@
+# typed: false
+def skip_building_extension?
+  # We don't support JRuby for profiling, and JRuby doesn't support native extensions, so let's just skip this entire
+  # thing so that JRuby users of dd-trace-rb aren't impacted.
+  on_jruby = RUBY_ENGINE == 'jruby'
+
+  # Experimental toggle to disable building the extension.
+  # Disabling the extension will lead to the profiler not working in future releases.
+  # If you needed to use this, please tell us why on <https://github.com/DataDog/dd-trace-rb/issues/new>.
+  disabled_via_env = ENV['DD_PROFILING_NO_EXTENSION'].to_s.downcase == 'true'
+
+  on_jruby || disabled_via_env
+end
+
+if skip_building_extension?
+  File.write('Makefile', 'all install clean: # dummy makefile that does nothing')
+  return
+end
+
+# NOTE: we MUST NOT require 'mkmf' before we check the #skip_building_extension? because the require triggers checks
+# that may fail on an environment not properly setup for building Ruby extensions.
+require 'mkmf'
+
+# Tag the native extension library with the Ruby version and Ruby platform.
+# This makes it easier for development (avoids "oops I forgot to rebuild when I switched my Ruby") and ensures that
+# the wrong library is never loaded.
+# When requiring, we need to use the exact same string, including the version and the platform.
+create_makefile "ddtrace_profiling_native_extension.#{RUBY_VERSION}_#{RUBY_PLATFORM}"

data/ext/ddtrace_profiling_native_extension/profiling.c

@@ -0,0 +1,17 @@
+#include <ruby.h>
+#include <stdio.h>
+
+static VALUE native_working_p(VALUE self);
+
+void Init_ddtrace_profiling_native_extension(void) {
+  VALUE datadog_module = rb_define_module("Datadog");
+  VALUE profiling_module = rb_define_module_under(datadog_module, "Profiling");
+  VALUE native_extension_module = rb_define_module_under(profiling_module, "NativeExtension");
+
+  rb_define_singleton_method(native_extension_module, "native_working?", native_working_p, 0);
+  rb_funcall(native_extension_module, rb_intern("private_class_method"), 1, ID2SYM(rb_intern("native_working?")));
+}
+
+static VALUE native_working_p(VALUE self) {
+  return Qtrue;
+}

data/lib/datadog/ci.rb

@@ -0,0 +1,17 @@
+# typed: strict
+require 'ddtrace'
+
+module Datadog
+  # Namespace for Datadog CI instrumentation:
+  # e.g. rspec, cucumber, etc...
+  module CI
+  end
+end
+
+# Integrations
+require 'datadog/ci/contrib/cucumber/integration'
+require 'datadog/ci/contrib/rspec/integration'
+
+# Extensions
+require 'datadog/ci/extensions'
+Datadog::CI::Extensions.activate!

data/lib/datadog/ci/configuration/components.rb

@@ -0,0 +1,31 @@
+# typed: true
+require 'datadog/ci/context_flush'
+
+module Datadog
+  module CI
+    module Configuration
+      # Adds CI behavior to Datadog trace components
+      module Components
+        def initialize(settings)
+          # Activate CI mode if enabled
+          activate_ci_mode!(settings) if settings.ci_mode.enabled
+
+          # Initialize normally
+          super
+        end
+
+        def activate_ci_mode!(settings)
+          # Activate underlying tracing test mode
+          settings.test_mode.enabled = true
+
+          # Choose user defined ContextFlush or default to CI ContextFlush
+          settings.test_mode.context_flush = settings.ci_mode.context_flush \
+                                             || Datadog::CI::ContextFlush::Finished.new
+
+          # Pass through any other options
+          settings.test_mode.writer_options = settings.ci_mode.writer_options
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/configuration/settings.rb

@@ -0,0 +1,37 @@
+# typed: false
+require 'datadog/ci/ext/settings'
+
+module Datadog
+  module CI
+    module Configuration
+      # Adds CI behavior to Datadog trace settings
+      module Settings
+        def self.extended(base)
+          base = base.singleton_class unless base.is_a?(Class)
+          add_settings!(base)
+        end
+
+        def self.add_settings!(base)
+          base.class_eval do
+            settings :ci_mode do
+              option :enabled do |o|
+                o.default { env_to_bool(Datadog::CI::Ext::Settings::ENV_MODE_ENABLED, false) }
+                o.lazy
+              end
+
+              option :context_flush do |o|
+                o.default { nil }
+                o.lazy
+              end
+
+              option :writer_options do |o|
+                o.default { {} }
+                o.lazy
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/context_flush.rb

@@ -0,0 +1,29 @@
+# typed: true
+require 'ddtrace/context_flush'
+
+module Datadog
+  module CI
+    module ContextFlush
+      # Common behavior for CI flushing
+      module Tagging
+        # Decorate a trace with CI tags
+        def get_trace(context)
+          context.get do |trace|
+            # Origin tag is required on every span
+            trace.each { |span| context.attach_origin(span) } if trace
+          end
+        end
+      end
+
+      # Consumes only completed traces (where all spans have finished)
+      class Finished < Datadog::ContextFlush::Finished
+        prepend Tagging
+      end
+
+      # Performs partial trace flushing to avoid large traces residing in memory for too long
+      class Partial < Datadog::ContextFlush::Partial
+        prepend Tagging
+      end
+    end
+  end
+end

data/lib/datadog/ci/contrib/cucumber/configuration/settings.rb

@@ -0,0 +1,31 @@
+# typed: false
+require 'ddtrace/contrib/configuration/settings'
+require 'datadog/ci/contrib/cucumber/ext'
+
+module Datadog
+  module CI
+    module Contrib
+      module Cucumber
+        module Configuration
+          # Custom settings for the Cucumber integration
+          class Settings < Datadog::Contrib::Configuration::Settings
+            option :enabled do |o|
+              o.default { env_to_bool(Ext::ENV_ENABLED, true) }
+              o.lazy
+            end
+
+            option :service_name do |o|
+              o.default { Datadog.configuration.service_without_fallback || Ext::SERVICE_NAME }
+              o.lazy
+            end
+
+            option :operation_name do |o|
+              o.default { ENV.key?(Ext::ENV_OPERATION_NAME) ? ENV[Ext::ENV_OPERATION_NAME] : Ext::OPERATION_NAME }
+              o.lazy
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/contrib/cucumber/ext.rb

@@ -0,0 +1,20 @@
+# typed: true
+module Datadog
+  module CI
+    module Contrib
+      module Cucumber
+        # Cucumber integration constants
+        module Ext
+          APP = 'cucumber'.freeze
+          ENV_ENABLED = 'DD_TRACE_CUCUMBER_ENABLED'.freeze
+          ENV_OPERATION_NAME = 'DD_TRACE_CUCUMBER_OPERATION_NAME'.freeze
+          FRAMEWORK = 'cucumber'.freeze
+          OPERATION_NAME = 'cucumber.test'.freeze
+          SERVICE_NAME = 'cucumber'.freeze
+          STEP_SPAN_TYPE = 'step'.freeze
+          TEST_TYPE = 'test'.freeze
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/contrib/cucumber/formatter.rb

@@ -0,0 +1,98 @@
+# typed: true
+require 'datadog/ci/test'
+require 'datadog/ci/ext/app_types'
+require 'datadog/ci/ext/environment'
+require 'datadog/ci/ext/test'
+require 'datadog/ci/contrib/cucumber/ext'
+
+module Datadog
+  module CI
+    module Contrib
+      module Cucumber
+        # Defines collection of instrumented Cucumber events
+        class Formatter
+          attr_reader :config, :current_feature_span, :current_step_span
+          private :config
+          private :current_feature_span, :current_step_span
+
+          def initialize(config)
+            @config = config
+
+            bind_events(config)
+          end
+
+          def bind_events(config)
+            config.on_event :test_case_started, &method(:on_test_case_started)
+            config.on_event :test_case_finished, &method(:on_test_case_finished)
+            config.on_event :test_step_started, &method(:on_test_step_started)
+            config.on_event :test_step_finished, &method(:on_test_step_finished)
+          end
+
+          def on_test_case_started(event)
+            @current_feature_span = CI::Test.trace(
+              tracer,
+              configuration[:operation_name],
+              {
+                span_options: {
+                  app: Ext::APP,
+                  resource: event.test_case.name,
+                  service: configuration[:service_name]
+                },
+                framework: Ext::FRAMEWORK,
+                test_name: event.test_case.name,
+                test_suite: event.test_case.location.file,
+                test_type: Ext::TEST_TYPE
+              }
+            )
+          end
+
+          def on_test_case_finished(event)
+            return if @current_feature_span.nil?
+
+            if event.result.skipped?
+              CI::Test.skipped!(@current_feature_span)
+            elsif event.result.ok?
+              CI::Test.passed!(@current_feature_span)
+            elsif event.result.failed?
+              CI::Test.failed!(@current_feature_span)
+            end
+
+            @current_feature_span.finish
+          end
+
+          def on_test_step_started(event)
+            trace_options = {
+              resource: event.test_step.to_s,
+              span_type: Ext::STEP_SPAN_TYPE
+            }
+            @current_step_span = tracer.trace(Ext::STEP_SPAN_TYPE, trace_options)
+          end
+
+          def on_test_step_finished(event)
+            return if @current_step_span.nil?
+
+            if event.result.skipped?
+              CI::Test.skipped!(@current_step_span, event.result.exception)
+            elsif event.result.ok?
+              CI::Test.passed!(@current_step_span)
+            elsif event.result.failed?
+              CI::Test.failed!(@current_step_span, event.result.exception)
+            end
+
+            @current_step_span.finish
+          end
+
+          private
+
+          def configuration
+            Datadog.configuration[:cucumber]
+          end
+
+          def tracer
+            configuration[:tracer]
+          end
+        end
+      end
+    end
+  end
+end

data/lib/datadog/ci/contrib/cucumber/instrumentation.rb

@@ -0,0 +1,27 @@
+# typed: true
+require 'datadog/ci/contrib/cucumber/formatter'
+
+module Datadog
+  module CI
+    module Contrib
+      module Cucumber
+        # Instrumentation for Cucumber
+        module Instrumentation
+          def self.included(base)
+            base.prepend(InstanceMethods)
+          end
+
+          # Instance methods for configuration
+          module InstanceMethods
+            attr_reader :datadog_formatter
+
+            def formatters
+              @datadog_formatter ||= Datadog::CI::Contrib::Cucumber::Formatter.new(@configuration)
+              [@datadog_formatter] + super
+            end
+          end
+        end
+      end
+    end
+  end
+end