ddtrace 0.35.2 → 1.1.0

data/CONTRIBUTING.md

@@ -20,8 +20,6 @@ Additionally, if you can, include:
  - Links to similar features that would serve as a good comparison
  - (Any other details that would be useful for implementing this feature!)
 
-Feature requests will be reviewed, discussed, and then added to [our Community project](https://github.com/DataDog/dd-trace-rb/projects/2).
-
 ## Found a bug?
 
 For any urgent matters (such as outages) or issues concerning the Datadog service or UI, contact our support team via https://docs.datadoghq.com/help/ for direct, faster assistance.
@@ -46,7 +44,7 @@ If at all possible, also provide:
  - Code sample or test that reproduces the problem
  - An explanation of what causes the bug and/or how it can be fixed
 
-Reports that include rich detail are better, and ones with code that reproduce the bug are best. Bug requests are triaged, reviewed, and added to [our Community project](https://github.com/DataDog/dd-trace-rb/projects/2).
+Reports that include rich detail are better, and ones with code that reproduce the bug are best.
 
 ## Have a patch?
 
@@ -56,7 +54,7 @@ We welcome code contributions to the library, which you can [submit as a pull re
 2. **Make any changes** for your patch.
 3. **Write tests** that demonstrate how the feature works or how the bug is fixed.
 4. **Update any documentation** such as `docs/GettingStarted.md`, especially for new features.
-5. **Submit the pull request** from your fork back to https://github.com/DataDog/dd-trace-rb. Bugs should merge back to `master`, and new features should merge back to the latest `dev` branch (e.g. `0.24-dev`).
+5. **Submit the pull request** from your fork back to the latest revision of the `master` branch on https://github.com/DataDog/dd-trace-rb.
 
 The pull request will be run through our CI pipeline, and a project member will review the changes with you. At a minimum, to be accepted and merged, pull requests must:
 
@@ -74,9 +72,7 @@ We also recommend that you share in your description:
  - Benchmarks if the feature is anticipated to have performance implications
  - Any limitations, constraints or risks that are important to consider
 
-Pull requests will be reviewed and added to [our Community project](https://github.com/DataDog/dd-trace-rb/projects/2).
-
-For more information on common topics such as debugging locally, or how to write new integrations, check out [our development guide](https://github.com/DataDog/dd-trace-rb/docs/DevelopmentGuide.md). If at any point you have a question or need assistance with your pull request, feel free to mention a project member! We're always happy to help contributors with their pull requests.
+For more information on common topics such as debugging locally, or how to write new integrations, check out [our development guide](https://github.com/DataDog/dd-trace-rb/blob/master/docs/DevelopmentGuide.md). If at any point you have a question or need assistance with your pull request, feel free to mention a project member! We're always happy to help contributors with their pull requests.
 
 ## Final word
 

data/LICENSE-3rdparty.csv

@@ -0,0 +1,4 @@
+Component,Origin,License,Copyright
+lib/datadog/core/vendor/multipart-post,https://github.com/socketry/multipart-post,MIT,"Copyright (c) 2007-2013 Nick Sieger."
+lib/datadog/tracing/contrib/active_record/vendor,https://github.com/rails/rails/,MIT,"Copyright (c) 2005-2018 David Heinemeier Hansson"
+ext/ddtrace_profiling_native_extension/private_vm_api_access,https://github.com/ruby/ruby,BSD-2-Clause,"Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved."

data/README.md

@@ -1,6 +1,11 @@
+***Version 1.0.0 has recently been released. Check out our [upgrade guide](https://github.com/DataDog/dd-trace-rb/blob/master/docs/UpgradeGuide.md#from-0x-to-10) for more details.***
+
 # Datadog Trace Client
 
+[![Gem](https://img.shields.io/gem/v/ddtrace)](https://rubygems.org/gems/ddtrace/)
 [![CircleCI](https://circleci.com/gh/DataDog/dd-trace-rb/tree/master.svg?style=svg&circle-token=b0bd5ef866ec7f7b018f48731bb495f2d1372cc1)](https://circleci.com/gh/DataDog/dd-trace-rb/tree/master)
+[![codecov](https://codecov.io/gh/DataDog/dd-trace-rb/branch/master/graph/badge.svg)](https://app.codecov.io/gh/DataDog/dd-trace-rb/branch/master)
+[![YARD documentation](https://img.shields.io/badge/YARD-documentation-blue)](https://www.rubydoc.info/gems/ddtrace/)
 
 ``ddtrace`` is Datadog’s tracing client for Ruby. It is used to trace requests as they flow across web servers,
 databases and microservices so that developers have great visiblity into bottlenecks and troublesome requests.

data/bin/ddtracerb

@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+require 'datadog/profiling/tasks/exec'
+require 'datadog/profiling/tasks/help'
+
+command = ARGV.shift
+
+case command
+when 'exec'
+  Datadog::Profiling::Tasks::Exec.new(ARGV).run
+when 'help', '--help'
+  Datadog::Profiling::Tasks::Help.new.run
+else
+  puts "Command '#{command}' is not valid for ddtrace."
+  Datadog::Profiling::Tasks::Help.new.run
+end

data/ddtrace.gemspec

@@ -6,8 +6,8 @@ require 'ddtrace/version'
 
 Gem::Specification.new do |spec|
   spec.name                  = 'ddtrace'
-  spec.version               = Datadog::VERSION::STRING
-  spec.required_ruby_version = ">= #{Datadog::VERSION::MINIMUM_RUBY_VERSION}"
+  spec.version               = DDTrace::VERSION::STRING
+  spec.required_ruby_version = [">= #{DDTrace::VERSION::MINIMUM_RUBY_VERSION}", "< #{DDTrace::VERSION::MAXIMUM_RUBY_VERSION}"]
   spec.required_rubygems_version = '>= 2.0.0'
   spec.authors               = ['Datadog, Inc.']
   spec.email                 = ['dev@datadoghq.com']
@@ -28,39 +28,41 @@ Gem::Specification.new do |spec|
     raise 'RubyGems 2.0 or newer is required to protect against public gem pushes.'
   end
 
-  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
-  spec.bindir        = 'exe'
-  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.files =
+    `git ls-files -z`
+    .split("\x0")
+    .reject { |f| f.match(%r{^(test|spec|features|[.]circleci|[.]github|[.]dd-ci|benchmarks|gemfiles|integration|tasks|sorbet|yard)/}) }
+    .reject do |f|
+      ['.dockerignore', '.env', '.gitattributes', '.gitlab-ci.yml', '.rspec', '.rubocop.yml',
+       '.rubocop_todo.yml', '.simplecov', 'Appraisals', 'Gemfile', 'Rakefile', 'docker-compose.yml', '.pryrc', '.yardopts'].include?(f)
+    end
+  spec.executables   = ['ddtracerb']
   spec.require_paths = ['lib']
 
-  spec.add_dependency 'msgpack'
-
-  # Optional extensions
-  # TODO: Move this to Appraisals?
-  spec.add_development_dependency 'dogstatsd-ruby', '>= 3.3.0'
-  spec.add_development_dependency 'opentracing', '>= 0.4.1'
-
-  # Development dependencies
-  spec.add_development_dependency 'rake', '>= 10.5'
-  spec.add_development_dependency 'rubocop', '= 0.49.1' if RUBY_VERSION >= '2.1.0'
-  spec.add_development_dependency 'rspec', '~> 3.0'
-  spec.add_development_dependency 'rspec-collection_matchers', '~> 1.1'
-  spec.add_development_dependency 'minitest', '= 5.10.1'
-  spec.add_development_dependency 'appraisal', '~> 2.2'
-  spec.add_development_dependency 'yard', '~> 0.9'
-  spec.add_development_dependency 'webmock', '~> 2.0'
-  spec.add_development_dependency 'builder'
-  if RUBY_PLATFORM != 'java'
-    spec.add_development_dependency 'sqlite3', '~> 1.3.6'
+  # Important note: This `if` ONLY works for development. When packaging up the gem, Ruby runs this code, and hardcodes
+  # the output in the `.gem` file that gets uploaded to rubygems. So the decision here gets hardcoded, we don't actually
+  # pick a version depending on the Ruby customers are running, as it may appear.
+  # For more context, see the discussion in
+  # * https://github.com/DataDog/dd-trace-rb/pull/1739
+  # * https://github.com/DataDog/dd-trace-rb/pull/1336
+  if RUBY_VERSION >= '2.2.0'
+    spec.add_dependency 'msgpack'
   else
-    spec.add_development_dependency 'jdbc-sqlite3', '~> 3'
+    # msgpack 1.4 fails for Ruby 2.1: https://github.com/msgpack/msgpack-ruby/issues/205
+    spec.add_dependency 'msgpack', '< 1.4'
   end
-  spec.add_development_dependency 'climate_control', '~> 0.2.0'
 
-  # locking transitive dependency of webmock
-  spec.add_development_dependency 'addressable', '~> 2.4.0'
-  spec.add_development_dependency 'redcarpet', '~> 3.4' if RUBY_PLATFORM != 'java'
-  spec.add_development_dependency 'pry', '~> 0.10.4'
-  spec.add_development_dependency 'pry-stack_explorer', '~> 0.4.9.2'
-  spec.add_development_dependency 'warning', '~> 1' if RUBY_VERSION >= '2.5.0'
+  # Used by the profiler native extension to support older Rubies (see NativeExtensionDesign.md for notes)
+  #
+  # Because we only use this for older Rubies, and we consider it "feature-complete" for those older Rubies,
+  # we're pinning it at the latest available version and will manually bump the dependency as needed.
+  spec.add_dependency 'debase-ruby_core_source', '<= 0.10.16'
+
+  # Used by appsec
+  spec.add_dependency 'libddwaf', '~> 1.3.0.2.0'
+
+  # Used by profiling
+  spec.add_dependency 'libddprof', '~> 0.6.0.1.0'
+
+  spec.extensions = ['ext/ddtrace_profiling_native_extension/extconf.rb', 'ext/ddtrace_profiling_loader/extconf.rb']
 end

data/docs/AutoInstrumentation.md

@@ -0,0 +1,36 @@
+# Auto Instrumentation
+
+`ddtrace` can automatically instrument all available libraries, without requiring the manual setup of each one.
+
+## Rails
+
+Add the following to your `Gemfile`/`gems.rb` file:
+
+```ruby
+source 'https://rubygems.org'
+gem 'ddtrace', require: 'ddtrace/auto_instrument'
+```
+
+## Ruby
+
+Require `'ddtrace/auto_instrument'` after all gems that you'd like to instrument have been loaded:
+
+```ruby
+# Example libraries with supported integrations
+require 'sinatra'
+require 'faraday'
+require 'redis'
+
+require 'ddtrace/auto_instrument'
+```
+
+## Additional configuration
+
+You can reconfigure, override, or disable any specific integration settings by adding
+a [`Datadog.configure`](GettingStarted.md#integration-instrumentation) call after `ddtrace/auto_instrument` is activated.
+
+## Custom integrations
+
+Custom integrations that want to support auto instrumentation need to be registered
+(e.g. invoke their {Datadog::Tracing::Contrib::Registerable::ClassMethods#register_as} method) before
+the tracer requires `'ddtrace/auto_instrument'`.

data/docs/Deprecation.md

@@ -0,0 +1,8 @@
+# Deprecation
+
+Deprecated API objects are scheduled to be **removed in the next major version release**.
+
+Deprecated objects can still be safely used until the next major version.
+
+Objects will receive the deprecation tag in a minor version release,
+allowing for deprecation warnings to be surfaced to users before the next major release.

data/docs/DevelopmentGuide.md

@@ -9,6 +9,8 @@ This guide covers some of the common how-tos and technical reference material fo
      - [Writing tests](#writing-tests)
      - [Running tests](#running-tests)
      - [Checking code quality](#checking-code-quality)
+     - [Running benchmarks](#running-benchmarks)
+     - [Type checking](#type-checking)
  - [Appendix](#appendix)
      - [Writing new integrations](#writing-new-integrations)
      - [Custom transport adapters](#custom-transport-adapters)
@@ -40,9 +42,7 @@ Then within this container you can [run tests](#running-tests), or [run code qua
 
 ## Testing
 
-The test suite uses both [Minitest](https://github.com/seattlerb/minitest) and [RSpec](https://rspec.info/) tests to verify the correctness of both the core trace library and its integrations.
-
-Minitest is deprecated in favor of RSpec; all new tests should be written in RSpec, and only existing minitests should be updated.
+The test suite uses [RSpec](https://rspec.info/) tests to verify the correctness of both the core trace library and its integrations.
 
 ### Writing tests
 
@@ -66,9 +66,6 @@ Simplest way to run tests is to run `bundle exec rake ci`, which will run the en
 Run the tests for the core library with:
 
 ```
-# Run Minitest
-$ bundle exec rake test:main
-# Run RSpec
 $ bundle exec rake spec:main
 ```
 
@@ -98,6 +95,50 @@ $ bundle exec appraisal contrib rake spec:redis'[--seed,1234]'
 
 This can be useful for replicating conditions from CI or isolating certain tests.
 
+**Checking test coverage**
+
+You can check test code coverage by creating a report _after_ running a test suite:
+```
+# Run the desired test suite
+$ bundle exec appraisal contrib rake spec:redis
+# Generate report for the suite executed
+$ bundle exec rake coverage:report
+```
+
+A webpage will be generated at `coverage/report/index.html` with the resulting report.
+
+Because you are likely not running all tests locally, your report will contain partial coverage results.
+You *must* check the CI step `coverage` for the complete test coverage report, ensuring coverage is not
+decreased.
+
+**Ensuring tests don't leak resources**
+
+Tests execution can create resources that are hard to track: threads, sockets, files, etc. Because these resources can come
+from the both the test setup as well as the code under test, making sure all resources are properly disposed is important
+to prevent the application from inadvertently creating cumulative resources during its execution.
+
+When running tests that utilize threads, you might see an error message similar to this one:
+
+```
+Test leaked 1 thread: "Datadog::Workers::AsyncTransport integration tests"
+Ensure all threads are terminated when test finishes:
+1: #<Thread:0x00007fcbc99863d0 /Users/marco.costa/work/dd-trace-rb/spec/spec_helper.rb:145 sleep> (Thread)
+Thread Creation Site:
+        ./dd-trace-rb/spec/ddtrace/workers_integration_spec.rb:245:in 'new'
+        ./dd-trace-rb/spec/ddtrace/workers_integration_spec.rb:245:in 'block (4 levels) in <top (required)>'
+Thread Backtrace:
+        ./dd-trace-rb/spec/ddtrace/workers_integration_spec.rb:262:in 'sleep'
+        .dd-trace-rb/spec/ddtrace/workers_integration_spec.rb:262:in 'block (5 levels) in <top (required)>'
+        ./dd-trace-rb/spec/spec_helper.rb:147:in 'block in initialize'
+```
+
+This means that this test did not finish all threads by the time the test had finished. In this case, the thread
+creation can be traced to `workers_integration_spec.rb:245:in 'new'`. The thread itself is sleeping at `workers_integration_spec.rb:262:in 'sleep'`.
+
+The actionable in this case would be to ensure that the thread created in `workers_integration_spec.rb:245` is properly terminated by invoking `Thread#join` during the test tear down, which will wait for the thread to finish before returning.
+
+Depending on the situation, the thread in question might need to be forced to terminate. It's recommended to have a mechanism in place to terminate it (a shared variable that changes value when the thread should exit), but as a last resort, `Thread#terminate` forces the thread to finish. Keep in mind that regardless of the termination method, `Thread#join` must be called to ensure that the thread has completely finished its shutdown process.
+
 ### Checking code quality
 
 **Linting**
@@ -108,6 +149,29 @@ The trace library uses Rubocop to enforce [code style](https://github.com/bbatso
 $ bundle exec rake rubocop
 ```
 
+### Running benchmarks
+
+If your changes can have a measurable performance impact, we recommend running our benchmark suite:
+
+```
+$ bundle exec rake spec:benchmark
+```
+
+Results are printed to STDOUT as well as written to the `./tmp/benchmark/` directory.
+
+## Type checking
+
+This library uses the [Sorbet](https://sorbet.org/) type checker. Sorbet can be run with `bundle exec srb tc` (or `bundle exec rake
+typecheck`). There's also Language Server Protocol support, if your editor supports it.
+
+Type checking can be controlled on a file-by-file manner, using a `# typed: ...` comment. The default (when none is provided) is assuming `# typed: false`.
+
+Things to note:
+
+* For compatibility with older Rubies, we use Sorbet but do not yet allow type annotations in the codebase. If Sorbet is blocking you, feel free to use `# typed: false` or `# typed: ignore` with a quick note on why this was needed. In many cases, Sorbet can typecheck a file correctly with no extra type annotations.
+
+* Most integration-specific code will reference optional external dependencies which Sorbet cannot see into. You'll probably need to use `# typed: false` or `# typed: ignore`  for those files as well.
+
 ## Appendix
 
 ### Writing new integrations
@@ -125,9 +189,9 @@ To get started quickly, it's perfectly fine to copy-paste an existing integratio
 
 Once you have it working in your application, you can [add unit tests](#writing-tests), [run them locally](#running-tests), and [check for code quality](#checking-code-quality) using Docker Compose.
 
-Then [open a pull request](https://github.com/DataDog/dd-trace-rb/CONTRIBUTING.md#have-a-patch) and be sure to add the following to the description:
+Then [open a pull request](../CONTRIBUTING.md#have-a-patch) and be sure to add the following to the description:
 
- - [Documentation](https://github.com/DataDog/dd-trace-rb/docs/GettingStarted.md) for the integration, including versions supported.
+ - [Documentation](./GettingStarted.md) for the integration, including versions supported.
  - Links to the repository/website of the library being integrated
  - Screenshots showing a sample trace
  - Any additional code snippets, sample apps, benchmarks, or other resources that demonstrate its implementation are a huge plus!
@@ -183,7 +247,7 @@ Then pass an adapter instance to the tracer configuration:
 
 ```ruby
 Datadog.configure do |c|
-  c.tracer.transport_options = proc { |t|
+  c.tracing.transport_options = proc { |t|
     # By name
     t.adapter :custom