ddtrace 1.4.1 → 1.6.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '01987691359b14541248048bb2f720031b4f0f0ca0be8fade0f6d0daf7aef8cc'
-  data.tar.gz: b2cce425177f3d619d0064d88ffb52073515b8d390e054117a160eac21e939d5
+  metadata.gz: 7a13232b0a3e34d8c05f4e9cd14083d3da7899e4cd4a8a52a95495193dce81b4
+  data.tar.gz: '0975d24547f318e657422c35ef54f8f68cff797436594d30514d163b55170220'
 SHA512:
-  metadata.gz: 7232b588ee13d8a47a1fccbe39a8753a19a4e9c3b0b2dee2cb278a64b183500ec9859cfecffda8c128971bf88c93b148e6f915c2d7906139d1df050b173df42c
-  data.tar.gz: 2605640bffc586a0fd8bb38139d49fa6dad9f04bfe91a0e672cdd04a6c8a8a36896be5322a4b78ecab37175cd600bdc144109646712265b49bad4e103cc1ab5e
+  metadata.gz: da8917c06511999a28152037efa177fceba67d0329534bc8a462d17ffb151bfad0f360318262be095fc73ce42990051e6856b2cc3663317091b552d76730116c
+  data.tar.gz: 44a8062936ae2f20fdca4d524d6390825b87092d58ebb934d2e4b715f442b595be020752b3a2c0fd432d4dc3901ad611725adf4d70139c095d78e7b2b127fdd1

data/CHANGELOG.md

@@ -2,6 +2,107 @@
 
 ## [Unreleased]
 
+## [1.6.1] - 2022-11-16
+
+### Changed
+
+* Limit `redis` version support to less than 5
+
+### Fixed
+
+* [redis]: Fix frozen input for `Redis.new(...)`
+
+## [1.6.0] - 2022-11-15
+
+### Added
+
+* Trace level tags propagation in distributed tracing  ([#2260][])
+* [hanami]: Hanami 1.x instrumentation ([#2230][])
+* [pg, mysql2]: option `comment_propagation` for SQL comment propagation, default is `disabled` ([#2339][])([#2324][])
+
+### Changed
+
+* [rack, sinatra]: Squash nested spans and improve patching mechanism.<br> No need to `register Datadog::Tracing::Contrib::Sinatra::Tracer`([#2217][])
+* [rails, rack]: Fix Non-GET request method with rails exception controller ([#2317][])
+* Upgrade to libdatadog 0.9.0.1.0 ([#2302][])
+* Remove legacy profiling transport ([#2062][])
+
+### Fixed
+
+* [redis]: Fix redis instance configuration, not on `client` ([#2363][])
+```
+# Change your code from
+Datadog.configure_onto(redis.client, service_name: '...')
+# to
+Datadog.configure_onto(redis, service_name: '...')
+```
+* Allow `DD_TAGS` values to have the colon character ([#2292][])
+* Ensure that `TraceSegment` can be reported correctly when they are dropped ([#2335][])
+* Docs: Fixes upgrade guide on configure_onto ([#2307][])
+* Fix environment logger with IO transport ([#2313][])
+
+## [1.5.2] - 2022-10-27
+
+### Deprecation notice
+
+- `DD_TRACE_CLIENT_IP_HEADER_DISABLED` was changed to `DD_TRACE_CLIENT_IP_ENABLED`. Although the former still works we encourage usage of the latter instead.
+
+### Changed
+
+- `http.client_ip` tag collection is made opt-in for APM. Note that `http.client_ip` is always collected when ASM is enabled as part of the security service provided ([#2321][], [#2331][])
+
+### Fixed
+
+- Handle REQUEST_URI with base url ([#2328][], [#2330][])
+
+## [1.5.1] - 2022-10-19
+
+### Changed
+
+* Update libddwaf to 1.5.1 ([#2306][])
+* Improve libddwaf extension memory management ([#2306][])
+
+### Fixed
+
+* Fix `URI::InvalidURIError` ([#2310][], [#2318][]) ([@yujideveloper][])
+* Handle URLs with invalid characters ([#2311][], [#2319][])
+* Fix missing appsec.event tag ([#2306][])
+* Fix missing Rack and Rails request body parsing for AppSec analysis ([#2306][])
+* Fix unneeded AppSec call in a Rack context when AppSec is disabled ([#2306][])
+* Fix spurious AppSec instrumentation ([#2306][])
+
+## [1.5.0] - 2022-09-29
+
+### Deprecation notice
+
+* `c.tracing.instrument :rack, { quantize: { base: ... } }` will change its default from `:exclude` to `:show` in a future version. Voluntarily moving to `:show` is recommended.
+* `c.tracing.instrument :rack, { quantize: { query: { show: ... } }` will change its default to `:all` in a future version, together with `quantize.query.obfuscate` changing to `:internal`. Voluntarily moving to these future values is recommended.
+
+### Added
+
+* Feature: Single Span Sampling ([#2128][])
+* Add query string automatic redaction ([#2283][])
+* Use full URL in `http.url` tag ([#2265][])
+* Add `http.useragent` tag ([#2252][])
+* Add `http.client_ip` tag for Rack-based frameworks ([#2248][])
+* Ci-app: CI: Fetch committer and author in Bitrise ([#2258][])
+
+### Changed
+
+* Bump allowed version of debase-ruby_core_source to include v0.10.17 ([#2267][])
+
+### Fixed
+
+* Bug: Fix `service_nam` typo to `service_name` ([#2296][])
+* Bug: Check AppSec Rails for railties instead of rails meta gem ([#2293][]) ([@seuros][])
+* Ci-app: Correctly extract commit message from AppVeyor ([#2257][])
+
+## [1.4.2] - 2022-09-27
+
+### Fixed
+
+OpenTracing context propagation ([#2191][], [#2289][])
+
 ## [1.4.1] - 2022-09-15
 
 ### Fixed
@@ -2097,7 +2198,13 @@ Release notes: https://github.com/DataDog/dd-trace-rb/releases/tag/v0.3.1
 
 Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 
-[Unreleased]: https://github.com/DataDog/dd-trace-rb/compare/v1.4.1...master
+[Unreleased]: https://github.com/DataDog/dd-trace-rb/compare/v1.6.1...master
+[1.6.1]: https://github.com/DataDog/dd-trace-rb/compare/v1.6.0...v1.6.1
+[1.6.0]: https://github.com/DataDog/dd-trace-rb/compare/v1.5.2...v1.6.0
+[1.5.2]: https://github.com/DataDog/dd-trace-rb/compare/v1.5.1...v1.5.2
+[1.5.1]: https://github.com/DataDog/dd-trace-rb/compare/v1.5.0...v1.5.1
+[1.5.0]: https://github.com/DataDog/dd-trace-rb/compare/v1.4.2...v1.5.0
+[1.4.1]: https://github.com/DataDog/dd-trace-rb/compare/v1.4.1...v1.4.2
 [1.4.1]: https://github.com/DataDog/dd-trace-rb/compare/v1.4.0...v1.4.1
 [1.4.0]: https://github.com/DataDog/dd-trace-rb/compare/v1.3.0...v1.4.0
 [1.3.0]: https://github.com/DataDog/dd-trace-rb/compare/v1.2.0...v1.3.0
@@ -2952,6 +3059,7 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 [#2054]: https://github.com/DataDog/dd-trace-rb/issues/2054
 [#2059]: https://github.com/DataDog/dd-trace-rb/issues/2059
 [#2061]: https://github.com/DataDog/dd-trace-rb/issues/2061
+[#2062]: https://github.com/DataDog/dd-trace-rb/issues/2062
 [#2066]: https://github.com/DataDog/dd-trace-rb/issues/2066
 [#2069]: https://github.com/DataDog/dd-trace-rb/issues/2069
 [#2070]: https://github.com/DataDog/dd-trace-rb/issues/2070
@@ -2966,6 +3074,7 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 [#2113]: https://github.com/DataDog/dd-trace-rb/issues/2113
 [#2118]: https://github.com/DataDog/dd-trace-rb/issues/2118
 [#2125]: https://github.com/DataDog/dd-trace-rb/issues/2125
+[#2128]: https://github.com/DataDog/dd-trace-rb/issues/2128
 [#2134]: https://github.com/DataDog/dd-trace-rb/issues/2134
 [#2138]: https://github.com/DataDog/dd-trace-rb/issues/2138
 [#2140]: https://github.com/DataDog/dd-trace-rb/issues/2140
@@ -2978,12 +3087,44 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 [#2173]: https://github.com/DataDog/dd-trace-rb/issues/2173
 [#2174]: https://github.com/DataDog/dd-trace-rb/issues/2174
 [#2180]: https://github.com/DataDog/dd-trace-rb/issues/2180
+[#2191]: https://github.com/DataDog/dd-trace-rb/issues/2191
 [#2200]: https://github.com/DataDog/dd-trace-rb/issues/2200
 [#2201]: https://github.com/DataDog/dd-trace-rb/issues/2201
+[#2217]: https://github.com/DataDog/dd-trace-rb/issues/2217
 [#2219]: https://github.com/DataDog/dd-trace-rb/issues/2219
 [#2229]: https://github.com/DataDog/dd-trace-rb/issues/2229
+[#2230]: https://github.com/DataDog/dd-trace-rb/issues/2230
+[#2248]: https://github.com/DataDog/dd-trace-rb/issues/2248
 [#2250]: https://github.com/DataDog/dd-trace-rb/issues/2250
+[#2252]: https://github.com/DataDog/dd-trace-rb/issues/2252
+[#2257]: https://github.com/DataDog/dd-trace-rb/issues/2257
+[#2258]: https://github.com/DataDog/dd-trace-rb/issues/2258
+[#2260]: https://github.com/DataDog/dd-trace-rb/issues/2260
+[#2265]: https://github.com/DataDog/dd-trace-rb/issues/2265
+[#2267]: https://github.com/DataDog/dd-trace-rb/issues/2267
 [#2279]: https://github.com/DataDog/dd-trace-rb/issues/2279
+[#2283]: https://github.com/DataDog/dd-trace-rb/issues/2283
+[#2289]: https://github.com/DataDog/dd-trace-rb/issues/2289
+[#2292]: https://github.com/DataDog/dd-trace-rb/issues/2292
+[#2293]: https://github.com/DataDog/dd-trace-rb/issues/2293
+[#2296]: https://github.com/DataDog/dd-trace-rb/issues/2296
+[#2302]: https://github.com/DataDog/dd-trace-rb/issues/2302
+[#2306]: https://github.com/DataDog/dd-trace-rb/issues/2306
+[#2307]: https://github.com/DataDog/dd-trace-rb/issues/2307
+[#2310]: https://github.com/DataDog/dd-trace-rb/issues/2310
+[#2311]: https://github.com/DataDog/dd-trace-rb/issues/2311
+[#2313]: https://github.com/DataDog/dd-trace-rb/issues/2313
+[#2317]: https://github.com/DataDog/dd-trace-rb/issues/2317
+[#2318]: https://github.com/DataDog/dd-trace-rb/issues/2318
+[#2319]: https://github.com/DataDog/dd-trace-rb/issues/2319
+[#2321]: https://github.com/DataDog/dd-trace-rb/issues/2321
+[#2324]: https://github.com/DataDog/dd-trace-rb/issues/2324
+[#2328]: https://github.com/DataDog/dd-trace-rb/issues/2328
+[#2330]: https://github.com/DataDog/dd-trace-rb/issues/2330
+[#2331]: https://github.com/DataDog/dd-trace-rb/issues/2331
+[#2335]: https://github.com/DataDog/dd-trace-rb/issues/2335
+[#2339]: https://github.com/DataDog/dd-trace-rb/issues/2339
+[#2363]: https://github.com/DataDog/dd-trace-rb/issues/2363
 [@AdrianLC]: https://github.com/AdrianLC
 [@Azure7111]: https://github.com/Azure7111
 [@BabyGroot]: https://github.com/BabyGroot
@@ -3105,6 +3246,7 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 [@saturnflyer]: https://github.com/saturnflyer
 [@sco11morgan]: https://github.com/sco11morgan
 [@senny]: https://github.com/senny
+[@seuros]: https://github.com/seuros
 [@shayonj]: https://github.com/shayonj
 [@sinsoku]: https://github.com/sinsoku
 [@skcc321]: https://github.com/skcc321
@@ -3124,5 +3266,6 @@ Git diff: https://github.com/DataDog/dd-trace-rb/compare/v0.3.0...v0.3.1
 [@vramaiah]: https://github.com/vramaiah
 [@walterking]: https://github.com/walterking
 [@y-yagi]: https://github.com/y-yagi
+[@yujideveloper]: https://github.com/yujideveloper
 [@yukimurasawa]: https://github.com/yukimurasawa
 [@zachmccormick]: https://github.com/zachmccormick

data/LICENSE-3rdparty.csv

@@ -1,6 +1,7 @@
 Component,Origin,License,Copyright
 lib/datadog/core/vendor/multipart-post,https://github.com/socketry/multipart-post,MIT,"Copyright (c) 2007-2013 Nick Sieger."
 lib/datadog/tracing/contrib/active_record/vendor,https://github.com/rails/rails/,MIT,"Copyright (c) 2005-2018 David Heinemeier Hansson"
+lib/datadog/tracing/contrib/utils/quantization/http.rb,https://github.com/ruby/uri,BSD-2-Clause,"Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved."
 ext/ddtrace_profiling_native_extension/private_vm_api_access,https://github.com/ruby/ruby,BSD-2-Clause,"Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved."
 msgpack,https://rubygems.org/gems/msgpack,Apache-2.0,"Copyright (c) 2008-2015 Sadayuki Furuhashi"
 debase-ruby_core_source,https://rubygems.org/gems/debase-ruby_core_source,MIT for gem and BSD-2-Clause for Ruby sources,"Copyright (c) 2012 Gabriel Horner. Files from Ruby sources are Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved."

data/ext/ddtrace_profiling_loader/ddtrace_profiling_loader.c

@@ -29,10 +29,17 @@
   #define RTLD_DEEPBIND 0
 #endif
 
+// Used to mark function arguments that are deliberately left unused
+#ifdef __GNUC__
+  #define DDTRACE_UNUSED  __attribute__((unused))
+#else
+  #define DDTRACE_UNUSED
+#endif
+
 static VALUE ok_symbol = Qnil; // :ok in Ruby
 static VALUE error_symbol = Qnil; // :error in Ruby
 
-static VALUE _native_load(VALUE self, VALUE ruby_path, VALUE ruby_init_name);
+static VALUE _native_load(DDTRACE_UNUSED VALUE self, VALUE ruby_path, VALUE ruby_init_name);
 static bool failed_to_load(void *handle, VALUE *failure_details);
 static bool incompatible_library(void *handle, VALUE *failure_details);
 static bool failed_to_initialize(void *handle, char *init_name, VALUE *failure_details);
@@ -51,7 +58,7 @@ void DDTRACE_EXPORT Init_ddtrace_profiling_loader(void) {
   error_symbol = ID2SYM(rb_intern_const("error"));
 }
 
-static VALUE _native_load(VALUE self, VALUE ruby_path, VALUE ruby_init_name) {
+static VALUE _native_load(DDTRACE_UNUSED VALUE self, VALUE ruby_path, VALUE ruby_init_name) {
   Check_Type(ruby_path, T_STRING);
   Check_Type(ruby_init_name, T_STRING);
 

data/ext/ddtrace_profiling_loader/extconf.rb

@@ -24,6 +24,16 @@ def add_compiler_flag(flag)
   end
 end
 
+# Because we can't control what compiler versions our customers use, shipping with -Werror by default is a no-go.
+# But we can enable it in CI, so that we quickly spot any new warnings that just got introduced.
+add_compiler_flag '-Werror' if ENV['CI'] == 'true'
+
+# Older gcc releases may not default to C99 and we need to ask for this. This is also used:
+# * by upstream Ruby -- search for gnu99 in the codebase
+# * by msgpack, another ddtrace dependency
+#   (https://github.com/msgpack/msgpack-ruby/blob/18ce08f6d612fe973843c366ac9a0b74c4e50599/ext/msgpack/extconf.rb#L8)
+add_compiler_flag '-std=gnu99'
+
 # Gets really noisy when we include the MJIT header, let's omit it
 add_compiler_flag '-Wno-unused-function'
 
@@ -34,6 +44,9 @@ add_compiler_flag '-Wno-declaration-after-statement'
 # cause a segfault later. Let's ensure that never happens.
 add_compiler_flag '-Werror-implicit-function-declaration'
 
+# Warn on unused parameters to functions. Use `DDTRACE_UNUSED` to mark things as known-to-not-be-used.
+add_compiler_flag '-Wunused-parameter'
+
 # The native extension is not intended to expose any symbols/functions for other native libraries to use;
 # the sole exception being `Init_ddtrace_profiling_loader` which needs to be visible for Ruby to call it when
 # it `dlopen`s the library.
@@ -42,6 +55,10 @@ add_compiler_flag '-Werror-implicit-function-declaration'
 # For more details see https://gcc.gnu.org/wiki/Visibility
 add_compiler_flag '-fvisibility=hidden'
 
+# Enable all other compiler warnings
+add_compiler_flag '-Wall'
+add_compiler_flag '-Wextra'
+
 # Tag the native extension library with the Ruby version and Ruby platform.
 # This makes it easier for development (avoids "oops I forgot to rebuild when I switched my Ruby") and ensures that
 # the wrong library is never loaded.

data/ext/ddtrace_profiling_native_extension/NativeExtensionDesign.md

@@ -1,6 +1,7 @@
 # Profiling Native Extension Design
 
 The profiling native extension is used to:
+
 1. Implement features which are expensive (in terms of resources) or otherwise impossible to implement using Ruby code.
 2. Bridge between Ruby-specific profiling features and [`libdatadog`](https://github.com/DataDog/libdatadog), a Rust-based
 library with common profiling functionality.
@@ -20,8 +21,8 @@ and disabling the extension will disable profiling.
 
 ## Who is this page for?
 
-This documentation is intended to be used by dd-trace-rb developers. Please see the `docs/` folder for user-level
-documentation.
+**This documentation is intended to be used by dd-trace-rb developers. Please see the `docs/` folder for user-level
+documentation.**
 
 ## Must not block or break users that cannot use it
 
@@ -118,3 +119,38 @@ Note that `pthread_getcpuclockid()` is not available on macOS (nor, obviously, o
 is currently Linux-specific. Thus, in the <clock_id_from_pthread.c> file we implement the feature for supported Ruby
 setups but if something is missing we instead compile in <clock_id_noop.c> that includes a no-op implementation of the
 feature.
+
+## Fork-safety
+
+It's common for Ruby applications to create child processes via the use of `fork`. For instance, this strategy is used
+by the puma webserver and the resque job processing tool.
+
+Thus, the profiler needs to be designed to take this into account. I'll call out two important parts of this design:
+
+1. Automatically propagate profiler to child processes. To make onboarding easier, we monkey patch the Ruby `fork` APIs
+so that the profiler is automatically restarted in child processes. This way, the user only needs to start profiling at
+the beginning of their application, and automatically forks are profiled as well.
+
+2. The profiler must ensure correctness and stability even if the application forks. There must be no impact on the
+application or incorrect data generated.
+
+### Fork-safety for libdatadog
+
+Since libdatadog is built in native code (Rust), special care needs to be take to consider how we're using it and how
+it can be affected by the use of `fork`.
+
+* Profile-related APIs: `Profile_new` and `Profile_add` and `Profile_free` are only called with the Ruby Global VM Lock
+being held. Thus, if Ruby APIs are being used for fork, this prevents any concurrency between profile mutation and
+forking, because if we’re holding the lock, then no other thread can call into the fork APIs.
+(Calling libc `fork()` directly from a native extension is possible but would break the VM as well, since it does need
+to do some of its own work when forking happens, so we’ll ignore that one)
+
+* Exporter-related APIs: Explicitly to make sure we had no issues with forking, we create a new `CancellationToken_new`
+and `ProfileExporterV3_new` for every report. We do release the Global VM Lock during exporting, so it's possible for
+forking and exporting to be concurrent.
+
+  Both the CancellationToken and ProfileExporter are only referenced on the stack of the thread doing the exporting, so
+  they will not be reused in the child process after the fork. In the worst case, if a report is concurrent with a fork,
+  then it's possible a small amount of memory will not be cleaned up in the child process.
+
+  Because there is no leftover undefined state, we guarantee correctness for the exporter APIs.

data/ext/ddtrace_profiling_native_extension/clock_id.h

@@ -20,5 +20,6 @@ void self_test_clock_id(void);
 // TODO: Remove this after the OldStack profiler gets removed
 VALUE clock_id_for(VALUE self, VALUE thread);
 
+// Safety: This function is assumed never to raise exceptions by callers
 thread_cpu_time_id thread_cpu_time_id_for(VALUE thread);
 thread_cpu_time thread_cpu_time_for(thread_cpu_time_id time_id);

data/ext/ddtrace_profiling_native_extension/clock_id_from_pthread.c

@@ -43,6 +43,7 @@ VALUE clock_id_for(DDTRACE_UNUSED VALUE _self, VALUE thread) {
   }
 }
 
+// Safety: This function is assumed never to raise exceptions by callers
 thread_cpu_time_id thread_cpu_time_id_for(VALUE thread) {
   rb_nativethread_id_t thread_id = pthread_id_for(thread);
   clockid_t clock_id;