elastic-apm 3.3.0 → 3.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. checksums.yaml +4 -4
  2. data/.ci/.jenkins_exclude.yml +4 -4
  3. data/.ci/.jenkins_ruby.yml +1 -1
  4. data/.ci/Jenkinsfile +5 -3
  5. data/.ci/jobs/apm-agent-ruby-downstream.yml +1 -0
  6. data/.ci/jobs/apm-agent-ruby-linting-mbp.yml +1 -0
  7. data/.ci/jobs/apm-agent-ruby-mbp.yml +1 -0
  8. data/.ci/prepare-git-context.sh +5 -2
  9. data/.github/ISSUE_TEMPLATE/Bug_report.md +38 -0
  10. data/.github/ISSUE_TEMPLATE/Feature_request.md +17 -0
  11. data/.github/PULL_REQUEST_TEMPLATE.md +14 -0
  12. data/.gitignore +3 -0
  13. data/.rubocop.yml +6 -0
  14. data/CHANGELOG.asciidoc +25 -1
  15. data/Gemfile +6 -2
  16. data/bench/sql.rb +49 -0
  17. data/bin/build_docs +1 -1
  18. data/codecov.yml +32 -0
  19. data/docs/api.asciidoc +37 -0
  20. data/docs/configuration.asciidoc +18 -1
  21. data/docs/supported-technologies.asciidoc +20 -1
  22. data/lib/elastic_apm.rb +29 -5
  23. data/lib/elastic_apm/agent.rb +6 -2
  24. data/lib/elastic_apm/child_durations.rb +9 -4
  25. data/lib/elastic_apm/config.rb +8 -1
  26. data/lib/elastic_apm/config/options.rb +3 -4
  27. data/lib/elastic_apm/context/response.rb +10 -2
  28. data/lib/elastic_apm/instrumenter.rb +20 -11
  29. data/lib/elastic_apm/normalizers/rails/active_record.rb +12 -5
  30. data/lib/elastic_apm/rails.rb +1 -10
  31. data/lib/elastic_apm/railtie.rb +1 -1
  32. data/lib/elastic_apm/span.rb +3 -2
  33. data/lib/elastic_apm/span/context.rb +26 -44
  34. data/lib/elastic_apm/span/context/db.rb +19 -0
  35. data/lib/elastic_apm/span/context/destination.rb +44 -0
  36. data/lib/elastic_apm/span/context/http.rb +26 -0
  37. data/lib/elastic_apm/spies/elasticsearch.rb +18 -5
  38. data/lib/elastic_apm/spies/faraday.rb +36 -18
  39. data/lib/elastic_apm/spies/http.rb +16 -2
  40. data/lib/elastic_apm/spies/mongo.rb +5 -0
  41. data/lib/elastic_apm/spies/net_http.rb +27 -7
  42. data/lib/elastic_apm/spies/sequel.rb +25 -15
  43. data/lib/elastic_apm/spies/shoryuken.rb +48 -0
  44. data/lib/elastic_apm/spies/sneakers.rb +57 -0
  45. data/lib/elastic_apm/sql.rb +19 -0
  46. data/lib/elastic_apm/sql/signature.rb +152 -0
  47. data/lib/elastic_apm/sql/tokenizer.rb +247 -0
  48. data/lib/elastic_apm/sql/tokens.rb +46 -0
  49. data/lib/elastic_apm/sql_summarizer.rb +1 -2
  50. data/lib/elastic_apm/transaction.rb +11 -11
  51. data/lib/elastic_apm/transport/connection/proxy_pipe.rb +2 -2
  52. data/lib/elastic_apm/transport/headers.rb +4 -0
  53. data/lib/elastic_apm/transport/serializers/span_serializer.rb +24 -7
  54. data/lib/elastic_apm/version.rb +1 -1
  55. metadata +16 -3
  56. data/.github/workflows/main.yml +0 -14
@@ -145,7 +145,7 @@ One example to generate a secure secret token is:
145
145
  ruby -r securerandom -e 'print SecureRandom.uuid'
146
146
  ----
147
147
 
148
- WARNING: Secret tokens only provide any real security if your APM server use TLS.
148
+ WARNING: Secret tokens only provide any real security if your APM server uses TLS.
149
149
 
150
150
  [float]
151
151
  [[config-active]]
@@ -726,6 +726,23 @@ between `0.0` and `1.0`.
726
726
  We still record overall time and the result for unsampled transactions, but no
727
727
  context information, tags, or spans.
728
728
 
729
+ [float]
730
+ [[config-use-experimental-sql-parser]]
731
+ ==== `use_experimental_sql_parser`
732
+ |============
733
+ | Environment | `Config` key | Default
734
+ | `ELASTIC_APM_USE_EXPERIMENTAL_SQL_PARSER` | `use_experimental_sql_parser` | `false`
735
+ |============
736
+
737
+ Use a newer, more precise but still experimental approach to generating summaries of
738
+ your app's SQL statements.
739
+ Without this, your SQL statements will still be summarized albeit less accurately.
740
+
741
+ The summaries become the spans' names -- what it says in the waterfall view in Kibana.
742
+
743
+ NOTE: This should work just fine but is still deamed experimental. That means it is
744
+ subject to change. Please let us know if you come across any issues.
745
+
729
746
  [float]
730
747
  [[config-verify-server-cert]]
731
748
  ==== `verify_server_cert`
@@ -26,7 +26,7 @@ https://www.ruby-lang.org/en/downloads/branches/[Ruby Maintenance Branches].
26
26
  We have automatic support for Ruby on Rails and all Rack compatible web
27
27
  frameworks.
28
28
 
29
- We test against all supported minor versions of Rails and Sinatra.
29
+ We test against all supported minor versions of Rails, Sinatra, and Grape.
30
30
 
31
31
  [float]
32
32
  [[supported-technologies-rails]]
@@ -45,6 +45,14 @@ We currently support all versions of Sinatra since 1.0.
45
45
 
46
46
  See <<getting-started-rack>>.
47
47
 
48
+ [float]
49
+ [[supported-technologies-grape]]
50
+ ==== Grape
51
+
52
+ We currently support all versions of Grape since 1.2.
53
+
54
+ See <<getting-started-grape>>.
55
+
48
56
  [float]
49
57
  [[supported-technologies-databases]]
50
58
  === Databases
@@ -67,3 +75,14 @@ requests using these libraries:
67
75
  - `net/http`
68
76
  - Http.rb (v0.6+)
69
77
  - Faraday (v0.2.1+)
78
+
79
+ [float]
80
+ [[supported-technologies-backgroud-processing]]
81
+ === Background Processing
82
+
83
+ We automatically instrument background processing using:
84
+
85
+ - DelayedJob
86
+ - Sidekiq
87
+ - Shoryuken
88
+ - Sneakers (v2.12.0+) (Experimental, see {pull}676[#676])
@@ -97,6 +97,8 @@ module ElasticAPM
97
97
  # @param type [String] The kind of the transaction, eg `app.request.get` or
98
98
  # `db.mysql2.query`
99
99
  # @param context [Context] An optional [Context]
100
+ # @param trace_context [TraceContext] An optional [TraceContext] object for
101
+ # Distributed Tracing.
100
102
  # @return [Transaction]
101
103
  def start_transaction(
102
104
  name = nil,
@@ -127,6 +129,8 @@ module ElasticAPM
127
129
  # @param type [String] The kind of the transaction, eg `app.request.get` or
128
130
  # `db.mysql2.query`
129
131
  # @param context [Context] An optional [Context]
132
+ # @param trace_context [TraceContext] An optional [TraceContext] object for
133
+ # Distributed Tracing.
130
134
  # @yield [Transaction]
131
135
  # @return result of block
132
136
  def with_transaction(
@@ -165,6 +169,11 @@ module ElasticAPM
165
169
  # @param action [String] The span action type, eq `connect` or `query`
166
170
  # @param context [Span::Context] Context information about the span
167
171
  # @param include_stacktrace [Boolean] Whether or not to capture a stacktrace
172
+ # @param trace_context [TraceContext] An optional [TraceContext] object for
173
+ # Distributed Tracing.
174
+ # @param parent [Transaction,Span] The parent transaction or span.
175
+ # Relevant when the span is created in another thread.
176
+ # @param sync [Boolean] Whether the span is created synchronously or not.
168
177
  # @return [Span]
169
178
  def start_span(
170
179
  name,
@@ -173,7 +182,9 @@ module ElasticAPM
173
182
  action: nil,
174
183
  context: nil,
175
184
  include_stacktrace: true,
176
- trace_context: nil
185
+ trace_context: nil,
186
+ parent: nil,
187
+ sync: nil
177
188
  )
178
189
  agent&.start_span(
179
190
  name,
@@ -181,7 +192,9 @@ module ElasticAPM
181
192
  subtype: subtype,
182
193
  action: action,
183
194
  context: context,
184
- trace_context: trace_context
195
+ trace_context: trace_context,
196
+ parent: parent,
197
+ sync: sync
185
198
  ).tap do |span|
186
199
  break unless span && include_stacktrace
187
200
  break unless agent.config.span_frames_min_duration?
@@ -202,9 +215,16 @@ module ElasticAPM
202
215
  # Wrap a block in a Span, ending it after the block
203
216
  #
204
217
  # @param name [String] A description of the span, eq `SELECT FROM "users"`
205
- # @param type [String] The kind of span, eq `db.mysql2.query`
218
+ # @param type [String] The kind of span, eq `db`
219
+ # @param subtype [String] The subtype of span eg. `postgresql`.
220
+ # @param action [String] The action type of span eg. `connect` or `query`.
206
221
  # @param context [Span::Context] Context information about the span
207
222
  # @param include_stacktrace [Boolean] Whether or not to capture a stacktrace
223
+ # @param trace_context [TraceContext] An optional [TraceContext] object for
224
+ # Distributed Tracing.
225
+ # @param parent [Transaction,Span] The parent transaction or span.
226
+ # Relevant when the span is created in another thread.
227
+ # @param sync [Boolean] Whether the span is created synchronously or not.
208
228
  # @yield [Span]
209
229
  # @return Result of block
210
230
  def with_span(
@@ -214,7 +234,9 @@ module ElasticAPM
214
234
  action: nil,
215
235
  context: nil,
216
236
  include_stacktrace: true,
217
- trace_context: nil
237
+ trace_context: nil,
238
+ parent: nil,
239
+ sync: nil
218
240
  )
219
241
  unless block_given?
220
242
  raise ArgumentError,
@@ -232,7 +254,9 @@ module ElasticAPM
232
254
  action: action,
233
255
  context: context,
234
256
  include_stacktrace: include_stacktrace,
235
- trace_context: trace_context
257
+ trace_context: trace_context,
258
+ parent: parent,
259
+ sync: sync
236
260
  )
237
261
  yield span
238
262
  ensure
@@ -165,7 +165,9 @@ module ElasticAPM
165
165
  action: nil,
166
166
  backtrace: nil,
167
167
  context: nil,
168
- trace_context: nil
168
+ trace_context: nil,
169
+ parent: nil,
170
+ sync: nil
169
171
  )
170
172
  instrumenter.start_span(
171
173
  name,
@@ -174,7 +176,9 @@ module ElasticAPM
174
176
  action: action,
175
177
  backtrace: backtrace,
176
178
  context: context,
177
- trace_context: trace_context
179
+ trace_context: trace_context,
180
+ parent: parent,
181
+ sync: sync
178
182
  )
179
183
  end
180
184
  # rubocop:enable Metrics/ParameterLists
@@ -24,18 +24,23 @@ module ElasticAPM
24
24
  @nesting_level = 0
25
25
  @start = nil
26
26
  @duration = 0
27
+ @mutex = Mutex.new
27
28
  end
28
29
 
29
30
  attr_reader :duration
30
31
 
31
32
  def start
32
- @nesting_level += 1
33
- @start = Util.micros if @nesting_level == 1
33
+ @mutex.synchronize do
34
+ @nesting_level += 1
35
+ @start = Util.micros if @nesting_level == 1
36
+ end
34
37
  end
35
38
 
36
39
  def stop
37
- @nesting_level -= 1
38
- @duration = (Util.micros - @start) if @nesting_level == 0
40
+ @mutex.synchronize do
41
+ @nesting_level -= 1
42
+ @duration = (Util.micros - @start) if @nesting_level == 0
43
+ end
39
44
  end
40
45
  end
41
46
  end
@@ -17,6 +17,7 @@ module ElasticAPM
17
17
  option :config_file, type: :string, default: 'config/elastic_apm.yml'
18
18
  option :server_url, type: :url, default: 'http://localhost:8200'
19
19
  option :secret_token, type: :string
20
+ option :api_key, type: :string
20
21
 
21
22
  option :active, type: :bool, default: true
22
23
  option :api_buffer_size, type: :int, default: 256
@@ -69,6 +70,9 @@ module ElasticAPM
69
70
  option :transaction_max_spans, type: :int, default: 500
70
71
  option :transaction_sample_rate, type: :float, default: 1.0
71
72
  option :verify_server_cert, type: :bool, default: true
73
+
74
+ option :use_experimental_sql_parser, type: :bool, default: false
75
+
72
76
  # rubocop:enable Metrics/LineLength, Layout/ExtraSpacing
73
77
  def initialize(options = {})
74
78
  @options = load_schema
@@ -105,6 +109,7 @@ module ElasticAPM
105
109
 
106
110
  def available_instrumentations
107
111
  %w[
112
+ action_dispatch
108
113
  delayed_job
109
114
  elasticsearch
110
115
  faraday
@@ -112,12 +117,14 @@ module ElasticAPM
112
117
  json
113
118
  mongo
114
119
  net_http
120
+ rake
115
121
  redis
116
122
  sequel
123
+ shoryuken
117
124
  sidekiq
118
125
  sinatra
126
+ sneakers
119
127
  tilt
120
- rake
121
128
  ]
122
129
  end
123
130
 
@@ -83,9 +83,8 @@ module ElasticAPM
83
83
  @schema ||= {}
84
84
  end
85
85
 
86
- def option(*args)
87
- key = args.shift
88
- schema[key] = *args
86
+ def option(key, **args)
87
+ schema[key] = args
89
88
  end
90
89
  end
91
90
 
@@ -93,7 +92,7 @@ module ElasticAPM
93
92
  module InstanceMethods
94
93
  def load_schema
95
94
  Hash[self.class.schema.map do |key, args|
96
- [key, Option.new(key, *args)]
95
+ [key, Option.new(key, **args)]
97
96
  end]
98
97
  end
99
98
 
@@ -11,12 +11,20 @@ module ElasticAPM
11
11
  finished: true
12
12
  )
13
13
  @status_code = status_code
14
- @headers = headers
15
14
  @headers_sent = headers_sent
16
15
  @finished = finished
16
+
17
+ self.headers = headers
17
18
  end
18
19
 
19
- attr_accessor :status_code, :headers, :headers_sent, :finished
20
+ attr_accessor :status_code, :headers_sent, :finished
21
+ attr_reader :headers
22
+
23
+ def headers=(headers)
24
+ @headers = headers.each_with_object({}) do |(k, v), hsh|
25
+ hsh[k] = v.to_s
26
+ end
27
+ end
20
28
  end
21
29
  end
22
30
  end
@@ -136,6 +136,7 @@ module ElasticAPM
136
136
  end
137
137
 
138
138
  # rubocop:disable Metrics/CyclomaticComplexity
139
+ # rubocop:disable Metrics/PerceivedComplexity
139
140
  # rubocop:disable Metrics/ParameterLists
140
141
  def start_span(
141
142
  name,
@@ -144,19 +145,25 @@ module ElasticAPM
144
145
  action: nil,
145
146
  backtrace: nil,
146
147
  context: nil,
147
- trace_context: nil
148
+ trace_context: nil,
149
+ parent: nil,
150
+ sync: nil
148
151
  )
149
- return unless (transaction = current_transaction)
150
- return unless transaction.sampled?
151
-
152
- transaction.inc_started_spans!
153
152
 
154
- if transaction.max_spans_reached?
155
- transaction.inc_dropped_spans!
156
- return
157
- end
153
+ transaction =
154
+ case parent
155
+ when Span
156
+ parent.transaction
157
+ when Transaction
158
+ parent
159
+ else
160
+ current_transaction
161
+ end
162
+ return unless transaction
163
+ return unless transaction.sampled?
164
+ return unless transaction.inc_started_spans!
158
165
 
159
- parent = current_span || transaction
166
+ parent ||= (current_span || current_transaction)
160
167
 
161
168
  span = Span.new(
162
169
  name: name,
@@ -167,7 +174,8 @@ module ElasticAPM
167
174
  trace_context: trace_context,
168
175
  type: type,
169
176
  context: context,
170
- stacktrace_builder: stacktrace_builder
177
+ stacktrace_builder: stacktrace_builder,
178
+ sync: sync
171
179
  )
172
180
 
173
181
  if backtrace && transaction.config.span_frames_min_duration?
@@ -180,6 +188,7 @@ module ElasticAPM
180
188
  end
181
189
  # rubocop:enable Metrics/ParameterLists
182
190
  # rubocop:enable Metrics/CyclomaticComplexity
191
+ # rubocop:enable Metrics/PerceivedComplexity
183
192
 
184
193
  def end_span
185
194
  return unless (span = current_spans.pop)
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'elastic_apm/sql_summarizer'
3
+ require 'elastic_apm/sql'
4
4
 
5
5
  module ElasticAPM
6
6
  module Normalizers
@@ -17,7 +17,8 @@ module ElasticAPM
17
17
  def initialize(*args)
18
18
  super
19
19
 
20
- @summarizer = SqlSummarizer.new
20
+ @summarizer = Sql.summarizer
21
+
21
22
  @adapters = {}
22
23
  end
23
24
 
@@ -25,14 +26,20 @@ module ElasticAPM
25
26
  return :skip if SKIP_NAMES.include?(payload[:name])
26
27
 
27
28
  name = summarize(payload[:sql]) || payload[:name]
29
+ subtype = subtype_for(payload)
30
+
28
31
  context =
29
- Span::Context.new(db: { statement: payload[:sql], type: 'sql' })
30
- [name, TYPE, subtype(payload), ACTION, context]
32
+ Span::Context.new(
33
+ db: { statement: payload[:sql], type: 'sql' },
34
+ destination: { name: subtype, resource: subtype, type: TYPE }
35
+ )
36
+
37
+ [name, TYPE, subtype, ACTION, context]
31
38
  end
32
39
 
33
40
  private
34
41
 
35
- def subtype(payload)
42
+ def subtype_for(payload)
36
43
  cached_adapter_name(
37
44
  payload[:connection]&.adapter_name ||
38
45
  ::ActiveRecord::Base.connection_config[:adapter]
@@ -9,7 +9,6 @@ module ElasticAPM
9
9
  # It is recommended to use the Railtie instead.
10
10
  module Rails
11
11
  extend self
12
- # rubocop:disable Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
13
12
  # Start the ElasticAPM agent and hook into Rails.
14
13
  # Note that the agent won't be started if the Rails console is being used.
15
14
  #
@@ -31,13 +30,6 @@ module ElasticAPM
31
30
  attach_subscriber(agent)
32
31
  end
33
32
 
34
- if ElasticAPM.running? &&
35
- !ElasticAPM.agent.config.disabled_instrumentations.include?(
36
- 'action_dispatch'
37
- )
38
- require 'elastic_apm/spies/action_dispatch'
39
- end
40
-
41
33
  ElasticAPM.running?
42
34
  rescue StandardError => e
43
35
  if config.disable_start_message?
@@ -48,12 +40,11 @@ module ElasticAPM
48
40
  puts "Backtrace:\n" + e.backtrace.join("\n")
49
41
  end
50
42
  end
51
- # rubocop:enable Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
52
43
 
53
44
  private
54
45
 
55
46
  def should_skip?(_config)
56
- if ::Rails.const_defined? 'Rails::Console'
47
+ if ::Rails.const_defined?('Console', false)
57
48
  return 'Rails console'
58
49
  end
59
50
 
@@ -7,7 +7,7 @@ module ElasticAPM
7
7
 
8
8
  Config.schema.each do |key, args|
9
9
  next unless args.length > 1
10
- config.elastic_apm[key] = args.last[:default]
10
+ config.elastic_apm[key] = args[:default]
11
11
  end
12
12
 
13
13
  initializer 'elastic_apm.initialize' do |app|
@@ -20,7 +20,8 @@ module ElasticAPM
20
20
  subtype: nil,
21
21
  action: nil,
22
22
  context: nil,
23
- stacktrace_builder: nil
23
+ stacktrace_builder: nil,
24
+ sync: nil
24
25
  )
25
26
  @name = name
26
27
 
@@ -36,7 +37,7 @@ module ElasticAPM
36
37
  @parent = parent
37
38
  @trace_context = trace_context || parent.trace_context.child
38
39
 
39
- @context = context || Span::Context.new
40
+ @context = context || Span::Context.new(sync: sync)
40
41
  @stacktrace_builder = stacktrace_builder
41
42
  end
42
43
  # rubocop:enable Metrics/ParameterLists