sidekiq-instrument 0.4.1 → 0.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 55f5707e3e987499dee665c604a5d7453a643f91573fd0f1bec37287bef38309
4
- data.tar.gz: '095f5c3bb1dff919143ba4a7a798a00f9a3ed78e467a3165a7942267d75864f4'
3
+ metadata.gz: dec762f020453b49d17551ce8dba6146fad0b4fdb5c06fbc5ce17ad57bdafae5
4
+ data.tar.gz: 98e6f98439a8a886a8ec4adc0b8daa83d181629ebe7f2e4020861dc66379c533
5
5
  SHA512:
6
- metadata.gz: 55f8eab12d63ce0d4dc37ec3323f3126d6026aa1be42c101cd300195275ca94625b706b9beefada7a8764860b1a8c7ab0a02c7ced1d4bd2b02a9ccc5932181e1
7
- data.tar.gz: 9ba846d4af8f326c83157fa605b1982068baac4ed4cf07b05fa31cf06cafd5d95a484c60e29e9860462762ab401b041034272b5d87aedb187a3788d17d410dde
6
+ metadata.gz: 123de131fd9d2aa45d1256c2b4c53c25e1cdc1181b02ab16e230ca6e2cbdeda2785c8b2b58a20b27a5056a8dc2bfc48dfaff8ad26c887e60addb4684acb20162
7
+ data.tar.gz: 56b1cb9ac84cb8ee016da9ab99fabe7e000b59c46b1c0b23a107798193f8ab979c9e782f3525b4ff8041455533ebe8837e375803e3313478e670e9137b203852
@@ -0,0 +1,36 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - main
7
+ pull_request:
8
+
9
+ permissions:
10
+ contents: read
11
+
12
+ jobs:
13
+ test:
14
+ runs-on: ubuntu-latest
15
+ strategy:
16
+ matrix:
17
+ ruby-version: [2.6, 2.7, 3.0]
18
+ redis-version: [4, 5, 6]
19
+
20
+ steps:
21
+ - name: Checkout project
22
+ uses: actions/checkout@v3
23
+
24
+ - name: Set up Ruby
25
+ uses: ruby/setup-ruby@v1
26
+ with:
27
+ ruby-version: ${{ matrix.ruby-version }}
28
+ bundler-cache: true # runs 'bundle install' and caches installed gems automatically
29
+
30
+ - name: Start Redis
31
+ uses: supercharge/redis-github-action@1.4.0
32
+ with:
33
+ redis-version: ${{ matrix.redis-version }}
34
+
35
+ - name: Run tests
36
+ run: bundle exec rake
data/.simplecov CHANGED
@@ -1,8 +1,11 @@
1
- require 'coveralls'
1
+ require 'simplecov'
2
+ require 'simplecov-cobertura'
2
3
 
3
4
  SimpleCov.formatters = [
4
5
  SimpleCov::Formatter::HTMLFormatter,
5
- Coveralls::SimpleCov::Formatter
6
+ SimpleCov::Formatter::CoberturaFormatter
6
7
  ]
7
8
 
8
- SimpleCov.start
9
+ SimpleCov.start do
10
+ add_filter "/spec/"
11
+ end
data/README.md CHANGED
@@ -1,30 +1,33 @@
1
1
  # Sidekiq::Instrument
2
- [![Build Status](https://travis-ci.org/enova/sidekiq-instrument.svg?branch=master)](https://travis-ci.org/enova/sidekiq-instrument)
3
- [![Coverage Status](https://coveralls.io/repos/github/enova/sidekiq-instrument/badge.svg?branch=master)](https://coveralls.io/github/enova/sidekiq-instrument?branch=master)
4
2
 
5
- Reports job metrics using Shopify's [statsd-instrument][statsd-instrument] library, incrementing a counter for each enqueue and dequeue per job type, and timing the full runtime of your perform method.
3
+ Reports job metrics using Shopify's [statsd-instrument][statsd-instrument] library and \[optionally\] DataDog's [dogstatsd-ruby](https://github.com/DataDog/dogstatsd-ruby), incrementing a counter for each enqueue and dequeue per job type, and timing the full runtime of your perform method.
6
4
 
7
5
  ## Installation
8
6
 
9
- Add this line to your application's Gemfile:
7
+ Add the following to your application's Gemfile:
10
8
 
11
9
  ```ruby
12
10
  gem 'sidekiq-instrument'
11
+ gem 'dogstatsd-ruby' # optional
13
12
  ```
14
13
 
15
14
  And then execute:
16
15
 
17
16
  $ bundle
18
17
 
19
- Or install it yourself as:
18
+ Or install the gem(s) yourself as:
20
19
 
21
20
  $ gem install sidekiq-instrument
21
+ $ gem install dogstatsd-ruby # again, optional
22
22
 
23
23
  ## Usage
24
24
 
25
25
  For now, this library assumes you have already initialized `StatsD` on your own;
26
26
  the `statsd-instrument` gem may have chosen reasonable defaults for you already. If not,
27
- a typical Rails app would just use an initializer:
27
+ a typical Rails app would just use an initializer and set the `StatsD` and optional `DogStatsD`
28
+ clients via this gem's `Statter` class:
29
+
30
+ ### StatsD
28
31
 
29
32
  ```ruby
30
33
  # config/initializers/statsd.rb
@@ -33,11 +36,22 @@ StatsD.prefix = 'my-app'
33
36
  StatsD.backend = StatsD::Instrument::Backends::UDPBackend.new('some-server:8125')
34
37
  ```
35
38
 
39
+ ### DogStatsD
40
+
41
+ ```ruby
42
+ # config/initializers/dogstatsd.rb
43
+ require 'datadog/statsd'
44
+ DogStatsD = Datadog::Statsd.new('localhost', 8125, tags: {app_name: 'my_app', env: 'production'})
45
+ ```
46
+
36
47
  Then add the client and server middlewares in your Sidekiq initializer:
37
48
 
38
49
  ```ruby
39
50
  require 'sidekiq/instrument'
40
51
 
52
+ Sidekiq::Instrument::Statter.statsd = StatsD # optional, Statter will fall back to a global StatsD
53
+ Sidekiq::Instrument::Statter.dogstatsd = DogStatsD # optional, dogstatsd can be nil if not desired
54
+
41
55
  Sidekiq.configure_server do |config|
42
56
  config.server_middleware do |chain|
43
57
  chain.add Sidekiq::Instrument::ServerMiddleware
@@ -74,6 +88,22 @@ For each queue, the following metrics will be reported:
74
88
  1. **shared.sidekiq._queue_.size**: gauge of how many jobs are in the queue
75
89
  1. **shared.sidekiq._queue_.latency**: gauge of how long the oldest job has been in the queue
76
90
 
91
+ ## DogStatsD Keys
92
+ For each job, the following metrics and tags will be reported:
93
+
94
+ 1. **sidekiq.enqueue (tags: {queue: _queue_, worker: _job_})**: counter incremented each time a
95
+ job is pushed onto the queue.
96
+ 2. **sidekiq.dequeue (tags: {queue: _queue_, worker: _job_})**: counter incremented just before
97
+ worker begins performing a job.
98
+ 3. **sidekiq.runtime (tags: {queue: _queue_, worker: _job_})**: timer of the total time spent
99
+ in `perform`, in milliseconds.
100
+ 3. **sidekiq.error (tags: {queue: _queue_, worker: _job_})**: counter incremented each time a
101
+ job fails.
102
+
103
+ For each queue, the following metrics and tags will be reported:
104
+ 1. **sidekiq.queue_size (tags: {queue: _queue_})**: gauge of how many jobs are in the queue
105
+ 1. **sidekiq.queue_latency (tags: {queue: _queue_})**: gauge of how long the oldest job has been in the queue
106
+
77
107
  ## Worker
78
108
  There is a worker, `Sidekiq::Instrument::Worker`, that submits gauges
79
109
  for various interesting statistics; namely, the bulk of the information in `Sidekiq::Stats`
data/bin/console CHANGED
@@ -1,14 +1,15 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
- require "bundler/setup"
4
- require "sidekiq/instrument"
3
+ require 'bundler/setup'
4
+ require 'sidekiq/instrument'
5
+ require 'datadog/statsd'
5
6
 
6
7
  # You can add fixtures and/or initialization code here to make experimenting
7
8
  # with your gem easier. You can also use a different console, if you like.
8
9
 
9
10
  # (If you use this, don't forget to add pry to your Gemfile!)
10
- # require "pry"
11
+ # require 'pry'
11
12
  # Pry.start
12
13
 
13
- require "irb"
14
+ require 'irb'
14
15
  IRB.start
@@ -7,7 +7,9 @@ module Sidekiq::Instrument
7
7
  def call(worker_class, job, queue, redis_pool)
8
8
  # worker_class is a const in sidekiq >= 6.x
9
9
  klass = Object.const_get(worker_class.to_s)
10
- StatsD.increment metric_name(klass.new, 'enqueue')
10
+ class_instance = klass.new
11
+ Statter.statsd.increment(metric_name(class_instance, 'enqueue'))
12
+ Statter.dogstatsd&.increment('sidekiq.enqueue', worker_dog_options(class_instance))
11
13
 
12
14
  yield
13
15
  end
@@ -5,12 +5,19 @@ module Sidekiq::Instrument
5
5
  include Sidekiq::Instrument::MetricNames
6
6
 
7
7
  def call(worker, job, queue, &block)
8
- StatsD.increment(metric_name(worker, 'dequeue'))
8
+ Statter.statsd.increment(metric_name(worker, 'dequeue'))
9
+ Statter.dogstatsd&.increment('sidekiq.dequeue', worker_dog_options(worker))
9
10
 
10
- StatsD.measure(metric_name(worker,'runtime'), &block)
11
+ start_time = Time.now
12
+ yield block
13
+ execution_time_ms = (Time.now - start_time) * 1000
14
+ Statter.statsd.measure(metric_name(worker, 'runtime'), execution_time_ms)
15
+ Statter.dogstatsd&.timing('sidekiq.runtime', execution_time_ms, worker_dog_options(worker))
11
16
  rescue StandardError => e
12
- StatsD.increment(metric_name(worker, 'error'))
17
+ Statter.statsd.increment(metric_name(worker, 'error'))
18
+ Statter.dogstatsd&.increment('sidekiq.error', worker_dog_options(worker))
13
19
  raise e
14
20
  end
15
21
  end
16
22
  end
23
+
@@ -4,11 +4,22 @@ module Sidekiq::Instrument
4
4
  if worker.respond_to?(:statsd_metric_name)
5
5
  worker.send(:statsd_metric_name, event)
6
6
  else
7
- queue = worker.class.get_sidekiq_options['queue']
8
- name = worker.class.name.gsub('::', '_')
9
-
10
- "shared.sidekiq.#{queue}.#{name}.#{event}"
7
+ "shared.sidekiq.#{queue_name(worker)}.#{class_name(worker)}.#{event}"
11
8
  end
12
9
  end
10
+
11
+ def worker_dog_options(worker)
12
+ { tags: ["queue:#{queue_name(worker)}", "worker:#{class_name(worker)}"] }
13
+ end
14
+
15
+ private
16
+
17
+ def queue_name(worker)
18
+ worker.class.get_sidekiq_options['queue']
19
+ end
20
+
21
+ def class_name(worker)
22
+ worker.class.name.gsub('::', '_')
23
+ end
13
24
  end
14
25
  end
@@ -0,0 +1,15 @@
1
+ module Sidekiq::Instrument
2
+ class Statter
3
+ # @!scope class
4
+ # @!attribute [rw]
5
+ # StatsD client for emitting metrics related to Sidekiq queue operations.
6
+ class_attribute :statsd
7
+
8
+ # @!scope class
9
+ # @!attribute [rw]
10
+ # Optional DogStatsD client for emitting metrics related to Sidekiq queue operations.
11
+ class_attribute :dogstatsd
12
+
13
+ self.statsd ||= StatsD
14
+ end
15
+ end
@@ -1,5 +1,5 @@
1
1
  module Sidekiq
2
2
  module Instrument
3
- VERSION = "0.4.1"
3
+ VERSION = "0.5.0"
4
4
  end
5
5
  end
@@ -21,18 +21,22 @@ module Sidekiq::Instrument
21
21
  self.class::METRIC_NAMES.each do |method, stat|
22
22
  stat ||= method
23
23
 
24
- StatsD.gauge "shared.sidekiq.stats.#{stat}", info.send(method)
24
+ Statter.statsd.gauge("shared.sidekiq.stats.#{stat}", info.send(method))
25
+ Statter.dogstatsd&.gauge("sidekiq.#{stat}", info.send(method))
25
26
  end
26
27
 
27
28
  working = Sidekiq::ProcessSet.new.select { |p| p[:busy] == 1 }.count
28
- StatsD.gauge "shared.sidekiq.stats.working", working
29
+ Statter.statsd.gauge('shared.sidekiq.stats.working', working)
30
+ Statter.dogstatsd&.gauge('sidekiq.working', working)
29
31
 
30
32
  info.queues.each do |name, size|
31
- StatsD.gauge "shared.sidekiq.#{name}.size", size
33
+ Statter.statsd.gauge("shared.sidekiq.#{name}.size", size)
34
+ Statter.dogstatsd&.gauge('sidekiq.queue.size', size, tags: ["queue:#{name}"])
32
35
  end
33
36
 
34
37
  Sidekiq::Queue.all.each do |queue|
35
- StatsD.gauge "shared.sidekiq.#{queue.name}.latency", queue.latency
38
+ Statter.statsd.gauge("shared.sidekiq.#{queue.name}.latency", queue.latency)
39
+ Statter.dogstatsd&.gauge('sidekiq.queue.latency', queue.latency, tags: ["queue:#{queue.name}"])
36
40
  end
37
41
  end
38
42
  end
@@ -1,3 +1,6 @@
1
+ require 'active_support/core_ext/class/attribute'
2
+
3
+ require "sidekiq/instrument/statter"
1
4
  require "sidekiq/instrument/version"
2
5
  require "sidekiq/instrument/worker"
3
6
  require "sidekiq/instrument/middleware/client"
@@ -6,10 +6,10 @@ require 'sidekiq/instrument/version'
6
6
  Gem::Specification.new do |spec|
7
7
  spec.name = 'sidekiq-instrument'
8
8
  spec.version = Sidekiq::Instrument::VERSION
9
- spec.authors = ['Matt Larraz']
10
- spec.email = ['mlarraz@enova.com']
9
+ spec.authors = ['Loan Application Services']
10
+ spec.email = ['application_services@enova.com']
11
11
 
12
- spec.summary = 'StatsD instrumentation for Sidekiq'
12
+ spec.summary = 'StatsD & DogStatsD Instrumentation for Sidekiq'
13
13
  spec.homepage = 'https://github.com/enova/sidekiq-instrument'
14
14
  spec.license = 'MIT'
15
15
 
@@ -18,10 +18,13 @@ Gem::Specification.new do |spec|
18
18
 
19
19
  spec.add_dependency 'sidekiq', '>= 4.2', '< 7'
20
20
  spec.add_dependency 'statsd-instrument', '>= 2.0.4'
21
+ spec.add_dependency 'dogstatsd-ruby', '~> 5.5.0'
22
+ spec.add_dependency 'activesupport', '>= 5.1', '< 7'
21
23
 
22
24
  spec.add_development_dependency 'bundler', '~> 2.0', '>= 2.0.2'
23
25
  spec.add_development_dependency 'rake', '~> 12.0'
24
26
  spec.add_development_dependency 'rspec', '~> 3.0'
25
27
  spec.add_development_dependency 'pry-byebug', '~> 3.4'
26
- spec.add_development_dependency 'coveralls', '~> 0.8'
28
+ spec.add_development_dependency 'simplecov'
29
+ spec.add_development_dependency 'simplecov-cobertura'
27
30
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sidekiq-instrument
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.1
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
- - Matt Larraz
7
+ - Loan Application Services
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-07-28 00:00:00.000000000 Z
11
+ date: 2022-11-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sidekiq
@@ -44,6 +44,40 @@ dependencies:
44
44
  - - ">="
45
45
  - !ruby/object:Gem::Version
46
46
  version: 2.0.4
47
+ - !ruby/object:Gem::Dependency
48
+ name: dogstatsd-ruby
49
+ requirement: !ruby/object:Gem::Requirement
50
+ requirements:
51
+ - - "~>"
52
+ - !ruby/object:Gem::Version
53
+ version: 5.5.0
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - "~>"
59
+ - !ruby/object:Gem::Version
60
+ version: 5.5.0
61
+ - !ruby/object:Gem::Dependency
62
+ name: activesupport
63
+ requirement: !ruby/object:Gem::Requirement
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: '5.1'
68
+ - - "<"
69
+ - !ruby/object:Gem::Version
70
+ version: '7'
71
+ type: :runtime
72
+ prerelease: false
73
+ version_requirements: !ruby/object:Gem::Requirement
74
+ requirements:
75
+ - - ">="
76
+ - !ruby/object:Gem::Version
77
+ version: '5.1'
78
+ - - "<"
79
+ - !ruby/object:Gem::Version
80
+ version: '7'
47
81
  - !ruby/object:Gem::Dependency
48
82
  name: bundler
49
83
  requirement: !ruby/object:Gem::Requirement
@@ -107,30 +141,44 @@ dependencies:
107
141
  - !ruby/object:Gem::Version
108
142
  version: '3.4'
109
143
  - !ruby/object:Gem::Dependency
110
- name: coveralls
144
+ name: simplecov
111
145
  requirement: !ruby/object:Gem::Requirement
112
146
  requirements:
113
- - - "~>"
147
+ - - ">="
114
148
  - !ruby/object:Gem::Version
115
- version: '0.8'
149
+ version: '0'
116
150
  type: :development
117
151
  prerelease: false
118
152
  version_requirements: !ruby/object:Gem::Requirement
119
153
  requirements:
120
- - - "~>"
154
+ - - ">="
155
+ - !ruby/object:Gem::Version
156
+ version: '0'
157
+ - !ruby/object:Gem::Dependency
158
+ name: simplecov-cobertura
159
+ requirement: !ruby/object:Gem::Requirement
160
+ requirements:
161
+ - - ">="
162
+ - !ruby/object:Gem::Version
163
+ version: '0'
164
+ type: :development
165
+ prerelease: false
166
+ version_requirements: !ruby/object:Gem::Requirement
167
+ requirements:
168
+ - - ">="
121
169
  - !ruby/object:Gem::Version
122
- version: '0.8'
170
+ version: '0'
123
171
  description:
124
172
  email:
125
- - mlarraz@enova.com
173
+ - application_services@enova.com
126
174
  executables: []
127
175
  extensions: []
128
176
  extra_rdoc_files: []
129
177
  files:
178
+ - ".github/workflows/ci.yml"
130
179
  - ".gitignore"
131
180
  - ".rspec"
132
181
  - ".simplecov"
133
- - ".travis.yml"
134
182
  - Gemfile
135
183
  - LICENSE.txt
136
184
  - README.md
@@ -141,6 +189,7 @@ files:
141
189
  - lib/sidekiq/instrument/middleware/client.rb
142
190
  - lib/sidekiq/instrument/middleware/server.rb
143
191
  - lib/sidekiq/instrument/mixin.rb
192
+ - lib/sidekiq/instrument/statter.rb
144
193
  - lib/sidekiq/instrument/version.rb
145
194
  - lib/sidekiq/instrument/worker.rb
146
195
  - sidekiq-instrument.gemspec
@@ -166,5 +215,5 @@ requirements: []
166
215
  rubygems_version: 3.1.6
167
216
  signing_key:
168
217
  specification_version: 4
169
- summary: StatsD instrumentation for Sidekiq
218
+ summary: StatsD & DogStatsD Instrumentation for Sidekiq
170
219
  test_files: []
data/.travis.yml DELETED
@@ -1,20 +0,0 @@
1
- sudo: false
2
- language: ruby
3
- rvm:
4
- - 2.5.8
5
- - 2.6.6
6
- - 2.7.2
7
-
8
- services:
9
- - redis-server
10
-
11
- before_install: gem install bundler -v 2.0.2
12
-
13
- deploy:
14
- provider: rubygems
15
- api_key:
16
- secure: T0S2ipyvGkY2qL6oxcgZgBl+krGR3qFlL4vNyzy+rvaZAV8YNijQiM7YyggTLhnKai3SxuHjzN1ZiT7vzlklJn9k2qRhU1uVq4xW+YX6yVNBBbugJGqNuyw3p2BUOga83ES2jA/Kby9mLAJEebS5LmupapSCeElsmhYzXIlV9SYH1mZV4CHQ06Luv3r1DQ8z1cwrB+5N+Y9TNV1wXa/LbuxJPl6jgxN0oJ+RFU7lTHtovMwyL+wRXLiRpZc2k6dH0IUo03QxYVCUDIvShdX26WfzF+fpSPRgOHnbJeaBTbeuxntFzEDy6Td2hpRQWP31i+/8BzPWGCLqUfS55uQqFM06UI/vnZnbe8NlI57gzw6yJHPsFoyg8Cw/3TABhw+SEOoMyBdued4RGKF6EgYsx3hW+KUhEvncnBg6F7jRdZgtHmk6s8iU+/d39pgQh4AiCJQOhINErAnnH2P5hM3TDoZYQlkUjw0FQ0F8bTmRKJq4JhMyvtwMutvMuSIN+8q7utNTxzFquvYcmAcNYafsmuC7Qtvpqi0iVzQ3f3c+SCljcvEnUEf66+xufkn79eSAYcCrL/FdZOdBEiXpaY2wdtJO7Kr66BMhzGCjlO8YQoDw0pgsES9Uy5H1JtfphjWzHWYdiYDBoofFURE1jO/pJ9V9dp9ezzOcvW4faZhL8ho=
17
- gem: sidekiq-instrument
18
- on:
19
- tags: true
20
- repo: enova/sidekiq-instrument