yabeda-sidekiq 0.1.2 → 0.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +59 -0
- data/README.md +28 -1
- data/lib/yabeda/sidekiq.rb +32 -12
- data/lib/yabeda/sidekiq/server_middleware.rb +2 -0
- data/lib/yabeda/sidekiq/version.rb +1 -1
- data/yabeda-sidekiq.gemspec +1 -1
- metadata +11 -11
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 63dde1ca9f23aca58fe86e630c4cbaa48426ad3621fee5bc0b6fa5dc804d0a3d
|
|
4
|
+
data.tar.gz: 67ab69e2ba72d772ed577636edb12f28b06ab829c9e31ba3e18fe74f711c01f7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c03295246b723eb55b6b93fa3bf0068c8306aa01ddd11915d228dba4c8d70640b961d2f63cb8448d88c130f98d3560ab6fca0cf01fb94fef00751ddf6390ea39
|
|
7
|
+
data.tar.gz: e010471f073e8becf98515531e2c532025cdaac64fac393fdad485074b0705964ed60e564d6b523da50b5421be14b6a3dd6838d1f228b535be5d9926d3218cee
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
|
|
6
|
+
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## 0.6.0 - 2020-07-15
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- Ability to override or add tags for every job via `yabeda_tags` method. @Envek
|
|
13
|
+
|
|
14
|
+
## 0.5.0 - 2020-02-20
|
|
15
|
+
|
|
16
|
+
### Added
|
|
17
|
+
|
|
18
|
+
- New `sidekiq_job_latency` histogram to track latency statistics of different job classes. [#9](https://github.com/yabeda-rb/yabeda-sidekiq/pull/9) by [@asusikov]
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
|
|
22
|
+
- **BREAKING CHANGE!** Renamed `sidekiq_jobs_latency` gauge to `sidekiq_queue_latency` to better describe its purpose and differentiate with the new histogram. [#9](https://github.com/yabeda-rb/yabeda-sidekiq/pull/9) by [@asusikov]
|
|
23
|
+
|
|
24
|
+
## 0.2.0 - 2020-01-14
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- Added `tags` option to metric declarations for compatibility with yabeda and yabeda-prometheus 0.2. @Envek
|
|
29
|
+
|
|
30
|
+
## 0.1.4 - 2019-10-07
|
|
31
|
+
|
|
32
|
+
### Added
|
|
33
|
+
|
|
34
|
+
- Require of core yabeda gem [#4](https://github.com/yabeda-rb/yabeda-sidekiq/pull/4). [@dsalahutdinov]
|
|
35
|
+
|
|
36
|
+
## 0.1.3 - 2018-10-25
|
|
37
|
+
|
|
38
|
+
### Fixed
|
|
39
|
+
|
|
40
|
+
- Require of core yabeda gem [#1](https://github.com/yabeda-rb/yabeda-sidekiq/issues/1). @Envek
|
|
41
|
+
|
|
42
|
+
## 0.1.2 - 2018-10-17
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- Renamed evil-metrics-sidekiq gem to yabeda-sidekiq. @Envek
|
|
47
|
+
|
|
48
|
+
## 0.1.1 - 2018-10-05
|
|
49
|
+
|
|
50
|
+
### Changed
|
|
51
|
+
|
|
52
|
+
- Automatic add client and server middlewares to Sidekiq. @Envek
|
|
53
|
+
|
|
54
|
+
## 0.1.0 - 2018-10-03
|
|
55
|
+
|
|
56
|
+
- Initial release of evil-metrics-sidekiq gem. @Envek
|
|
57
|
+
|
|
58
|
+
[@dsalahutdinov]: https://github.com/dsalahutdinov "Salahutdinov Dmitry"
|
|
59
|
+
[@asusikov]: https://github.com/asusikov "Alexander Susikov"
|
data/README.md
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
Built-in metrics for [Sidekiq] monitoring out of the box! Part of the [yabeda] suite.
|
|
4
4
|
|
|
5
|
+
Sample Grafana dashboard ID: [11667](https://grafana.com/grafana/dashboards/11667)
|
|
6
|
+
|
|
5
7
|
## Installation
|
|
6
8
|
|
|
7
9
|
```ruby
|
|
8
|
-
gem 'yabeda-
|
|
10
|
+
gem 'yabeda-sidekiq'
|
|
9
11
|
# Then add monitoring system adapter, e.g.:
|
|
10
12
|
# gem 'yabeda-prometheus'
|
|
11
13
|
```
|
|
@@ -31,6 +33,8 @@ end
|
|
|
31
33
|
- Number of jobs have been finished successfully: `sidekiq_jobs_success_total` (segmented by queue and class name)
|
|
32
34
|
- Number of jobs have been failed: `sidekiq_jobs_failed_total` (segmented by queue and class name)
|
|
33
35
|
- Time of job run: `sidekiq_job_runtime` (seconds per job execution, segmented by queue and class name)
|
|
36
|
+
- Time of the queue latency `sidekiq_queue_latency` (the difference in seconds since the oldest job in the queue was enqueued)
|
|
37
|
+
- Time of the job latency `sidekiq_job_latency` (the difference in seconds since the enqueuing until running job)
|
|
34
38
|
- Number of jobs in queues: `sidekiq_jobs_waiting_count` (segmented by queue)
|
|
35
39
|
- Number of scheduled jobs:`sidekiq_jobs_scheduled_count`
|
|
36
40
|
- Number of jobs in retry set: `sidekiq_jobs_retry_count`
|
|
@@ -38,6 +42,29 @@ end
|
|
|
38
42
|
- Active workers count: `sidekiq_active_processes`
|
|
39
43
|
- Active processes count: `sidekiq_active_workers_count`
|
|
40
44
|
|
|
45
|
+
## Custom tags
|
|
46
|
+
|
|
47
|
+
You can add additional tags to these metrics by declaring `yabeda_tags` method in your worker.
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
# This block is optional but some adapters (like Prometheus) requires that all tags should be declared in advance
|
|
51
|
+
Yabeda.configure do
|
|
52
|
+
default_tag :importance, nil
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
class MyWorker
|
|
56
|
+
include Sidekiq::Worker
|
|
57
|
+
|
|
58
|
+
def yabeda_tags(*params) # This method will be called first, before +perform+
|
|
59
|
+
{ importance: extract_importance(params) }
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def perform(*params)
|
|
63
|
+
# Your logic here
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
41
68
|
# Roadmap (TODO or Help wanted)
|
|
42
69
|
|
|
43
70
|
- Implement optional segmentation of retry/schedule/dead sets
|
data/lib/yabeda/sidekiq.rb
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "sidekiq"
|
|
4
4
|
require "sidekiq/api"
|
|
5
5
|
|
|
6
|
+
require "yabeda"
|
|
6
7
|
require "yabeda/sidekiq/version"
|
|
7
8
|
require "yabeda/sidekiq/client_middleware"
|
|
8
9
|
require "yabeda/sidekiq/server_middleware"
|
|
@@ -17,20 +18,29 @@ module Yabeda
|
|
|
17
18
|
Yabeda.configure do
|
|
18
19
|
group :sidekiq
|
|
19
20
|
|
|
20
|
-
counter :jobs_enqueued_total, comment: "A counter of the total number of jobs sidekiq enqueued."
|
|
21
|
+
counter :jobs_enqueued_total, tags: %i[queue worker], comment: "A counter of the total number of jobs sidekiq enqueued."
|
|
21
22
|
|
|
22
23
|
next unless ::Sidekiq.server?
|
|
23
24
|
|
|
24
|
-
counter :jobs_executed_total, comment: "A counter of the total number of jobs sidekiq executed."
|
|
25
|
-
counter :jobs_success_total, comment: "A counter of the total number of jobs successfully processed by sidekiq."
|
|
26
|
-
counter :jobs_failed_total, comment: "A counter of the total number of jobs failed in sidekiq."
|
|
27
|
-
|
|
28
|
-
gauge :
|
|
29
|
-
gauge :
|
|
30
|
-
gauge :
|
|
31
|
-
gauge :
|
|
32
|
-
gauge :
|
|
33
|
-
|
|
25
|
+
counter :jobs_executed_total, tags: %i[queue worker], comment: "A counter of the total number of jobs sidekiq executed."
|
|
26
|
+
counter :jobs_success_total, tags: %i[queue worker], comment: "A counter of the total number of jobs successfully processed by sidekiq."
|
|
27
|
+
counter :jobs_failed_total, tags: %i[queue worker], comment: "A counter of the total number of jobs failed in sidekiq."
|
|
28
|
+
|
|
29
|
+
gauge :jobs_waiting_count, tags: %i[queue], comment: "The number of jobs waiting to process in sidekiq."
|
|
30
|
+
gauge :active_workers_count, tags: [], comment: "The number of currently running machines with sidekiq workers."
|
|
31
|
+
gauge :jobs_scheduled_count, tags: [], comment: "The number of jobs scheduled for later execution."
|
|
32
|
+
gauge :jobs_retry_count, tags: [], comment: "The number of failed jobs waiting to be retried"
|
|
33
|
+
gauge :jobs_dead_count, tags: [], comment: "The number of jobs exceeded their retry count."
|
|
34
|
+
gauge :active_processes, tags: [], comment: "The number of active Sidekiq worker processes."
|
|
35
|
+
gauge :queue_latency, tags: %i[queue], comment: "The queue latency, the difference in seconds since the oldest job in the queue was enqueued"
|
|
36
|
+
|
|
37
|
+
histogram :job_latency, comment: "The job latency, the difference in seconds between enqueued and running time",
|
|
38
|
+
unit: :seconds, per: :job,
|
|
39
|
+
tags: %i[queue worker],
|
|
40
|
+
buckets: LONG_RUNNING_JOB_RUNTIME_BUCKETS
|
|
41
|
+
histogram :job_runtime, comment: "A histogram of the job execution time.",
|
|
42
|
+
unit: :seconds, per: :job,
|
|
43
|
+
tags: %i[queue worker],
|
|
34
44
|
buckets: LONG_RUNNING_JOB_RUNTIME_BUCKETS
|
|
35
45
|
|
|
36
46
|
collect do
|
|
@@ -45,6 +55,10 @@ module Yabeda
|
|
|
45
55
|
sidekiq_active_processes.set({}, stats.processes_size)
|
|
46
56
|
sidekiq_jobs_retry_count.set({}, stats.retry_size)
|
|
47
57
|
|
|
58
|
+
::Sidekiq::Queue.all.each do |queue|
|
|
59
|
+
sidekiq_queue_latency.set({ queue: queue.name }, queue.latency)
|
|
60
|
+
end
|
|
61
|
+
|
|
48
62
|
# That is quite slow if your retry set is large
|
|
49
63
|
# I don't want to enable it by default
|
|
50
64
|
# retries_by_queues =
|
|
@@ -74,7 +88,7 @@ module Yabeda
|
|
|
74
88
|
|
|
75
89
|
class << self
|
|
76
90
|
def labelize(worker, job, queue)
|
|
77
|
-
{ queue: queue, worker: worker_class(worker, job) }
|
|
91
|
+
{ queue: queue, worker: worker_class(worker, job), **custom_tags(worker, job).to_h }
|
|
78
92
|
end
|
|
79
93
|
|
|
80
94
|
def worker_class(worker, job)
|
|
@@ -83,6 +97,12 @@ module Yabeda
|
|
|
83
97
|
end
|
|
84
98
|
(worker.is_a?(String) ? worker : worker.class).to_s
|
|
85
99
|
end
|
|
100
|
+
|
|
101
|
+
def custom_tags(worker, job)
|
|
102
|
+
return {} unless worker.respond_to?(:yabeda_tags)
|
|
103
|
+
|
|
104
|
+
worker.method(:yabeda_tags).arity.zero? ? worker.yabeda_tags : worker.yabeda_tags(*job["args"])
|
|
105
|
+
end
|
|
86
106
|
end
|
|
87
107
|
end
|
|
88
108
|
end
|
|
@@ -8,6 +8,8 @@ module Yabeda
|
|
|
8
8
|
labels = Yabeda::Sidekiq.labelize(worker, job, queue)
|
|
9
9
|
start = Time.now
|
|
10
10
|
begin
|
|
11
|
+
job_instance = ::Sidekiq::Job.new(job)
|
|
12
|
+
Yabeda.sidekiq_job_latency.measure(labels, job_instance.latency)
|
|
11
13
|
yield
|
|
12
14
|
Yabeda.sidekiq_jobs_success_total.increment(labels)
|
|
13
15
|
rescue Exception # rubocop: disable Lint/RescueException
|
data/yabeda-sidekiq.gemspec
CHANGED
|
@@ -22,7 +22,7 @@ Gem::Specification.new do |spec|
|
|
|
22
22
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
|
23
23
|
spec.require_paths = ["lib"]
|
|
24
24
|
|
|
25
|
-
spec.add_dependency "yabeda"
|
|
25
|
+
spec.add_dependency "yabeda", "~> 0.2"
|
|
26
26
|
spec.add_dependency "sidekiq"
|
|
27
27
|
|
|
28
28
|
spec.add_development_dependency "bundler", "~> 1.16"
|
metadata
CHANGED
|
@@ -1,29 +1,29 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: yabeda-sidekiq
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Andrey Novikov
|
|
8
|
-
autorequire:
|
|
8
|
+
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2020-07-15 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: yabeda
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
16
16
|
requirements:
|
|
17
|
-
- - "
|
|
17
|
+
- - "~>"
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version: '0'
|
|
19
|
+
version: '0.2'
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
|
-
- - "
|
|
24
|
+
- - "~>"
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
|
-
version: '0'
|
|
26
|
+
version: '0.2'
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
28
28
|
name: sidekiq
|
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -92,6 +92,7 @@ files:
|
|
|
92
92
|
- ".rspec"
|
|
93
93
|
- ".rubocop.yml"
|
|
94
94
|
- ".travis.yml"
|
|
95
|
+
- CHANGELOG.md
|
|
95
96
|
- Gemfile
|
|
96
97
|
- LICENSE.txt
|
|
97
98
|
- README.md
|
|
@@ -107,7 +108,7 @@ homepage: https://github.com/yabeda-rb/yabeda-sidekiq
|
|
|
107
108
|
licenses:
|
|
108
109
|
- MIT
|
|
109
110
|
metadata: {}
|
|
110
|
-
post_install_message:
|
|
111
|
+
post_install_message:
|
|
111
112
|
rdoc_options: []
|
|
112
113
|
require_paths:
|
|
113
114
|
- lib
|
|
@@ -122,9 +123,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
122
123
|
- !ruby/object:Gem::Version
|
|
123
124
|
version: '0'
|
|
124
125
|
requirements: []
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
signing_key:
|
|
126
|
+
rubygems_version: 3.1.2
|
|
127
|
+
signing_key:
|
|
128
128
|
specification_version: 4
|
|
129
129
|
summary: Extensible Prometheus exporter for monitoring your Sidekiq
|
|
130
130
|
test_files: []
|