yabeda-sidekiq 0.1.2 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +59 -0
- data/README.md +28 -1
- data/lib/yabeda/sidekiq.rb +32 -12
- data/lib/yabeda/sidekiq/server_middleware.rb +2 -0
- data/lib/yabeda/sidekiq/version.rb +1 -1
- data/yabeda-sidekiq.gemspec +1 -1
- metadata +11 -11
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 63dde1ca9f23aca58fe86e630c4cbaa48426ad3621fee5bc0b6fa5dc804d0a3d
|
|
4
|
+
data.tar.gz: 67ab69e2ba72d772ed577636edb12f28b06ab829c9e31ba3e18fe74f711c01f7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c03295246b723eb55b6b93fa3bf0068c8306aa01ddd11915d228dba4c8d70640b961d2f63cb8448d88c130f98d3560ab6fca0cf01fb94fef00751ddf6390ea39
|
|
7
|
+
data.tar.gz: e010471f073e8becf98515531e2c532025cdaac64fac393fdad485074b0705964ed60e564d6b523da50b5421be14b6a3dd6838d1f228b535be5d9926d3218cee
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
|
|
6
|
+
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## 0.6.0 - 2020-07-15
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- Ability to override or add tags for every job via `yabeda_tags` method. @Envek
|
|
13
|
+
|
|
14
|
+
## 0.5.0 - 2020-02-20
|
|
15
|
+
|
|
16
|
+
### Added
|
|
17
|
+
|
|
18
|
+
- New `sidekiq_job_latency` histogram to track latency statistics of different job classes. [#9](https://github.com/yabeda-rb/yabeda-sidekiq/pull/9) by [@asusikov]
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
|
|
22
|
+
- **BREAKING CHANGE!** Renamed `sidekiq_jobs_latency` gauge to `sidekiq_queue_latency` to better describe its purpose and differentiate with the new histogram. [#9](https://github.com/yabeda-rb/yabeda-sidekiq/pull/9) by [@asusikov]
|
|
23
|
+
|
|
24
|
+
## 0.2.0 - 2020-01-14
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- Added `tags` option to metric declarations for compatibility with yabeda and yabeda-prometheus 0.2. @Envek
|
|
29
|
+
|
|
30
|
+
## 0.1.4 - 2019-10-07
|
|
31
|
+
|
|
32
|
+
### Added
|
|
33
|
+
|
|
34
|
+
- Require of core yabeda gem [#4](https://github.com/yabeda-rb/yabeda-sidekiq/pull/4). [@dsalahutdinov]
|
|
35
|
+
|
|
36
|
+
## 0.1.3 - 2018-10-25
|
|
37
|
+
|
|
38
|
+
### Fixed
|
|
39
|
+
|
|
40
|
+
- Require of core yabeda gem [#1](https://github.com/yabeda-rb/yabeda-sidekiq/issues/1). @Envek
|
|
41
|
+
|
|
42
|
+
## 0.1.2 - 2018-10-17
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- Renamed evil-metrics-sidekiq gem to yabeda-sidekiq. @Envek
|
|
47
|
+
|
|
48
|
+
## 0.1.1 - 2018-10-05
|
|
49
|
+
|
|
50
|
+
### Changed
|
|
51
|
+
|
|
52
|
+
- Automatic add client and server middlewares to Sidekiq. @Envek
|
|
53
|
+
|
|
54
|
+
## 0.1.0 - 2018-10-03
|
|
55
|
+
|
|
56
|
+
- Initial release of evil-metrics-sidekiq gem. @Envek
|
|
57
|
+
|
|
58
|
+
[@dsalahutdinov]: https://github.com/dsalahutdinov "Salahutdinov Dmitry"
|
|
59
|
+
[@asusikov]: https://github.com/asusikov "Alexander Susikov"
|
data/README.md
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
Built-in metrics for [Sidekiq] monitoring out of the box! Part of the [yabeda] suite.
|
|
4
4
|
|
|
5
|
+
Sample Grafana dashboard ID: [11667](https://grafana.com/grafana/dashboards/11667)
|
|
6
|
+
|
|
5
7
|
## Installation
|
|
6
8
|
|
|
7
9
|
```ruby
|
|
8
|
-
gem 'yabeda-
|
|
10
|
+
gem 'yabeda-sidekiq'
|
|
9
11
|
# Then add monitoring system adapter, e.g.:
|
|
10
12
|
# gem 'yabeda-prometheus'
|
|
11
13
|
```
|
|
@@ -31,6 +33,8 @@ end
|
|
|
31
33
|
- Number of jobs have been finished successfully: `sidekiq_jobs_success_total` (segmented by queue and class name)
|
|
32
34
|
- Number of jobs have been failed: `sidekiq_jobs_failed_total` (segmented by queue and class name)
|
|
33
35
|
- Time of job run: `sidekiq_job_runtime` (seconds per job execution, segmented by queue and class name)
|
|
36
|
+
- Time of the queue latency `sidekiq_queue_latency` (the difference in seconds since the oldest job in the queue was enqueued)
|
|
37
|
+
- Time of the job latency `sidekiq_job_latency` (the difference in seconds since the enqueuing until running job)
|
|
34
38
|
- Number of jobs in queues: `sidekiq_jobs_waiting_count` (segmented by queue)
|
|
35
39
|
- Number of scheduled jobs:`sidekiq_jobs_scheduled_count`
|
|
36
40
|
- Number of jobs in retry set: `sidekiq_jobs_retry_count`
|
|
@@ -38,6 +42,29 @@ end
|
|
|
38
42
|
- Active workers count: `sidekiq_active_processes`
|
|
39
43
|
- Active processes count: `sidekiq_active_workers_count`
|
|
40
44
|
|
|
45
|
+
## Custom tags
|
|
46
|
+
|
|
47
|
+
You can add additional tags to these metrics by declaring `yabeda_tags` method in your worker.
|
|
48
|
+
|
|
49
|
+
```ruby
|
|
50
|
+
# This block is optional but some adapters (like Prometheus) requires that all tags should be declared in advance
|
|
51
|
+
Yabeda.configure do
|
|
52
|
+
default_tag :importance, nil
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
class MyWorker
|
|
56
|
+
include Sidekiq::Worker
|
|
57
|
+
|
|
58
|
+
def yabeda_tags(*params) # This method will be called first, before +perform+
|
|
59
|
+
{ importance: extract_importance(params) }
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def perform(*params)
|
|
63
|
+
# Your logic here
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
41
68
|
# Roadmap (TODO or Help wanted)
|
|
42
69
|
|
|
43
70
|
- Implement optional segmentation of retry/schedule/dead sets
|
data/lib/yabeda/sidekiq.rb
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "sidekiq"
|
|
4
4
|
require "sidekiq/api"
|
|
5
5
|
|
|
6
|
+
require "yabeda"
|
|
6
7
|
require "yabeda/sidekiq/version"
|
|
7
8
|
require "yabeda/sidekiq/client_middleware"
|
|
8
9
|
require "yabeda/sidekiq/server_middleware"
|
|
@@ -17,20 +18,29 @@ module Yabeda
|
|
|
17
18
|
Yabeda.configure do
|
|
18
19
|
group :sidekiq
|
|
19
20
|
|
|
20
|
-
counter :jobs_enqueued_total, comment: "A counter of the total number of jobs sidekiq enqueued."
|
|
21
|
+
counter :jobs_enqueued_total, tags: %i[queue worker], comment: "A counter of the total number of jobs sidekiq enqueued."
|
|
21
22
|
|
|
22
23
|
next unless ::Sidekiq.server?
|
|
23
24
|
|
|
24
|
-
counter :jobs_executed_total, comment: "A counter of the total number of jobs sidekiq executed."
|
|
25
|
-
counter :jobs_success_total, comment: "A counter of the total number of jobs successfully processed by sidekiq."
|
|
26
|
-
counter :jobs_failed_total, comment: "A counter of the total number of jobs failed in sidekiq."
|
|
27
|
-
|
|
28
|
-
gauge :
|
|
29
|
-
gauge :
|
|
30
|
-
gauge :
|
|
31
|
-
gauge :
|
|
32
|
-
gauge :
|
|
33
|
-
|
|
25
|
+
counter :jobs_executed_total, tags: %i[queue worker], comment: "A counter of the total number of jobs sidekiq executed."
|
|
26
|
+
counter :jobs_success_total, tags: %i[queue worker], comment: "A counter of the total number of jobs successfully processed by sidekiq."
|
|
27
|
+
counter :jobs_failed_total, tags: %i[queue worker], comment: "A counter of the total number of jobs failed in sidekiq."
|
|
28
|
+
|
|
29
|
+
gauge :jobs_waiting_count, tags: %i[queue], comment: "The number of jobs waiting to process in sidekiq."
|
|
30
|
+
gauge :active_workers_count, tags: [], comment: "The number of currently running machines with sidekiq workers."
|
|
31
|
+
gauge :jobs_scheduled_count, tags: [], comment: "The number of jobs scheduled for later execution."
|
|
32
|
+
gauge :jobs_retry_count, tags: [], comment: "The number of failed jobs waiting to be retried"
|
|
33
|
+
gauge :jobs_dead_count, tags: [], comment: "The number of jobs exceeded their retry count."
|
|
34
|
+
gauge :active_processes, tags: [], comment: "The number of active Sidekiq worker processes."
|
|
35
|
+
gauge :queue_latency, tags: %i[queue], comment: "The queue latency, the difference in seconds since the oldest job in the queue was enqueued"
|
|
36
|
+
|
|
37
|
+
histogram :job_latency, comment: "The job latency, the difference in seconds between enqueued and running time",
|
|
38
|
+
unit: :seconds, per: :job,
|
|
39
|
+
tags: %i[queue worker],
|
|
40
|
+
buckets: LONG_RUNNING_JOB_RUNTIME_BUCKETS
|
|
41
|
+
histogram :job_runtime, comment: "A histogram of the job execution time.",
|
|
42
|
+
unit: :seconds, per: :job,
|
|
43
|
+
tags: %i[queue worker],
|
|
34
44
|
buckets: LONG_RUNNING_JOB_RUNTIME_BUCKETS
|
|
35
45
|
|
|
36
46
|
collect do
|
|
@@ -45,6 +55,10 @@ module Yabeda
|
|
|
45
55
|
sidekiq_active_processes.set({}, stats.processes_size)
|
|
46
56
|
sidekiq_jobs_retry_count.set({}, stats.retry_size)
|
|
47
57
|
|
|
58
|
+
::Sidekiq::Queue.all.each do |queue|
|
|
59
|
+
sidekiq_queue_latency.set({ queue: queue.name }, queue.latency)
|
|
60
|
+
end
|
|
61
|
+
|
|
48
62
|
# That is quite slow if your retry set is large
|
|
49
63
|
# I don't want to enable it by default
|
|
50
64
|
# retries_by_queues =
|
|
@@ -74,7 +88,7 @@ module Yabeda
|
|
|
74
88
|
|
|
75
89
|
class << self
|
|
76
90
|
def labelize(worker, job, queue)
|
|
77
|
-
{ queue: queue, worker: worker_class(worker, job) }
|
|
91
|
+
{ queue: queue, worker: worker_class(worker, job), **custom_tags(worker, job).to_h }
|
|
78
92
|
end
|
|
79
93
|
|
|
80
94
|
def worker_class(worker, job)
|
|
@@ -83,6 +97,12 @@ module Yabeda
|
|
|
83
97
|
end
|
|
84
98
|
(worker.is_a?(String) ? worker : worker.class).to_s
|
|
85
99
|
end
|
|
100
|
+
|
|
101
|
+
def custom_tags(worker, job)
|
|
102
|
+
return {} unless worker.respond_to?(:yabeda_tags)
|
|
103
|
+
|
|
104
|
+
worker.method(:yabeda_tags).arity.zero? ? worker.yabeda_tags : worker.yabeda_tags(*job["args"])
|
|
105
|
+
end
|
|
86
106
|
end
|
|
87
107
|
end
|
|
88
108
|
end
|
|
@@ -8,6 +8,8 @@ module Yabeda
|
|
|
8
8
|
labels = Yabeda::Sidekiq.labelize(worker, job, queue)
|
|
9
9
|
start = Time.now
|
|
10
10
|
begin
|
|
11
|
+
job_instance = ::Sidekiq::Job.new(job)
|
|
12
|
+
Yabeda.sidekiq_job_latency.measure(labels, job_instance.latency)
|
|
11
13
|
yield
|
|
12
14
|
Yabeda.sidekiq_jobs_success_total.increment(labels)
|
|
13
15
|
rescue Exception # rubocop: disable Lint/RescueException
|
data/yabeda-sidekiq.gemspec
CHANGED
|
@@ -22,7 +22,7 @@ Gem::Specification.new do |spec|
|
|
|
22
22
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
|
23
23
|
spec.require_paths = ["lib"]
|
|
24
24
|
|
|
25
|
-
spec.add_dependency "yabeda"
|
|
25
|
+
spec.add_dependency "yabeda", "~> 0.2"
|
|
26
26
|
spec.add_dependency "sidekiq"
|
|
27
27
|
|
|
28
28
|
spec.add_development_dependency "bundler", "~> 1.16"
|
metadata
CHANGED
|
@@ -1,29 +1,29 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: yabeda-sidekiq
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Andrey Novikov
|
|
8
|
-
autorequire:
|
|
8
|
+
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2020-07-15 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: yabeda
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
16
16
|
requirements:
|
|
17
|
-
- - "
|
|
17
|
+
- - "~>"
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version: '0'
|
|
19
|
+
version: '0.2'
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
|
-
- - "
|
|
24
|
+
- - "~>"
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
|
-
version: '0'
|
|
26
|
+
version: '0.2'
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
28
28
|
name: sidekiq
|
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -92,6 +92,7 @@ files:
|
|
|
92
92
|
- ".rspec"
|
|
93
93
|
- ".rubocop.yml"
|
|
94
94
|
- ".travis.yml"
|
|
95
|
+
- CHANGELOG.md
|
|
95
96
|
- Gemfile
|
|
96
97
|
- LICENSE.txt
|
|
97
98
|
- README.md
|
|
@@ -107,7 +108,7 @@ homepage: https://github.com/yabeda-rb/yabeda-sidekiq
|
|
|
107
108
|
licenses:
|
|
108
109
|
- MIT
|
|
109
110
|
metadata: {}
|
|
110
|
-
post_install_message:
|
|
111
|
+
post_install_message:
|
|
111
112
|
rdoc_options: []
|
|
112
113
|
require_paths:
|
|
113
114
|
- lib
|
|
@@ -122,9 +123,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
122
123
|
- !ruby/object:Gem::Version
|
|
123
124
|
version: '0'
|
|
124
125
|
requirements: []
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
signing_key:
|
|
126
|
+
rubygems_version: 3.1.2
|
|
127
|
+
signing_key:
|
|
128
128
|
specification_version: 4
|
|
129
129
|
summary: Extensible Prometheus exporter for monitoring your Sidekiq
|
|
130
130
|
test_files: []
|