sidekiq-prometheus-exporter 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/CLAUDE.md +24 -0
- data/README.md +25 -21
- data/Rakefile +44 -3
- data/lib/sidekiq/prometheus/exporter/standard.rb +18 -0
- data/lib/sidekiq/prometheus/exporter/templates/standard.erb +20 -0
- data/lib/sidekiq/prometheus/exporter/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 8fa8d0cd726729ad9e980344384cabd91fc4e71c7b713e3482f61a43f7a0bf5d
|
|
4
|
+
data.tar.gz: 8f7810985f4497f597e6767eaa664e9b5fb33c95623355af07f197ccbf19eb3f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 58e8b3b83cde2b42c88d83ceac3e7f8ff373baa5aa9261a33472b3f9ff322f25fd70764b540014f2a64ac57e85ea7e158c62c09e066e929b543f484eec30d612
|
|
7
|
+
data.tar.gz: 20151dc55a16609602a7ddeace1196a80e60a1951030b3eba6e33bdf557c43f73eae4ccf2f327f6b5aee80363d29f44a378df80024488587bb461452d1b21c50
|
data/.gitignore
CHANGED
data/CLAUDE.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Architecture
|
|
2
|
+
|
|
3
|
+
Ruby gem + Docker image + Helm chart. Gem is the core product, Docker wraps it as standalone Rack app, Helm deploys the Docker image to Kubernetes.
|
|
4
|
+
|
|
5
|
+
## Gem
|
|
6
|
+
|
|
7
|
+
Rack app that scrapes Sidekiq internals and renders Prometheus text format via ERB templates.
|
|
8
|
+
|
|
9
|
+
- Entry point: `lib/sidekiq/prometheus/exporter.rb` — Rack `call` interface, also mounts into Sidekiq::Web
|
|
10
|
+
- Exporters: standard (always on), cron (sidekiq-cron), scheduler (sidekiq-scheduler) — auto-detected via `available?`
|
|
11
|
+
- Metrics output: ERB templates in `lib/sidekiq/prometheus/exporter/templates/`
|
|
12
|
+
- Multi-version support: Appraisal gemfiles for Sidekiq 4.x through 8.x+latest, Rakefile picks spec directory by Sidekiq version
|
|
13
|
+
|
|
14
|
+
## Versioning
|
|
15
|
+
|
|
16
|
+
- `VERSION` in `lib/sidekiq/prometheus/exporter/version.rb` — gem, Docker base, Helm appVersion
|
|
17
|
+
- `DOCKER_PATCH_VERSION` — bumped for Docker-only changes, resets on VERSION bump
|
|
18
|
+
- Helm chart `version` uses `-X` suffix for chart-only changes (e.g. `0.3.1-1`)
|
|
19
|
+
|
|
20
|
+
## Commands
|
|
21
|
+
|
|
22
|
+
- `bundle exec rake spec` — run specs (auto-selects spec dir by Sidekiq version)
|
|
23
|
+
- `bundle exec rake docker:release` — build+push multi-arch Docker image
|
|
24
|
+
- `bundle exec rake helm:generate` — package chart and regenerate index.yaml
|
data/README.md
CHANGED
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
[](https://codeclimate.com/github/Strech/sidekiq-prometheus-exporter/maintainability)
|
|
3
3
|
|
|
4
4
|
[Sidekiq Enterprise]: https://sidekiq.org/products/enterprise.html
|
|
5
|
+
[Sidekiq v6.5.2]: https://github.com/sidekiq/sidekiq/releases/tag/v6.5.2
|
|
5
6
|
|
|
6
7
|
# Sidekiq Prometheus Exporter
|
|
7
8
|
|
|
@@ -11,7 +12,7 @@
|
|
|
11
12
|
|
|
12
13
|
|
|
13
14
|
|
|
14
|
-
Open [dashboard example file](/examples/sidekiq-dashboard.grafana-
|
|
15
|
+
Open [dashboard example file](/examples/sidekiq-dashboard.grafana-11.json) (grafana 11), then open `https://<your grafana-url>/dashboard/import` and paste the content of the file.
|
|
15
16
|
|
|
16
17
|
---
|
|
17
18
|
|
|
@@ -26,26 +27,29 @@ _(starting Sidekiq `v4.1.0`)_
|
|
|
26
27
|
|
|
27
28
|
## Standard
|
|
28
29
|
|
|
29
|
-
| Name | Type | Description
|
|
30
|
-
| ----------------------------------------- | ------- |
|
|
31
|
-
| sidekiq_processed_jobs_total | counter | The total number of processed jobs
|
|
32
|
-
| sidekiq_failed_jobs_total | counter | The total number of failed jobs
|
|
33
|
-
| sidekiq_workers | gauge | The number of workers across all the processes
|
|
34
|
-
| sidekiq_processes | gauge | The number of processes
|
|
35
|
-
| sidekiq_host_processes | gauge | The number of processes running on the host (labels: `host`, `quiet`)
|
|
36
|
-
| sidekiq_host_processes_memory_usage_bytes | gauge | The amount of real memory (resident set) used by the processes running on the host (labels: `host`)
|
|
37
|
-
| sidekiq_busy_workers | gauge | The number of workers performing the job
|
|
38
|
-
| sidekiq_enqueued_jobs | gauge | The number of enqueued jobs
|
|
39
|
-
| sidekiq_scheduled_jobs | gauge | The number of jobs scheduled for a future execution
|
|
40
|
-
| sidekiq_retry_jobs | gauge | The number of jobs scheduled for the next try
|
|
41
|
-
| sidekiq_dead_jobs | gauge | The number of jobs being dead
|
|
42
|
-
| sidekiq_queue_latency_seconds | gauge | The number of seconds between oldest job being pushed to the queue and current time (labels: `name`)
|
|
43
|
-
| sidekiq_queue_max_processing_time_seconds | gauge | The number of seconds between oldest job of the queue being executed and current time (labels: `name`)
|
|
44
|
-
| sidekiq_queue_enqueued_jobs | gauge | The number of enqueued jobs in the queue (labels: `name`)
|
|
45
|
-
| sidekiq_queue_workers | gauge | The number of workers serving the queue (labels: `name`)
|
|
46
|
-
| sidekiq_queue_processes | gauge | The number of processes serving the queue (labels: `name`)
|
|
47
|
-
| sidekiq_queue_busy_workers | gauge | The number of workers performing the job for the queue (labels: `name`)
|
|
48
|
-
| sidekiq_leader_lifetime_seconds | gauge | The number of seconds since cluster leader has been created (
|
|
30
|
+
| Name | Type | Description |
|
|
31
|
+
| ----------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------- |
|
|
32
|
+
| sidekiq_processed_jobs_total | counter | The total number of processed jobs |
|
|
33
|
+
| sidekiq_failed_jobs_total | counter | The total number of failed jobs |
|
|
34
|
+
| sidekiq_workers | gauge | The number of workers across all the processes |
|
|
35
|
+
| sidekiq_processes | gauge | The number of processes |
|
|
36
|
+
| sidekiq_host_processes | gauge | The number of processes running on the host (labels: `host`, `quiet`) |
|
|
37
|
+
| sidekiq_host_processes_memory_usage_bytes | gauge | The amount of real memory (resident set) used by the processes running on the host (labels: `host`) |
|
|
38
|
+
| sidekiq_busy_workers | gauge | The number of workers performing the job |
|
|
39
|
+
| sidekiq_enqueued_jobs | gauge | The number of enqueued jobs |
|
|
40
|
+
| sidekiq_scheduled_jobs | gauge | The number of jobs scheduled for a future execution |
|
|
41
|
+
| sidekiq_retry_jobs | gauge | The number of jobs scheduled for the next try |
|
|
42
|
+
| sidekiq_dead_jobs | gauge | The number of jobs being dead |
|
|
43
|
+
| sidekiq_queue_latency_seconds | gauge | The number of seconds between oldest job being pushed to the queue and current time (labels: `name`) |
|
|
44
|
+
| sidekiq_queue_max_processing_time_seconds | gauge | The number of seconds between oldest job of the queue being executed and current time (labels: `name`) |
|
|
45
|
+
| sidekiq_queue_enqueued_jobs | gauge | The number of enqueued jobs in the queue (labels: `name`) |
|
|
46
|
+
| sidekiq_queue_workers | gauge | The number of workers serving the queue (labels: `name`) |
|
|
47
|
+
| sidekiq_queue_processes | gauge | The number of processes serving the queue (labels: `name`) |
|
|
48
|
+
| sidekiq_queue_busy_workers | gauge | The number of workers performing the job for the queue (labels: `name`) |
|
|
49
|
+
| sidekiq_leader_lifetime_seconds | gauge | The number of seconds since cluster leader has been created (only [Sidekiq Enterprise]) |
|
|
50
|
+
| sidekiq_job_processed | gauge | The number of processed jobs per job class over the last hour (labels: `class`, since [Sidekiq v6.5.2]) |
|
|
51
|
+
| sidekiq_job_failed | gauge | The number of failed jobs per job class over the last hour (labels: `class`, since [Sidekiq v6.5.2]) |
|
|
52
|
+
| sidekiq_job_execution_time_seconds | gauge | The total execution time in seconds per job class over the last hour (labels: `class`, since [Sidekiq v6.5.2]) |
|
|
49
53
|
|
|
50
54
|
<details>
|
|
51
55
|
<summary>Click to expand for all available contribs</summary>
|
data/Rakefile
CHANGED
|
@@ -22,6 +22,7 @@ task default: :spec
|
|
|
22
22
|
|
|
23
23
|
require 'English'
|
|
24
24
|
require 'fileutils'
|
|
25
|
+
require 'yaml'
|
|
25
26
|
require_relative 'lib/sidekiq/prometheus/exporter/version'
|
|
26
27
|
|
|
27
28
|
VERSION = Sidekiq::Prometheus::Exporter::VERSION
|
|
@@ -78,10 +79,50 @@ namespace :docker do
|
|
|
78
79
|
end
|
|
79
80
|
|
|
80
81
|
namespace :helm do
|
|
82
|
+
desc "Release Helm chart: package, update gh-pages index, attach archive to GitHub release v#{VERSION}"
|
|
83
|
+
task :release, %i(version) do |_, args|
|
|
84
|
+
args.with_defaults(version: VERSION)
|
|
85
|
+
|
|
86
|
+
system('git diff-index --quiet HEAD --') or abort('Working tree has uncommitted changes')
|
|
87
|
+
|
|
88
|
+
current_branch = execute('git branch --show-current').strip
|
|
89
|
+
archive_dir_suffix = Time.now.to_i
|
|
90
|
+
archive_dir_path = File.expand_path("./tmp/archive-#{archive_dir_suffix}")
|
|
91
|
+
chart_version = YAML.load_file(
|
|
92
|
+
File.expand_path('./helm/sidekiq-prometheus-exporter/Chart.yaml')
|
|
93
|
+
).fetch('version')
|
|
94
|
+
|
|
95
|
+
Rake::Task['helm:generate'].invoke(args.version, archive_dir_suffix)
|
|
96
|
+
|
|
97
|
+
tgz = Dir.glob(File.join(archive_dir_path, '*.tgz')).first
|
|
98
|
+
abort 'No .tgz file found' unless tgz
|
|
99
|
+
|
|
100
|
+
index = File.join(archive_dir_path, 'index.yaml')
|
|
101
|
+
|
|
102
|
+
begin
|
|
103
|
+
execute('git checkout gh-pages')
|
|
104
|
+
FileUtils.cp(index, './index.yaml')
|
|
105
|
+
|
|
106
|
+
if execute('git diff index.yaml').strip.empty?
|
|
107
|
+
puts 'index.yaml unchanged, skipping commit'
|
|
108
|
+
else
|
|
109
|
+
execute('git add index.yaml')
|
|
110
|
+
execute("git commit -m 'Bump Helm chart version to #{chart_version}'")
|
|
111
|
+
execute('git push origin gh-pages')
|
|
112
|
+
puts "gh-pages updated with chart version #{chart_version}"
|
|
113
|
+
end
|
|
114
|
+
ensure
|
|
115
|
+
execute("git checkout #{current_branch}")
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
execute("gh release upload v#{VERSION} #{tgz}")
|
|
119
|
+
puts "#{File.basename(tgz)} attached to release v#{VERSION}"
|
|
120
|
+
end
|
|
121
|
+
|
|
81
122
|
desc 'Generate new Helm repo index'
|
|
82
|
-
task :generate, %i(version) do |_, args|
|
|
83
|
-
args.with_defaults(version: docker_version)
|
|
84
|
-
archive_dir = File.expand_path("./tmp/archive-#{
|
|
123
|
+
task :generate, %i(version suffix) do |_, args|
|
|
124
|
+
args.with_defaults(version: docker_version, suffix: Time.now.to_i)
|
|
125
|
+
archive_dir = File.expand_path("./tmp/archive-#{args.suffix}")
|
|
85
126
|
|
|
86
127
|
Rake::Task['helm:package'].invoke(archive_dir)
|
|
87
128
|
Rake::Task['helm:index'].invoke(archive_dir, args.version)
|
|
@@ -8,7 +8,9 @@ module Sidekiq
|
|
|
8
8
|
module Exporter
|
|
9
9
|
class Standard
|
|
10
10
|
UNKNOWN_IDENTITY = 'unknown-identity'.freeze
|
|
11
|
+
MINUTES_IN_HOUR = 60
|
|
11
12
|
BYTES_IN_KILOBYTE = 1024
|
|
13
|
+
MILLISECONDS_IN_SECOND = 1000
|
|
12
14
|
OMIT_NEWLINES_MODE = '<>'.freeze
|
|
13
15
|
TEMPLATE = ERB.new(
|
|
14
16
|
File.read(File.expand_path('templates/standard.erb', __dir__)),
|
|
@@ -23,6 +25,7 @@ module Sidekiq
|
|
|
23
25
|
WorkersStats = Struct.new(
|
|
24
26
|
:total_workers, :by_queue, :by_host, :leader_lifetime, keyword_init: true
|
|
25
27
|
)
|
|
28
|
+
JobStats = Struct.new(:processed, :failed, :execution_time, keyword_init: true)
|
|
26
29
|
|
|
27
30
|
def self.available?
|
|
28
31
|
true
|
|
@@ -33,6 +36,7 @@ module Sidekiq
|
|
|
33
36
|
@show_memory_usage = false
|
|
34
37
|
|
|
35
38
|
@overview_stats = Sidekiq::Stats.new
|
|
39
|
+
@jobs_stats = jobs_stats
|
|
36
40
|
@queues_stats = queues_stats
|
|
37
41
|
@workers_stats = workers_stats
|
|
38
42
|
@max_processing_times = max_processing_times
|
|
@@ -98,6 +102,20 @@ module Sidekiq
|
|
|
98
102
|
kilobytes * BYTES_IN_KILOBYTE
|
|
99
103
|
end
|
|
100
104
|
|
|
105
|
+
# NOTE: available only starting v6.5.2
|
|
106
|
+
def jobs_stats
|
|
107
|
+
return unless defined?(Sidekiq::Metrics::Query)
|
|
108
|
+
|
|
109
|
+
Sidekiq::Metrics::Query.new.top_jobs(minutes: MINUTES_IN_HOUR)
|
|
110
|
+
.job_results.transform_values! do |result|
|
|
111
|
+
JobStats.new(
|
|
112
|
+
processed: result.totals['p'],
|
|
113
|
+
failed: result.totals['f'],
|
|
114
|
+
execution_time: result.totals['ms'].fdiv(MILLISECONDS_IN_SECOND)
|
|
115
|
+
)
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
|
|
101
119
|
def max_processing_times
|
|
102
120
|
return new_max_processing_times if Sidekiq.const_defined?(:Work)
|
|
103
121
|
|
|
@@ -91,3 +91,23 @@ sidekiq_queue_busy_workers{name="<%= queue %>"} <%= format('%i', stats.busy_work
|
|
|
91
91
|
# TYPE sidekiq_leader_lifetime_seconds gauge
|
|
92
92
|
sidekiq_leader_lifetime_seconds <%= format('%d', @workers_stats.leader_lifetime) %>
|
|
93
93
|
<% end %>
|
|
94
|
+
<% if @jobs_stats %>
|
|
95
|
+
|
|
96
|
+
# HELP sidekiq_job_processed The number of processed jobs per job class over the last hour.
|
|
97
|
+
# TYPE sidekiq_job_processed gauge
|
|
98
|
+
<% @jobs_stats.each do |klass, stats| %>
|
|
99
|
+
sidekiq_job_processed{class="<%= klass %>"} <%= format('%d', stats.processed) %>
|
|
100
|
+
<% end %>
|
|
101
|
+
|
|
102
|
+
# HELP sidekiq_job_failed The number of failed jobs per job class over the last hour.
|
|
103
|
+
# TYPE sidekiq_job_failed gauge
|
|
104
|
+
<% @jobs_stats.each do |klass, stats| %>
|
|
105
|
+
sidekiq_job_failed{class="<%= klass %>"} <%= format('%d', stats.failed) %>
|
|
106
|
+
<% end %>
|
|
107
|
+
|
|
108
|
+
# HELP sidekiq_job_execution_time_seconds The total execution time in seconds per job class over the last hour.
|
|
109
|
+
# TYPE sidekiq_job_execution_time_seconds gauge
|
|
110
|
+
<% @jobs_stats.each do |klass, stats| %>
|
|
111
|
+
sidekiq_job_execution_time_seconds{class="<%= klass %>"} <%= format('%.3f', stats.execution_time) %>
|
|
112
|
+
<% end %>
|
|
113
|
+
<% end %>
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: sidekiq-prometheus-exporter
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Sergey Fedorov
|
|
@@ -48,6 +48,7 @@ files:
|
|
|
48
48
|
- ".rspec"
|
|
49
49
|
- ".rubocop.yml"
|
|
50
50
|
- Appraisals
|
|
51
|
+
- CLAUDE.md
|
|
51
52
|
- Gemfile
|
|
52
53
|
- LICENSE
|
|
53
54
|
- README.md
|