sidekiq-prometheus-exporter 0.1.12 → 0.1.16

data/README.md

@@ -1,6 +1,5 @@
-[![Build Status](https://travis-ci.org/Strech/sidekiq-prometheus-exporter.svg?branch=master)](https://travis-ci.org/Strech/sidekiq-prometheus-exporter)
+![CI status](https://github.com/Strech/sidekiq-prometheus-exporter/workflows/CI/badge.svg)
 [![Maintainability](https://api.codeclimate.com/v1/badges/bb1b30cd7aca8ecc9413/maintainability)](https://codeclimate.com/github/Strech/sidekiq-prometheus-exporter/maintainability)
-[![Test Coverage](https://api.codeclimate.com/v1/badges/bb1b30cd7aca8ecc9413/test_coverage)](https://codeclimate.com/github/Strech/sidekiq-prometheus-exporter/test_coverage)
 
 # Sidekiq Prometheus Exporter
 
@@ -10,42 +9,57 @@
 
 ![Grafana dashboard example](/examples/screenshot.png)
 
-Open [dashboard example file](/examples/sidekiq.json), then open `https://<your grafana-url>/dashboard/import` and paste the content of the file.
+Open [dashboard example file](/examples/sidekiq-dashboard.grafana-7.json) (grafana 7), then open `https://<your grafana-url>/dashboard/import` and paste the content of the file.
+
+---
+
+#### If you like the project and want to support me on my sleepless nights, you can
+
+[![Support via PayPal](https://cdn.rawgit.com/twolfson/paypal-github-button/1.0.0/dist/button.svg)](https://www.paypal.com/paypalme/onistrech/eur5.0)
+[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/W7W8367XJ)
 
 # Available metrics
 
-*(starting Sidekiq `v3.3.1`)*
+_(starting Sidekiq `v3.3.1`)_
 
-### Standard
+## Standard
 
-| Name                                      | Type    | Description             |
-|-------------------------------------------|---------|-------------------------|
-| sidekiq_processed_jobs_total              | counter | The total number of processed jobs
-| sidekiq_failed_jobs_total                 | counter | The total number of failed jobs
-| sidekiq_workers                           | gauge   | The number of workers across all the processes
-| sidekiq_processes                         | gauge   | The number of processes
-| sidekiq_busy_workers                      | gauge   | The number of workers performing the job
-| sidekiq_enqueued_jobs                     | gauge   | The number of enqueued jobs
-| sidekiq_scheduled_jobs                    | gauge   | The number of jobs scheduled for a future execution
-| sidekiq_retry_jobs                        | gauge   | The number of jobs scheduled for the next try
-| sidekiq_dead_jobs                         | gauge   | The number of jobs being dead
-| sidekiq_queue_latency_seconds             | gauge   | The number of seconds between oldest job being pushed to the queue and current time (labels: `name`)
-| sidekiq_queue_max_processing_time_seconds | gauge   | The number of seconds between oldest job of the queue being executed and current time (labels: `name`)
-| sidekiq_queue_enqueued_jobs               | gauge   | The number of enqueued jobs in the queue (labels: `name`)
+| Name                                      | Type    | Description                                                                                            |
+| ----------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------ |
+| sidekiq_processed_jobs_total              | counter | The total number of processed jobs                                                                     |
+| sidekiq_failed_jobs_total                 | counter | The total number of failed jobs                                                                        |
+| sidekiq_workers                           | gauge   | The number of workers across all the processes                                                         |
+| sidekiq_processes                         | gauge   | The number of processes                                                                                |
+| sidekiq_busy_workers                      | gauge   | The number of workers performing the job                                                               |
+| sidekiq_enqueued_jobs                     | gauge   | The number of enqueued jobs                                                                            |
+| sidekiq_scheduled_jobs                    | gauge   | The number of jobs scheduled for a future execution                                                    |
+| sidekiq_retry_jobs                        | gauge   | The number of jobs scheduled for the next try                                                          |
+| sidekiq_dead_jobs                         | gauge   | The number of jobs being dead                                                                          |
+| sidekiq_queue_latency_seconds             | gauge   | The number of seconds between oldest job being pushed to the queue and current time (labels: `name`)   |
+| sidekiq_queue_max_processing_time_seconds | gauge   | The number of seconds between oldest job of the queue being executed and current time (labels: `name`) |
+| sidekiq_queue_enqueued_jobs               | gauge   | The number of enqueued jobs in the queue (labels: `name`)                                              |
+| sidekiq_queue_workers                     | gauge   | The number of workers serving the queue (labels: `name`)                                               |
+| sidekiq_queue_processes                   | gauge   | The number of processes serving the queue (labels: `name`)                                             |
+| sidekiq_queue_busy_workers                | gauge   | The number of workers performing the job for the queue (labels: `name`)                                |
 
-### [Scheduler](https://github.com/moove-it/sidekiq-scheduler)
+<details>
+  <summary>Click to expand for all available contribs</summary>
 
-| Name                                          | Type  | Description           |
-|-----------------------------------------------|-------|-----------------------|
-| sidekiq_scheduler_jobs                        | gauge | The number of recurring jobs
-| sidekiq_scheduler_enabled_jobs                | gauge | The number of enabled recurring jobs
-| sidekiq_scheduler_time_since_last_run_minutes | gauge | The number of minutes since the last recurring job was executed and current time (labels: `name`)
+## [Scheduler](https://github.com/moove-it/sidekiq-scheduler)
 
-### [Cron](https://github.com/ondrejbartas/sidekiq-cron)
+| Name                                          | Type  | Description                                                                                       |
+| --------------------------------------------- | ----- | ------------------------------------------------------------------------------------------------- |
+| sidekiq_scheduler_jobs                        | gauge | The number of recurring jobs                                                                      |
+| sidekiq_scheduler_enabled_jobs                | gauge | The number of enabled recurring jobs                                                              |
+| sidekiq_scheduler_time_since_last_run_minutes | gauge | The number of minutes since the last recurring job was executed and current time (labels: `name`) |
 
-| Name                                      | Type    | Description             |
-|-------------------------------------------|---------|-------------------------|
-| sidekiq_cron_jobs                         | gauge   | The number of cron jobs
+## [Cron](https://github.com/ondrejbartas/sidekiq-cron)
+
+| Name              | Type  | Description             |
+| ----------------- | ----- | ----------------------- |
+| sidekiq_cron_jobs | gauge | The number of cron jobs |
+
+</details>
 
 # Installation
 
@@ -57,17 +71,17 @@ gem 'sidekiq-prometheus-exporter', '~> 0.1'
 
 And then execute:
 
-```bash
+```console
 $ bundle
 ```
 
 Or install it yourself as:
 
-```bash
+```console
 $ gem install sidekiq-prometheus-exporter -v '~> 0.1'
 ```
 
-# Rack application
+## Rack application
 
 For a fresh new application to expose metrics create `config.ru` file with
 next code inside
@@ -85,13 +99,13 @@ run Sidekiq::Prometheus::Exporter.to_app
 
 Use your favorite server to start it up, like this
 
-```bash
+```console
 $ bundle exec rackup -p9292 -o0.0.0.0
 ```
 
 and then `curl https://0.0.0.0:9292/metrics`
 
-# Rails application
+## Rails application
 
 When you have rails application, it's possible to mount exporter
 as a rack application in your `routes.rb`
@@ -109,13 +123,13 @@ end
 
 Use rails server from `bin` folder to start it up, like this
 
-```bash
+```console
 $ ./bin/rails s -p 9292 -b 0.0.0.0
 ```
 
 and then `curl https://0.0.0.0:9292/metrics`
 
-# Sidekiq Web (extream)
+## Sidekiq Web (extream)
 
 If you are ok with metrics being exposed via Sidekiq web dashboard because
 you have it inside your private network or only Prometheus scraper will have access
@@ -130,7 +144,52 @@ Sidekiq::Web.register(Sidekiq::Prometheus::Exporter)
 
 and then `curl https://<your-sidekiq-web-uri>/metrics`
 
-# Sidekiq Contribs
+## Docker
+
+If we are talking about isolation you can run already prepared official
+rack application in the Docker container by using the [public image](https://hub.docker.com/r/strech/sidekiq-prometheus-exporter)
+(check out this [README](/docker/README.md) for more)
+
+```bash
+$ docker run -it --rm \
+             -p 9292:9292 \
+             -e REDIS_URL=redis://<your-redis-host>:6379/0 \
+             strech/sidekiq-prometheus-exporter
+```
+
+and then `curl https://0.0.0.0:9292/metrics`
+
+## Helm
+
+And finally the cloud solution _(who don't these days)_. Easy to install, easy
+to use. A fully-functioning Helm-package based on official [Docker
+image](https://hub.docker.com/r/strech/sidekiq-prometheus-exporter), comes with lots of [configuration
+options](https://github.com/Strech/sidekiq-prometheus-exporter/blob/master/helm/sidekiq-prometheus-exporter/README.md)
+
+```console
+$ helm repo add strech https://strech.github.io/sidekiq-prometheus-exporter
+"strech" has been added to your repositories
+
+$ helm install strech/sidekiq-prometheus-exporter --name sidekiq-metrics
+```
+
+to `curl` your metrics, please follow the post-installation guide
+
+# Tips&Tricks
+
+If you want to see at the exporter startup time a banner about which exporters
+are enabled add this call to your `config.ru` (but after exporter `configure` statement)
+
+```ruby
+require 'sidekiq/prometheus/exporter'
+
+puts Sidekiq::Prometheus::Exporter.banner
+```
+
+:anger: if you don't see your banner try to output into `STDERR` instead of
+`STDOUT`
+
+## Sidekiq Contribs
 
 By default we try to detect as many as possible [sidekiq contribs](https://github.com/mperham/sidekiq/wiki/Related-Projects)
 and add their metrics to the output.
@@ -150,20 +209,9 @@ Sidekiq::Prometheus::Exporter.configure do |config|
 end
 ```
 
-# Tips&Tricks
-
-If you want to see at the exporter startup time a banner about which exporters
-are enabled add this call to your `config.ru` (but after exporter `configure` statement)
-
-```ruby
-require 'sidekiq/prometheus/exporter'
-
-puts Sidekiq::Prometheus::Exporter.banner
-```
-
-:anger: if you don't see your banner try to output into `STDERR` instead of `STDOUT`
+:bulb: if you did't find the contrib you would like to see, don't hesitate to [open an issue](https://github.com/Strech/sidekiq-prometheus-exporter/issues/new) and describe what do you think we should export.
 
-## Contributing
+# Contributing
 
 Bug reports and pull requests to support earlier versions of Sidekiq are welcome on GitHub at https://github.com/Strech/sidekiq-prometheus-exporter/issues.
 

data/Rakefile

@@ -4,3 +4,86 @@ require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec)
 
 task default: :spec
+
+# ==============================================================================
+
+require 'English'
+require 'fileutils'
+require_relative 'lib/sidekiq/prometheus/exporter/version'
+
+VERSION = Sidekiq::Prometheus::Exporter::VERSION
+
+def execute(command)
+  output = `#{command}`
+
+  unless $CHILD_STATUS.success?
+    warn output
+    exit 1
+  end
+
+  output
+end
+
+namespace :docker do
+  desc "Release new Docker image strech/sidekiq-prometheus-exporter:#{VERSION} (latest)"
+  task :release, %i(patch) do |_, args|
+    version = [VERSION, args.patch].compact.join('-')
+
+    Rake::Task['docker:build'].invoke(version)
+    Rake::Task['docker:push'].invoke(version)
+  end
+
+  task :build, %i(version) do |_, args|
+    args.with_defaults(version: VERSION)
+    image = 'strech/sidekiq-prometheus-exporter'
+
+    Dir.chdir(File.expand_path('./docker')) do
+      execute("docker build -t #{image}:#{args.version} -t #{image}:latest .")
+    end
+
+    puts "Successfully built strech/sidekiq-prometheus-exporter and tagged #{args.version} (latest)"
+  end
+
+  task :push, %i(version) do |_, args|
+    args.with_defaults(version: VERSION)
+    image = 'strech/sidekiq-prometheus-exporter'
+
+    execute("docker push #{image}:#{args.version}")
+    execute("docker push #{image}:latest")
+
+    puts "Successfully pushed strech/sidekiq-prometheus-exporter:#{args.version} (latest)"
+  end
+end
+
+namespace :helm do
+  desc 'Generate new Helm repo index'
+  task :generate, %i(patch) do |_, args|
+    version = [VERSION, args.patch].compact.join('-')
+    archive_dir = File.expand_path("./tmp/archive-#{Time.now.to_i}")
+
+    Rake::Task['helm:package'].invoke(archive_dir)
+    Rake::Task['helm:index'].invoke(archive_dir, version)
+
+    puts "New index generated: #{File.join(archive_dir, 'index.yaml')}"
+  end
+
+  task :package, %i(directory) do |_, args|
+    chart_dir = File.expand_path('./helm/sidekiq-prometheus-exporter')
+    archive_dir = args.fetch(:directory) { File.expand_path("./tmp/archive-#{Time.now.to_i}") }
+
+    FileUtils.mkdir_p(archive_dir)
+
+    execute("helm package #{chart_dir} -d #{archive_dir}")
+  end
+
+  task :index, %i(directory version) do |_, args|
+    args.with_defaults(version: VERSION)
+
+    Dir.chdir(args.fetch(:directory)) do
+      url = "https://github.com/Strech/sidekiq-prometheus-exporter/releases/download/v#{args.version}"
+
+      execute('git show gh-pages:index.yaml > existing-index.yaml')
+      execute("helm repo index . --url #{url} --merge existing-index.yaml")
+    end
+  end
+end

data/docker/Dockerfile

@@ -0,0 +1,24 @@
+FROM ruby:2.7.3-alpine
+
+LABEL maintainer="Sergey Fedorov <oni.strech@gmail.com>"
+LABEL repository="strech/sidekiq-prometheus-exporter"
+
+ENV RACK_VERSION 2.0.9
+ENV SIDEKIQ_VERSION 6.2.1
+ENV REDIS_NAMESPACE_VERSION 1.8.1
+ENV SIDEKIQ_PROMETHEUS_EXPORTER_VERSION 0.1.16
+
+RUN    addgroup -S exporter \
+    && adduser -s /bin/false -SDHg exporter exporter \
+    && gem install "rack:$RACK_VERSION" \
+    && gem install "sidekiq:$SIDEKIQ_VERSION" \
+    && gem install "redis-namespace:$REDIS_NAMESPACE_VERSION" \
+    && gem install "sidekiq-prometheus-exporter:$SIDEKIQ_PROMETHEUS_EXPORTER_VERSION"
+
+USER exporter
+WORKDIR /app
+COPY config.ru config.ru
+
+EXPOSE 9292
+ENTRYPOINT [ "rackup" ]
+CMD [ "-p", "9292", "-o", "0.0.0.0", "config.ru" ]

data/docker/README.md

@@ -0,0 +1,61 @@
+# Official Docker image
+
+[0]: https://github.com/Strech/sidekiq-prometheus-exporter
+[1]: https://www.iana.org/assignments/uri-schemes/prov/redis
+[2]: https://github.com/resque/redis-namespace
+[3]: https://github.com/redis/redis-rb/tree/v4.1.3#sentinel-support
+[4]: https://github.com/mperham/sidekiq/issues/4479
+
+This is an official Docker image for [sidekiq-prometheus-exporter][0]
+gem.
+
+It combines some approaches which were already published as images by other
+devs and at the same time brings more flexibility for the stock configuration
+and applies recommended Docker best practices.
+
+:warning: The Docker image currently supports **only** standards metrics.
+
+## Supported ENVs
+
+Required
+
+- `REDIS_URL` - [RFC 3986 generic URI][1], exclusive with `REDIS_HOST`
+- `REDIS_HOST` - a Redis host, exclusive with `REDIS_URL` (default: `localhost`)
+
+Optional
+
+- `REDIS_SSL` - a flag to enable Redis SSL URL scheme (default: `false`)
+- `REDIS_PORT` - a Redis port (default: `6379`)
+- `REDIS_PASSWORD` - a Redis password (if you need one)
+- `REDIS_DB_NUMBER` - a Redis database number (default: `0`)
+- `REDIS_NAMESPACE` - a Redis [namespace][2] name (if you have separated sidekiq)
+- `REDIS_SENTINELS` - a list of comma separated Redis urls (like `REDIS_URL`, but for sentinels)
+- `REDIS_SENTINEL_ROLE` - a role within the [sentinel][3] to connect (default: `master`)
+- `REDIS_DISABLE_CLIENT_ID` - a flag to disable `CLIENT` command in [Sidekiq][4]
+
+:bulb: Note, that `REDIS_URL` and `REDIS_HOST` are exclusive. Since `REDIS_HOST` is more
+atomic value it will be checked after `REDIS_URL`.
+
+:bulb: `REDIS_SENTINELS` will be parsed with `URI`, because of that it's
+mandatory for them to be formatted with protocol `redis://...`.
+
+:bulb: `REDIS_DISABLE_CLIENT_ID` might be required if a SaaS Redis provider is being used.
+
+## Examples
+
+If you don't have a running Redis instance, you can quickly spin an empty to
+practice.
+
+```bash
+$ docker run -d --rm --name redis-instance redis
+```
+
+and then run an exporter
+
+```bash
+$ docker run -it --rm \
+             --link redis-instance \
+             -p 9292:9292 \
+             -e REDIS_URL=redis://redis-instance \
+             strech/sidekiq-prometheus-exporter
+```

data/docker/config.ru

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require 'sidekiq'
+require 'sidekiq/prometheus/exporter'
+
+config = {}
+config[:url] = ENV['REDIS_URL'] if ENV.key?('REDIS_URL')
+
+unless config.key?(:url)
+  scheme = (ENV.fetch('REDIS_SSL', 'false') == 'true') ? 'rediss' : 'redis'
+  host = ENV.fetch('REDIS_HOST', 'localhost')
+  port = ENV.fetch('REDIS_PORT', 6379)
+  db_number = ENV.fetch('REDIS_DB_NUMBER', 0)
+  password = ":#{ENV['REDIS_PASSWORD']}" if ENV.key?('REDIS_PASSWORD')
+
+  config[:url] = "#{scheme}://#{password}@#{host}:#{port}/#{db_number}"
+end
+
+if ENV.key?('REDIS_NAMESPACE')
+  require 'redis-namespace'
+
+  config[:namespace] = ENV['REDIS_NAMESPACE']
+end
+
+if ENV.key?('REDIS_SENTINELS')
+  require 'uri'
+
+  config[:role] = ENV.fetch('REDIS_SENTINEL_ROLE', :master).to_sym
+  config[:sentinels] = ENV['REDIS_SENTINELS'].split(',').map do |url|
+    uri = URI.parse(url.strip)
+    cfg = {host: uri.host || 'localhost', port: uri.port || 26379} # rubocop:disable Style/NumericLiterals
+    cfg[:password] = ":#{uri.password}" if uri.password
+    cfg
+  end
+end
+
+config[:id] = nil if ENV.key?('REDIS_DISABLE_CLIENT_ID')
+
+Sidekiq.configure_client { |client| client.redis = config }
+
+run Sidekiq::Prometheus::Exporter.to_app

data/examples/docker-compose.yml

@@ -1,5 +1,5 @@
 ---
-version: '3.4'
+version: "3.4"
 
 services:
   metrics:
@@ -31,7 +31,7 @@ services:
       - metrics
 
   grafana:
-    image: grafana/grafana
+    image: grafana/grafana:7.1.1
     volumes:
       - grafana-data:/var/lib/grafana
     ports:

data/examples/metrics/config.ru

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require 'sidekiq'
 require 'sidekiq/prometheus/exporter'
 

data/examples/sidekiq/sidekiq.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class SleepyWorker
   include Sidekiq::Worker
 

data/examples/{sidekiq.json → sidekiq-dashboard.grafana-6.json}

@@ -22,11 +22,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": false,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -102,11 +98,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": false,
-      "colors": [
-        "#d44a3a",
-        "rgba(237, 129, 40, 0.89)",
-        "#299c46"
-      ],
+      "colors": ["#d44a3a", "rgba(237, 129, 40, 0.89)", "#299c46"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -182,11 +174,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": true,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -262,11 +250,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": true,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -342,11 +326,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": true,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -422,11 +402,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": false,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -502,11 +478,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": false,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -582,11 +554,7 @@
       "cacheTimeout": null,
       "colorBackground": false,
       "colorValue": false,
-      "colors": [
-        "#299c46",
-        "rgba(237, 129, 40, 0.89)",
-        "#d44a3a"
-      ],
+      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
       "datasource": null,
       "format": "none",
       "gauge": {
@@ -860,20 +828,10 @@
       "2h",
       "1d"
     ],
-    "time_options": [
-      "5m",
-      "15m",
-      "1h",
-      "6h",
-      "12h",
-      "24h",
-      "2d",
-      "7d",
-      "30d"
-    ]
+    "time_options": ["5m", "15m", "1h", "6h", "12h", "24h", "2d", "7d", "30d"]
   },
   "timezone": "",
-  "title": "Sidekiq",
+  "title": "Sidekiq Dashboard",
   "uid": "SwCwV7qkz",
   "version": 2
 }