promenade 0.12.17 → 0.12.19

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ee4df86abaae8acf1703fec0a3cffd7bc3a5ac74195d40b30a86b3fca1a9aa74
-  data.tar.gz: 6dc0b47820758b7a6288fc08f46c65a8b2bfa22a228eeb46c441de244f6734ae
+  metadata.gz: 5a7547269d759c687b4c98f3e71ee3e273c41fa9c4501c7bc8af3dee4a84467b
+  data.tar.gz: c0c4abe54da62a38090e00d542fafbd0b27453c14561954a83e113b7eb74e89b
 SHA512:
-  metadata.gz: 4469b6beed99690e9d739c3a3c86603d3c1033dbbabbe8db0061302ef281ea9f13c195ccc5ab595025b78dfadb884cc17c5508c0093df10d7e1a0abe90ce72b8
-  data.tar.gz: b67feeb7a2ad2b9e57a36363b695b8ec4693d712c53ac37e42614f2f5471a77e0cb97f9e7d3622204e1b67cfe2976c54f5e9e721021bf1c1399ab3a83829a456
+  metadata.gz: 51e565bc41e56d8f1ffb88d1243668584bb96e624239cf0e798e2444e6461f48f4982646b1cf12d706cd31645fddf85f7d394ea19141a7b0c43564eefc701499
+  data.tar.gz: c08c64ab3f836b6f7ff41513f86b8fadea0651cbcb0779e6df485f6fa9647106c25bb3c94971ef6e23945e7cd5664c839b523997f19fcefba8c68c54e28de42f

data/README.md

@@ -6,25 +6,22 @@
 
 Promenade is a library to simplify instrumenting Ruby applications with Prometheus.
 
-It is currently under development.
-
 ## Usage
 
-Add promenade to your Gemfle:
+Add promenade to your Gemfile:
 
-```
+```ruby
 gem "promenade"
 ```
 
 ### Built in instrumentation
 
-Promenade includes some built in instrumentation that can be used by requiring it (for example in an initializer).
+Promenade includes built-in instrumentation for several libraries. Require the relevant file in an initializer to enable it.
 
-Currently there is support for [ruby-kafka](https://github.com/zendesk/ruby-kafka), but I plan to support other things soon.
-
-```
-# Instrument the ruby-kafka libary
-require "promenade/kafka"
+```ruby
+require "promenade/kafka"     # ruby-kafka
+require "promenade/karafka"   # Karafka consumers
+require "promenade/waterdrop"  # WaterDrop producers
 ```
 
 ### Instrumentation DSL
@@ -52,7 +49,7 @@ class WidgetService
   end
 
   def batch_create
-    You can increment by more than 1 at a time if you need
+    # You can increment by more than 1 at a time if you need
     Promenade.metric(:widgets_created).increment({ type: "guinness" }, 100)
   end
 end
@@ -86,7 +83,7 @@ of all observed values.
 class Calculator
   Promenade.histogram :calculator_time_taken do
     doc "Records how long it takes to do the adding"
-    # promenade also has some bucket presets like :network and :memory for common usecases
+    # promenade also has some bucket presets like :network and :memory for common use cases
     buckets [0.25, 0.5, 1, 2, 4]
   end
 
@@ -122,13 +119,32 @@ end
 
 ### Exporter
 
-Because promenade is based on prometheus-client you can add the `Prometheus::Client::Rack::Exporter` middleware to your rack middleware stack to expose metrics.
-
-There is also a stand alone exporter that can be run with the `promenade` command.
-
-This is ideal if you are worried about accidentally exposing your metrics, are concerned about the performance impact prometheus scrapes might have on your application, or for applications without a web server (like background processing jobs). It does mean that you have another process to manage on your server though 🤷.
+The recommended way to expose metrics is the **Go exporter sidecar** in [`exporter/`](exporter/). It runs as a separate container that reads the `.db` files written by the Ruby application and exposes them at `:9394/metrics`. This keeps scrape overhead entirely off the Ruby application process and also collects TCP connection metrics (busy/queued workers) via Linux netlink without any native extension.
+
+The exporter sidecar shares a network namespace and tmpfs volume with your app container. See [`compose.yml`](compose.yml) for a reference deployment:
+
+```yaml
+services:
+  app:
+    # your Ruby app; writes metrics to tmp/promenade on the shared tmpfs
+    volumes:
+      - tmp:/app/tmp
+    environment:
+      PROMETHEUS_MULTIPROC_DIR: /app/tmp/promenade
+  exporter:
+    image: ghcr.io/errm/promenade:latest
+    network_mode: service:app   # shares the app's network namespace
+    volumes:
+      - tmp:/app/tmp
+volumes:
+  tmp:
+    driver: local
+    driver_opts:
+      type: tmpfs
+      device: tmpfs
+```
 
-The exporter runs by default on port `9394` and the metrics are available at the standard path of `/metrics`, the stand-alone exporter is configured to use gzip.
+The exporter is configured with `--multiprocess-dir` (or `PROMETHEUS_MULTIPROC_DIR`) and `--metrics-port` (default `9394`).
 
 
 ### Rails Middleware
@@ -137,15 +153,18 @@ Promenade provides custom Rack middleware to track HTTP response times for reque
 
 This was originally inspired by [prometheus-client-mmap](https://gitlab.com/gitlab-org/prometheus-client-mmap/-/blob/master/lib/prometheus/client/rack/collector.rb).
 
-**This middleware is automatically added to your Rack stack if your application is a Ruby on Rails app.**
+**The following middleware is automatically added to your Rack stack if your application is a Ruby on Rails app:**
 
-We recommend you add the middleware after `ActionDispatch::ShowExceptions` in your stack, so you can accurately record the controller and action where an exception was raised.
+- `Promenade::Client::Rack::HTTPRequestQueueTimeCollector` — inserted at the front of the stack, records time spent in the request queue (via `X-Request-Start` / `X-Queue-Start` headers).
+- `Promenade::Client::Rack::HTTPRequestDurationCollector` — inserted after `ActionDispatch::ShowExceptions`, records response duration and HTTP exception counts.
+- `Promenade::YJIT::Middleware` — appended, records YJIT stats (enabled only when `RubyVM::YJIT` is defined).
+- `Promenade::Pitchfork::Middleware` — appended, records worker and memory metrics (enabled only when Pitchfork is present).
 
-If you want to change the position, or customise the labels and exception handling behaviour, simply remove the middleware from the stack and re-insert it with your own preferences.
+If you want to change the position of `HTTPRequestDurationCollector`, or customise its labels and exception handling behaviour, simply remove it from the stack and re-insert it with your own preferences.
 
 ``` ruby
-Rails.application.middleware.delete(Promenade::Client::Rack::Collector)
-Rails.application.middleware.insert_after(Rails::Rack::Logger, Promenade::Client::Rack::Collector)
+Rails.application.middleware.delete(Promenade::Client::Rack::HTTPRequestDurationCollector)
+Rails.application.middleware.insert_after(Rails::Rack::Logger, Promenade::Client::Rack::HTTPRequestDurationCollector)
 ```
 
 #### Customising the labels recorded for each request
@@ -162,29 +181,29 @@ label_builder = Proc.new do |env|
   }
 end
 Rails.application.config.middleware.insert_after ActionDispatch::ShowExceptions,
-        Promenade::Client::Rack::Collector
+        Promenade::Client::Rack::HTTPRequestDurationCollector,
         label_builder: label_builder
 ```
 
 #### Customising how the middleware handles exceptions
 
-The default implementation will capture exceptions, count the execption class name (e.g. `"StandardError"`), and then re-raise the exception.
+The default implementation will capture exceptions, count the exception class name (e.g. `"StandardError"`), and then re-raise the exception.
 
 If you would like to customise this behaviour, you may do so by customising the middleware installation:
 
 ``` ruby
-exception_handler = Proc.new do |exception, exception_counter, env_hash, request_duration_seconds|
-  # This simple example just re-raises the execption
+exception_handler = Proc.new do |exception, env_hash, duration|
+  # This simple example just re-raises the exception
   raise exception
 end
 Rails.application.config.middleware.insert_after ActionDispatch::ShowExceptions,
-        Promenade::Client::Rack::Collector
+        Promenade::Client::Rack::HTTPRequestDurationCollector,
         exception_handler: exception_handler
 ```
 
 #### Customising the histogram buckets
 
-The default buckets cover a range of latencies from 5 ms to 10s see [Promenade::Configuration::DEFAULT_RACK_LATENCY_BUCKETS](https://github.com/errm/promenade/blob/ea7eb54c04257770a601b7e28b3e13db5d2430bb/lib/promenade/configuration.rb#L5). This is intended to capture the typical range of latencies for a web application. However, this might not be suitable for your Service-Level Agreements (SLAs), and other bucket size intervals may be required (see [histogram bins](https://en.wikipedia.org/wiki/Histogram#Number_of_bins_and_width)).
+The default buckets cover a range of latencies from 5 ms to 10s see [Promenade::Configuration::DEFAULT_RACK_LATENCY_BUCKETS](https://github.com/errm/promenade/blob/master/lib/promenade/configuration.rb#L5) and [Promenade::Configuration::DEFAULT_QUEUE_TIME_BUCKETS](https://github.com/errm/promenade/blob/master/lib/promenade/configuration.rb#L7). This is intended to capture the typical range of latencies for a web application. However, this might not be suitable for your Service-Level Agreements (SLAs), and other bucket size intervals may be required (see [histogram bins](https://en.wikipedia.org/wiki/Histogram#Number_of_bins_and_width)).
 
 If you would like to customise the histogram buckets, you can do so by configuring Promenade in an initializer:
 
@@ -192,21 +211,21 @@ If you would like to customise the histogram buckets, you can do so by configuri
 # config/initializers/promenade.rb
 
 Promenade.configure do |config|
-  config.rack_latency_buckets = [0.25, 0.350, 0.5, 1, 1.5, 2.5, 5, 10, 15, 19]
+  config.rack_latency_buckets = [0.1, 0.25, 0.5, 1, 2.5, 5, 10]
+  config.queue_time_buckets = [0.01, 0.5, 1.0, 10.0, 30.0]  # optional, for queue time collector
 end
 ```
 
 ### Configuration
 
-If you are using rails it should load a railtie and configure promenade.
+If you are using Rails it should load a railtie and configure promenade.
 
-If are not using rails you should call `Promenade.setup` after your environment has loaded.
+If you are not using Rails you should call `Promenade.setup` after your environment has loaded.
 
 In a typical development environment there should be nothing for you to do. Promenade stores its state files in `tmp/promenade` and will create that directory if it does not exist.
 
 In a production environment you should try to store the state files on tmpfs for performance, you can configure the path that promenade will write to by setting the `PROMETHEUS_MULTIPROC_DIR` environment variable.
 
-If you are running the stand-alone exporter, you may also set the `PORT` environment variable to bind to a port other than the default (`9394`).
 
 ## Development
 
@@ -218,8 +237,6 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/errm/promenade.
 
-This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
-
 ## Acknowledgements
 
 The original code for the Rack middleware collector class was copied from [Prometheus Client MMap](https://gitlab.com/gitlab-org/prometheus-client-mmap/-/blob/master/lib/prometheus/client/rack/collector.rb).
@@ -227,7 +244,3 @@ The original code for the Rack middleware collector class was copied from [Prome
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-
-## Code of Conduct
-
-Everyone interacting in the Promenade project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/promenade/blob/master/CODE_OF_CONDUCT.md).

data/lib/promenade/client/rack/http_request_queue_time_collector.rb

@@ -14,35 +14,24 @@ module Promenade
         def initialize(app,
                        registry: ::Prometheus::Client.registry,
                        label_builder: RequestLabeler)
-          @queue_time_buckets = Promenade.configuration.queue_time_buckets
-
           super
         end
 
         private
 
-          attr_reader :queue_time_buckets
-
           def trace(env)
-            start_timestamp = Time.now.utc
-            response = yield
-            record_request_queue_time(labels: labels(env, response),
-              env: env,
-              request_received_time: start_timestamp)
-            response
+            record_request_queue_time(env:)
+            yield
           end
 
-          def record_request_queue_time(labels:, env:, request_received_time:)
-            request_queue_duration = QueueTimeDuration.new(env:,
-              request_received_time:)
-            return unless request_queue_duration.valid_header_present?
-
-            queue_time_histogram.observe(labels, request_queue_duration.queue_time_seconds)
+          def record_request_queue_time(env:)
+            queue_time_seconds = QueueTimeDuration.new(env:).queue_time_seconds
+            queue_time_seconds && queue_time_histogram.observe(label_builder.call(env), queue_time_seconds)
           end
 
           def register_metrics!
             registry.histogram(REQUEST_QUEUE_TIME_HISTOGRAM_NAME,
-              "A histogram of request queue time", {}, queue_time_buckets)
+              "A histogram of request queue time", {}, Promenade.configuration.queue_time_buckets)
           end
 
           def queue_time_histogram

data/lib/promenade/client/rack/queue_time_duration.rb

@@ -3,40 +3,36 @@ module Promenade
     module Rack
       class QueueTimeDuration
         REQUEST_START_HEADER = "HTTP_X_REQUEST_START".freeze
-
         QUEUE_START_HEADER = "HTTP_X_QUEUE_START".freeze
 
         HEADER_VALUE_MATCHER = /^(?:t=)(?<timestamp>\d{10}(?:\.\d+))$/
 
-        def initialize(env:, request_received_time:)
-          @env = env
-          @request_queued_time_ms = extract_request_queued_time_from_env(env)
-          @valid_header_present = @request_queued_time_ms.is_a?(Float)
-          @request_received_time_ms = request_received_time.utc.to_f
-
+        def initialize(env:, request_received_time: Time.now.utc)
+          @request_enqueued_time = request_enqueued_time_from(env)
+          @request_received_time = request_received_time.utc.to_f
           freeze
         end
 
-        def valid_header_present?
-          @valid_header_present
-        end
-
         def queue_time_seconds
-          return unless valid_header_present?
+          # Enqueued time could not be parsed from headers
+          return unless request_enqueued_time
+
+          # A negative queue time is not valid
+          return if queue_time < 0
 
           queue_time.round(3)
         end
 
         private
 
-          attr_reader :env, :request_queued_time_ms, :request_received_time_ms
+          attr_reader :request_enqueued_time, :request_received_time
 
           def queue_time
-            request_received_time_ms - request_queued_time_ms
+            request_received_time - request_enqueued_time
           end
 
-          def extract_request_queued_time_from_env(env_hash)
-            header_value = env_hash[REQUEST_START_HEADER] || env_hash[QUEUE_START_HEADER]
+          def request_enqueued_time_from(env)
+            header_value = env.values_at(REQUEST_START_HEADER, QUEUE_START_HEADER).compact.first
             return if header_value.nil?
 
             header_time_match = header_value.to_s.match(HEADER_VALUE_MATCHER)

data/lib/promenade/client/rack/request_controller_action_labeler.rb

@@ -15,11 +15,14 @@ module Promenade
 
         SEPARATOR = "#".freeze
 
-        private_constant :PARAMS_KEY, :CONTROLLER, :ACTION, :UNKNOWN, :SEPARATOR
+        REQUEST_METHOD = "REQUEST_METHOD".freeze
+
+        private_constant :PARAMS_KEY, :CONTROLLER, :ACTION, :UNKNOWN, :SEPARATOR, :REQUEST_METHOD
 
         def call(env)
           super.merge({
             controller_action: controller_action_from_env(env),
+            method: env[REQUEST_METHOD].to_s.downcase,
           })
         end
 

data/lib/promenade/client/rack/request_labeler.rb

@@ -5,15 +5,12 @@ module Promenade
         require_relative "singleton_caller"
         extend SingletonCaller
 
-        REQUEST_METHOD = "REQUEST_METHOD".freeze
-
         HTTP_HOST = "HTTP_HOST".freeze
 
-        private_constant :REQUEST_METHOD, :HTTP_HOST
+        private_constant :HTTP_HOST
 
         def call(env)
           {
-            method: env[REQUEST_METHOD].to_s.downcase,
             host: env[HTTP_HOST].to_s,
           }
         end

data/lib/promenade/version.rb

@@ -1,3 +1,3 @@
 module Promenade
-  VERSION = "0.12.17".freeze
+  VERSION = "0.12.19".freeze
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: promenade
 version: !ruby/object:Gem::Version
-  version: 0.12.17
+  version: 0.12.19
 platform: ruby
 authors:
 - Ed Robinson
@@ -311,7 +311,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.3
+rubygems_version: 4.0.6
 specification_version: 4
 summary: Promenade makes it simple to instrument Ruby apps for prometheus scraping
 test_files: []