speedshop-cloudwatch 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: cb7393054526aee599b81c7f6031475244449ec5e2f6137f509b0fde8811c496
+  data.tar.gz: 45eababb5f976f5fe8eeacf13d43a13e1edc3a06b3a57e5f7a127ffa0385324e
+SHA512:
+  metadata.gz: f09b6d50481cb2ea0bb2d25a01fe93955321c1f528145a6e864b959d81724e87067e0851d8dd13d78d3864bbf60d6342bba737696350d2f10be50f6b5e06f1ec
+  data.tar.gz: 9c719ebf214f5112f95c15b31327ef71dff873b5eeb71ac2c5fbdf2ad1dffcba908abb74867f1429406095b8c7b3712d999650a8ca6d38af6be405ac68cd426e

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Nate Berkopec
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,278 @@
+# Speedshop::Cloudwatch
+
+<p align="center">
+  <img src="https://github.com/user-attachments/assets/146ce110-311a-4acb-8000-66cd098f8b45" alt="Sit around and watch the clouds all day...">
+</p>
+
+This gem helps integrate your Ruby application with AWS CloudWatch for the purposes of auto-scaling. There are integrations for **Puma**, **Rack**, **Sidekiq** and **ActiveJob**.
+
+This gem is for **infrastructure and queue metrics**, not application performance metrics, like response times, job execution times, or error rates. Use your APM for that stuff.
+
+CloudWatch is unusually difficult to integrate with properly in Ruby, because the AWS library makes a synchronous HTTP request to AWS every time you record a metric. This is unlike the statsd or UDP-based models used by Datadog or other providers, which return more-or-less-instantaneously and are a lot less dangerous to use. Naively implementing this stuff yourself, you could end up adding 20-50ms of delay to your jobs or responses!
+
+This library helps you avoid that latency by reporting to CloudWatch in a background thread.
+
+This library supports **Ruby 2.7+, Sidekiq 7+, and Puma 6+**.
+
+## Metrics
+
+For a full explanation of every metric, [read about them in the code.](./lib/speedshop/cloudwatch/metrics.rb)
+
+By default, only essential queue metrics are enabled:
+
+```ruby
+config.metrics[:puma] = [] # Disabled by default
+config.metrics[:sidekiq] = [:QueueLatency] # Per queue
+config.metrics[:rack] = [:RequestQueueTime]
+config.metrics[:active_job] = [:QueueLatency] # Per queue
+```
+
+To enable additional metrics, configure them explicitly:
+
+```ruby
+# Enable all Puma metrics. These are based on reading Puma.stats.
+config.metrics[:puma] = [
+  :Workers, :BootedWorkers, :OldWorkers, :Running, :Backlog, :PoolCapacity, :MaxThreads
+]
+
+# Enable additional Sidekiq metrics
+config.metrics[:sidekiq] = [
+  :EnqueuedJobs, :ProcessedJobs, :FailedJobs, :ScheduledJobs, :RetryJobs,
+  :DeadJobs, :Workers, :Processes, :DefaultQueueLatency, :Capacity,
+  :Utilization, :QueueLatency, :QueueSize
+]
+```
+
+## Installation
+
+```ruby
+gem 'speedshop-cloudwatch'
+```
+
+See each integration below for instructions on how to setup and configure that integration.
+
+## Configuration
+
+You'll need to [configure your CloudWatch API credentials](https://github.com/aws/aws-sdk-ruby?tab=readme-ov-file#configuration), which is usually done via ENV var. If you're using one of their supported auto-config methods, you're good to go. If you're not, you'll need to provide your own `Aws::Cloudwatch::Client` object to the config (see below).
+
+```ruby
+Speedshop::Cloudwatch.configure do |config|
+  config.client = Aws::CloudWatch::Client.new
+  config.interval = 60
+
+  # Optional: Custom logger (defaults to Rails.logger if available, otherwise STDOUT)
+  config.logger = Logger.new(Rails.root.join("log", "cloudwatch.log"))
+
+  # Customize which metrics to report (whitelist)
+  # Puma metrics are disabled by default, enable them explicitly:
+  config.metrics[:puma] = [:Workers, :BootedWorkers, :Running, :Backlog]
+  # Sidekiq defaults to [:QueueLatency], add more as needed:
+  config.metrics[:sidekiq] = [:EnqueuedJobs, :QueueLatency, :QueueSize]
+
+  # Customize which Sidekiq queues to monitor (all queues by default)
+  config.sidekiq_queues = ["critical", "default", "low_priority"]
+
+  # Customize CloudWatch namespaces
+  config.namespaces[:puma] = "MyApp/Puma"
+  config.namespaces[:sidekiq] = "MyApp/Sidekiq"
+  config.namespaces[:rack] = "MyApp/Rack"
+  config.namespaces[:active_job] = "MyApp/ActiveJob"
+
+  # Optional: Add custom dimensions to all metrics
+  config.dimensions[:Env] = ENV["RAILS_ENV"] || "development"
+end
+```
+
+> [!WARNING]
+> Setting `config.interval` to less than 60 seconds automatically enables [high-resolution storage](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics) (1-second granularity) in CloudWatch, which incurs additional costs.
+
+### Environment Control
+
+**By default, the reporter only runs in production.** The environment is detected from `RAILS_ENV`, `RACK_ENV`, and defaults to `"development"`.
+
+```ruby
+Speedshop::Cloudwatch.configure do |config|
+  config.enabled_environments = ["production", "staging"]
+  config.environment = "staging" # optional override
+end
+```
+
+## Puma
+
+Puma metrics are disabled by default. You must explicitly enable them in your configuration.
+
+Add to your `config/puma.rb`:
+
+```ruby
+require_relative "../config/environment"
+
+Speedshop::Cloudwatch.configure do |config|
+  config.collectors << :puma
+  # Enable Puma metrics (disabled by default)
+  config.metrics[:puma] = [
+    :Workers, :BootedWorkers, :OldWorkers, :Running, :Backlog, :PoolCapacity, :MaxThreads
+  ]
+end
+
+# Start the reporter so Puma metrics are collected
+Speedshop::Cloudwatch.start!
+```
+
+Collection runs in the master process and reports per-worker metrics (see below). This works correctly with both `preload_app true` and `false`, as well as single and cluster modes.
+
+This reports the following metrics:
+
+```
+Workers - Number of workers configured (Count)
+BootedWorkers - Number of workers currently booted (Count)
+OldWorkers - Number of workers that are old/being phased out (Count)
+Running - Number of threads currently running (Count) [per worker]
+Backlog - Number of requests in the backlog (Count) [per worker]
+PoolCapacity - Current thread pool capacity (Count) [per worker]
+MaxThreads - Maximum number of threads configured (Count) [per worker]
+```
+
+## Rack
+
+If you're using Rails, we'll automatically insert the correct middleware into the stack.
+
+If you're using some other Rack-based framework, insert the `Speedshop::Cloudwatch::Rack` high up (i.e. first) in the stack.
+
+You will need a reverse proxy, such as nginx, adding an `X-Request-Start` or `X-Queue-Start` header (containing the time since the Unix epoch in milliseconds) to incoming requests. See [New Relic's instructions](https://docs.newrelic.com/docs/apm/applications-menu/features/configure-request-queue-reporting/) for more about how to do this.
+
+We report the following metrics:
+
+```
+RequestQueueTime - Time spent waiting in the request queue (Milliseconds)
+```
+
+### Sidekiq Integration
+
+In Sidekiq server processes, this integration auto-registers lifecycle hooks. On startup, it adds the `:sidekiq` collector and starts the reporter (leader-only when using Sidekiq Enterprise).
+
+If you're using Sidekiq as your ActiveJob adapter, prefer this integration instead of the ActiveJob integration.
+
+By default, only `QueueLatency` is reported. To enable additional metrics, configure them explicitly:
+
+```ruby
+Speedshop::Cloudwatch.configure do |config|
+  config.metrics[:sidekiq] = [
+    :EnqueuedJobs, :ProcessedJobs, :FailedJobs, :ScheduledJobs, :RetryJobs,
+    :DeadJobs, :Workers, :Processes, :DefaultQueueLatency, :Capacity,
+    :Utilization, :QueueLatency, :QueueSize
+  ]
+end
+```
+
+We report the following metrics:
+
+```
+EnqueuedJobs - Number of jobs currently enqueued (Count)
+ProcessedJobs - Total number of jobs processed (Count)
+FailedJobs - Total number of failed jobs (Count)
+ScheduledJobs - Number of scheduled jobs (Count)
+RetryJobs - Number of jobs in retry queue (Count)
+DeadJobs - Number of dead jobs (Count)
+Workers - Number of Sidekiq workers (Count)
+Processes - Number of Sidekiq processes (Count)
+DefaultQueueLatency - Latency for the default queue (Seconds)
+Capacity - Total concurrency across all processes (Count)
+Utilization - Average utilization across all processes (Percent)
+QueueLatency - Latency for each queue (Seconds) [per queue]
+QueueSize - Size of each queue (Count) [per queue]
+```
+
+Metrics marked [per queue] include a QueueName dimension.
+Utilization metrics include Tag and/or Hostname dimensions.
+
+## ActiveJob
+
+> [!WARNING]
+> If you're using Sidekiq, just use that integration and do not include the ActiveJob module.
+
+In your ApplicationJob:
+
+```ruby
+include Speedshop::Cloudwatch::ActiveJob
+```
+
+We report the following metrics:
+
+```
+QueueLatency - Time job spent waiting in queue before execution (Seconds)
+```
+
+This metric includes QueueName dimension and is aggregated per interval using CloudWatch StatisticSets.
+
+## Rails
+
+When running in a Rails app we:
+
+1. Automatically insert the Rack middleware at index 0.
+2. Respect your configuration for enabled metrics and collectors. The reporter starts automatically the first time a metric is reported (e.g., via Rack middleware) or when you call `Speedshop::Cloudwatch.start!` yourself (e.g., in Puma or initializers).
+
+If you want full control over these behaviors, add `require: false` to your Gemfile:
+
+```ruby
+gem 'speedshop-cloudwatch', require: false
+```
+
+Then manually require the core module without the railtie:
+
+```ruby
+# config/initializers/speedshop-cloudwatch.rb
+require 'speedshop/cloudwatch'
+
+# Insert middleware manually (if using Rack integration)
+Rails.application.config.middleware.insert_before 0, Speedshop::Cloudwatch::Rack
+
+Rails.application.configure do
+  config.after_initialize do
+    Speedshop::Cloudwatch.start!
+  end
+end
+```
+
+## Non-Rails Apps
+
+For Rack apps (Sinatra, etc.):
+
+- Insert `Speedshop::Cloudwatch::Rack` at the top of your middleware stack.
+- Configure collectors and start the reporter during app boot.
+
+Example config:
+
+```ruby
+require 'speedshop/cloudwatch'
+
+Speedshop::Cloudwatch.configure do |config|
+  # ...
+end
+
+Speedshop::Cloudwatch.start!
+```
+
+## Disabling Automatic Integration
+
+You can disable the auto-integration of Sidekiq and Puma by not requiring them:
+
+```ruby
+gem 'speedshop-cloudwatch', require: false
+```
+
+```ruby
+# some_initializer.rb
+require 'speedshop/cloudwatch'
+require 'speedshop/cloudwatch/puma'
+require 'speedshop/cloudwatch/active_job'
+require 'speedshop/cloudwatch/rack'
+# require 'speedshop/cloudwatch/sidekiq'
+```
+
+## Bibliography
+
+This library was developed with reference to and inspiration from these excellent projects:
+
+- [sidekiq-cloudwatchmetrics](https://github.com/sj26/sidekiq-cloudwatchmetrics) - Sidekiq CloudWatch metrics integration (portions adapted, see lib/speedshop/cloudwatch/sidekiq.rb)
+- [puma-cloudwatch](https://github.com/boltops-tools/puma-cloudwatch) - Puma CloudWatch metrics reporter
+- [judoscale-ruby](https://github.com/judoscale/judoscale-ruby) - Autoscaling metrics collection patterns

data/Rakefile

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+require "standard/rake"
+
+FLOG_THRESHOLD = (ENV["FLOG_THRESHOLD"] || 50).to_i
+FLAY_THRESHOLD = (ENV["FLAY_THRESHOLD"] || 100).to_i
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+desc "Run flog"
+task :flog do
+  flog_output = `bundle exec flog -a lib test`
+  puts flog_output
+  method_scores = flog_output.lines.grep(/^\s+[0-9]+\.[0-9]+:.*#/).reject { |line| line.include?("main#none") }.map { |line| line.split.first.to_f }
+  max_score = method_scores.max
+  if max_score && max_score > FLOG_THRESHOLD
+    abort "flog failed: highest complexity (#{max_score}) exceeds threshold (#{FLOG_THRESHOLD})"
+  end
+  puts "flog passed (max complexity: #{max_score}, threshold: #{FLOG_THRESHOLD})"
+end
+
+desc "Run flay"
+task :flay do
+  flay_output = `bundle exec flay lib test`
+  puts flay_output
+  flay_score = flay_output[/Total score.*?=\s*(\d+)/, 1]&.to_i
+  if flay_score && flay_score > FLAY_THRESHOLD
+    abort "flay failed: duplication score (#{flay_score}) exceeds threshold (#{FLAY_THRESHOLD})"
+  end
+  puts "flay passed (duplication score: #{flay_score}, threshold: #{FLAY_THRESHOLD})"
+end
+
+task default: [:test, :standard, :flog, :flay]

data/bin/console

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "speedshop/cloudwatch/all"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Start Redis in Docker if port 6379 is free and docker is available
+if ! nc -z localhost 6379 2>/dev/null; then
+  if command -v docker &>/dev/null; then
+    docker run -d --name speedshop-cloudwatch-redis -p 6379:6379 redis 2>/dev/null || true
+  fi
+fi
+
+# Do any other automated setup that you need to do here

data/lib/speedshop/cloudwatch/active_job.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Speedshop
+  module Cloudwatch
+    module ActiveJob
+      def self.included(base)
+        base.around_perform :report_job_metrics
+      end
+
+      def report_job_metrics
+        begin
+          if enqueued_at
+            queue_time = Time.now.to_f - enqueued_at.to_f
+            # Drop JobClass to reduce time series cardinality and allow aggregation into StatisticSets per queue
+            Reporter.instance.report(metric: :QueueLatency, value: queue_time, dimensions: {QueueName: queue_name}, integration: :active_job)
+          end
+        rescue => e
+          Speedshop::Cloudwatch.log_error("Failed to collect ActiveJob metrics: #{e.message}", e)
+        end
+        yield
+      end
+    end
+  end
+end

data/lib/speedshop/cloudwatch/all.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "speedshop/cloudwatch"
+require "speedshop/cloudwatch/active_job"
+require "speedshop/cloudwatch/puma"
+require "speedshop/cloudwatch/rack"
+require "speedshop/cloudwatch/sidekiq"
+require "speedshop/cloudwatch/railtie" if defined?(Rails)

data/lib/speedshop/cloudwatch/config.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require "logger"
+require "singleton"
+
+module Speedshop
+  module Cloudwatch
+    class Config
+      include Singleton
+
+      attr_accessor :interval, :metrics, :namespaces, :logger, :queue_max_size, :sidekiq_queues, :dimensions,
+        :collectors, :enabled_environments, :environment
+      attr_writer :client
+
+      def initialize
+        reset
+      end
+
+      def client
+        @client ||= Aws::CloudWatch::Client.new
+      end
+
+      def reset
+        @interval = 60
+        @queue_max_size = 1000
+        @client = nil
+        @metrics = {
+          puma: [],
+          sidekiq: [:QueueLatency],
+          rack: [:RequestQueueTime],
+          active_job: [:QueueLatency]
+        }
+        @namespaces = {puma: "Puma", sidekiq: "Sidekiq", rack: "Rack", active_job: "ActiveJob"}
+        @sidekiq_queues = nil
+        @dimensions = {}
+        @logger = (defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger) ? Rails.logger : Logger.new($stdout)
+        @collectors = [] # [:puma, :sidekiq]
+        @enabled_environments = ["production"]
+        @environment = detect_environment
+      end
+
+      def environment_enabled?
+        enabled_environments.include?(environment)
+      end
+
+      def self.reset
+        if instance_variable_defined?(:@singleton__instance__)
+          config = instance_variable_get(:@singleton__instance__)
+          config&.reset
+        end
+        instance_variable_set(:@singleton__instance__, nil)
+      end
+
+      private
+
+      def detect_environment
+        ENV.fetch("RAILS_ENV", ENV.fetch("RACK_ENV", "development"))
+      end
+    end
+  end
+end

data/lib/speedshop/cloudwatch/metrics.rb

@@ -0,0 +1,181 @@
+# frozen_string_literal: true
+
+module Speedshop
+  module Cloudwatch
+    Metric = Struct.new(:name, :unit, :description, :source, keyword_init: true)
+
+    METRICS = {
+      puma: [
+        # Cluster-only metrics that report on the overall Puma process
+        Metric.new(
+          name: :Workers,
+          unit: "Count",
+          description: "Total number of workers (child processes) configured in " \
+                       "Puma. This is a static configuration value.",
+          source: "Puma.stats_hash[:workers]"
+        ),
+        Metric.new(
+          name: :BootedWorkers,
+          unit: "Count",
+          description: "Number of worker processes currently running and ready to " \
+                       "handle requests. Should match Workers count under normal " \
+                       "operation.",
+          source: "Puma.stats_hash[:booted_workers]"
+        ),
+        Metric.new(
+          name: :OldWorkers,
+          unit: "Count",
+          description: "Number of worker processes being phased out during a " \
+                       "restart. Zero during normal operation.",
+          source: "Puma.stats_hash[:old_workers]"
+        ),
+
+        # Per-worker metrics (also available in single mode with WorkerIndex=0)
+        Metric.new(
+          name: :Running,
+          unit: "Count",
+          description: "Number of threads currently processing requests in this " \
+                       "worker. Compare against MaxThreads to understand " \
+                       "utilization.",
+          source: "worker_stats[:running]"
+        ),
+        Metric.new(
+          name: :Backlog,
+          unit: "Count",
+          description: "Number of requests waiting to be processed by this worker. " \
+                       "Sustained backlog suggests over capacity.",
+          source: "worker_stats[:backlog]"
+        ),
+        Metric.new(
+          name: :PoolCapacity,
+          unit: "Count",
+          description: "Number of threads available to handle new requests. When " \
+                       "this reaches zero, requests queue in the backlog.",
+          source: "worker_stats[:pool_capacity]"
+        ),
+        Metric.new(
+          name: :MaxThreads,
+          unit: "Count",
+          description: "Maximum number of threads configured for this worker. This " \
+                       "is a static configuration value.",
+          source: "worker_stats[:max_threads]"
+        )
+      ],
+
+      sidekiq: [
+        # Overall Sidekiq statistics
+        Metric.new(
+          name: :EnqueuedJobs,
+          unit: "Count",
+          description: "Total number of jobs currently enqueued across all queues, waiting to be processed.",
+          source: "Sidekiq::Stats.new.enqueued"
+        ),
+        Metric.new(
+          name: :ProcessedJobs,
+          unit: "Count",
+          description: "Cumulative count of all jobs successfully processed since Sidekiq started.",
+          source: "Sidekiq::Stats.new.processed"
+        ),
+        Metric.new(
+          name: :FailedJobs,
+          unit: "Count",
+          description: "Cumulative count of all jobs that have failed since Sidekiq started.",
+          source: "Sidekiq::Stats.new.failed"
+        ),
+        Metric.new(
+          name: :ScheduledJobs,
+          unit: "Count",
+          description: "Number of jobs scheduled to run at a future time. Will move " \
+                       "to enqueued when their scheduled time arrives.",
+          source: "Sidekiq::Stats.new.scheduled_size"
+        ),
+        Metric.new(
+          name: :RetryJobs,
+          unit: "Count",
+          description: "Number of jobs in the retry queue. These jobs failed but " \
+                       "have not exhausted their retry attempts.",
+          source: "Sidekiq::Stats.new.retry_size"
+        ),
+        Metric.new(
+          name: :DeadJobs,
+          unit: "Count",
+          description: "Number of jobs in the dead queue. These jobs exhausted all retry attempts.",
+          source: "Sidekiq::Stats.new.dead_size"
+        ),
+        Metric.new(
+          name: :Workers,
+          unit: "Count",
+          description: "Total number of worker threads currently processing jobs across all Sidekiq processes.",
+          source: "Sidekiq::Stats.new.workers_size"
+        ),
+        Metric.new(
+          name: :Processes,
+          unit: "Count",
+          description: "Number of Sidekiq server processes currently running.",
+          source: "Sidekiq::Stats.new.processes_size"
+        ),
+        Metric.new(
+          name: :DefaultQueueLatency,
+          unit: "Seconds",
+          description: "Time the oldest job in the default queue has been waiting. Zero if the queue is empty.",
+          source: "Sidekiq::Stats.new.default_queue_latency"
+        ),
+
+        # Process-level metrics
+        Metric.new(
+          name: :Capacity,
+          unit: "Count",
+          description: "Total number of worker threads available across all Sidekiq " \
+                       "processes. Can be tagged by process tag.",
+          source: "Sum of process['concurrency'] across processes"
+        ),
+        Metric.new(
+          name: :Utilization,
+          unit: "Percent",
+          description: "Average percentage of worker threads currently busy. 100% " \
+                       "means all workers are busy. Can be reported by tag or " \
+                       "hostname.",
+          source: "Average of busy / concurrency * 100 across processes"
+        ),
+
+        # Queue-specific metrics
+        Metric.new(
+          name: :QueueLatency,
+          unit: "Seconds",
+          description: "Time the oldest job in this queue has been waiting. High " \
+                       "latency indicates jobs are backing up.",
+          source: "Sidekiq::Queue#latency"
+        ),
+        Metric.new(
+          name: :QueueSize,
+          unit: "Count",
+          description: "Number of jobs currently waiting in this queue. Growing " \
+                       "size indicates jobs arriving faster than processing.",
+          source: "Sidekiq::Queue#size"
+        )
+      ],
+
+      rack: [
+        Metric.new(
+          name: :RequestQueueTime,
+          unit: "Milliseconds",
+          description: "Time a request spent waiting in the reverse proxy before " \
+                       "reaching the application. High values indicate requests " \
+                       "backing up before reaching your application server.",
+          source: "(Time.now.to_f * 1000) - HTTP_X_REQUEST_START"
+        )
+      ],
+
+      active_job: [
+        Metric.new(
+          name: :QueueLatency,
+          unit: "Seconds",
+          description: "Time a job spent waiting in the queue before execution " \
+                       "started. Values are aggregated into CloudWatch " \
+                       "StatisticSets per reporting interval.",
+          source: "Time.now.to_f - job.enqueued_at"
+        )
+      ]
+    }.freeze
+  end
+end