deployangel 0.1.28 → 0.1.30

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fde8310f8129827b4b759369146c2346542a776ab13ac91fe7001617e93a7ebb
-  data.tar.gz: 15a216435704ea45fce03e0a1433d615dc28f6c07fdd6e33c90bebed23361779
+  metadata.gz: cb73e91bc627176c422a77379e49c662588f866524d591d367f4e6e44368ee57
+  data.tar.gz: ffff6fa5fe0ce44ba2f17fb58e689eb541c5ad04549bb684ffa17d6995bc3bc4
 SHA512:
-  metadata.gz: e6e7ae24a04e3999af5865e2cf5be14054cdfc92b5644b1a4f5d8bd56c407faf874b122db23508bbcb3b268e461a89d68a16a3a235e166cf457e10e063a4c3f9
-  data.tar.gz: 3e2bb9e11c4d8b35134fdb9088beabcaf0ba5b99b484c9e4b0e2dfc3b710aa544507b55f6a3a222c0446d4b6edbec0f935f0049f03e2039cc19e253767560530
+  metadata.gz: e4d40abb95f27a7b2ee692016943edb4f5970edbb57aa85bd75ce66946438604863cc8ae8c8bb89d4d75f01aa8697a2cd9d496ca1393cae39b58dd4d9f9255a7
+  data.tar.gz: 4bccc9b61268107aa9948087d4b42caee50998e08060b47157f2551a0f37f8050676dac8634e5961df06e8ce5cfff8792a6287ed1b1be5d51f4e75ba169fdc92

data/CHANGELOG.md

@@ -1,5 +1,40 @@
 # Changelog
 
+## 0.1.30 (2026-10-16)
+
+- Hanami 2.2 or later: `require "deployangel/hanami"` and
+  `DeployAngel::Hanami.install(self)` in the app's class. Requests are
+  recorded by the route pattern they matched, scopes' and slices' prefixes
+  included, through a router resolver that builds on any the app configured;
+  exceptions Hanami renders as error pages count as unhandled, and the 404
+  Hanami raises for an unknown path counts as a 404. The agent starts when
+  the app boots. The metadata lists each route with its action class and the
+  paths of its action, view, and template files, and digests Hanami's layout
+  (`app/`, `slices/`, `config/`, `lib/`).
+- Sinatra 3.0 or later: `require "deployangel/sinatra"` and
+  `register DeployAngel::Sinatra`, in modular and classic apps. Requests are
+  recorded by the route Sinatra ran, with the app's mount point; exceptions
+  Sinatra renders as error pages count as unhandled. The metadata lists
+  each route with the file it's defined in.
+- The Rack base middleware asks an adapter what status a raised exception
+  will be rendered as (`raised_status`); only a 5xx counts as unhandled.
+  Rails keeps 500. It also records a request once when it passes through
+  more than one DeployAngel middleware (nested Sinatra apps, or a Sinatra
+  app mounted in Rails).
+
+## 0.1.29 (2026-10-16)
+
+- Reports stuck work, so DeployAngel can fail a release in which a job hangs
+  or the queue keeps growing while every job still succeeds. Each minute's
+  payload says how many jobs are still running and how long the oldest of
+  each class has been running (ActiveJob on any adapter, and native Sidekiq
+  jobs). Sidekiq and Solid Queue worker processes also read the queue backlog,
+  the jobs ready to run and waiting, once a minute from the reporter thread:
+  one pipelined Redis call for Sidekiq, one `COUNT` of
+  `solid_queue_ready_executions` for Solid Queue. Web processes don't read
+  it. Needs a DeployAngel cloud that understands the new fields; older ones
+  ignore them.
+
 ## 0.1.28 (2026-10-12)
 
 - Clearer setup for coding agents, from a first-day rehearsal: `setup` says

data/README.md

@@ -1,6 +1,6 @@
-# DeployAngel for Rails
+# DeployAngel for Rails, Hanami, and Sinatra
 
-The DeployAngel agent. It watches what your Rails app does in production and
+The DeployAngel agent. It watches what your Rails, Hanami, or Sinatra app does in production and
 reports one small aggregated payload per process per minute. DeployAngel uses
 that to verify every deployment, and to tell you (or your coding agent) when a
 release is **cleared** and you can stop watching it.
@@ -20,6 +20,59 @@ Set these in production:
 DEPLOYANGEL_TOKEN=da_live_...           # an ingestion token with the telemetry scope
 ```
 
+### Hanami
+
+Requires Hanami 2.2 or later. Hanami has no Railtie, so install it where the
+app is configured:
+
+```ruby
+# config/app.rb
+require "hanami"
+require "deployangel/hanami"
+
+module Bookshelf
+  class App < Hanami::App
+    DeployAngel::Hanami.install(self)
+  end
+end
+```
+
+It names each request by the route it matched (through a router resolver that
+builds on any you've configured), records requests and the exceptions Hanami
+renders as error pages, and starts the agent when the app boots (as
+`config.ru`'s `hanami/boot` does). A process that loads the app without booting
+it, such as `hanami console`, doesn't report. Native Sidekiq jobs are recorded
+as in Rails; there's nothing else to set up for them.
+
+### Sinatra
+
+Requires Sinatra 3.0 or later. Register the extension before the app's
+routes, so it knows the file each route is defined in:
+
+```ruby
+require "sinatra/base"
+require "deployangel/sinatra"
+
+class Bookshelf < Sinatra::Base
+  register DeployAngel::Sinatra
+
+  get "/books/:id" do
+    # ...
+  end
+end
+```
+
+A classic app registers it at the top level, after `require "sinatra"`. It
+records each request by the route Sinatra ran (after `pass`, the route that
+answered), with the app's mount point from `config.ru` (`map "/api"`), and the
+exceptions Sinatra renders as error pages. A request that passes through more
+than one app (one used as another's middleware) is recorded once. File paths
+are relative to the working directory, the app's root in most deploys. An
+exception raised directly in a route block is named by its file, since the
+block has no method name, so two raised in different routes of the same file
+share a fingerprint; ones raised in helpers or other classes are told apart as
+usual.
+
 The agent must know which release it is running. It finds it in this order:
 
 1. `DEPLOYANGEL_REVISION` (the commit SHA) and `DEPLOYANGEL_RELEASE_VERSION`, if you set them
@@ -83,11 +136,16 @@ the dashboard says so.
   and a latency histogram, both per app and per route.
 - Routes are recorded as the matched pattern (`GET /users/:id`), never the raw
   path. At most 100 routes are sent per payload; the rest are folded into
-  `__other__`.
+  `__other__`. In Hanami, a request no route matched is a 404, not an
+  exception, though Hanami raises one to render it.
 - Background jobs: attempts, failed attempts, discarded jobs, duration, and
   queue latency, per job class. Works with every ActiveJob adapter (Solid
   Queue, Sidekiq, GoodJob, ...) and with native Sidekiq jobs through a server
   middleware. Failures that `retry_on` or `discard_on` handle still count.
+- Jobs still running at the end of each minute, and how long the oldest of
+  each class has been running, so a job that hangs is noticed. Sidekiq and
+  Solid Queue worker processes also send the queue backlog (how many jobs are
+  waiting), read once a minute off the request path.
 - Release identity, runtime versions, and a per-process instance ID.
 
 - Exceptions: a stable fingerprint, the exception class, the first line of the
@@ -97,9 +155,11 @@ the dashboard says so.
   job switched to. Other words are kept: `Payment failed for Jane Doe` is sent
   as it is. If your app's messages might hold personal or health data, [turn
   messages off](#exception-messages).
-- Once per process: the route table, job classes, recurring schedules
-  declared for Solid Queue, GoodJob, sidekiq-cron, sidekiq-scheduler, or
-  whenever, critical
+- Once per process: the route table (in Hanami, with each route's action
+  class and the paths of its action, view, and template files; in Sinatra,
+  with the path of the file each route is defined in), job classes,
+  recurring schedules declared for Solid Queue, GoodJob, sidekiq-cron,
+  sidekiq-scheduler, or whenever, critical
   flows, and file digests (relative paths and hashes, never file contents) so
   DeployAngel can tell which routes changed in a release.
   Disable digests with `DEPLOYANGEL_FILE_DIGESTS=false`.

data/lib/deployangel/agent.rb

@@ -86,6 +86,25 @@ module DeployAngel
       warn_once(:record_job, "DeployAngel failed to record a job: #{e.class}: #{e.message}")
     end
 
+    # Marks a job as running until #job_finished, so one that hangs is still
+    # reported. Returns the token to pass to it.
+    def job_started(job_class)
+      return unless @active
+
+      after_fork! if Process.pid != @pid
+      start_reporter
+      @running_jobs.start(job_class)
+    rescue StandardError => e
+      warn_once(:job_started, "DeployAngel failed to record a running job: #{e.class}: #{e.message}")
+      nil
+    end
+
+    def job_finished(token)
+      @running_jobs.finish(token) if token
+    rescue StandardError => e
+      warn_once(:job_finished, "DeployAngel failed to record a finished job: #{e.class}: #{e.message}")
+    end
+
     def record_discard(job_class:)
       return unless @active
 
@@ -135,7 +154,9 @@ module DeployAngel
     def flush(include_current: false)
       return 0 unless @active
 
-      @aggregator.drain(include_current: include_current, max_periods: config.max_queued_payloads).each do |period|
+      periods = @aggregator.drain(include_current: include_current, max_periods: config.max_queued_payloads)
+      add_stuck_work(periods.last, include_current) if periods.any?
+      periods.each do |period|
         @buffer.push(Transport.encode(Protocol.telemetry(period, instance: @instance, release: release,
           runtime: @runtime, capabilities: capabilities)))
       end
@@ -198,10 +219,24 @@ module DeployAngel
     end
 
     private
+      # What's still running, on the latest period only: as of its end, or
+      # now for the in-progress one at shutdown. The backlog is read here, in
+      # the reporter thread, but not at shutdown, which has seconds to send.
+      def add_stuck_work(period, include_current)
+        ended_at = [ period.started_at + Aggregator::PERIOD_SECONDS, @clock.call ].min
+        @running_jobs.snapshot(ended_at).each do |job_class, (count, oldest_ms)|
+          period.record_running(job_class, count, oldest_ms, config.max_routes)
+        end
+        period.queue_backlog = QueueBacklog.read unless include_current
+      rescue StandardError => e
+        warn_once(:stuck_work, "DeployAngel failed to read running jobs: #{e.class}: #{e.message}")
+      end
+
       def reset_process_state
         @pid = Process.pid
         @instance = Instance.new(env: @env, now: @clock.call)
         @aggregator = Aggregator.new(max_routes: config.max_routes, clock: @clock)
+        @running_jobs = RunningJobs.new(clock: @clock)
         @buffer = Buffer.new(config.max_queued_payloads)
         @paused_until = nil
         @metadata_sent = false

data/lib/deployangel/core/aggregator.rb

@@ -19,9 +19,16 @@ module DeployAngel
       end
     end
 
-    JobStats = Struct.new(:processed, :failed, :discarded, :duration, :queue_latency) do
+    # running and oldest_running_ms: jobs still running as the period is
+    # sent, and how long the oldest had been running at its end.
+    JobStats = Struct.new(:processed, :failed, :discarded, :duration, :queue_latency, :running, :oldest_running_ms) do
       def self.empty
-        new(0, 0, 0, Histogram.new, Histogram.new)
+        new(0, 0, 0, Histogram.new, Histogram.new, 0, nil)
+      end
+
+      def add_running(count, oldest_ms)
+        self.running += count
+        self.oldest_running_ms = [ oldest_running_ms, oldest_ms ].compact.max
       end
 
       def record(failed, discarded, duration_ms, queue_latency_ms)
@@ -54,6 +61,8 @@ module DeployAngel
     class Period
       attr_reader :started_at, :requests, :status_counts, :unhandled_exceptions, :histogram, :routes,
         :jobs, :job_classes, :exceptions, :exceptions_truncated, :checkpoints
+      # Jobs waiting in the job backend's queues, read as the period is sent.
+      attr_accessor :queue_backlog
 
       def initialize(started_at)
         @started_at = started_at
@@ -97,6 +106,14 @@ module DeployAngel
         (@job_classes[key] ||= JobStats.empty).record(failed, discarded, duration_ms, queue_latency_ms)
       end
 
+      # A class with jobs running and none finished is listed too, so a job
+      # that hangs is reported.
+      def record_running(job_class, count, oldest_ms, max_classes)
+        @jobs.add_running(count, oldest_ms)
+        key = @job_classes.key?(job_class) || @job_classes.size < max_classes - 1 ? job_class : OTHER_ROUTE
+        (@job_classes[key] ||= JobStats.empty).add_running(count, oldest_ms)
+      end
+
       def record_discard(job_class, max_classes)
         @jobs.record_discard
         key = @job_classes.key?(job_class) || @job_classes.size < max_classes - 1 ? job_class : OTHER_ROUTE

data/lib/deployangel/core/protocol.rb

@@ -32,7 +32,7 @@ module DeployAngel
         end,
         "exceptions" => period.exceptions.values.map { |e| e.merge("sources" => e["sources"].to_h).compact },
         "exceptions_truncated" => period.exceptions_truncated,
-        "jobs" => job_stats(period.jobs),
+        "jobs" => job_stats(period.jobs).merge({ "queue_backlog" => period.queue_backlog }.compact),
         "job_classes" => period.job_classes.map { |key, stats| { "key" => key }.merge(job_stats(stats)) },
         # http and job are always sent, 0 when none, so the cloud can tell
         # this agent from one that doesn't say where checkpoints happened.
@@ -42,14 +42,18 @@ module DeployAngel
       }
     end
 
+    # running and oldest_running_ms only when jobs were running.
     def job_stats(stats)
-      {
+      fields = {
         "processed" => stats.processed,
         "failed" => stats.failed,
         "discarded" => stats.discarded,
         "duration_histogram" => stats.duration.to_protocol,
         "queue_latency_histogram" => stats.queue_latency.to_protocol
       }
+      return fields unless stats.running.positive?
+
+      fields.merge("running" => stats.running, "oldest_running_ms" => stats.oldest_running_ms)
     end
 
     def runtime(framework: nil, framework_version: nil)

data/lib/deployangel/core/running_jobs.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  # The jobs this process is running now, so each payload can say how long
+  # the oldest of each class has been running: a job that hangs never
+  # finishes, so it never shows up in the counts. Starting and finishing is
+  # a hash update under a mutex.
+  class RunningJobs
+    # More than any process runs at once; past it, new jobs go untracked
+    # rather than growing memory.
+    MAX = 1_000
+
+    def initialize(clock:)
+      @clock = clock
+      @jobs = {}.compare_by_identity
+      @mutex = Mutex.new
+    end
+
+    # Returns the token to pass to #finish.
+    def start(job_class)
+      token = Object.new
+      @mutex.synchronize { @jobs[token] = [ job_class.to_s, @clock.call ] if @jobs.size < MAX }
+      token
+    end
+
+    def finish(token)
+      @mutex.synchronize { @jobs.delete(token) }
+    end
+
+    # { job_class => [running, oldest_running_ms] } for jobs that started by
+    # at and are still running.
+    def snapshot(at)
+      @mutex.synchronize do
+        @jobs.each_value.with_object({}) do |(job_class, started_at), running|
+          next if started_at > at
+
+          age_ms = ((at - started_at) * 1000).round
+          count, oldest = running[job_class]
+          running[job_class] = [ count.to_i + 1, [ oldest.to_i, age_ms ].max ]
+        end
+      end
+    end
+  end
+end

data/lib/deployangel/hanami/http.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  module Hanami
+    # The Hanami adapter: route identity from DeployAngel's resolver, and the
+    # status Hanami renders a raised exception as. Everything else about
+    # recording a request is in DeployAngel::Rack::Http.
+    #
+    # Hanami renders exceptions in middleware outside the app's own, so this
+    # one sees each exception raised, before it's rendered. Hanami also
+    # raises to say a route wasn't found, and renders that as a 404, so the
+    # status comes from the same mapping Hanami uses.
+    class Http < ::DeployAngel::Rack::Http
+      def initialize(app, hanami_config = nil)
+        super(app)
+        @hanami_config = hanami_config
+      end
+
+      private
+        def route_pattern(env)
+          env[Resolver::ROUTE_PATTERN]
+        end
+
+        # config.render_error_responses, by the exception class's exact name,
+        # as Hanami reads it. Without render_errors (development, tests),
+        # the exception reaches the server, which answers 500.
+        def raised_status(_env, exception)
+          return 500 unless @hanami_config&.render_errors
+
+          status = @hanami_config.render_error_responses[exception.class.name] || :internal_server_error
+          ::Rack::Utils.status_code(status)
+        rescue StandardError
+          500
+        end
+    end
+  end
+end

data/lib/deployangel/hanami/metadata.rb

@@ -0,0 +1,131 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  module Hanami
+    # The Hanami adapter: the route table, each route's action and the files
+    # behind it, and Hanami's layout for the file digests. Schedules read on
+    # every framework, and the digests themselves, are in
+    # DeployAngel::Metadata.
+    class Metadata < ::DeployAngel::Metadata
+      DIGEST_GLOBS = %w[app/**/* slices/**/* config/**/* lib/**/* Gemfile.lock].freeze
+      SKIPPED_METHODS = %w[HEAD OPTIONS].freeze
+
+      def initialize(app:, config:, root:, environment:)
+        super(config: config, root: root, environment: environment)
+        @app = app
+      end
+
+      def digest_globs
+        DIGEST_GLOBS
+      end
+
+      # The routes as `hanami routes` lists them: a second router, built
+      # only to be inspected, whose endpoints are never resolved or called.
+      def routes
+        RouteTable.new(@app).routes.filter_map do |http_method, path, endpoint|
+          next if SKIPPED_METHODS.include?(http_method)
+
+          key = "#{http_method} #{path}"
+          next if health_check_path?(path) || @config.ignored_route?(key)
+
+          route = { "key" => key }
+          route["controller"] = endpoint[:class_name] if endpoint[:class_name]
+          files = endpoint[:files].filter_map { |file| app_relative(file) }.uniq
+          route["files"] = files unless files.empty?
+          route
+        end.uniq { |route| route["key"] }
+      rescue StandardError
+        []
+      end
+    end
+
+    # [http_method, path, endpoint] for each route, where endpoint is
+    # { class_name:, files: }: the action class and its file, view, and
+    # templates, or the file a lambda or Rack app is defined in.
+    class RouteTable
+      # Takes each route's definition from the router.
+      class Inspector
+        attr_reader :routes
+
+        def initialize
+          @routes = []
+        end
+
+        def add_route(route)
+          @routes << route
+        end
+
+        def call(*) = ""
+      end
+
+      def initialize(app)
+        @app = app
+      end
+
+      def routes
+        require "hanami/slice/router"
+        require "hanami/slice/routing/resolver"
+
+        targets = {}
+        inspector = Inspector.new
+        resolver = Class.new(::Hanami::Slice::Routing::Resolver) do
+          # Which slice each `to:` belongs to; nothing is resolved.
+          define_method(:call) do |path, endpoint|
+            targets[[ path.to_s, endpoint ]] ||= slice
+            ->(*) {}
+          end
+        end
+        options = { inspector: inspector, routes: @app.routes, resolver: resolver.new(slice: @app) }
+        # Hanami 3's slice router needs the app's inflector; 2.2's takes none.
+        options[:inflector] = @app.inflector if router_takes?(:inflector)
+        ::Hanami::Slice::Router.new(**options) { nil }
+
+        inspector.routes.map do |route|
+          path = route.path.to_s
+          [ route.http_method.to_s.upcase, path, endpoint(route.to, targets[[ path, route.to ]] || @app) ]
+        end
+      end
+
+      private
+        def router_takes?(keyword)
+          ::Hanami::Slice::Router.instance_method(:initialize).parameters.any? { |type, name| %i[key keyreq].include?(type) && name == keyword }
+        end
+
+        def endpoint(to, slice)
+          case to
+          when String then action(to, slice)
+          when Proc then { class_name: nil, files: [ to.source_location&.first ] }
+          when Module then { class_name: to.name, files: [ source_file(to.name) ] }
+          else { class_name: to.class.name, files: [ source_file(to.class.name) ] }
+          end
+        end
+
+        # "books.show" in a slice is its Actions::Books::Show, in
+        # app/actions/books/show.rb (slices/<name>/actions/... in a slice),
+        # rendered by the view and templates at the same name.
+        def action(key, slice)
+          parts = key.split(".")
+          name = [ slice.namespace.name, "Actions", *parts.map { |part| slice.inflector.camelize(part) } ].join("::")
+          file = source_file(name)
+          files = [ file ]
+          if file&.include?("/actions/")
+            files << file.sub("/actions/", "/views/")
+            templates = File.dirname(file.sub("/actions/", "/templates/"))
+            files.concat(Dir.glob(File.join(templates, "#{File.basename(file, ".rb")}.*")))
+          end
+          { class_name: name, files: files.compact.select { |path| File.file?(path) } }
+        end
+
+        # Loads the class (Zeitwerk autoloads it; nothing is instantiated),
+        # since a constant not loaded yet has no source location.
+        def source_file(name)
+          return unless name
+
+          Object.const_get(name)
+          Object.const_source_location(name)&.first
+        rescue NameError, StandardError
+          nil
+        end
+    end
+  end
+end

data/lib/deployangel/hanami/resolver.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  module Hanami
+    # Hanami's router doesn't record which route a request matched, only its
+    # parameters. Its resolver, which turns each route's `to:` into an
+    # endpoint as the routes are defined, is told the route's full pattern
+    # (scopes' and slices' prefixes included), so this one wraps each
+    # endpoint to note the pattern on the request, as Rails' router does.
+    # It subclasses whatever resolver the app has configured.
+    module Resolver
+      ROUTE_PATTERN = "deployangel.route_pattern"
+
+      def self.for(base)
+        return base if base <= Tagging
+
+        Class.new(base) do
+          prepend Tagging
+
+          def self.name = "DeployAngel::Hanami::Resolver(#{superclass.name})"
+        end
+      end
+
+      module Tagging
+        def call(path, endpoint)
+          resolved = super
+          pattern = path.to_s
+          lambda do |env, *rest|
+            env[ROUTE_PATTERN] = pattern
+            resolved.call(env, *rest)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/deployangel/hanami.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+require_relative "../deployangel"
+require_relative "hanami/resolver"
+require_relative "hanami/http"
+require_relative "hanami/metadata"
+
+module DeployAngel
+  # The Hanami integration, for Hanami 2.2 or later. Hanami has no Railtie,
+  # so the app installs it where it's configured:
+  #
+  #   # config/app.rb
+  #   require "hanami"
+  #   require "deployangel/hanami"
+  #
+  #   module Shop
+  #     class App < Hanami::App
+  #       DeployAngel::Hanami.install(self)
+  #     end
+  #   end
+  #
+  # Inside this namespace a bare Hanami means DeployAngel::Hanami, so reach
+  # for the framework itself as ::Hanami.
+  module Hanami
+    class << self
+      # Names each request's route (through the router's resolver), records
+      # requests (through the app's middleware), and starts the agent once
+      # the app boots (its container is finalized, as config.ru's
+      # hanami/boot does). Safe to call more than once.
+      def install(app)
+        @installed ||= []
+        return app if @installed.include?(app)
+
+        @installed << app
+        config = app.config
+        config.router.resolver = Resolver.for(config.router.resolver)
+        config.middleware.use(Http, config)
+        app.container.after(:finalize) { DeployAngel::Hanami.start(app) }
+        DeployAngel::Sidekiq.install
+        DeployAngel::QueueBacklog.install
+        app
+      rescue StandardError => e
+        DeployAngel.configuration.logger&.warn("DeployAngel failed to install in Hanami: #{e.class}: #{e.message}")
+        app
+      end
+
+      # Called as the app boots. The metadata goes in at start, before the
+      # reporter thread can run: the code fingerprint is built from its file
+      # digests.
+      def start(app)
+        root = app.root.to_s
+        environment = ::Hanami.env.to_s
+        DeployAngel.start(
+          environment: environment,
+          root: root,
+          framework: "hanami",
+          framework_version: (::Hanami::VERSION if defined?(::Hanami::VERSION)),
+          logger: logger(app),
+          metadata: Metadata.new(app: app, config: DeployAngel.configuration, root: root, environment: environment)
+        )
+      end
+
+      private
+        def logger(app)
+          app["logger"] if app.key?("logger")
+        rescue StandardError
+          nil
+        end
+    end
+  end
+end

data/lib/deployangel/metadata.rb

@@ -214,6 +214,21 @@ module DeployAngel
       nil
     end
 
+    # A route at a conventional health-check path ("/healthz"), left out of
+    # the route table as the request middleware leaves it out of telemetry.
+    def health_check_path?(path)
+      segments = path.to_s.split("/").reject(&:empty?)
+      Rack::Http::HEALTH_CHECK_PATHS.include?(segments.last) && segments.none? { |segment| segment.start_with?(":", "*") }
+    end
+
+    # A source file in the app, relative to its root; nil for one in a gem.
+    def app_relative(file)
+      return unless file&.start_with?("#{@root}/")
+
+      path = file.delete_prefix("#{@root}/")
+      path unless path.start_with?("vendor/", ".bundle/")
+    end
+
     # The path, or the same name with .yaml if only that exists.
     def yaml_path(path)
       [ path, path.sub(/\.yml\z/, ".yaml") ].find { |candidate| File.file?(candidate) }

data/lib/deployangel/queue_backlog.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  # Reads how many jobs are ready to run and waiting in the job backend's
+  # queues. Only job worker processes read it, once a minute from the
+  # reporter thread as they send telemetry, never in a request or job. Every
+  # worker sees the same queues, and the cloud keeps the largest reading.
+  module QueueBacklog
+    module_function
+
+    # Solid Queue runs each worker in its own process, forked by the
+    # supervisor, so it's marked as each one starts.
+    def install
+      return if @installed || !defined?(::SolidQueue) || !::SolidQueue.respond_to?(:on_worker_start)
+
+      @installed = true
+      ::SolidQueue.on_worker_start { worker!(:solid_queue) }
+    rescue StandardError
+      nil
+    end
+
+    # Called where the backend says this process works its queues: Sidekiq's
+    # server configuration, or a Solid Queue worker starting.
+    def worker!(backend)
+      @backend = backend
+    end
+
+    def backend
+      @backend
+    end
+
+    # Jobs waiting, or nil when this process doesn't work a queue it can read.
+    def read
+      case @backend
+      when :sidekiq then sidekiq
+      when :solid_queue then solid_queue
+      end
+    rescue StandardError, LoadError
+      nil
+    end
+
+    # One pipelined call for every queue's length.
+    def sidekiq
+      require "sidekiq/api"
+      ::Sidekiq::Stats.new.enqueued
+    end
+
+    # Ready executions only: scheduled and blocked jobs aren't waiting on a
+    # worker. The connection goes straight back to the pool.
+    def solid_queue
+      model = ::SolidQueue::ReadyExecution
+      model.connection_pool.with_connection { model.count }
+    end
+  end
+end

data/lib/deployangel/rack/http.rb

@@ -10,7 +10,9 @@ module DeployAngel
     # An adapter subclasses it, answers route_pattern (and rendered_exception
     # if the framework renders exceptions itself), and is inserted at the top
     # of the stack, so it sees the final status after the framework's own
-    # error pages. DeployAngel::Rails::Http is the Rails adapter.
+    # error pages. A framework that renders exceptions outside its app
+    # middleware instead answers raised_status. DeployAngel::Rails::Http and
+    # DeployAngel::Hanami::Http are the adapters.
     #
     # A request is only counted against a route pattern the framework
     # matched, never the raw path, which keeps IDs out of route keys.
@@ -26,9 +28,16 @@ module DeployAngel
         @app = app
       end
 
+      # Set on a request once a DeployAngel middleware records it. Apps nest
+      # (a Sinatra app used as another's middleware, or mounted inside
+      # Rails), so a request can pass through more than one; only the
+      # outermost records it.
+      RECORDED = "deployangel.recorded"
+
       def call(env)
-        return @app.call(env) unless DeployAngel.recording?
+        return @app.call(env) if !DeployAngel.recording? || env[RECORDED]
 
+        env[RECORDED] = true
         started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
         Redaction.request_host = env["HTTP_HOST"]
         # Checkpoints recorded from here until the app returns count as
@@ -37,7 +46,10 @@ module DeployAngel
         begin
           status, headers, body = @app.call(env)
         rescue Exception => e # rubocop:disable Lint/RescueException -- recorded, then re-raised untouched
-          record(env, 500, started, unhandled: true, exception: e)
+          # A framework that renders exceptions outside this middleware turns
+          # some into a 4xx; only one that ends in a 5xx is unhandled.
+          status = raised_status(env, e)
+          record(env, status, started, unhandled: status >= 500, exception: (e if status >= 500))
           raise
         ensure
           ExecutionContext.restore(previous_context)
@@ -65,6 +77,12 @@ module DeployAngel
           nil
         end
 
+        # The status a raised exception will be rendered as, by a framework
+        # that renders it after this middleware. 500 unless an adapter knows.
+        def raised_status(_env, _exception)
+          500
+        end
+
         def route_key(env)
           pattern = route_pattern(env) or return nil
 

data/lib/deployangel/rails/active_job.rb

@@ -12,12 +12,14 @@ module DeployAngel
     module ActiveJob
       STATE = :@__deployangel_job_state
       PREVIOUS_CONTEXT = :@__deployangel_previous_context
+      RUNNING = :@__deployangel_running
 
       # Marks the fiber as running a job for as long as perform.active_job
-      # lasts, so checkpoints recorded by the job say so. ActiveSupport calls
-      # finish from an ensure, so the previous context (an HTTP request, for
-      # a job performed inline) comes back even when the job raises. It is
-      # kept on the job, so only a perform that started here restores.
+      # lasts, so checkpoints recorded by the job say so, and the job as
+      # running, so one that hangs is reported. ActiveSupport calls finish
+      # from an ensure, so the previous context (an HTTP request, for a job
+      # performed inline) comes back even when the job raises. Both are kept
+      # on the job, so only a perform that started here finishes them.
       module ContextListener
         module_function
 
@@ -26,6 +28,7 @@ module DeployAngel
           return unless job && DeployAngel.recording?
 
           job.instance_variable_set(PREVIOUS_CONTEXT, ExecutionContext.enter(ExecutionContext::JOB))
+          job.instance_variable_set(RUNNING, DeployAngel.job_started(job.class.name))
         rescue StandardError
           nil
         end
@@ -35,6 +38,7 @@ module DeployAngel
           return unless job&.instance_variable_defined?(PREVIOUS_CONTEXT)
 
           ExecutionContext.restore(job.remove_instance_variable(PREVIOUS_CONTEXT))
+          DeployAngel.job_finished(job.remove_instance_variable(RUNNING)) if job.instance_variable_defined?(RUNNING)
         rescue StandardError
           nil
         end

data/lib/deployangel/rails/railtie.rb

@@ -12,6 +12,7 @@ module DeployAngel
       initializer "deployangel.jobs" do
         ActiveSupport.on_load(:active_job) { DeployAngel::Rails::ActiveJob.install }
         DeployAngel::Sidekiq.install
+        DeployAngel::QueueBacklog.install
       end
 
       # Handled reports (Rails.error.handle / Rails.error.report) are shown for

data/lib/deployangel/sidekiq.rb

@@ -17,6 +17,7 @@ module DeployAngel
         started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
         latency = DeployAngel::Sidekiq.queue_latency_ms(job)
         previous_context = ExecutionContext.enter(ExecutionContext::JOB)
+        running = DeployAngel.job_started(job["class"])
         begin
           yield
         rescue Exception => e # rubocop:disable Lint/RescueException -- recorded, then re-raised untouched
@@ -25,6 +26,7 @@ module DeployAngel
           raise
         ensure
           ExecutionContext.restore(previous_context)
+          DeployAngel.job_finished(running)
         end
         record(job, started, latency, failed: false)
       end
@@ -46,6 +48,7 @@ module DeployAngel
       @installed = true
       DeployAngel.add_capability("jobs")
       ::Sidekiq.configure_server do |config|
+        QueueBacklog.worker!(:sidekiq)
         config.server_middleware { |chain| chain.add(ServerMiddleware) }
         # Jobs that exhausted their retries are discarded.
         config.death_handlers << ->(job, _error) { record_death(job) } if config.respond_to?(:death_handlers)

data/lib/deployangel/sinatra/http.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  module Sinatra
+    # The Sinatra adapter: route identity from the route Sinatra matched,
+    # with the app's mount point, and the exceptions Sinatra renders itself.
+    # Everything else about recording a request is in DeployAngel::Rack::Http.
+    class Http < ::DeployAngel::Rack::Http
+      def initialize(app, sinatra_app = nil)
+        super(app)
+        @sinatra_app = sinatra_app
+      end
+
+      private
+        # Sinatra names the route it ran ("GET /books/:id"), the last one when
+        # a route passes, but not the mount point (SCRIPT_NAME), which
+        # Sinatra strips from the path it matches.
+        def route_pattern(env)
+          route = env["sinatra.route"] or return nil
+
+          pattern = route.split(" ", 2).last
+          mount = env["SCRIPT_NAME"].to_s.chomp("/")
+          note_mount(mount) unless mount.empty?
+          "#{mount}#{pattern}"
+        end
+
+        # Sinatra rescues an exception, renders its error page, and keeps the
+        # exception here.
+        def rendered_exception(env)
+          env["sinatra.error"]
+        end
+
+        def note_mount(mount)
+          DeployAngel::Sinatra.mounts[@sinatra_app] ||= mount if @sinatra_app
+        rescue StandardError
+          nil
+        end
+    end
+  end
+end

data/lib/deployangel/sinatra/metadata.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module DeployAngel
+  module Sinatra
+    # The Sinatra adapter: the route table of every app that registered the
+    # extension, with each route's file, and a typical Sinatra layout for the
+    # file digests. Schedules read on every framework, and the digests
+    # themselves, are in DeployAngel::Metadata.
+    class Metadata < ::DeployAngel::Metadata
+      DIGEST_GLOBS = %w[*.rb *.ru app/**/* lib/**/* config/**/* routes/**/* views/**/* models/**/* helpers/**/*
+                        controllers/**/* db/migrate/**/* Gemfile.lock].freeze
+      SKIPPED_METHODS = %w[HEAD OPTIONS].freeze
+
+      def digest_globs
+        DIGEST_GLOBS
+      end
+
+      def routes
+        DeployAngel::Sinatra.apps.flat_map { |app| app_routes(app) }
+          .uniq { |route| route["key"] }
+      rescue StandardError
+        []
+      end
+
+      private
+        def app_routes(app)
+          mount = DeployAngel::Sinatra.mounts[app].to_s
+          app.routes.flat_map do |verb, signatures|
+            next [] if SKIPPED_METHODS.include?(verb)
+
+            signatures.filter_map do |pattern, *|
+              path = pattern.to_s
+              key = "#{verb} #{mount}#{path}"
+              next if health_check_path?(path) || @config.ignored_route?(key)
+
+              route = { "key" => key }
+              files = Array(DeployAngel::Sinatra.route_files[[ verb, path ]]).filter_map { |file| app_relative(file) }.uniq
+              route["files"] = files unless files.empty?
+              route
+            end
+          end
+        rescue StandardError
+          []
+        end
+
+    end
+  end
+end

data/lib/deployangel/sinatra.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+require_relative "../deployangel"
+require_relative "sinatra/http"
+require_relative "sinatra/metadata"
+
+module DeployAngel
+  # The Sinatra integration, for Sinatra 3.0 or later, as an extension:
+  #
+  #   require "sinatra/base"
+  #   require "deployangel/sinatra"
+  #
+  #   class Bookshelf < Sinatra::Base
+  #     register DeployAngel::Sinatra
+  #     # routes after it, so their files are known
+  #   end
+  #
+  # A classic app registers it at the top level, after `require "sinatra"`.
+  # Inside this namespace a bare Sinatra means DeployAngel::Sinatra, so reach
+  # for the framework itself as ::Sinatra.
+  module Sinatra
+    class << self
+      # The apps that registered the extension, and each route's file, by
+      # [verb, path]: Sinatra hands an extension each route's own block as
+      # it's defined, while the route table holds only Sinatra's wrapper.
+      def apps
+        @apps ||= []
+      end
+
+      def route_files
+        @route_files ||= {}
+      end
+
+      # Where each app was mounted (a map "/api" in config.ru), as its
+      # requests show it, since the routes it declares don't include it.
+      def mounts
+        @mounts ||= {}
+      end
+
+      # Records the app's requests (through its middleware) and starts the
+      # agent, once per process.
+      def registered(app)
+        return if apps.include?(app)
+
+        apps << app
+        app.use(Http, app)
+        DeployAngel::Sidekiq.install
+        DeployAngel::QueueBacklog.install
+        start(app) unless DeployAngel.agent
+      rescue StandardError => e
+        DeployAngel.configuration.logger&.warn("DeployAngel failed to install in Sinatra: #{e.class}: #{e.message}")
+      end
+
+      def route_added(verb, path, block)
+        file = block&.source_location&.first
+        (route_files[[ verb.to_s, path.to_s ]] ||= []) << file if file
+      rescue StandardError
+        nil
+      end
+
+      # The metadata goes in at start, before the reporter thread can run:
+      # the code fingerprint is built from its file digests. The working
+      # directory is the app's root, as in most deploys.
+      def start(app)
+        root = Dir.pwd
+        environment = app.environment.to_s
+        DeployAngel.start(
+          environment: environment,
+          root: root,
+          framework: "sinatra",
+          framework_version: (::Sinatra::VERSION if defined?(::Sinatra::VERSION)),
+          metadata: Metadata.new(config: DeployAngel.configuration, root: root, environment: environment)
+        )
+      end
+    end
+  end
+end

data/lib/deployangel/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DeployAngel
-  VERSION = "0.1.28"
+  VERSION = "0.1.30"
 end

data/lib/deployangel.rb

@@ -8,6 +8,7 @@ require_relative "deployangel/core/fingerprint"
 require_relative "deployangel/core/release"
 require_relative "deployangel/core/instance"
 require_relative "deployangel/core/aggregator"
+require_relative "deployangel/core/running_jobs"
 require_relative "deployangel/core/protocol"
 require_relative "deployangel/core/execution_context"
 require_relative "deployangel/core/buffer"
@@ -57,6 +58,14 @@ module DeployAngel
       agent&.record_job(**attributes)
     end
 
+    def job_started(job_class)
+      agent&.job_started(job_class)
+    end
+
+    def job_finished(token)
+      agent&.job_finished(token)
+    end
+
     def record_exception(exception, **options)
       agent&.record_exception(exception, **options)
     end
@@ -128,6 +137,7 @@ require_relative "deployangel/rails/error_subscriber"
 require_relative "deployangel/tasks"
 require_relative "deployangel/metadata"
 require_relative "deployangel/rails/metadata"
+require_relative "deployangel/queue_backlog"
 require_relative "deployangel/sidekiq"
 require_relative "deployangel/apartment"
 require_relative "deployangel/rails/railtie" if defined?(::Rails::Railtie)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: deployangel
 version: !ruby/object:Gem::Version
-  version: 0.1.28
+  version: 0.1.30
 platform: ruby
 authors:
 - Jordan Owens
@@ -45,13 +45,19 @@ files:
 - lib/deployangel/core/protocol.rb
 - lib/deployangel/core/redaction.rb
 - lib/deployangel/core/release.rb
+- lib/deployangel/core/running_jobs.rb
 - lib/deployangel/core/transport.rb
 - lib/deployangel/credentials.rb
 - lib/deployangel/doctor.rb
 - lib/deployangel/exerciser.rb
 - lib/deployangel/fork_hook.rb
+- lib/deployangel/hanami.rb
+- lib/deployangel/hanami/http.rb
+- lib/deployangel/hanami/metadata.rb
+- lib/deployangel/hanami/resolver.rb
 - lib/deployangel/mcp.rb
 - lib/deployangel/metadata.rb
+- lib/deployangel/queue_backlog.rb
 - lib/deployangel/rack/http.rb
 - lib/deployangel/rails/active_job.rb
 - lib/deployangel/rails/error_subscriber.rb
@@ -59,6 +65,9 @@ files:
 - lib/deployangel/rails/metadata.rb
 - lib/deployangel/rails/railtie.rb
 - lib/deployangel/sidekiq.rb
+- lib/deployangel/sinatra.rb
+- lib/deployangel/sinatra/http.rb
+- lib/deployangel/sinatra/metadata.rb
 - lib/deployangel/tasks.rb
 - lib/deployangel/verification_waiter.rb
 - lib/deployangel/version.rb
@@ -88,5 +97,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 3.6.9
 specification_version: 4
-summary: 'DeployAngel agent for Rails: production verification for every deployment.'
+summary: 'DeployAngel agent for Rails, Hanami, and Sinatra: production verification
+  for every deployment.'
 test_files: []