goodsorcery 0.1.1 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1413354a65863f49ea7822ea12cf89b824a71ccec864719ed8dcb0443a54fb48
-  data.tar.gz: e5f2d0445ba8261748f55dd09d15e2ae100bfa8349cf76a55d778fe13ebe51e2
+  metadata.gz: edbecd397465b4b9853689dcaeb10b832770545b51c4801f3f3dd22d6c126b08
+  data.tar.gz: fd3b3fdf18956d5e9dd9db3af4e042a5385459f531ad6791851551949590ab6c
 SHA512:
-  metadata.gz: b7df0121fdb191079e323cb6b6242134c21735aa461aa8c6a9bbed04572c50a820a863d90957a3e7fa57cb9d9b241ea6c9399d7ce71a5974c6c93cde833d956d
-  data.tar.gz: 4533d8744e3d5b8b1b93402b02cd18d6b6c8468222a5945cfb98c417b33a752a43ac72289e4beccf1a0915605d844e4e16656970cdc06a132406ff8bfae3fcf2
+  metadata.gz: dcb53d171bd4f36a48c7208a9cd2b3950bd1be25741b9946945da0db0cba745b75ee8159a6058a12288fda63258d5cdcd3fe413d25f538c9c7d7601592c2fbb8
+  data.tar.gz: 9767d439220b934647dc06cb805fca3d0ba808f97438cb320efc3749e220e0b9d7487fc613cfbb711781a795029cd9b0cfd834d559d2417d79af8f65a176819b

data/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## 0.3.0 (2026-11-07)
+
+- The `:active_job` delivery: `GoodSorcery::DeliveryJob`, `config.active_job_queue` and `GoodSorcery::RetryableDeliveryError`. Use `:async` (the default). Errors raised in your own jobs are still reported, and jobs are still measured by performance monitoring.
+- Custom delivery objects (`config.delivery = <object responding to #deliver>`).
+- `config.delivery` now accepts only `:async`, `:sync` and `:test`; anything else, including `:active_job` or an object, raises `ArgumentError` from `GoodSorcery.configure` with a message saying what to use instead. Setting `config.active_job_queue` raises `NoMethodError`.
+
+## 0.2.0 (2026-11-07)
+
+- **Performance monitoring** (opt-in, `config.performance_monitoring = true` or `GOODSORCERY_PERFORMANCE_MONITORING=1`). Records every Rails request and every Active Job and Sidekiq job: count, failures, total time and a shared log-scale histogram (`gs-v1`) with database, external HTTP, view and Ruby time per bucket, plus queue wait for jobs. Metrics are aggregated per transaction per minute in process and sent once a minute to `POST /api/v1/apm/metrics` from a background thread (`goodsorcery-apm`).
+- **Traces** of the slowest request per transaction per minute (`config.performance_traces`, default on), sent to `POST /api/v1/apm/traces`: SQL, `Net::HTTP`, render and enqueue spans; repeated queries grouped with their count, total and app call site (with code context under `send_code_context`); SQL normalised so no literal values leave the process; HTTP spans and the request path without query strings; GC, allocations, host and process; the id of the error reported during the request.
+- `config.performance_ignore_transactions` (default `["Rails::HealthController#show"]`). `GoodSorcery::DeliveryJob` is never recorded.
+- `GoodSorcery.flush_performance` and, with the `:test` delivery, `GoodSorcery.test_performance_payloads`.
+- The gem's own HTTP requests are never measured. Payloads keep their `batch_id` across retries; `429`, `5xx`, `404` (server without performance endpoints: monitoring turns off) and rejected payloads are handled as documented; data is bounded to five minutes and flushed at exit; fork-safe.
+- `bundle exec rake bench:performance` measures the per-request cost.
+
 ## 0.1.1 (2026-10-11)
 
 - `Sidekiq::JobRetry::Handled` and `Sidekiq::JobRetry::Skip` are ignored by default. Sidekiq raises them internally after its retry handling, so reporting them only duplicated the original error.

data/README.md

@@ -7,6 +7,7 @@ The Ruby client for the [Good Sorcery](https://app.goodsorcery.com) error tracke
 - The queue is bounded and **drops rather than blocks** when it is full. Rate limits (`429` + `Retry-After`) and server errors are retried with backoff.
 - It has no runtime dependencies (standard library only). It needs Ruby 3.2 or newer and works with or without Rails.
 - Rails, Rack/Sinatra, Active Job, Action Mailer, Action Cable, Sidekiq and Rake are covered out of the box (see [Integrations](#integrations)).
+- Optional, opt-in [performance monitoring](#performance-monitoring) for Rails apps: response times of every request and job, and timelines of the slowest ones.
 
 ## Installation
 
@@ -98,24 +99,26 @@ Change settings inside `GoodSorcery.configure`: it recompiles the filters and re
 | `send_job_arguments` | `true` | `false` never sends Active Job, mailer or Sidekiq arguments. |
 | `install_check` | `:verify` | `:verify`, `:test_event` or `false`; see [Install check](#install-check). |
 | `logger` | `Logger.new($stderr)` at `WARN` (`Rails.logger` in Rails) | The gem's own messages, prefixed `[goodsorcery]`. |
-| `delivery` | `:async` | See [Delivery](#delivery). |
+| `delivery` | `:async` | `:async`, `:sync` or `:test`; see [Delivery](#delivery). |
 | `batch_size` | `25` | Events per request, 1–100. |
 | `flush_interval` | `2.0` s | Longest an event waits in the queue. |
 | `max_queue_size` | `1_000` | Events beyond this are dropped and counted. |
 | `max_retries` | `3` | For `5xx`/network failures (exponential backoff with jitter, capped at 60 s). |
 | `shutdown_timeout` | `5` s | How long the exit hook waits for the queue to drain. |
 | `open_timeout` / `read_timeout` | `2` s / `5` s | HTTP timeouts (writes use `read_timeout`). |
-| `active_job_queue` | `"default"` | Queue for `delivery = :active_job`. |
+| `performance_monitoring` | `false` (or `ENV["GOODSORCERY_PERFORMANCE_MONITORING"]` `1`/`0`) | Records every request and job and sends metrics once a minute; see [Performance monitoring](#performance-monitoring). Read when the app boots. |
+| `performance_traces` | `true` | Also sends the timeline of the slowest request per transaction per minute. `false` sends metrics only. |
+| `performance_ignore_transactions` | `["Rails::HealthController#show"]` | Transaction names (exact strings) or regexps never recorded. |
 
 ### Delivery
 
 | `config.delivery` | What it does | When to use |
 |---|---|---|
 | `:async` (default) | Pushes events onto a bounded in-memory queue. One background thread (`goodsorcery-worker`) sends them to `POST /api/v1/events/batch` when `batch_size` events are waiting or `flush_interval` has passed. Restarts after `fork`; drains at exit. | Web apps and job workers (almost everyone). |
-| `:sync` | Sends each event immediately in the calling thread. | Scripts, rake tasks, serverless functions, debugging. |
-| `:active_job` | Enqueues `GoodSorcery::DeliveryJob` with the filtered event as JSON; the job sends it and retries `429`/`5xx` (5 attempts). Falls back to `:async` if enqueueing fails. **The filtered event is stored in your job backend.** | Apps that want delivery to survive a crash and already run a durable queue. |
+| `:sync` | Sends each event in the calling thread, before `notify` returns. | Serverless functions and other short-lived processes where a background thread doesn't get to run, such as AWS Lambda (the process is frozen between invocations). **Not for normal web apps:** every report adds an HTTPS round trip to the request that hit the error. |
 | `:test` | Keeps events in memory (`GoodSorcery.test_events`). Never touches the network and ignores the environment check. | Your app's test suite. |
-| any object | Must respond to `deliver(event_hash)`; may respond to `flush(timeout)` and `shutdown(timeout)`. | Custom transports. |
+
+Any other value raises an `ArgumentError` from `GoodSorcery.configure`, when the app boots. The `:active_job` delivery and custom delivery objects were removed; use `:async`.
 
 ### Environments
 
@@ -205,7 +208,7 @@ Nothing to set up beyond the initializer. The Railtie:
 - inserts `GoodSorcery::Rack::Middleware` right after `ActionDispatch::ShowExceptions`, so it sees exceptions before Rails renders the error page. Errors that a lower layer turned into a 5xx response (`env["action_dispatch.exception"]`) are reported too;
 - subscribes to `Rails.error`: unhandled errors are reported at level `error`, handled ones (`Rails.error.handle`, `Rails.error.report(handled: true)`) at their severity (`warning` by default). Set `report_handled_errors = false` to skip handled ones;
 - finds the user with the controller's `user_method` (default `current_user`, private methods too), else `Current.user`, and sends only `id`, `email`, `username` and `name`;
-- wraps Active Job's `perform_now` outside the job's own `rescue_from`/`retry_on`, so a job is reported once when it finally fails, not on every attempt it retries itself. `GoodSorcery::DeliveryJob` is never reported;
+- wraps Active Job's `perform_now` outside the job's own `rescue_from`/`retry_on`, so a job is reported once when it finally fails, not on every attempt it retries itself;
 - wraps Action Cable channel subscribe, unsubscribe and actions; errors your channel's `rescue_from` handles are not reported;
 - records breadcrumbs from `ActiveSupport::Notifications` inside requests, jobs and channel actions: `start_processing.action_controller`, `sql.active_record` (without `SCHEMA`, `CACHE` and `TRANSACTION` queries; SQL capped at 1,000 chars, with quoted strings and numbers replaced by `?` so inlined values never leave the app), `perform.active_job` and `deliver.action_mailer`. Turn them off with `config.breadcrumbs = false`;
 - loads the `goodsorcery:verify` and `goodsorcery:test` tasks and reports exceptions that end a rake task.
@@ -230,6 +233,30 @@ Hooked in automatically when Sidekiq is loaded before `goodsorcery` or in a Rail
 
 Rails apps get this automatically; elsewhere `require "goodsorcery/tasks"` in your Rakefile. An exception that ends a rake run is reported with `transaction = "rake <tasks>"` (task arguments stay out of the name and go, filtered, to `context.rake`), and the queue is flushed, for up to `shutdown_timeout`, before rake exits.
 
+## Performance monitoring
+
+Off by default. Turn it on in the initializer (or with `GOODSORCERY_PERFORMANCE_MONITORING=1`) and restart:
+
+```ruby
+GoodSorcery.configure do |config|
+  config.performance_monitoring = true
+  # config.performance_traces = true                  # false = metrics only
+  # config.performance_ignore_transactions += ["HealthController#show", /\AAdmin::/]
+end
+```
+
+It follows the same environment rules as error reporting and needs an API key: in a disabled environment nothing is subscribed, no thread starts and a request costs one boolean check.
+
+**What is measured.** Each controller action (`CheckoutsController#create`, with the HTTP method and the route pattern such as `/checkout`, never the real path) and each job (Active Job with any adapter, and Sidekiq jobs not wrapped by Active Job; with the adapter, queue and queue wait). A request or job counts as failed when an exception escapes it (for a request, one Rails answers with a 5xx: a `RecordNotFound` is recorded as a 404, not a failure), when a request answers with a 5xx status, or when Active Job's `retry_on`/`discard_on` handles a job's exception. Time is split into database (`sql.active_record`, without `SCHEMA` and cached queries), external HTTP (`Net::HTTP` calls), views (template renders, without the queries and HTTP calls made from them) and Ruby (the rest). Requests that never reach a controller (routing 404s, assets) and a job run inline inside a request are not separate transactions.
+
+**How it is sent.** Numbers are aggregated in your process per transaction per minute (count, failures, total time and a log-scale histogram with the time per layer) and sent once a minute from one background thread (`goodsorcery-apm`) as a small gzip'd request. The slowest request per transaction per minute also sends its **trace** (built after the response has been sent, on servers such as Puma that allow it): a timeline of its SQL queries, HTTP calls, template renders and job enqueues. Identical queries are grouped with their count, total time and the line of your code that ran them, so N+1 queries stand out. Unsent data is kept for at most five minutes; the payloads are retried with the same batch id.
+
+**What a trace never contains.** No params, headers, cookies, session, request or response bodies, query strings, job arguments or user data. SQL is normalised before it leaves the process: string and number literals become `?`, `IN (…)` lists and `VALUES` rows collapse, comments are removed; bind placeholders such as `$1` stay. Quotes are read the way your database reads them (from the connection's adapter), and when that cannot be told, everything after a string holding a backslash becomes `?`. HTTP calls are method, host and path only (never credentials, even when the app passes a whole URL). Code lines around a repeated query's call site are sent only with `send_code_context` (the default). `before_send` and `filter_params` don't apply, because there is nothing key/value-shaped to filter.
+
+**Cost.** Per request, a few microseconds of bookkeeping plus about 2 µs per query (`bundle exec rake bench:performance` measures it on your machine). Memory is bounded: at most 2,000 spans and 4 MiB of SQL text per running request, 500 transaction names per minute and kind (the rest are counted as `(other)`), five minutes of metrics and 30 traces per minute.
+
+Outside Rails, call `GoodSorcery::Performance.install` after `GoodSorcery.configure`; jobs and `Net::HTTP` are then measured, but web requests need Rails to get their names. In your test suite, the `:test` delivery keeps the payloads in memory: `GoodSorcery.flush_performance` builds them into `GoodSorcery.test_performance_payloads` (`[{ "path" => "/api/v1/apm/metrics", "body" => {...} }]`).
+
 ## Install check
 
 `bin/rails goodsorcery:verify` (or `rake goodsorcery:verify`) checks your setup on demand; see the end of this section.
@@ -270,7 +297,7 @@ event = GoodSorcery.test_events.last
 assert_equal "Payments::DeclinedError", event["exception"]["class"]
 ```
 
-The `:test` delivery ignores the environment check, stores the filtered event hashes exactly as they would be sent, and never touches the network.
+The `:test` delivery ignores the environment check, stores the filtered event hashes exactly as they would be sent, and never touches the network. With performance monitoring started (`GoodSorcery::Performance.install` after setting `config.performance_monitoring = true`), `GoodSorcery.flush_performance` stores the performance payloads in `GoodSorcery.test_performance_payloads`; `clear_test_events` clears both.
 
 ## Self-hosted servers
 
@@ -291,6 +318,7 @@ These are for integrations and advanced setups:
 bin/setup              # bundle install, then the whole test suite
 bundle exec rake test  # core, then Rack/Sidekiq/Rake, then Rails tests, each in its own process
 bundle exec rubocop
+bundle exec rake bench:performance  # cost of performance monitoring per request (not run in CI)
 bin/console
 ```
 

data/lib/generators/goodsorcery/install/templates/initializer.rb

@@ -27,7 +27,7 @@ GoodSorcery.configure do |config|
   # config.send_job_arguments = true                     # false = never send job arguments
 
   # --- Delivery --------------------------------------------------------------
-  # config.delivery = :async                             # :sync, :active_job, :test or a custom object
+  # config.delivery = :async                             # :sync (serverless, e.g. AWS Lambda) or :test
   # config.batch_size = 25
   # config.flush_interval = 2.0
   # config.max_queue_size = 1_000
@@ -35,7 +35,6 @@ GoodSorcery.configure do |config|
   # config.shutdown_timeout = 5
   # config.open_timeout = 2
   # config.read_timeout = 5
-  # config.active_job_queue = "default"                  # with delivery = :active_job
 
   # --- What gets reported ----------------------------------------------------
   # config.release = ENV["GIT_SHA"]                      # auto-detected from deploy env vars or REVISION
@@ -48,6 +47,13 @@ GoodSorcery.configure do |config|
   # config.breadcrumbs = true
   # config.max_breadcrumbs = 40
 
+  # --- Performance monitoring --------------------------------------------------
+  # Off by default. Records every request and job and sends the slowest
+  # requests' timelines once a minute (also ENV["GOODSORCERY_PERFORMANCE_MONITORING"]=1).
+  # config.performance_monitoring = true
+  # config.performance_traces = true                     # false = metrics only, no traces
+  # config.performance_ignore_transactions += ["HealthController#show", /\AAdmin::/]
+
   # --- Development -----------------------------------------------------------
   # config.install_check = :verify                       # after bundle install: :verify, :test_event or false
   # config.logger = Rails.logger

data/lib/goodsorcery/client.rb

@@ -4,7 +4,6 @@ module GoodSorcery
   # Wires configuration to the event pipeline and the delivery. One client
   # per process, built lazily; GoodSorcery.configure rebuilds its parts.
   class Client
-    NETWORK_DELIVERIES = %i[async sync active_job].freeze
     REPORTING_KEY = :__goodsorcery_reporting
 
     attr_reader :config, :log
@@ -69,9 +68,18 @@ module GoodSorcery
     end
 
     def shutdown(timeout = config.shutdown_timeout)
+      Performance.shutdown(timeout)
       @delivery ? @delivery.shutdown(timeout) : true
     end
 
+    # Whether performance monitoring may run (DESIGN.md 8.1): with the :test
+    # delivery, or in an enabled environment with a key.
+    def performance_allowed?
+      return true if test_delivery?
+
+      @gate.enabled? && !config.api_key.to_s.empty?
+    end
+
     def verify
       Verification.run(config, transport)
     end
@@ -123,7 +131,9 @@ module GoodSorcery
       event = prepare_event(exception, options)
       return nil unless event
 
-      delivery.deliver(event) == false ? nil : event["event_id"]
+      event_id = delivery.deliver(event) == false ? nil : event["event_id"]
+      Performance.note_error(event_id)
+      event_id
     rescue StandardError => e
       log.error("notify failed with #{e.class}; the error was not reported")
       nil
@@ -144,7 +154,7 @@ module GoodSorcery
     end
 
     def skip?(exception)
-      ReportedMark.reported?(exception) || exception.is_a?(RetryableDeliveryError) || ignore_list.ignore?(exception)
+      ReportedMark.reported?(exception) || ignore_list.ignore?(exception)
     end
 
     def test_delivery?
@@ -152,7 +162,6 @@ module GoodSorcery
     end
 
     def credentials_present?
-      return true unless NETWORK_DELIVERIES.include?(config.delivery)
       return true unless config.api_key.to_s.empty?
 
       log.warn_once(:missing_api_key,

data/lib/goodsorcery/configuration/performance_settings.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module GoodSorcery
+  class Configuration
+    # The performance monitoring settings (DESIGN.md 8.1). Monitoring is off
+    # unless set here or with GOODSORCERY_PERFORMANCE_MONITORING=1.
+    module PerformanceSettings
+      def self.included(base)
+        base.attr_accessor :performance_ignore_transactions
+        base.attr_reader :performance_traces
+        base.boolean_setting :performance_traces
+      end
+
+      def performance_monitoring
+        return @performance_monitoring unless @performance_monitoring.nil?
+
+        EnvFlag.parse(ENV.fetch("GOODSORCERY_PERFORMANCE_MONITORING", nil)) || false
+      end
+
+      def performance_monitoring=(value)
+        unless [true, false, nil].include?(value)
+          raise ArgumentError, "performance_monitoring must be true, false or nil (got #{value.inspect})"
+        end
+
+        @performance_monitoring = value
+      end
+
+      private
+
+      def assign_performance_defaults
+        @performance_monitoring = nil
+        @performance_traces = true
+        @performance_ignore_transactions = Defaults::PERFORMANCE_IGNORE_TRANSACTIONS.dup
+      end
+    end
+  end
+end

data/lib/goodsorcery/configuration/validation.rb

@@ -44,6 +44,16 @@ module GoodSorcery
         false
       end
 
+      def boolean_setting(name)
+        define_method(:"#{name}=") do |value|
+          unless [true, false].include?(value)
+            raise ArgumentError, "#{name} must be true or false (got #{value.inspect})"
+          end
+
+          instance_variable_set(:"@#{name}", value)
+        end
+      end
+
       def choice_setting(name, choices)
         define_method(:"#{name}=") do |value|
           normalized = value.is_a?(String) ? value.to_sym : value

data/lib/goodsorcery/configuration.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require_relative "configuration/validation"
+require_relative "configuration/performance_settings"
 
 module GoodSorcery
   # Every setting the gem reads (DESIGN.md 2 and 3). Unset credentials,
@@ -8,18 +9,19 @@ module GoodSorcery
   # environment variables (and Rails) when they are read.
   class Configuration
     extend Validation
+    include PerformanceSettings
 
-    DELIVERIES = %i[async sync active_job test].freeze
+    DELIVERIES = %i[async sync test].freeze
     DELIVERY_SETTINGS = %i[
       delivery api_key endpoint batch_size flush_interval max_queue_size shutdown_timeout
-      max_retries open_timeout read_timeout active_job_queue
+      max_retries open_timeout read_timeout
     ].freeze
 
     attr_writer :api_key, :environment, :release, :logger, :root
     attr_accessor :disabled_environments, :enabled_environments, :filter_params,
                   :use_rails_filter_parameters, :extra_filters, :ignore_exceptions, :tags,
                   :user_method, :send_code_context, :breadcrumbs,
-                  :report_handled_errors, :active_job_queue, :request_extractor,
+                  :report_handled_errors, :request_extractor,
                   :send_request_params, :send_request_headers, :send_session, :send_job_arguments
     attr_reader :delivery, :batch_size, :flush_interval, :max_queue_size,
                 :shutdown_timeout, :max_retries, :open_timeout, :read_timeout,
@@ -119,9 +121,8 @@ module GoodSorcery
     # Never shows the API key (or endpoint credentials): consoles, error
     # pages and loggers print objects with #inspect.
     def inspect
-      delivery_name = delivery.is_a?(Symbol) ? delivery.inspect : delivery.class.name
       "#<#{self.class.name} api_key=#{masked_api_key} endpoint=#{endpoint_label} " \
-        "environment=#{environment} delivery=#{delivery_name}>"
+        "environment=#{environment} delivery=#{delivery.inspect}>"
     end
 
     private
@@ -144,7 +145,6 @@ module GoodSorcery
       @max_retries = 3
       @open_timeout = 2
       @read_timeout = 5
-      @active_job_queue = "default"
     end
 
     def assign_behaviour_defaults
@@ -156,6 +156,7 @@ module GoodSorcery
       @install_check = :verify
       @report_handled_errors = true
       assign_capture_defaults
+      assign_performance_defaults
     end
 
     def assign_capture_defaults
@@ -190,15 +191,28 @@ module GoodSorcery
     end
   end
 
-  # Normalizes and validates config.delivery.
+  # Normalizes and validates config.delivery. The message names a rejected
+  # symbol but only the class of anything else, which may hold credentials.
   module DeliverySetting
+    REMOVED = {
+      active_job: "The :active_job delivery was removed: use :async (the default), " \
+                  "which sends events from a background thread."
+    }.freeze
+    CUSTOM_REMOVED = "Custom delivery objects are no longer supported: use :async (the default), " \
+                     ":sync in serverless functions or :test in your test suite."
+
     def self.normalize(value)
       symbol = value.is_a?(String) ? value.to_sym : value
       return symbol if Configuration::DELIVERIES.include?(symbol)
-      return value if !value.is_a?(Symbol) && value.respond_to?(:deliver)
 
-      raise ArgumentError, "delivery must be one of #{Configuration::DELIVERIES.map(&:inspect).join(", ")} " \
-                           "or an object responding to #deliver (got a #{value.class})"
+      raise ArgumentError, message(symbol, value)
+    end
+
+    def self.message(symbol, value)
+      got = symbol.is_a?(Symbol) ? symbol.inspect : "an instance of #{value.class}"
+      hint = symbol.is_a?(Symbol) ? REMOVED[symbol] : (CUSTOM_REMOVED if value.respond_to?(:deliver))
+      ["delivery must be one of #{Configuration::DELIVERIES.map(&:inspect).join(", ")} (got #{got}).", hint]
+        .compact.join(" ")
     end
   end
 end

data/lib/goodsorcery/defaults.rb

@@ -36,6 +36,10 @@ module GoodSorcery
       SystemExit
     ].freeze
 
+    # Transactions (exact names or regexps) that performance monitoring
+    # never records.
+    PERFORMANCE_IGNORE_TRANSACTIONS = ["Rails::HealthController#show"].freeze
+
     DISABLED_ENVIRONMENTS = %w[development test].freeze
     ENDPOINT = "https://app.goodsorcery.com"
   end

data/lib/goodsorcery/delivery.rb

@@ -6,20 +6,17 @@ require_relative "delivery/worker"
 require_relative "delivery/async"
 require_relative "delivery/sync"
 require_relative "delivery/test"
-require_relative "delivery/custom"
-require_relative "delivery/active_job_queue"
 
 module GoodSorcery
-  # Delivery strategies (config.delivery). Each exposes deliver(event),
+  # Delivery strategies (config.delivery: :async, :sync or :test; the
+  # setter rejects anything else). Each exposes deliver(event),
   # flush(timeout) and shutdown(timeout).
   module Delivery
     def self.build(config, client)
       case config.delivery
-      when :async then async(config, client.transport, client.serializer, client.log)
       when :sync then Sync.new(transport: client.transport, serializer: client.serializer, log: client.log)
       when :test then Test.new
-      when :active_job then active_job(config, client)
-      else Custom.new(config.delivery)
+      else async(config, client.transport, client.serializer, client.log)
       end
     end
 
@@ -29,11 +26,5 @@ module GoodSorcery
                 max_queue_size: config.max_queue_size, max_retries: config.max_retries,
                 shutdown_timeout: config.shutdown_timeout)
     end
-
-    def self.active_job(config, client)
-      ActiveJobQueue.new(serializer: client.serializer, queue_name: config.active_job_queue, log: client.log) do
-        async(config, client.transport, client.serializer, client.log)
-      end
-    end
   end
 end

data/lib/goodsorcery/errors.rb

@@ -7,8 +7,4 @@ module GoodSorcery
   # Raised (and rescued) by `goodsorcery:test` and the install check to send a
   # realistic test exception.
   class TestError < StandardError; end
-
-  # Raised by GoodSorcery::DeliveryJob so ActiveJob retries a delivery that
-  # failed with a 429, a 5xx or a network error.
-  class RetryableDeliveryError < Error; end
 end

data/lib/goodsorcery/integrations/active_job.rb

@@ -25,9 +25,11 @@ module GoodSorcery
         end
 
         def perform_now
-          return super if Integrations.delivery_job?(self)
+          return Integrations.run_unit_of_work(JobDetails.new(self)) { super } unless Performance.active?
 
-          Integrations.run_unit_of_work(JobDetails.new(self)) { super }
+          Performance.measure_job(Performance::ActiveJobTiming.new(self)) do
+            Integrations.run_unit_of_work(JobDetails.new(self)) { super }
+          end
         end
       end
 

data/lib/goodsorcery/integrations/rails/error_subscriber.rb

@@ -12,7 +12,7 @@ module GoodSorcery
 
         def report(error, handled:, severity: nil, context: {}, source: nil)
           return if handled && !GoodSorcery.configuration.report_handled_errors
-          return if ReportedMark.reported?(error) || from_delivery_job?(context)
+          return if ReportedMark.reported?(error)
 
           GoodSorcery.notify(error, level: level(handled, severity),
                                     context: { "rails_error" => rails_error_context(context, source) })
@@ -24,10 +24,6 @@ module GoodSorcery
           handled ? LEVELS.fetch(severity.to_s.to_sym, "warning") : "error"
         end
 
-        def from_delivery_job?(context)
-          context.is_a?(Hash) && Integrations.delivery_job?(context[:job])
-        end
-
         # Controllers and jobs from Rails' execution context become
         # "#<ClassName>"; records become GlobalID strings.
         def rails_error_context(context, source)

data/lib/goodsorcery/integrations/rails.rb

@@ -31,13 +31,14 @@ module GoodSorcery
       end
 
       # After the app's initializers ran: subscribe to Rails.error and to
-      # notifications, hook Sidekiq, and recompile the filters so the final
-      # filter_parameters apply.
+      # notifications, hook Sidekiq, recompile the filters so the final
+      # filter_parameters apply, and start performance monitoring if it is on.
       def after_initialize
         subscribe_to_error_reporter
         NotificationBreadcrumbs.subscribe if GoodSorcery.configuration.breadcrumbs
         install_sidekiq if defined?(::Sidekiq)
         GoodSorcery.client.reconfigure(rebuild_delivery: false)
+        GoodSorcery::Performance.install if GoodSorcery.configuration.performance_monitoring
       end
 
       def subscribe_to_error_reporter

data/lib/goodsorcery/integrations/sidekiq.rb

@@ -30,9 +30,25 @@ module GoodSorcery
         attr_accessor :config
 
         def call(worker, job, queue, &)
-          return yield if Sidekiq.active_job_wrapper?(job)
+          return wrapped(job, &) if Sidekiq.active_job_wrapper?(job)
 
-          Integrations.run_unit_of_work(JobDetails.new(job, queue: queue, worker: worker), &)
+          details = JobDetails.new(job, queue: queue, worker: worker)
+          return Integrations.run_unit_of_work(details, &) unless Performance.active?
+
+          Performance.measure_job(Performance::SidekiqTiming.new(job, queue, worker)) do
+            Integrations.run_unit_of_work(details, &)
+          end
+        end
+
+        private
+
+        # A Sidekiq retry runs an Active Job with its original job data, so
+        # its enqueued_at is the first enqueue; Sidekiq's own enqueued_at is
+        # when the retry was queued, which the queue wait is measured from.
+        def wrapped(job, &)
+          return yield unless Performance.active? && job.key?("retry_count")
+
+          Performance::ActiveJobTiming.requeued_at(job["enqueued_at"], &)
         end
       end
 

data/lib/goodsorcery/integrations.rb

@@ -5,12 +5,6 @@ module GoodSorcery
   # unit of work and leaves an enricher that reads its details only when an
   # error is reported.
   module Integrations
-    # GoodSorcery::DeliveryJob sends events; errors it raises are never
-    # reported (no recursion).
-    def self.delivery_job?(object)
-      GoodSorcery.const_defined?(:DeliveryJob, false) && object.is_a?(GoodSorcery::DeliveryJob)
-    end
-
     # Runs the block in a fresh scope prepared by +details+ (an object with
     # +open(scope)+), reporting any exception that escapes it, then
     # re-raising.

data/lib/goodsorcery/performance/agent.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+module GoodSorcery
+  module Performance
+    # The per-process state: metrics, traces and the flusher. A forked child
+    # gets a fresh, empty agent (the parent keeps and sends what it
+    # recorded), and its flusher thread starts on the child's first record.
+    class Agent
+      attr_reader :pid, :aggregator, :trace_buffer, :flusher, :context
+
+      def initialize(config, log)
+        @pid = Process.pid
+        @aggregator = Aggregator.new
+        @trace_buffer = TraceBuffer.new
+        @context = TraceContext.new(config)
+        @flusher = Flusher.new(agent: self, log: log)
+      end
+
+      def current?
+        @pid == Process.pid
+      end
+
+      def record(recorder, background:)
+        minute = (recorder.finished_wall / 60).floor
+        @aggregator.record(minute, Sample.from(recorder))
+        offer_trace(minute, recorder) if recorder.traces?
+        @flusher.ensure_started if background
+      end
+
+      private
+
+      # A request's trace is built once the response has gone out, when the
+      # server offers a hook for that, so the slowest requests are not kept
+      # waiting for it.
+      def offer_trace(minute, recorder)
+        key = [recorder.kind, recorder.name]
+        return unless @trace_buffer.candidate?(minute, key, recorder.duration_ms)
+
+        hooks = AfterReply.hooks(recorder.env)
+        return build_trace(minute, key, recorder) unless hooks
+
+        hooks << AfterReply.callback { build_trace(minute, key, recorder) }
+      end
+
+      def build_trace(minute, key, recorder)
+        return unless @trace_buffer.candidate?(minute, key, recorder.duration_ms)
+
+        trace = TraceBuilder.new(recorder, minute, @context).build
+        @trace_buffer.offer(minute, key, recorder.duration_ms, trace)
+      end
+    end
+
+    # The Rack server's hooks run after the response is sent: Rack 3's
+    # rack.response_finished (callables get env, status, headers, error)
+    # or Puma's rack.after_reply (no arguments).
+    module AfterReply
+      KEYS = %w[rack.response_finished rack.after_reply].freeze
+
+      def self.hooks(env)
+        return nil unless env.is_a?(Hash)
+
+        KEYS.each do |key|
+          hooks = env[key]
+          return hooks if hooks.is_a?(Array) && !hooks.frozen?
+        end
+        nil
+      end
+
+      # A hook that never raises into the server.
+      def self.callback(&work)
+        proc do |*|
+          work.call
+        rescue StandardError => e
+          GoodSorcery.log.warn_every(:apm_trace_error, 60, "could not build a performance trace (#{e.class})")
+        end
+      end
+    end
+
+    # performance_ignore_transactions: exact names or regexps. Answers are
+    # cached per name.
+    class IgnoreMatcher
+      MAX_CACHED = 1_000
+
+      def initialize(list)
+        entries = Array(list)
+        @names = entries.grep_v(Regexp).map(&:to_s).to_h { |name| [name, true] }
+        @patterns = entries.grep(Regexp)
+        @cache = {}
+      end
+
+      def ignored?(name)
+        return true if @names.key?(name)
+        return false if @patterns.empty?
+
+        cached = @cache[name]
+        return cached unless cached.nil?
+
+        @cache.clear if @cache.size >= MAX_CACHED
+        @cache[name] = @patterns.any? { |pattern| pattern.match?(name) }
+      end
+    end
+  end
+end