dead_bro 0.2.30 → 0.2.32

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cccae4ac857f38e889e12c292943b25fad11f7e372d7f13e4eb838d13c6330ee
-  data.tar.gz: 4d459c4d049219747ed5c1b6f10e1417648282eca2e3e81c7eeef132e8a347bd
+  metadata.gz: 22b021f41cc62097a8fc1f1aedf489e8ee1dca10b2b2abc6edbce6809763b39a
+  data.tar.gz: 30c442462399195cec1edb2f9505aa51896230da92d18ad97f5eca35d2749221
 SHA512:
-  metadata.gz: d12c33c0902c95f318cd716d7c88c78925543b235729ee90525d0453b16ed46521e2b00cc1daa5131b673abb7ee7e74e0968f299d56da055fd36452c2e314b90
-  data.tar.gz: e04ca78806e4d9a07757c0fedc177c2e4b77b295437d415fab94546d84add717ceab84791e34991f4955197ab1d8036ad498d679b089e795e5356beb0662d286
+  metadata.gz: 8d0cbef4fcc8b05de9e8a4c1c357352a17cd133b372ab0239b2402b4d8e7322728a881f0679cbfe0df812a0c3e4143e78d35279a7780e525810efc438b6d5d90
+  data.tar.gz: 81bd8d1e49b663c5ae1d651e26abcaa2eb3dcc10f7bf71155d79da9a59d03aafaee9dc282670fd376b9b99991a3074763ed2abf5461f405022d26f86fe4ecc23

data/CHANGELOG.md

@@ -1,5 +1,18 @@
 ## [Unreleased]
 
+## [0.2.32] - 2026-09-22
+
+### Fixed
+- **Errored requests now ship the SQL/cache/memory/etc data captured before the exception, not just the exception itself.** Previously, as soon as a request raised (e.g. an `ActiveRecord::LockWaitTimeout` mid-transaction), the subscriber discarded every already-captured `sql_queries`, `cache_events`, `ar_instantiation_count`, `memory_events`, `gc_stats`, etc. entry and sent only exception metadata — so error pages showed a backtrace with zero query context even when the gem had captured it. The error payload now carries the same detail fields a successful request would.
+- **Background job failures were reported to the dashboard as successful completions, not dropped.** There is no `exception.active_job` event anywhere in Rails/ActiveJob — the handler that subscribed to it never fired. `perform.active_job` (the real event) fires whether or not the job raised, but its handler built `status: "completed"` unconditionally and never inspected `data[:exception_object]`, so a failing job wasn't invisible — it was recorded as if it had succeeded. This silently skewed job success-rate and duration numbers for every account on a version prior to this one; there's nothing to backfill, since no exception data was ever sent for those occurrences. The handler now branches on `data[:exception_object]`: a failing job reports `status: "failed"` with `error`, `fingerprint`, and `cause_chain`, and ships regardless of sampling.
+  - **Known limitation:** this only covers a job whose exception escapes `perform_now` uncaught. A job using `retry_on`, `discard_on`, or a custom `rescue_from` has the exception handled *inside* `perform_now`, before `perform.active_job`'s payload is ever built — so a retried or discarded job still reports `status: "completed"` for every handled attempt. Only the final attempt of a `retry_on` job that exhausts its retries with no block given re-raises far enough to be visible. Covering retried/discarded attempts would need separate instrumentation on `enqueue_retry.active_job` / `retry_stopped.active_job` / `discard.active_job` — not done here.
+- **A request opening 5 or more transactions/savepoints could trip a false N+1 flag.** Every Rails adapter checked (MySQL/PostgreSQL/SQLite3, 7.1–8.1) logs transaction-control statements under the single name `"TRANSACTION"`, which the old skip list (`data[:name] == "BEGIN" || ...`) never matched — so `BEGIN`/`COMMIT`/etc. weren't ignored, they were tracked as ordinary queries: sanitized, counted toward N+1 detection, and folded into one aggregate per normalized statement. Five or more transactions in one request hit `N_PLUS_ONE_THRESHOLD` and flagged that aggregate `n_plus_one: true`, which the backend trusts directly for the aggregate payload format. Fixed as a consequence of the `TXN_CONTROL_NAMES` match below, not a separate change.
+
+### Added
+- Transaction-control statements (`BEGIN`/`COMMIT`/`ROLLBACK`/`SAVEPOINT`/`RELEASE`) are now captured as lightweight breadcrumbs (`transaction_events`, with offsets) instead of being tracked as ordinary queries — so a transaction that rolled back before an error shows up explicitly in the request trace, instead of as an anonymous, sanitized `sql_queries` entry.
+
+## [0.2.31] - 2026-08-30
+
 ### Added
 - Monitor thread now sends a synchronous heartbeat on startup before the first collection tick. This ensures remote settings — including `monitor_enabled` — are applied from the very first reporting cycle, so Sidekiq workers and other non-web processes that have not yet sent any metrics still receive the correct configuration immediately on boot rather than waiting up to 60 seconds for the first scheduled tick.
 

data/README.md

@@ -65,115 +65,9 @@ Use the DeadBro UI to turn features on or off, set sample rates, define controll
 
 You can still set `config.enabled` in Ruby if you need to force the integration off in a given environment before any remote settings arrive; otherwise the dashboard can control `enabled` like other remote settings.
 
-## Optional: configuration in Ruby
+## Request Sampling, Exclusions & Whitelisting
 
-The sections below describe the same knobs you can manage in the DeadBro app. Use them only when you want values in source control, per-environment initializer logic, or other overrides outside the UI.
-
-## Request Sampling
-
-DeadBro supports configurable request sampling to reduce the volume of metrics sent to your APM endpoint, which is useful for high-traffic applications. Prefer setting this in the DeadBro app; use Ruby if you need a local override.
-
-### Configuration
-
-Set the sample rate as a percentage (1-100):
-
-```ruby
-# Track 50% of requests
-DeadBro.configure do |config|
-  config.sample_rate = 50
-end
-
-# Track 10% of requests (useful for high-traffic apps)
-DeadBro.configure do |config|
-  config.sample_rate = 10
-end
-
-# Track all requests (default)
-DeadBro.configure do |config|
-  config.sample_rate = 100
-end
-```
-
-### How It Works
-
-- **Random Sampling**: Each request has a random chance of being tracked based on the sample rate
-- **Consistent Per-Request**: The sampling decision is made once per request and applies to all metrics for that request
-- **Debug Logging**: Skipped requests do not count towards the montly limit
-- **Error Tracking**: Errors are still tracked regardless of sampling
-
-### Use Cases
-
-- **High-Traffic Applications**: Reduce APM data volume and costs
-- **Development/Staging**: Sample fewer requests to reduce noise
-- **Performance Testing**: Track a subset of requests during load testing
-- **Cost Optimization**: Balance monitoring coverage with data costs
-
-
-## Excluding Controllers and Jobs
-
-You can exclude specific controllers and jobs from APM tracking (dashboard first; Ruby optional).
-
-### Configuration
-
-
-```ruby
-DeadBro.configure do |config|
-  # Controller-only or controller#action patterns in one list (wildcards supported)
-  config.excluded_controllers = [
-    "HealthChecksController",
-    "Admin::*",
-    "UsersController#show",
-    "Admin::ReportsController#index",
-    "Admin::*#*"
-  ]
-
-  config.excluded_jobs = [
-    "ActiveStorage::AnalyzeJob",
-    "Admin::*"
-  ]
-end
-```
-
-Notes:
-- Wildcards `*` are supported (e.g., `Admin::*`, `Admin::*#*`).
-- Matching uses full names like `UsersController`, `Admin::ReportsController#index`, `MyJob`.
-
-## Exclusive Tracking (Whitelist Mode)
-
-You can configure DeadBro to **only** track specific controllers, actions, or jobs. Prefer the dashboard; use Ruby for overrides.
-
-### Configuration
-
-```ruby
-DeadBro.configure do |config|
-  # Only track these controllers/actions (patterns can include #action or wildcards)
-  config.exclusive_controllers = [
-    "UsersController#show",
-    "UsersController#index",
-    "Admin::ReportsController#*",
-    "Api::*#*"
-  ]
-
-  config.exclusive_jobs = [
-    "PaymentProcessingJob",
-    "EmailDeliveryJob",
-    "Admin::*"
-  ]
-end
-```
-
-### How It Works
-
-- **If `exclusive_controllers` or `exclusive_jobs` is empty/not defined**: All controllers/actions/jobs are tracked (default behavior)
-- **If `exclusive_controllers` or `exclusive_jobs` is defined with values**: Only matching controllers/actions/jobs are tracked
-- **Exclusion takes precedence**: If something matches both `excluded_*` and `exclusive_*`, it is excluded (exclusion is checked first)
-
-### Use Cases
-
-- **Focus on Critical Paths**: Monitor only your most important endpoints
-- **Cost Optimization**: Track only specific high-value operations
-- **Debugging**: Temporarily focus on specific controllers/jobs during investigation
-- **Compliance**: Track only operations that require monitoring for compliance reasons
+To control data volume, DeadBro supports request sampling (track a percentage of requests), excluding specific controllers/jobs from tracking, and whitelisting (tracking *only* specific controllers/jobs). All of this is configured in the DeadBro dashboard — sample rate, `excluded_controllers`/`excluded_jobs`, and `exclusive_controllers`/`exclusive_jobs` patterns (wildcards like `Admin::*` and `Admin::*#*` supported). Exclusion always takes precedence over whitelisting when a pattern matches both.
 
 ## SQL Query Tracking
 
@@ -198,43 +92,19 @@ DeadBro can automatically capture the query plan (`EXPLAIN`) of slow SELECT quer
 - **Value scrubbing**: quoted string literals echoed in plan text (e.g. `Filter: email = 'a@b.com'`) are replaced with `?` before the plan leaves your app.
 - **Bounded overhead**: plans are captured on background threads (max 3 concurrent per request); at request end DeadBro waits at most 0.5s for stragglers, then drops them.
 
-### How It Works
-
-- **Automatic Detection**: when a SELECT exceeds `slow_query_threshold_ms`, DeadBro captures its plan in the background
-- **Database Support**: PostgreSQL, MySQL, SQLite (`EXPLAIN QUERY PLAN`), and any adapter with a standard `EXPLAIN`
-
 ### Configuration
 
-- **`explain_analyze_enabled`** (default: `false`) - local opt-in for automatic EXPLAIN plan capture. Must be set in Ruby; the DeadBro UI toggle can only turn capture off, not on.
-- **`slow_query_threshold_ms`** (default: `500`) - queries taking longer than this threshold will have their execution plan captured
+- **`explain_analyze_enabled`** (default: `false`) — this is the one setting that must be turned on in Ruby; the dashboard toggle can only turn capture *off*, never on, as an extra safety rail:
 
-### Example Configuration
-
-```ruby
-DeadBro.configure do |config|
-  # Capture EXPLAIN plans for SELECTs slower than 500ms
-  config.explain_analyze_enabled = true
-  config.slow_query_threshold_ms = 500
-
-  # Or use a higher threshold for production
-  # config.slow_query_threshold_ms = 1000  # Only explain queries > 1 second
-end
-```
+  ```ruby
+  DeadBro.configure do |config|
+    config.explain_analyze_enabled = true
+  end
+  ```
 
-### What You Get
+- Everything else — the `slow_query_threshold_ms` cutoff and enabling/disabling the feature — is managed from the dashboard.
 
-When a slow query is detected, the `explain_plan` field in the SQL query data will contain:
-- **PostgreSQL / MySQL**: `EXPLAIN` output — the planner's chosen strategy, estimated costs and row counts
-- **SQLite**: `EXPLAIN QUERY PLAN` output
-- **Other databases**: standard `EXPLAIN` output
-
-This execution plan helps you:
-- Identify missing indexes
-- Understand query execution order
-- Spot full table scans
-- Optimize JOIN operations
-
-Because the statement is not executed, plans show the planner's *estimates* rather than actual runtimes — usually exactly what you need to spot a missing index or an unexpected sequential scan.
+When a slow query is detected, the `explain_plan` field in the SQL query data contains the database's `EXPLAIN` (or `EXPLAIN QUERY PLAN` on SQLite) output — the planner's chosen strategy, estimated costs and row counts — useful for spotting missing indexes or unexpected sequential scans. Because the statement is never executed, plans show estimates rather than actual runtimes.
 
 ## View Rendering Tracking
 
@@ -259,20 +129,7 @@ By default, DeadBro uses **lightweight memory tracking** that has minimal perfor
 - **GC Efficiency Analysis**: Monitor garbage collection effectiveness
 - **Zero Allocation Tracking**: No object allocation tracking by default (can be enabled)
 
-### Configuration Options
-
-Usually managed in the dashboard; Ruby example:
-
-```ruby
-DeadBro.configure do |config|
-  config.memory_tracking_enabled = true        # Enable lightweight memory tracking (default: true)
-  config.allocation_tracking_enabled = false   # Enable detailed allocation tracking (default: false)
-  config.allocation_sample_rate = 100          # % of requests that pay for allocation tracking when enabled (1-100, default: 100)
-
-  # Sampling configuration
-  config.sample_rate = 100                     # Percentage of requests to track (1-100, default: 100)
-end
-```
+`memory_tracking_enabled`, `allocation_tracking_enabled`, and `allocation_sample_rate` are all managed from the dashboard.
 
 **Performance Impact:**
 - **Lightweight mode** (`memory_tracking_enabled`, ~0.1ms overhead per request): RSS before/after, GC pressure, the retained-vs-transient signals (`heap_live_slots_growth`, `malloc_increase_bytes`), and per-phase allocation attribution (`allocation_phases` — which of `sql`/`view`/`elasticsearch` allocated the request's objects).
@@ -310,25 +167,7 @@ Everything is **best effort** and designed to be **safe and low overhead**:
 
 ### Configuration
 
-Enable or disable collectors in the DeadBro app, or use `DeadBro.configure` for code-based overrides:
-
-```ruby
-DeadBro.configure do |config|
-  # Enable the periodic job queue monitor (disabled by default)
-  config.job_queue_monitoring_enabled = true
-
-  # Enable best-effort collectors (all default to false)
-  config.enable_db_stats      = true   # ActiveRecord pool + ping latency
-  config.enable_process_stats = true   # pid, hostname, RSS, GC, threads, fds
-  config.enable_system_stats  = true   # CPU%, memory, disk, network
-
-  # Filesystem paths to report disk usage for (default: ["/"])
-  config.disk_paths = ["/", "/var"]
-
-  # Network interfaces to ignore when computing rx/tx stats
-  config.interfaces_ignore = %w[lo docker0]
-end
-```
+Enable or disable the job queue monitor and the individual collectors (`enable_db_stats`, `enable_process_stats`, `enable_system_stats`), plus `disk_paths` and `interfaces_ignore`, from the DeadBro dashboard.
 
 ### Example Payload Shape
 

data/lib/dead_bro/job_subscriber.rb

@@ -9,7 +9,6 @@ end
 module DeadBro
   class JobSubscriber
     JOB_EVENT_NAME = "perform.active_job"
-    JOB_EXCEPTION_EVENT_NAME = "exception.active_job"
 
     def self.subscribe!(client: Client.new)
       # Snap GC state before the job runs so stop_request_tracking gets a valid diff
@@ -19,7 +18,28 @@ module DeadBro
       rescue
       end
 
-      # Track job execution
+      # Track job execution — success AND failure both land here. ActiveJob wraps
+      # perform with `instrument(:perform) { super }` (see
+      # ActiveJob::Instrumentation#instrument), and AS::Notifications.instrument
+      # still runs its "finish" listeners (with :exception / :exception_object set
+      # in the payload) when the block raises, then re-raises. There is no separate
+      # "exception.active_job" event anywhere in Rails/ActiveJob — a prior version
+      # of this file subscribed to one, so it never fired; the job wasn't dropped,
+      # it fired *this* event and built status: "completed" unconditionally,
+      # silently reporting every failing job as a success. Branching on
+      # data[:exception_object] here is the only place a job failure can actually
+      # be detected.
+      #
+      # Known gap: this only sees an exception that escapes perform_now uncaught.
+      # ActiveJob::Base#perform_now (Execution) rescues internally and hands off to
+      # rescue_with_handler — which is exactly what retry_on/discard_on/rescue_from
+      # are built on (ActiveJob::Exceptions) — before Instrumentation's `super`
+      # even returns. A handled retry or discard returns normally with no
+      # exception attached, so perform.active_job still reports status:
+      # "completed" for every handled attempt; only a retry_on job's final,
+      # unhandled raise (attempts exhausted, no block given) is visible here.
+      # Covering handled attempts would need separate instrumentation on
+      # enqueue_retry.active_job / retry_stopped.active_job / discard.active_job.
       ActiveSupport::Notifications.subscribe(JOB_EVENT_NAME) do |name, started, finished, _unique_id, data|
         begin
           if DeadBro.configuration.skip_tracking?
@@ -40,12 +60,14 @@ module DeadBro
         rescue
         end
 
+        exception = data[:exception_object]
+        has_error = !exception.nil?
+
         # Skip out via sampling before we build any payload — jobs can be chatty
         # enough that even the "cheap" stop/analyze work matters under load.
-        # Completions have no exception attached; the exception subscriber below
-        # always sends errors with force: true.
+        # Errors always ship regardless of sampling, matching Subscriber's web path.
         job_type_key = "#{job_class_name}#perform"
-        unless DeadBro.configuration.should_sample?(job_type_key)
+        unless has_error || DeadBro.configuration.should_sample?(job_type_key)
           drain_job_tracking
           next
         end
@@ -71,6 +93,7 @@ module DeadBro
 
         # Get SQL queries executed during this job
         sql_queries = DeadBro::SqlSubscriber.stop_request_tracking
+        transaction_events = DeadBro::SqlSubscriber.last_transaction_events
         dependency_events = job_dependency_payload
         db_connection_stats = defined?(DeadBro::DbConnectionSubscriber) ? DeadBro::DbConnectionSubscriber.stop_request_tracking : {}
         gc_pressure = defined?(DeadBro::GcTracker) ? DeadBro::GcTracker.stop_request_tracking : {}
@@ -117,8 +140,9 @@ module DeadBro
           db_connection_checkouts: db_connection_stats[:checkouts],
           gc_pressure: gc_pressure,
           ar_instantiation_count: ar_instantiation_count,
-          status: "completed",
+          status: has_error ? "failed" : "completed",
           sql_queries: sql_queries,
+          transaction_events: transaction_events,
           rails_env: DeadBro.env,
           host: DeadBro.safe_hostname,
           process_kind: DeadBro.process_kind,
@@ -130,115 +154,19 @@ module DeadBro
           logs: DeadBro.logger.logs
         }.merge(dependency_events)
 
-        # force: true — the sampling decision above already accounted for any
-        # per-job-type override; client#post_metric must not re-roll it globally.
-        client.post_metric(event_name: name, payload: payload, force: true)
-      end
-
-      # Track job exceptions
-      ActiveSupport::Notifications.subscribe(JOB_EXCEPTION_EVENT_NAME) do |name, started, finished, _unique_id, data|
-        begin
-          if DeadBro.configuration.skip_tracking?
-            drain_job_tracking
-            next
-          end
-
-          job_class_name = data[:job].class.name
-          if DeadBro.configuration.excluded_job?(job_class_name)
-            next
-          end
-          # If exclusive_jobs is defined and not empty, only track matching jobs
-          unless DeadBro.configuration.exclusive_job?(job_class_name)
-            next
-          end
-        rescue
-        end
-
-        duration_ms = ((finished - started) * 1000.0).round(2)
-        exception = data[:exception_object]
-        queue_duration_ms = job_queue_duration_ms(data[:job], started)
-
-        # Ensure tracking was started (fallback if perform_start.active_job didn't fire)
-        unless DeadBro::SqlSubscriber.tracking_active?
-          DeadBro.logger.clear
-          Thread.current[DeadBro::TRACKING_START_TIME_KEY] = Time.now
-          DeadBro::SqlSubscriber.start_request_tracking
-          start_job_dependency_tracking
-          DeadBro::DbConnectionSubscriber.start_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
-          DeadBro::WatchTracker.start_request_tracking if defined?(DeadBro::WatchTracker)
-          if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
-            DeadBro::MemoryTrackingSubscriber.start_request_tracking
-          else
-            DeadBro::LightweightMemoryTracker.start_request_tracking if defined?(DeadBro::LightweightMemoryTracker)
-          end
-        end
-
-        # Get SQL queries executed during this job
-        sql_queries = DeadBro::SqlSubscriber.stop_request_tracking
-        dependency_events = job_dependency_payload
-        db_connection_stats = defined?(DeadBro::DbConnectionSubscriber) ? DeadBro::DbConnectionSubscriber.stop_request_tracking : {}
-        gc_pressure = defined?(DeadBro::GcTracker) ? DeadBro::GcTracker.stop_request_tracking : {}
-        ar_instantiation_count = defined?(DeadBro::ArObjectTracker) ? DeadBro::ArObjectTracker.stop_request_tracking : nil
-        watch_events = defined?(DeadBro::WatchTracker) ? DeadBro::WatchTracker.stop_request_tracking : []
-
-        # Stop memory tracking and get collected memory data
-        if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
-          detailed_memory = DeadBro::MemoryTrackingSubscriber.stop_request_tracking
-          memory_performance = DeadBro::MemoryTrackingSubscriber.analyze_memory_performance(detailed_memory)
-          # Keep memory_events compact and user-friendly (no large raw arrays)
-          memory_events = {
-            memory_before: detailed_memory[:memory_before],
-            memory_after: detailed_memory[:memory_after],
-            duration_seconds: detailed_memory[:duration_seconds],
-            allocations_count: (detailed_memory[:allocations] || []).length,
-            memory_snapshots_count: (detailed_memory[:memory_snapshots] || []).length,
-            large_objects_count: (detailed_memory[:large_objects] || []).length
-          }
-        else
-          lightweight_memory = DeadBro::LightweightMemoryTracker.stop_request_tracking
-          # Separate raw readings from derived performance metrics to avoid duplicating data
-          memory_events = {
-            memory_before: lightweight_memory[:memory_before],
-            memory_after: lightweight_memory[:memory_after]
-          }
-          memory_performance = {
-            memory_growth_mb: lightweight_memory[:memory_growth_mb],
-            gc_count_increase: lightweight_memory[:gc_count_increase],
-            heap_pages_increase: lightweight_memory[:heap_pages_increase],
-            duration_seconds: lightweight_memory[:duration_seconds]
-          }
+        if has_error
+          payload[:exception_class] = exception.class.name
+          payload[:message] = exception.message.to_s[0, 1000]
+          payload[:backtrace] = Array(exception.backtrace).first(50)
+          payload[:fingerprint] = DeadBro::Subscriber.compute_error_fingerprint(exception)
+          payload[:cause_chain] = DeadBro::Subscriber.build_cause_chain(exception)
+          payload[:error] = true
         end
 
-        payload = {
-          job_class: data[:job].class.name,
-          job_id: data[:job].job_id,
-          queue_name: data[:job].queue_name,
-          arguments: safe_arguments(data[:job].arguments),
-          started_at: started.utc.iso8601(3),
-          duration_ms: duration_ms,
-          queue_duration_ms: queue_duration_ms,
-          db_connection_wait_ms: db_connection_stats[:wait_ms],
-          db_connection_checkouts: db_connection_stats[:checkouts],
-          gc_pressure: gc_pressure,
-          ar_instantiation_count: ar_instantiation_count,
-          status: "failed",
-          sql_queries: sql_queries,
-          exception_class: exception&.class&.name,
-          message: exception&.message&.to_s&.[](0, 1000),
-          backtrace: Array(exception&.backtrace).first(50),
-          rails_env: DeadBro.env,
-          host: DeadBro.safe_hostname,
-          process_kind: DeadBro.process_kind,
-          memory_usage: memory_usage_mb,
-          gc_stats: gc_stats,
-          memory_events: memory_events,
-          memory_performance: memory_performance,
-          watch_events: watch_events,
-          logs: DeadBro.logger.logs
-        }.merge(dependency_events)
-
-        event_name = exception&.class&.name || "ActiveJob::Exception"
-        client.post_metric(event_name: event_name, payload: payload, force: true)
+        # force: true — errors always ship, bypassing sampling by design; for
+        # completions the sampling decision above already accounted for any
+        # per-job-type override, so client#post_metric must not re-roll it globally.
+        client.post_metric(event_name: name, payload: payload, force: true)
       end
     rescue
       # Never raise from instrumentation install
@@ -248,6 +176,7 @@ module DeadBro
     # build a payload (excluded job / sampled out). Matches Subscriber.drain_request_tracking.
     def self.drain_job_tracking
       # wait_for_explains: false — result is discarded, don't block on pending plans.
+      # stop_request_tracking also pops the transaction-events stack (see its comment).
       DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
       Thread.current[:dead_bro_http_events] = nil
       DeadBro::CacheSubscriber.stop_request_tracking if defined?(DeadBro::CacheSubscriber)

data/lib/dead_bro/sql_subscriber.rb

@@ -16,7 +16,11 @@ module DeadBro
     THREAD_LOCAL_EXPLAIN_PENDING_KEY = :dead_bro_explain_pending
     THREAD_LOCAL_CALL_COUNTS_KEY     = :dead_bro_sql_call_counts
     THREAD_LOCAL_AGGREGATES_KEY      = :dead_bro_sql_aggregates
+    THREAD_LOCAL_TXN_EVENTS_KEY      = :dead_bro_sql_txn_events
     MAX_TRACKED_QUERIES = 1000
+    # Transaction control breadcrumbs (BEGIN/COMMIT/ROLLBACK/...) are cheap and rare
+    # compared to queries, but a pathological retry loop could still spam them.
+    MAX_TRACKED_TXN_EVENTS = 200
 
     # Number of identical queries within one request that triggers N+1 detection.
     N_PLUS_ONE_THRESHOLD = 5
@@ -37,6 +41,43 @@ module DeadBro
     SANITIZE_SKIP_SENSITIVE_WHEN_NO_KEYWORDS = /password|token|secret|key|ssn|credit_card/i
     SANITIZE_SKIP_WHERE_WHEN_NO_KEYWORD = /WHERE/i
 
+    # Rails' own adapters (MySQL, PostgreSQL, SQLite3, at least 7.1 through 8.1 —
+    # the versions checked directly in this repo) log every transaction-control
+    # statement under the single generic name "TRANSACTION"
+    # (`internal_execute("BEGIN", "TRANSACTION", ...)`, `internal_execute("COMMIT",
+    # "TRANSACTION", ...)`, etc. — see AbstractMysqlAdapter/PostgreSQL::DatabaseStatements/
+    # SQLite3::DatabaseStatements). "BEGIN"/"COMMIT"/"ROLLBACK"/"SAVEPOINT"/"RELEASE"
+    # as literal `name` values are kept here only as a defensive fallback for
+    # adapters/older Rails versions that might still emit them directly — the actual
+    # operation for a "TRANSACTION"-named event is derived from the SQL text itself
+    # (see transaction_operation_for below), since the name alone can't distinguish
+    # BEGIN from COMMIT from a SAVEPOINT.
+    TXN_CONTROL_NAMES = %w[BEGIN COMMIT ROLLBACK SAVEPOINT RELEASE TRANSACTION].freeze
+
+    TXN_OPERATION_FROM_SQL = [
+      [/\A\s*ROLLBACK\s+TO\s+SAVEPOINT/i, "ROLLBACK TO SAVEPOINT"],
+      [/\A\s*RELEASE\s+SAVEPOINT/i, "RELEASE SAVEPOINT"],
+      [/\A\s*SAVEPOINT/i, "SAVEPOINT"],
+      [/\A\s*BEGIN/i, "BEGIN"],
+      [/\A\s*COMMIT/i, "COMMIT"],
+      [/\A\s*ROLLBACK/i, "ROLLBACK"]
+    ].freeze
+
+    # data[:name] is only ever "TRANSACTION" in current Rails — this derives the
+    # actual verb (BEGIN/COMMIT/ROLLBACK/SAVEPOINT/RELEASE SAVEPOINT/ROLLBACK TO
+    # SAVEPOINT) from the SQL text so breadcrumbs are still meaningful. A literal
+    # non-"TRANSACTION" name (older Rails/adapter) is passed through unchanged.
+    def self.transaction_operation_for(name, sql)
+      return name unless name == "TRANSACTION"
+      sql_str = sql.to_s
+      TXN_OPERATION_FROM_SQL.each do |re, op|
+        return op if sql_str.match?(re)
+      end
+      name
+    rescue
+      name
+    end
+
     # True when there is at least one active tracking context (e.g. for nested jobs).
     def self.tracking_active?
       stack = Thread.current[THREAD_LOCAL_KEY]
@@ -50,6 +91,13 @@ module DeadBro
       stack.last
     end
 
+    # Current transaction-events array (top of stack); nil if no active tracking.
+    def self.current_txn_events_array
+      stack = Thread.current[THREAD_LOCAL_TXN_EVENTS_KEY]
+      return nil unless stack.is_a?(Array) && stack.any?
+      stack.last
+    end
+
     # Sum of SQL counts and durations recorded so far in the active tracking
     # context. Used by WatchTracker to attribute SQL to a DeadBro.watch block.
     def self.current_sql_metrics
@@ -96,6 +144,27 @@ module DeadBro
       sql.to_s.downcase
     end
 
+    # Records a lightweight breadcrumb for a transaction-control statement
+    # (BEGIN/COMMIT/ROLLBACK/SAVEPOINT/RELEASE) — enough to show, e.g., that the
+    # queries preceding an error ran inside a transaction that was rolled back,
+    # without the cost of treating it like a tracked query (no backtrace/EXPLAIN).
+    def self.record_transaction_event(name, started, finished)
+      current = current_txn_events_array
+      return unless current
+      return if current.length >= MAX_TRACKED_TXN_EVENTS
+
+      tracking_start = Thread.current[DeadBro::TRACKING_START_TIME_KEY]
+      start_offset_ms = tracking_start ? ((started - tracking_start) * 1000.0).round(2) : nil
+
+      current << {
+        event: name,
+        duration_ms: ((finished - started) * 1000.0).round(2),
+        start_offset_ms: start_offset_ms
+      }
+    rescue
+      nil
+    end
+
     def self.subscribe!
       # Subscribe with a start/finish listener to measure allocations per query
       if ActiveSupport::Notifications.notifier.respond_to?(:subscribe)
@@ -106,7 +175,13 @@ module DeadBro
       end
 
       ActiveSupport::Notifications.subscribe(SQL_EVENT_NAME) do |name, started, finished, _unique_id, data|
-        next if data[:name] == "SCHEMA" || data[:name] == "CACHE" || data[:name] == "BEGIN" || data[:name] == "COMMIT" || data[:name] == "ROLLBACK" || data[:name] == "SAVEPOINT" || data[:name] == "RELEASE"
+        next if data[:name] == "SCHEMA" || data[:name] == "CACHE"
+
+        if TXN_CONTROL_NAMES.include?(data[:name])
+          record_transaction_event(transaction_operation_for(data[:name], data[:sql]), started, finished)
+          next
+        end
+
         # Only track queries that are part of the current request (top of stack for nested jobs)
         current = current_queries_array
         next unless current
@@ -208,6 +283,7 @@ module DeadBro
       Thread.current[THREAD_LOCAL_EXPLAIN_PENDING_KEY] = []
       (Thread.current[THREAD_LOCAL_CALL_COUNTS_KEY] ||= []) << {}
       (Thread.current[THREAD_LOCAL_AGGREGATES_KEY]  ||= []) << {}
+      (Thread.current[THREAD_LOCAL_TXN_EVENTS_KEY]  ||= []) << []
     end
 
     # wait_for_explains: false on drain paths (excluded / sampled-out requests)
@@ -223,6 +299,21 @@ module DeadBro
       cc_stack     = Thread.current[THREAD_LOCAL_CALL_COUNTS_KEY]
       cc_stack.pop if cc_stack.is_a?(Array) && cc_stack.any?
 
+      # Transaction-control breadcrumbs are popped here, as part of this method's
+      # existing lifecycle, rather than through a separate stop_transaction_tracking
+      # call. start_request_tracking has many callers across this gem (and its own
+      # specs) that only know about query tracking; requiring every one of them to
+      # also remember a second, separately-paired stop call is exactly how a frame
+      # gets pushed and never popped — leaking across every later request/job on
+      # the same thread. Piggybacking on stop_request_tracking, which is already
+      # reliably paired with every start_request_tracking call, means there is
+      # nothing new for any caller to remember. A caller that wants the events
+      # reads last_transaction_events immediately afterward.
+      txn_stack = Thread.current[THREAD_LOCAL_TXN_EVENTS_KEY]
+      Thread.current[:dead_bro_last_transaction_events] =
+        (txn_stack.is_a?(Array) && txn_stack.any?) ? txn_stack.pop : []
+      Thread.current[THREAD_LOCAL_TXN_EVENTS_KEY] = nil if txn_stack.nil? || txn_stack.empty?
+
       # Fold any completed EXPLAIN plans from raw queries into their aggregate entry
       raw_queries.each do |q|
         next unless q[:explain_plan]
@@ -244,6 +335,17 @@ module DeadBro
       aggregates_h.values.sort_by { |a| -a[:total_duration_ms] }
     end
 
+    # The transaction-control breadcrumbs captured during the tracking window that
+    # this thread's most recent stop_request_tracking call just ended. Consumes
+    # (clears) the thread-local on read, not just on write: otherwise it would sit
+    # pinned in Thread.current between requests (a Puma/Sidekiq thread is reused),
+    # and — worse — any caller that reads it without an immediately-preceding
+    # stop_request_tracking would silently get the *previous* request's
+    # breadcrumbs instead of an empty result.
+    def self.last_transaction_events
+      Thread.current[:dead_bro_last_transaction_events].tap { Thread.current[:dead_bro_last_transaction_events] = nil } || []
+    end
+
     # Upper bound on pending EXPLAIN threads per request. Each thread checks
     # out an AR pool connection, so this must stay well below common pool
     # sizes (Rails default is 5) to avoid starving the app under a storm.

data/lib/dead_bro/sql_tracking_middleware.rb

@@ -94,6 +94,7 @@ module DeadBro
       if defined?(DeadBro::SqlSubscriber)
         Thread.current[:dead_bro_sql_queries]
         Thread.current[:dead_bro_sql_queries] = nil
+        Thread.current[DeadBro::SqlSubscriber::THREAD_LOCAL_TXN_EVENTS_KEY] = nil
       end
 
       if defined?(DeadBro::CacheSubscriber)

data/lib/dead_bro/subscriber.rb

@@ -58,6 +58,7 @@ module DeadBro
 
         # Stop SQL tracking and get collected queries (this was started by the request)
         sql_queries = DeadBro::SqlSubscriber.stop_request_tracking
+        transaction_events = DeadBro::SqlSubscriber.last_transaction_events
 
         # Stop cache, redis, and elasticsearch tracking
         cache_events = defined?(DeadBro::CacheSubscriber) ? DeadBro::CacheSubscriber.stop_request_tracking : []
@@ -147,6 +148,42 @@ module DeadBro
           })
         end
 
+        # Everything captured during the request regardless of outcome — shared between
+        # the error and success payloads below so an errored request (e.g. a lock wait
+        # timeout raised mid-transaction) still ships the SQL/cache/memory/etc data that
+        # led up to it, not just the exception. This used to be built only for the
+        # success path, so every errored request shipped with sql_count: 0 and no trace
+        # even though the gem had already captured it.
+        detail_fields = {
+          view_runtime_ms: data[:view_runtime],
+          db_runtime_ms: data[:db_runtime],
+          memory_usage: memory_usage_mb,
+          gc_stats: gc_stats,
+          sql_count: sql_count(data),
+          sql_queries: sql_queries,
+          transaction_events: transaction_events,
+          http_outgoing: Thread.current[:dead_bro_http_events] || [],
+          cache_events: cache_events,
+          redis_events: redis_events,
+          elasticsearch_events: elasticsearch_events,
+          cache_hits: cache_hits(data),
+          cache_misses: cache_misses(data),
+          view_events: view_events,
+          view_performance: view_performance,
+          memory_events: memory_events,
+          memory_performance: memory_performance,
+          allocation_phases: allocation_phases,
+          rack_duration_ms: rack_duration_ms,
+          queue_duration_ms: Thread.current[:dead_bro_queue_duration_ms],
+          db_connection_wait_ms: db_connection_stats[:wait_ms],
+          db_connection_checkouts: db_connection_stats[:checkouts],
+          gc_pressure: gc_pressure,
+          ar_instantiation_count: ar_instantiation_count,
+          cpu_time_ms: cpu_time_ms,
+          watch_events: watch_events,
+          logs: DeadBro.logger.logs
+        }
+
         # Report exceptions attached to this action (e.g. controller/view errors)
         if data[:exception] || data[:exception_object]
           begin
@@ -154,7 +191,7 @@ module DeadBro
             exception_obj = data[:exception_object]
             backtrace = Array(exception_obj&.backtrace).first(50)
 
-            error_payload = {
+            error_payload = detail_fields.merge(
               controller: data[:controller],
               action: data[:action],
               format: data[:format],
@@ -164,6 +201,7 @@ module DeadBro
               duration_ms: duration_ms,
               rails_env: DeadBro.env,
               host: DeadBro.safe_hostname,
+              request_host: safe_request_host(data),
               process_kind: DeadBro.process_kind,
               params: safe_params(data),
               user_agent: safe_user_agent(data),
@@ -173,9 +211,8 @@ module DeadBro
               backtrace: backtrace,
               fingerprint: compute_error_fingerprint(exception_obj),
               cause_chain: build_cause_chain(exception_obj),
-              error: true,
-              logs: DeadBro.logger.logs
-            }
+              error: true
+            )
 
             event_name = (exception_class || exception_obj&.class&.name || "exception").to_s
             client.post_metric(event_name: event_name, payload: error_payload, force: true)
@@ -185,7 +222,7 @@ module DeadBro
           end
         end
 
-        payload = {
+        payload = detail_fields.merge(
           controller: data[:controller],
           action: data[:action],
           format: data[:format],
@@ -194,39 +231,14 @@ module DeadBro
           status: data[:status],
           started_at: started.utc.iso8601(3),
           duration_ms: duration_ms,
-          view_runtime_ms: data[:view_runtime],
-          db_runtime_ms: data[:db_runtime],
           host: DeadBro.safe_hostname,
+          request_host: safe_request_host(data),
           rails_env: DeadBro.env,
           process_kind: DeadBro.process_kind,
           params: safe_params(data),
           user_agent: safe_user_agent(data),
-          user_id: extract_user_id(data),
-          memory_usage: memory_usage_mb,
-          gc_stats: gc_stats,
-          sql_count: sql_count(data),
-          sql_queries: sql_queries,
-          http_outgoing: Thread.current[:dead_bro_http_events] || [],
-          cache_events: cache_events,
-          redis_events: redis_events,
-          elasticsearch_events: elasticsearch_events,
-          cache_hits: cache_hits(data),
-          cache_misses: cache_misses(data),
-          view_events: view_events,
-          view_performance: view_performance,
-          memory_events: memory_events,
-          memory_performance: memory_performance,
-          allocation_phases: allocation_phases,
-          rack_duration_ms: rack_duration_ms,
-          queue_duration_ms: Thread.current[:dead_bro_queue_duration_ms],
-          db_connection_wait_ms: db_connection_stats[:wait_ms],
-          db_connection_checkouts: db_connection_stats[:checkouts],
-          gc_pressure: gc_pressure,
-          ar_instantiation_count: ar_instantiation_count,
-          cpu_time_ms: cpu_time_ms,
-          watch_events: watch_events,
-          logs: DeadBro.logger.logs
-        }
+          user_id: extract_user_id(data)
+        )
         # force: true — the sampling decision (global or per-request-type) was
         # already made above; client#post_metric must not re-roll it with the
         # global-only rate, which would silently override a per-type sample rate.
@@ -240,6 +252,7 @@ module DeadBro
     def self.drain_request_tracking
       # wait_for_explains: false — the result is discarded, so don't block this
       # thread waiting on pending EXPLAIN plans.
+      # stop_request_tracking also pops the transaction-events stack (see its comment).
       DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
       DeadBro::CacheSubscriber.stop_request_tracking if defined?(DeadBro::CacheSubscriber)
       DeadBro::RedisSubscriber.stop_request_tracking if defined?(DeadBro::RedisSubscriber)
@@ -330,6 +343,37 @@ module DeadBro
       end
     end
 
+    # The domain the request was served on (request.host) — distinct from `host`,
+    # which is the machine's OS hostname. Lets one app that answers on several
+    # domains be sliced by domain in the dashboard. Port and userinfo are dropped;
+    # value is lowercased so "Example.com" and "example.com" aren't two rows.
+    def self.safe_request_host(data)
+      raw =
+        if data[:request] && data[:request].respond_to?(:host)
+          data[:request].host
+        elsif data[:headers]
+          headers = data[:headers]
+          if headers.respond_to?(:[])
+            headers["HTTP_HOST"] || headers["Host"] || headers["host"]
+          elsif headers.respond_to?(:env)
+            headers.env && headers.env["HTTP_HOST"]
+          end
+        elsif data[:env].is_a?(Hash)
+          data[:env]["HTTP_HOST"]
+        end
+
+      host = sanitize_string(raw)
+      return "" if host.empty?
+
+      # Strip any userinfo@ and a trailing :port so only the hostname remains.
+      # Only a numeric trailing port is removed, so IPv6 literals ([::1]) survive.
+      host = host.split("@").last.to_s
+      host = host.sub(/:\d+\z/, "")
+      host.downcase[0, 255]
+    rescue
+      ""
+    end
+
     def self.safe_user_agent(data)
       begin
         # Prefer request object if available

data/lib/dead_bro/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DeadBro
-  VERSION = "0.2.30"
+  VERSION = "0.2.32"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dead_bro
 version: !ruby/object:Gem::Version
-  version: 0.2.30
+  version: 0.2.32
 platform: ruby
 authors:
 - Emanuel Comsa