dead_bro 0.2.27 → 0.2.28

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 780f95894158f81ca92c1c1778ba4807f1dce3db4f0f3734606cf47aea3db0e4
-  data.tar.gz: 7ab0fbc93effdd1fbc0590aa30a9abc7fb1e7274f7c7f1513bdcfd2604e9200c
+  metadata.gz: 6b96d1dee673157c1b4c46ad86f7e0d4a4feb51774038de260c5464353ff292e
+  data.tar.gz: f922d71e3c1f132642bf5756aff381a5d372538672c3e0affd010cb31f30b397
 SHA512:
-  metadata.gz: 6552ad55cf89e93470103ee33654249c796246f416f072b9d358e3f81a49aaf5651cf52593f931ed5989800c19aea97f0857782e68989c577211b9a5d7eadb42
-  data.tar.gz: d5001a9656d7bc5407694f209fb3355d76cafc93331ae46c87485d4e187aad9592c03b0beeb305eaecafd5368ad5096ca6bb79a06ac58daadf5907e9fb0811a2
+  metadata.gz: 20f9d81a03bb468cc4ded983ad77b1ad9191dde3ba5c5320b7dd1e11bb5295b5d90289f597df036770cb0d6740ec10f426cf75ba5c941a1922cce8785ee21fe8
+  data.tar.gz: c5325f9e3c8126236fb4386b9214d9637a4086176ae6e2675d4344958bc09d16bbb5718f6461f8bb8eaf75da4ef7e295d3ae012b15a5bb0ab1fbdea8ef0b8f48

data/README.md

@@ -184,31 +184,35 @@ DeadBro automatically tracks SQL queries executed during each request and job. E
 - `cached` - Whether the query was cached
 - `connection_id` - Database connection ID
 - `trace` - Call stack showing where the query was executed
-- `explain_plan` - Query execution plan (when EXPLAIN ANALYZE is enabled, see below)
+- `explain_plan` - Query execution plan (when EXPLAIN capture is enabled, see below)
 
-## Automatic EXPLAIN ANALYZE for Slow Queries
+## Automatic EXPLAIN for Slow Queries
 
-DeadBro can automatically run `EXPLAIN ANALYZE` on slow SQL queries to help you understand query performance and identify optimization opportunities. This feature runs in the background and doesn't block your application requests.
+DeadBro can automatically capture the query plan (`EXPLAIN`) of slow SELECT queries to help you understand query performance and identify optimization opportunities.
+
+### Safety guarantees
+
+- **Plan-only, never executed**: DeadBro runs plain `EXPLAIN` (never `EXPLAIN ANALYZE`), which asks the database for its plan without executing the statement. Your queries are never re-run and your data is never touched.
+- **SELECT-only allowlist**: only `SELECT` statements (and read-only `WITH ... SELECT` CTEs) are ever explained. INSERT/UPDATE/DELETE, DDL, and multi-statement strings are always skipped.
+- **Local opt-in required**: the feature is off unless you enable it in your own `DeadBro.configure` block. The DeadBro backend can remotely *disable* it (e.g. plan gating), but can never switch it on.
+- **Value scrubbing**: quoted string literals echoed in plan text (e.g. `Filter: email = 'a@b.com'`) are replaced with `?` before the plan leaves your app.
+- **Bounded overhead**: plans are captured on background threads (max 3 concurrent per request); at request end DeadBro waits at most 0.5s for stragglers, then drops them.
 
 ### How It Works
 
-- **Automatic Detection**: When a query exceeds the configured threshold, DeadBro automatically captures its execution plan
-- **Background Execution**: EXPLAIN ANALYZE runs in a separate thread using a dedicated database connection, so it never blocks your application
-- **Database Support**: Works with PostgreSQL, MySQL, SQLite, and other databases
-- **Smart Filtering**: Automatically skips transaction queries (BEGIN, COMMIT, ROLLBACK) and other queries that don't benefit from EXPLAIN
+- **Automatic Detection**: when a SELECT exceeds `slow_query_threshold_ms`, DeadBro captures its plan in the background
+- **Database Support**: PostgreSQL, MySQL, SQLite (`EXPLAIN QUERY PLAN`), and any adapter with a standard `EXPLAIN`
 
 ### Configuration
 
-These options are usually set in the DeadBro UI. In Ruby:
-
-- **`explain_analyze_enabled`** (default: `false`) - Set to `true` to enable automatic EXPLAIN ANALYZE
-- **`slow_query_threshold_ms`** (default: `500`) - Queries taking longer than this threshold will have their execution plan captured
+- **`explain_analyze_enabled`** (default: `false`) - local opt-in for automatic EXPLAIN plan capture. Must be set in Ruby; the DeadBro UI toggle can only turn capture off, not on.
+- **`slow_query_threshold_ms`** (default: `500`) - queries taking longer than this threshold will have their execution plan captured
 
 ### Example Configuration
 
 ```ruby
 DeadBro.configure do |config|
-  # Enable EXPLAIN ANALYZE for queries slower than 500ms
+  # Capture EXPLAIN plans for SELECTs slower than 500ms
   config.explain_analyze_enabled = true
   config.slow_query_threshold_ms = 500
 
@@ -220,17 +224,17 @@ end
 ### What You Get
 
 When a slow query is detected, the `explain_plan` field in the SQL query data will contain:
-- **PostgreSQL**: Full EXPLAIN ANALYZE output with buffer usage statistics
-- **MySQL**: EXPLAIN ANALYZE output showing actual execution times
-- **SQLite**: EXPLAIN QUERY PLAN output
-- **Other databases**: Standard EXPLAIN output
+- **PostgreSQL / MySQL**: `EXPLAIN` output — the planner's chosen strategy, estimated costs and row counts
+- **SQLite**: `EXPLAIN QUERY PLAN` output
+- **Other databases**: standard `EXPLAIN` output
 
 This execution plan helps you:
 - Identify missing indexes
 - Understand query execution order
 - Spot full table scans
 - Optimize JOIN operations
-- Analyze buffer and cache usage (PostgreSQL)
+
+Because the statement is not executed, plans show the planner's *estimates* rather than actual runtimes — usually exactly what you need to spot a missing index or an unexpected sequential scan.
 
 ## View Rendering Tracking
 

data/lib/dead_bro/configuration.rb

@@ -9,9 +9,15 @@ module DeadBro
       :circuit_breaker_enabled, :circuit_breaker_failure_threshold, :circuit_breaker_recovery_timeout,
       :circuit_breaker_retry_timeout, :disk_paths, :interfaces_ignore
 
+    # Local-only opt-in for EXPLAIN plan capture on slow queries. The gem runs
+    # DB statements (plan-only EXPLAIN) when this is on, so it must be enabled
+    # in the app's own config — remote settings can only turn it OFF, never on
+    # (see apply_remote_settings). Effective state: #explain_analyze_active?
+    attr_accessor :explain_analyze_enabled
+
     # Remote-managed settings (overwritten by backend JSON `settings` on successful API responses)
     attr_accessor :memory_tracking_enabled, :allocation_tracking_enabled, :allocation_sample_rate,
-      :sample_rate, :slow_query_threshold_ms, :explain_analyze_enabled,
+      :sample_rate, :slow_query_threshold_ms,
       :monitor_enabled, :enable_db_stats, :enable_process_stats, :enable_system_stats,
       :max_sql_queries_to_send, :max_logs_to_send
 
@@ -89,6 +95,10 @@ module DeadBro
       # ~2-5ms overhead can be capped without turning the feature fully off.
       @allocation_sample_rate = 100
       @explain_analyze_enabled = false
+      # Remote kill switch for EXPLAIN capture. Defaults to true so a local
+      # opt-in works against older backends that never send the key; any
+      # response that sends explain_analyze_enabled: false turns capture off.
+      @remote_explain_analyze_enabled = true
       @slow_query_threshold_ms = 500
       @max_sql_queries_to_send = 500
       @max_logs_to_send = 100
@@ -163,7 +173,12 @@ module DeadBro
           case k
           when "sample_rate", "allocation_sample_rate", "slow_query_threshold_ms", "max_sql_queries_to_send", "max_logs_to_send"
             send(:"#{k}=", value.to_i)
-          when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled", "explain_analyze_enabled",
+          when "explain_analyze_enabled"
+            # EXPLAIN runs statements against the customer DB, so the backend
+            # must never be able to switch it on — only off. The local opt-in
+            # (explain_analyze_enabled) stays untouched; see #explain_analyze_active?
+            @remote_explain_analyze_enabled = !!value
+          when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled",
                "monitor_enabled", "enable_db_stats", "enable_process_stats", "enable_system_stats"
             send(:"#{k}=", !!value)
           when "excluded_controllers", "excluded_jobs", "exclusive_controllers", "exclusive_jobs"
@@ -175,6 +190,12 @@ module DeadBro
       end
     end
 
+    # EXPLAIN capture requires BOTH the local opt-in and the remote flag —
+    # locally off means off no matter what the backend sends.
+    def explain_analyze_active?
+      !!(@explain_analyze_enabled && @remote_explain_analyze_enabled)
+    end
+
     def heartbeat_due?
       return false if api_key.nil?
       last_heartbeat_attempt_at.nil? || (Time.now.utc - last_heartbeat_attempt_at) >= HEARTBEAT_INTERVAL

data/lib/dead_bro/job_subscriber.rb

@@ -237,7 +237,8 @@ module DeadBro
     # Release job-side thread-local tracking state when we've decided not to
     # build a payload (excluded job / sampled out). Matches Subscriber.drain_request_tracking.
     def self.drain_job_tracking
-      DeadBro::SqlSubscriber.stop_request_tracking if defined?(DeadBro::SqlSubscriber)
+      # wait_for_explains: false — result is discarded, don't block on pending plans.
+      DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
       DeadBro::DbConnectionSubscriber.stop_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
       DeadBro::GcTracker.stop_request_tracking if defined?(DeadBro::GcTracker)
       DeadBro::ArObjectTracker.stop_request_tracking if defined?(DeadBro::ArObjectTracker)

data/lib/dead_bro/sql_subscriber.rb

@@ -146,7 +146,7 @@ module DeadBro
         if should_explain_query?(duration_ms, original_sql)
           query_info[:explain_plan] = nil
           binds = data[:type_casted_binds] || data[:binds]
-          start_explain_analyze_background(original_sql, data[:connection_id], query_info, binds)
+          start_explain_background(original_sql, data[:connection_id], query_info, binds)
         end
 
         if aggregates_h && normalized_key
@@ -195,8 +195,10 @@ module DeadBro
       (Thread.current[THREAD_LOCAL_AGGREGATES_KEY]  ||= []) << {}
     end
 
-    def self.stop_request_tracking
-      wait_for_pending_explains(EXPLAIN_WAIT_TIMEOUT_SECONDS)
+    # wait_for_explains: false on drain paths (excluded / sampled-out requests)
+    # where the result is discarded — no reason to block on pending plans.
+    def self.stop_request_tracking(wait_for_explains: true)
+      wait_for_pending_explains(EXPLAIN_WAIT_TIMEOUT_SECONDS) if wait_for_explains
 
       stack = Thread.current[THREAD_LOCAL_KEY]
       raw_queries = (stack.is_a?(Array) && stack.any?) ? stack.pop : []
@@ -227,12 +229,14 @@ module DeadBro
       aggregates_h.values.sort_by { |a| -a[:total_duration_ms] }
     end
 
-    # Upper bound on pending EXPLAIN threads per request — stops a slow-query
-    # storm from spawning unbounded background threads.
-    MAX_PENDING_EXPLAINS = 20
+    # Upper bound on pending EXPLAIN threads per request. Each thread checks
+    # out an AR pool connection, so this must stay well below common pool
+    # sizes (Rails default is 5) to avoid starving the app under a storm.
+    MAX_PENDING_EXPLAINS = 3
     # Overall wall-clock we're willing to block the request thread for pending
-    # EXPLAINs. If the plan isn't ready by then, skip it rather than stall the request.
-    EXPLAIN_WAIT_TIMEOUT_SECONDS = 5.0
+    # EXPLAINs. Plan-only EXPLAIN is planning work (typically <10ms); if a plan
+    # isn't ready by then, drop it rather than stall the request.
+    EXPLAIN_WAIT_TIMEOUT_SECONDS = 0.5
 
     def self.wait_for_pending_explains(timeout_seconds)
       pending = Thread.current[THREAD_LOCAL_EXPLAIN_PENDING_KEY]
@@ -246,7 +250,7 @@ module DeadBro
         begin
           thread.join(remaining_time)
         rescue => e
-          DeadBro.logger.debug("Error waiting for EXPLAIN ANALYZE: #{e.message}")
+          DeadBro.logger.debug("Error waiting for EXPLAIN plan: #{e.message}")
         end
       end
     end
@@ -276,25 +280,31 @@ module DeadBro
       sql
     end
 
+    # Read-only statements we are willing to EXPLAIN. Everything else —
+    # INSERT/UPDATE/DELETE, DDL, transaction control, EXPLAIN itself — is
+    # rejected by not matching (allowlist, not denylist).
+    EXPLAIN_ALLOWED_SQL_RE = /\A(SELECT|WITH)\b/i.freeze
+    # A WITH statement can hide data-modifying CTEs (WITH x AS (DELETE ...)).
+    # Plan-only EXPLAIN wouldn't run them, but keep them out entirely.
+    EXPLAIN_DML_KEYWORD_RE = /\b(INSERT|UPDATE|DELETE|MERGE)\b/i.freeze
+
     def self.should_explain_query?(duration_ms, sql)
-      return false unless DeadBro.configuration.explain_analyze_enabled
+      return false unless DeadBro.configuration.explain_analyze_active?
       return false if duration_ms < DeadBro.configuration.slow_query_threshold_ms
       return false unless sql.is_a?(String)
-      return false if sql.strip.empty?
 
-      # Skip EXPLAIN for certain query types that don't benefit from it
-      sql_upper = sql.upcase.strip
-      return false if sql_upper.start_with?("EXPLAIN")
-      return false if sql_upper.start_with?("BEGIN")
-      return false if sql_upper.start_with?("COMMIT")
-      return false if sql_upper.start_with?("ROLLBACK")
-      return false if sql_upper.start_with?("SAVEPOINT")
-      return false if sql_upper.start_with?("RELEASE")
+      stripped = sql.strip
+      stripped = stripped.chomp(";").rstrip
+      return false if stripped.empty?
+      # No multi-statement strings — EXPLAIN must cover exactly one statement.
+      return false if stripped.include?(";")
+      return false unless stripped.match?(EXPLAIN_ALLOWED_SQL_RE)
+      return false if stripped.match?(/\AWITH\b/i) && stripped.match?(EXPLAIN_DML_KEYWORD_RE)
 
       true
     end
 
-    def self.start_explain_analyze_background(sql, connection_id, query_info, binds = nil)
+    def self.start_explain_background(sql, connection_id, query_info, binds = nil)
       return unless defined?(ActiveRecord)
       return unless ActiveRecord::Base.respond_to?(:connection)
 
@@ -327,14 +337,14 @@ module DeadBro
               connection.execute(explain_sql)
             end
 
-            explain_plan = format_explain_result(result, connection)
+            explain_plan = scrub_explain_plan(format_explain_result(result, connection))
             query_info[:explain_plan] = if explain_plan && !explain_plan.to_s.strip.empty?
               explain_plan
             end
           end
         rescue => e
           # Silently fail — don't let EXPLAIN break the application.
-          append_log_to_thread(main_thread, :debug, "Failed to capture EXPLAIN ANALYZE: #{e.message}")
+          append_log_to_thread(main_thread, :debug, "Failed to capture EXPLAIN plan: #{e.message}")
           query_info[:explain_plan] = nil
         end
       end
@@ -342,7 +352,22 @@ module DeadBro
       pending << explain_thread
     rescue => e
       # Use DeadBro.logger here since we're still in the main thread
-      DeadBro.logger.debug("Failed to start EXPLAIN ANALYZE thread: #{e.message}")
+      DeadBro.logger.debug("Failed to start EXPLAIN thread: #{e.message}")
+    end
+
+    # Quoted string literal in plan text, e.g. Filter: ((email)::text = 'a@b.com'::text).
+    # '' is SQL escaping for a single quote inside a literal.
+    EXPLAIN_PLAN_QUOTED_LITERAL_RE = /'(?:[^']|'')*'/.freeze
+
+    # Plan text echoes the interpolated bind values in Filter / Index Cond
+    # lines, bypassing sanitize_sql. Blank out quoted literals before the plan
+    # is shipped. (Bare numbers are left alone — indistinguishable from the
+    # plan's own cost/row estimates.)
+    def self.scrub_explain_plan(plan)
+      return plan unless plan.is_a?(String)
+      plan.gsub(EXPLAIN_PLAN_QUOTED_LITERAL_RE, "?")
+    rescue
+      plan
     end
 
     # Append a log entry directly to a specific thread's log storage
@@ -389,18 +414,13 @@ module DeadBro
     def self.build_explain_query(sql, connection)
       adapter_name = connection.adapter_name.downcase
 
+      # Plan-only EXPLAIN everywhere. EXPLAIN ANALYZE *executes* the statement
+      # on PostgreSQL and MySQL 8 — a monitoring agent must never re-run
+      # customer queries, so ANALYZE is deliberately not used for any adapter.
       case adapter_name
-      when "postgresql", "postgis"
-        # PostgreSQL supports ANALYZE and BUFFERS
-        "EXPLAIN (ANALYZE, BUFFERS) #{sql}"
-      when "mysql", "mysql2", "trilogy"
-        # MySQL uses different syntax - ANALYZE is a separate keyword
-        "EXPLAIN ANALYZE #{sql}"
       when "sqlite3"
-        # SQLite supports EXPLAIN QUERY PLAN
         "EXPLAIN QUERY PLAN #{sql}"
       else
-        # Generic fallback - just EXPLAIN
         "EXPLAIN #{sql}"
       end
     end

data/lib/dead_bro/subscriber.rb

@@ -235,7 +235,9 @@ module DeadBro
     # a payload (disabled / excluded / sampled out). Without this, a subsequent
     # request reusing the same Puma thread would see stale queries/events.
     def self.drain_request_tracking
-      DeadBro::SqlSubscriber.stop_request_tracking if defined?(DeadBro::SqlSubscriber)
+      # wait_for_explains: false — the result is discarded, so don't block this
+      # thread waiting on pending EXPLAIN plans.
+      DeadBro::SqlSubscriber.stop_request_tracking(wait_for_explains: false) if defined?(DeadBro::SqlSubscriber)
       DeadBro::CacheSubscriber.stop_request_tracking if defined?(DeadBro::CacheSubscriber)
       DeadBro::RedisSubscriber.stop_request_tracking if defined?(DeadBro::RedisSubscriber)
       DeadBro::ElasticsearchSubscriber.stop_request_tracking if defined?(DeadBro::ElasticsearchSubscriber)

data/lib/dead_bro/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DeadBro
-  VERSION = "0.2.27"
+  VERSION = "0.2.28"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dead_bro
 version: !ruby/object:Gem::Version
-  version: 0.2.27
+  version: 0.2.28
 platform: ruby
 authors:
 - Emanuel Comsa