dead_bro 0.2.23 → 0.2.25

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d17b4a0fa119f9917bfdf77cc66078019bf72e87ca5c2752183b9e3b85629f5
-  data.tar.gz: 26bec0d029e000bd2ecff843ca242b8647862bf39727b31cef6408baa871cd13
+  metadata.gz: aec6a4f779a49726e225b3d01e9e1a6516c56639a451837615a8d8c08cb59e8a
+  data.tar.gz: 186f7fd09ef2b4c3d2ed9390c1278a9f5bbc577a0e05572e63cea7461689fccc
 SHA512:
-  metadata.gz: 97c4aba35a1f39c680d8445a9a31dc3ae35d9e0ea29b89eee0f3f9740086db8de498b810d7e38171665106d9f8ff3d15caaef47cbc257c6d39bea31da2852d64
-  data.tar.gz: 321aee532718f23c430a81fdec50ab3d27fe5373e02fab0f1598e884d3d0d02b10d175691d1ec08d0bffde97a0cbb5e7488d1b0125d29b6552009921c2dba4e2
+  metadata.gz: 2b7028050d353c623587cd3f083b2b8666fcd05abfa68b7cf05d7bdda5979f29118bdabcbb37c0165a10eca829e30305ff061874c08d9a45e65d89193acd3ad3
+  data.tar.gz: 2f16e4716988194147e087396dc419c4d20fdb4bd2a958d38f866e158d76e2f233d0187a08eb54c6b7f40a9eb9637dc0079e857257cb9d3f2091a05b9490968b

data/CHANGELOG.md

@@ -3,6 +3,26 @@
 ### Added
 - Monitor thread now sends a synchronous heartbeat on startup before the first collection tick. This ensures remote settings — including `monitor_enabled` — are applied from the very first reporting cycle, so Sidekiq workers and other non-web processes that have not yet sent any metrics still receive the correct configuration immediately on boot rather than waiting up to 60 seconds for the first scheduled tick.
 
+## [0.2.25] - 2026-06-14
+
+### Added
+- **Memory diagnostics for "what is allocating all this memory?"** A request can grow RSS by hundreds of MB while instantiating only a few thousand ActiveRecord objects — the existing AR object count cannot explain it because the memory lives in transient strings/hashes (e.g. deserialized Elasticsearch responses, large JSON response bodies), not AR models. These additions localize that growth. They are organised into two performance tiers so the default path stays fast:
+
+  **Under `memory_tracking_enabled` (~0.1ms overhead, on by default):**
+  - **Retained-vs-transient GC signals.** `GcTracker` now enriches the per-request `gc_pressure` payload with three additional `GC.stat`-derived fields:
+    - `heap_live_slots_growth` — net change in live heap slots over the request. A small value alongside a large `allocated_objects` means the memory was *transient* (allocated then reclaimed by GC, with RSS held by allocator fragmentation); a large value means objects were *retained* — the real leak signal. This reframes a large RSS delta that the previous metrics could not characterise.
+    - `malloc_increase_bytes` / `oldmalloc_increase_bytes` — request-end gauges of memory malloc'd outside the Ruby object heap (large strings/buffers), pointing at off-heap pressure such as parsed response bodies.
+    - These fields are captured only when `memory_tracking_enabled`; the base GC pressure fields (`minor_gc_runs`, `major_gc_runs`, `allocated_objects`, `gc_time_ms`) remain always-on and unchanged.
+  - **Per-phase allocation attribution.** New `MemoryPhaseTracker` charges each request's object allocations to the phase that produced them — `sql`, `view`, or `elasticsearch` — emitted as a new `allocation_phases` field on the request payload (e.g. `{ elasticsearch: 412_000, sql: 9_000, view: 2_500 }`). Attribution is **exclusive**: a `sql.active_record` event nested inside a view render is charged only to `:sql`, never double-counted into `:view`, via a thread-local stack that pauses the parent phase while a child is active. Whatever isn't captured by an instrumented phase is controller/application code and is derivable on the backend as `gc_pressure.allocated_objects` minus the sum of the buckets. Overhead is two single-key `GC.stat(:total_allocated_objects)` reads per instrumented event (no hash allocation); listeners are installed at boot but no-op via a thread-local check unless a request opts in.
+
+  **Under `allocation_tracking_enabled` + the new `allocation_sample_rate` (~2–5ms overhead, off by default):**
+  - **By-bytes object-type breakdown.** New `AllocationSourceSampler` produces `memsize_by_type` — total retained bytes summed per Ruby class via `ObjectSpace.memsize_of`. This catches the "death by a million small strings" pattern (each object well under any size threshold, but enormous in aggregate) that the existing >1MB single-object scan structurally misses.
+  - **Allocation-source attribution.** `allocation_sources` reports the top allocation sites (`file:line`) by retained bytes, using `ObjectSpace.trace_object_allocations`. This is the gold-standard "this line allocated 300MB" answer. The expensive heap walk is additionally gated on actual memory growth (`memory_growth_mb >= 50` by default), so even with the flag on, a request that didn't move memory pays nothing for the walk. Sampled object counts/bytes are reported alongside `allocation_sample_rate` so consumers can extrapolate to the full heap.
+  - **`allocation_sample_rate` configuration (default `100`, remote-manageable).** When allocation tracking is enabled, the heavy per-request work now runs on this percentage of requests, so the cost can be capped across traffic instead of being all-or-nothing. The decision is made once at request start (`Configuration#allocation_tracking_active?`) and cached in a thread-local so the matching stop agrees with the start.
+
+### Changed
+- **`objspace` is now loaded only under `allocation_tracking_enabled`.** `ObjectSpace.memsize_of`, `trace_object_allocations_*`, and `allocation_source*` live in the `objspace` stdlib extension, which is not loaded by default. It is now required exclusively via `AllocationSourceSampler` (loaded only under the allocation flag), keeping the heavyweight extension off the default and memory-tracking paths. As a consequence, the gem's pre-existing large-object scan (in `MemoryTrackingSubscriber` / `DeadBro.analyze`), which also depends on `ObjectSpace.memsize_of`, likewise functions only when allocation tracking is enabled — consistent with it already living behind that flag.
+
 ## [0.2.21] - 2026-06-02
 
 ### Added

data/README.md

@@ -14,13 +14,21 @@ Add to your Gemfile:
 gem "dead_bro", git: "https://github.com/rubydevro/dead_bro.git"
 ```
 
+Then run the install generator to create the initializer:
+
+```sh
+bin/rails generate dead_bro:install
+```
+
+This creates `config/initializers/dead_bro.rb`. Set `DEAD_BRO_API_KEY` in your environment and you're done.
+
 ## Usage
 
 By default, if Rails is present, DeadBro auto-subscribes to `process_action.action_controller` and posts metrics asynchronously.
 
 ### Required: API key in your app
 
-Add an initializer (for example `config/initializers/dead_bro.rb`) and set your API key from the environment, [Rails credentials](https://guides.rubyonrails.org/security.html#custom-credentials), or another secret store:
+The install generator above creates `config/initializers/dead_bro.rb` for you. If you prefer to add it manually, create the file and set your API key from the environment, [Rails credentials](https://guides.rubyonrails.org/security.html#custom-credentials), or another secret store:
 
 ```ruby
 DeadBro.configure do |config|
@@ -255,15 +263,16 @@ Usually managed in the dashboard; Ruby example:
 DeadBro.configure do |config|
   config.memory_tracking_enabled = true        # Enable lightweight memory tracking (default: true)
   config.allocation_tracking_enabled = false   # Enable detailed allocation tracking (default: false)
-  
+  config.allocation_sample_rate = 100          # % of requests that pay for allocation tracking when enabled (1-100, default: 100)
+
   # Sampling configuration
   config.sample_rate = 100                     # Percentage of requests to track (1-100, default: 100)
 end
 ```
 
 **Performance Impact:**
-- **Lightweight mode**: ~0.1ms overhead per request
-- **Allocation tracking**: ~2-5ms overhead per request (only enable when needed)
+- **Lightweight mode** (`memory_tracking_enabled`, ~0.1ms overhead per request): RSS before/after, GC pressure, the retained-vs-transient signals (`heap_live_slots_growth`, `malloc_increase_bytes`), and per-phase allocation attribution (`allocation_phases` — which of `sql`/`view`/`elasticsearch` allocated the request's objects).
+- **Allocation tracking** (`allocation_tracking_enabled`, ~2-5ms overhead per request — only enable when needed): adds by-bytes object-type breakdown (`memsize_by_type`) and allocation-source attribution (`allocation_sources`, `file:line`) using the `objspace` extension, which is loaded only on this path. Use `allocation_sample_rate` to spread that cost across a fraction of traffic.
 
 ## Job Tracking
 

data/lib/dead_bro/allocation_source_sampler.rb

@@ -0,0 +1,140 @@
+# frozen_string_literal: true
+
+# ObjectSpace.memsize_of / trace_object_allocations_* / allocation_source* all
+# live in the `objspace` stdlib extension, which is NOT loaded by default.
+# Requiring it only defines the methods — it does not start any tracing or add
+# runtime overhead until we explicitly call trace_object_allocations_start.
+begin
+  require "objspace"
+rescue LoadError
+  # Not available on this Ruby — available? will report false.
+end
+
+module DeadBro
+  # Deep, opt-in memory diagnostics that answer "what code allocated this?".
+  # Only active when allocation tracking is on (see Configuration
+  # #allocation_tracking_active?), because turning on object allocation tracing
+  # adds ~2-5ms of per-request overhead.
+  #
+  # Two complementary breakdowns, both produced from a single ObjectSpace walk
+  # after the request finishes:
+  #
+  #   * by_type_bytes — total *retained bytes* per Ruby class. This catches the
+  #     "death by a million small strings" pattern (e.g. a deserialized
+  #     Elasticsearch response) that MemoryTrackingSubscriber's >1MB
+  #     single-object scan structurally misses, because it sums bytes per type
+  #     instead of flagging individually-large objects.
+  #
+  #   * by_source — top allocation sites (file:line) by retained bytes. This is
+  #     the gold-standard "this line allocated 300MB" attribution, available
+  #     because trace_object_allocations was running for the request.
+  module AllocationSourceSampler
+    # Fraction of live objects inspected during the post-request walk. Reported
+    # back as sample_rate so the consumer can extrapolate to the full heap.
+    SAMPLE_RATE = 0.10
+    MAX_RESULTS = 20
+    # Only report a type if its sampled retained bytes clear this floor (keeps
+    # the breakdown to things that actually matter).
+    LARGE_TYPE_MIN_BYTES = 100_000
+    # Skip the walk entirely below this growth — no point profiling a request
+    # that didn't move memory. Gates the expensive path even when the flag is on.
+    DEFAULT_MIN_GROWTH_MB = 50
+
+    def self.available?
+      defined?(ObjectSpace) &&
+        ObjectSpace.respond_to?(:trace_object_allocations_start) &&
+        ObjectSpace.respond_to?(:memsize_of)
+    rescue StandardError
+      false
+    end
+
+    # Begin recording allocation source locations. Must be called before the
+    # request allocates the objects we want to attribute.
+    def self.start
+      return unless available?
+      ObjectSpace.trace_object_allocations_start
+    rescue StandardError
+      # Best-effort only.
+    end
+
+    # Stop and discard recorded allocation data. Call this AFTER analyze, since
+    # clearing wipes the source locations analyze reads.
+    def self.stop
+      return unless available?
+      ObjectSpace.trace_object_allocations_stop
+      ObjectSpace.trace_object_allocations_clear
+    rescue StandardError
+      # Best-effort only.
+    end
+
+    # Walk live objects once and build the two breakdowns. Returns {} when
+    # tracing is unavailable, or {skipped: ...} when growth was below threshold.
+    def self.analyze(memory_growth_mb: nil, min_growth_mb: DEFAULT_MIN_GROWTH_MB)
+      return {} unless available?
+      if memory_growth_mb && memory_growth_mb < min_growth_mb
+        return {skipped: "memory_growth_below_threshold", min_growth_mb: min_growth_mb}
+      end
+
+      by_type = Hash.new { |h, k| h[k] = {count: 0, bytes: 0} }
+      by_source = Hash.new { |h, k| h[k] = {count: 0, bytes: 0} }
+
+      ObjectSpace.each_object do |obj|
+        next unless rand < SAMPLE_RATE
+
+        size = begin
+          ObjectSpace.memsize_of(obj)
+        rescue StandardError
+          0
+        end
+        next unless size && size > 0
+
+        klass = begin
+          obj.class.name
+        rescue StandardError
+          nil
+        end || "Unknown"
+        type_bucket = by_type[klass]
+        type_bucket[:count] += 1
+        type_bucket[:bytes] += size
+
+        file = begin
+          ObjectSpace.allocation_sourcefile(obj)
+        rescue StandardError
+          nil
+        end
+        next unless file
+
+        line = begin
+          ObjectSpace.allocation_sourceline(obj)
+        rescue StandardError
+          nil
+        end
+        source_bucket = by_source["#{file}:#{line}"]
+        source_bucket[:count] += 1
+        source_bucket[:bytes] += size
+      end
+
+      {
+        sample_rate: SAMPLE_RATE,
+        by_type_bytes: top_by_bytes(by_type, LARGE_TYPE_MIN_BYTES),
+        by_source: top_by_bytes(by_source, 0)
+      }
+    rescue StandardError
+      {}
+    end
+
+    def self.top_by_bytes(hash, min_bytes)
+      hash.select { |_, v| v[:bytes] >= min_bytes }
+        .sort_by { |_, v| -v[:bytes] }
+        .first(MAX_RESULTS)
+        .map do |key, v|
+          {
+            name: key,
+            count: v[:count],
+            bytes: v[:bytes],
+            mb: (v[:bytes] / 1_000_000.0).round(2)
+          }
+        end
+    end
+  end
+end

data/lib/dead_bro/configuration.rb

@@ -10,7 +10,7 @@ module DeadBro
       :circuit_breaker_retry_timeout, :disk_paths, :interfaces_ignore
 
     # Remote-managed settings (overwritten by backend JSON `settings` on successful API responses)
-    attr_accessor :memory_tracking_enabled, :allocation_tracking_enabled,
+    attr_accessor :memory_tracking_enabled, :allocation_tracking_enabled, :allocation_sample_rate,
       :sample_rate, :slow_query_threshold_ms, :explain_analyze_enabled,
       :monitor_enabled, :enable_db_stats, :enable_process_stats, :enable_system_stats,
       :max_sql_queries_to_send, :max_logs_to_send
@@ -55,7 +55,7 @@ module DeadBro
     ].freeze
 
     REMOTE_SETTING_KEYS = %w[
-      enabled sample_rate memory_tracking_enabled allocation_tracking_enabled
+      enabled sample_rate memory_tracking_enabled allocation_tracking_enabled allocation_sample_rate
       explain_analyze_enabled slow_query_threshold_ms max_sql_queries_to_send max_logs_to_send
       excluded_controllers excluded_jobs exclusive_controllers exclusive_jobs
       monitor_enabled enable_db_stats enable_process_stats enable_system_stats
@@ -79,6 +79,10 @@ module DeadBro
       @sample_rate = 100
       @memory_tracking_enabled = true
       @allocation_tracking_enabled = false
+      # When allocation tracking is on, the heavy per-request work (object-space
+      # sampling, allocation-source tracing) runs on this % of requests so the
+      # ~2-5ms overhead can be capped without turning the feature fully off.
+      @allocation_sample_rate = 100
       @explain_analyze_enabled = false
       @slow_query_threshold_ms = 500
       @max_sql_queries_to_send = 500
@@ -142,7 +146,7 @@ module DeadBro
           next unless REMOTE_SETTING_KEYS.include?(k)
 
           case k
-          when "sample_rate", "slow_query_threshold_ms", "max_sql_queries_to_send", "max_logs_to_send"
+          when "sample_rate", "allocation_sample_rate", "slow_query_threshold_ms", "max_sql_queries_to_send", "max_logs_to_send"
             send(:"#{k}=", value.to_i)
           when "enabled", "memory_tracking_enabled", "allocation_tracking_enabled", "explain_analyze_enabled",
                "monitor_enabled", "enable_db_stats", "enable_process_stats", "enable_system_stats"
@@ -239,6 +243,20 @@ module DeadBro
       rand(1..100) <= sample_rate
     end
 
+    # Per-request decision: should this request pay for the heavy allocation
+    # tracking (object-space sampling + allocation-source tracing)? Combines the
+    # on/off flag with allocation_sample_rate. Decide once at request start and
+    # reuse the cached result for the matching stop, so start/stop agree.
+    def allocation_tracking_active?
+      return false unless allocation_tracking_enabled
+
+      rate = allocation_sample_rate.to_i
+      return true if rate >= 100
+      return false if rate <= 0
+
+      rand(1..100) <= rate
+    end
+
     # Returns the configured sample_rate only (no ENV fallback). Use DeadBro.configure or remote settings.
     def resolve_sample_rate
       @sample_rate

data/lib/dead_bro/error_middleware.rb

@@ -111,17 +111,29 @@ module DeadBro
       return {} unless req
 
       params = req.params || {}
-      sensitive_keys = %w[password password_confirmation token secret key authorization api_key]
-      filtered = params.dup
-      sensitive_keys.each do |k|
-        filtered.delete(k)
-        filtered.delete(k.to_sym)
-      end
-      JSON.parse(JSON.dump(filtered)) # ensure JSON-safe
+      # Redact at every nesting level (e.g. user[password]) before serializing.
+      JSON.parse(JSON.dump(redact_sensitive(params)))
     rescue
       {}
     end
 
+    # Matches a key segment so nested/prefixed/suffixed sensitive keys are caught
+    # without redacting innocent keys like passenger_count.
+    SENSITIVE_SEGMENT_RE = /(?:\A|[_\-\[])(password|passwd|secret|token|api_?key|access_?key|auth|authorization|credential|ssn|credit_?card|card_?number|cvv|cvc)(?:\z|[_\-\]])/i
+
+    def redact_sensitive(value)
+      case value
+      when Hash
+        value.each_with_object({}) do |(k, v), memo|
+          memo[k] = SENSITIVE_SEGMENT_RE.match?(k.to_s) ? "[FILTERED]" : redact_sensitive(v)
+        end
+      when Array
+        value.map { |v| redact_sensitive(v) }
+      else
+        value
+      end
+    end
+
     def truncate(str, max)
       return str if str.nil? || str.length <= max
       str[0..(max - 1)]

data/lib/dead_bro/gc_tracker.rb

@@ -19,12 +19,27 @@ module DeadBro
     def self.snapshot
       return {} unless defined?(GC) && GC.respond_to?(:stat)
       stat = GC.stat
-      {
+      base = {
         minor_gc_count: stat[:minor_gc_count] || 0,
         major_gc_count: stat[:major_gc_count] || 0,
         total_allocated_objects: stat[:total_allocated_objects] || 0,
         gc_time_ns: GC.respond_to?(:total_time) ? GC.total_time : nil
       }
+
+      # Memory-tracking enrichment (a few extra GC.stat reads). Only the base
+      # GC pressure fields above are truly always-on.
+      if memory_tracking_enabled?
+        # Live heap slots — net retained objects. Comparing this delta against
+        # allocated_objects separates transient churn from genuine retention.
+        base[:heap_live_slots] = stat[:heap_live_slots] || 0
+        # Bytes malloc'd outside the Ruby object heap (big strings/buffers, e.g.
+        # parsed JSON response bodies). These are point-in-time gauges reset by
+        # GC, so we report the request-end value rather than a diff.
+        base[:malloc_increase_bytes] = stat[:malloc_increase_bytes] || 0
+        base[:oldmalloc_increase_bytes] = stat[:oldmalloc_increase_bytes] || 0
+      end
+
+      base
     rescue
       {}
     end
@@ -34,14 +49,33 @@ module DeadBro
       gc_time_ms = if before[:gc_time_ns] && after[:gc_time_ns]
         ((after[:gc_time_ns] - before[:gc_time_ns]) / 1_000_000.0).round(3)
       end
-      {
+      result = {
         minor_gc_runs: (after[:minor_gc_count] || 0) - (before[:minor_gc_count] || 0),
         major_gc_runs: (after[:major_gc_count] || 0) - (before[:major_gc_count] || 0),
         allocated_objects: (after[:total_allocated_objects] || 0) - (before[:total_allocated_objects] || 0),
         gc_time_ms: gc_time_ms
       }
+
+      # Present only when the enrichment was captured (memory tracking enabled).
+      if after.key?(:heap_live_slots) || before.key?(:heap_live_slots)
+        # Net change in live slots over the request. A small value alongside a
+        # large allocated_objects means the memory was transient (reclaimed by
+        # GC); a large value means objects were retained — the real leak signal.
+        result[:heap_live_slots_growth] = (after[:heap_live_slots] || 0) - (before[:heap_live_slots] || 0)
+        # Off-heap malloc pressure pending at request end (see snapshot).
+        result[:malloc_increase_bytes] = after[:malloc_increase_bytes] || 0
+        result[:oldmalloc_increase_bytes] = after[:oldmalloc_increase_bytes] || 0
+      end
+
+      result
     rescue
       {}
     end
+
+    def self.memory_tracking_enabled?
+      DeadBro.configuration.memory_tracking_enabled
+    rescue
+      false
+    end
   end
 end

data/lib/dead_bro/job_subscriber.rb

@@ -105,6 +105,7 @@ module DeadBro
           job_id: data[:job].job_id,
           queue_name: data[:job].queue_name,
           arguments: safe_arguments(data[:job].arguments),
+          started_at: started.utc.iso8601(3),
           duration_ms: duration_ms,
           queue_duration_ms: queue_duration_ms,
           db_connection_wait_ms: db_connection_stats[:wait_ms],
@@ -201,6 +202,7 @@ module DeadBro
           job_id: data[:job].job_id,
           queue_name: data[:job].queue_name,
           arguments: safe_arguments(data[:job].arguments),
+          started_at: started.utc.iso8601(3),
           duration_ms: duration_ms,
           queue_duration_ms: queue_duration_ms,
           db_connection_wait_ms: db_connection_stats[:wait_ms],

data/lib/dead_bro/memory_phase_tracker.rb

@@ -0,0 +1,131 @@
+# frozen_string_literal: true
+
+require "active_support/notifications"
+
+module DeadBro
+  # Attributes a request's object allocations to the phase that produced them —
+  # e.g. "92% of this request's allocations happened during Elasticsearch".
+  #
+  # This is the always-on, low-overhead companion to GcTracker: GcTracker tells
+  # you *how many* objects a request allocated and whether they were retained;
+  # MemoryPhaseTracker tells you *where* they were allocated, so a 400MB request
+  # can be localized to ES deserialization vs view rendering vs controller code
+  # without running a full allocation profiler.
+  #
+  # Attribution is *exclusive*: a sql.active_record event nested inside a view
+  # render is charged only to :sql, not to both. A thread-local stack records
+  # the allocation counter when each phase becomes active; entering a child
+  # phase flushes the parent's accumulated delta and pauses it, leaving the
+  # child resumes the parent. Whatever isn't captured by an instrumented phase
+  # stays "unattributed" (controller/application code) and is derivable on the
+  # backend as gc_pressure.allocated_objects minus the sum of these buckets.
+  module MemoryPhaseTracker
+    THREAD_KEY = :dead_bro_memory_phases
+
+    # ActiveSupport event name => phase bucket. Each maps to a coarse phase so
+    # the breakdown stays readable (all view render events collapse to :view).
+    EVENT_PHASES = {
+      "sql.active_record" => :sql,
+      "render_template.action_view" => :view,
+      "render_partial.action_view" => :view,
+      "render_collection.action_view" => :view,
+      "render_layout.action_view" => :view,
+      "request.elasticsearch" => :elasticsearch,
+      "request.elastic_transport" => :elasticsearch
+    }.freeze
+
+    # Bridges ActiveSupport's evented-listener protocol (start/finish) onto our
+    # enter/leave accounting. Registered once per event name at boot.
+    class Listener
+      def initialize(phase)
+        @phase = phase
+      end
+
+      def start(_name, _id, _payload)
+        MemoryPhaseTracker.enter(@phase)
+      end
+
+      def finish(_name, _id, _payload)
+        MemoryPhaseTracker.leave(@phase)
+      end
+    end
+
+    def self.subscribe!
+      return if @subscribed
+      @subscribed = true
+      return unless allocation_counter_available?
+
+      EVENT_PHASES.each do |event_name, phase|
+        ActiveSupport::Notifications.subscribe(event_name, Listener.new(phase))
+      end
+    rescue StandardError
+      # Never raise from instrumentation install.
+    end
+
+    def self.start_request_tracking
+      Thread.current[THREAD_KEY] = {buckets: Hash.new(0), stack: []}
+    end
+
+    # Returns { sql: n, view: n, elasticsearch: n } of objects allocated
+    # exclusively within each phase, omitting phases that allocated nothing.
+    def self.stop_request_tracking
+      state = Thread.current[THREAD_KEY]
+      return {} unless state.is_a?(Hash)
+
+      buckets = state[:buckets]
+      buckets.each_with_object({}) do |(phase, count), result|
+        result[phase] = count if count.positive?
+      end
+    ensure
+      Thread.current[THREAD_KEY] = nil
+    end
+
+    def self.enter(phase)
+      state = Thread.current[THREAD_KEY]
+      return unless state.is_a?(Hash)
+
+      now = allocated_objects
+      stack = state[:stack]
+      if (parent = stack.last)
+        # Pause the parent: bank what it allocated up to this point.
+        state[:buckets][parent[:phase]] += now - parent[:checkpoint]
+      end
+      stack << {phase: phase, checkpoint: now}
+    rescue StandardError
+      # Best-effort only.
+    end
+
+    def self.leave(phase)
+      state = Thread.current[THREAD_KEY]
+      return unless state.is_a?(Hash)
+
+      stack = state[:stack]
+      frame = stack.pop
+      return unless frame
+
+      now = allocated_objects
+      state[:buckets][frame[:phase]] += now - frame[:checkpoint]
+      # Resume the parent from this point so the child's allocations aren't
+      # double-counted into it.
+      if (parent = stack.last)
+        parent[:checkpoint] = now
+      end
+    rescue StandardError
+      # Best-effort only.
+    end
+
+    # Single-key GC.stat returns just the Integer (no hash allocation), so this
+    # is cheap enough to call on every instrumented event boundary.
+    def self.allocated_objects
+      GC.stat(:total_allocated_objects)
+    rescue StandardError
+      0
+    end
+
+    def self.allocation_counter_available?
+      defined?(GC) && GC.respond_to?(:stat) && !GC.stat(:total_allocated_objects).nil?
+    rescue StandardError
+      false
+    end
+  end
+end

data/lib/dead_bro/railtie.rb

@@ -44,6 +44,11 @@ if defined?(Rails) && defined?(Rails::Railtie)
           require "dead_bro/ar_object_tracker"
           DeadBro::ArObjectTracker.subscribe!
 
+          # Install per-phase allocation attribution. Listeners are cheap and
+          # no-op unless a request opts in (gated on memory_tracking_enabled).
+          require "dead_bro/memory_phase_tracker"
+          DeadBro::MemoryPhaseTracker.subscribe!
+
           # Install view rendering tracking
           require "dead_bro/view_rendering_subscriber"
           DeadBro::ViewRenderingSubscriber.subscribe!(client: shared_client)
@@ -53,10 +58,13 @@ if defined?(Rails) && defined?(Rails::Railtie)
           require "dead_bro/memory_leak_detector"
           DeadBro::MemoryLeakDetector.initialize_history
 
-          # Install detailed memory tracking only if enabled
+          # Install detailed memory + allocation-source tracking only if
+          # enabled. This is also where `objspace` gets loaded (via the
+          # sampler), so the heavyweight extension stays off the default path.
           if DeadBro.configuration.allocation_tracking_enabled
             require "dead_bro/memory_tracking_subscriber"
             DeadBro::MemoryTrackingSubscriber.subscribe!(client: shared_client)
+            require "dead_bro/allocation_source_sampler"
           end
 
           # Install job tracking if ActiveJob is available

data/lib/dead_bro/sql_tracking_middleware.rb

@@ -46,9 +46,16 @@ module DeadBro
         DeadBro::LightweightMemoryTracker.start_request_tracking
       end
 
-      # Start detailed memory tracking when allocation tracking is enabled
-      if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
-        DeadBro::MemoryTrackingSubscriber.start_request_tracking
+      # Decide once whether this request pays for heavy allocation tracking
+      # (flag + per-request sampling). Cache the decision so the matching stop
+      # in Subscriber agrees with this start.
+      alloc_active = DeadBro.configuration.allocation_tracking_active?
+      Thread.current[:dead_bro_alloc_active] = alloc_active
+
+      # Start detailed memory + allocation-source tracking when active
+      if alloc_active
+        DeadBro::MemoryTrackingSubscriber.start_request_tracking if defined?(DeadBro::MemoryTrackingSubscriber)
+        DeadBro::AllocationSourceSampler.start if defined?(DeadBro::AllocationSourceSampler)
       end
 
       # Start Elasticsearch tracking for this request
@@ -64,6 +71,11 @@ module DeadBro
       # Start GC pressure tracking — snapshot before any app code runs
       DeadBro::GcTracker.start_request_tracking if defined?(DeadBro::GcTracker)
 
+      # Start per-phase allocation attribution (~0.1ms; under memory tracking)
+      if DeadBro.configuration.memory_tracking_enabled && defined?(DeadBro::MemoryPhaseTracker)
+        DeadBro::MemoryPhaseTracker.start_request_tracking
+      end
+
       # Start AR object instantiation counting for this request
       DeadBro::ArObjectTracker.start_request_tracking if defined?(DeadBro::ArObjectTracker)
 
@@ -110,6 +122,13 @@ module DeadBro
       # Bypass stop_request_tracking intentionally — cleanup only, no return value needed here.
       Thread.current[DeadBro::ArObjectTracker::THREAD_KEY] = nil if defined?(DeadBro::ArObjectTracker)
       Thread.current[DeadBro::CpuTracker::THREAD_KEY] = nil if defined?(DeadBro::CpuTracker)
+      Thread.current[DeadBro::MemoryPhaseTracker::THREAD_KEY] = nil if defined?(DeadBro::MemoryPhaseTracker)
+      # Safety net: ensure allocation tracing is never left running across
+      # requests (Subscriber normally stops it after analyzing).
+      if Thread.current[:dead_bro_alloc_active]
+        DeadBro::AllocationSourceSampler.stop if defined?(DeadBro::AllocationSourceSampler)
+      end
+      Thread.current[:dead_bro_alloc_active] = nil
       Thread.current[DeadBro::TRACKING_START_TIME_KEY] = nil
     end
 

data/lib/dead_bro/subscriber.rb

@@ -79,10 +79,34 @@ module DeadBro
         view_events = DeadBro::ViewRenderingSubscriber.stop_request_tracking
         view_performance = DeadBro::ViewRenderingSubscriber.analyze_view_performance(view_events)
 
-        # Stop memory tracking and get collected memory data
-        if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
+        # Per-phase allocation attribution (under memory tracking) — which phase
+        # allocated the request's objects (sql / view / elasticsearch).
+        allocation_phases = if DeadBro.configuration.memory_tracking_enabled && defined?(DeadBro::MemoryPhaseTracker)
+          DeadBro::MemoryPhaseTracker.stop_request_tracking
+        else
+          {}
+        end
+
+        # Stop memory tracking and get collected memory data. The decision to do
+        # heavy allocation tracking was made (and sampled) at request start.
+        if Thread.current[:dead_bro_alloc_active] && defined?(DeadBro::MemoryTrackingSubscriber)
           detailed_memory = DeadBro::MemoryTrackingSubscriber.stop_request_tracking
           memory_performance = DeadBro::MemoryTrackingSubscriber.analyze_memory_performance(detailed_memory)
+
+          # Allocation-source + by-bytes-type diagnostics. Read the trace data
+          # before stopping the sampler (stop clears the source locations).
+          if defined?(DeadBro::AllocationSourceSampler)
+            growth = (detailed_memory[:memory_after].to_f - detailed_memory[:memory_before].to_f)
+            source_analysis = DeadBro::AllocationSourceSampler.analyze(memory_growth_mb: growth)
+            DeadBro::AllocationSourceSampler.stop
+            if source_analysis.is_a?(Hash) && source_analysis.any?
+              memory_performance[:allocation_sources] = source_analysis[:by_source]
+              memory_performance[:memsize_by_type] = source_analysis[:by_type_bytes]
+              memory_performance[:allocation_sample_rate] = source_analysis[:sample_rate]
+              memory_performance[:allocation_sources_skipped] = source_analysis[:skipped] if source_analysis[:skipped]
+            end
+          end
+
           # Keep memory_events compact and user-friendly (no large raw arrays)
           memory_events = {
             memory_before: detailed_memory[:memory_before],
@@ -165,6 +189,7 @@ module DeadBro
           method: data[:method],
           path: safe_path(data),
           status: data[:status],
+          started_at: started.utc.iso8601(3),
           duration_ms: duration_ms,
           view_runtime_ms: data[:view_runtime],
           db_runtime_ms: data[:db_runtime],
@@ -188,6 +213,7 @@ module DeadBro
           view_performance: view_performance,
           memory_events: memory_events,
           memory_performance: memory_performance,
+          allocation_phases: allocation_phases,
           rack_duration_ms: rack_duration_ms,
           queue_duration_ms: Thread.current[:dead_bro_queue_duration_ms],
           db_connection_wait_ms: db_connection_stats[:wait_ms],
@@ -211,9 +237,12 @@ module DeadBro
       DeadBro::ElasticsearchSubscriber.stop_request_tracking if defined?(DeadBro::ElasticsearchSubscriber)
       DeadBro::ViewRenderingSubscriber.stop_request_tracking if defined?(DeadBro::ViewRenderingSubscriber)
       DeadBro::LightweightMemoryTracker.stop_request_tracking if defined?(DeadBro::LightweightMemoryTracker)
-      if DeadBro.configuration.allocation_tracking_enabled && defined?(DeadBro::MemoryTrackingSubscriber)
-        DeadBro::MemoryTrackingSubscriber.stop_request_tracking
+      DeadBro::MemoryPhaseTracker.stop_request_tracking if defined?(DeadBro::MemoryPhaseTracker)
+      if Thread.current[:dead_bro_alloc_active]
+        DeadBro::MemoryTrackingSubscriber.stop_request_tracking if defined?(DeadBro::MemoryTrackingSubscriber)
+        DeadBro::AllocationSourceSampler.stop if defined?(DeadBro::AllocationSourceSampler)
       end
+      Thread.current[:dead_bro_alloc_active] = nil
       Thread.current[:dead_bro_http_events] = nil
       DeadBro::DbConnectionSubscriber.stop_request_tracking if defined?(DeadBro::DbConnectionSubscriber)
       DeadBro::GcTracker.stop_request_tracking if defined?(DeadBro::GcTracker)
@@ -246,14 +275,11 @@ module DeadBro
       # Remove router-provided keys that we already send at top-level
       router_keys = %w[controller action format]
 
-      # Filter out sensitive parameters
-      sensitive_keys = %w[password password_confirmation token secret key]
-
       filtered = params.dup
       router_keys.each { |k| filtered.delete(k) || filtered.delete(k.to_sym) }
-      filtered = filtered.except(*sensitive_keys, *sensitive_keys.map(&:to_sym)) if filtered.respond_to?(:except)
 
-      # Truncate deeply to keep payload small and safe
+      # Truncate deeply to keep payload small and safe. truncate_value also redacts
+      # sensitive keys at every nesting level (e.g. user[password]).
       truncate_value(filtered)
     rescue
       {}
@@ -263,7 +289,18 @@ module DeadBro
       str.to_s.gsub("\x00", "")
     end
 
-    # Recursively truncate values to reasonable sizes to avoid huge payloads
+    # Matched against a key segment (delimited by start/end, underscore, dash, or
+    # bracket) so nested and prefixed/suffixed keys are caught — e.g. user[password],
+    # access_token, client_secret — without redacting innocent keys like
+    # passenger_count or cardinality.
+    SENSITIVE_SEGMENT_RE = /(?:\A|[_\-\[])(password|passwd|secret|token|api_?key|access_?key|auth|authorization|credential|ssn|credit_?card|card_?number|cvv|cvc)(?:\z|[_\-\]])/i
+
+    def self.sensitive_key?(key)
+      SENSITIVE_SEGMENT_RE.match?(key.to_s)
+    end
+
+    # Recursively truncate values to reasonable sizes to avoid huge payloads, and
+    # redact values whose key looks sensitive at any nesting level.
     def self.truncate_value(value, max_str: 200, max_array: 20, max_hash_keys: 30)
       case value
       when String
@@ -276,7 +313,7 @@ module DeadBro
       when Hash
         entries = value.to_a[0, max_hash_keys]
         entries.each_with_object({}) do |(k, v), memo|
-          memo[k] = truncate_value(v, max_str: max_str, max_array: max_array, max_hash_keys: max_hash_keys)
+          memo[k] = sensitive_key?(k) ? "[FILTERED]" : truncate_value(v, max_str: max_str, max_array: max_array, max_hash_keys: max_hash_keys)
         end
       else
         (value.to_s.length > max_str) ? value.to_s[0, max_str] + "…" : value.to_s

data/lib/dead_bro/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DeadBro
-  VERSION = "0.2.23"
+  VERSION = "0.2.25"
 end

data/lib/dead_bro.rb

@@ -19,6 +19,8 @@ module DeadBro
   autoload :MemoryLeakDetector, "dead_bro/memory_leak_detector"
   autoload :LightweightMemoryTracker, "dead_bro/lightweight_memory_tracker"
   autoload :GcTracker, "dead_bro/gc_tracker"
+  autoload :MemoryPhaseTracker, "dead_bro/memory_phase_tracker"
+  autoload :AllocationSourceSampler, "dead_bro/allocation_source_sampler"
   autoload :ArObjectTracker, "dead_bro/ar_object_tracker"
   autoload :CpuTracker, "dead_bro/cpu_tracker"
   autoload :MemoryHelpers, "dead_bro/memory_helpers"

data/lib/generators/dead_bro/install/install_generator.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+
+module DeadBro
+  module Generators
+    class InstallGenerator < ::Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Creates a DeadBro initializer in config/initializers/dead_bro.rb"
+
+      def create_initializer
+        template "dead_bro.rb", "config/initializers/dead_bro.rb"
+      end
+    end
+  end
+end

data/lib/generators/dead_bro/install/templates/dead_bro.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+DeadBro.configure do |cfg|
+  cfg.api_key = ENV["DEAD_BRO_API_KEY"]
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dead_bro
 version: !ruby/object:Gem::Version
-  version: 0.2.23
+  version: 0.2.25
 platform: ruby
 authors:
 - Emanuel Comsa
@@ -18,9 +18,9 @@ extensions: []
 extra_rdoc_files: []
 files:
 - CHANGELOG.md
-- FEATURES.md
 - README.md
 - lib/dead_bro.rb
+- lib/dead_bro/allocation_source_sampler.rb
 - lib/dead_bro/ar_object_tracker.rb
 - lib/dead_bro/cache_subscriber.rb
 - lib/dead_bro/circuit_breaker.rb
@@ -48,6 +48,7 @@ files:
 - lib/dead_bro/memory_details.rb
 - lib/dead_bro/memory_helpers.rb
 - lib/dead_bro/memory_leak_detector.rb
+- lib/dead_bro/memory_phase_tracker.rb
 - lib/dead_bro/memory_tracking_subscriber.rb
 - lib/dead_bro/monitor.rb
 - lib/dead_bro/railtie.rb
@@ -57,6 +58,8 @@ files:
 - lib/dead_bro/subscriber.rb
 - lib/dead_bro/version.rb
 - lib/dead_bro/view_rendering_subscriber.rb
+- lib/generators/dead_bro/install/install_generator.rb
+- lib/generators/dead_bro/install/templates/dead_bro.rb
 homepage: https://www.deadbro.com
 licenses: []
 metadata:
@@ -76,7 +79,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.9
+rubygems_version: 4.0.10
 specification_version: 4
 summary: Minimal APM for Rails apps.
 test_files: []

data/FEATURES.md

@@ -1,333 +0,0 @@
-# ApmBro Feature List
-
-A comprehensive feature list for comparing ApmBro with other APM (Application Performance Monitoring) tools.
-
-## Core Architecture
-
-- **Rails Integration**: Automatic subscription to Rails events via ActiveSupport::Notifications
-- **Zero-Configuration Setup**: Works out of the box with minimal configuration
-- **Asynchronous Metrics Posting**: Non-blocking HTTP requests using background threads
-- **Thread-Local Storage**: Per-request metric collection using thread-local variables
-- **Circuit Breaker Pattern**: Built-in circuit breaker to prevent cascading failures when APM endpoint is down
-- **Deploy Tracking**: Automatic deploy ID resolution from multiple sources (Rails settings, ENV vars, Heroku, Git)
-
-## Request Tracking
-
-### Controller Action Monitoring
-- **Automatic Tracking**: Tracks all controller actions automatically
-- **Request Duration**: Measures total request processing time
-- **HTTP Method & Path**: Captures HTTP method and request path
-- **Status Codes**: Tracks HTTP response status codes
-- **View Runtime**: Separate tracking of view rendering time
-- **Database Runtime**: Separate tracking of database query time
-- **Request Parameters**: Captures request parameters (with sensitive data filtering)
-- **User Agent**: Tracks user agent strings
-- **User ID Extraction**: Extracts authenticated user ID (supports Warden)
-- **Environment Context**: Tracks Rails environment (development, staging, production)
-
-### Request Sampling
-- **Configurable Sample Rate**: Percentage-based sampling (1-100%)
-- **Random Sampling**: Each request has random chance of being tracked
-- **Consistent Per-Request**: Sampling decision applies to all metrics for a request
-- **Error Override**: Errors are always tracked regardless of sampling
-- **Cost Optimization**: Reduces data volume and costs for high-traffic applications
-
-### Exclusion Rules
-- **Controller Exclusion**: Exclude entire controllers from tracking
-- **Action Exclusion**: Exclude specific controller#action combinations
-- **Wildcard Support**: Pattern matching with `*` wildcards (e.g., `Admin::*`, `Admin::*#*`)
-- **Job Exclusion**: Exclude specific background jobs from tracking
-- **Flexible Configuration**: Configure via initializer, Rails settings, or environment variables
-
-## SQL Query Tracking
-
-### Query Details
-- **Full SQL Tracking**: Captures all SQL queries executed during requests and jobs
-- **Query Sanitization**: Automatically sanitizes SQL to remove sensitive data
-- **Query Name**: Tracks query names (e.g., "User Load", "User Update")
-- **Duration Measurement**: Precise query execution time in milliseconds
-- **Cache Detection**: Identifies cached queries
-- **Connection ID**: Tracks database connection ID
-- **Call Stack Traces**: Full backtrace showing where queries were executed
-- **Object Allocations**: Optional tracking of object allocations per query
-
-### Query Performance Analysis
-- **Slow Query Detection**: Configurable threshold for identifying slow queries
-- **EXPLAIN ANALYZE**: Automatic execution plan capture for slow queries
-- **Background Execution**: EXPLAIN ANALYZE runs in separate thread (non-blocking)
-- **Multi-Database Support**: Works with PostgreSQL, MySQL, SQLite, and others
-- **Smart Filtering**: Automatically skips transaction queries (BEGIN, COMMIT, ROLLBACK)
-- **Execution Plan Details**: 
-  - PostgreSQL: Full EXPLAIN ANALYZE with buffer usage statistics
-  - MySQL: EXPLAIN ANALYZE with actual execution times
-  - SQLite: EXPLAIN QUERY PLAN output
-- **Query Optimization Insights**: Helps identify missing indexes, full table scans, JOIN issues
-
-## View Rendering Tracking
-
-### View Performance
-- **Template Rendering**: Tracks main template rendering
-- **Partial Rendering**: Tracks partial template rendering with cache key information
-- **Collection Rendering**: Tracks collection rendering (partials in loops)
-- **Rendering Duration**: Precise timing for each view component
-- **Virtual Path Tracking**: Tracks view virtual paths
-- **Layout Information**: Captures layout usage
-
-### View Analysis
-- **Slow View Detection**: Identifies the slowest rendering views
-- **Frequency Analysis**: Tracks most frequently rendered views
-- **Cache Hit Rate**: Calculates cache hit rates for partials
-- **Collection Cache Analysis**: Tracks cache hit rates for collection rendering
-- **Performance Metrics**: 
-  - Total views rendered per request
-  - Total view rendering duration
-  - Average view rendering duration
-  - Breakdown by view type (template, partial, collection)
-
-## Memory Tracking & Leak Detection
-
-### Lightweight Memory Tracking (Default)
-- **Memory Usage Monitoring**: Tracks memory consumption per request using GC stats
-- **Memory Growth Tracking**: Measures memory growth during request processing
-- **GC Statistics**: Tracks garbage collection count and heap pages
-- **Minimal Performance Impact**: ~0.1ms overhead per request
-- **Memory Before/After**: Captures memory state at request start and end
-
-### Detailed Allocation Tracking (Optional)
-- **Object Allocation Tracking**: Detailed tracking of object allocations (disabled by default)
-- **Allocation Sampling**: Configurable sampling rate for allocations
-- **Large Object Detection**: Identifies objects larger than 1MB threshold
-- **Memory Snapshots**: Periodic memory snapshots during request processing
-- **Object Count Tracking**: Tracks object counts before and after requests
-- **Performance Impact**: ~2-5ms overhead per request (only when enabled)
-
-### Memory Leak Detection
-- **Pattern Detection**: Detects growing memory patterns over time
-- **GC Efficiency Analysis**: Monitors garbage collection effectiveness
-- **Heap Page Tracking**: Tracks heap page growth
-- **Request Correlation**: Correlates memory growth with specific controllers/actions
-
-## Background Job Tracking
-
-### Job Execution Monitoring
-- **ActiveJob Integration**: Automatic tracking when ActiveJob is available
-- **Job Class Tracking**: Tracks job class names
-- **Job ID**: Captures unique job identifiers
-- **Queue Name**: Tracks which queue processed the job
-- **Job Arguments**: Captures job arguments (with sensitive data filtering)
-- **Duration Measurement**: Precise job execution time in milliseconds
-- **Status Tracking**: Tracks job status (completed or failed)
-
-### Job Error Tracking
-- **Exception Capture**: Captures exceptions from failed jobs
-- **Exception Class**: Tracks exception class names
-- **Exception Messages**: Captures exception messages (truncated to 1000 chars)
-- **Backtraces**: Full exception backtraces (first 50 lines)
-- **SQL Query Context**: Includes SQL queries executed during failed jobs
-- **Memory Context**: Includes memory usage during job execution
-
-### Job SQL Tracking
-- **SQL Query Tracking**: Tracks all SQL queries executed during job processing
-- **Query Details**: Same detailed SQL tracking as request tracking
-- **Query Context**: Full context of database operations in background jobs
-
-## Cache Tracking
-
-### Cache Operations
-- **Read Operations**: Tracks cache read operations
-- **Write Operations**: Tracks cache write operations
-- **Delete Operations**: Tracks cache delete operations
-- **Existence Checks**: Tracks cache existence checks
-- **Fetch Operations**: Tracks cache fetch with hit/miss detection
-- **Multi-Read Operations**: Tracks cache read_multi operations
-- **Multi-Write Operations**: Tracks cache write_multi operations
-- **Cache Generation**: Tracks cache generation events
-
-### Cache Analysis
-- **Cache Hit Detection**: Identifies cache hits vs misses
-- **Cache Key Tracking**: Tracks cache keys (truncated to 200 chars)
-- **Store Information**: Identifies which cache store was used
-- **Namespace Tracking**: Tracks cache namespaces
-- **Duration Measurement**: Precise timing for each cache operation
-- **Hit Rate Calculation**: Calculates cache hit rates per request
-
-## Redis Tracking
-
-### Redis Command Tracking
-- **Command Monitoring**: Tracks all Redis commands executed
-- **Command Name**: Captures Redis command names (GET, SET, etc.)
-- **Key Tracking**: Tracks Redis keys (truncated to 200 chars)
-- **Argument Count**: Tracks number of arguments per command
-- **Database Selection**: Tracks which Redis database is used
-- **Duration Measurement**: Precise timing for each Redis command
-- **Error Tracking**: Captures Redis command errors
-
-### Advanced Redis Features
-- **Pipeline Support**: Tracks Redis pipeline operations with command counts
-- **Multi/Transaction Support**: Tracks Redis MULTI/EXEC transactions
-- **ActiveSupport Integration**: Subscribes to ActiveSupport::Notifications for Redis events
-- **Client Instrumentation**: Direct instrumentation of Redis::Client for comprehensive coverage
-
-## Error Tracking
-
-### Exception Handling
-- **Automatic Exception Capture**: Captures exceptions from controller actions
-- **Exception Class**: Tracks exception class names
-- **Exception Messages**: Captures exception messages (truncated to 1000 chars)
-- **Full Backtraces**: Captures complete exception backtraces (first 50 lines)
-- **Request Context**: Includes full request context with exceptions
-- **Error Flagging**: Errors are marked and always tracked (even with sampling)
-
-### Error Context
-- **Controller/Action**: Identifies where the error occurred
-- **Request Parameters**: Includes request parameters at time of error
-- **User Information**: Includes user ID if available
-- **SQL Queries**: Includes SQL queries executed before error
-- **Memory State**: Includes memory usage at time of error
-- **Log Messages**: Includes application logs captured during request
-
-## HTTP Instrumentation
-
-### Outgoing HTTP Tracking
-- **HTTP Request Tracking**: Tracks outgoing HTTP requests (via middleware)
-- **Request Context**: Captures HTTP request details
-- **Response Context**: Captures HTTP response details
-- **Duration Measurement**: Tracks HTTP request duration
-
-## Configuration & Flexibility
-
-### Configuration Options
-- **API Key Management**: Multiple sources (config, Rails credentials, ENV)
-- **Endpoint Configuration**: Configurable endpoint URL
-- **Timeout Settings**: Configurable open and read timeouts
-- **Enable/Disable Toggle**: Can be enabled/disabled via configuration
-- **Environment Detection**: Automatic Rails environment detection
-
-### Circuit Breaker Configuration
-- **Failure Threshold**: Configurable failure threshold (default: 3)
-- **Recovery Timeout**: Configurable recovery timeout (default: 60 seconds)
-- **Retry Timeout**: Configurable retry timeout (default: 300 seconds)
-- **Enable/Disable**: Can enable/disable circuit breaker
-
-### Memory Tracking Configuration
-- **Memory Tracking Toggle**: Enable/disable memory tracking
-- **Allocation Tracking Toggle**: Enable/disable detailed allocation tracking
-- **Sampling Configuration**: Configurable request sampling rate
-
-### Query Analysis Configuration
-- **Slow Query Threshold**: Configurable threshold in milliseconds (default: 500ms)
-- **EXPLAIN ANALYZE Toggle**: Enable/disable automatic EXPLAIN ANALYZE
-
-## Data Safety & Privacy
-
-### Data Sanitization
-- **SQL Sanitization**: Automatically sanitizes SQL queries
-- **Parameter Filtering**: Filters sensitive parameters (password, token, secret, key)
-- **Argument Truncation**: Limits and truncates job arguments
-- **Key Truncation**: Truncates cache and Redis keys to 200 characters
-- **Value Truncation**: Recursively truncates nested values to prevent huge payloads
-- **String Limits**: Limits string values (e.g., user agent to 200 chars, messages to 1000 chars)
-
-### Data Limits
-- **Array Limits**: Limits array sizes (e.g., first 10 job arguments, first 5 array elements)
-- **Hash Limits**: Limits hash key counts (e.g., first 20 hash keys, first 30 params)
-- **Backtrace Limits**: Limits backtraces to first 50 lines
-- **Allocation Limits**: Limits allocations tracked per request (max 1000)
-
-## Performance & Reliability
-
-### Performance Optimizations
-- **Asynchronous Posting**: Non-blocking HTTP requests
-- **Lightweight Default Mode**: Minimal overhead in default configuration
-- **Sampling Support**: Reduces data volume for high-traffic applications
-- **Thread-Local Storage**: Efficient per-request data collection
-- **Background EXPLAIN**: EXPLAIN ANALYZE runs in background thread
-
-### Reliability Features
-- **Circuit Breaker**: Prevents cascading failures
-- **Error Handling**: Comprehensive error handling to prevent instrumentation failures
-- **Graceful Degradation**: Continues working even if some features fail
-- **Timeout Protection**: Configurable timeouts prevent hanging requests
-
-## Integration & Compatibility
-
-### Framework Support
-- **Rails Integration**: Full Rails integration via Railtie
-- **ActiveSupport Notifications**: Uses ActiveSupport::Notifications for event subscription
-- **ActiveRecord Integration**: Tracks ActiveRecord SQL queries
-- **ActiveJob Integration**: Tracks ActiveJob background jobs
-- **ActionView Integration**: Tracks ActionView rendering
-
-### Database Support
-- **PostgreSQL**: Full support with EXPLAIN ANALYZE
-- **MySQL**: Full support with EXPLAIN ANALYZE
-- **SQLite**: Full support with EXPLAIN QUERY PLAN
-- **Other Databases**: Basic support with standard EXPLAIN
-
-### Cache Store Support
-- **All Cache Stores**: Works with any Rails cache store
-- **Multi-Store Support**: Tracks cache operations across different stores
-
-### Redis Support
-- **Redis Gem**: Works with redis gem
-- **Client Instrumentation**: Direct instrumentation of Redis::Client
-- **Pipeline Support**: Tracks Redis pipelines
-- **Transaction Support**: Tracks Redis MULTI/EXEC transactions
-
-## Logging & Debugging
-
-### Application Logging
-- **Log Capture**: Captures application logs during request processing
-- **Log Context**: Includes logs in metric payloads
-- **Debug Logging**: Optional debug logging for skipped requests
-
-## Deployment & Environment
-
-### Deploy Tracking
-- **Deploy ID Resolution**: Multiple sources for deploy identification (`Configuration#deploy_id=` wins when set, then ENV in `Configuration::DEPLOY_REVISION_ENV_KEYS` order—including `DEAD_BRO_DEPLOY_ID`, git/CI vars, `DD_VERSION`, etc.), otherwise a **per-process UUID** (fine for single dyno/process; unusable alone for fleets like ECS replicas)
-- **Revision Tracking**: Includes deploy/revision ID in all metric payloads
-
-### Environment Support
-- **Rails Environment**: Automatic Rails environment detection
-- **Rack Environment**: Fallback to RACK_ENV or RAILS_ENV
-- **Environment Context**: Includes environment in all metric payloads
-
-## Data Collection & Transmission
-
-### Metric Payload Structure
-- **Structured Data**: Well-structured JSON payloads
-- **Event Names**: Descriptive event names for different metric types
-- **Timestamp Tracking**: ISO8601 timestamps for all metrics
-- **Metadata**: Rich metadata including environment, host, deploy ID
-
-### HTTP Client
-- **HTTPS Support**: Secure HTTPS communication
-- **Bearer Token Auth**: API key authentication via Bearer tokens
-- **JSON Encoding**: JSON-encoded payloads
-- **Custom Headers**: Proper Content-Type and Authorization headers
-
-## Comparison-Ready Features
-
-### Unique Differentiators
-1. **Automatic EXPLAIN ANALYZE**: Background execution plan capture for slow queries
-2. **Lightweight Memory Tracking**: Low-overhead memory monitoring by default
-3. **Comprehensive Cache Tracking**: Detailed cache operation tracking
-4. **Redis Instrumentation**: Full Redis command tracking including pipelines
-5. **View Rendering Analysis**: Detailed view performance analysis with cache hit rates
-6. **Flexible Exclusion Rules**: Wildcard support for controller/job exclusion
-7. **Request Sampling**: Configurable percentage-based sampling
-8. **Circuit Breaker**: Built-in resilience for APM endpoint failures
-9. **Multi-Source Configuration**: Flexible configuration from multiple sources
-10. **Deploy Tracking**: Automatic deploy ID resolution from multiple sources
-
-### Standard APM Features
-- Request/response tracking
-- SQL query tracking
-- Error tracking
-- Background job tracking
-- Memory tracking
-- Performance metrics
-- Exception handling
-- User context
-- Environment tracking
-