debugbundle 1.4.1 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cae31875b7ffe9d2005b62f2107fb4810061ec4362e7bf48c4dd521e43d415f6
-  data.tar.gz: 2e005a0112e3197aa9b19fed791b71cea7dbf1eefcfc6edebc5a2ce7c764f3d0
+  metadata.gz: 154069d58e3649c8b90ab0b5567749fef0b0ae27756354467d2f54e347c20bdb
+  data.tar.gz: 10364db06a596a352b0b96cdfda8b54fe6ec228ab2a4682b89740d38ca0f2a09
 SHA512:
-  metadata.gz: e2f99cad665ebbcd10ea0965ed97f16c1930567344848ead03aa2ff9d1ac297bf9260ba2915a72454d6843e2805e41ebc648525ab0250077c29ea3f8897a291a
-  data.tar.gz: 25a171fc0e6cd96c74a8cc7a85fde1dbfe52837d4c46cef69926e2b4c736df49912ffce294613b3c7738d4f6f7b8cdc2f4121f6898a1563c4e7056f237d52a55
+  metadata.gz: 529aa156d0361a75d06a6b9c1f1dd9950269c1c0c82a97367436a6987a8546c0f9274631c1027e0e5585147b18e34ce5a441eca453891a442e105617370f373f
+  data.tar.gz: 3d768b54c0ca11c5ed227bbe02ece3ad61a40a77597028366e900c1cc42a4e4777bb93fc837dfc2eeee030e610c0756993fb363f0d2eab5e17457b34b0c66e02

data/README.md

@@ -4,6 +4,12 @@ Ruby SDK for DebugBundle.
 
 Use this gem to capture Ruby backend exceptions, request metadata, structured logs, runtime context, probe data, and browser relay traffic. It supports a singleton facade plus instance clients for Rack, Rails, Sidekiq, and explicit Ruby instrumentation.
 
+The source test matrix covers Ruby 3.1 through 4.0. Rails 7.x remains an installed-base lane; the current Rails 8.1 lane exercises the relay and Rack middleware on Ruby 4.0. `make compat` runs the Rack, Rails, and Sidekiq Docker lanes, including Rails 8.1.
+
+Version 2.0 changes capture and hook timing. Existing installed 1.5.x applications continue to use their pinned gem until upgraded. Review [the migration guide](MIGRATION-2.0.md) before upgrading.
+
+`make sdk-safety-perf` runs a Docker-backed host-safety budget with 10,000 filtered INFO calls, 1,000 accepted ERROR calls with and without custom privacy fields, and 10,000 concurrent ERROR calls against a full queue while transport is held. CI and the release workflow run this gate before publication. These bounded synthetic checks supplement, but do not replace, installed runtime and workload qualification.
+
 ## Automatic capture and application filtering
 
 `capture_logger(logger)` respects the logger level, Rails silence and disabled output. Lazy message blocks are evaluated by the native logger: zero times when suppressed and once when accepted. Native return values and application block exceptions are preserved. DebugBundle applies its own configured minimum capture level after the logger accepts a message.
@@ -38,7 +44,7 @@ DebugBundle.capture_message("worker started", level: :info)
 DebugBundle.flush
 ```
 
-`DebugBundle.init(...)` arms best-effort process exception capture automatically through `at_exit` and thread exception hooks.
+`DebugBundle.init(...)` arms best-effort process exception capture automatically through `at_exit` and thread exception hooks. These hooks wake the sender without waiting for delivery; shutdown exceptions can be lost if the process exits before the sender finishes. Automatic delivery runs on one background sender thread when the configured batch size or flush interval is reached. Remote configuration uses a separate bounded poller so a slow fetch cannot hold delivery. An explicit `flush` waits for the sender for at most five seconds and reports whether delivery completed. Call it during a controlled shutdown window when delivery is required. A forked child starts fresh workers on its first SDK use and drops inherited parent events, context, and probes while retaining the parent's capture restrictions until remote configuration refreshes.
 
 ## Framework Integrations
 
@@ -73,7 +79,7 @@ Capture-policy fields are server-owned and must not be supplied in local SDK con
 | `local_events_dir` | `.debugbundle/local/events` | Local event file destination. |
 | `spool_dir` | `.debugbundle/local/browser-relay-spool` | Relay durable spool destination. |
 | `redact_fields` | `[]` | Additional sensitive field names merged with built-in redaction defaults. Rails `filter_parameters` are added automatically. |
-| `batch_size` | `25` | Max events per flush batch. |
+| `batch_size` | `25` | Wake the background sender when this many events are buffered. |
 | `flush_interval` | `5` | Flush interval in seconds. |
 | `sample_rate` | `1.0` | Fraction of events kept before transport. |
 | `log_level` | `warning` | Minimum captured log severity. |
@@ -318,10 +324,10 @@ This repository also ships a clean-install app-driven smoke harness that validat
 
 ```sh
 make smoke
-make smoke-published VERSION=1.4.1
+make smoke-published VERSION=1.5.0
 ```
 
-`make smoke` builds the gem, installs it into a fresh RubyGems home, drives a Rack request plus a browser relay batch through the public SDK surface, validates event envelope shape, and confirms the mock ingestion endpoint receives the expected service, environment, SDK metadata, and correlation fields.
+`make smoke` builds the gem, installs it into a fresh RubyGems home, drives a Rack request plus a browser relay batch through the public SDK surface, validates event envelope shape, and confirms the mock ingestion endpoint receives the expected service, environment, SDK metadata, and correlation fields. For version 2.0, it also forks the installed client and verifies exactly one child event without replaying the parent's buffered event.
 
 ## Examples
 
@@ -350,7 +356,7 @@ The repository ships a GitHub Actions release workflow at `.github/workflows/rel
 
 - Push a `v*` tag or run the workflow manually with a `version` input.
 - Configure the `RUBYGEMS_API_KEY` repository secret before enabling publish.
-- The workflow runs lint, tests, gem build, `make smoke`, RubyGems publish, and `make smoke-published VERSION=<tag>` before creating the GitHub release.
+- The workflow runs lint, tests, the full `make compat` framework/runtime matrix, gem build, `make smoke`, RubyGems publish, and `make smoke-published VERSION=<tag>` before creating the GitHub release.
 
 ## Documentation
 

data/lib/debugbundle/before_send.rb

@@ -27,7 +27,7 @@ module DebugBundle
         rejection_reason dom_context probe_data
       ],
       'deploy_metadata' => %w[commit_sha version branch environment deployed_at],
-      'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen device],
+      'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen reason level device],
       'probe_event' => %w[label data activation_id probe_label_pattern device]
     }.freeze
     ROOT_FIELDS = %w[

data/lib/debugbundle/client.rb

@@ -1,19 +1,28 @@
 # frozen_string_literal: true
 
 require 'digest'
+require 'json'
+require 'set'
 require 'time'
-require 'uri'
 
 require 'debugbundle/client_event_support'
+require 'debugbundle/client_process_support'
+require 'debugbundle/client_queue_support'
+require 'debugbundle/config_worker'
+require 'debugbundle/delivery_worker'
 require 'debugbundle/runtime'
 
 module DebugBundle
   class Client
+    include QueueSupport
+    include ProcessSupport
+
     SCHEMA_VERSION = '2026-03-01'
     SDK_NAME = '@debugbundle/sdk-ruby'
     DEFAULT_SERVICE_NAME = 'ruby-service'
     DEFAULT_ENVIRONMENT = 'development'
     MAX_BUFFER_SIZE = 1_000
+    MAX_BUFFER_BYTES = 8 * 1_024 * 1_024
     RETRY_AFTER_CAP_SECONDS = 300
     DEFAULT_HEADER_ALLOWLIST = %w[
       user-agent
@@ -80,6 +89,7 @@ module DebugBundle
     end
 
     def initialize(transport: nil, time_provider: nil, random_provider: nil, config_fetcher: nil, **options)
+      @owner_pid = Process.pid
       @config = Config.new(**options)
       @time_provider = time_provider || -> { Time.now.utc }
       @random_provider = random_provider || -> { rand }
@@ -90,6 +100,14 @@ module DebugBundle
       @config_fetcher = config_fetcher || build_default_config_fetcher(custom_transport: !transport.nil?)
       @context = {}
       @buffer = []
+      @buffer_bytes = 0
+      @buffer_priority_counts = [0, 0, 0, 0]
+      @inflight_event_ids = Set.new
+      @inflight_priority_counts = [0, 0, 0, 0]
+      @event_sizes = {}
+      @pressure_drops = {}
+      @hook_bypass = Set.new
+      @finalized_events = {}
       @buffer_mutex = Mutex.new
       @flush_mutex = Mutex.new
       @probe_buffers = {}
@@ -105,10 +123,22 @@ module DebugBundle
       @next_remote_config_poll_at = nil
       @remote_config_etag = nil
       @remote_config = RemoteConfig::Snapshot.default
-      @capture_policy = @remote_config.capture_policy
+      @capture_policy = @config_fetcher ? RemoteConfig.minimal_capture_policy : @remote_config.capture_policy
+      @initial_remote_config_pending = !@config_fetcher.nil?
+      @config_worker = nil
+      return unless capture_enabled?
 
-      refresh_remote_config!
-      @capture_policy = RemoteConfig.minimal_capture_policy if @config_fetcher && @remote_config_etag.nil?
+      @delivery_worker = DeliveryWorker.new(
+        interval: config.flush_interval,
+        before_work: -> {},
+        &method(:flush_now)
+      )
+      return unless @config_fetcher
+
+      @config_worker = ConfigWorker.new(
+        next_wait_seconds: method(:next_remote_config_wait_seconds),
+        &method(:refresh_remote_config_on_worker)
+      )
     end
 
     def capture_exception(error, context: nil, handled: true)
@@ -117,14 +147,15 @@ module DebugBundle
 
     def capture_exception_internal(error, context:, handled:, run_before_send:)
       return unless capture_enabled?
+      return unless preflight_capacity?(3, 'exception')
 
-      poll_remote_config_if_due!
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       payload = {
-        'name' => error.class.name,
-        'message' => error.message.to_s,
-        'stack' => Array(error.backtrace).join("\n"),
+        'name' => safe_exception_name(error),
+        'message' => safe_exception_message(error),
+        'stack' => safe_exception_stack(error),
         'handled' => handled,
         'request' => request_payload(merged_context['request']),
         'response' => response_payload(merged_context['response']),
@@ -140,8 +171,6 @@ module DebugBundle
       extra_context['causes'] = causes unless causes.empty?
 
       event = base_event('backend_exception', payload, extra_context)
-      event = apply_before_send(event) if run_before_send
-      return if event.nil?
 
       event_payload = event.fetch('payload')
       suppression_key = [
@@ -152,7 +181,7 @@ module DebugBundle
       ].join(':')
       return unless @suppression.should_capture(suppression_key, now: monotonic_now)
 
-      enqueue_event(event)
+      enqueue_event(event, skip_before_send: !run_before_send)
     end
     private :capture_exception_internal
 
@@ -161,26 +190,39 @@ module DebugBundle
     def capture_log(message, level: :warning, context: nil)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
-
       normalized_level = normalize_level(level || :warning)
+      return unless level_enabled?(normalized_level)
+
+      priority = LOG_LEVEL_RANKS.fetch(normalized_level) >= LOG_LEVEL_RANKS.fetch(:error) ? 2 : 0
+      return unless preflight_capacity?(priority, normalized_level.to_s)
+
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       payload = {
         'level' => normalized_level.to_s,
-        'message' => message.to_s,
+        'message' => safe_log_message(message),
         'attributes' => merged_context
       }
-      event = apply_before_send(base_event('log_event', payload, merged_context))
-      return if event.nil? || !level_enabled?(normalized_level)
-
-      enqueue_event(event)
+      enqueue_event(base_event('log_event', payload, merged_context))
     end
 
     def capture_request(request, response, context: nil)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
+      status = if response.is_a?(Hash)
+                 response[:status_code] || response['status_code'] || response[:status] || response['status']
+               end
+      priority = if status.is_a?(Integer)
+                   status >= 400 ? 2 : 1
+                 elsif response.nil?
+                   1
+                 else
+                   3
+                 end
+      return unless preflight_capacity?(priority, 'request')
+
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       sanitized_request = request_payload(request)
@@ -199,12 +241,9 @@ module DebugBundle
         'response_headers' => sanitized_response['headers'],
         'response_body' => sanitized_response['body']
       }
-      event = apply_before_send(
-        base_event('request_event', payload, merged_context.merge('request' => sanitized_request))
-      )
-      return if event.nil? || !capture_request_event?(response_status, sanitized_request)
+      return unless capture_request_event?(response_status, sanitized_request)
 
-      enqueue_event(event)
+      enqueue_event(base_event('request_event', payload, merged_context.merge('request' => sanitized_request)))
     end
 
     def capture_message(message, level: nil, context: nil)
@@ -212,14 +251,23 @@ module DebugBundle
     end
 
     def set_context(key, value)
-      @context[key.to_s] = @redactor.redact_value(value)
+      ensure_current_process!
+      safe_key = SafeInput.key(key)
+      return value unless safe_key
+
+      safe = TelemetryPrivacy.protect(
+        { safe_key => @redactor.redact_value(value) }, additional_fields: config.redact_fields
+      )
+      @context[safe_key] = safe[safe_key] if safe.key?(safe_key)
+      value
+    rescue StandardError
       value
     end
 
     def probe(label, data = nil, heavy: false, &block)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
+      request_remote_config_poll_if_due
       return unless @remote_config.probes_enabled
 
       matching_directives = matching_probe_directives(label)
@@ -238,17 +286,20 @@ module DebugBundle
       resolved, raw_value = resolve_probe_value(data, block)
       return unless resolved
 
+      safe_label = TelemetryPrivacy.protect(label.to_s, additional_fields: config.redact_fields)
       entry = {
-        'label' => label.to_s,
+        'label' => safe_label,
         'data' => normalize_probe_data(raw_value),
         'timestamp' => now.iso8601
       }
 
-      bucket = (@probe_buffers[label.to_s] ||= [])
+      bucket = (@probe_buffers[safe_label] ||= [])
       bucket << entry
       bucket.shift while bucket.length > config.max_probe_entries_per_label
 
-      emit_probe_events(label.to_s, entry['data'], matching_directives)
+      emit_probe_events(safe_label, entry['data'], matching_directives)
+    rescue StandardError
+      nil
     end
 
     def capture_exceptions
@@ -274,7 +325,9 @@ module DebugBundle
           handled: false,
           run_before_send: false
         )
-        client.flush
+        client.__send__(:wake_sender)
+      rescue StandardError
+        nil
       end
       true
     end
@@ -292,7 +345,7 @@ module DebugBundle
     end
 
     def with_request_trigger(request)
-      poll_remote_config_if_due! if capture_enabled?
+      request_remote_config_poll_if_due if capture_enabled?
 
       directives = TriggerToken.resolve_request_directives(
         request: request,
@@ -344,19 +397,46 @@ module DebugBundle
     end
 
     def flush
+      ensure_current_process!
+      @delivery_worker&.flush || false
+    end
+
+    def close
+      ensure_current_process!
+      @delivery_worker&.close
+      @config_worker&.close
+    end
+
+    def flush_now
       # rubocop:disable Metrics/BlockLength
       @flush_mutex.synchronize do
         append_suppression_aggregates
-        batch = buffered_batch
-        return true if batch.empty?
+        append_pressure_aggregates
+        candidates = reserve_buffered_batch
+        return true if candidates.empty?
         return false if @transport.nil?
         return false if rate_limited?
 
+        # Replace consumed snapshot slots so dropped events do not remain owned
+        # by this batch while later callbacks run or transport waits.
+        prepared = candidates.map! do |event|
+          finalized = finalized_event_for(event)
+          if finalized.nil?
+            remove_buffered_events([event])
+            next
+          end
+          [event, finalized]
+        end.compact
+        return true if prepared.empty?
+
+        batch = prepared.map(&:first)
+        wire_events = prepared.map(&:last)
+
         result = Transport.coerce_result(
           @transport.call(
             project_token: config.project_token,
             service_name: service_name,
-            events: batch.map(&:dup)
+            events: wire_events.map(&:dup)
           )
         )
 
@@ -382,9 +462,43 @@ module DebugBundle
     rescue StandardError
       @consecutive_failures += 1
       false
+    ensure
+      release_buffered_batch
     end
+    private :flush_now
+
+    def finalized_event_for(event)
+      event_id = event['event_id']
+      cached, bypass = @buffer_mutex.synchronize do
+        [@finalized_events[event_id], @hook_bypass.include?(event_id)]
+      end
+      return cached if cached
+
+      finalized = bypass || !config.before_send ? event : apply_before_send(event)
+      return nil if finalized.nil?
+      return nil unless post_hook_event_allowed?(finalized)
+
+      cache_finalized_event(event, finalized)
+    end
+    private :finalized_event_for
+
+    def post_hook_event_allowed?(event)
+      case event['event_type']
+      when 'log_event'
+        return false if @capture_policy.capture_logs == 'off'
+
+        level_enabled?(normalize_level(event.dig('payload', 'level')))
+      when 'request_event'
+        payload = event['payload']
+        capture_request_event?(payload['response_status'].to_i, payload)
+      else
+        true
+      end
+    end
+    private :post_hook_event_allowed?
 
     def status
+      ensure_current_process!
       return :disconnected unless config.enabled?
       return :degraded unless config.configured?
       return @acknowledgement_state if @acknowledgement_state
@@ -394,7 +508,10 @@ module DebugBundle
       :healthy
     end
 
-    def buffered_event_count = @buffer_mutex.synchronize { @buffer.length }
+    def buffered_event_count
+      ensure_current_process!
+      @buffer_mutex.synchronize { @buffer.length }
+    end
 
     private
 
@@ -497,8 +614,8 @@ module DebugBundle
     def matching_immediate_client_error_path_rule?(status_code, request)
       return false unless (400..499).cover?(status_code)
 
-      path = normalize_request_path(request['path'] || request['url'])
-      method = request['method'].to_s.upcase
+      path = SafeInput.request_path(request['path'] || request['url'])
+      method = SafeInput.key(request['method'])&.upcase || ''
       Array(@capture_policy.immediate_client_error_path_rules).any? do |rule|
         next false unless rule.status_code == status_code
         next false if !rule.http_methods.empty? && !rule.http_methods.include?(method)
@@ -511,19 +628,6 @@ module DebugBundle
       end
     end
 
-    def normalize_request_path(value)
-      begin
-        uri = URI.parse(value.to_s)
-        return uri.path if uri.path && !uri.path.empty?
-      rescue URI::InvalidURIError
-        # Fall through to the lightweight path-only fallback.
-      end
-      fallback = value.to_s.split('?', 2).first.to_s.split('#', 2).first
-      return fallback if fallback.start_with?('/') && !fallback.empty?
-
-      '/'
-    end
-
     def matching_probe_directives(label)
       active_directives = @remote_config.directives + current_request_trigger_directives
 
@@ -546,21 +650,16 @@ module DebugBundle
       return if candidate_directives.empty?
 
       candidate_directives.each do |directive|
-        event = apply_before_send(
-          base_event(
-            'probe_event',
-            {
-              'label' => label,
-              'data' => data,
-              'activation_id' => directive.id,
-              'probe_label_pattern' => directive.label_pattern
-            },
-            {}
-          )
-        )
         allowed = request_directives.include?(directive) ||
                   @capture_policy.capture_probe_events == 'standalone_when_activated'
-        enqueue_event(event) if event && allowed
+        next unless allowed
+
+        enqueue_event(base_event('probe_event', {
+                                   'label' => label,
+                                   'data' => data,
+                                   'activation_id' => directive.id,
+                                   'probe_label_pattern' => directive.label_pattern
+                                 }, {}))
       end
     end
 
@@ -575,7 +674,22 @@ module DebugBundle
 
     def local_environment? = LOCAL_ENVIRONMENTS.include?(environment_name.to_s)
 
-    def poll_remote_config_if_due!
+    def request_remote_config_poll_if_due
+      return unless @next_remote_config_poll_at && @next_remote_config_poll_at <= now
+
+      @config_worker&.wake
+    end
+
+    def refresh_remote_config_on_worker
+      if @initial_remote_config_pending
+        @initial_remote_config_pending = false
+        refresh_remote_config!
+      else
+        refresh_remote_config_if_due!
+      end
+    end
+
+    def refresh_remote_config_if_due!
       return unless @config_fetcher
       return unless @next_remote_config_poll_at && @next_remote_config_poll_at <= now
 
@@ -592,6 +706,12 @@ module DebugBundle
       @next_remote_config_poll_at = interval_seconds ? now + interval_seconds : nil
     end
 
+    def next_remote_config_wait_seconds
+      return nil unless @next_remote_config_poll_at
+
+      [@next_remote_config_poll_at - now, 0].max
+    end
+
     def capture_thread_exceptions
       return false if @thread_exception_registered
 
@@ -604,11 +724,15 @@ module DebugBundle
 
     def capture_thread_exception(error)
       capture_exception_internal(error, context: nil, handled: false, run_before_send: false)
-      flush
+      wake_sender
     rescue StandardError
       nil
     end
 
+    def wake_sender
+      @delivery_worker&.wake
+    end
+
     def now = @time_provider.call
 
     def monotonic_now = Process.clock_gettime(Process::CLOCK_MONOTONIC)