debugbundle 1.5.0 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f8040734d7a652566f0c06328713b3b7fa5bc559b1e30f1432c45b71f8decab
-  data.tar.gz: b53fba03cb4013cb375610e1e9acef0264f79cc87ad98c987b72ae1968e9a411
+  metadata.gz: de914935eb26ac5a4a4c7a21e91e13f92fcdef0949c8a19cc9566adf70fa7b5d
+  data.tar.gz: ff2d61415de5d618556d00f4918e4d3a29af38ae7be2bddafc088304a7b11e2b
 SHA512:
-  metadata.gz: e76d4d31fb538e7a9261b3a544e51440f9406c4ba3fc61508dd34477052e84d4036955568188f1baf988c6a30c30fa8fdbe08d4f286a26131adac3f9ef0f26aa
-  data.tar.gz: 64d7f18d44af78e8bd267fde9b967996d90b308efc0df021fa3d1cc2205f0cad69929a9b896ad5819df0ee4618ca399f9f6229dd372ce956e50397e770cd3db0
+  metadata.gz: 7655a43d83ce82e3d7cd6febebc560a2c6c31dde18df9b3a579897fa236c1094c36bc170aa203f1a412caca0ff86c1d6bf529745e9b69e48d55c88799ba84531
+  data.tar.gz: 06b7e321fe0bae17dfe91bd4ffecd4288f84c0151a9909def3d18c02fc1e5f4e262171b7318a2bed337876a0ed7740c9d1d31d0ab518b187a05f72e0db0fcf49

data/README.md

@@ -4,7 +4,11 @@ Ruby SDK for DebugBundle.
 
 Use this gem to capture Ruby backend exceptions, request metadata, structured logs, runtime context, probe data, and browser relay traffic. It supports a singleton facade plus instance clients for Rack, Rails, Sidekiq, and explicit Ruby instrumentation.
 
-The source test matrix covers Ruby 3.1 through 4.0. Rails 7.x remains an installed-base lane; the current Rails 8.1 lane exercises the relay and Rack middleware on Ruby 4.0.
+The source test matrix covers Ruby 3.1 through 4.0. Rails 7.x remains an installed-base lane; the current Rails 8.1 lane exercises the relay and Rack middleware on Ruby 4.0. `make compat` runs the Rack, Rails, and Sidekiq Docker lanes, including Rails 8.1.
+
+Version 2.0 changes capture and hook timing. Existing installed 1.5.x applications continue to use their pinned gem until upgraded. Review [the migration guide](MIGRATION-2.0.md) before upgrading.
+
+`make sdk-safety-perf` runs a Docker-backed host-safety budget with 10,000 filtered INFO calls, 1,000 accepted ERROR calls with and without custom privacy fields, and 10,000 concurrent ERROR calls against a full queue while transport is held. CI and the release workflow run this gate before publication. These bounded synthetic checks supplement, but do not replace, installed runtime and workload qualification.
 
 ## Automatic capture and application filtering
 
@@ -40,7 +44,7 @@ DebugBundle.capture_message("worker started", level: :info)
 DebugBundle.flush
 ```
 
-`DebugBundle.init(...)` arms best-effort process exception capture automatically through `at_exit` and thread exception hooks.
+`DebugBundle.init(...)` arms best-effort process exception capture automatically through `at_exit` and thread exception hooks. These hooks wake the sender without waiting for delivery; shutdown exceptions can be lost if the process exits before the sender finishes. Automatic delivery runs on one background sender thread when the configured batch size or flush interval is reached. Remote configuration uses a separate bounded poller so a slow fetch cannot hold delivery. An explicit `flush` waits for the sender for at most five seconds and reports whether delivery completed. Call it during a controlled shutdown window when delivery is required. A forked child starts fresh workers on its first SDK use and drops inherited parent events, context, and probes while retaining the parent's capture restrictions until remote configuration refreshes.
 
 ## Framework Integrations
 
@@ -75,7 +79,7 @@ Capture-policy fields are server-owned and must not be supplied in local SDK con
 | `local_events_dir` | `.debugbundle/local/events` | Local event file destination. |
 | `spool_dir` | `.debugbundle/local/browser-relay-spool` | Relay durable spool destination. |
 | `redact_fields` | `[]` | Additional sensitive field names merged with built-in redaction defaults. Rails `filter_parameters` are added automatically. |
-| `batch_size` | `25` | Max events per flush batch. |
+| `batch_size` | `25` | Wake the background sender when this many events are buffered. |
 | `flush_interval` | `5` | Flush interval in seconds. |
 | `sample_rate` | `1.0` | Fraction of events kept before transport. |
 | `log_level` | `warning` | Minimum captured log severity. |
@@ -323,7 +327,7 @@ make smoke
 make smoke-published VERSION=1.5.0
 ```
 
-`make smoke` builds the gem, installs it into a fresh RubyGems home, drives a Rack request plus a browser relay batch through the public SDK surface, validates event envelope shape, and confirms the mock ingestion endpoint receives the expected service, environment, SDK metadata, and correlation fields.
+`make smoke` builds the gem, installs it into a fresh RubyGems home, drives a Rack request plus a browser relay batch through the public SDK surface, validates event envelope shape, and confirms the mock ingestion endpoint receives the expected service, environment, SDK metadata, and correlation fields. For version 2.0, it also forks the installed client and verifies exactly one child event without replaying the parent's buffered event.
 
 ## Examples
 
@@ -352,7 +356,7 @@ The repository ships a GitHub Actions release workflow at `.github/workflows/rel
 
 - Push a `v*` tag or run the workflow manually with a `version` input.
 - Configure the `RUBYGEMS_API_KEY` repository secret before enabling publish.
-- The workflow runs lint, tests, gem build, `make smoke`, RubyGems publish, and `make smoke-published VERSION=<tag>` before creating the GitHub release.
+- The workflow runs lint, tests, the full `make compat` framework/runtime matrix, gem build, `make smoke`, RubyGems publish, and `make smoke-published VERSION=<tag>` before creating the GitHub release.
 
 ## Documentation
 
@@ -364,3 +368,9 @@ The repository ships a GitHub Actions release workflow at `.github/workflows/rel
 ## License
 
 Apache-2.0. See LICENSE.
+
+## Delivery acknowledgement and retry limits
+
+The built-in HTTP transport requires the canonical ingestion acknowledgement (`accepted`, `rejected`, and `errors`). Empty responses, unrelated JSON, malformed counts, or invalid rejection indices retain the full batch with backoff. Valid acknowledgements remove accepted and terminally rejected events and retry only the indexed retryable rejections; an all-rejected batch does not advance `lastEventAt`.
+
+File transports and explicitly supplied custom/legacy transports retain their documented bodyless success fallback. A custom transport that returns acknowledgement fields must return the complete canonical shape. Retry hints (delay seconds or HTTP dates) are bounded to five minutes and measured from response receipt, including partial/protocol acknowledgements and retryable server failures. Without a server hint, existing retry timing is preserved. Failures remain contained within the SDK's existing delivery path.

data/lib/debugbundle/acknowledgement.rb

@@ -11,8 +11,8 @@ module DebugBundle
 
     module_function
 
-    def decide(body, batch_length)
-      return { kind: :legacy } unless body.is_a?(Hash) && FIELDS.any? { |field| body.key?(field) }
+    def decide(body, batch_length, required: false)
+      return missing_acknowledgement(required) unless body.is_a?(Hash) && FIELDS.any? { |field| body.key?(field) }
       return protocol_failure unless FIELDS.all? { |field| body.key?(field) }
 
       accepted = body['accepted']
@@ -55,5 +55,9 @@ module DebugBundle
     def protocol_failure
       { kind: :protocol_failure }
     end
+
+    def missing_acknowledgement(required)
+      required ? protocol_failure : { kind: :legacy }
+    end
   end
 end

data/lib/debugbundle/before_send.rb

@@ -27,7 +27,7 @@ module DebugBundle
         rejection_reason dom_context probe_data
       ],
       'deploy_metadata' => %w[commit_sha version branch environment deployed_at],
-      'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen device],
+      'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen reason level device],
       'probe_event' => %w[label data activation_id probe_label_pattern device]
     }.freeze
     ROOT_FIELDS = %w[

data/lib/debugbundle/client.rb

@@ -1,19 +1,28 @@
 # frozen_string_literal: true
 
 require 'digest'
+require 'json'
+require 'set'
 require 'time'
-require 'uri'
 
 require 'debugbundle/client_event_support'
+require 'debugbundle/client_process_support'
+require 'debugbundle/client_queue_support'
+require 'debugbundle/config_worker'
+require 'debugbundle/delivery_worker'
 require 'debugbundle/runtime'
 
 module DebugBundle
   class Client
+    include QueueSupport
+    include ProcessSupport
+
     SCHEMA_VERSION = '2026-03-01'
     SDK_NAME = '@debugbundle/sdk-ruby'
     DEFAULT_SERVICE_NAME = 'ruby-service'
     DEFAULT_ENVIRONMENT = 'development'
     MAX_BUFFER_SIZE = 1_000
+    MAX_BUFFER_BYTES = 8 * 1_024 * 1_024
     RETRY_AFTER_CAP_SECONDS = 300
     DEFAULT_HEADER_ALLOWLIST = %w[
       user-agent
@@ -80,6 +89,7 @@ module DebugBundle
     end
 
     def initialize(transport: nil, time_provider: nil, random_provider: nil, config_fetcher: nil, **options)
+      @owner_pid = Process.pid
       @config = Config.new(**options)
       @time_provider = time_provider || -> { Time.now.utc }
       @random_provider = random_provider || -> { rand }
@@ -90,6 +100,14 @@ module DebugBundle
       @config_fetcher = config_fetcher || build_default_config_fetcher(custom_transport: !transport.nil?)
       @context = {}
       @buffer = []
+      @buffer_bytes = 0
+      @buffer_priority_counts = [0, 0, 0, 0]
+      @inflight_event_ids = Set.new
+      @inflight_priority_counts = [0, 0, 0, 0]
+      @event_sizes = {}
+      @pressure_drops = {}
+      @hook_bypass = Set.new
+      @finalized_events = {}
       @buffer_mutex = Mutex.new
       @flush_mutex = Mutex.new
       @probe_buffers = {}
@@ -105,10 +123,22 @@ module DebugBundle
       @next_remote_config_poll_at = nil
       @remote_config_etag = nil
       @remote_config = RemoteConfig::Snapshot.default
-      @capture_policy = @remote_config.capture_policy
+      @capture_policy = @config_fetcher ? RemoteConfig.minimal_capture_policy : @remote_config.capture_policy
+      @initial_remote_config_pending = !@config_fetcher.nil?
+      @config_worker = nil
+      return unless capture_enabled?
 
-      refresh_remote_config!
-      @capture_policy = RemoteConfig.minimal_capture_policy if @config_fetcher && @remote_config_etag.nil?
+      @delivery_worker = DeliveryWorker.new(
+        interval: config.flush_interval,
+        before_work: -> {},
+        &method(:flush_now)
+      )
+      return unless @config_fetcher
+
+      @config_worker = ConfigWorker.new(
+        next_wait_seconds: method(:next_remote_config_wait_seconds),
+        &method(:refresh_remote_config_on_worker)
+      )
     end
 
     def capture_exception(error, context: nil, handled: true)
@@ -117,14 +147,15 @@ module DebugBundle
 
     def capture_exception_internal(error, context:, handled:, run_before_send:)
       return unless capture_enabled?
+      return unless preflight_capacity?(3, 'exception')
 
-      poll_remote_config_if_due!
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       payload = {
-        'name' => error.class.name,
-        'message' => error.message.to_s,
-        'stack' => Array(error.backtrace).join("\n"),
+        'name' => safe_exception_name(error),
+        'message' => safe_exception_message(error),
+        'stack' => safe_exception_stack(error),
         'handled' => handled,
         'request' => request_payload(merged_context['request']),
         'response' => response_payload(merged_context['response']),
@@ -140,8 +171,6 @@ module DebugBundle
       extra_context['causes'] = causes unless causes.empty?
 
       event = base_event('backend_exception', payload, extra_context)
-      event = apply_before_send(event) if run_before_send
-      return if event.nil?
 
       event_payload = event.fetch('payload')
       suppression_key = [
@@ -152,7 +181,7 @@ module DebugBundle
       ].join(':')
       return unless @suppression.should_capture(suppression_key, now: monotonic_now)
 
-      enqueue_event(event)
+      enqueue_event(event, skip_before_send: !run_before_send)
     end
     private :capture_exception_internal
 
@@ -161,26 +190,39 @@ module DebugBundle
     def capture_log(message, level: :warning, context: nil)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
-
       normalized_level = normalize_level(level || :warning)
+      return unless level_enabled?(normalized_level)
+
+      priority = LOG_LEVEL_RANKS.fetch(normalized_level) >= LOG_LEVEL_RANKS.fetch(:error) ? 2 : 0
+      return unless preflight_capacity?(priority, normalized_level.to_s)
+
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       payload = {
         'level' => normalized_level.to_s,
-        'message' => message.to_s,
+        'message' => safe_log_message(message),
         'attributes' => merged_context
       }
-      event = apply_before_send(base_event('log_event', payload, merged_context))
-      return if event.nil? || !level_enabled?(normalized_level)
-
-      enqueue_event(event)
+      enqueue_event(base_event('log_event', payload, merged_context))
     end
 
     def capture_request(request, response, context: nil)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
+      status = if response.is_a?(Hash)
+                 response[:status_code] || response['status_code'] || response[:status] || response['status']
+               end
+      priority = if status.is_a?(Integer)
+                   status >= 400 ? 2 : 1
+                 elsif response.nil?
+                   1
+                 else
+                   3
+                 end
+      return unless preflight_capacity?(priority, 'request')
+
+      request_remote_config_poll_if_due
 
       merged_context = merge_context(context)
       sanitized_request = request_payload(request)
@@ -199,12 +241,9 @@ module DebugBundle
         'response_headers' => sanitized_response['headers'],
         'response_body' => sanitized_response['body']
       }
-      event = apply_before_send(
-        base_event('request_event', payload, merged_context.merge('request' => sanitized_request))
-      )
-      return if event.nil? || !capture_request_event?(response_status, sanitized_request)
+      return unless capture_request_event?(response_status, sanitized_request)
 
-      enqueue_event(event)
+      enqueue_event(base_event('request_event', payload, merged_context.merge('request' => sanitized_request)))
     end
 
     def capture_message(message, level: nil, context: nil)
@@ -212,10 +251,14 @@ module DebugBundle
     end
 
     def set_context(key, value)
+      ensure_current_process!
+      safe_key = SafeInput.key(key)
+      return value unless safe_key
+
       safe = TelemetryPrivacy.protect(
-        { key.to_s => @redactor.redact_value(value) }, additional_fields: config.redact_fields
+        { safe_key => @redactor.redact_value(value) }, additional_fields: config.redact_fields
       )
-      @context[key.to_s] = safe[key.to_s] if safe.key?(key.to_s)
+      @context[safe_key] = safe[safe_key] if safe.key?(safe_key)
       value
     rescue StandardError
       value
@@ -224,7 +267,7 @@ module DebugBundle
     def probe(label, data = nil, heavy: false, &block)
       return unless capture_enabled?
 
-      poll_remote_config_if_due!
+      request_remote_config_poll_if_due
       return unless @remote_config.probes_enabled
 
       matching_directives = matching_probe_directives(label)
@@ -282,7 +325,9 @@ module DebugBundle
           handled: false,
           run_before_send: false
         )
-        client.flush
+        client.__send__(:wake_sender)
+      rescue StandardError
+        nil
       end
       true
     end
@@ -300,7 +345,7 @@ module DebugBundle
     end
 
     def with_request_trigger(request)
-      poll_remote_config_if_due! if capture_enabled?
+      request_remote_config_poll_if_due if capture_enabled?
 
       directives = TriggerToken.resolve_request_directives(
         request: request,
@@ -352,29 +397,56 @@ module DebugBundle
     end
 
     def flush
+      ensure_current_process!
+      @delivery_worker&.flush || false
+    end
+
+    def close
+      ensure_current_process!
+      @delivery_worker&.close
+      @config_worker&.close
+    end
+
+    def flush_now
       # rubocop:disable Metrics/BlockLength
       @flush_mutex.synchronize do
         append_suppression_aggregates
-        batch = buffered_batch
-        return true if batch.empty?
+        append_pressure_aggregates
+        candidates = reserve_buffered_batch
+        return true if candidates.empty?
         return false if @transport.nil?
         return false if rate_limited?
 
+        # Replace consumed snapshot slots so dropped events do not remain owned
+        # by this batch while later callbacks run or transport waits.
+        prepared = candidates.map! do |event|
+          finalized = finalized_event_for(event)
+          if finalized.nil?
+            remove_buffered_events([event])
+            next
+          end
+          [event, finalized]
+        end.compact
+        return true if prepared.empty?
+
+        batch = prepared.map(&:first)
+        wire_events = prepared.map(&:last)
+
         result = Transport.coerce_result(
           @transport.call(
             project_token: config.project_token,
             service_name: service_name,
-            events: batch.map(&:dup)
+            events: wire_events.map(&:dup)
           )
         )
 
         case result.status_code
         when 200..299
           handle_successful_result(result, batch)
-        when 429
+        when 429, 500..599
           @consecutive_failures += 1
-          retry_after_seconds = (result.retry_after_seconds || 1).clamp(1, RETRY_AFTER_CAP_SECONDS)
-          @retry_at = now + retry_after_seconds
+          retry_after_seconds = bounded_retry_hint(result.retry_after_seconds)
+          @retry_at = now + retry_after_seconds if result.status_code == 429 || !result.retry_after_seconds.nil?
           false
         when 400..499
           remove_buffered_events(batch)
@@ -390,9 +462,43 @@ module DebugBundle
     rescue StandardError
       @consecutive_failures += 1
       false
+    ensure
+      release_buffered_batch
+    end
+    private :flush_now
+
+    def finalized_event_for(event)
+      event_id = event['event_id']
+      cached, bypass = @buffer_mutex.synchronize do
+        [@finalized_events[event_id], @hook_bypass.include?(event_id)]
+      end
+      return cached if cached
+
+      finalized = bypass || !config.before_send ? event : apply_before_send(event)
+      return nil if finalized.nil?
+      return nil unless post_hook_event_allowed?(finalized)
+
+      cache_finalized_event(event, finalized)
     end
+    private :finalized_event_for
+
+    def post_hook_event_allowed?(event)
+      case event['event_type']
+      when 'log_event'
+        return false if @capture_policy.capture_logs == 'off'
+
+        level_enabled?(normalize_level(event.dig('payload', 'level')))
+      when 'request_event'
+        payload = event['payload']
+        capture_request_event?(payload['response_status'].to_i, payload)
+      else
+        true
+      end
+    end
+    private :post_hook_event_allowed?
 
     def status
+      ensure_current_process!
       return :disconnected unless config.enabled?
       return :degraded unless config.configured?
       return @acknowledgement_state if @acknowledgement_state
@@ -402,13 +508,22 @@ module DebugBundle
       :healthy
     end
 
-    def buffered_event_count = @buffer_mutex.synchronize { @buffer.length }
+    def buffered_event_count
+      ensure_current_process!
+      @buffer_mutex.synchronize { @buffer.length }
+    end
 
     private
 
+    def bounded_retry_hint(value)
+      return 1 unless value.is_a?(Numeric) && value.real? && value.finite?
+
+      value.clamp(1, RETRY_AFTER_CAP_SECONDS)
+    end
+
     def handle_successful_result(result, batch)
-      decision = Acknowledgement.decide(result.body, batch.length)
-      return handle_protocol_failure if decision[:kind] == :protocol_failure
+      decision = Acknowledgement.decide(result.body, batch.length, required: @transport.is_a?(Transport::HttpTransport))
+      return handle_protocol_failure(result.retry_after_seconds) if decision[:kind] == :protocol_failure
 
       if decision[:kind] == :legacy
         remove_buffered_events(batch)
@@ -423,7 +538,7 @@ module DebugBundle
 
       if retryable_events.any?
         @consecutive_failures += 1
-        @retry_at = now + 1
+        @retry_at = now + bounded_retry_hint(result.retry_after_seconds)
         @acknowledgement_state = :degraded
         false
       else
@@ -434,9 +549,9 @@ module DebugBundle
       end
     end
 
-    def handle_protocol_failure
+    def handle_protocol_failure(retry_hint)
       @consecutive_failures += 1
-      @retry_at = now + 1
+      @retry_at = now + bounded_retry_hint(retry_hint)
       @acknowledgement_state = :degraded
       false
     end
@@ -505,8 +620,8 @@ module DebugBundle
     def matching_immediate_client_error_path_rule?(status_code, request)
       return false unless (400..499).cover?(status_code)
 
-      path = normalize_request_path(request['path'] || request['url'])
-      method = request['method'].to_s.upcase
+      path = SafeInput.request_path(request['path'] || request['url'])
+      method = SafeInput.key(request['method'])&.upcase || ''
       Array(@capture_policy.immediate_client_error_path_rules).any? do |rule|
         next false unless rule.status_code == status_code
         next false if !rule.http_methods.empty? && !rule.http_methods.include?(method)
@@ -519,19 +634,6 @@ module DebugBundle
       end
     end
 
-    def normalize_request_path(value)
-      begin
-        uri = URI.parse(value.to_s)
-        return uri.path if uri.path && !uri.path.empty?
-      rescue URI::InvalidURIError
-        # Fall through to the lightweight path-only fallback.
-      end
-      fallback = value.to_s.split('?', 2).first.to_s.split('#', 2).first
-      return fallback if fallback.start_with?('/') && !fallback.empty?
-
-      '/'
-    end
-
     def matching_probe_directives(label)
       active_directives = @remote_config.directives + current_request_trigger_directives
 
@@ -554,21 +656,16 @@ module DebugBundle
       return if candidate_directives.empty?
 
       candidate_directives.each do |directive|
-        event = apply_before_send(
-          base_event(
-            'probe_event',
-            {
-              'label' => label,
-              'data' => data,
-              'activation_id' => directive.id,
-              'probe_label_pattern' => directive.label_pattern
-            },
-            {}
-          )
-        )
         allowed = request_directives.include?(directive) ||
                   @capture_policy.capture_probe_events == 'standalone_when_activated'
-        enqueue_event(event) if event && allowed
+        next unless allowed
+
+        enqueue_event(base_event('probe_event', {
+                                   'label' => label,
+                                   'data' => data,
+                                   'activation_id' => directive.id,
+                                   'probe_label_pattern' => directive.label_pattern
+                                 }, {}))
       end
     end
 
@@ -583,7 +680,22 @@ module DebugBundle
 
     def local_environment? = LOCAL_ENVIRONMENTS.include?(environment_name.to_s)
 
-    def poll_remote_config_if_due!
+    def request_remote_config_poll_if_due
+      return unless @next_remote_config_poll_at && @next_remote_config_poll_at <= now
+
+      @config_worker&.wake
+    end
+
+    def refresh_remote_config_on_worker
+      if @initial_remote_config_pending
+        @initial_remote_config_pending = false
+        refresh_remote_config!
+      else
+        refresh_remote_config_if_due!
+      end
+    end
+
+    def refresh_remote_config_if_due!
       return unless @config_fetcher
       return unless @next_remote_config_poll_at && @next_remote_config_poll_at <= now
 
@@ -600,6 +712,12 @@ module DebugBundle
       @next_remote_config_poll_at = interval_seconds ? now + interval_seconds : nil
     end
 
+    def next_remote_config_wait_seconds
+      return nil unless @next_remote_config_poll_at
+
+      [@next_remote_config_poll_at - now, 0].max
+    end
+
     def capture_thread_exceptions
       return false if @thread_exception_registered
 
@@ -612,11 +730,15 @@ module DebugBundle
 
     def capture_thread_exception(error)
       capture_exception_internal(error, context: nil, handled: false, run_before_send: false)
-      flush
+      wake_sender
     rescue StandardError
       nil
     end
 
+    def wake_sender
+      @delivery_worker&.wake
+    end
+
     def now = @time_provider.call
 
     def monotonic_now = Process.clock_gettime(Process::CLOCK_MONOTONIC)