debugbundle 2.0.0 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 154069d58e3649c8b90ab0b5567749fef0b0ae27756354467d2f54e347c20bdb
-  data.tar.gz: 10364db06a596a352b0b96cdfda8b54fe6ec228ab2a4682b89740d38ca0f2a09
+  metadata.gz: fa08dbcf220cb84131821cf546272d4d08d6f15f01ba4e8c7f41f77fbc53a1c0
+  data.tar.gz: fe1a46ab795df53b5dd1ac5bee6684b43ea35501d031a345b52c576e9a8881f9
 SHA512:
-  metadata.gz: 529aa156d0361a75d06a6b9c1f1dd9950269c1c0c82a97367436a6987a8546c0f9274631c1027e0e5585147b18e34ce5a441eca453891a442e105617370f373f
-  data.tar.gz: 3d768b54c0ca11c5ed227bbe02ece3ad61a40a77597028366e900c1cc42a4e4777bb93fc837dfc2eeee030e610c0756993fb363f0d2eab5e17457b34b0c66e02
+  metadata.gz: 69e7d24b4c4615dc8ff2da958c90427c0a19d9f781629ff31f9e2918912e97f4298d6a44bba5fd2f19e56cef5fcb426796ed26e7f71b73eefb6384eaa26e0ca7
+  data.tar.gz: 190296bc690b90b3113a59b99c7062ed5f3f95705bb633b4998577d884743f5f64570a727ca81334fb45fe7c3d02f13be4d38cda714244cca79998ad94a2887d

data/README.md

@@ -10,6 +10,114 @@ Version 2.0 changes capture and hook timing. Existing installed 1.5.x applicatio
 
 `make sdk-safety-perf` runs a Docker-backed host-safety budget with 10,000 filtered INFO calls, 1,000 accepted ERROR calls with and without custom privacy fields, and 10,000 concurrent ERROR calls against a full queue while transport is held. CI and the release workflow run this gate before publication. These bounded synthetic checks supplement, but do not replace, installed runtime and workload qualification.
 
+`make evidence-performance` compares the same exception/recovery calls with
+evidence disabled and enabled, including a full warning queue. It reports capture
+time, allocated Ruby objects and retained bytes, and verifies original-error
+priority and the existing queue/diagnostic ceilings. Setup and cleanup are excluded;
+the existing `sdk-safety-perf` timing limits remain unchanged.
+
+## Unreleased evidence candidate
+
+The working candidate adds structured exception evidence and explicit recovery;
+it is not a published release. Enrichment requires a valid capability descriptor
+from the destination on an existing config/ingestion response. No extra discovery
+request is made. Unknown paths keep ordinary exception capture. The optional
+`evidence_capabilities: { version: 1, features: [...] }` pin asserts that every
+receiver, relay, spool/local processor and reader on that path has been qualified;
+it does not upgrade any receiver. `capture_exception_evidence: false` disables
+the new exception fields independently.
+
+`capture_recovery_observations` defaults to `false`. With destination proof and
+the option enabled, report application-observed outcomes inside an operation:
+
+```ruby
+require "securerandom"
+
+client = DebugBundle::Client.new(
+  project_token: ENV.fetch("DEBUGBUNDLE_TOKEN"),
+  capture_recovery_observations: true
+)
+
+client.with_operation_context(kind: "job", name: "asset.prepare", limit_ms: 5000) do
+  begin
+    prepare_asset
+  rescue => error
+    failure_ref = SecureRandom.uuid
+    attempt_id = SecureRandom.uuid
+    client.capture_exception(error, context: { failure_ref: failure_ref })
+    client.capture_recovery(failure_ref: failure_ref, attempt_id: attempt_id, state: "attempt_started")
+    result = application_recovery
+    client.capture_recovery(failure_ref: failure_ref, attempt_id: attempt_id, state: "attempt_completed")
+    client.capture_recovery(
+      failure_ref: failure_ref, attempt_id: attempt_id,
+      state: result.usable? ? "recovery_observed" : "failure_persisted"
+    )
+  end
+end
+```
+
+The application owns recovery and its success check. `attempt_completed` and a
+successful HTTP status do not establish recovery. `attempt_failed` records a
+failed attempt. Observations need no subsequent exception and do not open or
+resolve incidents. Use fresh UUID-v4 failure, attempt and scope IDs. For reporting
+outside the active operation, provide the same explicit `evidence_scope` at
+failure capture and in each observation. Page scopes additionally require the
+same nonblank session correlation; the string `"0"` is valid. Recovery accepts a
+Hash or keyword fields; scope/context Hash keys may be strings or symbols.
+
+Operations retain only bounded kind/name/elapsed/known-limit and correlation
+metadata. They preserve the block's return value and exceptions. Rack/Rails
+request and Sidekiq job capture runs inside its owned scope; other Fibers/threads
+do not inherit it. Completed scopes and process resets revoke ownership. Graphs
+use retained scalar exception snapshots, with no additional custom rendering or
+source/local-variable inspection. Ruby's linear cause path has at most five
+nodes and 32 frames per node, within a combined 32 KiB evidence budget. Fatal
+hooks use at most two nodes, eight frames per node and 4 KiB of total new evidence;
+delivery remains best-effort at process exit. Original sanitised stacks remain.
+Engine metadata is preserved under `payload.runtime.framework_extras`.
+
+Diagnostic metadata and pending events share 64 KiB, at most 50 failure windows,
+five attempts and 20 observations per window, and a five-minute capture window.
+Policy `capture_diagnostic_context: false` and the minimal preset disable recovery.
+Canonical `before_send` replacements may change identity and event type; the SDK
+reapplies privacy, policy, path proof and bounded ownership. An acknowledged
+capability downgrade may retry an original exception once without optional
+evidence, using the same identity and no second hook invocation.
+
+Hook replacements pass the complete vendored canonical schema and its conditional
+frontend identity checks before and after mandatory privacy. Invalid replacements
+fall back to the protected original; explicit `nil` still drops an event. Existing
+queue-pressure aggregate `reason` and `level` fields remain supported.
+
+Outgoing requests stay within the receiver's 256 KiB body limit, counting actual
+escaped UTF-8 JSON. The sender keeps ordered byte-sized batches and schedules
+remaining work on its existing worker. Acknowledgement indices apply to each
+submitted chunk. Oversized optional evidence is removed first; a valid hook
+replacement that still cannot fit is withheld without restoring pre-hook content.
+Invalid or throwing hooks retain the established safe-original fallback. Flush remains a bounded,
+best-effort wait; queued tails can continue after it returns. Local file transport
+keeps its existing bounded ownership and file semantics; the HTTP body limit
+does not narrow admissible local captures.
+
+Native HTTP config/ACK replies request identity encoding and retain at most
+256 KiB before JSON parsing. Existing native connect/read timeouts remain five
+seconds; relay chunks and downgrade retries share one five-second monotonic budget,
+passed to native response reads and remaining connect/read timeouts. A body deadline
+is checked between streamed chunks. Native DNS/socket waits remain advisory. Explicit
+`flush` still waits at most five seconds. `close` stops this process's sender,
+discards queued work and retains active-send accounting until completion. A
+forked child follows the existing fresh-worker contract. `dispose` permanently
+retires a client across process generations; singleton `DebugBundle.init` disposes
+its superseded client. Reinstall explicitly configured logger/integration
+bindings for a replacement client. Independently constructed clients remain
+available for applications that intentionally own multiple projects.
+
+Rack and Rails browser relays preserve validated evidence and server-owned
+credentials. Downstream chunks account for server metadata expansion; a durable
+spool is marked delivered only when every chunk has a complete canonical
+acknowledgement. Legacy custom transport success remains compatible and cannot
+create a delivered marker. Final installed/runtime qualification is in progress.
+
 ## Automatic capture and application filtering
 
 `capture_logger(logger)` respects the logger level, Rails silence and disabled output. Lazy message blocks are evaluated by the native logger: zero times when suppressed and once when accepted. Native return values and application block exceptions are preserved. DebugBundle applies its own configured minimum capture level after the logger accepts a message.
@@ -368,3 +476,28 @@ The repository ships a GitHub Actions release workflow at `.github/workflows/rel
 ## License
 
 Apache-2.0. See LICENSE.
+
+## Delivery acknowledgement and retry limits
+
+The built-in HTTP transport requires the canonical ingestion acknowledgement (`accepted`, `rejected`, and `errors`). Empty responses, unrelated JSON, malformed counts, or invalid rejection indices retain the full batch with backoff. Valid acknowledgements remove accepted and terminally rejected events and retry only the indexed retryable rejections; an all-rejected batch does not advance `lastEventAt`.
+
+File transports and explicitly supplied custom/legacy transports retain their documented bodyless success fallback. A custom transport that returns acknowledgement fields must return the complete canonical shape. Retry hints (delay seconds or HTTP dates) are bounded to five minutes and measured from response receipt, including partial/protocol acknowledgements and retryable server failures. Without a server hint, existing retry timing is preserved. Failures remain contained within the SDK's existing delivery path.
+
+## Probe checkpoint guidance
+
+Application instrumentation supplies probe labels and bounded data. Deploy a new
+checkpoint before activating it. Remote activation affects future matching calls
+when policy permits; it does not inject code or recover unretained measurements.
+Built-in runtime/error/device collectors keep their documented SDK settings.
+
+Use fixed labels and allowlisted technical states: `media.preview.checkpoint`
+(stage/input kind/attempt), `auth.session.checkpoint` (observed session presence
+and result), `job.dependency.checkpoint` (named dependency/attempt/result), or
+`cache.preview.checkpoint` (hit/miss/coarse age bucket). Omit customer identifiers,
+tokens/cookies, cache keys, URLs and raw payloads. HTTP 200 or attempt completion
+alone does not establish semantic recovery.
+
+Empty probe data means no matching retained observation for the selected sample
+window. It does not confirm activation delivery to every SDK or execution of a
+label. Check the deployed SDK/wrapper version, instrumentation, scope, expiry,
+policy, sampling and delivery receipts. See the [probe recipes](https://debugbundle.com/docs/probes#bounded-checkpoint-recipes).

data/lib/debugbundle/acknowledgement.rb

@@ -11,8 +11,8 @@ module DebugBundle
 
     module_function
 
-    def decide(body, batch_length)
-      return { kind: :legacy } unless body.is_a?(Hash) && FIELDS.any? { |field| body.key?(field) }
+    def decide(body, batch_length, required: false)
+      return missing_acknowledgement(required) unless body.is_a?(Hash) && FIELDS.any? { |field| body.key?(field) }
       return protocol_failure unless FIELDS.all? { |field| body.key?(field) }
 
       accepted = body['accepted']
@@ -55,5 +55,9 @@ module DebugBundle
     def protocol_failure
       { kind: :protocol_failure }
     end
+
+    def missing_acknowledgement(required)
+      required ? protocol_failure : { kind: :legacy }
+    end
   end
 end

data/lib/debugbundle/before_send.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 require 'time'
+require 'debugbundle/evidence_wire'
+require 'debugbundle/canonical_event_schema'
 
 module DebugBundle
   module BeforeSend
@@ -12,10 +14,12 @@ module DebugBundle
       'frontend_exception' => %w[name message stack],
       'deploy_metadata' => %w[commit_sha version branch environment deployed_at],
       'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen],
-      'probe_event' => %w[label data activation_id probe_label_pattern]
+      'probe_event' => %w[label data activation_id probe_label_pattern],
+      'diagnostic_context' => %w[version kind failure_ref scope]
     }.freeze
     ALLOWED_PAYLOAD_FIELDS = {
-      'backend_exception' => %w[name message stack handled request response runtime probe_data],
+      'backend_exception' => %w[name message stack handled request response runtime probe_data exception_evidence
+                                execution],
       'request_event' => %w[
         method path query headers body response_status duration_ms route_template
         response_headers response_body device
@@ -24,17 +28,20 @@ module DebugBundle
       'frontend_breadcrumb' => %w[breadcrumb_type route data device],
       'frontend_exception' => %w[
         name message stack route browser breadcrumbs device browser_event
-        rejection_reason dom_context probe_data
+        rejection_reason dom_context probe_data exception_evidence resource_observations
       ],
       'deploy_metadata' => %w[commit_sha version branch environment deployed_at],
       'error_suppressed' => %w[fingerprint suppressed_count window_seconds first_seen last_seen reason level device],
-      'probe_event' => %w[label data activation_id probe_label_pattern device]
+      'probe_event' => %w[label data activation_id probe_label_pattern device],
+      'diagnostic_context' => %w[version kind failure_ref scope attempt_id state operation status_code duration_ms
+                                 source_event_id observations]
     }.freeze
     ROOT_FIELDS = %w[
       schema_version event_id event_type project_token project_id sdk_name sdk_version
       service occurred_at correlation context payload
     ].freeze
-    UUID_PATTERN = /\A[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\z/i
+    UUID_PATTERN = /\A[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\z/i
+    MOBILE_SDK_NAMES = %w[@debugbundle/sdk-android @debugbundle/sdk-swift @debugbundle/sdk-react-native].freeze
 
     module_function
 
@@ -44,35 +51,56 @@ module DebugBundle
       result = hook.call(Marshal.load(Marshal.dump(event)))
       return nil if result.nil?
 
-      valid?(result) ? result : event
+      checked = normalization_result(result)
+      return nil if checked.equal?(EvidenceValue::EXHAUSTED)
+
+      checked || event
     rescue StandardError
       event
     end
 
     def valid?(event)
-      return false unless event.is_a?(Hash)
-      return false unless (event.keys - ROOT_FIELDS).empty?
-      return false unless %w[schema_version event_id event_type occurred_at sdk_name sdk_version].all? do |field|
-        event[field].is_a?(String) && !event[field].empty?
-      end
-      return false unless UUID_PATTERN.match?(event['event_id'])
+      !normalized(event).nil?
+    end
+
+    def normalized(event)
+      result = normalization_result(event)
+      EvidenceValue.type?(result, Hash) ? result : nil
+    end
+
+    def normalization_result(event)
+      return nil unless EvidenceValue.type?(event, Hash)
+      return nil if Hash.instance_method(:size).bind_call(event) > ROOT_FIELDS.length
+
+      # Local/queued events keep their established ceiling independently of
+      # the smaller evidence DTO and HTTP budgets enforced at those boundaries.
+      owned = EvidenceValue.copy_with_budget(event, symbol_keys: true, max_bytes: 8 * 1024 * 1024,
+                                                    max_string_bytes: 8 * 1024 * 1024)
+      return owned if owned.equal?(EvidenceValue::EXHAUSTED)
+      return nil unless EvidenceValue.type?(owned, Hash)
+      return nil unless CanonicalEventSchema.valid?(owned) && EvidenceWire.valid_optional?(owned)
+      return nil unless required_frontend_context?(owned)
+
+      owned
+    rescue StandardError
+      nil
+    end
+
+    # The canonical Zod contract refines this conditional outside its shape;
+    # the generated JSON Schema alone cannot establish frontend identity.
+    def required_frontend_context?(event)
+      return true unless event['event_type'] == 'frontend_exception'
 
-      Time.iso8601(event['occurred_at'])
-      service = event['service']
       payload = event['payload']
-      fields = REQUIRED_PAYLOAD_FIELDS[event['event_type']]
-      allowed_fields = ALLOWED_PAYLOAD_FIELDS[event['event_type']]
-      service.is_a?(Hash) &&
-        service['name'].is_a?(String) && !service['name'].empty? &&
-        service['environment'].is_a?(String) && !service['environment'].empty? &&
-        payload.is_a?(Hash) &&
-        fields &&
-        allowed_fields &&
-        (payload.keys - allowed_fields).empty? &&
-        fields.all? { |field| payload.key?(field) } &&
-        valid_payload_shape?(event['event_type'], payload)
-    rescue ArgumentError
-      false
+      MOBILE_SDK_NAMES.include?(event['sdk_name']) ? payload['device'].is_a?(Hash) : payload.key?('browser')
+    end
+
+    # Existing shape helpers stay available to integrations. Full acceptance
+    # above uses the vendored canonical contract, including nested fields.
+    def valid_identity?(event)
+      %w[schema_version event_id event_type occurred_at sdk_name sdk_version].all? do |field|
+        event[field].is_a?(String) && !event[field].empty?
+      end && UUID_PATTERN.match?(event['event_id']) && EvidenceWire.timestamp(event['occurred_at'])
     end
 
     def valid_payload_shape?(event_type, payload)
@@ -93,6 +121,8 @@ module DebugBundle
         valid_error_suppressed?(payload)
       when 'probe_event'
         valid_probe_event?(payload)
+      when 'diagnostic_context'
+        EvidenceWire.diagnostic(payload)
       else
         false
       end
@@ -160,9 +190,7 @@ module DebugBundle
     end
 
     def timestamp?(value)
-      value.is_a?(String) && Time.iso8601(value)
-    rescue ArgumentError
-      false
+      EvidenceWire.timestamp(value)
     end
 
     def nullable_uuid?(value)

data/lib/debugbundle/bounded_http_response.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module DebugBundle
+  # Native HTTP adapters request identity encoding and stream existing replies.
+  # This cap precedes JSON parsing; no extra discovery request is introduced.
+  module BoundedHttpResponse
+    class LimitExceeded < StandardError; end
+
+    module_function
+
+    def read(deadline: Process.clock_gettime(Process::CLOCK_MONOTONIC) + 5)
+      body = +''
+      response = yield lambda { |chunk|
+        raise LimitExceeded if Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
+        raise LimitExceeded if body.bytesize + chunk.bytesize > 262_144
+
+        body << chunk
+      }
+      raise LimitExceeded if Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
+
+      [response, body]
+    end
+  end
+end

data/lib/debugbundle/canonical_event_schema.rb

@@ -0,0 +1,137 @@
+# frozen_string_literal: true
+
+require 'json'
+require 'debugbundle/evidence_wire'
+
+module DebugBundle
+  # A small closed interpreter for the vendored accepting Draft 7 contract.
+  # No remote refs, application serializers or runtime dependency are needed.
+  # Unknown schema keywords fail closed when a future contract is generated.
+  class CanonicalEventSchema
+    KEYWORDS = %w[$schema $id $comment $ref type anyOf const enum properties required additionalProperties
+                  items minimum maximum exclusiveMinimum minLength maxLength minItems maxItems format pattern].freeze
+    TYPES = %w[null boolean number integer string array object].freeze
+    LOAD_MUTEX = Mutex.new
+
+    def self.valid?(value)
+      unless defined?(@engine)
+        LOAD_MUTEX.synchronize do
+          unless defined?(@engine)
+            begin
+              @engine = new(JSON.parse(File.read(File.join(__dir__, 'event-envelope.schema.json'))))
+            rescue StandardError
+              @engine = nil
+            end
+          end
+        end
+      end
+      !!(@engine && @engine.accepts?(value))
+    rescue StandardError
+      false
+    end
+
+    def initialize(schema)
+      @root = schema
+      @references = {}
+      @patterns = {}
+      compile(schema)
+    end
+
+    def compile(node)
+      return if [true, false].include?(node)
+
+      raise ArgumentError, 'unsupported schema keyword' unless (node.keys - KEYWORDS).empty?
+      raise ArgumentError, 'unsupported schema type' unless (Array(node['type']) - TYPES).empty?
+      raise ArgumentError, 'unsupported schema format' unless [nil, 'uuid', 'date-time'].include?(node['format'])
+
+      @patterns[node['pattern']] ||= Regexp.new(node['pattern']) if node.key?('pattern')
+      resolve(node['$ref']) if node.key?('$ref')
+      node.fetch('properties', {}).each_value { |child| compile(child) }
+      node.fetch('anyOf', []).each { |child| compile(child) }
+      %w[items additionalProperties].each { |key| compile(node[key]) if node.key?(key) }
+    end
+
+    def resolve(reference)
+      raise ArgumentError, 'external schema ref' unless reference.start_with?('#/')
+
+      @references[reference] ||= reference.delete_prefix('#/').split('/').reduce(@root) do |node, part|
+        node.fetch(node.is_a?(Array) ? Integer(part) : part.gsub('~1', '/').gsub('~0', '~'))
+      end
+    end
+
+    def accepts?(value)
+      check(@root, value, { steps: 100_000 }, 0)
+    rescue StandardError
+      false
+    end
+
+    def check(node, value, budget, depth)
+      budget[:steps] -= 1
+      return false if depth > 64 || budget[:steps].negative?
+      return node if [true, false].include?(node)
+      return check(resolve(node['$ref']), value, budget, depth + 1) if node.key?('$ref')
+
+      return false if node.key?('anyOf') && node['anyOf'].none? { |child| check(child, value, budget, depth + 1) }
+      return false if node.key?('type') && Array(node['type']).none? { |type| matches_type?(value, type) }
+      return false if node.key?('const') && value != node['const']
+      return false if node.key?('enum') && !node['enum'].include?(value)
+
+      case value
+      when Hash then object?(node, value, budget, depth)
+      when Array then array?(node, value, budget, depth)
+      when String then string?(node, value)
+      when Integer, Float then number?(node, value)
+      else true
+      end
+    end
+
+    def matches_type?(value, type)
+      case type
+      when 'null' then value.nil?
+      when 'boolean' then [true, false].include?(value)
+      when 'string' then value.is_a?(String)
+      when 'object' then value.is_a?(Hash)
+      when 'array' then value.is_a?(Array)
+      when 'number' then (value.is_a?(Integer) || value.is_a?(Float)) && value.to_f.finite?
+      when 'integer' then matches_type?(value, 'number') && value == value.floor
+      else false
+      end
+    end
+
+    def object?(node, value, budget, depth)
+      return false unless node.fetch('required', []).all? { |key| value.key?(key) }
+
+      properties = node.fetch('properties', {})
+      value.all? do |key, child|
+        if properties.key?(key)
+          check(properties[key], child, budget, depth + 1)
+        else
+          check(node.fetch('additionalProperties', true), child, budget, depth + 1)
+        end
+      end
+    end
+
+    def array?(node, value, budget, depth)
+      return false if node.key?('minItems') && value.length < node['minItems']
+      return false if node.key?('maxItems') && value.length > node['maxItems']
+
+      !node.key?('items') || value.all? { |child| check(node['items'], child, budget, depth + 1) }
+    end
+
+    def string?(node, value)
+      return false if node.key?('minLength') && value.length < node['minLength']
+      return false if node.key?('maxLength') && value.length > node['maxLength']
+      return false if node.key?('pattern') && !@patterns.fetch(node['pattern']).match?(value)
+      return EvidenceWire.uuid(value) if node['format'] == 'uuid'
+      return EvidenceWire.timestamp(value) if node['format'] == 'date-time'
+
+      true
+    end
+
+    def number?(node, value)
+      (!node.key?('minimum') || value >= node['minimum']) &&
+        (!node.key?('maximum') || value <= node['maximum']) &&
+        (!node.key?('exclusiveMinimum') || value > node['exclusiveMinimum'])
+    end
+  end
+end