logstruct 0.2.2 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c9e72813c2641a2cc336184ab2e6f15b0d38e1f005da1ade37e136487a9c4369
-  data.tar.gz: b91cbaa7d64ea075db6937d415db0d0e7f86e58902083d54e0b985570a2573c5
+  metadata.gz: 647c97ba4368b67d39333204ca35a5161b0905dda2c8ef72d01e15d58665d367
+  data.tar.gz: e86e688d516ef584c5f8994ecb0739599c09595a021ebc949f494e6d631f349d
 SHA512:
-  metadata.gz: cb4d9ebb39a98d73f022c853daac9069599fe83528d0ca6234f2f63517803f48df932762827b37e5b4aae3b89e1da16c6d3b85005f54da7cab95dcfe8ee263d0
-  data.tar.gz: 7843625a239673d36ac68c092568927a9f5c112c830dd1b00caaf3c0e9d495fd6c9fb7514c18b5119000dc940f9a6706bbc2fb987506d55fefdd6264d9f0c568
+  metadata.gz: 6709f7868f43c348891761749400bb93c762e3df6f2d7093dab12f51c50a2ce33e26bfd94dab009093a1c4207f746735764c56ce20768394a9b8e81cce5d346d
+  data.tar.gz: 7ab05d71a64c08cd4ad747c7684d3b75553e2740c0539cefa2cab2f906f2c5023dd8696bb78129790472896f83008addde117f9e41229d563ad9003a76911fd9

data/CHANGELOG.md

@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-10-23
+
+### Changed
+
+- **Breaking**: CarrierWave lines log `file_id` as an HMAC of the file's identifier (keyed like email hashes, see `hash_salt`), and no longer log `filename`, `store_path` or `ext`. CarrierWave names stored files after the uploaded file by default, and an uploaded file's name often names a person. `LogField::Filename`, `LogField::StorePath` and `LogField::Extension` are removed.
+- **Breaking**: A filtered Hash's summary has its `_keys_count` but not its first 10 keys (`_keys`): a Hash's keys can be secrets themselves, like tokens or email addresses used as keys.
+- **Breaking**: `additional_data` can't set any field of its line's own type, whether or not the line has it (an error line's `msg`, `backtrace` or `error_message`), or `req_id`. Like `src`, `evt`, `lvl` and `ts`, such a key is left out and handled as a LogStruct error (raised in `local_environments`). It used to replace the field. Other types' field names, like `name` on a plain line, can still be used.
+- **Filtering**: Without a `hash_salt`, values hashed before the Rails app has its `secret_key_base` use a random key for each process, instead of a fixed key that was public in LogStruct's source. Set `hash_salt` to keep these hashes the same across processes.
+- **Development**: `task ci` checks the locked Ruby and docs dependencies for known advisories, with `bundler-audit` and `pnpm audit`.
+
+### Fixed
+
+- **Formatting**: Hash keys longer than `max_string_bytes` are left out with their values, and counted in `_oversized_keys`. A request's parameter names could make a line any size.
+- **Formatting**: `ActionController::Parameters` are written like a Hash, within the size limits, instead of being converted whole first. Lograge's `params` and Ahoy's `properties` are converted at their first level only, and only as many keys as the line keeps; `_more_keys` and `_oversized_keys` count the rest.
+- **Filtering**: A Rails filter block with three arguments gets a copy of the parameters that stops at 10,000 values, 20 levels of nesting or a value that contains itself. Past those, every key the block would check is filtered. The parameters used to be copied whole.
+- **Errors**: An error raised while formatting, serializing or filtering a value is reported with its class and backtrace but not its message, which can quote the value. Errors in Rails filter blocks and LogStruct's filter matchers aren't reported with the key, which can be the secret. An object whose `as_json` raises is logged as `[SERIALIZATION_ERROR]` instead of its `to_s`, and its `inspect` isn't reported.
+- **Formatting**: A Hash key that isn't a String or Symbol is written as its string, scrubbed, so an email address in its `to_s` is scrubbed.
+- **Filtering**: A value longer than `max_string_bytes` is hashed by its first `max_string_bytes` bytes and its length.
+- **Development**: Update Next.js to 16.4.1 for the docs site, which fixes three advisories in features the static site doesn't use, and the test-only Dalli to 3.2.13.
+
 ## [0.2.2] - 2026-10-21
 
 ### Changed

data/lib/log_struct/bounded_input.rb

@@ -0,0 +1,101 @@
+# typed: strict
+# frozen_string_literal: true
+
+require "action_controller"
+
+module LogStruct
+  # Copies of logged Hashes and Arrays that stop before an input too large to copy in full
+  module BoundedInput
+    extend T::Sig
+
+    # A Rails filter block with three arguments gets every parameter, so a copy for it keeps what the formatter
+    # leaves out of the line. These limits only stop values no request sends: nesting deeper than the formatter
+    # writes, and more values than a form or JSON body has. A copy that reaches one is truncated, and the
+    # filters that read it fail closed.
+    MAX_DEPTH = 20
+    MAX_NODES = 10_000
+
+    # The keys that count what a Hash's copy left out. The formatter writes them as it writes its own counts:
+    # Rails' default filter_parameters include :_key, which would filter them.
+    COUNT_KEYS = T.let(Set[:_more_keys, :_oversized_keys].freeze, T::Set[Symbol])
+
+    class State < T::Struct
+      prop :remaining_nodes, Integer, default: MAX_NODES
+      prop :truncated, T::Boolean, default: false
+    end
+
+    sig do
+      params(value: T.untyped, state: State, depth: Integer, ancestors: T::Hash[T.untyped, T::Boolean]).returns(T.untyped)
+    end
+    def self.copy(value, state:, depth: 0, ancestors: {}.compare_by_identity)
+      if depth > MAX_DEPTH || ancestors.key?(value) || state.remaining_nodes <= 0
+        state.truncated = true
+        return nil
+      end
+      state.remaining_nodes -= 1
+
+      case value
+      when Hash, ::ActionController::Parameters
+        ancestors[value] = true
+        begin
+          result = {}
+          value.each_pair do |key, child|
+            break if state.truncated
+
+            result[key] = copy(child, state: state, depth: depth + 1, ancestors: ancestors)
+          end
+          result
+        ensure
+          ancestors.delete(value)
+        end
+      when Array
+        ancestors[value] = true
+        begin
+          result = []
+          value.each do |child|
+            break if state.truncated
+
+            result << copy(child, state: state, depth: depth + 1, ancestors: ancestors)
+          end
+          result
+        ensure
+          ancestors.delete(value)
+        end
+      else
+        value
+      end
+    end
+
+    # How many keys a Hash or ActionController::Parameters has, without converting its values
+    sig { params(hash: T.untyped).returns(Integer) }
+    def self.key_count(hash)
+      hash.is_a?(Hash) ? hash.size : hash.keys.size
+    end
+
+    # The first keys of a Hash or ActionController::Parameters as Symbols, for a log struct's Hash field, without
+    # converting the keys the formatter would leave out. The formatter writes max_hash_keys entries of a Hash,
+    # and the counts of the keys left out here are entries too: the last key's slot goes to _more_keys, and an
+    # omitted oversized key's slot to _oversized_keys, so the formatter writes both counts as they are.
+    sig { params(hash: T.untyped).returns(T::Hash[Symbol, T.untyped]) }
+    def self.symbolize_keys(hash)
+      limits = LogStruct.config.limits
+      total = key_count(hash)
+      examined = (total > limits.max_hash_keys) ? [limits.max_hash_keys - 1, 0].max : total
+      result = T.let({}, T::Hash[Symbol, T.untyped])
+      oversized_keys = 0
+      hash.each_pair.with_index do |(key, value), index|
+        break if index >= examined
+
+        if key.to_s.bytesize > limits.max_string_bytes
+          oversized_keys += 1
+          next
+        end
+
+        result[key.to_s.to_sym] = value
+      end
+      result[:_more_keys] = total - examined if examined < total
+      result[:_oversized_keys] = oversized_keys if oversized_keys.positive?
+      result
+    end
+  end
+end

data/lib/log_struct/concerns/configuration.rb

@@ -259,7 +259,7 @@ module LogStruct
             result = filter.call(key, value, root)
             !!result || (value.is_a?(String) && value != original_value)
           rescue => e
-            handle_filter_error(e, filter, original_key || "")
+            handle_filter_error(e, filter)
             true
           end
           ConfigStruct::PathFilterMatcher.new(callable: callable, label: filter.inspect)
@@ -290,11 +290,11 @@ module LogStruct
             begin
               !!filter.call(key)
             rescue => e
-              handle_filter_error(e, filter, key)
+              handle_filter_error(e, filter)
               true
             end
           rescue => e
-            handle_filter_error(e, filter, key)
+            handle_filter_error(e, filter)
             true
           end
           ConfigStruct::FilterMatcher.new(callable: callable, label: filter.inspect)
@@ -305,11 +305,11 @@ module LogStruct
           filter.respond_to?(:arity) ? filter.arity : 2
         end
 
-        sig { params(error: StandardError, filter: T.untyped, key: String).void }
-        def handle_filter_error(error, filter, key)
+        # The key isn't reported: a key can be the secret (an email address or a token used as a key)
+        sig { params(error: StandardError, filter: T.untyped).void }
+        def handle_filter_error(error, filter)
           context = {
             filter: filter.class.name,
-            key: key,
             filter_label: begin
               filter.inspect
             rescue
@@ -317,7 +317,7 @@ module LogStruct
             end
           }
 
-          LogStruct.handle_exception(error, source: Source::Internal, context: context)
+          LogStruct.handle_value_error(error, "filter_parameters entry failed", context: context)
         end
       end
     end

data/lib/log_struct/concerns/error_handling.rb

@@ -89,6 +89,16 @@ module LogStruct
           Thread.current[:logstruct_formatting_internal_error] = previous_internal_error
         end
 
+        # Handle an error raised while LogStruct read a value it was logging, without the error's message: the
+        # message can quote that value, as a NoMethodError quotes its receiver's inspect before Ruby 3.4.
+        # failure says what failed, and the reported error keeps the original's class and backtrace.
+        sig { params(error: Exception, failure: String, context: T.nilable(T::Hash[Symbol, T.untyped])).void }
+        def handle_value_error(error, failure, context: nil)
+          reported = RuntimeError.new("#{error.class}: #{failure}")
+          reported.set_backtrace(error.backtrace)
+          handle_exception(reported, source: Source::Internal, context: context)
+        end
+
         # Handle an error according to the configured error handling mode (log, report, raise, etc).
         # Pass `log: false` when the caller has already logged the error in its own structured line.
         sig { params(error: StandardError, source: Source, context: T.nilable(T::Hash[Symbol, T.untyped]), log: T::Boolean).void }

data/lib/log_struct/config_struct/filters.rb

@@ -2,6 +2,7 @@
 # frozen_string_literal: true
 
 require "active_support/hash_with_indifferent_access"
+require_relative "../bounded_input"
 
 module LogStruct
   # A key's words, lowercase and joined by underscores, so that keys match however they're written:
@@ -38,8 +39,8 @@ module LogStruct
     # A filter that needs more than the key: Rails' entries with a dot (`card.number`) match a key by its
     # path, and Rails gives filter blocks with three arguments the parameters around the key. The path
     # starts at the line's top-level field (`["params", "card", "number"]`); `root` is that field's value,
-    # or the whole line for a top-level key, with indifferent access as Rails gives it. Building it copies
-    # every value inside it, so a matcher that doesn't read it sets uses_root to false and gets nil.
+    # or the whole line for a top-level key, with indifferent access as Rails gives it. A matcher
+    # that doesn't read the root sets uses_root to false and gets nil.
     class PathFilterMatcher < T::Struct
       extend T::Sig
 
@@ -50,7 +51,10 @@ module LogStruct
       sig { params(path: T::Array[String], value: T.untyped, root: T.untyped).returns(T::Boolean) }
       def matches?(path, value, root)
         root = nil unless uses_root
-        callable.call(path, value, root.is_a?(LazyRoot) ? root.value : root)
+        root = root.value if root.is_a?(LazyRoot)
+        return true if root.equal?(LazyRoot::UNAVAILABLE)
+
+        callable.call(path, value, root)
       end
     end
 
@@ -58,6 +62,8 @@ module LogStruct
     class LazyRoot
       extend T::Sig
 
+      UNAVAILABLE = T.let(Object.new.freeze, Object)
+
       sig { params(value: T.untyped).void }
       def initialize(value)
         @source = value
@@ -70,12 +76,20 @@ module LogStruct
       def value
         return @value if @built
 
-        @built = true
-        @value = begin
-          @source.is_a?(Hash) ? ActiveSupport::HashWithIndifferentAccess.new(@source) : @source
-        rescue SystemStackError, StandardError
-          @source
+        state = BoundedInput::State.new
+        bounded = BoundedInput.copy(@source, state: state)
+        @value = if state.truncated
+          UNAVAILABLE
+        elsif bounded.is_a?(Hash)
+          ActiveSupport::HashWithIndifferentAccess.new(bounded)
+        else
+          bounded
         end
+        @built = true
+        @value
+      rescue SystemStackError, StandardError
+        @built = true
+        @value = UNAVAILABLE
       end
     end
 
@@ -86,7 +100,7 @@ module LogStruct
       # A filtered value is replaced by a summary of it, with no length (which would say how long a
       # secret is):
       #
-      # { _filtered: { _class: "Hash", _keys_count: 3, _keys: [:key1, :key2, :key3] } }  # first 10 keys
+      # { _filtered: { _class: "Hash", _keys_count: 3 } }
       # { _filtered: { _class: "Array", _count: 2 } }
       # { _filtered: { _class: "String" } }  # plus its _hash for filter_keys_with_hashes
       #

data/lib/log_struct/config_struct/limits.rb

@@ -21,7 +21,7 @@ module LogStruct
       # Default: 20
       prop :max_backtrace_lines, Integer, default: 20
 
-      # Strings longer than this many bytes are cut, and say how long they were
+      # Strings longer than this many bytes are cut; oversized Hash keys and their values are omitted
       # Default: 32768 (32 KB)
       prop :max_string_bytes, Integer, default: 32_768
     end

data/lib/log_struct/enums/log_field.rb

@@ -120,7 +120,6 @@ module LogStruct
       Storage = new(:storage)
       Operation = new(:op)
       FileId = new(:file_id)
-      Filename = new(:filename)
       MimeType = new(:mime_type)
       Size = new(:size)
       Metadata = new(:metadata)
@@ -134,8 +133,6 @@ module LogStruct
       Model = new(:model)
       MountPoint = new(:mount_point)
       Version = new(:version)
-      StorePath = new(:store_path)
-      Extension = new(:ext)
 
       # SQL-specific fields
       Sql = new(:sql)

data/lib/log_struct/formatter.rb

@@ -10,6 +10,7 @@ require_relative "enums/source"
 require_relative "enums/event"
 require_relative "enums/log_field"
 require_relative "string_scrubber"
+require_relative "bounded_input"
 require_relative "log"
 require_relative "param_filters"
 require_relative "multi_error_reporter"
@@ -80,13 +81,13 @@ module LogStruct
       return summarize_unprocessed_value(arg) if recursion_depth > MAX_RECURSION_DEPTH || ancestors.key?(arg)
 
       case arg
-      when Hash, Array
+      when Hash, Array, ::ActionController::Parameters
         ancestors[arg] = true
         begin
-          if arg.is_a?(Hash)
-            process_hash(arg, recursion_depth: recursion_depth, ancestors: ancestors, path: path, root: root)
-          else
+          if arg.is_a?(Array)
             process_array(arg, recursion_depth: recursion_depth, ancestors: ancestors, path: path, root: root)
+          else
+            process_hash(arg, recursion_depth: recursion_depth, ancestors: ancestors, path: path, root: root)
           end
         ensure
           ancestors.delete(arg)
@@ -107,7 +108,7 @@ module LogStruct
               scrub_string(String(object))
             end
           rescue => e
-            LogStruct.handle_exception(e, source: Source::Internal)
+            LogStruct.handle_value_error(e, "GlobalID formatting failed")
             "[GLOBALID_ERROR]"
           end
         end
@@ -125,26 +126,24 @@ module LogStruct
       when Numeric, TrueClass, FalseClass, NilClass
         arg
       else
-        # Structs, ActionController::Parameters and other objects: filter and scrub what
+        # Structs and other objects: filter and scrub what
         # to_json would write
         process_values(arg.as_json, recursion_depth: recursion_depth + 1, ancestors: ancestors, path: path, root: root)
       end
     rescue SystemStackError, StandardError => e
       # A SystemStackError comes from an object whose as_json refers back to itself
-      e = RuntimeError.new("#{e.class}: #{e.message}") unless e.is_a?(StandardError)
-      # Report error through LogStruct's framework
       context = {
         processor_method: "process_values",
         value_type: arg.class.name,
         recursion_depth: recursion_depth
       }
-      LogStruct.handle_exception(e, source: Source::Internal, context: context)
+      LogStruct.handle_value_error(e, "value formatting failed", context: context)
       summarize_unprocessed_value(arg)
     end
 
     sig do
       params(
-        hash: T::Hash[T.untyped, T.untyped],
+        hash: T.untyped,
         recursion_depth: Integer,
         ancestors: T::Hash[T.untyped, T::Boolean],
         path: T.nilable(T::Array[String]),
@@ -158,13 +157,30 @@ module LogStruct
       line_root = (path && recursion_depth == 0) ? ConfigStruct::LazyRoot.new(hash) : nil
 
       max_keys = LogStruct.config.limits.max_hash_keys
-      hash.each_with_index do |(key, value), index|
+      max_key_bytes = LogStruct.config.limits.max_string_bytes
+      oversized_keys = 0
+      hash.each_pair.with_index do |(key, value), index|
         if index >= max_keys
-          result[:_more_keys] = hash.size - max_keys
+          result[:_more_keys] = BoundedInput.key_count(hash) - max_keys
           break
         end
 
+        if BoundedInput::COUNT_KEYS.include?(key) && value.is_a?(Integer)
+          result[key] = value
+          next
+        end
+
+        if key.to_s.bytesize > max_key_bytes
+          oversized_keys += 1
+          next
+        end
+
+        # Checked again: each byte that isn't valid UTF-8 becomes a 3-byte U+FFFD
         key = valid_utf8_key(key)
+        if key.to_s.bytesize > max_key_bytes
+          oversized_keys += 1
+          next
+        end
         logstruct_field = recursion_depth == 0 && LOG_FIELD_KEYS.include?(key)
         key_path = path && [*path, key.to_s]
         # A top-level key's root is the line; the keys below it get the top-level field's value
@@ -173,7 +189,9 @@ module LogStruct
         # Check if this key should be filtered at any depth
         filtered = ParamFilters.should_filter_key?(key, value, logstruct_field: logstruct_field, path: key_path, root: key_root)
         # Keys are scrubbed too: an email address can be a key
-        result[StringScrubber.scrub_name(key)] = if !filtered
+        scrubbed_key = StringScrubber.scrub_name(key)
+        scrubbed_key = scrub_limited_string(scrubbed_key) if scrubbed_key.is_a?(String) && scrubbed_key.bytesize > max_key_bytes
+        result[scrubbed_key] = if !filtered
           process_values(value, recursion_depth: recursion_depth + 1, ancestors: ancestors, path: key_path, root: children_root)
         elsif value == RAILS_FILTERED_MASK
           # Rails' parameter filter has already replaced the value
@@ -183,6 +201,8 @@ module LogStruct
         end
       end
 
+      result[:_oversized_keys] = oversized_keys if oversized_keys.positive?
+
       result
     end
 
@@ -205,7 +225,7 @@ module LogStruct
         fixed = valid_utf8(name)
         fixed.equal?(name) ? key : fixed.to_sym
       else
-        key
+        valid_utf8(key.to_s)
       end
     end
 
@@ -295,15 +315,10 @@ module LogStruct
               log_value.to_s
             end
           rescue => e
-            # Handle serialization errors
-            context = {
-              object_class: log_value.class.name,
-              object_inspect: log_value.inspect.truncate(100)
-            }
-            LogStruct.handle_exception(e, source: Source::Internal, context: context)
-
-            # Fall back to the string representation to ensure we continue processing
-            log_value.to_s
+            LogStruct.handle_value_error(e, "value serialization failed", context: {object_class: log_value.class.name})
+
+            # An object's string representation may contain the same secret that broke serialization.
+            "[SERIALIZATION_ERROR]"
           end
         end
 

data/lib/log_struct/hash_utils.rb

@@ -2,13 +2,14 @@
 # frozen_string_literal: true
 
 require "openssl"
+require "securerandom"
 
 module LogStruct
   # Utility module for hashing sensitive data
   module HashUtils
-    # The key when there's neither a configured hash_salt nor a Rails app with a secret_key_base.
-    # It's public, so these hashes can be matched to a list of candidate values.
-    FALLBACK_KEY = "l0g5t0p"
+    # The key without a hash_salt, before the Rails app has its secret_key_base. A fixed key would be public,
+    # and anyone could match its hashes to a list of candidate emails; these hashes differ between processes.
+    FALLBACK_KEY = T.let(::SecureRandom.random_bytes(32).freeze, String)
 
     class << self
       extend T::Sig
@@ -17,7 +18,13 @@ module LogStruct
       sig { params(value: String).returns(String) }
       def hash_value(value)
         length = LogStruct.config.filters.hash_length
-        OpenSSL::HMAC.hexdigest("SHA256", hash_key, value)[0...length] || "error"
+        max_bytes = LogStruct.config.limits.max_string_bytes
+        input = if value.bytesize > max_bytes
+          value.byteslice(0, max_bytes).to_s.b + "\0".b + value.bytesize.to_s.b
+        else
+          value
+        end
+        OpenSSL::HMAC.hexdigest("SHA256", hash_key, input)[0...length] || "error"
       end
 
       # The configured hash_salt, or a key derived from the app's secret_key_base

data/lib/log_struct/integrations/ahoy.rb

@@ -1,6 +1,8 @@
 # typed: strict
 # frozen_string_literal: true
 
+require_relative "../bounded_input"
+
 module LogStruct
   module Integrations
     # Ahoy analytics integration. If Ahoy is present, prepend a small hook to
@@ -45,10 +47,9 @@ module LogStruct
       # Event properties can come from the browser, so they can be anything; a non-Hash is logged as its class
       sig { params(properties: T.untyped).returns(T.nilable(T::Hash[Symbol, T.untyped])) }
       def self.properties_for_log(properties)
-        properties = properties.to_unsafe_h if properties.respond_to?(:to_unsafe_h)
         case properties
         when nil then nil
-        when Hash then properties.to_h { |key, value| [key.to_s.to_sym, value] }
+        when Hash, ::ActionController::Parameters then BoundedInput.symbolize_keys(properties)
         else {_class: properties.class.name}
         end
       end

data/lib/log_struct/integrations/carrierwave.rb

@@ -6,6 +6,7 @@ begin
 rescue LoadError
   # CarrierWave gem is not available, integration will be skipped
 end
+require_relative "../hash_utils"
 
 module LogStruct
   module Integrations
@@ -49,7 +50,6 @@ module LogStruct
             # file_fields is a Hash Sorbet can't check against the struct's keyword arguments
             Log::CarrierWave::Upload.new(
               **T.unsafe(file_fields),
-              filename: file&.filename,
               mime_type: file&.content_type,
               size: file&.size,
               duration_ms: duration_ms
@@ -72,7 +72,6 @@ module LogStruct
           log_file_operation do
             Log::CarrierWave::Download.new(
               **T.unsafe(file_fields),
-              filename: file&.filename,
               mime_type: file&.content_type,
               size: file&.size
             )
@@ -87,13 +86,11 @@ module LogStruct
         def file_fields
           {
             storage: storage.class.name.to_s.split("::").last.to_s.downcase.to_sym,
-            file_id: identifier.to_s,
+            file_id: HashUtils.hash_value(identifier.to_s),
             uploader: self.class.name,
             model: model&.class&.name,
             mount_point: mounted_as&.to_s,
-            version: version_name&.to_s,
-            store_path: store_path,
-            extension: file&.extension
+            version: version_name&.to_s
           }
         end
 

data/lib/log_struct/integrations/lograge.rb

@@ -6,6 +6,7 @@ begin
 rescue LoadError
   # Lograge gem is not available, integration will be skipped
 end
+require_relative "../bounded_input"
 
 module LogStruct
   module Integrations
@@ -196,12 +197,12 @@ module LogStruct
           result = custom_options_proc.call(event, options)
           return if !result.is_a?(Hash) || result.equal?(options)
 
-          options.merge!(result.to_h { |key, value| [key.to_s.to_sym, value] })
+          options.merge!(BoundedInput.symbolize_keys(result))
         end
 
         sig { params(data: T::Hash[T.any(Symbol, String), T.untyped]).returns(Log::Request) }
         def build_request_log(data)
-          normalized_data = normalize_lograge_data(data)
+          normalized_data = BoundedInput.symbolize_keys(data)
 
           # Coerce common fields to expected types
           status = ((s = normalized_data[:status]) && s.respond_to?(:to_i)) ? s.to_i : s
@@ -210,7 +211,7 @@ module LogStruct
           db = ((b = normalized_data[:db]) && b.respond_to?(:to_f)) ? b.to_f : b
 
           params = normalized_data[:params]
-          params = params.deep_symbolize_keys if params&.respond_to?(:deep_symbolize_keys)
+          params = BoundedInput.symbolize_keys(params) if params.is_a?(Hash) || params.is_a?(::ActionController::Parameters)
 
           additional_data = extract_additional_data(normalized_data)
 
@@ -237,13 +238,6 @@ module LogStruct
           )
         end
 
-        sig { params(data: T::Hash[T.any(Symbol, String), T.untyped]).returns(T::Hash[Symbol, T.untyped]) }
-        def normalize_lograge_data(data)
-          data.each_with_object({}) do |(key, value), normalized|
-            normalized[key.to_s.to_sym] = value
-          end
-        end
-
         sig { params(data: T::Hash[Symbol, T.untyped]).returns(T.nilable(T::Hash[Symbol, T.untyped])) }
         def extract_additional_data(data)
           extras = T.let({}, T::Hash[Symbol, T.untyped])

data/lib/log_struct/log/action_mailer/delivered.rb

@@ -42,6 +42,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::To, LogField::From, LogField::Subject, LogField::MessageId, LogField::MailerClass, LogField::MailerAction, LogField::AttachmentCount].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/action_mailer/delivery.rb

@@ -42,6 +42,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::To, LogField::From, LogField::Subject, LogField::MessageId, LogField::MailerClass, LogField::MailerAction, LogField::AttachmentCount].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/action_mailer/error.rb

@@ -47,6 +47,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::To, LogField::From, LogField::Subject, LogField::MessageId, LogField::MailerClass, LogField::MailerAction, LogField::AttachmentCount, LogField::ErrorClass, LogField::Message, LogField::Backtrace].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/carrierwave/delete.rb

@@ -35,8 +35,6 @@ module LogStruct
         const :model, T.nilable(String), default: nil
         const :mount_point, T.nilable(String), default: nil
         const :version, T.nilable(String), default: nil
-        const :store_path, T.nilable(String), default: nil
-        const :extension, T.nilable(String), default: nil
 
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
@@ -51,8 +49,6 @@ module LogStruct
           h[LogField::Model] = model unless model.nil?
           h[LogField::MountPoint] = mount_point unless mount_point.nil?
           h[LogField::Version] = version unless version.nil?
-          h[LogField::StorePath] = store_path unless store_path.nil?
-          h[LogField::Extension] = extension unless extension.nil?
           h
         end
       end

data/lib/log_struct/log/carrierwave/download.rb

@@ -35,11 +35,8 @@ module LogStruct
         const :model, T.nilable(String), default: nil
         const :mount_point, T.nilable(String), default: nil
         const :version, T.nilable(String), default: nil
-        const :store_path, T.nilable(String), default: nil
-        const :extension, T.nilable(String), default: nil
 
         # Event-specific fields
-        const :filename, T.nilable(String), default: nil
         const :mime_type, T.nilable(String), default: nil
         const :size, T.nilable(Integer), default: nil
 
@@ -56,9 +53,6 @@ module LogStruct
           h[LogField::Model] = model unless model.nil?
           h[LogField::MountPoint] = mount_point unless mount_point.nil?
           h[LogField::Version] = version unless version.nil?
-          h[LogField::StorePath] = store_path unless store_path.nil?
-          h[LogField::Extension] = extension unless extension.nil?
-          h[LogField::Filename] = filename unless filename.nil?
           h[LogField::MimeType] = mime_type unless mime_type.nil?
           h[LogField::Size] = size unless size.nil?
           h

data/lib/log_struct/log/carrierwave/upload.rb

@@ -35,11 +35,8 @@ module LogStruct
         const :model, T.nilable(String), default: nil
         const :mount_point, T.nilable(String), default: nil
         const :version, T.nilable(String), default: nil
-        const :store_path, T.nilable(String), default: nil
-        const :extension, T.nilable(String), default: nil
 
         # Event-specific fields
-        const :filename, T.nilable(String), default: nil
         const :mime_type, T.nilable(String), default: nil
         const :size, T.nilable(Integer), default: nil
         const :metadata, T.nilable(T::Hash[String, T.untyped]), default: nil
@@ -58,9 +55,6 @@ module LogStruct
           h[LogField::Model] = model unless model.nil?
           h[LogField::MountPoint] = mount_point unless mount_point.nil?
           h[LogField::Version] = version unless version.nil?
-          h[LogField::StorePath] = store_path unless store_path.nil?
-          h[LogField::Extension] = extension unless extension.nil?
-          h[LogField::Filename] = filename unless filename.nil?
           h[LogField::MimeType] = mime_type unless mime_type.nil?
           h[LogField::Size] = size unless size.nil?
           h[LogField::Metadata] = metadata unless metadata.nil?

data/lib/log_struct/log/carrierwave.rb

@@ -21,8 +21,6 @@ module LogStruct
         const :model, T.nilable(String), default: nil
         const :mount_point, T.nilable(String), default: nil
         const :version, T.nilable(String), default: nil
-        const :store_path, T.nilable(String), default: nil
-        const :extension, T.nilable(String), default: nil
 
         Kwargs = T.type_alias do
           {
@@ -31,9 +29,7 @@ module LogStruct
             uploader: T.nilable(String),
             model: T.nilable(String),
             mount_point: T.nilable(String),
-            version: T.nilable(String),
-            store_path: T.nilable(String),
-            extension: T.nilable(String)
+            version: T.nilable(String)
           }
         end
 
@@ -45,9 +41,7 @@ module LogStruct
             uploader: uploader,
             model: model,
             mount_point: mount_point,
-            version: version,
-            store_path: store_path,
-            extension: extension
+            version: version
           }
         end
       end

data/lib/log_struct/log/error.rb

@@ -48,6 +48,16 @@ module LogStruct
       const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
       include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+      FIELD_KEYS = T.let(
+        [LogField::Path, LogField::HttpMethod, LogField::SourceIp, LogField::UserAgent, LogField::Referer, LogField::ErrorClass, LogField::Message, LogField::ErrorMessage, LogField::Backtrace, LogField::Status, LogField::CauseClass, LogField::CauseMessage].to_set(&:serialize).freeze,
+        T::Set[Symbol]
+      )
+
+      sig { override.returns(T::Set[Symbol]) }
+      def field_keys
+        FIELD_KEYS
+      end
+
       # Request fields (optional)
       include LogStruct::Log::Interfaces::RequestFields
 

data/lib/log_struct/log/plain.rb

@@ -36,6 +36,16 @@ module LogStruct
       const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
       include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+      FIELD_KEYS = T.let(
+        [LogField::Message].to_set(&:serialize).freeze,
+        T::Set[Symbol]
+      )
+
+      sig { override.returns(T::Set[Symbol]) }
+      def field_keys
+        FIELD_KEYS
+      end
+
       # Serialize shared fields
       include LogStruct::Log::Interfaces::CommonFields
       include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/puma/error.rb

@@ -42,6 +42,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::ErrorClass, LogField::Message, LogField::Backtrace, LogField::ErrorContext, LogField::HttpMethod, LogField::Path].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/puma/shutdown.rb

@@ -37,6 +37,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::ProcessId].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/puma/start.rb

@@ -45,6 +45,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::Mode, LogField::PumaVersion, LogField::PumaCodename, LogField::RubyVersion, LogField::MinThreads, LogField::MaxThreads, LogField::Environment, LogField::ProcessId, LogField::ListeningAddresses].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/request.rb

@@ -52,6 +52,16 @@ module LogStruct
       const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
       include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+      FIELD_KEYS = T.let(
+        [LogField::Path, LogField::HttpMethod, LogField::SourceIp, LogField::UserAgent, LogField::Referer, LogField::Format, LogField::Controller, LogField::Action, LogField::Status, LogField::DurationMs, LogField::View, LogField::Database, LogField::Params, LogField::Host, LogField::ContentType, LogField::Accept].to_set(&:serialize).freeze,
+        T::Set[Symbol]
+      )
+
+      sig { override.returns(T::Set[Symbol]) }
+      def field_keys
+        FIELD_KEYS
+      end
+
       # Request fields (optional)
       include LogStruct::Log::Interfaces::RequestFields
 

data/lib/log_struct/log/security/blocked_host.rb

@@ -46,6 +46,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::Path, LogField::HttpMethod, LogField::SourceIp, LogField::UserAgent, LogField::Referer, LogField::Message, LogField::BlockedHost, LogField::BlockedHosts, LogField::XForwardedFor].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Request fields (optional)
         include LogStruct::Log::Interfaces::RequestFields
 

data/lib/log_struct/log/security/csrf_violation.rb

@@ -43,6 +43,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::Path, LogField::HttpMethod, LogField::SourceIp, LogField::UserAgent, LogField::Referer, LogField::Message].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Request fields (optional)
         include LogStruct::Log::Interfaces::RequestFields
 

data/lib/log_struct/log/security/ip_spoof.rb

@@ -45,6 +45,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::Path, LogField::HttpMethod, LogField::SourceIp, LogField::UserAgent, LogField::Referer, LogField::Message, LogField::ClientIp, LogField::XForwardedFor].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Request fields (optional)
         include LogStruct::Log::Interfaces::RequestFields
 

data/lib/log_struct/log/sidekiq/error.rb

@@ -42,6 +42,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::ErrorClass, LogField::Message, LogField::Backtrace, LogField::ErrorContext, LogField::QueueName, LogField::RetryCount].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/sidekiq/log.rb

@@ -40,6 +40,16 @@ module LogStruct
         const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
         include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+        FIELD_KEYS = T.let(
+          [LogField::Message, LogField::Context, LogField::ProcessId, LogField::ThreadId].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        sig { override.returns(T::Set[Symbol]) }
+        def field_keys
+          FIELD_KEYS
+        end
+
         # Serialize shared fields
         include LogStruct::Log::Interfaces::CommonFields
         include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/log/sql.rb

@@ -47,6 +47,16 @@ module LogStruct
       const :additional_data, T.nilable(T::Hash[T.any(String, Symbol), T.untyped]), default: nil
       include LogStruct::Log::Shared::MergeAdditionalDataFields
 
+      FIELD_KEYS = T.let(
+        [LogField::Message, LogField::Sql, LogField::Name, LogField::DurationMs, LogField::RowCount, LogField::Adapter, LogField::BindParams, LogField::DatabaseName, LogField::ConnectionPoolSize, LogField::ActiveConnections, LogField::OperationType, LogField::TableNames].to_set(&:serialize).freeze,
+        T::Set[Symbol]
+      )
+
+      sig { override.returns(T::Set[Symbol]) }
+      def field_keys
+        FIELD_KEYS
+      end
+
       # Serialize shared fields
       include LogStruct::Log::Interfaces::CommonFields
       include LogStruct::Log::Shared::SerializeCommon

data/lib/log_struct/param_filters.rb

@@ -43,7 +43,7 @@ module LogStruct
 
           matcher.matches?(normalized_key, value)
         rescue => e
-          handle_filter_matcher_error(e, matcher.label, normalized_key)
+          handle_filter_matcher_error(e, matcher.label)
           # An unsuccessful filter must never expose the value it was meant to protect.
           true
         end
@@ -55,7 +55,7 @@ module LogStruct
         filters.path_matchers.any? do |matcher|
           matcher.matches?(path, value, root)
         rescue => e
-          handle_filter_matcher_error(e, matcher.label, normalized_key)
+          handle_filter_matcher_error(e, matcher.label)
           true
         end
       end
@@ -94,16 +94,14 @@ module LogStruct
         filtered_string
       end
 
-      # Summarize a filtered Hash: its first 10 keys and how many there are. No size, which would say
-      # how long the secrets in it are.
+      # A filtered Hash's child keys can themselves contain secrets.
       sig { params(hash: T::Hash[T.untyped, T.untyped]).returns(T::Hash[Symbol, T.untyped]) }
       def summarize_hash(hash)
         return {_class: "Hash", _empty: true} if hash.empty?
 
         {
           _class: Hash,
-          _keys_count: hash.size,
-          _keys: hash.each_key.take(10).map { |key| normalize_summary_key(StringScrubber.scrub_name(key)) }
+          _keys_count: hash.size
         }
       end
 
@@ -120,26 +118,9 @@ module LogStruct
 
       private
 
-      sig { params(key: T.any(String, Symbol, Integer, T.untyped)).returns(T.any(Symbol, String)) }
-      def normalize_summary_key(key)
-        if key.is_a?(Symbol)
-          key
-        elsif key.respond_to?(:to_sym)
-          key.to_sym
-        else
-          key.to_s
-        end
-      rescue
-        key.to_s
-      end
-
-      sig { params(error: StandardError, label: String, key: String).void }
-      def handle_filter_matcher_error(error, label, key)
-        context = {
-          matcher: label,
-          key: key
-        }
-        LogStruct.handle_exception(error, source: Source::Internal, context: context)
+      sig { params(error: StandardError, label: String).void }
+      def handle_filter_matcher_error(error, label)
+        LogStruct.handle_value_error(error, "filter matcher failed", context: {matcher: label})
       end
     end
   end

data/lib/log_struct/shared/merge_additional_data_fields.rb

@@ -3,6 +3,7 @@
 
 require_relative "interfaces/additional_data_field"
 require_relative "../enums/source"
+require_relative "../enums/log_field"
 
 module LogStruct
   module Log
@@ -14,17 +15,32 @@ module LogStruct
         requires_ancestor { T::Struct }
         requires_ancestor { Interfaces::AdditionalDataField }
 
-        # Reserved keys that cannot be overwritten by additional_data.
-        # These are the core log structure fields that must not be modified.
-        RESERVED_KEYS = T.let(%i[src evt lvl ts].freeze, T::Array[Symbol])
+        # Every line's own fields, and the request ID the formatter adds unless the line has one
+        RESERVED_KEYS = T.let(
+          [LogField::Source, LogField::Event, LogField::Level, LogField::Timestamp, LogField::RequestId].to_set(&:serialize).freeze,
+          T::Set[Symbol]
+        )
+
+        NO_FIELD_KEYS = T.let(Set.new.freeze, T::Set[Symbol])
+
+        # The keys of every field of the struct's type, set or not on this line, which additional_data can't
+        # set: it would replace the field, or stand in for one the type didn't write. LogStruct's log types list
+        # theirs. A custom type's serialize writes its own fields after merging additional_data, so they win.
+        sig { overridable.returns(T::Set[Symbol]) }
+        def field_keys
+          NO_FIELD_KEYS
+        end
 
         sig { params(hash: T::Hash[Symbol, T.untyped]).void }
         def merge_additional_data_fields(hash)
           ad = additional_data
           return unless ad
-          ad.each do |key, value|
+          ad.each_pair.with_index do |(key, value), index|
+            break if index >= LogStruct.config.limits.max_hash_keys
+            next if key.to_s.bytesize > LogStruct.config.limits.max_string_bytes
+
             sym_key = key.to_sym
-            if RESERVED_KEYS.include?(sym_key)
+            if RESERVED_KEYS.include?(sym_key) || field_keys.include?(sym_key)
               LogStruct.handle_exception(
                 ArgumentError.new("additional_data attempted to overwrite reserved key: #{sym_key}"),
                 source: Source::Internal

data/lib/log_struct/version.rb

@@ -2,5 +2,5 @@
 # frozen_string_literal: true
 
 module LogStruct
-  VERSION = "0.2.2"
+  VERSION = "0.3.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: logstruct
 version: !ruby/object:Gem::Version
-  version: 0.2.2
+  version: 0.3.0
 platform: ruby
 authors:
 - DocSpring
@@ -283,6 +283,7 @@ files:
 - README.md
 - lib/log_struct.rb
 - lib/log_struct/boot_buffer.rb
+- lib/log_struct/bounded_input.rb
 - lib/log_struct/concerns/configuration.rb
 - lib/log_struct/concerns/error_handling.rb
 - lib/log_struct/concerns/logging.rb