standard_audit 0.9.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a1d506a4bb5ad3811dd4a70b6be817374ec9a88ced8d1ffd9415bfddd28cc50b
-  data.tar.gz: 52c8aaa70f86810c6b308c86fa81e10c368ab5ddb011d16f422745e7681c88da
+  metadata.gz: b89dc74bd119d2e0f1ff5080cef706439e68b61c79d2a6b653fcd5a74e265ae1
+  data.tar.gz: 86686ea3bb611a13e8c8947ef3e0b63aad63eb47bfa1e48b16abba377a426f7f
 SHA512:
-  metadata.gz: 2db9ee0ef4983ce2bd9a94d195bbbf973016f32a3e9e2a53b45476df6c9cc6d30f7edd2bfb7e9442d6839e05cc491263b91bb644462946510740c187eb861e29
-  data.tar.gz: cce577cbc3f1b5829429ab30765c451f4a0fdca18b9f6283f075e1b3487b3fb763ceaeca846e39768c53707361bfc87024da65e0f8e662885a47766f814dd5c1
+  metadata.gz: b06e7ee9a3b5d79191cb47a532cf81efb95e3bb722015540be79ce4bb9412ab13b8e69985a66d368c8dcdfb099f34f1ca1d3f0d894bfc90cc409615782dde921
+  data.tar.gz: bb1e9316eba648a8d238becb75d33f705b82f509310e8c3e0d562656accac2b44339e49551e026ae0357bf57dda8fe8251f6f1d6c7df655777d5b157c3d17b0a

data/CHANGELOG.md

@@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.11.0] - 2026-07-31
+
+### Security
+
+- **ActiveRecord objects in audit metadata are no longer written with all of their attributes.** Any record found in metadata — at any depth, and inside Arrays, Hashes and `ActiveRecord::Relation`s — is now replaced with a reference: `{ "gid" => "gid://app/Account/1", "type" => "Account", "id" => "1" }`. Applies to both write paths (`ActiveSupport::Notifications` and `StandardAudit.record`).
+
+  `standard_id` publishes live records under payload keys like `account:`, `current_account:`, `session:` and `code_challenge:`, and `Subscriber#extract_metadata` excluded only `actor`/`target`/`scope`/`request_id`/`ip_address`/`user_agent`/`session_id` — every other key was written whole. Confirmed present in real `audit_logs` rows: `account.password_digest`, `account.password_reset_token_digest`, `session.token_digest`, `session.lookup_hash`, `code_challenge.code`.
+
+  All three existing defences missed, each for a different reason: `sensitive_keys` matches key names exactly and the secrets are attributes *underneath* `account:`; `filter_nested_metadata` is off by default so the filter never descended to them; and `account:` does not look sensitive at the top level. This is not a redaction bug — key-based redaction is the wrong tool for a value that is an entire database row — so the value is replaced by type, before any key filtering runs.
+
+  **This fix stops the bleeding; it does not clean up.** `audit_logs` is append-only by design (`before_update` and `before_destroy` raise `ReadOnlyRecord`), so every row already written keeps its digests, for the whole of a retention window that is intentionally long. Upgrading changes only what is written from now on. Assessing and remediating existing rows is a separate, host-side exercise.
+
+  `sensitive_keys` semantics are untouched: still exact-match, still no substring mode. See rarebit-one/rarebit-ops#296.
+
+### Added
+
+- **`config.dereference_record_metadata`** (default `true`) — the escape hatch for the above. It defaults to the SAFE behaviour, unlike `filter_nested_metadata`, because the values it catches are ones no host asked to record: they arrive as a side effect of a payload carrying `account:` or `session:`, and an append-only row cannot be walked back. Set it to `false` only if an app genuinely depends on record attributes in metadata and has satisfied itself no secret-bearing column can reach a row.
+
+### Changed
+
+- **Consumer-visible beyond the secrets disappearing:** any metadata key that used to hold a record's attribute hash now holds a three-key reference, so dashboards, reports or queries reading e.g. `metadata->'account'->>'email'` will read `NULL` on new rows (old rows are unchanged, which makes the change look like a data gap rather than a schema change). Recover the specific fields you need with `metadata_builder`, which runs BEFORE dereferencing and still receives the record: `->(metadata) { metadata.merge(account_email: metadata[:account]&.email) }`. `actor`/`target`/`scope` are unaffected — they were already stored as GlobalIDs.
+
+## [0.10.0] - 2026-07-31
+
+### Added
+
+- **`config.audit_error_context_key`** (default `:audit_action`) — renames the `Rails.error.report` context key naming the audit action, without needing a whole handler. Two apps had overridden `audit_write_error_handler` for nothing but this: both tag every OTHER audit-error report site with `audit_event:` (four sites in one, twelve across ten files in the other), so adopting the gem name left the operations layer as the only place forking the convention. A handler written to rename one key also silently opts out of every future improvement to the built-in reporter.
+
+### Changed
+
+- **The built-in reporter no longer fires when `raise_on_audit_write_error` re-raises.** The caller receives the error and owns it, so reporting as well produced two events for one failure in every host that both sets the flag and reports on what it catches — which is most hosts that set it at all, since the flag exists for hosts treating an unaudited write as a failure worth handling. Two of the five apps hit this independently and each wrote a log-only handler to work around it. A host that does NOT catch the error still gets a report, via its framework unhandled-error path. An explicitly configured `audit_write_error_handler` is unaffected and still runs under either policy — only the built-in reporter steps aside.
+
+### Fixed
+
+- **The write-site scan no longer reads comments as calls.** An operation declaring `audit_none!` that explained itself in prose naming `audit!` was flagged by `unexpected_write_sites` — a false failure, which hosts worked around by backticking the token in their own comments. Source is now lexed with `Ripper` and comment tokens dropped before scanning; a file that cannot be lexed falls back to raw source (the old behaviour). Lexing rather than stripping `#` to end-of-line, because the naive form eats `"#{interpolation}"` and `%w[#]` and would trade a false failure for a false pass.
+- **Corrected a docstring that was wrong for half its own surface.** It claimed the source-scanning predicates "err towards a false pass rather than a false failure". That held only for `missing_write_sites`; in the `unexpected_write_sites` direction the same match produced a false failure. Worse than the bug itself, since it told anyone hitting the failure not to suspect the scanner.
+
 ## [0.9.0] - 2026-07-31
 
 ### Added

data/README.md

@@ -332,6 +332,10 @@ StandardAudit.configure(baseline: true) do |config|
   # `metadata: { stripe: { client_secret: ... } }` is written intact.
   config.filter_nested_metadata = true
 
+  # Replace ActiveRecord objects in metadata with a reference instead of
+  # serialising every attribute. ON by default; see "Records in metadata".
+  config.dereference_record_metadata = true
+
   # -- Write-time hooks --
   # Run between the UUID assignment and the checksum computation, so a hook MAY
   # set a checksummed column and the row still passes `verify_chain`. No
@@ -695,6 +699,38 @@ Nested metadata is **not** redacted unless `filter_nested_metadata` is enabled
 — `metadata: { stripe: { client_secret: ... } }` passes both write paths under
 exact matching by default.
 
+### Records in metadata
+
+An ActiveRecord object appearing anywhere in metadata is replaced, by default,
+with a reference rather than a snapshot of the row:
+
+```ruby
+{ account: account }
+# => { account: { "gid" => "gid://myapp/Account/1", "type" => "Account", "id" => "1" } }
+```
+
+This applies at any depth, inside Arrays, Hashes and `ActiveRecord::Relation`s,
+on both write paths. It is on by default because key-based redaction cannot
+reach the problem: a payload key like `account:` looks like exactly the kind of
+key you want on an audit row, while the value serialises with
+`password_digest`, `token_digest`, `lookup_hash` and anything else the table
+happens to hold. Audit rows are append-only, so an unsafe default cannot be
+walked back.
+
+On the notifications path, records are dereferenced **after**
+`metadata_builder` runs, so a builder that needs real attributes still gets the
+record (`metadata_builder` has never applied to a direct `StandardAudit.record`
+call — pass the attributes you want in `metadata` there):
+
+```ruby
+config.metadata_builder = ->(metadata) {
+  metadata.merge(account_email: metadata[:account]&.email)
+}
+```
+
+`config.dereference_record_metadata = false` restores the pre-0.11.0 behaviour
+of writing full attributes. Prefer `metadata_builder` over turning it off.
+
 **Performance**: For high-volume applications, enable async processing and ensure your `audit_logs` table has appropriate indexes (the install generator adds them by default). Consider partitioning by `occurred_at` for very large tables.
 
 **Retention**: Set `retention_days` in your configuration and run `rake standard_audit:cleanup` via a scheduled job (e.g., cron or SolidQueue recurring). Archive before deleting if you need long-term storage.

data/lib/standard_audit/configuration.rb

@@ -7,10 +7,12 @@ module StandardAudit
                   :current_session_id_resolver,
                   :sensitive_keys, :sensitive_key_patterns,
                   :sensitive_key_exceptions, :filter_nested_metadata,
+                  :dereference_record_metadata,
                   :metadata_builder, :before_checksum_hooks,
                   :anonymizable_metadata_keys, :retention_days,
                   :audit_catalogue, :verify_audit_declarations,
-                  :raise_on_audit_write_error, :audit_write_error_handler
+                  :raise_on_audit_write_error, :audit_write_error_handler,
+                  :audit_error_context_key
 
     def initialize
       @subscriptions = []
@@ -71,6 +73,25 @@ module StandardAudit
       # append-only. Reserved keys are never descended into.
       @filter_nested_metadata = false
 
+      # When true (the default), an ActiveRecord object appearing anywhere in
+      # audit metadata is replaced by a reference —
+      # `{ "gid" => …, "type" => …, "id" => … }` — instead of being serialised
+      # with all of its attributes. See StandardAudit::RecordReference.
+      #
+      # Defaults to the SAFE behaviour, unlike `filter_nested_metadata`,
+      # because the values this catches are ones no host asked to record:
+      # `password_digest`, `token_digest`, `lookup_hash` and PKCE verifiers
+      # arriving as a side effect of a payload carrying `account:` or
+      # `session:`. Audit rows are append-only, so an unsafe default cannot be
+      # walked back — the rows already written keep whatever was in them.
+      #
+      # Set to false ONLY if an app genuinely depends on record attributes in
+      # metadata and has satisfied itself that no secret-bearing column can
+      # reach a row. Preferred alternative: keep this on and use
+      # `metadata_builder` to pull the specific attributes you want, which runs
+      # BEFORE dereferencing and still sees the record.
+      @dereference_record_metadata = true
+
       @metadata_builder = nil
 
       # Callables (or Symbols naming an AuditLog instance method) run on
@@ -124,6 +145,23 @@ module StandardAudit
       # write failure under either policy.
       @audit_write_error_handler = nil
 
+      # The `Rails.error.report` context key naming the audit action, for the
+      # built-in reporter. Exists because two apps independently overrode
+      # `audit_write_error_handler` for nothing but this key: both tag every
+      # OTHER audit-error report site with `audit_event:` (four sites in one
+      # app, twelve across ten files in the other), so adopting the gem's name
+      # left the operations layer as the only place forking the convention.
+      #
+      # That divergence fails silently and asymmetrically — a saved error-tracker
+      # search grouped on the host's key keeps working and simply stops
+      # containing operation write failures. Nothing errors, nothing goes red,
+      # and the search still looks healthy.
+      #
+      # A whole handler to rename one key is a lot of ceremony, and a handler
+      # written for that reason also silently opts out of every future
+      # improvement to the built-in reporter. Default keeps existing behaviour.
+      @audit_error_context_key = :audit_action
+
       # Retention defaults from ENV so it can be set per-environment without a
       # code change. Unset/blank/non-positive => nil (infinite retention, the
       # compliance-safe default that never auto-deletes). A host app can still

data/lib/standard_audit/operation/audit.rb

@@ -1,3 +1,5 @@
+require "ripper"
+
 module StandardAudit
   module Operation
     # The registry, the catalogue resolver, the write-error policy, and the
@@ -19,10 +21,20 @@ module StandardAudit
       #
       # The scan is FILE-scoped, not class-scoped — two operations defined in
       # one file share a verdict. Zeitwerk requires one class per file in a
-      # real app, so this only matters for fixtures. A commented-out or
-      # documented `audit!` in the same file also counts as a write site; the
-      # predicates deliberately err towards a false pass rather than a false
-      # failure, since the runtime guard is the authoritative check.
+      # real app, so this only matters for fixtures.
+      #
+      # Comments are stripped before scanning (see .strip_comments), so a
+      # documented or commented-out `audit!` is not a write site. Until 0.9.1
+      # it was, and the docstring here claimed the predicates "err towards a
+      # false pass rather than a false failure" — which was only ever true of
+      # {.missing_write_sites}. In the {.unexpected_write_sites} direction the
+      # same match produces a false FAILURE, so an `audit_none!` class that
+      # explained itself in prose failed the check. The claim was wrong for
+      # half its own surface, which is worse than the bug: it told anyone
+      # hitting the failure not to suspect the scanner.
+      #
+      # What remains genuinely heuristic: a string literal containing `audit!`
+      # still counts, and the runtime guard is still the authoritative check.
       WRITE_SITE_PATTERN = /(?<![\w.:])audit!\s*(?:\(|["':@$\w])/
 
       class << self
@@ -118,15 +130,24 @@ module StandardAudit
         # @raise [StandardError] the original error when
         #   `config.raise_on_audit_write_error` is true
         def handle_write_error(error, action:, operation:)
+          raising = StandardAudit.config.raise_on_audit_write_error
           handler = StandardAudit.config.audit_write_error_handler
 
           if handler
             handler.call(error, action: action, operation: operation)
-          else
+          elsif !raising
+            # Deliberately NOT reported when re-raising: the caller receives the
+            # error and owns it from here. Reporting as well produced two events
+            # for one failure in every host that both sets
+            # `raise_on_audit_write_error` and reports on the error it catches —
+            # which is most hosts that set the flag at all, since the flag exists
+            # for hosts that treat an unaudited write as a failure worth
+            # handling. And a host that does NOT catch it still gets the report,
+            # via its framework's own unhandled-error path.
             report_write_error(error, action: action, operation: operation)
           end
 
-          raise error if StandardAudit.config.raise_on_audit_write_error
+          raise error if raising
 
           nil
         end
@@ -228,7 +249,29 @@ module StandardAudit
           path = source_path(klass)
           return nil unless path && File.exist?(path)
 
-          File.read(path)
+          strip_comments(File.read(path))
+        end
+
+        # Comments are not write sites. Scanning raw source made a class that
+        # declares `audit_none!` and *explains why* in prose ("No direct audit!
+        # here") fail `unexpected_write_sites` — the scanner matched the word in
+        # the comment. That is a false failure, and it forced hosts to backtick
+        # the token in their own comments to appease a static check.
+        #
+        # Lexing rather than regex-stripping `#` to end-of-line, because the
+        # naive form eats `"#{interpolation}"` and `%w[#]`, which would trade a
+        # false failure for a false pass. A file Ripper cannot lex (syntax the
+        # running Ruby doesn't accept) falls back to the raw source: the old
+        # behaviour, which is conservative in the `audits` direction.
+        def strip_comments(src)
+          tokens = Ripper.lex(src)
+          return src if tokens.nil?
+
+          tokens.reject { |(_pos, type, _tok, _state)| type == :on_comment }
+                .map { |(_pos, _type, tok, _state)| tok }
+                .join
+        rescue StandardError
+          src
         end
 
         def subclassed?(klass)
@@ -244,7 +287,8 @@ module StandardAudit
           Rails.error.report(
             error,
             handled: true,
-            context: { audit_action: action, operation: operation.class.name }
+            context: { StandardAudit.config.audit_error_context_key => action,
+                       operation: operation.class.name }
           )
         end
       end

data/lib/standard_audit/record_reference.rb

@@ -0,0 +1,141 @@
+module StandardAudit
+  # Replaces ActiveRecord objects found in audit metadata with a stable
+  # *reference* instead of a snapshot of the record's whole attribute set.
+  #
+  # == Why this exists
+  #
+  # `ActiveSupport::Notifications` payloads routinely carry live records —
+  # `standard_id` publishes `account:`, `current_account:`, `session:` and
+  # `code_challenge:` — and an AR object serialises with every attribute it
+  # has. That put `account.password_digest`,
+  # `account.password_reset_token_digest`, `session.token_digest`,
+  # `session.lookup_hash` and `code_challenge.code` into `audit_logs` rows
+  # across the estate (rarebit-one/rarebit-ops#296).
+  #
+  # None of the three existing defences fired: `sensitive_keys` matches keys
+  # exactly and the secrets are attributes *underneath* `account:`;
+  # `filter_nested_metadata` is off by default so the filter never descended
+  # to them; and `account:` does not look sensitive at the top level.
+  #
+  # Key-based redaction is the wrong tool here — the leak is not "this key is
+  # sensitive", it is "this value is an entire database row". So the value is
+  # replaced wholesale, by type, before any key filtering happens.
+  #
+  # == What replaces a record
+  #
+  #   { "gid" => "gid://dummy/Account/1", "type" => "Account", "id" => "1" }
+  #
+  # A GlobalID string is the identifier this gem already uses for `actor`,
+  # `target` and `scope` (`actor_gid`), so an audit row stays resolvable with
+  # `GlobalID::Locator`. `type` and `id` ride along because they survive a
+  # record being deleted, an app whose GlobalID app name changes, and records
+  # that have no GlobalID at all (unpersisted ones): for those, `gid` is
+  # simply absent and the reference is still meaningful.
+  module RecordReference
+    # Written in place of a container that contains itself. Depth alone is NOT
+    # used as the guard: a plain depth cap would rewrite deeply-but-finitely
+    # nested metadata that holds no records at all, and permanently lose audit
+    # content in an append-only row to protect against a structure that is
+    # merely large. Only a genuine cycle — the one thing that cannot be
+    # serialised anyway — is replaced.
+    CIRCULAR = "[standard_audit: circular reference]".freeze
+
+    class << self
+      # Returns a copy of `value` with every ActiveRecord object — at any
+      # depth, inside Arrays, Hashes and Relations — replaced by a reference
+      # Hash. Structures containing no records are returned untouched (same
+      # object), so this is a no-op for ordinary metadata.
+      #
+      # Never mutates the input: notification payloads are shared with every
+      # other subscriber.
+      def call(value, ancestors = nil)
+        return reference_for(value) if record?(value)
+
+        container = relation?(value) || value.is_a?(Array) || hash_like?(value)
+        return value unless container
+
+        ancestors ||= {}.compare_by_identity
+        return CIRCULAR if ancestors.key?(value)
+
+        ancestors[value] = true
+        begin
+          if relation?(value)
+            map_collection(value.to_a, ancestors)
+          elsif value.is_a?(Array)
+            map_collection(value, ancestors)
+          else
+            map_hash(value, ancestors)
+          end
+        ensure
+          ancestors.delete(value)
+        end
+      end
+
+      # The reference written in place of a record.
+      def reference_for(record)
+        {
+          "gid" => global_id_for(record),
+          "type" => record.class.name,
+          "id" => record.id&.to_s
+        }.compact
+      end
+
+      private
+
+      def record?(value)
+        defined?(ActiveRecord::Base) && value.is_a?(ActiveRecord::Base)
+      end
+
+      def relation?(value)
+        defined?(ActiveRecord::Relation) && value.is_a?(ActiveRecord::Relation)
+      end
+
+      def hash_like?(value)
+        value.respond_to?(:each_pair) && value.respond_to?(:key?)
+      end
+
+      def map_collection(array, ancestors)
+        changed = false
+        mapped = array.map do |element|
+          replacement = call(element, ancestors)
+          changed ||= !replacement.equal?(element)
+          replacement
+        end
+
+        changed ? mapped : array
+      end
+
+      def map_hash(hash, ancestors)
+        changed = false
+        mapped = {}
+
+        hash.each_pair do |key, value|
+          # Reserved keys are gem-owned (`_tags`, `_source`) and are left
+          # exactly alone here, as they are in MetadataFilter.
+          if StandardAudit::RESERVED_METADATA_KEYS.include?(key.to_s)
+            mapped[key] = value
+            next
+          end
+
+          replacement = call(value, ancestors)
+          changed ||= !replacement.equal?(value)
+          mapped[key] = replacement
+        end
+
+        changed ? mapped : hash
+      end
+
+      # `to_global_id` raises for an unpersisted record and for a model that
+      # does not include GlobalID::Identification. Neither is a reason to fail
+      # an audit write, and neither is a reason to fall back to writing the
+      # attributes.
+      def global_id_for(record)
+        return nil unless record.respond_to?(:to_global_id)
+
+        record.to_global_id.to_s
+      rescue StandardError
+        nil
+      end
+    end
+  end
+end

data/lib/standard_audit/subscriber.rb

@@ -79,6 +79,16 @@ module StandardAudit
         raw_metadata = config.metadata_builder.call(raw_metadata)
       end
 
+      # ActiveRecord objects in the payload are replaced by a reference BEFORE
+      # any key-based redaction, because the leak they cause is not a
+      # sensitive *key* — it is a value that serialises as an entire database
+      # row (rarebit-one/rarebit-ops#296). Runs after `metadata_builder` so a
+      # host that derives fields from a record (`payload[:account].email`)
+      # still sees the record.
+      if config.dereference_record_metadata
+        raw_metadata = StandardAudit::RecordReference.call(raw_metadata)
+      end
+
       # Redaction lives in MetadataFilter, shared with StandardAudit.record.
       # This path previously carried its own copy that did *not* subtract
       # RESERVED_METADATA_KEYS, so `_tags`/`_source` were strippable here and

data/lib/standard_audit/version.rb

@@ -1,3 +1,3 @@
 module StandardAudit
-  VERSION = "0.9.0"
+  VERSION = "0.11.0"
 end

data/lib/standard_audit.rb

@@ -2,6 +2,7 @@ require "standard_audit/version"
 require "standard_audit/engine"
 require "standard_audit/configuration"
 require "standard_audit/metadata_filter"
+require "standard_audit/record_reference"
 require "standard_audit/sensitive_keys_dry_run"
 require "standard_audit/subscriber"
 require "standard_audit/event_subscriber"
@@ -52,9 +53,26 @@ module StandardAudit
 
       actor ||= config.current_actor_resolver.call
 
+      if block_given?
+        # Block form: instrument via ActiveSupport::Notifications and let the
+        # Subscriber write the row, which it does with its own dereferencing and
+        # filtering. Nothing built below would be used, so it is not built —
+        # dereferencing a Relation here would load it eagerly, before the block
+        # has run, purely to discard the result.
+        ActiveSupport::Notifications.instrument(event_type, metadata.merge(
+          actor: actor, target: target, scope: scope
+        )) do
+          yield
+        end
+        return
+      end
+
       # Redaction lives in MetadataFilter, shared with Subscriber, so the two
-      # write paths cannot drift apart.
-      filtered_metadata = MetadataFilter.call(metadata, config: config)
+      # write paths cannot drift apart. Record dereferencing is applied on both
+      # paths for the same reason: a snapshot of a whole row is as unrecoverable
+      # here as it is on the notifications path.
+      dereferenced = config.dereference_record_metadata ? RecordReference.call(metadata) : metadata
+      filtered_metadata = MetadataFilter.call(dereferenced, config: config)
 
       attrs = {
         event_type: event_type,
@@ -66,16 +84,6 @@ module StandardAudit
         metadata: filtered_metadata
       }
 
-      if block_given?
-        # Block form: instrument via ActiveSupport::Notifications
-        ActiveSupport::Notifications.instrument(event_type, metadata.merge(
-          actor: actor, target: target, scope: scope
-        )) do
-          yield
-        end
-        return
-      end
-
       gid_attrs = {
         actor_gid: actor&.to_global_id&.to_s,
         actor_type: actor&.class&.name,

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: standard_audit
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.11.0
 platform: ruby
 authors:
 - Jaryl Sim
@@ -114,6 +114,7 @@ files:
 - lib/standard_audit/metadata_filter.rb
 - lib/standard_audit/operation.rb
 - lib/standard_audit/operation/audit.rb
+- lib/standard_audit/record_reference.rb
 - lib/standard_audit/reference_preloading.rb
 - lib/standard_audit/rspec.rb
 - lib/standard_audit/rspec/operation.rb