standard_audit 0.14.1 → 0.15.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 052265cb7a2b77f0bdbbec540a7603112a26ad20173828a0009a5aee72d68270
4
- data.tar.gz: 3dd110875c9356705e363437c97ef35b27e8412eed4f54c523894d9dc8173ca2
3
+ metadata.gz: 3d166172ff4fb5afad44ee41bdc696458000759d778d26978feba930a1f0f15d
4
+ data.tar.gz: 04f546a81c899d026a38d5db3ae9ec18d0e8be7968bd6dc00fbb8bd521b01704
5
5
  SHA512:
6
- metadata.gz: abba2f5dac90739c1a96b1470030f81bcc967261af6effab9a3561edc1ec57115e19695aa7cebebb716b20a406bdfca2200a89afda7d826acd7de9a5379d5206
7
- data.tar.gz: 60985a94ecd78c1c47a9e66ffe7fb0df0ffd26928361c4a0b4ee1417a14eea31762aca75cb1e4805e117af3df668cd5e99f473ff3ddf05b6a920e48c35ef2f40
6
+ metadata.gz: 0ff1c20d8dcb9b23d6b6aac9e5c25c9c64ab2a31a3117f5a646ae6018d99bca5bdc09fe8c11010ccfeab83231d63bbd7354872c9dafac249719d54e18da97dbd
7
+ data.tar.gz: d546320a519c833a3015a01887fea2459d9fe83c936e417eaae240059c94ed1fab08666d71da0773229a475424edd04834c19396287a6454ba317d1f868301fc
data/CHANGELOG.md CHANGED
@@ -7,6 +7,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.15.1] - 2026-10-28
11
+
12
+ ### Fixed
13
+
14
+ - **`anonymize_actor!` also strips anonymizable keys from nested metadata.**
15
+ It used to drop `anonymizable_metadata_keys` (`email`, `name`, `ip_address`)
16
+ from the top level only, so a publisher that nests the subject's details
17
+ (standard_id's `social.auth.completed` carries `social_info: { email: ... }`)
18
+ kept them after erasure. With `filter_nested_metadata` on (the default since
19
+ 0.14.0) it now walks nested hashes and arrays, as the write-side filter does.
20
+
21
+ ## [0.15.0] - 2026-10-28
22
+
23
+ ### Security
24
+
25
+ - **`id_token` and `reset_url` are redacted by default.** Both are now in the
26
+ default `sensitive_keys`. An OIDC `id_token` is a bearer credential like the
27
+ `access_token`/`refresh_token` already listed, and a password-reset
28
+ `reset_url` embeds the live reset token, so a host subscribed to an event
29
+ that carries either wrote it into append-only rows. Nested metadata is
30
+ filtered by default, so `metadata: { tokens: { id_token: ... } }` is covered
31
+ too. Rows written before the upgrade are unchanged.
32
+
33
+ ### Added
34
+
35
+ - **A missed upgrade migration is no longer silent.** The first audit write in
36
+ each process checks `audit_logs` for the `previous_checksum` column and the
37
+ `[created_at, id]` and `checksum` indexes, and logs one warning naming
38
+ `rails generate standard_audit:add_previous_checksum` when any is missing
39
+ (`StandardAudit::SchemaCheck`; one cached `connection.indexes` read per
40
+ process). Without them no chain parents are recorded, so fork-tolerant
41
+ `verify_chain` is inactive. `config.schema_check` is `:warn` by default;
42
+ `:raise` fails writes with `StandardAudit::MissingUpgradeMigration`, `false`
43
+ skips the check. `StandardAudit::Checks::Schema` reports the same gap to
44
+ StandardHealth as a non-critical `:warn`.
45
+
46
+ ### Changed
47
+
48
+ - **`CreateAuditLogJob` no longer re-locates actor, target and scope.** The
49
+ enqueued attributes already carry the gid and type strings the row stores,
50
+ so the job writes them as they are instead of loading each record only to
51
+ serialise it back: up to three fewer SELECTs per async audit row. A gid whose
52
+ class has since been removed is now written instead of failing the job with
53
+ `NameError`. A `before_checksum` hook that reads `log.actor` (or target,
54
+ scope) still gets the record, located lazily on first read.
55
+
10
56
  ## [0.14.1] - 2026-10-22
11
57
 
12
58
  ### Changed
data/README.md CHANGED
@@ -683,6 +683,12 @@ StandardAudit.configure(baseline: true) do |config|
683
683
  # Only move it if your rollout slips, only to a future time, and never after
684
684
  # it has passed. See "Checksum algorithm versions".
685
685
  # config.canonical_checksum_since = Time.utc(2026, 10, 15)
686
+
687
+ # -- Schema check --
688
+ # What the first write in a process does when audit_logs lacks an upgrade
689
+ # migration: :warn (default) logs once, :raise fails writes, false skips.
690
+ # See "Detecting a missed upgrade".
691
+ # config.schema_check = :raise
686
692
  end
687
693
  ```
688
694
 
@@ -1141,6 +1147,25 @@ since it was written, so it adds no attestation the rows did not already carry.
1141
1147
  Rows it cannot resolve are reported as `unresolved` and keep failing
1142
1148
  verification — which is the point.
1143
1149
 
1150
+ #### Detecting a missed upgrade
1151
+
1152
+ The first audit write in each process checks that `audit_logs` has the
1153
+ `previous_checksum` column and the `[created_at, id]` and `checksum` indexes
1154
+ (`StandardAudit::SchemaCheck`, one cached `connection.indexes` read). When any
1155
+ is missing it logs one warning naming the generator above. Writes carry on
1156
+ either way. `config.schema_check` controls it: `:warn` (default), `:raise` (fail
1157
+ every write with `StandardAudit::MissingUpgradeMigration`), or `false` (skip).
1158
+
1159
+ On PostgreSQL with a large table, add `disable_ddl_transaction!` and
1160
+ `algorithm: :concurrently` to the generated `add_index` calls.
1161
+
1162
+ To surface the gap in `/health/ready` as well, register the non-critical
1163
+ `StandardAudit::Checks::Schema` next to `Checks::Retention`:
1164
+
1165
+ ```ruby
1166
+ c.register_check :audit_schema, StandardAudit::Checks::Schema, critical: false
1167
+ ```
1168
+
1144
1169
  **Do not run `backfill_checksums!` to make a red `verify_chain` go green.** It
1145
1170
  re-signs rows from their current contents, so it attests only that a script ran.
1146
1171
  It is for rows that never had a checksum at all (pre-feature data).
@@ -2,46 +2,15 @@ module StandardAudit
2
2
  class CreateAuditLogJob < ActiveJob::Base
3
3
  queue_as { StandardAudit.config.queue_name }
4
4
 
5
+ # `attrs` already carries actor/target/scope as the gid and type strings
6
+ # the row stores (StandardAudit.persist serialised them at enqueue time),
7
+ # so they are written as they are. Earlier versions located each record
8
+ # only to serialise it back to the same strings, which cost up to three
9
+ # SELECTs per row and failed the job outright (NameError) when a gid's
10
+ # class no longer existed. A `before_checksum` hook that reads `log.actor` still gets the
11
+ # record: the reader locates it lazily on first use.
5
12
  def perform(attrs)
6
- attrs = attrs.symbolize_keys
7
-
8
- actor_gid = attrs.delete(:actor_gid)
9
- target_gid = attrs.delete(:target_gid)
10
- scope_gid = attrs.delete(:scope_gid)
11
- actor_type = attrs.delete(:actor_type)
12
- target_type = attrs.delete(:target_type)
13
- scope_type = attrs.delete(:scope_type)
14
-
15
- log = StandardAudit::AuditLog.new(attrs)
16
-
17
- if actor_gid.present?
18
- begin
19
- log.actor = GlobalID::Locator.locate(actor_gid)
20
- rescue ActiveRecord::RecordNotFound
21
- log.actor_gid = actor_gid
22
- log.actor_type = actor_type
23
- end
24
- end
25
-
26
- if target_gid.present?
27
- begin
28
- log.target = GlobalID::Locator.locate(target_gid)
29
- rescue ActiveRecord::RecordNotFound
30
- log.target_gid = target_gid
31
- log.target_type = target_type
32
- end
33
- end
34
-
35
- if scope_gid.present?
36
- begin
37
- log.scope = GlobalID::Locator.locate(scope_gid)
38
- rescue ActiveRecord::RecordNotFound
39
- log.scope_gid = scope_gid
40
- log.scope_type = scope_type
41
- end
42
- end
43
-
44
- log.save!
13
+ StandardAudit::AuditLog.new(attrs).save!
45
14
  end
46
15
  end
47
16
  end
@@ -124,8 +124,7 @@ module StandardAudit
124
124
  attrs[:target_type] = "[anonymized]" if log.target_gid == gid
125
125
 
126
126
  if log.metadata.present? && anonymizable_keys.any?
127
- cleaned_metadata = log.metadata.reject { |k, _| anonymizable_keys.include?(k.to_s) }
128
- attrs[:metadata] = cleaned_metadata
127
+ attrs[:metadata] = strip_anonymizable_keys(log.metadata, anonymizable_keys)
129
128
  end
130
129
 
131
130
  attrs[:anonymized_at] = log.anonymized_at || stamp if stamp
@@ -136,6 +135,31 @@ module StandardAudit
136
135
  count
137
136
  end
138
137
 
138
+ # Drops the anonymizable keys from +metadata+. Nested hashes (and hashes
139
+ # inside arrays) are walked too when `filter_nested_metadata` is on, as the
140
+ # write-side MetadataFilter does: a publisher that nests the subject's
141
+ # email (standard_id's social `social_info: { email: ... }`) would
142
+ # otherwise keep it after erasure.
143
+ def self.strip_anonymizable_keys(metadata, keys)
144
+ cleaned = metadata.reject { |k, _| keys.include?(k.to_s) }
145
+ return cleaned unless StandardAudit.config.filter_nested_metadata
146
+
147
+ cleaned.to_h { |k, v| [k, strip_anonymizable_descendant(v, keys)] }
148
+ end
149
+
150
+ def self.strip_anonymizable_descendant(value, keys)
151
+ case value
152
+ when Hash
153
+ value.reject { |k, _| keys.include?(k.to_s) }
154
+ .to_h { |k, v| [k, strip_anonymizable_descendant(v, keys)] }
155
+ when Array
156
+ value.map { |element| strip_anonymizable_descendant(element, keys) }
157
+ else
158
+ value
159
+ end
160
+ end
161
+ private_class_method :strip_anonymizable_keys, :strip_anonymizable_descendant
162
+
139
163
  # True when the table carries `anonymized_at` (the
140
164
  # `standard_audit:add_anonymized_at` migration, or a 0.12+ install).
141
165
  def self.anonymization_column?
@@ -679,6 +703,7 @@ module StandardAudit
679
703
  # Record keeps a timestamp that is already set) so the decision is made
680
704
  # from exactly the value that is stored and that verification re-reads.
681
705
  def compute_checksum
706
+ StandardAudit::SchemaCheck.verify!
682
707
  self.created_at ||= Time.current if has_attribute?(:created_at)
683
708
  previous = self.class.chain_tip_checksum
684
709
  self.previous_checksum = previous if self.class.chain_parent_column?
@@ -1,18 +1,23 @@
1
1
  class AddPreviousChecksumToAuditLogs < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
2
- def change
3
- # On PostgreSQL with a large audit_logs table, add
4
- # `disable_ddl_transaction!` above and pass `algorithm: :concurrently` to
5
- # the add_index calls — StrongMigrations will ask for it.
6
- #
7
- # Nullable and index-only: no data is rewritten. Existing rows keep a NULL
8
- # parent and are verified by position and parent recovery, exactly as they
9
- # are today. Run `rake standard_audit:relink_checksums` afterwards to
10
- # record the parent each existing row was actually signed against.
11
- add_column :audit_logs, :previous_checksum, :string, limit: 64
2
+ # On PostgreSQL with a large audit_logs table, add
3
+ # `disable_ddl_transaction!` here and pass `algorithm: :concurrently` to
4
+ # the add_index calls — StrongMigrations will ask for it.
5
+ #
6
+ # Nullable and index-only: no data is rewritten. Existing rows keep a NULL
7
+ # parent and are verified by position and parent recovery, exactly as they
8
+ # are today. Run `rake standard_audit:relink_checksums` afterwards to
9
+ # record the parent each existing row was actually signed against.
10
+ #
11
+ # Every step in `up` is guarded, so the migration also repairs a table that
12
+ # has the column but is missing an index (StandardAudit::SchemaCheck warns
13
+ # about either gap with this same generator). `up`/`down` rather than
14
+ # `change`, because guards inside `change` would make the rollback a no-op.
15
+ def up
16
+ add_column :audit_logs, :previous_checksum, :string, limit: 64 unless column_exists?(:audit_logs, :previous_checksum)
12
17
  # Not used by the gem itself: verification reads previous_checksum off rows
13
18
  # it has already loaded. It is here for forensics — "what forked off this
14
19
  # row?" — which is the question you ask once verification flags something.
15
- add_index :audit_logs, :previous_checksum
20
+ add_index :audit_logs, :previous_checksum unless index_exists?(:audit_logs, :previous_checksum)
16
21
  # verify_chain / backfill_checksums! walk the table with a keyset cursor
17
22
  # ordered by (created_at, id).
18
23
  add_index :audit_logs, [:created_at, :id] unless index_exists?(:audit_logs, [:created_at, :id])
@@ -20,4 +25,12 @@ class AddPreviousChecksumToAuditLogs < ActiveRecord::Migration[<%= ActiveRecord:
20
25
  # the in-memory recovery window — the path that reports a removed row.
21
26
  add_index :audit_logs, :checksum unless index_exists?(:audit_logs, :checksum)
22
27
  end
28
+
29
+ # Removes the column (and its index). The [created_at, id] and checksum
30
+ # indexes stay: the table may have had them before this migration, and
31
+ # they cost nothing to keep.
32
+ def down
33
+ remove_index :audit_logs, :previous_checksum if index_exists?(:audit_logs, :previous_checksum)
34
+ remove_column :audit_logs, :previous_checksum if column_exists?(:audit_logs, :previous_checksum)
35
+ end
23
36
  end
@@ -38,8 +38,8 @@ StandardAudit.configure(baseline: true) do |config|
38
38
  #
39
39
  # Keys to strip from metadata. Matching is EXACT on the key name (defaults
40
40
  # include: password, password_confirmation, token, secret, api_key,
41
- # access_token, refresh_token, private_key, certificate_chain, ssn,
42
- # credit_card, authorization).
41
+ # access_token, refresh_token, id_token, reset_url, private_key,
42
+ # certificate_chain, ssn, credit_card, authorization).
43
43
  # config.sensitive_keys += %i[my_custom_secret]
44
44
  #
45
45
  # Regexps matched against every key name, in addition to the exact list.
@@ -0,0 +1,29 @@
1
+ module StandardAudit
2
+ module Checks
3
+ # A StandardHealth-compatible readiness check that warns when audit_logs
4
+ # is missing an upgrade migration (see StandardAudit::SchemaCheck).
5
+ #
6
+ # Duck-typed like Checks::Retention: `#initialize(name:, critical:)` +
7
+ # `#run`, with no dependency on standard_health. Register it NON-critical:
8
+ #
9
+ # c.register_check :audit_schema,
10
+ # StandardAudit::Checks::Schema,
11
+ # critical: false
12
+ #
13
+ # A :warn rolls /health/ready up to :degraded (still HTTP 200), so the gap
14
+ # shows in the readiness JSON without failing the probe.
15
+ class Schema
16
+ def initialize(name: :audit_schema, critical: false)
17
+ @name = name
18
+ @critical = critical
19
+ end
20
+
21
+ def run
22
+ gaps = StandardAudit::SchemaCheck.missing
23
+ return { status: :ok } if gaps.empty?
24
+
25
+ { status: :warn, missing: gaps, message: StandardAudit::SchemaCheck.message(gaps) }
26
+ end
27
+ end
28
+ end
29
+ end
@@ -17,7 +17,7 @@ module StandardAudit
17
17
  :audit_catalogue, :verify_audit_declarations,
18
18
  :raise_on_audit_write_error, :audit_write_error_handler,
19
19
  :audit_error_context_key, :error_reporter,
20
- :canonical_checksum_since
20
+ :canonical_checksum_since, :schema_check
21
21
 
22
22
  def initialize
23
23
  @subscriptions = []
@@ -86,7 +86,8 @@ module StandardAudit
86
86
  # rename it (e.g. :authorization_policy) to avoid accidental filtering.
87
87
  @sensitive_keys = %i[
88
88
  password password_confirmation token secret
89
- api_key access_token refresh_token
89
+ api_key access_token refresh_token id_token
90
+ reset_url
90
91
  private_key certificate_chain
91
92
  ssn credit_card authorization
92
93
  ]
@@ -253,6 +254,11 @@ module StandardAudit
253
254
  # advisory (StandardAudit::Checks::Retention) only warns on nil, which
254
255
  # means nobody chose.
255
256
  @retention_days = self.class.retention_days_from_env
257
+
258
+ # What the first audit write in a process does when audit_logs lacks an
259
+ # upgrade migration (StandardAudit::SchemaCheck): :warn logs one warning
260
+ # naming the generator, :raise fails every write, false skips the check.
261
+ @schema_check = :warn
256
262
  end
257
263
 
258
264
  # The retention window as a positive Integer of days, or nil when rows are
@@ -0,0 +1,93 @@
1
+ module StandardAudit
2
+ # Raised by SchemaCheck when `config.schema_check = :raise` and audit_logs
3
+ # is missing an upgrade migration.
4
+ class MissingUpgradeMigration < StandardError; end
5
+
6
+ # Detects an audit_logs table that predates the gem's upgrade migrations.
7
+ #
8
+ # Fresh installs get the `previous_checksum` column and the
9
+ # `[created_at, id]` and `checksum` indexes from the install template, but an
10
+ # older install only gets them by running the upgrade generator, and nothing
11
+ # used to notice when a host never did. The gem keeps writing either way
12
+ # (`chain_parent_column?` is simply false), so the cost is silent: no chain
13
+ # parents are recorded, fork-tolerant `verify_chain` is inactive, and
14
+ # `verify_chain` / `backfill_checksums!` page through the table without the
15
+ # index their keyset cursor needs.
16
+ #
17
+ # `verify!` runs on the first audit write in each process (both the
18
+ # single-row and the batch path) and is cached, so it costs one
19
+ # `connection.indexes` read per process. What it does is governed by
20
+ # `config.schema_check`:
21
+ #
22
+ # :warn (default) log one warning naming the generator to run
23
+ # :raise raise MissingUpgradeMigration on every write
24
+ # false skip the check
25
+ #
26
+ # The same result backs the StandardAudit::Checks::Schema readiness check.
27
+ module SchemaCheck
28
+ UPGRADE_COMMAND = "bin/rails generate standard_audit:add_previous_checksum && " \
29
+ "bin/rails db:migrate && bin/rails standard_audit:relink_checksums".freeze
30
+
31
+ class << self
32
+ # The upgrade pieces audit_logs lacks, as human-readable names; empty
33
+ # when the table is current. Cached per process; `reset!` clears it.
34
+ # Returns an empty Array, uncached, when the table cannot be introspected
35
+ # (no connection, no table yet): the write itself reports that failure,
36
+ # and the next call tries again.
37
+ def missing
38
+ return @missing if defined?(@missing)
39
+
40
+ gaps = detect_missing
41
+ gaps.nil? ? [] : (@missing = gaps)
42
+ end
43
+
44
+ def verify!
45
+ mode = StandardAudit.config.schema_check
46
+ return if mode.nil? || mode == false
47
+
48
+ gaps = missing
49
+ return if gaps.empty?
50
+
51
+ raise MissingUpgradeMigration, message(gaps) if mode == :raise
52
+ return if @warned
53
+
54
+ @warned = true
55
+ log_warning(message(gaps))
56
+ end
57
+
58
+ def message(gaps = missing)
59
+ "standard_audit: audit_logs is missing #{gaps.join(", ")} from an upgrade migration. " \
60
+ "Chain parents are not recorded, so fork-tolerant verify_chain is inactive, and " \
61
+ "verify_chain/backfill_checksums! walk the table without their index. " \
62
+ "Run: #{UPGRADE_COMMAND} (on PostgreSQL, add disable_ddl_transaction! and " \
63
+ "algorithm: :concurrently to the generated add_index calls). " \
64
+ "Set config.schema_check = :raise to fail writes instead, or false to silence this."
65
+ end
66
+
67
+ def reset!
68
+ remove_instance_variable(:@missing) if defined?(@missing)
69
+ @warned = false
70
+ end
71
+
72
+ private
73
+
74
+ def detect_missing
75
+ model = StandardAudit::AuditLog
76
+ indexes = model.connection.indexes(model.table_name).map { |index| Array(index.columns).map(&:to_s) }
77
+
78
+ gaps = []
79
+ gaps << "the previous_checksum column" unless model.column_names.include?("previous_checksum")
80
+ gaps << "the [created_at, id] index" unless indexes.any? { |cols| cols.first(2) == %w[created_at id] }
81
+ gaps << "the checksum index" unless indexes.any? { |cols| cols.first == "checksum" }
82
+ gaps
83
+ rescue ActiveRecord::ActiveRecordError
84
+ nil
85
+ end
86
+
87
+ def log_warning(text)
88
+ logger = defined?(Rails) && Rails.respond_to?(:logger) ? Rails.logger : nil
89
+ logger ? logger.warn(text) : Kernel.warn(text)
90
+ end
91
+ end
92
+ end
93
+ end
@@ -1,3 +1,3 @@
1
1
  module StandardAudit
2
- VERSION = "0.14.1"
2
+ VERSION = "0.15.1"
3
3
  end
@@ -27,7 +27,9 @@ require "standard_audit/reference_preloading"
27
27
  require "standard_audit/operation"
28
28
  require "standard_audit/auditable"
29
29
  require "standard_audit/audit_scope"
30
+ require "standard_audit/schema_check"
30
31
  require "standard_audit/checks/retention"
32
+ require "standard_audit/checks/schema"
31
33
  require "standard_audit/auth_failure"
32
34
 
33
35
  module StandardAudit
@@ -335,6 +337,7 @@ module StandardAudit
335
337
  end
336
338
 
337
339
  def flush_batch(buffer)
340
+ StandardAudit::SchemaCheck.verify!
338
341
  now = Time.current
339
342
  records_parent = StandardAudit::AuditLog.chain_parent_column?
340
343
  previous_checksum = StandardAudit::AuditLog.chain_tip_checksum
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: standard_audit
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.14.1
4
+ version: 0.15.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jaryl Sim
@@ -109,6 +109,7 @@ files:
109
109
  - lib/standard_audit/auditable.rb
110
110
  - lib/standard_audit/auth_failure.rb
111
111
  - lib/standard_audit/checks/retention.rb
112
+ - lib/standard_audit/checks/schema.rb
112
113
  - lib/standard_audit/checksum.rb
113
114
  - lib/standard_audit/checksum/key_order_search.rb
114
115
  - lib/standard_audit/configuration.rb
@@ -124,6 +125,7 @@ files:
124
125
  - lib/standard_audit/rspec/coverage.rb
125
126
  - lib/standard_audit/rspec/matchers.rb
126
127
  - lib/standard_audit/rspec/operation.rb
128
+ - lib/standard_audit/schema_check.rb
127
129
  - lib/standard_audit/sensitive_keys_dry_run.rb
128
130
  - lib/standard_audit/subscriber.rb
129
131
  - lib/standard_audit/version.rb