ez_logs_agent 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +38 -0
- data/lib/ez_logs_agent/buffer.rb +14 -0
- data/lib/ez_logs_agent/bulk_sql_parser.rb +11 -1
- data/lib/ez_logs_agent/capturers/active_job_capturer.rb +28 -3
- data/lib/ez_logs_agent/capturers/bulk_database_capturer.rb +229 -19
- data/lib/ez_logs_agent/capturers/database_capturer.rb +36 -10
- data/lib/ez_logs_agent/sensitive_patterns.rb +25 -7
- data/lib/ez_logs_agent/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 96fd8717fab4330e842769b9a4ae03902ccb2487e24c418d54601c99862da306
|
|
4
|
+
data.tar.gz: e0967b4bfe22b2abb5946c213378a615f3d399a9c0b8d8dd3f002f731bda9444
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bc502ab7e6ec65dab0f691005e4e0a71a23182a184c2ce1a188933ebe01db1155b3e4e500a59d23e7275c1a836450317a63e262d0d0babbf5123120be1718809
|
|
7
|
+
data.tar.gz: cc13f82eece53f6deb53c07ad324cf74fce412d06112abfa86c5ca76a829ed01a1cb421a63125489d969ee2e21efb3d9cb03f5b65412db9792330edca30be271
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,44 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.2.1] — 2026-06-05
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
- **Bulk-op row counts on Postgres.** The PG adapter does not populate
|
|
9
|
+
`payload[:row_count]` for plain `DELETE`/`UPDATE` notifications, so
|
|
10
|
+
the first 0.2.0 builds shipped `row_count: 0` for the cases that
|
|
11
|
+
matter most. `BulkDatabaseCapturer` now prepends a tiny shim onto
|
|
12
|
+
`ActiveRecord::Relation#delete_all` / `#update_all` that stashes the
|
|
13
|
+
returned row count and back-fills it onto the just-pushed event via
|
|
14
|
+
`Buffer.peek_last`. No change to the wire shape — `row_count` now
|
|
15
|
+
carries the real number.
|
|
16
|
+
- **Model resolution in Rails dev mode.** `ActiveRecord::Base.descendants`
|
|
17
|
+
only sees eager-loaded models. The resolver now falls through to
|
|
18
|
+
`safe_constantize` of the classified table name and verifies the
|
|
19
|
+
reconstructed class actually owns that table, so bulk ops on
|
|
20
|
+
lazy-autoloaded models in dev / test no longer silently drop.
|
|
21
|
+
- **Rails Query Log Tags noise in `where_template`.** The parser now
|
|
22
|
+
strips `/*application='X',action='Y'*/` comments before extracting
|
|
23
|
+
the WHERE clause, so the humanized filter line on the server reads
|
|
24
|
+
cleanly instead of leaking the framework's instrumentation tags.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- **Framework-rewrite filter narrowed.** The 0.2.0 filter that swallowed
|
|
28
|
+
Rails-generated `SET col = NULL` writes was too aggressive — it also
|
|
29
|
+
hid deliberate "null this column out" operations the customer wrote.
|
|
30
|
+
The filter now only drops `COALESCE`-shaped counter bumps and
|
|
31
|
+
empty-`SET` shells; honest `SET col = NULL` writes are captured and
|
|
32
|
+
render as "Cleared col" on the timeline.
|
|
33
|
+
- **Hot-path performance.** `capture_jobs`, `capture_database`, the
|
|
34
|
+
excluded-tables / excluded-job-classes / display-name maps, and the
|
|
35
|
+
user-extended sensitive-key patterns are now memoized at install
|
|
36
|
+
time across `ActiveJobCapturer`, `DatabaseCapturer`,
|
|
37
|
+
`BulkDatabaseCapturer`, and `SensitivePatterns`. The
|
|
38
|
+
`sql.active_record` subscriber is also tuned: 5-arity block to avoid
|
|
39
|
+
splat allocation, cheap `end_with?` name-prefilter ahead of any
|
|
40
|
+
parsing, and an early bail before `safe_constantize`. Bulk capture
|
|
41
|
+
overhead measured below the noise floor on a 10 ms reference query.
|
|
42
|
+
|
|
5
43
|
## [0.2.0] — 2026-06-05
|
|
6
44
|
|
|
7
45
|
### Added
|
data/lib/ez_logs_agent/buffer.rb
CHANGED
|
@@ -59,6 +59,20 @@ module EzLogsAgent
|
|
|
59
59
|
# Best effort, ignore failures
|
|
60
60
|
end
|
|
61
61
|
|
|
62
|
+
# Returns the most recently pushed event WITHOUT removing it.
|
|
63
|
+
# Used by BulkDatabaseCapturer to backfill the affected-row count
|
|
64
|
+
# after the relation method's return value is known (Rails'
|
|
65
|
+
# payload[:row_count] is unreliable for plain DELETE on PG).
|
|
66
|
+
# Mutating the returned hash in place is intentional and safe —
|
|
67
|
+
# the event hasn't been flushed yet.
|
|
68
|
+
# @return [Hash, nil] The last event, or nil if empty.
|
|
69
|
+
def peek_last
|
|
70
|
+
@monitor.synchronize { @queue.last }
|
|
71
|
+
rescue => error
|
|
72
|
+
log_error("[Buffer] peek_last failed: #{error.message}")
|
|
73
|
+
nil
|
|
74
|
+
end
|
|
75
|
+
|
|
62
76
|
private
|
|
63
77
|
|
|
64
78
|
def max_size
|
|
@@ -63,7 +63,7 @@ module EzLogsAgent
|
|
|
63
63
|
def parse(sql:, type_casted_binds:)
|
|
64
64
|
return { unparseable: true } if sql.nil? || sql.empty?
|
|
65
65
|
|
|
66
|
-
sql_stripped = sql.strip
|
|
66
|
+
sql_stripped = strip_query_log_tags(sql.strip)
|
|
67
67
|
|
|
68
68
|
case sql_stripped
|
|
69
69
|
when /\ADELETE FROM /i
|
|
@@ -84,6 +84,16 @@ module EzLogsAgent
|
|
|
84
84
|
{ unparseable: true }
|
|
85
85
|
end
|
|
86
86
|
|
|
87
|
+
# Strip Rails 7+ Query Log Tags (`/*application='X',action='Y'*/`)
|
|
88
|
+
# AND any other trailing SQL comments. They land at the end of every
|
|
89
|
+
# statement when `config.active_record.query_log_tags_enabled = true`
|
|
90
|
+
# and are pure noise on the timeline — they leak the host app's name
|
|
91
|
+
# and controller into the user-visible filter line. Removing them at
|
|
92
|
+
# parse time means we never ship them on the wire.
|
|
93
|
+
def strip_query_log_tags(sql)
|
|
94
|
+
sql.gsub(%r{/\*.*?\*/}m, "").rstrip
|
|
95
|
+
end
|
|
96
|
+
|
|
87
97
|
# Returns the symbolic operation name we expect downstream. Detected
|
|
88
98
|
# from SQL shape, independent of the `payload[:name]` Rails version
|
|
89
99
|
# variance (see plan §"insert_all/upsert_all payload :name varies").
|
|
@@ -51,6 +51,25 @@ module EzLogsAgent
|
|
|
51
51
|
def install
|
|
52
52
|
return unless defined?(ActiveJob)
|
|
53
53
|
|
|
54
|
+
# Memoize config values that the per-job hot path reads on
|
|
55
|
+
# every execute. Without this, capture_execution dispatches
|
|
56
|
+
# into EzLogsAgent.configuration twice per job (once for
|
|
57
|
+
# capture_jobs, once for all_excluded_job_classes). On
|
|
58
|
+
# job-heavy apps that's measurable. Runtime mutations need
|
|
59
|
+
# uninstall! + install.
|
|
60
|
+
@capture_enabled =
|
|
61
|
+
begin
|
|
62
|
+
EzLogsAgent.configuration.capture_jobs
|
|
63
|
+
rescue StandardError
|
|
64
|
+
false
|
|
65
|
+
end
|
|
66
|
+
@excluded_job_classes =
|
|
67
|
+
begin
|
|
68
|
+
EzLogsAgent.configuration.all_excluded_job_classes.dup.freeze
|
|
69
|
+
rescue StandardError
|
|
70
|
+
[].freeze
|
|
71
|
+
end
|
|
72
|
+
|
|
54
73
|
install_serialization_hooks unless @serialization_installed
|
|
55
74
|
|
|
56
75
|
ActiveJob::Base.before_enqueue do |job|
|
|
@@ -101,7 +120,11 @@ module EzLogsAgent
|
|
|
101
120
|
# @param block [Proc] The job execution block
|
|
102
121
|
# @return [Object] The result of the job execution
|
|
103
122
|
def capture_execution(job, block)
|
|
104
|
-
|
|
123
|
+
# Memoized at install time for hot-path perf. If we haven't
|
|
124
|
+
# installed yet (specs that test capture_execution directly),
|
|
125
|
+
# fall back to a live config read so the behavior matches.
|
|
126
|
+
enabled = defined?(@capture_enabled) ? @capture_enabled : EzLogsAgent.configuration.capture_jobs
|
|
127
|
+
return block.call unless enabled
|
|
105
128
|
|
|
106
129
|
if sidekiq_adapter?(job)
|
|
107
130
|
EzLogsAgent::Logger.debug("[ActiveJobCapturer] Skipping capture (Sidekiq adapter)")
|
|
@@ -196,8 +219,10 @@ module EzLogsAgent
|
|
|
196
219
|
# @param job [ActiveJob::Base] The job instance
|
|
197
220
|
# @return [Boolean] true if excluded, false otherwise
|
|
198
221
|
def excluded_job_class?(job)
|
|
199
|
-
|
|
200
|
-
|
|
222
|
+
# Memoized at install time for perf. Fall back to a live read
|
|
223
|
+
# for specs that don't call install (see capture_execution).
|
|
224
|
+
excluded = defined?(@excluded_job_classes) ? @excluded_job_classes : EzLogsAgent.configuration.all_excluded_job_classes
|
|
225
|
+
excluded.include?(job.class.name)
|
|
201
226
|
rescue StandardError
|
|
202
227
|
false
|
|
203
228
|
end
|
|
@@ -108,16 +108,107 @@ module EzLogsAgent
|
|
|
108
108
|
def install
|
|
109
109
|
return if @installed
|
|
110
110
|
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
111
|
+
# Cache configuration values that the hot path checks per
|
|
112
|
+
# notification. Re-read on every install (specs reinstall
|
|
113
|
+
# after toggling config). Runtime mutations to these settings
|
|
114
|
+
# require uninstall! + install to take effect — acceptable
|
|
115
|
+
# because nobody flips this at runtime in production.
|
|
116
|
+
@capture_enabled =
|
|
117
|
+
begin
|
|
118
|
+
::EzLogsAgent.configuration.capture_database
|
|
119
|
+
rescue StandardError
|
|
120
|
+
false
|
|
121
|
+
end
|
|
122
|
+
@excluded_tables =
|
|
123
|
+
begin
|
|
124
|
+
::EzLogsAgent.configuration.all_excluded_tables.dup.freeze
|
|
125
|
+
rescue StandardError
|
|
126
|
+
[].freeze
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
install_row_count_capture!
|
|
130
|
+
|
|
131
|
+
# 5-arity block bypasses the `*args` splat allocation per
|
|
132
|
+
# notification — measurable on hot paths where we ignore
|
|
133
|
+
# ~99% of events. Block accepts positional args matching the
|
|
134
|
+
# AS::N convention: (name, start, finish, id, payload).
|
|
135
|
+
@subscriber = ::ActiveSupport::Notifications.subscribe("sql.active_record") do |name, started, finished, _id, payload|
|
|
136
|
+
handle_notification(name, started, finished, payload)
|
|
117
137
|
end
|
|
118
138
|
@installed = true
|
|
119
139
|
end
|
|
120
140
|
|
|
141
|
+
# Patches ActiveRecord::Relation's bulk methods to backfill the
|
|
142
|
+
# affected-row count on the most recently captured bulk_database
|
|
143
|
+
# event. See the comment on RelationRowCountStash below for why
|
|
144
|
+
# this is necessary (Rails' payload[:row_count] is unreliable
|
|
145
|
+
# for plain DELETE/UPDATE on PG).
|
|
146
|
+
def install_row_count_capture!
|
|
147
|
+
return if @relation_patched
|
|
148
|
+
return unless defined?(::ActiveRecord::Relation)
|
|
149
|
+
|
|
150
|
+
::ActiveRecord::Relation.prepend(RelationRowCountStash)
|
|
151
|
+
@relation_patched = true
|
|
152
|
+
rescue StandardError => e
|
|
153
|
+
# Patching is best-effort — if AR's Relation isn't there or the
|
|
154
|
+
# prepend raises, the capturer still works with payload[:row_count].
|
|
155
|
+
::EzLogsAgent::Logger.debug("[BulkDatabaseCapturer] could not patch Relation: #{e.class}: #{e.message}")
|
|
156
|
+
end
|
|
157
|
+
|
|
158
|
+
# Patches the row_count on the most recently buffered bulk_database
|
|
159
|
+
# event when its model matches `model_class`. Called from the
|
|
160
|
+
# RelationRowCountStash module immediately after `super` returns
|
|
161
|
+
# from delete_all / update_all. The buffer's `peek_last` API is
|
|
162
|
+
# the lightweight read path; we mutate in place because the
|
|
163
|
+
# event hash hasn't been serialized yet.
|
|
164
|
+
def backfill_row_count(real_count, model_class)
|
|
165
|
+
return if real_count.nil?
|
|
166
|
+
|
|
167
|
+
tail = ::EzLogsAgent::Buffer.peek_last
|
|
168
|
+
return unless tail.is_a?(Hash)
|
|
169
|
+
return unless tail[:source_type] == "bulk_database"
|
|
170
|
+
return unless tail.dig(:source_data, :model_class) == model_class.name
|
|
171
|
+
|
|
172
|
+
tail[:source_data][:row_count] = real_count
|
|
173
|
+
if (rids = tail[:resource_ids]).is_a?(Array) && rids.first.is_a?(Hash)
|
|
174
|
+
rids.first[:resource_id] = "bulk:#{real_count}"
|
|
175
|
+
end
|
|
176
|
+
rescue StandardError => e
|
|
177
|
+
::EzLogsAgent::Logger.debug("[BulkDatabaseCapturer] backfill_row_count failed: #{e.class}: #{e.message}")
|
|
178
|
+
end
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# Backfills the affected-row count on the most recently captured
|
|
182
|
+
# bulk_database event. Order of operations:
|
|
183
|
+
#
|
|
184
|
+
# 1. Customer calls Relation#delete_all (or update_all).
|
|
185
|
+
# 2. AR runs the SQL → sql.active_record fires → our AS::N
|
|
186
|
+
# handler captures the event with row_count=payload[:row_count]
|
|
187
|
+
# (often 0 on PG for plain DELETE).
|
|
188
|
+
# 3. delete_all's super returns the affected count to us.
|
|
189
|
+
# 4. We patch the most-recently-buffered bulk_database event's
|
|
190
|
+
# source_data[:row_count] AND its sentinel resource_id so the
|
|
191
|
+
# timeline shows the real number.
|
|
192
|
+
#
|
|
193
|
+
# This is a thin shim — only the count is touched, never the SQL,
|
|
194
|
+
# context, or wire shape. If the buffer's tail isn't a bulk_database
|
|
195
|
+
# event for our model (e.g. AS::N skipped it), we leave it alone.
|
|
196
|
+
module RelationRowCountStash
|
|
197
|
+
def delete_all
|
|
198
|
+
result = super
|
|
199
|
+
::EzLogsAgent::Capturers::BulkDatabaseCapturer.backfill_row_count(result, klass) if result.is_a?(Integer)
|
|
200
|
+
result
|
|
201
|
+
end
|
|
202
|
+
|
|
203
|
+
def update_all(*args, **kwargs, &block)
|
|
204
|
+
result = super
|
|
205
|
+
::EzLogsAgent::Capturers::BulkDatabaseCapturer.backfill_row_count(result, klass) if result.is_a?(Integer)
|
|
206
|
+
result
|
|
207
|
+
end
|
|
208
|
+
end
|
|
209
|
+
|
|
210
|
+
class << self
|
|
211
|
+
|
|
121
212
|
# Removes the subscription. Specs use this between examples to
|
|
122
213
|
# avoid leaked subscribers; production never calls it.
|
|
123
214
|
def uninstall!
|
|
@@ -132,8 +223,22 @@ module EzLogsAgent
|
|
|
132
223
|
# tools listening on the same channel. Hard rule: bulk capture failures
|
|
133
224
|
# never propagate.
|
|
134
225
|
def handle_notification(_event_name, started, finished, payload)
|
|
135
|
-
|
|
136
|
-
|
|
226
|
+
# Hot path: this runs once per SQL statement in the host app —
|
|
227
|
+
# often thousands of times per second on a busy tenant. EVERY
|
|
228
|
+
# branch above the cheapest filter has to be fast. Order is:
|
|
229
|
+
#
|
|
230
|
+
# 1. Cheapest possible name check (string suffix, no regex,
|
|
231
|
+
# no method dispatch on the configuration object). This
|
|
232
|
+
# rejects ~99% of notifications in <1 µs.
|
|
233
|
+
# 2. Then capture_enabled? (config check).
|
|
234
|
+
# 3. Then everything else.
|
|
235
|
+
# Cheapest possible early-exit: a single instance-variable
|
|
236
|
+
# read. This branch fires on EVERY SQL statement in the host
|
|
237
|
+
# app and must not allocate or call into configuration.
|
|
238
|
+
return unless @capture_enabled
|
|
239
|
+
return unless payload.is_a?(Hash)
|
|
240
|
+
name = payload[:name]
|
|
241
|
+
return unless name && name_eligible?(name)
|
|
137
242
|
|
|
138
243
|
operation = ::EzLogsAgent::BulkSqlParser.detect_operation(payload[:sql])
|
|
139
244
|
return unless operation
|
|
@@ -147,6 +252,27 @@ module EzLogsAgent
|
|
|
147
252
|
type_casted_binds: payload[:type_casted_binds]
|
|
148
253
|
)
|
|
149
254
|
|
|
255
|
+
# Drop Rails framework rewrites that aren't real business activity:
|
|
256
|
+
#
|
|
257
|
+
# * Foreign-key nullification — when a parent has `dependent:
|
|
258
|
+
# :restrict_with_error` AND the child's `belongs_to :parent`
|
|
259
|
+
# is `optional: true`, Rails rewrites `child.delete_all`
|
|
260
|
+
# into `UPDATE children SET parent_id = NULL WHERE ...`
|
|
261
|
+
# before the destroy. There is no business meaning here;
|
|
262
|
+
# it's framework cleanup before the parent goes away.
|
|
263
|
+
#
|
|
264
|
+
# * Counter-cache / increment! — `Model.increment_counter(:x)`
|
|
265
|
+
# and `record.increment!(:x)` compile to
|
|
266
|
+
# `UPDATE ... SET x = COALESCE(x, 0) + N WHERE id = ?`,
|
|
267
|
+
# which is Rails plumbing for a numeric counter bump, not
|
|
268
|
+
# a user-visible change.
|
|
269
|
+
#
|
|
270
|
+
# Both produce high-volume noise on the timeline (see EZLogs's
|
|
271
|
+
# own dogfood where Company#increment_actions_count! fires
|
|
272
|
+
# per ingest batch). Filtering at the capturer means they
|
|
273
|
+
# don't ride the wire at all.
|
|
274
|
+
return if framework_rewrite?(operation, parse_result)
|
|
275
|
+
|
|
150
276
|
source_data = build_source_data(
|
|
151
277
|
operation: operation,
|
|
152
278
|
model_class: model_class,
|
|
@@ -183,11 +309,21 @@ module EzLogsAgent
|
|
|
183
309
|
# @return [Boolean]
|
|
184
310
|
def eligible_payload?(payload)
|
|
185
311
|
return false unless payload.is_a?(Hash)
|
|
312
|
+
name = payload[:name]
|
|
313
|
+
name.is_a?(String) && name_eligible?(name)
|
|
314
|
+
end
|
|
186
315
|
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
316
|
+
# Cheapest possible bulk-name check — string `end_with?` calls,
|
|
317
|
+
# no regex compilation, no method dispatch. Runs on the hot path.
|
|
318
|
+
# The four shapes we care about (per AR convention):
|
|
319
|
+
# "<Model> Delete All", "<Model> Update All",
|
|
320
|
+
# "<Model> Insert", "<Model> Upsert"
|
|
321
|
+
# (Older Rails 7 also used " Bulk Insert" / " Bulk Upsert" — those
|
|
322
|
+
# still end with "Insert"/"Upsert" so end_with? catches them.)
|
|
323
|
+
# Per-row CRUD names like "<Model> Create", "<Model> Update",
|
|
324
|
+
# "<Model> Destroy" fail all four suffix checks and bail in <1 µs.
|
|
325
|
+
def name_eligible?(name)
|
|
326
|
+
name.end_with?(" All") || name.end_with?(" Insert") || name.end_with?(" Upsert")
|
|
191
327
|
end
|
|
192
328
|
|
|
193
329
|
# Looks up the model class from the SQL's table name. Returns nil
|
|
@@ -200,9 +336,26 @@ module EzLogsAgent
|
|
|
200
336
|
table = extract_table_name(sql)
|
|
201
337
|
return nil if table.nil?
|
|
202
338
|
|
|
203
|
-
|
|
339
|
+
# Try the descendants list first — cheap and works in environments
|
|
340
|
+
# where models are already eager-loaded (production, Sidekiq workers).
|
|
341
|
+
loaded = ::ActiveRecord::Base.descendants.find do |klass|
|
|
204
342
|
klass.respond_to?(:table_name) && klass.table_name == table && !klass.abstract_class?
|
|
205
343
|
end
|
|
344
|
+
return loaded if loaded
|
|
345
|
+
|
|
346
|
+
# Fallback for development mode and any lazy-autoload path: derive
|
|
347
|
+
# the model class name from the table name via Rails' inflector
|
|
348
|
+
# and try to safe_constantize it. ActiveRecord's reflection class
|
|
349
|
+
# caches it on first call, so subsequent bulk ops on the same
|
|
350
|
+
# table hit the descendants path above.
|
|
351
|
+
constant_name = table.to_s.classify
|
|
352
|
+
klass = constant_name.safe_constantize
|
|
353
|
+
return klass if klass.is_a?(Class) &&
|
|
354
|
+
klass < ::ActiveRecord::Base &&
|
|
355
|
+
klass.respond_to?(:table_name) &&
|
|
356
|
+
klass.table_name == table
|
|
357
|
+
|
|
358
|
+
nil
|
|
206
359
|
rescue StandardError
|
|
207
360
|
nil
|
|
208
361
|
end
|
|
@@ -310,11 +463,12 @@ module EzLogsAgent
|
|
|
310
463
|
end
|
|
311
464
|
|
|
312
465
|
# Builds the sentinel resource entry. row_count may be nil (Rails
|
|
313
|
-
# < 7 didn't ship it; some adapters still don't)
|
|
314
|
-
#
|
|
315
|
-
#
|
|
466
|
+
# < 7 didn't ship it; some adapters still don't), and 0 is also
|
|
467
|
+
# not informative (PG's update_all returns 0 in some paths) —
|
|
468
|
+
# fall back to "many" so the display reads naturally and the
|
|
469
|
+
# server-side ResourceAggregationStage doesn't drop it.
|
|
316
470
|
def build_resource_ids(model_class, row_count)
|
|
317
|
-
count_str = row_count.is_a?(Integer) ? row_count.to_s : "
|
|
471
|
+
count_str = (row_count.is_a?(Integer) && row_count > 0) ? row_count.to_s : "many"
|
|
318
472
|
[{ resource_type: model_class.name, resource_id: "bulk:#{count_str}" }]
|
|
319
473
|
end
|
|
320
474
|
|
|
@@ -327,15 +481,71 @@ module EzLogsAgent
|
|
|
327
481
|
end
|
|
328
482
|
|
|
329
483
|
# Uses DatabaseCapturer's existing all_excluded_tables list — one
|
|
330
|
-
# config knob, both capturers obey it.
|
|
484
|
+
# config knob, both capturers obey it. The list is memoized at
|
|
485
|
+
# install time (`@excluded_tables`) so we don't pay a Hash
|
|
486
|
+
# method dispatch on every captured event. Customers who change
|
|
487
|
+
# config at runtime need to call uninstall! + install — the same
|
|
488
|
+
# constraint that already applies to `capture_database`.
|
|
331
489
|
def table_excluded?(model_class)
|
|
332
490
|
return false unless model_class.respond_to?(:table_name)
|
|
333
491
|
|
|
334
|
-
|
|
492
|
+
@excluded_tables.include?(model_class.table_name)
|
|
335
493
|
rescue StandardError
|
|
336
494
|
false
|
|
337
495
|
end
|
|
338
496
|
|
|
497
|
+
# Detects Rails-generated update_all SQL that we can't render
|
|
498
|
+
# meaningfully OR that is pure framework noise. We are deliberately
|
|
499
|
+
# narrow here — anything that COULD be a real change a customer
|
|
500
|
+
# cares about (even SET col = NULL on N rows) we keep.
|
|
501
|
+
#
|
|
502
|
+
# Filtered shapes:
|
|
503
|
+
#
|
|
504
|
+
# 1. Empty SET hash — the parser couldn't extract any column →
|
|
505
|
+
# value pairs (e.g. unquoted column names in raw SQL). The
|
|
506
|
+
# captured event would render with no "Set X to Y" detail
|
|
507
|
+
# and no humanized filter, just "Updated 26 things" with an
|
|
508
|
+
# empty Operation block. That's misleading.
|
|
509
|
+
#
|
|
510
|
+
# 2. Counter cache / increment! / decrement! — any SET value is
|
|
511
|
+
# a `COALESCE(<col>, 0) + N` expression. Rails uses this
|
|
512
|
+
# shape for `increment_counter`, `increment!`,
|
|
513
|
+
# `update_counters`. The semantics are "bump a number by N",
|
|
514
|
+
# which is plumbing-grade noise: high volume (per-request
|
|
515
|
+
# counter bumps fire dozens of times per minute on a busy
|
|
516
|
+
# tenant) and zero business meaning to a non-technical
|
|
517
|
+
# reader. On the EZLogs server alone these would dominate
|
|
518
|
+
# the timeline.
|
|
519
|
+
#
|
|
520
|
+
# SET <fk> = NULL on N rows IS captured — even though Rails sometimes
|
|
521
|
+
# generates it implicitly as cleanup before a destroy, it's also
|
|
522
|
+
# the shape of a real customer-issued nullification (soft-orphaning,
|
|
523
|
+
# disassociating tags from an item, etc.), and from the SQL alone
|
|
524
|
+
# we can't tell the two apart. Showing it honestly is the right call:
|
|
525
|
+
# the reader sees that N rows had their column X set to NULL.
|
|
526
|
+
#
|
|
527
|
+
# @return [Boolean]
|
|
528
|
+
def framework_rewrite?(operation, parse_result)
|
|
529
|
+
return false unless operation == :update_all
|
|
530
|
+
return false unless parse_result.is_a?(Hash)
|
|
531
|
+
|
|
532
|
+
set = parse_result[:set]
|
|
533
|
+
|
|
534
|
+
# Empty SET hash: parser bailed. Drop — would render as an
|
|
535
|
+
# empty Operation block.
|
|
536
|
+
return true if set.is_a?(Hash) && set.empty? && !parse_result[:unparseable]
|
|
537
|
+
|
|
538
|
+
return false unless set.is_a?(Hash) && set.any?
|
|
539
|
+
|
|
540
|
+
# Counter-cache / increment! — any value contains COALESCE().
|
|
541
|
+
# Distinct from a deliberate UPDATE because the SET expression
|
|
542
|
+
# references the column on its own RHS, which no business
|
|
543
|
+
# update_all ever does.
|
|
544
|
+
return true if set.values.any? { |v| v.to_s.include?("COALESCE(") }
|
|
545
|
+
|
|
546
|
+
false
|
|
547
|
+
end
|
|
548
|
+
|
|
339
549
|
# Same formatter as DatabaseCapturer. Keeps Date / Time / BigDecimal
|
|
340
550
|
# from collapsing to "[Object]" when they reach Sanitizer / wire.
|
|
341
551
|
def format_value_for_json(value)
|
|
@@ -78,6 +78,33 @@ module EzLogsAgent
|
|
|
78
78
|
return unless defined?(ActiveRecord::Base)
|
|
79
79
|
return if @installed
|
|
80
80
|
|
|
81
|
+
# Memoize config values that the per-row hot path reads on
|
|
82
|
+
# every callback. Without this, each captured create / update
|
|
83
|
+
# / destroy pays a method dispatch into the configuration
|
|
84
|
+
# object (`EzLogsAgent.configuration.capture_database`) plus
|
|
85
|
+
# an `all_excluded_tables.include?` Hash dispatch. On a request
|
|
86
|
+
# that touches dozens of rows the overhead is real.
|
|
87
|
+
# Runtime mutations require uninstall! + install to take effect
|
|
88
|
+
# (acceptable — nobody flips capture_database at runtime).
|
|
89
|
+
@capture_enabled =
|
|
90
|
+
begin
|
|
91
|
+
EzLogsAgent.configuration.capture_database
|
|
92
|
+
rescue StandardError
|
|
93
|
+
false
|
|
94
|
+
end
|
|
95
|
+
@excluded_tables =
|
|
96
|
+
begin
|
|
97
|
+
EzLogsAgent.configuration.all_excluded_tables.dup.freeze
|
|
98
|
+
rescue StandardError
|
|
99
|
+
[].freeze
|
|
100
|
+
end
|
|
101
|
+
@display_name_for =
|
|
102
|
+
begin
|
|
103
|
+
(EzLogsAgent.configuration.display_name_for || {}).dup.freeze
|
|
104
|
+
rescue StandardError
|
|
105
|
+
{}.freeze
|
|
106
|
+
end
|
|
107
|
+
|
|
81
108
|
# Only register callbacks once per Ruby process
|
|
82
109
|
unless @callbacks_registered
|
|
83
110
|
ActiveRecord::Base.class_eval do
|
|
@@ -137,24 +164,24 @@ module EzLogsAgent
|
|
|
137
164
|
|
|
138
165
|
private
|
|
139
166
|
|
|
140
|
-
# Checks if database capture is enabled
|
|
167
|
+
# Checks if database capture is enabled. Reads the memoized
|
|
168
|
+
# value set at install time (no config-object dispatch on the
|
|
169
|
+
# hot path).
|
|
141
170
|
#
|
|
142
171
|
# @return [Boolean]
|
|
143
172
|
def capture_enabled?
|
|
144
|
-
|
|
145
|
-
rescue StandardError
|
|
146
|
-
false
|
|
173
|
+
@capture_enabled
|
|
147
174
|
end
|
|
148
175
|
|
|
149
|
-
# Checks if the model's table is in the excluded_tables list
|
|
150
|
-
#
|
|
176
|
+
# Checks if the model's table is in the excluded_tables list.
|
|
177
|
+
# Reads the memoized list set at install time.
|
|
151
178
|
#
|
|
152
179
|
# @param model [ActiveRecord::Base] The model instance
|
|
153
180
|
# @return [Boolean]
|
|
154
181
|
def table_excluded?(model)
|
|
155
182
|
return false unless model.class.respond_to?(:table_name)
|
|
156
183
|
|
|
157
|
-
|
|
184
|
+
@excluded_tables.include?(model.class.table_name)
|
|
158
185
|
rescue StandardError
|
|
159
186
|
false
|
|
160
187
|
end
|
|
@@ -219,9 +246,8 @@ module EzLogsAgent
|
|
|
219
246
|
# @param model [ActiveRecord::Base] The model instance
|
|
220
247
|
# @return [String, nil] The display name, or nil if no meaningful name found
|
|
221
248
|
def resolve_display_name(model)
|
|
222
|
-
# Check for configured custom field
|
|
223
|
-
|
|
224
|
-
custom_field = display_name_config[model.class.name]
|
|
249
|
+
# Check for configured custom field (memoized list, see install).
|
|
250
|
+
custom_field = @display_name_for[model.class.name]
|
|
225
251
|
|
|
226
252
|
if custom_field && model.respond_to?(custom_field)
|
|
227
253
|
value = model.public_send(custom_field)
|
|
@@ -48,17 +48,35 @@ module EzLogsAgent
|
|
|
48
48
|
key_lower = key.to_s.downcase
|
|
49
49
|
return true if PATTERNS.any? { |pattern| key_lower.include?(pattern) }
|
|
50
50
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
# rescue would silently fall back to "no extra patterns" on a
|
|
54
|
-
# config bug and the outer rescue would never fire, which means
|
|
55
|
-
# the broken-config path becomes "leak", not "mask".
|
|
56
|
-
user_patterns = EzLogsAgent.configuration.excluded_graphql_variable_keys || []
|
|
57
|
-
user_patterns.any? { |pattern| key_lower.include?(pattern.to_s.downcase) }
|
|
51
|
+
user_patterns = user_patterns_cache
|
|
52
|
+
user_patterns.any? { |pattern| key_lower.include?(pattern) }
|
|
58
53
|
rescue StandardError
|
|
59
54
|
# Defensive: if configuration access raises, treat as sensitive.
|
|
60
55
|
# Better to over-mask than to leak.
|
|
61
56
|
true
|
|
62
57
|
end
|
|
58
|
+
|
|
59
|
+
# Memoized lookup of user-configured patterns, already lowercased.
|
|
60
|
+
# Called per HTTP param, per DB attribute, per job arg key. Without
|
|
61
|
+
# the cache it would dispatch into EzLogsAgent.configuration on
|
|
62
|
+
# every check.
|
|
63
|
+
#
|
|
64
|
+
# The cache is keyed by `object_id` of the configured array, so
|
|
65
|
+
# `EzLogsAgent.configure { |c| c.excluded_graphql_variable_keys = [...] }`
|
|
66
|
+
# invalidates it naturally (assigning a new array changes its id).
|
|
67
|
+
# No explicit invalidation needed.
|
|
68
|
+
def user_patterns_cache
|
|
69
|
+
configured = EzLogsAgent.configuration.excluded_graphql_variable_keys
|
|
70
|
+
return @cached_user_patterns if configured.nil? && @cached_source_id.nil?
|
|
71
|
+
|
|
72
|
+
source_id = configured&.object_id
|
|
73
|
+
if source_id != @cached_source_id
|
|
74
|
+
@cached_user_patterns =
|
|
75
|
+
(configured || []).map { |p| p.to_s.downcase }.freeze
|
|
76
|
+
@cached_source_id = source_id
|
|
77
|
+
end
|
|
78
|
+
@cached_user_patterns
|
|
79
|
+
end
|
|
80
|
+
private_class_method :user_patterns_cache
|
|
63
81
|
end
|
|
64
82
|
end
|