waldit 0.0.25 → 0.0.27

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ba555412a6c079daefae27957b707136b94b086ec4fb351c1243e7efb14470d0
4
- data.tar.gz: 189649ec353443111a932cb10fbe6e3beb59efee8550453bb282da148a1d8858
3
+ metadata.gz: 18bbabfc0b8a6f7386c30773ebb4515f45ea2ff03b3991c28b676eed77c2d001
4
+ data.tar.gz: a69f4a66db8b50305a9b5386183c4626b2a13fff018d1d17d84c95f8e64a2db9
5
5
  SHA512:
6
- metadata.gz: 2839fd6125953177e69c733438fb042a40364b236712f6e2a8d8c0c2490b4917d82be47d265422de4a19cad27c4ce58189fc922d0d28288a476775e85e119a20
7
- data.tar.gz: b49559cc7f60ab7b889fa3e183039214494f40dadc1ca50e00c006e8f62704d7b8ca709719ded76bf6c948835a9705617c7b673988ce97edd1e39fa332206bef
6
+ metadata.gz: 356750f5feefe2a477ed06ac7d90d78ab55fa7e7bafd9e6fa3edb2759fa77e6813e59a15b53f94f081c3aeba94baac2e1b2e2da968d8e6e0b304d0acdb929bfc
7
+ data.tar.gz: d1b1115e71044bce7ad3eb2f9efe3f6e47eb23687966818a4ee8218a8add9c5fbb65a449742d6c988eb86c8d5384447d66d73a35e037122ee9cd8c1cbbb5b3ee
data/README.md CHANGED
@@ -1,117 +1,231 @@
1
1
  # Waldit
2
2
 
3
- [![Gem Version](https://badge.fury.io/rb/waldit.svg)](https://badge.fury.io/rb/waldit)
3
+ Waldit is a Postgres-based audit trail for Rails.
4
4
 
5
- Waldit is a Ruby gem that provides a simple and extensible way to audit changes to your ActiveRecord models. It leverages PostgreSQL's logical replication capabilities to capture changes directly from your database with 100% consistency.
5
+ It hooks into [Postgres logical replication](https://www.postgresql.org/docs/current/logical-replication.html) via the [`wal`](https://github.com/reu/wal) gem to capture every `insert`, `update`, and `delete` directly from the WAL. Unlike ActiveRecord callbacks, these events are guaranteed by Postgres to be 100% consistent -- even changes that bypass Rails entirely are captured.
6
6
 
7
- ## Features
7
+ ## Getting started
8
8
 
9
- - **Automatic Auditing:** Automatically track `create`, `update`, and `delete` operations on your models.
10
- - **Contextual Auditing:** Add custom context to your audit records to understand who made the change and why.
11
- - **Flexible Configuration:** Configure which tables and columns to watch, and how to store audit information.
12
- - **High Performance:** Built on top of [`wal`](https://github.com/reu/wal), which uses PostgreSQL's logical replication for minimal overhead.
9
+ ### Installation
13
10
 
14
- ## Installation
15
-
16
- Add this line to your application's Gemfile:
11
+ Add `waldit` to your application's Gemfile:
17
12
 
18
13
  ```ruby
19
14
  gem "waldit"
20
15
  ```
21
16
 
22
- And then execute:
17
+ ### Database adapter
18
+
19
+ Waldit ships a custom database adapter that injects audit context into your transactions. Update your `config/database.yml`:
20
+
21
+ ```yaml
22
+ default: &default
23
+ adapter: waldit
24
+ # ... rest of your config
25
+ ```
26
+
27
+ ### Migrations
28
+
29
+ Waldit provides migration helpers. First, create the audit table and publication:
30
+
31
+ ```ruby
32
+ class SetupWaldit < ActiveRecord::Migration[7.0]
33
+ def change
34
+ create_waldit_table
35
+ create_waldit_publication
36
+ end
37
+ end
38
+ ```
39
+
40
+ Then, for each table you want to audit:
41
+
42
+ ```ruby
43
+ class AuditUsers < ActiveRecord::Migration[7.0]
44
+ def change
45
+ add_table_to_waldit :users
46
+ end
47
+ end
48
+ ```
49
+
50
+ This sets `REPLICA IDENTITY FULL` on the table and adds it to the Waldit publication.
23
51
 
24
- $ bundle
52
+ ### Running the watcher
25
53
 
26
- Or install it yourself as:
54
+ Create a `config/waldit.yml`:
27
55
 
28
- $ gem install waldit
56
+ ```yaml
57
+ slots:
58
+ audit:
59
+ publications: [waldit_publication]
60
+ watcher: Waldit::Watcher
61
+ ```
29
62
 
30
- ## Usage
63
+ Then start the process:
31
64
 
32
- 1. **Configure your database adapter:**
65
+ ```bash
66
+ bundle exec wal start config/waldit.yml
67
+ ```
33
68
 
34
- First step is to configure in your `config/database.yml` and change your adapter to `waldit`, which is a special adapter that allows injecting `waldit` contextual information on your transactions:
69
+ That's it. Every change to your audited tables is now being recorded.
35
70
 
36
- ```yaml
37
- default: &default
38
- adapter: waldit
39
- # ...
40
- ```
71
+ ## Adding context
41
72
 
42
- 2. **Create an audit table:**
73
+ Wrap your operations with `Waldit.with_context` to record who made the change and why:
43
74
 
44
- Generate a migration to create the `waldit` table:
75
+ ```ruby
76
+ Waldit.with_context(user_id: current_user.id, reason: "Profile update") do
77
+ user.update(name: "New Name")
78
+ end
79
+ ```
45
80
 
46
- ```bash
47
- rails generate migration create_waldit
48
- ```
81
+ Context can be nested and updated mid-transaction:
49
82
 
50
- And then add the following to your migration file:
83
+ ```ruby
84
+ Waldit.with_context(user_id: current_user.id) do
85
+ user.update(name: "New Name")
51
86
 
52
- ```ruby
53
- class CreateWalditTable < ActiveRecord::Migration[7.0]
54
- def change
55
- create_table :waldit do |t|
56
- t.bigint :transaction_id, null: false
57
- t.bigint :lsn, null: false
58
- t.string :action, null: false
59
- t.jsonb :context, default: {}
60
- t.string :table_name, null: false
61
- t.string :primary_key, null: false
62
- t.jsonb :old, default: {}
63
- t.jsonb :new, default: {}
64
- t.timestamp :commited_at
87
+ Waldit.with_context(via: "admin_panel") do
88
+ account.update(plan: "premium") # context: { user_id: 1, via: "admin_panel" }
89
+ end
65
90
 
66
- t.index [:table_name, :primary_key, :transaction_id], unique: true
67
- end
68
- end
69
- end
70
- ```
91
+ Waldit.add_context(batch: true)
92
+ other_user.update(name: "Other") # context: { user_id: 1, batch: true }
93
+ end
94
+ ```
71
95
 
72
- 3. **Configure Waldit:**
96
+ ### Sidekiq integration
73
97
 
74
- Create an initializer file at `config/initializers/waldit.rb`:
98
+ Waldit can propagate context into background jobs:
75
99
 
76
- ```ruby
77
- Waldit.configure do |config|
78
- # A callback that returns true if a table should be watched.
79
- config.watched_tables = ->(table) { table != "waldit" }
100
+ ```ruby
101
+ # config/initializers/sidekiq.rb
102
+ Sidekiq.configure_client do |config|
103
+ config.client_middleware do |chain|
104
+ chain.add Waldit::Sidekiq::SaveContext
105
+ end
106
+ end
107
+
108
+ Sidekiq.configure_server do |config|
109
+ config.server_middleware do |chain|
110
+ chain.add Waldit::Sidekiq::LoadContext
111
+ end
112
+ end
113
+ ```
80
114
 
81
- # A callback that returns an array of columns to ignore for a given table.
82
- config.ignored_columns = ->(table) { %w[created_at updated_at] }
83
- end
84
- ```
115
+ ## Querying the audit trail
85
116
 
86
- 4. **Add context to your changes:**
117
+ Waldit provides scopes on the audit model:
87
118
 
88
- Use the `with_context` method to add context to your database operations:
119
+ ```ruby
120
+ # All audit records for a specific record
121
+ Waldit.model.for(user)
89
122
 
90
- ```ruby
91
- Waldit.with_context(user_id: 1, reason: "User updated their profile") do
92
- user.update(name: "New Name")
93
- end
94
- ```
123
+ # All audit records for a table
124
+ Waldit.model.from_model(User)
95
125
 
96
- 5. **Start the watcher:**
126
+ # All audit records with a specific context
127
+ Waldit.model.with_context(user_id: 1)
128
+ ```
129
+
130
+ Each audit record exposes:
131
+
132
+ ```ruby
133
+ audit = Waldit.model.for(user).last
134
+
135
+ audit.action # "insert", "update", or "delete"
136
+ audit.old # previous attributes (updates and deletes)
137
+ audit.new # new attributes (inserts and updates)
138
+ audit.diff # changed attributes as { "name" => ["old", "new"] }
139
+ audit.context # the context hash
140
+ audit.committed_at # when the transaction was committed
141
+ audit.primary_key # the record's primary key
142
+ ```
143
+
144
+ The `old`, `new`, and `diff` accessors are smart -- if you only store `:diff`, calling `.old` or `.new` will compute the values from the diff, and vice versa.
145
+
146
+ ## Configuration
147
+
148
+ ```ruby
149
+ # config/initializers/waldit.rb
150
+ Waldit.configure do |config|
151
+ # Which tables to watch (default: all except "waldit")
152
+ config.watched_tables = -> table { table != "waldit" }
153
+
154
+ # Columns to exclude from audit records (default: created_at, updated_at)
155
+ config.ignored_columns = -> table { %w[created_at updated_at] }
156
+
157
+ # What to store per table (default: [:old, :new])
158
+ # Options: :old, :new, :diff (any combination)
159
+ config.store_changes = [:old, :new]
160
+
161
+ # WAL byte threshold for switching to streaming mode (default: 10MB)
162
+ # Transactions smaller than this are processed in memory for better performance
163
+ config.large_transaction_threshold = 10_000_000
164
+ end
165
+ ```
166
+
167
+ ### Storage policies
168
+
169
+ By default, Waldit stores both `old` and `new` attributes for every change. You can reduce storage by only keeping what you need:
170
+
171
+ ```ruby
172
+ # Only store diffs for updates (most compact)
173
+ config.store_changes = :diff
174
+
175
+ # Per-table policies
176
+ config.store_changes = -> table {
177
+ case table
178
+ when "events" then [:new]
179
+ when "logs" then [:diff]
180
+ else [:old, :new]
181
+ end
182
+ }
183
+ ```
184
+
185
+ ### Per-table ignored columns
186
+
187
+ ```ruby
188
+ config.ignored_columns = -> table {
189
+ case table
190
+ when "users" then %w[created_at updated_at last_sign_in_at]
191
+ else %w[created_at updated_at]
192
+ end
193
+ }
194
+ ```
195
+
196
+ ### Custom audit model
197
+
198
+ You can provide your own model class if you need custom methods or a different table name:
199
+
200
+ ```ruby
201
+ class AuditRecord < ApplicationRecord
202
+ include Waldit::Record
203
+ self.table_name = "waldit"
204
+ end
205
+
206
+ Waldit.configure do |config|
207
+ config.model = AuditRecord
208
+ end
209
+ ```
97
210
 
98
- To process the events, you need to start a WAL watcher. The recommended way is to have a config/waldit.yml
211
+ ## How it works
99
212
 
100
- ```yml
101
- slots:
102
- audit:
103
- publications: [waldit_publication]
104
- watcher: Waldit::Watcher
105
- ```
213
+ Waldit uses Postgres logical replication to stream changes from the WAL (Write-Ahead Log). The flow is:
106
214
 
107
- And then run:
215
+ 1. The custom database adapter sets a `waldit_context` session variable before each write operation
216
+ 2. Postgres captures the change and the context in the WAL
217
+ 3. `Waldit::Watcher` receives the events via a replication slot
218
+ 4. Events are deduplicated per-transaction (multiple updates to the same record produce a single audit entry)
219
+ 5. The final audit records are persisted to the `waldit` table
108
220
 
109
- ```bash
110
- bundle exec wal start config/waldit.yml
111
- ```
221
+ For small transactions, events are accumulated in memory and persisted in a single batch insert. For large transactions (configurable via `large_transaction_threshold`), events are streamed and persisted individually to avoid memory pressure.
112
222
 
113
- ## How it Works
223
+ ### Transaction-level deduplication
114
224
 
115
- Waldit uses a custom PostgreSQL adapter to set the `waldit_context` session variable before each transaction. This context is then captured by the logical replication slot and stored in the `waldit` table by the `Waldit::Watcher`.
225
+ Within a single database transaction, Waldit collapses events intelligently:
116
226
 
117
- The `Waldit::Watcher` is a streaming watcher that listens for changes in the logical replication slot and creates audit records in the `waldit` table. It processes events in batches to minimize the number of database transactions.
227
+ - **Insert then update** -- recorded as a single `insert` with the final state
228
+ - **Multiple updates** -- recorded as a single `update` with the original `old` and final `new`
229
+ - **Insert then delete** -- not recorded (the record never existed outside the transaction)
230
+ - **Update then delete** -- recorded as a `delete` with the original `old` values
231
+ - **Update that reverts to original** -- not recorded (no net change)
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Waldit
4
- VERSION = "0.0.25"
4
+ VERSION = "0.0.27"
5
5
  end
@@ -6,46 +6,207 @@ module Waldit
6
6
  class Watcher < Wal::StreamingWatcher
7
7
  include Wal
8
8
 
9
- def audit_event(event)
10
- return unless event.primary_key
11
- primary_key = event.primary_key.to_json
9
+ def initialize(*)
10
+ super
11
+ initialize_connection
12
+ @retry = false
13
+ end
12
14
 
13
- audit = [event.transaction_id, event.lsn, event.table, primary_key, event.context.to_json]
15
+ def on_transaction_events(events)
16
+ begin_event = events.next
17
+ # The commit LSN identifies the transaction. The xid from the replication stream is the raw
18
+ # 32-bit counter, which wraps around every ~4 billion transactions, so it cannot be used as
19
+ # a durable identity.
20
+ if begin_event.estimated_size < Waldit.large_transaction_threshold
21
+ process_in_memory(begin_event.final_lsn, events)
22
+ else
23
+ process_streaming(begin_event.final_lsn, events)
24
+ end
25
+ rescue PG::ConnectionBad
26
+ raise if @retry
27
+ initialize_connection
28
+ @retry = true
29
+ retry
30
+ end
14
31
 
15
- case event
16
- when InsertEvent
17
- new_attributes = clean_attributes(event.table, event.new)
18
- @connection.exec_prepared("waldit_insert", audit + [new_attributes.to_json])
19
- true
32
+ def should_watch_table?(table)
33
+ Waldit.watched_tables.call(table)
34
+ end
20
35
 
21
- when UpdateEvent
22
- return if event.diff.without(ignored_columns(event.table)).empty?
23
- old_attributes = clean_attributes(event.table, event.old)
24
- new_attributes = clean_attributes(event.table, event.new)
36
+ def valid_context_prefix?(prefix)
37
+ prefix == Waldit.context_prefix
38
+ end
25
39
 
26
- @connection.exec_prepared("waldit_update", audit + [old_attributes.to_json, new_attributes.to_json])
27
- true
40
+ def ignored_columns(table)
41
+ (@ignored_columns_cache ||= {})[table] ||= Waldit.ignored_columns.call(table)
42
+ end
28
43
 
29
- when DeleteEvent
30
- case @connection.exec_prepared("waldit_delete_cleanup", [event.transaction_id, event.table, primary_key]).values
31
- in [["update", previous_old]]
32
- @connection.exec_prepared("waldit_delete", audit + [previous_old])
33
- in []
34
- @connection.exec_prepared("waldit_delete", audit + [clean_attributes(event.table, event.old).to_json])
35
- else
36
- # Don't need to audit anything on this case
44
+ def store_changes(table)
45
+ (@store_changes_cache ||= {})[table] ||= Waldit.store_changes.call(table)
46
+ end
47
+
48
+ def clean_attributes(table, attributes)
49
+ attributes.without(ignored_columns(table))
50
+ end
51
+
52
+ def record
53
+ Waldit.model
54
+ end
55
+
56
+ private
57
+
58
+ COLUMNS = %w[
59
+ transaction_id
60
+ lsn
61
+ table_name
62
+ primary_key
63
+ action
64
+ context
65
+ committed_at
66
+ old
67
+ new
68
+ diff
69
+ ].freeze
70
+
71
+ PARAMS_PER_ROW = COLUMNS.size
72
+ MAX_ROWS_PER_BATCH = 65535 / PARAMS_PER_ROW
73
+
74
+ def process_in_memory(transaction_id, events)
75
+ records = {}
76
+
77
+ events.each do |event|
78
+ case event
79
+ when InsertEvent
80
+ next unless event.primary_key
81
+ key = [event.full_table_name, event.primary_key.to_json]
82
+ records[key] = event
83
+
84
+ when UpdateEvent
85
+ next unless event.primary_key
86
+ next if event.diff.without(ignored_columns(event.table)).empty?
87
+ key = [event.full_table_name, event.primary_key.to_json]
88
+ records[key] = case (existing_event = records[key])
89
+ when InsertEvent
90
+ # A record inserted on this transaction is being updated, which means it should still reflect as a insert
91
+ # event, we just change the information to reflect the most current data that was just updated.
92
+ existing_event.with(new: event.new)
93
+ when UpdateEvent
94
+ # We are updating again a event that was already updated on this transaction.
95
+ # Same as the insert, we keep the old data from the previous update and the new data from the new one.
96
+ existing_event.with(new: event.new)
97
+ else
98
+ event
99
+ end
100
+
101
+ when DeleteEvent
102
+ next unless event.primary_key
103
+ key = [event.full_table_name, event.primary_key.to_json]
104
+ records[key] = case (existing_event = records[key])
105
+ when InsertEvent
106
+ # We are removing a record that was inserted on this transaction, we should not even report this change, as
107
+ # this record never existed outside this transaction anyways.
108
+ nil
109
+ when UpdateEvent
110
+ # Deleting a record that was previously updated by this transaction. Just store the previous data while
111
+ # keeping the record as deleted.
112
+ event.with(old: existing_event.old)
113
+ else
114
+ event
115
+ end
116
+
117
+ when CommitTransactionEvent
118
+ rows = records.compact.values.filter_map do |evt|
119
+ table = evt.full_table_name
120
+ store = store_changes(table)
121
+
122
+ rec = {
123
+ committed_at: event.timestamp,
124
+ transaction_id:,
125
+ lsn: evt.lsn,
126
+ table_name: table,
127
+ primary_key: evt.primary_key.to_json,
128
+ context: evt.context,
129
+ }
130
+
131
+ case evt
132
+ when InsertEvent
133
+ { **rec, action: "insert", new: clean_attributes(table, evt.new) }
134
+ when UpdateEvent
135
+ rec = {
136
+ **rec,
137
+ action: "update",
138
+ old: evt.old&.then { |attrs| clean_attributes(table, attrs) } || {},
139
+ new: evt.new&.then { |attrs| clean_attributes(table, attrs) } || {},
140
+ }
141
+ next if rec[:old] == rec[:new]
142
+ rec[:old] = nil unless store.include? :old
143
+ rec[:new] = nil unless store.include? :new
144
+ rec[:diff] = clean_attributes(table, evt.diff) if store.include? :diff
145
+ rec
146
+ when DeleteEvent
147
+ { **rec, action: "delete", old: clean_attributes(table, evt.old) }
148
+ end
149
+ end
150
+
151
+ unless rows.empty?
152
+ if rows.size <= MAX_ROWS_PER_BATCH
153
+ insert_batch(rows)
154
+ else
155
+ @connection.transaction do
156
+ rows.each_slice(MAX_ROWS_PER_BATCH) { |batch| insert_batch(batch) }
157
+ end
158
+ end
159
+ end
160
+
161
+ @retry = false
37
162
  end
38
- true
39
163
  end
40
164
  end
41
165
 
42
- def initialize(*)
43
- super
44
- initialize_connection
45
- @retry = false
166
+ def insert_batch(batch)
167
+ rows = batch.each_with_index.map do |_, i|
168
+ o = i * PARAMS_PER_ROW
169
+ row = [
170
+ "$#{o + 1}",
171
+ "$#{o + 2}",
172
+ "$#{o + 3}",
173
+ "$#{o + 4}",
174
+ "$#{o + 5}::waldit_action",
175
+ "$#{o + 6}::jsonb",
176
+ "$#{o + 7}",
177
+ "$#{o + 8}::jsonb",
178
+ "$#{o + 9}::jsonb",
179
+ "$#{o + 10}::jsonb",
180
+ ].join(",")
181
+ "(#{row})"
182
+ end
183
+
184
+ params = batch.flat_map do |r|
185
+ [
186
+ r[:transaction_id],
187
+ r[:lsn],
188
+ r[:table_name],
189
+ r[:primary_key],
190
+ r[:action],
191
+ r[:context]&.to_json,
192
+ r[:committed_at],
193
+ r[:old]&.to_json,
194
+ r[:new]&.to_json,
195
+ r[:diff]&.to_json,
196
+ ]
197
+ end
198
+
199
+ @connection.exec_params(<<~SQL, params)
200
+ INSERT INTO #{record.table_name} (#{COLUMNS.join(",")})
201
+ VALUES #{rows.join(",")}
202
+ ON CONFLICT (table_name, primary_key, transaction_id)
203
+ DO NOTHING
204
+ SQL
46
205
  end
47
206
 
48
- def on_transaction_events(events)
207
+ def process_streaming(transaction_id, events)
208
+ ensure_streaming_statements_prepared
209
+
49
210
  @connection.transaction do
50
211
  tables = Set.new
51
212
 
@@ -54,80 +215,90 @@ module Waldit
54
215
  when CommitTransactionEvent
55
216
  unless tables.empty?
56
217
  changes = [:old, :new, :diff]
57
- .map { |diff| [diff, tables.filter { |table| Waldit.store_changes.call(table).include? diff }] }
218
+ .map { |diff| [diff, tables.filter { |table| store_changes(table).include? diff }] }
58
219
  .to_h
59
220
 
60
- log_new = (changes[:new] || []).map { |table| "#{table}" }
61
- log_old = (changes[:old] || []).map { |table| "#{table}" }
62
- log_diff = (changes[:diff] || []).map { |table| "#{table}" }
63
-
64
221
  @connection.exec_prepared("waldit_finish", [
65
- event.transaction_id,
222
+ transaction_id,
66
223
  event.timestamp,
67
- "{#{log_new.join(",")}}",
68
- "{#{log_old.join(",")}}",
69
- "{#{log_diff.join(",")}}",
224
+ "{#{changes[:new].join(",")}}",
225
+ "{#{changes[:old].join(",")}}",
226
+ "{#{changes[:diff].join(",")}}",
227
+ "{#{tables.join(",")}}",
70
228
  ])
71
229
 
72
230
  @connection.exec_prepared("waldit_cleanup", [
73
- event.transaction_id,
74
- "{#{(log_new + log_old).join(",")}}",
75
- "{#{log_diff.join(",")}}",
231
+ transaction_id,
232
+ "{#{(changes[:new] + changes[:old]).join(",")}}",
233
+ "{#{changes[:diff].join(",")}}",
234
+ "{#{tables.join(",")}}",
76
235
  ])
77
236
  end
78
237
 
79
- # We sucessful retried a connection, let's reset our retry state
80
238
  @retry = false
81
239
 
82
240
  when InsertEvent
83
- tables << event.table if audit_event(event)
241
+ tables << event.table if audit_event(transaction_id, event)
84
242
 
85
243
  when UpdateEvent
86
- tables << event.table if audit_event(event)
244
+ tables << event.table if audit_event(transaction_id, event)
87
245
 
88
246
  when DeleteEvent
89
- tables << event.table if audit_event(event)
247
+ tables << event.table if audit_event(transaction_id, event)
90
248
  end
91
249
  end
92
250
  end
93
- rescue PG::ConnectionBad
94
- raise if @retry
95
- # Let's try to fetch a new connection and reprocess the transaction
96
- initialize_connection
97
- @retry = true
98
- retry
99
251
  end
100
252
 
101
- def should_watch_table?(table)
102
- Waldit.watched_tables.call(table)
103
- end
253
+ def audit_event(transaction_id, event)
254
+ return unless event.primary_key
255
+ primary_key = event.primary_key.to_json
256
+ table = event.full_table_name
104
257
 
105
- def valid_context_prefix?(prefix)
106
- prefix == Waldit.context_prefix
107
- end
258
+ audit = [transaction_id, event.lsn, table, primary_key, event.context.to_json]
108
259
 
109
- def ignored_columns(table)
110
- Waldit.ignored_columns.call(table)
111
- end
260
+ case event
261
+ when InsertEvent
262
+ new_attributes = clean_attributes(table, event.new)
263
+ @connection.exec_prepared("waldit_insert", audit + [new_attributes.to_json])
264
+ true
112
265
 
113
- def clean_attributes(table, attributes)
114
- attributes.without(ignored_columns(table))
115
- end
266
+ when UpdateEvent
267
+ return if event.diff.without(ignored_columns(table)).empty?
268
+ old_attributes = clean_attributes(table, event.old)
269
+ new_attributes = clean_attributes(table, event.new)
116
270
 
117
- def record
118
- Waldit.model
119
- end
271
+ @connection.exec_prepared("waldit_update", audit + [old_attributes.to_json, new_attributes.to_json])
272
+ true
120
273
 
121
- private
274
+ when DeleteEvent
275
+ case @connection.exec_prepared("waldit_delete_cleanup", [transaction_id, table, primary_key]).values
276
+ in [["update", previous_old]]
277
+ @connection.exec_prepared("waldit_delete", audit + [previous_old])
278
+ in []
279
+ @connection.exec_prepared("waldit_delete", audit + [clean_attributes(table, event.old).to_json])
280
+ else
281
+ # Don't need to audit anything on this case
282
+ end
283
+ true
284
+ end
285
+ end
122
286
 
123
287
  def initialize_connection
124
288
  @connection = record.connection_pool.checkout.raw_connection
289
+ @streaming_statements_prepared = false
290
+ end
291
+
292
+ def ensure_streaming_statements_prepared
293
+ return if @streaming_statements_prepared
294
+
125
295
  prepare_insert
126
296
  prepare_update
127
297
  prepare_delete
128
298
  prepare_delete_cleanup
129
299
  prepare_finish
130
300
  prepare_cleanup
301
+ @streaming_statements_prepared = true
131
302
  end
132
303
 
133
304
  def prepare_insert
@@ -193,7 +364,7 @@ module Waldit
193
364
  )
194
365
  ELSE null
195
366
  END
196
- WHERE transaction_id = $1
367
+ WHERE transaction_id = $1 AND table_name = ANY ($6::varchar[])
197
368
  SQL
198
369
  rescue PG::DuplicatePstatement
199
370
  end
@@ -204,6 +375,7 @@ module Waldit
204
375
  WHERE
205
376
  transaction_id = $1
206
377
  AND action = 'update'::waldit_action
378
+ AND table_name = ANY ($4::varchar[])
207
379
  AND (
208
380
  (diff IS NULL AND table_name = ANY ($3::varchar[]))
209
381
  OR
data/lib/waldit.rb CHANGED
@@ -39,6 +39,7 @@ module Waldit
39
39
  attr_accessor :ignored_columns
40
40
  attr_accessor :model
41
41
  attr_accessor :context_prefix
42
+ attr_accessor :large_transaction_threshold
42
43
  end
43
44
 
44
45
  def self.configure(&block)
@@ -54,6 +55,8 @@ module Waldit
54
55
 
55
56
  config.ignored_columns = -> table { %w[created_at updated_at] }
56
57
 
58
+ config.large_transaction_threshold = 10_000_000
59
+
57
60
  config.model = Class.new(ActiveRecord::Base) do
58
61
  include Waldit::Record
59
62
  self.table_name = "waldit"
data/rbi/waldit.rbi CHANGED
@@ -2,7 +2,7 @@
2
2
  module Waldit
3
3
  extend T::Sig
4
4
  extend Waldit::Context
5
- VERSION = "0.0.25"
5
+ VERSION = "0.0.27"
6
6
 
7
7
  class << self
8
8
  sig { returns(String) }
@@ -19,6 +19,9 @@ module Waldit
19
19
 
20
20
  sig { returns(T.class_of(ActiveRecord::Base)) }
21
21
  attr_accessor :model
22
+
23
+ sig { returns(Integer) }
24
+ attr_accessor :large_transaction_threshold
22
25
  end
23
26
 
24
27
  sig { params(tables: T.any(T::Array[String], T.proc.params(table: String).returns(T::Boolean))).void }
@@ -102,8 +105,8 @@ module Waldit
102
105
  class Watcher < Wal::StreamingWatcher
103
106
  extend T::Sig
104
107
 
105
- sig { params(event: T.any(InsertEvent, UpdateEvent, DeleteEvent)).void }
106
- def audit_event(event); end
108
+ sig { params(transaction_id: Integer, event: T.any(InsertEvent, UpdateEvent, DeleteEvent)).void }
109
+ def audit_event(transaction_id, event); end
107
110
 
108
111
  sig { override.params(events: T::Enumerator[Event]).void }
109
112
  def on_transaction_events(events); end
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: waldit
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.25
4
+ version: 0.0.27
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rodrigo Navarro
8
8
  bindir: exe
9
9
  cert_chain: []
10
- date: 2026-02-12 00:00:00.000000000 Z
10
+ date: 2026-07-31 00:00:00.000000000 Z
11
11
  dependencies:
12
12
  - !ruby/object:Gem::Dependency
13
13
  name: wal