paper_trail-bulk_writes 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +10 -0
- data/LICENSE.txt +21 -0
- data/README.md +140 -0
- data/Rakefile +10 -0
- data/config/default.yml +7 -0
- data/lib/paper_trail/bulk_writes/adapter.rb +93 -0
- data/lib/paper_trail/bulk_writes/configuration.rb +15 -0
- data/lib/paper_trail/bulk_writes/delete.rb +53 -0
- data/lib/paper_trail/bulk_writes/insert.rb +56 -0
- data/lib/paper_trail/bulk_writes/model.rb +25 -0
- data/lib/paper_trail/bulk_writes/row_source.rb +39 -0
- data/lib/paper_trail/bulk_writes/rubocop.rb +24 -0
- data/lib/paper_trail/bulk_writes/scoped.rb +19 -0
- data/lib/paper_trail/bulk_writes/update.rb +73 -0
- data/lib/paper_trail/bulk_writes/version.rb +7 -0
- data/lib/paper_trail/bulk_writes/versioned.rb +27 -0
- data/lib/paper_trail/bulk_writes.rb +37 -0
- data/lib/rubocop/cop/paper_trail/bulk_writes/no_bulk_write_on_audited_model.rb +87 -0
- metadata +95 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 055ac91334d2baf03ac6070aa705f234296a25eebf790e05e52b7248f2be61b5
|
|
4
|
+
data.tar.gz: f5739caa8f59629cebd71a0dd16c8c9dafb7fb92e4b45051eddfff9e36e587a6
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 20a07a365217a304a6920df25fdd6cb6d55da74f09d0fe79d33ca86adefb5e2aa140d4b51fc96f77f59cfc0d97eea40abfdcad87943ea0f2faa83ef87967f7c2
|
|
7
|
+
data.tar.gz: 349d1dd5cc22cfe873d731243540e8bda174d38cfd8893a5ab19edc7a6ddbb7f960dd039bff032b80bf7c163690fddd0bfca2d0196386acef46034b5e0f28a8a
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 0.1.0
|
|
6
|
+
|
|
7
|
+
- `PaperTrail::BulkWrites::Model` with `audited_update_all`, `audited_delete_all` and `audited_insert_all`.
|
|
8
|
+
- Honours `only:`, `ignore:`, `skip:`, `meta:` and `versions:` from `has_paper_trail`; sets `item_subtype` for STI.
|
|
9
|
+
- `PaperTrail::BulkWrites.configure { |c| c.batch_size = ... }`.
|
|
10
|
+
- `PaperTrail/BulkWrites/NoBulkWriteOnAuditedModel` RuboCop cop, configured with a `Models:` list; flags `update_all`, `delete_all`, `insert_all`, `insert_all!` and `upsert_all`.
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 TODO: Write your name
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Paper Trail Bulk Writes
|
|
2
|
+
|
|
3
|
+
PaperTrail versions for `update_all`, `delete_all` and `insert_all`.
|
|
4
|
+
|
|
5
|
+
## The problem
|
|
6
|
+
|
|
7
|
+
[PaperTrail](https://github.com/paper-trail-gem/paper_trail) records versions from ActiveRecord callbacks. Bulk writes skip callbacks, so they leave no audit trail:
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
Shift.where(id: ids).update_all(status: :filled) # no version row
|
|
11
|
+
Shift.where(id: ids).delete_all # no version row
|
|
12
|
+
Shift.insert_all(rows) # no version row
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
The usual workaround is `find_each(&:update!)`, which is N queries and N version inserts. PaperTrail's maintainers have [declined to add bulk support](https://github.com/paper-trail-gem/paper_trail/issues/1533).
|
|
16
|
+
|
|
17
|
+
## What this gem does
|
|
18
|
+
|
|
19
|
+
```ruby
|
|
20
|
+
Shift.where(id: ids).audited_update_all(attributes: { status: :filled })
|
|
21
|
+
Shift.where(id: ids).audited_delete_all
|
|
22
|
+
Shift.audited_insert_all(rows)
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Each call runs the bulk write and inserts one version row per record, in the same transaction, using two statements per batch instead of 2N. The version rows match what the callback path would have written:
|
|
26
|
+
|
|
27
|
+
- `object_changes` holds `[before, after]` per column, read from the database before the write (or from the loaded records, if you pass an array).
|
|
28
|
+
- `only:`, `ignore:` and `skip:` from `has_paper_trail` are honoured.
|
|
29
|
+
- `meta:` symbols and lambdas are resolved per record.
|
|
30
|
+
- A custom `versions: { class_name: }` is written to instead of `PaperTrail::Version`.
|
|
31
|
+
- `item_subtype` is set for single-table-inheritance models.
|
|
32
|
+
- `whodunnit` defaults to `PaperTrail.request.whodunnit`.
|
|
33
|
+
- `audited_update_all` touches `updated_at` like `update!` does (pass `touch: false` to skip).
|
|
34
|
+
|
|
35
|
+
## Install
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
gem 'paper_trail-bulk_writes'
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
```ruby
|
|
42
|
+
class Shift < ApplicationRecord
|
|
43
|
+
include PaperTrail::BulkWrites::Model
|
|
44
|
+
|
|
45
|
+
has_paper_trail
|
|
46
|
+
end
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Usage
|
|
50
|
+
|
|
51
|
+
```ruby
|
|
52
|
+
# Relation or array of loaded records; returns the number of rows written.
|
|
53
|
+
Shift.where(company_id: company.id).audited_update_all(attributes: { archived: true })
|
|
54
|
+
Shift.where(id: ids).audited_update_all(attributes: { status: :filled }, touch: false, whodunnit: admin.id)
|
|
55
|
+
|
|
56
|
+
# Returns the number of rows deleted.
|
|
57
|
+
Shift.where(id: ids).audited_delete_all
|
|
58
|
+
|
|
59
|
+
# Returns the new ids, in input order. Rows are inserted with insert_all!.
|
|
60
|
+
Shift.audited_insert_all([{ name: 'a', status: 'draft' }, { name: 'b', status: 'draft' }])
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Batches default to 1,000 rows:
|
|
64
|
+
|
|
65
|
+
```ruby
|
|
66
|
+
# config/initializers/paper_trail_bulk_writes.rb
|
|
67
|
+
PaperTrail::BulkWrites.configure do |config|
|
|
68
|
+
config.batch_size = 500
|
|
69
|
+
end
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
### Semantics worth knowing
|
|
73
|
+
|
|
74
|
+
- **Loaded records are written by id**, exactly as `update!` would be, even if they no longer match the scope they were loaded from. An unloaded relation re-applies its predicate per batch, so a row that stopped matching is skipped.
|
|
75
|
+
- **Previous values come from the loaded record** when you pass an array, so if the row changed underneath you the version reports the stale value — the same thing PaperTrail's dirty tracking does.
|
|
76
|
+
- **A touch-only update writes a version** whose `object_changes` is just `updated_at`, like `touch` does. Ignore `updated_at` in `has_paper_trail` (or `PaperTrail.config.has_paper_trail_defaults`) if you don't want that.
|
|
77
|
+
- **Data and versions are one transaction.** If the version insert fails, the data write rolls back, and vice versa.
|
|
78
|
+
|
|
79
|
+
## RuboCop cop
|
|
80
|
+
|
|
81
|
+
The gem ships a cop that flags bare bulk writes on models you list as audited:
|
|
82
|
+
|
|
83
|
+
```yaml
|
|
84
|
+
# .rubocop.yml
|
|
85
|
+
require:
|
|
86
|
+
- paper_trail/bulk_writes/rubocop
|
|
87
|
+
|
|
88
|
+
PaperTrail/BulkWrites/NoBulkWriteOnAuditedModel:
|
|
89
|
+
Models:
|
|
90
|
+
- Shift
|
|
91
|
+
- Timesheet
|
|
92
|
+
- Tasks::Task
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
Shift.where(id: ids).update_all(status: :filled)
|
|
97
|
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `update_all` on Shift skips callbacks, so PaperTrail records no version. ...
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
It catches `update_all`, `delete_all`, `insert_all`, `insert_all!` and `upsert_all` when the call chain starts from a listed constant. It does not see relations held in locals or reached through associations (`scope.update_all`, `company.shifts.delete_all`) — the receiver has to be the model.
|
|
101
|
+
|
|
102
|
+
To keep `Models:` honest, add a spec that fails when a model gains `has_paper_trail` without being listed:
|
|
103
|
+
|
|
104
|
+
```ruby
|
|
105
|
+
RSpec.describe 'audited models are listed for the bulk-write cop' do
|
|
106
|
+
it 'lists every model in app/models with has_paper_trail' do
|
|
107
|
+
config = RuboCop::ConfigLoader.load_file('.rubocop.yml')
|
|
108
|
+
listed = config['PaperTrail/BulkWrites/NoBulkWriteOnAuditedModel']['Models'].sort
|
|
109
|
+
|
|
110
|
+
on_disk = Dir[Rails.root.join('app/models/**/*.rb')].filter_map do |path|
|
|
111
|
+
next unless File.foreach(path).any? { |line| line.match?(/^\s*has_paper_trail\b/) }
|
|
112
|
+
|
|
113
|
+
path.delete_prefix(Rails.root.join('app/models/').to_s).delete_suffix('.rb').camelize
|
|
114
|
+
end.sort
|
|
115
|
+
|
|
116
|
+
expect(listed).to eq(on_disk)
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
## Limitations
|
|
122
|
+
|
|
123
|
+
- **Postgres and SQLite (3.35+) only.** `audited_insert_all` relies on `INSERT ... RETURNING`. MySQL is untested.
|
|
124
|
+
- **`object` is not written**, only `object_changes`. `version.reify` won't work on bulk-written versions; use `where_object_changes` and friends.
|
|
125
|
+
- **`has_paper_trail on:`** is not consulted — every bulk write is versioned.
|
|
126
|
+
- Requires Ruby 3.2+, ActiveRecord 7.1+, PaperTrail 15+.
|
|
127
|
+
|
|
128
|
+
## Development
|
|
129
|
+
|
|
130
|
+
```sh
|
|
131
|
+
bundle install
|
|
132
|
+
bundle exec rspec # SQLite in memory
|
|
133
|
+
DB=postgres DATABASE_URL=postgres://localhost/paper_trail_bulk_writes_test bundle exec rspec
|
|
134
|
+
bundle exec appraisal install && bundle exec appraisal rspec # every Rails × PaperTrail combination
|
|
135
|
+
bundle exec rubocop
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## License
|
|
139
|
+
|
|
140
|
+
MIT. Extracted from production code at [Florence](https://www.florence.co.uk).
|
data/Rakefile
ADDED
data/config/default.yml
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
class Adapter
|
|
6
|
+
def initialize(model, whodunnit:)
|
|
7
|
+
@model = model
|
|
8
|
+
@whodunnit = whodunnit
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
def tracked(candidates)
|
|
12
|
+
tracked = candidates.map(&:to_s) - ignored
|
|
13
|
+
only.any? ? tracked & only : tracked
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def meta
|
|
17
|
+
@meta ||= versioned? ? model.paper_trail_options[:meta].to_h : {}
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def meta_for(record)
|
|
21
|
+
meta.transform_values do |value|
|
|
22
|
+
next value.call(record) if value.respond_to?(:call)
|
|
23
|
+
next record.send(value) if value.is_a?(Symbol)
|
|
24
|
+
|
|
25
|
+
value
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def build_row(id:, event:, object_changes:, extra:, subtype: nil)
|
|
30
|
+
row = {
|
|
31
|
+
item_type: model.base_class.name,
|
|
32
|
+
item_id: id,
|
|
33
|
+
event: event,
|
|
34
|
+
whodunnit: whodunnit,
|
|
35
|
+
object_changes: serialize(object_changes),
|
|
36
|
+
created_at: Time.current
|
|
37
|
+
}
|
|
38
|
+
row[:item_subtype] = subtype || default_subtype if subtype_column?
|
|
39
|
+
row.merge(extra)
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def insert(rows)
|
|
43
|
+
return unless versioned? && rows.any?
|
|
44
|
+
|
|
45
|
+
version_class.insert_all(rows)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
private
|
|
49
|
+
|
|
50
|
+
attr_reader :model, :whodunnit
|
|
51
|
+
|
|
52
|
+
def versioned?
|
|
53
|
+
model.respond_to?(:paper_trail_options)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def version_class
|
|
57
|
+
model.reflect_on_association(model.versions_association_name).klass
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def subtype_column?
|
|
61
|
+
versioned? && version_class.column_names.include?('item_subtype')
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def default_subtype
|
|
65
|
+
model.name unless model == model.base_class
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def serialize(object_changes)
|
|
69
|
+
return object_changes if !versioned? || json_column?
|
|
70
|
+
|
|
71
|
+
PaperTrail.serializer.dump(object_changes)
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def json_column?
|
|
75
|
+
return @json_column if defined?(@json_column)
|
|
76
|
+
|
|
77
|
+
@json_column = %i[json jsonb].include?(version_class.type_for_attribute('object_changes').type)
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
def ignored
|
|
81
|
+
@ignored ||= versioned? ? option_names(:ignore) + option_names(:skip) : []
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def only
|
|
85
|
+
@only ||= versioned? ? option_names(:only) : []
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
def option_names(key)
|
|
89
|
+
Array(model.paper_trail_options[key]).grep_v(Hash).map(&:to_s)
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
end
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
class Delete
|
|
6
|
+
include Versioned
|
|
7
|
+
include Scoped
|
|
8
|
+
|
|
9
|
+
def initialize(scope:, whodunnit: nil, batch_size: nil)
|
|
10
|
+
@scope = scope
|
|
11
|
+
@whodunnit = whodunnit
|
|
12
|
+
@batch_size = batch_size || default_batch_size
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def call
|
|
16
|
+
return 0 if model.nil?
|
|
17
|
+
|
|
18
|
+
written = 0
|
|
19
|
+
rows.each_batch(batch_size) { |batch| written += write(batch) }
|
|
20
|
+
written
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
private
|
|
24
|
+
|
|
25
|
+
attr_reader :scope, :whodunnit, :batch_size
|
|
26
|
+
|
|
27
|
+
def write(batch)
|
|
28
|
+
ids = batch.map(&:first)
|
|
29
|
+
return 0 if ids.empty?
|
|
30
|
+
|
|
31
|
+
model.transaction do
|
|
32
|
+
model.where(id: ids).delete_all
|
|
33
|
+
adapter.insert(batch.map { |id, previous, extra| version_row(id, previous, extra) })
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
ids.size
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def version_row(id, previous, extra)
|
|
40
|
+
adapter.build_row(
|
|
41
|
+
id: id,
|
|
42
|
+
event: 'destroy',
|
|
43
|
+
object_changes: previous.transform_values { |was| [was, nil] },
|
|
44
|
+
extra: extra
|
|
45
|
+
)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def tracked
|
|
49
|
+
@tracked ||= adapter.tracked(model.column_names)
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
class Insert
|
|
6
|
+
include Versioned
|
|
7
|
+
|
|
8
|
+
def initialize(model:, rows:, whodunnit: nil, batch_size: nil)
|
|
9
|
+
@model = model
|
|
10
|
+
@rows = rows
|
|
11
|
+
@whodunnit = whodunnit
|
|
12
|
+
@batch_size = batch_size || default_batch_size
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def call
|
|
16
|
+
return [] if rows.empty?
|
|
17
|
+
|
|
18
|
+
model.transaction { rows.each_slice(batch_size).flat_map { |batch| write(batch) } }
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
private
|
|
22
|
+
|
|
23
|
+
attr_reader :model, :rows, :whodunnit, :batch_size
|
|
24
|
+
|
|
25
|
+
def tracked
|
|
26
|
+
@tracked ||= adapter.tracked(model.column_names) - ['id']
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def write(batch)
|
|
30
|
+
inserted = model.insert_all!(batch, returning: ['id', *tracked]).to_a
|
|
31
|
+
changed = changed_columns(batch)
|
|
32
|
+
adapter.insert(inserted.map { |attributes| version_row(attributes, changed) })
|
|
33
|
+
inserted.pluck('id')
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def changed_columns(batch)
|
|
37
|
+
supplied = batch.flat_map { |row| row.keys.map(&:to_s) }.uniq
|
|
38
|
+
(supplied + %w[id created_at updated_at]) & ['id', *tracked]
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def version_row(attributes, changed)
|
|
42
|
+
adapter.build_row(
|
|
43
|
+
id: attributes['id'],
|
|
44
|
+
event: 'create',
|
|
45
|
+
object_changes: object_changes_for(attributes.slice(*changed)),
|
|
46
|
+
extra: adapter.meta.any? ? adapter.meta_for(model.new(attributes)) : {},
|
|
47
|
+
subtype: attributes[model.inheritance_column]
|
|
48
|
+
)
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def object_changes_for(attributes)
|
|
52
|
+
attributes.compact.to_h { |name, value| [name, [nil, model.type_for_attribute(name).cast(value)]] }
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
module Model
|
|
6
|
+
def self.included(base)
|
|
7
|
+
base.extend(ClassMethods)
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
module ClassMethods
|
|
11
|
+
def audited_insert_all(rows, whodunnit: PaperTrail.request.whodunnit)
|
|
12
|
+
Insert.call(model: self, rows: rows, whodunnit: whodunnit)
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def audited_update_all(attributes:, whodunnit: PaperTrail.request.whodunnit, touch: true)
|
|
16
|
+
Update.call(scope: all, attributes: attributes, whodunnit: whodunnit, touch: touch)
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def audited_delete_all(whodunnit: PaperTrail.request.whodunnit)
|
|
20
|
+
Delete.call(scope: all, whodunnit: whodunnit)
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
class RowSource
|
|
6
|
+
def initialize(scope, tracked:, adapter:)
|
|
7
|
+
@scope = scope
|
|
8
|
+
@tracked = tracked
|
|
9
|
+
@adapter = adapter
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def each_batch(batch_size)
|
|
13
|
+
if scope.is_a?(ActiveRecord::Relation) && !scope.loaded?
|
|
14
|
+
scope.in_batches(of: batch_size) { |relation| yield(rows_for_relation(relation)) }
|
|
15
|
+
else
|
|
16
|
+
scope.to_a.each_slice(batch_size) { |records| yield(loaded_rows(records)) }
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
private
|
|
21
|
+
|
|
22
|
+
attr_reader :scope, :tracked, :adapter
|
|
23
|
+
|
|
24
|
+
def rows_for_relation(relation)
|
|
25
|
+
return loaded_rows(relation.to_a) if adapter.meta.any?
|
|
26
|
+
|
|
27
|
+
relation.pluck(:id, *tracked).map do |id, *values|
|
|
28
|
+
[id, tracked.zip(values).to_h, {}]
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def loaded_rows(records)
|
|
33
|
+
records.map do |record|
|
|
34
|
+
[record.id, tracked.index_with { |name| record.read_attribute(name) }, adapter.meta_for(record)]
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'rubocop'
|
|
4
|
+
require_relative '../../rubocop/cop/paper_trail/bulk_writes/no_bulk_write_on_audited_model'
|
|
5
|
+
|
|
6
|
+
module PaperTrail
|
|
7
|
+
module BulkWrites
|
|
8
|
+
module RuboCop
|
|
9
|
+
CONFIG_DEFAULT = File.expand_path('../../../config/default.yml', __dir__)
|
|
10
|
+
|
|
11
|
+
class Inject
|
|
12
|
+
def self.defaults!
|
|
13
|
+
hash = ::RuboCop::ConfigLoader.send(:load_yaml_configuration, CONFIG_DEFAULT)
|
|
14
|
+
config = ::RuboCop::Config.new(hash, CONFIG_DEFAULT)
|
|
15
|
+
puts "configuration from #{CONFIG_DEFAULT}" if ::RuboCop::ConfigLoader.debug?
|
|
16
|
+
config = ::RuboCop::ConfigLoader.merge_with_default(config, CONFIG_DEFAULT)
|
|
17
|
+
::RuboCop::ConfigLoader.instance_variable_set(:@default_configuration, config)
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
PaperTrail::BulkWrites::RuboCop::Inject.defaults!
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
module Scoped
|
|
6
|
+
private
|
|
7
|
+
|
|
8
|
+
def model
|
|
9
|
+
return @model if defined?(@model)
|
|
10
|
+
|
|
11
|
+
@model = BulkWrites.model_for(scope)
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def rows
|
|
15
|
+
RowSource.new(scope, tracked: tracked, adapter: adapter)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
class Update
|
|
6
|
+
include Versioned
|
|
7
|
+
include Scoped
|
|
8
|
+
|
|
9
|
+
def initialize(scope:, attributes:, whodunnit: nil, batch_size: nil, touch: true)
|
|
10
|
+
@scope = scope
|
|
11
|
+
@attributes = attributes.stringify_keys
|
|
12
|
+
@whodunnit = whodunnit
|
|
13
|
+
@batch_size = batch_size || default_batch_size
|
|
14
|
+
@touch = touch
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def call
|
|
18
|
+
return 0 if model.nil?
|
|
19
|
+
|
|
20
|
+
written = 0
|
|
21
|
+
rows.each_batch(batch_size) { |batch| written += write(batch) }
|
|
22
|
+
written
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
private
|
|
26
|
+
|
|
27
|
+
attr_reader :scope, :attributes, :whodunnit, :batch_size, :touch
|
|
28
|
+
|
|
29
|
+
def write(batch)
|
|
30
|
+
ids = batch.map(&:first)
|
|
31
|
+
return 0 if ids.empty?
|
|
32
|
+
|
|
33
|
+
model.transaction do
|
|
34
|
+
model.where(id: ids).update_all(write_attributes)
|
|
35
|
+
adapter.insert(batch.filter_map { |id, previous, extra| version_row(id, previous, extra) })
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
ids.size
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def write_attributes
|
|
42
|
+
@write_attributes ||= if touch
|
|
43
|
+
attributes.merge('updated_at' => model.current_time_from_proper_timezone)
|
|
44
|
+
else
|
|
45
|
+
attributes
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def version_row(id, previous, extra)
|
|
50
|
+
changes = object_changes_for(previous)
|
|
51
|
+
return if changes.empty?
|
|
52
|
+
|
|
53
|
+
adapter.build_row(id: id, event: 'update', object_changes: changes, extra: extra)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def tracked
|
|
57
|
+
@tracked ||= adapter.tracked(write_attributes.keys)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def new_values
|
|
61
|
+
@new_values ||= tracked.index_with { |name| model.type_for_attribute(name).cast(write_attributes[name]) }
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def object_changes_for(previous)
|
|
65
|
+
tracked.each_with_object({}) do |name, changes|
|
|
66
|
+
was = previous[name]
|
|
67
|
+
now = new_values[name]
|
|
68
|
+
changes[name] = [was, now] unless was == now
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module PaperTrail
|
|
4
|
+
module BulkWrites
|
|
5
|
+
module Versioned
|
|
6
|
+
def self.included(base)
|
|
7
|
+
base.extend(ClassMethods)
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
module ClassMethods
|
|
11
|
+
def call(...)
|
|
12
|
+
new(...).call
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
private
|
|
17
|
+
|
|
18
|
+
def adapter
|
|
19
|
+
@adapter ||= Adapter.new(model, whodunnit: whodunnit)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def default_batch_size
|
|
23
|
+
BulkWrites.configuration.batch_size
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'active_record'
|
|
4
|
+
require 'paper_trail'
|
|
5
|
+
|
|
6
|
+
require_relative 'bulk_writes/version'
|
|
7
|
+
require_relative 'bulk_writes/configuration'
|
|
8
|
+
require_relative 'bulk_writes/adapter'
|
|
9
|
+
require_relative 'bulk_writes/row_source'
|
|
10
|
+
require_relative 'bulk_writes/versioned'
|
|
11
|
+
require_relative 'bulk_writes/scoped'
|
|
12
|
+
require_relative 'bulk_writes/insert'
|
|
13
|
+
require_relative 'bulk_writes/update'
|
|
14
|
+
require_relative 'bulk_writes/delete'
|
|
15
|
+
require_relative 'bulk_writes/model'
|
|
16
|
+
|
|
17
|
+
module PaperTrail
|
|
18
|
+
module BulkWrites
|
|
19
|
+
class << self
|
|
20
|
+
def configuration
|
|
21
|
+
@configuration ||= Configuration.new
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def configure
|
|
25
|
+
yield configuration
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def reset_configuration!
|
|
29
|
+
@configuration = Configuration.new
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def model_for(scope)
|
|
33
|
+
scope.is_a?(ActiveRecord::Relation) ? scope.klass : scope.first&.class
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RuboCop
|
|
4
|
+
module Cop
|
|
5
|
+
module PaperTrail
|
|
6
|
+
module BulkWrites
|
|
7
|
+
# Flags update_all / delete_all / insert_all / upsert_all on a model listed in `Models`.
|
|
8
|
+
# Those writes skip callbacks, so PaperTrail records nothing. Use the audited_* helpers
|
|
9
|
+
# from PaperTrail::BulkWrites::Model instead.
|
|
10
|
+
#
|
|
11
|
+
# @example
|
|
12
|
+
# # bad
|
|
13
|
+
# Shift.where(id: ids).update_all(status: :filled)
|
|
14
|
+
# Shift.where(id: ids).delete_all
|
|
15
|
+
# Shift.insert_all(rows)
|
|
16
|
+
#
|
|
17
|
+
# # good
|
|
18
|
+
# Shift.where(id: ids).audited_update_all(attributes: { status: :filled })
|
|
19
|
+
# Shift.where(id: ids).audited_delete_all
|
|
20
|
+
# Shift.audited_insert_all(rows)
|
|
21
|
+
class NoBulkWriteOnAuditedModel < Base
|
|
22
|
+
MSG = '`%<method>s` on %<model>s skips callbacks, so PaperTrail records no version. ' \
|
|
23
|
+
'Use `%<replacement>s` (include `PaperTrail::BulkWrites::Model` on the model), ' \
|
|
24
|
+
'or `%<callback_alternative>s` when you want the record callbacks to run too.'
|
|
25
|
+
|
|
26
|
+
REPLACEMENTS = {
|
|
27
|
+
update_all: 'audited_update_all',
|
|
28
|
+
delete_all: 'audited_delete_all',
|
|
29
|
+
insert_all: 'audited_insert_all',
|
|
30
|
+
insert_all!: 'audited_insert_all',
|
|
31
|
+
upsert_all: 'audited_insert_all'
|
|
32
|
+
}.freeze
|
|
33
|
+
|
|
34
|
+
CALLBACK_ALTERNATIVES = {
|
|
35
|
+
update_all: 'each(&:update!)',
|
|
36
|
+
delete_all: 'destroy_all',
|
|
37
|
+
insert_all: 'create!',
|
|
38
|
+
insert_all!: 'create!',
|
|
39
|
+
upsert_all: 'create!'
|
|
40
|
+
}.freeze
|
|
41
|
+
|
|
42
|
+
RESTRICT_ON_SEND = REPLACEMENTS.keys.freeze
|
|
43
|
+
|
|
44
|
+
def on_send(node)
|
|
45
|
+
model = audited_chain_root(node)
|
|
46
|
+
return unless model
|
|
47
|
+
|
|
48
|
+
add_offense(node, message: message_for(node.method_name, model))
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
private
|
|
52
|
+
|
|
53
|
+
def message_for(method, model)
|
|
54
|
+
format(
|
|
55
|
+
MSG,
|
|
56
|
+
method: method,
|
|
57
|
+
model: model,
|
|
58
|
+
replacement: REPLACEMENTS.fetch(method),
|
|
59
|
+
callback_alternative: CALLBACK_ALTERNATIVES.fetch(method)
|
|
60
|
+
)
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def audited_chain_root(node)
|
|
64
|
+
receiver = node.receiver
|
|
65
|
+
|
|
66
|
+
while receiver
|
|
67
|
+
return unless receiver.send_type? || receiver.const_type?
|
|
68
|
+
|
|
69
|
+
break if receiver.const_type?
|
|
70
|
+
|
|
71
|
+
receiver = receiver.receiver
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
return unless receiver
|
|
75
|
+
|
|
76
|
+
name = receiver.source.delete_prefix('::')
|
|
77
|
+
name if audited_models.include?(name)
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
def audited_models
|
|
81
|
+
@audited_models ||= Array(cop_config['Models']).map(&:to_s)
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: paper_trail-bulk_writes
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Sam Robinson
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: activerecord
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '7.1'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '7.1'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: paper_trail
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - ">="
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '15'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - ">="
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '15'
|
|
40
|
+
description: |
|
|
41
|
+
Bulk ActiveRecord writes skip callbacks, so PaperTrail records nothing. This gem adds
|
|
42
|
+
audited_update_all, audited_delete_all and audited_insert_all, which run the bulk write
|
|
43
|
+
and insert one version row per record in the same transaction, honouring the model's
|
|
44
|
+
only:, ignore:, skip:, meta: and versions: options. Ships a RuboCop cop that flags
|
|
45
|
+
bare bulk writes on audited models.
|
|
46
|
+
email:
|
|
47
|
+
- samrobinsonwork@outlook.com
|
|
48
|
+
executables: []
|
|
49
|
+
extensions: []
|
|
50
|
+
extra_rdoc_files: []
|
|
51
|
+
files:
|
|
52
|
+
- CHANGELOG.md
|
|
53
|
+
- LICENSE.txt
|
|
54
|
+
- README.md
|
|
55
|
+
- Rakefile
|
|
56
|
+
- config/default.yml
|
|
57
|
+
- lib/paper_trail/bulk_writes.rb
|
|
58
|
+
- lib/paper_trail/bulk_writes/adapter.rb
|
|
59
|
+
- lib/paper_trail/bulk_writes/configuration.rb
|
|
60
|
+
- lib/paper_trail/bulk_writes/delete.rb
|
|
61
|
+
- lib/paper_trail/bulk_writes/insert.rb
|
|
62
|
+
- lib/paper_trail/bulk_writes/model.rb
|
|
63
|
+
- lib/paper_trail/bulk_writes/row_source.rb
|
|
64
|
+
- lib/paper_trail/bulk_writes/rubocop.rb
|
|
65
|
+
- lib/paper_trail/bulk_writes/scoped.rb
|
|
66
|
+
- lib/paper_trail/bulk_writes/update.rb
|
|
67
|
+
- lib/paper_trail/bulk_writes/version.rb
|
|
68
|
+
- lib/paper_trail/bulk_writes/versioned.rb
|
|
69
|
+
- lib/rubocop/cop/paper_trail/bulk_writes/no_bulk_write_on_audited_model.rb
|
|
70
|
+
homepage: https://github.com/Sam-Robin/paper_trail-bulk_writes
|
|
71
|
+
licenses:
|
|
72
|
+
- MIT
|
|
73
|
+
metadata:
|
|
74
|
+
homepage_uri: https://github.com/Sam-Robin/paper_trail-bulk_writes
|
|
75
|
+
source_code_uri: https://github.com/Sam-Robin/paper_trail-bulk_writes
|
|
76
|
+
changelog_uri: https://github.com/Sam-Robin/paper_trail-bulk_writes/blob/main/CHANGELOG.md
|
|
77
|
+
rubygems_mfa_required: 'true'
|
|
78
|
+
rdoc_options: []
|
|
79
|
+
require_paths:
|
|
80
|
+
- lib
|
|
81
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
82
|
+
requirements:
|
|
83
|
+
- - ">="
|
|
84
|
+
- !ruby/object:Gem::Version
|
|
85
|
+
version: 3.2.0
|
|
86
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
87
|
+
requirements:
|
|
88
|
+
- - ">="
|
|
89
|
+
- !ruby/object:Gem::Version
|
|
90
|
+
version: '0'
|
|
91
|
+
requirements: []
|
|
92
|
+
rubygems_version: 4.0.16
|
|
93
|
+
specification_version: 4
|
|
94
|
+
summary: PaperTrail versions for update_all, delete_all and insert_all.
|
|
95
|
+
test_files: []
|