paper_trail-bulk_writes 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 055ac91334d2baf03ac6070aa705f234296a25eebf790e05e52b7248f2be61b5
-  data.tar.gz: f5739caa8f59629cebd71a0dd16c8c9dafb7fb92e4b45051eddfff9e36e587a6
+  metadata.gz: 5af6edbca4b17762cc0f647c69ee04b52284cd924b826b6a997a65b4b92925ba
+  data.tar.gz: 2baef624ade4173c84c5b54252ee67930f625ee1d97d5e2ed70788d75891900e
 SHA512:
-  metadata.gz: 20a07a365217a304a6920df25fdd6cb6d55da74f09d0fe79d33ca86adefb5e2aa140d4b51fc96f77f59cfc0d97eea40abfdcad87943ea0f2faa83ef87967f7c2
-  data.tar.gz: 349d1dd5cc22cfe873d731243540e8bda174d38cfd8893a5ab19edc7a6ddbb7f960dd039bff032b80bf7c163690fddd0bfca2d0196386acef46034b5e0f28a8a
+  metadata.gz: 46b8fe04d97334c518edd7344f1a2b2e0a771bb3a6c524611e3fe47000c627dd3cfda33358c88c1889ff9ec654fb8e7d8fd2c8aa732a76959410ca2a0f85669e
+  data.tar.gz: 8b16c7ea07f84084e99f2de9a062e1500e3d018ce024d9d8f48302975a72c9791b3472597d1a51d3b1f72babd7c4187fca3bfb3b11a95c0744ee2aac1da85ccf

data/CHANGELOG.md

@@ -1,6 +1,12 @@
 # Changelog
 
-## Unreleased
+## 0.3.0
+
+- Added: `audited_insert_all` takes `returning:`, a list of column names, and returns an `ActiveRecord::Result` of those columns in input order, like `insert_all`. Without it, it still returns the ids. Works on MySQL too, by reading the rows back.
+
+## 0.2.0
+
+- Fixed: on databases without `INSERT ... RETURNING`, `audited_insert_all` raised `KeyError` when the caller supplied ids typed differently to the column, such as strings out of a CSV or a JSON payload.
 
 ## 0.1.0
 

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+samrobinsonwork@outlook.com.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

data/README.md

@@ -2,6 +2,8 @@
 
 PaperTrail versions for `update_all`, `delete_all` and `insert_all`.
 
+Working on PostgreSQL (full support), SQLite 3.35+ and MySQL 5-7+, MariaDB 10.5+.
+
 ## The problem
 
 [PaperTrail](https://github.com/paper-trail-gem/paper_trail) records versions from ActiveRecord callbacks. Bulk writes skip callbacks, so they leave no audit trail:
@@ -58,6 +60,9 @@ Shift.where(id: ids).audited_delete_all
 
 # Returns the new ids, in input order. Rows are inserted with insert_all!.
 Shift.audited_insert_all([{ name: 'a', status: 'draft' }, { name: 'b', status: 'draft' }])
+
+# Pass returning: for other columns; returns an ActiveRecord::Result, like insert_all.
+Shift.audited_insert_all(rows, returning: %w[id status])
 ```
 
 Batches default to 1,000 rows:
@@ -120,7 +125,6 @@ end
 
 ## Limitations
 
-- **Postgres and SQLite (3.35+) only.** `audited_insert_all` relies on `INSERT ... RETURNING`. MySQL is untested.
 - **`object` is not written**, only `object_changes`. `version.reify` won't work on bulk-written versions; use `where_object_changes` and friends.
 - **`has_paper_trail on:`** is not consulted — every bulk write is versioned.
 - Requires Ruby 3.2+, ActiveRecord 7.1+, PaperTrail 15+.
@@ -130,11 +134,22 @@ end
 ```sh
 bundle install
 bundle exec rspec                                   # SQLite in memory
-DB=postgres DATABASE_URL=postgres://localhost/paper_trail_bulk_writes_test bundle exec rspec
 bundle exec appraisal install && bundle exec appraisal rspec   # every Rails × PaperTrail combination
 bundle exec rubocop
 ```
 
+`pg` and `mysql2` build against client libraries the SQLite suite does not need, so they sit in optional groups. Opt in per driver to run the suite against Postgres or MySQL:
+
+```sh
+bundle config set --local with postgres
+bundle install
+DB=postgres DATABASE_URL=postgres://localhost/paper_trail_bulk_writes_test bundle exec rspec
+
+bundle config set --local with mysql
+bundle install
+DB=mysql DATABASE_URL=mysql2://root:root@127.0.0.1:3306/paper_trail_bulk_writes_test bundle exec rspec
+```
+
 ## License
 
 MIT. Extracted from production code at [Florence](https://www.florence.co.uk).

data/lib/paper_trail/bulk_writes/insert.rb

@@ -5,32 +5,86 @@ module PaperTrail
     class Insert
       include Versioned
 
-      def initialize(model:, rows:, whodunnit: nil, batch_size: nil)
+      def initialize(model:, rows:, whodunnit: nil, batch_size: nil, returning: nil)
         @model = model
         @rows = rows
         @whodunnit = whodunnit
         @batch_size = batch_size || default_batch_size
+        @returning = returning && Array(returning).map(&:to_s)
       end
 
       def call
-        return [] if rows.empty?
+        inserted = insert_all
+        return inserted.pluck('id') unless returning
 
-        model.transaction { rows.each_slice(batch_size).flat_map { |batch| write(batch) } }
+        ActiveRecord::Result.new(returning, inserted.map { |attributes| attributes.values_at(*returning) })
       end
 
       private
 
-      attr_reader :model, :rows, :whodunnit, :batch_size
+      attr_reader :model, :rows, :whodunnit, :batch_size, :returning
 
       def tracked
         @tracked ||= adapter.tracked(model.column_names) - ['id']
       end
 
+      def fetched
+        @fetched ||= ['id', *tracked] | returning.to_a
+      end
+
+      def insert_all
+        return [] if rows.empty?
+
+        model.transaction { rows.each_slice(batch_size).flat_map { |batch| write(batch) } }
+      end
+
       def write(batch)
-        inserted = model.insert_all!(batch, returning: ['id', *tracked]).to_a
+        inserted = insert_and_fetch(batch)
         changed = changed_columns(batch)
         adapter.insert(inserted.map { |attributes| version_row(attributes, changed) })
-        inserted.pluck('id')
+        inserted
+      end
+
+      # Databases with RETURNING (Postgres, SQLite 3.35+, MariaDB 10.5+) hand the
+      # inserted rows back.
+      def insert_and_fetch(batch)
+        if model.connection.supports_insert_returning?
+          model.insert_all!(batch, returning: fetched).to_a
+        else
+          supplied = supplied_ids(batch)
+          model.insert_all!(batch)
+          fetch_rows(supplied || derive_ids(batch.size))
+        end
+      end
+
+      # MySQL only - it doesn't give us the inserted rows back for free.
+      def supplied_ids(batch)
+        ids = batch.map { |row| row[:id] || row['id'] }
+        return nil if ids.all?(&:nil?)
+        raise ArgumentError, 'rows must all supply an id or none of them' unless ids.none?(&:nil?)
+
+        cast_ids(ids)
+      end
+
+      def cast_ids(ids)
+        type = model.type_for_attribute('id')
+        ids.map { |id| type.cast(id) }
+      end
+
+      def derive_ids(count)
+        first = select_integer('SELECT LAST_INSERT_ID()')
+        step = select_integer('SELECT @@auto_increment_increment')
+        Array.new(count) { |i| first + (i * step) }
+      end
+
+      def select_integer(sql)
+        model.connection.select_value(sql).to_i
+      end
+
+      def fetch_rows(ids)
+        relation = model.unscoped.where(id: ids).select(*fetched)
+        by_id = model.connection.select_all(relation).to_a.index_by { |row| row['id'] }
+        ids.map { |id| by_id.fetch(id) }
       end
 
       def changed_columns(batch)

data/lib/paper_trail/bulk_writes/model.rb

@@ -8,8 +8,8 @@ module PaperTrail
       end
 
       module ClassMethods
-        def audited_insert_all(rows, whodunnit: PaperTrail.request.whodunnit)
-          Insert.call(model: self, rows: rows, whodunnit: whodunnit)
+        def audited_insert_all(rows, whodunnit: PaperTrail.request.whodunnit, returning: nil)
+          Insert.call(model: self, rows: rows, whodunnit: whodunnit, returning: returning)
         end
 
         def audited_update_all(attributes:, whodunnit: PaperTrail.request.whodunnit, touch: true)

data/lib/paper_trail/bulk_writes/version.rb

@@ -2,6 +2,6 @@
 
 module PaperTrail
   module BulkWrites
-    VERSION = '0.1.0'
+    VERSION = '0.3.0'
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: paper_trail-bulk_writes
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Sam Robinson
@@ -50,6 +50,7 @@ extensions: []
 extra_rdoc_files: []
 files:
 - CHANGELOG.md
+- CODE_OF_CONDUCT.md
 - LICENSE.txt
 - README.md
 - Rakefile