saseo 0.5.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b329bf7dd271331dda62ef081a05d9f7e0c64af9
-  data.tar.gz: ed09bc9155ee23828beee19b3fa32e0c781307e5
+  metadata.gz: 67445e3765a335a0ec968f0f052890632e27a2bf
+  data.tar.gz: 98cec85032a774a25e3ae7e5c2906dad60903ab1
 SHA512:
-  metadata.gz: 993d46fb9c8d728324a334c627bdfefae21d53668d9c431bc615bd343370d04ea3c0636e83ecb1cc15d6014232498acaf5e26765c4e3410b67334e670e5ace9f
-  data.tar.gz: 8c270a1382502571d728940c10509d00f5f8a18c4d0bcb181b11088fe98180d08d259d112b3ebbab4be463491b6ac9ae526235e846b814ac34bc97a9f23db8ce
+  metadata.gz: 95b70a7ea7208c7f1329009752fa4fb1a320bc1c9ee814bdc131acdd6743bbf1ff0e722a7df8252262077970359a3f31a71a23b86949c3815c1881362cf77f66
+  data.tar.gz: 6bfabc3a0858e4eb251850ef98753a3efb0766ec41f6f8f978edbc75e960d3f907e0fd20b5b024a9c4faa251b7fb8895ae598beded0e1585397aa3f407503ee2

data/.travis.yml

@@ -12,7 +12,7 @@ rvm:
 - ruby-head
 - jruby-head
 script:
-- bundle exec rake DATABASE=saseo_source DB=test db:create db:migrate && bundle exec rake DATABASE=saseo DB=test db:create db:migrate && DATABASE=saseo bundle exec rspec
+- bundle exec rake DATABASE=saseo_source RACK_ENV=test db:create db:migrate && bundle exec rake DATABASE=saseo RACK_ENV=test db:create db:migrate && DATABASE=saseo bundle exec rake
 addons:
   postgresql: "9.4"
   code_climate:

data/README.md

@@ -9,6 +9,62 @@ RabbitMQ based PaperTrail replacement
 [![License](http://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org)
 
 
+## Overview
+Saseo has two primary responsibilities:
+
+1. Publishing all changes from a database
+1. Persisting the changes to a separate data store
+
+
+### Publishing
+Saseo uses triggers to temporarily store changes to a local staging table. Saseo provides a stateless executable, `saseo_publisher`, that reads the records from that table, publishes them to a RabbitMQ Headers Exchange, then deletes them from the local staging table.
+
+__Because the messages are published in this way, the data is available to *ANY AND ALL* clients that can connect to RabbitMQ.__
+
+#### Published Message Format
+
+| Attribute | Details | Header* | Payload** |
+|-----------|---------|--------|---------|
+| table_name| name of the table the record is from | X | |
+| action| `INSERT`, `UPDATE`, or `DELETE`| X | |
+| whodunnit | the user or other entity responsible for the change | X | |
+| item_id | value, if any, of the id field from the record| X | | 
+| item_uuid | value, if any, of the id field from the record| X | | 
+| transaction_id | transaction_id from the source DB | X | | 
+| locale | I18n.locale of the publisher| X | | 
+| database | source database name for the record| X | | 
+| schema | source schema name for the record| X | | 
+| id | uuid of the saseo version|  | X | 
+| old_data | JSON representation of the record before the change|  | X | 
+| new_data | JSON representation of the record after the change|  | X | 
+| action_timestamp | timestamp from the transaction|  | X | 
+
+> _* Header: attribute that can be subscribed to_  
+> _** Payload: attribute that can NOT be subscribed to, only accessible once the message is received_
+
+### Persistence
+Saseo provides a stateless executable, `saseo_consumer`, that subscribes to RabbitMQ to pick up the canges published by `saseo_publisher` and persists them to a database specified by config. Saseo currently creates one table for every tracked table.
+
+
+#### Saseo table structure
+|              Column              |           Type           |              Modifiers              |
+|----------------------------------|--------------------------|-------------------------------------|
+| id                               | uuid                     | not null default uuid_generate_v4() |
+| transaction_id                   | bigint                   | not null                            |
+| item_id                          | bigint                   |                                     |
+| item_uuid                        | uuid                     |                                     |
+| action                           | character varying        | not null                            |
+| action_timestamp                 | timestamp with time zone | not null                            |
+| whodunnit                        | character varying        | not null                            |
+| changeset                        | jsonb                    | not null                            |
+| new_data                         | jsonb                    |                                     |
+
+
+
+### Sample Setup
+![](https://raw.githubusercontent.com/avantcredit/saseo/master/sample_saseo_setup.png)
+
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -25,19 +81,19 @@ Or install it yourself as:
 
     $ gem install saseo
 
+
 ## Usage
 
 TODO: Write usage instructions here
 
-## Sample Setup
-![](https://raw.githubusercontent.com/avantcredit/saseo/master/sample_saseo_setup.png)
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
-## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/saseo.
+## Contributing
 
+Bug reports and pull requests are welcome on GitHub at https://github.com/avantcredit/saseo.

data/db/saseo/migrate/20151201191940_add_changeset_column.rb

@@ -0,0 +1,5 @@
+class AddChangesetColumn < ActiveRecord::Migration
+  def change
+    add_column :saseo_versions, :changeset, :jsonb, null: false
+  end
+end

data/db/saseo/migrate/20151201212857_drop_old_data.rb

@@ -0,0 +1,9 @@
+class DropOldData < ActiveRecord::Migration
+  def up
+    remove_column :saseo_versions, :old_data
+  end
+
+  def down
+    add_column :saseo_versions, :old_data, :jsonb
+  end
+end

data/db/saseo/migrate/20151201221009_use_time_zones.rb

@@ -0,0 +1,9 @@
+class UseTimeZones < ActiveRecord::Migration
+  def up
+    change_column :saseo_versions, :action_timestamp, 'TIMESTAMP WITH TIME ZONE', null: false
+  end
+
+  def down
+    change_column :saseo_versions, :action_timestamp, :datetime, null: false
+  end
+end

data/db/saseo/migrate/20151201224236_use_big_int_for_item_id.rb

@@ -0,0 +1,9 @@
+class UseBigIntForItemId < ActiveRecord::Migration
+  def up
+    change_column :saseo_versions, :item_id, :integer, limit: 8
+  end
+
+  def down
+    change_column :saseo_versions, :item_id, :integer
+  end
+end

data/db/saseo/schema.rb

@@ -11,7 +11,7 @@
 #
 # It's strongly recommended that you check this file into your version control system.
 
-ActiveRecord::Schema.define(version: 20151116235958) do
+ActiveRecord::Schema.define(version: 20151201224236) do
 
   # These are extensions that must be enabled in order to support this database
   enable_extension "plpgsql"
@@ -19,14 +19,14 @@ ActiveRecord::Schema.define(version: 20151116235958) do
 
   create_table "saseo_versions", id: :uuid, default: "uuid_generate_v4()", force: :cascade do |t|
     t.integer  "transaction_id",   limit: 8, null: false
-    t.integer  "item_id"
+    t.integer  "item_id",          limit: 8
     t.uuid     "item_uuid"
     t.string   "table_name",                 null: false
     t.string   "action",                     null: false
     t.datetime "action_timestamp",           null: false
     t.string   "whodunnit",                  null: false
-    t.jsonb    "old_data"
     t.jsonb    "new_data"
+    t.jsonb    "changeset",                  null: false
   end
 
   add_index "saseo_versions", ["table_name", "action_timestamp", "item_id"], name: "saseo_item_id_idx", using: :btree

data/db/saseo_source/migrate/20151028181502_initial_schema.rb

@@ -1,22 +1,102 @@
 class InitialSchema < ActiveRecord::Migration
   def up
-    enable_extension 'uuid-ossp'
-    execute 'CREATE SCHEMA saseo'
-
-    create_table 'saseo.saseo_source_versions', id: :uuid, force: true do |t|
-      t.integer 'transaction_id', null: false, limit: 8 # bigint
-      t.string 'table_name', null: false
-      t.string 'action', null: false
-      t.datetime 'action_timestamp', null: false
-      t.string 'whodunnit', null: false
-      t.jsonb 'old_data'
-      t.jsonb 'new_data'
-    end
+
+    uuid_extension_sql = 'CREATE EXTENSION IF NOT EXISTS "uuid-ossp"'
+
+    audit_table_sql = <<SQL
+CREATE schema IF NOT EXISTS saseo;
+DROP TABLE IF EXISTS saseo.saseo_source_versions;
+
+CREATE TABLE saseo.saseo_source_versions (
+  id uuid PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
+  transaction_id bigint NOT NULL,
+  table_name text NOT NULL,
+  action_timestamp timestamp WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  action text NOT NULL CHECK (action IN ('INSERT','DELETE','UPDATE')),
+  whodunnit text NOT NULL,
+  old_data jsonb,
+  new_data jsonb
+);
+
+SQL
+    # originally copied from:
+    # https://wiki.postgresql.org/wiki/Audit_trigger
+    #
+    # modified to use NOTIFY
+    #
+    # a trigger must be added to every table
+    # CREATE TRIGGER identities_saseo_trigger AFTER INSERT OR UPDATE OR DELETE ON identities FOR EACH ROW EXECUTE PROCEDURE saseo.audit_func();
+    whodunnit_sql = <<SQL
+CREATE OR REPLACE FUNCTION saseo.whodunnit() RETURNS TEXT AS $body$
+BEGIN
+  return current_setting('saseo.whodunnit');
+EXCEPTION WHEN undefined_object THEN
+  return current_user;
+END;
+$body$
+LANGUAGE plpgsql
+SQL
+    audit_sql = <<SQL
+CREATE OR REPLACE FUNCTION saseo.audit_func() RETURNS TRIGGER AS $body$
+DECLARE
+  v_old_data json;
+  v_new_data json;
+  v_whodunnit text;
+  v_audit_uuid uuid;
+BEGIN
+   v_whodunnit := saseo.whodunnit();
+   v_audit_uuid := uuid_generate_v4();
+
+  IF (TG_OP = 'UPDATE') THEN
+    v_old_data := row_to_json(OLD);
+    v_new_data := row_to_json(NEW);
+
+    INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, v_old_data::JSONB, v_new_data::JSONB);
+
+    RETURN NEW;
+  ELSIF (TG_OP = 'DELETE') THEN
+    v_old_data := row_to_json(OLD);
+
+    INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, v_old_data::JSONB, NULL);
+
+    RETURN OLD;
+  ELSIF (TG_OP = 'INSERT') THEN
+    v_new_data := row_to_json(NEW);
+
+    INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, NULL, v_new_data::JSONB);
+
+    RETURN NEW;
+  ELSE
+      RAISE WARNING '[SASEO.AUDIT_FUNC] - Other action occurred: %, at %',TG_OP,now();
+      RETURN NULL;
+  END IF;
+
+EXCEPTION
+  WHEN data_exception THEN
+    RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [DATA EXCEPTION] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
+    RETURN NULL;
+  WHEN unique_violation THEN
+    RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [UNIQUE] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
+    RETURN NULL;
+  WHEN OTHERS THEN
+    RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [OTHER] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
+    RETURN NULL;
+END;
+$body$
+LANGUAGE plpgsql
+SQL
+    execute(uuid_extension_sql)
+    execute(audit_table_sql)
+    execute(whodunnit_sql)
+    execute(audit_sql)
   end
 
   def down
-    drop_table 'saseo.saseo_source_versions'
-    disable_extension 'uuid-ossp'
-    execute 'DROP SCHEMA IF EXISTS saseo'
+    execute('DROP FUNCTION IF EXISTS saseo.audit_func() CASCADE')
+    execute('DROP FUNCTION IF EXISTS saseo.whodunnit() CASCADE')
+    execute('DROP TABLE IF EXISTS saseo.saseo_source_versions CASCADE')
+    execute('DROP SCHEMA IF EXISTS saseo CASCADE')
+    execute('DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE')
+
   end
-end
+end

data/lib/generators/saseo/trigger_generator.rb

@@ -15,7 +15,7 @@ module Saseo
     def create_migration_file
       tables = table_names.dup
       tables = Saseo::Models::Source::Base.connection.tables if tables.include? 'all'
-      tables -= [Saseo.config.source_table_name, Saseo.config.table_name]
+      tables -= [Saseo.config.source_table_name]
 
       @table_names = tables
       add_saseo_migration("add_saseo_trigger")

data/lib/saseo/config/defaults.rb

@@ -7,11 +7,13 @@ module Saseo
       SOURCE_DATABASE_CONFIG_PATH    = 'config/database.yml'
       DATABASE_URL                   = nil
       DATABASE_CONFIG_PATH           = 'saseo/config/saseo_database.yml'
-      TABLE_NAME                     = 'saseo_versions'
+      TABLE_PREFIX                   = nil
+      TABLE_SUFFIX                   = nil
       SOURCE_TABLE_SCHEMA            = 'saseo'
       SOURCE_TABLE_NAME              = 'saseo_source_versions'
       CONSUMER_PHILOTIC_SUBSCRIPTION = 'saseo_audit'
       IGNORE_FIELDS_CONFIG_PATH      = nil
+      LOCALE                         = nil
 
       def defaults
         @defaults ||= Hash[Saseo::Config::Defaults.constants.map do |c|

data/lib/saseo/extensions/active_record/adapter/mixin.rb

@@ -0,0 +1,19 @@
+require 'active_record/base'
+require 'saseo/whodunnit'
+
+module Saseo
+  module Extensions
+    module ActiveRecord
+      module Adapter
+        module Mixin
+          extend self
+
+          def begin_db_transaction
+            execute 'BEGIN'
+            Saseo::Whodunnit.set_db_whodunnit
+          end
+        end
+      end
+    end
+  end
+end

data/lib/saseo/extensions/active_record/adapter.rb

@@ -0,0 +1,8 @@
+require 'saseo/extensions/active_record/detector'
+
+  if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
+    require 'active_record/connection_adapters/postgresql_adapter'
+    require 'saseo/extensions/active_record/adapter/mixin'
+
+    ::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend ::Saseo::Extensions::ActiveRecord::Adapter::Mixin
+  end

data/lib/saseo/extensions/active_record/migration/mixin.rb

@@ -0,0 +1,29 @@
+require 'active_record/base'
+require 'saseo/whodunnit'
+require 'saseo/config'
+
+module Saseo
+  module Extensions
+    module ActiveRecord
+      module Migration
+        module Mixin
+          extend self
+
+          def create_table(table_name, options = {}, &block)
+            super
+            add_trigger(table_name) if connection.schema_exists?(Saseo.config.source_table_schema)
+          end
+
+          def add_trigger(table_name)
+            drop_trigger(table_name)
+            execute("CREATE TRIGGER #{table_name}_saseo_trigger AFTER INSERT OR UPDATE OR DELETE ON #{table_name} FOR EACH ROW EXECUTE PROCEDURE saseo.audit_func()")
+          end
+
+          def drop_trigger(table_name)
+            execute("DROP TRIGGER IF EXISTS #{table_name}_saseo_trigger ON #{table_name} CASCADE")
+          end
+        end
+      end
+    end
+  end
+end

data/lib/saseo/extensions/active_record/migration.rb

@@ -0,0 +1,8 @@
+require 'saseo/extensions/active_record/detector'
+
+if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
+  require 'active_record/migration'
+  require 'saseo/extensions/active_record/migration/mixin'
+
+  ::ActiveRecord::Migration.prepend ::Saseo::Extensions::ActiveRecord::Migration::Mixin
+end

data/lib/saseo/extensions/active_record.rb

@@ -1,8 +1,2 @@
-require 'saseo/extensions/active_record/detector'
-
-if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
-  require 'active_record/connection_adapters/postgresql_adapter'
-  require 'saseo/extensions/active_record/adapter_mixin'
-
-  ::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend ::Saseo::Extensions::ActiveRecord::AdapterMixin
-end
+require 'saseo/extensions/active_record/adapter'
+require 'saseo/extensions/active_record/migration'

data/lib/saseo/models/base.rb

@@ -51,18 +51,23 @@ module Saseo
       end
 
       validates :transaction_id, presence: true
-      validates :table_name, presence: true
       validates :action_timestamp, presence: true
       validates :action, presence: true, inclusion: {in: %w[INSERT UPDATE DELETE]}
       validates :whodunnit, presence: true
-      validates :old_data, presence: true, unless: ->(version) { version.new_data.present? }
-      validates :new_data, presence: true, unless: ->(version) { version.old_data.present? }
+
 
       validate :validate_has_changes
 
       establish_connection database_config
 
-      def changes
+      def has_changeset?
+        changeset.present?
+      end
+
+      def build_changeset
+        old_data = (self.old_data && self.old_data.stringify_keys)
+        new_data = (self.new_data && self.new_data.stringify_keys)
+
         old_keys = old_data ? old_data.keys : []
         new_keys = new_data ? new_data.keys : []
         (old_keys | new_keys).reduce({}) do |changes, key|
@@ -73,13 +78,9 @@ module Saseo
         end
       end
 
-      def has_changes?
-        changes.any?
-      end
-
       private
       def validate_has_changes
-        errors.add(:new_data, 'can not be the same as old_data') unless has_changes?
+        errors.add(:new_data, 'can not be the same as old_data') unless has_changeset?
       end
     end
   end

data/lib/saseo/models/source/base.rb

@@ -16,6 +16,8 @@ module Saseo
         end
 
         establish_connection database_config
+
+        validates :table_name, presence: true
       end
     end
   end

data/lib/saseo/models/source/version.rb

@@ -6,6 +6,19 @@ module Saseo
     module Source
       class Version < Saseo::Models::Source::Base
         self.table_name = "#{Saseo.config.source_table_schema}.#{Saseo.config.source_table_name}"
+
+        validates :old_data, presence: true, unless: ->(version) { version.new_data.present? }
+        validates :new_data, presence: true, unless: ->(version) { version.old_data.present? }
+
+        alias_method :changeset, :build_changeset
+
+        def database
+          self.class.connection.current_database
+        end
+
+        def schema
+          self.class.connection.current_schema
+        end
       end
     end
   end

data/lib/saseo/models/version.rb

@@ -1,10 +1,126 @@
 require 'saseo/models/base'
 require 'saseo/config'
 
+require 'active_record/migration'
+
 module Saseo
   module Models
     class Version < Saseo::Models::Base
-      self.table_name = Saseo.config.table_name
+      self.abstract_class = true
+
+      attr_reader :old_data
+
+      validates :changeset, presence: true
+      before_validation :build_changeset!
+
+      class << self
+
+        def column_definitions
+          {
+            id:               {type: :uuid, args: {default: 'uuid_generate_v4()'}},
+            transaction_id:   {type: :integer, args: [:transaction_id, {limit: 8, null: false}]},
+            item_id:          {type: :integer, args: [:item_id, limit: 8]},
+            item_uuid:        {type: :uuid, args: [:item_uuid]},
+            action:           {type: :string, args: [:action, {null: false}]},
+            action_timestamp: {type: :column, args: [:action_timestamp, 'TIMESTAMP WITH TIME ZONE', {null: false}]},
+            whodunnit:        {type: :string, args: [:whodunnit, {null: false}]},
+            changeset:        {type: :jsonb, args: [:changeset, {null: false}]},
+            new_data:         {type: :jsonb, args: [:new_data]},
+
+          }
+        end
+
+        def migration_indexes
+          quoted_table_name = migration.connection.quote_table_name(table_name)
+
+          # use raw execute for indexes to let PostgreSQL generate index names
+          [
+            "CREATE INDEX ON #{quoted_table_name} (transaction_id)",
+            "CREATE INDEX ON #{quoted_table_name} (item_id, action_timestamp)",
+            "CREATE INDEX ON #{quoted_table_name} (item_uuid, action_timestamp)",
+            "CREATE INDEX ON #{quoted_table_name} USING GIN(changeset)",
+            "CREATE INDEX ON #{quoted_table_name} USING GIN(new_data)",
+          ]
+        end
+
+        def migration
+          @migration ||= ActiveRecord::Migration.new
+        end
+
+        def class_for_table(table_name)
+          table_class_name = "#{Saseo.config.table_prefix}#{table_name}#{Saseo.config.table_suffix}".classify
+
+          return Saseo::Models::Version.const_get(table_class_name) if Saseo::Models::Version.const_defined?(table_class_name)
+          Class.new(Saseo::Models::Version).tap do |klass|
+            Saseo::Models::Version.const_set(table_class_name, klass)
+            klass.table_name = table_name
+            klass.ensure_table!
+          end
+
+        end
+
+        def unmarshal_from_message(message)
+          class_for_table(message.table_name).new.tap do |version|
+
+            column_definitions.each_key do |attr|
+              version.send("#{attr}=", message.send(attr)) if message.respond_to?(attr)
+            end
+
+            # ActiveRecord 3 doesn't handle jsonb columns properly
+            version.old_data = message.old_data && Oj.dump(message.old_data)
+            version.new_data = message.new_data && Oj.dump(message.new_data)
+
+            version
+          end
+        end
+
+        def ensure_table!
+          ActiveRecord::Base.establish_connection database_config
+          return if connection.table_exists? table_name
+
+          create_table!
+          create_table_indexes!
+        end
+
+        def create_table!
+          migration.transaction do
+            migration.create_table table_name, create_table_options do |t|
+
+              column_definitions.each_pair do |column, definition|
+                next if column == :id
+                t.send(definition[:type], *definition[:args])
+              end
+            end
+          end
+        end
+
+        def create_table_options
+          {id: column_definitions[:id][:type], force: false}.merge(column_definitions[:id][:args])
+        end
+
+        def create_table_indexes!
+          migration_indexes.each do |index|
+            migration.execute index
+          end
+        end
+      end
+
+      def changeset
+        self[:changeset] || build_changeset!
+      end
+
+      def old_data=(val)
+        @old_data = val.is_a?(String) ? Oj.load(val) : val
+      end
+
+      def table_name
+        self.class.table_name
+      end
+
+      private
+      def build_changeset!
+        self.changeset = build_changeset
+      end
     end
   end
 end

data/lib/saseo/persistence/persistor.rb

@@ -34,25 +34,13 @@ module Saseo
       end
 
       def persist_message!(message)
-        save_version!(build_version(message))
-      end
-
-      def build_version(message)
-        Saseo::Models::Version.new.tap do |version|
-
-          version_attributes.each do |attr|
-            version.send("#{attr}=", message.send(attr))
-          end
-
-          # ActiveRecord 3 doesn't handle jsonb columns properly
-          version.old_data = message.old_data && Oj.dump(message.old_data)
-          version.new_data = message.new_data && Oj.dump(message.new_data)
-        end
+        save_version!(Saseo::Models::Version.unmarshal_from_message(message))
       end
 
       def save_version!(version)
-        if version.has_changes?
+        if version.has_changeset?
           begin
+            version.class.ensure_table!
             version.save!
             logger.debug { "saved version: #{version.id}" }
 

data/lib/saseo/publishing/data_change_message.rb

@@ -1,4 +1,5 @@
 require 'philotic/message'
+require 'saseo/config'
 require 'oj'
 
 module Saseo
@@ -8,7 +9,7 @@ module Saseo
       COMPONENT    = :saseo
       MESSAGE_TYPE = :'saseo.record_audit'
 
-      attr_routable :table_name, :action, :whodunnit, :item_id, :item_uuid, :transaction_id
+      attr_routable :table_name, :action, :whodunnit, :item_id, :item_uuid, :transaction_id, :locale, :database, :schema
       attr_payload :id, :old_data, :new_data, :action_timestamp
 
       def initialize(version)
@@ -17,13 +18,16 @@ module Saseo
         @id               = version.id
         @transaction_id   = version.transaction_id
         @table_name       = version.table_name
+        @locale           = Saseo.config.locale
+        @database         = version.database
+        @schema           = version.schema
         @action           = version.action
         @whodunnit        = version.whodunnit
         @action_timestamp = version.action_timestamp
 
         # ActiveRecord 3 doesn't handle jsonb columns properly
-        @old_data         = ensure_json_load version.old_data
-        @new_data         = ensure_json_load version.new_data
+        @old_data         = version.old_data
+        @new_data         = version.new_data
 
         @item_id   = @new_data ? @new_data['id'] : @old_data['id']
         @item_uuid = @new_data ? @new_data['uuid'] : @old_data['uuid']
@@ -32,12 +36,6 @@ module Saseo
         @philotic_message_type = MESSAGE_TYPE
 
       end
-
-
-      def ensure_json_load(val)
-        return val unless val.is_a? String
-        Oj.load val
-      end
     end
   end
 end

data/lib/saseo/version.rb

@@ -1,3 +1,3 @@
 module Saseo
-  VERSION = "0.5.0"
+  VERSION = "0.8.0"
 end

data/lib/saseo/whodunnit.rb

@@ -26,11 +26,13 @@ module Saseo
 
     def set_db_whodunnit(who = nil)
       who ||= whodunnit
-      connection.execute "SET saseo.whodunnit TO '#{connection.quote_string who.to_s}'"
+      connection && connection.execute("SET saseo.whodunnit TO '#{connection.quote_string who.to_s}'")
     end
 
     def connection
-      return ActiveRecord::Base.connection if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && ActiveRecord::Base.connected?
+      if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && ActiveRecord::Base.connected?
+        ActiveRecord::Base.connection
+      end
     end
   end
 

data/saseo.example.yml

@@ -4,7 +4,8 @@ defaults: &defaults
   source_database_config_path: /path/to/source/db/config
   database_url: postgres://user:pass@host:port/saseo
   source_database_config_path: /path/to/saseo/db/config
-  table_name: saseo_audit_versions
+  table_prefix: saseo_
+  table_suffix: _versions
   source_table_schema: audits
   source_table_name: audits
 

data/saseo.gemspec

@@ -22,6 +22,8 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'bundler', '~> 1.10'
   spec.add_development_dependency 'codeclimate-test-reporter', '~> 0.4'
   spec.add_development_dependency 'database_cleaner'
+  spec.add_development_dependency 'factory_girl_rails'
+  spec.add_development_dependency 'faker'
   spec.add_development_dependency 'generator_spec'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rspec'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: saseo
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.8.0
 platform: ruby
 authors:
 - Nathan Keyes
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2015-11-30 00:00:00.000000000 Z
+date: 2015-12-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: awesome_print
@@ -66,6 +66,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: factory_girl_rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: faker
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: generator_spec
   requirement: !ruby/object:Gem::Requirement
@@ -231,6 +259,10 @@ files:
 - db/saseo/migrate/20151028181502_initial_schema.rb
 - db/saseo/migrate/20151108210618_add_indexes.rb
 - db/saseo/migrate/20151116235958_allow_null_ids.rb
+- db/saseo/migrate/20151201191940_add_changeset_column.rb
+- db/saseo/migrate/20151201212857_drop_old_data.rb
+- db/saseo/migrate/20151201221009_use_time_zones.rb
+- db/saseo/migrate/20151201224236_use_big_int_for_item_id.rb
 - db/saseo/schema.rb
 - db/saseo_source/migrate/20151028181502_initial_schema.rb
 - db/saseo_source/schema.rb
@@ -247,8 +279,11 @@ files:
 - lib/saseo/config/saseo_source_database.yml
 - lib/saseo/extensions.rb
 - lib/saseo/extensions/active_record.rb
-- lib/saseo/extensions/active_record/adapter_mixin.rb
+- lib/saseo/extensions/active_record/adapter.rb
+- lib/saseo/extensions/active_record/adapter/mixin.rb
 - lib/saseo/extensions/active_record/detector.rb
+- lib/saseo/extensions/active_record/migration.rb
+- lib/saseo/extensions/active_record/migration/mixin.rb
 - lib/saseo/models/base.rb
 - lib/saseo/models/source/base.rb
 - lib/saseo/models/source/version.rb

data/lib/saseo/extensions/active_record/adapter_mixin.rb

@@ -1,17 +0,0 @@
-require 'active_record/base'
-require 'saseo/whodunnit'
-
-module Saseo
-  module Extensions
-    module ActiveRecord
-      module AdapterMixin
-        extend self
-
-        def begin_db_transaction
-          execute 'BEGIN'
-          Saseo::Whodunnit.set_db_whodunnit
-        end
-      end
-    end
-  end
-end