saseo 0.5.0 → 0.8.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: b329bf7dd271331dda62ef081a05d9f7e0c64af9
4
- data.tar.gz: ed09bc9155ee23828beee19b3fa32e0c781307e5
3
+ metadata.gz: 67445e3765a335a0ec968f0f052890632e27a2bf
4
+ data.tar.gz: 98cec85032a774a25e3ae7e5c2906dad60903ab1
5
5
  SHA512:
6
- metadata.gz: 993d46fb9c8d728324a334c627bdfefae21d53668d9c431bc615bd343370d04ea3c0636e83ecb1cc15d6014232498acaf5e26765c4e3410b67334e670e5ace9f
7
- data.tar.gz: 8c270a1382502571d728940c10509d00f5f8a18c4d0bcb181b11088fe98180d08d259d112b3ebbab4be463491b6ac9ae526235e846b814ac34bc97a9f23db8ce
6
+ metadata.gz: 95b70a7ea7208c7f1329009752fa4fb1a320bc1c9ee814bdc131acdd6743bbf1ff0e722a7df8252262077970359a3f31a71a23b86949c3815c1881362cf77f66
7
+ data.tar.gz: 6bfabc3a0858e4eb251850ef98753a3efb0766ec41f6f8f978edbc75e960d3f907e0fd20b5b024a9c4faa251b7fb8895ae598beded0e1585397aa3f407503ee2
data/.travis.yml CHANGED
@@ -12,7 +12,7 @@ rvm:
12
12
  - ruby-head
13
13
  - jruby-head
14
14
  script:
15
- - bundle exec rake DATABASE=saseo_source DB=test db:create db:migrate && bundle exec rake DATABASE=saseo DB=test db:create db:migrate && DATABASE=saseo bundle exec rspec
15
+ - bundle exec rake DATABASE=saseo_source RACK_ENV=test db:create db:migrate && bundle exec rake DATABASE=saseo RACK_ENV=test db:create db:migrate && DATABASE=saseo bundle exec rake
16
16
  addons:
17
17
  postgresql: "9.4"
18
18
  code_climate:
data/README.md CHANGED
@@ -9,6 +9,62 @@ RabbitMQ based PaperTrail replacement
9
9
  [![License](http://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org)
10
10
 
11
11
 
12
+ ## Overview
13
+ Saseo has two primary responsibilities:
14
+
15
+ 1. Publishing all changes from a database
16
+ 1. Persisting the changes to a separate data store
17
+
18
+
19
+ ### Publishing
20
+ Saseo uses triggers to temporarily store changes to a local staging table. Saseo provides a stateless executable, `saseo_publisher`, that reads the records from that table, publishes them to a RabbitMQ Headers Exchange, then deletes them from the local staging table.
21
+
22
+ __Because the messages are published in this way, the data is available to *ANY AND ALL* clients that can connect to RabbitMQ.__
23
+
24
+ #### Published Message Format
25
+
26
+ | Attribute | Details | Header* | Payload** |
27
+ |-----------|---------|--------|---------|
28
+ | table_name| name of the table the record is from | X | |
29
+ | action| `INSERT`, `UPDATE`, or `DELETE`| X | |
30
+ | whodunnit | the user or other entity responsible for the change | X | |
31
+ | item_id | value, if any, of the id field from the record| X | |
32
+ | item_uuid | value, if any, of the id field from the record| X | |
33
+ | transaction_id | transaction_id from the source DB | X | |
34
+ | locale | I18n.locale of the publisher| X | |
35
+ | database | source database name for the record| X | |
36
+ | schema | source schema name for the record| X | |
37
+ | id | uuid of the saseo version| | X |
38
+ | old_data | JSON representation of the record before the change| | X |
39
+ | new_data | JSON representation of the record after the change| | X |
40
+ | action_timestamp | timestamp from the transaction| | X |
41
+
42
+ > _* Header: attribute that can be subscribed to_
43
+ > _** Payload: attribute that can NOT be subscribed to, only accessible once the message is received_
44
+
45
+ ### Persistence
46
+ Saseo provides a stateless executable, `saseo_consumer`, that subscribes to RabbitMQ to pick up the canges published by `saseo_publisher` and persists them to a database specified by config. Saseo currently creates one table for every tracked table.
47
+
48
+
49
+ #### Saseo table structure
50
+ | Column | Type | Modifiers |
51
+ |----------------------------------|--------------------------|-------------------------------------|
52
+ | id | uuid | not null default uuid_generate_v4() |
53
+ | transaction_id | bigint | not null |
54
+ | item_id | bigint | |
55
+ | item_uuid | uuid | |
56
+ | action | character varying | not null |
57
+ | action_timestamp | timestamp with time zone | not null |
58
+ | whodunnit | character varying | not null |
59
+ | changeset | jsonb | not null |
60
+ | new_data | jsonb | |
61
+
62
+
63
+
64
+ ### Sample Setup
65
+ ![](https://raw.githubusercontent.com/avantcredit/saseo/master/sample_saseo_setup.png)
66
+
67
+
12
68
  ## Installation
13
69
 
14
70
  Add this line to your application's Gemfile:
@@ -25,19 +81,19 @@ Or install it yourself as:
25
81
 
26
82
  $ gem install saseo
27
83
 
84
+
28
85
  ## Usage
29
86
 
30
87
  TODO: Write usage instructions here
31
88
 
32
- ## Sample Setup
33
- ![](https://raw.githubusercontent.com/avantcredit/saseo/master/sample_saseo_setup.png)
89
+
34
90
  ## Development
35
91
 
36
92
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
37
93
 
38
94
  To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
39
95
 
40
- ## Contributing
41
96
 
42
- Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/saseo.
97
+ ## Contributing
43
98
 
99
+ Bug reports and pull requests are welcome on GitHub at https://github.com/avantcredit/saseo.
@@ -0,0 +1,5 @@
1
+ class AddChangesetColumn < ActiveRecord::Migration
2
+ def change
3
+ add_column :saseo_versions, :changeset, :jsonb, null: false
4
+ end
5
+ end
@@ -0,0 +1,9 @@
1
+ class DropOldData < ActiveRecord::Migration
2
+ def up
3
+ remove_column :saseo_versions, :old_data
4
+ end
5
+
6
+ def down
7
+ add_column :saseo_versions, :old_data, :jsonb
8
+ end
9
+ end
@@ -0,0 +1,9 @@
1
+ class UseTimeZones < ActiveRecord::Migration
2
+ def up
3
+ change_column :saseo_versions, :action_timestamp, 'TIMESTAMP WITH TIME ZONE', null: false
4
+ end
5
+
6
+ def down
7
+ change_column :saseo_versions, :action_timestamp, :datetime, null: false
8
+ end
9
+ end
@@ -0,0 +1,9 @@
1
+ class UseBigIntForItemId < ActiveRecord::Migration
2
+ def up
3
+ change_column :saseo_versions, :item_id, :integer, limit: 8
4
+ end
5
+
6
+ def down
7
+ change_column :saseo_versions, :item_id, :integer
8
+ end
9
+ end
data/db/saseo/schema.rb CHANGED
@@ -11,7 +11,7 @@
11
11
  #
12
12
  # It's strongly recommended that you check this file into your version control system.
13
13
 
14
- ActiveRecord::Schema.define(version: 20151116235958) do
14
+ ActiveRecord::Schema.define(version: 20151201224236) do
15
15
 
16
16
  # These are extensions that must be enabled in order to support this database
17
17
  enable_extension "plpgsql"
@@ -19,14 +19,14 @@ ActiveRecord::Schema.define(version: 20151116235958) do
19
19
 
20
20
  create_table "saseo_versions", id: :uuid, default: "uuid_generate_v4()", force: :cascade do |t|
21
21
  t.integer "transaction_id", limit: 8, null: false
22
- t.integer "item_id"
22
+ t.integer "item_id", limit: 8
23
23
  t.uuid "item_uuid"
24
24
  t.string "table_name", null: false
25
25
  t.string "action", null: false
26
26
  t.datetime "action_timestamp", null: false
27
27
  t.string "whodunnit", null: false
28
- t.jsonb "old_data"
29
28
  t.jsonb "new_data"
29
+ t.jsonb "changeset", null: false
30
30
  end
31
31
 
32
32
  add_index "saseo_versions", ["table_name", "action_timestamp", "item_id"], name: "saseo_item_id_idx", using: :btree
@@ -1,22 +1,102 @@
1
1
  class InitialSchema < ActiveRecord::Migration
2
2
  def up
3
- enable_extension 'uuid-ossp'
4
- execute 'CREATE SCHEMA saseo'
5
-
6
- create_table 'saseo.saseo_source_versions', id: :uuid, force: true do |t|
7
- t.integer 'transaction_id', null: false, limit: 8 # bigint
8
- t.string 'table_name', null: false
9
- t.string 'action', null: false
10
- t.datetime 'action_timestamp', null: false
11
- t.string 'whodunnit', null: false
12
- t.jsonb 'old_data'
13
- t.jsonb 'new_data'
14
- end
3
+
4
+ uuid_extension_sql = 'CREATE EXTENSION IF NOT EXISTS "uuid-ossp"'
5
+
6
+ audit_table_sql = <<SQL
7
+ CREATE schema IF NOT EXISTS saseo;
8
+ DROP TABLE IF EXISTS saseo.saseo_source_versions;
9
+
10
+ CREATE TABLE saseo.saseo_source_versions (
11
+ id uuid PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
12
+ transaction_id bigint NOT NULL,
13
+ table_name text NOT NULL,
14
+ action_timestamp timestamp WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
15
+ action text NOT NULL CHECK (action IN ('INSERT','DELETE','UPDATE')),
16
+ whodunnit text NOT NULL,
17
+ old_data jsonb,
18
+ new_data jsonb
19
+ );
20
+
21
+ SQL
22
+ # originally copied from:
23
+ # https://wiki.postgresql.org/wiki/Audit_trigger
24
+ #
25
+ # modified to use NOTIFY
26
+ #
27
+ # a trigger must be added to every table
28
+ # CREATE TRIGGER identities_saseo_trigger AFTER INSERT OR UPDATE OR DELETE ON identities FOR EACH ROW EXECUTE PROCEDURE saseo.audit_func();
29
+ whodunnit_sql = <<SQL
30
+ CREATE OR REPLACE FUNCTION saseo.whodunnit() RETURNS TEXT AS $body$
31
+ BEGIN
32
+ return current_setting('saseo.whodunnit');
33
+ EXCEPTION WHEN undefined_object THEN
34
+ return current_user;
35
+ END;
36
+ $body$
37
+ LANGUAGE plpgsql
38
+ SQL
39
+ audit_sql = <<SQL
40
+ CREATE OR REPLACE FUNCTION saseo.audit_func() RETURNS TRIGGER AS $body$
41
+ DECLARE
42
+ v_old_data json;
43
+ v_new_data json;
44
+ v_whodunnit text;
45
+ v_audit_uuid uuid;
46
+ BEGIN
47
+ v_whodunnit := saseo.whodunnit();
48
+ v_audit_uuid := uuid_generate_v4();
49
+
50
+ IF (TG_OP = 'UPDATE') THEN
51
+ v_old_data := row_to_json(OLD);
52
+ v_new_data := row_to_json(NEW);
53
+
54
+ INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, v_old_data::JSONB, v_new_data::JSONB);
55
+
56
+ RETURN NEW;
57
+ ELSIF (TG_OP = 'DELETE') THEN
58
+ v_old_data := row_to_json(OLD);
59
+
60
+ INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, v_old_data::JSONB, NULL);
61
+
62
+ RETURN OLD;
63
+ ELSIF (TG_OP = 'INSERT') THEN
64
+ v_new_data := row_to_json(NEW);
65
+
66
+ INSERT INTO saseo.saseo_source_versions VALUES(v_audit_uuid, txid_current(), TG_TABLE_NAME, DEFAULT, TG_OP, v_whodunnit, NULL, v_new_data::JSONB);
67
+
68
+ RETURN NEW;
69
+ ELSE
70
+ RAISE WARNING '[SASEO.AUDIT_FUNC] - Other action occurred: %, at %',TG_OP,now();
71
+ RETURN NULL;
72
+ END IF;
73
+
74
+ EXCEPTION
75
+ WHEN data_exception THEN
76
+ RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [DATA EXCEPTION] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
77
+ RETURN NULL;
78
+ WHEN unique_violation THEN
79
+ RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [UNIQUE] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
80
+ RETURN NULL;
81
+ WHEN OTHERS THEN
82
+ RAISE WARNING '[SASEO.AUDIT_FUNC] - UDF ERROR [OTHER] - SQLSTATE: %, SQLERRM: %',SQLSTATE,SQLERRM;
83
+ RETURN NULL;
84
+ END;
85
+ $body$
86
+ LANGUAGE plpgsql
87
+ SQL
88
+ execute(uuid_extension_sql)
89
+ execute(audit_table_sql)
90
+ execute(whodunnit_sql)
91
+ execute(audit_sql)
15
92
  end
16
93
 
17
94
  def down
18
- drop_table 'saseo.saseo_source_versions'
19
- disable_extension 'uuid-ossp'
20
- execute 'DROP SCHEMA IF EXISTS saseo'
95
+ execute('DROP FUNCTION IF EXISTS saseo.audit_func() CASCADE')
96
+ execute('DROP FUNCTION IF EXISTS saseo.whodunnit() CASCADE')
97
+ execute('DROP TABLE IF EXISTS saseo.saseo_source_versions CASCADE')
98
+ execute('DROP SCHEMA IF EXISTS saseo CASCADE')
99
+ execute('DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE')
100
+
21
101
  end
22
- end
102
+ end
@@ -15,7 +15,7 @@ module Saseo
15
15
  def create_migration_file
16
16
  tables = table_names.dup
17
17
  tables = Saseo::Models::Source::Base.connection.tables if tables.include? 'all'
18
- tables -= [Saseo.config.source_table_name, Saseo.config.table_name]
18
+ tables -= [Saseo.config.source_table_name]
19
19
 
20
20
  @table_names = tables
21
21
  add_saseo_migration("add_saseo_trigger")
@@ -7,11 +7,13 @@ module Saseo
7
7
  SOURCE_DATABASE_CONFIG_PATH = 'config/database.yml'
8
8
  DATABASE_URL = nil
9
9
  DATABASE_CONFIG_PATH = 'saseo/config/saseo_database.yml'
10
- TABLE_NAME = 'saseo_versions'
10
+ TABLE_PREFIX = nil
11
+ TABLE_SUFFIX = nil
11
12
  SOURCE_TABLE_SCHEMA = 'saseo'
12
13
  SOURCE_TABLE_NAME = 'saseo_source_versions'
13
14
  CONSUMER_PHILOTIC_SUBSCRIPTION = 'saseo_audit'
14
15
  IGNORE_FIELDS_CONFIG_PATH = nil
16
+ LOCALE = nil
15
17
 
16
18
  def defaults
17
19
  @defaults ||= Hash[Saseo::Config::Defaults.constants.map do |c|
@@ -0,0 +1,19 @@
1
+ require 'active_record/base'
2
+ require 'saseo/whodunnit'
3
+
4
+ module Saseo
5
+ module Extensions
6
+ module ActiveRecord
7
+ module Adapter
8
+ module Mixin
9
+ extend self
10
+
11
+ def begin_db_transaction
12
+ execute 'BEGIN'
13
+ Saseo::Whodunnit.set_db_whodunnit
14
+ end
15
+ end
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,8 @@
1
+ require 'saseo/extensions/active_record/detector'
2
+
3
+ if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
4
+ require 'active_record/connection_adapters/postgresql_adapter'
5
+ require 'saseo/extensions/active_record/adapter/mixin'
6
+
7
+ ::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend ::Saseo::Extensions::ActiveRecord::Adapter::Mixin
8
+ end
@@ -0,0 +1,29 @@
1
+ require 'active_record/base'
2
+ require 'saseo/whodunnit'
3
+ require 'saseo/config'
4
+
5
+ module Saseo
6
+ module Extensions
7
+ module ActiveRecord
8
+ module Migration
9
+ module Mixin
10
+ extend self
11
+
12
+ def create_table(table_name, options = {}, &block)
13
+ super
14
+ add_trigger(table_name) if connection.schema_exists?(Saseo.config.source_table_schema)
15
+ end
16
+
17
+ def add_trigger(table_name)
18
+ drop_trigger(table_name)
19
+ execute("CREATE TRIGGER #{table_name}_saseo_trigger AFTER INSERT OR UPDATE OR DELETE ON #{table_name} FOR EACH ROW EXECUTE PROCEDURE saseo.audit_func()")
20
+ end
21
+
22
+ def drop_trigger(table_name)
23
+ execute("DROP TRIGGER IF EXISTS #{table_name}_saseo_trigger ON #{table_name} CASCADE")
24
+ end
25
+ end
26
+ end
27
+ end
28
+ end
29
+ end
@@ -0,0 +1,8 @@
1
+ require 'saseo/extensions/active_record/detector'
2
+
3
+ if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
4
+ require 'active_record/migration'
5
+ require 'saseo/extensions/active_record/migration/mixin'
6
+
7
+ ::ActiveRecord::Migration.prepend ::Saseo::Extensions::ActiveRecord::Migration::Mixin
8
+ end
@@ -1,8 +1,2 @@
1
- require 'saseo/extensions/active_record/detector'
2
-
3
- if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && Saseo::Extensions::ActiveRecord::Detector.active_record_version >= 3
4
- require 'active_record/connection_adapters/postgresql_adapter'
5
- require 'saseo/extensions/active_record/adapter_mixin'
6
-
7
- ::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend ::Saseo::Extensions::ActiveRecord::AdapterMixin
8
- end
1
+ require 'saseo/extensions/active_record/adapter'
2
+ require 'saseo/extensions/active_record/migration'
@@ -51,18 +51,23 @@ module Saseo
51
51
  end
52
52
 
53
53
  validates :transaction_id, presence: true
54
- validates :table_name, presence: true
55
54
  validates :action_timestamp, presence: true
56
55
  validates :action, presence: true, inclusion: {in: %w[INSERT UPDATE DELETE]}
57
56
  validates :whodunnit, presence: true
58
- validates :old_data, presence: true, unless: ->(version) { version.new_data.present? }
59
- validates :new_data, presence: true, unless: ->(version) { version.old_data.present? }
57
+
60
58
 
61
59
  validate :validate_has_changes
62
60
 
63
61
  establish_connection database_config
64
62
 
65
- def changes
63
+ def has_changeset?
64
+ changeset.present?
65
+ end
66
+
67
+ def build_changeset
68
+ old_data = (self.old_data && self.old_data.stringify_keys)
69
+ new_data = (self.new_data && self.new_data.stringify_keys)
70
+
66
71
  old_keys = old_data ? old_data.keys : []
67
72
  new_keys = new_data ? new_data.keys : []
68
73
  (old_keys | new_keys).reduce({}) do |changes, key|
@@ -73,13 +78,9 @@ module Saseo
73
78
  end
74
79
  end
75
80
 
76
- def has_changes?
77
- changes.any?
78
- end
79
-
80
81
  private
81
82
  def validate_has_changes
82
- errors.add(:new_data, 'can not be the same as old_data') unless has_changes?
83
+ errors.add(:new_data, 'can not be the same as old_data') unless has_changeset?
83
84
  end
84
85
  end
85
86
  end
@@ -16,6 +16,8 @@ module Saseo
16
16
  end
17
17
 
18
18
  establish_connection database_config
19
+
20
+ validates :table_name, presence: true
19
21
  end
20
22
  end
21
23
  end
@@ -6,6 +6,19 @@ module Saseo
6
6
  module Source
7
7
  class Version < Saseo::Models::Source::Base
8
8
  self.table_name = "#{Saseo.config.source_table_schema}.#{Saseo.config.source_table_name}"
9
+
10
+ validates :old_data, presence: true, unless: ->(version) { version.new_data.present? }
11
+ validates :new_data, presence: true, unless: ->(version) { version.old_data.present? }
12
+
13
+ alias_method :changeset, :build_changeset
14
+
15
+ def database
16
+ self.class.connection.current_database
17
+ end
18
+
19
+ def schema
20
+ self.class.connection.current_schema
21
+ end
9
22
  end
10
23
  end
11
24
  end
@@ -1,10 +1,126 @@
1
1
  require 'saseo/models/base'
2
2
  require 'saseo/config'
3
3
 
4
+ require 'active_record/migration'
5
+
4
6
  module Saseo
5
7
  module Models
6
8
  class Version < Saseo::Models::Base
7
- self.table_name = Saseo.config.table_name
9
+ self.abstract_class = true
10
+
11
+ attr_reader :old_data
12
+
13
+ validates :changeset, presence: true
14
+ before_validation :build_changeset!
15
+
16
+ class << self
17
+
18
+ def column_definitions
19
+ {
20
+ id: {type: :uuid, args: {default: 'uuid_generate_v4()'}},
21
+ transaction_id: {type: :integer, args: [:transaction_id, {limit: 8, null: false}]},
22
+ item_id: {type: :integer, args: [:item_id, limit: 8]},
23
+ item_uuid: {type: :uuid, args: [:item_uuid]},
24
+ action: {type: :string, args: [:action, {null: false}]},
25
+ action_timestamp: {type: :column, args: [:action_timestamp, 'TIMESTAMP WITH TIME ZONE', {null: false}]},
26
+ whodunnit: {type: :string, args: [:whodunnit, {null: false}]},
27
+ changeset: {type: :jsonb, args: [:changeset, {null: false}]},
28
+ new_data: {type: :jsonb, args: [:new_data]},
29
+
30
+ }
31
+ end
32
+
33
+ def migration_indexes
34
+ quoted_table_name = migration.connection.quote_table_name(table_name)
35
+
36
+ # use raw execute for indexes to let PostgreSQL generate index names
37
+ [
38
+ "CREATE INDEX ON #{quoted_table_name} (transaction_id)",
39
+ "CREATE INDEX ON #{quoted_table_name} (item_id, action_timestamp)",
40
+ "CREATE INDEX ON #{quoted_table_name} (item_uuid, action_timestamp)",
41
+ "CREATE INDEX ON #{quoted_table_name} USING GIN(changeset)",
42
+ "CREATE INDEX ON #{quoted_table_name} USING GIN(new_data)",
43
+ ]
44
+ end
45
+
46
+ def migration
47
+ @migration ||= ActiveRecord::Migration.new
48
+ end
49
+
50
+ def class_for_table(table_name)
51
+ table_class_name = "#{Saseo.config.table_prefix}#{table_name}#{Saseo.config.table_suffix}".classify
52
+
53
+ return Saseo::Models::Version.const_get(table_class_name) if Saseo::Models::Version.const_defined?(table_class_name)
54
+ Class.new(Saseo::Models::Version).tap do |klass|
55
+ Saseo::Models::Version.const_set(table_class_name, klass)
56
+ klass.table_name = table_name
57
+ klass.ensure_table!
58
+ end
59
+
60
+ end
61
+
62
+ def unmarshal_from_message(message)
63
+ class_for_table(message.table_name).new.tap do |version|
64
+
65
+ column_definitions.each_key do |attr|
66
+ version.send("#{attr}=", message.send(attr)) if message.respond_to?(attr)
67
+ end
68
+
69
+ # ActiveRecord 3 doesn't handle jsonb columns properly
70
+ version.old_data = message.old_data && Oj.dump(message.old_data)
71
+ version.new_data = message.new_data && Oj.dump(message.new_data)
72
+
73
+ version
74
+ end
75
+ end
76
+
77
+ def ensure_table!
78
+ ActiveRecord::Base.establish_connection database_config
79
+ return if connection.table_exists? table_name
80
+
81
+ create_table!
82
+ create_table_indexes!
83
+ end
84
+
85
+ def create_table!
86
+ migration.transaction do
87
+ migration.create_table table_name, create_table_options do |t|
88
+
89
+ column_definitions.each_pair do |column, definition|
90
+ next if column == :id
91
+ t.send(definition[:type], *definition[:args])
92
+ end
93
+ end
94
+ end
95
+ end
96
+
97
+ def create_table_options
98
+ {id: column_definitions[:id][:type], force: false}.merge(column_definitions[:id][:args])
99
+ end
100
+
101
+ def create_table_indexes!
102
+ migration_indexes.each do |index|
103
+ migration.execute index
104
+ end
105
+ end
106
+ end
107
+
108
+ def changeset
109
+ self[:changeset] || build_changeset!
110
+ end
111
+
112
+ def old_data=(val)
113
+ @old_data = val.is_a?(String) ? Oj.load(val) : val
114
+ end
115
+
116
+ def table_name
117
+ self.class.table_name
118
+ end
119
+
120
+ private
121
+ def build_changeset!
122
+ self.changeset = build_changeset
123
+ end
8
124
  end
9
125
  end
10
126
  end
@@ -34,25 +34,13 @@ module Saseo
34
34
  end
35
35
 
36
36
  def persist_message!(message)
37
- save_version!(build_version(message))
38
- end
39
-
40
- def build_version(message)
41
- Saseo::Models::Version.new.tap do |version|
42
-
43
- version_attributes.each do |attr|
44
- version.send("#{attr}=", message.send(attr))
45
- end
46
-
47
- # ActiveRecord 3 doesn't handle jsonb columns properly
48
- version.old_data = message.old_data && Oj.dump(message.old_data)
49
- version.new_data = message.new_data && Oj.dump(message.new_data)
50
- end
37
+ save_version!(Saseo::Models::Version.unmarshal_from_message(message))
51
38
  end
52
39
 
53
40
  def save_version!(version)
54
- if version.has_changes?
41
+ if version.has_changeset?
55
42
  begin
43
+ version.class.ensure_table!
56
44
  version.save!
57
45
  logger.debug { "saved version: #{version.id}" }
58
46
 
@@ -1,4 +1,5 @@
1
1
  require 'philotic/message'
2
+ require 'saseo/config'
2
3
  require 'oj'
3
4
 
4
5
  module Saseo
@@ -8,7 +9,7 @@ module Saseo
8
9
  COMPONENT = :saseo
9
10
  MESSAGE_TYPE = :'saseo.record_audit'
10
11
 
11
- attr_routable :table_name, :action, :whodunnit, :item_id, :item_uuid, :transaction_id
12
+ attr_routable :table_name, :action, :whodunnit, :item_id, :item_uuid, :transaction_id, :locale, :database, :schema
12
13
  attr_payload :id, :old_data, :new_data, :action_timestamp
13
14
 
14
15
  def initialize(version)
@@ -17,13 +18,16 @@ module Saseo
17
18
  @id = version.id
18
19
  @transaction_id = version.transaction_id
19
20
  @table_name = version.table_name
21
+ @locale = Saseo.config.locale
22
+ @database = version.database
23
+ @schema = version.schema
20
24
  @action = version.action
21
25
  @whodunnit = version.whodunnit
22
26
  @action_timestamp = version.action_timestamp
23
27
 
24
28
  # ActiveRecord 3 doesn't handle jsonb columns properly
25
- @old_data = ensure_json_load version.old_data
26
- @new_data = ensure_json_load version.new_data
29
+ @old_data = version.old_data
30
+ @new_data = version.new_data
27
31
 
28
32
  @item_id = @new_data ? @new_data['id'] : @old_data['id']
29
33
  @item_uuid = @new_data ? @new_data['uuid'] : @old_data['uuid']
@@ -32,12 +36,6 @@ module Saseo
32
36
  @philotic_message_type = MESSAGE_TYPE
33
37
 
34
38
  end
35
-
36
-
37
- def ensure_json_load(val)
38
- return val unless val.is_a? String
39
- Oj.load val
40
- end
41
39
  end
42
40
  end
43
41
  end
data/lib/saseo/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module Saseo
2
- VERSION = "0.5.0"
2
+ VERSION = "0.8.0"
3
3
  end
@@ -26,11 +26,13 @@ module Saseo
26
26
 
27
27
  def set_db_whodunnit(who = nil)
28
28
  who ||= whodunnit
29
- connection.execute "SET saseo.whodunnit TO '#{connection.quote_string who.to_s}'"
29
+ connection && connection.execute("SET saseo.whodunnit TO '#{connection.quote_string who.to_s}'")
30
30
  end
31
31
 
32
32
  def connection
33
- return ActiveRecord::Base.connection if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && ActiveRecord::Base.connected?
33
+ if Saseo::Extensions::ActiveRecord::Detector.active_record_detected? && ActiveRecord::Base.connected?
34
+ ActiveRecord::Base.connection
35
+ end
34
36
  end
35
37
  end
36
38
 
data/saseo.example.yml CHANGED
@@ -4,7 +4,8 @@ defaults: &defaults
4
4
  source_database_config_path: /path/to/source/db/config
5
5
  database_url: postgres://user:pass@host:port/saseo
6
6
  source_database_config_path: /path/to/saseo/db/config
7
- table_name: saseo_audit_versions
7
+ table_prefix: saseo_
8
+ table_suffix: _versions
8
9
  source_table_schema: audits
9
10
  source_table_name: audits
10
11
 
data/saseo.gemspec CHANGED
@@ -22,6 +22,8 @@ Gem::Specification.new do |spec|
22
22
  spec.add_development_dependency 'bundler', '~> 1.10'
23
23
  spec.add_development_dependency 'codeclimate-test-reporter', '~> 0.4'
24
24
  spec.add_development_dependency 'database_cleaner'
25
+ spec.add_development_dependency 'factory_girl_rails'
26
+ spec.add_development_dependency 'faker'
25
27
  spec.add_development_dependency 'generator_spec'
26
28
  spec.add_development_dependency 'rake', '~> 10.0'
27
29
  spec.add_development_dependency 'rspec'
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: saseo
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Nathan Keyes
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2015-11-30 00:00:00.000000000 Z
11
+ date: 2015-12-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: awesome_print
@@ -66,6 +66,34 @@ dependencies:
66
66
  - - ">="
67
67
  - !ruby/object:Gem::Version
68
68
  version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: factory_girl_rails
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: faker
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
69
97
  - !ruby/object:Gem::Dependency
70
98
  name: generator_spec
71
99
  requirement: !ruby/object:Gem::Requirement
@@ -231,6 +259,10 @@ files:
231
259
  - db/saseo/migrate/20151028181502_initial_schema.rb
232
260
  - db/saseo/migrate/20151108210618_add_indexes.rb
233
261
  - db/saseo/migrate/20151116235958_allow_null_ids.rb
262
+ - db/saseo/migrate/20151201191940_add_changeset_column.rb
263
+ - db/saseo/migrate/20151201212857_drop_old_data.rb
264
+ - db/saseo/migrate/20151201221009_use_time_zones.rb
265
+ - db/saseo/migrate/20151201224236_use_big_int_for_item_id.rb
234
266
  - db/saseo/schema.rb
235
267
  - db/saseo_source/migrate/20151028181502_initial_schema.rb
236
268
  - db/saseo_source/schema.rb
@@ -247,8 +279,11 @@ files:
247
279
  - lib/saseo/config/saseo_source_database.yml
248
280
  - lib/saseo/extensions.rb
249
281
  - lib/saseo/extensions/active_record.rb
250
- - lib/saseo/extensions/active_record/adapter_mixin.rb
282
+ - lib/saseo/extensions/active_record/adapter.rb
283
+ - lib/saseo/extensions/active_record/adapter/mixin.rb
251
284
  - lib/saseo/extensions/active_record/detector.rb
285
+ - lib/saseo/extensions/active_record/migration.rb
286
+ - lib/saseo/extensions/active_record/migration/mixin.rb
252
287
  - lib/saseo/models/base.rb
253
288
  - lib/saseo/models/source/base.rb
254
289
  - lib/saseo/models/source/version.rb
@@ -1,17 +0,0 @@
1
- require 'active_record/base'
2
- require 'saseo/whodunnit'
3
-
4
- module Saseo
5
- module Extensions
6
- module ActiveRecord
7
- module AdapterMixin
8
- extend self
9
-
10
- def begin_db_transaction
11
- execute 'BEGIN'
12
- Saseo::Whodunnit.set_db_whodunnit
13
- end
14
- end
15
- end
16
- end
17
- end