logidze 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 662f0ec1e4b6945cffe599484e53691651d70599
4
- data.tar.gz: 6e82b77620011fa0b758376d810cf7faa5bf305e
3
+ metadata.gz: 8df5f1ac5aa84c04a0874f222011418bb4ef0fb7
4
+ data.tar.gz: f354e24b771387fddfc094dd5ea902827d1f4deb
5
5
  SHA512:
6
- metadata.gz: dd2b1e23e0ce3bef64d331cdb9e2ad19eb18ab7c0ca39264813ef75167fc6c4a2848dfc136ed1bd50f0bb85ad92e7e19d2894a8ed37dc767c15570c24e59c259
7
- data.tar.gz: 99597d63737e5cd7d35e88e5466a26a286895a3dac5252805d038c15c651ac375f67785303beee55585198836a543b74fcd207182d9179acd6e5818e48405242
6
+ metadata.gz: 971a0acde6848091d4944632b579a37ab71335a9d7cab813aa43eae054fa4ae5359b8d654403d1ed8d7145c53afb87917366404a616f73fc71bb6af84536e0e4
7
+ data.tar.gz: 61d8ad229bb3d56bb5468b005e7e4907af9195bae3d187934a2acb7906b381628fd188ce99d2fb0af013bff8e22f750ae3c00f3fb7496725605b754ef9bad5bb
data/.gitignore CHANGED
@@ -1,9 +1,40 @@
1
- /.bundle/
2
- /.yardoc
3
- /Gemfile.lock
4
- /_yardoc/
5
- /coverage/
6
- /doc/
7
- /pkg/
8
- /spec/reports/
9
- /tmp/
1
+ # Numerous always-ignore extensions
2
+ *.diff
3
+ *.err
4
+ *.orig
5
+ *.log
6
+ *.rej
7
+ *.swo
8
+ *.swp
9
+ *.vi
10
+ *~
11
+ *.sass-cache
12
+ *.iml
13
+ .idea/
14
+
15
+ # Sublime
16
+ *.sublime-project
17
+ *.sublime-workspace
18
+
19
+ # OS or Editor folders
20
+ .DS_Store
21
+ .cache
22
+ .project
23
+ .settings
24
+ .tmproj
25
+ Thumbs.db
26
+
27
+ .bundle/
28
+ log/*.log
29
+ *.gz
30
+ pkg/
31
+ spec/dummy/db/*.sqlite3
32
+ spec/dummy/db/*.sqlite3-journal
33
+ spec/dummy/tmp/
34
+
35
+ Gemfile.lock
36
+ Gemfile.local
37
+ .rspec
38
+ *.gem
39
+ tmp/
40
+ coverage/
data/.rubocop.yml CHANGED
@@ -7,9 +7,11 @@ AllCops:
7
7
  Exclude:
8
8
  - 'bin/**/*'
9
9
  - 'spec/dummy/**/*'
10
- RunRailsCops: true
10
+ - 'tmp/**/*'
11
+ - 'bench/**/*'
11
12
  DisplayCopNames: true
12
13
  StyleGuideCopsOnly: false
14
+ TargetRubyVersion: 2.3
13
15
 
14
16
  Style/AccessorMethodName:
15
17
  Enabled: false
data/.travis.yml CHANGED
@@ -1,4 +1,37 @@
1
1
  language: ruby
2
- rvm:
3
- - 2.3.0
4
- before_install: gem install bundler -v 1.11.2
2
+ cache: bundler
3
+
4
+ env:
5
+ global:
6
+ - LOGIDZE_DB_USER=postgres
7
+ - LOGIDZE_DB_NAME=logidze
8
+
9
+ before_install:
10
+ - sudo /etc/init.d/postgresql stop
11
+ - sudo apt-get -y remove --purge postgresql-9.1
12
+ - sudo apt-get -y remove --purge postgresql-9.2
13
+ - sudo apt-get -y remove --purge postgresql-9.3
14
+ - sudo apt-get -y remove --purge postgresql-9.4
15
+ - sudo apt-get -y autoremove
16
+ - sudo apt-key adv --keyserver keys.gnupg.net --recv-keys 7FCC7D46ACCC4CF8
17
+ - sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main 9.5" >> /etc/apt/sources.list.d/postgresql.list'
18
+ - sudo apt-get update
19
+ - sudo apt-get -y install postgresql-9.5
20
+ - sudo sh -c 'echo "local all postgres trust" > /etc/postgresql/9.5/main/pg_hba.conf'
21
+ - sudo sh -c 'echo -n "host all all 127.0.0.1/32 trust" >> /etc/postgresql/9.5/main/pg_hba.conf'
22
+ - sudo /etc/init.d/postgresql restart
23
+ - psql --version
24
+
25
+ before_script:
26
+ - bundle exec rake dummy:db:create
27
+ - psql -U postgres -d logidze -c 'CREATE EXTENSION IF NOT EXISTS hstore;'
28
+ - bundle exec rake dummy:db:test:prepare
29
+
30
+ matrix:
31
+ include:
32
+ - rvm: 2.3.0
33
+ gemfile: gemfiles/rails5.gemfile
34
+ - rvm: 2.3.0
35
+ gemfile: gemfiles/rails42.gemfile
36
+ allow_failures:
37
+ - gemfile: gemfiles/rails5.gemfile
data/CHANGELOG.md ADDED
File without changes
data/Gemfile CHANGED
@@ -1,6 +1,5 @@
1
1
  source 'https://rubygems.org'
2
2
 
3
- gem 'pry'
4
3
  # Specify your gem's dependencies in logidze.gemspec
5
4
  gemspec
6
5
 
data/README.md CHANGED
@@ -1,35 +1,170 @@
1
- [![Gem Version](https://badge.fury.io/rb/logidze.svg)](https://rubygems.org/gems/logidze) [![Vexor Status](https://ci.vexor.io/projects/163dc587-613e-4643-a071-a56edbe12f34/status.svg)](https://ci.vexor.io/ui/projects/163dc587-613e-4643-a071-a56edbe12f34/builds)
1
+ [![Gem Version](https://badge.fury.io/rb/logidze.svg)](https://rubygems.org/gems/logidze) [![Build Status](https://travis-ci.org/palkan/logidze.svg?branch=master)](https://travis-ci.org/palkan/logidze) [![Circle CI](https://circleci.com/gh/palkan/logidze/tree/master.svg?style=svg)](https://circleci.com/gh/palkan/logidze/tree/master)
2
2
 
3
3
  # Logidze
4
4
 
5
- TBD
5
+ Logidze provides tools for logging DB records changes.
6
+ **This is not [audited](https://github.com/collectiveidea/audited) or [paper_trail](https://github.com/airblade/paper_trail) alternative!**
7
+
8
+ Logidze allows you to create DB-level log (using triggers) and gives you an API to browse this log.
9
+ Log is stored with the record itself in JSONB column. No additional tables required.
10
+ Currently, only PostgreSQL 9.5+ is supported.
11
+
12
+ Other requirements:
13
+ - Ruby ~>2.3;
14
+ - Rails ~>4.2;
15
+
16
+ <a href="https://evilmartians.com/">
17
+ <img src="https://evilmartians.com/badges/sponsored-by-evil-martians.svg" alt="Sponsored by Evil Martians" width="236" height="54"></a>
6
18
 
7
19
  ## Installation
8
20
 
9
- Add this line to your application's Gemfile:
21
+ 1. Add Logidze your application's Gemfile:
10
22
 
11
23
  ```ruby
12
24
  gem 'logidze'
13
25
  ```
14
26
 
15
- And then execute:
27
+ 2. Install required DB extensions and create trigger function:
28
+
29
+ ```ruby
30
+ rails generate logidze:install
31
+ ```
32
+
33
+ This creates migration for adding trigger function and enabling hstore extension.
34
+
35
+ Run migrations:
36
+
37
+ ```ruby
38
+ rake db:migrate
39
+ ```
40
+
41
+ 3. Add log column and triggers to the model:
16
42
 
17
- $ bundle
43
+ ```ruby
44
+ rails generate logidze:model Post
45
+ rake db:migrate
46
+ ```
18
47
 
19
- Or install it yourself as:
48
+ You can provide `limit` option to `generate` to limit the size of the log (by default it's unlimited):
20
49
 
21
- $ gem install logidze
50
+ ```ruby
51
+ rails generate logidze:mode Post --limit=10
52
+ ```
53
+
54
+ This also adds `has_logidze` line to your model, which adds methods for working with logs.
22
55
 
23
56
  ## Usage
24
57
 
25
- TBD
58
+ Your model now has `log_data` column which stores changes log.
59
+
60
+ To retrieve record version at a given time use `#at` or `#at!` methods:
61
+
62
+ ```ruby
63
+ post = Post.find(27)
64
+
65
+ # Show current version
66
+ post.log_version #=> 3
67
+
68
+ # Show log size (number of versions)
69
+ post.log_size #=> 3
70
+
71
+ # Get copy of a record at a given time
72
+ old_post = post.at(2.days.ago)
73
+
74
+ # or revert the record itself to the previous state (without committing to DB)
75
+ post.at!('201-04-15 12:00:00')
76
+
77
+ # If no version found
78
+ post.at('1945-05-09 09:00:00') #=> nil
79
+ ```
80
+
81
+ You can also get revision by version number:
82
+
83
+ ```ruby
84
+ post.at_version(2)
85
+ ```
86
+
87
+ It is also possible to get version for relations:
88
+
89
+ ```ruby
90
+ Post.where(active: true).at(1.month.ago)
91
+ ```
92
+
93
+ You can also get diff from specified time:
94
+
95
+ ```ruby
96
+ post.diff_from(1.hour.ago)
97
+ #=> { "id" => 27, "changes" => { "title" => { "old" => "Logidze sucks!", "new" => "Logidze rulz!" } } }
98
+
99
+ # the same for relations
100
+ Post.where(created_at: Time.zone.today.all_day).diff_from(1.hour.ago)
101
+ ```
102
+
103
+ There are also `#undo!` and `#redo!` options (and more general `#switch_to!`):
104
+
105
+ ```ruby
106
+ # Revert record to the previous state (and stores this state in DB)
107
+ post.undo!
108
+
109
+ # You can now user redo! to revert back
110
+ post.redo!
111
+
112
+ # More generally you can revert record to arbitrary version
113
+ post.switch_to!(2)
114
+ ```
115
+
116
+ If you update record after `#undo!` or `#switch_to!` you lose all "future" versions and `#redo!` is no longer possible.
117
+
118
+ ## Disable logging temporary
119
+
120
+ If you want to make update without logging (e.g. mass update), you can turn it off the following way:
121
+
122
+ ```ruby
123
+ Logidze.without_logging { Post.update_all(seen: true) }
124
+
125
+ # or
126
+
127
+ Post.without_logging { Post.update_all(seen: true) }
128
+ ```
129
+
130
+ ## Log format
131
+
132
+ The `log_data` column has the following format:
133
+
134
+ ```js
135
+ {
136
+ "v": 2, // current record version,
137
+ "h": // list of changes
138
+ [
139
+ {
140
+ "v": 1, // change number
141
+ "ts": 1460805759352, // change timestamp in milliseconds
142
+ "c": {
143
+ "attr": "new value", // updated fields with new values
144
+ "attr2": "new value"
145
+ }
146
+ }
147
+ ]
148
+ }
149
+ ```
150
+
151
+ If you specified the limit in you trigger definition then log size would not exceed the specified size. When a new change occurs, and there is no more room for it, the two oldest changes get merged.
152
+
153
+ ## Development
154
+
155
+ For development setup run `./bin/setup`. This runs `bundle install` and creates test DB.
26
156
 
27
157
  ## Contributing
28
158
 
29
159
  Bug reports and pull requests are welcome on GitHub at https://github.com/palkan/logidze.
30
160
 
31
161
 
162
+ ## TODO
163
+
164
+ - Exclude columns from log
165
+ - Enhance `update_all` to support mass-logging
166
+ - Other DB adapters
167
+
32
168
  ## License
33
169
 
34
170
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
35
-
data/Rakefile CHANGED
@@ -3,4 +3,23 @@ require "rspec/core/rake_task"
3
3
 
4
4
  RSpec::Core::RakeTask.new(:spec)
5
5
 
6
- task :default => :spec
6
+ namespace :dummy do
7
+ require_relative "spec/dummy/config/application"
8
+ Dummy::Application.load_tasks
9
+ end
10
+
11
+ task(:spec).clear
12
+ desc "Run specs other than spec/acceptance"
13
+ RSpec::Core::RakeTask.new("spec") do |task|
14
+ task.exclude_pattern = "spec/acceptance/**/*_spec.rb"
15
+ task.verbose = false
16
+ end
17
+
18
+ desc "Run acceptance specs in spec/acceptance"
19
+ RSpec::Core::RakeTask.new("spec:acceptance") do |task|
20
+ task.pattern = "spec/acceptance/**/*_spec.rb"
21
+ task.verbose = false
22
+ end
23
+
24
+ desc "Run the specs and acceptance tests"
25
+ task default: %w(spec spec:acceptance)
data/bench/Makefile CHANGED
@@ -8,21 +8,49 @@ ifndef T
8
8
  T = 10000
9
9
  endif
10
10
 
11
- all:
12
- make plain
13
- make hstore
14
- make keys
11
+ all: plain hstore jsonb jsonb2 keys keys2
15
12
 
16
13
  setup:
17
- pgbench -i -q $(DB)
14
+ createdb $(DB) -w
15
+ psql -q -d $(DB) -c 'CREATE EXTENSION IF NOT EXISTS hstore;'
18
16
 
19
- plain: setup
17
+ plain:
18
+ $(info )
19
+ $(info ====== [START] Update without triggers ======)
20
+ pgbench -i -q $(DB)
20
21
  pgbench -f bench.sql -t $(T) -r $(DB)
21
22
 
22
- hstore: setup
23
+ hstore:
24
+ $(info )
25
+ $(info ====== [START] Update with hstore-based triggers ======)
26
+ pgbench -i -q $(DB)
23
27
  psql -q -d $(DB) -f hstore_trigger_setup.sql
24
- pgbench -f bench.sql -t $(T) -r $(DB)
28
+ pgbench -f bench.sql -t $(T) -r $(DB)
29
+
30
+ jsonb:
31
+ $(info )
32
+ $(info ====== [START] Update with jsonb-minus triggers ======)
33
+ pgbench -i -q $(DB)
34
+ psql -q -d $(DB) -f jsonb_minus_setup.sql
35
+ pgbench -f bench.sql -t $(T) -r $(DB)
25
36
 
26
- keys: setup
37
+ jsonb2:
38
+ $(info )
39
+ $(info ====== [START] Update with jsonb-minus triggers ======)
40
+ pgbench -i -q $(DB)
41
+ psql -q -d $(DB) -f jsonb_minus_2_setup.sql
42
+ pgbench -f bench.sql -t $(T) -r $(DB)
43
+
44
+ keys:
45
+ $(info )
46
+ $(info ====== [START] Update with loop thru keys triggers (v1) ======)
47
+ pgbench -i -q $(DB)
27
48
  psql -q -d $(DB) -f keys_trigger_setup.sql
49
+ pgbench -f bench.sql -t $(T) -r $(DB)
50
+
51
+ keys2:
52
+ $(info )
53
+ $(info ====== [START] Update with loop thru keys triggers (v2) ======)
54
+ pgbench -i -q $(DB)
55
+ psql -q -d $(DB) -f keys2_trigger_setup.sql
28
56
  pgbench -f bench.sql -t $(T) -r $(DB)
data/bench/Readme.md ADDED
@@ -0,0 +1,58 @@
1
+ # Triggers benchmarks
2
+
3
+ This benchmark uses standard _pg\_bench_ table `pgbench_accounts`.
4
+ We consider several approaches for calculating records diff: one uses _hstore_ extension, two uses jsonb functions and two others iterate through record fields.
5
+
6
+ # Usage
7
+
8
+ Create database:
9
+
10
+ ```sh
11
+ make setup
12
+ ```
13
+
14
+ You can provide database name through `DB` variable (by default "logidze_bench").
15
+
16
+ Run all benchmarks:
17
+
18
+ ```sh
19
+ make
20
+ ```
21
+
22
+ or separate benchmark:
23
+
24
+ ```sh
25
+ make hstore
26
+
27
+ make jsonb
28
+
29
+ make jsonb2
30
+
31
+ make keys
32
+
33
+ make keys2
34
+
35
+ # raw update, no triggers
36
+ make plain
37
+ ```
38
+
39
+ You can specify the number of transactions by `T` variable (defaults to 10000):
40
+
41
+ ```sh
42
+ make T=1000000
43
+ ```
44
+
45
+ # Results
46
+
47
+ The benchmark shows that hstore and jsonb variants are of the same efficiency (running on MacPro 2013, 2.4 GHz Core i5, 4GB, SSD, 1 million transactions per test):
48
+
49
+ |Mode | TPS | Statement latency (ms) |
50
+ |--------|------|------------------------|
51
+ | plain | 3805 | 0.106 |
52
+ | hstore | 3061 | 0.165 |
53
+ | jsonb | 3079 | 0.165 |
54
+ | jsonb2 | 3057 | 0.166 |
55
+ | keys | 2606 | 0.209 |
56
+ | keys2 | 2610 | 0.216 |
57
+
58
+ _Logidze_ uses jsonb variant.