logidze 0.9.0 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (61) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +101 -5
  3. data/LICENSE.txt +1 -1
  4. data/README.md +309 -89
  5. data/lib/generators/logidze/fx_helper.rb +17 -0
  6. data/lib/generators/logidze/inject_sql.rb +18 -0
  7. data/lib/generators/logidze/install/USAGE +6 -1
  8. data/lib/generators/logidze/install/functions/logidze_compact_history.sql +38 -0
  9. data/lib/generators/logidze/install/functions/logidze_filter_keys.sql +27 -0
  10. data/lib/generators/logidze/install/functions/logidze_logger.sql +150 -0
  11. data/lib/generators/logidze/install/functions/logidze_snapshot.sql +24 -0
  12. data/lib/generators/logidze/install/functions/logidze_version.sql +20 -0
  13. data/lib/generators/logidze/install/install_generator.rb +61 -3
  14. data/lib/generators/logidze/install/templates/hstore.rb.erb +1 -1
  15. data/lib/generators/logidze/install/templates/migration.rb.erb +19 -232
  16. data/lib/generators/logidze/install/templates/migration_fx.rb.erb +41 -0
  17. data/lib/generators/logidze/model/model_generator.rb +60 -20
  18. data/lib/generators/logidze/model/templates/migration.rb.erb +57 -36
  19. data/lib/generators/logidze/model/triggers/logidze.sql +6 -0
  20. data/lib/logidze.rb +43 -21
  21. data/lib/logidze/engine.rb +4 -1
  22. data/lib/logidze/has_logidze.rb +12 -4
  23. data/lib/logidze/history.rb +7 -15
  24. data/lib/logidze/history/type.rb +1 -1
  25. data/lib/logidze/history/version.rb +6 -5
  26. data/lib/logidze/ignore_log_data.rb +11 -19
  27. data/lib/logidze/meta.rb +44 -17
  28. data/lib/logidze/model.rb +63 -46
  29. data/lib/logidze/version.rb +2 -1
  30. data/lib/logidze/versioned_association.rb +0 -1
  31. metadata +43 -103
  32. data/.gitignore +0 -40
  33. data/.hound.yml +0 -3
  34. data/.rubocop.yml +0 -94
  35. data/.travis.yml +0 -39
  36. data/Gemfile +0 -13
  37. data/Rakefile +0 -28
  38. data/bench/performance/README.md +0 -109
  39. data/bench/performance/diff_bench.rb +0 -36
  40. data/bench/performance/insert_bench.rb +0 -20
  41. data/bench/performance/memory_profile.rb +0 -53
  42. data/bench/performance/setup.rb +0 -308
  43. data/bench/performance/update_bench.rb +0 -36
  44. data/bench/triggers/Makefile +0 -56
  45. data/bench/triggers/Readme.md +0 -58
  46. data/bench/triggers/bench.sql +0 -6
  47. data/bench/triggers/hstore_trigger_setup.sql +0 -38
  48. data/bench/triggers/jsonb_minus_2_setup.sql +0 -47
  49. data/bench/triggers/jsonb_minus_setup.sql +0 -49
  50. data/bench/triggers/keys2_trigger_setup.sql +0 -44
  51. data/bench/triggers/keys_trigger_setup.sql +0 -50
  52. data/bin/console +0 -8
  53. data/bin/setup +0 -9
  54. data/gemfiles/rails42.gemfile +0 -5
  55. data/gemfiles/rails5.gemfile +0 -6
  56. data/gemfiles/rails52.gemfile +0 -6
  57. data/gemfiles/railsmaster.gemfile +0 -7
  58. data/lib/logidze/ignore_log_data/ignored_columns.rb +0 -46
  59. data/lib/logidze/ignore_log_data/missing_attribute_patch.rb +0 -16
  60. data/lib/logidze/migration.rb +0 -19
  61. data/logidze.gemspec +0 -33
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0f853c7aed1f6d284150d85530b3fdc0fb8f59b856a4d962e35004507cb8c7f7
4
- data.tar.gz: 513c193a93b6ff059a27463ee96ab754cb88d995b74a8c73b2abf16441aad346
3
+ metadata.gz: c273494bd5a85f1f9d967a8fb474bc8ef4df75d42bf756a96f4a83e6cc9ed1a2
4
+ data.tar.gz: da791cced5e41ff2025ba349d16b394f6398a06b22782a5e78b7be1f18edbb48
5
5
  SHA512:
6
- metadata.gz: 54443a9ddd4b9bea05a9f6e8acd5f0549c359f2476e93d6cb11452597ba8d0b2580b4ac3db78a0bc78e0ec01f368e9e8c6c0de08868e4d6e7679641515c22329
7
- data.tar.gz: 27f28a1e2b8be8b9eb8095ef8ffd192e4397ff3389298cf290991fc76de7998f9c512fe1d153505dfdd6a69186b06365d5085c71d03472b573ec3c6cb3f523b3
6
+ metadata.gz: 30e03d1adf876881e88e630774963d3a347b3c3114a2e345177d9c6b6480bab9bfd76d572fe681dd5a9b006cd2c212ed07e1a340fbc0a9275df48c6da0d11553
7
+ data.tar.gz: 52663d375ac840f39d783eb2441b104b9ab910d3ee93d8ca309be9d9be626396caa6d9daac72d348e0d225945f37948bb197f18928bc865a9d2114735520cecb
@@ -1,6 +1,101 @@
1
1
  # Change log
2
2
 
3
- ## master
3
+ ## 1.0.0 (2020-11-09)
4
+
5
+ - Add `--name` option to model generator to specify the migration name. ([@palkan][])
6
+
7
+ When you update Logidze installation for a model multiple times, you might hit the `DuplicateMigrationNameError` (see [#167](https://github.com/palkan/logidze/issues/167)).
8
+
9
+ - Add `.with_full_snapshot` to add full snapshots to the log instead of diffs. ([@palkan][])
10
+
11
+ Useful in combination with `.without_logging`: first, you perform multiple updates without logging, then
12
+ you do something like `with_full_snapshot { record.touch }` to create a log entry with the current state.
13
+
14
+ - Add `#create_logidze_snapshot!` and `.create_logidze_snapshot` methods. ([@palkan][])
15
+
16
+ - Add integration with `fx` gem. ([@palkan][])
17
+
18
+ Now it's possible to use Logidze with `schema.rb`. Add `fx` gem to the project, and new migrations will be
19
+ using Fx `create_function` / `create_trigger` functions.
20
+
21
+ - Refactored columns filtering. ([@palkan][])
22
+
23
+ Renamed `--whitelist/--blacklist` to `--only/--except` correspondingly.
24
+
25
+ The _only_-logic has been changed: previously we collected the list of columns to ignore at the migration generation time,
26
+ now we filter the columns within the trigger function (thus, schema changes do not affect the columns being tracked).
27
+
28
+ - **Dropped support for Rails 4.2, Ruby 2.4 and PostgreSQL 9.5**. ([@palkan][])
29
+
30
+ ## 0.12.0 (2020-01-02)
31
+
32
+ - PR [#143](https://github.com/palkan/logidze/pull/143) Add `:transactional` option to `#with_meta` and `#with_responsible` ([@oleg-kiviljov][])
33
+
34
+ Now it's possible to set meta and responsible without wrapping the block into a DB transaction. For backward compatibility `:transactional` option by default is set to `true`.
35
+
36
+ Usage:
37
+
38
+ ```ruby
39
+ Logidze.with_meta({ip: request.ip}, transactional: false) do
40
+ post.save!
41
+ end
42
+ ```
43
+
44
+ or
45
+
46
+ ```ruby
47
+ Logidze.with_responsible(user.id, transactional: false) do
48
+ post.save!
49
+ end
50
+ ```
51
+
52
+ ## 0.11.0 (2019-08-15)
53
+
54
+ - **Breaking** Return `nil` when `log_data` is not loaded instead of raising an exception. ([@palkan][])
55
+
56
+ We cannot distinguish between not loaded `log_data` and not-yet-created (i.e. for new records).
57
+ The latter could be used in frameworks/gems ([example](https://github.com/palkan/logidze/issues/127#issuecomment-518798640)).
58
+
59
+ - **Breaking** Only allow specifying `ignore_log_data` at boot time without runtime modifications. ([@palkan][])
60
+
61
+ Playing with ActiveRecord default scopes wasn't a good idea. We fallback to a more explicit way of _telling_ AR
62
+ when to load or ignore the `log_data` column.
63
+
64
+ This change removes `Logidze.with_log_data` method.
65
+
66
+ ## 0.10.0 (2019-05-15)
67
+
68
+ - **Ruby >= 2.4 is required**
69
+
70
+ - PR [#111](https://github.com/palkan/logidze/pull/111) Global configuration for `:ignore_log_data` option ([@dmitrytsepelev][])
71
+
72
+ Now it's possible to avoid loading `log_data` from the DB by default with
73
+
74
+ ```ruby
75
+ Logidze.ignore_log_data_by_default = true
76
+ ```
77
+
78
+ In cases when `ignore_log_data: false` is explicitly passed to the `ignore_log_data` the default setting is being overriden. Also, it's possible to change it inside the block:
79
+
80
+ ```ruby
81
+ Logidze.with_log_data do
82
+ Post.find(params[:id]).log_data
83
+ end
84
+ ```
85
+
86
+ - PR [#110](https://github.com/palkan/logidze/pull/110) Add `reset_log_data` API to nullify log_data column ([@Arkweid][])
87
+
88
+ Usage:
89
+
90
+ Reset the history for a record (or records):
91
+
92
+ ```ruby
93
+ # for single record
94
+ record.reset_log_data
95
+
96
+ # for relation
97
+ User.where(active: true).reset_log_data
98
+ ```
4
99
 
5
100
  ## 0.9.0 (2018-11-28)
6
101
 
@@ -53,7 +148,7 @@ Please run `rails generate logidze:install --update` to regenerate stored functi
53
148
  This feature checks if several logs came in within a debounce time period then only keep the latest one
54
149
  by merging the latest in previous others.
55
150
 
56
- The concept is similar to https://underscorejs.org/#debounce
151
+ The concept is similar to [https://underscorejs.org/#debounce](https://underscorejs.org/#debounce).
57
152
 
58
153
  without `debounce_time`
59
154
 
@@ -114,7 +209,7 @@ with `debounce_time` of `10ms`
114
209
 
115
210
  ## 0.7.0 (2018-08-29)
116
211
 
117
- - [Fixes [#75](https://github.com/palkan/logidze/issues/70)] Fix association versioning with an optional belongs to ([@ankursethi-uscis][])
212
+ - [Fixes [#75](https://github.com/palkan/logidze/issues/70)] Fix association versioning with an optional belongs to ([@amalagaura][])
118
213
 
119
214
  - [PR [#79](https://github.com/palkan/logidze/pull/13)] Allow adding meta information to versions using `with_meta` (addressed [Issue [#60]](https://github.com/palkan/logidze/issues/60)). ([@DmitryTsepelev][])
120
215
 
@@ -182,7 +277,7 @@ This is a quick fix for a more general problem (see [#59](https://github.com/pal
182
277
 
183
278
  ## 0.5.1 (2017-06-15)
184
279
 
185
- - _(Fix)_ Drop _all_ created functions upon rolling back (https://github.com/palkan/logidze/commit/b8e150cc18b3316a8cf0c78f7117339481fb49c6). ([@vassilevsky][])
280
+ - _(Fix)_ Drop _all_ created functions upon rolling back ([commit](https://github.com/palkan/logidze/commit/b8e150cc18b3316a8cf0c78f7117339481fb49c6)). ([@vassilevsky][])
186
281
 
187
282
  ## 0.5.0 (2017-03-28)
188
283
 
@@ -232,7 +327,8 @@ This is a quick fix for a more general problem (see [#59](https://github.com/pal
232
327
  [@charlie-wasp]: https://github.com/charlie-wasp
233
328
  [@akxcv]: https://github.com/akxcv
234
329
  [@vassilevsky]: https://github.com/vassilevsky
235
- [@ankursethi-uscis]: https://github.com/ankursethi-uscis
330
+ [@amalagaura]: https://github.com/amalagaura
236
331
  [@dmitrytsepelev]: https://github.com/DmitryTsepelev
237
332
  [@zocoi]: https://github.com/zocoi
238
333
  [@duderman]: https://github.com/duderman
334
+ [@oleg-kiviljov]: https://github.com/oleg-kiviljov
@@ -1,6 +1,6 @@
1
1
  The MIT License (MIT)
2
2
 
3
- Copyright (c) 2016 palkan
3
+ Copyright (c) 2016-2020 Vladimir Dementyev
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -1,134 +1,187 @@
1
- [![Cult Of Martians](http://cultofmartians.com/assets/badges/badge.svg)](http://cultofmartians.com) [![Gem Version](https://badge.fury.io/rb/logidze.svg)](https://rubygems.org/gems/logidze) [![Build Status](https://travis-ci.org/palkan/logidze.svg?branch=master)](https://travis-ci.org/palkan/logidze) [![Open Source Helpers](https://www.codetriage.com/palkan/logidze/badges/users.svg)](https://www.codetriage.com/palkan/logidze)
1
+ [![Cult Of Martians](http://cultofmartians.com/assets/badges/badge.svg)](http://cultofmartians.com)
2
+ [![Gem Version](https://badge.fury.io/rb/logidze.svg)](https://rubygems.org/gems/logidze)
3
+ ![Build](https://github.com/palkan/logidze/workflows/Build/badge.svg)
4
+ [![Open Source Helpers](https://www.codetriage.com/palkan/logidze/badges/users.svg)](https://www.codetriage.com/palkan/logidze)
2
5
 
3
6
  # Logidze
4
7
 
5
- Logidze provides tools for logging DB records changes. Just like [audited](https://github.com/collectiveidea/audited) and [paper_trail](https://github.com/airblade/paper_trail) do (but [faster](bench/performance)).
8
+ Logidze provides tools for logging DB records changes when using PostgreSQL (>=9.6). Just like [audited](https://github.com/collectiveidea/audited) and [paper_trail](https://github.com/airblade/paper_trail) do (but [faster](bench/performance)).
6
9
 
7
10
  Logidze allows you to create a DB-level log (using triggers) and gives you an API to browse this log.
8
11
  The log is stored with the record itself in JSONB column. No additional tables required.
9
- Currently, only PostgreSQL 9.5+ is supported (for PostgreSQL 9.4 try [jsonbx](http://www.pgxn.org/dist/jsonbx/1.0.0/) extension).
10
12
 
11
- [Read the story behind Logidze](https://evilmartians.com/chronicles/introducing-logidze?utm_source=logidze)
12
-
13
- [How is Logidze pronounced?](https://github.com/palkan/logidze/issues/73)
13
+ 🤔 [How is Logidze pronounced?](https://github.com/palkan/logidze/issues/73)
14
14
 
15
15
  Other requirements:
16
- - Ruby ~> 2.1
17
- - Rails >= 4.2
16
+
17
+ - Ruby ~> 2.5
18
+ - Rails >= 5.0 (for Rails 4.2 use version <=0.12.0)
18
19
 
19
20
  <a href="https://evilmartians.com/">
20
21
  <img src="https://evilmartians.com/badges/sponsored-by-evil-martians.svg" alt="Sponsored by Evil Martians" width="236" height="54"></a>
21
22
 
23
+ ## Links
24
+
25
+ - [Logidze: for all those tired of versioning data](https://evilmartians.com/chronicles/introducing-logidze?utm_source=logidze)
26
+
27
+ ## Table of contents
28
+
29
+ - [Main concepts](#main-concepts)
30
+ - [Installation & Configuration](#installation)
31
+ - [Using with schema.rb](#using-with-schemarb)
32
+ - [Configuring models](#configuring-models)
33
+ - [Backfill data](#backfill-data)
34
+ - [Log size limit](#log-size-limit)
35
+ - [Tracking only selected columns](#tracking-only-selected-columns)
36
+ - [Logs timestamps](#logs-timestamps)
37
+ - [Usage](#usage)
38
+ - [Basic API](#basic-api)
39
+ - [Track meta information](#track-meta-information)
40
+ - [Track responsibility](#track-responsibility)
41
+ - [Disable logging temporary](#disable-logging-temporary)
42
+ - [Reset log](#reset-log)
43
+ - [Creating full snapshot instead of diffs](#full-snapshots)
44
+ - [Associations versioning](#associations-versioning)
45
+ - [Dealing with large logs](#dealing-with-large-logs)
46
+ - [Handling records deletion](#handling-records-deletion)
47
+ - [Upgrading](#upgrading)
48
+ - [Log format](#log-format)
49
+ - [Troubleshooting 🚨](#troubleshooting)
50
+ - [Development](#development)
51
+
22
52
  ## Installation
23
53
 
24
54
  Add Logidze to your application's Gemfile:
25
55
 
26
56
  ```ruby
27
- gem 'logidze'
57
+ gem "logidze", "~> 1.0.0"
28
58
  ```
29
59
 
30
60
  Install required DB extensions and create trigger function:
31
61
 
32
62
  ```sh
33
- rails generate logidze:install
63
+ bundle exec rails generate logidze:install
34
64
  ```
35
65
 
36
- This creates migration for adding trigger function and enabling hstore extension.
66
+ This creates a migration for adding trigger function and enabling the hstore extension.
37
67
 
38
68
  Run migrations:
39
69
 
40
70
  ```sh
41
- rake db:migrate
71
+ bundle exec rails db:migrate
42
72
  ```
43
73
 
44
- **NOTE:** you **must** use SQL schema format since Logidze uses DB functions and triggers:
74
+ **NOTE:** Logidze uses DB functions and triggers, hence you need to use SQL format for a schema dump:
45
75
 
46
76
  ```ruby
47
77
  # application.rb
48
78
  config.active_record.schema_format = :sql
49
79
  ```
50
80
 
51
- 3. Add log column and triggers to the model:
81
+ ### Using with schema.rb
82
+
83
+ Logidze seamlessly integrates with [fx][] gem to make it possible to continue using `schema.rb` for the database schema dump.
84
+
85
+ Add `fx` gem to your Gemfile and run the same Logidze generators: `rails g logidze:install` or `rails g logidze:model`.
86
+
87
+ If for some reason Logidze couldn't detect the presence of Fx in your bundle, you can enforce it by passing `--fx` option to generators.
88
+
89
+ On the other hand, if you have `fx` gem but don't want Logidze to use it—pass `--no-fx` option.
90
+
91
+ ### Configuring models
92
+
93
+ Run the following migration to enable changes tracking for an Active Record model and adding a `log_data::jsonb` column to the table:
52
94
 
53
95
  ```sh
54
- rails generate logidze:model Post
55
- rake db:migrate
96
+ bundle exec rails generate logidze:model Post
97
+ bundle exec rails db:migrate
56
98
  ```
57
99
 
58
100
  This also adds `has_logidze` line to your model, which adds methods for working with logs.
59
101
 
60
- You can provide `limit` option to `generate` to limit the size of the log (by default it's unlimited):
102
+ By default, Logidze tries to infer the path to the model file from the model name and may fail, for example, if you have unconventional project structure. In that case, you should specify the path explicitly:
61
103
 
62
104
  ```sh
63
- rails generate logidze:model Post --limit=10
105
+ bundle exec rails generate logidze:model Post --path "app/models/custom/post.rb"
64
106
  ```
65
107
 
66
- To backfill table data (i.e. create initial snapshots) add `backfill` option:
108
+ ### Backfill data
109
+
110
+ To backfill table data (i.e., create initial snapshots) add `backfill` option to the generator:
67
111
 
68
112
  ```sh
69
- rails generate logidze:model Post --backfill
113
+ bundle exec rails generate logidze:model Post --backfill
70
114
  ```
71
115
 
72
- You can log only particular columns changes. There are mutually exclusive `blacklist` and `whitelist` options for this:
116
+ Now your migration should contain and `UPDATE ...` statement to populate the `log_data` column with the current state.
73
117
 
74
- ```sh
75
- # track all columns, except `created_at` and `active`
76
- rails generate logidze:model Post --blacklist=created_at active
77
- # track only `title` and `body` columns
78
- rails generate logidze:model Post --whitelist=title body
79
- ```
118
+ Otherwise a full snapshot will be created the first time the record is updated.
80
119
 
81
- By default, Logidze tries to infer the path to the model file from the model name and may fail, for example, if you have unconventional project structure. In that case you should specify the path explicitly:
120
+ You can create a snapshot manually by performing the following query:
82
121
 
83
- ```sh
84
- rails generate logidze:model Post --path "app/models/custom/post.rb"
122
+ ```sql
123
+ UPDATE <my_table> as t
124
+ SET log_data = logidze_snapshot(to_jsonb(t))
85
125
  ```
86
126
 
87
- By default, Logidze tries to get a timestamp for a version from record's `updated_at` field whenever appropriate. If
88
- your model does not have that column, Logidze will gracefully fall back to `statement_timestamp()`.
89
- To change the column name or disable this feature completely, you can use the `timestamp_column` option:
127
+ Or by using the following methods:
90
128
 
91
- ```sh
92
- # will try to get the timestamp value from `time` column
93
- rails generate logidze:model Post --timestamp_column time
94
- # will always set version timestamp to `statement_timestamp()`
95
- rails generate logidze:model Post --timestamp_column nil # "null" and "false" will also work
96
- ```
129
+ ```ruby
130
+ Model.create_logidze_snapshot
97
131
 
98
- If you want to update Logidze settings for the model, run migration with `--update` flag:
132
+ # specify the timestamp column to use for the initial version (by default the current time is used)
133
+ Model.create_logidze_snapshot(timestamp: :created_at)
99
134
 
100
- ```sh
101
- rails generate logidze:model Post --update --whitelist=title body rating
135
+ # filter columns
136
+ Model.create_logidze_snapshot(only: %(name))
137
+ Model.create_logidze_snapshot(except: %(password))
138
+
139
+ # or call a similar method (but with !) on a record
140
+
141
+ my_model = Model.find(params[:id])
142
+ my_model.create_logidze_snapshot!(timestamp: :created_at)
102
143
  ```
103
144
 
104
- Logidze also supports associations versioning. It is experimental feature, and disabled by default. You can learn more
105
- in the [wiki](https://github.com/palkan/logidze/wiki/Associations-versioning).
145
+ A snapshot is only created if `log_data` is null.
106
146
 
107
- ## Troubleshooting
147
+ ### Log size limits
108
148
 
109
- The most common problem is `"permission denied to set parameter "logidze.xxx"` caused by `ALTER DATABASE ...` query.
110
- Logidze requires at least database owner privileges (which is not always possible).
149
+ You can provide the `limit` option to `generate` to limit the size of the log (by default it's unlimited):
111
150
 
112
- Here is a quick and straightforward [workaround](https://github.com/palkan/logidze/issues/11#issuecomment-260703464) by [@nobodyzzz](https://github.com/nobodyzzz).
151
+ ```sh
152
+ bundle exec rails generate logidze:model Post --limit=10
153
+ ```
113
154
 
114
- **NOTE**: if you're using PostgreSQL >= 9.6 you need neither the workaround nor owner privileges because Logidze (>= 0.3.1) can work without `ALTER DATABASE ...`.
155
+ ### Tracking only selected columns
115
156
 
116
- Nevertheless, you still need super-user privileges to enable `hstore` extension (or you can use [PostgreSQL Extension Whitelisting](https://github.com/dimitri/pgextwlist)).
157
+ You can log only particular columns changes. There are mutually exclusive `except` and `only` options for this:
117
158
 
159
+ ```sh
160
+ # track all columns, except `created_at` and `active`
161
+ bundle exec rails generate logidze:model Post --except=created_at,active
162
+ # track only `title` and `body` columns
163
+ bundle exec rails generate logidze:model Post --only=title,body
164
+ ```
118
165
 
119
- ## Upgrade from previous versions
166
+ ### Logs timestamps
120
167
 
121
- We try to make upgrade process as simple as possible. For now, the only required action is to create and run a migration:
168
+ By default, Logidze tries to get a timestamp for a version from record's `updated_at` field whenever appropriate. If
169
+ your model does not have that column, Logidze will gracefully fall back to `statement_timestamp()`.
170
+
171
+ To change the column name or disable this feature completely, you can use the `timestamp_column` option:
122
172
 
123
173
  ```sh
124
- rails generate logidze:install --update
174
+ # will try to get the timestamp value from `time` column
175
+ bundle exec rails generate logidze:model Post --timestamp_column time
176
+ # will always set version timestamp to `statement_timestamp()`
177
+ bundle exec rails generate logidze:model Post --timestamp_column nil # "null" and "false" will also work
125
178
  ```
126
179
 
127
- This updates core `logdize_logger` DB function. No need to update tables or triggers.
128
-
129
180
  ## Usage
130
181
 
131
- Your model now has `log_data` column which stores changes log.
182
+ ### Basic API
183
+
184
+ Your model now has `log_data` column, which stores changes log.
132
185
 
133
186
  To retrieve record version at a given time use `#at` or `#at!` methods:
134
187
 
@@ -142,13 +195,13 @@ post.log_version #=> 3
142
195
  post.log_size #=> 3
143
196
 
144
197
  # Get copy of a record at a given time
145
- old_post = post.at(time: 2.days.ago)
198
+ post.at(time: 2.days.ago)
146
199
 
147
200
  # or revert the record itself to the previous state (without committing to DB)
148
- post.at!(time: '201-04-15 12:00:00')
201
+ post.at!(time: "2018-04-15 12:00:00")
149
202
 
150
203
  # If no version found
151
- post.at(time: '1945-05-09 09:00:00') #=> nil
204
+ post.at(time: "1945-05-09 09:00:00") #=> nil
152
205
  ```
153
206
 
154
207
  You can also get revision by version number:
@@ -157,6 +210,9 @@ You can also get revision by version number:
157
210
  post.at(version: 2)
158
211
  ```
159
212
 
213
+ **NOTE:** If `log_data` is nil, `#at(time:)` returns self and `#at(version:)` returns `nil`.
214
+ You can opt-in to return `nil` for time-based `#at` as well by setting `Logidze.return_self_if_log_data_is_empty = false`.
215
+
160
216
  It is also possible to get version for relations:
161
217
 
162
218
  ```ruby
@@ -173,6 +229,8 @@ post.diff_from(time: 1.hour.ago)
173
229
  Post.where(created_at: Time.zone.today.all_day).diff_from(time: 1.hour.ago)
174
230
  ```
175
231
 
232
+ **NOTE:** If `log_data` is nil, `#diff_from` returns an empty Hash as `"changes"`.
233
+
176
234
  There are also `#undo!` and `#redo!` options (and more general `#switch_to!`):
177
235
 
178
236
  ```ruby
@@ -192,48 +250,45 @@ You can initiate reloading of `log_data` from the DB:
192
250
  post.reload_log_data # => returns the latest log data value
193
251
  ```
194
252
 
195
- Normally, if you update record after `#undo!` or `#switch_to!` you lose all "future" versions and `#redo!` is no
253
+ Typically, if you update record after `#undo!` or `#switch_to!` you lose all "future" versions and `#redo!` is no
196
254
  longer possible. However, you can provide an `append: true` option to `#undo!` or `#switch_to!`, which will
197
255
  create a new version with old data. Caveat: when switching to a newer version, `append` will have no effect.
198
256
 
199
257
  ```ruby
200
- post = Post.create!(title: 'first post') # v1
201
- post.update!(title: 'new title') # v2
202
- post.undo!(append: true) # v3 (with same attributes as v1)
258
+ post = Post.create!(title: "first post") # v1
259
+ post.update!(title: "new title") # v2
260
+ post.undo!(append: true) # v3 (with same attributes as v1)
203
261
  ```
204
262
 
205
263
  Note that `redo!` will not work after `undo!(append: true)` because the latter will create a new version
206
264
  instead of rolling back to an old one.
207
- Alternatively, you can configure Logidze to always default to `append: true`.
265
+ Alternatively, you can configure Logidze always to default to `append: true`.
208
266
 
209
267
  ```ruby
210
268
  Logidze.append_on_undo = true
211
269
  ```
212
270
 
213
- If you want to reduce the data loaded from the DB, you can turn off automatic loading of `log_data` column in the following way:
271
+ ### Track meta information
272
+
273
+ You can store any meta information you want inside your version (it could be IP address, user agent, etc.). To add it you should wrap your code with a block:
214
274
 
215
275
  ```ruby
216
- class User < ActiveRecord::Base
217
- has_logidze ignore_log_data: true
276
+ Logidze.with_meta(ip: request.ip) do
277
+ post.save!
218
278
  end
219
279
  ```
220
280
 
221
- After that, each time you use `User.all` (or any other relation method) `log_data` won't be loaded from the DB. If you try to call `#log_data` on the model loaded in a such way, you'll get `ActiveModel::MissingAttributeError`, but if you really need it (e.g. during the console debugging) - use `user.reload_log_data`, which forces loading the column from the DB. If you need to select `log_data` during the initial load - use a special scope `User.with_log_data`.
222
-
223
-
224
- ## Track meta information
281
+ Meta expects a hash to be passed so you won't need to encode and decode JSON manually.
225
282
 
226
- You can store any meta information you want inside your version (it could be IP address, user agent etc). In order to add it you should wrap your code with a block:
283
+ By default `.with_meta` wraps the block into a DB transaction. That could lead to an unexpected behavior, especially, when using `.with_meta` within an around_action. To avoid wrapping the block into a DB transaction use `transactional: false` option.
227
284
 
228
285
  ```ruby
229
- Logidze.with_meta(ip: request.ip) do
286
+ Logidze.with_meta({ip: request.ip}, transactional: false) do
230
287
  post.save!
231
288
  end
232
289
  ```
233
290
 
234
- Meta expects a hash to be passed so you won't need to encode and decode JSON manually.
235
-
236
- ## Track responsibility (aka _whodunnit_)
291
+ ### Track responsibility
237
292
 
238
293
  A special application of meta information is storing the author of the change, which is called _Responsible ID_. There is more likely that you would like to store the `current_user.id` that way.
239
294
 
@@ -251,7 +306,7 @@ And then to retrieve `responsible_id`:
251
306
  post.log_data.responsible_id
252
307
  ```
253
308
 
254
- Logidze does not require `responsible_id` to be `SomeModel` ID. It can be anything. Thus Logidze does not provide methods for retrieving the corresponding object. However, you can easy write it yourself:
309
+ Logidze does not require `responsible_id` to be `SomeModel` ID. It can be anything. Thus Logidze does not provide methods for retrieving the corresponding object. However, you can easily write it yourself:
255
310
 
256
311
  ```ruby
257
312
  class Post < ActiveRecord::Base
@@ -268,17 +323,25 @@ And in your controller:
268
323
 
269
324
  ```ruby
270
325
  class ApplicationController < ActionController::Base
271
- around_action :set_logidze_responsible, only: %i[create update]
326
+ around_action :use_logidze_responsible, only: %i[create update]
272
327
 
273
- def set_logidze_responsible(&block)
328
+ def use_logidze_responsible(&block)
274
329
  Logidze.with_responsible(current_user&.id, &block)
275
330
  end
276
331
  end
277
332
  ```
278
333
 
279
- ## Disable logging temporary
334
+ By default `.with_responsible` wraps the block into a DB transaction. That could lead to an unexpected behavior, especially, when using `.with_responsible` within an around_action. To avoid wrapping the block into a DB transaction use `transactional: false` option.
280
335
 
281
- If you want to make update without logging (e.g. mass update), you can turn it off the following way:
336
+ ```ruby
337
+ Logidze.with_responsible(user.id, transactional: false) do
338
+ post.save!
339
+ end
340
+ ```
341
+
342
+ ### Disable logging temporary
343
+
344
+ If you want to make update without logging (e.g., mass update), you can turn it off the following way:
282
345
 
283
346
  ```ruby
284
347
  Logidze.without_logging { Post.update_all(seen: true) }
@@ -288,6 +351,134 @@ Logidze.without_logging { Post.update_all(seen: true) }
288
351
  Post.without_logging { Post.update_all(seen: true) }
289
352
  ```
290
353
 
354
+ ### Reset log
355
+
356
+ Reset the history for a record (or records):
357
+
358
+ ```ruby
359
+ # for a single record
360
+ record.reset_log_data
361
+
362
+ # for relation
363
+ User.where(active: true).reset_log_data
364
+ ```
365
+
366
+ ### Full snapshots
367
+
368
+ You can instruct Logidze to create a full snapshot instead of a diff for a particular log entry.
369
+
370
+ It could be useful in combination with `.without_logging`: first, you perform multiple updates without logging, then
371
+ you want to create a log entry with the current state. To do that, you should use the `Logidze.with_full_snapshot` method:
372
+
373
+ ```ruby
374
+ record = Model.find(params[:id])
375
+
376
+ Logidze.without_logging do
377
+ # perform multiple write operations with record
378
+ end
379
+
380
+ Logidze.with_full_snapshot do
381
+ record.touch
382
+ end
383
+ ```
384
+
385
+ ### Associations versioning
386
+
387
+ Logidze also supports associations versioning. This feature is disabled by default (due to the number of edge cases). You can learn more
388
+ in the [wiki](https://github.com/palkan/logidze/wiki/Associations-versioning).
389
+
390
+ ## Dealing with large logs
391
+
392
+ By default, Active Record _selects_ all the table columns when no explicit `select` statement specified.
393
+
394
+ That could slow down queries execution if you have field values which exceed the size of the data block (typically 8KB). PostgreSQL turns on its [TOAST](https://wiki.postgresql.org/wiki/TOAST) mechanism), which requires reading from multiple physical locations for fetching the row's data.
395
+
396
+ If you do not use compaction (`generate logidze:model ... --limit N`) for `log_data`, you're likely to face this problem.
397
+
398
+ Logidze provides a way to avoid loading `log_data` by default (and load it on demand):
399
+
400
+ ```ruby
401
+ class User < ActiveRecord::Base
402
+ # Add `ignore_log_data` option to macros
403
+ has_logidze ignore_log_data: true
404
+ end
405
+ ```
406
+
407
+ If you want Logidze to behave this way by default, configure the global option:
408
+
409
+ ```ruby
410
+ # config/initializers/logidze.rb
411
+ Logidze.ignore_log_data_by_default = true
412
+
413
+ # or
414
+
415
+ # config/application.rb
416
+ config.logidze.ignore_log_data_by_default = true
417
+ ```
418
+
419
+ However, you can override it by explicitly passing `ignore_log_data: false` to the `ignore_log_data`.
420
+ You can also enforce loading `log_data` in-place by using the `.with_log_data` scope, e.g. `User.all.with_log_data` loads all
421
+ the _users_ with `log_data` included.
422
+
423
+ The chart below shows the difference in PG query time before and after turning `ignore_log_data` on. (Special thanks to [@aderyabin](https://github.com/aderyabin) for sharing it.)
424
+
425
+ ![](./assets/pg_log_data_chart.png)
426
+
427
+ If you try to call `#log_data` on the model loaded in such way, you'll get `nil`. If you want to fetch log data (e.g., during the console debugging)–use **`user.reload_log_data`**, which forces loading this column from the DB.
428
+
429
+ ## Handling records deletion
430
+
431
+ Unlike, for example, PaperTrail, Logidze is designed to **only track changes**. If the record has been deleted, **everything is lost**.
432
+
433
+ If you want to keep changes history after records deletion as well, consider using specialized tools for soft-delete, such as, [Discard](https://github.com/jhawthorn/discard) or [Paranoia](https://github.com/rubysherpas/paranoia).
434
+
435
+ See also the discussion: [#61](https://github.com/palkan/logidze/issues/61).
436
+
437
+ ## Upgrading
438
+
439
+ We try to make an upgrade process as simple as possible. For now, the only required action is to create and run a migration:
440
+
441
+ ```sh
442
+ rails generate logidze:install --update
443
+ ```
444
+
445
+ This updates core `logdize_logger` DB function. No need to update tables or triggers.
446
+
447
+ **NOTE:** When using `fx`, you can omit the `--update` flag. The migration containing only the updated functions would be created.
448
+
449
+ If you want to update Logidze settings for the model, run migration with `--update` flag:
450
+
451
+ ```sh
452
+ rails generate logidze:model Post --update --only=title,body,rating
453
+ ```
454
+
455
+ You can also use the `--name` option to specify the migration name to avoid duplicate migration names:
456
+
457
+ ```sh
458
+ $ rails generate logidze:model Post --update --only=title,body,rating --name add_only_filter_to_posts_log_data
459
+
460
+ create db/migrate/20202309142344_add_only_filter_to_posts_log_data.rb
461
+ ```
462
+
463
+ ### Upgrading from 0.x to 1.0 (edge)
464
+
465
+ #### Schema and migrations
466
+
467
+ Most SQL functions definitions has changed without backward compatibility.
468
+ Perform the following steps to upgrade:
469
+
470
+ 1. Re-install Logidze: `rails generate logidze:install --update`.
471
+
472
+ 1. Re-install Logidze triggers **for all models**: `rails generate logidze:model <model> --update`.
473
+
474
+ 1. Remove the `include Logidze::Migration` line from the old migration files (if any)—this module has been removed.
475
+
476
+ Rewrite the migrations to not use the `#current_setting(name)` and `#current_setting_missing_supported?` methods or copy them from the latest [0.x release](https://github.com/palkan/logidze/blob/0-stable/lib/logidze/migration.rb).
477
+
478
+ #### API changes
479
+
480
+ The deprecated `time` positional argument has been removed from `#at` and `#diff_from` methods. Now you need to use keyword arguments, i.e., `model.at(some_tome) -> model.at(time: some_time)`.
481
+
291
482
  ## Log format
292
483
 
293
484
  The `log_data` column has the following format:
@@ -314,22 +505,51 @@ The `log_data` column has the following format:
314
505
  }
315
506
  ```
316
507
 
317
- If you specify the limit in the trigger definition then log size will not exceed the specified size. When a new change occurs, and there is no more room for it, the two oldest changes will be merged.
508
+ If you specify the limit in the trigger definition, then log size will not exceed the specified size. When a new change occurs, and there is no more room for it, the two oldest changes will be merged.
318
509
 
319
- ## Development
510
+ ## Troubleshooting
320
511
 
321
- For development setup run `./bin/setup`. This runs `bundle install` and creates test DB.
512
+ ### `log_data` is nil when using Rails fixtures
322
513
 
323
- ## Contributing
514
+ Rails fixtures are populated with triggers disabled. Thus, `log_data` is null initially for all records.
515
+ You can use `#create_logidze_snapshot` manually to build initial snapshots.
516
+
517
+ ### How to make this work with Apartment 🤔
518
+
519
+ First, read [Apartment docs](https://github.com/influitive/apartment#installing-extensions-into-persistent-schemas) on installing PostgreSQL extensions. You need to use the described approach to install Hstore (and drop the migration provided by Logidze during installation).
520
+
521
+ Secondly, set `config.use_sql = true` in the Apartment configuration.
522
+
523
+ Finally, when using `fx` along with `schema.rb`, you might face a problem with duplicate trigger definitions (for different schemas).
524
+ Here is a patch to fix this: [dump_triggers.rake](etc/dump_triggers.rake).
525
+
526
+ Related issues: [#50](https://github.com/palkan/logidze/issues/50).
527
+
528
+ ### `PG::UntranslatableCharacter: ERROR`
324
529
 
325
- Bug reports and pull requests are welcome on GitHub at https://github.com/palkan/logidze.
530
+ That could happen when your row data contain null bytes. You should sanitize the data before writing to the database.
531
+ From the [PostgreSQL docs](https://www.postgresql.org/docs/current/datatype-json.html): `jsonb type also rejects \u0000 (because that cannot be represented in PostgreSQL's text type)`.
326
532
 
533
+ Related issues: [#155](https://github.com/palkan/logidze/issues/155).
327
534
 
328
- ## TODO
535
+ ### `pg_restore` fails to restore a dump
329
536
 
330
- - Enhance update_all to support mass-logging.
331
- - Other DB adapters.
537
+ First, when restoring data dumps you should consider using `--disable-triggers` option (unless you have a strong reason to invoke the triggers).
538
+
539
+ When restoring data dumps for a particular PostgreSQL schema (e.g., when using Apartment), you may encounter the issue with non-existent Logidze functions. That happens because `pg_dump` adds `SELECT pg_catalog.set_config('search_path', '', false);`, and, thus, breaks our existing triggers/functions, because they live either in "public" or in a tenant's namespace (see [this thread](https://postgrespro.com/list/thread-id/2448092)).
540
+
541
+ ## Development
542
+
543
+ We use [Dip](https://github.com/bibendi/dip) for development. Provision the project by running `dip provision` and then use `dip bundle`, `dip rspec` or `dip bash` to interact with a Docker development environment.
544
+
545
+ If you prefer developing on your local machine, make user you have Postgres installed and run `./bin/setup`.
546
+
547
+ ## Contributing
548
+
549
+ Bug reports and pull requests are welcome on GitHub at [https://github.com/palkan/logidze](https://github.com/palkan/logidze).
332
550
 
333
551
  ## License
334
552
 
335
553
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
554
+
555
+ [fx]: https://github.com/teoljungberg/fx