logidze 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +17 -1
- data/lib/generators/logidze/install/functions/logidze_capture_exception.sql +23 -0
- data/lib/generators/logidze/install/functions/logidze_logger.sql +69 -16
- data/lib/generators/logidze/install/functions/logidze_snapshot.sql +2 -1
- data/lib/generators/logidze/install/functions/logidze_version.sql +2 -1
- data/lib/logidze/history.rb +1 -1
- data/lib/logidze/model.rb +1 -1
- data/lib/logidze/version.rb +1 -1
- metadata +7 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7130810a9954a68eb38b0c9484b58091a599d662d36e61d4f2072dbe12612807
|
4
|
+
data.tar.gz: a4bc08a8a998826263441aa50cb263e5d3bcc36f48ede24bdad60ef4d7e3f640
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 1136d0509508787e18f3839f63293b384f315438cb2675f4ff517b2ca3afa9da9e7e71d013a234a745fac860c499bf29ebcb0949bc245576c53d02a2217f30d9
|
7
|
+
data.tar.gz: 9ee0339acaddf4c442da9699485e33acc57bc0ae20c98a80b5b08ea940cc59a6bc6a93c5d2d5d56d6eac95d4f86f4704bf85536f601304908b6cb5fc28dfb913
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,17 @@
|
|
2
2
|
|
3
3
|
## master (unreleased)
|
4
4
|
|
5
|
+
## 1.2.0 (2021-06-11)
|
6
|
+
|
7
|
+
- Add user-defined exception handling ([@skryukov][])
|
8
|
+
|
9
|
+
By default, Logidze raises an exception which causes the entire transaction to fail.
|
10
|
+
To change this behavior, it's now possible to override `logidze_capture_exception(error_data jsonb)` function.
|
11
|
+
|
12
|
+
- [Fixes [#69](https://github.com/palkan/logidze/issues/69)] Fallback on NUMERIC_VALUE_OUT_OF_RANGE exception ([@skryukov][])
|
13
|
+
|
14
|
+
- [Fixes [#192](https://github.com/palkan/logidze/issues/192)] Skip `log_data` column during `apply_column_diff` ([@skryukov][])
|
15
|
+
|
5
16
|
## 1.1.0 (2021-03-31)
|
6
17
|
|
7
18
|
- Add pending upgrade checks [Experimental]. ([@skryukov][])
|
@@ -347,3 +358,4 @@ This is a quick fix for a more general problem (see [#59](https://github.com/pal
|
|
347
358
|
[@duderman]: https://github.com/duderman
|
348
359
|
[@oleg-kiviljov]: https://github.com/oleg-kiviljov
|
349
360
|
[@skryukov]: https://github.com/skryukov
|
361
|
+
[@bf4]: https://github.com/bf4
|
data/README.md
CHANGED
@@ -44,6 +44,7 @@ Other requirements:
|
|
44
44
|
- [Associations versioning](#associations-versioning)
|
45
45
|
- [Dealing with large logs](#dealing-with-large-logs)
|
46
46
|
- [Handling records deletion](#handling-records-deletion)
|
47
|
+
- [Handling PG exceptions](#handling-pg-exceptions)
|
47
48
|
- [Upgrading](#upgrading)
|
48
49
|
- [Log format](#log-format)
|
49
50
|
- [Troubleshooting 🚨](#troubleshooting)
|
@@ -54,7 +55,7 @@ Other requirements:
|
|
54
55
|
Add Logidze to your application's Gemfile:
|
55
56
|
|
56
57
|
```ruby
|
57
|
-
gem "logidze", "~> 1.
|
58
|
+
gem "logidze", "~> 1.1"
|
58
59
|
```
|
59
60
|
|
60
61
|
Install required DB extensions and create trigger function:
|
@@ -434,6 +435,15 @@ If you want to keep changes history after records deletion as well, consider usi
|
|
434
435
|
|
435
436
|
See also the discussion: [#61](https://github.com/palkan/logidze/issues/61).
|
436
437
|
|
438
|
+
## Handling PG exceptions
|
439
|
+
|
440
|
+
By default, Logidze raises an exception which causes the entire transaction to fail.
|
441
|
+
To change this behavior, it's now possible to override `logidze_capture_exception(error_data jsonb)` function.
|
442
|
+
|
443
|
+
For example, you may want to raise a warning instead of an exception and complete the transaction without updating log_data.
|
444
|
+
|
445
|
+
Related issues: [#193](https://github.com/palkan/logidze/issues/193)
|
446
|
+
|
437
447
|
## Upgrading
|
438
448
|
|
439
449
|
We try to make an upgrade process as simple as possible. For now, the only required action is to create and run a migration:
|
@@ -544,6 +554,12 @@ First, when restoring data dumps you should consider using `--disable-triggers`
|
|
544
554
|
|
545
555
|
When restoring data dumps for a particular PostgreSQL schema (e.g., when using Apartment), you may encounter the issue with non-existent Logidze functions. That happens because `pg_dump` adds `SELECT pg_catalog.set_config('search_path', '', false);`, and, thus, breaks our existing triggers/functions, because they live either in "public" or in a tenant's namespace (see [this thread](https://postgrespro.com/list/thread-id/2448092)).
|
546
556
|
|
557
|
+
### `PG::NumericValueOutOfRange: ERROR: value overflows numeric format`
|
558
|
+
|
559
|
+
Due to the usage of `hstore_to_jsonb_loose` under the hood, there could be a situation when you have a string representing a number in the scientific notation (e.g., "557236406134e62000323100"). Postgres would try to convert it to a number (a pretty big one, for sure) and fail with the exception.
|
560
|
+
|
561
|
+
Related issues: [#69](https://github.com/palkan/logidze/issues/69).
|
562
|
+
|
547
563
|
## Development
|
548
564
|
|
549
565
|
We use [Dip](https://github.com/bibendi/dip) for development. Provision the project by running `dip provision` and then use `dip bundle`, `dip rspec` or `dip bash` to interact with a Docker development environment.
|
@@ -0,0 +1,23 @@
|
|
1
|
+
CREATE OR REPLACE FUNCTION logidze_capture_exception(error_data jsonb) RETURNS boolean AS $body$
|
2
|
+
-- version: 1
|
3
|
+
BEGIN
|
4
|
+
-- Feel free to change this function to change Logidze behavior on exception.
|
5
|
+
--
|
6
|
+
-- Return `false` to raise exception or `true` to commit record changes.
|
7
|
+
--
|
8
|
+
-- `error_data` contains:
|
9
|
+
-- - returned_sqlstate
|
10
|
+
-- - message_text
|
11
|
+
-- - pg_exception_detail
|
12
|
+
-- - pg_exception_hint
|
13
|
+
-- - pg_exception_context
|
14
|
+
-- - schema_name
|
15
|
+
-- - table_name
|
16
|
+
-- Learn more about available keys:
|
17
|
+
-- https://www.postgresql.org/docs/9.6/plpgsql-control-structures.html#PLPGSQL-EXCEPTION-DIAGNOSTICS-VALUES
|
18
|
+
--
|
19
|
+
|
20
|
+
return false;
|
21
|
+
END;
|
22
|
+
$body$
|
23
|
+
LANGUAGE plpgsql;
|
@@ -1,5 +1,5 @@
|
|
1
1
|
CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
|
2
|
-
-- version:
|
2
|
+
-- version: 2
|
3
3
|
DECLARE
|
4
4
|
changes jsonb;
|
5
5
|
version jsonb;
|
@@ -9,26 +9,32 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
|
|
9
9
|
history_limit integer;
|
10
10
|
debounce_time integer;
|
11
11
|
current_version integer;
|
12
|
-
|
12
|
+
k text;
|
13
13
|
iterator integer;
|
14
14
|
item record;
|
15
15
|
columns text[];
|
16
16
|
include_columns boolean;
|
17
17
|
ts timestamp with time zone;
|
18
18
|
ts_column text;
|
19
|
+
err_sqlstate text;
|
20
|
+
err_message text;
|
21
|
+
err_detail text;
|
22
|
+
err_hint text;
|
23
|
+
err_context text;
|
24
|
+
err_table_name text;
|
25
|
+
err_schema_name text;
|
26
|
+
err_jsonb jsonb;
|
27
|
+
err_captured boolean;
|
19
28
|
BEGIN
|
20
29
|
ts_column := NULLIF(TG_ARGV[1], 'null');
|
21
30
|
columns := NULLIF(TG_ARGV[2], 'null');
|
22
31
|
include_columns := NULLIF(TG_ARGV[3], 'null');
|
23
32
|
|
24
33
|
IF TG_OP = 'INSERT' THEN
|
25
|
-
-- always exclude log_data column
|
26
|
-
changes := to_jsonb(NEW.*) - 'log_data';
|
27
|
-
|
28
34
|
IF columns IS NOT NULL THEN
|
29
|
-
snapshot = logidze_snapshot(
|
35
|
+
snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column, columns, include_columns);
|
30
36
|
ELSE
|
31
|
-
snapshot = logidze_snapshot(
|
37
|
+
snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column);
|
32
38
|
END IF;
|
33
39
|
|
34
40
|
IF snapshot#>>'{h, -1, c}' != '{}' THEN
|
@@ -38,13 +44,10 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
|
|
38
44
|
ELSIF TG_OP = 'UPDATE' THEN
|
39
45
|
|
40
46
|
IF OLD.log_data is NULL OR OLD.log_data = '{}'::jsonb THEN
|
41
|
-
-- always exclude log_data column
|
42
|
-
changes := to_jsonb(NEW.*) - 'log_data';
|
43
|
-
|
44
47
|
IF columns IS NOT NULL THEN
|
45
|
-
snapshot = logidze_snapshot(
|
48
|
+
snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column, columns, include_columns);
|
46
49
|
ELSE
|
47
|
-
snapshot = logidze_snapshot(
|
50
|
+
snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column);
|
48
51
|
END IF;
|
49
52
|
|
50
53
|
IF snapshot#>>'{h, -1, c}' != '{}' THEN
|
@@ -89,11 +92,37 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
|
|
89
92
|
changes := '{}';
|
90
93
|
|
91
94
|
IF (coalesce(current_setting('logidze.full_snapshot', true), '') = 'on') THEN
|
92
|
-
|
95
|
+
BEGIN
|
96
|
+
changes = hstore_to_jsonb_loose(hstore(NEW.*));
|
97
|
+
EXCEPTION
|
98
|
+
WHEN NUMERIC_VALUE_OUT_OF_RANGE THEN
|
99
|
+
changes = row_to_json(NEW.*)::jsonb;
|
100
|
+
FOR k IN (SELECT key FROM jsonb_each(changes))
|
101
|
+
LOOP
|
102
|
+
IF jsonb_typeof(changes->k) = 'object' THEN
|
103
|
+
changes = jsonb_set(changes, ARRAY[k], to_jsonb(changes->>k));
|
104
|
+
END IF;
|
105
|
+
END LOOP;
|
106
|
+
END;
|
93
107
|
ELSE
|
94
|
-
|
95
|
-
|
96
|
-
|
108
|
+
BEGIN
|
109
|
+
changes = hstore_to_jsonb_loose(
|
110
|
+
hstore(NEW.*) - hstore(OLD.*)
|
111
|
+
);
|
112
|
+
EXCEPTION
|
113
|
+
WHEN NUMERIC_VALUE_OUT_OF_RANGE THEN
|
114
|
+
changes = (SELECT
|
115
|
+
COALESCE(json_object_agg(key, value), '{}')::jsonb
|
116
|
+
FROM
|
117
|
+
jsonb_each(row_to_json(NEW.*)::jsonb)
|
118
|
+
WHERE NOT jsonb_build_object(key, value) <@ row_to_json(OLD.*)::jsonb);
|
119
|
+
FOR k IN (SELECT key FROM jsonb_each(changes))
|
120
|
+
LOOP
|
121
|
+
IF jsonb_typeof(changes->k) = 'object' THEN
|
122
|
+
changes = jsonb_set(changes, ARRAY[k], to_jsonb(changes->>k));
|
123
|
+
END IF;
|
124
|
+
END LOOP;
|
125
|
+
END;
|
97
126
|
END IF;
|
98
127
|
|
99
128
|
changes = changes - 'log_data';
|
@@ -145,6 +174,30 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
|
|
145
174
|
END IF;
|
146
175
|
|
147
176
|
return NEW;
|
177
|
+
EXCEPTION
|
178
|
+
WHEN OTHERS THEN
|
179
|
+
GET STACKED DIAGNOSTICS err_sqlstate = RETURNED_SQLSTATE,
|
180
|
+
err_message = MESSAGE_TEXT,
|
181
|
+
err_detail = PG_EXCEPTION_DETAIL,
|
182
|
+
err_hint = PG_EXCEPTION_HINT,
|
183
|
+
err_context = PG_EXCEPTION_CONTEXT,
|
184
|
+
err_schema_name = SCHEMA_NAME,
|
185
|
+
err_table_name = TABLE_NAME;
|
186
|
+
err_jsonb := jsonb_build_object(
|
187
|
+
'returned_sqlstate', err_sqlstate,
|
188
|
+
'message_text', err_message,
|
189
|
+
'pg_exception_detail', err_detail,
|
190
|
+
'pg_exception_hint', err_hint,
|
191
|
+
'pg_exception_context', err_context,
|
192
|
+
'schema_name', err_schema_name,
|
193
|
+
'table_name', err_table_name
|
194
|
+
);
|
195
|
+
err_captured = logidze_capture_exception(err_jsonb);
|
196
|
+
IF err_captured THEN
|
197
|
+
return NEW;
|
198
|
+
ELSE
|
199
|
+
RAISE;
|
200
|
+
END IF;
|
148
201
|
END;
|
149
202
|
$body$
|
150
203
|
LANGUAGE plpgsql;
|
@@ -1,9 +1,10 @@
|
|
1
1
|
CREATE OR REPLACE FUNCTION logidze_snapshot(item jsonb, ts_column text DEFAULT NULL, columns text[] DEFAULT NULL, include_columns boolean DEFAULT false) RETURNS jsonb AS $body$
|
2
|
-
-- version:
|
2
|
+
-- version: 3
|
3
3
|
DECLARE
|
4
4
|
ts timestamp with time zone;
|
5
5
|
k text;
|
6
6
|
BEGIN
|
7
|
+
item = item - 'log_data';
|
7
8
|
IF ts_column IS NULL THEN
|
8
9
|
ts := statement_timestamp();
|
9
10
|
ELSE
|
@@ -1,8 +1,9 @@
|
|
1
1
|
CREATE OR REPLACE FUNCTION logidze_version(v bigint, data jsonb, ts timestamp with time zone) RETURNS jsonb AS $body$
|
2
|
-
-- version:
|
2
|
+
-- version: 2
|
3
3
|
DECLARE
|
4
4
|
buf jsonb;
|
5
5
|
BEGIN
|
6
|
+
data = data - 'log_data';
|
6
7
|
buf := jsonb_build_object(
|
7
8
|
'ts',
|
8
9
|
(extract(epoch from ts) * 1000)::bigint,
|
data/lib/logidze/history.rb
CHANGED
@@ -48,7 +48,7 @@ module Logidze
|
|
48
48
|
end
|
49
49
|
|
50
50
|
# Return diff from the initial state to specified time or version.
|
51
|
-
# Optional `data`
|
51
|
+
# Optional `data` parameter can be used as initial diff state.
|
52
52
|
def changes_to(time: nil, version: nil, data: {}, from: 0)
|
53
53
|
raise ArgumentError, "Time or version must be specified" if time.nil? && version.nil?
|
54
54
|
|
data/lib/logidze/model.rb
CHANGED
data/lib/logidze/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logidze
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- palkan
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-06-11 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: railties
|
@@ -163,6 +163,7 @@ files:
|
|
163
163
|
- lib/generators/logidze/fx_helper.rb
|
164
164
|
- lib/generators/logidze/inject_sql.rb
|
165
165
|
- lib/generators/logidze/install/USAGE
|
166
|
+
- lib/generators/logidze/install/functions/logidze_capture_exception.sql
|
166
167
|
- lib/generators/logidze/install/functions/logidze_compact_history.sql
|
167
168
|
- lib/generators/logidze/install/functions/logidze_filter_keys.sql
|
168
169
|
- lib/generators/logidze/install/functions/logidze_logger.sql
|
@@ -200,7 +201,7 @@ metadata:
|
|
200
201
|
documentation_uri: http://github.com/palkan/logidze
|
201
202
|
homepage_uri: http://github.com/palkan/logidze
|
202
203
|
source_code_uri: http://github.com/palkan/logidze
|
203
|
-
post_install_message:
|
204
|
+
post_install_message:
|
204
205
|
rdoc_options: []
|
205
206
|
require_paths:
|
206
207
|
- lib
|
@@ -215,8 +216,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
215
216
|
- !ruby/object:Gem::Version
|
216
217
|
version: '0'
|
217
218
|
requirements: []
|
218
|
-
rubygems_version: 3.
|
219
|
-
signing_key:
|
219
|
+
rubygems_version: 3.2.10
|
220
|
+
signing_key:
|
220
221
|
specification_version: 4
|
221
222
|
summary: PostgreSQL JSONB-based model changes tracking
|
222
223
|
test_files: []
|