logidze 1.1.0 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e6c441d5e6e60ef695eb354e4bfb5be9fa182bd8b70755d829aee6eb2781e401
4
- data.tar.gz: d64374a0d6327f2f713fd7341c31de93472f32d045cb5c825c35874c756307d4
3
+ metadata.gz: 7130810a9954a68eb38b0c9484b58091a599d662d36e61d4f2072dbe12612807
4
+ data.tar.gz: a4bc08a8a998826263441aa50cb263e5d3bcc36f48ede24bdad60ef4d7e3f640
5
5
  SHA512:
6
- metadata.gz: f3f453e410de263ed8b0704aece35e6274dfc19c9074a8f75450e84033318b149ae81c9e1f48cc9d515e523b95c98c14cbd3453a7c41db1690febfe8060f640d
7
- data.tar.gz: dbf4a22b357889bc0d2aaeda52bffc8bc245cdc12d2cb34d0f7c9ccec7c0f60807ecc3cbd5272fee567420024309f8be05f6e1e8c9f45334ab3d2ad155a97a12
6
+ metadata.gz: 1136d0509508787e18f3839f63293b384f315438cb2675f4ff517b2ca3afa9da9e7e71d013a234a745fac860c499bf29ebcb0949bc245576c53d02a2217f30d9
7
+ data.tar.gz: 9ee0339acaddf4c442da9699485e33acc57bc0ae20c98a80b5b08ea940cc59a6bc6a93c5d2d5d56d6eac95d4f86f4704bf85536f601304908b6cb5fc28dfb913
data/CHANGELOG.md CHANGED
@@ -2,6 +2,17 @@
2
2
 
3
3
  ## master (unreleased)
4
4
 
5
+ ## 1.2.0 (2021-06-11)
6
+
7
+ - Add user-defined exception handling ([@skryukov][])
8
+
9
+ By default, Logidze raises an exception which causes the entire transaction to fail.
10
+ To change this behavior, it's now possible to override `logidze_capture_exception(error_data jsonb)` function.
11
+
12
+ - [Fixes [#69](https://github.com/palkan/logidze/issues/69)] Fallback on NUMERIC_VALUE_OUT_OF_RANGE exception ([@skryukov][])
13
+
14
+ - [Fixes [#192](https://github.com/palkan/logidze/issues/192)] Skip `log_data` column during `apply_column_diff` ([@skryukov][])
15
+
5
16
  ## 1.1.0 (2021-03-31)
6
17
 
7
18
  - Add pending upgrade checks [Experimental]. ([@skryukov][])
@@ -347,3 +358,4 @@ This is a quick fix for a more general problem (see [#59](https://github.com/pal
347
358
  [@duderman]: https://github.com/duderman
348
359
  [@oleg-kiviljov]: https://github.com/oleg-kiviljov
349
360
  [@skryukov]: https://github.com/skryukov
361
+ [@bf4]: https://github.com/bf4
data/README.md CHANGED
@@ -44,6 +44,7 @@ Other requirements:
44
44
  - [Associations versioning](#associations-versioning)
45
45
  - [Dealing with large logs](#dealing-with-large-logs)
46
46
  - [Handling records deletion](#handling-records-deletion)
47
+ - [Handling PG exceptions](#handling-pg-exceptions)
47
48
  - [Upgrading](#upgrading)
48
49
  - [Log format](#log-format)
49
50
  - [Troubleshooting 🚨](#troubleshooting)
@@ -54,7 +55,7 @@ Other requirements:
54
55
  Add Logidze to your application's Gemfile:
55
56
 
56
57
  ```ruby
57
- gem "logidze", "~> 1.0.0"
58
+ gem "logidze", "~> 1.1"
58
59
  ```
59
60
 
60
61
  Install required DB extensions and create trigger function:
@@ -434,6 +435,15 @@ If you want to keep changes history after records deletion as well, consider usi
434
435
 
435
436
  See also the discussion: [#61](https://github.com/palkan/logidze/issues/61).
436
437
 
438
+ ## Handling PG exceptions
439
+
440
+ By default, Logidze raises an exception which causes the entire transaction to fail.
441
+ To change this behavior, it's now possible to override `logidze_capture_exception(error_data jsonb)` function.
442
+
443
+ For example, you may want to raise a warning instead of an exception and complete the transaction without updating log_data.
444
+
445
+ Related issues: [#193](https://github.com/palkan/logidze/issues/193)
446
+
437
447
  ## Upgrading
438
448
 
439
449
  We try to make an upgrade process as simple as possible. For now, the only required action is to create and run a migration:
@@ -544,6 +554,12 @@ First, when restoring data dumps you should consider using `--disable-triggers`
544
554
 
545
555
  When restoring data dumps for a particular PostgreSQL schema (e.g., when using Apartment), you may encounter the issue with non-existent Logidze functions. That happens because `pg_dump` adds `SELECT pg_catalog.set_config('search_path', '', false);`, and, thus, breaks our existing triggers/functions, because they live either in "public" or in a tenant's namespace (see [this thread](https://postgrespro.com/list/thread-id/2448092)).
546
556
 
557
+ ### `PG::NumericValueOutOfRange: ERROR: value overflows numeric format`
558
+
559
+ Due to the usage of `hstore_to_jsonb_loose` under the hood, there could be a situation when you have a string representing a number in the scientific notation (e.g., "557236406134e62000323100"). Postgres would try to convert it to a number (a pretty big one, for sure) and fail with the exception.
560
+
561
+ Related issues: [#69](https://github.com/palkan/logidze/issues/69).
562
+
547
563
  ## Development
548
564
 
549
565
  We use [Dip](https://github.com/bibendi/dip) for development. Provision the project by running `dip provision` and then use `dip bundle`, `dip rspec` or `dip bash` to interact with a Docker development environment.
@@ -0,0 +1,23 @@
1
+ CREATE OR REPLACE FUNCTION logidze_capture_exception(error_data jsonb) RETURNS boolean AS $body$
2
+ -- version: 1
3
+ BEGIN
4
+ -- Feel free to change this function to change Logidze behavior on exception.
5
+ --
6
+ -- Return `false` to raise exception or `true` to commit record changes.
7
+ --
8
+ -- `error_data` contains:
9
+ -- - returned_sqlstate
10
+ -- - message_text
11
+ -- - pg_exception_detail
12
+ -- - pg_exception_hint
13
+ -- - pg_exception_context
14
+ -- - schema_name
15
+ -- - table_name
16
+ -- Learn more about available keys:
17
+ -- https://www.postgresql.org/docs/9.6/plpgsql-control-structures.html#PLPGSQL-EXCEPTION-DIAGNOSTICS-VALUES
18
+ --
19
+
20
+ return false;
21
+ END;
22
+ $body$
23
+ LANGUAGE plpgsql;
@@ -1,5 +1,5 @@
1
1
  CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
2
- -- version: 1
2
+ -- version: 2
3
3
  DECLARE
4
4
  changes jsonb;
5
5
  version jsonb;
@@ -9,26 +9,32 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
9
9
  history_limit integer;
10
10
  debounce_time integer;
11
11
  current_version integer;
12
- merged jsonb;
12
+ k text;
13
13
  iterator integer;
14
14
  item record;
15
15
  columns text[];
16
16
  include_columns boolean;
17
17
  ts timestamp with time zone;
18
18
  ts_column text;
19
+ err_sqlstate text;
20
+ err_message text;
21
+ err_detail text;
22
+ err_hint text;
23
+ err_context text;
24
+ err_table_name text;
25
+ err_schema_name text;
26
+ err_jsonb jsonb;
27
+ err_captured boolean;
19
28
  BEGIN
20
29
  ts_column := NULLIF(TG_ARGV[1], 'null');
21
30
  columns := NULLIF(TG_ARGV[2], 'null');
22
31
  include_columns := NULLIF(TG_ARGV[3], 'null');
23
32
 
24
33
  IF TG_OP = 'INSERT' THEN
25
- -- always exclude log_data column
26
- changes := to_jsonb(NEW.*) - 'log_data';
27
-
28
34
  IF columns IS NOT NULL THEN
29
- snapshot = logidze_snapshot(changes, ts_column, columns, include_columns);
35
+ snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column, columns, include_columns);
30
36
  ELSE
31
- snapshot = logidze_snapshot(changes, ts_column);
37
+ snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column);
32
38
  END IF;
33
39
 
34
40
  IF snapshot#>>'{h, -1, c}' != '{}' THEN
@@ -38,13 +44,10 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
38
44
  ELSIF TG_OP = 'UPDATE' THEN
39
45
 
40
46
  IF OLD.log_data is NULL OR OLD.log_data = '{}'::jsonb THEN
41
- -- always exclude log_data column
42
- changes := to_jsonb(NEW.*) - 'log_data';
43
-
44
47
  IF columns IS NOT NULL THEN
45
- snapshot = logidze_snapshot(changes, ts_column, columns, include_columns);
48
+ snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column, columns, include_columns);
46
49
  ELSE
47
- snapshot = logidze_snapshot(changes, ts_column);
50
+ snapshot = logidze_snapshot(to_jsonb(NEW.*), ts_column);
48
51
  END IF;
49
52
 
50
53
  IF snapshot#>>'{h, -1, c}' != '{}' THEN
@@ -89,11 +92,37 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
89
92
  changes := '{}';
90
93
 
91
94
  IF (coalesce(current_setting('logidze.full_snapshot', true), '') = 'on') THEN
92
- changes = hstore_to_jsonb_loose(hstore(NEW.*));
95
+ BEGIN
96
+ changes = hstore_to_jsonb_loose(hstore(NEW.*));
97
+ EXCEPTION
98
+ WHEN NUMERIC_VALUE_OUT_OF_RANGE THEN
99
+ changes = row_to_json(NEW.*)::jsonb;
100
+ FOR k IN (SELECT key FROM jsonb_each(changes))
101
+ LOOP
102
+ IF jsonb_typeof(changes->k) = 'object' THEN
103
+ changes = jsonb_set(changes, ARRAY[k], to_jsonb(changes->>k));
104
+ END IF;
105
+ END LOOP;
106
+ END;
93
107
  ELSE
94
- changes = hstore_to_jsonb_loose(
95
- hstore(NEW.*) - hstore(OLD.*)
96
- );
108
+ BEGIN
109
+ changes = hstore_to_jsonb_loose(
110
+ hstore(NEW.*) - hstore(OLD.*)
111
+ );
112
+ EXCEPTION
113
+ WHEN NUMERIC_VALUE_OUT_OF_RANGE THEN
114
+ changes = (SELECT
115
+ COALESCE(json_object_agg(key, value), '{}')::jsonb
116
+ FROM
117
+ jsonb_each(row_to_json(NEW.*)::jsonb)
118
+ WHERE NOT jsonb_build_object(key, value) <@ row_to_json(OLD.*)::jsonb);
119
+ FOR k IN (SELECT key FROM jsonb_each(changes))
120
+ LOOP
121
+ IF jsonb_typeof(changes->k) = 'object' THEN
122
+ changes = jsonb_set(changes, ARRAY[k], to_jsonb(changes->>k));
123
+ END IF;
124
+ END LOOP;
125
+ END;
97
126
  END IF;
98
127
 
99
128
  changes = changes - 'log_data';
@@ -145,6 +174,30 @@ CREATE OR REPLACE FUNCTION logidze_logger() RETURNS TRIGGER AS $body$
145
174
  END IF;
146
175
 
147
176
  return NEW;
177
+ EXCEPTION
178
+ WHEN OTHERS THEN
179
+ GET STACKED DIAGNOSTICS err_sqlstate = RETURNED_SQLSTATE,
180
+ err_message = MESSAGE_TEXT,
181
+ err_detail = PG_EXCEPTION_DETAIL,
182
+ err_hint = PG_EXCEPTION_HINT,
183
+ err_context = PG_EXCEPTION_CONTEXT,
184
+ err_schema_name = SCHEMA_NAME,
185
+ err_table_name = TABLE_NAME;
186
+ err_jsonb := jsonb_build_object(
187
+ 'returned_sqlstate', err_sqlstate,
188
+ 'message_text', err_message,
189
+ 'pg_exception_detail', err_detail,
190
+ 'pg_exception_hint', err_hint,
191
+ 'pg_exception_context', err_context,
192
+ 'schema_name', err_schema_name,
193
+ 'table_name', err_table_name
194
+ );
195
+ err_captured = logidze_capture_exception(err_jsonb);
196
+ IF err_captured THEN
197
+ return NEW;
198
+ ELSE
199
+ RAISE;
200
+ END IF;
148
201
  END;
149
202
  $body$
150
203
  LANGUAGE plpgsql;
@@ -1,9 +1,10 @@
1
1
  CREATE OR REPLACE FUNCTION logidze_snapshot(item jsonb, ts_column text DEFAULT NULL, columns text[] DEFAULT NULL, include_columns boolean DEFAULT false) RETURNS jsonb AS $body$
2
- -- version: 2
2
+ -- version: 3
3
3
  DECLARE
4
4
  ts timestamp with time zone;
5
5
  k text;
6
6
  BEGIN
7
+ item = item - 'log_data';
7
8
  IF ts_column IS NULL THEN
8
9
  ts := statement_timestamp();
9
10
  ELSE
@@ -1,8 +1,9 @@
1
1
  CREATE OR REPLACE FUNCTION logidze_version(v bigint, data jsonb, ts timestamp with time zone) RETURNS jsonb AS $body$
2
- -- version: 1
2
+ -- version: 2
3
3
  DECLARE
4
4
  buf jsonb;
5
5
  BEGIN
6
+ data = data - 'log_data';
6
7
  buf := jsonb_build_object(
7
8
  'ts',
8
9
  (extract(epoch from ts) * 1000)::bigint,
@@ -48,7 +48,7 @@ module Logidze
48
48
  end
49
49
 
50
50
  # Return diff from the initial state to specified time or version.
51
- # Optional `data` paramater can be used as initial diff state.
51
+ # Optional `data` parameter can be used as initial diff state.
52
52
  def changes_to(time: nil, version: nil, data: {}, from: 0)
53
53
  raise ArgumentError, "Time or version must be specified" if time.nil? && version.nil?
54
54
 
data/lib/logidze/model.rb CHANGED
@@ -239,7 +239,7 @@ module Logidze
239
239
  end
240
240
 
241
241
  def apply_column_diff(column, value)
242
- return if deleted_column?(column)
242
+ return if deleted_column?(column) || column == "log_data"
243
243
 
244
244
  write_attribute column, deserialize_value(column, value)
245
245
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Logidze
4
- VERSION = "1.1.0"
4
+ VERSION = "1.2.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: logidze
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - palkan
8
- autorequire:
8
+ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-03-31 00:00:00.000000000 Z
11
+ date: 2021-06-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: railties
@@ -163,6 +163,7 @@ files:
163
163
  - lib/generators/logidze/fx_helper.rb
164
164
  - lib/generators/logidze/inject_sql.rb
165
165
  - lib/generators/logidze/install/USAGE
166
+ - lib/generators/logidze/install/functions/logidze_capture_exception.sql
166
167
  - lib/generators/logidze/install/functions/logidze_compact_history.sql
167
168
  - lib/generators/logidze/install/functions/logidze_filter_keys.sql
168
169
  - lib/generators/logidze/install/functions/logidze_logger.sql
@@ -200,7 +201,7 @@ metadata:
200
201
  documentation_uri: http://github.com/palkan/logidze
201
202
  homepage_uri: http://github.com/palkan/logidze
202
203
  source_code_uri: http://github.com/palkan/logidze
203
- post_install_message:
204
+ post_install_message:
204
205
  rdoc_options: []
205
206
  require_paths:
206
207
  - lib
@@ -215,8 +216,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
215
216
  - !ruby/object:Gem::Version
216
217
  version: '0'
217
218
  requirements: []
218
- rubygems_version: 3.0.6
219
- signing_key:
219
+ rubygems_version: 3.2.10
220
+ signing_key:
220
221
  specification_version: 4
221
222
  summary: PostgreSQL JSONB-based model changes tracking
222
223
  test_files: []