pub_sub_model_sync 0.5.10 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c3cb9b467f173cf208d051aead1d069868b9dfd6cc319311aac4fdaa3431d05f
4
- data.tar.gz: b5f6dad938ee545ab4f99c6cd51b84e574856c63f044f5a46e784ca17a01076c
3
+ metadata.gz: c5b72a2c8c7d97a09c17b3985ce466ecf1fba62573dbb860fae9ccab064e84d9
4
+ data.tar.gz: 4f0f801bc9d51fee36b5f279c4f995d9bf441f608ea26d16314dd2182bf67dc7
5
5
  SHA512:
6
- metadata.gz: c548cd499bfde36a4d0db4a8dc75e8ad3cadc389cfffb55c95db9ab306ecc7ce338274603e66ebb1f45932fd1de61cc5f6d4ad123c6ed0931ad3284cbdc957f3
7
- data.tar.gz: 62ced05bba4175c69d4453e35f5c9fa3414b1d2f123d40ada555e75bc876488d78f388e87a20a01c1bf895ab2991eeb1a0f2287cbd593aaec8d0429aec7519ee
6
+ metadata.gz: bdb68283d5feca9b506a11478671e8deea6e425065ececefd2162a227b01a7be837fdd533fd5b51e58e60e3d438ffa8e89009f690e0b68696b06cc6e204f5c24
7
+ data.tar.gz: 99baadbbd02b889af06b4cd1b17847a61efe074d58c3f7cc6b0e3eb693f277604f5420b8924bca383cfc8cdbba550d8c16755231335e0a1e056663cb0a5f4deb
@@ -24,7 +24,7 @@ jobs:
24
24
  steps:
25
25
  - uses: actions/checkout@v2
26
26
  - name: Set up Ruby
27
- uses: actions/setup-ruby@v1
27
+ uses: ruby/setup-ruby@v1
28
28
  with:
29
29
  ruby-version: ${{ matrix.ruby }}
30
30
  - name: Install sqlite3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  # Change Log
2
2
 
3
+ # 0.6.0 (March 03, 2021)
4
+ - feat: add support to include custom payload headers
5
+ - feat: add pubsub transactions to process all payloads inside in the same order they were published
6
+ - feat: when a model is created/updated/destroyed, process all related payloads in a single transaction
7
+ - feat: add method to save processed payload (:ps_processed_payload) when saving sync
8
+ - feat: add "ordering_key" support to process all payloads with the same key in the same order
9
+ - feat: start multiple workers to process async kafka messages when starting service listeners
10
+ - feat: make async publisher by reusing exchange connection (rabbit)
11
+ - feat: add support for forced_ordering_key to always be used as the ordering_key if defined
12
+ - feat: add feature to publish a message to a custom and/or multiple topics
13
+ - feat: add model custom action subscriber and publisher
14
+ - feat: add docker compose settings
15
+
3
16
  # 0.5.10 (February 13, 2021)
4
17
  - feat: remove duplicated callback :ps_before_save_sync (same result can be achieved with :ps_before_save_sync)
5
18
  - feat: improve message starter to retry when failed or exit system when persists
data/Dockerfile ADDED
@@ -0,0 +1,6 @@
1
+ FROM ruby:2.7-buster
2
+ RUN apt-get update -qq
3
+ WORKDIR /myapp
4
+ COPY . /myapp
5
+ RUN gem update bundler
6
+ RUN bundle install
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- pub_sub_model_sync (0.5.10)
4
+ pub_sub_model_sync (0.6.0)
5
5
  rails
6
6
 
7
7
  GEM
@@ -243,6 +243,7 @@ DEPENDENCIES
243
243
  database_cleaner-active_record
244
244
  google-cloud-pubsub (> 2.0)
245
245
  pub_sub_model_sync!
246
+ rails (~> 6)
246
247
  rake
247
248
  rspec
248
249
  rubocop (~> 1.6.0)
data/README.md CHANGED
@@ -6,8 +6,9 @@ Note: This gem is based on [MultipleMan](https://github.com/influitive/multiple_
6
6
  - [**PubSubModelSync**](#pubsubmodelsync)
7
7
  - [**Features**](#features)
8
8
  - [**Installation**](#installation)
9
- - [**Usage**](#usage)
10
- - [**Examples**](#examples)
9
+ - [**Configuration**](#configuration)
10
+ - [**Notifications Diagram**](#notifications-diagram)
11
+ - [**Basic Example**](#basic-example)
11
12
  - [**Advanced Example**](#advanced-example)
12
13
  - [**API**](#api)
13
14
  - [**Subscribers**](#subscribers)
@@ -33,6 +34,10 @@ Note: This gem is based on [MultipleMan](https://github.com/influitive/multiple_
33
34
  - Ability to make class level communication
34
35
  Example: If User from App1 wants to generate_email, this can be listened on App2, App3, ... to make corresponding actions
35
36
  - Change pub/sub service at any time
37
+ - Support for transactions: Permits to group all payloads with the same ordering_key and be processed in the same order they are published by the subscribers.
38
+ Grouping by ordering_key allows us to enable multiple workers in our Pub/Sub service(s), and still guarantee that related payloads will be processed in the correct order, despite of the multiple threads.
39
+ This thanks to the fact that Pub/Sub services will always send messages with the same `ordering_key` into the same worker/thread.
40
+ - Ability to send notifications to a specific topic or multiple topics
36
41
 
37
42
  ## **Installation**
38
43
  Add this line to your application's Gemfile:
@@ -46,7 +51,7 @@ gem 'ruby-kafka' # to use apache kafka pub/sub service
46
51
  And then execute: $ bundle install
47
52
 
48
53
 
49
- ## **Usage**
54
+ ## **Configuration**
50
55
 
51
56
  - Configuration for google pub/sub (You need google pub/sub service account)
52
57
  ```ruby
@@ -54,7 +59,8 @@ And then execute: $ bundle install
54
59
  PubSubModelSync::Config.service_name = :google
55
60
  PubSubModelSync::Config.project = 'google-project-id'
56
61
  PubSubModelSync::Config.credentials = 'path-to-the-config'
57
- PubSubModelSync::Config.topic_name = 'sample-topic'
62
+ PubSubModelSync::Config.topic_name = 'sample-topic'
63
+ PubSubModelSync::Config.subscription_name = 'my-app1'
58
64
  ```
59
65
  See details here:
60
66
  https://github.com/googleapis/google-cloud-ruby/tree/master/google-cloud-pubsub
@@ -63,8 +69,8 @@ And then execute: $ bundle install
63
69
  ```ruby
64
70
  PubSubModelSync::Config.service_name = :rabbitmq
65
71
  PubSubModelSync::Config.bunny_connection = 'amqp://guest:guest@localhost'
66
- PubSubModelSync::Config.queue_name = 'model-sync'
67
72
  PubSubModelSync::Config.topic_name = 'sample-topic'
73
+ PubSubModelSync::Config.subscription_name = 'my-app2'
68
74
  ```
69
75
  See details here: https://github.com/ruby-amqp/bunny
70
76
 
@@ -73,27 +79,25 @@ And then execute: $ bundle install
73
79
  PubSubModelSync::Config.service_name = :kafka
74
80
  PubSubModelSync::Config.kafka_connection = [["kafka1:9092", "localhost:2121"], { logger: Rails.logger }]
75
81
  PubSubModelSync::Config.topic_name = 'sample-topic'
82
+ PubSubModelSync::Config.subscription_name = 'my-app3'
76
83
  ```
77
84
  See details here: https://github.com/zendesk/ruby-kafka
78
85
 
79
86
  - Add publishers/subscribers to your models (See examples below)
80
87
 
81
88
  - Start subscribers to listen for publishers (Only in the app that has subscribers)
82
- ```ruby
83
- rake pub_sub_model_sync:start
84
- ```
85
- Note: Publishers do not need todo this
86
- Note2 (Rails 6+): Due to Zeitwerk, you need to load listeners manually when syncing without mentioned task (like rails console)
87
- ```ruby
88
- # PubSubModelSync::Config.subscribers ==> []
89
- PubSubModelSync::Runner.preload_listeners
90
- # PubSubModelSync::Config.subscribers ==> [#<PubSubModelSync::Subscriber:0x000.. @klass="Article", @action=:create..., ....]
89
+ ```bash
90
+ DB_POOL=20 bundle exec rake pub_sub_model_sync:start
91
91
  ```
92
+ Note: You need more than 15 DB pools to avoid "could not obtain a connection from the pool within 5.000 seconds". https://devcenter.heroku.com/articles/concurrency-and-database-connections
92
93
 
93
94
  - Check the service status with:
94
95
  ```PubSubModelSync::MessagePublisher.publish_data('Test message', {sample_value: 10}, :create)```
95
96
 
96
- ## **Examples**
97
+ ## **Notifications Diagram**
98
+ ![Diagram](/docs/notifications-diagram.png?raw=true)
99
+
100
+ ## **Basic Example**
97
101
  ```ruby
98
102
  # App 1 (Publisher)
99
103
  # attributes: name email age
@@ -105,20 +109,25 @@ end
105
109
  # App 2 (Subscriber)
106
110
  class User < ActiveRecord::Base
107
111
  include PubSubModelSync::SubscriberConcern
108
- ps_subscribe(%i[name])
109
- ps_class_subscribe(:greeting)
112
+ ps_subscribe(%i[name]) # crud notifications
113
+ ps_subscribe_custom(:say_welcome) # custom instance notification
114
+ ps_class_subscribe(:greeting) # class notification
110
115
 
111
116
  def self.greeting(data)
112
117
  puts 'Class message called'
113
118
  end
119
+
120
+ def say_welcome(data)
121
+ UserMailer.deliver(id, data)
122
+ end
114
123
  end
115
124
 
116
125
  # Samples
117
126
  User.create(name: 'test user', email: 'sample@gmail.com') # Review your App 2 to see the created user (only name will be saved)
118
127
  User.new(name: 'test user').ps_perform_sync(:create) # similar to above to perform sync on demand
119
128
 
120
- User.ps_class_publish({ msg: 'Hello' }, action: :greeting) # User.greeting method (Class method) will be called in App2
121
- PubSubModelSync::MessagePublisher.publish_data(User, { msg: 'Hello' }, :greeting) # similar to above when not included publisher concern
129
+ PubSubModelSync::MessagePublisher.publish_model_data(my_user, { id:10, msg: 'Hello' }, :say_welcome, { as_klass: 'RegisteredUser' }) # custom model action notification
130
+ PubSubModelSync::MessagePublisher.publish_data(User, { msg: 'Hello' }, :greeting) # custom data notification
122
131
  ```
123
132
 
124
133
  ## **Advanced Example**
@@ -127,7 +136,7 @@ PubSubModelSync::MessagePublisher.publish_data(User, { msg: 'Hello' }, :greeting
127
136
  class User < ActiveRecord::Base
128
137
  self.table_name = 'publisher_users'
129
138
  include PubSubModelSync::PublisherConcern
130
- ps_publish(%i[id:client_id name:full_name email], actions: %i[update], as_klass: 'Client')
139
+ ps_publish(%i[id:client_id name:full_name email], actions: %i[update], as_klass: 'Client', headers: { topic_name: ['topic1', 'topic N'] })
131
140
 
132
141
  def ps_skip_callback?(_action)
133
142
  false # here logic with action to skip push message
@@ -144,11 +153,16 @@ class User < ActiveRecord::Base
144
153
  include PubSubModelSync::SubscriberConcern
145
154
  ps_subscribe(%i[name], actions: %i[update], from_klass: 'Client', id: %i[client_id email])
146
155
  ps_class_subscribe(:greeting, from_action: :custom_greeting, from_klass: 'CustomUser')
156
+ ps_subscribe_custom(:send_welcome, from_klass: 'CustomUser', id: :id, from_action: :say_welcome)
147
157
  alias_attribute :full_name, :name
148
158
 
149
159
  def self.greeting(data)
150
160
  puts 'Class message called through custom_greeting'
151
161
  end
162
+
163
+ def send_welcome(data)
164
+ UserMailer.deliver(id, data)
165
+ end
152
166
 
153
167
  # def self.ps_find_model(data)
154
168
  # where(email: data[:email], ...).first_or_initialize
@@ -156,136 +170,207 @@ class User < ActiveRecord::Base
156
170
  end
157
171
  ```
158
172
 
159
- Note: Be careful with collision of names
160
- ```
161
- # ps_publish %i[name_data:name name:key] # key will be replaced with name_data
162
- ps_publish %i[name_data:name key_data:key] # use alias to avoid collision
163
- ```
164
-
165
173
  ## **API**
166
174
  ### **Subscribers**
167
175
 
168
- #### **Registering Subscription Callbacks**
176
+ #### **Registering Subscriptions**
169
177
 
170
- - Configure model-level subscriptions
178
+ - Configure class subscriptions
171
179
  ```ruby
172
180
  class MyModel < ActiveRecord::Base
173
181
  ps_class_subscribe(action_name, from_action: nil, from_klass: nil)
174
182
  end
175
183
  ```
176
- * `from_action`: (Optional) Source method name
177
- * `from_klass`: (Optional) Source class name
184
+ When Class receives the corresponding notification, `action` method will be called on the Class. Like: `User.action(data)`
185
+ * `action_name`: (String|Sym/Optional) Action name
186
+ * `from_klass`: (String/Optional) Source class name (Default `model.class.name`)
187
+ * `from_action`: (Sym/Optional) Source method name. Default `action`
178
188
 
179
- - Configure instance-level subscriptions (CRUD)
189
+ - Configure CRUD subscriptions
180
190
  ```ruby
181
191
  class MyModel < ActiveRecord::Base
182
192
  ps_subscribe(attrs, from_klass: nil, actions: nil, id: nil)
183
193
  end
184
194
  ```
195
+ When model receives the corresponding notification, `action` method will be called on the model. Like: `model.destroy`
185
196
  * `attrs`: (Array/Required) Array of all attributes to be synced
186
- * `from_klass`: (String/Optional) Source class name (Instead of the model class name, will use this value)
197
+ * `from_klass`: (String/Optional) Source class name (Default `model.class.name`)
187
198
  * `actions`: (Array/Optional, default: create/update/destroy) permit to customize action names
188
199
  * `id`: (Sym|Array/Optional, default: id) Attr identifier(s) to find the corresponding model
189
200
 
190
- - Configure a custom model finder
201
+ - Configure custom model subscriptions
191
202
  ```ruby
192
203
  class MyModel < ActiveRecord::Base
193
- ps_find_model(data)
204
+ ps_subscribe_custom(action, from_klass: name, id: :id, from_action: nil)
194
205
  end
195
206
  ```
196
- * `data`: (Hash) Data received from sync
197
- Must return an existent or a new model object
207
+ When model receives the corresponding notification, `action` method will be called on the model. Like: `model.action(data)`
208
+ * `action`: (String/Required) Action name
209
+ * `from_klass`: (String/Optional) Source class name (Default `model.class.name`)
210
+ * `from_action`: (Sym/Optional) Source method name. Default `action`
211
+ * `id`: (Sym|Array/Optional, default: id) Attr identifier(s) to find the corresponding model
198
212
 
199
- #### **Class Methods**
200
- - Configure CRUD subscription for the class
213
+ - Perform custom actions before saving sync of the model (`:cancel` can be returned to skip sync)
201
214
  ```ruby
202
- MyModel.ps_subscriber(action_name)
215
+ class MyModel < ActiveRecord::Base
216
+ def ps_before_save_sync(action, payload)
217
+ # puts payload.data[:id]
218
+ end
219
+ end
203
220
  ```
204
- * `action_name` (default :create, :sym): can be :create, :update, :destroy
221
+
222
+ - Configure a custom model finder (optional)
223
+ ```ruby
224
+ class MyModel < ActiveRecord::Base
225
+ def ps_find_model(data)
226
+ where(custom_finder: data[:custom_value]).first_or_initialize
227
+ end
228
+ end
229
+ ```
230
+ * `data`: (Hash) Data received from sync
231
+ Must return an existent or a new model object
205
232
 
206
- - Inspect all configured subscribers
233
+ #### **Subscription helpers**
234
+ - Inspect all configured subscriptions
207
235
  ```ruby
208
236
  PubSubModelSync::Config.subscribers
209
237
  ```
210
-
211
- #### **Instance Methods**
212
-
213
- - Perform custom actions before saving sync of the model (On-demand, `:cancel` can be returned to skip sync)
238
+ - Manually process or reprocess a notification
214
239
  ```ruby
215
- my_instance.ps_before_save_sync(payload)
240
+ payload = PubSubModelSync::Payload.new(data, attributes, headers)
241
+ payload.process!
216
242
  ```
217
243
 
244
+
218
245
  ### **Publishers**
219
246
 
220
- #### **Registering Publishing Callbacks**
221
- - You can register Model-level lifecycle callbacks (CRUD) that will trigger publishing events like this:
247
+ #### **Registering Publishers **
248
+ - Register CRUD publishers that will trigger configured notifications
222
249
  ```ruby
223
- ps_publish(attrs, actions: nil, as_klass: nil)
250
+ class MyModel < ActiveRecord::Base
251
+ ps_publish([:id, 'created_at:published_at', :full_name], actions: [:update], as_klass: nil, headers: { ordering_key: 'custom-key', topic_name: 'my-custom-topic' })
252
+ def full_name
253
+ [first_name, last_name].join(' ')
254
+ end
255
+ end
224
256
  ```
225
- * `attrs`: (Array/Required) Array of attributes to be published
226
- * `actions`: (Array/Optional, default: create/update/destroy) permit to customize action names
257
+ * `attrs`: (Array/Required) Array of attributes to be published. Supports for:
258
+ - aliases: permits to publish with different names, sample: "created_at:published_at" where "created_at" will be published as "published_at"
259
+ - methods: permits to publish method values as attributes, sample: "full_name"
260
+ * `actions`: (Array/Optional, default: %i[create update destroy]) permit to define action names
227
261
  * `as_klass`: (String/Optional) Output class name (Instead of the model class name, will use this value)
262
+ * `headers`: (Hash/Optional) Notification settings which permit to customize the way and the target of the notification (Refer Payload.headers)
263
+
228
264
 
265
+ #### **Publishing notifications**
266
+ - CRUD notifications
267
+ ```ruby
268
+ MyModel.create!(...)
269
+ ```
270
+ "Create" notification will be delivered with the configured attributes as the payload data
229
271
 
230
- #### **Instance Methods**
272
+ - Manual CRUD notifications
273
+ ```ruby
274
+ MyModel.ps_perform_sync(action, custom_data: {}, custom_headers: {})
275
+ ```
276
+ * `action`: (Sym) CRUD action name (create, update or destroy)
277
+ * `custom_data`: custom_data (nil|Hash) If present custom_data will be used as the payload data. I.E. data generator will be ignored
278
+ * `custom_headers`: (Hash, optional) override default headers. Refer `payload.headers`
279
+
280
+ - Class notifications
281
+ ```ruby
282
+ PubSubModelSync::MessagePublisher.publish_data((klass, data, action, headers: )
283
+ ```
284
+ Publishes any data to be listened at a class level.
285
+ - `klass`: (String) Class name to be used
286
+ - `data`: (Hash) Data to be delivered
287
+ - `action`: (Sym) Action name
288
+ - `headers`: (Hash, optional) Notification settings (Refer Payload.headers)
289
+
290
+ - Model custom action notifications
291
+ ```ruby
292
+ PubSubModelSync::MessagePublisher.publish_model_data(model, data, action, as_klass:, headers:)
293
+ ```
294
+ Publishes model custom action to be listened at an instance level.
295
+ - `model`: (ActiveRecord) model owner of the data
296
+ - `data`: (Hash) Data to be delivered
297
+ - `action`: (Sym) Action name
298
+ - `as_klass`: (String, optional) if not provided, `model.class.name` will be used instead
299
+ - `headers`: (Hash, optional) Notification settings (Refer Payload.headers)
300
+
301
+ - Manually publish or republish a notification
302
+ ```ruby
303
+ payload = PubSubModelSync::Payload.new(data, attributes, headers)
304
+ payload.publish!
305
+ ```
306
+
307
+ #### ** publishing callbacks**
231
308
 
232
- - **Prevent PS-related callback** (On-demand, before the callback gets triggered)
309
+ - Prevent CRUD sync at model callback level (Called right after :after_create, :after_update, :after_destroy).
310
+ If returns "true", sync will be cancelled.
233
311
  ```ruby
234
- model_instance.ps_skip_callback?(action)
312
+ class MyModel < ActiveRecord::Base
313
+ def ps_skip_callback?(action)
314
+ # logic here
315
+ end
316
+ end
235
317
  ```
236
- Default: False
237
- Note: Return true to cancel sync
238
318
 
239
- - **Prevent sync after create/update/destroy action** (On-demand, before the sync gets triggered)
319
+ - Prevent CRUD sync before processing payload (Affects model.ps_perform_sync(...))).
320
+ If returns "true", sync will be cancelled
240
321
  ```ruby
241
- model_instance.ps_skip_sync?(action)
322
+ class MyModel < ActiveRecord::Base
323
+ def ps_skip_sync?(action)
324
+ # logic here
325
+ end
326
+ end
242
327
  ```
243
- Note: return true to cancel sync
244
328
 
245
- - **Execute a callback before sync** (On-demand, before sync is executed, but after payload is received )
329
+ - Do some actions before publishing a CRUD notification.
330
+ If returns ":cancel", sync will be cancelled
246
331
  ```ruby
247
- model_instance.ps_before_sync(action, data_to_deliver)
332
+ class MyModel < ActiveRecord::Base
333
+ def ps_before_sync(action, payload)
334
+ # logic here
335
+ end
336
+ end
248
337
  ```
249
- Note: If the method returns ```:cancel```, the sync will be stopped (message will not be published)
250
338
 
251
- - **Execute a callback after sync**
339
+ - Do some actions after CRUD notification was published.
252
340
  ```ruby
253
- model_instance.ps_after_sync(action, data_delivered)
341
+ class MyModel < ActiveRecord::Base
342
+ def ps_after_sync(action, payload)
343
+ # logic here
344
+ end
345
+ end
254
346
  ```
255
347
 
256
- - **Trigger a sync on-demand** (:create, :update, :destroy):
257
- The target model will receive a notification to perform the indicated action
258
- ```ruby
259
- model_instance.ps_perform_sync(action_name, custom_settings = {})
260
- ```
261
- * `custom_settings`: override default settings defined for action_name ({ attrs: [], as_klass: nil })
262
348
 
263
- #### **Class Methods**
349
+ ### **Payload**
350
+ Any notification before delivering is transformed as a Payload for a better portability.
264
351
 
265
- - **Publish a class level notification**:
352
+ - Initialize
266
353
  ```ruby
267
- User.ps_class_publish(data, action: action_name, as_klass: custom_klass_name)
354
+ payload = PubSubModelSync::Payload.new(data, attributes, headers)
268
355
  ```
269
- Target class ```User.action_name``` will be called when message is received
270
- * `data`: (required, :hash) message value to deliver
271
- * `action_name`: (required, :sim) Action name
272
- * `as_klass`: (optional, :string) Custom class name (Default current model name)
273
-
274
- #### **Payload actions**
356
+ * `data`: (Hash) Data to be published or processed
357
+ * `attributes`: (Hash) Includes class and method info
358
+ - `action`: (String) action name
359
+ - `klass`: (String) class name
360
+ * `headers`: (Hash) Notification settings that defines how the notification will be processed or delivered.
361
+ - `key`: (String, optional) identifier of the payload, default: `<klass_name>/<action>` when class message, `<model.class.name>/<action>/<model.id>` when model message (Useful for caching techniques).
362
+ - `ordering_key`: (String, optional): messages with the same key are processed in the same order they were delivered, default: `klass_name` when class message, `<model.class.name>/<model.id>` when model message
363
+ - `topic_name`: (String|Array<String>, optional): Specific topic name to be used when delivering the message (default first topic from config).
364
+ - `forced_ordering_key`: (String, optional): Will force to use this value as the `ordering_key`, even withing transactions. Default `nil`.
365
+
366
+ - Actions for payloads
275
367
  ```ruby
276
- payload = PubSubModelSync::Payload.new({ title: 'hello' }, { action: :greeting, klass: 'User' })
277
368
  payload.publish! # publishes notification data. It raises exception if fails and does not call ```:on_error_publishing``` callback
278
369
  payload.publish # publishes notification data. On error does not raise exception but calls ```:on_error_publishing``` callback
279
370
  payload.process! # process a notification data. It raises exception if fails and does not call ```.on_error_processing``` callback
280
371
  payload.publish # process a notification data. It does not raise exception if fails but calls ```.on_error_processing``` callback
281
372
  ```
282
373
 
283
- - Get crud publisher configured for the class
284
- ```ruby
285
- User.ps_publisher(action_name)
286
- ```
287
- * `action_name` (default :create, :sym): can be :create, :update, :destroy
288
-
289
374
  ## **Testing with RSpec**
290
375
  - Config: (spec/rails_helper.rb)
291
376
  ```ruby
@@ -341,7 +426,7 @@ Note: Be careful with collision of names
341
426
  publisher = PubSubModelSync::MessagePublisher
342
427
  data = {msg: 'hello'}
343
428
  action = :greeting
344
- User.ps_class_publish(data, action: action)
429
+ PubSubModelSync::MessagePublisher.publish_data('User', data, action)
345
430
  expect(publisher).to receive(:publish_data).with('User', data, action)
346
431
  end
347
432
  ```
@@ -351,9 +436,12 @@ Note: Be careful with collision of names
351
436
  config = PubSubModelSync::Config
352
437
  config.debug = true
353
438
  ```
354
-
355
- - ```.subscription_name = 'app-2'```
356
- Permit to define a custom consumer identifier (Default: Rails application name)
439
+ - `.topic_name = ['topic1', 'topic 2']`: (String|Array<String>)
440
+ Topic name(s) to be used to listen all notifications from when listening. Additional first topic name is used as the default topic name when publishing a notification.
441
+ - `.subscription_name = "my-app-1"`: (String, default Rails.application.name)
442
+ Subscriber's identifier which helps to:
443
+ * skip self messages
444
+ * continue the sync from the last synced notification when service was restarted.
357
445
  - ```.debug = true```
358
446
  (true/false*) => show advanced log messages
359
447
  - ```.logger = Rails.logger```
@@ -378,18 +466,15 @@ config.debug = true
378
466
  - Add flag ```model.ps_process_payload``` to retrieve the payload used to process the pub/sub sync
379
467
  - Auto publish update only if payload has changed
380
468
  - On delete, payload must only be composed by ids
381
- - Feature to publish multiple message at a time with the ability to exclude similar messages by klass and action (use the last one)
382
- PubSubModelSync::MessagePublisher.batch_publish({ same_keys: :use_last_as_first|:use_last|:use_first_as_last|:keep*, same_data: :use_last_as_first*|:use_last|:use_first_as_last|:keep })
383
- - Add DB table to use as a shield to skip publishing similar notifications or publish partial notifications (similar idea when processing notif)
469
+ - Improve transactions to exclude similar messages by klass and action. Sample:
470
+ ```PubSubModelSync::MessagePublisher.transaction(key, { same_keys: :use_last_as_first|:use_last|:use_first_as_last|:keep*, same_data: :use_last_as_first*|:use_last|:use_first_as_last|:keep })```
471
+ - Add DB table to use as a shield to prevent publishing similar notifications and publish partial notifications (similar idea when processing notif)
384
472
  - add callback: on_message_received(payload)
385
473
 
386
474
  ## **Q&A**
387
475
  - I'm getting error "could not obtain a connection from the pool within 5.000 seconds"... what does this mean?
388
476
  This problem occurs because pub/sub dependencies (kafka, google-pubsub, rabbitmq) use many threads to perform notifications where the qty of threads is greater than qty of DB pools ([Google pubsub info](https://github.com/googleapis/google-cloud-ruby/blob/master/google-cloud-pubsub/lib/google/cloud/pubsub/subscription.rb#L888))
389
- To fix the problem, edit config/database.yml and increase the quantity of ```pool: 10```
390
- - Google pubsub: How to process notifications parallely and not sequentially (default 1 thread)?
391
- ```ruby PubSubModelSync::ServiceGoogle::LISTEN_SETTINGS = { threads: { callback: qty_threads } } ```
392
- Note: by this way some notifications can be processed before others thus missing relationship errors can appear
477
+ To fix the problem, edit config/database.yml and increase the quantity of ```pool: 20```
393
478
  - How to retry failed syncs with sidekiq?
394
479
  ```ruby
395
480
  # lib/initializers/pub_sub_config.rb