cyclone_lariat 0.3.10 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,85 +1,142 @@
1
1
  # Cyclone lariat
2
2
 
3
- This is gem work like middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It save all events to database. And catch and produce all exceptions.
3
+ This gem work in few scenarios:
4
+ - As middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It saves all events to the database and also catches and throws all exceptions.
5
+ - As a middleware, it can log all incoming messages.
6
+ - As a client that can send messages to SNS topics and SQS queues.
7
+ - Also it can help you with CI\CD for theme, queue and subscription management like database migration.
4
8
 
5
9
  ![Cyclone lariat](docs/_imgs/lariat.jpg)
6
10
 
7
11
 
12
+ # Client / Publisher
13
+ At first lets understand what the difference between SQS and SNS:
14
+ - Amazon Simple Queue Service (SQS) lets you send, store, and receive messages between software components at any
15
+ volume, without losing messages or requiring other services to be available.
16
+ - Amazon Simple Notification Service (SNS) sends notifications two ways Application2Person (like send sms).
17
+ And the second way is Application2Application, that's way more important for us. In this way you case use
18
+ SNS service like fanout.
19
+
20
+ ![SQS/SNS](docs/_imgs/sqs_sns_diagram.png)
21
+
22
+ For use **cyclone_lariat** as _Publisher_ lets make install CycloneLariat.
23
+
24
+ ## Install cyclone_lariat
25
+ Edit Gemfile:
8
26
  ```ruby
9
27
  # Gemfile
28
+ gem 'sequel'
29
+ gem 'cyclone_lariat'
30
+ ```
31
+ And run in console:
32
+ ```bash
33
+ $ bundle install
34
+ $ cyclone_lariat install
35
+ ```
10
36
 
11
- # If use client or middleware
12
- gem 'cyclone_lariat', require: false
37
+ Last command will create 2 files:
38
+ - ./lib/tasks/cyclone_lariat.rake - Rake tasks, for management migrations
39
+ - ./config/initializers/cyclone_lariat.rb - Configuration default values for cyclone lariat usage
13
40
 
14
- # If use client
15
- gem 'cyclone_lariat'
41
+ ## Configuration
42
+ ```ruby
43
+ # frozen_string_literal: true
44
+
45
+ CycloneLariat.tap do |cl|
46
+ cl.default_version = 1 # api version
47
+ cl.aws_key = ENV['AWS_KEY'] # aws key
48
+ cl.aws_account_id = ENV['AWS_ACCOUNT_ID'] # aws account id
49
+ cl.aws_secret_key = ENV['AWS_SECRET_KEY'] # aws secret
50
+ cl.aws_default_region = ENV['AWS_REGION'] # aws default region
51
+ cl.publisher = ENV['APP_NAME'] # name of your publishers, usually name of your application
52
+ cl.default_instance = ENV['INSTANCE'] # stage, production, test
53
+ cl.events_dataset = DB[:events] # sequel dataset for store income messages, for receiver
54
+ cl.versions_dataset = DB[:lariat_versions] # sequel dataset for migrations, for publisher
55
+ end
56
+ ```
57
+ If you are only using your application as a publisher, you may not need to set the `events_dataset`
58
+ parameter.
59
+
60
+ Before creating the first migration, let's explain what `CycloneLariat::Message` is.
61
+
62
+ ## Messages
63
+ Message in Amazon SQS\SNS service it's a
64
+ [object](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-attributes)
65
+ that has several attributes. The main attributes are the **body**, which consists of the published
66
+ data. The body is a _String_, but we can use it as a _JSON_ object. **Cyclone_lariat** use this scheme:
67
+
68
+ ```json
69
+ {
70
+ "uuid": "f2ce3813-0905-4d81-a60e-f289f2431f50", // Uniq message identificator
71
+ "publisher": "sample_app", // Publisher application name
72
+ "request_id": "51285005-8a06-4181-b5fd-bf29f3b1a45a", // Optional: X-Request-Id
73
+ "type": "event_note_created", // Type of Event or Command
74
+ "version": 1, // Version of data structure
75
+ "data": {
76
+ "id": 12,
77
+ "text": "Sample of published data",
78
+ "attributes": ["one", "two", "three"]
79
+ },
80
+ "sent_at": "2022-11-09T11:42:18.203+01:00" // Time when message was sended in ISO8601 Standard
81
+ }
16
82
  ```
17
83
 
18
- ## Logic
84
+ Idea about X-Request-Id you can see at
85
+ [StackOverflow](https://stackoverflow.com/questions/25433258/what-is-the-x-request-id-http-header).
86
+
87
+ As you see, type has prefix 'event_' in cyclone lariat you has two kinds of messages - _Event_ and _Command_.
19
88
 
20
- ![diagram](docs/_imgs/diagram.png)
89
+ ### Command vs Event
90
+ Commands and events are both simple domain structures that contain solely data for reading. That means
91
+ they contain no behaviour or business logic.
21
92
 
22
- ## Command vs Event
23
- Commands and events are both simple domain structures that contain solely data for reading. That means they contain no
24
- behaviour or business logic.
93
+ A command is an object that is sent to the domain for a state change which is handled by a command
94
+ handler. They should be named with a verb in an imperative mood plus the aggregate name which it
95
+ operates on. Such request can be rejected due to the data the command holds being invalid/inconsistent.
96
+ There should be exactly 1 handler for each command. Once the command has been executed, the consumer
97
+ can then carry out whatever the task is depending on the output of the command.
25
98
 
26
- A command is an object that is sent to the domain for a state change which is handled by a command handler. They should
27
- be named with a verb in an imperative mood plus the aggregate name which it operates on. Such request can be rejected
28
- due to the data the command holds being invalid/inconsistent. There should be exactly 1 handler for each command.
29
- Once the command has been executed, the consumer can then carry out whatever the task is depending on the output of the
30
- command.
99
+ An event is a statement of fact about what change has been made to the domain state. They are named
100
+ with the aggregate name where the change took place plus the verb past-participle. An event happens off
101
+ the back of a command.
102
+ A command can emit any number of events. The sender of the event does not care who receives it or
103
+ whether it has been received at all.
31
104
 
32
- An event is a statement of fact about what change has been made to the domain state. They are named with the aggregate
33
- name where the change took place plus the verb past-participle. An event happens off the back of a command.
34
- A command can emit any number of events. The sender of the event does not care who receives it or whether it has been
35
- received at all.
105
+ ### Publish
106
+ For publishing _Event_ or _Commands_, you have two ways, send _Message_ directly:
36
107
 
37
- ## Configure
38
108
  ```ruby
39
- # 'config/initializers/cyclone_lariat.rb'
40
109
  CycloneLariat.tap do |cl|
41
- cl.default_version = 1 # api version default is 1
42
- cl.aws_key = # aws key
43
- cl.aws_secret_key = # aws secret
44
- cl.aws_client_id = # aws client id
45
- cl.aws_default_region = # aws default region
46
- cl.publisher = 'auth' # name of your publishers, usually name of your application
47
- cl.default_instance = APP_INSTANCE # stage, production, test
110
+ # Config app here
48
111
  end
49
- ```
50
112
 
51
- ## SnsClient
52
- You can use client directly
113
+ client = CycloneLariat::SnsClient.new(instance: 'auth', version: 2)
53
114
 
54
- ```ruby
55
- require 'cyclone_lariat/sns_client' # If require: false in Gemfile
56
-
57
- client = CycloneLariat::SnsClient.new(
58
- key: APP_CONF.aws.key,
59
- secret_key: APP_CONF.aws.secret_key,
60
- region: APP_CONF.aws.region,
61
- version: 1, # at default 1
62
- publisher: 'pilot',
63
- instance: INSTANCE # at default :prod
115
+ client.publish_command('register_user', data: {
116
+ first_name: 'John',
117
+ last_name: 'Doe',
118
+ mail: 'john.doe@example.com'
119
+ }, fifo: false
64
120
  )
65
121
  ```
66
122
 
67
- You can don't define topic, and it's name will be defined automatically
68
- ```ruby
69
- # event_type data topic
70
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
71
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
72
- client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' } # prod-command-fanout-pilot-delete_user
73
- ```
74
- Or you can define it by handle. For example, if you want to send different events to same channel.
75
- ```ruby
76
- # event_type data topic
77
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
78
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
79
- client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' }, topic: 'prod-command-fanout-pilot-emails'
123
+ That's call, will generate a message body:
124
+ ```json
125
+ {
126
+ "uuid": "f2ce3813-0905-4d81-a60e-f289f2431f50",
127
+ "publisher": "auth",
128
+ "type": "command_register_user",
129
+ "version": 2,
130
+ "data": {
131
+ "first_name": "John",
132
+ "last_name": "Doe",
133
+ "mail": "john.doe@example.com"
134
+ },
135
+ "sent_at": "2022-11-09T11:42:18.203+01:00" // Time when message was sended in ISO8601 Standard
136
+ }
80
137
  ```
81
138
 
82
- Or you can use client as Repo.
139
+ Or is it better to make your own client, like a [Repository](https://deviq.com/design-patterns/repository-pattern) pattern.
83
140
 
84
141
  ```ruby
85
142
  require 'cyclone_lariat/sns_client' # If require: false in Gemfile
@@ -90,30 +147,21 @@ class YourClient < CycloneLariat::SnsClient
90
147
  instance 'stage'
91
148
 
92
149
  def email_is_created(mail)
93
- publish event('email_is_created',
94
- data: { mail: mail }
95
- ),
96
- to: APP_CONF.aws.fanout.emails
150
+ publish event('email_is_created', data: { mail: mail }), fifo: true
97
151
  end
98
152
 
99
153
  def email_is_removed(mail)
100
- publish event('email_is_removed',
101
- data: { mail: mail }
102
- ),
103
- to: APP_CONF.aws.fanout.email
154
+ publish event('email_is_removed', data: { mail: mail }), fifo: true
104
155
  end
105
156
 
106
157
 
107
158
  def delete_user(mail)
108
- publish command('delete_user',
109
- data: { mail: mail }
110
- ),
111
- to: APP_CONF.aws.fanout.email
159
+ publish command('delete_user', data: { mail: mail }), fifo: false
112
160
  end
113
161
  end
114
162
 
115
163
  # Init repo
116
- client = YourClient.new(key: APP_CONF.aws.key, secret_key: APP_CONF.aws.secret_key, region: APP_CONF.aws.region)
164
+ client = YourClient.new
117
165
 
118
166
  # And send topics
119
167
  client.email_is_created 'john.doe@example.com'
@@ -121,76 +169,394 @@ client.email_is_removed 'john.doe@example.com'
121
169
  client.delete_user 'john.doe@example.com'
122
170
  ```
123
171
 
172
+ ### Topics and Queue
173
+ An Amazon SNS topic and SQS queue is a logical access point that acts as a communication channel. Both
174
+ of them has specific address ARN.
175
+
176
+ ```
177
+ # Topic example
178
+ arn:aws:sns:eu-west-1:247602342345:test-event-fanout-cyclone_lariat-note_added.fifo
179
+
180
+ # Queue example
181
+ arn:aws:sqs:eu-west-1:247602342345:test-event-queue-cyclone_lariat-note_added-notifier.fifo
182
+ ```
183
+
184
+ Split ARN:
185
+ - `arn:aws:sns` - Prefix for SNS Topics
186
+ - `arn:aws:sqs` - Prefix for SQS Queues
187
+ - `eu-west-1` -
188
+ [AWS Region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions)
189
+ - `247602342345` - [AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html)
190
+ - `test-event-fanout-cyclone_lariat-note_added` - Topic \ Queue name
191
+ - `.fifo` - if Topic or queue is
192
+ [FIFO](https://aws.amazon.com/blogs/aws/introducing-amazon-sns-fifo-first-in-first-out-pub-sub-messaging/), they must
193
+ has that suffix.
194
+
195
+ Region and client_id usually set using the **cyclone_lariat** [configuration](#Configuration).
196
+
197
+ ## Declaration for topic and queues name
198
+ In **cyclone_lariat** we have a declaration for defining topic and queue names.
199
+ This can help in organizing the order.
200
+
201
+ ```ruby
202
+ CycloneLariat.tap do |cfg|
203
+ cfg.instance = 'test'
204
+ cfg.publisher = 'cyclone_lariat'
205
+ # ...
206
+ end
207
+
208
+ CycloneLariat::SnsClient.new.publish_command('register_user', data: {
209
+ first_name: 'John',
210
+ last_name: 'Doe',
211
+ mail: 'john.doe@example.com'
212
+ }, fifo: true
213
+ )
214
+
215
+ # or in repository-like style:
216
+
217
+ class YourClient < CycloneLariat::SnsClient
218
+ def register_user(first:, last:, mail:)
219
+ publish command('register_user', data: { mail: mail }), fifo: true
220
+ end
221
+ end
222
+ ```
223
+
224
+
225
+ We will publish a message on this topic: `test-command-fanout-cyclone_lariat-register_user`.
124
226
 
125
- # SqsClient
126
- SqsClient is really similar to SnsClient. It can be initialized in same way:
227
+ Let's split the topic title:
228
+ - `test` - instance;
229
+ - `command` - kind - [event or command](#command-vs-event);
230
+ - `fanount` - resource type - fanout for SNS topics;
231
+ - `cyclone_lariat` - publisher name;
232
+ - `regiser_user` - message type.
127
233
 
234
+ For queues you also can define destination.
128
235
  ```ruby
129
- require 'cyclone_lariat/sns_client' # If require: false in Gemfile
236
+ CycloneLariat::SqsClient.new.publish_event(
237
+ 'register_user', data: { mail: 'john.doe@example.com' },
238
+ dest: :mailer, fifo: true
239
+ )
240
+
241
+
242
+ # or in repository-like style:
243
+
244
+ class YourClient < CycloneLariat::SnsClient
245
+ # ...
246
+
247
+ def register_user(first:, last:, mail:)
248
+ publish event('register_user', data: { mail: mail }), fifo: true
249
+ end
250
+ end
251
+ ```
252
+
253
+ We will publish a message on this queue: `test-event-queue-cyclone_lariat-register_user-mailer`.
130
254
 
131
- client = CycloneLariat::SqsClient.new(
132
- key: APP_CONF.aws.key,
133
- secret_key: APP_CONF.aws.secret_key,
134
- region: APP_CONF.aws.region,
135
- version: 1, # at default 1
136
- publisher: 'pilot',
137
- instance: INSTANCE # at default :prod
255
+ Let's split the queue title:
256
+ - `test` - instance;
257
+ - `event` - kind - [event or command](#command-vs-event);
258
+ - `queue` - resource type - queue for SQS;
259
+ - `cyclone_lariat` - publisher name;
260
+ - `regiser_user` - message type.
261
+ - `mailer` - destination
262
+
263
+ You also can sent message to queue with custom name. But this way does not recommended.
264
+
265
+ ```ruby
266
+ # Directly
267
+ CycloneLariat::SqsClient.new.publish_event(
268
+ 'register_user', data: { mail: 'john.doe@example.com' },
269
+ dest: :mailer, topic: 'custom_topic_name.fifo', fifo: true
138
270
  )
271
+
272
+ # Repository
273
+ class YourClient < CycloneLariat::SnsClient
274
+ # ...
275
+
276
+ def register_user(first:, last:, mail:)
277
+ publish event('register_user', data: { mail: mail }),
278
+ topic: 'custom_topic_name.fifo', fifo: true
279
+ end
280
+ end
139
281
  ```
282
+ Will publish message on queue: `custom_topic` with fifo suffix.
140
283
 
141
- As you see all params identity. And you can easily change your sqs-queue to sns-topic when you start work with more
142
- subscribes. But you should define destination.
284
+ # Migrations
285
+
286
+ With **cyclone_lariat** you can use migrations that can create, delete, and subscribe to your queues and topics, just like database migrations do.
287
+ To store versions of **cyclone_lariat** migrations, you need to create a database table.
143
288
 
144
289
  ```ruby
145
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, dest: 'notify_service'
146
- # prod-event-queue-pilot-email_is_created-notify_service
290
+ # frozen_string_literal: true
291
+
292
+ Sequel.migration do
293
+ change do
294
+ create_table :lariat_versions do
295
+ Integer :version, null: false, unique: true
296
+ end
297
+ end
298
+ end
147
299
  ```
300
+ After migrate this database migration, create **cyclone_lariat** migration.
301
+
302
+ ```bash
303
+ $ cyclone_lariat generate migration user_created
304
+ ```
305
+
306
+ This command should create a migration file, let's edit it.
148
307
 
149
- Or you can define topic directly:
150
308
  ```ruby
151
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
309
+ # ./lariat/migrate/1668097991_user_created_queue.rb
310
+
311
+ # frozen_string_literal: true
312
+
313
+ class UserCreatedQueue < CycloneLariat::Migration
314
+ def up
315
+ create queue(:user_created, dest: :mailer, fifo: true)
316
+ end
317
+
318
+ def down
319
+ delete queue(:user_created, dest: :mailer, fifo: true)
320
+ end
321
+ end
322
+ ```
323
+
324
+ To apply migration use:
325
+ ```bash
326
+ $ rake cyclone_lariat:migrate
327
+ ```
328
+
329
+ To decline migration use:
330
+ ```bash
331
+ $ rake cyclone_lariat:rollback
152
332
  ```
153
333
 
334
+ Since the SNS\SQS management does not support an ACID transaction (in the sense of a database),
335
+ I highly recommend using the atomic schema:
336
+
337
+ ```ruby
338
+ # BAD:
339
+ class UserCreated < CycloneLariat::Migration
340
+ def up
341
+ create queue(:user_created, dest: :mailer, fifo: true)
342
+ create topic(:user_created, fifo: true)
343
+
344
+ subscribe topic: topic(:user_created, fifo: true),
345
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
346
+ end
347
+
348
+ def down
349
+ unsubscribe topic: topic(:user_created, fifo: true),
350
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
351
+
352
+ delete topic(:user_created, fifo: true)
353
+ delete queue(:user_created, dest: :mailer, fifo: true)
354
+ end
355
+ end
356
+
357
+ # GOOD:
358
+ class UserCreatedQueue < CycloneLariat::Migration
359
+ def up
360
+ create queue(:user_created, dest: :mailer, fifo: true)
361
+ end
362
+
363
+ def down
364
+ delete queue(:user_created, dest: :mailer, fifo: true)
365
+ end
366
+ end
367
+
368
+ class UserCreatedTopic < CycloneLariat::Migration
369
+ def up
370
+ create topic(:user_created, fifo: true)
371
+ end
372
+
373
+ def down
374
+ delete topic(:user_created, fifo: true)
375
+ end
376
+ end
377
+
378
+ class UserCreatedSubscription < CycloneLariat::Migration
379
+ def up
380
+ subscribe topic: topic(:user_created, fifo: true),
381
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
382
+ end
383
+
384
+ def down
385
+ unsubscribe topic: topic(:user_created, fifo: true),
386
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
387
+ end
388
+ end
389
+ ```
390
+
391
+
392
+ #### Example: one-to-many
393
+
394
+ The first example is when your _registration_ service creates new user. You also have two services:
395
+ _mailer_ - sending a welcome email, and _statistics_ service.
396
+
397
+ ```ruby
398
+ create topic(:user_created, fifo: true)
399
+ create queue(:user_created, dest: :mailer, fifo: true)
400
+ create queue(:user_created, dest: :stat, fifo: true)
401
+
402
+ subscribe topic: topic(:user_created, fifo: true),
403
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
404
+
405
+
406
+ subscribe topic: topic(:user_created, fifo: true),
407
+ endpoint: queue(:user_created, dest: :statistic, fifo: true)
408
+ ```
409
+ ![one2many](docs/_imgs/graphviz_01.png)
410
+
411
+ #### Example: many-to-one
412
+
413
+ The second example is when you have three services: _registration_ - creates new users, _order_
414
+ service - allows you to create new orders, _statistics_ service collects all statistics.
415
+
416
+ ```ruby
417
+ create topic(:user_created, fifo: false)
418
+ create topic(:order_created, fifo: false)
419
+ create queue(publisher: :any, dest: :statistic, fifo: false)
420
+
421
+ subscribe topic: topic(:user_created, fifo: false),
422
+ endpoint: queue(publisher: :any, dest: :statistic, fifo: false)
423
+
424
+ subscribe topic: topic(:order_created, fifo: false),
425
+ endpoint: queue(publisher: :any, dest: :statistic, fifo: false)
426
+ ```
427
+ ![one2many](docs/_imgs/graphviz_02.png)
428
+
429
+ If queue receives messages from multiple sources you must specify publisher as `:any`. If the
430
+ subscriber receives messages with different types, `cyclone_lariat` uses a specific keyword - `all`.
431
+
432
+ #### Example fanout-to-fanout
433
+
434
+ For better organisation you can subscribe topic on topic. For example, you have _management_panel_
435
+ and _client_panel_ services. Each of these services can register a user with predefined roles.
436
+ And you want to send this information to the _mailer_ and _statistics_ services.
437
+
438
+ ```ruby
439
+ create topic(:client_created, fifo: false)
440
+ create topic(:manager_created, fifo: false)
441
+ create topic(:user_created, publisher: :any, fifo: false)
442
+ create queue(:user_created, publisher: :any, dest: :mailer, fifo: false)
443
+ create queue(:user_created, publisher: :any, dest: :stat, fifo: false)
444
+
445
+ subscribe topic: topic(:user_created, fifo: false),
446
+ endpoint: topic(:user_created, publisher: :any, fifo: false)
447
+
448
+ subscribe topic: topic(:manager_created, fifo: false),
449
+ endpoint: topic(:user_created, publisher: :any, fifo: false)
450
+
451
+ subscribe topic: topic(:user_created, publisher: :any, fifo: false),
452
+ endpoint: queue(:user_created, publisher: :any, dest: :mailer, fifo: false)
453
+
454
+ subscribe topic: topic(:user_created, publisher: :any, fifo: false),
455
+ endpoint: queue(:user_created, publisher: :any, dest: :stat, fifo: false)
456
+ ```
457
+
458
+ ![one2many](docs/_imgs/graphviz_03.png)
459
+
460
+ ### Create and remove custom Topics and Queues
461
+
462
+ You can create Topic and Queues with custom names. That way recommended for:
463
+ - Remove old resources
464
+ - Receive messages from external sources
465
+
466
+ ```ruby
467
+ create custom_topic('custom_topic_name')
468
+ delete custom_queue('custom_topic_name')
469
+ ```
470
+
471
+ ### Where should the migration be?
472
+
473
+ We recommend locate migration on:
474
+ - **topic** - on Publisher side;
475
+ - **queue** - on Subscriber side;
476
+ - **subscription** - on Subscriber side.
477
+
478
+ # Console tasks
479
+
480
+ ```bash
481
+ $ cyclone_lariat install - install cyclone_lariat
482
+ $ cyclone_lariat generate migration - generate new migration
483
+
484
+ $ rake cyclone_lariat:list:queues # List all queues
485
+ $ rake cyclone_lariat:list:subscriptions # List all subscriptions
486
+ $ rake cyclone_lariat:list:topics # List all topics
487
+ $ rake cyclone_lariat:migrate # Migrate topics for SQS/SNS
488
+ $ rake cyclone_lariat:rollback[version] # Rollback topics for SQS/SNS
489
+ $ rake cyclone_lariat:graph # Make graph
490
+ ```
491
+
492
+ Graph generated in [grpahviz](https://graphviz.org/) format for the entry scheme. You should install
493
+ it on your system. For convert it in png use:
494
+ ```bash
495
+ $ rake cyclone_lariat:list:subscriptions | dot -Tpng -o foo.png
496
+ ```
497
+
498
+ # Subscriber
499
+
500
+ This is gem work like middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It save all events to
501
+ database. And catch and produce all exceptions.
502
+
503
+ The logic of lariat as a subscriber. Imagine that you are working with an http server. And it gives you various response
504
+ codes. You have the following processing:
505
+
506
+ - 2xx - success, we process the page.
507
+ - 4хх - Logic error send the error to the developer and wait until he fixes it
508
+ - 5xx - Send an error and try again
509
+
510
+
511
+ ![diagram](docs/_imgs/logic.png)
154
512
 
155
513
  # Middleware
156
514
  If you use middleware:
157
515
  - Store all events to dataset
158
516
  - Notify every input sqs message
159
- - Notify every error
517
+ - Notify every error
160
518
 
161
519
  ```ruby
162
520
  require 'cyclone_lariat/middleware' # If require: false in Gemfile
163
-
521
+ require 'cyclone_lariat/sqs_client' # If you want use queue name helper
164
522
 
165
523
  class Receiver
166
524
  include Shoryuken::Worker
167
-
525
+
168
526
  DB = Sequel.connect(host: 'localhost', user: 'ruby')
169
527
 
170
528
  shoryuken_options auto_delete: true,
171
529
  body_parser: ->(sqs_msg) {
172
530
  JSON.parse(sqs_msg.body, symbolize_names: true)
173
531
  },
174
- queue: 'your_sqs_queue_name'
532
+ queue: 'your_sqs_queue_name.fifo'
533
+ # or
534
+ # queue: CycloneLariat::SqsClient.new.queue('user_added', fifo: true).name
175
535
 
176
536
  server_middleware do |chain|
177
537
  # Options dataset, errors_notifier and message_notifier is optionals.
178
538
  # If you dont define notifiers - middleware does not notify
179
- # If you dont define dataset - middleware does store events in db
539
+ # If you dont define dataset - middleware does not store events in db
180
540
  chain.add CycloneLariat::Middleware,
181
541
  dataset: DB[:events],
182
542
  errors_notifier: LunaPark::Notifiers::Sentry.new,
183
543
  message_notifier: LunaPark::Notifiers::Log.new(min_lvl: :debug, format: :pretty_json)
184
544
  end
185
545
 
546
+ class UserIsNotRegistered < LunaPark::Errors::Business
547
+ end
548
+
186
549
  def perform(sqs_message, sqs_message_body)
187
550
  # Your logic here
551
+
552
+ # If you want to raise business error
553
+ raise UserIsNotRegistered.new(first_name: 'John', last_name: 'Doe')
188
554
  end
189
555
  end
190
556
  ```
191
557
 
192
558
  ## Migrations
193
- Before use events storage add and apply this two migrations
559
+ Before using the event store, add and apply these two migrations:
194
560
 
195
561
  ```ruby
196
562
 
@@ -229,9 +595,18 @@ Sequel.migration do
229
595
  end
230
596
  ```
231
597
 
598
+ And don't forget to add it to the config file:
599
+
600
+ ```ruby
601
+ # 'config/initializers/cyclone_lariat.rb'
602
+ CycloneLariat.tap do |cl|
603
+ cl.events_dataset = DB[:async_messages]
604
+ end
605
+ ```
606
+
232
607
  ### Rake tasks
233
608
 
234
- For simplify write some Rake tasks you can use CycloneLariat::Repo.
609
+ For simplify write some Rake tasks you can use `CycloneLariat::Repo`.
235
610
 
236
611
  ```ruby
237
612
  # For retry all unprocessed
data/Rakefile CHANGED
@@ -7,9 +7,6 @@ require 'rspec/core/rake_task'
7
7
  RSpec::Core::RakeTask.new(:spec)
8
8
 
9
9
  # tasks from lib directory
10
- Dir[File.expand_path('lib/tasks/**/*.rake', __dir__)].each do |entity|
11
- print "#{entity} : "
12
- puts load entity
13
- end
10
+ Rake.add_rakelib 'lib/tasks'
14
11
 
15
- task default: %i[spec] # rubocop]
12
+ task default: %i[spec]