cyclone_lariat 0.3.10 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,85 +1,142 @@
1
1
  # Cyclone lariat
2
2
 
3
- This is gem work like middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It save all events to database. And catch and produce all exceptions.
3
+ This gem work in few scenarios:
4
+ - As middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It saves all events to the database and also catches and throws all exceptions.
5
+ - As a middleware, it can log all incoming messages.
6
+ - As a client that can send messages to SNS topics and SQS queues.
7
+ - Also it can help you with CI\CD for theme, queue and subscription management like database migration.
4
8
 
5
9
  ![Cyclone lariat](docs/_imgs/lariat.jpg)
6
10
 
7
11
 
12
+ # Client / Publisher
13
+ At first lets understand what the difference between SQS and SNS:
14
+ - Amazon Simple Queue Service (SQS) lets you send, store, and receive messages between software components at any
15
+ volume, without losing messages or requiring other services to be available.
16
+ - Amazon Simple Notification Service (SNS) sends notifications two ways Application2Person (like send sms).
17
+ And the second way is Application2Application, that's way more important for us. In this way you case use
18
+ SNS service like fanout.
19
+
20
+ ![SQS/SNS](docs/_imgs/sqs_sns_diagram.png)
21
+
22
+ For use **cyclone_lariat** as _Publisher_ lets make install CycloneLariat.
23
+
24
+ ## Install cyclone_lariat
25
+ Edit Gemfile:
8
26
  ```ruby
9
27
  # Gemfile
28
+ gem 'sequel'
29
+ gem 'cyclone_lariat'
30
+ ```
31
+ And run in console:
32
+ ```bash
33
+ $ bundle install
34
+ $ cyclone_lariat install
35
+ ```
10
36
 
11
- # If use client or middleware
12
- gem 'cyclone_lariat', require: false
37
+ Last command will create 2 files:
38
+ - ./lib/tasks/cyclone_lariat.rake - Rake tasks, for management migrations
39
+ - ./config/initializers/cyclone_lariat.rb - Configuration default values for cyclone lariat usage
13
40
 
14
- # If use client
15
- gem 'cyclone_lariat'
41
+ ## Configuration
42
+ ```ruby
43
+ # frozen_string_literal: true
44
+
45
+ CycloneLariat.tap do |cl|
46
+ cl.default_version = 1 # api version
47
+ cl.aws_key = ENV['AWS_KEY'] # aws key
48
+ cl.aws_account_id = ENV['AWS_ACCOUNT_ID'] # aws account id
49
+ cl.aws_secret_key = ENV['AWS_SECRET_KEY'] # aws secret
50
+ cl.aws_default_region = ENV['AWS_REGION'] # aws default region
51
+ cl.publisher = ENV['APP_NAME'] # name of your publishers, usually name of your application
52
+ cl.default_instance = ENV['INSTANCE'] # stage, production, test
53
+ cl.events_dataset = DB[:events] # sequel dataset for store income messages, for receiver
54
+ cl.versions_dataset = DB[:lariat_versions] # sequel dataset for migrations, for publisher
55
+ end
56
+ ```
57
+ If you are only using your application as a publisher, you may not need to set the `events_dataset`
58
+ parameter.
59
+
60
+ Before creating the first migration, let's explain what `CycloneLariat::Message` is.
61
+
62
+ ## Messages
63
+ Message in Amazon SQS\SNS service it's a
64
+ [object](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-attributes)
65
+ that has several attributes. The main attributes are the **body**, which consists of the published
66
+ data. The body is a _String_, but we can use it as a _JSON_ object. **Cyclone_lariat** use this scheme:
67
+
68
+ ```json
69
+ {
70
+ "uuid": "f2ce3813-0905-4d81-a60e-f289f2431f50", // Uniq message identificator
71
+ "publisher": "sample_app", // Publisher application name
72
+ "request_id": "51285005-8a06-4181-b5fd-bf29f3b1a45a", // Optional: X-Request-Id
73
+ "type": "event_note_created", // Type of Event or Command
74
+ "version": 1, // Version of data structure
75
+ "data": {
76
+ "id": 12,
77
+ "text": "Sample of published data",
78
+ "attributes": ["one", "two", "three"]
79
+ },
80
+ "sent_at": "2022-11-09T11:42:18.203+01:00" // Time when message was sended in ISO8601 Standard
81
+ }
16
82
  ```
17
83
 
18
- ## Logic
84
+ Idea about X-Request-Id you can see at
85
+ [StackOverflow](https://stackoverflow.com/questions/25433258/what-is-the-x-request-id-http-header).
86
+
87
+ As you see, type has prefix 'event_' in cyclone lariat you has two kinds of messages - _Event_ and _Command_.
19
88
 
20
- ![diagram](docs/_imgs/diagram.png)
89
+ ### Command vs Event
90
+ Commands and events are both simple domain structures that contain solely data for reading. That means
91
+ they contain no behaviour or business logic.
21
92
 
22
- ## Command vs Event
23
- Commands and events are both simple domain structures that contain solely data for reading. That means they contain no
24
- behaviour or business logic.
93
+ A command is an object that is sent to the domain for a state change which is handled by a command
94
+ handler. They should be named with a verb in an imperative mood plus the aggregate name which it
95
+ operates on. Such request can be rejected due to the data the command holds being invalid/inconsistent.
96
+ There should be exactly 1 handler for each command. Once the command has been executed, the consumer
97
+ can then carry out whatever the task is depending on the output of the command.
25
98
 
26
- A command is an object that is sent to the domain for a state change which is handled by a command handler. They should
27
- be named with a verb in an imperative mood plus the aggregate name which it operates on. Such request can be rejected
28
- due to the data the command holds being invalid/inconsistent. There should be exactly 1 handler for each command.
29
- Once the command has been executed, the consumer can then carry out whatever the task is depending on the output of the
30
- command.
99
+ An event is a statement of fact about what change has been made to the domain state. They are named
100
+ with the aggregate name where the change took place plus the verb past-participle. An event happens off
101
+ the back of a command.
102
+ A command can emit any number of events. The sender of the event does not care who receives it or
103
+ whether it has been received at all.
31
104
 
32
- An event is a statement of fact about what change has been made to the domain state. They are named with the aggregate
33
- name where the change took place plus the verb past-participle. An event happens off the back of a command.
34
- A command can emit any number of events. The sender of the event does not care who receives it or whether it has been
35
- received at all.
105
+ ### Publish
106
+ For publishing _Event_ or _Commands_, you have two ways, send _Message_ directly:
36
107
 
37
- ## Configure
38
108
  ```ruby
39
- # 'config/initializers/cyclone_lariat.rb'
40
109
  CycloneLariat.tap do |cl|
41
- cl.default_version = 1 # api version default is 1
42
- cl.aws_key = # aws key
43
- cl.aws_secret_key = # aws secret
44
- cl.aws_client_id = # aws client id
45
- cl.aws_default_region = # aws default region
46
- cl.publisher = 'auth' # name of your publishers, usually name of your application
47
- cl.default_instance = APP_INSTANCE # stage, production, test
110
+ # Config app here
48
111
  end
49
- ```
50
112
 
51
- ## SnsClient
52
- You can use client directly
113
+ client = CycloneLariat::SnsClient.new(instance: 'auth', version: 2)
53
114
 
54
- ```ruby
55
- require 'cyclone_lariat/sns_client' # If require: false in Gemfile
56
-
57
- client = CycloneLariat::SnsClient.new(
58
- key: APP_CONF.aws.key,
59
- secret_key: APP_CONF.aws.secret_key,
60
- region: APP_CONF.aws.region,
61
- version: 1, # at default 1
62
- publisher: 'pilot',
63
- instance: INSTANCE # at default :prod
115
+ client.publish_command('register_user', data: {
116
+ first_name: 'John',
117
+ last_name: 'Doe',
118
+ mail: 'john.doe@example.com'
119
+ }, fifo: false
64
120
  )
65
121
  ```
66
122
 
67
- You can don't define topic, and it's name will be defined automatically
68
- ```ruby
69
- # event_type data topic
70
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
71
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
72
- client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' } # prod-command-fanout-pilot-delete_user
73
- ```
74
- Or you can define it by handle. For example, if you want to send different events to same channel.
75
- ```ruby
76
- # event_type data topic
77
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
78
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
79
- client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' }, topic: 'prod-command-fanout-pilot-emails'
123
+ That's call, will generate a message body:
124
+ ```json
125
+ {
126
+ "uuid": "f2ce3813-0905-4d81-a60e-f289f2431f50",
127
+ "publisher": "auth",
128
+ "type": "command_register_user",
129
+ "version": 2,
130
+ "data": {
131
+ "first_name": "John",
132
+ "last_name": "Doe",
133
+ "mail": "john.doe@example.com"
134
+ },
135
+ "sent_at": "2022-11-09T11:42:18.203+01:00" // Time when message was sended in ISO8601 Standard
136
+ }
80
137
  ```
81
138
 
82
- Or you can use client as Repo.
139
+ Or is it better to make your own client, like a [Repository](https://deviq.com/design-patterns/repository-pattern) pattern.
83
140
 
84
141
  ```ruby
85
142
  require 'cyclone_lariat/sns_client' # If require: false in Gemfile
@@ -90,30 +147,21 @@ class YourClient < CycloneLariat::SnsClient
90
147
  instance 'stage'
91
148
 
92
149
  def email_is_created(mail)
93
- publish event('email_is_created',
94
- data: { mail: mail }
95
- ),
96
- to: APP_CONF.aws.fanout.emails
150
+ publish event('email_is_created', data: { mail: mail }), fifo: true
97
151
  end
98
152
 
99
153
  def email_is_removed(mail)
100
- publish event('email_is_removed',
101
- data: { mail: mail }
102
- ),
103
- to: APP_CONF.aws.fanout.email
154
+ publish event('email_is_removed', data: { mail: mail }), fifo: true
104
155
  end
105
156
 
106
157
 
107
158
  def delete_user(mail)
108
- publish command('delete_user',
109
- data: { mail: mail }
110
- ),
111
- to: APP_CONF.aws.fanout.email
159
+ publish command('delete_user', data: { mail: mail }), fifo: false
112
160
  end
113
161
  end
114
162
 
115
163
  # Init repo
116
- client = YourClient.new(key: APP_CONF.aws.key, secret_key: APP_CONF.aws.secret_key, region: APP_CONF.aws.region)
164
+ client = YourClient.new
117
165
 
118
166
  # And send topics
119
167
  client.email_is_created 'john.doe@example.com'
@@ -121,76 +169,394 @@ client.email_is_removed 'john.doe@example.com'
121
169
  client.delete_user 'john.doe@example.com'
122
170
  ```
123
171
 
172
+ ### Topics and Queue
173
+ An Amazon SNS topic and SQS queue is a logical access point that acts as a communication channel. Both
174
+ of them has specific address ARN.
175
+
176
+ ```
177
+ # Topic example
178
+ arn:aws:sns:eu-west-1:247602342345:test-event-fanout-cyclone_lariat-note_added.fifo
179
+
180
+ # Queue example
181
+ arn:aws:sqs:eu-west-1:247602342345:test-event-queue-cyclone_lariat-note_added-notifier.fifo
182
+ ```
183
+
184
+ Split ARN:
185
+ - `arn:aws:sns` - Prefix for SNS Topics
186
+ - `arn:aws:sqs` - Prefix for SQS Queues
187
+ - `eu-west-1` -
188
+ [AWS Region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions)
189
+ - `247602342345` - [AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html)
190
+ - `test-event-fanout-cyclone_lariat-note_added` - Topic \ Queue name
191
+ - `.fifo` - if Topic or queue is
192
+ [FIFO](https://aws.amazon.com/blogs/aws/introducing-amazon-sns-fifo-first-in-first-out-pub-sub-messaging/), they must
193
+ has that suffix.
194
+
195
+ Region and client_id usually set using the **cyclone_lariat** [configuration](#Configuration).
196
+
197
+ ## Declaration for topic and queues name
198
+ In **cyclone_lariat** we have a declaration for defining topic and queue names.
199
+ This can help in organizing the order.
200
+
201
+ ```ruby
202
+ CycloneLariat.tap do |cfg|
203
+ cfg.instance = 'test'
204
+ cfg.publisher = 'cyclone_lariat'
205
+ # ...
206
+ end
207
+
208
+ CycloneLariat::SnsClient.new.publish_command('register_user', data: {
209
+ first_name: 'John',
210
+ last_name: 'Doe',
211
+ mail: 'john.doe@example.com'
212
+ }, fifo: true
213
+ )
214
+
215
+ # or in repository-like style:
216
+
217
+ class YourClient < CycloneLariat::SnsClient
218
+ def register_user(first:, last:, mail:)
219
+ publish command('register_user', data: { mail: mail }), fifo: true
220
+ end
221
+ end
222
+ ```
223
+
224
+
225
+ We will publish a message on this topic: `test-command-fanout-cyclone_lariat-register_user`.
124
226
 
125
- # SqsClient
126
- SqsClient is really similar to SnsClient. It can be initialized in same way:
227
+ Let's split the topic title:
228
+ - `test` - instance;
229
+ - `command` - kind - [event or command](#command-vs-event);
230
+ - `fanount` - resource type - fanout for SNS topics;
231
+ - `cyclone_lariat` - publisher name;
232
+ - `regiser_user` - message type.
127
233
 
234
+ For queues you also can define destination.
128
235
  ```ruby
129
- require 'cyclone_lariat/sns_client' # If require: false in Gemfile
236
+ CycloneLariat::SqsClient.new.publish_event(
237
+ 'register_user', data: { mail: 'john.doe@example.com' },
238
+ dest: :mailer, fifo: true
239
+ )
240
+
241
+
242
+ # or in repository-like style:
243
+
244
+ class YourClient < CycloneLariat::SnsClient
245
+ # ...
246
+
247
+ def register_user(first:, last:, mail:)
248
+ publish event('register_user', data: { mail: mail }), fifo: true
249
+ end
250
+ end
251
+ ```
252
+
253
+ We will publish a message on this queue: `test-event-queue-cyclone_lariat-register_user-mailer`.
130
254
 
131
- client = CycloneLariat::SqsClient.new(
132
- key: APP_CONF.aws.key,
133
- secret_key: APP_CONF.aws.secret_key,
134
- region: APP_CONF.aws.region,
135
- version: 1, # at default 1
136
- publisher: 'pilot',
137
- instance: INSTANCE # at default :prod
255
+ Let's split the queue title:
256
+ - `test` - instance;
257
+ - `event` - kind - [event or command](#command-vs-event);
258
+ - `queue` - resource type - queue for SQS;
259
+ - `cyclone_lariat` - publisher name;
260
+ - `regiser_user` - message type.
261
+ - `mailer` - destination
262
+
263
+ You also can sent message to queue with custom name. But this way does not recommended.
264
+
265
+ ```ruby
266
+ # Directly
267
+ CycloneLariat::SqsClient.new.publish_event(
268
+ 'register_user', data: { mail: 'john.doe@example.com' },
269
+ dest: :mailer, topic: 'custom_topic_name.fifo', fifo: true
138
270
  )
271
+
272
+ # Repository
273
+ class YourClient < CycloneLariat::SnsClient
274
+ # ...
275
+
276
+ def register_user(first:, last:, mail:)
277
+ publish event('register_user', data: { mail: mail }),
278
+ topic: 'custom_topic_name.fifo', fifo: true
279
+ end
280
+ end
139
281
  ```
282
+ Will publish message on queue: `custom_topic` with fifo suffix.
140
283
 
141
- As you see all params identity. And you can easily change your sqs-queue to sns-topic when you start work with more
142
- subscribes. But you should define destination.
284
+ # Migrations
285
+
286
+ With **cyclone_lariat** you can use migrations that can create, delete, and subscribe to your queues and topics, just like database migrations do.
287
+ To store versions of **cyclone_lariat** migrations, you need to create a database table.
143
288
 
144
289
  ```ruby
145
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, dest: 'notify_service'
146
- # prod-event-queue-pilot-email_is_created-notify_service
290
+ # frozen_string_literal: true
291
+
292
+ Sequel.migration do
293
+ change do
294
+ create_table :lariat_versions do
295
+ Integer :version, null: false, unique: true
296
+ end
297
+ end
298
+ end
147
299
  ```
300
+ After migrate this database migration, create **cyclone_lariat** migration.
301
+
302
+ ```bash
303
+ $ cyclone_lariat generate migration user_created
304
+ ```
305
+
306
+ This command should create a migration file, let's edit it.
148
307
 
149
- Or you can define topic directly:
150
308
  ```ruby
151
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
309
+ # ./lariat/migrate/1668097991_user_created_queue.rb
310
+
311
+ # frozen_string_literal: true
312
+
313
+ class UserCreatedQueue < CycloneLariat::Migration
314
+ def up
315
+ create queue(:user_created, dest: :mailer, fifo: true)
316
+ end
317
+
318
+ def down
319
+ delete queue(:user_created, dest: :mailer, fifo: true)
320
+ end
321
+ end
322
+ ```
323
+
324
+ To apply migration use:
325
+ ```bash
326
+ $ rake cyclone_lariat:migrate
327
+ ```
328
+
329
+ To decline migration use:
330
+ ```bash
331
+ $ rake cyclone_lariat:rollback
152
332
  ```
153
333
 
334
+ Since the SNS\SQS management does not support an ACID transaction (in the sense of a database),
335
+ I highly recommend using the atomic schema:
336
+
337
+ ```ruby
338
+ # BAD:
339
+ class UserCreated < CycloneLariat::Migration
340
+ def up
341
+ create queue(:user_created, dest: :mailer, fifo: true)
342
+ create topic(:user_created, fifo: true)
343
+
344
+ subscribe topic: topic(:user_created, fifo: true),
345
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
346
+ end
347
+
348
+ def down
349
+ unsubscribe topic: topic(:user_created, fifo: true),
350
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
351
+
352
+ delete topic(:user_created, fifo: true)
353
+ delete queue(:user_created, dest: :mailer, fifo: true)
354
+ end
355
+ end
356
+
357
+ # GOOD:
358
+ class UserCreatedQueue < CycloneLariat::Migration
359
+ def up
360
+ create queue(:user_created, dest: :mailer, fifo: true)
361
+ end
362
+
363
+ def down
364
+ delete queue(:user_created, dest: :mailer, fifo: true)
365
+ end
366
+ end
367
+
368
+ class UserCreatedTopic < CycloneLariat::Migration
369
+ def up
370
+ create topic(:user_created, fifo: true)
371
+ end
372
+
373
+ def down
374
+ delete topic(:user_created, fifo: true)
375
+ end
376
+ end
377
+
378
+ class UserCreatedSubscription < CycloneLariat::Migration
379
+ def up
380
+ subscribe topic: topic(:user_created, fifo: true),
381
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
382
+ end
383
+
384
+ def down
385
+ unsubscribe topic: topic(:user_created, fifo: true),
386
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
387
+ end
388
+ end
389
+ ```
390
+
391
+
392
+ #### Example: one-to-many
393
+
394
+ The first example is when your _registration_ service creates new user. You also have two services:
395
+ _mailer_ - sending a welcome email, and _statistics_ service.
396
+
397
+ ```ruby
398
+ create topic(:user_created, fifo: true)
399
+ create queue(:user_created, dest: :mailer, fifo: true)
400
+ create queue(:user_created, dest: :stat, fifo: true)
401
+
402
+ subscribe topic: topic(:user_created, fifo: true),
403
+ endpoint: queue(:user_created, dest: :mailer, fifo: true)
404
+
405
+
406
+ subscribe topic: topic(:user_created, fifo: true),
407
+ endpoint: queue(:user_created, dest: :statistic, fifo: true)
408
+ ```
409
+ ![one2many](docs/_imgs/graphviz_01.png)
410
+
411
+ #### Example: many-to-one
412
+
413
+ The second example is when you have three services: _registration_ - creates new users, _order_
414
+ service - allows you to create new orders, _statistics_ service collects all statistics.
415
+
416
+ ```ruby
417
+ create topic(:user_created, fifo: false)
418
+ create topic(:order_created, fifo: false)
419
+ create queue(publisher: :any, dest: :statistic, fifo: false)
420
+
421
+ subscribe topic: topic(:user_created, fifo: false),
422
+ endpoint: queue(publisher: :any, dest: :statistic, fifo: false)
423
+
424
+ subscribe topic: topic(:order_created, fifo: false),
425
+ endpoint: queue(publisher: :any, dest: :statistic, fifo: false)
426
+ ```
427
+ ![one2many](docs/_imgs/graphviz_02.png)
428
+
429
+ If queue receives messages from multiple sources you must specify publisher as `:any`. If the
430
+ subscriber receives messages with different types, `cyclone_lariat` uses a specific keyword - `all`.
431
+
432
+ #### Example fanout-to-fanout
433
+
434
+ For better organisation you can subscribe topic on topic. For example, you have _management_panel_
435
+ and _client_panel_ services. Each of these services can register a user with predefined roles.
436
+ And you want to send this information to the _mailer_ and _statistics_ services.
437
+
438
+ ```ruby
439
+ create topic(:client_created, fifo: false)
440
+ create topic(:manager_created, fifo: false)
441
+ create topic(:user_created, publisher: :any, fifo: false)
442
+ create queue(:user_created, publisher: :any, dest: :mailer, fifo: false)
443
+ create queue(:user_created, publisher: :any, dest: :stat, fifo: false)
444
+
445
+ subscribe topic: topic(:user_created, fifo: false),
446
+ endpoint: topic(:user_created, publisher: :any, fifo: false)
447
+
448
+ subscribe topic: topic(:manager_created, fifo: false),
449
+ endpoint: topic(:user_created, publisher: :any, fifo: false)
450
+
451
+ subscribe topic: topic(:user_created, publisher: :any, fifo: false),
452
+ endpoint: queue(:user_created, publisher: :any, dest: :mailer, fifo: false)
453
+
454
+ subscribe topic: topic(:user_created, publisher: :any, fifo: false),
455
+ endpoint: queue(:user_created, publisher: :any, dest: :stat, fifo: false)
456
+ ```
457
+
458
+ ![one2many](docs/_imgs/graphviz_03.png)
459
+
460
+ ### Create and remove custom Topics and Queues
461
+
462
+ You can create Topic and Queues with custom names. That way recommended for:
463
+ - Remove old resources
464
+ - Receive messages from external sources
465
+
466
+ ```ruby
467
+ create custom_topic('custom_topic_name')
468
+ delete custom_queue('custom_topic_name')
469
+ ```
470
+
471
+ ### Where should the migration be?
472
+
473
+ We recommend locate migration on:
474
+ - **topic** - on Publisher side;
475
+ - **queue** - on Subscriber side;
476
+ - **subscription** - on Subscriber side.
477
+
478
+ # Console tasks
479
+
480
+ ```bash
481
+ $ cyclone_lariat install - install cyclone_lariat
482
+ $ cyclone_lariat generate migration - generate new migration
483
+
484
+ $ rake cyclone_lariat:list:queues # List all queues
485
+ $ rake cyclone_lariat:list:subscriptions # List all subscriptions
486
+ $ rake cyclone_lariat:list:topics # List all topics
487
+ $ rake cyclone_lariat:migrate # Migrate topics for SQS/SNS
488
+ $ rake cyclone_lariat:rollback[version] # Rollback topics for SQS/SNS
489
+ $ rake cyclone_lariat:graph # Make graph
490
+ ```
491
+
492
+ Graph generated in [grpahviz](https://graphviz.org/) format for the entry scheme. You should install
493
+ it on your system. For convert it in png use:
494
+ ```bash
495
+ $ rake cyclone_lariat:list:subscriptions | dot -Tpng -o foo.png
496
+ ```
497
+
498
+ # Subscriber
499
+
500
+ This is gem work like middleware for [shoryuken](https://github.com/ruby-shoryuken/shoryuken). It save all events to
501
+ database. And catch and produce all exceptions.
502
+
503
+ The logic of lariat as a subscriber. Imagine that you are working with an http server. And it gives you various response
504
+ codes. You have the following processing:
505
+
506
+ - 2xx - success, we process the page.
507
+ - 4хх - Logic error send the error to the developer and wait until he fixes it
508
+ - 5xx - Send an error and try again
509
+
510
+
511
+ ![diagram](docs/_imgs/logic.png)
154
512
 
155
513
  # Middleware
156
514
  If you use middleware:
157
515
  - Store all events to dataset
158
516
  - Notify every input sqs message
159
- - Notify every error
517
+ - Notify every error
160
518
 
161
519
  ```ruby
162
520
  require 'cyclone_lariat/middleware' # If require: false in Gemfile
163
-
521
+ require 'cyclone_lariat/sqs_client' # If you want use queue name helper
164
522
 
165
523
  class Receiver
166
524
  include Shoryuken::Worker
167
-
525
+
168
526
  DB = Sequel.connect(host: 'localhost', user: 'ruby')
169
527
 
170
528
  shoryuken_options auto_delete: true,
171
529
  body_parser: ->(sqs_msg) {
172
530
  JSON.parse(sqs_msg.body, symbolize_names: true)
173
531
  },
174
- queue: 'your_sqs_queue_name'
532
+ queue: 'your_sqs_queue_name.fifo'
533
+ # or
534
+ # queue: CycloneLariat::SqsClient.new.queue('user_added', fifo: true).name
175
535
 
176
536
  server_middleware do |chain|
177
537
  # Options dataset, errors_notifier and message_notifier is optionals.
178
538
  # If you dont define notifiers - middleware does not notify
179
- # If you dont define dataset - middleware does store events in db
539
+ # If you dont define dataset - middleware does not store events in db
180
540
  chain.add CycloneLariat::Middleware,
181
541
  dataset: DB[:events],
182
542
  errors_notifier: LunaPark::Notifiers::Sentry.new,
183
543
  message_notifier: LunaPark::Notifiers::Log.new(min_lvl: :debug, format: :pretty_json)
184
544
  end
185
545
 
546
+ class UserIsNotRegistered < LunaPark::Errors::Business
547
+ end
548
+
186
549
  def perform(sqs_message, sqs_message_body)
187
550
  # Your logic here
551
+
552
+ # If you want to raise business error
553
+ raise UserIsNotRegistered.new(first_name: 'John', last_name: 'Doe')
188
554
  end
189
555
  end
190
556
  ```
191
557
 
192
558
  ## Migrations
193
- Before use events storage add and apply this two migrations
559
+ Before using the event store, add and apply these two migrations:
194
560
 
195
561
  ```ruby
196
562
 
@@ -229,9 +595,18 @@ Sequel.migration do
229
595
  end
230
596
  ```
231
597
 
598
+ And don't forget to add it to the config file:
599
+
600
+ ```ruby
601
+ # 'config/initializers/cyclone_lariat.rb'
602
+ CycloneLariat.tap do |cl|
603
+ cl.events_dataset = DB[:async_messages]
604
+ end
605
+ ```
606
+
232
607
  ### Rake tasks
233
608
 
234
- For simplify write some Rake tasks you can use CycloneLariat::Repo.
609
+ For simplify write some Rake tasks you can use `CycloneLariat::Repo`.
235
610
 
236
611
  ```ruby
237
612
  # For retry all unprocessed
data/Rakefile CHANGED
@@ -7,9 +7,6 @@ require 'rspec/core/rake_task'
7
7
  RSpec::Core::RakeTask.new(:spec)
8
8
 
9
9
  # tasks from lib directory
10
- Dir[File.expand_path('lib/tasks/**/*.rake', __dir__)].each do |entity|
11
- print "#{entity} : "
12
- puts load entity
13
- end
10
+ Rake.add_rakelib 'lib/tasks'
14
11
 
15
- task default: %i[spec] # rubocop]
12
+ task default: %i[spec]