cyclone_lariat 0.2.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9a416593e31f7f6fd67eb7e453971b5867d9169bdb37e5708876fdea03b8021f
4
- data.tar.gz: 6ba164622a11d2bd4d63214ba391372bfda1c5447d8979332ef66a4e9e9d2c75
3
+ metadata.gz: 51c2975656b4c950bc5dd8959d06450260fb5ac823005d9c2372b263944b64e4
4
+ data.tar.gz: 4ac631178ba3087ba437cbf9c2e0a502032c53ee133b71d46adcdceb7e1079bc
5
5
  SHA512:
6
- metadata.gz: b7fe001ac8bfc6f373d2f9e7eeb8be68bdea77bca68d4c7b9a7af36c41fd606e82fff5355fc0757e6b4c0b8ca7b00798a7f90c47b3b1a8d141b519389747a117
7
- data.tar.gz: 19f361ff6fa6509439f8aa6122f743175678c81aee9948b71e5f2fb4e3ab8f852b86920fdcbd656c16a3ac1c30705c4da96c542c2e4a373ab012040544128049
6
+ metadata.gz: 125fdc253fdbd93896c7de8c715f0d1cf64288e658f0b870ef1d6ae32351e85d24611be9ababca096b813c1c863f346ca92af604ac91dc60dc0282c8f2c4e3dc
7
+ data.tar.gz: 7435eeadbf5047dc7b9e99488d32a8886cd2a90cef40f627f65aaa20459f220893bd417fbfe1b6b4f2d1a5665051d2cbdc34e7b96968a4a303352f579456b7ea
data/.gitignore CHANGED
@@ -9,7 +9,7 @@
9
9
  /tmp/
10
10
  /.tmp/
11
11
  .byebug_history
12
- config/database.yml
12
+ config/db.rb
13
13
 
14
14
  # rspec failure tracking
15
15
  .rspec_status
data/CHANGELOG.md CHANGED
@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5
5
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+
8
+ ## [0.3.0] - 2021-06-09
9
+ Added
10
+ - Command
11
+ - SqsClient
12
+
13
+ Changed:
14
+ - Client renamed to SnsClient
15
+ - `to:` renamed to `topic:`
16
+
7
17
  ## [0.2.3] - 2021-06-09
8
18
  Added
9
19
  - Skip on empty message with error notify
data/Gemfile.lock CHANGED
@@ -1,8 +1,9 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- cyclone_lariat (0.2.3)
4
+ cyclone_lariat (0.3.0)
5
5
  aws-sdk-sns
6
+ aws-sdk-sqs
6
7
  luna_park (~> 0.11)
7
8
 
8
9
  GEM
@@ -21,6 +22,9 @@ GEM
21
22
  aws-sdk-sns (1.40.0)
22
23
  aws-sdk-core (~> 3, >= 3.112.0)
23
24
  aws-sigv4 (~> 1.1)
25
+ aws-sdk-sqs (1.39.0)
26
+ aws-sdk-core (~> 3, >= 3.112.0)
27
+ aws-sigv4 (~> 1.1)
24
28
  aws-sigv4 (1.2.3)
25
29
  aws-eventstream (~> 1, >= 1.0.2)
26
30
  byebug (11.1.3)
data/README.md CHANGED
@@ -19,59 +19,82 @@ gem 'cyclone_lariat'
19
19
 
20
20
  ![diagram](docs/_imgs/diagram.png)
21
21
 
22
-
23
- ## Client
24
-
22
+ ## Command vs Event
23
+ Commands and events are both simple domain structures that contain solely data for reading. That means they contain no
24
+ behaviour or business logic.
25
+
26
+ A command is an object that is sent to the domain for a state change which is handled by a command handler. They should
27
+ be named with a verb in an imperative mood plus the aggregate name which it operates on. Such request can be rejected
28
+ due to the data the command holds being invalid/inconsistent. There should be exactly 1 handler for each command.
29
+ Once the command has been executed, the consumer can then carry out whatever the task is depending on the output of the
30
+ command.
31
+
32
+ An event is a statement of fact about what change has been made to the domain state. They are named with the aggregate
33
+ name where the change took place plus the verb past-participle. An event happens off the back of a command.
34
+ A command can emit any number of events. The sender of the event does not care who receives it or whether it has been
35
+ received at all.
36
+
37
+ ## SnsClient
25
38
  You can use client directly
26
39
 
27
40
  ```ruby
28
- require 'cyclone_lariat/client' # If require: false in Gemfile
41
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
29
42
 
30
- client = CycloneLariat::Client.new(
31
- key: APP_CONF.aws.key,
43
+ client = CycloneLariat::SnsClient.new(
44
+ key: APP_CONF.aws.key,
32
45
  secret_key: APP_CONF.aws.secret_key,
33
- region: APP_CONF.aws.region,
34
- version: 1, # at default 1
35
- publisher: 'pilot',
36
- instance: INSTANCE # at default :prod
46
+ region: APP_CONF.aws.region,
47
+ version: 1, # at default 1
48
+ publisher: 'pilot',
49
+ instance: INSTANCE # at default :prod
37
50
  )
38
51
  ```
39
52
 
40
53
  You can don't define topic, and it's name will be defined automatically
41
54
  ```ruby
42
55
  # event_type data topic
43
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
44
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
56
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
57
+ client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
58
+ client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' } # prod-command-fanout-pilot-delete_user
45
59
  ```
46
60
  Or you can define it by handle. For example, if you want to send different events to same channel.
47
61
  ```ruby
48
62
  # event_type data topic
49
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, to: 'prod-event-fanout-pilot-emails'
50
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, to: 'prod-event-fanout-pilot-emails'
63
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
64
+ client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
65
+ client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' }, topic: 'prod-command-fanout-pilot-emails'
51
66
  ```
52
67
 
53
68
  Or you can use client as Repo.
54
69
 
55
70
  ```ruby
56
- require 'cyclone_lariat/client' # If require: false in Gemfile
71
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
57
72
 
58
- class YourClient < CycloneLariat::Client
59
- version 1
73
+ class YourClient < CycloneLariat::SnsClient
74
+ version 1
60
75
  publisher 'pilot'
61
- instance 'stage'
62
-
76
+ instance 'stage'
77
+
63
78
  def email_is_created(mail)
64
- publish event( 'email_is_created',
65
- data: { mail: mail }
66
- ),
67
- to: APP_CONF.aws.fanout.emails
79
+ publish event('email_is_created',
80
+ data: { mail: mail }
81
+ ),
82
+ to: APP_CONF.aws.fanout.emails
68
83
  end
69
-
84
+
70
85
  def email_is_removed(mail)
71
- publish event( 'email_is_removed',
72
- data: { mail: mail }
73
- ),
74
- to: APP_CONF.aws.fanout.email
86
+ publish event('email_is_removed',
87
+ data: { mail: mail }
88
+ ),
89
+ to: APP_CONF.aws.fanout.email
90
+ end
91
+
92
+
93
+ def delete_user(mail)
94
+ publish command('delete_user',
95
+ data: { mail: mail }
96
+ ),
97
+ to: APP_CONF.aws.fanout.email
75
98
  end
76
99
  end
77
100
 
@@ -81,8 +104,39 @@ client = YourClient.new(key: APP_CONF.aws.key, secret_key: APP_CONF.aws.secret_k
81
104
  # And send topics
82
105
  client.email_is_created 'john.doe@example.com'
83
106
  client.email_is_removed 'john.doe@example.com'
107
+ client.delete_user 'john.doe@example.com'
84
108
  ```
85
109
 
110
+
111
+ # SqsClient
112
+ SqsClient is really similar to SnsClient. It can be initialized in same way:
113
+
114
+ ```ruby
115
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
116
+
117
+ client = CycloneLariat::SqsClient.new(
118
+ key: APP_CONF.aws.key,
119
+ secret_key: APP_CONF.aws.secret_key,
120
+ region: APP_CONF.aws.region,
121
+ version: 1, # at default 1
122
+ publisher: 'pilot',
123
+ instance: INSTANCE # at default :prod
124
+ )
125
+ ```
126
+
127
+ As you see all params identity. And you can easily change your sqs-queue to sns-topic when you start work with more
128
+ subscribes. But you should define destination.
129
+
130
+ ```ruby
131
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, dest: 'notify_service'
132
+ ```
133
+
134
+ Or you can define topic directly:
135
+ ```ruby
136
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
137
+ ```
138
+
139
+
86
140
  # Middleware
87
141
  If you use middleware:
88
142
  - Store all events to dataset
@@ -144,7 +198,7 @@ end
144
198
  # The second one:
145
199
  Sequel.migration do
146
200
  change do
147
- create_table :events do
201
+ create_table :async_messages do
148
202
  column :uuid, :uuid, primary_key: true
149
203
  String :type, null: false
150
204
  Integer :version, null: false
@@ -33,6 +33,7 @@ Gem::Specification.new do |spec|
33
33
  spec.require_paths = ['lib']
34
34
 
35
35
  spec.add_dependency 'aws-sdk-sns'
36
+ spec.add_dependency 'aws-sdk-sqs'
36
37
  spec.add_dependency 'luna_park', '~> 0.11'
37
38
 
38
39
  spec.add_development_dependency 'bundler', '~> 2.1'
@@ -2,8 +2,9 @@
2
2
 
3
3
  Sequel.migration do
4
4
  change do
5
- create_table :events do
5
+ create_table :async_messages do
6
6
  column :uuid, :uuid, primary_key: true
7
+ String :kind, null: false
7
8
  String :type, null: false
8
9
  Integer :version, null: false
9
10
  String :publisher, null: false
@@ -1,9 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative 'cyclone_lariat/client'
3
+ require_relative 'cyclone_lariat/sns_client'
4
4
  require_relative 'cyclone_lariat/errors'
5
5
  require_relative 'cyclone_lariat/event'
6
- require_relative 'cyclone_lariat/events_repo'
6
+ require_relative 'cyclone_lariat/messages_repo'
7
7
  require_relative 'cyclone_lariat/middleware'
8
8
  require_relative 'cyclone_lariat/version'
9
9
 
@@ -0,0 +1,94 @@
1
+ # frozen_string_literal: true
2
+
3
+ # require 'aws-sdk-sns'
4
+ require 'luna_park/extensions/injector'
5
+ require_relative '../event'
6
+ require_relative '../command'
7
+ require_relative '../errors'
8
+
9
+ module CycloneLariat
10
+ module Abstract
11
+ class Client
12
+ include LunaPark::Extensions::Injector
13
+
14
+ dependency(:aws_client_class) { raise ArgumentError, 'Client class should be defined' }
15
+ dependency(:aws_credentials_class) { Aws::Credentials }
16
+
17
+ DEFAULT_VERSION = 1
18
+ DEFAULT_INSTANCE = :prod
19
+
20
+ def initialize(key:, secret_key:, region:, version: nil, publisher: nil, instance: nil)
21
+ @key = key
22
+ @secret_key = secret_key
23
+ @region = region
24
+ @version = version
25
+ @publisher = publisher
26
+ @instance = instance
27
+ end
28
+
29
+ def event(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
30
+ Event.wrap(
31
+ uuid: uuid,
32
+ type: type,
33
+ sent_at: Time.now.iso8601,
34
+ version: version,
35
+ publisher: publisher,
36
+ data: data
37
+ )
38
+ end
39
+
40
+ def command(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
41
+ Command.wrap(
42
+ uuid: uuid,
43
+ type: type,
44
+ sent_at: Time.now.iso8601,
45
+ version: version,
46
+ publisher: publisher,
47
+ data: data
48
+ )
49
+ end
50
+
51
+ def publish
52
+ raise LunaPark::Errors::AbstractMethod, 'Publish method should be defined'
53
+ end
54
+
55
+ class << self
56
+ def version(version = nil)
57
+ version.nil? ? @version || DEFAULT_VERSION : @version = version
58
+ end
59
+
60
+ def instance(instance = nil)
61
+ instance.nil? ? @instance || DEFAULT_INSTANCE : @instance = instance
62
+ end
63
+
64
+ def publisher(publisher = nil)
65
+ publisher.nil? ? @publisher || (raise 'You should define publisher') : @publisher = publisher
66
+ end
67
+ end
68
+
69
+ def version
70
+ @version ||= self.class.version
71
+ end
72
+
73
+ def publisher
74
+ @publisher ||= self.class.publisher
75
+ end
76
+
77
+ def instance
78
+ @instance ||= self.class.instance
79
+ end
80
+
81
+ private
82
+
83
+ attr_reader :key, :secret_key, :region
84
+
85
+ def aws_client
86
+ @aws_client ||= aws_client_class.new(credentials: aws_credentials, region: region)
87
+ end
88
+
89
+ def aws_credentials
90
+ @aws_credentials ||= aws_credentials_class.new(key, secret_key)
91
+ end
92
+ end
93
+ end
94
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'luna_park/entities/attributable'
4
+ require_relative '../errors'
5
+
6
+ module CycloneLariat
7
+ module Abstract
8
+ class Message < LunaPark::Entities::Attributable
9
+ attr :uuid, String, :new
10
+ attr :publisher, String, :new
11
+ attr :type, String, :new
12
+ attr :client_error
13
+ attr :version
14
+ attr :data
15
+
16
+ attr_reader :sent_at,
17
+ :processed_at,
18
+ :received_at
19
+
20
+ def kind
21
+ raise LunaPark::Errors::AbstractMethod
22
+ end
23
+
24
+ def version=(value)
25
+ @version = Integer(value)
26
+ end
27
+
28
+ def sent_at=(value)
29
+ @sent_at = wrap_time(value)
30
+ end
31
+
32
+ def received_at=(value)
33
+ @received_at = wrap_time(value)
34
+ end
35
+
36
+ def processed_at=(value)
37
+ @processed_at = wrap_time(value)
38
+ end
39
+
40
+ def client_error_message=(txt)
41
+ return unless txt
42
+
43
+ @client_error ||= Errors::ClientError.new
44
+ @client_error.message = txt
45
+ end
46
+
47
+ def client_error_details=(details)
48
+ return unless details
49
+
50
+ @client_error ||= Errors::ClientError.new
51
+ @client_error.details = details
52
+ end
53
+
54
+ def ==(other)
55
+ kind == other.kind &&
56
+ uuid == other.uuid &&
57
+ publisher == other.publisher &&
58
+ type == other.type &&
59
+ client_error&.message == other.client_error&.message &&
60
+ client_error&.details == other.client_error&.details &&
61
+ version == other.version &&
62
+ sent_at.to_i == other.sent_at.to_i &&
63
+ received_at.to_i == other.received_at.to_i
64
+ processed_at.to_i == other.processed_at.to_i
65
+ end
66
+
67
+ def to_json(*args)
68
+ hash = serialize
69
+ hash[:type] = [kind, hash[:type]].join '_'
70
+ hash.to_json(*args)
71
+ end
72
+
73
+ private
74
+
75
+ def wrap_time(value)
76
+ case value
77
+ when String then Time.parse(value)
78
+ when Time then value
79
+ when NilClass then nil
80
+ else raise Argumentevent.rbError, "Unknown type `#{value.class}`"
81
+ end
82
+ end
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'abstract/message'
4
+
5
+ module CycloneLariat
6
+ class Command < Abstract::Message
7
+ KIND = 'command'
8
+
9
+ def kind
10
+ KIND
11
+ end
12
+ end
13
+ end
@@ -1,85 +1,13 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'luna_park/entities/attributable'
4
- require_relative 'errors'
3
+ require_relative 'abstract/message'
5
4
 
6
5
  module CycloneLariat
7
- class Event < LunaPark::Entities::Attributable
6
+ class Event < Abstract::Message
8
7
  KIND = 'event'
9
8
 
10
- attr :uuid, String, :new
11
- attr :publisher, String, :new
12
- attr :type, String, :new
13
- attr :client_error
14
- attr :version
15
- attr :data
16
-
17
- attr_reader :sent_at,
18
- :processed_at,
19
- :received_at
20
-
21
9
  def kind
22
10
  KIND
23
11
  end
24
-
25
- def version=(value)
26
- @version = Integer(value)
27
- end
28
-
29
- def sent_at=(value)
30
- @sent_at = wrap_time(value)
31
- end
32
-
33
- def received_at=(value)
34
- @received_at = wrap_time(value)
35
- end
36
-
37
- def processed_at=(value)
38
- @processed_at = wrap_time(value)
39
- end
40
-
41
- def client_error_message=(txt)
42
- return unless txt
43
-
44
- @client_error ||= Errors::ClientError.new
45
- @client_error.message = txt
46
- end
47
-
48
- def client_error_details=(details)
49
- return unless details
50
-
51
- @client_error ||= Errors::ClientError.new
52
- @client_error.details = details
53
- end
54
-
55
- def ==(other)
56
- kind == other.kind &&
57
- uuid == other.uuid &&
58
- publisher == other.publisher &&
59
- type == other.type &&
60
- client_error&.message == other.client_error&.message &&
61
- client_error&.details == other.client_error&.details &&
62
- version == other.version &&
63
- sent_at.to_i == other.sent_at.to_i &&
64
- received_at.to_i == other.received_at.to_i
65
- processed_at.to_i == other.processed_at.to_i
66
- end
67
-
68
- def to_json(*args)
69
- hash = serialize
70
- hash[:type] = [kind, hash[:type]].join '_'
71
- hash.to_json(*args)
72
- end
73
-
74
- private
75
-
76
- def wrap_time(value)
77
- case value
78
- when String then Time.parse(value)
79
- when Time then value
80
- when NilClass then nil
81
- else raise ArgumentError, "Unknown type `#{value.class}`"
82
- end
83
- end
84
12
  end
85
13
  end
@@ -0,0 +1,79 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'event'
4
+
5
+ module CycloneLariat
6
+ class MessagesRepo
7
+ attr_reader :dataset
8
+
9
+ def initialize(dataset)
10
+ @dataset = dataset
11
+ end
12
+
13
+ def create(msg)
14
+ dataset.insert(
15
+ uuid: msg.uuid,
16
+ kind: msg.kind,
17
+ type: msg.type,
18
+ publisher: msg.publisher,
19
+ data: JSON.generate(msg.data),
20
+ client_error_message: msg.client_error&.message,
21
+ client_error_details: JSON.generate(msg.client_error&.details),
22
+ version: msg.version,
23
+ sent_at: msg.sent_at
24
+ )
25
+ end
26
+
27
+ def exists?(uuid:)
28
+ dataset.where(uuid: uuid).limit(1).any?
29
+ end
30
+
31
+ def processed!(uuid:, error: nil)
32
+ data = { processed_at: Sequel.function(:NOW) }
33
+ data.merge!(client_error_message: error.message, client_error_details: JSON.generate(error.details)) if error
34
+
35
+ !dataset.where(uuid: uuid).update(data).zero?
36
+ end
37
+
38
+ def find(uuid:)
39
+ raw = dataset.where(uuid: uuid).first
40
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
41
+ if raw[:client_error_details]
42
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
43
+ end
44
+ build raw
45
+ end
46
+
47
+ def each_unprocessed
48
+ dataset.where(processed_at: nil).each do |raw|
49
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
50
+ if raw[:client_error_details]
51
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
52
+ end
53
+ msg = build raw
54
+ yield(msg)
55
+ end
56
+ end
57
+
58
+ def each_with_client_errors
59
+ dataset.where { (processed_at !~ nil) & (client_error_message !~ nil) }.each do |raw|
60
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
61
+ if raw[:client_error_details]
62
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
63
+ end
64
+ msg = build raw
65
+ yield(msg)
66
+ end
67
+ end
68
+
69
+ private
70
+
71
+ def build(raw)
72
+ case kind = raw.delete(:kind)
73
+ when 'event' then Event.wrap raw
74
+ when 'command' then Command.wrap raw
75
+ else raise ArgumentError, "Unknown kind `#{kind}` of message"
76
+ end
77
+ end
78
+ end
79
+ end
@@ -1,12 +1,12 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative 'events_repo'
3
+ require_relative 'messages_repo'
4
4
  require 'luna_park/errors'
5
5
  require 'json'
6
6
 
7
7
  module CycloneLariat
8
8
  class Middleware
9
- def initialize(dataset: nil, errors_notifier: nil, message_notifier: nil, repo: EventsRepo)
9
+ def initialize(dataset: nil, errors_notifier: nil, message_notifier: nil, repo: MessagesRepo)
10
10
  @events_repo = repo.new(dataset) if dataset
11
11
  @message_notifier = message_notifier
12
12
  @errors_notifier = errors_notifier
@@ -17,7 +17,7 @@ module CycloneLariat
17
17
 
18
18
  catch_standard_error(queue, body) do
19
19
  return true unless check(body[:Message])
20
-
20
+
21
21
  event = Event.wrap(JSON.parse(body[:Message]))
22
22
 
23
23
  catch_business_error(event) do
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'aws-sdk-sns'
4
+ require_relative 'abstract/client'
5
+
6
+ module CycloneLariat
7
+ class SnsClient < Abstract::Client
8
+ include LunaPark::Extensions::Injector
9
+
10
+ dependency(:aws_client_class) { Aws::SNS::Client }
11
+
12
+ SNS_SUFFIX = :fanout
13
+
14
+ def publish(msg, topic: nil)
15
+ topic ||= [instance, msg.kind, SNS_SUFFIX, publisher, msg.type].join('-')
16
+
17
+ aws_client.publish(
18
+ topic_arn: topic_arn(topic),
19
+ message: msg.to_json
20
+ )
21
+ end
22
+
23
+ def publish_event(type, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
24
+ publish event(type, data: data, version: version, uuid: uuid), topic: topic
25
+ end
26
+
27
+ def publish_command(type, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
28
+ publish command(type, data: data, version: version, uuid: uuid), topic: topic
29
+ end
30
+
31
+ private
32
+
33
+ def topic_arn(topic_name)
34
+ list = aws_client.list_topics.topics
35
+ topic = list.find { |t| t.topic_arn.match?(topic_name) }
36
+ raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: list.map(&:topic_arn)) if topic.nil?
37
+
38
+ topic.topic_arn
39
+ end
40
+ end
41
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'aws-sdk-sqs'
4
+ require_relative 'abstract/client'
5
+
6
+ module CycloneLariat
7
+ class SqsClient < Abstract::Client
8
+ include LunaPark::Extensions::Injector
9
+
10
+ dependency(:aws_client_class) { Aws::SQS::Client }
11
+
12
+ SQS_SUFFIX = :queue
13
+
14
+ def publish(msg, dest: nil, topic: nil)
15
+ raise ArgumentError, 'You should define dest or topic' if dest.nil? && topic.nil?
16
+
17
+ topic ||= [instance, msg.kind, SQS_SUFFIX, publisher, msg.type, dest].join('-')
18
+
19
+ aws_client.send_message(
20
+ queue_url: url(topic),
21
+ message_body: msg.to_json
22
+ )
23
+ end
24
+
25
+ def publish_event(type, dest: nil, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
26
+ publish event(type, data: data, version: version, uuid: uuid), dest: dest, topic: topic
27
+ end
28
+
29
+ def publish_command(type, dest: nil, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
30
+ publish command(type, data: data, version: version, uuid: uuid), dest: dest, topic: topic
31
+ end
32
+
33
+ private
34
+
35
+ def url(topic_name)
36
+ aws_client.get_queue_url(queue_name: topic_name).queue_url
37
+ rescue Aws::SQS::Errors::NonExistentQueue => _e
38
+ raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: aws_client.list_queues)
39
+ end
40
+ end
41
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module CycloneLariat
4
- VERSION = '0.2.3'
4
+ VERSION = '0.3.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: cyclone_lariat
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.3
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexander Kudrin
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2021-06-09 00:00:00.000000000 Z
12
+ date: 2021-06-10 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: aws-sdk-sns
@@ -25,6 +25,20 @@ dependencies:
25
25
  - - ">="
26
26
  - !ruby/object:Gem::Version
27
27
  version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: aws-sdk-sqs
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :runtime
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
28
42
  - !ruby/object:Gem::Dependency
29
43
  name: luna_park
30
44
  requirement: !ruby/object:Gem::Requirement
@@ -260,11 +274,15 @@ files:
260
274
  - docs/_imgs/diagram.png
261
275
  - docs/_imgs/lariat.jpg
262
276
  - lib/cyclone_lariat.rb
263
- - lib/cyclone_lariat/client.rb
277
+ - lib/cyclone_lariat/abstract/client.rb
278
+ - lib/cyclone_lariat/abstract/message.rb
279
+ - lib/cyclone_lariat/command.rb
264
280
  - lib/cyclone_lariat/errors.rb
265
281
  - lib/cyclone_lariat/event.rb
266
- - lib/cyclone_lariat/events_repo.rb
282
+ - lib/cyclone_lariat/messages_repo.rb
267
283
  - lib/cyclone_lariat/middleware.rb
284
+ - lib/cyclone_lariat/sns_client.rb
285
+ - lib/cyclone_lariat/sqs_client.rb
268
286
  - lib/cyclone_lariat/version.rb
269
287
  - lib/tasks/db.rake
270
288
  homepage: https://am-team.github.io/cyclone_lariat/#/
@@ -1,98 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require 'aws-sdk-sns'
4
- require 'luna_park/extensions/injector'
5
- require_relative 'event'
6
- require_relative 'errors'
7
-
8
- module CycloneLariat
9
- class Client
10
- include LunaPark::Extensions::Injector
11
-
12
- dependency(:aws_sns_client_class) { Aws::SNS::Client }
13
- dependency(:aws_credentials_class) { Aws::Credentials }
14
-
15
- DEFAULT_VERSION = 1
16
- DEFAULT_INSTANCE = :prod
17
- SNS_SUFFIX = :fanout
18
-
19
- def initialize(key:, secret_key:, region:, version: nil, publisher: nil, instance: nil)
20
- @key = key
21
- @secret_key = secret_key
22
- @region = region
23
- @version = version
24
- @publisher = publisher
25
- @instance = instance
26
- end
27
-
28
- def event(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
29
- Event.wrap(
30
- uuid: uuid,
31
- type: type,
32
- sent_at: Time.now.iso8601,
33
- version: version,
34
- publisher: publisher,
35
- data: data
36
- )
37
- end
38
-
39
- def publish(msg, to: nil)
40
- topic = to || [instance, msg.kind, SNS_SUFFIX, publisher, msg.type].join('-')
41
-
42
- aws_client.publish(
43
- topic_arn: topic_arn(topic),
44
- message: msg.to_json
45
- )
46
- end
47
-
48
- def publish_event(type, data: {}, version: self.version, uuid: SecureRandom.uuid, to: nil)
49
- publish event(type, data: data, version: version, uuid: uuid), to: to
50
- end
51
-
52
- class << self
53
- def version(version = nil)
54
- version.nil? ? @version || DEFAULT_VERSION : @version = version
55
- end
56
-
57
- def instance(instance = nil)
58
- instance.nil? ? @instance || DEFAULT_INSTANCE : @instance = instance
59
- end
60
-
61
- def publisher(publisher = nil)
62
- publisher.nil? ? @publisher || (raise 'You should define publisher') : @publisher = publisher
63
- end
64
- end
65
-
66
- def version
67
- @version ||= self.class.version
68
- end
69
-
70
- def publisher
71
- @publisher ||= self.class.publisher
72
- end
73
-
74
- def instance
75
- @instance ||= self.class.instance
76
- end
77
-
78
- private
79
-
80
- attr_reader :key, :secret_key, :region
81
-
82
- def topic_arn(topic_name)
83
- list = aws_client.list_topics.topics
84
- topic = list.find { |t| t.topic_arn.match?(topic_name) }
85
- raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: list.map(&:topic_arn)) if topic.nil?
86
-
87
- topic.topic_arn
88
- end
89
-
90
- def aws_client
91
- @aws_client ||= aws_sns_client_class.new(credentials: aws_credentials, region: region)
92
- end
93
-
94
- def aws_credentials
95
- @aws_credentials ||= aws_credentials_class.new(key, secret_key)
96
- end
97
- end
98
- end
@@ -1,65 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require_relative 'event'
4
-
5
- module CycloneLariat
6
- class EventsRepo
7
- attr_reader :dataset
8
-
9
- def initialize(dataset)
10
- @dataset = dataset
11
- end
12
-
13
- def create(event)
14
- dataset.insert(
15
- uuid: event.uuid,
16
- type: event.type,
17
- publisher: event.publisher,
18
- data: JSON.generate(event.data),
19
- client_error_message: event.client_error&.message,
20
- client_error_details: JSON.generate(event.client_error&.details),
21
- version: event.version,
22
- sent_at: event.sent_at
23
- )
24
- end
25
-
26
- def exists?(uuid:)
27
- dataset.where(uuid: uuid).limit(1).any?
28
- end
29
-
30
- def processed!(uuid:, error: nil)
31
- data = { processed_at: Sequel.function(:NOW) }
32
- data.merge!(
33
- client_error_message: error&.message,
34
- client_error_details: JSON.generate(error&.details),
35
- ) if error
36
-
37
- !dataset.where(uuid: uuid).update(data).zero?
38
- end
39
-
40
- def find(uuid:)
41
- raw = dataset.where(uuid: uuid).first
42
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
43
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
44
- Event.wrap raw
45
- end
46
-
47
- def each_unprocessed
48
- dataset.where(processed_at: nil).each do |raw|
49
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
50
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
51
- event = Event.wrap(raw)
52
- yield(event)
53
- end
54
- end
55
-
56
- def each_with_client_errors
57
- dataset.where { (processed_at !~ nil) & (client_error_message !~ nil) }.each do |raw|
58
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
59
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
60
- event = Event.wrap(raw)
61
- yield(event)
62
- end
63
- end
64
- end
65
- end