cyclone_lariat 0.2.3 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9a416593e31f7f6fd67eb7e453971b5867d9169bdb37e5708876fdea03b8021f
4
- data.tar.gz: 6ba164622a11d2bd4d63214ba391372bfda1c5447d8979332ef66a4e9e9d2c75
3
+ metadata.gz: 51c2975656b4c950bc5dd8959d06450260fb5ac823005d9c2372b263944b64e4
4
+ data.tar.gz: 4ac631178ba3087ba437cbf9c2e0a502032c53ee133b71d46adcdceb7e1079bc
5
5
  SHA512:
6
- metadata.gz: b7fe001ac8bfc6f373d2f9e7eeb8be68bdea77bca68d4c7b9a7af36c41fd606e82fff5355fc0757e6b4c0b8ca7b00798a7f90c47b3b1a8d141b519389747a117
7
- data.tar.gz: 19f361ff6fa6509439f8aa6122f743175678c81aee9948b71e5f2fb4e3ab8f852b86920fdcbd656c16a3ac1c30705c4da96c542c2e4a373ab012040544128049
6
+ metadata.gz: 125fdc253fdbd93896c7de8c715f0d1cf64288e658f0b870ef1d6ae32351e85d24611be9ababca096b813c1c863f346ca92af604ac91dc60dc0282c8f2c4e3dc
7
+ data.tar.gz: 7435eeadbf5047dc7b9e99488d32a8886cd2a90cef40f627f65aaa20459f220893bd417fbfe1b6b4f2d1a5665051d2cbdc34e7b96968a4a303352f579456b7ea
data/.gitignore CHANGED
@@ -9,7 +9,7 @@
9
9
  /tmp/
10
10
  /.tmp/
11
11
  .byebug_history
12
- config/database.yml
12
+ config/db.rb
13
13
 
14
14
  # rspec failure tracking
15
15
  .rspec_status
data/CHANGELOG.md CHANGED
@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5
5
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+
8
+ ## [0.3.0] - 2021-06-09
9
+ Added
10
+ - Command
11
+ - SqsClient
12
+
13
+ Changed:
14
+ - Client renamed to SnsClient
15
+ - `to:` renamed to `topic:`
16
+
7
17
  ## [0.2.3] - 2021-06-09
8
18
  Added
9
19
  - Skip on empty message with error notify
data/Gemfile.lock CHANGED
@@ -1,8 +1,9 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- cyclone_lariat (0.2.3)
4
+ cyclone_lariat (0.3.0)
5
5
  aws-sdk-sns
6
+ aws-sdk-sqs
6
7
  luna_park (~> 0.11)
7
8
 
8
9
  GEM
@@ -21,6 +22,9 @@ GEM
21
22
  aws-sdk-sns (1.40.0)
22
23
  aws-sdk-core (~> 3, >= 3.112.0)
23
24
  aws-sigv4 (~> 1.1)
25
+ aws-sdk-sqs (1.39.0)
26
+ aws-sdk-core (~> 3, >= 3.112.0)
27
+ aws-sigv4 (~> 1.1)
24
28
  aws-sigv4 (1.2.3)
25
29
  aws-eventstream (~> 1, >= 1.0.2)
26
30
  byebug (11.1.3)
data/README.md CHANGED
@@ -19,59 +19,82 @@ gem 'cyclone_lariat'
19
19
 
20
20
  ![diagram](docs/_imgs/diagram.png)
21
21
 
22
-
23
- ## Client
24
-
22
+ ## Command vs Event
23
+ Commands and events are both simple domain structures that contain solely data for reading. That means they contain no
24
+ behaviour or business logic.
25
+
26
+ A command is an object that is sent to the domain for a state change which is handled by a command handler. They should
27
+ be named with a verb in an imperative mood plus the aggregate name which it operates on. Such request can be rejected
28
+ due to the data the command holds being invalid/inconsistent. There should be exactly 1 handler for each command.
29
+ Once the command has been executed, the consumer can then carry out whatever the task is depending on the output of the
30
+ command.
31
+
32
+ An event is a statement of fact about what change has been made to the domain state. They are named with the aggregate
33
+ name where the change took place plus the verb past-participle. An event happens off the back of a command.
34
+ A command can emit any number of events. The sender of the event does not care who receives it or whether it has been
35
+ received at all.
36
+
37
+ ## SnsClient
25
38
  You can use client directly
26
39
 
27
40
  ```ruby
28
- require 'cyclone_lariat/client' # If require: false in Gemfile
41
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
29
42
 
30
- client = CycloneLariat::Client.new(
31
- key: APP_CONF.aws.key,
43
+ client = CycloneLariat::SnsClient.new(
44
+ key: APP_CONF.aws.key,
32
45
  secret_key: APP_CONF.aws.secret_key,
33
- region: APP_CONF.aws.region,
34
- version: 1, # at default 1
35
- publisher: 'pilot',
36
- instance: INSTANCE # at default :prod
46
+ region: APP_CONF.aws.region,
47
+ version: 1, # at default 1
48
+ publisher: 'pilot',
49
+ instance: INSTANCE # at default :prod
37
50
  )
38
51
  ```
39
52
 
40
53
  You can don't define topic, and it's name will be defined automatically
41
54
  ```ruby
42
55
  # event_type data topic
43
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
44
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
56
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_created
57
+ client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' } # prod-event-fanout-pilot-email_is_removed
58
+ client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' } # prod-command-fanout-pilot-delete_user
45
59
  ```
46
60
  Or you can define it by handle. For example, if you want to send different events to same channel.
47
61
  ```ruby
48
62
  # event_type data topic
49
- client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, to: 'prod-event-fanout-pilot-emails'
50
- client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, to: 'prod-event-fanout-pilot-emails'
63
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
64
+ client.publish_event 'email_is_removed', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
65
+ client.publish_command 'delete_user', data: { mail: 'john.doe@example.com' }, topic: 'prod-command-fanout-pilot-emails'
51
66
  ```
52
67
 
53
68
  Or you can use client as Repo.
54
69
 
55
70
  ```ruby
56
- require 'cyclone_lariat/client' # If require: false in Gemfile
71
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
57
72
 
58
- class YourClient < CycloneLariat::Client
59
- version 1
73
+ class YourClient < CycloneLariat::SnsClient
74
+ version 1
60
75
  publisher 'pilot'
61
- instance 'stage'
62
-
76
+ instance 'stage'
77
+
63
78
  def email_is_created(mail)
64
- publish event( 'email_is_created',
65
- data: { mail: mail }
66
- ),
67
- to: APP_CONF.aws.fanout.emails
79
+ publish event('email_is_created',
80
+ data: { mail: mail }
81
+ ),
82
+ to: APP_CONF.aws.fanout.emails
68
83
  end
69
-
84
+
70
85
  def email_is_removed(mail)
71
- publish event( 'email_is_removed',
72
- data: { mail: mail }
73
- ),
74
- to: APP_CONF.aws.fanout.email
86
+ publish event('email_is_removed',
87
+ data: { mail: mail }
88
+ ),
89
+ to: APP_CONF.aws.fanout.email
90
+ end
91
+
92
+
93
+ def delete_user(mail)
94
+ publish command('delete_user',
95
+ data: { mail: mail }
96
+ ),
97
+ to: APP_CONF.aws.fanout.email
75
98
  end
76
99
  end
77
100
 
@@ -81,8 +104,39 @@ client = YourClient.new(key: APP_CONF.aws.key, secret_key: APP_CONF.aws.secret_k
81
104
  # And send topics
82
105
  client.email_is_created 'john.doe@example.com'
83
106
  client.email_is_removed 'john.doe@example.com'
107
+ client.delete_user 'john.doe@example.com'
84
108
  ```
85
109
 
110
+
111
+ # SqsClient
112
+ SqsClient is really similar to SnsClient. It can be initialized in same way:
113
+
114
+ ```ruby
115
+ require 'cyclone_lariat/sns_client' # If require: false in Gemfile
116
+
117
+ client = CycloneLariat::SqsClient.new(
118
+ key: APP_CONF.aws.key,
119
+ secret_key: APP_CONF.aws.secret_key,
120
+ region: APP_CONF.aws.region,
121
+ version: 1, # at default 1
122
+ publisher: 'pilot',
123
+ instance: INSTANCE # at default :prod
124
+ )
125
+ ```
126
+
127
+ As you see all params identity. And you can easily change your sqs-queue to sns-topic when you start work with more
128
+ subscribes. But you should define destination.
129
+
130
+ ```ruby
131
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, dest: 'notify_service'
132
+ ```
133
+
134
+ Or you can define topic directly:
135
+ ```ruby
136
+ client.publish_event 'email_is_created', data: { mail: 'john.doe@example.com' }, topic: 'prod-event-fanout-pilot-emails'
137
+ ```
138
+
139
+
86
140
  # Middleware
87
141
  If you use middleware:
88
142
  - Store all events to dataset
@@ -144,7 +198,7 @@ end
144
198
  # The second one:
145
199
  Sequel.migration do
146
200
  change do
147
- create_table :events do
201
+ create_table :async_messages do
148
202
  column :uuid, :uuid, primary_key: true
149
203
  String :type, null: false
150
204
  Integer :version, null: false
@@ -33,6 +33,7 @@ Gem::Specification.new do |spec|
33
33
  spec.require_paths = ['lib']
34
34
 
35
35
  spec.add_dependency 'aws-sdk-sns'
36
+ spec.add_dependency 'aws-sdk-sqs'
36
37
  spec.add_dependency 'luna_park', '~> 0.11'
37
38
 
38
39
  spec.add_development_dependency 'bundler', '~> 2.1'
@@ -2,8 +2,9 @@
2
2
 
3
3
  Sequel.migration do
4
4
  change do
5
- create_table :events do
5
+ create_table :async_messages do
6
6
  column :uuid, :uuid, primary_key: true
7
+ String :kind, null: false
7
8
  String :type, null: false
8
9
  Integer :version, null: false
9
10
  String :publisher, null: false
@@ -1,9 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative 'cyclone_lariat/client'
3
+ require_relative 'cyclone_lariat/sns_client'
4
4
  require_relative 'cyclone_lariat/errors'
5
5
  require_relative 'cyclone_lariat/event'
6
- require_relative 'cyclone_lariat/events_repo'
6
+ require_relative 'cyclone_lariat/messages_repo'
7
7
  require_relative 'cyclone_lariat/middleware'
8
8
  require_relative 'cyclone_lariat/version'
9
9
 
@@ -0,0 +1,94 @@
1
+ # frozen_string_literal: true
2
+
3
+ # require 'aws-sdk-sns'
4
+ require 'luna_park/extensions/injector'
5
+ require_relative '../event'
6
+ require_relative '../command'
7
+ require_relative '../errors'
8
+
9
+ module CycloneLariat
10
+ module Abstract
11
+ class Client
12
+ include LunaPark::Extensions::Injector
13
+
14
+ dependency(:aws_client_class) { raise ArgumentError, 'Client class should be defined' }
15
+ dependency(:aws_credentials_class) { Aws::Credentials }
16
+
17
+ DEFAULT_VERSION = 1
18
+ DEFAULT_INSTANCE = :prod
19
+
20
+ def initialize(key:, secret_key:, region:, version: nil, publisher: nil, instance: nil)
21
+ @key = key
22
+ @secret_key = secret_key
23
+ @region = region
24
+ @version = version
25
+ @publisher = publisher
26
+ @instance = instance
27
+ end
28
+
29
+ def event(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
30
+ Event.wrap(
31
+ uuid: uuid,
32
+ type: type,
33
+ sent_at: Time.now.iso8601,
34
+ version: version,
35
+ publisher: publisher,
36
+ data: data
37
+ )
38
+ end
39
+
40
+ def command(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
41
+ Command.wrap(
42
+ uuid: uuid,
43
+ type: type,
44
+ sent_at: Time.now.iso8601,
45
+ version: version,
46
+ publisher: publisher,
47
+ data: data
48
+ )
49
+ end
50
+
51
+ def publish
52
+ raise LunaPark::Errors::AbstractMethod, 'Publish method should be defined'
53
+ end
54
+
55
+ class << self
56
+ def version(version = nil)
57
+ version.nil? ? @version || DEFAULT_VERSION : @version = version
58
+ end
59
+
60
+ def instance(instance = nil)
61
+ instance.nil? ? @instance || DEFAULT_INSTANCE : @instance = instance
62
+ end
63
+
64
+ def publisher(publisher = nil)
65
+ publisher.nil? ? @publisher || (raise 'You should define publisher') : @publisher = publisher
66
+ end
67
+ end
68
+
69
+ def version
70
+ @version ||= self.class.version
71
+ end
72
+
73
+ def publisher
74
+ @publisher ||= self.class.publisher
75
+ end
76
+
77
+ def instance
78
+ @instance ||= self.class.instance
79
+ end
80
+
81
+ private
82
+
83
+ attr_reader :key, :secret_key, :region
84
+
85
+ def aws_client
86
+ @aws_client ||= aws_client_class.new(credentials: aws_credentials, region: region)
87
+ end
88
+
89
+ def aws_credentials
90
+ @aws_credentials ||= aws_credentials_class.new(key, secret_key)
91
+ end
92
+ end
93
+ end
94
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'luna_park/entities/attributable'
4
+ require_relative '../errors'
5
+
6
+ module CycloneLariat
7
+ module Abstract
8
+ class Message < LunaPark::Entities::Attributable
9
+ attr :uuid, String, :new
10
+ attr :publisher, String, :new
11
+ attr :type, String, :new
12
+ attr :client_error
13
+ attr :version
14
+ attr :data
15
+
16
+ attr_reader :sent_at,
17
+ :processed_at,
18
+ :received_at
19
+
20
+ def kind
21
+ raise LunaPark::Errors::AbstractMethod
22
+ end
23
+
24
+ def version=(value)
25
+ @version = Integer(value)
26
+ end
27
+
28
+ def sent_at=(value)
29
+ @sent_at = wrap_time(value)
30
+ end
31
+
32
+ def received_at=(value)
33
+ @received_at = wrap_time(value)
34
+ end
35
+
36
+ def processed_at=(value)
37
+ @processed_at = wrap_time(value)
38
+ end
39
+
40
+ def client_error_message=(txt)
41
+ return unless txt
42
+
43
+ @client_error ||= Errors::ClientError.new
44
+ @client_error.message = txt
45
+ end
46
+
47
+ def client_error_details=(details)
48
+ return unless details
49
+
50
+ @client_error ||= Errors::ClientError.new
51
+ @client_error.details = details
52
+ end
53
+
54
+ def ==(other)
55
+ kind == other.kind &&
56
+ uuid == other.uuid &&
57
+ publisher == other.publisher &&
58
+ type == other.type &&
59
+ client_error&.message == other.client_error&.message &&
60
+ client_error&.details == other.client_error&.details &&
61
+ version == other.version &&
62
+ sent_at.to_i == other.sent_at.to_i &&
63
+ received_at.to_i == other.received_at.to_i
64
+ processed_at.to_i == other.processed_at.to_i
65
+ end
66
+
67
+ def to_json(*args)
68
+ hash = serialize
69
+ hash[:type] = [kind, hash[:type]].join '_'
70
+ hash.to_json(*args)
71
+ end
72
+
73
+ private
74
+
75
+ def wrap_time(value)
76
+ case value
77
+ when String then Time.parse(value)
78
+ when Time then value
79
+ when NilClass then nil
80
+ else raise Argumentevent.rbError, "Unknown type `#{value.class}`"
81
+ end
82
+ end
83
+ end
84
+ end
85
+ end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'abstract/message'
4
+
5
+ module CycloneLariat
6
+ class Command < Abstract::Message
7
+ KIND = 'command'
8
+
9
+ def kind
10
+ KIND
11
+ end
12
+ end
13
+ end
@@ -1,85 +1,13 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'luna_park/entities/attributable'
4
- require_relative 'errors'
3
+ require_relative 'abstract/message'
5
4
 
6
5
  module CycloneLariat
7
- class Event < LunaPark::Entities::Attributable
6
+ class Event < Abstract::Message
8
7
  KIND = 'event'
9
8
 
10
- attr :uuid, String, :new
11
- attr :publisher, String, :new
12
- attr :type, String, :new
13
- attr :client_error
14
- attr :version
15
- attr :data
16
-
17
- attr_reader :sent_at,
18
- :processed_at,
19
- :received_at
20
-
21
9
  def kind
22
10
  KIND
23
11
  end
24
-
25
- def version=(value)
26
- @version = Integer(value)
27
- end
28
-
29
- def sent_at=(value)
30
- @sent_at = wrap_time(value)
31
- end
32
-
33
- def received_at=(value)
34
- @received_at = wrap_time(value)
35
- end
36
-
37
- def processed_at=(value)
38
- @processed_at = wrap_time(value)
39
- end
40
-
41
- def client_error_message=(txt)
42
- return unless txt
43
-
44
- @client_error ||= Errors::ClientError.new
45
- @client_error.message = txt
46
- end
47
-
48
- def client_error_details=(details)
49
- return unless details
50
-
51
- @client_error ||= Errors::ClientError.new
52
- @client_error.details = details
53
- end
54
-
55
- def ==(other)
56
- kind == other.kind &&
57
- uuid == other.uuid &&
58
- publisher == other.publisher &&
59
- type == other.type &&
60
- client_error&.message == other.client_error&.message &&
61
- client_error&.details == other.client_error&.details &&
62
- version == other.version &&
63
- sent_at.to_i == other.sent_at.to_i &&
64
- received_at.to_i == other.received_at.to_i
65
- processed_at.to_i == other.processed_at.to_i
66
- end
67
-
68
- def to_json(*args)
69
- hash = serialize
70
- hash[:type] = [kind, hash[:type]].join '_'
71
- hash.to_json(*args)
72
- end
73
-
74
- private
75
-
76
- def wrap_time(value)
77
- case value
78
- when String then Time.parse(value)
79
- when Time then value
80
- when NilClass then nil
81
- else raise ArgumentError, "Unknown type `#{value.class}`"
82
- end
83
- end
84
12
  end
85
13
  end
@@ -0,0 +1,79 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'event'
4
+
5
+ module CycloneLariat
6
+ class MessagesRepo
7
+ attr_reader :dataset
8
+
9
+ def initialize(dataset)
10
+ @dataset = dataset
11
+ end
12
+
13
+ def create(msg)
14
+ dataset.insert(
15
+ uuid: msg.uuid,
16
+ kind: msg.kind,
17
+ type: msg.type,
18
+ publisher: msg.publisher,
19
+ data: JSON.generate(msg.data),
20
+ client_error_message: msg.client_error&.message,
21
+ client_error_details: JSON.generate(msg.client_error&.details),
22
+ version: msg.version,
23
+ sent_at: msg.sent_at
24
+ )
25
+ end
26
+
27
+ def exists?(uuid:)
28
+ dataset.where(uuid: uuid).limit(1).any?
29
+ end
30
+
31
+ def processed!(uuid:, error: nil)
32
+ data = { processed_at: Sequel.function(:NOW) }
33
+ data.merge!(client_error_message: error.message, client_error_details: JSON.generate(error.details)) if error
34
+
35
+ !dataset.where(uuid: uuid).update(data).zero?
36
+ end
37
+
38
+ def find(uuid:)
39
+ raw = dataset.where(uuid: uuid).first
40
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
41
+ if raw[:client_error_details]
42
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
43
+ end
44
+ build raw
45
+ end
46
+
47
+ def each_unprocessed
48
+ dataset.where(processed_at: nil).each do |raw|
49
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
50
+ if raw[:client_error_details]
51
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
52
+ end
53
+ msg = build raw
54
+ yield(msg)
55
+ end
56
+ end
57
+
58
+ def each_with_client_errors
59
+ dataset.where { (processed_at !~ nil) & (client_error_message !~ nil) }.each do |raw|
60
+ raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
61
+ if raw[:client_error_details]
62
+ raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true)
63
+ end
64
+ msg = build raw
65
+ yield(msg)
66
+ end
67
+ end
68
+
69
+ private
70
+
71
+ def build(raw)
72
+ case kind = raw.delete(:kind)
73
+ when 'event' then Event.wrap raw
74
+ when 'command' then Command.wrap raw
75
+ else raise ArgumentError, "Unknown kind `#{kind}` of message"
76
+ end
77
+ end
78
+ end
79
+ end
@@ -1,12 +1,12 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative 'events_repo'
3
+ require_relative 'messages_repo'
4
4
  require 'luna_park/errors'
5
5
  require 'json'
6
6
 
7
7
  module CycloneLariat
8
8
  class Middleware
9
- def initialize(dataset: nil, errors_notifier: nil, message_notifier: nil, repo: EventsRepo)
9
+ def initialize(dataset: nil, errors_notifier: nil, message_notifier: nil, repo: MessagesRepo)
10
10
  @events_repo = repo.new(dataset) if dataset
11
11
  @message_notifier = message_notifier
12
12
  @errors_notifier = errors_notifier
@@ -17,7 +17,7 @@ module CycloneLariat
17
17
 
18
18
  catch_standard_error(queue, body) do
19
19
  return true unless check(body[:Message])
20
-
20
+
21
21
  event = Event.wrap(JSON.parse(body[:Message]))
22
22
 
23
23
  catch_business_error(event) do
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'aws-sdk-sns'
4
+ require_relative 'abstract/client'
5
+
6
+ module CycloneLariat
7
+ class SnsClient < Abstract::Client
8
+ include LunaPark::Extensions::Injector
9
+
10
+ dependency(:aws_client_class) { Aws::SNS::Client }
11
+
12
+ SNS_SUFFIX = :fanout
13
+
14
+ def publish(msg, topic: nil)
15
+ topic ||= [instance, msg.kind, SNS_SUFFIX, publisher, msg.type].join('-')
16
+
17
+ aws_client.publish(
18
+ topic_arn: topic_arn(topic),
19
+ message: msg.to_json
20
+ )
21
+ end
22
+
23
+ def publish_event(type, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
24
+ publish event(type, data: data, version: version, uuid: uuid), topic: topic
25
+ end
26
+
27
+ def publish_command(type, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
28
+ publish command(type, data: data, version: version, uuid: uuid), topic: topic
29
+ end
30
+
31
+ private
32
+
33
+ def topic_arn(topic_name)
34
+ list = aws_client.list_topics.topics
35
+ topic = list.find { |t| t.topic_arn.match?(topic_name) }
36
+ raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: list.map(&:topic_arn)) if topic.nil?
37
+
38
+ topic.topic_arn
39
+ end
40
+ end
41
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'aws-sdk-sqs'
4
+ require_relative 'abstract/client'
5
+
6
+ module CycloneLariat
7
+ class SqsClient < Abstract::Client
8
+ include LunaPark::Extensions::Injector
9
+
10
+ dependency(:aws_client_class) { Aws::SQS::Client }
11
+
12
+ SQS_SUFFIX = :queue
13
+
14
+ def publish(msg, dest: nil, topic: nil)
15
+ raise ArgumentError, 'You should define dest or topic' if dest.nil? && topic.nil?
16
+
17
+ topic ||= [instance, msg.kind, SQS_SUFFIX, publisher, msg.type, dest].join('-')
18
+
19
+ aws_client.send_message(
20
+ queue_url: url(topic),
21
+ message_body: msg.to_json
22
+ )
23
+ end
24
+
25
+ def publish_event(type, dest: nil, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
26
+ publish event(type, data: data, version: version, uuid: uuid), dest: dest, topic: topic
27
+ end
28
+
29
+ def publish_command(type, dest: nil, data: {}, version: self.version, uuid: SecureRandom.uuid, topic: nil)
30
+ publish command(type, data: data, version: version, uuid: uuid), dest: dest, topic: topic
31
+ end
32
+
33
+ private
34
+
35
+ def url(topic_name)
36
+ aws_client.get_queue_url(queue_name: topic_name).queue_url
37
+ rescue Aws::SQS::Errors::NonExistentQueue => _e
38
+ raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: aws_client.list_queues)
39
+ end
40
+ end
41
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module CycloneLariat
4
- VERSION = '0.2.3'
4
+ VERSION = '0.3.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: cyclone_lariat
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.3
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alexander Kudrin
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2021-06-09 00:00:00.000000000 Z
12
+ date: 2021-06-10 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: aws-sdk-sns
@@ -25,6 +25,20 @@ dependencies:
25
25
  - - ">="
26
26
  - !ruby/object:Gem::Version
27
27
  version: '0'
28
+ - !ruby/object:Gem::Dependency
29
+ name: aws-sdk-sqs
30
+ requirement: !ruby/object:Gem::Requirement
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: '0'
35
+ type: :runtime
36
+ prerelease: false
37
+ version_requirements: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
28
42
  - !ruby/object:Gem::Dependency
29
43
  name: luna_park
30
44
  requirement: !ruby/object:Gem::Requirement
@@ -260,11 +274,15 @@ files:
260
274
  - docs/_imgs/diagram.png
261
275
  - docs/_imgs/lariat.jpg
262
276
  - lib/cyclone_lariat.rb
263
- - lib/cyclone_lariat/client.rb
277
+ - lib/cyclone_lariat/abstract/client.rb
278
+ - lib/cyclone_lariat/abstract/message.rb
279
+ - lib/cyclone_lariat/command.rb
264
280
  - lib/cyclone_lariat/errors.rb
265
281
  - lib/cyclone_lariat/event.rb
266
- - lib/cyclone_lariat/events_repo.rb
282
+ - lib/cyclone_lariat/messages_repo.rb
267
283
  - lib/cyclone_lariat/middleware.rb
284
+ - lib/cyclone_lariat/sns_client.rb
285
+ - lib/cyclone_lariat/sqs_client.rb
268
286
  - lib/cyclone_lariat/version.rb
269
287
  - lib/tasks/db.rake
270
288
  homepage: https://am-team.github.io/cyclone_lariat/#/
@@ -1,98 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require 'aws-sdk-sns'
4
- require 'luna_park/extensions/injector'
5
- require_relative 'event'
6
- require_relative 'errors'
7
-
8
- module CycloneLariat
9
- class Client
10
- include LunaPark::Extensions::Injector
11
-
12
- dependency(:aws_sns_client_class) { Aws::SNS::Client }
13
- dependency(:aws_credentials_class) { Aws::Credentials }
14
-
15
- DEFAULT_VERSION = 1
16
- DEFAULT_INSTANCE = :prod
17
- SNS_SUFFIX = :fanout
18
-
19
- def initialize(key:, secret_key:, region:, version: nil, publisher: nil, instance: nil)
20
- @key = key
21
- @secret_key = secret_key
22
- @region = region
23
- @version = version
24
- @publisher = publisher
25
- @instance = instance
26
- end
27
-
28
- def event(type, data: {}, version: self.version, uuid: SecureRandom.uuid)
29
- Event.wrap(
30
- uuid: uuid,
31
- type: type,
32
- sent_at: Time.now.iso8601,
33
- version: version,
34
- publisher: publisher,
35
- data: data
36
- )
37
- end
38
-
39
- def publish(msg, to: nil)
40
- topic = to || [instance, msg.kind, SNS_SUFFIX, publisher, msg.type].join('-')
41
-
42
- aws_client.publish(
43
- topic_arn: topic_arn(topic),
44
- message: msg.to_json
45
- )
46
- end
47
-
48
- def publish_event(type, data: {}, version: self.version, uuid: SecureRandom.uuid, to: nil)
49
- publish event(type, data: data, version: version, uuid: uuid), to: to
50
- end
51
-
52
- class << self
53
- def version(version = nil)
54
- version.nil? ? @version || DEFAULT_VERSION : @version = version
55
- end
56
-
57
- def instance(instance = nil)
58
- instance.nil? ? @instance || DEFAULT_INSTANCE : @instance = instance
59
- end
60
-
61
- def publisher(publisher = nil)
62
- publisher.nil? ? @publisher || (raise 'You should define publisher') : @publisher = publisher
63
- end
64
- end
65
-
66
- def version
67
- @version ||= self.class.version
68
- end
69
-
70
- def publisher
71
- @publisher ||= self.class.publisher
72
- end
73
-
74
- def instance
75
- @instance ||= self.class.instance
76
- end
77
-
78
- private
79
-
80
- attr_reader :key, :secret_key, :region
81
-
82
- def topic_arn(topic_name)
83
- list = aws_client.list_topics.topics
84
- topic = list.find { |t| t.topic_arn.match?(topic_name) }
85
- raise Errors::TopicNotFound.new(expected_topic: topic_name, existed_topics: list.map(&:topic_arn)) if topic.nil?
86
-
87
- topic.topic_arn
88
- end
89
-
90
- def aws_client
91
- @aws_client ||= aws_sns_client_class.new(credentials: aws_credentials, region: region)
92
- end
93
-
94
- def aws_credentials
95
- @aws_credentials ||= aws_credentials_class.new(key, secret_key)
96
- end
97
- end
98
- end
@@ -1,65 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require_relative 'event'
4
-
5
- module CycloneLariat
6
- class EventsRepo
7
- attr_reader :dataset
8
-
9
- def initialize(dataset)
10
- @dataset = dataset
11
- end
12
-
13
- def create(event)
14
- dataset.insert(
15
- uuid: event.uuid,
16
- type: event.type,
17
- publisher: event.publisher,
18
- data: JSON.generate(event.data),
19
- client_error_message: event.client_error&.message,
20
- client_error_details: JSON.generate(event.client_error&.details),
21
- version: event.version,
22
- sent_at: event.sent_at
23
- )
24
- end
25
-
26
- def exists?(uuid:)
27
- dataset.where(uuid: uuid).limit(1).any?
28
- end
29
-
30
- def processed!(uuid:, error: nil)
31
- data = { processed_at: Sequel.function(:NOW) }
32
- data.merge!(
33
- client_error_message: error&.message,
34
- client_error_details: JSON.generate(error&.details),
35
- ) if error
36
-
37
- !dataset.where(uuid: uuid).update(data).zero?
38
- end
39
-
40
- def find(uuid:)
41
- raw = dataset.where(uuid: uuid).first
42
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
43
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
44
- Event.wrap raw
45
- end
46
-
47
- def each_unprocessed
48
- dataset.where(processed_at: nil).each do |raw|
49
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
50
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
51
- event = Event.wrap(raw)
52
- yield(event)
53
- end
54
- end
55
-
56
- def each_with_client_errors
57
- dataset.where { (processed_at !~ nil) & (client_error_message !~ nil) }.each do |raw|
58
- raw[:data] = JSON.parse(raw[:data], symbolize_names: true)
59
- raw[:client_error_details] = JSON.parse(raw[:client_error_details], symbolize_names: true) if raw[:client_error_details]
60
- event = Event.wrap(raw)
61
- yield(event)
62
- end
63
- end
64
- end
65
- end