artery 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fd74a0468234a04d072dc0235885426aff27b04353a10f3a32176ada5ebd932e
4
- data.tar.gz: 70193e8d9f5a01ac73157cf4f105252ed66e59a211b169e10b5e5e95a618b5d2
3
+ metadata.gz: 4f1a11d03719b7ed31e9defaab67715e12d45a94bf44d6ef006487345aac4620
4
+ data.tar.gz: 8c7293da3839b5da9867b106a8a49c55ecab34aced06c79020ef09e9f8ccdef9
5
5
  SHA512:
6
- metadata.gz: f93909d8a746b73ac81fcf4cecb2e92d6b7c5b761b16a1d8c244527b18f460ad21498ac5b2556f12ad3cafc75eca7a69724447540d6a5c1a2bb086d0a3bf660b
7
- data.tar.gz: 0a006ed60df6714bcb0459fb34106ef970c760eb58895437b0ac8ca08303ddd6c15559c657e7ce2353c9427621bd84c514102f0326de803b86e39e0249cc54ef
6
+ metadata.gz: eefc0283414fed7b76b869ee6fd5e78d0e755b31f294364d691df36fb840807401c6e883f9cc3da2e7606d2eb239364b582162e71d3aec6f53a782541e38cff9
7
+ data.tar.gz: 225827156bb7a3f6db5e1dd90beb5bb1b62d35b9761098f0101f56ed679dda10e978b504b4c5f4a856b90591db455fbdf563ecccd700474115d1e3cd7cfcb071
data/README.md CHANGED
@@ -36,6 +36,82 @@ mount Artery::Engine => '/artery'
36
36
  ```
37
37
  And then you can access it by url `http(s)://{ your_app_url }/artery/`.
38
38
 
39
+ ## Logging
40
+
41
+ Artery uses `ActiveSupport::Notifications` for instrumentation and `ActiveSupport::TaggedLogging` for request-scoped log tagging.
42
+
43
+ ### Configuration
44
+
45
+ ```ruby
46
+ Artery.configure do |config|
47
+ config.service_name = :my_service
48
+
49
+ # Log every message (publish/request/subscribe/response).
50
+ # When false, only lifecycle events (errors, sync, connect/disconnect) are logged.
51
+ # Default: true
52
+ config.log_messages = true
53
+
54
+ # Maximum bytes of message body included in logs.
55
+ # nil = no limit (full dumps). Default: nil
56
+ config.message_body_max_size = 1024
57
+ end
58
+ ```
59
+
60
+ ### Log levels
61
+
62
+ | Level | What is logged |
63
+ |-------|----------------|
64
+ | `debug` | Message payloads (request, publish, received, response), skip reasons, sync page details |
65
+ | `info` | Lifecycle events: backend connected/reconnected, worker started, sync started/completed |
66
+ | `warn` | Backend disconnected, request errors, no subscriptions defined |
67
+ | `error` | Exception handling (via `ErrorHandler`/`SentryErrorHandler`) |
68
+
69
+ ### Request-scoped tagging
70
+
71
+ All logs emitted during message processing are automatically tagged with the request ID (`reply_to` or a generated hex ID). This includes nested operations (enrich, sub-requests) and any Rails logs (e.g., ActiveRecord queries) that go through the shared logger:
72
+
73
+ ```
74
+ [Artery] [Worker] [abc123] [INBOX.xyz789] [RECV] <svc.model.update> {"uuid":"..."}
75
+ [Artery] [Worker] [abc123] [INBOX.xyz789] Source Load (0.5ms) SELECT ...
76
+ [Artery] [Worker] [abc123] [INBOX.xyz789] [DONE] <svc.model.update> (12.3ms)
77
+ ```
78
+
79
+ On Rails 7.0+ with `config.active_support.isolation_level = :fiber`, this tagging is fiber-safe.
80
+
81
+ ### Instrumentation events
82
+
83
+ All events follow the `event_name.artery` convention. You can subscribe to them for metrics, tracing, or custom logging:
84
+
85
+ ```ruby
86
+ ActiveSupport::Notifications.subscribe('request.artery') do |event|
87
+ StatsD.measure('artery.request', event.duration, tags: { route: event.payload[:route] })
88
+ end
89
+ ```
90
+
91
+ Available events (each uses a `stage:`, `state:`, or `action:` payload key to distinguish sub-stages):
92
+
93
+ | Event | Key | Values | Other payload | Description |
94
+ |-------|-----|--------|---------------|-------------|
95
+ | `request.artery` | `stage` | `:sent` | `route`, `data` | Outbound request sent |
96
+ | | | `:response` | `route`, `data` | Response received |
97
+ | | | `:error` | `route`, `error` | Request timeout or error (always logged) |
98
+ | `publish.artery` | — | — | `route`, `data` | Fire-and-forget publish |
99
+ | `message.artery` | `stage` | `:received` | `route`, `data`, `request_id` | Incoming message |
100
+ | | | `:handled` | `route`, `request_id` | Finished processing (block, has duration) |
101
+ | | | `:skipped` | `reason` | Message skipped |
102
+ | `sync.artery` | `stage` | `:receive_all` | `route` | Full sync (block, has duration) |
103
+ | | | `:receive_updates` | `route` | Incremental sync (block, has duration) |
104
+ | | | `:page` | `route`, `page` | Page received |
105
+ | | | `:continue` | — | Not all updates received, continuing |
106
+ | `connection.artery` | `state` | `:connected` | `server` | Connected to backend |
107
+ | | | `:disconnected` | — | Disconnected from backend |
108
+ | | | `:reconnected` | `server` | Reconnected to backend |
109
+ | | | `:closed` | — | Connection closed |
110
+ | `worker.artery` | `action` | `:started` | `worker_id` | Worker started |
111
+ | | | `:subscribing` | `route` | Subscribing to route |
112
+ | `lock.artery` | `state` | `:waiting` | `latest_index` | Waiting for subscription lock |
113
+ | | | `:acquired` | `latest_index` | Lock acquired |
114
+
39
115
  ## Contributing
40
116
  Contribution directions go here.
41
117
 
@@ -0,0 +1,23 @@
1
+ # frozen_string_literal: true
2
+
3
+ class CreateArteryModelInfos < ActiveRecord::Migration[5.2]
4
+ def up
5
+ create_table :artery_model_infos do |t|
6
+ t.string :model, null: false
7
+ t.bigint :latest_index, null: false, default: 0
8
+ end
9
+
10
+ add_index :artery_model_infos, :model, unique: true
11
+
12
+ execute <<~SQL.squish
13
+ INSERT INTO artery_model_infos (model, latest_index)
14
+ SELECT model, COALESCE(MAX(id), 0)
15
+ FROM artery_messages
16
+ GROUP BY model
17
+ SQL
18
+ end
19
+
20
+ def down
21
+ drop_table :artery_model_infos
22
+ end
23
+ end
@@ -10,6 +10,9 @@ module Artery
10
10
  serialize :data, coder: JSON
11
11
 
12
12
  after_commit :send_to_artery, on: :create
13
+ around_create :lock_on_model
14
+
15
+ attr_accessor :cached_previous_index
13
16
 
14
17
  alias index id
15
18
 
@@ -20,7 +23,7 @@ module Artery
20
23
  end
21
24
 
22
25
  def latest_index(model)
23
- where(model: model).last&.id.to_i
26
+ Artery.model_info_class.find_by(model: model)&.latest_index.to_i
24
27
  end
25
28
 
26
29
  def delete_old
@@ -29,13 +32,23 @@ module Artery
29
32
  end
30
33
  end
31
34
 
32
- # It is used in after_commit, so we always know previous index based on our current index
33
35
  def previous_index
36
+ return cached_previous_index if cached_previous_index
37
+
34
38
  scope = self.class.where(model: model).order(:id)
35
39
  scope = scope.where(self.class.arel_table[:id].lt(index)) if index
36
40
 
37
41
  scope.select(:id).last&.id.to_i
38
42
  end
43
+
44
+ private
45
+
46
+ def lock_on_model
47
+ lock_row = Artery.model_info_class.acquire_lock!(model)
48
+ self.cached_previous_index = lock_row.latest_index
49
+ yield
50
+ lock_row.update!(latest_index: id)
51
+ end
39
52
  end
40
53
  end
41
54
  end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Artery
4
+ module ActiveRecord
5
+ class ModelInfo < ::ActiveRecord::Base
6
+ self.table_name = 'artery_model_infos'
7
+
8
+ def self.acquire_lock!(model_name)
9
+ lock_row = lock('FOR UPDATE').find_by(model: model_name)
10
+ return lock_row if lock_row
11
+
12
+ begin
13
+ create!(model: model_name, latest_index: Artery.message_class.latest_index(model_name))
14
+ rescue ::ActiveRecord::RecordNotUnique
15
+ # concurrent insert — fine
16
+ end
17
+
18
+ lock('FOR UPDATE').find_by!(model: model_name)
19
+ end
20
+ end
21
+ end
22
+ end
@@ -23,11 +23,11 @@ module Artery
23
23
  def with_lock
24
24
  self.class.transaction do
25
25
  unless (was_locked = @locked) # prevent double lock to reduce selects
26
- Artery.logger.debug "WAITING FOR LOCK... [LATEST_INDEX: #{latest_index}]"
26
+ Artery::Instrumentation.instrument(:lock, state: :waiting, latest_index: latest_index)
27
27
 
28
- reload lock: true # explicitely reload record
29
-
30
- Artery.logger.debug "GOT LOCK! [LATEST_INDEX: #{latest_index}]"
28
+ Artery::Instrumentation.instrument(:lock, state: :acquired, latest_index: latest_index) do
29
+ reload lock: true # explicitely reload record
30
+ end
31
31
 
32
32
  @locked = true
33
33
  end
@@ -3,6 +3,7 @@
3
3
  module Artery
4
4
  module ActiveRecord
5
5
  autoload :Message, 'artery/active_record/message'
6
+ autoload :ModelInfo, 'artery/active_record/model_info'
6
7
  autoload :SubscriptionInfo, 'artery/active_record/subscription_info'
7
8
  end
8
9
  end
@@ -58,14 +58,18 @@ module Artery
58
58
  yield(handler)
59
59
 
60
60
  data ||= {}
61
- Artery.logger.debug "REQUESTED: <#{uri.to_route}> #{data.to_json}"
61
+ Artery::Instrumentation.instrument(:request, stage: :sent, route: uri.to_route, data: data)
62
62
 
63
+ request_start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
63
64
  backend.request(uri.to_route, data.to_json, options) do |message|
65
+ duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - request_start) * 1000
64
66
  if message.is_a?(Error) # timeout case
65
- Artery.logger.debug "REQUEST ERROR: <#{uri.to_route}> #{message.message}"
67
+ Artery::Instrumentation.instrument(:request, stage: :error, route: uri.to_route,
68
+ error: message.message, duration_ms: duration_ms)
66
69
  handler.call :error, message
67
70
  else
68
- Artery.logger.debug "REQUEST RESPONSE: <#{uri.to_route}> #{message}"
71
+ Artery::Instrumentation.instrument(:request, stage: :response, route: uri.to_route,
72
+ data: message, duration_ms: duration_ms)
69
73
  begin
70
74
  message ||= '{}'
71
75
  response = JSON.parse(message).with_indifferent_access
@@ -89,7 +93,7 @@ module Artery
89
93
 
90
94
  def publish(route, data)
91
95
  backend.publish(route, data.to_json)
92
- Artery.logger.debug "PUBLISHED: <#{route}> #{data.to_json}"
96
+ Artery::Instrumentation.instrument(:publish, route: route, data: data)
93
97
  end
94
98
  end
95
99
  end
@@ -16,20 +16,20 @@ module Artery
16
16
  client = ::NATS.connect(options)
17
17
 
18
18
  client.on_reconnect do
19
- Artery.logger.debug "Reconnected to server at #{client.connected_server}"
19
+ Artery::Instrumentation.instrument(:connection, state: :reconnected, server: client.connected_server)
20
20
  end
21
21
 
22
22
  client.on_disconnect do
23
- Artery.logger.debug 'Disconnected!'
23
+ Artery::Instrumentation.instrument(:connection, state: :disconnected)
24
24
  end
25
25
 
26
26
  client.on_close do
27
- Artery.logger.debug 'Connection to NATS closed'
27
+ Artery::Instrumentation.instrument(:connection, state: :closed)
28
28
  end
29
29
  client
30
30
  end
31
31
 
32
- Artery.logger.debug "Connected to #{@client.connected_server}"
32
+ Artery::Instrumentation.instrument(:connection, state: :connected, server: @client.connected_server)
33
33
  @client.connect unless @client.connected?
34
34
 
35
35
  @client
data/lib/artery/check.rb CHANGED
@@ -31,18 +31,17 @@ module Artery
31
31
  end
32
32
 
33
33
  def self.run(services)
34
- Artery.logger.push_tags('Check')
35
- result = Artery::Check.new.execute services
36
-
37
- errors = result.select { |_service, res| res[:status] == :error }
38
- return if errors.blank?
39
-
40
- Artery.logger.error "There were errors:\n\t#{errors.map do |service, result|
41
- "#{service}: #{result[:message]}"
42
- end.join("\n\t")}"
43
- exit 1
44
- ensure
45
- Artery.logger.pop_tags
34
+ Artery.logger.tagged('Check') do
35
+ result = Artery::Check.new.execute services
36
+
37
+ errors = result.select { |_service, res| res[:status] == :error }
38
+ return if errors.blank?
39
+
40
+ Artery.logger.error "There were errors:\n\t#{errors.map do |service, res|
41
+ "#{service}: #{res[:message]}"
42
+ end.join("\n\t")}"
43
+ exit 1
44
+ end
46
45
  end
47
46
  end
48
47
  end
data/lib/artery/config.rb CHANGED
@@ -6,14 +6,18 @@ module Artery
6
6
 
7
7
  included do # rubocop:disable Metrics/BlockLength
8
8
  class << self
9
- attr_accessor :message_class, :subscription_info_class, :service_name, :backend_config, :request_timeout,
10
- :error_handler
9
+ attr_accessor :message_class, :model_info_class, :subscription_info_class, :service_name, :backend_config,
10
+ :request_timeout, :error_handler
11
11
 
12
12
  # Ability to redefine message class (for example, for non-activerecord applications)
13
13
  def message_class
14
14
  @message_class || get_model_class(:Message)
15
15
  end
16
16
 
17
+ def model_info_class
18
+ @model_info_class || get_model_class(:ModelInfo)
19
+ end
20
+
17
21
  def subscription_info_class
18
22
  @subscription_info_class || get_model_class(:SubscriptionInfo)
19
23
  end
@@ -35,6 +39,16 @@ module Artery
35
39
  @request_timeout || ENV.fetch('ARTERY_REQUEST_TIMEOUT', '15').to_i
36
40
  end
37
41
 
42
+ attr_writer :log_messages, :message_body_max_size
43
+
44
+ def log_messages?
45
+ instance_variable_defined?(:@log_messages) ? @log_messages : true
46
+ end
47
+
48
+ def message_body_max_size
49
+ instance_variable_defined?(:@message_body_max_size) ? @message_body_max_size : nil
50
+ end
51
+
38
52
  def error_handler
39
53
  @error_handler || (defined?(Artery::SentryErrorHandler) ? Artery::SentryErrorHandler : Artery::ErrorHandler)
40
54
  end
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Artery
4
+ module Instrumentation
5
+ NAMESPACE = 'artery'
6
+
7
+ module_function
8
+
9
+ def instrument(event, payload = {}, &block)
10
+ ActiveSupport::Notifications.instrument("#{event}.#{NAMESPACE}", payload, &block)
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,93 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Artery
4
+ class LogSubscriber < ActiveSupport::LogSubscriber
5
+ def request(event)
6
+ p = event.payload
7
+
8
+ case p[:stage]
9
+ when :sent then debug "[REQ] <#{p[:route]}> #{truncate_body(p[:data])}"
10
+ when :response then debug "[RESP] <#{p[:route]}> #{truncate_body(p[:data])} (#{p[:duration_ms].round(1)}ms)"
11
+ when :error then warn "[REQ ERR] <#{p[:route]}> #{p[:error]} (#{p[:duration_ms].round(1)}ms)"
12
+ end
13
+ end
14
+
15
+ def publish(event)
16
+ debug "[PUB] <#{event.payload[:route]}> #{truncate_body(event.payload[:data])}"
17
+ end
18
+
19
+ def message(event)
20
+ p = event.payload
21
+
22
+ case p[:stage]
23
+ when :received then debug "[RECV] <#{p[:route]}> #{truncate_body(p[:data])}"
24
+ when :handled then debug "[DONE] <#{p[:route]}> (#{event.duration.round(1)}ms)"
25
+ when :skipped then debug "[SKIP] #{p[:reason]}"
26
+ end
27
+ end
28
+
29
+ def sync(event)
30
+ p = event.payload
31
+
32
+ case p[:stage]
33
+ when :receive_all then info "[SYNC] receive_all <#{p[:route]}> (#{event.duration.round(1)}ms)"
34
+ when :receive_updates then info "[SYNC] receive_updates <#{p[:route]}> (#{event.duration.round(1)}ms)"
35
+ when :page then debug "[SYNC] page #{p[:page]} received for <#{p[:route]}>"
36
+ when :continue then debug '[SYNC] not all updates received, continuing...'
37
+ end
38
+ end
39
+
40
+ def connection(event)
41
+ p = event.payload
42
+
43
+ case p[:state]
44
+ when :connected then info "[Backend] connected to #{p[:server]}"
45
+ when :disconnected then warn '[Backend] disconnected'
46
+ when :reconnected then info "[Backend] reconnected to #{p[:server]}"
47
+ when :closed then info '[Backend] connection closed'
48
+ end
49
+ end
50
+
51
+ def worker(event)
52
+ p = event.payload
53
+
54
+ case p[:action]
55
+ when :started then info "started (id=#{p[:worker_id]})"
56
+ when :subscribing then debug "[SUB] <#{p[:route]}>"
57
+ end
58
+ end
59
+
60
+ def lock(event)
61
+ p = event.payload
62
+
63
+ case p[:state]
64
+ when :waiting then debug "[LOCK] waiting (latest_index: #{p[:latest_index]})"
65
+ when :acquired then debug "[LOCK] acquired (latest_index: #{p[:latest_index]}, #{event.duration.round(1)}ms)"
66
+ end
67
+ end
68
+
69
+ private
70
+
71
+ def debug(msg)
72
+ return unless Artery.log_messages?
73
+
74
+ super
75
+ end
76
+
77
+ def truncate_body(data)
78
+ return '' if data.nil?
79
+
80
+ json = data.is_a?(String) ? data : data.to_json
81
+ max = Artery.message_body_max_size
82
+ return json if max.nil? || max <= 0 || json.length <= max
83
+
84
+ "#{json[0...max]}... (truncated, #{json.length} bytes total)"
85
+ end
86
+
87
+ def logger
88
+ Artery.logger
89
+ end
90
+ end
91
+ end
92
+
93
+ Artery::LogSubscriber.attach_to :artery
@@ -14,26 +14,42 @@ module Artery
14
14
  after_archive :artery_on_archive
15
15
  after_unarchive :artery_on_unarchive
16
16
  end
17
+
18
+ before_commit :artery_send_pending_notifications
17
19
  end
18
20
 
19
21
  def artery_on_create
20
- artery_notify_message(:create)
22
+ artery_pending_notifications << [:create]
21
23
  end
22
24
 
23
25
  def artery_on_update
24
- artery_notify_message(:update)
26
+ artery_pending_notifications << [:update]
25
27
  end
26
28
 
27
29
  def artery_on_archive
28
- artery_notify_message(:archive, archived_at: archived_at.to_f)
30
+ artery_pending_notifications << [:archive, { archived_at: archived_at.to_f }]
29
31
  end
30
32
 
31
33
  def artery_on_unarchive
32
- artery_notify_message(:unarchive)
34
+ artery_pending_notifications << [:unarchive]
33
35
  end
34
36
 
35
37
  def artery_on_destroy
36
- artery_notify_message(:delete)
38
+ artery_pending_notifications << [:delete]
39
+ end
40
+
41
+ private
42
+
43
+ def artery_pending_notifications
44
+ @artery_pending_notifications ||= []
45
+ end
46
+
47
+ def artery_send_pending_notifications
48
+ artery_pending_notifications.each do |action, extra_data|
49
+ artery_notify_message(action, extra_data || {})
50
+ end
51
+ ensure
52
+ @artery_pending_notifications = nil
37
53
  end
38
54
  end
39
55
  end
@@ -53,8 +53,7 @@ module Artery
53
53
 
54
54
  # rubocop:disable Metrics/AbcSize,Metrics/MethodLength,Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity
55
55
  def artery_add_get_subscriptions
56
- artery_add_subscription Routing.uri(model: artery_model_name_plural, action: :get) do |data, reply, sub|
57
- Artery.logger.info "HEY-HEY-HEY, message on GET with arguments: `#{[data, reply, sub].inspect}`!"
56
+ artery_add_subscription Routing.uri(model: artery_model_name_plural, action: :get) do |data, reply, _sub|
58
57
  obj = artery_find data['uuid']
59
58
 
60
59
  representation = data['representation']
@@ -64,9 +63,7 @@ module Artery
64
63
  Artery.publish(reply, data)
65
64
  end
66
65
 
67
- artery_add_subscription Routing.uri(model: artery_model_name_plural, action: :get_all) do |data, reply, sub|
68
- Artery.logger.info "HEY-HEY-HEY, message on GET_ALL with arguments: `#{[data, reply, sub].inspect}`!"
69
-
66
+ artery_add_subscription Routing.uri(model: artery_model_name_plural, action: :get_all) do |data, reply, _sub|
70
67
  scope = "artery_#{data['scope'] || 'all'}"
71
68
  per_page = data['per_page']
72
69
  page = data['page'] || 0
@@ -90,9 +87,7 @@ module Artery
90
87
  end
91
88
 
92
89
  artery_add_subscription Routing.uri(model: artery_model_name_plural,
93
- action: :get_updates) do |data, reply, sub|
94
- Artery.logger.info "HEY-HEY-HEY, message on GET_UPDATES with arguments: `#{[data, reply, sub].inspect}`!"
95
-
90
+ action: :get_updates) do |data, reply, _sub|
96
91
  index = data['after_index'].to_i
97
92
  autoenrich = data['representation'].present?
98
93
  per_page = data['per_page'] || (autoenrich ? ARTERY_MAX_AUTOENRICHED_UPDATES_SYNC : ARTERY_MAX_UPDATES_SYNC)
@@ -112,8 +107,6 @@ module Artery
112
107
  latest_index = Artery.message_class.latest_index(artery_model_name)
113
108
  updates_latest_index = messages.last&.index || latest_index
114
109
 
115
- Artery.logger.info "MESSAGES: #{messages.inspect}"
116
-
117
110
  # Autoenrich data
118
111
  if autoenrich
119
112
  scope = "artery_#{data['scope'] || 'all'}"
@@ -83,13 +83,18 @@ module Artery
83
83
  def receive_all
84
84
  synchronization_in_progress! unless synchronization_in_progress?
85
85
 
86
- while receive_all_once == :continue; end
86
+ Artery::Instrumentation.instrument(:sync, stage: :receive_all, route: uri.to_route) do
87
+ reset_latest_index!
88
+ while receive_all_once == :continue; end
89
+ end
87
90
  end
88
91
 
89
92
  def receive_updates
90
93
  synchronization_in_progress!
91
94
 
92
- while receive_updates_once == :continue; end
95
+ Artery::Instrumentation.instrument(:sync, stage: :receive_updates, route: uri.to_route) do
96
+ while receive_updates_once == :continue; end
97
+ end
93
98
  end
94
99
 
95
100
  private
@@ -127,15 +132,13 @@ module Artery
127
132
 
128
133
  Artery.request all_uri.to_route, all_data do |on|
129
134
  on.success do |data|
130
- Artery.logger.debug "HEY-HEY, ALL OBJECTS: <#{all_uri.to_route}> #{[data].inspect}"
131
-
132
135
  objects = data[:objects].map(&:with_indifferent_access)
133
136
 
134
137
  synchronization_transaction { handler.call(:synchronization, objects, page) }
135
138
 
136
139
  if synchronization_per_page && objects.any?
137
140
  synchronization_page_update!(page)
138
- Artery.logger.debug "PAGE #{page} RECEIVED, WILL CONTINUE..."
141
+ Artery::Instrumentation.instrument(:sync, stage: :page, route: all_uri.to_route, page: page)
139
142
  should_continue = true
140
143
  else
141
144
  synchronization_page_update!(nil) if synchronization_per_page
@@ -183,8 +186,6 @@ module Artery
183
186
 
184
187
  Artery.request updates_uri.to_route, updates_data do |on|
185
188
  on.success do |data|
186
- Artery.logger.debug "HEY-HEY, LAST_UPDATES: <#{updates_uri.to_route}> #{[data].inspect}"
187
-
188
189
  updates = data[:updates].map(&:with_indifferent_access)
189
190
  synchronization_transaction do
190
191
  updates.sort_by { |u| u[:_index] }.each do |update|
@@ -196,7 +197,7 @@ module Artery
196
197
  end
197
198
 
198
199
  if data[:_continue]
199
- Artery.logger.debug 'NOT ALL UPDATES RECEIVED, WILL CONTINUE...'
200
+ Artery::Instrumentation.instrument(:sync, stage: :continue)
200
201
  should_continue = true
201
202
  else
202
203
  synchronization_in_progress!(false)
@@ -220,6 +221,12 @@ module Artery
220
221
  end
221
222
  :continue if should_continue
222
223
  end
224
+
225
+ def reset_latest_index!
226
+ return unless latest_message_index.positive?
227
+
228
+ info.update!(latest_index: 0)
229
+ end
223
230
  end
224
231
  end
225
232
  end
@@ -60,39 +60,43 @@ module Artery
60
60
  end
61
61
 
62
62
  def handle(message) # rubocop:disable Metrics/AbcSize
63
- Artery.logger.push_tags message.reply
64
- Artery.logger.debug "GOT MESSAGE: #{message.inspect}"
65
-
66
- info.lock_for_message(message) do
67
- if !message.from_updates? && synchronization_in_progress?
68
- Artery.logger.debug 'SKIPPING MESSAGE RECEIVED WHILE SYNC IN PROGRESS'
69
- return
70
- end
71
- return if !message.from_updates? && !validate_index(message)
63
+ request_id = message.reply || SecureRandom.hex(8)
64
+ Artery.logger.tagged(request_id) do
65
+ Artery::Instrumentation.instrument(
66
+ :message, stage: :received, route: message.from, data: message.data, request_id: request_id
67
+ )
68
+
69
+ info.lock_for_message(message) do
70
+ if !message.from_updates? && synchronization_in_progress?
71
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'sync in progress')
72
+ return
73
+ end
74
+ return if !message.from_updates? && !validate_index(message)
72
75
 
73
- if message.update_by_us?
74
- Artery.logger.debug 'SKIPPING UPDATE MADE BY US'
75
- update_info_by_message!(message)
76
- return
77
- end
76
+ if message.update_by_us?
77
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'update by us')
78
+ update_info_by_message!(message)
79
+ return
80
+ end
78
81
 
79
- unless handler.has_block?(message.action) || handler.has_block?(:_default)
80
- Artery.logger.debug 'SKIPPING MESSAGE WE ARE NOT LISTENING TO'
81
- update_info_by_message!(message)
82
- return
83
- end
82
+ unless handler.has_block?(message.action) || handler.has_block?(:_default)
83
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'no listener for action')
84
+ update_info_by_message!(message)
85
+ return
86
+ end
84
87
 
85
- case message.action
86
- when :create, :update
87
- message.enrich_data do |attributes|
88
- handle_data(message, attributes)
88
+ Artery::Instrumentation.instrument(:message, stage: :handled, route: message.from, request_id: request_id) do
89
+ case message.action
90
+ when :create, :update
91
+ message.enrich_data do |attributes|
92
+ handle_data(message, attributes)
93
+ end
94
+ else
95
+ handle_data(message)
96
+ end
89
97
  end
90
- else
91
- handle_data(message)
92
98
  end
93
99
  end
94
- ensure
95
- Artery.logger.pop_tags
96
100
  end
97
101
 
98
102
  protected
@@ -101,13 +105,11 @@ module Artery
101
105
  return true unless message.previous_index.positive? && latest_message_index.positive?
102
106
 
103
107
  if message.previous_index > latest_message_index
104
- Artery.logger.debug 'WE\'VE GOT FUTURE MESSAGE, REQUESTING ALL MISSED'
105
-
106
- synchronize! # this will include current message
108
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'future message, requesting missed')
109
+ synchronize!
107
110
  false
108
111
  elsif message.previous_index < latest_message_index
109
- Artery.logger.debug 'WE\'VE GOT PREVIOUS MESSAGE AND ALREADY HANDLED IT, SKIPPING'
110
-
112
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'duplicate message, already handled')
111
113
  false
112
114
  else
113
115
  true
@@ -118,8 +120,8 @@ module Artery
118
120
  data ||= message.data
119
121
 
120
122
  info.lock_for_message(message) do
121
- if data == :not_found # special data when enrich failed with not_found
122
- Artery.logger.debug 'SKIP HANDLING MESSAGE BECAUSE ENRICH DATA IS BLANK'
123
+ if data == :not_found
124
+ Artery::Instrumentation.instrument(:message, stage: :skipped, reason: 'enrich data not found')
123
125
  else
124
126
  handler.call(:_before_action, message.action, data, message.reply, message.from)
125
127
 
data/lib/artery/sync.rb CHANGED
@@ -25,11 +25,11 @@ module Artery
25
25
 
26
26
  def self.run(subscriptions)
27
27
  sync_id = SecureRandom.hex
28
- Artery.logger.push_tags('Sync', sync_id)
29
- Artery::Sync.new(sync_id).execute subscriptions
30
- ensure
31
- Artery.clear_synchronizing_subscriptions!
32
- Artery.logger.pop_tags
28
+ Artery.logger.tagged('Sync', sync_id) do
29
+ Artery::Sync.new(sync_id).execute subscriptions
30
+ ensure
31
+ Artery.clear_synchronizing_subscriptions!
32
+ end
33
33
  end
34
34
  end
35
35
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Artery
4
- VERSION = '1.2.1'
4
+ VERSION = '1.3.0'
5
5
  end
data/lib/artery/worker.rb CHANGED
@@ -37,38 +37,40 @@ module Artery
37
37
  private
38
38
 
39
39
  def worker_cycle(services, subscriptions_on_services)
40
- Artery.logger.push_tags 'Worker', worker_id
41
- tries = 0
42
- begin
43
- subscribe_healthz
44
-
45
- @sync.execute services
46
-
47
- subscriptions_on_services.each do |uri, subscriptions|
48
- Artery.logger.debug "Subscribing on '#{uri}'"
49
- Artery.subscribe uri.to_route, queue: "#{Artery.service_name}.worker" do |data, reply, from|
50
- subscriptions.each do |subscription|
51
- message = Subscription::IncomingMessage.new subscription, data, reply, from
52
-
53
- subscription.handle(message)
54
- rescue StandardError => e
55
- Artery.handle_error Error.new("Error in subscription handling: #{e.inspect}",
56
- original_exception: e,
57
- subscription: {
58
- subscriber: subscription.subscriber.to_s,
59
- route: from,
60
- data: data.to_json
61
- })
40
+ Artery.logger.tagged('Worker', worker_id) do
41
+ Artery::Instrumentation.instrument(:worker, action: :started, worker_id: worker_id)
42
+ tries = 0
43
+ begin
44
+ subscribe_healthz
45
+
46
+ @sync.execute services
47
+
48
+ subscriptions_on_services.each do |uri, subscriptions|
49
+ Artery::Instrumentation.instrument(:worker, action: :subscribing, route: uri.to_s)
50
+ Artery.subscribe uri.to_route, queue: "#{Artery.service_name}.worker" do |data, reply, from|
51
+ subscriptions.each do |subscription|
52
+ message = Subscription::IncomingMessage.new subscription, data, reply, from
53
+
54
+ subscription.handle(message)
55
+ rescue StandardError => e
56
+ Artery.handle_error Error.new("Error in subscription handling: #{e.inspect}",
57
+ original_exception: e,
58
+ subscription: {
59
+ subscriber: subscription.subscriber.to_s,
60
+ route: from,
61
+ data: data.to_json
62
+ })
63
+ end
62
64
  end
63
65
  end
64
- end
65
- rescue StandardError => e
66
- tries += 1
66
+ rescue StandardError => e
67
+ tries += 1
67
68
 
68
- Artery.handle_error Error.new("WORKER ERROR: #{e.inspect}", original_exception: e)
69
- retry if tries <= 5
69
+ Artery.handle_error Error.new("WORKER ERROR: #{e.inspect}", original_exception: e)
70
+ retry if tries <= 5
70
71
 
71
- Artery.handle_error Error.new('Worker failed 5 times and exited.')
72
+ Artery.handle_error Error.new('Worker failed 5 times and exited.')
73
+ end
72
74
  end
73
75
  end
74
76
  end
data/lib/artery.rb CHANGED
@@ -4,6 +4,8 @@ require_relative 'artery/engine' if defined?(Rails)
4
4
 
5
5
  require_relative 'artery/errors'
6
6
  require_relative 'artery/backends/base'
7
+ require_relative 'artery/instrumentation'
8
+ require_relative 'artery/log_subscriber'
7
9
 
8
10
  require 'multiblock'
9
11
  require_relative 'multiblock_has_block'
@@ -44,7 +46,7 @@ module Artery
44
46
  def handle_signals
45
47
  %w[TERM INT].each do |sig|
46
48
  trap(sig) do
47
- puts "Caught #{sig} signal, exiting..."
49
+ Artery.logger.info "Caught #{sig} signal, exiting..."
48
50
 
49
51
  yield if block_given?
50
52
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: artery
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.2.1
4
+ version: 1.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sergey Gnuskov
@@ -100,6 +100,7 @@ files:
100
100
  - db/migrate/20200109120304_add_index_on_model_to_artery_messages.rb
101
101
  - db/migrate/20200109120305_remove_last_message_at_from_artery_subscription_infos.rb
102
102
  - db/migrate/20240411120304_add_synchronization_heartbeat_to_artery_subscription_infos.rb
103
+ - db/migrate/20250320120000_create_artery_model_infos.rb
103
104
  - exe/artery-check
104
105
  - exe/artery-clean
105
106
  - exe/artery-sync
@@ -107,6 +108,7 @@ files:
107
108
  - lib/artery.rb
108
109
  - lib/artery/active_record.rb
109
110
  - lib/artery/active_record/message.rb
111
+ - lib/artery/active_record/model_info.rb
110
112
  - lib/artery/active_record/subscription_info.rb
111
113
  - lib/artery/backend.rb
112
114
  - lib/artery/backends/base.rb
@@ -117,6 +119,8 @@ files:
117
119
  - lib/artery/engine.rb
118
120
  - lib/artery/errors.rb
119
121
  - lib/artery/healthz_subscription.rb
122
+ - lib/artery/instrumentation.rb
123
+ - lib/artery/log_subscriber.rb
120
124
  - lib/artery/model.rb
121
125
  - lib/artery/model/callbacks.rb
122
126
  - lib/artery/model/subscriptions.rb