pg-locks-monitor 0.0.1 → 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 38fe7be18449c8a637a4f6a756228055f217f4e2fa1c47eb84609009c31793cf
4
- data.tar.gz: 19e9b7bcacde528b1b0c34d4c2c28f2f482e6573742fe10536af51cbeb68db75
3
+ metadata.gz: dcbbfc8c0f63bbfd2f2c06abfb5db0e98ed2f5d6da39dc227089501f9734ec86
4
+ data.tar.gz: 370e8706b301323367dd9685be5ac5e5e8fcb7c47a73c06e6a337cf590db331c
5
5
  SHA512:
6
- metadata.gz: 9b947dcacbdaccc1d2ce67244c7c054882b9120a996ea3a8c7b3b99ae9e516065920201fa42d5c0f536fcef3296c86a6a4066c3319d464a3d726f19ca2e6e481
7
- data.tar.gz: ad98c65cf24043c9381579442731e74d26784df9886a6633913fc9c9fb1ba20e76470a53b12b11b24b588fa3d8014364194eca34f0b7c96caaa8e632e1edd66f
6
+ metadata.gz: e4ce44f0a435f65a9cbb3e69b185bcfe6939a7e4e44cb5ac687abeab7e2a7b83eb794b2407fae7fae34db152e5fd4b0332e65c01e895e851e5c2dc1a791d8a30
7
+ data.tar.gz: 585cd6b764cc4a42582b8f0eca28ec4569f8e5cb9b7175df15745aef4af1a0e5634803c4dfe06af4737e759ad8cc4d2ea9d992ef39f6fa21e0b1ee4e8bd57186
data/README.md CHANGED
@@ -1,3 +1,200 @@
1
1
  # PG Locks Monitor [![Gem Version](https://badge.fury.io/rb/pg-locks-monitor.svg)](https://badge.fury.io/rb/pg-locks-monitor) [![GH Actions](https://github.com/pawurb/pg-locks-monitor/actions/workflows/ci.yml/badge.svg)](https://github.com/pawurb/pg-locks-monitor/actions)
2
2
 
3
- WIP
3
+ This gem allows to observe database locks generated by a Rails application. By default, locks data is not persisted anywhere in the PostgreSQL logs, so the only way to monitor it is via analyzing the transient state of the `pg_locks` metadata table. `pg-locks-monitor` is a simple tool that makes this process quick to implement and adjust to each app's individual requirements.
4
+
5
+ ## Usage
6
+
7
+ `PgLocksMonitor` class provides a `snapshot!` method, which notifies selected channels about database locks that match configured criteria.
8
+
9
+ Start by adding the gem to your Gemfile:
10
+
11
+ ```ruby
12
+ gem "pg-locks-monitor"
13
+ ```
14
+
15
+ Then run `bundle install` and `rake pg_locks_monitor:init`. It creates a default configuration file with the following settings:
16
+
17
+ `config/initializers/pg_locks_monitor.rb`
18
+ ```ruby
19
+ PgLocksMonitor.configure do |config|
20
+ config.locks_limit = 5
21
+
22
+ config.monitor_locks = true
23
+ config.locks_min_duration_ms = 200
24
+
25
+ config.monitor_blocking = true
26
+ config.blocking_min_duration_ms = 100
27
+
28
+ config.notify_logs = true
29
+
30
+ config.notify_slack = false
31
+ config.slack_webhook_url = ""
32
+ config.slack_channel = ""
33
+
34
+ config.notifier_class = PgLocksMonitor::DefaultNotifier
35
+ end
36
+ ```
37
+
38
+ - `locks_limit` - specify the max number of locks to report in a single notification
39
+ - `notify_locks` - observe database locks even if they don't conflict with a different SQL query
40
+ - `locks_min_duration_ms` - notify about locks that execeed this duration threshold in milliseconds
41
+ - `notify_blocking` - observe database locks which cause other SQL query to wait from them to release
42
+ - `blocking_min_duration_ms` - notify about blocking locks that execeed this duration threshold in milliseconds
43
+ - `notify_logs` - send notifications about detected locks using `Rails.logger.info` method
44
+ - `notify_slack` - send notifications about detected locks to the configured Slack channel
45
+ - `slack_webhook_url` - webhook necessary for Slack notification to work
46
+ - `slack_channel` - the name of the target Slack channel
47
+ - `notifier_class` - customizable notifier class
48
+
49
+
50
+ ## Testing the notification channels
51
+
52
+ Before configuring a recurring invocation of the `snapshot!` method, it's recommended to first manually trigger the notification to test the configured channels.
53
+
54
+ You can generate an _"artificial"_ blocking lock and observe it by running the following code in the Rails console:
55
+
56
+ ```ruby
57
+ user = User.last
58
+
59
+ Thread.new do
60
+ User.transaction do
61
+ user.update(email: "email-#{SecureRandom.hex(2)}@example.com")
62
+ sleep 5
63
+ raise ActiveRecord::Rollback
64
+ end
65
+ end
66
+
67
+ Thread.new do
68
+ User.transaction do
69
+ user.update(email: "email-#{SecureRandom.hex(2)}@example.com")
70
+ sleep 5
71
+ raise ActiveRecord::Rollback
72
+ end
73
+ end
74
+
75
+ sleep 0.5
76
+ PgLocksMonitor.snapshot!
77
+ ```
78
+
79
+ Please remember to adjust the update operation to match your app's schema.
80
+
81
+ As a result of running the above snippet, you should receive a notification about the acquired blocking database lock.
82
+
83
+ ### Sample notification
84
+
85
+ Received notifications contain data helpful in debugging the cause of long lasting-locks.
86
+
87
+ And here's a sample blocking lock notification:
88
+
89
+ ```ruby
90
+ [
91
+ {
92
+ # PID of the process which was blocking another query
93
+ "blocking_pid": 29,
94
+ # SQL query blocking other SQL query
95
+ "blocking_statement": "UPDATE \"users\" SET \"updated_at\" = $1 WHERE \"users\".\"id\" = $2 from/sidekiq_job:UserUpdater/",
96
+ # the duration of blocking SQL query
97
+ "blocking_duration": "PT0.972116S",
98
+ # app that triggered the blocking SQL query
99
+ "blocking_sql_app": "bin/sidekiq",
100
+
101
+ # PID of the process which was blocked by another query
102
+ "blocked_pid": 30,
103
+ # SQL query blocked by other SQL query
104
+ "blocked_statement": "UPDATE \"users\" SET \"last_active_at\" = $1 WHERE \"users\".\"id\" = $2 from/controller_with_namespace:UsersController,action:update/",
105
+ # the duration of the blocked SQL query
106
+ "blocked_duration": "PT0.483309S",
107
+ # app that triggered the blocked SQL query
108
+ "blocked_sql_app": "bin/puma"
109
+ }
110
+ ]
111
+ ```
112
+
113
+ This sample blocking notification shows than a query originating from the `UserUpdater` Sidekiq job is blocking an update operation on the same user for the `UsersController#update` action. Remember to configure the [marginalia gem](https://github.com/basecamp/marginalia) to enable these helpful query source annotations.
114
+
115
+ Here's a sample lock notification:
116
+
117
+ ```ruby
118
+ [
119
+ {
120
+ # PID of the process which acquired the lock
121
+ "pid": 50,
122
+ # name of affected table/index
123
+ "relname": "users",
124
+ # ID of the source transaction
125
+ "transactionid": null,
126
+ # bool indicating if the lock is already granted
127
+ "granted": true,
128
+ # type of the acquired lock
129
+ "mode": "RowExclusiveLock",
130
+ # SQL query which acquired the lock
131
+ "query_snippet": "UPDATE \"users\" SET \"updated_at\" = $1 WHERE \"users\".\"id\" = $2 from/sidekiq_job:UserUpdater/",
132
+ # age of the lock
133
+ "age": "PT0.94945S",
134
+ # app that acquired the lock
135
+ "application": "bin/sidekiq"
136
+ },
137
+ ```
138
+
139
+ You can read [this blogpost](https://pawelurbanek.com/rails-postgresql-locks) for more detailed info on locks in the Rails apps.
140
+
141
+ ## Background job config
142
+
143
+ This gem is intended to be used via a recurring background job, but it is agnostic to the background job provider. Here's a sample Sidekiq implementation:
144
+
145
+ `app/jobs/pg_locks_monitor_job.rb`
146
+ ```ruby
147
+ require 'pg-locks-monitor'
148
+
149
+ class PgLocksMonitoringJob
150
+ include Sidekiq::Worker
151
+ sidekiq_options retry: false
152
+
153
+ def perform
154
+ PgLocksMonitor.snapshot!
155
+ ensure
156
+ if ENV["PG_LOCKS_MONITOR_ENABLED"]
157
+ PgLocksMonitoringJob.perform_in(1.minute)
158
+ end
159
+ end
160
+ end
161
+ ```
162
+
163
+ Remember to schedule this job when your app starts:
164
+ `config/pg_locks_monitor.rb`
165
+
166
+ ```ruby
167
+ #...
168
+
169
+ if ENV["PG_LOCKS_MONITOR_ENABLED"]
170
+ PgLocksMonitoringJob.perform
171
+ end
172
+ ```
173
+
174
+ A background job that schedules itself is not the cleanest pattern. So alternatively you can use [sidekiq-cron](https://github.com/sidekiq-cron/sidekiq-cron), [whenever](https://github.com/javan/whenever) or [clockwork](https://github.com/adamwiggins/clockwork) gems to trigger the `PgLocksMonitor.snapshot!` invocation periodically.
175
+
176
+ A recommended frequency of invocation depends on your app's traffic. From my experience, even 1 minute apart snapshots can provide a lot of valuable data, but it all depends on how often the locks are occurring in your Rails application.
177
+
178
+ ## Custom notifier class
179
+
180
+ `PgLocksMonitor::DefaultNotifier` supports sending lock notifications with `Rails.logger` or to a Slack channel. If you want to use different notification channels you can define your custom notifier like that:
181
+
182
+ `config/initializers/pg_locks_monitor.rb`
183
+ ```ruby
184
+ class PgLocksEmailNotifier
185
+ def self.call(locks_data)
186
+ LocksMailer.with(locks_data: locks_data).notification.deliver_now
187
+ end
188
+ end
189
+
190
+ PgLocksMonitor.configure do |config|
191
+ # ...
192
+
193
+ config.notifier_class = PgLocksEmailNotifier
194
+ end
195
+
196
+ ```
197
+
198
+ ## Contributions
199
+
200
+ This gem is in a very early stage of development so feedback and PRs are welcome.
@@ -4,4 +4,37 @@ require "uri"
4
4
  require "pg"
5
5
 
6
6
  module PgLocksMonitor
7
+ def self.snapshot!
8
+ locks = RailsPgExtras.locks(
9
+ in_format: :hash, args: {
10
+ limit: configuration.locks_limit,
11
+ },
12
+ ).select do |lock|
13
+ (ActiveSupport::Duration.parse(lock.fetch("age")).to_f * 1000) > configuration.locks_min_duration_ms
14
+ end
15
+
16
+ if locks.present? && configuration.monitor_locks
17
+ configuration.notifier_class.call(locks)
18
+ end
19
+
20
+ blocking = RailsPgExtras.blocking(in_format: :hash).select do |block|
21
+ (ActiveSupport::Duration.parse(block.fetch("blocking_duration")).to_f * 1000) > configuration.blocking_min_duration_ms
22
+ end
23
+
24
+ if blocking.present? && configuration.monitor_blocking
25
+ configuration.notifier_class.call(blocking)
26
+ end
27
+ end
28
+
29
+ def self.configuration
30
+ @configuration ||= Configuration.new(Configuration::DEFAULT)
31
+ end
32
+
33
+ def self.configure
34
+ yield(configuration)
35
+ end
7
36
  end
37
+
38
+ require "pg_locks_monitor/default_notifier"
39
+ require "pg_locks_monitor/configuration"
40
+ require "pg_locks_monitor/railtie" if defined?(Rails)
@@ -0,0 +1,49 @@
1
+ # frozen_string_literal: true
2
+
3
+ module PgLocksMonitor
4
+ class Configuration
5
+ DEFAULT = {
6
+ locks_limit: 5,
7
+ monitor_locks: true,
8
+ monitor_blocking: true,
9
+ locks_min_duration_ms: 200,
10
+ blocking_min_duration_ms: 100,
11
+ notify_logs: true,
12
+ notify_slack: false,
13
+ slack_webhook_url: nil,
14
+ slack_channel: nil,
15
+ notifier_class: PgLocksMonitor::DefaultNotifier,
16
+ }
17
+
18
+ attr_accessor *DEFAULT.keys
19
+
20
+ def initialize(attrs)
21
+ DEFAULT.keys.each do |key|
22
+ value = attrs.fetch(key) { DEFAULT.fetch(key) }
23
+ public_send("#{key}=", value)
24
+ end
25
+ end
26
+
27
+ DEFAULT_CONFIG_FILE = <<-CONFIG
28
+ # Configuration for pg-locks-monitor
29
+
30
+ PgLocksMonitor.configure do |config|
31
+ config.locks_limit = #{DEFAULT[:locks_limit]}
32
+
33
+ config.monitor_locks = #{DEFAULT[:monitor_locks]}
34
+ config.monitor_blocking = #{DEFAULT[:monitor_blocking]}
35
+
36
+ config.locks_min_duration_ms = #{DEFAULT[:locks_min_duration_ms]}
37
+ config.blocking_min_duration_ms = #{DEFAULT[:blocking_min_duration_ms]}
38
+
39
+ config.notify_logs = #{DEFAULT[:notify_logs]}
40
+
41
+ config.notify_slack = #{DEFAULT[:notify_slack]}
42
+ config.slack_webhook_url = "#{DEFAULT[:slack_webhook_url]}"
43
+ config.slack_channel = "#{DEFAULT[:slack_channel]}"
44
+
45
+ config.notifier_class = #{DEFAULT[:notifier_class]}
46
+ end
47
+ CONFIG
48
+ end
49
+ end
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "slack-notifier"
4
+
5
+ module PgLocksMonitor
6
+ class DefaultNotifier
7
+ def self.call(locks_data)
8
+ config = PgLocksMonitor.configuration
9
+
10
+ if config.notify_logs && defined?(Rails)
11
+ Rails.logger.info locks_data.to_s
12
+ end
13
+
14
+ if config.notify_slack
15
+ slack_webhook_url = config.slack_webhook_url
16
+ if slack_webhook_url.nil? || slack_webhook_url.strip.length == 0
17
+ raise "Missing pg-locks-monitor slack_webhook_url config"
18
+ end
19
+
20
+ slack_channel = config.slack_channel
21
+ if slack_channel.nil? || slack_channel.strip.length == 0
22
+ raise "Missing pg-locks-monitor slack_channel config"
23
+ end
24
+
25
+ Slack::Notifier.new(
26
+ slack_webhook_url,
27
+ channel: slack_channel,
28
+ ).ping JSON.pretty_generate(locks_data)
29
+ end
30
+ end
31
+ end
32
+ end
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ class PgLocksMonitor::Railtie < Rails::Railtie
4
+ rake_tasks do
5
+ load "pg_locks_monitor/tasks/all.rake"
6
+ end
7
+ end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pg-locks-monitor"
4
+ require "fileutils"
5
+
6
+ namespace :pg_locks_monitor do
7
+ desc "Initialize a config file"
8
+ task :init do
9
+ file_path = "config/initializers/pg_locks_monitor.rb"
10
+ if File.exist?(file_path)
11
+ puts "#{file_path} config file has already been initialized!"
12
+ else
13
+ File.write(file_path, PgLocksMonitor::Configuration::DEFAULT_CONFIG_FILE)
14
+ puts "Config file created at #{file_path}"
15
+ end
16
+ end
17
+
18
+ desc "Check for currently active locks"
19
+ task :snapshot do
20
+ PgLocksMonitor.snapshot!
21
+ end
22
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module PgLocksMonitor
4
- VERSION = "0.0.1"
4
+ VERSION = "0.1.0"
5
5
  end
@@ -8,7 +8,7 @@ Gem::Specification.new do |s|
8
8
  s.version = PgLocksMonitor::VERSION
9
9
  s.authors = ["pawurb"]
10
10
  s.email = ["contact@pawelurbanek.com"]
11
- s.summary = %q{ Observe PostgreSQL database locks obtained by your Ruby application. }
11
+ s.summary = %q{ Observe PostgreSQL database locks obtained by a Rails application. }
12
12
  s.description = %q{ This gem allows to monitor and notify about PostgreSQL database locks which meet certain criteria. You can report locks which are held for a certain amount of time, or locks which are held by a certain query. }
13
13
  s.homepage = "http://github.com/pawurb/pg-locks-monitor"
14
14
  s.files = `git ls-files`.split("\n")
@@ -16,6 +16,7 @@ Gem::Specification.new do |s|
16
16
  s.require_paths = ["lib"]
17
17
  s.license = "MIT"
18
18
  s.add_dependency "ruby-pg-extras"
19
+ s.add_dependency "slack-notifier"
19
20
  s.add_development_dependency "rake"
20
21
  s.add_development_dependency "rspec"
21
22
  s.add_development_dependency "rufo"
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "spec_helper"
4
+
5
+ describe "PgLocksMonitor::Configuration" do
6
+ it "has a default configuration" do
7
+ config = PgLocksMonitor.configuration
8
+ expect(config.monitor_locks).to eq true
9
+ expect(config.monitor_blocking).to eq true
10
+ expect(config.locks_min_duration_ms).to eq 200
11
+ expect(config.blocking_min_duration_ms).to eq 100
12
+ expect(config.notifier_class).to eq PgLocksMonitor::DefaultNotifier
13
+ end
14
+
15
+ it "can be configured" do
16
+ PgLocksMonitor.configure do |config|
17
+ config.monitor_locks = false
18
+ end
19
+
20
+ expect(PgLocksMonitor.configuration.monitor_locks).to eq false
21
+ end
22
+ end
@@ -0,0 +1,37 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "spec_helper"
4
+
5
+ describe PgLocksMonitor::DefaultNotifier do
6
+ before do
7
+ # Mock Rails and its logger
8
+ Rails = nil
9
+ logger_double = double("Logger")
10
+ allow(logger_double).to receive(:info)
11
+ allow(Rails).to receive(:logger).and_return(logger_double)
12
+ end
13
+
14
+ it "requires correct config if Slack notifications enabled" do
15
+ expect {
16
+ PgLocksMonitor::DefaultNotifier.call({})
17
+ }.not_to raise_error
18
+ PgLocksMonitor.configure do |config|
19
+ config.notify_slack = true
20
+ end
21
+
22
+ expect {
23
+ PgLocksMonitor::DefaultNotifier.call({})
24
+ }.to raise_error(RuntimeError)
25
+ end
26
+
27
+ it "sends the Slack notification if enabled" do
28
+ PgLocksMonitor.configure do |config|
29
+ config.notify_slack = true
30
+ config.slack_webhook_url = "https://hooks.slack.com/services/123456789/123456789/123456789"
31
+ config.slack_channel = "pg-locks-monitor"
32
+ end
33
+
34
+ expect_any_instance_of(Slack::Notifier).to receive(:ping)
35
+ PgLocksMonitor::DefaultNotifier.call({ locks: "data" })
36
+ end
37
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pg-locks-monitor
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - pawurb
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-08-20 00:00:00.000000000 Z
11
+ date: 2024-08-27 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: ruby-pg-extras
@@ -24,6 +24,20 @@ dependencies:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
26
  version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: slack-notifier
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
27
41
  - !ruby/object:Gem::Dependency
28
42
  name: rake
29
43
  requirement: !ruby/object:Gem::Requirement
@@ -83,8 +97,14 @@ files:
83
97
  - Rakefile
84
98
  - docker-compose.yml.sample
85
99
  - lib/pg-locks-monitor.rb
100
+ - lib/pg_locks_monitor/configuration.rb
101
+ - lib/pg_locks_monitor/default_notifier.rb
102
+ - lib/pg_locks_monitor/railtie.rb
103
+ - lib/pg_locks_monitor/tasks/all.rake
86
104
  - lib/pg_locks_monitor/version.rb
87
105
  - pg-locks-monitor.gemspec
106
+ - spec/configuration_spec.rb
107
+ - spec/default_notifier_spec.rb
88
108
  - spec/smoke_spec.rb
89
109
  - spec/spec_helper.rb
90
110
  homepage: http://github.com/pawurb/pg-locks-monitor
@@ -110,7 +130,9 @@ requirements: []
110
130
  rubygems_version: 3.5.4
111
131
  signing_key:
112
132
  specification_version: 4
113
- summary: Observe PostgreSQL database locks obtained by your Ruby application.
133
+ summary: Observe PostgreSQL database locks obtained by a Rails application.
114
134
  test_files:
135
+ - spec/configuration_spec.rb
136
+ - spec/default_notifier_spec.rb
115
137
  - spec/smoke_spec.rb
116
138
  - spec/spec_helper.rb