active_monitoring 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: d713464ea142fd4d40db7b8b12b786f3fd21c85d052cc9fbe2eb9fcb5ece2a87
4
+ data.tar.gz: 94c592b0e41fe8be9c609525eb429979cb67f382179baaae902abcd4b256b031
5
+ SHA512:
6
+ metadata.gz: 56afc9fc77d56d52437671acfe644308e83f13d1a0fe91b2b38ded82b0f007156eec4784d75d6ba041c57bed97ba301a3334bb1dd88a8ce4a4e824993d01ff02
7
+ data.tar.gz: b1b763e859633c3255693fa5b3453cadccf6184788f5482f18913bbcc8c7cc8dc88d1429fe4a9fa50b721f646ca39a4befb36fb20fc1f76a27122803a87b63fb
data/MIT-LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright 2020 Christian Bruckmayer
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,182 @@
1
+ # ActiveMonitoring
2
+
3
+ I gave this talk at [RailsConf 2020 Couch Edition](https://railsconf.com/) and this blog article contains the source code and step by step tutorial.
4
+ The talk and tutorial is inspired by my work on [influxdb-rails](https://github.com/influxdata/influxdb-rails) and contains several patterns and learnings from it.
5
+
6
+ By the end of this tutorial, we will have a very basic Performance Monitoring tool which measures the response and SQL query time of a Rails application.
7
+
8
+ ## Abstract
9
+ > Building a Performance Analytics Tool with ActiveSupport
10
+
11
+ > Setting up a performance analytics tool like NewRelic or Skylight is one of the first things many developers do in a new project. However, have you ever wondered how these tools work under the hood?
12
+
13
+ > In this talk, we will build a basic performance analytics tool from scratch. We will deep dive into ActiveSupport instrumentations to collect, group and normalise metrics from your app. To persist these metrics, we will use a time series database and visualise the data on a dashboard. By the end of the talk, you will know how your favourite performance analytics tool works.
14
+
15
+ ## Chapter 1: Collecting data
16
+ ### Notification Framework
17
+ As a first step we need to implement a notification framework which we can use to instrument and subscribe to performance metrics.
18
+
19
+ A basic example of the functionality of our notification framework would be this code:
20
+ ```ruby
21
+ scenario "It can susbcribe and instrument events" do
22
+ events = []
23
+ ActiveMonitoring::Notifications.subscribe("test_event") do |name, start, finish, id, payload|
24
+ events << { name: name, start: start, finish: finish, id: id, payload: payload }
25
+ end
26
+
27
+ ActiveMonitoring::Notifications.instrument("test_event", payload: :payload) do
28
+ 1 + 1
29
+ end
30
+
31
+ expect(metrics[0].name).to eq("test_event")
32
+ expect(metrics[0].payload).to include(payload: :payload)
33
+ end
34
+ ```
35
+
36
+ The implementation can be found in commit [32c3](https://github.com/ChrisBr/active_monitoring/commit/32c32dcec7529ac7a8e0125b1aaab28cd4219363).
37
+ For the sake of simplicity, this framework is not thread safe of course.
38
+
39
+ This framework is inspired by [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) which already provides various hooks into Rails internals.
40
+ The implementatition of Active Support Notifications can be found [here](https://github.com/rails/rails/tree/master/activesupport/lib/active_support/notifications).
41
+
42
+ ### Instrument events
43
+ [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) provides already
44
+ several hooks to subscribe to e.g. processing of controller actions, SQL queries or view rendering.
45
+ However, Rails does not provide hooks for external libraries like Redis or Memcache.
46
+ In [ef20](https://github.com/ChrisBr/active_monitoring/commit/ef200ae5bf58f9afd2a69003e2c3620d0d9c86eb) and [46c3](https://github.com/ChrisBr/active_monitoring/commit/46c338f8fa5870bae7dc0e251e637d1d4be551db) we monkey patch Rails
47
+ to subscribe to controller actions and SQL queries.
48
+
49
+ ## Chapter 2: Storage
50
+ In chapter two we discussed an appropriate storage engine for our data.
51
+ Our data would look for example like this
52
+
53
+ | Timestamp | Hook | Location | Request ID | Value |
54
+ | ------------- | ------------- |------------- |------------- |------------- |
55
+ | 01.04.2020 00:31:30 CET | process_action | BooksController#show | cf6d6dcd | 100 ms
56
+ | 01.04.2020 00:32:30 CET | sql | BooksController#show | c7c7c841 | 80 ms
57
+
58
+ This data looks tabular, so using a relational database to store it would make sense.
59
+ However, there are several downsides to this approach.
60
+
61
+ ### Relational Database
62
+
63
+ Relational databases perform very well with applications which map to the basic CRUD operations of the database like a Rails controller.
64
+
65
+ ```ruby
66
+ class BooksController < ApplicationController
67
+ def new; end
68
+ def create; end
69
+ def show; end
70
+ def edit; end
71
+ def update; end
72
+ def destroy; end
73
+ end
74
+ ```
75
+
76
+ However, our plugin does not really use all of these traditional database operations or in a different way we would in a normal Rails app.
77
+
78
+ #### Create
79
+ For each request, we will write one process action and several SQL query data points.
80
+ If we extend this application to also cover view render times or background jobs, it's not unlikely we would write 10+ metrics per request.
81
+ Depending of how many requests your app does process, we could easily create millions of metrics per year.
82
+
83
+ #### Read
84
+ We will read the metrics in a dashboard for a certain time range like today or last three hours but not randomly.
85
+
86
+ #### Update
87
+ We will almost never go back to update on of the metrics.
88
+
89
+ #### Delete
90
+ We will only delete metrics in batches but not single metrics.
91
+ For instance, we are only interested in metrics for the last three months so once a month we can do a 'garbage collection' and delete metrics older than three months.
92
+
93
+ ### Time Series Database
94
+
95
+ With this all said, a relational database might not be the best storage engine for this kind of data as
96
+
97
+ * We will heavily write to the database which would cause to frequently create new pages in a [B-Tree](https://en.wikipedia.org/wiki/B-tree) like most relational databases use internally
98
+ * We would need to implement some sort of garbage collection to remove old data
99
+ * Compression of data would not be very efficient
100
+
101
+ For this kind of data, a [log based storage engine](https://en.wikipedia.org/wiki/Log-structured_merge-tree) like most [Time Series databases](https://db-engines.com/en/ranking/time+series+dbms) implement it would be more efficient.
102
+
103
+ However, for the sake of simplicity, we store our metrics in a relational database in this tutorial, the implementation can be found in [5967](https://github.com/ChrisBr/active_monitoring/commit/5967190a1484009476f2378c02e3e0ea96d21624).
104
+
105
+ ## Chapter 3: Cleaning, Normalizing and Grouping
106
+ In the previous chapter we already stored the controller metrics.
107
+ Before we can store SQL metrics, we need to do some more additional work.
108
+
109
+ ### Cleaning
110
+ ActiveRecord does a lot of 'magic' behind the scenes to abstract persisting objects to the database.
111
+ Some of this 'magic' requires to execute additional database queries, for instance
112
+
113
+ * Current version of the database engine
114
+ * Which migrations did already run
115
+ * Which environment
116
+
117
+ We're only interested to store application metrics so we need to filter these queries out.
118
+ The implementation can be found in [156d](https://github.com/ChrisBr/active_monitoring/commit/156d8f7fb0bd3b013524a20effc65dad05506b3d).
119
+
120
+ ### Normalizing
121
+ Depending on the database adapter you use, the queries might contain values.
122
+ To group the same queries together, we need to normalize these queries.
123
+
124
+ ```SQL
125
+ SELECT * FROM books WHERE id = 1;
126
+ SELECT * FROM books WHERE id = 2;
127
+ ```
128
+
129
+ For our performance monitoring tool, these two queries should be treated the same and we need to normalize them to
130
+ ```SQL
131
+ SELECT * FROM books WHERE id = xxx;
132
+ ```
133
+
134
+ In [66a8](https://github.com/ChrisBr/active_monitoring/commit/66a833a46b40072a8888fb6b6c6bb02ee60bda17) we implement a simple query normalizer.
135
+
136
+ ### Grouping
137
+ ActiveRecord is a standalone framework, therefore the payload of the SQL event does not contain a `request_id`.
138
+ We can use ActiveRecord for instance in migrations or background jobs so it is perfectly valid to use it outside of a request response cycle.
139
+ However, in our Performance Monitoring tool we would like to group requests and SQL queries together so we can see if a query causes a slow response.
140
+
141
+ Luckily, we also implemented a `start_processing` event in [ef20](https://github.com/ChrisBr/active_monitoring/commit/ef200ae5bf58f9afd2a69003e2c3620d0d9c86eb).
142
+ We can now subscribe to this event and set the `request_id` and `location` in a cache which we later read
143
+ in when writing the SQL metrics.
144
+ In [b1f1](https://github.com/ChrisBr/active_monitoring/commit/b1f1847270935beacc236bc54535ced1eb83c5ef) we implement a `CurrentAttributes` class and eventually write the SQL metrics.
145
+
146
+ ActiveSupport ships with [CurrentAttributes](https://github.com/rails/rails/blob/157920aead96865e3135f496c09ace607d5620dc/activesupport/lib/active_support/current_attributes.rb) out of the box since Rails 5.
147
+
148
+ ## Chapter 4: Visualization
149
+ So we now have several metrics in our data base written which would look something like
150
+
151
+ | Timestamp | Hook | Location | Request ID | Value |
152
+ | ------------- | ------------- |------------- |------------- |------------- |
153
+ | 01.04.2020 00:31:30 CET | process_action | BooksController#show | cf6d6dcd | 100 ms
154
+ | 01.04.2020 00:32:30 CET | sql | BooksController#show | c7c7c841 | 80 ms
155
+
156
+ It would be very hard to spot now problems or discover pattern so eventually we need to visualize our collected data.
157
+ In [cfca](https://github.com/ChrisBr/active_monitoring/commit/cfcafd2f4664e6927e7d297a4b7da2e18bdd5205) we implement a dashboard to show percentiles and the slowest queries of our Rails app.
158
+ A very good blog article about data visualization for performance metrics from Richard Schneemann can be found [here](https://www.schneems.com/2020/03/17/lies-damned-lies-and-averages-perc50-perc95-explained-for-programmers/).
159
+
160
+ In a real world application, I would strongly recommend to use a dashboard software like [Grafana](https://github.com/grafana/grafana).
161
+
162
+ ## Summary
163
+ Congratulations, we implemented a very basic Performance Monitoring tool in just a few hundred lines of code now.
164
+ We deep dived into [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) and hooked into Rails events to write request and SQL query metrics in our data storage.
165
+ As data storage, we compared relational databases with time series databases like InfluxDB.
166
+ Before we could visualize our data, we needed to clean, normalize and group the metrics.
167
+
168
+ From here, we can easily add more metrics like [rendering of views](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#action-view),
169
+ [caching](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#active-support) or [background jobs](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#active-job).
170
+
171
+ As initially mentioned, this tutorial is heavily influenced by our work on https://github.com/influxdata/influxdb-rails.
172
+ If this made you curious, we always look for new contributors.
173
+
174
+ ## Further information
175
+ * [Profiling and Benchmarking 101 by Nate Berkopec](https://youtu.be/XL51vf-XBTs)
176
+ * [Lies, Damned Lies, and Averages: Perc50, Perc95 explained for Programmers by Richard Schneeman](https://www.schneems.com/2020/03/17/lies-damned-lies-and-averages-perc50-perc95-explained-for-programmers/)
177
+ * https://github.com/influxdata/influxdb-ruby
178
+ * https://github.com/influxdata/influxdb-rails
179
+ * https://docs.influxdata.com/influxdb/v1.7/concepts/storage_engine/
180
+
181
+ ## License
182
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,22 @@
1
+ begin
2
+ require "bundler/setup"
3
+ rescue LoadError
4
+ puts "You must `gem install bundler` and `bundle install` to run rake tasks"
5
+ end
6
+
7
+ require "rdoc/task"
8
+
9
+ RDoc::Task.new(:rdoc) do |rdoc|
10
+ rdoc.rdoc_dir = "rdoc"
11
+ rdoc.title = "ActiveMonitoring"
12
+ rdoc.options << "--line-numbers"
13
+ rdoc.rdoc_files.include("README.md")
14
+ rdoc.rdoc_files.include("lib/**/*.rb")
15
+ end
16
+
17
+ APP_RAKEFILE = File.expand_path("spec/dummy/Rakefile", __dir__)
18
+ load "rails/tasks/engine.rake"
19
+
20
+ load "rails/tasks/statistics.rake"
21
+
22
+ require "bundler/gem_tasks"
@@ -0,0 +1,5 @@
1
+ module ActiveMonitoring
2
+ class ApplicationController < ActionController::Base
3
+ protect_from_forgery with: :exception
4
+ end
5
+ end
@@ -0,0 +1,9 @@
1
+ require_dependency "active_monitoring/application_controller"
2
+
3
+ module ActiveMonitoring
4
+ class DashboardController < ApplicationController
5
+ def show
6
+ @dashboard = Dashboard.new
7
+ end
8
+ end
9
+ end
@@ -0,0 +1,5 @@
1
+ module ActiveMonitoring
2
+ class ApplicationRecord < ActiveRecord::Base
3
+ self.abstract_class = true
4
+ end
5
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class Dashboard
3
+ LIMIT = 10
4
+
5
+ def initialize(date = Date.current)
6
+ @date = date
7
+ end
8
+
9
+ def percentile(value)
10
+ response_metrics.percentile(value)
11
+ end
12
+
13
+ def slow_sql_queries
14
+ sql_metrics.order(:value).limit(LIMIT)
15
+ end
16
+
17
+ private
18
+
19
+ attr_reader :date
20
+
21
+ def sql_metrics
22
+ metrics.where(name: "sql.active_record")
23
+ end
24
+
25
+ def response_metrics
26
+ metrics.where(name: "process_action.action_controller")
27
+ end
28
+
29
+ def metrics
30
+ Metric.where(created_at: date.beginning_of_day..date.end_of_day)
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,7 @@
1
+ module ActiveMonitoring
2
+ class Metric < ApplicationRecord
3
+ def self.percentile(value)
4
+ order(:value).offset(count * value / 10 - 1).limit(1).pluck(:value).first
5
+ end
6
+ end
7
+ end
@@ -0,0 +1,23 @@
1
+ <h1>Dashboard</h1>
2
+
3
+ <h2>Response time</h2>
4
+ <ul>
5
+ <li>90th Percentile: <%= @dashboard.percentile(9) %><li>
6
+ <li>50th Percentile: <%= @dashboard.percentile(5) %><li>
7
+ </ul>
8
+
9
+ <h2>Slow queriess</h2>
10
+ <table>
11
+ <tr>
12
+ <th>SQL</th>
13
+ <th>Location</th>
14
+ <th>Time</th>
15
+ </tr>
16
+ <% @dashboard.slow_sql_queries.each do |metric| %>
17
+ <tr>
18
+ <td><%= metric.sql_query %></td>
19
+ <td><%= metric.location %></td>
20
+ <td><%= metric.value %></td>
21
+ </tr>
22
+ <% end %>
23
+ <table>
@@ -0,0 +1,13 @@
1
+ <!DOCTYPE html>
2
+ <html>
3
+ <head>
4
+ <title>Active monitoring</title>
5
+ <%= csrf_meta_tags %>
6
+ <%= csp_meta_tag %>
7
+ </head>
8
+ <body>
9
+
10
+ <%= yield %>
11
+
12
+ </body>
13
+ </html>
data/config/routes.rb ADDED
@@ -0,0 +1,3 @@
1
+ ActiveMonitoring::Engine.routes.draw do
2
+ resource :dashboard, controller: :dashboard, only: :show
3
+ end
@@ -0,0 +1,13 @@
1
+ class CreateActiveMonitoringMetrics < ActiveRecord::Migration[6.0]
2
+ def change
3
+ create_table :active_monitoring_metrics do |t|
4
+ t.string :name
5
+ t.string :request_id
6
+ t.string :location
7
+ t.string :sql_query
8
+ t.integer :value
9
+
10
+ t.timestamps
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,6 @@
1
+ require "active_monitoring/engine"
2
+ require "active_monitoring/notifications"
3
+
4
+ module ActiveMonitoring
5
+ # Your code goes here...
6
+ end
@@ -0,0 +1,19 @@
1
+ module ActiveMonitoring
2
+ module CoreExtensions
3
+ module ActionController
4
+ module Instrumentation
5
+ def process_action(*)
6
+ payload = {
7
+ controller: self.class.name,
8
+ action: action_name,
9
+ request_id: request.uuid
10
+ }
11
+ ::ActiveMonitoring::Notifications.instrument("start_processing.action_controller", payload)
12
+ ::ActiveMonitoring::Notifications.instrument("process_action.action_controller", payload) do
13
+ super
14
+ end
15
+ end
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,13 @@
1
+ module ActiveMonitoring
2
+ module CoreExtensions
3
+ module ActiveRecord
4
+ module Instrumentation
5
+ def log(sql, name = "SQL", binds = [], type_casted_binds = [], statement_name = nil)
6
+ ::ActiveMonitoring::Notifications.instrument("sql.active_record", sql: sql, name: name) do
7
+ super
8
+ end
9
+ end
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,31 @@
1
+ require_relative "core_extensions/action_controller"
2
+ require_relative "core_extensions/active_record"
3
+ require_relative "sql_query"
4
+ require_relative "current"
5
+
6
+ module ActiveMonitoring
7
+ class Current
8
+ class << self
9
+ def request_id
10
+ store[:request_id]
11
+ end
12
+
13
+ def request_id=(value)
14
+ store[:request_id] = value
15
+ end
16
+
17
+ def location
18
+ store[:location]
19
+ end
20
+
21
+ def location=(value)
22
+ store[:location] = value
23
+ end
24
+
25
+ def store
26
+ Thread.current[:active_monitoring_store] ||= {}
27
+ Thread.current[:active_monitoring_store]
28
+ end
29
+ end
30
+ end
31
+ end
@@ -0,0 +1,47 @@
1
+ require_relative "core_extensions/action_controller"
2
+ require_relative "core_extensions/active_record"
3
+ require_relative "sql_query"
4
+ require_relative "current"
5
+
6
+ module ActiveMonitoring
7
+ class Engine < ::Rails::Engine
8
+ isolate_namespace ActiveMonitoring
9
+
10
+ ActiveSupport.on_load(:active_record) do
11
+ ActiveRecord::ConnectionAdapters::AbstractAdapter.prepend(::ActiveMonitoring::CoreExtensions::ActiveRecord::Instrumentation)
12
+ end
13
+
14
+ ActiveSupport.on_load(:action_controller) do
15
+ ActionController::Base.prepend(::ActiveMonitoring::CoreExtensions::ActionController::Instrumentation)
16
+
17
+ ActiveMonitoring::Notifications.subscribe("start_processing.action_controller") do |_, _, _, _, payload|
18
+ Current.request_id = payload[:request_id]
19
+ Current.location = "#{payload[:controller]}##{payload[:action]}"
20
+ end
21
+
22
+ ActiveMonitoring::Notifications.subscribe("process_action.action_controller") do |name, start, finish, _id, payload|
23
+ Metric.create(
24
+ name: name,
25
+ value: finish - start,
26
+ request_id: payload[:request_id],
27
+ location: "#{payload[:controller]}##{payload[:action]}",
28
+ created_at: finish
29
+ )
30
+ end
31
+
32
+ ActiveMonitoring::Notifications.subscribe("sql.active_record") do |name, start, finish, _id, payload|
33
+ query = ActiveMonitoring::SqlQuery.new(name: payload[:name].dup, query: payload[:sql].dup)
34
+ if query.track?
35
+ Metric.create(
36
+ name: name,
37
+ value: finish - start,
38
+ request_id: Current.request_id,
39
+ sql_query: query.normalized_query,
40
+ location: Current.location,
41
+ created_at: finish
42
+ )
43
+ end
44
+ end
45
+ end
46
+ end
47
+ end
@@ -0,0 +1,19 @@
1
+ require_relative "notifier"
2
+
3
+ module ActiveMonitoring
4
+ class Notifications
5
+ class << self
6
+ attr_accessor :notifier
7
+
8
+ def subscribe(name, &block)
9
+ notifier.subscribe(name, &block)
10
+ end
11
+
12
+ def instrument(name, payload = {}, &block)
13
+ notifier.instrument(name, payload, &block)
14
+ end
15
+ end
16
+
17
+ self.notifier = ActiveMonitoring::Notifier.new
18
+ end
19
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class Notifier
3
+ def initialize
4
+ @subscribers = {}
5
+ end
6
+
7
+ def subscribe(name, &block)
8
+ subscribers[name] ||= []
9
+ subscribers[name] << block
10
+ end
11
+
12
+ def instrument(name, payload)
13
+ id = SecureRandom.hex(10)
14
+ start = Time.current
15
+ result = yield if block_given?
16
+ finish = Time.current
17
+
18
+ subscribers_for(name).each do |callback|
19
+ callback.call(name, start, finish, id, payload)
20
+ end
21
+
22
+ result
23
+ end
24
+
25
+ private
26
+
27
+ attr_reader :subscribers
28
+
29
+ def subscribers_for(name)
30
+ subscribers[name].to_a
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,23 @@
1
+ module ActiveMonitoring
2
+ class SqlNormalizer
3
+ def initialize(query:)
4
+ @query = query
5
+ end
6
+
7
+ def to_s
8
+ query.squish!
9
+ query.gsub!(/(\s(=|>|<|>=|<=|<>|!=)\s)('[^']+'|[\$\+\-\w\.]+)/, '\1xxx')
10
+ query.gsub!(/(\sIN\s)\([^\(\)]+\)/i, '\1(xxx)')
11
+ regex = /(\sBETWEEN\s)('[^']+'|[\+\-\w\.]+)(\sAND\s)('[^']+'|[\+\-\w\.]+)/i
12
+ query.gsub!(regex, '\1xxx\3xxx')
13
+ query.gsub!(/(\sVALUES\s)\(.+\)/i, '\1(xxx)')
14
+ query.gsub!(/(\s(LIKE|ILIKE|SIMILAR TO|NOT SIMILAR TO)\s)('[^']+')/i, '\1xxx')
15
+ query.gsub!(/(\s(LIMIT|OFFSET)\s)(\d+)/i, '\1xxx')
16
+ query
17
+ end
18
+
19
+ private
20
+
21
+ attr_reader :query
22
+ end
23
+ end
@@ -0,0 +1,23 @@
1
+ require_relative "sql_tracker"
2
+ require_relative "sql_normalizer"
3
+
4
+ module ActiveMonitoring
5
+ class SqlQuery
6
+ def initialize(query:, name:)
7
+ @query = query
8
+ @name = name
9
+ end
10
+
11
+ def track?
12
+ SqlTracker.new(query: query, name: name).track?
13
+ end
14
+
15
+ def normalized_query
16
+ SqlNormalizer.new(query: query).to_s
17
+ end
18
+
19
+ private
20
+
21
+ attr_reader :query, :name
22
+ end
23
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class SqlTracker
3
+ TRACKED_SQL_COMMANDS = %w(SELECT INSERT UPDATE DELETE).freeze
4
+ UNTRACKED_NAMES = %w(SCHEMA).freeze
5
+ UNTRACKED_TABLES = %w(
6
+ SCHEMA_MIGRATIONS
7
+ SQLITE_MASTER
8
+ ACTIVE_MONITORING_METRICS
9
+ SQLITE_TEMP_MASTER
10
+ SQLITE_VERSION
11
+ AR_INTERNAL_METADATA
12
+ ).freeze
13
+
14
+ def initialize(query:, name:)
15
+ @query = query.to_s.upcase
16
+ @name = name.to_s.upcase
17
+ end
18
+
19
+ def track?
20
+ query.start_with?(*TRACKED_SQL_COMMANDS) &&
21
+ !name.start_with?(*UNTRACKED_NAMES) &&
22
+ !untracked_tables?
23
+ end
24
+
25
+ private
26
+
27
+ attr_reader :query, :name
28
+
29
+ def untracked_tables?
30
+ UNTRACKED_TABLES.any? { |table| query.include?(table) }
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,3 @@
1
+ module ActiveMonitoring
2
+ VERSION = "0.1.0".freeze
3
+ end
metadata ADDED
@@ -0,0 +1,158 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: active_monitoring
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Christian Bruckmayer
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2020-04-17 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: rails
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: 6.0.2
20
+ - - ">="
21
+ - !ruby/object:Gem::Version
22
+ version: 6.0.2.2
23
+ type: :runtime
24
+ prerelease: false
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - "~>"
28
+ - !ruby/object:Gem::Version
29
+ version: 6.0.2
30
+ - - ">="
31
+ - !ruby/object:Gem::Version
32
+ version: 6.0.2.2
33
+ - !ruby/object:Gem::Dependency
34
+ name: byebug
35
+ requirement: !ruby/object:Gem::Requirement
36
+ requirements:
37
+ - - ">="
38
+ - !ruby/object:Gem::Version
39
+ version: '0'
40
+ type: :development
41
+ prerelease: false
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - ">="
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ - !ruby/object:Gem::Dependency
48
+ name: rspec-rails
49
+ requirement: !ruby/object:Gem::Requirement
50
+ requirements:
51
+ - - ">="
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :development
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - ">="
59
+ - !ruby/object:Gem::Version
60
+ version: '0'
61
+ - !ruby/object:Gem::Dependency
62
+ name: rubocop-performance
63
+ requirement: !ruby/object:Gem::Requirement
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: '0'
68
+ type: :development
69
+ prerelease: false
70
+ version_requirements: !ruby/object:Gem::Requirement
71
+ requirements:
72
+ - - ">="
73
+ - !ruby/object:Gem::Version
74
+ version: '0'
75
+ - !ruby/object:Gem::Dependency
76
+ name: rubocop-rails
77
+ requirement: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - ">="
80
+ - !ruby/object:Gem::Version
81
+ version: '0'
82
+ type: :development
83
+ prerelease: false
84
+ version_requirements: !ruby/object:Gem::Requirement
85
+ requirements:
86
+ - - ">="
87
+ - !ruby/object:Gem::Version
88
+ version: '0'
89
+ - !ruby/object:Gem::Dependency
90
+ name: sqlite3
91
+ requirement: !ruby/object:Gem::Requirement
92
+ requirements:
93
+ - - ">="
94
+ - !ruby/object:Gem::Version
95
+ version: '0'
96
+ type: :development
97
+ prerelease: false
98
+ version_requirements: !ruby/object:Gem::Requirement
99
+ requirements:
100
+ - - ">="
101
+ - !ruby/object:Gem::Version
102
+ version: '0'
103
+ description: his is an example gem used in my RailsConf 2020 talk. Please don't use
104
+ in production! https://bruckmayer.net/rails-conf-2020
105
+ email:
106
+ - christian@bruckmayer.net
107
+ executables: []
108
+ extensions: []
109
+ extra_rdoc_files: []
110
+ files:
111
+ - MIT-LICENSE
112
+ - README.md
113
+ - Rakefile
114
+ - app/controllers/active_monitoring/application_controller.rb
115
+ - app/controllers/active_monitoring/dashboard_controller.rb
116
+ - app/models/active_monitoring/application_record.rb
117
+ - app/models/active_monitoring/dashboard.rb
118
+ - app/models/active_monitoring/metric.rb
119
+ - app/views/active_monitoring/dashboard/show.html.erb
120
+ - app/views/layouts/active_monitoring/application.html.erb
121
+ - config/routes.rb
122
+ - db/migrate/20200401191741_create_active_monitoring_metrics.rb
123
+ - lib/active_monitoring.rb
124
+ - lib/active_monitoring/core_extensions/action_controller.rb
125
+ - lib/active_monitoring/core_extensions/active_record.rb
126
+ - lib/active_monitoring/current.rb
127
+ - lib/active_monitoring/engine.rb
128
+ - lib/active_monitoring/notifications.rb
129
+ - lib/active_monitoring/notifier.rb
130
+ - lib/active_monitoring/sql_normalizer.rb
131
+ - lib/active_monitoring/sql_query.rb
132
+ - lib/active_monitoring/sql_tracker.rb
133
+ - lib/active_monitoring/version.rb
134
+ homepage: https://github.com/ChrisBr/active_monitoring
135
+ licenses:
136
+ - MIT
137
+ metadata: {}
138
+ post_install_message:
139
+ rdoc_options: []
140
+ require_paths:
141
+ - lib
142
+ required_ruby_version: !ruby/object:Gem::Requirement
143
+ requirements:
144
+ - - ">="
145
+ - !ruby/object:Gem::Version
146
+ version: '0'
147
+ required_rubygems_version: !ruby/object:Gem::Requirement
148
+ requirements:
149
+ - - ">="
150
+ - !ruby/object:Gem::Version
151
+ version: '0'
152
+ requirements: []
153
+ rubygems_version: 3.1.2
154
+ signing_key:
155
+ specification_version: 4
156
+ summary: This is an example gem used in my RailsConf 2020 talk. Please don't use in
157
+ production!
158
+ test_files: []