active_monitoring 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: d713464ea142fd4d40db7b8b12b786f3fd21c85d052cc9fbe2eb9fcb5ece2a87
4
+ data.tar.gz: 94c592b0e41fe8be9c609525eb429979cb67f382179baaae902abcd4b256b031
5
+ SHA512:
6
+ metadata.gz: 56afc9fc77d56d52437671acfe644308e83f13d1a0fe91b2b38ded82b0f007156eec4784d75d6ba041c57bed97ba301a3334bb1dd88a8ce4a4e824993d01ff02
7
+ data.tar.gz: b1b763e859633c3255693fa5b3453cadccf6184788f5482f18913bbcc8c7cc8dc88d1429fe4a9fa50b721f646ca39a4befb36fb20fc1f76a27122803a87b63fb
data/MIT-LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright 2020 Christian Bruckmayer
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,182 @@
1
+ # ActiveMonitoring
2
+
3
+ I gave this talk at [RailsConf 2020 Couch Edition](https://railsconf.com/) and this blog article contains the source code and step by step tutorial.
4
+ The talk and tutorial is inspired by my work on [influxdb-rails](https://github.com/influxdata/influxdb-rails) and contains several patterns and learnings from it.
5
+
6
+ By the end of this tutorial, we will have a very basic Performance Monitoring tool which measures the response and SQL query time of a Rails application.
7
+
8
+ ## Abstract
9
+ > Building a Performance Analytics Tool with ActiveSupport
10
+
11
+ > Setting up a performance analytics tool like NewRelic or Skylight is one of the first things many developers do in a new project. However, have you ever wondered how these tools work under the hood?
12
+
13
+ > In this talk, we will build a basic performance analytics tool from scratch. We will deep dive into ActiveSupport instrumentations to collect, group and normalise metrics from your app. To persist these metrics, we will use a time series database and visualise the data on a dashboard. By the end of the talk, you will know how your favourite performance analytics tool works.
14
+
15
+ ## Chapter 1: Collecting data
16
+ ### Notification Framework
17
+ As a first step we need to implement a notification framework which we can use to instrument and subscribe to performance metrics.
18
+
19
+ A basic example of the functionality of our notification framework would be this code:
20
+ ```ruby
21
+ scenario "It can susbcribe and instrument events" do
22
+ events = []
23
+ ActiveMonitoring::Notifications.subscribe("test_event") do |name, start, finish, id, payload|
24
+ events << { name: name, start: start, finish: finish, id: id, payload: payload }
25
+ end
26
+
27
+ ActiveMonitoring::Notifications.instrument("test_event", payload: :payload) do
28
+ 1 + 1
29
+ end
30
+
31
+ expect(metrics[0].name).to eq("test_event")
32
+ expect(metrics[0].payload).to include(payload: :payload)
33
+ end
34
+ ```
35
+
36
+ The implementation can be found in commit [32c3](https://github.com/ChrisBr/active_monitoring/commit/32c32dcec7529ac7a8e0125b1aaab28cd4219363).
37
+ For the sake of simplicity, this framework is not thread safe of course.
38
+
39
+ This framework is inspired by [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) which already provides various hooks into Rails internals.
40
+ The implementatition of Active Support Notifications can be found [here](https://github.com/rails/rails/tree/master/activesupport/lib/active_support/notifications).
41
+
42
+ ### Instrument events
43
+ [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) provides already
44
+ several hooks to subscribe to e.g. processing of controller actions, SQL queries or view rendering.
45
+ However, Rails does not provide hooks for external libraries like Redis or Memcache.
46
+ In [ef20](https://github.com/ChrisBr/active_monitoring/commit/ef200ae5bf58f9afd2a69003e2c3620d0d9c86eb) and [46c3](https://github.com/ChrisBr/active_monitoring/commit/46c338f8fa5870bae7dc0e251e637d1d4be551db) we monkey patch Rails
47
+ to subscribe to controller actions and SQL queries.
48
+
49
+ ## Chapter 2: Storage
50
+ In chapter two we discussed an appropriate storage engine for our data.
51
+ Our data would look for example like this
52
+
53
+ | Timestamp | Hook | Location | Request ID | Value |
54
+ | ------------- | ------------- |------------- |------------- |------------- |
55
+ | 01.04.2020 00:31:30 CET | process_action | BooksController#show | cf6d6dcd | 100 ms
56
+ | 01.04.2020 00:32:30 CET | sql | BooksController#show | c7c7c841 | 80 ms
57
+
58
+ This data looks tabular, so using a relational database to store it would make sense.
59
+ However, there are several downsides to this approach.
60
+
61
+ ### Relational Database
62
+
63
+ Relational databases perform very well with applications which map to the basic CRUD operations of the database like a Rails controller.
64
+
65
+ ```ruby
66
+ class BooksController < ApplicationController
67
+ def new; end
68
+ def create; end
69
+ def show; end
70
+ def edit; end
71
+ def update; end
72
+ def destroy; end
73
+ end
74
+ ```
75
+
76
+ However, our plugin does not really use all of these traditional database operations or in a different way we would in a normal Rails app.
77
+
78
+ #### Create
79
+ For each request, we will write one process action and several SQL query data points.
80
+ If we extend this application to also cover view render times or background jobs, it's not unlikely we would write 10+ metrics per request.
81
+ Depending of how many requests your app does process, we could easily create millions of metrics per year.
82
+
83
+ #### Read
84
+ We will read the metrics in a dashboard for a certain time range like today or last three hours but not randomly.
85
+
86
+ #### Update
87
+ We will almost never go back to update on of the metrics.
88
+
89
+ #### Delete
90
+ We will only delete metrics in batches but not single metrics.
91
+ For instance, we are only interested in metrics for the last three months so once a month we can do a 'garbage collection' and delete metrics older than three months.
92
+
93
+ ### Time Series Database
94
+
95
+ With this all said, a relational database might not be the best storage engine for this kind of data as
96
+
97
+ * We will heavily write to the database which would cause to frequently create new pages in a [B-Tree](https://en.wikipedia.org/wiki/B-tree) like most relational databases use internally
98
+ * We would need to implement some sort of garbage collection to remove old data
99
+ * Compression of data would not be very efficient
100
+
101
+ For this kind of data, a [log based storage engine](https://en.wikipedia.org/wiki/Log-structured_merge-tree) like most [Time Series databases](https://db-engines.com/en/ranking/time+series+dbms) implement it would be more efficient.
102
+
103
+ However, for the sake of simplicity, we store our metrics in a relational database in this tutorial, the implementation can be found in [5967](https://github.com/ChrisBr/active_monitoring/commit/5967190a1484009476f2378c02e3e0ea96d21624).
104
+
105
+ ## Chapter 3: Cleaning, Normalizing and Grouping
106
+ In the previous chapter we already stored the controller metrics.
107
+ Before we can store SQL metrics, we need to do some more additional work.
108
+
109
+ ### Cleaning
110
+ ActiveRecord does a lot of 'magic' behind the scenes to abstract persisting objects to the database.
111
+ Some of this 'magic' requires to execute additional database queries, for instance
112
+
113
+ * Current version of the database engine
114
+ * Which migrations did already run
115
+ * Which environment
116
+
117
+ We're only interested to store application metrics so we need to filter these queries out.
118
+ The implementation can be found in [156d](https://github.com/ChrisBr/active_monitoring/commit/156d8f7fb0bd3b013524a20effc65dad05506b3d).
119
+
120
+ ### Normalizing
121
+ Depending on the database adapter you use, the queries might contain values.
122
+ To group the same queries together, we need to normalize these queries.
123
+
124
+ ```SQL
125
+ SELECT * FROM books WHERE id = 1;
126
+ SELECT * FROM books WHERE id = 2;
127
+ ```
128
+
129
+ For our performance monitoring tool, these two queries should be treated the same and we need to normalize them to
130
+ ```SQL
131
+ SELECT * FROM books WHERE id = xxx;
132
+ ```
133
+
134
+ In [66a8](https://github.com/ChrisBr/active_monitoring/commit/66a833a46b40072a8888fb6b6c6bb02ee60bda17) we implement a simple query normalizer.
135
+
136
+ ### Grouping
137
+ ActiveRecord is a standalone framework, therefore the payload of the SQL event does not contain a `request_id`.
138
+ We can use ActiveRecord for instance in migrations or background jobs so it is perfectly valid to use it outside of a request response cycle.
139
+ However, in our Performance Monitoring tool we would like to group requests and SQL queries together so we can see if a query causes a slow response.
140
+
141
+ Luckily, we also implemented a `start_processing` event in [ef20](https://github.com/ChrisBr/active_monitoring/commit/ef200ae5bf58f9afd2a69003e2c3620d0d9c86eb).
142
+ We can now subscribe to this event and set the `request_id` and `location` in a cache which we later read
143
+ in when writing the SQL metrics.
144
+ In [b1f1](https://github.com/ChrisBr/active_monitoring/commit/b1f1847270935beacc236bc54535ced1eb83c5ef) we implement a `CurrentAttributes` class and eventually write the SQL metrics.
145
+
146
+ ActiveSupport ships with [CurrentAttributes](https://github.com/rails/rails/blob/157920aead96865e3135f496c09ace607d5620dc/activesupport/lib/active_support/current_attributes.rb) out of the box since Rails 5.
147
+
148
+ ## Chapter 4: Visualization
149
+ So we now have several metrics in our data base written which would look something like
150
+
151
+ | Timestamp | Hook | Location | Request ID | Value |
152
+ | ------------- | ------------- |------------- |------------- |------------- |
153
+ | 01.04.2020 00:31:30 CET | process_action | BooksController#show | cf6d6dcd | 100 ms
154
+ | 01.04.2020 00:32:30 CET | sql | BooksController#show | c7c7c841 | 80 ms
155
+
156
+ It would be very hard to spot now problems or discover pattern so eventually we need to visualize our collected data.
157
+ In [cfca](https://github.com/ChrisBr/active_monitoring/commit/cfcafd2f4664e6927e7d297a4b7da2e18bdd5205) we implement a dashboard to show percentiles and the slowest queries of our Rails app.
158
+ A very good blog article about data visualization for performance metrics from Richard Schneemann can be found [here](https://www.schneems.com/2020/03/17/lies-damned-lies-and-averages-perc50-perc95-explained-for-programmers/).
159
+
160
+ In a real world application, I would strongly recommend to use a dashboard software like [Grafana](https://github.com/grafana/grafana).
161
+
162
+ ## Summary
163
+ Congratulations, we implemented a very basic Performance Monitoring tool in just a few hundred lines of code now.
164
+ We deep dived into [ActiveSupport Notifications](https://edgeguides.rubyonrails.org/active_support_instrumentation.html) and hooked into Rails events to write request and SQL query metrics in our data storage.
165
+ As data storage, we compared relational databases with time series databases like InfluxDB.
166
+ Before we could visualize our data, we needed to clean, normalize and group the metrics.
167
+
168
+ From here, we can easily add more metrics like [rendering of views](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#action-view),
169
+ [caching](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#active-support) or [background jobs](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#active-job).
170
+
171
+ As initially mentioned, this tutorial is heavily influenced by our work on https://github.com/influxdata/influxdb-rails.
172
+ If this made you curious, we always look for new contributors.
173
+
174
+ ## Further information
175
+ * [Profiling and Benchmarking 101 by Nate Berkopec](https://youtu.be/XL51vf-XBTs)
176
+ * [Lies, Damned Lies, and Averages: Perc50, Perc95 explained for Programmers by Richard Schneeman](https://www.schneems.com/2020/03/17/lies-damned-lies-and-averages-perc50-perc95-explained-for-programmers/)
177
+ * https://github.com/influxdata/influxdb-ruby
178
+ * https://github.com/influxdata/influxdb-rails
179
+ * https://docs.influxdata.com/influxdb/v1.7/concepts/storage_engine/
180
+
181
+ ## License
182
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,22 @@
1
+ begin
2
+ require "bundler/setup"
3
+ rescue LoadError
4
+ puts "You must `gem install bundler` and `bundle install` to run rake tasks"
5
+ end
6
+
7
+ require "rdoc/task"
8
+
9
+ RDoc::Task.new(:rdoc) do |rdoc|
10
+ rdoc.rdoc_dir = "rdoc"
11
+ rdoc.title = "ActiveMonitoring"
12
+ rdoc.options << "--line-numbers"
13
+ rdoc.rdoc_files.include("README.md")
14
+ rdoc.rdoc_files.include("lib/**/*.rb")
15
+ end
16
+
17
+ APP_RAKEFILE = File.expand_path("spec/dummy/Rakefile", __dir__)
18
+ load "rails/tasks/engine.rake"
19
+
20
+ load "rails/tasks/statistics.rake"
21
+
22
+ require "bundler/gem_tasks"
@@ -0,0 +1,5 @@
1
+ module ActiveMonitoring
2
+ class ApplicationController < ActionController::Base
3
+ protect_from_forgery with: :exception
4
+ end
5
+ end
@@ -0,0 +1,9 @@
1
+ require_dependency "active_monitoring/application_controller"
2
+
3
+ module ActiveMonitoring
4
+ class DashboardController < ApplicationController
5
+ def show
6
+ @dashboard = Dashboard.new
7
+ end
8
+ end
9
+ end
@@ -0,0 +1,5 @@
1
+ module ActiveMonitoring
2
+ class ApplicationRecord < ActiveRecord::Base
3
+ self.abstract_class = true
4
+ end
5
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class Dashboard
3
+ LIMIT = 10
4
+
5
+ def initialize(date = Date.current)
6
+ @date = date
7
+ end
8
+
9
+ def percentile(value)
10
+ response_metrics.percentile(value)
11
+ end
12
+
13
+ def slow_sql_queries
14
+ sql_metrics.order(:value).limit(LIMIT)
15
+ end
16
+
17
+ private
18
+
19
+ attr_reader :date
20
+
21
+ def sql_metrics
22
+ metrics.where(name: "sql.active_record")
23
+ end
24
+
25
+ def response_metrics
26
+ metrics.where(name: "process_action.action_controller")
27
+ end
28
+
29
+ def metrics
30
+ Metric.where(created_at: date.beginning_of_day..date.end_of_day)
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,7 @@
1
+ module ActiveMonitoring
2
+ class Metric < ApplicationRecord
3
+ def self.percentile(value)
4
+ order(:value).offset(count * value / 10 - 1).limit(1).pluck(:value).first
5
+ end
6
+ end
7
+ end
@@ -0,0 +1,23 @@
1
+ <h1>Dashboard</h1>
2
+
3
+ <h2>Response time</h2>
4
+ <ul>
5
+ <li>90th Percentile: <%= @dashboard.percentile(9) %><li>
6
+ <li>50th Percentile: <%= @dashboard.percentile(5) %><li>
7
+ </ul>
8
+
9
+ <h2>Slow queriess</h2>
10
+ <table>
11
+ <tr>
12
+ <th>SQL</th>
13
+ <th>Location</th>
14
+ <th>Time</th>
15
+ </tr>
16
+ <% @dashboard.slow_sql_queries.each do |metric| %>
17
+ <tr>
18
+ <td><%= metric.sql_query %></td>
19
+ <td><%= metric.location %></td>
20
+ <td><%= metric.value %></td>
21
+ </tr>
22
+ <% end %>
23
+ <table>
@@ -0,0 +1,13 @@
1
+ <!DOCTYPE html>
2
+ <html>
3
+ <head>
4
+ <title>Active monitoring</title>
5
+ <%= csrf_meta_tags %>
6
+ <%= csp_meta_tag %>
7
+ </head>
8
+ <body>
9
+
10
+ <%= yield %>
11
+
12
+ </body>
13
+ </html>
data/config/routes.rb ADDED
@@ -0,0 +1,3 @@
1
+ ActiveMonitoring::Engine.routes.draw do
2
+ resource :dashboard, controller: :dashboard, only: :show
3
+ end
@@ -0,0 +1,13 @@
1
+ class CreateActiveMonitoringMetrics < ActiveRecord::Migration[6.0]
2
+ def change
3
+ create_table :active_monitoring_metrics do |t|
4
+ t.string :name
5
+ t.string :request_id
6
+ t.string :location
7
+ t.string :sql_query
8
+ t.integer :value
9
+
10
+ t.timestamps
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,6 @@
1
+ require "active_monitoring/engine"
2
+ require "active_monitoring/notifications"
3
+
4
+ module ActiveMonitoring
5
+ # Your code goes here...
6
+ end
@@ -0,0 +1,19 @@
1
+ module ActiveMonitoring
2
+ module CoreExtensions
3
+ module ActionController
4
+ module Instrumentation
5
+ def process_action(*)
6
+ payload = {
7
+ controller: self.class.name,
8
+ action: action_name,
9
+ request_id: request.uuid
10
+ }
11
+ ::ActiveMonitoring::Notifications.instrument("start_processing.action_controller", payload)
12
+ ::ActiveMonitoring::Notifications.instrument("process_action.action_controller", payload) do
13
+ super
14
+ end
15
+ end
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,13 @@
1
+ module ActiveMonitoring
2
+ module CoreExtensions
3
+ module ActiveRecord
4
+ module Instrumentation
5
+ def log(sql, name = "SQL", binds = [], type_casted_binds = [], statement_name = nil)
6
+ ::ActiveMonitoring::Notifications.instrument("sql.active_record", sql: sql, name: name) do
7
+ super
8
+ end
9
+ end
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,31 @@
1
+ require_relative "core_extensions/action_controller"
2
+ require_relative "core_extensions/active_record"
3
+ require_relative "sql_query"
4
+ require_relative "current"
5
+
6
+ module ActiveMonitoring
7
+ class Current
8
+ class << self
9
+ def request_id
10
+ store[:request_id]
11
+ end
12
+
13
+ def request_id=(value)
14
+ store[:request_id] = value
15
+ end
16
+
17
+ def location
18
+ store[:location]
19
+ end
20
+
21
+ def location=(value)
22
+ store[:location] = value
23
+ end
24
+
25
+ def store
26
+ Thread.current[:active_monitoring_store] ||= {}
27
+ Thread.current[:active_monitoring_store]
28
+ end
29
+ end
30
+ end
31
+ end
@@ -0,0 +1,47 @@
1
+ require_relative "core_extensions/action_controller"
2
+ require_relative "core_extensions/active_record"
3
+ require_relative "sql_query"
4
+ require_relative "current"
5
+
6
+ module ActiveMonitoring
7
+ class Engine < ::Rails::Engine
8
+ isolate_namespace ActiveMonitoring
9
+
10
+ ActiveSupport.on_load(:active_record) do
11
+ ActiveRecord::ConnectionAdapters::AbstractAdapter.prepend(::ActiveMonitoring::CoreExtensions::ActiveRecord::Instrumentation)
12
+ end
13
+
14
+ ActiveSupport.on_load(:action_controller) do
15
+ ActionController::Base.prepend(::ActiveMonitoring::CoreExtensions::ActionController::Instrumentation)
16
+
17
+ ActiveMonitoring::Notifications.subscribe("start_processing.action_controller") do |_, _, _, _, payload|
18
+ Current.request_id = payload[:request_id]
19
+ Current.location = "#{payload[:controller]}##{payload[:action]}"
20
+ end
21
+
22
+ ActiveMonitoring::Notifications.subscribe("process_action.action_controller") do |name, start, finish, _id, payload|
23
+ Metric.create(
24
+ name: name,
25
+ value: finish - start,
26
+ request_id: payload[:request_id],
27
+ location: "#{payload[:controller]}##{payload[:action]}",
28
+ created_at: finish
29
+ )
30
+ end
31
+
32
+ ActiveMonitoring::Notifications.subscribe("sql.active_record") do |name, start, finish, _id, payload|
33
+ query = ActiveMonitoring::SqlQuery.new(name: payload[:name].dup, query: payload[:sql].dup)
34
+ if query.track?
35
+ Metric.create(
36
+ name: name,
37
+ value: finish - start,
38
+ request_id: Current.request_id,
39
+ sql_query: query.normalized_query,
40
+ location: Current.location,
41
+ created_at: finish
42
+ )
43
+ end
44
+ end
45
+ end
46
+ end
47
+ end
@@ -0,0 +1,19 @@
1
+ require_relative "notifier"
2
+
3
+ module ActiveMonitoring
4
+ class Notifications
5
+ class << self
6
+ attr_accessor :notifier
7
+
8
+ def subscribe(name, &block)
9
+ notifier.subscribe(name, &block)
10
+ end
11
+
12
+ def instrument(name, payload = {}, &block)
13
+ notifier.instrument(name, payload, &block)
14
+ end
15
+ end
16
+
17
+ self.notifier = ActiveMonitoring::Notifier.new
18
+ end
19
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class Notifier
3
+ def initialize
4
+ @subscribers = {}
5
+ end
6
+
7
+ def subscribe(name, &block)
8
+ subscribers[name] ||= []
9
+ subscribers[name] << block
10
+ end
11
+
12
+ def instrument(name, payload)
13
+ id = SecureRandom.hex(10)
14
+ start = Time.current
15
+ result = yield if block_given?
16
+ finish = Time.current
17
+
18
+ subscribers_for(name).each do |callback|
19
+ callback.call(name, start, finish, id, payload)
20
+ end
21
+
22
+ result
23
+ end
24
+
25
+ private
26
+
27
+ attr_reader :subscribers
28
+
29
+ def subscribers_for(name)
30
+ subscribers[name].to_a
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,23 @@
1
+ module ActiveMonitoring
2
+ class SqlNormalizer
3
+ def initialize(query:)
4
+ @query = query
5
+ end
6
+
7
+ def to_s
8
+ query.squish!
9
+ query.gsub!(/(\s(=|>|<|>=|<=|<>|!=)\s)('[^']+'|[\$\+\-\w\.]+)/, '\1xxx')
10
+ query.gsub!(/(\sIN\s)\([^\(\)]+\)/i, '\1(xxx)')
11
+ regex = /(\sBETWEEN\s)('[^']+'|[\+\-\w\.]+)(\sAND\s)('[^']+'|[\+\-\w\.]+)/i
12
+ query.gsub!(regex, '\1xxx\3xxx')
13
+ query.gsub!(/(\sVALUES\s)\(.+\)/i, '\1(xxx)')
14
+ query.gsub!(/(\s(LIKE|ILIKE|SIMILAR TO|NOT SIMILAR TO)\s)('[^']+')/i, '\1xxx')
15
+ query.gsub!(/(\s(LIMIT|OFFSET)\s)(\d+)/i, '\1xxx')
16
+ query
17
+ end
18
+
19
+ private
20
+
21
+ attr_reader :query
22
+ end
23
+ end
@@ -0,0 +1,23 @@
1
+ require_relative "sql_tracker"
2
+ require_relative "sql_normalizer"
3
+
4
+ module ActiveMonitoring
5
+ class SqlQuery
6
+ def initialize(query:, name:)
7
+ @query = query
8
+ @name = name
9
+ end
10
+
11
+ def track?
12
+ SqlTracker.new(query: query, name: name).track?
13
+ end
14
+
15
+ def normalized_query
16
+ SqlNormalizer.new(query: query).to_s
17
+ end
18
+
19
+ private
20
+
21
+ attr_reader :query, :name
22
+ end
23
+ end
@@ -0,0 +1,33 @@
1
+ module ActiveMonitoring
2
+ class SqlTracker
3
+ TRACKED_SQL_COMMANDS = %w(SELECT INSERT UPDATE DELETE).freeze
4
+ UNTRACKED_NAMES = %w(SCHEMA).freeze
5
+ UNTRACKED_TABLES = %w(
6
+ SCHEMA_MIGRATIONS
7
+ SQLITE_MASTER
8
+ ACTIVE_MONITORING_METRICS
9
+ SQLITE_TEMP_MASTER
10
+ SQLITE_VERSION
11
+ AR_INTERNAL_METADATA
12
+ ).freeze
13
+
14
+ def initialize(query:, name:)
15
+ @query = query.to_s.upcase
16
+ @name = name.to_s.upcase
17
+ end
18
+
19
+ def track?
20
+ query.start_with?(*TRACKED_SQL_COMMANDS) &&
21
+ !name.start_with?(*UNTRACKED_NAMES) &&
22
+ !untracked_tables?
23
+ end
24
+
25
+ private
26
+
27
+ attr_reader :query, :name
28
+
29
+ def untracked_tables?
30
+ UNTRACKED_TABLES.any? { |table| query.include?(table) }
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,3 @@
1
+ module ActiveMonitoring
2
+ VERSION = "0.1.0".freeze
3
+ end
metadata ADDED
@@ -0,0 +1,158 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: active_monitoring
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Christian Bruckmayer
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2020-04-17 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: rails
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: 6.0.2
20
+ - - ">="
21
+ - !ruby/object:Gem::Version
22
+ version: 6.0.2.2
23
+ type: :runtime
24
+ prerelease: false
25
+ version_requirements: !ruby/object:Gem::Requirement
26
+ requirements:
27
+ - - "~>"
28
+ - !ruby/object:Gem::Version
29
+ version: 6.0.2
30
+ - - ">="
31
+ - !ruby/object:Gem::Version
32
+ version: 6.0.2.2
33
+ - !ruby/object:Gem::Dependency
34
+ name: byebug
35
+ requirement: !ruby/object:Gem::Requirement
36
+ requirements:
37
+ - - ">="
38
+ - !ruby/object:Gem::Version
39
+ version: '0'
40
+ type: :development
41
+ prerelease: false
42
+ version_requirements: !ruby/object:Gem::Requirement
43
+ requirements:
44
+ - - ">="
45
+ - !ruby/object:Gem::Version
46
+ version: '0'
47
+ - !ruby/object:Gem::Dependency
48
+ name: rspec-rails
49
+ requirement: !ruby/object:Gem::Requirement
50
+ requirements:
51
+ - - ">="
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :development
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - ">="
59
+ - !ruby/object:Gem::Version
60
+ version: '0'
61
+ - !ruby/object:Gem::Dependency
62
+ name: rubocop-performance
63
+ requirement: !ruby/object:Gem::Requirement
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: '0'
68
+ type: :development
69
+ prerelease: false
70
+ version_requirements: !ruby/object:Gem::Requirement
71
+ requirements:
72
+ - - ">="
73
+ - !ruby/object:Gem::Version
74
+ version: '0'
75
+ - !ruby/object:Gem::Dependency
76
+ name: rubocop-rails
77
+ requirement: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - ">="
80
+ - !ruby/object:Gem::Version
81
+ version: '0'
82
+ type: :development
83
+ prerelease: false
84
+ version_requirements: !ruby/object:Gem::Requirement
85
+ requirements:
86
+ - - ">="
87
+ - !ruby/object:Gem::Version
88
+ version: '0'
89
+ - !ruby/object:Gem::Dependency
90
+ name: sqlite3
91
+ requirement: !ruby/object:Gem::Requirement
92
+ requirements:
93
+ - - ">="
94
+ - !ruby/object:Gem::Version
95
+ version: '0'
96
+ type: :development
97
+ prerelease: false
98
+ version_requirements: !ruby/object:Gem::Requirement
99
+ requirements:
100
+ - - ">="
101
+ - !ruby/object:Gem::Version
102
+ version: '0'
103
+ description: his is an example gem used in my RailsConf 2020 talk. Please don't use
104
+ in production! https://bruckmayer.net/rails-conf-2020
105
+ email:
106
+ - christian@bruckmayer.net
107
+ executables: []
108
+ extensions: []
109
+ extra_rdoc_files: []
110
+ files:
111
+ - MIT-LICENSE
112
+ - README.md
113
+ - Rakefile
114
+ - app/controllers/active_monitoring/application_controller.rb
115
+ - app/controllers/active_monitoring/dashboard_controller.rb
116
+ - app/models/active_monitoring/application_record.rb
117
+ - app/models/active_monitoring/dashboard.rb
118
+ - app/models/active_monitoring/metric.rb
119
+ - app/views/active_monitoring/dashboard/show.html.erb
120
+ - app/views/layouts/active_monitoring/application.html.erb
121
+ - config/routes.rb
122
+ - db/migrate/20200401191741_create_active_monitoring_metrics.rb
123
+ - lib/active_monitoring.rb
124
+ - lib/active_monitoring/core_extensions/action_controller.rb
125
+ - lib/active_monitoring/core_extensions/active_record.rb
126
+ - lib/active_monitoring/current.rb
127
+ - lib/active_monitoring/engine.rb
128
+ - lib/active_monitoring/notifications.rb
129
+ - lib/active_monitoring/notifier.rb
130
+ - lib/active_monitoring/sql_normalizer.rb
131
+ - lib/active_monitoring/sql_query.rb
132
+ - lib/active_monitoring/sql_tracker.rb
133
+ - lib/active_monitoring/version.rb
134
+ homepage: https://github.com/ChrisBr/active_monitoring
135
+ licenses:
136
+ - MIT
137
+ metadata: {}
138
+ post_install_message:
139
+ rdoc_options: []
140
+ require_paths:
141
+ - lib
142
+ required_ruby_version: !ruby/object:Gem::Requirement
143
+ requirements:
144
+ - - ">="
145
+ - !ruby/object:Gem::Version
146
+ version: '0'
147
+ required_rubygems_version: !ruby/object:Gem::Requirement
148
+ requirements:
149
+ - - ">="
150
+ - !ruby/object:Gem::Version
151
+ version: '0'
152
+ requirements: []
153
+ rubygems_version: 3.1.2
154
+ signing_key:
155
+ specification_version: 4
156
+ summary: This is an example gem used in my RailsConf 2020 talk. Please don't use in
157
+ production!
158
+ test_files: []