railswatch 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: fa66ff41888554ca29ee2d634bbf31701298ca13dd94973a9700e97ac8d26375
+  data.tar.gz: 1548b8107dae00b59b9cef9c2effea9223e724486bf619752dc4c8d681a68217
+SHA512:
+  metadata.gz: 9bd95a2eddaa365ca48eda575e0ad0371ad2d90a0167deb8beff351f198f4b4b993ccfad1f7dd9abc5c9377da666836f839a00f88cf2eae2ef3b13f99f9ea355
+  data.tar.gz: e87f1c5cec9055f9caff00b13007b64f4858f04052666f2f90a95b1b48458284a39590f1fd95e78ff623821dd07366ebdbf34dbc4164bfc76ffeceae2730bb44

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2020 Sourabh Patware
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,485 @@
+# Railswatch
+
+[![Tests](https://github.com/100rabhg/railswatch/actions/workflows/ruby.yml/badge.svg)](https://github.com/100rabhg/railswatch/actions/workflows/ruby.yml)
+
+A self-hosted tool to monitor the performance of your Ruby on Rails application.
+
+This is a **simple and free alternative** to the New Relic APM, Datadog or other similar services.
+
+![Home](docs/railswatch_home.png)
+
+P50, P90, P99, throughput, and more is available.
+
+Detailed p50, p90, p99 response time information.
+
+![Recent Requests](docs/railswatch_recent_requests.png)
+
+Per-controller breakdown with response time percentiles.
+
+![Requests Breakdown](docs/recent_requests.png)
+
+(more screenshots below)
+
+It allows you to track:
+
+- real-time monitoring on the Recent tab
+- see your p50, p90, p99 response time
+- monitor system resources (CPU, memory, disk)
+- monitor slow requests
+- throughput report (see amount of RPM (requests per minute))
+- an average response time
+- the slowest controllers & actions
+- total duration of time spent per request, views rendering, DB
+- SQL queries, rendering logs in "Recent Requests" section
+- simple 500-crashes reports
+- deployment events (or custom events)
+- Sidekiq jobs
+- Delayed Job jobs
+- Grape API inside Rails app
+- Rake tasks performance
+- Custom events wrapped with `Railswatch.measure do .. end` block
+- Active Record-backed storage, with optional separate database support on modern Rails multi-database setups
+
+All data are stored in your application's database and not sent to any 3rd party servers.
+
+## Production
+
+Gem is production-ready. At least in my 2 applications with ~1000 unique users per day it works perfectly.
+
+Just don't forget to protect performance dashboard with http basic auth or check of current_user.
+
+## Quick Start
+
+1. Add the gem to your application's `Gemfile`
+2. Run `bundle install`
+3. Run `bin/rails generate railswatch:install`
+4. Run your database migrations for the storage option you chose
+5. Start the app, make a few requests, and open `/railswatch`
+
+Railswatch stores monitoring data in your application's database by default.
+You do not need Redis for Railswatch storage.
+
+If your application uses Sidekiq, Sidekiq itself still uses Redis, but Railswatch stores the captured job metrics in your database.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'railswatch'
+
+# or
+
+group :development, :production do
+  gem 'railswatch'
+end
+```
+
+Install dependencies:
+
+```bash
+$ bundle install
+```
+
+Generate the initializer and migration:
+
+```bash
+$ bin/rails generate railswatch:install
+```
+
+That generator creates:
+
+- `config/initializers/railswatch.rb`
+- `db/migrate/*_create_railswatch_tables.rb`
+
+## Setup Guide
+
+### 1. Choose where monitoring data is stored
+
+#### Option A: Use your primary application database
+
+This is the default and simplest setup:
+
+```ruby
+config.database_connection_name = nil
+```
+
+Then run:
+
+```bash
+$ bin/rails db:migrate
+```
+
+#### Option B: Use a separate monitoring database
+
+Add a dedicated database to `config/database.yml`:
+
+```yaml
+development:
+  primary:
+    <<: *default
+    database: storage/development.sqlite3
+  railswatch:
+    <<: *default
+    database: storage/development_railswatch.sqlite3
+
+test:
+  primary:
+    <<: *default
+    database: storage/test.sqlite3
+  railswatch:
+    <<: *default
+    database: storage/test_railswatch.sqlite3
+```
+
+Then set:
+
+```ruby
+config.database_connection_name = :railswatch
+```
+
+Run migrations for both databases:
+
+```bash
+$ bin/rails db:migrate
+$ bin/rails db:migrate:railswatch
+```
+
+You can use any connection name you prefer. `config.database_connection_name` must match the database name in `config/database.yml`.
+
+### 2. Start the app
+
+After installation and configuration:
+
+```bash
+$ bin/rails server
+```
+
+Make a few requests to your app, then open:
+
+```text
+http://localhost:3000/railswatch
+```
+
+### 3. Protect the dashboard
+
+The dashboard should not be publicly accessible in production.
+
+You can protect it with HTTP Basic auth:
+
+```ruby
+config.http_basic_authentication_enabled = true
+config.http_basic_authentication_user_name = 'railswatch'
+config.http_basic_authentication_password = ENV.fetch('RAILSWATCH_PASSWORD')
+```
+
+Or with your own app-level authorization:
+
+```ruby
+config.verify_access_proc = proc do |controller|
+  controller.current_user&.admin?
+end
+```
+
+## Configuration
+
+Default configuration is listed below. You can override it in `config/initializers/railswatch.rb`:
+
+```ruby
+Railswatch.setup do |config|
+  # Use the primary application database by default.
+  # To use a dedicated database, define it in config/database.yml and set:
+  # config.database_connection_name = :railswatch
+  config.database_connection_name = nil
+
+  config.duration = 4.hours
+
+  config.debug    = false
+  config.enabled  = true
+
+  # configure Recent tab (time window and limit of requests)
+  # config.recent_requests_time_window = 60.minutes
+  # config.recent_requests_limit = nil # or 1000
+
+  # configure Slow Requests tab (time window, limit of requests and threshold)
+  # config.slow_requests_time_window = 4.hours # time window for slow requests
+  # config.slow_requests_limit = 500 # number of max rows
+  # config.slow_requests_threshold = 500 # number of ms
+
+  # default path where to mount gem,
+  # alternatively you can mount the Railswatch::Engine in your routes.rb
+  config.mount_at = '/railswatch'
+
+  # protect your Performance Dashboard with HTTP BASIC password
+  config.http_basic_authentication_enabled   = false
+  config.http_basic_authentication_user_name = 'railswatch'
+  config.http_basic_authentication_password  = 'password12'
+
+  # if you need an additional rules to check user permissions
+  config.verify_access_proc = proc { |controller| true }
+  # for example when you have `current_user`
+  # config.verify_access_proc = proc { |controller| controller.current_user && controller.current_user.admin? }
+
+  # You can ignore endpoints with Rails standard notation controller#action
+  # config.ignored_endpoints = ['HomeController#contact']
+
+  # You can ignore request paths by specifying the beginning of the path.
+  # For example, all routes starting with '/admin' can be ignored:
+  config.ignored_paths = ['/railswatch', '/admin']
+
+  # store custom data for the request
+  # config.custom_data_proc = proc do |env|
+  #   request = Rack::Request.new(env)
+  #   {
+  #     email: request.env['warden'].user&.email, # if you are using Devise for example
+  #     user_agent: request.env['HTTP_USER_AGENT']
+  #   }
+  # end
+
+  # config home button link
+  config.home_link = '/'
+
+  # To skip some Rake tasks from monitoring
+  config.skipable_rake_tasks = ['webpacker:compile']
+
+  # To monitor rake tasks performance, you need to include rake tasks
+  # config.include_rake_tasks = false
+
+  # To monitor custom events with `Railswatch.measure` block
+  # config.include_custom_events = true
+
+  # To monitor system resources (CPU, memory, disk)
+  # to enabled add required gems (see README)
+  # config.system_monitor_duration = 24.hours
+
+  config.retention = {
+    requests: config.duration,
+    sidekiq: config.duration,
+    delayed_job: config.duration,
+    grape: config.duration,
+    rake: config.duration,
+    custom: config.duration,
+    traces: config.recent_requests_time_window,
+    resources: 24.hours,
+    events: nil
+  }
+end if defined?(Railswatch)
+```
+
+Additionally you might need to configure app time zone. You can do it in `config/application.rb`:
+
+```ruby
+config.time_zone = 'Eastern Time (US & Canada)'
+```
+
+Gem will present charts/tables in the app timezone. If it's not set, it will use UTC.
+
+## Optional Integrations
+
+### Sidekiq
+
+If your app uses Sidekiq, Railswatch will capture Sidekiq job metrics automatically when Sidekiq is present.
+
+Railswatch does not require Redis for its own storage, but Sidekiq still requires Redis as usual.
+
+![Sidekiq](docs/sidekiq.png)
+
+### Delayed Job
+
+If your app uses Delayed Job, add the adapter gem to your app:
+
+```ruby
+gem 'delayed_job_active_record'
+```
+
+Railswatch stores captured Delayed Job metrics in your database.
+
+### System resource monitoring
+
+To show CPU, memory, and disk charts on the dashboard, add:
+
+```ruby
+gem 'sys-filesystem'
+gem 'sys-cpu'
+gem 'get_process_mem'
+```
+
+Once these gems are installed, Railswatch will collect and display system resource metrics.
+
+### Retention and pruning
+
+Retention is configured per record type with `config.retention`. Set a duration to prune old records, or `nil` to keep that record type forever.
+
+```ruby
+config.retention = {
+  requests: 4.hours,
+  sidekiq: 4.hours,
+  delayed_job: 4.hours,
+  grape: 4.hours,
+  rake: 4.hours,
+  custom: 4.hours,
+  traces: 60.minutes,
+  resources: 24.hours,
+  events: nil
+}
+```
+
+Prune old records manually or from a scheduler:
+
+```bash
+$ bin/rails railswatch:prune
+```
+
+### Alternative: Mounting the engine yourself
+
+If you, for whatever reason (company policy, devise, ...) need to mount Railswatch yourself, feel free to do so by using the following snippet as inspiration.
+You can skip the `mount_at` and `http_basic_authentication_*` configurations then, if you like.
+Under certain constraints (i.e. subdomains) it may be necessary to set `url_options` in the config so that Railswatch can generate links correctly.
+
+```ruby
+# config/routes.rb
+Rails.application.routes.draw do
+  ...
+  # example for usage with Devise
+  authenticate :user, -> (user) { user.admin? } do
+    mount Railswatch::Engine, at: 'railswatch'
+  end
+end
+```
+
+### 500 Errors and request context
+
+Railswatch captures every 500 error with a full backtrace, the request path, controller/action, and per-request context: client IP, user-agent, and filtered request params.
+
+![Errors](docs/errors.png)
+
+### Custom data
+
+You can configure `config.custom_data_proc` to capture additional data per request (e.g. `current_user`, email). This proc is executed inside middleware and has access to the Rack `env`.
+
+
+### Server Monitoring
+
+You can monitor system resources (CPU, memory, disk) by adding these gems to your Gemfile:
+
+```ruby
+gem "sys-filesystem"
+gem "sys-cpu"
+gem "get_process_mem"
+```
+
+Once you add these gems, Railswatch will track and show system resources on the dashboard.
+
+If you have multiple servers running the same app, it will use store metrics per server. You can configure the env variable `ENV["RAILSWATCH_SERVER_ID"]` or use `hostname`.
+
+Basically using this code:
+
+```ruby
+      def server_id
+        @server_id ||= ENV["RAILSWATCH_SERVER_ID"] || `hostname`.strip
+      end
+```
+
+For Kamal for example:
+
+```yaml
+env:
+  clear:
+    RAILSWATCH_SERVER_ID: "server"
+```
+
+You can also specify custom "context" and "role" for monitoring, by changing the env variables:
+
+```ruby
+Railswatch::SystemMonitor::ResourcesMonitor.new(
+  ENV["RAILSWATCH_SERVER_CONTEXT"].presence || "rails",
+  ENV["RAILSWATCH_SERVER_ROLE"].presence || "web"
+)
+```
+
+More information here: `lib/railswatch/engine.rb`.
+
+PS: right now it can only distinguish between web app servers and the sidekiq servers.
+
+#### Deployment Events + Custom Events on the Charts
+
+#### For Kamal
+
+- edit `.kamal/hooks/post-deploy` (rename .sample if needed)
+- add `kamal app exec -p './bin/rails runner "Railswatch.create_event(name: \"Deploy\")"'`
+- kamal deploy
+
+#### Custom Events on the Charts
+
+You can specify colors, orientation for the event label.
+
+```ruby
+Railswatch.create_event(name: "Deploy", options: {
+  borderColor: "#00E396",
+  label: {
+    borderColor: "#00E396",
+    orientation: "horizontal",
+    text: "Deploy"
+  }
+})
+```
+
+### Custom events
+
+```ruby
+Railswatch.measure("some label", "some namespace") do
+  # your code
+end
+```
+
+## How it works
+
+![Schema](docs/railswatch.png)
+
+In addition it's wrapping gems internal methods and collecting performance information. See `./lib/railswatch/gems/*` for more information.
+
+## Limitations
+
+- it doesn't track params of POST/PUT requests
+- it doesn't track ElasticSearch or other apps
+- it can't compare historical data
+- depending on your load you may need to reduce how long you keep data, especially if you retain large request volumes for reporting
+
+## Development & Testing
+
+Just clone the repo, setup dummy app (`rails db:migrate`).
+
+After this:
+
+- rails s
+- rake test
+
+If you need to clear collected data during development, you can delete rows from the `railswatch_*` tables or run a short retention window with `bin/rails railswatch:prune`.
+
+Like a regular web development.
+
+Please note that to simplify integration with other apps all CSS/JS are bundled inside, and delivered in body of the request. This is to avoid integration with assets pipeline or webpacker.
+
+For UI changes you need to use Bulma CSS (https://bulma.io/documentation).
+
+## Why
+
+The idea of this gem grew from curiosity how many RPM my app receiving per day. Later it evolved to something more powerful.
+
+## Development
+
+1. Clone the repo
+2. Run `bundle install`
+3. Set up the dummy app with `cd test/dummy && bundle install && rails db:create && rails db:migrate`
+4. Run `rails s` in the project root
+5. Run `rails test`
+
+## Contributing
+
+You are welcome to contribute. I've a big list of TODO.
+
+If the storage schema changes in a breaking way, add a migration and document the upgrade path in the changelog and README.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'rdoc/task'
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = 'Railswatch'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.md')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+APP_RAKEFILE = File.expand_path('test/dummy/Rakefile', __dir__)
+load 'rails/tasks/engine.rake'
+load 'rails/tasks/statistics.rake'
+
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+begin
+  require 'standard/rake'
+rescue LoadError
+  nil
+end
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+task default: [:test]

data/app/assets/images/activity.svg

@@ -0,0 +1,13 @@
+<?xml version="1.0" ?><svg height="48" id="activity" viewBox="0 0 48 48" width="48" xmlns="http://www.w3.org/2000/svg"><defs><style>
+      .vi-primary {
+        fill: #FF6E6E;
+      }
+
+      .vi-primary, .vi-accent {
+        fill-rule: evenodd;
+      }
+
+      .vi-accent {
+        fill: #0C0058;
+      }
+    </style></defs><path class="vi-primary" d="M20,33a2,2,0,0,1-1.788-1.1l-3.6-7.183-2.047,2.551A2,2,0,0,1,11,28.013H7a1.994,1.994,0,1,1,0-3.988h3.039l3.4-4.237a2,2,0,0,1,3.351.354l2.7,5.386,2.588-9.03a2,2,0,0,1,3.75-.262l3.473,7.79H32a1.994,1.994,0,1,1,0,3.988H28a2,2,0,0,1-1.827-1.184L24.41,22.874l-2.487,8.678a2,2,0,0,1-1.737,1.437C20.123,33,20.061,33,20,33Z"/><path class="vi-accent" d="M37,32a6,6,0,1,1,6-6A6,6,0,0,1,37,32Zm0-8.717A2.72,2.72,0,1,0,39.728,26,2.724,2.724,0,0,0,37,23.283Z"/></svg>

data/app/assets/images/bot.svg

@@ -0,0 +1 @@
+<?xml version="1.0" ?><svg height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="M21.928 11.607c-.202-.488-.635-.605-.928-.633V8c0-1.103-.897-2-2-2h-6V4.61c.305-.274.5-.668.5-1.11a1.5 1.5 0 0 0-3 0c0 .442.195.836.5 1.11V6H5c-1.103 0-2 .897-2 2v2.997l-.082.006A1 1 0 0 0 1.99 12v2a1 1 0 0 0 1 1H3v5c0 1.103.897 2 2 2h14c1.103 0 2-.897 2-2v-5a1 1 0 0 0 1-1v-1.938a1.006 1.006 0 0 0-.072-.455zM5 20V8h14l.001 3.996L19 12v2l.001.005.001 5.995H5z"/><ellipse cx="8.5" cy="12" rx="1.5" ry="2"/><ellipse cx="15.5" cy="12" rx="1.5" ry="2"/><path d="M8 16h8v2H8z"/></svg>

data/app/assets/images/close.svg

@@ -0,0 +1,13 @@
+<?xml version="1.0" ?><svg height="48" id="close" viewBox="0 0 48 48" width="48" xmlns="http://www.w3.org/2000/svg"><defs><style>
+      .vi-primary {
+        fill: #FF6E6E;
+      }
+
+      .vi-primary, .vi-accent {
+        fill-rule: evenodd;
+      }
+
+      .vi-accent {
+        fill: #0C0058;
+      }
+    </style></defs><path class="vi-primary" d="M35,32l-3,3-7.5-7.5L17,35l-3-3,7.5-7.5L14,17l3-3,7.5,7.5L32,14l3,3-7.5,7.5Z"/><path class="vi-accent" d="M20,32l-3,3-3-3,3-3ZM35,17l-3,3-3-3,3-3ZM29,32l3,3,3-3-3-3ZM14,17l3,3,3-3-3-3Z"/></svg>

data/app/assets/images/details.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="size-6">
+  <path stroke-linecap="round" stroke-linejoin="round" d="m21 21-5.197-5.197m0 0A7.5 7.5 0 1 0 5.196 5.196a7.5 7.5 0 0 0 10.607 10.607ZM10.5 7.5v6m3-3h-6" />
+</svg>

data/app/assets/images/download.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="size-6">
+  <path stroke-linecap="round" stroke-linejoin="round" d="M3 16.5v2.25A2.25 2.25 0 0 0 5.25 21h13.5A2.25 2.25 0 0 0 21 18.75V16.5M16.5 12 12 16.5m0 0L7.5 12m4.5 4.5V3" />
+</svg>

data/app/assets/images/export.svg

@@ -0,0 +1,13 @@
+<?xml version="1.0" ?><svg height="48" id="share" viewBox="0 0 48 48" width="48" xmlns="http://www.w3.org/2000/svg"><defs><style>
+      .vi-primary {
+        fill: #FF6E6E;
+        stroke: #fff;
+        stroke-linecap: round;
+        stroke-width: 0;
+      }
+
+      .vi-accent {
+        fill: #0C0058;
+        fill-rule: evenodd;
+      }
+    </style></defs><rect class="vi-primary" height="28" width="28" x="5" y="10"/><path class="vi-accent" d="M17,25H39l-3,3,2,2,6-6V23l-6-6-2,2,3,3H17v3Z"/></svg>

data/app/assets/images/external.svg

@@ -0,0 +1 @@
+<?xml version="1.0"?><svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><g><path d="M0 0h24v24H0z" fill="none"/><path d="M10 6v2H5v11h11v-5h2v6a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1V7a1 1 0 0 1 1-1h6zm11-3v8h-2V6.413l-7.793 7.794-1.414-1.414L17.585 5H13V3h8z"/></g></svg>