rails_db_inspector 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 406b9d3f33d6809dfd3d245f95d82c01ddf3a007775a1a8a74b83c41da5263cd
+  data.tar.gz: 4d5913bcc76b28f7042b7e280c966fba438848db53788f598d836fdf222d9578
+SHA512:
+  metadata.gz: aabc8ed324f44a464c9042b40198f096222338375a7e9e8fbcad2a3690c11a290a07a2b22c4d3cefa6681a53c1c3626504c8b91d2f99f27ae5f7c74c5b712fcb
+  data.tar.gz: 7f6507b9b1ab845396b26dd54d8a83a5ec1a317505ced5d0075f85c85da8a13051319a72682cabf76a1d1b134ef57fd2a3afe10116b6a7838962330748e26d7e

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright samuel-murphy
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,232 @@
+# Rails DB Inspector
+
+A mountable Rails engine that gives you a built-in dashboard for **SQL query monitoring**, **N+1 detection**, **EXPLAIN / EXPLAIN ANALYZE plans**, and **interactive schema visualization** — no external services required.
+
+Supports **PostgreSQL** and **MySQL**.
+
+![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.1-red)
+![Rails](https://img.shields.io/badge/rails-%3E%3D%207.1-red)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+
+---
+
+## Features
+
+- **Real-time SQL Query Capture** — every query your app executes is logged with SQL text, duration, bind parameters, and timestamps
+- **N+1 Query Detection** — automatically identifies repeated query patterns and highlights the worst offenders
+- **Query Grouping** — queries are grouped by controller action using Rails marginal annotations
+- **EXPLAIN Plans** — run `EXPLAIN` on any captured query to see the execution plan (PostgreSQL JSON format, MySQL tabular)
+- **EXPLAIN ANALYZE** — optionally run `EXPLAIN ANALYZE` to get real execution statistics, buffer usage, and timing (opt-in, SELECT only)
+- **Plan Analysis** — rich visual rendering of PostgreSQL plans including cost breakdown, row estimate accuracy, index usage analysis, performance hotspots, buffer statistics, and actionable recommendations
+- **Interactive Schema / ERD Visualization** — drag-and-drop entity relationship diagram with pan, zoom, search, column expansion, heat-map by row count, missing index warnings, polymorphic detection, and SVG export
+- **Dev Widget** — floating button injected into your app's pages in development for quick access to the dashboard
+- **Zero Dependencies** — no JavaScript build step, no external CSS frameworks, everything is self-contained
+
+---
+
+## Installation
+
+Add to your Gemfile. It is **strongly recommended** to restrict this to `:development` (and optionally `:test`):
+
+```ruby
+group :development do
+  gem "rails_db_inspector"
+end
+```
+
+Then run:
+
+```bash
+bundle install
+```
+
+---
+
+## Setup
+
+### 1. Mount the Engine
+
+Add the engine route to your `config/routes.rb`:
+
+```ruby
+Rails.application.routes.draw do
+  # ... your app routes ...
+
+  if Rails.env.development?
+    mount RailsDbInspector::Engine, at: "/inspect"
+  end
+end
+```
+
+You can use any mount path — `/inspect`, `/db`, `/rails_db_inspector`, etc.
+
+### 2. Create an Initializer (Optional but Recommended)
+
+Create `config/initializers/rails_db_inspector.rb`:
+
+```ruby
+RailsDbInspector.configure do |config|
+  # Enable or disable the engine entirely.
+  # Default: true
+  config.enabled = Rails.env.development?
+
+  # Maximum number of queries to keep in memory.
+  # Older queries are trimmed when this limit is exceeded.
+  # Default: 2000
+  config.max_queries = 2_000
+
+  # Allow EXPLAIN ANALYZE to run on captured queries.
+  # This actually executes the query, so it is disabled by default for safety.
+  # Only SELECT statements are permitted even when enabled.
+  # Default: false
+  config.allow_explain_analyze = true
+
+  # Show the floating dev widget on your app's pages in development.
+  # The widget provides quick links to the query monitor and schema viewer.
+  # Default: true
+  config.show_widget = true
+end
+```
+
+### Configuration Options
+
+| Option                  | Type    | Default  | Description                                                                 |
+|-------------------------|---------|----------|-----------------------------------------------------------------------------|
+| `enabled`               | Boolean | `true`   | Master switch — disables SQL subscription and widget when `false`           |
+| `max_queries`           | Integer | `2000`   | Max queries stored in memory (FIFO eviction)                                |
+| `allow_explain_analyze` | Boolean | `false`  | Permit EXPLAIN ANALYZE (executes the query — SELECT only)                   |
+| `show_widget`           | Boolean | `true`   | Inject floating widget into HTML pages in development                       |
+
+---
+
+## Usage
+
+### Accessing the Dashboard
+
+Once mounted, visit the engine in your browser:
+
+```
+http://localhost:3000/inspect
+```
+
+(Replace `/inspect` with whatever mount path you chose.)
+
+### Query Monitor
+
+The root page shows all captured SQL queries in reverse-chronological order.
+
+- **Grouped by Controller Action** — queries are automatically grouped using Rails' marginal SQL comments (`controller='...'`, `action='...'`). Enable annotations in your app with:
+
+  ```ruby
+  # config/application.rb or config/environments/development.rb
+  config.active_record.query_log_tags_enabled = true
+  config.active_record.query_log_tags = [
+    { controller: ->(context) { context[:controller]&.controller_name } },
+    { action:     ->(context) { context[:controller]&.action_name } }
+  ]
+  ```
+
+- **N+1 Detection** — the dashboard flags queries that appear 3+ times with the same normalized SQL pattern, showing the count, total duration, and table name
+- **Query Type Badges** — each query is tagged with its operation (`SELECT`, `INSERT`, `UPDATE`, `DELETE`, `CTE`) and complexity hints (`JOIN`, `SUBQUERY`, `AGGREGATE`, `ORDER BY`, `WINDOW`)
+- **Clear Queries** — use the "Clear" button to reset the in-memory query store
+
+### Running EXPLAIN
+
+Click on any query to view its details, then click **Explain** to get the execution plan.
+
+- **EXPLAIN** — shows the planned execution without running the query (always available)
+- **EXPLAIN ANALYZE** — shows actual execution statistics (requires `allow_explain_analyze = true` in the initializer)
+
+For PostgreSQL, the plan is rendered with:
+- Visual tree of plan nodes with cost, rows, and width
+- Timing and buffer statistics (ANALYZE mode)
+- Row estimate accuracy indicators (color-coded)
+- Warning badges for sequential scans, large sorts, nested loops, etc.
+- Index usage analysis
+- Performance hotspot identification
+- Cache hit ratio
+- Actionable recommendations (e.g., "Create index on `orders.status`")
+
+> **⚠️ Safety:** EXPLAIN ANALYZE actually executes the query. Only `SELECT` statements are allowed — `INSERT`, `UPDATE`, and `DELETE` queries are blocked even when analyze is enabled.
+
+### Schema Visualization
+
+Navigate to the **Schema** page to see an interactive entity relationship diagram:
+
+- **Drag & drop** nodes to rearrange
+- **Pan & zoom** with mouse wheel or controls
+- **Search** tables with `/` keyboard shortcut
+- **Click** a table to see columns, types, indexes, foreign keys, associations, and row count in the detail panel
+- **Double-click** a node to expand/collapse its columns
+- **Relationships** drawn from foreign keys (solid blue lines) and Rails conventions (dashed gray lines)
+- **Heat map** — node headers are color-coded by row count (green → red)
+- **Missing index warnings** — yellow badge on tables with `_id` columns lacking an index
+- **Polymorphic detection** — purple "P" badge on tables with matching `_type`/`_id` column pairs
+- **Health summary** — table count, column count, index count, total rows, missing indexes, tables without timestamps, tables without primary keys, polymorphic columns
+- **Export SVG** — download the diagram as an SVG file
+
+### Dev Widget
+
+In development, a floating blue button (🛢️) appears in the bottom-right corner of every page. Click it to reveal quick links to:
+
+- **Query Monitor** — opens the query dashboard
+- **Schema Visualization** — opens the ERD viewer
+
+The widget is automatically injected via Rack middleware and only appears in `development` environment. Disable it with `config.show_widget = false`.
+
+---
+
+## Supported Databases
+
+| Adapter    | EXPLAIN | EXPLAIN ANALYZE | Schema / ERD |
+|------------|---------|-----------------|--------------|
+| PostgreSQL | ✅      | ✅               | ✅            |
+| MySQL      | ✅      | ✅               | ✅            |
+| SQLite     | ❌      | ❌               | ✅            |
+
+EXPLAIN uses `FORMAT JSON` for PostgreSQL and standard `EXPLAIN` for MySQL.
+
+---
+
+## How It Works
+
+1. **SQL Subscriber** — uses `ActiveSupport::Notifications.subscribe("sql.active_record")` to capture every query. Schema, transaction, cached, and EXPLAIN queries are automatically filtered out.
+2. **Query Store** — an in-memory singleton (`QueryStore`) stores captured queries with thread-safe access. Oldest queries are evicted when `max_queries` is exceeded.
+3. **Explain** — wraps the captured SQL in an `EXPLAIN` statement appropriate for the database adapter and parses the result.
+4. **Schema Inspector** — introspects `ActiveRecord::Base.connection` for tables, columns, indexes, foreign keys, primary keys, row counts, associations, polymorphic columns, and missing indexes.
+5. **Dev Widget Middleware** — a Rack middleware that injects a small HTML snippet before `</body>` on HTML responses in development.
+
+---
+
+## Development / Contributing
+
+```bash
+# Clone the repo
+git clone https://github.com/h0m1c1de/rails_db_inspector.git
+cd rails_db_inspector
+
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Run linter
+bin/rubocop
+```
+
+### Running Tests
+
+The test suite uses RSpec with SimpleCov for coverage:
+
+```bash
+bundle exec rspec
+```
+
+Coverage targets: **95% line**, **85% branch**.
+
+---
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,3 @@
+require "bundler/setup"
+
+require "bundler/gem_tasks"

data/app/assets/stylesheets/rails_db_inspector/application.css

@@ -0,0 +1,41 @@
+/*
+ * Rails DB Inspector - Tailwind CSS Application
+ * 
+ * This gem now uses Tailwind CSS for modern, professional styling.
+ * Custom styles for plan trees and specific components are included below.
+ *= require_self
+ */
+
+/* Custom styles for plan tree interactions */
+.expand-toggle {
+  transition: all 0.2s ease-in-out;
+}
+
+.plan-node-header.collapsed + .plan-node-body {
+  display: none;
+}
+
+/* Custom animations */
+@keyframes fadeIn {
+  from { opacity: 0; transform: translateY(-10px); }
+  to { opacity: 1; transform: translateY(0); }
+}
+
+.fade-in {
+  animation: fadeIn 0.3s ease-out;
+}
+
+/* Print styles */
+@media print {
+  .no-print {
+    display: none !important;
+  }
+  
+  .bg-gray-50 {
+    background-color: white !important;
+  }
+  
+  .shadow, .shadow-sm {
+    box-shadow: none !important;
+  }
+}

data/app/controllers/rails_db_inspector/application_controller.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module RailsDbInspector
+  class ApplicationController < ActionController::Base
+    protect_from_forgery with: :exception
+
+    before_action :ensure_enabled!
+
+    private
+
+    def ensure_enabled!
+      head :not_found unless RailsDbInspector.configuration.enabled
+    end
+  end
+end

data/app/controllers/rails_db_inspector/queries_controller.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module RailsDbInspector
+  class QueriesController < ApplicationController
+    include RailsDbInspector::ApplicationHelper
+
+    def index
+      all_queries = RailsDbInspector::QueryStore.instance.all.reverse
+      @queries = all_queries
+      @query_groups = group_queries_by_action(all_queries)
+      @n_plus_ones = detect_n_plus_one(all_queries)
+    end
+
+    def show
+      @query = RailsDbInspector::QueryStore.instance.find(params[:id])
+      head :not_found unless @query
+    end
+
+    def explain
+      @query = RailsDbInspector::QueryStore.instance.find(params[:id])
+      return head :not_found unless @query
+
+      analyze = ActiveModel::Type::Boolean.new.cast(params[:analyze])
+
+      if analyze && !RailsDbInspector.configuration.allow_explain_analyze
+        return render plain: "EXPLAIN ANALYZE is disabled. Enable RailsDbInspector.configuration.allow_explain_analyze = true", status: :forbidden
+      end
+
+      explainer = RailsDbInspector::Explain.for_connection(ActiveRecord::Base.connection)
+      @explain = explainer.explain(@query.sql, analyze: analyze)
+    rescue RailsDbInspector::Explain::DangerousQuery => e
+      render plain: e.message, status: :unprocessable_entity
+    rescue RailsDbInspector::Explain::UnsupportedAdapter => e
+      render plain: e.message, status: :not_implemented
+    end
+
+    def clear
+      RailsDbInspector::QueryStore.instance.clear!
+      redirect_to queries_path
+    end
+  end
+end

data/app/controllers/rails_db_inspector/schema_controller.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module RailsDbInspector
+  class SchemaController < ApplicationController
+    include RailsDbInspector::ApplicationHelper
+
+    def index
+      inspector = RailsDbInspector::SchemaInspector.new
+      @schema = inspector.introspect
+      @relationships = inspector.relationships
+    end
+  end
+end

data/app/helpers/rails_db_inspector/application_helper.rb

@@ -0,0 +1,274 @@
+require_relative "plan_renderer"
+
+module RailsDbInspector
+  module ApplicationHelper
+    def render_postgres_plan(plan_data)
+      renderer = RailsDbInspector::ApplicationHelper::PostgresPlanRenderer.new(plan_data)
+      (renderer.render_summary + renderer.render_tree).html_safe
+    end
+
+    def render_query_type(query)
+      sql = query.sql.downcase.strip
+
+      # Determine the primary operation
+      operation = case sql
+      when /^select\b/
+        "SELECT"
+      when /^insert\b/
+        "INSERT"
+      when /^update\b/
+        "UPDATE"
+      when /^delete\b/
+        "DELETE"
+      when /^with\b/
+        "CTE"
+      else
+        "OTHER"
+      end
+
+      # Add complexity indicators for SELECT queries
+      complexity_hints = []
+
+      if operation == "SELECT"
+        complexity_hints << "JOIN" if sql.include?(" join ")
+        complexity_hints << "SUBQUERY" if sql.include?("(select") || sql.include?("( select")
+        complexity_hints << "AGGREGATE" if sql.match?(/\b(count|sum|avg|max|min|group by)\b/)
+        complexity_hints << "ORDER BY" if sql.include?(" order by ")
+        complexity_hints << "WINDOW" if sql.include?(" over(") || sql.include?(" over (")
+      end
+
+      # Render the operation with complexity hints
+      result_html = "<span class=\"inline-flex items-center px-2 py-1 rounded text-xs font-medium bg-gray-100 text-gray-800\">#{operation}</span>"
+
+      complexity_hints.each do |hint|
+        case hint
+        when "JOIN"
+          result_html += " <span class=\"inline-flex items-center px-1.5 py-0.5 rounded text-xs bg-blue-100 text-blue-800\">#{hint}</span>"
+        when "AGGREGATE", "WINDOW"
+          result_html += " <span class=\"inline-flex items-center px-1.5 py-0.5 rounded text-xs bg-yellow-100 text-yellow-800\">#{hint}</span>"
+        when "SUBQUERY"
+          result_html += " <span class=\"inline-flex items-center px-1.5 py-0.5 rounded text-xs bg-red-100 text-red-800\">#{hint}</span>"
+        else
+          result_html += " <span class=\"inline-flex items-center px-1.5 py-0.5 rounded text-xs bg-green-100 text-green-800\">#{hint}</span>"
+        end
+      end
+
+      result_html.html_safe
+    end
+
+    def group_queries_by_action(queries)
+      return [] if queries.empty?
+
+      groups = []
+      current_group = nil
+
+      queries.each do |query|
+        controller_action = extract_controller_action_from_sql(query)
+
+        # Start a new group if:
+        # 1. No current group
+        # 2. Different controller/action
+        # 3. Time gap of more than 10 seconds from the last query in the group
+        if current_group.nil? ||
+           current_group[:action] != controller_action ||
+           time_gap_too_large?(query, current_group[:queries].last, 10.0)
+
+          current_group = {
+            action: controller_action,
+            queries: [],
+            start_time: query.timestamp,
+            request_type: determine_request_type_from_action(controller_action)
+          }
+          groups << current_group
+        end
+
+        current_group[:queries] << query
+      end
+
+      groups
+    end
+
+    def detect_n_plus_one(queries)
+      return [] if queries.length < 3
+
+      # Normalize queries by replacing literal values with placeholders
+      normalized = queries.map do |q|
+        {
+          query: q,
+          normalized: normalize_sql(q.sql)
+        }
+      end
+
+      # Group by normalized SQL
+      groups = normalized.group_by { |entry| entry[:normalized] }
+
+      # Find N+1 patterns: same normalized query appearing 3+ times
+      n_plus_ones = []
+      groups.each do |normalized_sql, entries|
+        next if entries.length < 3
+        next if normalized_sql.strip.empty?
+
+        # Skip schema/transaction queries
+        next if normalized_sql =~ /\A(BEGIN|COMMIT|ROLLBACK|SET|SHOW)\b/i
+
+        sample_query = entries.first[:query]
+        total_duration = entries.sum { |e| e[:query].duration_ms }
+
+        # Try to extract the table name
+        table = normalized_sql.match(/FROM\s+"?(\w+)"?/i)&.captures&.first || "unknown"
+
+        n_plus_ones << {
+          normalized_sql: normalized_sql,
+          count: entries.length,
+          queries: entries.map { |e| e[:query] },
+          sample: sample_query,
+          total_duration_ms: total_duration,
+          table: table,
+          name: sample_query.name
+        }
+      end
+
+      # Sort by count descending (worst offenders first)
+      n_plus_ones.sort_by { |n| -n[:count] }
+    end
+
+    private
+
+    def normalize_sql(sql)
+      normalized = sql.dup
+      # Remove SQL comments like /*...*/ (Rails marginal annotations)
+      normalized.gsub!(/\/\*.*?\*\//m, "")
+      # Replace string literals 'value' with ?
+      normalized.gsub!(/'[^']*'/, "?")
+      # Replace numeric literals (integers and floats)
+      normalized.gsub!(/\b\d+(\.\d+)?\b/, "?")
+      # Replace $1, $2 style bind params
+      normalized.gsub!(/\$\d+/, "?")
+      # Collapse whitespace
+      normalized.gsub!(/\s+/, " ")
+      normalized.strip
+    end
+
+    def extract_controller_action_from_sql(query)
+      sql = query.sql.to_s
+
+      # Look for controller and action in SQL comments
+      if match = sql.match(/\/\*.*?controller='([^']+)'.*?action='([^']+)'.*?\*\//)
+        controller = match[1]
+        action = match[2]
+
+        # Handle namespaced controllers properly
+        if controller.include?("/")
+          # Convert api/users to Api::UsersController
+          controller_parts = controller.split("/")
+          namespaced_controller = controller_parts.map(&:camelize).join("::") + "Controller"
+        else
+          namespaced_controller = "#{controller.camelize}Controller"
+        end
+
+        return "#{namespaced_controller}##{action}"
+      end
+
+      # Fallback to query name if no controller/action found
+      query.name.to_s.presence || "Unknown Query"
+    end
+
+    def determine_request_type_from_action(action_name)
+      case action_name
+      when /API::|Api::|\/api\//i
+        :api
+      when /Controller#/
+        # Check if it looks like an API endpoint based on common patterns
+        if action_name.match(/(show|index|create|update|destroy)/) &&
+           !action_name.match(/(new|edit)/)
+          # Could be API if no render actions (new/edit are typically web-only)
+          :web_or_api
+        else
+          :web_request
+        end
+      when /Load|Create|Update|Delete|Destroy/
+        :model_operation
+      when /Schema|Migration/i
+        :schema
+      else
+        :other
+      end
+    end
+
+    def group_icon(request_type)
+      case request_type
+      when :api
+        "🔗"
+      when :web_request
+        "🌐"
+      when :web_or_api
+        "🔀"  # Mixed/ambiguous
+      when :model_operation
+        "📊"
+      when :schema
+        "🗂️"
+      else
+        "💾"
+      end
+    end
+
+    def time_gap_too_large?(current_query, last_query, gap_seconds = 5.0)
+      return false if last_query.nil?
+
+      current_time = parse_timestamp(current_query.timestamp)
+      last_time = parse_timestamp(last_query.timestamp)
+
+      (current_time - last_time).abs > gap_seconds
+    end
+
+    def parse_timestamp(timestamp)
+      case timestamp
+      when Time
+        timestamp
+      when Numeric
+        Time.at(timestamp)
+      else
+        Time.parse(timestamp.to_s)
+      end
+    rescue
+      Time.now
+    end
+
+    def format_group_time_range(group)
+      return "" if group[:queries].empty?
+
+      start_time = parse_timestamp(group[:start_time])
+      end_time = parse_timestamp(group[:queries].last.timestamp)
+
+      if (end_time - start_time) < 1.0
+        start_time.strftime("%H:%M:%S")
+      else
+        "#{start_time.strftime('%H:%M:%S')} - #{end_time.strftime('%H:%M:%S')}"
+      end
+    end
+
+    # JSON serialization helpers for schema visualization
+    def schema_to_json(schema)
+      result = {}
+      schema.each do |table_name, info|
+        result[table_name] = {
+          columns: info[:columns].map { |c| { name: c[:name], type: c[:type], nullable: c[:nullable], default: c[:default] } },
+          indexes: info[:indexes].map { |i| { name: i[:name], columns: i[:columns], unique: i[:unique] } },
+          foreign_keys: info[:foreign_keys].map { |fk| { column: fk[:column], to_table: fk[:to_table], primary_key: fk[:primary_key] } },
+          primary_key: info[:primary_key],
+          row_count: info[:row_count],
+          associations: (info[:associations] || []).map { |a| { name: a[:name], macro: a[:macro], target_table: a[:target_table], foreign_key: a[:foreign_key], through: a[:through] } },
+          missing_indexes: info[:missing_indexes] || [],
+          polymorphic_columns: (info[:polymorphic_columns] || []).map { |p| { name: p[:name], type_column: p[:type_column], id_column: p[:id_column] } }
+        }
+      end
+      result.to_json
+    end
+
+    def relationships_to_json(relationships)
+      relationships.map do |r|
+        { from_table: r[:from_table], from_column: r[:from_column], to_table: r[:to_table], to_column: r[:to_column], type: r[:type].to_s }
+      end.to_json
+    end
+  end
+end