pg_insights 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3575be5a5a333d43c44c555f5f736c83e5458788143d92b5ca5ec765e3cbd083
+  data.tar.gz: be967ed421de1a002b52ef7b476fd58a02c866e6cf429ddeef053672831679a5
+SHA512:
+  metadata.gz: b1756de4bb9c8aa2bceab70f936d79a85d678376917f0976bf2b9fa90ef4cc54fe4683387166ba6b93fe2e72f50d879a7185c89f292a87b60f782e92790c38df
+  data.tar.gz: 17594af5eeeb6ed8812a6a7659635ce5efa032c6f67bb882fd8caf52dc617fe54b2dd2773c6883047be386c736f0826d6224cece06ebe06b0c6e0a5e49a2203c

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright MezbahAlam
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,183 @@
+# PgInsights
+
+**PostgreSQL performance monitoring for Rails apps**
+
+[![Gem Version](https://badge.fury.io/rb/pg_insights.svg)](https://badge.fury.io/rb/pg_insights)
+[![CI](https://github.com/mezbahalam/pg_insights/actions/workflows/ci.yml/badge.svg)](https://github.com/mezbahalam/pg_insights/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+PgInsights is a Rails engine that gives you a web dashboard for monitoring your PostgreSQL database performance. Think of it as a lightweight alternative to external monitoring tools that lives right inside your Rails app.
+
+## Why I built this
+
+I got tired of switching between different tools to check database performance. Sometimes you just want to quickly see which indexes aren't being used, or find slow queries without setting up a whole monitoring infrastructure. PgInsights gives you that - a simple dashboard you can access at `/pg_insights` in your Rails app.
+
+## What you get
+
+**Health Dashboard**
+- Find unused indexes that are wasting space
+- Spot tables that might need indexes (high sequential scans)
+- Identify slow queries (if you have pg_stat_statements enabled)
+- Check for table bloat that needs cleanup
+- Review PostgreSQL configuration settings
+
+**Query Runner**
+- Run your own SELECT queries safely
+- Built-in queries for common performance checks
+- Save queries you use frequently
+- Results displayed as tables or charts
+
+**Smart execution**
+- Runs health checks in background jobs if you have them set up
+- Falls back to running directly if you don't
+- Caches results so repeated visits are fast
+- Configurable timeouts to prevent slow queries from hanging
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem 'pg_insights'
+```
+
+Run the installer:
+
+```bash
+bundle install
+rails generate pg_insights:install
+rails db:migrate
+```
+
+That's it. Visit `/pg_insights` in your browser.
+
+## Configuration
+
+The engine works out of the box, but you can customize it:
+
+```ruby
+# config/initializers/pg_insights.rb
+PgInsights.configure do |config|
+  # Run health checks in background (default: true)
+  config.enable_background_jobs = true
+  
+  # How long to cache results (default: 5 minutes)
+  config.health_cache_expiry = 10.minutes
+  
+  # Timeout for health check queries (default: 10 seconds)
+  config.health_check_timeout = 15.seconds
+  
+  # Queue name for background jobs (default: :pg_insights_health)
+  config.background_job_queue = :low_priority
+end
+```
+
+## How Background Jobs Work
+
+**PgInsights uses on-demand background jobs, not automatic scheduling.**
+
+### When health checks run:
+- ✅ **When you visit the health dashboard `/pg_insights/health`** and cached data is older than `health_cache_expiry` (default: 5 minutes)
+- ✅ **When you click the "Refresh" button** in the dashboard
+- ✅ **When you run** `rails pg_insights:health_check` manually
+- ❌ **NOT automatically** - PgInsights doesn't run background jobs on its own
+
+### How caching works:
+```
+Visit at 2:00 PM → Runs health checks, caches results for 5 minutes
+Visit at 2:03 PM → Uses cached results (still fresh)
+Visit at 2:06 PM → Data is stale, triggers new background jobs
+```
+
+### Background job setup (optional but recommended):
+
+If your app has background jobs (Sidekiq, Resque, etc.), PgInsights will use them for better performance:
+
+```bash
+# Check if background jobs are working
+rails pg_insights:status
+```
+
+**Without background jobs**: Health checks run synchronously when you visit the page (slower but works)  
+**With background jobs**: Health checks run asynchronously (faster, non-blocking)
+
+### Optional: Automatic recurring checks
+
+If you want health checks to run automatically (not just on-demand), set up a scheduler:
+
+```ruby
+# Using whenever (runs every hour)
+every 1.hour do
+  runner "PgInsights::RecurringHealthChecksJob.perform_later"
+end
+
+# Using sidekiq-cron
+Sidekiq::Cron::Job.create(
+  name: 'PgInsights Health Checks',
+  cron: '0 * * * *',
+  class: 'PgInsights::RecurringHealthChecksJob'
+)
+```
+
+**Note**: Even with automatic scheduling, the jobs are smart - they only run expensive queries if the cached data is actually stale.
+
+## Usage
+
+Navigate to `/pg_insights` in your app. The interface is pretty straightforward:
+
+- **Main page**: Run queries and see results as tables or charts
+- **Health tab**: Database performance overview
+- **Query examples**: Pre-built queries for common checks
+
+All queries are read-only (SELECT statements only) and have timeouts to prevent issues.
+
+## Available rake tasks
+
+```bash
+rails pg_insights:status        # Check configuration
+rails pg_insights:health_check  # Run health checks manually
+rails pg_insights:stats         # Show usage statistics
+rails pg_insights:clear_data    # Clear stored data and caches
+```
+
+## Safety
+
+- Only SELECT queries allowed
+- Query timeouts prevent long-running queries
+- Focuses on public schema by default
+- No modification of your data
+
+## Uninstalling
+
+```bash
+rails generate pg_insights:clean
+rails db:rollback STEP=2
+# Remove gem from Gemfile
+```
+
+## Requirements
+
+- Rails 6.1+
+- PostgreSQL
+- For slow query detection: pg_stat_statements extension (optional)
+
+## Contributing
+
+Found a bug or have an idea? Open an issue or send a pull request. The codebase is pretty straightforward.
+
+Development setup:
+
+```bash
+git clone https://github.com/mezbahalam/pg_insights.git
+cd pg_insights
+bundle install
+bundle exec rake spec
+```
+
+## License
+
+MIT License. See [LICENSE](MIT-LICENSE) file.
+
+---
+
+Built by [Mezbah Alam](https://github.com/mezbahalam). Inspired by pg_hero and other database monitoring tools.

data/Rakefile

@@ -0,0 +1,8 @@
+require "bundler/setup"
+
+APP_RAKEFILE = File.expand_path("spec/dummy/Rakefile", __dir__)
+load "rails/tasks/engine.rake"
+
+load "rails/tasks/statistics.rake"
+
+require "bundler/gem_tasks"

data/app/assets/javascripts/pg_insights/application.js

@@ -0,0 +1,436 @@
+//= require chartkick
+//= require Chart.bundle
+//= require_tree .
+
+
+// PG Insights JavaScript
+document.addEventListener('DOMContentLoaded', function() {
+  const InsightsApp = {
+    // Configuration - will be set from data attribute
+    config: {
+      queries: [],
+      savedQueries: []
+    },
+    // Keep track of the currently loaded query
+    currentQueryState: {
+      id: null,
+      type: null, // 'built-in' or 'saved'
+      name: ''
+    },
+
+    // Initialize the application
+    init() {
+      this.loadQueriesFromDataAttribute();
+      this.loadSavedQueries();
+      this.bindEvents();
+      this.validateInitialQuery();
+      this.setupQueryExamples();
+      this.loadTableNames();
+    },
+
+    // Load queries from data attribute
+    loadQueriesFromDataAttribute() {
+      const container = document.querySelector('.insights-container');
+      if (container && container.dataset.queries) {
+        try {
+          this.config.queries = JSON.parse(container.dataset.queries);
+        } catch (e) {
+          console.error('Failed to parse queries data:', e);
+          this.config.queries = [];
+        }
+      }
+    },
+
+    // Load saved queries from localStorage (keeping existing method)
+    loadSavedQueries() {
+      // Keep this for backward compatibility if needed elsewhere
+    },
+
+    // Copy current query functionality
+    copyCurrentQuery() {
+      const textarea = document.querySelector('.sql-editor');
+      const btn = document.querySelector('.btn-icon.btn-copy');
+      
+      if (!textarea?.value.trim()) return;
+      
+      btn.disabled = true;
+      btn.textContent = '✓';
+      
+      navigator.clipboard.writeText(textarea.value).then(() => {
+        setTimeout(() => {
+          btn.disabled = false;
+          btn.textContent = '📋';
+        }, 1000);
+      });
+    },
+
+    // Save or Update the current query
+    saveCurrentQuery() {
+      const textarea = document.querySelector('.sql-editor');
+      const sql = textarea?.value.trim();
+      if (!sql) return;
+
+      const isUpdate = this.currentQueryState.type === 'saved';
+      let name;
+
+      if (isUpdate) {
+        name = prompt("Update query name, or confirm current name:", this.currentQueryState.name);
+      } else {
+        name = prompt("Enter a name for this new saved query:");
+      }
+
+      if (!name) return; // User cancelled prompt
+
+      const method = isUpdate ? 'PATCH' : 'POST';
+      const url = isUpdate ? `/pg_insights/queries/${this.currentQueryState.id}` : '/pg_insights/queries';
+
+      const body = {
+        query: {
+          name: name.trim(),
+          sql: sql,
+          // For now, description is not editable in the UI
+          description: isUpdate ? (this.currentQueryState.description || '') : 'User saved query',
+          category: 'saved'
+        }
+      };
+
+      const btn = document.querySelector('.btn-icon.btn-save');
+      btn.disabled = true;
+      btn.textContent = '⏳';
+
+      fetch(url, {
+        method: method,
+        headers: {
+          'Content-Type': 'application/json',
+          'X-CSRF-Token': document.querySelector('meta[name="csrf-token"]').getAttribute('content')
+        },
+        body: JSON.stringify(body)
+      })
+      .then(response => {
+        if (!response.ok) {
+          return response.json().then(err => { throw err; });
+        }
+        return response.json();
+      })
+      .then(data => {
+        if (data.success) {
+          btn.textContent = '✓';
+          location.reload(); // Easiest way to show updated query list
+        }
+      })
+      .catch(error => {
+        console.error('Save query error:', error);
+        btn.textContent = '✗';
+        const errorMessage = error.errors ? error.errors.join(', ') : 'A server error occurred.';
+        alert(`Failed to save query: ${errorMessage}`);
+        
+        // Restore button after a delay on failure
+        setTimeout(() => {
+          btn.disabled = false;
+          const icon = this.currentQueryState.type === 'saved' ? '📝' : '💾';
+          btn.textContent = icon;
+        }, 2000);
+      });
+    },
+
+    // Query validation
+    validateQuery(sql) {
+      if (!sql || !sql.trim()) {
+        return { valid: false, message: "Please enter a SQL query" };
+      }
+
+      const trimmedSql = sql.trim();
+
+      // Check if it starts with SELECT (case insensitive)
+      if (!trimmedSql.toLowerCase().startsWith('select')) {
+        return { valid: false, message: "Only SELECT statements are allowed" };
+      }
+
+      // Check for semicolons
+      if (trimmedSql.includes(';')) {
+        return { valid: false, message: "Semicolons (;) are not allowed. Please use a single SELECT statement only." };
+      }
+
+      // Check for forbidden keywords
+      const forbiddenWords = /\b(insert|update|delete|alter|drop|create|grant|revoke)\b/i;
+      const match = trimmedSql.match(forbiddenWords);
+      if (match) {
+        return { valid: false, message: `${match[1].toUpperCase()} statements are not allowed. Only SELECT queries are permitted.` };
+      }
+
+      return { valid: true };
+    },
+
+    // UI Updates
+    updateExecuteButton(isValid) {
+      const executeBtn = document.getElementById('execute-btn');
+      if (executeBtn) {
+        executeBtn.disabled = !isValid;
+        executeBtn.title = isValid ? 'Execute query' : 'Please fix query errors first';
+      }
+    },
+
+    validateAndUpdateUI(sql) {
+      const validation = this.validateQuery(sql);
+
+      if (validation.valid) {
+        this.hideValidationError();
+        this.updateExecuteButton(true);
+      } else {
+        this.showValidationError(validation.message);
+        this.updateExecuteButton(false);
+      }
+
+      return validation;
+    },
+
+    showValidationError(message) {
+      // Remove existing error
+      const existingError = document.querySelector('.validation-error');
+      if (existingError) {
+        existingError.remove();
+      }
+
+      // Create and show new error
+      const errorDiv = document.createElement('div');
+      errorDiv.className = 'validation-error';
+      errorDiv.innerHTML = `<span>⚠️ ${message}</span>`;
+
+      const textarea = document.querySelector('.sql-editor');
+      if (textarea && textarea.parentNode) {
+        textarea.parentNode.insertBefore(errorDiv, textarea.nextSibling);
+      }
+    },
+
+    hideValidationError() {
+      const existingError = document.querySelector('.validation-error');
+      if (existingError) {
+        existingError.remove();
+      }
+    },
+
+    // Query management
+    clearQuery() {
+      const textarea = document.querySelector('.sql-editor');
+      if (textarea) {
+        textarea.value = '';
+        textarea.focus();
+        this.validateAndUpdateUI('');
+      }
+      
+      // Reset state
+      this.currentQueryState = { id: null, type: null, name: '' };
+      
+      // Reset save button
+      const saveBtn = document.querySelector('.btn-icon.btn-save');
+      if (saveBtn) {
+          saveBtn.innerHTML = '💾';
+          saveBtn.title = 'Save query';
+      }
+    },
+
+    // Load table names for preview dropdown
+    loadTableNames() {
+      fetch('/pg_insights/table_names')
+        .then(res => res.json())
+        .then(data => {
+          const select = document.getElementById('table-preview-select');
+          if (select && data.tables) {
+            // Clear existing options except the first one
+            while (select.children.length > 1) {
+              select.removeChild(select.lastChild);
+            }
+            
+            // Add table options
+            data.tables.forEach(table => {
+              const option = document.createElement('option');
+              option.value = table;
+              option.textContent = table;
+              select.appendChild(option);
+            });
+          }
+        })
+        .catch(error => {
+          console.error('Failed to load table names:', error);
+        });
+    },
+
+    // Preview selected table
+    previewTable(tableName) {
+      if (!tableName) return;
+      
+      const sql = `SELECT * FROM ${tableName} LIMIT 10`;
+      const textarea = document.querySelector('.sql-editor');
+      
+      if (textarea) {
+        textarea.value = sql;
+        
+        // Validate the query
+        this.validateAndUpdateUI(sql);
+        
+        // Auto-resize textarea
+        textarea.style.height = 'auto';
+        textarea.style.height = Math.max(160, textarea.scrollHeight) + 'px';
+        
+        // Auto-execute the query
+        const executeBtn = document.getElementById('execute-btn');
+        if (executeBtn && !executeBtn.disabled) {
+          executeBtn.click();
+        }
+      }
+      
+      // Reset the dropdown to the default option
+      const select = document.getElementById('table-preview-select');
+      if (select) {
+        select.value = '';
+      }
+    },
+
+    loadQueryById(queryId) {
+      const query = this.config.queries.find(q => q.id.toString() === queryId.toString());
+      
+      if (!query) {
+        console.error('Query not found:', queryId);
+        return;
+      }
+
+      // Set current state
+      this.currentQueryState.id = query.id;
+      this.currentQueryState.name = query.name;
+      this.currentQueryState.description = query.description;
+      // Database IDs are numbers, built-in IDs are strings
+      this.currentQueryState.type = (typeof query.id === 'number') ? 'saved' : 'built-in';
+      
+      // Update save button
+      const saveBtn = document.querySelector('.btn-icon.btn-save');
+      if (saveBtn) {
+          if (this.currentQueryState.type === 'saved') {
+              saveBtn.innerHTML = '📝';
+              saveBtn.title = 'Update saved query';
+          } else {
+              saveBtn.innerHTML = '💾';
+              saveBtn.title = 'Save query as new';
+          }
+      }
+
+      const textarea = document.querySelector('.sql-editor');
+      if (textarea) {
+        textarea.value = query.sql;
+        textarea.focus();
+
+        // Validate the loaded query
+        this.validateAndUpdateUI(query.sql);
+
+        // Trigger auto-resize if available
+        const event = new Event('input', { bubbles: true });
+        textarea.dispatchEvent(event);
+      }
+    },
+
+    // Query examples setup
+    setupQueryExamples() {
+      // Setup category filtering
+      document.querySelectorAll('.filter-btn').forEach(button => {
+        button.addEventListener('click', () => {
+          const category = button.getAttribute('data-category');
+
+          // Update active state
+          document.querySelectorAll('.filter-btn').forEach(btn => btn.classList.remove('active'));
+          button.classList.add('active');
+
+          // Filter query buttons
+          document.querySelectorAll('.query-example-btn').forEach(queryBtn => {
+            const queryCategory = queryBtn.getAttribute('data-category');
+            if (category === 'all' || queryCategory === category) {
+              queryBtn.classList.remove('hidden');
+            } else {
+              queryBtn.classList.add('hidden');
+            }
+          });
+        });
+      });
+
+      // Setup query example button clicks
+      document.querySelectorAll('.query-example-btn').forEach(button => {
+        button.addEventListener('click', () => {
+          const queryId = button.getAttribute('data-query-id');
+          if (queryId) {
+            this.loadQueryById(queryId);
+          }
+        });
+      });
+    },
+
+    // Event binding
+    bindEvents() {
+      const textarea = document.querySelector('.sql-editor');
+      const executeBtn = document.getElementById('execute-btn');
+
+      // Real-time validation on input
+      if (textarea) {
+        textarea.addEventListener('input', () => {
+          // Auto-resize
+          textarea.style.height = 'auto';
+          textarea.style.height = Math.max(160, textarea.scrollHeight) + 'px';
+
+          // Instant validation
+          this.validateAndUpdateUI(textarea.value);
+        });
+
+        // Validation on paste
+        textarea.addEventListener('paste', () => {
+          // Small delay to let paste complete
+          setTimeout(() => {
+            this.validateAndUpdateUI(textarea.value);
+          }, 10);
+        });
+      }
+
+      // Prevent form submission if button is disabled
+      if (executeBtn) {
+        executeBtn.addEventListener('click', (event) => {
+          if (executeBtn.disabled) {
+            event.preventDefault();
+            if (textarea) {
+              textarea.focus();
+            }
+            return false;
+          }
+        });
+      }
+
+      // Global functions - expose methods to global scope for onclick handlers
+      window.clearQuery = () => this.clearQuery();
+      window.copyCurrentQuery = () => this.copyCurrentQuery();
+      window.saveCurrentQuery = () => this.saveCurrentQuery();
+
+      // Table preview dropdown
+      const tableSelect = document.getElementById('table-preview-select');
+      if (tableSelect) {
+        tableSelect.addEventListener('change', (event) => {
+          const tableName = event.target.value;
+          if (tableName) {
+            this.previewTable(tableName);
+          }
+        });
+      }
+    },
+
+    // Initial validation
+    validateInitialQuery() {
+      const textarea = document.querySelector('.sql-editor');
+      if (textarea) {
+        const initialValue = textarea.value.trim();
+        if (initialValue) {
+          this.validateAndUpdateUI(initialValue);
+        } else {
+          this.updateExecuteButton(false);
+          textarea.focus();
+        }
+      }
+    },
+  };
+
+  // Initialize the application
+  InsightsApp.init();
+});