query_guard 0.4.2 → 0.5.1

data/README.md

@@ -1,63 +1,612 @@
-# query_guard
+# QueryGuard: Migration Safety for Rails
 
-Guardrails for ActiveRecord queries per request: maximum query count, slow query flagging, and optional `SELECT *` blocking.
+**Catch risky database changes before they reach production.**
+
+QueryGuard automatically analyzes your Rails migrations in CI to detect safety issues, preventing schema problems from entering your codebase. Built with deep query intelligence and security features for comprehensive database safety.
+
+## 🎯 What Makes QueryGuard Different?
+
+**v1.0 Focus: Migration Safety + Query Security**
+
+QueryGuard runs in your CI pipeline to:
+- **Analyze Migrations**: Detect risky patterns like non-nullable columns without defaults, missing rollbacks, performance issues
+- **Detect Security Issues**: Monitor for SQL injection, unusual query patterns, data exfiltration, mass assignment attacks
+- **Monitor Query Performance**: Track slow queries and unusual database activity
 
 ## Installation
 
-Add to your Gemfile:
+Add QueryGuard to your Gemfile:
 
 ```ruby
-gem "query_guard"
+group :development, :test do
+  gem 'query_guard'
+end
 ```
 
-## ⚙️ Configuration File
+Then install and generate the initializer:
 
-To enable and configure `QueryGuard` in your Rails application,  
-you need to create an initializer file with the configuration options below.
+```bash
+bundle install
+bundle exec rails generate query_guard:install
+```
 
----
+This creates `config/initializers/query_guard.rb` with essential configuration.
+
+## Configuration
+
+The default configuration works for most Rails applications. Edit `config/initializers/query_guard.rb` to customize:
+
+```ruby
+QueryGuard.configure do |config|
+  # Directory where migrations live (default: db/migrate)
+  config.migrations_directory = 'db/migrate'
+
+  # Environments to analyze (default: [:development, :test])
+  config.enabled_environments = [:development, :test]
+
+  # Max queries to check per request (prevent timeouts)
+  config.max_queries_per_request = 100
+
+  # Max query duration in ms before flagging as slow
+  config.max_duration_ms_per_query = 5000
+end
+```
+
+## Using QueryGuard
+
+### In CI (Recommended)
+
+Add to your CI workflow to check migrations automatically:
+
+```yaml
+# .github/workflows/db-safety.yml
+name: Database Safety
+on: [pull_request]
 
-### 1️⃣ Create the configuration file
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:14
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        env:
+          POSTGRES_PASSWORD: postgres
 
-Run this command inside your Rails app:
+    steps:
+      - uses: actions/checkout@v3
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: 3.3.0
+          bundler-cache: true
+
+      - name: Analyze migrations
+        run: bundle exec queryguard analyze
+        env:
+          DATABASE_URL: postgres://postgres:postgres@localhost/test_db
+```
+
+### Locally
 
 ```bash
-touch config/initializers/query_guard.rb
+# Analyze migrations in your current environment
+bundle exec queryguard analyze
+
+# Get JSON output for integration with CI dashboards
+bundle exec queryguard analyze --format json
 ```
 
-### 2️⃣ Add the following code inside that file:
+## JSON Output
+
+QueryGuard can output analysis results as JSON for integration with CI dashboards, security tools, or custom workflows:
+
+```bash
+bundle exec queryguard analyze --format json
+```
+
+### JSON Schema
+
+```json
+{
+  "status": "success",
+  "findings": [
+    {
+      "type": "risky_migration",
+      "severity": "high",
+      "message": "Adding non-nullable column without default",
+      "file": "db/migrate/20240101120000_add_user_status.rb",
+      "line": 4,
+      "context": {
+        "migration_name": "AddUserStatus",
+        "environment": "test"
+      }
+    }
+  ],
+  "metadata": {
+    "environment": "test",
+    "database": "postgresql",
+    "schema_version": "20240101120000"
+  }
+}
+```
+
+## What's Coming Next
+
+**v2.0 (Future)**
+- Query analysis for N+1 problems and missing indexes
+- Live application query monitoring
+- Performance recommendations based on actual query patterns
+- SaaS dashboard integration
+
+**v1.0** focuses exclusively on migration safety because:
+1. Migrations are deterministic and analyzable in CI
+2. Migration problems are expensive to fix in production
+3. This gives you immediate, measurable value
+
+For query monitoring in your application, see QueryGuard's query analysis (coming v2.0) or integrate with tools like:
+- [New Relic APM](https://newrelic.com)
+- [DataDog APM](https://www.datadoghq.com)
+- [Sentry Performance](https://sentry.io)
+
+## Development
+
+To set up the development environment:
+
+```bash
+git clone https://github.com/yourusername/query_guard.git
+cd query_guard
+bundle install
+bundle exec rake spec
+```
+
+All 186+ tests should pass. QueryGuard maintains 100% test coverage for core analysis logic.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
+
+## License
+
+MIT License - See [LICENSE.txt](LICENSE.txt) for details.
+
+## Support
+
+- **Issues**: [GitHub Issues](https://github.com/yourusername/query_guard/issues)
+- **Discussions**: [GitHub Discussions](https://github.com/yourusername/query_guard/discussions)
+- **Docs**: See [INDEX.md](INDEX.md) for complete documentation
+
+---
+
+### Why QueryGuard?
+
+Database migrations are a critical part of your deployment pipeline. A single unsafe migration can:
+- Lock production tables for hours
+- Cause deployment failures requiring emergency rollbacks
+- Add months of technical debt to your schema
+
+QueryGuard catches these issues in CI, before they reach production.
+
+**Stop deploying database surprises.** Start using QueryGuard.
+=======
+QueryGuard fills a unique niche between traditional APMs and Rails profilers:
+
+- **vs Datadog APM**: Datadog excels at distributed tracing, logs/metrics correlation, and infrastructure monitoring, but is generic across languages. QueryGuard is **deeply Rails and ActiveRecord-aware**, providing schema intelligence and query budgets that Datadog doesn't offer.
+
+- **vs Skylight**: Skylight is an excellent Rails profiler with request timelines, deploy tracking, and background job support. QueryGuard complements this by adding **query budgets/SLOs per endpoint**, **schema-aware linting** (missing indexes, wide scans), and **first-class test/CI support** via RSpec matchers.
+
+- **vs Grafana**: Grafana provides powerful multi-signal visualization and metrics↔traces workflows via exemplars, but isn't Rails-specific. QueryGuard offers **Rails-native tooling** that works in development, tests, and production with minimal configuration.
+
+**QueryGuard's Focus**: SQL + Schema + Budget enforcement + Developer UX (console, tests, CI).
+
+## 📦 Installation
+
+Add to your Gemfile:
 
 ```ruby
-# config/initializers/query_guard.rb
+gem "query_guard"
+```
+
+Then run:
 
-# Configure QueryGuard settings
+```bash
+bundle install
+```
+
+## ⚙️ Configuration
+
+Create an initializer at `config/initializers/query_guard.rb`:
+
+```ruby
 QueryGuard.configure do |config|
   # Environments where QueryGuard should be active
-  # By default: [:development, :test]
-  config.enabled_environments = %i[development test]
+  config.enabled_environments = %i[development test production]
 
-  # Maximum number of SQL queries allowed per request
-  # Use nil to disable this limit
-  config.max_queries_per_request = 100
+  # === Budget System (New in v2) ===
+  # Define query budgets/SLOs for specific endpoints or jobs
+  
+  # Controller actions
+  config.budget.for("users#index", count: 10, duration_ms: 500)
+  config.budget.for("posts#show", count: 5, duration_ms: 200)
+  config.budget.for("admin/reports#dashboard", count: 50, duration_ms: 2000)
+  
+  # Background jobs
+  config.budget.for_job("EmailJob", count: 20, duration_ms: 1000)
+  config.budget.for_job("DataExportJob", count: 100, duration_ms: 5000)
+  
+  # Budget enforcement mode
+  config.budget.mode = :log       # :log (warn only), :notify (callback), :raise (exception)
+  
+  # Optional: callback for :notify mode
+  config.budget.on_violation = ->(key, violation) {
+    # Send to error tracker, metrics service, etc.
+    Honeybadger.notify("Budget violation", context: { key: key, violation: violation })
+  }
 
-  # Maximum duration (milliseconds) for a single SQL query
-  # Logs as a slow query if exceeded
+  # === Legacy Limits (Still Supported) ===
+  config.max_queries_per_request = 100
   config.max_duration_ms_per_query = 100.0
-
-  # Whether to flag or block SELECT * statements
   config.block_select_star = true
-
-  # Ignore certain SQL patterns (e.g., schema and transaction queries)
+  
+  # Ignore certain SQL patterns
   config.ignored_sql = [
-    /^PRAGMA /i,  # SQLite schema queries
+    /^PRAGMA /i,
     /^BEGIN/i,
-    /^COMMIT/i
+    /^COMMIT/i,
+    /^SHOW /i
   ]
 
-  # Raise exception on violation instead of just logging
-  config.raise_on_violation = false
+  # === Security Features ===
+  config.enable_security = true
+  config.detect_sql_injection = true
+  config.detect_unusual_query_pattern = true
+  config.detect_data_exfiltration = true
+  config.detect_mass_assignment = true
 
-  # Prefix for log messages in Rails logs
+  # === Export Configuration ===
+  config.base_url = ENV["QUERY_GUARD_API_URL"]
+  config.api_key = ENV["QUERY_GUARD_API_KEY"]
+  config.project = "my_app"
+  config.env = Rails.env
+  
+  # Logging
+  config.raise_on_violation = false  # Set to true in CI
   config.log_prefix = "[QueryGuard]"
 end
 ```
+
+## 🚀 Features
+
+### 1. Query Budgets & SLOs
+
+Define query budgets for specific endpoints or background jobs:
+
+```ruby
+# In config/initializers/query_guard.rb
+QueryGuard.configure do |config|
+  # Set budgets for controller actions
+  config.budget.for("users#index", count: 10, duration_ms: 500)
+  config.budget.for("posts#show", count: 5, duration_ms: 200)
+  
+  # Set budgets for background jobs
+  config.budget.for_job("EmailJob", count: 20)
+  config.budget.for_job("ReportJob", count: 100, duration_ms: 5000)
+  
+  # Choose enforcement mode
+  config.budget.mode = :log  # Options: :log, :notify, :raise
+end
+```
+
+**Enforcement Modes**:
+
+- `:log` - Logs warnings when budgets are exceeded (default, safe for production)
+- `:notify` - Calls a custom callback for integration with error trackers
+- `:raise` - Raises an exception (useful in test/CI environments)
+
+**With Callbacks**:
+
+```ruby
+config.budget.mode = :notify
+config.budget.on_violation = ->(key, violation) {
+  # Send to your monitoring service
+  Datadog::Statsd.new.increment("query_guard.budget.exceeded", tags: ["endpoint:#{key}"])
+  
+  # Or send to error tracker
+  Sentry.capture_message("Budget exceeded", extra: { key: key, violation: violation })
+}
+```
+
+### 2. Trace API (Console & Testing)
+
+Manually trace query performance in any context:
+
+```ruby
+# In Rails console or tests
+result, report = QueryGuard.trace("load active users") do
+  User.where(active: true).limit(100).to_a
+end
+
+puts "Queries executed: #{report.query_count}"
+puts "Total duration: #{report.total_duration_ms}ms"
+puts "Violations: #{report.violations.inspect}"
+puts "Queries:"
+report.queries.each do |q|
+  puts "  #{q[:duration_ms]}ms: #{q[:sql]}"
+end
+```
+
+**With Context**:
+
+```ruby
+result, report = QueryGuard.trace("process batch", context: { batch_id: 123, user_id: 456 }) do
+  Batch.find(123).process!
+end
+
+# Context is included in the report for correlation
+puts report.context  # => { batch_id: 123, user_id: 456 }
+```
+
+### 3. RSpec Matchers
+
+Test query performance in your specs:
+
+```ruby
+require "query_guard/rspec"
+
+RSpec.describe UsersController, type: :controller do
+  describe "GET #index" do
+    it "stays within query budget" do
+      expect {
+        get :index
+      }.to_not exceed_query_budget(count: 10, duration_ms: 500)
+    end
+    
+    # Or use named budgets defined in config
+    it "respects users#index budget" do
+      expect {
+        get :index
+      }.to_not exceed_query_budget("users#index")
+    end
+  end
+end
+```
+
+**Helper Method**:
+
+```ruby
+RSpec.describe "batch processing" do
+  it "processes batch efficiently" do
+    report = within_query_budget(count: 50, duration_ms: 2000) do
+      Batch.process_all
+    end
+    
+    expect(report.query_count).to be < 50
+  end
+end
+```
+
+### 4. SQL Fingerprinting & Statistics
+
+Track query patterns across your application:
+
+```ruby
+# In console or background job
+QueryGuard::Fingerprint.record("SELECT * FROM users WHERE id = 123", 45.2)
+QueryGuard::Fingerprint.record("SELECT * FROM users WHERE id = 456", 32.1)
+
+# Get stats for a specific fingerprint
+fp = QueryGuard::Fingerprint.generate("SELECT * FROM users WHERE id = ?")
+stats = QueryGuard::Fingerprint.stats_for(fp)
+
+puts stats[:count]              # => 2
+puts stats[:total_duration_ms]  # => 77.3
+puts stats[:min_duration_ms]    # => 32.1
+puts stats[:max_duration_ms]    # => 45.2
+puts stats[:first_seen_at]
+puts stats[:last_seen_at]
+
+# Get top queries by various metrics
+QueryGuard::Fingerprint.top_by_count(10)         # Most frequently executed
+QueryGuard::Fingerprint.top_by_duration(10)      # Highest total time
+QueryGuard::Fingerprint.top_by_avg_duration(10)  # Slowest on average
+```
+
+**Fingerprinting normalizes SQL**:
+- Removes string and numeric literals
+- Collapses whitespace
+- Normalizes `IN (...)` lists
+- Returns consistent SHA1 hash
+
+```ruby
+# These all produce the same fingerprint:
+QueryGuard::Fingerprint.generate("SELECT * FROM users WHERE id = 1")
+QueryGuard::Fingerprint.generate("SELECT * FROM users WHERE id = 999")
+QueryGuard::Fingerprint.generate("SELECT  *  FROM users WHERE id = 42")
+# All normalize to: "select * from users where id = ?"
+```
+
+### 5. Security Features (Existing)
+
+QueryGuard includes built-in security detection:
+
+- **SQL Injection Detection**: Flags suspicious patterns (OR 1=1, UNION SELECT, etc.)
+- **Unusual Query Patterns**: Rate limiting per actor (IP/user)
+- **Data Exfiltration**: Monitors large responses and suspicious endpoints
+- **Mass Assignment**: Detects unpermitted parameters
+
+```ruby
+config.enable_security = true
+config.detect_sql_injection = true
+config.detect_unusual_query_pattern = true
+config.max_queries_per_minute_per_actor = 300
+
+# Custom actor resolver
+config.actor_resolver = ->(env) {
+  env["warden"].user&.id || env["action_dispatch.remote_ip"]
+}
+```
+
+### 6. Export & Monitoring
+
+Export query data to external services:
+
+```ruby
+config.base_url = "https://your-monitoring-service.com"
+config.api_key = ENV["MONITORING_API_KEY"]
+config.project = "my_rails_app"
+config.env = Rails.env
+config.export_mode = :async  # Don't block requests
+```
+
+Exported data includes:
+- Query statements with fingerprints
+- Durations and timestamps
+- Budget violations
+- Security threat events
+- Request context (controller, action, user, etc.)
+
+## 📊 Use Cases
+
+### Development
+
+```ruby
+# In Rails console
+result, report = QueryGuard.trace("diagnose N+1") do
+  Post.limit(10).each { |post| post.comments.to_a }
+end
+
+puts "Queries: #{report.query_count}"  # Spot N+1 problems immediately
+report.queries.each { |q| puts q[:sql] }
+```
+
+### Testing (CI)
+
+```ruby
+# spec/support/query_guard.rb
+RSpec.configure do |config|
+  config.around(:each, :query_budget) do |example|
+    metadata = example.metadata
+    budget = metadata[:query_budget]
+    
+    expect {
+      example.run
+    }.to_not exceed_query_budget(**budget)
+  end
+end
+
+# spec/controllers/users_controller_spec.rb
+RSpec.describe UsersController do
+  describe "GET #index", :query_budget, query_budget: { count: 5, duration_ms: 200 } do
+    it "loads users" do
+      get :index
+      expect(response).to be_successful
+    end
+  end
+end
+```
+
+### Production Monitoring
+
+```ruby
+# config/initializers/query_guard.rb
+QueryGuard.configure do |config|
+  config.enabled_environments = [:production]
+  config.budget.mode = :notify
+  
+  config.budget.for("api/v1/users#index", count: 10, duration_ms: 100)
+  config.budget.for("api/v1/posts#feed", count: 15, duration_ms: 150)
+  
+  config.budget.on_violation = ->(key, violation) {
+    # Alert when budgets exceeded in production
+    Datadog::Statsd.new.increment("query.budget.exceeded", tags: [
+      "endpoint:#{key}",
+      "type:#{violation[:type]}"
+    ])
+  }
+end
+```
+
+## 🔧 API Reference
+
+### QueryGuard.configure
+
+Configure QueryGuard settings. See Configuration section above.
+
+### QueryGuard.trace(label, context: {}, &block)
+
+Trace a block of code and capture query statistics.
+
+**Arguments**:
+- `label` (String): Descriptive label for the trace
+- `context` (Hash): Additional context (user_id, batch_id, etc.)
+- `&block`: Code to trace
+
+**Returns**: `[result, report]` tuple
+
+### QueryGuard::Budget
+
+**Methods**:
+- `.for(key, **limits)`: Define budget for controller action
+- `.for_job(job, **limits)`: Define budget for background job
+- `.mode=`: Set enforcement mode (`:log`, `:notify`, `:raise`)
+- `.on_violation=`: Set callback for `:notify` mode
+
+### QueryGuard::Fingerprint
+
+**Methods**:
+- `.generate(sql)`: Generate fingerprint for SQL
+- `.normalize(sql)`: Normalize SQL query
+- `.record(sql, duration_ms)`: Record query execution
+- `.stats_for(fingerprint)`: Get stats for fingerprint
+- `.top_by_count(limit)`: Top queries by count
+- `.top_by_duration(limit)`: Top queries by total duration
+- `.top_by_avg_duration(limit)`: Top queries by average duration
+- `.reset!`: Clear all stats
+
+### RSpec Matchers
+
+```ruby
+require "query_guard/rspec"
+
+expect { code }.to_not exceed_query_budget(count: 10)
+expect { code }.to_not exceed_query_budget(count: 10, duration_ms: 500)
+expect { code }.to_not exceed_query_budget("users#index")
+
+report = within_query_budget(count: 10) { code }
+```
+
+## 🧪 Testing
+
+Run the test suite:
+
+```bash
+bundle exec rspec
+```
+
+## 📄 License
+
+MIT License. See [LICENSE.txt](LICENSE.txt) for details.
+
+## 🙏 Contributing
+
+Contributions welcome! Please:
+
+1. Fork the repository
+2. Create a feature branch
+3. Write tests for your changes
+4. Submit a pull request
+
+## 📮 Support
+
+- Issues: [GitHub Issues](https://github.com/Chitradevi36/query_guard/issues)
+- Documentation: This README
+- Example Rails App: [Coming soon]
+
+---
+
+Built with ❤️ to make Rails query performance monitoring delightful.
+>>>>>>> 2fd3b3b87eaac58f779bc220c6bff2389c1b1c7b
+

data/exe/queryguard

@@ -0,0 +1,23 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Handle --version early (before loading Rails)
+if ARGV.first == '--version' || ARGV.first == '-v'
+  $LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+  require 'query_guard/version'
+  puts "QueryGuard v#{QueryGuard::VERSION}"
+  exit 0
+end
+
+# Now load everything else
+$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+require 'query_guard'
+require 'query_guard/cli'
+
+begin
+  QueryGuard::CLI.run(ARGV)
+rescue => e
+  puts "Error: #{e.message}"
+  puts e.backtrace.join("\n") if ENV['DEBUG']
+  exit 1
+end

data/lib/query_guard/action_controller_subscriber.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+require "active_support/notifications"
+
+module QueryGuard
+  module ActionControllerSubscriber
+    EVENT = "unpermitted_parameters.action_controller"
+
+    def self.install!(config)
+      return if @installed
+      @subscriber = ActiveSupport::Notifications.subscribe(EVENT) do |_name, _start, _finish, _id, payload|
+        stats = Thread.current[:query_guard_stats]
+        next unless stats
+        next unless config.enable_security && config.detect_mass_assignment
+
+        keys = Array(payload[:keys]).map(&:to_s)
+        sensitive = keys & Array(config.sensitive_param_keys)
+
+        stats[:violations] << {
+          type: :mass_assignment_unpermitted_params,
+          keys: keys.take(50),
+          sensitive_keys: sensitive.take(50)
+        }
+      end
+      @installed = true
+    end
+  end
+end