solid_queue_heroku_autoscaler 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d232e8e59b0f059186709890d2b293b9498cbee779620582e58540908af2ce49
-  data.tar.gz: 9fad9708b81ca955208c77095a1f395c513e89d3524cfdb7d30d8357fb4fe346
+  metadata.gz: b7b31454d3f2b02f066a5ebf62b70e758a572f5a0ab406b6d090ffdb12e819c8
+  data.tar.gz: 90567a30026883b659cfac5b4fe3b1204e330781da31b1dfe4db0960f2336531
 SHA512:
-  metadata.gz: a61ccb5462fe1dfa0941c857aa1c60cf61a62e50338d13b122904b2d0208c3a9dd615a72dd320ceb124d6540b36453e070ca76657e66160b83a8aaeb04be622b
-  data.tar.gz: aec6a58d0243d07ef9ecc289b7dff1bbfa87dc8e031d2d92d44a497ef48efcf7d914399611b938be372ad5fca578a58cb5cc34a50b3a6ff7a0aae24ae03c0cf7
+  metadata.gz: 6c28ebe76d4529f674d91405bd2307e513ad6de767faa1a34bfc5ae64c600a95d49da1fe0b5dfb05067bfad64a651cc33816576b2d13efc341911332679af1d7
+  data.tar.gz: f7a12818f46e209751ede56e5d9cca3b8e36960690c166ef0104c09992de63a767ed03a29c0a9b921d98502c0e252e436476b985afbd12dd626873b09e2d3297

data/CHANGELOG.md

@@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **Dashboard UI**: Web-based dashboard for monitoring autoscaler events and status
+  - Overview page with real-time metrics, worker status, and recent events
+  - Workers page with detailed status, configuration, and cooldown info
+  - Events log with filtering by worker type
+  - Manual scale trigger from the UI
+- **ScaleEvent model**: Tracks all scaling decisions in the database
+- **Event recording configuration**: `record_events` and `record_all_events` options
+- **New rake tasks**: `events`, `cleanup_events` for managing scale event history
+- **Dashboard generator**: `rails generate solid_queue_heroku_autoscaler:dashboard`
+
 ## [0.1.0] - 2025-01-XX
 
 ### Added

data/README.md

@@ -41,6 +41,27 @@ rails db:migrate
 
 This creates a `solid_queue_autoscaler_state` table to store cooldown timestamps.
 
+### Dashboard Setup (Optional)
+
+For a web UI to monitor autoscaler events and status:
+
+```bash
+rails generate solid_queue_heroku_autoscaler:dashboard
+rails db:migrate
+```
+
+Then mount the dashboard in `config/routes.rb`:
+
+```ruby
+# With authentication (recommended)
+authenticate :user, ->(u) { u.admin? } do
+  mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/autoscaler"
+end
+
+# Or without authentication
+mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/autoscaler"
+```
+
 ## Quick Start
 
 ### Basic Configuration (Single Worker)
@@ -417,6 +438,64 @@ end
 
 In dry-run mode, all decisions are logged but no platform API calls are made.
 
+## Dashboard
+
+The optional dashboard provides a web UI for monitoring the autoscaler:
+
+### Features
+
+- **Overview Dashboard**: Real-time metrics, worker status, and recent events
+- **Workers View**: Detailed status for each worker type with configuration and cooldowns
+- **Events Log**: Historical record of all scaling decisions with filtering
+- **Manual Scaling**: Trigger scale operations directly from the UI
+
+### Setup
+
+1. Generate the dashboard migration:
+
+```bash
+rails generate solid_queue_heroku_autoscaler:dashboard
+rails db:migrate
+```
+
+2. Mount the engine in `config/routes.rb`:
+
+```ruby
+authenticate :user, ->(u) { u.admin? } do
+  mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/autoscaler"
+end
+```
+
+3. Visit `/autoscaler` in your browser
+
+### Event Recording
+
+By default, all scaling events are recorded to the database. Configure in your initializer:
+
+```ruby
+SolidQueueHerokuAutoscaler.configure do |config|
+  # Record scale_up, scale_down, skipped, and error events (default: true)
+  config.record_events = true
+  
+  # Also record no_change events (verbose, default: false)
+  config.record_all_events = false
+end
+```
+
+### Rake Tasks for Events
+
+```bash
+# View recent scale events
+bundle exec rake solid_queue_autoscaler:events
+
+# View events for a specific worker
+WORKER=critical_worker bundle exec rake solid_queue_autoscaler:events
+
+# Cleanup old events (default: keep 30 days)
+bundle exec rake solid_queue_autoscaler:cleanup_events
+KEEP_DAYS=7 bundle exec rake solid_queue_autoscaler:cleanup_events
+```
+
 ## Troubleshooting
 
 ### "Could not acquire advisory lock"

data/lib/generators/solid_queue_heroku_autoscaler/dashboard_generator.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+require 'rails/generators'
+require 'rails/generators/active_record'
+
+module SolidQueueHerokuAutoscaler
+  module Generators
+    # Generator for the dashboard migrations.
+    # Creates the scale events table for tracking autoscaler history.
+    #
+    # @example Run the generator
+    #   rails generate solid_queue_heroku_autoscaler:dashboard
+    class DashboardGenerator < Rails::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path('templates', __dir__)
+
+      desc 'Creates migrations for SolidQueueHerokuAutoscaler dashboard (events table)'
+
+      def create_migration_file
+        migration_template 'create_solid_queue_autoscaler_events.rb.erb',
+                           'db/migrate/create_solid_queue_autoscaler_events.rb'
+      end
+
+      def show_post_install
+        say ''
+        say '=== Solid Queue Autoscaler Dashboard Setup ==='
+        say ''
+        say 'Next steps:'
+        say '  1. Run migrations: rails db:migrate'
+        say '  2. Mount the dashboard in config/routes.rb:'
+        say ''
+        say '     mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/autoscaler"'
+        say ''
+        say '  3. For authentication, wrap in a constraint:'
+        say ''
+        say '     authenticate :user, ->(u) { u.admin? } do'
+        say '       mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/autoscaler"'
+        say '     end'
+        say ''
+        say 'View the dashboard at: /autoscaler'
+        say ''
+      end
+
+      private
+
+      def migration_version
+        return unless defined?(ActiveRecord::VERSION)
+
+        "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
+      end
+    end
+  end
+end

data/lib/generators/solid_queue_heroku_autoscaler/templates/create_solid_queue_autoscaler_events.rb.erb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+class CreateSolidQueueAutoscalerEvents < ActiveRecord::Migration<%= migration_version %>
+  def change
+    create_table :solid_queue_autoscaler_events do |t|
+      t.string :worker_name, null: false
+      t.string :action, null: false
+      t.integer :from_workers, null: false, default: 0
+      t.integer :to_workers, null: false, default: 0
+      t.text :reason
+      t.integer :queue_depth, default: 0
+      t.float :latency_seconds, default: 0.0
+      t.jsonb :metrics_json
+      t.boolean :dry_run, default: false
+
+      t.datetime :created_at, null: false
+    end
+
+    add_index :solid_queue_autoscaler_events, :worker_name
+    add_index :solid_queue_autoscaler_events, :action
+    add_index :solid_queue_autoscaler_events, :created_at
+    add_index :solid_queue_autoscaler_events, %i[worker_name created_at]
+  end
+end

data/lib/generators/solid_queue_heroku_autoscaler/templates/initializer.rb

@@ -49,4 +49,10 @@ SolidQueueHerokuAutoscaler.configure do |config|
 
   # Optional: Custom logger
   # config.logger = Rails.logger
+
+  # Dashboard & Event Recording
+  # Record scale events to database for dashboard (requires dashboard migration)
+  config.record_events = true
+  # Also record no_change events (verbose, generates many records)
+  # config.record_all_events = false
 end

data/lib/solid_queue_heroku_autoscaler/configuration.rb

@@ -60,6 +60,9 @@ module SolidQueueHerokuAutoscaler
     attr_accessor :enabled, :logger
     attr_writer :lock_key
 
+    # Dashboard/event recording settings
+    attr_accessor :record_events, :record_all_events
+
     def initialize
       # Configuration name (auto-set when using named configurations)
       @name = :default
@@ -121,6 +124,10 @@ module SolidQueueHerokuAutoscaler
       @kubernetes_namespace = ENV['K8S_NAMESPACE'] || 'default'
       @kubernetes_context = ENV.fetch('K8S_CONTEXT', nil)
       @kubernetes_kubeconfig = ENV.fetch('KUBECONFIG', nil)
+
+      # Dashboard/event recording settings
+      @record_events = true # Record scale events to database
+      @record_all_events = false # Also record no_change events (verbose)
     end
 
     # Returns the lock key, auto-generating based on name if not explicitly set
@@ -186,6 +193,23 @@ module SolidQueueHerokuAutoscaler
       enabled
     end
 
+    def record_events?
+      record_events && connection_available?
+    end
+
+    def record_all_events?
+      record_all_events && record_events?
+    end
+
+    def connection_available?
+      return true if database_connection
+      return false unless defined?(ActiveRecord::Base)
+
+      ActiveRecord::Base.connected?
+    rescue StandardError
+      false
+    end
+
     # Returns the configured adapter instance.
     # Creates a new instance from adapter_class if not set.
     # Defaults to Heroku adapter.

data/lib/solid_queue_heroku_autoscaler/dashboard/engine.rb

@@ -0,0 +1,136 @@
+# frozen_string_literal: true
+
+require 'action_controller/railtie'
+require 'action_view/railtie'
+
+module SolidQueueHerokuAutoscaler
+  module Dashboard
+    # Rails engine that provides the autoscaler dashboard.
+    # Mount at /solid_queue_autoscaler or integrate with Mission Control.
+    #
+    # @example Mount in routes.rb
+    #   mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/solid_queue_autoscaler"
+    #
+    # @example With authentication
+    #   authenticate :user, ->(u) { u.admin? } do
+    #     mount SolidQueueHerokuAutoscaler::Dashboard::Engine => "/solid_queue_autoscaler"
+    #   end
+    class Engine < ::Rails::Engine
+      isolate_namespace SolidQueueHerokuAutoscaler::Dashboard
+
+      # Engine configuration
+      config.solid_queue_autoscaler_dashboard = ActiveSupport::OrderedOptions.new
+      config.solid_queue_autoscaler_dashboard.title = 'Solid Queue Autoscaler'
+
+      # Configure view paths
+      config.paths['app/views'] = File.expand_path('views', __dir__)
+
+      initializer 'solid_queue_autoscaler.dashboard.view_paths' do
+        ActiveSupport.on_load(:action_controller) do
+          append_view_path File.expand_path('views', __dir__)
+        end
+      end
+
+      initializer 'solid_queue_autoscaler.dashboard.integration' do
+        # Auto-integrate with Mission Control if available
+        ActiveSupport.on_load(:mission_control) do
+          # Register with Mission Control's tab system if available
+        end
+      end
+    end
+
+    # Application controller for dashboard
+    class ApplicationController < ActionController::Base
+      protect_from_forgery with: :exception
+
+      layout 'solid_queue_heroku_autoscaler/dashboard/application'
+
+      private
+
+      def autoscaler_status
+        @autoscaler_status ||= SolidQueueHerokuAutoscaler::Dashboard.status
+      end
+      helper_method :autoscaler_status
+
+      def events_available?
+        @events_available ||= SolidQueueHerokuAutoscaler::Dashboard.events_table_available?
+      end
+      helper_method :events_available?
+    end
+
+    # Main dashboard controller
+    class DashboardController < ApplicationController
+      def index
+        @status = autoscaler_status
+        @stats = SolidQueueHerokuAutoscaler::Dashboard.event_stats(since: 24.hours.ago)
+        @recent_events = SolidQueueHerokuAutoscaler::Dashboard.recent_events(limit: 10)
+      end
+    end
+
+    # Events controller
+    class EventsController < ApplicationController
+      def index
+        @worker_filter = params[:worker]
+        @events = SolidQueueHerokuAutoscaler::Dashboard.recent_events(
+          limit: params.fetch(:limit, 100).to_i,
+          worker_name: @worker_filter
+        )
+        @stats = SolidQueueHerokuAutoscaler::Dashboard.event_stats(
+          since: 24.hours.ago,
+          worker_name: @worker_filter
+        )
+      end
+    end
+
+    # Workers controller
+    class WorkersController < ApplicationController
+      def index
+        @workers = autoscaler_status
+      end
+
+      def show
+        worker_name = params[:id].to_sym
+        @worker = SolidQueueHerokuAutoscaler::Dashboard.worker_status(worker_name)
+        @events = SolidQueueHerokuAutoscaler::Dashboard.recent_events(
+          limit: 20,
+          worker_name: worker_name.to_s
+        )
+      end
+
+      def scale
+        worker_name = params[:id].to_sym
+        @result = SolidQueueHerokuAutoscaler.scale!(worker_name)
+        redirect_to worker_path(worker_name), notice: scale_notice(@result)
+      end
+
+      private
+
+      def scale_notice(result)
+        if result.success?
+          if result.scaled?
+            "Scaled from #{result.decision.from} to #{result.decision.to} workers"
+          elsif result.skipped?
+            "Skipped: #{result.skipped_reason}"
+          else
+            "No change needed: #{result.decision&.reason}"
+          end
+        else
+          "Error: #{result.error}"
+        end
+      end
+    end
+  end
+end
+
+# Define routes for the engine
+SolidQueueHerokuAutoscaler::Dashboard::Engine.routes.draw do
+  root to: 'dashboard#index'
+
+  resources :events, only: [:index]
+
+  resources :workers, only: %i[index show] do
+    member do
+      post :scale
+    end
+  end
+end

data/lib/solid_queue_heroku_autoscaler/dashboard/views/layouts/solid_queue_heroku_autoscaler/dashboard/application.html.erb

@@ -0,0 +1,206 @@
+<!DOCTYPE html>
+<html>
+<head>
+  <title>Solid Queue Autoscaler</title>
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <%= csrf_meta_tags %>
+  <style>
+    :root {
+      --bg-primary: #1a1a2e;
+      --bg-secondary: #16213e;
+      --bg-card: #0f3460;
+      --text-primary: #eee;
+      --text-secondary: #aaa;
+      --accent: #e94560;
+      --success: #00d26a;
+      --warning: #ffbe0b;
+      --info: #00b4d8;
+      --border: #333;
+    }
+    * { box-sizing: border-box; margin: 0; padding: 0; }
+    body {
+      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+      background: var(--bg-primary);
+      color: var(--text-primary);
+      line-height: 1.6;
+    }
+    .container { max-width: 1400px; margin: 0 auto; padding: 20px; }
+    header {
+      background: var(--bg-secondary);
+      padding: 15px 0;
+      border-bottom: 1px solid var(--border);
+      margin-bottom: 30px;
+    }
+    header .container {
+      display: flex;
+      justify-content: space-between;
+      align-items: center;
+    }
+    header h1 {
+      font-size: 1.5rem;
+      display: flex;
+      align-items: center;
+      gap: 10px;
+    }
+    header h1 span { color: var(--accent); }
+    nav { display: flex; gap: 20px; }
+    nav a {
+      color: var(--text-secondary);
+      text-decoration: none;
+      padding: 8px 16px;
+      border-radius: 6px;
+      transition: all 0.2s;
+    }
+    nav a:hover, nav a.active {
+      color: var(--text-primary);
+      background: var(--bg-card);
+    }
+    .flash {
+      padding: 12px 20px;
+      border-radius: 6px;
+      margin-bottom: 20px;
+    }
+    .flash.notice { background: rgba(0, 210, 106, 0.2); border: 1px solid var(--success); }
+    .flash.alert { background: rgba(233, 69, 96, 0.2); border: 1px solid var(--accent); }
+    .grid { display: grid; gap: 20px; }
+    .grid-2 { grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); }
+    .grid-3 { grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); }
+    .grid-4 { grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); }
+    .card {
+      background: var(--bg-card);
+      border-radius: 12px;
+      padding: 20px;
+      border: 1px solid var(--border);
+    }
+    .card h2, .card h3 {
+      margin-bottom: 15px;
+      display: flex;
+      align-items: center;
+      gap: 10px;
+    }
+    .stat-card {
+      text-align: center;
+      padding: 25px;
+    }
+    .stat-card .value {
+      font-size: 2.5rem;
+      font-weight: bold;
+      color: var(--accent);
+    }
+    .stat-card .label {
+      color: var(--text-secondary);
+      font-size: 0.9rem;
+      margin-top: 5px;
+    }
+    .stat-card.success .value { color: var(--success); }
+    .stat-card.warning .value { color: var(--warning); }
+    .stat-card.info .value { color: var(--info); }
+    table {
+      width: 100%;
+      border-collapse: collapse;
+    }
+    th, td {
+      padding: 12px;
+      text-align: left;
+      border-bottom: 1px solid var(--border);
+    }
+    th {
+      color: var(--text-secondary);
+      font-weight: 500;
+      font-size: 0.85rem;
+      text-transform: uppercase;
+    }
+    tr:hover { background: rgba(255,255,255,0.02); }
+    .badge {
+      display: inline-block;
+      padding: 4px 10px;
+      border-radius: 20px;
+      font-size: 0.8rem;
+      font-weight: 500;
+    }
+    .badge-success { background: rgba(0,210,106,0.2); color: var(--success); }
+    .badge-warning { background: rgba(255,190,11,0.2); color: var(--warning); }
+    .badge-danger { background: rgba(233,69,96,0.2); color: var(--accent); }
+    .badge-info { background: rgba(0,180,216,0.2); color: var(--info); }
+    .badge-neutral { background: rgba(170,170,170,0.2); color: var(--text-secondary); }
+    .btn {
+      display: inline-block;
+      padding: 10px 20px;
+      border-radius: 6px;
+      border: none;
+      cursor: pointer;
+      font-size: 0.9rem;
+      text-decoration: none;
+      transition: all 0.2s;
+    }
+    .btn-primary {
+      background: var(--accent);
+      color: white;
+    }
+    .btn-primary:hover { background: #d63654; }
+    .btn-secondary {
+      background: var(--bg-secondary);
+      color: var(--text-primary);
+      border: 1px solid var(--border);
+    }
+    .btn-secondary:hover { background: var(--bg-card); }
+    .btn-sm { padding: 6px 12px; font-size: 0.8rem; }
+    .progress-bar {
+      height: 8px;
+      background: var(--bg-secondary);
+      border-radius: 4px;
+      overflow: hidden;
+      margin-top: 8px;
+    }
+    .progress-bar .fill {
+      height: 100%;
+      border-radius: 4px;
+      transition: width 0.3s;
+    }
+    .text-muted { color: var(--text-secondary); }
+    .text-success { color: var(--success); }
+    .text-warning { color: var(--warning); }
+    .text-danger { color: var(--accent); }
+    .text-info { color: var(--info); }
+    .mb-1 { margin-bottom: 0.5rem; }
+    .mb-2 { margin-bottom: 1rem; }
+    .mb-3 { margin-bottom: 1.5rem; }
+    .mt-2 { margin-top: 1rem; }
+    .d-flex { display: flex; }
+    .justify-between { justify-content: space-between; }
+    .align-center { align-items: center; }
+    .gap-2 { gap: 1rem; }
+    .time-ago { font-size: 0.85rem; color: var(--text-secondary); }
+  </style>
+</head>
+<body>
+  <header>
+    <div class="container">
+      <h1>
+        <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
+          <path d="M12 2L2 7l10 5 10-5-10-5z"/>
+          <path d="M2 17l10 5 10-5"/>
+          <path d="M2 12l10 5 10-5"/>
+        </svg>
+        <span>Solid Queue</span> Autoscaler
+      </h1>
+      <nav>
+        <%= link_to 'Dashboard', main_app.respond_to?(:solid_queue_autoscaler_path) ? main_app.solid_queue_autoscaler_path : root_path, class: request.path == root_path ? 'active' : '' %>
+        <%= link_to 'Workers', workers_path, class: request.path.include?('/workers') ? 'active' : '' %>
+        <%= link_to 'Events', events_path, class: request.path.include?('/events') ? 'active' : '' %>
+      </nav>
+    </div>
+  </header>
+
+  <main class="container">
+    <% if notice %>
+      <div class="flash notice"><%= notice %></div>
+    <% end %>
+    <% if alert %>
+      <div class="flash alert"><%= alert %></div>
+    <% end %>
+
+    <%= yield %>
+  </main>
+</body>
+</html>