pghero 0.1.10 → 1.0.0

data/guides/Query-Stats.md

@@ -0,0 +1,60 @@
+# Query Stats
+
+The [pg_stat_statements module](http://www.postgresql.org/docs/9.3/static/pgstatstatements.html) is used for query stats.
+
+## Installation
+
+If you have trouble enabling query stats from the dashboard, try doing it manually.
+
+Add the following to your `postgresql.conf`:
+
+```conf
+shared_preload_libraries = 'pg_stat_statements'
+pg_stat_statements.track = all
+pg_stat_statements.max = 10000
+track_activity_query_size = 2048
+```
+
+Then restart PostgreSQL. As a superuser from the `psql` console, run:
+
+```psql
+CREATE extension pg_stat_statements;
+```
+
+#### Amazon RDS
+
+Change `shared_preload_libraries` to `pg_stat_statements` in your [Parameter Group](https://console.aws.amazon.com/rds/home?region=us-east-1#parameter-groups:) and restart the database instance.
+
+As a superuser from the `psql` console, run:
+
+```psql
+CREATE extension pg_stat_statements;
+```
+
+## Common Issues
+
+#### pg_stat_statements must be loaded via shared_preload_libraries
+
+Follow the instructions above.
+
+#### FATAL: could not access file "pg_stat_statements": No such file or directory
+
+Run `apt-get install postgresql-contrib-9.3` and follow the instructions above.
+
+#### The database user does not have permission to ...
+
+The database user is not a superuser.  You can manually enable stats from the `psql` console with:
+
+```psql
+CREATE extension pg_stat_statements;
+```
+
+and reset stats with:
+
+```psql
+SELECT pg_stat_statements_reset();
+```
+
+#### Queries show up as `<insufficient privilege>`
+
+For security reasons, only superusers can see queries executed by other users.

data/guides/Rails.md

@@ -0,0 +1,187 @@
+# PgHero for Rails
+
+:gem:
+
+## Installation
+
+Add this line to your application’s Gemfile:
+
+```ruby
+gem 'pghero'
+```
+
+And mount the dashboard in your `config/routes.rb`:
+
+```ruby
+mount PgHero::Engine, at: "pghero"
+```
+
+Be sure to [secure the dashboard](#security) in production.
+
+## Insights
+
+```ruby
+PgHero.running_queries
+PgHero.long_running_queries
+PgHero.index_usage
+PgHero.missing_indexes
+PgHero.unused_indexes
+PgHero.unused_tables
+PgHero.database_size
+PgHero.relation_sizes
+PgHero.index_hit_rate
+PgHero.table_hit_rate
+PgHero.total_connections
+```
+
+Kill queries
+
+```ruby
+PgHero.kill(pid)
+PgHero.kill_long_running_queries
+PgHero.kill_all
+```
+
+Query stats
+
+```ruby
+PgHero.query_stats_enabled?
+PgHero.enable_query_stats
+PgHero.disable_query_stats
+PgHero.reset_query_stats
+PgHero.query_stats
+PgHero.slow_queries
+```
+
+Security
+
+```ruby
+PgHero.ssl_used?
+```
+
+Replication
+
+```ruby
+PgHero.replica?
+PgHero.replication_lag
+```
+
+## Users
+
+Create a user
+
+```ruby
+PgHero.create_user("link")
+# {password: "zbTrNHk2tvMgNabFgCo0ws7T"}
+```
+
+This generates and returns a secure password.  The user has full access to the `public` schema.
+
+Read-only access
+
+```ruby
+PgHero.create_user("epona", readonly: true)
+```
+
+Set the password
+
+```ruby
+PgHero.create_user("zelda", password: "hyrule")
+```
+
+Drop a user
+
+```ruby
+PgHero.drop_user("ganondorf")
+```
+
+## Security
+
+#### Basic Authentication
+
+Set the following variables in your environment or an initializer.
+
+```ruby
+ENV["PGHERO_USERNAME"] = "link"
+ENV["PGHERO_PASSWORD"] = "hyrule"
+```
+
+#### Devise
+
+```ruby
+authenticate :user, lambda { |user| user.admin? } do
+  mount PgHero::Engine, at: "pghero"
+end
+```
+
+## Query Stats
+
+Query stats can be enabled from the dashboard. If you run into issues, [view the guide](Query-Stats.md).
+
+## System Stats
+
+CPU usage is available for Amazon RDS.  Add these lines to your application’s Gemfile:
+
+```ruby
+gem 'aws-sdk'
+gem 'chartkick'
+```
+
+And add these variables to your environment:
+
+```sh
+PGHERO_ACCESS_KEY_ID=accesskey123
+PGHERO_SECRET_ACCESS_KEY=secret123
+PGHERO_DB_INSTANCE_IDENTIFIER=epona
+```
+
+## Multiple Databases
+
+Create `config/pghero.yml` with:
+
+```yml
+default: &default
+  databases:
+    primary:
+      url: <%= ENV["PGHERO_DATABASE_URL"] %>
+    replica:
+      url: <%= ENV["REPLICA_DATABASE_URL"] %>
+
+development:
+  <<: *default
+
+production:
+  <<: *default
+```
+
+## Customize
+
+Minimum time for long running queries
+
+```ruby
+PgHero.long_running_query_sec = 60 # default
+```
+
+Minimum average time for slow queries
+
+```ruby
+PgHero.slow_query_ms = 20 # default
+```
+
+Minimum calls for slow queries
+
+```ruby
+PgHero.slow_query_calls = 100 # default
+```
+
+Minimum connections for high connections warning
+
+```ruby
+PgHero.total_connections_threshold = 100 # default
+```
+
+## Bonus
+
+- See where queries come from with [Marginalia](https://github.com/basecamp/marginalia) - comments appear on the Live Queries tab.
+- Get weekly news and articles with [Postgres Weekly](http://postgresweekly.com)
+- Optimize your configuration with [PgTune](http://pgtune.leopard.in.ua) and [pgBench](http://www.postgresql.org/docs/devel/static/pgbench.html)

data/lib/pghero.rb

@@ -5,18 +5,73 @@ require "pghero/engine" if defined?(Rails)
 module PgHero
   # hack for connection
   class Connection < ActiveRecord::Base
-    establish_connection ENV["PGHERO_DATABASE_URL"] if ENV["PGHERO_DATABASE_URL"]
+    self.abstract_class = true
   end
 
   class << self
-    attr_accessor :long_running_query_sec, :slow_query_ms, :slow_query_calls, :total_connections_threshold
+    attr_accessor :long_running_query_sec, :slow_query_ms, :slow_query_calls, :total_connections_threshold, :env
   end
-  self.long_running_query_sec = 60
-  self.slow_query_ms = 20
-  self.slow_query_calls = 100
-  self.total_connections_threshold = 100
+  self.long_running_query_sec = (ENV["PGHERO_LONG_RUNNING_QUERY_SEC"] || 60).to_i
+  self.slow_query_ms = (ENV["PGHERO_SLOW_QUERY_MS"] || 20).to_i
+  self.slow_query_calls = (ENV["PGHERO_SLOW_QUERY_CALLS"] || 100).to_i
+  self.total_connections_threshold = (ENV["PGHERO_TOTAL_CONNECTIONS_THRESHOLD"] || 100).to_i
+  self.env = ENV["RAILS_ENV"] || ENV["RACK_ENV"] || "development"
 
   class << self
+    def config
+      Thread.current[:pghero_config] ||= begin
+        path = "config/pghero.yml"
+
+        config =
+          if File.exist?(path)
+            YAML.load(ERB.new(File.read(path)).result)[env]
+          end
+
+        if config
+          config
+        else
+          {
+            "databases" => {
+              "primary" => {
+                "url" => ENV["PGHERO_DATABASE_URL"], # nil reverts to default config
+                "db_instance_identifier" => ENV["PGHERO_DB_INSTANCE_IDENTIFIER"]
+              }
+            }
+          }
+        end
+      end
+    end
+
+    def primary_database
+      config["databases"].keys.first
+    end
+
+    def current_database
+      Thread.current[:pghero_current_database] ||= primary_database
+    end
+
+    def current_database=(database)
+      raise "Database not found" unless database_config(database)
+      Thread.current[:pghero_current_database] = database.to_s
+      Thread.current[:pghero_connection] = nil
+      database
+    end
+
+    def database_config(database)
+      config["databases"][database.to_s]
+    end
+
+    def current_config
+      database_config(current_database)
+    end
+
+    def with(database)
+      self.current_database = database
+      yield
+    ensure
+      self.current_database = primary_database
+    end
+
     def running_queries
       select_all <<-SQL
         SELECT
@@ -225,6 +280,11 @@ module PgHero
       execute("SELECT pg_terminate_backend(#{pid.to_i})").first["pg_terminate_backend"] == "t"
     end
 
+    def kill_long_running_queries
+      long_running_queries.each { |query| kill(query["pid"]) }
+      true
+    end
+
     def kill_all
       select_all <<-SQL
         SELECT
@@ -340,6 +400,16 @@ module PgHero
       end
     end
 
+    def ssl_used?
+      ssl_used = nil
+      Connection.transaction do
+        execute("CREATE EXTENSION IF NOT EXISTS sslinfo")
+        ssl_used = select_all("SELECT ssl_is_used()").first["ssl_is_used"] == "t"
+        raise ActiveRecord::Rollback
+      end
+      ssl_used
+    end
+
     def cpu_usage
       rds_stats("CPUUtilization")
     end
@@ -348,6 +418,10 @@ module PgHero
       rds_stats("DatabaseConnections")
     end
 
+    def replication_lag_stats
+      rds_stats("ReplicaLag")
+    end
+
     def rds_stats(metric_name)
       if system_stats_enabled?
         cw = AWS::CloudWatch.new(access_key_id: access_key_id, secret_access_key: secret_access_key)
@@ -387,7 +461,7 @@ module PgHero
 
       commands =
         [
-          "CREATE ROLE #{user} LOGIN PASSWORD #{Connection.connection.quote(password)}",
+          "CREATE ROLE #{user} LOGIN PASSWORD #{connection.quote(password)}",
           "GRANT CONNECT ON DATABASE #{database} TO #{user}",
           "GRANT USAGE ON SCHEMA #{schema} TO #{user}"
         ]
@@ -448,7 +522,7 @@ module PgHero
     end
 
     def db_instance_identifier
-      ENV["PGHERO_DB_INSTANCE_IDENTIFIER"]
+      current_config["db_instance_identifier"]
     end
 
     def explain(sql)
@@ -485,6 +559,15 @@ module PgHero
       Hash[names.map { |name| [name, values[name]] }]
     end
 
+    def replica?
+      select_all("SELECT setting FROM pg_settings WHERE name = 'hot_standby'").first["setting"] == "on"
+    end
+
+    # http://www.niwi.be/2013/02/16/replication-status-in-postgresql/
+    def replication_lag
+      select_all("SELECT EXTRACT(EPOCH FROM NOW() - pg_last_xact_replay_timestamp()) AS replication_lag").first["replication_lag"].to_f
+    end
+
     def friendly_value(setting, unit)
       if %w(kB 8kB).include?(unit)
         value = setting.to_i
@@ -512,7 +595,10 @@ module PgHero
     end
 
     def connection
-      @connection ||= Connection.connection
+      Thread.current[:pghero_connection] ||= begin
+        Connection.establish_connection(current_config["url"])
+        Connection.connection
+      end
     end
 
     # from ActiveSupport

data/lib/pghero/version.rb

@@ -1,3 +1,3 @@
 module PgHero
-  VERSION = "0.1.10"
+  VERSION = "1.0.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pghero
 version: !ruby/object:Gem::Version
-  version: 0.1.10
+  version: 1.0.0
 platform: ruby
 authors:
 - Andrew Kane
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-03-20 00:00:00.000000000 Z
+date: 2015-04-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -96,21 +96,26 @@ files:
 - app/controllers/pg_hero/home_controller.rb
 - app/views/layouts/pg_hero/application.html.erb
 - app/views/pg_hero/home/_connections_table.html.erb
+- app/views/pg_hero/home/_live_queries_table.html.erb
 - app/views/pg_hero/home/_queries_table.html.erb
-- app/views/pg_hero/home/_query_stats_table.html.erb
 - app/views/pg_hero/home/connections.html.erb
 - app/views/pg_hero/home/explain.html.erb
 - app/views/pg_hero/home/index.html.erb
-- app/views/pg_hero/home/indexes.html.erb
+- app/views/pg_hero/home/index_usage.html.erb
+- app/views/pg_hero/home/live_queries.html.erb
 - app/views/pg_hero/home/queries.html.erb
-- app/views/pg_hero/home/query_stats.html.erb
 - app/views/pg_hero/home/space.html.erb
-- app/views/pg_hero/home/system_stats.html.erb
+- app/views/pg_hero/home/system.html.erb
 - app/views/pg_hero/home/tune.html.erb
 - config/routes.rb
 - gemfiles/activerecord31.gemfile
 - gemfiles/activerecord32.gemfile
 - gemfiles/activerecord40.gemfile
+- guides/Docker.md
+- guides/Heroku.md
+- guides/Linux.md
+- guides/Query-Stats.md
+- guides/Rails.md
 - lib/pghero.rb
 - lib/pghero/engine.rb
 - lib/pghero/version.rb