ruby-pg-extras 1.4.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 943bbbbd7e2de25570164a5bbd5e6015708ca2f808e7f0b25523a4125ad1bbf0
-  data.tar.gz: dd4e28f9900f2b50eae39232828c47ebb7c6b57dbb9ca4042be074cda1214e75
+  metadata.gz: 517f1d39dfe5d78d1e8f7650767664b9bc349bf062efb60b421e568afe65be82
+  data.tar.gz: b60152d78952c4dc8bcee791d445d1aa0028d60bb0af47736095c5b587f605bd
 SHA512:
-  metadata.gz: 12dd59913848f989fe6c186fefcdeb6418ab41cd6082b65f072b48d7d961e5d43b5c41822f8527bfda64e4557d284d52e74c0c3bd6dc1ca722fc958fa77d6b2a
-  data.tar.gz: 6c2d5445c0acb5eb93f614f778464c1059807112ca310efac9f135ca025985783ee553b7dca99e269d932a7d447c736011b54e791897e7e8671832492ee87add
+  metadata.gz: 8d827609746ab9a203a80c40945ca4b394d9014443d3fb89ae938734d996e3dc3e7f9bce584a7d98297a490365b76e8209e3d43501f932c9c7d39078891f77d4
+  data.tar.gz: 95dd7b3bb562e194ddac894b9f1e66d05d37d933561333a127ae9dca649c0e2b2259a46b518117b06bcce72729574250b46be935e3f2245eaf9c812ed48346ad

data/README.md

@@ -109,6 +109,8 @@ RubyPGExtras.cache_hit
 
 This command provides information on the efficiency of the buffer cache, for both index reads (`index hit rate`) as well as table reads (`table hit rate`). A low buffer cache hit ratio can be a sign that the Postgres instance is too small for the workload.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#cache-hit)
+
 ### `index_cache_hit`
 
 ```ruby
@@ -125,6 +127,8 @@ RubyPGExtras.index_cache_hit
 
 The same as `cache_hit` with each table's indexes cache hit info displayed separately.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#cache-hit)
+
 ### `table_cache_hit`
 
 ```ruby
@@ -141,6 +145,28 @@ RubyPGExtras.table_cache_hit
 
 The same as `cache_hit` with each table's cache hit info displayed seperately.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#cache-hit)
+
+### `db_settings`
+
+```ruby
+
+RubyPGExtras.db_settings
+
+             name             | setting | unit |
+------------------------------+---------+------+
+ checkpoint_completion_target | 0.7     |      |
+ default_statistics_target    | 100     |      |
+ effective_cache_size         | 1350000 | 8kB  |
+ effective_io_concurrency     | 1       |      |
+(truncated results for brevity)
+
+```
+
+This method displays values for selected PostgreSQL settings. You can compare them with settings recommended by [PGTune](https://pgtune.leopard.in.ua/#/) and tweak values to improve performance.
+
+[More info](https://pawelurbanek.com/postgresql-fix-performance#cache-hit)
+
 ### `index_usage`
 
 ```ruby
@@ -178,6 +204,8 @@ RubyPGExtras.locks
 
 This command displays queries that have taken out an exclusive lock on a relation. Exclusive locks typically prevent other operations on that relation from taking place, and can be a cause of "hung" queries that are waiting for a lock to be granted.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#deadlocks)
+
 ### `all_locks`
 
 ```ruby
@@ -210,6 +238,8 @@ This command displays statements, obtained from `pg_stat_statements`, ordered by
 
 Typically, an efficient query will have an appropriate ratio of calls to total execution time, with as little time spent on I/O as possible. Queries that have a high total execution time but low call count should be investigated to improve their performance. Queries that have a high proportion of execution time being spent on synchronous I/O should also be investigated.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#missing-indexes)
+
 ### `calls`
 
 ```ruby
@@ -230,6 +260,8 @@ RubyPGExtras.calls(args: { limit: 10 })
 
 This command is much like `pg:outliers`, but ordered by the number of times a statement has been called.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#missing-indexes)
+
 ### `blocking`
 
 ```ruby
@@ -244,6 +276,8 @@ RubyPGExtras.blocking
 
 This command displays statements that are currently holding locks that other statements are waiting to be released. This can be used in conjunction with `pg:locks` to determine which statements need to be terminated in order to resolve lock contention.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#deadlocks)
+
 ### `total_index_size`
 
 ```ruby
@@ -351,6 +385,26 @@ RubyPGExtras.unused_indexes(args: { min_scans: 20 })
 
 This command displays indexes that have < 50 scans recorded against them, and are greater than 5 pages in size, ordered by size relative to the number of index scans. This command is generally useful for eliminating indexes that are unused, which can impact write performance, as well as read performance should they occupy space in memory.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#unused-indexes)
+
+### `null_indexes`
+
+```ruby
+
+RubyPGExtras.null_indexes(args: { min_relation_size_mb: 10 })
+
+   oid   |         index      | index_size | unique | indexed_column | null_frac | expected_saving
+---------+--------------------+------------+--------+----------------+-----------+-----------------
+  183764 | users_reset_token  | 1445 MB    | t      | reset_token    |   97.00%  | 1401 MB
+   88732 | plan_cancelled_at  | 539 MB     | f      | cancelled_at   |    8.30%  | 44 MB
+ 9827345 | users_email        | 18 MB      | t      | email          |   28.67%  | 5160 kB
+
+```
+
+This command displays indexes that contain `NULL` values. A high ratio of `NULL` values means that using a partial index excluding them will be beneficial in case they are not used for searching.
+
+[More info](https://pawelurbanek.com/postgresql-fix-performance#null-indexes)
+
 ### `seq_scans`
 
 ```ruby
@@ -373,11 +427,13 @@ RubyPGExtras.seq_scans
 
 This command displays the number of sequential scans recorded against all tables, descending by count of sequential scans. Tables that have very high numbers of sequential scans may be under-indexed, and it may be worth investigating queries that read from these tables.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#missing-indexes)
+
 ### `long_running_queries`
 
 ```ruby
 
-RubyPGExtras.long_running_queries
+RubyPGExtras.long_running_queries(args: { threshold: "200 milliseconds" })
 
 
   pid  |    duration     |                                      query
@@ -428,6 +484,8 @@ RubyPGExtras.bloat
 
 This command displays an estimation of table "bloat" – space allocated to a relation that is full of dead tuples, that has yet to be reclaimed. Tables that have a high bloat ratio, typically 10 or greater, should be investigated to see if vacuuming is aggressive enough, and can be a sign of high table churn.
 
+[More info](https://pawelurbanek.com/postgresql-fix-performance#bloat)
+
 ### `vacuum_stats`
 
 ```ruby
@@ -475,3 +533,19 @@ RubyPGExtras.mandelbrot
 ```
 
 This command outputs the Mandelbrot set, calculated through SQL.
+
+### `buffercache_stats`
+
+This command shows the relations buffered in database share buffer, ordered by percentage taken. It also shows that how much of the whole relation is buffered.
+
+```ruby
+RubyPGExtras.buffercache_stats(args: { limit: 10 })
+```
+
+### `buffercache_usage`
+
+This command calculates how many blocks from which table are currently cached.
+
+```ruby
+RubyPGExtras.buffercache_usage(args: { limit: 20 })
+```

data/lib/ruby-pg-extras.rb

@@ -8,20 +8,24 @@ module RubyPGExtras
   @@database_url = nil
 
   QUERIES = %i(
-    bloat blocking cache_hit
+    bloat blocking cache_hit db_settings
     calls extensions table_cache_hit index_cache_hit
-    index_size index_usage locks all_locks
+    index_size index_usage null_indexes locks all_locks
     long_running_queries mandelbrot outliers
     records_rank seq_scans table_indexes_size
     table_size total_index_size total_table_size
     unused_indexes vacuum_stats kill_all
+    buffercache_stats buffercache_usage
   )
 
   DEFAULT_ARGS = Hash.new({}).merge({
     calls: { limit: 10 },
     long_running_queries: { threshold: "500 milliseconds" },
     outliers: { limit: 10 },
-    unused_indexes: { min_scans: 50 }
+    buffercache_stats: { limit: 10 },
+    buffercache_usage: { limit: 20 },
+    unused_indexes: { min_scans: 50 },
+    null_indexes: { min_relation_size_mb: 10 }
   })
 
   QUERIES.each do |query_name|

data/lib/ruby-pg-extras/queries/buffercache_stats.sql

@@ -0,0 +1,13 @@
+/* Calculates percentages of relations buffered in database share buffer */
+
+SELECT
+  c.relname,
+  pg_size_pretty(count(*) * 8192) AS buffered,
+  round(100.0 * count(*) / (SELECT setting FROM pg_settings WHERE name = 'shared_buffers')::integer, 1) AS buffer_percent,
+  round(100.0 * count(*) * 8192 / pg_table_size(c.oid), 1) AS percent_of_relation
+FROM pg_class c
+INNER JOIN pg_buffercache b ON b.relfilenode = c.relfilenode
+INNER JOIN pg_database d ON (b.reldatabase = d.oid AND d.datname = current_database())
+GROUP BY c.oid,c.relname
+ORDER BY 3 DESC
+LIMIT %{limit};

data/lib/ruby-pg-extras/queries/buffercache_usage.sql

@@ -0,0 +1,9 @@
+/* Calculate how many blocks from which table are currently cached */
+
+SELECT c.relname, count(*) AS buffers
+FROM pg_class c
+INNER JOIN pg_buffercache b ON b.relfilenode = c.relfilenode
+INNER JOIN pg_database d ON (b.reldatabase = d.oid AND d.datname = current_database())
+GROUP BY c.relname
+ORDER BY 2 DESC
+LIMIT %{limit};

data/lib/ruby-pg-extras/queries/db_settings.sql

@@ -0,0 +1,9 @@
+/* Values of selected PostgreSQL settings */
+
+SELECT name, setting, unit, short_desc FROM pg_settings
+WHERE name IN (
+  'max_connections', 'shared_buffers', 'effective_cache_size',
+  'maintenance_work_mem', 'checkpoint_completion_target', 'wal_buffers',
+  'default_statistics_target', 'random_page_cost', 'effective_io_concurrency',
+  'work_mem', 'min_wal_size', 'max_wal_size'
+);

data/lib/ruby-pg-extras/queries/null_indexes.sql

@@ -0,0 +1,33 @@
+/* Find indexes with a high ratio of NULL values */
+
+SELECT
+    c.oid,
+    c.relname AS index,
+    pg_size_pretty(pg_relation_size(c.oid)) AS index_size,
+    i.indisunique AS unique,
+    a.attname AS indexed_column,
+    CASE s.null_frac
+        WHEN 0 THEN ''
+        ELSE to_char(s.null_frac * 100, '999.00%%')
+    END AS null_frac,
+    pg_size_pretty((pg_relation_size(c.oid) * s.null_frac)::bigint) AS expected_saving
+FROM
+    pg_class c
+    JOIN pg_index i ON i.indexrelid = c.oid
+    JOIN pg_attribute a ON a.attrelid = c.oid
+    JOIN pg_class c_table ON c_table.oid = i.indrelid
+    JOIN pg_indexes ixs ON c.relname = ixs.indexname
+    LEFT JOIN pg_stats s ON s.tablename = c_table.relname AND a.attname = s.attname
+WHERE
+    -- Primary key cannot be partial
+    NOT i.indisprimary
+    -- Exclude already partial indexes
+    AND i.indpred IS NULL
+    -- Exclude composite indexes
+    AND array_length(i.indkey, 1) = 1
+    -- Exclude indexes without null_frac ratio
+    AND coalesce(s.null_frac, 0) != 0
+    -- Larger than threshold
+    AND pg_relation_size(c.oid) > %{min_relation_size_mb} * 1024 ^ 2
+ORDER BY
+  pg_relation_size(c.oid) * s.null_frac DESC;

data/lib/ruby-pg-extras/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RubyPGExtras
-  VERSION = "1.4.0"
+  VERSION = "1.6.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ruby-pg-extras
 version: !ruby/object:Gem::Version
-  version: 1.4.0
+  version: 1.6.0
 platform: ruby
 authors:
 - pawurb
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-01-26 00:00:00.000000000 Z
+date: 2021-03-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: pg
@@ -86,8 +86,11 @@ files:
 - lib/ruby-pg-extras/queries/all_locks.sql
 - lib/ruby-pg-extras/queries/bloat.sql
 - lib/ruby-pg-extras/queries/blocking.sql
+- lib/ruby-pg-extras/queries/buffercache_stats.sql
+- lib/ruby-pg-extras/queries/buffercache_usage.sql
 - lib/ruby-pg-extras/queries/cache_hit.sql
 - lib/ruby-pg-extras/queries/calls.sql
+- lib/ruby-pg-extras/queries/db_settings.sql
 - lib/ruby-pg-extras/queries/extensions.sql
 - lib/ruby-pg-extras/queries/index_cache_hit.sql
 - lib/ruby-pg-extras/queries/index_size.sql
@@ -96,6 +99,7 @@ files:
 - lib/ruby-pg-extras/queries/locks.sql
 - lib/ruby-pg-extras/queries/long_running_queries.sql
 - lib/ruby-pg-extras/queries/mandelbrot.sql
+- lib/ruby-pg-extras/queries/null_indexes.sql
 - lib/ruby-pg-extras/queries/outliers.sql
 - lib/ruby-pg-extras/queries/records_rank.sql
 - lib/ruby-pg-extras/queries/seq_scans.sql