pghero 2.1.1 → 2.2.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of pghero might be problematic. Click here for more details.

Files changed (41) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +8 -0
  3. data/CONTRIBUTING.md +9 -7
  4. data/README.md +6 -4
  5. data/app/controllers/pg_hero/home_controller.rb +10 -2
  6. data/app/views/pg_hero/home/explain.html.erb +1 -1
  7. data/app/views/pg_hero/home/index.html.erb +16 -6
  8. data/lib/generators/pghero/templates/{config.yml → config.yml.tt} +3 -3
  9. data/lib/generators/pghero/templates/{query_stats.rb → query_stats.rb.tt} +0 -0
  10. data/lib/generators/pghero/templates/{space_stats.rb → space_stats.rb.tt} +0 -0
  11. data/lib/pghero.rb +2 -2
  12. data/lib/pghero/methods/connections.rb +16 -0
  13. data/lib/pghero/methods/indexes.rb +32 -26
  14. data/lib/pghero/methods/maintenance.rb +1 -1
  15. data/lib/pghero/methods/query_stats.rb +2 -2
  16. data/lib/pghero/methods/replication.rb +1 -1
  17. data/lib/pghero/methods/sequences.rb +6 -2
  18. data/lib/pghero/methods/suggested_indexes.rb +1 -5
  19. data/lib/pghero/version.rb +1 -1
  20. metadata +9 -45
  21. data/.gitattributes +0 -1
  22. data/.github/ISSUE_TEMPLATE.md +0 -7
  23. data/.gitignore +0 -22
  24. data/.travis.yml +0 -16
  25. data/Gemfile +0 -6
  26. data/Rakefile +0 -8
  27. data/guides/Contributing.md +0 -16
  28. data/guides/Docker.md +0 -89
  29. data/guides/Heroku.md +0 -102
  30. data/guides/Linux.md +0 -296
  31. data/guides/Permissions.md +0 -57
  32. data/guides/Query-Stats.md +0 -60
  33. data/guides/Rails.md +0 -339
  34. data/guides/Suggested-Indexes.md +0 -19
  35. data/pghero.gemspec +0 -35
  36. data/test/basic_test.rb +0 -38
  37. data/test/best_index_test.rb +0 -180
  38. data/test/gemfiles/activerecord41.gemfile +0 -6
  39. data/test/gemfiles/activerecord42.gemfile +0 -6
  40. data/test/suggested_indexes_test.rb +0 -18
  41. data/test/test_helper.rb +0 -66
@@ -1,57 +0,0 @@
1
- # Permissions
2
-
3
- For security, Postgres doesn’t allow you to see queries from other users without being a superuser. However, you likely don’t want to run PgHero as a superuser. You can use `SECURITY DEFINER` to give non-superusers access to superuser functions.
4
-
5
- With a superuser, run:
6
-
7
- ```sql
8
- CREATE SCHEMA pghero;
9
-
10
- -- view queries
11
- CREATE OR REPLACE FUNCTION pghero.pg_stat_activity() RETURNS SETOF pg_stat_activity AS
12
- $$
13
- SELECT * FROM pg_catalog.pg_stat_activity;
14
- $$ LANGUAGE sql VOLATILE SECURITY DEFINER;
15
-
16
- CREATE VIEW pghero.pg_stat_activity AS SELECT * FROM pghero.pg_stat_activity();
17
-
18
- -- kill queries
19
- CREATE OR REPLACE FUNCTION pghero.pg_terminate_backend(pid int) RETURNS boolean AS
20
- $$
21
- SELECT * FROM pg_catalog.pg_terminate_backend(pid);
22
- $$ LANGUAGE sql VOLATILE SECURITY DEFINER;
23
-
24
- -- query stats
25
- CREATE OR REPLACE FUNCTION pghero.pg_stat_statements() RETURNS SETOF pg_stat_statements AS
26
- $$
27
- SELECT * FROM public.pg_stat_statements;
28
- $$ LANGUAGE sql VOLATILE SECURITY DEFINER;
29
-
30
- CREATE VIEW pghero.pg_stat_statements AS SELECT * FROM pghero.pg_stat_statements();
31
-
32
- -- query stats reset
33
- CREATE OR REPLACE FUNCTION pghero.pg_stat_statements_reset() RETURNS void AS
34
- $$
35
- SELECT public.pg_stat_statements_reset();
36
- $$ LANGUAGE sql VOLATILE SECURITY DEFINER;
37
-
38
- -- suggested indexes
39
- CREATE OR REPLACE FUNCTION pghero.pg_stats() RETURNS
40
- TABLE(schemaname name, tablename name, attname name, null_frac real, avg_width integer, n_distinct real) AS
41
- $$
42
- SELECT schemaname, tablename, attname, null_frac, avg_width, n_distinct FROM pg_catalog.pg_stats;
43
- $$ LANGUAGE sql VOLATILE SECURITY DEFINER;
44
-
45
- CREATE VIEW pghero.pg_stats AS SELECT * FROM pghero.pg_stats();
46
-
47
- -- create user
48
- CREATE ROLE pghero WITH LOGIN ENCRYPTED PASSWORD 'secret';
49
- GRANT CONNECT ON DATABASE <dbname> TO pghero;
50
- ALTER ROLE pghero SET search_path = pghero, pg_catalog, public;
51
- GRANT USAGE ON SCHEMA pghero TO pghero;
52
- GRANT SELECT ON ALL TABLES IN SCHEMA pghero TO pghero;
53
- ```
54
-
55
- ## Thanks
56
-
57
- A big thanks to [pganalyze](https://github.com/pganalyze/collector#setting-up-a-restricted-monitoring-user) for coming up with this approach for their collector.
@@ -1,60 +0,0 @@
1
- # Query Stats
2
-
3
- The [pg_stat_statements module](https://www.postgresql.org/docs/current/static/pgstatstatements.html) is used for query stats.
4
-
5
- ## Installation
6
-
7
- If you have trouble enabling query stats from the dashboard, try doing it manually.
8
-
9
- Add the following to your `postgresql.conf`:
10
-
11
- ```conf
12
- shared_preload_libraries = 'pg_stat_statements'
13
- pg_stat_statements.track = all
14
- pg_stat_statements.max = 10000
15
- track_activity_query_size = 2048
16
- ```
17
-
18
- Then restart PostgreSQL. As a superuser from the `psql` console, run:
19
-
20
- ```psql
21
- CREATE extension pg_stat_statements;
22
- ```
23
-
24
- #### Amazon RDS
25
-
26
- Change `shared_preload_libraries` to `pg_stat_statements` in your [Parameter Group](https://console.aws.amazon.com/rds/home?region=us-east-1#parameter-groups:) and restart the database instance.
27
-
28
- As a superuser from the `psql` console, run:
29
-
30
- ```psql
31
- CREATE extension pg_stat_statements;
32
- ```
33
-
34
- ## Common Issues
35
-
36
- #### pg_stat_statements must be loaded via shared_preload_libraries
37
-
38
- Follow the instructions above.
39
-
40
- #### FATAL: could not access file "pg_stat_statements": No such file or directory
41
-
42
- Run `apt-get install postgresql-contrib-9.3` and follow the instructions above.
43
-
44
- #### The database user does not have permission to ...
45
-
46
- The database user is not a superuser. You can manually enable stats from the `psql` console with:
47
-
48
- ```psql
49
- CREATE extension pg_stat_statements;
50
- ```
51
-
52
- and reset stats with:
53
-
54
- ```psql
55
- SELECT pg_stat_statements_reset();
56
- ```
57
-
58
- #### Queries show up as `<insufficient privilege>`
59
-
60
- For security reasons, only superusers can see queries executed by other users.
@@ -1,339 +0,0 @@
1
- # PgHero for Rails
2
-
3
- Add this line to your application’s Gemfile:
4
-
5
- ```ruby
6
- gem 'pghero'
7
- ```
8
-
9
- And mount the dashboard in your `config/routes.rb`:
10
-
11
- ```ruby
12
- mount PgHero::Engine, at: "pghero"
13
- ```
14
-
15
- Be sure to [secure the dashboard](#security) in production.
16
-
17
- ### Suggested Indexes
18
-
19
- PgHero can suggest indexes to add. To enable, add to your Gemfile:
20
-
21
- ```ruby
22
- gem 'pg_query', '>= 0.9.0'
23
- ```
24
-
25
- and make sure [query stats](#query-stats) are enabled. Read about how it works [here](Suggested-Indexes.md).
26
-
27
- ## Security
28
-
29
- #### Basic Authentication
30
-
31
- Set the following variables in your environment or an initializer.
32
-
33
- ```ruby
34
- ENV["PGHERO_USERNAME"] = "link"
35
- ENV["PGHERO_PASSWORD"] = "hyrule"
36
- ```
37
-
38
- #### Devise
39
-
40
- ```ruby
41
- authenticate :user, -> (user) { user.admin? } do
42
- mount PgHero::Engine, at: "pghero"
43
- end
44
- ```
45
-
46
- ## Query Stats
47
-
48
- Query stats can be enabled from the dashboard. If you run into issues, [view the guide](Query-Stats.md).
49
-
50
- ## Historical Query Stats
51
-
52
- To track query stats over time, run:
53
-
54
- ```sh
55
- rails generate pghero:query_stats
56
- rake db:migrate
57
- ```
58
-
59
- And schedule the task below to run every 5 minutes.
60
-
61
- ```sh
62
- rake pghero:capture_query_stats
63
- ```
64
-
65
- Or with a scheduler like Clockwork, use:
66
-
67
- ```ruby
68
- PgHero.capture_query_stats
69
- ```
70
-
71
- After this, a time range slider will appear on the Queries tab.
72
-
73
- By default, query stats are stored in your app’s database. Change this with:
74
-
75
- ```ruby
76
- ENV["PGHERO_STATS_DATABASE_URL"]
77
- ```
78
-
79
- ## Historical Space Stats
80
-
81
- To track space stats over time, run:
82
-
83
- ```sh
84
- rails generate pghero:space_stats
85
- rake db:migrate
86
- ```
87
-
88
- And schedule the task below to run once a day.
89
-
90
- ```sh
91
- rake pghero:capture_space_stats
92
- ```
93
-
94
- Or with a scheduler like Clockwork, use:
95
-
96
- ```ruby
97
- PgHero.capture_space_stats
98
- ```
99
-
100
- ## System Stats
101
-
102
- CPU usage, IOPS, and other stats are available for Amazon RDS. Add these lines to your application’s Gemfile:
103
-
104
- ```ruby
105
- gem 'aws-sdk-cloudwatch'
106
- # or
107
- gem 'aws-sdk'
108
- ```
109
-
110
- And add these variables to your environment:
111
-
112
- ```sh
113
- PGHERO_ACCESS_KEY_ID=accesskey123
114
- PGHERO_SECRET_ACCESS_KEY=secret123
115
- PGHERO_DB_INSTANCE_IDENTIFIER=epona
116
- ```
117
-
118
- This requires the following IAM policy:
119
-
120
- ```json
121
- {
122
- "Version": "2012-10-17",
123
- "Statement": [
124
- {
125
- "Effect": "Allow",
126
- "Action": "cloudwatch:GetMetricStatistics",
127
- "Resource": "*"
128
- }
129
- ]
130
- }
131
- ```
132
-
133
- ## Multiple Databases
134
-
135
- Create `config/pghero.yml` with:
136
-
137
- ```yml
138
- databases:
139
- primary:
140
- url: <%= ENV["PGHERO_DATABASE_URL"] %>
141
- replica:
142
- url: <%= ENV["REPLICA_DATABASE_URL"] %>
143
- ```
144
-
145
- ## Permissions
146
-
147
- We recommend [setting up a dedicated user](Permissions.md) for PgHero.
148
-
149
- ## Customize
150
-
151
- Minimum time for long running queries
152
-
153
- ```ruby
154
- PgHero.long_running_query_sec = 60 # default
155
- ```
156
-
157
- Minimum average time for slow queries
158
-
159
- ```ruby
160
- PgHero.slow_query_ms = 20 # default
161
- ```
162
-
163
- Minimum calls for slow queries
164
-
165
- ```ruby
166
- PgHero.slow_query_calls = 100 # default
167
- ```
168
-
169
- Minimum connections for high connections warning
170
-
171
- ```ruby
172
- PgHero.total_connections_threshold = 100 # default
173
- ```
174
-
175
- Statement timeout for explain
176
-
177
- ```ruby
178
- PgHero.explain_timeout_sec = 10 # default
179
- ```
180
-
181
- ## Methods
182
-
183
- Insights
184
-
185
- ```ruby
186
- PgHero.running_queries
187
- PgHero.long_running_queries
188
- PgHero.index_usage
189
- PgHero.invalid_indexes
190
- PgHero.missing_indexes
191
- PgHero.unused_indexes
192
- PgHero.unused_tables
193
- PgHero.database_size
194
- PgHero.relation_sizes
195
- PgHero.index_hit_rate
196
- PgHero.table_hit_rate
197
- PgHero.total_connections
198
- PgHero.locks
199
- ```
200
-
201
- Kill queries
202
-
203
- ```ruby
204
- PgHero.kill(pid)
205
- PgHero.kill_long_running_queries
206
- PgHero.kill_all
207
- ```
208
-
209
- Query stats
210
-
211
- ```ruby
212
- PgHero.query_stats_enabled?
213
- PgHero.enable_query_stats
214
- PgHero.disable_query_stats
215
- PgHero.reset_query_stats
216
- PgHero.query_stats
217
- PgHero.slow_queries
218
- ```
219
-
220
- Suggested indexes
221
-
222
- ```ruby
223
- PgHero.suggested_indexes
224
- PgHero.best_index(query)
225
- ```
226
-
227
- Security
228
-
229
- ```ruby
230
- PgHero.ssl_used?
231
- ```
232
-
233
- Replication
234
-
235
- ```ruby
236
- PgHero.replica?
237
- PgHero.replication_lag
238
- ```
239
-
240
- If you have multiple databases, specify a database with:
241
-
242
- ```ruby
243
- PgHero.databases["db2"].running_queries
244
- ```
245
-
246
- ## Users
247
-
248
- **Note:** It’s unsafe to pass user input to these commands.
249
-
250
- Create a user
251
-
252
- ```ruby
253
- PgHero.create_user("link")
254
- # {password: "zbTrNHk2tvMgNabFgCo0ws7T"}
255
- ```
256
-
257
- This generates and returns a secure password. The user has full access to the `public` schema.
258
-
259
- Read-only access
260
-
261
- ```ruby
262
- PgHero.create_user("epona", readonly: true)
263
- ```
264
-
265
- Set the password
266
-
267
- ```ruby
268
- PgHero.create_user("zelda", password: "hyrule")
269
- ```
270
-
271
- Grant access to only certain tables
272
-
273
- ```ruby
274
- PgHero.create_user("navi", tables: ["triforce"])
275
- ```
276
-
277
- Drop a user
278
-
279
- ```ruby
280
- PgHero.drop_user("ganondorf")
281
- ```
282
-
283
- ## Upgrading
284
-
285
- ### 2.0.0
286
-
287
- New features
288
-
289
- - Query details page
290
-
291
- Breaking changes
292
-
293
- - Methods now return symbols for keys instead of strings
294
- - Methods raise `PgHero::NotEnabled` error when a feature isn’t enabled
295
- - Requires pg_query 0.9.0+ for suggested indexes
296
- - Historical query stats require the `pghero_query_stats` table to have `query_hash` and `user` columns
297
- - Removed `with` option - use:
298
-
299
- ```ruby
300
- PgHero.databases[:database2].running_queries
301
- ```
302
-
303
- instead of
304
-
305
- ```ruby
306
- PgHero.with(:database2) { PgHero.running_queries }
307
- ```
308
-
309
- - Removed options from `connection_sources` method
310
- - Removed `locks` method
311
-
312
- ### 1.5.0
313
-
314
- For query stats grouping by user, create a migration with:
315
-
316
- ```ruby
317
- add_column :pghero_query_stats, :user, :text
318
- ```
319
-
320
- ### 1.3.0
321
-
322
- For better query stats grouping with Postgres 9.4+, create a migration with:
323
-
324
- ```ruby
325
- add_column :pghero_query_stats, :query_hash, :integer, limit: 8
326
- ```
327
-
328
- If you get an error with `queryid`, recreate the `pg_stat_statements` extension.
329
-
330
- ```sql
331
- DROP EXTENSION pg_stat_statements;
332
- CREATE EXTENSION pg_stat_statements;
333
- ```
334
-
335
- ## Bonus
336
-
337
- - See where queries come from with [Marginalia](https://github.com/basecamp/marginalia) - comments appear on the Live Queries tab.
338
- - Get weekly news and articles with [Postgres Weekly](https://postgresweekly.com/)
339
- - Optimize your configuration with [PgTune](https://pgtune.leopard.in.ua/) and [pgBench](https://www.postgresql.org/docs/devel/static/pgbench.html)