pghero 2.1.1 → 2.2.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of pghero might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/CONTRIBUTING.md +9 -7
- data/README.md +6 -4
- data/app/controllers/pg_hero/home_controller.rb +10 -2
- data/app/views/pg_hero/home/explain.html.erb +1 -1
- data/app/views/pg_hero/home/index.html.erb +16 -6
- data/lib/generators/pghero/templates/{config.yml → config.yml.tt} +3 -3
- data/lib/generators/pghero/templates/{query_stats.rb → query_stats.rb.tt} +0 -0
- data/lib/generators/pghero/templates/{space_stats.rb → space_stats.rb.tt} +0 -0
- data/lib/pghero.rb +2 -2
- data/lib/pghero/methods/connections.rb +16 -0
- data/lib/pghero/methods/indexes.rb +32 -26
- data/lib/pghero/methods/maintenance.rb +1 -1
- data/lib/pghero/methods/query_stats.rb +2 -2
- data/lib/pghero/methods/replication.rb +1 -1
- data/lib/pghero/methods/sequences.rb +6 -2
- data/lib/pghero/methods/suggested_indexes.rb +1 -5
- data/lib/pghero/version.rb +1 -1
- metadata +9 -45
- data/.gitattributes +0 -1
- data/.github/ISSUE_TEMPLATE.md +0 -7
- data/.gitignore +0 -22
- data/.travis.yml +0 -16
- data/Gemfile +0 -6
- data/Rakefile +0 -8
- data/guides/Contributing.md +0 -16
- data/guides/Docker.md +0 -89
- data/guides/Heroku.md +0 -102
- data/guides/Linux.md +0 -296
- data/guides/Permissions.md +0 -57
- data/guides/Query-Stats.md +0 -60
- data/guides/Rails.md +0 -339
- data/guides/Suggested-Indexes.md +0 -19
- data/pghero.gemspec +0 -35
- data/test/basic_test.rb +0 -38
- data/test/best_index_test.rb +0 -180
- data/test/gemfiles/activerecord41.gemfile +0 -6
- data/test/gemfiles/activerecord42.gemfile +0 -6
- data/test/suggested_indexes_test.rb +0 -18
- data/test/test_helper.rb +0 -66
data/guides/Permissions.md
DELETED
@@ -1,57 +0,0 @@
|
|
1
|
-
# Permissions
|
2
|
-
|
3
|
-
For security, Postgres doesn’t allow you to see queries from other users without being a superuser. However, you likely don’t want to run PgHero as a superuser. You can use `SECURITY DEFINER` to give non-superusers access to superuser functions.
|
4
|
-
|
5
|
-
With a superuser, run:
|
6
|
-
|
7
|
-
```sql
|
8
|
-
CREATE SCHEMA pghero;
|
9
|
-
|
10
|
-
-- view queries
|
11
|
-
CREATE OR REPLACE FUNCTION pghero.pg_stat_activity() RETURNS SETOF pg_stat_activity AS
|
12
|
-
$$
|
13
|
-
SELECT * FROM pg_catalog.pg_stat_activity;
|
14
|
-
$$ LANGUAGE sql VOLATILE SECURITY DEFINER;
|
15
|
-
|
16
|
-
CREATE VIEW pghero.pg_stat_activity AS SELECT * FROM pghero.pg_stat_activity();
|
17
|
-
|
18
|
-
-- kill queries
|
19
|
-
CREATE OR REPLACE FUNCTION pghero.pg_terminate_backend(pid int) RETURNS boolean AS
|
20
|
-
$$
|
21
|
-
SELECT * FROM pg_catalog.pg_terminate_backend(pid);
|
22
|
-
$$ LANGUAGE sql VOLATILE SECURITY DEFINER;
|
23
|
-
|
24
|
-
-- query stats
|
25
|
-
CREATE OR REPLACE FUNCTION pghero.pg_stat_statements() RETURNS SETOF pg_stat_statements AS
|
26
|
-
$$
|
27
|
-
SELECT * FROM public.pg_stat_statements;
|
28
|
-
$$ LANGUAGE sql VOLATILE SECURITY DEFINER;
|
29
|
-
|
30
|
-
CREATE VIEW pghero.pg_stat_statements AS SELECT * FROM pghero.pg_stat_statements();
|
31
|
-
|
32
|
-
-- query stats reset
|
33
|
-
CREATE OR REPLACE FUNCTION pghero.pg_stat_statements_reset() RETURNS void AS
|
34
|
-
$$
|
35
|
-
SELECT public.pg_stat_statements_reset();
|
36
|
-
$$ LANGUAGE sql VOLATILE SECURITY DEFINER;
|
37
|
-
|
38
|
-
-- suggested indexes
|
39
|
-
CREATE OR REPLACE FUNCTION pghero.pg_stats() RETURNS
|
40
|
-
TABLE(schemaname name, tablename name, attname name, null_frac real, avg_width integer, n_distinct real) AS
|
41
|
-
$$
|
42
|
-
SELECT schemaname, tablename, attname, null_frac, avg_width, n_distinct FROM pg_catalog.pg_stats;
|
43
|
-
$$ LANGUAGE sql VOLATILE SECURITY DEFINER;
|
44
|
-
|
45
|
-
CREATE VIEW pghero.pg_stats AS SELECT * FROM pghero.pg_stats();
|
46
|
-
|
47
|
-
-- create user
|
48
|
-
CREATE ROLE pghero WITH LOGIN ENCRYPTED PASSWORD 'secret';
|
49
|
-
GRANT CONNECT ON DATABASE <dbname> TO pghero;
|
50
|
-
ALTER ROLE pghero SET search_path = pghero, pg_catalog, public;
|
51
|
-
GRANT USAGE ON SCHEMA pghero TO pghero;
|
52
|
-
GRANT SELECT ON ALL TABLES IN SCHEMA pghero TO pghero;
|
53
|
-
```
|
54
|
-
|
55
|
-
## Thanks
|
56
|
-
|
57
|
-
A big thanks to [pganalyze](https://github.com/pganalyze/collector#setting-up-a-restricted-monitoring-user) for coming up with this approach for their collector.
|
data/guides/Query-Stats.md
DELETED
@@ -1,60 +0,0 @@
|
|
1
|
-
# Query Stats
|
2
|
-
|
3
|
-
The [pg_stat_statements module](https://www.postgresql.org/docs/current/static/pgstatstatements.html) is used for query stats.
|
4
|
-
|
5
|
-
## Installation
|
6
|
-
|
7
|
-
If you have trouble enabling query stats from the dashboard, try doing it manually.
|
8
|
-
|
9
|
-
Add the following to your `postgresql.conf`:
|
10
|
-
|
11
|
-
```conf
|
12
|
-
shared_preload_libraries = 'pg_stat_statements'
|
13
|
-
pg_stat_statements.track = all
|
14
|
-
pg_stat_statements.max = 10000
|
15
|
-
track_activity_query_size = 2048
|
16
|
-
```
|
17
|
-
|
18
|
-
Then restart PostgreSQL. As a superuser from the `psql` console, run:
|
19
|
-
|
20
|
-
```psql
|
21
|
-
CREATE extension pg_stat_statements;
|
22
|
-
```
|
23
|
-
|
24
|
-
#### Amazon RDS
|
25
|
-
|
26
|
-
Change `shared_preload_libraries` to `pg_stat_statements` in your [Parameter Group](https://console.aws.amazon.com/rds/home?region=us-east-1#parameter-groups:) and restart the database instance.
|
27
|
-
|
28
|
-
As a superuser from the `psql` console, run:
|
29
|
-
|
30
|
-
```psql
|
31
|
-
CREATE extension pg_stat_statements;
|
32
|
-
```
|
33
|
-
|
34
|
-
## Common Issues
|
35
|
-
|
36
|
-
#### pg_stat_statements must be loaded via shared_preload_libraries
|
37
|
-
|
38
|
-
Follow the instructions above.
|
39
|
-
|
40
|
-
#### FATAL: could not access file "pg_stat_statements": No such file or directory
|
41
|
-
|
42
|
-
Run `apt-get install postgresql-contrib-9.3` and follow the instructions above.
|
43
|
-
|
44
|
-
#### The database user does not have permission to ...
|
45
|
-
|
46
|
-
The database user is not a superuser. You can manually enable stats from the `psql` console with:
|
47
|
-
|
48
|
-
```psql
|
49
|
-
CREATE extension pg_stat_statements;
|
50
|
-
```
|
51
|
-
|
52
|
-
and reset stats with:
|
53
|
-
|
54
|
-
```psql
|
55
|
-
SELECT pg_stat_statements_reset();
|
56
|
-
```
|
57
|
-
|
58
|
-
#### Queries show up as `<insufficient privilege>`
|
59
|
-
|
60
|
-
For security reasons, only superusers can see queries executed by other users.
|
data/guides/Rails.md
DELETED
@@ -1,339 +0,0 @@
|
|
1
|
-
# PgHero for Rails
|
2
|
-
|
3
|
-
Add this line to your application’s Gemfile:
|
4
|
-
|
5
|
-
```ruby
|
6
|
-
gem 'pghero'
|
7
|
-
```
|
8
|
-
|
9
|
-
And mount the dashboard in your `config/routes.rb`:
|
10
|
-
|
11
|
-
```ruby
|
12
|
-
mount PgHero::Engine, at: "pghero"
|
13
|
-
```
|
14
|
-
|
15
|
-
Be sure to [secure the dashboard](#security) in production.
|
16
|
-
|
17
|
-
### Suggested Indexes
|
18
|
-
|
19
|
-
PgHero can suggest indexes to add. To enable, add to your Gemfile:
|
20
|
-
|
21
|
-
```ruby
|
22
|
-
gem 'pg_query', '>= 0.9.0'
|
23
|
-
```
|
24
|
-
|
25
|
-
and make sure [query stats](#query-stats) are enabled. Read about how it works [here](Suggested-Indexes.md).
|
26
|
-
|
27
|
-
## Security
|
28
|
-
|
29
|
-
#### Basic Authentication
|
30
|
-
|
31
|
-
Set the following variables in your environment or an initializer.
|
32
|
-
|
33
|
-
```ruby
|
34
|
-
ENV["PGHERO_USERNAME"] = "link"
|
35
|
-
ENV["PGHERO_PASSWORD"] = "hyrule"
|
36
|
-
```
|
37
|
-
|
38
|
-
#### Devise
|
39
|
-
|
40
|
-
```ruby
|
41
|
-
authenticate :user, -> (user) { user.admin? } do
|
42
|
-
mount PgHero::Engine, at: "pghero"
|
43
|
-
end
|
44
|
-
```
|
45
|
-
|
46
|
-
## Query Stats
|
47
|
-
|
48
|
-
Query stats can be enabled from the dashboard. If you run into issues, [view the guide](Query-Stats.md).
|
49
|
-
|
50
|
-
## Historical Query Stats
|
51
|
-
|
52
|
-
To track query stats over time, run:
|
53
|
-
|
54
|
-
```sh
|
55
|
-
rails generate pghero:query_stats
|
56
|
-
rake db:migrate
|
57
|
-
```
|
58
|
-
|
59
|
-
And schedule the task below to run every 5 minutes.
|
60
|
-
|
61
|
-
```sh
|
62
|
-
rake pghero:capture_query_stats
|
63
|
-
```
|
64
|
-
|
65
|
-
Or with a scheduler like Clockwork, use:
|
66
|
-
|
67
|
-
```ruby
|
68
|
-
PgHero.capture_query_stats
|
69
|
-
```
|
70
|
-
|
71
|
-
After this, a time range slider will appear on the Queries tab.
|
72
|
-
|
73
|
-
By default, query stats are stored in your app’s database. Change this with:
|
74
|
-
|
75
|
-
```ruby
|
76
|
-
ENV["PGHERO_STATS_DATABASE_URL"]
|
77
|
-
```
|
78
|
-
|
79
|
-
## Historical Space Stats
|
80
|
-
|
81
|
-
To track space stats over time, run:
|
82
|
-
|
83
|
-
```sh
|
84
|
-
rails generate pghero:space_stats
|
85
|
-
rake db:migrate
|
86
|
-
```
|
87
|
-
|
88
|
-
And schedule the task below to run once a day.
|
89
|
-
|
90
|
-
```sh
|
91
|
-
rake pghero:capture_space_stats
|
92
|
-
```
|
93
|
-
|
94
|
-
Or with a scheduler like Clockwork, use:
|
95
|
-
|
96
|
-
```ruby
|
97
|
-
PgHero.capture_space_stats
|
98
|
-
```
|
99
|
-
|
100
|
-
## System Stats
|
101
|
-
|
102
|
-
CPU usage, IOPS, and other stats are available for Amazon RDS. Add these lines to your application’s Gemfile:
|
103
|
-
|
104
|
-
```ruby
|
105
|
-
gem 'aws-sdk-cloudwatch'
|
106
|
-
# or
|
107
|
-
gem 'aws-sdk'
|
108
|
-
```
|
109
|
-
|
110
|
-
And add these variables to your environment:
|
111
|
-
|
112
|
-
```sh
|
113
|
-
PGHERO_ACCESS_KEY_ID=accesskey123
|
114
|
-
PGHERO_SECRET_ACCESS_KEY=secret123
|
115
|
-
PGHERO_DB_INSTANCE_IDENTIFIER=epona
|
116
|
-
```
|
117
|
-
|
118
|
-
This requires the following IAM policy:
|
119
|
-
|
120
|
-
```json
|
121
|
-
{
|
122
|
-
"Version": "2012-10-17",
|
123
|
-
"Statement": [
|
124
|
-
{
|
125
|
-
"Effect": "Allow",
|
126
|
-
"Action": "cloudwatch:GetMetricStatistics",
|
127
|
-
"Resource": "*"
|
128
|
-
}
|
129
|
-
]
|
130
|
-
}
|
131
|
-
```
|
132
|
-
|
133
|
-
## Multiple Databases
|
134
|
-
|
135
|
-
Create `config/pghero.yml` with:
|
136
|
-
|
137
|
-
```yml
|
138
|
-
databases:
|
139
|
-
primary:
|
140
|
-
url: <%= ENV["PGHERO_DATABASE_URL"] %>
|
141
|
-
replica:
|
142
|
-
url: <%= ENV["REPLICA_DATABASE_URL"] %>
|
143
|
-
```
|
144
|
-
|
145
|
-
## Permissions
|
146
|
-
|
147
|
-
We recommend [setting up a dedicated user](Permissions.md) for PgHero.
|
148
|
-
|
149
|
-
## Customize
|
150
|
-
|
151
|
-
Minimum time for long running queries
|
152
|
-
|
153
|
-
```ruby
|
154
|
-
PgHero.long_running_query_sec = 60 # default
|
155
|
-
```
|
156
|
-
|
157
|
-
Minimum average time for slow queries
|
158
|
-
|
159
|
-
```ruby
|
160
|
-
PgHero.slow_query_ms = 20 # default
|
161
|
-
```
|
162
|
-
|
163
|
-
Minimum calls for slow queries
|
164
|
-
|
165
|
-
```ruby
|
166
|
-
PgHero.slow_query_calls = 100 # default
|
167
|
-
```
|
168
|
-
|
169
|
-
Minimum connections for high connections warning
|
170
|
-
|
171
|
-
```ruby
|
172
|
-
PgHero.total_connections_threshold = 100 # default
|
173
|
-
```
|
174
|
-
|
175
|
-
Statement timeout for explain
|
176
|
-
|
177
|
-
```ruby
|
178
|
-
PgHero.explain_timeout_sec = 10 # default
|
179
|
-
```
|
180
|
-
|
181
|
-
## Methods
|
182
|
-
|
183
|
-
Insights
|
184
|
-
|
185
|
-
```ruby
|
186
|
-
PgHero.running_queries
|
187
|
-
PgHero.long_running_queries
|
188
|
-
PgHero.index_usage
|
189
|
-
PgHero.invalid_indexes
|
190
|
-
PgHero.missing_indexes
|
191
|
-
PgHero.unused_indexes
|
192
|
-
PgHero.unused_tables
|
193
|
-
PgHero.database_size
|
194
|
-
PgHero.relation_sizes
|
195
|
-
PgHero.index_hit_rate
|
196
|
-
PgHero.table_hit_rate
|
197
|
-
PgHero.total_connections
|
198
|
-
PgHero.locks
|
199
|
-
```
|
200
|
-
|
201
|
-
Kill queries
|
202
|
-
|
203
|
-
```ruby
|
204
|
-
PgHero.kill(pid)
|
205
|
-
PgHero.kill_long_running_queries
|
206
|
-
PgHero.kill_all
|
207
|
-
```
|
208
|
-
|
209
|
-
Query stats
|
210
|
-
|
211
|
-
```ruby
|
212
|
-
PgHero.query_stats_enabled?
|
213
|
-
PgHero.enable_query_stats
|
214
|
-
PgHero.disable_query_stats
|
215
|
-
PgHero.reset_query_stats
|
216
|
-
PgHero.query_stats
|
217
|
-
PgHero.slow_queries
|
218
|
-
```
|
219
|
-
|
220
|
-
Suggested indexes
|
221
|
-
|
222
|
-
```ruby
|
223
|
-
PgHero.suggested_indexes
|
224
|
-
PgHero.best_index(query)
|
225
|
-
```
|
226
|
-
|
227
|
-
Security
|
228
|
-
|
229
|
-
```ruby
|
230
|
-
PgHero.ssl_used?
|
231
|
-
```
|
232
|
-
|
233
|
-
Replication
|
234
|
-
|
235
|
-
```ruby
|
236
|
-
PgHero.replica?
|
237
|
-
PgHero.replication_lag
|
238
|
-
```
|
239
|
-
|
240
|
-
If you have multiple databases, specify a database with:
|
241
|
-
|
242
|
-
```ruby
|
243
|
-
PgHero.databases["db2"].running_queries
|
244
|
-
```
|
245
|
-
|
246
|
-
## Users
|
247
|
-
|
248
|
-
**Note:** It’s unsafe to pass user input to these commands.
|
249
|
-
|
250
|
-
Create a user
|
251
|
-
|
252
|
-
```ruby
|
253
|
-
PgHero.create_user("link")
|
254
|
-
# {password: "zbTrNHk2tvMgNabFgCo0ws7T"}
|
255
|
-
```
|
256
|
-
|
257
|
-
This generates and returns a secure password. The user has full access to the `public` schema.
|
258
|
-
|
259
|
-
Read-only access
|
260
|
-
|
261
|
-
```ruby
|
262
|
-
PgHero.create_user("epona", readonly: true)
|
263
|
-
```
|
264
|
-
|
265
|
-
Set the password
|
266
|
-
|
267
|
-
```ruby
|
268
|
-
PgHero.create_user("zelda", password: "hyrule")
|
269
|
-
```
|
270
|
-
|
271
|
-
Grant access to only certain tables
|
272
|
-
|
273
|
-
```ruby
|
274
|
-
PgHero.create_user("navi", tables: ["triforce"])
|
275
|
-
```
|
276
|
-
|
277
|
-
Drop a user
|
278
|
-
|
279
|
-
```ruby
|
280
|
-
PgHero.drop_user("ganondorf")
|
281
|
-
```
|
282
|
-
|
283
|
-
## Upgrading
|
284
|
-
|
285
|
-
### 2.0.0
|
286
|
-
|
287
|
-
New features
|
288
|
-
|
289
|
-
- Query details page
|
290
|
-
|
291
|
-
Breaking changes
|
292
|
-
|
293
|
-
- Methods now return symbols for keys instead of strings
|
294
|
-
- Methods raise `PgHero::NotEnabled` error when a feature isn’t enabled
|
295
|
-
- Requires pg_query 0.9.0+ for suggested indexes
|
296
|
-
- Historical query stats require the `pghero_query_stats` table to have `query_hash` and `user` columns
|
297
|
-
- Removed `with` option - use:
|
298
|
-
|
299
|
-
```ruby
|
300
|
-
PgHero.databases[:database2].running_queries
|
301
|
-
```
|
302
|
-
|
303
|
-
instead of
|
304
|
-
|
305
|
-
```ruby
|
306
|
-
PgHero.with(:database2) { PgHero.running_queries }
|
307
|
-
```
|
308
|
-
|
309
|
-
- Removed options from `connection_sources` method
|
310
|
-
- Removed `locks` method
|
311
|
-
|
312
|
-
### 1.5.0
|
313
|
-
|
314
|
-
For query stats grouping by user, create a migration with:
|
315
|
-
|
316
|
-
```ruby
|
317
|
-
add_column :pghero_query_stats, :user, :text
|
318
|
-
```
|
319
|
-
|
320
|
-
### 1.3.0
|
321
|
-
|
322
|
-
For better query stats grouping with Postgres 9.4+, create a migration with:
|
323
|
-
|
324
|
-
```ruby
|
325
|
-
add_column :pghero_query_stats, :query_hash, :integer, limit: 8
|
326
|
-
```
|
327
|
-
|
328
|
-
If you get an error with `queryid`, recreate the `pg_stat_statements` extension.
|
329
|
-
|
330
|
-
```sql
|
331
|
-
DROP EXTENSION pg_stat_statements;
|
332
|
-
CREATE EXTENSION pg_stat_statements;
|
333
|
-
```
|
334
|
-
|
335
|
-
## Bonus
|
336
|
-
|
337
|
-
- See where queries come from with [Marginalia](https://github.com/basecamp/marginalia) - comments appear on the Live Queries tab.
|
338
|
-
- Get weekly news and articles with [Postgres Weekly](https://postgresweekly.com/)
|
339
|
-
- Optimize your configuration with [PgTune](https://pgtune.leopard.in.ua/) and [pgBench](https://www.postgresql.org/docs/devel/static/pgbench.html)
|