pgslice 0.4.7 → 0.4.8
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/LICENSE.txt +1 -1
- data/README.md +36 -13
- data/lib/pgslice/helpers.rb +1 -1
- data/lib/pgslice/version.rb +1 -1
- metadata +8 -50
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4d3a279841af6ee62759c9570149cda73460c674c383aef78d7b99fa5c9840ec
|
4
|
+
data.tar.gz: 48462eb02783c097d0553071880e75650263d662cf755e8fa977b1d4e25ccc0e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9c95fe04d9c90f524c9cd778f5a5e05013ff4655982cd4ed1420a024c5c17a97d3de679c86e1d66800a9ee3ab7e29eec8abec8676041d7e07d7ebce96052315f
|
7
|
+
data.tar.gz: 5d43e4d5dc94f29538abd25bd2e3676dc47e1b5cdce646e345539d7543f4b21fb2d9d33ecb08c69df81283150643ad423817179719f712d963b0aba791c10d89
|
data/CHANGELOG.md
CHANGED
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -4,7 +4,7 @@ Postgres partitioning as easy as pie. Works great for both new and existing tabl
|
|
4
4
|
|
5
5
|
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
|
6
6
|
|
7
|
-
[![Build Status](https://
|
7
|
+
[![Build Status](https://github.com/ankane/pgslice/workflows/build/badge.svg?branch=master)](https://github.com/ankane/pgslice/actions)
|
8
8
|
|
9
9
|
## Install
|
10
10
|
|
@@ -14,7 +14,7 @@ pgslice is a command line tool. To install, run:
|
|
14
14
|
gem install pgslice
|
15
15
|
```
|
16
16
|
|
17
|
-
This will give you the `pgslice` command. If installation fails, you may need to install [dependencies](#dependencies).
|
17
|
+
This will give you the `pgslice` command. You can also install it with [Homebrew](#homebrew). If installation fails, you may need to install [dependencies](#dependencies).
|
18
18
|
|
19
19
|
## Steps
|
20
20
|
|
@@ -227,6 +227,15 @@ If you use [Amazon S3](https://aws.amazon.com/s3/) for backups, [s3cmd](https://
|
|
227
227
|
s3cmd put <table>_201809.dump s3://<s3-bucket>/<table>_201809.dump
|
228
228
|
```
|
229
229
|
|
230
|
+
## Schema Updates
|
231
|
+
|
232
|
+
Once a table is partitioned, make schema updates on the master table only (not partitions). This includes adding, removing, and modifying columns, as well as adding and removing indexes and foreign keys.
|
233
|
+
|
234
|
+
A few exceptions are:
|
235
|
+
|
236
|
+
- For Postgres 10, make index and foreign key updates on partitions only
|
237
|
+
- For Postgres < 10, make index and foreign key updates on the master table and all partitions
|
238
|
+
|
230
239
|
## Additional Commands
|
231
240
|
|
232
241
|
To undo prep (which will delete partitions), use:
|
@@ -241,6 +250,14 @@ To undo swap, use:
|
|
241
250
|
pgslice unswap <table>
|
242
251
|
```
|
243
252
|
|
253
|
+
## Additional Options
|
254
|
+
|
255
|
+
Set the tablespace when adding partitions
|
256
|
+
|
257
|
+
```sh
|
258
|
+
pgslice add_partitions <table> --tablespace fastspace
|
259
|
+
```
|
260
|
+
|
244
261
|
## App Considerations
|
245
262
|
|
246
263
|
This set up allows you to read and write with the original table name with no knowledge it’s partitioned. However, there are a few things to be aware of.
|
@@ -271,7 +288,9 @@ Before Postgres 10, if you use `INSERT` statements with a `RETURNING` clause (as
|
|
271
288
|
1. Insert directly into the partition
|
272
289
|
2. Get value before the insert with `SELECT nextval('sequence_name')` (for multiple rows, append `FROM generate_series(1, n)`)
|
273
290
|
|
274
|
-
##
|
291
|
+
## Frameworks
|
292
|
+
|
293
|
+
### Rails
|
275
294
|
|
276
295
|
For Postgres 10+, specify the primary key for partitioned models to ensure it’s returned.
|
277
296
|
|
@@ -291,6 +310,10 @@ class Visit < ApplicationRecord
|
|
291
310
|
end
|
292
311
|
```
|
293
312
|
|
313
|
+
### Other Frameworks
|
314
|
+
|
315
|
+
Please submit a PR if additional configuration is needed.
|
316
|
+
|
294
317
|
## One Off Tasks
|
295
318
|
|
296
319
|
You can also use pgslice to reduce the size of a table without partitioning by creating a new table, filling it with a subset of records, and swapping it in.
|
@@ -301,17 +324,9 @@ pgslice fill <table> --where "id > 1000" # use any conditions
|
|
301
324
|
pgslice swap <table>
|
302
325
|
```
|
303
326
|
|
304
|
-
##
|
305
|
-
|
306
|
-
Once a table is partitioned, here’s how to change the schema:
|
307
|
-
|
308
|
-
To add, remove, or modify a column, make the update on the master table only.
|
309
|
-
|
310
|
-
To add or remove an index or foreign key:
|
327
|
+
## Triggers
|
311
328
|
|
312
|
-
|
313
|
-
- For Postgres 10, make the update on partitions only.
|
314
|
-
- For Postgres < 10, make the update on the master table and all partitions.
|
329
|
+
Triggers aren’t copied from the original table. You can set up triggers on the intermediate table if needed. Note that Postgres doesn’t support `BEFORE / FOR EACH ROW` triggers on partitioned tables.
|
315
330
|
|
316
331
|
## Declarative Partitioning
|
317
332
|
|
@@ -321,6 +336,14 @@ Postgres 10 introduces [declarative partitioning](https://www.postgresql.org/doc
|
|
321
336
|
|
322
337
|
Always make sure your [connection is secure](https://ankane.org/postgres-sslmode-explained) when connecting to a database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.
|
323
338
|
|
339
|
+
## Homebrew
|
340
|
+
|
341
|
+
On Mac, you can use:
|
342
|
+
|
343
|
+
```sh
|
344
|
+
brew install ankane/brew/pgslice
|
345
|
+
```
|
346
|
+
|
324
347
|
## Dependencies
|
325
348
|
|
326
349
|
If installation fails, your system may be missing Ruby or libpq.
|
data/lib/pgslice/helpers.rb
CHANGED
@@ -33,7 +33,7 @@ module PgSlice
|
|
33
33
|
params = CGI.parse(uri.query.to_s)
|
34
34
|
# remove schema
|
35
35
|
@schema = Array(params.delete("schema") || "public")[0]
|
36
|
-
uri.query = URI.encode_www_form(params)
|
36
|
+
uri.query = params.any? ? URI.encode_www_form(params) : nil
|
37
37
|
|
38
38
|
ENV["PGCONNECT_TIMEOUT"] ||= "1"
|
39
39
|
conn = PG::Connection.new(uri.to_s)
|
data/lib/pgslice/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pgslice
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.4.
|
4
|
+
version: 0.4.8
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2022-02-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: thor
|
@@ -38,50 +38,8 @@ dependencies:
|
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: 0.18.2
|
41
|
-
|
42
|
-
|
43
|
-
requirement: !ruby/object:Gem::Requirement
|
44
|
-
requirements:
|
45
|
-
- - ">="
|
46
|
-
- !ruby/object:Gem::Version
|
47
|
-
version: '0'
|
48
|
-
type: :development
|
49
|
-
prerelease: false
|
50
|
-
version_requirements: !ruby/object:Gem::Requirement
|
51
|
-
requirements:
|
52
|
-
- - ">="
|
53
|
-
- !ruby/object:Gem::Version
|
54
|
-
version: '0'
|
55
|
-
- !ruby/object:Gem::Dependency
|
56
|
-
name: rake
|
57
|
-
requirement: !ruby/object:Gem::Requirement
|
58
|
-
requirements:
|
59
|
-
- - ">="
|
60
|
-
- !ruby/object:Gem::Version
|
61
|
-
version: '0'
|
62
|
-
type: :development
|
63
|
-
prerelease: false
|
64
|
-
version_requirements: !ruby/object:Gem::Requirement
|
65
|
-
requirements:
|
66
|
-
- - ">="
|
67
|
-
- !ruby/object:Gem::Version
|
68
|
-
version: '0'
|
69
|
-
- !ruby/object:Gem::Dependency
|
70
|
-
name: minitest
|
71
|
-
requirement: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - ">="
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '0'
|
76
|
-
type: :development
|
77
|
-
prerelease: false
|
78
|
-
version_requirements: !ruby/object:Gem::Requirement
|
79
|
-
requirements:
|
80
|
-
- - ">="
|
81
|
-
- !ruby/object:Gem::Version
|
82
|
-
version: '0'
|
83
|
-
description:
|
84
|
-
email: andrew@chartkick.com
|
41
|
+
description:
|
42
|
+
email: andrew@ankane.org
|
85
43
|
executables:
|
86
44
|
- pgslice
|
87
45
|
extensions: []
|
@@ -107,7 +65,7 @@ homepage: https://github.com/ankane/pgslice
|
|
107
65
|
licenses:
|
108
66
|
- MIT
|
109
67
|
metadata: {}
|
110
|
-
post_install_message:
|
68
|
+
post_install_message:
|
111
69
|
rdoc_options: []
|
112
70
|
require_paths:
|
113
71
|
- lib
|
@@ -122,8 +80,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
122
80
|
- !ruby/object:Gem::Version
|
123
81
|
version: '0'
|
124
82
|
requirements: []
|
125
|
-
rubygems_version: 3.
|
126
|
-
signing_key:
|
83
|
+
rubygems_version: 3.3.3
|
84
|
+
signing_key:
|
127
85
|
specification_version: 4
|
128
86
|
summary: Postgres partitioning as easy as pie
|
129
87
|
test_files: []
|