pg_online_schema_change 0.2.0 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +2 -2
- data/Dockerfile +5 -0
- data/Gemfile +0 -13
- data/Gemfile.lock +13 -13
- data/README.md +49 -11
- data/diagrams/how-it-works.excalidraw +500 -508
- data/diagrams/how-it-works.png +0 -0
- data/lib/pg_online_schema_change/version.rb +1 -1
- data/scripts/release.sh +22 -0
- metadata +177 -9
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b06d39169d55c016d78969968c71cac7f0c49a3c716d0bc8c5004905ef820fb0
|
4
|
+
data.tar.gz: 78fcc0a928a5efa645ec201cfca093f4ef078b8d683f0776a7f9ad91bb11ef99
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 179fcaf09bcc36b69186ba48494835841a88c095e010756f0e6f322a146279d0790403f2c195f23f483e89a1cb74c10b50a3ceee26ad6d311e3d9977af1b8c07
|
7
|
+
data.tar.gz: 157255f9d901ec2cb980d84bd664cc0eaae32854bdd79ebd262b2112eef965f8dc2d123277a35446fefd6ecdac4a4ba4c94769589461053241d1eedbe40efa52
|
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
## [0.2.0] - 2022-02-17
|
2
2
|
|
3
|
-
- Use ISOLATION LEVEL SERIALIZABLE ([#42](https://github.com/shayonj/pg-
|
3
|
+
- Use ISOLATION LEVEL SERIALIZABLE ([#42](https://github.com/shayonj/pg-osc/pull/42)) (props to @jfrost)
|
4
4
|
|
5
5
|
## [0.1.0] - 2022-02-16
|
6
6
|
|
@@ -10,4 +10,4 @@ pg-online-schema-change (`pg-osc`) is a tool for making schema changes (any `ALT
|
|
10
10
|
|
11
11
|
`pg-osc` uses the concept of shadow table to perform schema changes. At a high level, it copies the contents from a primary table to a shadow table, performs the schema change on the shadow table and swaps the table names in the end while preserving all changes to the primary table using triggers (via audit table).
|
12
12
|
|
13
|
-
Checkout [Readme](https://github.com/shayonj/pg-
|
13
|
+
Checkout [Readme](https://github.com/shayonj/pg-osc#readme) for more details.
|
data/Dockerfile
ADDED
data/Gemfile
CHANGED
@@ -3,16 +3,3 @@
|
|
3
3
|
source "https://rubygems.org"
|
4
4
|
|
5
5
|
gemspec
|
6
|
-
|
7
|
-
gem "ougai", "~> 2.0.0"
|
8
|
-
gem "pg", "~> 1.0"
|
9
|
-
gem "pg_query", "~> 2.1.2"
|
10
|
-
gem "pry"
|
11
|
-
gem "rake", "~> 13.0"
|
12
|
-
gem "rspec", "~> 3.0"
|
13
|
-
gem "rubocop", "~> 1.23.0"
|
14
|
-
gem "rubocop-packaging", "~> 0.5.1"
|
15
|
-
gem "rubocop-performance", "~> 1.12.0"
|
16
|
-
gem "rubocop-rake", "~> 0.6.0"
|
17
|
-
gem "rubocop-rspec", "~> 2.7.0"
|
18
|
-
gem "thor", "~> 1.1.0"
|
data/Gemfile.lock
CHANGED
@@ -1,7 +1,11 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
pg_online_schema_change (0.
|
4
|
+
pg_online_schema_change (0.2.0)
|
5
|
+
ougai (~> 2.0.0)
|
6
|
+
pg (~> 1.3.2)
|
7
|
+
pg_query (~> 2.1.3)
|
8
|
+
thor (~> 1.2.1)
|
5
9
|
|
6
10
|
GEM
|
7
11
|
remote: https://rubygems.org/
|
@@ -9,18 +13,18 @@ GEM
|
|
9
13
|
ast (2.4.2)
|
10
14
|
coderay (1.1.3)
|
11
15
|
diff-lcs (1.5.0)
|
12
|
-
google-protobuf (3.19.
|
13
|
-
google-protobuf (3.19.
|
16
|
+
google-protobuf (3.19.4)
|
17
|
+
google-protobuf (3.19.4-x86_64-linux)
|
14
18
|
method_source (1.0.0)
|
15
|
-
oj (3.13.
|
19
|
+
oj (3.13.11)
|
16
20
|
ougai (2.0.0)
|
17
21
|
oj (~> 3.10)
|
18
22
|
parallel (1.21.0)
|
19
23
|
parser (3.0.3.2)
|
20
24
|
ast (~> 2.4.1)
|
21
|
-
pg (1.2
|
22
|
-
pg_query (2.1.
|
23
|
-
google-protobuf (>= 3.
|
25
|
+
pg (1.3.2)
|
26
|
+
pg_query (2.1.3)
|
27
|
+
google-protobuf (>= 3.19.2)
|
24
28
|
pry (0.14.1)
|
25
29
|
coderay (~> 1.1)
|
26
30
|
method_source (~> 1.0)
|
@@ -62,7 +66,7 @@ GEM
|
|
62
66
|
rubocop-rspec (2.7.0)
|
63
67
|
rubocop (~> 1.19)
|
64
68
|
ruby-progressbar (1.11.0)
|
65
|
-
thor (1.1
|
69
|
+
thor (1.2.1)
|
66
70
|
unicode-display_width (2.1.0)
|
67
71
|
|
68
72
|
PLATFORMS
|
@@ -70,11 +74,8 @@ PLATFORMS
|
|
70
74
|
x86_64-linux
|
71
75
|
|
72
76
|
DEPENDENCIES
|
73
|
-
ougai (~> 2.0.0)
|
74
|
-
pg (~> 1.0)
|
75
77
|
pg_online_schema_change!
|
76
|
-
|
77
|
-
pry
|
78
|
+
pry (~> 0.14.1)
|
78
79
|
rake (~> 13.0)
|
79
80
|
rspec (~> 3.0)
|
80
81
|
rubocop (~> 1.23.0)
|
@@ -82,7 +83,6 @@ DEPENDENCIES
|
|
82
83
|
rubocop-performance (~> 1.12.0)
|
83
84
|
rubocop-rake (~> 0.6.0)
|
84
85
|
rubocop-rspec (~> 2.7.0)
|
85
|
-
thor (~> 1.1.0)
|
86
86
|
|
87
87
|
BUNDLED WITH
|
88
88
|
2.3.3
|
data/README.md
CHANGED
@@ -1,5 +1,6 @@
|
|
1
|
-
# pg-
|
2
|
-
|
1
|
+
# pg-osc
|
2
|
+
|
3
|
+
[![CircleCI](https://circleci.com/gh/shayonj/pg-osc/tree/main.svg?style=shield)](https://circleci.com/gh/shayonj/pg-osc/tree/main)
|
3
4
|
[![Gem Version](https://badge.fury.io/rb/pg_online_schema_change.svg)](https://badge.fury.io/rb/pg_online_schema_change)
|
4
5
|
|
5
6
|
pg-online-schema-change (`pg-osc`) is a tool for making schema changes (any `ALTER` statements) in Postgres tables with minimal locks, thus helping achieve zero downtime schema changes against production workloads.
|
@@ -10,6 +11,25 @@ pg-online-schema-change (`pg-osc`) is a tool for making schema changes (any `ALT
|
|
10
11
|
|
11
12
|
⚠️ Proceed with caution when using this on production like workloads. Best to try on similar setup or staging like environment first. Read on below for some examples and caveats.
|
12
13
|
|
14
|
+
## Table of Contents
|
15
|
+
|
16
|
+
- [Installation](#installation)
|
17
|
+
- [Requirements](#requirements)
|
18
|
+
- [Usage](#usage)
|
19
|
+
- [How does it work](#how-does-it-work)
|
20
|
+
- [Prominent features](#prominent-features)
|
21
|
+
- [Examples](#examples)
|
22
|
+
* [Renaming a column](#renaming-a-column)
|
23
|
+
* [Multiple ALTER statements](#multiple-alter-statements)
|
24
|
+
* [Kill other backends after 5s](#kill-other-backends-after-5s)
|
25
|
+
* [Backfill data](#backfill-data)
|
26
|
+
* [Running using Docker](#running-using-docker)
|
27
|
+
- [Caveats](#caveats)
|
28
|
+
- [Development](#development)
|
29
|
+
- [Releasing](#releasing)
|
30
|
+
- [Contributing](#contributing)
|
31
|
+
- [License](#license)
|
32
|
+
- [Code of Conduct](#code-of-conduct)
|
13
33
|
## Installation
|
14
34
|
|
15
35
|
Add this line to your application's Gemfile:
|
@@ -26,6 +46,13 @@ Or install it yourself as:
|
|
26
46
|
|
27
47
|
$ gem install pg_online_schema_change
|
28
48
|
|
49
|
+
This will include all dependencies accordingly as well. Make sure the following requirements are satisfied.
|
50
|
+
|
51
|
+
Or via Docker:
|
52
|
+
|
53
|
+
docker pull shayonj/pg-osc:latest
|
54
|
+
|
55
|
+
https://hub.docker.com/r/shayonj/pg-osc
|
29
56
|
## Requirements
|
30
57
|
- PostgreSQL 9.6 and later
|
31
58
|
- Ruby 2.6 and later
|
@@ -92,7 +119,7 @@ print the version
|
|
92
119
|
- Copies over indexes and Foreign keys.
|
93
120
|
- Optionally drop or retain old tables in the end.
|
94
121
|
- Backfill old/new columns as data is copied from primary table to shadow table, and then perform the swap. [Example](#backfill-data)
|
95
|
-
- **TBD**: Ability to reverse the change with no data loss. [tracking issue](https://github.com/shayonj/pg-
|
122
|
+
- **TBD**: Ability to reverse the change with no data loss. [tracking issue](https://github.com/shayonj/pg-osc/issues/14)
|
96
123
|
|
97
124
|
## Examples
|
98
125
|
|
@@ -160,7 +187,20 @@ pg-online-schema-change perform \
|
|
160
187
|
--copy-statement "/src/query.sql" \
|
161
188
|
--drop
|
162
189
|
```
|
163
|
-
|
190
|
+
|
191
|
+
### Running using Docker
|
192
|
+
|
193
|
+
```
|
194
|
+
docker run --network host -it --rm shayonj/pg-osc:latest \
|
195
|
+
pg-online-schema-change perform \
|
196
|
+
--alter-statement 'ALTER TABLE books ADD COLUMN "purchased" BOOLEAN DEFAULT FALSE; ALTER TABLE books RENAME COLUMN email TO new_email;' \
|
197
|
+
--dbname "postgres" \
|
198
|
+
--host "localhost" \
|
199
|
+
--username "jamesbond" \
|
200
|
+
--password "" \
|
201
|
+
--drop
|
202
|
+
```
|
203
|
+
## Caveats
|
164
204
|
- A primary key should exist on the table; without it, `pg-osc` will raise an exception
|
165
205
|
- This is because - currently there is no other way to uniquely identify rows during replay.
|
166
206
|
- `pg-osc` will acquire `ACCESS EXCLUSIVE` lock on the parent table twice during the operation.
|
@@ -195,16 +235,14 @@ To install this gem onto your local machine, run `bundle exec rake install`.
|
|
195
235
|
## Releasing
|
196
236
|
|
197
237
|
- Bump version in `version.rb`
|
198
|
-
-
|
199
|
-
-
|
200
|
-
- `gem build pg_online_schema_change.gemspec`
|
201
|
-
- `gem push pg_online_schema_change-0.1.0.gem`
|
238
|
+
- Commit
|
239
|
+
- `./scripts/release.sh 0.2.0`
|
202
240
|
- Update `CHANGELOG.md`
|
203
|
-
- Create a new release - https://github.com/shayonj/pg-
|
241
|
+
- Create a new release - https://github.com/shayonj/pg-osc/releases/new
|
204
242
|
|
205
243
|
## Contributing
|
206
244
|
|
207
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/shayonj/pg-
|
245
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/shayonj/pg-osc.
|
208
246
|
|
209
247
|
## License
|
210
248
|
|
@@ -212,4 +250,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
|
|
212
250
|
|
213
251
|
## Code of Conduct
|
214
252
|
|
215
|
-
Everyone interacting in the PgOnlineSchemaChange project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/shayonj/pg-
|
253
|
+
Everyone interacting in the PgOnlineSchemaChange project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/shayonj/pg-osc/blob/main/CODE_OF_CONDUCT.md).
|