pg_ha_migrations 0.2.0 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.ruby-version +1 -1
- data/.travis.yml +5 -1
- data/README.md +24 -2
- data/lib/pg_ha_migrations.rb +11 -0
- data/lib/pg_ha_migrations/hacks/disable_ddl_transaction.rb +15 -0
- data/lib/pg_ha_migrations/safe_statements.rb +0 -2
- data/lib/pg_ha_migrations/unsafe_statements.rb +55 -20
- data/lib/pg_ha_migrations/version.rb +1 -1
- data/pg_ha_migrations.gemspec +11 -3
- metadata +13 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dc85b0812b8317e3f129a5ee82749513549298437822ae30df47c80c1150dfd3
|
4
|
+
data.tar.gz: 88f379fb400d43d0fcf435646a6b102558d51909363a4daec4d519a865cac823
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 15e3cc9067bf0befe6b348fc1743e74d1c1511c26d089907054e427560db4eee05f4f82fc68c29e56165ee08416c7dfb195e94859539999fd9fd1474d23cc0bd
|
7
|
+
data.tar.gz: bcb14d8320a4aeb19617787076bcd01215c34e110a9e9d0562a24de87347958e86af2ee756a96d835e5f77844038a508bd76d9e623406ee5f8c3b3f4b49dfa5e
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
ruby-2.
|
1
|
+
ruby-2.4.3
|
data/.travis.yml
CHANGED
data/README.md
CHANGED
@@ -1,6 +1,12 @@
|
|
1
1
|
# PgHaMigrations
|
2
2
|
|
3
|
-
|
3
|
+
[![Build Status](https://travis-ci.org/braintree/pg_ha_migrations.svg?branch=master)](https://travis-ci.org/braintree/pg_ha_migrations/)
|
4
|
+
|
5
|
+
We've documented our learned best practices for applying schema changes without downtime in the post [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680) on the [Braintree Product and Technology Blog](https://medium.com/braintree-product-technology). Many of the approaches we take and choices we've made are explained in much greater depth there than in this README.
|
6
|
+
|
7
|
+
Internally we apply those best practices to our Rails applications through this gem which updates ActiveRecord migrations to clearly delineate safe and unsafe DDL as well as provide safe alternatives where possible.
|
8
|
+
|
9
|
+
Some projects attempt to hide complexity by having code determine the intent and magically do the right series of operations. But we (and by extension this gem) take the approach that it's better to understand exactly what the database is doing so that (particularly long running) operations are not a surprise during your deploy cycle.
|
4
10
|
|
5
11
|
Provided functionality:
|
6
12
|
- [Migrations](#migrations)
|
@@ -26,6 +32,20 @@ Or install it yourself as:
|
|
26
32
|
|
27
33
|
## Usage
|
28
34
|
|
35
|
+
### Rollback
|
36
|
+
|
37
|
+
Because we require that ["Rollback strategies do not involve reverting the database schema to its previous version"](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#360a), PgHaMigrations does not support ActiveRecord's automatic migration rollback capability.
|
38
|
+
|
39
|
+
Instead we write all of our migrations with only an `def up` method like:
|
40
|
+
|
41
|
+
```
|
42
|
+
def up
|
43
|
+
safe_add_column :table, :column
|
44
|
+
end
|
45
|
+
```
|
46
|
+
|
47
|
+
and never use `def change`. We believe that this is the only safe approach in production environments. For development environments we iterate by recreating the database from scratch every time we make a change.
|
48
|
+
|
29
49
|
### Migrations
|
30
50
|
|
31
51
|
In general, existing migrations are prefixed with `unsafe_` and safer alternatives are provided prefixed with `safe_`.
|
@@ -34,6 +54,8 @@ Migrations prefixed with `unsafe_` will warn when invoked. The API is designed t
|
|
34
54
|
|
35
55
|
Migrations prefixed with `safe_` prefer concurrent operations where available, set low lock timeouts where appropriate, and decompose operations into multiple safe steps.
|
36
56
|
|
57
|
+
[Running multiple DDL statements inside a transaction acquires exclusive locks on all of the modified objects](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#cc22). For that reason, this gem [disables DDL transactions](./lib/pg_ha_migrations.rb:8) by default. You can change this by resetting `ActiveRecord::Migration.disable_ddl_transaction` in your application.
|
58
|
+
|
37
59
|
The following functionality is currently unsupported:
|
38
60
|
|
39
61
|
- Rollbacks
|
@@ -112,7 +134,7 @@ unsafe_make_column_not_nullable :table, :column
|
|
112
134
|
|
113
135
|
#### safe\_add\_concurrent\_index
|
114
136
|
|
115
|
-
Add an index concurrently.
|
137
|
+
Add an index concurrently.
|
116
138
|
|
117
139
|
```ruby
|
118
140
|
safe_add_concurrent_index :table, :column
|
data/lib/pg_ha_migrations.rb
CHANGED
@@ -27,4 +27,15 @@ require "pg_ha_migrations/unsafe_statements"
|
|
27
27
|
require "pg_ha_migrations/safe_statements"
|
28
28
|
require "pg_ha_migrations/allowed_versions"
|
29
29
|
require "pg_ha_migrations/railtie"
|
30
|
+
require "pg_ha_migrations/hacks/disable_ddl_transaction"
|
30
31
|
|
32
|
+
module PgHaMigrations::AutoIncluder
|
33
|
+
def inherited(klass)
|
34
|
+
super(klass) if defined?(super)
|
35
|
+
|
36
|
+
klass.prepend(PgHaMigrations::SafeStatements)
|
37
|
+
klass.prepend(PgHaMigrations::UnsafeStatements)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
ActiveRecord::Migration.singleton_class.prepend(PgHaMigrations::AutoIncluder)
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require "active_record/migration"
|
2
|
+
|
3
|
+
module PgHaMigrations
|
4
|
+
module ActiveRecordHacks
|
5
|
+
module DisableDdlTransaction
|
6
|
+
def disable_ddl_transaction
|
7
|
+
# Default to disabled unless someone has set it elsewhere
|
8
|
+
defined?(@disable_ddl_transaction) ? @disable_ddl_transaction : true
|
9
|
+
end
|
10
|
+
end
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
ActiveRecord::Migration.singleton_class.prepend(PgHaMigrations::ActiveRecordHacks::DisableDdlTransaction)
|
15
|
+
|
@@ -1,45 +1,78 @@
|
|
1
1
|
module PgHaMigrations::UnsafeStatements
|
2
|
-
def self.
|
2
|
+
def self.delegate_unsafe_method_to_migration_base_class(method_name)
|
3
3
|
define_method("unsafe_#{method_name}") do |*args, &block|
|
4
|
-
|
5
|
-
#
|
6
|
-
|
7
|
-
|
4
|
+
# Dispatching here is a bit complicated: we need to execute the method
|
5
|
+
# belonging to the first member of the inheritance chain (besides
|
6
|
+
# UnsafeStatements). If don't find the method in the inheritance chain,
|
7
|
+
# we need to rely on the ActiveRecord::Migration#method_missing
|
8
|
+
# implementation since much of ActiveRecord::Migration's functionality
|
9
|
+
# is not implemented in real methods but rather by proxying.
|
10
|
+
#
|
11
|
+
# For example, ActiveRecord::Migration doesn't define #create_table.
|
12
|
+
# Instead ActiveRecord::Migration#method_missing proxies the method
|
13
|
+
# to the connection. However some migration compatibility version
|
14
|
+
# subclasses _do_ explicitly define #create_table, so we can't rely
|
15
|
+
# on only one way of finding the proper dispatch target.
|
16
|
+
|
17
|
+
# Exclude our `raise` guard implementations.
|
18
|
+
ancestors_without_unsafe_statements = self.class.ancestors - [PgHaMigrations::UnsafeStatements]
|
19
|
+
|
20
|
+
delegate_method = self.method(method_name)
|
21
|
+
candidate_method = delegate_method
|
22
|
+
|
23
|
+
# Find the first usable method in the ancestor chain
|
24
|
+
# or stop looking if there are no more possible
|
25
|
+
# implementations.
|
26
|
+
until candidate_method.nil? || ancestors_without_unsafe_statements.include?(candidate_method.owner)
|
27
|
+
candidate_method = candidate_method.super_method
|
28
|
+
end
|
29
|
+
|
30
|
+
if candidate_method
|
31
|
+
delegate_method = candidate_method
|
32
|
+
end
|
33
|
+
|
34
|
+
# If we failed to find a concrete implementation from the
|
35
|
+
# inheritance chain, use ActiveRecord::Migrations# method_missing
|
36
|
+
# otherwise use the method from the inheritance chain.
|
37
|
+
if delegate_method.owner == PgHaMigrations::UnsafeStatements
|
38
|
+
method_missing(method_name, *args, &block)
|
39
|
+
else
|
40
|
+
delegate_method.call(*args, &block)
|
8
41
|
end
|
9
42
|
end
|
10
43
|
end
|
11
44
|
|
12
|
-
|
45
|
+
delegate_unsafe_method_to_migration_base_class :create_table
|
13
46
|
def create_table(name, options={})
|
14
47
|
raise PgHaMigrations::UnsafeMigrationError.new(":create_table is NOT SAFE! Use safe_create_table instead")
|
15
48
|
end
|
16
49
|
|
17
|
-
|
50
|
+
delegate_unsafe_method_to_migration_base_class :add_column
|
18
51
|
def add_column(table, column, type, options={})
|
19
52
|
raise PgHaMigrations::UnsafeMigrationError.new(":add_column is NOT SAFE! Use safe_add_column instead")
|
20
53
|
end
|
21
54
|
|
22
|
-
|
55
|
+
delegate_unsafe_method_to_migration_base_class :change_table
|
23
56
|
def change_table(name, options={})
|
24
57
|
raise PgHaMigrations::UnsafeMigrationError.new(":change_table is NOT SAFE! Use a combination of safe and explicit unsafe migration methods instead")
|
25
58
|
end
|
26
59
|
|
27
|
-
|
60
|
+
delegate_unsafe_method_to_migration_base_class :drop_table
|
28
61
|
def drop_table(name)
|
29
62
|
raise PgHaMigrations::UnsafeMigrationError.new(":drop_table is NOT SAFE! Explicitly call :unsafe_drop_table to proceed")
|
30
63
|
end
|
31
64
|
|
32
|
-
|
65
|
+
delegate_unsafe_method_to_migration_base_class :rename_table
|
33
66
|
def rename_table(old_name, new_name)
|
34
67
|
raise PgHaMigrations::UnsafeMigrationError.new(":rename_table is NOT SAFE! Explicitly call :unsafe_rename_table to proceed")
|
35
68
|
end
|
36
69
|
|
37
|
-
|
70
|
+
delegate_unsafe_method_to_migration_base_class :rename_column
|
38
71
|
def rename_column(table_name, column_name, new_column_name)
|
39
72
|
raise PgHaMigrations::UnsafeMigrationError.new(":rename_column is NOT SAFE! Explicitly call :unsafe_rename_column to proceed")
|
40
73
|
end
|
41
74
|
|
42
|
-
|
75
|
+
delegate_unsafe_method_to_migration_base_class :change_column
|
43
76
|
def change_column(table_name, column_name, type, options={})
|
44
77
|
raise PgHaMigrations::UnsafeMigrationError.new(":change_column is NOT SAFE! Use a combination of safe and explicit unsafe migration methods instead")
|
45
78
|
end
|
@@ -50,30 +83,32 @@ module PgHaMigrations::UnsafeStatements
|
|
50
83
|
EOS
|
51
84
|
end
|
52
85
|
|
53
|
-
|
86
|
+
delegate_unsafe_method_to_migration_base_class :remove_column
|
54
87
|
def remove_column(table_name, column_name, type, options={})
|
55
88
|
raise PgHaMigrations::UnsafeMigrationError.new(":remove_column is NOT SAFE! Explicitly call :unsafe_remove_column to proceed")
|
56
89
|
end
|
57
90
|
|
58
|
-
|
91
|
+
delegate_unsafe_method_to_migration_base_class :add_index
|
59
92
|
def add_index(table_name, column_names, options={})
|
60
93
|
raise PgHaMigrations::UnsafeMigrationError.new(":add_index is NOT SAFE! Use safe_add_concurrent_index instead")
|
61
94
|
end
|
62
95
|
|
63
|
-
|
96
|
+
delegate_unsafe_method_to_migration_base_class :execute
|
64
97
|
def execute(sql, name = nil)
|
65
|
-
|
98
|
+
if caller[0] =~ /lib\/active_record\/migration\/compatibility.rb/
|
99
|
+
super
|
100
|
+
else
|
101
|
+
raise PgHaMigrations::UnsafeMigrationError.new(":execute is NOT SAFE! Explicitly call :unsafe_execute to proceed")
|
102
|
+
end
|
66
103
|
end
|
67
104
|
|
68
|
-
|
105
|
+
delegate_unsafe_method_to_migration_base_class :remove_index
|
69
106
|
def remove_index(table_name, options={})
|
70
107
|
raise PgHaMigrations::UnsafeMigrationError.new(":remove_index is NOT SAFE! Use safe_remove_concurrent_index instead for Postgres 9.6 databases; Explicitly call :unsafe_remove_index to proceed on Postgres 9.1")
|
71
108
|
end
|
72
109
|
|
73
|
-
|
110
|
+
delegate_unsafe_method_to_migration_base_class :add_foreign_key
|
74
111
|
def add_foreign_key(from_table, to_table, options)
|
75
112
|
raise PgHaMigrations::UnsafeMigrationError.new(":add_foreign_key is NOT SAFE! Explicitly call :unsafe_add_foreign_key only if you have guidance from a migration reviewer in #service-app-db.")
|
76
113
|
end
|
77
114
|
end
|
78
|
-
|
79
|
-
ActiveRecord::Migration.send(:prepend, PgHaMigrations::UnsafeStatements)
|
data/pg_ha_migrations.gemspec
CHANGED
@@ -6,11 +6,19 @@ require "pg_ha_migrations/version"
|
|
6
6
|
Gem::Specification.new do |spec|
|
7
7
|
spec.name = "pg_ha_migrations"
|
8
8
|
spec.version = PgHaMigrations::VERSION
|
9
|
-
spec.authors =
|
9
|
+
spec.authors = %w{
|
10
|
+
celeen
|
11
|
+
cosgroveb
|
12
|
+
jaronkk
|
13
|
+
jcoleman
|
14
|
+
kexline4710
|
15
|
+
mgates
|
16
|
+
redneckbeard
|
17
|
+
}
|
10
18
|
spec.email = ["code@getbraintree.com"]
|
11
19
|
|
12
|
-
spec.summary = %q{}
|
13
|
-
spec.description = %q{}
|
20
|
+
spec.summary = %q{Enforces DDL/migration safety in Ruby on Rails project with an emphasis on explicitly choosing trade-offs and avoiding unnecessary magic.}
|
21
|
+
spec.description = %q{Enforces DDL/migration safety in Ruby on Rails project with an emphasis on explicitly choosing trade-offs and avoiding unnecessary magic.}
|
14
22
|
spec.homepage = ""
|
15
23
|
spec.license = "MIT"
|
16
24
|
|
metadata
CHANGED
@@ -1,14 +1,20 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pg_ha_migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
|
+
- celeen
|
8
|
+
- cosgroveb
|
9
|
+
- jaronkk
|
7
10
|
- jcoleman
|
11
|
+
- kexline4710
|
12
|
+
- mgates
|
13
|
+
- redneckbeard
|
8
14
|
autorequire:
|
9
15
|
bindir: exe
|
10
16
|
cert_chain: []
|
11
|
-
date: 2019-
|
17
|
+
date: 2019-03-04 00:00:00.000000000 Z
|
12
18
|
dependencies:
|
13
19
|
- !ruby/object:Gem::Dependency
|
14
20
|
name: bundler
|
@@ -114,7 +120,8 @@ dependencies:
|
|
114
120
|
- - ">="
|
115
121
|
- !ruby/object:Gem::Version
|
116
122
|
version: '0'
|
117
|
-
description:
|
123
|
+
description: Enforces DDL/migration safety in Ruby on Rails project with an emphasis
|
124
|
+
on explicitly choosing trade-offs and avoiding unnecessary magic.
|
118
125
|
email:
|
119
126
|
- code@getbraintree.com
|
120
127
|
executables: []
|
@@ -136,6 +143,7 @@ files:
|
|
136
143
|
- lib/pg_ha_migrations/allowed_versions.rb
|
137
144
|
- lib/pg_ha_migrations/blocking_database_transactions.rb
|
138
145
|
- lib/pg_ha_migrations/blocking_database_transactions_reporter.rb
|
146
|
+
- lib/pg_ha_migrations/hacks/disable_ddl_transaction.rb
|
139
147
|
- lib/pg_ha_migrations/railtie.rb
|
140
148
|
- lib/pg_ha_migrations/safe_statements.rb
|
141
149
|
- lib/pg_ha_migrations/unsafe_statements.rb
|
@@ -165,5 +173,6 @@ rubyforge_project:
|
|
165
173
|
rubygems_version: 2.7.8
|
166
174
|
signing_key:
|
167
175
|
specification_version: 4
|
168
|
-
summary:
|
176
|
+
summary: Enforces DDL/migration safety in Ruby on Rails project with an emphasis on
|
177
|
+
explicitly choosing trade-offs and avoiding unnecessary magic.
|
169
178
|
test_files: []
|