pg_ha_migrations 0.1.3

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 764006f9563011eaf0c11d51dfdac477b273f6e63687df9d9448861129640615
+  data.tar.gz: 2ddde5f6e1e39ab07034ff6d43bdd5a9c5f0b71ab75b6c9a5100f4ebfd3c2bec
+SHA512:
+  metadata.gz: 993b63e38c85ae14660d300e122c3ad6d748e88d12a62182624e1b80b18862a61fe337c88ecfca7e0e149f64b575d08ec09248972e3dc44fab844d7bc03a5954
+  data.tar.gz: 2bdccc2af3fa5e6834a31761c7e19ef4dc1983f5a84dc2b7f7dd8c188ac3c6a2e5ff745de1907bc2ff66c92976668f070eedabb40e08acb5bfe6d02617f8c6eb

data/.gitignore

@@ -0,0 +1,12 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+# rspec failure tracking
+.rspec_status

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.ruby-version

@@ -0,0 +1 @@
+ruby-2.2.3

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 1.9.3
+before_install: gem install bundler -v 1.15.4

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at code@getbraintree.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,7 @@
+source "https://rubygems.org"
+
+git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
+
+# Specify your gem's dependencies in pg_ha_migrations.gemspec
+gemspec
+

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2018 Braintree Payments
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,213 @@
+# PgHaMigrations
+
+PgHaMigrations is a gem that applies learned best practices to ActiveRecord Migrations.
+
+Provided functionality:
+- [Migrations](#migrations)
+- [Utilities](#utilities)
+- [Rake Tasks](#rake-tasks)
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'pg_ha_migrations'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install pg_ha_migrations
+
+## Usage
+
+### Migrations
+
+In general, existing migrations are prefixed with `unsafe_` and safer alternatives are provided prefixed with `safe_`. 
+
+Migrations prefixed with `unsafe_` will warn when invoked. The API is designed to be explicit yet remain flexible. There may be situations where invoking the unsafe migration is preferred.
+
+Migrations prefixed with `safe_` prefer concurrent operations where available, set low lock timeouts where appropriate, and decompose operations into multiple safe steps.
+
+The following functionality is currently unsupported:
+
+- Rollbacks
+- Generators
+- schema.rb
+
+#### safe\_create\_table
+
+Safely creates a new table.
+
+```ruby
+safe_create_table :table do |t|
+  t.type :column
+end
+```
+
+#### safe\_create\_enum\_type
+
+Safely create a new enum without values.
+
+```ruby
+safe_create_enum_type :enum
+```
+Or, safely create the enum with values.
+```ruby
+safe_create_enum_type :enum, ["value1", "value2"]
+```
+
+#### safe\_add\_enum\_value
+
+Safely add a new enum value.
+
+```ruby
+safe_add_enum_value :enum, "value"
+```
+
+#### safe\_add\_column
+
+Safely add a column.
+
+```ruby
+safe_add_column :table, :column, :type
+```
+
+#### unsafe\_add\_column
+
+Unsafely add a column, but do so with a lock that is safely acquired.
+
+```ruby
+unsafe_add_column :table, :column, :type
+```
+
+#### safe\_change\_column\_default
+
+Safely change the default value for a column.
+
+```ruby
+safe_change_column_default :table, :column, "value"
+```
+
+#### safe\_make\_column\_nullable
+
+Safely make the column nullable.
+
+```ruby
+safe_make_column_nullable :table, :column
+```
+
+#### unsafe\_make\_column\_not\_nullable
+
+Unsafely make a column not nullable.
+
+```ruby
+unsafe_make_column_not_nullable :table, :column
+```
+
+#### safe\_add\_concurrent\_index
+
+Add an index concurrently. Migrations that contain this statement must also include `disable_ddl_transaction!`.
+
+```ruby
+safe_add_concurrent_index :table, :column
+```
+
+Add a composite btree index.
+
+```ruby
+safe_add_concurrent_index :table, [:column1, :column2], name: "index_name", using: :btree
+```
+
+#### safe\_remove\_concurrent\_index
+
+Safely remove an index. Migrations that contain this statement must also include `disable_ddl_transaction!`.
+
+```ruby
+safe_remove_concurrent_index :table, :name => :index_name
+```
+
+
+### Utilities
+
+#### safely\_acquire\_lock\_for\_table
+
+Safely acquire a lock for a table.
+
+```ruby
+safely_acquire_lock_for_table(:table) do
+  ...
+end
+```
+
+#### adjust\_lock\_timeout
+
+Adjust lock timeout.
+
+```ruby
+adjust_lock_timeout(seconds) do
+  ...
+end
+```
+
+#### adjust\_statement\_timeout
+
+Adjust statement timeout.
+
+```ruby
+adjust_statement_timeout(seconds) do
+  ...
+end
+```
+
+#### safe\_set\_maintenance\_work\_mem\_gb
+
+Set maintenance work mem.
+
+```ruby
+safe_set_maintenance_work_mem_gb 1
+```
+
+
+### Rake Tasks
+
+Use this to check for blocking transactions before migrating.
+
+    $ bundle exec rake pg_ha_migrations:check_blocking_database_transactions
+
+This rake task expects that you already have a connection open to your database. We suggest that you add another rake task to open the connection and then add that as a prerequisite for `pg_ha_migrations:check_blocking_database_transactions`.
+
+```ruby
+namespace :db do
+  desc "Establish a database connection"
+  task :establish_connection do
+    ActiveRecord::Base.establish_connection
+  end
+end
+
+Rake::Task["pg_ha_migrations:check_blocking_database_transactions"].enhance ["db:establish_connection"]
+```
+
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies and start a postgres docker container. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/braintreeps/pg_ha_migrations. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the PgHaMigrations project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/braintreeps/pg_ha_migrations/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,8 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require_relative File.join("lib", "pg_ha_migrations")
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec
+

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "pg_ha_migrations"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here
+
+# Launch a blank postgres image for testing
+docker run -d -p 127.0.0.1:5432:5432 postgres:10

data/lib/pg_ha_migrations.rb

@@ -0,0 +1,32 @@
+require "pg_ha_migrations/version"
+require "rails"
+require "active_record"
+require "active_record/migration"
+require "relation_to_struct"
+
+module PgHaMigrations
+  LOCK_TIMEOUT_SECONDS = 5
+  LOCK_FAILURE_RETRY_DELAY_MULTLIPLIER = 5
+
+  # Safe versus unsafe in this context specifically means the following:
+  # - Safe operations will not block for long periods of time.
+  # - Unsafe operations _may_ block for long periods of time.
+  UnsafeMigrationError = Class.new(Exception)
+
+  # Invalid migrations are operations which we expect to put the schema
+  # into a state inconsistent with our current guidelines; e.g., using
+  # a 32-bit foreign key value when the referenced primary key column
+  # is a 64-bit value.
+  InvalidMigrationError = Class.new(Exception)
+
+  # This gem only supports the PostgreSQL adapter at this time.
+  UnsupportedAdapter = Class.new(Exception)
+end
+
+require "pg_ha_migrations/blocking_database_transactions"
+require "pg_ha_migrations/blocking_database_transactions_reporter"
+require "pg_ha_migrations/unsafe_statements"
+require "pg_ha_migrations/safe_statements"
+require "pg_ha_migrations/allowed_versions"
+require "pg_ha_migrations/railtie"
+

data/lib/pg_ha_migrations/allowed_versions.rb

@@ -0,0 +1,23 @@
+require "active_record/migration/compatibility"
+
+module PgHaMigrations::AllowedVersions
+  ALLOWED_VERSIONS = [4.2, 5.0, 5.1].map do |v|
+    begin
+      ActiveRecord::Migration[v]
+    rescue ArgumentError
+      nil
+    end
+  end.compact
+
+  def inherited(subclass)
+    super
+    unless ALLOWED_VERSIONS.include?(subclass.superclass)
+      raise StandardError, "#{subclass.superclass} is not a permitted migration class\n" \
+        "\n" \
+        "To add a new version update the ALLOWED_VERSIONS constant in #{__FILE__}\n" \
+        "Currently allowed versions: #{ALLOWED_VERSIONS.map { |v| "ActiveRecord::Migration[#{v.current_version}]" }.join(', ')}"
+    end
+  end
+end
+
+ActiveRecord::Migration.singleton_class.send(:prepend, PgHaMigrations::AllowedVersions)

data/lib/pg_ha_migrations/blocking_database_transactions.rb

@@ -0,0 +1,54 @@
+module PgHaMigrations
+  class BlockingDatabaseTransactions
+    LongRunningTransaction = Struct.new(:database, :current_query, :transaction_age, :tables_with_locks) do
+      def description
+        "#{database} | tables (#{tables_with_locks.join(', ')}) have been locked for #{transaction_age} | query: #{current_query}"
+      end
+
+      def concurrent_index_creation?
+        !!current_query.match(/create\s+index\s+concurrently/i)
+      end
+    end
+
+    def self.autovacuum_regex
+      "^autovacuum: (?!.*to prevent wraparound)"
+    end
+
+    def self.find_blocking_transactions(minimum_transaction_age = "0 seconds")
+      pid_column, state_column = if ActiveRecord::Base.connection.select_value("SHOW server_version") =~ /9\.1/
+        ["procpid", "current_query"]
+      else
+        ["pid", "query"]
+      end
+
+      raw_query = <<-SQL
+        SELECT
+          psa.datname as database, -- Will only ever be one database
+          psa.#{state_column} as current_query,
+          clock_timestamp() - psa.xact_start AS transaction_age,
+          array_agg(distinct c.relname) AS tables_with_locks
+        FROM pg_stat_activity psa -- Cluster wide
+          JOIN pg_locks l ON (psa.#{pid_column} = l.pid)  -- Cluster wide
+          JOIN pg_class c ON (l.relation = c.oid) -- Database wide
+          JOIN pg_namespace ns ON (c.relnamespace = ns.oid) -- Database wide
+        WHERE psa.#{pid_column} != pg_backend_pid()
+          AND ns.nspname != 'pg_catalog'
+          AND c.relkind = 'r'
+          AND psa.xact_start < clock_timestamp() - ?::interval
+          AND psa.#{state_column} !~ ?
+          AND (
+            -- Be explicit about this being for a single database -- it's already implicit in
+            -- the relations used, and if we don't restrict this we could get incorrect results
+            -- with oid collisions from pg_namespace and pg_class.
+            l.database = 0
+            OR l.database = (SELECT d.oid FROM pg_database d WHERE d.datname = current_database())
+          )
+        GROUP BY psa.datname, psa.#{state_column}, psa.xact_start
+      SQL
+
+      query = ActiveRecord::Base.send(:sanitize_sql_for_conditions, [raw_query, minimum_transaction_age, autovacuum_regex])
+
+      ActiveRecord::Base.structs_from_sql(LongRunningTransaction, query)
+    end
+  end
+end

data/lib/pg_ha_migrations/blocking_database_transactions_reporter.rb

@@ -0,0 +1,52 @@
+require 'stringio'
+
+module PgHaMigrations
+  class BlockingDatabaseTransactionsReporter
+    CHECK_DURATION = "30 seconds"
+
+    def self.run
+      blocking_transactions = get_blocking_transactions
+      has_transactions = blocking_transactions.values.flatten.present?
+      _puts(report(blocking_transactions)) if has_transactions
+    end
+
+    def self.report(transactions)
+      report = StringIO.new
+      report << "Potentially blocking transactions:\n"
+      transactions.each do |db_description, blocking_transactions|
+        report << "#{db_description}:\n"
+        if blocking_transactions.empty?
+          report << "\t(no long running transactions)\n\n"
+        else
+          blocking_transactions.each do |transaction|
+            report << "\t#{transaction.description}\n\n"
+          end
+        end
+
+        if blocking_transactions.any?(&:concurrent_index_creation?)
+          report << <<-eos.strip_heredoc.lines.map { |line| "\t#{line}" }.join
+            Warning: concurrent indexes are currently being built. If you have any other
+                     migrations in this deploy that will attempt to create additional
+                     concurrent indexes on the same physical database (even if the table
+                     being indexes is on another dimension) those migrations will not be
+                     able to complete until the in-progress index creations finish.
+
+                     For more information, see #service-db in Slack.\n
+          eos
+          report << "\n" # Blank line intentional
+        end
+      end
+      report.string
+    end
+
+    def self.get_blocking_transactions
+      {
+        "Primary database" => PgHaMigrations::BlockingDatabaseTransactions.find_blocking_transactions(CHECK_DURATION)
+      }
+    end
+
+    def self._puts(msg)
+      puts msg
+    end
+  end
+end

data/lib/pg_ha_migrations/railtie.rb

@@ -0,0 +1,5 @@
+class PgHaMigrations::Railtie < Rails::Railtie
+  rake_tasks do
+    load 'tasks/blocking_transactions.rake'
+  end
+end

data/lib/pg_ha_migrations/safe_statements.rb

@@ -0,0 +1,192 @@
+module PgHaMigrations::SafeStatements
+  def safe_create_table(table, options={}, &block)
+    unsafe_create_table(table, options, &block)
+  end
+
+  def safe_create_enum_type(name, values=nil)
+    case values
+    when nil
+      raise ArgumentError, "safe_create_enum_type expects a set of values; if you want an enum with no values please pass an empty array"
+    when []
+      unsafe_execute("CREATE TYPE #{PG::Connection.quote_ident(name.to_s)} AS ENUM ()")
+    else
+      escaped_values = values.map do |value|
+        "'#{PG::Connection.escape_string(value.to_s)}'"
+      end
+      unsafe_execute("CREATE TYPE #{PG::Connection.quote_ident(name.to_s)} AS ENUM (#{escaped_values.join(',')})")
+    end
+  end
+
+  def safe_add_enum_value(name, value)
+    unsafe_execute("ALTER TYPE #{PG::Connection.quote_ident(name.to_s)} ADD VALUE '#{PG::Connection.escape_string(value)}'")
+  end
+
+  def safe_add_column(table, column, type, options = {})
+    if options.has_key? :default
+      raise PgHaMigrations::UnsafeMigrationError.new(":default is NOT SAFE! Use safe_change_column_default afterwards then backfill the data to prevent locking the table")
+    end
+    if options[:null] == false
+      raise PgHaMigrations::UnsafeMigrationError.new(":null => false is NOT SAFE if the table has data! If you _really_ want to do this, use unsafe_make_column_not_nullable")
+    end
+
+    unsafe_add_column(table, column, type, options)
+  end
+
+  def unsafe_add_column(table, column, type, options = {})
+    safely_acquire_lock_for_table(table) do
+      super(table, column, type, options)
+    end
+  end
+
+  def safe_change_column_default(table, column, default_value)
+    escaped_value = case default_value
+    when Proc
+      PG::Connection.escape_string(default_value.call.to_s)
+    else
+      "'#{PG::Connection.escape_string(default_value.to_s)}'"
+    end
+
+    safely_acquire_lock_for_table(table) do
+      unsafe_execute <<-SQL.strip_heredoc
+        ALTER TABLE #{PG::Connection.quote_ident(table.to_s)}
+        ALTER COLUMN #{PG::Connection.quote_ident(column.to_s)}
+        SET DEFAULT #{escaped_value}
+      SQL
+    end
+  end
+
+  def safe_make_column_nullable(table, column)
+    safely_acquire_lock_for_table(table) do
+      unsafe_execute "ALTER TABLE #{table} ALTER COLUMN #{column} DROP NOT NULL"
+    end
+  end
+
+  def unsafe_make_column_not_nullable(table, column, options={}) # options arg is only present for backwards compatiblity
+    safely_acquire_lock_for_table(table) do
+      unsafe_execute "ALTER TABLE #{table} ALTER COLUMN #{column} SET NOT NULL"
+    end
+  end
+
+  def safe_add_concurrent_index(table, columns, options={})
+    unsafe_add_index(table, columns, options.merge(:algorithm => :concurrently))
+  end
+
+  def safe_remove_concurrent_index(table, options={})
+    unless options.is_a?(Hash) && options.key?(:name)
+      raise ArgumentError, "Expected safe_remove_concurrent_index to be called with arguments (table_name, :name => ...)"
+    end
+    unless ActiveRecord::Base.connection.postgresql_version >= 90600
+      raise PgHaMigrations::InvalidMigrationError, "Removing an index concurrently is not supported on Postgres 9.1 databases"
+    end
+    index_size = select_value("SELECT pg_size_pretty(pg_relation_size('#{options[:name]}'))")
+    say "Preparing to drop index #{options[:name]} which is #{index_size} on disk..."
+    unsafe_remove_index(table, options.merge(:algorithm => :concurrently))
+  end
+
+  def safe_set_maintenance_work_mem_gb(gigabytes)
+    unsafe_execute("SET maintenance_work_mem = '#{PG::Connection.escape_string(gigabytes.to_s)} GB'")
+  end
+
+  def _per_migration_caller
+    @_per_migration_caller ||= Kernel.caller
+  end
+
+  def _check_postgres_adapter!
+    expected_adapter = "PostgreSQL"
+    actual_adapter = ActiveRecord::Base.connection.adapter_name
+    raise PgHaMigrations::UnsupportedAdapter, "This gem only works with the #{expected_adapter} adapter, found #{actual_adapter} instead" unless actual_adapter == expected_adapter
+  end
+
+  def exec_migration(conn, direction)
+    _check_postgres_adapter!
+    super(conn, direction)
+  end
+
+  def safely_acquire_lock_for_table(table, &block)
+    _check_postgres_adapter!
+    table = table.to_s
+    quoted_table_name = connection.quote_table_name(table)
+
+    successfully_acquired_lock = false
+
+    until successfully_acquired_lock
+      while (
+        blocking_transactions = PgHaMigrations::BlockingDatabaseTransactions.find_blocking_transactions("#{PgHaMigrations::LOCK_TIMEOUT_SECONDS} seconds")
+        blocking_transactions.any? { |query| query.tables_with_locks.include?(table) }
+      )
+        say "Waiting on blocking transactions:"
+        blocking_transactions.each do |blocking_transaction|
+          say blocking_transaction.description
+        end
+        sleep(PgHaMigrations::LOCK_TIMEOUT_SECONDS)
+      end
+
+      connection.transaction do
+        adjust_timeout_method = connection.postgresql_version >= 90300 ? :adjust_lock_timeout : :adjust_statement_timeout
+        begin
+          method(adjust_timeout_method).call(PgHaMigrations::LOCK_TIMEOUT_SECONDS) do
+            connection.execute("LOCK #{quoted_table_name};")
+          end
+          successfully_acquired_lock = true
+        rescue ActiveRecord::StatementInvalid => e
+          if e.message =~ /PG::LockNotAvailable.+ lock timeout/ || e.message =~ /PG::QueryCanceled.+ statement timeout/
+            sleep_seconds = PgHaMigrations::LOCK_FAILURE_RETRY_DELAY_MULTLIPLIER * PgHaMigrations::LOCK_TIMEOUT_SECONDS
+            say "Timed out trying to acquire an exclusive lock on the #{quoted_table_name} table."
+            say "Sleeping for #{sleep_seconds}s to allow potentially queued up queries to finish before continuing."
+            sleep(sleep_seconds)
+
+            raise ActiveRecord::Rollback
+          else
+            raise e
+          end
+        end
+
+        if successfully_acquired_lock
+          block.call
+        end
+      end
+    end
+  end
+
+  def adjust_lock_timeout(timeout_seconds = PgHaMigrations::LOCK_TIMEOUT_SECONDS, &block)
+    _check_postgres_adapter!
+    original_timeout = ActiveRecord::Base.value_from_sql("SHOW lock_timeout").sub(/s\Z/, '').to_i * 1000
+    begin
+      connection.execute("SET lock_timeout = #{PG::Connection.escape_string((timeout_seconds * 1000).to_s)};")
+      block.call
+    ensure
+      begin
+        connection.execute("SET lock_timeout = #{original_timeout};")
+      rescue ActiveRecord::StatementInvalid => e
+        if e.message =~ /PG::InFailedSqlTransaction/
+          # If we're in a failed transaction the `SET lock_timeout` will be rolled back,
+          # so we don't need to worry about cleaning up, and we can't execute SQL anyway.
+        else
+          raise e
+        end
+      end
+    end
+  end
+
+  def adjust_statement_timeout(timeout_seconds, &block)
+    _check_postgres_adapter!
+    original_timeout = ActiveRecord::Base.value_from_sql("SHOW statement_timeout").sub(/s\Z/, '').to_i * 1000
+    begin
+      connection.execute("SET statement_timeout = #{PG::Connection.escape_string((timeout_seconds * 1000).to_s)};")
+      block.call
+    ensure
+      begin
+        connection.execute("SET statement_timeout = #{original_timeout};")
+      rescue ActiveRecord::StatementInvalid => e
+        if e.message =~ /PG::InFailedSqlTransaction/
+          # If we're in a failed transaction the `SET lock_timeout` will be rolled back,
+          # so we don't need to worry about cleaning up, and we can't execute SQL anyway.
+        else
+          raise e
+        end
+      end
+    end
+  end
+end
+
+ActiveRecord::Migration.send(:prepend, PgHaMigrations::SafeStatements)

data/lib/pg_ha_migrations/unsafe_statements.rb

@@ -0,0 +1,79 @@
+module PgHaMigrations::UnsafeStatements
+  def self.delegate_unsafe_method_to_connection(method_name)
+    define_method("unsafe_#{method_name}") do |*args, &block|
+      arg_list = args.map { |arg| arg.inspect }.join(', ')
+      # say_with_time args taken from https://github.com/rails/rails/blob/4-2-stable/activerecord/lib/active_record/migration.rb#L654
+      say_with_time "#{method_name}(#{arg_list})" do
+        connection.send(method_name, *args, &block)
+      end
+    end
+  end
+
+  delegate_unsafe_method_to_connection :create_table
+  def create_table(name, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":create_table is NOT SAFE! Use safe_create_table instead")
+  end
+
+  delegate_unsafe_method_to_connection :add_column
+  def add_column(table, column, type, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":add_column is NOT SAFE! Use safe_add_column instead")
+  end
+
+  delegate_unsafe_method_to_connection :change_table
+  def change_table(name, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":change_table is NOT SAFE! Use a combination of safe and explicit unsafe migration methods instead")
+  end
+
+  delegate_unsafe_method_to_connection :drop_table
+  def drop_table(name)
+    raise PgHaMigrations::UnsafeMigrationError.new(":drop_table is NOT SAFE! Explicitly call :unsafe_drop_table to proceed")
+  end
+
+  delegate_unsafe_method_to_connection :rename_table
+  def rename_table(old_name, new_name)
+    raise PgHaMigrations::UnsafeMigrationError.new(":rename_table is NOT SAFE! Explicitly call :unsafe_rename_table to proceed")
+  end
+
+  delegate_unsafe_method_to_connection :rename_column
+  def rename_column(table_name, column_name, new_column_name)
+    raise PgHaMigrations::UnsafeMigrationError.new(":rename_column is NOT SAFE! Explicitly call :unsafe_rename_column to proceed")
+  end
+
+  delegate_unsafe_method_to_connection :change_column
+  def change_column(table_name, column_name, type, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":change_column is NOT SAFE! Use a combination of safe and explicit unsafe migration methods instead")
+  end
+
+  def change_column_null(table_name, column_name, null, default = nil)
+    raise PgHaMigrations::UnsafeMigrationError.new(<<-EOS.strip_heredoc)
+    :change_column_null is NOT (guaranteed to be) SAFE! Either use :safe_make_column_nullable or explicitly call :unsafe_make_column_not_nullable to proceed
+    EOS
+  end
+
+  delegate_unsafe_method_to_connection :remove_column
+  def remove_column(table_name, column_name, type, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":remove_column is NOT SAFE! Explicitly call :unsafe_remove_column to proceed")
+  end
+
+  delegate_unsafe_method_to_connection :add_index
+  def add_index(table_name, column_names, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":add_index is NOT SAFE! Use safe_add_concurrent_index instead")
+  end
+
+  delegate_unsafe_method_to_connection :execute
+  def execute(sql, name = nil)
+    raise PgHaMigrations::UnsafeMigrationError.new(":execute is NOT SAFE! Explicitly call :unsafe_execute to proceed")
+  end
+
+  delegate_unsafe_method_to_connection :remove_index
+  def remove_index(table_name, options={})
+    raise PgHaMigrations::UnsafeMigrationError.new(":remove_index is NOT SAFE! Use safe_remove_concurrent_index instead for Postgres 9.6 databases; Explicitly call :unsafe_remove_index to proceed on Postgres 9.1")
+  end
+
+  delegate_unsafe_method_to_connection :add_foreign_key
+  def add_foreign_key(from_table, to_table, options)
+    raise PgHaMigrations::UnsafeMigrationError.new(":add_foreign_key is NOT SAFE! Explicitly call :unsafe_add_foreign_key only if you have guidance from a migration reviewer in #service-app-db.")
+  end
+end
+
+ActiveRecord::Migration.send(:prepend, PgHaMigrations::UnsafeStatements)

data/lib/pg_ha_migrations/version.rb

@@ -0,0 +1,3 @@
+module PgHaMigrations
+  VERSION = "0.1.3"
+end

data/lib/tasks/blocking_transactions.rake

@@ -0,0 +1,9 @@
+require_relative File.join("..", "pg_ha_migrations")
+
+namespace :pg_ha_migrations do
+  desc "Check if blocking database transactions exist"
+  task :check_blocking_database_transactions do
+    PgHaMigrations::BlockingDatabaseTransactionsReporter.run
+  end
+end
+

data/pg_ha_migrations.gemspec

@@ -0,0 +1,33 @@
+# coding: utf-8
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "pg_ha_migrations/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "pg_ha_migrations"
+  spec.version       = PgHaMigrations::VERSION
+  spec.authors       = ["jcoleman"]
+  spec.email         = ["code@getbraintree.com"]
+
+  spec.summary       = %q{}
+  spec.description   = %q{}
+  spec.homepage      = ""
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.15"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency "pg"
+  spec.add_development_dependency "db-query-matchers", "~> 0.9.0"
+
+
+  spec.add_dependency "rails", ">= 5.0", "< 5.2"
+  spec.add_dependency "relation_to_struct"
+end

metadata

@@ -0,0 +1,169 @@
+--- !ruby/object:Gem::Specification
+name: pg_ha_migrations
+version: !ruby/object:Gem::Version
+  version: 0.1.3
+platform: ruby
+authors:
+- jcoleman
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2018-11-12 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.15'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.15'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: db-query-matchers
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.9.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.9.0
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '5.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '5.2'
+- !ruby/object:Gem::Dependency
+  name: relation_to_struct
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: ''
+email:
+- code@getbraintree.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".ruby-version"
+- ".travis.yml"
+- CODE_OF_CONDUCT.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- lib/pg_ha_migrations.rb
+- lib/pg_ha_migrations/allowed_versions.rb
+- lib/pg_ha_migrations/blocking_database_transactions.rb
+- lib/pg_ha_migrations/blocking_database_transactions_reporter.rb
+- lib/pg_ha_migrations/railtie.rb
+- lib/pg_ha_migrations/safe_statements.rb
+- lib/pg_ha_migrations/unsafe_statements.rb
+- lib/pg_ha_migrations/version.rb
+- lib/tasks/blocking_transactions.rake
+- pg_ha_migrations.gemspec
+homepage: ''
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.7.8
+signing_key: 
+specification_version: 4
+summary: ''
+test_files: []