arrival 0.1.0

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'codeclimate-test-reporter', '~> 1.0.3', group: :test, require: nil
+gem 'rubocop', '~> 0.49.1', require: false

data/LICENSE.txt

@@ -0,0 +1,22 @@
+The MIT License (MIT)
+
+Copyright for portions of Arrival are held by Redbooth, Inc., 2015-2017. All
+other copyright for Arrival are held by Pau Pérez, 2017.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,209 @@
+# Arrival [![Build Status](https://travis-ci.com/WeTransfer/arrival.svg?token=Mq6F1pVeWbb6XsB9HonL&branch=main)](https://travis-ci.com/WeTransfer/arrival)
+
+Arrival is an **ActiveRecord connection adapter** that allows running
+**MySQL online and non-blocking DDL** through `ActiveRecord::Migration` without needing
+    to use a different DSL other than Rails' migrations DSL.
+
+It uses `pt-online-schema-change` command-line tool of
+[Percona
+Toolkit](https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html)
+which runs MySQL alter table statements without downtime.
+
+## Rename from "Percona Migrator"
+
+This project was formerly known as "Percona Migrator", but this incurs in an
+infringement of Percona's trade mark policy and thus has to be renamed. Said
+name is likely to cause confusion as to the source of the wrapper.
+
+The next major versions will use "Arrival" as gem name.
+
+## Installation
+
+Arrival relies on `pt-online-schema-change` from [Percona
+Toolkit](https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html)
+
+### Mac
+
+`brew install percona-toolkit`
+
+If when running a migration you see an error like:
+
+```
+PerconaMigrator::Error: Cannot connect to MySQL: Cannot connect to MySQL because
+the Perl DBI module is not installed or not found.
+```
+
+You also need to install the DBI and DBD::MySQL modules from `cpan`.
+
+```
+$ sudo cpan
+cpan> install DBI
+cpan> install DBD::mysql
+```
+
+### Linux
+
+#### Ubuntu/Debian based
+
+`apt-get install percona-toolkit`
+
+#### Arch Linux
+
+`pacman -S percona-toolkit perl-dbd-mysql`
+
+#### Other distros
+
+For other Linux distributions check out the [Percona Toolkit download
+page](https://www.percona.com/downloads/percona-toolkit/) to find the package
+that fits your distribution.
+
+You can also get it from [Percona's apt repository](https://www.percona.com/doc/percona-xtradb-cluster/5.5/installation/apt_repo.html)
+
+Once installed, add this line to your application's Gemfile:
+
+```ruby
+gem 'arrival'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install arrival
+
+## Usage
+
+Once you added it to your app's Gemfile, you can create and run Rails migrations
+as usual.
+
+All the `ALTER TABLE` statements will be executed with
+`pt-online-schema-change`, which will provide additional output to the
+migration.
+
+### pt-online-schema-change arguments
+
+#### with environment variable
+
+You can specify any `pt-online-schema-change` arguments when running the
+migration. All what you pass in the PERCONA_ARGS env var, will be bypassed to the
+binary, overwriting any default values. Note the format is the same as in
+`pt-online-schema-change`. Check the full list in [Percona Toolkit
+documentation](https://www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html#options)
+
+```ruby
+$ PERCONA_ARGS='--chunk-time=1' bundle exec rake db:migrate:up VERSION=xxx
+```
+
+or even mulitple arguments
+
+```ruby
+$ PERCONA_ARGS='--chunk-time=1 --critical-load=55' bundle exec rake db:migrate:up VERSION=xxx
+```
+
+Use caution when using PERCONA_ARGS with `db:migrate`, as your args will be applied
+to every call that Arrival makes to pt-osc.
+
+#### with global configuration
+
+You can specify any `pt-online-schema-change` arguments in global gem configuration
+using `global_percona_args` option.
+
+```ruby
+Arrival.configure do |config|
+  config.global_percona_args = '--chunk-time=1 --critical-load=55'
+end
+```
+
+Unlike using `PERCONA_ARGS`, options provided with global configuration will be applied
+every time sql command is executed via `pt-online-schema-change`.
+
+Arguments provided in global configuration can be overwritten with `PERCONA_ARGS` env variable.
+
+We recommend using this option with caution and only when you understand the consequences.
+
+### LHM support
+
+If you moved to Soundcloud's [Lhm](https://github.com/soundcloud/lhm) already,
+we got you covered. Arrival overrides Lhm's DSL so that all the alter
+statements also go through `pt-online-schema-change` as well.
+
+You can keep your Lhm migrations and start using Rails migration's DSL back
+again in your next migration.
+
+## Configuration
+
+You can override any of the default values from an initializer:
+
+```ruby
+Arrival.configure do |config|
+  config.tmp_path = '/tmp/'
+end
+```
+
+It's strongly recommended to name it after this gems name, such as
+`config/initializers/arrival.rb`
+
+## How it works
+
+When booting your Rails app, Arrival extends the
+`ActiveRecord::Migration#migrate` method to reset the connection and reestablish
+it using the `ArrivalAdapter` instead of the one you defined in your
+`config/database.yml`.
+
+Then, when any migration DSL methods such as `add_column` or `create_table` are
+executed, they all go to the
+[ArrivalAdapter](https://github.com/arrivalrb/arrival/blob/master/lib/active_record/connection_adapters/arrival_adapter.rb).
+There, the methods that require `ALTER TABLE` SQL statements, like `add_column`,
+are overriden to get executed with
+[Arrival::Runner](https://github.com/arrivalrb/arrival/blob/master/lib/arrival/runner.rb),
+which deals with the `pt-online-schema-change` binary. All the others, like
+`create_table`, are delegated to the ActiveRecord's built in Mysql2Adapter and
+so they follow the regular path.
+
+[Arrival::Runner](https://github.com/arrivalrb/arrival/blob/master/lib/arrival/runner.rb)
+spawns a new process that runs the `pt-online-schema-change` binary present in
+the system, with the appropriate arguments for the generated SQL.
+
+When any errors occur, an `ActiveRecord::StatementInvalid` exception is
+raised and the migration is aborted, as all other ActiveRecord connection
+adapters.
+
+## Trouleshooting
+
+### Error creating new table: DBD::mysql::db do failed: Can't write; duplicate key in table (TABLE_NAME)
+There is a [known bug](https://bugs.launchpad.net/percona-toolkit/+bug/1498128) in percona-toolkit version 2.2.15
+that prevents schema changes when a table has constraints. You should upgrade to a version later than 2.2.17 to fix the issue.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run
+`rake spec` to run the tests. You can also run `bin/console` for an interactive
+prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To
+release a new version, update the version number in `version.rb`, and then run
+`bundle exec rake release`, which will create a git tag for the version, push
+git commits and tags, and push the `.gem` file to
+[rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at
+https://github.com/arrivalrb/arrival. They need to be opened against
+`master` or `v3.2` only if the changes fix a bug in Rails 3.2 apps.
+
+Please note that this project is released with a Contributor Code of Conduct. By
+participating in this project you agree to abide by its terms.
+
+Check the code of conduct [here](CODE_OF_CONDUCT.md)
+
+## Changelog
+
+You can consult the changelog [here](CHANGELOG.md)
+
+## License
+
+The gem is available as open source under the terms of the [MIT
+License](http://opensource.org/licenses/MIT).

data/RELEASING.md

@@ -0,0 +1,17 @@
+# Releasing Arrival
+
+All releases come from the master branch. All other branches won't be maintained
+and will receive bug fix releases only.
+
+In order to give support to a new major Rails version, we'll branch off of
+master, name it following the Rails repo convention, such as `v4.2`, and
+we'll keep it open for bug fixes.
+
+1. Update `lib/arrival/version.rb` accordingly
+2. Review the `CHANGELOG.md` and add a new section following the format
+   `[version] - YYYY-MM-DD`. We conform to the guidelines of
+   http://keepachangelog.com/
+3. Commit the changes with the message `Prepare release VERSION`
+4. Execute the release rake task as `bundle exec rake release`. It creates the
+   tag, builds and pushes the gem to Rubygems.
+5. Announce it! :tada:

data/Rakefile

@@ -0,0 +1,17 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+require './configuration'
+require './test_database'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec
+
+namespace :db do
+  desc 'Create the test database'
+  task :create do
+    config = Configuration.new
+    TestDatabase.new(config).setup_test_database
+  end
+end

data/arrival.gemspec

@@ -0,0 +1,29 @@
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require 'arrival/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'arrival'
+  spec.version       = Arrival::VERSION
+  spec.authors       = ['Arno Fleming']
+  spec.email         = ['arno@wetransfer.com']
+
+  spec.summary       = %(pt-gh-ost runner for ActiveRecord migrations)
+  spec.description   = %(Execute your ActiveRecord migrations with Github's `gh-ost`, a triggerless online schema changer)
+  spec.homepage      = 'https://github.com/wetransfer/arrival'
+  spec.license       = 'MIT'
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.require_paths = ['lib']
+
+  spec.add_runtime_dependency 'railties', '~> 5.2.0'
+  spec.add_runtime_dependency 'activerecord', '~> 5.2.0'
+  spec.add_runtime_dependency 'mysql2', '>= 0.4.0', '<= 0.5.2'
+
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.4', '>= 3.4.0'
+  spec.add_development_dependency 'rspec-its', '~> 1.2'
+  spec.add_development_dependency 'pry' #,'~> 8.2', '>= 8.2.1'
+  spec.add_development_dependency 'climate_control', '~> 0.0.3'
+end

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require 'bundler/setup'
+require 'percona_migrator'
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+require "pry"
+Pry.start
+
+# require 'irb'
+# IRB.start

data/bin/rspec

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# This file was generated by Bundler.
+#
+# The application 'rspec' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require 'pathname'
+ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', Pathname.new(__FILE__).realpath)
+
+require 'rubygems'
+require 'bundler/setup'
+
+load Gem.bin_path('rspec-core', 'rspec')

data/bin/setup

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+bundle install
+
+bundle exec rake db:create

data/config.yml.erb

@@ -0,0 +1,4 @@
+username: <%= ENV['PERCONA_DB_USER'] || 'root' %>
+password: <%= ENV['PERCONA_DB_PASSWORD'] || '' %>
+database: <%= ENV['PERCONA_DB_NAME'] || 'arrival_test' %>
+hostname: <%= ENV['PERCONA_DB_HOST'] || 'localhost' %>

data/configuration.rb

@@ -0,0 +1,16 @@
+require 'yaml'
+require 'erb'
+
+class Configuration
+  CONFIG_PATH = 'config.yml.erb'.freeze
+
+  attr_reader :config
+
+  def initialize
+    @config = YAML.load(ERB.new(File.read(CONFIG_PATH)).result).freeze
+  end
+
+  def [](key)
+    config[key]
+  end
+end

data/docker-compose-inspiration.yml

@@ -0,0 +1,44 @@
+version: '3'
+services:
+  web:
+    build: 'web'
+    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
+    container_name: "web"
+    volumes:
+      - ./web:/myapp
+    ports:
+      - "3000:3000"
+    depends_on:
+      - mysql_master
+
+  ghost:
+    build: ghost
+    depends_on:
+      - mysql_master
+      - mysql_slave
+
+  mysql_master:
+    image: mysql:5.7
+    env_file:
+      - ./master/mysql_master.env
+    container_name: "mysql_master"
+    restart: "no"
+    ports:
+      - 4406:3306
+    volumes:
+      - ./master/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./master/data:/var/lib/mysql
+
+  mysql_slave:
+    image: mysql:5.7
+    env_file:
+      - ./slave/mysql_slave.env
+    container_name: "mysql_slave"
+    restart: "no"
+    ports:
+      - 5506:3306
+    depends_on:
+      - mysql_master
+    volumes:
+      - ./slave/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./slave/data:/var/lib/mysql

data/docker-compose.yml

@@ -0,0 +1,40 @@
+version: '3'
+services:
+  mysql_main:
+    image: mysql:5.7
+    env_file:
+      - ./main/mysql_main.env
+    container_name: "mysql_main"
+    restart: "no"
+    # ports:
+    #   - 4406:3306
+    volumes:
+      - ./main/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./main/data:/var/lib/mysql
+
+  mysql_replica:
+    image: mysql:5.7
+    env_file:
+      - ./replica/mysql_replica.env
+    container_name: "mysql_replica"
+    restart: "no"
+    # ports:
+    #   - 5506:3306
+    depends_on:
+      - mysql_main
+    volumes:
+      - ./replica/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./replica/data:/var/lib/mysql
+
+  arrival:
+    build: .
+    depends_on:
+      - mysql_main
+      - mysql_replica
+    # tty: true
+    # stdin_open: true
+    # environment:
+    #   PERCONA_DB_USER: root
+    #   PERCONA_DB_HOST: db
+    #   PERCONA_DB_PASSWORD:
+    #   PERCONA_DB_NAME: arrival_test