departure 1.0.1 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 04cbb41217f09b9e06bb015359b9189f8342d5b1
-  data.tar.gz: 280d997d365765664307e0851a71c86a631e473f
+  metadata.gz: ef3cc8883b3b4dc294f1b94b7d93744e98812849
+  data.tar.gz: 242eac0339dd6f9aedf020d321cd9e6cc0bc24fe
 SHA512:
-  metadata.gz: 145620cb84bcb3bf28d3504e3a35713bcfcb265e3627da6faa2c9a01b77bf1bd4e2b8ffa34d4ecd2bff6fe39df68f454374694a4198286a475029fa90353ef3e
-  data.tar.gz: 9eac5663f4e2fa00d1d2f47a3222f63f65f16144c49f9bcb4438d49d3e4c996abb236074616972848b971c0e2992db586bdd2574f05048079afe002b9b90d518
+  metadata.gz: 515ab293664312a15f3482f1b0cc0156c325b59df452fa82a9a700adbff81197d1cfffcb4b1296a1afdad47f828a6aeffe0e5b831b61ff37dd89fab0127abdd4
+  data.tar.gz: 35b2746785ba419acddf8e2629e564f9382e7107fd4e57dcd75750e7440f347b3d1e4ce4c96a22a330096bec2bb383ef039070244a522dfd27df87018f850dd1

data/.gitignore

@@ -9,4 +9,4 @@
 /tmp/
 .byebug_history
 tags
-percona_migrator_error.log
+departure_error.log

data/CHANGELOG.md

@@ -11,45 +11,50 @@ Please follow the format in [Keep a Changelog](http://keepachangelog.com/)
 ### Removed
 ### Fixed
 
-## [1.0.1] - 2017-04-19
+## [2.0.0 - 2017-04-20]
 
 ### Added
 ### Changed
+
+- Rename the gem from percona_migrator to departure
+
 ### Removed
 
 - Percona_migrator's deprecation warnings when installing and running the gem.
 
 ### Fixed
 
-## [1.0.0] - 2017-04-19
+## [1.1.0] - 2017-04-07
 
 ### Added
-### Changed
 
-- Renamed gem to Departure
+- Allow passing any `pt-online-schema-change`'s arguments through the
+   `PERCONA_ARGS` env var when executing a migration with `rake db:migrate:up`
+   or `db:migrate:down`.
+- Allow setting global percona arguments via gem configuration
+- Filter MySQL's password from logs
 
-### Removed
-### Fixed
+### Changed
+
+- Enable default pt-online-schema-change replicas discovering mechanism.
+    So far, this was purposely set to `none`. To keep this same behaviour
+    provide the `PERCONA_ARGS=--recursion-method=none` env var when running the
+    migration.
 
-## [0.1.1] - 2017-03-10
+## [1.0.0] - 2016-11-30
 
 ### Added
 
+- Show pt-online-schema-change's stdout while the migration is running instead
+    of at then and all at once.
 - Store pt-online-schema-change's stderr to percona_migrator_error.log in the
     default Rails tmp folder.
-- Allow configuring the tmp directory where the error log gets written into.
-- Output a deprecation warning when installing and running the gem. It will be
-    replaced by Departure soon.
-
-### Changed
-
-- No longer a hard dependency on mysql2 0.3.20. Will accept 0.3.20 or any
-    higher patch revisions.
-
-### Fixed
-
-- Output pt-online-schema-change's stdout while the migration is running instead
-    of all of it at the end.
+- Allow configuring the tmp directory where the error log gets written into,
+    with the `tmp_path` configuration setting.
+- Support for ActiveRecord 4.0. Adds the following migration methods:
+  - #rename_index, #change_column_null, #add_reference, #remove_reference,
+    #set_field_encoding, #add_timestamps, #remove_timestamps, #rename_table,
+    #rename_column
 
 ## [0.1.0.rc.7] - 2016-09-15
 

data/README.md

@@ -26,6 +26,21 @@ Toolkit](https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change
 
 `brew install percona-toolkit`
 
+If when running a migration you see an error like:
+
+```
+PerconaMigrator::Error: Cannot connect to MySQL: Cannot connect to MySQL because
+the Perl DBI module is not installed or not found.
+```
+
+You also need to install the DBI and DBD::MySQL modules from `cpan`.
+
+```
+$ sudo cpan
+cpan> install DBI
+cpan> install DBD::mysql
+```
+
 ### Linux
 
 #### Ubuntu/Debian based
@@ -67,6 +82,48 @@ All the `ALTER TABLE` statements will be executed with
 `pt-online-schema-change`, which will provide additional output to the
 migration.
 
+### pt-online-schema-change arguments
+
+#### with environment variable
+
+You can specify any `pt-online-schema-change` arguments when running the
+migration. All what you pass in the PERCONA_ARGS env var, will be bypassed to the
+binary, overwriting any default values. Note the format is the same as in
+`pt-online-schema-change`. Check the full list in [Percona Toolkit
+documentation](https://www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html#options)
+
+```ruby
+$ PERCONA_ARGS='--chunk-time=1' bundle exec rake db:migrate:up VERSION=xxx
+```
+
+or even mulitple arguments
+
+```ruby
+$ PERCONA_ARGS='--chunk-time=1 --critical-load=55' bundle exec rake db:migrate:up VERSION=xxx
+```
+
+This however, only works for `db:migrate:up` or `db:migrate:down` rake tasks and
+not with `db:migrate`. The settings you provide can't be generalized as these
+vary depending on the database table and the kind of changes you apply.
+
+#### with global configuration
+
+You can specify any `pt-online-schema-change` arguments in global gem configuration
+using `global_percona_args` option.
+
+```ruby
+PerconaMigrator.configure do |config|
+  config.global_percona_args = '--chunk-time=1 --critical-load=55'
+end
+```
+
+Unlike using `PERCONA_ARGS`, options provided with global configuration will be applied 
+every time sql command is executed via `pt-online-schema-change`.
+ 
+Arguments provided in global configuration can be overwritten with `PERCONA_ARGS` env variable. 
+
+We recommend using this option with caution and only when you understand the consequences.
+
 ### LHM support
 
 If you moved to Soundcloud's [Lhm](https://github.com/soundcloud/lhm) already,
@@ -129,7 +186,8 @@ git commits and tags, and push the `.gem` file to
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at
-https://github.com/redbooth/percona_migrator.
+https://github.com/redbooth/percona_migrator. They need to be opened against
+`master` or `v3.2` only if the changes fix a bug in Rails 3.2 apps.
 
 Please note that this project is released with a Contributor Code of Conduct. By
 participating in this project you agree to abide by its terms.

data/RELEASING.md

@@ -0,0 +1,17 @@
+# Releasing Percona Migrator
+
+All releases come from the master branch. All other branches won't be maintained
+and will receive bug fix releases only.
+
+In order to give support to a new major Rails version, we'll branch off of
+master, name it following the Rails repo convention, such as `v4.2`, and
+we'll keep it open for bug fixes.
+
+1. Update `lib/percona_migrator/version.rb` accordingly
+2. Review the `CHANGELOG.md` and add a new section following the format
+   `[version] - YYYY-MM-DD`. We conform to the guidelines of
+   http://keepachangelog.com/
+3. Commit the changes with the message `Prepare release VERSION`
+4. Execute the release rake task as `bundle exec rake release`. It creates the
+   tag, builds and pushes the gem to Rubygems.
+5. Announce it! :tada:

data/bin/rspec

@@ -0,0 +1,17 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+#
+# This file was generated by Bundler.
+#
+# The application 'rspec' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require "pathname"
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
+  Pathname.new(__FILE__).realpath)
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("rspec-core", "rspec")

data/config.yml

@@ -1,3 +1,3 @@
 username: 'root'
 password: ''
-database: 'percona_migrator_test'
+database: 'departure_test'

data/departure.gemspec

@@ -7,8 +7,8 @@ require 'departure/version'
 Gem::Specification.new do |spec|
   spec.name          = 'departure'
   spec.version       = Departure::VERSION
-  spec.authors       = ['Ilya Zayats', 'Pau Pérez', 'Fran Casas', 'Jorge Morante']
-  spec.email         = ['ilya.zayats@redbooth.com', 'pau.perez@redbooth.com', 'fran.casas@redbooth.com', 'jorge.morante@redbooth.com']
+  spec.authors       = ['Ilya Zayats', 'Pau Pérez', 'Fran Casas', 'Jorge Morante', 'Adrian Serafin']
+  spec.email         = ['ilya.zayats@redbooth.com', 'pau.perez@redbooth.com', 'fran.casas@redbooth.com', 'jorge.morante@redbooth.com', 'adrian@softmad.pl']
 
   spec.summary       = %q{pt-online-schema-change runner for ActiveRecord migrations}
   spec.description   = %q{Execute your ActiveRecord migrations with Percona's pt-online-schema-change. Formerly known as Percona Migrator.}
@@ -18,12 +18,13 @@ Gem::Specification.new do |spec|
   spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
   spec.require_paths = ['lib']
 
-  spec.add_runtime_dependency 'rails', '~>3.2.22'
-  spec.add_runtime_dependency 'mysql2', '~>0.3.20'
+  spec.add_runtime_dependency 'rails', '~> 4.0.0'
+  spec.add_runtime_dependency 'mysql2', '0.3.20'
 
   spec.add_development_dependency 'bundler', '~> 1.10'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rspec', '~> 3.4', '>= 3.4.0'
   spec.add_development_dependency 'rspec-its', '~> 1.2'
   spec.add_development_dependency 'byebug', '~> 8.2', '>= 8.2.1'
+  spec.add_development_dependency 'climate_control', '~> 0.0.3'
 end

data/lib/active_record/connection_adapters/percona_adapter.rb

@@ -5,15 +5,21 @@ require 'departure'
 require 'forwardable'
 
 module ActiveRecord
-  class Base
+  module ConnectionHandling
     # Establishes a connection to the database that's used by all Active
     # Record objects.
-    def self.percona_connection(config)
+    def percona_connection(config)
       mysql2_connection = mysql2_connection(config)
 
+      config[:username] = 'root' if config[:username].nil?
+
+      connection_details = Departure::ConnectionDetails.new(config)
       verbose = ActiveRecord::Migration.verbose
-      percona_logger = Departure::LoggerFactory.build(verbose: verbose)
-      cli_generator = Departure::CliGenerator.new(config)
+      sanitizers = [
+        Departure::LogSanitizers::PasswordSanitizer.new(connection_details)
+      ]
+      percona_logger = Departure::LoggerFactory.build(sanitizers: sanitizers, verbose: verbose)
+      cli_generator = Departure::CliGenerator.new(connection_details)
 
       runner = Departure::Runner.new(
         percona_logger,
@@ -45,7 +51,7 @@ module ActiveRecord
 
       ADAPTER_NAME = 'Percona'.freeze
 
-      def_delegators :mysql_adapter, :last_inserted_id, :each_hash
+      def_delegators :mysql_adapter, :last_inserted_id, :each_hash, :set_field_encoding
 
       def initialize(connection, _logger, connection_options, _config)
         super
@@ -59,7 +65,7 @@ module ActiveRecord
       end
       alias :exec_update :exec_delete
 
-      def exec_insert(sql, name, binds)
+      def exec_insert(sql, name, binds, pk = nil, sequence_name = nil)
         execute(to_sql(sql, binds), name)
       end
 
@@ -77,7 +83,7 @@ module ActiveRecord
       # Executes a SELECT query and returns an array of record hashes with the
       # column names as keys and column values as values.
       def select(sql, name = nil, binds = [])
-        exec_query(sql, name, binds).to_a
+        exec_query(sql, name, binds)
       end
 
       # Returns true, as this adapter supports migrations
@@ -85,8 +91,8 @@ module ActiveRecord
         true
       end
 
-      def new_column(field, default, type, null, collation)
-        Column.new(field, default, type, null, collation)
+      def new_column(field, default, type, null, collation, extra = "")
+        Column.new(field, default, type, null, collation, extra)
       end
 
       # Adds a new index to the table

data/lib/departure.rb

@@ -2,6 +2,7 @@ require 'active_record'
 require 'active_support/all'
 
 require 'departure/version'
+require 'departure/log_sanitizers/password_sanitizer'
 require 'departure/runner'
 require 'departure/cli_generator'
 require 'departure/logger'
@@ -9,6 +10,7 @@ require 'departure/null_logger'
 require 'departure/logger_factory'
 require 'departure/configuration'
 require 'departure/errors'
+require 'departure/command'
 
 require 'departure/railtie' if defined?(Rails)
 
@@ -28,6 +30,21 @@ module Departure
   # Hooks Percona Migrator into Rails migrations by replacing the configured
   # database adapter
   def self.load
+    ActiveRecord::Migrator.instance_eval do
+      class << self
+        alias_method(:original_migrate, :migrate)
+      end
+
+      # Checks whether arguments are being passed through PERCONA_ARGS when running
+      # the db:migrate rake task
+      #
+      # @raise [ArgumentsNotSupported] if PERCONA_ARGS has any value
+      def migrate(migrations_paths, target_version = nil, &block)
+        raise ArgumentsNotSupported if ENV['PERCONA_ARGS'].present?
+        original_migrate(migrations_paths, target_version, &block)
+      end
+    end
+
     ActiveRecord::Migration.class_eval do
       alias_method :original_migrate, :migrate
 
@@ -55,10 +72,9 @@ module Departure
       # Make all connections in the connection pool to use PerconaAdapter
       # instead of the current adapter.
       def reconnect_with_percona
-        connection_config = ActiveRecord::Base.connection_config
-        ActiveRecord::Base.establish_connection(
-          connection_config.merge(adapter: 'percona')
-        )
+        connection_config = ActiveRecord::Base
+          .connection_config.merge(adapter: 'percona')
+        ActiveRecord::Base.establish_connection(connection_config)
       end
     end
   end

data/lib/departure/cli_generator.rb

@@ -1,49 +1,36 @@
+require 'departure/dsn'
+require 'departure/option'
 require 'departure/alter_argument'
+require 'departure/connection_details'
+require 'departure/user_options'
 
 module Departure
 
-  # Represents the 'DSN' argument of Percona's pt-online-schema-change
-  # See https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html#dsn-options
-  class DSN
-
-    # Constructor
-    #
-    # @param database [String, Symbol]
-    # @param table_name [String, Symbol]
-    def initialize(database, table_name)
-      @database = database
-      @table_name = table_name
-    end
-
-    # Returns the pt-online-schema-change DSN string. See
-    # https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html#dsn-options
-    def to_s
-      "D=#{database},t=#{table_name}"
-    end
-
-    private
-
-    attr_reader :table_name, :database
-  end
-
   # Generates the equivalent Percona's pt-online-schema-change command to the
   # given SQL statement
-  class CliGenerator # Command
-    BASE_COMMAND = 'pt-online-schema-change'
-    BASE_OPTIONS = %w(
-      --execute
-      --statistics
-      --recursion-method=none
-      --alter-foreign-keys-method=auto
-    )
-
-    # Constructor
+  #
+  # --no-check-alter is used to allow running CHANGE COLUMN statements. For
+  #   more details, check: www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html#cmdoption-pt-online-schema-change--[no]check-alter
+  #
+  class CliGenerator
+    COMMAND_NAME = 'pt-online-schema-change'.freeze
+    DEFAULT_OPTIONS = Set.new(
+      [
+        Option.new('execute'),
+        Option.new('statistics'),
+        Option.new('alter-foreign-keys-method', 'auto'),
+        Option.new('no-check-alter')
+      ]
+    ).freeze
+
+    # TODO: Better doc.
+    #
+    # Constructor. Specify any arguments to pass to pt-online-schema-change
+    # passing the PERCONA_ARGS env var when executing the migration
     #
     # @param connection_data [Hash]
-    def initialize(connection_data)
-      @connection_data = connection_data
-      init_base_command
-      add_connection_details
+    def initialize(connection_details)
+      @connection_details = connection_details
     end
 
     # Generates the percona command. Fills all the connection credentials from
@@ -57,9 +44,9 @@ module Departure
     # @return [String]
     def generate(table_name, statement)
       alter_argument = AlterArgument.new(statement)
-      dsn = DSN.new(database, table_name)
+      dsn = DSN.new(connection_details.database, table_name)
 
-      "#{self} #{dsn} #{alter_argument}"
+      "#{command} #{all_options} #{dsn} #{alter_argument}"
     end
 
     # Generates the percona command for a raw MySQL statement. Fills all the
@@ -72,63 +59,27 @@ module Departure
     # @return [String]
     def parse_statement(statement)
       alter_argument = AlterArgument.new(statement)
-      dsn = DSN.new(database, alter_argument.table_name)
+      dsn = DSN.new(connection_details.database, alter_argument.table_name)
 
-      "#{self} #{dsn} #{alter_argument}"
+      "#{command} #{all_options} #{dsn} #{alter_argument}"
     end
 
     private
 
-    attr_reader :connection_data
-
-    # Sets up the command with its options
-    def init_base_command
-      @command = [BASE_COMMAND, BASE_OPTIONS.join(' ')]
-    end
-
-    # Adds the host, user and password, if present, to the command
-    def add_connection_details
-      @command.push("-h #{host}")
-      @command.push("-u #{user}")
-      @command.push("-p #{password}") if password.present?
-    end
-
-    # Returns the command as a string that can be executed in a shell
-    #
-    # @return [String]
-    def to_s
-      @command.join(' ')
-    end
-
-    # Returns the database host name, defaulting to localhost. If PERCONA_DB_HOST
-    # is passed its value will be used instead
-    #
-    # @return [String]
-    def host
-      ENV['PERCONA_DB_HOST'] || connection_data[:host] || 'localhost'
-    end
+    attr_reader :connection_details
 
-    # Returns the database user. If PERCONA_DB_USER is passed its value will be
-    # used instead
-    #
-    # @return [String]
-    def user
-      ENV['PERCONA_DB_USER'] || connection_data[:username]
+    def command
+      "#{COMMAND_NAME} #{connection_details}"
     end
 
-    # Returns the database user's password. If PERCONA_DB_PASSWORD is passed its
-    # value will be used instead
+    # Returns all the arguments to execute pt-online-schema-change with
     #
     # @return [String]
-    def password
-      ENV['PERCONA_DB_PASSWORD'] || connection_data[:password]
-    end
-
-    # TODO: Doesn't the abstract adapter already handle this somehow?
-    # Returns the database name. If PERCONA_DB_NAME is passed its value will be
-    # used instead
-    def database
-      ENV['PERCONA_DB_NAME'] || connection_data[:database]
+    def all_options
+      env_variable_options = UserOptions.new
+      global_configuration_options = UserOptions.new(Departure.configuration.global_percona_args)
+      options = env_variable_options.merge(global_configuration_options).merge(DEFAULT_OPTIONS)
+      options.to_a.join(' ')
     end
   end
 end

data/lib/departure/command.rb

@@ -0,0 +1,96 @@
+module Departure
+  # Executes the given command returning it's status and errors
+  class Command
+    COMMAND_NOT_FOUND = 127
+
+    # Constructor
+    #
+    # @param command_line [String]
+    # @param error_log_path [String]
+    # @param logger [#write_no_newline]
+    def initialize(command_line, error_log_path, logger)
+      @command_line = command_line
+      @error_log_path = error_log_path
+      @logger = logger
+    end
+
+    # Executes the command returning its status. It also prints its stdout to
+    # the logger and its stderr to the file specified in error_log_path.
+    #
+    # @raise [NoStatusError] if the spawned process' status can't be retrieved
+    # @raise [SignalError] if the spawned process received a signal
+    # @raise [CommandNotFoundError] if pt-online-schema-change can't be found
+    #
+    # @return [Process::Status]
+    def run
+      log_started
+
+      run_in_process
+
+      log_finished
+
+      validate_status!
+      status
+    end
+
+    private
+
+    attr_reader :command_line, :error_log_path, :logger, :status
+
+    # Runs the command in a separate process, capturing its stdout and
+    # execution status
+    def run_in_process
+      Open3.popen3(full_command) do |_stdin, stdout, _stderr, waith_thr|
+        begin
+          loop do
+            IO.select([stdout])
+            data = stdout.read_nonblock(8)
+            logger.write_no_newline(data)
+          end
+        rescue EOFError
+          # noop
+        ensure
+          @status = waith_thr.value
+        end
+      end
+    end
+
+    # Builds the actual command including stderr redirection to the specified
+    # log file
+    #
+    # @return [String]
+    def full_command
+      "#{command_line} 2> #{error_log_path}"
+    end
+
+    # Validates the status of the execution
+    #
+    # @raise [NoStatusError] if the spawned process' status can't be retrieved
+    # @raise [SignalError] if the spawned process received a signal
+    # @raise [CommandNotFoundError] if pt-online-schema-change can't be found
+    def validate_status!
+      raise SignalError.new(status) if status.signaled?
+      raise CommandNotFoundError if status.exitstatus == COMMAND_NOT_FOUND
+      raise Error, error_message unless status.success?
+    end
+
+    # Returns the error message that appeared in the process' stderr
+    #
+    # @return [String]
+    def error_message
+      File.read(error_log_path)
+    end
+
+    # Logs when the execution started
+    def log_started
+      logger.write("\n")
+      logger.say("Running #{command_line}\n\n", true)
+    end
+
+    # Prints a line break to keep the logs separate from the execution time
+    # print by the migration
+    def log_finished
+      logger.write("\n")
+    end
+  end
+end

data/lib/departure/configuration.rb

@@ -1,10 +1,11 @@
 module Departure
   class Configuration
-    attr_accessor :tmp_path
+    attr_accessor :tmp_path, :global_percona_args
 
     def initialize
       @tmp_path = '.'.freeze
       @error_log_filename = 'departure_error.log'.freeze
+      @global_percona_args = nil
     end
 
     def error_log_path

data/lib/departure/connection_details.rb

@@ -0,0 +1,70 @@
+module Departure
+  # Holds the parameters of the DB connection and formats them to string
+  class ConnectionDetails
+
+    # Constructor
+    #
+    # @param [Hash] connection parametes as used in #establish_conneciton
+    def initialize(connection_data)
+      @connection_data = connection_data
+    end
+
+    # Returns the details formatted as an string to be used with
+    # pt-online-schema-change. It follows the mysql client's format.
+    #
+    # @return [String]
+    def to_s
+      @to_s ||= "-h #{host} -u #{user} #{password_argument}"
+    end
+
+    # TODO: Doesn't the abstract adapter already handle this somehow?
+    # Returns the database name. If PERCONA_DB_NAME is passed its value will be
+    # used instead
+    #
+    # Returns the database name
+    #
+    # @return [String]
+    def database
+      ENV.fetch('PERCONA_DB_NAME', connection_data[:database])
+    end
+
+    # Returns the password fragment of the details string if a password is passed
+    #
+    # @return [String]
+    def password_argument
+      if password.present?
+        "-p #{password}"
+      else
+        ''
+      end
+    end
+
+    private
+
+    attr_reader :connection_data
+
+    # Returns the database host name, defaulting to localhost. If PERCONA_DB_HOST
+    # is passed its value will be used instead
+    #
+    # @return [String]
+    def host
+      ENV.fetch('PERCONA_DB_HOST', connection_data[:host]) || 'localhost'
+    end
+
+    # Returns the database user. If PERCONA_DB_USER is passed its value will be
+    # used instead
+    #
+    # @return [String]
+    def user
+      ENV.fetch('PERCONA_DB_USER', connection_data[:username])
+    end
+
+    # Returns the database user's password. If PERCONA_DB_PASSWORD is passed its
+    # value will be used instead
+    #
+    # @return [String]
+    def password
+      ENV.fetch('PERCONA_DB_PASSWORD', connection_data[:password])
+    end
+  end
+end

data/lib/departure/dsn.rb

@@ -0,0 +1,26 @@
+module Departure
+
+  # Represents the 'DSN' argument of Percona's pt-online-schema-change
+  # See https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html#dsn-options
+  class DSN
+
+    # Constructor
+    #
+    # @param database [String, Symbol]
+    # @param table_name [String, Symbol]
+    def initialize(database, table_name)
+      @database = database
+      @table_name = table_name
+    end
+
+    # Returns the pt-online-schema-change DSN string. See
+    # https://www.percona.com/doc/percona-toolkit/2.0/pt-online-schema-change.html#dsn-options
+    def to_s
+      "D=#{database},t=#{table_name}"
+    end
+
+    private
+
+    attr_reader :table_name, :database
+  end
+end

data/lib/departure/errors.rb

@@ -32,4 +32,8 @@ module Departure
       'Please install pt-online-schema-change. Check: https://www.percona.com/doc/percona-toolkit for further details'
     end
   end
+
+  # Used to prevent running the db:migrate rake task when providing arguments
+  # through PERCONA_ARGS env var
+  class ArgumentsNotSupported < Error; end
 end

data/lib/departure/log_sanitizers/password_sanitizer.rb

@@ -0,0 +1,21 @@
+module Departure
+  module LogSanitizers
+    class PasswordSanitizer
+      PASSWORD_REPLACEMENT = '[filtered_password]'
+
+      delegate :password_argument, to: :connection_details
+
+      def initialize(connection_details)
+        @connection_details = connection_details
+      end
+
+      def execute(log_statement)
+        return log_statement if password_argument.blank?
+        log_statement.gsub(password_argument, PASSWORD_REPLACEMENT)
+      end
+
+      private
+      attr_accessor :connection_details
+    end
+  end
+end

data/lib/departure/logger.rb

@@ -5,6 +5,10 @@ module Departure
   # seen from the connection adapter.
   class Logger
 
+    def initialize(sanitizers)
+      @sanitizers = sanitizers
+    end
+
     # Outputs the message through the stdout, following the
     # ActiveRecord::Migration log format
     #
@@ -18,14 +22,22 @@ module Departure
     #
     # @param text [String]
     def write(text = '')
-      puts(text)
+      puts(sanitize(text))
     end
 
     # Outputs the text through the stdout without adding a new line at the end
     #
     # @param text [String]
     def write_no_newline(text)
-      print(text)
+      print(sanitize(text))
+    end
+
+    private
+
+    attr_accessor :sanitizers
+
+    def sanitize(text)
+      sanitizers.inject(text) { |memo, sanitizer| sanitizer.execute(memo) }
     end
   end
 end

data/lib/departure/logger_factory.rb

@@ -6,9 +6,9 @@ module Departure
     #
     # @param verbose [Boolean]
     # @return [#say, #write]
-    def self.build(verbose: true)
+    def self.build(sanitizers: [], verbose: true)
       if verbose
-        Departure::Logger.new
+        Departure::Logger.new(sanitizers)
       else
         Departure::NullLogger.new
       end

data/lib/departure/option.rb

@@ -0,0 +1,63 @@
+module Departure
+  class Option
+    attr_reader :name, :value
+
+    # Builds an instance by parsing its name and value out of the given string.
+    # Note the string must conform to "--<arg>=<value>" format.
+    #
+    # @param string [String]
+    # @return [Option]
+    def self.from_string(string)
+      pair = string.split('=')
+      name = pair[0][2..-1]
+      value = pair[1]
+
+      new(name, value)
+    end
+
+    # Constructor
+    #
+    # @param name [String]
+    # @param optional value [String]
+    def initialize(name, value = nil)
+      @name = name
+      @value = value
+    end
+
+    # Compares two options
+    #
+    # @param [Option]
+    # @return [Boolean]
+    def ==(another_option)
+      name == another_option.name
+    end
+    alias :eql? :==
+
+    # Returns the option's hash
+    #
+    # @return [Fixnum]
+    def hash
+      name.hash
+    end
+
+    # Returns the option as string following the "--<name>=<value>" format
+    #
+    # @return [String]
+    def to_s
+      "--#{name}#{value_as_string}"
+    end
+
+    private
+
+    # Returns the value fragment of the option string if any value is specified
+    #
+    # @return [String]
+    def value_as_string
+      if value.nil?
+        ''
+      else
+        "=#{value}"
+      end
+    end
+  end
+end

data/lib/departure/runner.rb

@@ -5,7 +5,6 @@ module Departure
   # It executes pt-online-schema-change commands in a new process and gets its
   # output and status
   class Runner
-    COMMAND_NOT_FOUND = 127
 
     # Constructor
     #
@@ -17,8 +16,7 @@ module Departure
       @logger = logger
       @cli_generator = cli_generator
       @mysql_adapter = mysql_adapter
-      @status = nil
-      @config = config
+      @error_log_path = config.error_log_path
     end
 
     # Executes the passed sql statement using pt-online-schema-change for ALTER
@@ -27,8 +25,8 @@ module Departure
     # @param sql [String]
     def query(sql)
       if alter_statement?(sql)
-        command = cli_generator.parse_statement(sql)
-        execute(command)
+        command_line = cli_generator.parse_statement(sql)
+        execute(command_line)
       else
         mysql_adapter.execute(sql)
       end
@@ -45,18 +43,15 @@ module Departure
     # TODO: rename it so we don't confuse it with AR's #execute
     # Runs and logs the given command
     #
-    # @param command [String]
+    # @param command_line [String]
     # @return [Boolean]
-    def execute(command)
-      @command = command
-      logging { run_command }
-      validate_status
-      status
+    def execute(command_line)
+      Command.new(command_line, error_log_path, logger).run
     end
 
     private
 
-    attr_reader :command, :logger, :status, :cli_generator, :mysql_adapter, :config
+    attr_reader :logger, :cli_generator, :mysql_adapter, :error_log_path
 
     # Checks whether the sql statement is an ALTER TABLE
     #
@@ -65,66 +60,5 @@ module Departure
     def alter_statement?(sql)
       sql =~ /\Aalter table/i
     end
-
-    # Logs the start and end of the execution
-    #
-    # @yield
-    def logging
-      log_started
-      yield
-      log_finished
-    end
-
-    # Logs when the execution started
-    def log_started
-      logger.write("\n")
-      logger.say("Running #{command}\n\n", true)
-    end
-
-    # Executes the command and prints its output to the stdout
-    def run_command
-      Open3.popen3("#{command} 2> #{error_log_path}") do |_stdin, stdout, _stderr, waith_thr|
-        begin
-          loop do
-            IO.select([stdout])
-            data = stdout.read_nonblock(8)
-            logger.write_no_newline(data)
-          end
-        rescue EOFError
-          # noop
-        ensure
-          @status = waith_thr.value
-        end
-      end
-    end
-
-    # Validates the status of the execution
-    #
-    # @raise [NoStatusError] if the spawned process' status can't be retrieved
-    # @raise [SignalError] if the spawned process received a signal
-    # @raise [CommandNotFoundError] if pt-online-schema-change can't be found
-    def validate_status
-      raise SignalError.new(status) if status.signaled?
-      raise CommandNotFoundError if status.exitstatus == COMMAND_NOT_FOUND
-      raise Error, error_message unless status.success?
-    end
-
-    # Prints a line break to keep the logs separate from the execution time
-    # print by the migration
-    def log_finished
-      logger.write("\n")
-    end
-
-    # The path where the percona toolkit stderr will be written
-    #
-    # @return [String]
-    def error_log_path
-      config.error_log_path
-    end
-
-    # @return [String]
-    def error_message
-      File.read(error_log_path)
-    end
   end
 end

data/lib/departure/user_options.rb

@@ -0,0 +1,44 @@
+module Departure
+  # Encapsulates the pt-online-schema-change options defined by the user
+  class UserOptions
+    delegate :each, :merge, to: :to_set
+
+    # Constructor
+    #
+    # @param arguments [String]
+    def initialize(arguments = ENV['PERCONA_ARGS'])
+      @arguments = arguments
+    end
+
+    private
+
+    attr_reader :arguments
+
+    # Returns the arguments the user defined but without duplicates
+    #
+    # @return [Set]
+    def to_set
+      Set.new(user_options)
+    end
+
+    # Returns Option instances from the arguments the user specified, if any
+    #
+    # @return [Array]
+    def user_options
+      if arguments
+        build_options
+      else
+        []
+      end
+    end
+
+    # Builds Option instances from the user arguments
+    #
+    # @return [Array<Option>]
+    def build_options
+      arguments.split(' ').map do |argument|
+        Option.from_string(argument)
+      end
+    end
+  end
+end

data/lib/departure/version.rb

@@ -1,3 +1,3 @@
 module Departure
-  VERSION = '1.0.1'.freeze
+  VERSION = '2.0.0'.freeze
 end

data/test_database.rb

@@ -19,7 +19,7 @@ class TestDatabase
     drop_and_create_schema_migrations_table
   end
 
-  # Creates the test database and the comments table in it.
+  # Creates the #{database} database and the comments table in it.
   # Before, it drops both if they already exist
   def setup_test_database
     drop_and_create_test_database

metadata

@@ -1,17 +1,18 @@
 --- !ruby/object:Gem::Specification
 name: departure
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 2.0.0
 platform: ruby
 authors:
 - Ilya Zayats
 - Pau Pérez
 - Fran Casas
 - Jorge Morante
+- Adrian Serafin
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-04-19 00:00:00.000000000 Z
+date: 2017-04-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -19,26 +20,26 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.2.22
+        version: 4.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.2.22
+        version: 4.0.0
 - !ruby/object:Gem::Dependency
   name: mysql2
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
         version: 0.3.20
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
         version: 0.3.20
 - !ruby/object:Gem::Dependency
@@ -123,6 +124,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 8.2.1
+- !ruby/object:Gem::Dependency
+  name: climate_control
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.0.3
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.0.3
 description: Execute your ActiveRecord migrations with Percona's pt-online-schema-change.
   Formerly known as Percona Migrator.
 email:
@@ -130,6 +145,7 @@ email:
 - pau.perez@redbooth.com
 - fran.casas@redbooth.com
 - jorge.morante@redbooth.com
+- adrian@softmad.pl
 executables: []
 extensions: []
 extra_rdoc_files: []
@@ -142,23 +158,32 @@ files:
 - Gemfile
 - LICENSE.txt
 - README.md
+- RELEASING.md
 - Rakefile
 - bin/console
+- bin/rspec
 - bin/setup
 - config.yml
 - configuration.rb
 - departure.gemspec
+- departure_error.log
 - lib/active_record/connection_adapters/percona_adapter.rb
 - lib/departure.rb
 - lib/departure/alter_argument.rb
 - lib/departure/cli_generator.rb
+- lib/departure/command.rb
 - lib/departure/configuration.rb
+- lib/departure/connection_details.rb
+- lib/departure/dsn.rb
 - lib/departure/errors.rb
+- lib/departure/log_sanitizers/password_sanitizer.rb
 - lib/departure/logger.rb
 - lib/departure/logger_factory.rb
 - lib/departure/null_logger.rb
+- lib/departure/option.rb
 - lib/departure/railtie.rb
 - lib/departure/runner.rb
+- lib/departure/user_options.rb
 - lib/departure/version.rb
 - lib/lhm.rb
 - lib/lhm/adapter.rb