lhm 1.0.3 → 1.1.0

data/.travis.yml

@@ -1,16 +1,10 @@
+language: ruby
 before_script:
   - "mysql -e 'create database lhm;'"
 rvm:
   - 1.8.7
   - 1.9.3
 gemfile:
-  - gemfiles/ar-2.3.gemfile
-  - gemfiles/ar-3.1.gemfile
-matrix:
-  exclude:
-    - rvm: 1.8.7
-      gemfile: gemfiles/ar-2.3.gemfile
-branches:
-  only:
-    - master
-    - stable
+  - gemfiles/ar-2.3_mysql.gemfile
+  - gemfiles/ar-3.2_mysql.gemfile
+  - gemfiles/ar-3.2_mysql2.gemfile

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+# 1.1.0 (April 29, 2012)
+
+* Add option to specify custom index name
+* Add mysql2 compatibility
+* Add AtomicSwitcher
+
 # 1.0.3 (February 23, 2012)
 
 * Improve change_column

data/README.md

@@ -1,6 +1,4 @@
-# Large Hadron Migrator [![Build Status](https://secure.travis-ci.org/soundcloud/large-hadron-migrator.png)](http://travis-ci.org/soundcloud/large-hadron-migrator)
-
-Update: There is currently [An issue](https://github.com/soundcloud/large-hadron-migrator/issues/11) with the migration. Fix coming up.
+# Large Hadron Migrator [![Build Status](https://secure.travis-ci.org/soundcloud/large-hadron-migrator.png)][4]
 
 Rails style database migrations are a useful way to evolve your data schema in
 an agile manner. Most Rails projects start like this, and at first, making
@@ -22,7 +20,7 @@ table. The InnoDB Plugin provides facilities for online index creation, which
 is great if you are using this engine, but only solves half the problem.
 
 At SoundCloud we started having migration pains quite a while ago, and after
-looking around for third party solutions [0] [1] [2], we decided to create our
+looking around for third party solutions, we decided to create our
 own. We called it Large Hadron Migrator, and it is a gem for online
 ActiveRecord migrations.
 
@@ -33,62 +31,100 @@ ActiveRecord migrations.
 ## The idea
 
 The basic idea is to perform the migration online while the system is live,
-without locking the table. Similar to OAK (online alter table) [2] and the
-facebook tool [0], we use a copy table, triggers and a journal table.
+without locking the table. In contrast to [OAK][0] and the
+[facebook tool][1], we only use a copy table and triggers.
 
 The Large Hadron is a test driven Ruby solution which can easily be dropped
 into an ActiveRecord migration. It presumes a single auto incremented
 numerical primary key called id as per the Rails convention. Unlike the
-twitter solution [1], it does not require the presence of an indexed
+[twitter solution][2], it does not require the presence of an indexed
 `updated_at` column.
 
+## Requirements
+
+Lhm currently only works with MySQL databases and requires an established
+ActiveRecord connection.
+
+It is compatible and [continuously tested][4] with Ruby 1.8.7 and Ruby 1.9.x,
+ActiveRecord 2.3.x and 3.x as well as mysql and mysql2 adapters.
+
+## Installation
+
+Install it via `gem install lhm` or add `gem "lhm"` to your Gemfile.
+
 ## Usage
 
-You can invoke Lhm directly from a plain ruby file after connecting active
-record to your mysql instance:
+You can invoke Lhm directly from a plain ruby file after connecting ActiveRecord
+to your mysql instance:
 
-    require 'lhm'
+```ruby
+require 'lhm'
 
-    ActiveRecord::Base.establish_connection(
-      :adapter => 'mysql',
-      :host => '127.0.0.1',
-      :database => 'lhm'
-    )
+ActiveRecord::Base.establish_connection(
+  :adapter => 'mysql',
+  :host => '127.0.0.1',
+  :database => 'lhm'
+)
 
-    Lhm.change_table(:users) do |m|
-      m.add_column(:arbitrary, "INT(12)")
-      m.add_index([:arbitrary, :created_at])
-      m.ddl("alter table %s add column flag tinyint(1)" % m.name)
-    end
+Lhm.change_table :users do |m|
+  m.add_column :arbitrary, "INT(12)"
+  m.add_index  [:arbitrary_id, :created_at]
+  m.ddl("alter table %s add column flag tinyint(1)" % m.name)
+end
+```
 
 To use Lhm from an ActiveRecord::Migration in a Rails project, add it to your
 Gemfile, then invoke as follows:
 
-    class MigrateUsers < ActiveRecord::Migration
-
-      def self.up
-        Lhm.change_table(:users) do |m|
-          m.add_column(:arbitrary, "INT(12)")
-          m.add_index([:arbitrary, :created_at])
-          m.ddl("alter table %s add column flag tinyint(1)" % m.name)
-        end
-      end
-
-      def self.down
-        Lhm.change_table(:users) do |m|
-          m.remove_index([:arbitrary, :created_at])
-          m.remove_column(:arbitrary)
-        end
-      end
+```ruby
+require 'lhm'
+
+class MigrateUsers < ActiveRecord::Migration
+  def self.up
+    Lhm.change_table :users do |m|
+      m.add_column :arbitrary, "INT(12)"
+      m.add_index  [:arbitrary_id, :created_at]
+      m.ddl("alter table %s add column flag tinyint(1)" % m.name)
+    end
+  end
+
+  def self.down
+    Lhm.change_table :users do |m|
+      m.remove_index  [:arbitrary_id, :created_at]
+      m.remove_column :arbitrary)
     end
+  end
+end
+```
+
+## Table rename strategies
+
+There are two different table rename strategies available: LockedSwitcher and
+AtomicSwitcher.
+
+For all setups which use replication and a MySQL version
+affected by the the [binlog bug #39675](http://bugs.mysql.com/bug.php?id=39675),
+we recommend the LockedSwitcher strategy to avoid replication issues. This
+strategy locks the table being migrated and issues two ALTER TABLE statements.
+The AtomicSwitcher uses a single atomic RENAME TABLE query and should be favored
+in setups which do not suffer from the mentioned replication bug.
+
+Lhm chooses the strategy automatically based on the used MySQL server version,
+but you can override the behavior with an option:
+
+```ruby
+Lhm.change_table :users, :atomic_switch => true do |m|
+  # ...
+end
+```
 
 ## Contributing
 
 We'll check out your contribution if you:
 
-- Provide a comprehensive suite of tests for your fork.
-- Have a clear and documented rationale for your changes.
-- Package these up in a pull request.
+  * Provide a comprehensive suite of tests for your fork.
+  * Have a clear and documented rationale for your changes.
+  * Package these up in a pull request.
 
 We'll do our best to help you out with any contribution issues you may have.
 
@@ -96,9 +132,15 @@ We'll do our best to help you out with any contribution issues you may have.
 
 The license is included as LICENSE in this directory.
 
-## Footnotes
+## Similar solutions
 
-[0]: http://www.facebook.com/note.php?note\_id=430801045932 "Facebook"
-[1]: https://github.com/freels/table\_migrator "Twitter"
-[2]: http://openarkkit.googlecode.com "OAK online alter table"
+  * [OAK: online alter table][0]
+  * [Facebook][1]
+  * [Twitter][2]
+  * [pt-online-schema-change][3]
 
+[0]: http://openarkkit.googlecode.com
+[1]: http://www.facebook.com/note.php?note\_id=430801045932
+[2]: https://github.com/freels/table_migrator
+[3]: http://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html
+[4]: http://travis-ci.org/soundcloud/large-hadron-migrator

data/bin/lhm-test-all.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+for gemfile in gemfiles/*.gemfile
+do
+  if !(BUNDLE_GEMFILE=$gemfile bundle install &&
+       BUNDLE_GEMFILE=$gemfile bundle exec rake)
+  then
+    exit 1
+  fi
+done

data/gemfiles/{ar-2.3.gemfile → ar-2.3_mysql.gemfile}

@@ -1,4 +1,5 @@
-source "http://rubygems.org"
+source :rubygems
 
+gem "mysql", "~> 2.8.1"
 gem "activerecord", "~> 2.3.14"
 gemspec :path=>"../"

data/gemfiles/ar-3.2_mysql.gemfile

@@ -0,0 +1,5 @@
+source :rubygems
+
+gem "mysql", "~> 2.8.1"
+gem "activerecord", "~> 3.2.2"
+gemspec :path=>"../"

data/gemfiles/ar-3.2_mysql2.gemfile

@@ -0,0 +1,5 @@
+source :rubygems
+
+gem "mysql2", "~> 0.3.11"
+gem "activerecord", "~> 3.2.2"
+gemspec :path=>"../"

data/lhm.gemspec

@@ -19,8 +19,6 @@ Gem::Specification.new do |s|
   s.require_paths = ["lib"]
   s.executables   = ["lhm-kill-queue"]
 
-  # this should be a real dependency, but we're using a different gem in our code
-  s.add_development_dependency "mysql", "~> 2.8.1"
   s.add_development_dependency "minitest", "= 2.10.0"
   s.add_development_dependency "rake"
 

data/lib/lhm.rb

@@ -21,22 +21,25 @@ module Lhm
   # Alters a table with the changes described in the block
   #
   # @param [String, Symbol] table_name Name of the table
-  # @param [Hash] chunk_options Optional options to alter the chunk behavior
-  # @option chunk_options [Fixnum] :stride
+  # @param [Hash] options Optional options to alter the chunk / switch behavior
+  # @option options [Fixnum] :stride
   #   Size of a chunk (defaults to: 40,000)
-  # @option chunk_options [Fixnum] :throttle
+  # @option options [Fixnum] :throttle
   #   Time to wait between chunks in milliseconds (defaults to: 100)
+  # @option options [Boolean] :atomic_switch
+  #   Use atomic switch to rename tables (defaults to: true)
+  #   If using a version of mysql affected by atomic switch bug, LHM forces user
+  #   to set this option (see SqlHelper#supports_atomic_switch?)
   # @yield [Migrator] Yielded Migrator object records the changes
   # @return [Boolean] Returns true if the migration finishes
   # @raise [Error] Raises Lhm::Error in case of a error and aborts the migration
-  def self.change_table(table_name, chunk_options = {}, &block)
+  def self.change_table(table_name, options = {}, &block)
     connection = ActiveRecord::Base.connection
     origin = Table.parse(table_name, connection)
     invoker = Invoker.new(origin, connection)
     block.call(invoker.migrator)
-    invoker.run(chunk_options)
+    invoker.run(options)
 
     true
   end
 end
-

data/lib/lhm/atomic_switcher.rb

@@ -0,0 +1,49 @@
+# Copyright (c) 2011, SoundCloud Ltd., Rany Keddo, Tobias Bielohlawek, Tobias
+# Schmidt
+
+require 'lhm/command'
+require 'lhm/migration'
+require 'lhm/sql_helper'
+
+module Lhm
+  # Switches origin with destination table using an atomic rename.
+  #
+  # It should only be used if the MySQL server version is not affected by the
+  # bin log affecting bug #39675. This can be verified using
+  # Lhm::SqlHelper.supports_atomic_switch?.
+  class AtomicSwitcher
+    include Command
+    include SqlHelper
+
+    attr_reader :connection
+
+    def initialize(migration, connection = nil)
+      @migration = migration
+      @connection = connection
+      @origin = migration.origin
+      @destination = migration.destination
+    end
+
+    def statements
+      atomic_switch
+    end
+
+    def atomic_switch
+      [
+        "rename table `#{ @origin.name }` to `#{ @migration.archive_name }`, " +
+        "`#{ @destination.name }` to `#{ @origin.name }`"
+      ]
+    end
+
+    def validate
+      unless table?(@origin.name) && table?(@destination.name)
+        error "`#{ @origin.name }` and `#{ @destination.name }` must exist"
+      end
+    end
+
+  private
+    def execute
+      sql statements
+    end
+  end
+end

data/lib/lhm/chunker.rb

@@ -91,6 +91,7 @@ module Lhm
 
         print "."
       end
+      print "\n"
     end
   end
 end

data/lib/lhm/invoker.rb

@@ -3,6 +3,7 @@
 
 require 'lhm/chunker'
 require 'lhm/entangler'
+require 'lhm/atomic_switcher'
 require 'lhm/locked_switcher'
 require 'lhm/migrator'
 
@@ -13,19 +14,35 @@ module Lhm
   # Once the origin and destination tables have converged, origin is archived
   # and replaced by destination.
   class Invoker
-    attr_reader :migrator
+    include SqlHelper
+
+    attr_reader :migrator, :connection
 
     def initialize(origin, connection)
       @connection = connection
       @migrator = Migrator.new(origin, connection)
     end
 
-    def run(chunk_options = {})
+    def run(options = {})
+      if !options.include?(:atomic_switch)
+        if supports_atomic_switch?
+          options[:atomic_switch] = true
+        else
+          raise Error.new(
+            "Using mysql #{version_string}. You must explicitly set " +
+            "options[:atomic_switch] (re SqlHelper#supports_atomic_switch?)")
+        end
+      end
+
       migration = @migrator.run
 
       Entangler.new(migration, @connection).run do
-        Chunker.new(migration, @connection, chunk_options).run
-        LockedSwitcher.new(migration, @connection).run
+        Chunker.new(migration, @connection, options).run
+        if options[:atomic_switch]
+          AtomicSwitcher.new(migration, @connection).run
+        else
+          LockedSwitcher.new(migration, @connection).run
+        end
       end
     end
   end

data/lib/lhm/locked_switcher.rb

@@ -6,11 +6,7 @@ require 'lhm/migration'
 require 'lhm/sql_helper'
 
 module Lhm
-  # Switches origin with destination table with a write lock. Use this as
-  # a safe alternative to rename, which can cause slave inconsistencies:
-  #
-  #   http://bugs.mysql.com/bug.php?id=39675
-  #
+  # Switches origin with destination table nonatomically using a locked write.
   # LockedSwitcher adopts the Facebook strategy, with the following caveat:
   #
   #   "Since alter table causes an implicit commit in innodb, innodb locks get

data/lib/lhm/migrator.rb

@@ -95,8 +95,10 @@ module Lhm
     # @param [String, Symbol, Array<String, Symbol>] columns
     #   A column name given as String or Symbol. An Array of Strings or Symbols
     #   for compound indexes. It's possible to pass a length limit.
-    def add_index(columns)
-      ddl(index_ddl(columns))
+    # @param [String, Symbol] index_name
+    #   Optional name of the index to be created
+    def add_index(columns, index_name = nil)
+      ddl(index_ddl(columns, false, index_name))
     end
 
     # Add a unique index to a table
@@ -112,8 +114,10 @@ module Lhm
     # @param [String, Symbol, Array<String, Symbol>] columns
     #   A column name given as String or Symbol. An Array of Strings or Symbols
     #   for compound indexes. It's possible to pass a length limit.
-    def add_unique_index(columns)
-      ddl(index_ddl(columns, :unique))
+    # @param [String, Symbol] index_name
+    #   Optional name of the index to be created
+    def add_unique_index(columns, index_name = nil)
+      ddl(index_ddl(columns, true, index_name))
     end
 
     # Remove an index from a table
@@ -128,8 +132,11 @@ module Lhm
     # @param [String, Symbol, Array<String, Symbol>] columns
     #   A column name given as String or Symbol. An Array of Strings or Symbols
     #   for compound indexes.
-    def remove_index(columns)
-      ddl("drop index `%s` on `%s`" % [idx_name(@origin.name, columns), @name])
+    # @param [String, Symbol] index_name
+    #   Optional name of the index to be removed
+    def remove_index(columns, index_name = nil)
+      index_name ||= idx_name(@origin.name, columns)
+      ddl("drop index `%s` on `%s`" % [index_name, @name])
     end
 
   private
@@ -167,9 +174,10 @@ module Lhm
       Table.parse(@origin.destination_name, connection)
     end
 
-    def index_ddl(cols, unique = nil)
+    def index_ddl(cols, unique = nil, index_name = nil)
       type = unique ? "unique index" : "index"
-      parts = [type, idx_name(@origin.name, cols), @name, idx_spec(cols)]
+      index_name ||= idx_name(@origin.name, cols)
+      parts = [type, index_name, @name, idx_spec(cols)]
       "create %s `%s` on `%s` (%s)" % parts
     end
   end

data/lib/lhm/sql_helper.rb

@@ -28,7 +28,7 @@ module Lhm
       [statements].flatten.each do |statement|
         connection.execute(tagged(statement))
       end
-    rescue ActiveRecord::StatementInvalid, Mysql::Error => e
+    rescue ActiveRecord::StatementInvalid => e
       error e.message
     end
 
@@ -36,10 +36,14 @@ module Lhm
       [statements].flatten.inject(0) do |memo, statement|
         memo += connection.update(tagged(statement))
       end
-    rescue ActiveRecord::StatementInvalid, Mysql::Error => e
+    rescue ActiveRecord::StatementInvalid => e
       error e.message
     end
 
+    def version_string
+      connection.select_one("show variables like 'version'")["Value"]
+    end
+
   private
 
     def tagged(statement)
@@ -51,5 +55,31 @@ module Lhm
         column.to_s.match(/`?([^\(]+)`?(\([^\)]+\))?/).captures
       end
     end
+
+    # Older versions of MySQL contain an atomic rename bug affecting bin
+    # log order. Affected versions extracted from bug report:
+    #
+    #   http://bugs.mysql.com/bug.php?id=39675
+    #
+    # More Info: http://dev.mysql.com/doc/refman/5.5/en/metadata-locking.html
+    def supports_atomic_switch?
+      major, minor, tiny = version_string.split('.').map(&:to_i)
+
+      case major
+      when 4 then return false if minor and minor < 2
+      when 5
+        case minor
+        when 0 then return false if tiny and tiny < 52
+        when 1 then return false
+        when 4 then return false if tiny and tiny < 4
+        when 5 then return false if tiny and tiny < 3
+        end
+      when 6
+        case minor
+        when 0 then return false if tiny and tiny < 11
+        end
+      end
+      return true
+    end
   end
 end

data/lib/lhm/table.rb

@@ -38,7 +38,9 @@ module Lhm
 
       def ddl
         sql = "show create table `#{ @table_name }`"
-        @connection.execute(sql).fetch_row.last
+        specification = nil
+        @connection.execute(sql).each { |row| specification = row.last }
+        specification
       end
 
       def parse

data/lib/lhm/version.rb

@@ -2,5 +2,5 @@
 # Schmidt
 
 module Lhm
-  VERSION = "1.0.3"
+  VERSION = "1.1.0"
 end

data/spec/integration/atomic_switcher_spec.rb

@@ -0,0 +1,42 @@
+# Copyright (c) 2011, SoundCloud Ltd., Rany Keddo, Tobias Bielohlawek, Tobias
+# Schmidt
+
+require File.expand_path(File.dirname(__FILE__)) + '/integration_helper'
+
+require 'lhm/table'
+require 'lhm/migration'
+require 'lhm/atomic_switcher'
+
+describe Lhm::AtomicSwitcher do
+  include IntegrationHelper
+
+  before(:each) { connect_master! }
+
+  describe "switching" do
+    before(:each) do
+      @origin = table_create("origin")
+      @destination = table_create("destination")
+      @migration = Lhm::Migration.new(@origin, @destination)
+    end
+
+    it "rename origin to archive" do
+      switcher = Lhm::AtomicSwitcher.new(@migration, connection)
+      switcher.run
+
+      slave do
+        table_exists?(@origin).must_equal true
+        table_read(@migration.archive_name).columns.keys.must_include "origin"
+      end
+    end
+
+    it "rename destination to origin" do
+      switcher = Lhm::AtomicSwitcher.new(@migration, connection)
+      switcher.run
+
+      slave do
+        table_exists?(@destination).must_equal false
+        table_read(@origin.name).columns.keys.must_include "destination"
+      end
+    end
+  end
+end

data/spec/integration/integration_helper.rb

@@ -4,34 +4,39 @@
 require File.expand_path(File.dirname(__FILE__)) + "/../bootstrap"
 
 require 'active_record'
+begin
+  require 'mysql2'
+rescue LoadError
+  require 'mysql'
+end
 require 'lhm/table'
 require 'lhm/sql_helper'
 
 module IntegrationHelper
-  attr_accessor :connection
-
   #
   # Connectivity
   #
 
+  def connection
+    ActiveRecord::Base.connection
+  end
+
   def connect_master!
-    @connection = connect!(3306)
+    connect!(3306)
   end
 
   def connect_slave!
-    @connection = connect!(3307)
+    connect!(3307)
   end
 
   def connect!(port)
     ActiveRecord::Base.establish_connection(
-      :adapter => 'mysql',
+      :adapter => defined?(Mysql2) ? 'mysql2' : 'mysql',
       :host => '127.0.0.1',
       :database => 'lhm',
       :username => '',
       :port => port
     )
-
-    ActiveRecord::Base.connection
   end
 
   def select_one(*args)
@@ -43,7 +48,17 @@ module IntegrationHelper
   end
 
   def execute(*args)
-    connection.execute(*args)
+    retries = 10
+    begin
+      connection.execute(*args)
+    rescue ActiveRecord::StatementInvalid => e
+      if (retries -= 1) > 0 && e.message =~ /Table '.*?' doesn't exist/
+        sleep 0.1
+        retry
+      else
+        raise
+      end
+    end
   end
 
   def slave(&block)
@@ -99,11 +114,16 @@ module IntegrationHelper
     select_value(query).to_i
   end
 
-  def key?(table_name, cols, type = :non_unique)
-    non_unique = type == :non_unique ? 1 : 0
+  def index_on_columns?(table_name, cols, type = :non_unique)
     key_name = Lhm::SqlHelper.idx_name(table_name, cols)
 
-    !!select_value(%Q<
+    index?(table_name, key_name, type)
+  end
+
+  def index?(table_name, key_name, type = :non_unique)
+    non_unique = type == :non_unique ? 1 : 0
+
+    !!select_one(%Q<
       show indexes in #{ table_name }
      where key_name = '#{ key_name }'
        and non_unique = #{ non_unique }

data/spec/integration/lhm_spec.rb

@@ -16,7 +16,7 @@ describe Lhm do
     end
 
     it "should add a column" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.add_column(:logins, "INT(12) DEFAULT '0'")
       end
 
@@ -32,7 +32,7 @@ describe Lhm do
     it "should copy all rows" do
       23.times { |n| execute("insert into users set reference = '#{ n }'") }
 
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.add_column(:logins, "INT(12) DEFAULT '0'")
       end
 
@@ -42,7 +42,7 @@ describe Lhm do
     end
 
     it "should remove a column" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.remove_column(:comment)
       end
 
@@ -52,47 +52,67 @@ describe Lhm do
     end
 
     it "should add an index" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.add_index([:comment, :created_at])
       end
 
       slave do
-        key?(:users, [:comment, :created_at]).must_equal(true)
+        index_on_columns?(:users, [:comment, :created_at]).must_equal(true)
+      end
+    end
+
+    it "should add an index with a custom name" do
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
+        t.add_index([:comment, :created_at], :my_index_name)
+      end
+
+      slave do
+        index?(:users, :my_index_name).must_equal(true)
       end
     end
 
     it "should add an index on a column with a reserved name" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.add_index(:group)
       end
 
       slave do
-        key?(:users, :group).must_equal(true)
+        index_on_columns?(:users, :group).must_equal(true)
       end
     end
 
     it "should add a unqiue index" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.add_unique_index(:comment)
       end
 
       slave do
-        key?(:users, :comment, :unique).must_equal(true)
+        index_on_columns?(:users, :comment, :unique).must_equal(true)
       end
     end
 
     it "should remove an index" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.remove_index([:username, :created_at])
       end
 
       slave do
-        key?(:users, [:username, :created_at]).must_equal(false)
+        index_on_columns?(:users, [:username, :created_at]).must_equal(false)
+      end
+    end
+
+    it "should remove an index with a custom name" do
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
+        t.remove_index(:reference, :index_users_on_reference)
+      end
+
+      slave do
+        index?(:users, :index_users_on_reference).must_equal(false)
       end
     end
 
     it "should apply a ddl statement" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.ddl("alter table %s add column flag tinyint(1)" % t.name)
       end
 
@@ -106,7 +126,7 @@ describe Lhm do
     end
 
     it "should change a column" do
-      Lhm.change_table(:users) do |t|
+      Lhm.change_table(:users, :atomic_switch => false) do |t|
         t.change_column(:comment, "varchar(20) DEFAULT 'none' NOT NULL")
       end
 
@@ -122,7 +142,7 @@ describe Lhm do
     it "should change the last column in a table" do
       table_create(:small_table)
 
-      Lhm.change_table(:small_table) do |t|
+      Lhm.change_table(:small_table, :atomic_switch => false) do |t|
         t.change_column(:id, "int(5)")
       end
 
@@ -146,7 +166,8 @@ describe Lhm do
           end
         end
 
-        Lhm.change_table(:users, :stride => 10, :throttle => 97) do |t|
+        options = { :stride => 10, :throttle => 97, :atomic_switch => false }
+        Lhm.change_table(:users, options) do |t|
           t.add_column(:parallel, "INT(10) DEFAULT '0'")
         end
 
@@ -167,7 +188,8 @@ describe Lhm do
           end
         end
 
-        Lhm.change_table(:users, :stride => 10, :throttle => 97) do |t|
+        options = { :stride => 10, :throttle => 97, :atomic_switch => false }
+        Lhm.change_table(:users, options) do |t|
           t.add_column(:parallel, "INT(10) DEFAULT '0'")
         end
 

data/spec/unit/atomic_switcher_spec.rb

@@ -0,0 +1,31 @@
+# Copyright (c) 2011, SoundCloud Ltd., Rany Keddo, Tobias Bielohlawek, Tobias
+# Schmidt
+
+require File.expand_path(File.dirname(__FILE__)) + '/unit_helper'
+
+require 'lhm/table'
+require 'lhm/migration'
+require 'lhm/atomic_switcher'
+
+describe Lhm::AtomicSwitcher do
+  include UnitHelper
+
+  before(:each) do
+    @start       = Time.now
+    @origin      = Lhm::Table.new("origin")
+    @destination = Lhm::Table.new("destination")
+    @migration   = Lhm::Migration.new(@origin, @destination, @start)
+    @switcher    = Lhm::AtomicSwitcher.new(@migration, nil)
+  end
+
+  describe "atomic switch" do
+    it "should perform a single atomic rename" do
+      @switcher.
+        statements.
+        must_equal([
+          "rename table `origin` to `#{ @migration.archive_name }`, " +
+          "`destination` to `origin`"
+        ])
+    end
+  end
+end

data/spec/unit/locked_switcher_spec.rb

@@ -15,7 +15,7 @@ describe Lhm::LockedSwitcher do
     @origin      = Lhm::Table.new("origin")
     @destination = Lhm::Table.new("destination")
     @migration   = Lhm::Migration.new(@origin, @destination, @start)
-    @switcher    = Lhm::LockedSwitcher.new(@migration)
+    @switcher    = Lhm::LockedSwitcher.new(@migration, nil)
   end
 
   describe "uncommitted" do

data/spec/unit/migrator_spec.rb

@@ -16,14 +16,22 @@ describe Lhm::Migrator do
 
   describe "index changes" do
     it "should add an index" do
-      @creator.add_index(["a", "b"])
+      @creator.add_index(:a)
+
+      @creator.statements.must_equal([
+        "create index `index_alt_on_a` on `lhmn_alt` (`a`)"
+      ])
+    end
+
+    it "should add a composite index" do
+      @creator.add_index([:a, :b])
 
       @creator.statements.must_equal([
         "create index `index_alt_on_a_and_b` on `lhmn_alt` (`a`, `b`)"
       ])
     end
 
-    it "should add an index with prefixed columns" do
+    it "should add an index with prefix length" do
       @creator.add_index(["a(10)", "b"])
 
       @creator.statements.must_equal([
@@ -31,7 +39,15 @@ describe Lhm::Migrator do
       ])
     end
 
-    it "should add an unique index" do
+    it "should add an index with a custom name" do
+      @creator.add_index([:a, :b], :custom_index_name)
+
+      @creator.statements.must_equal([
+        "create index `custom_index_name` on `lhmn_alt` (`a`, `b`)"
+      ])
+    end
+
+    it "should add a unique index" do
       @creator.add_unique_index(["a(5)", :b])
 
       @creator.statements.must_equal([
@@ -39,6 +55,14 @@ describe Lhm::Migrator do
       ])
     end
 
+    it "should add a unique index with a custom name" do
+      @creator.add_unique_index([:a, :b], :custom_index_name)
+
+      @creator.statements.must_equal([
+        "create unique index `custom_index_name` on `lhmn_alt` (`a`, `b`)"
+      ])
+    end
+
     it "should remove an index" do
       @creator.remove_index(["b", "a"])
 
@@ -46,6 +70,14 @@ describe Lhm::Migrator do
         "drop index `index_alt_on_b_and_a` on `lhmn_alt`"
       ])
     end
+
+    it "should remove an index with a custom name" do
+      @creator.remove_index([:a, :b], :custom_index_name)
+
+      @creator.statements.must_equal([
+        "drop index `custom_index_name` on `lhmn_alt`"
+      ])
+    end
   end
 
   describe "column changes" do

metadata

@@ -1,10 +1,10 @@
---- !ruby/object:Gem::Specification 
+--- !ruby/object:Gem::Specification
 name: lhm
-version: !ruby/object:Gem::Version 
+version: !ruby/object:Gem::Version
+  version: 1.1.0
   prerelease: 
-  version: 1.0.3
 platform: ruby
-authors: 
+authors:
 - SoundCloud
 - Rany Keddo
 - Tobias Bielohlawek
@@ -12,62 +12,65 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-
-date: 2012-02-22 00:00:00 Z
-dependencies: 
-- !ruby/object:Gem::Dependency 
-  name: mysql
-  prerelease: false
-  requirement: &id001 !ruby/object:Gem::Requirement 
+date: 2012-04-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
     none: false
-    requirements: 
-    - - ~>
-      - !ruby/object:Gem::Version 
-        version: 2.8.1
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 2.10.0
   type: :development
-  version_requirements: *id001
-- !ruby/object:Gem::Dependency 
-  name: minitest
   prerelease: false
-  requirement: &id002 !ruby/object:Gem::Requirement 
+  version_requirements: !ruby/object:Gem::Requirement
     none: false
-    requirements: 
-    - - "="
-      - !ruby/object:Gem::Version 
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
         version: 2.10.0
-  type: :development
-  version_requirements: *id002
-- !ruby/object:Gem::Dependency 
+- !ruby/object:Gem::Dependency
   name: rake
-  prerelease: false
-  requirement: &id003 !ruby/object:Gem::Requirement 
+  requirement: !ruby/object:Gem::Requirement
     none: false
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        version: "0"
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :development
-  version_requirements: *id003
-- !ruby/object:Gem::Dependency 
-  name: activerecord
   prerelease: false
-  requirement: &id004 !ruby/object:Gem::Requirement 
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
     none: false
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        version: "0"
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :runtime
-  version_requirements: *id004
-description: Migrate large tables without downtime by copying to a temporary table in chunks. The old table is not dropped. Instead, it is moved to timestamp_table_name for verification.
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Migrate large tables without downtime by copying to a temporary table
+  in chunks. The old table is not dropped. Instead, it is moved to timestamp_table_name
+  for verification.
 email: rany@soundcloud.com, tobi@soundcloud.com, ts@soundcloud.com
-executables: 
+executables:
 - lhm-kill-queue
 extensions: []
-
 extra_rdoc_files: []
-
-files: 
+files:
 - .gitignore
 - .travis.yml
 - CHANGELOG.md
@@ -78,10 +81,13 @@ files:
 - bin/lhm-spec-clobber.sh
 - bin/lhm-spec-grants.sh
 - bin/lhm-spec-setup-cluster.sh
-- gemfiles/ar-2.3.gemfile
-- gemfiles/ar-3.1.gemfile
+- bin/lhm-test-all.sh
+- gemfiles/ar-2.3_mysql.gemfile
+- gemfiles/ar-3.2_mysql.gemfile
+- gemfiles/ar-3.2_mysql2.gemfile
 - lhm.gemspec
 - lib/lhm.rb
+- lib/lhm/atomic_switcher.rb
 - lib/lhm/chunker.rb
 - lib/lhm/command.rb
 - lib/lhm/entangler.rb
@@ -100,12 +106,14 @@ files:
 - spec/fixtures/origin.ddl
 - spec/fixtures/small_table.ddl
 - spec/fixtures/users.ddl
+- spec/integration/atomic_switcher_spec.rb
 - spec/integration/chunker_spec.rb
 - spec/integration/entangler_spec.rb
 - spec/integration/integration_helper.rb
 - spec/integration/lhm_spec.rb
 - spec/integration/locked_switcher_spec.rb
 - spec/integration/table_spec.rb
+- spec/unit/atomic_switcher_spec.rb
 - spec/unit/chunker_spec.rb
 - spec/unit/entangler_spec.rb
 - spec/unit/intersection_spec.rb
@@ -117,44 +125,43 @@ files:
 - spec/unit/unit_helper.rb
 homepage: http://github.com/soundcloud/large-hadron-migrator
 licenses: []
-
 post_install_message: 
 rdoc_options: []
-
-require_paths: 
+require_paths:
 - lib
-required_ruby_version: !ruby/object:Gem::Requirement 
+required_ruby_version: !ruby/object:Gem::Requirement
   none: false
-  requirements: 
-  - - ">="
-    - !ruby/object:Gem::Version 
-      version: "0"
-required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
-  requirements: 
-  - - ">="
-    - !ruby/object:Gem::Version 
-      version: "0"
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
 requirements: []
-
 rubyforge_project: 
-rubygems_version: 1.8.15
+rubygems_version: 1.8.21
 signing_key: 
 specification_version: 3
 summary: online schema changer for mysql
-test_files: 
+test_files:
 - spec/README.md
 - spec/bootstrap.rb
 - spec/fixtures/destination.ddl
 - spec/fixtures/origin.ddl
 - spec/fixtures/small_table.ddl
 - spec/fixtures/users.ddl
+- spec/integration/atomic_switcher_spec.rb
 - spec/integration/chunker_spec.rb
 - spec/integration/entangler_spec.rb
 - spec/integration/integration_helper.rb
 - spec/integration/lhm_spec.rb
 - spec/integration/locked_switcher_spec.rb
 - spec/integration/table_spec.rb
+- spec/unit/atomic_switcher_spec.rb
 - spec/unit/chunker_spec.rb
 - spec/unit/entangler_spec.rb
 - spec/unit/intersection_spec.rb

data/gemfiles/ar-3.1.gemfile

@@ -1,4 +0,0 @@
-source "http://rubygems.org"
-
-gem "activerecord", "~> 3.1.3"
-gemspec :path=>"../"