data_migrator 0.0.1

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 [name of plugin creator]
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,42 @@
+Data Migrations
+===============
+
+Every Rails developer is familiar with Rails’ invaluable schema migration tool but, as I’m sure many have experienced, it does not address the need for data migrations. Mixing data migrations into schema migrations is simply unacceptable. There is of course the seed data tool, but this is intended for the loading of initial data, much like fixtures. We needed a way to deal with ongoing data migrations to support schema changes over time. This called for a tool similar to schema migrations to be used exclusively for data migrations. The behavior is almost identical to schema migrations.  They are controlled via a versioning mechanism like schema migrations.  They can be run up and down and they load your Rails environment so all your models are fair game.  Anything you can do with your models you can do in the migrations i.e. create, update destroy/delete.
+It's easy to get started just install the plugin and your ready to go.  There is a generator just like with schema migrations:
+
+script/generate data_migration create_sample_users
+
+It will generate the versioning for you and place the file in db/data_migrations (don't worry if the folder isn't there it will create it for you).  Simply add your code to the generated file
+
+class CreateSampleUsers
+  def self.up
+	User.create(:login => 'awesome_dude' , :email => 'awesome@dude.com')
+  end
+  def self.down
+	User.destroy_all(:conditions => ['login = ?','awesome_dude'])
+  end
+end
+You can even use the Migration class if you wish:
+class CreateSampleUsers < ActiveRecord::Migration
+  def self.up
+    execute "UPDATE users SET name=’Fred' WHERE id=2" 
+  end
+end
+
+As you can see it has an up and a down just like schema migrations.  To run the data migrations simply run the rake task
+rake db:migrate_data
+
+All the schema migration features work here too
+Migrate up to and including specified version, running all migrations that have not been run:
+rake db:migrate VERSION=10101010101
+Migrate only the specified migration if it has not been run:
+rake db:migrate_data:up VERSION=1010101010101
+Migrate down only the specified version if it has not been run:
+rake db:migrate_data:down VERSION=10101010101
+
+One added benefit that data migrations have over regular schema migrations is that if you like to develop in engines as I do, it will pick up and run any of the data migrations living in your engines as well.  Just create a folder in your engine db/data_migrations and you’re all set.  When you run the rake task it will copy any files located in db/data_migrations of your engines up to RAILS_ROOT/db/data_migrations and run them.  When it is done it will remove them from RAILS_ROOT /db/data_migrations leaving them down in your engines.  Yet another cool feature (I know this is starting to sound like an infomercial), if you don't want to run some of your migrations, say you have some test data that you want only ran from your environment you can place them in folder called db/data_migrations/ignore and as the folder suggests the rake task will ignore anything and everything in that folder.  So when you commit your changes, your colleagues won't groan when your migrations won't run in their environment.  It will also recursively go through folders looking for migrations so you can organize them if you like
+
+db/data_migrations/users/010101010101_example_users.rb
+db/data_migrations/roles/010101010101_example_roles.rb
+
+That about wraps it up. Have at it my fellow developers and as always happy coding.

data/db/migrate/20100819181805_create_data_migrations_table.rb

@@ -0,0 +1,15 @@
+class CreateDataMigrationsTable < ActiveRecord::Migration
+  def self.up
+    unless table_exists?(:data_migrations)
+     create_table :data_migrations, {:id => false} do |t|
+        t.column :version,  :string
+      end
+    end
+  end
+
+  def self.down
+    if table_exists?(:data_migrations)
+     drop_table :data_migrations
+    end
+  end
+end

data/lib/data_migrator.rb

@@ -0,0 +1,303 @@
+require "benchmark"
+
+class DataMigrator
+  DEFAULT_MIGRATIONS_PATH = "#{Rails.root}/db/data_migrations"
+  REMOVE_FILES_REGEX = /^\./
+  
+  def self.migrate(passed_location=nil, passed_version=nil)
+    self.setup
+   
+    location = passed_location.nil? ? DEFAULT_MIGRATIONS_PATH  : passed_location
+    @@current_version = get_current_version
+
+    if passed_version.nil? || @@current_version.nil?
+      self.run_all_non_migrated(passed_version)
+    elsif passed_version < @@current_version
+      self.handle_lower_passed_version(passed_version)
+    elsif passed_version > @@current_version
+      self.handle_higher_passed_version(passed_version)
+    end
+  end
+  
+  def self.pending_migrations
+    versions = []
+    files = self.get_all_files
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      unless version_has_been_migrated?(version)
+        versions << filename
+      end
+    end
+
+    versions
+  end
+
+  def self.get_current_version
+    result = ActiveRecord::Base.connection.select_all("select max(version) as current_version from data_migrations")
+
+    current_version = result[0]['current_version'] unless result == -1
+
+    current_version = current_version.to_i unless current_version.nil?
+
+    current_version
+  end
+  
+  def self.next_migration_number
+	Time.now.utc.strftime("%Y%m%d%H%M%S")
+  end
+  
+  def self.run_up(passed_version)
+    self.setup
+    
+    raise "VERSION is required" unless passed_version
+    
+    files = self.get_all_files
+    found = false
+    
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      if passed_version == version
+        found = true
+        unless self.version_has_been_migrated?(version)
+          self.handle_action(file, klass_name, version, :up)
+        else
+          puts "** Version #{passed_version} has already been migrated"
+        end
+      end
+    end
+    
+    unless found
+      puts "** Version #{passed_version} not found"
+    end
+    
+  end
+  
+  def self.run_down(passed_version)
+    self.setup
+    
+    raise "VERSION is required" unless passed_version
+    
+    files = self.get_all_files
+    found = false 
+
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      if passed_version == version
+        found = true
+        if self.version_has_been_migrated?(version)
+          self.handle_action(file, klass_name, version, :down)
+        else
+          puts "** Version #{passed_version} has not been migrated"
+        end
+      end
+    end
+    
+    unless found
+      puts "** Version #{passed_version} not found"
+    end
+    
+  end
+  
+  def self.prepare_migrations
+    target = "#{Rails.root}/db/data_migrations/"
+
+    # first copy all app data_migrations away
+    files = Dir["#{target}*.rb"]
+
+    unless files.empty?
+      FileUtils.mkdir_p "#{target}/ignore/app/"
+      FileUtils.cp files, "#{target}/ignore/app/"
+      puts "copied #{files.size} data_migrations to db/data_migrations/ignore/app"
+    end
+
+    dirs = Rails::Application::Railties.engines.map{|p| p.config.root.to_s}
+    files = Dir["{#{dirs.join(',')}}/db/data_migrations/*.rb"]
+
+    unless files.empty?
+      FileUtils.mkdir_p target
+      FileUtils.cp files, target
+      puts "copied #{files.size} migrations to db/data_migrations"
+    end
+  end
+  
+  def self.cleanup_migrations
+    target = "#{Rails.root}/db/data_migrations/"
+      
+	  files = Dir["#{target}*.rb"]
+    unless files.empty?
+      FileUtils.rm files
+      puts "removed #{files.size} data_migrations from db/data_migrations"
+    end
+    files = Dir["#{target}/ignore/app/*.rb"]
+    unless files.empty?
+      FileUtils.cp files, target
+      puts "copied #{files.size} data_migrations back to db/data_migrations"
+    end
+    FileUtils.rm_rf "#{target}/ignore/app"
+  end
+	
+  private
+  
+  def self.setup
+    unless self.data_migrations_table_exists?
+      self.create_data_migrations_table
+    end
+    
+    unless File.directory? DEFAULT_MIGRATIONS_PATH
+      FileUtils.mkdir_p(DEFAULT_MIGRATIONS_PATH)
+      #create ignore folder
+      FileUtils.mkdir_p(DEFAULT_MIGRATIONS_PATH + '/ignore/')
+    end
+  end
+  
+  def self.handle_higher_passed_version(passed_version)
+    files = self.get_all_files
+
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      if version <= passed_version
+        unless self.version_has_been_migrated?(version)
+          self.handle_action(file, klass_name, version, :up)
+        end
+      end
+    end
+
+  end
+
+  def self.run_all_non_migrated(passed_version)
+    files = self.get_all_files
+
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      if passed_version.nil? or version <= passed_version
+        if !self.version_has_been_migrated?(version) 
+          self.handle_action(file, klass_name, version, :up)
+        end
+      end
+    end
+  end
+
+  def self.handle_lower_passed_version(passed_version)
+    files = self.get_all_files
+
+    files.each do |file|
+      filename, version, klass_name = self.seperate_file_parts(file)
+      if version > passed_version
+        if self.version_has_been_migrated?(version)
+          self.handle_action(file, klass_name, version, :down)
+        end
+      end
+    end
+
+  end
+
+  def self.handle_action(file, klass_name, version, action)
+    require file
+    klass = klass_name.camelize.constantize
+    puts "=================Migrating #{klass.to_s} #{action.to_s.upcase}============"
+    begin
+      time = Benchmark.measure do
+        ActiveRecord::Base.transaction do
+          klass.send(action.to_s)
+        end
+      end
+    rescue Exception=>ex
+      self.cleanup_migrations
+      raise ex
+    end
+    time_str = "(%.4fs)" % time.real
+    puts "================Finished #{klass.to_s} in #{time_str}=="
+    self.insert_migration_version(version)
+  end
+  
+  def self.insert_migration_version(version)
+    ActiveRecord::Base.connection.execute("insert into data_migrations (version) values ('#{version}')")
+  end
+
+  def self.remove_migration_version(version)
+    ActiveRecord::Base.connection.execute("delete from data_migrations where version = '#{version}'")
+  end
+
+  def self.version_has_been_migrated?(version)
+    result = true
+
+    db_result = ActiveRecord::Base.connection.select_all("select count(*) as num_rows from data_migrations where version = '#{version}'")
+
+    num_rows = db_result[0]['num_rows'] unless db_result == -1
+
+    if num_rows.nil? || num_rows.to_i == 0
+      result = false
+    end
+
+    result
+  end
+
+  def self.data_migrations_table_exists?
+    table_names =  ActiveRecord::Base.connection.tables
+    table_names.include?('data_migrations')
+  end
+
+  def self.create_data_migrations_table
+    puts "** data_migrations table missing creating now...."
+    puts ActiveRecord::Migrator.run(:up, File.join(File.dirname(__FILE__),'../db/migrate/'), 20100819181805)
+    puts "** done"
+  end
+
+  def self.seperate_file_parts(file)
+    paths = file.split('/')
+    filename = paths[paths.length - 1]
+    version = filename.split('_')[0]
+    klass_name = filename.gsub(/#{version}/, "").gsub(/.rb/, "")[1..filename.length]
+
+    return filename, version.to_i, klass_name
+  end
+
+  def self.get_all_files
+    files = []
+    
+    if File.directory? DEFAULT_MIGRATIONS_PATH
+    
+      files_or_directories = Dir.entries(DEFAULT_MIGRATIONS_PATH).map{|directory| directory}
+
+      files_or_directories.delete_if{|name| name =~ REMOVE_FILES_REGEX} #remove any file leading with . or ..
+      files_or_directories.delete_if{|name| name == 'ignore'} #ignore the ignore folder
+
+      
+      files_or_directories.each do |file_or_directory|
+        file_or_directory = DEFAULT_MIGRATIONS_PATH + "/" + file_or_directory
+        files = self.get_files_in_directory(file_or_directory, files)
+      end
+
+      files.sort! {|x,y| self.get_file_name(x) <=> self.get_file_name(y)}
+    end
+
+    files
+  end
+
+  def self.get_files_in_directory(file_or_directory, files)
+    unless file_or_directory =~ /\w\.rb/
+      files_or_directories = Dir.entries(file_or_directory).map{|directory| directory}
+
+      files_or_directories.delete_if{|name| name =~ REMOVE_FILES_REGEX} #remove any file leading with . or ..
+      files_or_directories.delete_if{|name| name == 'ignore'} #ignore the ignore folder
+
+      files_or_directories.each do |_file_or_directory|
+        _file_or_directory = file_or_directory + "/" + _file_or_directory
+        files = self.get_files_in_directory(_file_or_directory, files)
+      end
+    else
+      files << file_or_directory
+    end
+
+    files
+  end
+
+  def self.get_file_name(dir)
+    file_name = dir
+
+    paths = dir.split('/')
+    file_name = paths[paths.length - 1]
+
+    file_name
+  end
+end

data/lib/generators/data_migration/USAGE

@@ -0,0 +1,9 @@
+Description:
+    Used to generate migration to insert data into database
+	To run the migration db:migrate_data
+
+Example:
+    rails g data_migration my_new_migration
+
+    This will create:
+        db/data_migrations/[timestamp]_my_new_migration.rb

data/lib/generators/data_migration/data_migration_generator.rb

@@ -0,0 +1,7 @@
+class DataMigrationGenerator < Rails::Generators::NamedBase
+  source_root File.expand_path('../templates', __FILE__)
+  
+  def generate_layout  
+    template "migration_template.rb", "db/data_migrations/#{DataMigrator.next_migration_number}_#{file_name}.rb" 
+  end  
+end

data/lib/generators/data_migration/templates/migration_template.rb

@@ -0,0 +1,11 @@
+class <%=class_name %>
+  
+  def self.up
+    #insert data here
+  end
+  
+  def self.down
+    #remove data here
+  end
+
+end

data/lib/tasks/data_migrator.rake

@@ -0,0 +1,54 @@
+namespace :db do
+  desc 'migrates data into database'
+  task :migrate_data => :environment do
+      
+    passed_version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
+    DataMigrator.prepare_migrations
+    DataMigrator.migrate(nil, passed_version)
+	  DataMigrator.cleanup_migrations
+      
+  end#end task
+    
+  namespace :migrate_data do
+	task :list_pending => :environment do
+      DataMigrator.prepare_migrations
+      pending_migrations = DataMigrator.pending_migrations
+      puts "================Pending Data Migrations=========="
+      puts pending_migrations
+      puts "================================================="
+      DataMigrator.cleanup_migrations
+    end
+
+    task :version => :environment do
+      version = DataMigrator.get_current_version
+
+      unless version.nil?
+        puts "** Current version #{version}"
+      else
+        puts "** No migrations ran"
+      end
+    end
+      
+    task :up => :environment do
+      passed_version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
+      raise "VERSION is required" unless passed_version
+        
+      DataMigrator.prepare_migrations
+      DataMigrator.run_up(passed_version)
+      DataMigrator.cleanup_migrations
+        
+    end#end task
+      
+    task :down => :environment do
+      passed_version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
+      raise "VERSION is required" unless passed_version
+		
+      DataMigrator.prepare_migrations
+      DataMigrator.run_down(passed_version)
+      DataMigrator.cleanup_migrations
+        
+    end#end task
+  end#end namespace
+end
+
+

data/lib/version.rb

@@ -0,0 +1,3 @@
+module DataMigrator
+  VERSION = '0.0.1' unless defined?(::DataMigrator::VERSION)
+end

data/test/data_migrator_test.rb

@@ -0,0 +1,8 @@
+require 'test_helper'
+
+class DataMigratorTest < ActiveSupport::TestCase
+  # Replace this with your real tests.
+  test "the truth" do
+    assert true
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,3 @@
+require 'rubygems'
+require 'active_support'
+require 'active_support/test_case'

metadata

@@ -0,0 +1,93 @@
+--- !ruby/object:Gem::Specification 
+name: data_migrator
+version: !ruby/object:Gem::Version 
+  hash: 29
+  prerelease: false
+  segments: 
+  - 0
+  - 0
+  - 1
+  version: 0.0.1
+platform: ruby
+authors: 
+- Russell Holmes, Adam Hull
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2011-08-19 00:00:00 -04:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: activerecord
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 9
+        segments: 
+        - 2
+        - 3
+        - 5
+        version: 2.3.5
+  type: :runtime
+  version_requirements: *id001
+description: Allows you to create data migrations that can be run up and down to insert data into the database.
+email: russellfholmes@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: []
+
+files: 
+- db/migrate/20100819181805_create_data_migrations_table.rb
+- lib/data_migrator.rb
+- lib/generators/data_migration/data_migration_generator.rb
+- lib/generators/data_migration/templates/migration_template.rb
+- lib/generators/data_migration/USAGE
+- lib/tasks/data_migrator.rake
+- lib/version.rb
+- MIT-LICENSE
+- README.rdoc
+- test/data_migrator_test.rb
+- test/test_helper.rb
+has_rdoc: true
+homepage: https://github.com/portablemind/data_migrator
+licenses: []
+
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.7
+signing_key: 
+specification_version: 3
+summary: Creates Data Migrations for data similar to schema migrations.
+test_files: 
+- test/data_migrator_test.rb
+- test/test_helper.rb