data_migrate 1.0.0 → 1.1.0

data/README.md

@@ -3,25 +3,90 @@ Data Migrate
 
 Run data migrations alongside schema migrations.
 
-Data migrations are stored in db/data. They act like schema migrations, except they should be reserved for data migrations. For instance, if you realize you need to titleize all yours titles, this is the place to do it.
-
-Data migrations can be created at the same time as schema migrations, or independently.  Database (db:) tasks have been added and extended to run on data migrations only, or in conjunction with the schema migration.  For instance, `rake db:migrate:with_data` will run both schema and data migrations in the proper order.
+Data migrations are stored in db/data. They act like schema
+migrations, except they should be reserved for data migrations. For
+instance, if you realize you need to titleize all yours titles, this
+is the place to do it.
+
+Why should I use this?
+----------------------
+
+Its seems when a project hits a certain size, I get to manipulate data
+outside the application itself.  Changing defaults, new validations,
+one-to-one to one-to-many... I found it a pain and dodgy to have to
+step up migrations one by one, run a ruby script of some sort, then
+resume migrations.  It tanks a lot of the automation of deploy.
+
+If you don't use the one off scripts, you could do it as a regular
+migration.  It'd be much better to keep concerns separate. The benefit
+of having them separate has to do with your data model.
+
+For instance, lets take an absurd example, to illustrate: You have
+your infamous [Rails blog](http://media.rubyonrails.org/video/rails-0-5.mov)
+that has posts with many comments.  After some use, you decide you are
+going to be a trend setter, and want only one comment per post, and
+just the text. "Frist!" rules the day. Given that you:
+- write a migration to add a comment column to Post
+- write a migration to move the contents of the first comments to the Post
+- drop the column_id column from Post
+- drop the Comment model
+- fix all your test/controller/view mojo.
+
+You've just got bit.  When you rake setup:development, the mess gets
+mad at you after it creates your database, and starts cranking through
+migrations.  It gets to the part where you iterate over the comments
+and it blows up.  You don't have a comment model anymore for it to
+even try and get 'all' from.  You think you are smarter, and wrap the
+AR call in a conditional based on the environment. That's fine until
+you get that QA gal, and she wants her own thing. Then the UI people
+get tired of waiting for the full stack to load on page refreshes, so
+you have to edit past migrations...
+
+With Data Migrate, you have the control.  You can generate your
+migrations as schema or data as you would as your work flow. For
+setting tasks that don't require any intermediate AR activity, like
+dev and test, you stick with db:migrate.  For your prod, and qa, you
+change their scripts to `db:migrate:with_data`.  Of course you want to
+test your migration, so you have the choice of `db:migrate:with_data` or
+`data:migrate` to just capture that data change.
+
+What's it do?
+-------------
+
+Data migrations are stored in db/data. They act like schema
+migrations, except they should be reserved for data migrations. For
+instance, if you realize you need to titleize all yours titles, this
+is the place to do it. Running any of the provided rake tasks also
+creates a data schema table to mirror the usual schema migrations
+table to track all the goodness.
+
+Data migrations can be created at the same time as schema migrations,
+or independently.  Database (db:) tasks have been added and extended
+to run on data migrations only, or in conjunction with the schema
+migration.  For instance, `rake db:migrate:with_data` will run both
+schema and data migrations in the proper order.
 
 Note: If a data and schema migration share the same version number, schema gets precedence when migrating up. Data does down.
 
 Rails 3 and Ruby 1.9
 --------------------
 
-Data Migrate is Rails 3.0.0 - 3.0.7,  and Ruby 1.9 compatible
+Data Migrate is Rails 3.0.0 - 3.0.7, and Ruby 1.9 compatible
 
 Installation
 ------------
-After adding Data Migrate to your project,
+Add the gem to your project
+
+    # Gemfile
+    gem 'data_migrate'
 
-    rails g data_migrate:install
-    rake db:migrate
+Then `bundle install` and you are ready to go.
 
-A table 'data_migrations' table will be generated.
+So you know, when you use one of the provide rake tasks, a table
+called 'data_migrations' will be created in your database. This
+is to mirror the way the standard 'db' rake tasks work. If you've
+installed previous to v1.1.0, you'll want to delete the
+'create\_data\_migrations_table' migration.
 
 Usage
 -----
@@ -76,3 +141,8 @@ Running `rake db:migrate:up:with_data VERSION=20110419021211` would execute the
 Going down instead of up would be the opposite.
 
 `rake db:migrate:status:with_data` provides and additional column to indicate which type of migration.
+
+Thanks
+------
+[Jeremy Durham](http://jeremydurham.com/) for fleshing out the idea with me, and providing guidance.
+You!  Yes, you. Thanks for checking it out.

data/lib/data_migrate/version.rb

@@ -1,3 +1,3 @@
 module DataMigrate
-  VERSION = "1.0.0"
+  VERSION = "1.1.0"
 end

data/lib/generators/data_migrate.rb

@@ -1,13 +1,6 @@
-require 'rails/generators/base'
 require 'rails/generators/named_base'
 module DataMigrate
   module Generators
-    class InstallGenerator < Rails::Generators::Base #:nodoc:
-      def self.source_root
-         @_data_migrate_source_root ||= File.expand_path(File.join(File.dirname(__FILE__), 'data_migrate', generator_name, 'templates'))
-      end
-    end
-
     class DataMigrationGenerator < Rails::Generators::NamedBase #:nodoc:
       def self.source_root
          @_data_migrate_source_root ||= File.expand_path(File.join(File.dirname(__FILE__), generator_name, 'templates'))

data/lib/generators/data_migration/data_migration_generator.rb

@@ -9,7 +9,7 @@ module DataMigrate
       include Rails::Generators::Migration
 
       argument :attributes, :type => :array, :default => [], :banner => "field:type field:type"
-      class_option :skip_schema_migration, :desc => 'Dont generate database migration file.', :type => :boolean
+      class_option :skip_schema_migration, :desc => 'Dont generate database schema migration file.', :type => :boolean
 
       def create_data_migration
         set_local_assigns!

data/tasks/databases.rake

@@ -2,8 +2,7 @@ namespace :db do
   namespace :migrate do
     desc "Migrate the database data and schema (options: VERSION=x, VERBOSE=false)."
     task :with_data => :environment do
-      config = connect_to_database
-      next unless config
+      assure_data_schema_table
 
       ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
       target_version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
@@ -60,6 +59,7 @@ namespace :db do
     namespace :redo do
       desc 'Rollbacks the database one migration and re migrate up (options: STEP=x, VERSION=x).'
       task :with_data => :environment do
+      assure_data_schema_table
         if ENV["VERSION"]
           Rake::Task["db:migrate:down:with_data"].invoke
           Rake::Task["db:migrate:up:with_data"].invoke
@@ -75,7 +75,7 @@ namespace :db do
       task :with_data => :environment do
         version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
         raise "VERSION is required" unless version
-        config = connect_to_database
+        assure_data_schema_table
         run_both = ENV["BOTH"] == "true"
         migrations = pending_migrations.keep_if{|m| m[:version] == version}
 
@@ -100,7 +100,7 @@ namespace :db do
       task :with_data => :environment do
         version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
         raise "VERSION is required" unless version
-        config = connect_to_database
+        assure_data_schema_table
         run_both = ENV["BOTH"] == "true"
         migrations = past_migrations.keep_if{|m| m[:version] == version}
 
@@ -170,8 +170,7 @@ namespace :db do
     desc 'Rolls the schema back to the previous version (specify steps w/ STEP=n).'
     task :with_data => :environment do
       step = ENV['STEP'] ? ENV['STEP'].to_i : 1
-      config = connect_to_database
-      next unless config
+      assure_data_schema_table
       past_migrations[0..(step - 1)].each do | past_migration |
         if past_migration[:kind] == :data
           ActiveRecord::Migration.write("== %s %s" % ['Data', "=" * 71])
@@ -187,6 +186,7 @@ namespace :db do
   namespace :forward do
     desc 'Pushes the schema to the next version (specify steps w/ STEP=n).'
     task :with_data => :environment do
+      assure_data_schema_table
       # TODO: No worky for .forward
       step = ENV['STEP'] ? ENV['STEP'].to_i : 1
       # DataMigrate::DataMigrator.forward('db/data/', step)
@@ -206,6 +206,7 @@ namespace :db do
   namespace :version do
     desc "Retrieves the current schema version numbers for data and schema migrations"
     task :with_data => :environment do
+      assure_data_schema_table
       puts "Current Schema version: #{ActiveRecord::Migrator.current_version}"
       puts "Current Data version: #{DataMigrate::DataMigrator.current_version}"
     end
@@ -214,6 +215,7 @@ end
 
 namespace :data do
   task :migrate => :environment do
+    assure_data_schema_table
     ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
     DataMigrate::DataMigrator.migrate("db/data/", ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
   end
@@ -221,6 +223,7 @@ namespace :data do
   namespace :migrate do
     desc  'Rollbacks the database one migration and re migrate up (options: STEP=x, VERSION=x).'
     task :redo => :environment do
+      assure_data_schema_table
       if ENV["VERSION"]
         Rake::Task["data:migrate:down"].invoke
         Rake::Task["data:migrate:up"].invoke
@@ -232,6 +235,7 @@ namespace :data do
 
     desc 'Runs the "up" for a given migration VERSION.'
     task :up => :environment do
+      assure_data_schema_table
       version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
       raise "VERSION is required" unless version
       DataMigrate::DataMigrator.run(:up, "db/data/", version)
@@ -241,6 +245,7 @@ namespace :data do
     task :down => :environment do
       version = ENV["VERSION"] ? ENV["VERSION"].to_i : nil
       raise "VERSION is required" unless version
+      assure_data_schema_table
       DataMigrate::DataMigrator.run(:down, "db/data/", version)
     end
 
@@ -277,14 +282,16 @@ namespace :data do
 
   desc 'Rolls the schema back to the previous version (specify steps w/ STEP=n).'
   task :rollback => :environment do
+    assure_data_schema_table
     step = ENV['STEP'] ? ENV['STEP'].to_i : 1
     DataMigrate::DataMigrator.rollback('db/data/', step)
   end
 
   desc 'Pushes the schema to the next version (specify steps w/ STEP=n).'
   task :forward => :environment do
-    # TODO: No worky for .forward
+    assure_data_schema_table
     step = ENV['STEP'] ? ENV['STEP'].to_i : 1
+    # TODO: No worky for .forward
     # DataMigrate::DataMigrator.forward('db/data/', step)
     migrations = pending_data_migrations.reverse.pop(step).reverse
     migrations.each do | pending_migration |
@@ -294,6 +301,7 @@ namespace :data do
 
   desc "Retrieves the current schema version number for data migrations"
   task :version => :environment do
+    assure_data_schema_table
     puts "Current data version: #{DataMigrate::DataMigrator.current_version}"
   end
 end
@@ -342,3 +350,16 @@ def past_migrations sort=nil
 
   sort == 'asc' ? sort_migrations(migrations) : sort_migrations(migrations).reverse
 end
+
+def assure_data_schema_table
+  config = ActiveRecord::Base.configurations[Rails.env || 'development']
+  ActiveRecord::Base.establish_connection(config)
+  sm_table = DataMigrate::DataMigrator.schema_migrations_table_name
+
+  unless ActiveRecord::Base.connection.table_exists?(sm_table)
+    ActiveRecord::Base.connection.create_table(sm_table, :id => false) do |schema_migrations_table|
+      schema_migrations_table.column :version, :string, :null => false
+    end
+    ActiveRecord::Base.connection.add_index sm_table, :version, :unique => true, :name => "#{ActiveRecord::Base.table_name_prefix}unique_data_migrations#{ActiveRecord::Base.table_name_suffix}"
+  end
+end

metadata

@@ -2,7 +2,7 @@
 name: data_migrate
 version: !ruby/object:Gem::Version 
   prerelease: 
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors: 
 - Andrew J Vargo
@@ -10,7 +10,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-05-02 00:00:00 Z
+date: 2011-05-05 00:00:00 Z
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: rails
@@ -47,9 +47,6 @@ files:
 - lib/data_migrate/railtie.rb
 - lib/data_migrate/version.rb
 - lib/generators/data_migrate.rb
-- lib/generators/data_migrate/USAGE
-- lib/generators/data_migrate/install/install_generator.rb
-- lib/generators/data_migrate/install/templates/install_migration.rb
 - lib/generators/data_migration/data_migration_generator.rb
 - lib/generators/data_migration/templates/data_migration.rb
 - lib/generators/data_migration/templates/migration.rb

data/lib/generators/data_migrate/USAGE

@@ -1,8 +0,0 @@
-Description:
-    Explain the generator
-
-Example:
-    rails generate data_migration Thing
-
-    This will create:
-        what/will/it/create

data/lib/generators/data_migrate/install/install_generator.rb

@@ -1,24 +0,0 @@
-require 'generators/data_migrate'
-require 'rails/generators'
-require 'rails/generators/migration'
-
-module DataMigrate
-  module Generators
-    class InstallGenerator < Rails::Generators::Base
-      include Rails::Generators::Migration
-
-      def create_migration_file
-        migration_template "install_migration.rb", "db/migrate/create_data_migrations.rb"
-      end
-
-      protected
-      def self.next_migration_number(dirname)
-        if ActiveRecord::Base.timestamped_migrations
-          Time.new.utc.strftime("%Y%m%d%H%M%S")
-        else
-          "%.3d" % (current_migration_number(dirname) + 1)
-        end
-      end
-    end
-  end
-end

data/lib/generators/data_migrate/install/templates/install_migration.rb

@@ -1,13 +0,0 @@
-class CreateDataMigrations < ActiveRecord::Migration
-  def self.up
-    create_table :data_migrations do |t|
-      t.string :version, :null => false
-    end
-
-    add_index :data_migrations, :version, :unique => true, :name => "<%= ActiveRecord::Base.table_name_prefix %>unique_data_migrations<%= ActiveRecord::Base.table_name_suffix %>"
-  end
-
-  def self.down
-    drop_table :data_migrations
-  end
-end