sqrbl 0.1.0

data/README.txt

@@ -0,0 +1,85 @@
+== SQrbL:  Making database migrations suck less since July 2009!
+
+Copyright (c) 2009 raSANTIAGO + Associates LLC (http://www.rasantiago.com)
+
+The source code for SQrbL can be found on GitHub at:
+1. http://github.com/rasantiago/sqrbl/
+2. http://github.com/geeksam/sqrbl/
+
+SQrbL's creator and primary author is Sam Livingston-Gray.
+    
+== DESCRIPTION:
+
+SQrbL was created to help manage an extremely specific problem:  managing SQL-based database migrations.
+
+In essence, SQrbL is a tool for managing multiple SQL queries using Ruby.  SQrbL borrows some terminology and ideas from ActiveRecord's schema migrations, but where ActiveRecord manages changes to your database schema over time, SQrbL was written to manage the process of transforming your data from one schema to another.  (Of course, you could use SQrbL for the former case as well -- just use it to write DDL queries -- but ActiveRecord has better tools for figuring out which migrations have already been applied.)
+
+===How It Works:
+
+You describe the steps in your migration in a SQrbL file.  Each step can produce as much or as little SQL as you like.  Each step has an "up" and a "down" part -- so you can do and undo each step as many times as you need to, until you get it just right.  When you run your SQrbL file, it creates a tree of *.sql files containing the output from your migration steps, one file per step.  It also creates the combined files "all_up.sql" and "all_down.sql"; these contain all of your steps combined into one giant file so that, when Cutover Day arrives, you can copy/paste the whole thing into your SQL client and run it all at once.
+
+===Why It Exists:
+
+SQL is a fantastic DSL for describing queries.  It's not bad at doing transformations, either.  Unfortunately, SQL is, um... rather verbose.  And, it lacks tools for reducing duplication -- if you have to run five similar queries that differ only in their parameters, you have to figure out how to use a prepared statement, but if you have to run five similar queries that differ in the field names they use, that's a bit more work.  SQrbL lets you use SQL for the things SQL is good at, and Ruby for the other stuff.
+
+===About the Name:
+
+SQL.rb seemed a bit too pretentious, so I went with SQrbL -- as in, "You got Ruby in my SQL."  I pronounce it "squirble" (rhymes with "squirrel").  The proper capitalization can be annoying to type, so I've aliased the SQrbL class as Sqrbl.
+
+== SYNOPSIS:
+
+<i>(Note that, in the following code sample, I'm using the convention that do/end blocks are used primarily for their side effects, and curly-brace blocks are used primarily for their return value.)</i>
+
+  Sqrbl.migration "Convert from old widgets to new widgets" do
+    set_output_directory '/path/to/generated/sql'
+
+    group "Widgets" do
+      step "Create widgets" do
+        up do
+          action "Migrate old_widgets" {
+            <<-SQL
+              #{
+                insert_into("new_widgets", {
+                  :name     => 'widget_name',
+                  :part_num => 'CONCAT("X_", part_number)',
+                  :note     => '"Imported from old_widgets"',
+                })
+              }
+              FROM old_widgets
+            SQL
+          }
+        end
+
+        down do
+          action "Drop imported organizational contacts" {
+            'DELETE FROM new_widgets WHERE note LIKE "Imported from old_widgets"'
+          }
+        end
+      end
+    end
+  end
+
+The above code sample describes a migration with one step:  migrating the data in an `old_widgets` table to a `new_widgets` table.  When run, this will generate a set of *.sql files in the directory /path/to/generated/sql.
+
+== REQUIREMENTS:
+
+* Ruby.  (SQrbL was written using MRI Ruby 1.8.6, and has not been tested using other versions.)
+
+== INSTALL:
+
+  sudo gem install rasantiago-sqrbl --source http://gems.github.com
+
+== LICENSE:
+
+SQrbL is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.

data/Rakefile

@@ -0,0 +1,32 @@
+# Look in the tasks/setup.rb file for the various options that can be
+# configured in this Rakefile. The .rake files in the tasks directory
+# are where the options are used.
+
+begin
+  require 'bones'
+  Bones.setup
+rescue LoadError
+  begin
+    load 'tasks/setup.rb'
+  rescue LoadError
+    raise RuntimeError, '### please install the "bones" gem ###'
+  end
+end
+
+ensure_in_path 'lib'
+require 'sqrbl'
+
+task :default => 'spec:run'
+
+PROJ.rubyforge.name = 'sqrbl'
+PROJ.name           = 'sqrbl'
+PROJ.version        = Sqrbl::VERSION
+PROJ.authors        = 'Sam Livingston-Gray'
+PROJ.email          = 'geeksam@gmail.com'
+PROJ.url            = 'http://sqrbl.rubyforge.org'
+PROJ.ignore_file    = '.gitignore'
+
+
+PROJ.spec.opts << '--color'
+
+# EOF

data/TODO.txt

@@ -0,0 +1,11 @@
+- bin/sqrbl should create an empty SQrbL file with the following simple template:
+    require 'rubygems'
+    require 'sqrbl'
+    Sqrbl.migration do
+      # (copy the rest from README.txt)
+    end
+
+- Writers should create some sort of output to let you know that something happened.
+
+- Define #helper(&block) as a simple wrapper to instance_eval on the same object.  (Purpose:  allow you to define helpers in a block that can be code-folded.)
+

data/bin/sqrbl

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+
+require File.expand_path(
+    File.join(File.dirname(__FILE__), %w[.. lib sqrbl]))
+
+# Put your code here
+
+# EOF

data/lib/core_exts/object.rb

@@ -0,0 +1,44 @@
+# Swiped from Rails' ActiveSupport 2.3.2
+
+class Object
+  # Returns +value+ after yielding +value+ to the block. This simplifies the
+  # process of constructing an object, performing work on the object, and then
+  # returning the object from a method. It is a Ruby-ized realization of the K
+  # combinator, courtesy of Mikael Brockman.
+  #
+  # ==== Examples
+  #
+  #  # Without returning
+  #  def foo
+  #    values = []
+  #    values << "bar"
+  #    values << "baz"
+  #    return values
+  #  end
+  #
+  #  foo # => ['bar', 'baz']
+  #
+  #  # returning with a local variable
+  #  def foo
+  #    returning values = [] do
+  #      values << 'bar'
+  #      values << 'baz'
+  #    end
+  #  end
+  #
+  #  foo # => ['bar', 'baz']
+  #
+  #  # returning with a block argument
+  #  def foo
+  #    returning [] do |values|
+  #      values << 'bar'
+  #      values << 'baz'
+  #    end
+  #  end
+  #
+  #  foo # => ['bar', 'baz']
+  def returning(value)
+    yield(value)
+    value
+  end
+end

data/lib/core_exts/symbol.rb

@@ -0,0 +1,16 @@
+# Swiped from Rails' ActiveSupport 2.3.2
+
+unless :to_proc.respond_to?(:to_proc)
+  class Symbol
+    # Turns the symbol into a simple proc, which is especially useful for enumerations. Examples:
+    #
+    #   # The same as people.collect { |p| p.name }
+    #   people.collect(&:name)
+    #
+    #   # The same as people.select { |p| p.manager? }.collect { |p| p.salary }
+    #   people.select(&:manager?).collect(&:salary)
+    def to_proc
+      Proc.new { |*args| args.shift.__send__(self, *args) }
+    end
+  end
+end

data/lib/sqrbl/base_migration_writer.rb

@@ -0,0 +1,52 @@
+require 'fileutils'
+
+module Sqrbl
+  # Base class for other migration writers.
+  class BaseMigrationWriter
+    attr_accessor :migration, :output_directory
+
+    class << self
+      # List all classes that inherit from this one
+      def subclasses
+        @@subclasses ||= []
+      end
+
+      def inherited(subclass) #:nodoc:
+        subclasses << subclass
+      end
+
+      # Convenience method:  create a new instance and invoke <tt>write!</tt> on it.
+      def write_migration!(migration)
+        new(migration).write!
+      end
+    end
+
+    def initialize(migration) #:nodoc:
+      @migration = migration
+      set_default_output_dir!
+    end
+
+    protected
+    # Set the output_directory attribute based on the output_directory or creating_file
+    # attributes of the given migration, in that order
+    def set_default_output_dir!
+      raise "No migration given!" unless migration
+      self.output_directory ||= migration.output_directory
+      self.output_directory ||= migration.creating_file && File.join(File.expand_path(File.dirname(migration.creating_file)), 'sql')
+      raise "Unable to determine output directory!" unless self.output_directory
+    end
+
+    # Make sure that the given directory exists
+    def ensure_dir_exists(dirname)
+      raise ArgumentError.new("dirname cannot be nil!") if dirname.nil?
+      dirname = File.expand_path(dirname)
+      return if File.directory?(dirname)
+      FileUtils.makedirs(dirname)
+    end
+
+    # Open +filename+ and write +contents+ to it
+    def write_file(filename, contents)
+      File.open(filename, 'w') { |f| f << contents }
+    end
+  end
+end

data/lib/sqrbl/call_stack.rb

@@ -0,0 +1,21 @@
+module Sqrbl
+  # A small set of tools for working with the call stack.
+  module CallStack
+    module_function
+
+    # Find the first item on the call stack that isn't in the Sqrbl library path
+    def first_non_sqrbl_caller(call_stack = caller)
+      call_stack.detect { |call| ! call.include?(Sqrbl::LIBPATH) }
+    end
+
+    # For a given entry in the call stack, get the file
+    def caller_file(caller_item)
+      caller_item.split(':')[0]
+    end
+
+    # For a given entry in the call stack, get the line number
+    def caller_line(caller_item)
+      caller_item.split(':')[1]
+    end
+  end
+end

data/lib/sqrbl/group.rb

@@ -0,0 +1,42 @@
+# This file is part of SQrbL, which is free software licensed under v3 of the GNU GPL.
+# For more information, please see README.txt and LICENSE.txt in the project's root directory.
+
+
+module Sqrbl
+  # Like the Migration class, Group doesn't do much on its own.
+  # It's basically a container for a list of StepPair objects, which are created using #step.
+  #
+  # Group delegates +method_missing+ calls to its +migration+ object.
+  # For more information, see MethodMissingDelegation.
+  class Group
+    attr_reader :migration, :description, :block, :steps
+
+    include Sqrbl::ExpectsBlockWithNew
+    include Sqrbl::MethodMissingDelegation
+    delegate_method_missing_to :migration
+    include HasTodos
+
+    def initialize(migration, description, options = {}, &block)
+      @migration   = migration
+      @description = description
+      @block       = lambda(&block)
+      @steps       = []
+
+      eval_block_on_initialize(options)
+    end
+
+    # Creates a StepPair object, passing it the step_description and block arguments.
+    def step(step_description, &block)
+      steps << StepPair.new(self, step_description, &block)
+    end
+
+    # A Group is valid if it contains at least one StepPair object, and all of those objects are themselves valid.
+    def valid?
+      !steps.empty? && steps.all? { |step| step.kind_of?(StepPair) && step.valid? }
+    end
+
+    def unix_name
+      Sqrbl.calculate_unix_name(description)
+    end
+  end
+end

data/lib/sqrbl/individual_migration_writer.rb

@@ -0,0 +1,48 @@
+module Sqrbl
+  # Writes one [numbered] file per migration step, in two main directories:  sql/up/ and sql/down/.
+  # Migration steps are contained in a [numbered] subfolder that corresponds to the group.
+  class IndividualMigrationWriter < BaseMigrationWriter
+    # Prepare the up and down directories and populate them with individual files.
+    #
+    # WARNING:  <b>RECURSIVELY DELETES ALL FILES</b> in output_directory/up and output_directory/down unless passed :safe_mode => true in the options hash!
+    def write!(options = {})
+      ensure_dir_exists(output_directory)
+      clear_dirs!(options)
+      write_individual_files!
+    end
+
+    protected
+    # Recursively delete all files in output_directory/up and output_directory/down unless passed :safe_mode => true in the options hash.
+    def clear_dirs!(options)
+      return if options[:safe_mode]
+      base_dir = File.expand_path(output_directory)
+      FileUtils.rm_rf(File.join(base_dir, 'up'))
+      FileUtils.rm_rf(File.join(base_dir, 'down'))
+    end
+
+    # Write out the contents of the individual files, each in its group's subfolder
+    def write_individual_files!
+      migration.groups.each_with_index do |group, idx|
+        group_dir   = "%0#{group_num_width}d_%s" % [idx + 1, group.unix_name]
+        up_subdir   = File.join(output_directory, 'up',   group_dir)
+        down_subdir = File.join(output_directory, 'down', group_dir)
+        ensure_dir_exists(up_subdir)
+        ensure_dir_exists(down_subdir)
+
+        group.steps.each_with_index do |step, idx|
+          step_filename = "%0#{step_num_width(step)}d_%s.sql" % [idx + 1, step.unix_name]
+          write_file(File.join(up_subdir,   step_filename), step.up_step.output  )
+          write_file(File.join(down_subdir, step_filename), step.down_step.output)
+        end
+      end
+    end
+
+    def group_num_width # :nodoc:
+      migration.groups.length.to_s.length
+    end
+
+    def step_num_width(step) # :nodoc:
+      step.group.steps.length.to_s.length
+    end
+  end
+end

data/lib/sqrbl/migration.rb

@@ -0,0 +1,62 @@
+# This file is part of SQrbL, which is free software licensed under v3 of the GNU GPL.
+# For more information, please see README.txt and LICENSE.txt in the project's root directory.
+
+
+module Sqrbl
+
+  # The Migration class doesn't do much on its own; it's basically just a
+  # container for a list of Group objects, which are created using #group.
+  #
+  # Note also that because Group includes the MethodMissingDelegation mixin,
+  # instance methods defined in the block given to Migration.build will become
+  # available to all groups (and their related objects) that belong to a
+  # Migration instance.  For more information, see MethodMissingDelegation.
+  class Migration
+    attr_reader :groups, :creating_file, :output_directory
+
+    include HasTodos
+
+    # Creates a new Migration object and evaluates the block in its binding.
+    # As a result, any methods called within the block will affect the state
+    # of that object (and that object only).
+    #
+    # <i>(Eventually, this will also pass the migration to a helper object that
+    # will create the tree of output *.sql files.)</i>
+    def self.build(&block)
+      returning(self.new) do |migration|
+        migration.instance_eval(&block)
+      end
+    end
+
+    def initialize # :nodoc:
+      @groups = []
+      creating_caller = CallStack.first_non_sqrbl_caller
+      @creating_file  = File.expand_path(CallStack.caller_file(creating_caller))
+    end
+
+    # Convenience method:  set the +output_directory+ attribute
+    def set_output_directory(dirname)
+      self.output_directory = File.expand_path(dirname)
+    end
+
+    # Creates a Group object, passing it the name and block arguments.
+    def group(name, &block)
+      groups << Group.new(self, name, &block)
+    end
+
+    # Convenience method:  iterate all StepPair objects in the migration
+    def step_pairs
+      groups.map(&:steps).flatten
+    end
+
+    # Convenience method:  iterate all 'up' steps in the migration
+    def up_steps
+      step_pairs.map(&:up_step)
+    end
+
+    # Convenience method:  iterate all 'down' steps in the migration
+    def down_steps
+      step_pairs.map(&:down_step)
+    end
+  end
+end

data/lib/sqrbl/mixins/expects_block_with_new.rb

@@ -0,0 +1,27 @@
+# This file is part of SQrbL, which is free software licensed under v3 of the GNU GPL.
+# For more information, please see README.txt and LICENSE.txt in the project's root directory.
+
+
+module Sqrbl
+  # This module primarily exists to help with testing.
+  # It encapsulates the common pattern of a method on one object
+  # that takes a block, creates a new object, and passes the block
+  # to the new object.
+  #
+  # In normal use, the new object should then immediately instance_eval
+  # the block; however, for testing, it is sometimes useful to delay
+  # block evaluation until later so we can set up mocking beforehand.
+  #
+  # (Search the /spec directory for "skip_block_evaluation" for examples.)
+  module ExpectsBlockWithNew
+    protected
+
+    def eval_block_on_initialize(options = {})
+      evaluate_block! unless options[:skip_block_evaluation]
+    end
+
+    def evaluate_block!
+      instance_eval(&block) if block
+    end
+  end
+end

data/lib/sqrbl/mixins/has_todos.rb

@@ -0,0 +1,65 @@
+# This file is part of SQrbL, which is free software licensed under v3 of the GNU GPL.
+# For more information, please see README.txt and LICENSE.txt in the project's root directory.
+
+
+module Sqrbl
+
+  module HasTodos
+    class Todo
+      attr_accessor :message, :call_stack, :type
+      def initialize(message, call_stack, type)
+        @message    = message
+        @call_stack = call_stack
+        @type       = type
+      end
+
+      # Returns the first item from the call stack that isn't inside the Sqrbl library.
+      # This lets us output a pointer back to the place where this instance was created.
+      def location
+        @location ||= call_stack.detect { |call| ! call.include?(Sqrbl::LIBPATH) }
+      end
+
+      # Return just the line number from +location+.
+      def calling_line
+        CallStack.caller_line(location)
+      end
+
+      # Return just the filename from +location+.
+      def creating_file
+        CallStack.caller_file(location)
+      end
+
+      # Is this Todo of type <tt>:todo</tt>?
+      def todo?
+        type == :todo
+      end
+
+      # Is this Todo of type <tt>:warning</tt>?
+      def warning?
+        type == :warning
+      end
+    end
+
+    # Return the list of Todo items.
+    def todos
+      @todos ||= []
+    end
+
+    # Create a new Todo item (of type <tt>:todo</tt>) and add it to +todos+.
+    def todo(message)
+      add_todo(message, caller, :todo)
+    end
+
+    # Create a new Todo item (of type <tt>:warning</tt>) and add it to +todos+.
+    def warning(message)
+      add_todo(message, caller, :warning)
+    end
+
+    protected
+    def add_todo(message, caller, type)
+      returning Todo.new(message, caller, type) do |new_todo|
+        todos << new_todo
+      end
+    end
+  end
+end

data/lib/sqrbl/mixins/method_missing_delegation.rb

@@ -0,0 +1,83 @@
+# This file is part of SQrbL, which is free software licensed under v3 of the GNU GPL.
+# For more information, please see README.txt and LICENSE.txt in the project's root directory.
+
+
+module Sqrbl
+  # When used as directed, Sqrbl will create a graph of objects with bidirectional
+  # references to one another.  The graph will have the following general shape
+  # (where --> indicates a relationship that is generally one-to-many):
+  #
+  # Migration --> Group --> StepPair --> Step
+  #
+  # Also, because these various objects are created with blocks that are
+  # instance_eval'd, those blocks can define helper methods that will be
+  # added to the appropriate objects in the graph.
+  #
+  # Taken together, these two features (back-references and instance_eval)
+  # let us provide a useful facility for defining helper methods inside a
+  # block, and accessing them using scoping rules that work in an intuitive
+  # manner.  For example, if a Migration defines a method #foo, which is
+  # later called from inside a Step's block, the Step can catch that call
+  # using method_missing and delegate it to its StepPair, which in turn
+  # will delegate to its Group, which in turn will delegate to its Migration.
+  #
+  # Confused yet?  Here's a slightly modified version of the example in README.txt:
+  #
+  #   Sqrbl.migration "Convert from old widgets to new widgets" do
+  #     def new_widget_insert()
+  #       insert_into("new_widgets", {
+  #         :name     => 'widget_name',
+  #         :part_num => 'CONCAT("X_", part_number)',
+  #         :note     => '"Imported from old_widgets"',
+  #       })
+  #     end
+  #
+  #     group "Widgets" do
+  #       step "Create widgets" do
+  #         up do
+  #           action "Migrate old_widgets" {
+  #             "#{new_widget_insert()} FROM old_widgets"
+  #           }
+  #         end
+  #
+  #         down do
+  #           action "Drop imported organizational contacts" {
+  #             'DELETE FROM new_widgets WHERE note LIKE "Imported from old_widgets"'
+  #           }
+  #         end
+  #       end
+  #     end
+  #   end
+  #
+  # Note that the call to new_widget_insert occurs several layers of
+  # nesting down from its definition.  By using this method_missing
+  # delegation strategy, we can effectively hide Sqrbl's object model
+  # from the user and provide something that "just works."
+  #
+  # (Without this strategy, the above example would need to define
+  # new_widget_insert as a lambda function that gets invoked using
+  # either #call or the square-bracket syntax, but I find that more
+  # awkward -- hence this wee bit of metaprogramming.)
+  module MethodMissingDelegation
+    def self.included(receiver)
+      receiver.class_eval(<<-EOF, __FILE__, __LINE__)
+        @@mm_delegate_accessor = nil
+
+        # Defines the accessor method that instances of this class should use
+        # when delegating +method_missing+ calls.
+        def self.delegate_method_missing_to(accessor)
+          @@mm_delegate_accessor = accessor
+        end
+
+        # If +delegate_method_missing_to+ was called on the class,
+        # use the accessor defined there to find a delegate and
+        # pass the unknown method to it.
+        def method_missing(method, *args, &block)
+          return super unless defined?(@@mm_delegate_accessor) && !@@mm_delegate_accessor.nil?
+          delegate = self.send(@@mm_delegate_accessor)
+          delegate.send(method, *args, &block)
+        end
+      EOF
+    end
+  end
+end