tern 0.6.0

data/.gitignore

@@ -0,0 +1,4 @@
+pkg/*
+*.gem
+.bundle
+

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Jack Christensen
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.markdown

@@ -0,0 +1,189 @@
+Tern - The SQL Fan's Migrator
+===============================
+
+Tern is designed to simplify migrating database schemas with views, functions,
+triggers, constraints and other objects where altering one may require dropping
+and recreating others. For example, if view A selects from view B which selects
+from table C, then altering  C could require five steps: drop A, drop B, alter
+C, recreate B, and recreate A. Tern can be told to alter C and it will
+automatically perform the other four steps.
+
+Installation
+============
+
+    gem install tern
+
+Requirements
+============
+
+Ruby 1.8.7+
+
+Tern is built on top of [Sequel][1], so it can run on any database
+[Sequel][1] supports. However, Tern relies on transactional DDL to keep the
+database schema in a consistent state should something go wrong mid-migration.
+[PostgreSQL][2] supports this. MySQL does not. Using Tern on a database
+without transactional DDL is not recommended.
+
+Creating a Tern Project
+=======================
+
+    tern new my_project
+
+Edit config.yml and set up your database environments. Database environment
+parameters are passed directly to Sequel.connect so it uses the same
+[options][3].
+
+    alterations:
+      table: tern_alterations
+      column: version
+    definitions:
+      table: tern_definitions
+    environments:
+      development:
+        adapter: postgres
+        database: my_project_development
+      test:
+        adapter: postgres
+        database: my_project_test
+      production:
+        adapter: postgres
+        database: my_project_production
+
+Migration Types
+===============
+
+Tern divides potential database changes into alterations and definitions. An
+alteration is something that cannot be reversed without potentially losing data.
+For example, table creation is an alteration because dropping a table could
+result in data loss. Alterations correspond directly with the migration style
+popularized by Ruby on Rails. View creation is a definition, because it can be
+dropped without possibility of data loss.
+
+Both types of migrations have an extremely simple file format. They are simply
+the create and drop SQL statements divided by a magic comment.
+
+    ---- CREATE above / DROP below ----
+
+Alterations
+-----------
+
+Alterations should be used to create, drop, and alter tables and any other
+potentially irreversible migration.
+
+    tern generate alteration create_people
+
+or abbreviate
+
+    tern g a create_people
+
+This will create a numbered alteration in the alterations directory. Simply edit
+the file and place the create code above the magic comment and the drop code
+below it.
+
+    CREATE TABLE people(
+      id serial PRIMARY KEY,
+      name varchar NOT NULL
+    );
+
+    ---- CREATE above / DROP below ----
+
+    DROP TABLE people;
+
+If this alteration is irreversible such as a drop table, simply delete the magic
+comment.
+
+    DROP TABLE widgets;
+
+Definitions
+-----------
+
+Definitions should be used to specify the desired views, functions, triggers,
+constraints, and other database objects that are reversible without possibility
+of data loss.
+
+    tern generate definition create_ultimate_answer
+
+or abbreviate
+
+    tern g d create_ultimate_answer
+
+
+This will create the file create_ultimate_answer.sql in the definitions
+directory. Add the create and drop commands around the magic comment.
+
+    CREATE FUNCTION ultimate_answer() RETURNS integer AS $$
+      SELECT 42;
+    $$ LANGUAGE SQL;
+
+    ---- CREATE above / DROP below ----
+
+    DROP FUNCTION ultimate_answer();
+
+Definitions need to be created in dropped in a particular order. This order is
+defined in the file definitions/sequence.yml. Add this new definition to the
+sequences file.
+
+    default:
+      - ultimate_answer.sql
+
+Migrating
+=========
+
+    tern migrate
+
+To run alterations to a specific version:
+
+    tern migrate --alteration-version=42
+
+To migrate a particular database environment:
+
+    tern migrate --environment=test
+
+How it Works
+============
+
+Tern in migrates in three steps.
+
+1. Drop all definitions in the reverse order they were created
+2. Run alterations
+3. Create all definitions
+
+Tern stores the drop command for definitions in the database when it is first
+created. This allows it to totally reverse all definitions even without the
+original definition files. To make changes to the definitions just change the
+files and rerun tern migrate. Tern will drop all definitions it has previously
+created and create your new definitions.
+
+Multiple Definition Sequences
+=============================
+
+Definitions such as a check constraint on a table with many rows may be too
+time-consuming to drop and recreate for every migration. Tern allows you to
+define multiple definition sequences in the sequence.yml file.
+
+    # default:
+    #   - ultimate_answer.sql
+    #   - my_first_view.sql
+    #   - my_second_view.sql
+    # expensive:
+    #   - super_slow_check_constraint.sql
+
+Only the default sequence will normally run. To migrate the expensive
+definition sequence use the --definition-sequences option. Note that default
+will not run unless specified when using this option.
+
+    tern migrate --definition-sequences=expensive
+
+Multiple sequences may be specified and they will be run in the order they are
+listed.
+
+    tern migrate --definition-sequences=expensive default
+
+License
+=======
+
+Copyright (c) 2011 Jack Christensen, released under the MIT license
+
+[1]: http://sequel.rubyforge.org/
+[2]: http://www.postgresql.org/
+[3]: http://sequel.rubyforge.org/rdoc/files/doc/opening_databases_rdoc.html

data/Rakefile

@@ -0,0 +1,2 @@
+require 'bundler'
+Bundler::GemHelper.install_tasks

data/bin/tern

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require 'tern'
+require 'tern/app'
+
+App.start

data/lib/tern.rb

@@ -0,0 +1,207 @@
+require 'sequel'
+require 'yaml'
+
+class Change
+  SPLIT_MARKER = '---- CREATE above / DROP below ----'
+
+  def self.parse(string)
+    create_sql, drop_sql = string.split('---- CREATE above / DROP below ----')
+    [create_sql, drop_sql]
+  end
+end
+
+class Definition < Change
+  class << self
+    attr_accessor :table_name
+
+    def table
+      DB[table_name]
+    end
+
+    def ensure_table_exists
+      DB.create_table? table_name do
+        primary_key :id
+        column :sequence, :text, :null => false
+        column :create_sql, :text, :null => false
+        column :drop_sql, :text, :null => false
+      end
+    end
+
+    def load_existing
+      table.order(:id).all.map do |row|
+        new row[:id], row[:sequence], row[:create_sql], row[:drop_sql]
+      end.group_by { |d| d.sequence }
+    end
+
+    def load_targets(sequence_yml_path)
+      definition_sequences = YAML.load(File.read(sequence_yml_path))
+      sequence_dir = File.dirname(sequence_yml_path)
+
+      definition_sequences.keys.each do |sequence|
+        definition_sequences[sequence] = definition_sequences[sequence].map do |f|
+          create_sql, drop_sql = parse File.read(File.join(sequence_dir, f))
+          Definition.new nil, sequence, create_sql, drop_sql
+        end
+      end
+
+      definition_sequences
+    end
+  end
+
+  attr_reader :id
+  attr_reader :sequence
+  attr_reader :create_sql
+  attr_reader :drop_sql
+
+  def initialize(id, sequence, create_sql, drop_sql)
+    @id = id
+    @sequence = sequence
+    @create_sql = create_sql
+    @drop_sql = drop_sql
+  end
+
+  def create
+    DB.run create_sql
+    table.insert :sequence => sequence, :create_sql => create_sql, :drop_sql => drop_sql
+  end
+
+  def drop
+    DB.run drop_sql
+    table.filter(:id => id).delete
+  end
+
+  def table
+    self.class.table
+  end
+end
+
+class Alteration < Change
+  class IrreversibleAlteration < StandardError
+  end
+
+  class MissingAlteration < StandardError
+  end
+
+  class DuplicateAlteration < StandardError
+  end
+
+  class << self
+    attr_accessor :table_name
+    attr_accessor :version_column
+
+    def table
+      DB[table_name]
+    end
+
+    def ensure_table_exists
+      vc = version_column # because create_table? block is run with different binding and can't access version_column
+      DB.create_table? table_name do
+        column vc, :integer, :null => false
+      end
+      table.insert version_column => 0
+    end
+
+    def version
+      table.get(version_column)
+    end
+
+    def version=(new_version)
+      table.update version_column => new_version
+    end
+
+    def load(alterations_path)
+      alterations = Dir.glob("#{alterations_path}/[0-9]*.sql").map do |path|
+        raise "This can't happen" unless File.basename(path) =~ /^(\d+)/
+        version = $1.to_i
+        create_sql, drop_sql = parse File.read(path)
+        new version, create_sql, drop_sql
+      end.sort_by(&:version)
+
+      alterations.each_with_index do |a, i|
+        expected = i+1
+        raise DuplicateAlteration, "Alteration #{a.version.to_s.rjust(3, "0")} is duplicated" if a.version < expected
+        raise MissingAlteration, "Alteration #{expected.to_s.rjust(3, "0")} is missing" if a.version > expected
+      end
+
+      alterations
+    end
+  end
+
+  attr_reader :version
+  attr_reader :create_sql
+  attr_reader :drop_sql
+
+  def initialize(version, create_sql, drop_sql)
+    @version = version
+    @create_sql = create_sql
+    @drop_sql = drop_sql
+  end
+
+  def create
+    DB.run create_sql
+    Alteration.version = version
+  end
+
+  def drop
+    raise IrreversibleAlteration, "Alteration #{version.to_s.rjust(3, "0")} is irreversible" unless drop_sql
+    DB.run drop_sql
+    Alteration.version = version - 1
+  end
+end
+
+class Tern
+  def initialize(alterations_table, alterations_column, definitions_table)
+    Alteration.table_name = alterations_table.to_sym
+    Alteration.version_column = alterations_column.to_sym
+    Alteration.ensure_table_exists
+    @alterations = Alteration.load 'alterations'
+
+    Definition.table_name = definitions_table.to_sym
+    Definition.ensure_table_exists
+    @existing_definitions = Definition.load_existing
+    @target_definitions = Definition.load_targets 'definitions/sequence.yml'
+  end
+
+  def migrate(options={})
+    sequences = options[:sequences] || ['default']
+    DB.transaction do
+      drop_existing_definitions(sequences)
+      run_alterations(options[:version])
+      create_target_definitions(sequences)
+    end
+  end
+
+  private
+    def run_alterations(version=nil)
+      return if @alterations.empty?
+      version ||= @alterations.size
+
+      if Alteration.version < version
+        @alterations[Alteration.version..version].each(&:create)
+      elsif
+        @alterations[version..(Alteration.version-1)].reverse.each(&:drop)
+      end
+    end
+
+    def drop_existing_definitions(sequences)
+      sequences.each do |s|
+        sequence = @existing_definitions[s]
+        if sequence
+          sequence.reverse.each do |definition|
+            definition.drop
+          end
+        end
+      end
+    end
+
+    def create_target_definitions(sequences)
+      sequences.each do |s|
+        sequence = @target_definitions[s]
+        if sequence
+          sequence.each do |definition|
+            definition.create
+          end
+        end
+      end
+    end
+end

data/lib/tern/app.rb

@@ -0,0 +1,65 @@
+require 'thor'
+require 'thor/group'
+
+class App < Thor
+  include Thor::Actions
+
+  source_root(File.join(File.dirname(__FILE__), "generators"))
+
+  attr_accessor :app_name
+
+  desc "new PATH", "Create a new Tern project"
+  def new(path)
+    self.app_name = File.basename path
+    self.destination_root = path
+    directory "new", "."
+  end
+
+  map "m" => "migrate"
+  desc "migrate", "Drop definitions, run alterations, then recreate definitions"
+  method_option :environment, :type => :string, :desc => "Database environment to load", :default => "development", :aliases => "-e"
+  method_option :alteration_version, :type => :numeric, :desc => "Target alteration version", :aliases => "-a"
+  method_option :definition_sequences, :type => :array, :desc => "Definition sequences to drop and create", :default => ["default"], :aliases => "-d"
+  def migrate
+    require 'yaml'
+
+    unless File.exist?('config.yml')
+      say 'This directory does not appear to be a Tern project. config.yml not found.'
+      return
+    end
+    config = YAML.load(File.read('config.yml'))
+    ::Kernel.const_set("DB", Sequel.connect(config['environments'][options["environment"]])) # using const_set to avoid dynamic constant assignment error
+
+    begin
+      tern = Tern.new(config['alterations']['table'], config['alterations']['column'], config['definitions']['table'])
+      tern.migrate(:version => options["alteration_version"], :sequences => options["definition_sequences"])
+    rescue Alteration::IrreversibleAlteration, Alteration::MissingAlteration, Alteration::DuplicateAlteration
+      say $!, :red
+    end
+  end
+
+  map "g" => "generate"
+  desc "generate TYPE NAME", "Generate files"
+  def generate(type, name)
+    unless File.exist?('config.yml')
+      say 'This directory does not appear to be a Tern project. config.yml not found.', :red
+      return
+    end
+
+    case type
+    when 'a', 'alteration'
+      current_version = Dir.entries('alterations').map do |f|
+        f =~ /^(\d+)_.*.sql$/ ? $1.to_i : 0
+      end.max
+      zero_padded_next_version = (current_version+1).to_s.rjust(3, "0")
+      file_name = "#{zero_padded_next_version}_#{name}.sql"
+      copy_file "change.sql", "alterations/#{file_name}"
+    when 'd', 'definition'
+      file_name = "#{name}.sql"
+      copy_file "change.sql", "definitions/#{file_name}"
+      say "Remember to add #{file_name} in your sequence.yml file."
+    else
+      say "#{type} is not a valid TYPE", :red
+    end
+  end
+end