db_leftovers 0.6.0 → 0.7.0

data/README.html

@@ -1,10 +1,10 @@
 <h1>db_leftovers</h1>
 
-<p>db_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
+<p>db_leftovers lets you define indexes and foreign keys for your Rails app
 in one place using an easy-to-read DSL,
 then run a rake task to bring your database up-to-date.
-I wrote this because I didn't want indexes and constraints scattered throughout my migrations or buried in my <code>schema.rb</code>, and I wanted a command I could run to ensure that they matched across my development, test, staging, and production databases.
-This was particularly a problem for Heroku projects, because <code>db:push</code> and <code>db:pull</code> do not transfer your foreign keys or other constraints.
+I wrote this because I didn't want indexes and foreign keys scattered throughout my migrations or buried in my <code>schema.rb</code>, and I wanted a command I could run to ensure that they matched across my development, test, staging, and production databases.
+This was particularly a problem for Heroku projects, because <code>db:push</code> and <code>db:pull</code> do not transfer your foreign keys.
 But now that it's written, I'm finding it useful on non-Heroku projects as well.</p>
 
 <p>At present db_leftovers works only on PostgreSQL databases,
@@ -12,9 +12,9 @@ but it could easily be extended to cover other RDBMSes.</p>
 
 <h2>Configuration File</h2>
 
-<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. There are only a few methods:</p>
+<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and foreign keys you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. There are only a few methods:</p>
 
-<h3>index(table_name, columns, [opts])</h3>
+<h3>index table_name, columns, [opts]</h3>
 
 <p>This ensures that you have an index on the given table and column(s). The <code>columns</code> parameter can be either a string or a list of strings. Opts is a hash with the following possible keys:</p>
 
@@ -31,48 +31,38 @@ index :books, [:publisher_id, :published_at]
 index :books, :isbn, :unique =&gt; true
 </code></pre>
 
-<h3>foreign_key(from_table, from_column, to_table, [to_column, [opts]])</h3>
+<h3>foreign_key from_table, from_column, to_table, [to_column, [opts]]</h3>
 
 <p>This ensures that you have a foreign key relating the given tables and columns.
-All parameters are strings/symbols except <code>opts</code>, which is a hash.
+All parameters are strings except <code>opts</code>, which is a hash.
 If you don't pass anything for <code>opts</code>, you can leave off the <code>to_column</code> parameter, and it will default to <code>:id</code>.
-The only option that is supported is <code>:on_delete</code>, which may have any of these values:</p>
+These options are supported:</p>
 
 <ul>
-<li><code>nil</code> Indicates that attempting to delete the referenced row should fail (the default).</li>
 <li><code>:set_null</code> Indicates that the foreign key should be set to null if the referenced row is deleted.</li>
 <li><code>:cascade</code> Indicates that the referencing row should be deleted if the referenced row is deleted.</li>
 </ul>
 
+<p>These options are mutually exclusive. They should probably be consolidated into a single option like <code>:on_delete</code>.</p>
+
 <h4>Examples</h4>
 
 <pre><code>foreign_key :books, :author_id, :authors, :id
 foreign_key :books, :publisher_id, :publishers
-foreign_key :pages, :book_id, :books, :id, :on_delete =&gt; :cascade
-</code></pre>
-
-<h3>check(constraint_name, on_table, expression)</h3>
-
-<p>This ensures that you have a CHECK constraint on the given table with the given name and expression.
-All parameters are strings or symbols.</p>
-
-<h4>Examples</h4>
-
-<pre><code>check :books, :books_have_positive_pages, 'page_count &gt; 0'
+foreign_key :pages, :book_id, :books, :id, :cascade =&gt; true
 </code></pre>
 
-<h3>table(table_name, &amp;block)</h3>
+<h3>table table_name, &amp;block</h3>
 
 <p>The <code>table</code> call is just a convenience so you can group all a table's indexes and foreign keys together and not keep repeating the table name. You use it like this:</p>
 
 <pre><code>table :books do
   index :author_id
   foreign_key :publisher_id, :publishers
-  check :books_have_positive_pages, 'page_count &gt; 0'
 end
 </code></pre>
 
-<p>You can repeat <code>table</code> calls for the same table several times if you like. This lets you put your indexes in one place and your foreign keys in another, for example.</p>
+<p>You can repeat <code>table</code> calls for the same table several times if you like. This lets you put your indexes in one place and your foreign keys in another.</p>
 
 <h2>Running db_leftovers</h2>
 
@@ -81,35 +71,17 @@ end
 <pre><code>rake db:leftovers
 </code></pre>
 
-<p>To print messages even about indexes/foreign keys/constraints that haven't changed, you can say:</p>
-
-<pre><code>rake db:leftovers VERBOSE=true
-</code></pre>
-
-<p>or</p>
-
-<pre><code>rake db:leftovers DB_LEFTOVERS_VERBOSE=true
-</code></pre>
-
 <h2>Known Issues</h2>
 
 <ul>
 <li><p>db_leftovers only supports PostgreSQL databases.
 If you want to add support for something else, just send me a pull request!</p></li>
-<li><p>db_leftovers will not notice if an foreign key/constraint definition changes.
-Right now it only checks for existence/non-existence.
-You can get around this by adding a version number to your constraint names,
-so if you want to force books to have at least 12 pages, you can say this:</p>
-
-<p><code>check :books_have_positive_pages2, 'page_count &gt;= 12'</code></p>
-
-<p>Then the old constraint will be dropped and the new one will be added.</p>
-
-<p>However, db_leftovers <em>does</em> check for index definitions, so if you
-make an existing index unique, add a column, remove a WHERE clause, or
-anything else, it will notice and drop and re-create the index.
-I'm working on doing the same thing for foreign keys/constraints,
-but it's not done just yet.</p></li>
+<li><p>db_leftovers will not notice if an index/foreign key definition changes.
+Right now it only checks for existence/non-existence.</p></li>
+<li><p>If your database is mostly up-to-date, then running the Rake task will spam
+you with messages about how this index and that foreign key already exist.
+These should be hidden by default and shown only if you request a higher
+verbosity.</p></li>
 </ul>
 
 <h2>Contributing to db_leftovers</h2>
@@ -126,5 +98,5 @@ but it's not done just yet.</p></li>
 
 <h2>Copyright</h2>
 
-<p>Copyright (c) 2012 Paul A. Jungwirth.
-See LICENSE.txt for further details.</p>
+<p>Copyright (c) 2012 Paul A. Jungwirth. See LICENSE.txt for
+further details.</p>

data/TODO

@@ -2,7 +2,17 @@
 - Support altering index and foreign keys if their definition changes
 - don't show "{index,fk} already exists" unless you set the verbosity higher
 
-- Use Sequel to introspect databases for their indexes, to get easy support for non-Postgres databases.
-  - BUT: Sequel can't return partial indexes (i.e. indexes created with a WHERE clause), and using those kind of indexes is a major feature for db_leftovers. So this will have to wait until I can get a patch accepted to let at least the Postgres Sequel adapter optionally query for those type of indexes.
+- Abbreviated notation for conventional foreign keys:
+
+      table :books do
+        foreign_key :authors
+        foreign_key :publishers, :on_delete => :cascade
+
+        # and with a funny FK column but implicit PK of 'id':
+
+        foreign_key :ghost_writer_id, :authors, :on_delete => :cascade
+      end
+
+  In other words, the params are foreign_key(from_table, [from_column], to_table, [to_column], [opts]).
 
 

data/VERSION

@@ -1 +1 @@
-0.6.0
+0.7.0

data/db_leftovers.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = "db_leftovers"
-  s.version = "0.6.0"
+  s.version = "0.7.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Paul A. Jungwirth"]
-  s.date = "2012-09-27"
+  s.date = "2012-09-29"
   s.description = "        Define indexes and foreign keys for your Rails app\n        in one place using an easy-to-read DSL,\n        then run a rake task to bring your database up-to-date.\n"
   s.email = "pj@illuminatedcomputing.com"
   s.extra_rdoc_files = [

data/lib/db_leftovers/constraint.rb

@@ -8,6 +8,16 @@ module DBLeftovers
       @on_table = on_table.to_s
       @check = check
     end
-  end
 
+    def equals(other)
+      other.constraint_name == constraint_name and
+      other.on_table == on_table and
+      other.check == check
+    end
+
+    def to_s
+      "<#{@constraint_name}: #{@on_table} CHECK (#{@check})>"
+    end
+
+  end
 end

data/lib/db_leftovers/database_interface.rb

@@ -3,6 +3,7 @@ module DBLeftovers
   class DatabaseInterface
 
     def lookup_all_indexes
+      # TODO: Constraint it to the database for the current Rails project:
       ret = {}
       sql = <<-EOQ
           SELECT  ix.indexrelid,
@@ -49,18 +50,48 @@ module DBLeftovers
 
 
     def lookup_all_foreign_keys
+      # confdeltype: a=nil, c=cascade, n=null
       ret = {}
+      # TODO: Support multi-column foreign keys:
+      # TODO: Constraint it to the database for the current Rails project:
       sql = <<-EOQ
-          SELECT  t.constraint_name, t.table_name, k.column_name, t.constraint_type, c.table_name, c.column_name
-          FROM    information_schema.table_constraints t,
-                  information_schema.constraint_column_usage c,
-                  information_schema.key_column_usage k
-          WHERE   t.constraint_name = c.constraint_name
-          AND     k.constraint_name = c.constraint_name
-          AND     t.constraint_type = 'FOREIGN KEY'
+          SELECT  c.conname,
+                  t1.relname,
+                  a1.attname,
+                  t2.relname,
+                  a2.attname,
+                  c.confdeltype
+          FROM    pg_catalog.pg_constraint c,
+                  pg_catalog.pg_class t1,
+                  pg_catalog.pg_class t2,
+                  pg_catalog.pg_attribute a1,
+                  pg_catalog.pg_attribute a2,
+                  pg_catalog.pg_namespace n1,
+                  pg_catalog.pg_namespace n2
+          WHERE   c.conrelid = t1.oid
+          AND     c.confrelid = t2.oid
+          AND     c.contype = 'f'
+          AND     a1.attrelid = t1.oid
+          AND     a1.attnum = ANY(c.conkey)
+          AND     a2.attrelid = t2.oid
+          AND     a2.attnum = ANY(c.confkey)
+          AND     t1.relkind = 'r'
+          AND     t2.relkind = 'r'
+          AND     n1.oid = t1.relnamespace
+          AND     n2.oid = t2.relnamespace
+          AND     n1.nspname NOT IN ('pg_catalog', 'pg_toast')
+          AND     n2.nspname NOT IN ('pg_catalog', 'pg_toast')
+          AND     pg_catalog.pg_table_is_visible(t1.oid)
+          AND     pg_catalog.pg_table_is_visible(t2.oid)
       EOQ
-      ActiveRecord::Base.connection.select_rows(sql).each do |constr_name, from_table, from_column, constr_type, to_table, to_column|
-        ret[constr_name] = ForeignKey.new(constr_name, from_table, from_column, to_table, to_column)
+      ActiveRecord::Base.connection.select_rows(sql).each do |constr_name, from_table, from_column, to_table, to_column, del_type|
+        del_type = case del_type
+                   when 'a'; nil
+                   when 'c'; :cascade
+                   when 'n'; :set_null
+                   else; raise "Unknown del type: #{del_type}"
+                   end
+        ret[constr_name] = ForeignKey.new(constr_name, from_table, from_column, to_table, to_column, :on_delete => del_type)
       end
       return ret
     end

data/lib/db_leftovers/dsl.rb

@@ -80,9 +80,14 @@ module DBLeftovers
       # First create any new foreign keys:
       @foreign_keys_by_table.each do |table_name, fks|
         fks.each do |fk|
-          if foreign_key_exists?(fk)
+          case foreign_key_status(fk)
+          when STATUS_EXISTS
             puts "Foreign Key already exists: #{fk.constraint_name} on #{fk.from_table}" if @verbose
-          else
+          when STATUS_CHANGED
+            @db.execute_drop_foreign_key(fk.constraint_name, fk.from_table, fk.from_column)
+            @db.execute_add_foreign_key(fk)
+            puts "Dropped & re-created foreign key: #{fk.constraint_name} on #{fk.from_table}"
+          when STATUS_NEW
             @db.execute_add_foreign_key(fk)
             puts "Created foreign key: #{fk.constraint_name} on #{fk.from_table}"
           end
@@ -103,9 +108,14 @@ module DBLeftovers
       # First create any new constraints:
       @constraints_by_table.each do |table_name, chks|
         chks.each do |chk|
-          if constraint_exists?(chk)
+          case constraint_status(chk)
+          when STATUS_EXISTS
             puts "Constraint already exists: #{chk.constraint_name} on #{chk.on_table}" if @verbose
-          else
+          when STATUS_CHANGED
+            @db.execute_drop_constraint(constraint_name, chk.on_table)
+            @db.execute_add_constraint(chk)
+            puts "Dropped & re-created CHECK constraint: #{chk.constraint_name} on #{chk.on_table}"
+          when STATUS_NEW
             @db.execute_add_constraint(chk)
             puts "Created CHECK constraint: #{chk.constraint_name} on #{chk.on_table}"
           end
@@ -152,12 +162,22 @@ module DBLeftovers
       end
     end
 
-    def foreign_key_exists?(fk)
-      @old_foreign_keys[fk.constraint_name]
+    def foreign_key_status(fk)
+      old = @old_foreign_keys[fk.constraint_name]
+      if old
+        return old.equals(fk) ? STATUS_EXISTS : STATUS_CHANGED
+      else
+        return STATUS_NEW
+      end
     end
 
-    def constraint_exists?(chk)
-      @old_constraints[chk.constraint_name]
+    def constraint_status(chk)
+      old = @old_constraints[chk.constraint_name]
+      if old
+        return old.equals(chk) ? STATUS_EXISTS : STATUS_CHANGED
+      else
+        return STATUS_NEW
+      end
     end
 
     def name_constraint(from_table, from_column)

data/lib/db_leftovers/foreign_key.rb

@@ -31,7 +31,12 @@ module DBLeftovers
       other.from_column == from_column and
       other.to_table == to_table and
       other.to_column == to_column and
-      other.opts == opts
+      other.set_null == set_null and
+      other.cascade == cascade
+    end
+
+    def to_s
+      "<#{@constraint_name}: from #{@from_table}.#{@from_column} to #{@to_table}.#{@to_column} #{if @set_null; "ON DELETE SET NULL "; elsif @cascade; "ON DELETE CASCADE "; else ""; end}>"
     end
 
   end

data/spec/db_leftovers_spec.rb

@@ -13,15 +13,13 @@ class DBLeftovers::DatabaseInterface
   end
 
   def execute_sql(sql)
-    @@sqls << sql
+    @@sqls << DBLeftovers::DatabaseInterface.normal_whitespace(sql)
   end
 
   def self.saw_sql(sql)
     # puts sqls.join("\n\n\n")
     # Don't fail if only the whitespace is different:
-    sqls.map{|x| x.gsub(/\s+/m, ' ').strip}.include?(
-      sql.gsub(/\s+/m, ' ').strip
-    )
+    sqls.include?(normal_whitespace(sql))
   end
 
   def self.starts_with(indexes=[], foreign_keys=[], constraints=[])
@@ -43,6 +41,12 @@ class DBLeftovers::DatabaseInterface
     @@constraints
   end
 
+  private
+
+  def self.normal_whitespace(sql)
+    sql.gsub(/\s/m, ' ').gsub(/  +/, ' ').strip
+  end
+
 end
 
 RSpec::Matchers.define :have_seen_sql do |sql|
@@ -292,6 +296,26 @@ describe DBLeftovers do
 
 
 
+  it "should create foreign keys when they have been redefined" do
+    DBLeftovers::DatabaseInterface.starts_with([], [
+      DBLeftovers::ForeignKey.new('fk_books_shelf_id', 'books', 'shelf_id', 'shelves', 'id'),
+      DBLeftovers::ForeignKey.new('fk_books_author_id', 'books', 'author_id', 'authors', 'id')
+    ])
+    DBLeftovers::Definition.define do
+      table :books do
+        foreign_key :shelf_id, :shelves, :id, :on_delete => :cascade
+        foreign_key :author_id, :authors, :id, :on_delete => :set_null
+      end
+    end
+    DBLeftovers::DatabaseInterface.sqls.should have(4).items
+    DBLeftovers::DatabaseInterface.sqls[0].should =~ /ALTER TABLE books DROP CONSTRAINT fk_books_shelf_id/
+    DBLeftovers::DatabaseInterface.sqls[1].should =~ /ALTER TABLE books ADD CONSTRAINT fk_books_shelf_id/
+    DBLeftovers::DatabaseInterface.sqls[2].should =~ /ALTER TABLE books DROP CONSTRAINT fk_books_author_id/
+    DBLeftovers::DatabaseInterface.sqls[3].should =~ /ALTER TABLE books ADD CONSTRAINT fk_books_author_id/
+  end
+
+
+
   it "should support creating multi-column indexes" do
     DBLeftovers::DatabaseInterface.starts_with
     DBLeftovers::Definition.define do

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: db_leftovers
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-09-27 00:00:00.000000000 Z
+date: 2012-09-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -139,7 +139,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2522100486146355365
+      hash: 258395923
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements: