db_leftovers 0.7.0 → 0.8.0

data/README.html

@@ -1,10 +1,10 @@
 <h1>db_leftovers</h1>
 
-<p>db_leftovers lets you define indexes and foreign keys for your Rails app
+<p>db_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
 in one place using an easy-to-read DSL,
 then run a rake task to bring your database up-to-date.
-I wrote this because I didn't want indexes and foreign keys scattered throughout my migrations or buried in my <code>schema.rb</code>, and I wanted a command I could run to ensure that they matched across my development, test, staging, and production databases.
-This was particularly a problem for Heroku projects, because <code>db:push</code> and <code>db:pull</code> do not transfer your foreign keys.
+I wrote this because I didn't want indexes and constraints scattered throughout my migrations or buried in my <code>schema.rb</code>, and I wanted a command I could run to ensure that they matched across my development, test, staging, and production databases.
+This was particularly a problem for Heroku projects, because <code>db:push</code> and <code>db:pull</code> do not transfer your foreign keys or other constraints.
 But now that it's written, I'm finding it useful on non-Heroku projects as well.</p>
 
 <p>At present db_leftovers works only on PostgreSQL databases,
@@ -12,9 +12,9 @@ but it could easily be extended to cover other RDBMSes.</p>
 
 <h2>Configuration File</h2>
 
-<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and foreign keys you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. There are only a few methods:</p>
+<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. There are only a few methods:</p>
 
-<h3>index table_name, columns, [opts]</h3>
+<h3>index(table_name, columns, [opts])</h3>
 
 <p>This ensures that you have an index on the given table and column(s). The <code>columns</code> parameter can be either a string or a list of strings. Opts is a hash with the following possible keys:</p>
 
@@ -31,38 +31,48 @@ index :books, [:publisher_id, :published_at]
 index :books, :isbn, :unique =&gt; true
 </code></pre>
 
-<h3>foreign_key from_table, from_column, to_table, [to_column, [opts]]</h3>
+<h3>foreign_key(from_table, from_column, to_table, [to_column, [opts]])</h3>
 
 <p>This ensures that you have a foreign key relating the given tables and columns.
-All parameters are strings except <code>opts</code>, which is a hash.
+All parameters are strings/symbols except <code>opts</code>, which is a hash.
 If you don't pass anything for <code>opts</code>, you can leave off the <code>to_column</code> parameter, and it will default to <code>:id</code>.
-These options are supported:</p>
+The only option that is supported is <code>:on_delete</code>, which may have any of these values:</p>
 
 <ul>
+<li><code>nil</code> Indicates that attempting to delete the referenced row should fail (the default).</li>
 <li><code>:set_null</code> Indicates that the foreign key should be set to null if the referenced row is deleted.</li>
 <li><code>:cascade</code> Indicates that the referencing row should be deleted if the referenced row is deleted.</li>
 </ul>
 
-<p>These options are mutually exclusive. They should probably be consolidated into a single option like <code>:on_delete</code>.</p>
-
 <h4>Examples</h4>
 
 <pre><code>foreign_key :books, :author_id, :authors, :id
 foreign_key :books, :publisher_id, :publishers
-foreign_key :pages, :book_id, :books, :id, :cascade =&gt; true
+foreign_key :pages, :book_id, :books, :id, :on_delete =&gt; :cascade
 </code></pre>
 
-<h3>table table_name, &amp;block</h3>
+<h3>check(constraint_name, on_table, expression)</h3>
+
+<p>This ensures that you have a CHECK constraint on the given table with the given name and expression.
+All parameters are strings or symbols.</p>
+
+<h4>Examples</h4>
+
+<pre><code>check :books, :books_have_positive_pages, 'page_count &gt; 0'
+</code></pre>
+
+<h3>table(table_name, &amp;block)</h3>
 
 <p>The <code>table</code> call is just a convenience so you can group all a table's indexes and foreign keys together and not keep repeating the table name. You use it like this:</p>
 
 <pre><code>table :books do
   index :author_id
   foreign_key :publisher_id, :publishers
+  check :books_have_positive_pages, 'page_count &gt; 0'
 end
 </code></pre>
 
-<p>You can repeat <code>table</code> calls for the same table several times if you like. This lets you put your indexes in one place and your foreign keys in another.</p>
+<p>You can repeat <code>table</code> calls for the same table several times if you like. This lets you put your indexes in one place and your foreign keys in another, for example.</p>
 
 <h2>Running db_leftovers</h2>
 
@@ -71,17 +81,39 @@ end
 <pre><code>rake db:leftovers
 </code></pre>
 
+<p>db_leftovers will notice whenever an index, foreign key, or CHECK constraint needs to be added, dropped, or changed.
+It will even catch cases where the name of the managed object is the same, but its attributes have changed.
+For instance, if you previously required books to have at least 1 page, but now you are introducing a "pamphlet"
+and want books to have at least 100 pages, you can change your config file to say:</p>
+
+<pre><code>check :books, :books_have_positive_pages, 'page_count &gt;= 100'
+</code></pre>
+
+<p>and db_leftovers will notice the changed expression. It will drop and re-add the constraint.</p>
+
+<p>One caveat, however: we pull the current expression from the database, and sometimes Postgres does things like
+add type conversions. If instance, suppose you said <code>check :users, :email_length, 'LENGTH(email) &gt; 2'</code>.
+The second time you run db_leftovers, it will read the expression from Postgres and get <code>LENGTH((email)::text) &gt; 2</code>,
+and so it will drop and re-create the constraint.
+It will drop and re-create it every time you run the rake task.
+To get around this, make sure your config file uses the same expression as printed by db_leftovers in the rake output.
+This can also happen for index WHERE clauses, fixable by a similar workaround.</p>
+
+<p>To print messages even about indexes/foreign keys/constraints that haven't changed, you can say:</p>
+
+<pre><code>rake db:leftovers VERBOSE=true
+</code></pre>
+
+<p>or</p>
+
+<pre><code>rake db:leftovers DB_LEFTOVERS_VERBOSE=true
+</code></pre>
+
 <h2>Known Issues</h2>
 
 <ul>
-<li><p>db_leftovers only supports PostgreSQL databases.
-If you want to add support for something else, just send me a pull request!</p></li>
-<li><p>db_leftovers will not notice if an index/foreign key definition changes.
-Right now it only checks for existence/non-existence.</p></li>
-<li><p>If your database is mostly up-to-date, then running the Rake task will spam
-you with messages about how this index and that foreign key already exist.
-These should be hidden by default and shown only if you request a higher
-verbosity.</p></li>
+<li>db_leftovers only supports PostgreSQL databases.
+If you want to add support for something else, just send me a pull request!</li>
 </ul>
 
 <h2>Contributing to db_leftovers</h2>
@@ -98,5 +130,5 @@ verbosity.</p></li>
 
 <h2>Copyright</h2>
 
-<p>Copyright (c) 2012 Paul A. Jungwirth. See LICENSE.txt for
-further details.</p>
+<p>Copyright (c) 2012 Paul A. Jungwirth.
+See LICENSE.txt for further details.</p>

data/README.md

@@ -78,6 +78,23 @@ db\_leftovers comes with a Rake task named `db:leftovers`. So you can update you
 
     rake db:leftovers
 
+db\_leftovers will notice whenever an index, foreign key, or CHECK constraint needs to be added, dropped, or changed.
+It will even catch cases where the name of the managed object is the same, but its attributes have changed.
+For instance, if you previously required books to have at least 1 page, but now you are introducing a "pamphlet"
+and want books to have at least 100 pages, you can change your config file to say:
+
+    check :books, :books_have_positive_pages, 'page_count >= 100'
+
+and db\_leftovers will notice the changed expression. It will drop and re-add the constraint.
+
+One caveat, however: we pull the current expression from the database, and sometimes Postgres does things like
+add type conversions. If instance, suppose you said `check :users, :email_length, 'LENGTH(email) > 2'`.
+The second time you run db\_leftovers, it will read the expression from Postgres and get `LENGTH((email)::text) > 2`,
+and so it will drop and re-create the constraint.
+It will drop and re-create it every time you run the rake task.
+To get around this, make sure your config file uses the same expression as printed by db\_leftovers in the rake output.
+This can also happen for index WHERE clauses, fixable by a similar workaround.
+
 To print messages even about indexes/foreign keys/constraints that haven't changed, you can say:
 
     rake db:leftovers VERBOSE=true
@@ -87,26 +104,14 @@ or
     rake db:leftovers DB_LEFTOVERS_VERBOSE=true
 
 
+
+
 Known Issues
 ------------
 
 * db\_leftovers only supports PostgreSQL databases.
   If you want to add support for something else, just send me a pull request!
 
-* db\_leftovers will not notice if an foreign key/constraint definition changes.
-  Right now it only checks for existence/non-existence.
-  You can get around this by adding a version number to your constraint names,
-  so if you want to force books to have at least 12 pages, you can say this:
-
-  `check :books_have_positive_pages2, 'page_count >= 12'`
-
-  Then the old constraint will be dropped and the new one will be added.
-
-  However, db\_leftovers *does* check for index definitions, so if you
-  make an existing index unique, add a column, remove a WHERE clause, or
-  anything else, it will notice and drop and re-create the index.
-  I'm working on doing the same thing for foreign keys/constraints,
-  but it's not done just yet.
   
 
 

data/TODO

@@ -1,7 +1,3 @@
-- Write the README
-- Support altering index and foreign keys if their definition changes
-- don't show "{index,fk} already exists" unless you set the verbosity higher
-
 - Abbreviated notation for conventional foreign keys:
 
       table :books do
@@ -16,3 +12,14 @@
   In other words, the params are foreign_key(from_table, [from_column], to_table, [to_column], [opts]).
 
 
+- Print the actual WHERE-clause/CHECK expression used by postgres whenever the index/constraint is created (even the first time).
+
+- Refactor DatabaseInterface so all methods are instance methods and Definition/DSL uses an instance, set according to the Rails database adapter and also settable to a test subclass, so we can change the definition from test to test.
+
+- Add optional tests that use a real Postgres/MySQL/SQLite if available.
+  - Run all the same tests, but against a real DatabaseInterface instance.
+
+- Make sure everything works if the current db user has access to multiple databases: only run the code on the db given in the Rails database.yml.
+
+- Support MySQL
+

data/VERSION

@@ -1 +1 @@
-0.7.0
+0.8.0

data/db_leftovers.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = "db_leftovers"
-  s.version = "0.7.0"
+  s.version = "0.8.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Paul A. Jungwirth"]
-  s.date = "2012-09-29"
+  s.date = "2012-09-30"
   s.description = "        Define indexes and foreign keys for your Rails app\n        in one place using an easy-to-read DSL,\n        then run a rake task to bring your database up-to-date.\n"
   s.email = "pj@illuminatedcomputing.com"
   s.extra_rdoc_files = [
@@ -25,6 +25,7 @@ Gem::Specification.new do |s|
     "LICENSE.txt",
     "README.md",
     "Rakefile",
+    "TODO",
     "VERSION",
     "db_leftovers.gemspec",
     "lib/db_leftovers.rb",

data/lib/db_leftovers/database_interface.rb

@@ -35,7 +35,7 @@ module DBLeftovers
           ORDER BY t.relname, i.relname
       EOQ
       ActiveRecord::Base.connection.select_rows(sql).each do |indexrelid, indrelid, table_name, index_name, is_unique, column_numbers, where_clause|
-        where_clause = where_clause.gsub(/^\((.*)\)$/, '\1') if where_clause    # strip surrounding parens
+        where_clause = remove_outer_parens(where_clause) if where_clause
         ret[index_name] = Index.new(
           table_name,
           column_names_for_index(indrelid, column_numbers.split(",")),
@@ -97,17 +97,23 @@ module DBLeftovers
     end
 
     def lookup_all_constraints
+      # TODO: Constraint it to the database for the current Rails project:
       ret = {}
       sql = <<-EOQ
-          SELECT  t.constraint_name, t.table_name
-          FROM    information_schema.table_constraints t
-          WHERE   t.constraint_type = 'CHECK'
-          AND     EXISTS (SELECT  1
-                          FROM    information_schema.constraint_column_usage c
-                          WHERE   t.constraint_name = c.constraint_name)
+          SELECT  c.conname,
+                  t.relname
+          FROM    pg_catalog.pg_constraint c,
+                  pg_catalog.pg_class t,
+                  pg_catalog.pg_namespace n
+          WHERE   c.contype = 'c'
+          AND     c.conrelid = t.oid
+          AND     t.relkind = 'r'
+          AND     n.oid = t.relnamespace
+          AND     n.nspname NOT IN ('pg_catalog', 'pg_toast')
+          AND     pg_catalog.pg_table_is_visible(t.oid)
       EOQ
-      ActiveRecord::Base.connection.select_rows(sql).each do |constr_name, on_table|
-        ret[constr_name] = Constraint.new(constr_name, on_table, nil)
+      ActiveRecord::Base.connection.select_rows(sql).each do |constr_name, on_table, check_expr|
+        ret[constr_name] = Constraint.new(constr_name, on_table, remove_outer_parens(check_expr))
       end
       return ret
     end
@@ -175,6 +181,9 @@ module DBLeftovers
       end
     end
 
+    def remove_outer_parens(str)
+      str.gsub(/^\((.*)\)$/, '\1')
+    end
 
 
   end

data/lib/db_leftovers/dsl.rb

@@ -112,9 +112,9 @@ module DBLeftovers
           when STATUS_EXISTS
             puts "Constraint already exists: #{chk.constraint_name} on #{chk.on_table}" if @verbose
           when STATUS_CHANGED
-            @db.execute_drop_constraint(constraint_name, chk.on_table)
+            @db.execute_drop_constraint(chk.constraint_name, chk.on_table)
             @db.execute_add_constraint(chk)
-            puts "Dropped & re-created CHECK constraint: #{chk.constraint_name} on #{chk.on_table}"
+            puts "Dropped & re-created CHECK constraint: #{chk.constraint_name} on #{chk.on_table} as #{chk.check}"
           when STATUS_NEW
             @db.execute_add_constraint(chk)
             puts "Created CHECK constraint: #{chk.constraint_name} on #{chk.on_table}"

data/spec/db_leftovers_spec.rb

@@ -425,4 +425,56 @@ describe DBLeftovers do
     }.should raise_error(RuntimeError, "`:cascade => true` should now be `:on_delete => :cascade`")
   end
 
+  it "should create CHECK constraints on an empty database" do
+    DBLeftovers::DatabaseInterface.starts_with([], [], [])
+    DBLeftovers::Definition.define do
+      check :books, :books_have_positive_pages, 'pages_count > 0'
+    end
+    DBLeftovers::DatabaseInterface.sqls.should have(1).item
+    DBLeftovers::DatabaseInterface.should have_seen_sql <<-EOQ
+        ALTER TABLE books ADD CONSTRAINT books_have_positive_pages CHECK (pages_count > 0)
+    EOQ
+  end
+
+  it "should create CHECK constraints inside a table block" do
+    DBLeftovers::DatabaseInterface.starts_with([], [], [])
+    DBLeftovers::Definition.define do
+      table :books do
+        check :books_have_positive_pages, 'pages_count > 0'
+      end
+    end
+    DBLeftovers::DatabaseInterface.sqls.should have(1).item
+    DBLeftovers::DatabaseInterface.should have_seen_sql <<-EOQ
+        ALTER TABLE books ADD CONSTRAINT books_have_positive_pages CHECK (pages_count > 0)
+    EOQ
+  end
+
+  it "should remove obsolete CHECK constraints" do
+    DBLeftovers::DatabaseInterface.starts_with([], [], [
+      DBLeftovers::Constraint.new(:books_have_positive_pages, :books, 'pages_count > 0')
+    ])
+    DBLeftovers::Definition.define do
+    end
+    DBLeftovers::DatabaseInterface.sqls.should have(1).item
+    DBLeftovers::DatabaseInterface.should have_seen_sql <<-EOQ
+        ALTER TABLE books DROP CONSTRAINT books_have_positive_pages
+    EOQ
+  end
+
+  it "should drop and re-create changed CHECK constraints" do
+    DBLeftovers::DatabaseInterface.starts_with([], [], [
+      DBLeftovers::Constraint.new(:books_have_positive_pages, :books, 'pages_count > 0')
+    ])
+    DBLeftovers::Definition.define do
+      check :books, :books_have_positive_pages, 'pages_count > 12'
+    end
+    DBLeftovers::DatabaseInterface.sqls.should have(2).items
+    DBLeftovers::DatabaseInterface.should have_seen_sql <<-EOQ
+        ALTER TABLE books DROP CONSTRAINT books_have_positive_pages
+    EOQ
+    DBLeftovers::DatabaseInterface.should have_seen_sql <<-EOQ
+        ALTER TABLE books ADD CONSTRAINT books_have_positive_pages CHECK (pages_count > 12)
+    EOQ
+  end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: db_leftovers
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.8.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-09-29 00:00:00.000000000 Z
+date: 2012-09-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -109,6 +109,7 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
+- TODO
 - VERSION
 - db_leftovers.gemspec
 - lib/db_leftovers.rb
@@ -123,7 +124,6 @@ files:
 - spec/db_leftovers_spec.rb
 - spec/spec_helper.rb
 - README.html
-- TODO
 homepage: http://github.com/pjungwir/db_leftovers
 licenses:
 - MIT
@@ -139,7 +139,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 258395923
+      hash: 856836431
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements: