db_leftovers 1.0.0 → 1.0.1

data/README.html

@@ -1,29 +1,51 @@
 <h1>db_leftovers</h1>
 
-<p>db_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
+<p>Db_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
 in one place using an easy-to-read DSL,
 then run a rake task to bring your database up-to-date.
-Whenever you edit the DSL, you can re-run the rake task and db_leftovers will alter your database accorindgly.
+Whenever you edit the DSL, you can re-run the rake task and db_leftovers will alter your database accordingly.
 This is useful because of the following limitations in vanilla Rails:</p>
 
 <ul>
 <li>There are no built-in migration methods to create foreign keys or CHECK constraints.</li>
 <li>Even if created, foreign keys and CHECK constraints won't appear in your schema.rb.</li>
+<li>The built-in <code>add_index</code> method doesn't support WHERE clauses on your indexes.</li>
 <li>If you're using Heroku, <code>db:push</code> and <code>db:pull</code> won't transfer your foreign keys and CHECK constraints.</li>
 <li>Creating indexes in your migrations makes it hard to manage them.</li>
 </ul>
 
-<p>That last point deserves some elaboration. Using <code>create_index</code> in your migrations is bug-prone because without rare developer discipline (My rule is "never change a migration after a <code>git push</code>, but I haven't seen this followed elsewhere."), you wind up missing indexes in some environments. It also means you don't have a central place to see all your indexes so you can analyze which are needed. With db_leftovers, you can rest assured that each environment conforms to a definition that is easy to read and checked into version control.</p>
+<p>That last point deserves some elaboration. First, using <code>add_index</code> in your migrations is bug-prone because without rare developer discipline, you wind up missing indexes in some environments. (My rule is "never change a migration after a <code>git push</code>," but I haven't seen this followed elsewhere, and there is nothing that automatically enforces it.) Also, since missing indexes don't cause errors, it's easy to be missing one and not notice until users start complaining about performance.</p>
+
+<p>Second, Scattering <code>add_index</code> methods throughout migrations also doesn't match the workflow of optimizing database queries. Hopefully you create appropriate indexes when you set up your tables originally, but in practice you often need to go back later and add/remove indexes according to your database usage patterns. Or you just forget the indexes, because you're thinking about modeling the data, not optimizing the queries.
+It's easier to vet and analyze database indexes if you can see them all in one place,
+and db_leftovers lets you do that easily.
+And since you can rest assured that each environment conforms to the same definition, you don't need to second-guess yourself about indexes that are present in development but missing in production.
+Db_leftovers lets you rest assured that each environment conforms to a definition that is easy to read and checked into version control.</p>
 
 <p>At present db_leftovers supports PostgreSQL and MySQL, although since MySQL doesn't support index WHERE clauses or CHECK constraints, using that functionality will raise errors. (If you need to share the same definitions across Postgres and MySQL, you can run arbitrary Ruby code inside the DSL to avoid defining unsupported objects when run against MySQL.)</p>
 
 <h2>Configuration File</h2>
 
-<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. There are only a few methods:</p>
+<p>db_leftovers reads a file named <code>config/db_leftovers.rb</code> to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like <code>config/routes.rb</code>. It should look something like this:</p>
+
+<pre><code>DBLeftovers::Definition.define do
+
+  table :users do
+    index :email, :unique =&gt; true
+    check :registered_at_unless_guest, "role_name = 'guest' OR registered_at IS NOT NULL"
+  end
+
+  foreign_key :orders, :users
+
+  # . . 
+end
+</code></pre>
+
+<p>Within the DSL file, the following methods are supported:</p>
 
 <h3>index(table_name, columns, [opts])</h3>
 
-<p>This ensures that you have an index on the given table and column(s). The <code>columns</code> parameter can be either a string or a list of strings. Opts is a hash with the following possible keys:</p>
+<p>This ensures that you have an index on the given table and column(s). The <code>columns</code> parameter can be either a string/symbol or a list of strings/symbols. Opts is a hash with the following possible keys:</p>
 
 <ul>
 <li><p><code>:name</code> The name of the index. Defaults to <code>index_</code><em>table_name</em><code>_on_</code><em>column_names</em>, like the <code>add_index</code> method from Rails migrations.</p></li>
@@ -124,10 +146,30 @@ MySQL doesn't have this problem because it doesn't support CHECK constraints or
 <pre><code>rake db:leftovers DB_LEFTOVERS_VERBOSE=true
 </code></pre>
 
+<h2>Capistrano Integration</h2>
+
+<p>I recommend running <code>rake db:migrate</code> any time you deploy, and then running <code>rake db:leftovers</code> after that. Here is what you need in your <code>config/deploy.rb</code> to make that happen:</p>
+
+<pre><code>set :rails_env, "production"
+
+namespace :db do
+  desc "Set up constraints and indexes"
+  task :leftovers do
+    run("cd #{deploy_to}/current &amp;&amp; bundle exec rake db:leftovers RAILS_ENV=#{rails_env}")  
+  end
+end
+
+after :deploy, 'deploy:migrate'
+after 'deploy:migrate', 'db:leftovers'
+</code></pre>
+
+<p>You could also change this code to <em>not</em> run migrations after each deploy, if you like. But in that case I'd recommend not running db:leftovers until after the latest migrations (if any), since new entries in the DSL are likely to reference newly-created tables/columns.</p>
+
 <h2>Known Issues</h2>
 
 <ul>
-<li>When db_leftovers interrogates your database for the currently-defined indexes et cetera, it doesn't filter things by the current database name. So if you have mutliple Rails projects all accessible to the same user, you'll wind up changing more than you like (probably by DROPing things).</li>
+<li><p>When db_leftovers interrogates your database for the currently-defined indexes et cetera, it doesn't filter things by the current database name. So if you have mutliple Rails projects all accessible to the same user, you'll wind up changing more than you like (probably by DROPing things).</p></li>
+<li><p>It'd be nice to have another Rake task that will read your database and generate a new <code>db_leftovers.rb</code> file, to help people migrating existing projects that already have lots of tables.</p></li>
 </ul>
 
 <h2>Contributing to db_leftovers</h2>

data/README.md

@@ -1,29 +1,50 @@
 db\_leftovers
 =============
 
-db\_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
+Db\_leftovers lets you define indexes, foreign keys, and CHECK constraints for your Rails app
 in one place using an easy-to-read DSL,
 then run a rake task to bring your database up-to-date.
-Whenever you edit the DSL, you can re-run the rake task and db\_leftovers will alter your database accorindgly.
+Whenever you edit the DSL, you can re-run the rake task and db\_leftovers will alter your database accordingly.
 This is useful because of the following limitations in vanilla Rails:
 
   * There are no built-in migration methods to create foreign keys or CHECK constraints.
   * Even if created, foreign keys and CHECK constraints won't appear in your schema.rb.
+  * The built-in `add_index` method doesn't support WHERE clauses on your indexes.
   * If you're using Heroku, `db:push` and `db:pull` won't transfer your foreign keys and CHECK constraints.
   * Creating indexes in your migrations makes it hard to manage them.
   
-That last point deserves some elaboration. Using `create_index` in your migrations is bug-prone because without rare developer discipline (My rule is "never change a migration after a `git push`, but I haven't seen this followed elsewhere."), you wind up missing indexes in some environments. It also means you don't have a central place to see all your indexes so you can analyze which are needed. With db\_leftovers, you can rest assured that each environment conforms to a definition that is easy to read and checked into version control.
+That last point deserves some elaboration. First, using `add_index` in your migrations is bug-prone because without rare developer discipline, you wind up missing indexes in some environments. (My rule is "never change a migration after a `git push`," but I haven't seen this followed elsewhere, and there is nothing that automatically enforces it.) Also, since missing indexes don't cause errors, it's easy to be missing one and not notice until users start complaining about performance.
+
+Second, scattering `add_index` methods throughout migrations also doesn't match the workflow of optimizing database queries. Hopefully you create appropriate indexes when you set up your tables originally, but in practice you often need to go back later and add/remove indexes according to your database usage patterns. Or you just forget the indexes, because you're thinking about modeling the data, not optimizing the queries.
+It's easier to vet and analyze database indexes if you can see them all in one place,
+and db\_leftovers lets you do that easily.
+And since you can rest assured that each environment conforms to the same definition, you don't need to second-guess yourself about indexes that are present in development but missing in production.
+Db\_leftovers gives you confidence that your database matches a definition that is easy to read and checked into version control.
 
 At present db\_leftovers supports PostgreSQL and MySQL, although since MySQL doesn't support index WHERE clauses or CHECK constraints, using that functionality will raise errors. (If you need to share the same definitions across Postgres and MySQL, you can run arbitrary Ruby code inside the DSL to avoid defining unsupported objects when run against MySQL.)
 
 Configuration File
 ------------------
 
-db\_leftovers reads a file named `config/db_leftovers.rb` to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like `config/routes.rb`. There are only a few methods:
+db\_leftovers reads a file named `config/db_leftovers.rb` to find out which indexes and constraints you want in your database. This file is a DSL implemented in Ruby, sort of like `config/routes.rb`. It should look something like this:
+
+    DBLeftovers::Definition.define do
+
+      table :users do
+        index :email, :unique => true
+        check :registered_at_unless_guest, "role_name = 'guest' OR registered_at IS NOT NULL"
+      end
+
+      foreign_key :orders, :users
+
+      # . . 
+    end
+
+Within the DSL file, the following methods are supported:
 
 ### index(table\_name, columns, [opts])
 
-This ensures that you have an index on the given table and column(s). The `columns` parameter can be either a string or a list of strings. Opts is a hash with the following possible keys:
+This ensures that you have an index on the given table and column(s). The `columns` parameter can be either a string/symbol or a list of strings/symbols. Opts is a hash with the following possible keys:
 
 * `:name` The name of the index. Defaults to `index_`*table\_name*`_on_`*column\_names*, like the `add_index` method from Rails migrations.
 
@@ -60,7 +81,7 @@ With implicit column names:
     foreign_key :books, :co_author_id, :authors
     foreign_key :books, :co_author_id, :authors, :on_delete => :cascade
 
-### check(constraint\_name, on\_table, expression)
+### check(on\_table, constraint\_name, expression)
 
 This ensures that you have a CHECK constraint on the given table with the given name and expression.
 All parameters are strings or symbols.
@@ -99,7 +120,7 @@ and want books to have at least 100 pages, you can change your config file to sa
 and db\_leftovers will notice the changed expression. It will drop and re-add the constraint.
 
 One caveat, however: we pull the current expression from the database, and sometimes Postgres does things like
-add type conversions. If instance, suppose you said `check :users, :email_length, 'LENGTH(email) > 2'`.
+add type conversions and extra parentheses. If instance, suppose you said `check :users, :email_length, 'LENGTH(email) > 2'`.
 The second time you run db\_leftovers, it will read the expression from Postgres and get `LENGTH((email)::text) > 2`,
 and so it will drop and re-create the constraint.
 It will drop and re-create it every time you run the rake task.
@@ -118,12 +139,33 @@ or
 
 
 
+Capistrano Integration
+----------------------
+
+I recommend running `rake db:migrate` any time you deploy, and then running `rake db:leftovers` after that. Here is what you need in your `config/deploy.rb` to make that happen:
+
+    set :rails_env, "production"
+
+    namespace :db do
+      desc "Set up constraints and indexes"
+      task :leftovers do
+        run("cd #{deploy_to}/current && bundle exec rake db:leftovers RAILS_ENV=#{rails_env}")  
+      end
+    end
+
+    after :deploy, 'deploy:migrate'
+    after 'deploy:migrate', 'db:leftovers'
+
+You could also change this code to *not* run migrations after each deploy, if you like. But in that case I'd recommend not running db:leftovers until after the latest migrations (if any), since new entries in the DSL are likely to reference newly-created tables/columns.
+
+
+
 Known Issues
 ------------
 
-* When db\_leftovers interrogates your database for the currently-defined indexes et cetera, it doesn't filter things by the current database name. So if you have mutliple Rails projects all accessible to the same user, you'll wind up changing more than you like (probably by DROPing things).
+* Multi-column foreign keys are not supported. This shouldn't be a problem for a Rails project, unless you are using a legacy database. If you need this functionality, let me know and I'll look into adding it.
 
-  
+* It'd be nice to have another Rake task that will read your database and generate a new `db_leftovers.rb` file, to help people migrating existing projects that already have lots of tables.
 
 
 Contributing to db\_leftovers

data/TODO

@@ -1,2 +1,11 @@
-- Make sure everything works if the current db user has access to multiple databases: only run the code on the db given in the Rails database.yml.
+- It'd be nice to have another Rake task that will read your database and generate a new `db_leftovers.rb` file, to help people migrating existing projects that already have lots of tables.
+
+- Support multi-column foreign keys.
+
+- Support custom names for foreign keys. (But then watch out excluding these from the list of indexes on MySQL!)
+
+- Support CREATE INDEX CONCURRENTLY on Postgres as an option to Rake or the `define` call.
+
+- Support NOT VALID for Postgres 9.1+ foreign keys and Postgres 9.2+ CHECK constraints.
+
 

data/VERSION

@@ -1 +1 @@
-1.0.0
+1.0.1

data/db_leftovers.gemspec

@@ -5,7 +5,7 @@
 
 Gem::Specification.new do |s|
   s.name = "db_leftovers"
-  s.version = "1.0.0"
+  s.version = "1.0.1"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Paul A. Jungwirth"]

data/lib/db_leftovers/mysql_database_interface.rb

@@ -2,12 +2,12 @@ module DBLeftovers
 
   class MysqlDatabaseInterface < GenericDatabaseInterface
 
-    def initialize(conn=nil)
+    def initialize(conn=nil, database_name=nil)
       @conn = conn || ActiveRecord::Base.connection
+      @db_name = database_name || ActiveRecord::Base.configurations[Rails.env]['database']
     end
 
     def lookup_all_indexes
-      # TODO: Constrain it to the database for the current Rails project:
       ret = {}
       @conn.select_values("SHOW TABLES").each do |table_name|
         indexes = {}
@@ -36,7 +36,6 @@ module DBLeftovers
 
     def lookup_all_foreign_keys
       # TODO: Support multi-column foreign keys:
-      # TODO: Constrain it to the database for the current Rails project:
       ret = {}
       sql = <<-EOQ
           SELECT  c.constraint_name,
@@ -49,6 +48,7 @@ module DBLeftovers
                   information_schema.key_column_usage k
           WHERE   c.constraint_schema = k.constraint_schema
           AND     c.constraint_name = k.constraint_name
+          AND     c.constraint_schema IN (#{target_databases_quoted})
       EOQ
       @conn.select_rows(sql).each do |constr_name, from_table, from_column, to_table, to_column, del_type|
         del_type = case del_type
@@ -79,5 +79,16 @@ module DBLeftovers
       execute_sql %{ALTER TABLE #{from_table} DROP FOREIGN KEY #{constraint_name}}
     end
 
+    private
+
+    def target_databases_quoted
+      case @db_name
+      when Array
+        @db_name.map{|x| "'#{x}'"}.join(", ")
+      else
+        "'#{@db_name}'"
+      end
+    end
+
   end
 end

data/lib/db_leftovers/postgres_database_interface.rb

@@ -7,8 +7,6 @@ module DBLeftovers
     end
 
     def lookup_all_indexes
-      # TODO: Constrain it to the database for the current Rails project:
-      #       (current_database(), current_schema())
       ret = {}
       sql = <<-EOQ
           SELECT  ix.indexrelid,
@@ -58,8 +56,6 @@ module DBLeftovers
       # confdeltype: a=nil, c=cascade, n=null
       ret = {}
       # TODO: Support multi-column foreign keys:
-      # TODO: Constrain it to the database for the current Rails project:
-      #       (current_database(), current_schema())
       sql = <<-EOQ
           SELECT  c.conname,
                   t1.relname,
@@ -103,8 +99,6 @@ module DBLeftovers
     end
 
     def lookup_all_constraints
-      # TODO: Constrain it to the database for the current Rails project:
-      #       (current_database(), current_schema())
       ret = {}
       sql = <<-EOQ
           SELECT  c.conname,

data/spec/mysql_spec.rb

@@ -18,9 +18,9 @@ describe DBLeftovers::MysqlDatabaseInterface do
     it "WARN: Skipping MySQL tests because no database found. Use spec/config/database.yml to configure one."
   else
     before do
-       y = test_database_yml('mysql')
+      y = test_database_yml('mysql')
       @conn = test_db_connection(nil, y)
-      @db = DBLeftovers::MysqlDatabaseInterface.new(@conn)
+      @db = DBLeftovers::MysqlDatabaseInterface.new(@conn, y['database'])
       drop_all_mysql_tables(@conn, y['database'])
       fresh_tables(@conn, y['database'])
     end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: db_leftovers
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.1
   prerelease: 
 platform: ruby
 authors:
@@ -147,7 +147,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 3241801701891816563
+      hash: -672806461406731014
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements: