couch_tomato 0.1.5 → 0.2.0

data/README.md

@@ -1,88 +1,108 @@
-Couch Tomato
-============
+#Couch Tomato
+*A Ruby persistence layer for CouchDB, inspired by and forked from Couch Potato*
 
-## TODO
+##Quick Start
+###Installing Couch Tomato
+Couch Tomato is hosted on [gemcutter.org](http://gemcutter.org), and can be installed as follows:
 
-### Documentation
+    sudo gem install couch_tomato --source http://gemcutter.org
 
-- Quick start
-- Data migration mechanism for managing (implicit) schema and data changes between team members and deployed systems, complete with generator and Rake tasks
-- Rake tasks that facilitate replication of local and remote CouchDB databases
-- Removed model property dirty tracking (added significant complexity and we haven't needed it...yet?)
-- Shoulda instead of RSpec
-- RR for mocks/stubs
+###Post Installation Requirements
+`root` in a path refers to `Rails.root` if you are using Rails, and the root level of any Ruby project if you are not using Rails. With the couch\_tomato gem installed, enable the Thor tasks by creating a file `couch_tomato.thor` as shown below:
 
-### Code
+####couch\_tomato.thor
 
-- default "server" to `http://localhost:5984`
+    couch_tomato_gem = Gem.searcher.find('couch_tomato')
+    Dir["#{couch_tomato_gem.full_gem_path}/lib/tasks/*.thor"].each { |ext| load ext } if couch_tomato_gem
 
+`couch_tomato.thor` can be saved to `Rails.root/lib/tasks` for Rails projects or to the root level of a regular Ruby app. Thor tasks associated with Couch Tomato are available under the `ct` namespace. To setup the Couch Tomato folder structure and config file in a Rails project, run the following:
 
-## Potato/Tomato
+    thor ct:init
+    
+The above will create a folder `couchdb` in `root`, along with `root/couchdb/migrate` and `root/couchdb/views`. The init task will also generate a sample Couch Tomato config file `couch_tomato.yml.example` in `root/config` as given below: 
 
-We're huge fans of Couch Potato. We love it's advocacy for using Couch naturally (not trying to make it look like a SQL database). Originally a Couch Potato fork, Couch Tomato supports our own production needs.
+####couch\_tomato.yml.example
 
-### Multi-Database Support
+    defaults: &defaults
+      couchdb_address:         127.0.0.1
+      couchdb_port:            5984
+      couchdb_basename:        your_project_name
+
+    development:
+      <<: *defaults
+
+    test:
+      <<: *defaults
+
+    production:
+      <<: *defaults
+
+Modify `couchdb_address`, `couchdb_port`, and `couchdb_basename` to correspond to the ip/address, port number, and name of your project respectively. You can optionally choose to suffix all your databases names by adding a `couchdb_suffix` field. Rename/copy `couch_tomato.yml.example` to `couch_tomato.yml`. 
+
+Finally, you will need to populate the values in CouchTomato::Config. Put
 
+    CouchTomato::Config.set_config_yml path
+    
+somewhere in your app (i.e. in an initializer for Rails). This will load `couch_tomato.yml` into CouchTomato::Config. If path is not specified, Couch Tomato will look for the default `root/config/couch_tomato.yml`. If you chose to not create a `couch_tomato.yml`, you can populate the fields of `CouchTomato::Config` manually. Couch Tomato is now ready to be used.
+
+##Using Couch Tomato
+### Multi-Database Support
 CouchDB makes it dead-simple to manage multiple databases. For large data-sets, it's very important to separate unrelated documents into separate databases. Couch Tomato assumes (but doesn't force) the use of multiple databases.
 
     class UserDb < CouchTomato::Database
       name "users"
-      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
+      ...
     end
 
     class StatDb < CouchTomato::Database
-      name "stats"
-      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
+      ...
     end
 
     UserDb.save_doc(User.new({:name => 'Joe'}))
     5_000.times { StatDb.save_doc(Stat.new({:metric => 10_000 * rand})) }
 
-### Each view determines the model for its values
+A name can be specified for a specific database as shown in UserDb, otherwise, the class name is used.
 
+###Each view determines the model for its values
 Views return arbitrary hashes. Often a view's value is an entire document (or more correctly, utilize `emit(key, null)` combined with `:include_docs => true`). But, a view's value is also often completely independent of the structure of the underlying documents.
 
 Define views on the database rather than inside a model (this is arguably more Couch-like). Each views declaration stipulates whether their results should be 'raw' hashes or a particular model type.
 
     class UserDb < CouchTomato::Database
       name "users"
-      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
       
       view :by_created_at, User
       view :count # raw
     end
 
-### Store view definitions on the file system
-
+###Store view definitions on the file system
 Rather than having Ruby generate JavaScript or writing JavaScript in our Ruby code as a string, define views in files on the file system:
 
-    RAILS_ROOT/couchdb/views/users/*-map.js
-    RAILS_ROOT/couchdb/views/users/*-reduce.js
+    root/couchdb/views/users/*-map.js
+    root/couchdb/views/users/*-reduce.js
     
 The reduce is optional. If you want to define views in a specific design document (called 'lazy'), you can do so:
 
-    RAILS_ROOT/couchdb/views/users/lazy/*.js
+    root/couchdb/views/users/lazy/*.js
 
 There's a handy generator:
 
-    script/generate view users by_created_at
-    script/generate view users/lazy by_birthday
-    script/generate view users by_created_on map reduce
-
-Rake tasks apply the views on the file system to Couch, skipping views that aren't dirty:
+    script/generate couch_view users by_created_at
+    script/generate couch_view users/lazy by_birthday
+    script/generate couch_view users by_created_on map reduce
 
-    rake couch_tomato:push
+Thor tasks apply the views on the file system to CouchDb, skipping views that aren't dirty:
 
-You can also view the differences between the views in Couch and those on the file system:
+    thor ct:push
 
-    rake couch_tomato:diff
+You can also view the differences between the views in CouchDb and those on the file system:
 
-### Remove dynamically generated views
+    thor ct:diff
 
+###Remove dynamically generated views
 We almost always need to write JavaScript to get the view behavior we need, and, for both conceptual and implementation complexity reasons, we value having all the views contained in one place--the file system. This also simplifies deployment and collaboration workflows.
 
-### Multiple design documents per database
-
+###Multiple design documents per database
 CouchDB supports multiple design documents per database. There's an important semantic consideration: all views in a design document are updated if any one view needs to be updated. To improve the read performance of couch views under high-volume reads and writes, you could organize views that don't need to be as timely into a separate design document named 'lazy', and always include the `stale=true` couch option in queries to views defined in the 'lazy' design document. You could then have a script that ran periodically to trigger the 'lazy' views to update.
 
     class UserDb > CouchTomato::Database
@@ -93,4 +113,113 @@ CouchDB supports multiple design documents per database. There's an important se
       view 'lazy/count_created_by_date'
     end
 
-We have not had a use for this, nor have we demonstrated that the claimed performance benefit actually exists (it originated from the CouchDB docs, wiki or list or some-such). But, it is instance where this approach to representing views maps fairly directly to CouchDB functionality.
+###Migrations
+Couch Tomato migrations are similar to ActiveRecord migrations, however, Couch Tomato migrations modify existing fields of documents instead of a "schema". There is a handy generator available for migrators as well.
+
+    script/generate couch_migration users by_created_at
+
+Migration come with two methods, up and down, each with a document hash. Up/down method will be run on every document in a database, with changes to the document hash committed to a database if the method does not return false. A migration can be accessed by thor ct:migrate and the -v (version) option. The version is simply the prefixed number in front of the generated view file.
+
+##Thor Tasks
+All Thor tasks associated with Couch Tomato are available under the namespace "ct". The `-e` option specifies an environment (i.e. for Rails)
+
+###ct:init
+The init task creates the folder structure required for managing views and migrations and a sample `couch_tomato.yml`.
+
+Example:
+
+    thor ct:init
+
+###ct:push
+The push tasks syncs CouchDB with the view structure present on the file system.
+
+Example:
+
+    thor ct:push -e development
+  
+###ct:diff
+The diff tasks `git status` type diff between the filesystem view structure and the current structure in CouchDB. The diff is with respect to the file system, that is, the file system is always assumed to be the most up to date.
+
+Example:
+
+    thor ct:diff -e development
+  
+###ct:drop
+The drop task will remove a specified database within the given environment from CouchDB. The -r option can be specified to remove via regex, and no arguments can be supplied to remove all databases.
+
+Examples:
+
+    # Remove all databases (you will be prompted first)
+    thor ct:drop -e development
+
+    # Remove all databases ending "\_bak"
+    thor ct:drop -e development -r .*\_bak
+
+###ct:migrate
+The migrate tasks runs migrations from your `couchdb/migrate` folder.
+
+Examples:
+
+    # Apply all migrations
+    thor ct:migrate -e development
+    
+    # Undo migration "20090911201227"
+    thor ct:migrate -e development --down -v 20090911201227
+    
+    # Redo the last 5 migrations
+    thor ct:migrate -e development --redo -s 5
+    
+    # Reset all databases using all available migrations to the "development" environment
+    thor ct:migrate -e development --reset
+
+###ct:rollback
+The rollback tasks will revert to a previous migration from the current version. Specify the number of steps with the -s option.
+
+Examples:
+
+    # Undo the previous migration
+    thor ct:rollback -e development
+    
+    # Undo the last 5 migrations
+    thor ct:rollback -e development -s 5
+
+###ct:forward
+The forward task will roll forward to the next version. Specify the number of steps with the -s option.
+
+Example:
+
+    # Roll forward to the next migration
+    thor ct:forward -e development
+
+###ct:replicate
+The replicate task facilitates the duplication of databases across application environments. The source and target server are always required for replication. Replicate operates in three different functions:
+
+1. If a source and destination database are provided, then the source database from the source server will be copied onto the destination database on the target server. Note that the destination database needs to already have been created.
+2. If not 1., but the the source and target servers are the same, then all databases on the common server are duplicated; the duplicate databases are postfixed with a "\_bak".
+3. If neither 1. or 2., then the assumption is that the user wants to clone all databases from the remote source server onto the specified target server.
+
+Examples:
+
+    # Copy database "example" from source server "11.11.11.11" to "example_1" in localhost
+    thor ct:replicate -e development -s 11.11.11.11 -t localhost -c example -v example_1
+    
+    # Back up all databases in localhost
+    thor ct:replicate -e development -s localhost -t localhost
+    
+    # Duplicate databases from "11.11.11.11" to localhost
+    thor ct:replicate -e development -s 11.11.11.11 -t localhost
+
+###ct:touch
+The touch task will initiate the building of views for a given database. Touch will query the first view of the each design doc in a db which will cause all remaining views to be built as well.
+
+Examples:
+
+    # Build all design documents in databases "example" and "test"
+    thor ct:touch -e development -d example test
+    
+    # Build all design documents in "example" and specify a 24 hours timeout
+    thor ct:touch -e development -d example -t 86400
+    
+    # Build all design documents in "example" asynchronously
+    thor ct:touch -e development -d example --async
+    

data/README.md_old

@@ -0,0 +1,96 @@
+Couch Tomato
+============
+
+## TODO
+
+### Documentation
+
+- Quick start
+- Data migration mechanism for managing (implicit) schema and data changes between team members and deployed systems, complete with generator and Rake tasks
+- Rake tasks that facilitate replication of local and remote CouchDB databases
+- Removed model property dirty tracking (added significant complexity and we haven't needed it...yet?)
+- Shoulda instead of RSpec
+- RR for mocks/stubs
+
+### Code
+
+- default "server" to `http://localhost:5984`
+
+
+## Potato/Tomato
+
+We're huge fans of Couch Potato. We love it's advocacy for using Couch naturally (not trying to make it look like a SQL database). Originally a Couch Potato fork, Couch Tomato supports our own production needs.
+
+### Multi-Database Support
+
+CouchDB makes it dead-simple to manage multiple databases. For large data-sets, it's very important to separate unrelated documents into separate databases. Couch Tomato assumes (but doesn't force) the use of multiple databases.
+
+    class UserDb < CouchTomato::Database
+      name "users"
+      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
+    end
+
+    class StatDb < CouchTomato::Database
+      name "stats"
+      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
+    end
+
+    UserDb.save_doc(User.new({:name => 'Joe'}))
+    5_000.times { StatDb.save_doc(Stat.new({:metric => 10_000 * rand})) }
+
+### Each view determines the model for its values
+
+Views return arbitrary hashes. Often a view's value is an entire document (or more correctly, utilize `emit(key, null)` combined with `:include_docs => true`). But, a view's value is also often completely independent of the structure of the underlying documents.
+
+Define views on the database rather than inside a model (this is arguably more Couch-like). Each views declaration stipulates whether their results should be 'raw' hashes or a particular model type.
+
+    class UserDb < CouchTomato::Database
+      name "users"
+      server "http://#{APP_CONFIG["couchdb_address"]}:#{APP_CONFIG["couchdb_port"]}"
+      
+      view :by_created_at, User
+      view :count # raw
+    end
+
+### Store view definitions on the file system
+
+Rather than having Ruby generate JavaScript or writing JavaScript in our Ruby code as a string, define views in files on the file system:
+
+    RAILS_ROOT/couchdb/views/users/*-map.js
+    RAILS_ROOT/couchdb/views/users/*-reduce.js
+    
+The reduce is optional. If you want to define views in a specific design document (called 'lazy'), you can do so:
+
+    RAILS_ROOT/couchdb/views/users/lazy/*.js
+
+There's a handy generator:
+
+    script/generate view users by_created_at
+    script/generate view users/lazy by_birthday
+    script/generate view users by_created_on map reduce
+
+Rake tasks apply the views on the file system to Couch, skipping views that aren't dirty:
+
+    rake couch_tomato:push
+
+You can also view the differences between the views in Couch and those on the file system:
+
+    rake couch_tomato:diff
+
+### Remove dynamically generated views
+
+We almost always need to write JavaScript to get the view behavior we need, and, for both conceptual and implementation complexity reasons, we value having all the views contained in one place--the file system. This also simplifies deployment and collaboration workflows.
+
+### Multiple design documents per database
+
+CouchDB supports multiple design documents per database. There's an important semantic consideration: all views in a design document are updated if any one view needs to be updated. To improve the read performance of couch views under high-volume reads and writes, you could organize views that don't need to be as timely into a separate design document named 'lazy', and always include the `stale=true` couch option in queries to views defined in the 'lazy' design document. You could then have a script that ran periodically to trigger the 'lazy' views to update.
+
+    class UserDb > CouchTomato::Database
+      name :users
+      
+      view :by_created_at, User
+      view :count # raw
+      view 'lazy/count_created_by_date'
+    end
+
+We have not had a use for this, nor have we demonstrated that the claimed performance benefit actually exists (it originated from the CouchDB docs, wiki or list or some-such). But, it is instance where this approach to representing views maps fairly directly to CouchDB functionality.

data/lib/couch_tomato.rb

@@ -2,6 +2,7 @@ require 'couchrest'
 require 'json'
 require 'json/add/core'
 require 'json/add/rails'
+require 'active_support'
 
 # require 'ostruct'
 
@@ -42,5 +43,7 @@ require File.dirname(__FILE__) + '/core_ext/symbol'
 require File.dirname(__FILE__) + '/core_ext/extract_options'
 require File.dirname(__FILE__) + '/core_ext/duplicable'
 require File.dirname(__FILE__) + '/core_ext/inheritable_attributes'
+require File.dirname(__FILE__) + '/couch_tomato/config'
 require File.dirname(__FILE__) + '/couch_tomato/persistence'
 require File.dirname(__FILE__) + '/couch_tomato/js_view_source'
+

data/lib/couch_tomato/config.rb

@@ -0,0 +1,11 @@
+module CouchTomato
+  class Config
+    cattr_accessor :url
+    cattr_accessor :prefix
+    cattr_accessor :suffix
+
+    @@url    = "http://127.0.0.1:5984"
+    @@prefix = ""
+    @@suffix = defined?(Rails) ? Rails.env : ""
+  end
+end

data/lib/couch_tomato/database.rb

@@ -1,73 +1,48 @@
-require 'couchrest'
-require 'pp'
+require "couchrest"
 
 module CouchTomato
   class Database
-
     class ValidationsFailedError < ::StandardError; end
 
-    # Database
-    class_inheritable_accessor :prefix_string
-    class_inheritable_accessor :database_name
-    class_inheritable_accessor :database_server
-    class_inheritable_accessor :couchrest_db
-    class_inheritable_accessor :views
+    class << self
+      attr_accessor :url
+      attr_accessor :prefix
+      attr_accessor :suffix
+      attr_accessor :name
+      attr_accessor :views
 
-    self.views = {}
-    self.prefix_string = ''
+      attr_accessor :couchrest_db
+    end
+
+    def self.inherited(c)
+      c.url    = CouchTomato::Config.url
+      c.prefix = CouchTomato::Config.prefix
+      c.suffix = CouchTomato::Config.suffix
+      c.name   = c.to_s.underscore
+      c.views  = {}
+    end
 
     def self.database
       return self.couchrest_db if self.couchrest_db
-      
-      self.prefix_string ||= ''
 
-      tmp_prefix = self.prefix_string + '_'  unless self.prefix_string.empty?
-      tmp_server = self.database_server + '/' unless self.database_server.match(/\/$/)
-      tmp_suffix = '_' + Rails.env  if defined?(Rails)
+      path = "#{self.url.gsub(/\/\s*$/, "")}/#{([self.prefix.to_s, self.name, self.suffix.to_s] - [""]).join("_")}"
+
+      self.couchrest_db = CouchRest.database(path)
 
-      self.couchrest_db = CouchRest.database("#{tmp_server}#{tmp_prefix}#{self.database_name}#{tmp_suffix}")
       begin
         self.couchrest_db.info
       rescue RestClient::ResourceNotFound
-        raise "Database '#{tmp_prefix}#{self.database_name}#{tmp_suffix}' does not exist."
+        raise "Database '#{path}' does not exist."
       end
-      
-      self.couchrest_db
-    end
 
-    def self.prefix (name)
-       self.prefix_string =  name || ''
+      return self.couchrest_db
     end
-    def self.name (name)
-      raise 'You need to provide a database name' if name.nil?
-      self.database_name   = (name.class == Symbol)? name.to_s : name
-    end
-
-    # TODO: specify db=>host mapping in yaml, and allow to differ per environment
-    def self.server (route='http://127.0.0.1:5984/')
-      self.database_server = route
-      
-      # self.prefix_string ||= ''
-      # 
-      # tmp_prefix = self.prefix_string + '_'  unless self.prefix_string.empty?
-      # tmp_server = self.database_server + '/' unless self.database_server.match(/\/$/)
-      # tmp_suffix = '_' + Rails.env  if defined?(Rails)
-      # 
-      # 
-      # self.couchrest_db ||= CouchRest.database("#{tmp_server}#{tmp_prefix}#{self.database_name}#{tmp_suffix}")
-      # begin
-      #   self.couchrest_db.info
-      # rescue RestClient::ResourceNotFound
-      #   raise "Database '#{tmp_prefix}#{self.database_name}#{tmp_suffix}' does not exist."
-      # end
-
-    end
-
 
     def self.view(name, options={})
-      raise 'A View nemonic must be specified' if name.nil?
+      raise "A View nemonic must be specified" if name.nil?
+
       self.views[name] = {}
-      self.views[name][:design_doc] = !options[:design_doc] ? self.database_name.to_sym : options.delete(:design_doc).to_sym
+      self.views[name][:design_doc] = !options[:design_doc] ? self.name.to_sym : options.delete(:design_doc).to_sym
       self.views[name][:view_name] = options.delete(:view_name) || name.to_s
       self.views[name][:model] = options.delete(:model)
       self.views[name][:couch_options] = options
@@ -149,11 +124,9 @@ module CouchTomato
     end
 
     def self.inspect
-      super
-      puts 'Database name: ' + (self.database_name || 'nil')
-      puts 'Database server: ' + (self.database_server || 'nil')
-      puts 'Views:'
-      pp self.views
+      puts "Database server: #{self.url || "nil"}"
+      puts "Database name:   #{self.name || "nil"}"
+      puts "Views: #{self.views.inspect}"
     end
 
     def self.query_view!(name, options={})

data/lib/couch_tomato/js_view_source.rb

@@ -1,5 +1,7 @@
 require 'digest/sha1'
+require 'patron'
 
+STDOUT.sync = true
 module CouchTomato
   class JsViewSource
     # todo: provide a 'dirty?' method that can be called in an initializer and warn the developer that view are out of sync
@@ -22,11 +24,11 @@ module CouchTomato
 
           if fs_doc['views'].empty?
             next unless fs_doc['_rev']
-            puts "DELETE #{fs_doc['_id']}" unless silent
+            puts "DELETE #{database_name}/#{fs_doc['_id']}" unless silent
             db.delete_doc(fs_doc)
           else
             if changed_views?(fs_doc, db_doc)
-              puts "UPDATE #{fs_doc['_id']}" unless silent
+              puts "UPDATE #{database_name}/#{fs_doc['_id']}" unless silent
               db.save_doc(fs_doc)
             end
           end
@@ -48,22 +50,36 @@ module CouchTomato
     end
 
     def self.diff
+      status_dict = {
+        :NEW_DOC  => "new design:", :NEW_VIEW => "new view:", 
+        :MOD_VIEW => "modified:",   :DEL_DOC  => "deleted:", 
+        :DEL_VIEW => "deleted:",    :NEW_DB   => "new db:"
+      }
+      types = ["map", "reduce"]
+      puts "# Changes with respect to the filesystem:"
+      
       fs_database_names.each do |database_name|
-        db = database!(database_name)
-
+        db = database(database_name)
+        puts "#\t#{setw(status_dict[:NEW_DB], 14)}#{database_name}" unless is_db?(db)
+        
         fs_docs = fs_design_docs(database_name)
-        db_docs = db_design_docs(db)
+        db_docs = is_db?(db) ? db_design_docs(db) : {}
 
+        diff = []
         # design docs on fs but not in db
         (fs_docs.keys - db_docs.keys).each do |design_name|
           unless fs_docs[design_name]['views'].empty?
-            puts "    NEW: #{database_name}/_#{design_name}: #{fs_docs[design_name]['views'].keys.join(', ')}"
+            diff.push [:NEW_DOC, "#{database_name}/_#{design_name}"]
+            fs_docs[design_name]['views'].keys.each do |view| 
+              (fs_docs[design_name]['views'][view].keys & types).each {|type|
+                diff.push [:NEW_VIEW, "#{database_name}/_#{design_name}/#{view}-#{type}"]}
+            end
           end
         end
 
         # design docs in db but not on fs
         (db_docs.keys - fs_docs.keys).each do |design_name|
-          puts "REMOVED: #{database_name}/_#{design_name}"
+          diff.push [:DEL_DOC, "#{database_name}/_#{design_name}"] unless (design_name.to_s.include? "migrations")
         end
 
         # design docs in both db and fs
@@ -74,16 +90,16 @@ module CouchTomato
 
           unless fs_only_view_keys.empty?
             methods = fs_only_view_keys.map do |key|
-              %w(map reduce).map {|method| fs_docs[design_name]['views'][key][method].nil? ? nil : "#{key}.#{method}()"}.compact
+              %w(map reduce).map {|method| fs_docs[design_name]['views'][key][method].nil? ? nil : "#{key}-#{method}"}.compact
             end.flatten
-            puts "  ADDED: #{database_name}/_#{design_name}: #{methods.join(', ')}"
+            methods.each {|method| diff.push [:NEW_VIEW, "#{database_name}/_#{design_name}/#{method}"] }
           end
 
           unless db_only_view_keys.empty?
             methods = db_only_view_keys.map do |key|
-              %w(map reduce).map {|method| db_docs[design_name]['views'][key][method] ? "#{key}.#{method}()" : nil}.compact
+              %w(map reduce).map {|method| db_docs[design_name]['views'][key][method] ? "#{key}-#{method}" : nil}.compact
             end
-            puts "REMOVED: #{database_name}/_#{design_name}: #{methods.join(', ')}"
+            methods.each {|method| diff.push [:DEL_VIEW, "#{database_name}/_#{design_name}/#{method}"] }
           end
 
           common_view_keys.each do |common_key|
@@ -95,28 +111,36 @@ module CouchTomato
             # has either the map or reduce been added or removed
             %w(map reduce).each do |method|
               if db_view[method] && !fs_view[method]
-                puts "REMOVED: #{database_name}/_#{design_name}:#{method}()" and next
+                diff.push [:DEL_VIEW, "#{database_name}/_#{design_name}/#{common_key}-#{method}"]  and next
               end
 
               if fs_view[method] && !db_view[method]
-                puts "  ADDED: #{database_name}/_#{design_name}:#{method}()" and next
+                diff.push [:NEW_VIEW, "#{database_name}/_#{design_name}/#{common_key}-#{method}"]  and next
               end
 
               if fs_view["sha1-#{method}"] != db_view["sha1-#{method}"]
-                puts "OUTDATED: #{database_name}/_#{design_name}:#{method}()" and next
+                diff.push [:MOD_VIEW, "#{database_name}/_#{design_name}/#{common_key}-#{method}"]  and next
               end
             end
           end
-
         end
-
+        
+        diff.uniq!
+        diff.each do |status|
+          puts "#\t#{setw(status_dict[status.first], 14)}#{status.last}"
+        end
       end
     end
 
     private
+    
+    def self.setw(str, w)
+      spaces = w - str.length
+      (spaces > 0) ? str + (" " * spaces) : str
+    end
 
     def self.path(db_name="")
-      "#{Rails.root}/couchdb/views/#{db_name}" if Rails
+      "#{Rails.root rescue nil || "."}/couchdb/views/#{db_name}"
     end
 
     def self.fs_database_names
@@ -136,14 +160,17 @@ module CouchTomato
     # :clicks => {'by_date' => {'map' => ..., 'reduce' => ..., sha1-map => ..., sha1-reduce => ...} }
     def self.fs_design_docs(db_name)
       design_docs = {}
-
-      path = "#{RAILS_ROOT}/couchdb/views/#{db_name}"
-      Dir[path + "/**"].each do |file|
-        throw "Invalid filename '#{File.basename(file)}': expecting '-map.js' or '-reduce.js' suffix" unless file.match(/-((map)|(reduce))\.js$/)
-
-        design_name = db_name.to_sym
-        design_docs[design_name] ||= {'_id' => "_design/#{db_name}", 'views' => {}}
-        fs_view(design_docs[design_name], file)
+      
+      path = "#{Rails.root rescue nil || "."}/couchdb/views/#{db_name}"
+      doc_folders = Dir["#{path}/**/"].map {|ddoc| ddoc.chop! }
+      doc_folders.each do |design_doc|
+        (Dir["#{design_doc}/**"] - doc_folders).each do |file|
+          throw "Invalid filename '#{File.basename(file)}': expecting '-map.js' or '-reduce.js' suffix" unless file.match(/-((map)|(reduce))\.js$/)
+
+          design_name = File.basename(design_doc)
+          design_docs[design_name.to_sym] ||= {'_id' => "_design/#{design_name}", 'views' => {}}
+          fs_view(design_docs[design_name.to_sym], file)
+        end
       end
 
       design_docs.each do |db, design|
@@ -172,12 +199,58 @@ module CouchTomato
       design_doc['views'][name]["sha1-#{type}"] = sha1
       design_doc
     end
+    
+    def self.touch(dbs, async=false, timeout=nil)
+      s = Patron::Session.new
+      s.timeout = timeout.nil? ? 86400 : timeout.to_i
+      s.timeout = 1 if async
+       
+      dbs.each do |db_str|
+        db = database(db_str)
+        design_docs = db_design_docs(db)
+        
+        design_docs.each do |ddoc_sym, ddoc|
+          puts ddoc_sym.to_s
+          next if ddoc_sym.to_s == "migrations"
+          doc_id = ddoc["_id"]
+          view = ddoc["views"].keys.first
+          
+          print "Building #{db_str}/#{doc_id}... "
+          begin
+            s.get("#{db_url(db_str)}/#{doc_id}/_view/#{view}?limit=0")
+            puts "finished!"
+          rescue Patron::TimeoutError
+            if async
+              puts "task started asynchronously."
+            else
+              puts "the view could not be built within the specified timeout (#{s.timeout} seconds). The view is still being built in the background."
+            end
+          end          
+        end
+      end
+    end
 
-    # todo: don't depend on "proprietary" APP_CONFIG
-    def self.database!(database_name)
-      CouchRest.database!("http://" + APP_CONFIG["couchdb_address"] + ":" + APP_CONFIG["couchdb_port"].to_s \
-       + "/" + APP_CONFIG["couchdb_basename"] + "_" + database_name + "_" + RAILS_ENV)
+    def self.db_url(name)
+      "#{CouchTomato::Config.url.gsub(/\/\s*$/, "")}/#{([CouchTomato::Config.prefix.to_s, name, CouchTomato::Config.suffix.to_s] - [""]).join("_")}"
     end
 
+    def self.database(database_name, force=false)
+      raise "Database names (#{database_name}) cannot contain uppercase letters." unless database_name == database_name.downcase
+      url = db_url(database_name)
+      force ? CouchRest.database!(url) : CouchRest.database(url)
+    end
+    
+    def self.database!(database_name)
+      database(database_name, true);
+    end
+    
+    def self.is_db?(db) 
+      begin
+        db.info
+      rescue
+        return false
+      end
+      return true
+    end
   end
 end

data/lib/tasks/couch_tomato.thor

@@ -0,0 +1,241 @@
+Ct_Yml_Example = %(defaults: &defaults
+  couchdb_address:         127.0.0.1
+  couchdb_port:            5984
+  couchdb_basename:        your_project_name
+
+development:
+  <<: *defaults
+
+test:
+  <<: *defaults
+
+production:
+  <<: *defaults
+)
+
+STDOUT.sync = true
+class CouchTomatoApp < Thor
+  include Thor::Actions
+  namespace :ct
+  
+  desc 'init', 'Sets up the required structure for Couch Tomato'
+  def init
+    load_env
+    project_root = ::Rails.root rescue nil || "."
+    
+    ct_yml_path = "#{project_root}/config/couch_tomato.yml.example"
+    couch_folder = "#{project_root}/couchdb"
+    
+    print "Generating a sample couch_tomato yml... " 
+    unless File.exist?(ct_yml_path)
+      FileUtils.mkdir "#{project_root}/config"
+      File.open(ct_yml_path, 'w') {|f| f.write(Ct_Yml_Example) }
+      puts "#{ct_yml_path} created"
+    else
+      puts "#{ct_yml_path} already exists"
+    end
+
+    print "Generating the couchdb folder.......... " 
+    puts (File.directory? couch_folder) ? "#{couch_folder} already exists" : "#{FileUtils.mkdir "#{couch_folder}"} created"
+    print "Generating couchdb/migrate............. " 
+    puts (File.directory? "#{couch_folder}/migrate") ? "#{couch_folder}/migrate already exists" : "#{FileUtils.mkdir "#{couch_folder}/migrate"} created"
+    print "Generating couchdb/views............... " 
+    puts (File.directory? "#{couch_folder}/views") ? "#{couch_folder}/views already exists" : "#{FileUtils.mkdir "#{couch_folder}/views"} created"
+  end
+  
+  desc 'push', 'Inserts the views into CouchDB'
+  method_options %w(RAILS_ENV -e) => :string
+  def push
+    load_env(options)
+    CouchTomato::JsViewSource.push
+  end
+  
+  desc 'diff', 'Compares views in DB and the File System'
+  method_options %w(RAILS_ENV -e) => :string
+  def diff
+    load_env(options)
+    CouchTomato::JsViewSource.diff
+  end
+  
+  desc 'drop', 'Drops databases for the current RAILS_ENV; ' + 
+       'If no databases are specified, user will be prompted if all databases should be removed; ' + 
+       'If the -r option is specified, all databases matching the regex will be dropped.'
+    method_options %w(RAILS_ENV -e) => :string, %w(DBS -d) => :array, %w(REGEX -r) => :string
+  def drop
+    load_env(options)
+    dbs = options["DBS"]
+    rm_all = false
+    rm_all = (yes? "Drop all databases?") if (options['REGEX'].nil? && dbs.nil?)
+    
+    regex = Regexp.new(options['REGEX'].to_s)
+    databases(dbs) do |db, dir|
+      if (rm_all || db.name == db.name[regex]) && is_db?(db)
+        db.delete!
+        puts "Dropped #{db.name}"
+      end
+    end
+  end
+  
+  desc 'migrate', 'Runs migrations'
+  method_options %w(RAILS_ENV -e) => :string, %w(VERSION -v) => :string, %w(STEP -s) => :string, 
+    :redo => :boolean, :reset => :boolean, :up => :boolean, :down => :boolean
+  def migrate
+    load_env(options)
+    supported_args = %w(redo reset up down)
+    action = supported_args & options.keys
+    raise "Cannot provide more than one action." if action.length > 1
+    
+    action = (action.empty?) ? nil : action.first
+    case action
+      when nil
+        migrate_helper
+      #Rollbacks the database one migration and re migrate up. If you want to rollback more than one step, define STEP=x. Target specific version with VERSION=x.
+      when "redo"
+        if ENV['VERSION']
+          down_helper
+          up_helper
+        else
+          rollback_helper
+          migrate_helper
+        end
+      #Resets your database using your migrations for the current environment
+      when "reset"
+        invoke :drop
+        invoke :push
+        migrate_helper
+      #Runs the "up" for a given migration VERSION.
+      when "up"
+        up_helper
+      #Runs the "down" for a given migration VERSION.
+      when "down"
+        down_helper
+    end        
+  end
+    
+  desc 'rollback', 'Rolls back to the previous version. Specify the number of steps with STEP=n'
+  method_options %w(RAILS_ENV -e) => :string, %w(STEP -s) => :string 
+  def rollback
+    load_env(options)
+    rollback_helper
+  end
+  
+  desc 'forward', 'Rolls forward to the next version. Specify the number of steps with STEP=n'
+  method_options %w(RAILS_ENV -e) => :string, %w(STEP -s) => :string 
+  def forward
+    load_env(options)
+    databases do |db, dir|
+      CouchTomato::Migrator.forward(db, dir, ENV['STEP'] ? ENV['STEP'].to_i : 1)
+    end
+  end
+  
+  desc 'replicate', 'Replicate databases between app environments'
+  method_options %w(RAILS_ENV -e) => :string, %w(SRC_DB -c) => :string, 
+    %w(DST_DB -v) => :string , %w(SRC_SERVER -s) => :string , %w(DST_SERVER -t) => :string 
+  def replicate
+    load_env(options)
+    src_server, dst_server = servers
+
+    src_db = options['SRC_DB']
+    dst_db = options['DST_DB'] || (src_server == dst_server ? "#{src_db}_bak" : src_db)
+
+    replicator = CouchTomato::Replicator.new(src_server, dst_server)
+    
+    if src_db
+      puts "== Replicating '#{src_server}/#{src_db}' to '#{dst_server}/#{dst_db}'"
+      replicator.replicate(src_db, dst_db)
+    elsif src_server == dst_server
+      puts "== Replicating all databases at '#{src_server}' using '_bak' suffix for replicated database names"
+      replicator.replicate_all('_bak')
+    else
+      puts "== Replicating all databases from '#{src_server}' to '#{dst_server}'"
+      replicator.replicate_all
+    end
+  end
+  
+  desc 'touch', 'Initiates the building of a design document'
+  method_options %w(RAILS_ENV -e) => :string, %w(DBS -d) => :array, :async => :boolean, %w(TIMEOUT -t) => :numeric
+  def touch
+    load_env(options)
+    view_path = "couchdb/views"
+    valid_dbs = options["DBS"] & (Dir["couchdb/views/**"].map {|db| File.basename(db) })
+    CouchTomato::JsViewSource.touch(valid_dbs, options.async?, options['TIMEOUT'])
+  end
+  
+  private
+  def up_helper
+    version = ENV['VERSION'] ? ENV['VERSION'].to_i : nil
+    raise 'VERSION is required' unless version
+
+    databases do |db, dir|
+      CouchTomato::Migrator.run(:up, db, dir, version)
+    end
+  end
+  
+  def down_helper
+    version = ENV['VERSION'] ? ENV['VERSION'].to_i : nil
+    raise 'VERSION is required' unless version
+
+    databases do |db, dir|
+      CouchTomato::Migrator.run(:down, db, dir, version)
+    end
+  end
+  
+  def rollback_helper
+    databases do |db, dir|
+      CouchTomato::Migrator.rollback(db, dir, ENV['STEP'] ? ENV['STEP'].to_i : 1)
+    end
+  end
+  
+  def migrate_helper
+    databases do |db, dir|
+      CouchTomato::Migrator.migrate(db, dir, ENV['VERSION'] ? ENV['VERSION'].to_i : nil)
+    end
+  end
+  
+  def load_env(options=nil)
+    unless (Rails rescue nil)
+      print "Loading enviornment... "  
+      %w(RAILS_ENV VERSION SRC_SERVER DST_SERVER STEP).each {|var| ENV[var] =
+        options[var] unless options[var].nil? } unless options.nil?
+
+      begin
+        require(File.join(ENV['PWD'], 'config', 'boot'))
+        require(File.join(ENV['PWD'], 'config', 'environment'))
+        puts "done."
+      rescue LoadError
+        puts "could not find environment files. Assuming direct use."
+      ensure
+        CouchTomato::Config.set_config_yml unless CouchTomato::Config.loaded?
+      end
+    end
+  end
+
+  def servers
+    local_server = "http://#{CouchTomato::Config.couch_address}:#{CouchTomato::Config.couch_port}"
+    src_server = (ENV['SRC_SERVER'] || local_server).gsub(/\s*\/\s*$/, '')
+    dst_server = (ENV['DST_SERVER'] || local_server).gsub(/\s*\/\s*$/, '')
+
+    return src_server, dst_server
+  end
+
+  def databases(db_names=nil)
+    dirs = Dir['couchdb/migrate/*']
+    
+    db_map = dirs.inject({}) {|map, dir| map[File.basename(dir)] = dir; map }
+    db_names ||= db_map.keys
+    db_names.each do |db_name|
+      db = CouchTomato::JsViewSource.database(db_name)
+      yield db, db_map[db_name]
+    end
+  end
+
+  def is_db?(db) 
+    begin
+      db.info
+    rescue
+      return false
+    end
+    return true
+  end
+
+end

data/rails/init.rb

@@ -1,7 +1,3 @@
 # this is for rails only
-
 require File.dirname(__FILE__) + '/../lib/couch_tomato'
-
-# CouchTomato::Config.database_name = YAML::load(File.read(Rails.root.to_s + '/config/couchdb.yml'))[RAILS_ENV]
-
 RAILS_DEFAULT_LOGGER.info "** couch_tomato: initialized from #{__FILE__}"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: couch_tomato
 version: !ruby/object:Gem::Version 
-  version: 0.1.5
+  version: 0.2.0
 platform: ruby
 authors: 
 - Plastic Trophy
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-11-16 00:00:00 -08:00
+date: 2010-01-21 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -50,9 +50,11 @@ extensions: []
 
 extra_rdoc_files: 
 - README.md
+- README.md_old
 files: 
 - MIT-LICENSE.txt
 - README.md
+- README.md_old
 - generators/couch_migration/couch_migration_generator.rb
 - generators/couch_migration/templates/migration.rb
 - generators/couch_view/couch_view_generator.rb
@@ -68,6 +70,7 @@ files:
 - lib/core_ext/symbol.rb
 - lib/core_ext/time.rb
 - lib/couch_tomato.rb
+- lib/couch_tomato/config.rb
 - lib/couch_tomato/database.rb
 - lib/couch_tomato/js_view_source.rb
 - lib/couch_tomato/migration.rb
@@ -84,6 +87,7 @@ files:
 - lib/couch_tomato/persistence/validation.rb
 - lib/couch_tomato/replicator.rb
 - lib/tasks/couch_tomato.rake
+- lib/tasks/couch_tomato.thor
 - rails/init.rb
 has_rdoc: true
 homepage: http://github.com/plastictrophy/couch_tomato