ar-octopus 0.0.23 → 0.0.24

data/.gitignore

@@ -1,3 +1,4 @@
 pkg/*.*
 tmp/*
 *.sqlite3
+.rvmrc

data/README.mkdn

@@ -3,7 +3,7 @@
 <p> Octopus is a better way to do Database Sharding in ActiveRecord. Sharding allows multiple databases in the same rails application. While there are several projects that implement Sharding (e.g. DbCharmer, DataFabric, MultiDb), each project has its own limitations. The main goal of octopus project is to provide a nice and clean way of doing Database Sharding.</p>
 
 <h2>Feature list: </h2>
-<p> The design of the api is made to be simple as possible. Octopus is focusing in the end user, giving the power of multiple databases, but with reliable code and flexibility. Octopus is focused on Rails 3, is compatible with Rails 2.x.</p>
+<p> The design of the api is made to be simple as possible. Octopus is focusing in the end user, giving the power of multiple databases, but with reliable code and flexibility. Octopus is focused on Rails 3, but is compatible with Rails 2.x.</p>
 
 <p> Octopus supports: </p>
 
@@ -12,38 +12,95 @@
 - Moving data between shards with migrations.
 - Tools to manage database configurations. (soon)
 
-<p> To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/tchandy/octopus/"> Wiki</a></p>
-<p>Wanna see sample rails applications using octopus features? please check it out: <a href="http://github.com/tchandy/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/tchandy/octopus_replication_example">Replication Example</a> </p> 
-  
-<h2>Thanks</h2>
+<h3> Replication </h3>
+<p> When using replication, all writes queries will be sent to master, and read queries to slaves. More info could be found at: <a href="http://wiki.github.com/tchandy/octopus/replication"> Wiki</a> </p>
 
-This project is sponsored by the <a href="http://www.rubysoc.org">Ruby Summer of Code</a>,
-and my mentors <a href="http://github.com/mperham">Mike Perham</a> and <a href="http://github.com/amitagarwal">Amit Agarwal</a>.
+<h3> Sharding </h3>
+<p> When using sharding, you need to specify what shard the query will be sent. Octopus support selecting the shard inside a controller, or manually in each object. More could be found at <a href="http://wiki.github.com/tchandy/octopus/sharding"> Wiki</a> </p>
+
+<h2> Install </h2>
+
+<h3> Rails 2.x </h3>
+
+Install  the octopus gem:
+<pre> sudo gem install ar-octopus </pre>
+
+Add this line to enviroment.rb:
+<pre>config.gem 'ar-octopus', :lib => "octopus"</pre>
 
+<h3> Rails 3.x </h3>
 
-<h3>If you are writing code that smells using octopus, the EVIL Octopus is going behind you, be careful.</h3>
+Add this line to Gemfile:
+<pre>gem 'ar-octopus', :require => "octopus"</pre>
+
+Runs a bundle install:
+<pre>bundle install</pre>
+
+<h2> How to use Octopus? </h2>
+<p>First, you need to create a config file, shards.yml, inside your config/ directory. to see the syntax and how this file should look, please checkout <a href="http://wiki.github.com/tchandy/octopus/config-file">this page on wiki</a></p>.
+
+<h3> Syntax </h3>
+<p>Octopus adds a method to each AR Class and object. the using method is used to select the shard, like this: </p>
+<pre>User.where(:name => "Thiago").limit(3).using(:slave_one) </pre>
+
+<p> Octopus also supports queries inside block. When you pass a block to the using method, all queries inside the block will be sent to the specified shard. </p>
 <pre>
-                                _________________
-                      ___       | I'm so EVIL!  |
-                   .-'   `'.    |______________ |
-                  /         \   /
-                  |         ;  /
-                  |         | /         ___.--,
-         _.._     |0) ~ (0) |    _.---'`__.-( (_.
-  __.--'`_.. '.__.\    '--. \_.-' ,.--'`     `""`
- ( ,.--'`   ',__ /./;   ;, '.__.'`    __
- _`) )  .---.__.' / |   |\   \__..--""  """--.,_
-`---' .'.''-._.-'`_./  /\ '.  \ _.-~~~````~~~-._`-.__.'
-      | |  .' _.-' |  |  \  \  '.               `~---`
-       \ \/ .'     \  \   '. '-._)
-        \/ /        \  \    `=.__`~-.
-        / /\         `) )    / / `"".`\
-  , _.-'.'\ \        / /    ( (     / /
-   `--~`   ) )    .-'.'      '.'.  | (
-          (/`    ( (`          ) )  '-;
-           `      '-;         (-'
+User.using(:slave_two) do 
+  User.create(:name => "Mike")
+end
 </pre>
 
+<p> Each object knows where is your shard, so you could to thinks like this:</p>
+<pre> 
+# This will find the user in the shard1
+@user = User.using(:shard1).find_by_name("Joao")
+
+# This will find the user in the master database
+@user2 = User.find_by_name("Jose")
+
+#Sets the name
+@user.name = "Mike"
+
+# Save the user in the correct shard, shard1. 
+@user.save()
+</pre>
+
+<p> In migrations, you also have access to the using method. The syntax is basically the same. This migration will run in brazil and canada shards.</p>
+<pre>
+  class CreateUsersOnBothShards < ActiveRecord::Migration
+    using(:brazil, :canada)
+
+    def self.up
+      User.create!(:name => "Both")
+    end
+
+    def self.down
+      User.delete_all()
+    end
+  end
+</pre>
+<p> You also could send a migration to a group of shards.  This migration will be sent to all shards that belongs to history_shards group, specified in shards.yml: </p>
+<pre>
+  class CreateUsersOnMultiplesGroups < ActiveRecord::Migration
+    using_group(:history_shards)
+
+    def self.up
+      User.create!(:name => "MultipleGroup")
+    end
+
+    def self.down
+      User.delete_all()
+    end
+  end
+</pre>
+
+<p> To see the complete list of features and syntax, please check out our <a href="http://wiki.github.com/tchandy/octopus/"> Wiki</a></p>
+<p>Wanna see sample rails applications using octopus features? please check it out: <a href="http://github.com/tchandy/octopus_sharding_example">Sharding Example</a> and <a href="http://github.com/tchandy/octopus_replication_example">Replication Example</a> </p>
+  
+<h2>Thanks</h2>
+
+This project is sponsored by the <a href="http://www.rubysoc.org">Ruby Summer of Code</a>,
+and my mentors <a href="http://github.com/mperham">Mike Perham</a> and <a href="http://github.com/amitagarwal">Amit Agarwal</a>.
 
 <h2>Copyright</h2>
 

data/Rakefile

@@ -29,7 +29,7 @@ begin
     gem.authors = ["Thiago Pradi", "Mike Perham", "Amit Agarwal"]
     gem.add_development_dependency "rspec", ">= 1.2.9"
     gem.add_dependency('activerecord', '>= 3.0.0beta')
-    gem.version = "0.0.23"
+    gem.version = "0.0.24"
   end
   Jeweler::GemcutterTasks.new
 rescue LoadError

data/ar-octopus.gemspec

@@ -5,7 +5,7 @@
 
 Gem::Specification.new do |s|
   s.name = %q{ar-octopus}
-  s.version = "0.0.23"
+  s.version = "0.0.24"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Thiago Pradi", "Mike Perham", "Amit Agarwal"]
@@ -46,6 +46,7 @@ Gem::Specification.new do |s|
      "spec/migrations/10_create_users_using_replication.rb",
      "spec/migrations/11_add_field_in_all_slaves.rb",
      "spec/migrations/12_create_users_using_block.rb",
+     "spec/migrations/13_create_users_using_block_and_using.rb",
      "spec/migrations/1_create_users_on_master.rb",
      "spec/migrations/2_create_users_on_canada.rb",
      "spec/migrations/3_create_users_on_both_shards.rb",
@@ -78,6 +79,7 @@ Gem::Specification.new do |s|
      "spec/migrations/10_create_users_using_replication.rb",
      "spec/migrations/11_add_field_in_all_slaves.rb",
      "spec/migrations/12_create_users_using_block.rb",
+     "spec/migrations/13_create_users_using_block_and_using.rb",
      "spec/migrations/1_create_users_on_master.rb",
      "spec/migrations/2_create_users_on_canada.rb",
      "spec/migrations/3_create_users_on_both_shards.rb",

data/lib/octopus.rb

@@ -12,8 +12,8 @@ module Octopus
   def self.config()
     @config ||= HashWithIndifferentAccess.new(YAML.load_file(Octopus.directory() + "/config/shards.yml"))
     
-    if !@config[Octopus.env].nil? && @config[Octopus.env()]['octopus_enviroments']
-      self.octopus_enviroments = @config[Octopus.env()]['octopus_enviroments']
+    if !@config[Octopus.env].nil? && @config[Octopus.env()]['enviroments']
+      self.enviroments = @config[Octopus.env()]['enviroments']
     end
     
     @config
@@ -27,17 +27,17 @@ module Octopus
   
   # This is the default way to do Octopus Setup
   # Available variables:
-  # :excluded_enviroments => the enviroments that octopus will not run. default: :development, :cucumber and :test
+  # :enviroments => the enviroments that octopus will run. default: :production
   def self.setup
     yield self
   end
   
-  def self.octopus_enviroments=(octopus_enviroments)
-    @octopus_enviroments = octopus_enviroments.map { |element| element.to_s }
+  def self.enviroments=(enviroments)
+    @enviroments = enviroments.map { |element| element.to_s }
   end
   
-  def self.octopus_enviroments
-    @octopus_enviroments || ['production']
+  def self.enviroments
+    @enviroments || ['production']
   end
   
   def self.rails3?

data/lib/octopus/association.rb

@@ -1,12 +1,12 @@
 module Octopus::Association
   def has_many(association_id, options = {}, &extension)
     default_octopus_opts(options)
-    super(association_id, options, &extension)
+    super
   end
 
   def has_and_belongs_to_many(association_id, options = {}, &extension)
     default_octopus_opts(options)
-    super(association_id, options, &extension)    
+    super
   end
 
   def default_octopus_opts(options)

data/lib/octopus/migration.rb

@@ -1,20 +1,17 @@
 module Octopus::Migration  
   def self.extended(base)
     class << base
-      alias_method_chain :migrate, :octopus
-      
-      def announce(message)
-        version = defined?(@version) ? @version : nil
-
-        text = "#{version} #{name}: #{message} - #{get_current_shard}"
-        length = [0, 75 - text.length].max
-        write "== %s %s" % [text, "=" * length]
+      def announce_with_octopus(message)
+        announce_without_octopus("#{message} - #{get_current_shard}")
       end
+      
+      alias_method_chain :migrate, :octopus
+      alias_method_chain :announce, :octopus
     end
   end
 
   def using(*args, &block)
-    if self.connection().is_a?(Octopus::Proxy)
+    if self.connection().is_a?(Octopus::Proxy) && !block_given?
       args.each do |shard|
         self.connection().check_schema_migrations(shard)
       end
@@ -23,13 +20,14 @@ module Octopus::Migration
       self.connection().current_shard = args        
     end
     
-    yield if block_given?
+    if block_given?
+      self.connection.run_queries_on_shard(args, &block)
+    end 
 
     return self
   end
 
   def using_group(*args)
-    
     if self.connection().is_a?(Octopus::Proxy)
       args.each do |group_shard|
         shards = self.connection().instance_variable_get(:@groups)[group_shard] || []
@@ -55,19 +53,21 @@ module Octopus::Migration
     conn = ActiveRecord::Base.connection
     groups = conn.instance_variable_get(:@groups)
     
-    return migrate_without_octopus(direction) unless conn.is_a?(Octopus::Proxy)
+    begin
+      return migrate_without_octopus(direction) unless conn.is_a?(Octopus::Proxy)
     
-    if conn.current_group.is_a?(Array)
-      conn.current_group.each { |group| conn.send_queries_to_multiple_shards(groups[group]) { migrate_without_octopus(direction) } } 
-    elsif conn.current_group.is_a?(Symbol)       
-      conn.send_queries_to_multiple_shards(groups[conn.current_group]) { migrate_without_octopus(direction) }     
-    elsif conn.current_shard.is_a?(Array)
-      conn.send_queries_to_multiple_shards(conn.current_shard) { migrate_without_octopus(direction) }     
-    else
-      migrate_without_octopus(direction)
+      if conn.current_group.is_a?(Array)
+        conn.current_group.each { |group| conn.send_queries_to_multiple_shards(groups[group]) { migrate_without_octopus(direction) } } 
+      elsif conn.current_group.is_a?(Symbol)       
+        conn.send_queries_to_multiple_shards(groups[conn.current_group]) { migrate_without_octopus(direction) }     
+      elsif conn.current_shard.is_a?(Array)
+        conn.send_queries_to_multiple_shards(conn.current_shard) { migrate_without_octopus(direction) }     
+      else
+        migrate_without_octopus(direction)
+      end
+    ensure
+      conn.clean_proxy()
     end
-
-    conn.clean_proxy()
   end
 end
 

data/lib/octopus/model.rb

@@ -15,7 +15,7 @@ module Octopus::Model
     end
 
     def using(shard, &block)
-      return self if defined?(::Rails) && !Octopus.octopus_enviroments.include?(Rails.env.to_s)
+      return self if defined?(::Rails) && !Octopus.enviroments.include?(Rails.env.to_s)
 
       hijack_connection()  
       clean_table_name()
@@ -58,7 +58,7 @@ module Octopus::Model
       def self.connection_with_octopus()
         if defined?(::Rails) 
           Octopus.config()
-          if Octopus.octopus_enviroments.include?(Rails.env.to_s)
+          if Octopus.enviroments.include?(Rails.env.to_s)
             self.connection_proxy().current_model = self
             return self.connection_proxy()
           else

data/lib/octopus/rails3/persistence.rb

@@ -7,17 +7,17 @@ module Octopus
 
       def update_attribute(name, value)
         reload_connection()
-        super(name, value)
+        super
       end
 
       def update_attributes(attributes)
         reload_connection()
-        super(attributes)
+        super
       end
 
       def update_attributes!(attributes)
         reload_connection()
-        super(attributes)
+        super
       end
 
       def reload

data/spec/config/shards.yml

@@ -104,7 +104,7 @@ production_entire_replicated:
       
 octopus_rails:
   replicated: true
-  octopus_enviroments:
+  enviroments:
     - staging
     - production
 

data/spec/migrations/13_create_users_using_block_and_using.rb

@@ -0,0 +1,15 @@
+class CreateUsersUsingBlockAndUsing < ActiveRecord::Migration
+  using(:brazil)
+  
+  def self.up
+    using(:canada) do
+      User.create!(:name => "Canada")
+    end
+    
+    User.create!(:name => "Brazil")
+  end
+
+  def self.down
+    User.delete_all()
+  end
+end

data/spec/octopus/migration_spec.rb

@@ -46,6 +46,15 @@ describe Octopus::Migration do
       User.using(:canada).find(:all, :conditions => {:name => "UsingCanada2"}).size.should == 1
     end
   end
+  
+  it "should send the query to the correct shard" do
+    migrating_to_version 13 do 
+      User.using(:brazil).find(:all, :conditions => {:name => "Brazil"}).size.should == 1
+      User.using(:brazil).find(:all, :conditions => {:name => "Canada"}).size.should == 0
+      User.using(:canada).find(:all, :conditions => {:name => "Brazil"}).size.should == 0
+      User.using(:canada).find(:all, :conditions => {:name => "Canada"}).size.should == 1
+    end    
+  end
 
   describe "should raise a exception when" do
     it "you specify a invalid shard name" do

data/spec/octopus/octopus_spec.rb

@@ -21,18 +21,18 @@ describe Octopus do
   
   describe "#setup method" do    
     it "should have the default octopus enviroment as production" do
-      Octopus.octopus_enviroments.should == ["production"]
+      Octopus.enviroments.should == ["production"]
     end
     
     it "should allow the user to configure the octopus enviroments" do
       Octopus.setup do |config|
-        config.octopus_enviroments = [:production, :staging]
+        config.enviroments = [:production, :staging]
       end
       
-      Octopus.octopus_enviroments.should == ['production', 'staging']      
+      Octopus.enviroments.should == ['production', 'staging']      
 
       Octopus.setup do |config|
-        config.octopus_enviroments = [:production]
+        config.enviroments = [:production]
       end
     end
   end

data/spec/octopus/proxy_spec.rb

@@ -71,7 +71,7 @@ describe Octopus::Proxy do
       Octopus.config()
       
       proxy.instance_variable_get(:@replicated).should be_true
-      Octopus.octopus_enviroments.should == ["staging", "production"] 
+      Octopus.enviroments.should == ["staging", "production"] 
     end
     
     it "should initialize correctly the shards for the staging enviroment" do

data/spec/spec_helper.rb

@@ -9,7 +9,7 @@ end
 
 require 'spec'
 require 'spec/autorun'
-require "spec/database_connection"
+require File.expand_path(File.dirname(__FILE__)) + "/database_connection"
 require "action_controller"
 require 'octopus'
 require "octopus_helper"

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: ar-octopus
 version: !ruby/object:Gem::Version 
-  hash: 49
+  hash: 47
   prerelease: false
   segments: 
   - 0
   - 0
-  - 23
-  version: 0.0.23
+  - 24
+  version: 0.0.24
 platform: ruby
 authors: 
 - Thiago Pradi
@@ -91,6 +91,7 @@ files:
 - spec/migrations/10_create_users_using_replication.rb
 - spec/migrations/11_add_field_in_all_slaves.rb
 - spec/migrations/12_create_users_using_block.rb
+- spec/migrations/13_create_users_using_block_and_using.rb
 - spec/migrations/1_create_users_on_master.rb
 - spec/migrations/2_create_users_on_canada.rb
 - spec/migrations/3_create_users_on_both_shards.rb
@@ -151,6 +152,7 @@ test_files:
 - spec/migrations/10_create_users_using_replication.rb
 - spec/migrations/11_add_field_in_all_slaves.rb
 - spec/migrations/12_create_users_using_block.rb
+- spec/migrations/13_create_users_using_block_and_using.rb
 - spec/migrations/1_create_users_on_master.rb
 - spec/migrations/2_create_users_on_canada.rb
 - spec/migrations/3_create_users_on_both_shards.rb