lookout-data_fabric 1.5.0

checksums.yaml

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    MWI0YzJiY2Q3NDg2NWY2NjY5ODdkMzJiNDNkMDUxNTA2YTY5MzgyOQ==
+  data.tar.gz: !binary |-
+    NjU1MDY3Y2ViNjMxNWQ0NDM5YWE3NmM5OTM2MTg4NTg5MWFjMWE4OA==
+SHA512:
+  metadata.gz: !binary |-
+    ZDBiMmExYjEzNmRkMTFmMGQ5ZjBjNzc5YmQ1MzU2NmFmMTJhYzk0NDgzMmFk
+    ZTU3Njg1M2IyNWFlODZlZDQzYTkzNWEyY2NjYTZlYjNjNzcyOGM1MjJhZTAy
+    YTMxNjBkMTM5OTdlOGFjN2IxMmRjMTg0YWNlZmM2NTQ0YTAyNjI=
+  data.tar.gz: !binary |-
+    MmNlNjAwMmU1ZjRlY2ZlZTYzNGIzMDI4OTBmNmE0OTliNThkYzAxNjgyOWYw
+    Nzk2YTM4NWE1YTdiMmEwNzY1ZmEzNjM0NGIyNWE2ZDgwZDBlNGZlYWQ4M2U4
+    MjljNzdjNjcyOTU2YzMyZDA4MmFhZjI3MmVhZTY4M2Y4Y2I0ZTU=

data/CHANGELOG

@@ -0,0 +1,79 @@
+DataFabric changelog
+
+v1.3.3 - 2011-10-29
+
+- Added ConnectionProxy#respond_to? which delegates to underlying connection
+(Paul Gross and Dan Manges)
+- Remove specific handling of nested transactions and let the underlying
+connection handle it (Paul Gross and Dan Manges)
+
+v1.3.2 - 2011-06-02
+
+- Fix problem fetching spec for connection pool.
+- Fix issues with Rails 3.1.
+
+v1.3.1 - 2011-02-10
+
+- Add some missing ConnectionPool methods (e.g. clear_reloadable_connections!)
+
+v1.3.0 - 2010-11-22
+
+- Major implementation refactoring
+- Fix issues under Rails 3.0.  Tested with 2.3.10, 3.0.0 and 3.0.3.
+- Drop support for Rails 2.2 (may work but untested)
+- Remove jeweler dependency
+
+v1.2.7 - 2010-08-10
+
+- NO FUNCTIONAL CHANGES.
+- Fix test suite under Ruby 1.9.2.
+- Update example22 application to example23 and Rails 2.3.
+
+v1.2.6 - Never released
+
+- STI fixes.
+
+v1.2.5 - 2009-03-28
+
+- data_fabric should now work as a gem or plugin.
+- Move from echoe to Jeweler for gem maintenance.
+
+v1.2.4 - 2009-03-09
+
+- Add FiveRuns Dash recipe for data_fabric.  See http://dash.fiveruns.com for details.
+- Fixed incorrect remove_connection method signature.
+- The official data_fabric repository is now http://github.com/mperham/data_fabric.
+
+v1.2.3 - 2009-02-05
+
+- Release connections at the end of each Rails action.  If you see 5 second
+  pauses in your application, it's because the connection pool is timing out
+  and releasing checked out connections.
+
+- NB: DataFabric passes all tests against Rails 2.3.0.
+
+v1.2.2 - 2008-12-16
+
+- Fix multithreading issue with AR 2.2.
+
+v1.2.1 - 2008-12-01
+
+- Clean up gem of superfluous files, trims gem size from 160k to 50k.
+  No functional changes.
+
+v1.2.0 - 2008-12-01
+
+- Now supports ActiveRecord 2.2.
+
+v1.1.0 - 2008-11-22
+
+- Cache connections so we don't have to reconnect constantly. (Justin Balthrop, Geni)
+- Clean up logging API (not backwards-compatible)
+- Verify wrapped connections automatically so cached mysql connections older than 8 hours
+  don't die due to timeout.
+- Please note this version is not Rails 2.2-compatible.  DataFabric 2.0 will be.
+
+
+Detailed commit history:
+
+http://github.com/mperham/data_fabric/commits/master

data/README.rdoc

@@ -0,0 +1,125 @@
+= data_fabric
+
+---------------------------
+I'm looking for someone to take over maintenance and responsibility
+for data_fabric.  If you use data_fabric or have an interest in its ongoing
+development, please let me know.
+---------------------------
+
+
+DataFabric provides flexible database connection switching for ActiveRecord.
+
+We needed two features to scale our mysql database: application-level sharding and 
+master/slave replication.  Sharding is the process of splitting a dataset across many 
+independent databases.  This often happens based on geographical region (e.g. 
+craigslist) or category (e.g. ebay).  Replication provides a near-real-time copy 
+of a database which can be used for fault tolerance and to reduce load on the master 
+node.  Combined, you get a scalable database solution which does not require huge 
+hardware to scale to huge volumes.  Or: DPAYEIOB - don't put all your eggs in one 
+basket.  :-)
+
+== Installation
+
+  gem install data_fabric
+
+then in config/environment.rb, add a config.gem entry:
+
+  Rails::Initializer.run do |config|
+    ...
+    config.gem 'data_fabric'
+
+or add data_fabric to your Gemfile:
+
+  gem 'data_fabric'
+
+data_fabric has been tested with Rails 2.3.10 and 3.0.7.
+
+== How does it work?
+
+You describe the topology for your database infrastructure in your model(s).  
+Different models can use different topologies.
+
+  class MyHugeVolumeOfDataModel < ActiveRecord::Base
+    data_fabric :replicated => true, :shard_by => :city
+  end
+
+There are four supported modes of operation, depending on the options given to the 
+data_fabric method.  The plugin will look for connections in your 
+config/database.yml with the following convention:
+
+No connection topology:
+#{environment} - this is the default, as with ActiveRecord, e.g. "production"
+
+  data_fabric :replicated => true
+#{environment}_#{role} - no sharding, just replication, where role is "master" or "slave", e.g. "production_master"
+
+  data_fabric :shard_by => :city
+#{group}_#{shard}_#{environment} - sharding, no replication, e.g. "city_austin_production"
+
+  data_fabric :replicated => true, :shard_by => :city
+#{group}_#{shard}_#{environment}_#{role} - sharding with replication, e.g. "city_austin_production_master"
+
+
+When marked as replicated, all write and transactional operations for the model 
+go to the master, whereas read operations go to the slave.
+
+Since sharding is an application-level concern, your application must set the shard
+to use based on the current request or environment.  The current shard
+is set on a thread local variable.  For example, you can set the shard in an 
+ActionController around_filter based on the user as follows:
+
+  class ApplicationController < ActionController::Base
+    around_filter :select_shard
+
+    private
+    def select_shard(&block)
+      DataFabric.activate_shard(:city => @current_user.city, &block)
+    end
+  end
+
+
+== Warnings
+
+* Sharded models should never be placed in the session store or you will get "Shard not set" errors when the session is persisted.
+* DataFabric does not work in development mode with cache_classes = false.  You will get an error in ActiveRecord's clear_reloadable_connections! method.
+
+== Testing and Bug Reports
+
+If you think you've found a problem with data_fabric, please use the example23 or example30 application to reproduce the bug and send me the diff.
+
+
+== Thanks to...
+
+Rick Olsen of Rails Core:: for the Masochism plugin, which showed me how to bend AR's connection handling to my will
+Bradley Taylor of RailsMachine:: for the advice to shard while at acts_as_conference
+
+
+== Author
+
+Mike Perham <mperham@gmail.com>
+http://mikeperham.com
+http://twitter.com/mperham
+
+
+== LICENSE:
+
+Copyright (c) 2010 Mike Perham
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+ 
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,93 @@
+require 'fileutils'
+include FileUtils::Verbose
+
+require 'rake/testtask'
+Rake::TestTask.new do |t|
+  t.verbose = true
+  t.libs << 'test'
+  t.test_files = FileList['test/*_test.rb']
+end
+
+task :clean do
+  rm_f Dir['*.gem']
+  rm_f Dir['test/*.db']
+  rm_rf 'coverage'
+
+  puts `cd example30 ; rake app:clean`
+end
+
+task :default => :test
+task :test => [:pretest]
+
+desc "Test all versions of ActiveRecord installed locally"
+task :test_all do
+  Gem.source_index.search(Gem::Dependency.new('activerecord', '>=2.0')).each do |spec|
+    puts `rake test AR_VERSION=#{spec.version}`
+  end
+end
+
+task :pretest do
+  setup(false)
+end
+
+task :create_db do
+  setup(true)
+end
+
+def load_database_yml
+  filename = "test/database.yml"
+  if !File.exist?(filename)
+    STDERR.puts "\n*** ERROR ***:\n" <<
+      "You must have a 'test/database.yml' file in order to create the test database. " <<
+      "An example is provided in 'test/database.yml.mysql'.\n\n"
+    exit 1
+  end
+  YAML::load(ERB.new(IO.read(filename)).result)
+end
+
+def setup_connection
+  require 'erb'
+  require 'logger'
+  require 'active_record'
+  ActiveRecord::Base.configurations = load_database_yml
+  ActiveRecord::Base.logger = Logger.new(STDOUT)
+  ActiveRecord::Base.logger.level = Logger::INFO
+end
+
+def using_connection(database_identifier, &block)
+  ActiveRecord::Base.establish_connection(database_identifier)
+  ActiveRecord::Base.connection.instance_eval(&block)
+end
+
+def setup(create = false)
+  setup_connection
+
+  ActiveRecord::Base.configurations.each_pair do |identifier, config|
+    using_connection(identifier) do
+      send("create_#{config['adapter']}", create, config['database'])
+    end
+  end
+end
+
+def create_sqlite3(create, db_name)
+  execute "drop table if exists the_whole_burritos"
+  execute "drop table if exists enchiladas"
+  execute "create table enchiladas (id integer not null primary key, name varchar(30) not null)"
+  execute "insert into enchiladas (id, name) values (1, '#{db_name}')"
+  execute "create table the_whole_burritos (id integer not null primary key, name varchar(30) not null)"
+  execute "insert into the_whole_burritos (id, name) values (1, '#{db_name}')"
+end
+
+def create_mysql(create, db_name)
+  if create
+    execute "drop database if exists #{db_name}"
+    execute "create database #{db_name}"
+  end
+  execute "use #{db_name}"
+  execute "drop table if exists the_whole_burritos"
+  execute "drop table if exists enchiladas"
+  execute "create table enchiladas (id integer not null auto_increment, name varchar(30) not null, primary key(id))"
+  execute "insert into enchiladas (id, name) values (1, '#{db_name}')"
+  execute "create table the_whole_burritos (id integer not null auto_increment, name varchar(30) not null, primary key(id))"
+  execute "insert into the_whole_burritos (id, name) values (1, '#{db_name}')"
+end

data/TESTING.rdoc

@@ -0,0 +1,30 @@
+data_fabric has two layers of tests: unit tests and integration tests.
+
+
+== Running the Unit Tests
+
+The unit tests test both with and without an actual database.  test/database_test.rb
+tests against a database.  The other unit tests mock AR so no actual database is 
+required.  You can use the standard test/database.yml which tests against SQLite3
+or customize the provided test/database.yml.mysql.  The "rake create_db" task will
+set up the necessary databases and tables.
+
+
+== Running the Integration Tests
+
+data_fabric ships with an example application which shows an actual Rails application
+using data_fabric.  You can customize config/database.yml but by default it uses a local
+Sqlite3 database so there should be no customization necessary.
+
+Run the Rake task to prepare the application for testing.  This will delete existing SQLite
+databases and migrate all the databases from scratch.  You should then be able to run the
+example application's tests.
+
+ cd example
+ rake app:prepare
+ rake test
+
+
+== Submitting Bugs
+
+If you think you've found a problem with data_fabric, contact me at mperham AT gmail.com.

data/example30/Gemfile

@@ -0,0 +1,5 @@
+source 'http://rubygems.org'
+
+gem 'rails', '3.0.3'
+gem 'sqlite3-ruby', :require => 'sqlite3'
+gem 'data_fabric', :path => '..'

data/example30/Gemfile.lock

@@ -0,0 +1,79 @@
+PATH
+  remote: ..
+  specs:
+    data_fabric (1.3.0)
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    abstract (1.0.0)
+    actionmailer (3.0.3)
+      actionpack (= 3.0.3)
+      mail (~> 2.2.9)
+    actionpack (3.0.3)
+      activemodel (= 3.0.3)
+      activesupport (= 3.0.3)
+      builder (~> 2.1.2)
+      erubis (~> 2.6.6)
+      i18n (~> 0.4)
+      rack (~> 1.2.1)
+      rack-mount (~> 0.6.13)
+      rack-test (~> 0.5.6)
+      tzinfo (~> 0.3.23)
+    activemodel (3.0.3)
+      activesupport (= 3.0.3)
+      builder (~> 2.1.2)
+      i18n (~> 0.4)
+    activerecord (3.0.3)
+      activemodel (= 3.0.3)
+      activesupport (= 3.0.3)
+      arel (~> 2.0.2)
+      tzinfo (~> 0.3.23)
+    activeresource (3.0.3)
+      activemodel (= 3.0.3)
+      activesupport (= 3.0.3)
+    activesupport (3.0.3)
+    arel (2.0.4)
+    builder (2.1.2)
+    erubis (2.6.6)
+      abstract (>= 1.0.0)
+    i18n (0.4.1)
+    mail (2.2.10)
+      activesupport (>= 2.3.6)
+      i18n (~> 0.4.1)
+      mime-types (~> 1.16)
+      treetop (~> 1.4.8)
+    mime-types (1.16)
+    polyglot (0.3.1)
+    rack (1.2.1)
+    rack-mount (0.6.13)
+      rack (>= 1.0.0)
+    rack-test (0.5.6)
+      rack (>= 1.0)
+    rails (3.0.3)
+      actionmailer (= 3.0.3)
+      actionpack (= 3.0.3)
+      activerecord (= 3.0.3)
+      activeresource (= 3.0.3)
+      activesupport (= 3.0.3)
+      bundler (~> 1.0)
+      railties (= 3.0.3)
+    railties (3.0.3)
+      actionpack (= 3.0.3)
+      activesupport (= 3.0.3)
+      rake (>= 0.8.7)
+      thor (~> 0.14.4)
+    rake (0.8.7)
+    sqlite3-ruby (1.3.1)
+    thor (0.14.4)
+    treetop (1.4.8)
+      polyglot (>= 0.3.1)
+    tzinfo (0.3.23)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  data_fabric!
+  rails (= 3.0.3)
+  sqlite3-ruby

data/example30/README

@@ -0,0 +1,256 @@
+== Welcome to Rails
+
+Rails is a web-application framework that includes everything needed to create
+database-backed web applications according to the Model-View-Control pattern.
+
+This pattern splits the view (also called the presentation) into "dumb"
+templates that are primarily responsible for inserting pre-built data in between
+HTML tags. The model contains the "smart" domain objects (such as Account,
+Product, Person, Post) that holds all the business logic and knows how to
+persist themselves to a database. The controller handles the incoming requests
+(such as Save New Account, Update Product, Show Post) by manipulating the model
+and directing data to the view.
+
+In Rails, the model is handled by what's called an object-relational mapping
+layer entitled Active Record. This layer allows you to present the data from
+database rows as objects and embellish these data objects with business logic
+methods. You can read more about Active Record in
+link:files/vendor/rails/activerecord/README.html.
+
+The controller and view are handled by the Action Pack, which handles both
+layers by its two parts: Action View and Action Controller. These two layers
+are bundled in a single package due to their heavy interdependence. This is
+unlike the relationship between the Active Record and Action Pack that is much
+more separate. Each of these packages can be used independently outside of
+Rails. You can read more about Action Pack in
+link:files/vendor/rails/actionpack/README.html.
+
+
+== Getting Started
+
+1. At the command prompt, create a new Rails application:
+       <tt>rails new myapp</tt> (where <tt>myapp</tt> is the application name)
+
+2. Change directory to <tt>myapp</tt> and start the web server:
+       <tt>cd myapp; rails server</tt> (run with --help for options)
+
+3. Go to http://localhost:3000/ and you'll see:
+       "Welcome aboard: You're riding Ruby on Rails!"
+
+4. Follow the guidelines to start developing your application. You can find
+the following resources handy:
+
+* The Getting Started Guide: http://guides.rubyonrails.org/getting_started.html
+* Ruby on Rails Tutorial Book: http://www.railstutorial.org/
+
+
+== Debugging Rails
+
+Sometimes your application goes wrong. Fortunately there are a lot of tools that
+will help you debug it and get it back on the rails.
+
+First area to check is the application log files. Have "tail -f" commands
+running on the server.log and development.log. Rails will automatically display
+debugging and runtime information to these files. Debugging info will also be
+shown in the browser on requests from 127.0.0.1.
+
+You can also log your own messages directly into the log file from your code
+using the Ruby logger class from inside your controllers. Example:
+
+  class WeblogController < ActionController::Base
+    def destroy
+      @weblog = Weblog.find(params[:id])
+      @weblog.destroy
+      logger.info("#{Time.now} Destroyed Weblog ID ##{@weblog.id}!")
+    end
+  end
+
+The result will be a message in your log file along the lines of:
+
+  Mon Oct 08 14:22:29 +1000 2007 Destroyed Weblog ID #1!
+
+More information on how to use the logger is at http://www.ruby-doc.org/core/
+
+Also, Ruby documentation can be found at http://www.ruby-lang.org/. There are
+several books available online as well:
+
+* Programming Ruby: http://www.ruby-doc.org/docs/ProgrammingRuby/ (Pickaxe)
+* Learn to Program: http://pine.fm/LearnToProgram/ (a beginners guide)
+
+These two books will bring you up to speed on the Ruby language and also on
+programming in general.
+
+
+== Debugger
+
+Debugger support is available through the debugger command when you start your
+Mongrel or WEBrick server with --debugger. This means that you can break out of
+execution at any point in the code, investigate and change the model, and then,
+resume execution! You need to install ruby-debug to run the server in debugging
+mode. With gems, use <tt>sudo gem install ruby-debug</tt>. Example:
+
+  class WeblogController < ActionController::Base
+    def index
+      @posts = Post.find(:all)
+      debugger
+    end
+  end
+
+So the controller will accept the action, run the first line, then present you
+with a IRB prompt in the server window. Here you can do things like:
+
+  >> @posts.inspect
+  => "[#<Post:0x14a6be8
+          @attributes={"title"=>nil, "body"=>nil, "id"=>"1"}>,
+       #<Post:0x14a6620
+          @attributes={"title"=>"Rails", "body"=>"Only ten..", "id"=>"2"}>]"
+  >> @posts.first.title = "hello from a debugger"
+  => "hello from a debugger"
+
+...and even better, you can examine how your runtime objects actually work:
+
+  >> f = @posts.first
+  => #<Post:0x13630c4 @attributes={"title"=>nil, "body"=>nil, "id"=>"1"}>
+  >> f.
+  Display all 152 possibilities? (y or n)
+
+Finally, when you're ready to resume execution, you can enter "cont".
+
+
+== Console
+
+The console is a Ruby shell, which allows you to interact with your
+application's domain model. Here you'll have all parts of the application
+configured, just like it is when the application is running. You can inspect
+domain models, change values, and save to the database. Starting the script
+without arguments will launch it in the development environment.
+
+To start the console, run <tt>rails console</tt> from the application
+directory.
+
+Options:
+
+* Passing the <tt>-s, --sandbox</tt> argument will rollback any modifications
+  made to the database.
+* Passing an environment name as an argument will load the corresponding
+  environment. Example: <tt>rails console production</tt>.
+
+To reload your controllers and models after launching the console run
+<tt>reload!</tt>
+
+More information about irb can be found at:
+link:http://www.rubycentral.com/pickaxe/irb.html
+
+
+== dbconsole
+
+You can go to the command line of your database directly through <tt>rails
+dbconsole</tt>. You would be connected to the database with the credentials
+defined in database.yml. Starting the script without arguments will connect you
+to the development database. Passing an argument will connect you to a different
+database, like <tt>rails dbconsole production</tt>. Currently works for MySQL,
+PostgreSQL and SQLite 3.
+
+== Description of Contents
+
+The default directory structure of a generated Ruby on Rails application:
+
+  |-- app
+  |   |-- controllers
+  |   |-- helpers
+  |   |-- mailers
+  |   |-- models
+  |   `-- views
+  |       `-- layouts
+  |-- config
+  |   |-- environments
+  |   |-- initializers
+  |   `-- locales
+  |-- db
+  |-- doc
+  |-- lib
+  |   `-- tasks
+  |-- log
+  |-- public
+  |   |-- images
+  |   |-- javascripts
+  |   `-- stylesheets
+  |-- script
+  |-- test
+  |   |-- fixtures
+  |   |-- functional
+  |   |-- integration
+  |   |-- performance
+  |   `-- unit
+  |-- tmp
+  |   |-- cache
+  |   |-- pids
+  |   |-- sessions
+  |   `-- sockets
+  `-- vendor
+      `-- plugins
+
+app
+  Holds all the code that's specific to this particular application.
+
+app/controllers
+  Holds controllers that should be named like weblogs_controller.rb for
+  automated URL mapping. All controllers should descend from
+  ApplicationController which itself descends from ActionController::Base.
+
+app/models
+  Holds models that should be named like post.rb. Models descend from
+  ActiveRecord::Base by default.
+
+app/views
+  Holds the template files for the view that should be named like
+  weblogs/index.html.erb for the WeblogsController#index action. All views use
+  eRuby syntax by default.
+
+app/views/layouts
+  Holds the template files for layouts to be used with views. This models the
+  common header/footer method of wrapping views. In your views, define a layout
+  using the <tt>layout :default</tt> and create a file named default.html.erb.
+  Inside default.html.erb, call <% yield %> to render the view using this
+  layout.
+
+app/helpers
+  Holds view helpers that should be named like weblogs_helper.rb. These are
+  generated for you automatically when using generators for controllers.
+  Helpers can be used to wrap functionality for your views into methods.
+
+config
+  Configuration files for the Rails environment, the routing map, the database,
+  and other dependencies.
+
+db
+  Contains the database schema in schema.rb. db/migrate contains all the
+  sequence of Migrations for your schema.
+
+doc
+  This directory is where your application documentation will be stored when
+  generated using <tt>rake doc:app</tt>
+
+lib
+  Application specific libraries. Basically, any kind of custom code that
+  doesn't belong under controllers, models, or helpers. This directory is in
+  the load path.
+
+public
+  The directory available for the web server. Contains subdirectories for
+  images, stylesheets, and javascripts. Also contains the dispatchers and the
+  default HTML files. This should be set as the DOCUMENT_ROOT of your web
+  server.
+
+script
+  Helper scripts for automation and generation.
+
+test
+  Unit and functional tests along with fixtures. When using the rails generate
+  command, template test files will be generated for you and placed in this
+  directory.
+
+vendor
+  External libraries that the application depends on. Also includes the plugins
+  subdirectory. If the app has frozen rails, those gems also go here, under
+  vendor/rails/. This directory is in the load path.