db-charmer 1.7.0.pre7 → 1.7.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +1 -0
- data/CHANGES +1 -1
- data/LICENSE +1 -1
- data/README.rdoc +50 -40
- data/lib/db_charmer/version.rb +1 -1
- metadata +8 -12
data/.gitignore
CHANGED
data/CHANGES
CHANGED
data/LICENSE
CHANGED
data/README.rdoc
CHANGED
@@ -1,33 +1,33 @@
|
|
1
1
|
= DB Charmer - ActiveRecord Connection Magic Plugin
|
2
2
|
|
3
|
-
+DbCharmer+ is a simple yet powerful plugin for ActiveRecord that
|
3
|
+
+DbCharmer+ is a simple yet powerful plugin for ActiveRecord that significantly extends its ability to work with
|
4
|
+
multiple databases and/or database servers. The major features we add to ActiveRecord are:
|
4
5
|
|
5
|
-
1.
|
6
|
-
2.
|
7
|
-
3.
|
8
|
-
4.
|
9
|
-
5.
|
6
|
+
1. Simple management for AR model connections (+switch_connection_to+ method)
|
7
|
+
2. Switching of default AR model connections to separate servers/databases
|
8
|
+
3. Ability to easily choose where your query should go (<tt>Model.on_*</tt> methods family)
|
9
|
+
4. Automated master/slave queries routing (selects go to a slave, updates handled by the master).
|
10
|
+
5. Multiple database migrations with very flexible query routing controls.
|
11
|
+
6. Simple database sharding functionality with multiple sharding methods (value, range, mapping table).
|
10
12
|
|
13
|
+
For more information on the project, you can check out our web site at http://kovyrin.github.com/db-charmer.
|
11
14
|
|
12
|
-
== Installation
|
13
|
-
|
14
|
-
There are two options when approaching db-charmer installation:
|
15
|
-
* using the gem (recommended)
|
16
|
-
* install as a Rails plugin
|
17
15
|
|
18
|
-
|
16
|
+
== Installation
|
19
17
|
|
20
|
-
|
18
|
+
There are two options when approaching +DbCharmer+ installation:
|
19
|
+
* using the gem (recommended and the only way of using it with Rails 3.0+)
|
20
|
+
* install as a Rails plugin (works in Rails 2.x only)
|
21
21
|
|
22
|
-
|
22
|
+
To install as a gem, add this to your Gemfile:
|
23
23
|
|
24
|
-
|
24
|
+
gem 'db-charmer', :require => 'db_charmer'
|
25
25
|
|
26
|
-
To install
|
26
|
+
To install +DbCharmer+ as a Rails plugin use the following command:
|
27
27
|
|
28
|
-
script/plugin install git://github.com/kovyrin/db-charmer.git
|
28
|
+
./script/plugin install git://github.com/kovyrin/db-charmer.git
|
29
29
|
|
30
|
-
_Notice_: If you use
|
30
|
+
_Notice_: If you use +DbCharmer+ in a non-rails project, you may need to set <tt>DbCharmer.env</tt> to a correct value
|
31
31
|
before using any of its connection management methods. Correct value here is a valid <tt>database.yml</tt>
|
32
32
|
first-level section name.
|
33
33
|
|
@@ -395,29 +395,35 @@ impact, but it is worth noting.
|
|
395
395
|
== Simple Sharding Support
|
396
396
|
|
397
397
|
Starting with the release 1.6.0 of +DbCharmer+ we have added support for simple database sharding
|
398
|
-
to our ActiveRecord extensions.
|
399
|
-
|
398
|
+
to our ActiveRecord extensions. Even though this feature is tested in production, we do not recommend
|
399
|
+
using it in your applications without complete understanding of the principles of its work.
|
400
400
|
|
401
|
-
At this point we support
|
401
|
+
At this point we support four sharding methods:
|
402
402
|
|
403
|
-
1) range - really simple sharding method that allows you to take a table, slice is to a set of
|
403
|
+
1) +range+ - really simple sharding method that allows you to take a table, slice is to a set of
|
404
404
|
smaller tables with pre-defined ranges of primary keys and then put those smaller tables to
|
405
405
|
different databases/servers. This could be useful for situations where you have a huge table that
|
406
406
|
is slowly growing and you just want to keep it simple and split the table load into a few servers
|
407
407
|
without building any complex sharding schemes.
|
408
408
|
|
409
|
-
2) hash_map - pretty simple sharding method that allows you to take a table and slice it to a set
|
409
|
+
2) +hash_map+ - pretty simple sharding method that allows you to take a table and slice it to a set
|
410
410
|
of smaller tables by some key that has a pre-defined key of values. For example, list of US mailing
|
411
411
|
addresses could be sharded by states, where you'd be able to define which states are stored in which
|
412
412
|
databases/servers.
|
413
413
|
|
414
|
-
3) db_block_map - this is a really complex sharding method that allows you to shard your table into a
|
414
|
+
3) +db_block_map+ - this is a really complex sharding method that allows you to shard your table into a
|
415
415
|
set of small fixed-size blocks that then would be assigned to a set of shards (databases/servers).
|
416
416
|
Whenever you would need an additional blocks they would be allocated automatically and then balanced
|
417
417
|
across the shards you have defined in your database. This method could be used to scale out huge
|
418
418
|
tables with hundreds of millions to billions of rows and allows relatively easy re-sharding techniques
|
419
419
|
to be implemented on top.
|
420
420
|
|
421
|
+
4) +db_block_group_map+ - really similar to the +db_block_map+ method with a single difference: this method
|
422
|
+
allows you to have a set of databases (table groups) on each server and every group would be handled as a
|
423
|
+
separate shard of data. This approach is really useful for pre-sharding of your data before scaling your
|
424
|
+
application out. You can easily start with one server, having 10-20-50 separate databases, and then
|
425
|
+
move those databases to different servers as you see your database outgrow one machine.
|
426
|
+
|
421
427
|
|
422
428
|
=== How to enable sharding?
|
423
429
|
|
@@ -514,17 +520,17 @@ allows you to switch to the default shard:
|
|
514
520
|
Text.on_default_shard.create(:body => 'hello', :user_id => 123)
|
515
521
|
|
516
522
|
And finally, there is a method that allows you to run your code on each shard in the system (at this
|
517
|
-
point the method is supported in db_block_map
|
523
|
+
point the method is supported in db_block_map and db_block_group_map methods only):
|
518
524
|
|
519
525
|
Event.on_each_shard { |event| event.delete_all }
|
520
526
|
|
521
527
|
|
522
528
|
=== Defining your own sharding methods
|
523
529
|
|
524
|
-
It is
|
530
|
+
It is possible with +DbCharmer+ for the users to define their own sharding methods. You need to do a
|
525
531
|
few things to implement your very own sharding scheme:
|
526
532
|
|
527
|
-
1) Create a class with a name DbCharmer::Sharding::Method::YourOwnName
|
533
|
+
1) Create a class with a name <tt>DbCharmer::Sharding::Method::YourOwnName</tt>
|
528
534
|
|
529
535
|
2) Implement at least a constructor <tt>initialize(config)</tt> and a lookup instance
|
530
536
|
method <tt>shard_for_key(key)</tt> that would return either a connection name from <tt>database.yml</tt>
|
@@ -534,7 +540,7 @@ file or just a hash of connection parameters for rails connection adapters.
|
|
534
540
|
|
535
541
|
DbCharmer::Sharding.register_connection(
|
536
542
|
:name => :some_name,
|
537
|
-
:method => :your_own_name, # your
|
543
|
+
:method => :your_own_name, # your sharding method class name in lower case
|
538
544
|
... some additional parameters if needed ...
|
539
545
|
)
|
540
546
|
|
@@ -555,32 +561,36 @@ would return +true+ if you do support default shard specification and +false+ ot
|
|
555
561
|
=== Adding support for shards enumeration in your custom sharding methods
|
556
562
|
|
557
563
|
To add shards enumeration support to your custom-sharded models you need to implement
|
558
|
-
|
559
|
-
|
560
|
-
establish shard connections in a loop.
|
564
|
+
an instance method +shard_connections+ on your class. This method should return an array of
|
565
|
+
sharding connection names or connection configurations to be used to establish connections in a loop.
|
561
566
|
|
562
567
|
|
563
568
|
== Documentation/Questions
|
564
569
|
|
565
|
-
For more information
|
566
|
-
|
567
|
-
|
568
|
-
|
570
|
+
For more information about the library, please visit our site at http://kovyrin.github.com/db-charmer.
|
571
|
+
If you need more defails on DbCharmer internals, please check out the source code. All the plugin's
|
572
|
+
code is ~100% covered with tests (located in a separate staging rails project
|
573
|
+
at http://github.com/kovyrin/db-charmer-sandbox). The project has unit tests for all or at
|
574
|
+
least the most actively used code paths.
|
569
575
|
|
570
576
|
If you have any questions regarding this project, you could contact the author using
|
571
577
|
the DbCharmer Users Group mailing list:
|
572
578
|
|
573
579
|
- Group Info: http://groups.google.com/group/db-charmer
|
574
|
-
- Subscribe using the info page or by sending an email to db-charmer-subscribe@googlegroups.com
|
580
|
+
- Subscribe using the info page or by sending an email to mailto:db-charmer-subscribe@googlegroups.com
|
575
581
|
|
576
582
|
|
577
583
|
== What Ruby and Rails implementations does it work for?
|
578
584
|
|
579
|
-
We have a continuous integration
|
580
|
-
|
581
|
-
|
585
|
+
We have a continuous integration setup for this gem on with Rails 2.2, 2.3 and 3.0 using a few
|
586
|
+
different versions of Ruby. For more information about the build matrix please visit
|
587
|
+
http://github.com/kovyrin/db-charmer-sandbox web site.
|
588
|
+
|
589
|
+
In addition to CI testing, we use this gem in production on Scribd.com (one of the largest RoR
|
590
|
+
sites in the world) with Ruby Enterprise Edition and Rails 2.2, Rails 2.3, Sinatra and plain
|
591
|
+
Rack applications.
|
582
592
|
|
583
|
-
Starting with version 1.7.0 we support Rails 3.0, but until further notice we consider it a
|
593
|
+
Starting with version 1.7.0 we support Rails 3.0, but until further notice we consider it a
|
584
594
|
beta-quality feature since we do not run any production software on this version of Rails. If you
|
585
595
|
run your application on Rails 3.0, please contact the author.
|
586
596
|
|
data/lib/db_charmer/version.rb
CHANGED
metadata
CHANGED
@@ -1,15 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: db-charmer
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
5
|
-
prerelease:
|
4
|
+
hash: 11
|
5
|
+
prerelease:
|
6
6
|
segments:
|
7
7
|
- 1
|
8
8
|
- 7
|
9
9
|
- 0
|
10
|
-
|
11
|
-
- 7
|
12
|
-
version: 1.7.0.pre7
|
10
|
+
version: 1.7.0
|
13
11
|
platform: ruby
|
14
12
|
authors:
|
15
13
|
- Oleksiy Kovyrin
|
@@ -17,7 +15,7 @@ autorequire:
|
|
17
15
|
bindir: bin
|
18
16
|
cert_chain: []
|
19
17
|
|
20
|
-
date: 2011-08-
|
18
|
+
date: 2011-08-29 00:00:00 Z
|
21
19
|
dependencies:
|
22
20
|
- !ruby/object:Gem::Dependency
|
23
21
|
name: activesupport
|
@@ -160,14 +158,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
160
158
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
161
159
|
none: false
|
162
160
|
requirements:
|
163
|
-
- - "
|
161
|
+
- - ">="
|
164
162
|
- !ruby/object:Gem::Version
|
165
|
-
hash:
|
163
|
+
hash: 3
|
166
164
|
segments:
|
167
|
-
-
|
168
|
-
|
169
|
-
- 1
|
170
|
-
version: 1.3.1
|
165
|
+
- 0
|
166
|
+
version: "0"
|
171
167
|
requirements: []
|
172
168
|
|
173
169
|
rubyforge_project:
|