db-charmer 1.7.0.pre7 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +1 -0
- data/CHANGES +1 -1
- data/LICENSE +1 -1
- data/README.rdoc +50 -40
- data/lib/db_charmer/version.rb +1 -1
- metadata +8 -12
data/.gitignore
CHANGED
data/CHANGES
CHANGED
data/LICENSE
CHANGED
data/README.rdoc
CHANGED
@@ -1,33 +1,33 @@
|
|
1
1
|
= DB Charmer - ActiveRecord Connection Magic Plugin
|
2
2
|
|
3
|
-
+DbCharmer+ is a simple yet powerful plugin for ActiveRecord that
|
3
|
+
+DbCharmer+ is a simple yet powerful plugin for ActiveRecord that significantly extends its ability to work with
|
4
|
+
multiple databases and/or database servers. The major features we add to ActiveRecord are:
|
4
5
|
|
5
|
-
1.
|
6
|
-
2.
|
7
|
-
3.
|
8
|
-
4.
|
9
|
-
5.
|
6
|
+
1. Simple management for AR model connections (+switch_connection_to+ method)
|
7
|
+
2. Switching of default AR model connections to separate servers/databases
|
8
|
+
3. Ability to easily choose where your query should go (<tt>Model.on_*</tt> methods family)
|
9
|
+
4. Automated master/slave queries routing (selects go to a slave, updates handled by the master).
|
10
|
+
5. Multiple database migrations with very flexible query routing controls.
|
11
|
+
6. Simple database sharding functionality with multiple sharding methods (value, range, mapping table).
|
10
12
|
|
13
|
+
For more information on the project, you can check out our web site at http://kovyrin.github.com/db-charmer.
|
11
14
|
|
12
|
-
== Installation
|
13
|
-
|
14
|
-
There are two options when approaching db-charmer installation:
|
15
|
-
* using the gem (recommended)
|
16
|
-
* install as a Rails plugin
|
17
15
|
|
18
|
-
|
16
|
+
== Installation
|
19
17
|
|
20
|
-
|
18
|
+
There are two options when approaching +DbCharmer+ installation:
|
19
|
+
* using the gem (recommended and the only way of using it with Rails 3.0+)
|
20
|
+
* install as a Rails plugin (works in Rails 2.x only)
|
21
21
|
|
22
|
-
|
22
|
+
To install as a gem, add this to your Gemfile:
|
23
23
|
|
24
|
-
|
24
|
+
gem 'db-charmer', :require => 'db_charmer'
|
25
25
|
|
26
|
-
To install
|
26
|
+
To install +DbCharmer+ as a Rails plugin use the following command:
|
27
27
|
|
28
|
-
script/plugin install git://github.com/kovyrin/db-charmer.git
|
28
|
+
./script/plugin install git://github.com/kovyrin/db-charmer.git
|
29
29
|
|
30
|
-
_Notice_: If you use
|
30
|
+
_Notice_: If you use +DbCharmer+ in a non-rails project, you may need to set <tt>DbCharmer.env</tt> to a correct value
|
31
31
|
before using any of its connection management methods. Correct value here is a valid <tt>database.yml</tt>
|
32
32
|
first-level section name.
|
33
33
|
|
@@ -395,29 +395,35 @@ impact, but it is worth noting.
|
|
395
395
|
== Simple Sharding Support
|
396
396
|
|
397
397
|
Starting with the release 1.6.0 of +DbCharmer+ we have added support for simple database sharding
|
398
|
-
to our ActiveRecord extensions.
|
399
|
-
|
398
|
+
to our ActiveRecord extensions. Even though this feature is tested in production, we do not recommend
|
399
|
+
using it in your applications without complete understanding of the principles of its work.
|
400
400
|
|
401
|
-
At this point we support
|
401
|
+
At this point we support four sharding methods:
|
402
402
|
|
403
|
-
1) range - really simple sharding method that allows you to take a table, slice is to a set of
|
403
|
+
1) +range+ - really simple sharding method that allows you to take a table, slice is to a set of
|
404
404
|
smaller tables with pre-defined ranges of primary keys and then put those smaller tables to
|
405
405
|
different databases/servers. This could be useful for situations where you have a huge table that
|
406
406
|
is slowly growing and you just want to keep it simple and split the table load into a few servers
|
407
407
|
without building any complex sharding schemes.
|
408
408
|
|
409
|
-
2) hash_map - pretty simple sharding method that allows you to take a table and slice it to a set
|
409
|
+
2) +hash_map+ - pretty simple sharding method that allows you to take a table and slice it to a set
|
410
410
|
of smaller tables by some key that has a pre-defined key of values. For example, list of US mailing
|
411
411
|
addresses could be sharded by states, where you'd be able to define which states are stored in which
|
412
412
|
databases/servers.
|
413
413
|
|
414
|
-
3) db_block_map - this is a really complex sharding method that allows you to shard your table into a
|
414
|
+
3) +db_block_map+ - this is a really complex sharding method that allows you to shard your table into a
|
415
415
|
set of small fixed-size blocks that then would be assigned to a set of shards (databases/servers).
|
416
416
|
Whenever you would need an additional blocks they would be allocated automatically and then balanced
|
417
417
|
across the shards you have defined in your database. This method could be used to scale out huge
|
418
418
|
tables with hundreds of millions to billions of rows and allows relatively easy re-sharding techniques
|
419
419
|
to be implemented on top.
|
420
420
|
|
421
|
+
4) +db_block_group_map+ - really similar to the +db_block_map+ method with a single difference: this method
|
422
|
+
allows you to have a set of databases (table groups) on each server and every group would be handled as a
|
423
|
+
separate shard of data. This approach is really useful for pre-sharding of your data before scaling your
|
424
|
+
application out. You can easily start with one server, having 10-20-50 separate databases, and then
|
425
|
+
move those databases to different servers as you see your database outgrow one machine.
|
426
|
+
|
421
427
|
|
422
428
|
=== How to enable sharding?
|
423
429
|
|
@@ -514,17 +520,17 @@ allows you to switch to the default shard:
|
|
514
520
|
Text.on_default_shard.create(:body => 'hello', :user_id => 123)
|
515
521
|
|
516
522
|
And finally, there is a method that allows you to run your code on each shard in the system (at this
|
517
|
-
point the method is supported in db_block_map
|
523
|
+
point the method is supported in db_block_map and db_block_group_map methods only):
|
518
524
|
|
519
525
|
Event.on_each_shard { |event| event.delete_all }
|
520
526
|
|
521
527
|
|
522
528
|
=== Defining your own sharding methods
|
523
529
|
|
524
|
-
It is
|
530
|
+
It is possible with +DbCharmer+ for the users to define their own sharding methods. You need to do a
|
525
531
|
few things to implement your very own sharding scheme:
|
526
532
|
|
527
|
-
1) Create a class with a name DbCharmer::Sharding::Method::YourOwnName
|
533
|
+
1) Create a class with a name <tt>DbCharmer::Sharding::Method::YourOwnName</tt>
|
528
534
|
|
529
535
|
2) Implement at least a constructor <tt>initialize(config)</tt> and a lookup instance
|
530
536
|
method <tt>shard_for_key(key)</tt> that would return either a connection name from <tt>database.yml</tt>
|
@@ -534,7 +540,7 @@ file or just a hash of connection parameters for rails connection adapters.
|
|
534
540
|
|
535
541
|
DbCharmer::Sharding.register_connection(
|
536
542
|
:name => :some_name,
|
537
|
-
:method => :your_own_name, # your
|
543
|
+
:method => :your_own_name, # your sharding method class name in lower case
|
538
544
|
... some additional parameters if needed ...
|
539
545
|
)
|
540
546
|
|
@@ -555,32 +561,36 @@ would return +true+ if you do support default shard specification and +false+ ot
|
|
555
561
|
=== Adding support for shards enumeration in your custom sharding methods
|
556
562
|
|
557
563
|
To add shards enumeration support to your custom-sharded models you need to implement
|
558
|
-
|
559
|
-
|
560
|
-
establish shard connections in a loop.
|
564
|
+
an instance method +shard_connections+ on your class. This method should return an array of
|
565
|
+
sharding connection names or connection configurations to be used to establish connections in a loop.
|
561
566
|
|
562
567
|
|
563
568
|
== Documentation/Questions
|
564
569
|
|
565
|
-
For more information
|
566
|
-
|
567
|
-
|
568
|
-
|
570
|
+
For more information about the library, please visit our site at http://kovyrin.github.com/db-charmer.
|
571
|
+
If you need more defails on DbCharmer internals, please check out the source code. All the plugin's
|
572
|
+
code is ~100% covered with tests (located in a separate staging rails project
|
573
|
+
at http://github.com/kovyrin/db-charmer-sandbox). The project has unit tests for all or at
|
574
|
+
least the most actively used code paths.
|
569
575
|
|
570
576
|
If you have any questions regarding this project, you could contact the author using
|
571
577
|
the DbCharmer Users Group mailing list:
|
572
578
|
|
573
579
|
- Group Info: http://groups.google.com/group/db-charmer
|
574
|
-
- Subscribe using the info page or by sending an email to db-charmer-subscribe@googlegroups.com
|
580
|
+
- Subscribe using the info page or by sending an email to mailto:db-charmer-subscribe@googlegroups.com
|
575
581
|
|
576
582
|
|
577
583
|
== What Ruby and Rails implementations does it work for?
|
578
584
|
|
579
|
-
We have a continuous integration
|
580
|
-
|
581
|
-
|
585
|
+
We have a continuous integration setup for this gem on with Rails 2.2, 2.3 and 3.0 using a few
|
586
|
+
different versions of Ruby. For more information about the build matrix please visit
|
587
|
+
http://github.com/kovyrin/db-charmer-sandbox web site.
|
588
|
+
|
589
|
+
In addition to CI testing, we use this gem in production on Scribd.com (one of the largest RoR
|
590
|
+
sites in the world) with Ruby Enterprise Edition and Rails 2.2, Rails 2.3, Sinatra and plain
|
591
|
+
Rack applications.
|
582
592
|
|
583
|
-
Starting with version 1.7.0 we support Rails 3.0, but until further notice we consider it a
|
593
|
+
Starting with version 1.7.0 we support Rails 3.0, but until further notice we consider it a
|
584
594
|
beta-quality feature since we do not run any production software on this version of Rails. If you
|
585
595
|
run your application on Rails 3.0, please contact the author.
|
586
596
|
|
data/lib/db_charmer/version.rb
CHANGED
metadata
CHANGED
@@ -1,15 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: db-charmer
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
5
|
-
prerelease:
|
4
|
+
hash: 11
|
5
|
+
prerelease:
|
6
6
|
segments:
|
7
7
|
- 1
|
8
8
|
- 7
|
9
9
|
- 0
|
10
|
-
|
11
|
-
- 7
|
12
|
-
version: 1.7.0.pre7
|
10
|
+
version: 1.7.0
|
13
11
|
platform: ruby
|
14
12
|
authors:
|
15
13
|
- Oleksiy Kovyrin
|
@@ -17,7 +15,7 @@ autorequire:
|
|
17
15
|
bindir: bin
|
18
16
|
cert_chain: []
|
19
17
|
|
20
|
-
date: 2011-08-
|
18
|
+
date: 2011-08-29 00:00:00 Z
|
21
19
|
dependencies:
|
22
20
|
- !ruby/object:Gem::Dependency
|
23
21
|
name: activesupport
|
@@ -160,14 +158,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
160
158
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
161
159
|
none: false
|
162
160
|
requirements:
|
163
|
-
- - "
|
161
|
+
- - ">="
|
164
162
|
- !ruby/object:Gem::Version
|
165
|
-
hash:
|
163
|
+
hash: 3
|
166
164
|
segments:
|
167
|
-
-
|
168
|
-
|
169
|
-
- 1
|
170
|
-
version: 1.3.1
|
165
|
+
- 0
|
166
|
+
version: "0"
|
171
167
|
requirements: []
|
172
168
|
|
173
169
|
rubyforge_project:
|