mongo 1.1.5 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. data/README.md +15 -15
  2. data/Rakefile +38 -17
  3. data/docs/FAQ.md +4 -0
  4. data/docs/HISTORY.md +59 -0
  5. data/docs/RELEASES.md +33 -0
  6. data/docs/REPLICA_SETS.md +13 -16
  7. data/lib/mongo/collection.rb +157 -69
  8. data/lib/mongo/connection.rb +189 -65
  9. data/lib/mongo/cursor.rb +43 -29
  10. data/lib/mongo/db.rb +63 -43
  11. data/lib/mongo/exceptions.rb +4 -1
  12. data/lib/mongo/gridfs/grid.rb +1 -1
  13. data/lib/mongo/gridfs/grid_ext.rb +1 -1
  14. data/lib/mongo/gridfs/grid_file_system.rb +1 -1
  15. data/lib/mongo/gridfs/grid_io.rb +89 -8
  16. data/lib/mongo/gridfs/grid_io_fix.rb +1 -1
  17. data/lib/mongo/repl_set_connection.rb +72 -20
  18. data/lib/mongo/test.rb +20 -0
  19. data/lib/mongo/util/conversions.rb +1 -1
  20. data/lib/mongo/util/core_ext.rb +11 -1
  21. data/lib/mongo/util/pool.rb +67 -15
  22. data/lib/mongo/util/server_version.rb +1 -1
  23. data/lib/mongo/util/support.rb +1 -1
  24. data/lib/mongo/util/uri_parser.rb +127 -13
  25. data/lib/mongo.rb +38 -2
  26. data/test/async/collection_test.rb +224 -0
  27. data/test/async/connection_test.rb +24 -0
  28. data/test/async/cursor_test.rb +162 -0
  29. data/test/async/worker_pool_test.rb +99 -0
  30. data/test/auxillary/fork_test.rb +30 -0
  31. data/test/auxillary/repl_set_auth_test.rb +58 -0
  32. data/test/auxillary/threaded_authentication_test.rb +101 -0
  33. data/test/bson/bson_test.rb +140 -28
  34. data/test/bson/byte_buffer_test.rb +18 -0
  35. data/test/bson/object_id_test.rb +14 -1
  36. data/test/bson/ordered_hash_test.rb +7 -0
  37. data/test/bson/timestamp_test.rb +24 -0
  38. data/test/collection_test.rb +104 -15
  39. data/test/connection_test.rb +78 -2
  40. data/test/conversions_test.rb +10 -11
  41. data/test/cursor_fail_test.rb +1 -1
  42. data/test/cursor_message_test.rb +1 -1
  43. data/test/cursor_test.rb +33 -4
  44. data/test/db_api_test.rb +30 -52
  45. data/test/db_test.rb +3 -3
  46. data/test/grid_file_system_test.rb +0 -1
  47. data/test/grid_io_test.rb +72 -1
  48. data/test/grid_test.rb +16 -16
  49. data/test/load/resque/load.rb +21 -0
  50. data/test/load/resque/processor.rb +26 -0
  51. data/test/load/thin/load.rb +24 -0
  52. data/test/load/unicorn/load.rb +23 -0
  53. data/test/load/unicorn/unicorn.rb +29 -0
  54. data/test/replica_sets/connect_test.rb +11 -1
  55. data/test/replica_sets/connection_string_test.rb +32 -0
  56. data/test/replica_sets/query_secondaries.rb +16 -0
  57. data/test/replica_sets/query_test.rb +10 -0
  58. data/test/replica_sets/replication_ack_test.rb +2 -0
  59. data/test/replica_sets/rs_test_helper.rb +9 -11
  60. data/test/support/hash_with_indifferent_access.rb +0 -13
  61. data/test/support_test.rb +0 -1
  62. data/test/test_helper.rb +27 -8
  63. data/test/tools/auth_repl_set_manager.rb +14 -0
  64. data/test/tools/load.rb +58 -0
  65. data/test/tools/repl_set_manager.rb +34 -9
  66. data/test/tools/sharding_manager.rb +202 -0
  67. data/test/tools/test.rb +3 -12
  68. data/test/unit/collection_test.rb +20 -24
  69. data/test/unit/connection_test.rb +4 -18
  70. data/test/unit/cursor_test.rb +16 -6
  71. data/test/unit/db_test.rb +10 -11
  72. data/test/unit/repl_set_connection_test.rb +0 -23
  73. data/test/unit/safe_test.rb +3 -3
  74. data/test/uri_test.rb +91 -0
  75. metadata +49 -12
  76. data/docs/1.0_UPGRADE.md +0 -21
data/README.md CHANGED
@@ -10,9 +10,10 @@ This documentation includes other articles of interest, include:
10
10
  4. [GridFS in Ruby](http://api.mongodb.org/ruby/current/file.GridFS.html).
11
11
  5. [Frequently Asked Questions](http://api.mongodb.org/ruby/current/file.FAQ.html).
12
12
  6. [History](http://api.mongodb.org/ruby/current/file.HISTORY.html).
13
+ 6. [Release plan](http://api.mongodb.org/ruby/current/file.RELEASES.html).
13
14
  7. [Credits](http://api.mongodb.org/ruby/current/file.CREDITS.html).
14
15
 
15
- Here's a quick code sample. Again, see the [MongoDB Ruby Tutorial](file.TUTORIAL.html)
16
+ Here's a quick code sample. Again, see the [MongoDB Ruby Tutorial](http://api.mongodb.org/ruby/current/file.TUTORIAL.html)
16
17
  for much more:
17
18
 
18
19
  require 'rubygems'
@@ -129,7 +130,7 @@ The driver is thread-safe.
129
130
 
130
131
  ## Connection Pooling
131
132
 
132
- As of v0.18, the driver implements connection pooling. By default, only one
133
+ The driver implements connection pooling. By default, only one
133
134
  socket connection will be opened to MongoDB. However, if you're running a
134
135
  multi-threaded application, you can specify a maximum pool size and a maximum
135
136
  timeout for waiting for old connections to be released to the pool.
@@ -141,20 +142,11 @@ To set up a pooled connection to a single MongoDB instance:
141
142
  Though the pooling architecture will undoubtedly evolve, it currently owes much credit
142
143
  to the connection pooling implementations in ActiveRecord and PyMongo.
143
144
 
144
- ## Using with Phusion Passenger and Unicorn
145
+ ## Forking
145
146
 
146
- When Passenger and Unicorn are in smart spawning mode you need to be sure that child
147
- processes will create a new connection to the database. In Passenger, this can be handled like so:
148
-
149
- if defined?(PhusionPassenger)
150
- PhusionPassenger.on_event(:starting_worker_process) do |forked|
151
- if forked
152
- # Create new connection here
153
- end
154
- end
155
- end
156
-
157
- The above code should be put into a Rails initializer or other initialization script.
147
+ Certain Ruby application servers work by forking, and it has long been necessary to
148
+ re-establish the child process's connection to the database after fork. But with the release
149
+ of v1.3.0, the Ruby driver detects forking and reconnects automatically.
158
150
 
159
151
  ## String Encoding
160
152
 
@@ -259,6 +251,14 @@ Notes:
259
251
  * Cursors will timeout on the server after 10 minutes. If you need to keep a cursor
260
252
  open for more than 10 minutes, specify `:timeout => false` when you create the cursor.
261
253
 
254
+ ## Socket timeouts
255
+
256
+ The Ruby driver support timeouts on socket read operations. To enable them, set the
257
+ `:op_timeout` option when you create a `Mongo::Connection` object.
258
+
259
+ If implementing higher-level timeouts, using tools like `Rack::Timeout`, it's very important
260
+ to call `Mongo::Connection#close` to prevent the subsequent operation from receiving the previous
261
+ request.
262
262
 
263
263
  # Testing
264
264
 
data/Rakefile CHANGED
@@ -5,10 +5,6 @@ require 'fileutils'
5
5
  require 'rake'
6
6
  require 'rake/testtask'
7
7
  require 'rake/gempackagetask'
8
- begin
9
- require 'rake/contrib/rubyforgepublisher'
10
- rescue LoadError
11
- end
12
8
  require 'rbconfig'
13
9
  include Config
14
10
  ENV['TEST_MODE'] = 'TRUE'
@@ -26,7 +22,7 @@ namespace :build do
26
22
  jar_dir = File.join(java_dir, 'jar')
27
23
 
28
24
  jruby_jar = File.join(jar_dir, 'jruby.jar')
29
- mongo_jar = File.join(jar_dir, 'mongo-2.2.jar')
25
+ mongo_jar = File.join(jar_dir, 'mongo-2.4.jar')
30
26
  bson_jar = File.join(jar_dir, 'bson-2.2.jar')
31
27
 
32
28
  src_base = File.join(java_dir, 'src')
@@ -77,41 +73,49 @@ namespace :test do
77
73
  Rake::TestTask.new(:rs) do |t|
78
74
  t.test_files = FileList['test/replica_sets/*_test.rb']
79
75
  t.verbose = true
76
+ t.ruby_opts << '-w'
80
77
  end
81
78
 
82
79
  Rake::TestTask.new(:unit) do |t|
83
80
  t.test_files = FileList['test/unit/*_test.rb']
84
81
  t.verbose = true
82
+ t.ruby_opts << '-w'
85
83
  end
86
84
 
87
85
  Rake::TestTask.new(:functional) do |t|
88
86
  t.test_files = FileList['test/*_test.rb']
89
87
  t.verbose = true
88
+ t.ruby_opts << '-w'
90
89
  end
91
90
 
92
91
  Rake::TestTask.new(:pooled_threading) do |t|
93
92
  t.test_files = FileList['test/threading/*_test.rb']
94
93
  t.verbose = true
94
+ t.ruby_opts << '-w'
95
95
  end
96
96
 
97
97
  Rake::TestTask.new(:auto_reconnect) do |t|
98
98
  t.test_files = FileList['test/auxillary/autoreconnect_test.rb']
99
99
  t.verbose = true
100
+ t.ruby_opts << '-w'
100
101
  end
101
102
 
102
103
  Rake::TestTask.new(:authentication) do |t|
103
104
  t.test_files = FileList['test/auxillary/authentication_test.rb']
104
105
  t.verbose = true
106
+ t.ruby_opts << '-w'
105
107
  end
106
108
 
107
109
  Rake::TestTask.new(:new_features) do |t|
108
110
  t.test_files = FileList['test/auxillary/1.4_features.rb']
109
111
  t.verbose = true
112
+ t.ruby_opts << '-w'
110
113
  end
111
114
 
112
115
  Rake::TestTask.new(:bson) do |t|
113
116
  t.test_files = FileList['test/bson/*_test.rb']
114
117
  t.verbose = true
118
+ t.ruby_opts << '-w'
115
119
  end
116
120
 
117
121
  task :drop_databases do |t|
@@ -138,16 +142,24 @@ task :ydoc do
138
142
  require File.join(File.dirname(__FILE__), 'lib', 'mongo')
139
143
  out = File.join('ydoc', Mongo::VERSION)
140
144
  FileUtils.rm_rf('ydoc')
141
- system "yardoc lib/**/*.rb lib/mongo/**/*.rb lib/bson/**/*.rb -e yard/yard_ext.rb -p yard/templates -o #{out} --title MongoRuby-#{Mongo::VERSION} --files docs/TUTORIAL.md,docs/GridFS.md,docs/FAQ.md,docs/REPLICA_SETS.md,docs/WRITE_CONCERN.md,docs/HISTORY.md,docs/CREDITS.md,docs/1.0_UPGRADE.md"
145
+ system "yardoc lib/**/*.rb lib/mongo/**/*.rb lib/bson/**/*.rb -e yard/yard_ext.rb -p yard/templates -o #{out} --title MongoRuby-#{Mongo::VERSION} --files docs/TUTORIAL.md,docs/GridFS.md,docs/FAQ.md,docs/REPLICA_SETS.md,docs/WRITE_CONCERN.md,docs/HISTORY.md,docs/CREDITS.md,docs/RELEASES.md"
142
146
  end
143
147
 
144
148
  namespace :bamboo do
149
+ task :ci_reporter do
150
+ begin
151
+ require 'ci/reporter/rake/test_unit'
152
+ rescue LoadError
153
+ warn "Warning: Unable to load ci_reporter gem."
154
+ end
155
+ end
156
+
145
157
  namespace :test do
146
- task :ruby do
158
+ task :ruby => [:ci_reporter, "ci:setup:testunit"] do
147
159
  Rake::Task['test:ruby'].invoke
148
160
  end
149
161
 
150
- task :c do
162
+ task :c => [:ci_reporter, "ci:setup:testunit"] do
151
163
  Rake::Task['gem:install_extensions'].invoke
152
164
  Rake::Task['test:c'].invoke
153
165
  end
@@ -158,21 +170,30 @@ namespace :gem do
158
170
 
159
171
  desc "Install the gem locally"
160
172
  task :install do
161
- sh "gem build bson.gemspec"
162
- sh "gem install --no-rdoc --no-ri bson-*.gem"
173
+ `gem build bson.gemspec`
174
+ `gem install --no-rdoc --no-ri bson-*.gem`
163
175
 
164
- sh "gem build mongo.gemspec"
165
- sh "gem install --no-rdoc --no-ri mongo-*.gem"
176
+ `gem build mongo.gemspec`
177
+ `gem install --no-rdoc --no-ri mongo-*.gem`
166
178
 
167
- sh "rm mongo-*.gem"
168
- sh "rm bson-*.gem"
179
+ `rm mongo-*.gem`
180
+ `rm bson-*.gem`
169
181
  end
170
182
 
171
183
  desc "Install the optional c extensions"
172
184
  task :install_extensions do
173
- sh "gem build bson_ext.gemspec"
174
- sh "gem install --no-rdoc --no-ri bson_ext-*.gem"
175
- sh "rm bson_ext-*.gem"
185
+ `gem uninstall bson_ext`
186
+ `gem build bson_ext.gemspec`
187
+ `gem install --no-rdoc --no-ri bson_ext-*.gem`
188
+ `rm bson_ext-*.gem`
189
+ end
190
+
191
+ desc "Build all gems"
192
+ task :build_all do
193
+ `gem build mongo.gemspec`
194
+ `gem build bson.gemspec`
195
+ `gem build bson.java.gemspec`
196
+ `gem build bson_ext.gemspec`
176
197
  end
177
198
 
178
199
  end
data/docs/FAQ.md CHANGED
@@ -110,3 +110,7 @@ Without further investigation, it's impossible to know exactly what has caused t
110
110
  Because of the indeterminacy involved, the MongoDB drivers will not retry operations on connection failure. How connection failures should be handled is entirely dependent on the application. Therefore, we leave it to the application developers to make the best decision in this case.
111
111
 
112
112
  The drivers will reconnect on the subsequent operation.
113
+
114
+ #### I ocassionally get an error saying that responses are out of order. What's happening?
115
+
116
+ See (this JIRA issue)[http://jira.mongodb.org/browse/RUBY-221].
data/docs/HISTORY.md CHANGED
@@ -1,5 +1,64 @@
1
1
  # MongoDB Ruby Driver History
2
2
 
3
+ ### 1.3.0
4
+ 2011-4-04
5
+
6
+ * Add option to set timeouts on socket read calls using the
7
+ Mongo::Connection :op_timeout option.
8
+ * Add StringIO methods to GridIO objects
9
+ * Support for BSON timestamp type with BSON::Timestamp
10
+ * Change the BSON binary subtype from 2 to 0
11
+ * Remove private method Connection#reset_conection
12
+ and deprecate public method ReplSetConnection#reset_connection
13
+ * ByteBuffer#== and OrderedHash#dup (Hongli Lai)
14
+ * Better check for UTF8 validity in Ruby 1.9
15
+ * Added previously removed Connection#host and Connection#port
16
+ * Added transformers to allow Mongo::Cursor to allow instantiated objects (John Nunemaker)
17
+ * Automated reconnection on fork
18
+ * Added Cursor#next alias for Cursor#next_document
19
+ * Audit tests after enabling warnings (Wojciech Piekutowski)
20
+ * Various bug fixes thanks to Datanoise, Hongli Lai, and Mauro Pompilio
21
+
22
+ ### 1.2.4
23
+ 2011-2-23
24
+
25
+ * Fix the exception message shown when there's an IOError (Mauro Pompilio)
26
+ * Another update to map-reduce docs for v1.8. Note that if you use the new
27
+ output option {:out => {:inline => true}}, then you must also specify
28
+ :raw => true.
29
+
30
+ ### 1.2.3
31
+ 2011-2-22
32
+
33
+ * Update docs for map-reduce command
34
+ * Minor doc fix
35
+
36
+ ### 1.2.2
37
+ 2011-2-15
38
+
39
+ * Improved replica set failover for edge case.
40
+ * Fix for REE on OSX (Hongli Lai)
41
+
42
+ ### 1.2.1
43
+ 2011-1-18
44
+
45
+ * Enable authentication with connection pooling.
46
+ * Allow custom logging with Connection#instrument (CodeMonkeySteve)
47
+ * Minor fixes and doc improvements.
48
+
49
+ ### 1.2.0
50
+ 2011-1-18
51
+
52
+ * Some minor improvements. See commit history.
53
+
54
+ ### 1.2.rc0
55
+ 2011-1-5
56
+
57
+ Lots of cleanup and minor bug fixes.
58
+ * Issues resolved: http://jira.mongodb.org/browse/RUBY/fixforversion/10222
59
+ * Updated Java BSON to Java driver 2.4.
60
+ * Platform gem for JRuby bson.
61
+
3
62
  ### 1.1.5
4
63
  2010-12-15
5
64
 
data/docs/RELEASES.md ADDED
@@ -0,0 +1,33 @@
1
+ # MongoDB Ruby Driver Release Plan
2
+
3
+ This is a description of a formalized release plan that will take effect
4
+ with version 1.3.0.
5
+
6
+ ## Semantic versioning
7
+
8
+ The most significant difference is that releases will now adhere to the conventions of
9
+ [semantic versioning](http://semver.org). In particular, we will strictly abide by the
10
+ following release rules:
11
+
12
+ 1. Patch versions of the driver (Z in x.y.Z) will be released only when backward-compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior.
13
+
14
+ 2. Minor versions (Y in x.Y.z) will be released if new, backward-compatible functionality is introduced to the public API.
15
+
16
+ 3. Major versions (X in X.y.z) will be incremented if any backward-incompatibl changes are introduced to the public API.
17
+
18
+ This policy will clearly indicate to users when an upgrade may affect their code. As a side effect, version numbers will climb more quickly than before.
19
+
20
+
21
+ ## Release checklist
22
+
23
+ Before each relese to Rubygems.org, the following steps will be taken:
24
+
25
+ 1. All driver tests will be run on Linux, OS X, and Windows via continuous integration system.
26
+
27
+ 2. HISTORY file will document all significant commits.
28
+
29
+ 3. Version number will be incremented per the semantic version spec described above.
30
+
31
+ 4. Appropriate branches and tags will be created in Git repository, as necessary.
32
+
33
+ 5. Docs will be updated to the latest version of the driver and posted [online](http://api.mongodb.org/ruby/current/index.html).
data/docs/REPLICA_SETS.md CHANGED
@@ -6,12 +6,13 @@ Here follow a few considerations for those using the MongoDB Ruby driver with [r
6
6
 
7
7
  First, make sure that you've configured and initialized a replica set.
8
8
 
9
- Use `ReplSetConnection.new` to connect to a replica set:
9
+ Use `ReplSetConnection.new` to connect to a replica set. This method, which accepts a variable number of arugments,
10
+ takes a list of seed nodes followed by any connection options. You'll want to specify at least two seed nodes. This gives
11
+ the driver more chances to connect in the event that any one seed node is offline. Once the driver connects, it will
12
+ cache the replica set topology as reported by the given seed node and use that information if a failover is later required.
10
13
 
11
14
  @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017])
12
15
 
13
- The driver will attempt to connect to a master node and, when found, will replace all seed nodes with known members of the replica set.
14
-
15
16
  ### Read slaves
16
17
 
17
18
  If you want to read from a seconday node, you can pass :read_secondary => true to ReplSetConnection#new.
@@ -38,18 +39,14 @@ every half second and time out after thirty seconds.
38
39
 
39
40
  # Ensure retry upon failure
40
41
  def rescue_connection_failure(max_retries=60)
41
- success = false
42
- retries = 0
43
- while !success
44
- begin
45
- yield
46
- success = true
47
- rescue Mongo::ConnectionFailure => ex
48
- retries += 1
49
- raise ex if retries >= max_retries
50
- sleep(0.5)
51
- end
52
- end
42
+ retries = 0
43
+ begin
44
+ yield
45
+ rescue Mongo::ConnectionFailure => ex
46
+ retries += 1
47
+ raise ex if retries > max_retries
48
+ sleep(0.5)
49
+ retry
53
50
  end
54
51
  end
55
52
 
@@ -67,7 +64,7 @@ The Ruby driver (>= 1.1.5) includes unit tests for verifying replica set behavio
67
64
  rake test:rs
68
65
 
69
66
  The suite will set up a five-node replica set by itself and ensure that driver behaves correctly even in the face
70
- of individual node failures. Node that the `mongod` executable must be in the search path for this to work.
67
+ of individual node failures. Note that the `mongod` executable must be in the search path for this to work.
71
68
 
72
69
  ### Further Reading
73
70