jonbell-mongo 1.3.1.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (88) hide show
  1. data/LICENSE.txt +190 -0
  2. data/README.md +333 -0
  3. data/Rakefile +215 -0
  4. data/bin/mongo_console +21 -0
  5. data/docs/CREDITS.md +123 -0
  6. data/docs/FAQ.md +116 -0
  7. data/docs/GridFS.md +158 -0
  8. data/docs/HISTORY.md +263 -0
  9. data/docs/RELEASES.md +33 -0
  10. data/docs/REPLICA_SETS.md +72 -0
  11. data/docs/TUTORIAL.md +247 -0
  12. data/docs/WRITE_CONCERN.md +28 -0
  13. data/lib/mongo.rb +97 -0
  14. data/lib/mongo/collection.rb +895 -0
  15. data/lib/mongo/connection.rb +926 -0
  16. data/lib/mongo/cursor.rb +474 -0
  17. data/lib/mongo/db.rb +617 -0
  18. data/lib/mongo/exceptions.rb +71 -0
  19. data/lib/mongo/gridfs/grid.rb +107 -0
  20. data/lib/mongo/gridfs/grid_ext.rb +57 -0
  21. data/lib/mongo/gridfs/grid_file_system.rb +146 -0
  22. data/lib/mongo/gridfs/grid_io.rb +485 -0
  23. data/lib/mongo/gridfs/grid_io_fix.rb +38 -0
  24. data/lib/mongo/repl_set_connection.rb +356 -0
  25. data/lib/mongo/util/conversions.rb +89 -0
  26. data/lib/mongo/util/core_ext.rb +60 -0
  27. data/lib/mongo/util/pool.rb +177 -0
  28. data/lib/mongo/util/server_version.rb +71 -0
  29. data/lib/mongo/util/support.rb +82 -0
  30. data/lib/mongo/util/uri_parser.rb +185 -0
  31. data/mongo.gemspec +34 -0
  32. data/test/auxillary/1.4_features.rb +166 -0
  33. data/test/auxillary/authentication_test.rb +68 -0
  34. data/test/auxillary/autoreconnect_test.rb +41 -0
  35. data/test/auxillary/fork_test.rb +30 -0
  36. data/test/auxillary/repl_set_auth_test.rb +58 -0
  37. data/test/auxillary/slave_connection_test.rb +36 -0
  38. data/test/auxillary/threaded_authentication_test.rb +101 -0
  39. data/test/bson/binary_test.rb +15 -0
  40. data/test/bson/bson_test.rb +654 -0
  41. data/test/bson/byte_buffer_test.rb +208 -0
  42. data/test/bson/hash_with_indifferent_access_test.rb +38 -0
  43. data/test/bson/json_test.rb +17 -0
  44. data/test/bson/object_id_test.rb +154 -0
  45. data/test/bson/ordered_hash_test.rb +210 -0
  46. data/test/bson/timestamp_test.rb +24 -0
  47. data/test/collection_test.rb +910 -0
  48. data/test/connection_test.rb +324 -0
  49. data/test/conversions_test.rb +119 -0
  50. data/test/cursor_fail_test.rb +75 -0
  51. data/test/cursor_message_test.rb +43 -0
  52. data/test/cursor_test.rb +483 -0
  53. data/test/db_api_test.rb +738 -0
  54. data/test/db_connection_test.rb +15 -0
  55. data/test/db_test.rb +315 -0
  56. data/test/grid_file_system_test.rb +259 -0
  57. data/test/grid_io_test.rb +209 -0
  58. data/test/grid_test.rb +258 -0
  59. data/test/load/thin/load.rb +24 -0
  60. data/test/load/unicorn/load.rb +23 -0
  61. data/test/replica_sets/connect_test.rb +112 -0
  62. data/test/replica_sets/connection_string_test.rb +32 -0
  63. data/test/replica_sets/count_test.rb +35 -0
  64. data/test/replica_sets/insert_test.rb +53 -0
  65. data/test/replica_sets/pooled_insert_test.rb +55 -0
  66. data/test/replica_sets/query_secondaries.rb +108 -0
  67. data/test/replica_sets/query_test.rb +51 -0
  68. data/test/replica_sets/replication_ack_test.rb +66 -0
  69. data/test/replica_sets/rs_test_helper.rb +27 -0
  70. data/test/safe_test.rb +68 -0
  71. data/test/support/hash_with_indifferent_access.rb +186 -0
  72. data/test/support/keys.rb +45 -0
  73. data/test/support_test.rb +18 -0
  74. data/test/test_helper.rb +102 -0
  75. data/test/threading/threading_with_large_pool_test.rb +90 -0
  76. data/test/threading_test.rb +87 -0
  77. data/test/tools/auth_repl_set_manager.rb +14 -0
  78. data/test/tools/repl_set_manager.rb +266 -0
  79. data/test/unit/collection_test.rb +130 -0
  80. data/test/unit/connection_test.rb +85 -0
  81. data/test/unit/cursor_test.rb +109 -0
  82. data/test/unit/db_test.rb +94 -0
  83. data/test/unit/grid_test.rb +49 -0
  84. data/test/unit/pool_test.rb +9 -0
  85. data/test/unit/repl_set_connection_test.rb +59 -0
  86. data/test/unit/safe_test.rb +125 -0
  87. data/test/uri_test.rb +91 -0
  88. metadata +224 -0
data/docs/GridFS.md ADDED
@@ -0,0 +1,158 @@
1
+ # GridFS in Ruby
2
+
3
+ GridFS, which stands for "Grid File Store," is a specification for storing large files in MongoDB. It works by dividing a file into manageable chunks and storing each of those chunks as a separate document. GridFS requires two collections to achieve this: one collection stores each file's metadata (e.g., name, size, etc.) and another stores the chunks themselves. If you're interested in more details, check out the [GridFS Specification](http://www.mongodb.org/display/DOCS/GridFS+Specification).
4
+
5
+ ### The Grid class
6
+
7
+ The [Grid class](Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid:
8
+
9
+
10
+ @db = Mongo::Connection.new.db('social_site')
11
+ @grid = Grid.new(@db)
12
+
13
+ #### Saving files
14
+ Once you have a Grid object, you can start saving data to it. The data can be either a string or an IO-like object that responds to a #read method:
15
+
16
+
17
+ # Saving string data
18
+ id = @grid.put("here's some string / binary data")
19
+
20
+ # Saving IO data and including the optional filename
21
+ image = File.open("me.jpg")
22
+ id2 = @grid.put(image, :filename => "me.jpg")
23
+
24
+
25
+ Grid#put returns an object id, which you can use to retrieve the file:
26
+
27
+
28
+ # Get the string we saved
29
+ file = @grid.get(id)
30
+
31
+ # Get the file we saved
32
+ image = @grid.get(id2)
33
+
34
+
35
+ #### File metadata
36
+
37
+ There are accessors for the various file attributes:
38
+
39
+
40
+ image.filename
41
+ # => "me.jpg"
42
+
43
+ image.content_type
44
+ # => "image/jpg"
45
+
46
+ image.file_length
47
+ # => 502357
48
+
49
+ image.upload_date
50
+ # => Mon Mar 01 16:18:30 UTC 2010
51
+
52
+ # Read all the image's data at once
53
+ image.read
54
+
55
+ # Read the first 100k bytes of the image
56
+ image.read(100 * 1024)
57
+
58
+
59
+ When putting a file, you can set many of these attributes and write arbitrary metadata:
60
+
61
+
62
+ # Saving IO data
63
+ file = File.open("me.jpg")
64
+ id2 = @grid.put(file,
65
+ :filename => "my-avatar.jpg"
66
+ :content_type => "application/jpg",
67
+ :_id => 'a-unique-id-to-use-in-lieu-of-a-random-one',
68
+ :chunk_size => 100 * 1024,
69
+ :metadata => {'description' => "taken after a game of ultimate"})
70
+
71
+
72
+ #### Safe mode
73
+
74
+ A kind of safe mode is built into the GridFS specification. When you save a file, and MD5 hash is created on the server. If you save the file in safe mode, an MD5 will be created on the client for comparison with the server version. If the two hashes don't match, an exception will be raised.
75
+
76
+
77
+ image = File.open("me.jpg")
78
+ id2 = @grid.put(image, "my-avatar.jpg", :safe => true)
79
+
80
+
81
+ #### Deleting files
82
+
83
+ Deleting a file is as simple as providing the id:
84
+
85
+
86
+ @grid.delete(id2)
87
+
88
+
89
+ ### The GridFileSystem class
90
+
91
+ [GridFileSystem](http://api.mongodb.org/ruby/current/Mongo/GridFileSystem.html) is a light emulation of a file system and therefore has a couple of unique properties. The first is that filenames are assumed to be unique. The second, a consequence of the first, is that files are versioned. To see what this means, let's create a GridFileSystem instance:
92
+
93
+ #### Saving files
94
+
95
+ @db = Mongo::Connection.new.db("social_site")
96
+ @fs = GridFileSystem.new(@db)
97
+
98
+ Now suppose we want to save the file 'me.jpg.' This is easily done using a filesystem-like API:
99
+
100
+
101
+ image = File.open("me.jpg")
102
+ @fs.open("me.jpg", "w") do |f|
103
+ f.write image
104
+ end
105
+
106
+
107
+ We can then retrieve the file by filename:
108
+
109
+
110
+ image = @fs.open("me.jpg", "r") {|f| f.read }
111
+
112
+
113
+ No problems there. But what if we need to replace the file? That too is straightforward:
114
+
115
+
116
+ image = File.open("me-dancing.jpg")
117
+ @fs.open("me.jpg", "w") do |f|
118
+ f.write image
119
+ end
120
+
121
+
122
+ But a couple things need to be kept in mind. First is that the original 'me.jpg' will be available until the new 'me.jpg' saves. From then on, calls to the #open method will always return the most recently saved version of a file. But, and this the second point, old versions of the file won't be deleted. So if you're going to be rewriting files often, you could end up with a lot of old versions piling up. One solution to this is to use the :delete_old options when writing a file:
123
+
124
+
125
+ image = File.open("me-dancing.jpg")
126
+ @fs.open("me.jpg", "w", :delete_old => true) do |f|
127
+ f.write image
128
+ end
129
+
130
+
131
+ This will delete all but the latest version of the file.
132
+
133
+
134
+ #### Deleting files
135
+
136
+ When you delete a file by name, you delete all versions of that file:
137
+
138
+
139
+ @fs.delete("me.jpg")
140
+
141
+
142
+ #### Metadata and safe mode
143
+
144
+ All of the options for storing metadata and saving in safe mode are available for the GridFileSystem class:
145
+
146
+
147
+ image = File.open("me.jpg")
148
+ @fs.open('my-avatar.jpg', w,
149
+ :content_type => "application/jpg",
150
+ :metadata => {'description' => "taken on 3/1/2010 after a game of ultimate"},
151
+ :_id => 'a-unique-id-to-use-instead-of-the-automatically-generated-one',
152
+ :safe => true) { |f| f.write image }
153
+
154
+
155
+ ### Advanced Users
156
+
157
+ Astute code readers will notice that the Grid and GridFileSystem classes are merely thin wrappers around an underlying [GridIO class](http://api.mongodb.org/ruby/current/Mongo/GridIO.html). This means that it's easy to customize the GridFS implementation presented here; just use GridIO for all the low-level work, and build the API you need in an external manager class similar to Grid or GridFileSystem.
158
+
data/docs/HISTORY.md ADDED
@@ -0,0 +1,263 @@
1
+ # MongoDB Ruby Driver History
2
+
3
+ ### 1.3.1
4
+ 2011-5-10
5
+
6
+ * Fix GridIO#gets infinite loop error (Ryan McGeary)
7
+ * Fix BSON::OrderedHash#reject! leaving keys with null values (rpt. by Ben Poweski)
8
+ * Minor semantic fix for OrderedHash#reject!
9
+ * Fix Mongo::DB to allow symbols in method traversing collection names (rpt. by Chris Griego)
10
+ * Support new server regex option "s" (dotall). This is folded in with \m in Ruby.
11
+ * Fix so that Cursor#close hits the right node when :read_secondary is enabled.
12
+ * Support maxScan, showDiskLoc, and returnKey cursor options.
13
+ * Make DB#validate_collection compatible with server v1.9.1.
14
+ * Fix so that GridIO#gets returns local md5 with md5 matches server md5 (Steve Tantra).
15
+ * Fix bug in BSON::OrderedHash that prevents YAML.load (Ian Warshak).
16
+ * Fix example from /examples.
17
+ * Ensure that we do not modify hash arguments by calling Hash#dup when appropriate.
18
+ * Ensure that JRuby deserializer preserves binary subtypes properly.
19
+ * Fix for streaming an empty file into GridFS (Daniël van de Burgt).
20
+ * Minor doc fixes.
21
+
22
+ ### 1.3.0
23
+ 2011-4-04
24
+
25
+ * Add option to set timeouts on socket read calls using the
26
+ Mongo::Connection :op_timeout option.
27
+ * Add StringIO methods to GridIO objects
28
+ * Support for BSON timestamp type with BSON::Timestamp
29
+ * Change the BSON binary subtype from 2 to 0
30
+ * Remove private method Connection#reset_conection
31
+ and deprecate public method ReplSetConnection#reset_connection
32
+ * ByteBuffer#== and OrderedHash#dup (Hongli Lai)
33
+ * Better check for UTF8 validity in Ruby 1.9
34
+ * Added previously removed Connection#host and Connection#port
35
+ * Added transformers to allow Mongo::Cursor to allow instantiated objects (John Nunemaker)
36
+ * Automated reconnection on fork
37
+ * Added Cursor#next alias for Cursor#next_document
38
+ * Audit tests after enabling warnings (Wojciech Piekutowski)
39
+ * Various bug fixes thanks to Datanoise, Hongli Lai, and Mauro Pompilio
40
+
41
+ ### 1.2.4
42
+ 2011-2-23
43
+
44
+ * Fix the exception message shown when there's an IOError (Mauro Pompilio)
45
+ * Another update to map-reduce docs for v1.8. Note that if you use the new
46
+ output option {:out => {:inline => true}}, then you must also specify
47
+ :raw => true.
48
+
49
+ ### 1.2.3
50
+ 2011-2-22
51
+
52
+ * Update docs for map-reduce command
53
+ * Minor doc fix
54
+
55
+ ### 1.2.2
56
+ 2011-2-15
57
+
58
+ * Improved replica set failover for edge case.
59
+ * Fix for REE on OSX (Hongli Lai)
60
+
61
+ ### 1.2.1
62
+ 2011-1-18
63
+
64
+ * Enable authentication with connection pooling.
65
+ * Allow custom logging with Connection#instrument (CodeMonkeySteve)
66
+ * Minor fixes and doc improvements.
67
+
68
+ ### 1.2.0
69
+ 2011-1-18
70
+
71
+ * Some minor improvements. See commit history.
72
+
73
+ ### 1.2.rc0
74
+ 2011-1-5
75
+
76
+ Lots of cleanup and minor bug fixes.
77
+ * Issues resolved: http://jira.mongodb.org/browse/RUBY/fixforversion/10222
78
+ * Updated Java BSON to Java driver 2.4.
79
+ * Platform gem for JRuby bson.
80
+
81
+ ### 1.1.5
82
+ 2010-12-15
83
+
84
+ * ReplSetConnection class. This must be used for replica set connections from
85
+ now on. You can still use Connection.multi, but that method has been deprecated.
86
+ * Automated replica set tests. rake test:rs
87
+ * Check that request and response ids match.
88
+ * Several bug fixes. See the commit history for details.
89
+
90
+ ### 1.1.4
91
+ 2010-11-30
92
+
93
+ * Important connection failure fix.
94
+ * ObjectId#to_s optimization (David Cuadrado).
95
+
96
+ ### 1.1.3
97
+ 2010-11-29
98
+
99
+ * Distributed reads for replica set secondaries. See /docs/examples/replica_set.rb and
100
+ http://api.mongodb.org/ruby/current/file.REPLICA_SETS.html for details.
101
+ * Note: when connecting to a replica set, you must use Connection#multi.
102
+ * Cursor#count takes optional skip and limit
103
+ * Collection#ensure_index for caching index creation calls
104
+ * Collection#update and Collection#remove now return error object when using safe mode
105
+ * Important fix for int/long serialization on bug introduced in 1.0.9
106
+ * Numerous tweaks and bug fixes.
107
+
108
+ ### 1.1.2
109
+ 2010-11-4
110
+
111
+ * Two critical fixes to automated failover and replica sets.
112
+ * Bug passing :timeout to Cursor.
113
+ * Permit safe mode specification on Connection, Collection, and DB levels.
114
+ * Specify replica set name on connect to verify connection to the right set.
115
+ * Misc. reorganization of project and docs.
116
+
117
+ ### 1.1.1
118
+ 2010-10-14
119
+
120
+ * Several critical JRuby bug fixes
121
+ * Fixes for JRuby in 1.9 mode
122
+ * Check keys and move id only when necessary for JRuby encoder
123
+
124
+ ## 1.1
125
+ 2010-10-4
126
+
127
+ * Official JRuby support via Java extensons for BSON (beta)
128
+ * Connection#lock! and Connection#unlock! for easy fsync lock
129
+ * Note: BSON::Code is no longer a subclass of String.
130
+
131
+ ### 1.0.9
132
+ 2010-9-20
133
+
134
+ * Significant performance improvements (with a lot of help from Hongli Lai)
135
+
136
+ ### 1.0.8
137
+ 2010-8-27
138
+
139
+ * Cursor#rewind! and more consistent Cursor Enumberable behavior
140
+ * Deprecated ObjectID for ObjectId
141
+ * Numerous minor bug fixes.
142
+
143
+ ### 1.0.7
144
+ 2010-8-4
145
+
146
+ * A few minor test/doc fixes.
147
+ * Better tests for replica sets and replication acknowledgment.
148
+ * Deprecated DB#error and DB#last_status
149
+
150
+ ### 1.0.6
151
+ 2010-7-26
152
+
153
+ * Replica set support.
154
+ * Collection#map_reduce bug fix.
155
+
156
+ ### 1.0.5
157
+ 2010-7-13
158
+
159
+ * Fix for bug introduced in 1.0.4.
160
+
161
+ ### 1.0.4
162
+ 2010-7-13
163
+
164
+ * Removed deprecated
165
+ * Cursor admin option
166
+ * DB#query
167
+ * DB#create_index (use Collection#create_index)
168
+ * DB#command only takes hash options now
169
+ * j2bson executable (neomantra)
170
+ * Fixed bson_ext compilation on Solaris (slyphon)
171
+ * System JS helpers (neovintage)
172
+ * Use one mutex per thread on pooled connections (cremes)
173
+ * Check for CursorNotFound response flag
174
+ * MapReduce can return raw command output using :raw
175
+ * BSON::OrderedHash equality with other Ruby hashes (Ryan Angilly)
176
+ * Fix for broken Socket.send with large payloads (Frédéric De Jaeger)
177
+ * Lots of minor improvements. See commmits.
178
+
179
+ ### 1.0.3
180
+ 2010-6-15
181
+
182
+ * Optimiztion for BSON::OrderedHash
183
+ * Some important fixes.
184
+
185
+ ### 1.0.2
186
+ 2010-6-5
187
+
188
+ This is a minor release for fixing an incompatibility with MongoDB v1.5.2
189
+
190
+ * Fix for boolean response on commands for core server v1.5.2
191
+ * BSON.read_bson_document and b2json executable (neomantra)
192
+ * BSON::ObjectID() shortcut for BSON::ObjectID.from_string (tmm1)
193
+ * Various bug fixes.
194
+
195
+ ### 1.0.1
196
+ 2010-5-7
197
+
198
+ * set Encoding.default_internal
199
+ * DEPRECATE JavaScript string on Collection#find. You now must specify $where explicitly.
200
+ * Added Grid#exist? and GridFileSystem#exist?
201
+ * Support for replication acknowledgment
202
+ * Support for $slice
203
+ * Namespaced OrderedHash under BSON (sleverbor)
204
+
205
+ ## 1.0
206
+ 2010-4-29
207
+ Note: if upgrading from versions prior to 0.20, be sure to upgrade
208
+ to 0.20 before upgrading to 1.0.
209
+
210
+ * Inspected ObjectID is represented in MongoDB extended json format.
211
+ * Support for tailable cursors.
212
+ * Configurable query response batch size (thx. to Aman Gupta)
213
+
214
+ * bson_ext installs on early release of Ruby 1.8.5 (dfitzgibbon)
215
+ * Deprecated DB#create_index. Use Collection#create_index index.
216
+ * Removed deprecated Grid#put syntax; no longer requires a filename.
217
+
218
+ ### 0.20.1
219
+ 2010-4-7
220
+
221
+ * Added bson gem dependency.
222
+
223
+ ### 0.20
224
+ 2010-4-7
225
+
226
+ If upgrading from a previous version of the Ruby driver, please read these notes carefully,
227
+ along with the 0.20_UPGRADE doc.
228
+
229
+ * Support for new commands:
230
+ * Collection#find_and_modify
231
+ * Collection#stats
232
+ * DB#stats
233
+ * Query :fields options allows for values of 0 to exclude fields (houdini, railsjedi).
234
+ * GridFS
235
+ * Option to delete old versions of GridFileSystem entries.
236
+ * Filename is now optional for Grid#put.
237
+ * Option to write arbitrary attributes to a file: @grid.put(@data, :favorite_phrase => "blimey!")
238
+ * Indexes created on the chunks collection are now unique. If you have an existing chunks collection,
239
+ you may want to remove
240
+ * Removed the following deprecated items:
241
+ * GridStore class
242
+ * RegexpOfHolding class
243
+ * Paired connections must now be initialized with Connection.paired
244
+
245
+ * BSON-related code extracted into two separate gems: bson and bson_ext (thx to Chuck Remes).
246
+ * mongo_ext no longer exists.
247
+ * BSON::Binary constructor can now take a string, which will be packed into an array.
248
+ * Exception class adjustments:
249
+ * Mongo::InvalidObjectID moved to BSON::InvalidObjectID
250
+ * Mongo::InvalidDocument moved to BSON::InvalidDocument
251
+ * Mongo::InvalidStringEncoding moved to BSON::InvalidStringEncoding
252
+ * Mongo::InvalidName replaced by Mongo::InvalidNSName and BSON::InvalidKeyName
253
+ * BSON types are now namespaced under the BSON module. These types include:
254
+ * Binary
255
+ * ObjectID
256
+ * Code
257
+ * DBRef
258
+ * MinKey and MaxKey
259
+ * Extensions compile on Rubinius (Chuck Remes).
260
+
261
+ ## Prior to 0.20
262
+
263
+ See git revisions.
data/docs/RELEASES.md ADDED
@@ -0,0 +1,33 @@
1
+ # MongoDB Ruby Driver Release Plan
2
+
3
+ This is a description of a formalized release plan that will take effect
4
+ with version 1.3.0.
5
+
6
+ ## Semantic versioning
7
+
8
+ The most significant difference is that releases will now adhere to the conventions of
9
+ [semantic versioning](http://semver.org). In particular, we will strictly abide by the
10
+ following release rules:
11
+
12
+ 1. Patch versions of the driver (Z in x.y.Z) will be released only when backward-compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior.
13
+
14
+ 2. Minor versions (Y in x.Y.z) will be released if new, backward-compatible functionality is introduced to the public API.
15
+
16
+ 3. Major versions (X in X.y.z) will be incremented if any backward-incompatibl changes are introduced to the public API.
17
+
18
+ This policy will clearly indicate to users when an upgrade may affect their code. As a side effect, version numbers will climb more quickly than before.
19
+
20
+
21
+ ## Release checklist
22
+
23
+ Before each relese to Rubygems.org, the following steps will be taken:
24
+
25
+ 1. All driver tests will be run on Linux, OS X, and Windows via continuous integration system.
26
+
27
+ 2. HISTORY file will document all significant commits.
28
+
29
+ 3. Version number will be incremented per the semantic version spec described above.
30
+
31
+ 4. Appropriate branches and tags will be created in Git repository, as necessary.
32
+
33
+ 5. Docs will be updated to the latest version of the driver and posted [online](http://api.mongodb.org/ruby/current/index.html).