monga 0.0.10 → 0.0.11

data/README.md

@@ -1,11 +1,5 @@
 [![Build Status](https://travis-ci.org/fl00r/monga.png?branch=master)](https://travis-ci.org/fl00r/monga)
 
-This client is under development. You can try 
-
-  * [em-mongo](https://github.com/bcg/em-mongo) with Eventmachine inside
-  * Official [mongo-ruby-driver](https://github.com/mongodb/mongo-ruby-driver) from 10gen
-  * [Moped](http://mongoid.org/en/moped/) from Mongoid guys
-
 # Monga
 
 Yet another [MongoDB](http://www.mongodb.org/) Ruby Client.
@@ -16,6 +10,12 @@ It supports three kind of interfaces:
   * Synchronous (on Fibers)
   * Blocking (over TCPSocket, [kgio](http://bogomips.org/kgio/) actually)
 
+You can also try:
+
+  * [em-mongo](https://github.com/bcg/em-mongo) with Eventmachine inside
+  * Official [mongo-ruby-driver](https://github.com/mongodb/mongo-ruby-driver) from 10gen
+  * [Moped](http://mongoid.org/en/moped/) from Mongoid guys
+
 ## Introduction
 
 Asynchronous API will be familiar to Node.js developers. Instead of Deferrable Object you will receive `err, response` into callback.
@@ -103,110 +103,148 @@ end
 puts "We have got #{docs.size} documents in this pretty array"
 ```
 
-## To Do List
-
-* [ ] Write a Wiki
-* [ ] Write comments
-* [ ] Grammar improvement ;)
-
-### Clients
-* [x] Client (Single instance connection)
-* [x] ReplicaSetClient
-* [ ] MasterSlaveClient
-* [x] ReadPref
-* [ ] Sharding Support
-
-### Connection
-* [x] Connection
-* [x] Autoreconnect
-* [x] Connection Pool
-
-### Protocol
-* [x] OP_QUERY
-* [x] OP_GET_MORE
-* [x] OP_KILL_CURSORS
-* [x] OP_INSERT
-* [x] OP_UPDATE
-* [x] OP_DELETE
-* [x] OP_REPLY
-
-### Database
-* [x] create_collection
-* [x] drop_collection
-* [x] get_last_error
-* [x] drop_indexes
-* [x] get_indexes
-* Authentication
-    * [ ] login
-    * [ ] logout
-    * [ ] add_user
-* [ ] check maxBsonSize / validate
-* [x] cmd
-* [x] eval
-* [x] aggregate
-* [x] distinct
-* [x] group
-* [x] mapReduce
-* [x] text
-* [ ] gridfs?
-
-### Collection
-* QUERY_OP
-    * [x] find
-    * [x] find_one (first)
-    * [x] sorting
-* INSERT_OP
-    * [x] insert (single)
-    * [x] insert (batch)
-    * [x] safe_insert
-    * FLAGS
-        * [x] continue_on_error
-* UPDATE_OP
-    * [x] update
-    * [x] safe_update
-    * FLAGS
-        * [x] upsert
-        * [x] multi_update
-* DELETE_OP
-    * [x] delete
-    * [x] safe_delete
-    * FLAGS
-        * [x] single_remove
-* INDEXES
-    * [x] ensure_index
-    * [x] drop_index
-    * [x] drop_indexes
-    * [x] get_indexes
-* [x] count
-* [x] all
-* [x] cursor
-* [ ] DBRef
-
-### Cursor
-* [x] limit
-* [x] skip
-* [x] batch_size
-* [x] get_more
-* [x] next_document
-* [x] next_batch
-* [x] each_doc
-* [x] kill
-* [x] mark_to_kill
-* [x] batch_kill
-* [x] explain
-* [x] hint
-* Flags
-    * [x] tailable_cursor
-    * [x] slave_ok
-    * [x] no_cursor_timeout
-    * [x] await_data
-    * [x] exhaust
-    * [x] partial
-
-# ISSUES handled with
-
-Some commands, such as `db.getLastError`, `db.count` and other `db.commands` requires `numberToReturn` in OP_QUERY to be setted as `-1`. Also this commands should return a response. If nothing returned it should be interpreted as an Exception. Also, in contrast to the other queries it could return `errmsg` which should be interpreted as an Exception too. Other query methods could return `err` response.
-
-To create index you can't call any `db.ensureIndex` command but you should insert a document into `sytem.indexes` collection manually. To get list of indexes you should fetch all documents from this collection. But to drop index you should call specific `db.dropIndexes` command.
-
-`multi_update` flag works only with `$` commands (i.e. `$set: { title: "blahblah" }`)
+## Find
+
+`find` method allways returns Cursor.
+You can chain `skip`, `limit`, `batch_size` methods.
+`all` will return all matching documents.
+`each_doc` will return document into block.
+`each_batch` will return batch into block.
+For big collections iterating with small batches is a good choice.
+
+```ruby
+# All docs
+collection.find.all
+# All matching docs
+collection.find(name: "Peter").all
+# skip and limit
+collection.find(moderated: true).skip(20).limit(10).all
+# iterating over cursor
+collection.find(country: "Japan").each_doc do |doc|
+  puts doc.inspect
+end
+# Iterating over cursor with predefined batch size
+collection.find(country: "China").batch_size(10_000).skip(1_000_000).each_doc do |chineese|
+  puts chineese.name
+end
+```
+
+## Insert
+
+`insert` method will puts data into socket without waiting for response.
+`safe_insert` method will send current request and then `getLastError` request. So it will be blocked till MongoDB server returns response. If response contains an error it will raise it.
+
+You could use `continue_on_error` flag if you use "batch" insert. In this case MongoDB will try to insert all items in batch and then returns an error if any happened. Otherwise MongoDB will fail on first bad insert and won't continue.
+
+Also you could pass following flags for `safe_insert` method:
+
+  * j
+  * fsync
+  * w
+  * wtimeout
+
+More info about safe methods http://docs.mongodb.org/manual/reference/command/getLastError/#dbcmd.getLastError
+
+```ruby
+collection.insert(_id: 1, name: "Peter")
+collection.safe_insert(_id: 1, name: "Peter")
+#=> Duplicate key error
+collection.safe_insert(_id: 2, name: "Ivan")
+
+# Batch insert
+batch = [
+  { _id: 3, name: "Nick" },
+  { _id: 2, name: "Mary" },
+  { _id: 4, name: "Kate" }
+]
+collection.safe_insert(batch)
+#=> Duplicate error key
+collection.first(_id: 3)
+#=> { _id: 3, name: "Nick" }
+collection.first(_id: 4)
+#=> nil
+
+# Batch insert with `continue_on_error` flag
+collection.safe_insert(batch, continue_on_error: true)
+#=> Duplicate key error
+# but all non existing docs are saved
+collection.first(_id: 4)
+#=> { _id: 4, name: "Kate" }
+```
+
+## Update
+
+`update` method will also only puts data into socket without waiting for any response.
+`safe_update` will raise an error if MongoDB can't update document.
+You could use `upsert` and `multi_update` flags. With `upsert` it will insert current document if it doesn't present in database. With `multi_update` it will update all matching documents, otherwise only firtst will be updated.
+Also `j`, `fsync`, `w`, `wtimeout` flags are available for `safe_update` mthod.
+
+```ruby
+collection.insert(_id: 1, name: "Peter", job: "Dancer")
+collection.insert(_id: 2, name: "Peter", job: "Painter")
+
+collection.update( 
+  { name: "Peter" }, 
+  { "$set" => { job: "Driver" } }
+)
+collection.find(name: "Peter").all
+#=> [ 
+#=>   { _id: 1, name: "Peter", job: "Driver" },
+#=>   { _id: 2, name: "Peter", job: "Painter" }
+#=> ]
+collection.update( 
+  { name: "Peter" }, 
+  { "$set" => { job: "Singer" } },
+  { multi_update: true }
+)
+collection.find(name: "Peter").all
+#=> [ 
+#=>   { _id: 1, name: "Peter", job: "Singer" },
+#=>   { _id: 2, name: "Peter", job: "Singer" }
+#=> ]
+collection.update(
+  { name: "Biork" },
+  { "$set" => { job: "Artist" } }
+)
+collection.first(name: "Bjork")
+#=> nil
+collection.update(
+  { name: "Biork" },
+  { "$set" => { job: "Artist" } },
+  { upsert: true }
+)
+collection.first(name: "Bjork")
+#=> { _id: "Some id", name: "Bjork", job: "Artist" }
+```
+
+## Delete
+Same as `insert` and `update` it has got `safe_delete` method and `j`, `fsync`, `w`, `wtimeout` flags.
+Also it supports `single_remove` flag if you want to delete first matching dcument.
+
+```ruby
+batch = [
+  { _id: 1, name: "Antonio" },
+  { _id: 2, name: "Antonio" },
+  { _id: 3, name: "Antonio" }
+]
+collection.safe_insert(batch)
+collection.count
+#=> 3
+collection.safe_delete({ name: "Antonio" }, single_remove: true)
+collection.count
+#=> 2
+collection.safe_delete(name: "Antonio")
+collection.count
+#=> 0
+```
+
+## Counting
+
+```ruby
+# All items
+collection.count
+# Query
+collection.count(query: { name: "Peter" })
+# Limit, skip
+collection.count(query: { name: "Peter" }, limit: 10, skip: 5)
+```

data/benchmarks/inserts.rb

@@ -1,7 +1,8 @@
 require 'benchmark'
+require 'em-synchrony'
 
-TOTAL_INSERTS = 10000
-TOTAL_READS = 50
+TOTAL_INSERTS = 5000
+TOTAL_READS = 40
 
 chars = ('a'..'z').to_a
 DOCS = [10, 100, 1000, 10000].map do |size|
@@ -53,14 +54,14 @@ fork do
 
     DOCS.each do |size, doc|
       GC.start
-      x.report("Monga: Inserting #{size}b document") do
+      x.report("Monga (blocking): Inserting #{size}b document") do
         TOTAL_INSERTS.times do
           collection.safe_insert(doc)
         end
       end
 
       GC.start
-      x.report("Monga: Reading #{size}b documents") do
+      x.report("Monga (blocking): Reading #{size}b documents") do
         TOTAL_READS.times do
           collection.find.all
         end

data/lib/monga/client.rb

@@ -18,6 +18,7 @@ module Monga
     def initialize(opts = {})
       @opts = opts
       @opts[:type] ||= :block
+      @opts[:timeout] ||= 10
 
       sanitize_opts!
       create_client

data/lib/monga/protocol/insert.rb

@@ -18,7 +18,6 @@ module Monga::Protocol
             msg << doc.to_bson
           end
         when Hash
-          # msg << BSON::BSON_C.serialize(documents).to_s
           msg << documents.to_bson
         end
         msg

data/lib/monga/request.rb

@@ -2,6 +2,7 @@ module Monga
   class Request
     attr_reader :request_id, :connection
 
+    FLAGS = {}
     OP_CODES = {
       reply:           1,
       msg:          1000,
@@ -20,6 +21,7 @@ module Monga
       @collection_name = collection_name
       @options = options
 
+      check_flags
       @request_id = self.class.request_id
     end
 
@@ -69,6 +71,15 @@ module Monga
 
     private
 
+    # Ouch!
+    def check_flags
+      return  unless @options[:query]
+      self.class::FLAGS.each do |k, byte|
+        v = @options[:query].delete(k)
+        @options[k] = v  if v
+      end
+    end
+
     def flags
       flags = 0
       self.class::FLAGS.each do |k, byte|

data/monga.gemspec

@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name          = "monga"
-  spec.version       = "0.0.10"
+  spec.version       = "0.0.11"
   spec.authors       = ["Petr Yanovich"]
   spec.email         = ["fl00r@yandex.ru"]
   spec.description   = %q{Yet another MongoDB Ruby Client}
@@ -22,7 +22,7 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "kgio"
   spec.add_development_dependency "em-synchrony"
 
-  spec.add_dependency "bson", ["~> 2.0.0.rc1"]
+  spec.add_dependency "bson", ["2.0.0.rc1"]
   # spec.add_dependency "bson"
   # spec.add_dependency "bson_ext"
   spec.add_dependency "bin_utils"

data/spec/monga/block/collection_spec.rb

@@ -124,6 +124,10 @@ describe Monga::Collection do
     it "should count all docs with limit and skip" do
       @collection.count(query: { artist: "Madonna" }, limit: 5, skip: 6).must_equal 4
     end
+
+    it "should work with flags" do
+      @collection.count(query: { artist: "Madonna" }, limit: 5, skip: 6, slave_ok: true)
+    end
   end
 
   # ENSURE/DROP INDEX

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: monga
 version: !ruby/object:Gem::Version
-  version: 0.0.10
+  version: 0.0.11
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-07-09 00:00:00.000000000 Z
+date: 2013-09-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -80,7 +80,7 @@ dependencies:
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ~>
+    - - '='
       - !ruby/object:Gem::Version
         version: 2.0.0.rc1
   type: :runtime
@@ -88,7 +88,7 @@ dependencies:
   version_requirements: !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ~>
+    - - '='
       - !ruby/object:Gem::Version
         version: 2.0.0.rc1
 - !ruby/object:Gem::Dependency
@@ -121,7 +121,6 @@ files:
 - README.md
 - Rakefile
 - benchmarks/inserts.rb
-- benchmarks/prof.rb
 - lib/monga.rb
 - lib/monga/client.rb
 - lib/monga/clients/master_slave_client.rb
@@ -181,7 +180,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -4608881135826432488
+      hash: -3001858726288794131
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -190,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -4608881135826432488
+      hash: -3001858726288794131
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25

data/benchmarks/prof.rb

@@ -1,26 +0,0 @@
-require 'ruby-prof'
-require File.expand_path('../../lib/monga', __FILE__)
-require 'mongo'
-include Mongo
-
-total = 100
-# monga_collection = Monga::Client.new(type: :block).get_database("dbTest").get_collection("testCollection")
-mongo_collection = MongoClient.new.db("dbTest").collection("testCollection")
-
-total.times do |i|
-  mongo_collection.insert(title: "Row #{i}")
-end
-RubyProf.start
-mongo_collection.find.to_a
-
-result = RubyProf.stop
-mongo_collection.drop
-
-# Print a flat profile to text
-printer = RubyProf::FlatPrinter.new(result)
-printer.print(STDOUT)
-
-
-
-  mongo_collection = MongoClient.new.db("dbTest").collection("testCollection")
-  mongo_collection.find.to_a