mongo 2.8.0 → 2.9.0.rc0

data/lib/mongo/version.rb

@@ -17,5 +17,5 @@ module Mongo
   # The current version of the driver.
   #
   # @since 2.0.0
-  VERSION = '2.8.0'.freeze
+  VERSION = '2.9.0.rc0'.freeze
 end

data/spec/README.md

@@ -1,85 +1,212 @@
-The tests run against a MongoDB cluster which is
-configured and started externally to the test suite. This allows
-running the entire test suite against, for example, a standalone
-mongod as well as a replica set. The flip side to this is the
-test suite will not work without a running mongo cluster, and
-tests which are not applicable to or cannot be performed on the
-running mongo cluster are skipped.
+# Running Ruby Driver Tests
 
-Not only does the test suite require an externally launched cluster,
-the test suite must also be told how the cluster is configured
-via MONGODB_URI, TOPOLOGY, MONGODB_ADDRESSES, RS_ENABLED, RS_NAME and/or
-SHARDED_ENABLED environment variables.
+## Quick Start
 
-The test suite attempts to provide diagnostics when it is not able to
-connect to the cluster it is configured to use.
+To run the test suite against a local MongoDB deployment listening on port
+27017, run:
 
-Additionally some of the tests assume that the seed list (given in
-MONGODB_URI or MONGODB_ADDRESSES) encompasses all servers in the cluster,
-and will fail when MONGODB_URI includes only one host of a replica set.
-It is best to include all hosts of the cluster in MONGODB_URI and
-MONGODB_ADDRESSES.
+    rake
 
-It is best to have the test suite configured to connect to exactly
-the hostnames configured in the cluster. If, for example, the test suite
-is configured to use IP addresses but the cluster is configured with
-hostnames, the tests should still work (by using SDAM to discover correct
-cluster configuration) but will spend a significant amount of extra time
-on server discovery.
+When run without options, the test suite will automatically detect deployment
+topology and configure itself appropriately. Standalone, replica set and
+sharded cluster topologies are supported (though the test suite will presently
+use only the first listed shard in a sharded cluster if given a seed list,
+or the one running on port 27017 if not given a seed list).
 
-In order to run spec tests, the mongo cluster needs to have fail points
-enabled. This is accomplished by starting mongod with the following option:
-  --setParameter enableTestCommands=1
+TLS, authentication and other options can be configured via URI options by
+setting `MONGODB_URI` environment variable appropriately. Examples of such
+configuration are given later in this document.
 
-Use the following environment variables to configure the tests:
+## MongoDB Server Deployment
 
-CLIENT_DEBUG: Show debug messages from the client.
+The tests require a running MongoDB deployment, configured and started
+externally to the test suite. Tests that are not appropriate for the running
+deployment will be skipped.
 
-    CLIENT_DEBUG=1
+## Starting MongoDB Deployment
 
-MONGODB_URI: Connection string to use. This must be a valid MongoDB URI;
-mongodb:// and mongodb+srv:// are both supported.
-RS_ENABLED and SHARDED_ENABLED are NOT honored if using MONGODB_URI -
-specify replica set name in the URI and to specify a sharded topology
-set TOPOLOGY=sharded_cluster environment variable.
+There are many ways in which MongoDB can be started. The instructions below
+are for manually launching `mongod` instances and using
+[mlaunch](http://blog.rueckstiess.com/mtools/mlaunch.html)
+(part of [mtools](https://github.com/rueckstiess/mtools)) for more complex
+deployments, but other tools like
+[mongodb-runner](https://github.com/mongodb-js/runner) and
+[Mongo Orchestration](https://github.com/10gen/mongo-orchestration) can
+in principle also work.
 
-    MONGODB_URI=mongodb://127.0.0.1:27001/?replicaSet=test
-    MONGODB_URI=mongodb://127.0.0.1:27001,127.0.0.1:27002/ TOPOLOGY=sharded_cluster
+### Standalone
 
-MONGODB_ADDRESSES: Specify addresses to connect to. Use RS_ENABLED,
-RS_NAME and SHARDED_ENABLED to configure the topology.
+The simplest possible deployment is a standalone `mongod`, which can be
+launched as follows:
 
-    MONGODB_ADDRESSES=127.0.0.1:27017,127.0.0.1:27018
-    MONGODB_ADDRESSES=127.0.0.1:27017,127.0.0.1:27018 RS_ENABLED=1
-    MONGODB_ADDRESSES=127.0.0.1:27017,127.0.0.1:27018 RS_ENABLED=1 RS_NAME=test
-    MONGODB_ADDRESSES=127.0.0.1:27017,127.0.0.1:27018 SHARDED_ENABLED=1
+    # Launch mongod in one terminal
+    mkdir /tmp/mdb
+    mongod --dbpath /tmp/mdb
+    
+    # Run tests in another terminal
+    rake
 
-RS_ENABLED: Instruct the test suite to connect to a replica set.
-RS_ENABLED is only honored when not using MONGODB_URI; to connect to a
-replica set with MONGODB_URI, specify the replica set name in the URI
-(despite the Ruby driver performing topology discovery by default, it
-doesn't do so in the test suite).
-RS_NAME can be given to specify the replica set name; the default is
-ruby-driver-rs.
+A standalone deployment is a good starting point, however a great many tests
+require a replica set deployment and will be skipped on a standalone deployment.
 
-    RS_ENABLED=1
-    RS_ENABLED=1 RS_NAME=test
+### Replica Set
 
-SHARDED_ENABLED: Instruct the test suite to connect to the sharded cluster.
-Set MONGODB_URI appropriately as well.
+While a replica set can be started and configured by hand, doing so is
+cumbersome. The examples below use
+[mlaunch](http://blog.rueckstiess.com/mtools/mlaunch.html)
+to start a replica set.
 
-    SHARDED_ENABLED=1
+First, install [mtools](https://github.com/rueckstiess/mtools):
 
-SSL_ENABLED: Instruct the test suite to connect to the cluster via SSL.
+    pip install mtools --user
+    export PATH=~/.local/bin:$PATH
+    
+Then, launch a replica set:
 
-    SSL_ENABLED=1
-    # Also acceptable:
-    SSL=ssl
+    mlaunch init --replicaset --name ruby-driver-rs \
+      --dir /tmp/mdb-rs --setParameter enableTestCommands=1
 
-Note: SSL can also be enabled by giving ssl=true in the MONGODB_URI options.
+The test suite willl automatically detect the topology, no explicit
+configuration is needed:
 
-EXTERNAL_DISABLED: Run the tests without making any external connections
-(for example, external connections are required to test DNS seedlists and SRV
-URIs).
+    rake
+
+### Sharded Cluster
+
+A sharded cluster can be configured with mlaunch:
+
+    mlaunch init --replicaset --name ruby-driver-rs --sharded 1 \
+      --dir /tmp/mdb-sc --setParameter enableTestCommands=1
+
+As with the replica set, the test suite will automatically detect sharded
+cluster topology.
+
+## TLS With Verification
+
+The test suite includes a set of TLS certificates for configuring a server
+and a client to perform full TLS verification. The server can be started as
+follows, if the current directory is the top of the driver source tree:
+
+    mlaunch init --single --dir /tmp/mdb-ssl --sslMode requireSSL \
+      --sslPEMKeyFile `pwd`/spec/support/certificates/server.pem \
+      --sslCAFile `pwd`/spec/support/certificates/ca.pem \
+      --sslClientCertificate `pwd`/spec/support/certificates/client.pem
+
+## TLS Without Verification
+
+It is also possible to enable TLS but omit certificate verification. In this
+case a standalone server can be started as follows:
+
+    mlaunch init --single --dir /tmp/mdb-ssl --sslMode requireSSL \
+      --sslPEMKeyFile `pwd`/spec/support/certificates/server.pem \
+      --sslCAFile `pwd`/spec/support/certificates/ca.pem \
+      --sslAllowConnectionsWithoutCertificates \
+      --sslAllowInvalidCertificates
+
+To run the test suite against such a server, also omitting certificate
+verification, run:
+
+    MONGODB_URI='mongodb://localhost:27017/?tls=true&tlsInsecure=true' rake 
+
+## Authentication
+
+mlaunch can configure authentication on the server:
+
+    mlaunch init --single --dir /tmp/mdb-auth --auth --username dev --password dev
+
+To run the test suite against such a server, run:
+
+    MONGODB_URI='mongodb://dev:dev@localhost:27017/' rake 
+
+## Compression
+
+To be written.
+
+## Other Options
+
+Generally, all URI options recognized by the driver may be set for a test run,
+and will cause the clients created by the test suite to have those options
+by default. For example, retryable writes may be turned on and off as follows:
+
+    MONGODB_URI='mongodb://localhost:27017/?retryWrites=true' rake 
+
+    MONGODB_URI='mongodb://localhost:27017/?retryWrites=false' rake 
+
+Individual tests may override options that the test suite uses as defaults.
+For example, retryable writes tests may create clients with the retry writes
+option set to true or false as needed regardless of what the default is for
+the entire test run.
+
+It is also possible to, for example, reference non-default hosts and replica
+set names:
+
+    MONGODB_URI='mongodb://test.host:27017,test.host:27018/?replicaSet=fooset' rake 
+
+However, as noted in the caveats section, changing the database name used by
+the test suite is not supported.
+
+Some tests require internet connectivity, for example to test DNS seed lists
+and SRV URIs. These tests can be skipped by setting the following environment
+variable:
 
     EXTERNAL_DISABLED=true
+
+## Caveats
+
+### Socket Permission Errors
+
+If you get permission errors connecting to `mongod`'s socket, adjust its
+permissions:
+
+    sudo chmod 0666 /tmp/mongodb-27017.sock
+
+### Non-Identical Hostnames
+
+The test suite should be configured to connect to exactly the hostnames
+configured in the cluster. If, for example, the test suite is configured
+to use IP addresses but the cluster is configured with hostnames, most tests
+would still work (by using SDAM to discover correct cluster configuration)
+but will spend a significant amount of extra time on server discovery.
+
+Some tests perform address assertions and will fail if hostnames configured
+in the test suite do not match hostnames configured in the cluster.
+For the same reason, each node in server configuration should have its port
+specified.
+
+### Database Name
+
+The test suite currently does not allow changing the database name that it
+uses, which is `ruby-driver`. Attempts to specify a different database name
+in the URI for example will lead to some of the tests failing.
+
+### Fail Points
+
+In order to run some of the tests, the mongo cluster needs to have fail points
+enabled. This is accomplished by starting `mongod` with the following option:
+
+    --setParameter enableTestCommands=1
+
+### Log Output
+
+The test suite is run with the driver log level set to WARN by default.
+This produces a fair amount of output as many tests trigger various conditions
+resulting in the driver outputting warnings. This is expected behavior.
+
+## Running Individual Examples
+
+Individual examples can be run by invoking `rspec` instead of `rake`. Prior
+to running `rspec`, ensure the test suite created users for itself - this
+is done by the `rake` command automatically, or you can manually invoke the
+Rake task which configures the deployment for testing:
+
+    rake spec:prepare
+
+Then, any of the standard RSpec invocations will work:
+
+    rspec path/to/file_spec.rb
+
+## Configuration Reporting
+
+To have the test suite report its current configuration, run:
+
+    rake spec:config

data/spec/integration/change_stream_spec.rb

@@ -366,4 +366,68 @@ describe 'Change stream integration', retry: 4 do
       end.to raise_error(ArgumentError, 'Time must be a Time or a BSON::Timestamp instance')
     end
   end
+
+  describe ':start_after option' do
+    require_topology :replica_set
+    min_server_version '4.1'
+
+    let(:start_after) do
+      stream = authorized_collection.watch([])
+      authorized_collection.insert_one(x: 1)
+      start_after = stream.to_enum.next['_id']
+    end
+
+    let(:stream) do
+      authorized_collection.watch([], { start_after: start_after })
+    end
+
+    let(:events) do
+      start_after
+
+      subscriber = EventSubscriber.new
+      authorized_client.subscribe(Mongo::Monitoring::COMMAND, subscriber)
+      use_stream
+
+      subscriber.started_events.select { |e| e.command_name == 'aggregate' }
+    end
+
+    context 'when an initial aggregation is run' do
+      let(:use_stream) do
+        stream
+      end
+
+      it 'sends startAfter' do
+        expect(events.size >= 1).to eq(true)
+
+        command = events.first.command
+        expect(command['pipeline'].size == 1).to eq(true)
+        expect(command['pipeline'].first.key?('$changeStream')).to eq(true)
+        expect(command['pipeline'].first['$changeStream'].key?('startAfter')).to eq(true)
+      end
+    end
+
+    context 'when resuming' do
+      let(:use_stream) do
+        stream
+
+        authorized_collection.insert_one(x: 1)
+        stream.to_enum.next
+
+        authorized_collection.insert_one(x: 1)
+        authorized_collection.client.use(:admin).command(fail_point_base_command.merge(
+          :mode => {:times => 1},
+          :data => {:failCommands => ['getMore'], errorCode: 100}))
+        stream.to_enum.next
+      end
+
+      it 'does not startAfter even when passed in' do
+        expect(events.size == 2).to eq(true)
+
+        command = events.last.command
+        expect(command['pipeline'].size == 1).to eq(true)
+        expect(command['pipeline'].first.key?('$changeStream')).to eq(true)
+        expect(command['pipeline'].first['$changeStream'].key?('startAfter')).to eq(false)
+      end
+    end
+  end
 end

data/spec/integration/command_spec.rb

@@ -62,12 +62,6 @@ describe 'Command' do
         min_server_fcv '3.6'
 
         let(:session) { authorized_client.start_session }
-        # w will be 2 for a RS topology with 3 nodes
-        let(:w) do
-          authorized_client.write_concern.options[:w].tap do |w|
-            expect(w).to be_a(Integer)
-          end
-        end
 
         let(:expected_payload) do
           {
@@ -75,7 +69,6 @@ describe 'Command' do
               'commitTransaction' => 1,
               'lsid' => session.session_id,
               'txnNumber' => BSON::Int64.new(123),
-              'writeConcern' => {'w' => w},
             },
             'command_name' => 'commitTransaction',
             'database_name' => 'admin',

data/spec/integration/error_detection_spec.rb

@@ -0,0 +1,39 @@
+require 'spec_helper'
+
+describe 'Error detection' do
+  context 'document contains a not master/node recovering code' do
+    let(:document) { {code: 91} }
+
+    let(:coll) { authorized_client_without_any_retries['error-detection'] }
+
+    before do
+      coll.delete_many
+    end
+
+    context 'cursors not used' do
+
+      before do
+        coll.insert_one(document)
+      end
+
+      it 'is not treated as an error when retrieved' do
+        actual = coll.find.first
+        expect(actual['code']).to eq(91)
+      end
+    end
+
+    context 'cursors used' do
+
+      before do
+        10.times do
+          coll.insert_one(document)
+        end
+      end
+
+      it 'is not treated as an error when retrieved' do
+        actual = coll.find({}, batch_size: 2).first
+        expect(actual['code']).to eq(91)
+      end
+    end
+  end
+end

data/spec/integration/read_concern.rb

@@ -0,0 +1,83 @@
+require 'spec_helper'
+
+describe 'read concern' do
+  let(:subscriber) do
+    EventSubscriber.new
+  end
+
+  let(:specified_read_concern) do
+    { 'level' => 'local' }
+  end
+
+  let(:sent_read_concern) do
+    subscriber.clear_events!
+    collection.count_documents({})
+    subscriber.started_events.find { |c| c.command_name == 'aggregate' }.command[:readConcern]
+  end
+
+  shared_examples_for 'a read concern is specified' do
+    it 'sends a read concern to the server' do
+      expect(sent_read_concern).to eq(specified_read_concern)
+    end
+  end
+
+  shared_examples_for 'no read concern is specified' do
+    it 'does not send a read concern to the server' do
+      expect(sent_read_concern).to be_nil
+    end
+  end
+
+  context 'when the client has no read concern specified' do
+
+    let(:client) do
+      authorized_client.tap do |client|
+        client.subscribe(Mongo::Monitoring::COMMAND, subscriber)
+      end
+    end
+
+    context 'when the collection has no read concern specified' do
+
+      let(:collection) do
+        client[TEST_COLL]
+      end
+
+      it_behaves_like 'no read concern is specified'
+    end
+
+    context 'when the collection has a read concern specified' do
+
+      let(:collection) do
+        client[TEST_COLL].with(read_concern: specified_read_concern)
+      end
+
+      it_behaves_like 'a read concern is specified'
+    end
+  end
+
+  context 'when the client has a read concern specified' do
+
+    let(:client) do
+      authorized_client.with(read_concern: specified_read_concern).tap do |client|
+        client.subscribe(Mongo::Monitoring::COMMAND, subscriber)
+      end
+    end
+
+    context 'when the collection has no read concern specified' do
+
+      let(:collection) do
+        client[TEST_COLL]
+      end
+
+      it_behaves_like 'a read concern is specified'
+    end
+
+    context 'when the collection has a read concern specified' do
+
+      let(:collection) do
+        client[TEST_COLL].with(read_concern: specified_read_concern)
+      end
+
+      it_behaves_like 'a read concern is specified'
+    end
+  end
+end