mongo 2.10.5 → 2.15.1

data/spec/USERS.md

@@ -0,0 +1,72 @@
+# Test Users
+
+The Mongo Ruby Driver tests assume the presence of two `Mongo::Auth::User` objects:
+`root_user` and `test_user`. This document details the roles and privileges granted
+to those users as well as how they are created and used in the tests.
+
+Both users are defined in the [spec_config](support/spec_config.rb#L376) file.
+
+## root_user
+`root_user` is the test user with the most privileges. It is created with the following roles:
+- userAdminAnyDatabase
+- dbAdminAnyDatabase
+- readWriteAnyDatabase
+- clusterAdmin
+
+By default, `root_user` is given a username of `root-user` and a password of `password`.
+However, you may override these defaults by specifying a username and password in the
+`MONGODB_URI` environment variable while running your tests. For example, if you set `MONGODB_URI` to: `mongodb://alanturing:enigma@localhost:27017/`, the username of `root_user` would be set to `alanturing`, and the password would be set to `enigma`.
+
+## test_user
+`test_user` is the user created with a more limited set of privileges. It is created with the following
+roles:
+- readWrite on the ruby-driver database
+- dbAdmin on the ruby-driver database
+
+It is also granted the following roles against a database called "invalid_database." These permissions are used for the purpose of running tests against a database that doesn't exist.
+- readWrite on the invalid_database database
+- dbAdmin on the invalid_database database
+
+`test_user` also has the following roles, which are exclusively used to test transactions:
+- readWrite on the hr database
+- dbAdmin on the hr database
+- readWrite on the reporting database
+- dbAdmin on the reporting database
+
+The `test_user` has the username `test-user` and the password `password`; these values are not customizable without changing the source code.
+
+## User Creation
+
+Both users are typically created in the [spec_setup](support/spec_setup.rb) script, which can be
+run in two ways: either by running `bundle exec rake spec:prepare`, which only runs spec setup without
+running any actual tests, or by running `rake`, which runs spec setup and the entire test suite.
+
+First, the `spec_setup` script attempts to create the `root_user`. If this user already exists (for example,
+if you have already created this user in your test instance), `spec_setup` will skip this step. Once
+the script has verified the existence of `root_user`, it will create a client authenticated with the `root_user` and use that client to create a second user, `test_user`. Because `root_user` has the `userAdminAnyDatabase` role, it has the permissions necessary to create and destroy users on your MongoDB instance. If you have already created a user with the same credentials as `test_user` prior to running
+the `spec_setup` script, the script will delete this user and re-create it.
+
+The `root_user` is created in the `admin` database, while the `test_user` is created in the `ruby-driver`
+database.
+
+The authentication mechanism used to store the user credentials is going to change depending on the version of MongoDB running on your deployment. If you are running tests against a MongoDB instance with a server version older than 3.0, the users will be created using the `MONGODB-CR` authentication mechanism. If your server version is between 3.0 and 3.6 (inclusive), the test users will be created using the `SCRAM-SHA-1` mechanism, which was introduced as the new default starting in MongoDB version 3.0. If you are running a version of MongoDB newer than 4.0, test users will be authenticated using either `SCRAM-SHA-1` or `SCRAM-SHA-256`.
+
+**Note:** (m-launch)[http://blog.rueckstiess.com/mtools/mlaunch.html], the client tool we use to spin up MongoDB instances for our tests, creates users EXCLUSIVELY with the `SCRAM-SHA-1` mechanism, even when `SCRAM-SHA-256` is enabled on the test server. This should not impact your ability to run the Mongo Ruby Driver test suite.
+
+## Test Usage
+
+`root_user` is used in the Mongo Ruby Driver tests to perform functionality that requires its high-level
+roles and privileges (if your client is set up with authentication), such as creating and destroying users and database administration. To easily set up a `Mongo::Client` object authenticated with the roles and privileges of `root_user`, you can initialize a client using the `ClientRegistry` module as follows:
+
+```
+client = ClientRegistry.instance.global_client('root_authorized')
+```
+
+Of course, not every test will require you to create a client with so many privileges. Often, it is enough
+to have a user who is only authorized to read and write to a specific test database. In this case, it is preferable to use `test_user`. To initialize a `Mongo::Client` object authenticated with the `test_user` object, use the `ClientRegistry` module as follows:
+
+```
+client = ClientRegistry.instance.global_client('authorized')
+```
+
+Once you have initialized these client objects, you may use them to perform functionality required by your tests.

data/spec/atlas/atlas_connectivity_spec.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
 require 'lite_spec_helper'
 
 describe 'Atlas connectivity' do

data/spec/atlas/operations_spec.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
 require 'lite_spec_helper'
 
 describe 'Operations' do

data/spec/integration/auth_spec.rb

@@ -1,13 +1,24 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
 require 'spec_helper'
 
 describe 'Auth' do
+  # User creation with a password fails on the server if, for example,
+  # only MONGODB-AWS auth mechanism is allowed in server configuration.
+  require_no_external_user
+
   describe 'Unauthorized exception message' do
     let(:server) do
       authorized_client.cluster.next_primary
     end
 
+    let(:base_options) do
+      SpecConfig.instance.monitoring_options
+    end
+
     let(:connection) do
-      Mongo::Server::Connection.new(server, options)
+      Mongo::Server::Connection.new(server, base_options.merge(options))
     end
 
     before(:all) do
@@ -27,8 +38,9 @@ describe 'Auth' do
     context 'user mechanism not provided' do
 
       context 'user does not exist' do
-        let(:options) { SpecConfig.instance.ssl_options.merge(
-          user: 'nonexistent_user') }
+        let(:options) do
+          {user: 'nonexistent_user' }
+        end
 
         before do
           expect(connection.app_metadata.send(:document)[:saslSupportedMechs]).to eq('admin.nonexistent_user')
@@ -41,7 +53,7 @@ describe 'Auth' do
           it 'indicates scram-sha-1 was used' do
             expect do
               connection.connect!
-            end.to raise_error(Mongo::Auth::Unauthorized, 'User nonexistent_user (mechanism: scram) is not authorized to access admin (used mechanism: SCRAM-SHA-1)')
+            end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
           end
         end
 
@@ -53,14 +65,15 @@ describe 'Auth' do
           it 'indicates scram-sha-1 was used' do
             expect do
               connection.connect!
-            end.to raise_error(Mongo::Auth::Unauthorized, 'User nonexistent_user (mechanism: scram) is not authorized to access admin (used mechanism: SCRAM-SHA-1)')
+            end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
           end
         end
       end
 
       context 'user exists' do
-        let(:options) { SpecConfig.instance.ssl_options.merge(
-          user: 'existing_user', password: 'bogus') }
+        let(:options) do
+          {user: 'existing_user', password: 'bogus'}
+        end
 
         before do
           expect(connection.app_metadata.send(:document)[:saslSupportedMechs]).to eq("admin.existing_user")
@@ -73,7 +86,7 @@ describe 'Auth' do
           it 'indicates scram-sha-1 was used' do
             expect do
               connection.connect!
-            end.to raise_error(Mongo::Auth::Unauthorized, "User existing_user (mechanism: scram) is not authorized to access admin (used mechanism: SCRAM-SHA-1)")
+            end.to raise_error(Mongo::Auth::Unauthorized, /User existing_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
           end
         end
 
@@ -85,7 +98,7 @@ describe 'Auth' do
           it 'indicates scram-sha-256 was used' do
             expect do
               connection.connect!
-            end.to raise_error(Mongo::Auth::Unauthorized, "User existing_user (mechanism: scram256) is not authorized to access admin (used mechanism: SCRAM-SHA-256)")
+            end.to raise_error(Mongo::Auth::Unauthorized, /User existing_user \(mechanism: scram256\) is not authorized to access admin.*used mechanism: SCRAM-SHA-256/)
           end
         end
       end
@@ -95,32 +108,69 @@ describe 'Auth' do
       min_server_fcv '3.0'
 
       context 'scram-sha-1 requested' do
-        let(:options) { SpecConfig.instance.ssl_options.merge(
-          user: 'nonexistent_user', auth_mech: :scram) }
+        let(:options) do
+          {user: 'nonexistent_user', auth_mech: :scram}
+        end
 
         it 'indicates scram-sha-1 was requested and used' do
           expect do
             connection.connect!
-          end.to raise_error(Mongo::Auth::Unauthorized, 'User nonexistent_user (mechanism: scram) is not authorized to access admin (used mechanism: SCRAM-SHA-1)')
+          end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
         end
       end
 
       context 'scram-sha-256 requested' do
         min_server_fcv '4.0'
 
-        let(:options) { SpecConfig.instance.ssl_options.merge(
-          user: 'nonexistent_user', auth_mech: :scram256) }
+        let(:options) do
+          {user: 'nonexistent_user', auth_mech: :scram256}
+        end
 
         it 'indicates scram-sha-256 was requested and used' do
           expect do
             connection.connect!
-          end.to raise_error(Mongo::Auth::Unauthorized, 'User nonexistent_user (mechanism: scram256) is not authorized to access admin (used mechanism: SCRAM-SHA-256)')
+          end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram256\) is not authorized to access admin.*used mechanism: SCRAM-SHA-256/)
+        end
+      end
+    end
+
+    context 'when authentication fails' do
+      let(:options) do
+        {user: 'nonexistent_user', password: 'foo'}
+      end
+
+      it 'reports which server authentication was attempted against' do
+        expect do
+          connection.connect!
+        end.to raise_error(Mongo::Auth::Unauthorized, /used server: #{connection.address.to_s}/)
+      end
+
+      context 'with default auth source' do
+        it 'reports auth source used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /auth source: admin/)
+        end
+      end
+
+      context 'with custom auth source' do
+        let(:options) do
+          {user: 'nonexistent_user', password: 'foo', auth_source: 'authdb'}
+        end
+
+        it 'reports auth source used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /auth source: authdb/)
         end
       end
     end
 
     context 'attempting to connect to a non-tls server with tls' do
       require_no_tls
+      # The exception raised is SocketTimeout on 3.6 server for whatever reason,
+      # run the test on 4.0+ only.
+      min_server_fcv '4.0'
 
       let(:options) { {ssl: true} }
 
@@ -140,7 +190,7 @@ describe 'Auth' do
     context 'attempting to connect to a tls server without tls' do
       require_tls
 
-      let(:options) { {} }
+      let(:options) { {ssl: false} }
 
       it 'reports host, port and tls status' do
         begin
@@ -157,4 +207,73 @@ describe 'Auth' do
       end
     end
   end
+
+  shared_examples_for 'caches client key' do
+    it 'caches' do
+      client.close
+      Mongo::Auth::CredentialCache.clear
+
+      RSpec::Mocks.with_temporary_scope do
+        expect_any_instance_of(conversation_class).to receive(:hi).exactly(:once).and_call_original
+
+        client.reconnect
+        server = client.cluster.next_primary
+        server.with_connection do
+          server.with_connection do
+            # nothing
+          end
+        end
+      end
+    end
+  end
+
+  describe 'scram-sha-1 client key caching' do
+    clean_slate
+    min_server_version '3.0'
+    require_no_external_user
+
+    let(:client) { authorized_client.with(max_pool_size: 2, auth_mech: :scram) }
+    let(:conversation_class) { Mongo::Auth::Scram::Conversation }
+
+    it_behaves_like 'caches client key'
+  end
+
+  describe 'scram-sha-256 client key caching' do
+    clean_slate
+    min_server_version '4.0'
+    require_no_external_user
+
+    let(:client) { authorized_client.with(max_pool_size: 2, auth_mech: :scram256) }
+    let(:conversation_class) { Mongo::Auth::Scram256::Conversation }
+
+    it_behaves_like 'caches client key'
+  end
+
+  context 'when only auth source is specified' do
+    require_no_auth
+
+    let(:client) do
+      new_local_client(SpecConfig.instance.addresses, SpecConfig.instance.monitoring_options.merge(
+        auth_source: 'foo'))
+    end
+
+    it 'does not authenticate' do
+      expect(Mongo::Auth::User).not_to receive(:new)
+      client.database.command(ping: 1)
+    end
+  end
+
+  context 'when only auth mechanism is specified' do
+    require_x509_auth
+
+    let(:client) do
+      new_local_client(SpecConfig.instance.addresses, base_options.merge(
+        auth_mech: :mongodb_x509))
+    end
+
+    it 'authenticates' do
+      expect(Mongo::Auth::User).to receive(:new).and_call_original
+      client.database.command(ping: 1)
+    end
+  end
 end

data/spec/integration/awaited_ismaster_spec.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
+require 'spec_helper'
+
+describe 'awaited hello' do
+  min_server_fcv '4.4'
+
+  # If we send the consecutive hello commands to different mongoses,
+  # they have different process ids, and so the awaited one would return
+  # immediately.
+  require_no_multi_shard
+
+  let(:client) { authorized_client }
+
+  it 'waits' do
+    # Perform a regular hello to get topology version
+    resp = client.database.command(hello: 1)
+    doc = resp.replies.first.documents.first
+    tv = Mongo::TopologyVersion.new(doc['topologyVersion'])
+    tv.should be_a(BSON::Document)
+
+    elapsed_time = Benchmark.realtime do
+      resp = client.database.command(hello: 1,
+        topologyVersion: tv.to_doc, maxAwaitTimeMS: 500)
+    end
+    doc = resp.replies.first.documents.first
+
+    elapsed_time.should > 0.5
+  end
+end

data/spec/integration/aws_auth_request_spec.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
+require 'lite_spec_helper'
+require 'net/http'
+
+describe Mongo::Auth::Aws::Request do
+  require_aws_auth
+
+  before(:all) do
+    if ENV['AUTH'] =~ /aws-(ec2|ecs)/
+      skip "This test requires explicit credentials to be provided"
+    end
+  end
+
+  let(:access_key_id) { ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID') }
+  let(:secret_access_key) { ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY') }
+  let(:session_token) { ENV['MONGO_RUBY_DRIVER_AWS_AUTH_SESSION_TOKEN'] }
+
+  describe '#authorization' do
+    let(:request) do
+      described_class.new(
+        access_key_id: access_key_id,
+        secret_access_key: secret_access_key,
+        session_token: session_token,
+        host: 'sts.amazonaws.com',
+        server_nonce: 'aaaaaaaaaaafake',
+      )
+    end
+
+    let(:sts_request) do
+      Net::HTTP::Post.new("https://sts.amazonaws.com").tap do |req|
+        request.headers.each do |k, v|
+          req[k] = v
+        end
+        req['authorization'] = request.authorization
+        req['accept'] = 'application/json'
+        req.body = described_class::STS_REQUEST_BODY
+      end
+    end
+
+    let(:sts_response) do
+      http = Net::HTTP.new('sts.amazonaws.com', 443)
+      http.use_ssl = true
+
+      # Uncomment to log complete request headers and the response.
+      # WARNING: do not enable this in Evergreen as this can expose real
+      # AWS credentias.
+      #http.set_debug_output(STDERR)
+
+      http.start do
+        resp = http.request(sts_request)
+      end
+    end
+
+    let(:sts_response_payload) do
+      JSON.parse(sts_response.body)
+    end
+
+    let(:result) do
+      sts_response_payload['GetCallerIdentityResponse']['GetCallerIdentityResult']
+    end
+
+    it 'is usable' do
+      # This assertion intentionally does not use payload so that if it fails,
+      # the entire response is printed for diagnostic purposes.
+      sts_response.body.should_not =~ /"Error"/
+
+      sts_response.code.should == '200'
+      result['Arn'].should =~ /^arn:aws:(iam|sts)::/
+      result['Account'].should be_a(String)
+      result['UserId'].should =~ /^A/
+
+      puts "STS request successful with ARN #{result['Arn']}"
+    end
+  end
+end

data/spec/integration/aws_credentials_retriever_spec.rb

@@ -0,0 +1,106 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
+require 'lite_spec_helper'
+require 'support/aws_utils'
+
+describe Mongo::Auth::Aws::CredentialsRetriever do
+  require_aws_auth
+
+  let(:retriever) do
+    described_class.new(user)
+  end
+
+  let(:credentials) do
+    retriever.credentials
+  end
+
+  context 'when user is not given' do
+    let(:user) do
+      Mongo::Auth::User.new(auth_mech: :aws)
+    end
+
+    shared_examples_for 'retrieves the credentials' do
+      it 'retrieves' do
+        credentials.should be_a(Mongo::Auth::Aws::Credentials)
+
+        # When user is not given, credentials retrieved are always temporary.
+        retriever.credentials.access_key_id.should =~ /^ASIA/
+        retriever.credentials.secret_access_key.should =~ /./
+        retriever.credentials.session_token.should =~ /./
+      end
+
+      let(:request) do
+        Mongo::Auth::Aws::Request.new(
+          access_key_id: credentials.access_key_id,
+          secret_access_key: credentials.secret_access_key,
+          session_token: credentials.session_token,
+          host: 'sts.amazonaws.com',
+          server_nonce: 'test',
+        )
+      end
+
+      it 'produces valid credentials' do
+        result = request.validate!
+        puts "STS request successful with ARN #{result['Arn']}"
+      end
+    end
+
+    context 'ec2 instance role' do
+      require_ec2_host
+
+      before(:all) do
+        unless ENV['AUTH'] == 'aws-ec2'
+          skip "Set AUTH=aws-ec2 in environment to run EC2 instance role tests"
+        end
+      end
+
+      context 'when instance profile is not assigned' do
+        before(:all) do
+          orchestrator = AwsUtils::Orchestrator.new(
+            region: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_REGION'),
+            access_key_id: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID'),
+            secret_access_key: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY'),
+          )
+
+          orchestrator.clear_instance_profile(Utils.ec2_instance_id)
+          Utils.wait_for_no_instance_profile
+        end
+
+        it 'raises an error' do
+          lambda do
+            credentials
+          end.should raise_error(Mongo::Auth::InvalidConfiguration, /Could not locate AWS credentials/)
+        end
+      end
+
+      context 'when instance profile is assigned' do
+        before(:all) do
+          orchestrator = AwsUtils::Orchestrator.new(
+            region: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_REGION'),
+            access_key_id: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID'),
+            secret_access_key: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY'),
+          )
+
+          orchestrator.set_instance_profile(Utils.ec2_instance_id,
+            instance_profile_name: nil,
+            instance_profile_arn: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_INSTANCE_PROFILE_ARN'),
+          )
+          Utils.wait_for_instance_profile
+        end
+
+        it_behaves_like 'retrieves the credentials'
+      end
+    end
+
+    context 'ecs task role' do
+      before(:all) do
+        unless ENV['AUTH'] == 'aws-ecs'
+          skip "Set AUTH=aws-ecs in environment to run ECS task role tests"
+        end
+      end
+
+      it_behaves_like 'retrieves the credentials'
+    end
+  end
+end

data/spec/integration/bson_symbol_spec.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
 require 'spec_helper'
 
 describe 'Symbol encoding to BSON' do
@@ -12,7 +15,7 @@ describe 'Symbol encoding to BSON' do
   end
 
   let(:expected) do
-    "\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00foo\x00\x00".force_encoding('binary')
+    (+"\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00foo\x00\x00").force_encoding('binary')
   end
 
   it 'encodes symbol to BSON symbol' do
@@ -25,8 +28,10 @@ describe 'Symbol encoding to BSON' do
   end
 
   it 'round-trips symbol values using the same byte buffer' do
-    if BSON::Environment.jruby?
-      pending 'https://jira.mongodb.org/browse/RUBY-2128'
+    if BSON::Environment.jruby? && (BSON::VERSION.split('.').map(&:to_i) <=> [4, 11, 0]) < 0
+      skip 'This test is only relevant to bson versions that increment ByteBuffer '\
+       'read and write positions separately in JRuby, as implemented in ' \
+       'bson version 4.11.0. For more information, see https://jira.mongodb.org/browse/RUBY-2128'
     end
 
     Hash.from_bson(hash.to_bson).should == hash

data/spec/integration/bulk_insert_spec.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+# encoding: utf-8
+
 require 'spec_helper'
 
 describe 'Bulk insert' do

data/spec/integration/bulk_write_error_message_spec.rb

@@ -0,0 +1,41 @@
+require 'spec_helper'
+
+describe 'BulkWriteError message' do
+  let(:client) { authorized_client }
+  let(:collection_name) { 'bulk_write_error_message_spec' }
+  let(:collection) { client[collection_name] }
+
+  before do
+    collection.delete_many
+  end
+
+  context 'a bulk write with one error' do
+    it 'reports code name, code and message' do
+      begin
+        collection.insert_many([
+          {_id: 1},
+          {_id: 1},
+          {_id: 1},
+        ], ordered: true)
+        fail('Should have raised')
+      rescue Mongo::Error::BulkWriteError => e
+        e.message.should =~ %r,\A\[11000\]: (insertDocument :: caused by :: 11000 )?E11000 duplicate key error (collection|index):,
+      end
+    end
+  end
+
+  context 'a bulk write with multiple errors' do
+    it 'reports code name, code and message' do
+      begin
+        collection.insert_many([
+          {_id: 1},
+          {_id: 1},
+          {_id: 1},
+        ], ordered: false)
+        fail('Should have raised')
+      rescue Mongo::Error::BulkWriteError => e
+        e.message.should =~ %r,\AMultiple errors: \[11000\]: (insertDocument :: caused by :: 11000 )?E11000 duplicate key error (collection|index):.*\[11000\]: (insertDocument :: caused by :: 11000 )?E11000 duplicate key error (collection|index):,
+      end
+    end
+  end
+end