mongo 2.4.1 → 2.14.0

data/spec/USERS.md

@@ -0,0 +1,72 @@
+# Test Users
+
+The Mongo Ruby Driver tests assume the presence of two `Mongo::Auth::User` objects:
+`root_user` and `test_user`. This document details the roles and privileges granted
+to those users as well as how they are created and used in the tests.
+
+Both users are defined in the [spec_config](support/spec_config.rb#L376) file.
+
+## root_user
+`root_user` is the test user with the most privileges. It is created with the following roles:
+- userAdminAnyDatabase
+- dbAdminAnyDatabase
+- readWriteAnyDatabase
+- clusterAdmin
+
+By default, `root_user` is given a username of `root-user` and a password of `password`.
+However, you may override these defaults by specifying a username and password in the
+`MONGODB_URI` environment variable while running your tests. For example, if you set `MONGODB_URI` to: `mongodb://alanturing:enigma@localhost:27017/`, the username of `root_user` would be set to `alanturing`, and the password would be set to `enigma`.
+
+## test_user
+`test_user` is the user created with a more limited set of privileges. It is created with the following
+roles:
+- readWrite on the ruby-driver database
+- dbAdmin on the ruby-driver database
+
+It is also granted the following roles against a database called "invalid_database." These permissions are used for the purpose of running tests against a database that doesn't exist.
+- readWrite on the invalid_database database
+- dbAdmin on the invalid_database database
+
+`test_user` also has the following roles, which are exclusively used to test transactions:
+- readWrite on the hr database
+- dbAdmin on the hr database
+- readWrite on the reporting database
+- dbAdmin on the reporting database
+
+The `test_user` has the username `test-user` and the password `password`; these values are not customizable without changing the source code.
+
+## User Creation
+
+Both users are typically created in the [spec_setup](support/spec_setup.rb) script, which can be
+run in two ways: either by running `bundle exec rake spec:prepare`, which only runs spec setup without
+running any actual tests, or by running `rake`, which runs spec setup and the entire test suite.
+
+First, the `spec_setup` script attempts to create the `root_user`. If this user already exists (for example,
+if you have already created this user in your test instance), `spec_setup` will skip this step. Once
+the script has verified the existence of `root_user`, it will create a client authenticated with the `root_user` and use that client to create a second user, `test_user`. Because `root_user` has the `userAdminAnyDatabase` role, it has the permissions necessary to create and destroy users on your MongoDB instance. If you have already created a user with the same credentials as `test_user` prior to running
+the `spec_setup` script, the script will delete this user and re-create it.
+
+The `root_user` is created in the `admin` database, while the `test_user` is created in the `ruby-driver`
+database.
+
+The authentication mechanism used to store the user credentials is going to change depending on the version of MongoDB running on your deployment. If you are running tests against a MongoDB instance with a server version older than 3.0, the users will be created using the `MONGODB-CR` authentication mechanism. If your server version is between 3.0 and 3.6 (inclusive), the test users will be created using the `SCRAM-SHA-1` mechanism, which was introduced as the new default starting in MongoDB version 3.0. If you are running a version of MongoDB newer than 4.0, test users will be authenticated using either `SCRAM-SHA-1` or `SCRAM-SHA-256`.
+
+**Note:** (m-launch)[http://blog.rueckstiess.com/mtools/mlaunch.html], the client tool we use to spin up MongoDB instances for our tests, creates users EXCLUSIVELY with the `SCRAM-SHA-1` mechanism, even when `SCRAM-SHA-256` is enabled on the test server. This should not impact your ability to run the Mongo Ruby Driver test suite.
+
+## Test Usage
+
+`root_user` is used in the Mongo Ruby Driver tests to perform functionality that requires its high-level
+roles and privileges (if your client is set up with authentication), such as creating and destroying users and database administration. To easily set up a `Mongo::Client` object authenticated with the roles and privileges of `root_user`, you can initialize a client using the `ClientRegistry` module as follows:
+
+```
+client = ClientRegistry.instance.global_client('root_authorized')
+```
+
+Of course, not every test will require you to create a client with so many privileges. Often, it is enough
+to have a user who is only authorized to read and write to a specific test database. In this case, it is preferable to use `test_user`. To initialize a `Mongo::Client` object authenticated with the `test_user` object, use the `ClientRegistry` module as follows:
+
+```
+client = ClientRegistry.instance.global_client('authorized')
+```
+
+Once you have initialized these client objects, you may use them to perform functionality required by your tests.

data/spec/atlas/atlas_connectivity_spec.rb

@@ -0,0 +1,24 @@
+require 'lite_spec_helper'
+
+describe 'Atlas connectivity' do
+  let(:uri) { ENV['ATLAS_URI'] }
+  let(:client) { Mongo::Client.new(uri) }
+
+  before do
+    if uri.nil?
+      skip "ATLAS_URI not set in environment"
+    end
+  end
+
+  describe 'connection to Atlas' do
+    it 'runs ismaster successfully' do
+      result = client.database.command(:ismaster => 1)
+      expect(result.documents.first['ismaster']).to be true
+    end
+
+    it 'runs findOne successfully' do
+      result = client.use(:test)['test'].find.to_a
+      expect(result).to be_a(Array)
+    end
+  end
+end

data/spec/atlas/operations_spec.rb

@@ -0,0 +1,25 @@
+require 'lite_spec_helper'
+
+describe 'Operations' do
+  let(:uri) { ENV['ATLAS_URI'] }
+  let(:client) { Mongo::Client.new(uri) }
+
+  before do
+    if uri.nil?
+      skip "ATLAS_URI not set in environment"
+    end
+  end
+
+  describe 'list_collections' do
+    # Atlas free tier proxy enforces restrictions on list_collections
+    # arguments. This tests verifies that list_collections works on Atlas
+
+    it 'works' do
+      # We are not allowed to mutate the database, therefore the list of
+      # collections would generally be empty.
+      expect do
+        client.database.list_collections
+      end.not_to raise_error
+    end
+  end
+end

data/spec/integration/auth_spec.rb

@@ -0,0 +1,266 @@
+require 'spec_helper'
+
+describe 'Auth' do
+  # User creation with a password fails on the server if, for example,
+  # only MONGODB-AWS auth mechanism is allowed in server configuration.
+  require_no_external_user
+
+  describe 'Unauthorized exception message' do
+    let(:server) do
+      authorized_client.cluster.next_primary
+    end
+
+    let(:connection) do
+      Mongo::Server::Connection.new(server, options)
+    end
+
+    before(:all) do
+      # If auth is configured, the test suite uses the configured user
+      # and does not create its own users. However, the configured user may
+      # not have the auth mechanisms we need. Therefore we create a user
+      # for this test without specifying auth mechanisms, which gets us
+      # server default (scram for 4.0, scram & scram256 for 4.2).
+
+      users = ClientRegistry.instance.global_client('root_authorized').use(:admin).database.users
+      unless users.info('existing_user').empty?
+        users.remove('existing_user')
+      end
+      users.create('existing_user', password: 'password')
+    end
+
+    context 'user mechanism not provided' do
+
+      context 'user does not exist' do
+        let(:options) { SpecConfig.instance.ssl_options.merge(
+          user: 'nonexistent_user') }
+
+        before do
+          expect(connection.app_metadata.send(:document)[:saslSupportedMechs]).to eq('admin.nonexistent_user')
+        end
+
+        context 'scram-sha-1 only server' do
+          min_server_fcv '3.0'
+          max_server_version '3.6'
+
+          it 'indicates scram-sha-1 was used' do
+            expect do
+              connection.connect!
+            end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
+          end
+        end
+
+        context 'scram-sha-256 server' do
+          min_server_fcv '4.0'
+
+          # An existing user on 4.0+ will negotiate scram-sha-256.
+          # A non-existing user on 4.0+ will negotiate scram-sha-1.
+          it 'indicates scram-sha-1 was used' do
+            expect do
+              connection.connect!
+            end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
+          end
+        end
+      end
+
+      context 'user exists' do
+        let(:options) { SpecConfig.instance.ssl_options.merge(
+          user: 'existing_user', password: 'bogus') }
+
+        before do
+          expect(connection.app_metadata.send(:document)[:saslSupportedMechs]).to eq("admin.existing_user")
+        end
+
+        context 'scram-sha-1 only server' do
+          min_server_fcv '3.0'
+          max_server_version '3.6'
+
+          it 'indicates scram-sha-1 was used' do
+            expect do
+              connection.connect!
+            end.to raise_error(Mongo::Auth::Unauthorized, /User existing_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
+          end
+        end
+
+        context 'scram-sha-256 server' do
+          min_server_fcv '4.0'
+
+          # An existing user on 4.0+ will negotiate scram-sha-256.
+          # A non-existing user on 4.0+ will negotiate scram-sha-1.
+          it 'indicates scram-sha-256 was used' do
+            expect do
+              connection.connect!
+            end.to raise_error(Mongo::Auth::Unauthorized, /User existing_user \(mechanism: scram256\) is not authorized to access admin.*used mechanism: SCRAM-SHA-256/)
+          end
+        end
+      end
+    end
+
+    context 'user mechanism is provided' do
+      min_server_fcv '3.0'
+
+      context 'scram-sha-1 requested' do
+        let(:options) { SpecConfig.instance.ssl_options.merge(
+          user: 'nonexistent_user', auth_mech: :scram) }
+
+        it 'indicates scram-sha-1 was requested and used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram\) is not authorized to access admin.*used mechanism: SCRAM-SHA-1/)
+        end
+      end
+
+      context 'scram-sha-256 requested' do
+        min_server_fcv '4.0'
+
+        let(:options) { SpecConfig.instance.ssl_options.merge(
+          user: 'nonexistent_user', auth_mech: :scram256) }
+
+        it 'indicates scram-sha-256 was requested and used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /User nonexistent_user \(mechanism: scram256\) is not authorized to access admin.*used mechanism: SCRAM-SHA-256/)
+        end
+      end
+    end
+
+    context 'when authentication fails' do
+      let(:options) { SpecConfig.instance.ssl_options.merge(
+        user: 'nonexistent_user', password: 'foo') }
+
+      it 'reports which server authentication was attempted against' do
+        expect do
+          connection.connect!
+        end.to raise_error(Mongo::Auth::Unauthorized, /used server: #{connection.address.to_s}/)
+      end
+
+      context 'with default auth source' do
+        it 'reports auth source used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /auth source: admin/)
+        end
+      end
+
+      context 'with custom auth source' do
+        let(:options) { SpecConfig.instance.ssl_options.merge(
+          user: 'nonexistent_user', password: 'foo', auth_source: 'authdb') }
+
+        it 'reports auth source used' do
+          expect do
+            connection.connect!
+          end.to raise_error(Mongo::Auth::Unauthorized, /auth source: authdb/)
+        end
+      end
+    end
+
+    context 'attempting to connect to a non-tls server with tls' do
+      require_no_tls
+      # The exception raised is SocketTimeout on 3.6 server for whatever reason,
+      # run the test on 4.0+ only.
+      min_server_fcv '4.0'
+
+      let(:options) { {ssl: true} }
+
+      it 'reports host, port and tls status' do
+        begin
+          connection.connect!
+        rescue Mongo::Error::SocketError => exc
+        end
+        expect(exc).not_to be nil
+        expect(exc.message).to include('OpenSSL::SSL::SSLError')
+        expect(exc.message).to include(server.address.to_s)
+        expect(exc.message).to include('TLS')
+        expect(exc.message).not_to include('no TLS')
+      end
+    end
+
+    context 'attempting to connect to a tls server without tls' do
+      require_tls
+
+      let(:options) { {ssl: false} }
+
+      it 'reports host, port and tls status' do
+        begin
+          connection.connect!
+        rescue Mongo::Error::SocketError => exc
+        end
+        expect(exc).not_to be nil
+        expect(exc.message).not_to include('OpenSSL::SSL::SSLError')
+        addresses = Socket.getaddrinfo(server.address.host, nil)
+        expect(addresses.any? do |address|
+          exc.message.include?("#{address[2]}:#{server.address.port}")
+        end).to be true
+        expect(exc.message).to include('no TLS')
+      end
+    end
+  end
+
+  shared_examples_for 'caches client key' do
+    it 'caches' do
+      client.close
+      Mongo::Auth::CredentialCache.clear
+
+      RSpec::Mocks.with_temporary_scope do
+        expect_any_instance_of(conversation_class).to receive(:hi).exactly(:once).and_call_original
+
+        client.reconnect
+        server = client.cluster.next_primary
+        server.with_connection do
+          server.with_connection do
+            # nothing
+          end
+        end
+      end
+    end
+  end
+
+  describe 'scram-sha-1 client key caching' do
+    clean_slate
+    min_server_version '3.0'
+    require_no_external_user
+
+    let(:client) { authorized_client.with(max_pool_size: 2, auth_mech: :scram) }
+    let(:conversation_class) { Mongo::Auth::Scram::Conversation }
+
+    it_behaves_like 'caches client key'
+  end
+
+  describe 'scram-sha-256 client key caching' do
+    clean_slate
+    min_server_version '4.0'
+    require_no_external_user
+
+    let(:client) { authorized_client.with(max_pool_size: 2, auth_mech: :scram256) }
+    let(:conversation_class) { Mongo::Auth::Scram256::Conversation }
+
+    it_behaves_like 'caches client key'
+  end
+
+  context 'when only auth source is specified' do
+    require_no_auth
+
+    let(:client) do
+      new_local_client(SpecConfig.instance.addresses, SpecConfig.instance.ssl_options.merge(
+        auth_source: 'foo'))
+    end
+
+    it 'does not authenticate' do
+      expect(Mongo::Auth::User).not_to receive(:new)
+      client.database.command(ping: 1)
+    end
+  end
+
+  context 'when only auth mechanism is specified' do
+    require_x509_auth
+
+    let(:client) do
+      new_local_client(SpecConfig.instance.addresses, SpecConfig.instance.ssl_options.merge(
+        auth_mech: :mongodb_x509))
+    end
+
+    it 'authenticates' do
+      expect(Mongo::Auth::User).to receive(:new).and_call_original
+      client.database.command(ping: 1)
+    end
+  end
+end

data/spec/integration/awaited_ismaster_spec.rb

@@ -0,0 +1,28 @@
+require 'spec_helper'
+
+describe 'awaited ismaster' do
+  min_server_fcv '4.4'
+
+  # If we send the consecutive ismasters to different mongoses,
+  # they have different process ids, and so the awaited one would return
+  # immediately.
+  require_no_multi_shard
+
+  let(:client) { authorized_client }
+
+  it 'waits' do
+    # Perform a regular ismaster to get topology version
+    resp = client.database.command(ismaster: 1)
+    doc = resp.replies.first.documents.first
+    tv = Mongo::TopologyVersion.new(doc['topologyVersion'])
+    tv.should be_a(BSON::Document)
+
+    elapsed_time = Benchmark.realtime do
+      resp = client.database.command(ismaster: 1,
+        topologyVersion: tv.to_doc, maxAwaitTimeMS: 500)
+    end
+    doc = resp.replies.first.documents.first
+
+    elapsed_time.should > 0.5
+  end
+end

data/spec/integration/aws_auth_request_spec.rb

@@ -0,0 +1,74 @@
+require 'lite_spec_helper'
+require 'net/http'
+
+describe Mongo::Auth::Aws::Request do
+  require_aws_auth
+
+  before(:all) do
+    if ENV['AUTH'] =~ /aws-(ec2|ecs)/
+      skip "This test requires explicit credentials to be provided"
+    end
+  end
+
+  let(:access_key_id) { ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID') }
+  let(:secret_access_key) { ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY') }
+  let(:session_token) { ENV['MONGO_RUBY_DRIVER_AWS_AUTH_SESSION_TOKEN'] }
+
+  describe '#authorization' do
+    let(:request) do
+      described_class.new(
+        access_key_id: access_key_id,
+        secret_access_key: secret_access_key,
+        session_token: session_token,
+        host: 'sts.amazonaws.com',
+        server_nonce: 'aaaaaaaaaaafake',
+      )
+    end
+
+    let(:sts_request) do
+      Net::HTTP::Post.new("https://sts.amazonaws.com").tap do |req|
+        request.headers.each do |k, v|
+          req[k] = v
+        end
+        req['authorization'] = request.authorization
+        req['accept'] = 'application/json'
+        req.body = described_class::STS_REQUEST_BODY
+      end
+    end
+
+    let(:sts_response) do
+      http = Net::HTTP.new('sts.amazonaws.com', 443)
+      http.use_ssl = true
+
+      # Uncomment to log complete request headers and the response.
+      # WARNING: do not enable this in Evergreen as this can expose real
+      # AWS credentias.
+      #http.set_debug_output(STDERR)
+
+      http.start do
+        resp = http.request(sts_request)
+      end
+    end
+
+    let(:sts_response_payload) do
+      JSON.parse(sts_response.body)
+    end
+
+    let(:result) do
+      sts_response_payload['GetCallerIdentityResponse']['GetCallerIdentityResult']
+    end
+
+    it 'is usable' do
+      # This assertion intentionally does not use payload so that if it fails,
+      # the entire response is printed for diagnostic purposes.
+      sts_response.body.should_not =~ /"Error"/
+
+      sts_response.code.should == '200'
+      result['Arn'].should =~ /^arn:aws:(iam|sts)::/
+      result['Account'].should be_a(String)
+      result['UserId'].should =~ /^A/
+
+      puts "STS request successful with ARN #{result['Arn']}"
+    end
+  end
+end

data/spec/integration/aws_credentials_retriever_spec.rb

@@ -0,0 +1,103 @@
+require 'lite_spec_helper'
+require 'support/aws_utils'
+
+describe Mongo::Auth::Aws::CredentialsRetriever do
+  require_aws_auth
+
+  let(:retriever) do
+    described_class.new(user)
+  end
+
+  let(:credentials) do
+    retriever.credentials
+  end
+
+  context 'when user is not given' do
+    let(:user) do
+      Mongo::Auth::User.new(auth_mech: :aws)
+    end
+
+    shared_examples_for 'retrieves the credentials' do
+      it 'retrieves' do
+        credentials.should be_a(Mongo::Auth::Aws::Credentials)
+
+        # When user is not given, credentials retrieved are always temporary.
+        retriever.credentials.access_key_id.should =~ /^ASIA/
+        retriever.credentials.secret_access_key.should =~ /./
+        retriever.credentials.session_token.should =~ /./
+      end
+
+      let(:request) do
+        Mongo::Auth::Aws::Request.new(
+          access_key_id: credentials.access_key_id,
+          secret_access_key: credentials.secret_access_key,
+          session_token: credentials.session_token,
+          host: 'sts.amazonaws.com',
+          server_nonce: 'test',
+        )
+      end
+
+      it 'produces valid credentials' do
+        result = request.validate!
+        puts "STS request successful with ARN #{result['Arn']}"
+      end
+    end
+
+    context 'ec2 instance role' do
+      require_ec2_host
+
+      before(:all) do
+        unless ENV['AUTH'] == 'aws-ec2'
+          skip "Set AUTH=aws-ec2 in environment to run EC2 instance role tests"
+        end
+      end
+
+      context 'when instance profile is not assigned' do
+        before(:all) do
+          orchestrator = AwsUtils::Orchestrator.new(
+            region: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_REGION'),
+            access_key_id: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID'),
+            secret_access_key: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY'),
+          )
+
+          orchestrator.clear_instance_profile(Utils.ec2_instance_id)
+          Utils.wait_for_no_instance_profile
+        end
+
+        it 'raises an error' do
+          lambda do
+            credentials
+          end.should raise_error(Mongo::Auth::InvalidConfiguration, /Could not locate AWS credentials/)
+        end
+      end
+
+      context 'when instance profile is assigned' do
+        before(:all) do
+          orchestrator = AwsUtils::Orchestrator.new(
+            region: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_REGION'),
+            access_key_id: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_ACCESS_KEY_ID'),
+            secret_access_key: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_SECRET_ACCESS_KEY'),
+          )
+
+          orchestrator.set_instance_profile(Utils.ec2_instance_id,
+            instance_profile_name: nil,
+            instance_profile_arn: ENV.fetch('MONGO_RUBY_DRIVER_AWS_AUTH_INSTANCE_PROFILE_ARN'),
+          )
+          Utils.wait_for_instance_profile
+        end
+
+        it_behaves_like 'retrieves the credentials'
+      end
+    end
+
+    context 'ecs task role' do
+      before(:all) do
+        unless ENV['AUTH'] == 'aws-ecs'
+          skip "Set AUTH=aws-ecs in environment to run ECS task role tests"
+        end
+      end
+
+      it_behaves_like 'retrieves the credentials'
+    end
+  end
+end

data/spec/integration/bson_symbol_spec.rb

@@ -0,0 +1,36 @@
+require 'spec_helper'
+
+describe 'Symbol encoding to BSON' do
+  let(:value) { :foo }
+
+  let(:hash) do
+    {'foo' => value}
+  end
+
+  let(:serialized) do
+    hash.to_bson.to_s
+  end
+
+  let(:expected) do
+    "\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00foo\x00\x00".force_encoding('binary')
+  end
+
+  it 'encodes symbol to BSON symbol' do
+    serialized.should == expected
+  end
+
+  it 'round-trips symbol values' do
+    buffer = BSON::ByteBuffer.new(serialized)
+    Hash.from_bson(buffer).should == hash
+  end
+
+  it 'round-trips symbol values using the same byte buffer' do
+    if BSON::Environment.jruby? && (BSON::VERSION.split('.').map(&:to_i) <=> [4, 11, 0]) < 0
+      skip 'This test is only relevant to bson versions that increment ByteBuffer '\
+       'read and write positions separately in JRuby, as implemented in ' \
+       'bson version 4.11.0. For more information, see https://jira.mongodb.org/browse/RUBY-2128'
+    end
+
+    Hash.from_bson(hash.to_bson).should == hash
+  end
+end