mongo 2.4.1 → 2.14.0

data/spec/NOTES.aws-auth.md

@@ -0,0 +1,296 @@
+# AWS Authentication Implementation Notes
+
+## AWS Account
+
+Per [its documentation](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html,
+the GetCallerIdentity API call that the server makes to STS to authenticate
+the user using MONGODB-AWS auth mechanism requires no privileges. This means
+in order to test authentication using non-temporary credentials (i.e.,
+AWS access key id and secret access key only) it is sufficient to create an
+IAM user that has no permissions but does have programmatic access enabled
+(i.e. has an access key id and secret access key).
+
+## AWS Signature V4
+
+The driver implements the AWS signature v4 internally rather than relying on
+a third-party library (such as the
+[AWS SDK for Ruby](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/index.html))
+to provide the signature implementation. The implementation is quite compact
+but getting it working took some effort due to:
+
+1. [The server not logging AWS responses when authentication fails
+](https://jira.mongodb.org/browse/SERVER-46909)
+2. Some of the messages from STS being quite cryptic (I could not figure out
+what the problem was for either "Request is missing Authentication Token" or
+"Request must contain a signature that conforms to AWS standards", and
+ultimately resolved these problems by comparing my requests to those produced
+by the AWS SDK).
+3. Amazon's own documentation not providing an example signature calculation
+that could be followed to verify correctness, especially since this is a
+multi-step process and all kinds of subtle errors are possible in many of the
+steps like using a date instead of a time, hex-encoding a MAC in an
+intermediate step or not separating header values from the list of signed
+headers by two newlines.
+
+### Reference Implementation - AWS SDK
+
+To see actual working STS requests I used Amazon's
+[AWS SDK for Ruby](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/index.html)
+([API docs for STS client](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/EC2/Client.html),
+[configuration documentation](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html))
+as follows:
+
+1. Set the credentials in the environment (note that the region must be
+explicitly provided):
+
+    export AWS_ACCESS_KEY_ID=AKIAREALKEY
+    export AWS_SECRET_ACCESS_KEY=Sweee/realsecret
+    export AWS_REGION=us-east-1
+
+2. Install the correct gem and launch IRb:
+
+    gem install aws-sdk-core
+    irb -raws-sdk-core -Iaws/sts
+
+3. Send a GetCallerIdentity request, as used by MongoDB server:
+
+    Aws::STS::Client.new(
+      logger: Logger.new(STDERR, level: :debug),
+      http_wire_trace: true,
+    ).get_caller_identity
+
+This call enables HTTP request and response logging and produces output
+similar to the following:
+
+    opening connection to sts.amazonaws.com:443...
+    opened
+    starting SSL for sts.amazonaws.com:443...
+    SSL established, protocol: TLSv1.2, cipher: ECDHE-RSA-AES128-SHA
+    <- "POST / HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded; charset=utf-8\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.91.1 ruby/2.7.0 x86_64-linux aws-sdk-core/3.91.1\r\nHost: sts.amazonaws.com\r\nX-Amz-Date: 20200317T194745Z\r\nX-Amz-Content-Sha256: ab821ae955788b0e33ebd34c208442ccfc2d406e2edc5e7a39bd6458fbb4f843\r\nAuthorization: AWS4-HMAC-SHA256 Credential=AKIAREALKEY/20200317/us-east-1/sts/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=6cd3a60a2d7dfba0dcd17f9c4c42d0186de5830cf99545332253a327bba14131\r\nContent-Length: 43\r\nAccept: */*\r\n\r\n"
+    -> "HTTP/1.1 200 OK\r\n"
+    -> "x-amzn-RequestId: c56f5d68-8763-4032-a835-fd95efd83fa6\r\n"
+    -> "Content-Type: text/xml\r\n"
+    -> "Content-Length: 401\r\n"
+    -> "Date: Tue, 17 Mar 2020 19:47:44 GMT\r\n"
+    -> "\r\n"
+    reading 401 bytes...
+    -> ""
+    -> "<GetCallerIdentityResponse xmlns=\"https://sts.amazonaws.com/doc/2011-06-15/\">\n  <GetCallerIdentityResult>\n    <Arn>arn:aws:iam::5851234356:user/test</Arn>\n    <UserId>AIDAREALUSERID</UserId>\n    <Account>5851234356</Account>\n  </GetCallerIdentityResult>\n  <ResponseMetadata>\n    <RequestId>c56f5d68-8763-4032-a835-fd95efd83fa6</RequestId>\n  </ResponseMetadata>\n</GetCallerIdentityResponse>\n"
+    read 401 bytes
+    Conn keep-alive
+    I, [2020-03-17T15:47:45.275421 #9815]  INFO -- : [Aws::STS::Client 200 0.091573 0 retries] get_caller_identity()  
+
+    => #<struct Aws::STS::Types::GetCallerIdentityResponse user_id="AIDAREALUSERID", account="5851234356", arn="arn:aws:iam::5851234356:user/test">
+
+Note that:
+
+1. The set of headers sent by the AWS SDK differs from the set
+  of headers that the MONGODB-AWS auth mechanism specification mentions.
+  I used the AWS SDK implementation as a guide to determine the correct shape
+  of the request to STS and in particular the `Authorization` header.
+  The source code of Amazon's implementation is
+  [here](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sigv4/lib/aws-sigv4/signer.rb)
+  and it generates, in particular, the x-amz-content-sha256` header
+  which the MONGODB-AWS auth mechanism specification does not mention.
+2. This is a working request which can be replayed, making it possible
+  to send this request that was created by the AWS SDK repeatedly with minor
+  alterations to study STS error reporting behavior. STS as of this writing
+  allows a 15 minute window during which a request may be replayed.
+3. The printed request only shows the headers and not the request body.
+  In case of the GetCallerIdentity, the payload is fixed and is the same as
+  what the MONGODB-AWS auth mechanism specification requires
+  (`Action=GetCallerIdentity&Version=2011-06-15`).
+
+Because the AWS SDK includes a different set of headers in its requests,
+it not feasible to compare the canonical requests generated by AWS SDK
+verbatim to the canonical requests generated by the driver.
+
+### Manual Requests
+
+It is possible to manually send requests to STS using OpenSSL `s_client`
+tool in combination with the [printf](https://linux.die.net/man/3/printf)
+utility to transform the newline escapes. A sample command replaying the
+request printed above is as follows:
+
+    (printf "POST / HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded; charset=utf-8\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.91.1 ruby/2.7.0 x86_64-linux aws-sdk-core/3.91.1\r\nHost: sts.amazonaws.com\r\nX-Amz-Date: 20200317T194745Z\r\nX-Amz-Content-Sha256: ab821ae955788b0e33ebd34c208442ccfc2d406e2edc5e7a39bd6458fbb4f843\r\nAuthorization: AWS4-HMAC-SHA256 Credential=AKIAREALKEY/20200317/us-east-1/sts/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=6cd3a60a2d7dfba0dcd17f9c4c42d0186de5830cf99545332253a327bba14131\r\nContent-Length: 43\r\nAccept: */*\r\n\r\n" &&
+      echo "Action=GetCallerIdentity&Version=2011-06-15" &&
+      sleep 5) |openssl s_client -connect sts.amazonaws.com:443
+
+Note the sleep call - `s_client` does not wait for the remote end to provide
+a response before exiting, thus the sleep on the input side allows 5 seconds
+for STS to process the request and respond.
+
+For reference, Amazon provides [GetCallerIdentity API documentation
+](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html).
+
+### Integration Test - Signature Generation
+
+The Ruby driver includes an integration test for signature generation, where
+the driver makes the call to `GetCallerIdentity` STS endpoint using the
+provided AWS credentials. This test is in
+`spec/integration/aws_auth_request_spec.rb`.
+
+### STS Error Responses
+
+The error responses produced by STS sometimes do not clearly indicate the
+problem. Below are some of the puzzling responses I encountered:
+
+- *Request is missing Authentication Token*: request is missing the
+  `Authorization` header, or the value of the header does not begin with
+  `AWS4-`. For example, this error is produced if the signature algorithm
+  is erroneously given as `AWS-HMAC-SHA256` instead of `AWS4-HMAC-SHA256`
+  with the remainder of the header value being correctly constructed.
+  This error is also produced if the value of the header erroneously includes
+  the name of the header (i.e. the header name is specified twice in the header
+  line) but the value is otherwise completely valid. This error has no relation
+  to the "session token" or "security token" as used with temporary AWS
+  credentials.
+- *The security token included in the request is invalid*: this error can be
+  produced in several circumstances:
+  - When the AWS access key id, as specified in the scope part of the
+    `Authorization` header, is not a valid access key id. In the case of
+    non-temporary credentials being used for authentication, the error refers to
+    a "security token" but the authentication process does not actually use a
+    security token as this term is used in the AWS documentation describing
+    temporary credentials.
+  - When using temporary credentials and the security token is not provided
+    in the STS request at all (x-amz-security-token header).
+- *Signature expired: 20200317T000000Z is now earlier than 20200317T222541Z
+  (20200317T224041Z - 15 min.)*: This error happens when `x-amz-date` header
+  value is the formatted date (`YYYYMMDD`) rather than the ISO8601 formatted
+  time (`YYYYMMDDTHHMMSSZ`). Note that the string `20200317T000000Z` is never
+  explicitly provided in the request - it is derived by AWS from the provided
+  header `x-amz-date: 20200317`.
+- *The request signature we calculated does not match the signature
+  you provided. Check your AWS Secret Access Key and signing method. Consult
+  the service documentation for details*: this is the error produced when
+  the signature is not calculated correctly but everything else in the
+  request is valid. If a different error is produced, most likely the problem
+  is in something other than signature calculation.
+- *The security token included in the request is expired*: this error is
+  produced when temporary credentials are used and the credentials have
+  expired.
+
+See also [AWS documentation for STS error messages](https://docs.aws.amazon.com/STS/latest/APIReference/CommonErrors.html).
+
+### Resources
+
+Generally I found Amazon's own documentation to be the best for implementing
+the signature calculation. The following documents should be read in order:
+
+- [Signing AWS requests overview](https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html)
+- [Creating canonical request](https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html)
+- [Creating string to sign](https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html)
+- [Calculating signature](https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html)
+
+### Signature Debugger
+
+The most excellent [awssignature.com](http://www.awssignature.com/) was
+indispensable in debugging the actual signature calculation process.
+
+### MongoDB Server
+
+MongoDB server internally defines the set of headers that it is prepared to
+handle when it is processing AWS authentication. Headers that are not part
+of that set cause the server to reject driver's payloads.
+
+The error reporting when additional headers are provided and when the
+correct set of headers is provided but the headers are not ordered
+lexicographically [can be misleading](https://jira.mongodb.org/browse/SERVER-47488).
+
+## Direct AWS Requests
+
+[STS GetCallerIdentity API docs](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html)
+
+When making direct requests to AWS, adding `Accept: application/json`
+header will return the results in the JSON format, including the errors.
+
+## AWS CLI
+
+[Configuration reference](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html)
+
+Note that AWS CLI uses `AWS_DEFAULT_REGION` environment variable to configure
+the region used for operations.
+
+## AWS Ruby SDK
+
+[Configuration reference](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html)
+
+Note that AWS Ruby SDK uses `AWS_REGION` environment variable to configure
+the region used for operations.
+
+[STS::Client#assume_role documentation](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/STS/Client.html#assume_role-instance_method)
+
+## IMDSv2
+
+`X-aws-ec2-metadata-token-ttl-seconds` is a required header when using
+IMDSv2 EC2 instance metadata requests. This header is used in the examples
+on [Amazon's page describing
+IMDSv2](https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/),
+but is not explicitly stated as being required.
+
+Not providing this header fails the PUT requests with HTTP code 400.
+
+## IAM Roles For EC2 Instances
+
+### Metadata Rate Limit
+
+[Amazon documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html#instancedata-throttling)
+states that the EC2 instance metadata endpoint is rate limited. Since the
+driver accesses it to obtain credentials whenever a connection is established,
+rate limits may adversely affect the driver's ability to establish connections.
+
+### Instance Profile Assignment
+
+It can take over 5 seconds for an instance to see its instance profile change
+reflected in the instance metadata. Evergreen test runs seem to experience
+this delay to a significantly larger extent than testing in a standalone
+AWS account.
+
+## IAM Roles For ECS Tasks
+
+### ECS Task Roles
+
+When an ECS task (or more precisely, the task definition) is created,
+it is possible to specify an *execution role* and a *task role*. The two are
+completely separate; an execution role is required to, for example, be
+able to send container logs to CloudWatch if the container is running in
+Fargate, and a task role is required for AWS authentication purposes.
+
+The ECS task role is also separate from EC2 instance role and the IAM role
+for a user to assume a role - these roles all require different configuration.
+
+### `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` Scope
+
+As stated in [this Amazon support document](https://aws.amazon.com/premiumsupport/knowledge-center/ecs-iam-task-roles-config-errors/),
+the `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` environment variable is only
+available to the PID 1 process in the container. Other processes need to
+extract it from PID 1's environment:
+
+    strings /proc/1/environment
+
+### Other ECS Metadata
+
+`strings /proc/1/environment` also shows a number of other enviroment
+variables available in the container with metadata. For example a test
+container yields:
+
+    HOSTNAME=f893c90ec4bd
+    ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/5fb0b11b-c4c8-4cdb-b68b-edf70b3f4937
+    AWS_DEFAULT_REGION=us-east-2
+    AWS_EXECUTION_ENV=AWS_ECS_FARGATE
+    AWS_REGION=us-east-2
+    AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/v2/credentials/f17b5770-9a0d-498c-8d26-eea69f8d0924
+
+### Metadata Rate Limit
+
+[Amazon documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/troubleshoot-task-iam-roles.html)
+states that ECS task metadata endpoint is subject to rate limiting,
+which is configured via [ECS_TASK_METADATA_RPS_LIMIT container agent
+parameter](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html).
+When the rate limit is reached, requests fail with `429 Too Many Requests`
+HTTP status code.
+
+Since the driver accesses this endpoint to obtain credentials whenever
+a connection is established, rate limits may adversely affect the driver's
+ability to establish connections.

data/spec/README.aws-auth.md

@@ -0,0 +1,318 @@
+# Testing AWS Authentication
+
+## Server Configuration
+
+AWS authentication requires the following to be done on the server side:
+
+1. The AWS authentication mechanism must be enabled on the server. This
+is done by adding `MONGODB-AWS` to the values in `authenticationMechanisms`
+server parameter.
+
+2. A user must be created in the `$external` database with the ARN matching
+the IAM user or role that the client will authenticate as.
+
+Note that the server does not need to have AWS keys provided to it - it
+uses the keys that the client provides during authentication.
+
+An easy way to configure the deployment in the required fashion is to
+configure the deployment to accept both password authentication and
+AWS authentication, and add a bootstrap user:
+
+    mlaunch init --single --auth --username root --password toor \
+      --setParameter authenticationMechanisms=MONGODB-AWS,SCRAM-SHA-1,SCRAM-SHA-256 \
+      --dir /tmp/db
+
+Then connect as the bootstrap user and create AWS-mapped users:
+
+    mongo mongodb://root:toor@localhost:27017
+    
+    # In the mongo shell:
+    use $external
+    db.createUser({
+      user: 'arn:aws:iam::1234567890:user/test',
+      roles: [{role:'root', db:'admin'}]})
+
+The ARN can be retrieved from the AWS management console. Alternatively,
+if the IAM user's access and secret keys are known, trying to authenticate
+as the user will log the user's ARN into the server log when authentication
+fails; this ARN can be then used to create the server user.
+
+With the server user created, it is possible to authenticate using AWS.
+The following example uses regular user credentials for an IAM user
+created as described in the next section;
+
+    mongo 'mongodb://AKIAAAAAAAAAAAA:t9t2mawssecretkey@localhost:27017/?authMechanism=MONGODB-AWS&authsource=$external'
+
+To authenticate, provide the IAM user's access key id as the username and
+secret access key as the password. Note that the username and the password
+must be percent-escaped when they are passed in the URI as the examples here
+show. Also note that the user's ARN is not explicitly specified by the client
+during authentication - the server determines the ARN from the acess
+key id and the secret access key provided by the client.
+
+## Provisioning Tools
+
+The Ruby driver includes tools that set up the resources needed to test
+AWS authentication. These are exposed by the `.evergreen/aws` script.
+To use this script, it must be provided AWS credentials and the region
+to operate in. The credentials and region can be given as command-line
+arguments or set in the environment, as follows:
+
+    export AWS_ACCESS_KEY_ID=AKIAYOURACCESSKEY
+    export AWS_SECRET_ACCESS_KEY=YOURSECRETACCESSKEY
+    export AWS_REGION=us-east-1
+
+If you also perform manual testing (for example by following some of the
+instructions in this file), ensure AWS_SESSION_TOKEN is not set
+unless you are intending to invoke the `.evergreen/aws` script with
+temporary credentials:
+
+    unset AWS_SESSION_TOKEN
+
+Note that [AWS CLI](https://aws.amazon.com/cli/) uses a different environment
+variable for the region - `AWS_DEFAULT_REGION` rather than `AWS_REGION`.
+If you also intend to use the AWS CLI, execute:
+
+    export AWS_DEFAULT_REGION=$AWS_REGION
+
+To verify that credentials are correctly set in the environment, you can
+perform the following operations:
+
+    # Test driver tooling
+    ./.evergreen/aws key-pairs
+    
+    # Test AWS CLI
+    aws sts get-caller-identity
+
+Alternatively, to provide the credentials on each call to the driver's
+`aws` script, use the `-a` and `-s` arguments as follows:
+
+    ./.evergreen/aws -a KEY-ID -s SECRET-KEY key-pairs
+
+## Common Setup
+
+In order to test all AWS authentication scenarios, a large number of AWS
+objects needs to be configured. This configuration is split into two parts:
+common setup and scenario-specific setup.
+
+The common setup is performed by running:
+
+    ./.evergreen/aws setup-resources
+
+This creates resources like security groups, IAM users and CloudWatch
+log groups that do not cost money. It is possible to test authentication
+with regular credentials and temporary credentials obtained via an
+AssumeRole request using these resources. In order to test authentication
+from an EC2 instance or an ECS task, the instance and/or the task need
+to be started which costs money and is performed as separate steps as
+detailed below.
+
+## Regular Credentials - IAM User
+
+AWS authentication as a regular IAM user requires having an IAM user to
+authenticate as. This user can be created using the AWS management console.
+The IAM user requires no permissions, but it must have the programmatic
+access enabled (i.e. have an access key ID and the secret access key).
+
+An IAM user is created as part of the common setup described earlier.
+To reset and retrieve the access key ID and secret access key for the
+created user, run:
+
+    ./.evergreen/aws reset-keys
+
+Note that if the user already had an access key, the old credentials are
+removed and replaced with new credentials.
+
+Given the credentials for the test user, the URI for running the driver
+test suite can be formed as follows:
+
+    export "MONGODB_URI=mongodb://$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY@localhost:27017/?authMechanism=MONGODB-AWS&authsource=$external"
+
+## Temporary Credentials - AssumeRole Request
+
+To test a user authenticating with an assumed role, you can follow
+[the example provided in Amazon documentation](https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/)
+to set up the assumed role and related objects and obtain temporary credentials
+or use the driver's tooling using the commands given below.
+Since the temporary credentials expire, the role needs to be re-assumed
+periodically during testing and the new credentials and session token retrieved.
+
+If following the example in Amazon's documentation,
+[jq](https://stedolan.github.io/jq/) can be used to efficiently place the
+credentials from the AssumeRole request into the environment, as follows:
+
+    # Call given in the example guide
+    aws sts assume-role --role-arn arn:aws:iam::YOUR-ACCOUNT-ID:role/example-role --role-session-name AWSCLI-Session >~/.aws-assumed-role.json
+    
+    # Extract the credentials
+    export AWS_ACCESS_KEY_ID=`jq .Credentials.AccessKeyId  ~/.aws-assumed-role.json -r`
+    export AWS_SECRET_ACCESS_KEY=`jq .Credentials.SecretAccessKey  ~/.aws-assumed-role.json -r`
+    export AWS_SESSION_TOKEN=`jq .Credentials.SessionToken ~/.aws-assumed-role.json -r`
+
+Alternatively, the `./evergreen/aws` script can be used to assume the role.
+By default, it will assume the role that `setup-resources` action configured.
+
+Note: The ability to assume this role is granted to the
+[IAM user](#regular-credentials-iam-user) that the provisioning tool creates.
+Therefore the shell must be configured with credentials of the test user,
+not with credentials of the master user that performed the provisioning.
+
+To assume the role created by the common setup, run:
+
+    ./.evergreen/aws assume-role
+
+It is also possible to specify the ARN of the role to assume manually, if
+you created the role using other means:
+
+    ./.evergreen/aws assume-role ASSUME-ROLE-ARN
+
+To place the credentials into the environment:
+
+    eval $(./.evergreen/aws assume-role)
+    export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
+
+With the credentials in the environment, to verify that the role was assumed
+and the credentials are complete and correct, perform a `GetCallerIdentity`
+call:
+
+    aws sts get-caller-identity
+
+Given the credentials for the test user, the URI for running the driver
+test suite can be formed as follows:
+
+    export "MONGODB_URI=mongodb://$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY@localhost:27017/?authMechanism=MONGODB-AWS&authsource=$external&authMechanismProperties=AWS_SESSION_TOKEN:$AWS_SESSION_TOKEN"
+
+## Temporary Credentials - EC2 Instance Role
+
+To test authentication [using temporary credentials for an EC2 instance
+role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html),
+an EC2 instance launched with an IAM role or an EC2 instance configured
+with an instance profile is required. No permissions are needed for the
+IAM role used with the EC2 instance.
+
+To create an EC2 instance with an attached role using the AWS console:
+
+1. Crate an IAM role that the instance will use. It is not necessary to
+specify any permissions.
+2. Launch an instance, choosing the IAM role created in the launch wizard.
+
+To define an instance profile which allows adding and removing an IAM role
+to/from an instance at runtime, follow Amazon documentation
+[here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#attach-iam-role).
+To test temporary credentials obtained via an EC2 instance role in Evergreen,
+an instance profile must be associated with the running instance as per
+this guide.
+
+The driver provides tooling to configure a suitable instance profile and
+launch an EC2 instance that can have this instance profile attached to it.
+
+The instance profile and associated IAM role are created by the common
+setup described above. To launch an EC2 instance suitable for testing
+authentication via an EC2 role, run:
+
+    ./.evergreen/aws launch-ec2 path/to/ssh.key.pub
+
+The `launch-ec2` command takes one argument which is the path to the
+public key for the key pair to use for SSH access to the instance.
+
+This script will output the instance ID of the launched instance. The
+instance initially does not have an instance profile assigned; to assign
+the instance profile created in the common setup to the instance, run:
+
+    ./.evergreen/aws set-instance-profile i-instanceid
+
+To remove the instance profile from the instance, run:
+
+    ./.evergreen/aws clear-instance-profile i-instanceid
+
+To provision the instance for running the driver's test suite via Docker, run:
+
+    ip=12.34.56.78
+    ./.evergreen/provision-remote ubuntu@$ip docker
+
+To run the AWS auth tests using the EC2 instance role credentials, run:
+
+    ./.evergreen/test-docker-remote ubuntu@$ip \
+      MONGODB_VERSION=4.4 AUTH=aws-ec2 \
+      -s .evergreen/run-tests-aws-auth.sh \
+      -a .env.private
+
+Note that if if you are not using MongoDB AWS account for testing, you
+would need to specify MONGO_RUBY_DRIVER_AWS_AUTH_USER_ARN in your
+`.env.private` file with the ARN of the user to add to MongoDB. The easiest
+way to find out this value is to run the tests and note which username the
+test suite is trying to authenticate as.
+
+To terminate the instance, run:
+
+    ./.evergreen/aws stop-ec2
+
+## Temporary Credentials - ECS Task Role
+
+The basic procedure for setting up an ECS cluster is described in
+[this guide](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_AWSCLI_Fargate.html).
+For testing AWS auth, the ECS task must have a role assigned to it which is
+covered in [this guide](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html)
+and additionally [here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html).
+
+Although not required for testing AWS auth specifically, it is very helpful
+for general troubleshooting of ECS provisioning to have log output from the
+tasks. Logging to CloudWatch is covered by [this Amazon guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html)
+with these potentially helpful [additional](https://stackoverflow.com/questions/50397217/how-to-determine-the-cloudwatch-log-stream-for-a-fargate-service#50704804)
+[resources](https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/AWS_Fargate_log_collection).
+A log group must be manually created, the steps for which are described
+[here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html).
+
+Additional references:
+
+- [Task definition CPU and memory values](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html)
+
+The common setup creates all of the necessary prerequisites to test
+authentication using ECS task credentials, which includes an empty ECS
+cluster. To test authentication, a service needs to be created in the
+ECS cluster that runs the SSH daemon, which can be done by running:
+
+    ./.evergreen/aws launch-ecs path/to/ssh.key.pub
+
+The `launch-ecs` command takes one argument which is the path to the
+public key for the key pair to use for SSH access to the instance.
+
+This script generally produces no output if it succeeds. As the service takes
+some time to start, run the following command to check its status:
+
+    ./.evergreen/aws ecs-status
+
+The status output shows the tasks running in the ECS cluster ordered by their
+generation, with the newest ones first. Event log for the cluster is displayed,
+as well as event stream for the running task of the latest available generation
+which includes the Docker execution output collected via CloudWatch.
+The status output includes the public IP of the running task once it is
+available, which can be used to SSH into the container and run the tests.
+
+Note that when AWS auth from an ECS task is tested in Evergreen, the task is
+accessed via its private IP; when the test is performed using the provisioning
+tooling described in this document, the task is accessed via its public IP.
+
+If the public IP address is in the `IP` shell variable, provision the task:
+
+    ./.evergreen/provision-remote root@$IP local
+
+To run the credentials retrieval test on the ECS task, execute:
+
+    ./.evergreen/test-remote root@$IP env AUTH=aws-ecs RVM_RUBY=ruby-2.7 MONGODB_VERSION=4.4 TEST_CMD='rspec spec/integration/aws*spec.rb' .evergreen/run-tests.sh
+
+To run the test again without rebuilding the remote environment, execute:
+
+    ./.evergreen/test-remote -e root@$IP \
+      env AUTH=aws-ecs RVM_RUBY=ruby-2.7 sh -c '\
+        export PATH=`pwd`/rubies/ruby-2.7/bin:$PATH && \
+        eval export `strings /proc/1/environ |grep ^AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` && \
+        bundle exec rspec spec/integration/aws*spec.rb'
+
+Note that this command retrieves the value of `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`
+from the PID 1 environment and places it into the current environment prior to
+running the tests.
+
+To terminate the AWS auth-related ECS tasks, run:
+
+    ./.evergreen/aws stop-ecs