RubyGems - configure-s3-website - Versions diffs - 1.2.0 → 1.3.0 - Mend

configure-s3-website 1.2.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (27) hide show

data/.travis.yml +1 -1
data/README.md +44 -33
data/bin/configure-s3-website +2 -24
data/changelog.md +5 -0
data/features/cassettes/cucumber_tags/create-cf-dist.yml +134 -0
data/features/configure_bucket.feature +18 -11
data/features/create_cloudfront_dist.feature +164 -0
data/features/step_definitions/steps.rb +44 -7
data/features/support/env.rb +4 -0
data/features/support/sample_config_files/config_with_cloudfront_distribution_id.yml +4 -0
data/features/support/sample_config_files/create_cf_dist.yml +3 -0
data/features/support/vcr.rb +1 -0
data/lib/configure-s3-website.rb +5 -0
data/lib/configure-s3-website/cli.rb +35 -0
data/lib/configure-s3-website/cloudfront_client.rb +117 -0
data/lib/configure-s3-website/config_source/config_source.rb +9 -0
data/lib/configure-s3-website/config_source/file_config_source.rb +19 -3
data/lib/configure-s3-website/http_helper.rb +83 -0
data/lib/configure-s3-website/runner.rb +17 -0
data/lib/configure-s3-website/s3_client.rb +9 -68
data/lib/configure-s3-website/version.rb +1 -1
data/lib/configure-s3-website/xml_helper.rb +15 -0
data/spec/config_source/file_config_source_spec.rb +22 -0
data/spec/s3_client_spec.rb +2 -2
data/spec/xml_helper_spec.rb +39 -0
metadata +21 -6
data/spec/config_extractor/file_config_extractor_spec.rb +0 -11

data/.travis.yml CHANGED

@@ -1,5 +1,5 @@
 language: ruby
 rvm:
-- 1.8.7
 - 1.9.2
 - 1.9.3
+- 2.0.0

data/README.md CHANGED

@@ -31,6 +31,33 @@ following command:
 Congratulations! You now have an S3 bucket that can act as a website server for
 you.
+### Deliver your website via CloudFront
+`configure-s3-website` can create a CloudFront distribution for you. It will ask
+you whether you want to deliver your website via the CDN. If you answer yes,
+`configure-s3-website` will create a CloudFront distribution that has the
+configured S3 bucket as its origin. In addition, it will add the entry
+`cloudfront_distribution_id: [id-of-the-new-distribution]` into your
+configuration file.
+CloudFront can be configured in various ways. However, the distribution created
+by `configure-s3-website` uses sensible defaults for an S3-based website and
+thus saves you the burden of figuring out how to configure CloudFront. For
+example, it assumes that your default root object is *index.html*.
+You can see all the settings this gem applies on the new distribution by running
+the command in verbose mode:
+    configure-s3-website --config-file config.yml --verbose
+Note that if you already have the key `cloudfront_distribution_id` in your
+configuration file, `configure-s3-website` will not create a new distribution.
+Conversely, if you remove the `cloudfront_distribution_id` key from the file and
+run `configure-s3-website` again, it will create you a new distribution.
+If you want to, you can tune the distribution settings on the management console
+at <https://console.aws.amazon.com/cloudfront>.
 ### Specifying a non-standard S3 endpoint
 By default, `configure-s3-website` creates the S3 website into the US Standard
@@ -67,43 +94,23 @@ supports. All you have to do is to replace the uppercase letter in AWS XML with
 an underscore and an undercase version of the same letter. For example,
 `KeyPrefixEquals` becomes `key_prefix_equals` in the config file.
+Apply the rules by invoking `configure-s3-website --config [your-config-file]`
+on the command-line interface. You can verify the results by looking at your
+bucket on the [S3 console](https://console.aws.amazon.com/s3/home).
 ## How does `configure-s3-website` work?
-It calls the [PUT Bucket
-website](http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUTwebsite.html)
-API with the following XML:
-```xml
-<WebsiteConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>
-  <IndexDocument>
-    <Suffix>index.html</Suffix>
-  </IndexDocument>
-  <ErrorDocument>
-    <Key>error.html</Key>
-  </ErrorDocument>
-</WebsiteConfiguration>
-```
+`configure-s3-website` uses the AWS REST API of S3 for creating and modifying
+the bucket. In brief, it does the following things:
-Then **it makes all the objects on the bucket visible to the whole world** by
-calling the [PUT Bucket
-policy](http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUTpolicy.html)
-API with the following JSON:
-```json
-{
-  "Version":"2008-10-17",
-  "Statement":[{
-    "Sid":"PublicReadForGetBucketObjects",
-    "Effect":"Allow",
-    "Principal": { "AWS": "*" },
-    "Action":["s3:GetObject"],
-    "Resource":["arn:aws:s3:::your-bucket-name/*"]
-  }]
-}
-```
+1. Create a bucket for you (if it does not yet exist)
+2. Add the website configuration on the bucket via the [website REST
+   API](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTwebsite.html)
+3. Make the bucket **readable to the whole world**
+4. Apply the redirect (a.k.a routing) rules on the bucket website
-If you define `routing_rules` in the config file, `configure-s3-website` will
-make an additional call to the AWS API.
+In addition, if you instruct `configure-s3-website` to create a CloudFront
+distribution to you, it will call the [CloudFront POST Distribution](http://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/CreateDistribution.html) API.
 ## Development
@@ -118,6 +125,10 @@ Big thanks to the following contributors (in alphabetical order):
 * SlawD
 * Steve Schwartz
+## Supported Ruby versions
+The file `.travis.yml` defines the supported Ruby versions.
 ## License
 See the file LICENSE.

data/bin/configure-s3-website CHANGED

@@ -5,30 +5,8 @@ $:.unshift File.join(File.dirname(__FILE__), *%w[.. lib])
 require 'optparse'
 require 'configure-s3-website'
-options = {}
-banner = <<END
-Usage: #{File.basename(__FILE__)} arguments
-Configure your S3 bucket to function as a web site
-Arguments:
-END
-optparse = OptionParser.new do |opts|
-  opts.banner = banner
-  opts.on('-f', '--config-file FILE',
-          'Pick credentials and the S3 bucket name from a config file') do
-          |yaml_file_path|
-    options[:config_source] =
-      ConfigureS3Website::FileConfigSource.new yaml_file_path
-  end
-  opts.on('--help', 'Display this screen') do
-    puts opts
-    exit
-  end
-end
+options, optparse = ConfigureS3Website::CLI.optparse_and_options
 optparse.parse!
-ConfigureS3Website::S3Client.configure_website(options[:config_source])
+ConfigureS3Website::Runner.run(options)

data/changelog.md CHANGED

@@ -2,6 +2,11 @@
 This project uses [Semantic Versioning](http://semver.org).
+## 1.3.0
+* Create a CloudFront distro if the user wants to deliver his S3 website via the
+  CDN
 ## 1.2.0
 * Support configuring redirects on the S3 website

data/features/cassettes/cucumber_tags/create-cf-dist.yml ADDED

@@ -0,0 +1,134 @@
+---
+http_interactions:
+- request:
+    method: put
+    uri: https://s3.amazonaws.com/website-via-cf/?website
+    body:
+      encoding: US-ASCII
+      string: ! "\n        <WebsiteConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>\n
+        \         <IndexDocument>\n            <Suffix>index.html</Suffix>\n          </IndexDocument>\n
+        \         <ErrorDocument>\n            <Key>error.html</Key>\n          </ErrorDocument>\n
+        \       </WebsiteConfiguration>\n      "
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:54 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '298'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      X-Amz-Id-2:
+      - vB5gARfon866SjZUI2QIDzjxJjParAzDxo8vLjDC+SBujim60dH0XzrF9uPIlj7u
+      X-Amz-Request-Id:
+      - 16950B9C61E9FF87
+      Date:
+      - Sat, 18 May 2013 20:07:59 GMT
+      Content-Length:
+      - '0'
+      Server:
+      - AmazonS3
+    body:
+      encoding: US-ASCII
+      string: ''
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:55 GMT
+- request:
+    method: put
+    uri: https://s3.amazonaws.com/website-via-cf/?policy
+    body:
+      encoding: US-ASCII
+      string: ! "{\n        \"Version\":\"2008-10-17\",\n        \"Statement\":[{\n
+        \         \"Sid\":\"PublicReadForGetBucketObjects\",\n          \"Effect\":\"Allow\",\n
+        \         \"Principal\": { \"AWS\": \"*\" },\n          \"Action\":[\"s3:GetObject\"],\n
+        \         \"Resource\":[\"arn:aws:s3:::website-via-cf/*\"]\n        }]\n      }"
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:55 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '283'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 204
+      message: No Content
+    headers:
+      X-Amz-Id-2:
+      - v7kj1pp5cB/kl2WbrQeDfrog3f7gVx17znu+S0bvUsf0CxaZtg+1Lk650LalVyQH
+      X-Amz-Request-Id:
+      - 25D798662824A94C
+      Date:
+      - Sat, 18 May 2013 20:08:00 GMT
+      Server:
+      - AmazonS3
+    body:
+      encoding: US-ASCII
+      string: ''
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:56 GMT
+- request:
+    method: post
+    uri: https://cloudfront.amazonaws.com/2012-07-01/distribution
+    body:
+      encoding: US-ASCII
+      string: ! "\n        <DistributionConfig xmlns=\"http://cloudfront.amazonaws.com/doc/2012-07-01/\">\n
+        \         <Origins>\n            <Quantity>1</Quantity>\n            <Items>\n
+        \             <Origin>\n                <Id>website-via-cf-S3-origin</Id>\n
+        \               <DomainName>website-via-cf.s3.amazonaws.com</DomainName>\n
+        \               <S3OriginConfig>\n                  <OriginAccessIdentity></OriginAccessIdentity>\n
+        \               </S3OriginConfig>\n              </Origin>\n            </Items>\n
+        \         </Origins>\n          \n<CallerReference>configure-s3-website gem
+        2013-05-18 23:07:56 +0300</CallerReference>\n<DefaultRootObject>index.html</DefaultRootObject>\n<Logging>\n
+        \ <Enabled>false</Enabled>\n  <IncludeCookies>false</IncludeCookies>\n  <Bucket></Bucket>\n
+        \ <Prefix></Prefix></Logging>\n<Enabled>true</Enabled>\n<Comment>Created by
+        the configure-s3-website gem</Comment>\n<Aliases>\n  <Quantity>0</Quantity></Aliases>\n<DefaultCacheBehavior>\n
+        \ <TargetOriginId>website-via-cf-S3-origin</TargetOriginId>\n  <TrustedSigners>\n
+        \   <Enabled>false</Enabled>\n    <Quantity>0</Quantity></TrustedSigners>\n
+        \ <ForwardedValues>\n    <QueryString>true</QueryString>\n    <Cookies>\n
+        \     <Forward>all</Forward></Cookies></ForwardedValues>\n  <ViewerProtocolPolicy>allow-all</ViewerProtocolPolicy>\n
+        \ <MinTTL>86400</MinTTL></DefaultCacheBehavior>\n<CacheBehaviors>\n  <Quantity>0</Quantity></CacheBehaviors>\n<PriceClass>PriceClass_All</PriceClass>\n
+        \       </DistributionConfig>\n        "
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:56 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '1451'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 201
+      message: Created
+    headers:
+      X-Amzn-Requestid:
+      - a3ab794f-bff6-11e2-abb8-3518185a092c
+      Etag:
+      - E1N3VW3E5YJ58U
+      Location:
+      - https://cloudfront.amazonaws.com/2012-07-01/distribution/E45H2VN49KPDU
+      Content-Type:
+      - text/xml
+      Content-Length:
+      - '1536'
+      Date:
+      - Sat, 18 May 2013 20:08:00 GMT
+    body:
+      encoding: US-ASCII
+      string: ! '<?xml version="1.0"?>
+        <Distribution xmlns="http://cloudfront.amazonaws.com/doc/2012-07-01/"><Id>E45H2VN49KPDU</Id><Status>InProgress</Status><LastModifiedTime>2013-05-18T20:08:00.350Z</LastModifiedTime><InProgressInvalidationBatches>0</InProgressInvalidationBatches><DomainName>d3feoe9t5ufu01.cloudfront.net</DomainName><ActiveTrustedSigners><Enabled>false</Enabled><Quantity>0</Quantity></ActiveTrustedSigners><DistributionConfig><CallerReference>configure-s3-website
+        gem 2013-05-18 23:07:56 +0300</CallerReference><Aliases><Quantity>0</Quantity></Aliases><DefaultRootObject>index.html</DefaultRootObject><Origins><Quantity>1</Quantity><Items><Origin><Id>website-via-cf-S3-origin</Id><DomainName>website-via-cf.s3.amazonaws.com</DomainName><S3OriginConfig><OriginAccessIdentity></OriginAccessIdentity></S3OriginConfig></Origin></Items></Origins><DefaultCacheBehavior><TargetOriginId>website-via-cf-S3-origin</TargetOriginId><ForwardedValues><QueryString>true</QueryString><Cookies><Forward>all</Forward></Cookies></ForwardedValues><TrustedSigners><Enabled>false</Enabled><Quantity>0</Quantity></TrustedSigners><ViewerProtocolPolicy>allow-all</ViewerProtocolPolicy><MinTTL>86400</MinTTL></DefaultCacheBehavior><CacheBehaviors><Quantity>0</Quantity></CacheBehaviors><Comment>Created
+        by the configure-s3-website gem</Comment><Logging><Enabled>false</Enabled><IncludeCookies>false</IncludeCookies><Bucket></Bucket><Prefix></Prefix></Logging><PriceClass>PriceClass_All</PriceClass><Enabled>true</Enabled></DistributionConfig></Distribution>'
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:57 GMT
+recorded_with: VCR 2.3.0

data/features/configure_bucket.feature CHANGED

@@ -2,21 +2,24 @@ Feature: configure an S3 bucket to function as a website
   @bucket-does-not-exist
   Scenario: The bucket does not yet exist
-    Given my config file is in "features/support/sample_config_files/s3_config_with_non-existing_bucket.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                       |
+      | --config-file | features/support/sample_config_files/s3_config_with_non-existing_bucket.yml |
     Then the output should be
       """
       Created bucket name-of-a-new-bucket in the US Standard Region
       Bucket name-of-a-new-bucket now functions as a website
       Bucket name-of-a-new-bucket is now readable to the whole world
       No redirects to configure for name-of-a-new-bucket bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
       """
   @bucket-does-not-exist
   Scenario: The configuration does not contain the 's3_endpoint' setting
-    Given my config file is in "features/support/sample_config_files/s3_config_with_non-existing_bucket.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                       |
+      | --config-file | features/support/sample_config_files/s3_config_with_non-existing_bucket.yml |
     Then the output should include
       """
       Created bucket name-of-a-new-bucket in the US Standard Region
@@ -24,8 +27,9 @@ Feature: configure an S3 bucket to function as a website
   @bucket-does-not-exist-in-tokyo
   Scenario: Create bucket into the Tokyo region
-    Given my config file is in "features/support/sample_config_files/endpoint_tokyo.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/endpoint_tokyo.yml |
     Then the output should include
       """
       Created bucket name-of-a-new-bucket in the Asia Pacific (Tokyo) Region
@@ -33,9 +37,10 @@ Feature: configure an S3 bucket to function as a website
   @bucket-exists
   Scenario: The bucket already exists
-    Given my config file is in "features/support/sample_config_files/s3_config_with_existing_bucket.yml"
-    When I run the configure-s3-website command
-    Then the output should be
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                   |
+      | --config-file | features/support/sample_config_files/s3_config_with_existing_bucket.yml |
+    Then the output should include
       """
       Bucket name-of-an-existing-bucket now functions as a website
       Bucket name-of-an-existing-bucket is now readable to the whole world
@@ -45,13 +50,15 @@ Feature: configure an S3 bucket to function as a website
   @redirects
   Scenario: The user wants to configure redirects for the S3 website
-    Given my config file is in "features/support/sample_config_files/redirects.yml"
-    When I run the configure-s3-website command
+    And I run the configure-s3-website command with parameters
+      | option        | value                                              |
+      | --config-file | features/support/sample_config_files/redirects.yml |
     Then the output should be
       """
       Created bucket website-with-redirects in the US Standard Region
       Bucket website-with-redirects now functions as a website
       Bucket website-with-redirects is now readable to the whole world
       1 redirects configured for website-with-redirects bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
       """

data/features/create_cloudfront_dist.feature ADDED

@@ -0,0 +1,164 @@
+Feature: Create CloudFront distribution
+  @create-cf-dist
+  Scenario: The user wants to deliver his website via CloudFront
+    Given I answer 'yes' to 'do you want to use CloudFront'
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/create_cf_dist.yml |
+    Then the output should be
+      """
+      Bucket website-via-cf now functions as a website
+      Bucket website-via-cf is now readable to the whole world
+      No redirects to configure for website-via-cf bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
+        The distribution E45H2VN49KPDU at d3feoe9t5ufu01.cloudfront.net now delivers the bucket website-via-cf
+          Please allow up to 15 minutes for the distribution to initialise
+          For more information on the distribution, see https://console.aws.amazon.com/cloudfront
+        Added setting 'cloudfront_distribution_id: E45H2VN49KPDU' into features/support/sample_config_files/create_cf_dist.yml
+      """
+  @create-cf-dist
+  Scenario: The user wants to deliver his website via CloudFront and see details on the new distribution
+    Given I answer 'yes' to 'do you want to use CloudFront'
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/create_cf_dist.yml |
+      | --verbose     |                                                         |
+    Then the output should be
+      """
+      Bucket website-via-cf now functions as a website
+      Bucket website-via-cf is now readable to the whole world
+      No redirects to configure for website-via-cf bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
+        The distribution E45H2VN49KPDU at d3feoe9t5ufu01.cloudfront.net now delivers the bucket website-via-cf
+          Please allow up to 15 minutes for the distribution to initialise
+          For more information on the distribution, see https://console.aws.amazon.com/cloudfront
+        Below is the response from the CloudFront API:
+          <?xml version='1.0'?>
+          <Distribution xmlns='http://cloudfront.amazonaws.com/doc/2012-07-01/'>
+            <Id>
+              E45H2VN49KPDU
+            </Id>
+            <Status>
+              InProgress
+            </Status>
+            <LastModifiedTime>
+              2013-05-18T20:08:00.350Z
+            </LastModifiedTime>
+            <InProgressInvalidationBatches>
+              0
+            </InProgressInvalidationBatches>
+            <DomainName>
+              d3feoe9t5ufu01.cloudfront.net
+            </DomainName>
+            <ActiveTrustedSigners>
+              <Enabled>
+                false
+              </Enabled>
+              <Quantity>
+                0
+              </Quantity>
+            </ActiveTrustedSigners>
+            <DistributionConfig>
+              <CallerReference>
+                configure-s3-website gem 2013-05-18 23:07:56 +0300
+              </CallerReference>
+              <Aliases>
+                <Quantity>
+                  0
+                </Quantity>
+              </Aliases>
+              <DefaultRootObject>
+                index.html
+              </DefaultRootObject>
+              <Origins>
+                <Quantity>
+                  1
+                </Quantity>
+                <Items>
+                  <Origin>
+                    <Id>
+                      website-via-cf-S3-origin
+                    </Id>
+                    <DomainName>
+                      website-via-cf.s3.amazonaws.com
+                    </DomainName>
+                    <S3OriginConfig>
+                      <OriginAccessIdentity/>
+                    </S3OriginConfig>
+                  </Origin>
+                </Items>
+              </Origins>
+              <DefaultCacheBehavior>
+                <TargetOriginId>
+                  website-via-cf-S3-origin
+                </TargetOriginId>
+                <ForwardedValues>
+                  <QueryString>
+                    true
+                  </QueryString>
+                  <Cookies>
+                    <Forward>
+                      all
+                    </Forward>
+                  </Cookies>
+                </ForwardedValues>
+                <TrustedSigners>
+                  <Enabled>
+                    false
+                  </Enabled>
+                  <Quantity>
+                    0
+                  </Quantity>
+                </TrustedSigners>
+                <ViewerProtocolPolicy>
+                  allow-all
+                </ViewerProtocolPolicy>
+                <MinTTL>
+                  86400
+                </MinTTL>
+              </DefaultCacheBehavior>
+              <CacheBehaviors>
+                <Quantity>
+                  0
+                </Quantity>
+              </CacheBehaviors>
+              <Comment>
+                Created by the configure-s3-website gem
+              </Comment>
+              <Logging>
+                <Enabled>
+                  false
+                </Enabled>
+                <IncludeCookies>
+                  false
+                </IncludeCookies>
+                <Bucket/>
+                <Prefix/>
+              </Logging>
+              <PriceClass>
+                PriceClass_All
+              </PriceClass>
+              <Enabled>
+                true
+              </Enabled>
+            </DistributionConfig>
+          </Distribution>
+        Added setting 'cloudfront_distribution_id: E45H2VN49KPDU' into features/support/sample_config_files/create_cf_dist.yml
+      """
+  @bucket-exists
+  Scenario: The user already has CloudFront configured in his configuration file
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                           |
+      | --config-file | features/support/sample_config_files/config_with_cloudfront_distribution_id.yml |
+    Then the output should be
+      """
+      Bucket name-of-an-existing-bucket now functions as a website
+      Bucket name-of-an-existing-bucket is now readable to the whole world
+      No redirects to configure for name-of-an-existing-bucket bucket
+      """