RubyGems - configure-s3-website - Versions diffs - 1.2.0 → 1.3.0 - Mend

configure-s3-website 1.2.0 → 1.3.0

Files changed (27) hide show

data/.travis.yml +1 -1
data/README.md +44 -33
data/bin/configure-s3-website +2 -24
data/changelog.md +5 -0
data/features/cassettes/cucumber_tags/create-cf-dist.yml +134 -0
data/features/configure_bucket.feature +18 -11
data/features/create_cloudfront_dist.feature +164 -0
data/features/step_definitions/steps.rb +44 -7
data/features/support/env.rb +4 -0
data/features/support/sample_config_files/config_with_cloudfront_distribution_id.yml +4 -0
data/features/support/sample_config_files/create_cf_dist.yml +3 -0
data/features/support/vcr.rb +1 -0
data/lib/configure-s3-website.rb +5 -0
data/lib/configure-s3-website/cli.rb +35 -0
data/lib/configure-s3-website/cloudfront_client.rb +117 -0
data/lib/configure-s3-website/config_source/config_source.rb +9 -0
data/lib/configure-s3-website/config_source/file_config_source.rb +19 -3
data/lib/configure-s3-website/http_helper.rb +83 -0
data/lib/configure-s3-website/runner.rb +17 -0
data/lib/configure-s3-website/s3_client.rb +9 -68
data/lib/configure-s3-website/version.rb +1 -1
data/lib/configure-s3-website/xml_helper.rb +15 -0
data/spec/config_source/file_config_source_spec.rb +22 -0
data/spec/s3_client_spec.rb +2 -2
data/spec/xml_helper_spec.rb +39 -0
metadata +21 -6
data/spec/config_extractor/file_config_extractor_spec.rb +0 -11

data/.travis.yml CHANGED

@@ -1,5 +1,5 @@
 language: ruby
 rvm:
-- 1.8.7
 - 1.9.2
 - 1.9.3
+- 2.0.0

data/README.md CHANGED

@@ -31,6 +31,33 @@ following command:
 Congratulations! You now have an S3 bucket that can act as a website server for
 you.
+### Deliver your website via CloudFront
+`configure-s3-website` can create a CloudFront distribution for you. It will ask
+you whether you want to deliver your website via the CDN. If you answer yes,
+`configure-s3-website` will create a CloudFront distribution that has the
+configured S3 bucket as its origin. In addition, it will add the entry
+`cloudfront_distribution_id: [id-of-the-new-distribution]` into your
+configuration file.
+CloudFront can be configured in various ways. However, the distribution created
+by `configure-s3-website` uses sensible defaults for an S3-based website and
+thus saves you the burden of figuring out how to configure CloudFront. For
+example, it assumes that your default root object is *index.html*.
+You can see all the settings this gem applies on the new distribution by running
+the command in verbose mode:
+    configure-s3-website --config-file config.yml --verbose
+Note that if you already have the key `cloudfront_distribution_id` in your
+configuration file, `configure-s3-website` will not create a new distribution.
+Conversely, if you remove the `cloudfront_distribution_id` key from the file and
+run `configure-s3-website` again, it will create you a new distribution.
+If you want to, you can tune the distribution settings on the management console
+at <https://console.aws.amazon.com/cloudfront>.
 ### Specifying a non-standard S3 endpoint
 By default, `configure-s3-website` creates the S3 website into the US Standard
@@ -67,43 +94,23 @@ supports. All you have to do is to replace the uppercase letter in AWS XML with
 an underscore and an undercase version of the same letter. For example,
 `KeyPrefixEquals` becomes `key_prefix_equals` in the config file.
+Apply the rules by invoking `configure-s3-website --config [your-config-file]`
+on the command-line interface. You can verify the results by looking at your
+bucket on the [S3 console](https://console.aws.amazon.com/s3/home).
 ## How does `configure-s3-website` work?
-It calls the [PUT Bucket
-website](http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUTwebsite.html)
-API with the following XML:
-```xml
-<WebsiteConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>
-  <IndexDocument>
-    <Suffix>index.html</Suffix>
-  </IndexDocument>
-  <ErrorDocument>
-    <Key>error.html</Key>
-  </ErrorDocument>
-</WebsiteConfiguration>
-```
+`configure-s3-website` uses the AWS REST API of S3 for creating and modifying
+the bucket. In brief, it does the following things:
-Then **it makes all the objects on the bucket visible to the whole world** by
-calling the [PUT Bucket
-policy](http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUTpolicy.html)
-API with the following JSON:
-```json
-{
-  "Version":"2008-10-17",
-  "Statement":[{
-    "Sid":"PublicReadForGetBucketObjects",
-    "Effect":"Allow",
-    "Principal": { "AWS": "*" },
-    "Action":["s3:GetObject"],
-    "Resource":["arn:aws:s3:::your-bucket-name/*"]
-  }]
-}
-```
+1. Create a bucket for you (if it does not yet exist)
+2. Add the website configuration on the bucket via the [website REST
+   API](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTwebsite.html)
+3. Make the bucket **readable to the whole world**
+4. Apply the redirect (a.k.a routing) rules on the bucket website
-If you define `routing_rules` in the config file, `configure-s3-website` will
-make an additional call to the AWS API.
+In addition, if you instruct `configure-s3-website` to create a CloudFront
+distribution to you, it will call the [CloudFront POST Distribution](http://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/CreateDistribution.html) API.
 ## Development
@@ -118,6 +125,10 @@ Big thanks to the following contributors (in alphabetical order):
 * SlawD
 * Steve Schwartz
+## Supported Ruby versions
+The file `.travis.yml` defines the supported Ruby versions.
 ## License
 See the file LICENSE.

data/bin/configure-s3-website CHANGED

@@ -5,30 +5,8 @@ $:.unshift File.join(File.dirname(__FILE__), *%w[.. lib])
 require 'optparse'
 require 'configure-s3-website'
-options = {}
-banner = <<END
-Usage: #{File.basename(__FILE__)} arguments
-Configure your S3 bucket to function as a web site
-Arguments:
-END
-optparse = OptionParser.new do |opts|
-  opts.banner = banner
-  opts.on('-f', '--config-file FILE',
-          'Pick credentials and the S3 bucket name from a config file') do
-          |yaml_file_path|
-    options[:config_source] =
-      ConfigureS3Website::FileConfigSource.new yaml_file_path
-  end
-  opts.on('--help', 'Display this screen') do
-    puts opts
-    exit
-  end
-end
+options, optparse = ConfigureS3Website::CLI.optparse_and_options
 optparse.parse!
-ConfigureS3Website::S3Client.configure_website(options[:config_source])
+ConfigureS3Website::Runner.run(options)

data/changelog.md CHANGED

@@ -2,6 +2,11 @@
 This project uses [Semantic Versioning](http://semver.org).
+## 1.3.0
+* Create a CloudFront distro if the user wants to deliver his S3 website via the
+  CDN
 ## 1.2.0
 * Support configuring redirects on the S3 website

data/features/cassettes/cucumber_tags/create-cf-dist.yml ADDED

@@ -0,0 +1,134 @@
+---
+http_interactions:
+- request:
+    method: put
+    uri: https://s3.amazonaws.com/website-via-cf/?website
+    body:
+      encoding: US-ASCII
+      string: ! "\n        <WebsiteConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>\n
+        \         <IndexDocument>\n            <Suffix>index.html</Suffix>\n          </IndexDocument>\n
+        \         <ErrorDocument>\n            <Key>error.html</Key>\n          </ErrorDocument>\n
+        \       </WebsiteConfiguration>\n      "
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:54 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '298'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      X-Amz-Id-2:
+      - vB5gARfon866SjZUI2QIDzjxJjParAzDxo8vLjDC+SBujim60dH0XzrF9uPIlj7u
+      X-Amz-Request-Id:
+      - 16950B9C61E9FF87
+      Date:
+      - Sat, 18 May 2013 20:07:59 GMT
+      Content-Length:
+      - '0'
+      Server:
+      - AmazonS3
+    body:
+      encoding: US-ASCII
+      string: ''
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:55 GMT
+- request:
+    method: put
+    uri: https://s3.amazonaws.com/website-via-cf/?policy
+    body:
+      encoding: US-ASCII
+      string: ! "{\n        \"Version\":\"2008-10-17\",\n        \"Statement\":[{\n
+        \         \"Sid\":\"PublicReadForGetBucketObjects\",\n          \"Effect\":\"Allow\",\n
+        \         \"Principal\": { \"AWS\": \"*\" },\n          \"Action\":[\"s3:GetObject\"],\n
+        \         \"Resource\":[\"arn:aws:s3:::website-via-cf/*\"]\n        }]\n      }"
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:55 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '283'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 204
+      message: No Content
+    headers:
+      X-Amz-Id-2:
+      - v7kj1pp5cB/kl2WbrQeDfrog3f7gVx17znu+S0bvUsf0CxaZtg+1Lk650LalVyQH
+      X-Amz-Request-Id:
+      - 25D798662824A94C
+      Date:
+      - Sat, 18 May 2013 20:08:00 GMT
+      Server:
+      - AmazonS3
+    body:
+      encoding: US-ASCII
+      string: ''
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:56 GMT
+- request:
+    method: post
+    uri: https://cloudfront.amazonaws.com/2012-07-01/distribution
+    body:
+      encoding: US-ASCII
+      string: ! "\n        <DistributionConfig xmlns=\"http://cloudfront.amazonaws.com/doc/2012-07-01/\">\n
+        \         <Origins>\n            <Quantity>1</Quantity>\n            <Items>\n
+        \             <Origin>\n                <Id>website-via-cf-S3-origin</Id>\n
+        \               <DomainName>website-via-cf.s3.amazonaws.com</DomainName>\n
+        \               <S3OriginConfig>\n                  <OriginAccessIdentity></OriginAccessIdentity>\n
+        \               </S3OriginConfig>\n              </Origin>\n            </Items>\n
+        \         </Origins>\n          \n<CallerReference>configure-s3-website gem
+        2013-05-18 23:07:56 +0300</CallerReference>\n<DefaultRootObject>index.html</DefaultRootObject>\n<Logging>\n
+        \ <Enabled>false</Enabled>\n  <IncludeCookies>false</IncludeCookies>\n  <Bucket></Bucket>\n
+        \ <Prefix></Prefix></Logging>\n<Enabled>true</Enabled>\n<Comment>Created by
+        the configure-s3-website gem</Comment>\n<Aliases>\n  <Quantity>0</Quantity></Aliases>\n<DefaultCacheBehavior>\n
+        \ <TargetOriginId>website-via-cf-S3-origin</TargetOriginId>\n  <TrustedSigners>\n
+        \   <Enabled>false</Enabled>\n    <Quantity>0</Quantity></TrustedSigners>\n
+        \ <ForwardedValues>\n    <QueryString>true</QueryString>\n    <Cookies>\n
+        \     <Forward>all</Forward></Cookies></ForwardedValues>\n  <ViewerProtocolPolicy>allow-all</ViewerProtocolPolicy>\n
+        \ <MinTTL>86400</MinTTL></DefaultCacheBehavior>\n<CacheBehaviors>\n  <Quantity>0</Quantity></CacheBehaviors>\n<PriceClass>PriceClass_All</PriceClass>\n
+        \       </DistributionConfig>\n        "
+    headers:
+      Date:
+      - Sat, 18 May 2013 20:07:56 UTC
+      Content-Type:
+      - ''
+      Content-Length:
+      - '1451'
+      Authorization:
+      - AWS foo:foo
+  response:
+    status:
+      code: 201
+      message: Created
+    headers:
+      X-Amzn-Requestid:
+      - a3ab794f-bff6-11e2-abb8-3518185a092c
+      Etag:
+      - E1N3VW3E5YJ58U
+      Location:
+      - https://cloudfront.amazonaws.com/2012-07-01/distribution/E45H2VN49KPDU
+      Content-Type:
+      - text/xml
+      Content-Length:
+      - '1536'
+      Date:
+      - Sat, 18 May 2013 20:08:00 GMT
+    body:
+      encoding: US-ASCII
+      string: ! '<?xml version="1.0"?>
+        <Distribution xmlns="http://cloudfront.amazonaws.com/doc/2012-07-01/"><Id>E45H2VN49KPDU</Id><Status>InProgress</Status><LastModifiedTime>2013-05-18T20:08:00.350Z</LastModifiedTime><InProgressInvalidationBatches>0</InProgressInvalidationBatches><DomainName>d3feoe9t5ufu01.cloudfront.net</DomainName><ActiveTrustedSigners><Enabled>false</Enabled><Quantity>0</Quantity></ActiveTrustedSigners><DistributionConfig><CallerReference>configure-s3-website
+        gem 2013-05-18 23:07:56 +0300</CallerReference><Aliases><Quantity>0</Quantity></Aliases><DefaultRootObject>index.html</DefaultRootObject><Origins><Quantity>1</Quantity><Items><Origin><Id>website-via-cf-S3-origin</Id><DomainName>website-via-cf.s3.amazonaws.com</DomainName><S3OriginConfig><OriginAccessIdentity></OriginAccessIdentity></S3OriginConfig></Origin></Items></Origins><DefaultCacheBehavior><TargetOriginId>website-via-cf-S3-origin</TargetOriginId><ForwardedValues><QueryString>true</QueryString><Cookies><Forward>all</Forward></Cookies></ForwardedValues><TrustedSigners><Enabled>false</Enabled><Quantity>0</Quantity></TrustedSigners><ViewerProtocolPolicy>allow-all</ViewerProtocolPolicy><MinTTL>86400</MinTTL></DefaultCacheBehavior><CacheBehaviors><Quantity>0</Quantity></CacheBehaviors><Comment>Created
+        by the configure-s3-website gem</Comment><Logging><Enabled>false</Enabled><IncludeCookies>false</IncludeCookies><Bucket></Bucket><Prefix></Prefix></Logging><PriceClass>PriceClass_All</PriceClass><Enabled>true</Enabled></DistributionConfig></Distribution>'
+    http_version:
+  recorded_at: Sat, 18 May 2013 20:07:57 GMT
+recorded_with: VCR 2.3.0

data/features/configure_bucket.feature CHANGED

@@ -2,21 +2,24 @@ Feature: configure an S3 bucket to function as a website
   @bucket-does-not-exist
   Scenario: The bucket does not yet exist
-    Given my config file is in "features/support/sample_config_files/s3_config_with_non-existing_bucket.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                       |
+      | --config-file | features/support/sample_config_files/s3_config_with_non-existing_bucket.yml |
     Then the output should be
       """
       Created bucket name-of-a-new-bucket in the US Standard Region
       Bucket name-of-a-new-bucket now functions as a website
       Bucket name-of-a-new-bucket is now readable to the whole world
       No redirects to configure for name-of-a-new-bucket bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
       """
   @bucket-does-not-exist
   Scenario: The configuration does not contain the 's3_endpoint' setting
-    Given my config file is in "features/support/sample_config_files/s3_config_with_non-existing_bucket.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                       |
+      | --config-file | features/support/sample_config_files/s3_config_with_non-existing_bucket.yml |
     Then the output should include
       """
       Created bucket name-of-a-new-bucket in the US Standard Region
@@ -24,8 +27,9 @@ Feature: configure an S3 bucket to function as a website
   @bucket-does-not-exist-in-tokyo
   Scenario: Create bucket into the Tokyo region
-    Given my config file is in "features/support/sample_config_files/endpoint_tokyo.yml"
-    When I run the configure-s3-website command
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/endpoint_tokyo.yml |
     Then the output should include
       """
       Created bucket name-of-a-new-bucket in the Asia Pacific (Tokyo) Region
@@ -33,9 +37,10 @@ Feature: configure an S3 bucket to function as a website
   @bucket-exists
   Scenario: The bucket already exists
-    Given my config file is in "features/support/sample_config_files/s3_config_with_existing_bucket.yml"
-    When I run the configure-s3-website command
-    Then the output should be
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                   |
+      | --config-file | features/support/sample_config_files/s3_config_with_existing_bucket.yml |
+    Then the output should include
       """
       Bucket name-of-an-existing-bucket now functions as a website
       Bucket name-of-an-existing-bucket is now readable to the whole world
@@ -45,13 +50,15 @@ Feature: configure an S3 bucket to function as a website
   @redirects
   Scenario: The user wants to configure redirects for the S3 website
-    Given my config file is in "features/support/sample_config_files/redirects.yml"
-    When I run the configure-s3-website command
+    And I run the configure-s3-website command with parameters
+      | option        | value                                              |
+      | --config-file | features/support/sample_config_files/redirects.yml |
     Then the output should be
       """
       Created bucket website-with-redirects in the US Standard Region
       Bucket website-with-redirects now functions as a website
       Bucket website-with-redirects is now readable to the whole world
       1 redirects configured for website-with-redirects bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
       """

data/features/create_cloudfront_dist.feature ADDED

@@ -0,0 +1,164 @@
+Feature: Create CloudFront distribution
+  @create-cf-dist
+  Scenario: The user wants to deliver his website via CloudFront
+    Given I answer 'yes' to 'do you want to use CloudFront'
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/create_cf_dist.yml |
+    Then the output should be
+      """
+      Bucket website-via-cf now functions as a website
+      Bucket website-via-cf is now readable to the whole world
+      No redirects to configure for website-via-cf bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
+        The distribution E45H2VN49KPDU at d3feoe9t5ufu01.cloudfront.net now delivers the bucket website-via-cf
+          Please allow up to 15 minutes for the distribution to initialise
+          For more information on the distribution, see https://console.aws.amazon.com/cloudfront
+        Added setting 'cloudfront_distribution_id: E45H2VN49KPDU' into features/support/sample_config_files/create_cf_dist.yml
+      """
+  @create-cf-dist
+  Scenario: The user wants to deliver his website via CloudFront and see details on the new distribution
+    Given I answer 'yes' to 'do you want to use CloudFront'
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                   |
+      | --config-file | features/support/sample_config_files/create_cf_dist.yml |
+      | --verbose     |                                                         |
+    Then the output should be
+      """
+      Bucket website-via-cf now functions as a website
+      Bucket website-via-cf is now readable to the whole world
+      No redirects to configure for website-via-cf bucket
+      Do you want to deliver your website via CloudFront, the CDN of Amazon? [y/N]
+        The distribution E45H2VN49KPDU at d3feoe9t5ufu01.cloudfront.net now delivers the bucket website-via-cf
+          Please allow up to 15 minutes for the distribution to initialise
+          For more information on the distribution, see https://console.aws.amazon.com/cloudfront
+        Below is the response from the CloudFront API:
+          <?xml version='1.0'?>
+          <Distribution xmlns='http://cloudfront.amazonaws.com/doc/2012-07-01/'>
+            <Id>
+              E45H2VN49KPDU
+            </Id>
+            <Status>
+              InProgress
+            </Status>
+            <LastModifiedTime>
+              2013-05-18T20:08:00.350Z
+            </LastModifiedTime>
+            <InProgressInvalidationBatches>
+              0
+            </InProgressInvalidationBatches>
+            <DomainName>
+              d3feoe9t5ufu01.cloudfront.net
+            </DomainName>
+            <ActiveTrustedSigners>
+              <Enabled>
+                false
+              </Enabled>
+              <Quantity>
+                0
+              </Quantity>
+            </ActiveTrustedSigners>
+            <DistributionConfig>
+              <CallerReference>
+                configure-s3-website gem 2013-05-18 23:07:56 +0300
+              </CallerReference>
+              <Aliases>
+                <Quantity>
+                  0
+                </Quantity>
+              </Aliases>
+              <DefaultRootObject>
+                index.html
+              </DefaultRootObject>
+              <Origins>
+                <Quantity>
+                  1
+                </Quantity>
+                <Items>
+                  <Origin>
+                    <Id>
+                      website-via-cf-S3-origin
+                    </Id>
+                    <DomainName>
+                      website-via-cf.s3.amazonaws.com
+                    </DomainName>
+                    <S3OriginConfig>
+                      <OriginAccessIdentity/>
+                    </S3OriginConfig>
+                  </Origin>
+                </Items>
+              </Origins>
+              <DefaultCacheBehavior>
+                <TargetOriginId>
+                  website-via-cf-S3-origin
+                </TargetOriginId>
+                <ForwardedValues>
+                  <QueryString>
+                    true
+                  </QueryString>
+                  <Cookies>
+                    <Forward>
+                      all
+                    </Forward>
+                  </Cookies>
+                </ForwardedValues>
+                <TrustedSigners>
+                  <Enabled>
+                    false
+                  </Enabled>
+                  <Quantity>
+                    0
+                  </Quantity>
+                </TrustedSigners>
+                <ViewerProtocolPolicy>
+                  allow-all
+                </ViewerProtocolPolicy>
+                <MinTTL>
+                  86400
+                </MinTTL>
+              </DefaultCacheBehavior>
+              <CacheBehaviors>
+                <Quantity>
+                  0
+                </Quantity>
+              </CacheBehaviors>
+              <Comment>
+                Created by the configure-s3-website gem
+              </Comment>
+              <Logging>
+                <Enabled>
+                  false
+                </Enabled>
+                <IncludeCookies>
+                  false
+                </IncludeCookies>
+                <Bucket/>
+                <Prefix/>
+              </Logging>
+              <PriceClass>
+                PriceClass_All
+              </PriceClass>
+              <Enabled>
+                true
+              </Enabled>
+            </DistributionConfig>
+          </Distribution>
+        Added setting 'cloudfront_distribution_id: E45H2VN49KPDU' into features/support/sample_config_files/create_cf_dist.yml
+      """
+  @bucket-exists
+  Scenario: The user already has CloudFront configured in his configuration file
+    When I run the configure-s3-website command with parameters
+      | option        | value                                                                           |
+      | --config-file | features/support/sample_config_files/config_with_cloudfront_distribution_id.yml |
+    Then the output should be
+      """
+      Bucket name-of-an-existing-bucket now functions as a website
+      Bucket name-of-an-existing-bucket is now readable to the whole world
+      No redirects to configure for name-of-an-existing-bucket bucket
+      """