s3_website_revived 4.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (48) hide show
  1. checksums.yaml +7 -0
  2. data/.gitignore +15 -0
  3. data/.travis.yml +5 -0
  4. data/Gemfile +3 -0
  5. data/LICENSE +42 -0
  6. data/README.md +591 -0
  7. data/Rakefile +2 -0
  8. data/additional-docs/debugging.md +21 -0
  9. data/additional-docs/development.md +29 -0
  10. data/additional-docs/example-configurations.md +113 -0
  11. data/additional-docs/running-from-ec2-with-dropbox.md +6 -0
  12. data/additional-docs/setting-up-aws-credentials.md +52 -0
  13. data/assembly.sbt +3 -0
  14. data/bin/s3_website +285 -0
  15. data/build.sbt +48 -0
  16. data/changelog.md +596 -0
  17. data/lib/s3_website/version.rb +3 -0
  18. data/lib/s3_website.rb +7 -0
  19. data/project/assembly.sbt +1 -0
  20. data/project/build.properties +1 -0
  21. data/project/plugins.sbt +1 -0
  22. data/release +41 -0
  23. data/resources/configuration_file_template.yml +67 -0
  24. data/resources/s3_website.jar.md5 +1 -0
  25. data/s3_website-4.0.0.jar +0 -0
  26. data/s3_website.gemspec +34 -0
  27. data/sbt +3 -0
  28. data/src/main/resources/log4j.properties +6 -0
  29. data/src/main/scala/s3/website/ByteHelper.scala +18 -0
  30. data/src/main/scala/s3/website/CloudFront.scala +144 -0
  31. data/src/main/scala/s3/website/Logger.scala +67 -0
  32. data/src/main/scala/s3/website/Push.scala +246 -0
  33. data/src/main/scala/s3/website/Ruby.scala +14 -0
  34. data/src/main/scala/s3/website/S3.scala +239 -0
  35. data/src/main/scala/s3/website/UploadHelper.scala +76 -0
  36. data/src/main/scala/s3/website/model/Config.scala +249 -0
  37. data/src/main/scala/s3/website/model/S3Endpoint.scala +35 -0
  38. data/src/main/scala/s3/website/model/Site.scala +159 -0
  39. data/src/main/scala/s3/website/model/push.scala +225 -0
  40. data/src/main/scala/s3/website/model/ssg.scala +30 -0
  41. data/src/main/scala/s3/website/package.scala +182 -0
  42. data/src/test/scala/s3/website/AwsSdkSpec.scala +15 -0
  43. data/src/test/scala/s3/website/ConfigSpec.scala +150 -0
  44. data/src/test/scala/s3/website/S3EndpointSpec.scala +15 -0
  45. data/src/test/scala/s3/website/S3WebsiteSpec.scala +1480 -0
  46. data/src/test/scala/s3/website/UnitTest.scala +11 -0
  47. data/vagrant/Vagrantfile +25 -0
  48. metadata +195 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 92d5c603441b2716994cbd0f628b598c2147ffc437bc4d77daf0a6f9a1d10a26
4
+ data.tar.gz: a577ae2c664baec4d88511884da7213afae5f1cc613efb408118604da37b53b0
5
+ SHA512:
6
+ metadata.gz: b7df4dff8a63d83dced5988e637c8e6c5337d6c6555ce19ea14b203435d0a7b8d7ed1535bc91a90123bc693397761e91414ea15021b3cd777bf742f7193eeccd
7
+ data.tar.gz: 42da54ada02c1b1c526523f22b56bc3c4197ad3b48d1fcfb485716b025b60ad05751c47890f6e452371f35f52f95e4721776ec698f3e7f100f3e568cf2dead0c
data/.gitignore ADDED
@@ -0,0 +1,15 @@
1
+ .rvmrc
2
+ *.gem
3
+ .bundle
4
+ pkg/*
5
+ tmp
6
+ Gemfile.lock
7
+
8
+ .idea
9
+ target
10
+ .idea_modules
11
+ .vagrant
12
+
13
+ # The checksum is unique to each build, and we don't store builds in git.
14
+ # Consequently, ignore the checksum file.
15
+ resources/s3_website.jar.md5
data/.travis.yml ADDED
@@ -0,0 +1,5 @@
1
+ language: scala
2
+ scala:
3
+ 2.11.2
4
+ jdk:
5
+ - oraclejdk8
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,42 @@
1
+ Copyright (c) 2013 Lauri Lehmijoki
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy
4
+ of this software and associated documentation files (the "Software"), to deal
5
+ in the Software without restriction, including without limitation the rights
6
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7
+ copies of the Software, and to permit persons to whom the Software is
8
+ furnished to do so, subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in
11
+ all copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19
+ THE SOFTWARE.
20
+
21
+ (This software is derived from the Ruby gem jekyll-s3, which was created by
22
+ Philippe Creux. Below is the MIT license of jekyll-s3.)
23
+
24
+ Copyright (c) 2011 VersaPay, Philippe Creux.
25
+
26
+ Permission is hereby granted, free of charge, to any person obtaining a copy
27
+ of this software and associated documentation files (the "Software"), to deal
28
+ in the Software without restriction, including without limitation the rights
29
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
30
+ copies of the Software, and to permit persons to whom the Software is
31
+ furnished to do so, subject to the following conditions:
32
+
33
+ The above copyright notice and this permission notice shall be included in
34
+ all copies or substantial portions of the Software.
35
+
36
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
37
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
38
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
39
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
40
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
41
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
42
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,591 @@
1
+ # Deploy your website to S3
2
+
3
+ [![Gem Version](https://fury-badge.herokuapp.com/rb/s3_website_revived.png)](http://badge.fury.io/rb/s3_website_revived)
4
+
5
+ # This is a fork of s3_website with fixed support for Ruby 3.1 and Java 11 and 17. Feedback is welcome. Big thanks to the original author Lauri Lehmijoki!
6
+
7
+ ## What `s3_website_revived` can do for you
8
+
9
+ * Create and configure an S3 website for you
10
+ * Upload your static website to AWS S3
11
+ * [Jekyll](http://jekyllrb.com/), [Nanoc](http://nanoc.ws/), and [Middleman](https://middlemanapp.com/) are automatically supported
12
+ * Help you use AWS Cloudfront to distribute your website
13
+ * Improve page speed with HTTP cache control and gzipping
14
+ * Set HTTP redirects for your website
15
+ * (for other features, see the documentation below)
16
+
17
+ ## Install
18
+
19
+ gem install s3_website_revived
20
+
21
+ s3_website_revived needs both [Ruby](https://www.ruby-lang.org/en/downloads/)
22
+ and [Java](http://java.com) to run. (S3_website is partly written in Scala, hence the need for Java.)
23
+
24
+ ## Usage
25
+
26
+ Here's how you can get started:
27
+
28
+ * Create API credentials that have sufficient permissions to S3. More info
29
+ [here](https://github.com/ivoanjo/s3_website_revived/tree/master/additional-docs/setting-up-aws-credentials.md).
30
+ * Go to your website directory
31
+ * Run `s3_website cfg create`. This generates a configuration file called `s3_website.yml`.
32
+ * Put your AWS credentials and the S3 bucket name into the file
33
+ * Run `s3_website cfg apply`. This will configure your bucket to function as an
34
+ S3 website. If the bucket does not exist, the command will create it for you.
35
+ * Run `s3_website push` to push your website to S3. Congratulations! You are live.
36
+ * At any later time when you would like to synchronise your local website with
37
+ the S3 website, simply run `s3_website push` again.
38
+ (It will calculate the difference, update the changed files,
39
+ upload the new files and delete the obsolete files.)
40
+
41
+ ### Specifying the location of your website
42
+
43
+ S3_website will automatically discover websites in the *_site* and
44
+ *public/output* directories.
45
+
46
+ If your website is not in either of those directories, you can
47
+ point the location of your website in two ways:
48
+
49
+ 1. Add the line `site: path-to-your-website` into the `s3_website.yml` file
50
+ 2. Or, use the `--site=path-to-your-site` command-line argument
51
+
52
+ If you want to store the `s3_website.yml` file in a directory other than
53
+ the project's root you can specify the directory like so:
54
+ `s3_website push --config-dir config`.
55
+
56
+ ### Using standard AWS credentials
57
+
58
+ If you omit `s3_id` from your `s3_website.yml`, S3_website will fall back to reading from the [default AWS SDK locations](http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html). For instance, if you've used `aws configure` to set up credentials in `~/.aws/credentials`, S3_website can use these.
59
+
60
+ ### Using an AWS profile or a profile that assumes a role
61
+
62
+ If you omit `s3_id`, `s3_secret`, and `session_token` you can specify an AWS credentials profile to use via the `profile` configuration variable, eg:
63
+
64
+ profile: name_of_aws_profile
65
+
66
+ In addition, if you want this profile to assume a role before executing against S3, use the `profile_assume_role_arn` variable, eg:
67
+
68
+ profile_assume_role_arn: arn_of_role_to_assume
69
+
70
+ (Note: you have to use a regular profile with an ID and SECRET and specify the role ARN via a variable like this instead of a profile that specifies a `role_arn` as documented [here](http://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html) since it does not look like the Java SDK supports that format, yet...)
71
+
72
+ ### Using environment variables
73
+
74
+ You can use ERB in your `s3_website.yml` file which incorporates environment variables:
75
+
76
+ ```yaml
77
+ s3_id: <%= ENV['S3_ID'] %>
78
+ s3_secret: <%= ENV['S3_SECRET'] %>
79
+ s3_bucket: blog.example.com
80
+ ```
81
+
82
+ (If you are using `s3_website_revived` on an [EC2 instance with IAM
83
+ roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UsingIAM.html#UsingIAMrolesWithAmazonEC2Instances),
84
+ you can omit the `s3_id` and `s3_secret` keys in the config file.)
85
+
86
+ S3_website implements support for reading environment variables from a file using
87
+ the [dotenv](https://github.com/bkeepers/dotenv) gem. You can create a `.env` file
88
+ in the project's root directory to take advantage of this feature. Please have
89
+ a look at [dotenv's usage guide](https://github.com/bkeepers/dotenv#usage) for
90
+ syntax information.
91
+
92
+ Your `.env` file should containing the following variables:
93
+
94
+ S3_ID=FOO
95
+ S3_SECRET=BAR
96
+
97
+ S3_website will also honor environment variables named `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` (if using STS) automatically if `s3_id` is ommitted from `s3_website.yml`.
98
+
99
+ ## Project goals
100
+
101
+ * Provide a command-line interface tool for deploying and managing S3 websites
102
+ * Let the user have all the S3 website configurations in a file
103
+ * Minimise or remove the need to use the AWS Console
104
+ * Allow the user to deliver the website via CloudFront
105
+ * Automatically detect the most common static website tools, such as [Jekyll](http://jekyllrb.com/), [Nanoc](http://nanoc.ws/), and [Middleman](https://middlemanapp.com/).
106
+ * Be simple to use: require only the S3 credentials and the name of the S3
107
+ bucket
108
+ * Let the power users benefit from advanced S3 website features such as
109
+ redirects, Cache-Control headers and gzip support
110
+ * Be as fast as possible. Do in parallel all that can be done in parallel.
111
+
112
+ `s3_website` attempts to be a command-line interface tool that is easy to
113
+ understand and use. For example, `s3_website --help` should print you all the
114
+ things it can perform. Please create an issue if you think the tool is
115
+ incomprehensible or inconsistent.
116
+
117
+ ## Additional features
118
+
119
+ ### Cache Control
120
+
121
+ You can use either the setting `max_age` or `cache_control`to enable more
122
+ effective browser caching of your static assets.
123
+
124
+ #### max_age
125
+
126
+ There are two possible ways to use the option:
127
+ you can specify a single age (in seconds) like so:
128
+
129
+ ```yaml
130
+ max_age: 300
131
+ ```
132
+
133
+ Or you can specify a hash of globs, and all files matching those globs will have
134
+ the specified age:
135
+
136
+ ```yaml
137
+ max_age:
138
+ "assets/*": 6000
139
+ "*": 300
140
+ ```
141
+
142
+ After changing the `max_age` setting, push with the `--force` option.
143
+ Force-pushing allows you to update the S3 object metadata of existing files.
144
+
145
+ #### cache_control
146
+
147
+ The `cache_control` setting allows you to define an arbitrary string that s3_website_revived
148
+ will put on all the S3 objects of your website.
149
+
150
+ Here's an example:
151
+
152
+ ```yaml
153
+ cache_control: public, no-transform, max-age=1200, s-maxage=1200
154
+ ```
155
+
156
+ You can also specify a hash of globs, and all files matching those globs will have
157
+ the specified cache-control string:
158
+
159
+ ```yaml
160
+ cache_control:
161
+ "assets/*": public, max-age=3600
162
+ "*": no-cache, no-store
163
+ ```
164
+
165
+ After changing the `cache_control` setting, push with the `--force` option.
166
+ Force-pushing allows you to update the S3 object metadata of existing files.
167
+
168
+ ### Content type detection
169
+
170
+ By default, s3_website_revived automatically detects the content type of a file with the help of Apache Tika.
171
+
172
+ For some file types Tika's auto detection does not work correctly. Should this problem affect you, use the `content_type`
173
+ setting to override Tika's decision:
174
+
175
+ ```yaml
176
+ content_type:
177
+ "*.myextension": application/my-custom-type
178
+ ```
179
+
180
+ ### Gzip Compression
181
+
182
+ If you choose, you can use compress certain file types before uploading them to
183
+ S3. This is a recommended practice for maximizing page speed and minimizing
184
+ bandwidth usage.
185
+
186
+ To enable Gzip compression, simply add a `gzip` option to your `s3_website.yml`
187
+ configuration file:
188
+
189
+ ```yaml
190
+ gzip: true
191
+ ```
192
+
193
+ Note that you can additionally specify the file extensions you want to Gzip
194
+ (`.html`, `.css`, `.js`, `.ico`, and `.txt` will be compressed when `gzip: true`):
195
+
196
+ ```yaml
197
+ gzip:
198
+ - .html
199
+ - .css
200
+ - .md
201
+ ```
202
+
203
+ Remember that the extensions here are referring to the *compiled* extensions,
204
+ not the pre-processed extensions.
205
+
206
+ After changing the `gzip` setting, push with the `--force` option.
207
+
208
+ s3_website_revived will not gzip a file that is already gzipped. This is useful in the
209
+ situations where your build tools gzip a file before you invoke `s3_website push`.
210
+
211
+ ### Using non-standard AWS regions
212
+
213
+ By default, `s3_website` uses the US Standard Region. You can upload your
214
+ website to other regions by adding the setting `s3_endpoint` into the
215
+ `s3_website.yml` file.
216
+
217
+ For example, the following line in `s3_website.yml` will instruct `s3_website` to
218
+ push your site into the Tokyo region:
219
+
220
+ ```yaml
221
+ s3_endpoint: ap-northeast-1
222
+ ```
223
+
224
+ The valid `s3_endpoint` values consist of the [S3 location constraint
225
+ values](http://docs.amazonwebservices.com/general/latest/gr/rande.html#s3_region).
226
+
227
+ ### Ignoring files you want to keep on AWS
228
+
229
+ Sometimes there are files or directories you want to keep on S3, but not on
230
+ your local machine. You may define a regular expression to ignore files like so:
231
+
232
+ ```yaml
233
+ ignore_on_server: that_folder_of_stuff_i_dont_keep_locally
234
+ ```
235
+
236
+ You may also specify the values as a list:
237
+
238
+ ```yaml
239
+ ignore_on_server:
240
+ - that_folder_of_stuff_i_dont_keep_locally
241
+ - file_managed_by_somebody_else
242
+ ```
243
+
244
+ If you add the magic word `ignore_on_server: _DELETE_NOTHING_ON_THE_S3_BUCKET_`,
245
+ `s3_website push` will never delete any objects on the bucket.
246
+
247
+ ### Excluding files from upload
248
+
249
+ You can instruct `s3_website` not to push certain files:
250
+
251
+ ```yaml
252
+ exclude_from_upload: test
253
+ ```
254
+
255
+ The value can be a regex, and you can specify many of them:
256
+
257
+ ```yaml
258
+ exclude_from_upload:
259
+ - test
260
+ - (draft|secret)
261
+ ```
262
+
263
+ ### Reduced Redundancy
264
+
265
+ You can reduce the cost of hosting your blog on S3 by using Reduced Redundancy Storage:
266
+
267
+ * In `s3_website.yml`, set `s3_reduced_redundancy: true`
268
+ * All objects uploaded after this change will use the Reduced Redundancy Storage.
269
+ * If you want to change all of the files in the bucket, you can change them through the AWS console, or update the timestamp on the files before running `s3_website` again
270
+
271
+ After changing the `s3_reduced_redundancy` setting, push with the `--force`
272
+ option.
273
+
274
+ ### How to use Cloudfront to deliver your blog
275
+
276
+ It is easy to deliver your S3-based web site via Cloudfront, the CDN of Amazon.
277
+
278
+ #### Creating a new CloudFront distribution
279
+
280
+ When you run the command `s3_website cfg apply`, it will ask you whether you
281
+ want to deliver your website via CloudFront. If you answer yes, the command will
282
+ create a CloudFront distribution for you.
283
+
284
+ If you do not want to receive this prompt, or if you are running the command in a non-interactive session, you can use `s3_website cfg apply --headless` (and optionally also use `--autocreate-cloudfront-dist` if desired).
285
+
286
+ #### Using your existing CloudFront distribution
287
+
288
+ If you already have a CloudFront distribution that serves data from your website
289
+ S3 bucket, just add the following line into the file `s3_website.yml`:
290
+
291
+ ```yaml
292
+ cloudfront_distribution_id: your-dist-id
293
+ ```
294
+
295
+ Next time you run `s3_website push`, it will invalidate the items on CloudFront and
296
+ thus force the CDN system to reload the changes from your website S3 bucket.
297
+
298
+ #### Specifying custom settings for your CloudFront distribution
299
+
300
+ `s3_website` lets you define custom settings for your CloudFront distribution.
301
+
302
+ For example, like this you can define your own TTL and CNAME:
303
+
304
+ ```yaml
305
+ cloudfront_distribution_config:
306
+ default_cache_behavior:
307
+ min_ttl: <%= 60 * 60 * 24 %>
308
+ aliases:
309
+ quantity: 1
310
+ items:
311
+ CNAME: your.website.com
312
+ ```
313
+
314
+ Once you've saved the configuration into `s3_website.yml`, you can apply them by
315
+ running `s3_website cfg apply`.
316
+
317
+ #### Invalidating all CloudFront resources (wildcard invalidation)
318
+
319
+ The following setting is recommended for most users:
320
+
321
+ ```yaml
322
+ cloudfront_wildcard_invalidation: true
323
+ ```
324
+
325
+ Over time, it can reduce your AWS bill significantly. For more information, see <http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html>.
326
+
327
+ #### Invalidating root resources instead of index.htmls
328
+
329
+ By default, `s3_website push` calls the CloudFront invalidation API with the
330
+ file-name-as-it-is. This means that if your file is *article/index.html*, the
331
+ push command will call the invalidation API on the resource
332
+ *article/index.html*.
333
+
334
+ You can instruct the push command to invalidate the root resource instead of the
335
+ *index.html* resource by adding the following setting into the configuration
336
+ file:
337
+
338
+ ```yaml
339
+ cloudfront_invalidate_root: true
340
+ ```
341
+
342
+ To recap, this setting instructs s3_website_revived to invalidate the root resource
343
+ (e.g., *article/*) instead of the filename'd resource (e.g.,
344
+ *article/index.html*).
345
+
346
+ No more index.htmls in your URLs!
347
+
348
+ *Note*: If the root resource on your folder displays an error instead of the
349
+ index file, your source bucket in Cloudfront likely is pointing to the S3 Origin,
350
+ *example.com.s3.amazonaws.com*. Update the source to the S3 Website Endpoint,
351
+ *e.g. example.com.s3-website-us-east-1.amazonaws.com*, to fix this.
352
+
353
+ ### Configuring redirects on your S3 website
354
+
355
+ You can set HTTP redirects on your S3 website in three ways.
356
+
357
+ #### Exact page match for moving a single page
358
+ If a request is received matching a string e.g. /heated-towel-rail/ redirect to a page e.g. /
359
+
360
+ This kind of redirect is created in the s3_website.yml file under the ```redirects:``` section as follows:
361
+
362
+ ```yaml
363
+ redirects:
364
+ index.php: /
365
+ about.php: /about.html
366
+ music-files/promo.mp4: http://www.youtube.com/watch?v=dQw4w9WgXcQ
367
+ heated-towel-rail/index.html: /
368
+
369
+ ```
370
+
371
+ Note that the root forward slash is omitted in the requested page path and included in the redirect-to path. Note also that in the heated-towel-rail example this also matches heated-towel-rail/ since S3 appends index.html to request URLs terminated with a slash.
372
+
373
+ This redirect will be created as a 301 redirect from the first URL to the destination URL on the same server with the same http protocol.
374
+
375
+ Under the hood `s3_website_revived` creates a zero-byte index.html page for each path you want redirected with the appropriate `x-amz-website-redirect-location` value in the metadata for the object. See Amazon S3's
376
+ [`x-amz-website-redirect-location`](http://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html)
377
+ documentation for more details.
378
+
379
+ On terminology: the left value is the redirect source and the right value is the redirect
380
+ target. For example above, *about.php* is the redirect source and */about.html* the target.
381
+
382
+ If the `s3_key_prefix` setting is defined, it will be applied to the redirect
383
+ target if and only if the redirect target points to a site-local resource and
384
+ does not start with a slash. E.g., `about.php: about.html` will be translated
385
+ into `about.php: VALUE-OF-S3_KEY_PREFIX/about.html`.
386
+
387
+ #### Prefix replacement for moving a folder of pages
388
+ Common to content migrations, content pages often move from one subdirectory to another. For example if you're moving all the case studies on your site under /portfolio/work/ to /work/. In this case we use a prefix replacement such that /portfolio/work/walkjogrun/ gets 301 redirected to /work/walkjogrun/.
389
+
390
+ To do this we add a new rule to the routing_rules: section as follows:
391
+
392
+ ```
393
+ - condition:
394
+ key_prefix_equals: portfolio/work/
395
+ redirect:
396
+ protocol: https
397
+ host_name: <%= ENV['REDIRECT_DOMAIN_NAME'] %>
398
+ replace_key_prefix_with: work/
399
+ http_redirect_code: 301
400
+ ```
401
+
402
+ Here:
403
+
404
+ * ```-condition:``` indicates the start of a new rule.
405
+ * ```key_prefix_equals:``` introduces the path prefix (also without the leading / per the exact page match). Note that this prefix matches anything underneath it so every case study under that path will be handled by the subsequent redirect
406
+ * ```redirect:``` indicates the start of the redirect definition
407
+ * ```protocol:``` is optional and defaults to http.
408
+ * ```host_name:``` is optional but the default is the amazonaws.com bucket name not the actual domain name so this also effectively required for our site. In this example we use an environment variable to store the server hostname to support building to different environments. ```REDIRECT_DOMAIN_NAME``` can be configured on a CI server as well any CodePipelines responsible for building the site to different environments. If you're running locally you'll need to set ```REDIRECT_DOMAIN_NAME=local.myhostname.com```
409
+ * ```replace_key_prefix_with:``` indicates the substitution to use in place of the matched prefix. This is the only field required by `s3_website_revived`, so effectively this rule works like a replace rule e.g. replace portfolio/work with /work in the string portfolio/work/walkjogrun
410
+ * ```http_redirect_code:``` is optional and defaults to 302 Temporary redirect **which is terrible for SEO** since your content temporarily vanishes from the Google index until the response changes for the URL. This is almost never what you want. You *can* use this to temporarily redirect any content you haven't migrated to the new site yet as long as you remove or replace the 302 with a link to a permanent home. This tells Google to forget the old location of the page and use the new content at the new URL. For pages that move you'll see little if any discrepancy in Google traffic.
411
+
412
+ After adding the configuration, run the command `s3_website cfg apply` on your
413
+ command-line interface. This will apply the routing rules on your S3 bucket.
414
+
415
+ For more information on configuring redirects, see the documentation of the
416
+ [configure-s3-website](https://github.com/laurilehmijoki/configure-s3-website#configuring-redirects)
417
+ gem, which comes as a transitive dependency of the `s3_website_revived` gem. (The
418
+ command `s3_website cfg apply` internally calls the `configure-s3-website` gem.)
419
+
420
+ #### Prefix coallescing for deleting pages (or consolidating)
421
+ If you 301 redirect lots of content into one new path you're telling Google that the old pages are gone so only the destination page is important moving forward. E.g. if you had 10 services pages and consolidate them into 1 services listing page you'll lose the 10 pages uniquely optimized for different sets of keywords and retain just 1 page with no real keyword focus and hence less SEO value.
422
+
423
+ For example, we're not porting the entire set of pages under the folder /experience to the new website. Some of these pages still get traffic from either Google or inbound links so we don't want to just show a 404 content not found error. We will 301 redirect them to the most useful replacement page. In the case of /experience we don't have anything better to show than just the home page so that is how the redirect is configured.
424
+
425
+ Here's how to redirect to indicate a deleted page:
426
+
427
+ ```
428
+ - condition:
429
+ key_prefix_equals: experience/
430
+ redirect:
431
+ protocol: https
432
+ host_name: <%= ENV['REDIRECT_DOMAIN_NAME'] %>
433
+ **replace_key_with**: /
434
+ http_redirect_code: 301
435
+ ```
436
+
437
+ Note the only difference is that instead of using ```replace_key_prefix_with``` we use ```replace_key_with``` to effectively say "replace the entire path matching the prefix specfied in the condition with the new path".
438
+
439
+ #### On skipping application of redirects
440
+
441
+ If your website has a lot of redirects, you may find the following setting
442
+ helpful:
443
+
444
+ ```yaml
445
+ treat_zero_length_objects_as_redirects: true
446
+ ```
447
+
448
+ The setting allows `s3_website push` to infer whether a redirect exists on the S3 bucket.
449
+ You will experience faster `push` performance when this setting is `true`.
450
+
451
+ If this setting is enabled and you modify the `redirects` setting in
452
+ *s3_website.yml*, use `push --force` to apply the modified values.
453
+
454
+ For backward-compatibility reasons, this setting is `false` by default.
455
+
456
+ In this context, the word *object* refers to object on S3, not file-system file.
457
+
458
+ ### Specifying custom concurrency level
459
+
460
+ By default, `s3_website_revived` does 3 operations in parallel. An operation can be an
461
+ HTTP PUT operation against the S3 API, for example.
462
+
463
+ You can increase the concurrency level by adding the following setting into the
464
+ `s3_website.yml` file:
465
+
466
+ ```yaml
467
+ concurrency_level: <integer>
468
+ ```
469
+
470
+ However, because S3 throttles connections, there's an upper limit to the
471
+ level of parallelism. If you start to see end-of-file errors, decrease the
472
+ concurrency level. Conversely, if you don't experience any errors, you can
473
+ increase the concurrency level and thus benefit from faster uploads.
474
+
475
+ If you experience the "too many open files" error, either increase the amount of
476
+ maximum open files (on Unix-like systems, see `man ulimit`) or decrease the
477
+ `concurrency_level` setting.
478
+
479
+ ### Simulating deployments
480
+
481
+ You can simulate the `s3_website push` operation by adding the
482
+ `--dry-run` switch. The dry run mode will not apply any modifications on your S3
483
+ bucket or CloudFront distribution. It will merely print out what the `push`
484
+ operation would actually do if run without the dry switch.
485
+
486
+ You can use the dry run mode if you are unsure what kind of effects the `push`
487
+ operation would cause to your live website.
488
+
489
+ ### S3 website in a subdirectory of the bucket
490
+
491
+ If your S3 website shares the same S3 bucket with other applications, you can
492
+ push your website into a "subdirectory" on the bucket.
493
+
494
+ Define the subdirectory like so:
495
+
496
+ ```yaml
497
+ s3_key_prefix: your-subdirectory
498
+ ```
499
+
500
+ ### Temporary security credentials with Session Token
501
+
502
+ [AWS temporary security credentials](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html) (eg: when [assuming IAM roles](http://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html))
503
+
504
+ Usage:
505
+
506
+ ```yaml
507
+ session_token: your-token
508
+ ```
509
+
510
+ ## Migrating from v1 to v2
511
+
512
+ Please read the [release note](/changelog.md#200) on version 2. It contains
513
+ information on backward incompatible changes.
514
+
515
+ You can find the v1 branch
516
+ [here](https://github.com/laurilehmijoki/s3_website/tree/1.x). It's in
517
+ maintenance mode. This means that v1 will see only critical bugfix releases.
518
+
519
+ ## Example configurations
520
+
521
+ - [Minimal configuration](additional-docs/example-configurations.md#minimal)
522
+ - [CloudFront optimisation](additional-docs/example-configurations.md#optimised-for-speed)
523
+ - [CloudFront multiple CNAMEs](additional-docs/example-configurations.md#multiple-cnames)
524
+ - [Using redirects](additional-docs/example-configurations.md#using-redirects)
525
+
526
+ See more [example-configurations](additional-docs/example-configurations.md)
527
+
528
+ ## On security
529
+
530
+ If the source code of your website is publicly
531
+ available, ensure that the `s3_website.yml` file is in the list of ignored files.
532
+ For git users this means that the file `.gitignore` should mention the
533
+ `s3_website.yml` file.
534
+
535
+ If you use the .dotenv gem, ensure that you do not push the `.env` file to a
536
+ public git repository.
537
+
538
+ ## Known issues
539
+
540
+ Please create an issue and send a pull request if you spot any.
541
+
542
+ ## Versioning
543
+
544
+ s3_website_revived uses [Semantic Versioning](http://semver.org).
545
+
546
+ In the spirit of semantic versioning, here is the definition of public API for
547
+ s3_website_revived: Within a major version, s3_website_revived will not break
548
+ backwards-compatibility of anything that is mentioned in this README file.
549
+
550
+ ## Development
551
+
552
+ See [development](additional-docs/development.md).
553
+
554
+ ### Contributing
555
+
556
+ We (users and developers of s3_website_revived) welcome patches, pull requests and
557
+ ideas for improvement.
558
+
559
+ When sending pull requests, please accompany them with tests. Favor BDD style
560
+ in test descriptions. Use VCR-backed integration tests where possible. For
561
+ reference, you can look at the existing s3_website_revived tests.
562
+
563
+ If you are not sure how to test your pull request, you can ask the [gem owners
564
+ ](http://rubygems.org/gems/s3_website_revived) to supplement the request with tests.
565
+ However, by including proper tests, you increase the chances of your pull
566
+ request being incorporated into future releases.
567
+
568
+ ## Alternatives
569
+
570
+ * <https://pages.github.com/>
571
+ * <https://pages.gitlab.io/>
572
+
573
+ ## License
574
+
575
+ MIT. See the LICENSE file for more information.
576
+
577
+ ## Contributors
578
+
579
+ This gem was originally created by Lauri Lehmijoki. Without the valuable work of [Philippe
580
+ Creux](https://github.com/pcreux) on
581
+ [jekyll-s3](https://github.com/laurilehmijoki/jekyll-s3), this project would not
582
+ exist.
583
+
584
+ See the [Contributors](https://github.com/ivoanjo/s3_website_revived/graphs/contributors).
585
+
586
+ ## Community articles
587
+
588
+ * [Deploying websites to FTP or Amazon S3 with BitBucket Pipelines](https://www.savjee.be/2016/06/Deploying-website-to-ftp-or-amazon-s3-with-BitBucket-Pipelines/)
589
+ * [How To: Hosting on Amazon S3 with CloudFront](https://paulstamatiou.com/hosting-on-amazon-s3-with-cloudfront/)
590
+ * [Zero to HTTP/2 with AWS and Hugo](https://habd.as/zero-to-http-2-aws-hugo/)
591
+
data/Rakefile ADDED
@@ -0,0 +1,2 @@
1
+ require 'bundler'
2
+ Bundler::GemHelper.install_tasks