middleman-s3_sync 3.3.10 → 4.0.1.rc.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Changelog.md +16 -0
- data/README.md +73 -54
- data/lib/middleman-s3_sync/commands.rb +78 -24
- data/lib/middleman-s3_sync/extension.rb +4 -15
- data/lib/middleman/s3_sync.rb +3 -22
- data/lib/middleman/s3_sync/caching_policy.rb +1 -2
- data/lib/middleman/s3_sync/options.rb +1 -3
- data/lib/middleman/s3_sync/resource.rb +6 -6
- data/lib/middleman/s3_sync/status.rb +1 -1
- data/lib/middleman/s3_sync/version.rb +1 -1
- data/middleman-s3_sync.gemspec +3 -3
- data/spec/caching_policy_spec.rb +55 -95
- metadata +15 -15
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5aa620355fa36d4086923227d59faf63c45d32d1
|
4
|
+
data.tar.gz: 0eba8d4e743a9d1b73a7a6ee5ef5fcf28447cdf1
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 19f333be406383826315d40513bc14ab10cce8dfb9cc68767078caab99ed27b095336599d10f9370c5c7d8d12bd83bb2ef7d90478dbbbc339c84ff534748ab48
|
7
|
+
data.tar.gz: b9606e5f31b4537dcac23e603d75e9a532d49675348733225ee6d1d5325f9ba9a78bcfd7c61de8fdea60a8767e540a3f4869e0c8f4cb3b4d5e3124b38e25e2ef
|
data/Changelog.md
CHANGED
@@ -2,6 +2,22 @@
|
|
2
2
|
|
3
3
|
The gem that tries really hard not to push files to S3.
|
4
4
|
|
5
|
+
## v4.0.1
|
6
|
+
|
7
|
+
* Fix order of manipulator chain so that S3 Sync is always the last action
|
8
|
+
* Add --aws_access_key_id / --aws_secret_access_key as command line switches
|
9
|
+
* Add --build command line switch, which triggers a build before syncing
|
10
|
+
* Bump dependency version of middleman-core to 4.0.0.rc.1
|
11
|
+
|
12
|
+
## v4.0.0
|
13
|
+
|
14
|
+
* Initial support for Middleman v4
|
15
|
+
* Fix errors related to ANSI format on Windows
|
16
|
+
* Fix null reference errors in resource class
|
17
|
+
* Add --environment and --instrument command line switches
|
18
|
+
* Remove duplicate extension registration
|
19
|
+
* Remove after_s3_sync hook as hooks are no longer supported in Middleman v4
|
20
|
+
|
5
21
|
## v3.0.22
|
6
22
|
|
7
23
|
* Fixes a bug where files were not closed, leading to an exhaustion of
|
data/README.md
CHANGED
@@ -6,7 +6,7 @@ This gem determines which files need to be added, updated and optionally deleted
|
|
6
6
|
and only transfer these files up. This reduces the impact of an update
|
7
7
|
on a web site hosted on S3.
|
8
8
|
|
9
|
-
|
9
|
+
#### Why not Middleman Sync?
|
10
10
|
|
11
11
|
[Middleman Sync](https://github.com/karlfreeman/middleman-sync) does a
|
12
12
|
great job to push [Middleman](http://middlemanapp.com) generated
|
@@ -14,6 +14,11 @@ websites to S3. The only issue I have with it is that it pushes
|
|
14
14
|
every files under build to S3 and doesn't seem to properly delete files
|
15
15
|
that are no longer needed.
|
16
16
|
|
17
|
+
#### Version Support
|
18
|
+
|
19
|
+
* Use middleman-s3_sync version 4.x for Middleman 4.x
|
20
|
+
* Use middleman-s3_sync version 3.x for Middleman 3.x
|
21
|
+
|
17
22
|
## Installation
|
18
23
|
|
19
24
|
Add this line to your application's Gemfile:
|
@@ -47,22 +52,20 @@ activate :s3_sync do |s3_sync|
|
|
47
52
|
s3_sync.encryption = false
|
48
53
|
s3_sync.prefix = ''
|
49
54
|
s3_sync.version_bucket = false
|
50
|
-
s3_sync.index_document = 'index.html'
|
51
|
-
s3_sync.error_document = '404.html'
|
52
55
|
end
|
53
56
|
```
|
54
57
|
|
55
58
|
You can then start synchronizing files with S3 through ```middleman s3_sync```.
|
56
59
|
|
57
|
-
|
60
|
+
#### Configuration Defaults
|
58
61
|
|
59
62
|
The following defaults apply to the configuration items:
|
60
63
|
|
61
64
|
| Setting | Default |
|
62
65
|
| ----------------- | ---------------------------- |
|
63
|
-
| aws_access_key_id |
|
64
|
-
| aws_secret_access_key |
|
65
|
-
| bucket |
|
66
|
+
| aws_access_key_id | - |
|
67
|
+
| aws_secret_access_key | - |
|
68
|
+
| bucket | - |
|
66
69
|
| delete | ```true``` |
|
67
70
|
| after_build | ```false``` |
|
68
71
|
| prefer_gzip | ```true``` |
|
@@ -72,20 +75,11 @@ The following defaults apply to the configuration items:
|
|
72
75
|
| acl | ```'public-read'``` |
|
73
76
|
| version_bucket | ```false``` |
|
74
77
|
|
75
|
-
|
76
|
-
simplify the configuration of the extension:
|
77
|
-
|
78
|
-
```ruby
|
79
|
-
activate :s3_sync do |s3_sync|
|
80
|
-
s3_sync.bucket = 'my.bucket.com'
|
81
|
-
end
|
82
|
-
```
|
83
|
-
|
84
|
-
### Providing AWS Credentials
|
78
|
+
## Setting AWS Credentials
|
85
79
|
|
86
|
-
There are
|
80
|
+
There are several ways to provide the AWS credentials for s3_sync:
|
87
81
|
|
88
|
-
#### Through
|
82
|
+
#### Through `config.rb`
|
89
83
|
|
90
84
|
You can set the aws_access_key_id and aws_secret_access_key in the block
|
91
85
|
that is passed to the activate method.
|
@@ -93,20 +87,26 @@ that is passed to the activate method.
|
|
93
87
|
> I strongly discourage using this method. This will lead you to add and commit these changes
|
94
88
|
> to your SCM and potentially expose sensitive information to the world.
|
95
89
|
|
96
|
-
#### Through
|
90
|
+
#### Through `.s3_sync` File
|
97
91
|
|
98
|
-
You can create a
|
92
|
+
You can create a `.s3_sync` at the root of your middleman project.
|
99
93
|
The credentials are passed in the YAML format. The keys match the
|
100
94
|
options keys.
|
101
95
|
|
102
96
|
The .s3_sync file takes precedence to the configuration passed in the
|
103
97
|
activate method.
|
104
98
|
|
105
|
-
A sample
|
99
|
+
A sample `.s3_sync` file is included at the root of this repo.
|
106
100
|
|
107
101
|
> Make sure to add .s3_sync to your ignore list if you choose this approach. Not doing so may expose
|
108
102
|
> credentials to the world.
|
109
103
|
|
104
|
+
#### Through the Command Line
|
105
|
+
|
106
|
+
The aws credentials can also be passed via a command line options
|
107
|
+
`--aws_access_key_id` (`-k`) and `--aws_secret_access_key` (`-s`). They should override
|
108
|
+
any other settings if specified.
|
109
|
+
|
110
110
|
#### Through Environment
|
111
111
|
|
112
112
|
You can also pass the credentials through environment variables. They
|
@@ -123,9 +123,9 @@ method or passed through the ```.s3_sync``` configuration file.
|
|
123
123
|
|
124
124
|
#### Through IAM role
|
125
125
|
|
126
|
-
Alternatively, if you are running builds on EC2 instance which has approrpiate IAM role, then you don't need to think about specifying credentials at all – they will be pulled from AWS metadata
|
126
|
+
Alternatively, if you are running builds on EC2 instance which has approrpiate IAM role, then you don't need to think about specifying credentials at all – they will be pulled from AWS metadata service.
|
127
127
|
|
128
|
-
|
128
|
+
#### IAM Policy
|
129
129
|
|
130
130
|
Here's a sample IAM policy that will allow a user to update the site
|
131
131
|
contained in a bucket named "mysite.com":
|
@@ -150,28 +150,56 @@ contained in a bucket named "mysite.com":
|
|
150
150
|
|
151
151
|
This will give full access to both the bucket and it's contents.
|
152
152
|
|
153
|
-
##
|
153
|
+
## Command Line Usage
|
154
|
+
|
155
|
+
#### Push All Content to S3
|
154
156
|
|
155
157
|
There are situations where you might need to push the files to S3. In
|
156
|
-
such case, you can pass the
|
158
|
+
such case, you can pass the `--force` (`-f`) option:
|
157
159
|
|
158
160
|
$ middleman s3_sync --force
|
159
161
|
|
160
|
-
|
162
|
+
#### Overriding the destination bucket
|
161
163
|
|
162
|
-
You can override the destination bucket using the
|
164
|
+
You can override the destination bucket using the `--bucket` (`-b`) switch.
|
163
165
|
The command is:
|
164
166
|
|
165
167
|
$ middleman s3_sync --bucket=my.new.bucket
|
166
168
|
|
167
|
-
|
169
|
+
#### Overriding the destination prefix
|
168
170
|
|
169
|
-
You can override the destination prefix using the `--prefix` switch. The
|
171
|
+
You can override the destination prefix using the `--prefix` (`-p`) switch. The
|
170
172
|
command is:
|
171
173
|
|
172
174
|
$ middleman s3_sync --prefix=my/new/prefix
|
173
175
|
|
174
|
-
|
176
|
+
#### Specify a Middleman environment
|
177
|
+
|
178
|
+
You can specify which environment to run Middleman under using the
|
179
|
+
`--environment` (`-e`) option:
|
180
|
+
|
181
|
+
$ middleman s3_sync --environment=production
|
182
|
+
|
183
|
+
#### Dry Run
|
184
|
+
|
185
|
+
You can perform a dry run to see what would be the result of a sync
|
186
|
+
operation using the `--dry_run` (`-d`) option:
|
187
|
+
|
188
|
+
$ middleman s3_sync --dry_run
|
189
|
+
|
190
|
+
#### Print instrument messages
|
191
|
+
|
192
|
+
The `--instrument` (`-i`) option will output more information about Middleman
|
193
|
+
and s3_sync.
|
194
|
+
|
195
|
+
#### Run build before synchronizing
|
196
|
+
|
197
|
+
The `--build` (`-B`) option will ensure that Middleman build is run before the
|
198
|
+
synchronization with the S3 bucket.
|
199
|
+
|
200
|
+
## AWS Configuration
|
201
|
+
|
202
|
+
#### Pushing to a folder within a bucket
|
175
203
|
|
176
204
|
You can push to a folder within an S3 bucket by adding using the prefix
|
177
205
|
option in the config block:
|
@@ -183,24 +211,24 @@ activate :s3_sync do |s3_sync|
|
|
183
211
|
end
|
184
212
|
```
|
185
213
|
|
186
|
-
|
214
|
+
#### Bucket Versioning
|
187
215
|
|
188
|
-
You can enable bucket versioning by setting the
|
216
|
+
You can enable bucket versioning by setting the `version_bucket`
|
189
217
|
setting to true within the bucket configuration.
|
190
218
|
|
191
219
|
Versioning is enabled at the bucket level, not at the object level.
|
192
220
|
|
193
221
|
You can [find out more about versioning here](https://aws.amazon.com/about-aws/whats-new/2010/02/08/versioning-feature-for-amazon-s3-now-available/).
|
194
222
|
|
195
|
-
|
223
|
+
#### HTTP Caching
|
196
224
|
|
197
|
-
By default,
|
225
|
+
By default, `middleman-s3_sync` does not set caching headers. In
|
198
226
|
general, the default settings are sufficient. However, there are
|
199
227
|
situations where you might want to set a different HTTP caching policy.
|
200
|
-
This may be very helpful if you are using the
|
228
|
+
This may be very helpful if you are using the `asset_hash`
|
201
229
|
extension.
|
202
230
|
|
203
|
-
|
231
|
+
#### Setting a policy based on the mime-type of a file
|
204
232
|
|
205
233
|
You can set a caching policy for every files that match a certain
|
206
234
|
mime-type. For example, setting max-age to 0 and kindly asking the
|
@@ -213,7 +241,7 @@ caching_policy 'text/html', max_age: 0, must_revalidate: true
|
|
213
241
|
|
214
242
|
As a result, the following ```Cache-Control``` header would be set to ```max-age:0, must-revalidate```
|
215
243
|
|
216
|
-
|
244
|
+
#### Setting a Default Policy
|
217
245
|
|
218
246
|
You can set the default policy by passing an options hash to ```default_caching_policy``` in your ```config.rb``` file after the ```activate :s3_sync ... end``` block:
|
219
247
|
|
@@ -224,7 +252,7 @@ default_caching_policy max_age:(60 * 60 * 24 * 365)
|
|
224
252
|
This will apply the policy to any file that do not have a mime-type
|
225
253
|
specific policy.
|
226
254
|
|
227
|
-
|
255
|
+
#### Caching Policies
|
228
256
|
|
229
257
|
The [Caching Tutorial](http://www.mnot.net/cache_docs/) is a great
|
230
258
|
introduction to HTTP caching. The caching policy code in this gem is
|
@@ -243,7 +271,7 @@ The following keys can be set:
|
|
243
271
|
| `must_revalidate` | boolean | `must-revalidate` | Tells the caches that they must obey any freshness information you give them about a representation. |
|
244
272
|
| `proxy_revalidate` | boolean | `proxy-revalidate` | Similar as `must-revalidate`, but only for proxies. |
|
245
273
|
|
246
|
-
|
274
|
+
#### Setting `Expires` Header
|
247
275
|
|
248
276
|
You can pass the `expires` key to the `caching_policy` and
|
249
277
|
`default_caching_policy` methods if you insist on setting the expires
|
@@ -253,7 +281,7 @@ when the resourse is set to expire.
|
|
253
281
|
> Note that the `Cache-Control` header will take precedence over the
|
254
282
|
> `Expires` header if both are present.
|
255
283
|
|
256
|
-
|
284
|
+
#### A Note About Browser Caching
|
257
285
|
|
258
286
|
Browser caching is well specified. It hasn't always been the case.
|
259
287
|
Still, even modern browsers have different behaviors if it suits it's
|
@@ -263,7 +291,7 @@ are (I'm looking at you Chrome!). Setting the `Cache-Control` or
|
|
263
291
|
that stand between them and your content will behave the way you want
|
264
292
|
them to. YMMV.
|
265
293
|
|
266
|
-
|
294
|
+
#### Path Specific Content Type
|
267
295
|
|
268
296
|
You can now set the content type of a path through the
|
269
297
|
```s3_sync.content_types``` hash. This hasi will take precendence over
|
@@ -271,7 +299,7 @@ the content type discovered by the mime_types gem. The [associated pull
|
|
271
299
|
request](https://github.com/fredjean/middleman-s3_sync/pull/87) has a
|
272
300
|
few good examples on how to use this feature.
|
273
301
|
|
274
|
-
|
302
|
+
#### ACLs
|
275
303
|
|
276
304
|
```middleman-s3_sync``` will set the resources's ACL to ```public-read``` by default. You
|
277
305
|
can specificy a different ACL via the ```acl``` configuration option.
|
@@ -288,7 +316,7 @@ The full values and their semantics are [documented on AWS's
|
|
288
316
|
documentation
|
289
317
|
site](http://docs.aws.amazon.com/AmazonS3/latest/dev/ACLOverview.html#CannedACL).
|
290
318
|
|
291
|
-
|
319
|
+
#### Encryption
|
292
320
|
|
293
321
|
You can ask Amazon to encrypt your files at rest by setting the
|
294
322
|
```encryption``` option to true. [Server side encryption is documented
|
@@ -296,7 +324,7 @@ on the AWS documentation
|
|
296
324
|
site](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html)
|
297
325
|
.
|
298
326
|
|
299
|
-
|
327
|
+
#### GZipped Content Encoding
|
300
328
|
|
301
329
|
You can set the ```prefer_gzip``` option to look for a gzipped version
|
302
330
|
of a resource. The gzipped version of the resource will be pushed to S3
|
@@ -305,15 +333,6 @@ headers will be set correctly. This will cause Amazon to serve the
|
|
305
333
|
compressed version of the resource. In order for this to work, you need to
|
306
334
|
have the `:gzip` extension activated in your `config.rb`.
|
307
335
|
|
308
|
-
#### Custom S3 Index and Error Documents
|
309
|
-
|
310
|
-
You can enable a custom [index document](http://docs.aws.amazon.com/AmazonS3/latest/dev/IndexDocumentSupport.html)
|
311
|
-
and [error document](http://docs.aws.amazon.com/AmazonS3/latest/dev/CustomErrorDocSupport.html)
|
312
|
-
settings. The ```index_document``` option tells which file name gets used as
|
313
|
-
the index document of a directory (typically, ```index.html```), while
|
314
|
-
```error_document``` specifies the document to display for 4xx errors (ie,
|
315
|
-
the 404 page).
|
316
|
-
|
317
336
|
## A Debt of Gratitude
|
318
337
|
|
319
338
|
I used Middleman Sync as a template for building a Middleman extension.
|
@@ -1,10 +1,9 @@
|
|
1
|
-
require 'middleman-
|
1
|
+
require 'middleman-cli'
|
2
2
|
|
3
3
|
module Middleman
|
4
4
|
module Cli
|
5
|
-
Base.map("sync" => "s3_sync")
|
6
5
|
|
7
|
-
class S3Sync < Thor
|
6
|
+
class S3Sync < Thor::Group
|
8
7
|
include Thor::Actions
|
9
8
|
|
10
9
|
check_unknown_options!
|
@@ -15,49 +14,104 @@ module Middleman
|
|
15
14
|
true
|
16
15
|
end
|
17
16
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
17
|
+
class_option :environment,
|
18
|
+
aliases: '-e',
|
19
|
+
type: :string,
|
20
|
+
default: ENV['MM_ENV'] || ENV['RACK_ENV'] || 'production',
|
21
|
+
desc: 'The environment to deploy.'
|
22
|
+
|
23
|
+
class_option :build,
|
24
|
+
type: :boolean,
|
25
|
+
aliases: '-B',
|
26
|
+
desc: 'Run `middleman build` before the sync step'
|
27
|
+
|
28
|
+
class_option :force,
|
29
|
+
aliases: '-f',
|
30
|
+
type: :boolean,
|
31
|
+
desc: 'Push all local files to the server.'
|
32
|
+
|
33
|
+
class_option :aws_access_key_id,
|
34
|
+
aliases: '-k',
|
35
|
+
type: :string,
|
36
|
+
desc: 'Specify aws_access_key_id, and overrides the configured value.'
|
37
|
+
|
38
|
+
class_option :aws_secret_access_key,
|
39
|
+
aliases: '-s',
|
40
|
+
type: :string,
|
41
|
+
desc: 'Specify aws_secret_access_key, and overrides the configured value.'
|
42
|
+
|
43
|
+
class_option :bucket,
|
44
|
+
aliases: '-b',
|
45
|
+
type: :string,
|
46
|
+
desc: 'Specify which bucket to use, overrides the configured bucket.'
|
47
|
+
|
48
|
+
class_option :prefix,
|
49
|
+
aliases: '-p',
|
50
|
+
type: :string,
|
51
|
+
desc: 'Specify which prefix to use, overrides the configured prefix.'
|
52
|
+
|
53
|
+
class_option :verbose,
|
54
|
+
aliases: '-v',
|
55
|
+
type: :boolean,
|
56
|
+
desc: 'Enables verbose log output.'
|
57
|
+
|
58
|
+
class_option :dry_run,
|
59
|
+
aliases: '-n',
|
60
|
+
type: :boolean,
|
61
|
+
desc: 'Performs a dry run of the sync.'
|
62
|
+
|
63
|
+
class_option :instrument,
|
64
|
+
aliases: '-i',
|
65
|
+
type: :string,
|
66
|
+
desc: 'Print instrument messages.'
|
34
67
|
|
35
68
|
def s3_sync
|
36
|
-
|
37
|
-
|
69
|
+
env = options[:environment].to_s.to_sym
|
70
|
+
verbose = options[:verbose] ? 0 : 1
|
71
|
+
instrument = options[:instrument]
|
72
|
+
|
73
|
+
::Middleman::S3Sync.app = ::Middleman::Application.new do
|
74
|
+
config[:mode] = :build
|
75
|
+
config[:environment] = env
|
76
|
+
::Middleman::Logger.singleton(verbose, instrument)
|
38
77
|
end
|
39
78
|
|
79
|
+
build(options)
|
80
|
+
|
40
81
|
s3_sync_options = ::Middleman::S3Sync.s3_sync_options
|
41
82
|
|
42
83
|
bucket = s3_sync_options.bucket rescue nil
|
43
84
|
|
44
85
|
unless bucket
|
45
|
-
raise Thor::Error.new
|
86
|
+
raise Thor::Error.new 'You must provide the bucket name.'
|
46
87
|
end
|
47
88
|
|
48
|
-
|
49
89
|
# Override options based on what was passed on the command line...
|
50
90
|
s3_sync_options.force = options[:force] if options[:force]
|
91
|
+
s3_sync_options.aws_access_key_id = options[:aws_access_key_id] if options[:aws_access_key_id]
|
92
|
+
s3_sync_options.aws_secret_access_key = options[:aws_secret_access_key] if options[:aws_secret_access_key]
|
51
93
|
s3_sync_options.bucket = options[:bucket] if options[:bucket]
|
52
94
|
s3_sync_options.verbose = options[:verbose] if options[:verbose]
|
53
95
|
if options[:prefix]
|
54
96
|
s3_sync_options.prefix = options[:prefix] if options[:prefix]
|
55
|
-
s3_sync_options.prefix = s3_sync_options.prefix.end_with?(
|
97
|
+
s3_sync_options.prefix = s3_sync_options.prefix.end_with?('/') ? s3_sync_options.prefix : s3_sync_options.prefix + '/'
|
56
98
|
end
|
57
99
|
s3_sync_options.dry_run = options[:dry_run] if options[:dry_run]
|
58
100
|
|
59
101
|
::Middleman::S3Sync.sync()
|
60
102
|
end
|
103
|
+
|
104
|
+
def build(options = {})
|
105
|
+
if options[:build]
|
106
|
+
run("middleman build -e #{options[:environment]}") || exit(1)
|
107
|
+
end
|
108
|
+
end
|
61
109
|
end
|
110
|
+
|
111
|
+
# Add to CLI
|
112
|
+
Base.register(Middleman::Cli::S3Sync, 's3_sync', 's3_sync [options]', 'Synchronizes a middleman site to an AWS S3 bucket')
|
113
|
+
|
114
|
+
# Alias "sync" to "s3_sync"
|
115
|
+
Base.map('sync' => 's3_sync')
|
62
116
|
end
|
63
117
|
end
|
@@ -3,7 +3,7 @@ require 'middleman/s3_sync'
|
|
3
3
|
require 'map'
|
4
4
|
|
5
5
|
module Middleman
|
6
|
-
class S3SyncExtension < Extension
|
6
|
+
class S3SyncExtension < ::Middleman::Extension
|
7
7
|
# Options supported by the extension...
|
8
8
|
option :prefix, nil, 'Path prefix of the resource we are looking for on the server.'
|
9
9
|
option :http_prefix, nil, 'Path prefix of the resources'
|
@@ -23,15 +23,12 @@ module Middleman
|
|
23
23
|
option :version_bucket, false, 'Whether to enable versionning on the S3 bucket content'
|
24
24
|
option :verbose, false, 'Whether to provide more verbose output'
|
25
25
|
option :dry_run, false, 'Whether to perform a dry-run'
|
26
|
-
|
27
|
-
|
28
|
-
|
26
|
+
|
27
|
+
# S3Sync must be the last action in the manipulator chain
|
28
|
+
self.resource_list_manipulator_priority = 9999
|
29
29
|
|
30
30
|
def initialize(app, options_hash = {}, &block)
|
31
31
|
super
|
32
|
-
app.define_hook :after_s3_sync
|
33
|
-
# Temporary workaround for 3.3 and 3.4.
|
34
|
-
app.send :include, ClassMethods
|
35
32
|
end
|
36
33
|
|
37
34
|
def after_configuration
|
@@ -43,7 +40,6 @@ module Middleman
|
|
43
40
|
options.build_dir ||= app.build_dir if app.respond_to? :build_dir
|
44
41
|
if options.prefix
|
45
42
|
options.prefix = options.prefix.end_with?("/") ? options.prefix : options.prefix + "/"
|
46
|
-
options.prefix = "" if options.prefix == "/"
|
47
43
|
end
|
48
44
|
::Middleman::S3Sync.s3_sync_options = s3_sync_options
|
49
45
|
end
|
@@ -89,10 +85,6 @@ module Middleman
|
|
89
85
|
::Middleman::S3SyncExtension.s3_sync_options
|
90
86
|
end
|
91
87
|
|
92
|
-
def content_types
|
93
|
-
::Middleman::S3Sync.content_types
|
94
|
-
end
|
95
|
-
|
96
88
|
def default_caching_policy(policy = {})
|
97
89
|
::Middleman::S3Sync.add_caching_policy(:default, policy)
|
98
90
|
end
|
@@ -102,7 +94,4 @@ module Middleman
|
|
102
94
|
end
|
103
95
|
end
|
104
96
|
end
|
105
|
-
|
106
|
-
::Middleman::Extensions.register(:s3_sync, S3SyncExtension)
|
107
97
|
end
|
108
|
-
|
data/lib/middleman/s3_sync.rb
CHANGED
@@ -25,6 +25,8 @@ module Middleman
|
|
25
25
|
attr_reader :app
|
26
26
|
|
27
27
|
def sync()
|
28
|
+
@app ||= ::Middleman::Application.new
|
29
|
+
|
28
30
|
say_status "Let's see if there's work to be done..."
|
29
31
|
unless work_to_be_done?
|
30
32
|
say_status "All S3 files are up to date."
|
@@ -35,17 +37,10 @@ module Middleman
|
|
35
37
|
|
36
38
|
update_bucket_versioning
|
37
39
|
|
38
|
-
update_bucket_website
|
39
|
-
|
40
40
|
ignore_resources
|
41
41
|
create_resources
|
42
42
|
update_resources
|
43
43
|
delete_resources
|
44
|
-
|
45
|
-
app.run_hook :after_s3_sync, ignored: files_to_ignore.map(&:path),
|
46
|
-
created: files_to_create.map(&:path),
|
47
|
-
updated: files_to_update.map(&:path),
|
48
|
-
deleted: files_to_delete.map(&:path)
|
49
44
|
end
|
50
45
|
|
51
46
|
def bucket
|
@@ -80,21 +75,6 @@ module Middleman
|
|
80
75
|
connection.put_bucket_versioning(s3_sync_options.bucket, "Enabled") if s3_sync_options.version_bucket
|
81
76
|
end
|
82
77
|
|
83
|
-
def update_bucket_website
|
84
|
-
opts = {}
|
85
|
-
opts[:IndexDocument] = s3_sync_options.index_document if s3_sync_options.index_document
|
86
|
-
opts[:ErrorDocument] = s3_sync_options.error_document if s3_sync_options.error_document
|
87
|
-
|
88
|
-
if opts[:ErrorDocument] && !opts[:IndexDocument]
|
89
|
-
raise 'S3 requires `index_document` if `error_document` is specified'
|
90
|
-
end
|
91
|
-
|
92
|
-
unless opts.empty?
|
93
|
-
say_status "Putting bucket website: #{opts.to_json}"
|
94
|
-
connection.put_bucket_website(s3_sync_options.bucket, opts)
|
95
|
-
end
|
96
|
-
end
|
97
|
-
|
98
78
|
def connection
|
99
79
|
connection_options = {
|
100
80
|
:region => s3_sync_options.region,
|
@@ -205,3 +185,4 @@ module Middleman
|
|
205
185
|
end
|
206
186
|
end
|
207
187
|
end
|
188
|
+
|
@@ -8,8 +8,7 @@ module Middleman
|
|
8
8
|
end
|
9
9
|
|
10
10
|
def caching_policy_for(content_type)
|
11
|
-
|
12
|
-
caching_policies.fetch(content_type.to_s.split(';').first.strip, caching_policies[:default])
|
11
|
+
caching_policies.fetch(content_type.to_s, caching_policies[:default])
|
13
12
|
end
|
14
13
|
|
15
14
|
def default_caching_policy
|
@@ -58,7 +58,7 @@ module Middleman
|
|
58
58
|
|
59
59
|
def update!
|
60
60
|
local_content { |body|
|
61
|
-
say_status ANSI.blue{"Updating"}
|
61
|
+
say_status "#{ANSI.blue{"Updating"}} #{remote_path}#{ gzipped ? ANSI.white {' (gzipped)'} : ''}"
|
62
62
|
s3_resource.merge_attributes(to_h)
|
63
63
|
s3_resource.body = body
|
64
64
|
|
@@ -76,12 +76,12 @@ module Middleman
|
|
76
76
|
end
|
77
77
|
|
78
78
|
def destroy!
|
79
|
-
say_status ANSI.red
|
79
|
+
say_status "#{ANSI.red{"Deleting"}} #{remote_path}"
|
80
80
|
bucket.files.destroy remote_path unless options.dry_run
|
81
81
|
end
|
82
82
|
|
83
83
|
def create!
|
84
|
-
say_status ANSI.green
|
84
|
+
say_status "#{ANSI.green{"Creating"}} #{remote_path}#{ gzipped ? ANSI.white {' (gzipped)'} : ''}"
|
85
85
|
local_content { |body|
|
86
86
|
bucket.files.create(to_h.merge(body: body)) unless options.dry_run
|
87
87
|
}
|
@@ -94,7 +94,7 @@ module Middleman
|
|
94
94
|
elsif directory?
|
95
95
|
:directory
|
96
96
|
end
|
97
|
-
say_status ANSI.yellow
|
97
|
+
say_status "#{ANSI.yellow{"Ignoring"}} #{remote_path} #{ reason ? ANSI.white {"(#{reason})" } : "" }"
|
98
98
|
end
|
99
99
|
end
|
100
100
|
|
@@ -174,7 +174,7 @@ module Middleman
|
|
174
174
|
end
|
175
175
|
|
176
176
|
def redirect?
|
177
|
-
full_s3_resource.metadata.has_key?('x-amz-website-redirect-location')
|
177
|
+
full_s3_resource && full_s3_resource.metadata.has_key?('x-amz-website-redirect-location')
|
178
178
|
end
|
179
179
|
|
180
180
|
def directory?
|
@@ -232,7 +232,7 @@ module Middleman
|
|
232
232
|
end
|
233
233
|
|
234
234
|
def build_dir
|
235
|
-
options.build_dir
|
235
|
+
options.build_dir || 'build'
|
236
236
|
end
|
237
237
|
|
238
238
|
def options
|
data/middleman-s3_sync.gemspec
CHANGED
@@ -18,10 +18,10 @@ Gem::Specification.new do |gem|
|
|
18
18
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
19
19
|
gem.require_paths = ["lib"]
|
20
20
|
|
21
|
-
gem.add_runtime_dependency 'middleman-core', '
|
21
|
+
gem.add_runtime_dependency 'middleman-core', '>= 4.0.0.rc.1'
|
22
|
+
gem.add_runtime_dependency 'middleman-cli'
|
22
23
|
gem.add_runtime_dependency 'unf'
|
23
|
-
gem.add_runtime_dependency '
|
24
|
-
gem.add_runtime_dependency 'fog-aws', '>= 0.7.0'
|
24
|
+
gem.add_runtime_dependency 'fog-aws', '>= 0.1.1'
|
25
25
|
gem.add_runtime_dependency 'map'
|
26
26
|
gem.add_runtime_dependency 'parallel'
|
27
27
|
gem.add_runtime_dependency 'ruby-progressbar'
|
data/spec/caching_policy_spec.rb
CHANGED
@@ -2,119 +2,79 @@ require 'spec_helper'
|
|
2
2
|
require 'middleman/s3_sync/caching_policy'
|
3
3
|
|
4
4
|
describe Middleman::S3Sync::BrowserCachePolicy do
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
end
|
22
|
-
|
23
|
-
context "setting max-age" do
|
24
|
-
let(:options) { { max_age: 300 } }
|
25
|
-
|
26
|
-
its(:to_s) { should =~ /max-age=300/ }
|
27
|
-
end
|
28
|
-
|
29
|
-
context "setting s-maxage" do
|
30
|
-
let(:options) { { s_maxage: 300 } }
|
31
|
-
|
32
|
-
its(:to_s) { should =~ /s-maxage=300/ }
|
33
|
-
end
|
34
|
-
|
35
|
-
context "set public flag" do
|
36
|
-
let(:options) { { public: true } }
|
37
|
-
its(:to_s) { should =~ /public/ }
|
38
|
-
end
|
39
|
-
|
40
|
-
context "it should set the private flag if it is set to true" do
|
41
|
-
let(:options) { { private: true } }
|
42
|
-
its(:to_s) { should =~ /private/ }
|
43
|
-
end
|
44
|
-
|
45
|
-
context "it should set the no-cache flag when set property" do
|
46
|
-
let(:options) { { no_cache: true }}
|
47
|
-
its(:to_s) { should =~ /no-cache/ }
|
48
|
-
end
|
49
|
-
|
50
|
-
context "setting the no-store flag" do
|
51
|
-
let(:options) { { no_store: true } }
|
52
|
-
its(:to_s) { should =~ /no-store/ }
|
53
|
-
end
|
54
|
-
|
55
|
-
context "setting the must-revalidate policy" do
|
56
|
-
let(:options) { { must_revalidate: true } }
|
57
|
-
its(:to_s) { should =~ /must-revalidate/ }
|
58
|
-
end
|
59
|
-
|
60
|
-
context "setting the proxy-revalidate policy" do
|
61
|
-
let(:options) { { proxy_revalidate: true } }
|
62
|
-
its(:to_s) { should =~ /proxy-revalidate/ }
|
63
|
-
end
|
5
|
+
let(:options) { Hash.new }
|
6
|
+
subject(:policy) { Middleman::S3Sync::BrowserCachePolicy.new(options) }
|
7
|
+
|
8
|
+
it "should be blank" do
|
9
|
+
policy.should_not be_nil
|
10
|
+
|
11
|
+
policy.to_s.should_not =~ /max-age=/
|
12
|
+
policy.to_s.should_not =~ /s-maxage=/
|
13
|
+
policy.to_s.should_not =~ /public/
|
14
|
+
policy.to_s.should_not =~ /private/
|
15
|
+
policy.to_s.should_not =~ /no-cache/
|
16
|
+
policy.to_s.should_not =~ /no-store/
|
17
|
+
policy.to_s.should_not =~ /must-revalidate/
|
18
|
+
policy.to_s.should_not =~ /proxy-revalidate/
|
19
|
+
policy.expires.should be_nil
|
20
|
+
end
|
64
21
|
|
65
|
-
|
66
|
-
|
22
|
+
context "setting max-age" do
|
23
|
+
let(:options) { { max_age: 300 } }
|
67
24
|
|
68
|
-
|
69
|
-
|
70
|
-
policies.length.should == 2
|
71
|
-
policies.first.should == 'max-age=300'
|
72
|
-
policies.last.should == 'public'
|
73
|
-
end
|
74
|
-
end
|
25
|
+
its(:to_s) { should =~ /max-age=300/ }
|
26
|
+
end
|
75
27
|
|
76
|
-
|
77
|
-
|
28
|
+
context "setting s-maxage" do
|
29
|
+
let(:options) { { s_maxage: 300 } }
|
78
30
|
|
79
|
-
|
80
|
-
end
|
31
|
+
its(:to_s) { should =~ /s-maxage=300/ }
|
81
32
|
end
|
82
|
-
end
|
83
33
|
|
84
|
-
|
85
|
-
|
86
|
-
|
34
|
+
context "set public flag" do
|
35
|
+
let(:options) { { public: true } }
|
36
|
+
its(:to_s) { should =~ /public/ }
|
87
37
|
end
|
88
38
|
|
89
|
-
|
90
|
-
|
39
|
+
context "it should set the private flag if it is set to true" do
|
40
|
+
let(:options) { { private: true } }
|
41
|
+
its(:to_s) { should =~ /private/ }
|
42
|
+
end
|
91
43
|
|
92
|
-
|
93
|
-
|
44
|
+
context "it should set the no-cache flag when set property" do
|
45
|
+
let(:options) { { no_cache: true }}
|
46
|
+
its(:to_s) { should =~ /no-cache/ }
|
47
|
+
end
|
94
48
|
|
95
|
-
|
96
|
-
|
49
|
+
context "setting the no-store flag" do
|
50
|
+
let(:options) { { no_store: true } }
|
51
|
+
its(:to_s) { should =~ /no-store/ }
|
97
52
|
end
|
98
|
-
end
|
99
53
|
|
100
|
-
|
101
|
-
|
102
|
-
|
54
|
+
context "setting the must-revalidate policy" do
|
55
|
+
let(:options) { { must_revalidate: true } }
|
56
|
+
its(:to_s) { should =~ /must-revalidate/ }
|
103
57
|
end
|
104
58
|
|
105
|
-
|
59
|
+
context "setting the proxy-revalidate policy" do
|
60
|
+
let(:options) { { proxy_revalidate: true } }
|
61
|
+
its(:to_s) { should =~ /proxy-revalidate/ }
|
62
|
+
end
|
106
63
|
|
107
|
-
|
108
|
-
|
64
|
+
context "divide caching policiies with a comma and a space" do
|
65
|
+
let(:options) { { :max_age => 300, :public => true } }
|
109
66
|
|
110
|
-
|
111
|
-
|
67
|
+
it "splits policies eith commans and spaces" do
|
68
|
+
policies = policy.to_s.split(/, /)
|
69
|
+
policies.length.should == 2
|
70
|
+
policies.first.should == 'max-age=300'
|
71
|
+
policies.last.should == 'public'
|
72
|
+
end
|
112
73
|
end
|
113
74
|
|
114
|
-
|
115
|
-
|
75
|
+
context "set the expiration date" do
|
76
|
+
let(:options) { { expires: 1.years.from_now } }
|
116
77
|
|
117
|
-
|
118
|
-
expect(caching_policy.caching_policy_for("").policies[:max_age]).to eq(60 * 60 * 24 * 365)
|
78
|
+
its(:expires) { should == CGI.rfc1123_date(1.year.from_now )}
|
119
79
|
end
|
120
80
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: middleman-s3_sync
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 4.0.1.rc.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Frederic Jean
|
@@ -9,24 +9,24 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2015-10-30 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: middleman-core
|
16
16
|
requirement: !ruby/object:Gem::Requirement
|
17
17
|
requirements:
|
18
|
-
- - "
|
18
|
+
- - ">="
|
19
19
|
- !ruby/object:Gem::Version
|
20
|
-
version:
|
20
|
+
version: 4.0.0.rc.1
|
21
21
|
type: :runtime
|
22
22
|
prerelease: false
|
23
23
|
version_requirements: !ruby/object:Gem::Requirement
|
24
24
|
requirements:
|
25
|
-
- - "
|
25
|
+
- - ">="
|
26
26
|
- !ruby/object:Gem::Version
|
27
|
-
version:
|
27
|
+
version: 4.0.0.rc.1
|
28
28
|
- !ruby/object:Gem::Dependency
|
29
|
-
name:
|
29
|
+
name: middleman-cli
|
30
30
|
requirement: !ruby/object:Gem::Requirement
|
31
31
|
requirements:
|
32
32
|
- - ">="
|
@@ -40,33 +40,33 @@ dependencies:
|
|
40
40
|
- !ruby/object:Gem::Version
|
41
41
|
version: '0'
|
42
42
|
- !ruby/object:Gem::Dependency
|
43
|
-
name:
|
43
|
+
name: unf
|
44
44
|
requirement: !ruby/object:Gem::Requirement
|
45
45
|
requirements:
|
46
46
|
- - ">="
|
47
47
|
- !ruby/object:Gem::Version
|
48
|
-
version: '
|
48
|
+
version: '0'
|
49
49
|
type: :runtime
|
50
50
|
prerelease: false
|
51
51
|
version_requirements: !ruby/object:Gem::Requirement
|
52
52
|
requirements:
|
53
53
|
- - ">="
|
54
54
|
- !ruby/object:Gem::Version
|
55
|
-
version: '
|
55
|
+
version: '0'
|
56
56
|
- !ruby/object:Gem::Dependency
|
57
57
|
name: fog-aws
|
58
58
|
requirement: !ruby/object:Gem::Requirement
|
59
59
|
requirements:
|
60
60
|
- - ">="
|
61
61
|
- !ruby/object:Gem::Version
|
62
|
-
version: 0.
|
62
|
+
version: 0.1.1
|
63
63
|
type: :runtime
|
64
64
|
prerelease: false
|
65
65
|
version_requirements: !ruby/object:Gem::Requirement
|
66
66
|
requirements:
|
67
67
|
- - ">="
|
68
68
|
- !ruby/object:Gem::Version
|
69
|
-
version: 0.
|
69
|
+
version: 0.1.1
|
70
70
|
- !ruby/object:Gem::Dependency
|
71
71
|
name: map
|
72
72
|
requirement: !ruby/object:Gem::Requirement
|
@@ -294,12 +294,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
294
294
|
version: '0'
|
295
295
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
296
296
|
requirements:
|
297
|
-
- - "
|
297
|
+
- - ">"
|
298
298
|
- !ruby/object:Gem::Version
|
299
|
-
version:
|
299
|
+
version: 1.3.1
|
300
300
|
requirements: []
|
301
301
|
rubyforge_project:
|
302
|
-
rubygems_version: 2.5
|
302
|
+
rubygems_version: 2.4.5
|
303
303
|
signing_key:
|
304
304
|
specification_version: 4
|
305
305
|
summary: Tries really, really hard not to push files to S3.
|