middleman-s3_sync 3.3.10 → 4.0.1.rc.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Changelog.md +16 -0
- data/README.md +73 -54
- data/lib/middleman-s3_sync/commands.rb +78 -24
- data/lib/middleman-s3_sync/extension.rb +4 -15
- data/lib/middleman/s3_sync.rb +3 -22
- data/lib/middleman/s3_sync/caching_policy.rb +1 -2
- data/lib/middleman/s3_sync/options.rb +1 -3
- data/lib/middleman/s3_sync/resource.rb +6 -6
- data/lib/middleman/s3_sync/status.rb +1 -1
- data/lib/middleman/s3_sync/version.rb +1 -1
- data/middleman-s3_sync.gemspec +3 -3
- data/spec/caching_policy_spec.rb +55 -95
- metadata +15 -15
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5aa620355fa36d4086923227d59faf63c45d32d1
|
4
|
+
data.tar.gz: 0eba8d4e743a9d1b73a7a6ee5ef5fcf28447cdf1
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 19f333be406383826315d40513bc14ab10cce8dfb9cc68767078caab99ed27b095336599d10f9370c5c7d8d12bd83bb2ef7d90478dbbbc339c84ff534748ab48
|
7
|
+
data.tar.gz: b9606e5f31b4537dcac23e603d75e9a532d49675348733225ee6d1d5325f9ba9a78bcfd7c61de8fdea60a8767e540a3f4869e0c8f4cb3b4d5e3124b38e25e2ef
|
data/Changelog.md
CHANGED
@@ -2,6 +2,22 @@
|
|
2
2
|
|
3
3
|
The gem that tries really hard not to push files to S3.
|
4
4
|
|
5
|
+
## v4.0.1
|
6
|
+
|
7
|
+
* Fix order of manipulator chain so that S3 Sync is always the last action
|
8
|
+
* Add --aws_access_key_id / --aws_secret_access_key as command line switches
|
9
|
+
* Add --build command line switch, which triggers a build before syncing
|
10
|
+
* Bump dependency version of middleman-core to 4.0.0.rc.1
|
11
|
+
|
12
|
+
## v4.0.0
|
13
|
+
|
14
|
+
* Initial support for Middleman v4
|
15
|
+
* Fix errors related to ANSI format on Windows
|
16
|
+
* Fix null reference errors in resource class
|
17
|
+
* Add --environment and --instrument command line switches
|
18
|
+
* Remove duplicate extension registration
|
19
|
+
* Remove after_s3_sync hook as hooks are no longer supported in Middleman v4
|
20
|
+
|
5
21
|
## v3.0.22
|
6
22
|
|
7
23
|
* Fixes a bug where files were not closed, leading to an exhaustion of
|
data/README.md
CHANGED
@@ -6,7 +6,7 @@ This gem determines which files need to be added, updated and optionally deleted
|
|
6
6
|
and only transfer these files up. This reduces the impact of an update
|
7
7
|
on a web site hosted on S3.
|
8
8
|
|
9
|
-
|
9
|
+
#### Why not Middleman Sync?
|
10
10
|
|
11
11
|
[Middleman Sync](https://github.com/karlfreeman/middleman-sync) does a
|
12
12
|
great job to push [Middleman](http://middlemanapp.com) generated
|
@@ -14,6 +14,11 @@ websites to S3. The only issue I have with it is that it pushes
|
|
14
14
|
every files under build to S3 and doesn't seem to properly delete files
|
15
15
|
that are no longer needed.
|
16
16
|
|
17
|
+
#### Version Support
|
18
|
+
|
19
|
+
* Use middleman-s3_sync version 4.x for Middleman 4.x
|
20
|
+
* Use middleman-s3_sync version 3.x for Middleman 3.x
|
21
|
+
|
17
22
|
## Installation
|
18
23
|
|
19
24
|
Add this line to your application's Gemfile:
|
@@ -47,22 +52,20 @@ activate :s3_sync do |s3_sync|
|
|
47
52
|
s3_sync.encryption = false
|
48
53
|
s3_sync.prefix = ''
|
49
54
|
s3_sync.version_bucket = false
|
50
|
-
s3_sync.index_document = 'index.html'
|
51
|
-
s3_sync.error_document = '404.html'
|
52
55
|
end
|
53
56
|
```
|
54
57
|
|
55
58
|
You can then start synchronizing files with S3 through ```middleman s3_sync```.
|
56
59
|
|
57
|
-
|
60
|
+
#### Configuration Defaults
|
58
61
|
|
59
62
|
The following defaults apply to the configuration items:
|
60
63
|
|
61
64
|
| Setting | Default |
|
62
65
|
| ----------------- | ---------------------------- |
|
63
|
-
| aws_access_key_id |
|
64
|
-
| aws_secret_access_key |
|
65
|
-
| bucket |
|
66
|
+
| aws_access_key_id | - |
|
67
|
+
| aws_secret_access_key | - |
|
68
|
+
| bucket | - |
|
66
69
|
| delete | ```true``` |
|
67
70
|
| after_build | ```false``` |
|
68
71
|
| prefer_gzip | ```true``` |
|
@@ -72,20 +75,11 @@ The following defaults apply to the configuration items:
|
|
72
75
|
| acl | ```'public-read'``` |
|
73
76
|
| version_bucket | ```false``` |
|
74
77
|
|
75
|
-
|
76
|
-
simplify the configuration of the extension:
|
77
|
-
|
78
|
-
```ruby
|
79
|
-
activate :s3_sync do |s3_sync|
|
80
|
-
s3_sync.bucket = 'my.bucket.com'
|
81
|
-
end
|
82
|
-
```
|
83
|
-
|
84
|
-
### Providing AWS Credentials
|
78
|
+
## Setting AWS Credentials
|
85
79
|
|
86
|
-
There are
|
80
|
+
There are several ways to provide the AWS credentials for s3_sync:
|
87
81
|
|
88
|
-
#### Through
|
82
|
+
#### Through `config.rb`
|
89
83
|
|
90
84
|
You can set the aws_access_key_id and aws_secret_access_key in the block
|
91
85
|
that is passed to the activate method.
|
@@ -93,20 +87,26 @@ that is passed to the activate method.
|
|
93
87
|
> I strongly discourage using this method. This will lead you to add and commit these changes
|
94
88
|
> to your SCM and potentially expose sensitive information to the world.
|
95
89
|
|
96
|
-
#### Through
|
90
|
+
#### Through `.s3_sync` File
|
97
91
|
|
98
|
-
You can create a
|
92
|
+
You can create a `.s3_sync` at the root of your middleman project.
|
99
93
|
The credentials are passed in the YAML format. The keys match the
|
100
94
|
options keys.
|
101
95
|
|
102
96
|
The .s3_sync file takes precedence to the configuration passed in the
|
103
97
|
activate method.
|
104
98
|
|
105
|
-
A sample
|
99
|
+
A sample `.s3_sync` file is included at the root of this repo.
|
106
100
|
|
107
101
|
> Make sure to add .s3_sync to your ignore list if you choose this approach. Not doing so may expose
|
108
102
|
> credentials to the world.
|
109
103
|
|
104
|
+
#### Through the Command Line
|
105
|
+
|
106
|
+
The aws credentials can also be passed via a command line options
|
107
|
+
`--aws_access_key_id` (`-k`) and `--aws_secret_access_key` (`-s`). They should override
|
108
|
+
any other settings if specified.
|
109
|
+
|
110
110
|
#### Through Environment
|
111
111
|
|
112
112
|
You can also pass the credentials through environment variables. They
|
@@ -123,9 +123,9 @@ method or passed through the ```.s3_sync``` configuration file.
|
|
123
123
|
|
124
124
|
#### Through IAM role
|
125
125
|
|
126
|
-
Alternatively, if you are running builds on EC2 instance which has approrpiate IAM role, then you don't need to think about specifying credentials at all – they will be pulled from AWS metadata
|
126
|
+
Alternatively, if you are running builds on EC2 instance which has approrpiate IAM role, then you don't need to think about specifying credentials at all – they will be pulled from AWS metadata service.
|
127
127
|
|
128
|
-
|
128
|
+
#### IAM Policy
|
129
129
|
|
130
130
|
Here's a sample IAM policy that will allow a user to update the site
|
131
131
|
contained in a bucket named "mysite.com":
|
@@ -150,28 +150,56 @@ contained in a bucket named "mysite.com":
|
|
150
150
|
|
151
151
|
This will give full access to both the bucket and it's contents.
|
152
152
|
|
153
|
-
##
|
153
|
+
## Command Line Usage
|
154
|
+
|
155
|
+
#### Push All Content to S3
|
154
156
|
|
155
157
|
There are situations where you might need to push the files to S3. In
|
156
|
-
such case, you can pass the
|
158
|
+
such case, you can pass the `--force` (`-f`) option:
|
157
159
|
|
158
160
|
$ middleman s3_sync --force
|
159
161
|
|
160
|
-
|
162
|
+
#### Overriding the destination bucket
|
161
163
|
|
162
|
-
You can override the destination bucket using the
|
164
|
+
You can override the destination bucket using the `--bucket` (`-b`) switch.
|
163
165
|
The command is:
|
164
166
|
|
165
167
|
$ middleman s3_sync --bucket=my.new.bucket
|
166
168
|
|
167
|
-
|
169
|
+
#### Overriding the destination prefix
|
168
170
|
|
169
|
-
You can override the destination prefix using the `--prefix` switch. The
|
171
|
+
You can override the destination prefix using the `--prefix` (`-p`) switch. The
|
170
172
|
command is:
|
171
173
|
|
172
174
|
$ middleman s3_sync --prefix=my/new/prefix
|
173
175
|
|
174
|
-
|
176
|
+
#### Specify a Middleman environment
|
177
|
+
|
178
|
+
You can specify which environment to run Middleman under using the
|
179
|
+
`--environment` (`-e`) option:
|
180
|
+
|
181
|
+
$ middleman s3_sync --environment=production
|
182
|
+
|
183
|
+
#### Dry Run
|
184
|
+
|
185
|
+
You can perform a dry run to see what would be the result of a sync
|
186
|
+
operation using the `--dry_run` (`-d`) option:
|
187
|
+
|
188
|
+
$ middleman s3_sync --dry_run
|
189
|
+
|
190
|
+
#### Print instrument messages
|
191
|
+
|
192
|
+
The `--instrument` (`-i`) option will output more information about Middleman
|
193
|
+
and s3_sync.
|
194
|
+
|
195
|
+
#### Run build before synchronizing
|
196
|
+
|
197
|
+
The `--build` (`-B`) option will ensure that Middleman build is run before the
|
198
|
+
synchronization with the S3 bucket.
|
199
|
+
|
200
|
+
## AWS Configuration
|
201
|
+
|
202
|
+
#### Pushing to a folder within a bucket
|
175
203
|
|
176
204
|
You can push to a folder within an S3 bucket by adding using the prefix
|
177
205
|
option in the config block:
|
@@ -183,24 +211,24 @@ activate :s3_sync do |s3_sync|
|
|
183
211
|
end
|
184
212
|
```
|
185
213
|
|
186
|
-
|
214
|
+
#### Bucket Versioning
|
187
215
|
|
188
|
-
You can enable bucket versioning by setting the
|
216
|
+
You can enable bucket versioning by setting the `version_bucket`
|
189
217
|
setting to true within the bucket configuration.
|
190
218
|
|
191
219
|
Versioning is enabled at the bucket level, not at the object level.
|
192
220
|
|
193
221
|
You can [find out more about versioning here](https://aws.amazon.com/about-aws/whats-new/2010/02/08/versioning-feature-for-amazon-s3-now-available/).
|
194
222
|
|
195
|
-
|
223
|
+
#### HTTP Caching
|
196
224
|
|
197
|
-
By default,
|
225
|
+
By default, `middleman-s3_sync` does not set caching headers. In
|
198
226
|
general, the default settings are sufficient. However, there are
|
199
227
|
situations where you might want to set a different HTTP caching policy.
|
200
|
-
This may be very helpful if you are using the
|
228
|
+
This may be very helpful if you are using the `asset_hash`
|
201
229
|
extension.
|
202
230
|
|
203
|
-
|
231
|
+
#### Setting a policy based on the mime-type of a file
|
204
232
|
|
205
233
|
You can set a caching policy for every files that match a certain
|
206
234
|
mime-type. For example, setting max-age to 0 and kindly asking the
|
@@ -213,7 +241,7 @@ caching_policy 'text/html', max_age: 0, must_revalidate: true
|
|
213
241
|
|
214
242
|
As a result, the following ```Cache-Control``` header would be set to ```max-age:0, must-revalidate```
|
215
243
|
|
216
|
-
|
244
|
+
#### Setting a Default Policy
|
217
245
|
|
218
246
|
You can set the default policy by passing an options hash to ```default_caching_policy``` in your ```config.rb``` file after the ```activate :s3_sync ... end``` block:
|
219
247
|
|
@@ -224,7 +252,7 @@ default_caching_policy max_age:(60 * 60 * 24 * 365)
|
|
224
252
|
This will apply the policy to any file that do not have a mime-type
|
225
253
|
specific policy.
|
226
254
|
|
227
|
-
|
255
|
+
#### Caching Policies
|
228
256
|
|
229
257
|
The [Caching Tutorial](http://www.mnot.net/cache_docs/) is a great
|
230
258
|
introduction to HTTP caching. The caching policy code in this gem is
|
@@ -243,7 +271,7 @@ The following keys can be set:
|
|
243
271
|
| `must_revalidate` | boolean | `must-revalidate` | Tells the caches that they must obey any freshness information you give them about a representation. |
|
244
272
|
| `proxy_revalidate` | boolean | `proxy-revalidate` | Similar as `must-revalidate`, but only for proxies. |
|
245
273
|
|
246
|
-
|
274
|
+
#### Setting `Expires` Header
|
247
275
|
|
248
276
|
You can pass the `expires` key to the `caching_policy` and
|
249
277
|
`default_caching_policy` methods if you insist on setting the expires
|
@@ -253,7 +281,7 @@ when the resourse is set to expire.
|
|
253
281
|
> Note that the `Cache-Control` header will take precedence over the
|
254
282
|
> `Expires` header if both are present.
|
255
283
|
|
256
|
-
|
284
|
+
#### A Note About Browser Caching
|
257
285
|
|
258
286
|
Browser caching is well specified. It hasn't always been the case.
|
259
287
|
Still, even modern browsers have different behaviors if it suits it's
|
@@ -263,7 +291,7 @@ are (I'm looking at you Chrome!). Setting the `Cache-Control` or
|
|
263
291
|
that stand between them and your content will behave the way you want
|
264
292
|
them to. YMMV.
|
265
293
|
|
266
|
-
|
294
|
+
#### Path Specific Content Type
|
267
295
|
|
268
296
|
You can now set the content type of a path through the
|
269
297
|
```s3_sync.content_types``` hash. This hasi will take precendence over
|
@@ -271,7 +299,7 @@ the content type discovered by the mime_types gem. The [associated pull
|
|
271
299
|
request](https://github.com/fredjean/middleman-s3_sync/pull/87) has a
|
272
300
|
few good examples on how to use this feature.
|
273
301
|
|
274
|
-
|
302
|
+
#### ACLs
|
275
303
|
|
276
304
|
```middleman-s3_sync``` will set the resources's ACL to ```public-read``` by default. You
|
277
305
|
can specificy a different ACL via the ```acl``` configuration option.
|
@@ -288,7 +316,7 @@ The full values and their semantics are [documented on AWS's
|
|
288
316
|
documentation
|
289
317
|
site](http://docs.aws.amazon.com/AmazonS3/latest/dev/ACLOverview.html#CannedACL).
|
290
318
|
|
291
|
-
|
319
|
+
#### Encryption
|
292
320
|
|
293
321
|
You can ask Amazon to encrypt your files at rest by setting the
|
294
322
|
```encryption``` option to true. [Server side encryption is documented
|
@@ -296,7 +324,7 @@ on the AWS documentation
|
|
296
324
|
site](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html)
|
297
325
|
.
|
298
326
|
|
299
|
-
|
327
|
+
#### GZipped Content Encoding
|
300
328
|
|
301
329
|
You can set the ```prefer_gzip``` option to look for a gzipped version
|
302
330
|
of a resource. The gzipped version of the resource will be pushed to S3
|
@@ -305,15 +333,6 @@ headers will be set correctly. This will cause Amazon to serve the
|
|
305
333
|
compressed version of the resource. In order for this to work, you need to
|
306
334
|
have the `:gzip` extension activated in your `config.rb`.
|
307
335
|
|
308
|
-
#### Custom S3 Index and Error Documents
|
309
|
-
|
310
|
-
You can enable a custom [index document](http://docs.aws.amazon.com/AmazonS3/latest/dev/IndexDocumentSupport.html)
|
311
|
-
and [error document](http://docs.aws.amazon.com/AmazonS3/latest/dev/CustomErrorDocSupport.html)
|
312
|
-
settings. The ```index_document``` option tells which file name gets used as
|
313
|
-
the index document of a directory (typically, ```index.html```), while
|
314
|
-
```error_document``` specifies the document to display for 4xx errors (ie,
|
315
|
-
the 404 page).
|
316
|
-
|
317
336
|
## A Debt of Gratitude
|
318
337
|
|
319
338
|
I used Middleman Sync as a template for building a Middleman extension.
|
@@ -1,10 +1,9 @@
|
|
1
|
-
require 'middleman-
|
1
|
+
require 'middleman-cli'
|
2
2
|
|
3
3
|
module Middleman
|
4
4
|
module Cli
|
5
|
-
Base.map("sync" => "s3_sync")
|
6
5
|
|
7
|
-
class S3Sync < Thor
|
6
|
+
class S3Sync < Thor::Group
|
8
7
|
include Thor::Actions
|
9
8
|
|
10
9
|
check_unknown_options!
|
@@ -15,49 +14,104 @@ module Middleman
|
|
15
14
|
true
|
16
15
|
end
|
17
16
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
17
|
+
class_option :environment,
|
18
|
+
aliases: '-e',
|
19
|
+
type: :string,
|
20
|
+
default: ENV['MM_ENV'] || ENV['RACK_ENV'] || 'production',
|
21
|
+
desc: 'The environment to deploy.'
|
22
|
+
|
23
|
+
class_option :build,
|
24
|
+
type: :boolean,
|
25
|
+
aliases: '-B',
|
26
|
+
desc: 'Run `middleman build` before the sync step'
|
27
|
+
|
28
|
+
class_option :force,
|
29
|
+
aliases: '-f',
|
30
|
+
type: :boolean,
|
31
|
+
desc: 'Push all local files to the server.'
|
32
|
+
|
33
|
+
class_option :aws_access_key_id,
|
34
|
+
aliases: '-k',
|
35
|
+
type: :string,
|
36
|
+
desc: 'Specify aws_access_key_id, and overrides the configured value.'
|
37
|
+
|
38
|
+
class_option :aws_secret_access_key,
|
39
|
+
aliases: '-s',
|
40
|
+
type: :string,
|
41
|
+
desc: 'Specify aws_secret_access_key, and overrides the configured value.'
|
42
|
+
|
43
|
+
class_option :bucket,
|
44
|
+
aliases: '-b',
|
45
|
+
type: :string,
|
46
|
+
desc: 'Specify which bucket to use, overrides the configured bucket.'
|
47
|
+
|
48
|
+
class_option :prefix,
|
49
|
+
aliases: '-p',
|
50
|
+
type: :string,
|
51
|
+
desc: 'Specify which prefix to use, overrides the configured prefix.'
|
52
|
+
|
53
|
+
class_option :verbose,
|
54
|
+
aliases: '-v',
|
55
|
+
type: :boolean,
|
56
|
+
desc: 'Enables verbose log output.'
|
57
|
+
|
58
|
+
class_option :dry_run,
|
59
|
+
aliases: '-n',
|
60
|
+
type: :boolean,
|
61
|
+
desc: 'Performs a dry run of the sync.'
|
62
|
+
|
63
|
+
class_option :instrument,
|
64
|
+
aliases: '-i',
|
65
|
+
type: :string,
|
66
|
+
desc: 'Print instrument messages.'
|
34
67
|
|
35
68
|
def s3_sync
|
36
|
-
|
37
|
-
|
69
|
+
env = options[:environment].to_s.to_sym
|
70
|
+
verbose = options[:verbose] ? 0 : 1
|
71
|
+
instrument = options[:instrument]
|
72
|
+
|
73
|
+
::Middleman::S3Sync.app = ::Middleman::Application.new do
|
74
|
+
config[:mode] = :build
|
75
|
+
config[:environment] = env
|
76
|
+
::Middleman::Logger.singleton(verbose, instrument)
|
38
77
|
end
|
39
78
|
|
79
|
+
build(options)
|
80
|
+
|
40
81
|
s3_sync_options = ::Middleman::S3Sync.s3_sync_options
|
41
82
|
|
42
83
|
bucket = s3_sync_options.bucket rescue nil
|
43
84
|
|
44
85
|
unless bucket
|
45
|
-
raise Thor::Error.new
|
86
|
+
raise Thor::Error.new 'You must provide the bucket name.'
|
46
87
|
end
|
47
88
|
|
48
|
-
|
49
89
|
# Override options based on what was passed on the command line...
|
50
90
|
s3_sync_options.force = options[:force] if options[:force]
|
91
|
+
s3_sync_options.aws_access_key_id = options[:aws_access_key_id] if options[:aws_access_key_id]
|
92
|
+
s3_sync_options.aws_secret_access_key = options[:aws_secret_access_key] if options[:aws_secret_access_key]
|
51
93
|
s3_sync_options.bucket = options[:bucket] if options[:bucket]
|
52
94
|
s3_sync_options.verbose = options[:verbose] if options[:verbose]
|
53
95
|
if options[:prefix]
|
54
96
|
s3_sync_options.prefix = options[:prefix] if options[:prefix]
|
55
|
-
s3_sync_options.prefix = s3_sync_options.prefix.end_with?(
|
97
|
+
s3_sync_options.prefix = s3_sync_options.prefix.end_with?('/') ? s3_sync_options.prefix : s3_sync_options.prefix + '/'
|
56
98
|
end
|
57
99
|
s3_sync_options.dry_run = options[:dry_run] if options[:dry_run]
|
58
100
|
|
59
101
|
::Middleman::S3Sync.sync()
|
60
102
|
end
|
103
|
+
|
104
|
+
def build(options = {})
|
105
|
+
if options[:build]
|
106
|
+
run("middleman build -e #{options[:environment]}") || exit(1)
|
107
|
+
end
|
108
|
+
end
|
61
109
|
end
|
110
|
+
|
111
|
+
# Add to CLI
|
112
|
+
Base.register(Middleman::Cli::S3Sync, 's3_sync', 's3_sync [options]', 'Synchronizes a middleman site to an AWS S3 bucket')
|
113
|
+
|
114
|
+
# Alias "sync" to "s3_sync"
|
115
|
+
Base.map('sync' => 's3_sync')
|
62
116
|
end
|
63
117
|
end
|
@@ -3,7 +3,7 @@ require 'middleman/s3_sync'
|
|
3
3
|
require 'map'
|
4
4
|
|
5
5
|
module Middleman
|
6
|
-
class S3SyncExtension < Extension
|
6
|
+
class S3SyncExtension < ::Middleman::Extension
|
7
7
|
# Options supported by the extension...
|
8
8
|
option :prefix, nil, 'Path prefix of the resource we are looking for on the server.'
|
9
9
|
option :http_prefix, nil, 'Path prefix of the resources'
|
@@ -23,15 +23,12 @@ module Middleman
|
|
23
23
|
option :version_bucket, false, 'Whether to enable versionning on the S3 bucket content'
|
24
24
|
option :verbose, false, 'Whether to provide more verbose output'
|
25
25
|
option :dry_run, false, 'Whether to perform a dry-run'
|
26
|
-
|
27
|
-
|
28
|
-
|
26
|
+
|
27
|
+
# S3Sync must be the last action in the manipulator chain
|
28
|
+
self.resource_list_manipulator_priority = 9999
|
29
29
|
|
30
30
|
def initialize(app, options_hash = {}, &block)
|
31
31
|
super
|
32
|
-
app.define_hook :after_s3_sync
|
33
|
-
# Temporary workaround for 3.3 and 3.4.
|
34
|
-
app.send :include, ClassMethods
|
35
32
|
end
|
36
33
|
|
37
34
|
def after_configuration
|
@@ -43,7 +40,6 @@ module Middleman
|
|
43
40
|
options.build_dir ||= app.build_dir if app.respond_to? :build_dir
|
44
41
|
if options.prefix
|
45
42
|
options.prefix = options.prefix.end_with?("/") ? options.prefix : options.prefix + "/"
|
46
|
-
options.prefix = "" if options.prefix == "/"
|
47
43
|
end
|
48
44
|
::Middleman::S3Sync.s3_sync_options = s3_sync_options
|
49
45
|
end
|
@@ -89,10 +85,6 @@ module Middleman
|
|
89
85
|
::Middleman::S3SyncExtension.s3_sync_options
|
90
86
|
end
|
91
87
|
|
92
|
-
def content_types
|
93
|
-
::Middleman::S3Sync.content_types
|
94
|
-
end
|
95
|
-
|
96
88
|
def default_caching_policy(policy = {})
|
97
89
|
::Middleman::S3Sync.add_caching_policy(:default, policy)
|
98
90
|
end
|
@@ -102,7 +94,4 @@ module Middleman
|
|
102
94
|
end
|
103
95
|
end
|
104
96
|
end
|
105
|
-
|
106
|
-
::Middleman::Extensions.register(:s3_sync, S3SyncExtension)
|
107
97
|
end
|
108
|
-
|
data/lib/middleman/s3_sync.rb
CHANGED
@@ -25,6 +25,8 @@ module Middleman
|
|
25
25
|
attr_reader :app
|
26
26
|
|
27
27
|
def sync()
|
28
|
+
@app ||= ::Middleman::Application.new
|
29
|
+
|
28
30
|
say_status "Let's see if there's work to be done..."
|
29
31
|
unless work_to_be_done?
|
30
32
|
say_status "All S3 files are up to date."
|
@@ -35,17 +37,10 @@ module Middleman
|
|
35
37
|
|
36
38
|
update_bucket_versioning
|
37
39
|
|
38
|
-
update_bucket_website
|
39
|
-
|
40
40
|
ignore_resources
|
41
41
|
create_resources
|
42
42
|
update_resources
|
43
43
|
delete_resources
|
44
|
-
|
45
|
-
app.run_hook :after_s3_sync, ignored: files_to_ignore.map(&:path),
|
46
|
-
created: files_to_create.map(&:path),
|
47
|
-
updated: files_to_update.map(&:path),
|
48
|
-
deleted: files_to_delete.map(&:path)
|
49
44
|
end
|
50
45
|
|
51
46
|
def bucket
|
@@ -80,21 +75,6 @@ module Middleman
|
|
80
75
|
connection.put_bucket_versioning(s3_sync_options.bucket, "Enabled") if s3_sync_options.version_bucket
|
81
76
|
end
|
82
77
|
|
83
|
-
def update_bucket_website
|
84
|
-
opts = {}
|
85
|
-
opts[:IndexDocument] = s3_sync_options.index_document if s3_sync_options.index_document
|
86
|
-
opts[:ErrorDocument] = s3_sync_options.error_document if s3_sync_options.error_document
|
87
|
-
|
88
|
-
if opts[:ErrorDocument] && !opts[:IndexDocument]
|
89
|
-
raise 'S3 requires `index_document` if `error_document` is specified'
|
90
|
-
end
|
91
|
-
|
92
|
-
unless opts.empty?
|
93
|
-
say_status "Putting bucket website: #{opts.to_json}"
|
94
|
-
connection.put_bucket_website(s3_sync_options.bucket, opts)
|
95
|
-
end
|
96
|
-
end
|
97
|
-
|
98
78
|
def connection
|
99
79
|
connection_options = {
|
100
80
|
:region => s3_sync_options.region,
|
@@ -205,3 +185,4 @@ module Middleman
|
|
205
185
|
end
|
206
186
|
end
|
207
187
|
end
|
188
|
+
|
@@ -8,8 +8,7 @@ module Middleman
|
|
8
8
|
end
|
9
9
|
|
10
10
|
def caching_policy_for(content_type)
|
11
|
-
|
12
|
-
caching_policies.fetch(content_type.to_s.split(';').first.strip, caching_policies[:default])
|
11
|
+
caching_policies.fetch(content_type.to_s, caching_policies[:default])
|
13
12
|
end
|
14
13
|
|
15
14
|
def default_caching_policy
|
@@ -58,7 +58,7 @@ module Middleman
|
|
58
58
|
|
59
59
|
def update!
|
60
60
|
local_content { |body|
|
61
|
-
say_status ANSI.blue{"Updating"}
|
61
|
+
say_status "#{ANSI.blue{"Updating"}} #{remote_path}#{ gzipped ? ANSI.white {' (gzipped)'} : ''}"
|
62
62
|
s3_resource.merge_attributes(to_h)
|
63
63
|
s3_resource.body = body
|
64
64
|
|
@@ -76,12 +76,12 @@ module Middleman
|
|
76
76
|
end
|
77
77
|
|
78
78
|
def destroy!
|
79
|
-
say_status ANSI.red
|
79
|
+
say_status "#{ANSI.red{"Deleting"}} #{remote_path}"
|
80
80
|
bucket.files.destroy remote_path unless options.dry_run
|
81
81
|
end
|
82
82
|
|
83
83
|
def create!
|
84
|
-
say_status ANSI.green
|
84
|
+
say_status "#{ANSI.green{"Creating"}} #{remote_path}#{ gzipped ? ANSI.white {' (gzipped)'} : ''}"
|
85
85
|
local_content { |body|
|
86
86
|
bucket.files.create(to_h.merge(body: body)) unless options.dry_run
|
87
87
|
}
|
@@ -94,7 +94,7 @@ module Middleman
|
|
94
94
|
elsif directory?
|
95
95
|
:directory
|
96
96
|
end
|
97
|
-
say_status ANSI.yellow
|
97
|
+
say_status "#{ANSI.yellow{"Ignoring"}} #{remote_path} #{ reason ? ANSI.white {"(#{reason})" } : "" }"
|
98
98
|
end
|
99
99
|
end
|
100
100
|
|
@@ -174,7 +174,7 @@ module Middleman
|
|
174
174
|
end
|
175
175
|
|
176
176
|
def redirect?
|
177
|
-
full_s3_resource.metadata.has_key?('x-amz-website-redirect-location')
|
177
|
+
full_s3_resource && full_s3_resource.metadata.has_key?('x-amz-website-redirect-location')
|
178
178
|
end
|
179
179
|
|
180
180
|
def directory?
|
@@ -232,7 +232,7 @@ module Middleman
|
|
232
232
|
end
|
233
233
|
|
234
234
|
def build_dir
|
235
|
-
options.build_dir
|
235
|
+
options.build_dir || 'build'
|
236
236
|
end
|
237
237
|
|
238
238
|
def options
|
data/middleman-s3_sync.gemspec
CHANGED
@@ -18,10 +18,10 @@ Gem::Specification.new do |gem|
|
|
18
18
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
19
19
|
gem.require_paths = ["lib"]
|
20
20
|
|
21
|
-
gem.add_runtime_dependency 'middleman-core', '
|
21
|
+
gem.add_runtime_dependency 'middleman-core', '>= 4.0.0.rc.1'
|
22
|
+
gem.add_runtime_dependency 'middleman-cli'
|
22
23
|
gem.add_runtime_dependency 'unf'
|
23
|
-
gem.add_runtime_dependency '
|
24
|
-
gem.add_runtime_dependency 'fog-aws', '>= 0.7.0'
|
24
|
+
gem.add_runtime_dependency 'fog-aws', '>= 0.1.1'
|
25
25
|
gem.add_runtime_dependency 'map'
|
26
26
|
gem.add_runtime_dependency 'parallel'
|
27
27
|
gem.add_runtime_dependency 'ruby-progressbar'
|
data/spec/caching_policy_spec.rb
CHANGED
@@ -2,119 +2,79 @@ require 'spec_helper'
|
|
2
2
|
require 'middleman/s3_sync/caching_policy'
|
3
3
|
|
4
4
|
describe Middleman::S3Sync::BrowserCachePolicy do
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
end
|
22
|
-
|
23
|
-
context "setting max-age" do
|
24
|
-
let(:options) { { max_age: 300 } }
|
25
|
-
|
26
|
-
its(:to_s) { should =~ /max-age=300/ }
|
27
|
-
end
|
28
|
-
|
29
|
-
context "setting s-maxage" do
|
30
|
-
let(:options) { { s_maxage: 300 } }
|
31
|
-
|
32
|
-
its(:to_s) { should =~ /s-maxage=300/ }
|
33
|
-
end
|
34
|
-
|
35
|
-
context "set public flag" do
|
36
|
-
let(:options) { { public: true } }
|
37
|
-
its(:to_s) { should =~ /public/ }
|
38
|
-
end
|
39
|
-
|
40
|
-
context "it should set the private flag if it is set to true" do
|
41
|
-
let(:options) { { private: true } }
|
42
|
-
its(:to_s) { should =~ /private/ }
|
43
|
-
end
|
44
|
-
|
45
|
-
context "it should set the no-cache flag when set property" do
|
46
|
-
let(:options) { { no_cache: true }}
|
47
|
-
its(:to_s) { should =~ /no-cache/ }
|
48
|
-
end
|
49
|
-
|
50
|
-
context "setting the no-store flag" do
|
51
|
-
let(:options) { { no_store: true } }
|
52
|
-
its(:to_s) { should =~ /no-store/ }
|
53
|
-
end
|
54
|
-
|
55
|
-
context "setting the must-revalidate policy" do
|
56
|
-
let(:options) { { must_revalidate: true } }
|
57
|
-
its(:to_s) { should =~ /must-revalidate/ }
|
58
|
-
end
|
59
|
-
|
60
|
-
context "setting the proxy-revalidate policy" do
|
61
|
-
let(:options) { { proxy_revalidate: true } }
|
62
|
-
its(:to_s) { should =~ /proxy-revalidate/ }
|
63
|
-
end
|
5
|
+
let(:options) { Hash.new }
|
6
|
+
subject(:policy) { Middleman::S3Sync::BrowserCachePolicy.new(options) }
|
7
|
+
|
8
|
+
it "should be blank" do
|
9
|
+
policy.should_not be_nil
|
10
|
+
|
11
|
+
policy.to_s.should_not =~ /max-age=/
|
12
|
+
policy.to_s.should_not =~ /s-maxage=/
|
13
|
+
policy.to_s.should_not =~ /public/
|
14
|
+
policy.to_s.should_not =~ /private/
|
15
|
+
policy.to_s.should_not =~ /no-cache/
|
16
|
+
policy.to_s.should_not =~ /no-store/
|
17
|
+
policy.to_s.should_not =~ /must-revalidate/
|
18
|
+
policy.to_s.should_not =~ /proxy-revalidate/
|
19
|
+
policy.expires.should be_nil
|
20
|
+
end
|
64
21
|
|
65
|
-
|
66
|
-
|
22
|
+
context "setting max-age" do
|
23
|
+
let(:options) { { max_age: 300 } }
|
67
24
|
|
68
|
-
|
69
|
-
|
70
|
-
policies.length.should == 2
|
71
|
-
policies.first.should == 'max-age=300'
|
72
|
-
policies.last.should == 'public'
|
73
|
-
end
|
74
|
-
end
|
25
|
+
its(:to_s) { should =~ /max-age=300/ }
|
26
|
+
end
|
75
27
|
|
76
|
-
|
77
|
-
|
28
|
+
context "setting s-maxage" do
|
29
|
+
let(:options) { { s_maxage: 300 } }
|
78
30
|
|
79
|
-
|
80
|
-
end
|
31
|
+
its(:to_s) { should =~ /s-maxage=300/ }
|
81
32
|
end
|
82
|
-
end
|
83
33
|
|
84
|
-
|
85
|
-
|
86
|
-
|
34
|
+
context "set public flag" do
|
35
|
+
let(:options) { { public: true } }
|
36
|
+
its(:to_s) { should =~ /public/ }
|
87
37
|
end
|
88
38
|
|
89
|
-
|
90
|
-
|
39
|
+
context "it should set the private flag if it is set to true" do
|
40
|
+
let(:options) { { private: true } }
|
41
|
+
its(:to_s) { should =~ /private/ }
|
42
|
+
end
|
91
43
|
|
92
|
-
|
93
|
-
|
44
|
+
context "it should set the no-cache flag when set property" do
|
45
|
+
let(:options) { { no_cache: true }}
|
46
|
+
its(:to_s) { should =~ /no-cache/ }
|
47
|
+
end
|
94
48
|
|
95
|
-
|
96
|
-
|
49
|
+
context "setting the no-store flag" do
|
50
|
+
let(:options) { { no_store: true } }
|
51
|
+
its(:to_s) { should =~ /no-store/ }
|
97
52
|
end
|
98
|
-
end
|
99
53
|
|
100
|
-
|
101
|
-
|
102
|
-
|
54
|
+
context "setting the must-revalidate policy" do
|
55
|
+
let(:options) { { must_revalidate: true } }
|
56
|
+
its(:to_s) { should =~ /must-revalidate/ }
|
103
57
|
end
|
104
58
|
|
105
|
-
|
59
|
+
context "setting the proxy-revalidate policy" do
|
60
|
+
let(:options) { { proxy_revalidate: true } }
|
61
|
+
its(:to_s) { should =~ /proxy-revalidate/ }
|
62
|
+
end
|
106
63
|
|
107
|
-
|
108
|
-
|
64
|
+
context "divide caching policiies with a comma and a space" do
|
65
|
+
let(:options) { { :max_age => 300, :public => true } }
|
109
66
|
|
110
|
-
|
111
|
-
|
67
|
+
it "splits policies eith commans and spaces" do
|
68
|
+
policies = policy.to_s.split(/, /)
|
69
|
+
policies.length.should == 2
|
70
|
+
policies.first.should == 'max-age=300'
|
71
|
+
policies.last.should == 'public'
|
72
|
+
end
|
112
73
|
end
|
113
74
|
|
114
|
-
|
115
|
-
|
75
|
+
context "set the expiration date" do
|
76
|
+
let(:options) { { expires: 1.years.from_now } }
|
116
77
|
|
117
|
-
|
118
|
-
expect(caching_policy.caching_policy_for("").policies[:max_age]).to eq(60 * 60 * 24 * 365)
|
78
|
+
its(:expires) { should == CGI.rfc1123_date(1.year.from_now )}
|
119
79
|
end
|
120
80
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: middleman-s3_sync
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 4.0.1.rc.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Frederic Jean
|
@@ -9,24 +9,24 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2015-10-30 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: middleman-core
|
16
16
|
requirement: !ruby/object:Gem::Requirement
|
17
17
|
requirements:
|
18
|
-
- - "
|
18
|
+
- - ">="
|
19
19
|
- !ruby/object:Gem::Version
|
20
|
-
version:
|
20
|
+
version: 4.0.0.rc.1
|
21
21
|
type: :runtime
|
22
22
|
prerelease: false
|
23
23
|
version_requirements: !ruby/object:Gem::Requirement
|
24
24
|
requirements:
|
25
|
-
- - "
|
25
|
+
- - ">="
|
26
26
|
- !ruby/object:Gem::Version
|
27
|
-
version:
|
27
|
+
version: 4.0.0.rc.1
|
28
28
|
- !ruby/object:Gem::Dependency
|
29
|
-
name:
|
29
|
+
name: middleman-cli
|
30
30
|
requirement: !ruby/object:Gem::Requirement
|
31
31
|
requirements:
|
32
32
|
- - ">="
|
@@ -40,33 +40,33 @@ dependencies:
|
|
40
40
|
- !ruby/object:Gem::Version
|
41
41
|
version: '0'
|
42
42
|
- !ruby/object:Gem::Dependency
|
43
|
-
name:
|
43
|
+
name: unf
|
44
44
|
requirement: !ruby/object:Gem::Requirement
|
45
45
|
requirements:
|
46
46
|
- - ">="
|
47
47
|
- !ruby/object:Gem::Version
|
48
|
-
version: '
|
48
|
+
version: '0'
|
49
49
|
type: :runtime
|
50
50
|
prerelease: false
|
51
51
|
version_requirements: !ruby/object:Gem::Requirement
|
52
52
|
requirements:
|
53
53
|
- - ">="
|
54
54
|
- !ruby/object:Gem::Version
|
55
|
-
version: '
|
55
|
+
version: '0'
|
56
56
|
- !ruby/object:Gem::Dependency
|
57
57
|
name: fog-aws
|
58
58
|
requirement: !ruby/object:Gem::Requirement
|
59
59
|
requirements:
|
60
60
|
- - ">="
|
61
61
|
- !ruby/object:Gem::Version
|
62
|
-
version: 0.
|
62
|
+
version: 0.1.1
|
63
63
|
type: :runtime
|
64
64
|
prerelease: false
|
65
65
|
version_requirements: !ruby/object:Gem::Requirement
|
66
66
|
requirements:
|
67
67
|
- - ">="
|
68
68
|
- !ruby/object:Gem::Version
|
69
|
-
version: 0.
|
69
|
+
version: 0.1.1
|
70
70
|
- !ruby/object:Gem::Dependency
|
71
71
|
name: map
|
72
72
|
requirement: !ruby/object:Gem::Requirement
|
@@ -294,12 +294,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
294
294
|
version: '0'
|
295
295
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
296
296
|
requirements:
|
297
|
-
- - "
|
297
|
+
- - ">"
|
298
298
|
- !ruby/object:Gem::Version
|
299
|
-
version:
|
299
|
+
version: 1.3.1
|
300
300
|
requirements: []
|
301
301
|
rubyforge_project:
|
302
|
-
rubygems_version: 2.5
|
302
|
+
rubygems_version: 2.4.5
|
303
303
|
signing_key:
|
304
304
|
specification_version: 4
|
305
305
|
summary: Tries really, really hard not to push files to S3.
|