s3_website 1.5.0 → 1.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +13 -5
- data/README.md +25 -0
- data/changelog.md +6 -0
- data/features/instructions-for-new-user.feature +4 -0
- data/features/push.feature +14 -0
- data/features/support/test_site_dirs/ignored-files.com/_site/css/styles.css +4 -0
- data/features/support/test_site_dirs/ignored-files.com/_site/index.html +8 -0
- data/features/support/test_site_dirs/ignored-files.com/s3_website.yml +5 -0
- data/lib/s3_website/diff_helper.rb +4 -4
- data/lib/s3_website/upload.rb +7 -4
- data/lib/s3_website/uploader.rb +10 -2
- data/resources/configuration_file_template.yml +6 -0
- data/s3_website.gemspec +1 -1
- data/spec/lib/upload_spec.rb +31 -1
- data/spec/lib/uploader_spec.rb +8 -1
- metadata +20 -14
checksums.yaml
CHANGED
|
@@ -1,7 +1,15 @@
|
|
|
1
1
|
---
|
|
2
|
-
|
|
3
|
-
metadata.gz:
|
|
4
|
-
|
|
2
|
+
!binary "U0hBMQ==":
|
|
3
|
+
metadata.gz: !binary |-
|
|
4
|
+
ZWVhZTNiMDNkODAzMThjMDY4MjkwNGUyOTZiNTBkODAyZGY2NDY4ZA==
|
|
5
|
+
data.tar.gz: !binary |-
|
|
6
|
+
NzIyNzNkODYxNGYzZDQxMDI3ODA2NDg2MmI0MGFlZDJkZjRlZDA1Zg==
|
|
5
7
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
|
|
8
|
+
metadata.gz: !binary |-
|
|
9
|
+
YjMyM2I2Mjg4YmJjZDdjZDM4ZTcwYjg5YThlY2M5NGM1NTFkNGE5ZTYwZDNm
|
|
10
|
+
YzFmYzM3ODE0MzhhZTNkMmI4MzcxY2YzNDIzNjM3ZjFiMmRhNGMyZmZkZWQ1
|
|
11
|
+
ZGZhNDA1NGQ3MDU4NzhjYTZlMDI4NWE1NDAwNjZjYzhjYTNhZmQ=
|
|
12
|
+
data.tar.gz: !binary |-
|
|
13
|
+
ZDFmYmYxNjc2MDdlZThmNWRhYzQ3OTE5N2ZjMjAwY2M3NDgwODg4MWQyN2U4
|
|
14
|
+
NWRjODg4NzY0ZWI4NDQ3MGJjYzNmYmFhY2I2NGY0NDQ0YTk4NzdmOTAxZWZi
|
|
15
|
+
MmRhZWRiODcxMzNjNzk2Zjk0NmYxNGE3YjI3MWJiZDlhZmUzNzI=
|
data/README.md
CHANGED
|
@@ -171,6 +171,30 @@ your local machine. You may define a regular expression to ignore files like so:
|
|
|
171
171
|
ignore_on_server: that_folder_of_stuff_i_dont_keep_locally
|
|
172
172
|
```
|
|
173
173
|
|
|
174
|
+
You may also specify the values as a list:
|
|
175
|
+
|
|
176
|
+
```yaml
|
|
177
|
+
ignore_on_server:
|
|
178
|
+
- that_folder_of_stuff_i_dont_keep_locally
|
|
179
|
+
- file_managed_by_somebody_else
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
### Excluding files from upload
|
|
183
|
+
|
|
184
|
+
You can instruct `s3_website` not to push certain files:
|
|
185
|
+
|
|
186
|
+
```yaml
|
|
187
|
+
exclude_from_upload: test
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The value can be a regex, and you can specify many of them:
|
|
191
|
+
|
|
192
|
+
```yaml
|
|
193
|
+
exclude_from_upload:
|
|
194
|
+
- test
|
|
195
|
+
- (draft|secret)
|
|
196
|
+
```
|
|
197
|
+
|
|
174
198
|
### Reduced Redundancy
|
|
175
199
|
|
|
176
200
|
You can reduce the cost of hosting your blog on S3 by using Reduced Redundancy Storage:
|
|
@@ -397,6 +421,7 @@ Contributors (in alphabetical order)
|
|
|
397
421
|
* Chris Moos
|
|
398
422
|
* David Michael Barr
|
|
399
423
|
* Greg Karékinian
|
|
424
|
+
* John Allison
|
|
400
425
|
* László Bácsi
|
|
401
426
|
* Mason Turner
|
|
402
427
|
* Michael Bleigh
|
data/changelog.md
CHANGED
|
@@ -2,6 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
This project uses [Semantic Versioning](http://semver.org).
|
|
4
4
|
|
|
5
|
+
## 1.6.0
|
|
6
|
+
|
|
7
|
+
* Add support for excluding files from upload
|
|
8
|
+
* s3_website.yml now supports `exclude_from_upload`
|
|
9
|
+
* Support multiple values on the `ignore_on_server` setting
|
|
10
|
+
|
|
5
11
|
## 1.5.0
|
|
6
12
|
|
|
7
13
|
* Add support for specifying the MIME type for extensionless files
|
|
@@ -99,6 +99,10 @@ Feature: Instructions for a new user
|
|
|
99
99
|
"""
|
|
100
100
|
# concurrency_level:
|
|
101
101
|
"""
|
|
102
|
+
Then the file "s3_website.yml" should contain:
|
|
103
|
+
"""
|
|
104
|
+
# extensionless_mime_type: text/html
|
|
105
|
+
"""
|
|
102
106
|
|
|
103
107
|
@starts-new-os-process
|
|
104
108
|
Scenario: Run s3_website with a malformed configuration file
|
data/features/push.feature
CHANGED
|
@@ -99,3 +99,17 @@ Feature: upload S3 website to S3
|
|
|
99
99
|
Done! Go visit: http://s3-website-test.net.s3-website-us-east-1.amazonaws.com/index.html
|
|
100
100
|
|
|
101
101
|
"""
|
|
102
|
+
|
|
103
|
+
@new-and-changed-files
|
|
104
|
+
Scenario: The blogger user does not want to upload certain files
|
|
105
|
+
When my S3 website is in "features/support/test_site_dirs/ignored-files.com"
|
|
106
|
+
Then s3_website will push my blog to S3
|
|
107
|
+
And the output should equal
|
|
108
|
+
"""
|
|
109
|
+
Deploying features/support/test_site_dirs/ignored-files.com/_site/* to s3-website-test.net
|
|
110
|
+
Calculating diff ... done
|
|
111
|
+
Uploading 1 changed file(s)
|
|
112
|
+
Upload css/styles.css: Success!
|
|
113
|
+
Done! Go visit: http://s3-website-test.net.s3-website-us-east-1.amazonaws.com/index.html
|
|
114
|
+
|
|
115
|
+
"""
|
|
@@ -17,16 +17,16 @@ module S3Website
|
|
|
17
17
|
s3_data_source
|
|
18
18
|
)
|
|
19
19
|
[
|
|
20
|
-
reject_blacklisted(
|
|
21
|
-
reject_blacklisted(
|
|
20
|
+
reject_blacklisted(changed_local_files, config),
|
|
21
|
+
reject_blacklisted(new_local_files, config)
|
|
22
22
|
]
|
|
23
23
|
}
|
|
24
24
|
end
|
|
25
25
|
|
|
26
26
|
private
|
|
27
27
|
|
|
28
|
-
def self.reject_blacklisted(file_paths)
|
|
29
|
-
file_paths.reject { |f| Upload.is_blacklisted
|
|
28
|
+
def self.reject_blacklisted(file_paths, config)
|
|
29
|
+
(normalise file_paths).reject { |f| Upload.is_blacklisted(f, config) }
|
|
30
30
|
end
|
|
31
31
|
|
|
32
32
|
def self.with_progress_indicator(diff_msg)
|
data/lib/s3_website/upload.rb
CHANGED
|
@@ -7,7 +7,7 @@ module S3Website
|
|
|
7
7
|
BLACKLISTED_FILES = ['s3_website.yml']
|
|
8
8
|
|
|
9
9
|
def initialize(path, s3, config, site_dir)
|
|
10
|
-
raise "May not upload #{path}, because it's blacklisted" if Upload.is_blacklisted
|
|
10
|
+
raise "May not upload #{path}, because it's blacklisted" if Upload.is_blacklisted(path, config)
|
|
11
11
|
@path = path
|
|
12
12
|
@full_path = "#{site_dir}/#{path}"
|
|
13
13
|
@file = File.open("#{site_dir}/#{path}")
|
|
@@ -25,9 +25,12 @@ module S3Website
|
|
|
25
25
|
"#{path}#{" [gzipped]" if gzip?}#{" [max-age=#{max_age}]" if cache_control?}"
|
|
26
26
|
end
|
|
27
27
|
|
|
28
|
-
def self.is_blacklisted(path)
|
|
29
|
-
|
|
30
|
-
|
|
28
|
+
def self.is_blacklisted(path, config)
|
|
29
|
+
[
|
|
30
|
+
config['exclude_from_upload'],
|
|
31
|
+
BLACKLISTED_FILES
|
|
32
|
+
].flatten.compact.any? do |blacklisted_file|
|
|
33
|
+
Regexp.new(blacklisted_file).match path
|
|
31
34
|
end
|
|
32
35
|
end
|
|
33
36
|
|
data/lib/s3_website/uploader.rb
CHANGED
|
@@ -148,9 +148,17 @@ module S3Website
|
|
|
148
148
|
end
|
|
149
149
|
|
|
150
150
|
def self.build_list_of_files_to_delete(remote_files, local_files, ignore_on_server = nil)
|
|
151
|
-
ignore_on_server = Regexp.new(ignore_on_server || "a_string_that_should_never_match_ever")
|
|
152
151
|
files_to_delete = remote_files - local_files
|
|
153
|
-
files_to_delete.reject { |file|
|
|
152
|
+
files_to_delete.reject { |file|
|
|
153
|
+
ignore_regexps(ignore_on_server).any? do |ignore_regexp|
|
|
154
|
+
Regexp.new(ignore_regexp).match file
|
|
155
|
+
end
|
|
156
|
+
}
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
def self.ignore_regexps(ignore_on_server)
|
|
160
|
+
ignore_regexps = ignore_on_server || "a_string_that_should_never_match_ever"
|
|
161
|
+
ignore_regexps.class == Array ? ignore_regexps : [ignore_regexps]
|
|
154
162
|
end
|
|
155
163
|
|
|
156
164
|
def self.delete_s3_object(s3, s3_bucket_name, s3_object_key)
|
|
@@ -18,6 +18,10 @@ s3_bucket: your.blog.bucket.com
|
|
|
18
18
|
|
|
19
19
|
# ignore_on_server: that_folder_of_stuff_i_dont_keep_locally
|
|
20
20
|
|
|
21
|
+
# exclude_from_upload:
|
|
22
|
+
# - those_folders_of_stuff
|
|
23
|
+
# - i_wouldnt_want_to_upload
|
|
24
|
+
|
|
21
25
|
# s3_reduced_redundancy: true
|
|
22
26
|
|
|
23
27
|
# cloudfront_distribution_id: your-dist-id
|
|
@@ -46,3 +50,5 @@ s3_bucket: your.blog.bucket.com
|
|
|
46
50
|
# host_name: blog.example.com
|
|
47
51
|
# replace_key_prefix_with: some_new_path/
|
|
48
52
|
# http_redirect_code: 301
|
|
53
|
+
|
|
54
|
+
# extensionless_mime_type: text/html
|
data/s3_website.gemspec
CHANGED
data/spec/lib/upload_spec.rb
CHANGED
|
@@ -8,10 +8,40 @@ describe S3Website::Upload do
|
|
|
8
8
|
it 'should fail if the upload file is s3_website.yml' do
|
|
9
9
|
blacklisted_files.each do |blacklisted_file|
|
|
10
10
|
expect {
|
|
11
|
-
S3Website::Upload.new blacklisted_file, mock(),
|
|
11
|
+
S3Website::Upload.new blacklisted_file, mock(), {}, mock()
|
|
12
12
|
}.to raise_error "May not upload #{blacklisted_file}, because it's blacklisted"
|
|
13
13
|
end
|
|
14
14
|
end
|
|
15
|
+
|
|
16
|
+
it 'should fail to upload configured blacklisted files' do
|
|
17
|
+
config = { 'exclude_from_upload' => 'vendor' }
|
|
18
|
+
|
|
19
|
+
expect {
|
|
20
|
+
S3Website::Upload.new "vendor/jquery/development.js", mock(), config, mock()
|
|
21
|
+
}.to raise_error "May not upload vendor/jquery/development.js, because it's blacklisted"
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
context 'the uploaded file matches a value in the exclude_from_upload setting' do
|
|
25
|
+
it 'should fail to upload any configured blacklisted files' do
|
|
26
|
+
config = { 'exclude_from_upload' => ['vendor', 'tests'] }
|
|
27
|
+
|
|
28
|
+
expect {
|
|
29
|
+
S3Website::Upload.new "vendor/jquery/development.js", mock(), config, mock()
|
|
30
|
+
}.to raise_error "May not upload vendor/jquery/development.js, because it's blacklisted"
|
|
31
|
+
|
|
32
|
+
expect {
|
|
33
|
+
S3Website::Upload.new "tests/spec_helper.js", mock(), config, mock()
|
|
34
|
+
}.to raise_error "May not upload tests/spec_helper.js, because it's blacklisted"
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
it 'supports regexes in the exclude_from_upload setting' do
|
|
38
|
+
config = { 'exclude_from_upload' => 'test.*' }
|
|
39
|
+
|
|
40
|
+
expect {
|
|
41
|
+
S3Website::Upload.new "tests/spec_helper.js", mock(), config, mock()
|
|
42
|
+
}.to raise_error "May not upload tests/spec_helper.js, because it's blacklisted"
|
|
43
|
+
end
|
|
44
|
+
end
|
|
15
45
|
end
|
|
16
46
|
|
|
17
47
|
describe 'reduced redundancy setting' do
|
data/spec/lib/uploader_spec.rb
CHANGED
|
@@ -17,11 +17,18 @@ describe S3Website::Uploader do
|
|
|
17
17
|
end
|
|
18
18
|
end
|
|
19
19
|
|
|
20
|
-
context "
|
|
20
|
+
context "honoring the ignore_on_server setting" do
|
|
21
21
|
it "ignores files which match a regular expression" do
|
|
22
22
|
files_to_delete = S3Website::Uploader.build_list_of_files_to_delete(["a", "b", "ignored"], ["a"], "ignored")
|
|
23
23
|
files_to_delete.should eq ["b"]
|
|
24
24
|
end
|
|
25
|
+
|
|
26
|
+
it "let's the user specify the regexes in a list" do
|
|
27
|
+
files_to_delete = S3Website::Uploader.build_list_of_files_to_delete(["a", "b", "ignored"], ["a"], ["ignored"])
|
|
28
|
+
files_to_delete.should eq ["b"]
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
|
|
25
32
|
it "does not ignore when you don't provide an ignored regex" do
|
|
26
33
|
files_to_delete = S3Website::Uploader.build_list_of_files_to_delete(["a", "b", "ignored"], ["a"])
|
|
27
34
|
files_to_delete.should eq ["b", "ignored"]
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: s3_website
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Lauri Lehmijoki
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2013-
|
|
11
|
+
date: 2013-11-22 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: aws-sdk
|
|
@@ -112,42 +112,42 @@ dependencies:
|
|
|
112
112
|
name: rspec
|
|
113
113
|
requirement: !ruby/object:Gem::Requirement
|
|
114
114
|
requirements:
|
|
115
|
-
- - '>='
|
|
115
|
+
- - ! '>='
|
|
116
116
|
- !ruby/object:Gem::Version
|
|
117
117
|
version: '0'
|
|
118
118
|
type: :development
|
|
119
119
|
prerelease: false
|
|
120
120
|
version_requirements: !ruby/object:Gem::Requirement
|
|
121
121
|
requirements:
|
|
122
|
-
- - '>='
|
|
122
|
+
- - ! '>='
|
|
123
123
|
- !ruby/object:Gem::Version
|
|
124
124
|
version: '0'
|
|
125
125
|
- !ruby/object:Gem::Dependency
|
|
126
126
|
name: rspec-expectations
|
|
127
127
|
requirement: !ruby/object:Gem::Requirement
|
|
128
128
|
requirements:
|
|
129
|
-
- - '>='
|
|
129
|
+
- - ! '>='
|
|
130
130
|
- !ruby/object:Gem::Version
|
|
131
131
|
version: '0'
|
|
132
132
|
type: :development
|
|
133
133
|
prerelease: false
|
|
134
134
|
version_requirements: !ruby/object:Gem::Requirement
|
|
135
135
|
requirements:
|
|
136
|
-
- - '>='
|
|
136
|
+
- - ! '>='
|
|
137
137
|
- !ruby/object:Gem::Version
|
|
138
138
|
version: '0'
|
|
139
139
|
- !ruby/object:Gem::Dependency
|
|
140
140
|
name: cucumber
|
|
141
141
|
requirement: !ruby/object:Gem::Requirement
|
|
142
142
|
requirements:
|
|
143
|
-
- - '>='
|
|
143
|
+
- - ! '>='
|
|
144
144
|
- !ruby/object:Gem::Version
|
|
145
145
|
version: '0'
|
|
146
146
|
type: :development
|
|
147
147
|
prerelease: false
|
|
148
148
|
version_requirements: !ruby/object:Gem::Requirement
|
|
149
149
|
requirements:
|
|
150
|
-
- - '>='
|
|
150
|
+
- - ! '>='
|
|
151
151
|
- !ruby/object:Gem::Version
|
|
152
152
|
version: '0'
|
|
153
153
|
- !ruby/object:Gem::Dependency
|
|
@@ -168,14 +168,14 @@ dependencies:
|
|
|
168
168
|
name: rake
|
|
169
169
|
requirement: !ruby/object:Gem::Requirement
|
|
170
170
|
requirements:
|
|
171
|
-
- - '>='
|
|
171
|
+
- - ! '>='
|
|
172
172
|
- !ruby/object:Gem::Version
|
|
173
173
|
version: '0'
|
|
174
174
|
type: :development
|
|
175
175
|
prerelease: false
|
|
176
176
|
version_requirements: !ruby/object:Gem::Requirement
|
|
177
177
|
requirements:
|
|
178
|
-
- - '>='
|
|
178
|
+
- - ! '>='
|
|
179
179
|
- !ruby/object:Gem::Version
|
|
180
180
|
version: '0'
|
|
181
181
|
- !ruby/object:Gem::Dependency
|
|
@@ -206,7 +206,7 @@ dependencies:
|
|
|
206
206
|
- - ~>
|
|
207
207
|
- !ruby/object:Gem::Version
|
|
208
208
|
version: 1.8.0
|
|
209
|
-
description: "\n Sync website files, set redirects, use HTTP performance optimisations,
|
|
209
|
+
description: ! "\n Sync website files, set redirects, use HTTP performance optimisations,
|
|
210
210
|
deliver via\n CloudFront.\n "
|
|
211
211
|
email:
|
|
212
212
|
- lauri.lehmijoki@iki.fi
|
|
@@ -254,6 +254,9 @@ files:
|
|
|
254
254
|
- features/support/test_site_dirs/cdn-powered.with-one-change.blog.fi/s3_website.yml
|
|
255
255
|
- features/support/test_site_dirs/create-redirects/_site/.gitkeep
|
|
256
256
|
- features/support/test_site_dirs/create-redirects/s3_website.yml
|
|
257
|
+
- features/support/test_site_dirs/ignored-files.com/_site/css/styles.css
|
|
258
|
+
- features/support/test_site_dirs/ignored-files.com/_site/index.html
|
|
259
|
+
- features/support/test_site_dirs/ignored-files.com/s3_website.yml
|
|
257
260
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/assets/picture.gif
|
|
258
261
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/css/styles.css
|
|
259
262
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/index.html
|
|
@@ -345,17 +348,17 @@ require_paths:
|
|
|
345
348
|
- lib
|
|
346
349
|
required_ruby_version: !ruby/object:Gem::Requirement
|
|
347
350
|
requirements:
|
|
348
|
-
- - '>='
|
|
351
|
+
- - ! '>='
|
|
349
352
|
- !ruby/object:Gem::Version
|
|
350
353
|
version: '0'
|
|
351
354
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
352
355
|
requirements:
|
|
353
|
-
- - '>='
|
|
356
|
+
- - ! '>='
|
|
354
357
|
- !ruby/object:Gem::Version
|
|
355
358
|
version: '0'
|
|
356
359
|
requirements: []
|
|
357
360
|
rubyforge_project:
|
|
358
|
-
rubygems_version: 2.
|
|
361
|
+
rubygems_version: 2.1.9
|
|
359
362
|
signing_key:
|
|
360
363
|
specification_version: 4
|
|
361
364
|
summary: Manage your S3 website
|
|
@@ -390,6 +393,9 @@ test_files:
|
|
|
390
393
|
- features/support/test_site_dirs/cdn-powered.with-one-change.blog.fi/s3_website.yml
|
|
391
394
|
- features/support/test_site_dirs/create-redirects/_site/.gitkeep
|
|
392
395
|
- features/support/test_site_dirs/create-redirects/s3_website.yml
|
|
396
|
+
- features/support/test_site_dirs/ignored-files.com/_site/css/styles.css
|
|
397
|
+
- features/support/test_site_dirs/ignored-files.com/_site/index.html
|
|
398
|
+
- features/support/test_site_dirs/ignored-files.com/s3_website.yml
|
|
393
399
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/assets/picture.gif
|
|
394
400
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/css/styles.css
|
|
395
401
|
- features/support/test_site_dirs/index-and-assets.blog.fi/_site/index.html
|