httpimagestore 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +115 -7
- data/VERSION +1 -1
- data/features/cache-control.feature +2 -2
- data/features/compatibility.feature +2 -2
- data/features/error-reporting.feature +2 -2
- data/features/facebook.feature +149 -0
- data/features/s3-store-and-thumbnail.feature +2 -2
- data/httpimagestore.gemspec +3 -2
- data/lib/httpimagestore/configuration/handler.rb +34 -12
- data/lib/httpimagestore/configuration/path.rb +1 -1
- data/lib/httpimagestore/configuration/s3.rb +8 -5
- data/spec/configuration_path_spec.rb +4 -4
- data/spec/configuration_s3_spec.rb +59 -0
- metadata +4 -3
data/README.md
CHANGED
@@ -14,6 +14,20 @@ It is using [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer) as
|
|
14
14
|
* storage under custom paths including image hash, content determined extension or used URL path
|
15
15
|
* based on [Unicorn HTTP server](http://unicorn.bogomips.org) with UNIX socket communication support
|
16
16
|
|
17
|
+
## Changelog
|
18
|
+
|
19
|
+
### 1.2.0
|
20
|
+
|
21
|
+
* matching for query string key and value
|
22
|
+
* getting query string key value into variable
|
23
|
+
* optional query string matchers with default value
|
24
|
+
* default values for optional component matchers
|
25
|
+
* S3 storage prefix support
|
26
|
+
|
27
|
+
### 1.1.0
|
28
|
+
|
29
|
+
* passing thumbnailer options via query string parameters
|
30
|
+
|
17
31
|
## Installing
|
18
32
|
|
19
33
|
HTTP Image Store is released as gem and can be installed from [RubyGems](http://rubygems.org) with:
|
@@ -113,13 +127,26 @@ Endpoints will be evaluated in order of definition until one is matched.
|
|
113
127
|
|
114
128
|
Statement should start with one of the following HTTP verbs in lowercase: `get`, `post`, `put`, `delete`, followed by list of URI component matchers:
|
115
129
|
|
116
|
-
*
|
117
|
-
*
|
118
|
-
*
|
119
|
-
*
|
130
|
+
* `<string>` - `<string>` literally match against URI component in that position for the endpoint to be evaluated
|
131
|
+
* `:<symbol>` - match any URI component in that position and store matched component value in variable named `<symbol>`
|
132
|
+
* `:<symbol>?[defalut]` - optionally match URI component in that position and store matched component value in variable named `<symbol>`; request URI may not contain component in that position (usually at the end of URI) to be matched for this endpoint to be evaluated; if `[default]` value is specified it will be used when no value was found in the URI, otherwise empty string will be used
|
133
|
+
* `:<symbol>/<regexp>/` - match URI component using `/` surrounded [regular expression](http://rubular.com) and store matched component value in variable named `<symbol>`
|
134
|
+
* `&<key>=<value>` - match this endpoint when query string contains key `<key>` with value of `<value>`
|
135
|
+
* `&:<key>` - match query string parameter of key `<key>` and store it's value in variable named `<key>`
|
136
|
+
* `&:<key>?[default]` - optionally match query string parameter of key `<key>`; when `[default]` is specified it will be used as variable value, otherwise empty string will be used
|
137
|
+
|
138
|
+
Note that any remaining unmatched URI is stored in `path` variable.
|
139
|
+
All query string parameters are available as variables named by their key.
|
140
|
+
Additionally `query_string_options` variable is build from query string parameters and can be used to specify options to [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer).
|
120
141
|
|
121
|
-
Note that
|
122
|
-
|
142
|
+
Note that variables can get overwritten in order of evaluation:
|
143
|
+
1. all query string parameters
|
144
|
+
2. `path` variable value
|
145
|
+
3. all matched URI components and query string parameters in order of specification from left to right
|
146
|
+
4. `query_string_options` variable value
|
147
|
+
|
148
|
+
Note that URI components are URI decoded after they are matched.
|
149
|
+
Query string parameter values are URI decoded before they are matched.
|
123
150
|
|
124
151
|
Example:
|
125
152
|
|
@@ -183,6 +210,7 @@ Options:
|
|
183
210
|
|
184
211
|
* `bucket` - name of bucket to source image from
|
185
212
|
* `path` - name of predefined path that will be used to generate key to object to source
|
213
|
+
* `prefix` - prefix object key with given prefix value; this does not affect fromat of output URL; prefix will not be included in source path output; default: ``
|
186
214
|
|
187
215
|
Example:
|
188
216
|
|
@@ -288,6 +316,7 @@ Options:
|
|
288
316
|
* `bucket` - name of bucket to store image in
|
289
317
|
* `path` - name of predefined path that will be used to generate key to store object under
|
290
318
|
* `public` - if set to `true` the image will be readable by everybody; this affects fromat of output URL; default: `false`
|
319
|
+
* `prefix` - prefix storeage key with given prefix value; this does not affect fromat of output URL; prefix will not be included in storage path output; default: ``
|
291
320
|
|
292
321
|
Example:
|
293
322
|
|
@@ -602,7 +631,7 @@ On demand API example:
|
|
602
631
|
$ curl -X PUT 10.1.1.24:3000/v1/original -q --data-binary @Pictures/compute.jpg
|
603
632
|
4006450256177f4a.jpg
|
604
633
|
|
605
|
-
#
|
634
|
+
# Getting fit operation 100x1000 thumbnail to /tmp/test.jpg
|
606
635
|
$ curl 10.1.1.24:3000/v1/thumbnail/4006450256177f4a.jpg/fit/100/1000 -v -s -o /tmp/test.jpg
|
607
636
|
* About to connect() to 10.1.1.24 port 3000 (#0)
|
608
637
|
* Trying 10.1.1.24... connected
|
@@ -653,6 +682,85 @@ $ identify /tmp/test.jpg
|
|
653
682
|
/tmp/test.jpg JPEG 100x100 100x100+0+0 8-bit sRGB 3.31KB 0.000u 0:00.000
|
654
683
|
```
|
655
684
|
|
685
|
+
## Facebook like API example
|
686
|
+
|
687
|
+
Based on [Facebook APIs](https://developers.facebook.com/docs/reference/api/using-pictures/#sizes).
|
688
|
+
|
689
|
+
```sdl
|
690
|
+
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
691
|
+
|
692
|
+
path "hash" "#{digest}"
|
693
|
+
path "path" "#{path}"
|
694
|
+
|
695
|
+
put "original" {
|
696
|
+
thumbnail "input" "original" operation="limit" width=100 height=100 format="jpeg" quality=95
|
697
|
+
|
698
|
+
store_s3 "original" bucket="mybucket_v1" path="hash"
|
699
|
+
|
700
|
+
output_store_path "original"
|
701
|
+
}
|
702
|
+
|
703
|
+
# type selected
|
704
|
+
get "&type=square" {
|
705
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
706
|
+
|
707
|
+
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
708
|
+
|
709
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
710
|
+
}
|
711
|
+
|
712
|
+
get "&type=small" {
|
713
|
+
source_s3 "original" bucket="mybucket_v1" path="path"
|
714
|
+
|
715
|
+
thumbnail "original" "thumbnail" operation="fit" width="50" height="2000" format="input"
|
716
|
+
|
717
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
718
|
+
}
|
719
|
+
|
720
|
+
get "&type=normall" {
|
721
|
+
source_s3 "original" bucket="mybucket_v1" path="path"
|
722
|
+
|
723
|
+
thumbnail "original" "thumbnail" operation="fit" width="100" height="2000" format="input"
|
724
|
+
|
725
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
726
|
+
}
|
727
|
+
|
728
|
+
get "&type=large" {
|
729
|
+
source_s3 "original" bucket="mybucket_v1" path="path"
|
730
|
+
|
731
|
+
thumbnail "original" "thumbnail" operation="fit" width="200" height="2000" format="input"
|
732
|
+
|
733
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
734
|
+
}
|
735
|
+
|
736
|
+
# crop to specified width and height
|
737
|
+
get "&:width" "&:height" {
|
738
|
+
source_s3 "original" bucket="mybucket_v1" path="path"
|
739
|
+
|
740
|
+
thumbnail "original" "thumbnail" operation="crop" width="#{width}" height="#{height}" format="input"
|
741
|
+
|
742
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
743
|
+
}
|
744
|
+
|
745
|
+
# fit to width when no height is specified
|
746
|
+
get "&:width" "&:height?1080" {
|
747
|
+
source_s3 "original" bucket="mybucket_v1" path="path"
|
748
|
+
|
749
|
+
thumbnail "original" "thumbnail" operation="fit" width="#{width}" height="#{height}" format="input"
|
750
|
+
|
751
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
752
|
+
}
|
753
|
+
|
754
|
+
# default to small class
|
755
|
+
get {
|
756
|
+
source_s3 "original" bucket="bmybucket_v1" path="path"
|
757
|
+
|
758
|
+
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
759
|
+
|
760
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
761
|
+
}
|
762
|
+
```
|
763
|
+
|
656
764
|
## Usage
|
657
765
|
|
658
766
|
After installation of the gem the `httpimagestore` executable is installed in **PATH**.
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
1.
|
1
|
+
1.2.0
|
@@ -3,7 +3,8 @@ Feature: S3 object Cache-Control header settings
|
|
3
3
|
|
4
4
|
Background:
|
5
5
|
Given S3 settings in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_TEST_BUCKET environment variables
|
6
|
-
Given
|
6
|
+
Given httpthumbnailer server is running at http://localhost:3100/health_check
|
7
|
+
Given httpimagestore server is running at http://localhost:3000/health_check with the following configuration
|
7
8
|
"""
|
8
9
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
9
10
|
|
@@ -21,7 +22,6 @@ Feature: S3 object Cache-Control header settings
|
|
21
22
|
store_s3 "cache" bucket="@AWS_S3_TEST_BUCKET@" public=true path="hash-name" cache-control="public, max-age=31557600, s-maxage=0"
|
22
23
|
}
|
23
24
|
"""
|
24
|
-
Given httpthumbnailer server is running at http://localhost:3100/
|
25
25
|
|
26
26
|
@cache-control
|
27
27
|
Scenario: Image files get don't get Cache-Control header by default
|
@@ -4,7 +4,8 @@ Feature: Image list based thumbnailing and S3 storage
|
|
4
4
|
|
5
5
|
Background:
|
6
6
|
Given S3 settings in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_TEST_BUCKET environment variables
|
7
|
-
Given
|
7
|
+
Given httpthumbnailer server is running at http://localhost:3100/health_check
|
8
|
+
Given httpimagestore server is running at http://localhost:3000/health_check with the following configuration
|
8
9
|
"""
|
9
10
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
10
11
|
|
@@ -49,7 +50,6 @@ Feature: Image list based thumbnailing and S3 storage
|
|
49
50
|
}
|
50
51
|
}
|
51
52
|
"""
|
52
|
-
Given httpthumbnailer server is running at http://localhost:3100/
|
53
53
|
|
54
54
|
@compatibility @test
|
55
55
|
Scenario: Putting original and its thumbnails to S3 bucket
|
@@ -4,7 +4,8 @@ Feature: Image list based thumbnailing and S3 storage
|
|
4
4
|
|
5
5
|
Background:
|
6
6
|
Given S3 settings in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_TEST_BUCKET environment variables
|
7
|
-
Given
|
7
|
+
Given httpthumbnailer server is running at http://localhost:3100/health_check
|
8
|
+
Given httpimagestore server is running at http://localhost:3000/health_check with the following configuration
|
8
9
|
"""
|
9
10
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
10
11
|
|
@@ -44,7 +45,6 @@ Feature: Image list based thumbnailing and S3 storage
|
|
44
45
|
source_file "original" root="/dev" path="zero"
|
45
46
|
}
|
46
47
|
"""
|
47
|
-
Given httpthumbnailer server is running at http://localhost:3100/
|
48
48
|
|
49
49
|
@error-reporting
|
50
50
|
Scenario: Reporting of missing resource
|
@@ -0,0 +1,149 @@
|
|
1
|
+
Feature: Store limited original image in S3 and thumbnail on facebook API
|
2
|
+
Similar API to described in https://developers.facebook.com/docs/reference/api/using-pictures/#sizes
|
3
|
+
|
4
|
+
Background:
|
5
|
+
Given S3 settings in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_TEST_BUCKET environment variables
|
6
|
+
Given httpthumbnailer server is running at http://localhost:3100/health_check
|
7
|
+
Given httpimagestore server is running at http://localhost:3000/health_check with the following configuration
|
8
|
+
"""
|
9
|
+
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
10
|
+
|
11
|
+
path "original-hash" "#{digest}"
|
12
|
+
path "path" "#{path}"
|
13
|
+
|
14
|
+
put "original" {
|
15
|
+
thumbnail "input" "original" operation="limit" width=100 height=100 format="jpeg" quality=95
|
16
|
+
|
17
|
+
store_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="original-hash"
|
18
|
+
|
19
|
+
output_store_path "original"
|
20
|
+
}
|
21
|
+
|
22
|
+
get "&type=square" {
|
23
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
24
|
+
|
25
|
+
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
26
|
+
|
27
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
28
|
+
}
|
29
|
+
|
30
|
+
get "&type=small" {
|
31
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
32
|
+
|
33
|
+
thumbnail "original" "thumbnail" operation="fit" width="50" height="2000" format="input"
|
34
|
+
|
35
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
36
|
+
}
|
37
|
+
|
38
|
+
get "&type=normall" {
|
39
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
40
|
+
|
41
|
+
thumbnail "original" "thumbnail" operation="fit" width="100" height="2000" format="input"
|
42
|
+
|
43
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
44
|
+
}
|
45
|
+
|
46
|
+
get "&type=large" {
|
47
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
48
|
+
|
49
|
+
thumbnail "original" "thumbnail" operation="fit" width="200" height="2000" format="input"
|
50
|
+
|
51
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
52
|
+
}
|
53
|
+
|
54
|
+
get "&:width" "&:height" {
|
55
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
56
|
+
|
57
|
+
thumbnail "original" "thumbnail" operation="crop" width="#{width}" height="#{height}" format="input"
|
58
|
+
|
59
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
60
|
+
}
|
61
|
+
|
62
|
+
get "&:width" "&:height?1080" {
|
63
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
64
|
+
|
65
|
+
thumbnail "original" "thumbnail" operation="fit" width="#{width}" height="#{height}" format="input"
|
66
|
+
|
67
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
68
|
+
}
|
69
|
+
|
70
|
+
get {
|
71
|
+
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
72
|
+
|
73
|
+
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
74
|
+
|
75
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
76
|
+
}
|
77
|
+
"""
|
78
|
+
|
79
|
+
@facebook @type
|
80
|
+
Scenario: Putting original to S3 bucket
|
81
|
+
Given there is no 4006450256177f4a file in S3 bucket
|
82
|
+
Given test.jpg file content as request body
|
83
|
+
When I do PUT request http://localhost:3000/original
|
84
|
+
Then response status will be 200
|
85
|
+
And response content type will be text/plain
|
86
|
+
And response body will be CRLF ended lines
|
87
|
+
"""
|
88
|
+
4006450256177f4a
|
89
|
+
"""
|
90
|
+
Then S3 object 4006450256177f4a will contain JPEG image of size 71x100
|
91
|
+
When I do GET request http://@AWS_S3_TEST_BUCKET@.s3.amazonaws.com/4006450256177f4a
|
92
|
+
Then response status will be 403
|
93
|
+
|
94
|
+
@facebook @type
|
95
|
+
Scenario: Getting square type tumbnail
|
96
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
97
|
+
When I do GET request http://localhost:3000/4006450256177f4a?type=square
|
98
|
+
Then response status will be 200
|
99
|
+
And response content type will be image/jpeg
|
100
|
+
Then response body will contain JPEG image of size 50x50
|
101
|
+
|
102
|
+
@facebook @type
|
103
|
+
Scenario: Getting small type tumbnail
|
104
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
105
|
+
When I do GET request http://localhost:3000/4006450256177f4a?type=small
|
106
|
+
Then response status will be 200
|
107
|
+
And response content type will be image/jpeg
|
108
|
+
Then response body will contain JPEG image of size 50x71
|
109
|
+
|
110
|
+
@facebook @type
|
111
|
+
Scenario: Getting normall type tumbnail
|
112
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
113
|
+
When I do GET request http://localhost:3000/4006450256177f4a?type=normall
|
114
|
+
Then response status will be 200
|
115
|
+
And response content type will be image/jpeg
|
116
|
+
Then response body will contain JPEG image of size 100x141
|
117
|
+
|
118
|
+
@facebook @type
|
119
|
+
Scenario: Getting large type tumbnail
|
120
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
121
|
+
When I do GET request http://localhost:3000/4006450256177f4a?type=large
|
122
|
+
Then response status will be 200
|
123
|
+
And response content type will be image/jpeg
|
124
|
+
Then response body will contain JPEG image of size 200x283
|
125
|
+
|
126
|
+
@facebook @type
|
127
|
+
Scenario: Getting square type tumbnail when no type is specified
|
128
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
129
|
+
When I do GET request http://localhost:3000/4006450256177f4a
|
130
|
+
Then response status will be 200
|
131
|
+
And response content type will be image/jpeg
|
132
|
+
Then response body will contain JPEG image of size 50x50
|
133
|
+
|
134
|
+
@facebook @size
|
135
|
+
Scenario: Getting custom size tumbnail
|
136
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
137
|
+
When I do GET request http://localhost:3000/4006450256177f4a?width=123&height=321
|
138
|
+
Then response status will be 200
|
139
|
+
And response content type will be image/jpeg
|
140
|
+
Then response body will contain JPEG image of size 123x321
|
141
|
+
|
142
|
+
@facebook @size
|
143
|
+
Scenario: Getting custom size tumbnail without height
|
144
|
+
Given test.jpg file content is stored in S3 under 4006450256177f4a
|
145
|
+
When I do GET request http://localhost:3000/4006450256177f4a?width=123
|
146
|
+
Then response status will be 200
|
147
|
+
And response content type will be image/jpeg
|
148
|
+
Then response body will contain JPEG image of size 123x174
|
149
|
+
|
@@ -5,7 +5,8 @@ Feature: Store limited original image in S3 and thumbnail based on request
|
|
5
5
|
|
6
6
|
Background:
|
7
7
|
Given S3 settings in AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_TEST_BUCKET environment variables
|
8
|
-
Given
|
8
|
+
Given httpthumbnailer server is running at http://localhost:3100/health_check
|
9
|
+
Given httpimagestore server is running at http://localhost:3000/health_check with the following configuration
|
9
10
|
"""
|
10
11
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
11
12
|
|
@@ -36,7 +37,6 @@ Feature: Store limited original image in S3 and thumbnail based on request
|
|
36
37
|
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
37
38
|
}
|
38
39
|
"""
|
39
|
-
Given httpthumbnailer server is running at http://localhost:3100/
|
40
40
|
|
41
41
|
@s3-store-and-thumbnail
|
42
42
|
Scenario: Putting original to S3 bucket
|
data/httpimagestore.gemspec
CHANGED
@@ -5,11 +5,11 @@
|
|
5
5
|
|
6
6
|
Gem::Specification.new do |s|
|
7
7
|
s.name = "httpimagestore"
|
8
|
-
s.version = "1.
|
8
|
+
s.version = "1.2.0"
|
9
9
|
|
10
10
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
11
11
|
s.authors = ["Jakub Pastuszek"]
|
12
|
-
s.date = "2013-07-
|
12
|
+
s.date = "2013-07-29"
|
13
13
|
s.description = "Thumbnails images using httpthumbnailer and stored data on HTTP server (S3)"
|
14
14
|
s.email = "jpastuszek@gmail.com"
|
15
15
|
s.executables = ["httpimagestore"]
|
@@ -30,6 +30,7 @@ Gem::Specification.new do |s|
|
|
30
30
|
"features/cache-control.feature",
|
31
31
|
"features/compatibility.feature",
|
32
32
|
"features/error-reporting.feature",
|
33
|
+
"features/facebook.feature",
|
33
34
|
"features/health-check.feature",
|
34
35
|
"features/s3-store-and-thumbnail.feature",
|
35
36
|
"features/step_definitions/httpimagestore_steps.rb",
|
@@ -42,7 +42,7 @@ module Configuration
|
|
42
42
|
@locals[:query_string_options] = query_string.sort.map{|kv| kv.join(':')}.join(',')
|
43
43
|
log.debug "processing request with body length: #{body.bytesize} bytes and locals: #{@locals} "
|
44
44
|
|
45
|
-
@locals[:
|
45
|
+
@locals[:_body] = body
|
46
46
|
|
47
47
|
@images = Images.new(memory_limit)
|
48
48
|
@memory_limit = memory_limit
|
@@ -81,8 +81,8 @@ module Configuration
|
|
81
81
|
|
82
82
|
class InputSource
|
83
83
|
def realize(request_state)
|
84
|
-
request_state.locals[:
|
85
|
-
request_state.images['input'] = Image.new(request_state.locals[:
|
84
|
+
request_state.locals[:_body].empty? and raise ZeroBodyLengthError
|
85
|
+
request_state.images['input'] = Image.new(request_state.locals[:_body])
|
86
86
|
end
|
87
87
|
end
|
88
88
|
|
@@ -202,24 +202,46 @@ module Configuration
|
|
202
202
|
handler_configuration.http_method = node.name
|
203
203
|
handler_configuration.uri_matchers = node.values.map do |matcher|
|
204
204
|
case matcher
|
205
|
-
|
206
|
-
|
205
|
+
# URI component matchers
|
206
|
+
when %r{^:([^/]+)/(.*)/$} # :foobar/.*/
|
207
|
+
name = $1
|
208
|
+
regexp = $2
|
207
209
|
Matcher.new(name.to_sym) do
|
208
210
|
Regexp.new("(#{regexp})")
|
209
211
|
end
|
210
|
-
when
|
211
|
-
name =
|
212
|
+
when /^:(.+)\?(.*)$/ # :foo?bar
|
213
|
+
name = $1.to_sym
|
214
|
+
default = $2
|
212
215
|
Matcher.new(name) do
|
213
|
-
->{match(name) || captures.push(
|
216
|
+
->{match(name) || captures.push(default)}
|
214
217
|
end
|
215
|
-
when
|
216
|
-
name =
|
218
|
+
when /^:(.+)$/ # :foobar
|
219
|
+
name = $1.to_sym
|
217
220
|
Matcher.new(name) do
|
218
221
|
name
|
219
222
|
end
|
220
|
-
|
223
|
+
# Query string matchers
|
224
|
+
when /^\&([^=]+)=(.+)$/# ?foo=bar
|
225
|
+
name = $1
|
226
|
+
value = $2
|
221
227
|
Matcher.new(nil) do
|
222
|
-
|
228
|
+
->{req[name] && req[name] == value}
|
229
|
+
end
|
230
|
+
when /^\&:(.+)\?(.*)$/# &:foo?bar
|
231
|
+
name = $1
|
232
|
+
default = $2
|
233
|
+
Matcher.new(name.to_sym) do
|
234
|
+
->{captures.push(req[name] || default)}
|
235
|
+
end
|
236
|
+
when /^\&:(.+)$/# &:foo
|
237
|
+
name = $1
|
238
|
+
Matcher.new(name.to_sym) do
|
239
|
+
->{req[name] && captures.push(req[name])}
|
240
|
+
end
|
241
|
+
# String URI component matcher
|
242
|
+
else # foobar
|
243
|
+
Matcher.new(nil) do
|
244
|
+
Regexp.escape(matcher)
|
223
245
|
end
|
224
246
|
end
|
225
247
|
end
|
@@ -62,7 +62,7 @@ module Configuration
|
|
62
62
|
Pathname.new(path).extname.delete('.')
|
63
63
|
when :digest
|
64
64
|
return locals[:_digest] if locals.include? :_digest
|
65
|
-
data = locals[:
|
65
|
+
data = locals[:_body] or raise NoMetaValueForPathTemplatePlaceholerError.new(path_name, template, :body, name)
|
66
66
|
digest = Digest::SHA2.new.update(data).to_s[0,16]
|
67
67
|
# cache digest in request locals
|
68
68
|
locals[:_digest] = digest
|
@@ -75,9 +75,10 @@ module Configuration
|
|
75
75
|
node.required_attributes('bucket', 'path')
|
76
76
|
node.valid_attribute_values('public_access', true, false, nil)
|
77
77
|
|
78
|
-
bucket, path_spec, cache_control,
|
79
|
-
*node.grab_attributes('bucket', 'path', 'cache-control', '
|
78
|
+
bucket, path_spec, public_access, cache_control, prefix, if_image_name_on =
|
79
|
+
*node.grab_attributes('bucket', 'path', 'public', 'cache-control', 'prefix', 'if-image-name-on')
|
80
80
|
public_access = false if public_access.nil?
|
81
|
+
prefix = '' if prefix.nil?
|
81
82
|
|
82
83
|
self.new(
|
83
84
|
configuration.global,
|
@@ -86,16 +87,18 @@ module Configuration
|
|
86
87
|
bucket,
|
87
88
|
path_spec,
|
88
89
|
public_access,
|
89
|
-
cache_control
|
90
|
+
cache_control,
|
91
|
+
prefix
|
90
92
|
)
|
91
93
|
end
|
92
94
|
|
93
|
-
def initialize(global, image_name, matcher, bucket, path_spec, public_access, cache_control)
|
95
|
+
def initialize(global, image_name, matcher, bucket, path_spec, public_access, cache_control, prefix)
|
94
96
|
super global, image_name, matcher
|
95
97
|
@bucket = bucket
|
96
98
|
@path_spec = path_spec
|
97
99
|
@public_access = public_access
|
98
100
|
@cache_control = cache_control
|
101
|
+
@prefix = prefix
|
99
102
|
local :bucket, @bucket
|
100
103
|
end
|
101
104
|
|
@@ -114,7 +117,7 @@ module Configuration
|
|
114
117
|
def object(path)
|
115
118
|
begin
|
116
119
|
bucket = client.buckets[@bucket]
|
117
|
-
yield bucket.objects[path]
|
120
|
+
yield bucket.objects[@prefix + path]
|
118
121
|
rescue AWS::S3::Errors::AccessDenied
|
119
122
|
raise S3AccessDenied.new(@bucket, path)
|
120
123
|
rescue AWS::S3::Errors::NoSuchBucket
|
@@ -18,10 +18,10 @@ describe Configuration do
|
|
18
18
|
EOF
|
19
19
|
|
20
20
|
subject.paths['uri'].render(path: 'test/abc.jpg').should == 'test/abc.jpg'
|
21
|
-
subject.paths['hash'].render(path: 'test/abc.jpg',
|
22
|
-
subject.paths['hash-name'].render(path: 'test/abc.jpg',
|
23
|
-
subject.paths['structured'].render(path: 'test/abc.jpg',
|
24
|
-
subject.paths['structured-name'].render(path: 'test/abc.jpg',
|
21
|
+
subject.paths['hash'].render(path: 'test/abc.jpg', _body: 'hello').should == '2cf24dba5fb0a30e.jpg'
|
22
|
+
subject.paths['hash-name'].render(path: 'test/abc.jpg', _body: 'hello', imagename: 'xbrna').should == '2cf24dba5fb0a30e/xbrna.jpg'
|
23
|
+
subject.paths['structured'].render(path: 'test/abc.jpg', _body: 'hello').should == 'test/2cf24dba5fb0a30e/abc.jpg'
|
24
|
+
subject.paths['structured-name'].render(path: 'test/abc.jpg', _body: 'hello', imagename: 'xbrna').should == 'test/2cf24dba5fb0a30e/abc-xbrna.jpg'
|
25
25
|
end
|
26
26
|
|
27
27
|
describe 'error handling' do
|
@@ -87,6 +87,7 @@ else
|
|
87
87
|
s3_client = AWS::S3.new(use_ssl: false)
|
88
88
|
s3_test_bucket = s3_client.buckets[ENV['AWS_S3_TEST_BUCKET']]
|
89
89
|
s3_test_bucket.objects['test.jpg'].write(@test_data, content_type: 'image/jpeg')
|
90
|
+
s3_test_bucket.objects['test_prefix/test.jpg'].write(@test_data, content_type: 'image/jpeg')
|
90
91
|
end
|
91
92
|
|
92
93
|
it 'should source image from S3 using path spec' do
|
@@ -113,6 +114,34 @@ else
|
|
113
114
|
status(state.images['original'].source_url).should == 200
|
114
115
|
end
|
115
116
|
|
117
|
+
describe 'storage prefix' do
|
118
|
+
subject do
|
119
|
+
Configuration.read(<<-EOF)
|
120
|
+
s3 key="#{ENV['AWS_ACCESS_KEY_ID']}" secret="#{ENV['AWS_SECRET_ACCESS_KEY']}"
|
121
|
+
path "hash" "\#{test_image}"
|
122
|
+
get {
|
123
|
+
source_s3 "original" bucket="#{ENV['AWS_S3_TEST_BUCKET']}" path="hash" prefix="test_prefix/"
|
124
|
+
}
|
125
|
+
EOF
|
126
|
+
end
|
127
|
+
|
128
|
+
it 'should still provide valid HTTPS URL incliding prefix' do
|
129
|
+
subject.handlers[0].image_sources[0].realize(state)
|
130
|
+
|
131
|
+
state.images['original'].source_url.should start_with "https://"
|
132
|
+
state.images['original'].source_url.should include ENV['AWS_S3_TEST_BUCKET']
|
133
|
+
state.images['original'].source_url.should include "/test_prefix/test.jpg"
|
134
|
+
state.images['original'].source_url.should include ENV['AWS_ACCESS_KEY_ID']
|
135
|
+
status(state.images['original'].source_url).should == 200
|
136
|
+
end
|
137
|
+
|
138
|
+
it 'should provide source path without prefix' do
|
139
|
+
subject.handlers[0].image_sources[0].realize(state)
|
140
|
+
|
141
|
+
state.images['original'].source_path.should == "test.jpg"
|
142
|
+
end
|
143
|
+
end
|
144
|
+
|
116
145
|
describe 'non encrypted connection mode' do
|
117
146
|
subject do
|
118
147
|
Configuration.read(<<-EOF)
|
@@ -292,6 +321,8 @@ else
|
|
292
321
|
s3_test_bucket = s3_client.buckets[ENV['AWS_S3_TEST_BUCKET']]
|
293
322
|
@test_object = s3_test_bucket.objects['test_out.jpg']
|
294
323
|
@test_object.delete
|
324
|
+
test_object = s3_test_bucket.objects['test_prefix/test_out.jpg']
|
325
|
+
test_object.delete
|
295
326
|
end
|
296
327
|
|
297
328
|
before :each do
|
@@ -323,6 +354,34 @@ else
|
|
323
354
|
status(state.images['input'].store_url).should == 200
|
324
355
|
end
|
325
356
|
|
357
|
+
describe 'storage prefix' do
|
358
|
+
subject do
|
359
|
+
Configuration.read(<<-EOF)
|
360
|
+
s3 key="#{ENV['AWS_ACCESS_KEY_ID']}" secret="#{ENV['AWS_SECRET_ACCESS_KEY']}"
|
361
|
+
path "hash" "\#{test_image}"
|
362
|
+
post {
|
363
|
+
store_s3 "input" bucket="#{ENV['AWS_S3_TEST_BUCKET']}" path="hash" prefix="test_prefix/"
|
364
|
+
}
|
365
|
+
EOF
|
366
|
+
end
|
367
|
+
|
368
|
+
it 'should still provide valid HTTPS URL incliding prefix' do
|
369
|
+
subject.handlers[0].stores[0].realize(state)
|
370
|
+
|
371
|
+
state.images['input'].store_url.should start_with "https://"
|
372
|
+
state.images['input'].store_url.should include ENV['AWS_S3_TEST_BUCKET']
|
373
|
+
state.images['input'].store_url.should include "test_prefix/test_out.jpg"
|
374
|
+
state.images['input'].store_url.should include ENV['AWS_ACCESS_KEY_ID']
|
375
|
+
status(state.images['input'].store_url).should == 200
|
376
|
+
end
|
377
|
+
|
378
|
+
it 'should provide storage path without prefix' do
|
379
|
+
subject.handlers[0].stores[0].realize(state)
|
380
|
+
|
381
|
+
state.images['input'].store_path.should == "test_out.jpg"
|
382
|
+
end
|
383
|
+
end
|
384
|
+
|
326
385
|
describe 'non encrypted connection mode' do
|
327
386
|
subject do
|
328
387
|
Configuration.read(<<-EOF)
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: httpimagestore
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2013-07-
|
12
|
+
date: 2013-07-29 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: unicorn-cuba-base
|
@@ -241,6 +241,7 @@ files:
|
|
241
241
|
- features/cache-control.feature
|
242
242
|
- features/compatibility.feature
|
243
243
|
- features/error-reporting.feature
|
244
|
+
- features/facebook.feature
|
244
245
|
- features/health-check.feature
|
245
246
|
- features/s3-store-and-thumbnail.feature
|
246
247
|
- features/step_definitions/httpimagestore_steps.rb
|
@@ -291,7 +292,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
291
292
|
version: '0'
|
292
293
|
segments:
|
293
294
|
- 0
|
294
|
-
hash:
|
295
|
+
hash: -1673888350520322204
|
295
296
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
296
297
|
none: false
|
297
298
|
requirements:
|