httpimagestore 1.2.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- data/Gemfile +5 -2
- data/Gemfile.lock +24 -14
- data/README.md +161 -105
- data/bin/httpimagestore +6 -2
- data/features/cache-control.feature +2 -2
- data/features/compatibility.feature +4 -4
- data/features/error-reporting.feature +49 -3
- data/features/flexi.feature +260 -0
- data/features/s3-store-and-thumbnail.feature +1 -1
- data/features/step_definitions/httpimagestore_steps.rb +24 -0
- data/features/storage.feature +199 -0
- data/features/support/test-large.jpg +0 -0
- data/features/support/test.png +0 -0
- data/httpimagestore.gemspec +13 -9
- data/lib/httpimagestore/configuration/file.rb +1 -1
- data/lib/httpimagestore/configuration/handler.rb +114 -20
- data/lib/httpimagestore/configuration/identify.rb +56 -0
- data/lib/httpimagestore/configuration/path.rb +7 -28
- data/lib/httpimagestore/configuration/s3.rb +230 -12
- data/lib/httpimagestore/configuration/thumbnailer.rb +17 -8
- data/lib/httpimagestore/error_reporter.rb +1 -11
- data/load_test/load_test.jmx +11 -11
- data/load_test/thumbnail_specs.csv +11 -11
- data/spec/configuration_file_spec.rb +20 -20
- data/spec/configuration_handler_spec.rb +156 -6
- data/spec/configuration_identify_spec.rb +45 -0
- data/spec/configuration_output_spec.rb +15 -15
- data/spec/configuration_path_spec.rb +70 -23
- data/spec/configuration_s3_spec.rb +187 -28
- data/spec/configuration_thumbnailer_spec.rb +22 -22
- data/spec/spec_helper.rb +11 -2
- data/spec/support/full.cfg +4 -4
- metadata +12 -8
- data/features/facebook.feature +0 -149
data/Gemfile
CHANGED
@@ -1,11 +1,14 @@
|
|
1
1
|
source "http://rubygems.org"
|
2
2
|
ruby "1.9.3"
|
3
3
|
|
4
|
-
gem "unicorn-cuba-base", "~> 1.0"
|
5
|
-
gem "
|
4
|
+
#gem "unicorn-cuba-base", "~> 1.0"
|
5
|
+
gem "unicorn-cuba-base", path: '../unicorn-cuba-base'
|
6
|
+
#gem "httpthumbnailer-client", "~> 1.0"
|
7
|
+
gem "httpthumbnailer-client", path: '../httpthumbnailer-client'
|
6
8
|
gem "aws-sdk", "~> 1.10"
|
7
9
|
gem "mime-types", "~> 1.17"
|
8
10
|
gem "sdl4r", "~> 0.9"
|
11
|
+
gem "msgpack", "~> 0.5"
|
9
12
|
|
10
13
|
# Add dependencies to develop your gem here.
|
11
14
|
# Include everything needed to run rake, tests, features, etc.
|
data/Gemfile.lock
CHANGED
@@ -5,6 +5,25 @@ PATH
|
|
5
5
|
rmagick (~> 2)
|
6
6
|
unicorn-cuba-base (~> 1.0)
|
7
7
|
|
8
|
+
PATH
|
9
|
+
remote: ../httpthumbnailer-client
|
10
|
+
specs:
|
11
|
+
httpthumbnailer-client (1.0.0)
|
12
|
+
cli (~> 1.1.0)
|
13
|
+
httpclient (>= 2.3)
|
14
|
+
multipart-parser (~> 0.1.1)
|
15
|
+
|
16
|
+
PATH
|
17
|
+
remote: ../unicorn-cuba-base
|
18
|
+
specs:
|
19
|
+
unicorn-cuba-base (1.0.0)
|
20
|
+
cli (~> 1.1.0)
|
21
|
+
cuba (~> 3.0)
|
22
|
+
facter (~> 1.6.11)
|
23
|
+
raindrops (~> 0.11)
|
24
|
+
ruby-ip (~> 0.9)
|
25
|
+
unicorn (>= 4.6.2)
|
26
|
+
|
8
27
|
GEM
|
9
28
|
remote: http://rubygems.org/
|
10
29
|
specs:
|
@@ -28,10 +47,6 @@ GEM
|
|
28
47
|
json (>= 1.4.6)
|
29
48
|
git (1.2.5)
|
30
49
|
httpclient (2.3.3)
|
31
|
-
httpthumbnailer-client (1.0.0)
|
32
|
-
cli (~> 1.1.0)
|
33
|
-
httpclient (>= 2.3)
|
34
|
-
multipart-parser (~> 0.1.1)
|
35
50
|
jeweler (1.8.4)
|
36
51
|
bundler (~> 1.0)
|
37
52
|
git (>= 1.2.5)
|
@@ -40,6 +55,7 @@ GEM
|
|
40
55
|
json (1.7.5)
|
41
56
|
kgio (2.8.0)
|
42
57
|
mime-types (1.17.2)
|
58
|
+
msgpack (0.5.5)
|
43
59
|
multipart-parser (0.1.1)
|
44
60
|
nokogiri (1.5.9)
|
45
61
|
prawn (0.8.4)
|
@@ -65,17 +81,10 @@ GEM
|
|
65
81
|
rspec-mocks (2.13.1)
|
66
82
|
ruby-ip (0.9.1)
|
67
83
|
sdl4r (0.9.11)
|
68
|
-
unicorn (4.6.
|
84
|
+
unicorn (4.6.3)
|
69
85
|
kgio (~> 2.6)
|
70
86
|
rack
|
71
87
|
raindrops (~> 0.7)
|
72
|
-
unicorn-cuba-base (1.0.0)
|
73
|
-
cli (~> 1.1.0)
|
74
|
-
cuba (~> 3.0)
|
75
|
-
facter (~> 1.6.11)
|
76
|
-
raindrops (~> 0.11)
|
77
|
-
ruby-ip (~> 0.9)
|
78
|
-
unicorn (>= 4.6.2)
|
79
88
|
uuidtools (2.1.4)
|
80
89
|
|
81
90
|
PLATFORMS
|
@@ -87,11 +96,12 @@ DEPENDENCIES
|
|
87
96
|
daemon (~> 1)
|
88
97
|
httpclient (>= 2.3)
|
89
98
|
httpthumbnailer!
|
90
|
-
httpthumbnailer-client
|
99
|
+
httpthumbnailer-client!
|
91
100
|
jeweler (~> 1.8.4)
|
92
101
|
mime-types (~> 1.17)
|
102
|
+
msgpack (~> 0.5)
|
93
103
|
prawn (= 0.8.4)
|
94
104
|
rdoc (~> 3.9)
|
95
105
|
rspec (~> 2.13)
|
96
106
|
sdl4r (~> 0.9)
|
97
|
-
unicorn-cuba-base
|
107
|
+
unicorn-cuba-base!
|
data/README.md
CHANGED
@@ -16,6 +16,13 @@ It is using [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer) as
|
|
16
16
|
|
17
17
|
## Changelog
|
18
18
|
|
19
|
+
### 1.3.0
|
20
|
+
|
21
|
+
* `identify` statement support (requires [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer) v1.1.0 or higher)
|
22
|
+
* changed variable names: `imagename` to `image_name`, `mimeextension` to `image_mime_extension`, `digest` to `input_digest` (old names can still be used)
|
23
|
+
* added support for new variables: `uuid`, `input_sha256`, `image_digest`, `image_sha256`, `image_width`, `image_height`, `input_image_mime_extension`, `input_image_width`, `input_image_height`
|
24
|
+
* improved variable processing and logging
|
25
|
+
|
19
26
|
### 1.2.0
|
20
27
|
|
21
28
|
* matching for query string key and value
|
@@ -96,25 +103,41 @@ Arguments:
|
|
96
103
|
|
97
104
|
Variables:
|
98
105
|
|
99
|
-
* `
|
106
|
+
* `input_digest` - input image digest based on it's content only; this is first 16 hex characters from SHA256 digest; ex. `9f86d081884c7d65`
|
107
|
+
* `digest` - same as `input_digest`; deprecated, please use `input_digest` instead
|
108
|
+
* `input_sha256` - input image digest based on it's content only; this is whole hex encoded SHA256 digest; ex. `9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08`
|
109
|
+
* `input_image_mime_extension` - extension of input image determined from image mime type
|
110
|
+
* `input_image_width` - width in pixels of the input image
|
111
|
+
* `input_image_height` - height in pixels of the input image
|
112
|
+
* `image_digest` - digest of image being stored based on it's content only; this is first 16 hex characters from SHA256 digest; ex. `9f86d081884c7d65`
|
113
|
+
* `image_sha256` - digest of image being stored based on it's content only; this is whole hex encoded SHA256 digest; ex. `9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08`
|
114
|
+
* `image_name` - name of the image that is being stored or sourced
|
115
|
+
* `imagename` - same as `image_name`; deprecated, please use `image_name` instead
|
116
|
+
* `image_mime_extension` - extension of image being stored determined from image mime type
|
117
|
+
* `mimeextension` - same as `image_mime_extension`; deprecated, please use `image_mime_extension` instead
|
118
|
+
* `image_width` - width in pixels of the image that is being stored
|
119
|
+
* `image_height` - height in pixels of the image that is being stored
|
100
120
|
* `path` - remaining (unmatched) request URL path
|
101
121
|
* `basename` - name of the file without it's extension determined from `path`
|
102
122
|
* `direname` - name of the directory determined form `path`
|
103
123
|
* `extension` - file extension determined from `path`
|
104
|
-
* `
|
105
|
-
* `imagename` - name of the image that is being stored or sourced
|
124
|
+
* `uuid` - unique ID for request; ex. `1b68a767-8163-425a-a32c-db6ed7200343`
|
106
125
|
* URL matches and query string parameters - other variables can be matched from URL pattern and query string parameters - see API configuration
|
107
126
|
|
127
|
+
Notes:
|
128
|
+
* Input based variables can only be calculated for non GET requests where body is not empty.
|
129
|
+
* Image mime type, width and height will only be know if that image was sent to or received from [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer) during thumbnailing or via `identify` statement.
|
130
|
+
|
108
131
|
Example:
|
109
132
|
|
110
133
|
```sdl
|
111
134
|
path "uri" "#{path}"
|
112
|
-
path "hash" "#{
|
135
|
+
path "hash" "#{input_digest}.#{extension}"
|
113
136
|
|
114
137
|
path {
|
115
|
-
"hash-name" "#{
|
116
|
-
"structured" "#{dirname}/#{
|
117
|
-
"structured-name" "#{dirname}/#{
|
138
|
+
"hash-name" "#{input_digest}/#{image_name}.#{extension}"
|
139
|
+
"structured" "#{dirname}/#{input_digest}/#{basename}.#{extension}"
|
140
|
+
"structured-name" "#{dirname}/#{input_digest}/#{basename}-#{image_name}.#{extension}"
|
118
141
|
}
|
119
142
|
```
|
120
143
|
|
@@ -225,6 +248,8 @@ get "small" {
|
|
225
248
|
|
226
249
|
Requesting `/small` URI will result with image fetched from S3 bucket `mybucket` and key `myimage.jpg` and named `original`.
|
227
250
|
|
251
|
+
### API endpoint processing operations
|
252
|
+
|
228
253
|
#### thumbnail
|
229
254
|
|
230
255
|
This source will provide new images based on already sourced images by processing them with [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer) backend.
|
@@ -270,6 +295,25 @@ put ":operation" ":width" ":height" ":options" {
|
|
270
295
|
}
|
271
296
|
```
|
272
297
|
|
298
|
+
#### identify
|
299
|
+
|
300
|
+
This statement allows for image mime type, width and height identification based on image content with use of [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer).
|
301
|
+
This is usefull when image is not taking part in thumbnailing process but variables that are based on meta data needs to be used for storage.
|
302
|
+
|
303
|
+
Example:
|
304
|
+
|
305
|
+
```sdl
|
306
|
+
path "size" "#{image_width}x#{image_height}.#{image_mime_extension}"
|
307
|
+
|
308
|
+
put "images" {
|
309
|
+
identify "input"
|
310
|
+
store_s3 "input" bucket="mybucket" path="size"
|
311
|
+
output_store_path "input"
|
312
|
+
}
|
313
|
+
```
|
314
|
+
|
315
|
+
In this example input image is indentified by [HTTP Thumbnailer](https://github.com/jpastuszek/httpthumbnailer). Received information is used to populate `image_width`, `image_height` and `image_mime_extension` variables used to generate S3 storage key.
|
316
|
+
|
273
317
|
### API endpoint storage operations
|
274
318
|
|
275
319
|
This statements are executed after all source statements are finished.
|
@@ -291,8 +335,8 @@ Options:
|
|
291
335
|
Example:
|
292
336
|
|
293
337
|
```sdl
|
294
|
-
path "imagename" "#{
|
295
|
-
path "hash" "#{
|
338
|
+
path "imagename" "#{image_name}"
|
339
|
+
path "hash" "#{input_digest}"
|
296
340
|
|
297
341
|
put "store" ":name" {
|
298
342
|
store_file "input" root="/srv/images" path="imagename"
|
@@ -323,8 +367,8 @@ Example:
|
|
323
367
|
```sdl
|
324
368
|
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
325
369
|
|
326
|
-
path "imagename" "#{
|
327
|
-
path "hash" "#{
|
370
|
+
path "imagename" "#{image_name}"
|
371
|
+
path "hash" "#{input_digest}"
|
328
372
|
|
329
373
|
put ":name" {
|
330
374
|
store_s3 "input" bucket="mybucket" path="imagename"
|
@@ -431,8 +475,8 @@ Example:
|
|
431
475
|
```sdl
|
432
476
|
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
433
477
|
|
434
|
-
path "hash" "#{
|
435
|
-
path "hash-name" "#{
|
478
|
+
path "hash" "#{input_digest}.#{image_mime_extension}"
|
479
|
+
path "hash-name" "#{input_digest}/#{image_name}.#{image_mime_extension}"
|
436
480
|
|
437
481
|
put "thumbnail" {
|
438
482
|
thumbnail "input" {
|
@@ -478,18 +522,115 @@ The list is in format `image name[,image name]*`.
|
|
478
522
|
|
479
523
|
This option is useful when building API that works on predefined set of image operations and allows to select witch set of operations to perform with list included in the URL.
|
480
524
|
|
481
|
-
|
525
|
+
## Configuration examples
|
526
|
+
|
527
|
+
### Flexible API example
|
528
|
+
|
529
|
+
Features two storage apporaches: with JPEG conversion and limiting in size - for user provided content - and storing literaly.
|
530
|
+
POST requests will end up with server side generated storage key based on input data digest.
|
531
|
+
PUT requsts can be used to store image under provided storage key.
|
532
|
+
Thumbnail GET API is similart to described in [Facebook APIs](https://developers.facebook.com/docs/reference/api/using-pictures/#sizes) for thumbnailing.
|
533
|
+
Stored object extension and content type is determined from image data.
|
534
|
+
|
535
|
+
```sdl
|
536
|
+
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
537
|
+
|
538
|
+
path "hash" "#{input_digest}.#{image_mime_extension}"
|
539
|
+
path "path" "#{path}"
|
540
|
+
|
541
|
+
## User uploaded content - always JPEG converted, not bigger than 2160x2160 and in hight quality compression
|
542
|
+
post "pictures" {
|
543
|
+
thumbnail "input" "original" operation="limit" width=2160 height=2160 format="jpeg" quality=95
|
544
|
+
store_s3 "original" bucket="mybucket" path="hash"
|
545
|
+
output_store_path "original"
|
546
|
+
}
|
547
|
+
|
548
|
+
put "pictures" {
|
549
|
+
thumbnail "input" "original" operation="limit" width=2160 height=2160 format="jpeg" quality=95
|
550
|
+
store_s3 "original" bucket="mybucket" path="path"
|
551
|
+
output_store_path "original"
|
552
|
+
}
|
553
|
+
|
554
|
+
## Uploaded by admin for use on the website for example - store whatever was send
|
555
|
+
post "images" {
|
556
|
+
identify "input"
|
557
|
+
store_s3 "input" bucket="mybucket" path="hash"
|
558
|
+
output_store_path "input"
|
559
|
+
}
|
560
|
+
|
561
|
+
put "images" {
|
562
|
+
identify "input"
|
563
|
+
store_s3 "input" bucket="mybucket" path="path"
|
564
|
+
output_store_path "input"
|
565
|
+
}
|
566
|
+
|
567
|
+
## Thumbailing - keep input format; default JPEG quality is 85
|
568
|
+
### Thumbnail specification from query string paramaters
|
569
|
+
get "pictures" "&:width" "&:height" "&:operation?crop" "&:background-color?white" {
|
570
|
+
source_s3 "original" bucket="mybucket" path="path"
|
571
|
+
thumbnail "original" "thumbnail" operation="#{operation}" width="#{width}" height="#{height}" options="background-color:#{background-color}"
|
572
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
573
|
+
}
|
574
|
+
|
575
|
+
get "pictures" "&:width" "&:height?1080" "&:operation?fit" "&:background-color?white" {
|
576
|
+
source_s3 "original" bucket="mybucket" path="path"
|
577
|
+
thumbnail "original" "thumbnail" operation="#{operation}" width="#{width}" height="#{height}" options="background-color:#{background-color}"
|
578
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
579
|
+
}
|
580
|
+
|
581
|
+
get "pictures" "&:height" "&:width?1080" "&:operation?fit" "&:background-color?white" {
|
582
|
+
source_s3 "original" bucket="mybucket" path="path"
|
583
|
+
thumbnail "original" "thumbnail" operation="#{operation}" width="#{width}" height="#{height}" options="background-color:#{background-color}"
|
584
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
585
|
+
}
|
586
|
+
|
587
|
+
### Predefined thumbnailing specification
|
588
|
+
get "pictures" "&type=square" {
|
589
|
+
source_s3 "original" bucket="mybucket" path="path"
|
590
|
+
thumbnail "original" "thumbnail" operation="crop" width="50" height="50"
|
591
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
592
|
+
}
|
593
|
+
|
594
|
+
get "pictures" "&type=small" {
|
595
|
+
source_s3 "original" bucket="mybucket" path="path"
|
596
|
+
thumbnail "original" "thumbnail" operation="fit" width="50" height="2000"
|
597
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
598
|
+
}
|
599
|
+
|
600
|
+
get "pictures" "&type=normall" {
|
601
|
+
source_s3 "original" bucket="mybucket" path="path"
|
602
|
+
thumbnail "original" "thumbnail" operation="fit" width="100" height="2000"
|
603
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
604
|
+
}
|
605
|
+
|
606
|
+
get "pictures" "&type=large" {
|
607
|
+
source_s3 "original" bucket="mybucket" path="path"
|
608
|
+
thumbnail "original" "thumbnail" operation="fit" width="200" height="2000"
|
609
|
+
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
610
|
+
}
|
611
|
+
|
612
|
+
## By default serve original image as is - JPEG for user content and what was send for admin uploaded images
|
613
|
+
get "pictures" {
|
614
|
+
source_s3 "original" bucket="mybucket" path="path"
|
615
|
+
output_image "original" cache-control="public, max-age=31557600, s-maxage=0"
|
616
|
+
}
|
617
|
+
```
|
618
|
+
|
619
|
+
For more information see [flexi feature test](https://github.com/jpastuszek/httpimagestore/blob/master/features/flexi.feature).
|
620
|
+
|
621
|
+
### Compatible and on demand API examples
|
482
622
|
|
483
|
-
This
|
623
|
+
This example can be used to create new API and migration between old and new approach.
|
624
|
+
It presents API compatible with previous version of httpimagestore (v0.5.0) and one possible thumbnail on demand approach API.
|
484
625
|
|
485
626
|
```sdl
|
486
627
|
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
487
628
|
|
488
629
|
# Compatibility API
|
489
|
-
path "compat-hash" "#{
|
490
|
-
path "compat-hash-name" "#{
|
491
|
-
path "compat-structured" "#{dirname}/#{
|
492
|
-
path "compat-structured-name" "#{dirname}/#{
|
630
|
+
path "compat-hash" "#{input_digest}.#{image_mime_extension}"
|
631
|
+
path "compat-hash-name" "#{input_digest}/#{image_name}.#{image_mime_extension}"
|
632
|
+
path "compat-structured" "#{dirname}/#{input_digest}/#{basename}.#{image_mime_extension}"
|
633
|
+
path "compat-structured-name" "#{dirname}/#{input_digest}/#{basename}-#{image_name}.#{image_mime_extension}"
|
493
634
|
|
494
635
|
put "thumbnail" ":name_list" ":path/.+/" {
|
495
636
|
thumbnail "input" {
|
@@ -552,30 +693,24 @@ put "thumbnail" ":name_list" {
|
|
552
693
|
}
|
553
694
|
|
554
695
|
# Thumbnail on demand API
|
555
|
-
path "hash" "#{
|
696
|
+
path "hash" "#{input_digest}.#{image_mime_extension}"
|
556
697
|
path "path" "#{path}"
|
557
698
|
|
558
699
|
put "v1" "original" {
|
559
700
|
thumbnail "input" "original" operation="limit" width=2160 height=2160 format="jpeg" quality=95
|
560
|
-
|
561
701
|
store_s3 "original" bucket="mybucket_v1" path="hash"
|
562
|
-
|
563
702
|
output_store_path "original"
|
564
703
|
}
|
565
704
|
|
566
705
|
get "v1" "thumbnail" ":path" ":operation" ":width" ":height" ":options?" {
|
567
706
|
source_s3 "original" bucket="mybucket_v1" path="path"
|
568
|
-
|
569
707
|
thumbnail "original" "thumbnail" operation="#{operation}" width="#{width}" height="#{height}" options="#{options}" quality=84 format="jpeg"
|
570
|
-
|
571
708
|
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
572
709
|
}
|
573
710
|
|
574
711
|
get "v2" "thumbnail" ":operation" ":width" ":height" {
|
575
712
|
source_s3 "original" bucket="mybucket_v1" path="path"
|
576
|
-
|
577
713
|
thumbnail "original" "thumbnail" operation="#{operation}" width="#{width}" height="#{height}" options="#{query_string_options}" quality=84 format="png"
|
578
|
-
|
579
714
|
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
580
715
|
}
|
581
716
|
```
|
@@ -682,85 +817,6 @@ $ identify /tmp/test.jpg
|
|
682
817
|
/tmp/test.jpg JPEG 100x100 100x100+0+0 8-bit sRGB 3.31KB 0.000u 0:00.000
|
683
818
|
```
|
684
819
|
|
685
|
-
## Facebook like API example
|
686
|
-
|
687
|
-
Based on [Facebook APIs](https://developers.facebook.com/docs/reference/api/using-pictures/#sizes).
|
688
|
-
|
689
|
-
```sdl
|
690
|
-
s3 key="AIAITCKMELYWQZPJP7HQ" secret="V37lCu0F48Tv9s7QVqIT/sLf/wwqhNSB4B0Em7Ei" ssl=false
|
691
|
-
|
692
|
-
path "hash" "#{digest}"
|
693
|
-
path "path" "#{path}"
|
694
|
-
|
695
|
-
put "original" {
|
696
|
-
thumbnail "input" "original" operation="limit" width=100 height=100 format="jpeg" quality=95
|
697
|
-
|
698
|
-
store_s3 "original" bucket="mybucket_v1" path="hash"
|
699
|
-
|
700
|
-
output_store_path "original"
|
701
|
-
}
|
702
|
-
|
703
|
-
# type selected
|
704
|
-
get "&type=square" {
|
705
|
-
source_s3 "original" bucket="@AWS_S3_TEST_BUCKET@" path="path"
|
706
|
-
|
707
|
-
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
708
|
-
|
709
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
710
|
-
}
|
711
|
-
|
712
|
-
get "&type=small" {
|
713
|
-
source_s3 "original" bucket="mybucket_v1" path="path"
|
714
|
-
|
715
|
-
thumbnail "original" "thumbnail" operation="fit" width="50" height="2000" format="input"
|
716
|
-
|
717
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
718
|
-
}
|
719
|
-
|
720
|
-
get "&type=normall" {
|
721
|
-
source_s3 "original" bucket="mybucket_v1" path="path"
|
722
|
-
|
723
|
-
thumbnail "original" "thumbnail" operation="fit" width="100" height="2000" format="input"
|
724
|
-
|
725
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
726
|
-
}
|
727
|
-
|
728
|
-
get "&type=large" {
|
729
|
-
source_s3 "original" bucket="mybucket_v1" path="path"
|
730
|
-
|
731
|
-
thumbnail "original" "thumbnail" operation="fit" width="200" height="2000" format="input"
|
732
|
-
|
733
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
734
|
-
}
|
735
|
-
|
736
|
-
# crop to specified width and height
|
737
|
-
get "&:width" "&:height" {
|
738
|
-
source_s3 "original" bucket="mybucket_v1" path="path"
|
739
|
-
|
740
|
-
thumbnail "original" "thumbnail" operation="crop" width="#{width}" height="#{height}" format="input"
|
741
|
-
|
742
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
743
|
-
}
|
744
|
-
|
745
|
-
# fit to width when no height is specified
|
746
|
-
get "&:width" "&:height?1080" {
|
747
|
-
source_s3 "original" bucket="mybucket_v1" path="path"
|
748
|
-
|
749
|
-
thumbnail "original" "thumbnail" operation="fit" width="#{width}" height="#{height}" format="input"
|
750
|
-
|
751
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
752
|
-
}
|
753
|
-
|
754
|
-
# default to small class
|
755
|
-
get {
|
756
|
-
source_s3 "original" bucket="bmybucket_v1" path="path"
|
757
|
-
|
758
|
-
thumbnail "original" "thumbnail" operation="crop" width="50" height="50" format="input"
|
759
|
-
|
760
|
-
output_image "thumbnail" cache-control="public, max-age=31557600, s-maxage=0"
|
761
|
-
}
|
762
|
-
```
|
763
|
-
|
764
820
|
## Usage
|
765
821
|
|
766
822
|
After installation of the gem the `httpimagestore` executable is installed in **PATH**.
|
data/bin/httpimagestore
CHANGED
@@ -84,8 +84,11 @@ Application.new('httpimagestore', port: 3000, processor_count_factor: 2) do
|
|
84
84
|
|
85
85
|
state = Configuration::RequestState.new(req.body.read, matches, path, query_string, memory_limit)
|
86
86
|
|
87
|
-
handler.
|
88
|
-
|
87
|
+
handler.sources.each do |source|
|
88
|
+
source.realize(state) unless source.respond_to? :excluded? and source.excluded?(state)
|
89
|
+
end
|
90
|
+
handler.processors.each do |processor|
|
91
|
+
processor.realize(state) unless processor.respond_to? :excluded? and processor.excluded?(state)
|
89
92
|
end
|
90
93
|
handler.stores.each do |store|
|
91
94
|
store.realize(state) unless store.respond_to? :excluded? and store.excluded?(state)
|
@@ -123,6 +126,7 @@ Application.new('httpimagestore', port: 3000, processor_count_factor: 2) do
|
|
123
126
|
require 'httpimagestore/configuration/path'
|
124
127
|
require 'httpimagestore/configuration/handler'
|
125
128
|
require 'httpimagestore/configuration/thumbnailer'
|
129
|
+
require 'httpimagestore/configuration/identify'
|
126
130
|
require 'httpimagestore/configuration/file'
|
127
131
|
require 'httpimagestore/configuration/output'
|
128
132
|
require 'httpimagestore/configuration/s3'
|
@@ -8,8 +8,8 @@ Feature: S3 object Cache-Control header settings
|
|
8
8
|
"""
|
9
9
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
10
10
|
|
11
|
-
path "hash" "#{
|
12
|
-
path "hash-name" "#{
|
11
|
+
path "hash" "#{input_digest}.#{image_mime_extension}"
|
12
|
+
path "hash-name" "#{input_digest}-#{image_name}.#{image_mime_extension}"
|
13
13
|
|
14
14
|
put "thumbnail" {
|
15
15
|
thumbnail "input" {
|
@@ -9,10 +9,10 @@ Feature: Image list based thumbnailing and S3 storage
|
|
9
9
|
"""
|
10
10
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
11
11
|
|
12
|
-
path "hash" "#{
|
13
|
-
path "hash-name" "#{
|
14
|
-
path "structured" "#{dirname}/#{
|
15
|
-
path "structured-name" "#{dirname}/#{
|
12
|
+
path "hash" "#{input_digest}.#{image_mime_extension}"
|
13
|
+
path "hash-name" "#{input_digest}/#{image_name}.#{image_mime_extension}"
|
14
|
+
path "structured" "#{dirname}/#{input_digest}/#{basename}.#{image_mime_extension}"
|
15
|
+
path "structured-name" "#{dirname}/#{input_digest}/#{basename}-#{image_name}.#{image_mime_extension}"
|
16
16
|
|
17
17
|
put "thumbnail" ":name_list" ":path/.+/" {
|
18
18
|
thumbnail "input" {
|
@@ -9,7 +9,7 @@ Feature: Image list based thumbnailing and S3 storage
|
|
9
9
|
"""
|
10
10
|
s3 key="@AWS_ACCESS_KEY_ID@" secret="@AWS_SECRET_ACCESS_KEY@" ssl=false
|
11
11
|
|
12
|
-
path "structured-name" "#{dirname}/#{
|
12
|
+
path "structured-name" "#{dirname}/#{input_digest}/#{basename}-#{image_name}.#{image_mime_extension}"
|
13
13
|
path "missing" "blah"
|
14
14
|
path "zero" "zero"
|
15
15
|
|
@@ -44,6 +44,21 @@ Feature: Image list based thumbnailing and S3 storage
|
|
44
44
|
get "zero" {
|
45
45
|
source_file "original" root="/dev" path="zero"
|
46
46
|
}
|
47
|
+
|
48
|
+
path "not_defined" "#{bogous}"
|
49
|
+
get "not_defined" {
|
50
|
+
source_file "original" root="/tmp" path="not_defined"
|
51
|
+
}
|
52
|
+
|
53
|
+
path "body" "#{input_digest}"
|
54
|
+
get "body" {
|
55
|
+
source_file "original" root="/tmp" path="body"
|
56
|
+
}
|
57
|
+
|
58
|
+
path "no_image_meta" "#{image_mime_extension}"
|
59
|
+
put "no_image_meta" {
|
60
|
+
store_file "input" root="/tmp" path="no_image_meta"
|
61
|
+
}
|
47
62
|
"""
|
48
63
|
|
49
64
|
@error-reporting
|
@@ -148,7 +163,7 @@ Feature: Image list based thumbnailing and S3 storage
|
|
148
163
|
thumbnailing of 'input' into 'bad_dim' failed: bad dimension value: 128x
|
149
164
|
"""
|
150
165
|
|
151
|
-
@error-reporting
|
166
|
+
@error-reporting @413 @load
|
152
167
|
Scenario: Too large image - uploaded image too big to fit in memory limit
|
153
168
|
Given test-large.jpg file content as request body
|
154
169
|
When I do PUT request http://localhost:3000/multipart/large_png
|
@@ -166,7 +181,7 @@ Feature: Image list based thumbnailing and S3 storage
|
|
166
181
|
thumbnailing of 'input' into 'large_png' failed: image too large: cache resources exhausted
|
167
182
|
"""
|
168
183
|
|
169
|
-
@error-reporting
|
184
|
+
@error-reporting @413 @thumbnail
|
170
185
|
Scenario: Too large image - memory exhausted when thmbnailing
|
171
186
|
Given test.jpg file content as request body
|
172
187
|
When I do PUT request http://localhost:3000/multipart/superlarge
|
@@ -205,3 +220,34 @@ Feature: Image list based thumbnailing and S3 storage
|
|
205
220
|
memory limit exceeded
|
206
221
|
"""
|
207
222
|
|
223
|
+
@error-reporting @variables
|
224
|
+
Scenario: Bad variable use - not defined
|
225
|
+
When I do GET request http://localhost:3000/not_defined
|
226
|
+
Then response status will be 500
|
227
|
+
And response content type will be text/plain
|
228
|
+
And response body will be CRLF ended lines like
|
229
|
+
"""
|
230
|
+
cannot generate path 'not_defined' from template '\#{bogous}': variable 'bogous' not defined
|
231
|
+
"""
|
232
|
+
|
233
|
+
@error-reporting @variables
|
234
|
+
Scenario: Bad variable use - no body
|
235
|
+
When I do GET request http://localhost:3000/body
|
236
|
+
Then response status will be 500
|
237
|
+
And response content type will be text/plain
|
238
|
+
And response body will be CRLF ended lines like
|
239
|
+
"""
|
240
|
+
cannot generate path 'body' from template '#{input_digest}': need not empty request body to generate value for 'input_digest'
|
241
|
+
"""
|
242
|
+
|
243
|
+
@error-reporting @variables
|
244
|
+
Scenario: Bad variable use - no image meta
|
245
|
+
Given test.jpg file content as request body
|
246
|
+
When I do PUT request http://localhost:3000/no_image_meta
|
247
|
+
Then response status will be 500
|
248
|
+
And response content type will be text/plain
|
249
|
+
And response body will be CRLF ended lines like
|
250
|
+
"""
|
251
|
+
cannot generate path 'no_image_meta' from template '#{image_mime_extension}': image 'input' does not have data for variable 'image_mime_extension'
|
252
|
+
"""
|
253
|
+
|