urlbox 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: cec773f6cb2fef66cd22acfe1af87316319bdc461508402f5f22241b743a2dc6
4
+ data.tar.gz: dd0907a3c9ffcbd56f7cb3d15e1cddb959737982851a1b1264521b609fa6b864
5
+ SHA512:
6
+ metadata.gz: d942867cdfc345c4707fb7d75d47276fee2ff8b4a96b9741fdc271c61ce5bc0475799d0b89c9cf3dc4e16568154f2dcb02f26091c99046afb7491aca72cc125f
7
+ data.tar.gz: c011c4fa50a6064099bd2d5c5aa70337b1e373962936133acdc385e5e11b8abc8c1dece8a0d95df0571c66fb5215979eb15e5973501574fc7bbe10c426e7ffbe
data/README.md ADDED
@@ -0,0 +1,245 @@
1
+ ![image](https://user-images.githubusercontent.com/1453680/143582241-f44bd8c6-c242-48f4-8f9a-ed5507948588.png)
2
+ # Urlbox Ruby Library
3
+ The Urlbox Ruby gem provides easy access to the <a href="https://urlbox.io/" target="_blank">Urlbox website screenshot API</a> from your Ruby/Rails application.
4
+
5
+ Now there's no need to muck around with http clients, etc...
6
+
7
+ Just initialise the Urlbox::Client and make a screenshot of a URL in seconds.
8
+
9
+
10
+ ## Documentation
11
+
12
+ See the <a href=https://urlbox.io/docs/overview target="_blank">Urlbox API Docs</a>.
13
+
14
+ ## Requirements
15
+
16
+ Ruby 2.5 and above.
17
+
18
+ ## Installation
19
+
20
+ Add this line to your application's Gemfile:
21
+
22
+ ```ruby
23
+ gem 'urlbox'
24
+ ```
25
+
26
+ And then run:
27
+ ```
28
+ $ bundle install
29
+ ```
30
+ Or just...
31
+ ```
32
+ $ gem install urlbox
33
+ ```
34
+
35
+ ## Usage
36
+
37
+ First, grab your Urlbox API key and API secret* found in your <a href="https://urlbox.io/dashboard/api" target="_blank">Urlbox Dashboard</a>.
38
+
39
+ *\* Requests will be automatically authenticated when you supply your API secret.*
40
+
41
+ ### Quick Start: Generate a Screenshot URL
42
+ For use directly in HTML templates, the browser etc.
43
+
44
+ ```ruby
45
+ require 'urlbox/client'
46
+
47
+ # Initialise the UrlboxClient
48
+ urlbox_client = Urlbox::Client.new(api_key: 'YOUR_API_KEY', api_secret: 'YOUR_API_SECRET')
49
+
50
+ # Generate a screenshot url
51
+ screenshot_url = urlbox_client.generate_url({url: 'http://example.com/'})
52
+
53
+ ```
54
+
55
+ In your erb/html template, use the screenshot_url generated above:
56
+ ```html
57
+ <%= image_tag screenshot_url %>
58
+ ```
59
+
60
+ ### Quick Start: Quickly Get a Screenshot of a URL
61
+ ```ruby
62
+ require 'urlbox/client'
63
+
64
+ urlbox_client = Urlbox::Client.newpay(api_key: 'YOUR_API_KEY', api_secret: 'YOUR_API_SECRET')
65
+
66
+ # Make a request to the UrlBox API
67
+ response = Urlbox::Client.get({url: 'http://example.com/'})
68
+
69
+ # Save your screenshot image to screenshot.png:
70
+ File.write('screenshot.png', response.content)
71
+ ```
72
+
73
+ All UrlboxClient methods require at least one argument: a hash that *must include either a "url", or "html" entry*, which the Urlbox API will render as a screenshot.
74
+
75
+ Additional options in the dictionary include:
76
+
77
+ "format" can be either: png, jpg or jpeg, avif, webp ,pdf, svg, html *(defaults to png if not provided).*
78
+
79
+ "full_page", "width", and many more.
80
+ See all available options here: https://urlbox.io/docs/options
81
+
82
+ eg:
83
+ ```ruby
84
+ {url: 'http://example.com/', full_page: true, width: 300}
85
+ ```
86
+
87
+
88
+ ### A More Extensive Get Request
89
+ ```ruby
90
+ options = {
91
+ url: "https://www.independent.co.uk/arts-entertainment/tv/news/squid-game-real-youtube-mrbeast-b1964007.html",
92
+ format: 'jpg',
93
+ full_page: false,
94
+ hide_cookie_banners: true,
95
+ block_ads: true
96
+ }
97
+
98
+ response = urlbox_client.get(options)
99
+
100
+ # The Urlbox API will return binary data as the response with the
101
+ # Content-Type header set to the relevant mime-type for the format requested.
102
+ # For example, if you requested jpg format, the Content-Type will be image/jpeg
103
+ # and response body will be the actual jpg binary data.
104
+
105
+ response.content # Your screenshot as binary image data which looks like 👇
106
+ ```
107
+ ![image](https://user-images.githubusercontent.com/1453680/143479491-78d8edbc-dfdc-48e3-9ae0-3b59bcf98e2c.png)
108
+
109
+
110
+ ## Other Methods/Requests
111
+ The UrlboxClient has the following public methods:
112
+
113
+ ### get(options)
114
+ *(as detailed in the above examples)*
115
+ Makes a GET request to the Urlbox API to create a screenshot for the url or html passed in the options dictionary.
116
+
117
+ Example request:
118
+ ```ruby
119
+ response = urlbox_client.get({url: 'http://example.com/'})
120
+ response.content # Your screenshot 🎉
121
+ ```
122
+
123
+ ### delete(options)
124
+ Removes a previously created screenshot from the cache.
125
+
126
+ Example request:
127
+ ```ruby
128
+ urlbox_client.delete({url: 'http://example.com/'})
129
+ ```
130
+ ### head(options)
131
+ If you just want to get the response status/headers without pulling down the full response body.
132
+
133
+ Example request:
134
+ ```ruby
135
+ response = urlbox_client.head({url: 'http://example.com/'})
136
+
137
+ puts(response.headers.to_s)
138
+
139
+ ```
140
+
141
+ Example response headers:
142
+
143
+ ```json
144
+ {
145
+ "Date":"Fri, 26 Nov 2021 16:22:56 GMT",
146
+ "Content-Type":"image/png",
147
+ "Content-Length":"1268491",
148
+ "Connection":"keep-alive",
149
+ "Cache-Control":"public, max-age=2592000",
150
+ "Expires":"Sun, 26 Dec 2021 16:16:09 GMT",
151
+ "Last-Modified":"Fri, 26 Nov 2021 16:14:56 GMT",
152
+ "X-Renders-Used":"60",
153
+ "X-Renders-Reset":"Sun Dec 05 2021 09:58:00 GMT+0000 (Coordinated Universal Time)",
154
+ "X-Renders-Allowed":"22000"
155
+ }
156
+ ```
157
+ You can use these headers to check how many renders you have used or your current rate limiting status, etc.
158
+
159
+ ### post(options)
160
+ Uses Urlbox's webhook functionality to initialise a render of a screenshot. You will need to provide a *"webhook_url"* entry in the options which Urlbox will post back to when the rendering of the screenshot is complete.
161
+
162
+ Example request:
163
+ ```ruby
164
+ urlbox_client.post({url: "http://twitter.com/", webhook_url: "http://yoursite.com/webhook"})
165
+ ```
166
+ Give it a couple of seconds, and you should receive, posted to the webhook_url specified in your request above, a post request with a JSON body similar to:
167
+ ```json
168
+ {
169
+ "event": "render.succeeded",
170
+ "renderId": "2cf5ffe2-7736-4d41-8c30-f13e16d35248",
171
+ "result": {
172
+ "renderUrl": "https://renders.urlbox.io/urlbox1/renders/61431b47b8538a00086c29dd/2021/11/25/e2dcec18-8353-435c-ba17-b549c849eec5.png"
173
+ },
174
+ "meta": {
175
+ "startTime": "2021-11-25T16:32:32.453Z",
176
+ "endTime": "2021-11-25T16:32:38.719Z"
177
+ }
178
+ }
179
+ ```
180
+ You can then parse the renderUrl value to access the your screenshot.
181
+
182
+
183
+ ## Secure Webhook Posts
184
+ The Urlbox API post to your webhook endpoint will include a header that you can use to ensure this is a genuine request from the Urlbox API, and not a malicious actor.
185
+
186
+ Using your http client of choice, access the *x-urlbox-signature* header. Its value will be something similar to:
187
+
188
+ `t=1637857959,sha256=1d721f99aa03122d494f8b49f201fdf806efaec609c614f0a0ec7b394f1d403a`
189
+
190
+ Use the *webhook_validator* helper function that is included, for no extra charge, in the urlbox package to verify that the webhook post is indeed a genuine request from the Urlbox API. Like so:
191
+
192
+ ```ruby
193
+ require 'urlbox/webhook_validator'
194
+
195
+ # extracted from the x-urlbox-signature header
196
+ header_signature = "t=1637857959,sha256=1d721f..."
197
+
198
+ # the raw JSON payload from the webhook request body
199
+ payload = {
200
+ "event": "render.succeeded",
201
+ "renderId": "794383cd-b09e-4aef-a12b-fadf8aad9d63",
202
+ "result": {
203
+ "renderUrl": "https://renders.urlbox.io/urlbox1/renders/foo.png"
204
+ },
205
+ "meta": {
206
+ "startTime": "2021-11-24T16:49:48.307Z",
207
+ "endTime": "2021-11-24T16:49:53.659Z",
208
+ },
209
+ }
210
+
211
+ # Your webhook secret - coming soon.
212
+ # NB: This is NOT your api_secret, that's different.
213
+ webhook_secret = "YOUR_WEBHOOK_SECRET"
214
+
215
+ # This will either return true (if the signature is genuinely from Urlbox)
216
+ # or it will raise a InvalidHeaderSignatureError (if the signature is not from Urlbox)
217
+ Urlbox::WebhookValidator.call(header_signature, payload, webhook_secret)
218
+ ```
219
+
220
+ ## Using Env Vars?
221
+
222
+ If you are using env vars, in your .env file, set:
223
+ ```yaml
224
+ URLBOX_API_KEY: YOUR_URLBOX_API_KEY
225
+ URLBOX_API_SECRET: YOUR_URLBOX_API_SECRET
226
+ URLBOX_API_HOST_NAME: YOUR_URLBOX_API_HOST_NAME # (optional, advanced usage)
227
+ ```
228
+
229
+ Then the Urlbox::Client will pick these up and you can use all the above Urlbox::Client class methods directly, without having to initialise the Urlbox::Client.
230
+ Eg:
231
+
232
+ ```ruby
233
+ require 'urlbox/client'
234
+
235
+ screenshot_url = Urlbox::Client.generate_url({url: "http://example.com/"})
236
+
237
+ Urlbox::Client.get(...)
238
+ Urlbox::Client.head(...)
239
+ # etc
240
+
241
+ ```
242
+ ## Feedback
243
+
244
+
245
+ Feel free to contact us if you spot a bug or have any suggestions at: support`[at]`urlbox.io.
@@ -0,0 +1,129 @@
1
+ require 'http'
2
+ require 'urlbox/errors'
3
+
4
+ module Urlbox
5
+ class Client
6
+ BASE_API_URL = 'https://api.urlbox.io/v1/'.freeze
7
+ POST_END_POINT = 'render'.freeze
8
+
9
+ def initialize(api_key: nil, api_secret: nil, api_host_name: nil)
10
+ if api_key.nil? && ENV['URLBOX_API_KEY'].nil?
11
+ raise Urlbox::Error, "Missing api_key or ENV['URLBOX_API_KEY'] not set"
12
+ end
13
+
14
+ @api_key = api_key || ENV['URLBOX_API_KEY']
15
+ @api_secret = api_secret || ENV['URLBOX_API_SECRET']
16
+ @base_api_url = init_base_api_url(api_host_name)
17
+ end
18
+
19
+ def get(options)
20
+ HTTP.timeout(100).follow.get(generate_url(options))
21
+ end
22
+
23
+ def delete(options)
24
+ processed_options, format = process_options(options)
25
+ HTTP.delete("#{@base_api_url}#{@api_key}/#{format}?#{processed_options}")
26
+ end
27
+
28
+ def head(options)
29
+ processed_options, format = process_options(options)
30
+ HTTP.timeout(100)
31
+ .follow
32
+ .head("#{@base_api_url}#{@api_key}/#{format}?#{processed_options}")
33
+ end
34
+
35
+ def post(options)
36
+ raise Urlbox::Error, Urlbox::Error.missing_api_secret_error_message if @api_secret.nil?
37
+
38
+ unless options.key?(:webhook_url)
39
+ warn('webhook_url not supplied, you will need to poll the statusUrl in order to get your result')
40
+ end
41
+
42
+ processed_options, _format = process_options_post_request(options)
43
+ HTTP.timeout(5)
44
+ .headers('Content-Type': 'application/json', 'Authorization': "Bearer #{@api_secret}")
45
+ .post("#{@base_api_url}#{POST_END_POINT}", json: processed_options)
46
+ end
47
+
48
+ def generate_url(options)
49
+ processed_options, format = process_options(options)
50
+
51
+ if @api_secret
52
+ "#{@base_api_url}" \
53
+ "#{@api_key}/#{token(processed_options)}/#{format}" \
54
+ "?#{processed_options}"
55
+ else
56
+ "#{@base_api_url}" \
57
+ "#{@api_key}/#{format}" \
58
+ "?#{processed_options}"
59
+ end
60
+ end
61
+
62
+ # class methods to allow easy env var based usage
63
+ class << self
64
+ %i[delete head get post generate_url].each do |method|
65
+ define_method(method) do |options|
66
+ new.send(method, options)
67
+ end
68
+ end
69
+ end
70
+
71
+ private
72
+
73
+ def init_base_api_url(api_host_name)
74
+ if api_host_name
75
+ "https://#{api_host_name}/"
76
+ elsif ENV['URLBOX_API_HOST_NAME']
77
+ ENV['URLBOX_API_HOST_NAME']
78
+ else
79
+ BASE_API_URL
80
+ end
81
+ end
82
+
83
+ def prepend_schema(url)
84
+ url.start_with?('http') ? url : "http://#{url}"
85
+ end
86
+
87
+ def process_options(options, url_encode_options: true)
88
+ processed_options = options.transform_keys(&:to_sym)
89
+
90
+ raise_key_error_if_missing_required_keys(processed_options)
91
+
92
+ processed_options[:url] = process_url(processed_options[:url]) if processed_options[:url]
93
+ processed_options[:format] = processed_options.fetch(:format, 'png')
94
+
95
+ if url_encode_options
96
+ [URI.encode_www_form(processed_options), processed_options[:format]]
97
+ else
98
+ [processed_options, processed_options[:format]]
99
+ end
100
+ end
101
+
102
+ def process_options_post_request(options)
103
+ process_options(options, url_encode_options: false)
104
+ end
105
+
106
+ def process_url(url)
107
+ url_parsed = prepend_schema(url.strip)
108
+
109
+ raise Urlbox::Error, "Invalid URL: #{url_parsed}" unless valid_url?(url_parsed)
110
+
111
+ url_parsed
112
+ end
113
+
114
+ def raise_key_error_if_missing_required_keys(options)
115
+ return unless options[:url].nil? && options[:html].nil?
116
+
117
+ raise Urlbox::Error, 'Missing url or html entry in options'
118
+ end
119
+
120
+ def token(url_encoded_options)
121
+ OpenSSL::HMAC.hexdigest('sha1', @api_secret.encode('UTF-8'), url_encoded_options.encode('UTF-8'))
122
+ end
123
+
124
+ def valid_url?(url)
125
+ parsed_url = URI.parse(url)
126
+ !parsed_url.host.nil? && parsed_url.host.include?('.')
127
+ end
128
+ end
129
+ end
@@ -0,0 +1,12 @@
1
+ module Urlbox
2
+ class Error < StandardError
3
+ def self.missing_api_secret_error_message
4
+ <<-ERROR_MESSAGE
5
+ Missing api_secret when initialising client or ENV['URLBOX_API_SECRET'] not set.
6
+ Required for authorised post request.
7
+ ERROR_MESSAGE
8
+ end
9
+ end
10
+
11
+ class InvalidHeaderSignatureError < StandardError; end
12
+ end
@@ -0,0 +1,53 @@
1
+ require 'urlbox/errors'
2
+
3
+ module Urlbox
4
+ class WebhookValidator
5
+ SIGNATURE_REGEX = /^sha256=[0-9a-zA-Z]{40,}$/.freeze
6
+ TIMESTAMP_REGEX = /^t=[0-9]+$/.freeze
7
+ WEBHOOK_AGE_MAX_MINUTES = 5
8
+
9
+ class << self
10
+ def call(header_signature, payload, webhook_secret)
11
+ timestamp, signature = header_signature.split(',')
12
+
13
+ check_timestamp(timestamp)
14
+ check_signature(signature, timestamp, payload, webhook_secret)
15
+
16
+ true
17
+ end
18
+
19
+ def check_signature(raw_signature, timestamp, payload, webhook_secret)
20
+ raise Urlbox::InvalidHeaderSignatureError, 'Invalid signature' unless SIGNATURE_REGEX.match?(raw_signature)
21
+
22
+ signature_webhook = raw_signature.split('=')[1]
23
+ timestamp_parsed = timestamp.split('=')[1]
24
+ signature_generated =
25
+ OpenSSL::HMAC.hexdigest('sha256',
26
+ webhook_secret.encode('UTF-8'),
27
+ "#{timestamp_parsed}.#{JSON.dump(payload).encode('UTF-8')}")
28
+
29
+ raise Urlbox::InvalidHeaderSignatureError, 'Invalid signature' unless signature_generated == signature_webhook
30
+ end
31
+
32
+ def check_timestamp(raw_timestamp)
33
+ raise Urlbox::InvalidHeaderSignatureError, 'Invalid timestamp' unless TIMESTAMP_REGEX.match?(raw_timestamp)
34
+
35
+ timestamp = (raw_timestamp.split('=')[1]).to_i
36
+
37
+ check_webhook_creation_time(timestamp)
38
+ end
39
+
40
+ def check_webhook_creation_time(header_timestamp)
41
+ current_timestamp = Time.now.to_i
42
+ webhook_posted = current_timestamp - header_timestamp
43
+ webhook_posted_minutes_ago = webhook_posted / 60
44
+
45
+ if webhook_posted_minutes_ago > WEBHOOK_AGE_MAX_MINUTES
46
+ raise Urlbox::InvalidHeaderSignatureError, 'Invalid timestamp'
47
+ end
48
+ end
49
+ end
50
+
51
+ private_class_method :check_signature, :check_timestamp, :check_webhook_creation_time
52
+ end
53
+ end
metadata ADDED
@@ -0,0 +1,47 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: urlbox
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Alan Donohoe
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2021-12-06 00:00:00.000000000 Z
12
+ dependencies: []
13
+ description:
14
+ email:
15
+ - alan@urlbox.io
16
+ executables: []
17
+ extensions: []
18
+ extra_rdoc_files: []
19
+ files:
20
+ - README.md
21
+ - lib/urlbox/client.rb
22
+ - lib/urlbox/errors.rb
23
+ - lib/urlbox/webhook_validator.rb
24
+ homepage: https://www.urlbox.io
25
+ licenses:
26
+ - MIT
27
+ metadata: {}
28
+ post_install_message:
29
+ rdoc_options: []
30
+ require_paths:
31
+ - lib
32
+ required_ruby_version: !ruby/object:Gem::Requirement
33
+ requirements:
34
+ - - ">="
35
+ - !ruby/object:Gem::Version
36
+ version: '2.5'
37
+ required_rubygems_version: !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: '0'
42
+ requirements: []
43
+ rubygems_version: 3.2.3
44
+ signing_key:
45
+ specification_version: 4
46
+ summary: Ruby wrapper for the Urlbox API
47
+ test_files: []