urlbox 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: cec773f6cb2fef66cd22acfe1af87316319bdc461508402f5f22241b743a2dc6
+  data.tar.gz: dd0907a3c9ffcbd56f7cb3d15e1cddb959737982851a1b1264521b609fa6b864
+SHA512:
+  metadata.gz: d942867cdfc345c4707fb7d75d47276fee2ff8b4a96b9741fdc271c61ce5bc0475799d0b89c9cf3dc4e16568154f2dcb02f26091c99046afb7491aca72cc125f
+  data.tar.gz: c011c4fa50a6064099bd2d5c5aa70337b1e373962936133acdc385e5e11b8abc8c1dece8a0d95df0571c66fb5215979eb15e5973501574fc7bbe10c426e7ffbe

data/README.md

@@ -0,0 +1,245 @@
+![image](https://user-images.githubusercontent.com/1453680/143582241-f44bd8c6-c242-48f4-8f9a-ed5507948588.png)
+# Urlbox Ruby Library
+The Urlbox Ruby gem provides easy access to the <a href="https://urlbox.io/" target="_blank">Urlbox website screenshot API</a> from your Ruby/Rails application.
+
+Now there's no need to muck around with http clients, etc...
+
+Just initialise the Urlbox::Client and make a screenshot of a URL in seconds.
+
+
+## Documentation
+
+See the <a href=https://urlbox.io/docs/overview target="_blank">Urlbox API Docs</a>.
+
+## Requirements
+
+Ruby 2.5 and above.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'urlbox'
+```
+
+And then run:
+```
+$ bundle install
+```
+Or just...
+```
+$ gem install urlbox
+```
+
+## Usage
+
+First, grab your Urlbox API key and API secret* found in your <a href="https://urlbox.io/dashboard/api" target="_blank">Urlbox Dashboard</a>.
+
+*\* Requests will be automatically authenticated when you supply your API secret.*
+
+### Quick Start:  Generate a Screenshot URL
+For use directly in HTML templates, the browser etc.
+
+```ruby
+require  'urlbox/client'
+
+# Initialise the UrlboxClient
+urlbox_client = Urlbox::Client.new(api_key: 'YOUR_API_KEY', api_secret: 'YOUR_API_SECRET')
+
+# Generate a screenshot url
+screenshot_url = urlbox_client.generate_url({url: 'http://example.com/'})
+
+```
+
+In your erb/html template, use the screenshot_url generated above:
+```html
+<%= image_tag screenshot_url %>
+```
+
+###  Quick Start: Quickly Get a Screenshot of a URL
+```ruby
+require  'urlbox/client'
+
+urlbox_client = Urlbox::Client.newpay(api_key: 'YOUR_API_KEY', api_secret: 'YOUR_API_SECRET')
+
+# Make a request to the UrlBox API
+response = Urlbox::Client.get({url: 'http://example.com/'})
+
+# Save your screenshot image to screenshot.png:
+File.write('screenshot.png', response.content)
+```
+
+All UrlboxClient methods require at least one argument: a hash that *must include either a "url", or "html" entry*, which the Urlbox API will render as a screenshot.
+
+Additional options in the dictionary include:
+
+"format" can be either: png, jpg or jpeg, avif, webp ,pdf, svg, html  *(defaults to png if not provided).*
+
+"full_page", "width", and many more.
+See all available options here: https://urlbox.io/docs/options
+
+eg:
+```ruby
+{url: 'http://example.com/', full_page: true, width: 300}
+```
+
+
+### A More Extensive Get Request
+```ruby
+options = {
+	url: "https://www.independent.co.uk/arts-entertainment/tv/news/squid-game-real-youtube-mrbeast-b1964007.html",
+	format: 'jpg',
+	full_page: false,
+	hide_cookie_banners: true,
+	block_ads: true
+}
+
+response = urlbox_client.get(options)
+
+# The Urlbox API will return binary data as the response with the
+# Content-Type header set to the relevant mime-type for the format requested.
+# For example, if you requested jpg format, the Content-Type will be image/jpeg
+# and response body will be the actual jpg binary data.
+
+response.content # Your screenshot as binary image data which looks like 👇
+```
+![image](https://user-images.githubusercontent.com/1453680/143479491-78d8edbc-dfdc-48e3-9ae0-3b59bcf98e2c.png)
+
+
+## Other Methods/Requests
+The UrlboxClient has the following public methods:
+
+### get(options)
+*(as detailed in the above examples)*
+Makes a GET request to the Urlbox API to create a screenshot for the url or html passed in the options dictionary.
+
+Example request:
+```ruby
+response = urlbox_client.get({url: 'http://example.com/'})
+response.content # Your screenshot 🎉
+```
+
+### delete(options)
+Removes a previously created screenshot from the cache.
+
+Example request:
+```ruby
+urlbox_client.delete({url: 'http://example.com/'})
+```
+### head(options)
+If you just want to get the response status/headers without pulling down the full response body.
+
+Example request:
+```ruby
+response = urlbox_client.head({url: 'http://example.com/'})
+
+puts(response.headers.to_s)
+
+```
+
+Example response headers:
+
+```json
+{
+   "Date":"Fri, 26 Nov 2021 16:22:56 GMT",
+   "Content-Type":"image/png",
+   "Content-Length":"1268491",
+   "Connection":"keep-alive",
+   "Cache-Control":"public, max-age=2592000",
+   "Expires":"Sun, 26 Dec 2021 16:16:09 GMT",
+   "Last-Modified":"Fri, 26 Nov 2021 16:14:56 GMT",
+   "X-Renders-Used":"60",
+   "X-Renders-Reset":"Sun Dec 05 2021 09:58:00 GMT+0000 (Coordinated Universal Time)",
+   "X-Renders-Allowed":"22000"
+}
+```
+You can use these headers to check how many renders you have used or your current rate limiting status, etc.
+
+### post(options)
+Uses Urlbox's webhook functionality to initialise a render of a screenshot. You will need to provide a *"webhook_url"* entry in the options which Urlbox will post back to when the rendering of the screenshot is complete.
+
+Example request:
+```ruby
+urlbox_client.post({url: "http://twitter.com/", webhook_url: "http://yoursite.com/webhook"})
+```
+Give it a couple of seconds, and you should receive, posted to the webhook_url specified in your request above, a post request with a JSON body similar to:
+```json
+{
+  "event": "render.succeeded",
+  "renderId": "2cf5ffe2-7736-4d41-8c30-f13e16d35248",
+  "result": {
+    "renderUrl": "https://renders.urlbox.io/urlbox1/renders/61431b47b8538a00086c29dd/2021/11/25/e2dcec18-8353-435c-ba17-b549c849eec5.png"
+  },
+  "meta": {
+    "startTime": "2021-11-25T16:32:32.453Z",
+    "endTime": "2021-11-25T16:32:38.719Z"
+  }
+}
+```
+You can then parse the renderUrl value to access the your screenshot.
+
+
+## Secure Webhook Posts
+The Urlbox API post to your webhook endpoint will include a header that you can use to  ensure this is a genuine request from the Urlbox API, and not a malicious actor.
+
+Using your http client of choice, access the *x-urlbox-signature* header. Its value will be something similar to:
+
+`t=1637857959,sha256=1d721f99aa03122d494f8b49f201fdf806efaec609c614f0a0ec7b394f1d403a`
+
+Use the *webhook_validator* helper function that is included, for no extra charge, in the urlbox package to verify that the webhook post is indeed a genuine request from the Urlbox API. Like so:
+
+```ruby
+require 'urlbox/webhook_validator'
+
+# extracted from the x-urlbox-signature header
+header_signature = "t=1637857959,sha256=1d721f..."
+
+# the raw JSON payload from the webhook request body
+payload = {
+  "event": "render.succeeded",
+  "renderId": "794383cd-b09e-4aef-a12b-fadf8aad9d63",
+  "result": {
+    "renderUrl": "https://renders.urlbox.io/urlbox1/renders/foo.png"
+  },
+  "meta": {
+    "startTime": "2021-11-24T16:49:48.307Z",
+    "endTime": "2021-11-24T16:49:53.659Z",
+  },
+}
+
+# Your webhook secret - coming soon.
+# NB: This is NOT your api_secret, that's different.
+webhook_secret = "YOUR_WEBHOOK_SECRET"
+
+# This will either return true (if the signature is genuinely from Urlbox)
+#   or it will raise a InvalidHeaderSignatureError (if the signature is not from Urlbox)
+Urlbox::WebhookValidator.call(header_signature, payload, webhook_secret)
+```
+
+## Using Env Vars?
+
+If you are using env vars, in your .env file, set:
+```yaml
+URLBOX_API_KEY: YOUR_URLBOX_API_KEY
+URLBOX_API_SECRET: YOUR_URLBOX_API_SECRET
+URLBOX_API_HOST_NAME: YOUR_URLBOX_API_HOST_NAME # (optional, advanced usage)
+```
+
+Then the Urlbox::Client will pick these up and you can use all the above Urlbox::Client class methods directly, without having to initialise the Urlbox::Client.
+Eg:
+
+```ruby
+require  'urlbox/client'
+
+screenshot_url = Urlbox::Client.generate_url({url: "http://example.com/"})
+
+Urlbox::Client.get(...)
+Urlbox::Client.head(...)
+# etc
+
+```
+## Feedback
+
+
+Feel free to contact us if you spot a bug or have any suggestions at: support`[at]`urlbox.io.

data/lib/urlbox/client.rb

@@ -0,0 +1,129 @@
+require 'http'
+require 'urlbox/errors'
+
+module Urlbox
+  class Client
+    BASE_API_URL = 'https://api.urlbox.io/v1/'.freeze
+    POST_END_POINT = 'render'.freeze
+
+    def initialize(api_key: nil, api_secret: nil, api_host_name: nil)
+      if api_key.nil? && ENV['URLBOX_API_KEY'].nil?
+        raise Urlbox::Error, "Missing api_key or ENV['URLBOX_API_KEY'] not set"
+      end
+
+      @api_key = api_key || ENV['URLBOX_API_KEY']
+      @api_secret = api_secret || ENV['URLBOX_API_SECRET']
+      @base_api_url = init_base_api_url(api_host_name)
+    end
+
+    def get(options)
+      HTTP.timeout(100).follow.get(generate_url(options))
+    end
+
+    def delete(options)
+      processed_options, format = process_options(options)
+      HTTP.delete("#{@base_api_url}#{@api_key}/#{format}?#{processed_options}")
+    end
+
+    def head(options)
+      processed_options, format = process_options(options)
+      HTTP.timeout(100)
+          .follow
+          .head("#{@base_api_url}#{@api_key}/#{format}?#{processed_options}")
+    end
+
+    def post(options)
+      raise Urlbox::Error, Urlbox::Error.missing_api_secret_error_message if @api_secret.nil?
+
+      unless options.key?(:webhook_url)
+        warn('webhook_url not supplied, you will need to poll the statusUrl in order to get your result')
+      end
+
+      processed_options, _format = process_options_post_request(options)
+      HTTP.timeout(5)
+          .headers('Content-Type': 'application/json', 'Authorization': "Bearer #{@api_secret}")
+          .post("#{@base_api_url}#{POST_END_POINT}", json: processed_options)
+    end
+
+    def generate_url(options)
+      processed_options, format = process_options(options)
+
+      if @api_secret
+        "#{@base_api_url}" \
+        "#{@api_key}/#{token(processed_options)}/#{format}" \
+        "?#{processed_options}"
+      else
+        "#{@base_api_url}" \
+        "#{@api_key}/#{format}" \
+        "?#{processed_options}"
+      end
+    end
+
+    # class methods to allow easy env var based usage
+    class << self
+      %i[delete head get post generate_url].each do |method|
+        define_method(method) do |options|
+          new.send(method, options)
+        end
+      end
+    end
+
+    private
+
+    def init_base_api_url(api_host_name)
+      if api_host_name
+        "https://#{api_host_name}/"
+      elsif ENV['URLBOX_API_HOST_NAME']
+        ENV['URLBOX_API_HOST_NAME']
+      else
+        BASE_API_URL
+      end
+    end
+
+    def prepend_schema(url)
+      url.start_with?('http') ? url : "http://#{url}"
+    end
+
+    def process_options(options, url_encode_options: true)
+      processed_options = options.transform_keys(&:to_sym)
+
+      raise_key_error_if_missing_required_keys(processed_options)
+
+      processed_options[:url] = process_url(processed_options[:url]) if processed_options[:url]
+      processed_options[:format] = processed_options.fetch(:format, 'png')
+
+      if url_encode_options
+        [URI.encode_www_form(processed_options), processed_options[:format]]
+      else
+        [processed_options, processed_options[:format]]
+      end
+    end
+
+    def process_options_post_request(options)
+      process_options(options, url_encode_options: false)
+    end
+
+    def process_url(url)
+      url_parsed = prepend_schema(url.strip)
+
+      raise Urlbox::Error, "Invalid URL: #{url_parsed}" unless valid_url?(url_parsed)
+
+      url_parsed
+    end
+
+    def raise_key_error_if_missing_required_keys(options)
+      return unless options[:url].nil? && options[:html].nil?
+
+      raise Urlbox::Error, 'Missing url or html entry in options'
+    end
+
+    def token(url_encoded_options)
+      OpenSSL::HMAC.hexdigest('sha1', @api_secret.encode('UTF-8'), url_encoded_options.encode('UTF-8'))
+    end
+
+    def valid_url?(url)
+      parsed_url = URI.parse(url)
+      !parsed_url.host.nil? && parsed_url.host.include?('.')
+    end
+  end
+end

data/lib/urlbox/errors.rb

@@ -0,0 +1,12 @@
+module Urlbox
+  class Error < StandardError
+    def self.missing_api_secret_error_message
+      <<-ERROR_MESSAGE
+        Missing api_secret when initialising client or ENV['URLBOX_API_SECRET'] not set.
+        Required for authorised post request.
+      ERROR_MESSAGE
+    end
+  end
+
+  class InvalidHeaderSignatureError < StandardError; end
+end

data/lib/urlbox/webhook_validator.rb

@@ -0,0 +1,53 @@
+require 'urlbox/errors'
+
+module Urlbox
+  class WebhookValidator
+    SIGNATURE_REGEX = /^sha256=[0-9a-zA-Z]{40,}$/.freeze
+    TIMESTAMP_REGEX = /^t=[0-9]+$/.freeze
+    WEBHOOK_AGE_MAX_MINUTES = 5
+
+    class << self
+      def call(header_signature, payload, webhook_secret)
+        timestamp, signature = header_signature.split(',')
+
+        check_timestamp(timestamp)
+        check_signature(signature, timestamp, payload, webhook_secret)
+
+        true
+      end
+
+      def check_signature(raw_signature, timestamp, payload, webhook_secret)
+        raise Urlbox::InvalidHeaderSignatureError, 'Invalid signature' unless SIGNATURE_REGEX.match?(raw_signature)
+
+        signature_webhook = raw_signature.split('=')[1]
+        timestamp_parsed = timestamp.split('=')[1]
+        signature_generated =
+          OpenSSL::HMAC.hexdigest('sha256',
+                                  webhook_secret.encode('UTF-8'),
+                                  "#{timestamp_parsed}.#{JSON.dump(payload).encode('UTF-8')}")
+
+        raise Urlbox::InvalidHeaderSignatureError, 'Invalid signature' unless signature_generated == signature_webhook
+      end
+
+      def check_timestamp(raw_timestamp)
+        raise Urlbox::InvalidHeaderSignatureError, 'Invalid timestamp' unless TIMESTAMP_REGEX.match?(raw_timestamp)
+
+        timestamp = (raw_timestamp.split('=')[1]).to_i
+
+        check_webhook_creation_time(timestamp)
+      end
+
+      def check_webhook_creation_time(header_timestamp)
+        current_timestamp = Time.now.to_i
+        webhook_posted = current_timestamp - header_timestamp
+        webhook_posted_minutes_ago = webhook_posted / 60
+
+        if webhook_posted_minutes_ago > WEBHOOK_AGE_MAX_MINUTES
+          raise Urlbox::InvalidHeaderSignatureError, 'Invalid timestamp'
+        end
+      end
+    end
+
+    private_class_method :check_signature, :check_timestamp, :check_webhook_creation_time
+  end
+end

metadata

@@ -0,0 +1,47 @@
+--- !ruby/object:Gem::Specification
+name: urlbox
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Alan Donohoe
+autorequire:
+bindir: exe
+cert_chain: []
+date: 2021-12-06 00:00:00.000000000 Z
+dependencies: []
+description:
+email:
+- alan@urlbox.io
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/urlbox/client.rb
+- lib/urlbox/errors.rb
+- lib/urlbox/webhook_validator.rb
+homepage: https://www.urlbox.io
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '2.5'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.2.3
+signing_key:
+specification_version: 4
+summary: Ruby wrapper for the Urlbox API
+test_files: []