RubyGems - telvue-prerender-rails - Versions diffs - 1.6.2 - Mend

telvue-prerender-rails 1.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (12) hide show

checksums.yaml +7 -0
data/.gitignore +17 -0
data/.travis.yml +3 -0
data/Gemfile +4 -0
data/LICENSE.txt +22 -0
data/README.md +176 -0
data/Rakefile +12 -0
data/lib/prerender_rails.rb +254 -0
data/prerender_rails.gemspec +24 -0
data/test/lib/prerender_rails.rb +224 -0
data/test/test_helper.rb +4 -0
metadata +129 -0

checksums.yaml ADDED

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 0171b8209477afcf52a4d850b8fc1d5f78543550532a0f3e1214a77ead6b9074
+  data.tar.gz: 599fd645634d6813200041a6048aec9d1055f0a0efe0e401d053bd210bb9acea
+SHA512:
+  metadata.gz: e775e031b72d27cc405466e786bb85634feb8a31c8c95150e0255d5492300cff8df9ca37ad576a7895138cdcac854b4f8a5a7516f0baa966f1a4cbc74b42ca9d
+  data.tar.gz: 0120664a13c7c77b05858f83ad4473396af39edff61ff348c190dcb50339295bf530db9f944ba26d3f60a5414dd430b75724c19e4f77fa561082851c2ed721f2

data/.gitignore ADDED

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/.travis.yml ADDED

@@ -0,0 +1,3 @@
+language: ruby
+rvm:
+  - 2.2.2

data/Gemfile ADDED

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+# Specify your gem's dependencies in prerender_rails.gemspec
+gemspec

data/LICENSE.txt ADDED

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Todd Hooper
+MIT License
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md ADDED

@@ -0,0 +1,176 @@
+Prerender Rails [![Build Status](https://travis-ci.org/prerender/prerender_rails.png)](https://travis-ci.org/prerender/prerender_rails) [![Gem Version](https://badge.fury.io/rb/prerender_rails.png)](http://badge.fury.io/rb/prerender_rails)
+===========================
+Google, Facebook, Twitter, Yahoo, and Bing are constantly trying to view your website... but they don't execute javascript. That's why we built Prerender. Prerender is perfect for AngularJS SEO, BackboneJS SEO, EmberJS SEO, and any other javascript framework.
+This middleware intercepts requests to your Rails website from crawlers, and then makes a call to the (external) Prerender Service to get the static HTML instead of the javascript for that page.
+Prerender adheres to google's `_escaped_fragment_` proposal, which we recommend you use. It's easy:
+- Just add &lt;meta name="fragment" content="!"> to the &lt;head> of all of your pages
+- If you use hash urls (#), change them to the hash-bang (#!)
+- That's it! Perfect SEO on javascript pages.
+`Note` Make sure you have more than one webserver thread/process running because the prerender service will make a request to your server to render the HTML.
+Add this line to your application's Gemfile:
+    gem 'prerender_rails'
+And in `config/environment/production.rb`, add this line:
+```ruby
+	config.middleware.use Rack::Prerender
+```
+or if you have an account on [prerender.io](http://prerender.io) and want to use your token:
+```ruby
+	config.middleware.use Rack::Prerender, prerender_token: 'YOUR_TOKEN'
+```
+`Note` If you're testing locally, you'll need to run the [prerender server](https://github.com/prerender/prerender) locally so that it has access to your server.
+## Testing
+When testing make sure you're not using a single threaded application server like default WEBrick one, use Puma or Unicorn.
+If your URLs use a hash-bang:
+	If you want to see `http://localhost:5000/#!/profiles/1234`
+	Then go to `http://localhost:5000/?_escaped_fragment_=/profiles/1234`
+If your URLs use push-state:
+	If you want to see `http://localhost:5000/profiles/1234`
+	Then go to `http://localhost:5000/profiles/1234?_escaped_fragment_=`
+## How it works
+1. The middleware checks to make sure we should show a prerendered page
+	1. The middleware checks if the request is from a crawler (`_escaped_fragment_` or agent string)
+	2. The middleware checks to make sure we aren't requesting a resource (js, css, etc...)
+	3. (optional) The middleware checks to make sure the url is in the whitelist
+	4. (optional) The middleware checks to make sure the url isn't in the blacklist
+2. The middleware makes a `GET` request to the [prerender service](https://github.com/prerender/prerender)(phantomjs server) for the page's prerendered HTML
+3. Return that HTML to the crawler
+# Customization
+### Whitelist
+Whitelist a single url path or multiple url paths. Compares using regex, so be specific when possible. If a whitelist is supplied, only url's containing a whitelist path will be prerendered.
+```ruby
+config.middleware.use Rack::Prerender, whitelist: '^/search'
+```
+```ruby
+config.middleware.use Rack::Prerender, whitelist: ['/search', '/users/.*/profile']
+```
+### Blacklist
+Blacklist a single url path or multiple url paths. Compares using regex, so be specific when possible. If a blacklist is supplied, all url's will be prerendered except ones containing a blacklist path.
+```ruby
+config.middleware.use Rack::Prerender, blacklist: '^/search'
+```
+```ruby
+config.middleware.use Rack::Prerender, blacklist: ['/search', '/users/.*/profile']
+```
+### before_render
+This method is intended to be used for caching, but could be used to save analytics or anything else you need to do for each crawler request. If you return a string from before_render, the middleware will serve that to the crawler instead of making a request to the prerender service.
+```ruby
+config.middleware.use Rack::Prerender,
+	before_render: (Proc.new do |env|
+		# do whatever you need to do.
+	end)
+```
+### after_render
+This method is intended to be used for caching, but could be used to save analytics or anything else you need to do for each crawler request. This method is a noop and is called after the prerender service returns HTML.
+```ruby
+config.middleware.use Rack::Prerender,
+	after_render: (Proc.new do |env, response|
+		# do whatever you need to do.
+	end)
+```
+### build_rack_response_from_prerender
+This method is intended to be used to modify the response before it is sent to the crawler. Use this method to add/remove response headers, or do anything else before the request is sent.
+```ruby
+config.middleware.use Rack::Prerender,
+	build_rack_response_from_prerender: (Proc.new do |response, prerender_response|
+		# response is already populated with the prerender status code, html, and headers
+		# prerender_response is the response that came back from the prerender service
+	end)
+```
+### protocol
+Option to hard-set the protocol for Prerender accessing your site instead of the middleware figuring out the protocol based on the request.
+```ruby
+config.middleware.use Rack::Prerender, protocol: 'https'
+```
+## Caching
+This rails middleware is ready to be used with [redis](http://redis.io/) or [memcached](http://memcached.org/) to return prerendered pages in milliseconds.
+When setting up the middleware in `config/environment/production.rb`, you can add a `before_render` method and `after_render` method for caching.
+Here's an example testing a local redis cache:
+_Put this in `config/environment/development.rb`, and add `gem 'redis'` to your Gemfile._
+```ruby
+require 'redis'
+@redis = Redis.new
+config.middleware.use Rack::Prerender,
+  before_render: (Proc.new do |env|
+    @redis.get(Rack::Request.new(env).url)
+  end),
+  after_render: (Proc.new do |env, response|
+    @redis.set(Rack::Request.new(env).url, response.body)
+  end)
+```
+## Using your own prerender service
+We host a Prerender server at [prerender.io](http://prerender.io) so that you can work on more important things, but if you've deployed the prerender service on your own... set the `PRERENDER_SERVICE_URL` environment variable so that this middleware points there instead. Otherwise, it will default to the service already deployed by [prerender.io](http://prerender.io).
+	$ export PRERENDER_SERVICE_URL=<new url>
+Or on heroku:
+	$ heroku config:add PRERENDER_SERVICE_URL=<new url>
+As an alternative, you can pass `prerender_service_url` in the options object during initialization of the middleware
+``` ruby
+config.middleware.use Rack::Prerender, prerender_service_url: '<new url>'
+```
+## License
+The MIT License (MIT)
+Copyright (c) 2013 Todd Hooper &lt;todd@prerender.io&gt;
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/Rakefile ADDED

@@ -0,0 +1,12 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"
+require 'rake/testtask'
+Rake::TestTask.new do |t|
+  t.libs << 'lib/prerender_rails'
+  t.test_files = FileList['test/lib/prerender_rails.rb']
+  t.verbose = true
+end
+task :default => :test

data/lib/prerender_rails.rb ADDED

@@ -0,0 +1,254 @@
+module Rack
+  class Prerender
+    require 'net/http'
+    require 'active_support'
+    def initialize(app, options={})
+      @crawler_user_agents = [
+        'googlebot',
+        'yahoo',
+        'bingbot',
+        'baiduspider',
+        'facebookexternalhit',
+        'twitterbot',
+        'rogerbot',
+        'linkedinbot',
+        'embedly',
+        'bufferbot',
+        'quora link preview',
+        'showyoubot',
+        'outbrain',
+        'pinterest/0.',
+        'developers.google.com/+/web/snippet',
+        'www.google.com/webmasters/tools/richsnippets',
+        'slackbot',
+        'vkShare',
+        'W3C_Validator',
+        'redditbot',
+        'Applebot',
+        'WhatsApp',
+        'flipboard',
+        'tumblr',
+        'bitlybot',
+        'SkypeUriPreview',
+        'nuzzel',
+        'Discordbot',
+        'Google Page Speed',
+        'Qwantify',
+        'Chrome-Lighthouse'
+      ]
+      @extensions_to_ignore = [
+        '.js',
+        '.css',
+        '.xml',
+        '.less',
+        '.png',
+        '.jpg',
+        '.jpeg',
+        '.gif',
+        '.pdf',
+        '.doc',
+        '.txt',
+        '.ico',
+        '.rss',
+        '.zip',
+        '.mp3',
+        '.rar',
+        '.exe',
+        '.wmv',
+        '.doc',
+        '.avi',
+        '.ppt',
+        '.mpg',
+        '.mpeg',
+        '.tif',
+        '.wav',
+        '.mov',
+        '.psd',
+        '.ai',
+        '.xls',
+        '.mp4',
+        '.m4a',
+        '.swf',
+        '.dat',
+        '.dmg',
+        '.iso',
+        '.flv',
+        '.m4v',
+        '.torrent'
+      ]
+      @options = options
+      @options[:whitelist] = [@options[:whitelist]] if @options[:whitelist].is_a? String
+      @options[:blacklist] = [@options[:blacklist]] if @options[:blacklist].is_a? String
+      @extensions_to_ignore = @options[:extensions_to_ignore] if @options[:extensions_to_ignore]
+      @crawler_user_agents = @options[:crawler_user_agents] if @options[:crawler_user_agents]
+      @app = app
+    end
+    def call(env)
+      if should_show_prerendered_page(env)
+        cached_response = before_render(env)
+        if cached_response
+          return cached_response.finish
+        end
+        prerendered_response = get_prerendered_page_response(env)
+        if prerendered_response
+          response = build_rack_response_from_prerender(prerendered_response)
+          after_render(env, prerendered_response)
+          return response.finish
+        end
+      end
+      @app.call(env)
+    end
+    def should_show_prerendered_page(env)
+      user_agent = env['HTTP_USER_AGENT']
+      buffer_agent = env['HTTP_X_BUFFERBOT']
+      prerender_agent = env['HTTP_X_PRERENDER']
+      is_requesting_prerendered_page = false
+      return false if !user_agent
+      return false if env['REQUEST_METHOD'] != 'GET'
+      request = Rack::Request.new(env)
+      is_requesting_prerendered_page = true if Rack::Utils.parse_query(request.query_string).has_key?('_escaped_fragment_')
+      #if it is a bot...show prerendered page
+      is_requesting_prerendered_page = true if @crawler_user_agents.any? { |crawler_user_agent| user_agent.downcase.include?(crawler_user_agent.downcase) }
+      #if it is BufferBot...show prerendered page
+      is_requesting_prerendered_page = true if buffer_agent
+      #if it is Prerender...don't show prerendered page
+      is_requesting_prerendered_page = false if prerender_agent
+      #if it is a bot and is requesting a resource...dont prerender
+      return false if @extensions_to_ignore.any? { |extension| request.fullpath.include? extension }
+      #if it is a bot and not requesting a resource and is not whitelisted...dont prerender
+      return false if @options[:whitelist].is_a?(Array) && @options[:whitelist].all? { |whitelisted| !Regexp.new(whitelisted).match(request.fullpath) }
+      #if it is a bot and not requesting a resource and is blacklisted(url or referer)...dont prerender
+      if @options[:blacklist].is_a?(Array) && @options[:blacklist].any? { |blacklisted|
+          blacklistedUrl = false
+          blacklistedReferer = false
+          regex = Regexp.new(blacklisted)
+          blacklistedUrl = !!regex.match(request.fullpath)
+          blacklistedReferer = !!regex.match(request.referer) if request.referer
+          blacklistedUrl || blacklistedReferer
+        }
+        return false
+      end
+      return is_requesting_prerendered_page
+    end
+    def get_prerendered_page_response(env)
+      begin
+        url = URI.parse(build_api_url(env))
+        headers = {
+          'User-Agent' => env['HTTP_USER_AGENT'],
+          'Accept-Encoding' => 'gzip'
+        }
+        headers['X-Prerender-Token'] = ENV['PRERENDER_TOKEN'] if ENV['PRERENDER_TOKEN']
+        headers['X-Prerender-Token'] = @options[:prerender_token] if @options[:prerender_token]
+        req = Net::HTTP::Get.new(url.request_uri, headers)
+        req.basic_auth(ENV['PRERENDER_USERNAME'], ENV['PRERENDER_PASSWORD']) if @options[:basic_auth]
+        http = Net::HTTP.new(url.host, url.port)
+        http.use_ssl = true if url.scheme == 'https'
+        response = http.request(req)
+        if response['Content-Encoding'] == 'gzip'
+          response.body = ActiveSupport::Gzip.decompress(response.body)
+          response['Content-Length'] = response.body.length
+          response.delete('Content-Encoding')
+        end
+        response
+      rescue
+        nil
+      end
+    end
+    def build_api_url(env)
+      new_env = env
+      if env["CF-VISITOR"]
+        match = /"scheme":"(http|https)"/.match(env['CF-VISITOR'])
+        new_env["HTTPS"] = true and new_env["rack.url_scheme"] = "https" and new_env["SERVER_PORT"] = 443 if (match && match[1] == "https")
+        new_env["HTTPS"] = false and new_env["rack.url_scheme"] = "http" and new_env["SERVER_PORT"] = 80 if (match && match[1] == "http")
+      end
+      if env["X-FORWARDED-PROTO"]
+        new_env["HTTPS"] = true and new_env["rack.url_scheme"] = "https" and new_env["SERVER_PORT"] = 443 if env["X-FORWARDED-PROTO"].split(',')[0] == "https"
+        new_env["HTTPS"] = false and new_env["rack.url_scheme"] = "http" and new_env["SERVER_PORT"] = 80 if env["X-FORWARDED-PROTO"].split(',')[0] == "http"
+      end
+      if @options[:protocol]
+        new_env["HTTPS"] = true and new_env["rack.url_scheme"] = "https" and new_env["SERVER_PORT"] = 443 if @options[:protocol] == "https"
+        new_env["HTTPS"] = false and new_env["rack.url_scheme"] = "http" and new_env["SERVER_PORT"] = 80 if @options[:protocol] == "http"
+      end
+      url = Rack::Request.new(new_env).url
+      prerender_url = get_prerender_service_url()
+      forward_slash = prerender_url[-1, 1] == '/' ? '' : '/'
+      if env['HTTP_USER_AGENT'].include?('facebookexternalhit')
+        if url.include?('?')
+          url = "#{url}&ua=fb_crawler"
+        else
+          url = "#{url}?ua=fb_crawler"
+        end
+      end
+      "#{prerender_url}#{forward_slash}#{url}"
+    end
+    def get_prerender_service_url
+      @options[:prerender_service_url] || ENV['PRERENDER_SERVICE_URL'] || 'http://service.prerender.io/'
+    end
+    def build_rack_response_from_prerender(prerendered_response)
+      response = Rack::Response.new(prerendered_response.body, prerendered_response.code, prerendered_response.header)
+      @options[:build_rack_response_from_prerender].call(response, prerendered_response) if @options[:build_rack_response_from_prerender]
+      response
+    end
+    def before_render(env)
+      return nil unless @options[:before_render]
+      cached_render = @options[:before_render].call(env)
+      if cached_render && cached_render.is_a?(String)
+        Rack::Response.new(cached_render, 200, { 'Content-Type' => 'text/html; charset=utf-8' })
+      elsif cached_render && cached_render.is_a?(Rack::Response)
+        cached_render
+      else
+        nil
+      end
+    end
+    def after_render(env, response)
+      return true unless @options[:after_render]
+      @options[:after_render].call(env, response)
+    end
+  end
+end

data/prerender_rails.gemspec ADDED

@@ -0,0 +1,24 @@
+# coding: utf-8
+Gem::Specification.new do |spec|
+  spec.name          = "telvue-prerender-rails"
+  spec.version       = "1.6.2"
+  spec.authors       = ["Todd Hooper", "Ben Liu"]
+  spec.email         = ["todd@prerender.io", "bliu@telvue.com"]
+  spec.description   = %q{Rails middleware to prerender your javascript heavy pages on the fly by a phantomjs service}
+  spec.summary       = %q{Prerender your backbone/angular/javascript rendered application on the fly when search engines crawl}
+  spec.homepage      = "https://github.com/prerender/prerender_rails"
+  spec.license       = "MIT"
+  spec.files         = `git ls-files`.split($/)
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+  spec.add_dependency 'rack', '>= 0'
+  spec.add_dependency 'activesupport', '>= 0'
+  spec.add_development_dependency "bundler", "~> 1.3"
+  spec.add_development_dependency "rake"
+  spec.add_development_dependency "webmock"
+end

data/test/lib/prerender_rails.rb ADDED

@@ -0,0 +1,224 @@
+require_relative '../test_helper'
+describe Rack::Prerender do
+  bot = 'Baiduspider+(+http://www.baidu.com/search/spider.htm)'
+  user = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.76 Safari/537.36'
+  before :each do
+    @app = lambda do |params|
+      [200, {}, ""]
+    end
+    @prerender = Rack::Prerender.new(@app)
+  end
+  it "should return a prerendered response for a crawler with the returned status code and headers" do
+    request = Rack::MockRequest.env_for "/", "HTTP_USER_AGENT" => bot
+    stub_request(:get, @prerender.build_api_url(request)).with(:headers => { 'User-Agent' => bot }).to_return(:body => "<html></html>", :status => 301, :headers => { 'Location' => 'http://google.com'})
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal response[2].body, ["<html></html>"]
+    assert_equal response[2].status, 301
+    assert_equal( { 'location' => 'http://google.com', 'Content-Length' => '13'}, response[2].headers )
+  end
+  it "should return a prerendered reponse if user is a bot by checking for _escaped_fragment_" do
+    request = Rack::MockRequest.env_for "/path?_escaped_fragment_=", "HTTP_USER_AGENT" => user
+    stub_request(:get, @prerender.build_api_url(request)).with(:headers => { 'User-Agent' => user }).to_return(:body => "<html></html>")
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal ["<html></html>"], response[2].body
+  end
+  it "should continue to app routes if the url is a bad url with _escaped_fragment_" do
+    request = Rack::MockRequest.env_for "/path?query=string?_escaped_fragment_=", "HTTP_USER_AGENT" => user
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if the request is not a GET" do
+    request = Rack::MockRequest.env_for "/path?_escaped_fragment_=", { "HTTP_USER_AGENT" => user, "REQUEST_METHOD" => "POST" }
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if user is not a bot by checking agent string" do
+    request = Rack::MockRequest.env_for "/", "HTTP_USER_AGENT" => user
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if contains X-Prerender header" do
+    request = Rack::MockRequest.env_for "/path?_escaped_fragment_=", "HTTP_USER_AGENT" => user, "HTTP_X_PRERENDER" => "1"
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if user is a bot, but the bot is requesting a resource file" do
+    request = Rack::MockRequest.env_for "/main.js?anyQueryParam=true", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if the url is not part of the regex specific whitelist" do
+    request = Rack::MockRequest.env_for "/saved/search/blah?_escaped_fragment_=", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app, whitelist: ['^/search', '/help']).call(request)
+    assert_equal "", response[2]
+  end
+  it "should set use_ssl to true for https prerender_service_url" do
+    @prerender = Rack::Prerender.new(@app, prerender_service_url: 'https://service.prerender.io/')
+    request = Rack::MockRequest.env_for "/search/things/123/page?_escaped_fragment_=", "HTTP_USER_AGENT" => bot
+    stub_request(:get, @prerender.build_api_url(request)).to_return(:body => "<html></html>")
+    response = @prerender.call(request)
+    assert_equal ["<html></html>"], response[2].body
+  end
+  it "should return a prerendered response if the url is part of the regex specific whitelist" do
+    request = Rack::MockRequest.env_for "/search/things/123/page?_escaped_fragment_=", "HTTP_USER_AGENT" => bot
+    stub_request(:get, @prerender.build_api_url(request)).to_return(:body => "<html></html>")
+    response = Rack::Prerender.new(@app, whitelist: ['^/search.*page', '/help']).call(request)
+    assert_equal ["<html></html>"], response[2].body
+  end
+  it "should continue to app routes if the url is part of the regex specific blacklist" do
+    request = Rack::MockRequest.env_for "/search/things/123/page", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app, blacklist: ['^/search', '/help']).call(request)
+    assert_equal "", response[2]
+  end
+  it "should continue to app routes if the hashbang url is part of the regex specific blacklist" do
+    request = Rack::MockRequest.env_for "?_escaped_fragment_=/search/things/123/page", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app, blacklist: ['/search', '/help']).call(request)
+    assert_equal "", response[2]
+  end
+  it "should return a prerendered response if the url is not part of the regex specific blacklist" do
+    request = Rack::MockRequest.env_for "/profile/search/blah", "HTTP_USER_AGENT" => bot
+    stub_request(:get, @prerender.build_api_url(request)).to_return(:body => "<html></html>")
+    response = Rack::Prerender.new(@app, blacklist: ['^/search', '/help']).call(request)
+    assert_equal ["<html></html>"], response[2].body
+  end
+  it "should continue to app routes if the referer is part of the regex specific blacklist" do
+    request = Rack::MockRequest.env_for "/api/results", "HTTP_USER_AGENT" => bot, "HTTP_REFERER" => '/search'
+    response = Rack::Prerender.new(@app, blacklist: ['^/search', '/help']).call(request)
+    assert_equal "", response[2]
+  end
+  it "should return a prerendered response if the referer is not part of the regex specific blacklist" do
+    request = Rack::MockRequest.env_for "/api/results", "HTTP_USER_AGENT" => bot, "HTTP_REFERER" => '/profile/search'
+    stub_request(:get, @prerender.build_api_url(request)).to_return(:body => "<html></html>")
+    response = Rack::Prerender.new(@app, blacklist: ['^/search', '/help']).call(request)
+    assert_equal ["<html></html>"], response[2].body
+  end
+  it "should return a prerendered response if a string is returned from before_render" do
+    request = Rack::MockRequest.env_for "/", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app, before_render: Proc.new do |env| '<html>cached</html>' end).call(request)
+    assert_equal ["<html>cached</html>"], response[2].body
+  end
+  it "should return a prerendered response if a response is returned from before_render" do
+    request = Rack::MockRequest.env_for "/", "HTTP_USER_AGENT" => bot
+    response = Rack::Prerender.new(@app, before_render: Proc.new do |env| Rack::Response.new('<html>cached2</html>', 200, { 'test' => 'test2Header'}) end).call(request)
+    assert_equal ["<html>cached2</html>"], response[2].body
+    assert_equal response[2].status, 200
+    assert_equal( { 'test' => 'test2Header', "Content-Length"=>"20"}, response[2].headers )
+  end
+  describe '#buildApiUrl' do
+    it "should build the correct api url with the default url" do
+      request = Rack::MockRequest.env_for "https://google.com/search?q=javascript"
+      ENV['PRERENDER_SERVICE_URL'] = nil
+      assert_equal 'http://service.prerender.io/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    it "should build the correct api url with an environment variable url" do
+      ENV['PRERENDER_SERVICE_URL'] = 'http://prerenderurl.com'
+      request = Rack::MockRequest.env_for "https://google.com/search?q=javascript"
+      assert_equal 'http://prerenderurl.com/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+      ENV['PRERENDER_SERVICE_URL'] = nil
+    end
+    it "should build the correct api url with an initialization variable url" do
+      @prerender = Rack::Prerender.new(@app, prerender_service_url: 'http://prerenderurl.com')
+      request = Rack::MockRequest.env_for "https://google.com/search?q=javascript"
+      assert_equal 'http://prerenderurl.com/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    it "should build the correct https api url with an initialization variable url" do
+      @prerender = Rack::Prerender.new(@app, prerender_service_url: 'https://prerenderurl.com')
+      request = Rack::MockRequest.env_for "https://google.com/search?q=javascript"
+      assert_equal 'https://prerenderurl.com/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    # Check CF-Visitor header in order to Work behind CloudFlare with Flexible SSL (https://support.cloudflare.com/hc/en-us/articles/200170536)
+    it "should build the correct api url for the Cloudflare Flexible SSL support" do
+      request = Rack::MockRequest.env_for "http://google.com/search?q=javascript", { 'CF-VISITOR' => '"scheme":"https"'}
+      ENV['PRERENDER_SERVICE_URL'] = nil
+      assert_equal 'http://service.prerender.io/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    # Check X-Forwarded-Proto because Heroku SSL Support terminates at the load balancer
+    it "should build the correct api url for the Heroku SSL Addon support with single value" do
+      request = Rack::MockRequest.env_for "http://google.com/search?q=javascript", { 'X-FORWARDED-PROTO' => 'https'}
+      ENV['PRERENDER_SERVICE_URL'] = nil
+      assert_equal 'http://service.prerender.io/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    # Check X-Forwarded-Proto because Heroku SSL Support terminates at the load balancer
+    it "should build the correct api url for the Heroku SSL Addon support with double value" do
+      request = Rack::MockRequest.env_for "http://google.com/search?q=javascript", { 'X-FORWARDED-PROTO' => 'https,http'}
+      ENV['PRERENDER_SERVICE_URL'] = nil
+      assert_equal 'http://service.prerender.io/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+    it "should build the correct api url for the protocol option" do
+      @prerender = Rack::Prerender.new(@app, protocol: 'https')
+      request = Rack::MockRequest.env_for "http://google.com/search?q=javascript"
+      ENV['PRERENDER_SERVICE_URL'] = nil
+      assert_equal 'http://service.prerender.io/https://google.com/search?q=javascript', @prerender.build_api_url(request)
+    end
+  end
+end

data/test/test_helper.rb ADDED

@@ -0,0 +1,4 @@
+require 'minitest/autorun'
+require 'webmock/minitest'
+require 'rack'
+require File.expand_path('../../lib/prerender_rails.rb', __FILE__)

metadata ADDED

@@ -0,0 +1,129 @@
+--- !ruby/object:Gem::Specification
+name: telvue-prerender-rails
+version: !ruby/object:Gem::Version
+  version: 1.6.2
+platform: ruby
+authors:
+- Todd Hooper
+- Ben Liu
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2019-10-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rack
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Rails middleware to prerender your javascript heavy pages on the fly
+  by a phantomjs service
+email:
+- todd@prerender.io
+- bliu@telvue.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/prerender_rails.rb
+- prerender_rails.gemspec
+- test/lib/prerender_rails.rb
+- test/test_helper.rb
+homepage: https://github.com/prerender/prerender_rails
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.6
+signing_key:
+specification_version: 4
+summary: Prerender your backbone/angular/javascript rendered application on the fly
+  when search engines crawl
+test_files:
+- test/lib/prerender_rails.rb
+- test/test_helper.rb