RubyGems - PageRankr - Versions diffs - 3.2.1 → 4.0.0 - Mend

PageRankr 3.2.1 → 4.0.0

Files changed (21) hide show

data/CHANGELOG.md +4 -0
data/PageRankr.gemspec +2 -1
data/README.md +19 -10
data/Rakefile +6 -1
data/lib/PageRankr.rb +1 -0
data/lib/page_rankr/proxy_services/random.rb +2 -0
data/lib/page_rankr/ranks/google.rb +1 -1
data/lib/page_rankr/request.rb +44 -0
data/lib/page_rankr/tracker.rb +11 -19
data/lib/page_rankr/trackers.rb +18 -13
data/lib/page_rankr/version.rb +1 -1
data/spec/fixtures/vcr_cassettes/alexa_ranks_edge_case_1.yml +191 -175
data/spec/fixtures/vcr_cassettes/failure_backlinks.yml +257 -4510
data/spec/fixtures/vcr_cassettes/failure_indexes.yml +221 -4509
data/spec/fixtures/vcr_cassettes/failure_ranks.yml +196 -127
data/spec/fixtures/vcr_cassettes/success_backlinks.yml +291 -5401
data/spec/fixtures/vcr_cassettes/success_indexes.yml +223 -5242
data/spec/fixtures/vcr_cassettes/success_ranks.yml +280 -217
data/spec/page_rankr_spec.rb +35 -5
data/spec/spec_helper.rb +13 -1
metadata +168 -88

data/CHANGELOG.md CHANGED

@@ -1,5 +1,9 @@
 # Change Log
+## Version 4.0.0
+* Fix for Google pagerank check. Query parts were omitted which made the some pages get the same PR as the domain.
+* Switch from Typheous to HTTParty to avoid memory leaks and have better support on Windows.
 ## Version 3.2.1
 * Fix issue where tracker calls proxy with class name rather than name defined on class instance.

data/PageRankr.gemspec CHANGED

@@ -16,11 +16,12 @@ Gem::Specification.new do |s|
   s.add_development_dependency "bundler", ">= 1.0.0"
   s.add_development_dependency "fuubar",  ">= 0.0.1"
   s.add_development_dependency "vcr"
+  s.add_development_dependency "fakeweb"
   s.add_runtime_dependency "nokogiri",              ">= 1.4.1"
   s.add_runtime_dependency "json",                  ">= 1.4.6"
   s.add_runtime_dependency "public_suffix",         ">= 0.9.0"
-  s.add_runtime_dependency "typhoeus",              ">= 0.2.1"
+  s.add_runtime_dependency "httparty",              ">= 0.9.0"
   s.add_runtime_dependency "jsonpath",              ">= 0.4.2"
   s.files         = `git ls-files`.split("\n")

data/README.md CHANGED

@@ -2,6 +2,8 @@
 Provides an easy way to retrieve Google Page Rank, Alexa Rank, backlink counts, and index counts.
+__Note: Version ~> 2.0 and ~> 3.0 used typheous internally which caused memory leak issues and failures on windows. 4.0.0 changes the implementation to use a Net::HTTP based library for better compatability.__
 Check out a little [web app][1] I wrote up that uses it or look at the [source][2].
 [1]: http://isitpopular.heroku.com
@@ -13,6 +15,12 @@ Check out a little [web app][1] I wrote up that uses it or look at the [source][
     gem install PageRankr
 ```
+## Gemfile
+``` ruby
+    gem 'PageRankr'
+```
 ## Use it!
 ``` ruby
@@ -215,19 +223,20 @@ Then, just make sure you require the class and PageRankr and whenever you call P
   (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
 * Send me a pull request. Bonus points for topic branches.
-## TODO Version 4
+## TODO Version 5
 * Detect request throttling
 ## Contributors
-* [Dru Ibarra](https://github.com/Druwerd) - Use Google Search API instead of scraping.
-* [Iteration Labs, LLC](https://github.com/iterationlabs) - Compete rank tracker and domain indexes.
-* [Marc Seeger](http://www.marc-seeger.de) ([Acquia](http://www.acquia.com)) - Ignore invalid ranks that Alexa returns for incorrect sites.
-* [Rémy Coutable](https://github.com/rymai) - Update public_suffix_service gem.
-* [Jonathan Rudenberg](https://github.com/titanous) - Fix compete scraper.
-* [Chris Corbyn](https://github.com/d11wtq) - Fix google page rank url.
-* [Hans Haselberg](https://github.com/i0rek) - Update typhoeus gem.
-* [Priit Haamer](https://github.com/priithaamer) - Fix google backlinks lookup.
-* [Marty McKenna](https://github.com/martyMM) - Idea for proxy service
+* [Dru Ibarra](https://github.com/Druwerd)
+* [Iteration Labs, LLC](https://github.com/iterationlabs)
+* [Marc Seeger](http://www.marc-seeger.de) ([Acquia](http://www.acquia.com))
+* [Rémy Coutable](https://github.com/rymai)
+* [Jonathan Rudenberg](https://github.com/titanous)
+* [Chris Corbyn](https://github.com/d11wtq)
+* [Hans Haselberg](https://github.com/i0rek)
+* [Priit Haamer](https://github.com/priithaamer)
+* [Marty McKenna](https://github.com/martyMM)
+* [Alexi Rahman](https://github.com/odynvolk)
 ## Shout Out
 Gotta give credit where credits due!

data/Rakefile CHANGED

@@ -4,4 +4,9 @@ Bundler::GemHelper.install_tasks
 require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec)
-task :default => :spec
+task :default => :spec
+desc "Open an irb session preloaded with this library"
+task :console do
+  sh "irb -rubygems -I lib -r page_rankr.rb"
+end

data/lib/PageRankr.rb ADDED

	@@ -0,0 +1 @@
1	+ require File.expand_path('../page_rankr', __FILE__)

data/lib/page_rankr/proxy_services/random.rb CHANGED

@@ -1,3 +1,5 @@
+require 'uri'
 module PageRankr
   module ProxyServices
     class Random

data/lib/page_rankr/ranks/google.rb CHANGED

@@ -14,7 +14,7 @@ module PageRankr
       end
       def supported_components
-        [:subdomain, :path]
+        [:subdomain, :path, :query]
       end
       def url

data/lib/page_rankr/request.rb ADDED

@@ -0,0 +1,44 @@
+require 'httparty'
+module PageRankr
+  class Request
+    def initialize(tracker, options)
+      @tracker = tracker
+      @options = options
+    end
+    def perform
+      method = tracker.method
+      url = tracker.url
+      response = HTTParty.send(method, url, construct_options(tracker))
+      yield response.body if block_given?
+    end
+  private
+    attr_reader :tracker
+    def construct_options(tracker)
+      proxy = tracker.proxy
+      params = tracker.params if tracker.respond_to?(:params)
+      options = default_options
+      options.merge!({
+        :http_proxyaddr => proxy.host,
+        :http_proxyport => proxy.port,
+        :http_proxyuser => proxy.user,
+        :http_proxypass => proxy.password
+      }) if proxy
+      options.merge!({:query => params}) if params
+      options.merge!(@options)
+    end
+    def default_options
+      {
+        :headers => {
+          'User-Agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.56.5 (KHTML, like Gecko) Version/5.1.6 Safari/534.56.5'
+        }
+      }
+    end
+  end
+end

data/lib/page_rankr/tracker.rb CHANGED

@@ -1,8 +1,9 @@
-require 'typhoeus'
 require 'nokogiri'
 require 'json'
 require 'jsonpath'
+require 'uri'
 require File.expand_path('../site', __FILE__)
+require File.expand_path('../request', __FILE__)
 module PageRankr
   module Tracker
@@ -12,21 +13,10 @@ module PageRankr
     def initialize(site, options = {})
       @site = PageRankr::Site(site)
+      @options = options
-      @options = {:method => method, :headers => {'User-Agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.12 Safari/535.11'}}
-      @options[:params] = params if respond_to? :params
-      @options[:proxy] = proxy
-      @options.merge!(options)
-      request.on_complete do |response|
-        self.body = response.body
-        self.raw = content(body)
-        self.tracked = clean(raw)
-      end
-    end
-    def request
-      @request ||= Typhoeus::Request.new(url, @options)
+      # Initialize proxy, so threads don't need to synchronize the proxy service.
+      proxy
     end
     def url
@@ -46,13 +36,15 @@ module PageRankr
     end
     def proxy
-      PageRankr.proxy_service.proxy(name, @site) if PageRankr.proxy_service
+      @proxy ||= URI.parse(PageRankr.proxy_service.proxy(name, @site)) if PageRankr.proxy_service
     end
     def run
-      hydra = Typhoeus::Hydra.new
-      hydra.queue request
-      hydra.run
+      PageRankr::Request.new(self, @options).perform do |body|
+        self.body = body
+        self.raw = content(body)
+        self.tracked = clean(raw)
+      end
       tracked
     end

data/lib/page_rankr/trackers.rb CHANGED

@@ -1,4 +1,4 @@
-require 'typhoeus'
+require 'thread'
 module PageRankr
   module Trackers
@@ -10,27 +10,32 @@ module PageRankr
     def lookup(site, *trackers)
       trackers = site_trackers if trackers.empty?
-      tracked = {}
-      hydra = Typhoeus::Hydra.new
-      trackers.each do |tracker|
+      tracked = trackers.map do |tracker|
         name, klass = constant_name(tracker), self.class
         next unless klass.const_defined? name
-        tracked[tracker] = klass.const_get(name).new(site)
-        hydra.queue tracked[tracker].request
-      end
-      hydra.run
-      tracked.keys.each do |tracker|
-        tracked[tracker] = tracked[tracker].tracked
+        [
+          tracker,
+          build_thread(tracker, klass.const_get(name), site)
+        ]
+      end.each do |_, thread|
+        thread.join
+      end.map do |tracker, thread|
+        [tracker, thread.value]
       end
-      tracked
+      Hash[tracked]
     end
     private
+    def build_thread(tracker, instance, site)
+      Thread.new(tracker, instance, site) do |t, i, s|
+        i.new(s).run
+      end
+    end
     def symbol_for(klass)
       word = klass.to_s.dup

data/lib/page_rankr/version.rb CHANGED

@@ -1,3 +1,3 @@
 module PageRankr
-  VERSION = "3.2.1"
+  VERSION = "4.0.0"
 end

data/spec/fixtures/vcr_cassettes/alexa_ranks_edge_case_1.yml CHANGED

@@ -1,181 +1,197 @@
----
-http_interactions:
-- request:
+---
+http_interactions:
+- request:
     method: get
     uri: http://data.alexa.com/data?cli=10&dat=snbamz&url=slocourts.net
-    body:
-      encoding: US-ASCII
-      string: ''
-    headers:
-      User-Agent:
-      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML,
-        like Gecko) Chrome/17.0.963.12 Safari/535.11
-  response:
-    status:
+    body:
+      string: ""
+    headers:
+      user-agent:
+      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.56.5 (KHTML, like Gecko) Version/5.1.6 Safari/534.56.5
+  response:
+    status:
       code: 200
-      message: !binary |-
-        T0s=
-    headers:
-      !binary "Q29udGVudC1UeXBl":
-      - !binary |-
-        dGV4dC94bWw=
-      !binary "Q29udGVudC1MZW5ndGg=":
-      - !binary |-
-        MjU2Nw==
-      !binary "Q29ubmVjdGlvbg==":
-      - !binary |-
-        a2VlcC1hbGl2ZQ==
-    body:
-      encoding: ASCII-8BIT
-      string: !binary |-
-        PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4NCg0KPEFM
-        RVhBIFZFUj0iMC45IiBVUkw9InNsb2NvdXJ0cy5jYS5nb3YvIiBIT01FPSIw
-        IiBBSUQ9Ij0iPg0KPFJMUyBQUkVGSVg9Imh0dHA6Ly8iIG1vcmU9IjIzIj4K
-        PFJMIEhSRUY9ImNvdXJ0cy5jb3VudHlvZnZlbnR1cmEub3JnLyIgVElUTEU9
-        IlN1cGVyaW9yIENvdXJ0LCBWZW50dXJhIi8+CjxSTCBIUkVGPSJ3d3cuY28u
-        c2FuLWJlcm5hcmRpbm8uY2EudXMvQ291cnRzIiBUSVRMRT0iU3VwZXJpb3Ig
-        Q291cnQsIFNhbiBCZXJuYXJkaW5vIi8+CjxSTCBIUkVGPSJ3d3cuY28ucml2
-        ZXJzaWRlLmNhLnVzL2RlcHRzL2NvdXJ0cyIgVElUTEU9IlN1cGVyaW9yIENv
-        dXJ0IE9mIENhbGlmb3JuaWEgQ291bnR5IE9mIFJpdmVyc2lkZSIvPgo8Ukwg
-        SFJFRj0id3d3LmNpLnNmLmNhLnVzL2NvdXJ0cyIgVElUTEU9IlN1cGVyaW9y
-        IENvdXJ0LCBTYW4gRnJhbmNpc2NvIi8+CjxSTCBIUkVGPSJ3d3cuc2Jjb3Vy
-        dHMub3JnLyIgVElUTEU9IlN1cGVyaW9yIENvdXJ0LCBTYW50YSBCYXJiYXJh
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvc2hlcmlmZi5vcmcvIiBUSVRMRT0iU2Fu
-        IEx1aXMgT2Jpc3BvIENvdW50eSBTaGVyaWZmJmFwb3M7cyBEZXBhcnRtZW50
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvZHNzLm9yZy8iIFRJVExFPSJTYW4gTHVp
-        cyBPYmlzcG8gQ291bnR5IERlcGFydG1lbnQgb2YgU29jaWFsIFNlcnZpY2Vz
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvY291bnR5cHdkLm9yZy8iIFRJVExFPSJT
-        YW4gTHVpcyBPYmlzcG8gQ291bnR5IERlcGFydG1lbnQgb2YgUHVibGljIFdv
-        cmtzIGFuZCBUcmFuc3BvcnRhdGlvbiIvPgo8UkwgSFJFRj0id3d3LnNsb2Nv
-        dW50eWZyYW5jaGlzZWFkbWluaXN0cmF0aW9uLm9yZy8iIFRJVExFPSJTYW4g
-        THVpcyBPYmlzcG8gQ291bnR5IEZyYW5jaGlzZSBBZG1pbmlzdHJhdGlvbiIv
-        Pgo8UkwgSFJFRj0id3d3LnNsb2NvZy5vcmcvIiBUSVRMRT0iU0xPQ09HIi8+
-        CjxSTCBIUkVGPSJ3d3cuYnV0dG9ud2FyZS5jb20vIiBUSVRMRT0id3d3LmJ1
-        dHRvbndhcmUuY29tLyIvPgo8L1JMUz4KPFNEIFRJVExFPSJBIiBGTEFHUz0i
-        RE1PWiIgSE9TVD0ic2xvY291cnRzLmNhLmdvdiI+CjxUSVRMRSBURVhUPSJT
-        dXBlcmlvciBDb3VydCwgU2FuIEx1aXMgT2Jpc3BvIi8+CjxBRERSIFNUUkVF
-        VD0iMzEwMSBHb2xkIENhbXAgRHIiIENJVFk9IlJhbmNobyBDb3Jkb3ZhIiBT
-        VEFURT0iQ0EiIFpJUD0iOTU2NzAiIENPVU5UUlk9IlVTIiAvPgo8Q1JFQVRF
-        RCBEQVRFPSIxNi1KdW4tMjAwMCIgREFZPSIxNiIgTU9OVEg9IjA2IiBZRUFS
-        PSIyMDAwIi8+CjxQSE9ORSBOVU1CRVI9IisxIDkxNiA0NjQgNDIzOSIvPgo8
-        T1dORVIgTkFNRT0iU3RhdGUgb2YgQ2FsaWZvcm5pYSIvPgo8RU1BSUwgQURE
-        Uj0ibGJhdGNoZWxkb3JAY28uc2xvLmNhLnVzIi8+CjxMQU5HIExFWD0iZW4i
-        IENPREU9InVzLWFzY2lpIi8+CjxMSU5LU0lOIE5VTT0iNjAiLz4KPFNQRUVE
-        IFRFWFQ9Ijc3NyIgUENUPSI4MiIvPgo8UkVWSUVXUyBBVkc9IjUuMCIgTlVN
-        PSIiLz4KPENISUxEIFNSQVRJTkc9IjAiLz4KPC9TRD4KCjxLRVlXT1JEUz4K
-        PEtFWVdPUkQgVkFMPSJTYW4gTHVpcyBPYmlzcG8iLz4KPC9LRVlXT1JEUz48
-        RE1PWj4KPFNJVEUgQkFTRT0ic2xvY291cnRzLmNhLmdvdi8iIFRJVExFPSJT
-        TE8gQ291bnR5IFN1cGVyaW9yIENvdXJ0IiBERVNDPSJBZG1pbmlzdHJhdGl2
-        ZSBzaXRlIG9mIHRoZSBTdXBlcmlvciBDb3VydCwgd2l0aCBpbmZvcm1hdGlv
-        biwgcHVibGljIG5vdGljZXMsIGFuZCBiYWlsIGFuZCBmZWUgc2NoZWR1bGVz
-        LiI+CjxDQVRTPgo8Q0FUIElEPSJUb3AvUmVnaW9uYWwvTm9ydGhfQW1lcmlj
-        YS9Vbml0ZWRfU3RhdGVzL0NhbGlmb3JuaWEvR292ZXJubWVudC9Db3VydHMi
-        IFRJVExFPSJHb3Zlcm5tZW50L0NvdXJ0cyIgQ0lEPSI2MjA3MTciLz4KPENB
-        VCBJRD0iVG9wL1JlZ2lvbmFsL05vcnRoX0FtZXJpY2EvVW5pdGVkX1N0YXRl
-        cy9DYWxpZm9ybmlhL0NvdW50aWVzL1Nhbl9MdWlzX09iaXNwby9Hb3Zlcm5t
-        ZW50IiBUSVRMRT0iU2FuIEx1aXMgT2Jpc3BvL0dvdmVybm1lbnQiIENJRD0i
-        NjIwNTEwIi8+CjxDQVQgSUQ9IlRvcC9SZWdpb25hbC9Ob3J0aF9BbWVyaWNh
-        L1VuaXRlZF9TdGF0ZXMvQ2FsaWZvcm5pYSIgVElUTEU9IlVuaXRlZCBTdGF0
-        ZXMvQ2FsaWZvcm5pYSIgQ0lEPSI2MTkwODciLz4KPENBVCBJRD0iVG9wL1Jl
-        Z2lvbmFsL05vcnRoX0FtZXJpY2EvVW5pdGVkX1N0YXRlcy9DYWxpZm9ybmlh
-        L0dvdmVybm1lbnQiIFRJVExFPSJDYWxpZm9ybmlhL0dvdmVybm1lbnQiIENJ
-        RD0iNjIwNjA4Ii8+CjxDQVQgSUQ9IlRvcC9Xb3JsZC9Fc3Bhw7FvbC9SZWdp
-        b25hbC9BbcOpcmljYS9Fc3RhZG9zX1VuaWRvcy9Fc3RhZG9zL0NhbGlmb3Ju
-        aWEiIFRJVExFPSJFc3RhZG9zL0NhbGlmb3JuaWEiIENJRD0iMjYyNDkiLz4K
-        PC9DQVRTPgo8L1NJVEU+CjwvRE1PWj4KPFNEPgo8UE9QVUxBUklUWSBVUkw9
-        ImNhLmdvdi8iIFRFWFQ9Ijc2MCIgU09VUkNFPSJwYW5lbCIvPgo8UkVBQ0gg
-        UkFOSz0iODQ0Ii8+CjxSQU5LIERFTFRBPSItODgiLz4KPC9TRD4KPC9BTEVY
-        QT4=
-    http_version: !binary |-
-      MS4x
-  recorded_at: Sat, 12 May 2012 18:39:16 GMT
-- request:
+      message: OK
+    headers:
+      connection:
+      - Close
+      content-type:
+      - text/xml
+      content-length:
+      - "2620"
+    body:
+      string: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n\
+        \r\n\
+        <ALEXA VER=\"0.9\" URL=\"slocourts.ca.gov/\" HOME=\"0\" AID=\"=\">\r\n\
+        <RLS PREFIX=\"http://\" more=\"23\">\n\
+        <RL HREF=\"www.co.riverside.ca.us/depts/courts\" TITLE=\"Superior Court Of California County Of Riverside\"/>\n\
+        <RL HREF=\"courts.countyofventura.org/\" TITLE=\"Superior Court, Ventura\"/>\n\
+        <RL HREF=\"www.co.san-bernardino.ca.us/Courts\" TITLE=\"Superior Court, San Bernardino\"/>\n\
+        <RL HREF=\"www.ci.sf.ca.us/courts\" TITLE=\"Superior Court, San Francisco\"/>\n\
+        <RL HREF=\"www.sbcourts.org/\" TITLE=\"Superior Court, Santa Barbara\"/>\n\
+        <RL HREF=\"www.slosheriff.org/\" TITLE=\"San Luis Obispo County Sheriff&apos;s Department\"/>\n\
+        <RL HREF=\"www.slodss.org/\" TITLE=\"San Luis Obispo County Department of Social Services\"/>\n\
+        <RL HREF=\"www.slocountypwd.org/\" TITLE=\"San Luis Obispo County Department of Public Works and Transportation\"/>\n\
+        <RL HREF=\"www.slocountyfranchiseadministration.org/\" TITLE=\"San Luis Obispo County Franchise Administration\"/>\n\
+        <RL HREF=\"www.slocog.org/\" TITLE=\"SLOCOG\"/>\n\
+        <RL HREF=\"www.buttonware.com/\" TITLE=\"www.buttonware.com/\"/>\n\
+        </RLS>\n\
+        <SD TITLE=\"A\" FLAGS=\"DMOZ\" HOST=\"slocourts.ca.gov\">\n\
+        <TITLE TEXT=\"Superior Court, San Luis Obispo\"/>\n\
+        <ADDR STREET=\"3101 Gold Camp Dr\" CITY=\"Rancho Cordova\" STATE=\"CA\" ZIP=\"95670\" COUNTRY=\"US\" />\n\
+        <CREATED DATE=\"16-Jun-2000\" DAY=\"16\" MONTH=\"06\" YEAR=\"2000\"/>\n\
+        <PHONE NUMBER=\"+1 916 464 4239\"/>\n\
+        <OWNER NAME=\"State of California\"/>\n\
+        <EMAIL ADDR=\"lbatcheldor@co.slo.ca.us\"/>\n\
+        <LANG LEX=\"en\" CODE=\"us-ascii\"/>\n\
+        <LINKSIN NUM=\"12\"/>\n\
+        <SPEED TEXT=\"716\" PCT=\"83\"/>\n\
+        <REVIEWS AVG=\"4.5\" NUM=\"\"/>\n\
+        <CHILD SRATING=\"0\"/>\n\
+        </SD>\n\n\
+        <KEYWORDS>\n\
+        <KEYWORD VAL=\"San Luis Obispo\"/>\n\
+        </KEYWORDS><DMOZ>\n\
+        <SITE BASE=\"slocourts.ca.gov/\" TITLE=\"SLO County Superior Court\" DESC=\"Administrative site of the Superior Court, with information, public notices, and bail and fee schedules.\">\n\
+        <CATS>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Government/Courts\" TITLE=\"Government/Courts\" CID=\"620717\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Counties/San_Luis_Obispo/Government\" TITLE=\"San Luis Obispo/Government\" CID=\"620510\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California\" TITLE=\"United States/California\" CID=\"619087\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Government\" TITLE=\"California/Government\" CID=\"620608\"/>\n\
+        <CAT ID=\"Top/World/Espa\xC3\xB1ol/Regional/Am\xC3\xA9rica/Estados_Unidos/Estados/California\" TITLE=\"Estados/California\" CID=\"26249\"/>\n\
+        </CATS>\n\
+        </SITE>\n\
+        </DMOZ>\n\
+        <SD>\n\
+        <POPULARITY URL=\"ca.gov/\" TEXT=\"775\" SOURCE=\"panel\"/>\n\
+        <REACH RANK=\"896\"/>\n\
+        <RANK DELTA=\"-10\"/>\n\
+        <COUNTRY CODE=\"US\" NAME=\"United States\" RANK=\"197\"/>\n\
+        </SD>\n\
+        </ALEXA>"
+    http_version: "1.1"
+  recorded_at: Mon, 17 Dec 2012 17:03:43 GMT
+- request:
     method: get
     uri: http://data.alexa.com/data?cli=10&dat=snbamz&url=slocourts.net
-    body:
-      encoding: US-ASCII
-      string: ''
-    headers:
-      User-Agent:
-      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML,
-        like Gecko) Chrome/17.0.963.12 Safari/535.11
-  response:
-    status:
+    body:
+      string: ""
+    headers:
+      user-agent:
+      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.56.5 (KHTML, like Gecko) Version/5.1.6 Safari/534.56.5
+  response:
+    status:
       code: 200
-      message: !binary |-
-        T0s=
-    headers:
-      !binary "Q29udGVudC1UeXBl":
-      - !binary |-
-        dGV4dC94bWw=
-      !binary "Q29udGVudC1MZW5ndGg=":
-      - !binary |-
-        MjU2Nw==
-      !binary "Q29ubmVjdGlvbg==":
-      - !binary |-
-        a2VlcC1hbGl2ZQ==
-    body:
-      encoding: ASCII-8BIT
-      string: !binary |-
-        PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4NCg0KPEFM
-        RVhBIFZFUj0iMC45IiBVUkw9InNsb2NvdXJ0cy5jYS5nb3YvIiBIT01FPSIw
-        IiBBSUQ9Ij0iPg0KPFJMUyBQUkVGSVg9Imh0dHA6Ly8iIG1vcmU9IjIzIj4K
-        PFJMIEhSRUY9ImNvdXJ0cy5jb3VudHlvZnZlbnR1cmEub3JnLyIgVElUTEU9
-        IlN1cGVyaW9yIENvdXJ0LCBWZW50dXJhIi8+CjxSTCBIUkVGPSJ3d3cuY28u
-        c2FuLWJlcm5hcmRpbm8uY2EudXMvQ291cnRzIiBUSVRMRT0iU3VwZXJpb3Ig
-        Q291cnQsIFNhbiBCZXJuYXJkaW5vIi8+CjxSTCBIUkVGPSJ3d3cuY28ucml2
-        ZXJzaWRlLmNhLnVzL2RlcHRzL2NvdXJ0cyIgVElUTEU9IlN1cGVyaW9yIENv
-        dXJ0IE9mIENhbGlmb3JuaWEgQ291bnR5IE9mIFJpdmVyc2lkZSIvPgo8Ukwg
-        SFJFRj0id3d3LmNpLnNmLmNhLnVzL2NvdXJ0cyIgVElUTEU9IlN1cGVyaW9y
-        IENvdXJ0LCBTYW4gRnJhbmNpc2NvIi8+CjxSTCBIUkVGPSJ3d3cuc2Jjb3Vy
-        dHMub3JnLyIgVElUTEU9IlN1cGVyaW9yIENvdXJ0LCBTYW50YSBCYXJiYXJh
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvc2hlcmlmZi5vcmcvIiBUSVRMRT0iU2Fu
-        IEx1aXMgT2Jpc3BvIENvdW50eSBTaGVyaWZmJmFwb3M7cyBEZXBhcnRtZW50
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvZHNzLm9yZy8iIFRJVExFPSJTYW4gTHVp
-        cyBPYmlzcG8gQ291bnR5IERlcGFydG1lbnQgb2YgU29jaWFsIFNlcnZpY2Vz
-        Ii8+CjxSTCBIUkVGPSJ3d3cuc2xvY291bnR5cHdkLm9yZy8iIFRJVExFPSJT
-        YW4gTHVpcyBPYmlzcG8gQ291bnR5IERlcGFydG1lbnQgb2YgUHVibGljIFdv
-        cmtzIGFuZCBUcmFuc3BvcnRhdGlvbiIvPgo8UkwgSFJFRj0id3d3LnNsb2Nv
-        dW50eWZyYW5jaGlzZWFkbWluaXN0cmF0aW9uLm9yZy8iIFRJVExFPSJTYW4g
-        THVpcyBPYmlzcG8gQ291bnR5IEZyYW5jaGlzZSBBZG1pbmlzdHJhdGlvbiIv
-        Pgo8UkwgSFJFRj0id3d3LnNsb2NvZy5vcmcvIiBUSVRMRT0iU0xPQ09HIi8+
-        CjxSTCBIUkVGPSJ3d3cuYnV0dG9ud2FyZS5jb20vIiBUSVRMRT0id3d3LmJ1
-        dHRvbndhcmUuY29tLyIvPgo8L1JMUz4KPFNEIFRJVExFPSJBIiBGTEFHUz0i
-        RE1PWiIgSE9TVD0ic2xvY291cnRzLmNhLmdvdiI+CjxUSVRMRSBURVhUPSJT
-        dXBlcmlvciBDb3VydCwgU2FuIEx1aXMgT2Jpc3BvIi8+CjxBRERSIFNUUkVF
-        VD0iMzEwMSBHb2xkIENhbXAgRHIiIENJVFk9IlJhbmNobyBDb3Jkb3ZhIiBT
-        VEFURT0iQ0EiIFpJUD0iOTU2NzAiIENPVU5UUlk9IlVTIiAvPgo8Q1JFQVRF
-        RCBEQVRFPSIxNi1KdW4tMjAwMCIgREFZPSIxNiIgTU9OVEg9IjA2IiBZRUFS
-        PSIyMDAwIi8+CjxQSE9ORSBOVU1CRVI9IisxIDkxNiA0NjQgNDIzOSIvPgo8
-        T1dORVIgTkFNRT0iU3RhdGUgb2YgQ2FsaWZvcm5pYSIvPgo8RU1BSUwgQURE
-        Uj0ibGJhdGNoZWxkb3JAY28uc2xvLmNhLnVzIi8+CjxMQU5HIExFWD0iZW4i
-        IENPREU9InVzLWFzY2lpIi8+CjxMSU5LU0lOIE5VTT0iNjAiLz4KPFNQRUVE
-        IFRFWFQ9Ijc3NyIgUENUPSI4MiIvPgo8UkVWSUVXUyBBVkc9IjUuMCIgTlVN
-        PSIiLz4KPENISUxEIFNSQVRJTkc9IjAiLz4KPC9TRD4KCjxLRVlXT1JEUz4K
-        PEtFWVdPUkQgVkFMPSJTYW4gTHVpcyBPYmlzcG8iLz4KPC9LRVlXT1JEUz48
-        RE1PWj4KPFNJVEUgQkFTRT0ic2xvY291cnRzLmNhLmdvdi8iIFRJVExFPSJT
-        TE8gQ291bnR5IFN1cGVyaW9yIENvdXJ0IiBERVNDPSJBZG1pbmlzdHJhdGl2
-        ZSBzaXRlIG9mIHRoZSBTdXBlcmlvciBDb3VydCwgd2l0aCBpbmZvcm1hdGlv
-        biwgcHVibGljIG5vdGljZXMsIGFuZCBiYWlsIGFuZCBmZWUgc2NoZWR1bGVz
-        LiI+CjxDQVRTPgo8Q0FUIElEPSJUb3AvUmVnaW9uYWwvTm9ydGhfQW1lcmlj
-        YS9Vbml0ZWRfU3RhdGVzL0NhbGlmb3JuaWEvR292ZXJubWVudC9Db3VydHMi
-        IFRJVExFPSJHb3Zlcm5tZW50L0NvdXJ0cyIgQ0lEPSI2MjA3MTciLz4KPENB
-        VCBJRD0iVG9wL1JlZ2lvbmFsL05vcnRoX0FtZXJpY2EvVW5pdGVkX1N0YXRl
-        cy9DYWxpZm9ybmlhL0NvdW50aWVzL1Nhbl9MdWlzX09iaXNwby9Hb3Zlcm5t
-        ZW50IiBUSVRMRT0iU2FuIEx1aXMgT2Jpc3BvL0dvdmVybm1lbnQiIENJRD0i
-        NjIwNTEwIi8+CjxDQVQgSUQ9IlRvcC9SZWdpb25hbC9Ob3J0aF9BbWVyaWNh
-        L1VuaXRlZF9TdGF0ZXMvQ2FsaWZvcm5pYSIgVElUTEU9IlVuaXRlZCBTdGF0
-        ZXMvQ2FsaWZvcm5pYSIgQ0lEPSI2MTkwODciLz4KPENBVCBJRD0iVG9wL1Jl
-        Z2lvbmFsL05vcnRoX0FtZXJpY2EvVW5pdGVkX1N0YXRlcy9DYWxpZm9ybmlh
-        L0dvdmVybm1lbnQiIFRJVExFPSJDYWxpZm9ybmlhL0dvdmVybm1lbnQiIENJ
-        RD0iNjIwNjA4Ii8+CjxDQVQgSUQ9IlRvcC9Xb3JsZC9Fc3Bhw7FvbC9SZWdp
-        b25hbC9BbcOpcmljYS9Fc3RhZG9zX1VuaWRvcy9Fc3RhZG9zL0NhbGlmb3Ju
-        aWEiIFRJVExFPSJFc3RhZG9zL0NhbGlmb3JuaWEiIENJRD0iMjYyNDkiLz4K
-        PC9DQVRTPgo8L1NJVEU+CjwvRE1PWj4KPFNEPgo8UE9QVUxBUklUWSBVUkw9
-        ImNhLmdvdi8iIFRFWFQ9Ijc2MCIgU09VUkNFPSJwYW5lbCIvPgo8UkVBQ0gg
-        UkFOSz0iODQ0Ii8+CjxSQU5LIERFTFRBPSItODgiLz4KPC9TRD4KPC9BTEVY
-        QT4=
-    http_version: !binary |-
-      MS4x
-  recorded_at: Sat, 12 May 2012 18:39:16 GMT
-recorded_with: VCR 2.0.1
+      message: OK
+    headers:
+      connection:
+      - Close
+      content-type:
+      - text/xml
+      content-length:
+      - "2620"
+    body:
+      string: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n\
+        \r\n\
+        <ALEXA VER=\"0.9\" URL=\"slocourts.ca.gov/\" HOME=\"0\" AID=\"=\">\r\n\
+        <RLS PREFIX=\"http://\" more=\"23\">\n\
+        <RL HREF=\"www.co.riverside.ca.us/depts/courts\" TITLE=\"Superior Court Of California County Of Riverside\"/>\n\
+        <RL HREF=\"courts.countyofventura.org/\" TITLE=\"Superior Court, Ventura\"/>\n\
+        <RL HREF=\"www.co.san-bernardino.ca.us/Courts\" TITLE=\"Superior Court, San Bernardino\"/>\n\
+        <RL HREF=\"www.ci.sf.ca.us/courts\" TITLE=\"Superior Court, San Francisco\"/>\n\
+        <RL HREF=\"www.sbcourts.org/\" TITLE=\"Superior Court, Santa Barbara\"/>\n\
+        <RL HREF=\"www.slosheriff.org/\" TITLE=\"San Luis Obispo County Sheriff&apos;s Department\"/>\n\
+        <RL HREF=\"www.slodss.org/\" TITLE=\"San Luis Obispo County Department of Social Services\"/>\n\
+        <RL HREF=\"www.slocountypwd.org/\" TITLE=\"San Luis Obispo County Department of Public Works and Transportation\"/>\n\
+        <RL HREF=\"www.slocountyfranchiseadministration.org/\" TITLE=\"San Luis Obispo County Franchise Administration\"/>\n\
+        <RL HREF=\"www.slocog.org/\" TITLE=\"SLOCOG\"/>\n\
+        <RL HREF=\"www.buttonware.com/\" TITLE=\"www.buttonware.com/\"/>\n\
+        </RLS>\n\
+        <SD TITLE=\"A\" FLAGS=\"DMOZ\" HOST=\"slocourts.ca.gov\">\n\
+        <TITLE TEXT=\"Superior Court, San Luis Obispo\"/>\n\
+        <ADDR STREET=\"3101 Gold Camp Dr\" CITY=\"Rancho Cordova\" STATE=\"CA\" ZIP=\"95670\" COUNTRY=\"US\" />\n\
+        <CREATED DATE=\"16-Jun-2000\" DAY=\"16\" MONTH=\"06\" YEAR=\"2000\"/>\n\
+        <PHONE NUMBER=\"+1 916 464 4239\"/>\n\
+        <OWNER NAME=\"State of California\"/>\n\
+        <EMAIL ADDR=\"lbatcheldor@co.slo.ca.us\"/>\n\
+        <LANG LEX=\"en\" CODE=\"us-ascii\"/>\n\
+        <LINKSIN NUM=\"12\"/>\n\
+        <SPEED TEXT=\"716\" PCT=\"83\"/>\n\
+        <REVIEWS AVG=\"4.5\" NUM=\"\"/>\n\
+        <CHILD SRATING=\"0\"/>\n\
+        </SD>\n\n\
+        <KEYWORDS>\n\
+        <KEYWORD VAL=\"San Luis Obispo\"/>\n\
+        </KEYWORDS><DMOZ>\n\
+        <SITE BASE=\"slocourts.ca.gov/\" TITLE=\"SLO County Superior Court\" DESC=\"Administrative site of the Superior Court, with information, public notices, and bail and fee schedules.\">\n\
+        <CATS>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Government/Courts\" TITLE=\"Government/Courts\" CID=\"620717\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Counties/San_Luis_Obispo/Government\" TITLE=\"San Luis Obispo/Government\" CID=\"620510\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California\" TITLE=\"United States/California\" CID=\"619087\"/>\n\
+        <CAT ID=\"Top/Regional/North_America/United_States/California/Government\" TITLE=\"California/Government\" CID=\"620608\"/>\n\
+        <CAT ID=\"Top/World/Espa\xC3\xB1ol/Regional/Am\xC3\xA9rica/Estados_Unidos/Estados/California\" TITLE=\"Estados/California\" CID=\"26249\"/>\n\
+        </CATS>\n\
+        </SITE>\n\
+        </DMOZ>\n\
+        <SD>\n\
+        <POPULARITY URL=\"ca.gov/\" TEXT=\"775\" SOURCE=\"panel\"/>\n\
+        <REACH RANK=\"896\"/>\n\
+        <RANK DELTA=\"-10\"/>\n\
+        <COUNTRY CODE=\"US\" NAME=\"United States\" RANK=\"197\"/>\n\
+        </SD>\n\
+        </ALEXA>"
+    http_version: "1.1"
+  recorded_at: Mon, 17 Dec 2012 17:03:43 GMT
+- request:
+    method: get
+    uri: http://data.alexa.com/data?dat=snbamz&url=slocourts.net&cli=10
+    body:
+      string: ""
+    headers:
+      user-agent:
+      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.56.5 (KHTML, like Gecko) Version/5.1.6 Safari/534.56.5
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      connection:
+      - Close
+      content-type:
+      - text/xml
+      content-length:
+      - "30"
+    body:
+      string: |
+        <ALEXA VER="0.9" URL="404"/>
+    http_version: "1.1"
+  recorded_at: Tue, 18 Dec 2012 01:21:40 GMT
+- request:
+    method: get
+    uri: http://data.alexa.com/data?dat=snbamz&url=slocourts.net&cli=10
+    body:
+      string: ""
+    headers:
+      user-agent:
+      - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.56.5 (KHTML, like Gecko) Version/5.1.6 Safari/534.56.5
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      connection:
+      - Close
+      content-type:
+      - text/xml
+      content-length:
+      - "30"
+    body:
+      string: |
+        <ALEXA VER="0.9" URL="404"/>
+    http_version: "1.1"
+  recorded_at: Tue, 18 Dec 2012 01:21:40 GMT
+recorded_with: VCR 2.3.0