rapgenius 0.0.2 → 0.0.3

data/.gitignore

@@ -0,0 +1,2 @@
+Gemfile.lock
+spec/support/cassettes

data/CHANGELOG.md

@@ -0,0 +1,14 @@
+# Changelog
+
+__v0.0.1__ (17th August 2013)
+
+* Initial version
+
+__v0.0.2__ (17th August 2013)
+
+* Adds `RapGenius::Song.find` to replicate behaviour in `RapGenius::Annotation`
+
+__v0.0.3__ (22nd August 2013, *contributed by [tsigo](https://github.com/tsigo)*)
+
+* Improves implementation of HTTParty
+* Reorganises specs to use VCR

data/README.md

@@ -0,0 +1,95 @@
+# rapgenius
+
+![Rap Genius logo](http://f.cl.ly/items/303W0c1i2r100j2u3Y0y/Screen%20Shot%202013-08-17%20at%2016.01.19.png)
+
+## What does this do?
+
+It's a Ruby gem for accessing lyrics and explanations on
+[Rap Genius](http://rapgenius.com). 
+
+They very sadly [don't have an API](https://twitter.com/RapGenius/status/245057326321655808) so I decided to replicate one for myself
+with a nice bit of screen scraping with [Nokogiri](https://github.com/sparklemotion/nokogiri), much like my [amex](https://github.com/timrogers/amex), [ucas](https://github.com/timrogers/ucas) and [lloydstsb](https://github.com/timrogers/lloydstsb) gems.
+
+## Installation
+
+Install the gem, and you're ready to go. Simply add the following to your
+Gemfile:
+
+`gem "rapgenius", "~> 0.0.2"`
+
+## Usage
+
+Songs on Rap Genius don't have numeric identifiers as far as I can tell - they're identified by a URL slug featuring the artist and song name, for instance "Big-sean-control-lyrics". We use this to fetch a particular track, like so:
+
+```ruby
+require 'rapgenius'
+song = RapGenius::Song.find("Big-sean-control-lyrics")
+```
+
+Once you've got the song, you can easily load details about it. This uses
+Nokogiri to fetch the song's page and then parse it:
+
+```ruby
+song.title
+# => "Control"
+
+song.artist
+# => "Big Sean"
+
+song.full_artist
+# => "Big Sean (Ft. Jay Electronica & Kendrick Lamar)"
+
+song.images
+# => ["http://s3.amazonaws.com/rapgenius/1376434983_jay-electronica.jpg", "http://s3.amazonaws.com/rapgenius/1375029260_Big%20Sean.png", "http://s3.amazonaws.com/rapgenius/Kendrick-Lamar-1024x680.jpg"]
+
+song.description
+# => "The non-album cut from Sean that basically blew up the Internet due to a world-beating verse by Kendrick Lamar...
+```
+
+The `#annotations` accessor on a Song returns an array of RapGenius::Annotation
+objects corresponding to different annotated lines of the song, identified by
+their `id`.
+
+You can look these up manually using `RapGenius::Annotation.find("id")`. You
+can grab the ID for a lyric from a RapGenius page by right clicking on an annotation, copying the shortcut and then finding the number after "http://rapgenius.com".
+
+```ruby
+song.annotations
+# => [<RapGenius::Annotation>, <RapGenius::Annotation>...]
+
+annotation = song.annotations[99]
+
+annotation.lyric
+# => "And that goes for Jermaine Cole, Big KRIT, Wale\nPusha T, Meek Millz, A$AP Rocky, Drake\nBig Sean, Jay Electron', Tyler, Mac Miller"
+
+annotation.explanation
+# => "Kendrick calls out some of the biggest names in present day Hip-hop...""
+
+annotation.song == song # You can get back to the song from the annotation...
+# => true
+
+annotation.id
+# => "2093001"
+
+annotation2 = RapGenius::Annotation.find("2093001") # Fetching directly...
+
+annotation == annotations2
+# => true
+```
+
+## Contributing
+
+There are a few things I'd love to see added to this gem:
+
+* __Searching__ - having to know the path to a particular track's lyrics isn't super intuitive
+* __Support for *\*Genius*__ - RapG enius also have other sites on subdomains like [News Genius](http://news.rapgenius.com) and [Poetry Genius](http://poetry.rapgenius.com). These could very easily be supported, since theyre identical in terms of markup.
+
+This gem is open source, so feel free to add anything you want, then make a pull request. A few quick tips:
+
+* Don't update the version numbers before your pull request - I'll sort that part out for you!
+* Make sure you write specs, then run them with `$ bundle exec rake`
+* Update this README.md file so I, and users, know how your changes work
+
+## Get in touch
+
+Any questions, thoughts or comments? Email me at <me@timrogers.co.uk>.

data/lib/rapgenius/annotation.rb

@@ -26,11 +26,12 @@ module RapGenius
     end
 
     def song
-      entry_path = document.css('meta[property="rap_genius:song"]').
-        attr('content').to_s
-
-      @song ||= Song.new(entry_path)
+      @song ||= Song.new(song_url)
     end
 
+    def song_url
+      @song_url ||= document.css('meta[property="rap_genius:song"]').
+        attr('content').to_s
+    end
   end
-end
+end

data/lib/rapgenius/scraper.rb

@@ -3,33 +3,55 @@ require 'httparty'
 
 module RapGenius
   module Scraper
-    BASE_URL = "http://rapgenius.com/".freeze
+    # Custom HTTParty parser that parses the returned body with Nokogiri
+    class NokogiriParser < HTTParty::Parser
+      SupportedFormats.merge!('text/html' => :html)
 
-    attr_reader :url
+      def html
+        Nokogiri::HTML(body)
+      end
+    end
+
+    # HTTParty client
+    #
+    # Sets some useful defaults for all of our requests.
+    #
+    # See Scraper#fetch
+    class Client
+      include HTTParty
+
+      format   :html
+      parser   NokogiriParser
+      base_uri 'http://rapgenius.com'
+      headers  'User-Agent' => "rapgenius.rb v#{RapGenius::VERSION}"
+    end
 
+    BASE_URL = Client.base_uri + "/".freeze
+
+    attr_reader :url
 
     def url=(url)
-      if !(url =~ /^https?:\/\//)
-        @url = "#{BASE_URL}#{url}" 
+      unless url =~ /^https?:\/\//
+        @url = BASE_URL + url
       else
         @url = url
       end
     end
 
     def document
-      @document ||= Nokogiri::HTML(fetch(@url))
+      @document ||= fetch(@url)
     end
 
     private
+
     def fetch(url)
-      response = HTTParty.get(url)
+      response = Client.get(url)
 
       if response.code != 200
         raise ScraperError, "Received a #{response.code} HTTP response"
       end
 
-      response.body
+      response.parsed_response
     end
-
   end
-end
+end

data/lib/rapgenius/song.rb

@@ -11,7 +11,6 @@ module RapGenius
       self.url = path
     end
 
-
     def artist
       document.css('.song_title a').text
     end
@@ -43,7 +42,5 @@ module RapGenius
         )
       end
     end
-
-
   end
-end
+end

data/lib/rapgenius/version.rb

@@ -1,3 +1,3 @@
 module RapGenius
-  VERSION = "0.0.2"
+  VERSION = "0.0.3"
 end

data/rapgenius.gemspec

@@ -14,13 +14,15 @@ Gem::Specification.new do |s|
     "working at Rap Genius is the API". With this magical screen-scraping gem,
     you can access the wealth of data on the internet Talmud in Ruby.}
 
-  s.add_runtime_dependency "nokogiri", "~>1.6.0"
-  s.add_runtime_dependency "httparty", "~>0.11.0"
-  s.add_development_dependency "rspec", "~>2.14.1"
-  s.add_development_dependency "mocha", "~>0.14.0"
+  s.add_runtime_dependency "nokogiri",    "~>1.6.0"
+  s.add_runtime_dependency "httparty",    "~>0.11.0"
+  s.add_development_dependency "rspec",   "~>2.14.1"
+  s.add_development_dependency "mocha",   "~>0.14.0"
+  s.add_development_dependency "webmock", "~>1.11.0"
+  s.add_development_dependency "vcr",     "~>2.5.0"
 
   s.files         = `git ls-files`.split("\n")
   s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib"]
-end
+end

data/spec/rapgenius/annotation_spec.rb

@@ -0,0 +1,41 @@
+require 'spec_helper'
+
+module RapGenius
+  describe Annotation, vcr: {cassette_name: "big-sean-annotation"} do
+
+    let(:annotation) { described_class.new(id: "2092393") }
+    subject { annotation }
+
+    its(:id)       { should eq "2092393" }
+    its(:url)      { should eq "http://rapgenius.com/2092393" }
+    its(:song)     { should be_a Song }
+    its(:song_url) { should eq "http://rapgenius.com/Big-sean-control-lyrics" }
+
+    describe "#lyric" do
+      it "should have the correct lyric" do
+        annotation.lyric.should eq "You gon' get this rain like it's May weather,"
+      end
+    end
+
+    describe "#explanation" do
+      it "should have the correct explanation" do
+        annotation.explanation.should include "making it rain"
+      end
+    end
+
+    describe '.find' do
+      it "returns a new instance at the specified path" do
+        i = described_class.find("foobar")
+        i.should be_an Annotation
+        i.id.should eq "foobar"
+      end
+    end
+
+    context "with additional parameters passed into the constructor" do
+      let(:annotation) { described_class.new(id: "5678", lyric: "foo") }
+
+      its(:id)    { should eq "5678" }
+      its(:lyric) { should eq "foo" }
+    end
+  end
+end

data/spec/rapgenius/scraper_spec.rb

@@ -0,0 +1,54 @@
+require 'spec_helper'
+
+class ScraperTester
+  include RapGenius::Scraper
+end
+
+module RapGenius
+  describe Scraper do
+
+    let(:scraper) { ScraperTester.new }
+
+    describe "#url=" do
+      it "forms the URL with the base URL, if the current path is relative" do
+        scraper.url = "foobar"
+        scraper.url.should include RapGenius::Scraper::BASE_URL
+      end
+
+      it "leaves the URL as it is if already complete" do
+        scraper.url = "http://foobar.com/baz"
+        scraper.url.should eq "http://foobar.com/baz"
+      end
+    end
+
+    describe "#document" do
+      before do
+        scraper.url = "http://foo.bar/"
+      end
+
+      context "with a successful request" do
+        before do
+          stub_request(:get, "http://foo.bar").to_return({body: 'ok', status: 200})
+        end
+
+        it "returns a Nokogiri document object" do
+          scraper.document.should be_a Nokogiri::HTML::Document
+        end
+
+        it "contains the tags in page received back from the HTTP request" do
+          scraper.document.css('body').length.should eq 1
+        end
+      end
+
+      context "with a failed request" do
+        before do
+          stub_request(:get, "http://foo.bar").to_return({body: '', status: 404})
+        end
+
+        it "raises a ScraperError" do
+          expect { scraper.document }.to raise_error(RapGenius::ScraperError)
+        end
+      end
+    end
+  end
+end

data/spec/rapgenius/song_spec.rb

@@ -0,0 +1,46 @@
+require 'spec_helper'
+
+module RapGenius
+  describe Song do
+    context "given Big Sean's Control", vcr: {cassette_name: "big-sean-control-lyrics"} do
+      subject { described_class.new("Big-sean-control-lyrics") }
+
+      its(:url)         { should eq "http://rapgenius.com/Big-sean-control-lyrics" }
+      its(:title)       { should eq "Control" }
+      its(:artist)      { should eq "Big Sean" }
+      its(:description) { should include "blew up the Internet" }
+      its(:full_artist) { should include "(Ft. Jay Electronica & Kendrick Lamar)"}
+
+      describe "#images" do
+        it "should be an Array" do
+          subject.images.should be_an Array
+        end
+
+        it "should include Big Sean's picture" do
+          subject.images.should include "http://s3.amazonaws.com/rapgenius/1375029260_Big%20Sean.png"
+        end
+      end
+
+      describe "#annotations" do
+        it "should be an Array of Annotation objects" do
+          subject.annotations.should be_an Array
+          subject.annotations.first.should be_a Annotation
+        end
+
+        it "should be of a valid length" do
+          # Annotations get added and removed from the live site; we want our
+          # count to be somewhat accurate, within reason.
+          subject.annotations.length.should be_within(15).of(130)
+        end
+      end
+    end
+
+    describe '.find' do
+      it "returns a new instance at the specified path" do
+        i = described_class.find("foobar")
+        i.should be_a Song
+        i.url.should eq 'http://rapgenius.com/foobar'
+      end
+    end
+  end
+end

data/spec/spec_helper.rb

@@ -1,6 +1,9 @@
 require 'rapgenius'
 require 'mocha/api'
+require 'webmock/rspec'
+
+Dir[File.expand_path('../support/**/*.rb', __FILE__)].each { |f| require f }
 
 RSpec.configure do |config|
   config.mock_framework = :mocha
-end
+end

data/spec/support/vcr.rb

@@ -0,0 +1,11 @@
+require 'vcr'
+
+VCR.configure do |c|
+  c.default_cassette_options = {
+    record: :new_episodes,
+    re_record_interval: 24 * 60 * 60
+  }
+  c.cassette_library_dir = File.expand_path('../cassettes/', __FILE__)
+  c.hook_into :webmock
+  c.configure_rspec_metadata!
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rapgenius
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.0.3
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-08-17 00:00:00.000000000 Z
+date: 2013-08-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -75,6 +75,38 @@ dependencies:
     - - ~>
       - !ruby/object:Gem::Version
         version: 0.14.0
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.11.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.11.0
+- !ruby/object:Gem::Dependency
+  name: vcr
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 2.5.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 2.5.0
 description: ! "Up until until now, to quote RapGenius themselves,\n    \"working
   at Rap Genius is the API\". With this magical screen-scraping gem,\n    you can
   access the wealth of data on the internet Talmud in Ruby."
@@ -84,9 +116,11 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- .gitignore
+- CHANGELOG.md
 - Gemfile
-- Gemfile.lock
 - LICENSE
+- README.md
 - Rakefile
 - lib/rapgenius.rb
 - lib/rapgenius/annotation.rb
@@ -95,13 +129,13 @@ files:
 - lib/rapgenius/song.rb
 - lib/rapgenius/version.rb
 - pkg/rapgenius-0.0.1.gem
+- pkg/rapgenius-0.0.2.gem
 - rapgenius.gemspec
-- spec/annotation_spec.rb
-- spec/scraper_spec.rb
-- spec/song_spec.rb
+- spec/rapgenius/annotation_spec.rb
+- spec/rapgenius/scraper_spec.rb
+- spec/rapgenius/song_spec.rb
 - spec/spec_helper.rb
-- spec/support/annotation.html
-- spec/support/song.html
+- spec/support/vcr.rb
 homepage: http://timrogers.co.uk
 licenses: []
 post_install_message: 
@@ -127,9 +161,8 @@ signing_key:
 specification_version: 3
 summary: A gem for accessing lyrics and explanations on RapGenius.com
 test_files:
-- spec/annotation_spec.rb
-- spec/scraper_spec.rb
-- spec/song_spec.rb
+- spec/rapgenius/annotation_spec.rb
+- spec/rapgenius/scraper_spec.rb
+- spec/rapgenius/song_spec.rb
 - spec/spec_helper.rb
-- spec/support/annotation.html
-- spec/support/song.html
+- spec/support/vcr.rb