rapgenius 0.1.0 → 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: cba91ddf9f2be805d1fdf8c884a3474b893ec741
+  data.tar.gz: 41f02951bd8769319f5ed6a4f4a28a84e38baac3
+SHA512:
+  metadata.gz: 2632e2bac0a238c1d1f7989cbe3e3e72e34b7c8b7e4fb442cfa40473fc26472532220ad46b9007849ba630c1cf6cc94f765bcb967d4ac5d13256ff69f0314950
+  data.tar.gz: d4f89f5534f08a6ee69a6abefef724ba659f5994dbe766ab58af6bd64dc2be4bb941231ded1ced2d409ce211252711d5e94b451c4c14562dc3508671660207ac

data/.gitignore

@@ -1,2 +1,4 @@
 Gemfile.lock
 spec/support/cassettes
+.bundle/
+pkg/

data/CHANGELOG.md

@@ -15,4 +15,11 @@ __v0.0.3__ (22nd August 2013, *contributed by [tsigo](https://github.com/tsigo)*
 
 __v0.1.0__ (29th August 2013, *contributed by [tsigo](https://github.com/tsigo)*)
 
-* Adds support for searching for songs with `RapGenius::Song.search("Song, artist name or other query")`
+* Adds support for searching for songs with `RapGenius::Song.search("Song, artist name or other query")`
+
+__v1.0.0__ (27th January 2014)
+
+* Switches to using the private REST API used by the soon to be released
+[Genius iOS app](http://rapgenius.com/static/app).
+* Vastly improves quality of data available on songs and their lyrics
+* Provides access to media items and song artists

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2013 Tim Rogers
+Copyright (c) 2013-2014 Tim Rogers
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,128 +1,151 @@
-# rapgenius
+# rapgenius.rb
 
 ![Rap Genius logo](http://f.cl.ly/items/303W0c1i2r100j2u3Y0y/Screen%20Shot%202013-08-17%20at%2016.01.19.png)
 
 ## What does this do?
 
-It's a Ruby gem for accessing lyrics and explanations on
-[Rap Genius](http://rapgenius.com). 
+It's a Ruby gem for accessing lyrics, artists and explanations on
+[Rap Genius](http://rapgenius.com).
 
-They very sadly [don't have an API](https://twitter.com/RapGenius/status/245057326321655808) so I decided to replicate one for myself
-with a nice bit of screen scraping with [Nokogiri](https://github.com/sparklemotion/nokogiri), much like my [amex](https://github.com/timrogers/amex), [ucas](https://github.com/timrogers/ucas) and [lloydstsb](https://github.com/timrogers/lloydstsb) gems.
+In pre-1.0.0 versions, this gem used Nokogiri to scrape Rap Genius's pages,
+but no more. The new [Genius iOS app](http://rapgenius.com/static/app) uses
+a private API, which this gem makes use of.
 
 ## Installation
 
 Install the gem, and you're ready to go. Simply add the following to your
 Gemfile:
 
-`gem "rapgenius", "~> 0.1.0"`
+`gem "rapgenius", "~> 1.0.0"`
 
 ## Usage
 
-Songs on Rap Genius don't have numeric identifiers as far as I can tell - they're identified by a URL slug featuring the artist and song name, for instance "Big-sean-control-lyrics". We use this to fetch a particular track, like so:
+__The best way to get a decent idea of the attributes available on `Song` and
+the other objects is by looking through the files in `lib/rapgenius`.__
+
+### Searching
+
+You can search for songs by various fields. All of these
+methods return an array of `Song` objects...:
+
+```ruby
+RapGenius.search("Versace")
+RapGenius.search_by_title("Versace")
+RapGenius.search_by_artist("Migos")
+RapGenius.search_by_lyrics("medusa")
+```
+
+If more than 20 results are returned, you won't be able to get access to
+records beyond the 20th. There doesn't appear to be any pagination support
+in the API.
+
+### Songs
+
+Songs on Rap Genius have unique identifiers. They're not especially
+easy to find, but if you hover over the "pyong" button near the top of the page,
+you'll see the song's ID in the URL. Once you have an ID, you can load a
+song via the API:
 
 ```ruby
 require 'rapgenius'
-song = RapGenius::Song.find("Big-sean-control-lyrics")
+
+song = RapGenius::Song.find(176872)
+song.title # => "Versace"
+song.artists.map(&:name) # => ["Migos", "Drake", "Zaytoven"]
 ```
 
-Once you've got the song, you can easily load details about it. This uses
-Nokogiri to fetch the song's page and then parse it:
+Once you've found the song you're seeking, there's plenty of other useful
+details you can access:
 
 ```ruby
 song.title
-# => "Control"
-
-song.artist
-# => "Big Sean"
+# => "Versace"
 
-song.full_artist
-# => "Big Sean (Ft. Jay Electronica & Kendrick Lamar)"
+song.url
+# => "http://rapgenius.com/Migos-versace-lyrics""
 
-song.images
-# => ["http://s3.amazonaws.com/rapgenius/1376434983_jay-electronica.jpg", "http://s3.amazonaws.com/rapgenius/1375029260_Big%20Sean.png", "http://s3.amazonaws.com/rapgenius/Kendrick-Lamar-1024x680.jpg"]
+song.pyongs
+# => 22
 
 song.description
-# => "The non-album cut from Sean that basically blew up the Internet due to a world-beating verse by Kendrick Lamar...
+# => "Released in June 2013, not only did they take the beat from Soulja Boy’s OMG part 2 but they absolutely killed it."
 ```
 
-The `#annotations` accessor on a Song returns an array of RapGenius::Annotation
-objects corresponding to different annotated lines of the song, identified by
-their `id`.
 
-You can look these up manually using `RapGenius::Annotation.find("id")`. You
-can grab the ID for a lyric from a RapGenius page by right clicking on an annotation, copying the shortcut and then finding the number after "http://rapgenius.com".
+### Lines
+
+Once you've got a song, you can then go through its "lines", which includes
+annotated and unannotated parts of the content.
 
 ```ruby
-song.annotations
-# => [<RapGenius::Annotation>, <RapGenius::Annotation>...]
+line = song.lines[1]
 
-annotation = song.annotations[99]
+line.lyric
+# => Versace, Versace, Medusa head on me like I'm 'luminati
 
-annotation.lyric
-# => "And that goes for Jermaine Cole, Big KRIT, Wale\nPusha T, Meek Millz, A$AP Rocky, Drake\nBig Sean, Jay Electron', Tyler, Mac Miller"
+line.annotations
+# => ["Read about how this collaboration came to pass here..."]
 
-annotation.explanation
-# => "Kendrick calls out some of the biggest names in present day Hip-hop...""
+line.song
+# => #<RapGenius::Song:0x007fccdba50a50 @id=176872...
+```
 
-annotation.song == song # You can get back to the song from the annotation...
-# => true
+### Media
 
-annotation.id
-# => "2093001"
+Rap Genius provides great access to media like MP3s on Soundcloud or videos
+on YouTube direct from songs.
 
-annotation2 = RapGenius::Annotation.find("2093001") # Fetching directly...
+```ruby
+media = song.media.first
 
-annotation == annotations2
-# => true
+media.type
+# => "audio"
+
+media.provider
+# => "soundcloud"
+
+media.url
+# => "https://soundcloud.com/mixtapemechaniks/migos-ft-drake-versace-remix"
 ```
 
-You can search for songs by artist and/or title.
+### Artist
+
+You can navigate from a song to its artist, and then to other songs by that
+artist. Magic, huh?!
 
 ```ruby
-results = RapGenius::Song.search("Big Sean Control")
-# => [#<RapGenius::Song:0x007fbe4b9195e0
-#  @artist="Big Sean (Ft. Jay Electronica & Kendrick Lamar)",
-#  @title="Control",
-#  @url="http://rapgenius.com/Big-sean-control-lyrics">,
-# #<RapGenius::Song:0x007fbe4b920f70
-#  @artist="Big Sean (Ft. Jay Electronica & Kendrick Lamar)",
-#  @title="Control (French Version)",
-#  @url="http://rapgenius.com/Big-sean-control-french-version-lyrics">,
-# #<RapGenius::Song:0x007fbe4b920958
-#  @artist="Big Sean (Ft. Crobar, Jay Electronica & Kendrick Lamar)",
-#  @title="Control (Remix) [Kendrick Diss]",
-#  @url="http://rapgenius.com/Big-sean-control-remix-kendrick-diss-lyrics">,
-# #<RapGenius::Song:0x007fbe4b920250
-#  @artist=
-#   "Sa-roc (Ft. Big Sean - No I.D., Jay Electronica, Kendrick Lamar & Sa-roc)",
-#  @title="CONTROL",
-#  @url="http://rapgenius.com/Sa-roc-control-lyrics">,
-# #<RapGenius::Song:0x007fbe4b91ff30
-#  @artist="C3",
-#  @title=
-#   "Control ( Disses Kendrick Lamar , Jay-Z, Tyler The Creator, Big Sean, Meek Mill & More )",
-#  @url=
-#   "http://rapgenius.com/C3-control-disses-kendrick-lamar-jay-z-tyler-the-creator-big-sean-meek-mill-and-more-lyrics">]
-
-results[0].description
-# => "The non-album cut from Sean that basically blew up the Internet due to a world-beating verse by Kendrick Lamar...
-```
+artist = song.artist # or song.artists, song.featured_artists or song.producer_artists
 
-## Contributing
+artist.name
+# => "Migos"
+
+artist.type
+# => :primary
 
-After the last few contributions, there's one core thing I'd like to add to the gem:
+artist.url
+# => "http://rapgenius.com/artists/Migos"
 
-* __Support for *\*Genius*__ - Rap Genius also have other sites on subdomains like [News Genius](http://news.rapgenius.com) and [Poetry Genius](http://poetry.rapgenius.com). These could very easily be supported, since theyre identical in terms of markup.
+artist.description
+# => "Migos are an American hip-hop group from Atlanta, Georgia..."
+
+artist.songs
+# => [#<RapGenius::Song:0x007fccdb884398...]
+```
+
+## Contributing
 
 If you'd like to contribute anything else, go ahead or better still, make an issue and we can talk it over and spec it out! A few quick tips:
 
-* Don't update the version numbers before your pull request - I'll sort that part out for you!
+* Don't update the version numbers before your pull request - I'll sort that part out for you.
 * Make sure you write specs, then run them with `$ bundle exec rake`
 * Update this README.md file so I, and users, know how your changes work
 
+## Copyright
+
+Copyright (c) 2013-2014 Tim Rogers. See LICENSE for details.
+
 ## Get in touch
 
 [timrogers](https://github.com/timrogers) and [tsigo](https://github.com/tsigo) are the gem's primary contributors.
 
-Any questions, thoughts or comments? Email me at <me+rapgenius@timrogers.co.uk> or create an issue.
+Any questions, thoughts or comments? Email me at <me@timrogers.co.uk> or create an issue.

data/lib/rapgenius.rb

@@ -1,5 +1,42 @@
 require 'rapgenius/version'
-require 'rapgenius/scraper'
-require 'rapgenius/annotation'
+require 'rapgenius/client'
+require 'rapgenius/line'
 require 'rapgenius/song'
-require 'rapgenius/exceptions'
+require 'rapgenius/artist'
+require 'rapgenius/media'
+require 'rapgenius/exceptions'
+
+module RapGenius
+  extend RapGenius::Client
+
+  def self.search(query, options={})
+    response = Client::HTTPClient.get("/search", query: {q: query}.merge(options))
+
+    response["response"]["hits"].map do |song|
+      result = song["result"]
+
+      Song.new(
+        id: result["id"],
+        name: result["name"],
+        artist: Artist.new(
+          id: result["primary_artist"]["id"],
+          name: result["primary_artist"]["name"],
+          type: :primary
+        ),
+        title: result["title"]
+      )
+    end
+  end
+
+  def self.search_by_artist(query)
+    self.search(query, field: "primary_artist_name")
+  end
+
+  def self.search_by_title(query)
+    self.search(query, field: "title")
+  end
+
+  def self.search_by_lyrics(query)
+    self.search(query, field: "lyrics")
+  end
+end

data/lib/rapgenius/artist.rb

@@ -0,0 +1,54 @@
+# encoding: utf-8
+module RapGenius
+  class Artist
+    include RapGenius::Client
+
+    def self.find(id)
+      self.new(id: id).tap { |artist| artist.document }
+    end
+
+    def initialize(kwargs = {})
+      @id = kwargs.delete(:id)
+      @name = kwargs.delete(:name)
+      @type = kwargs.delete(:type)
+      self.url = "artists/#{@id}"
+    end
+
+    def response
+      document["response"]["artist"]
+    end
+
+    def name
+      @name ||= response["name"]
+    end
+
+    def image
+      @image ||= response["image_url"]
+    end
+
+    def url
+      response["url"]
+    end
+
+    def description
+      @description ||= response["description"]["dom"]["children"].map do |node|
+        parse_description(node)
+      end.flatten.join("")
+    end
+
+    def songs
+      @songs ||= fetch("/artists/#{@id}/songs")["response"]["songs"].map do |song|
+        Song.new(
+          artist: Artist.new(
+            name: song["primary_artist"]["name"],
+            id: song["primary_artist"]["id"],
+            type: :primary
+          ),
+          title: song["title"],
+          id: song["id"]
+        )
+      end
+    end
+
+  end
+end

data/lib/rapgenius/client.rb

@@ -0,0 +1,57 @@
+require 'httparty'
+
+module RapGenius
+  module Client
+    # HTTParty client
+    #
+    # Sets some useful defaults for all of our requests.
+    #
+    # See Scraper#fetch
+    class HTTPClient
+      include HTTParty
+
+      format   :json
+      base_uri 'https://api.rapgenius.com'
+      headers  'User-Agent' => "rapgenius.rb v#{RapGenius::VERSION}"
+    end
+
+    BASE_URL = HTTPClient.base_uri + "/".freeze
+
+    def url=(url)
+      unless url =~ /^https?:\/\//
+        @url = BASE_URL + url.gsub(/^\//, '')
+      else
+        @url = url
+      end
+    end
+
+    def document
+      @document ||= fetch(@url)
+    end
+
+    def fetch(url)
+      response = HTTPClient.get(url)
+
+      if response.code != 200
+        raise RapGenius::Error, "Received a #{response.code} HTTP response"
+      end
+
+      response.parsed_response
+    end
+
+    # Descriptions are formatted in an irritating way, encapsulating the
+    # various kinds of HTML tag that can be included. This parses that
+    # into text, but some content may be lost.
+    def parse_description(node)
+      if node.is_a? String
+        node
+      elsif node.is_a? Array
+        node.map { |subnode| parse_description(subnode) }
+      elsif node.is_a? Hash
+        return unless node.key? "children"
+        parse_description(node["children"])
+      end
+    end
+
+  end
+end

data/lib/rapgenius/exceptions.rb

@@ -1,4 +1,4 @@
 module RapGenius
-  class ScraperError < StandardError
+  class Error < StandardError
   end
 end

data/lib/rapgenius/line.rb

@@ -0,0 +1,59 @@
+module RapGenius
+  class Line
+    include RapGenius::Client
+
+    attr_reader :id
+
+    def self.find(id)
+      self.new(id: id).tap { |line| line.document }
+    end
+
+    def initialize(kwargs)
+      @id = kwargs.delete(:id)
+      @song = kwargs.delete(:song)
+      @lyric = kwargs.delete(:lyric)
+      self.url = "referents/#{@id}" if @id
+    end
+
+    def response
+      return nil unless @id
+      document["response"]["referent"]
+    end
+
+    def lyric
+      if @id
+        @lyric ||= response["fragment"]
+      else
+        @lyric
+      end
+    end
+
+    def annotated?
+      !!@id
+    end
+
+    alias_method :explained?, :annotated?
+
+    # A line can have multiple annotations, usually if it has a community one
+    # and a verified one. Ideally, these would be encapsulated into an
+    # Annotation class, but I don't have time for now.
+    def explanations
+      return nil unless @id
+      @explanation ||= response["annotations"].map do |annotation|
+        annotation["body"]["dom"]["children"].map do |node|
+          parse_description(node)
+        end.join("")
+      end.flatten
+    end
+
+    alias_method :annotations, :explanations
+
+    def song
+      if @id
+        @song ||= Song.find(response['song_id'])
+      else
+        @song
+      end
+    end
+  end
+end