bnet_scraper 0.6.0 → 0.7.0

data/README.md

@@ -41,8 +41,9 @@ just for you.  Call `BnetScraper::Starcraft2#full_profile_scrape` with the usual
 ProfileScraper would take, and it will eager-load the achievements, matches, and leagues.
 
 ``` ruby
-scraper = BnetScraper::Starcraft2.full_profile_scrape(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
-profile = scraper.scrape
+profile = BnetScraper::Starcraft2.full_profile_scrape(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
+profile.class.name # => 'BnetScraper::Starcraft2::Profile'
+profile.leagues.first.name # => 'Changeling Bravo'
 ```
 
 Alternatively, these scrapers can be accessed in isolation.
@@ -60,12 +61,12 @@ All of the scrapers take an options hash, and can be created by either passing a
 passing the account information in the options hash.  Thus, either of these two approaches work:
 
 ``` ruby
-BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/12345/1/TestAccount/')
-BnetScraper::Starcraft2::ProfileScraper.new(bnet_id: '12345', account: 'TestAccount', region: 'na')
+scraper1 = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
+scraper2 = BnetScraper::Starcraft2::ProfileScraper.new(bnet_id: '2377239', account: 'Demon', region: 'na')
 ```
 
-All scrapers have a `#scrape` method that triggers the scraping and storage.  By default they will return the result,
-but an additional `#output` method exists to retrieve the results subsequent times without re-scraping.
+All scrapers have a `#scrape` method that triggers the scraping and storage.  The `#scrape` method will return an
+object containing the scraped data result.
 
 ### BnetScraper::Starcraft2::ProfileScraper
 
@@ -78,6 +79,17 @@ profile = scraper.scrape
 profile.class.name # => BnetScraper::Starcraft2::Profile
 ```
 
+Additionally, the resulting `BnetScraper::Starcraft2::Profile` object has methods to scrape additional
+information without the need of creating another scraper.  For example, if you need to pull league information up
+on a player, you may call `BnetScraper::Starcraft2::Profile#leagues` and it will scrape and store the information
+for memoized access.
+
+``` ruby
+scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
+profile = scraper.scrape
+profile.leagues.map(&:division) #=> ['Bronze']
+```
+
 ### BnetScraper::Starcraft2::LeagueScraper
 
 This pulls information on a specific league for a specific account.  It is best used either in conjunction with a
@@ -86,15 +98,15 @@ profile scrape that profiles a URL, or if you happen to know the specific league
 ``` ruby
 scraper = BnetScraper::Starcraft2::LeagueScraper.new(league_id: '12345', account: 'Demon', bnet_id: '2377239')
 scraper.scrape
-# => {
-  season: '6',
-  name: 'Aleksander Pepper',
-  division: 'Diamond',
-  size: '4v4',
-  random: false,
-  bnet_id: '2377239',
-  account: 'Demon'
-}
+
+# => #<BnetScraper::Starcraft2::League:0x007f89eab7a680
+@account="Demon",
+@bnet_id="2377239",
+@division="Bronze",
+@name="Changeling Bravo",
+@random=false,
+@season="2013 Season 4",
+@size="3v3">
 ```
 
 ### BnetScraper::Starcraft2::AchievementScraper
@@ -103,33 +115,30 @@ This pulls achievement information for an account.  Note that currently only ret
 not the in-depth, by-category achievement information.
 
 ``` ruby
-scraper = BnetScraper::Starcraft2::AchievementScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
-scraper.scrape
-# => {
-  recent: [
-    { title: 'Blink of an Eye', description: 'Complete round 24 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
-    { title: 'Whack-a-Roach', description: 'Complete round 9 in "Starcraft Master" in under 45 seconds', earned: '3/5/2012' },
-    { title: 'Safe Zone', description: 'Complete round 8 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
-    { title: 'Starcraft Master', description: 'Complete all 30 rounds in "Starcraft Master"', earned: '3/5/2012' },
-    { title: 'Starcraft Expert', description: 'Complete any 25 rounds in "Starcraft Master"', earned: '3/5/2012' },
-    { title: 'Starcraft Apprentice', description: 'Complete any 20 rounds in "Starcraft Master"', earned: '3/5/2012' }
-  ],
-  showcase: [
-    { title: 'Hot Shot', description: 'Finish a Qualification Round with an undefeated record.' },
-    { title: 'Starcraft Master', description: 'Complete all rounds in "Starcraft Master"' },
-    { title: 'Team Protoss 500', description: 'Win 500 team league matches as Protoss' },
-    { title: 'Night of the Living III', description: 'Survive 15 Infested Horde Attacks in the "Night 2 Die" mode of the "Left 2 Die" scenario.' },
-    { title: 'Team Top 100 Diamond', description: 'Finish a Season in Team Diamond Division' }
-												
-  ],
-  progress: {
-    liberty_campaign: '1580',
-    exploration: '480',
-    custom_game: '330',
-    cooperative: '660',
-    quick_match: '170'
-  }
-}
+scraper = BnetScraper::Starcraft2::AchievementScraper.new(
+  url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/'
+)
+achievement_information = scraper.scrape
+achievement_information[:recent].size # => 6
+achievement_information[:recent].first
+# => #<BnetScraper::Starcraft2::Achievement:0x007fef52b0b488
+@description="Win 50 Team Unranked or Ranked games as Zerg.",
+@earned=#<Date: 2013-04-04 ((2456387j,0s,0n),+0s,2299161j)>,
+@title="50 Wins: Team Zerg">
+
+achievement_information[:progress]
+# => {:liberty_campaign=>1580,
+:swarm_campaign=>1120,
+:matchmaking=>1410,
+:custom_game=>120,
+:arcade=>220,
+:exploration=>530}
+
+achievement_information[:showcase].size # => 5
+achievement_information[:showcase].first
+# => #<BnetScraper::Starcraft2::Achievement:0x007fef52abcb08
+@description="Finish a Qualification Round with an undefeated record.",
+@title="Hot Shot">
 ```
 
 ### BnetScraper::Starcraft2::MatchHistoryScraper
@@ -138,18 +147,20 @@ This pulls the 25 most recent matches played for an account. Note that this is o
 will likely not be as fast as in-game.
 
 ``` ruby
-scraper = BnetScraper::Starcraft2::MatchHistoryScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
-scraper.scrape
-# => {
-  wins: '15',
-  losses: '10',
-  matches: [
-    { map_name: 'Bx Monobattle - Sand Canyon (Fix)', outcome: :win, type: 'Custom', date: '3/12/2012' },
-    { map_name: 'Deadlock Ridge', outcome: :loss, type: '4v4', date: '3/12/2012' },
-    { map_name: 'District 10', outcome: :win, type: '4v4', date: '3/12/2012' },
-    # ...
-  ]
-}
+scraper = BnetScraper::Starcraft2::MatchHistoryScraper.new(
+  url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/'
+)
+matches = scraper.scrape
+matches.size # => 25
+wins = matches.count { |m| m.outcome == :win } # => 15
+losses = matches.count { |m| m.outcome == :loss } # => 10
+
+matches.first
+# =>  #<BnetScraper::Starcraft2::Match:0x007fef55113428
+@date="5/24/2013",
+@map_name="Queen's Nest",
+@outcome=:win,
+@type="3v3">
 ```
 
 ## BnetScraper::Starcraft2::Status

data/bnet_scraper.gemspec

@@ -3,12 +3,13 @@ $:.push File.expand_path("../lib", __FILE__)
 
 Gem::Specification.new do |s|
   s.name        = "bnet_scraper"
-  s.version     = "0.6.0"
+  s.version     = "0.7.0"
   s.authors     = ["Andrew Nordman"]
   s.email       = ["cadwallion@gmail.com"]
   s.homepage    = "https://github.com/agoragames/bnet_scraper/"
   s.summary     = %q{Battle.net Profile Scraper}
   s.description = %q{BnetScraper is a Nokogiri-based scraper of Battle.net profile information. Currently this only includes Starcraft2.}
+  s.license     = 'MIT'
 
   s.files         = `git ls-files`.split("\n")
   s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")

data/lib/bnet_scraper/starcraft2.rb

@@ -28,8 +28,6 @@ module BnetScraper
       'tw.battle.net' => 'fea'
     }
 
-    
-      
     # This is a convenience method that chains calls to ProfileScraper,
     # followed by a scrape of each league returned in the `leagues` array
     # in the profile_data.  The end result is a fully scraped profile with
@@ -38,16 +36,14 @@ module BnetScraper
     # See `BnetScraper::Starcraft2::ProfileScraper` for more information on
     # the parameters being sent to `#full_profile_scrape`.
     #
-    # @param bnet_id - Battle.net Account ID 
-    # @param account - Battle.net Account Name
-    # @param region  - Battle.net Account Region
-    # @return profile_data - Hash containing complete profile and league data
-    #   scraped from the website
-    def self.full_profile_scrape bnet_id, account, region = 'na'
-      profile_scraper = ProfileScraper.new bnet_id: bnet_id, account: account, region: region
+    # @param options - Hash of profile options (url, bnet_id/account/region, etc).  See 
+    #   `BnetScraper::Starcraft::BaseScraper` for more information on hash options.
+    # @return [BnetScraper::Profile] profile_data - Profile object containing complete profile and league data
+    def self.full_profile_scrape options = {}
+      profile_scraper = ProfileScraper.new options
       profile = profile_scraper.scrape
       profile.leagues.each do |league|
-        league.scrape_league
+        league.scrape
       end
       profile.achievements
       profile.match_history

data/lib/bnet_scraper/starcraft2/achievement.rb

@@ -1,24 +1,13 @@
-require 'date'
 module BnetScraper
   module Starcraft2
     class Achievement
-      attr_accessor :title, :description
-      attr_reader :earned
+      attr_accessor :title, :description, :earned
 
       def initialize options = {}
         options.each_key do |key|
           self.send "#{key}=", options[key]
         end
       end
-
-      def earned= date
-        @earned = convert_date date
-      end
-
-      def convert_date date
-        month, day, year = date.scan(/(\d+)\/(\d+)\/(\d+)/).first.map(&:to_i)
-        Date.new year, month, day
-      end
     end
   end
 end

data/lib/bnet_scraper/starcraft2/achievement_scraper.rb

@@ -1,38 +1,37 @@
 require 'bnet_scraper/starcraft2/achievement'
+require 'date'
 
 module BnetScraper
   module Starcraft2
     # This pulls achievement information for an account.  Note that currently only returns the overall achievements,
     # not the in-depth, by-category achievement information.
     #
-    #  scraper = BnetScraper::Starcraft2::AchievementScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
-    #  scraper.scrape
-    #  # => {
-    #    recent: [
-    #      { title: 'Blink of an Eye', description: 'Complete round 24 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
-    #      { title: 'Whack-a-Roach', description: 'Complete round 9 in "Starcraft Master" in under 45 seconds', earned: '3/5/2012' },
-    #      { title: 'Safe Zone', description: 'Complete round 8 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
-    #      { title: 'Starcraft Master', description: 'Complete all 30 rounds in "Starcraft Master"', earned: '3/5/2012' },
-    #      { title: 'Starcraft Expert', description: 'Complete any 25 rounds in "Starcraft Master"', earned: '3/5/2012' },
-    #      { title: 'Starcraft Apprentice', description: 'Complete any 20 rounds in "Starcraft Master"', earned: '3/5/2012' }
-    #    ],
-    #    showcase: [
-    #      { title: 'Hot Shot', description: 'Finish a Qualification Round with an undefeated record.' },
-    #      { title: 'Starcraft Master', description: 'Complete all rounds in "Starcraft Master"' },
-    #      { title: 'Team Protoss 500', description: 'Win 500 team league matches as Protoss' },
-    #      { title: 'Night of the Living III', description: 'Survive 15 Infested Horde Attacks in the "Night 2 Die" mode of the "Left 2 Die" scenario.' },
-    #      { title: 'Team Top 100 Diamond', description: 'Finish a Season in Team Diamond Division' }
-    #                          
-    #    ],
-    #    progress: {
-    #      liberty_campaign: '1580',
-    #      swarm_campaign: '480',
-    #      matchmaking: '1100',
-    #      custom_game: '330',
-    #      arcade: '660',
-    #      exploration: '170'
-    #    }
-    #  }
+    # ``` ruby
+    # scraper = BnetScraper::Starcraft2::AchievementScraper.new(
+    #     url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/'
+    # )
+    # achievement_information = scraper.scrape
+    # achievement_information[:recent].size # => 6
+    # achievement_information[:recent].first
+    # # => #<BnetScraper::Starcraft2::Achievement:0x007fef52b0b488
+    # @description="Win 50 Team Unranked or Ranked games as Zerg.",
+    #   @earned=#<Date: 2013-04-04 ((2456387j,0s,0n),+0s,2299161j)>,
+    #   @title="50 Wins: Team Zerg">
+    #
+    # achievement_information[:progress]
+    # # => {:liberty_campaign=>1580,
+    # :swarm_campaign=>1120,
+    #   :matchmaking=>1410,
+    #   :custom_game=>120,
+    #   :arcade=>220,
+    #   :exploration=>530}
+    #
+    # achievement_information[:showcase].size # => 5
+    # achievement_information[:showcase].first
+    # # => #<BnetScraper::Starcraft2::Achievement:0x007fef52abcb08
+    # @description="Finish a Qualification Round with an undefeated record.",
+    #   @title="Hot Shot">
+    # ```
     class AchievementScraper < BaseScraper
       attr_reader :recent, :progress, :showcase, :response
 
@@ -78,11 +77,23 @@ module BnetScraper
           Achievement.new({
             title: div.children[1].inner_text,
             description: div.children[2].inner_text.strip,
-            earned: response.css(".recent-tile")[num].css('span')[1].inner_text
+            earned: convert_date(response.css(".recent-tile")[num].css('span')[1].inner_text)
           })
         end
       end
 
+      def convert_date date
+        dates = date.scan(/(\d+)\/(\d+)\/(\d+)/).first.map(&:to_i)
+        
+        if region == 'na'
+          month, day, year = dates
+        else
+          day, month, year = dates
+        end
+          
+        Date.new year, month, day
+      end
+
 
 
       # Scrapes the progress of each achievement category from the account's achievements
@@ -106,10 +117,12 @@ module BnetScraper
       # @return [Array<Achievement>] Array containing all the showcased achievements
       def scrape_showcase
         @showcase = response.css("#showcase-module .progress-tile").map do |achievement|
-          Achievement.new({
-            title: achievement.css('.tooltip-title').inner_text.strip,
-            description: achievement.children[3].children[2].inner_text.strip
-          })
+          if !achievement.values.first.split.include? 'empty'
+            Achievement.new({
+              title: achievement.css('.tooltip-title').inner_text.strip,
+              description: achievement.children[3].children[2].inner_text.strip
+            })
+          end
         end
         @showcase
       end
@@ -121,6 +134,7 @@ module BnetScraper
           showcase: @showcase
         }
       end
+
     end
   end
 end

data/lib/bnet_scraper/starcraft2/base_scraper.rb

@@ -1,3 +1,5 @@
+require 'net/http'
+
 module BnetScraper
   module Starcraft2
     # BaseScraper handles the account information extraction. Each scraper can either be passed a profile URL or

data/lib/bnet_scraper/starcraft2/league.rb

@@ -38,13 +38,15 @@ module BnetScraper
         if self.instance_variable_get(attribute)
           return self.instance_variable_get(attribute)
         else
-          scrape_league
+          scrape
           self.instance_variable_get(attribute)
         end
       end
 
-      def scrape_league
-        scraped_data = LeagueScraper.new(url: href).scrape
+      def scrape
+        scraper = LeagueScraper.new(url: href)
+        scraper.scrape
+        scraped_data = scraper.output
         scraped_data.each_key do |key|
           self.send "#{key}=", scraped_data[key]
         end

data/lib/bnet_scraper/starcraft2/league_scraper.rb

@@ -5,17 +5,19 @@ module BnetScraper
     # This pulls information on a specific league for a specific account.  It is best used either in conjunction with a
     # profile scrape that profiles a URL, or if you happen to know the specific league\_id and can pass it as an option.
     #
-    #  scraper = BnetScraper::Starcraft2::LeagueScraper.new(league_id: '12345', account: 'Demon', bnet_id: '2377239')
-    #  scraper.scrape
-    #  # => {
-    #    season: '6',
-    #    name: 'Aleksander Pepper',
-    #    division: 'Diamond',
-    #    size: '4v4',
-    #    random: false,
-    #    bnet_id: '2377239',
-    #    account: 'Demon'
-    #  }
+    #   ``` ruby
+    #   scraper = BnetScraper::Starcraft2::LeagueScraper.new(league_id: '12345', account: 'Demon', bnet_id: '2377239')
+    #   scraper.scrape
+    #
+    #   # => #<BnetScraper::Starcraft2::League:0x007f89eab7a680
+    #   @account="Demon",
+    #   @bnet_id="2377239",
+    #   @division="Bronze",
+    #   @name="Changeling Bravo",
+    #   @random=false,
+    #   @season="2013 Season 4",
+    #   @size="3v3">
+    #   ```
     class LeagueScraper < BaseScraper
       attr_reader :league_id, :season, :size, :random, :name, :division
 
@@ -28,6 +30,7 @@ module BnetScraper
           @league_id = options[:url].match(/http:\/\/.+\/sc2\/.+\/profile\/.+\/\d{1}\/.+\/ladder\/(.+)(#current-rank)?/).to_a[1]
         else
           @league_id = options[:league_id]
+          @url = "#{profile_url}ladder/#{@league_id}"
         end
       end
 
@@ -42,7 +45,7 @@ module BnetScraper
           @season, @size, @random, @division, @name = header_values
           
           @random = !@random.nil?
-          output
+          League.new(output)
         else
           raise BnetScraper::InvalidProfileError
         end

data/lib/bnet_scraper/starcraft2/profile.rb

@@ -1,4 +1,3 @@
-
 module BnetScraper
   module Starcraft2
     class Profile
@@ -6,7 +5,7 @@ module BnetScraper
         :highest_solo_league, :current_team_league, :highest_team_league,
         :career_games, :games_this_season, :terran_swarm_level, :protoss_swarm_level,
         :zerg_swarm_level, :leagues, :swarm_levels, :terran_campaign_completion,
-        :zerg_campaign_completion
+        :zerg_campaign_completion, :clan_tag, :clan_name, :account
 
       def initialize options = {}
         options.each_key do |key|
@@ -58,6 +57,16 @@ module BnetScraper
           false
         end
       end
+
+      def scrape
+        scraper = ProfileScraper.new(url: @url)
+        scraper.scrape
+
+        scraped_data = scraper.output
+        scraped_data.each_key do |key|
+          self.send "#{key}=", scraped_data[key]
+        end
+      end
     end
   end
 end