greenmonster 0.1.0.pre → 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
data/Readme.markdown ADDED
@@ -0,0 +1,60 @@
1
+ Greenmonster
2
+ ============
3
+
4
+ Greenmonster is a toolkit for baseball stat enthusiasts or sabermetricians to build a database of play-by-play stats from MLB's [Gameday XML data](http://gd.mlb.com/components/game/).
5
+
6
+ The gem currently spiders games from MLB's servers. Over the season, I will be extracting pieces out of my toolkit to parse Gameday XML data to produce play-by-play database of MLB and MiLB stats.
7
+
8
+ Usage
9
+ -----
10
+
11
+ If you don't want to specify a download location every time you run the spider, you can set a default games folder location using Greenmonster.set_games_location:
12
+
13
+ # Set games folder location
14
+ >> Greenmonster.set_games_folder('/Users/geoff/games/')
15
+
16
+ The spider utility has three public class methods: Spider.pull_game, Spider.pull_day and Spider.pull_days.
17
+
18
+ Spider.pull_game takes a game_id (the folder name of the game on the Gameday server) and a hash of options as arguments. If for some reason the game does not fall in the expected folder for the game's date or sport code, you can add those options to the arguments hash. Other options include :games_folder and :print_games (if false, game IDs are not printed to screen).
19
+
20
+ # Pulls MLB's 7/4/2011 Toronto @ Boston game
21
+ >> Greenmonster::Spider.pull_game('gid_2011_07_04_tormlb_bosmlb_1', {:print_games => false})
22
+
23
+ Spider.pull_day takes an hash of options as an argument. Greenmonster will create subfolders by MLB "sport_code" (MLB games fall under 'mlb', various minor league games and non-MLB/MiLB games fall under other sport code designations), and then children folders for years, months, days, and specific games.
24
+
25
+ # Pulls all MLB games for today
26
+ >> Greenmonster::Spider.pull_day({:date => Date.today, :games_folder => './home/geoff/games'})
27
+
28
+ # Pulls all rookie league games for today
29
+ >> Greenmonster::Spider.pull_day({:sport_code => 'rok', :date => Date.today, :games_folder => './home/geoff/games'})
30
+
31
+
32
+ Spider.pull_days takes a range of dates to process as an argument, plus a hash of arguments to pass to Spider.pull.
33
+
34
+ # Pulls all MLB games for in April, 2012
35
+ >> Greenmonster::Spider.pull_days((Date.new(2012,4,1)..Date.new(2012,4,30)), {:games_folder => './home/geoff/games'})
36
+
37
+
38
+ Requirements
39
+ ------------
40
+ - Ruby 1.9
41
+ - Nokogiri
42
+ - HTTParty
43
+
44
+ Testing
45
+ -------
46
+
47
+ The test suite downloads a few days of data, so it is not fast to execute.
48
+
49
+
50
+ License
51
+ -------
52
+ (The MIT License)
53
+
54
+ Copyright © [Geoff Harcourt](http://github.com/geoffharcourt) 2012
55
+
56
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
57
+
58
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
59
+
60
+ THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -1,77 +1,96 @@
1
1
  # The Gameday XML Spider utility
2
2
  class Greenmonster::Spider
3
3
  include HTTParty
4
+
5
+ ##
6
+ # Pull Gameday XML files for a given game, specified by the game
7
+ # ID. If date and sport code are not specified as options, these
8
+ # values are guessed from the game ID string using the home team's
9
+ # sport code and the date from the scheduled date values in the game
10
+ # ID.
11
+ #
12
+ # Example:
13
+ # >> Gameday::Spider.pull_game('',{:games_folder => })
14
+
15
+ def self.pull_game(game_id,args = {})
16
+ args = {
17
+ :date => args[:date] || Date.new(game_id[4,4].to_i, game_id[9,2].to_i, game_id[12,2].to_i),
18
+ :sport_code => args[:sport_code] || game_id[25,3],
19
+ :print_games => true,
20
+ :games_folder => Greenmonster.games_folder
21
+ }.merge(args)
22
+
23
+ puts game_id if args[:print_games]
24
+
25
+ raise "Games folder location required." if args[:games_folder].nil?
26
+
27
+ paths = {
28
+ :localGameFolder => "#{args[:games_folder]}/#{args[:sport_code]}/#{format_date_as_folder(args[:date])}/#{game_id}/",
29
+ :mlbGameFolder => "#{gameday_league_and_date_url(args)}/#{game_id}/"
30
+ }
31
+
32
+ FileUtils.mkdir_p paths[:localGameFolder] + 'inning'
33
+
34
+ begin
35
+ # Always copy linescore first. If we can't get this
36
+ # data, all other game data is useless.
37
+ copy_gameday_xml('linescore.xml',paths)
38
+
39
+ if args[:date].year > 2007
40
+ copy_gameday_xml('inning_all.xml',paths)
41
+ copy_gameday_xml('inning_hit.xml',paths)
42
+ else
43
+ # Iterate through the inning files, but skip inning
44
+ # files numbered 0 (some bad spring training data).
45
+ # Necessary for games prior to 2008 because there is
46
+ # no inning_all.xml file in older games.
47
+ (Nokogiri::XML(self.get("#{paths[:mlbGameFolder]}/inning/").body).search('a')).each do |ic|
48
+ copy_gameday_xml(ic.attribute('href').value,paths) if ic.attribute('href').value[-3,3] == "xml" unless ic.attribute('href').value[-6,6] == "_0.xml" or ic.attribute('href').value.include?('Score')
49
+ end
50
+ end
51
+
52
+ # Copy base data files
53
+ # (if inning data wasn't there, this gets skipped)
54
+ ['boxscore.xml','eventLog.xml','players.xml'].each do |file|
55
+ copy_gameday_xml(file,paths)
56
+ end
57
+ rescue StandardError => bang
58
+ puts "Unable to download some data for #{e.attribute('href').value}"
59
+ end
60
+
61
+ return game_id
62
+ end
4
63
 
5
64
  ##
6
65
  # Pull Gameday XML files for a given date. Default options for
7
66
  # the spider are to pull games with sport_code of 'mlb' (games
8
67
  # played by MLB games rather than MiLB teams or foreign teams)
9
- # and to pull games on the current date. You must specify a
10
- # :games_folder argument that is the root folder under which
11
- # all games will be placed.
68
+ # and to pull games on the current date.
12
69
  #
13
70
  # Example:
14
71
  # # Pull games from July 4, 2011
15
- # >> Gameday::Spider.pull({:date => Date.new(2011,7,1), :games_location => '/Users/geoff/games'})
72
+ # >> Gameday::Spider.pull_day({:date => Date.new(2011,7,1), :games_folder => '/Users/geoff/games'})
16
73
  #
17
74
  # Arguments:
18
75
  # args: (Hash)
76
+ #
19
77
 
20
78
  def self.pull_day(args = {})
21
79
  args = {
22
80
  :date => Date.today,
23
- :league => 'mlb',
24
- :print_games => true
81
+ :sport_code => 'mlb',
25
82
  }.merge(args)
26
83
 
27
- raise "Games folder location required." if args[:games_folder].nil?
28
-
29
- game_day_url_for_date = "http://gd2.mlb.com/components/game/#{args[:league]}/#{format_date_as_folder(args[:date])}"
30
-
31
84
  # Iterate through every hyperlink on the page.
32
85
  # These links represent the individual game folders
33
86
  # for each date. Reject any links that aren't to game
34
87
  # folders or that are to what look like backup game
35
88
  # folders.
36
- (Nokogiri::XML(self.get(game_day_url_for_date))/"a").reject{|l| l.attribute('href').value[0,4] != "gid_" or l.attribute('href').value[-5,4] == "_bak"}.each do |e|
37
- puts e.attribute('href').value.gsub('/','') if args[:print_games]
38
-
39
- paths = {
40
- :localGameFolder => "#{args[:games_folder]}/#{args[:league]}/#{format_date_as_folder(args[:date])}/#{e.attribute('href').value}",
41
- :mlbGameFolder => "#{game_day_url_for_date}/#{e.attribute('href').value}"
42
- }
43
-
44
- FileUtils.mkdir_p paths[:localGameFolder] + 'inning'
45
-
46
- begin
47
- # Always copy linescore first. If we can't get this
48
- # data, all other game data is useless.
49
- copy_gameday_xml('linescore.xml',paths)
50
-
51
- if args[:date].year > 2007
52
- copy_gameday_xml('inning_all.xml',paths)
53
- copy_gameday_xml('inning_hit.xml',paths)
54
- else
55
- # Iterate through the inning files, but skip inning
56
- # files numbered 0 (some bad spring training data).
57
- # Necessary for games prior to 2008 because there is
58
- # no inning_all.xml file in older games.
59
- (Nokogiri::XML(self.get("#{paths[:mlbGameFolder]}/inning/").body).search('a')).each do |ic|
60
- copy_gameday_xml(ic.attribute('href').value,paths) if ic.attribute('href').value[-3,3] == "xml" unless ic.attribute('href').value[-6,6] == "_0.xml" or ic.attribute('href').value.include?('Score')
61
- end
62
- end
63
-
64
- # Copy base data files
65
- # (if inning data wasn't there, this gets skipped)
66
- ['boxscore.xml','eventLog.xml','players.xml'].each do |file|
67
- copy_gameday_xml(file,paths)
68
- end
69
- rescue StandardError => bang
70
- puts "Unable to download some data for #{e.attribute('href').value}"
71
- end
89
+ (Nokogiri::XML(self.get(gameday_league_and_date_url(args)))/"a").reject{|l| l.attribute('href').value[0,4] != "gid_" or l.attribute('href').value[-5,4] == "_bak"}.each do |e|
90
+ self.pull_game(e.attribute('href').value.gsub('/',''),args)
72
91
  end
73
-
74
- game_day_url_for_date
92
+
93
+ return gameday_league_and_date_url(args)
75
94
  end
76
95
 
77
96
  ##
@@ -80,7 +99,7 @@ class Greenmonster::Spider
80
99
  #
81
100
  # Example:
82
101
  # # Pull all games in MLB in July 2011
83
- # >> Gameday::Spider.pull_days(Date.new(2011,7,1)..Date.new(2011,7,31), {:games_location => '/Users/geoff/games'})
102
+ # >> Gameday::Spider.pull_days(Date.new(2011,7,1)..Date.new(2011,7,31), {:games_folder => '/Users/geoff/games'})
84
103
  #
85
104
  # Arguments:
86
105
  # range: (Range)
@@ -91,11 +110,27 @@ class Greenmonster::Spider
91
110
  end
92
111
 
93
112
  private
113
+ ##
114
+ # Return the Gameday URL for a given sport_code
115
+ # and date. Argument must include :date, but
116
+ # :sport_code will be assumed to be 'mlb' if not
117
+ # specified.
118
+ #
119
+ # Arguments:
120
+ # args: (Hash)
121
+ #
122
+ def self.gameday_league_and_date_url(args = {})
123
+ raise "Date required." unless args[:date]
124
+ args[:sport_code] ||= 'mlb'
125
+
126
+ "http://gd2.mlb.com/components/game/#{args[:sport_code]}/#{format_date_as_folder(args[:date])}"
127
+ end
128
+
94
129
  ##
95
130
  # Copy XML files from the Gameday severs to your
96
131
  # local machine for parsing and analysis. This method
97
- # is used by Spider.pull to put files on your local
98
- # machine in a similar layout to the one used by
132
+ # is used by Spider pull class methods to put files on
133
+ # your local machine in a similar layout to the one used by
99
134
  # Gameday. The paths argument gets built by Spider.pull
100
135
  # during the pull process.
101
136
  #
@@ -113,7 +148,7 @@ class Greenmonster::Spider
113
148
  # Output a folder format similar to the one used by Gameday.
114
149
  #
115
150
  # Example:
116
- # Spider.format_date_as_folder(Date.new(2011,7,4))
151
+ # >> Spider.format_date_as_folder(Date.new(2011,7,4))
117
152
  # => "year_2011/month_07/day_04"
118
153
  #
119
154
  # Arguments:
data/lib/greenmonster.rb CHANGED
@@ -3,8 +3,35 @@ require 'nokogiri'
3
3
  require 'fileutils'
4
4
 
5
5
  module Greenmonster
6
+ @@games_folder = nil
6
7
 
8
+ ##
9
+ # Set the default folder to which games
10
+ # are saved after being downloaded from
11
+ # the server.
12
+ #
13
+ # Example:
14
+ # => Greenmonster.set_games_folder("/Users/geoff/game_data")
15
+ #
16
+ # Arguments:
17
+ # location: (String)
18
+ #
7
19
 
20
+ def self.set_games_folder(location)
21
+ @@games_folder = location
22
+ end
23
+
24
+ ##
25
+ # Return the default games folder location
26
+ #
27
+ # Example:
28
+ # >> Greenmonster.set_games_folder("/Users/geoff/game_data")
29
+ # >> Greenmonster.games_folder
30
+ # => "/Users/geoff/game_data"
31
+
32
+ def self.games_folder
33
+ @@games_folder
34
+ end
8
35
  end
9
36
 
10
37
  require 'greenmonster/spider'
@@ -1,78 +1,15 @@
1
1
  require 'minitest/autorun'
2
2
  require 'greenmonster'
3
3
 
4
- class SpiderTest < MiniTest::Unit::TestCase
4
+ class GreenmonsterTest < MiniTest::Unit::TestCase
5
5
  def setup
6
- @local_test_data_location = "./greenmonster_test_games"
7
- FileUtils.mkdir_p @local_test_data_location
8
6
  end
9
7
 
10
- def test_format_date_as_folder
11
- assert_equal "year_2011/month_07/day_04", Greenmonster::Spider.format_date_as_folder(Date.new(2011,7,4))
12
- assert_equal "year_2012/month_12/day_31", Greenmonster::Spider.format_date_as_folder(Date.new(2012,12,31))
13
- end
14
-
15
- def test_copy_gameday_xml
16
- paths = {
17
- :localGameFolder => "#{@local_test_data_location}/mlb/year_2011/month_05/day_03/gid_2011_05_03_sfnmlb_nynmlb_1/",
18
- :mlbGameFolder => "http://gd2.mlb.com/components/game/mlb/year_2011/month_05/day_03/gid_2011_05_03_sfnmlb_nynmlb_1/"
19
- }
20
- FileUtils.mkdir_p paths[:localGameFolder] + 'inning'
21
-
22
- %w(linescore.xml inning_all.xml).each do |f|
23
- Greenmonster::Spider.copy_gameday_xml(f,paths)
24
- end
25
-
26
- assert_equal 287337, Nokogiri::XML(open(paths[:localGameFolder] + 'linescore.xml')).search("game").first.attribute('game_pk').value.to_i
27
- assert_equal 10, Nokogiri::XML(open(paths[:localGameFolder] + 'inning/inning_all.xml')).search("inning").count
28
- assert_equal 400023, Nokogiri::XML(open(paths[:localGameFolder] + 'inning/inning_all.xml')).search("atbat").first.attribute('batter').value.to_i
29
- end
30
-
31
- def test_pull_day
32
- Greenmonster::Spider.pull_day({:print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2011,6,7)})
33
-
34
- assert_equal 17, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_06/day_07/').count
35
-
36
- game_location = @local_test_data_location + '/mlb/year_2011/month_06/day_07/gid_2011_06_07_bosmlb_nyamlb_1'
37
- %w(linescore.xml boxscore.xml players.xml eventLog.xml inning).each do |f|
38
- assert Dir.entries(game_location).include? f
39
- end
40
-
41
- boxscore = Nokogiri::XML(open(game_location + '/boxscore.xml'))
42
- eventlog = Nokogiri::XML(open(game_location + '/eventLog.xml'))
43
- players = Nokogiri::XML(open(game_location + '/players.xml'))
44
- innings = Nokogiri::XML(open(game_location + '/inning/inning_all.xml'))
45
-
46
- assert_equal 8, boxscore.search('pitcher').count
47
- assert_equal 147, boxscore.search('boxscore').first.attribute('home_id').value.to_i
48
- assert eventlog.search("event[number='19']").attribute('description').value.include?('Pedroia walks.')
49
- assert_equal 'Magadan', players.search("team[type='away']").search("coach[position='batting_coach']").attribute('last').value
50
- assert_equal 'Strikeout', innings.search('atbat').last.attribute('event').value
51
- end
52
-
53
- def test_pull_minor_league_games
54
- Greenmonster::Spider.pull_day({:league => 'aaa', :print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2011,9,1)})
55
-
56
- assert_equal 13, Dir.entries(@local_test_data_location + '/aaa/year_2011/month_09/day_01/').count
57
- %w(linescore.xml boxscore.xml players.xml eventLog.xml inning).each do |f|
58
- assert Dir.entries(@local_test_data_location + '/aaa/year_2011/month_09/day_01/gid_2011_09_01_albaaa_mrbaaa_1/').include? f
59
- end
60
- end
61
-
62
- def test_pull_games_prior_to_2008
63
- Greenmonster::Spider.pull_day({:print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2007,4,15)})
64
-
65
- assert_equal 12, Dir.entries(@local_test_data_location + '/mlb/year_2007/month_04/day_15/gid_2007_04_15_detmlb_tormlb_1/inning/').count
66
- end
67
-
68
- def test_pull_days
69
- Greenmonster::Spider.pull_days((Date.new(2011,8,4)..Date.new(2011,8,5)), {:print_games => false, :games_folder => @local_test_data_location})
70
-
71
- assert_equal 4, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_08/').count
72
- assert_equal 17, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_08/day_05').count
8
+ def test_set_games_folder_location
9
+ Greenmonster.set_games_folder('./test_greenmonster_folder')
10
+ assert_equal './test_greenmonster_folder', Greenmonster.games_folder
73
11
  end
74
12
 
75
13
  def teardown
76
- FileUtils.remove_dir @local_test_data_location
77
14
  end
78
- end
15
+ end
@@ -0,0 +1,89 @@
1
+ require 'minitest/autorun'
2
+ require 'greenmonster'
3
+
4
+ class GreenmonsterSpiderTest < MiniTest::Unit::TestCase
5
+ def setup
6
+ @local_test_data_location = "./greenmonster_test_games"
7
+ FileUtils.mkdir_p @local_test_data_location
8
+ end
9
+
10
+ def test_format_date_as_folder
11
+ assert_equal "year_2011/month_07/day_04", Greenmonster::Spider.format_date_as_folder(Date.new(2011,7,4))
12
+ assert_equal "year_2012/month_12/day_31", Greenmonster::Spider.format_date_as_folder(Date.new(2012,12,31))
13
+ end
14
+
15
+ def test_copy_gameday_xml
16
+ paths = {
17
+ :localGameFolder => "#{@local_test_data_location}/mlb/year_2011/month_05/day_03/gid_2011_05_03_sfnmlb_nynmlb_1/",
18
+ :mlbGameFolder => "http://gd2.mlb.com/components/game/mlb/year_2011/month_05/day_03/gid_2011_05_03_sfnmlb_nynmlb_1/"
19
+ }
20
+ FileUtils.mkdir_p paths[:localGameFolder] + 'inning'
21
+
22
+ %w(linescore.xml inning_all.xml).each do |f|
23
+ Greenmonster::Spider.copy_gameday_xml(f,paths)
24
+ end
25
+
26
+ assert_equal 287337, Nokogiri::XML(open(paths[:localGameFolder] + 'linescore.xml')).search("game").first.attribute('game_pk').value.to_i
27
+ assert_equal 10, Nokogiri::XML(open(paths[:localGameFolder] + 'inning/inning_all.xml')).search("inning").count
28
+ assert_equal 400023, Nokogiri::XML(open(paths[:localGameFolder] + 'inning/inning_all.xml')).search("atbat").first.attribute('batter').value.to_i
29
+ end
30
+
31
+ def test_pull_day
32
+ Greenmonster::Spider.pull_day({:print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2011,6,7)})
33
+
34
+ assert_equal 17, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_06/day_07/').count
35
+
36
+ game_location = @local_test_data_location + '/mlb/year_2011/month_06/day_07/gid_2011_06_07_bosmlb_nyamlb_1'
37
+ %w(linescore.xml boxscore.xml players.xml eventLog.xml inning).each do |f|
38
+ assert Dir.entries(game_location).include? f
39
+ end
40
+
41
+ boxscore = Nokogiri::XML(open(game_location + '/boxscore.xml'))
42
+ eventlog = Nokogiri::XML(open(game_location + '/eventLog.xml'))
43
+ players = Nokogiri::XML(open(game_location + '/players.xml'))
44
+ innings = Nokogiri::XML(open(game_location + '/inning/inning_all.xml'))
45
+
46
+ assert_equal 8, boxscore.search('pitcher').count
47
+ assert_equal 147, boxscore.search('boxscore').first.attribute('home_id').value.to_i
48
+ assert eventlog.search("event[number='19']").attribute('description').value.include?('Pedroia walks.')
49
+ assert_equal 'Magadan', players.search("team[type='away']").search("coach[position='batting_coach']").attribute('last').value
50
+ assert_equal 'Strikeout', innings.search('atbat').last.attribute('event').value
51
+ end
52
+
53
+ def test_pull_non_mlb_sport_code_games
54
+ Greenmonster::Spider.pull_day({:sport_code => 'aaa', :print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2011,9,1)})
55
+
56
+ assert_equal 13, Dir.entries(@local_test_data_location + '/aaa/year_2011/month_09/day_01/').count
57
+ %w(linescore.xml boxscore.xml players.xml eventLog.xml inning).each do |f|
58
+ assert Dir.entries(@local_test_data_location + '/aaa/year_2011/month_09/day_01/gid_2011_09_01_albaaa_mrbaaa_1/').include? f
59
+ end
60
+ end
61
+
62
+ def test_pull_games_prior_to_2008
63
+ Greenmonster::Spider.pull_day({:print_games => false, :games_folder => @local_test_data_location, :date => Date.new(2007,4,15)})
64
+
65
+ assert_equal 12, Dir.entries(@local_test_data_location + '/mlb/year_2007/month_04/day_15/gid_2007_04_15_detmlb_tormlb_1/inning/').count
66
+ end
67
+
68
+ def test_pull_days
69
+ Greenmonster::Spider.pull_days((Date.new(2011,8,4)..Date.new(2011,8,5)), {:print_games => false, :games_folder => @local_test_data_location})
70
+
71
+ assert_equal 4, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_08/').count
72
+ assert_equal 17, Dir.entries(@local_test_data_location + '/mlb/year_2011/month_08/day_05').count
73
+ end
74
+
75
+ def test_pull_day_with_default_folder_location
76
+ Greenmonster.set_games_folder(@local_test_data_location)
77
+ Greenmonster::Spider.pull_day({:date => Date.new(2011,7,4), :print_games => false})
78
+
79
+ assert_equal 288186, Nokogiri::XML(open(@local_test_data_location + '/mlb/year_2011/month_07/day_04/gid_2011_07_04_tormlb_bosmlb_1/boxscore.xml')).search('boxscore').first.attribute('game_pk').value.to_i
80
+ end
81
+
82
+ def test_pull_single_game_by_game_id
83
+ Greenmonster::Spider.pull_game('gid_2011_07_04_tormlb_bos_mlb_1', {:games_folder => @local_test_data_location, :print_games => false})
84
+ end
85
+
86
+ def teardown
87
+ FileUtils.remove_dir @local_test_data_location
88
+ end
89
+ end
metadata CHANGED
@@ -1,50 +1,29 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: greenmonster
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.pre
5
- prerelease: 6
4
+ version: 0.1.0
5
+ prerelease:
6
6
  platform: ruby
7
7
  authors:
8
8
  - Geoff Harcourt
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2012-03-11 00:00:00.000000000 Z
13
- dependencies:
14
- - !ruby/object:Gem::Dependency
15
- name: nokogiri
16
- requirement: &70119326330320 !ruby/object:Gem::Requirement
17
- none: false
18
- requirements:
19
- - - ! '>='
20
- - !ruby/object:Gem::Version
21
- version: 1.5.2
22
- type: :runtime
23
- prerelease: false
24
- version_requirements: *70119326330320
25
- - !ruby/object:Gem::Dependency
26
- name: httparty
27
- requirement: &70119326329200 !ruby/object:Gem::Requirement
28
- none: false
29
- requirements:
30
- - - ! '>='
31
- - !ruby/object:Gem::Version
32
- version: 0.8.1
33
- type: :runtime
34
- prerelease: false
35
- version_requirements: *70119326329200
36
- description: A utility for spidering and parsing MLB Gameday XML files.
12
+ date: 2012-03-13 00:00:00.000000000 Z
13
+ dependencies: []
14
+ description: A utility for working with MLB Gameday XML data.
37
15
  email: geoff.harcourt@gmail.com
38
16
  executables: []
39
17
  extensions: []
40
18
  extra_rdoc_files: []
41
19
  files:
42
20
  - Rakefile
43
- - README.markdown
44
- - lib/greenmonster/spider.rb
21
+ - Readme.markdown
45
22
  - lib/greenmonster.rb
23
+ - lib/greenmonster/spider.rb
46
24
  - test/test_greenmonster.rb
47
- homepage: http://rubygems.org/gems/greenmonster
25
+ - test/test_greenmonster_spider.rb
26
+ homepage: http://github.com/geoffharcourt/greenmonster
48
27
  licenses: []
49
28
  post_install_message:
50
29
  rdoc_options: []
@@ -59,13 +38,13 @@ required_ruby_version: !ruby/object:Gem::Requirement
59
38
  required_rubygems_version: !ruby/object:Gem::Requirement
60
39
  none: false
61
40
  requirements:
62
- - - ! '>'
41
+ - - ! '>='
63
42
  - !ruby/object:Gem::Version
64
- version: 1.3.1
43
+ version: '0'
65
44
  requirements: []
66
45
  rubyforge_project:
67
46
  rubygems_version: 1.8.11
68
47
  signing_key:
69
48
  specification_version: 3
70
- summary: A utility for spidering and parsing MLB Gameday XML files.
49
+ summary: A utility for working with MLB Gameday XML data.
71
50
  test_files: []
data/README.markdown DELETED
@@ -1,57 +0,0 @@
1
- Greenmonster
2
- ============
3
-
4
- Greenmonster is a toolkit for baseball stat enthusiasts or sabermetricians to build a database of
5
- play-by-play stats from MLB's [Gameday XML data](http://gd.mlb.com/components/game/).
6
-
7
- The pre-release gem is currently spiders games from MLB's servers. I will be extracting pieces out
8
- of my toolkit to parse Gameday XML data to produce play-by-play database of MLB and MiLB stats.
9
-
10
- Usage
11
- -----
12
-
13
- The spider utility has two public class methods: Spider.pull_day and Spider.pull_days.
14
-
15
- Spider.pull_day takes an hash of options as an argument. The only mandatory argument option
16
- is to specify :games_folder, which is the local folder where you want to store the game files.
17
- Greenmonster will create subfolders by MLB "sport_code" (MLB games fall under 'mlb', various
18
- minor league games and non-MLB/MiLB games fall under other sport code designations), and then
19
- children folders for years, months, days, and specific games.
20
-
21
- # Pulls all MLB games for today
22
- => Greenmonster::Spider.pull_day({:date => Date.today, :games_folder => './home/geoff/games'})
23
-
24
- # Pulls all rookie league games for today
25
- > Greenmonster::Spider.pull_day({:sport_code => 'rok', :date => Date.today, :games_folder => './home/geoff/games'})
26
-
27
-
28
- Spider.pull_days takes a range of dates to process as an argument, plus a hash of arguments to pass
29
- to Spider.pull.
30
-
31
- # Pulls all MLB games for in April, 2012
32
- => Greenmonster::Spider.pull_days((Date.new(2012,4,1)..Date.new(2012,4,30)), {:games_folder => './home/geoff/games'})
33
-
34
-
35
- Requirements
36
- ------------
37
- - Ruby 1.9
38
- - Nokogiri
39
- - HTTParty
40
-
41
- Testing
42
- -------
43
-
44
- The test suite downloads a few days of data, so it is not fast to execute.
45
-
46
-
47
- License
48
- -------
49
- (The MIT License)
50
-
51
- Copyright &copy; [Geoff Harcourt](http://github.com/geoffharcourt) 2012
52
-
53
- Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
54
-
55
- The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
56
-
57
- THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.