sportdb-readers 1.0.0 → 1.1.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (16) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +86 -20
  3. data/Rakefile +1 -1
  4. data/lib/sportdb/readers.rb +6 -6
  5. data/lib/sportdb/readers/conf_reader.rb +2 -2
  6. data/lib/sportdb/readers/match_reader.rb +18 -10
  7. data/lib/sportdb/readers/package.rb +2 -1
  8. data/lib/sportdb/readers/version.rb +2 -2
  9. data/test/test_conf_reader.rb +2 -2
  10. data/test/test_match_reader_champs.rb +2 -2
  11. data/test/test_match_reader_eng.rb +2 -2
  12. data/test/test_match_reader_euro.rb +1 -1
  13. data/test/test_match_reader_mu.rb +1 -1
  14. data/test/test_reader.rb +2 -2
  15. data/test/test_reader_champs.rb +2 -2
  16. metadata +4 -4
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 976e1c02d5e34f69d999581d0036b1dce641d27b
4
- data.tar.gz: f84742df381b09fa244d963d6624d5a0137c3159
3
+ metadata.gz: d51d795ec384d281363b3af32bb6560bb0e7707a
4
+ data.tar.gz: 6579ab0a20f45673f24ae4b444bc01d5c0f43d19
5
5
  SHA512:
6
- metadata.gz: 040b202ef7acac2ce56a67f33b88c9c93fe961173cfb5ff28983dee540b643963ed5581273be2d6ee5c6f8d257f1559b62f7ea5c340fc8b02c7c279768add7f7
7
- data.tar.gz: d0f3cfa4c66a261f5d4a30b24e7e5f86aef8ebc78730dd4782ffc33d00d44205e5d23a44275640664851fb0e7e10f83b5cc1a789293031c6b638dcf53de36b18
6
+ metadata.gz: 5b5ca27813390ab7fe3aa41e63897bbd0aa03c39d78dabed8a321eb3853528a2c1700c8dea4be57fe7fd3eb6122631878bdfb2beeed03ac81e545b4c9049dd4f
7
+ data.tar.gz: 5c2a40f3fd42a0ec3eac82e942088a00bd2e7a6bf4bd07a2d993d217ac2b17a3503ac41d7befd1016a90511ded5c8ea2d4df7a577bbd1ec3dd994f84c4ea1113
data/README.md CHANGED
@@ -31,50 +31,116 @@ Let's read in some leagues, seasons, clubs, and match schedules and results.
31
31
  Let's use the public domain football.db datasets for England (see [`openfootball/england`](https://github.com/openfootball/england)), as an example:
32
32
 
33
33
 
34
- ``` ruby
35
- ## turn on logging to console
36
- ActiveRecord::Base.logger = Logger.new( STDOUT )
34
+ ```
35
+ = English Premier League 2015/16
36
+
37
+ Matchday 1
38
+
39
+ [Sat Aug 8]
40
+ Manchester United 1-0 Tottenham Hotspur
41
+ AFC Bournemouth 0-1 Aston Villa
42
+ Everton FC 2-2 Watford FC
43
+ Leicester City 4-2 Sunderland AFC
44
+ Norwich City 1-3 Crystal Palace
45
+ Chelsea FC 2-2 Swansea City
46
+ [Sun Aug 9]
47
+ Arsenal FC 0-2 West Ham United
48
+ Newcastle United 2-2 Southampton FC
49
+ Stoke City 0-1 Liverpool FC
50
+ [Mon Aug 10]
51
+ West Bromwich Albion 0-3 Manchester City
52
+
53
+ ...
54
+ ```
55
+
56
+ (Source: [england/2015-16/1-premierleague-i.txt](https://github.com/openfootball/england/blob/master/2015-16/1-premierleague-i.txt))
37
57
 
58
+ and let's try:
59
+
60
+ ``` ruby
38
61
  ## assumes football.db datasets for England in ./england directory
39
62
  ## see github.com/openfootball/england
40
- SportDb::ConfReaderV2.read( './england/2015-16/.conf.txt' )
41
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-i.txt' )
42
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-ii.txt' )
63
+ SportDb.read( './england/2015-16/1-premierleague-i.txt' )
64
+ SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
43
65
 
44
66
  ## let's try another season
45
- SportDb::ConfReaderV2.read( './england/2019-20/.conf.txt' )
46
- SportDb::MatchReaderV2.read( './england/2019-20/1-premierleague.txt' )
67
+ SportDb.read( './england/2019-20/1-premierleague.txt' )
47
68
  ```
48
69
 
49
70
  All leagues, seasons, clubs, match days and rounds, match fixtures and results,
50
71
  and more are now in your (SQL) database of choice.
51
72
 
52
-
53
- Or as an alternative use the `read` convenience all-in-one shortcut helper:
73
+ The proof of the pudding - Let's query the (SQL) database using the sport.db ActiveRecord models:
54
74
 
55
75
  ``` ruby
56
- ## assumes football.db datasets for England in ./england directory
57
- ## see github.com/openfootball/england
58
- SportDb.read( './england/2015-16/.conf.txt' )
59
- SportDb.read( './england/2015-16/1-premierleague-i.txt' )
60
- SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
76
+ include SportDb::Models
61
77
 
62
- ## let's try another season
63
- SportDb.read( './england/2019-20/.conf.txt' )
64
- SportDb.read( './england/2019-20/1-premierleague.txt' )
78
+ pl_2015_16 = Event.find_by( key: 'eng.1.2015/16' )
79
+ #=> SELECT * FROM events WHERE key = 'eng.1.2015/16' LIMIT 1
80
+
81
+ pl_2015_16.teams.count #=> 20
82
+ #=> SELECT COUNT(*) FROM teams
83
+ # INNER JOIN events_teams ON teams.id = events_teams.team_id
84
+ # WHERE events_teams.event_id = 1
85
+
86
+ pl_2015_16.games.count #=> 380
87
+ #=> SELECT COUNT(*) FROM games
88
+ # INNER JOIN rounds ON games.round_id = rounds.id
89
+ # WHERE rounds.event_id = 1
90
+
91
+ pl_2019_20 = Event.find_by( key: 'eng.1.2019/20' )
92
+ pl_2015_16.teams.count #=> 20
93
+ pl_2015_16.games.count #=> 380
94
+
95
+ # -or-
96
+
97
+ pl = League.find_by( key: 'eng.1' )
98
+ #=> SELECT * FROM leagues WHERE key = 'eng.1' LIMIT 1
99
+
100
+ pl.seasons.count #=> 2
101
+ #=> SELECT COUNT(*) FROM seasons
102
+ # INNER JOIN events ON seasons.id = events.season_id
103
+ # WHERE events.league_id = 1
104
+
105
+ # and so on and so forth.
65
106
  ```
66
107
 
67
- Or as an alternative pass in the "package" directory and let `read` figure
68
- out what datafiles to read:
108
+ Bonus: As an alternative pass in the "package" directory or a zip archive and let `read` figure
109
+ out what datafiles to read in:
69
110
 
70
111
  ``` ruby
71
112
  ## assumes football.db datasets for England in ./england directory
72
113
  ## see github.com/openfootball/england
73
114
  SportDb.read( './england' )
115
+ ## -or- use a zip archive download
116
+ SportDb.read( './england.zip' )
74
117
  ```
75
118
 
76
119
  That's it.
77
120
 
121
+ ## Frequently Asked Questions (F.A.Q.s) and Answers
122
+
123
+ Q: What about reading in datasets in comma-separated values (CSV) format?
124
+ Example:
125
+
126
+ ```
127
+ Round, Date, Team 1, FT, HT, Team 2
128
+ 1, (Fri) 9 Aug 2019, Liverpool FC, 4-1, 4-0, Norwich City FC
129
+ 1, (Sat) 10 Aug 2019, West Ham United FC, 0-5, 0-1, Manchester City FC
130
+ 1, (Sat) 10 Aug 2019, AFC Bournemouth, 1-1, 0-0, Sheffield United FC
131
+ 1, (Sat) 10 Aug 2019, Burnley FC, 3-0, 0-0, Southampton FC
132
+ 1, (Sat) 10 Aug 2019, Crystal Palace FC, 0-0, 0-0, Everton FC
133
+ 1, (Sat) 10 Aug 2019, Watford FC, 0-3, 0-1, Brighton & Hove Albion FC
134
+ 1, (Sat) 10 Aug 2019, Tottenham Hotspur FC, 3-1, 0-1, Aston Villa FC
135
+ 1, (Sun) 11 Aug 2019, Leicester City FC, 0-0, 0-0, Wolverhampton Wanderers FC
136
+ 1, (Sun) 11 Aug 2019, Newcastle United FC, 0-1, 0-0, Arsenal FC
137
+ 1, (Sun) 11 Aug 2019, Manchester United FC, 4-0, 1-0, Chelsea FC
138
+ ...
139
+ ```
140
+ (Source: [england/2019-20/eng.1.csv](https://github.com/footballcsv/england/blob/master/2010s/2019-20/eng.1.csv))
141
+
142
+ Yes, you can. See the [sportdb-importers library / gem »](https://github.com/sportdb/sport.db/tree/master/sportdb-importers)
143
+
78
144
 
79
145
 
80
146
  ## License
data/Rakefile CHANGED
@@ -20,7 +20,7 @@ Hoe.spec 'sportdb-readers' do
20
20
  self.licenses = ['Public Domain']
21
21
 
22
22
  self.extra_deps = [
23
- ['sportdb-sync', '>= 1.0.0'],
23
+ ['sportdb-sync', '>= 1.1.0'],
24
24
  ]
25
25
 
26
26
  self.spec_extras = {
data/lib/sportdb/readers.rb CHANGED
@@ -16,18 +16,18 @@ require 'sportdb/readers/package'
16
16
  ##
17
17
  ## add convenience shortcut helpers
18
18
  module SportDb
19
- def self.read_conf( path, season: nil ) ConfReaderV2.read( path, season: season ); end
20
- def self.parse_conf( txt, season: nil ) ConfReaderV2.parse( txt, season: season ); end
19
+ def self.read_conf( path, season: nil ) ConfReader.read( path, season: season ); end
20
+ def self.parse_conf( txt, season: nil ) ConfReader.parse( txt, season: season ); end
21
21
 
22
22
  ### todo/check: add alias read_matches - why? why not?
23
- def self.read_match( path, season: nil ) MatchReaderV2.read( path, season: season ); end
24
- def self.parse_match( txt, season: nil ) MatchReaderV2.parse( txt, season: season ); end
23
+ def self.read_match( path, season: nil ) MatchReader.read( path, season: season ); end
24
+ def self.parse_match( txt, season: nil ) MatchReader.parse( txt, season: season ); end
25
25
 
26
26
  def self.read_club_props( path ) Import::ClubPropsReader.read( path ); end
27
27
  def self.parse_club_props( txt ) Import::ClubPropsReader.parse( txt ); end
28
28
 
29
- def self.parse_leagues( txt ) recs = Import::LeagueReader.parse( txt ); Import::config.leagues.add( recs ); end
30
- def self.parse_clubs( txt ) recs = Import::ClubReader.parse( txt ); Import::config.clubs.add( recs ); end
29
+ def self.parse_leagues( txt ) recs = Import::LeagueReader.parse( txt ); Import.catalog.leagues.add( recs ); end
30
+ def self.parse_clubs( txt ) recs = Import::ClubReader.parse( txt ); Import.catalog.clubs.add( recs ); end
31
31
 
32
32
 
33
33
  def self.read( path, season: nil )
data/lib/sportdb/readers/conf_reader.rb CHANGED
@@ -3,7 +3,7 @@
3
3
  module SportDb
4
4
 
5
5
 
6
- class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
6
+ class ConfReader ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
7
7
 
8
8
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
9
9
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -96,5 +96,5 @@ class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why?
96
96
 
97
97
  def catalog() Import.catalog; end ## shortcut convenience helper
98
98
 
99
- end # class ConfReaderV2
99
+ end # class ConfReader
100
100
  end # module SportDb
data/lib/sportdb/readers/match_reader.rb CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  module SportDb
4
4
 
5
- class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
5
+ class MatchReader ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
6
6
 
7
7
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
8
8
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -31,12 +31,13 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
31
31
  langs = { ## map country keys to lang codes
32
32
  'de' => 'de', ## de - Deutsch (German)
33
33
  'at' => 'de',
34
+ 'ch' => 'de',
34
35
  'fr' => 'fr', ## fr - French
35
36
  'it' => 'it', ## it - Italian
36
37
  'es' => 'es', ## es - Español (Spanish)
37
38
  'mx' => 'es',
38
39
  'pt' => 'pt', ## pt - Português (Portuguese)
39
- 'br' => 'br'
40
+ 'br' => 'br' ## !!! todo/check - br exists?? use pt portuges??
40
41
  }
41
42
 
42
43
  secs.each do |sec| ## sec(tion)s
@@ -53,11 +54,19 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
53
54
  Import.config.lang = langs[ league.country.key ] || 'en'
54
55
  end
55
56
 
57
+ ### check if event info availabe - use start_date;
58
+ ## otherwise we have to guess (use a "synthetic" start_date)
59
+ event_info = catalog.events.find_by( season: season,
60
+ league: league )
56
61
 
57
- start = if season.year?
58
- Date.new( season.start_year, 1, 1 )
62
+ start = if event_info && event_info.start_date
63
+ event_info.start_date
59
64
  else
60
- Date.new( season.start_year, 7, 1 )
65
+ if season.year?
66
+ Date.new( season.start_year, 1, 1 )
67
+ else
68
+ Date.new( season.start_year, 7, 1 )
69
+ end
61
70
  end
62
71
 
63
72
  auto_conf_teams, _ = AutoConfParser.parse( lines,
@@ -87,7 +96,7 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
87
96
  team_mapping = auto_conf_teams.keys.zip( teams ).to_h
88
97
 
89
98
 
90
- parser = MatchParserSimpleV2.new( lines,
99
+ parser = MatchParser.new( lines,
91
100
  team_mapping,
92
101
  start ) ## note: keep season start_at date for now (no need for more specific stage date need for now)
93
102
 
@@ -125,8 +134,6 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
125
134
 
126
135
 
127
136
  rounds.each do |round|
128
- ## quick hack: if pos missing fill with dummy 999 for now
129
- round.pos = 999 if round.pos.nil?
130
137
  round_rec = Sync::Round.find_or_create( round, event: event_rec ) ## check: use/rename to EventRound why? why not?
131
138
  end
132
139
 
@@ -135,7 +142,8 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
135
142
  end
136
143
 
137
144
  matches.each do |match|
138
- ## todo/fix: pass along stage (if present): stage - optional!!!!
145
+ ## note: pass along stage (if present): stage - optional from heading!!!!
146
+ match = match.update( stage: stage ) if stage
139
147
  match_rec = Sync::Match.create_or_update( match, event: event_rec )
140
148
  end
141
149
  end
@@ -149,5 +157,5 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
149
157
 
150
158
  def catalog() Import.catalog; end
151
159
 
152
- end # class MatchReaderV2
160
+ end # class MatchReader
153
161
  end # module SportDb
data/lib/sportdb/readers/package.rb CHANGED
@@ -49,7 +49,8 @@ module SportDb
49
49
  read_leagues()
50
50
  read_clubs()
51
51
  read_club_props()
52
- read_conf( season: season )
52
+ ## note: skip conf(iguration)s for now!!!!!!!
53
+ ## read_conf( season: season )
53
54
  read_match( season: season )
54
55
  else
55
56
  names.each do |name|
data/lib/sportdb/readers/version.rb CHANGED
@@ -5,8 +5,8 @@ module SportDb
5
5
  module Readers
6
6
 
7
7
  MAJOR = 1 ## todo: namespace inside version or something - why? why not??
8
- MINOR = 0
9
- PATCH = 0
8
+ MINOR = 1
9
+ PATCH = 1
10
10
  VERSION = [MAJOR,MINOR,PATCH].join('.')
11
11
 
12
12
  def self.version
data/test/test_conf_reader.rb CHANGED
@@ -25,7 +25,7 @@ class TestConfReader < MiniTest::Test
25
25
  # path = "../../../openfootball/england/2017-18/.conf.txt"
26
26
  # path = "../../../openfootball/england/2018-19/.conf.txt"
27
27
  # path = "../../../openfootball/england/2019-20/.conf.txt"
28
- SportDb::ConfReaderV2.read( path )
28
+ SportDb::ConfReader.read( path )
29
29
  end # method test_read
30
30
 
31
31
 
@@ -73,6 +73,6 @@ Monaco › MCO
73
73
  APOEL › CYP
74
74
  TXT
75
75
 
76
- SportDb::ConfReaderV2.parse( txt )
76
+ SportDb::ConfReader.parse( txt )
77
77
  end
78
78
  end # class TestConfReader
data/test/test_match_reader_champs.rb CHANGED
@@ -363,7 +363,7 @@ Group H:
363
363
  [Llorente 20' Son Heung-min 37' N'Koudou 80']
364
364
  TXT
365
365
 
366
- SportDb::MatchReaderV2.parse( txt )
366
+ SportDb::MatchReader.parse( txt )
367
367
  end
368
368
 
369
369
  def test_read_finals
@@ -480,7 +480,7 @@ Final
480
480
  [Benzema 51' Bale 64', 83'; Mané 55']
481
481
  TXT
482
482
 
483
- SportDb::MatchReaderV2.parse( txt )
483
+ SportDb::MatchReader.parse( txt )
484
484
  end
485
485
 
486
486
 
data/test/test_match_reader_eng.rb CHANGED
@@ -58,7 +58,7 @@ Matchday 2
58
58
  Manchester City 1-1 Everton FC
59
59
  TXT
60
60
 
61
- SportDb::MatchReaderV2.parse( txt )
61
+ SportDb::MatchReader.parse( txt )
62
62
  end # method test_read_eng
63
63
 
64
64
  def test_read_eng_2012_13
@@ -87,7 +87,7 @@ Matchday 20
87
87
  QPR 0-3 Liverpool
88
88
  TXT
89
89
 
90
- SportDb::MatchReaderV2.parse( txt )
90
+ SportDb::MatchReader.parse( txt )
91
91
 
92
92
  puts SportDb::Model::Game.count
93
93
  end
data/test/test_match_reader_euro.rb CHANGED
@@ -149,7 +149,7 @@ Final
149
149
  (51) Jul/10 21:00 Portugal 0:0 1:0nV France @ Stade de France, Saint-Denis
150
150
  TXT
151
151
 
152
- SportDb::MatchReaderV2.parse( txt )
152
+ SportDb::MatchReader.parse( txt )
153
153
  end
154
154
  end # class TestMatchReaderEuro
155
155
 
data/test/test_match_reader_mu.rb CHANGED
@@ -91,7 +91,7 @@ Matchday 3
91
91
  AS Port-Louis 2000 1-0 La Cure Sylvester
92
92
  TXT
93
93
 
94
- SportDb::MatchReaderV2.parse( txt )
94
+ SportDb::MatchReader.parse( txt )
95
95
  end # method test_read_mauritius
96
96
 
97
97
  end # class TestMatchReaderMu
data/test/test_reader.rb CHANGED
@@ -24,13 +24,13 @@ class TestReader < MiniTest::Test
24
24
  # path = "../../../openfootball/england/2017-18/.conf.txt"
25
25
  # path = "../../../openfootball/england/2018-19/.conf.txt"
26
26
  # path = "../../../openfootball/england/2019-20/.conf.txt"
27
- recs = SportDb::ConfReaderV2.read( path )
27
+ recs = SportDb::ConfReader.read( path )
28
28
  # path = "../../../openfootball/austria/2018-19/1-bundesliga.txt"
29
29
  path = "../../../openfootball/england/2015-16/1-premierleague-i.txt"
30
30
  # path = "../../../openfootball/england/2017-18/1-premierleague-i.txt"
31
31
  # path = "../../../openfootball/england/2018-19/1-premierleague.txt"
32
32
  # path = "../../../openfootball/england/2019-20/1-premierleague.txt"
33
- recs = SportDb::MatchReaderV2.read( path )
33
+ recs = SportDb::MatchReader.read( path )
34
34
  # path = "../../../openfootball/england/2017-18/1-premierleague-ii.txt"
35
35
  #recs = SportDb::MatchReaderV2.read( path )
36
36
  end # method test_read
data/test/test_reader_champs.rb CHANGED
@@ -64,7 +64,7 @@ Monaco › MCO
64
64
  APOEL › CYP
65
65
  TXT
66
66
 
67
- SportDb::ConfReaderV2.parse( txt )
67
+ SportDb::ConfReader.parse( txt )
68
68
 
69
69
 
70
70
  txt =<<TXT
@@ -180,7 +180,7 @@ Final
180
180
  [Benzema 51' Bale 64', 83'; Mané 55']
181
181
  TXT
182
182
 
183
- SportDb::MatchReaderV2.parse( txt )
183
+ SportDb::MatchReader.parse( txt )
184
184
  end
185
185
 
186
186
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sportdb-readers
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0
4
+ version: 1.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Gerald Bauer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-05-11 00:00:00.000000000 Z
11
+ date: 2020-07-07 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sportdb-sync
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 1.0.0
19
+ version: 1.1.0
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 1.0.0
26
+ version: 1.1.0
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: rdoc
29
29
  requirement: !ruby/object:Gem::Requirement