sportdb-readers 1.0.1 → 1.1.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 59763aec58a5729bbe978c4f81ac60de0b883e3f
4
- data.tar.gz: 8fe8c7a0362bb664620f0392865a2435d8d54ef7
3
+ metadata.gz: b92858d91cd8a8229f492aba44a58e3dd6be1b40
4
+ data.tar.gz: 36d490bff1de4f88216e4092abc73927a1fb0de6
5
5
  SHA512:
6
- metadata.gz: f700cc86b085a7b882211c0d2bc7033287e4b523403374eba95ff58ef2fe9b930fd09d6e4c57483e5b5a978aa487825b7fcdcd34ea873299a94604d443403f83
7
- data.tar.gz: '08a052e7b179189c2eae987c24c6e90915a74f3666a87dd116bef60a4b7266c693d7419415df0311e91f35c287c2c19b0b5b8a7faa86c2f0345f88bd5e672090'
6
+ metadata.gz: 8b509fe042ac69758777e838f245d929807c23ccc3276d1216d0dc127fd54f1758a3474a56469b5c42fb073862431a96bcd85fa78b15979e61272251cfb61237
7
+ data.tar.gz: cb9b1868690d2cd36c9ae2cc50a2cef6962192f072bf5cd70710f9942439d62dbfbb8f6fa67908eddb69853f6a1284840a6502f5daf237864059a716aaf2511e
data/README.md CHANGED
@@ -31,50 +31,116 @@ Let's read in some leagues, seasons, clubs, and match schedules and results.
31
31
  Let's use the public domain football.db datasets for England (see [`openfootball/england`](https://github.com/openfootball/england)), as an example:
32
32
 
33
33
 
34
- ``` ruby
35
- ## turn on logging to console
36
- ActiveRecord::Base.logger = Logger.new( STDOUT )
34
+ ```
35
+ = English Premier League 2015/16
36
+
37
+ Matchday 1
38
+
39
+ [Sat Aug 8]
40
+ Manchester United 1-0 Tottenham Hotspur
41
+ AFC Bournemouth 0-1 Aston Villa
42
+ Everton FC 2-2 Watford FC
43
+ Leicester City 4-2 Sunderland AFC
44
+ Norwich City 1-3 Crystal Palace
45
+ Chelsea FC 2-2 Swansea City
46
+ [Sun Aug 9]
47
+ Arsenal FC 0-2 West Ham United
48
+ Newcastle United 2-2 Southampton FC
49
+ Stoke City 0-1 Liverpool FC
50
+ [Mon Aug 10]
51
+ West Bromwich Albion 0-3 Manchester City
52
+
53
+ ...
54
+ ```
55
+
56
+ (Source: [england/2015-16/1-premierleague-i.txt](https://github.com/openfootball/england/blob/master/2015-16/1-premierleague-i.txt))
37
57
 
58
+ and let's try:
59
+
60
+ ``` ruby
38
61
  ## assumes football.db datasets for England in ./england directory
39
62
  ## see github.com/openfootball/england
40
- SportDb::ConfReaderV2.read( './england/2015-16/.conf.txt' )
41
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-i.txt' )
42
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-ii.txt' )
63
+ SportDb.read( './england/2015-16/1-premierleague-i.txt' )
64
+ SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
43
65
 
44
66
  ## let's try another season
45
- SportDb::ConfReaderV2.read( './england/2019-20/.conf.txt' )
46
- SportDb::MatchReaderV2.read( './england/2019-20/1-premierleague.txt' )
67
+ SportDb.read( './england/2019-20/1-premierleague.txt' )
47
68
  ```
48
69
 
49
70
  All leagues, seasons, clubs, match days and rounds, match fixtures and results,
50
71
  and more are now in your (SQL) database of choice.
51
72
 
52
-
53
- Or as an alternative use the `read` convenience all-in-one shortcut helper:
73
+ The proof of the pudding - Let's query the (SQL) database using the sport.db ActiveRecord models:
54
74
 
55
75
  ``` ruby
56
- ## assumes football.db datasets for England in ./england directory
57
- ## see github.com/openfootball/england
58
- SportDb.read( './england/2015-16/.conf.txt' )
59
- SportDb.read( './england/2015-16/1-premierleague-i.txt' )
60
- SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
76
+ include SportDb::Models
61
77
 
62
- ## let's try another season
63
- SportDb.read( './england/2019-20/.conf.txt' )
64
- SportDb.read( './england/2019-20/1-premierleague.txt' )
78
+ pl_2015_16 = Event.find_by( key: 'eng.1.2015/16' )
79
+ #=> SELECT * FROM events WHERE key = 'eng.1.2015/16' LIMIT 1
80
+
81
+ pl_2015_16.teams.count #=> 20
82
+ #=> SELECT COUNT(*) FROM teams
83
+ # INNER JOIN events_teams ON teams.id = events_teams.team_id
84
+ # WHERE events_teams.event_id = 1
85
+
86
+ pl_2015_16.games.count #=> 380
87
+ #=> SELECT COUNT(*) FROM games
88
+ # INNER JOIN rounds ON games.round_id = rounds.id
89
+ # WHERE rounds.event_id = 1
90
+
91
+ pl_2019_20 = Event.find_by( key: 'eng.1.2019/20' )
92
+ pl_2015_16.teams.count #=> 20
93
+ pl_2015_16.games.count #=> 380
94
+
95
+ # -or-
96
+
97
+ pl = League.find_by( key: 'eng.1' )
98
+ #=> SELECT * FROM leagues WHERE key = 'eng.1' LIMIT 1
99
+
100
+ pl.seasons.count #=> 2
101
+ #=> SELECT COUNT(*) FROM seasons
102
+ # INNER JOIN events ON seasons.id = events.season_id
103
+ # WHERE events.league_id = 1
104
+
105
+ # and so on and so forth.
65
106
  ```
66
107
 
67
- Or as an alternative pass in the "package" directory and let `read` figure
68
- out what datafiles to read:
108
+ Bonus: As an alternative pass in the "package" directory or a zip archive and let `read` figure
109
+ out what datafiles to read in:
69
110
 
70
111
  ``` ruby
71
112
  ## assumes football.db datasets for England in ./england directory
72
113
  ## see github.com/openfootball/england
73
114
  SportDb.read( './england' )
115
+ ## -or- use a zip archive download
116
+ SportDb.read( './england.zip' )
74
117
  ```
75
118
 
76
119
  That's it.
77
120
 
121
+ ## Frequently Asked Questions (F.A.Q.s) and Answers
122
+
123
+ Q: What about reading in datasets in comma-separated values (CSV) format?
124
+ Example:
125
+
126
+ ```
127
+ Round, Date, Team 1, FT, HT, Team 2
128
+ 1, (Fri) 9 Aug 2019, Liverpool FC, 4-1, 4-0, Norwich City FC
129
+ 1, (Sat) 10 Aug 2019, West Ham United FC, 0-5, 0-1, Manchester City FC
130
+ 1, (Sat) 10 Aug 2019, AFC Bournemouth, 1-1, 0-0, Sheffield United FC
131
+ 1, (Sat) 10 Aug 2019, Burnley FC, 3-0, 0-0, Southampton FC
132
+ 1, (Sat) 10 Aug 2019, Crystal Palace FC, 0-0, 0-0, Everton FC
133
+ 1, (Sat) 10 Aug 2019, Watford FC, 0-3, 0-1, Brighton & Hove Albion FC
134
+ 1, (Sat) 10 Aug 2019, Tottenham Hotspur FC, 3-1, 0-1, Aston Villa FC
135
+ 1, (Sun) 11 Aug 2019, Leicester City FC, 0-0, 0-0, Wolverhampton Wanderers FC
136
+ 1, (Sun) 11 Aug 2019, Newcastle United FC, 0-1, 0-0, Arsenal FC
137
+ 1, (Sun) 11 Aug 2019, Manchester United FC, 4-0, 1-0, Chelsea FC
138
+ ...
139
+ ```
140
+ (Source: [england/2019-20/eng.1.csv](https://github.com/footballcsv/england/blob/master/2010s/2019-20/eng.1.csv))
141
+
142
+ Yes, you can. See the [sportdb-importers library / gem »](https://github.com/sportdb/sport.db/tree/master/sportdb-importers)
143
+
78
144
 
79
145
 
80
146
  ## License
data/Rakefile CHANGED
@@ -20,7 +20,7 @@ Hoe.spec 'sportdb-readers' do
20
20
  self.licenses = ['Public Domain']
21
21
 
22
22
  self.extra_deps = [
23
- ['sportdb-sync', '>= 1.0.0'],
23
+ ['sportdb-sync', '>= 1.1.1'],
24
24
  ]
25
25
 
26
26
  self.spec_extras = {
@@ -16,12 +16,12 @@ require 'sportdb/readers/package'
16
16
  ##
17
17
  ## add convenience shortcut helpers
18
18
  module SportDb
19
- def self.read_conf( path, season: nil ) ConfReaderV2.read( path, season: season ); end
20
- def self.parse_conf( txt, season: nil ) ConfReaderV2.parse( txt, season: season ); end
19
+ def self.read_conf( path, season: nil ) ConfReader.read( path, season: season ); end
20
+ def self.parse_conf( txt, season: nil ) ConfReader.parse( txt, season: season ); end
21
21
 
22
22
  ### todo/check: add alias read_matches - why? why not?
23
- def self.read_match( path, season: nil ) MatchReaderV2.read( path, season: season ); end
24
- def self.parse_match( txt, season: nil ) MatchReaderV2.parse( txt, season: season ); end
23
+ def self.read_match( path, season: nil ) MatchReader.read( path, season: season ); end
24
+ def self.parse_match( txt, season: nil ) MatchReader.parse( txt, season: season ); end
25
25
 
26
26
  def self.read_club_props( path ) Import::ClubPropsReader.read( path ); end
27
27
  def self.parse_club_props( txt ) Import::ClubPropsReader.parse( txt ); end
@@ -3,7 +3,7 @@
3
3
  module SportDb
4
4
 
5
5
 
6
- class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
6
+ class ConfReader ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
7
7
 
8
8
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
9
9
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -96,5 +96,5 @@ class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why?
96
96
 
97
97
  def catalog() Import.catalog; end ## shortcut convenience helper
98
98
 
99
- end # class ConfReaderV2
99
+ end # class ConfReader
100
100
  end # module SportDb
@@ -2,7 +2,7 @@
2
2
 
3
3
  module SportDb
4
4
 
5
- class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
5
+ class MatchReader ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
6
6
 
7
7
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
8
8
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -31,12 +31,14 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
31
31
  langs = { ## map country keys to lang codes
32
32
  'de' => 'de', ## de - Deutsch (German)
33
33
  'at' => 'de',
34
+ 'ch' => 'de',
34
35
  'fr' => 'fr', ## fr - French
35
36
  'it' => 'it', ## it - Italian
36
37
  'es' => 'es', ## es - Español (Spanish)
37
38
  'mx' => 'es',
39
+ 'ar' => 'es', ## Argentina
38
40
  'pt' => 'pt', ## pt - Português (Portuguese)
39
- 'br' => 'br'
41
+ 'br' => 'pt',
40
42
  }
41
43
 
42
44
  secs.each do |sec| ## sec(tion)s
@@ -53,13 +55,23 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
53
55
  Import.config.lang = langs[ league.country.key ] || 'en'
54
56
  end
55
57
 
58
+ ### check if event info availabe - use start_date;
59
+ ## otherwise we have to guess (use a "synthetic" start_date)
60
+ event_info = catalog.events.find_by( season: season,
61
+ league: league )
56
62
 
57
- start = if season.year?
58
- Date.new( season.start_year, 1, 1 )
63
+ start = if event_info && event_info.start_date
64
+ event_info.start_date
59
65
  else
60
- Date.new( season.start_year, 7, 1 )
66
+ if season.year?
67
+ Date.new( season.start_year, 1, 1 )
68
+ else
69
+ Date.new( season.start_year, 7, 1 )
70
+ end
61
71
  end
62
72
 
73
+
74
+ ### todo/check: make sure team include teams from group def too!!!!!
63
75
  auto_conf_teams, _ = AutoConfParser.parse( lines,
64
76
  start: start )
65
77
 
@@ -72,7 +84,10 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
72
84
  ## todo/fix: make more generic / reuseable!!!!
73
85
  mods = {}
74
86
  ## europa league uses same mods as champions league
75
- mods[ 'uefa.el' ] = mods[ 'uefa.cl' ] = catalog.clubs.build_mods(
87
+ mods[ 'uefa.el.quali' ] =
88
+ mods[ 'uefa.cl.quali' ] =
89
+ mods[ 'uefa.el' ] =
90
+ mods[ 'uefa.cl' ] = catalog.clubs.build_mods(
76
91
  { 'Liverpool | Liverpool FC' => 'Liverpool FC, ENG',
77
92
  'Arsenal | Arsenal FC' => 'Arsenal FC, ENG',
78
93
  'Barcelona' => 'FC Barcelona, ESP',
@@ -87,7 +102,7 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
87
102
  team_mapping = auto_conf_teams.keys.zip( teams ).to_h
88
103
 
89
104
 
90
- parser = MatchParserSimpleV2.new( lines,
105
+ parser = MatchParser.new( lines,
91
106
  team_mapping,
92
107
  start ) ## note: keep season start_at date for now (no need for more specific stage date need for now)
93
108
 
@@ -124,18 +139,37 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
124
139
  end
125
140
 
126
141
 
127
- rounds.each do |round|
128
- ## quick hack: if pos missing fill with dummy 999 for now
129
- round.pos = 999 if round.pos.nil?
130
- round_rec = Sync::Round.find_or_create( round, event: event_rec ) ## check: use/rename to EventRound why? why not?
131
- end
142
+ ## build a lookup cache for team_recs (used by group lookup)
143
+ ## lookup by "canonical" name
144
+ cache_team_recs = new_team_recs.reduce({}) { |h,rec| h[rec.name] = rec; h }
132
145
 
133
146
  groups.each do |group|
134
147
  group_rec = Sync::Group.find_or_create( group, event: event_rec ) ## check: use/rename to EventGroup why? why not?
148
+
149
+ ########
150
+ ## add/update teams - todo/fix/clean-up - find a better way or move to sync? - possible?
151
+ ## e.g. group.teams assumes an array of team names e.g.
152
+ ## ["Spain", "Czech Republic", "Turkey", "Croatia"]
153
+ group_team_ids = []
154
+ group.teams.each do |team_name|
155
+ team_rec = cache_team_recs[ team_name ]
156
+ if team_rec.nil? ## assume team MUST always be present/known in mapping (via autoconfig parser)
157
+ puts "!! ERROR - no (cached) team rec found for team in group >#{group.name}< for >#{team_name}<"
158
+ exit 1
159
+ end
160
+ group_team_ids << team_rec.id
161
+ end
162
+ group_rec.team_ids = group_team_ids ## reset/update all teams at once (via ids)
163
+ end
164
+
165
+
166
+ rounds.each do |round|
167
+ round_rec = Sync::Round.find_or_create( round, event: event_rec ) ## check: use/rename to EventRound why? why not?
135
168
  end
136
169
 
137
170
  matches.each do |match|
138
- ## todo/fix: pass along stage (if present): stage - optional!!!!
171
+ ## note: pass along stage (if present): stage - optional from heading!!!!
172
+ match = match.update( stage: stage ) if stage
139
173
  match_rec = Sync::Match.create_or_update( match, event: event_rec )
140
174
  end
141
175
  end
@@ -149,5 +183,5 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
149
183
 
150
184
  def catalog() Import.catalog; end
151
185
 
152
- end # class MatchReaderV2
186
+ end # class MatchReader
153
187
  end # module SportDb
@@ -49,7 +49,8 @@ module SportDb
49
49
  read_leagues()
50
50
  read_clubs()
51
51
  read_club_props()
52
- read_conf( season: season )
52
+ ## note: skip conf(iguration)s for now!!!!!!!
53
+ ## read_conf( season: season )
53
54
  read_match( season: season )
54
55
  else
55
56
  names.each do |name|
@@ -5,8 +5,8 @@ module SportDb
5
5
  module Readers
6
6
 
7
7
  MAJOR = 1 ## todo: namespace inside version or something - why? why not??
8
- MINOR = 0
9
- PATCH = 1
8
+ MINOR = 1
9
+ PATCH = 2
10
10
  VERSION = [MAJOR,MINOR,PATCH].join('.')
11
11
 
12
12
  def self.version
@@ -1,6 +1,6 @@
1
1
  ## note: use the local version of sportdb gems
2
2
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-formats/lib' ))
3
- $LOAD_PATH.unshift( File.expand_path( '../sportdb-config/lib' ))
3
+ $LOAD_PATH.unshift( File.expand_path( '../sportdb-catalogs/lib' ))
4
4
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-models/lib' ))
5
5
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-sync/lib' ))
6
6
 
@@ -25,7 +25,7 @@ class TestConfReader < MiniTest::Test
25
25
  # path = "../../../openfootball/england/2017-18/.conf.txt"
26
26
  # path = "../../../openfootball/england/2018-19/.conf.txt"
27
27
  # path = "../../../openfootball/england/2019-20/.conf.txt"
28
- SportDb::ConfReaderV2.read( path )
28
+ SportDb::ConfReader.read( path )
29
29
  end # method test_read
30
30
 
31
31
 
@@ -73,6 +73,6 @@ Monaco › MCO
73
73
  APOEL › CYP
74
74
  TXT
75
75
 
76
- SportDb::ConfReaderV2.parse( txt )
76
+ SportDb::ConfReader.parse( txt )
77
77
  end
78
78
  end # class TestConfReader
@@ -363,7 +363,7 @@ Group H:
363
363
  [Llorente 20' Son Heung-min 37' N'Koudou 80']
364
364
  TXT
365
365
 
366
- SportDb::MatchReaderV2.parse( txt )
366
+ SportDb::MatchReader.parse( txt )
367
367
  end
368
368
 
369
369
  def test_read_finals
@@ -480,7 +480,7 @@ Final
480
480
  [Benzema 51' Bale 64', 83'; Mané 55']
481
481
  TXT
482
482
 
483
- SportDb::MatchReaderV2.parse( txt )
483
+ SportDb::MatchReader.parse( txt )
484
484
  end
485
485
 
486
486
 
@@ -58,7 +58,7 @@ Matchday 2
58
58
  Manchester City 1-1 Everton FC
59
59
  TXT
60
60
 
61
- SportDb::MatchReaderV2.parse( txt )
61
+ SportDb::MatchReader.parse( txt )
62
62
  end # method test_read_eng
63
63
 
64
64
  def test_read_eng_2012_13
@@ -87,7 +87,7 @@ Matchday 20
87
87
  QPR 0-3 Liverpool
88
88
  TXT
89
89
 
90
- SportDb::MatchReaderV2.parse( txt )
90
+ SportDb::MatchReader.parse( txt )
91
91
 
92
92
  puts SportDb::Model::Game.count
93
93
  end
@@ -149,7 +149,7 @@ Final
149
149
  (51) Jul/10 21:00 Portugal 0:0 1:0nV France @ Stade de France, Saint-Denis
150
150
  TXT
151
151
 
152
- SportDb::MatchReaderV2.parse( txt )
152
+ SportDb::MatchReader.parse( txt )
153
153
  end
154
154
  end # class TestMatchReaderEuro
155
155
 
@@ -91,7 +91,7 @@ Matchday 3
91
91
  AS Port-Louis 2000 1-0 La Cure Sylvester
92
92
  TXT
93
93
 
94
- SportDb::MatchReaderV2.parse( txt )
94
+ SportDb::MatchReader.parse( txt )
95
95
  end # method test_read_mauritius
96
96
 
97
97
  end # class TestMatchReaderMu
@@ -24,13 +24,13 @@ class TestReader < MiniTest::Test
24
24
  # path = "../../../openfootball/england/2017-18/.conf.txt"
25
25
  # path = "../../../openfootball/england/2018-19/.conf.txt"
26
26
  # path = "../../../openfootball/england/2019-20/.conf.txt"
27
- recs = SportDb::ConfReaderV2.read( path )
27
+ recs = SportDb::ConfReader.read( path )
28
28
  # path = "../../../openfootball/austria/2018-19/1-bundesliga.txt"
29
29
  path = "../../../openfootball/england/2015-16/1-premierleague-i.txt"
30
30
  # path = "../../../openfootball/england/2017-18/1-premierleague-i.txt"
31
31
  # path = "../../../openfootball/england/2018-19/1-premierleague.txt"
32
32
  # path = "../../../openfootball/england/2019-20/1-premierleague.txt"
33
- recs = SportDb::MatchReaderV2.read( path )
33
+ recs = SportDb::MatchReader.read( path )
34
34
  # path = "../../../openfootball/england/2017-18/1-premierleague-ii.txt"
35
35
  #recs = SportDb::MatchReaderV2.read( path )
36
36
  end # method test_read
@@ -64,7 +64,7 @@ Monaco › MCO
64
64
  APOEL › CYP
65
65
  TXT
66
66
 
67
- SportDb::ConfReaderV2.parse( txt )
67
+ SportDb::ConfReader.parse( txt )
68
68
 
69
69
 
70
70
  txt =<<TXT
@@ -180,7 +180,7 @@ Final
180
180
  [Benzema 51' Bale 64', 83'; Mané 55']
181
181
  TXT
182
182
 
183
- SportDb::MatchReaderV2.parse( txt )
183
+ SportDb::MatchReader.parse( txt )
184
184
  end
185
185
 
186
186
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sportdb-readers
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.1
4
+ version: 1.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Gerald Bauer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-05-11 00:00:00.000000000 Z
11
+ date: 2020-08-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sportdb-sync
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 1.0.0
19
+ version: 1.1.1
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 1.0.0
26
+ version: 1.1.1
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: rdoc
29
29
  requirement: !ruby/object:Gem::Requirement