sportdb-readers 1.0.1 → 1.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 59763aec58a5729bbe978c4f81ac60de0b883e3f
4
- data.tar.gz: 8fe8c7a0362bb664620f0392865a2435d8d54ef7
3
+ metadata.gz: b92858d91cd8a8229f492aba44a58e3dd6be1b40
4
+ data.tar.gz: 36d490bff1de4f88216e4092abc73927a1fb0de6
5
5
  SHA512:
6
- metadata.gz: f700cc86b085a7b882211c0d2bc7033287e4b523403374eba95ff58ef2fe9b930fd09d6e4c57483e5b5a978aa487825b7fcdcd34ea873299a94604d443403f83
7
- data.tar.gz: '08a052e7b179189c2eae987c24c6e90915a74f3666a87dd116bef60a4b7266c693d7419415df0311e91f35c287c2c19b0b5b8a7faa86c2f0345f88bd5e672090'
6
+ metadata.gz: 8b509fe042ac69758777e838f245d929807c23ccc3276d1216d0dc127fd54f1758a3474a56469b5c42fb073862431a96bcd85fa78b15979e61272251cfb61237
7
+ data.tar.gz: cb9b1868690d2cd36c9ae2cc50a2cef6962192f072bf5cd70710f9942439d62dbfbb8f6fa67908eddb69853f6a1284840a6502f5daf237864059a716aaf2511e
data/README.md CHANGED
@@ -31,50 +31,116 @@ Let's read in some leagues, seasons, clubs, and match schedules and results.
31
31
  Let's use the public domain football.db datasets for England (see [`openfootball/england`](https://github.com/openfootball/england)), as an example:
32
32
 
33
33
 
34
- ``` ruby
35
- ## turn on logging to console
36
- ActiveRecord::Base.logger = Logger.new( STDOUT )
34
+ ```
35
+ = English Premier League 2015/16
36
+
37
+ Matchday 1
38
+
39
+ [Sat Aug 8]
40
+ Manchester United 1-0 Tottenham Hotspur
41
+ AFC Bournemouth 0-1 Aston Villa
42
+ Everton FC 2-2 Watford FC
43
+ Leicester City 4-2 Sunderland AFC
44
+ Norwich City 1-3 Crystal Palace
45
+ Chelsea FC 2-2 Swansea City
46
+ [Sun Aug 9]
47
+ Arsenal FC 0-2 West Ham United
48
+ Newcastle United 2-2 Southampton FC
49
+ Stoke City 0-1 Liverpool FC
50
+ [Mon Aug 10]
51
+ West Bromwich Albion 0-3 Manchester City
52
+
53
+ ...
54
+ ```
55
+
56
+ (Source: [england/2015-16/1-premierleague-i.txt](https://github.com/openfootball/england/blob/master/2015-16/1-premierleague-i.txt))
37
57
 
58
+ and let's try:
59
+
60
+ ``` ruby
38
61
  ## assumes football.db datasets for England in ./england directory
39
62
  ## see github.com/openfootball/england
40
- SportDb::ConfReaderV2.read( './england/2015-16/.conf.txt' )
41
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-i.txt' )
42
- SportDb::MatchReaderV2.read( './england/2015-16/1-premierleague-ii.txt' )
63
+ SportDb.read( './england/2015-16/1-premierleague-i.txt' )
64
+ SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
43
65
 
44
66
  ## let's try another season
45
- SportDb::ConfReaderV2.read( './england/2019-20/.conf.txt' )
46
- SportDb::MatchReaderV2.read( './england/2019-20/1-premierleague.txt' )
67
+ SportDb.read( './england/2019-20/1-premierleague.txt' )
47
68
  ```
48
69
 
49
70
  All leagues, seasons, clubs, match days and rounds, match fixtures and results,
50
71
  and more are now in your (SQL) database of choice.
51
72
 
52
-
53
- Or as an alternative use the `read` convenience all-in-one shortcut helper:
73
+ The proof of the pudding - Let's query the (SQL) database using the sport.db ActiveRecord models:
54
74
 
55
75
  ``` ruby
56
- ## assumes football.db datasets for England in ./england directory
57
- ## see github.com/openfootball/england
58
- SportDb.read( './england/2015-16/.conf.txt' )
59
- SportDb.read( './england/2015-16/1-premierleague-i.txt' )
60
- SportDb.read( './england/2015-16/1-premierleague-ii.txt' )
76
+ include SportDb::Models
61
77
 
62
- ## let's try another season
63
- SportDb.read( './england/2019-20/.conf.txt' )
64
- SportDb.read( './england/2019-20/1-premierleague.txt' )
78
+ pl_2015_16 = Event.find_by( key: 'eng.1.2015/16' )
79
+ #=> SELECT * FROM events WHERE key = 'eng.1.2015/16' LIMIT 1
80
+
81
+ pl_2015_16.teams.count #=> 20
82
+ #=> SELECT COUNT(*) FROM teams
83
+ # INNER JOIN events_teams ON teams.id = events_teams.team_id
84
+ # WHERE events_teams.event_id = 1
85
+
86
+ pl_2015_16.games.count #=> 380
87
+ #=> SELECT COUNT(*) FROM games
88
+ # INNER JOIN rounds ON games.round_id = rounds.id
89
+ # WHERE rounds.event_id = 1
90
+
91
+ pl_2019_20 = Event.find_by( key: 'eng.1.2019/20' )
92
+ pl_2015_16.teams.count #=> 20
93
+ pl_2015_16.games.count #=> 380
94
+
95
+ # -or-
96
+
97
+ pl = League.find_by( key: 'eng.1' )
98
+ #=> SELECT * FROM leagues WHERE key = 'eng.1' LIMIT 1
99
+
100
+ pl.seasons.count #=> 2
101
+ #=> SELECT COUNT(*) FROM seasons
102
+ # INNER JOIN events ON seasons.id = events.season_id
103
+ # WHERE events.league_id = 1
104
+
105
+ # and so on and so forth.
65
106
  ```
66
107
 
67
- Or as an alternative pass in the "package" directory and let `read` figure
68
- out what datafiles to read:
108
+ Bonus: As an alternative pass in the "package" directory or a zip archive and let `read` figure
109
+ out what datafiles to read in:
69
110
 
70
111
  ``` ruby
71
112
  ## assumes football.db datasets for England in ./england directory
72
113
  ## see github.com/openfootball/england
73
114
  SportDb.read( './england' )
115
+ ## -or- use a zip archive download
116
+ SportDb.read( './england.zip' )
74
117
  ```
75
118
 
76
119
  That's it.
77
120
 
121
+ ## Frequently Asked Questions (F.A.Q.s) and Answers
122
+
123
+ Q: What about reading in datasets in comma-separated values (CSV) format?
124
+ Example:
125
+
126
+ ```
127
+ Round, Date, Team 1, FT, HT, Team 2
128
+ 1, (Fri) 9 Aug 2019, Liverpool FC, 4-1, 4-0, Norwich City FC
129
+ 1, (Sat) 10 Aug 2019, West Ham United FC, 0-5, 0-1, Manchester City FC
130
+ 1, (Sat) 10 Aug 2019, AFC Bournemouth, 1-1, 0-0, Sheffield United FC
131
+ 1, (Sat) 10 Aug 2019, Burnley FC, 3-0, 0-0, Southampton FC
132
+ 1, (Sat) 10 Aug 2019, Crystal Palace FC, 0-0, 0-0, Everton FC
133
+ 1, (Sat) 10 Aug 2019, Watford FC, 0-3, 0-1, Brighton & Hove Albion FC
134
+ 1, (Sat) 10 Aug 2019, Tottenham Hotspur FC, 3-1, 0-1, Aston Villa FC
135
+ 1, (Sun) 11 Aug 2019, Leicester City FC, 0-0, 0-0, Wolverhampton Wanderers FC
136
+ 1, (Sun) 11 Aug 2019, Newcastle United FC, 0-1, 0-0, Arsenal FC
137
+ 1, (Sun) 11 Aug 2019, Manchester United FC, 4-0, 1-0, Chelsea FC
138
+ ...
139
+ ```
140
+ (Source: [england/2019-20/eng.1.csv](https://github.com/footballcsv/england/blob/master/2010s/2019-20/eng.1.csv))
141
+
142
+ Yes, you can. See the [sportdb-importers library / gem »](https://github.com/sportdb/sport.db/tree/master/sportdb-importers)
143
+
78
144
 
79
145
 
80
146
  ## License
data/Rakefile CHANGED
@@ -20,7 +20,7 @@ Hoe.spec 'sportdb-readers' do
20
20
  self.licenses = ['Public Domain']
21
21
 
22
22
  self.extra_deps = [
23
- ['sportdb-sync', '>= 1.0.0'],
23
+ ['sportdb-sync', '>= 1.1.1'],
24
24
  ]
25
25
 
26
26
  self.spec_extras = {
@@ -16,12 +16,12 @@ require 'sportdb/readers/package'
16
16
  ##
17
17
  ## add convenience shortcut helpers
18
18
  module SportDb
19
- def self.read_conf( path, season: nil ) ConfReaderV2.read( path, season: season ); end
20
- def self.parse_conf( txt, season: nil ) ConfReaderV2.parse( txt, season: season ); end
19
+ def self.read_conf( path, season: nil ) ConfReader.read( path, season: season ); end
20
+ def self.parse_conf( txt, season: nil ) ConfReader.parse( txt, season: season ); end
21
21
 
22
22
  ### todo/check: add alias read_matches - why? why not?
23
- def self.read_match( path, season: nil ) MatchReaderV2.read( path, season: season ); end
24
- def self.parse_match( txt, season: nil ) MatchReaderV2.parse( txt, season: season ); end
23
+ def self.read_match( path, season: nil ) MatchReader.read( path, season: season ); end
24
+ def self.parse_match( txt, season: nil ) MatchReader.parse( txt, season: season ); end
25
25
 
26
26
  def self.read_club_props( path ) Import::ClubPropsReader.read( path ); end
27
27
  def self.parse_club_props( txt ) Import::ClubPropsReader.parse( txt ); end
@@ -3,7 +3,7 @@
3
3
  module SportDb
4
4
 
5
5
 
6
- class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
6
+ class ConfReader ## todo/check: rename to EventsReaderV2 (use plural?) why? why not?
7
7
 
8
8
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
9
9
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -96,5 +96,5 @@ class ConfReaderV2 ## todo/check: rename to EventsReaderV2 (use plural?) why?
96
96
 
97
97
  def catalog() Import.catalog; end ## shortcut convenience helper
98
98
 
99
- end # class ConfReaderV2
99
+ end # class ConfReader
100
100
  end # module SportDb
@@ -2,7 +2,7 @@
2
2
 
3
3
  module SportDb
4
4
 
5
- class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
5
+ class MatchReader ## todo/check: rename to MatchReaderV2 (use plural?) why? why not?
6
6
 
7
7
  def self.read( path, season: nil ) ## use - rename to read_file or from_file etc. - why? why not?
8
8
  txt = File.open( path, 'r:utf-8' ) {|f| f.read }
@@ -31,12 +31,14 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
31
31
  langs = { ## map country keys to lang codes
32
32
  'de' => 'de', ## de - Deutsch (German)
33
33
  'at' => 'de',
34
+ 'ch' => 'de',
34
35
  'fr' => 'fr', ## fr - French
35
36
  'it' => 'it', ## it - Italian
36
37
  'es' => 'es', ## es - Español (Spanish)
37
38
  'mx' => 'es',
39
+ 'ar' => 'es', ## Argentina
38
40
  'pt' => 'pt', ## pt - Português (Portuguese)
39
- 'br' => 'br'
41
+ 'br' => 'pt',
40
42
  }
41
43
 
42
44
  secs.each do |sec| ## sec(tion)s
@@ -53,13 +55,23 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
53
55
  Import.config.lang = langs[ league.country.key ] || 'en'
54
56
  end
55
57
 
58
+ ### check if event info availabe - use start_date;
59
+ ## otherwise we have to guess (use a "synthetic" start_date)
60
+ event_info = catalog.events.find_by( season: season,
61
+ league: league )
56
62
 
57
- start = if season.year?
58
- Date.new( season.start_year, 1, 1 )
63
+ start = if event_info && event_info.start_date
64
+ event_info.start_date
59
65
  else
60
- Date.new( season.start_year, 7, 1 )
66
+ if season.year?
67
+ Date.new( season.start_year, 1, 1 )
68
+ else
69
+ Date.new( season.start_year, 7, 1 )
70
+ end
61
71
  end
62
72
 
73
+
74
+ ### todo/check: make sure team include teams from group def too!!!!!
63
75
  auto_conf_teams, _ = AutoConfParser.parse( lines,
64
76
  start: start )
65
77
 
@@ -72,7 +84,10 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
72
84
  ## todo/fix: make more generic / reuseable!!!!
73
85
  mods = {}
74
86
  ## europa league uses same mods as champions league
75
- mods[ 'uefa.el' ] = mods[ 'uefa.cl' ] = catalog.clubs.build_mods(
87
+ mods[ 'uefa.el.quali' ] =
88
+ mods[ 'uefa.cl.quali' ] =
89
+ mods[ 'uefa.el' ] =
90
+ mods[ 'uefa.cl' ] = catalog.clubs.build_mods(
76
91
  { 'Liverpool | Liverpool FC' => 'Liverpool FC, ENG',
77
92
  'Arsenal | Arsenal FC' => 'Arsenal FC, ENG',
78
93
  'Barcelona' => 'FC Barcelona, ESP',
@@ -87,7 +102,7 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
87
102
  team_mapping = auto_conf_teams.keys.zip( teams ).to_h
88
103
 
89
104
 
90
- parser = MatchParserSimpleV2.new( lines,
105
+ parser = MatchParser.new( lines,
91
106
  team_mapping,
92
107
  start ) ## note: keep season start_at date for now (no need for more specific stage date need for now)
93
108
 
@@ -124,18 +139,37 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
124
139
  end
125
140
 
126
141
 
127
- rounds.each do |round|
128
- ## quick hack: if pos missing fill with dummy 999 for now
129
- round.pos = 999 if round.pos.nil?
130
- round_rec = Sync::Round.find_or_create( round, event: event_rec ) ## check: use/rename to EventRound why? why not?
131
- end
142
+ ## build a lookup cache for team_recs (used by group lookup)
143
+ ## lookup by "canonical" name
144
+ cache_team_recs = new_team_recs.reduce({}) { |h,rec| h[rec.name] = rec; h }
132
145
 
133
146
  groups.each do |group|
134
147
  group_rec = Sync::Group.find_or_create( group, event: event_rec ) ## check: use/rename to EventGroup why? why not?
148
+
149
+ ########
150
+ ## add/update teams - todo/fix/clean-up - find a better way or move to sync? - possible?
151
+ ## e.g. group.teams assumes an array of team names e.g.
152
+ ## ["Spain", "Czech Republic", "Turkey", "Croatia"]
153
+ group_team_ids = []
154
+ group.teams.each do |team_name|
155
+ team_rec = cache_team_recs[ team_name ]
156
+ if team_rec.nil? ## assume team MUST always be present/known in mapping (via autoconfig parser)
157
+ puts "!! ERROR - no (cached) team rec found for team in group >#{group.name}< for >#{team_name}<"
158
+ exit 1
159
+ end
160
+ group_team_ids << team_rec.id
161
+ end
162
+ group_rec.team_ids = group_team_ids ## reset/update all teams at once (via ids)
163
+ end
164
+
165
+
166
+ rounds.each do |round|
167
+ round_rec = Sync::Round.find_or_create( round, event: event_rec ) ## check: use/rename to EventRound why? why not?
135
168
  end
136
169
 
137
170
  matches.each do |match|
138
- ## todo/fix: pass along stage (if present): stage - optional!!!!
171
+ ## note: pass along stage (if present): stage - optional from heading!!!!
172
+ match = match.update( stage: stage ) if stage
139
173
  match_rec = Sync::Match.create_or_update( match, event: event_rec )
140
174
  end
141
175
  end
@@ -149,5 +183,5 @@ class MatchReaderV2 ## todo/check: rename to MatchReaderV2 (use plural?) why?
149
183
 
150
184
  def catalog() Import.catalog; end
151
185
 
152
- end # class MatchReaderV2
186
+ end # class MatchReader
153
187
  end # module SportDb
@@ -49,7 +49,8 @@ module SportDb
49
49
  read_leagues()
50
50
  read_clubs()
51
51
  read_club_props()
52
- read_conf( season: season )
52
+ ## note: skip conf(iguration)s for now!!!!!!!
53
+ ## read_conf( season: season )
53
54
  read_match( season: season )
54
55
  else
55
56
  names.each do |name|
@@ -5,8 +5,8 @@ module SportDb
5
5
  module Readers
6
6
 
7
7
  MAJOR = 1 ## todo: namespace inside version or something - why? why not??
8
- MINOR = 0
9
- PATCH = 1
8
+ MINOR = 1
9
+ PATCH = 2
10
10
  VERSION = [MAJOR,MINOR,PATCH].join('.')
11
11
 
12
12
  def self.version
@@ -1,6 +1,6 @@
1
1
  ## note: use the local version of sportdb gems
2
2
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-formats/lib' ))
3
- $LOAD_PATH.unshift( File.expand_path( '../sportdb-config/lib' ))
3
+ $LOAD_PATH.unshift( File.expand_path( '../sportdb-catalogs/lib' ))
4
4
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-models/lib' ))
5
5
  $LOAD_PATH.unshift( File.expand_path( '../sportdb-sync/lib' ))
6
6
 
@@ -25,7 +25,7 @@ class TestConfReader < MiniTest::Test
25
25
  # path = "../../../openfootball/england/2017-18/.conf.txt"
26
26
  # path = "../../../openfootball/england/2018-19/.conf.txt"
27
27
  # path = "../../../openfootball/england/2019-20/.conf.txt"
28
- SportDb::ConfReaderV2.read( path )
28
+ SportDb::ConfReader.read( path )
29
29
  end # method test_read
30
30
 
31
31
 
@@ -73,6 +73,6 @@ Monaco › MCO
73
73
  APOEL › CYP
74
74
  TXT
75
75
 
76
- SportDb::ConfReaderV2.parse( txt )
76
+ SportDb::ConfReader.parse( txt )
77
77
  end
78
78
  end # class TestConfReader
@@ -363,7 +363,7 @@ Group H:
363
363
  [Llorente 20' Son Heung-min 37' N'Koudou 80']
364
364
  TXT
365
365
 
366
- SportDb::MatchReaderV2.parse( txt )
366
+ SportDb::MatchReader.parse( txt )
367
367
  end
368
368
 
369
369
  def test_read_finals
@@ -480,7 +480,7 @@ Final
480
480
  [Benzema 51' Bale 64', 83'; Mané 55']
481
481
  TXT
482
482
 
483
- SportDb::MatchReaderV2.parse( txt )
483
+ SportDb::MatchReader.parse( txt )
484
484
  end
485
485
 
486
486
 
@@ -58,7 +58,7 @@ Matchday 2
58
58
  Manchester City 1-1 Everton FC
59
59
  TXT
60
60
 
61
- SportDb::MatchReaderV2.parse( txt )
61
+ SportDb::MatchReader.parse( txt )
62
62
  end # method test_read_eng
63
63
 
64
64
  def test_read_eng_2012_13
@@ -87,7 +87,7 @@ Matchday 20
87
87
  QPR 0-3 Liverpool
88
88
  TXT
89
89
 
90
- SportDb::MatchReaderV2.parse( txt )
90
+ SportDb::MatchReader.parse( txt )
91
91
 
92
92
  puts SportDb::Model::Game.count
93
93
  end
@@ -149,7 +149,7 @@ Final
149
149
  (51) Jul/10 21:00 Portugal 0:0 1:0nV France @ Stade de France, Saint-Denis
150
150
  TXT
151
151
 
152
- SportDb::MatchReaderV2.parse( txt )
152
+ SportDb::MatchReader.parse( txt )
153
153
  end
154
154
  end # class TestMatchReaderEuro
155
155
 
@@ -91,7 +91,7 @@ Matchday 3
91
91
  AS Port-Louis 2000 1-0 La Cure Sylvester
92
92
  TXT
93
93
 
94
- SportDb::MatchReaderV2.parse( txt )
94
+ SportDb::MatchReader.parse( txt )
95
95
  end # method test_read_mauritius
96
96
 
97
97
  end # class TestMatchReaderMu
@@ -24,13 +24,13 @@ class TestReader < MiniTest::Test
24
24
  # path = "../../../openfootball/england/2017-18/.conf.txt"
25
25
  # path = "../../../openfootball/england/2018-19/.conf.txt"
26
26
  # path = "../../../openfootball/england/2019-20/.conf.txt"
27
- recs = SportDb::ConfReaderV2.read( path )
27
+ recs = SportDb::ConfReader.read( path )
28
28
  # path = "../../../openfootball/austria/2018-19/1-bundesliga.txt"
29
29
  path = "../../../openfootball/england/2015-16/1-premierleague-i.txt"
30
30
  # path = "../../../openfootball/england/2017-18/1-premierleague-i.txt"
31
31
  # path = "../../../openfootball/england/2018-19/1-premierleague.txt"
32
32
  # path = "../../../openfootball/england/2019-20/1-premierleague.txt"
33
- recs = SportDb::MatchReaderV2.read( path )
33
+ recs = SportDb::MatchReader.read( path )
34
34
  # path = "../../../openfootball/england/2017-18/1-premierleague-ii.txt"
35
35
  #recs = SportDb::MatchReaderV2.read( path )
36
36
  end # method test_read
@@ -64,7 +64,7 @@ Monaco › MCO
64
64
  APOEL › CYP
65
65
  TXT
66
66
 
67
- SportDb::ConfReaderV2.parse( txt )
67
+ SportDb::ConfReader.parse( txt )
68
68
 
69
69
 
70
70
  txt =<<TXT
@@ -180,7 +180,7 @@ Final
180
180
  [Benzema 51' Bale 64', 83'; Mané 55']
181
181
  TXT
182
182
 
183
- SportDb::MatchReaderV2.parse( txt )
183
+ SportDb::MatchReader.parse( txt )
184
184
  end
185
185
 
186
186
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sportdb-readers
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.1
4
+ version: 1.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Gerald Bauer
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-05-11 00:00:00.000000000 Z
11
+ date: 2020-08-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sportdb-sync
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 1.0.0
19
+ version: 1.1.1
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 1.0.0
26
+ version: 1.1.1
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: rdoc
29
29
  requirement: !ruby/object:Gem::Requirement