kiwicourse 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (14) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +58 -7
  3. data/bin/kiwicourse +38 -9
  4. data/lib/coursesdesc/courses.rb +31 -34
  5. data/lib/coursesdesc/version.rb +1 -1
  6. data/spec/courses_date_data.rb +1 -1
  7. data/spec/courses_id_data.rb +220 -0
  8. data/spec/courses_name_data.rb +220 -0
  9. data/spec/courses_url_data.rb +220 -0
  10. data/spec/coursesdesc_spec.rb +6 -7
  11. metadata +7 -7
  12. data/spec/courses_id_test_data.rb +0 -219
  13. data/spec/courses_name_test_data.rb +0 -219
  14. data/spec/urls_test_data.rb +0 -219
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 5b0cda5893a3ecaf2a108d71053b49e94a4ef5f1
4
- data.tar.gz: 8ded148d071e55f5acd30d275b87b8c0e949ae09
3
+ metadata.gz: cf325fef6f456da234f7b16a171df31edf564fe3
4
+ data.tar.gz: 6ff8b1e49e58cfc77549283114f152b1416c7ac3
5
5
  SHA512:
6
- metadata.gz: 9746a1e72202559a3f3f2c933620dcefbd72d1ab34fd5313363216ca6ff42fd6f7e164b2ca604250109f2bddbc9abcb450816308938ba9c342e47dfc1d75c1fc
7
- data.tar.gz: 101dee9ca8aeb2e19cc1a6044c48d5008493e196e3d460a346a24e27807ee91edd245c514a63075186bd8b3200bbadaa0bf152381672e812be246d44791e0883
6
+ metadata.gz: 4f45126b2484ba0cf979ee3c2ac75d0891ee2eafe5692d43c4295184f024cdd9ae2f4fce3008351e4d707e46de5638eddd718135c053f4f390a3fc352a2743cc
7
+ data.tar.gz: d07c2bbd7ccee331b2fff2d3195556becb1b21ce683dd2361fdc4ae57013d821eb0235b5da53cf3eda47a08bf72cc9d8258d33c15713bbc7d269450a8a143b54
data/README.md CHANGED
@@ -3,7 +3,9 @@
3
3
  [![Build Status](https://travis-ci.org/Kiwi-Learn/kiwi-scraper.svg?branch=master)](https://travis-ci.org/Kiwi-Learn/kiwi-scraper)
4
4
  [![Gem Version](https://badge.fury.io/rb/kiwicourse.svg)](https://badge.fury.io/rb/kiwicourse)
5
5
 
6
- Kiwi Scraper is a great tool to get [Sharecourse](http://sharecourse.net/sharecourse/general/home/) description!
6
+ Kiwi Scraper is a great tool to get [Sharecourse](http://sharecourse.net/sharecourse/general/home/) information!
7
+
8
+ Yes, you can use CLI to browse course information even you are **offline**!!
7
9
 
8
10
  We respect Sharecourse's `robots.txt`
9
11
 
@@ -14,23 +16,72 @@ Install it with the following command:
14
16
  $ gem install kiwicourse
15
17
  ```
16
18
 
17
- Run it from the command line.
19
+ Run it from the command line. We provide 5 features to you to browse courses information on [Sharecourse](http://sharecourse.net/sharecourse/general/home/).
20
+
21
+ - list
22
+ - search
23
+ - info
24
+ - open
25
+ - update
26
+
27
+
28
+ Help
29
+ ```sh
30
+ $ kiwicourse
31
+ Commands:
32
+ kiwicourse help [COMMAND] # Describe available commands or one specific command
33
+ kiwicourse info ID # Display information about course.
34
+ kiwicourse list # List all courses on ShareCourse
35
+ kiwicourse open ID # Open the course page on browser with course id
36
+ kiwicourse search COURSENAME # Search a course on ShareCourse
37
+ kiwicourse update # Update the offline courses data
38
+ ```
39
+
40
+
41
+ List all courses on [Sharecourse](http://sharecourse.net/sharecourse/general/home/)
42
+ ```sh
43
+ $ kiwicourse list
44
+ MA02004 - 會計學原理
45
+ AO35004 - 行動磨課師【曠世名琴訴說的故事】
46
+ DM91002 - 方法對了，人人都可以是設計師
47
+
48
+ ...
49
+
50
+ CS01001 - 網路安全 Network Security
51
+ CS01002 - 作業系統 Operating Systems
52
+ EE62002 - 小型風力機系統與國際認證 (104 秋季班)
53
+ ```
18
54
 
19
55
  Search a course with a keyword
20
56
  ```sh
21
- $ kiwicourse search cellular
22
- 2015 細胞神經科學Cellular Neuroscience
57
+ $ kiwicourse search 會計
58
+ MA02004 - 會計學原理
23
59
  ```
24
60
 
25
- Open the course url in browser
61
+ Display information about the course.
62
+
26
63
  ```sh
27
- $ kiwicourse open 科學計算
64
+ $ kiwicourse info CS06001
65
+ Course ID: CS06001
66
+ Course: 資料結構
67
+ Course time: 2014-09-14 - 2015-02-28
68
+ Course webpagehttp://www.sharecourse.net/sharecourse/course/view/courseInfo/28
28
69
  ```
29
70
 
71
+ Open the course webpage in browser
72
+ ```sh
73
+ $ kiwicourse open MA02004
74
+ ```
75
+
76
+ Update the offline courses data
77
+ ```sh
78
+ $ kiwicourse update
79
+ Already up to date.
80
+ ```
30
81
 
31
82
  Use it from your Ruby code:
32
83
  ````ruby
33
- require 'kiwicourse'
84
+ require 'coursesdesc'
34
85
  sc = KiwiScraper::ShareCourse.new
35
86
 
36
87
  course_found = sc.course_name
data/bin/kiwicourse CHANGED
@@ -4,12 +4,19 @@ require 'thor'
4
4
  require 'fuzzy_match'
5
5
  require 'launchy'
6
6
  require 'coursesdesc' # for production
7
- # require '../lib/coursesdesc/courses.rb' # for cmd line testing purposes
7
+ # require './lib/coursesdesc/courses.rb' # for cmd line testing purposes
8
+
9
+ DUMP_FILE_PATH = Dir.home + '/.kiwicourse.dump'
8
10
 
9
11
  class KiwiCLI < Thor
10
12
  desc 'search COURSENAME', 'Search a course on ShareCourse'
11
13
  def search(coursename)
12
- sc = KiwiScraper::ShareCourse.new
14
+ if File.exist?(DUMP_FILE_PATH)
15
+ sc = Marshal.load(File.read(DUMP_FILE_PATH))
16
+ else
17
+ sc = KiwiScraper::ShareCourse.new
18
+ File.open(DUMP_FILE_PATH, 'w') { |f| f.write(Marshal.dump(sc)) }
19
+ end
13
20
  result = FuzzyMatch.new(sc.course_name).find(coursename)
14
21
  input_key = Digest::SHA256.digest result
15
22
  id = sc.courses_name_to_id_mapping[input_key]
@@ -18,28 +25,50 @@ class KiwiCLI < Thor
18
25
 
19
26
  desc 'list', 'List all courses on ShareCourse'
20
27
  def list
21
- sc = KiwiScraper::ShareCourse.new
22
- sc.courses_id_mapping.each do |key, info|
28
+ if File.exist?(DUMP_FILE_PATH)
29
+ sc = Marshal.load(File.read(DUMP_FILE_PATH))
30
+ else
31
+ sc = KiwiScraper::ShareCourse.new
32
+ File.open(DUMP_FILE_PATH, 'w') { |f| f.write(Marshal.dump(sc)) }
33
+ end
34
+ sc.courses_id_to_all_mapping.each do |key, info|
23
35
  puts "#{key} - #{info['name']}"
24
36
  end
25
37
  end
26
38
 
27
- desc 'open ID', 'open the course page on browser with course id'
39
+ desc 'open ID', 'Open the course page on browser with course id'
28
40
  def open(id)
29
- sc = KiwiScraper::ShareCourse.new
30
- course = sc.courses_id_mapping[id]
41
+ if File.exist?(DUMP_FILE_PATH)
42
+ sc = Marshal.load(File.read(DUMP_FILE_PATH))
43
+ else
44
+ sc = KiwiScraper::ShareCourse.new
45
+ File.open(DUMP_FILE_PATH, 'w') { |f| f.write(Marshal.dump(sc)) }
46
+ end
47
+ course = sc.courses_id_to_all_mapping[id]
31
48
  Launchy.open(course['url'])
32
49
  end
33
50
 
34
51
  desc 'info ID', 'Display information about course.'
35
52
  def info(id)
36
- sc = KiwiScraper::ShareCourse.new
37
- course = sc.courses_id_mapping[id]
53
+ if File.exist?(DUMP_FILE_PATH)
54
+ sc = Marshal.load(File.read(DUMP_FILE_PATH))
55
+ else
56
+ sc = KiwiScraper::ShareCourse.new
57
+ File.open(DUMP_FILE_PATH, 'w') { |f| f.write(Marshal.dump(sc)) }
58
+ end
59
+ course = sc.courses_id_to_all_mapping[id]
38
60
  puts "Course ID: #{course['id']}"
39
61
  puts "Course: #{course['name']}"
40
62
  puts "Course time: #{course['date']}"
41
63
  puts "Course webpage#{course['url']}"
42
64
  end
65
+
66
+ desc 'update', 'Update the offline courses data'
67
+ def update
68
+ sc = KiwiScraper::ShareCourse.new
69
+ File.open(DUMP_FILE_PATH, 'w') { |f| f.write(Marshal.dump(sc)) }
70
+ puts 'Already up to date.'
71
+ end
43
72
  end
44
73
 
45
74
  KiwiCLI.start(ARGV)
data/lib/coursesdesc/courses.rb CHANGED
@@ -9,11 +9,7 @@ module KiwiScraper
9
9
  URL = 'http://www.sharecourse.net/sharecourse/course/view/courseList'
10
10
 
11
11
  def initialize
12
- get_html
13
- end
14
-
15
- def get_html
16
- @document = Oga.parse_html(open(URL))
12
+ parse_html
17
13
  end
18
14
 
19
15
  def course_name
@@ -21,7 +17,7 @@ module KiwiScraper
21
17
  end
22
18
 
23
19
  def course_date
24
- @course_date ||= get_course_date
20
+ @course_date ||= parse_course_date
25
21
  end
26
22
 
27
23
  def course_url
@@ -29,19 +25,31 @@ module KiwiScraper
29
25
  end
30
26
 
31
27
  def courses_name_to_id_mapping
32
- @course_map ||= make_course_name_to_id_map
28
+ @course_map ||= map_course_name_to_id
33
29
  end
34
30
 
35
31
  def course_id
36
32
  @course_id ||= parse_course_id
37
33
  end
38
34
 
39
- def courses_id_mapping
40
- @course_id_map ||= map_course_id
35
+ def courses_id_to_all_mapping
36
+ @course_id_map ||= map_course_id_to_all
41
37
  end
42
38
 
43
39
  private
44
40
 
41
+ def parse_html
42
+ @document = Oga.parse_html(open(URL))
43
+ end
44
+
45
+ def parse_course_id
46
+ course_id = []
47
+ @document.xpath("//div[@class='gb_mid']").each do |course|
48
+ course_id << course.children[4].children[0].children.text.split('：')[1]
49
+ end
50
+ course_id
51
+ end
52
+
45
53
  def parse_course_name
46
54
  result = []
47
55
  @document.xpath("//h4[@id='courseName']").each do |course|
@@ -58,7 +66,17 @@ module KiwiScraper
58
66
  result
59
67
  end
60
68
 
61
- def make_course_name_to_id_map
69
+ def parse_course_date
70
+ # date type yyyy-mm-dd - yyyy-mm-dd
71
+ date = []
72
+ @document.xpath("//p[@id='courseDate']").each do |course|
73
+ longdate = course.text
74
+ date << longdate.split('：')[1]
75
+ end
76
+ date
77
+ end
78
+
79
+ def map_course_name_to_id
62
80
  name = parse_course_name
63
81
  course_id = parse_course_id
64
82
 
@@ -67,15 +85,13 @@ module KiwiScraper
67
85
  hash_name[index] = Digest::SHA256.digest name[index]
68
86
  end
69
87
 
70
- result = Hash[hash_name.zip(course_id)]
71
-
72
- result
88
+ Hash[hash_name.zip(course_id)] # return value
73
89
  end
74
90
 
75
- def map_course_id
91
+ def map_course_id_to_all
76
92
  name = parse_course_name
77
93
  course_id = parse_course_id
78
- date = get_course_date
94
+ date = parse_course_date
79
95
  url = parse_course_url
80
96
  info = {}
81
97
 
@@ -90,24 +106,5 @@ module KiwiScraper
90
106
  end
91
107
  info
92
108
  end
93
-
94
- def get_course_date
95
- #date type yyyy-mm-dd - yyyy-mm-dd
96
- date = []
97
- @document.xpath("//p[@id='courseDate']").each do |course|
98
- longdate = course.text
99
- date << longdate.split("：")[1]
100
- end
101
- date
102
- end
103
-
104
- def parse_course_id
105
- course_id = []
106
- @document.xpath("//div[@class='gb_mid']").each do |course|
107
- course_id << course.children[4].children[0].children.text.split("：")[1]
108
- end
109
- course_id
110
- end
111
-
112
109
  end
113
110
  end
data/lib/coursesdesc/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # Versioning
2
2
  module CoursesDesc
3
- VERSION = '0.3.0'
3
+ VERSION = '0.4.0'
4
4
  DATE = '2015-10-18'
5
5
  end
data/spec/courses_date_data.rb CHANGED
@@ -1,4 +1,4 @@
1
- COURSES_DATE = [
1
+ COURSES_DATE_LIST = [
2
2
  '2015-10-12 - 2015-01-31',
3
3
  '2015-10-19 - 2015-12-06',
4
4
  '2015-10-19 - 2015-12-06',
data/spec/courses_id_data.rb ADDED
@@ -0,0 +1,220 @@
1
+ COURSES_ID_LIST = [
2
+ 'MA02004',
3
+ 'AO35004',
4
+ 'DM91002',
5
+ 'WL33002',
6
+ 'SP31002',
7
+ 'SO31003',
8
+ 'SO31004',
9
+ 'AN54005',
10
+ 'CM81009',
11
+ 'MM39012',
12
+ 'MM39013',
13
+ 'ML22006',
14
+ 'CS08006',
15
+ 'OL48008',
16
+ 'OL48009',
17
+ 'OL48010',
18
+ 'CM99001',
19
+ 'AN99002',
20
+ 'WL32009',
21
+ 'OL32010',
22
+ 'JL91001',
23
+ 'MM93001',
24
+ 'PH05009',
25
+ 'CS01023',
26
+ 'CS92001',
27
+ 'CS121000',
28
+ 'CS01022',
29
+ 'CS01024',
30
+ 'CS08004',
31
+ 'SC46005',
32
+ 'SC46006',
33
+ 'OL33001',
34
+ 'CS01024',
35
+ 'HS05007',
36
+ 'PH05008',
37
+ 'AN54004',
38
+ 'CS13001',
39
+ 'CS81006',
40
+ 'CM81005',
41
+ 'MM81007',
42
+ 'CM81008',
43
+ 'CS08005',
44
+ 'ML22007',
45
+ 'ML34002',
46
+ 'SS39009',
47
+ 'MM39010',
48
+ 'AM39011',
49
+ 'ML98001',
50
+ 'CS01021',
51
+ 'CS08003',
52
+ 'CM81004',
53
+ 'OL48007',
54
+ 'CS32008',
55
+ 'LI01023',
56
+ 'MM81003',
57
+ 'CS81002',
58
+ 'WL32006',
59
+ 'ML22005',
60
+ 'CM81001',
61
+ 'CS08002',
62
+ 'CS05005',
63
+ 'CH54003',
64
+ 'CS08001',
65
+ 'ML22004',
66
+ 'CS01019',
67
+ 'OL31001',
68
+ 'CS02005',
69
+ 'CS05006',
70
+ 'AN54002',
71
+ 'EE57005',
72
+ 'OL48005',
73
+ 'OL48006',
74
+ 'SC46004',
75
+ 'SC46003',
76
+ 'MM10037',
77
+ 'EE10038',
78
+ 'AO35003',
79
+ 'AO35002',
80
+ 'CM55004',
81
+ 'AO35001',
82
+ 'CM55003',
83
+ 'EE57004',
84
+ 'SC46002',
85
+ 'EC05001',
86
+ 'ML22003',
87
+ 'LE58001',
88
+ 'CE02002',
89
+ 'CS02003',
90
+ 'TT10201',
91
+ 'TO60001',
92
+ 'OL59001',
93
+ 'EE57003',
94
+ 'MM16002',
95
+ 'ML22002',
96
+ 'MS57002',
97
+ 'OL48003',
98
+ 'AN54001',
99
+ 'OL48002',
100
+ 'NB13001',
101
+ 'MM16003',
102
+ 'MM16004',
103
+ 'OL48004',
104
+ 'EE10031',
105
+ 'CS10032',
106
+ 'EE10033',
107
+ 'EE10034',
108
+ 'EE10035',
109
+ 'EE10036',
110
+ 'CS05003',
111
+ 'CS05004',
112
+ 'EP57001',
113
+ 'WL32005',
114
+ 'ML22001',
115
+ 'OL32004',
116
+ 'EE62001',
117
+ 'CS51001',
118
+ 'PH05002',
119
+ 'EE50001',
120
+ 'OL48001',
121
+ 'CM55002',
122
+ 'CS01017',
123
+ 'SC46001',
124
+ 'CM55001',
125
+ 'GE10206',
126
+ 'CS06001',
127
+ 'GE10204',
128
+ 'EM01018',
129
+ 'CS32003',
130
+ 'GE10203',
131
+ 'HR10018',
132
+ 'HR10007',
133
+ 'HR10008',
134
+ 'HR10009',
135
+ 'HR10010',
136
+ 'HR10011',
137
+ 'HR10012',
138
+ 'HR10013',
139
+ 'HR10014',
140
+ 'HR10015',
141
+ 'HR10016',
142
+ 'HR10017',
143
+ 'CS32002',
144
+ 'HR10001',
145
+ 'HR10002',
146
+ 'HR10003',
147
+ 'HR10004',
148
+ 'HR10005',
149
+ 'HR10006',
150
+ 'CS10007',
151
+ 'CS10003',
152
+ 'CS05003',
153
+ 'EE10028',
154
+ 'CS05002',
155
+ 'EE10024',
156
+ 'EE10025',
157
+ 'EE10026',
158
+ 'EE10020',
159
+ 'EE10021',
160
+ 'EE10022',
161
+ 'EE10023',
162
+ 'EE10027',
163
+ 'CS10004',
164
+ 'EE10018',
165
+ 'CS10005',
166
+ 'EE10019',
167
+ 'EE10014',
168
+ 'EE10015',
169
+ 'EE10017',
170
+ 'EE10013',
171
+ 'EE10016',
172
+ 'CS10006',
173
+ 'EE10012',
174
+ 'EE10010',
175
+ 'EE10011',
176
+ 'EE10008',
177
+ 'EE10009',
178
+ 'EE10005',
179
+ 'CS32001',
180
+ 'EE10004',
181
+ 'EE10002',
182
+ 'EE10003',
183
+ 'EE10007',
184
+ 'EE10029',
185
+ 'EE10030',
186
+ 'EE10001',
187
+ 'EE10006',
188
+ 'CS10001',
189
+ 'CM10001',
190
+ 'ML10001',
191
+ 'CS25001',
192
+ 'CS10002',
193
+ 'MN09009',
194
+ 'CS01016',
195
+ 'CS02001',
196
+ 'CS01013',
197
+ 'CS01014',
198
+ 'CS01015',
199
+ 'CS18001',
200
+ 'CS01007',
201
+ 'CS16001',
202
+ 'CS05001',
203
+ 'CS01009',
204
+ 'CS01010',
205
+ 'CS07001',
206
+ 'MN09007',
207
+ 'HSS19001',
208
+ 'CS01012',
209
+ 'CS01011',
210
+ 'EE01003',
211
+ 'CS01004',
212
+ 'CS04001',
213
+ 'CS01005',
214
+ 'CS03001',
215
+ 'CS04001',
216
+ 'CS01008',
217
+ 'CS01006',
218
+ 'CS01001',
219
+ 'CS01002',
220
+ 'EE62002']