markify 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +55 -9
- data/examples/config.yml.example +2 -1
- data/lib/markify.rb +17 -6
- data/lib/markify/optparser.rb +3 -3
- data/lib/markify/scraper/base.rb +50 -0
- data/lib/markify/{scraper.rb → scraper/hbrs.rb} +16 -24
- data/lib/markify/settings.rb +4 -3
- data/lib/markify/version.rb +5 -6
- metadata +11 -11
data/README.md
CHANGED
|
@@ -1,9 +1,26 @@
|
|
|
1
1
|
# Markify
|
|
2
2
|
|
|
3
|
-
Markify is a ruby script to detect new marks in the
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
Markify is a ruby script to detect new marks or course assessments in the student information system of your university.
|
|
4
|
+
Supported universities are:
|
|
5
|
+
|
|
6
|
+
* [University of Applied Sciences Bonn-Rhein-Sieg](http://h-brs.de)
|
|
7
|
+
|
|
8
|
+
If new marks are detected, Markify will send you a [XMPP](http://xmpp.org/) message for every change.
|
|
9
|
+
|
|
10
|
+
The main reason for creating Markify is some kind of disappointing. Our university is able to publish the results of
|
|
11
|
+
exams online, but the web based student information system is not able to send notifications if something is changed.
|
|
12
|
+
That is very annoying especially in the time when you have to write exams, but you also waiting for results.
|
|
13
|
+
|
|
14
|
+
What Markify does is to do the same procedure which would you do to look for new marks.
|
|
15
|
+
|
|
16
|
+
1. Go to the student information system login page.
|
|
17
|
+
2. Login with your user data.
|
|
18
|
+
3. Go to your performance record.
|
|
19
|
+
4. Scrape the page.
|
|
20
|
+
5. Recognize changes.
|
|
21
|
+
6. Send notifications.
|
|
22
|
+
|
|
23
|
+
The benefit of Markify against your manual interaction is, that you can run Markify automatically per cron.
|
|
7
24
|
|
|
8
25
|
## Installation
|
|
9
26
|
|
|
@@ -13,11 +30,25 @@ Install Markify via rubygems:
|
|
|
13
30
|
|
|
14
31
|
## Usage
|
|
15
32
|
|
|
16
|
-
|
|
33
|
+
1. Create the initial Markify configuration.
|
|
34
|
+
|
|
35
|
+
$ markify --init
|
|
36
|
+
|
|
37
|
+
2. Add your student information system and XMPP login data to your configuration. Default path ``$HOME/markify/config.yml``.
|
|
38
|
+
|
|
39
|
+
3. Test your configuration.
|
|
17
40
|
|
|
18
|
-
|
|
41
|
+
$ markify --test
|
|
19
42
|
|
|
20
|
-
|
|
43
|
+
4. Run Markify to create hash database and insert all known marks.
|
|
44
|
+
|
|
45
|
+
$ markify -v
|
|
46
|
+
|
|
47
|
+
5. Configure cron job to run Markify every hour.
|
|
48
|
+
|
|
49
|
+
$ crontab -e
|
|
50
|
+
|
|
51
|
+
@hourly /usr/local/bin/bash -c "/usr/local/bin/markify -s"
|
|
21
52
|
|
|
22
53
|
Possible options to run markify:
|
|
23
54
|
|
|
@@ -25,7 +56,7 @@ Possible options to run markify:
|
|
|
25
56
|
|
|
26
57
|
Optional options:
|
|
27
58
|
-s, --send Send xmpp messages
|
|
28
|
-
-n, --noop No operation, only
|
|
59
|
+
-n, --noop No operation, only stdout output
|
|
29
60
|
-v, --[no-]verbose Run verbosely
|
|
30
61
|
|
|
31
62
|
-f, --config-file FILE Config file (default: ~/markify/config.yml)
|
|
@@ -34,7 +65,22 @@ Possible options to run markify:
|
|
|
34
65
|
--init Create example configuration
|
|
35
66
|
--test Test configuration and accounts
|
|
36
67
|
-h, --help Show this message
|
|
37
|
-
--version Show version
|
|
68
|
+
--version Show version information
|
|
69
|
+
|
|
70
|
+
## Notification example
|
|
71
|
+
|
|
72
|
+
exam: BCS-3-SPEZ Transportnetze und Hochgeschwindigkeitsnetze
|
|
73
|
+
mark: 1.7
|
|
74
|
+
passed: BE
|
|
75
|
+
try: 3
|
|
76
|
+
date: 2042-01-23
|
|
77
|
+
|
|
78
|
+
hash: 05d48ced524708314d4ff2e628f7200b06f02ed2e40a36efd92bbf449f474cce
|
|
79
|
+
|
|
80
|
+
## Contribution
|
|
81
|
+
|
|
82
|
+
The Markify source code is [hosted on GitHub](https://github.com/meise/markify). Please use the
|
|
83
|
+
[issue tracker](https://github.com/meise/markify/issues) and let me know about errors or ideas to improve this software.
|
|
38
84
|
|
|
39
85
|
## License
|
|
40
86
|
|
data/examples/config.yml.example
CHANGED
data/lib/markify.rb
CHANGED
|
@@ -24,32 +24,43 @@ require 'yaml'
|
|
|
24
24
|
require 'pathname'
|
|
25
25
|
|
|
26
26
|
require "markify/version"
|
|
27
|
-
require "markify/scraper"
|
|
28
27
|
require "markify/database"
|
|
29
28
|
require "markify/bot"
|
|
30
29
|
require "markify/settings"
|
|
31
30
|
require "markify/optparser"
|
|
32
31
|
require 'markify/mark'
|
|
32
|
+
require 'markify/scraper/base'
|
|
33
|
+
require 'markify/scraper/hbrs'
|
|
33
34
|
|
|
34
35
|
module Markify
|
|
35
36
|
|
|
36
37
|
def self.run!
|
|
37
38
|
@options = Markify::OptParser.parse!
|
|
38
39
|
@config = Markify::Settings.load!(@options[:config_file])
|
|
40
|
+
@scraper = nil
|
|
41
|
+
|
|
42
|
+
mark_database = Markify::Database.new(@config['general']['database_file'])
|
|
43
|
+
|
|
44
|
+
case @config['university']['acronym'].downcase
|
|
45
|
+
when /hbrs/
|
|
46
|
+
@scraper = Markify::Scraper::Hbrs.new(@config['university']['login_name'],
|
|
47
|
+
@config['university']['login_password'])
|
|
48
|
+
else
|
|
49
|
+
puts 'No support for your university.'
|
|
50
|
+
exit
|
|
51
|
+
end
|
|
39
52
|
|
|
40
53
|
if @options[:test]
|
|
41
|
-
Markify::Settings.test_settings(@config)
|
|
54
|
+
Markify::Settings.test_settings(@config, @scraper)
|
|
42
55
|
exit
|
|
43
56
|
end
|
|
44
57
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
all_marks = Markify::Scraper.new(@config['sis']['login_name'], @config['sis']['login_password']).scrape!
|
|
58
|
+
all_marks = @scraper.scrape!
|
|
48
59
|
new_marks = mark_database.check_for_new_marks(all_marks)
|
|
49
60
|
|
|
50
61
|
if new_marks.count == 0 && (@config['general']['verbose'] || @options[:noop])
|
|
51
62
|
puts "No new marks."
|
|
52
|
-
exit
|
|
63
|
+
exit 0
|
|
53
64
|
end
|
|
54
65
|
|
|
55
66
|
bot = Markify::Bot.new(@config['xmpp']['bot_id'], @config['xmpp']['bot_password']) if @options[:send]
|
data/lib/markify/optparser.rb
CHANGED
|
@@ -37,11 +37,11 @@ module Markify::OptParser
|
|
|
37
37
|
opts.separator ""
|
|
38
38
|
opts.separator "Optional options:"
|
|
39
39
|
|
|
40
|
-
opts.on('-s', '--send', 'Send
|
|
40
|
+
opts.on('-s', '--send', 'Send XMPP messages') do |s|
|
|
41
41
|
options[:send] = s
|
|
42
42
|
end
|
|
43
43
|
|
|
44
|
-
opts.on('-n', '--noop', 'No operation, only
|
|
44
|
+
opts.on('-n', '--noop', 'No operation, only stdout output') do |n|
|
|
45
45
|
options[:noop] = n
|
|
46
46
|
end
|
|
47
47
|
|
|
@@ -75,7 +75,7 @@ module Markify::OptParser
|
|
|
75
75
|
exit
|
|
76
76
|
end
|
|
77
77
|
|
|
78
|
-
opts.on_tail('--version', 'Show version
|
|
78
|
+
opts.on_tail('--version', 'Show version information') do
|
|
79
79
|
puts Markify::LICENCE
|
|
80
80
|
exit
|
|
81
81
|
end
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# encoding: utf-8
|
|
2
|
+
=begin
|
|
3
|
+
Copyright Daniel Meißner <meise+markify@3st.be>, 2013
|
|
4
|
+
|
|
5
|
+
This file is part of Markify.
|
|
6
|
+
|
|
7
|
+
Markify is free software: you can redistribute it and/or modify
|
|
8
|
+
it under the terms of the GNU General Public License as published by
|
|
9
|
+
the Free Software Foundation, either version 3 of the License, or
|
|
10
|
+
(at your option) any later version.
|
|
11
|
+
|
|
12
|
+
Markify is distributed in the hope that it will be useful,
|
|
13
|
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
14
|
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
15
|
+
GNU General Public License for more details.
|
|
16
|
+
|
|
17
|
+
You should have received a copy of the GNU General Public License
|
|
18
|
+
along with Markify. If not, see <http://www.gnu.org/licenses/>.
|
|
19
|
+
=end
|
|
20
|
+
|
|
21
|
+
require 'mechanize'
|
|
22
|
+
|
|
23
|
+
module Markify::Scraper
|
|
24
|
+
class Base
|
|
25
|
+
|
|
26
|
+
attr_reader :marks
|
|
27
|
+
|
|
28
|
+
def initialize(login_name, login_password, sis_login_page)
|
|
29
|
+
@agent = Mechanize.new
|
|
30
|
+
original, library = */(.*) \(.*\)$/.match(@agent.user_agent)
|
|
31
|
+
@agent.user_agent =
|
|
32
|
+
"#{Markify::NAME.capitalize}/#{Markify::VERSION} #{library} (https://github.com/meise/markify)"
|
|
33
|
+
|
|
34
|
+
@data = {}
|
|
35
|
+
@data[:login_page] = sis_login_page
|
|
36
|
+
@data[:login_name] = login_name
|
|
37
|
+
@data[:login_password] = login_password
|
|
38
|
+
|
|
39
|
+
@marks = []
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def scrape!
|
|
43
|
+
scrape
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def test_login
|
|
47
|
+
puts 'Scraper: Nothing to test.'
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -18,45 +18,33 @@ You should have received a copy of the GNU General Public License
|
|
|
18
18
|
along with Markify. If not, see <http://www.gnu.org/licenses/>.
|
|
19
19
|
=end
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
class Markify::Scraper::Hbrs < Markify::Scraper::Base
|
|
22
22
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
@agent = Mechanize.new
|
|
29
|
-
|
|
30
|
-
@sis = {}
|
|
31
|
-
@sis[:login_page] = sis_login_page || 'https://dias.fh-bonn-rhein-sieg.de/d3/SISEgo.asp?UserAcc=Gast&DokID=DiasSWeb&ADias2Dction=Login'
|
|
32
|
-
@sis[:login_name] = login_name
|
|
33
|
-
@sis[:login_password] = login_password
|
|
34
|
-
|
|
35
|
-
@marks = []
|
|
36
|
-
end
|
|
37
|
-
|
|
38
|
-
def scrape!
|
|
39
|
-
scrape
|
|
23
|
+
def initialize(login_name, login_password)
|
|
24
|
+
super(login_name,
|
|
25
|
+
login_password,
|
|
26
|
+
'https://dias.fh-bonn-rhein-sieg.de/d3/SISEgo.asp?UserAcc=Gast&DokID=DiasSWeb&ADias2Dction=Login'
|
|
27
|
+
)
|
|
40
28
|
end
|
|
41
29
|
|
|
42
30
|
def test_login
|
|
43
|
-
login_page = @agent.get(@
|
|
44
|
-
error_page = sis_login(@
|
|
31
|
+
login_page = @agent.get(@data[:login_page])
|
|
32
|
+
error_page = sis_login(@data[:login_name], @data[:login_password], login_page)
|
|
45
33
|
|
|
46
34
|
if error_page.search(".//*[@id='inhalt']/table/tbody/tr/td/form/center/table/tr[3]/td").text =~
|
|
47
35
|
/Benutzername unbekannt oder falsches Kennwort eingegeben!/
|
|
48
|
-
puts "
|
|
36
|
+
puts "Scraper: Username and password wrong."
|
|
49
37
|
else
|
|
50
|
-
puts "
|
|
38
|
+
puts "Scraper: Login works."
|
|
51
39
|
end
|
|
52
40
|
end
|
|
53
41
|
|
|
54
42
|
protected
|
|
55
43
|
|
|
56
44
|
def scrape
|
|
57
|
-
login_page = @agent.get(@
|
|
45
|
+
login_page = @agent.get(@data[:login_page])
|
|
58
46
|
|
|
59
|
-
first_sis_page = sis_login(@
|
|
47
|
+
first_sis_page = sis_login(@data[:login_name], @data[:login_password], login_page)
|
|
60
48
|
marks_table = get_marks_table(first_sis_page)
|
|
61
49
|
|
|
62
50
|
get_marks(marks_table)
|
|
@@ -71,6 +59,10 @@ class Markify::Scraper
|
|
|
71
59
|
next
|
|
72
60
|
end
|
|
73
61
|
|
|
62
|
+
if mark[7].text =~ /AN/
|
|
63
|
+
next
|
|
64
|
+
end
|
|
65
|
+
|
|
74
66
|
@marks << Markify::Mark.new(
|
|
75
67
|
mark[1].text, # curse name
|
|
76
68
|
mark[0].text, # id
|
data/lib/markify/settings.rb
CHANGED
|
@@ -48,7 +48,8 @@ module Markify::Settings
|
|
|
48
48
|
|
|
49
49
|
File.open(config_file, 'a+', 0600) do |file|
|
|
50
50
|
file.puts <<CONTENT
|
|
51
|
-
|
|
51
|
+
university:
|
|
52
|
+
acronym: hbrs
|
|
52
53
|
login_name: "foobaz2s"
|
|
53
54
|
login_password: "fnord2000"
|
|
54
55
|
|
|
@@ -70,8 +71,8 @@ CONTENT
|
|
|
70
71
|
end
|
|
71
72
|
end
|
|
72
73
|
|
|
73
|
-
def self.test_settings(config)
|
|
74
|
-
|
|
74
|
+
def self.test_settings(config, scraper)
|
|
75
|
+
scraper.test_login
|
|
75
76
|
|
|
76
77
|
begin
|
|
77
78
|
Markify::Bot.new(config['xmpp']['bot_id'],
|
data/lib/markify/version.rb
CHANGED
|
@@ -20,14 +20,13 @@ along with Markify. If not, see <http://www.gnu.org/licenses/>.
|
|
|
20
20
|
|
|
21
21
|
module Markify
|
|
22
22
|
|
|
23
|
-
VERSION = '0.
|
|
23
|
+
VERSION = '0.2.0'
|
|
24
24
|
NAME = 'markify'
|
|
25
25
|
|
|
26
26
|
DESCRIPTION =<<DESCRIPTION
|
|
27
|
-
#{Markify::NAME.capitalize} is a ruby script to detect new marks in the
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
message for every change.
|
|
27
|
+
#{Markify::NAME.capitalize} is a ruby script to detect new marks or course assessments in the student
|
|
28
|
+
information system of your university. If #{Markify::NAME} detects new marks, they will send you a XMPP message for
|
|
29
|
+
every change. Supported universities: University of Applied Sciences Bonn-Rhein-Sieg.
|
|
31
30
|
DESCRIPTION
|
|
32
31
|
|
|
33
32
|
LICENCE =<<LICENCE
|
|
@@ -36,7 +35,7 @@ Released under the GNU GENERAL PUBLIC LICENSE Version 3. © Daniel Meißner, 201
|
|
|
36
35
|
LICENCE
|
|
37
36
|
|
|
38
37
|
SUMMARY =<<SUMMARY
|
|
39
|
-
Mark notify script for
|
|
38
|
+
Mark notify script for different universities.
|
|
40
39
|
SUMMARY
|
|
41
40
|
|
|
42
41
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: markify
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
prerelease:
|
|
6
6
|
platform: ruby
|
|
7
7
|
authors:
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2013-
|
|
12
|
+
date: 2013-05-01 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: xmpp4r
|
|
@@ -123,13 +123,13 @@ dependencies:
|
|
|
123
123
|
- - ! '>='
|
|
124
124
|
- !ruby/object:Gem::Version
|
|
125
125
|
version: '0'
|
|
126
|
-
description: ! 'Markify is a ruby script to detect new marks
|
|
126
|
+
description: ! 'Markify is a ruby script to detect new marks or course assessments
|
|
127
|
+
in the student
|
|
127
128
|
|
|
128
|
-
|
|
129
|
+
information system of your university. If markify detects new marks, they will send
|
|
130
|
+
you a XMPP message for
|
|
129
131
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
message for every change.
|
|
132
|
+
every change. Supported universities: University of Applied Sciences Bonn-Rhein-Sieg.
|
|
133
133
|
|
|
134
134
|
'
|
|
135
135
|
email:
|
|
@@ -153,7 +153,8 @@ files:
|
|
|
153
153
|
- lib/markify/database.rb
|
|
154
154
|
- lib/markify/mark.rb
|
|
155
155
|
- lib/markify/optparser.rb
|
|
156
|
-
- lib/markify/scraper.rb
|
|
156
|
+
- lib/markify/scraper/base.rb
|
|
157
|
+
- lib/markify/scraper/hbrs.rb
|
|
157
158
|
- lib/markify/settings.rb
|
|
158
159
|
- lib/markify/version.rb
|
|
159
160
|
- markify.gemspec
|
|
@@ -181,14 +182,13 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
181
182
|
version: '0'
|
|
182
183
|
segments:
|
|
183
184
|
- 0
|
|
184
|
-
hash:
|
|
185
|
+
hash: 3264874439132320863
|
|
185
186
|
requirements: []
|
|
186
187
|
rubyforge_project:
|
|
187
188
|
rubygems_version: 1.8.23
|
|
188
189
|
signing_key:
|
|
189
190
|
specification_version: 3
|
|
190
|
-
summary: Mark notify script for
|
|
191
|
-
of Applied Sciences Bonn-Rhein-Sieg.
|
|
191
|
+
summary: Mark notify script for different universities.
|
|
192
192
|
test_files:
|
|
193
193
|
- spec/markify/database_spec.rb
|
|
194
194
|
- spec/markify/mark_spec.rb
|