opener-outlet 1.1.2 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +30 -33
- data/bin/outlet-daemon +5 -5
- data/bin/outlet-server +6 -4
- data/exec/outlet.rb +3 -3
- data/lib/opener/outlet/output.rb +2 -4
- data/lib/opener/outlet/server.rb +49 -87
- data/lib/opener/outlet/version.rb +1 -1
- data/lib/opener/outlet/visualizer.rb +0 -3
- data/lib/opener/outlet.rb +20 -4
- data/opener-outlet.gemspec +6 -9
- metadata +20 -91
- data/visualizer.rb +0 -278
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4f3d92a8588ca78a7e1d07bbffe6d634a83a989a
|
4
|
+
data.tar.gz: 811797473b1e4bd1ec23da927ecf9e1769046c72
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d47525b3693fcf0faf4f939a3f47481101bee5a850df55636cc521db404d1cbcdc4eab20a11ecfaa35dd59c87246051c0df03883a7cbc9620f9d2327ab3f25b3
|
7
|
+
data.tar.gz: c06881374a30bafbd37562f51d4114db03cf7b93a5a942d8066de59170598265aac9571d217fc326dabd87a6934d6a94a72ca07814b4065db9f434ffaef28fdc
|
data/README.md
CHANGED
@@ -1,20 +1,24 @@
|
|
1
|
-
Outlet
|
2
|
-
------------
|
1
|
+
# Outlet
|
3
2
|
|
4
|
-
Component that stores results from the Opener Web Services chain into a SQLite
|
3
|
+
Component that stores results from the Opener Web Services chain into a SQLite
|
4
|
+
Database and shows them into your browser.
|
5
5
|
|
6
|
-
|
6
|
+
## Confused by some terminology?
|
7
7
|
|
8
|
-
This software is part of a larger collection of natural language processing
|
8
|
+
This software is part of a larger collection of natural language processing
|
9
|
+
tools known as "the OpeNER project". You can find more information about the
|
10
|
+
project at [the OpeNER portal](http://opener-project.github.io). There you can
|
11
|
+
also find references to terms like KAF (an XML standard to represent linguistic
|
12
|
+
annotations in texts), component, cores, scenario's and pipelines.
|
9
13
|
|
10
|
-
Quick Use Example
|
11
|
-
-----------------
|
14
|
+
## Quick Use Example
|
12
15
|
|
13
16
|
Installing the outlet can be done by executing:
|
14
17
|
|
15
18
|
gem install opener-outlet
|
16
19
|
|
17
|
-
Please
|
20
|
+
Please keep in mind that all components in OpeNER take KAF as an input and
|
21
|
+
output KAF by default.
|
18
22
|
|
19
23
|
### Webservices
|
20
24
|
|
@@ -22,28 +26,32 @@ You can launch a webservice by executing:
|
|
22
26
|
|
23
27
|
outlet-server
|
24
28
|
|
25
|
-
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
29
|
+
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
30
|
+
so you can access it at <http://localhost:9292>.
|
26
31
|
|
27
|
-
To launch it on a different port provide the `-p [port-number]` option like
|
32
|
+
To launch it on a different port provide the `-p [port-number]` option like
|
33
|
+
this:
|
28
34
|
|
29
35
|
outlet-server -p 1234
|
30
36
|
|
31
37
|
It then launches at <http://localhost:1234>
|
32
38
|
|
33
|
-
Documentation on the Webservice is provided by surfing to the urls provided
|
34
|
-
|
39
|
+
Documentation on the Webservice is provided by surfing to the urls provided
|
40
|
+
above. For more information on how to launch a webservice run the command with
|
41
|
+
the `--help` option.
|
35
42
|
|
36
43
|
### Daemon
|
37
44
|
|
38
|
-
Last but not least the outlet comes shipped with a daemon that can read jobs
|
45
|
+
Last but not least the outlet comes shipped with a daemon that can read jobs
|
46
|
+
(and write) jobs to and from Amazon SQS queues. For more information type:
|
39
47
|
|
40
48
|
outlet-daemon -h
|
41
49
|
|
50
|
+
## Description of dependencies
|
42
51
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
This component runs best if you run it in an environment suited for OpeNER components. You can find an installation guide and helper tools in the [OpeNER installer](https://github.com/opener-project/opener-installer) and an
|
52
|
+
This component runs best if you run it in an environment suited for OpeNER
|
53
|
+
components. You can find an installation guide and helper tools in the [OpeNER
|
54
|
+
installer](https://github.com/opener-project/opener-installer) and an
|
47
55
|
installation guide on the [Opener Website](http://opener-project.github.io/getting-started/how-to/local-installation.html)
|
48
56
|
|
49
57
|
At least you need the following system setup:
|
@@ -53,28 +61,17 @@ At least you need the following system setup:
|
|
53
61
|
* JRuby 1.7.9 or newer
|
54
62
|
* Ruby 1.9.3 or newer
|
55
63
|
|
56
|
-
|
57
|
-
--------
|
58
|
-
|
59
|
-
The component is a fat wrapper around the actual language technology core. You can find the core technolies in the following repositories:
|
60
|
-
|
61
|
-
* (link to) Related Core readmes
|
62
|
-
|
63
|
-
Where to go from here
|
64
|
-
---------------------
|
64
|
+
## Where to go from here
|
65
65
|
|
66
66
|
* [Check the project website](http://opener-project.github.io)
|
67
67
|
* [Checkout the webservice](http://opener.olery.com/outlet)
|
68
68
|
|
69
|
-
Report problem/Get help
|
70
|
-
-----------------------
|
71
|
-
|
72
|
-
If you encounter problems, please email support@opener-project.eu or leave an issue in the
|
73
|
-
[issue tracker](https://github.com/opener-project/outlet/issues).
|
69
|
+
## Report problem/Get help
|
74
70
|
|
71
|
+
If you encounter problems, please email support@opener-project.eu or leave an
|
72
|
+
issue in the [issue tracker](https://github.com/opener-project/outlet/issues).
|
75
73
|
|
76
|
-
Contributing
|
77
|
-
------------
|
74
|
+
## Contributing
|
78
75
|
|
79
76
|
1. Fork it <http://github.com/opener-project/outlet/fork>
|
80
77
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
data/bin/outlet-daemon
CHANGED
@@ -2,9 +2,9 @@
|
|
2
2
|
|
3
3
|
require 'opener/daemons'
|
4
4
|
|
5
|
-
|
5
|
+
controller = Opener::Daemons::Controller.new(
|
6
|
+
:name => 'opener-outlet',
|
7
|
+
:exec_path => File.expand_path('../../exec/outlet.rb', __FILE__)
|
8
|
+
)
|
6
9
|
|
7
|
-
|
8
|
-
:name => 'outlet',
|
9
|
-
:exec_path => exec_path
|
10
|
-
)
|
10
|
+
controller.run
|
data/bin/outlet-server
CHANGED
@@ -1,8 +1,10 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
|
-
require '
|
3
|
+
require 'opener/webservice'
|
4
4
|
|
5
|
-
|
5
|
+
parser = Opener::Webservice::OptionParser.new(
|
6
|
+
'opener-outlet',
|
7
|
+
File.expand_path('../../config.ru', __FILE__)
|
8
|
+
)
|
6
9
|
|
7
|
-
|
8
|
-
cli.run
|
10
|
+
parser.run
|
data/exec/outlet.rb
CHANGED
@@ -1,9 +1,9 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
3
|
require 'opener/daemons'
|
4
|
+
|
4
5
|
require_relative '../lib/opener/outlet'
|
5
6
|
|
6
|
-
|
7
|
-
daemon = Opener::Daemons::Daemon.new(Opener::Outlet, options)
|
7
|
+
daemon = Opener::Daemons::Daemon.new(Opener::Outlet)
|
8
8
|
|
9
|
-
daemon.start
|
9
|
+
daemon.start
|
data/lib/opener/outlet/output.rb
CHANGED
data/lib/opener/outlet/server.rb
CHANGED
@@ -1,21 +1,10 @@
|
|
1
|
-
require 'sinatra'
|
2
|
-
require 'sinatra/json'
|
3
|
-
require 'nokogiri'
|
4
|
-
require 'stringio'
|
5
|
-
require_relative './visualizer'
|
6
|
-
|
7
|
-
require File.expand_path('../../../../config/database', __FILE__)
|
8
|
-
|
9
1
|
module Opener
|
10
2
|
class Outlet
|
11
|
-
class Server <
|
3
|
+
class Server < Webservice::Server
|
4
|
+
set :views, File.expand_path('../views', __FILE__)
|
12
5
|
|
13
|
-
|
14
|
-
|
15
|
-
output.uuid = params[:request_id]
|
16
|
-
output.text = params[:input]
|
17
|
-
output.save
|
18
|
-
end
|
6
|
+
self.text_processor = Outlet
|
7
|
+
self.accepted_params = [:request_id]
|
19
8
|
|
20
9
|
get '/' do
|
21
10
|
if params[:request_id]
|
@@ -24,99 +13,72 @@ module Opener
|
|
24
13
|
erb :index
|
25
14
|
end
|
26
15
|
end
|
27
|
-
|
16
|
+
|
28
17
|
get '/:request_id.json' do
|
29
18
|
unless params[:request_id] == 'favicon.ico'
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
error_callback = params[:error_callback]
|
42
|
-
|
43
|
-
submit_error(error_callback, error.message) if error_callback
|
44
|
-
|
45
|
-
raise(error)
|
19
|
+
output = Output.find_by_uuid(params[:request_id])
|
20
|
+
|
21
|
+
if output
|
22
|
+
content_type(:json)
|
23
|
+
|
24
|
+
kj = Opener::KafToJson.new
|
25
|
+
json = kj.run(output.text)
|
26
|
+
|
27
|
+
body(json)
|
28
|
+
else
|
29
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
46
30
|
end
|
47
31
|
end
|
48
32
|
end
|
49
33
|
|
50
34
|
get '/:request_id' do
|
51
35
|
unless params[:request_id] == 'favicon.ico'
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
rescue => error
|
61
|
-
error_callback = params[:error_callback]
|
62
|
-
|
63
|
-
submit_error(error_callback, error.message) if error_callback
|
64
|
-
|
65
|
-
raise(error)
|
36
|
+
output = Output.find_by_uuid(params[:request_id])
|
37
|
+
|
38
|
+
if output
|
39
|
+
content_type(:xml)
|
40
|
+
|
41
|
+
body(output.text)
|
42
|
+
else
|
43
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
66
44
|
end
|
67
45
|
end
|
68
46
|
end
|
69
47
|
|
70
48
|
get '/html/:request_id' do
|
71
49
|
unless params[:request_id] == 'favicon.ico'
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
error_callback = params[:error_callback]
|
86
|
-
|
87
|
-
submit_error(error_callback, error.message) if error_callback
|
88
|
-
|
89
|
-
raise(error)
|
50
|
+
output = Output.find_by_uuid(params[:request_id])
|
51
|
+
|
52
|
+
if output
|
53
|
+
output = StringIO.new(output.text)
|
54
|
+
parser = Opener::Kaf::Visualizer::Parser.new(output)
|
55
|
+
doc = parser.parse
|
56
|
+
html = Opener::Kaf::Visualizer::HTMLTextPresenter.new(doc)
|
57
|
+
|
58
|
+
@parsed = html.to_html
|
59
|
+
|
60
|
+
erb :show
|
61
|
+
else
|
62
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
90
63
|
end
|
91
64
|
end
|
92
65
|
end
|
93
|
-
|
66
|
+
|
94
67
|
get '/score/:request_id' do
|
95
68
|
unless params[:request_id] == 'favicon.ico'
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
processor = Opener::Scorer::OutputProcessor.new(output.text)
|
101
|
-
result = processor.process
|
102
|
-
body(result.to_json)
|
103
|
-
else
|
104
|
-
halt(404, "No record found for ID #{params[:request_id]}")
|
105
|
-
end
|
106
|
-
rescue => error
|
107
|
-
error_callback = params[:error_callback]
|
108
|
-
|
109
|
-
submit_error(error_callback, error.message) if error_callback
|
110
|
-
|
111
|
-
raise(error)
|
112
|
-
end
|
113
|
-
end
|
114
|
-
end
|
69
|
+
output = Output.find_by_uuid(params[:request_id])
|
70
|
+
|
71
|
+
if output
|
72
|
+
content_type('text/json')
|
115
73
|
|
116
|
-
|
74
|
+
processor = Opener::Scorer::OutputProcessor.new(output.text)
|
75
|
+
result = processor.process
|
117
76
|
|
118
|
-
|
119
|
-
|
77
|
+
body(result.to_json)
|
78
|
+
else
|
79
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
80
|
+
end
|
81
|
+
end
|
120
82
|
end
|
121
83
|
end # Server
|
122
84
|
end # Outlet
|
data/lib/opener/outlet.rb
CHANGED
@@ -1,14 +1,30 @@
|
|
1
|
+
require 'securerandom'
|
2
|
+
require 'stringio'
|
3
|
+
|
4
|
+
require 'active_record'
|
5
|
+
require 'nokogiri'
|
6
|
+
require 'opener/kaf_to_json'
|
7
|
+
require 'opener/scorer'
|
8
|
+
|
1
9
|
require_relative 'outlet/output'
|
2
10
|
require_relative 'outlet/version'
|
3
11
|
require_relative 'outlet/server'
|
4
|
-
|
5
|
-
|
12
|
+
require_relative 'outlet/visualizer'
|
13
|
+
|
14
|
+
require_relative '../../config/database'
|
6
15
|
|
7
16
|
module Opener
|
8
17
|
class Outlet
|
9
|
-
|
18
|
+
attr_reader :uuid
|
19
|
+
|
20
|
+
def initialize(options = {})
|
21
|
+
@uuid = options[:request_id] || SecureRandom.hex
|
22
|
+
end
|
23
|
+
|
24
|
+
def run(input)
|
10
25
|
output = Output.new(:uuid=>uuid, :text=>input)
|
11
|
-
|
26
|
+
|
27
|
+
output.save!
|
12
28
|
|
13
29
|
return input
|
14
30
|
end
|
data/opener-outlet.gemspec
CHANGED
@@ -26,18 +26,15 @@ Gem::Specification.new do |gem|
|
|
26
26
|
gem.executables = Dir.glob('bin/*').map { |file| File.basename(file) }
|
27
27
|
|
28
28
|
gem.add_dependency 'builder', '~>3.0.0'
|
29
|
-
gem.add_dependency 'sinatra', '~>1.4.2'
|
30
|
-
gem.add_dependency 'sinatra-contrib'
|
31
29
|
gem.add_dependency 'nokogiri'
|
32
|
-
gem.add_dependency 'puma'
|
33
|
-
gem.add_dependency 'opener-daemons'
|
34
|
-
gem.add_dependency 'httpclient'
|
35
|
-
gem.add_dependency 'uuidtools'
|
36
30
|
gem.add_dependency 'jdbc-mysql'
|
37
31
|
gem.add_dependency 'activerecord-jdbcmysql-adapter'
|
38
|
-
gem.add_dependency 'activerecord', '~>3.2'
|
39
|
-
|
40
|
-
gem.add_dependency 'opener-
|
32
|
+
gem.add_dependency 'activerecord', '~> 3.2'
|
33
|
+
|
34
|
+
gem.add_dependency 'opener-daemons', '~> 2.2'
|
35
|
+
gem.add_dependency 'opener-webservice', '~> 2.1'
|
36
|
+
gem.add_dependency 'opener-core', '~> 2.2'
|
37
|
+
|
41
38
|
gem.add_dependency 'opener-kaf2json'
|
42
39
|
gem.add_dependency 'opener-scorer'
|
43
40
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: opener-outlet
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- development@olery.com
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-11-25 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: builder
|
@@ -24,34 +24,6 @@ dependencies:
|
|
24
24
|
version: 3.0.0
|
25
25
|
prerelease: false
|
26
26
|
type: :runtime
|
27
|
-
- !ruby/object:Gem::Dependency
|
28
|
-
name: sinatra
|
29
|
-
version_requirements: !ruby/object:Gem::Requirement
|
30
|
-
requirements:
|
31
|
-
- - ~>
|
32
|
-
- !ruby/object:Gem::Version
|
33
|
-
version: 1.4.2
|
34
|
-
requirement: !ruby/object:Gem::Requirement
|
35
|
-
requirements:
|
36
|
-
- - ~>
|
37
|
-
- !ruby/object:Gem::Version
|
38
|
-
version: 1.4.2
|
39
|
-
prerelease: false
|
40
|
-
type: :runtime
|
41
|
-
- !ruby/object:Gem::Dependency
|
42
|
-
name: sinatra-contrib
|
43
|
-
version_requirements: !ruby/object:Gem::Requirement
|
44
|
-
requirements:
|
45
|
-
- - '>='
|
46
|
-
- !ruby/object:Gem::Version
|
47
|
-
version: '0'
|
48
|
-
requirement: !ruby/object:Gem::Requirement
|
49
|
-
requirements:
|
50
|
-
- - '>='
|
51
|
-
- !ruby/object:Gem::Version
|
52
|
-
version: '0'
|
53
|
-
prerelease: false
|
54
|
-
type: :runtime
|
55
27
|
- !ruby/object:Gem::Dependency
|
56
28
|
name: nokogiri
|
57
29
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -67,49 +39,7 @@ dependencies:
|
|
67
39
|
prerelease: false
|
68
40
|
type: :runtime
|
69
41
|
- !ruby/object:Gem::Dependency
|
70
|
-
name:
|
71
|
-
version_requirements: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - '>='
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '0'
|
76
|
-
requirement: !ruby/object:Gem::Requirement
|
77
|
-
requirements:
|
78
|
-
- - '>='
|
79
|
-
- !ruby/object:Gem::Version
|
80
|
-
version: '0'
|
81
|
-
prerelease: false
|
82
|
-
type: :runtime
|
83
|
-
- !ruby/object:Gem::Dependency
|
84
|
-
name: opener-daemons
|
85
|
-
version_requirements: !ruby/object:Gem::Requirement
|
86
|
-
requirements:
|
87
|
-
- - '>='
|
88
|
-
- !ruby/object:Gem::Version
|
89
|
-
version: '0'
|
90
|
-
requirement: !ruby/object:Gem::Requirement
|
91
|
-
requirements:
|
92
|
-
- - '>='
|
93
|
-
- !ruby/object:Gem::Version
|
94
|
-
version: '0'
|
95
|
-
prerelease: false
|
96
|
-
type: :runtime
|
97
|
-
- !ruby/object:Gem::Dependency
|
98
|
-
name: httpclient
|
99
|
-
version_requirements: !ruby/object:Gem::Requirement
|
100
|
-
requirements:
|
101
|
-
- - '>='
|
102
|
-
- !ruby/object:Gem::Version
|
103
|
-
version: '0'
|
104
|
-
requirement: !ruby/object:Gem::Requirement
|
105
|
-
requirements:
|
106
|
-
- - '>='
|
107
|
-
- !ruby/object:Gem::Version
|
108
|
-
version: '0'
|
109
|
-
prerelease: false
|
110
|
-
type: :runtime
|
111
|
-
- !ruby/object:Gem::Dependency
|
112
|
-
name: uuidtools
|
42
|
+
name: jdbc-mysql
|
113
43
|
version_requirements: !ruby/object:Gem::Requirement
|
114
44
|
requirements:
|
115
45
|
- - '>='
|
@@ -123,7 +53,7 @@ dependencies:
|
|
123
53
|
prerelease: false
|
124
54
|
type: :runtime
|
125
55
|
- !ruby/object:Gem::Dependency
|
126
|
-
name:
|
56
|
+
name: activerecord-jdbcmysql-adapter
|
127
57
|
version_requirements: !ruby/object:Gem::Requirement
|
128
58
|
requirements:
|
129
59
|
- - '>='
|
@@ -137,59 +67,59 @@ dependencies:
|
|
137
67
|
prerelease: false
|
138
68
|
type: :runtime
|
139
69
|
- !ruby/object:Gem::Dependency
|
140
|
-
name: activerecord
|
70
|
+
name: activerecord
|
141
71
|
version_requirements: !ruby/object:Gem::Requirement
|
142
72
|
requirements:
|
143
|
-
- -
|
73
|
+
- - ~>
|
144
74
|
- !ruby/object:Gem::Version
|
145
|
-
version: '
|
75
|
+
version: '3.2'
|
146
76
|
requirement: !ruby/object:Gem::Requirement
|
147
77
|
requirements:
|
148
|
-
- -
|
78
|
+
- - ~>
|
149
79
|
- !ruby/object:Gem::Version
|
150
|
-
version: '
|
80
|
+
version: '3.2'
|
151
81
|
prerelease: false
|
152
82
|
type: :runtime
|
153
83
|
- !ruby/object:Gem::Dependency
|
154
|
-
name:
|
84
|
+
name: opener-daemons
|
155
85
|
version_requirements: !ruby/object:Gem::Requirement
|
156
86
|
requirements:
|
157
87
|
- - ~>
|
158
88
|
- !ruby/object:Gem::Version
|
159
|
-
version: '
|
89
|
+
version: '2.2'
|
160
90
|
requirement: !ruby/object:Gem::Requirement
|
161
91
|
requirements:
|
162
92
|
- - ~>
|
163
93
|
- !ruby/object:Gem::Version
|
164
|
-
version: '
|
94
|
+
version: '2.2'
|
165
95
|
prerelease: false
|
166
96
|
type: :runtime
|
167
97
|
- !ruby/object:Gem::Dependency
|
168
|
-
name:
|
98
|
+
name: opener-webservice
|
169
99
|
version_requirements: !ruby/object:Gem::Requirement
|
170
100
|
requirements:
|
171
101
|
- - ~>
|
172
102
|
- !ruby/object:Gem::Version
|
173
|
-
version: '
|
103
|
+
version: '2.1'
|
174
104
|
requirement: !ruby/object:Gem::Requirement
|
175
105
|
requirements:
|
176
106
|
- - ~>
|
177
107
|
- !ruby/object:Gem::Version
|
178
|
-
version: '
|
108
|
+
version: '2.1'
|
179
109
|
prerelease: false
|
180
110
|
type: :runtime
|
181
111
|
- !ruby/object:Gem::Dependency
|
182
|
-
name: opener-
|
112
|
+
name: opener-core
|
183
113
|
version_requirements: !ruby/object:Gem::Requirement
|
184
114
|
requirements:
|
185
|
-
- -
|
115
|
+
- - ~>
|
186
116
|
- !ruby/object:Gem::Version
|
187
|
-
version: '
|
117
|
+
version: '2.2'
|
188
118
|
requirement: !ruby/object:Gem::Requirement
|
189
119
|
requirements:
|
190
|
-
- -
|
120
|
+
- - ~>
|
191
121
|
- !ruby/object:Gem::Version
|
192
|
-
version: '
|
122
|
+
version: '2.2'
|
193
123
|
prerelease: false
|
194
124
|
type: :runtime
|
195
125
|
- !ruby/object:Gem::Dependency
|
@@ -299,7 +229,6 @@ files:
|
|
299
229
|
- README.md
|
300
230
|
- LICENSE.txt
|
301
231
|
- exec/outlet.rb
|
302
|
-
- visualizer.rb
|
303
232
|
- bin/outlet-server
|
304
233
|
- bin/outlet-daemon
|
305
234
|
homepage: http://opener-project.github.com/
|
data/visualizer.rb
DELETED
@@ -1,278 +0,0 @@
|
|
1
|
-
require 'nokogiri'
|
2
|
-
|
3
|
-
module Opener
|
4
|
-
module Kaf
|
5
|
-
module Visualizer
|
6
|
-
class Parser
|
7
|
-
attr_reader :doc
|
8
|
-
attr_reader :words, :terms, :entities, :properties, :opinions, :document
|
9
|
-
|
10
|
-
def initialize(input_file_handler)
|
11
|
-
@doc = Nokogiri::XML(input_file_handler)
|
12
|
-
end
|
13
|
-
|
14
|
-
def parse
|
15
|
-
@words = parse_words
|
16
|
-
@terms = parse_terms
|
17
|
-
@entities = parse_entities
|
18
|
-
@properties = parse_properties
|
19
|
-
@opinions = parse_opinions
|
20
|
-
@document = KAFDocument.new(
|
21
|
-
:words => words,
|
22
|
-
:terms => terms,
|
23
|
-
:entities => entities,
|
24
|
-
:properties => properties,
|
25
|
-
:opinions => opinions
|
26
|
-
)
|
27
|
-
|
28
|
-
return document
|
29
|
-
end
|
30
|
-
|
31
|
-
def parse_words
|
32
|
-
parse_elements("//wf", Word)
|
33
|
-
end
|
34
|
-
|
35
|
-
def parse_terms
|
36
|
-
# Of course terms should be words here.
|
37
|
-
# Dirty Hack, sufficient for now.
|
38
|
-
parse_elements("//term", Term, :terms=>words)
|
39
|
-
end
|
40
|
-
|
41
|
-
def parse_entities
|
42
|
-
parse_elements("//entity", Entity, :terms=>terms)
|
43
|
-
end
|
44
|
-
|
45
|
-
def parse_properties
|
46
|
-
parse_elements("//property", Property, :terms=>terms)
|
47
|
-
end
|
48
|
-
|
49
|
-
def parse_opinions
|
50
|
-
parse_elements("//opinion", Opinion, :terms=>terms)
|
51
|
-
end
|
52
|
-
|
53
|
-
def parse_elements(xpath, klass, opts={})
|
54
|
-
elements = doc.xpath(xpath)
|
55
|
-
lookup_table = Hash.new
|
56
|
-
elements.each do |element|
|
57
|
-
instance = klass.new(element, opts)
|
58
|
-
lookup_table[instance.id] = instance
|
59
|
-
end
|
60
|
-
lookup_table
|
61
|
-
end
|
62
|
-
|
63
|
-
end
|
64
|
-
|
65
|
-
class KAFNode
|
66
|
-
attr_reader :content, :targets, :tag, :references
|
67
|
-
|
68
|
-
def initialize(tag, references)
|
69
|
-
@references = references
|
70
|
-
@tag = tag
|
71
|
-
|
72
|
-
set_instance_variables
|
73
|
-
set_content
|
74
|
-
set_targets
|
75
|
-
process_subnodes
|
76
|
-
end
|
77
|
-
|
78
|
-
def set_content
|
79
|
-
@content = tag.content
|
80
|
-
end
|
81
|
-
|
82
|
-
def set_instance_variables
|
83
|
-
tag.keys.each do |key|
|
84
|
-
if respond_to?("#{key}=".to_sym)
|
85
|
-
send("#{key}=".to_sym, tag[key])
|
86
|
-
else
|
87
|
-
instance_variable_set("@#{key}", tag[key])
|
88
|
-
end
|
89
|
-
end
|
90
|
-
end
|
91
|
-
|
92
|
-
def set_targets
|
93
|
-
@targets = []
|
94
|
-
tag.css("span target").each do |target|
|
95
|
-
id = target["id"]
|
96
|
-
@targets << references[:terms][id]
|
97
|
-
end
|
98
|
-
end
|
99
|
-
|
100
|
-
def process_subnodes
|
101
|
-
end
|
102
|
-
|
103
|
-
def has_target?(*ids)
|
104
|
-
ids.flatten.each do |id|
|
105
|
-
return true if target_ids.include?(id)
|
106
|
-
end
|
107
|
-
return false
|
108
|
-
end
|
109
|
-
|
110
|
-
def to_s
|
111
|
-
if targets.size > 0
|
112
|
-
return targets.map(&:to_s).join(" ")
|
113
|
-
else
|
114
|
-
return content
|
115
|
-
end
|
116
|
-
end
|
117
|
-
|
118
|
-
def target_ids
|
119
|
-
@targets.map(&:id)
|
120
|
-
end
|
121
|
-
|
122
|
-
end
|
123
|
-
|
124
|
-
class Word < KAFNode
|
125
|
-
attr_reader :wid, :sent, :para, :offset
|
126
|
-
|
127
|
-
def id
|
128
|
-
wid
|
129
|
-
end
|
130
|
-
|
131
|
-
def offset=(offset)
|
132
|
-
@offset = offset.to_i
|
133
|
-
end
|
134
|
-
|
135
|
-
def length
|
136
|
-
content.nil? ? 0 : content.length
|
137
|
-
end
|
138
|
-
|
139
|
-
end
|
140
|
-
|
141
|
-
class Term < KAFNode
|
142
|
-
attr_reader :tid, :type, :lemma, :pos, :morphofeat
|
143
|
-
|
144
|
-
def id
|
145
|
-
tid
|
146
|
-
end
|
147
|
-
end
|
148
|
-
|
149
|
-
class Entity < KAFNode
|
150
|
-
attr_reader :eid, :type
|
151
|
-
|
152
|
-
def id
|
153
|
-
eid
|
154
|
-
end
|
155
|
-
|
156
|
-
def to_s
|
157
|
-
"#{type}: #{targets.map(&:to_s).join(", ")}"
|
158
|
-
end
|
159
|
-
end
|
160
|
-
|
161
|
-
class Property < KAFNode
|
162
|
-
attr_reader :pid, :lemma
|
163
|
-
|
164
|
-
def id
|
165
|
-
pid
|
166
|
-
end
|
167
|
-
|
168
|
-
def to_s
|
169
|
-
"#{lemma}: #{targets.map(&:to_s).join(", ")}"
|
170
|
-
end
|
171
|
-
end
|
172
|
-
|
173
|
-
class Opinion < KAFNode
|
174
|
-
attr_reader :oid, :expression
|
175
|
-
|
176
|
-
def id
|
177
|
-
oid
|
178
|
-
end
|
179
|
-
|
180
|
-
def process_subnodes
|
181
|
-
@expression = tag.xpath("//opinion_expression").first["polarity"].to_sym
|
182
|
-
end
|
183
|
-
|
184
|
-
def to_s
|
185
|
-
"#{expression}: #{targets.map(&:to_s).join(", ")}"
|
186
|
-
end
|
187
|
-
|
188
|
-
end
|
189
|
-
|
190
|
-
class KAFDocument
|
191
|
-
attr_reader :words, :terms, :entities, :properties, :opinions
|
192
|
-
|
193
|
-
def initialize(opts={})
|
194
|
-
@words = opts.fetch(:words)
|
195
|
-
@terms = opts.fetch(:terms)
|
196
|
-
@entities = opts.fetch(:entities)
|
197
|
-
@properties = opts.fetch(:properties)
|
198
|
-
@opinions = opts.fetch(:opinions)
|
199
|
-
end
|
200
|
-
|
201
|
-
end
|
202
|
-
|
203
|
-
class HTMLTextPresenter
|
204
|
-
attr_reader :document
|
205
|
-
def initialize(document)
|
206
|
-
@document = document
|
207
|
-
end
|
208
|
-
|
209
|
-
def to_html
|
210
|
-
offset = 0
|
211
|
-
prev = Struct.new(:offset).new(0)
|
212
|
-
|
213
|
-
builder = Nokogiri::HTML::Builder.new do |html|
|
214
|
-
html.div(:class=>"opener") do
|
215
|
-
html.p do
|
216
|
-
document.words.values.sort_by(&:offset).each do |word|
|
217
|
-
if offset < word.offset
|
218
|
-
spacer = word.offset - offset
|
219
|
-
spacer = Array.new(spacer, " ").join
|
220
|
-
html.span(spacer)
|
221
|
-
end
|
222
|
-
|
223
|
-
terms = targets_for(:terms, word.id)
|
224
|
-
entities = targets_for(:entities, terms)
|
225
|
-
opinions = opinions_for(terms)
|
226
|
-
properties = targets_for(:properties, terms)
|
227
|
-
|
228
|
-
generics = []
|
229
|
-
generics << "term" if terms.size > 0
|
230
|
-
generics << "entity" if entities.size > 0
|
231
|
-
generics << "opinion" if opinions.size > 0
|
232
|
-
generics << "property" if properties.size > 0
|
233
|
-
|
234
|
-
classes = [terms, entities, opinions, properties, generics].flatten.uniq
|
235
|
-
|
236
|
-
word_annotations = classes.join(" ")
|
237
|
-
|
238
|
-
html.span(word.content, :class=>word_annotations, :id=>word.id)
|
239
|
-
offset = word.offset + word.length
|
240
|
-
end
|
241
|
-
end
|
242
|
-
|
243
|
-
[:entities, :opinions, :properties].each do |sym|
|
244
|
-
html.div(:class=>sym) do
|
245
|
-
document.public_send(sym).values.each do |entity|
|
246
|
-
html.div(entity.to_s, :id=>entity.id, :class=>entity.target_ids.join(" "))
|
247
|
-
end
|
248
|
-
end
|
249
|
-
end
|
250
|
-
end
|
251
|
-
end
|
252
|
-
|
253
|
-
builder.to_html
|
254
|
-
end
|
255
|
-
|
256
|
-
def targets_for(variable, *ids)
|
257
|
-
targets = document.public_send(variable.to_sym).values.select do |value|
|
258
|
-
value.has_target?(ids.flatten)
|
259
|
-
end
|
260
|
-
|
261
|
-
targets.map(&:id)
|
262
|
-
end
|
263
|
-
|
264
|
-
def opinions_for(*ids)
|
265
|
-
targets = document.opinions.values.select do |value|
|
266
|
-
value.has_target?(ids.flatten)
|
267
|
-
end
|
268
|
-
|
269
|
-
ids = targets.map(&:id)
|
270
|
-
sentiments = targets.map(&:expression)
|
271
|
-
return ids.concat(sentiments).uniq
|
272
|
-
end
|
273
|
-
|
274
|
-
end
|
275
|
-
end
|
276
|
-
end
|
277
|
-
end
|
278
|
-
|