opener-polarity-tagger 2.2.4 → 2.3.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +71 -1
- data/lib/opener/polarity_tagger.rb +10 -1
- data/lib/opener/polarity_tagger/cli.rb +1 -9
- data/lib/opener/polarity_tagger/error_layer.rb +90 -0
- data/lib/opener/polarity_tagger/version.rb +1 -1
- data/opener-polarity-tagger.gemspec +1 -0
- metadata +77 -63
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 434ab9d00e6c29ba6076e75900cd47b0be4c0d4d
|
4
|
+
data.tar.gz: 4d7b34075f158fb5e6fd1efcd1a85377323e4f0f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 75c5f72d0b0b6797e7619719eb2e320d1477835a6f2acc15d2a4efedf42f3479a3db4e86add4f8c87656d5a14f1d15bda2471b27bee243ae4231171ab71223ed
|
7
|
+
data.tar.gz: 2f68c3cbd2e7d8370746b083e3363c0eeb450efc44f7a35b55330a715f498739aa68015ff0729c1ae51ce3d01454de281d69050802ceecc7f0f788c43bda9e72
|
data/README.md
CHANGED
@@ -15,6 +15,69 @@ The polarity tagger supports the following languages:
|
|
15
15
|
* Italian
|
16
16
|
* Spanish
|
17
17
|
|
18
|
+
##How-to##
|
19
|
+
|
20
|
+
The main script of this tool is a python file, which accepts a set of parameters to determine which features
|
21
|
+
or options we want to use. The language is read from the KAF file, so it doesn't need to be specified as a parameter
|
22
|
+
The program reads a KAF file from the standard input and writes the resulting KAf in the standard output.
|
23
|
+
To see the options you can call to the main script with the -h or --help option
|
24
|
+
````shell
|
25
|
+
$ python core/poltagger-basic-multi.py -h
|
26
|
+
usage: poltagger-basic-multi.py [-h] [--no-time] [--ignore-pos]
|
27
|
+
[--show-lexicons {nl,en,de,es,it,fr}]
|
28
|
+
[--lexicon LEXICON] [--silent] [--version]
|
29
|
+
|
30
|
+
Tags a text with polarities at lemma level
|
31
|
+
|
32
|
+
optional arguments:
|
33
|
+
-h, --help show this help message and exit
|
34
|
+
--no-time For not including timestamp in header
|
35
|
+
--ignore-pos Ignore the pos labels
|
36
|
+
--show-lexicons {nl,en,de,es,it,fr}
|
37
|
+
Show lexicons for the given language and exit
|
38
|
+
--lexicon LEXICON Lexicon identifier, check with --show-lexicons LANG
|
39
|
+
for options
|
40
|
+
--lexicon-path LEXICON The path of the lexicons
|
41
|
+
--silent Turn off debug info
|
42
|
+
--version show program's version number and exit
|
43
|
+
````
|
44
|
+
|
45
|
+
The `--ignore-pos` parameter must be used when want to ignore the part-of-speech information assigned to the lemmas, and we want to assign polarities
|
46
|
+
just to the lemmas, not considering the POS tag. This could be useful when the information provided by the pos-tagger is not accurate or the pos-tagging
|
47
|
+
has not been processed.
|
48
|
+
|
49
|
+
The main options are those concerning with the usage of different lexicons. The lexicons are provided by the
|
50
|
+
VU-sentiment-lexicon library (https://github.com/opener-project/VU-sentiment-lexicon), which needs to be installed.
|
51
|
+
You can see what the lexicons available for a given language are by calling to the program with the option --show-lexicons LANG,
|
52
|
+
for instance:
|
53
|
+
````shell
|
54
|
+
core/poltagger-basic-multi.py --show-lexicons nl
|
55
|
+
|
56
|
+
##############################
|
57
|
+
Available lexicons for nl
|
58
|
+
Identifier: "hotel" (Default)
|
59
|
+
Desc: Hotel domain lexicon for Dutch
|
60
|
+
Res: VUA_olery_lexicon_nl_lmf
|
61
|
+
File: /Users/ruben/python_envs/python2.7/lib/python2.7/VUSentimentLexicon/NL-lexicon/Sentiment-Dutch-HotelDomain.xml
|
62
|
+
|
63
|
+
Identifier:"general"
|
64
|
+
Desc: General lexicon for Dutch
|
65
|
+
Res: VUA_olery_lexicon_nl_lmf
|
66
|
+
File: /Users/ruben/python_envs/python2.7/lib/python2.7/VUSentimentLexicon/NL-lexicon/Sentiment-Dutch-general.xml
|
67
|
+
|
68
|
+
##############################
|
69
|
+
````
|
70
|
+
|
71
|
+
Then you can use the lexicon identifiers to select the proper lexicon, with the option --lexicon
|
72
|
+
````shell
|
73
|
+
cat my_input.nl.kaf | core/poltagger-basic-multi.py --lexicon general
|
74
|
+
````
|
75
|
+
|
76
|
+
This command will call to the polarity tagger using the general lexicon for Dutch. The lexicon identifiers are unique only per language.
|
77
|
+
If the lexicon id is not specified(you skip the --lexicon option), or you provide a wrong identifier, the default lexicon will be loaded.
|
78
|
+
If there is no lexicon marked as default in the --show-lexicon options, the first one in the list will be used. Check the VU-sentiment-lexicon
|
79
|
+
for further information about how to manage lexicons and add new ones
|
80
|
+
|
18
81
|
### Confused by some terminology?
|
19
82
|
|
20
83
|
This software is part of a larger collection of natural language processing tools known as "the OpeNER project". You can find more information about the project at [the OpeNER portal](http://opener-project.github.io). There you can also find references to terms like KAF (an XML standard to represent linguistic annotations in texts), component, cores, scenario's and pipelines.
|
@@ -51,6 +114,13 @@ This will output:
|
|
51
114
|
</term>
|
52
115
|
```
|
53
116
|
|
117
|
+
##Requirements##
|
118
|
+
|
119
|
+
* Python 2.7.0 or newer
|
120
|
+
* Ruby 1.9.2 or newer
|
121
|
+
* pip
|
122
|
+
* libxml2
|
123
|
+
|
54
124
|
### Webservices
|
55
125
|
|
56
126
|
You can launch a webservice by executing:
|
@@ -106,7 +176,7 @@ The component is a fat wrapper around the actual language technology core. You c
|
|
106
176
|
Where to go from here
|
107
177
|
---------------------
|
108
178
|
|
109
|
-
* [Check the project
|
179
|
+
* [Check the project website](http://opener-project.github.io)
|
110
180
|
* [Checkout the webservice](http://opener.olery.com/polarity-tagger)
|
111
181
|
|
112
182
|
Report problem/Get help
|
@@ -2,6 +2,7 @@ require 'open3'
|
|
2
2
|
|
3
3
|
require_relative 'polarity_tagger/version'
|
4
4
|
require_relative 'polarity_tagger/cli'
|
5
|
+
require_relative 'polarity_tagger/error_layer'
|
5
6
|
|
6
7
|
module Opener
|
7
8
|
##
|
@@ -36,6 +37,8 @@ module Opener
|
|
36
37
|
def lexicon_path
|
37
38
|
if path = options[:resource_path]
|
38
39
|
return "--lexicon-path #{path}"
|
40
|
+
elsif path = ENV['POLARITY_LEXICON_PATH']
|
41
|
+
return "--lexicon-path #{path}"
|
39
42
|
else
|
40
43
|
return nil
|
41
44
|
end
|
@@ -49,7 +52,13 @@ module Opener
|
|
49
52
|
# @return [Array]
|
50
53
|
#
|
51
54
|
def run(input)
|
52
|
-
|
55
|
+
begin
|
56
|
+
stdout, stderr, process = capture(input)
|
57
|
+
raise stderr unless process.success?
|
58
|
+
return stdout
|
59
|
+
rescue Exception => error
|
60
|
+
return ErrorLayer.new(input, error.message, self.class).add
|
61
|
+
end
|
53
62
|
end
|
54
63
|
|
55
64
|
protected
|
@@ -55,15 +55,7 @@ module Opener
|
|
55
55
|
|
56
56
|
stdout, stderr, process = tagger.run(input)
|
57
57
|
|
58
|
-
|
59
|
-
puts stdout
|
60
|
-
|
61
|
-
if options[:logging] and !stderr.empty?
|
62
|
-
STDERR.puts(stderr)
|
63
|
-
end
|
64
|
-
else
|
65
|
-
abort stderr
|
66
|
-
end
|
58
|
+
puts stdout
|
67
59
|
end
|
68
60
|
|
69
61
|
private
|
@@ -0,0 +1,90 @@
|
|
1
|
+
require 'nokogiri'
|
2
|
+
|
3
|
+
module Opener
|
4
|
+
class PolarityTagger
|
5
|
+
##
|
6
|
+
# Add Error Layer to KAF file instead of throwing an error.
|
7
|
+
#
|
8
|
+
class ErrorLayer
|
9
|
+
attr_accessor :input, :document, :error, :klass
|
10
|
+
|
11
|
+
def initialize(input, error, klass)
|
12
|
+
@input = input.to_s
|
13
|
+
# Make sure there is always a document, even if it is empty.
|
14
|
+
@document = Nokogiri::XML(input) rescue Nokogiri::XML(nil)
|
15
|
+
@error = error
|
16
|
+
@klass = klass
|
17
|
+
end
|
18
|
+
|
19
|
+
def add
|
20
|
+
if is_xml?
|
21
|
+
unless has_errors_layer?
|
22
|
+
add_errors_layer
|
23
|
+
end
|
24
|
+
else
|
25
|
+
add_root
|
26
|
+
add_text
|
27
|
+
add_errors_layer
|
28
|
+
end
|
29
|
+
add_error
|
30
|
+
|
31
|
+
xml = !!document.encoding ? document.to_xml : document.to_xml(:encoding => "UTF-8")
|
32
|
+
|
33
|
+
return xml
|
34
|
+
end
|
35
|
+
|
36
|
+
##
|
37
|
+
# Check if the document is a valid XML file.
|
38
|
+
#
|
39
|
+
def is_xml?
|
40
|
+
!!document.root
|
41
|
+
end
|
42
|
+
|
43
|
+
##
|
44
|
+
# Add root element to the XML file.
|
45
|
+
#
|
46
|
+
def add_root
|
47
|
+
root = Nokogiri::XML::Node.new "KAF", document
|
48
|
+
document.add_child(root)
|
49
|
+
end
|
50
|
+
|
51
|
+
##
|
52
|
+
# Check if the document already has an errors layer.
|
53
|
+
#
|
54
|
+
def has_errors_layer?
|
55
|
+
!!document.at('errors')
|
56
|
+
end
|
57
|
+
|
58
|
+
##
|
59
|
+
# Add errors element to the XML file.
|
60
|
+
#
|
61
|
+
def add_errors_layer
|
62
|
+
node = Nokogiri::XML::Node.new "errors", document
|
63
|
+
document.root.add_child(node)
|
64
|
+
end
|
65
|
+
|
66
|
+
##
|
67
|
+
# Add the text file incase it is not a valid XML document. More
|
68
|
+
# info for debugging.
|
69
|
+
#
|
70
|
+
def add_text
|
71
|
+
node = Nokogiri::XML::Node.new "raw", document
|
72
|
+
node.inner_html = input
|
73
|
+
document.root.add_child(node)
|
74
|
+
|
75
|
+
end
|
76
|
+
|
77
|
+
##
|
78
|
+
# Add the actual error to the errors layer.
|
79
|
+
#
|
80
|
+
def add_error
|
81
|
+
node = document.at('errors')
|
82
|
+
error_node = Nokogiri::XML::Node.new "error", node
|
83
|
+
error_node['class'] = "#{klass.to_s} #{klass::VERSION}"
|
84
|
+
error_node.inner_html = error
|
85
|
+
node.add_child(error_node)
|
86
|
+
end
|
87
|
+
|
88
|
+
end # ErrorLayer
|
89
|
+
end # PolarityTagger
|
90
|
+
end # Opener
|
@@ -32,6 +32,7 @@ Gem::Specification.new do |gem|
|
|
32
32
|
gem.add_dependency 'opener-daemons'
|
33
33
|
gem.add_dependency 'opener-webservice'
|
34
34
|
gem.add_dependency 'opener-core', ['>= 0.1.1']
|
35
|
+
gem.add_dependency 'nokogiri'
|
35
36
|
|
36
37
|
gem.add_development_dependency 'rspec'
|
37
38
|
gem.add_development_dependency 'cucumber'
|
metadata
CHANGED
@@ -1,161 +1,175 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: opener-polarity-tagger
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.3.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- development@olery.com
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-06-11 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: opener-build-tools
|
15
|
-
|
15
|
+
version_requirements: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- -
|
17
|
+
- - '>='
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: 1.0.1
|
20
|
-
|
21
|
-
prerelease: false
|
22
|
-
version_requirements: !ruby/object:Gem::Requirement
|
20
|
+
requirement: !ruby/object:Gem::Requirement
|
23
21
|
requirements:
|
24
|
-
- -
|
22
|
+
- - '>='
|
25
23
|
- !ruby/object:Gem::Version
|
26
24
|
version: 1.0.1
|
25
|
+
prerelease: false
|
26
|
+
type: :runtime
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: rake
|
29
|
+
version_requirements: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - '>='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
29
34
|
requirement: !ruby/object:Gem::Requirement
|
30
35
|
requirements:
|
31
|
-
- -
|
36
|
+
- - '>='
|
32
37
|
- !ruby/object:Gem::Version
|
33
38
|
version: '0'
|
34
|
-
type: :runtime
|
35
39
|
prerelease: false
|
40
|
+
type: :runtime
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: sinatra
|
36
43
|
version_requirements: !ruby/object:Gem::Requirement
|
37
44
|
requirements:
|
38
|
-
- -
|
45
|
+
- - '>='
|
39
46
|
- !ruby/object:Gem::Version
|
40
47
|
version: '0'
|
41
|
-
- !ruby/object:Gem::Dependency
|
42
|
-
name: sinatra
|
43
48
|
requirement: !ruby/object:Gem::Requirement
|
44
49
|
requirements:
|
45
|
-
- -
|
50
|
+
- - '>='
|
46
51
|
- !ruby/object:Gem::Version
|
47
52
|
version: '0'
|
48
|
-
type: :runtime
|
49
53
|
prerelease: false
|
54
|
+
type: :runtime
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: httpclient
|
50
57
|
version_requirements: !ruby/object:Gem::Requirement
|
51
58
|
requirements:
|
52
|
-
- -
|
59
|
+
- - '>='
|
53
60
|
- !ruby/object:Gem::Version
|
54
61
|
version: '0'
|
55
|
-
- !ruby/object:Gem::Dependency
|
56
|
-
name: httpclient
|
57
62
|
requirement: !ruby/object:Gem::Requirement
|
58
63
|
requirements:
|
59
|
-
- -
|
64
|
+
- - '>='
|
60
65
|
- !ruby/object:Gem::Version
|
61
66
|
version: '0'
|
62
|
-
type: :runtime
|
63
67
|
prerelease: false
|
68
|
+
type: :runtime
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: puma
|
64
71
|
version_requirements: !ruby/object:Gem::Requirement
|
65
72
|
requirements:
|
66
|
-
- -
|
73
|
+
- - '>='
|
67
74
|
- !ruby/object:Gem::Version
|
68
75
|
version: '0'
|
69
|
-
- !ruby/object:Gem::Dependency
|
70
|
-
name: puma
|
71
76
|
requirement: !ruby/object:Gem::Requirement
|
72
77
|
requirements:
|
73
|
-
- -
|
78
|
+
- - '>='
|
74
79
|
- !ruby/object:Gem::Version
|
75
80
|
version: '0'
|
76
|
-
type: :runtime
|
77
81
|
prerelease: false
|
82
|
+
type: :runtime
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: opener-daemons
|
78
85
|
version_requirements: !ruby/object:Gem::Requirement
|
79
86
|
requirements:
|
80
|
-
- -
|
87
|
+
- - '>='
|
81
88
|
- !ruby/object:Gem::Version
|
82
89
|
version: '0'
|
83
|
-
- !ruby/object:Gem::Dependency
|
84
|
-
name: opener-daemons
|
85
90
|
requirement: !ruby/object:Gem::Requirement
|
86
91
|
requirements:
|
87
|
-
- -
|
92
|
+
- - '>='
|
88
93
|
- !ruby/object:Gem::Version
|
89
94
|
version: '0'
|
90
|
-
type: :runtime
|
91
95
|
prerelease: false
|
96
|
+
type: :runtime
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: opener-webservice
|
92
99
|
version_requirements: !ruby/object:Gem::Requirement
|
93
100
|
requirements:
|
94
|
-
- -
|
101
|
+
- - '>='
|
95
102
|
- !ruby/object:Gem::Version
|
96
103
|
version: '0'
|
97
|
-
- !ruby/object:Gem::Dependency
|
98
|
-
name: opener-webservice
|
99
104
|
requirement: !ruby/object:Gem::Requirement
|
100
105
|
requirements:
|
101
|
-
- -
|
106
|
+
- - '>='
|
102
107
|
- !ruby/object:Gem::Version
|
103
108
|
version: '0'
|
104
|
-
type: :runtime
|
105
109
|
prerelease: false
|
110
|
+
type: :runtime
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: opener-core
|
106
113
|
version_requirements: !ruby/object:Gem::Requirement
|
107
114
|
requirements:
|
108
|
-
- -
|
115
|
+
- - '>='
|
109
116
|
- !ruby/object:Gem::Version
|
110
|
-
version:
|
111
|
-
- !ruby/object:Gem::Dependency
|
112
|
-
name: opener-core
|
117
|
+
version: 0.1.1
|
113
118
|
requirement: !ruby/object:Gem::Requirement
|
114
119
|
requirements:
|
115
|
-
- -
|
120
|
+
- - '>='
|
116
121
|
- !ruby/object:Gem::Version
|
117
122
|
version: 0.1.1
|
118
|
-
type: :runtime
|
119
123
|
prerelease: false
|
124
|
+
type: :runtime
|
125
|
+
- !ruby/object:Gem::Dependency
|
126
|
+
name: nokogiri
|
120
127
|
version_requirements: !ruby/object:Gem::Requirement
|
121
128
|
requirements:
|
122
|
-
- -
|
129
|
+
- - '>='
|
123
130
|
- !ruby/object:Gem::Version
|
124
|
-
version: 0
|
125
|
-
- !ruby/object:Gem::Dependency
|
126
|
-
name: rspec
|
131
|
+
version: '0'
|
127
132
|
requirement: !ruby/object:Gem::Requirement
|
128
133
|
requirements:
|
129
|
-
- -
|
134
|
+
- - '>='
|
130
135
|
- !ruby/object:Gem::Version
|
131
136
|
version: '0'
|
132
|
-
type: :development
|
133
137
|
prerelease: false
|
138
|
+
type: :runtime
|
139
|
+
- !ruby/object:Gem::Dependency
|
140
|
+
name: rspec
|
134
141
|
version_requirements: !ruby/object:Gem::Requirement
|
135
142
|
requirements:
|
136
|
-
- -
|
143
|
+
- - '>='
|
137
144
|
- !ruby/object:Gem::Version
|
138
145
|
version: '0'
|
139
|
-
- !ruby/object:Gem::Dependency
|
140
|
-
name: cucumber
|
141
146
|
requirement: !ruby/object:Gem::Requirement
|
142
147
|
requirements:
|
143
|
-
- -
|
148
|
+
- - '>='
|
144
149
|
- !ruby/object:Gem::Version
|
145
150
|
version: '0'
|
146
|
-
type: :development
|
147
151
|
prerelease: false
|
152
|
+
type: :development
|
153
|
+
- !ruby/object:Gem::Dependency
|
154
|
+
name: cucumber
|
148
155
|
version_requirements: !ruby/object:Gem::Requirement
|
149
156
|
requirements:
|
150
|
-
- -
|
157
|
+
- - '>='
|
151
158
|
- !ruby/object:Gem::Version
|
152
159
|
version: '0'
|
160
|
+
requirement: !ruby/object:Gem::Requirement
|
161
|
+
requirements:
|
162
|
+
- - '>='
|
163
|
+
- !ruby/object:Gem::Version
|
164
|
+
version: '0'
|
165
|
+
prerelease: false
|
166
|
+
type: :development
|
153
167
|
description: Polarity tagger for various languages.
|
154
|
-
email:
|
168
|
+
email:
|
155
169
|
executables:
|
156
|
-
- polarity-tagger-server
|
157
|
-
- polarity-tagger-daemon
|
158
170
|
- polarity-tagger
|
171
|
+
- polarity-tagger-daemon
|
172
|
+
- polarity-tagger-server
|
159
173
|
extensions:
|
160
174
|
- ext/hack/Rakefile
|
161
175
|
extra_rdoc_files: []
|
@@ -171,6 +185,7 @@ files:
|
|
171
185
|
- ext/hack/support.rb
|
172
186
|
- lib/opener/polarity_tagger.rb
|
173
187
|
- lib/opener/polarity_tagger/cli.rb
|
188
|
+
- lib/opener/polarity_tagger/error_layer.rb
|
174
189
|
- lib/opener/polarity_tagger/public/markdown.css
|
175
190
|
- lib/opener/polarity_tagger/server.rb
|
176
191
|
- lib/opener/polarity_tagger/version.rb
|
@@ -181,25 +196,24 @@ files:
|
|
181
196
|
homepage: http://opener-project.github.com/
|
182
197
|
licenses: []
|
183
198
|
metadata: {}
|
184
|
-
post_install_message:
|
199
|
+
post_install_message:
|
185
200
|
rdoc_options: []
|
186
201
|
require_paths:
|
187
202
|
- lib
|
188
203
|
required_ruby_version: !ruby/object:Gem::Requirement
|
189
204
|
requirements:
|
190
|
-
- -
|
205
|
+
- - '>='
|
191
206
|
- !ruby/object:Gem::Version
|
192
207
|
version: 1.9.2
|
193
208
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
194
209
|
requirements:
|
195
|
-
- -
|
210
|
+
- - '>='
|
196
211
|
- !ruby/object:Gem::Version
|
197
212
|
version: '0'
|
198
213
|
requirements: []
|
199
|
-
rubyforge_project:
|
214
|
+
rubyforge_project:
|
200
215
|
rubygems_version: 2.2.2
|
201
|
-
signing_key:
|
216
|
+
signing_key:
|
202
217
|
specification_version: 4
|
203
218
|
summary: Polarity tagger for various languages.
|
204
219
|
test_files: []
|
205
|
-
has_rdoc:
|