opener-ned 2.0.4 → 2.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +34 -17
- data/bin/ned-server +6 -4
- data/core/target/ehu-ned-1.0.jar +0 -0
- data/lib/opener/ned/version.rb +1 -1
- data/lib/opener/ned.rb +14 -20
- data/opener-ned.gemspec +2 -6
- metadata +4 -50
- data/bin/ned-daemon +0 -10
- data/exec/ned.rb +0 -9
- data/lib/opener/ned/error_layer.rb +0 -89
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a32557f4c5975993619a7c34fd8445c4a34c2772
|
4
|
+
data.tar.gz: a1bf198c6c521deb21c295b6570e772ff2444cb6
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 39f0013bf01f194047ecdd5921bca6618201045dd8197635fb2a34a023cbe4bf7a2d5a4365b338e9052b05c7fc8b6e57b7b6337b96105abca3231f5469b8adf3
|
7
|
+
data.tar.gz: 56facef0fb80a8b52e50b7337ac82c3901f2a8938edce8a4f662ef593033658c8e25a04f7a39327f9fee65629b81eabdef8506500818e2f9455dd68de695d404
|
data/README.md
CHANGED
@@ -1,12 +1,17 @@
|
|
1
1
|
Introduction
|
2
2
|
------------
|
3
3
|
|
4
|
-
This repository contains a Named Entity Disambiguation tool that queries a
|
4
|
+
This repository contains a Named Entity Disambiguation tool that queries a
|
5
|
+
DBpedia spotlight server. The client takes KAF as input (containing
|
6
|
+
`<entities>` nodes).
|
5
7
|
|
6
8
|
### Confused by some terminology?
|
7
9
|
|
8
|
-
This software is part of a larger collection of natural language processing
|
9
|
-
|
10
|
+
This software is part of a larger collection of natural language processing
|
11
|
+
tools known as "the OpeNER project". You can find more information about the
|
12
|
+
project at (the OpeNER portal)[http://opener-project.github.io]. There you can
|
13
|
+
also find references to terms like KAF (an XML standard to represent linguistic
|
14
|
+
annotations in texts), component, cores, scenario's and pipelines.
|
10
15
|
|
11
16
|
Quick Use Example
|
12
17
|
-----------------
|
@@ -15,12 +20,13 @@ Installing the ned can be done by executing:
|
|
15
20
|
|
16
21
|
gem install opener-ned
|
17
22
|
|
18
|
-
Please bare in mind that all components in OpeNER take KAF as an input and
|
23
|
+
Please bare in mind that all components in OpeNER take KAF as an input and
|
24
|
+
output KAF by default.
|
19
25
|
|
20
26
|
|
21
27
|
### Command line interface
|
22
28
|
|
23
|
-
The NED client connects to the
|
29
|
+
The NED client connects to the (DBPedia spotlight demo servers)[http://dbpedia-spotlight.github.io/demo/].
|
24
30
|
|
25
31
|
A simple example:
|
26
32
|
|
@@ -50,20 +56,26 @@ You can launch a language identification webservice by executing:
|
|
50
56
|
|
51
57
|
ned-server
|
52
58
|
|
53
|
-
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
59
|
+
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
60
|
+
so you can access it at <http://localhost:9292>.
|
54
61
|
|
55
|
-
To launch it on a different port provide the `-p [port-number]` option like
|
62
|
+
To launch it on a different port provide the `-p [port-number]` option like
|
63
|
+
this:
|
56
64
|
|
57
65
|
ned-server -p 1234
|
58
66
|
|
59
67
|
It then launches at <http://localhost:1234>
|
60
68
|
|
61
|
-
Documentation on the Webservice is provided by surfing to the urls provided
|
69
|
+
Documentation on the Webservice is provided by surfing to the urls provided
|
70
|
+
above. For more information on how to launch a webservice run the command with
|
71
|
+
the ```-h``` option.
|
62
72
|
|
63
73
|
|
64
74
|
### Daemon
|
65
75
|
|
66
|
-
Last but not least the NED comes shipped with a daemon that
|
76
|
+
Last but not least the NED comes shipped with a daemon that
|
77
|
+
can read jobs (and write) jobs to and from Amazon SQS queues. For more
|
78
|
+
information type:
|
67
79
|
|
68
80
|
ned-daemon -h
|
69
81
|
|
@@ -71,8 +83,11 @@ Last but not least the NED comes shipped with a daemon that can read jobs (and w
|
|
71
83
|
Description of dependencies
|
72
84
|
---------------------------
|
73
85
|
|
74
|
-
This component runs best if you run it in an environment suited for OpeNER
|
75
|
-
|
86
|
+
This component runs best if you run it in an environment suited for OpeNER
|
87
|
+
components. You can find an installation guide and helper tools in the (OpeNER
|
88
|
+
installer)[https://github.com/opener-project/opener-installer] and (an
|
89
|
+
installation guide on the Opener
|
90
|
+
Website)[http://opener-project.github.io/getting-started/how-to/local-installation.html]
|
76
91
|
|
77
92
|
At least you need the following system setup:
|
78
93
|
|
@@ -87,7 +102,9 @@ At least you need the following system setup:
|
|
87
102
|
|
88
103
|
### Structure
|
89
104
|
|
90
|
-
This repository comes in two parts: a collection of Java source files and Ruby
|
105
|
+
This repository comes in two parts: a collection of Java source files and Ruby
|
106
|
+
source files. The Java code can be found in the `core/` directory, everything
|
107
|
+
else will be Ruby source code.
|
91
108
|
|
92
109
|
Language Extension
|
93
110
|
------------------
|
@@ -97,20 +114,20 @@ Language Extension
|
|
97
114
|
Where to go from here
|
98
115
|
---------------------
|
99
116
|
|
100
|
-
*
|
101
|
-
*
|
117
|
+
* Check (the project websitere)[http://opener-project.github.io]
|
118
|
+
* (Checkout the webservice)[http://opener.olery.com/ned]
|
102
119
|
|
103
120
|
Report problem/Get help
|
104
121
|
-----------------------
|
105
122
|
|
106
|
-
If you encounter problems, please email
|
107
|
-
|
123
|
+
If you encounter problems, please email support@opener-project.eu or leave an
|
124
|
+
issue in the (issue tracker)[https://github.com/opener-project/ned/issues].
|
108
125
|
|
109
126
|
|
110
127
|
Contributing
|
111
128
|
------------
|
112
129
|
|
113
|
-
1. Fork it
|
130
|
+
1. Fork it ( http://github.com/opener-project/ned/fork )
|
114
131
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
115
132
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
116
133
|
4. Push to the branch (`git push origin my-new-feature`)
|
data/bin/ned-server
CHANGED
@@ -1,8 +1,10 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
|
-
|
3
|
+
require_relative '../lib/opener/ned/server'
|
4
4
|
|
5
|
-
|
5
|
+
# Without calling `Rack::Server#options` manually the CLI arguments will never
|
6
|
+
# be passed, thus the application can't be specified as a constructor argument.
|
7
|
+
server = Rack::Server.new
|
8
|
+
server.options[:config] = File.expand_path('../../config.ru', __FILE__)
|
6
9
|
|
7
|
-
|
8
|
-
cli.run
|
10
|
+
server.start
|
data/core/target/ehu-ned-1.0.jar
CHANGED
Binary file
|
data/lib/opener/ned/version.rb
CHANGED
data/lib/opener/ned.rb
CHANGED
@@ -7,7 +7,6 @@ require File.expand_path('../../../core/target/ehu-ned-1.0.jar', __FILE__)
|
|
7
7
|
|
8
8
|
require_relative 'ned/version'
|
9
9
|
require_relative 'ned/cli'
|
10
|
-
require_relative 'ned/error_layer'
|
11
10
|
|
12
11
|
import 'java.io.InputStreamReader'
|
13
12
|
|
@@ -61,30 +60,25 @@ module Opener
|
|
61
60
|
end
|
62
61
|
|
63
62
|
def run(input)
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
end
|
63
|
+
if !input or input.strip.empty?
|
64
|
+
raise ArgumentError, 'No input specified'
|
65
|
+
end
|
68
66
|
|
69
|
-
|
67
|
+
language = language_from_kaf(input)
|
70
68
|
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
69
|
+
input_io = StringIO.new(input)
|
70
|
+
reader = InputStreamReader.new(input_io.to_inputstream)
|
71
|
+
document = KAFDocument.create_from_stream(reader)
|
72
|
+
annotator = Java::ehu.ned.Annotate.new
|
75
73
|
|
76
|
-
|
74
|
+
endpoint = @options.fetch(:endpoint, uri_for_language(language))
|
77
75
|
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
76
|
+
annotator.disambiguateNEsToKAF(
|
77
|
+
document,
|
78
|
+
endpoint.to_s
|
79
|
+
)
|
82
80
|
|
83
|
-
|
84
|
-
|
85
|
-
rescue Exception => error
|
86
|
-
return ErrorLayer.new(input, error.message, self.class).add
|
87
|
-
end
|
81
|
+
return document.to_string
|
88
82
|
end
|
89
83
|
|
90
84
|
private
|
data/opener-ned.gemspec
CHANGED
@@ -15,8 +15,7 @@ Gem::Specification.new do |gem|
|
|
15
15
|
'lib/**/*',
|
16
16
|
'*.gemspec',
|
17
17
|
'config.ru',
|
18
|
-
'README.md'
|
19
|
-
'exec/**/*'
|
18
|
+
'README.md'
|
20
19
|
]).select { |file| File.file?(file) }
|
21
20
|
|
22
21
|
gem.executables = Dir.glob('bin/*').map { |file| File.basename(file) }
|
@@ -25,10 +24,7 @@ Gem::Specification.new do |gem|
|
|
25
24
|
gem.add_dependency 'httpclient'
|
26
25
|
gem.add_dependency 'opener-webservice'
|
27
26
|
gem.add_dependency 'nokogiri'
|
28
|
-
gem.add_dependency 'puma'
|
29
|
-
gem.add_dependency 'opener-daemons'
|
30
|
-
gem.add_dependency 'opener-core', ['>= 0.1.2']
|
31
27
|
|
28
|
+
gem.add_development_dependency 'opener-build-tools'
|
32
29
|
gem.add_development_dependency 'rake'
|
33
|
-
gem.add_development_dependency 'cliver'
|
34
30
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: opener-ned
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- development@olery.com
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-05-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: sinatra
|
@@ -67,49 +67,7 @@ dependencies:
|
|
67
67
|
prerelease: false
|
68
68
|
type: :runtime
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
|
-
name:
|
71
|
-
version_requirements: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - '>='
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '0'
|
76
|
-
requirement: !ruby/object:Gem::Requirement
|
77
|
-
requirements:
|
78
|
-
- - '>='
|
79
|
-
- !ruby/object:Gem::Version
|
80
|
-
version: '0'
|
81
|
-
prerelease: false
|
82
|
-
type: :runtime
|
83
|
-
- !ruby/object:Gem::Dependency
|
84
|
-
name: opener-daemons
|
85
|
-
version_requirements: !ruby/object:Gem::Requirement
|
86
|
-
requirements:
|
87
|
-
- - '>='
|
88
|
-
- !ruby/object:Gem::Version
|
89
|
-
version: '0'
|
90
|
-
requirement: !ruby/object:Gem::Requirement
|
91
|
-
requirements:
|
92
|
-
- - '>='
|
93
|
-
- !ruby/object:Gem::Version
|
94
|
-
version: '0'
|
95
|
-
prerelease: false
|
96
|
-
type: :runtime
|
97
|
-
- !ruby/object:Gem::Dependency
|
98
|
-
name: opener-core
|
99
|
-
version_requirements: !ruby/object:Gem::Requirement
|
100
|
-
requirements:
|
101
|
-
- - '>='
|
102
|
-
- !ruby/object:Gem::Version
|
103
|
-
version: 0.1.2
|
104
|
-
requirement: !ruby/object:Gem::Requirement
|
105
|
-
requirements:
|
106
|
-
- - '>='
|
107
|
-
- !ruby/object:Gem::Version
|
108
|
-
version: 0.1.2
|
109
|
-
prerelease: false
|
110
|
-
type: :runtime
|
111
|
-
- !ruby/object:Gem::Dependency
|
112
|
-
name: rake
|
70
|
+
name: opener-build-tools
|
113
71
|
version_requirements: !ruby/object:Gem::Requirement
|
114
72
|
requirements:
|
115
73
|
- - '>='
|
@@ -123,7 +81,7 @@ dependencies:
|
|
123
81
|
prerelease: false
|
124
82
|
type: :development
|
125
83
|
- !ruby/object:Gem::Dependency
|
126
|
-
name:
|
84
|
+
name: rake
|
127
85
|
version_requirements: !ruby/object:Gem::Requirement
|
128
86
|
requirements:
|
129
87
|
- - '>='
|
@@ -140,21 +98,17 @@ description: NED client using DBpedia
|
|
140
98
|
email:
|
141
99
|
executables:
|
142
100
|
- ned
|
143
|
-
- ned-daemon
|
144
101
|
- ned-server
|
145
102
|
extensions: []
|
146
103
|
extra_rdoc_files: []
|
147
104
|
files:
|
148
105
|
- README.md
|
149
106
|
- bin/ned
|
150
|
-
- bin/ned-daemon
|
151
107
|
- bin/ned-server
|
152
108
|
- config.ru
|
153
109
|
- core/target/ehu-ned-1.0.jar
|
154
|
-
- exec/ned.rb
|
155
110
|
- lib/opener/ned.rb
|
156
111
|
- lib/opener/ned/cli.rb
|
157
|
-
- lib/opener/ned/error_layer.rb
|
158
112
|
- lib/opener/ned/public/markdown.css
|
159
113
|
- lib/opener/ned/server.rb
|
160
114
|
- lib/opener/ned/version.rb
|
data/bin/ned-daemon
DELETED
data/exec/ned.rb
DELETED
@@ -1,89 +0,0 @@
|
|
1
|
-
module Opener
|
2
|
-
class Ned
|
3
|
-
##
|
4
|
-
# Add Error Layer to KAF file instead of throwing an error.
|
5
|
-
#
|
6
|
-
class ErrorLayer
|
7
|
-
attr_accessor :input, :document, :error, :klass
|
8
|
-
|
9
|
-
def initialize(input, error, klass)
|
10
|
-
@input = input.to_s
|
11
|
-
# Make sure there is always a document, even if it is empty.
|
12
|
-
@document = Nokogiri::XML(input) rescue Nokogiri::XML(nil)
|
13
|
-
@error = error
|
14
|
-
@klass = klass
|
15
|
-
end
|
16
|
-
|
17
|
-
def add
|
18
|
-
if is_xml?
|
19
|
-
unless has_errors_layer?
|
20
|
-
add_errors_layer
|
21
|
-
end
|
22
|
-
else
|
23
|
-
add_root
|
24
|
-
add_text
|
25
|
-
add_errors_layer
|
26
|
-
end
|
27
|
-
add_error
|
28
|
-
|
29
|
-
xml = !!document.encoding ? document.to_xml : document.to_xml(:encoding => "UTF-8")
|
30
|
-
|
31
|
-
return xml
|
32
|
-
end
|
33
|
-
|
34
|
-
##
|
35
|
-
# Check if the document is a valid XML file.
|
36
|
-
#
|
37
|
-
def is_xml?
|
38
|
-
!!document.root
|
39
|
-
end
|
40
|
-
|
41
|
-
##
|
42
|
-
# Add root element to the XML file.
|
43
|
-
#
|
44
|
-
def add_root
|
45
|
-
root = Nokogiri::XML::Node.new "KAF", document
|
46
|
-
document.add_child(root)
|
47
|
-
end
|
48
|
-
|
49
|
-
##
|
50
|
-
# Check if the document already has an errors layer.
|
51
|
-
#
|
52
|
-
def has_errors_layer?
|
53
|
-
!!document.at('errors')
|
54
|
-
end
|
55
|
-
|
56
|
-
##
|
57
|
-
# Add errors element to the XML file.
|
58
|
-
#
|
59
|
-
def add_errors_layer
|
60
|
-
node = Nokogiri::XML::Node.new "errors", document
|
61
|
-
document.root.add_child(node)
|
62
|
-
end
|
63
|
-
|
64
|
-
##
|
65
|
-
# Add the text file incase it is not a valid XML document. More
|
66
|
-
# info for debugging.
|
67
|
-
#
|
68
|
-
def add_text
|
69
|
-
node = Nokogiri::XML::Node.new "raw", document
|
70
|
-
node.inner_html = input
|
71
|
-
document.root.add_child(node)
|
72
|
-
|
73
|
-
end
|
74
|
-
|
75
|
-
##
|
76
|
-
# Add the actual error to the errors layer.
|
77
|
-
#
|
78
|
-
def add_error
|
79
|
-
node = document.at('errors')
|
80
|
-
error_node = Nokogiri::XML::Node.new "error", node
|
81
|
-
error_node['class'] = klass.to_s
|
82
|
-
error_node['version'] = klass::VERSION
|
83
|
-
error_node.inner_html = error
|
84
|
-
node.add_child(error_node)
|
85
|
-
end
|
86
|
-
|
87
|
-
end # ErrorLayer
|
88
|
-
end # Ned
|
89
|
-
end # Opener
|