opener-s3-outlet 1.1.3 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE.txt +13 -0
- data/README.md +29 -34
- data/bin/s3-outlet-daemon +6 -5
- data/bin/s3-outlet-server +6 -4
- data/exec/s3-outlet.rb +2 -26
- data/lib/opener/s3_outlet.rb +9 -4
- data/lib/opener/s3_outlet/s3_output.rb +1 -3
- data/lib/opener/s3_outlet/server.rb +30 -62
- data/lib/opener/s3_outlet/version.rb +1 -1
- data/lib/opener/s3_outlet/visualizer.rb +0 -3
- data/opener-s3-outlet.gemspec +15 -19
- metadata +30 -126
- data/visualizer.rb +0 -278
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: cfcdb5e05c9bd45c647de2ad0bd02b139a621aa8
|
4
|
+
data.tar.gz: 780fe19a0e15dbf9df35093361878fe595ffde15
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eed2a58ef5ae8652bbeed1f66c0b57f4883e5909ce6186b2971a8d7f08ecf8c5e3d54d012c45debe0811ef261c59124719d0d70782e81dd70e9e4a9420c3bb85
|
7
|
+
data.tar.gz: 94cffc0ee2646f2924ba0da0badfae6f0ad24ca75e938c0c2bb025c353f11c96bf58037cfdd55e7aa796d499ee70d19d65cf8784419e85de0722ef95ead041dc
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
Copyright 2014 OpeNER Project Consortium
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/README.md
CHANGED
@@ -1,21 +1,23 @@
|
|
1
|
-
S3-Outlet
|
2
|
-
------------
|
1
|
+
# S3-Outlet
|
3
2
|
|
4
|
-
Component
|
5
|
-
Database and shows them into your browser.
|
3
|
+
Component for storing KAF documents in Amazon S3.
|
6
4
|
|
7
|
-
|
5
|
+
## Confused by some terminology?
|
8
6
|
|
9
|
-
This software is part of a larger collection of natural language processing
|
7
|
+
This software is part of a larger collection of natural language processing
|
8
|
+
tools known as "the OpeNER project". You can find more information about the
|
9
|
+
project at [the OpeNER portal](http://opener-project.github.io). There you can
|
10
|
+
also find references to terms like KAF (an XML standard to represent linguistic
|
11
|
+
annotations in texts), component, cores, scenario's and pipelines.
|
10
12
|
|
11
|
-
Quick Use Example
|
12
|
-
-----------------
|
13
|
+
## Quick Use Example
|
13
14
|
|
14
15
|
Installing the s3-outlet can be done by executing:
|
15
16
|
|
16
17
|
gem install opener-s3-outlet
|
17
18
|
|
18
|
-
Please
|
19
|
+
Please keep in mind that all components in OpeNER take KAF as an input and
|
20
|
+
output KAF by default.
|
19
21
|
|
20
22
|
### Webservices
|
21
23
|
|
@@ -23,28 +25,32 @@ You can launch a webservice by executing:
|
|
23
25
|
|
24
26
|
s3-outlet-server
|
25
27
|
|
26
|
-
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
28
|
+
This will launch a mini webserver with the webservice. It defaults to port 9292,
|
29
|
+
so you can access it at <http://localhost:9292>.
|
27
30
|
|
28
|
-
To launch it on a different port provide the `-p [port-number]` option like
|
31
|
+
To launch it on a different port provide the `-p [port-number]` option like
|
32
|
+
this:
|
29
33
|
|
30
34
|
s3-outlet-server -p 1234
|
31
35
|
|
32
36
|
It then launches at <http://localhost:1234>
|
33
37
|
|
34
|
-
Documentation on the Webservice is provided by surfing to the urls provided
|
35
|
-
|
38
|
+
Documentation on the Webservice is provided by surfing to the urls provided
|
39
|
+
above. For more information on how to launch a webservice run the command with
|
40
|
+
the `--help` option.
|
36
41
|
|
37
42
|
### Daemon
|
38
43
|
|
39
|
-
Last but not least the s3-outlet comes shipped with a daemon that can read jobs
|
44
|
+
Last but not least the s3-outlet comes shipped with a daemon that can read jobs
|
45
|
+
(and write) jobs to and from Amazon SQS queues. For more information type:
|
40
46
|
|
41
47
|
s3-outlet-daemon -h
|
42
48
|
|
49
|
+
## Description of dependencies
|
43
50
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
This component runs best if you run it in an environment suited for OpeNER components. You can find an installation guide and helper tools in the [OpeNER installer](https://github.com/opener-project/opener-installer) and an
|
51
|
+
This component runs best if you run it in an environment suited for OpeNER
|
52
|
+
components. You can find an installation guide and helper tools in the
|
53
|
+
[OpeNER installer](https://github.com/opener-project/opener-installer) and an
|
48
54
|
[installation guide on the Opener Website](http://opener-project.github.io/getting-started/how-to/local-installation.html)
|
49
55
|
|
50
56
|
At least you need the following system setup:
|
@@ -54,28 +60,17 @@ At least you need the following system setup:
|
|
54
60
|
* JRuby 1.7.9 or newer
|
55
61
|
* Ruby 1.9.3 or newer
|
56
62
|
|
57
|
-
|
58
|
-
--------
|
59
|
-
|
60
|
-
The component is a fat wrapper around the actual language technology core. You can find the core technologies in the following repositories:
|
61
|
-
|
62
|
-
* (link to) Related Core readmes
|
63
|
-
|
64
|
-
Where to go from here
|
65
|
-
---------------------
|
63
|
+
## Where to go from here
|
66
64
|
|
67
65
|
* [Check the project website](http://opener-project.github.io)
|
68
66
|
* [Checkout the webservice](http://opener.olery.com/s3-outlet)
|
69
67
|
|
70
|
-
Report problem/Get help
|
71
|
-
-----------------------
|
72
|
-
|
73
|
-
If you encounter problems, please email support@opener-project.eu or leave an issue in the
|
74
|
-
[issue tracker](https://github.com/opener-project/s3-outlet/issues).
|
68
|
+
## Report problem/Get help
|
75
69
|
|
70
|
+
If you encounter problems, please email support@opener-project.eu or leave an
|
71
|
+
issue in the [issue tracker](https://github.com/opener-project/s3-outlet/issues).
|
76
72
|
|
77
|
-
Contributing
|
78
|
-
------------
|
73
|
+
## Contributing
|
79
74
|
|
80
75
|
1. Fork it <http://github.com/opener-project/s3-outlet/fork>
|
81
76
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
data/bin/s3-outlet-daemon
CHANGED
@@ -1,9 +1,10 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
|
-
|
3
|
-
require 'rubygems'
|
2
|
+
|
4
3
|
require 'opener/daemons'
|
5
4
|
|
6
|
-
|
7
|
-
|
8
|
-
|
5
|
+
controller = Opener::Daemons::Controller.new(
|
6
|
+
:name => 'opener-s3-outlet',
|
7
|
+
:exec_path => File.expand_path('../../exec/s3-outlet.rb', __FILE__)
|
8
|
+
)
|
9
9
|
|
10
|
+
controller.run
|
data/bin/s3-outlet-server
CHANGED
@@ -1,8 +1,10 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
|
-
require '
|
3
|
+
require 'opener/webservice'
|
4
4
|
|
5
|
-
|
5
|
+
parser = Opener::Webservice::OptionParser.new(
|
6
|
+
'opener-s3-outlet',
|
7
|
+
File.expand_path('../../config.ru', __FILE__)
|
8
|
+
)
|
6
9
|
|
7
|
-
|
8
|
-
cli.run
|
10
|
+
parser.run
|
data/exec/s3-outlet.rb
CHANGED
@@ -1,33 +1,9 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
3
|
require 'opener/daemons'
|
4
|
-
require_relative '../lib/opener/s3_outlet'
|
5
|
-
|
6
|
-
parser = Opener::Daemons::OptParser.new do |opts, options|
|
7
|
-
opts.on("-b", "--bucket NAME", "Bucket name") do |v|
|
8
|
-
options[:bucket] = v
|
9
|
-
end
|
10
|
-
opts.on("-d", "--directory NAME", "Directory name") do |v|
|
11
|
-
options[:directory] = v
|
12
|
-
end
|
13
|
-
end
|
14
|
-
|
15
|
-
class S3OutletFactory
|
16
|
-
attr_reader :dir, :bucket
|
17
4
|
|
18
|
-
|
19
|
-
@bucket = options.fetch(:bucket)
|
20
|
-
@dir = options.fetch(:directory)
|
21
|
-
end
|
22
|
-
|
23
|
-
def new
|
24
|
-
Opener::S3Outlet.new(:directory=>dir, :bucket=>bucket)
|
25
|
-
end
|
26
|
-
end
|
5
|
+
require_relative '../lib/opener/s3_outlet'
|
27
6
|
|
28
|
-
|
29
|
-
factory = S3OutletFactory.new(options)
|
7
|
+
daemon = Opener::Daemons::Daemon.new(Opener::S3Outlet)
|
30
8
|
|
31
|
-
daemon = Opener::Daemons::Daemon.new(factory, options)
|
32
9
|
daemon.start
|
33
|
-
|
data/lib/opener/s3_outlet.rb
CHANGED
@@ -1,9 +1,13 @@
|
|
1
|
+
require 'securerandom'
|
2
|
+
require 'stringio'
|
3
|
+
|
1
4
|
require 'aws-sdk'
|
5
|
+
require 'nokogiri'
|
6
|
+
require 'opener/kaf_to_json'
|
2
7
|
|
3
8
|
require_relative 's3_outlet/s3_output'
|
4
9
|
require_relative 's3_outlet/version'
|
5
10
|
require_relative 's3_outlet/server'
|
6
|
-
require 'opener/kaf_to_json'
|
7
11
|
|
8
12
|
module Opener
|
9
13
|
class S3Outlet
|
@@ -14,10 +18,11 @@ module Opener
|
|
14
18
|
end
|
15
19
|
|
16
20
|
def run(input)
|
17
|
-
|
18
|
-
|
21
|
+
S3Output.create(
|
22
|
+
options.merge(:text => input, :uuid => options[:request_id])
|
23
|
+
)
|
19
24
|
|
20
|
-
return input
|
25
|
+
return input
|
21
26
|
end
|
22
27
|
|
23
28
|
def self.s3
|
@@ -1,5 +1,3 @@
|
|
1
|
-
require 'uuidtools'
|
2
|
-
|
3
1
|
module Opener
|
4
2
|
class S3Outlet
|
5
3
|
class S3Output
|
@@ -7,7 +5,7 @@ module Opener
|
|
7
5
|
attr_reader :uuid, :text, :dir, :bucket, :params
|
8
6
|
|
9
7
|
def initialize(params = {})
|
10
|
-
@uuid = params
|
8
|
+
@uuid = params[:uuid] || SecureRandom.hex
|
11
9
|
@text = params.fetch(:text)
|
12
10
|
@dir = params.fetch(:directory, S3Outlet.dir)
|
13
11
|
|
@@ -1,15 +1,10 @@
|
|
1
|
-
require 'sinatra'
|
2
|
-
require 'nokogiri'
|
3
|
-
require 'stringio'
|
4
|
-
require_relative './visualizer'
|
5
|
-
|
6
1
|
module Opener
|
7
2
|
class S3Outlet
|
8
|
-
class Server <
|
3
|
+
class Server < Webservice::Server
|
4
|
+
set :views, File.expand_path('../views', __FILE__)
|
9
5
|
|
10
|
-
|
11
|
-
|
12
|
-
end
|
6
|
+
self.text_processor = S3Outlet
|
7
|
+
self.accepted_params = [:request_id]
|
13
8
|
|
14
9
|
get '/' do
|
15
10
|
if params[:request_id]
|
@@ -18,80 +13,53 @@ module Opener
|
|
18
13
|
erb :index
|
19
14
|
end
|
20
15
|
end
|
21
|
-
|
16
|
+
|
22
17
|
get '/:request_id.json' do
|
23
18
|
unless params[:request_id] == 'favicon.ico'
|
24
|
-
|
25
|
-
output = S3Output.find(params[:request_id])
|
19
|
+
output = S3Output.find(params[:request_id])
|
26
20
|
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
end
|
35
|
-
rescue => error
|
36
|
-
error_callback = params[:error_callback]
|
37
|
-
|
38
|
-
submit_error(error_callback, error.message) if error_callback
|
39
|
-
|
40
|
-
raise(error)
|
21
|
+
if output
|
22
|
+
content_type('text/json')
|
23
|
+
kj = Opener::KafToJson.new
|
24
|
+
json = kj.run(output.text) rescue output.text
|
25
|
+
body(json)
|
26
|
+
else
|
27
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
41
28
|
end
|
42
29
|
end
|
43
30
|
end
|
44
31
|
|
45
32
|
get '/:request_id' do
|
46
33
|
unless params[:request_id] == 'favicon.ico'
|
47
|
-
|
48
|
-
output = S3Output.find(params[:request_id])
|
49
|
-
|
50
|
-
if output
|
51
|
-
content_type(:xml)
|
52
|
-
body(output)
|
53
|
-
else
|
54
|
-
halt(404, "No record found for ID #{params[:request_id]}")
|
55
|
-
end
|
56
|
-
rescue => error
|
57
|
-
error_callback = params[:error_callback]
|
34
|
+
output = S3Output.find(params[:request_id])
|
58
35
|
|
59
|
-
|
60
|
-
|
61
|
-
|
36
|
+
if output
|
37
|
+
content_type(:xml)
|
38
|
+
body(output)
|
39
|
+
else
|
40
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
62
41
|
end
|
63
42
|
end
|
64
43
|
end
|
65
44
|
|
66
45
|
get '/html/:request_id' do
|
67
46
|
unless params[:request_id] == 'favicon.ico'
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
@parsed = html.to_html
|
76
|
-
erb :show
|
77
|
-
else
|
78
|
-
halt(404, "No record found for ID #{params[:request_id]}")
|
79
|
-
end
|
80
|
-
rescue => error
|
81
|
-
error_callback = params[:error_callback]
|
47
|
+
output = S3Output.find(params[:request_id])
|
48
|
+
|
49
|
+
if output
|
50
|
+
output = StringIO.new(output)
|
51
|
+
parser = Opener::Kaf::Visualizer::Parser.new(output)
|
52
|
+
doc = parser.parse
|
53
|
+
html = Opener::Kaf::Visualizer::HTMLTextPresenter.new(doc)
|
82
54
|
|
83
|
-
|
55
|
+
@parsed = html.to_html
|
84
56
|
|
85
|
-
|
57
|
+
erb :show
|
58
|
+
else
|
59
|
+
halt(404, "No record found for ID #{params[:request_id]}")
|
86
60
|
end
|
87
61
|
end
|
88
62
|
end
|
89
|
-
|
90
|
-
private
|
91
|
-
|
92
|
-
def submit_error(url, message)
|
93
|
-
HTTPClient.post(url, :body => {:error => message})
|
94
|
-
end
|
95
63
|
end # Server
|
96
64
|
end # S3Outlet
|
97
65
|
end # Opener
|
data/opener-s3-outlet.gemspec
CHANGED
@@ -1,13 +1,15 @@
|
|
1
1
|
require File.expand_path('../lib/opener/s3_outlet/version', __FILE__)
|
2
2
|
|
3
3
|
Gem::Specification.new do |gem|
|
4
|
-
gem.name
|
5
|
-
gem.version
|
6
|
-
gem.authors
|
7
|
-
gem.summary
|
8
|
-
gem.description
|
9
|
-
gem.homepage
|
10
|
-
gem.has_rdoc
|
4
|
+
gem.name = 'opener-s3-outlet'
|
5
|
+
gem.version = Opener::S3Outlet::VERSION
|
6
|
+
gem.authors = ['development@olery.com']
|
7
|
+
gem.summary = 'S3 Bucket data storing for the web services output when using callbacks.'
|
8
|
+
gem.description = gem.summary
|
9
|
+
gem.homepage = "http://opener-project.github.com/"
|
10
|
+
gem.has_rdoc = 'yard'
|
11
|
+
gem.license = 'Apache 2.0'
|
12
|
+
|
11
13
|
gem.required_ruby_version = '>= 1.9.2'
|
12
14
|
|
13
15
|
gem.files = Dir.glob([
|
@@ -16,24 +18,18 @@ Gem::Specification.new do |gem|
|
|
16
18
|
'config.ru',
|
17
19
|
'*.gemspec',
|
18
20
|
'README.md',
|
19
|
-
'
|
21
|
+
'LICENSE.txt'
|
20
22
|
]).select { |file| File.file?(file) }
|
21
23
|
|
22
24
|
gem.executables = Dir.glob('bin/*').map { |file| File.basename(file) }
|
23
25
|
|
26
|
+
gem.add_dependency 'opener-kaf2json'
|
27
|
+
gem.add_dependency 'opener-daemons', '~> 2.2'
|
28
|
+
gem.add_dependency 'opener-webservice', '~> 2.1'
|
29
|
+
gem.add_dependency 'opener-core', '~> 2.2'
|
30
|
+
|
24
31
|
gem.add_dependency 'builder'
|
25
|
-
gem.add_dependency 'sinatra', '~>1.4.2'
|
26
32
|
gem.add_dependency 'nokogiri'
|
27
|
-
gem.add_dependency 'httpclient'
|
28
|
-
gem.add_dependency 'uuidtools'
|
29
|
-
gem.add_dependency 'opener-webservice'
|
30
|
-
gem.add_dependency 'opener-daemons'
|
31
|
-
gem.add_dependency 'opener-kaf2json'
|
32
|
-
gem.add_dependency 'aws-sdk'
|
33
|
-
gem.add_dependency 'puma'
|
34
33
|
|
35
|
-
gem.add_development_dependency 'rspec'
|
36
|
-
gem.add_development_dependency 'cucumber'
|
37
|
-
gem.add_development_dependency 'pry'
|
38
34
|
gem.add_development_dependency 'rake'
|
39
35
|
end
|
metadata
CHANGED
@@ -1,17 +1,17 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: opener-s3-outlet
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- development@olery.com
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-11-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
|
-
name:
|
14
|
+
name: opener-kaf2json
|
15
15
|
version_requirements: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
17
|
- - '>='
|
@@ -25,105 +25,49 @@ dependencies:
|
|
25
25
|
prerelease: false
|
26
26
|
type: :runtime
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
|
-
name:
|
28
|
+
name: opener-daemons
|
29
29
|
version_requirements: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ~>
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
33
|
+
version: '2.2'
|
34
34
|
requirement: !ruby/object:Gem::Requirement
|
35
35
|
requirements:
|
36
36
|
- - ~>
|
37
37
|
- !ruby/object:Gem::Version
|
38
|
-
version:
|
39
|
-
prerelease: false
|
40
|
-
type: :runtime
|
41
|
-
- !ruby/object:Gem::Dependency
|
42
|
-
name: nokogiri
|
43
|
-
version_requirements: !ruby/object:Gem::Requirement
|
44
|
-
requirements:
|
45
|
-
- - '>='
|
46
|
-
- !ruby/object:Gem::Version
|
47
|
-
version: '0'
|
48
|
-
requirement: !ruby/object:Gem::Requirement
|
49
|
-
requirements:
|
50
|
-
- - '>='
|
51
|
-
- !ruby/object:Gem::Version
|
52
|
-
version: '0'
|
53
|
-
prerelease: false
|
54
|
-
type: :runtime
|
55
|
-
- !ruby/object:Gem::Dependency
|
56
|
-
name: httpclient
|
57
|
-
version_requirements: !ruby/object:Gem::Requirement
|
58
|
-
requirements:
|
59
|
-
- - '>='
|
60
|
-
- !ruby/object:Gem::Version
|
61
|
-
version: '0'
|
62
|
-
requirement: !ruby/object:Gem::Requirement
|
63
|
-
requirements:
|
64
|
-
- - '>='
|
65
|
-
- !ruby/object:Gem::Version
|
66
|
-
version: '0'
|
67
|
-
prerelease: false
|
68
|
-
type: :runtime
|
69
|
-
- !ruby/object:Gem::Dependency
|
70
|
-
name: uuidtools
|
71
|
-
version_requirements: !ruby/object:Gem::Requirement
|
72
|
-
requirements:
|
73
|
-
- - '>='
|
74
|
-
- !ruby/object:Gem::Version
|
75
|
-
version: '0'
|
76
|
-
requirement: !ruby/object:Gem::Requirement
|
77
|
-
requirements:
|
78
|
-
- - '>='
|
79
|
-
- !ruby/object:Gem::Version
|
80
|
-
version: '0'
|
38
|
+
version: '2.2'
|
81
39
|
prerelease: false
|
82
40
|
type: :runtime
|
83
41
|
- !ruby/object:Gem::Dependency
|
84
42
|
name: opener-webservice
|
85
43
|
version_requirements: !ruby/object:Gem::Requirement
|
86
44
|
requirements:
|
87
|
-
- -
|
88
|
-
- !ruby/object:Gem::Version
|
89
|
-
version: '0'
|
90
|
-
requirement: !ruby/object:Gem::Requirement
|
91
|
-
requirements:
|
92
|
-
- - '>='
|
93
|
-
- !ruby/object:Gem::Version
|
94
|
-
version: '0'
|
95
|
-
prerelease: false
|
96
|
-
type: :runtime
|
97
|
-
- !ruby/object:Gem::Dependency
|
98
|
-
name: opener-daemons
|
99
|
-
version_requirements: !ruby/object:Gem::Requirement
|
100
|
-
requirements:
|
101
|
-
- - '>='
|
45
|
+
- - ~>
|
102
46
|
- !ruby/object:Gem::Version
|
103
|
-
version: '
|
47
|
+
version: '2.1'
|
104
48
|
requirement: !ruby/object:Gem::Requirement
|
105
49
|
requirements:
|
106
|
-
- -
|
50
|
+
- - ~>
|
107
51
|
- !ruby/object:Gem::Version
|
108
|
-
version: '
|
52
|
+
version: '2.1'
|
109
53
|
prerelease: false
|
110
54
|
type: :runtime
|
111
55
|
- !ruby/object:Gem::Dependency
|
112
|
-
name: opener-
|
56
|
+
name: opener-core
|
113
57
|
version_requirements: !ruby/object:Gem::Requirement
|
114
58
|
requirements:
|
115
|
-
- -
|
59
|
+
- - ~>
|
116
60
|
- !ruby/object:Gem::Version
|
117
|
-
version: '
|
61
|
+
version: '2.2'
|
118
62
|
requirement: !ruby/object:Gem::Requirement
|
119
63
|
requirements:
|
120
|
-
- -
|
64
|
+
- - ~>
|
121
65
|
- !ruby/object:Gem::Version
|
122
|
-
version: '
|
66
|
+
version: '2.2'
|
123
67
|
prerelease: false
|
124
68
|
type: :runtime
|
125
69
|
- !ruby/object:Gem::Dependency
|
126
|
-
name:
|
70
|
+
name: builder
|
127
71
|
version_requirements: !ruby/object:Gem::Requirement
|
128
72
|
requirements:
|
129
73
|
- - '>='
|
@@ -137,7 +81,7 @@ dependencies:
|
|
137
81
|
prerelease: false
|
138
82
|
type: :runtime
|
139
83
|
- !ruby/object:Gem::Dependency
|
140
|
-
name:
|
84
|
+
name: nokogiri
|
141
85
|
version_requirements: !ruby/object:Gem::Requirement
|
142
86
|
requirements:
|
143
87
|
- - '>='
|
@@ -150,48 +94,6 @@ dependencies:
|
|
150
94
|
version: '0'
|
151
95
|
prerelease: false
|
152
96
|
type: :runtime
|
153
|
-
- !ruby/object:Gem::Dependency
|
154
|
-
name: rspec
|
155
|
-
version_requirements: !ruby/object:Gem::Requirement
|
156
|
-
requirements:
|
157
|
-
- - '>='
|
158
|
-
- !ruby/object:Gem::Version
|
159
|
-
version: '0'
|
160
|
-
requirement: !ruby/object:Gem::Requirement
|
161
|
-
requirements:
|
162
|
-
- - '>='
|
163
|
-
- !ruby/object:Gem::Version
|
164
|
-
version: '0'
|
165
|
-
prerelease: false
|
166
|
-
type: :development
|
167
|
-
- !ruby/object:Gem::Dependency
|
168
|
-
name: cucumber
|
169
|
-
version_requirements: !ruby/object:Gem::Requirement
|
170
|
-
requirements:
|
171
|
-
- - '>='
|
172
|
-
- !ruby/object:Gem::Version
|
173
|
-
version: '0'
|
174
|
-
requirement: !ruby/object:Gem::Requirement
|
175
|
-
requirements:
|
176
|
-
- - '>='
|
177
|
-
- !ruby/object:Gem::Version
|
178
|
-
version: '0'
|
179
|
-
prerelease: false
|
180
|
-
type: :development
|
181
|
-
- !ruby/object:Gem::Dependency
|
182
|
-
name: pry
|
183
|
-
version_requirements: !ruby/object:Gem::Requirement
|
184
|
-
requirements:
|
185
|
-
- - '>='
|
186
|
-
- !ruby/object:Gem::Version
|
187
|
-
version: '0'
|
188
|
-
requirement: !ruby/object:Gem::Requirement
|
189
|
-
requirements:
|
190
|
-
- - '>='
|
191
|
-
- !ruby/object:Gem::Version
|
192
|
-
version: '0'
|
193
|
-
prerelease: false
|
194
|
-
type: :development
|
195
97
|
- !ruby/object:Gem::Dependency
|
196
98
|
name: rake
|
197
99
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -214,24 +116,25 @@ executables:
|
|
214
116
|
extensions: []
|
215
117
|
extra_rdoc_files: []
|
216
118
|
files:
|
217
|
-
- README.md
|
218
|
-
- bin/s3-outlet-daemon
|
219
|
-
- bin/s3-outlet-server
|
220
|
-
- config.ru
|
221
119
|
- exec/s3-outlet.rb
|
222
120
|
- lib/opener/s3_outlet.rb
|
223
|
-
- lib/opener/s3_outlet/
|
121
|
+
- lib/opener/s3_outlet/visualizer.rb
|
122
|
+
- lib/opener/s3_outlet/version.rb
|
224
123
|
- lib/opener/s3_outlet/s3_output.rb
|
225
124
|
- lib/opener/s3_outlet/server.rb
|
226
|
-
- lib/opener/s3_outlet/
|
227
|
-
- lib/opener/s3_outlet/views/index.erb
|
125
|
+
- lib/opener/s3_outlet/public/outlet.css
|
228
126
|
- lib/opener/s3_outlet/views/show.erb
|
127
|
+
- lib/opener/s3_outlet/views/index.erb
|
229
128
|
- lib/opener/s3_outlet/views/show.html.erb
|
230
|
-
-
|
129
|
+
- config.ru
|
231
130
|
- opener-s3-outlet.gemspec
|
232
|
-
-
|
131
|
+
- README.md
|
132
|
+
- LICENSE.txt
|
133
|
+
- bin/s3-outlet-daemon
|
134
|
+
- bin/s3-outlet-server
|
233
135
|
homepage: http://opener-project.github.com/
|
234
|
-
licenses:
|
136
|
+
licenses:
|
137
|
+
- Apache 2.0
|
235
138
|
metadata: {}
|
236
139
|
post_install_message:
|
237
140
|
rdoc_options: []
|
@@ -249,8 +152,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
249
152
|
version: '0'
|
250
153
|
requirements: []
|
251
154
|
rubyforge_project:
|
252
|
-
rubygems_version: 2.
|
155
|
+
rubygems_version: 2.1.9
|
253
156
|
signing_key:
|
254
157
|
specification_version: 4
|
255
158
|
summary: S3 Bucket data storing for the web services output when using callbacks.
|
256
159
|
test_files: []
|
160
|
+
has_rdoc: yard
|
data/visualizer.rb
DELETED
@@ -1,278 +0,0 @@
|
|
1
|
-
require 'nokogiri'
|
2
|
-
|
3
|
-
module Opener
|
4
|
-
module Kaf
|
5
|
-
module Visualizer
|
6
|
-
class Parser
|
7
|
-
attr_reader :doc
|
8
|
-
attr_reader :words, :terms, :entities, :properties, :opinions, :document
|
9
|
-
|
10
|
-
def initialize(input_file_handler)
|
11
|
-
@doc = Nokogiri::XML(input_file_handler)
|
12
|
-
end
|
13
|
-
|
14
|
-
def parse
|
15
|
-
@words = parse_words
|
16
|
-
@terms = parse_terms
|
17
|
-
@entities = parse_entities
|
18
|
-
@properties = parse_properties
|
19
|
-
@opinions = parse_opinions
|
20
|
-
@document = KAFDocument.new(
|
21
|
-
:words => words,
|
22
|
-
:terms => terms,
|
23
|
-
:entities => entities,
|
24
|
-
:properties => properties,
|
25
|
-
:opinions => opinions
|
26
|
-
)
|
27
|
-
|
28
|
-
return document
|
29
|
-
end
|
30
|
-
|
31
|
-
def parse_words
|
32
|
-
parse_elements("//wf", Word)
|
33
|
-
end
|
34
|
-
|
35
|
-
def parse_terms
|
36
|
-
# Of course terms should be words here.
|
37
|
-
# Dirty Hack, sufficient for now.
|
38
|
-
parse_elements("//term", Term, :terms=>words)
|
39
|
-
end
|
40
|
-
|
41
|
-
def parse_entities
|
42
|
-
parse_elements("//entity", Entity, :terms=>terms)
|
43
|
-
end
|
44
|
-
|
45
|
-
def parse_properties
|
46
|
-
parse_elements("//property", Property, :terms=>terms)
|
47
|
-
end
|
48
|
-
|
49
|
-
def parse_opinions
|
50
|
-
parse_elements("//opinion", Opinion, :terms=>terms)
|
51
|
-
end
|
52
|
-
|
53
|
-
def parse_elements(xpath, klass, opts={})
|
54
|
-
elements = doc.xpath(xpath)
|
55
|
-
lookup_table = Hash.new
|
56
|
-
elements.each do |element|
|
57
|
-
instance = klass.new(element, opts)
|
58
|
-
lookup_table[instance.id] = instance
|
59
|
-
end
|
60
|
-
lookup_table
|
61
|
-
end
|
62
|
-
|
63
|
-
end
|
64
|
-
|
65
|
-
class KAFNode
|
66
|
-
attr_reader :content, :targets, :tag, :references
|
67
|
-
|
68
|
-
def initialize(tag, references)
|
69
|
-
@references = references
|
70
|
-
@tag = tag
|
71
|
-
|
72
|
-
set_instance_variables
|
73
|
-
set_content
|
74
|
-
set_targets
|
75
|
-
process_subnodes
|
76
|
-
end
|
77
|
-
|
78
|
-
def set_content
|
79
|
-
@content = tag.content
|
80
|
-
end
|
81
|
-
|
82
|
-
def set_instance_variables
|
83
|
-
tag.keys.each do |key|
|
84
|
-
if respond_to?("#{key}=".to_sym)
|
85
|
-
send("#{key}=".to_sym, tag[key])
|
86
|
-
else
|
87
|
-
instance_variable_set("@#{key}", tag[key])
|
88
|
-
end
|
89
|
-
end
|
90
|
-
end
|
91
|
-
|
92
|
-
def set_targets
|
93
|
-
@targets = []
|
94
|
-
tag.css("span target").each do |target|
|
95
|
-
id = target["id"]
|
96
|
-
@targets << references[:terms][id]
|
97
|
-
end
|
98
|
-
end
|
99
|
-
|
100
|
-
def process_subnodes
|
101
|
-
end
|
102
|
-
|
103
|
-
def has_target?(*ids)
|
104
|
-
ids.flatten.each do |id|
|
105
|
-
return true if target_ids.include?(id)
|
106
|
-
end
|
107
|
-
return false
|
108
|
-
end
|
109
|
-
|
110
|
-
def to_s
|
111
|
-
if targets.size > 0
|
112
|
-
return targets.map(&:to_s).join(" ")
|
113
|
-
else
|
114
|
-
return content
|
115
|
-
end
|
116
|
-
end
|
117
|
-
|
118
|
-
def target_ids
|
119
|
-
@targets.map(&:id)
|
120
|
-
end
|
121
|
-
|
122
|
-
end
|
123
|
-
|
124
|
-
class Word < KAFNode
|
125
|
-
attr_reader :wid, :sent, :para, :offset
|
126
|
-
|
127
|
-
def id
|
128
|
-
wid
|
129
|
-
end
|
130
|
-
|
131
|
-
def offset=(offset)
|
132
|
-
@offset = offset.to_i
|
133
|
-
end
|
134
|
-
|
135
|
-
def length
|
136
|
-
content.nil? ? 0 : content.length
|
137
|
-
end
|
138
|
-
|
139
|
-
end
|
140
|
-
|
141
|
-
class Term < KAFNode
|
142
|
-
attr_reader :tid, :type, :lemma, :pos, :morphofeat
|
143
|
-
|
144
|
-
def id
|
145
|
-
tid
|
146
|
-
end
|
147
|
-
end
|
148
|
-
|
149
|
-
class Entity < KAFNode
|
150
|
-
attr_reader :eid, :type
|
151
|
-
|
152
|
-
def id
|
153
|
-
eid
|
154
|
-
end
|
155
|
-
|
156
|
-
def to_s
|
157
|
-
"#{type}: #{targets.map(&:to_s).join(", ")}"
|
158
|
-
end
|
159
|
-
end
|
160
|
-
|
161
|
-
class Property < KAFNode
|
162
|
-
attr_reader :pid, :lemma
|
163
|
-
|
164
|
-
def id
|
165
|
-
pid
|
166
|
-
end
|
167
|
-
|
168
|
-
def to_s
|
169
|
-
"#{lemma}: #{targets.map(&:to_s).join(", ")}"
|
170
|
-
end
|
171
|
-
end
|
172
|
-
|
173
|
-
class Opinion < KAFNode
|
174
|
-
attr_reader :oid, :expression
|
175
|
-
|
176
|
-
def id
|
177
|
-
oid
|
178
|
-
end
|
179
|
-
|
180
|
-
def process_subnodes
|
181
|
-
@expression = tag.xpath("//opinion_expression").first["polarity"].to_sym
|
182
|
-
end
|
183
|
-
|
184
|
-
def to_s
|
185
|
-
"#{expression}: #{targets.map(&:to_s).join(", ")}"
|
186
|
-
end
|
187
|
-
|
188
|
-
end
|
189
|
-
|
190
|
-
class KAFDocument
|
191
|
-
attr_reader :words, :terms, :entities, :properties, :opinions
|
192
|
-
|
193
|
-
def initialize(opts={})
|
194
|
-
@words = opts.fetch(:words)
|
195
|
-
@terms = opts.fetch(:terms)
|
196
|
-
@entities = opts.fetch(:entities)
|
197
|
-
@properties = opts.fetch(:properties)
|
198
|
-
@opinions = opts.fetch(:opinions)
|
199
|
-
end
|
200
|
-
|
201
|
-
end
|
202
|
-
|
203
|
-
class HTMLTextPresenter
|
204
|
-
attr_reader :document
|
205
|
-
def initialize(document)
|
206
|
-
@document = document
|
207
|
-
end
|
208
|
-
|
209
|
-
def to_html
|
210
|
-
offset = 0
|
211
|
-
prev = Struct.new(:offset).new(0)
|
212
|
-
|
213
|
-
builder = Nokogiri::HTML::Builder.new do |html|
|
214
|
-
html.div(:class=>"opener") do
|
215
|
-
html.p do
|
216
|
-
document.words.values.sort_by(&:offset).each do |word|
|
217
|
-
if offset < word.offset
|
218
|
-
spacer = word.offset - offset
|
219
|
-
spacer = Array.new(spacer, " ").join
|
220
|
-
html.span(spacer)
|
221
|
-
end
|
222
|
-
|
223
|
-
terms = targets_for(:terms, word.id)
|
224
|
-
entities = targets_for(:entities, terms)
|
225
|
-
opinions = opinions_for(terms)
|
226
|
-
properties = targets_for(:properties, terms)
|
227
|
-
|
228
|
-
generics = []
|
229
|
-
generics << "term" if terms.size > 0
|
230
|
-
generics << "entity" if entities.size > 0
|
231
|
-
generics << "opinion" if opinions.size > 0
|
232
|
-
generics << "property" if properties.size > 0
|
233
|
-
|
234
|
-
classes = [terms, entities, opinions, properties, generics].flatten.uniq
|
235
|
-
|
236
|
-
word_annotations = classes.join(" ")
|
237
|
-
|
238
|
-
html.span(word.content, :class=>word_annotations, :id=>word.id)
|
239
|
-
offset = word.offset + word.length
|
240
|
-
end
|
241
|
-
end
|
242
|
-
|
243
|
-
[:entities, :opinions, :properties].each do |sym|
|
244
|
-
html.div(:class=>sym) do
|
245
|
-
document.public_send(sym).values.each do |entity|
|
246
|
-
html.div(entity.to_s, :id=>entity.id, :class=>entity.target_ids.join(" "))
|
247
|
-
end
|
248
|
-
end
|
249
|
-
end
|
250
|
-
end
|
251
|
-
end
|
252
|
-
|
253
|
-
builder.to_html
|
254
|
-
end
|
255
|
-
|
256
|
-
def targets_for(variable, *ids)
|
257
|
-
targets = document.public_send(variable.to_sym).values.select do |value|
|
258
|
-
value.has_target?(ids.flatten)
|
259
|
-
end
|
260
|
-
|
261
|
-
targets.map(&:id)
|
262
|
-
end
|
263
|
-
|
264
|
-
def opinions_for(*ids)
|
265
|
-
targets = document.opinions.values.select do |value|
|
266
|
-
value.has_target?(ids.flatten)
|
267
|
-
end
|
268
|
-
|
269
|
-
ids = targets.map(&:id)
|
270
|
-
sentiments = targets.map(&:expression)
|
271
|
-
return ids.concat(sentiments).uniq
|
272
|
-
end
|
273
|
-
|
274
|
-
end
|
275
|
-
end
|
276
|
-
end
|
277
|
-
end
|
278
|
-
|