informers 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +9 -4
- data/lib/informers/ner.rb +15 -0
- data/lib/informers/question_answering.rb +16 -1
- data/lib/informers/sentiment_analysis.rb +15 -0
- data/lib/informers/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7fab4014ceee446289bf0fb5a3c5b32630462eddba5c1b10e2104c3c42ee43ea
|
4
|
+
data.tar.gz: a09cdc2dc9676a91a5e6c0ab0c3712653b99d2dbae1fe91b1ac55b32777c8cb9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: eb4693b6ff9cd60ccdaf727de793a90b0b8576bdc57376a22e78759032144c4bc470b9750bdc38e2288229370dd43f5ca982b407d553ea7ca73b5fff0d5a9e3a
|
7
|
+
data.tar.gz: 7ad9384587b2c12ff09d21c4f1074b297758c7ef65142bfc19bb730e3034d7b9febd180253068b6181814803d6d599e4685a4ef736c734b1b45c41ee700fd3ec
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -11,6 +11,8 @@ Supports:
|
|
11
11
|
- Summarization - *in development*
|
12
12
|
- Translation - *in development*
|
13
13
|
|
14
|
+
[![Build Status](https://travis-ci.org/ankane/informers.svg?branch=master)](https://travis-ci.org/ankane/informers)
|
15
|
+
|
14
16
|
## Installation
|
15
17
|
|
16
18
|
Add this line to your application’s Gemfile:
|
@@ -33,7 +35,7 @@ brew install libomp
|
|
33
35
|
|
34
36
|
### Sentiment Analysis
|
35
37
|
|
36
|
-
First, download the [pretrained model](sentiment.onnx).
|
38
|
+
First, download the [pretrained model](https://github.com/ankane/informers/releases/download/v0.1.0/sentiment-analysis.onnx).
|
37
39
|
|
38
40
|
Predict sentiment
|
39
41
|
|
@@ -56,7 +58,7 @@ model.predict(["This is super cool", "I didn't like it"])
|
|
56
58
|
|
57
59
|
### Question Answering
|
58
60
|
|
59
|
-
First, download the [pretrained model](question-answering.onnx) and add Numo to your application’s Gemfile:
|
61
|
+
First, download the [pretrained model](https://github.com/ankane/informers/releases/download/v0.1.0/question-answering.onnx) and add Numo to your application’s Gemfile:
|
60
62
|
|
61
63
|
```ruby
|
62
64
|
gem 'numo-narray'
|
@@ -66,7 +68,10 @@ Ask a question with some context
|
|
66
68
|
|
67
69
|
```ruby
|
68
70
|
model = Informers::QuestionAnswering.new("question-answering.onnx")
|
69
|
-
model.predict(
|
71
|
+
model.predict(
|
72
|
+
question: "Who invented Ruby?",
|
73
|
+
context: "Ruby is a programming language created by Matz"
|
74
|
+
)
|
70
75
|
```
|
71
76
|
|
72
77
|
This returns
|
@@ -111,7 +116,7 @@ Models are [quantized](https://medium.com/microsoftazure/faster-and-smaller-quan
|
|
111
116
|
This project uses many state-of-the-art technologies:
|
112
117
|
|
113
118
|
- [Transformers](https://github.com/huggingface/transformers) for transformer models
|
114
|
-
- [
|
119
|
+
- [Bling Fire](https://github.com/microsoft/BlingFire) and [BERT](https://github.com/google-research/bert) for high-performance text tokenization
|
115
120
|
- [ONNX Runtime](https://github.com/Microsoft/onnxruntime) for high-performance inference
|
116
121
|
|
117
122
|
Some code was ported from Transformers and is available under the same license.
|
data/lib/informers/ner.rb
CHANGED
@@ -1,3 +1,18 @@
|
|
1
|
+
# Copyright 2018 The HuggingFace Inc. team.
|
2
|
+
# Copyright 2020 Andrew Kane.
|
3
|
+
#
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
#
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
#
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
1
16
|
module Informers
|
2
17
|
class NER
|
3
18
|
def initialize(model_path)
|
@@ -1,3 +1,18 @@
|
|
1
|
+
# Copyright 2018 The HuggingFace Inc. team.
|
2
|
+
# Copyright 2020 Andrew Kane.
|
3
|
+
#
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
#
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
#
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
1
16
|
module Informers
|
2
17
|
class QuestionAnswering
|
3
18
|
def initialize(model_path)
|
@@ -88,7 +103,7 @@ module Informers
|
|
88
103
|
starts, ends, scores = decode(start_, end_, topk, max_answer_len)
|
89
104
|
|
90
105
|
# char_to_word
|
91
|
-
doc_tokens, char_to_word_offset =
|
106
|
+
doc_tokens, char_to_word_offset = send(:doc_tokens, questions[i][:context])
|
92
107
|
char_to_word = Numo::Int64.cast(char_to_word_offset)
|
93
108
|
|
94
109
|
# token_to_orig_map
|
@@ -1,3 +1,18 @@
|
|
1
|
+
# Copyright 2018 The HuggingFace Inc. team.
|
2
|
+
# Copyright 2020 Andrew Kane.
|
3
|
+
#
|
4
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
5
|
+
# you may not use this file except in compliance with the License.
|
6
|
+
# You may obtain a copy of the License at
|
7
|
+
#
|
8
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
9
|
+
#
|
10
|
+
# Unless required by applicable law or agreed to in writing, software
|
11
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
12
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
13
|
+
# See the License for the specific language governing permissions and
|
14
|
+
# limitations under the License.
|
15
|
+
|
1
16
|
module Informers
|
2
17
|
class SentimentAnalysis
|
3
18
|
def initialize(model_path)
|
data/lib/informers/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: informers
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-10-
|
11
|
+
date: 2020-10-05 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: blingfire
|