rley 0.6.02 → 0.6.03

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 9764b3e89d5395584b93ca2a06d80a3381d58560
4
- data.tar.gz: 91eb805167c77da1d86cf23e04382d9c791f47c5
3
+ metadata.gz: 0100f83a11838d8f996fdb6057f3292cb8377562
4
+ data.tar.gz: d497a30317385d5aac6adbc247a0162abb59b57d
5
5
  SHA512:
6
- metadata.gz: 9dc181140698cab328974cbbca977ccbd0de1afc5eb0af628f2c7517fbb1ad93af058341e6146542540adcd7ea7268dd3a234131b3b046838c6a26e2cf05146e
7
- data.tar.gz: 90e957d8e37e957b50c769de6661c3ce5f5ad9d6c48beb5c7a8e7487622568273f715c5bacdbbd7ed382f410c6cd9b420218ac447e92fc279fd901b6e0a2baea
6
+ metadata.gz: db578e84d91d34009b07badaab7070ad2fbc6d0d5b042ac2f9e75edaa6a75d8bba15543b3c5cf017f4821bc8695791e91b48c8ebdac6896069e55ba8d96b51d1
7
+ data.tar.gz: efffade432255e86a4939bd36d306f1a35203267a6fcbeb72df9b60a62d92342a0ea56d978499a01b9d0dfce533e9b7858a38c8b1e3dacca5bc5524adfcf8802
@@ -3,10 +3,11 @@ rvm:
3
3
  - 2.0.0-p648
4
4
  - 2.1.10
5
5
  - 2.2.8
6
- - 2.3.5
6
+ - 2.3.6
7
7
  - 2.4.2
8
+ - 2.5.0
8
9
  - ruby-head
9
- - jruby-9.1.9.0
10
+ - jruby-9.1.13.0
10
11
  - jruby-head
11
12
  matrix:
12
13
  allow_failures:
@@ -1,5 +1,11 @@
1
+ ### 0.6.03 / 2018-03-10
2
+ * [CHANGE] File `lib/parser/error_reason.rb` Improved error location reporting
3
+ * [CHANGE] File `README.md` updated to reflect the improved error location reporting.
4
+ * [CHANGE] File `.travis.yml`: Added Ruby 2.5 build environment in Travis CI
5
+ * [CHANGE] File `appveyor.yml`: Added Ruby 2.5 build environment in Appveyor CI
6
+
1
7
  ### 0.6.02 / 2018-03-03
2
- * [FIX] File `examples/general/NLP/engtagger.rb` ode re-styling to remove most style offenses found by Rubocop 0.52.1
8
+ * [FIX] File `examples/general/NLP/engtagger.rb` code re-styling to remove most style offenses found by Rubocop 0.52.1
3
9
  * [DELETE] All files `examples/general/SRL` are removed. It will become a new gem by itself.
4
10
 
5
11
  ### 0.6.01 / 2018-03-03
data/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
  [![Gem Version](https://badge.fury.io/rb/rley.svg)](http://badge.fury.io/rb/rley)
7
7
  [![Dependency Status](https://gemnasium.com/famished-tiger/Rley.svg)](https://gemnasium.com/famished-tiger/Rley)
8
8
  [![Inline docs](http://inch-ci.org/github/famished-tiger/Rley.svg?branch=master)](http://inch-ci.org/github/famished-tiger/Rley)
9
- [![License](https://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](https://github.com/famished-tiger/Rley/blob/master/LICENSE.txt)
9
+ [![License](https://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat)](https://github.com/famished-tiger/SRL-Ruby/blob/master/LICENSE.txt)
10
10
 
11
11
  A Ruby library for constructing general parsers for _any_ context-free language.
12
12
 
@@ -38,7 +38,7 @@ handle
38
38
 
39
39
  In short, the foundations of Rley are strong enough to be useful in a large
40
40
  application range such as:
41
- * computer languages,
41
+ * computer languages -e.g. [Simple Regex Language](https://github.com/famished-tiger/SRL-Ruby) - ,
42
42
  * artificial intelligence and
43
43
  * Natural Language Processing.
44
44
 
@@ -56,7 +56,8 @@ Rley supports the following Ruby implementations:
56
56
  - MRI 2.1
57
57
  - MRI 2.2
58
58
  - MRI 2.3
59
- - MRI 2.4
59
+ - MRI 2.4
60
+ - MRI 2.5
60
61
  - JRuby 9.1+
61
62
 
62
63
  ---
@@ -374,7 +375,7 @@ ask it to generate a message.
374
375
 
375
376
  Re-running the example with the error, results in the error message:
376
377
  ```
377
- Syntax error at or near token 2 >>>Mary<<<
378
+ Syntax error at or near token line 1, column 6 >>>Mary<<<
378
379
  Expected one 'Verb', found a 'Proper-Noun' instead.
379
380
  ```
380
381
 
@@ -401,12 +402,10 @@ Let's experiment again with the original sentence but without the word
401
402
  This time, the following output is displayed:
402
403
  ```
403
404
  Parsing successful? false
404
- Premature end of input after 'a' at position 5
405
+ Premature end of input after 'a' at position line 1, column 20
405
406
  Expected one 'Noun'.
406
407
  ```
407
408
  Again, the resulting error message is user-friendly.
408
- Remark: currently, Rley reports an error position as the index of the
409
- input token with which the error was detected.
410
409
 
411
410
 
412
411
  ## Examples
@@ -11,7 +11,7 @@ environment:
11
11
  - Ruby_version: 23
12
12
  - Ruby_version: 23-x64
13
13
  - Ruby_version: 24
14
- - Ruby_version: 24-x64
14
+ - Ruby_version: 24-x64
15
15
 
16
16
  install:
17
17
  - set PATH=C:\Ruby%Ruby_version%\bin;%PATH%
@@ -1,4 +1,5 @@
1
1
  require 'rley' # Load Rley library
2
+ require 'strscan'
2
3
 
3
4
 
4
5
  ########################################
@@ -55,16 +56,40 @@ Lexicon = {
55
56
  'with' => 'Preposition'
56
57
  }.freeze
57
58
 
59
+ Position = Struct.new(:line, :column) do
60
+ def to_s()
61
+ "line #{line}, column #{column}"
62
+ end
63
+ end
64
+
65
+ class NLPToken < Rley::Lexical::Token
66
+ attr_reader(:position)
67
+
68
+ def initialize(theLexeme, aTerminal, aPosition)
69
+ super(theLexeme, aTerminal)
70
+ @position = aPosition
71
+ end
72
+ end
73
+
58
74
  ########################################
59
75
  # Step 4. Create a tokenizer
60
76
  # A tokenizer reads the input string and converts it into a sequence of tokens.
61
77
  # Remark: Rley doesn't provide tokenizer functionality.
62
78
  # Highly simplified tokenizer implementation
63
79
  def tokenizer(aTextToParse)
64
- tokens = aTextToParse.scan(/\S+/).map do |word|
80
+ scanner = StringScanner.new(aTextToParse)
81
+ tokens = []
82
+
83
+ loop do
84
+ scanner.skip(/\s+/)
85
+ curr_pos = scanner.pos
86
+ word = scanner.scan(/\S+/)
87
+ break unless word
88
+
65
89
  term_name = Lexicon[word]
66
90
  raise StandardError, "Word '#{word}' not found in lexicon" if term_name.nil?
67
- Rley::Lexical::Token.new(word, term_name)
91
+ pos = Position.new(1, curr_pos + 1)
92
+ tokens << NLPToken.new(word, term_name, pos)
68
93
  end
69
94
 
70
95
  return tokens
@@ -73,7 +98,7 @@ end
73
98
 
74
99
  ########################################
75
100
  # Step 5. Parse the input
76
- input_to_parse = 'John saw Mary with a telescope'
101
+ input_to_parse = 'John saw Mary with a '
77
102
  # input_to_parse = 'the dog saw a man in the park' # This one is ambiguous
78
103
  # Convert input text into a sequence of token objects...
79
104
  tokens = tokenizer(input_to_parse)
@@ -84,7 +109,7 @@ unless result.success?
84
109
  puts result.failure_reason.message
85
110
  exit(1)
86
111
  end
87
-
112
+
88
113
  ########################################
89
114
  # Step 6. Generating a parse tree from parse result
90
115
  ptree = engine.to_ptree(result)
@@ -3,7 +3,7 @@
3
3
 
4
4
  module Rley # Module used as a namespace
5
5
  # The version number of the gem.
6
- Version = '0.6.02'.freeze
6
+ Version = '0.6.03'.freeze
7
7
 
8
8
  # Brief description of the gem.
9
9
  Description = "Ruby implementation of the Earley's parsing algorithm".freeze
@@ -4,22 +4,22 @@ module Rley # Module used as a namespace
4
4
  # the likely cause of a parse error
5
5
  # detected by Rley.
6
6
  class ErrorReason
7
- # The position of the offending input token
8
- attr_reader(:position)
7
+ # @!attribute [r] rank
8
+ # @return [Fixnum] The rank number of the offending input token
9
+ attr_reader(:rank)
9
10
 
10
- # The failing production
11
- attr_reader(:production)
12
-
13
- def initialize(aPosition)
14
- @position = aPosition
11
+ # Constructor
12
+ # @param aPosition [Fixnum] The rank number of the offending input token
13
+ def initialize(aRank)
14
+ @rank = aRank
15
15
  end
16
16
 
17
- # Returns the result of invoking reason.to_s.
17
+ # @return [String] the result of invoking reason.to_s
18
18
  def message()
19
19
  return to_s
20
20
  end
21
21
 
22
- # Return this reason's class name and message
22
+ # @return [String] Return this reason's class name and message
23
23
  def inspect
24
24
  "#{self.class.name}: #{message}"
25
25
  end
@@ -49,8 +49,8 @@ module Rley # Module used as a namespace
49
49
  # The terminal symbols expected when error was occurred
50
50
  attr_reader(:expected_terminals)
51
51
 
52
- def initialize(aPosition, lastToken, expectedTerminals)
53
- super(aPosition)
52
+ def initialize(aRank, lastToken, expectedTerminals)
53
+ super(aRank)
54
54
  raise StandardError, 'Internal error: nil token' if lastToken.nil?
55
55
  @last_token = lastToken.dup
56
56
  @expected_terminals = expectedTerminals.dup
@@ -58,6 +58,11 @@ module Rley # Module used as a namespace
58
58
 
59
59
  protected
60
60
 
61
+ def position()
62
+ return last_token.position if last_token.respond_to?(:position)
63
+ rank + 1
64
+ end
65
+
61
66
  # Emit a text explaining the expected terminal symbols
62
67
  def expectations
63
68
  term_names = expected_terminals.map(&:name)
@@ -76,7 +81,7 @@ module Rley # Module used as a namespace
76
81
  class UnexpectedToken < ExpectationNotMet
77
82
  # Returns the reason's message.
78
83
  def to_s
79
- err_msg = "Syntax error at or near token #{position + 1} "
84
+ err_msg = "Syntax error at or near token #{position} "
80
85
  err_msg << ">>>#{last_token.lexeme}<<<\n"
81
86
  err_msg << expectations
82
87
  err_msg << ", found a '#{last_token.terminal.name}' instead."
@@ -91,7 +96,7 @@ module Rley # Module used as a namespace
91
96
  # Returns the reason's message.
92
97
  def to_s
93
98
  err_msg = "Premature end of input after '#{last_token.lexeme}'"
94
- err_msg << " at position #{position + 1}\n"
99
+ err_msg << " at position #{position}\n"
95
100
  err_msg << "#{expectations}."
96
101
 
97
102
  return err_msg
@@ -14,8 +14,8 @@ module Rley # Open this namespace to avoid module qualifier prefixes
14
14
  expect { NoInput.new }.not_to raise_error
15
15
  end
16
16
 
17
- it 'should know the error position' do
18
- expect(subject.position).to eq(0)
17
+ it 'should know the error token rank' do
18
+ expect(subject.rank).to eq(0)
19
19
  end
20
20
  end # context
21
21
 
@@ -50,7 +50,7 @@ module Rley # Open this namespace to avoid module qualifier prefixes
50
50
  end
51
51
 
52
52
  it 'should know the error position' do
53
- expect(subject.position).to eq(3)
53
+ expect(subject.rank).to eq(3)
54
54
  end
55
55
 
56
56
  it 'should know the expected terminals' do
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rley
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.02
4
+ version: 0.6.03
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dimitri Geshef
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2018-03-03 00:00:00.000000000 Z
11
+ date: 2018-03-10 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: coveralls