lexr 0.1.0 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (3) hide show
  1. data/README.md +13 -5
  2. data/lib/lexr.rb +23 -8
  3. metadata +4 -4
data/README.md CHANGED
@@ -1,8 +1,15 @@
1
- ## Lexr
1
+ # Lexr
2
2
 
3
3
  Lexr is a lightweight lexical analyser written in ruby, it has no dependencies, has good test coverage, looks pretty and reads well.
4
4
 
5
- # An example: Expressions
5
+ Install with
6
+
7
+ gem install lexr
8
+
9
+ ## An example: Expressions
10
+
11
+ require 'rubygems'
12
+ require 'lexr'
6
13
 
7
14
  ExpressionLexer = Lexr.that {
8
15
  ignores /\s+/ => :whitespace
@@ -17,8 +24,8 @@ Lexr is a lightweight lexical analyser written in ruby, it has no dependencies,
17
24
 
18
25
  lexer = ExpressionLexer.new("1 * 12.5 / (55 + 2 - 56)")
19
26
 
20
- while (token = lexer.next) != Lexr::Token.end
21
- puts token
27
+ until lexer.end?
28
+ puts lexer.next
22
29
  end
23
30
 
24
31
  results in an output of
@@ -34,9 +41,10 @@ results in an output of
34
41
  subtraction(-)
35
42
  number(56.0)
36
43
  right_parenthesis())
44
+ end()
37
45
 
38
46
  if you added a % in there somewhere, you'd get a Lexr::UnmatchableTextError with a message like this:
39
47
 
40
48
  => Unexpected character '%' at position 5
41
49
 
42
- and that pretty is every feature so far. Please let me know of any bugs or additions that you'd like to see!
50
+ and that is pretty much every feature so far. Please let me know of any bugs or additions that you'd like to see!
data/lib/lexr.rb CHANGED
@@ -7,36 +7,51 @@ class Lexr
7
7
 
8
8
  def initialize(text, rules)
9
9
  @text, @rules = text, rules
10
+ @current = nil
10
11
  @position = 0
11
12
  end
12
13
 
13
- def next
14
- return Lexr::Token.end if @position >= @text.length
14
+ def next(peeking = false)
15
+ return @current = Lexr::Token.end if @position >= @text.length
15
16
  @rules.each do |rule|
16
- if result = rule.pattern.instance_of?(Regexp) ? regexp_match(rule.pattern) : literal_match(rule.pattern)
17
+ if result = rule.pattern.instance_of?(Regexp) ? regexp_match(rule.pattern, peeking) : literal_match(rule.pattern, peeking)
17
18
  result = rule.converter[result] if rule.converter
18
19
  return self.send(:next) if rule.ignore?
19
- return Lexr::Token.new(result, rule.symbol)
20
+ token = Lexr::Token.new(result, rule.symbol)
21
+ return @current = token unless peeking
22
+ return token
20
23
  end
21
24
  end
22
25
  raise Lexr::UnmatchableTextError.new(unprocessed_text[0..0], @position)
23
26
  end
24
27
 
28
+ def peek
29
+ self.send(:next, true)
30
+ end
31
+
32
+ def current
33
+ @current
34
+ end
35
+
36
+ def end?
37
+ @current == Lexr::Token.end
38
+ end
39
+
25
40
  private
26
41
 
27
42
  def unprocessed_text
28
43
  @text[@position..-1]
29
44
  end
30
45
 
31
- def regexp_match(regex)
46
+ def regexp_match(regex, peeking = false)
32
47
  return nil unless m = unprocessed_text.match(/^#{regex}/)
33
- @position += m.end(0)
48
+ @position += m.end(0) unless peeking
34
49
  m[0]
35
50
  end
36
51
 
37
- def literal_match(lit)
52
+ def literal_match(lit, peeking = false)
38
53
  return nil unless unprocessed_text[0..lit.length-1] == lit
39
- @position += lit.length
54
+ @position += lit.length unless peeking
40
55
  lit
41
56
  end
42
57
 
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: lexr
3
3
  version: !ruby/object:Gem::Version
4
- hash: 27
4
+ hash: 23
5
5
  prerelease: false
6
6
  segments:
7
7
  - 0
8
- - 1
8
+ - 2
9
9
  - 0
10
- version: 0.1.0
10
+ version: 0.2.0
11
11
  platform: ruby
12
12
  authors:
13
13
  - Michael Baldry
@@ -15,7 +15,7 @@ autorequire:
15
15
  bindir: bin
16
16
  cert_chain: []
17
17
 
18
- date: 2010-11-12 00:00:00 +00:00
18
+ date: 2010-11-24 00:00:00 +00:00
19
19
  default_executable:
20
20
  dependencies:
21
21
  - !ruby/object:Gem::Dependency