dolos 0.2.1 → 0.3.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +2 -0
- data/README.md +5 -8
- data/docs/.nojekyll +0 -0
- data/docs/README.md +22 -0
- data/docs/_sidebar.md +4 -0
- data/docs/getting_started.md +52 -0
- data/docs/index.html +26 -0
- data/examples/letter.rb +1 -2
- data/lib/dolos/parsers.rb +12 -1
- data/lib/dolos/version.rb +1 -1
- data/lib/dolos.rb +43 -8
- data/lib/{dolos_common_parsers/common_parsers.rb → parsers/common.rb} +7 -1
- metadata +10 -4
- /data/docs/{dolos_stable_diff.png → images/dolos_stable_diff.png} +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f8e2e4aa7f86848de041ffcdb29a21c552608d6e7e48617763102cbd6fe29319
|
4
|
+
data.tar.gz: d94587b9583bf071deddf3cdc04bf3b2f91cfbab8ab81d15da82c4b59351f72a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e5676cdf9a97bbcd49fb2e52d68ee935093e30b0dd78431ae83442cc3a8b522e57db4c90610500407e114ad88a517151f0ddb1e84f5c2897f009946064317e01
|
7
|
+
data.tar.gz: 7231d5bb051cb2edab89bdea12630c7a6bac4fd6fb2477524c04906acc90a81c3af82ed123bd8d08b5f3910ca65e6c985189256534222ad201f910312c7099b2
|
data/.rubocop.yml
ADDED
data/README.md
CHANGED
@@ -1,6 +1,9 @@
|
|
1
1
|
# Dolos
|
2
|
+
[![Gem version](https://badge.fury.io/rb/dolos.svg)](https://rubygems.org/gems/dolos)
|
3
|
+
[![Build status](https://github.com/benetis/dolos/actions/workflows/ruby.yml/badge.svg)](https://github.com/benetis/dolos/actions)
|
2
4
|
|
3
|
-
|
5
|
+
|
6
|
+
<img height="256" src="docs/images/dolos_stable_diff.png" width="256"/>
|
4
7
|
|
5
8
|
|
6
9
|
### Disclaimer
|
@@ -35,18 +38,12 @@ greet_and_speak.run("Hello, Parsers are great!") # <Result::Success>
|
|
35
38
|
|
36
39
|
```ruby
|
37
40
|
require 'dolos'
|
38
|
-
require 'dolos_common_parsers/common_parsers'
|
39
|
-
|
40
|
-
include Dolos
|
41
|
-
# frozen_string_literal: true
|
42
|
-
require_relative 'dolos'
|
43
|
-
require_relative 'dolos_common_parsers/common_parsers'
|
44
41
|
|
45
42
|
include Dolos
|
46
43
|
|
47
44
|
# Include common parsers
|
48
45
|
# In future this can be more structured, moved them to separate module to prevent breaking changes
|
49
|
-
include Dolos::
|
46
|
+
include Dolos::Common
|
50
47
|
|
51
48
|
# Library usage example
|
52
49
|
# Parse out a name and address from a letter
|
data/docs/.nojekyll
ADDED
File without changes
|
data/docs/README.md
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
# Dolos
|
2
|
+
|
3
|
+
## What is Dolos?
|
4
|
+
Dolos is parser combinator library for Ruby. It is inspired by FastParse and Scala Parser Combinators.
|
5
|
+
|
6
|
+
## What are parser combinators?
|
7
|
+
Parser combinators are a way to build parsers from smaller parsers. For example, you can build a parser for a number from a parser for a digit.
|
8
|
+
This is a very simple example, but it can be used to build more complex parsers.
|
9
|
+
Parsers are lazy and only run when needed. This allows to build complex parsers before passing input to them.
|
10
|
+
```ruby
|
11
|
+
hello = string("Hello")
|
12
|
+
greeting = hello >> c(" ") >> string("Ruby developer!")
|
13
|
+
greeting.run("Hello Ruby developer!") # => Success
|
14
|
+
```
|
15
|
+
|
16
|
+
## What's different from alternatives?
|
17
|
+
This library focuses on two things:
|
18
|
+
- Parsers integrate well into Ruby code. There is no need to keep them in separate classes.
|
19
|
+
- Fine grained control over parsers. You can `map` and adjust each parser separately
|
20
|
+
- Two ways of capturing values: traditional `>>`, other product operators to construct value and `capture!`
|
21
|
+
- For simple parsers `capture!` can be used to very quickly capture values into flat arrays
|
22
|
+
- Running parsers will not throw exceptions and instead return a result object. Exceptions don't play well with parsing.
|
data/docs/_sidebar.md
ADDED
@@ -0,0 +1,52 @@
|
|
1
|
+
# Getting started
|
2
|
+
|
3
|
+
## Installation
|
4
|
+
|
5
|
+
Install the gem and add it to your Gemfile:
|
6
|
+
```shell
|
7
|
+
$ bundle add dolos
|
8
|
+
```
|
9
|
+
Or manually:
|
10
|
+
```ruby
|
11
|
+
gem 'dolos'
|
12
|
+
```
|
13
|
+
|
14
|
+
## Usage
|
15
|
+
|
16
|
+
Two things to do:
|
17
|
+
- require library
|
18
|
+
- include module `Dolos` and `Dolos::Common`
|
19
|
+
|
20
|
+
```ruby
|
21
|
+
require 'dolos'
|
22
|
+
|
23
|
+
include Dolos
|
24
|
+
include Dolos::Common # Common parsers
|
25
|
+
```
|
26
|
+
|
27
|
+
### Basic parsers
|
28
|
+
|
29
|
+
A simple parser which matches one word.
|
30
|
+
|
31
|
+
```ruby
|
32
|
+
require 'dolos'
|
33
|
+
include Dolos
|
34
|
+
|
35
|
+
hello = c("Hello") # c("") is an alias for string(""). Can be read as: case-sensitive string match
|
36
|
+
|
37
|
+
hello.run("Hello").success? # => true
|
38
|
+
|
39
|
+
hello.run("hello").success? # => failure
|
40
|
+
```
|
41
|
+
|
42
|
+
After defining parser, it can be ran with `run('my-input')` method. It returns a `Result` object.
|
43
|
+
|
44
|
+
### Result
|
45
|
+
|
46
|
+
Result can be either `Success` or `Failure`. It can be checked with `success?` or `failure?` methods.
|
47
|
+
|
48
|
+
Success will also have `value` property which will contain the result of the parser. There is also `captures`, but
|
49
|
+
that's for later.
|
50
|
+
|
51
|
+
|
52
|
+
Failure will have `inspect` method which will return a string with the error message. It will show error position as well.
|
data/docs/index.html
ADDED
@@ -0,0 +1,26 @@
|
|
1
|
+
<!DOCTYPE html>
|
2
|
+
<html lang="en">
|
3
|
+
<head>
|
4
|
+
<meta charset="UTF-8">
|
5
|
+
<title>Document</title>
|
6
|
+
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
|
7
|
+
<meta name="description" content="Description">
|
8
|
+
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
|
9
|
+
<link rel="stylesheet" href="//cdn.jsdelivr.net/npm/docsify@4/lib/themes/vue.css">
|
10
|
+
</head>
|
11
|
+
<body>
|
12
|
+
<div id="app"></div>
|
13
|
+
<script>
|
14
|
+
window.$docsify = {
|
15
|
+
name: '',
|
16
|
+
repo: ''
|
17
|
+
}
|
18
|
+
window.$docsify = {
|
19
|
+
loadSidebar: true
|
20
|
+
}
|
21
|
+
</script>
|
22
|
+
<!-- Docsify v4 -->
|
23
|
+
<script src="//cdn.jsdelivr.net/npm/docsify@4"></script>
|
24
|
+
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-ruby.min.js"></script>
|
25
|
+
</body>
|
26
|
+
</html>
|
data/examples/letter.rb
CHANGED
@@ -1,12 +1,11 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
require 'dolos'
|
3
|
-
require 'dolos_common_parsers/common_parsers'
|
4
3
|
|
5
4
|
include Dolos
|
6
5
|
|
7
6
|
# Include common parsers
|
8
7
|
# In future this can be more structured, moved them to separate module to prevent breaking changes
|
9
|
-
include Dolos::
|
8
|
+
include Dolos::Common
|
10
9
|
|
11
10
|
# Library usage example
|
12
11
|
# Parse out a name and address from a letter
|
data/lib/dolos/parsers.rb
CHANGED
@@ -2,6 +2,11 @@
|
|
2
2
|
|
3
3
|
module Dolos
|
4
4
|
module Parsers
|
5
|
+
|
6
|
+
# String parser
|
7
|
+
# Matches exactly the given string
|
8
|
+
# string('hello').run('hello') => Success.new('hello', 5)
|
9
|
+
# Alias: c, for case-sensitive. Ex: c('hello').run('hello') => Success.new('hello', 5)
|
5
10
|
def string(str)
|
6
11
|
utf8_str = str.encode('UTF-8')
|
7
12
|
|
@@ -21,9 +26,12 @@ module Dolos
|
|
21
26
|
end
|
22
27
|
end
|
23
28
|
end
|
24
|
-
|
25
29
|
alias_method :c, :string
|
26
30
|
|
31
|
+
# Regex parser
|
32
|
+
# Accepts a regex, matches the regex against the input
|
33
|
+
# parser = regex(/\d+/)
|
34
|
+
# result = parser.run('123') # => Success.new('123', 3)
|
27
35
|
def regex(pattern)
|
28
36
|
Parser.new do |state|
|
29
37
|
state.input.mark_offset
|
@@ -41,6 +49,8 @@ module Dolos
|
|
41
49
|
end
|
42
50
|
end
|
43
51
|
|
52
|
+
# Matches any character
|
53
|
+
# any_char.run('a') # => Success.new('a', 1)
|
44
54
|
def any_char
|
45
55
|
Parser.new do |state|
|
46
56
|
state.input.mark_offset
|
@@ -62,6 +72,7 @@ module Dolos
|
|
62
72
|
end
|
63
73
|
|
64
74
|
# Matches any character in a string
|
75
|
+
# Passed string can be imagined as a set of characters
|
65
76
|
# Example:
|
66
77
|
# char_in('abc').run('b') # => Success.new('b', 1)
|
67
78
|
def char_in(characters_string)
|
data/lib/dolos/version.rb
CHANGED
data/lib/dolos.rb
CHANGED
@@ -5,6 +5,7 @@ require_relative "dolos/parser_state"
|
|
5
5
|
require_relative "dolos/result"
|
6
6
|
require_relative "dolos/string_io_wrapper"
|
7
7
|
require_relative "dolos/parsers"
|
8
|
+
require_relative 'parsers/common'
|
8
9
|
|
9
10
|
module Dolos
|
10
11
|
include Parsers
|
@@ -15,16 +16,24 @@ module Dolos
|
|
15
16
|
@parser_proc = block
|
16
17
|
end
|
17
18
|
|
19
|
+
# Run the parser with the given input
|
20
|
+
# Returns a Result<Success|Failure>
|
21
|
+
# string("hello").run("hello") => Success.new("hello", 5)
|
18
22
|
def run(input)
|
19
23
|
run_with_state(ParserState.new(input))
|
20
24
|
end
|
21
25
|
|
26
|
+
|
22
27
|
def run_with_state(state)
|
23
28
|
result = @parser_proc.call(state)
|
24
29
|
state.last_success_position = state.input.offset if result.success?
|
25
30
|
result
|
26
31
|
end
|
27
32
|
|
33
|
+
# Capture the result of the parser
|
34
|
+
# p = string("hello").capture!
|
35
|
+
# p.run("hello").captures => ["hello"]
|
36
|
+
# Captures is a flat array of all captured values
|
28
37
|
def capture!(wrap_in = nil)
|
29
38
|
Parser.new do |state|
|
30
39
|
result = run_with_state(state)
|
@@ -32,7 +41,10 @@ module Dolos
|
|
32
41
|
end
|
33
42
|
end
|
34
43
|
|
35
|
-
#
|
44
|
+
# Map the captures of the parser
|
45
|
+
# p = string("hello").map_captures { |captures| captures.map(&:upcase) }
|
46
|
+
# p.run("hello") => Success.new("hello", 5, ["HELLO"])
|
47
|
+
# This only maps over captures, not the value
|
36
48
|
def map_captures(&block)
|
37
49
|
Parser.new do |state|
|
38
50
|
result = run_with_state(state)
|
@@ -40,7 +52,9 @@ module Dolos
|
|
40
52
|
end
|
41
53
|
end
|
42
54
|
|
43
|
-
#
|
55
|
+
# Map the result of the parser
|
56
|
+
# p = string("hello").map { |s| s.upcase }
|
57
|
+
# p.run("hello") => Success.new("HELLO", 5)
|
44
58
|
def map(&block)
|
45
59
|
Parser.new do |state|
|
46
60
|
result = run_with_state(state)
|
@@ -48,6 +62,7 @@ module Dolos
|
|
48
62
|
end
|
49
63
|
end
|
50
64
|
|
65
|
+
# Combine the result of the parser with another parser
|
51
66
|
def combine(&block)
|
52
67
|
Parser.new do |state|
|
53
68
|
result = run_with_state(state)
|
@@ -62,12 +77,10 @@ module Dolos
|
|
62
77
|
end
|
63
78
|
end
|
64
79
|
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
end
|
70
|
-
|
80
|
+
# Combine the result of the parser with another parser
|
81
|
+
# Has an alias of `&`
|
82
|
+
# p = string("hello") & string("world")
|
83
|
+
# p.run("helloworld") => Success.new(["hello", "world"], 10)
|
71
84
|
def product(other_parser)
|
72
85
|
combine do |value1, capture1|
|
73
86
|
other_parser.map do |value2|
|
@@ -79,6 +92,10 @@ module Dolos
|
|
79
92
|
end
|
80
93
|
alias_method :&, :product
|
81
94
|
|
95
|
+
|
96
|
+
# Combine the result of the parser with another parser
|
97
|
+
# Discards the result of the second parser
|
98
|
+
# p = string("hello") << string("world")
|
82
99
|
def product_l(other_parser)
|
83
100
|
combine do |value1, capture1|
|
84
101
|
other_parser.map do |_|
|
@@ -89,6 +106,9 @@ module Dolos
|
|
89
106
|
end
|
90
107
|
end
|
91
108
|
|
109
|
+
# Combine the result of the parser with another parser
|
110
|
+
# Discards the result of the first parser
|
111
|
+
# p = string("hello") >> string("world")
|
92
112
|
def product_r(other_parser)
|
93
113
|
combine do |_, capture1|
|
94
114
|
other_parser.map do |value2|
|
@@ -102,6 +122,10 @@ module Dolos
|
|
102
122
|
alias_method :<<, :product_l
|
103
123
|
alias_method :>>, :product_r
|
104
124
|
|
125
|
+
# Combine the result of the parser with another parser
|
126
|
+
# If the first parser fails, it will try the second parser
|
127
|
+
# p = string("hello") | string("world") | string("!")
|
128
|
+
# p.run("hello") => Success.new("hello", 5)
|
105
129
|
def choice(other_parser)
|
106
130
|
Parser.new do |state|
|
107
131
|
result = run_with_state(state)
|
@@ -114,6 +138,9 @@ module Dolos
|
|
114
138
|
end
|
115
139
|
alias_method :|, :choice
|
116
140
|
|
141
|
+
|
142
|
+
# Repeat the parser n times
|
143
|
+
# Separator is optional, its another parser that will be run between each repetition
|
117
144
|
# rep0 # 0 or more
|
118
145
|
# rep # 1 or more
|
119
146
|
# rep(n = 2) # exactly 2
|
@@ -156,11 +183,17 @@ module Dolos
|
|
156
183
|
end
|
157
184
|
end
|
158
185
|
end
|
186
|
+
|
187
|
+
# Repeat the parser zero or more times
|
188
|
+
# c(" ").rep0.run(" ") => Success.new([" ", " ", " "], 3)
|
159
189
|
def zero_or_more
|
160
190
|
repeat(n_min: 0, n_max: Float::INFINITY)
|
161
191
|
end
|
162
192
|
alias_method :rep0, :zero_or_more
|
163
193
|
|
194
|
+
# Repeat the parser one or more times
|
195
|
+
# Same as rep0, but must match at least once
|
196
|
+
# c(" ").rep.run("A") => Failure.new("...")
|
164
197
|
def one_or_more(exactly = nil)
|
165
198
|
if exactly.nil?
|
166
199
|
repeat(n_min: 1, n_max: Float::INFINITY)
|
@@ -170,6 +203,8 @@ module Dolos
|
|
170
203
|
end
|
171
204
|
alias_method :rep, :one_or_more
|
172
205
|
|
206
|
+
# Make parser optional
|
207
|
+
# c(" ").opt.run("A") => Success.new([], 0)
|
173
208
|
def optional
|
174
209
|
Parser.new do |state|
|
175
210
|
result = run_with_state(state.dup)
|
@@ -1,7 +1,13 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
module Dolos
|
4
|
-
|
4
|
+
# Common parsers
|
5
|
+
# Separated from the main library to improve them later on
|
6
|
+
# These will change, new ones will be added. Once API stabilises, we will see what to do
|
7
|
+
# We have to be careful what is in the scope when we include this main module
|
8
|
+
# Probably a package of parsers following some RFC will be added as well.
|
9
|
+
# Keeping them separate for now
|
10
|
+
module Common
|
5
11
|
def ws
|
6
12
|
regex(/\s/)
|
7
13
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: dolos
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.3.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- benetis
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-08-
|
11
|
+
date: 2023-08-23 00:00:00.000000000 Z
|
12
12
|
dependencies: []
|
13
13
|
description: Parser combinators library for Ruby. In active development, not stable
|
14
14
|
yet.
|
@@ -19,6 +19,7 @@ extensions: []
|
|
19
19
|
extra_rdoc_files: []
|
20
20
|
files:
|
21
21
|
- ".rspec"
|
22
|
+
- ".rubocop.yml"
|
22
23
|
- CHANGELOG.md
|
23
24
|
- LICENSE.txt
|
24
25
|
- README.md
|
@@ -27,7 +28,12 @@ files:
|
|
27
28
|
- benchmarks/json/nested_json_166.json
|
28
29
|
- benchmarks/json/nested_json_1m.json
|
29
30
|
- benchmarks/letter.rb
|
30
|
-
- docs
|
31
|
+
- docs/.nojekyll
|
32
|
+
- docs/README.md
|
33
|
+
- docs/_sidebar.md
|
34
|
+
- docs/getting_started.md
|
35
|
+
- docs/images/dolos_stable_diff.png
|
36
|
+
- docs/index.html
|
31
37
|
- examples/letter.rb
|
32
38
|
- lib/dolos.rb
|
33
39
|
- lib/dolos/parser_state.rb
|
@@ -35,7 +41,7 @@ files:
|
|
35
41
|
- lib/dolos/result.rb
|
36
42
|
- lib/dolos/string_io_wrapper.rb
|
37
43
|
- lib/dolos/version.rb
|
38
|
-
- lib/
|
44
|
+
- lib/parsers/common.rb
|
39
45
|
- sig/dolos.rbs
|
40
46
|
- sig/dolos/common_parsers.rbs
|
41
47
|
- sig/dolos/parser.rbs
|
File without changes
|