gen-text 0.0.6 → 0.0.7

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -9,6 +9,23 @@ Install
9
9
  - Install [Ruby](http://ruby-lang.org) 1.9.3 or higher.
10
10
  - `gem install gen-text`
11
11
 
12
+ Example
13
+ -------
14
+
15
+ Simple grammar:
16
+
17
+ expr =
18
+ expr ('+'|'-') expr |
19
+ expr ('*'|'/') expr |
20
+ '(' expr ')' |
21
+ 0...1000 ;
22
+
23
+ Sample output:
24
+
25
+ (914-548*155+131*799)*827
26
+
27
+ See more examples in "sample" directory.
28
+
12
29
  Usage
13
30
  -----
14
31
 
@@ -69,8 +86,9 @@ You may use the following expressions in the rules' right part:
69
86
  <tt>'str'</tt>
70
87
  </td>
71
88
  <td>
72
- <p>Generate a string.</p>
73
- <p>Following escape sequences are allowed: "\n", "\t", "\e" and "\." where "." is an arbitrary character.</p>
89
+ Generate a string.
90
+ <p></p>
91
+ Following escape sequences are allowed: "\n", "\t", "\e" and "\." where "." is an arbitrary character.
74
92
  </td>
75
93
  </tr>
76
94
  <tr>
@@ -78,7 +96,7 @@ You may use the following expressions in the rules' right part:
78
96
  <td>Generate an UTF-8 character sequence corresponding to the Unicode code. E. g.: <tt>U+000A</tt> is equivalent to <tt>"\n"</tt>.</td>
79
97
  </tr>
80
98
  <tr>
81
- <td><tt>n</tt> (a number)</td>
99
+ <td><tt>n</tt> (a number)</td>
82
100
  <td>Generate a number</td>
83
101
  </tr>
84
102
  <tr>
@@ -87,7 +105,7 @@ You may use the following expressions in the rules' right part:
87
105
  </tr>
88
106
  <tr>
89
107
  <td><tt>nonterm</tt></td>
90
- <td>–</td>
108
+ <td>—</td>
91
109
  </tr>
92
110
  <tr>
93
111
  <td colspan="2"><center><strong>Combinators</strong></center></td>
@@ -104,33 +122,28 @@ You may use the following expressions in the rules' right part:
104
122
  </tr>
105
123
  <tr>
106
124
  <td>
107
- <tt>
108
- | expr <br/>
109
- | expr
110
- </tt>
125
+ <tt><pre>| expr
126
+ | expr</pre></tt>
111
127
  </td>
112
128
  <td>Random choice (another form).</td>
113
129
  </tr>
114
130
  <tr>
115
131
  <td>
116
- <tt>
117
- | [m%] expr <br/>
118
- | [n%] expr <br/>
119
- | expr
120
- </tt>
132
+ <tt><pre>| [m%] expr
133
+ | [n%] expr
134
+ | expr</pre></tt>
121
135
  </td>
122
136
  <td>
123
- <p>Random choice with specific probabilities.</p>
124
- <p>If probability is unspecified then it is calculated automatically.</p>
137
+ Random choice with specific probabilities.
138
+ <p></p>
139
+ If probability is unspecified then it is calculated automatically.
125
140
  </td>
126
141
  </tr>
127
142
  <tr>
128
143
  <td>
129
- <tt>
130
- | [0.1] expr <br/>
131
- | [0.3] expr <br/>
132
- | expr
133
- </tt>
144
+ <tt><pre>| [0.1] expr
145
+ | [0.3] expr
146
+ | expr</pre></tt>
134
147
  </td>
135
148
  <td>
136
149
  The same as above. Probabilities may be specified as floating point numbers between 0.0 and 1.0.
@@ -145,7 +158,7 @@ You may use the following expressions in the rules' right part:
145
158
  <tt>expr*[m...n]</tt> <br/>
146
159
  </td>
147
160
  <td>
148
- <p>Repeat <tt>expr</tt> many times:</p>
161
+ Repeat <tt>expr</tt> many times:
149
162
  <ul>
150
163
  <li>0 or more times</li>
151
164
  <li>1 or more times</li>
@@ -153,17 +166,39 @@ You may use the following expressions in the rules' right part:
153
166
  <li>exactly <tt>n</tt> times</li>
154
167
  <li>between <tt>m</tt> and <tt>n</tt> times</li>
155
168
  </ul>
156
- <p><strong>Note:</strong> you may use <tt>inf</tt> ("infinity") instead of <tt>m</tt> or <tt>n</tt>.</p>
169
+ <p></p>
170
+ <strong>Note:</strong> you may use <tt>inf</tt> ("infinity") instead of <tt>m</tt> or <tt>n</tt>.
171
+ </td>
172
+ </tr>
173
+ <tr>
174
+ <td>
175
+ <tt>n:expr</tt>
176
+ </td>
177
+ <td>
178
+ Capture a string generated by <tt>expr</tt> into variable <tt>n</tt>. The variable may then be used in <a href="#Code_insertions">Ruby code insertions</a>.
179
+ <p></p>
180
+ You may also use names like <tt>$n</tt> and <tt>@n</tt>.
181
+ <p></p>
182
+ <strong>Note</strong>: presence of this expression turns on backtracking and output buffering and may result in enormous memory usage.
157
183
  </td>
158
184
  </tr>
159
185
  <tr>
160
- <td colspan="2"><center><strong>Ruby code insertions</strong></center></td>
186
+ <td>
187
+ <tt>n := expr</tt>
188
+ </td>
189
+ <td>
190
+ The same as above but the output generated by <tt>expr</tt> is discarded.
191
+ </td>
192
+ </tr>
193
+ <tr>
194
+ <td colspan="2"><center><strong><a id="Code_insertions"></a>Ruby code insertions</strong></center></td>
161
195
  </tr>
162
196
  <tr>
163
197
  <td><tt>{ code }</tt></td>
164
198
  <td>
165
- <p>Execute the code. Generate nothing.</p>
166
- <p><strong>Note</strong>: all code insertions inside a rule share the same scope.</p>
199
+ Execute the code. Generate nothing.
200
+ <p></p>
201
+ <strong>Note</strong>: all code insertions inside a rule share the same scope.
167
202
  </td>
168
203
  </tr>
169
204
  <tr>
@@ -173,15 +208,14 @@ You may use the following expressions in the rules' right part:
173
208
  <tr>
174
209
  <td><tt>{? code }</tt></td>
175
210
  <td>
176
- <p>Condition. A code which must evaluate to true.</p>
177
- <p><strong>Note</strong>: presence of this expression turns on backtracking and output buffering and may result in enormous memory usage.</p>
211
+ Condition. A code which must evaluate to true.
212
+ <p></p>
213
+ <strong>Note</strong>: presence of this expression turns on backtracking and output buffering and may result in enormous memory usage.
178
214
  </td>
179
215
  </tr>
180
216
  </tbody>
181
217
  </table>
182
218
 
183
- TODO: Capture the generated output.
184
-
185
219
  ### Alternate syntax ###
186
220
 
187
221
  You may use "<-" instead of "=":
@@ -194,13 +228,8 @@ and "/" instead of "|":
194
228
 
195
229
  `simple choice` <- "a" / "b" ;
196
230
 
197
- Examples
198
- --------
199
-
200
- See them in "sample" directory.
201
-
202
231
  Links
203
232
  -----
204
233
 
205
- - [Documentation](http://www.rubydoc.info/gems/gen-text/0.0.5)
234
+ - [Documentation](http://www.rubydoc.info/gems/gen-text/0.0.7)
206
235
  - [Source code](https://github.com/LavirtheWhiolet/gen-text)
@@ -256,6 +256,19 @@ module GenText
256
256
 
257
257
  end
258
258
 
259
+ Capture = ASTNode.new :var_name, :expr, :discard_output do
260
+
261
+ def to_vm_code(context)
262
+ [
263
+ *generated_from(pos),
264
+ [:push_pos],
265
+ *expr.to_vm_code(context),
266
+ [:capture, context.rule_scope, var_name, discard_output]
267
+ ]
268
+ end
269
+
270
+ end
271
+
259
272
  Choice = ASTNode.new :alternatives do
260
273
 
261
274
  def to_vm_code(context)
@@ -372,8 +385,8 @@ module GenText
372
385
  end
373
386
 
374
387
  rule :seq do
375
- e = repeat and many {
376
- e2 = repeat and e = _(Seq[to_seq_subexprs(e) + to_seq_subexprs(e2)])
388
+ e = capture and many {
389
+ e2 = capture and e = _(Seq[to_seq_subexprs(e) + to_seq_subexprs(e2)])
377
390
  } and
378
391
  e
379
392
  end
@@ -385,6 +398,18 @@ module GenText
385
398
  end
386
399
  end
387
400
 
401
+ rule :capture do
402
+ _{
403
+ n = ruby_var_name and
404
+ d = (
405
+ _{ colon_eq and :discard_output } or
406
+ _{ colon and :do_not_discard_output }
407
+ ) and
408
+ e = repeat and _(Capture[n, e, (d == :discard_output)])
409
+ } or
410
+ _{ repeat }
411
+ end
412
+
388
413
  rule :repeat do
389
414
  e = primary and many {
390
415
  _{
@@ -456,6 +481,8 @@ module GenText
456
481
  token :rbracket, "]"
457
482
  token :dot, "."
458
483
  token :larrow, "<-"
484
+ token :colon, ":"
485
+ token :colon_eq, ":="
459
486
 
460
487
  # Parses "#{start} #{code_part} } #{whitespace_and_comments}".
461
488
  # Returns the code_part.
@@ -479,6 +506,10 @@ module GenText
479
506
  }.join
480
507
  end
481
508
 
509
+ token :ruby_var_name, "variable name" do
510
+ scan(/[\@\$]?[a-z_][a-z_0-9]*/)
511
+ end
512
+
482
513
  token :string do
483
514
  _{ string0('"') } or
484
515
  _{ string0("'") } or
data/lib/gen_text/vm.rb CHANGED
@@ -7,12 +7,15 @@ module GenText
7
7
  # @return [Boolean] true if +program+ may result in calling
8
8
  # {IO#pos=} and false otherwise.
9
9
  def self.may_set_out_pos?(program)
10
- program.any? { |instruction| instruction.first == :rescue_ }
10
+ program.any? do |instruction|
11
+ [:rescue_, :capture].include? instruction.first
12
+ end
11
13
  end
12
14
 
13
15
  # Executes +program+.
14
16
  #
15
- # After the execution the +out+ may contain garbage after its {IO#pos}.
17
+ # If +program+ may result in calling {IO#pos=} (see {VM::may_set_out_pos?}
18
+ # then after the execution the +out+ may contain garbage after its {IO#pos}.
16
19
  # It is up to the caller to truncate the garbage or to copy the useful data.
17
20
  #
18
21
  # @param program Array of <code>[:method_id, *args]</code>.
@@ -165,6 +168,32 @@ module GenText
165
168
  @pc += 1
166
169
  end
167
170
 
171
+ # - p := {#pop};
172
+ # - +binding_+.eval(set variable named +name+ to a substring of {#out} from
173
+ # p to current position);
174
+ # - if +discard_output+ is true then set {IO#pos} of {#out} to p.
175
+ #
176
+ # @param [Binding] binding_
177
+ # @param [String] name
178
+ # @param [Boolean] discard_output
179
+ # @return [void]
180
+ def capture(binding_, name, discard_output)
181
+ capture_start_pos = @stack.pop
182
+ capture_end_pos = @out.pos
183
+ captured_string = begin
184
+ @out.pos = capture_start_pos
185
+ @out.read(capture_end_pos - capture_start_pos)
186
+ end
187
+ @out.pos =
188
+ if discard_output then
189
+ capture_start_pos
190
+ else
191
+ capture_end_pos
192
+ end
193
+ binding_.eval("#{name} = #{captured_string.inspect}")
194
+ @pc += 1
195
+ end
196
+
168
197
  # {#push}(eval(+ruby_code+, +file+, +line+))
169
198
  #
170
199
  # @param [Binding] binding_
@@ -177,7 +206,15 @@ module GenText
177
206
  @pc += 1
178
207
  end
179
208
 
180
- # {#push}({#out}'s {IO#pos} and {#pc} as {RescuePoint})
209
+ # {#push}({#out}'s {IO#pos})
210
+ #
211
+ # @return [void]
212
+ def push_pos
213
+ @stack.push(@out.pos)
214
+ @pc += 1
215
+ end
216
+
217
+ # {#push}({#out}'s {IO#pos}, {#pc} as {RescuePoint})
181
218
  #
182
219
  # @param [Integer, nil] pc if specified then it is pushed instead of {#pc}.
183
220
  # @return [void]
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: gen-text
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.6
4
+ version: 0.0.7
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2016-06-26 00:00:00.000000000 Z
12
+ date: 2016-07-09 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: parse-framework