ruby_parser 3.0.0 → 3.19.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- checksums.yaml.gz.sig +0 -0
- data/.autotest +36 -19
- data/History.rdoc +1297 -0
- data/Manifest.txt +35 -7
- data/{README.txt → README.rdoc} +44 -14
- data/Rakefile +308 -110
- data/bin/ruby_parse +3 -1
- data/bin/ruby_parse_extract_error +36 -16
- data/compare/normalize.rb +218 -0
- data/debugging.md +190 -0
- data/gauntlet.md +107 -0
- data/lib/.document +1 -0
- data/lib/rp_extensions.rb +53 -0
- data/lib/rp_stringscanner.rb +33 -0
- data/lib/ruby20_parser.rb +10973 -0
- data/lib/ruby20_parser.y +2683 -0
- data/lib/ruby21_parser.rb +10980 -0
- data/lib/ruby21_parser.y +2700 -0
- data/lib/ruby22_parser.rb +11123 -0
- data/lib/ruby22_parser.y +2711 -0
- data/lib/ruby23_parser.rb +11132 -0
- data/lib/ruby23_parser.y +2713 -0
- data/lib/ruby24_parser.rb +11231 -0
- data/lib/ruby24_parser.y +2721 -0
- data/lib/ruby25_parser.rb +11231 -0
- data/lib/ruby25_parser.y +2721 -0
- data/lib/ruby26_parser.rb +11253 -0
- data/lib/ruby26_parser.y +2736 -0
- data/lib/ruby27_parser.rb +12980 -0
- data/lib/ruby27_parser.y +3324 -0
- data/lib/ruby30_parser.rb +13242 -0
- data/lib/ruby30_parser.y +3447 -0
- data/lib/ruby31_parser.rb +13622 -0
- data/lib/ruby31_parser.y +3481 -0
- data/lib/ruby3_parser.yy +3536 -0
- data/lib/ruby_lexer.rb +933 -1232
- data/lib/ruby_lexer.rex +185 -0
- data/lib/ruby_lexer.rex.rb +399 -0
- data/lib/ruby_lexer_strings.rb +638 -0
- data/lib/ruby_parser.rb +97 -3
- data/lib/ruby_parser.yy +3465 -0
- data/lib/ruby_parser_extras.rb +1216 -687
- data/test/test_ruby_lexer.rb +2249 -1092
- data/test/test_ruby_parser.rb +5156 -975
- data/test/test_ruby_parser_extras.rb +47 -77
- data/tools/munge.rb +250 -0
- data/tools/ripper.rb +44 -0
- data.tar.gz.sig +1 -1
- metadata +200 -155
- metadata.gz.sig +0 -0
- data/.gemtest +0 -0
- data/History.txt +0 -482
- data/lib/gauntlet_rubyparser.rb +0 -120
- data/lib/ruby18_parser.rb +0 -5747
- data/lib/ruby18_parser.y +0 -1873
- data/lib/ruby19_parser.rb +0 -6110
- data/lib/ruby19_parser.y +0 -2078
data/History.txt
DELETED
@@ -1,482 +0,0 @@
|
|
1
|
-
=== 3.0.0 / 2012-11-02
|
2
|
-
|
3
|
-
I've hit 99.967% success rate against 558k files! 3.6σ!! 182 files
|
4
|
-
failed to parse and spot checking them shows that they're pretty much
|
5
|
-
lost causes. I'm sure I missed some stuff, but it is more important at
|
6
|
-
this point to release a version to get more people using it in more
|
7
|
-
diverse ways than I can come up with.
|
8
|
-
|
9
|
-
* 3 minor enhancements:
|
10
|
-
|
11
|
-
* Added RBStringScanner#charpos using #string_to_pos.
|
12
|
-
* Added RBStringScanner#string_to_pos using String#byteslice.
|
13
|
-
* Optimized regexp used for escape chars in strings. (presidentbeef)
|
14
|
-
|
15
|
-
* 3 bug fixes:
|
16
|
-
|
17
|
-
* Fixed current_line and unread_many depending on StringScanner#pos
|
18
|
-
* Fixed parsing of 'a[] = b'
|
19
|
-
* Fixed the worst abuse of heredocs EVER. Just because it is valid doesn't mean you should.
|
20
|
-
|
21
|
-
=== 3.0.0.a10 / 2012-10-26
|
22
|
-
|
23
|
-
* 1 major enhancement:
|
24
|
-
|
25
|
-
* iter nodes are now structurally the same as defs. Block args are COMPLETELY different.
|
26
|
-
|
27
|
-
* 4 minor enhancements:
|
28
|
-
|
29
|
-
* 1.8 and 1.9 now treat f { |(...)| } differently, per MRI.
|
30
|
-
* 1.9: Added __ENCODING__ support. (witlessbird)
|
31
|
-
* 1.9: Finished coverage for masgn variants in block args
|
32
|
-
* 1.9: covered all forms of block args: optional, splat, block, and destructuring
|
33
|
-
|
34
|
-
=== 3.0.0.a9 / 2012-10-22
|
35
|
-
|
36
|
-
Holy crap! I've hit a 99.92% success rate against 558k files! 492
|
37
|
-
failed parses to go. NOW is the time to start testing ruby_parser
|
38
|
-
against your code!
|
39
|
-
|
40
|
-
* 16 minor enhancements:
|
41
|
-
|
42
|
-
* 1.9 parser: Finished magic encoding and utf-8 bom support.
|
43
|
-
* 1.9: Added leading dot support eg "a\n.b".
|
44
|
-
* 1.9: Added partial handling for *arg and opt=val args in block args.
|
45
|
-
* 1.9: OMFG Encoding is the worst thing ever. Overhauled encoding support once again. It is much better at doing last ditch guessing of the encoding of the source file.
|
46
|
-
* 1.9: added String#grep for legacy support.
|
47
|
-
* Added Sexp#block_pass?
|
48
|
-
* Added ability to delete slow files when they timeout.
|
49
|
-
* Added block_dup_check and refactored grammar code to use it for block arg handling.
|
50
|
-
* Added in_lex_state?(*states) to clean up lexer code.
|
51
|
-
* Added tentative support for optional block args. THIS IS SUBJECT TO CHANGE!!
|
52
|
-
* Added toggleable debugging to StackState to make my life easier
|
53
|
-
* All untested uses have not moved but have been laced with poison.
|
54
|
-
* Finally got good tests for most forms of double-block arg errors.
|
55
|
-
* Moved all _known_ uses of SyntaxError to RubyParser::SyntaxError
|
56
|
-
* f_block_optarg now always returns a block node.
|
57
|
-
* ruby_parse_extract_error uses Find.find instead of globbing so dot files aren't missed.
|
58
|
-
|
59
|
-
* 12 bug fixes:
|
60
|
-
|
61
|
-
* 1.9: Completely ignore IndexError in unread_many because... you know... it sucks.
|
62
|
-
* 1.9: Fixed lex state after lexing ? in trinary.
|
63
|
-
* 1.9: Fixed lex state in some ternarys.
|
64
|
-
* 1.9: Fixed parsing of "1 ? b('') : 2\na d: 3"... ARE YOU NOT GLAD?!?!
|
65
|
-
* Fix Timeout differences between 1.8 and 1.9 :(
|
66
|
-
* Fixed emacs-style encodings to deal with no whitespace.
|
67
|
-
* Fixed error message for bad % codes. (whitequark)
|
68
|
-
* Fixed lexing of :a==>b vs :a===b vs :a==>b. P.S. Your space bar is broken. chump.
|
69
|
-
* Fixed lexing of rare numeric formats.
|
70
|
-
* Fixed magic comment encodings on DOS files ... :/
|
71
|
-
* Fixed ruby_parse_extract_error to exit non-zero on errors.
|
72
|
-
* Removed Symbol#is_argument and switch usage to is_arg?
|
73
|
-
|
74
|
-
=== 3.0.0.a8 / 2012-09-26
|
75
|
-
|
76
|
-
* 1 bug fix:
|
77
|
-
|
78
|
-
* Remove method #d from global namespace. (alexch)
|
79
|
-
|
80
|
-
=== 3.0.0.a7 / 2012-09-21
|
81
|
-
|
82
|
-
* 3 minor enhancements:
|
83
|
-
|
84
|
-
* Reorganized ruby_parse_extract_error so it will start much faster with a bunch of glob directories
|
85
|
-
* RubyParserStuff#process takes optional time arg and raises Timeout::Error if it goes too long. You should rescue that, ya know...
|
86
|
-
* ruby_parse_extract_error now checks *.rake and Rakefile on dir scan.
|
87
|
-
|
88
|
-
* 3 bug fixes:
|
89
|
-
|
90
|
-
* 1.9: Fixed ternary state tracking so {a:f{f()},b:nil} won't trip up the lexer.
|
91
|
-
* Fixed optional values in block args (no20/no21)
|
92
|
-
* ruby_parse_extract_error skips non-files. Some asshats put .rb on their dirs. :(
|
93
|
-
|
94
|
-
=== 3.0.0.a6 / 2012-08-20
|
95
|
-
|
96
|
-
* 2 minor enhancements:
|
97
|
-
|
98
|
-
* 1.8: Added basic encoding support to 1.8 parser. Assumes -Ku.
|
99
|
-
* 1.9: Added encoding magic comment support to 1.9 parser.
|
100
|
-
|
101
|
-
* 8 bug fixes:
|
102
|
-
|
103
|
-
* 1.9: Fixed lexing of -1 w/in conditionals. yeah... I dunno.
|
104
|
-
* 1.9: Fixed parsing of a do | | end.
|
105
|
-
* 1.9: Fixed parsing of not(x).
|
106
|
-
* 1.9: Fixed parsing of op_asgn + rescue: 'a ||= b rescue nil'
|
107
|
-
* 1.9: added \r to the EOL backslash handler. dos files blow
|
108
|
-
* 1.9: hacked in a workaround for 1.9 specific regexps running in 1.8.
|
109
|
-
* Added #reset to RubyParser proxy class
|
110
|
-
* Fixed lexing of conditional w/ %() tokens
|
111
|
-
|
112
|
-
=== 3.0.0.a5 / 2012-07-31
|
113
|
-
|
114
|
-
* 5 bug fixes:
|
115
|
-
|
116
|
-
* 1.9: Fix construction of 'f(:x, y: nil,)' w/ trailing comma.
|
117
|
-
* 1.9: cleaned up lexing exit lex_state handling. Fixes bug parsing 'if f :x; end'
|
118
|
-
* 1.9: fixed building of right-leaning masgns: 'f { |a, (b, c)| }'
|
119
|
-
* 1.9: fixed lexing 'when *splat'
|
120
|
-
* 1.9: fixed lexing of regexps in whens
|
121
|
-
|
122
|
-
=== 3.0.0.a4 / 2012-07-26
|
123
|
-
|
124
|
-
* 10 minor enhancements:
|
125
|
-
|
126
|
-
* 'rake debug' defaults to 1.9 parser since that's all I'm doing these days
|
127
|
-
* 1.9: Fixed f { |(a, b, ...), ...| ... } handling.
|
128
|
-
* Added 'rake extract F=path' task to quickly extract errors from large files
|
129
|
-
* Added on_error handler to provide more readable error message.
|
130
|
-
* Aliased #process to #parse.
|
131
|
-
* Renamed #parse to #process (legacy name), added default path of '(string)'
|
132
|
-
* cleaned ruby_parse_extract_error output and fixed to 1.9 parser
|
133
|
-
* ruby_parse_extract_error expands shell globs from ARGV
|
134
|
-
* ruby_parse_extract_error should also capture RuntimeError
|
135
|
-
* yyerror(msg) now warns with the message instead of ignoring it.
|
136
|
-
|
137
|
-
* 3 bug fixes:
|
138
|
-
|
139
|
-
* 1.9: Fixed bug lexing/parsing [ in rhs.
|
140
|
-
* 1.9: Fixed f { |((a, b), c)| ... } handling
|
141
|
-
* 1.9: fixed newline handling during expr_value
|
142
|
-
|
143
|
-
=== 3.0.0.a3 / 2012-07-03
|
144
|
-
|
145
|
-
* 1 major enhancement:
|
146
|
-
|
147
|
-
* OMG! Implemented all the 1.9 arg crap and I went from 500 lines of
|
148
|
-
structural diff to 32!
|
149
|
-
|
150
|
-
* 17 minor enhancements:
|
151
|
-
|
152
|
-
* 1.9: !(...) is now a method call. ugh
|
153
|
-
* 1.9: Added __ENCODING__ keyword (returns Unsupported atm).
|
154
|
-
* 1.9: Added support for "a.()" thingy. whatever you call it. Have I ever mentioned that ruby has too much syntax? I mean really...
|
155
|
-
* 1.9: Moved kRETURN/kBREAK/kNEXT from command_call to command
|
156
|
-
* 1.9: Removed when_args and refactored into args production.
|
157
|
-
* 1.9: Support for not/! being a call to #!. why matz? why?!?
|
158
|
-
* 1.9: Support for trailing comma support in opt call args.
|
159
|
-
* 1.9: Switched keyword support to deal with 1.8 vs 1.9 differences.
|
160
|
-
* 1.9: refactored and created command_asgn
|
161
|
-
* 1.9: removed case/else from primary
|
162
|
-
* 1.9: reworked mlhs and family. Now horrible things like 'a, *, c = f' work
|
163
|
-
* 1.9: reworked top level rules from program to expr. Lots of lame refactorings in mri.
|
164
|
-
* Moved 'a[]' from primary down to method_call
|
165
|
-
* Removed 'args trailer' production from assoc_list (eg {1, 2}).
|
166
|
-
* Removed 1.9 support for : as a terminator in do/then
|
167
|
-
* RubyParser no longer subclasses the 18 parser, so I had to change some consts around.
|
168
|
-
* Undeprecated RubyParser - but now it is a compound parser that tries 1.9 first.
|
169
|
-
|
170
|
-
* 3 bug fixes:
|
171
|
-
|
172
|
-
* 1.9: fixed bang method name.
|
173
|
-
* Fixed case/when/splat handling on both 1.8 and 1.9.
|
174
|
-
* Removed lambda_body production from lambda
|
175
|
-
|
176
|
-
=== 3.0.0.a2 / 2012-06-19
|
177
|
-
|
178
|
-
* 1 minor enhancement:
|
179
|
-
|
180
|
-
* Updated to sexp_processor ~> 4.0
|
181
|
-
|
182
|
-
* 1 bug fix:
|
183
|
-
|
184
|
-
* Fixed new_module to remove scope/block. Not enough tests on module variance.
|
185
|
-
|
186
|
-
* 1 known issue:
|
187
|
-
|
188
|
-
* No parsing of "a.()" thingy. (reported by julian7) (patches welcome!)
|
189
|
-
|
190
|
-
=== 3.0.0.a1 / 2012-05-22
|
191
|
-
|
192
|
-
This is the first alpha release of the 3.0.0 series. It is probably
|
193
|
-
woefully incomplete, bug ridden, and hasn't showered in several days.
|
194
|
-
Please please please beat the crap out of it and send
|
195
|
-
bugs/patches/complaints/suggestions.
|
196
|
-
|
197
|
-
* 5 major enhancements:
|
198
|
-
|
199
|
-
* 1.9 parsing! Thanks to EVERYONE for submitting patches for this!
|
200
|
-
* Removed :arglist from everything but :op_asgn1
|
201
|
-
* Removed :block from resbody
|
202
|
-
* Removed :block from when
|
203
|
-
* Removed :block nodes inside of scope nodes (defn/defs/class/sclass).
|
204
|
-
* Removed :scope nodes in defn/defs/class/sclass nodes.
|
205
|
-
* (probably more sexp cleanup to come before 3.0.0 final)
|
206
|
-
|
207
|
-
* 25 minor enhancements:
|
208
|
-
|
209
|
-
* 1.9: Fix \!a. (wanabe)
|
210
|
-
* 1.9: Method calling with postargs. (wanabe)
|
211
|
-
* 1.9: Method definition with postargs. (wanabe)
|
212
|
-
* 1.9: Support lambda args without parentheses. (wanabe)
|
213
|
-
* Added R arg to `rake debug` to debug ruby straight up
|
214
|
-
* Added RubyParser, subclassing Ruby18Parser but warning on instantiation.
|
215
|
-
* Added backref_assign_error (needs tests)
|
216
|
-
* Added bin/ruby_parse_extract_error to help with error submissions
|
217
|
-
* Added debug task to help quickly get into a bug
|
218
|
-
* Added more 18 vs 19 lexing tests for ?c.
|
219
|
-
* Added ruby_parser.rb that pulls everything together in proper order.
|
220
|
-
* Added tLABEL. (brynary)
|
221
|
-
* Branched ruby_parser.y to ruby18_parser.y
|
222
|
-
* Fix to pass test_lambda_args_block__19 test. (mrmargolis)
|
223
|
-
* Got rid of one instance of unread_many. (Confusion)
|
224
|
-
* Moved everything from RubyParser to RubyParserStuff and included module in both.
|
225
|
-
* Refactored 1.9 args handling
|
226
|
-
* Refactored and added new_resbody to ruby_parser_extras.
|
227
|
-
* Refactored and added new_when
|
228
|
-
* Refactored tests infrastructure and added both 1.8 and 1.9 test branches.
|
229
|
-
* Removed unused methods: unread, begin_of_line? was_begin_of_line. (YAY!) (Confusion)
|
230
|
-
* Renamed ruby_parser.y to ruby19_parser.y
|
231
|
-
* RubyLexer now takes a version specifier.
|
232
|
-
* Started doing comparative refactoring between MRI's 1.9 parser and RP's. Shouldn't differ functionally except where we missed stuff in RP.
|
233
|
-
* `rake debug` prints the output if it succeeds.
|
234
|
-
|
235
|
-
* 21 bug fixes:
|
236
|
-
|
237
|
-
* Added missing gvar arg error. (1.8)
|
238
|
-
* Attach parser files to isolate to ensure they can build
|
239
|
-
* Conditionalize handling of tLABEL to ruby19+. Fixes issue #33.
|
240
|
-
* DOH. I deactivated some tests and never reactivated them. (Confusion ftw)
|
241
|
-
* Duplicate the input so that heredoc processing doesn't morph original. (banister)
|
242
|
-
* Entirely reworked block arg handling. (1.8)
|
243
|
-
* Fix ?x char literal. (nobu)
|
244
|
-
* Fixed 4/5 of literal lambda tests (jamie)
|
245
|
-
* Fixed deps for parser
|
246
|
-
* Fixed lexing of ?c for ruby 1.8 and 1.9.
|
247
|
-
* Fixed more Ruby 1.9 args tests (brynary)
|
248
|
-
* Fixed reswords to match MRI (1.8, 1.9)
|
249
|
-
* Fixed symbols with no spaces in method calls (e.g. foo:bar) (YAY! brynary)
|
250
|
-
* Fixed ternary_nil_no_space and other ternary edge cases for 1.9. (lastobelus)
|
251
|
-
* Fixed test_call_not_equal__19. First bug bounty! (albus522)
|
252
|
-
* Made lambda w/o arg list zero out the arg slot.
|
253
|
-
* Renamed awords to qwords to match stupid MRI naming. (1.8, 1.9) :(
|
254
|
-
* Rolled out brynary's symbols-no-spaces (foo:bar) changes when parsing 1.8 code
|
255
|
-
* Split 1.8 from 1.9 open paren lexer. Gawd that's ugly code.
|
256
|
-
* Split block_var from for_var. (1.8, 1.9)
|
257
|
-
* Use binread (and emulate in ruby 1.8) to avoid encoding issues
|
258
|
-
|
259
|
-
=== 2.3.1 / 2011-09-21
|
260
|
-
|
261
|
-
* 2 bug fixes:
|
262
|
-
|
263
|
-
* Fixed line numbers at end of special var+whitespace (larsch)
|
264
|
-
* Holy crap I was smokin' something good... Fixed 1.9.3 warning
|
265
|
-
|
266
|
-
=== 2.3.0 / 2011-09-06
|
267
|
-
|
268
|
-
* 2 minor enhancements:
|
269
|
-
|
270
|
-
* Add -g flag to parser compile if DEBUG
|
271
|
-
* Lexer now embeds line number in yacc_value for keywords, helping fix up line numbers
|
272
|
-
|
273
|
-
* 3 bug fixes:
|
274
|
-
|
275
|
-
* Fix method line numbers when no args and no parens (quix)
|
276
|
-
* Fixed line numbers on return/break/next w/ result expr. (pjnz)
|
277
|
-
* Fixed some lexing state in order to parse: 'f (1), 2' as 'f(1, 2)'. (invernizzi)
|
278
|
-
|
279
|
-
=== 2.2.0 / 2011-08-23
|
280
|
-
|
281
|
-
* 2 minor enhancements:
|
282
|
-
|
283
|
-
* Moved Keyword, Environment, and StackState inside of RubyParser
|
284
|
-
* Added proper dsym and dsym->sym support.
|
285
|
-
|
286
|
-
* 3 bug fixes:
|
287
|
-
|
288
|
-
* Added extra (failing) tests for call/iter line number checking (quix)
|
289
|
-
* Fixed line numbers for certain call/iter edge cases
|
290
|
-
* Fixed parsing of: alias :"<<" :">>".
|
291
|
-
|
292
|
-
=== 2.1.0 / 2011-08-15
|
293
|
-
|
294
|
-
* 2 minor enhancements:
|
295
|
-
|
296
|
-
* Added new accessor canonicalize_conditions to toggle conditional canonicalization (on by default). (confused)
|
297
|
-
* Awesome cleanup: Replaced call to append_block by block_append. (Confusion)
|
298
|
-
|
299
|
-
* 2 bug fixes:
|
300
|
-
|
301
|
-
* Fixed handling last line of =begin/=end. (raybaxter)
|
302
|
-
* Fixed source line numbers after heredocs. (jbarreneche)
|
303
|
-
|
304
|
-
=== 2.0.6 / 2011-02-18
|
305
|
-
|
306
|
-
* 1 minor enhancement:
|
307
|
-
|
308
|
-
* Switched to hoe's racc plugin to clean up rakefile and builds
|
309
|
-
|
310
|
-
* 1 bug fix:
|
311
|
-
|
312
|
-
* Fixed empty =begin/end.
|
313
|
-
|
314
|
-
=== 2.0.5 / 2010-09-01
|
315
|
-
|
316
|
-
* 1 minor enhancement:
|
317
|
-
|
318
|
-
* Started merging like lexical cases to try to squeeze some optimization out
|
319
|
-
|
320
|
-
=== 2.0.4 / 2009-08-18
|
321
|
-
|
322
|
-
* 1 minor enhancement:
|
323
|
-
|
324
|
-
* Changed requires around to be more accurate.
|
325
|
-
|
326
|
-
* 4 bug fixes:
|
327
|
-
|
328
|
-
* Fixed .autotest for minitest
|
329
|
-
* Fixed emacs escape lexing bug: "\C-\\" (maglev/gemstone)
|
330
|
-
* Fixed octal lexing edgecases. (maglev/gemstone)
|
331
|
-
* Fixed regexp lexing edgecases. (maglev/gemstone)
|
332
|
-
|
333
|
-
=== 2.0.3 / 2009-06-23
|
334
|
-
|
335
|
-
* 4 minor enhancements:
|
336
|
-
|
337
|
-
* Removed dead code handling d/regex match node cruft.
|
338
|
-
* Switched to minitest
|
339
|
-
* Updated .autotest and rakefile wrt rcov for new hoe capabilities
|
340
|
-
* Updated hoe for new capabilities
|
341
|
-
|
342
|
-
* 4 bug fixes:
|
343
|
-
|
344
|
-
* Environment#all now deals with strange edge cases in RAD.
|
345
|
-
* Fixed packaging/compilation issue.
|
346
|
-
* Minor 1.9 fixes
|
347
|
-
* hoe -> flay -> rubyparser rakefile circularity fixed
|
348
|
-
|
349
|
-
=== 2.0.2 / 2009-01-20
|
350
|
-
|
351
|
-
* 2 minor enhancements:
|
352
|
-
|
353
|
-
* Added gauntlet_rubyparser plugin. YAY for easy massive bug-hunting.
|
354
|
-
* Promoted Sexp's file/line/comments to sexp_processor.
|
355
|
-
|
356
|
-
* 4 bug fixes:
|
357
|
-
|
358
|
-
* Fixed and improved the readme
|
359
|
-
* Fixed lexing heredoc newlines.
|
360
|
-
* Fixed line numbers on defns.
|
361
|
-
* Fixed rdoc generation bug pointed out by hugh sasse (who rocks)
|
362
|
-
|
363
|
-
=== 2.0.1 / 2008-11-04
|
364
|
-
|
365
|
-
* 2 minor enhancements:
|
366
|
-
|
367
|
-
* Updated for changes to splat node in many contexts.
|
368
|
-
* Made PT a developer dep
|
369
|
-
|
370
|
-
=== 2.0.0 / 2008-10-22
|
371
|
-
|
372
|
-
* 1 major enhancement
|
373
|
-
|
374
|
-
* Brought on the AWESOME! 4x faster! no known lexing/parsing bugs!
|
375
|
-
|
376
|
-
* 71 minor enhancements
|
377
|
-
|
378
|
-
* 1.9: Added Fixnum#ord.
|
379
|
-
* 1.9: Added missing Regexp constants and did it so it'd work on 1.9.
|
380
|
-
* Added #store_comment and #comments
|
381
|
-
* Added StringScanner #begin_of_line?
|
382
|
-
* Added a bunch of tests for regexp escape chars, #parse_string, #read_escape, ? numbers, ? whitespace.
|
383
|
-
* Added a hack for rubinius' r2l eval bug.
|
384
|
-
* Added a new token type tSTRING that bypasses tSTRING_BEG/END entirely. Only does non-interpolated strings and then falls back to the old way. MUCH cleaner tho.
|
385
|
-
* Added bin/ruby_parse
|
386
|
-
* Added compare rule to Rakefile.
|
387
|
-
* Added coverage files/dirs to clean rule.
|
388
|
-
* Added file and line numbers to all sexp nodes. Column/ranges to come.
|
389
|
-
* Added lex_state change for lvars at the end of yylex.
|
390
|
-
* Added lexed comments to defn/defs/class/module nodes.
|
391
|
-
* Added stats gathering for yylex. Reordered yylex for avg data
|
392
|
-
* Added tSYMBOL token type and parser rule to speed up symbol lexing.
|
393
|
-
* Added tally output for getch, unread, and unread_many.
|
394
|
-
* Added tests for ambigous uminus/uplus, backtick in cmdarg, square and curly brackets, numeric gvars, eos edge cases, string quoting %<> and %%%.
|
395
|
-
* All cases throughout yylex now return directly if they match, no passthroughs.
|
396
|
-
* All lexer cases now slurp entire token in one swoop.
|
397
|
-
* All zarrays are now just empty arrays.
|
398
|
-
* Changed s(:block_arg, :blah) to :"&blah" in args sexp.
|
399
|
-
* Cleaned up lexer error handling. Now just raises all over.
|
400
|
-
* Cleaned up read_escape and regx_options
|
401
|
-
* Cleaned up tokadd_string (for some definition of cleaned).
|
402
|
-
* Converted single quoted strings to new tSTRING token type.
|
403
|
-
* Coverage is currently 94.4% on lexer.
|
404
|
-
* Done what I can to clean up heredoc lexing... still sucks.
|
405
|
-
* Flattened resbodies in rescue node. Fixed .autotest file.
|
406
|
-
* Folded lex_keywords back in now that it screams.
|
407
|
-
* Found very last instanceof ILiteralNode in the code. haha!
|
408
|
-
* Got the tests subclassing PTTC and cleaned up a lot. YAY
|
409
|
-
* Handle yield(*ary) properly
|
410
|
-
* MASSIVELY cleaned out =begin/=end comment processor.
|
411
|
-
* Massive overhaul on Keyword class. All hail the mighty Hash!
|
412
|
-
* Massively cleaned up ident= edge cases and fixed a stupid bug from jruby.
|
413
|
-
* Merged @/@@ scanner together, going to try to do the same everywhere.
|
414
|
-
* Refactored fix_arg_lex_state, common across the lexer.
|
415
|
-
* Refactored new_fcall into new_call.
|
416
|
-
* Refactored some code to get better profile numbers.
|
417
|
-
* Refactored some more #fix_arg_lex_state.
|
418
|
-
* Refactored tail of yylex into its own method.
|
419
|
-
* Removed Module#kill
|
420
|
-
* Removed Token, replaced with Sexp.
|
421
|
-
* Removed all parse_number and parse_quote tests.
|
422
|
-
* Removed argspush, argscat. YAY!
|
423
|
-
* Removed as many token_buffer.split(//)'s as possible. 1 to go.
|
424
|
-
* Removed begins from compstmts
|
425
|
-
* Removed buffer arg for tokadd_string.
|
426
|
-
* Removed crufty (?) solo '@' token... wtf was that anyhow?
|
427
|
-
* Removed most jruby/stringio cruft from StringScanner.
|
428
|
-
* Removed one unread_many... 2 to go. They're harder.
|
429
|
-
* Removed store_comment, now done directly.
|
430
|
-
* Removed token_buffer. Now I just use token ivar.
|
431
|
-
* Removed use of s() from lexer. Changed the way line numbers are gathered.
|
432
|
-
* Renamed *qwords to *awords.
|
433
|
-
* Renamed StringScanner to RPStringScanner (a subclass) to fix namespace trashing.
|
434
|
-
* Renamed parse to process and aliased to parse.
|
435
|
-
* Renamed token_buffer to string_buffer since that arcane shit still needs it.
|
436
|
-
* Resolved the rest of the lexing issues I brought up w/ ruby-core.
|
437
|
-
* Revamped tokadd_escape.
|
438
|
-
* Rewrote Keyword and KWtable.
|
439
|
-
* Rewrote RubyLexer using StringScanner.
|
440
|
-
* Rewrote tokadd_escape. 79 lines down to 21.
|
441
|
-
* Split out lib/ruby_parser_extras.rb so lexer is standalone.
|
442
|
-
* Started to clean up the parser and make it as skinny as possible
|
443
|
-
* Stripped out as much code as possible.
|
444
|
-
* Stripped yylex of some dead code.
|
445
|
-
* Switched from StringIO to StringScanner.
|
446
|
-
* Updated rakefile for new hoe.
|
447
|
-
* Uses pure ruby racc if ENV['PURE_RUBY'], otherwise use c.
|
448
|
-
* Wrote a ton of lexer tests. Coverage is as close to 100% as possible.
|
449
|
-
* Wrote args to clean up the big nasty args processing grammar section.
|
450
|
-
* lex_strterm is now a plain array, removed RubyLexer#s(...).
|
451
|
-
* yield and super now flatten args.
|
452
|
-
|
453
|
-
* 21+ bug fixes:
|
454
|
-
|
455
|
-
* I'm sure this list is missing a lot:
|
456
|
-
* Fixed 2 bugs both involving attrasgn (and ilk) esp when lhs is an array.
|
457
|
-
* Fixed a bug in the lexer for strings with single digit hex escapes.
|
458
|
-
* Fixed a bug parsing: a (args) { expr }... the space caused a different route to be followed and all hell broke loose.
|
459
|
-
* Fixed a bug with x\n=beginvar not putting begin back.
|
460
|
-
* Fixed attrasgn to have arglists, not arrays.
|
461
|
-
* Fixed bug in defn/defs with block fixing.
|
462
|
-
* Fixed class/module's name slot if colon2/3.
|
463
|
-
* Fixed dstr with empty interpolation body.
|
464
|
-
* Fixed for 1.9 string/char changes.
|
465
|
-
* Fixed lexer BS wrt determining token type of words.
|
466
|
-
* Fixed lexer BS wrt pass through values and lexing words. SO STUPID.
|
467
|
-
* Fixed lexing of floats.
|
468
|
-
* Fixed lexing of identifiers followed by equals. I hope.
|
469
|
-
* Fixed masgn with splat on lhs
|
470
|
-
* Fixed new_super to deal with block_pass correctly.
|
471
|
-
* Fixed parser's treatment of :colon2 and :colon3.
|
472
|
-
* Fixed regexp scanning of escaped numbers, ANY number is valid, not just octs.
|
473
|
-
* Fixed string scanning of escaped octs, allowing 1-3 chars.
|
474
|
-
* Fixed unescape for \n
|
475
|
-
* Fixed: omg this is stupid. '()' was returning bare nil
|
476
|
-
* Fixed: remove_begin now goes to the end, not sure why it didn't before.
|
477
|
-
|
478
|
-
=== 1.0.0 / 2007-12-20
|
479
|
-
|
480
|
-
* 1 major enhancement
|
481
|
-
* Birthday!
|
482
|
-
|
data/lib/gauntlet_rubyparser.rb
DELETED
@@ -1,120 +0,0 @@
|
|
1
|
-
#!/usr/bin/ruby -ws
|
2
|
-
|
3
|
-
$f ||= false
|
4
|
-
|
5
|
-
$:.unshift "../../ruby_parser/dev/lib"
|
6
|
-
$:.unshift "../../ruby2ruby/dev/lib"
|
7
|
-
|
8
|
-
require 'rubygems'
|
9
|
-
require 'ruby2ruby'
|
10
|
-
require 'ruby_parser'
|
11
|
-
|
12
|
-
require 'gauntlet'
|
13
|
-
|
14
|
-
class RubyParserGauntlet < Gauntlet
|
15
|
-
def initialize
|
16
|
-
super
|
17
|
-
|
18
|
-
self.data = Hash.new { |h,k| h[k] = {} }
|
19
|
-
old_data = load_yaml data_file
|
20
|
-
self.data.merge! old_data
|
21
|
-
end
|
22
|
-
|
23
|
-
def should_skip? name
|
24
|
-
if $f then
|
25
|
-
if Hash === data[name] then
|
26
|
-
! data[name].empty?
|
27
|
-
else
|
28
|
-
data[name]
|
29
|
-
end
|
30
|
-
else
|
31
|
-
data[name] == true # yes, == true on purpose
|
32
|
-
end
|
33
|
-
end
|
34
|
-
|
35
|
-
def diff_pp o1, o2
|
36
|
-
require 'pp'
|
37
|
-
|
38
|
-
File.open("/tmp/a.#{$$}", "w") do |f|
|
39
|
-
PP.pp o1, f
|
40
|
-
end
|
41
|
-
|
42
|
-
File.open("/tmp/b.#{$$}", "w") do |f|
|
43
|
-
PP.pp o2, f
|
44
|
-
end
|
45
|
-
|
46
|
-
`diff -u /tmp/a.#{$$} /tmp/b.#{$$}`
|
47
|
-
ensure
|
48
|
-
File.unlink "/tmp/a.#{$$}" rescue nil
|
49
|
-
File.unlink "/tmp/b.#{$$}" rescue nil
|
50
|
-
end
|
51
|
-
|
52
|
-
def broke name, file, msg
|
53
|
-
warn "bad"
|
54
|
-
self.data[name][file] = msg
|
55
|
-
self.dirty = true
|
56
|
-
end
|
57
|
-
|
58
|
-
def process path, name
|
59
|
-
begin
|
60
|
-
$stderr.print " #{path}: "
|
61
|
-
rp = RubyParser.new
|
62
|
-
r2r = Ruby2Ruby.new
|
63
|
-
|
64
|
-
old_ruby = File.read(path)
|
65
|
-
|
66
|
-
begin
|
67
|
-
old_sexp = rp.process old_ruby
|
68
|
-
rescue Racc::ParseError => e
|
69
|
-
self.data[name][path] = :unparsable
|
70
|
-
self.dirty = true
|
71
|
-
return
|
72
|
-
end
|
73
|
-
|
74
|
-
new_ruby = r2r.process old_sexp.deep_clone
|
75
|
-
|
76
|
-
begin
|
77
|
-
new_sexp = rp.process new_ruby
|
78
|
-
rescue Racc::ParseError => e
|
79
|
-
broke name, path, "couldn't parse new_ruby: #{e.message.strip}"
|
80
|
-
return
|
81
|
-
end
|
82
|
-
|
83
|
-
if old_sexp != new_sexp then
|
84
|
-
broke name, path, diff_pp(old_sexp, new_sexp)
|
85
|
-
return
|
86
|
-
end
|
87
|
-
|
88
|
-
self.data[name][path] = true
|
89
|
-
self.dirty = true
|
90
|
-
|
91
|
-
warn "good"
|
92
|
-
rescue Interrupt
|
93
|
-
puts "User cancelled"
|
94
|
-
exit 1
|
95
|
-
rescue Exception => e
|
96
|
-
broke name, path, " UNKNOWN ERROR: #{e}: #{e.message.strip}"
|
97
|
-
end
|
98
|
-
end
|
99
|
-
|
100
|
-
def run name
|
101
|
-
warn name
|
102
|
-
Dir["**/*.rb"].sort.each do |path|
|
103
|
-
next if path =~ /gemspec.rb/ # HACK
|
104
|
-
next if data[name][path] == true
|
105
|
-
process path, name
|
106
|
-
end
|
107
|
-
|
108
|
-
if self.data[name].values.all? { |v| v == true } then
|
109
|
-
warn " ALL GOOD!"
|
110
|
-
self.data[name] = true
|
111
|
-
self.dirty = true
|
112
|
-
end
|
113
|
-
end
|
114
|
-
end
|
115
|
-
|
116
|
-
filter = ARGV.shift
|
117
|
-
filter = Regexp.new filter if filter
|
118
|
-
|
119
|
-
gauntlet = RubyParserGauntlet.new
|
120
|
-
gauntlet.run_the_gauntlet filter
|