parser 0.9.alpha

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: ac470264d9dbf4e1781557f2e4c6f93050ca1fca
+  data.tar.gz: 2af0ef8f8dee1d84f91305adbbd6f099f99b5c8f
+SHA512:
+  metadata.gz: cceb3bc547346c33f28c9392971b0d6df75c289c3e5046bebac559bdf3a7334c902efae7d2c02e294d40930589ff27db011ab95afb72ff0f9f82af6851db2e0c
+  data.tar.gz: 0121d2b3fea37bac9b97db352c1cc55469cfa0ca854f0f913f0d8cb9015740e6799d9180a2a29c6a9715285e8acbc50d5717fafd8759de3966af222fd963a3b4

data/.autotest

@@ -0,0 +1,50 @@
+# -*- ruby -*-
+
+require 'autotest/restart'
+require 'autotest/isolate'
+require 'autotest/rcov' if ENV['RCOV']
+
+Autotest.add_hook :initialize do |at|
+  at.extra_files << "../../sexp_processor/dev/lib/pt_testcase.rb"
+  at.libs << ":../../sexp_processor/dev/lib"
+  at.add_exception 'unit'
+  at.add_exception 'coverage'
+  at.add_exception 'coverage.info'
+  at.add_exception '.diff'
+  at.add_exception 'rubycorpus'
+
+  dirs = Dir["corpus*"] + Dir["gauntlet*"]
+
+  dirs.each do |f|
+    at.add_exception f
+  end
+
+  at.libs << ':../../minitest/dev/lib'
+  at.testlib = "minitest/autorun"
+
+  at.add_mapping(/^lib\/.*\.y$/) do |f, _|
+    at.files_matching %r%^test/.*#{File.basename(f, '.y').gsub '_', '_?'}.rb$%
+  end
+
+  at.add_mapping(/pt_testcase.rb/) do |f, _|
+    at.files_matching(/test_.*rb$/)
+  end
+
+  %w(TestEnvironment TestStackState).each do |klass|
+    at.extra_class_map[klass] = "test/test_ruby_parser_extras.rb"
+  end
+
+  %w(TestRuby18Parser TestRuby19Parser TestParseTree).each do |klass| # HACK
+    at.extra_class_map[klass] = "test/test_ruby_parser.rb"
+  end
+end
+
+Autotest.add_hook :run_command do |at, _|
+  system "rake parser DEBUG=1"
+end
+
+class Autotest
+  def ruby
+    File.expand_path "~/.multiruby/install/1.9.0-0/bin/ruby"
+  end
+end if ENV['ONENINE']

data/History.txt

@@ -0,0 +1,558 @@
+=== 3.1.2 / 2013-03-18
+
+* 1 minor enhancement:
+
+  * OMG A SECURITY ISSUE FOR CODE NOBODY USES... EVER
+
+=== 3.1.1 / 2012-12-19
+
+* 1 minor enhancement:
+
+  * Added MOVE_TIMEOUT env var for ruby_parse_extract_error to move slow files to a sibling directory
+
+* 4 bug fixes:
+
+  * 1.9: Fixed lexing of "0o". (whitequark)
+  * 1.9: Fixed parsing of unary plus on literals. (whitequark)
+  * Added timeout arg to RubyParser#process to pass through to the real parser
+  * Updated Synopsis to reflect new options for running RP. (louismullie)
+
+=== 3.1.0 / 2012-12-06
+
+* 2 minor enhancements:
+
+  * Added RubyParser.for_current_ruby to provide a parser that matches your runtime. (neilconway)
+  * Duck-typed IDENT_CHAR_RE instead of using RUBY_VERSION
+
+* 3 bug fixes:
+
+  * Cleared out body comments in class/module/defn/defs
+  * Flipped lexer tests to US-ASCII to avoid encoding hell
+  * yyerror is now an alias for syntax_error
+
+=== 3.0.4 / 2012-11-26
+
+* 1 bug fix:
+
+  * RPStringScanner#lineno was still using byte offset, not char offset. (brynary)
+
+=== 3.0.3 / 2012-11-23
+
+* 1 minor enhancement:
+
+  * Improved error output on invalid char in expression.
+
+* 1 bug fix:
+
+  * Fixed lexing of no-name ivars. (whitequark)
+
+=== 3.0.2 / 2012-11-21
+
+52 down makes 99.9767% or 3.7σ. 130 files failed to parse out of 558k.
+
+* 4 minor enhancements:
+
+  * Added RP_TIMEOUT env var to override default timeout of 10 seconds.
+  * Minor optimization to RubyLexer#parse_number
+  * Only output parseerror output to stderr if $DEBUG.
+  * ruby_parse_extract_error modified to include 'it' blocks in its search.
+
+* 7 bug fixes:
+
+  * 1.9: Fixed args in dot-call forms (eg f.(...)).
+  * 1.9: Fixed lexing stabby lambda w/ do/end
+  * Deal better with DOS files. Ugh.
+  * Fix line number of production after heredoc.
+  * Fixed RubyParser#process to reuse parser instances across calls.
+  * Fixed line numbers for several productions.
+  * new_call sets line number to smallest line number of members.
+
+=== 3.0.1 / 2012-11-02
+
+* 2 bug fixes -- both based on MRI bugs... Yay?:
+
+  * 1.9.2 doesn't have String#byteslice... so I have to do the *HORRIBLE*.
+  * Lexing regexps w/ bad character encodings and no declared options will fall back to /.../n.
+
+=== 3.0.0 / 2012-11-02
+
+I've hit 99.967% success rate against 558k files! 3.6σ!! 182 files
+failed to parse and spot checking them shows that they're pretty much
+lost causes. I'm sure I missed some stuff, but it is more important at
+this point to release a version to get more people using it in more
+diverse ways than I can come up with.
+
+* 3 minor enhancements:
+
+  * Added RBStringScanner#charpos using #string_to_pos.
+  * Added RBStringScanner#string_to_pos using String#byteslice.
+  * Optimized regexp used for escape chars in strings. (presidentbeef)
+
+* 3 bug fixes:
+
+  * Fixed current_line and unread_many depending on StringScanner#pos
+  * Fixed parsing of 'a[] = b'
+  * Fixed the worst abuse of heredocs EVER. Just because it is valid doesn't mean you should.
+
+=== 3.0.0.a10 / 2012-10-26
+
+* 1 major enhancement:
+
+  * iter nodes are now structurally the same as defs. Block args are COMPLETELY different.
+
+* 4 minor enhancements:
+
+  * 1.8 and 1.9 now treat f { |(...)| } differently, per MRI.
+  * 1.9: Added __ENCODING__ support. (witlessbird)
+  * 1.9: Finished coverage for masgn variants in block args
+  * 1.9: covered all forms of block args: optional, splat, block, and destructuring
+
+=== 3.0.0.a9 / 2012-10-22
+
+Holy crap! I've hit a 99.92% success rate against 558k files! 492
+failed parses to go. NOW is the time to start testing ruby_parser
+against your code!
+
+* 16 minor enhancements:
+
+  * 1.9 parser: Finished magic encoding and utf-8 bom support.
+  * 1.9: Added leading dot support eg "a\n.b".
+  * 1.9: Added partial handling for *arg and opt=val args in block args.
+  * 1.9: OMFG Encoding is the worst thing ever. Overhauled encoding support once again. It is much better at doing last ditch guessing of the encoding of the source file. 
+  * 1.9: added String#grep for legacy support.
+  * Added Sexp#block_pass?
+  * Added ability to delete slow files when they timeout.
+  * Added block_dup_check and refactored grammar code to use it for block arg handling.
+  * Added in_lex_state?(*states) to clean up lexer code.
+  * Added tentative support for optional block args. THIS IS SUBJECT TO CHANGE!!
+  * Added toggleable debugging to StackState to make my life easier
+  * All untested uses have not moved but have been laced with poison. 
+  * Finally got good tests for most forms of double-block arg errors.
+  * Moved all _known_ uses of SyntaxError to RubyParser::SyntaxError
+  * f_block_optarg now always returns a block node.
+  * ruby_parse_extract_error uses Find.find instead of globbing so dot files aren't missed.
+
+* 12 bug fixes:
+
+  * 1.9: Completely ignore IndexError in unread_many because... you know... it sucks.
+  * 1.9: Fixed lex state after lexing ? in trinary.
+  * 1.9: Fixed lex state in some ternarys.
+  * 1.9: Fixed parsing of "1 ? b('') : 2\na d: 3"... ARE YOU NOT GLAD?!?!
+  * Fix Timeout differences between 1.8 and 1.9 :(
+  * Fixed emacs-style encodings to deal with no whitespace.
+  * Fixed error message for bad % codes. (whitequark)
+  * Fixed lexing of :a==>b vs :a===b vs :a==>b. P.S. Your space bar is broken. chump.
+  * Fixed lexing of rare numeric formats.
+  * Fixed magic comment encodings on DOS files ... :/
+  * Fixed ruby_parse_extract_error to exit non-zero on errors.
+  * Removed Symbol#is_argument and switch usage to is_arg?
+
+=== 3.0.0.a8 / 2012-09-26
+
+* 1 bug fix:
+
+  * Remove method #d from global namespace. (alexch)
+
+=== 3.0.0.a7 / 2012-09-21
+
+* 3 minor enhancements:
+
+  * Reorganized ruby_parse_extract_error so it will start much faster with a bunch of glob directories
+  * RubyParserStuff#process takes optional time arg and raises Timeout::Error if it goes too long. You should rescue that, ya know...
+  * ruby_parse_extract_error now checks *.rake and Rakefile on dir scan.
+
+* 3 bug fixes:
+
+  * 1.9: Fixed ternary state tracking so {a:f{f()},b:nil} won't trip up the lexer.
+  * Fixed optional values in block args (no20/no21)
+  * ruby_parse_extract_error skips non-files. Some asshats put .rb on their dirs. :(
+
+=== 3.0.0.a6 / 2012-08-20
+
+* 2 minor enhancements:
+
+  * 1.8: Added basic encoding support to 1.8 parser. Assumes -Ku.
+  * 1.9: Added encoding magic comment support to 1.9 parser.
+
+* 8 bug fixes:
+
+  * 1.9: Fixed lexing of -1 w/in conditionals. yeah... I dunno.
+  * 1.9: Fixed parsing of a do | | end.
+  * 1.9: Fixed parsing of not(x).
+  * 1.9: Fixed parsing of op_asgn + rescue: 'a ||= b rescue nil'
+  * 1.9: added \r to the EOL backslash handler. dos files blow
+  * 1.9: hacked in a workaround for 1.9 specific regexps running in 1.8.
+  * Added #reset to RubyParser proxy class
+  * Fixed lexing of conditional w/ %() tokens
+
+=== 3.0.0.a5 / 2012-07-31
+
+* 5 bug fixes:
+
+  * 1.9: Fix construction of 'f(:x, y: nil,)' w/ trailing comma.
+  * 1.9: cleaned up lexing exit lex_state handling. Fixes bug parsing 'if f :x; end'
+  * 1.9: fixed building of right-leaning masgns: 'f { |a, (b, c)| }'
+  * 1.9: fixed lexing 'when *splat'
+  * 1.9: fixed lexing of regexps in whens
+
+=== 3.0.0.a4 / 2012-07-26
+
+* 10 minor enhancements:
+
+  * 'rake debug' defaults to 1.9 parser since that's all I'm doing these days
+  * 1.9: Fixed f { |(a, b, ...), ...| ... } handling.
+  * Added 'rake extract F=path' task to quickly extract errors from large files
+  * Added on_error handler to provide more readable error message.
+  * Aliased #process to #parse.
+  * Renamed #parse to #process (legacy name), added default path of '(string)'
+  * cleaned ruby_parse_extract_error  output and fixed to 1.9 parser
+  * ruby_parse_extract_error expands shell globs from ARGV
+  * ruby_parse_extract_error should also capture RuntimeError
+  * yyerror(msg) now warns with the message instead of ignoring it.
+
+* 3 bug fixes:
+
+  * 1.9: Fixed bug lexing/parsing [ in rhs.
+  * 1.9: Fixed f { |((a, b), c)| ... } handling
+  * 1.9: fixed newline handling during expr_value
+
+=== 3.0.0.a3 / 2012-07-03
+
+* 1 major enhancement:
+
+  * OMG! Implemented all the 1.9 arg crap and I went from 500 lines of
+    structural diff to 32!
+
+* 17 minor enhancements:
+
+  * 1.9: !(...) is now a method call. ugh
+  * 1.9: Added __ENCODING__ keyword (returns Unsupported atm).
+  * 1.9: Added support for "a.()" thingy. whatever you call it. Have I ever mentioned that ruby has too much syntax? I mean really...
+  * 1.9: Moved kRETURN/kBREAK/kNEXT from command_call to command
+  * 1.9: Removed when_args and refactored into args production.
+  * 1.9: Support for not/! being a call to #!. why matz? why?!?
+  * 1.9: Support for trailing comma support in opt call args.
+  * 1.9: Switched keyword support to deal with 1.8 vs 1.9 differences.
+  * 1.9: refactored and created command_asgn
+  * 1.9: removed case/else from primary
+  * 1.9: reworked mlhs and family. Now horrible things like 'a, *, c = f' work
+  * 1.9: reworked top level rules from program to expr. Lots of lame refactorings in mri.
+  * Moved 'a[]' from primary down to method_call
+  * Removed 'args trailer' production from assoc_list (eg {1, 2}).
+  * Removed 1.9 support for : as a terminator in do/then
+  * RubyParser no longer subclasses the 18 parser, so I had to change some consts around.
+  * Undeprecated RubyParser - but now it is a compound parser that tries 1.9 first.
+
+* 3 bug fixes:
+
+  * 1.9: fixed bang method name.
+  * Fixed case/when/splat handling on both 1.8 and 1.9.
+  * Removed lambda_body production from lambda
+
+=== 3.0.0.a2 / 2012-06-19
+
+* 1 minor enhancement:
+
+  * Updated to sexp_processor ~> 4.0
+
+* 1 bug fix:
+
+  * Fixed new_module to remove scope/block. Not enough tests on module variance.
+
+* 1 known issue:
+
+  * No parsing of "a.()" thingy. (reported by julian7) (patches welcome!)
+
+=== 3.0.0.a1 / 2012-05-22
+
+This is the first alpha release of the 3.0.0 series. It is probably
+woefully incomplete, bug ridden, and hasn't showered in several days.
+Please please please beat the crap out of it and send
+bugs/patches/complaints/suggestions.
+
+* 5 major enhancements:
+
+  * 1.9 parsing! Thanks to EVERYONE for submitting patches for this!
+  * Removed :arglist from everything but :op_asgn1
+  * Removed :block from resbody
+  * Removed :block from when
+  * Removed :block nodes inside of scope nodes (defn/defs/class/sclass).
+  * Removed :scope nodes in defn/defs/class/sclass nodes.
+  * (probably more sexp cleanup to come before 3.0.0 final)
+
+* 25 minor enhancements:
+
+  * 1.9: Fix \!a. (wanabe)
+  * 1.9: Method calling with postargs. (wanabe)
+  * 1.9: Method definition with postargs. (wanabe)
+  * 1.9: Support lambda args without parentheses. (wanabe)
+  * Added R arg to `rake debug` to debug ruby straight up
+  * Added RubyParser, subclassing Ruby18Parser but warning on instantiation.
+  * Added backref_assign_error (needs tests)
+  * Added bin/ruby_parse_extract_error to help with error submissions
+  * Added debug task to help quickly get into a bug
+  * Added more 18 vs 19 lexing tests for ?c.
+  * Added ruby_parser.rb that pulls everything together in proper order.
+  * Added tLABEL. (brynary)
+  * Branched ruby_parser.y to ruby18_parser.y
+  * Fix to pass test_lambda_args_block__19 test. (mrmargolis)
+  * Got rid of one instance of unread_many. (Confusion)
+  * Moved everything from RubyParser to RubyParserStuff and included module in both.
+  * Refactored 1.9 args handling
+  * Refactored and added new_resbody to ruby_parser_extras.
+  * Refactored and added new_when
+  * Refactored tests infrastructure and added both 1.8 and 1.9 test branches.
+  * Removed unused methods: unread, begin_of_line? was_begin_of_line. (YAY!) (Confusion)
+  * Renamed ruby_parser.y to ruby19_parser.y
+  * RubyLexer now takes a version specifier.
+  * Started doing comparative refactoring between MRI's 1.9 parser and RP's. Shouldn't differ functionally except where we missed stuff in RP.
+  * `rake debug` prints the output if it succeeds.
+
+* 21 bug fixes:
+
+  * Added missing gvar arg error. (1.8)
+  * Attach parser files to isolate to ensure they can build
+  * Conditionalize handling of tLABEL to ruby19+. Fixes issue #33.
+  * DOH. I deactivated some tests and never reactivated them. (Confusion ftw)
+  * Duplicate the input so that heredoc processing doesn't morph original. (banister)
+  * Entirely reworked block arg handling. (1.8)
+  * Fix ?x char literal. (nobu)
+  * Fixed 4/5 of literal lambda tests (jamie)
+  * Fixed deps for parser
+  * Fixed lexing of ?c for ruby 1.8 and 1.9.
+  * Fixed more Ruby 1.9 args tests (brynary)
+  * Fixed reswords to match MRI (1.8, 1.9)
+  * Fixed symbols with no spaces in method calls (e.g. foo:bar) (YAY! brynary)
+  * Fixed ternary_nil_no_space and other ternary edge cases for 1.9. (lastobelus)
+  * Fixed test_call_not_equal__19. First bug bounty! (albus522)
+  * Made lambda w/o arg list zero out the arg slot.
+  * Renamed awords to qwords to match stupid MRI naming. (1.8, 1.9) :(
+  * Rolled out brynary's symbols-no-spaces (foo:bar) changes when parsing 1.8 code
+  * Split 1.8 from 1.9 open paren lexer. Gawd that's ugly code.
+  * Split block_var from for_var. (1.8, 1.9)
+  * Use binread (and emulate in ruby 1.8) to avoid encoding issues
+
+=== 2.3.1 / 2011-09-21
+
+* 2 bug fixes:
+
+  * Fixed line numbers at end of special var+whitespace (larsch)
+  * Holy crap I was smokin' something good... Fixed 1.9.3 warning
+
+=== 2.3.0 / 2011-09-06
+
+* 2 minor enhancements:
+
+  * Add -g flag to parser compile if DEBUG
+  * Lexer now embeds line number in yacc_value for keywords, helping fix up line numbers
+
+* 3 bug fixes:
+
+  * Fix method line numbers when no args and no parens (quix)
+  * Fixed line numbers on return/break/next w/ result expr. (pjnz)
+  * Fixed some lexing state in order to parse: 'f (1), 2' as 'f(1, 2)'. (invernizzi)
+
+=== 2.2.0 / 2011-08-23
+
+* 2 minor enhancements:
+
+  * Moved Keyword, Environment, and StackState inside of RubyParser
+  * Added proper dsym and dsym->sym support.
+
+* 3 bug fixes:
+
+  * Added extra (failing) tests for call/iter line number checking (quix)
+  * Fixed line numbers for certain call/iter edge cases
+  * Fixed parsing of: alias :"<<" :">>".
+
+=== 2.1.0 / 2011-08-15
+
+* 2 minor enhancements:
+
+  * Added new accessor canonicalize_conditions to toggle conditional canonicalization (on by default). (confused)
+  * Awesome cleanup: Replaced call to append_block by block_append. (Confusion)
+
+* 2 bug fixes:
+
+  * Fixed handling last line of =begin/=end. (raybaxter)
+  * Fixed source line numbers after heredocs. (jbarreneche)
+
+=== 2.0.6 / 2011-02-18
+
+* 1 minor enhancement:
+
+  * Switched to hoe's racc plugin to clean up rakefile and builds
+
+* 1 bug fix:
+
+  * Fixed empty =begin/end.
+
+=== 2.0.5 / 2010-09-01
+
+* 1 minor enhancement:
+
+  * Started merging like lexical cases to try to squeeze some optimization out
+
+=== 2.0.4 / 2009-08-18
+
+* 1 minor enhancement:
+
+  * Changed requires around to be more accurate.
+
+* 4 bug fixes:
+
+  * Fixed .autotest for minitest
+  * Fixed emacs escape lexing bug: "\C-\\" (maglev/gemstone)
+  * Fixed octal lexing edgecases. (maglev/gemstone)
+  * Fixed regexp lexing edgecases. (maglev/gemstone)
+
+=== 2.0.3 / 2009-06-23
+
+* 4 minor enhancements:
+
+  * Removed dead code handling d/regex match node cruft.
+  * Switched to minitest
+  * Updated .autotest and rakefile wrt rcov for new hoe capabilities
+  * Updated hoe for new capabilities
+
+* 4 bug fixes:
+
+  * Environment#all now deals with strange edge cases in RAD.
+  * Fixed packaging/compilation issue.
+  * Minor 1.9 fixes
+  * hoe -> flay -> rubyparser rakefile circularity fixed
+
+=== 2.0.2 / 2009-01-20
+
+* 2 minor enhancements:
+
+  * Added gauntlet_rubyparser plugin. YAY for easy massive bug-hunting.
+  * Promoted Sexp's file/line/comments to sexp_processor.
+
+* 4 bug fixes:
+
+  * Fixed and improved the readme
+  * Fixed lexing heredoc newlines.
+  * Fixed line numbers on defns.
+  * Fixed rdoc generation bug pointed out by hugh sasse (who rocks)
+
+=== 2.0.1 / 2008-11-04
+
+* 2 minor enhancements:
+
+  * Updated for changes to splat node in many contexts.
+  * Made PT a developer dep
+
+=== 2.0.0 / 2008-10-22
+
+* 1 major enhancement
+
+  * Brought on the AWESOME! 4x faster! no known lexing/parsing bugs!
+
+* 71 minor enhancements
+
+  * 1.9: Added Fixnum#ord.
+  * 1.9: Added missing Regexp constants and did it so it'd work on 1.9.
+  * Added #store_comment and #comments
+  * Added StringScanner #begin_of_line?
+  * Added a bunch of tests for regexp escape chars, #parse_string, #read_escape, ? numbers, ? whitespace.
+  * Added a hack for rubinius' r2l eval bug.
+  * Added a new token type tSTRING that bypasses tSTRING_BEG/END entirely. Only does non-interpolated strings and then falls back to the old way. MUCH cleaner tho.
+  * Added bin/ruby_parse
+  * Added compare rule to Rakefile.
+  * Added coverage files/dirs to clean rule.
+  * Added file and line numbers to all sexp nodes. Column/ranges to come.
+  * Added lex_state change for lvars at the end of yylex.
+  * Added lexed comments to defn/defs/class/module nodes.
+  * Added stats gathering for yylex. Reordered yylex for avg data
+  * Added tSYMBOL token type and parser rule to speed up symbol lexing.
+  * Added tally output for getch, unread, and unread_many.
+  * Added tests for ambigous uminus/uplus, backtick in cmdarg, square and curly brackets, numeric gvars, eos edge cases, string quoting %<> and %%%.
+  * All cases throughout yylex now return directly if they match, no passthroughs.
+  * All lexer cases now slurp entire token in one swoop.
+  * All zarrays are now just empty arrays.
+  * Changed s(:block_arg, :blah) to :"&blah" in args sexp.
+  * Cleaned up lexer error handling. Now just raises all over.
+  * Cleaned up read_escape and regx_options
+  * Cleaned up tokadd_string (for some definition of cleaned).
+  * Converted single quoted strings to new tSTRING token type.
+  * Coverage is currently 94.4% on lexer.
+  * Done what I can to clean up heredoc lexing... still sucks.
+  * Flattened resbodies in rescue node. Fixed .autotest file.
+  * Folded lex_keywords back in now that it screams.
+  * Found very last instanceof ILiteralNode in the code. haha!
+  * Got the tests subclassing PTTC and cleaned up a lot. YAY
+  * Handle yield(*ary) properly
+  * MASSIVELY cleaned out =begin/=end comment processor.
+  * Massive overhaul on Keyword class. All hail the mighty Hash!
+  * Massively cleaned up ident= edge cases and fixed a stupid bug from jruby.
+  * Merged @/@@ scanner together, going to try to do the same everywhere.
+  * Refactored fix_arg_lex_state, common across the lexer.
+  * Refactored new_fcall into new_call.
+  * Refactored some code to get better profile numbers.
+  * Refactored some more #fix_arg_lex_state.
+  * Refactored tail of yylex into its own method.
+  * Removed Module#kill
+  * Removed Token, replaced with Sexp.
+  * Removed all parse_number and parse_quote tests.
+  * Removed argspush, argscat. YAY!
+  * Removed as many token_buffer.split(//)'s as possible. 1 to go.
+  * Removed begins from compstmts
+  * Removed buffer arg for tokadd_string.
+  * Removed crufty (?) solo '@' token... wtf was that anyhow?
+  * Removed most jruby/stringio cruft from StringScanner.
+  * Removed one unread_many... 2 to go. They're harder.
+  * Removed store_comment, now done directly.
+  * Removed token_buffer. Now I just use token ivar.
+  * Removed use of s() from lexer. Changed the way line numbers are gathered.
+  * Renamed *qwords to *awords.
+  * Renamed StringScanner to RPStringScanner (a subclass) to fix namespace trashing.
+  * Renamed parse to process and aliased to parse.
+  * Renamed token_buffer to string_buffer since that arcane shit still needs it.
+  * Resolved the rest of the lexing issues I brought up w/ ruby-core.
+  * Revamped tokadd_escape.
+  * Rewrote Keyword and KWtable.
+  * Rewrote RubyLexer using StringScanner.
+  * Rewrote tokadd_escape. 79 lines down to 21.
+  * Split out lib/ruby_parser_extras.rb so lexer is standalone.
+  * Started to clean up the parser and make it as skinny as possible
+  * Stripped out as much code as possible.
+  * Stripped yylex of some dead code.
+  * Switched from StringIO to StringScanner.
+  * Updated rakefile for new hoe.
+  * Uses pure ruby racc if ENV['PURE_RUBY'], otherwise use c.
+  * Wrote a ton of lexer tests. Coverage is as close to 100% as possible.
+  * Wrote args to clean up the big nasty args processing grammar section.
+  * lex_strterm is now a plain array, removed RubyLexer#s(...).
+  * yield and super now flatten args.
+
+* 21 bug fixes:
+
+  * I'm sure this list is missing a lot:
+  * Fixed 2 bugs both involving attrasgn (and ilk) esp when lhs is an array.
+  * Fixed a bug in the lexer for strings with single digit hex escapes.
+  * Fixed a bug parsing: a (args) { expr }... the space caused a different route to be followed and all hell broke loose.
+  * Fixed a bug with x\n=beginvar not putting begin back.
+  * Fixed attrasgn to have arglists, not arrays.
+  * Fixed bug in defn/defs with block fixing.
+  * Fixed class/module's name slot if colon2/3.
+  * Fixed dstr with empty interpolation body.
+  * Fixed for 1.9 string/char changes.
+  * Fixed lexer BS wrt determining token type of words.
+  * Fixed lexer BS wrt pass through values and lexing words. SO STUPID.
+  * Fixed lexing of floats.
+  * Fixed lexing of identifiers followed by equals. I hope.
+  * Fixed masgn with splat on lhs
+  * Fixed new_super to deal with block_pass correctly.
+  * Fixed parser's treatment of :colon2 and :colon3.
+  * Fixed regexp scanning of escaped numbers, ANY number is valid, not just octs.
+  * Fixed string scanning of escaped octs, allowing 1-3 chars.
+  * Fixed unescape for \n
+  * Fixed: omg this is stupid. '()' was returning bare nil
+  * Fixed: remove_begin now goes to the end, not sure why it didn't before.
+
+=== 1.0.0 / 2007-12-20
+
+* 1 major enhancement
+  * Birthday!
+