ruby_parser 3.18.1 → 3.19.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: dad59e40690dfb746fea4399e13d05f97b43cf562c2559c67d1f9b5868da86c5
-  data.tar.gz: 0c14605ecf34929f6d4865c20f085b6ef2a35533a70e530f261f9d04f44998cb
+  metadata.gz: 5e8b4b78bfb538253820b4ff34d49ebf7a91f9722c0ecfb63e3dadb3067530cb
+  data.tar.gz: e8e6e9cc42513094304d79a10fb4802706b10cb147084683c5dd3315c2ecde9a
 SHA512:
-  metadata.gz: 4fa69e55ac577e55fa2945ef9c135b47359a646e654bc8cf9f2ead973852adb97350cf67ac5da549592ea583f4824cd5f6930a167fa19cf3f914cb6d6888093d
-  data.tar.gz: 0c13eb02ffaf3b3c60ef358be2fa8f70b951e947170d3eeb8a94e5c37bec5dc8282376e4ab3f03de99e49b6e2aac2aa13169b31c30de1067e1a290c1fd65803a
+  metadata.gz: dd72b30995fbc5ad74f04dedc341572d03b73b962ddd2dd3ecd255c8d1babe1b4e190b780588b846f2c8b8942327ac4a50bda0e8401d02037299f9b16965e69b
+  data.tar.gz: 5793d0f9bf5e11f56db8eb5b53fd4b603664e72818b4bc43a3bfb61cf3c4a0c61f9ecee3757350a684271ae9e0c434782ef739ef0f075b3897fa338bcd2680c5

data/History.rdoc

@@ -1,3 +1,24 @@
+=== 3.19.0 / 2022-03-29
+
+* 1 major enhancement:
+
+  * Added tentative 3.1 support.
+
+* 7 minor enhancements:
+
+  * 3.1: bare RHS assoc: { y: } => s(:hash, s(:lit, :y), nil)
+  * 3.1: calls w/ unnamed block args (bare &)
+  * 3.1: endless defn/defs w/ paren-less calls (aka commands)
+  * 3.1: pattern capture to nonlocal vars, eg: ^@a, ^$b, ^@@c
+  * 3.1: pattern: ^(expr) => expr
+  * Improved steps for adding new versions.
+  * Improved steps for running gauntlets.
+
+* 2 bug fixes:
+
+  * Bumped 2.6+ cached versions for rake compare.
+  * Skip test_regexp_esc_C_slash on ruby 3.1.0 because of MRI bug.
+
 === 3.18.1 / 2021-11-10
 
 * 1 minor enhancement:

data/Manifest.txt

@@ -29,6 +29,8 @@ lib/ruby27_parser.rb
 lib/ruby27_parser.y
 lib/ruby30_parser.rb
 lib/ruby30_parser.y
+lib/ruby31_parser.rb
+lib/ruby31_parser.y
 lib/ruby3_parser.yy
 lib/ruby_lexer.rb
 lib/ruby_lexer.rex

data/README.rdoc

@@ -33,6 +33,9 @@ Tested against 801,039 files from the latest of all rubygems (as of 2013-05):
 * 1.9 parser is at 99.9940% accuracy, 4.013 sigma
 * 2.0 parser is at 99.9939% accuracy, 4.008 sigma
 * 2.6 parser is at 99.9972% accuracy, 4.191 sigma
+* 3.0 parser has a 100% parse rate.
+  * Tested against 2,672,412 unique ruby files across 167k gems.
+  * As do all the others now, basically.
 
 == FEATURES/PROBLEMS:
 
@@ -62,15 +65,14 @@ You can also use Ruby19Parser, Ruby18Parser, or RubyParser.for_current_ruby:
 
 To add a new version:
 
-* New parser should be generated from lib/ruby_parser.yy.
-* Extend lib/ruby_parser.yy with new class name.
-* Add new version number to V2 in Rakefile for rule creation.
+* New parser should be generated from lib/ruby[3]_parser.yy.
+* Extend lib/ruby[3]_parser.yy with new class name.
+* Add new version number to V2/V3 in Rakefile for rule creation.
+* Add new (full) version to `ruby_parse` section of Rakefile for rake compare
 * Require generated parser in lib/ruby_parser.rb.
 * Add empty TestRubyParserShared##Plus module and TestRubyParserV## to test/test_ruby_parser.rb.
 * Extend Manifest.txt with generated file names.
-* Extend sexp_processor's pt_testcase.rb to match version
-  * add_19tests needs to have the version added
-  * VER_RE needs to have the regexp expanded
+* Add new version number to sexp_processor's pt_testcase.rb in all_versions
 
 Until all of these are done, you won't have a clean test run.
 

data/Rakefile

@@ -14,7 +14,7 @@ Hoe.add_include_dirs "../../minitest/dev/lib"
 Hoe.add_include_dirs "../../oedipus_lex/dev/lib"
 
 V2   = %w[20 21 22 23 24 25 26 27]
-V3   = %w[30]
+V3   = %w[30 31]
 
 VERS = V2 + V3
 
@@ -184,8 +184,8 @@ def ruby_parse version
 
   file c_parse_y => c_tarball do
     in_compare do
-      extract_glob = case version
-                     when /2\.7|3\.0/
+      extract_glob = case
+                     when version > "2.7" then
                        "{id.h,parse.y,tool/{id2token.rb,lib/vpath.rb}}"
                      else
                        "{id.h,parse.y,tool/{id2token.rb,vpath.rb}}"
@@ -258,7 +258,7 @@ task :versions do
   require "net/http" # avoid require issues in threads
   require "net/https"
 
-  versions = %w[ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 3.0 ]
+  versions = VERS.map { |s| s.split(//).join "." }
 
   base_url = "https://cache.ruby-lang.org/pub/ruby"
 
@@ -291,9 +291,10 @@ ruby_parse "2.2.10"
 ruby_parse "2.3.8"
 ruby_parse "2.4.10"
 ruby_parse "2.5.9"
-ruby_parse "2.6.8"
-ruby_parse "2.7.4"
-ruby_parse "3.0.2"
+ruby_parse "2.6.9"
+ruby_parse "2.7.5"
+ruby_parse "3.0.3"
+ruby_parse "3.1.1"
 
 task :debug => :isolate do
   ENV["V"] ||= VERS.last

data/gauntlet.md

@@ -19,10 +19,10 @@ an external disk. Here is the config:
 And I update using rake:
 
 ```
-% cd ~/Work/git/rubygems/rubygems-mirror
+% cd GIT/rubygems/rubygems-mirror
 % git down
 % rake mirror:latest
-% /Volumes/StuffA/gauntlet/bin/cleanup.rb
+% /Volumes/StuffA/gauntlet/bin/cleanup.rb -y -v
 ```
 
 This rather quickly updates my mirror to the latest versions of
@@ -34,22 +34,24 @@ bit, but it is pretty minimal (currently ~20 bad gems).
 ## Curating an Archive of Ruby Files
 
 Next, I process the gem mirror into a much more digestable structure
-using `hash.rb` (TODO: needs a better name):
+using `unpack_gems.rb`.
 
 ```
-% cd RP
-% /Volumes/StuffA/gauntlet/bin/unpack_gems.rb
+% cd RP/gauntlet
+% time caffeinate /Volumes/StuffA/gauntlet/bin/unpack_gems.rb -v [-a] ; say done
 ... waaaait ...
-% mv hashed.noindex gauntlet.$(today).noindex
-% lrztar gauntlet.$(today).noindex
-% mv gauntlet.$(today).noindex.lrz /Volumes/StuffA/gauntlet/
-```
-
-This script filters all the newer gems (TODO: WHY?), unpacks them,
-finds all the files that look like they're valid ruby, ensures they're
-valid ruby (using the current version of ruby to compile them), and
-then moves them into a SHA dir structure that looks something like
-this:
+% DIR=gauntlet.$(today).(all|new).noindex
+% mv hashed.noindex $DIR
+% tar c $DIR | zstd -5 -T0 --long > archives/$DIR.tar.zst
+% tar vc -T <(fd . $DIR | sort) | zstd -5 -T0 --long > archives/$DIR.tar.zst
+% ./bin/sync.sh
+```
+
+This script filters all the newer (< 1 year old) gems (unless `-a` is
+used), unpacks them, finds all the files that look like they're valid
+ruby, ensures they're valid ruby (using the current version of ruby to
+compile them), and then moves them into a SHA dir structure that looks
+something like this:
 
 ```
 hashed.noindex/a/b/c/<full_file_sha>.rb