parser 2.6.4.1 → 2.7.0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d7b8b9f67b6aaacdb9a0d545465e924b4f648b111da6be699663c5c67c84e4f
-  data.tar.gz: 171070fa439abfe6279d7e0c713d46a28528669085cfc643ab808569d17357c1
+  metadata.gz: 2801cd6adbcd161d1a3b643dca79b9a9f7ff13210439bf632fcc22072837dffa
+  data.tar.gz: 20a0402b20903555a7cc29507205f7bf7176b26dfe039df79bf17fd503ed951e
 SHA512:
-  metadata.gz: ab3de4f196504df68763d5efaa347b4febf2e2feed3dc1f0b407565a6f93434d4b1709dbe7a1fa36f28e39b25d67434ad861818ca7d570e0b2b0d8b5b1e71ff9
-  data.tar.gz: 8634d57721f51fa9dd91c843960fb6c20a7bbb0331c6e62b3f29da3a770f5471f9adcaad14efb5a68904f2e93767e835022abcb9b739c113898822942085f112
+  metadata.gz: b254886d186ac704cb8a6dcc6bfdce7d2f9f079ce83edb456646a0ec04336a2893c140f1b525e8cea89f34c52bf01c1e6849dc8ce80d9f4f3fdb0af072e15ac2
+  data.tar.gz: 627b678f07bf6ad130235f3ab1440d563905cf671e0618f2cb3b549e0f34dc341e70f5659580d731259d8caa97fd3d221868b966b5df5188c5c985db836ecc2a

data/.travis.yml

@@ -11,14 +11,17 @@ matrix:
     - name: 2.3.8 / Parser tests
       rvm: 2.3.8
       script: bundle exec rake test_cov
-    - name: 2.4.7 / Parser tests
-      rvm: 2.4.7
+    - name: 2.4.9 / Parser tests
+      rvm: 2.4.9
       script: bundle exec rake test_cov
-    - name: 2.5.6 / Parser tests
-      rvm: 2.5.6
+    - name: 2.5.7 / Parser tests
+      rvm: 2.5.7
       script: bundle exec rake test_cov
-    - name: 2.6.4 / Parser tests
-      rvm: 2.6.4
+    - name: 2.6.5 / Parser tests
+      rvm: 2.6.5
+      script: bundle exec rake test_cov
+    - name: 2.7.0 / Parser tests
+      rvm: 2.7.0
       script: bundle exec rake test_cov
     - name: ruby-head / Parser tests
       rvm: ruby-head
@@ -29,11 +32,14 @@ matrix:
     - name: rbx-2 / Parser tests
       rvm: rbx-2
       script: bundle exec rake test_cov
-    - name: 2.5.6 / Rubocop tests
-      rvm: 2.5.6
+    - name: 2.5.7 / Rubocop tests
+      rvm: 2.5.7
+      script: ./ci/run_rubocop_specs
+    - name: 2.6.5 / Rubocop tests
+      rvm: 2.6.5
       script: ./ci/run_rubocop_specs
-    - name: 2.6.4 / Rubocop tests
-      rvm: 2.6.4
+    - name: 2.7.0 / Rubocop tests
+      rvm: 2.7.0
       script: ./ci/run_rubocop_specs
   allow_failures:
     - rvm: ruby-head

data/CHANGELOG.md

@@ -1,6 +1,140 @@
 Changelog
 =========
 
+v2.7.0.2 (2020-01-08)
+---------------------
+
+Bugs fixed:
+ * lexer.rl: fix paren_nest for curly braces (#646) (Ilya Bylich)
+
+v2.7.0.1 (2019-12-30)
+---------------------
+
+Bugs fixed:
+ * dedenter.rb: prevent `ArgumentError` when processing binary en… (#642) (Koichi ITO)
+
+v2.7.0.0 (2019-12-26)
+---------------------
+
+API modifications:
+ * README.md: documented compatibility issue with EOF chars after… (#637) (Ilya Bylich)
+ * ruby27.y: refactor logic around 'circular argument reference'(#628) (Ilya Bylich)
+
+Features implemented:
+ * ruby27.y: added pattern matching (#574) (Ilya Bylich)
+ * lexer.rl: parse embedded ivars/cvars starting with digit as str (#639) (Ilya Bylich)
+ * lexer.rl: warn on `...` at EOL. (#636) (Ilya Bylich)
+ * ruby27.y: removed opt_block_args_tail: tOROP rule. (#635) (Ilya Bylich)
+ * ruby27.y: reverted method reference operator (added in #634) (Ilya Bylich)
+ * ruby27.y: treat numparams as locals outside numblock. (#633) (Ilya Bylich)
+
+Bugs fixed:
+ * dedenter.rb: fixed over-dedenting of squiggly heredocs (#641) (Ilya Bylich)
+ * ruby27.y: added "arguments forwarding" (#625) (Ilya Bylich)
+ * ruby27.y: reject circular argument reference. (#622) (Ilya Bylich)
+ * ruby27.y: changed prefix of numparams to "_" (#620) (Ilya Bylich)
+
+v2.6.5.0 (2019-10-03)
+---------------------
+
+API modifications:
+ * Bump ruby versions to 2.4.9, 2.5.7 and 2.6.5. (#619) (Ilya Bylich)
+
+Features implemented:
+ * lexer.rl: changed max numparam to `@9` (#617) (Ilya Bylich)
+ * lexer.rl: support comments before leading dot in 27 mode. (#613) (Ilya Bylich)
+
+Bugs fixed:
+ * lexer.rl: emit tMETHREF as tDOT+tCOLON for rubies \< 27. (#614) (Ilya Bylich)
+
+v2.6.4.1 (2019-09-12)
+---------------------
+
+Bugs fixed:
+ * lexer.rl: fix parsing of 'm a + b do end' (#605) (Ilya Bylich)
+
+v2.6.4.0 (2019-08-30)
+---------------------
+
+API modifications:
+ * Added specs for heredocs with mixed encoding. (#581) (Ilya Bylich)
+
+Features implemented:
+ * ruby27.y: Revert "pipeline operator" (#601) (Koichi ITO)
+ * ruby27.y: Fix parsing of mutiple assignment with rescue modifier (#600) (Koichi ITO)
+ * ruby27.y: hoisted out f_rest_marg. (#594) (Ilya Bylich)
+ * ruby27.y: added pipeline operator. (#592) (Ilya Bylich)
+ * ruby27.y: reject safe navigator in LHS of mass-assignment. (#586) (Ilya Bylich)
+ * lexer.rl: reject whitespaces in meta and control chars. (#585) (Ilya Bylich)
+ * lexer.rl: Reject numparams as symbol literals. (#582) (Ilya Bylich)
+ * ruby27.y: Added numbered parameters support. (#565) (Ilya Bylich)
+ * lexer.rl: Reject \n and \r in heredoc identifiers starting from 2.7. (#575) (Ilya Bylich)
+
+Bugs fixed:
+ * ruby-parse: print empty string when --emit-json and empty input are given. (#590) (Ilya Bylich)
+ * AST_FORMAT: fixed documentation of the string with interpolation. (#589) (Ilya Bylich)
+ * builder.rb, processor.rb: Changed format of the procarg0 node. (#587) (Ilya Bylich)
+
+v2.6.3.0 (2019-04-28)
+---------------------
+
+Features implemented:
+ * ruby27.y: Added beginless ranges support. (#570) (Ilya Bylich)
+
+v2.6.2.1 (2019-04-05)
+---------------------
+
+API modifications:
+ * Bump 2.4 branch to 2.4.6. (#569) (Ilya Bylich)
+ * Lexer should know about current parsing context. (#566) (Ilya Bylich)
+
+v2.6.2.0 (2019-03-21)
+---------------------
+
+API modifications:
+ * Bump ruby versions to 2.5.5 and 2.6.2. (#563) (Ilya Bylich)
+ * Bump Ruby version to 2.6.1. (#554) (Ilya Bylich)
+
+Features implemented:
+ * ruby27.y: dsym should be treated as string. (#560) (Ilya Bylich)
+ * ruby27.y: Refactored symbol rules. (#557) (Ilya Bylich)
+ * ruby27.y: Added method reference operator. (#556) (Ilya Bylich)
+ * ruby27.y: branch parser. (#546) (Ilya Bylich)
+
+v2.6.0.0 (2019-01-16)
+---------------------
+
+API modifications:
+ * 2.6.0 was released, unmark is as -dev. (#538) (Ilya Bylich)
+
+Bugs fixed:
+ * Fix parsing of "\\\n" escaped sequences in various literals. (#539) (Ilya Bylich)
+
+v2.5.3.0 (2018-10-29)
+---------------------
+
+Bugs fixed:
+ * lexer.rl: Fix parsing of 'm :key => m do; m() do end; end'. (#526) (Ilya Bylich)
+ * lexer.rl: Fix parsing of ambiguous 1re. (#523) (Ilya Bylich)
+
+v2.5.1.2 (2018-07-10)
+---------------------
+
+Bugs fixed:
+ * lexer.rl: Partially revert 5ba072d and properly handle 'm = -> *args do end'. (Ilya Bylich)
+
+v2.5.1.1 (2018-07-10)
+---------------------
+
+Features implemented:
+ * ruby26.y: Endless ranges support. (Ilya Bylich)
+
+Bugs fixed:
+ * lexer.rl: Fix parsing of 'm = -> *args do end'. (Ilya Bylich)
+ * AST::Processor: Properly recurse into "kwsplat" nodes (Nelson Elhage)
+ * ruby24, ruby25, ruby26: Fix cmdargs after command_args followed by tLBRACE_ARG. This commit tracks upstream commit ruby/ruby@f168dbd. (Ilya Bylich)
+ * lexer.rl: Fix parsing of `let (:a) { m do; end }`. (Ilya Bylich)
+
 v2.5.1.0 (2018-04-12)
 ---------------------
 

data/README.md

@@ -234,7 +234,7 @@ Parser implements the MacRuby 0.12 and RubyMotion mid-2015 parsers precisely. Ho
 
 ## Known issues
 
-Adding support for the following Ruby MRI features in Parser would needlessly complicate it, and as they all are very specific and rarely occuring corner cases, this is not done.
+Adding support for the following Ruby MRI features in Parser would needlessly complicate it, and as they all are very specific and rarely occurring corner cases, this is not done.
 
 Parser has been extensively tested; in particular, it parses almost entire [Rubygems][rg] corpus. For every issue, a breakdown of affected gems is offered.
 
@@ -269,6 +269,13 @@ follows 2.1 behavior.
 
 No known code is affected by this issue.
 
+### EOF characters after embedded documents before 2.7
+
+Code like `"=begin\n""=end\0"` is invalid for all versions of Ruby before 2.7. Ruby 2.7 and later parses it
+normally. Parser follows 2.7 behavior.
+
+It is unknown whether any gems are affected by this issue.
+
 ## Contributors
 
 * [whitequark][]

data/doc/AST_FORMAT.md

@@ -602,18 +602,6 @@ Format:
  ~~~~~~~~~~~~~ expression
 ~~~
 
-### Method reference operator
-
-Format:
-
-~~~
-(meth-ref (self) :foo)
-"self.:foo"
-     ^^ dot
-       ^^^ selector
- ^^^^^^^^^ expression
-~~~
-
 ### Multiple assignment
 
 #### Multiple left hand side
@@ -1118,22 +1106,41 @@ Format:
 s(:numblock,
   s(:send, nil, :proc), 3,
   s(:send,
-    s(:numparam, 1), :+,
-    s(:numparam, 3)))
-"proc { @1 + @3 }"
+    s(:lvar, :_1), :+,
+    s(:lvar, :_3)))
+"proc { _1 + _3 }"
       ~ begin   ~ end
  ~~~~~~~~~~~~~~~~ expression
 ~~~
 
-### Numbered parameter
+## Forward arguments
+
+### Method definition accepting forwarding arguments
+
+Ruby 2.7 introduced a feature called "arguments forwarding".
+When a method takes any arguments for forwarding them in the future
+the whole `args` node gets replaced with `forward-args` node.
 
 Format:
 
 ~~~
-(numparam 10)
-"@10"
- ~~~ name
- ~~~ expression
+(def :foo
+  (forward-args) nil)
+"def foo(...); end"
+            ~ end
+        ~ begin
+        ~~~~~ expression
+~~~
+
+### Method call taking arguments of the currently forwarding method
+
+Format:
+
+~~~
+(send nil :foo
+  (forwarded-args))
+"foo(...)"
+     ~~~ expression
 ~~~
 
 ## Send
@@ -1814,3 +1821,360 @@ Format:
 "__ENCODING__"
  ~~~~~~~~~~~~ expression
 ~~~
+
+## Pattern matching
+
+### Using `in` modifier
+
+Format:
+
+~~~
+(in-match
+  (int 1)
+  (match-var :a))
+"1 in a"
+   ~~ operator
+ ~~~~~~ expression
+~~~
+
+### Case with pattern matching
+
+#### Without else
+
+Format:
+
+~~~
+(case-match
+  (str "str")
+  (in-pattern
+    (match-var :foo)
+    (lvar :bar)) nil)
+"case "str"; in foo; bar; end"
+ ~~~~ keyword             ~~~ end
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ expression
+~~~
+
+#### With else
+
+Format:
+
+~~~
+(case-match,
+  (str "str")
+  (in-pattern
+    (match-var :foo)
+    (lvar :bar))
+  (lvar :baz))
+"case "str"; in foo; bar; else; baz; end"
+ ~~~~ keyword             ~~~~ else  ~~~ end
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ expression
+~~~
+
+#### With empty else
+
+Empty `else` differs from the missing (or _implicit_) `else` for pattern matching, since
+the latter one raises a `NoMatchingPattern` exception. Thus, we need a way to distinguish this
+two cases in the resulting AST.
+
+Format:
+
+~~~
+(case-match,
+  (str "str")
+  (in-pattern
+    (match-var :foo)
+    (lvar :bar))
+  (empty-else))
+"case "str"; in foo; bar; else; end"
+ ~~~~ keyword             ~~~~ else
+                                ~~~ end
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ expression
+~~~
+
+### In clause
+
+Format:
+
+~~~
+(in-pattern
+  (match-var :foo)
+  (lvar :bar))
+"in foo then bar"
+ ~~ keyword
+        ~~~~ begin
+ ~~~~~~~~~~~~~~~ expression
+~~~
+
+### If guard
+
+This guard runs after matching, so it's not an `if` modifier.
+
+Format:
+
+~~~
+(in-pattern
+  (match-var :foo)
+  (if-guard
+    (lvar :bar)) nil)
+"in foo if bar"
+        ~~ keyword
+        ~~~~~~ expression
+~~~
+
+### Unless guard
+
+This guard runs after matching, so it's not an `unless` modifier.
+
+Format:
+
+~~~
+(in-pattern
+  (match-var :foo)
+  (unless-guard
+    (lvar :bar)) nil)
+"in foo unless bar"
+        ~~~~~~ keyword
+        ~~~~~~~~~~ expression
+~~~
+
+### Match variable
+
+Format:
+
+~~~
+(match-var :foo)
+"in foo"
+    ~~~ name
+    ~~~ expression
+~~~
+
+### Match rest
+
+#### With name
+
+Format:
+
+~~~
+(match-rest
+  (match-var :foo))
+"in *foo"
+    ~ operator
+    ~~~~ expression
+~~~
+
+#### Without name
+
+Format:
+
+~~~
+(match-rest)
+"in *"
+    ~ operator
+    ~ expression
+~~~
+
+### Pin operator
+
+Format:
+
+~~~
+(pin
+  (lvar :foo))
+"in ^foo"
+    ~ selector
+    ~~~~ expression
+~~~
+
+### Match alternative
+
+Format:
+
+~~~
+(match-alt
+  (pin
+    (lvar :foo))
+  (int 1))
+"in ^foo | 1"
+         ~ operator
+    ~~~~~~~~ expression
+~~~
+
+### Match with alias
+
+Format:
+
+~~~
+(match-as
+  (int 1)
+  (match-var :foo))
+"in 1 => foo"
+      ~~ operator
+    ~~~~~~~~ expression
+~~~
+
+### Match using array pattern
+
+#### Explicit
+
+Format:
+
+~~~
+(array-pattern
+  (pin
+    (lvar :foo))
+  (match-var :bar))
+"in [^foo, bar]"
+    ~ begin   ~ end
+    ~~~~~~~~~~~ expression
+~~~
+
+#### Explicit with tail
+
+Adding a trailing comma in the end works as `, *`
+
+Format:
+
+~~~
+(array-pattern-with-tail
+  (pin
+    (lvar :foo))
+  (match-var :bar))
+"in [^foo, bar,]"
+    ~ begin    ~ end
+    ~~~~~~~~~~~~ expression
+~~~
+
+#### Implicit
+
+Format:
+
+~~~
+(array-pattern
+  (pin
+    (lvar :foo))
+  (match-var :bar))
+"in ^foo, bar"
+    ~~~~~~~~~ expression
+~~~
+
+#### Implicit with tail
+
+Format:
+
+Adding a trailing comma in the end works as `, *`,
+so a single item match with comma gets interpreted as an array.
+
+~~~
+(array-pattern-with-tail
+  (match-var :foo))
+"in foo,"
+    ~~~ expression
+~~~
+
+### Matching using hash pattern
+
+#### Explicit
+
+Format:
+
+~~~
+(hash-pattern
+  (pair
+    (sym :a)
+    (int 10)))
+"in { a: 10 }"
+    ~ begin ~ end
+    ~~~~~~~~~ expression
+~~~
+
+#### Implicit
+
+Format:
+
+~~~
+(hash-pattern
+  (pair
+    (sym :a)
+    (int 10)))
+"in a: 10"
+    ~~~~~ expression
+~~~
+
+#### Assignment using hash pattern
+
+Format:
+
+~~~
+(hash-pattern
+  (match-var :a))
+"in a:"
+    ~ name (match-var)
+    ~~ expression (match-var)
+~~~
+
+#### Nil hash pattern
+
+Format:
+~~~
+(hash-pattern
+  (match-nil-pattern))
+"in **nil"
+    ~~~~~ expression (match-nil-pattern)
+      ~~~ name (match-nil-pattern)
+~~~
+
+### Matching using const pattern
+
+#### With array pattern
+
+Format:
+
+~~~
+(const-pattern
+  (const nil :X)
+  (array-pattern
+    (pin
+      (lvar :foo))
+    (match-var :bar)))
+"in X[^foo bar]"
+     ~ begin (const-pattern)
+               ~ end (const-pattern)
+     ~~~~~~~~~~~ expression (const-pattern)
+    ~ name (const-pattern.const)
+    ~ expression (const-pattern.const)
+~~~
+
+#### With hash pattern
+
+Format:
+
+~~~
+(const-pattern
+  (const nil :X)
+  (hash-pattern
+    (match-var :foo)
+    (match-var :bar)))
+"in X[foo:, bar:]"
+     ~ begin (const-pattern)
+                ~ end (const-pattern)
+     ~~~~~~~~~~~~ expression (const-pattern)
+    ~ name (const-pattern.const)
+    ~ expression (const-pattern.const)
+~~~
+
+#### With array pattern without elements
+
+Format:
+
+~~~
+(const-pattern
+  (const nil :X)
+  (array-pattern))
+"in X[]"
+     ~ begin (const-pattern)
+      ~ end (const-pattern)
+     ~~ expression (const-pattern)
+    ~ name (const-pattern.const)
+    ~ expression (const-pattern.const)
+     ~~ expression (const-pattern.array_pattern)
+~~~