ripper-plus 1.0.0

data/.document

@@ -0,0 +1,5 @@
+lib/**/*.rb
+bin/*
+- 
+features/**/*.feature
+LICENSE.txt

data/.rspec

@@ -0,0 +1 @@
+--color

data/Gemfile

@@ -0,0 +1,14 @@
+source "http://rubygems.org"
+# Add dependencies required to use your gem here.
+# Example:
+#   gem "activesupport", ">= 2.3.5"
+
+# Add dependencies to develop your gem here.
+# Include everything needed to run rake, tests, features, etc.
+group :development do
+  gem "rspec", "~> 2.3.0"
+  gem "yard", "~> 0.6.0"
+  gem "bundler", "~> 1.0.0"
+  gem "jeweler", "~> 1.5.2"
+  gem "rcov", ">= 0"
+end

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Michael Edgar
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,68 @@
+# ripper-plus
+
+Ripper is the Ruby parser library packaged with Ruby 1.9. While quite complete, it has some quirks that can make use frustrating. This gem intends to correct them. The most important is bareword resolution in Ruby ASTs.
+
+## Bareword Resolution
+
+In Ruby, a local variable is created when it is assigned to. If Ruby finds a bareword `foo` before an assignment `foo = 5`, then Ruby will call `self.foo()`.
+
+Kind of.
+
+What happens when you run this code?
+
+    def label
+      'hello'
+    end
+    def print_label
+      label = 'Label: ' + label
+      puts label
+    end
+    print_label
+
+If one takes the intuitive approach, one would assume it prints "Label: hello", right? Afraid not: it errors out. The intuitive approach says that `'Label: ' + label` is run before the assignment to the local variable `label`, so the `label` method will be called and appended to `"Label: "`. In reality, it won't, because local variables aren't created on assignment. Local variables are created *immediately when the Ruby parser parses a left-hand-side local variable*. Before the right-hand-side is even parsed, the local variable has been created. This also results in an infamous anomaly:
+
+    def say_what?
+      x = 'hello' unless defined?(x)
+      puts x
+    end
+    say_what?
+
+`say_what?` prints a blank line! This is because as soon as the `x` on the LHS is parsed, `x` is a local variable with value `nil`. By the time `defined?(x)` is executed, `x` has long since been a local variable!
+
+## Ripper's Mistake
+
+Ripper doesn't carry scope information as it parses, and as such, parses any lone identifier as an AST node of type `:var_ref`. It is up to consumers of the AST to figure out the meaning of the `:var_ref` node. It can be reasonably argued that the semantics of the `:var_ref` node should not be part of the AST, as my thesis adviser pointed out when I complained about this, as the two are syntactically identical. Unfortunately, the meaning of the `var_ref` node comes from the parser itself; any attempt to determine semantics based solely on the AST is simply re-creating the work the parser ordinarily does! Indeed, when Ruby parses the code for execution, it *does* create different internal node types upon seeing a method-call bareword and a local variable bareword!
+
+I'd like to see this behavior rolled into Ripper proper. Until then, `ripper-plus` is a reasonable replacement.
+
+## ripper-plus
+
+By using our knowledge of Ruby's parser, we can recreate the scope-tracking behavior by walking the AST generated by Ripper. We simply must do normal scope tracking, observe the creation of local variables, and ensure that we walk each node of the tree in the order in which those nodes would have been parsed. Most subtrees generated by Ripper are already in this order, with the exception of the modifier nodes (`foo.bar if baz`, `x = bar() while x != nil`). Most importantly, since everything is an expression in Ruby and local variable assignments can occur just about anywhere, exact scoping semantics must be recreated. Corner cases such as this:
+
+    def foo(x, y = y)
+    end
+
+Need to be properly resolved. Did you know that, unlike the `label = label` example above, the above code is equivalent to:
+
+    def foo(x, y = y())
+    end
+
+Anyway, ripper-plus turns all method-call `:var_ref` nodes into `zcall` nodes; the node structure is otherwise unchanged.
+
+The truth is, everybody who is using Ripper right now *should* be doing *all* of this. Anything short, and you have bugs. [Laser](https://github.com/michaeledgar/laser/) has bugs as a result. It's a pain in the ass to get it all right. So hopefully, in Ruby 1.9.x, this will be the default. For now, you *should* use ripper-plus.
+
+## Contributing to ripper-plus
+ 
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
+* Fork the project
+* Start a feature/bugfix branch
+* Commit and push until you are happy with your contribution
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+## Copyright
+
+Copyright (c) 2011 Michael Edgar. See LICENSE.txt for
+further details.
+

data/Rakefile

@@ -0,0 +1,45 @@
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts "Run `bundle install` to install missing gems"
+  exit e.status_code
+end
+require 'rake'
+
+require 'jeweler'
+Jeweler::Tasks.new do |gem|
+  # gem is a Gem::Specification... see http://docs.rubygems.org/read/chapter/20 for more options
+  gem.name = "ripper-plus"
+  gem.homepage = "http://github.com/michaeledgar/ripper-plus"
+  gem.license = "MIT"
+  gem.summary = %Q{Parses Ruby code into an improved Ripper AST format}
+  gem.description = %Q{Ripper is the Ruby parser library packaged
+with Ruby 1.9. While quite complete, it has some quirks that can
+make use frustrating. This gem intends to correct them.}
+  gem.email = "michael.j.edgar@dartmouth.edu"
+  gem.authors = ["Michael Edgar"]
+  # Include your dependencies below. Runtime dependencies are required when using your gem,
+  # and development dependencies are only needed for development (ie running rake tasks, tests, etc)
+  #  gem.add_runtime_dependency 'jabber4r', '> 0.1'
+  #  gem.add_development_dependency 'rspec', '> 1.2.3'
+end
+Jeweler::RubygemsDotOrgTasks.new
+
+require 'rspec/core'
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec) do |spec|
+  spec.pattern = FileList['spec/**/*_spec.rb']
+end
+
+RSpec::Core::RakeTask.new(:rcov) do |spec|
+  spec.pattern = 'spec/**/*_spec.rb'
+  spec.rcov = true
+end
+
+task :default => :spec
+
+require 'yard'
+YARD::Rake::YardocTask.new

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/lib/ripper-plus.rb

@@ -0,0 +1,7 @@
+if RUBY_VERSION < "1.9"
+  raise 'ripper-plus requires Ruby 1.9+'
+end
+require 'ripper'
+require 'ripper-plus/ripper-plus'
+require 'ripper-plus/scope_stack'
+require 'ripper-plus/transformer'

data/lib/ripper-plus/ripper-plus.rb

@@ -0,0 +1,13 @@
+# Top-level module for Ripper Plus. Provides global methods for
+# getting a RipperPlus AST for a given input program.
+module RipperPlus
+  # Parses the given Ruby code into a RipperPlus AST.
+  def self.sexp(text)
+    for_ripper_ast(Ripper.sexp(text))
+  end
+  
+  # Transforms the provided Ripper AST into a RipperPlus AST.
+  def self.for_ripper_ast(tree)
+    Transformer.transform(tree)
+  end
+end

data/lib/ripper-plus/scope_stack.rb

@@ -0,0 +1,54 @@
+module RipperPlus
+  # internal class that manages the current scopes.
+  class ScopeStack
+    SCOPE_BLOCKER_9000 = :scope_block
+    def initialize
+      # simplifies algorithm to have the scope blocker be the stack base
+      @stack = [SCOPE_BLOCKER_9000, Set.new]
+    end
+
+    def inspect
+      middle = @stack.map do |scope|
+        if SCOPE_BLOCKER_9000 == scope
+          '||'
+        else
+          "[ #{scope.to_a.sort.join(', ')} ]"
+        end
+      end.join(' ')
+      "< #{middle} >"
+    end
+
+    def add_variable(var)
+      @stack.last << var
+    end
+  
+    # An open scope permits reference to local variables in enclosing scopes.
+    def with_open_scope
+      @stack.push(Set.new)
+      yield
+    ensure
+      @stack.pop  # pops open scope
+    end
+  
+    # An open scope denies reference to local variables in enclosing scopes.
+    def with_closed_scope
+      @stack.push(SCOPE_BLOCKER_9000)
+      @stack.push(Set.new)
+      yield
+    ensure
+      @stack.pop  # pop closed scope
+      @stack.pop  # pop scope blocker
+    end
+  
+    # Checks if the given variable is in scope.
+    def has_variable?(var)
+      @stack.reverse_each do |scope|
+        if SCOPE_BLOCKER_9000 == scope
+          return false
+        elsif scope.include?(var)
+          return true
+        end
+      end
+    end
+  end
+end

data/lib/ripper-plus/transformer.rb

@@ -0,0 +1,201 @@
+require 'set'
+module RipperPlus
+  # Transforms a 1.9.2 Ripper AST into a RipperPlus AST. The only
+  # change as a result of this transformation is that the nodes for
+  # local variable references and zcalls (bareword calls to self)
+  # have different node types:
+  #
+  #    def foo(x)
+  #      y
+  #      y = x
+  #    end
+  #
+  # becomes
+  #
+  # [:def,
+  #   [:@ident, "foo", [1, 4]],
+  #   [:paren, [:params, [[:@ident, "x", [1, 8]]], nil, nil, nil, nil]],
+  #   [:bodystmt,
+  #    [[:zcall, [:@ident, "y", [2, 2]]],
+  #     [:assign,
+  #      [:var_field, [:@ident, "y", [3, 2]]],
+  #      [:var_ref, [:@ident, "x", [3, 6]]]]],
+  #    nil, nil, nil]]
+  #
+  # while:
+  #
+  #    def foo(x)
+  #      y = x
+  #      y
+  #    end
+  #
+  # becomes
+  #
+  # [:def,
+  #   [:@ident, "foo", [1, 4]],
+  #   [:paren, [:params, [[:@ident, "x", [1, 8]]], nil, nil, nil, nil]],
+  #   [:bodystmt,
+  #    [[:assign,
+  #      [:var_field, [:@ident, "y", [2, 2]]],
+  #      [:var_ref, [:@ident, "x", [2, 6]]]],
+  #     [:var_ref, [:@ident, "y", [3, 2]]]],
+  #    nil, nil, nil]]
+  module Transformer
+    extend self
+
+    # Transforms the given AST into a RipperPlus AST.
+    def transform(root)
+      new_copy = clone_sexp(root)
+      scope_stack = ScopeStack.new
+      transform_tree(new_copy, scope_stack)
+      new_copy
+    end
+
+    # Transforms the given tree into a RipperPlus AST.
+    def transform_tree(tree, scope_stack)
+      case tree[0]
+      when :assign, :massign
+        lhs, rhs = tree[1..2]
+        add_variables_from_lhs(lhs, scope_stack)
+        transform_tree(rhs, scope_stack)
+      when :for
+        vars, iterated, body = tree[1..3]
+        add_variables_from_lhs(vars, scope_stack)
+        transform_tree(iterated, scope_stack)
+        transform_tree(body, scope_stack)
+      when :var_ref
+        if tree[1][0] == :@ident && !scope_stack.has_variable?(tree[1][1])
+          tree[0] = :zcall
+        end
+      when :class
+        superclass, body = tree[2..3]
+        transform_tree(superclass, scope_stack) if superclass  # superclass node
+        scope_stack.with_closed_scope do
+          transform_tree(body, scope_stack)
+        end
+      when :module
+        scope_stack.with_closed_scope do
+          transform_tree(tree[2], scope_stack)  # body
+        end
+      when :sclass
+        singleton, body = tree[1..2]
+        transform_tree(singleton, scope_stack)
+        scope_stack.with_closed_scope do
+          transform_tree(body, scope_stack)
+        end
+      when :def
+        scope_stack.with_closed_scope do
+          param_node = tree[2]
+          body = tree[3]
+          transform_params(param_node, scope_stack)
+          transform_tree(body, scope_stack)
+        end
+      when :defs
+        transform_tree(tree[1], scope_stack)  # singleton could be a method call!
+        scope_stack.with_closed_scope do
+          param_node = tree[4]
+          body = tree[5]
+          transform_params(param_node, scope_stack)
+          transform_tree(body, scope_stack)
+        end
+      when :method_add_block
+        call, block = tree[1..2]
+        # first transform the call
+        transform_tree(call, scope_stack)
+        # then transform the block
+        block_args, block_body = block[1..2]
+        scope_stack.with_open_scope do
+          if block_args
+            transform_params(block_args[1], scope_stack)
+            if block_args[2]
+              block_args[2].each { |var| add_variables_from_lhs(var, scope_stack) }
+            end
+          end
+          transform_tree(block_body, scope_stack)
+        end
+      when :if_mod, :unless_mod, :while_mod, :until_mod, :rescue_mod
+        # The AST is the reverse of the parse order for these nodes.
+        transform_tree(tree[2], scope_stack)
+        transform_tree(tree[1], scope_stack)
+      else
+        transform_in_order(tree, scope_stack)
+      end
+    end
+
+    def add_variables_from_lhs(lhs, scope_stack)
+      case lhs[0]
+      when :@ident
+        scope_stack.add_variable(lhs[1])
+      when Array
+        lhs.each { |var| add_variables_from_lhs(var, scope_stack) }
+      when :mlhs_paren, :var_field, :rest_param, :blockarg
+        add_variables_from_lhs(lhs[1], scope_stack)
+      when :mlhs_add_star
+        pre_star, star, post_star = lhs[1..3]
+        pre_star.each { |var| add_variables_from_lhs(var, scope_stack) }
+        if star
+          add_variables_from_lhs(star, scope_stack)
+        end
+        if post_star
+          post_star.each { |var| add_variables_from_lhs(var, scope_stack) }
+        end
+      end
+    end
+
+    # If this node's subtrees are ordered as they are lexically, as most are,
+    # transform each subtree in order.
+    def transform_in_order(tree, scope_stack)
+      # some nodes have no type: include the first element in this case
+      range = Symbol === tree[0] ? 1..-1 : 0..-1
+      tree[range].each do |subtree|
+        # obviously don't transform literals or token locations
+        if Array === subtree && !(Fixnum === subtree[0])
+          transform_tree(subtree, scope_stack)
+        end
+      end
+    end
+
+    # Transforms a parameter list, and adds the new variables to current scope.
+    # Used by both block args and method args.
+    def transform_params(param_node, scope_stack)
+      param_node = param_node[1] if param_node[0] == :paren
+      if param_node
+        positional_1, optional, rest, positional_2, block = param_node[1..5]
+        if positional_1
+          positional_1.each { |var| add_variables_from_lhs(var, scope_stack) }
+        end
+        if optional
+          optional.each do |var, value|
+            # MUST walk value first. (def foo(y=y); end) == (def foo(y=y()); end)
+            transform_tree(value, scope_stack)
+            add_variables_from_lhs(var, scope_stack)
+          end
+        end
+        if rest && rest[1]
+          add_variables_from_lhs(rest, scope_stack)
+        end
+        if positional_2
+          positional_2.each { |var| add_variables_from_lhs(var, scope_stack) }
+        end
+        if block
+          add_variables_from_lhs(block, scope_stack)
+        end
+      end
+    end
+
+    # Deep-copies the sexp. I wish Array#clone did deep copies...
+    def clone_sexp(node)
+      node.map do |part|
+        # guess: arrays most common, then un-dupables, then dup-ables (strings only)
+        if Array === part
+          clone_sexp(part)
+        elsif [Symbol, Fixnum, TrueClass, FalseClass, NilClass].any? { |k| k === part }
+          part
+        else
+          part.dup
+        end
+      end
+    end
+    private :clone_sexp
+  end
+end

data/spec/ripper-plus_spec.rb

@@ -0,0 +1,5 @@
+require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
+
+describe "RipperPlus" do
+
+end

data/spec/scope_stack_spec.rb

@@ -0,0 +1,29 @@
+require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
+
+describe RipperPlus::ScopeStack do
+  before do
+    @stack = RipperPlus::ScopeStack.new
+  end
+  
+  it 'can see variables added at top level' do
+    @stack.should_not have_variable(:hello)
+    @stack.add_variable :hello
+    @stack.should have_variable(:hello)
+  end
+  
+  it 'can not see variables added beyond a closed scope' do
+    @stack.add_variable :hello
+    @stack.with_closed_scope do
+      @stack.should_not have_variable(:hello)
+    end
+    @stack.should have_variable(:hello)
+  end
+  
+  it 'can see variables added beyond an open scope' do
+    @stack.add_variable :hello
+    @stack.with_open_scope do
+      @stack.should have_variable(:hello)
+    end
+    @stack.should have_variable(:hello)
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,18 @@
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+$LOAD_PATH.unshift(File.dirname(__FILE__))
+require 'rspec'
+require 'ripper-plus'
+
+# Requires supporting files with custom matchers and macros, etc,
+# in ./support/ and its subdirectories.
+Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}
+
+RSpec::Matchers.define :transform_to do |output|
+  match do |input|
+    RipperPlus::Transformer.transform(input) == output
+  end
+end
+
+RSpec.configure do |config|
+  
+end

data/spec/transformer_spec.rb

@@ -0,0 +1,467 @@
+require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
+
+describe RipperPlus::Transformer do
+  it 'should transform a simple zcall in a method' do
+    input_tree =
+      [:program,
+       [[:def,
+         [:@ident, "foo", [1, 4]],
+         [:paren, [:params, [[:@ident, "x", [1, 8]]], nil, nil, nil, nil]],
+         [:bodystmt,
+          [[:void_stmt],
+           [:var_ref, [:@ident, "y", [1, 12]]],
+           [:assign,
+            [:var_field, [:@ident, "y", [1, 15]]],
+            [:var_ref, [:@ident, "x", [1, 19]]]]],
+          nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:def,
+         [:@ident, "foo", [1, 4]],
+         [:paren, [:params, [[:@ident, "x", [1, 8]]], nil, nil, nil, nil]],
+         [:bodystmt,
+          [[:void_stmt],
+           [:zcall, [:@ident, "y", [1, 12]]],
+           [:assign,
+            [:var_field, [:@ident, "y", [1, 15]]],
+            [:var_ref, [:@ident, "x", [1, 19]]]]],
+          nil, nil, nil]]]]
+    input_tree.should transform_to(output_tree)
+  end
+
+  it 'should respect argument order in method definitions' do
+    input_tree =
+      [:program,
+       [[:def,
+         [:@ident, "foo", [1, 4]],
+         [:paren,
+          [:params,
+           [[:@ident, "x", [1, 8]]],
+           [[[:@ident, "y", [1, 11]], [:var_ref, [:@ident, "z", [1, 13]]]],
+            [[:@ident, "z", [1, 16]], [:var_ref, [:@ident, "y", [1, 18]]]]],
+           nil,
+           nil,
+           nil]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:def,
+         [:@ident, "foo", [1, 4]],
+         [:paren,
+          [:params,
+           [[:@ident, "x", [1, 8]]],
+           [[[:@ident, "y", [1, 11]], [:zcall, [:@ident, "z", [1, 13]]]],
+            [[:@ident, "z", [1, 16]], [:var_ref, [:@ident, "y", [1, 18]]]]],
+           nil,
+           nil,
+           nil]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    input_tree.should transform_to(output_tree)
+  end
+
+  it 'should respect argument order in block argument definitions' do
+    input_tree =
+      [:program,
+       [[:method_add_block,
+         [:zsuper],
+         [:do_block,
+          [:block_var,
+           [:params,
+            [[:@ident, "x", [1, 10]]],
+            [[[:@ident, "y", [1, 13]], [:var_ref, [:@ident, "x", [1, 15]]]],
+             [[:@ident, "z", [1, 18]], [:var_ref, [:@ident, "a", [1, 20]]]]],
+            [:rest_param, [:@ident, "a", [1, 24]]],
+            nil,
+            nil],
+           nil],
+          [[:void_stmt],
+           [:command,
+            [:@ident, "p", [1, 28]],
+            [:args_add_block,
+             [[:var_ref, [:@ident, "x", [1, 30]]],
+              [:var_ref, [:@ident, "y", [1, 33]]],
+              [:var_ref, [:@ident, "z", [1, 36]]],
+              [:var_ref, [:@ident, "a", [1, 39]]]],
+             false]]]]]]]
+    output_tree =
+      [:program,
+       [[:method_add_block,
+         [:zsuper],
+         [:do_block,
+          [:block_var,
+           [:params,
+            [[:@ident, "x", [1, 10]]],
+            [[[:@ident, "y", [1, 13]], [:var_ref, [:@ident, "x", [1, 15]]]],
+             [[:@ident, "z", [1, 18]], [:zcall, [:@ident, "a", [1, 20]]]]],
+            [:rest_param, [:@ident, "a", [1, 24]]],
+            nil,
+            nil],
+           nil],
+          [[:void_stmt],
+           [:command,
+            [:@ident, "p", [1, 28]],
+            [:args_add_block,
+             [[:var_ref, [:@ident, "x", [1, 30]]],
+              [:var_ref, [:@ident, "y", [1, 33]]],
+              [:var_ref, [:@ident, "z", [1, 36]]],
+              [:var_ref, [:@ident, "a", [1, 39]]]],
+             false]]]]]]]
+    input_tree.should transform_to(output_tree)
+  end
+
+  it 'should transform singleton names in singleton method definitions' do
+    input_tree =
+      [:program,
+       [[:defs,
+         [:var_ref, [:@ident, "foo", [1, 4]]],
+         [:@period, ".", [1, 7]],
+         [:@ident, "silly", [1, 8]],
+         [:paren, [:params, nil, nil, nil, nil, nil]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:defs,
+         [:zcall, [:@ident, "foo", [1, 4]]],
+         [:@period, ".", [1, 7]],
+         [:@ident, "silly", [1, 8]],
+         [:paren, [:params, nil, nil, nil, nil, nil]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    input_tree.should transform_to(output_tree)
+  end
+  
+  it 'should not transform singleton names when an appropriate local variable exists' do
+    input_tree =
+      [:program,
+       [[:assign,
+         [:var_field, [:@ident, "foo", [1, 0]]],
+         [:var_ref, [:@kw, "self", [1, 6]]]],
+        [:defs,
+         [:var_ref, [:@ident, "foo", [1, 16]]],
+         [:@period, ".", [1, 19]],
+         [:@ident, "silly", [1, 20]],
+         [:params, nil, nil, nil, nil, nil],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    input_tree.should transform_to(input_tree)
+  end
+  
+  it 'does not transform the tricky x = 5 unless defined?(x) case' do
+    input_tree =
+      [:program,
+       [[:unless_mod,
+         [:defined, [:var_ref, [:@ident, "x", [1, 22]]]],
+         [:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]]]]]
+    input_tree.should transform_to(input_tree)
+  end
+  
+  it 'finds all LHS vars before transforming an MLHS' do
+    input_tree =
+      [:program,
+       [[:massign,
+         [[:@ident, "a", [1, 0]],
+          [:@const, "B", [1, 3]],
+          [:mlhs_paren,
+           [:mlhs_add_star,
+            [[:@gvar, "$c", [1, 7]]],
+            [:@ident, "rest", [1, 12]],
+            [[:@ident, "d", [1, 18]], [:@ident, "e", [1, 21]]]]]],
+         [:method_add_arg,
+          [:fcall, [:@ident, "foo", [1, 26]]],
+          [:arg_paren,
+           [:args_add_block,
+            [[:var_ref, [:@ident, "a", [1, 30]]],
+             [:var_ref, [:@ident, "c", [1, 33]]],
+             [:var_ref, [:@ident, "d", [1, 36]]]],
+            false]]]]]]
+    output_tree =
+      [:program,
+       [[:massign,
+         [[:@ident, "a", [1, 0]],
+          [:@const, "B", [1, 3]],
+          [:mlhs_paren,
+           [:mlhs_add_star,
+            [[:@gvar, "$c", [1, 7]]],
+            [:@ident, "rest", [1, 12]],
+            [[:@ident, "d", [1, 18]], [:@ident, "e", [1, 21]]]]]],
+         [:method_add_arg,
+          [:fcall, [:@ident, "foo", [1, 26]]],
+          [:arg_paren,
+           [:args_add_block,
+            [[:var_ref, [:@ident, "a", [1, 30]]],
+             [:zcall, [:@ident, "c", [1, 33]]],
+             [:var_ref, [:@ident, "d", [1, 36]]]],
+            false]]]]]]
+    input_tree.should transform_to(output_tree)
+  end
+  
+  it 'creates for-loop MLHS vars before transforming the iteratee' do
+    input_tree =
+      [:program,
+       [[:for,
+         [[:@ident, "a", [1, 4]],
+          [:@ident, "b", [1, 7]],
+          [:mlhs_paren,
+           [[:@ident, "c", [1, 11]],
+            [:mlhs_paren,
+             [:mlhs_add_star,
+              [],
+              [:@ident, "d", [1, 16]],
+              [[:@ident, "e", [1, 19]]]]]]]],
+         [:method_add_arg,
+          [:fcall, [:@ident, "foo", [1, 26]]],
+          [:arg_paren,
+           [:args_add_block,
+            [[:var_ref, [:@ident, "a", [1, 30]]],
+             [:var_ref, [:@ident, "b", [1, 33]]],
+             [:var_ref, [:@ident, "c", [1, 36]]],
+             [:var_ref, [:@ident, "d", [1, 39]]],
+             [:var_ref, [:@ident, "e", [1, 42]]],
+             [:var_ref, [:@ident, "f", [1, 45]]],
+             [:var_ref, [:@ident, "g", [1, 48]]]],
+            false]]],
+         [[:void_stmt]]]]]
+    output_tree =
+      [:program,
+       [[:for,
+         [[:@ident, "a", [1, 4]],
+          [:@ident, "b", [1, 7]],
+          [:mlhs_paren,
+           [[:@ident, "c", [1, 11]],
+            [:mlhs_paren,
+             [:mlhs_add_star,
+              [],
+              [:@ident, "d", [1, 16]],
+              [[:@ident, "e", [1, 19]]]]]]]],
+         [:method_add_arg,
+          [:fcall, [:@ident, "foo", [1, 26]]],
+          [:arg_paren,
+           [:args_add_block,
+            [[:var_ref, [:@ident, "a", [1, 30]]],
+             [:var_ref, [:@ident, "b", [1, 33]]],
+             [:var_ref, [:@ident, "c", [1, 36]]],
+             [:var_ref, [:@ident, "d", [1, 39]]],
+             [:var_ref, [:@ident, "e", [1, 42]]],
+             [:zcall, [:@ident, "f", [1, 45]]],
+             [:zcall, [:@ident, "g", [1, 48]]]],
+            false]]],
+         [[:void_stmt]]]]]
+    input_tree.should transform_to(output_tree)
+  end
+  
+  it 'should transform respecting subassignments in block arguments' do
+    input_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "10", [1, 4]]],
+        [:method_add_block,
+         [:call,
+          [:top_const_ref, [:@const, "ARR", [1, 10]]],
+          :".",
+          [:@ident, "each", [1, 14]]],
+         [:brace_block,
+          [:block_var,
+           [:params,
+            [[:@ident, "y", [1, 22]],
+             [:mlhs_paren,
+              [[:mlhs_paren, [:@ident, "z", [1, 26]]],
+               [:mlhs_paren, [:@ident, "a", [1, 29]]]]]],
+            [[[:@ident, "k", [1, 33]], [:var_ref, [:@ident, "z", [1, 37]]]],
+             [[:@ident, "j", [1, 40]], [:var_ref, [:@ident, "d", [1, 44]]]]],
+            [:rest_param, [:@ident, "r", [1, 48]]],
+            nil,
+            [:blockarg, [:@ident, "blk", [1, 52]]]],
+           [[:@ident, "local", [1, 57]]]],
+          [[:command,
+            [:@ident, "p", [1, 64]],
+            [:args_add_block,
+             [[:var_ref, [:@ident, "x", [1, 66]]],
+              [:var_ref, [:@ident, "y", [1, 69]]],
+              [:var_ref, [:@ident, "z", [1, 72]]],
+              [:var_ref, [:@ident, "a", [1, 75]]],
+              [:var_ref, [:@ident, "k", [1, 78]]],
+              [:var_ref, [:@ident, "j", [1, 81]]],
+              [:var_ref, [:@ident, "r", [1, 84]]],
+              [:var_ref, [:@ident, "blk", [1, 87]]],
+              [:var_ref, [:@ident, "local", [1, 92]]],
+              [:var_ref, [:@ident, "foo", [1, 99]]]],
+             false]]]]]]]
+    output_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "10", [1, 4]]],
+        [:method_add_block,
+         [:call,
+          [:top_const_ref, [:@const, "ARR", [1, 10]]],
+          :".",
+          [:@ident, "each", [1, 14]]],
+         [:brace_block,
+          [:block_var,
+           [:params,
+            [[:@ident, "y", [1, 22]],
+             [:mlhs_paren,
+              [[:mlhs_paren, [:@ident, "z", [1, 26]]],
+               [:mlhs_paren, [:@ident, "a", [1, 29]]]]]],
+            [[[:@ident, "k", [1, 33]], [:var_ref, [:@ident, "z", [1, 37]]]],
+             [[:@ident, "j", [1, 40]], [:zcall, [:@ident, "d", [1, 44]]]]],
+            [:rest_param, [:@ident, "r", [1, 48]]],
+            nil,
+            [:blockarg, [:@ident, "blk", [1, 52]]]],
+           [[:@ident, "local", [1, 57]]]],
+          [[:command,
+            [:@ident, "p", [1, 64]],
+            [:args_add_block,
+             [[:var_ref, [:@ident, "x", [1, 66]]],
+              [:var_ref, [:@ident, "y", [1, 69]]],
+              [:var_ref, [:@ident, "z", [1, 72]]],
+              [:var_ref, [:@ident, "a", [1, 75]]],
+              [:var_ref, [:@ident, "k", [1, 78]]],
+              [:var_ref, [:@ident, "j", [1, 81]]],
+              [:var_ref, [:@ident, "r", [1, 84]]],
+              [:var_ref, [:@ident, "blk", [1, 87]]],
+              [:var_ref, [:@ident, "local", [1, 92]]],
+              [:zcall, [:@ident, "foo", [1, 99]]]],
+             false]]]]]]]
+    input_tree.should transform_to(output_tree)  
+  end
+  
+  it 'does not let block variables escape into enclosing scopes' do
+    input_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:method_add_block,
+         [:call, [:array, nil], :".", [:@ident, "each", [1, 10]]],
+         [:brace_block,
+          [:block_var,
+           [:params, [[:@ident, "y", [1, 18]]], nil, nil, nil, nil],
+           [[:@ident, "z", [1, 20]], [:@ident, "x", [1, 23]]]],
+          [[:var_ref, [:@ident, "x", [1, 26]]],
+           [:var_ref, [:@ident, "y", [1, 28]]],
+           [:var_ref, [:@ident, "z", [1, 30]]]]]],
+        [:var_ref, [:@ident, "x", [1, 36]]],
+        [:var_ref, [:@ident, "y", [1, 39]]],
+        [:var_ref, [:@ident, "z", [1, 42]]]]]
+    output_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:method_add_block,
+         [:call, [:array, nil], :".", [:@ident, "each", [1, 10]]],
+         [:brace_block,
+          [:block_var,
+           [:params, [[:@ident, "y", [1, 18]]], nil, nil, nil, nil],
+           [[:@ident, "z", [1, 20]], [:@ident, "x", [1, 23]]]],
+          [[:var_ref, [:@ident, "x", [1, 26]]],
+           [:var_ref, [:@ident, "y", [1, 28]]],
+           [:var_ref, [:@ident, "z", [1, 30]]]]]],
+        [:var_ref, [:@ident, "x", [1, 36]]],
+        [:zcall, [:@ident, "y", [1, 39]]],
+        [:zcall, [:@ident, "z", [1, 42]]]]]
+    input_tree.should transform_to(output_tree)
+  end
+  
+  it 'creates closed scopes when classes are created' do
+    input_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:class,
+         [:const_ref, [:@const, "Foo", [1, 13]]],
+         nil,
+         [:bodystmt, [[:var_ref, [:@ident, "x", [1, 18]]]], nil, nil, nil]],
+        [:var_ref, [:@ident, "x", [1, 26]]]]]
+    output_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:class,
+         [:const_ref, [:@const, "Foo", [1, 13]]],
+         nil,
+         [:bodystmt, [[:zcall, [:@ident, "x", [1, 18]]]], nil, nil, nil]],
+        [:var_ref, [:@ident, "x", [1, 26]]]]]
+    input_tree.should transform_to(output_tree)
+  end
+  
+  it 'uses the previous scope for superclass specification' do
+    input_tree =
+      [:program,
+       [[:assign,
+         [:var_field, [:@ident, "x", [1, 0]]],
+         [:var_ref, [:@const, "String", [1, 4]]]],
+        [:class,
+         [:const_ref, [:@const, "A", [1, 18]]],
+         [:var_ref, [:@ident, "x", [1, 22]]],
+         [:bodystmt, [[:var_ref, [:@ident, "x", [1, 25]]]], nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:assign,
+         [:var_field, [:@ident, "x", [1, 0]]],
+         [:var_ref, [:@const, "String", [1, 4]]]],
+        [:class,
+         [:const_ref, [:@const, "A", [1, 18]]],
+         [:var_ref, [:@ident, "x", [1, 22]]],
+         [:bodystmt, [[:zcall, [:@ident, "x", [1, 25]]]], nil, nil, nil]]]]
+    input_tree.should transform_to output_tree
+  end
+  
+  it 'creates closed scopes when modules are created' do
+    input_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:module,
+         [:const_ref, [:@const, "Foo", [1, 14]]],
+         [:bodystmt,
+          [[:void_stmt], [:var_ref, [:@ident, "x", [1, 19]]]],
+          nil,
+          nil,
+          nil]],
+        [:var_ref, [:@ident, "x", [1, 27]]]]]
+    output_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:module,
+         [:const_ref, [:@const, "Foo", [1, 14]]],
+         [:bodystmt,
+          [[:void_stmt], [:zcall, [:@ident, "x", [1, 19]]]],
+          nil,
+          nil,
+          nil]],
+        [:var_ref, [:@ident, "x", [1, 27]]]]]
+    input_tree.should transform_to output_tree
+  end
+  
+  it 'creates closed scopes when singleton classes are opened' do
+    input_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:sclass,
+         [:var_ref, [:@const, "String", [1, 16]]],
+         [:bodystmt, [[:var_ref, [:@ident, "x", [1, 24]]]], nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:assign, [:var_field, [:@ident, "x", [1, 0]]], [:@int, "5", [1, 4]]],
+        [:sclass,
+         [:var_ref, [:@const, "String", [1, 16]]],
+         [:bodystmt, [[:zcall, [:@ident, "x", [1, 24]]]], nil, nil, nil]]]]
+    input_tree.should transform_to output_tree
+  end
+  
+  it 'refers to the enclosing environment to determine the singleton to open' do
+    input_tree =
+      [:program,
+       [[:assign,
+         [:var_field, [:@ident, "x", [1, 0]]],
+         [:string_literal, [:string_content, [:@tstring_content, "hi", [1, 5]]]]],
+        [:sclass,
+         [:var_ref, [:@ident, "x", [1, 19]]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    input_tree.should transform_to input_tree
+  end
+  
+  it 'refers to the enclosing environment to determine the singleton to open - resulting in zcall' do
+    input_tree =
+      [:program,
+       [[:sclass,
+         [:var_ref, [:@ident, "x", [1, 19]]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    output_tree =
+      [:program,
+       [[:sclass,
+         [:zcall, [:@ident, "x", [1, 19]]],
+         [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]
+    input_tree.should transform_to output_tree
+  end
+end

metadata

@@ -0,0 +1,132 @@
+--- !ruby/object:Gem::Specification 
+name: ripper-plus
+version: !ruby/object:Gem::Version 
+  prerelease: 
+  version: 1.0.0
+platform: ruby
+authors: 
+- Michael Edgar
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2011-04-27 00:00:00 Z
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: rspec
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 2.3.0
+  type: :development
+  prerelease: false
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
+  name: yard
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 0.6.0
+  type: :development
+  prerelease: false
+  version_requirements: *id002
+- !ruby/object:Gem::Dependency 
+  name: bundler
+  requirement: &id003 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 1.0.0
+  type: :development
+  prerelease: false
+  version_requirements: *id003
+- !ruby/object:Gem::Dependency 
+  name: jeweler
+  requirement: &id004 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ~>
+      - !ruby/object:Gem::Version 
+        version: 1.5.2
+  type: :development
+  prerelease: false
+  version_requirements: *id004
+- !ruby/object:Gem::Dependency 
+  name: rcov
+  requirement: &id005 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: "0"
+  type: :development
+  prerelease: false
+  version_requirements: *id005
+description: |-
+  Ripper is the Ruby parser library packaged
+  with Ruby 1.9. While quite complete, it has some quirks that can
+  make use frustrating. This gem intends to correct them.
+email: michael.j.edgar@dartmouth.edu
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE.txt
+- README.md
+files: 
+- .document
+- .rspec
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- VERSION
+- lib/ripper-plus.rb
+- lib/ripper-plus/ripper-plus.rb
+- lib/ripper-plus/scope_stack.rb
+- lib/ripper-plus/transformer.rb
+- spec/ripper-plus_spec.rb
+- spec/scope_stack_spec.rb
+- spec/spec_helper.rb
+- spec/transformer_spec.rb
+homepage: http://github.com/michaeledgar/ripper-plus
+licenses: 
+- MIT
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 505813799173151565
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.7.2
+signing_key: 
+specification_version: 3
+summary: Parses Ruby code into an improved Ripper AST format
+test_files: 
+- spec/ripper-plus_spec.rb
+- spec/scope_stack_spec.rb
+- spec/spec_helper.rb
+- spec/transformer_spec.rb