furnace-avm2 1.0.0 → 1.0.1

data/README.md

@@ -0,0 +1,122 @@
+Furnace-AVM2
+============
+
+Furnace-AVM2 is a library for manipulating Adobe Flash ActionScript 3 bytecode. The library contains routines for reading and writing the bytecode in native Flash binary format and transformations between various flavors of abstract syntax trees suited for automatic and manual analysis.
+
+This library can solve wide range of tasks, including but not limited to:
+
+ * **Deobfuscation.** Currently, only an opcode-level dead code eliminator is provided, which is nevertheless proved itself quite useful. A name normalization routine which transforms the names in the source code back to a human-readable form is also provided.
+ * **Decompilation.** One of the provided AST transformations emits back the source code in ActionScript 3. One of the development goals was to produce code which can be compiled back and perform the identical actions. It was (mostly) achieved.
+ * **Behavioral matching of code.** Furnace matchers provide a powerful [regular language](http://en.wikipedia.org/wiki/Regular_languages) and can be used to locate statement level or control flow level constructs. For example, a matcher could:
+
+    * Find all statement sequences of the form
+
+            var <var>:String = Loader.loadURL(<url>);
+            trace(<var>);
+
+      where `<x>` correspond to wildcards which capture and remember values.
+
+    * Find all loops of the form
+
+            for(var <var>:<type> in [object.getParent().getList()|object.getList()]) {
+              ...
+            	var descendant:<type> = <var>.getDescendant();
+            	...
+            }
+
+      where `<x>` correspond to wildcards as described above and `[y|x]` means that either of variants `y` or `x` is accepted in place of the construct.
+
+ * **Patching.** The library does not allow transformation of abstract syntax trees back to bytecode, but it retails a lot of information about origin of constructs and allows to modify every other aspect of the bitstream.
+
+The library supports all known AVM2 opcodes, including undocumented (generic types) and Alchemy ones. Most of these opcodes are supported down to source level, with a notable exception of some [E4X](http://en.wikipedia.org/wiki/E4X), which are just too braindead to implement.
+
+The library is extensible. A new transformation can easily be plugged in, shall such a need to arise. Adding support for a certain obfuscator boils down to adding one or two stages to the pipeline, which would normalize the mangled code.
+
+The library is portable and fast. It works on 1.9 rubies: MRI, JRuby and (if you're lucky) Rubinius. It can decompile circa 9000 methods per 30s (on JRuby 1.7 and 8-core Intel i7).
+
+Installation
+------------
+
+Furnace-AVM2 is written in Ruby 1.9. You will need to install a compatible Ruby implementation. JRuby is recommended as supports real multithreading mode, but Ruby MRI is also acceptable.
+
+Install the required gems:
+
+    $ gem install furnace-avm2 furnace-swf
+
+Command line interface
+----------------------
+
+Furnace-AVM2 has two main command-line utilites, `furnace-avm2` and `furnace-avm2-decompiler`. There is also a supplementary utility called `furnace-swf` which is contained in the gem with the same name.
+
+Note that this library only operates with raw bytecode. It does not know anything about SWF files nor can it parse them.
+
+To analyze a real-world file, which most certainly will be an SWF, you will need to use `furnace-swf` first. It can parse the whole SWF file (including compressed ones), but only supports DoABC2 tags which contain AVM2 bytecode.
+
+The `furnace-swf` utility currently has three subcommands, `abclist`, `abcextract` and `abcreplace`. They should be mostly self-explanatory; an example session is shown below.
+
+    $ furnace-swf -i sample.swf abclist
+    ABC tags:
+      "frame1": 1488672 byte(s)
+    $ furnace-swf -i sample.swf abcextract -n frame1 -o frame1.abc
+    $
+
+After you have extracted the AVM2 bytecode, you can use Furnace-AVM2 itself. First, if you think that the file might be obfuscated, you need to preprocess it to clean the obfuscation artifacts. Run the `furnace-avm2` utility in DCE mode (the names are normalized by default; if you don't need that, pass `-q`).
+
+    $ furnace-avm2 -i frame1.abc -d -o frame1.abc
+    $
+
+`furnace-avm2` generally works on the method level. It builds a set of methods to work on (`-O` and `-E` options), and then performs various transformations on them. If you need to determine why a particular method fails or to retrieve AST in free-text form, it's a right utility to use. Check its inline help and don't hesitate to experiment with different options.
+
+Contrary to that, `furnace-avm2-decompiler` works on a class level. You can include and exclude objects to decompile with the class granularity, and it doesn't have much more configuration than that.
+
+    $ furnace-avm2-decompiler -i frame1.abc -d -D funids >frame1.as
+    Reading input data...
+    Found 2434 classes and packages.
+    Decompiling... 2402/2434 /
+               Decompiled: 9167/9201 (99%)
+     Partially decompiled: 8/9201 (0%)
+                   Failed: 26/9201 (0%)
+    Time taken: 69.27s
+    $
+
+The `-D funids` option adds a comment with method body index for each decompiled method. It can be used for debugging decompiler failures.
+
+You'll notice that some methods probably will not get decompiled. (The file I used in this example is quite complex.) Not every possible bytecode sequence can be directly represented in ActionScript 3, and there are some corner cases yet to be described in the decompiler. For "partially decompiled" (i.e. where there were no control flow uncertainites, but some expressions were impossible to transform to ActionScript) the relevant NF-AST code is automatically emitted. You can look at it manually with `furnace-avm2 -n`. For "failed" methods there is no generated code, but you might try to look at control flow graph (`furnace-avm2 -C`, look for emitted `method-*.dot` file) in [Graphviz](http://en.wikipedia.org/wiki/Graphviz) format to understand the logic.
+
+Programming interface
+---------------------
+
+The programming interface will get an in-depth description later. For now, you are advised to look at the source code of [ABC metadata](https://github.com/whitequark/furnace-avm2/tree/master/lib/furnace-avm2/abc/metadata) parser/storage code and [Furnace](https://github.com/whitequark/furnace/tree/master/lib/furnace) source code. Neither of these are particularly large, and you will probably need to read it anyway.
+
+You can also use [Pry](http://pry.github.com/) to explore the interfaces. Try launching the utility `furnace-avm2-shell`.
+
+Contact
+-------
+
+If you experience any difficultes, you can ask me (*whitequark*) on channel `#ruby-lang` at `irc.freenode.net` or drop me a email.
+
+License
+-------
+
+Furnace-AVM2 is distributed under the terms of MIT license.
+
+    Copyright (c) 2012  Peter Zotov <whitequark@whitequark.org>
+
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the
+    "Software"), to deal in the Software without restriction, including
+    without limitation the rights to use, copy, modify, merge, publish,
+    distribute, sublicense, and/or sell copies of the Software, and to
+    permit persons to whom the Software is furnished to do so, subject to
+    the following conditions:
+
+    The above copyright notice and this permission notice shall be included
+    in all copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+    OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+    MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+    IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+    CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/furnace-avm2.gemspec

@@ -17,6 +17,6 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib"]
 
-  s.add_runtime_dependency "furnace", '= 0.2.3'
+  s.add_runtime_dependency "furnace", '= 0.2.4'
   s.add_runtime_dependency "trollop"
 end

data/lib/furnace-avm2/source/decompiler.rb

@@ -688,6 +688,41 @@ module Furnace::AVM2
     alias :expr_pre_increment_local  :expr_prepost_incdec_local
     alias :expr_pre_decrement_local  :expr_prepost_incdec_local
 
+    PrePostIncDecSlot = Matcher.new do
+      [any,
+        capture(:index),
+        either[
+          [:get_global_scope],
+          [:get_scope_object, capture(:scope_pos)]
+        ]
+      ]
+    end
+
+    def expr_prepost_incdec_slot(opcode)
+      if captures = PrePostIncDecSlot.match(opcode)
+        scope = @scopes[captures[:scope_pos] || 0]
+
+        if @closure_slots && scope == :activation
+          slot_trait = @closure_slots[captures[:index]]
+          slot = token(VariableNameToken, slot_trait.name.name)
+
+          if opcode.type == :post_increment_slot
+            token(UnaryPostOperatorToken, slot, "++")
+          elsif opcode.type == :post_decrement_slot
+            token(UnaryPostOperatorToken, slot, "--")
+          elsif opcode.type == :pre_increment_slot
+            token(UnaryOperatorToken, slot, "++")
+          elsif opcode.type == :pre_decrement_slot
+            token(UnaryOperatorToken, slot, "--")
+          end
+        end
+      end
+    end
+    alias :expr_post_increment_slot :expr_prepost_incdec_slot
+    alias :expr_post_decrement_slot :expr_prepost_incdec_slot
+    alias :expr_pre_increment_slot  :expr_prepost_incdec_slot
+    alias :expr_pre_decrement_slot  :expr_prepost_incdec_slot
+
     OPERATOR_MAP = {
       :and         => :"&&",
       :or          => :"||",

data/lib/furnace-avm2/transform/ast_build.rb

@@ -15,7 +15,8 @@ module Furnace::AVM2
       SHORT_ASSIGN_OPERATORS = [ :add, :add_i, :subtract, :subtract_i, :multiply, :multiply_i,
                                  :divide, :modulo,
                                  :set_local, :set_local_0, :set_local_1, :set_local_2, :set_local_3,
-                                 :new_catch, :new_activation ]
+                                 :new_catch, :new_activation,
+                                 :next_value ]
 
       def initialize(options)
         @validate = options[:validate] || false

data/lib/furnace-avm2/transform/cfg_build.rb

@@ -59,7 +59,7 @@ module Furnace::AVM2
           next_label = next_node.metadata[:label] if next_node
 
           case node.type
-          when :return_value, :return_void
+          when :return_value, :return_void, :throw
             cutoff(nil, [nil])
 
           when :jump

data/lib/furnace-avm2/transform/cfg_reduce.rb

@@ -112,7 +112,7 @@ module Furnace::AVM2
                     loop_stack.include?(@loop_tails[block])
       end
 
-      def extended_block(block, stopgap=nil, loop_stack=[], nesting=0, upper_exc=nil)
+      def extended_block(block, stopgap=nil, loop_stack=[], nesting=0, upper_exc=nil, options={})
         nodes = []
         prev_block = nil
         current_exception = upper_exc
@@ -125,13 +125,20 @@ module Furnace::AVM2
           log nesting, "BLOCK: #{block.inspect}"
 
           if is_loop_head?(block, loop_stack)
-            log nesting, "exit: loop head (continue stmt)"
+            if options[:infinite_loop_head]
+              # Infinite loop head is a special case where cti_block
+              # has back edges pointing to it, but just for once it
+              # should not be turned to (continue) statement.
+              options.delete(:infinite_loop_head)
+            else
+              log nesting, "exit: loop head (continue stmt)"
 
-            check_nonlocal_loop(loop_stack, block) do |params|
-              current_nodes << AST::Node.new(:continue, params)
-            end
+              check_nonlocal_loop(loop_stack, block) do |params|
+                current_nodes << AST::Node.new(:continue, params)
+              end
 
-            break
+              break
+            end
           elsif is_loop_tail?(block, loop_stack)
             log nesting, "exit: loop tail (break stmt)"
 
@@ -183,6 +190,8 @@ module Furnace::AVM2
             if block.cti.type == :lookup_switch
               log nesting, "is a switch"
 
+              append_instructions(block, current_nodes)
+
               # Group cases pointing to the same blocks of code.
               aliases = Hash[block.targets.each_index.
                     group_by { |index| block.targets[index] }.values.
@@ -270,8 +279,8 @@ module Furnace::AVM2
                     body.children << AST::Node.new(:break)
                   end
 
-                  main_index = case_branches.index(next_branch)
-                  body = case_bodies[main_index]
+                  main_index = block.targets.index(next_branch)
+                  body = case_bodies[case_branches.index(next_branch)]
 
                   [ main_index, *aliases[main_index] ].each do |index|
                     if index == 0
@@ -299,7 +308,11 @@ module Furnace::AVM2
               block = exit_point
             elsif @loops.include?(block) && !@postcond_heads.include?(block)
               # we're trapped in a strange loop
-              if block.insns.first == block.cti
+              if block.insns.first == block.cti &&
+                    !(@loops[block].include?(block.targets.first) &&
+                      @loops[block].include?(block.targets.last))
+                # Make sure that both branch targets don't reside within the
+                # loop. If they do, it's a do-while loop.
                 log nesting, "is a while loop"
 
                 loop_type = :head_cti
@@ -317,10 +330,15 @@ module Furnace::AVM2
                 end
 
                 if back_edges.count == 1
+                  log nesting, "is a do-while loop"
+
                   loop_type = :tail_cti
                   cti_block = back_edges.first
                 else
-                  raise "invalid back edge count"
+                  log nesting, "is an infinite loop"
+
+                  loop_type = :infinite
+                  cti_block = block
                 end
               end
 
@@ -335,9 +353,9 @@ module Furnace::AVM2
                 reverse = !cti_block.cti.children[0]
                 in_root, out_root = cti_block.targets
 
-                # One of the branch targets should reside within
-                # the loop.
                 if !@loops[block].include?(in_root)
+                  # One of the branch targets should reside within
+                  # the loop.
                   in_root, out_root = out_root, in_root
                   reverse = !reverse
                 end
@@ -366,7 +384,8 @@ module Furnace::AVM2
 
                 append_instructions(block, body.children)
               else
-                body = extended_block(in_root, nil, [ cti_block ] + loop_stack, nesting + 1, current_exception)
+                body = extended_block(in_root, nil, [ cti_block ] + loop_stack, nesting + 1, current_exception,
+                        { infinite_loop_head: (loop_type == :infinite) })
               end
 
               # [(label name)]
@@ -463,9 +482,11 @@ module Furnace::AVM2
               if completely_dominated?(right_root, block)
                 # Yes. Find merge point.
 
-                # The function technically finds two merge points,
-                # but in case of two heads they're identical.
-                merge, = find_merge_point([ left_root, right_root ])
+                merge_left, merge_right = find_merge_point([ left_root, right_root ])
+
+                # One or both of the merge points could be nil, but they will
+                # not be different.
+                merge = merge_left || merge_right
 
                 # If the merge search did not yield a valid node, use
                 # stopgap for the current block to avoid runaway code

data/lib/furnace-avm2/transform/nf_normalize.rb

@@ -23,10 +23,15 @@ module Furnace::AVM2
       end
 
       LocalIncDecMatcher = AST::Matcher.new do
-        [:set_local, capture(:index),
+        [ either_multi[
+            [:set_slot,  capture(:index), capture(:scope)],
+            [:set_local, capture(:index)],
+          ],
           either[
             [:convert, any,
               capture(:inner)],
+            [:coerce, :any,
+              capture(:inner)],
             capture(:inner)
           ]
         ]
@@ -36,13 +41,19 @@ module Furnace::AVM2
         [capture(:operator),
           either[
             [:convert, any,
-              [:get_local, backref(:index)]],
-            [:get_local, backref(:index)],
-            capture(:abnormal)
+              capture(:getter)],
+            capture(:getter),
           ]
         ]
       end
 
+      LocalIncDecGetterMatcher = AST::Matcher.new do
+        either[
+          [:get_slot,  backref(:index), backref(:scope)],
+          [:get_local, backref(:index)],
+        ]
+      end
+
       IncDecOperators = [
         :pre_increment, :post_increment,
         :pre_decrement, :post_decrement
@@ -51,22 +62,27 @@ module Furnace::AVM2
       def on_set_local(node)
         captures = {}
         if LocalIncDecMatcher.match(node, captures) &&
-              LocalIncDecInnerMatcher.match(captures[:inner], captures)
-          if IncDecOperators.include? captures[:operator]
-            if captures[:abnormal]
-              node.update(:add, [
-                AST::Node.new(:set_local, [
-                  captures[:index],
-                  captures[:abnormal]
-                ]),
-                AST::Node.new(:integer, [ 1 ])
-              ])
+              LocalIncDecInnerMatcher.match(captures[:inner], captures) &&
+              IncDecOperators.include?(captures[:operator])
+          if captures[:getter].is_a?(AST::Node) &&
+              LocalIncDecGetterMatcher.match(captures[:getter], captures)
+            if captures[:scope]
+              node.update(:"#{captures[:operator]}_slot", [ captures[:index], captures[:scope] ])
             else
               node.update(:"#{captures[:operator]}_local", [ captures[:index] ])
             end
+          else
+            node.update(:add, [
+              AST::Node.new(:set_local, [
+                captures[:index],
+                captures[:getter]
+              ]),
+              AST::Node.new(:integer, [ 1 ])
+            ])
           end
         end
       end
+      alias :on_set_slot :on_set_local
 
       ExpandedForInMatcher = AST::Matcher.new do
         [:if, [:has_next2, skip], skip]

data/lib/furnace-avm2/transform/propagate_constants.rb

@@ -5,45 +5,60 @@ module Furnace::AVM2
     class PropagateConstants
       include AST::Visitor
 
-      def transform(ast, *stuff)
-        @local_nonconst = Set.new
-        @local_sets = {}
-        @local_gets = Hash.new { |h,k| h[k] = [] }
+      class Replacer
+        include AST::Visitor
 
-        visit ast
+        def initialize(local_var, value)
+          @local_var, @value = local_var, value
+        end
 
-        @local_sets.each do |index, set_node|
-          *, value = set_node.children
+        def replace_in(nodes)
+          @nodes = nodes
+          @graceful_shutdown = true
 
-          unless @local_nonconst.include? index
-            @local_gets[index].each do |get_node|
-              get_node.update(:find_property_strict,
-                value.children.dup,
-                get_node.metadata)
+          catch(:stop) {
+            @nodes.each do |node|
+              visit node
             end
+          }
+
+          @graceful_shutdown
+        end
 
-            set_node.update(:nop, [])
+        def on_set_local(node)
+          index, value = node.children
+          if index == @local_var
+            @graceful_shutdown = @nodes.include?(node)
+            throw :stop
           end
         end
 
+        def on_get_local(node)
+          index, = node.children
+          if index == @local_var
+            node.update(@value.type, @value.children.dup, @value.metadata)
+          end
+        end
+      end
+
+      def transform(ast, *stuff)
+        visit ast
+
         [ ast, *stuff ]
       end
 
       def on_set_local(node)
         index, value = node.children
         if value.type == :find_property_strict
-          if @local_sets.has_key?(index)
-            @local_nonconst.add index
-          else
-            @local_sets[index] = node
+          block = node.parent
+          nodes = block.children[(block.children.index(node) + 1)..-1]
+
+          replacer = Replacer.new(index, value)
+          if replacer.replace_in(nodes)
+            node.update(:remove)
           end
         end
       end
-
-      def on_get_local(node)
-        index, = node.children
-        @local_gets[index].push node
-      end
     end
   end
 end

data/lib/furnace-avm2/version.rb

@@ -1,5 +1,5 @@
 module Furnace
   module AVM2
-    VERSION = "1.0.0"
+    VERSION = "1.0.1"
   end
 end