crystalizer 0.2.2

data/doc/optimizations.txt

@@ -0,0 +1,185 @@
+
+h1. Optimizations
+
+Without any optimizations enabled Ruby2CExtension tries to match Ruby's
+semantics as close as possible, but sometimes it might be desired to sacrifice
+some compatibility in exchange for faster execution.
+
+Ruby2CExtension offers different optimizations which can be enabled
+indivdually (see "rb2cx":rb2cx.html and "Eval2C":eval2c.html). All these
+optimizations should be safe, i.e. they won't produce segfaults or other C
+level failures, but, as mentioned above, they sacrifice compatibility for
+faster execution and thus can result in "wrong" behavior of the compiled Ruby
+code.
+
+
+h2. Constant Lookup Caching
+
+This optimization simply assumes that constants really are constant. So the
+constant lookup for each constant is only performed once and then the result
+is cached for later use.
+
+Because in Ruby constants can actually change their value, this optimization
+can lead to wrong results. So if your code depends on changing constants, then
+this optimization should not be enabled. Otherwise it can provide a small
+performance improvement.
+
+
+h2. Optimization of Calls to Built-in Methods
+
+This optimization is by far the most important one, because it can produce
+very significant speedups for most Ruby code. It replaces normal calls to many
+(C-implemented) methods of the built-in classes with (almost) direct calls to
+the C functions that implement these methods.
+
+The motivation behind this optimization is that method calls are generally
+considered to be one of the "slowest" parts of Ruby and since almost
+everything is method calls in Ruby it helps to improve at least some of them.
+A method call basically consists of two parts: first the method is looked up
+in the receivers class (or its ancestors) and then Ruby sets up the
+environment for the method and executes the code. For many of the
+C-implemented methods of the built-in classes the setup is not necessary and
+if we assume that they are generally not modified/redefined, we can also avoid
+the lookup. So we basically can directly call the C function that implements
+the method and thereby avoid most of the method call overhead in these cases.
+
+Because this can not be done for every C-implemented method, there is a list
+of method names (with arities and built-in types that have this method) for
+which the optimization can be applied. When a call to such a method is
+compiled, instead of doing a normal @rb_funcall()@, another C function is
+called which checks if the type of the receiver matches one of the built-in
+types that have this method and if yes, then the implementing C function is
+called directly otherwise a normal Ruby call is performed.
+
+To be able to directly call the C functions that implement the methods, the
+pointers to these C functions have to be looked up at least once. So, when the
+C extension is <code>require</code>d, for all methods that are used in this
+extension the pointers to the C functions are looked up. This is also kind of
+a sanity check: if that lookup fails or the lookup returns that the method is
+not implemented by a C function (e.g. because it was modified/overridden by a
+Ruby implementation), then this method will later be called by just using
+@rb_funcall()@.
+
+The built-in types which are included in this optimization are @Array@,
+@Bignum@, @FalseClass@, @Fixnum@, @Float@, @Hash@, @NilClass@, @Regexp@,
+@String@, @Symbol@ and @TrueClass@. For the detailed method list please see
+@lib/ruby2cext/plugins/builtin_methods.rb@.
+
+If the type of the receiver is known at compile time, then it is possibly to
+even further optimize by directly calling the implementing C function, instead
+of doing the indirect call with the runtime type checking. An example for such
+a case is the expression <code>1 == some_var</code>, in this expression the
+receiver is a fixnum. The compile time type detection also works for string
+literals, array literals, hash literals and regexp literals.
+
+As mentioned above, this optimization can speedup most Ruby code, but it also
+has downsides:
+
+* If the type of the receiver of such a method call is not one of the
+  supported built-in types, then the call will actually be slightly slower
+  than it would have been without the optimization (because of the additional
+  type check). But the overhead is very small.
+* If one of the supported methods is modified after the compiled C extension
+  is <code>require</code>d, then this modification will not be used in the
+  extension, the extension will just continue to use the C implementation.
+
+But in general this optimization can and should always be used, unless it is
+necessary to modify built-in methods after the extension is
+<code>require</code>d.
+
+
+h2. Inlining of Some Built-in Methods
+
+This optimization goes one step further than the previous optimization by
+avoiding the method calls completely and instead directly inlining the code
+for the following methods: @nil?@, @equal?@ and @__send__@ (without a block).
+
+Because these three methods should never be redefined in any class (according
+to their documentation), this optimization can be applied to all calls of
+these methods (with the correct arities) independent of the type of the
+receiver.
+
+Details:
+
+* Any call to @nil?@ with no arguments will be translated to a simple check
+  that determines if the receiver is @nil@.
+* Any call to @equal?@ with one argument will be translated to a simple check
+  that determines if the receiver is equal to the first argument (i.e. they
+  have the same id).
+* Any call to @__send__@ with one or more arguments and without a block will
+  be translated to code that directly performs the method call. As a result
+  <code>__send__(:foo, 1, 2, 3)</code> will perform practically as fast as
+  <code>foo(1, 2, 3)</code> in compiled code.
+
+In general this optimization can and should always be used, unless one of the
+affected methods is redefined somewhere.
+
+
+h2. Case Optimization
+
+This optimization can in some cases transform Ruby @case@/@when@ statements to
+real C @switch@/@case@ statements. This works for cases with Ruby immediate
+values, for which the integral value is known at compile time, i.e. @nil@,
+@true@, @false@ and fixnums.
+
+These cases have to be put at the beginning, all following non optimizable
+whens will be handled normally. Example:
+
+PRE
+case x
+when true
+  # opt
+when 2, 3, nil
+  # opt
+when false, 26
+  # opt
+when /foo/
+  # normal
+when 23
+  # normal because, it is after another normal case
+end
+PREEND
+
+This basically gets converted to the following C code:
+
+PRE
+switch (eval(x)) {
+case Qtrue:
+  code;
+  break;
+case LONG2FIX(2):
+case LONG2FIX(3):
+case Qnil:
+  code;
+  break;
+case Qfalse:
+case LONG2FIX(26):
+  code;
+  break;
+default:
+  other cases ...
+}
+PREEND
+
+There also is a fallback that ensures that cases like
+
+PRE
+case 2.0
+when 2
+  ...
+end
+PREEND
+
+still work.
+
+This optimization can be useful for some kind of opcode dispatch in an
+interpreter for example.
+
+It should always produce the correct results, unless the @===@ method of
+@Fixnum@, @TrueClass@, @FalseClass@ or @NilClass@ is modified. So it is
+generally okay to enable this optimization.
+
+When this optimization is combined with the built-in methods optimization,
+then the gained speedup is not really big (because the @===@ calls for the
+@when@ cases are optimized by the built-in methods optimization anyway), but
+it can be useful for larger @case@/@when@ statements.

data/doc/rb2cx.txt

@@ -0,0 +1,130 @@
+
+h1. rb2cx
+
+@rb2cx@ is the command line interface to the functionality of Ruby2CExtension.
+It takes one or more Ruby files, translates them to equivalent C extensions
+and optionally compiles them with the C compiler.
+
+
+h2. Overview
+
+The general usage is very simple, just run @rb2cx@ with the filenames of the
+Ruby files as arguments:
+
+PRE
+rb2cx file1.rb file2.rb
+PREEND
+
+This will produce @file1.c@, @file2.c@ and the compiled extensions @file1.so@,
+@file2.so@ (the file extension depends on the platform).
+
+Additionally it is possible to specify some options before the filenames.
+
+
+h2. General Options
+
+* @-h@/@--help@: a help message is printed and the program exits regardless of
+  other arguments.
+* @-c@/@--only-c@: only the translation to C code is performed, the compilation
+  to native extensions is omited.
+* @-v@/@--verbose@: some status messages are printed, e.g. which files are
+  currently translated or compiled.
+* @-w@/@--warnings@: warnings are printed for some things that might not work
+  as expected. The warnings do not cover everything mentioned in the
+  "limitations documentation":limitations.html.
+* @-V@/@--version@: the Ruby2CExtension version is printed.
+
+
+h2. Include Option
+
+Ruby2CExtension has an experimental feature that allows dependencies of a
+compiled Ruby file to be included in the same extension. This is best
+described with an example. Let's say we have 3 files: @a.rb@, @b.rb@ and
+@c.rb@:
+
+PRE
+# a.rb
+puts "a"
+class A; end
+PREEND
+
+PRE
+# b.rb
+puts "b"
+require "a"
+class B; end
+PREEND
+
+PRE
+# c.rb
+require "a"
+require "b"
+puts "c"
+PREEND
+
+The @require@-include feature is enabled if the @-I@/@--include@ option
+followed by a search path is given (possibly multiple times). The search paths
+can be absolute or relative paths, they are searched for <code>require</code>d
+files. So, if the example is compiled to a C extension with
+
+PRE
+rb2cx -I . c.rb
+PREEND
+
+then the following will happen. For each call to @require@ with no explicit
+receiver and one argument that is a simple string (i.e. no interpolation) the
+search paths are searched for a file that matches the argument to the
+@require@ call (with an algorithm similar to Ruby's). If no matching file is
+found, then the call to @require@ is compiled as usual. But if a matching file
+is found, then that file is read and it is translated to C and instead of
+compiling the original @require@ call that translated C code will be executed,
+unless a @require@ of that file was encountered before.
+
+So in the example we will get one C extension, that contains the code of all
+three files and the <code>require</code>s are performed at the right moment
+and in correct order. The output will be (notice that the <code>require
+"a"</code> in @b.rb@ does not result in a second execution of @a.rb@, as
+expected):
+
+PRE
+a
+b
+c
+PREEND
+
+As stated above, this feature is experimental, it should work well for many
+cases, e.g. for a library that has one main file which is always
+<code>require</code>d by user code, but is split into multiple files for
+maintenance. Such a library could be compiled into a single C extension. But
+it can break for various reasons, e.g. Ruby will not be aware, that the
+included files are already "<code>require</code>d" (so if a file is already
+included in a C extension, but also <code>require</code>d by other normal Ruby
+code, then that file will in effect execute twice).
+
+If the verbose mode is enabled (the @-v@/@--verbose@ option), then each
+inclusion of a file will be logged. This way one can easily check if the
+expected files are actually included.
+
+
+h2. Optimization Options
+
+Ruby2CExtension can use various "optimizations":optimizations.html to improve
+the performance of the resulting C extension. These optimizations are not
+enabled by default, because they can all result in wrong behavior. The
+optimizations are enabled by the @-O@/@--optimization@ option followed by one
+of the following optimization names:
+
+* @const_cache@: enables constant lookup caching
+* @builtin_methods@: enables optimization of calls to built-in methods
+* @inline_methods@: enables inlining of some built-in methods
+* @case_optimize@: enables case optimization
+* @all@: enables all of the above optimizations
+
+
+h2. Examples
+
+PRE
+rb2cx -wv file.rb
+rb2cx -I . -O all file.rb
+rb2cx -I . -I ../libs -O const_cache -O builtin_methods -w file.rb
+PREEND

data/doc/style.css

@@ -0,0 +1,27 @@
+body {
+	color: #333;
+	background-color: white;
+	margin: 0 2em;
+	min-width: 41em;
+}
+h1 {
+	color: black;
+	border-bottom: 2px solid red;
+}
+h2 {
+	color: black;
+}
+h3 {
+	color: black;
+}
+pre {
+	color: black;
+	background-color: #eee;
+	padding: 0.7em 1em;
+	margin: 0.8em 0;
+	border: 1px dotted #999;
+}
+code {
+	color: black;
+	background-color: #eee;
+}

data/lib/concretizer.rb

@@ -0,0 +1,3 @@
+require File.dirname(__FILE__) + "/ruby2cext/compiler" # includes concretizer
+# to use: >> 
+# Ruby2CExtension::Concretize.concretize_all!

data/lib/ruby2cext/c_function.rb

@@ -0,0 +1,617 @@
+
+module Ruby2CExtension
+
+	module CFunction
+		# contains all different Types of C functions that are compiled from ruby nodes
+
+		class Base
+			include CommonNodeComp
+			extend Tools::EnsureNodeTypeMixin
+			attr_reader :scope, :compiler, :closure_tbl
+			attr_accessor :need_res, :need_self, :need_cref, :need_class, :need_wrap
+			def initialize(compiler, scope)
+				@compiler = compiler
+				@scope = scope
+				@closure_tbl = []
+				@scope.closure_tbl = @closure_tbl
+				@lines = []
+				@while_stack = []
+			end
+
+			# some redirects to compiler
+			def un(str); compiler.un(str); end
+			def sym(sym); compiler.sym(sym); end
+			def global_const(str, register_gc = true)
+				compiler.global_const(str, register_gc)
+			end
+			def global_var(str)
+				compiler.global_var(str)
+			end
+			def add_helper(str); compiler.add_helper(str); end
+
+			def get_lines
+				@lines.join("\n")
+			end
+			
+			def l(line) # add_line
+				# ignore lines with only whitespace or only alnum chars (variable name)
+				unless line =~ /\A\s*\z/ || (line =~ /\A(\w*);?\z/ && !(%w[break continue].include? $1))
+					@lines << line
+				end
+			end
+
+			def push_while(redo_lbl, next_lbl, break_lbl)
+				@while_stack << [redo_lbl, next_lbl, break_lbl]
+			end
+			def pop_while
+				raise Ruby2CExtError::Bug, "popped from empty while stack" if @while_stack.empty?
+				@while_stack.pop
+			end
+			def in_while?(lbl_type = nil)
+				return false if @while_stack.empty?
+				case lbl_type
+				when nil
+					true
+				when :redo
+					@while_stack.last[0]
+				when :next
+					@while_stack.last[1]
+				when :break
+					@while_stack.last[2]
+				else
+					false
+				end
+			end
+
+			def break_allowed?(with_value)
+				in_while?(:break)
+			end
+			def comp_break(hash)
+				if (lbl = in_while?(:break))
+					l "while_res = #{comp(hash[:stts])};"
+					l "goto #{lbl};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "break is not supported here"
+				end
+			end
+
+			def next_allowed?
+				in_while?(:next)
+			end
+			def comp_next(hash)
+				if (lbl = in_while?(:next))
+					# hash[:stts] is silently ignored (as ruby does)
+					l "goto #{lbl};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "next is not supported here"
+				end
+			end
+
+			def redo_allowed?
+				in_while?(:redo)
+			end
+			def comp_redo(hash)
+				if (lbl = in_while?(:redo))
+					l "goto #{lbl};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "redo is not supported here"
+				end
+			end
+
+			def return_allowed?
+				false
+			end
+			
+			def comp_return(hash)
+				raise Ruby2CExtError::NotSupported, "return is not supported here"
+			end
+
+			def comp_retry(hash)
+				l "rb_jump_tag(0x4 /* TAG_RETRY */);"
+				"Qnil"
+			end
+
+			def need_closure_ptr
+				false # only needed in Block
+			end
+
+			def assign_res(str)
+				self.need_res = true
+				unless str.strip == "res"
+					l "res = #{str};"
+				end
+			end
+			def get_self
+				self.need_self = true
+				"self"
+			end
+			def get_cref
+				self.need_cref = true
+				get_cref_impl # subclass
+			end
+			def get_class
+				self.need_class = true
+				"s_class"
+			end
+			def get_cbase
+				"(#{get_cref}->nd_clss)"
+			end
+			def get_cvar_cbase
+				# there is always at least one real class in the cref chain
+				add_helper <<-EOC
+					static VALUE cvar_cbase(NODE *cref) {
+						while (FL_TEST(cref->nd_clss, FL_SINGLETON)) { cref = cref->nd_next; }
+						return cref->nd_clss;
+					}
+				EOC
+				"cvar_cbase(#{get_cref})"
+			end
+
+			def get_closure_ary_var
+				"my_closure_ary"
+			end
+
+			def get_wrap_ptr
+				"(&the_wrap)"
+			end
+
+			def add_closure_need(sym)
+				closure_tbl << sym unless closure_tbl.include? sym
+			end
+
+			def closure_buid_c_code
+				if closure_tbl.empty?
+					nil
+				else
+					res = ["#{get_closure_ary_var} = rb_ary_new2(#{closure_tbl.size});"]
+					closure_tbl.each_with_index { |entry, idx|
+						case entry
+						when Integer
+							res << "RARRAY(#{get_closure_ary_var})->ptr[#{idx}] = #{scope.get_dvar_ary(entry)};"
+						when :lvar
+							res << "RARRAY(#{get_closure_ary_var})->ptr[#{idx}] = #{scope.get_lvar_ary};"
+						when :self
+							res << "RARRAY(#{get_closure_ary_var})->ptr[#{idx}] = #{get_self};"
+						when :class
+							res << "RARRAY(#{get_closure_ary_var})->ptr[#{idx}] = #{get_class};"
+						when :cref
+							add_helper <<-EOC
+								static void cref_data_mark(NODE *n) {
+									rb_gc_mark((VALUE)n);
+								}
+							EOC
+							res << "RARRAY(#{get_closure_ary_var})->ptr[#{idx}] = " +
+								"Data_Wrap_Struct(rb_cObject, cref_data_mark, 0, #{get_cref});"
+						else
+							raise Ruby2CExtError::Bug, "unexpected closure_tbl entry: #{entry.inspect}"
+						end
+					}
+					res << "RARRAY(#{get_closure_ary_var})->len = #{closure_tbl.size};"
+					res.join("\n")
+				end
+			end
+
+			def wrap_buid_c_code
+				add_helper <<-EOC
+					struct wrap {
+						VALUE self;
+						VALUE s_class;
+						NODE *cref;
+						VALUE my_closure_ary;
+						VALUE *closure;
+						VALUE *var;
+						long state;
+					};
+				EOC
+				res = []
+				res << "the_wrap.self = #{get_self};" if need_self
+				res << "the_wrap.s_class = #{get_class};" if need_class
+				res << "the_wrap.cref = #{get_cref};" if need_cref
+				res << "the_wrap.my_closure_ary = #{get_closure_ary_var};" unless closure_tbl.empty?
+				res << "the_wrap.closure = closure;" if need_closure_ptr
+				res << "the_wrap.var = #{scope.var_ptr_for_wrap};" if scope.var_ptr_for_wrap
+				res.compact.join("\n")
+			end
+
+			def init_c_code
+				cb_c_code = closure_buid_c_code # must be called before the rest because it might change self or scope
+				res = []
+				res << "VALUE res;" if need_res
+				res << "VALUE s_class = (#{get_cref})->nd_clss;" if need_class
+				res << "VALUE #{get_closure_ary_var};" if cb_c_code
+				res << "struct wrap the_wrap;" if need_wrap
+				res << scope.init_c_code
+				res << cb_c_code if cb_c_code
+				res << wrap_buid_c_code if need_wrap
+				res.compact.join("\n")
+			end
+		end
+
+		class ClassModuleScope < Base
+			def self.compile(outer, scope_node, class_mod_var, is_class)
+				ensure_node_type(scope_node, :scope)
+				vmode_methods = Scopes::Scope::VMODES.dup
+				vmode_methods.delete :module_function if is_class
+				cf = self.new(outer.compiler, Scopes::Scope.new(scope_node.last[:tbl], vmode_methods))
+				cf.instance_eval {
+					block = make_block(scope_node.last[:next])
+					l "return #{comp(block)};"
+				}
+				body = "#{cf.init_c_code}\n#{cf.get_lines}"
+				args = []
+				args << "VALUE self" if cf.need_self
+				args << "NODE *cref" if cf.need_cref
+				sig = "static VALUE FUNNAME(#{args.join(", ")}) {"
+				fname = cf.compiler.add_fun("#{sig}\n#{body}\n}", "class_module_scope")
+				outer.instance_eval {
+					args = []
+					args << class_mod_var if cf.need_self
+					args << "NEW_NODE(NODE_CREF, #{class_mod_var}, 0, #{get_cref})" if cf.need_cref
+					assign_res("#{fname}(#{args.join(", ")})")
+				}
+				"res"
+			end
+
+			def get_cref_impl
+				"cref"
+			end
+
+		end
+
+		class ToplevelScope < ClassModuleScope
+			def self.compile(compiler, scope_node, private_vmode = true)
+				ensure_node_type(scope_node, :scope)
+				cf = self.new(compiler, Scopes::Scope.new(scope_node.last[:tbl], [:public, :private], private_vmode))
+				cf.instance_eval {
+					block = make_block(scope_node.last[:next])
+					l "#{comp(block)};"
+				}
+				body = "#{cf.init_c_code}\n#{cf.get_lines}"
+				sig = "static void FUNNAME(VALUE self, NODE *cref) {"
+				cf.compiler.add_fun("#{sig}\n#{body}\n}", "toplevel_scope") # and return the function name
+			end
+		end
+
+		class Method < Base
+			def self.compile(outer, scope_node, def_fun, class_var, mid)
+				ensure_node_type(scope_node, :scope)
+				cf = self.new(outer.compiler, Scopes::Scope.new(scope_node.last[:tbl], []))
+				cf.instance_eval {
+					block_array = make_block(scope_node.last[:next]).last.dup # dup the block_array to allow modification
+					arg = block_array.shift
+					ba = nil
+					unless block_array.empty? || block_array.first.first != :block_arg
+						ba = block_array.shift
+					end
+					handle_method_args(arg, ba)
+					l "return #{comp([:block, block_array])};"
+				}
+				body = "#{cf.init_c_code}\n#{cf.get_lines}"
+				sig = "static VALUE FUNNAME(int meth_argc, VALUE *meth_argv, VALUE self) {"
+				fname = cf.compiler.add_fun("#{sig}\n#{body}\n}", "method")
+				if cf.need_cref
+					outer.instance_eval {
+						add_helper <<-EOC
+							static void def_only_once(ID mid) {
+								rb_raise(rb_eTypeError, "def for \\"%s\\" can only be used once", rb_id2name(mid));
+							}
+						EOC
+						l "if (#{cf.cref_global_var}) def_only_once(#{sym(mid)});"
+						l "#{cf.cref_global_var} = (VALUE)(#{get_cref});"
+					}
+				end
+				outer.l "#{def_fun}(#{class_var}, #{mid.to_s.to_c_strlit}, #{fname}, -1);"
+				"Qnil"
+			end
+
+			def return_allowed?
+				true
+			end
+			def comp_return(hash)
+				l "return #{comp(hash[:stts])};"
+				"Qnil"
+			end
+
+			def get_cref_impl
+				@cref_global_var ||= global_var("Qfalse")
+				"(RNODE(#{@cref_global_var}))"
+			end
+			attr_reader :cref_global_var
+		end
+
+		class Block < Base
+			def self.compile(outer, block_node, var_node)
+				ensure_node_type(block_node, :block)
+				cf = self.new(outer.compiler, outer.scope.new_dyna_scope)
+				cf.instance_eval {
+					if Array === var_node
+						if var_node.first == :masgn
+							dup_hash = var_node.last.dup
+							c_if("ruby_current_node->nd_state != 1") { # 1 is YIELD_FUNC_AVALUE
+								# do "svalue_to_mrhs"
+								c_if("bl_val == Qundef") {
+									l "bl_val = rb_ary_new2(0);"
+								}
+								c_else {
+									#if dup_hash[:head] # TODO
+										l "VALUE tmp = rb_check_array_type(bl_val);"
+										l "bl_val = (NIL_P(tmp) ? rb_ary_new3(1, bl_val) : tmp);"
+									#else
+									#	l "bl_val = rb_ary_new3(1, bl_val);"
+									#end
+								}
+							}
+							dup_hash[:value] = "bl_val"
+							comp_masgn(dup_hash)
+						else
+							c_if("ruby_current_node->nd_state == 1") { # 1 is YIELD_FUNC_AVALUE
+								# do "avalue_to_svalue"
+								l "if (RARRAY(bl_val)->len == 0) bl_val = Qnil;"
+								l "else if (RARRAY(bl_val)->len == 1) bl_val = RARRAY(bl_val)->ptr[0];"
+							}
+							handle_assign(var_node, "bl_val")
+						end
+					end
+					l "block_redo:"
+					l "return #{comp_block(block_node.last)};"
+				}
+				body = "#{cf.init_c_code(outer)}\n#{cf.get_lines}"
+				sig = "static VALUE FUNNAME(VALUE bl_val, VALUE closure_ary, VALUE bl_self) {"
+				fname = cf.compiler.add_fun("#{sig}\n#{body}\n}", "block")
+				[fname, cf.need_closure_ptr]
+			end
+
+			def init_c_code(outer)
+				cb_c_code = closure_buid_c_code # must be called before the rest because it might change self or scope
+				outer.add_closure_need(:self) if need_self
+				outer.add_closure_need(:class) if need_class
+				outer.add_closure_need(:cref) if need_cref
+				res = []
+				res << "VALUE res;" if need_res
+				if need_closure_ptr
+					res << "VALUE *closure = RARRAY(closure_ary)->ptr;"
+				end
+				res << "VALUE self = (bl_self == Qundef ? closure[#{outer.closure_tbl.index(:self)}] : bl_self);" if need_self
+				res << "VALUE s_class = (bl_self == Qundef ? closure[#{outer.closure_tbl.index(:class)}] : ruby_class);" if need_class
+				if need_cref
+					# see #define Data_Get_Struct
+					res << "NODE *cref = (Check_Type(closure[#{outer.closure_tbl.index(:cref)}]," +
+						" T_DATA), (NODE*)DATA_PTR(closure[#{outer.closure_tbl.index(:cref)}]));"
+				end
+				res << "VALUE #{get_closure_ary_var};" if cb_c_code
+				res << "struct wrap the_wrap;" if need_wrap
+				res << scope.init_c_code
+				res << cb_c_code if cb_c_code
+				res << wrap_buid_c_code if need_wrap
+				res.compact.join("\n")
+			end
+
+			def break_allowed?(with_value)
+				super || !with_value
+			end
+			def comp_break(hash)
+				if in_while?(:break)
+					super
+				else
+					raise Ruby2CExtError::NotSupported, "break with a value is not supported in a block" if hash[:stts]
+					l "rb_iter_break();"
+					"Qnil"
+				end
+			end
+
+			def next_allowed?
+				true
+			end
+			def comp_next(hash)
+				if in_while?(:next)
+					super
+				else
+					l "return #{comp(hash[:stts])};"
+					"Qnil"
+				end
+			end
+
+			def redo_allowed?
+				true
+			end
+			def comp_redo(hash)
+				if in_while?(:redo)
+					super
+				else
+					l "goto block_redo;"
+					"Qnil"
+				end
+			end
+
+			def need_closure_ptr
+				scope.need_closure || need_self || need_class || need_cref
+			end
+
+			def get_cref_impl
+				"cref"
+			end
+		end
+
+		class Wrap < Base
+
+			def self.compile(outer, base_name, cflow_hash = nil)
+				cf = self.new(outer, cflow_hash)
+				cf.instance_eval {
+					if cf.cflow_hash
+						l "#{get_wrap_ptr}->state &= (1<<20)-1;" # set bits 20+ to 0
+					end
+					l "return #{yield cf};"
+				}
+				body = "#{cf.init_c_code}\n#{cf.get_lines}"
+				sig = "static VALUE FUNNAME(struct wrap *wrap_ptr) {"
+				outer.need_wrap = true
+				cf.compiler.add_fun("#{sig}\n#{body}\n}", base_name) # and return the function name
+			end
+
+			attr_reader :base_cfun, :outer_cfun, :cflow_hash
+
+			# the following attr_accessors from Base are redefined, to redirect to base_cfun
+			[:need_self, :need_cref, :need_class, :need_wrap].each { |a|
+				define_method(a) { base_cfun.send(a) }
+				asgn_sym = :"#{a}="
+				define_method(asgn_sym) { |arg| base_cfun.send(asgn_sym, arg) }
+			}
+
+			# cflow_hash can either be nil or a hash, if it is nil then break,
+			# next, redo and return are not possible from the wrap to
+			# outer_cfun.
+			#
+			# If cflow_hash is a hash then those are possible if outer_cfun
+			# allows them. For this to work only the bits 0-19 of
+			# wrap_ptr->state may be modified by code inside this wrap and
+			# after the call the code generated by Wrap.handle_wrap_cflow has
+			# to be inserted.
+			def initialize(outer_cfun, cflow_hash = nil)
+				@outer_cfun = outer_cfun
+				if Wrap === outer_cfun
+					@base_cfun = outer_cfun.base_cfun
+				else
+					@base_cfun = outer_cfun
+				end
+				@cflow_hash = cflow_hash
+				@compiler = base_cfun.compiler
+				@scope = Scopes::WrappedScope.new(base_cfun.scope)
+				@closure_tbl = base_cfun.closure_tbl
+				@lines = []
+				@while_stack = []
+			end
+
+			BREAK_FLAG  = 1 << 20
+			NEXT_FLAG   = 1 << 21
+			REDO_FLAG   = 1 << 22
+			RETURN_FLAG = 1 << 23
+
+			def break_allowed?(with_value)
+				super || (cflow_hash && outer_cfun.break_allowed?(with_value))
+			end
+			def comp_break(hash)
+				if in_while?(:break)
+					super
+				elsif break_allowed?(hash[:stts])
+					@cflow_hash[:break] ||= hash[:stts]
+					l "#{get_wrap_ptr}->state |= #{BREAK_FLAG};"
+					l "return #{comp(hash[:stts])};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "break #{hash[:stts] ? "with a value " : ""}is not supported here"
+				end
+			end
+
+			def next_allowed?
+				super || (cflow_hash && outer_cfun.next_allowed?)
+			end
+			def comp_next(hash)
+				if in_while?(:next)
+					super
+				elsif next_allowed?
+					@cflow_hash[:next] = true
+					l "#{get_wrap_ptr}->state |= #{NEXT_FLAG};"
+					# hash[:stts] might be evaluated unnecessarily (because it
+					# is ignored in while loops), but oh well ...
+					l "return #{comp(hash[:stts])};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "next is not supported here"
+				end
+			end
+
+			def redo_allowed?
+				super || (cflow_hash && outer_cfun.redo_allowed?)
+			end
+			def comp_redo(hash)
+				if in_while?(:redo)
+					super
+				elsif redo_allowed?
+					@cflow_hash[:redo] = true
+					l "#{get_wrap_ptr}->state |= #{REDO_FLAG};"
+					l "return Qnil;"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "redo is not supported here"
+				end
+			end
+
+			def return_allowed?
+				cflow_hash && outer_cfun.return_allowed?
+			end
+			def comp_return(hash)
+				if return_allowed?
+					@cflow_hash[:return] = true
+					l "#{get_wrap_ptr}->state |= #{RETURN_FLAG};"
+					l "return #{comp(hash[:stts])};"
+					"Qnil"
+				else
+					raise Ruby2CExtError::NotSupported, "return is not supported here"
+				end
+			end
+
+			# the result of the call to the wrap cfun is expected in "res"
+			def self.handle_wrap_cflow(cfun, cflow_hash)
+				cfun.instance_eval {
+					if cflow_hash.has_key?(:break)
+						c_if("#{get_wrap_ptr}->state & #{BREAK_FLAG}") {
+							l comp_break(:stts => (cflow_hash[:break] ? "res" : false))
+						}
+					end
+					if cflow_hash.has_key?(:next)
+						c_if("#{get_wrap_ptr}->state & #{NEXT_FLAG}") {
+							l comp_next(:stts => "res")
+						}
+					end
+					if cflow_hash.has_key?(:redo)
+						c_if("#{get_wrap_ptr}->state & #{REDO_FLAG}") {
+							l comp_redo({})
+						}
+					end
+					if cflow_hash.has_key?(:return)
+						c_if("#{get_wrap_ptr}->state & #{RETURN_FLAG}") {
+							l comp_return(:stts => "res")
+						}
+					end
+				}
+			end
+
+			def get_self
+				self.need_self = true
+				"(wrap_ptr->self)"
+			end
+			def get_cref_impl
+				"(wrap_ptr->cref)"
+			end
+			def get_class
+				self.need_class = true
+				"(wrap_ptr->s_class)"
+			end
+
+			def get_closure_ary_var
+				"(wrap_ptr->my_closure_ary)"
+			end
+
+			def get_wrap_ptr
+				"wrap_ptr"
+			end
+
+			[:closure_buid_c_code, :wrap_buid_c_code, :need_closure_ptr].each { |m|
+				define_method(m) {
+					raise Ruby2CExtError::Bug, "the method #{m} may not be called for an instance of Wrap"
+				}
+			}
+
+			def init_c_code
+				res = []
+				res << "VALUE res;" if need_res
+				res.compact.join("\n")
+			end
+		end
+
+	end
+
+end