crystalizer 0.2.2

data/doc/eval2c.txt

@@ -0,0 +1,246 @@
+
+h1. Eval2C
+
+Eval2C is a class that allows the compilation of Ruby code to a C extension at
+runtime. The compiled C extension will be <code>require</code>d automatically
+and the compiled code will be available as a Proc.
+
+It is easy to integrate Eval2C into existing scripts or libraries to improve
+performance. It is also pretty simple to provide fallback code for when
+Ruby2CExtension is not available.
+
+
+h2. Basic Usage
+
+Here is an example:
+
+PRE
+require "ruby2cext/eval2c"
+
+$e2c = Ruby2CExtension::Eval2C.new
+
+$e2c.toplevel_eval("puts 'hello'") # prints hello
+PREEND
+
+First you need to create an Eval2C instance, this instance basically stores
+the options/configuration. In the above example no options were given, so the
+defaults are used. The available options are explained below.
+
+The last line of the example does what the method name suggests: the string of
+Ruby code is evaluated at the toplevel (i.e. not inside any class scope). To
+be more precise, that last line is equivalent to this Ruby code:
+
+PRE
+$some_global_var = proc { puts 'hello' }
+$some_global_var.call
+PREEND
+
+The proc is compiled to a C extension, that C expension is then
+<code>require</code>d and finally the proc is called.
+
+The implementation of @toplevel_eval@ is actually pretty simple:
+
+PRE
+def toplevel_eval(code_str)
+  compile_to_proc(code_str).call
+end
+PREEND
+
+So the main work is done by @compile_to_proc@, which can also be used
+directly:
+
+PRE
+$e2c.compile_to_proc("|a,b| a+b").call(2, 3) # => 5
+PREEND
+
+There are some things to note: The proc won't have access to local variables
+and it will be defined at toplevel, which is important for constant lookup:
+
+PRE
+SOME_CONST = :toplevel
+
+class A
+  SOME_CONST = :A
+  $e2c.compile_to_proc("SOME_CONST").call # => :toplevel, not :A!
+end
+PREEND
+
+Eval2C also offers equivalents to Ruby's @module_eval@/@class_eval@ and
+@instance_eval@:
+
+PRE
+class A
+  $e2c.module_eval(self, %{
+    def initialize(x)
+      @x = x
+    end
+    def foo(y)
+      @x + y
+    end
+  })
+end
+
+A.new(5).foo(6) # => 11
+
+$e2c.instance_eval("4321", "reverse") # => "1234"
+PREEND
+
+Their implementations also use @compile_to_proc@ and they are very similar to
+the implementation of @toplevel_eval@:
+
+PRE
+def module_eval(mod, code_str)
+  mod.module_eval(&compile_to_proc(code_str))
+end
+alias :class_eval :module_eval
+
+def instance_eval(object, code_str)
+  object.instance_eval(&compile_to_proc(code_str))
+end
+PREEND
+
+With @module_eval@/@class_eval@ it is possible to selectively compile some
+performance critical methods to C and leave the rest in Ruby.
+
+But there is one more thing: @compile_methods@.
+
+Defining the methods using @module_eval@ as in the last example, has at least
+one downside: it won't play nice with syntax-highlighting text editors,
+because the code of the methods is actually in a string. It would be much
+nicer to @def@ the methods the normal way and then just compile them
+afterwards.
+
+Thanks to @compile_methods@ this is possible:
+
+PRE
+class A
+  def initialize(x)
+    @x = x
+  end
+  def foo(y)
+    @x + y
+  end
+
+  $e2c.compile_methods(self, :initialize, :foo)
+end
+
+A.new(5).foo(6) # => 11
+PREEND
+
+@compile_methods@ will grab the methods' node trees using RubyNode, compile
+them to C and then replace the old methods with the C versions. The visibility
+of the old methods will be preserved.
+
+There are some limitations: @compile_methods@ only works with methods defined
+using @def@ and the methods must not need a cref (for more informations on
+what crefs are please see the respective section in
+"limitations":limitations.html). This mainly means that constants must be
+prefixed with a double colon: @Array@ needs a cref, while @::Array@ does not.
+
+
+h2. Options
+
+Eval2C is configured with an option hash, the available options are:
+
+* @:path@: this is the path where the compiled extensions are stored, the
+  default is the directory @.ruby2cext@ in the current users home dir.
+* @:prefix@: this is the prefix for the names of the generated extensions, the
+  names of the extensions are generated from this prefix and a SHA1 digest of
+  the code and options. The default is @"eval2c"@ and it is generally not
+  necessary to change it.
+* @:plugins@: this is used to configure the options for the compilation
+  (compare "rb2cx":rb2cx.html) and it is another option hash. The default is
+  <code>{:optimizations => :all}</code>, but all of the following keys are
+  possible:
+** @:warnings@: if set to @true@, then warnings about possible problems will
+   be sent to the logger (see below).
+** @:require_include@: can be set to a single search path or an array of
+   search paths (equivalent to the @-I@ option of @rb2cx@).
+** @:optimizations@: yet another option hash to configure the
+   "optimizations":optimizations.html. The valid keys are @:const_cache@,
+   @:builtin_methods@, @:inline_methods@ and @:case_optimize@, each with
+   @true@ or @false@ as value. To just use all optimizations it is also
+   possible to just use the symbol @:all@ instead of the hash.
+* @:logger@: used for log messages, can be either @nil@ or an instance of
+  @Logger@ (<code>require "logger"</code>). The default is @nil@, which means
+  that no output is generated.
+* @:force_recompile@: if this is set to @true@, then each generated extension
+  is (re)compiled, even if it is already available from a previous run. The
+  default is @false@.
+
+Some examples:
+
+PRE
+Eval2C.new(:path => ".", :plugins => {})
+Eval2C.new(:plugins => {:require_include=>[".", "../lib"], :optimizations=>:all})
+Eval2C.new(:plugins => {:warnings=>true, :optimizations=>{:builtin_methods=>true}})
+Eval2C.new(:prefix => "my_lib_name", :logger => Logger.new(STDOUT))
+PREEND
+
+
+h2. Implementation Details
+
+The @compile_to_proc@ method works as follows:
+
+# The name of the extension is determined from the given prefix combined with
+  the SHA1 digest of the Ruby code and some extra data (Ruby version,
+  Ruby2CExtension version and plugin options). So the code determines the
+  extension name and if the same code is compiled with the same version and
+  settings, the name will be the same. This allows very simple and efficient
+  caching of the compiled extensions.
+# From the extension name a global variable name is constructed. The compiled
+  proc will be stored in that global variable by the generated C extension.
+# If the extension with the generated name already exists, then we are
+  basically done and can skip this step. Otherwise the Ruby proc code is
+  translated to C and compiled to a C extension in the specified path.
+# The generated or already existing C extension is <code>require</code>d.
+# The global variable that now contains the compiled proc is read and the
+  result is returned.
+
+The @compile_methods@ method works similar.
+
+As mentioned above, this scheme allows very efficient caching of the C
+extensions. Basically each extension is only compiled once when the
+program/library is executed for the first time. In later runs the extension
+will already exist and just be <code>require</code>d.
+
+Adding the Ruby version and the Ruby2CExtension version to the digest avoids
+problems if multiple versions of Ruby or Ruby2CExtension are installed and
+used in parallel and it also guarantees that the extensions are automatically
+recompiled if Ruby or Ruby2CExtension are updated.
+
+
+h2. Using Eval2C Only When It Is Available
+
+Having Ruby2CExtension as a hard dependency just for some speedup might not
+be acceptable for many projects, in particular since Ruby2CExtension requires
+that a C compiler is available at runtime.
+
+An alternative that avoids the hard dependency is to use Ruby2CExtension only
+if it is installed anyway and otherwise just execute the Ruby code as usual.
+This works works very well if Eval2C and @compile_methods@ is used, for
+example:
+
+PRE
+begin
+  require "ruby2cext/eval2c"
+  $e2c = Ruby2CExtension::Eval2C.new
+rescue LoadError
+  $e2c = nil
+end
+
+class A
+  def initialize(x)
+    @x = x
+  end
+  def foo(y)
+    @x + y
+  end
+
+  $e2c && $e2c.compile_methods(self, :initialize, :foo)
+end
+PREEND
+
+So if Ruby2CExtension is available, then the methods will be compiled and
+fast, otherwise the Ruby code will just work as usual (but it might be a bit
+slower of course).

data/doc/gen_html.rb

@@ -0,0 +1,26 @@
+require "redcloth"
+
+Dir.chdir(File.dirname(__FILE__))
+
+template = File.read("html_template")
+
+Dir["*.txt"].each { |fn|
+	txt = File.read(fn)
+	title = txt[/^h1. (.+?)$/, 1]
+	txt.gsub!(/^PRE\n/m, "<pre><code>")
+	txt.gsub!(/^PREEND$/, "</code></pre>")
+	cnt = 0
+	sections = []
+	txt.gsub!(/^h2\.\s(.+)$/) { sections << $1; "h2(#section#{cnt+=1}). #$1" }
+	unless sections.empty?
+		sect_list = "\nSections: "
+		cnt = 0
+		sect_list << sections.map { |s| "\"#{s}\":#section#{cnt+=1}" }.join(", ")
+		sect_list << ".\n\n"
+		txt.sub!(/\nh2\(/m, sect_list << "h2(")
+	end
+	html = RedCloth.new(txt).to_html
+	File.open("#{File.basename(fn, ".txt")}.html", "w") { |f|
+		f.puts(template % [title, html])
+	}
+}

data/doc/html_template

@@ -0,0 +1,10 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
+<html>
+<head>
+	<title>%s</title>
+	<link href="style.css" media="all" rel="Stylesheet" type="text/css">
+</head>
+<body>
+%s
+</body>
+</html>

data/doc/index.txt

@@ -0,0 +1,169 @@
+
+h1. Ruby2CExtension
+
+Ruby2CExtension is a Ruby to C extension translator/compiler. It takes any
+Ruby source file, parses it using Ruby's builtin parser and then translates
+the abstract syntax tree into "equivalent" C extension code.
+
+Let's say you have a Ruby file @foo.rb@. To translate it to a C extension and
+then compile it, just run:
+
+PRE
+rb2cx foo.rb
+PREEND
+
+This will produce the files @foo.c@ and @foo.so@ (on Linux). @foo.c@ is the
+generated C extension source code and @foo.so@ is the compiled C extension.
+
+If @foo.rb@ is a library, you can just rename, move or delete @foo.rb@ (only
+if you have a backup, of course) and your Ruby program will just use @foo.so@
+instead.
+
+If @foo.rb@ is a script, then it isn't possible to just run @foo.so@, because
+Ruby does not support running C extensions directly, but you can do this:
+
+PRE
+ruby -r foo.so -e ""
+PREEND
+
+which should produce the same output as
+
+PRE
+ruby -r foo.rb -e ""
+PREEND
+
+
+h2. Why?
+
+Well, like everybody else I wanted a faster Ruby and I also wanted to learn
+about Ruby's internals, so I thought translating Ruby to C might be worth a
+try...
+
+The initial results were not as good as I had hoped, but they weren't bad
+either: without optimizations the generated C extension is practically never
+slower than the Ruby code and I found cases where it is more than twice as
+fast, usually it is somewhere in between. But, starting from version 0.2.0,
+Ruby2CExtension can use "optimizations":optimizations.html to *significantly
+speedup* execution in many cases (*sometimes more than five times faster* than
+normal Ruby).
+
+Of course Ruby2CExtension can also be used as an obfuscator for Ruby code,
+though this was not my main motivation.
+
+
+h2. Requirements
+
+Ruby2CExtension is developed for the Ruby 1.8 versions (only 1.8.4 and later).
+It is currently tested with *Ruby 1.8.4, 1.8.5 and 1.8.6*. Only those versions
+should be used, because Ruby2CExtension depends on Ruby internals that can
+change between Ruby versions. If an untested Ruby version is used, then there
+will be a warning. It might work for later 1.8 versions, but it definitely
+won't work with Ruby 1.9.
+
+Ruby2CExtension requires RubyNode to access Ruby's node trees (the AST).
+RubyNode is available at
+"http://rubynode.rubyforge.org/":http://rubynode.rubyforge.org/.
+
+Ruby2CExtension is pure Ruby code, so it should work on all platforms that
+Ruby supports, but it is currently only tested on Linux. There might be
+problems with the automatic compilation of the generated C code on some
+platforms (in particular on Windows).
+
+
+h2. Download
+
+* "Project page":http://rubyforge.org/projects/ruby2cext/
+* "Download":http://rubyforge.org/frs/?group_id=1799
+
+
+h2. Installation
+
+Just run (as root):
+
+PRE
+gem install ruby2cext
+PREEND
+
+Or if you do not use the gem:
+
+PRE
+ruby setup.rb
+PREEND
+
+This will install Ruby2CExtension to the default locations. Optionally you
+can supply some options to @setup.rb@ to customize the installation (see
+@"ruby setup.rb --help"@).
+
+
+h2. Features
+
+Ruby2CExtension supports a very large subset of Ruby's features:
+
+* all the basics (classes, methods, ...)
+* blocks, closures
+* @instance_eval@, @define_method@, ... (only when the block is given directly)
+* correct constant and class variable lookup
+* @raise@, @rescue@, @retry@, @ensure@
+* ...
+
+Things that (currently) don't work:
+
+* @break@ with a value from inside a block
+* @return@ from inside a block
+* @return@, @break@, @next@, @redo@ inside @ensure@
+* @defined?@ is not implemented for all cases
+* block pass (passing a proc as block to a method) works, but only through a
+  hack and it doesn't work (correctly) for things like @instance_eval@
+
+Some of the above things might be fixed in future versions.
+
+Things that don't work as expected and probably never will:
+
+* methods like @local_variables@, that depend on a Ruby SCOPE
+* interoperability with @eval(str)@ and similar methods that parse a string and
+  eval it; these methods work, but they might not behave as expected (i.e.
+  access to local variables and similar things won't work, see above)
+* @callcc@ works but might behave strangely (in particular local variables
+  might behave wrong)
+* some more things, please see "limitations":limitations.html
+
+Ruby2CExtension will translate and compile Ruby files using one or more of the
+above functionalities without a warning (for some cases warnings can be
+enabled, see "rb2cx":rb2cx.html), so if your Ruby code uses such
+functionality, please verify that the compiled C extension works as expected.
+
+For more details please see "limitations":limitations.html.
+
+
+h2. Usage
+
+There are two "interfaces" to use Ruby2CExtension:
+
+* "rb2cx":rb2cx.html: the command line compiler.
+* "Eval2C":eval2c.html: a class that allows dynamic compilation of Ruby code
+  at runtime.
+
+Please see their respective documentations.
+
+
+h2. Feedback
+
+If you find a bug, think that something doesn't work as it should or have
+other suggestions, then please don't hesitate to "contact
+me":mailto:dbatml@remove_nospam.gmx.de and tell me about it.
+
+I am also interested to know if Ruby2CExtension works under Windows (or other
+non Linux platforms).
+
+
+h2. Thanks
+
+I would like to thank Eric Mahurin for various good ideas for
+"optimizations":optimizations.html and for inspiring "Eval2C":eval2c.html.
+
+
+h2. License
+
+Copyright 2006-2007 "Dominik Bathon":mailto:dbatml@remove_nospam.gmx.de.
+
+Ruby2CExtension is licensed under the same terms as Ruby.