serializable_proc 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.document ADDED
@@ -0,0 +1,5 @@
1
+ README.rdoc
2
+ lib/**/*.rb
3
+ bin/*
4
+ features/**/*.feature
5
+ LICENSE
data/.gitignore ADDED
@@ -0,0 +1,21 @@
1
+ ## MAC OS
2
+ .DS_Store
3
+
4
+ ## TEXTMATE
5
+ *.tmproj
6
+ tmtags
7
+
8
+ ## EMACS
9
+ *~
10
+ \#*
11
+ .\#*
12
+
13
+ ## VIM
14
+ *.swp
15
+
16
+ ## PROJECT::GENERAL
17
+ coverage
18
+ rdoc
19
+ pkg
20
+
21
+ ## PROJECT::SPECIFIC
data/HISTORY.txt ADDED
@@ -0,0 +1,4 @@
1
+ === 0.1.0 (Apr 14, 2010)
2
+
3
+ = 1st gem release !! [#ngty]
4
+
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2009 NgTzeYang
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.rdoc ADDED
@@ -0,0 +1,132 @@
1
+ = serializable_proc
2
+
3
+ As the name suggests, SerializableProc is a proc that can be serialized (marshalled).
4
+ A proc is a closure, which consists of the code block defining it, and binding of
5
+ local variables. SerializableProc's approach to serializability is to extract:
6
+
7
+ 1. the code from the proc (using ParseTree or RubyParser), and
8
+ 2. the local, instance, class & global variables reference within the proc from the
9
+ proc's binding, using deep copy via Marshal.load(Marshal.dump(var))
10
+
11
+ A SerializableProc differs from the vanilla Proc in the following 2 ways:
12
+
13
+ === 1. Isolated variables
14
+
15
+ Upon initializing, all variables (local, instance, class & global) within its context
16
+ are extracted from the proc's binding, and are isolated from changes outside the proc's
17
+ scope, thus, achieving a snapshot effect.
18
+
19
+ require 'rubygems'
20
+ require 'serializable_proc'
21
+
22
+ x, @x, @@x, $x = 'lx', 'ix', 'cx', 'gx'
23
+
24
+ s_proc = SerializableProc.new { [x, @x, @@x, $x].join(', ') }
25
+ v_proc = Proc.new { [x, @x, @@x, $x].join(', ') }
26
+
27
+ x, @x, @@x, $x = 'ly', 'iy', 'cy', 'gy'
28
+
29
+ s_proc.call # >> "lx, ix, cx, gx"
30
+ v_proc.call # >> "ly, iy, cy, gy"
31
+
32
+ === 2. Marshallable
33
+
34
+ No throwing of TypeError when marshalling a SerializableProc:
35
+
36
+ Marshal.load(Marshal.dump(s_proc)).call # >> "lx, ix, cx, gx"
37
+ Marshal.load(Marshal.dump(v_proc)).call # >> TypeError (cannot dump Proc)
38
+
39
+ == Installing It
40
+
41
+ The religiously standard way:
42
+
43
+ $ gem install ParseTree serializable_proc
44
+
45
+ Or on 1.9.* or JRuby:
46
+
47
+ $ gem install ruby_parser serializable_proc
48
+
49
+ By default, SerializableProc attempts to load ParseTree, which supports better
50
+ performance & offers many dynamic goodness. If ParseTree cannot be found,
51
+ SerializableProc falls back to the RubyParser-based which suffers some gotchas due
52
+ to its static analysis nature (see 'Gotchas' section).
53
+
54
+ == Performance
55
+
56
+ SerializableProc relies on ParseTree or RubyParser to do code extraction. While running
57
+ in ParseTree mode, thanks to the goodness of dynamic code analysis, SerializableProc
58
+ performs faster by a magnitude of abt 7.5 times for the same ruby, as illustrated with
59
+ the following benchmark results (obtained from running the specs suite):
60
+
61
+ MRI & implementation user system total real
62
+ 1.8.7p299 (ParseTree) 0.000000 0.000000 1.310000 1.312676
63
+ 1.8.7p299 (RubyParser) 0.000000 0.010000 9.560000 9.706455
64
+ 1.9.1p376 (RubyParser) 0.010000 0.010000 8.240000 8.288799
65
+
66
+ (the above is run on my x86_64-linux):
67
+
68
+ == Gotchas
69
+
70
+ As RubyParser does only static code analysis, quite a bit of regexp matchings are needed
71
+ to get SerializableProc to work in RubyParser mode. However, as our regexp kungfu is not
72
+ perfect (yet), pls take note of the following:
73
+
74
+ === 1. Cannot have multiple initializing code block per line
75
+
76
+ The following initializations throw SerializableProc::CannotAnalyseCodeError:
77
+
78
+ # Multiple SerializableProc.new per line
79
+ SerializableProc.new { x } ; SerializableProc.new { y }
80
+
81
+ # Multiple lambda per line (the same applies to proc & Proc.new)
82
+ x_proc = lambda { x } ; y_proc = lambda { y }
83
+ SerializableProc.new(&x_proc)
84
+
85
+ # Mixed lambda, proc & Proc.new per line
86
+ x_proc = proc { x } ; y_proc = lambda { y }
87
+ SerializableProc.new(&x_proc)
88
+
89
+ === 2. Limited ways to initialize code blocks
90
+
91
+ Code block must be initialized with lambda, proc, Proc.new & SerializableProc.new,
92
+ the following will throw SerializableProc::CannotAnalyseCodeError:
93
+
94
+ def create_serializable_proc(&block)
95
+ SerializableProc.new(&block)
96
+ end
97
+
98
+ create_serializable_proc { x }
99
+
100
+ But the following will work as expected:
101
+
102
+ x_proc = lambda { x }
103
+ create_serializable_proc(&x_proc)
104
+
105
+ == TODO (just brain-dumping)
106
+
107
+ 1. Provide flexibility for user to specify whether global variables should be isolated,
108
+ because globals are globals, it may sometimes be useful to use/manipulate the globals
109
+ within the context that the SerializableProc is called, instead of when it is
110
+ initialized
111
+ 2. The RubyParser-based implementation probably need alot more optimization to catch up
112
+ on ParseTree-based one
113
+ 3. Implementing alternative means of extracting the code block without requiring help
114
+ of ParseTree or RubyParser
115
+ 4. Implement workaround to tackle line-numbering bug in JRuby, which causes the
116
+ RubyParser-based implementation to fail, for more info abt JRuby's line-numbering
117
+ bug, see http://stackoverflow.com/questions/3454838/jruby-line-numbering-problem
118
+
119
+ == Note on Patches/Pull Requests
120
+
121
+ * Fork the project.
122
+ * Make your feature addition or bug fix.
123
+ * Add tests for it. This is important so I don't break it in a
124
+ future version unintentionally.
125
+ * Commit, do not mess with rakefile, version, or history.
126
+ (if you want to have your own version, that is fine but bump version in a commit by
127
+ itself I can ignore when I pull)
128
+ * Send me a pull request. Bonus points for topic branches.
129
+
130
+ == Copyright
131
+
132
+ Copyright (c) 2010 NgTzeYang. See LICENSE for details.
data/Rakefile ADDED
@@ -0,0 +1,136 @@
1
+ require 'rubygems'
2
+ require 'rake'
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |gem|
7
+ # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
8
+ gem.name = "serializable_proc"
9
+ gem.summary = %Q{Proc that can be serialized (as the name suggests)}
10
+ gem.description = %Q{
11
+ Give & take, serializing a ruby proc is possible, though not a perfect one.
12
+ Requires either ParseTree (faster) or RubyParser (& Ruby2Ruby).
13
+ }
14
+ gem.email = "ngty77@gmail.com"
15
+ gem.homepage = "http://github.com/ngty/serializable_proc"
16
+ gem.authors = ["NgTzeYang"]
17
+ gem.add_dependency "ruby2ruby", ">= 1.2.4"
18
+ gem.add_development_dependency "bacon", ">= 0"
19
+ # Plus one of the following groups:
20
+ #
21
+ # 1). ParseTree (better performance + dynamic goodness, but not supported on java & 1.9.*)
22
+ # >> gem.add_dependency "ParseTree", ">= 3.0.5"
23
+ #
24
+ # 2). RubyParser (supported for all)
25
+ # >> gem.add_dependency "ruby_parser", ">= 2.0.4"
26
+
27
+ unless RUBY_PLATFORM =~ /java/i or RUBY_VERSION.include?('1.9.')
28
+ gem.post_install_message = %Q{
29
+ /////////////////////////////////////////////////////////////////////////////////
30
+
31
+ ** SerializableProc **
32
+
33
+ You are installing SerializableProc on a ruby platform & version that supports
34
+ ParseTree. With ParseTree, u can enjoy better performance of SerializableProc,
35
+ as well as other dynamic code analysis goodness, as compared to the default
36
+ implementation using RubyParser's less flexible static code analysis.
37
+
38
+ Anyway, u have been informed, SerializableProc will fallback on its default
39
+ implementation using RubyParser.
40
+
41
+ /////////////////////////////////////////////////////////////////////////////////
42
+ }
43
+ end
44
+ end
45
+
46
+ Jeweler::GemcutterTasks.new
47
+ rescue LoadError
48
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
49
+ end
50
+
51
+ require 'rake/testtask'
52
+ Rake::TestTask.new(:spec) do |spec|
53
+ spec.libs << 'lib' << 'spec'
54
+ spec.pattern = 'spec/**/*_spec.rb'
55
+ spec.verbose = true
56
+ end
57
+
58
+ begin
59
+ require 'rcov/rcovtask'
60
+ Rcov::RcovTask.new do |spec|
61
+ spec.libs << 'spec'
62
+ spec.pattern = 'spec/**/*_spec.rb'
63
+ spec.verbose = true
64
+ end
65
+ rescue LoadError
66
+ task :rcov do
67
+ abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
68
+ end
69
+ end
70
+
71
+ task :spec => :check_dependencies
72
+
73
+ begin
74
+ require 'reek/adapters/rake_task'
75
+ Reek::RakeTask.new do |t|
76
+ t.fail_on_error = true
77
+ t.verbose = false
78
+ t.source_files = 'lib/**/*.rb'
79
+ end
80
+ rescue LoadError
81
+ task :reek do
82
+ abort "Reek is not available. In order to run reek, you must: sudo gem install reek"
83
+ end
84
+ end
85
+
86
+ begin
87
+ require 'roodi'
88
+ require 'roodi_task'
89
+ RoodiTask.new do |t|
90
+ t.verbose = false
91
+ end
92
+ rescue LoadError
93
+ task :roodi do
94
+ abort "Roodi is not available. In order to run roodi, you must: sudo gem install roodi"
95
+ end
96
+ end
97
+
98
+ task :default => :spec
99
+
100
+ require 'rake/rdoctask'
101
+ Rake::RDocTask.new do |rdoc|
102
+ version = File.exist?('VERSION') ? File.read('VERSION') : ""
103
+
104
+ rdoc.rdoc_dir = 'rdoc'
105
+ rdoc.title = "serializable_proc #{version}"
106
+ rdoc.rdoc_files.include('README*')
107
+ rdoc.rdoc_files.include('lib/**/*.rb')
108
+ end
109
+
110
+ # Benchmarking
111
+ task :benchmark, :task, :times do |t, args|
112
+ times, task = (args.times || 5).to_i.method(:times), args.task
113
+ title = " ~ Benchmark Results for Task :#{task} ~ "
114
+ results = [%w{nth}, %w{user}, %w{system}, %w{total}, %w{real}]
115
+
116
+ # Running benchmarking & collecting results
117
+ require 'benchmark'
118
+ times.call do |i|
119
+ result = Benchmark.measure{ Rake::Task[task].execute }.to_s
120
+ user, system, total, real =
121
+ result.match(/^\s*(\d+\.\d+)\s+(\d+\.\d+)\s+(\d+\.\d+)\s+\(\s*(\d+\.\d+)\)$/)[1..-1]
122
+ ["##{i.succ}", user, system, total, real].each_with_index{|val, j| results[j] << val }
123
+ end
124
+
125
+ # Formatting benchmarking results
126
+ formatted_results = results.map do |rs|
127
+ width = rs.map(&:to_s).map(&:size).max
128
+ rs.map{|r| ' ' + r.ljust(width, ' ') }
129
+ end.transpose.map{|row| row.join }
130
+
131
+ # Showdown .. printout
132
+ line = '=' * ([title.size, formatted_results.map(&:size).max].max + 2)
133
+ puts [line, title, formatted_results.join("\n"), line].join("\n\n")
134
+
135
+ end
136
+
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.1.0
@@ -0,0 +1,53 @@
1
+ class SerializableProc
2
+
3
+ class CannotSerializeVariableError < Exception ; end
4
+
5
+ class Binding
6
+
7
+ include Marshalable
8
+ marshal_attr :vars
9
+
10
+ def initialize(binding, sexp)
11
+ @vars, sexp_str = {}, sexp.inspect
12
+ while m = sexp_str.match(/^(.*?s\(:(?:l|g|c|i)var, :([^\)]+)\))/)
13
+ ignore, var = m[1..2]
14
+ sexp_str.sub!(ignore,'')
15
+ begin
16
+ val = binding.eval(var) rescue nil
17
+ @vars.update(Sandboxer.fvar(var) => mclone(val))
18
+ rescue TypeError
19
+ raise CannotSerializeVariableError.new("Variable #{var} cannot be serialized !!")
20
+ end
21
+ end
22
+ end
23
+
24
+ def eval!
25
+ @binding ||= (
26
+ set_vars = @vars.map{|(k,v)| "#{k} = Marshal.load(%|#{mdump(v)}|)" } * '; '
27
+ (binding = Kernel.binding).eval(set_vars)
28
+ binding.extend(Extensions)
29
+ )
30
+ end
31
+
32
+ module Extensions
33
+ def self.extended(base)
34
+ class << base
35
+
36
+ alias_method :orig_eval, :eval
37
+
38
+ def eval(str)
39
+ begin
40
+ @fvar = Sandboxer.fvar(str).to_s
41
+ orig_eval(@fvar)
42
+ rescue NameError => e
43
+ msg = e.message.sub(@fvar, str)
44
+ raise NameError.new(msg)
45
+ end
46
+ end
47
+
48
+ end
49
+ end
50
+ end
51
+
52
+ end
53
+ end
@@ -0,0 +1,51 @@
1
+ class SerializableProc
2
+ module Marshalable
3
+
4
+ def self.included(base)
5
+ base.class_eval do
6
+ extend ClassMethods
7
+ include InstanceMethods
8
+ end
9
+ end
10
+
11
+ module ClassMethods
12
+
13
+ protected
14
+
15
+ def marshal_attrs(*attrs)
16
+ attrs = attrs.map{|attr| :"@#{attr}" }
17
+ self.class_eval do
18
+ define_method(:marshalable_attrs) { attrs }
19
+ end
20
+ end
21
+
22
+ alias_method :marshal_attr, :marshal_attrs
23
+
24
+ end
25
+
26
+ module InstanceMethods
27
+
28
+ def marshal_dump
29
+ marshalable_attrs.map{|attr| instance_variable_get(attr) }
30
+ end
31
+
32
+ def marshal_load(data)
33
+ [data].flatten.each_with_index do |val, i|
34
+ instance_variable_set(marshalable_attrs[i], val)
35
+ end
36
+ end
37
+
38
+ protected
39
+
40
+ def mdump(val)
41
+ Marshal.dump(val).gsub('|','\|')
42
+ end
43
+
44
+ def mclone(val)
45
+ Marshal.load(mdump(val))
46
+ end
47
+
48
+ end
49
+
50
+ end
51
+ end
@@ -0,0 +1,16 @@
1
+ class SerializableProc
2
+ module Parsers
3
+ module PT
4
+ class << self
5
+ def process(block)
6
+ if Object.const_defined?(:ParseTree)
7
+ sexp = block.to_sexp
8
+ runnable_code = RUBY_2_RUBY.process(Sandboxer.fsexp(sexp))
9
+ extracted_code = RUBY_2_RUBY.process(eval(sexp.inspect))
10
+ [{:runnable => runnable_code, :extracted => extracted_code}, sexp]
11
+ end
12
+ end
13
+ end
14
+ end
15
+ end
16
+ end
@@ -0,0 +1,93 @@
1
+ class SerializableProc
2
+
3
+ class CannotAnalyseCodeError < Exception ; end
4
+
5
+ module Parsers
6
+ module RP
7
+ class << self
8
+
9
+ def process(klass, file, line)
10
+ const_set(:RUBY_PARSER, RubyParser.new) unless const_defined?(:RUBY_PARSER)
11
+ @klass, @file, @line = klass, file, line
12
+ extract_code_and_sexp
13
+ end
14
+
15
+ private
16
+
17
+ def extract_code_and_sexp
18
+ sexp_str, remaining, type, marker = extract_sexp_args
19
+ while frag = remaining[/^([^\)]*\))/,1]
20
+ begin
21
+ sexp = eval(sexp_str += frag) # this throws SyntaxError if sexp is invalid
22
+ runnable_code, extracted_code = [
23
+ RUBY_2_RUBY.process(Sandboxer.fsexp(sexp)),
24
+ RUBY_2_RUBY.process(eval(sexp.inspect))
25
+ ].map do |code|
26
+ unescape_magic_vars(code).sub(/#{marker}\s*;?/m,'').sub(type,'lambda')
27
+ end
28
+ return [{:runnable => runnable_code, :extracted => extracted_code}, sexp]
29
+ rescue SyntaxError
30
+ remaining.sub!(frag,'')
31
+ end
32
+ end
33
+ end
34
+
35
+ def extract_sexp_args
36
+ raw, type, marker = raw_sexp_and_marker
37
+ rq = lambda{|s| Regexp.quote(s) }
38
+ regexp = Regexp.new([
39
+ '^(.*(', (
40
+ case type
41
+ when /(#{@klass}|Proc)/ then rq["s(:iter, s(:call, s(:const, :#{$1}), :new, s(:arglist)),"]
42
+ else rq['s(:iter, s(:call, nil, :'] + '(?:proc|lambda)' + rq[', s(:arglist']
43
+ end
44
+ ),
45
+ '.*?',
46
+ rq["s(:call, nil, :#{marker}, s(:arglist))"],
47
+ '))(.*)$'
48
+ ].join, Regexp::MULTILINE)
49
+ [raw.match(regexp)[2..3], type, marker].flatten
50
+ end
51
+
52
+
53
+ def raw_sexp_and_marker
54
+ %W{#{@klass}\.new lambda|proc|Proc\.new}.each do |declarative|
55
+ regexp = /^(.*?(#{declarative})\s*(do|\{)\s*(\|([^\|]*)\|\s*)?)/
56
+ raw = raw_code
57
+ lines1, lines2 = [(0 .. (@line - 2)), (@line.pred .. -1)].map{|r| raw[r] }
58
+ match, type = lines2[0].match(regexp)[1..2] rescue next
59
+
60
+ if lines2[0] =~ /^(.*?\W)?(#{declarative})(\W.*?\W(#{declarative}))+(\W.*)?$/
61
+ msg = "Static code analysis can only handle single occurrence of '%s' per line !!" %
62
+ declarative.split('|').join("'/'")
63
+ raise CannotAnalyseCodeError.new(msg)
64
+ elsif lines2[0] =~ /^(.*?\W)?(#{declarative})(\W.*)?$/
65
+ marker = "__serializable_proc_marker_#{@line}__"
66
+ lines = lines1.join + escape_magic_vars(lines2[0].sub(match, match+marker+';') + lines2[1..-1].join)
67
+ return [RUBY_PARSER.parse(lines).inspect, type, marker]
68
+ end
69
+ end
70
+ raise CannotAnalyseCodeError.new('Cannot find specified initializer !!')
71
+ end
72
+
73
+ def escape_magic_vars(s)
74
+ %w{__FILE__ __LINE__}.inject(s) do |s, var|
75
+ s.gsub(var, "__serializable_proc_#{var.downcase}__")
76
+ end
77
+ end
78
+
79
+ def unescape_magic_vars(s)
80
+ %w{__FILE__ __LINE__}.inject(s) do |s, var|
81
+ s.gsub("__serializable_proc_#{var.downcase}__", var)
82
+ end
83
+ end
84
+
85
+ def raw_code
86
+ File.readlines(@file)
87
+ end
88
+
89
+ end
90
+ end
91
+ end
92
+ end
93
+
@@ -0,0 +1,8 @@
1
+ class SerializableProc
2
+ module Parsers
3
+ RUBY_2_RUBY = Ruby2Ruby.new
4
+ end
5
+ end
6
+
7
+ require 'serializable_proc/parsers/pt'
8
+ require 'serializable_proc/parsers/rp'
@@ -0,0 +1,24 @@
1
+ class SerializableProc
2
+ module Sandboxer
3
+ class << self
4
+
5
+ def fsexp(sexp)
6
+ n_sexp, t_sexp = nil, sexp.inspect
7
+ while m = t_sexp.match(/^(.*?s\(:)((i|l|c|g)(asgn|var|vdecl))(,\ :)((|@|@@|\$)([\w]+))(\)|,)/)
8
+ orig, prepend, _, type, declare, join, _, _, name, append = m[0..-1]
9
+ declare.sub!('vdecl','asgn')
10
+ n_sexp = "#{n_sexp}#{prepend}l#{declare}#{join}#{type}var_#{name}#{append}"
11
+ t_sexp.sub!(orig,'')
12
+ end
13
+ eval(n_sexp ? "#{n_sexp}#{t_sexp}" : sexp.inspect)
14
+ end
15
+
16
+ def fvar(var)
17
+ @translate_var_maps ||= {'@' => 'ivar_', '@@' => 'cvar_', '$' => 'gvar_', '' => 'lvar_'}
18
+ m = var.to_s.match(/^(|@|@@|\$)(\w+)$/)
19
+ var.to_s.sub(m[1], @translate_var_maps[m[1]]).to_sym
20
+ end
21
+
22
+ end
23
+ end
24
+ end