oleganza-emrpc 0.3
Sign up to get free protection for your applications and to get access to all the features.
- data/MIT-LICENSE +20 -0
- data/README +129 -0
- data/Rakefile +156 -0
- data/TODO +47 -0
- data/bin/emrpc +4 -0
- data/lib/emrpc.rb +15 -0
- data/lib/emrpc/archive/reference_savior.rb +48 -0
- data/lib/emrpc/archive/ring.rb +44 -0
- data/lib/emrpc/blocking_api.rb +3 -0
- data/lib/emrpc/blocking_api/method_proxy.rb +57 -0
- data/lib/emrpc/blocking_api/multithreaded_client.rb +52 -0
- data/lib/emrpc/blocking_api/singlethreaded_client.rb +68 -0
- data/lib/emrpc/client.rb +28 -0
- data/lib/emrpc/console.rb +32 -0
- data/lib/emrpc/evented_api.rb +14 -0
- data/lib/emrpc/evented_api/connection_mixin.rb +14 -0
- data/lib/emrpc/evented_api/debug_connection.rb +52 -0
- data/lib/emrpc/evented_api/debug_pid_callbacks.rb +39 -0
- data/lib/emrpc/evented_api/default_callbacks.rb +40 -0
- data/lib/emrpc/evented_api/evented_wrapper.rb +28 -0
- data/lib/emrpc/evented_api/local_connection.rb +48 -0
- data/lib/emrpc/evented_api/pid.rb +198 -0
- data/lib/emrpc/evented_api/protocol_mapper.rb +57 -0
- data/lib/emrpc/evented_api/reconnecting_pid.rb +105 -0
- data/lib/emrpc/evented_api/remote_connection.rb +73 -0
- data/lib/emrpc/evented_api/remote_pid.rb +38 -0
- data/lib/emrpc/evented_api/subscribable.rb +56 -0
- data/lib/emrpc/evented_api/timer.rb +23 -0
- data/lib/emrpc/protocols.rb +2 -0
- data/lib/emrpc/protocols/fast_message_protocol.rb +99 -0
- data/lib/emrpc/protocols/marshal_protocol.rb +33 -0
- data/lib/emrpc/server.rb +17 -0
- data/lib/emrpc/util.rb +7 -0
- data/lib/emrpc/util/blank_slate.rb +25 -0
- data/lib/emrpc/util/codec.rb +114 -0
- data/lib/emrpc/util/combine_modules.rb +11 -0
- data/lib/emrpc/util/em2rev.rb +48 -0
- data/lib/emrpc/util/em_start_stop_timeouts.rb +62 -0
- data/lib/emrpc/util/parsed_uri.rb +15 -0
- data/lib/emrpc/util/safe_run.rb +23 -0
- data/lib/emrpc/util/timers.rb +17 -0
- data/lib/emrpc/version.rb +3 -0
- data/spec/blocking_api/method_proxy_spec.rb +33 -0
- data/spec/blocking_api/multithreaded_client_spec.rb +52 -0
- data/spec/blocking_api/scenario_spec.rb +35 -0
- data/spec/blocking_api/singlethreaded_client_spec.rb +63 -0
- data/spec/blocking_api/spec_helper.rb +1 -0
- data/spec/blocking_api_test.rb +98 -0
- data/spec/evented_api/connection_mixin_spec.rb +34 -0
- data/spec/evented_api/default_callbacks_spec.rb +26 -0
- data/spec/evented_api/evented_wrapper_spec.rb +50 -0
- data/spec/evented_api/pid_spec.rb +194 -0
- data/spec/evented_api/reconnecting_pid_spec.rb +76 -0
- data/spec/evented_api/remote_connection_spec.rb +147 -0
- data/spec/evented_api/remote_pid_spec.rb +84 -0
- data/spec/evented_api/scenario_spec.rb +138 -0
- data/spec/evented_api/spec_helper.rb +10 -0
- data/spec/evented_api/subscribable_spec.rb +53 -0
- data/spec/server_spec.rb +7 -0
- data/spec/spec_helper.rb +96 -0
- data/spec/util/blank_slate_spec.rb +7 -0
- data/spec/util/codec_spec.rb +183 -0
- data/spec/util/fast_message_protocol_spec.rb +60 -0
- data/spec/util/marshal_protocol_spec.rb +50 -0
- data/spec/util/parsed_uri_spec.rb +19 -0
- data/spec/util/spec_helper.rb +1 -0
- metadata +164 -0
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008 Oleg Andreev <oleganza@gmail.com>
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
EMRPC is a EventMachine-based remote procedure call library.
|
2
|
+
It looks like DRb, but is much more efficient and provides
|
3
|
+
asynchronous erlang-like interface along with blocking synchronous interface.
|
4
|
+
|
5
|
+
Author: Oleg Andreev <oleganza@gmail.com>
|
6
|
+
|
7
|
+
|
8
|
+
FEATURES
|
9
|
+
|
10
|
+
0. Object-oriented evented API based on lightweight processes called Pids.
|
11
|
+
1. Automatically reconnecting clients (blocking and evented).
|
12
|
+
2. Support for both TCP and Unix sockets. (Use "emrpc://host:port" or "unix:///path/to/my.sock")
|
13
|
+
3. Modularity and good specs coverage: EMRPC is easy to learn, extend and optimize.
|
14
|
+
|
15
|
+
|
16
|
+
VERSION AND STATUS
|
17
|
+
|
18
|
+
This is v0.3 alpha. It is being used in several projects of my own, but is not tested on many interesting cases.
|
19
|
+
It has also several issues with performance and specs stability (see TODO file).
|
20
|
+
|
21
|
+
|
22
|
+
THANKS TO
|
23
|
+
|
24
|
+
* linkfeed.ru for the real-world EMRPC-based application.
|
25
|
+
* pierlis.com for the table/chair/wifi in Paris.
|
26
|
+
|
27
|
+
|
28
|
+
EXAMPLES
|
29
|
+
|
30
|
+
You may try out examples using bin/emrpc IRB session.
|
31
|
+
Just open the console and paste the code.
|
32
|
+
|
33
|
+
-------------------------------------------------------------------------------
|
34
|
+
|
35
|
+
HELLO WORLD (BLOCKING API)
|
36
|
+
|
37
|
+
class HelloWorld
|
38
|
+
def action
|
39
|
+
"Hello!"
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
server = EMRPC::Server.new(:address => 'emrpc://localhost:4000/',
|
44
|
+
:backend => HelloWorld.new)
|
45
|
+
EM::safe_run(:background) { }
|
46
|
+
|
47
|
+
server.start
|
48
|
+
|
49
|
+
client = EMRPC::Client.new('emrpc://localhost:4000/')
|
50
|
+
client.action == "Hello!" #=> true
|
51
|
+
|
52
|
+
|
53
|
+
-------------------------------------------------------------------------------
|
54
|
+
|
55
|
+
HELLO WORLD (EVENTED API)
|
56
|
+
|
57
|
+
class HelloWorld
|
58
|
+
include Pid
|
59
|
+
def action(sender)
|
60
|
+
puts "HelloWorld replies to the sender #{sender}..."
|
61
|
+
sender.reply(self, "Hello!")
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
class User
|
66
|
+
include Pid
|
67
|
+
def connected(pid)
|
68
|
+
puts "Pid #{pid} connected with the user."
|
69
|
+
pid.action(self)
|
70
|
+
end
|
71
|
+
def reply(pid, msg)
|
72
|
+
puts "Pid #{pid} replied: #{msg}"
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
76
|
+
EM::run do
|
77
|
+
hw = HelloWorld.new
|
78
|
+
hw.bind('emrpc://localhost:4000/') # bind a pid to the address
|
79
|
+
|
80
|
+
oleg = User.new
|
81
|
+
oleg.connect('emrpc://localhost:4000/') # connect to that address
|
82
|
+
end
|
83
|
+
|
84
|
+
# Output:
|
85
|
+
Pid #<EMRPC::RemotePid:0x143b740> connected with the user.
|
86
|
+
HelloWorld replies to the sender #<User:0x143b4d4>...
|
87
|
+
Pid #<HelloWorld:0x143d540> replied: Hello!
|
88
|
+
|
89
|
+
-------------------------------------------------------------------------------
|
90
|
+
|
91
|
+
HELLO WORLD (EVENTED WRAPPER, MIXED API)
|
92
|
+
|
93
|
+
# in first process:
|
94
|
+
class HelloWorld
|
95
|
+
def action
|
96
|
+
"Hello!"
|
97
|
+
end
|
98
|
+
end
|
99
|
+
|
100
|
+
server = EMRPC::Server.new(:address => 'emrpc://localhost:4000/',
|
101
|
+
:backend => HelloWorld.new)
|
102
|
+
|
103
|
+
EM::run do
|
104
|
+
server.start
|
105
|
+
end
|
106
|
+
|
107
|
+
# in the other process (actually, this works in the same process with the server too):
|
108
|
+
class User
|
109
|
+
include Pid
|
110
|
+
def connected(pid)
|
111
|
+
puts "Pid #{pid} connected to the user."
|
112
|
+
pid.send(self, :action)
|
113
|
+
end
|
114
|
+
def on_return(pid, msg)
|
115
|
+
puts "Pid #{pid} replied: #{msg}"
|
116
|
+
end
|
117
|
+
end
|
118
|
+
|
119
|
+
EM::run do
|
120
|
+
oleg = User.new
|
121
|
+
oleg.connect('emrpc://localhost:4000/') # connect to that address
|
122
|
+
end
|
123
|
+
|
124
|
+
# Output:
|
125
|
+
Pid #<EMRPC::RemotePid:0x143b740> connected to the user.
|
126
|
+
Pid #<HelloWorld:0x143d540> replied: Hello!
|
127
|
+
|
128
|
+
-------------------------------------------------------------------------------
|
129
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1,156 @@
|
|
1
|
+
require "rake"
|
2
|
+
require "rake/clean"
|
3
|
+
require "rake/gempackagetask"
|
4
|
+
require "rake/rdoctask"
|
5
|
+
require "rake/testtask"
|
6
|
+
require "spec/rake/spectask"
|
7
|
+
require "fileutils"
|
8
|
+
|
9
|
+
|
10
|
+
##############################################################################
|
11
|
+
# Automated tests
|
12
|
+
##############################################################################
|
13
|
+
|
14
|
+
desc "Run specs"
|
15
|
+
task :spec => :'specs:spec'
|
16
|
+
task :specs => :'specs:spec'
|
17
|
+
|
18
|
+
namespace :specs do
|
19
|
+
|
20
|
+
def spec_path
|
21
|
+
path = ENV['SPECS_PATH'] || "spec"
|
22
|
+
puts "Using #{path.inspect} path. (See SPECS_PATH environment variable.)"
|
23
|
+
path
|
24
|
+
end
|
25
|
+
|
26
|
+
desc "Run specs"
|
27
|
+
task :spec do
|
28
|
+
system("spec #{spec_path} -c")
|
29
|
+
end
|
30
|
+
|
31
|
+
desc "Runs specs set by SPECS_PATH (default is 'spec') in a loop detecting random errors"
|
32
|
+
task :loop do
|
33
|
+
def run_spec_iteration(path = "spec", counter = 0)
|
34
|
+
r = `spec #{path}`
|
35
|
+
if r =~ /(\A|[.PFE])[FE]/ || r =~ /[FE]([.PFE]|\z)/
|
36
|
+
puts r
|
37
|
+
counter + 1
|
38
|
+
else
|
39
|
+
counter
|
40
|
+
end
|
41
|
+
end
|
42
|
+
path = spec_path
|
43
|
+
iters = (ENV['SPECS_ITERS'] || 10_000).to_i
|
44
|
+
puts "#{iters} iterations. (See SPECS_ITERS environment variable.)"
|
45
|
+
fs = 0
|
46
|
+
iters.times do |i|
|
47
|
+
puts "Iterations: #{i} Failures: #{fs}"
|
48
|
+
fs = run_spec_iteration(path, fs)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
|
54
|
+
##############################################################################
|
55
|
+
# Packaging & Installation
|
56
|
+
##############################################################################
|
57
|
+
|
58
|
+
def __DIR__
|
59
|
+
File.dirname(__FILE__)
|
60
|
+
end
|
61
|
+
|
62
|
+
include FileUtils
|
63
|
+
|
64
|
+
require "lib/emrpc/version"
|
65
|
+
|
66
|
+
def sudo
|
67
|
+
ENV['EMRPC_SUDO'] ||= "sudo"
|
68
|
+
sudo = windows? ? "" : ENV['EMRPC_SUDO']
|
69
|
+
end
|
70
|
+
|
71
|
+
def windows?
|
72
|
+
(PLATFORM =~ /win32|cygwin/) rescue nil
|
73
|
+
end
|
74
|
+
|
75
|
+
def install_home
|
76
|
+
ENV['GEM_HOME'] ? "-i #{ENV['GEM_HOME']}" : ""
|
77
|
+
end
|
78
|
+
|
79
|
+
CLEAN.include ["**/.*.sw?", "pkg", "lib/*.bundle", "*.gem", "doc/rdoc", ".config", "coverage", "cache"]
|
80
|
+
|
81
|
+
desc "Run the specs."
|
82
|
+
task :default => :specs
|
83
|
+
|
84
|
+
task :emrpc => [:clean, :rdoc, :package]
|
85
|
+
|
86
|
+
RUBY_FORGE_PROJECT = "emrpc"
|
87
|
+
PROJECT_URL = "http://oleganza.com/"
|
88
|
+
PROJECT_SUMMARY = "Efficient RPC library with evented and blocking APIs. In all ways better than DRb."
|
89
|
+
PROJECT_DESCRIPTION = PROJECT_SUMMARY
|
90
|
+
|
91
|
+
AUTHOR = "Oleg Andreev"
|
92
|
+
EMAIL = "oleganza@gmail.com"
|
93
|
+
|
94
|
+
GEM_NAME = "emrpc"
|
95
|
+
PKG_BUILD = ENV['PKG_BUILD'] ? '.' + ENV['PKG_BUILD'] : ''
|
96
|
+
GEM_VERSION = EMRPC::VERSION + PKG_BUILD
|
97
|
+
|
98
|
+
RELEASE_NAME = "REL #{GEM_VERSION}"
|
99
|
+
|
100
|
+
require "extlib/tasks/release"
|
101
|
+
|
102
|
+
spec = Gem::Specification.new do |s|
|
103
|
+
s.name = GEM_NAME
|
104
|
+
s.version = GEM_VERSION
|
105
|
+
s.platform = Gem::Platform::RUBY
|
106
|
+
s.author = AUTHOR
|
107
|
+
s.email = EMAIL
|
108
|
+
s.homepage = PROJECT_URL
|
109
|
+
s.summary = PROJECT_SUMMARY
|
110
|
+
s.bindir = "bin"
|
111
|
+
s.description = s.summary
|
112
|
+
s.executables = %w( emrpc )
|
113
|
+
s.require_path = "lib"
|
114
|
+
s.files = %w( README Rakefile TODO MIT-LICENSE ) + Dir["{docs,bin,spec,lib,examples,script}/**/*"]
|
115
|
+
|
116
|
+
# rdoc
|
117
|
+
s.has_rdoc = true
|
118
|
+
s.extra_rdoc_files = %w( README TODO MIT-LICENSE )
|
119
|
+
#s.rdoc_options += RDOC_OPTS + ["--exclude", "^(app|uploads)"]
|
120
|
+
|
121
|
+
# Dependencies
|
122
|
+
s.add_dependency "eventmachine"
|
123
|
+
s.add_dependency "rake"
|
124
|
+
s.add_dependency "rspec"
|
125
|
+
|
126
|
+
# See sources on github.com/oleganza
|
127
|
+
s.add_dependency "gem_console"
|
128
|
+
|
129
|
+
# Requirements
|
130
|
+
s.requirements << "You need to install the json (or json_pure), yaml, rack gems to use related features."
|
131
|
+
s.required_ruby_version = ">= 1.8.4"
|
132
|
+
end
|
133
|
+
|
134
|
+
Rake::GemPackageTask.new(spec) do |package|
|
135
|
+
package.gem_spec = spec
|
136
|
+
end
|
137
|
+
|
138
|
+
desc "Generate *.gemspec file"
|
139
|
+
task :gemspec do
|
140
|
+
gemspec = "spec = " + spec.to_ruby
|
141
|
+
path = File.join(File.expand_path(File.dirname(__FILE__)), "emrpc.gemspec")
|
142
|
+
puts %{Writing "#{path}"}
|
143
|
+
File.open(path, "w") do |f|
|
144
|
+
f.write(gemspec)
|
145
|
+
end
|
146
|
+
end
|
147
|
+
|
148
|
+
desc "Run :package and install the resulting .gem"
|
149
|
+
task :install => :package do
|
150
|
+
sh %{#{sudo} gem install #{install_home} --local pkg/#{GEM_NAME}-#{GEM_VERSION}.gem --no-rdoc --no-ri}
|
151
|
+
end
|
152
|
+
|
153
|
+
desc "Run :clean and uninstall the .gem"
|
154
|
+
task :uninstall => :clean do
|
155
|
+
sh %{#{sudo} gem uninstall #{GEM_NAME}}
|
156
|
+
end
|
data/TODO
ADDED
@@ -0,0 +1,47 @@
|
|
1
|
+
|
2
|
+
IMPROVEMENTS
|
3
|
+
|
4
|
+
* Rev:
|
5
|
+
* Rev::Server#on_connection: on_connect is called before @block.call:
|
6
|
+
this is not how EM behaves. The order is crucial because we would like to
|
7
|
+
pass some configuration with @block and use it inside on_connect/connection_completed callback.
|
8
|
+
TODO: suggest a patch to Toni Arciery.
|
9
|
+
|
10
|
+
* lib/rev/loop.rb:60 typo: EVFLAG_NOENV should be used instead of EVFLAG_NOEV
|
11
|
+
TODO: suggest a patch to Toni Arciery.
|
12
|
+
|
13
|
+
* When running $ spec spec/ servers sometimes are not unbinded and some specs fail.
|
14
|
+
* fixed issue with reconnecting pid.
|
15
|
+
* MultithreadedClient spec fails randomly with 0 timed out threads:
|
16
|
+
'EMRPC::MultithreadedClient with PoolTimeout should raise ThreadTimeout' FAILED
|
17
|
+
expected: > 10,
|
18
|
+
got: 0
|
19
|
+
/Users/olegandreev/Work/emrpc.git/spec/blocking_api/multithreaded_client_spec.rb:49:
|
20
|
+
* Transactions for MultithreadedClient to enable scheduling the same backend for all messages in a transaction.
|
21
|
+
This can be implemented in two ways:
|
22
|
+
1. Blocking: when transaction starts, backend is taken out of the queue while transaction is in process.
|
23
|
+
2. Non-blocking: backend can be rescheduled to other messages while transaction is in process,
|
24
|
+
but when it is needed again, system waits for that particular backend.
|
25
|
+
Non-blocking option might seem more efficient, but it also might be less safe: what if you really should
|
26
|
+
not interleave communication with the backend with foreign messages?
|
27
|
+
|
28
|
+
* SinglethreadedClient creates a messaging thread for each Pid. We should use some kind of
|
29
|
+
shared channel for a number of pids.
|
30
|
+
* Try to use Thread.critical instead of Queue.new to achive better performance.
|
31
|
+
|
32
|
+
FEATURES
|
33
|
+
|
34
|
+
* filters for messages (erlang-like receive() pattern-matching)
|
35
|
+
* passing particular objects through TCP
|
36
|
+
* resource discovery
|
37
|
+
* ring/chord protocol
|
38
|
+
* rack http server & http client
|
39
|
+
* Async DNS support
|
40
|
+
* REv along with eventmachine backend
|
41
|
+
|
42
|
+
TASKS
|
43
|
+
|
44
|
+
* Documentation and examples!
|
45
|
+
* Benchmark against DRb.
|
46
|
+
* Publish gem somewhere.
|
47
|
+
* Total world domination.
|
data/bin/emrpc
ADDED
data/lib/emrpc.rb
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'eventmachine'
|
3
|
+
|
4
|
+
# add current dir to the load path
|
5
|
+
$LOAD_PATH.unshift(File.expand_path(File.join(File.dirname(__FILE__))))
|
6
|
+
|
7
|
+
$DEBUG ||= ENV['DEBUG']
|
8
|
+
|
9
|
+
require 'emrpc/version'
|
10
|
+
require 'emrpc/util'
|
11
|
+
require 'emrpc/protocols'
|
12
|
+
require 'emrpc/evented_api'
|
13
|
+
require 'emrpc/blocking_api'
|
14
|
+
require 'emrpc/server'
|
15
|
+
require 'emrpc/client'
|
@@ -0,0 +1,48 @@
|
|
1
|
+
module EMRPC
|
2
|
+
|
3
|
+
class StaleReference < StandardError; end
|
4
|
+
|
5
|
+
class ReferenceSavior
|
6
|
+
attr_accessor :timeout
|
7
|
+
# :timeout - specifies a time interval for keeping a reference to a value
|
8
|
+
def initialize(options)
|
9
|
+
@timeout = options[:timeout] || 60
|
10
|
+
@timer = options[:timer] || Timers::EVENTED
|
11
|
+
@timeout_thread = @timer.call(@timeout, method(:swap_pages))
|
12
|
+
# We keep two previous pages to ensure, that the value lives for at least +timeout+ seconds.
|
13
|
+
@page1 = new_page
|
14
|
+
@page2 = new_page
|
15
|
+
end
|
16
|
+
|
17
|
+
def set(val)
|
18
|
+
oid = val.object_id
|
19
|
+
ping(oid, val)
|
20
|
+
MethodProxy.new(Wrapper.new(oid))
|
21
|
+
end
|
22
|
+
|
23
|
+
def get(oid)
|
24
|
+
oid = oid.to_i
|
25
|
+
val = @page1[oid] || @page2[oid]
|
26
|
+
raise StaleReference.new("Object id #{oid} was not found in a ReferenceSavior cache. It could have been garbage collected after #{@timeout} sec.") unless val
|
27
|
+
ping(oid, val)
|
28
|
+
val
|
29
|
+
end
|
30
|
+
|
31
|
+
def ping(oid, val)
|
32
|
+
@page1[oid] = val
|
33
|
+
end
|
34
|
+
|
35
|
+
# @page2 disappears and will be garbage collected.
|
36
|
+
# @page1 is copied to @page2 for another @timeout seconds
|
37
|
+
# and is replaced by a brand new cache page.
|
38
|
+
def swap_pages
|
39
|
+
@page2 = @page1
|
40
|
+
@page1 = new_page
|
41
|
+
end
|
42
|
+
|
43
|
+
def new_page
|
44
|
+
Hash.new
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
@@ -0,0 +1,44 @@
|
|
1
|
+
module EMRPC
|
2
|
+
=begin
|
3
|
+
|
4
|
+
REPLICATION
|
5
|
+
|
6
|
+
Who does replication?
|
7
|
+
|
8
|
+
Master schedules replication streams:
|
9
|
+
+ replication is centralized
|
10
|
+
- replicas' existance is not guaranteed.
|
11
|
+
- overcomplication (?)
|
12
|
+
Primary chunkserver replicates data to secondary chunkservers:
|
13
|
+
- every chunkserver must know about other chunkservers
|
14
|
+
-
|
15
|
+
Client streams data to all the chunkservers:
|
16
|
+
- bandwidth!
|
17
|
+
+ most simple scheme
|
18
|
+
|
19
|
+
RELIABILITY PRINCIPLES
|
20
|
+
|
21
|
+
1. Client should be guaranteed, that data is completely replicated.
|
22
|
+
Therefore, it can do replication on its own.
|
23
|
+
For optimization reasons, it can delegate some streams to
|
24
|
+
other nodes. Say, it sends data to node A and tells it to replicate data
|
25
|
+
to node B. A must report finish of both A and B streamings.
|
26
|
+
2. Client may connect any node and try to upload the file there.
|
27
|
+
|
28
|
+
=end
|
29
|
+
|
30
|
+
module Ring
|
31
|
+
|
32
|
+
class Bucket
|
33
|
+
attr_accessor :range
|
34
|
+
def initialize
|
35
|
+
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
class Master
|
40
|
+
|
41
|
+
end
|
42
|
+
|
43
|
+
end
|
44
|
+
end
|