protoplasm-client 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,4 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
data/Gemfile ADDED
@@ -0,0 +1,5 @@
1
+ source "http://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in protoplasm.gemspec
4
+ gemspec :name => 'protoplasm-server'
5
+ gemspec :name => 'protoplasm-client'
@@ -0,0 +1,103 @@
1
+ # Protoplasm
2
+
3
+ Protoplasm makes is easy to define an RPC server/client which is backed by protobuf through Beefcake.
4
+
5
+ ## Defining your service endpoints
6
+
7
+ The current service model is very simple. You can send one protobuf object (the request object). This object must have an enum and a series of optional command fields to allow it to send commands. From the tests, this is a valid request object definition:
8
+
9
+ ```ruby
10
+ class Command
11
+ include Beefcake::Message
12
+ module Type
13
+ PING = 1
14
+ UPCASE = 2
15
+ EVEN = 3
16
+ end
17
+ required :type, Type, 1
18
+ optional :ping_command, PingCommand, 2
19
+ optional :upcase_command, UpcaseCommand, 3
20
+ optional :even_command, EvenCommand, 4
21
+ end
22
+ ```
23
+
24
+ In this case, your request object would be able to accept one, and only one subcommand object. Those types are `PingCommand`, `UpcaseCommand` and `EvenCommand`.
25
+
26
+ So, in order to mark this `Command` class as your request class, you'd do the following:
27
+
28
+ ```ruby
29
+ module Types
30
+ include Protoplasm::Types
31
+
32
+ # .. your actual classes would go here
33
+
34
+ request_class Command
35
+ request_type_field :type
36
+ end
37
+ ```
38
+
39
+ ## Defining your response objects
40
+
41
+ Every subcommand can choose to relay back no objects, one object, or stream any number of objects. Those objects must all be of the same type.
42
+
43
+ To define which objects you expect back, you must add the following.
44
+
45
+ ```ruby
46
+ module Types
47
+ rpc_map Command::Type::PING, :ping_command, nil
48
+ rpc_map Command::Type::UPCASE, :upcase_command, UpcaseResponse
49
+ rpc_map Command::Type::EVEN, :even_command, EvenResponse, :streaming => true
50
+ ```
51
+
52
+ In this case, this would define the ping command as returning no object, the upcase command returns a single object, of type `UpcaseResponse`, and the even command return any number of `EvenResponse` objects.
53
+
54
+ ## Server implementation
55
+
56
+ Currently there is a single server implementation `EMServer`, which defines a non-blocking EventMachine based server. To create an `EMServer`, you subclass `Protoplasm::EMServer` and setup handlers for each of your command types. For example, a worker server could look like this:
57
+
58
+ ```ruby
59
+ class Server < Protoplasm::EMServer
60
+ def process_ping_command(ping_command)
61
+ # do nothing
62
+ end
63
+
64
+ def process_upcase_command(upcase_command)
65
+ send_response(:response => cmd.word.upcase)
66
+ end
67
+
68
+ def process_even_command(even_command)
69
+ (1..even_command.top).each do |num|
70
+ send_response(:num => num) if num % 2 == 0
71
+ end
72
+ finish_streaming
73
+ end
74
+ end
75
+ ```
76
+
77
+ This server then could be started with `Server.start(3000)` which would start on port 3000 and process requests.
78
+
79
+ ## Client
80
+
81
+ Currently there is a single client implementation: `BlockingClient`. It defines a blocking `TCPSocket` based client. To create a client for this example, you would do the following.
82
+
83
+ ```ruby
84
+ class Client < Protoplasm::BlockingClient
85
+ def initialize(host, port)
86
+ super(Types, host, port)
87
+ end
88
+
89
+ def ping
90
+ send_request(:ping_command)
91
+ end
92
+
93
+ def upcase(word)
94
+ send_request(:upcase_command, :word => word).response
95
+ end
96
+
97
+ def evens(top)
98
+ send_request(:even_command, :top => top) { |resp| yield resp.num }
99
+ end
100
+ end
101
+ ```
102
+
103
+ Look at the full example under `test/test_helper.rb`.
@@ -0,0 +1,74 @@
1
+ require 'bundler'
2
+ require 'rake/testtask'
3
+ require './lib/protoplasm/version'
4
+
5
+ task :test do
6
+ Rake::TestTask.new do |t|
7
+ Dir['test/*_test.rb'].each{|f| require File.expand_path(f)}
8
+ end
9
+ end
10
+
11
+ def version
12
+ Protoplasm::VERSION
13
+ end
14
+
15
+ def version_tag
16
+ "v#{version}"
17
+ end
18
+
19
+ def tag_version
20
+ system("git tag -a -m \"Version #{version}\" #{version_tag}") or raise("Cannot tag version")
21
+ Bundler.ui.confirm "Tagged #{version_tag}"
22
+ yield
23
+ rescue
24
+ Bundler.ui.error "Untagged #{version_tag} due to error"
25
+ system("git tag -d #{version_tag}") or raise("Cannot untag version")
26
+ raise
27
+ end
28
+
29
+
30
+ desc "Release client & server (#{version})"
31
+ task :release do
32
+ tag_version do
33
+ Rake::Task["server:release_without_tagging"].invoke
34
+ Rake::Task["client:release_without_tagging"].invoke
35
+ end
36
+ end
37
+
38
+ desc "Install client & server (#{version})"
39
+ task :install do
40
+ Rake::Task["server:install"].invoke
41
+ Rake::Task["client:install"].invoke
42
+ end
43
+
44
+ desc "Build client & server (#{version})"
45
+ task :build do
46
+ Rake::Task["server:build"].invoke
47
+ Rake::Task["client:build"].invoke
48
+ end
49
+
50
+ namespace :server do
51
+ helper = Bundler::GemHelper.new(File.dirname(__FILE__), "protoplasm-server")
52
+ helper.install
53
+ helper.instance_eval do
54
+ task :release_without_tagging do
55
+ guard_clean
56
+ built_gem_path = build_gem
57
+ git_push
58
+ rubygem_push(built_gem_path)
59
+ end
60
+ end
61
+ end
62
+
63
+ namespace :client do
64
+ helper = Bundler::GemHelper.new(File.dirname(__FILE__), "protoplasm-client")
65
+ helper.install
66
+ helper.instance_eval do
67
+ task :release_without_tagging do
68
+ guard_clean
69
+ built_gem_path = build_gem
70
+ git_push
71
+ rubygem_push(built_gem_path)
72
+ end
73
+ end
74
+ end
@@ -0,0 +1,7 @@
1
+ require "protoplasm/version"
2
+
3
+ module Protoplasm
4
+ autoload :BlockingClient, "protoplasm/client/blocking_client"
5
+ autoload :EMServer, "protoplasm/server/em_server"
6
+ autoload :Types, "protoplasm/types/types"
7
+ end
@@ -0,0 +1,63 @@
1
+ require 'socket'
2
+
3
+ module Protoplasm
4
+ class BlockingClient
5
+ attr_reader :_types
6
+
7
+ def initialize(_types, host, port)
8
+ @_types, @host, @port = _types, host, port
9
+ end
10
+
11
+ private
12
+ def _socket(force_new = false)
13
+ if force_new
14
+ TCPSocket.open(@host, @port)
15
+ else
16
+ @_socket ||= TCPSocket.open(@host, @port)
17
+ end
18
+ end
19
+
20
+ def _master_socket
21
+ @_master_socket ||= _socket
22
+ end
23
+
24
+ def send_request(field, *args, &blk)
25
+ s = ''
26
+ type = _types.request_type_for_field(field)
27
+ cmd_class = type.command_class.fields.values.find{|f| f.name == field}
28
+ cmd = _types.request_class.new(_types.request_type_field => type.type, type.field => cmd_class.type.new(*args))
29
+ cmd.encode(s)
30
+ socket = _socket(type.streaming?)
31
+ socket.write([0, s.size].pack("CQ"))
32
+ socket.write s
33
+ socket.flush
34
+ unless type.void?
35
+ if type.streaming?
36
+ begin
37
+ until socket.eof?
38
+ len = socket.read(8).unpack("Q").first
39
+ buf = ''
40
+ while buf.size < len
41
+ socket.read(len - buf.size, buf)
42
+ end
43
+ yield type.response_class.decode(buf)
44
+ end
45
+ ensure
46
+ socket.close
47
+ end
48
+ else
49
+ len_buf = socket.readpartial(8)
50
+ while len_buf.size < 8
51
+ socket.readpartial(8 - len_buf.size, len_buf)
52
+ end
53
+ len = len_buf.unpack("Q").first
54
+ buf = ''
55
+ until buf.size == len or socket.eof?
56
+ socket.readpartial(len - buf.size, buf)
57
+ end
58
+ type.response_class.decode(buf)
59
+ end
60
+ end
61
+ end
62
+ end
63
+ end
@@ -0,0 +1,71 @@
1
+ require 'eventmachine'
2
+
3
+ module Protoplasm
4
+ class EMServer < EventMachine::Connection
5
+ CONTROL_REQUEST = 0
6
+
7
+ def self.start(types, port)
8
+ if EM.reactor_running?
9
+ EM::start_server("0.0.0.0", port, self) do |srv|
10
+ srv._types = types
11
+ yield srv if block_given?
12
+ end
13
+ else
14
+ begin
15
+ EM.run do
16
+ start(types, port)
17
+ end
18
+ rescue Interrupt
19
+ end
20
+ end
21
+ end
22
+
23
+ attr_accessor :_types
24
+
25
+ def post_init
26
+ @_response_types = []
27
+ @data = ''
28
+ end
29
+
30
+ def receive_data(data)
31
+ @data << data
32
+ data_ready
33
+ end
34
+
35
+ def finish_streaming
36
+ close_connection_after_writing
37
+ end
38
+
39
+ def data_ready
40
+ @control = @data.slice!(0, 1).unpack("C").first unless @control
41
+ case @control
42
+ when CONTROL_REQUEST
43
+ @size = @data.slice!(0, 8).unpack("Q").first unless @size
44
+
45
+ if @data.size >= @size
46
+ buf = @data.slice!(0, @size)
47
+ @size, @control = nil, nil
48
+ obj = _types.request_class.decode(buf)
49
+ type = _types.request_type_for_request(obj)
50
+ @_response_types << type unless type.void?
51
+ EM.next_tick { send(:"process_#{type.field}", obj.send(type.field)) }
52
+ data_ready unless @data.empty?
53
+ #EM.next_tick { data_ready } #todo left over data needs to be processed still
54
+ end
55
+ else
56
+ # illegal char
57
+ close_connection
58
+ end
59
+ end
60
+
61
+ def send_response(*args)
62
+ type = @_response_types.first
63
+ @_response_types.shift unless type.streaming?
64
+ obj = type.response_class.new(*args)
65
+ s = ''
66
+ obj.encode(s)
67
+ send_data [s.size].pack("Q")
68
+ send_data s
69
+ end
70
+ end
71
+ end
@@ -0,0 +1,69 @@
1
+ require 'beefcake'
2
+
3
+ module Protoplasm
4
+ module Types
5
+ # the truest thing i know ... the enum pattern matches to a response class
6
+ # therefore, you need to know the following
7
+ # where is the request class
8
+ # where is the field class
9
+ # how does the field map to response classes
10
+
11
+ def self.included(cls)
12
+ cls.extend(ClassMethods)
13
+ end
14
+
15
+ class RequestResponseType < Struct.new(:request_class, :response_class, :type, :field, :streaming)
16
+
17
+ alias_method :streaming?, :streaming
18
+
19
+ def command_class
20
+ request_class
21
+ end
22
+
23
+ def void?
24
+ response_class.nil?
25
+ end
26
+ end
27
+
28
+ module ClassMethods
29
+ def request_class(request_class = nil)
30
+ request_class ? @request_class = request_class : @request_class
31
+ end
32
+
33
+ def request_type(request_obj)
34
+ request_obj.send(@request_type_field)
35
+ end
36
+
37
+ def request_type_field(field = nil)
38
+ field ? @request_type_field = field : @request_type_field
39
+ end
40
+
41
+ def rpc_map(type, field, response_class, opts = nil)
42
+ @response_map_by_field ||= {}
43
+ @response_map_by_type ||= {}
44
+ streaming = opts && opts.key?(:streaming) ? opts[:streaming] : false
45
+ rrt = RequestResponseType.new(@request_class, response_class, type, field, streaming)
46
+ @response_map_by_field[field] = rrt
47
+ @response_map_by_type[type] = rrt
48
+ end
49
+
50
+ def request_type_for_field(field)
51
+ @response_map_by_field[field]
52
+ end
53
+
54
+ def request_type_for_request(req)
55
+ @response_map_by_type[req.send(@request_type_field)]
56
+ end
57
+ end
58
+ end
59
+ end
60
+
61
+ # module Unilogger
62
+ # module Types
63
+ # include Protoplasm::Types
64
+ #
65
+ # request_class RequestCommand
66
+ # request_type_field :command_type
67
+ # response_map LogCommand::CommandType::INSERT, nil
68
+ # end
69
+ # end
@@ -0,0 +1,3 @@
1
+ module Protoplasm
2
+ VERSION = "0.0.1"
3
+ end
@@ -0,0 +1,25 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "protoplasm/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "protoplasm-client"
7
+ s.version = Protoplasm::VERSION
8
+ s.authors = ["Josh Hull"]
9
+ s.email = ["joshbuddy@gmail.com"]
10
+ s.homepage = ""
11
+ s.summary = %q{The protoplasm client}
12
+ s.description = %q{The protoplasm client.}
13
+
14
+ s.rubyforge_project = "protoplasm"
15
+
16
+ s.files = `git ls-files`.split("\n")
17
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
18
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
19
+ s.require_paths = ["lib"]
20
+
21
+ s.add_dependency "beefcake", "~> 0.3.7"
22
+
23
+ s.add_development_dependency 'rake'
24
+ s.add_development_dependency 'minitest', "~> 2.6.1"
25
+ end
@@ -0,0 +1,27 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "protoplasm/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "protoplasm-server"
7
+ s.version = Protoplasm::VERSION
8
+ s.authors = ["Josh Hull"]
9
+ s.email = ["joshbuddy@gmail.com"]
10
+ s.homepage = ""
11
+ s.summary = %q{The protoplasm server}
12
+ s.description = %q{The protoplasm server.}
13
+
14
+ s.rubyforge_project = "protoplasm"
15
+
16
+ s.files = `git ls-files`.split("\n")
17
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
18
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
19
+ s.require_paths = ["lib"]
20
+
21
+ s.add_dependency "eventmachine"
22
+ s.add_dependency "beefcake", "~> 0.3.7"
23
+
24
+ s.add_development_dependency 'rake'
25
+ s.add_development_dependency 'minitest', "~> 2.6.1"
26
+
27
+ end
@@ -0,0 +1,46 @@
1
+ require File.expand_path('../test_helper', __FILE__)
2
+
3
+ describe "Protoplasm test server" do
4
+ it "should ping" do
5
+ with_proto_server(ProtoplasmTest::EMServer) do |port|
6
+ client = ProtoplasmTest::Client.new('127.0.0.1', port)
7
+ client.ping
8
+ pass
9
+ end
10
+ end
11
+
12
+ it "should upcase" do
13
+ with_proto_server(ProtoplasmTest::EMServer) do |port|
14
+ client = ProtoplasmTest::Client.new('127.0.0.1', port)
15
+ assert_equal "LOWER", client.upcase('lower')
16
+ end
17
+ end
18
+
19
+ it "should give you even numbers" do
20
+ with_proto_server(ProtoplasmTest::EMServer) do |port|
21
+ client = ProtoplasmTest::Client.new('127.0.0.1', port)
22
+ nums = []
23
+ client.evens(10) do |resp|
24
+ nums << resp
25
+ end
26
+ assert_equal [2, 4, 6, 8], nums
27
+ end
28
+ end
29
+
30
+ it "should allow multiple calls" do
31
+ with_proto_server(ProtoplasmTest::EMServer) do |port|
32
+ client = ProtoplasmTest::Client.new('127.0.0.1', port)
33
+ client.ping
34
+ assert_equal "LOWER", client.upcase('lower')
35
+ assert_equal "UPPER", client.upcase('upper')
36
+ nums = []
37
+ client.evens(10) do |resp|
38
+ nums << resp
39
+ end
40
+ assert_equal [2, 4, 6, 8], nums
41
+ client.ping
42
+ assert_equal "LOWER", client.upcase('lower')
43
+ end
44
+ end
45
+
46
+ end
@@ -0,0 +1,118 @@
1
+ require 'rubygems'
2
+ require 'minitest/autorun'
3
+ require 'protoplasm'
4
+ require 'timeout'
5
+
6
+ class ProtoplasmTest
7
+ module Types
8
+ include Protoplasm::Types
9
+
10
+ class PingCommand
11
+ include Beefcake::Message
12
+ end
13
+
14
+ class UpcaseCommand
15
+ include Beefcake::Message
16
+ required :word, :string, 1
17
+ end
18
+
19
+ class EvenCommand
20
+ include Beefcake::Message
21
+ required :top, :uint32, 1
22
+ end
23
+
24
+ class UpcaseResponse
25
+ include Beefcake::Message
26
+ required :response, :string, 1
27
+ end
28
+
29
+ class EvenResponse
30
+ include Beefcake::Message
31
+ required :num, :uint32, 1
32
+ end
33
+
34
+ class Command
35
+ include Beefcake::Message
36
+ module Type
37
+ PING = 1
38
+ UPCASE = 2
39
+ EVEN = 3
40
+ end
41
+ required :type, Type, 1
42
+ optional :ping_command, PingCommand, 2
43
+ optional :upcase_command, UpcaseCommand, 3
44
+ optional :even_command, EvenCommand, 4
45
+ end
46
+
47
+ request_class Command
48
+ request_type_field :type
49
+ rpc_map Command::Type::PING, :ping_command, nil
50
+ rpc_map Command::Type::UPCASE, :upcase_command, UpcaseResponse
51
+ rpc_map Command::Type::EVEN, :even_command, EvenResponse, :streaming => true
52
+ end
53
+
54
+ class EMServer < Protoplasm::EMServer
55
+ def process_upcase_command(cmd)
56
+ send_response(:response => cmd.word.upcase)
57
+ end
58
+
59
+ def process_even_command(cmd)
60
+ spit_out_even(1, cmd.top)
61
+ end
62
+
63
+ def process_ping_command(cmd)
64
+ # no op
65
+ end
66
+
67
+ def spit_out_even(cur, top)
68
+ EM.next_tick do
69
+ if cur == top
70
+ finish_streaming
71
+ else
72
+ if cur % 2 == 0
73
+ send_response(:num => cur)
74
+ end
75
+ spit_out_even(cur + 1, top)
76
+ end
77
+ end
78
+ end
79
+ end
80
+
81
+ class Client < Protoplasm::BlockingClient
82
+ def initialize(host, port)
83
+ super(Types, host, port)
84
+ end
85
+
86
+ def ping
87
+ send_request(:ping_command)
88
+ end
89
+
90
+ def upcase(word)
91
+ send_request(:upcase_command, :word => word).response
92
+ end
93
+
94
+ def evens(top)
95
+ send_request(:even_command, :top => top) { |resp| yield resp.num }
96
+ end
97
+ end
98
+ end
99
+
100
+ class MiniTest::Spec
101
+ def with_proto_server(cls)
102
+ port = 19866
103
+ pid = fork { cls.start(ProtoplasmTest::Types, port) }
104
+ begin
105
+ Timeout.timeout(10.0) {
106
+ begin
107
+ TCPSocket.open("127.0.0.1", port).close
108
+ rescue
109
+ sleep(0.1)
110
+ retry
111
+ end
112
+ }
113
+ yield port
114
+ ensure
115
+ Process.kill("INT", pid) if pid
116
+ end
117
+ end
118
+ end
metadata ADDED
@@ -0,0 +1,99 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: protoplasm-client
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Josh Hull
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2011-11-21 00:00:00.000000000Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: beefcake
16
+ requirement: &70270602546480 !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ~>
20
+ - !ruby/object:Gem::Version
21
+ version: 0.3.7
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: *70270602546480
25
+ - !ruby/object:Gem::Dependency
26
+ name: rake
27
+ requirement: &70270602545980 !ruby/object:Gem::Requirement
28
+ none: false
29
+ requirements:
30
+ - - ! '>='
31
+ - !ruby/object:Gem::Version
32
+ version: '0'
33
+ type: :development
34
+ prerelease: false
35
+ version_requirements: *70270602545980
36
+ - !ruby/object:Gem::Dependency
37
+ name: minitest
38
+ requirement: &70270602545400 !ruby/object:Gem::Requirement
39
+ none: false
40
+ requirements:
41
+ - - ~>
42
+ - !ruby/object:Gem::Version
43
+ version: 2.6.1
44
+ type: :development
45
+ prerelease: false
46
+ version_requirements: *70270602545400
47
+ description: The protoplasm client.
48
+ email:
49
+ - joshbuddy@gmail.com
50
+ executables: []
51
+ extensions: []
52
+ extra_rdoc_files: []
53
+ files:
54
+ - .gitignore
55
+ - Gemfile
56
+ - README.md
57
+ - Rakefile
58
+ - lib/protoplasm.rb
59
+ - lib/protoplasm/client/blocking_client.rb
60
+ - lib/protoplasm/server/em_server.rb
61
+ - lib/protoplasm/types/types.rb
62
+ - lib/protoplasm/version.rb
63
+ - protoplasm-client.gemspec
64
+ - protoplasm-server.gemspec
65
+ - test/protoplasm_test.rb
66
+ - test/test_helper.rb
67
+ homepage: ''
68
+ licenses: []
69
+ post_install_message:
70
+ rdoc_options: []
71
+ require_paths:
72
+ - lib
73
+ required_ruby_version: !ruby/object:Gem::Requirement
74
+ none: false
75
+ requirements:
76
+ - - ! '>='
77
+ - !ruby/object:Gem::Version
78
+ version: '0'
79
+ segments:
80
+ - 0
81
+ hash: -3078516638908246795
82
+ required_rubygems_version: !ruby/object:Gem::Requirement
83
+ none: false
84
+ requirements:
85
+ - - ! '>='
86
+ - !ruby/object:Gem::Version
87
+ version: '0'
88
+ segments:
89
+ - 0
90
+ hash: -3078516638908246795
91
+ requirements: []
92
+ rubyforge_project: protoplasm
93
+ rubygems_version: 1.8.10
94
+ signing_key:
95
+ specification_version: 3
96
+ summary: The protoplasm client
97
+ test_files:
98
+ - test/protoplasm_test.rb
99
+ - test/test_helper.rb