arya-pandemic 0.2.1 → 0.2.3

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,20 @@
1
+ lib/pandemic/client_side/cluster_connection.rb
2
+ lib/pandemic/client_side/config.rb
3
+ lib/pandemic/client_side/connection.rb
4
+ lib/pandemic/client_side/connection_proxy.rb
5
+ lib/pandemic/client_side/pandemize.rb
6
+ lib/pandemic/connection_pool.rb
7
+ lib/pandemic/mutex_counter.rb
8
+ lib/pandemic/server_side/client.rb
9
+ lib/pandemic/server_side/config.rb
10
+ lib/pandemic/server_side/handler.rb
11
+ lib/pandemic/server_side/peer.rb
12
+ lib/pandemic/server_side/request.rb
13
+ lib/pandemic/server_side/server.rb
14
+ lib/pandemic/util.rb
15
+ lib/pandemic.rb
16
+ Manifest
17
+ MIT-LICENSE
18
+ pandemic.gemspec
19
+ Rakefile
20
+ README.markdown
@@ -1,5 +1,9 @@
1
1
  # Pandemic
2
- Pandemic is a map-reduce framework. You give it the map, process, and reduce methods and it handles the rest. It works both in Ruby 1.8 and Ruby 1.9, and performs better on 1.9.
2
+ Pandemic is a map-reduce framework. You give it the map, process, and reduce methods and it handles the rest. It's designed to serve requests in real-time, but can also be used for offline tasks.
3
+
4
+ It's different from the typical map-reduce framework in that it doesn't have a master-worker structure. Every node can map, process, and reduce. It also doesn't have the concept of jobs, everything is a request.
5
+
6
+ The framework is designed to be as flexible as possible, there is no rigid request format, or API, you can specify it however you want. You can send it http-style headers and a body, you can send it JSON, or you can even just send it a single line and have it do whatever you want. The only requirement is that you write your handler to appropriately act on the request and return the response.
3
7
 
4
8
  ## Usage
5
9
  ### Server
@@ -53,7 +57,9 @@ Each value for the server list is the _host:port_ that a node can bind to. The s
53
57
  - host2:4000
54
58
  max_connections_per_server: 10
55
59
  min_connections_per_server: 1
56
- The min/max connections refers to how many connections to each node. If you're using Rails, then just use 1 for both min/max since it's single threaded.
60
+ response_timeout: 1
61
+
62
+ The min/max connections refers to how many connections to each node. If you're using the client in Rails, then just use 1 for both min/max since it's single threaded.
57
63
 
58
64
  ### More Config
59
65
  There are three ways to start a server:
@@ -66,7 +72,7 @@ The first refers to the index in the servers array:
66
72
 
67
73
  servers:
68
74
  - host1:4000 # started with ruby server.rb -i 0
69
- - host2:4000 # started with ruby server.rb -i 0
75
+ - host2:4000 # started with ruby server.rb -i 1
70
76
 
71
77
  The second refers to the index in the servers _hash_. This can be particularly useful if you use the hostname as the key.
72
78
 
@@ -96,4 +102,5 @@ And you can access these additional options using _config.get(keys)_ in your han
96
102
  def initialize
97
103
  @dbh = Mysql.real_connect(*config.get('host', 'username', 'password', 'database'))
98
104
  end
99
- end
105
+ end
106
+
data/Rakefile CHANGED
@@ -2,7 +2,7 @@ require 'rubygems'
2
2
  require 'rake'
3
3
  require 'echoe'
4
4
 
5
- Echoe.new('pandemic', '0.2.1') do |p|
5
+ Echoe.new('pandemic', '0.2.3') do |p|
6
6
  p.description = "Distribute MapReduce to any of the workers and it will spread, like a pandemic."
7
7
  p.url = "https://github.com/arya/pandemic/"
8
8
  p.author = "Arya Asemanfar"
@@ -30,6 +30,9 @@ require 'pandemic/client_side/pandemize'
30
30
  # - documentation
31
31
  # - PING/PONG?
32
32
 
33
+ TCP_NO_DELAY_AVAILABLE =
34
+ RUBY_VERSION < '1.9' ? Socket.constants.include?('TCP_NODELAY') : Socket.constants.include?(:TCP_NODELAY)
35
+
33
36
  def epidemic!
34
37
  if $pandemic_logger.nil?
35
38
  $pandemic_logger = Logger.new("pandemic.log")
@@ -16,7 +16,7 @@ module Pandemic
16
16
  @socket = begin
17
17
  connection = TCPSocket.new(@host, @port)
18
18
  if connection && !connection.closed?
19
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
19
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
20
20
  connection.write("CLIENT\n")
21
21
  connection
22
22
  else
@@ -106,7 +106,7 @@ module Pandemic
106
106
  warn("Unhandled exception in create connection block: #{e.inspect}")
107
107
  end
108
108
  if connection
109
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
109
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
110
110
  connection.write("SERVER #{@server.signature}\n")
111
111
  end
112
112
  connection
@@ -84,7 +84,7 @@ module Pandemic
84
84
 
85
85
  def handle_connection(connection)
86
86
  begin
87
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
87
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
88
88
 
89
89
  identification = connection.gets.strip
90
90
  info("Incoming connection from #{connection.peeraddr.values_at(3,1).join(":")} (#{identification})")
@@ -2,11 +2,11 @@
2
2
 
3
3
  Gem::Specification.new do |s|
4
4
  s.name = %q{pandemic}
5
- s.version = "0.2.1"
5
+ s.version = "0.2.3"
6
6
 
7
7
  s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
8
8
  s.authors = ["Arya Asemanfar"]
9
- s.date = %q{2009-03-26}
9
+ s.date = %q{2009-04-10}
10
10
  s.description = %q{Distribute MapReduce to any of the workers and it will spread, like a pandemic.}
11
11
  s.email = %q{aryaasemanfar@gmail.com}
12
12
  s.extra_rdoc_files = ["lib/pandemic/client_side/cluster_connection.rb", "lib/pandemic/client_side/config.rb", "lib/pandemic/client_side/connection.rb", "lib/pandemic/client_side/connection_proxy.rb", "lib/pandemic/client_side/pandemize.rb", "lib/pandemic/connection_pool.rb", "lib/pandemic/mutex_counter.rb", "lib/pandemic/server_side/client.rb", "lib/pandemic/server_side/config.rb", "lib/pandemic/server_side/handler.rb", "lib/pandemic/server_side/peer.rb", "lib/pandemic/server_side/request.rb", "lib/pandemic/server_side/server.rb", "lib/pandemic/util.rb", "lib/pandemic.rb", "README.markdown"]
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: arya-pandemic
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.1
4
+ version: 0.2.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Arya Asemanfar
@@ -9,7 +9,7 @@ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
11
 
12
- date: 2009-03-26 00:00:00 -07:00
12
+ date: 2009-04-10 00:00:00 -07:00
13
13
  default_executable:
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency