arya-pandemic 0.2.1 → 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,20 @@
1
+ lib/pandemic/client_side/cluster_connection.rb
2
+ lib/pandemic/client_side/config.rb
3
+ lib/pandemic/client_side/connection.rb
4
+ lib/pandemic/client_side/connection_proxy.rb
5
+ lib/pandemic/client_side/pandemize.rb
6
+ lib/pandemic/connection_pool.rb
7
+ lib/pandemic/mutex_counter.rb
8
+ lib/pandemic/server_side/client.rb
9
+ lib/pandemic/server_side/config.rb
10
+ lib/pandemic/server_side/handler.rb
11
+ lib/pandemic/server_side/peer.rb
12
+ lib/pandemic/server_side/request.rb
13
+ lib/pandemic/server_side/server.rb
14
+ lib/pandemic/util.rb
15
+ lib/pandemic.rb
16
+ Manifest
17
+ MIT-LICENSE
18
+ pandemic.gemspec
19
+ Rakefile
20
+ README.markdown
@@ -1,5 +1,9 @@
1
1
  # Pandemic
2
- Pandemic is a map-reduce framework. You give it the map, process, and reduce methods and it handles the rest. It works both in Ruby 1.8 and Ruby 1.9, and performs better on 1.9.
2
+ Pandemic is a map-reduce framework. You give it the map, process, and reduce methods and it handles the rest. It's designed to serve requests in real-time, but can also be used for offline tasks.
3
+
4
+ It's different from the typical map-reduce framework in that it doesn't have a master-worker structure. Every node can map, process, and reduce. It also doesn't have the concept of jobs, everything is a request.
5
+
6
+ The framework is designed to be as flexible as possible, there is no rigid request format, or API, you can specify it however you want. You can send it http-style headers and a body, you can send it JSON, or you can even just send it a single line and have it do whatever you want. The only requirement is that you write your handler to appropriately act on the request and return the response.
3
7
 
4
8
  ## Usage
5
9
  ### Server
@@ -53,7 +57,9 @@ Each value for the server list is the _host:port_ that a node can bind to. The s
53
57
  - host2:4000
54
58
  max_connections_per_server: 10
55
59
  min_connections_per_server: 1
56
- The min/max connections refers to how many connections to each node. If you're using Rails, then just use 1 for both min/max since it's single threaded.
60
+ response_timeout: 1
61
+
62
+ The min/max connections refers to how many connections to each node. If you're using the client in Rails, then just use 1 for both min/max since it's single threaded.
57
63
 
58
64
  ### More Config
59
65
  There are three ways to start a server:
@@ -66,7 +72,7 @@ The first refers to the index in the servers array:
66
72
 
67
73
  servers:
68
74
  - host1:4000 # started with ruby server.rb -i 0
69
- - host2:4000 # started with ruby server.rb -i 0
75
+ - host2:4000 # started with ruby server.rb -i 1
70
76
 
71
77
  The second refers to the index in the servers _hash_. This can be particularly useful if you use the hostname as the key.
72
78
 
@@ -96,4 +102,5 @@ And you can access these additional options using _config.get(keys)_ in your han
96
102
  def initialize
97
103
  @dbh = Mysql.real_connect(*config.get('host', 'username', 'password', 'database'))
98
104
  end
99
- end
105
+ end
106
+
data/Rakefile CHANGED
@@ -2,7 +2,7 @@ require 'rubygems'
2
2
  require 'rake'
3
3
  require 'echoe'
4
4
 
5
- Echoe.new('pandemic', '0.2.1') do |p|
5
+ Echoe.new('pandemic', '0.2.3') do |p|
6
6
  p.description = "Distribute MapReduce to any of the workers and it will spread, like a pandemic."
7
7
  p.url = "https://github.com/arya/pandemic/"
8
8
  p.author = "Arya Asemanfar"
@@ -30,6 +30,9 @@ require 'pandemic/client_side/pandemize'
30
30
  # - documentation
31
31
  # - PING/PONG?
32
32
 
33
+ TCP_NO_DELAY_AVAILABLE =
34
+ RUBY_VERSION < '1.9' ? Socket.constants.include?('TCP_NODELAY') : Socket.constants.include?(:TCP_NODELAY)
35
+
33
36
  def epidemic!
34
37
  if $pandemic_logger.nil?
35
38
  $pandemic_logger = Logger.new("pandemic.log")
@@ -16,7 +16,7 @@ module Pandemic
16
16
  @socket = begin
17
17
  connection = TCPSocket.new(@host, @port)
18
18
  if connection && !connection.closed?
19
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
19
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
20
20
  connection.write("CLIENT\n")
21
21
  connection
22
22
  else
@@ -106,7 +106,7 @@ module Pandemic
106
106
  warn("Unhandled exception in create connection block: #{e.inspect}")
107
107
  end
108
108
  if connection
109
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
109
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
110
110
  connection.write("SERVER #{@server.signature}\n")
111
111
  end
112
112
  connection
@@ -84,7 +84,7 @@ module Pandemic
84
84
 
85
85
  def handle_connection(connection)
86
86
  begin
87
- connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if Socket.constants.include?('TCP_NODELAY')
87
+ connection.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if TCP_NO_DELAY_AVAILABLE
88
88
 
89
89
  identification = connection.gets.strip
90
90
  info("Incoming connection from #{connection.peeraddr.values_at(3,1).join(":")} (#{identification})")
@@ -2,11 +2,11 @@
2
2
 
3
3
  Gem::Specification.new do |s|
4
4
  s.name = %q{pandemic}
5
- s.version = "0.2.1"
5
+ s.version = "0.2.3"
6
6
 
7
7
  s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
8
8
  s.authors = ["Arya Asemanfar"]
9
- s.date = %q{2009-03-26}
9
+ s.date = %q{2009-04-10}
10
10
  s.description = %q{Distribute MapReduce to any of the workers and it will spread, like a pandemic.}
11
11
  s.email = %q{aryaasemanfar@gmail.com}
12
12
  s.extra_rdoc_files = ["lib/pandemic/client_side/cluster_connection.rb", "lib/pandemic/client_side/config.rb", "lib/pandemic/client_side/connection.rb", "lib/pandemic/client_side/connection_proxy.rb", "lib/pandemic/client_side/pandemize.rb", "lib/pandemic/connection_pool.rb", "lib/pandemic/mutex_counter.rb", "lib/pandemic/server_side/client.rb", "lib/pandemic/server_side/config.rb", "lib/pandemic/server_side/handler.rb", "lib/pandemic/server_side/peer.rb", "lib/pandemic/server_side/request.rb", "lib/pandemic/server_side/server.rb", "lib/pandemic/util.rb", "lib/pandemic.rb", "README.markdown"]
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: arya-pandemic
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.1
4
+ version: 0.2.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Arya Asemanfar
@@ -9,7 +9,7 @@ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
11
 
12
- date: 2009-03-26 00:00:00 -07:00
12
+ date: 2009-04-10 00:00:00 -07:00
13
13
  default_executable:
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency