iodine 0.1.4 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1b7b0693d39f0937c97786d6070a972dfdadbefb
-  data.tar.gz: 9392682f01bd3f81f77ce6f4397aeef61df33ad5
+  metadata.gz: 3a9f87061376cb31f4e3d6b1cc7fd9dcca401151
+  data.tar.gz: 2cde195704434f6ff3af7b4a98a35c3b06f637af
 SHA512:
-  metadata.gz: c3cf5a23ae4547f126d11523a8926f974f602bf033a0e34012bd342559ada37701e5b6beb91e26c5a044083621d53e0187c87bd49ba66d2edb180642b8cc3a76
-  data.tar.gz: e9e14352d3460fdae6d1823b8e5f8a45a61dbb2b28cb0cf92e8f4eadb95000f3d2bb9a5d742b0312371003cde1fa698b202b1abcc2c6ede3b68e511e677c0a9e
+  metadata.gz: 0bb1203ab2d5d5ab58ec4b6dd97146b0328802db7b426031319a5e8be6936826811e43a86ae6326bbe86f0811a3013656d15a56fdfb0f29a1d0046d6bf9b485a
+  data.tar.gz: 773e868701784aca9b5a5899f0d027cafab8e586f50acf43a40f2efb51252350d689f0af6bc90caeff6f79a6a1e4a0d0bc5a683ca4b0003dde1be3182326d866

data/CHANGELOG.md

@@ -8,6 +8,14 @@ Please notice that this change log contains changes for upcoming releases as wel
 
 ***
 
+Change log v.0.1.5
+
+**Feature**: The Response#body can now be set to a File object, allowing Iodine to preserve memory when serving large static files from disc. Limited Range requests are also supported - together, these changes allow Iodine to serve media files (such as movies) while suffering a smaller memory penalty and supporting a wider variaty of players (Safari requires Range request support for it's media player).
+
+**Fix**: Fixed an issue where Iodine might take a long time to shut down after a Fatal Error during the server initialization.
+
+***
+
 Change log v.0.1.4
 
 **Fix**: fixed an issue with where the WebsocketClient#on_close wouldn't be called for a renewable Websocket connection during shutdown.

data/README.md

@@ -57,7 +57,7 @@ exit
 
 In this mode, Iodine will continue running until all the tasks have completed and than it will quite. Timer based tasks will be ignored.
 
-## Simple Usage: Task polling (unreleased version)
+## Simple Usage: Task polling
 
 This mode of operation is effective if want Iodine to periodically initiates new tasks, for instance if you cannot use `cron`.
 
@@ -129,7 +129,7 @@ Iodine::Http.on_websocket WSChatServer
 
 ### Security and limits
 
-Nobody wants their server to crash... Security measures are a fact of life as an internet entety. It is not only the theoretical malicious attacker from which a server must protect itself, but also from the unaware user or client.
+Nobody wants their server to crash... Security measures are a fact of life as an internet entity. It is not only the theoretical malicious attacker from which a server must protect itself, but also from the unaware user or client.
 
 Mostly, it is assumed that Iodine will run behind a proxy (i.e. within a Heroku Dyno or viaduct.io process), as such it is assumed that the proxy will protect the Iodine Http server from undue stress.
 
@@ -137,13 +137,13 @@ Having said that, Iodine is built with certain security measures in mind:
 
 - Iodine will not accept IO data (neither from new connections nor form existing ones) while still answering existing requests and performing tasks. This safeguards against task overloading and DoS attacks causing a global crash, allowing the server to resume normal operation once a DoS attack had run it's course (and potentially allowing legitimate requests to be answered while the attack is still underway).
 
-- Iodine will limit the query length (Http/1), header count and header data size as well as well as react to header overloading by immediate disconnections. Iodine's limits are hardcoded to be slightly more than double those of the common Proxies, so this counter-measure will only take effect should an attacker manage to bypass the Proxy.
+- Iodine will limit the query length, header count and header data size as well as well as react to header overloading by immediate disconnections. Iodine's limits are hardcoded to be slightly more than double those of common Proxies, so this counter-measure will only take effect should an attacker manage to bypass the Proxy.
 
 - Iodine limits every Http request body-size (file upload data, form data, etc') to ~0.5GB. This setting can be changed using `Iodine::Http.max_http_buffer`. This safeguard is meant to prevent Ruby from crashing due to insufficient memory (an error Iodine cannot, and should not, recover from).
 
    It is recommended that this number will be lowered substantially whenever possible, by using `Iodine::Http.max_http_buffer = new_value`
 
-   Do be aware that, at the moment, file uploads are passed through the memory when parsed. The parser's memory consumption will hopefully decrese in future releases, however, it is always recomended that large data be avoided when possible or handled using download/upload management protocols and services.
+   Do be aware that, at the moment, file upload data must passed through the memory on it's way to the temporary file. The parser's memory consumption will hopefully decrese in future releases, however, it is always recomended that large data be avoided when possible or handled using download/upload management protocols and services.
 
 ## Server Usage: Plug in your network protocol
 

data/bin/hello_world

@@ -7,6 +7,10 @@ Dir.chdir Root.join('..').to_s
 require "bundler/setup"
 require "iodine/http"
 
+
+# ~/ruby/wrk/wrk -c400 -d10 -t12 http://localhost:3000/
+
+
 # Iodine.ssl = true
 # Iodine.treads = 8
 # Iodine.protocol.on_http do |req, res|

data/lib/iodine.rb

@@ -39,10 +39,19 @@ require 'openssl'
 #
 #       # setting up the server is as easy as plugging in your Protocol class:
 #       Iodine.protocol = MyProtocol
+#
+#       # setting up cuncurrency can be done with threads - the default is a single thread):
+#       Iodine.threads = 8
+#
+#       # setting up cuncurrency can also be done with processes (forking) - the default is a single process:
+#       Iodine.processes = 4
 #       
 #       # if you are excecuting this script from IRB, exit IRB to start Iodine.
 #       exit
 #
+# You can test the example using `telnet`
+#
+# Before you write your own protocol, make sure you learn more about the {Iodine::Protocol} and all it's got to offer.
 module Iodine
   extend self
 end

data/lib/iodine/client.rb

@@ -0,0 +1,12 @@
+require 'iodine/http'
+
+module Iodine
+	# # Connect to a remote server or network socket
+	# # this method is defined under the `client` API.
+	# def connect url, protocol, ssl=false
+		
+	# end
+end
+
+Iodine.protocol = :client
+

data/lib/iodine/http/http1.rb

@@ -95,11 +95,11 @@ module Iodine
 			def send_response response
 				return false if response.headers.frozen?
 
+				body = response.extract_body
 				request = response.request
 				headers = response.headers
-				body = response.extract_body
 
-				headers['content-length'.freeze] ||= body.to_s.bytesize
+				headers['content-length'.freeze] ||= body.size if body
 
 				keep_alive = response.keep_alive
 				if (request[:version].to_f > 1 && request['connection'.freeze].nil?) || request['connection'.freeze].to_s =~ /ke/i || (headers['connection'.freeze] && headers['connection'.freeze] =~ /^ke/i)
@@ -111,13 +111,13 @@ module Iodine
 				end
 
 				send_headers response
-				return log_finished(response) if request.head?
-				if body
-					written = write(body)
-					return Iodine.warn "Http/1 couldn't send response because connection was lost." unless written
+				return log_finished(response) && (body && body.close) if request.head? || body.nil?
+				until body.eof?
+					written = write(body.read 65_536)
+					return Iodine.warn("Http/1 couldn't send response because connection was lost.") && body.close unless written
 					response.bytes_written += written
-					(body.frozen? || body.clear)
 				end
+				body.close
 				close unless keep_alive
 				log_finished response
 			end
@@ -130,17 +130,17 @@ module Iodine
 				end
 				return if response.request.head?
 				body = response.extract_body
-				if body
-					written = stream_data(body)
-					return Iodine.warn "Http/1 couldn't send response because connection was lost." unless written
+				until body.eof?
+					written = write(body.read 65_536)
+					return Iodine.warn("Http/1 couldn't send response because connection was lost.") && body.close unless written
 					response.bytes_written += written
-				end
+				end if body
 				if finish
 					response.bytes_written += stream_data('')
 					log_finished response
 					close unless response.keep_alive
 				end
-				(body.frozen? || body.clear) if body
+				body.close if body
 				true
 			end
 

data/lib/iodine/http/http2.rb

@@ -1,18 +1,6 @@
 module Iodine
 	module Http
 		class Http2 < ::Iodine::Protocol
-			def initialize io, original_request = nil
-				super(io)
-				return unless original_request
-				::Iodine.warn "Http/2: upgrade handshake settings not implemented. upgrade request:\n#{original_request}"
-				@last_stream = original_request[:stream_id] = 1
-				original_request[:io] = self
-				# deal with the request['http2-settings'] - NO ACK
-				# HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>
-
-				# dispatch the original request
-				::Iodine.run original_request, &(::Iodine::Http::Http2.dispatch)
-			end
 			def on_open
 				# not fully fanctional.
 				::Iodine.warn "Http/2 requested - support is still experimental."
@@ -52,6 +40,18 @@ module Iodine
 				# 0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a
 				# == PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n
 				# + SETTINGS frame
+
+				# The @options variable contains the original Http1 request, if exists.
+				return unless @options
+				::Iodine.warn "Http/2: upgrade handshake settings not implemented. upgrade request:\n#{@options}"
+				@last_stream = @options[:stream_id] = 1
+				@options[:io] = self
+				# deal with the request['http2-settings'] - NO ACK
+				# HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>
+
+				# dispatch the original request
+				::Iodine.run @options, &(::Iodine::Http::Http2.dispatch)
+				@options = nil
 			end
 			def on_message data
 				data = ::StringIO.new data
@@ -61,21 +61,35 @@ module Iodine
 			end
 
 			def send_response response
-				request = response.request
-				return false unless send_headers response, request
-				return nil if request.head?
+				return false if response.headers.frozen?
 				body = response.extract_body
-				return (log_finished(response) && body.clear) if request.head?
-				(response.bytes_written += emit_payload(body.to_s, request[:sid], 0, 1) ) && (body.frozen? || body.clear) if body
+				request = response.request
+				return body && body.close unless send_headers response, request
+				return log_finished(response) && body && body.close if request.head?
+				if body
+					until body.eof?
+						response.bytes_written += emit_payload(body.read(@settings[SETTINGS_MAX_FRAME_SIZE]), request[:sid], 0, (body.eof? ? 1 : 0))
+					end
+					body.close
+				else
+					emit_payload('', request[:sid], 0, 1)
+				end
 				log_finished response
 			end
+
 			def stream_response response, finish = false
 				request = response.request
-				send_headers response, request
-				return nil if request.head?
 				body = response.extract_body
-				# puts "should stream #{body}"
-				(response.bytes_written += emit_payload body.to_s, request[:sid], 0, (finish ? 1 : 0) ) && (body && !body.frozen? && body.clear) if body || finish
+				send_headers response, request
+				return body && body.close if request.head?
+				if body
+					until body.eof?
+						response.bytes_written += emit_payload(body.read(@settings[SETTINGS_MAX_FRAME_SIZE]), request[:sid], 0, ((finish && body.eof?) ? 1 : 0))
+					end
+					body.close
+				elsif finish
+					emit_payload('', request[:sid], 0, 1)
+				end
 				log_finished response if finish
 			end
 
@@ -132,8 +146,8 @@ module Iodine
 			end
 
 			def send_headers response, request
+				return false if response.headers.frozen?
 				headers = response.headers
-				return false if headers.frozen?
 				# headers[:status] = response.status.to_s
 				headers['set-cookie'] = response.extract_cookies
 				headers.freeze

data/lib/iodine/http/response.rb

@@ -307,27 +307,54 @@ module Iodine
 				511=>"Network Authentication Required".freeze
 			}
 
-			# This will return the Body object as a String... And set the body to `nil` (seeing as it was extracted from the response).
+			# This will return the Body object as an IO like object, such as StringIO (or File)... And set the body to `nil` (seeing as it was extracted from the response).
+			#
+			# This method will also attempts to set headers and update the response status in relation to the body, if applicable. Call this BEFORE getting any final data about the response or sending the headers.
 			def extract_body
-				if @body.is_a?(Array)
+				body_io = if @body.is_a?(Array)
 					return (@body = nil) if body.empty?
-					@body = @body.join
-					extract_body
+					StringIO.new @body.join
 				elsif @body.is_a?(String)
 					return (@body = nil) if body.empty?
-					tmp = @body
-					@body = nil
-					tmp
+					StringIO.new @body
 				elsif body.nil?
 					nil
-				elsif body.respond_to? :each
+				elsif  @body.is_a?(File) || @body.is_a?(Tempfile) || @body.is_a?(StringIO)
+					@body
+				elsif @body.respond_to? :each
 					tmp = ''
-					body.each {|s| tmp << s}
-					body.close if body.respond_to? :close
+					@body.each {|s| tmp << s}
+					@body.close if @body.respond_to? :close
 					@body = nil
 					return nil if tmp.empty? 
-					tmp
+					StringIO.new tmp
 				end
+				@body = nil
+				body_io.rewind
+
+				if !(@headers.frozen?) && @request['range'.freeze] && @request.get? && @status == 200 && @headers['content-length'.freeze].nil?
+					r = @request['range'.freeze].match(/^bytes=([\d]+)\-([\d]+)?$/i)
+					if r
+						old_size = body_io.size
+						start_pos = r[1].to_i
+						end_pos = (r[2] || (old_size - 1)).to_i
+						read_length = end_pos-start_pos+ 1
+						@status = 206 unless old_size == read_length
+						body_io.pos = start_pos
+						unless end_pos == old_size-1
+							new_body = body_io.read(read_length)
+							body_io.close
+							body_io = StringIO.new new_body
+							body_io.rewind
+						end
+						@headers['content-range'.freeze] = "bytes #{start_pos}-#{end_pos}/#{old_size}"
+						@headers['accept-ranges'.freeze] ||= 'bytes'
+					else
+						@headers['accept-ranges'.freeze] ||= 'none'
+					end
+				end
+
+				body_io
 			end
 
 			# This will return an array of cookie settings to be appended to `set-cookie` headers.

data/lib/iodine/http/websocket_client.rb

@@ -258,7 +258,7 @@ module Iodine
 
 				request[:ws_client_params] = options
 				client = self.new(request)
-				Iodine::Http::Websockets.new( ( ssl || socket), client, request )
+				Iodine::Http::Websockets.new( ( ssl || socket), handler: client, request: request )
 
 				return client	
 

data/lib/iodine/http/websocket_handler.rb

@@ -28,9 +28,16 @@ module Iodine
 			# cleanup, if needed, using this callback.
 			def on_close
 			end
+			# extra cleanup, if needed, when server is shutting down while the websocket is connected.
+			#
+			# You can use on_close unless some "going away" cleanup is required.
+			def on_shutdown
+			end
 
 			# This method allows the class itself to act as the Websocket handler, usable with:
-			#        Iodine::Http.on_websocket Iodine::Http::WebsocketEchoDemo
+			#
+			#           # Iodine::Http::WebsocketHandler's default implementation does nothing.
+			#           Iodine::Http.on_websocket Iodine::Http::WebsocketHandler
 			def self.call request, response
 				self.new request, response
 			end
@@ -43,7 +50,11 @@ module Iodine
 			def write data
 				# We leverage the fact that the Http response can be used to send Websocket data.
 				#
-				# you can also use Websocket#send_data or it's alias Websocket#<<
+				# you can also use Websocket#send_data or it's alias Websocket#<< for example:
+				#
+				#        # @request[:io] contains the Websockets Protocol instance
+				#        @request[:io] << data
+				#
 				# do NOT use Websocket#write (which writes the data directly, bypassing the protocol).
 				@response << data
 			end
@@ -53,18 +64,18 @@ module Iodine
 			# This implementation is limited to a single process on a single server.
 			# Consider using Redis for a scalable implementation.
 			def unicast id, data
-				# @request[:io] contains the Websocket Protocol
-				@request[:io].unicast id, data
+				::Iodine::Http::Websockets.unicast id, data
 			end
 			# Broadcast to all Websockets, except self.
 			#
 			# This implementation is limited to a single process on a single server.
 			# Consider using Redis for a scalable implementation.
 			def broadcast data
-				@request[:io].broadcast data
+				::Iodine::Http::Websockets.broadcast data, self
 			end
 			# Closes the connection
 			def close
+				# @request[:io] contains the Websockets Protocol instance
 				@request[:io].go_away
 			end
 		end

data/lib/iodine/http/websockets.rb

@@ -1,15 +1,11 @@
 module Iodine
 	module Http
 		class Websockets < ::Iodine::Protocol
-			# initialize the websocket protocol.
-			def initialize io, handler, request, ws_extentions = nil
-				@handler = handler
-				@ws_extentions = ws_extentions
-				request[:io] = self
-				super(io)
-			end
 			# continue to initialize the websocket protocol.
 			def on_open
+				@handler = @options[:handler]
+				@ws_extentions = @options[:ext]
+				@options[:request][:io] = self
 				@parser = {body: '', stage: 0, step: 0, mask_key: [], len_bytes: []}
 				set_timeout = self.class.default_timeout
 				@handler.on_open if @handler.respond_to? :on_open
@@ -46,7 +42,7 @@ module Iodine
 			# allow Http responses to be used for sending Websocket data.
 			def send_response response, finish = false
 				body = response.extract_body
-				send_data body
+				send_data body.read
 			end
 			alias :stream_response :send_response
 
@@ -164,7 +160,7 @@ module Iodine
 				response.session
 				# Iodine.log "#{@request[:client_ip]} [#{Time.now.utc}] - #{@connection.object_id} Upgraded HTTP to WebSockets.\n"
 				response.finish
-				self.new(io.io, handler, request, ws_extentions)
+				self.new(io.io, handler: handler, request: request, ext: ws_extentions)
 				return true
 			end
 

data/lib/iodine/io.rb

@@ -106,6 +106,7 @@ module Iodine
 			rescue => e
 				fatal e.message
 				fatal "Running existing tasks and exiting."
+				@queue << REACTOR
 				Process.kill("INT", 0)
 				next
 			end

data/lib/iodine/protocol.rb

@@ -4,6 +4,26 @@ module Iodine
 	#
 	# A new protocol instance will be created for every network connection.
 	#
+	# A new protocol might be initialized also when switching between protocols. In this use-case, the
+	# protocol can be initialized with an optional second `options` parameter (the first parameter MUST be the IO object used),
+	# allowing this data to be accessed within the {#on_open} method using the `@options` instance variable or the `options` accessor.
+	#
+	# For example, when switching protocols midstream (i.e. for implementing an Http Upgrade to another protocol such as Websockets):
+	# 
+	#      class MyNextProtocol
+	#         def on_open
+	#            @secret = options[:secret]
+	#         end
+	#      end
+	#
+	#      # in the old protocol:
+	#
+	#      class MyOriginalProtocol
+	#         def switch_to_next_protocol
+	#            MyNextProtocol.new @io, secret: "my secret data"
+	#         end
+	#      end
+	#
 	# The recommended use is to inherit this class and override any of the following:
 	# on_open:: called whenever the Protocol is initialized. Override this to initialize the Protocol object.
 	# on_message(data):: called whenever data is received from the IO. Override this to implement the actual network protocol.
@@ -24,6 +44,9 @@ module Iodine
 		#
 		# Using one of the Protocol methods {#write}, {#read}, {#close} is prefered over direct access.
 		attr_reader :io
+		# the argument or options Hash passed to the initializer as a second argument (the first argument MUST be the IO object).
+		# the value is usually `nil` unless the protocol instance was created by a different protocol while "upgrading" from one protocol to the next.
+		attr_reader :options
 
 		# Sets the timeout in seconds for IO activity (set timeout within {#on_open}).
 		#
@@ -131,11 +154,12 @@ module Iodine
 		# A new Protocol instance set itself up as the IO's protocol (replacing any previous protocol).
 		#
 		# Normally you won't need to override this method. Override {#on_open} instead.
-		def initialize io
+		def initialize io, options = nil
 			@timeout ||= nil
 			@send_locker = Mutex.new
 			@locker = Mutex.new
 			@io = io
+			@options = options
 			touch
 			@locker.synchronize do
 				Iodine.switch_protocol @io.to_io, self

data/lib/iodine/version.rb

@@ -1,3 +1,3 @@
 module Iodine
-  VERSION = "0.1.4"
+  VERSION = "0.1.5"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: iodine
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.1.5
 platform: ruby
 authors:
 - Boaz Segev
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2015-10-25 00:00:00.000000000 Z
+date: 2015-10-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -74,6 +74,7 @@ files:
 - bin/setup
 - iodine.gemspec
 - lib/iodine.rb
+- lib/iodine/client.rb
 - lib/iodine/core.rb
 - lib/iodine/http.rb
 - lib/iodine/http/hpack.rb