yahns 1.5.0 → 1.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Documentation/design_notes.txt +101 -0
- data/GIT-VERSION-FILE +1 -1
- data/GIT-VERSION-GEN +1 -1
- data/NEWS +32 -0
- data/README +1 -0
- data/extras/proxy_pass.rb +8 -9
- data/lib/yahns/acceptor.rb +4 -1
- data/lib/yahns/client_expire_generic.rb +1 -1
- data/lib/yahns/fdmap.rb +2 -2
- data/lib/yahns/http_client.rb +3 -2
- data/lib/yahns/server.rb +3 -3
- data/lib/yahns/stream_file.rb +4 -2
- data/lib/yahns/version.rb +1 -1
- data/lib/yahns/wbuf.rb +5 -5
- data/lib/yahns/wbuf_common.rb +3 -1
- data/lib/yahns.rb +17 -4
- data/test/server_helper.rb +3 -3
- data/test/test_extras_autoindex.rb +1 -0
- data/test/test_extras_proxy_pass.rb +100 -0
- data/test/test_extras_try_gzip_static.rb +1 -0
- data/test/test_output_buffering.rb +1 -1
- data/test/test_server.rb +39 -0
- data/test/test_ssl.rb +6 -3
- data/test/test_stream_file.rb +2 -2
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ed39f1e1cd058304e952eec64aa936763d1afd90
|
4
|
+
data.tar.gz: 97d35c0cbf569626a2ad5670f558c09736c76d97
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: bf372119bfdf2226d133e1b0ff4a74f51d2f79811901da42f601f4c92f872c13c2a31fbd872eb38b1f7fa2f34db5871037ebc4d1be282fdc14a690664238fe07
|
7
|
+
data.tar.gz: 5e4af052c543e3154ad7c9a5abdd4af95aa9c711a53236e9cd5b74ada45585e7ae4d6b0fbd279dc6fd19422a946aa1e2de5241c0fd5439d25579029e627a8fb7
|
@@ -0,0 +1,101 @@
|
|
1
|
+
event queues in yahns
|
2
|
+
---------------------
|
3
|
+
|
4
|
+
There are currently 2 classes of queues and 2 classes of thread pools
|
5
|
+
in yahns.
|
6
|
+
|
7
|
+
While non-blocking I/O with epoll or kqueue is a cheap way to handle
|
8
|
+
thousands of socket connections, multi-threading is required for many
|
9
|
+
existing APIs, including Rack and standard POSIX filesystem interfaces.
|
10
|
+
|
11
|
+
listen queue + accept() thread pool
|
12
|
+
-----------------------------------
|
13
|
+
|
14
|
+
Like all TCP servers, there is a standard listen queue for every listen
|
15
|
+
socket we have inside the kernel.
|
16
|
+
|
17
|
+
Each listen queue has a dedicated thread pool running _blocking_
|
18
|
+
accept(2) (or accept4(2)) syscall in a loop. We use dedicated threads
|
19
|
+
and blocking accept to benefit from "wake-one" behavior in the Linux
|
20
|
+
kernel. By default, this thread pool only has thread per-process, doing
|
21
|
+
nothing but accepting sockets and injecting into to the event queue
|
22
|
+
(used by epoll or kqueue).
|
23
|
+
|
24
|
+
worker thread pool
|
25
|
+
------------------
|
26
|
+
|
27
|
+
This is where all the interesting application dispatch happens in yahns.
|
28
|
+
epoll(2) (or kqueue(2)) descriptor is the heart of event queue. This
|
29
|
+
design allows clients to migrate between different threads as they
|
30
|
+
become active, preventing head-of-line blocking in traditional designs
|
31
|
+
where a client is pinned to a thread (at the cost of weaker cache
|
32
|
+
locality).
|
33
|
+
|
34
|
+
The critical component for implementing this thread pool is "one-shot"
|
35
|
+
notifications in the epoll and kqueue APIs, allowing them to be used as
|
36
|
+
readiness queues for feeding the thread pool. Used correctly, this
|
37
|
+
allows us to guarantee exclusive access to a client socket without
|
38
|
+
additional locks managed in userspace.
|
39
|
+
|
40
|
+
Idle threads will sit performing epoll_wait (or kqueue) indefinitely
|
41
|
+
until a socket is reported as "ready" by the kernel.
|
42
|
+
|
43
|
+
queue flow
|
44
|
+
----------
|
45
|
+
|
46
|
+
Once a client is accept(2)-ed, it is immediately pushed into the worker
|
47
|
+
thread pool (via EPOLL_CTL_ADD or EV_ADD). This mimics the effect of
|
48
|
+
TCP_DEFER_ACCEPT (in Linux) and the "dataready" accept filter (in
|
49
|
+
FreeBSD) from the perspective of the epoll_wait(2)/kqueue(2) caller.
|
50
|
+
No explicit locking controlled from userspace is necessary.
|
51
|
+
|
52
|
+
TCP_DEFER_ACCEPT/"dataready"/"httpready" themselves are not used as it
|
53
|
+
has some documented and unresolved issues (and adds latency).
|
54
|
+
|
55
|
+
https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/134274
|
56
|
+
http://labs.apnic.net/blabs/?p=57
|
57
|
+
|
58
|
+
Denial-of-Service and head-of-line blocking mitigation
|
59
|
+
------------------------------------------------------
|
60
|
+
|
61
|
+
As mentioned before, traditional uses of multi-threaded event loops may
|
62
|
+
suffer from head-of-line blocking because clients on a busy thread may
|
63
|
+
not be able to migrate to a non-busy thread. In yahns, a client
|
64
|
+
automatically migrates to the next available thread in the worker thread
|
65
|
+
pool.
|
66
|
+
|
67
|
+
yahns can safely yield a client after every HTTP request, forcing the
|
68
|
+
client to be rescheduled (via epoll/kqueue) after any existing clients
|
69
|
+
have completed processing.
|
70
|
+
|
71
|
+
"Yielding" a client is accomplished by re-arming the already "ready"
|
72
|
+
socket by using EPOLL_CTL_MOD (with EPOLLONESHOT) with a one-shot
|
73
|
+
notification requeues the descriptor at the end of the internal epoll
|
74
|
+
ready queue; achieving a similar effect to yielding a thread (via
|
75
|
+
sched_yield or Thread.pass) in a purely multi-threaded design.
|
76
|
+
|
77
|
+
Once the client is yielded, epoll_wait is called again to pull
|
78
|
+
the next client off the ready queue.
|
79
|
+
|
80
|
+
Output buffering notes
|
81
|
+
----------------------
|
82
|
+
|
83
|
+
yahns will not read data from a client socket if there is any outgoing
|
84
|
+
data buffered by yahns. This prevents clients from performing a DoS
|
85
|
+
sending a barrage of requests but not reading them (this should be
|
86
|
+
obvious behavior for any server!).
|
87
|
+
|
88
|
+
If outgoing data cannot fit into the kernel socket buffer, we buffer to
|
89
|
+
the filesystem immediately to avoid putting pressure on malloc (or the
|
90
|
+
Ruby GC). This also allows use of the sendfile(2) syscall to avoid
|
91
|
+
extra copies into the kernel.
|
92
|
+
|
93
|
+
Input buffering notes (for Rack)
|
94
|
+
--------------------------------
|
95
|
+
|
96
|
+
As seen by the famous "Slowloris" example, slow clients can ruin some
|
97
|
+
HTTP servers. By default, yahns will use non-blocking I/O to
|
98
|
+
fully-buffer an HTTP request before allowing the Rack 1.x application
|
99
|
+
dispatch to block a thread. This unfortunately means we double the
|
100
|
+
amount of data copied, but prevents us from being hogged by slow clients
|
101
|
+
due to the synchronous nature of Rack 1.x API for handling uploads.
|
data/GIT-VERSION-FILE
CHANGED
@@ -1 +1 @@
|
|
1
|
-
VERSION = 1.
|
1
|
+
VERSION = 1.6.0
|
data/GIT-VERSION-GEN
CHANGED
data/NEWS
CHANGED
@@ -1,3 +1,35 @@
|
|
1
|
+
yahns 1.6.0 - reduced allocations and bugfixes / 2015-03-09
|
2
|
+
-----------------------------------------------------------
|
3
|
+
|
4
|
+
This release fixes a bug where previously-configured-but-now-removed
|
5
|
+
listeners were inherited across USR2 upgrades are not shutdown
|
6
|
+
immediately in the child.
|
7
|
+
|
8
|
+
There are also minor reductions in allocations which can save a few
|
9
|
+
hundred bytes statically and also whenever write buffering is necessary
|
10
|
+
for large responses.
|
11
|
+
|
12
|
+
Some minor documentation updates improvements in extras, too.
|
13
|
+
|
14
|
+
shortlog of changes since 1.5.0:
|
15
|
+
README: add link to mailing list archives
|
16
|
+
test_ssl: factor out server SSLContext creation
|
17
|
+
doc: add design_notes document
|
18
|
+
reduce File::Stat object allocations
|
19
|
+
update comments about wbuf_close return values
|
20
|
+
wbuf: lazily (re)create temporary file
|
21
|
+
fix compatibility with unicorn.git
|
22
|
+
skip tests requiring String#b on 1.9.3
|
23
|
+
use the monotonic clock under Ruby 2.1+
|
24
|
+
favor Class.new for method-less classes
|
25
|
+
extras/proxy_pass: save memory in String#split arg
|
26
|
+
extras/proxy_pass: do not name unused variable
|
27
|
+
extras/proxy_pass: log exceptions leading to 502
|
28
|
+
extras/proxy_pass: flesh out upload support + tests
|
29
|
+
acceptor: close inherited-but-unneeded sockets
|
30
|
+
|
31
|
+
See the git repository for more: git clone git://yhbt.net/yahns
|
32
|
+
|
1
33
|
yahns 1.5.0 - initial OpenSSL support and bugfixes / 2014-12-21
|
2
34
|
---------------------------------------------------------------
|
3
35
|
|
data/README
CHANGED
@@ -68,6 +68,7 @@ You may subscribe by sending an email to:
|
|
68
68
|
|
69
69
|
You may also subscribe to the public-inbox at git://yhbt.net/yahns-public
|
70
70
|
using ssoma <http://ssoma.public-inbox.org/>
|
71
|
+
Mailing list archives browsable at: http://yhbt.net/yahns-public/
|
71
72
|
|
72
73
|
This README is our homepage, we would rather be working on HTTP servers
|
73
74
|
all day than worrying about the next browser vulnerability because
|
data/extras/proxy_pass.rb
CHANGED
@@ -117,7 +117,7 @@ class ProxyPass # :nodoc:
|
|
117
117
|
case dest
|
118
118
|
when %r{\Ahttp://([^/]+)(/.*)\z}
|
119
119
|
path = $2
|
120
|
-
host, port = $1.split(
|
120
|
+
host, port = $1.split(':')
|
121
121
|
@sockaddr = Socket.sockaddr_in(port || 80, host)
|
122
122
|
|
123
123
|
# methods from Rack::Request we want:
|
@@ -135,12 +135,7 @@ class ProxyPass # :nodoc:
|
|
135
135
|
end
|
136
136
|
|
137
137
|
def call(env)
|
138
|
-
|
139
|
-
when "GET", "HEAD" # OK
|
140
|
-
else
|
141
|
-
return [ 405, [%w(Content-Length 0), %w(Content-Length 0)], [] ]
|
142
|
-
end
|
143
|
-
|
138
|
+
request_method = env['REQUEST_METHOD']
|
144
139
|
req = Rack::Request.new(env)
|
145
140
|
path = @path.gsub(/\$(\w+)/) { req.__send__($1.to_sym) }
|
146
141
|
req = "#{request_method} #{path} HTTP/1.1\r\n" \
|
@@ -170,7 +165,7 @@ class ProxyPass # :nodoc:
|
|
170
165
|
send_body(env["rack.input"], ures, chunked) if chunked || clen
|
171
166
|
|
172
167
|
# wait for the response here
|
173
|
-
|
168
|
+
_, header, body = res = ures.rack
|
174
169
|
|
175
170
|
# don't let the upstream Connection and Keep-Alive headers leak through
|
176
171
|
header.delete_if do |k,_|
|
@@ -189,6 +184,10 @@ class ProxyPass # :nodoc:
|
|
189
184
|
res
|
190
185
|
rescue => e
|
191
186
|
retry if ures && ures.fail_retryable? && request_method != "POST"
|
187
|
+
if defined?(Yahns::Log)
|
188
|
+
logger = env['rack.logger'] and
|
189
|
+
Yahns::Log.exception(logger, 'proxy_pass', e)
|
190
|
+
end
|
192
191
|
ERROR_502
|
193
192
|
end
|
194
193
|
|
@@ -200,7 +199,7 @@ class ProxyPass # :nodoc:
|
|
200
199
|
buf.replace("#{buf.size.to_s(16)}\r\n#{buf}\r\n")
|
201
200
|
ures.req_write(buf, @timeout)
|
202
201
|
end
|
203
|
-
ures.req_write("0\r\n\r\n")
|
202
|
+
ures.req_write("0\r\n\r\n", @timeout)
|
204
203
|
else # common if we hit uploads
|
205
204
|
while input.read(16384, buf)
|
206
205
|
ures.req_write(buf, @timeout)
|
data/lib/yahns/acceptor.rb
CHANGED
@@ -19,7 +19,10 @@ module Yahns::Acceptor # :nodoc:
|
|
19
19
|
|
20
20
|
# just keep looping this on every acceptor until the associated thread dies
|
21
21
|
def ac_quit
|
22
|
-
|
22
|
+
unless defined?(@thrs) # acceptor has not started yet, freshly inherited
|
23
|
+
close
|
24
|
+
return true
|
25
|
+
end
|
23
26
|
@thrs.each { |t| t[:yahns_quit] = true }
|
24
27
|
return true if __ac_quit_done?
|
25
28
|
|
data/lib/yahns/fdmap.rb
CHANGED
@@ -81,7 +81,7 @@ class Yahns::Fdmap # :nodoc:
|
|
81
81
|
def __expire(timeout)
|
82
82
|
return if @count == 0
|
83
83
|
nr = 0
|
84
|
-
now =
|
84
|
+
now = Yahns.now
|
85
85
|
(now - @last_expire) >= 1.0 or return # don't expire too frequently
|
86
86
|
|
87
87
|
# @fdmap_ary may be huge, so always expire a bunch at once to
|
@@ -93,7 +93,7 @@ class Yahns::Fdmap # :nodoc:
|
|
93
93
|
nr += c.yahns_expire(tout)
|
94
94
|
end
|
95
95
|
|
96
|
-
@last_expire =
|
96
|
+
@last_expire = Yahns.now
|
97
97
|
msg = timeout ? "timeout=#{timeout}" : "client_timeout"
|
98
98
|
@logger.info("dropping #{nr} of #@count clients for #{msg}")
|
99
99
|
end
|
data/lib/yahns/http_client.rb
CHANGED
@@ -4,8 +4,9 @@
|
|
4
4
|
class Yahns::HttpClient < Kgio::Socket # :nodoc:
|
5
5
|
NULL_IO = StringIO.new("")
|
6
6
|
|
7
|
-
# FIXME: we shouldn't have this at all
|
8
|
-
Unicorn::HttpParser.keepalive_requests
|
7
|
+
# FIXME: we shouldn't have this at all when we go Unicorn 5-only
|
8
|
+
Unicorn::HttpParser.respond_to?(:keepalive_requests=) and
|
9
|
+
Unicorn::HttpParser.keepalive_requests = 0xffffffff
|
9
10
|
|
10
11
|
include Yahns::HttpResponse
|
11
12
|
QEV_FLAGS = Yahns::Queue::QEV_RD # used by acceptor
|
data/lib/yahns/server.rb
CHANGED
@@ -385,7 +385,7 @@ class Yahns::Server # :nodoc:
|
|
385
385
|
def quit_enter(alive)
|
386
386
|
if alive
|
387
387
|
@logger.info("gracefully exiting shutdown_timeout=#@shutdown_timeout")
|
388
|
-
@shutdown_expire ||=
|
388
|
+
@shutdown_expire ||= Yahns.now + @shutdown_timeout + 1
|
389
389
|
else # drop connections immediately if signaled twice
|
390
390
|
@logger.info("graceful exit aborted, exiting immediately")
|
391
391
|
# we will still call any app-defined at_exit hooks here
|
@@ -416,7 +416,7 @@ class Yahns::Server # :nodoc:
|
|
416
416
|
# response bodies out (e.g. "tail -F") Oh well, have a timeout
|
417
417
|
begin
|
418
418
|
@wthr.delete_if { |t| t.join(0.01) }
|
419
|
-
end while @wthr[0] &&
|
419
|
+
end while @wthr[0] && Yahns.now <= @shutdown_expire
|
420
420
|
|
421
421
|
# cleanup, our job is done
|
422
422
|
@queues.each(&:close).clear
|
@@ -466,7 +466,7 @@ class Yahns::Server # :nodoc:
|
|
466
466
|
|
467
467
|
def dropping(fdmap)
|
468
468
|
if drop_acceptors[0] || fdmap.size > 0
|
469
|
-
timeout = @shutdown_expire <
|
469
|
+
timeout = @shutdown_expire < Yahns.now ? -1 : @shutdown_timeout
|
470
470
|
fdmap.desperate_expire(timeout)
|
471
471
|
true
|
472
472
|
else
|
data/lib/yahns/stream_file.rb
CHANGED
@@ -31,12 +31,14 @@ class Yahns::StreamFile # :nodoc:
|
|
31
31
|
end
|
32
32
|
end
|
33
33
|
@sf_offset = offset
|
34
|
-
@sf_count = count || @tmpio.
|
34
|
+
@sf_count = count || @tmpio.size
|
35
35
|
@wbuf_persist = persist # whether or not we keep the connection alive
|
36
36
|
@body = body
|
37
37
|
end
|
38
38
|
|
39
|
-
# called by last wbuf_flush
|
39
|
+
# called by last wbuf_flush,
|
40
|
+
# returns true / false for persistent/non-persistent connections,
|
41
|
+
# :ignore for hijacked connections
|
40
42
|
def wbuf_close(client)
|
41
43
|
@tmpio.close if NeedClose === @tmpio
|
42
44
|
wbuf_close_common(client)
|
data/lib/yahns/version.rb
CHANGED
@@ -1 +1 @@
|
|
1
|
-
Yahns::VERSION = '1.
|
1
|
+
Yahns::VERSION = '1.6.0' # :nodoc:
|
data/lib/yahns/wbuf.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
# -*- encoding: binary -*-
|
2
|
-
# Copyright (C) 2013
|
2
|
+
# Copyright (C) 2013-2015 all contributors <yahns-public@yhbt.net>
|
3
3
|
# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
|
4
4
|
require_relative 'wbuf_common'
|
5
5
|
|
@@ -31,7 +31,7 @@ class Yahns::Wbuf # :nodoc:
|
|
31
31
|
include Yahns::WbufCommon
|
32
32
|
|
33
33
|
def initialize(body, persist, tmpdir)
|
34
|
-
@tmpio =
|
34
|
+
@tmpio = nil
|
35
35
|
@tmpdir = tmpdir
|
36
36
|
@sf_offset = @sf_count = 0
|
37
37
|
@wbuf_persist = persist # whether or not we keep the connection alive
|
@@ -50,6 +50,7 @@ class Yahns::Wbuf # :nodoc:
|
|
50
50
|
@bypass = false # ugh, continue to buffering to file
|
51
51
|
end while @bypass
|
52
52
|
|
53
|
+
@tmpio ||= Yahns::TmpIO.new(@tmpdir)
|
53
54
|
@sf_count += @tmpio.write(buf)
|
54
55
|
case rv = client.trysendfile(@tmpio, @sf_offset, @sf_count)
|
55
56
|
when Integer
|
@@ -64,16 +65,15 @@ class Yahns::Wbuf # :nodoc:
|
|
64
65
|
|
65
66
|
# we're all caught up, try to prevent dirty data from getting flushed
|
66
67
|
# to disk if we can help it.
|
67
|
-
@tmpio.close
|
68
|
+
@tmpio = @tmpio.close
|
68
69
|
@sf_offset = 0
|
69
|
-
@tmpio = Yahns::TmpIO.new(@tmpdir)
|
70
70
|
@bypass = true
|
71
71
|
nil
|
72
72
|
end
|
73
73
|
|
74
74
|
# called by last wbuf_flush
|
75
75
|
def wbuf_close(client)
|
76
|
-
@tmpio = @tmpio.close
|
76
|
+
@tmpio = @tmpio.close if @tmpio
|
77
77
|
wbuf_close_common(client)
|
78
78
|
end
|
79
79
|
end
|
data/lib/yahns/wbuf_common.rb
CHANGED
@@ -9,7 +9,9 @@ rescue LoadError
|
|
9
9
|
end
|
10
10
|
|
11
11
|
module Yahns::WbufCommon # :nodoc:
|
12
|
-
# returns
|
12
|
+
# returns true / false for persistent/non-persistent connections
|
13
|
+
# returns :wait_*able when blocked
|
14
|
+
# returns :ignore if hijacked
|
13
15
|
# currently, we rely on each thread having exclusive access to the
|
14
16
|
# client socket, so this is never called concurrently with wbuf_write
|
15
17
|
def wbuf_flush(client)
|
data/lib/yahns.rb
CHANGED
@@ -10,7 +10,9 @@ require 'sleepy_penguin'
|
|
10
10
|
# on unicorn at all
|
11
11
|
[ :ClientShutdown, :Const, :SocketHelper, :StreamInput, :TeeInput,
|
12
12
|
:SSLConfigurator, :Configurator, :TmpIO, :Util, :Worker, :SSLServer,
|
13
|
-
:HttpServer ].each
|
13
|
+
:HttpServer ].each do |sym|
|
14
|
+
Unicorn.__send__(:remove_const, sym) if Unicorn.const_defined?(sym)
|
15
|
+
end
|
14
16
|
|
15
17
|
# yahns exposes no user-visible API outside of the config file
|
16
18
|
# Internals are subject to change.
|
@@ -48,10 +50,21 @@ module Yahns # :nodoc:
|
|
48
50
|
# application dispatch. This is always raised with an empty backtrace
|
49
51
|
# since there is nothing in the application stack that is responsible
|
50
52
|
# for client shutdowns/disconnects.
|
51
|
-
|
52
|
-
|
53
|
+
ClientShutdown = Class.new(EOFError) # :nodoc:
|
54
|
+
|
55
|
+
ClientTimeout = Class.new(RuntimeError) # :nodoc:
|
53
56
|
|
54
|
-
|
57
|
+
# try to use the monotonic clock in Ruby >= 2.1, it is immune to clock
|
58
|
+
# offset adjustments and generates less garbage (Float vs Time object)
|
59
|
+
begin
|
60
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
61
|
+
def self.now
|
62
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
63
|
+
end
|
64
|
+
rescue NameError, NoMethodError
|
65
|
+
def self.now # Ruby <= 2.0
|
66
|
+
Time.now.to_f
|
67
|
+
end
|
55
68
|
end
|
56
69
|
end
|
57
70
|
|
data/test/server_helper.rb
CHANGED
@@ -72,10 +72,10 @@ module ServerHelper
|
|
72
72
|
@ru = nil
|
73
73
|
end
|
74
74
|
|
75
|
-
def mkserver(cfg)
|
75
|
+
def mkserver(cfg, srv = @srv)
|
76
76
|
fork do
|
77
|
-
ENV["YAHNS_FD"] =
|
78
|
-
|
77
|
+
ENV["YAHNS_FD"] = srv.fileno.to_s
|
78
|
+
srv.autoclose = false
|
79
79
|
yield if block_given?
|
80
80
|
Yahns::Server.new(cfg).start.join
|
81
81
|
end
|
@@ -0,0 +1,100 @@
|
|
1
|
+
# Copyright (C) 2015 all contributors <yahns-public@yhbt.net>
|
2
|
+
# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
|
3
|
+
require_relative 'server_helper'
|
4
|
+
|
5
|
+
class TestExtrasProxyPass < Testcase
|
6
|
+
ENV["N"].to_i > 1 and parallelize_me!
|
7
|
+
include ServerHelper
|
8
|
+
|
9
|
+
class ProxiedApp
|
10
|
+
def call(env)
|
11
|
+
h = [ %w(Content-Length 3), %w(Content-Type text/plain) ]
|
12
|
+
case env['REQUEST_METHOD']
|
13
|
+
when 'GET'
|
14
|
+
[ 200, h, [ "hi\n"] ]
|
15
|
+
when 'HEAD'
|
16
|
+
[ 200, h, [] ]
|
17
|
+
when 'PUT'
|
18
|
+
buf = env['rack.input'].read
|
19
|
+
[ 201, {
|
20
|
+
'Content-Length' => buf.bytesize.to_s,
|
21
|
+
'Content-Type' => 'text/plain',
|
22
|
+
}, [ buf ] ]
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
27
|
+
def setup
|
28
|
+
@srv2 = TCPServer.new(ENV["TEST_HOST"] || "127.0.0.1", 0)
|
29
|
+
server_helper_setup
|
30
|
+
end
|
31
|
+
|
32
|
+
def teardown
|
33
|
+
@srv2.close if defined?(@srv2) && !@srv2.closed?
|
34
|
+
server_helper_teardown
|
35
|
+
end
|
36
|
+
|
37
|
+
def test_proxy_pass
|
38
|
+
err, cfg, host, port = @err, Yahns::Config.new, @srv.addr[3], @srv.addr[1]
|
39
|
+
host2, port2 = @srv2.addr[3], @srv2.addr[1]
|
40
|
+
pid = mkserver(cfg) do
|
41
|
+
$LOAD_PATH.unshift "#{Dir.pwd}/extras"
|
42
|
+
require 'proxy_pass'
|
43
|
+
@srv2.close
|
44
|
+
cfg.instance_eval do
|
45
|
+
app(:rack, ProxyPass.new("http://#{host2}:#{port2}/")) do
|
46
|
+
listen "#{host}:#{port}"
|
47
|
+
end
|
48
|
+
stderr_path err.path
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
pid2 = mkserver(cfg, @srv2) do
|
53
|
+
@srv.close
|
54
|
+
cfg.instance_eval do
|
55
|
+
app(:rack, ProxiedApp.new) do
|
56
|
+
listen "#{host2}:#{port2}"
|
57
|
+
end
|
58
|
+
stderr_path err.path
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
gplv3 = File.open('COPYING')
|
63
|
+
|
64
|
+
Net::HTTP.start(host, port) do |http|
|
65
|
+
res = http.request(Net::HTTP::Get.new('/'))
|
66
|
+
assert_equal 200, res.code.to_i
|
67
|
+
n = res.body.bytesize
|
68
|
+
assert_operator n, :>, 1
|
69
|
+
res = http.request(Net::HTTP::Head.new('/'))
|
70
|
+
assert_equal 200, res.code.to_i
|
71
|
+
assert_equal n, res['Content-Length'].to_i
|
72
|
+
assert_nil res.body
|
73
|
+
|
74
|
+
# chunked encoding
|
75
|
+
req = Net::HTTP::Put.new('/')
|
76
|
+
req.body_stream = gplv3
|
77
|
+
req.content_type = 'application/octet-stream'
|
78
|
+
req['Transfer-Encoding'] = 'chunked'
|
79
|
+
res = http.request(req)
|
80
|
+
gplv3.rewind
|
81
|
+
assert_equal gplv3.read, res.body
|
82
|
+
assert_equal 201, res.code.to_i
|
83
|
+
|
84
|
+
# normal content-length
|
85
|
+
gplv3.rewind
|
86
|
+
req = Net::HTTP::Put.new('/')
|
87
|
+
req.body_stream = gplv3
|
88
|
+
req.content_type = 'application/octet-stream'
|
89
|
+
req.content_length = gplv3.size
|
90
|
+
res = http.request(req)
|
91
|
+
gplv3.rewind
|
92
|
+
assert_equal gplv3.read, res.body
|
93
|
+
assert_equal 201, res.code.to_i
|
94
|
+
end
|
95
|
+
ensure
|
96
|
+
gplv3.close if gplv3
|
97
|
+
quit_wait pid
|
98
|
+
quit_wait pid2
|
99
|
+
end
|
100
|
+
end
|
data/test/test_server.rb
CHANGED
@@ -804,4 +804,43 @@ class TestServer < Testcase
|
|
804
804
|
ensure
|
805
805
|
quit_wait(pid)
|
806
806
|
end
|
807
|
+
|
808
|
+
def test_inherit_too_many
|
809
|
+
err = @err
|
810
|
+
s2 = TCPServer.new(ENV["TEST_HOST"] || "127.0.0.1", 0)
|
811
|
+
cfg = Yahns::Config.new
|
812
|
+
host, port = @srv.addr[3], @srv.addr[1]
|
813
|
+
cfg.instance_eval do
|
814
|
+
ru = lambda { |_| [ 200, {'Content-Length'=>'2'}, ['HI'] ] }
|
815
|
+
GTL.synchronize { app(:rack, ru) { listen "#{host}:#{port}" } }
|
816
|
+
logger(Logger.new(err.path))
|
817
|
+
end
|
818
|
+
mkserver(cfg, @srv) do
|
819
|
+
s2.autoclose = false
|
820
|
+
ENV["YAHNS_FD"] = "#{@srv.fileno},#{s2.fileno}"
|
821
|
+
end
|
822
|
+
run_client(host, port) { |res| assert_equal "HI", res.body }
|
823
|
+
th = Thread.new do
|
824
|
+
c = s2.accept
|
825
|
+
c.readpartial(1234)
|
826
|
+
c.write "HTTP/1.0 666 OK\r\n\r\nGO AWAY"
|
827
|
+
c.close
|
828
|
+
:OK
|
829
|
+
end
|
830
|
+
Thread.pass
|
831
|
+
s2host, s2port = s2.addr[3], s2.addr[1]
|
832
|
+
Net::HTTP.start(s2host, s2port) do |http|
|
833
|
+
res = http.request(Net::HTTP::Get.new("/"))
|
834
|
+
assert_equal 666, res.code.to_i
|
835
|
+
assert_equal "GO AWAY", res.body
|
836
|
+
end
|
837
|
+
assert_equal :OK, th.value
|
838
|
+
tmpc = TCPSocket.new(s2host, s2port)
|
839
|
+
a2 = s2.accept
|
840
|
+
assert_nil IO.select([a2], nil, nil, 0.05)
|
841
|
+
tmpc.close
|
842
|
+
assert_nil a2.read(1)
|
843
|
+
a2.close
|
844
|
+
s2.close
|
845
|
+
end
|
807
846
|
end
|
data/test/test_ssl.rb
CHANGED
@@ -53,13 +53,16 @@ AQjjxMXhwULlmuR/K+WwlaZPiLIBYalLAZQ7ZbOPeVkJ8ePao0eLAgEC
|
|
53
53
|
ssl
|
54
54
|
end
|
55
55
|
|
56
|
-
def
|
57
|
-
err, cfg, host, port = @err, Yahns::Config.new, @srv.addr[3], @srv.addr[1]
|
58
|
-
host, port = @srv.addr[3], @srv.addr[1]
|
56
|
+
def srv_ctx
|
59
57
|
ctx = OpenSSL::SSL::SSLContext.new
|
60
58
|
ctx.ciphers = "ADH"
|
61
59
|
ctx.tmp_dh_callback = proc { TEST_KEY_DH1024 }
|
60
|
+
ctx
|
61
|
+
end
|
62
62
|
|
63
|
+
def test_ssl_basic
|
64
|
+
err, cfg, host, port = @err, Yahns::Config.new, @srv.addr[3], @srv.addr[1]
|
65
|
+
ctx = srv_ctx
|
63
66
|
pid = mkserver(cfg) do
|
64
67
|
cfg.instance_eval do
|
65
68
|
ru = lambda { |_| [ 200, {'Content-Length'=>'2'}, ['HI'] ] }
|
data/test/test_stream_file.rb
CHANGED
@@ -9,7 +9,7 @@ class TestStreamFile < Testcase
|
|
9
9
|
|
10
10
|
def test_stream_file
|
11
11
|
fp = File.open("COPYING")
|
12
|
-
sf = Yahns::StreamFile.new(fp, true, 0, fp.
|
12
|
+
sf = Yahns::StreamFile.new(fp, true, 0, fp.size)
|
13
13
|
refute sf.respond_to?(:close)
|
14
14
|
sf.wbuf_close(nil)
|
15
15
|
assert fp.closed?
|
@@ -18,7 +18,7 @@ class TestStreamFile < Testcase
|
|
18
18
|
def test_fd
|
19
19
|
fp = File.open("COPYING")
|
20
20
|
obj = DevFD.new("/dev/fd/#{fp.fileno}")
|
21
|
-
sf = Yahns::StreamFile.new(obj, true, 0, fp.
|
21
|
+
sf = Yahns::StreamFile.new(obj, true, 0, fp.size)
|
22
22
|
io = sf.instance_variable_get :@tmpio
|
23
23
|
assert_instance_of IO, io.to_io
|
24
24
|
assert_equal fp.fileno, io.fileno
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: yahns
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.6.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- yahns hackers
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-03-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: kgio
|
@@ -96,6 +96,7 @@ files:
|
|
96
96
|
- COPYING
|
97
97
|
- Documentation/.gitignore
|
98
98
|
- Documentation/GNUmakefile
|
99
|
+
- Documentation/design_notes.txt
|
99
100
|
- Documentation/yahns-rackup.txt
|
100
101
|
- Documentation/yahns.txt
|
101
102
|
- Documentation/yahns_config.txt
|
@@ -178,6 +179,7 @@ files:
|
|
178
179
|
- test/test_extras_autoindex.rb
|
179
180
|
- test/test_extras_exec_cgi.rb
|
180
181
|
- test/test_extras_exec_cgi.sh
|
182
|
+
- test/test_extras_proxy_pass.rb
|
181
183
|
- test/test_extras_try_gzip_static.rb
|
182
184
|
- test/test_fdmap.rb
|
183
185
|
- test/test_input.rb
|