by 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG +3 -0
- data/MIT-LICENSE +18 -0
- data/README.rdoc +284 -0
- data/bin/by +38 -0
- data/bin/by-server +6 -0
- data/bin/by-session +14 -0
- data/lib/by/server.rb +211 -0
- data/lib/by/worker.rb +164 -0
- metadata +104 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: '078f15faf5964b6b3bb4ac860c7733e069c0a8cb46556e9af7e555f8c3808d54'
|
4
|
+
data.tar.gz: e878d37b2417d4d655f138d3e2bae5bf51a76c34b7a3fab2956bc40c88a77dbb
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 128a3c27c4dfe729b62fa15d9b961f7384a75ec02030403ae85b376981b731c8d892305732e519cbfb84ab105b816275e6a910afff7f9187c2f723d7683eb12d
|
7
|
+
data.tar.gz: 61895959cd9a6097ded3430017d0a831affa73c668c8301579dd4a62f661a6c1fd3bb09871d5eb321ce9d97fb7a64c573fc8b92a60ae34b7adfcad835eb0ae08
|
data/CHANGELOG
ADDED
data/MIT-LICENSE
ADDED
@@ -0,0 +1,18 @@
|
|
1
|
+
Copyright (c) 2023 Jeremy Evans
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
of this software and associated documentation files (the "Software"), to
|
5
|
+
deal in the Software without restriction, including without limitation the
|
6
|
+
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
|
7
|
+
sell copies of the Software, and to permit persons to whom the Software is
|
8
|
+
furnished to do so, subject to the following conditions:
|
9
|
+
|
10
|
+
The above copyright notice and this permission notice shall be included in
|
11
|
+
all copies or substantial portions of the Software.
|
12
|
+
|
13
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
16
|
+
THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.rdoc
ADDED
@@ -0,0 +1,284 @@
|
|
1
|
+
= by
|
2
|
+
|
3
|
+
by is a library preloader for Ruby designed to speed up process startup.
|
4
|
+
It uses a client/server approach, where the server loads the libraries and
|
5
|
+
listens on a UNIX socket, and the client connects to that socket to run
|
6
|
+
a process. For each client connection, the server forks a worker process,
|
7
|
+
which uses the current directory, stdin, stdout, stderr, and environment
|
8
|
+
of the client process. The worker process then processes the arguments
|
9
|
+
provided by the client. The client process waits until the worker process
|
10
|
+
returns an exit code and closes the socket, and uses exit code 0 (normal
|
11
|
+
exit) if the worker process indicates success, or exit code 1 (error)
|
12
|
+
if the worker process indicates an error.
|
13
|
+
|
14
|
+
== Installation
|
15
|
+
|
16
|
+
gem install by
|
17
|
+
|
18
|
+
== Source Code
|
19
|
+
|
20
|
+
Source code is available on GitHub at https://github.com/jeremyevans/by
|
21
|
+
|
22
|
+
== Usage
|
23
|
+
|
24
|
+
To use +by+, you first start <tt>by-server</tt>, passing in libraries you would
|
25
|
+
like to preload.
|
26
|
+
|
27
|
+
$ by-server sequel roda capybara
|
28
|
+
|
29
|
+
Then you can run ruby with the libraries preloaded using +by+:
|
30
|
+
|
31
|
+
$ by -e 'p [Sequel, Roda, Capybara]'
|
32
|
+
[Sequel, Roda, Capybara]
|
33
|
+
|
34
|
+
The advantage of using +by+ is that the libraries are already loaded,
|
35
|
+
so Ruby doesn't have to find the libraries and parse the files in each
|
36
|
+
library on process startup. Here's a performance comparison:
|
37
|
+
|
38
|
+
$ /usr/bin/time ruby -e 'require "sequel"; require "roda"; require "capybara"'
|
39
|
+
1.67 real 0.93 user 0.66 sys
|
40
|
+
|
41
|
+
$ /usr/bin/time by -e 'require "sequel"; require "roda"; require "capybara"'
|
42
|
+
0.37 real 0.20 user 0.15 sys
|
43
|
+
|
44
|
+
The more libraries your program uses that you can preload in the server
|
45
|
+
program, the greater the speedup this offers.
|
46
|
+
|
47
|
+
== Speeding Things Up Even More By Avoiding Rubygems
|
48
|
+
|
49
|
+
Loading Rubygems is by far the slowest thing that Ruby does during
|
50
|
+
process initialization:
|
51
|
+
|
52
|
+
$ /usr/bin/time ruby -e ''
|
53
|
+
0.25 real 0.11 user 0.14 sys
|
54
|
+
|
55
|
+
$ /usr/bin/time ruby --disable-gems -e ''
|
56
|
+
0.03 real 0.02 user 0.01 sys
|
57
|
+
|
58
|
+
You can speedup +by+ by making it not require rubygems, since it only
|
59
|
+
needs the +socket+ standard library. The only issue with that is that
|
60
|
+
+by+ is distributed as a gem. There are a few workarounds.
|
61
|
+
|
62
|
+
1. Create a shell alias. How you create the alias will depend on
|
63
|
+
the shell you are using, but here's some Ruby code that will
|
64
|
+
output an alias command that will work for most shells:
|
65
|
+
|
66
|
+
require 'rbconfig'
|
67
|
+
by = Gem.activate_bin_path("by", "by")
|
68
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
69
|
+
puts "alias by='#{ruby} --disable-gems #{by}'"
|
70
|
+
|
71
|
+
Note that one issue with using a shell alias is that it only
|
72
|
+
works when loaded and used by the shell, it won't work if
|
73
|
+
executed by another program.
|
74
|
+
|
75
|
+
2. Copy the +by+ program and modify the shebang line to use the
|
76
|
+
path to your +ruby+ binary and <tt>--disable-gems</tt>. You can
|
77
|
+
get the path to the +by+ program with the following Ruby code.
|
78
|
+
|
79
|
+
puts Gem.activate_bin_path("by", "by")
|
80
|
+
|
81
|
+
You would copy that file to somewhere in your <tt>$PATH</tt>
|
82
|
+
before where the rubygems wrapper is installed, and then modify
|
83
|
+
the shebang.
|
84
|
+
|
85
|
+
3. Add your own shell wrapper program that calls +by+. Here's some
|
86
|
+
example Ruby code that may work, though whether it does depends
|
87
|
+
on your shell.
|
88
|
+
|
89
|
+
require 'rbconfig'
|
90
|
+
by = Gem.activate_bin_path("by", "by")
|
91
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
92
|
+
File.binwrite("by", "#!/bin/sh\nexec #{ruby} --disable-gems #{by} \"$@\"\n")
|
93
|
+
File.chmod(0755, "by")
|
94
|
+
|
95
|
+
With each of these approaches, you can get much faster program
|
96
|
+
execution:
|
97
|
+
|
98
|
+
$ /usr/bin/time ./by -e 'require "sequel"; require "roda"; require "capybara"'
|
99
|
+
0.08 real 0.05 user 0.03 sys
|
100
|
+
|
101
|
+
As you can see, by avoiding Rubygems, using +by+ to require the
|
102
|
+
three libraries executes three times faster than Ruby itself starts
|
103
|
+
if you are using Rubygems.
|
104
|
+
|
105
|
+
With each of these approaches, you need to update the alias/wrapper
|
106
|
+
any time you update the +by+ gem when the +by+ program itself has
|
107
|
+
changed. However, the +by+ program itself is quite small and simple
|
108
|
+
and unlikely to change.
|
109
|
+
|
110
|
+
== Argument Handling
|
111
|
+
|
112
|
+
<tt>by-server</tt> treats all arguments provided on the command line as
|
113
|
+
arguments to <tt>Kernel#require</tt>.
|
114
|
+
|
115
|
+
+by+ passes all arguments to the worker process over the UNIX socket.
|
116
|
+
|
117
|
+
The worker process handles arguments passed by the client in the following
|
118
|
+
way:
|
119
|
+
|
120
|
+
* If first argument is +m+ or matches <tt>/\.rb:\d+\z</tt>, uses the +m+
|
121
|
+
gem to run a single minitest test by line number, waiting until after
|
122
|
+
the test is run so that it can return the correct exit code.
|
123
|
+
* If first argument is +irb+, starts an IRB shell with remaining arguments
|
124
|
+
in ARGV.
|
125
|
+
* If first argument is <tt>-e</tt>, evaluates second argument as Ruby code,
|
126
|
+
with remaining arguments in ARGV.
|
127
|
+
* If no arguments are given, evaluates Ruby code provided on stdin.
|
128
|
+
* Otherwise, treats first argument as a file name, expands the file path,
|
129
|
+
and then requires that. If Minitest is loaded and set to autorun, waits
|
130
|
+
until after Minitest runs tests, so it can return the correct exit code.
|
131
|
+
If Minitest is not loaded or not set to autorun, exits after the file
|
132
|
+
is required.
|
133
|
+
|
134
|
+
=== Restarting the Server
|
135
|
+
|
136
|
+
If <tt>by-server</tt> is already running, running <tt>by-server</tt> will
|
137
|
+
shutdown the existing server and start a new server with the arguments it
|
138
|
+
is given.
|
139
|
+
|
140
|
+
=== Stopping the Server
|
141
|
+
|
142
|
+
Running <tt>by-server stop</tt> will stop an existing server without starting
|
143
|
+
a new server. If no server is running, <tt>by-server stop</tt> will exit
|
144
|
+
without doing anything.
|
145
|
+
|
146
|
+
You can also send a +TERM+ signal to the <tt>by-server</tt> process to shut
|
147
|
+
the server down gracefully. Be aware that by default, <tt>by-server</tt>
|
148
|
+
daemonizes, so the pid of the started <tt>by-server</tt> will not be the
|
149
|
+
pid <tt>by-server</tt> uses to run. For that reason, it is recommended to
|
150
|
+
use <tt>by-server stop</tt> to stop the server.
|
151
|
+
|
152
|
+
=== Running Multiple Servers
|
153
|
+
|
154
|
+
==== Manually
|
155
|
+
|
156
|
+
You can run multiple +by-server+ processes concurrently by making sure
|
157
|
+
they each use a separate UNIX socket, which you can configure with the
|
158
|
+
+BY_SOCKET+ environment variable:
|
159
|
+
|
160
|
+
$ BY_SOCKET=~/.by_sequel_socket by-server sequel
|
161
|
+
$ BY_SOCKET=~/.by_roda_socket by-server roda
|
162
|
+
$ BY_SOCKET=~/.by_sequel_socket by -e 'p [defined?(Sequel), defined?(Roda)]'
|
163
|
+
["constant", nil]
|
164
|
+
$ BY_SOCKET=~/.by_roda_socket by -e 'p [defined?(Sequel), defined?(Roda)]'
|
165
|
+
[nil, "constant"]
|
166
|
+
|
167
|
+
==== Using <tt>by-session</tt>
|
168
|
+
|
169
|
+
In many cases, it can be helpful to have a separate server process for
|
170
|
+
each application directory. <tt>by-session</tt> exists to make this easier.
|
171
|
+
<tt>by-session</tt> will call <tt>by-server</tt> with the arguments it is
|
172
|
+
given, using a socket in the current directory by default, and then open a
|
173
|
+
new shell. When the shell exits, <tt>by-session</tt> will stop the
|
174
|
+
<tt>by-server</tt> it spawned.
|
175
|
+
|
176
|
+
If the directory in which you are running <tt>by-session</tt> has a +Gemfile+,
|
177
|
+
you could add a file named <tt>.by-session-setup.rb</tt> in your home directory,
|
178
|
+
which contains:
|
179
|
+
|
180
|
+
require 'bundler/setup'
|
181
|
+
Bundler.require(:default)
|
182
|
+
|
183
|
+
When you to startup a <tt>by-session</tt> shell for the directory using the
|
184
|
+
+Gemfile+, you can use:
|
185
|
+
|
186
|
+
$ by-session ~/.by-session-setup
|
187
|
+
|
188
|
+
This will load all gems in the +Gemfile+ into the <tt>by-server</tt> process.
|
189
|
+
If you are doing this, you must be careful to only run this in a directory
|
190
|
+
that you trust.
|
191
|
+
|
192
|
+
If you don't want to specify the <tt>~/.by-session-setup</tt> argument every
|
193
|
+
time you start <tt>by-session</tt>, you can use the +BY_SERVER_AUTO_REQUIRE+
|
194
|
+
environment variable.
|
195
|
+
|
196
|
+
=== Environment Variables
|
197
|
+
|
198
|
+
+BY_SOCKET+ :: The path to the UNIX socket to listen on (<tt>by-server</tt>)
|
199
|
+
or connect to (+by+).
|
200
|
+
+DEBUG+ :: If set to +log+, logs <tt>$LOADED_FEATURES</tt> to stdout
|
201
|
+
after requiring libraries (<tt>by-server</tt>) or before worker
|
202
|
+
process shutdown (+by+).
|
203
|
+
|
204
|
+
==== <tt>by-server</tt>-Specific Environment Variables
|
205
|
+
|
206
|
+
+BY_SERVER_AUTO_REQUIRE+ :: Whitespace separated list of libraries for
|
207
|
+
<tt>by-server</tt> to require, before it requires
|
208
|
+
command line arguments.
|
209
|
+
+BY_SERVER_NO_DAEMON+ :: Do not daemonize if set.
|
210
|
+
+BY_SERVER_DAEMON_NO_CHDIR+ :: Do not change directory to <tt>/</tt>
|
211
|
+
when daemonizing if set.
|
212
|
+
+BY_SERVER_DAEMON_NO_REDIR_STDIO+ :: Do not redirect stdio to
|
213
|
+
<tt>/dev/null</tt> when daemonizing
|
214
|
+
if set.
|
215
|
+
|
216
|
+
== <tt>by-server</tt> Signals
|
217
|
+
|
218
|
+
+QUIT+ :: Close the socket (this is what <tt>by-server stop</tt> uses).
|
219
|
+
+TERM+ :: Delete the socket path and then close the socket.
|
220
|
+
|
221
|
+
== Internals
|
222
|
+
|
223
|
+
There are two classes, <tt>By::Server</tt> and <tt>By::Worker</tt>.
|
224
|
+
<tt>By::Server</tt> listens on the UNIX socket, forking worker
|
225
|
+
processes for each connection. <tt>By::Worker</tt> is run in each
|
226
|
+
worker process handling receiving data from the +by+ command line
|
227
|
+
program.
|
228
|
+
|
229
|
+
The +by+ command line program is self-contained, there is no
|
230
|
+
Ruby class for the behavior, to make sure startup is as fast as
|
231
|
+
possible. <tt>by-session</tt> is also self-contained.
|
232
|
+
|
233
|
+
== Customization
|
234
|
+
|
235
|
+
For custom handling of arguments, you can require <tt>by/server</tt>
|
236
|
+
and use the <tt>By::Server.with_argument_handler</tt> method. For example,
|
237
|
+
if you wanted to add support for an initial <tt>-I</tt> option to modify
|
238
|
+
the load path, and then use the standard argument handling:
|
239
|
+
|
240
|
+
require 'by/server'
|
241
|
+
|
242
|
+
By::Server.with_argument_handler do |args|
|
243
|
+
if args[0] == '-I'
|
244
|
+
args.shift
|
245
|
+
$LOAD_PATH.unshift(args.shift)
|
246
|
+
end
|
247
|
+
super(args)
|
248
|
+
end.new.run
|
249
|
+
|
250
|
+
Note that if you do this, you are responsible for making sure
|
251
|
+
to correctly communicate with the client socket. Otherwise, it's
|
252
|
+
possible the client socket may hang waiting on a response. Please
|
253
|
+
review the default argument handling in <tt>lib/by/worker.rb</tt>
|
254
|
+
before writing your own argument handler.
|
255
|
+
|
256
|
+
== Security
|
257
|
+
|
258
|
+
As with any program that forks without executing, the memory layout
|
259
|
+
is shared by the client and the server program, which can lead to
|
260
|
+
Blind Return Oriented Programming (BROP) attacks. You should avoid
|
261
|
+
using +by+ to run a program that deals with any untrusted input.
|
262
|
+
+by+ makes a deliberate choice to trade security to make process
|
263
|
+
startup as fast as possible.
|
264
|
+
|
265
|
+
The server socket is set to mode 0600, so it is only readable and
|
266
|
+
writable by the same user.
|
267
|
+
|
268
|
+
== Name
|
269
|
+
|
270
|
+
The name +by+ was chosen because it is +ruby+ with the +ru+ preloaded.
|
271
|
+
|
272
|
+
== Similar Projects
|
273
|
+
|
274
|
+
* Spring: https://github.com/rails/spring
|
275
|
+
* Spin: https://github.com/jstorimer/spin
|
276
|
+
* Spinoff: https://github.com/bernd/spinoff
|
277
|
+
|
278
|
+
== License
|
279
|
+
|
280
|
+
MIT
|
281
|
+
|
282
|
+
== Author
|
283
|
+
|
284
|
+
Jeremy Evans <code@jeremyevans.net>
|
data/bin/by
ADDED
@@ -0,0 +1,38 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
require 'socket'
|
5
|
+
|
6
|
+
by_server_path = ENV['BY_SOCKET'] || File.join(ENV["HOME"], '.by_socket')
|
7
|
+
s = UNIXSocket.new(by_server_path)
|
8
|
+
pid = s.readline("\0", chomp: true).to_i
|
9
|
+
|
10
|
+
unless pid > 1
|
11
|
+
$stderr.puts "Invalid by_server worker pid"
|
12
|
+
exit(1)
|
13
|
+
end
|
14
|
+
|
15
|
+
s.send_io($stdin)
|
16
|
+
s.send_io($stdout)
|
17
|
+
s.send_io($stderr)
|
18
|
+
|
19
|
+
s.write(Dir.pwd)
|
20
|
+
s.write("\0")
|
21
|
+
|
22
|
+
ENV.each do |k, v|
|
23
|
+
s.write(k)
|
24
|
+
s.write("=")
|
25
|
+
s.write(v)
|
26
|
+
s.write("\0")
|
27
|
+
end
|
28
|
+
s.write("\0")
|
29
|
+
|
30
|
+
ARGV.each do |arg|
|
31
|
+
s.write(arg)
|
32
|
+
s.write("\0")
|
33
|
+
end
|
34
|
+
|
35
|
+
s.shutdown(Socket::SHUT_WR)
|
36
|
+
exit_status = s.read
|
37
|
+
s.close
|
38
|
+
exit(exit_status == '0')
|
data/bin/by-server
ADDED
data/bin/by-session
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
require 'rbconfig'
|
5
|
+
ENV['BY_SOCKET'] ||= File.join(Dir.pwd, '.by_socket')
|
6
|
+
by_server = File.join(__dir__, 'by-server')
|
7
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
8
|
+
|
9
|
+
begin
|
10
|
+
system(ruby, by_server, *ARGV, exception: true)
|
11
|
+
system(ENV["SHELL"] || '/bin/sh')
|
12
|
+
ensure
|
13
|
+
system(ruby, by_server, 'stop', exception: true)
|
14
|
+
end
|
data/lib/by/server.rb
ADDED
@@ -0,0 +1,211 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'socket'
|
4
|
+
require_relative 'worker'
|
5
|
+
|
6
|
+
module By
|
7
|
+
class Server
|
8
|
+
# Return a subclass that will use a Worker subclass with
|
9
|
+
# a handle_args method defined by the given block. Allows
|
10
|
+
# for easily customizing/overriding the default argument
|
11
|
+
# handling.
|
12
|
+
def self.with_argument_handler(&block)
|
13
|
+
worker_subclass = Class.new(new.default_worker_class) do
|
14
|
+
define_method(:handle_args, &block)
|
15
|
+
end
|
16
|
+
Class.new(self) do
|
17
|
+
define_method(:default_worker_class){worker_subclass}
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
# Creates a new server. Arguments:
|
22
|
+
# socket_path: The path to the UNIX socket to create and listen on.
|
23
|
+
# argv: The arguments to the server, which are libraries to be required by default.
|
24
|
+
# debug: If set, operations on an existing server socket will be logged.
|
25
|
+
# If the value is <tt>'log'</tt>, <tt>$LOADED_FEATURES</tt> will also be logged to the stdout
|
26
|
+
# after libraries have been required.
|
27
|
+
# daemonize: Whether to daemonize, +true+ by default.
|
28
|
+
# daemon_args: Arguments to use when daemonizing, <tt>[false, false]</tt> by default.
|
29
|
+
# worker_class: The class to use for worker process handling, Worker by default.
|
30
|
+
def initialize(socket_path: default_socket_path, argv: default_argv, debug: default_debug,
|
31
|
+
daemonize: default_daemonize, daemon_args: default_daemon_args,
|
32
|
+
worker_class: default_worker_class)
|
33
|
+
@socket_path = socket_path
|
34
|
+
@argv = argv
|
35
|
+
@debug = debug
|
36
|
+
if @daemonize = !!daemonize
|
37
|
+
@daemon_args = Array(daemon_args)
|
38
|
+
end
|
39
|
+
@worker_class = worker_class
|
40
|
+
end
|
41
|
+
|
42
|
+
# The default socket path to use. Use the +BY_SOCKET+ environment variable if set,
|
43
|
+
# or <tt>~/.by_socket</tt> if not set.
|
44
|
+
def default_socket_path
|
45
|
+
ENV['BY_SOCKET'] || File.join(ENV["HOME"], '.by_socket')
|
46
|
+
end
|
47
|
+
|
48
|
+
# The default server arguments, uses +ARGV+ by default.
|
49
|
+
def default_argv
|
50
|
+
ARGV
|
51
|
+
end
|
52
|
+
|
53
|
+
# The default debug mode. This uses and removes the +DEBUG+ environment variable.
|
54
|
+
def default_debug
|
55
|
+
ENV.delete('DEBUG')
|
56
|
+
end
|
57
|
+
|
58
|
+
# The default for whether to daemonize. It is true if the +BY_SERVER_NO_DAEMON+
|
59
|
+
# environment variable is not set.
|
60
|
+
def default_daemonize
|
61
|
+
!ENV['BY_SERVER_NO_DAEMON']
|
62
|
+
end
|
63
|
+
|
64
|
+
# The default arguments when daemonizing. By default, considers the
|
65
|
+
# +BY_SERVER_DAEMON_NO_CHDIR+ and +BY_SERVER_DAEMON_NO_REDIR_STDIO+ environment
|
66
|
+
# variables.
|
67
|
+
def default_daemon_args
|
68
|
+
[!!ENV['BY_SERVER_DAEMON_NO_CHDIR'], !!ENV['BY_SERVER_DAEMON_NO_REDIR_STDIO']]
|
69
|
+
end
|
70
|
+
|
71
|
+
# The default worker class to use for worker processes, Worker by default.
|
72
|
+
def default_worker_class
|
73
|
+
Worker
|
74
|
+
end
|
75
|
+
|
76
|
+
# Runs the server. This will not terminate until the server receives SIGTERM
|
77
|
+
# or stop_accepting_clients! is called manually.
|
78
|
+
#
|
79
|
+
# If stop? is true, does not run a server, just handles an existing server.
|
80
|
+
def run
|
81
|
+
handle_existing_server
|
82
|
+
return if stop?
|
83
|
+
|
84
|
+
handle_argv
|
85
|
+
setup_server
|
86
|
+
daemonize if daemonize?
|
87
|
+
setup_signals
|
88
|
+
accept_clients
|
89
|
+
end
|
90
|
+
|
91
|
+
# Handle an existing server socket. This attempts to connect to the socket and
|
92
|
+
# then shutdown the server. If successful, it removes the socket. If unnecessful,
|
93
|
+
# it will print an error.
|
94
|
+
def handle_existing_server
|
95
|
+
if File.socket?(@socket_path)
|
96
|
+
begin
|
97
|
+
@socket = UNIXSocket.new(@socket_path)
|
98
|
+
print "Shutting down existing by_server at #{@socket_path}..." if @debug
|
99
|
+
raise "Invalid by_server worker pid" unless @socket.readline("\0", chomp: true).to_i > 1
|
100
|
+
@socket.send_io($stdin)
|
101
|
+
@socket.send_io($stdout)
|
102
|
+
@socket.send_io($stderr)
|
103
|
+
@socket.write("stop")
|
104
|
+
@socket.shutdown(Socket::SHUT_WR)
|
105
|
+
@socket.read
|
106
|
+
@socket.close
|
107
|
+
rescue => e
|
108
|
+
puts "FAILED!!!" if @debug
|
109
|
+
$stderr.puts "Error shutting down server on existing socket: #{e.class}: #{e.message}"
|
110
|
+
exit(1)
|
111
|
+
else
|
112
|
+
puts "Success!" if @debug
|
113
|
+
end
|
114
|
+
@socket = nil
|
115
|
+
File.delete(@socket_path)
|
116
|
+
end
|
117
|
+
end
|
118
|
+
|
119
|
+
# Whether to only stop an existing server and not start a new server.
|
120
|
+
def stop?
|
121
|
+
@argv == ['stop']
|
122
|
+
end
|
123
|
+
|
124
|
+
# Handle arguments provided to the server. Requires each argument by default.
|
125
|
+
def handle_argv
|
126
|
+
(auto_require_files + @argv).each{|f| require f}
|
127
|
+
print_loaded_features if @debug == 'log'
|
128
|
+
end
|
129
|
+
|
130
|
+
# Files to automatically require, uses the +BY_SERVER_AUTO_REQUIRE+ environment
|
131
|
+
# variable by default.
|
132
|
+
def auto_require_files
|
133
|
+
(ENV['BY_SERVER_AUTO_REQUIRE'] || '').split
|
134
|
+
end
|
135
|
+
|
136
|
+
# Creates and listens on the server socket.
|
137
|
+
def setup_server
|
138
|
+
# Prevent TOCTOU on server socket creation
|
139
|
+
umask = File.umask(077)
|
140
|
+
@socket = UNIXServer.new(@socket_path)
|
141
|
+
File.umask(umask)
|
142
|
+
system('chmod', '600', @socket_path)
|
143
|
+
end
|
144
|
+
|
145
|
+
# Daemonize with configured daemon args using Process.daemon.
|
146
|
+
def daemonize
|
147
|
+
Process.daemon(*@daemon_args)
|
148
|
+
end
|
149
|
+
|
150
|
+
# Whether to daemonize.
|
151
|
+
def daemonize?
|
152
|
+
!!@daemonize
|
153
|
+
end
|
154
|
+
|
155
|
+
# Trap SIGTERM and have it stop accepting clients.
|
156
|
+
# Trap SIGTERM and have it remove the socket and stop accepting clients.
|
157
|
+
def setup_signals
|
158
|
+
@sigquit_default = Signal.trap(:QUIT) do
|
159
|
+
stop_accepting_clients!
|
160
|
+
end
|
161
|
+
@sigterm_default = Signal.trap(:TERM) do
|
162
|
+
begin
|
163
|
+
File.delete(@socket_path)
|
164
|
+
rescue Errno::ENOENT
|
165
|
+
# server socket already deleted, ignore
|
166
|
+
end
|
167
|
+
stop_accepting_clients!
|
168
|
+
end
|
169
|
+
end
|
170
|
+
|
171
|
+
# Accept each client connection and fork a worker socket for it.
|
172
|
+
# Terminate loop when stop_accepting_clients! is called.
|
173
|
+
def accept_clients
|
174
|
+
while socket = accept_client
|
175
|
+
fork_worker(socket)
|
176
|
+
end
|
177
|
+
end
|
178
|
+
|
179
|
+
# Accept a new client connection, or return nil if
|
180
|
+
# stop_accepting_clients! has been called.
|
181
|
+
def accept_client
|
182
|
+
@socket.accept
|
183
|
+
rescue IOError, Errno::EBADF
|
184
|
+
# likely closed stream, return nil to exit accept_clients loop
|
185
|
+
nil
|
186
|
+
end
|
187
|
+
|
188
|
+
# Close the server socket. This will trigger the accept_clients
|
189
|
+
# loop to terminate.
|
190
|
+
def stop_accepting_clients!
|
191
|
+
@socket.close
|
192
|
+
end
|
193
|
+
|
194
|
+
# Fork a worker process to handle the client connection. Close the
|
195
|
+
# given socket after the fork, so the socket will open be open in
|
196
|
+
# the worker process.
|
197
|
+
def fork_worker(socket)
|
198
|
+
Process.detach(Process.fork do
|
199
|
+
Signal.trap(:QUIT, @sigquit_default)
|
200
|
+
Signal.trap(:TERM, @sigterm_default)
|
201
|
+
@worker_class.new(socket).run
|
202
|
+
end)
|
203
|
+
socket.close
|
204
|
+
end
|
205
|
+
|
206
|
+
# Print <tt>$LOADED_FEATURES</tt> to stdout.
|
207
|
+
def print_loaded_features
|
208
|
+
puts $LOADED_FEATURES
|
209
|
+
end
|
210
|
+
end
|
211
|
+
end
|
data/lib/by/worker.rb
ADDED
@@ -0,0 +1,164 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module By
|
4
|
+
class Worker
|
5
|
+
# Whether the worker process should signal a normal exit to the client.
|
6
|
+
# The default is nil, which signals a normal exit when the worker
|
7
|
+
# process exits normally. This can be set to false to signal an
|
8
|
+
# abnormal exit, such as to indicate test failures.
|
9
|
+
attr_writer :normal_exit
|
10
|
+
|
11
|
+
# Create the worker. Arguments
|
12
|
+
# socket :: The Unix socket to use to communicate with the client.
|
13
|
+
# sigterm_handler
|
14
|
+
def initialize(socket)
|
15
|
+
@socket = socket
|
16
|
+
@normal_exit = nil
|
17
|
+
end
|
18
|
+
|
19
|
+
# Run the worker process.
|
20
|
+
def run
|
21
|
+
write_pid
|
22
|
+
reopen_stdio
|
23
|
+
chdir_or_stop
|
24
|
+
replace_env
|
25
|
+
handle_args(get_args)
|
26
|
+
end
|
27
|
+
|
28
|
+
# Write the current process pid to the client, to signal that
|
29
|
+
# the worker process is ready.
|
30
|
+
def write_pid
|
31
|
+
@socket.write($$.to_s)
|
32
|
+
@socket.write("\0")
|
33
|
+
end
|
34
|
+
|
35
|
+
# Replace stdin, stdout, stderr with the IO values provided by the client.
|
36
|
+
def reopen_stdio
|
37
|
+
$stdin.reopen(@socket.recv_io(IO))
|
38
|
+
$stdout.reopen(@socket.recv_io(IO))
|
39
|
+
$stderr.reopen(@socket.recv_io(IO))
|
40
|
+
end
|
41
|
+
|
42
|
+
# Change to the given directory, unless the client is telling the
|
43
|
+
# worker to stop the server.
|
44
|
+
def chdir_or_stop
|
45
|
+
arg = @socket.readline("\0", chomp: true)
|
46
|
+
arg == 'stop' ? stop_server : chdir(arg)
|
47
|
+
end
|
48
|
+
|
49
|
+
# Stop the server process by sending it the SIGQUIT signal, then exit.
|
50
|
+
def stop_server
|
51
|
+
Process.kill(:QUIT, Process.ppid)
|
52
|
+
cleanup_proc.call
|
53
|
+
exit
|
54
|
+
end
|
55
|
+
|
56
|
+
# Change to the given directory.
|
57
|
+
def chdir(dir)
|
58
|
+
Dir.chdir(dir)
|
59
|
+
end
|
60
|
+
|
61
|
+
# Print <tt>$LOADED_FEATURES</tt> to stdout.
|
62
|
+
def print_loaded_features
|
63
|
+
puts $LOADED_FEATURES
|
64
|
+
end
|
65
|
+
|
66
|
+
# A proc for communicating exit status to the client, and
|
67
|
+
# printing the loaded features if configured. Used during
|
68
|
+
# process shutdown.
|
69
|
+
def cleanup_proc
|
70
|
+
proc do
|
71
|
+
if @normal_exit.nil?
|
72
|
+
@normal_exit = $!.nil? || ($!.is_a?(SystemExit) && $!.success?)
|
73
|
+
end
|
74
|
+
@socket.write(@normal_exit ? '0' : '1')
|
75
|
+
@socket.shutdown(Socket::SHUT_WR)
|
76
|
+
@socket.close
|
77
|
+
print_loaded_features if ENV['DEBUG'] == 'log'
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
# Replace ENV with the environment provided by the client.
|
82
|
+
def replace_env
|
83
|
+
env = {}
|
84
|
+
while line = @socket.readline("\0", chomp: true)
|
85
|
+
break if line.empty?
|
86
|
+
k, v = line.split("=")
|
87
|
+
env[k] = v
|
88
|
+
end
|
89
|
+
ENV.replace(env)
|
90
|
+
end
|
91
|
+
|
92
|
+
# An array of arguments provided by the client.
|
93
|
+
def get_args
|
94
|
+
args = []
|
95
|
+
while !@socket.eof?
|
96
|
+
args << @socket.readline("\0", chomp: true)
|
97
|
+
end
|
98
|
+
args
|
99
|
+
end
|
100
|
+
|
101
|
+
# Handle arguments provided by the client. Ensure correct
|
102
|
+
# client after handling the arguments.
|
103
|
+
def handle_args(args)
|
104
|
+
worker = self
|
105
|
+
cleanup = cleanup_proc
|
106
|
+
|
107
|
+
case arg = args.first
|
108
|
+
when 'm', /\.rb:\d+\z/
|
109
|
+
args.shift if arg == 'm'
|
110
|
+
ARGV.replace(args)
|
111
|
+
require 'm'
|
112
|
+
M.define_singleton_method(:exit!) do |exit_code|
|
113
|
+
worker.normal_exit = exit_code == true
|
114
|
+
cleanup.call
|
115
|
+
super(exit_code)
|
116
|
+
end
|
117
|
+
M.run(args)
|
118
|
+
when 'irb'
|
119
|
+
at_exit(&cleanup)
|
120
|
+
args.shift
|
121
|
+
ARGV.replace(args)
|
122
|
+
require 'irb'
|
123
|
+
IRB.start(__FILE__)
|
124
|
+
when '-e'
|
125
|
+
at_exit(&cleanup)
|
126
|
+
unless args.length >= 2
|
127
|
+
$stderr.puts 'no code specified for -e (RuntimeError)'
|
128
|
+
exit(1)
|
129
|
+
end
|
130
|
+
args.shift
|
131
|
+
code = args.shift
|
132
|
+
ARGV.replace(args)
|
133
|
+
::TOPLEVEL_BINDING.eval(code)
|
134
|
+
when String
|
135
|
+
args.shift
|
136
|
+
ARGV.replace(args)
|
137
|
+
|
138
|
+
begin
|
139
|
+
require File.expand_path(arg)
|
140
|
+
rescue
|
141
|
+
@normal_exit = false
|
142
|
+
at_exit(&cleanup)
|
143
|
+
raise
|
144
|
+
end
|
145
|
+
|
146
|
+
if defined?(Minitest) && Minitest.class_variable_get(:@@installed_at_exit)
|
147
|
+
Minitest.singleton_class.prepend(Module.new do
|
148
|
+
define_method(:run) do |argv|
|
149
|
+
super(argv).tap{|exit_code| p worker.normal_exit = exit_code == true}
|
150
|
+
end
|
151
|
+
end)
|
152
|
+
Minitest.after_run(&cleanup)
|
153
|
+
else
|
154
|
+
at_exit(&cleanup)
|
155
|
+
end
|
156
|
+
else
|
157
|
+
# no arguments
|
158
|
+
at_exit(&cleanup)
|
159
|
+
::TOPLEVEL_BINDING.eval($stdin.read)
|
160
|
+
end
|
161
|
+
end
|
162
|
+
end
|
163
|
+
end
|
164
|
+
|
metadata
ADDED
@@ -0,0 +1,104 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: by
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jeremy Evans
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2023-02-07 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: minitest-global_expectations
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: m
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
description: |
|
42
|
+
by is a library preloader for ruby designed to speed up process startup.
|
43
|
+
It uses a client/server approach, where the server loads the libraries and
|
44
|
+
listens on a UNIX socket, and the client connects to that socket to run
|
45
|
+
processes. For each client connection, the server forks a worker process,
|
46
|
+
which uses the current directory, stdin, stdout, stderr, and environment
|
47
|
+
of the client process. The worker process then processes the arguments
|
48
|
+
provided by the client. The client process waits until the worker process
|
49
|
+
closes the socket, which the worker process attempts to do right before
|
50
|
+
it exits.
|
51
|
+
email: code@jeremyevans.net
|
52
|
+
executables:
|
53
|
+
- by
|
54
|
+
- by-server
|
55
|
+
- by-session
|
56
|
+
extensions: []
|
57
|
+
extra_rdoc_files:
|
58
|
+
- README.rdoc
|
59
|
+
- CHANGELOG
|
60
|
+
- MIT-LICENSE
|
61
|
+
files:
|
62
|
+
- CHANGELOG
|
63
|
+
- MIT-LICENSE
|
64
|
+
- README.rdoc
|
65
|
+
- bin/by
|
66
|
+
- bin/by-server
|
67
|
+
- bin/by-session
|
68
|
+
- lib/by/server.rb
|
69
|
+
- lib/by/worker.rb
|
70
|
+
homepage: http://github.com/jeremyevans/by
|
71
|
+
licenses:
|
72
|
+
- MIT
|
73
|
+
metadata:
|
74
|
+
bug_tracker_uri: https://github.com/jeremyevans/by/issues
|
75
|
+
changelog_uri: https://github.com/jeremyevans/by/blob/master/CHANGELOG
|
76
|
+
mailing_list_uri: https://github.com/jeremyevans/by/discussions
|
77
|
+
source_code_uri: https://github.com/jeremyevans/by
|
78
|
+
post_install_message:
|
79
|
+
rdoc_options:
|
80
|
+
- "--quiet"
|
81
|
+
- "--line-numbers"
|
82
|
+
- "--inline-source"
|
83
|
+
- "--title"
|
84
|
+
- 'by: Ruby library preloader'
|
85
|
+
- "--main"
|
86
|
+
- README.rdoc
|
87
|
+
require_paths:
|
88
|
+
- lib
|
89
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
90
|
+
requirements:
|
91
|
+
- - ">="
|
92
|
+
- !ruby/object:Gem::Version
|
93
|
+
version: '2.6'
|
94
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
95
|
+
requirements:
|
96
|
+
- - ">="
|
97
|
+
- !ruby/object:Gem::Version
|
98
|
+
version: '0'
|
99
|
+
requirements: []
|
100
|
+
rubygems_version: 3.4.1
|
101
|
+
signing_key:
|
102
|
+
specification_version: 4
|
103
|
+
summary: Ruby library preloader
|
104
|
+
test_files: []
|