by 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG +3 -0
- data/MIT-LICENSE +18 -0
- data/README.rdoc +284 -0
- data/bin/by +38 -0
- data/bin/by-server +6 -0
- data/bin/by-session +14 -0
- data/lib/by/server.rb +211 -0
- data/lib/by/worker.rb +164 -0
- metadata +104 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: '078f15faf5964b6b3bb4ac860c7733e069c0a8cb46556e9af7e555f8c3808d54'
|
4
|
+
data.tar.gz: e878d37b2417d4d655f138d3e2bae5bf51a76c34b7a3fab2956bc40c88a77dbb
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 128a3c27c4dfe729b62fa15d9b961f7384a75ec02030403ae85b376981b731c8d892305732e519cbfb84ab105b816275e6a910afff7f9187c2f723d7683eb12d
|
7
|
+
data.tar.gz: 61895959cd9a6097ded3430017d0a831affa73c668c8301579dd4a62f661a6c1fd3bb09871d5eb321ce9d97fb7a64c573fc8b92a60ae34b7adfcad835eb0ae08
|
data/CHANGELOG
ADDED
data/MIT-LICENSE
ADDED
@@ -0,0 +1,18 @@
|
|
1
|
+
Copyright (c) 2023 Jeremy Evans
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
|
+
of this software and associated documentation files (the "Software"), to
|
5
|
+
deal in the Software without restriction, including without limitation the
|
6
|
+
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
|
7
|
+
sell copies of the Software, and to permit persons to whom the Software is
|
8
|
+
furnished to do so, subject to the following conditions:
|
9
|
+
|
10
|
+
The above copyright notice and this permission notice shall be included in
|
11
|
+
all copies or substantial portions of the Software.
|
12
|
+
|
13
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
15
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
16
|
+
THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.rdoc
ADDED
@@ -0,0 +1,284 @@
|
|
1
|
+
= by
|
2
|
+
|
3
|
+
by is a library preloader for Ruby designed to speed up process startup.
|
4
|
+
It uses a client/server approach, where the server loads the libraries and
|
5
|
+
listens on a UNIX socket, and the client connects to that socket to run
|
6
|
+
a process. For each client connection, the server forks a worker process,
|
7
|
+
which uses the current directory, stdin, stdout, stderr, and environment
|
8
|
+
of the client process. The worker process then processes the arguments
|
9
|
+
provided by the client. The client process waits until the worker process
|
10
|
+
returns an exit code and closes the socket, and uses exit code 0 (normal
|
11
|
+
exit) if the worker process indicates success, or exit code 1 (error)
|
12
|
+
if the worker process indicates an error.
|
13
|
+
|
14
|
+
== Installation
|
15
|
+
|
16
|
+
gem install by
|
17
|
+
|
18
|
+
== Source Code
|
19
|
+
|
20
|
+
Source code is available on GitHub at https://github.com/jeremyevans/by
|
21
|
+
|
22
|
+
== Usage
|
23
|
+
|
24
|
+
To use +by+, you first start <tt>by-server</tt>, passing in libraries you would
|
25
|
+
like to preload.
|
26
|
+
|
27
|
+
$ by-server sequel roda capybara
|
28
|
+
|
29
|
+
Then you can run ruby with the libraries preloaded using +by+:
|
30
|
+
|
31
|
+
$ by -e 'p [Sequel, Roda, Capybara]'
|
32
|
+
[Sequel, Roda, Capybara]
|
33
|
+
|
34
|
+
The advantage of using +by+ is that the libraries are already loaded,
|
35
|
+
so Ruby doesn't have to find the libraries and parse the files in each
|
36
|
+
library on process startup. Here's a performance comparison:
|
37
|
+
|
38
|
+
$ /usr/bin/time ruby -e 'require "sequel"; require "roda"; require "capybara"'
|
39
|
+
1.67 real 0.93 user 0.66 sys
|
40
|
+
|
41
|
+
$ /usr/bin/time by -e 'require "sequel"; require "roda"; require "capybara"'
|
42
|
+
0.37 real 0.20 user 0.15 sys
|
43
|
+
|
44
|
+
The more libraries your program uses that you can preload in the server
|
45
|
+
program, the greater the speedup this offers.
|
46
|
+
|
47
|
+
== Speeding Things Up Even More By Avoiding Rubygems
|
48
|
+
|
49
|
+
Loading Rubygems is by far the slowest thing that Ruby does during
|
50
|
+
process initialization:
|
51
|
+
|
52
|
+
$ /usr/bin/time ruby -e ''
|
53
|
+
0.25 real 0.11 user 0.14 sys
|
54
|
+
|
55
|
+
$ /usr/bin/time ruby --disable-gems -e ''
|
56
|
+
0.03 real 0.02 user 0.01 sys
|
57
|
+
|
58
|
+
You can speedup +by+ by making it not require rubygems, since it only
|
59
|
+
needs the +socket+ standard library. The only issue with that is that
|
60
|
+
+by+ is distributed as a gem. There are a few workarounds.
|
61
|
+
|
62
|
+
1. Create a shell alias. How you create the alias will depend on
|
63
|
+
the shell you are using, but here's some Ruby code that will
|
64
|
+
output an alias command that will work for most shells:
|
65
|
+
|
66
|
+
require 'rbconfig'
|
67
|
+
by = Gem.activate_bin_path("by", "by")
|
68
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
69
|
+
puts "alias by='#{ruby} --disable-gems #{by}'"
|
70
|
+
|
71
|
+
Note that one issue with using a shell alias is that it only
|
72
|
+
works when loaded and used by the shell, it won't work if
|
73
|
+
executed by another program.
|
74
|
+
|
75
|
+
2. Copy the +by+ program and modify the shebang line to use the
|
76
|
+
path to your +ruby+ binary and <tt>--disable-gems</tt>. You can
|
77
|
+
get the path to the +by+ program with the following Ruby code.
|
78
|
+
|
79
|
+
puts Gem.activate_bin_path("by", "by")
|
80
|
+
|
81
|
+
You would copy that file to somewhere in your <tt>$PATH</tt>
|
82
|
+
before where the rubygems wrapper is installed, and then modify
|
83
|
+
the shebang.
|
84
|
+
|
85
|
+
3. Add your own shell wrapper program that calls +by+. Here's some
|
86
|
+
example Ruby code that may work, though whether it does depends
|
87
|
+
on your shell.
|
88
|
+
|
89
|
+
require 'rbconfig'
|
90
|
+
by = Gem.activate_bin_path("by", "by")
|
91
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
92
|
+
File.binwrite("by", "#!/bin/sh\nexec #{ruby} --disable-gems #{by} \"$@\"\n")
|
93
|
+
File.chmod(0755, "by")
|
94
|
+
|
95
|
+
With each of these approaches, you can get much faster program
|
96
|
+
execution:
|
97
|
+
|
98
|
+
$ /usr/bin/time ./by -e 'require "sequel"; require "roda"; require "capybara"'
|
99
|
+
0.08 real 0.05 user 0.03 sys
|
100
|
+
|
101
|
+
As you can see, by avoiding Rubygems, using +by+ to require the
|
102
|
+
three libraries executes three times faster than Ruby itself starts
|
103
|
+
if you are using Rubygems.
|
104
|
+
|
105
|
+
With each of these approaches, you need to update the alias/wrapper
|
106
|
+
any time you update the +by+ gem when the +by+ program itself has
|
107
|
+
changed. However, the +by+ program itself is quite small and simple
|
108
|
+
and unlikely to change.
|
109
|
+
|
110
|
+
== Argument Handling
|
111
|
+
|
112
|
+
<tt>by-server</tt> treats all arguments provided on the command line as
|
113
|
+
arguments to <tt>Kernel#require</tt>.
|
114
|
+
|
115
|
+
+by+ passes all arguments to the worker process over the UNIX socket.
|
116
|
+
|
117
|
+
The worker process handles arguments passed by the client in the following
|
118
|
+
way:
|
119
|
+
|
120
|
+
* If first argument is +m+ or matches <tt>/\.rb:\d+\z</tt>, uses the +m+
|
121
|
+
gem to run a single minitest test by line number, waiting until after
|
122
|
+
the test is run so that it can return the correct exit code.
|
123
|
+
* If first argument is +irb+, starts an IRB shell with remaining arguments
|
124
|
+
in ARGV.
|
125
|
+
* If first argument is <tt>-e</tt>, evaluates second argument as Ruby code,
|
126
|
+
with remaining arguments in ARGV.
|
127
|
+
* If no arguments are given, evaluates Ruby code provided on stdin.
|
128
|
+
* Otherwise, treats first argument as a file name, expands the file path,
|
129
|
+
and then requires that. If Minitest is loaded and set to autorun, waits
|
130
|
+
until after Minitest runs tests, so it can return the correct exit code.
|
131
|
+
If Minitest is not loaded or not set to autorun, exits after the file
|
132
|
+
is required.
|
133
|
+
|
134
|
+
=== Restarting the Server
|
135
|
+
|
136
|
+
If <tt>by-server</tt> is already running, running <tt>by-server</tt> will
|
137
|
+
shutdown the existing server and start a new server with the arguments it
|
138
|
+
is given.
|
139
|
+
|
140
|
+
=== Stopping the Server
|
141
|
+
|
142
|
+
Running <tt>by-server stop</tt> will stop an existing server without starting
|
143
|
+
a new server. If no server is running, <tt>by-server stop</tt> will exit
|
144
|
+
without doing anything.
|
145
|
+
|
146
|
+
You can also send a +TERM+ signal to the <tt>by-server</tt> process to shut
|
147
|
+
the server down gracefully. Be aware that by default, <tt>by-server</tt>
|
148
|
+
daemonizes, so the pid of the started <tt>by-server</tt> will not be the
|
149
|
+
pid <tt>by-server</tt> uses to run. For that reason, it is recommended to
|
150
|
+
use <tt>by-server stop</tt> to stop the server.
|
151
|
+
|
152
|
+
=== Running Multiple Servers
|
153
|
+
|
154
|
+
==== Manually
|
155
|
+
|
156
|
+
You can run multiple +by-server+ processes concurrently by making sure
|
157
|
+
they each use a separate UNIX socket, which you can configure with the
|
158
|
+
+BY_SOCKET+ environment variable:
|
159
|
+
|
160
|
+
$ BY_SOCKET=~/.by_sequel_socket by-server sequel
|
161
|
+
$ BY_SOCKET=~/.by_roda_socket by-server roda
|
162
|
+
$ BY_SOCKET=~/.by_sequel_socket by -e 'p [defined?(Sequel), defined?(Roda)]'
|
163
|
+
["constant", nil]
|
164
|
+
$ BY_SOCKET=~/.by_roda_socket by -e 'p [defined?(Sequel), defined?(Roda)]'
|
165
|
+
[nil, "constant"]
|
166
|
+
|
167
|
+
==== Using <tt>by-session</tt>
|
168
|
+
|
169
|
+
In many cases, it can be helpful to have a separate server process for
|
170
|
+
each application directory. <tt>by-session</tt> exists to make this easier.
|
171
|
+
<tt>by-session</tt> will call <tt>by-server</tt> with the arguments it is
|
172
|
+
given, using a socket in the current directory by default, and then open a
|
173
|
+
new shell. When the shell exits, <tt>by-session</tt> will stop the
|
174
|
+
<tt>by-server</tt> it spawned.
|
175
|
+
|
176
|
+
If the directory in which you are running <tt>by-session</tt> has a +Gemfile+,
|
177
|
+
you could add a file named <tt>.by-session-setup.rb</tt> in your home directory,
|
178
|
+
which contains:
|
179
|
+
|
180
|
+
require 'bundler/setup'
|
181
|
+
Bundler.require(:default)
|
182
|
+
|
183
|
+
When you to startup a <tt>by-session</tt> shell for the directory using the
|
184
|
+
+Gemfile+, you can use:
|
185
|
+
|
186
|
+
$ by-session ~/.by-session-setup
|
187
|
+
|
188
|
+
This will load all gems in the +Gemfile+ into the <tt>by-server</tt> process.
|
189
|
+
If you are doing this, you must be careful to only run this in a directory
|
190
|
+
that you trust.
|
191
|
+
|
192
|
+
If you don't want to specify the <tt>~/.by-session-setup</tt> argument every
|
193
|
+
time you start <tt>by-session</tt>, you can use the +BY_SERVER_AUTO_REQUIRE+
|
194
|
+
environment variable.
|
195
|
+
|
196
|
+
=== Environment Variables
|
197
|
+
|
198
|
+
+BY_SOCKET+ :: The path to the UNIX socket to listen on (<tt>by-server</tt>)
|
199
|
+
or connect to (+by+).
|
200
|
+
+DEBUG+ :: If set to +log+, logs <tt>$LOADED_FEATURES</tt> to stdout
|
201
|
+
after requiring libraries (<tt>by-server</tt>) or before worker
|
202
|
+
process shutdown (+by+).
|
203
|
+
|
204
|
+
==== <tt>by-server</tt>-Specific Environment Variables
|
205
|
+
|
206
|
+
+BY_SERVER_AUTO_REQUIRE+ :: Whitespace separated list of libraries for
|
207
|
+
<tt>by-server</tt> to require, before it requires
|
208
|
+
command line arguments.
|
209
|
+
+BY_SERVER_NO_DAEMON+ :: Do not daemonize if set.
|
210
|
+
+BY_SERVER_DAEMON_NO_CHDIR+ :: Do not change directory to <tt>/</tt>
|
211
|
+
when daemonizing if set.
|
212
|
+
+BY_SERVER_DAEMON_NO_REDIR_STDIO+ :: Do not redirect stdio to
|
213
|
+
<tt>/dev/null</tt> when daemonizing
|
214
|
+
if set.
|
215
|
+
|
216
|
+
== <tt>by-server</tt> Signals
|
217
|
+
|
218
|
+
+QUIT+ :: Close the socket (this is what <tt>by-server stop</tt> uses).
|
219
|
+
+TERM+ :: Delete the socket path and then close the socket.
|
220
|
+
|
221
|
+
== Internals
|
222
|
+
|
223
|
+
There are two classes, <tt>By::Server</tt> and <tt>By::Worker</tt>.
|
224
|
+
<tt>By::Server</tt> listens on the UNIX socket, forking worker
|
225
|
+
processes for each connection. <tt>By::Worker</tt> is run in each
|
226
|
+
worker process handling receiving data from the +by+ command line
|
227
|
+
program.
|
228
|
+
|
229
|
+
The +by+ command line program is self-contained, there is no
|
230
|
+
Ruby class for the behavior, to make sure startup is as fast as
|
231
|
+
possible. <tt>by-session</tt> is also self-contained.
|
232
|
+
|
233
|
+
== Customization
|
234
|
+
|
235
|
+
For custom handling of arguments, you can require <tt>by/server</tt>
|
236
|
+
and use the <tt>By::Server.with_argument_handler</tt> method. For example,
|
237
|
+
if you wanted to add support for an initial <tt>-I</tt> option to modify
|
238
|
+
the load path, and then use the standard argument handling:
|
239
|
+
|
240
|
+
require 'by/server'
|
241
|
+
|
242
|
+
By::Server.with_argument_handler do |args|
|
243
|
+
if args[0] == '-I'
|
244
|
+
args.shift
|
245
|
+
$LOAD_PATH.unshift(args.shift)
|
246
|
+
end
|
247
|
+
super(args)
|
248
|
+
end.new.run
|
249
|
+
|
250
|
+
Note that if you do this, you are responsible for making sure
|
251
|
+
to correctly communicate with the client socket. Otherwise, it's
|
252
|
+
possible the client socket may hang waiting on a response. Please
|
253
|
+
review the default argument handling in <tt>lib/by/worker.rb</tt>
|
254
|
+
before writing your own argument handler.
|
255
|
+
|
256
|
+
== Security
|
257
|
+
|
258
|
+
As with any program that forks without executing, the memory layout
|
259
|
+
is shared by the client and the server program, which can lead to
|
260
|
+
Blind Return Oriented Programming (BROP) attacks. You should avoid
|
261
|
+
using +by+ to run a program that deals with any untrusted input.
|
262
|
+
+by+ makes a deliberate choice to trade security to make process
|
263
|
+
startup as fast as possible.
|
264
|
+
|
265
|
+
The server socket is set to mode 0600, so it is only readable and
|
266
|
+
writable by the same user.
|
267
|
+
|
268
|
+
== Name
|
269
|
+
|
270
|
+
The name +by+ was chosen because it is +ruby+ with the +ru+ preloaded.
|
271
|
+
|
272
|
+
== Similar Projects
|
273
|
+
|
274
|
+
* Spring: https://github.com/rails/spring
|
275
|
+
* Spin: https://github.com/jstorimer/spin
|
276
|
+
* Spinoff: https://github.com/bernd/spinoff
|
277
|
+
|
278
|
+
== License
|
279
|
+
|
280
|
+
MIT
|
281
|
+
|
282
|
+
== Author
|
283
|
+
|
284
|
+
Jeremy Evans <code@jeremyevans.net>
|
data/bin/by
ADDED
@@ -0,0 +1,38 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
require 'socket'
|
5
|
+
|
6
|
+
by_server_path = ENV['BY_SOCKET'] || File.join(ENV["HOME"], '.by_socket')
|
7
|
+
s = UNIXSocket.new(by_server_path)
|
8
|
+
pid = s.readline("\0", chomp: true).to_i
|
9
|
+
|
10
|
+
unless pid > 1
|
11
|
+
$stderr.puts "Invalid by_server worker pid"
|
12
|
+
exit(1)
|
13
|
+
end
|
14
|
+
|
15
|
+
s.send_io($stdin)
|
16
|
+
s.send_io($stdout)
|
17
|
+
s.send_io($stderr)
|
18
|
+
|
19
|
+
s.write(Dir.pwd)
|
20
|
+
s.write("\0")
|
21
|
+
|
22
|
+
ENV.each do |k, v|
|
23
|
+
s.write(k)
|
24
|
+
s.write("=")
|
25
|
+
s.write(v)
|
26
|
+
s.write("\0")
|
27
|
+
end
|
28
|
+
s.write("\0")
|
29
|
+
|
30
|
+
ARGV.each do |arg|
|
31
|
+
s.write(arg)
|
32
|
+
s.write("\0")
|
33
|
+
end
|
34
|
+
|
35
|
+
s.shutdown(Socket::SHUT_WR)
|
36
|
+
exit_status = s.read
|
37
|
+
s.close
|
38
|
+
exit(exit_status == '0')
|
data/bin/by-server
ADDED
data/bin/by-session
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
require 'rbconfig'
|
5
|
+
ENV['BY_SOCKET'] ||= File.join(Dir.pwd, '.by_socket')
|
6
|
+
by_server = File.join(__dir__, 'by-server')
|
7
|
+
ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['RUBY_INSTALL_NAME'])
|
8
|
+
|
9
|
+
begin
|
10
|
+
system(ruby, by_server, *ARGV, exception: true)
|
11
|
+
system(ENV["SHELL"] || '/bin/sh')
|
12
|
+
ensure
|
13
|
+
system(ruby, by_server, 'stop', exception: true)
|
14
|
+
end
|
data/lib/by/server.rb
ADDED
@@ -0,0 +1,211 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'socket'
|
4
|
+
require_relative 'worker'
|
5
|
+
|
6
|
+
module By
|
7
|
+
class Server
|
8
|
+
# Return a subclass that will use a Worker subclass with
|
9
|
+
# a handle_args method defined by the given block. Allows
|
10
|
+
# for easily customizing/overriding the default argument
|
11
|
+
# handling.
|
12
|
+
def self.with_argument_handler(&block)
|
13
|
+
worker_subclass = Class.new(new.default_worker_class) do
|
14
|
+
define_method(:handle_args, &block)
|
15
|
+
end
|
16
|
+
Class.new(self) do
|
17
|
+
define_method(:default_worker_class){worker_subclass}
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
# Creates a new server. Arguments:
|
22
|
+
# socket_path: The path to the UNIX socket to create and listen on.
|
23
|
+
# argv: The arguments to the server, which are libraries to be required by default.
|
24
|
+
# debug: If set, operations on an existing server socket will be logged.
|
25
|
+
# If the value is <tt>'log'</tt>, <tt>$LOADED_FEATURES</tt> will also be logged to the stdout
|
26
|
+
# after libraries have been required.
|
27
|
+
# daemonize: Whether to daemonize, +true+ by default.
|
28
|
+
# daemon_args: Arguments to use when daemonizing, <tt>[false, false]</tt> by default.
|
29
|
+
# worker_class: The class to use for worker process handling, Worker by default.
|
30
|
+
def initialize(socket_path: default_socket_path, argv: default_argv, debug: default_debug,
|
31
|
+
daemonize: default_daemonize, daemon_args: default_daemon_args,
|
32
|
+
worker_class: default_worker_class)
|
33
|
+
@socket_path = socket_path
|
34
|
+
@argv = argv
|
35
|
+
@debug = debug
|
36
|
+
if @daemonize = !!daemonize
|
37
|
+
@daemon_args = Array(daemon_args)
|
38
|
+
end
|
39
|
+
@worker_class = worker_class
|
40
|
+
end
|
41
|
+
|
42
|
+
# The default socket path to use. Use the +BY_SOCKET+ environment variable if set,
|
43
|
+
# or <tt>~/.by_socket</tt> if not set.
|
44
|
+
def default_socket_path
|
45
|
+
ENV['BY_SOCKET'] || File.join(ENV["HOME"], '.by_socket')
|
46
|
+
end
|
47
|
+
|
48
|
+
# The default server arguments, uses +ARGV+ by default.
|
49
|
+
def default_argv
|
50
|
+
ARGV
|
51
|
+
end
|
52
|
+
|
53
|
+
# The default debug mode. This uses and removes the +DEBUG+ environment variable.
|
54
|
+
def default_debug
|
55
|
+
ENV.delete('DEBUG')
|
56
|
+
end
|
57
|
+
|
58
|
+
# The default for whether to daemonize. It is true if the +BY_SERVER_NO_DAEMON+
|
59
|
+
# environment variable is not set.
|
60
|
+
def default_daemonize
|
61
|
+
!ENV['BY_SERVER_NO_DAEMON']
|
62
|
+
end
|
63
|
+
|
64
|
+
# The default arguments when daemonizing. By default, considers the
|
65
|
+
# +BY_SERVER_DAEMON_NO_CHDIR+ and +BY_SERVER_DAEMON_NO_REDIR_STDIO+ environment
|
66
|
+
# variables.
|
67
|
+
def default_daemon_args
|
68
|
+
[!!ENV['BY_SERVER_DAEMON_NO_CHDIR'], !!ENV['BY_SERVER_DAEMON_NO_REDIR_STDIO']]
|
69
|
+
end
|
70
|
+
|
71
|
+
# The default worker class to use for worker processes, Worker by default.
|
72
|
+
def default_worker_class
|
73
|
+
Worker
|
74
|
+
end
|
75
|
+
|
76
|
+
# Runs the server. This will not terminate until the server receives SIGTERM
|
77
|
+
# or stop_accepting_clients! is called manually.
|
78
|
+
#
|
79
|
+
# If stop? is true, does not run a server, just handles an existing server.
|
80
|
+
def run
|
81
|
+
handle_existing_server
|
82
|
+
return if stop?
|
83
|
+
|
84
|
+
handle_argv
|
85
|
+
setup_server
|
86
|
+
daemonize if daemonize?
|
87
|
+
setup_signals
|
88
|
+
accept_clients
|
89
|
+
end
|
90
|
+
|
91
|
+
# Handle an existing server socket. This attempts to connect to the socket and
|
92
|
+
# then shutdown the server. If successful, it removes the socket. If unnecessful,
|
93
|
+
# it will print an error.
|
94
|
+
def handle_existing_server
|
95
|
+
if File.socket?(@socket_path)
|
96
|
+
begin
|
97
|
+
@socket = UNIXSocket.new(@socket_path)
|
98
|
+
print "Shutting down existing by_server at #{@socket_path}..." if @debug
|
99
|
+
raise "Invalid by_server worker pid" unless @socket.readline("\0", chomp: true).to_i > 1
|
100
|
+
@socket.send_io($stdin)
|
101
|
+
@socket.send_io($stdout)
|
102
|
+
@socket.send_io($stderr)
|
103
|
+
@socket.write("stop")
|
104
|
+
@socket.shutdown(Socket::SHUT_WR)
|
105
|
+
@socket.read
|
106
|
+
@socket.close
|
107
|
+
rescue => e
|
108
|
+
puts "FAILED!!!" if @debug
|
109
|
+
$stderr.puts "Error shutting down server on existing socket: #{e.class}: #{e.message}"
|
110
|
+
exit(1)
|
111
|
+
else
|
112
|
+
puts "Success!" if @debug
|
113
|
+
end
|
114
|
+
@socket = nil
|
115
|
+
File.delete(@socket_path)
|
116
|
+
end
|
117
|
+
end
|
118
|
+
|
119
|
+
# Whether to only stop an existing server and not start a new server.
|
120
|
+
def stop?
|
121
|
+
@argv == ['stop']
|
122
|
+
end
|
123
|
+
|
124
|
+
# Handle arguments provided to the server. Requires each argument by default.
|
125
|
+
def handle_argv
|
126
|
+
(auto_require_files + @argv).each{|f| require f}
|
127
|
+
print_loaded_features if @debug == 'log'
|
128
|
+
end
|
129
|
+
|
130
|
+
# Files to automatically require, uses the +BY_SERVER_AUTO_REQUIRE+ environment
|
131
|
+
# variable by default.
|
132
|
+
def auto_require_files
|
133
|
+
(ENV['BY_SERVER_AUTO_REQUIRE'] || '').split
|
134
|
+
end
|
135
|
+
|
136
|
+
# Creates and listens on the server socket.
|
137
|
+
def setup_server
|
138
|
+
# Prevent TOCTOU on server socket creation
|
139
|
+
umask = File.umask(077)
|
140
|
+
@socket = UNIXServer.new(@socket_path)
|
141
|
+
File.umask(umask)
|
142
|
+
system('chmod', '600', @socket_path)
|
143
|
+
end
|
144
|
+
|
145
|
+
# Daemonize with configured daemon args using Process.daemon.
|
146
|
+
def daemonize
|
147
|
+
Process.daemon(*@daemon_args)
|
148
|
+
end
|
149
|
+
|
150
|
+
# Whether to daemonize.
|
151
|
+
def daemonize?
|
152
|
+
!!@daemonize
|
153
|
+
end
|
154
|
+
|
155
|
+
# Trap SIGTERM and have it stop accepting clients.
|
156
|
+
# Trap SIGTERM and have it remove the socket and stop accepting clients.
|
157
|
+
def setup_signals
|
158
|
+
@sigquit_default = Signal.trap(:QUIT) do
|
159
|
+
stop_accepting_clients!
|
160
|
+
end
|
161
|
+
@sigterm_default = Signal.trap(:TERM) do
|
162
|
+
begin
|
163
|
+
File.delete(@socket_path)
|
164
|
+
rescue Errno::ENOENT
|
165
|
+
# server socket already deleted, ignore
|
166
|
+
end
|
167
|
+
stop_accepting_clients!
|
168
|
+
end
|
169
|
+
end
|
170
|
+
|
171
|
+
# Accept each client connection and fork a worker socket for it.
|
172
|
+
# Terminate loop when stop_accepting_clients! is called.
|
173
|
+
def accept_clients
|
174
|
+
while socket = accept_client
|
175
|
+
fork_worker(socket)
|
176
|
+
end
|
177
|
+
end
|
178
|
+
|
179
|
+
# Accept a new client connection, or return nil if
|
180
|
+
# stop_accepting_clients! has been called.
|
181
|
+
def accept_client
|
182
|
+
@socket.accept
|
183
|
+
rescue IOError, Errno::EBADF
|
184
|
+
# likely closed stream, return nil to exit accept_clients loop
|
185
|
+
nil
|
186
|
+
end
|
187
|
+
|
188
|
+
# Close the server socket. This will trigger the accept_clients
|
189
|
+
# loop to terminate.
|
190
|
+
def stop_accepting_clients!
|
191
|
+
@socket.close
|
192
|
+
end
|
193
|
+
|
194
|
+
# Fork a worker process to handle the client connection. Close the
|
195
|
+
# given socket after the fork, so the socket will open be open in
|
196
|
+
# the worker process.
|
197
|
+
def fork_worker(socket)
|
198
|
+
Process.detach(Process.fork do
|
199
|
+
Signal.trap(:QUIT, @sigquit_default)
|
200
|
+
Signal.trap(:TERM, @sigterm_default)
|
201
|
+
@worker_class.new(socket).run
|
202
|
+
end)
|
203
|
+
socket.close
|
204
|
+
end
|
205
|
+
|
206
|
+
# Print <tt>$LOADED_FEATURES</tt> to stdout.
|
207
|
+
def print_loaded_features
|
208
|
+
puts $LOADED_FEATURES
|
209
|
+
end
|
210
|
+
end
|
211
|
+
end
|
data/lib/by/worker.rb
ADDED
@@ -0,0 +1,164 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module By
|
4
|
+
class Worker
|
5
|
+
# Whether the worker process should signal a normal exit to the client.
|
6
|
+
# The default is nil, which signals a normal exit when the worker
|
7
|
+
# process exits normally. This can be set to false to signal an
|
8
|
+
# abnormal exit, such as to indicate test failures.
|
9
|
+
attr_writer :normal_exit
|
10
|
+
|
11
|
+
# Create the worker. Arguments
|
12
|
+
# socket :: The Unix socket to use to communicate with the client.
|
13
|
+
# sigterm_handler
|
14
|
+
def initialize(socket)
|
15
|
+
@socket = socket
|
16
|
+
@normal_exit = nil
|
17
|
+
end
|
18
|
+
|
19
|
+
# Run the worker process.
|
20
|
+
def run
|
21
|
+
write_pid
|
22
|
+
reopen_stdio
|
23
|
+
chdir_or_stop
|
24
|
+
replace_env
|
25
|
+
handle_args(get_args)
|
26
|
+
end
|
27
|
+
|
28
|
+
# Write the current process pid to the client, to signal that
|
29
|
+
# the worker process is ready.
|
30
|
+
def write_pid
|
31
|
+
@socket.write($$.to_s)
|
32
|
+
@socket.write("\0")
|
33
|
+
end
|
34
|
+
|
35
|
+
# Replace stdin, stdout, stderr with the IO values provided by the client.
|
36
|
+
def reopen_stdio
|
37
|
+
$stdin.reopen(@socket.recv_io(IO))
|
38
|
+
$stdout.reopen(@socket.recv_io(IO))
|
39
|
+
$stderr.reopen(@socket.recv_io(IO))
|
40
|
+
end
|
41
|
+
|
42
|
+
# Change to the given directory, unless the client is telling the
|
43
|
+
# worker to stop the server.
|
44
|
+
def chdir_or_stop
|
45
|
+
arg = @socket.readline("\0", chomp: true)
|
46
|
+
arg == 'stop' ? stop_server : chdir(arg)
|
47
|
+
end
|
48
|
+
|
49
|
+
# Stop the server process by sending it the SIGQUIT signal, then exit.
|
50
|
+
def stop_server
|
51
|
+
Process.kill(:QUIT, Process.ppid)
|
52
|
+
cleanup_proc.call
|
53
|
+
exit
|
54
|
+
end
|
55
|
+
|
56
|
+
# Change to the given directory.
|
57
|
+
def chdir(dir)
|
58
|
+
Dir.chdir(dir)
|
59
|
+
end
|
60
|
+
|
61
|
+
# Print <tt>$LOADED_FEATURES</tt> to stdout.
|
62
|
+
def print_loaded_features
|
63
|
+
puts $LOADED_FEATURES
|
64
|
+
end
|
65
|
+
|
66
|
+
# A proc for communicating exit status to the client, and
|
67
|
+
# printing the loaded features if configured. Used during
|
68
|
+
# process shutdown.
|
69
|
+
def cleanup_proc
|
70
|
+
proc do
|
71
|
+
if @normal_exit.nil?
|
72
|
+
@normal_exit = $!.nil? || ($!.is_a?(SystemExit) && $!.success?)
|
73
|
+
end
|
74
|
+
@socket.write(@normal_exit ? '0' : '1')
|
75
|
+
@socket.shutdown(Socket::SHUT_WR)
|
76
|
+
@socket.close
|
77
|
+
print_loaded_features if ENV['DEBUG'] == 'log'
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
# Replace ENV with the environment provided by the client.
|
82
|
+
def replace_env
|
83
|
+
env = {}
|
84
|
+
while line = @socket.readline("\0", chomp: true)
|
85
|
+
break if line.empty?
|
86
|
+
k, v = line.split("=")
|
87
|
+
env[k] = v
|
88
|
+
end
|
89
|
+
ENV.replace(env)
|
90
|
+
end
|
91
|
+
|
92
|
+
# An array of arguments provided by the client.
|
93
|
+
def get_args
|
94
|
+
args = []
|
95
|
+
while !@socket.eof?
|
96
|
+
args << @socket.readline("\0", chomp: true)
|
97
|
+
end
|
98
|
+
args
|
99
|
+
end
|
100
|
+
|
101
|
+
# Handle arguments provided by the client. Ensure correct
|
102
|
+
# client after handling the arguments.
|
103
|
+
def handle_args(args)
|
104
|
+
worker = self
|
105
|
+
cleanup = cleanup_proc
|
106
|
+
|
107
|
+
case arg = args.first
|
108
|
+
when 'm', /\.rb:\d+\z/
|
109
|
+
args.shift if arg == 'm'
|
110
|
+
ARGV.replace(args)
|
111
|
+
require 'm'
|
112
|
+
M.define_singleton_method(:exit!) do |exit_code|
|
113
|
+
worker.normal_exit = exit_code == true
|
114
|
+
cleanup.call
|
115
|
+
super(exit_code)
|
116
|
+
end
|
117
|
+
M.run(args)
|
118
|
+
when 'irb'
|
119
|
+
at_exit(&cleanup)
|
120
|
+
args.shift
|
121
|
+
ARGV.replace(args)
|
122
|
+
require 'irb'
|
123
|
+
IRB.start(__FILE__)
|
124
|
+
when '-e'
|
125
|
+
at_exit(&cleanup)
|
126
|
+
unless args.length >= 2
|
127
|
+
$stderr.puts 'no code specified for -e (RuntimeError)'
|
128
|
+
exit(1)
|
129
|
+
end
|
130
|
+
args.shift
|
131
|
+
code = args.shift
|
132
|
+
ARGV.replace(args)
|
133
|
+
::TOPLEVEL_BINDING.eval(code)
|
134
|
+
when String
|
135
|
+
args.shift
|
136
|
+
ARGV.replace(args)
|
137
|
+
|
138
|
+
begin
|
139
|
+
require File.expand_path(arg)
|
140
|
+
rescue
|
141
|
+
@normal_exit = false
|
142
|
+
at_exit(&cleanup)
|
143
|
+
raise
|
144
|
+
end
|
145
|
+
|
146
|
+
if defined?(Minitest) && Minitest.class_variable_get(:@@installed_at_exit)
|
147
|
+
Minitest.singleton_class.prepend(Module.new do
|
148
|
+
define_method(:run) do |argv|
|
149
|
+
super(argv).tap{|exit_code| p worker.normal_exit = exit_code == true}
|
150
|
+
end
|
151
|
+
end)
|
152
|
+
Minitest.after_run(&cleanup)
|
153
|
+
else
|
154
|
+
at_exit(&cleanup)
|
155
|
+
end
|
156
|
+
else
|
157
|
+
# no arguments
|
158
|
+
at_exit(&cleanup)
|
159
|
+
::TOPLEVEL_BINDING.eval($stdin.read)
|
160
|
+
end
|
161
|
+
end
|
162
|
+
end
|
163
|
+
end
|
164
|
+
|
metadata
ADDED
@@ -0,0 +1,104 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: by
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jeremy Evans
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2023-02-07 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: minitest-global_expectations
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0'
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: m
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - ">="
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '0'
|
41
|
+
description: |
|
42
|
+
by is a library preloader for ruby designed to speed up process startup.
|
43
|
+
It uses a client/server approach, where the server loads the libraries and
|
44
|
+
listens on a UNIX socket, and the client connects to that socket to run
|
45
|
+
processes. For each client connection, the server forks a worker process,
|
46
|
+
which uses the current directory, stdin, stdout, stderr, and environment
|
47
|
+
of the client process. The worker process then processes the arguments
|
48
|
+
provided by the client. The client process waits until the worker process
|
49
|
+
closes the socket, which the worker process attempts to do right before
|
50
|
+
it exits.
|
51
|
+
email: code@jeremyevans.net
|
52
|
+
executables:
|
53
|
+
- by
|
54
|
+
- by-server
|
55
|
+
- by-session
|
56
|
+
extensions: []
|
57
|
+
extra_rdoc_files:
|
58
|
+
- README.rdoc
|
59
|
+
- CHANGELOG
|
60
|
+
- MIT-LICENSE
|
61
|
+
files:
|
62
|
+
- CHANGELOG
|
63
|
+
- MIT-LICENSE
|
64
|
+
- README.rdoc
|
65
|
+
- bin/by
|
66
|
+
- bin/by-server
|
67
|
+
- bin/by-session
|
68
|
+
- lib/by/server.rb
|
69
|
+
- lib/by/worker.rb
|
70
|
+
homepage: http://github.com/jeremyevans/by
|
71
|
+
licenses:
|
72
|
+
- MIT
|
73
|
+
metadata:
|
74
|
+
bug_tracker_uri: https://github.com/jeremyevans/by/issues
|
75
|
+
changelog_uri: https://github.com/jeremyevans/by/blob/master/CHANGELOG
|
76
|
+
mailing_list_uri: https://github.com/jeremyevans/by/discussions
|
77
|
+
source_code_uri: https://github.com/jeremyevans/by
|
78
|
+
post_install_message:
|
79
|
+
rdoc_options:
|
80
|
+
- "--quiet"
|
81
|
+
- "--line-numbers"
|
82
|
+
- "--inline-source"
|
83
|
+
- "--title"
|
84
|
+
- 'by: Ruby library preloader'
|
85
|
+
- "--main"
|
86
|
+
- README.rdoc
|
87
|
+
require_paths:
|
88
|
+
- lib
|
89
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
90
|
+
requirements:
|
91
|
+
- - ">="
|
92
|
+
- !ruby/object:Gem::Version
|
93
|
+
version: '2.6'
|
94
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
95
|
+
requirements:
|
96
|
+
- - ">="
|
97
|
+
- !ruby/object:Gem::Version
|
98
|
+
version: '0'
|
99
|
+
requirements: []
|
100
|
+
rubygems_version: 3.4.1
|
101
|
+
signing_key:
|
102
|
+
specification_version: 4
|
103
|
+
summary: Ruby library preloader
|
104
|
+
test_files: []
|