aw 1.0.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +16 -6
- data/lib/aw/child_error.rb +3 -2
- data/lib/aw/fork.rb +57 -13
- data/lib/aw.rb +5 -5
- metadata +4 -6
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9b3e173f5eb48d67ae27b75d5d59330b8483119700ac8f9215eaf1f2ab2c7980
|
|
4
|
+
data.tar.gz: d8249389baeeeacffaefe74222a1686e2d311d32292df8e9b2665ba68c2d34a4
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 48aadec08cd55c084eb66e894ec0189e8fdcab084bcf33b6cba4d42cd739af2045561f05fc14a1463be2593a648ba3fc49514bcd262ee20ef010441668ba22b4
|
|
7
|
+
data.tar.gz: 942922f6a3b48f6027f4269378bbab2ba7c81dea9a9e76555b587d00e7f952f265be27e41d3eaf9ddb14bededb21f29cbfbf6f6a9c6a903c867ece7c197e0188
|
data/README.md
CHANGED
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|

|
|
10
10
|
|
|
11
|
-
Runs a block in a forked child process and brings back what it returned, or re-raises what it raised. What the block does to the process stays in the child; what it does outside, to files or to other processes,
|
|
11
|
+
Runs a block in a forked child process and brings back what it returned, or re-raises what it raised. What the block does to the process stays in the child; what it does outside, to files or to other processes, happens.
|
|
12
12
|
|
|
13
13
|
## Installation
|
|
14
14
|
|
|
@@ -17,7 +17,7 @@ Runs a block in a forked child process and brings back what it returned, or re-r
|
|
|
17
17
|
gem "aw"
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
Aw requires Ruby 3.3 or later and `fork(2)`: it
|
|
20
|
+
Aw requires Ruby 3.3 or later and `fork(2)`: it is tested with CRuby on Linux and macOS, and declines where `fork` is missing (Windows, JRuby) with a `NotImplementedError`. `Aw.supported?` tells beforehand.
|
|
21
21
|
|
|
22
22
|
## Usage
|
|
23
23
|
|
|
@@ -57,7 +57,7 @@ It reports the child only: an exception raised in the parent while waiting, such
|
|
|
57
57
|
|
|
58
58
|
The result travels through a pipe with `Marshal`, so it must be a marshalable value: numbers, strings, symbols, arrays, hashes, and instances of named classes whose state is marshalable. A `Proc`, an `IO`, an instance of an anonymous class, or an exception carrying such a thing (a `KeyError` raised by a hash that holds a `Proc` carries that hash) cannot come back; `fork!` then raises `Aw::ChildError`, which says what the block returned or raised and why Marshal refused it. Neither can an instance of a class the parent does not know: a class defined or required inside the block exists in the child only, and the parent cannot load it (`Aw::ChildError: … undefined class/module Foo`). A constant the parent registered for autoload is the exception: `Marshal` autoloads it, and so runs that file, in the parent. What the parent raises while loading the result, short of an interruption, is an `Aw::ChildError` too, never mistaken for an exception of the block: a `marshal_load` of one of its classes, a failed autoload, a structure too deep or too large for the parent. `fork?` transfers neither the value nor the exception, and is not concerned.
|
|
59
59
|
|
|
60
|
-
`Aw::ChildError` is also raised when the child died without delivering a result: killed by a signal that Ruby does not turn into an exception (`KILL`, `SEGV`), left on its own with `exit!` or `exec`, or left the block with a `break`, a `return` or a `throw`. Its `status` is the child's `Process::Status`, and tells the cases apart: `signaled?` for a crash, `exitstatus` 70 (`EX_SOFTWARE`) for a block that left without returning or raising, 0 for a result that cannot be transferred or loaded, and the block's own code for an `exit!` or an `exec`: a block calling `exit! 70` is reported as one that left without returning, and `exit! 0` as a child that did not deliver a result.
|
|
60
|
+
`Aw::ChildError` is also raised when the child died without delivering a result: killed by a signal that Ruby does not turn into an exception (`KILL`, `SEGV`), left on its own with `exit!` or `exec`, or left the block with a `break`, a `return` or a `throw`. Its `status` is the child's `Process::Status`, and tells the cases apart: `signaled?` for a crash, `exitstatus` 70 (`EX_SOFTWARE`) for a block that left without returning or raising, 74 (`EX_IOERR`) for a result the child could not write because the block closed the pipe to the parent (a block that closes every descriptor above the standard ones, as a daemon does), 0 for a result that cannot be transferred or loaded, and the block's own code for an `exit!` or an `exec`: a block calling `exit! 70` is reported as one that left without returning, and `exit! 0` as a child that did not deliver a result.
|
|
61
61
|
|
|
62
62
|
```ruby
|
|
63
63
|
Aw.fork! { -> {} }
|
|
@@ -74,11 +74,11 @@ The value that comes back is a copy: an object returns as a new object, and its
|
|
|
74
74
|
- The child leaves with `exit!` after flushing its standard output and error, whatever the block does to leave (returning, raising, `exit`, `break`, `return`, `throw`, a redefined `exit!`, a replaced stream whose `flush` throws): the parent's `at_exit` handlers run once, in the parent. An `at_exit` registered by the block itself never runs.
|
|
75
75
|
- The child's standard output and error go where the parent's go. Any other buffered `IO` opened by the parent and written by the block, `STDOUT` included when the block replaced `$stdout`, is not flushed, and that output is lost: flush it in the block, and in the parent before the call, or what the parent had buffered is written twice.
|
|
76
76
|
- The block runs with the usual interrupt handling, whatever mask the caller set with `Thread.handle_interrupt`: a `Timeout` inside the block, a `Thread#raise` or a signal reach it. A signal that Ruby turns into an exception (`INT`, `TERM`, `HUP`) is then re-raised in the parent, like any exception of the block.
|
|
77
|
-
- The message is length-prefixed and binary, whatever the default encodings: a process the block spawned and left running does not delay the result, as long as it does not write to the pipe. When the child delivers no result, though, the parent waits for the end of the pipe, so a process the block forked without `exec` and left running delays the `Aw::ChildError` until it exits (`spawn` and `system` do not hold the pipe).
|
|
77
|
+
- The message is length-prefixed and binary, whatever the default encodings: a process the block spawned and left running does not delay the result, as long as it does not write to the pipe. Only the child delivers: a process the block forked without a block, and which let the block return, leaves at once with status 0 and writes nothing. When the child delivers no result, though, the parent waits for the end of the pipe, so a process the block forked without `exec` and left running delays the `Aw::ChildError` until it exits (`spawn` and `system` do not hold the pipe).
|
|
78
78
|
- Aw has no timeout of its own: `fork!` and `fork?` wait until the child exits, whatever keeps it alive (a block that loops or sleeps, a child stopped by `SIGSTOP`, a standard stream whose `flush` blocks, a thread that goes on after the result was written). Wrap code you do not trust in a `Timeout` of your own.
|
|
79
|
-
- If the parent is interrupted while waiting (`Interrupt`, `Timeout`, `Thread#kill`), the child is killed and reaped before the interruption proceeds, with no window between the fork and that guard. A second interruption during the reaping
|
|
79
|
+
- If the parent is interrupted while waiting (`Interrupt`, `Timeout`, `Thread#kill`), the child is asked to leave with `TERM`, killed with `KILL` if it is still there half a second later, and reaped, before the interruption proceeds, with no window between the fork and that guard. A second interruption during the reaping waits for it: no zombie is left.
|
|
80
80
|
- No file descriptor is left open after a call.
|
|
81
|
-
- Forks can be nested.
|
|
81
|
+
- Forks can be nested. The `TERM` an interrupted parent sends to an outer fork reaches it as a `SignalException`, which lets it ask its own children to leave and reap them in turn, within the same half second. A fork that ignores `TERM` leaves orphans: the children of an outer one, when it is killed, and an inner one itself, as the fork above it is likely to be killed before it can kill it.
|
|
82
82
|
- A fork costs a few milliseconds; the pages of the parent are shared copy-on-write, and the cost grows with the size of the parent (about 2 ms at 50 MB, 20 ms at 700 MB).
|
|
83
83
|
|
|
84
84
|
## Things to know about fork
|
|
@@ -92,6 +92,16 @@ The value that comes back is a copy: an object returns as a new object, and its
|
|
|
92
92
|
- SimpleCov stores the coverage of a process from an `at_exit` handler, which `exit!` skips: the code the child runs is not in the parent's report. With `merge_subprocesses true` (`enable_for_subprocesses true` before SimpleCov 1.3), call `SimpleCov.result` in an `ensure` of the block to store the child's share; the parent merges it.
|
|
93
93
|
- The parent deserializes, with `Marshal`, what the child wrote; the child ran your block, which could reach the pipe and write anything. Aw isolates the side effects of code you trust; it is not a sandbox for code you do not. A hostile block can make `fork!` return or raise anything, make `fork?` answer `true`, choose the message of an `Aw::ChildError`, make the parent hold as much memory as it writes, run the parent's `marshal_load` methods and autoloads, and, from a continuation the parent captured (`callcc`), run the parent's code in the child: only the `status` of an `Aw::ChildError` comes from the system. Running as the parent's user, it can also signal the parent and leave processes behind; Aw kills and reaps the child only.
|
|
94
94
|
|
|
95
|
+
## Changes in 1.0.2
|
|
96
|
+
|
|
97
|
+
- Documentation: a fork that ignores `TERM` leaves orphans whether it is an outer one or an inner one, where 1.0.1 said so of an outer one only; Aw is tested on Linux and macOS, the BSDs no longer being named; what the block does outside the process happens.
|
|
98
|
+
|
|
99
|
+
## Changes in 1.0.1
|
|
100
|
+
|
|
101
|
+
- A process the block forked without a block, and which let the block return, no longer delivers a result in place of the child: it leaves without writing, as the child alone writes.
|
|
102
|
+
- An interrupted parent asks the child to leave with `TERM` before killing it, so that a nested fork reaps its own children, and no second interruption leaves a zombie.
|
|
103
|
+
- A result the child could not write because the block closed the pipe is an `Aw::ChildError` that says so, with `exitstatus` 74, instead of one saying that the block left without returning.
|
|
104
|
+
|
|
95
105
|
## Changes in 1.0
|
|
96
106
|
|
|
97
107
|
- An exception *returned* by the block is returned, not raised; only what the block raises is re-raised.
|
data/lib/aw/child_error.rb
CHANGED
|
@@ -8,8 +8,9 @@ module Aw
|
|
|
8
8
|
# a `marshal_load` that raises).
|
|
9
9
|
#
|
|
10
10
|
# The status tells the cases apart: `signaled?` for a crash, `exitstatus`
|
|
11
|
-
# 70 for a block that left without returning or raising,
|
|
12
|
-
#
|
|
11
|
+
# 70 for a block that left without returning or raising, 74 for a result
|
|
12
|
+
# the child could not write because the block closed the pipe, 0 for a
|
|
13
|
+
# result that cannot be transferred or loaded, the block's own code for an
|
|
13
14
|
# `exit!` or an `exec` of the block.
|
|
14
15
|
class ChildError < ::StandardError
|
|
15
16
|
# @return [Process::Status] the status of the child
|
data/lib/aw/fork.rb
CHANGED
|
@@ -17,17 +17,25 @@ module Aw
|
|
|
17
17
|
HEADER_SIZE = 8
|
|
18
18
|
CHUNK = 65_536
|
|
19
19
|
|
|
20
|
-
# Exit
|
|
21
|
-
# that called `exit!` itself
|
|
20
|
+
# Exit statuses of a child that failed to deliver, as opposed to a block
|
|
21
|
+
# that called `exit!` itself: EX_SOFTWARE and EX_IOERR in sysexits(3).
|
|
22
22
|
FAILURE = 70
|
|
23
|
+
UNWRITTEN = 74
|
|
23
24
|
LEFT = "the block left without returning or raising (break, return or throw), " \
|
|
24
25
|
"so the child delivered no result"
|
|
26
|
+
LOST = "the child could not write its result to the parent (the block closed or " \
|
|
27
|
+
"replaced the pipe between them)"
|
|
28
|
+
|
|
29
|
+
# How long a child asked to leave with TERM is given before KILL, in
|
|
30
|
+
# seconds, and how often it is checked meanwhile.
|
|
31
|
+
GRACE = 0.5
|
|
32
|
+
TICK = 0.01
|
|
25
33
|
|
|
26
34
|
# Captured once, so that a block redefining `Process.exit!` in the child
|
|
27
35
|
# cannot make the child fall through to the parent's `at_exit` handlers.
|
|
28
36
|
EXIT = ::Process.method(:exit!)
|
|
29
37
|
|
|
30
|
-
private_constant :HEADER, :HEADER_SIZE, :CHUNK, :FAILURE, :LEFT, :EXIT
|
|
38
|
+
private_constant :HEADER, :HEADER_SIZE, :CHUNK, :FAILURE, :UNWRITTEN, :LEFT, :LOST, :GRACE, :TICK, :EXIT
|
|
31
39
|
|
|
32
40
|
# @param block [Proc] the code to run in the child
|
|
33
41
|
# @return [Array(Symbol, Object)] `[:return, value]` or `[:raise, exception]`
|
|
@@ -64,11 +72,11 @@ module Aw
|
|
|
64
72
|
# parent set does not see the block, which would otherwise hand the
|
|
65
73
|
# child over to the parent's other fibers.
|
|
66
74
|
def self.child(block, reader, writer)
|
|
75
|
+
pid = ::Process.pid
|
|
67
76
|
::Fiber.blocking do
|
|
68
77
|
::Thread.handle_interrupt(::Object => :immediate) do
|
|
69
78
|
reader.close
|
|
70
|
-
|
|
71
|
-
writer.write([data.bytesize].pack(HEADER), data)
|
|
79
|
+
write(writer, dump(evaluate(block)), pid)
|
|
72
80
|
leave(0)
|
|
73
81
|
ensure
|
|
74
82
|
leave(FAILURE)
|
|
@@ -76,6 +84,19 @@ module Aw
|
|
|
76
84
|
end
|
|
77
85
|
end
|
|
78
86
|
|
|
87
|
+
# Writes the length-prefixed message, from the child alone: a process
|
|
88
|
+
# the block forked without a block of its own, and which let the block
|
|
89
|
+
# return, reaches this code too, with a copy of the pipe, and leaves at
|
|
90
|
+
# once, writing nothing. A child that cannot write, because the block
|
|
91
|
+
# closed the pipe or replaced its file descriptor with another file,
|
|
92
|
+
# leaves with the status that says so.
|
|
93
|
+
def self.write(writer, data, pid)
|
|
94
|
+
leave(0) unless ::Process.pid == pid
|
|
95
|
+
writer.write([data.bytesize].pack(HEADER), data)
|
|
96
|
+
rescue ::Exception # rubocop:disable Lint/RescueException
|
|
97
|
+
leave(UNWRITTEN)
|
|
98
|
+
end
|
|
99
|
+
|
|
79
100
|
# Flushes the standard streams of the child and leaves with the status.
|
|
80
101
|
# The exit is in an ensure of its own, so that a `throw` from the
|
|
81
102
|
# `flush` of a stream the block replaced cannot skip it.
|
|
@@ -145,6 +166,7 @@ module Aw
|
|
|
145
166
|
# exited normally; raises a ChildError that says why otherwise.
|
|
146
167
|
def self.deliver(data, status)
|
|
147
168
|
raise ChildError.new(LEFT, status:) if status.exitstatus == FAILURE
|
|
169
|
+
raise ChildError.new(LOST, status:) if status.exitstatus == UNWRITTEN
|
|
148
170
|
raise ChildError.new("the child did not deliver a result", status:) unless status.success? && data
|
|
149
171
|
|
|
150
172
|
kind, value = load(data, status)
|
|
@@ -170,20 +192,42 @@ module Aw
|
|
|
170
192
|
end
|
|
171
193
|
end
|
|
172
194
|
|
|
173
|
-
# Reaps a child the parent gave up waiting for
|
|
174
|
-
# is still
|
|
175
|
-
#
|
|
176
|
-
#
|
|
195
|
+
# Reaps a child the parent gave up waiting for: asked to leave, then
|
|
196
|
+
# killed if it is still there when the grace is over. Nothing interrupts
|
|
197
|
+
# the reaping, so that no child is left behind: a second interruption of
|
|
198
|
+
# the parent waits for it.
|
|
177
199
|
def self.reap(pid)
|
|
178
200
|
return if pid.nil?
|
|
179
|
-
return if ::Process.wait2(pid, ::Process::WNOHANG)
|
|
180
201
|
|
|
181
|
-
::
|
|
182
|
-
|
|
202
|
+
::Thread.handle_interrupt(::Object => :never) do
|
|
203
|
+
next if left?(pid)
|
|
204
|
+
|
|
205
|
+
::Process.kill(:KILL, pid)
|
|
206
|
+
::Process.wait(pid)
|
|
207
|
+
end
|
|
183
208
|
rescue ::Errno::ESRCH, ::Errno::ECHILD
|
|
184
209
|
nil
|
|
185
210
|
end
|
|
186
211
|
|
|
187
|
-
|
|
212
|
+
# Asks a child still running to leave with TERM, which lets it reap
|
|
213
|
+
# children of its own, and tells whether it had left and was reaped by
|
|
214
|
+
# the end of the grace, measured on the clock: a sleep may last longer
|
|
215
|
+
# than asked. The child is checked before the signal: a wait2
|
|
216
|
+
# interrupted right after it reaped would otherwise lead to a signal to
|
|
217
|
+
# a pid that may already belong to another process.
|
|
218
|
+
def self.left?(pid)
|
|
219
|
+
return true if ::Process.wait2(pid, ::Process::WNOHANG)
|
|
220
|
+
|
|
221
|
+
::Process.kill(:TERM, pid)
|
|
222
|
+
deadline = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) + GRACE
|
|
223
|
+
until ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) > deadline
|
|
224
|
+
sleep(TICK)
|
|
225
|
+
return true if ::Process.wait2(pid, ::Process::WNOHANG)
|
|
226
|
+
end
|
|
227
|
+
false
|
|
228
|
+
end
|
|
229
|
+
|
|
230
|
+
private_class_method :child, :write, :leave, :evaluate, :dump, :opaque, :flush, :read, :deliver, :load, :reap,
|
|
231
|
+
:left?
|
|
188
232
|
end
|
|
189
233
|
end
|
data/lib/aw.rb
CHANGED
|
@@ -7,7 +7,7 @@ require_relative "aw/fork"
|
|
|
7
7
|
# or re-raises what it raised. Side effects of the block on the process
|
|
8
8
|
# (mutations, globals, constants, the current directory, `ENV`, a crash)
|
|
9
9
|
# stay in the child; what it does outside, to files or to other processes,
|
|
10
|
-
#
|
|
10
|
+
# happens.
|
|
11
11
|
#
|
|
12
12
|
# The result travels through a pipe with `Marshal`, so it must be a
|
|
13
13
|
# marshalable value; a `Proc`, an `IO` or an instance of an anonymous class
|
|
@@ -31,8 +31,8 @@ module Aw
|
|
|
31
31
|
# @raise [ArgumentError] when no block is given
|
|
32
32
|
# @raise [NotImplementedError] when this Ruby cannot fork
|
|
33
33
|
# @raise [Exception] whatever the block raised, re-raised in the parent
|
|
34
|
-
# @raise [Aw::ChildError] when the child delivered no result (it died, the block called `exit
|
|
35
|
-
# left with `break`, `return` or `throw
|
|
34
|
+
# @raise [Aw::ChildError] when the child delivered no result (it died, the block called `exit!`,
|
|
35
|
+
# left with `break`, `return` or `throw`, or closed the pipe), or its result cannot be transferred or loaded
|
|
36
36
|
def self.fork!(&block)
|
|
37
37
|
raise ::ArgumentError, "a block must be provided" if block.nil?
|
|
38
38
|
|
|
@@ -55,8 +55,8 @@ module Aw
|
|
|
55
55
|
# @return [Boolean] true when the block returned, false when it raised
|
|
56
56
|
# @raise [ArgumentError] when no block is given
|
|
57
57
|
# @raise [NotImplementedError] when this Ruby cannot fork
|
|
58
|
-
# @raise [Aw::ChildError] when the child died, or the block called `exit
|
|
59
|
-
# `return` or `throw
|
|
58
|
+
# @raise [Aw::ChildError] when the child died, or the block called `exit!`, left with `break`,
|
|
59
|
+
# `return` or `throw`, or closed the pipe
|
|
60
60
|
def self.fork?(&block)
|
|
61
61
|
raise ::ArgumentError, "a block must be provided" if block.nil?
|
|
62
62
|
|
metadata
CHANGED
|
@@ -1,14 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: aw
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Cyril Kato
|
|
8
|
-
autorequire:
|
|
9
8
|
bindir: bin
|
|
10
9
|
cert_chain: []
|
|
11
|
-
date:
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
12
11
|
dependencies: []
|
|
13
12
|
description: Runs a block in a forked child process and brings back what it returned,
|
|
14
13
|
or re-raises what it raised. Side effects on the process stay in the child.
|
|
@@ -27,10 +26,10 @@ licenses:
|
|
|
27
26
|
- MIT
|
|
28
27
|
metadata:
|
|
29
28
|
bug_tracker_uri: https://github.com/fixrb/aw/issues
|
|
29
|
+
changelog_uri: https://github.com/fixrb/aw#changes-in-102
|
|
30
30
|
documentation_uri: https://rubydoc.info/gems/aw
|
|
31
31
|
rubygems_mfa_required: 'true'
|
|
32
32
|
source_code_uri: https://github.com/fixrb/aw
|
|
33
|
-
post_install_message:
|
|
34
33
|
rdoc_options: []
|
|
35
34
|
require_paths:
|
|
36
35
|
- lib
|
|
@@ -45,8 +44,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
45
44
|
- !ruby/object:Gem::Version
|
|
46
45
|
version: '0'
|
|
47
46
|
requirements: []
|
|
48
|
-
rubygems_version:
|
|
49
|
-
signing_key:
|
|
47
|
+
rubygems_version: 4.0.23
|
|
50
48
|
specification_version: 4
|
|
51
49
|
summary: Aw, fork❗
|
|
52
50
|
test_files: []
|