process_executer 4.1.1 → 4.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a6dbea0ce77694bd73783f7191fc9b32957fefda87f9db6108d97ebf26bb1a9f
4
- data.tar.gz: e710378f301c56c649063d6bfef0525f0c0c8dcf6adf6329e12184ec24d02894
3
+ metadata.gz: 58884682b61a416e64d16dabe3f8192a129e4281d6e315c5cb676256dee9afd2
4
+ data.tar.gz: 2f08fa0b9a801eebd7ae2e24699c460edab33272e89801f4c59f839196b61a24
5
5
  SHA512:
6
- metadata.gz: 756a9aacc5d173fa2cedd68f178c9e321e141d3ef75f61ddcfe7e3ebe67f236a0603192f24c49aaedbc0dcf209d61dd16b593477f67d974fa1f2e875ec17dd7d
7
- data.tar.gz: a9ce354984f37839c54abfb453eef726d6467467dc529aaa2108643c2e153005022b097b9a1ecfea368391ff9ebbf8ba20b32b9cd73db11c5076394824d81528
6
+ metadata.gz: f5ab2e6ee671a90fba84d7531c38b9eb8928bd2f8f93948cb1a577551126aa996555566a0ca6a4fc95b5e536374638e53b2218ab4d45088c11932318b59c9085
7
+ data.tar.gz: f929f097160797d721c4918f030c597943c1cad830877869416d553bc4c9c0ffedc16ce989eae9b07ae7990c2ffdec023799b10dbad356fa06fe73749a52dcc9
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "4.1.1"
2
+ ".": "4.1.2"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -5,6 +5,24 @@ All notable changes to the process_executer gem will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [4.1.2](https://github.com/main-branch/process_executer/compare/v4.1.1...v4.1.2) (2026-11-15)
9
+
10
+
11
+ ### Bug Fixes
12
+
13
+ * Don't report SpawnError or leave the subprocess running when interrupted just after the spawn ([0941723](https://github.com/main-branch/process_executer/commit/09417234a869f4027b6904df863c380500daae44))
14
+ * Point the caller's stdout and stderr back when a failed spawn on Windows MRI redirects them ([590014f](https://github.com/main-branch/process_executer/commit/590014f6140108b3b51d2e05f0f49e1767605e28)), closes [#211](https://github.com/main-branch/process_executer/issues/211)
15
+ * Stop a missing chdir: from breaking the caller's stdout and stderr on Windows MRI ([eed3773](https://github.com/main-branch/process_executer/commit/eed377302ee68cb6706012cb1530ac2d56dc588a)), closes [#204](https://github.com/main-branch/process_executer/issues/204)
16
+
17
+
18
+ ### Other Changes
19
+
20
+ * Run the TruffleRuby 24 job on ubuntu-24.04, the newest image it has a build for ([83e3576](https://github.com/main-branch/process_executer/commit/83e357602ed6d54fa679b532b266e515be23fc94))
21
+ * Show that a bad chdir: on Windows MRI breaks the caller's stdout and stderr ([932e31b](https://github.com/main-branch/process_executer/commit/932e31bf98a23fe937c59dc0357f1260693765b7)), closes [#204](https://github.com/main-branch/process_executer/issues/204)
22
+ * Show that a chdir: failure the check can't see still breaks stdout and stderr on Windows MRI ([7e4e2a9](https://github.com/main-branch/process_executer/commit/7e4e2a94c2055930000e9278ad64380fb5a623bf)), closes [#211](https://github.com/main-branch/process_executer/issues/211)
23
+ * Show that an exception raised into the caller just after the spawn is reported as SpawnError ([5bb4799](https://github.com/main-branch/process_executer/commit/5bb4799f61a8a30f5c88fd84b22ae455b3eb55bd))
24
+ * Write to a closed stdout with the shell's echo so the example passes on Ubuntu 26.04 ([1bb0f77](https://github.com/main-branch/process_executer/commit/1bb0f77a655618df319d8866972bda0182ac3641))
25
+
8
26
  ## [4.1.1](https://github.com/main-branch/process_executer/compare/v4.1.0...v4.1.1) (2026-08-31)
9
27
 
10
28
 
@@ -4,6 +4,7 @@ require 'English'
4
4
 
5
5
  require_relative '../errors'
6
6
  require_relative 'spawn_with_timeout'
7
+ require_relative 'standard_stream_guard'
7
8
 
8
9
  module ProcessExecuter
9
10
  module Commands
@@ -60,6 +61,7 @@ module ProcessExecuter
60
61
  #
61
62
  def call
62
63
  @opened_pipes = {}
64
+ @standard_stream_guard = nil
63
65
  @redirection_overrides = internal_redirections
64
66
  wrap_stdout_stderr
65
67
  super.tap do
@@ -122,6 +124,32 @@ module ProcessExecuter
122
124
  #
123
125
  def spawn_options = super.merge(redirection_overrides)
124
126
 
127
+ # Spawn the subprocess, undoing a stdout or stderr leak this class caused
128
+ #
129
+ # On Windows, MRI points this process's stdout and stderr at the
130
+ # redirection targets before it spawns and doesn't point them back when
131
+ # a later option fails, which is [Ruby Bug
132
+ # #22540](https://bugs.ruby-lang.org/issues/22540). When the targets are
133
+ # this class's pipes, the caller would be left writing into pipes nobody
134
+ # reads, so the spawn runs inside a {StandardStreamGuard}. A call that
135
+ # attached no pipe -- no `out:` or `err:`, or only `[:child, fd]` and
136
+ # `:close` destinations -- spawns unguarded and fails as `Process.spawn`
137
+ # fails.
138
+ #
139
+ # Remove this override with {StandardStreamGuard}; the condition is in
140
+ # that class's comment.
141
+ #
142
+ # @return [Integer] the pid of the subprocess
143
+ #
144
+ def spawn_subprocess
145
+ return super unless windows_mri? && !opened_pipes.empty?
146
+
147
+ # :nocov: runs only on Windows MRI
148
+ @standard_stream_guard = StandardStreamGuard.new(opened_pipes)
149
+ @standard_stream_guard.call { super }
150
+ # :nocov:
151
+ end
152
+
125
153
  # Wrap the stdout and stderr redirection options with a MonitoredPipe
126
154
  #
127
155
  # Each wrapper pipe is recorded in two collections with different roles:
@@ -158,6 +186,11 @@ module ProcessExecuter
158
186
  # and that exception (not a pipe destination error or a pipe cleanup
159
187
  # error) must be the one the caller sees, so nothing is raised here.
160
188
  #
189
+ # Nothing is raised either when the standard stream guard closed the
190
+ # pipes: the spawn did not return, so `#call` is being unwound even when
191
+ # no exception is in flight (a `throw` or `Thread#kill`), and what the
192
+ # pipes hold is what the guard's leak check left in them.
193
+ #
161
194
  # @param in_flight_error [Exception, nil] the exception `#call` is unwinding from, if any
162
195
  #
163
196
  # @raise [ProcessExecuter::ProcessIOError] if a pipe recorded a destination
@@ -169,7 +202,7 @@ module ProcessExecuter
169
202
  #
170
203
  def close_pipes_and_check_errors(in_flight_error)
171
204
  close_error = close_pipes
172
- return if in_flight_error
205
+ return if in_flight_error || @standard_stream_guard&.pipes_closed?
173
206
 
174
207
  opened_pipes.each do |option_key, pipe|
175
208
  raise_pipe_error(option_key, pipe)
@@ -190,9 +223,14 @@ module ProcessExecuter
190
223
  # whose drain is cut short by the deadline records it via
191
224
  # {MonitoredPipe#truncated?}.
192
225
  #
226
+ # Does nothing when a {StandardStreamGuard} already closed every pipe
227
+ # while checking a failed spawn for a leak.
228
+ #
193
229
  # @return [StandardError, nil] the first error raised while closing, or nil if none was raised
194
230
  #
195
231
  def close_pipes
232
+ return nil if @standard_stream_guard&.pipes_closed?
233
+
196
234
  first_close_error = nil
197
235
  deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + MonitoredPipe::DEFAULT_CLOSE_TIMEOUT
198
236
  opened_pipes.each_value do |pipe|
@@ -47,13 +47,7 @@ module ProcessExecuter
47
47
  # @return [ProcessExecuter::Result] The result of the completed subprocess
48
48
  #
49
49
  def call
50
- begin
51
- @effective_spawn_options = spawn_options
52
- @pid = Process.spawn(*command, **effective_spawn_options)
53
- rescue StandardError => e
54
- raise ProcessExecuter::SpawnError, "Failed to spawn process: #{e.message}"
55
- end
56
-
50
+ spawn_and_handle_errors
57
51
  wait_for_process
58
52
  end
59
53
 
@@ -181,6 +175,117 @@ module ProcessExecuter
181
175
  windows? ? { new_pgroup: true } : { pgroup: true }
182
176
  end
183
177
 
178
+ # Spawn the subprocess, sorting a spawn failure from a later interruption
179
+ #
180
+ # An error raised before the subprocess exists is a spawn failure and is
181
+ # reported as a {ProcessExecuter::SpawnError}.
182
+ #
183
+ # An exception that arrives once {#pid} is set was not raised by the
184
+ # spawn: the subprocess is running and another thread raised into this
185
+ # one (`Thread#raise`, which `Timeout.timeout` also uses). It is treated
186
+ # as an abandoned wait -- {#kill_and_reap_abandoned_subprocess}, then the
187
+ # exception propagates as it is -- so the caller is never told that
188
+ # nothing ran when something did. {Run} makes this case likely on
189
+ # Windows MRI, where it defers such exceptions for the length of the
190
+ # spawn (see {StandardStreamGuard}).
191
+ #
192
+ # The cleanup is in an `ensure`, not a `rescue`, because the caller can
193
+ # also be unwound by a `throw`, which no `rescue` sees: `Timeout.timeout`
194
+ # with no exception class unwinds that way on Ruby 3.1 and 3.2.
195
+ #
196
+ # @return [void]
197
+ #
198
+ # @raise [ProcessExecuter::SpawnError] `Process.spawn` raised an error
199
+ # before the command was run
200
+ #
201
+ def spawn_and_handle_errors
202
+ @pid = nil
203
+ @effective_spawn_options = spawn_options
204
+ spawn_subprocess
205
+ spawn_completed = true
206
+ rescue StandardError => e
207
+ raise(pid ? e : ProcessExecuter::SpawnError.new("Failed to spawn process: #{e.message}"))
208
+ ensure
209
+ kill_and_reap_abandoned_subprocess if pid && !spawn_completed
210
+ end
211
+
212
+ # Spawn the subprocess and record its pid
213
+ #
214
+ # The pid is recorded here, in the same step as the spawn, so that a
215
+ # subclass that wraps this method in `Thread.handle_interrupt` has the
216
+ # pid on record before a deferred exception is delivered. Subclasses may
217
+ # override this method to wrap the spawn.
218
+ #
219
+ # @return [Integer] the pid of the subprocess
220
+ #
221
+ def spawn_subprocess
222
+ check_chdir if windows_mri?
223
+ @pid = Process.spawn(*command, **effective_spawn_options)
224
+ end
225
+
226
+ # Whether this is MRI on Windows
227
+ #
228
+ # @return [Boolean]
229
+ #
230
+ def windows_mri? = windows? && RUBY_ENGINE == 'ruby'
231
+
232
+ # :nocov: runs only on Windows MRI
233
+
234
+ # Raise the error Process.spawn would raise for a chdir: it can't enter
235
+ #
236
+ # A workaround for [Ruby Bug
237
+ # #22540](https://bugs.ruby-lang.org/issues/22540). On Windows, MRI
238
+ # applies env, umask, and redirections to this process before it changes
239
+ # directory and doesn't undo them when the change fails, so a bad
240
+ # directory must not reach `Process.spawn`.
241
+ #
242
+ # `File.directory?` answers for an ordinary directory with no side
243
+ # effects. It can't be the whole check: it says no to paths `Process.spawn`
244
+ # accepts (`...`, `//./NUL`), and the error class for a bad path can't be
245
+ # worked out from file tests. So for a path it rejects, a block-form
246
+ # `Dir.chdir` asks Windows the question `Process.spawn` would ask, which
247
+ # changes the working directory for a moment, as every spawn with
248
+ # `chdir:` does.
249
+ #
250
+ # What the check does not catch, each of which fails at `Process.spawn`
251
+ # as it would without the check ({Run} then repairs stdout and stderr,
252
+ # see {StandardStreamGuard}):
253
+ #
254
+ # * a directory the process can't enter, which passes `File.directory?`
255
+ # * a directory removed between the check and the spawn
256
+ # * any bad directory while another thread is inside a `Dir.chdir`
257
+ # block, because Ruby refuses the trial `Dir.chdir`
258
+ #
259
+ # A bad `chdir:` is reported ahead of a `TypeError` or `ArgumentError`
260
+ # that `Process.spawn` would have raised first for another option.
261
+ #
262
+ # Remove this method when the gem's minimum supported Ruby is a release
263
+ # line in which every patch version has the fix for Ruby Bug #22540.
264
+ # Until then, once the fixed versions are known, gate the call on
265
+ # `RUBY_VERSION`.
266
+ #
267
+ # @return [void]
268
+ #
269
+ # @raise [SystemCallError] the error `Process.spawn` would raise for the
270
+ # directory, with the path as its message
271
+ #
272
+ def check_chdir
273
+ return unless effective_spawn_options.key?(:chdir)
274
+
275
+ dir = effective_spawn_options[:chdir]
276
+ return if File.directory?(dir)
277
+
278
+ Dir.chdir(dir) { nil }
279
+ rescue SystemCallError => e
280
+ raise e.class, (dir.respond_to?(:to_path) ? dir.to_path : dir)
281
+ rescue StandardError
282
+ # Not a path, or another thread is inside a Dir.chdir block: let
283
+ # Process.spawn report it
284
+ nil
285
+ end
286
+
287
+ # :nocov:
288
+
184
289
  # Whether the current platform is Windows
185
290
  #
186
291
  # @return [Boolean]
@@ -0,0 +1,457 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'monitor'
4
+
5
+ module ProcessExecuter
6
+ module Commands
7
+ # Puts stdout and stderr back after a failed spawn left them pointed at pipes
8
+ #
9
+ # A workaround for [Ruby Bug #22540](https://bugs.ruby-lang.org/issues/22540),
10
+ # used only on Windows MRI. There, `Process.spawn` points this process's
11
+ # stdout and stderr at the redirection targets before it spawns and doesn't
12
+ # point them back when a later option fails, such as a `chdir:` the process
13
+ # can't enter. When the targets are {MonitoredPipe}s that {Run} attached, the
14
+ # caller is left writing into pipes nobody reads -- something
15
+ # `Process.spawn` given the caller's own options would not have done.
16
+ #
17
+ # {#call} copies descriptors 1 and 2, runs the spawn, and after a spawn that
18
+ # doesn't return, points the descriptors back at the copies -- but only when
19
+ # the pipes show that they leaked. A restore after every failed spawn would
20
+ # disturb other threads for failures that leak nothing: on Ruby 3.1 through
21
+ # 3.4, reopening descriptor 1 or 2 raises `IOError` in any thread that is
22
+ # writing to it.
23
+ #
24
+ # What this class does not repair:
25
+ #
26
+ # * Env, umask, and stdin. They leak the same way through `Process.spawn`,
27
+ # so this gem leaves the caller no worse off than `Process.spawn` does.
28
+ # * Ruby's descriptor leak. Each spawn that fails this way leaves open the
29
+ # two copies Ruby itself made of the descriptors it redirected, and Ruby
30
+ # code can't find them to close them. After about 4,090 such failures the
31
+ # process is out of descriptors and every later call that attaches a pipe
32
+ # raises {ProcessExecuter::SpawnError} (`Too many open files`) at once;
33
+ # stdout and stderr still work.
34
+ # * A thread writing to stdout or stderr while a leak is repaired. It gets
35
+ # an `Errno::EPIPE` from the pipe whose reader the leak check closed, and
36
+ # on Ruby 3.1 through 3.4 an `IOError` from the reopen. Without the repair
37
+ # every later write fails.
38
+ # * A stream the caller closed before the call. There is no copy to put
39
+ # back, so a leaked descriptor stays pointed at the pipe and closing the
40
+ # pipes waits out {MonitoredPipe::DEFAULT_CLOSE_TIMEOUT}.
41
+ #
42
+ # ## Known limits
43
+ #
44
+ # Each needs code that is unlikely to be written, and each fix would cost
45
+ # more than the case is worth:
46
+ #
47
+ # * Code that runs while {LOCK} is held must not wait for another thread
48
+ # to run a command through this class: the two deadlock. Such code runs
49
+ # in two places. `Process.spawn` calls coercion methods such as an env
50
+ # value's `#to_str`. And while a leak is checked, a pipe's destination
51
+ # may be in `#write` with what another thread wrote to the leaked
52
+ # stream; the check waits for that write to return. The thread that
53
+ # holds the lock can itself run a command.
54
+ # * A signal handler that raises `IOError` or a `SystemCallError` is taken
55
+ # for an I/O failure. Raised while a pipe is closed for the leak check,
56
+ # it is dropped and the caller gets the spawn's error. Raised during the
57
+ # restore, it ends the restore, and a leaked stream stays pointed at its
58
+ # pipe. A handler that raises any other class, or throws, is handled.
59
+ # * A signal handler that raises or throws while the copies are being
60
+ # closed leaves one or both of them open until garbage collection closes
61
+ # them. Stdout and stderr are not affected.
62
+ #
63
+ # ## Operations this class avoids
64
+ #
65
+ # While a descriptor is leaked, another thread may be writing to stdout or
66
+ # stderr. Ruby writes to descriptors 1 and 2 with a synchronous call, but
67
+ # the pipe they now point at was opened for overlapped I/O, and on such a
68
+ # handle a synchronous write that has to wait can miss its completion when
69
+ # any other operation touches the same pipe end. The writer then waits
70
+ # forever holding the C runtime's lock on the descriptor, and the next
71
+ # thread to use that descriptor while holding the GVL stops the process.
72
+ # That is a Ruby bug of its own, tracked in
73
+ # [issue 210](https://github.com/main-branch/process_executer/issues/210).
74
+ #
75
+ # Two things Ruby does are such operations, so nothing here does either to
76
+ # a pipe end or to a standard stream:
77
+ #
78
+ # * `IO#close`, which calls `UnlockFile` on the handle. A descriptor is
79
+ # pointed at the null device with `IO#reopen` first, which swaps the
80
+ # handle under the C runtime's lock, and then closed. That goes for the
81
+ # copies of the standard streams and for each pipe's own write end.
82
+ # * `IO.for_fd`, which queries the handle. Copies are made with `IO#dup`,
83
+ # and the two IO objects a restore reopens are made once per process,
84
+ # before any spawn.
85
+ #
86
+ # The code this leads to looks roundabout, and the direct form of each step
87
+ # (`copy.close`, `IO.for_fd(fd)`, `STDOUT.reopen`) passes every example.
88
+ # No example can show the difference: the hang is a race with no
89
+ # deterministic trigger. Before simplifying anything here, run the script
90
+ # in issue 210 against the change. Code outside this class that closes,
91
+ # stats, or wraps a standard descriptor while one is leaked can still
92
+ # trigger the hang.
93
+ #
94
+ # Remove this class, and its use in {Run}, when the gem's minimum supported
95
+ # Ruby is a release line in which every patch version has the fix for Ruby
96
+ # Bug #22540. Until then, once the fixed versions are known, gate its use on
97
+ # `RUBY_VERSION` so a fixed Ruby doesn't pay for the copies and the lock.
98
+ #
99
+ # @api private
100
+ #
101
+ class StandardStreamGuard
102
+ # Serializes the save, the spawn, and the restore across threads
103
+ #
104
+ # Without it, one thread can save descriptors that another thread's
105
+ # failed spawn has just leaked, and then restore the leak.
106
+ #
107
+ # A Monitor, not a Mutex, so the thread that holds it can enter again:
108
+ # `Process.spawn` calls coercion methods such as an env value's
109
+ # `#to_str` inside the lock, and one of those may itself run a command
110
+ # through this class.
111
+ #
112
+ # @return [Monitor]
113
+ #
114
+ LOCK = Monitor.new
115
+
116
+ # The descriptors to guard and the IO objects that own them
117
+ #
118
+ # The constants and not `$stdout` and `$stderr`: a caller can point the
119
+ # globals at a `StringIO`, and the leak is in descriptors 1 and 2.
120
+ #
121
+ # @return [Hash{Integer => IO}]
122
+ #
123
+ STREAMS = { 1 => STDOUT, 2 => STDERR }.freeze # rubocop:disable Style/GlobalStdStream
124
+
125
+ # :nocov: the rest of this class runs only on Windows MRI
126
+
127
+ # Create a guard for a spawn that redirects to the given pipes
128
+ #
129
+ # @example
130
+ # pipes = { out: ProcessExecuter::MonitoredPipe.new(StringIO.new) }
131
+ # guard = ProcessExecuter::Commands::StandardStreamGuard.new(pipes)
132
+ #
133
+ # @param pipes [Hash{Object => ProcessExecuter::MonitoredPipe}] the pipes
134
+ # the spawn redirects stdout and stderr to
135
+ #
136
+ def initialize(pipes)
137
+ @pipes = pipes
138
+ @pipes_closed = false
139
+ end
140
+
141
+ # Whether {#call} closed the pipes while checking them for a leak
142
+ #
143
+ # True after a spawn that didn't return: the leak check has to close every
144
+ # pipe to see whether it reaches end-of-file, so the owner of the pipes
145
+ # must not close them again or report what the check left in them.
146
+ #
147
+ # @example
148
+ # guard.pipes_closed? #=> false
149
+ #
150
+ # @return [Boolean]
151
+ #
152
+ def pipes_closed? = @pipes_closed
153
+
154
+ # Run the spawn in the block, undoing a stdout or stderr leak if it is unwound
155
+ #
156
+ # The block can be left by an exception of any class or by a `throw`; the
157
+ # leak is checked for and undone in each case.
158
+ #
159
+ # Exceptions raised from other threads are deferred for the length of the
160
+ # block (one spawn). One that landed between the failed spawn and the
161
+ # restore would leave the leak in place and then close the copies the
162
+ # restore needed. A deferred exception is delivered as this method
163
+ # returns, including after a spawn that succeeded, so the block must
164
+ # record the pid itself for the caller to know a subprocess is running.
165
+ # The mask is on `Object`, not `Exception`, so that it also defers
166
+ # `Thread#kill`, which is not delivered as an exception.
167
+ #
168
+ # A signal handler is not deferred: it runs on the main thread whenever
169
+ # the signal arrives, and may raise or throw there. So the check and the
170
+ # restore are started again, from an `ensure`, each time they are unwound
171
+ # before they finish; what unwound them carries on once they are done.
172
+ #
173
+ # @example
174
+ # pid = guard.call { Process.spawn('echo hello', out: pipes[:out]) }
175
+ #
176
+ # @yield the call to `Process.spawn`
177
+ #
178
+ # @return [Object] the value of the block
179
+ #
180
+ # @raise [Exception] whatever the block raised
181
+ #
182
+ # @raise [SystemCallError] the null device could not be opened or a
183
+ # stream could not be copied, as when the process is out of
184
+ # descriptors; the block was not run
185
+ #
186
+ def call(&spawn)
187
+ LOCK.synchronize do
188
+ # Opened now, not on first use: the leak check and the release both
189
+ # need it, and neither can do its work if the open fails then
190
+ self.class.null_device
191
+ Thread.handle_interrupt(Object => :never) { spawn_and_repair(save, &spawn) }
192
+ end
193
+ end
194
+
195
+ private
196
+
197
+ # The pipes the spawn redirects stdout and stderr to
198
+ #
199
+ # @return [Hash{Object => ProcessExecuter::MonitoredPipe}]
200
+ #
201
+ attr_reader :pipes
202
+
203
+ # Run the spawn, restore the saved descriptors if it leaked, and close the copies
204
+ #
205
+ # A flag and an `ensure`, not a `rescue`: a `throw` or a `Thread#kill`
206
+ # leaves the block without raising anything a `rescue` could see.
207
+ #
208
+ # @param copies [Hash{Integer => IO}] each descriptor and its copy
209
+ #
210
+ # @yield the call to `Process.spawn`
211
+ #
212
+ # @return [Object] the value of the block
213
+ #
214
+ def spawn_and_repair(copies)
215
+ completed = false
216
+ result = yield
217
+ completed = true
218
+ result
219
+ ensure
220
+ completed ? release(copies) : repair_and_release(copies)
221
+ end
222
+
223
+ # Restore the saved descriptors if the spawn leaked, then close the copies
224
+ #
225
+ # Starts again if it is unwound before it finishes, which a signal
226
+ # handler that raises or throws can do at any point. Every step can be
227
+ # repeated: closing a closed pipe and pointing a descriptor at its copy
228
+ # a second time change nothing.
229
+ #
230
+ # The leak check comes before the restore, and must: it closes each
231
+ # pipe's reader, which makes a write another thread has in progress on a
232
+ # leaked descriptor fail. Restoring first would wait for such a write
233
+ # with the GVL held, while the thread that could let it finish by
234
+ # reading the pipe waits for the GVL.
235
+ #
236
+ # An error from the restore or from closing a copy is raised without
237
+ # starting again, since a second try would fail the same way.
238
+ #
239
+ # @param copies [Hash{Integer => IO}] each descriptor and its copy
240
+ #
241
+ # @return [void]
242
+ #
243
+ def repair_and_release(copies)
244
+ restore(copies) if leaked?
245
+ release(copies)
246
+ finished = true
247
+ rescue IOError, SystemCallError
248
+ finished = true
249
+ release(copies)
250
+ raise
251
+ ensure
252
+ repair_and_release(copies) unless finished
253
+ end
254
+
255
+ # Close the copies of descriptors 1 and 2
256
+ #
257
+ # Each copy is taken out of the hash before it is closed, so that a call
258
+ # repeated after an interruption never closes a copy twice.
259
+ #
260
+ # @param copies [Hash{Integer => IO}] each descriptor and its copy
261
+ #
262
+ # @return [void]
263
+ #
264
+ def release(copies)
265
+ until copies.empty?
266
+ _fd, copy = copies.shift
267
+ discard(copy)
268
+ end
269
+ end
270
+
271
+ # Close a copy of a standard stream without touching the stream itself
272
+ #
273
+ # The copy is pointed at the null device before it is closed. Ruby's close
274
+ # on Windows calls `UnlockFile` on the handle, and the copy and the stream
275
+ # are the same open file; see "Operations this class avoids" in the class
276
+ # comment. Both lines are needed: `copy.close` alone is the call to avoid.
277
+ #
278
+ # @param copy [IO] a copy made by {#copy_of}
279
+ #
280
+ # @return [void]
281
+ #
282
+ def discard(copy)
283
+ copy.reopen(self.class.null_device)
284
+ copy.close
285
+ end
286
+
287
+ # Copy descriptors 1 and 2 so they can be put back
288
+ #
289
+ # A stream the caller closed has no copy.
290
+ #
291
+ # A copy that can't be made, as when the process is out of descriptors,
292
+ # is an error: a spawn run without it could leak a stream that nothing
293
+ # could put back. The copies already made are closed first.
294
+ #
295
+ # @return [Hash{Integer => IO}] each descriptor and its copy
296
+ #
297
+ # @raise [SystemCallError] a stream could not be copied
298
+ #
299
+ def save
300
+ copies = {}
301
+ STREAMS.each do |fd, io|
302
+ next if io.closed?
303
+
304
+ # The result is not used here. The call makes the object {#restore}
305
+ # will need, now, while the descriptor is not leaked.
306
+ self.class.target(fd)
307
+ copies[fd] = copy_of(io)
308
+ end
309
+ copies
310
+ rescue SystemCallError
311
+ release(copies)
312
+ raise
313
+ end
314
+
315
+ # Copy a standard stream to a descriptor above 2
316
+ #
317
+ # A copy that lands on 0, 1 or 2 (the caller closed one of them) could be
318
+ # overwritten by the spawn's own redirections, so it is held open until a
319
+ # higher one is made.
320
+ #
321
+ # `IO#dup` and not `fcntl(F_DUPFD, 3)`, which would do this in one step:
322
+ # the descriptor number it returns needs `IO.for_fd` to become an IO.
323
+ #
324
+ # @param io [IO] STDOUT or STDERR
325
+ #
326
+ # @return [IO] the copy
327
+ #
328
+ def copy_of(io)
329
+ copy = io.dup
330
+ return copy if copy.fileno > 2
331
+
332
+ begin
333
+ copy_of(io)
334
+ ensure
335
+ discard(copy)
336
+ end
337
+ end
338
+
339
+ # Whether the failed spawn left a descriptor pointed at one of the pipes
340
+ #
341
+ # The spawn failed, so there is no subprocess: once a pipe's own write
342
+ # end is closed, only a leaked descriptor in this process can keep the
343
+ # pipe from reaching end-of-file. Closing with a zero timeout reads the
344
+ # pipe once and reports anything short of end-of-file as truncated.
345
+ #
346
+ # A signal handler can also unwind the block just after a spawn that
347
+ # succeeded. The subprocess then holds the pipes open and this reports a
348
+ # leak that isn't there; the restore that follows points each descriptor
349
+ # at a copy of itself, which changes nothing.
350
+ #
351
+ # A pipe whose destination raised on a write also counts as leaked. With
352
+ # no subprocess the data it was writing can only have come through a
353
+ # leaked descriptor; after such an error the pipe stops reading, so it
354
+ # never reports itself truncated.
355
+ #
356
+ # The pipe is asked whether its destination failed, not whether it
357
+ # recorded an exception. {#close_pipe} takes the pipe's own write end
358
+ # away before closing the pipe, and when nothing leaked the reading
359
+ # thread can reach end-of-file in between and record an `EOFError`. A
360
+ # destination can raise that class too, so the class can't tell them
361
+ # apart.
362
+ #
363
+ # @return [Boolean]
364
+ #
365
+ def leaked?
366
+ pipes.each_value { |pipe| close_pipe(pipe) }
367
+ @pipes_closed = true
368
+ pipes.each_value.any? { |pipe| pipe.truncated? || pipe.destination_failed? }
369
+ end
370
+
371
+ # Close a pipe for the leak check, ignoring an error from the close
372
+ #
373
+ # The pipe's write end is pointed at the null device first, so that the
374
+ # close the pipe then does on it never reaches the pipe; see "Operations
375
+ # this class avoids" in the class comment. No other thread can be using
376
+ # that descriptor: {Run} made the pipe for this one spawn.
377
+ #
378
+ # This step can't move into {MonitoredPipe#close}. A pipe made by other
379
+ # code can have a thread blocked in {MonitoredPipe#write}, which holds
380
+ # the descriptor's lock, and the reopen would wait for it with the GVL
381
+ # held. `IO#close` interrupts such a thread first; on Ruby 3.1 `IO#reopen`
382
+ # doesn't, and an existing {MonitoredPipe} example hangs.
383
+ #
384
+ # An error here must not stop the remaining pipes from being checked or
385
+ # the leak from being repaired, and the spawn error already being raised
386
+ # is the one the caller must see. Only the errors a close raises are
387
+ # ignored, so an error of another class is not lost.
388
+ #
389
+ # @param pipe [ProcessExecuter::MonitoredPipe] the pipe to close
390
+ #
391
+ # @return [void]
392
+ #
393
+ def close_pipe(pipe)
394
+ writer = pipe.pipe_writer
395
+ writer.reopen(self.class.null_device) unless writer.closed?
396
+ pipe.close(timeout: 0)
397
+ rescue IOError, SystemCallError
398
+ nil
399
+ end
400
+
401
+ # Point descriptors 1 and 2 back at their saved copies
402
+ #
403
+ # Works through IO objects other than STDOUT and STDERR, so those keep
404
+ # their buffers, modes, and encodings. `IO#reopen` swaps the handle under
405
+ # the C runtime's lock, so it waits for a write in progress on the
406
+ # descriptor; with the pipe's reader already closed by the leak check,
407
+ # such a write fails at once.
408
+ #
409
+ # @param copies [Hash{Integer => IO}] each descriptor and its copy
410
+ #
411
+ # @return [void]
412
+ #
413
+ def restore(copies)
414
+ copies.each { |fd, copy| self.class.target(fd).reopen(copy) }
415
+ end
416
+
417
+ class << self
418
+ # An IO object for descriptor 1 or 2 that a restore reopens
419
+ #
420
+ # Made once and kept, and first asked for before a spawn: `IO.for_fd`
421
+ # queries the descriptor's handle, which must not happen while the
422
+ # descriptor is leaked. It doesn't own the descriptor, so garbage
423
+ # collection never closes a standard stream. Its mode doesn't matter:
424
+ # `IO#reopen` takes the mode of the copy.
425
+ #
426
+ # @example
427
+ # ProcessExecuter::Commands::StandardStreamGuard.target(2).fileno #=> 2
428
+ #
429
+ # @param descriptor [Integer] 1 or 2
430
+ #
431
+ # @return [IO]
432
+ #
433
+ # @api private
434
+ #
435
+ def target(descriptor)
436
+ (@targets ||= {})[descriptor] ||= IO.for_fd(descriptor, 'w', autoclose: false)
437
+ end
438
+
439
+ # An IO object open on the null device, shared by every guard
440
+ #
441
+ # @example
442
+ # ProcessExecuter::Commands::StandardStreamGuard.null_device.closed? #=> false
443
+ #
444
+ # @return [File]
445
+ #
446
+ # @api private
447
+ #
448
+ def null_device
449
+ @null_device = File.open(File::NULL, 'wb') if @null_device.nil? || @null_device.closed?
450
+ @null_device
451
+ end
452
+ end
453
+
454
+ # :nocov:
455
+ end
456
+ end
457
+ end
@@ -386,6 +386,25 @@ module ProcessExecuter
386
386
  #
387
387
  def truncated? = @truncated
388
388
 
389
+ # Whether the destination's `#write` raised
390
+ #
391
+ # {#exception} can't answer this: it also holds errors from the monitor
392
+ # loop and from pipe cleanup, and the loop records an `EOFError` of its
393
+ # own when the pipe's write end is closed or reopened from outside, the
394
+ # same class a destination may raise.
395
+ #
396
+ # @example
397
+ # data_collector = StringIO.new
398
+ # pipe = ProcessExecuter::MonitoredPipe.new(data_collector)
399
+ # pipe.close
400
+ # pipe.destination_failed? #=> false
401
+ #
402
+ # @return [Boolean]
403
+ #
404
+ # @api private
405
+ #
406
+ def destination_failed? = @destination_failed
407
+
389
408
  # @!attribute [r]
390
409
  #
391
410
  # The thread that monitors the pipe
@@ -502,6 +521,7 @@ module ProcessExecuter
502
521
 
503
522
  @state = :open
504
523
  @truncated = false
524
+ @destination_failed = false
505
525
  @thread = start_monitoring_thread
506
526
 
507
527
  self.class.add_open_instance(self)
@@ -694,6 +714,7 @@ module ProcessExecuter
694
714
  rescue Exception => e # rubocop:disable Lint/RescueException
695
715
  mutex.synchronize do
696
716
  @exception = e
717
+ @destination_failed = true
697
718
  @state = :closing
698
719
  end
699
720
  end
@@ -3,5 +3,5 @@
3
3
  module ProcessExecuter
4
4
  # The current Gem version
5
5
  # @return [String]
6
- VERSION = '4.1.1'
6
+ VERSION = '4.1.2'
7
7
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: process_executer
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.1.1
4
+ version: 4.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - James Couball
@@ -255,6 +255,7 @@ files:
255
255
  - lib/process_executer/commands/run.rb
256
256
  - lib/process_executer/commands/run_with_capture.rb
257
257
  - lib/process_executer/commands/spawn_with_timeout.rb
258
+ - lib/process_executer/commands/standard_stream_guard.rb
258
259
  - lib/process_executer/destinations.rb
259
260
  - lib/process_executer/destinations/child_redirection.rb
260
261
  - lib/process_executer/destinations/close.rb
@@ -295,8 +296,8 @@ metadata:
295
296
  allowed_push_host: https://rubygems.org
296
297
  homepage_uri: https://github.com/main-branch/process_executer
297
298
  source_code_uri: https://github.com/main-branch/process_executer
298
- documentation_uri: https://rubydoc.info/gems/process_executer/4.1.1
299
- changelog_uri: https://rubydoc.info/gems/process_executer/4.1.1/file/CHANGELOG.md
299
+ documentation_uri: https://rubydoc.info/gems/process_executer/4.1.2
300
+ changelog_uri: https://rubydoc.info/gems/process_executer/4.1.2/file/CHANGELOG.md
300
301
  rubygems_mfa_required: 'true'
301
302
  rdoc_options: []
302
303
  require_paths:
@@ -314,7 +315,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
314
315
  requirements:
315
316
  - 'Platform: Mac, Linux, or Windows'
316
317
  - 'Ruby: MRI 3.1 or later, TruffleRuby 24 or later, or JRuby 9.4 or later'
317
- rubygems_version: 4.0.16
318
+ rubygems_version: 4.0.20
318
319
  specification_version: 4
319
320
  summary: Enhanced subprocess execution with timeouts, output capture, and flexible
320
321
  redirection