uppercut-subprocess-runner 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,10 @@
1
+ .venv/
2
+ __pycache__/
3
+ *.egg-info/
4
+ dist/
5
+ build/
6
+ .pytest_cache/
7
+ .DS_Store
8
+ uv.lock
9
+ .env*
10
+ *.pem
@@ -0,0 +1,107 @@
1
+ Metadata-Version: 2.5
2
+ Name: uppercut-subprocess-runner
3
+ Version: 0.1.0
4
+ Summary: Run a subprocess with a timeout, an output cap and cancellation, and kill its whole process tree.
5
+ Project-URL: Source, https://github.com/uppercut-labs/py-kit/tree/main/packages/uppercut-subprocess-runner
6
+ Author-email: Uppercut Labs <packages@uppercut-labs.com>
7
+ License-Expression: MIT
8
+ Keywords: kill,process-tree,sigterm,subprocess,taskkill,timeout
9
+ Classifier: Development Status :: 4 - Beta
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Operating System :: MacOS
12
+ Classifier: Operating System :: Microsoft :: Windows
13
+ Classifier: Operating System :: POSIX
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.10
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Typing :: Typed
20
+ Requires-Python: >=3.10
21
+ Description-Content-Type: text/markdown
22
+
23
+ # uppercut-subprocess-runner
24
+
25
+ Runs a command with a timeout, an output cap and cancellation, and returns a result object with the exit code, the captured output, the last bytes of each stream and how the run ended. When it has to stop the command it kills the whole process tree: the process group on macOS and Linux, a process-group break followed by `taskkill /F /T` on Windows. It sends a soft signal first, waits a grace period, then forces the kill. Standard library only.
26
+
27
+ ```sh
28
+ pip install uppercut-subprocess-runner
29
+ ```
30
+
31
+ ```python
32
+ import threading
33
+ from uppercut_subprocess_runner import run
34
+
35
+ cancel = threading.Event() # call cancel.set() from another thread to stop the run
36
+
37
+ result = run(
38
+ ["python", "build.py", "--fast"],
39
+ cwd="project",
40
+ timeout=120,
41
+ max_output=2_000_000,
42
+ grace=5,
43
+ cancel=cancel,
44
+ )
45
+
46
+ if result.timed_out:
47
+ print("gave up after", round(result.duration), "seconds")
48
+ elif not result.ok:
49
+ print(result.error_tail())
50
+ ```
51
+
52
+ Use `check=True` to raise `CommandError` instead, with the tail of the output in the message and the full result in `error.result`.
53
+
54
+ ## Functions
55
+
56
+ | Function | Returns |
57
+ | --- | --- |
58
+ | `run(argv, **options)` | `RunResult` |
59
+ | `kill_tree(process_or_pid, grace=5.0)` | `KillInfo` |
60
+ | `popen_group(argv, **popen_options)` | a `subprocess.Popen` started in its own session or process group |
61
+ | `popen_group_kwargs(no_window=False)` | the `Popen` keyword arguments that `popen_group` adds |
62
+
63
+ ## Options for `run`
64
+
65
+ | Option | Default | Meaning |
66
+ | --- | --- | --- |
67
+ | `cwd`, `env` | inherited | Working directory and environment of the command |
68
+ | `input` | none | Bytes written to stdin. Without it stdin is closed |
69
+ | `timeout` | none | Seconds before the run is stopped with `stop_reason == "timeout"` |
70
+ | `max_output` | `10_000_000` | Bytes kept from the start of each stream. `None` keeps everything |
71
+ | `stop_on_output_limit` | `True` | Stop the command when a stream passes `max_output`. `False` keeps reading and drops the excess |
72
+ | `tail_bytes` | `8192` | Bytes kept from the end of each stream, even past `max_output` |
73
+ | `grace` | `5.0` | Seconds between the soft signal and the forced kill |
74
+ | `cancel` | none | A `threading.Event`. Setting it stops the run with `stop_reason == "cancelled"` |
75
+ | `on_output` | none | Called as `on_output("stdout" or "stderr", chunk)` from reader threads. Exceptions it raises are ignored |
76
+ | `kill_process_tree` | `True` | `False` signals only the command itself and leaves its descendants running |
77
+ | `drain_timeout` | `1.0` | Seconds to wait for the pipes to close after the command exits. If a descendant still holds them, the tree is killed |
78
+ | `no_window` | `False` | Windows only: start the command without a console window |
79
+ | `check` | `False` | Raise `CommandError` unless the command exits with code 0 |
80
+
81
+ ## RunResult
82
+
83
+ | Field | Meaning |
84
+ | --- | --- |
85
+ | `argv` | The command that ran |
86
+ | `exit_code` | Exit code. `None` if it never started. Negative signal number after a kill on macOS and Linux |
87
+ | `stop_reason` | `"exited"`, `"timeout"`, `"cancelled"`, `"output-limit"` or `"spawn-failed"` |
88
+ | `timed_out`, `cancelled`, `ok` | Shortcuts. `ok` means it exited by itself with code 0 |
89
+ | `truncated` | A stream produced more than `max_output` bytes |
90
+ | `duration` | Seconds from start to finish |
91
+ | `stdout`, `stderr` | Captured bytes, at most `max_output` each. `stdout_text` and `stderr_text` decode them |
92
+ | `stdout_tail`, `stderr_tail` | The last `tail_bytes` of each stream. `error_tail(limit)` returns the stderr tail as text, or the stdout tail if stderr is empty |
93
+ | `pid` | Process id of the command |
94
+ | `kill` | A `KillInfo` when the run was stopped, otherwise `None` |
95
+ | `error` | The operating system's message when `stop_reason == "spawn-failed"` |
96
+
97
+ `KillInfo` has `method`, `already_exited`, `graceful` (the soft signal was enough) and `forced` (SIGKILL or `taskkill /F` was needed).
98
+
99
+ ## Notes
100
+
101
+ - `kill_tree` on macOS and Linux needs the target to lead its own process group. Processes started by `run` and `popen_group` do. It raises `ValueError` rather than signal the calling process's own group.
102
+ - On Windows the soft step is `CTRL_BREAK_EVENT`, which ends most console programs. Descendants of a command that exits during the grace period cannot be found afterwards and keep running.
103
+ - Error messages from `check=True` name the program but not its arguments.
104
+
105
+ ## License
106
+
107
+ MIT
@@ -0,0 +1,85 @@
1
+ # uppercut-subprocess-runner
2
+
3
+ Runs a command with a timeout, an output cap and cancellation, and returns a result object with the exit code, the captured output, the last bytes of each stream and how the run ended. When it has to stop the command it kills the whole process tree: the process group on macOS and Linux, a process-group break followed by `taskkill /F /T` on Windows. It sends a soft signal first, waits a grace period, then forces the kill. Standard library only.
4
+
5
+ ```sh
6
+ pip install uppercut-subprocess-runner
7
+ ```
8
+
9
+ ```python
10
+ import threading
11
+ from uppercut_subprocess_runner import run
12
+
13
+ cancel = threading.Event() # call cancel.set() from another thread to stop the run
14
+
15
+ result = run(
16
+ ["python", "build.py", "--fast"],
17
+ cwd="project",
18
+ timeout=120,
19
+ max_output=2_000_000,
20
+ grace=5,
21
+ cancel=cancel,
22
+ )
23
+
24
+ if result.timed_out:
25
+ print("gave up after", round(result.duration), "seconds")
26
+ elif not result.ok:
27
+ print(result.error_tail())
28
+ ```
29
+
30
+ Use `check=True` to raise `CommandError` instead, with the tail of the output in the message and the full result in `error.result`.
31
+
32
+ ## Functions
33
+
34
+ | Function | Returns |
35
+ | --- | --- |
36
+ | `run(argv, **options)` | `RunResult` |
37
+ | `kill_tree(process_or_pid, grace=5.0)` | `KillInfo` |
38
+ | `popen_group(argv, **popen_options)` | a `subprocess.Popen` started in its own session or process group |
39
+ | `popen_group_kwargs(no_window=False)` | the `Popen` keyword arguments that `popen_group` adds |
40
+
41
+ ## Options for `run`
42
+
43
+ | Option | Default | Meaning |
44
+ | --- | --- | --- |
45
+ | `cwd`, `env` | inherited | Working directory and environment of the command |
46
+ | `input` | none | Bytes written to stdin. Without it stdin is closed |
47
+ | `timeout` | none | Seconds before the run is stopped with `stop_reason == "timeout"` |
48
+ | `max_output` | `10_000_000` | Bytes kept from the start of each stream. `None` keeps everything |
49
+ | `stop_on_output_limit` | `True` | Stop the command when a stream passes `max_output`. `False` keeps reading and drops the excess |
50
+ | `tail_bytes` | `8192` | Bytes kept from the end of each stream, even past `max_output` |
51
+ | `grace` | `5.0` | Seconds between the soft signal and the forced kill |
52
+ | `cancel` | none | A `threading.Event`. Setting it stops the run with `stop_reason == "cancelled"` |
53
+ | `on_output` | none | Called as `on_output("stdout" or "stderr", chunk)` from reader threads. Exceptions it raises are ignored |
54
+ | `kill_process_tree` | `True` | `False` signals only the command itself and leaves its descendants running |
55
+ | `drain_timeout` | `1.0` | Seconds to wait for the pipes to close after the command exits. If a descendant still holds them, the tree is killed |
56
+ | `no_window` | `False` | Windows only: start the command without a console window |
57
+ | `check` | `False` | Raise `CommandError` unless the command exits with code 0 |
58
+
59
+ ## RunResult
60
+
61
+ | Field | Meaning |
62
+ | --- | --- |
63
+ | `argv` | The command that ran |
64
+ | `exit_code` | Exit code. `None` if it never started. Negative signal number after a kill on macOS and Linux |
65
+ | `stop_reason` | `"exited"`, `"timeout"`, `"cancelled"`, `"output-limit"` or `"spawn-failed"` |
66
+ | `timed_out`, `cancelled`, `ok` | Shortcuts. `ok` means it exited by itself with code 0 |
67
+ | `truncated` | A stream produced more than `max_output` bytes |
68
+ | `duration` | Seconds from start to finish |
69
+ | `stdout`, `stderr` | Captured bytes, at most `max_output` each. `stdout_text` and `stderr_text` decode them |
70
+ | `stdout_tail`, `stderr_tail` | The last `tail_bytes` of each stream. `error_tail(limit)` returns the stderr tail as text, or the stdout tail if stderr is empty |
71
+ | `pid` | Process id of the command |
72
+ | `kill` | A `KillInfo` when the run was stopped, otherwise `None` |
73
+ | `error` | The operating system's message when `stop_reason == "spawn-failed"` |
74
+
75
+ `KillInfo` has `method`, `already_exited`, `graceful` (the soft signal was enough) and `forced` (SIGKILL or `taskkill /F` was needed).
76
+
77
+ ## Notes
78
+
79
+ - `kill_tree` on macOS and Linux needs the target to lead its own process group. Processes started by `run` and `popen_group` do. It raises `ValueError` rather than signal the calling process's own group.
80
+ - On Windows the soft step is `CTRL_BREAK_EVENT`, which ends most console programs. Descendants of a command that exits during the grace period cannot be found afterwards and keep running.
81
+ - Error messages from `check=True` name the program but not its arguments.
82
+
83
+ ## License
84
+
85
+ MIT
@@ -0,0 +1,36 @@
1
+ [build-system]
2
+ requires = ["hatchling>=1.25"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "uppercut-subprocess-runner"
7
+ version = "0.1.0"
8
+ description = "Run a subprocess with a timeout, an output cap and cancellation, and kill its whole process tree."
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ license = "MIT"
12
+ authors = [{ name = "Uppercut Labs", email = "packages@uppercut-labs.com" }]
13
+ keywords = ["subprocess", "timeout", "process-tree", "kill", "sigterm", "taskkill"]
14
+ dependencies = []
15
+ classifiers = [
16
+ "Development Status :: 4 - Beta",
17
+ "Intended Audience :: Developers",
18
+ "Operating System :: MacOS",
19
+ "Operating System :: Microsoft :: Windows",
20
+ "Operating System :: POSIX",
21
+ "Programming Language :: Python :: 3",
22
+ "Programming Language :: Python :: 3.10",
23
+ "Programming Language :: Python :: 3.11",
24
+ "Programming Language :: Python :: 3.12",
25
+ "Programming Language :: Python :: 3.13",
26
+ "Typing :: Typed",
27
+ ]
28
+
29
+ [project.urls]
30
+ Source = "https://github.com/uppercut-labs/py-kit/tree/main/packages/uppercut-subprocess-runner"
31
+
32
+ [tool.hatch.build.targets.wheel]
33
+ packages = ["src/uppercut_subprocess_runner"]
34
+
35
+ [tool.hatch.build.targets.sdist]
36
+ include = ["src", "tests", "README.md"]
@@ -0,0 +1,15 @@
1
+ """Run a subprocess with a timeout, an output cap and cancellation, and kill its whole process tree."""
2
+
3
+ from ._kill import KillInfo, kill_tree, popen_group, popen_group_kwargs
4
+ from ._runner import CommandError, RunResult, run
5
+
6
+ __all__ = [
7
+ "CommandError",
8
+ "KillInfo",
9
+ "RunResult",
10
+ "kill_tree",
11
+ "popen_group",
12
+ "popen_group_kwargs",
13
+ "run",
14
+ ]
15
+ __version__ = "0.1.0"
@@ -0,0 +1,184 @@
1
+ """Kill a process together with everything it started."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import os
6
+ import signal
7
+ import subprocess
8
+ import time
9
+ from dataclasses import dataclass
10
+ from typing import Any, Union
11
+
12
+ IS_WINDOWS = os.name == "nt"
13
+
14
+ _POLL = 0.02
15
+
16
+ Target = Union["subprocess.Popen[Any]", int]
17
+
18
+
19
+ @dataclass(frozen=True)
20
+ class KillInfo:
21
+ """What `kill_tree` did.
22
+
23
+ method: "killpg" (POSIX process group), "taskkill" (Windows), "terminate" (leader only) or "none" when
24
+ nothing was left to kill.
25
+ already_exited: the tree was already gone when the call started.
26
+ graceful: the soft signal was enough and no forced kill was needed.
27
+ forced: SIGKILL (POSIX) or taskkill /F (Windows) was used.
28
+ """
29
+
30
+ method: str
31
+ already_exited: bool = False
32
+ graceful: bool = False
33
+ forced: bool = False
34
+
35
+
36
+ def popen_group_kwargs(*, no_window: bool = False) -> dict[str, Any]:
37
+ """Popen keyword arguments that put the child in its own session or process group.
38
+
39
+ A child started this way can be killed together with its descendants by `kill_tree`.
40
+ `no_window` adds CREATE_NO_WINDOW on Windows and is ignored elsewhere.
41
+ """
42
+ if IS_WINDOWS:
43
+ flags = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
44
+ if no_window:
45
+ flags |= getattr(subprocess, "CREATE_NO_WINDOW", 0)
46
+ return {"creationflags": flags}
47
+ return {"start_new_session": True}
48
+
49
+
50
+ def popen_group(argv: Any, *, no_window: bool = False, **kwargs: Any) -> "subprocess.Popen[Any]":
51
+ """`subprocess.Popen` with `popen_group_kwargs` applied."""
52
+ kwargs.update(popen_group_kwargs(no_window=no_window))
53
+ return subprocess.Popen(argv, **kwargs) # noqa: S603
54
+
55
+
56
+ def kill_tree(target: Target, *, grace: float = 5.0) -> KillInfo:
57
+ """Stop a process and all of its descendants.
58
+
59
+ POSIX: sends SIGTERM to the process group, waits up to `grace` seconds for the group to empty, then sends
60
+ SIGKILL. The target must lead its own group (start it with `popen_group` or `start_new_session=True`).
61
+ The calling process's own group is never signalled; a target in it raises ValueError.
62
+
63
+ Windows: sends CTRL_BREAK_EVENT to a `Popen` started in a new process group, waits up to `grace` seconds,
64
+ then runs `taskkill /F /T`. Descendants of a leader that exited during the grace period are not reached.
65
+
66
+ A `Popen` target is reaped. An integer pid is only signalled. `grace=0` skips the soft step.
67
+ """
68
+ if grace < 0:
69
+ raise ValueError("grace must be zero or positive")
70
+ if IS_WINDOWS:
71
+ return _kill_windows(target, grace)
72
+ return _kill_posix(target, grace)
73
+
74
+
75
+ def _pid_of(target: Target) -> "tuple[int, subprocess.Popen[Any] | None]":
76
+ if isinstance(target, subprocess.Popen):
77
+ return target.pid, target
78
+ return int(target), None
79
+
80
+
81
+ def _group_exists(pgid: int) -> bool:
82
+ try:
83
+ os.killpg(pgid, 0)
84
+ except ProcessLookupError:
85
+ return False
86
+ except PermissionError:
87
+ # Members we cannot signal (or zombies on some systems): nothing more we can do.
88
+ return False
89
+ return True
90
+
91
+
92
+ def _signal_group(pgid: int, sig: int) -> None:
93
+ try:
94
+ os.killpg(pgid, sig)
95
+ except (ProcessLookupError, PermissionError):
96
+ pass
97
+
98
+
99
+ def _reap(proc: "subprocess.Popen[Any]") -> None:
100
+ try:
101
+ proc.wait(timeout=5)
102
+ except subprocess.TimeoutExpired:
103
+ pass
104
+
105
+
106
+ def _kill_posix(target: Target, grace: float) -> KillInfo:
107
+ pid, proc = _pid_of(target)
108
+ try:
109
+ pgid = os.getpgid(pid)
110
+ except ProcessLookupError:
111
+ # The leader is already gone; its group id stays the same while any member lives.
112
+ pgid = pid
113
+ if pgid == os.getpgrp():
114
+ raise ValueError(
115
+ "refusing to kill the calling process's own process group; start the child with popen_group()"
116
+ )
117
+ if proc is not None:
118
+ proc.poll()
119
+ if not _group_exists(pgid):
120
+ if proc is not None:
121
+ _reap(proc)
122
+ return KillInfo(method="none", already_exited=True)
123
+
124
+ if grace > 0:
125
+ _signal_group(pgid, signal.SIGTERM)
126
+ end = time.monotonic() + grace
127
+ while True:
128
+ if proc is not None:
129
+ proc.poll()
130
+ if not _group_exists(pgid):
131
+ if proc is not None:
132
+ _reap(proc)
133
+ return KillInfo(method="killpg", graceful=True)
134
+ if time.monotonic() >= end:
135
+ break
136
+ time.sleep(_POLL)
137
+ _signal_group(pgid, signal.SIGKILL)
138
+ if proc is not None:
139
+ _reap(proc)
140
+ end = time.monotonic() + 2.0
141
+ while _group_exists(pgid) and time.monotonic() < end:
142
+ time.sleep(_POLL)
143
+ return KillInfo(method="killpg", forced=True)
144
+
145
+
146
+ def _taskkill(pid: int) -> bool:
147
+ flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
148
+ try:
149
+ done = subprocess.run( # noqa: S603
150
+ ["taskkill", "/F", "/T", "/PID", str(pid)],
151
+ stdin=subprocess.DEVNULL,
152
+ stdout=subprocess.DEVNULL,
153
+ stderr=subprocess.DEVNULL,
154
+ timeout=15,
155
+ creationflags=flags,
156
+ )
157
+ except (OSError, subprocess.SubprocessError):
158
+ return False
159
+ return done.returncode == 0
160
+
161
+
162
+ def _kill_windows(target: Target, grace: float) -> KillInfo:
163
+ pid, proc = _pid_of(target)
164
+ if proc is not None and proc.poll() is not None:
165
+ return KillInfo(method="none", already_exited=True)
166
+ if proc is not None and grace > 0:
167
+ try:
168
+ proc.send_signal(signal.CTRL_BREAK_EVENT) # type: ignore[attr-defined]
169
+ except (OSError, ValueError, AttributeError):
170
+ pass
171
+ try:
172
+ proc.wait(timeout=grace)
173
+ _taskkill(pid) # sweeps any descendants that are still reachable
174
+ return KillInfo(method="taskkill", graceful=True)
175
+ except subprocess.TimeoutExpired:
176
+ pass
177
+ if not _taskkill(pid) and proc is not None:
178
+ try:
179
+ proc.kill()
180
+ except OSError:
181
+ pass
182
+ if proc is not None:
183
+ _reap(proc)
184
+ return KillInfo(method="taskkill", forced=True)
@@ -0,0 +1,304 @@
1
+ """Run one command with a timeout, an output cap and cancellation."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import os
6
+ import subprocess
7
+ import threading
8
+ import time
9
+ from dataclasses import dataclass
10
+ from typing import Any, Callable, Mapping, Sequence
11
+
12
+ from ._kill import KillInfo, kill_tree, popen_group_kwargs
13
+
14
+ OnOutput = Callable[[str, bytes], None]
15
+
16
+ _POLL = 0.02
17
+ _READ = 65536
18
+
19
+
20
+ class CommandError(RuntimeError):
21
+ """Raised by `run(check=True)` and `RunResult.check_returncode()`. The result is in `.result`."""
22
+
23
+ def __init__(self, message: str, result: "RunResult") -> None:
24
+ super().__init__(message)
25
+ self.result = result
26
+
27
+
28
+ @dataclass(frozen=True)
29
+ class RunResult:
30
+ """Outcome of `run`.
31
+
32
+ stop_reason is one of "exited", "timeout", "cancelled", "output-limit" or "spawn-failed".
33
+ stdout and stderr hold at most `max_output` bytes from the start of each stream. The tails hold the last
34
+ `tail_bytes` bytes of each stream, even when the stream was longer than `max_output`.
35
+ exit_code is None when the command never started; after a kill on POSIX it is the negative signal number.
36
+ """
37
+
38
+ argv: tuple[str, ...]
39
+ exit_code: int | None
40
+ stop_reason: str
41
+ duration: float
42
+ stdout: bytes
43
+ stderr: bytes
44
+ stdout_tail: bytes
45
+ stderr_tail: bytes
46
+ truncated: bool
47
+ pid: int | None
48
+ kill: KillInfo | None = None
49
+ error: str | None = None
50
+
51
+ @property
52
+ def timed_out(self) -> bool:
53
+ return self.stop_reason == "timeout"
54
+
55
+ @property
56
+ def cancelled(self) -> bool:
57
+ return self.stop_reason == "cancelled"
58
+
59
+ @property
60
+ def ok(self) -> bool:
61
+ """The command ran to the end by itself and exited with code 0."""
62
+ return self.stop_reason == "exited" and self.exit_code == 0
63
+
64
+ @property
65
+ def stdout_text(self) -> str:
66
+ return self.stdout.decode("utf-8", "replace")
67
+
68
+ @property
69
+ def stderr_text(self) -> str:
70
+ return self.stderr.decode("utf-8", "replace")
71
+
72
+ def error_tail(self, limit: int = 2000) -> str:
73
+ """The last `limit` characters of stderr, or of stdout when stderr is empty."""
74
+ data = self.stderr_tail or self.stdout_tail
75
+ return data.decode("utf-8", "replace")[-limit:]
76
+
77
+ def check_returncode(self, *, tail: int = 2000) -> "RunResult":
78
+ """Return self when `ok`, otherwise raise CommandError with the tail of the output."""
79
+ if self.ok:
80
+ return self
81
+ name = self.argv[0] if self.argv else "command"
82
+ if self.stop_reason == "exited":
83
+ what = "exited with code %s" % self.exit_code
84
+ elif self.stop_reason == "spawn-failed":
85
+ what = "could not start (%s)" % (self.error or "unknown error")
86
+ else:
87
+ what = "stopped: %s" % self.stop_reason
88
+ text = self.error_tail(tail).strip()
89
+ message = "%s %s" % (name, what) + (":\n" + text if text else "")
90
+ raise CommandError(message, self)
91
+
92
+
93
+ class _Stream:
94
+ """Reads one pipe on a thread, keeping the head up to a cap and a rolling tail."""
95
+
96
+ def __init__(
97
+ self,
98
+ name: str,
99
+ pipe: Any,
100
+ max_bytes: int | None,
101
+ tail_bytes: int,
102
+ stop_on_limit: bool,
103
+ limit_event: threading.Event,
104
+ on_output: OnOutput | None,
105
+ ) -> None:
106
+ self.name = name
107
+ self.head = bytearray()
108
+ self.tail = bytearray()
109
+ self.truncated = False
110
+ self._pipe = pipe
111
+ self._max = max_bytes
112
+ self._tail_bytes = tail_bytes
113
+ self._stop_on_limit = stop_on_limit
114
+ self._limit_event = limit_event
115
+ self._on_output = on_output
116
+ self.thread = threading.Thread(target=self._pump, daemon=True)
117
+ self.thread.start()
118
+
119
+ def _pump(self) -> None:
120
+ try:
121
+ while True:
122
+ chunk = self._pipe.read(_READ)
123
+ if not chunk:
124
+ return
125
+ self._feed(chunk)
126
+ except (OSError, ValueError):
127
+ return
128
+
129
+ def _feed(self, chunk: bytes) -> None:
130
+ if self._max is None:
131
+ self.head += chunk
132
+ else:
133
+ room = self._max - len(self.head)
134
+ if room > 0:
135
+ self.head += chunk[:room]
136
+ if len(chunk) > max(room, 0):
137
+ self.truncated = True
138
+ if self._stop_on_limit:
139
+ self._limit_event.set()
140
+ if self._tail_bytes > 0:
141
+ self.tail += chunk
142
+ if len(self.tail) > 2 * self._tail_bytes:
143
+ del self.tail[: -self._tail_bytes]
144
+ if self._on_output is not None:
145
+ try:
146
+ self._on_output(self.name, bytes(chunk))
147
+ except Exception: # a broken callback must not stop the capture
148
+ pass
149
+
150
+ def final_tail(self) -> bytes:
151
+ return bytes(self.tail[-self._tail_bytes :]) if self._tail_bytes > 0 else b""
152
+
153
+
154
+ def _feed_stdin(pipe: Any, data: bytes) -> None:
155
+ try:
156
+ pipe.write(data)
157
+ except (OSError, ValueError):
158
+ pass
159
+ finally:
160
+ try:
161
+ pipe.close()
162
+ except (OSError, ValueError):
163
+ pass
164
+
165
+
166
+ def _stop_leader_only(proc: "subprocess.Popen[Any]", grace: float) -> KillInfo:
167
+ if proc.poll() is not None:
168
+ return KillInfo(method="none", already_exited=True)
169
+ proc.terminate()
170
+ try:
171
+ proc.wait(timeout=grace)
172
+ return KillInfo(method="terminate", graceful=True)
173
+ except subprocess.TimeoutExpired:
174
+ proc.kill()
175
+ return KillInfo(method="terminate", forced=True)
176
+
177
+
178
+ def run(
179
+ argv: "Sequence[str | os.PathLike[str]]",
180
+ *,
181
+ cwd: "str | os.PathLike[str] | None" = None,
182
+ env: Mapping[str, str] | None = None,
183
+ input: bytes | None = None,
184
+ timeout: float | None = None,
185
+ max_output: int | None = 10_000_000,
186
+ tail_bytes: int = 8192,
187
+ stop_on_output_limit: bool = True,
188
+ grace: float = 5.0,
189
+ cancel: threading.Event | None = None,
190
+ on_output: OnOutput | None = None,
191
+ kill_process_tree: bool = True,
192
+ drain_timeout: float = 1.0,
193
+ no_window: bool = False,
194
+ check: bool = False,
195
+ ) -> RunResult:
196
+ """Run `argv` (no shell) and return a `RunResult`. Never raises for a failing command unless `check` is set.
197
+
198
+ The child starts in its own session (POSIX) or process group (Windows). It is stopped, together with its
199
+ descendants, when `timeout` seconds pass, when `cancel` is set, or when one output stream exceeds
200
+ `max_output` bytes (unless `stop_on_output_limit` is False, which keeps draining and discards the excess).
201
+ Stopping sends a soft signal, waits `grace` seconds, then forces the kill.
202
+
203
+ stdin is closed unless `input` is given. If the command exits but a descendant keeps its pipes open, the
204
+ tree is killed after `drain_timeout` seconds so the call returns.
205
+ """
206
+ args = tuple(os.fspath(a) for a in argv)
207
+ if not args:
208
+ raise ValueError("argv must not be empty")
209
+ if timeout is not None and timeout <= 0:
210
+ raise ValueError("timeout must be positive")
211
+ if max_output is not None and max_output <= 0:
212
+ raise ValueError("max_output must be positive")
213
+ if tail_bytes < 0:
214
+ raise ValueError("tail_bytes must not be negative")
215
+ if grace < 0:
216
+ raise ValueError("grace must be zero or positive")
217
+
218
+ start = time.monotonic()
219
+ try:
220
+ proc = subprocess.Popen( # noqa: S603
221
+ list(args),
222
+ cwd=os.fspath(cwd) if cwd is not None else None,
223
+ env=dict(env) if env is not None else None,
224
+ stdin=subprocess.PIPE if input is not None else subprocess.DEVNULL,
225
+ stdout=subprocess.PIPE,
226
+ stderr=subprocess.PIPE,
227
+ bufsize=0,
228
+ shell=False,
229
+ **popen_group_kwargs(no_window=no_window),
230
+ )
231
+ except (OSError, ValueError) as exc:
232
+ failed = RunResult(
233
+ argv=args,
234
+ exit_code=None,
235
+ stop_reason="spawn-failed",
236
+ duration=time.monotonic() - start,
237
+ stdout=b"",
238
+ stderr=b"",
239
+ stdout_tail=b"",
240
+ stderr_tail=b"",
241
+ truncated=False,
242
+ pid=None,
243
+ error=str(exc),
244
+ )
245
+ return failed.check_returncode() if check else failed
246
+
247
+ limit_event = threading.Event()
248
+ streams = [
249
+ _Stream(name, pipe, max_output, tail_bytes, stop_on_output_limit, limit_event, on_output)
250
+ for name, pipe in (("stdout", proc.stdout), ("stderr", proc.stderr))
251
+ ]
252
+ if input is not None:
253
+ threading.Thread(target=_feed_stdin, args=(proc.stdin, input), daemon=True).start()
254
+
255
+ deadline = start + timeout if timeout is not None else None
256
+ reason = "exited"
257
+ while proc.poll() is None:
258
+ if cancel is not None and cancel.is_set():
259
+ reason = "cancelled"
260
+ elif limit_event.is_set():
261
+ reason = "output-limit"
262
+ elif deadline is not None and time.monotonic() >= deadline:
263
+ reason = "timeout"
264
+ if reason != "exited":
265
+ break
266
+ try:
267
+ proc.wait(timeout=_POLL)
268
+ except subprocess.TimeoutExpired:
269
+ pass
270
+
271
+ kill: KillInfo | None = None
272
+ if reason != "exited":
273
+ kill = kill_tree(proc, grace=grace) if kill_process_tree else _stop_leader_only(proc, grace)
274
+ proc.wait()
275
+
276
+ for stream in streams:
277
+ stream.thread.join(drain_timeout)
278
+ if any(s.thread.is_alive() for s in streams):
279
+ # A descendant still holds the pipes open.
280
+ if kill_process_tree:
281
+ late = kill_tree(proc, grace=0)
282
+ kill = kill or late
283
+ for stream in streams:
284
+ stream.thread.join(1.0)
285
+ for pipe in (proc.stdout, proc.stderr):
286
+ try:
287
+ pipe.close()
288
+ except (OSError, ValueError):
289
+ pass
290
+
291
+ result = RunResult(
292
+ argv=args,
293
+ exit_code=proc.returncode,
294
+ stop_reason=reason,
295
+ duration=time.monotonic() - start,
296
+ stdout=bytes(streams[0].head),
297
+ stderr=bytes(streams[1].head),
298
+ stdout_tail=streams[0].final_tail(),
299
+ stderr_tail=streams[1].final_tail(),
300
+ truncated=any(s.truncated for s in streams),
301
+ pid=proc.pid,
302
+ kill=kill,
303
+ )
304
+ return result.check_returncode() if check else result
@@ -0,0 +1,380 @@
1
+ import os
2
+ import subprocess
3
+ import sys
4
+ import threading
5
+ import time
6
+
7
+ import pytest
8
+
9
+ from uppercut_subprocess_runner import CommandError, RunResult, kill_tree, popen_group, run
10
+
11
+ PY = sys.executable
12
+ POSIX = os.name != "nt"
13
+ posix_only = pytest.mark.skipif(not POSIX, reason="needs POSIX process groups and signals")
14
+
15
+
16
+ def py(code, **kwargs):
17
+ return run([PY, "-c", code], **kwargs)
18
+
19
+
20
+ def pid_alive(pid):
21
+ if POSIX:
22
+ try:
23
+ os.kill(pid, 0)
24
+ except ProcessLookupError:
25
+ return False
26
+ except PermissionError:
27
+ return True
28
+ return True
29
+ out = subprocess.run(["tasklist", "/FI", "PID eq %d" % pid], capture_output=True, text=True).stdout
30
+ return str(pid) in out
31
+
32
+
33
+ def wait_dead(pid, seconds=5.0):
34
+ end = time.monotonic() + seconds
35
+ while time.monotonic() < end:
36
+ if not pid_alive(pid):
37
+ return True
38
+ time.sleep(0.05)
39
+ return not pid_alive(pid)
40
+
41
+
42
+ # --- normal runs -------------------------------------------------------------------------------------------
43
+
44
+
45
+ def test_captures_stdout_stderr_and_exit_code():
46
+ r = py("import sys; print('out'); print('err', file=sys.stderr); sys.exit(3)")
47
+ assert isinstance(r, RunResult)
48
+ assert r.exit_code == 3
49
+ assert r.stop_reason == "exited"
50
+ assert r.stdout_text.strip() == "out"
51
+ assert r.stderr_text.strip() == "err"
52
+ assert not r.ok and not r.timed_out and not r.cancelled and not r.truncated
53
+ assert r.duration > 0 and r.pid
54
+
55
+
56
+ def test_ok_on_zero_exit():
57
+ r = py("print('hi')")
58
+ assert r.ok and r.exit_code == 0
59
+
60
+
61
+ def test_stdin_input_is_delivered_and_default_stdin_is_closed():
62
+ r = py("import sys; print(sys.stdin.read().upper())", input=b"hello")
63
+ assert r.stdout_text.strip() == "HELLO"
64
+ r = py("import sys; print(repr(sys.stdin.read()))", timeout=10)
65
+ assert r.stdout_text.strip() == "''"
66
+
67
+
68
+ def test_large_input_does_not_deadlock():
69
+ data = b"x" * 3_000_000
70
+ r = py("import sys; d = sys.stdin.buffer.read(); print(len(d))", input=data, timeout=30)
71
+ assert r.stdout_text.strip() == str(len(data))
72
+
73
+
74
+ def test_cwd_and_env(tmp_path):
75
+ r = py("import os; print(os.getcwd()); print(os.environ['RUNNER_TEST'])", cwd=tmp_path,
76
+ env={**os.environ, "RUNNER_TEST": "value"})
77
+ lines = r.stdout_text.split()
78
+ assert os.path.realpath(lines[0]) == os.path.realpath(str(tmp_path))
79
+ assert lines[1] == "value"
80
+
81
+
82
+ def test_accepts_path_like_in_argv(tmp_path):
83
+ script = tmp_path / "s.py"
84
+ script.write_text("print('ran')")
85
+ r = run([PY, script])
86
+ assert r.stdout_text.strip() == "ran"
87
+
88
+
89
+ def test_on_output_receives_stream_names_and_chunks():
90
+ seen = []
91
+ py("import sys; print('a'); print('b', file=sys.stderr)", on_output=lambda name, chunk: seen.append((name, chunk)))
92
+ assert ("stdout", b"a") in [(n, c.strip()) for n, c in seen]
93
+ assert ("stderr", b"b") in [(n, c.strip()) for n, c in seen]
94
+
95
+
96
+ def test_failing_callback_does_not_break_capture():
97
+ def boom(name, chunk):
98
+ raise RuntimeError("no")
99
+
100
+ r = py("print('still captured')", on_output=boom)
101
+ assert r.stdout_text.strip() == "still captured"
102
+
103
+
104
+ # --- spawn failure and check ---------------------------------------------------------------------------------
105
+
106
+
107
+ def test_spawn_failure_is_a_result():
108
+ r = run(["definitely-not-a-real-command-xyz"])
109
+ assert r.stop_reason == "spawn-failed"
110
+ assert r.exit_code is None and r.pid is None
111
+ assert r.error
112
+ assert not r.ok
113
+
114
+
115
+ def test_check_raises_with_stderr_tail():
116
+ with pytest.raises(CommandError) as info:
117
+ py("import sys; print('boom detail', file=sys.stderr); sys.exit(2)", check=True)
118
+ assert "exited with code 2" in str(info.value)
119
+ assert "boom detail" in str(info.value)
120
+ assert info.value.result.exit_code == 2
121
+
122
+
123
+ def test_check_falls_back_to_stdout_tail():
124
+ with pytest.raises(CommandError) as info:
125
+ py("print('only stdout'); raise SystemExit(1)", check=True)
126
+ assert "only stdout" in str(info.value)
127
+
128
+
129
+ def test_check_raises_on_spawn_failure_and_timeout():
130
+ with pytest.raises(CommandError):
131
+ run(["definitely-not-a-real-command-xyz"], check=True)
132
+ with pytest.raises(CommandError) as info:
133
+ py("import time; time.sleep(30)", timeout=0.3, grace=0.2, check=True)
134
+ assert "timeout" in str(info.value)
135
+
136
+
137
+ def test_check_returncode_returns_the_result_when_ok():
138
+ r = py("pass")
139
+ assert r.check_returncode() is r
140
+
141
+
142
+ def test_error_message_does_not_echo_arguments():
143
+ with pytest.raises(CommandError) as info:
144
+ run([PY, "-c", "import sys; sys.exit(1)", "SECRET-ARG"], check=True)
145
+ assert "SECRET-ARG" not in str(info.value)
146
+
147
+
148
+ # --- timeout -----------------------------------------------------------------------------------------------
149
+
150
+
151
+ def test_timeout_stops_the_command_and_keeps_partial_output():
152
+ r = py("import time; print('partial', flush=True); time.sleep(30)", timeout=1.0, grace=0.5)
153
+ assert r.timed_out
154
+ assert r.stop_reason == "timeout"
155
+ assert r.stdout_text.strip() == "partial"
156
+ assert r.duration < 10
157
+ assert r.kill is not None
158
+ assert not pid_alive(r.pid)
159
+
160
+
161
+ @posix_only
162
+ def test_term_ignoring_child_is_killed_after_the_grace_period():
163
+ code = (
164
+ "import signal, sys, time\n"
165
+ "signal.signal(signal.SIGTERM, signal.SIG_IGN)\n"
166
+ "print('ready', flush=True)\n"
167
+ "time.sleep(60)\n"
168
+ )
169
+ r = py(code, timeout=1.0, grace=0.5)
170
+ assert r.timed_out
171
+ assert r.kill.forced and not r.kill.graceful
172
+ assert r.exit_code == -9
173
+ assert r.duration >= 1.4
174
+
175
+
176
+ @posix_only
177
+ def test_well_behaved_child_stops_on_sigterm_without_force():
178
+ r = py("import time; print('x', flush=True); time.sleep(60)", timeout=0.8, grace=5.0)
179
+ assert r.timed_out
180
+ assert r.kill.graceful and not r.kill.forced
181
+ assert r.exit_code == -15
182
+ assert r.duration < 4
183
+
184
+
185
+ @posix_only
186
+ def test_timeout_kills_the_whole_process_tree():
187
+ code = (
188
+ "import subprocess, sys, time\n"
189
+ "child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n"
190
+ "print(child.pid, flush=True)\n"
191
+ "time.sleep(60)\n"
192
+ )
193
+ r = py(code, timeout=2.0, grace=0.5)
194
+ assert r.timed_out
195
+ grandchild = int(r.stdout_text.split()[0])
196
+ assert wait_dead(grandchild)
197
+
198
+
199
+ @posix_only
200
+ def test_descendant_holding_the_pipes_does_not_hang_the_call():
201
+ code = (
202
+ "import subprocess, sys\n"
203
+ "child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n"
204
+ "print(child.pid, flush=True)\n"
205
+ )
206
+ start = time.monotonic()
207
+ r = py(code, drain_timeout=0.3, grace=0.5)
208
+ assert time.monotonic() - start < 10
209
+ assert r.stop_reason == "exited" and r.exit_code == 0
210
+ assert wait_dead(int(r.stdout_text.split()[0]))
211
+
212
+
213
+ @posix_only
214
+ def test_leader_only_mode_leaves_descendants_alone():
215
+ code = (
216
+ "import subprocess, sys, time\n"
217
+ "child = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(8)'])\n"
218
+ "print(child.pid, flush=True)\n"
219
+ "time.sleep(60)\n"
220
+ )
221
+ r = py(code, timeout=1.5, grace=0.3, kill_process_tree=False, drain_timeout=0.2)
222
+ grandchild = int(r.stdout_text.split()[0])
223
+ try:
224
+ assert r.timed_out
225
+ assert pid_alive(grandchild)
226
+ finally:
227
+ os.kill(grandchild, 9)
228
+
229
+
230
+ # --- output cap --------------------------------------------------------------------------------------------
231
+
232
+
233
+ def test_output_limit_stops_the_command_and_marks_truncated():
234
+ code = "import sys\nwhile True:\n sys.stdout.write('x' * 1000)\n sys.stdout.flush()\n"
235
+ r = py(code, max_output=5000, timeout=20, grace=0.5)
236
+ assert r.stop_reason == "output-limit"
237
+ assert r.truncated
238
+ assert len(r.stdout) == 5000
239
+ assert not r.timed_out
240
+ assert r.duration < 15
241
+
242
+
243
+ def test_output_limit_without_kill_drains_and_discards_the_excess():
244
+ r = py("print('y' * 20000)", max_output=100, stop_on_output_limit=False, timeout=20)
245
+ assert r.stop_reason == "exited" and r.exit_code == 0
246
+ assert r.truncated
247
+ assert len(r.stdout) == 100
248
+ assert r.stdout_tail.endswith(b"y\n") or r.stdout_tail.endswith(b"y\r\n")
249
+
250
+
251
+ def test_output_exactly_at_the_cap_is_not_truncated():
252
+ r = py("import sys; sys.stdout.write('z' * 50)", max_output=50)
253
+ assert len(r.stdout) == 50
254
+ assert not r.truncated
255
+
256
+
257
+ def test_cap_applies_to_each_stream_and_stderr_can_trigger_it():
258
+ code = "import sys\nwhile True:\n sys.stderr.write('e' * 1000)\n sys.stderr.flush()\n"
259
+ r = py(code, max_output=3000, timeout=20, grace=0.5)
260
+ assert r.stop_reason == "output-limit" and r.truncated
261
+ assert len(r.stderr) == 3000
262
+
263
+
264
+ def test_no_cap_when_max_output_is_none():
265
+ r = py("print('q' * 200000)", max_output=None)
266
+ assert len(r.stdout) > 200000 and not r.truncated
267
+
268
+
269
+ def test_tail_keeps_the_end_even_when_the_head_is_capped():
270
+ r = py("print('a' * 5000 + 'THE-END')", max_output=100, tail_bytes=20, stop_on_output_limit=False)
271
+ assert b"THE-END" in r.stdout_tail
272
+ assert len(r.stdout_tail) <= 20
273
+ assert b"THE-END" not in r.stdout
274
+
275
+
276
+ def test_tail_bytes_zero_disables_the_tail():
277
+ r = py("print('hello')", tail_bytes=0)
278
+ assert r.stdout_tail == b""
279
+
280
+
281
+ # --- cancel ------------------------------------------------------------------------------------------------
282
+
283
+
284
+ def test_cancel_stops_the_command():
285
+ cancel = threading.Event()
286
+ threading.Timer(0.5, cancel.set).start()
287
+ r = py("import time; print('go', flush=True); time.sleep(60)", cancel=cancel, grace=0.5)
288
+ assert r.cancelled and r.stop_reason == "cancelled"
289
+ assert r.stdout_text.strip() == "go"
290
+ assert r.duration < 10
291
+ assert not pid_alive(r.pid)
292
+
293
+
294
+ def test_cancel_set_before_start_returns_quickly():
295
+ cancel = threading.Event()
296
+ cancel.set()
297
+ r = py("import time; time.sleep(60)", cancel=cancel, grace=0.3)
298
+ assert r.cancelled
299
+ assert r.duration < 10
300
+
301
+
302
+ def test_cancel_that_is_never_set_is_harmless():
303
+ r = py("print(1)", cancel=threading.Event())
304
+ assert r.ok
305
+
306
+
307
+ # --- argument validation ------------------------------------------------------------------------------------
308
+
309
+
310
+ @pytest.mark.parametrize(
311
+ "kwargs",
312
+ [{"timeout": 0}, {"timeout": -1}, {"max_output": 0}, {"tail_bytes": -1}, {"grace": -1}],
313
+ )
314
+ def test_invalid_options_raise_value_error(kwargs):
315
+ with pytest.raises(ValueError):
316
+ run([PY, "-c", "pass"], **kwargs)
317
+
318
+
319
+ def test_empty_argv_raises_value_error():
320
+ with pytest.raises(ValueError):
321
+ run([])
322
+
323
+
324
+ # --- kill_tree ---------------------------------------------------------------------------------------------
325
+
326
+
327
+ @posix_only
328
+ def test_kill_tree_stops_a_popen_group_and_its_children():
329
+ code = (
330
+ "import subprocess, sys, time\n"
331
+ "c = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n"
332
+ "print(c.pid, flush=True)\n"
333
+ "time.sleep(60)\n"
334
+ )
335
+ proc = popen_group([PY, "-c", code], stdout=subprocess.PIPE)
336
+ grandchild = int(proc.stdout.readline())
337
+ info = kill_tree(proc, grace=2.0)
338
+ assert proc.poll() is not None
339
+ assert info.method == "killpg"
340
+ assert wait_dead(grandchild)
341
+ proc.stdout.close()
342
+
343
+
344
+ def test_kill_tree_on_an_exited_process_reports_already_exited():
345
+ proc = popen_group([PY, "-c", "pass"])
346
+ proc.wait()
347
+ info = kill_tree(proc, grace=0.2)
348
+ assert info.already_exited
349
+
350
+
351
+ @posix_only
352
+ def test_kill_tree_refuses_the_callers_own_group():
353
+ proc = subprocess.Popen([PY, "-c", "import time; time.sleep(30)"]) # same group as the test run
354
+ try:
355
+ with pytest.raises(ValueError):
356
+ kill_tree(proc)
357
+ finally:
358
+ proc.kill()
359
+ proc.wait()
360
+
361
+
362
+ @posix_only
363
+ def test_kill_tree_with_zero_grace_goes_straight_to_sigkill():
364
+ proc = popen_group([PY, "-c", "import time; time.sleep(30)"])
365
+ info = kill_tree(proc, grace=0)
366
+ assert info.forced
367
+ assert proc.returncode == -9
368
+
369
+
370
+ def test_kill_tree_rejects_negative_grace():
371
+ with pytest.raises(ValueError):
372
+ kill_tree(1, grace=-1)
373
+
374
+
375
+ @posix_only
376
+ def test_kill_tree_accepts_a_plain_pid():
377
+ proc = popen_group([PY, "-c", "import time; time.sleep(30)"])
378
+ kill_tree(proc.pid, grace=1.0)
379
+ proc.wait(timeout=5)
380
+ assert proc.returncode is not None