git-rexec 1.0.3__tar.gz → 1.0.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {git_rexec-1.0.3/git_rexec.egg-info → git_rexec-1.0.4}/PKG-INFO +31 -31
- {git_rexec-1.0.3 → git_rexec-1.0.4}/README.md +30 -30
- {git_rexec-1.0.3 → git_rexec-1.0.4}/git-rexec +187 -141
- {git_rexec-1.0.3 → git_rexec-1.0.4/git_rexec.egg-info}/PKG-INFO +31 -31
- {git_rexec-1.0.3 → git_rexec-1.0.4}/git_rexec.egg-info/SOURCES.txt +0 -1
- {git_rexec-1.0.3 → git_rexec-1.0.4}/pyproject.toml +1 -1
- git_rexec-1.0.3/.gitignore +0 -6
- {git_rexec-1.0.3 → git_rexec-1.0.4}/git_rexec.egg-info/dependency_links.txt +0 -0
- {git_rexec-1.0.3 → git_rexec-1.0.4}/git_rexec.egg-info/requires.txt +0 -0
- {git_rexec-1.0.3 → git_rexec-1.0.4}/git_rexec.egg-info/top_level.txt +0 -0
- {git_rexec-1.0.3 → git_rexec-1.0.4}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: git-rexec
|
|
3
|
-
Version: 1.0.
|
|
3
|
+
Version: 1.0.4
|
|
4
4
|
Summary: Locate Git repositories and execute commands against them in parallel.
|
|
5
5
|
Author: James Cherti
|
|
6
6
|
License-Expression: GPL-3.0-or-later
|
|
@@ -21,36 +21,36 @@ Requires-Dist: setproctitle; extra == "extras"
|
|
|
21
21
|
The [git-rexec](https://github.com/jamescherti/git-rexec/) command-line tool recursively locates Git repositories within a directory and executes commands against them, either sequentially or in parallel.
|
|
22
22
|
|
|
23
23
|
Here are examples demonstrating how to use `git-rexec`:
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
24
|
+
|
|
25
|
+
Example 1: Execute `git status -s` across all discovered Git repositories (found by searching recursively under the current working directory) in parallel (`-p` or `--parallel`):
|
|
26
|
+
```
|
|
27
|
+
git-rexec -p -- git status -s
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Example 2: Fetch updates across all discovered repositories, limiting the concurrency to 5 background jobs (`-j 5`), which helps avoid network congestion or server rate limits when communicating with upstream Git remotes:
|
|
31
|
+
```bash
|
|
32
|
+
git-rexec -j 5 --parallel -- git fetch
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Example 3: Include sub-repositories (e.g., Git worktrees and submodules) alongside standard Git repositories during discovery (`-s` or `--include-sub-repos`), and execute `git status` against them:
|
|
36
|
+
```bash
|
|
37
|
+
git-rexec -s -p -- git status
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Example 4: Target a specific base directory (`~/projects`) using the `-C` flag to recursively discover repositories within it, while explicitly excluding a specific subfolder (`~/projects/archive`). This example executes `git status -s` in parallel for all discovered repositories except those within the excluded path:
|
|
41
|
+
```bash
|
|
42
|
+
git-rexec -C ~/projects --exclude-dir ~/projects/archive --parallel -- git status -s
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Example 5: Evaluate whether a `README.md` file exists in the repository (`sh -c "test -f README.md"`). If the condition returns an exit status of 0 (success), it counts the number of lines in that file (`wc -l README.md`):
|
|
46
|
+
```bash
|
|
47
|
+
git-rexec --if-exec 'sh -c "test -f README.md"' --parallel -- wc -l README.md
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Example 6: Print the paths of all discovered Git repositories:
|
|
51
|
+
```bash
|
|
52
|
+
git-rexec --print
|
|
53
|
+
```
|
|
54
54
|
|
|
55
55
|
If this helps your workflow, please support the project by **⭐ starring git-rexec on GitHub** and sharing it on your website, blog, Mastodon, Reddit, X, LinkedIn, or other social media platforms to help more Git users discover its benefits.
|
|
56
56
|
|
|
@@ -3,36 +3,36 @@
|
|
|
3
3
|
The [git-rexec](https://github.com/jamescherti/git-rexec/) command-line tool recursively locates Git repositories within a directory and executes commands against them, either sequentially or in parallel.
|
|
4
4
|
|
|
5
5
|
Here are examples demonstrating how to use `git-rexec`:
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
6
|
+
|
|
7
|
+
Example 1: Execute `git status -s` across all discovered Git repositories (found by searching recursively under the current working directory) in parallel (`-p` or `--parallel`):
|
|
8
|
+
```
|
|
9
|
+
git-rexec -p -- git status -s
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Example 2: Fetch updates across all discovered repositories, limiting the concurrency to 5 background jobs (`-j 5`), which helps avoid network congestion or server rate limits when communicating with upstream Git remotes:
|
|
13
|
+
```bash
|
|
14
|
+
git-rexec -j 5 --parallel -- git fetch
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Example 3: Include sub-repositories (e.g., Git worktrees and submodules) alongside standard Git repositories during discovery (`-s` or `--include-sub-repos`), and execute `git status` against them:
|
|
18
|
+
```bash
|
|
19
|
+
git-rexec -s -p -- git status
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Example 4: Target a specific base directory (`~/projects`) using the `-C` flag to recursively discover repositories within it, while explicitly excluding a specific subfolder (`~/projects/archive`). This example executes `git status -s` in parallel for all discovered repositories except those within the excluded path:
|
|
23
|
+
```bash
|
|
24
|
+
git-rexec -C ~/projects --exclude-dir ~/projects/archive --parallel -- git status -s
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Example 5: Evaluate whether a `README.md` file exists in the repository (`sh -c "test -f README.md"`). If the condition returns an exit status of 0 (success), it counts the number of lines in that file (`wc -l README.md`):
|
|
28
|
+
```bash
|
|
29
|
+
git-rexec --if-exec 'sh -c "test -f README.md"' --parallel -- wc -l README.md
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Example 6: Print the paths of all discovered Git repositories:
|
|
33
|
+
```bash
|
|
34
|
+
git-rexec --print
|
|
35
|
+
```
|
|
36
36
|
|
|
37
37
|
If this helps your workflow, please support the project by **⭐ starring git-rexec on GitHub** and sharing it on your website, blog, Mastodon, Reddit, X, LinkedIn, or other social media platforms to help more Git users discover its benefits.
|
|
38
38
|
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
#
|
|
7
7
|
# Author: James Cherti
|
|
8
8
|
# URL: https://github.com/jamescherti/git-rexec
|
|
9
|
-
# Version: 1.0.
|
|
9
|
+
# Version: 1.0.4
|
|
10
10
|
#
|
|
11
11
|
# Copyright (C) 2019-2026 James Cherti
|
|
12
12
|
#
|
|
@@ -30,6 +30,8 @@ and executing commands against them. It supports conditional filtering,
|
|
|
30
30
|
background execution, and sequential foreground execution.
|
|
31
31
|
"""
|
|
32
32
|
|
|
33
|
+
from __future__ import annotations
|
|
34
|
+
|
|
33
35
|
import argparse
|
|
34
36
|
import os
|
|
35
37
|
import shlex
|
|
@@ -41,16 +43,18 @@ from concurrent.futures import Future, ThreadPoolExecutor, as_completed
|
|
|
41
43
|
from dataclasses import dataclass
|
|
42
44
|
from multiprocessing import cpu_count
|
|
43
45
|
from pathlib import Path
|
|
44
|
-
from typing import Optional
|
|
45
46
|
|
|
46
47
|
try:
|
|
47
48
|
from colorama import Fore
|
|
48
49
|
from colorama import init as colorama_init
|
|
50
|
+
|
|
49
51
|
HAS_COLORAMA: bool = True
|
|
50
52
|
except ImportError:
|
|
51
53
|
HAS_COLORAMA = False
|
|
52
54
|
|
|
53
55
|
class Fore: # type: ignore
|
|
56
|
+
"""Provide fallback empty strings for missing colorama Fore attributes."""
|
|
57
|
+
|
|
54
58
|
YELLOW: str = ""
|
|
55
59
|
RED: str = ""
|
|
56
60
|
RESET: str = ""
|
|
@@ -65,6 +69,7 @@ class CommandResult:
|
|
|
65
69
|
:param stdout: Standard output content.
|
|
66
70
|
:param stderr: Standard error content.
|
|
67
71
|
"""
|
|
72
|
+
|
|
68
73
|
command: list[str]
|
|
69
74
|
returncode: int
|
|
70
75
|
stdout: str = ""
|
|
@@ -78,11 +83,12 @@ class RepoContext:
|
|
|
78
83
|
:param path: The absolute path to the repository root.
|
|
79
84
|
:param parallel_result: The result of the parallel execution, if any.
|
|
80
85
|
"""
|
|
86
|
+
|
|
81
87
|
path: Path
|
|
82
|
-
parallel_result:
|
|
88
|
+
parallel_result: CommandResult | None = None
|
|
83
89
|
|
|
84
90
|
|
|
85
|
-
def git_toplevel(repo_path: Path) ->
|
|
91
|
+
def git_toplevel(repo_path: Path) -> Path | None:
|
|
86
92
|
"""Return the absolute path to the top-level directory of a Git repository.
|
|
87
93
|
|
|
88
94
|
:param repo_path: Path inside the Git repository.
|
|
@@ -90,39 +96,78 @@ def git_toplevel(repo_path: Path) -> Optional[Path]:
|
|
|
90
96
|
Git repo.
|
|
91
97
|
"""
|
|
92
98
|
try:
|
|
93
|
-
proc: subprocess.CompletedProcess[str] =
|
|
94
|
-
["git", "-C", str(repo_path), "rev-parse",
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
+
proc: subprocess.CompletedProcess[str] = \
|
|
100
|
+
subprocess.run(["git", "-C", str(repo_path), "rev-parse",
|
|
101
|
+
"--show-toplevel"],
|
|
102
|
+
capture_output=True,
|
|
103
|
+
text=True,
|
|
104
|
+
check=True)
|
|
99
105
|
return Path(proc.stdout.strip()).resolve()
|
|
100
106
|
except subprocess.CalledProcessError:
|
|
101
107
|
return None
|
|
102
108
|
|
|
103
109
|
|
|
104
|
-
def
|
|
110
|
+
def git_find_repo(root: Path, exclude_paths: list[Path] | None = None) \
|
|
111
|
+
-> tuple[set[Path], set[Path]]:
|
|
105
112
|
"""Recursively find all Git repositories under the root path using Python.
|
|
106
113
|
|
|
107
114
|
:param root: Directory to search under.
|
|
115
|
+
:param exclude_paths: Directories to exclude to optimize traversal.
|
|
108
116
|
:return: A tuple containing standard repository paths and sub-repository
|
|
109
117
|
paths.
|
|
110
118
|
"""
|
|
119
|
+
root_res: Path = root.resolve()
|
|
120
|
+
|
|
121
|
+
# Paths are already resolved by discover_repos
|
|
122
|
+
ex_resolved: set[Path] = set(exclude_paths) if exclude_paths else set()
|
|
123
|
+
|
|
124
|
+
# Do not begin traversal if the root itself is excluded
|
|
111
125
|
repos: set[Path] = set()
|
|
112
126
|
sub_repos: set[Path] = set()
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
127
|
+
if ex_resolved and any((root_res == ex_path or
|
|
128
|
+
root_res.is_relative_to(ex_path))
|
|
129
|
+
for ex_path in ex_resolved):
|
|
130
|
+
return repos, sub_repos
|
|
131
|
+
|
|
132
|
+
# Store Path objects internally to simplify maintenance
|
|
133
|
+
dirs_to_visit: list[Path] = [root_res]
|
|
134
|
+
while dirs_to_visit:
|
|
135
|
+
current_dir: Path = dirs_to_visit.pop()
|
|
136
|
+
|
|
137
|
+
try:
|
|
138
|
+
with os.scandir(current_dir) as iter_scandir:
|
|
139
|
+
dirs_in_current: list[Path] = []
|
|
140
|
+
is_repo: bool = False
|
|
141
|
+
|
|
142
|
+
for entry in iter_scandir:
|
|
143
|
+
if entry.name == ".git":
|
|
144
|
+
is_repo = True
|
|
145
|
+
if entry.is_dir():
|
|
146
|
+
repos.add(current_dir)
|
|
147
|
+
elif entry.is_file():
|
|
148
|
+
sub_repos.add(current_dir)
|
|
149
|
+
|
|
150
|
+
# Found the repository, stop evaluating this directory
|
|
151
|
+
break
|
|
152
|
+
|
|
153
|
+
try:
|
|
154
|
+
# Using follow_symlinks=False avoids an extra
|
|
155
|
+
# is_symlink() call
|
|
156
|
+
if entry.is_dir(follow_symlinks=False):
|
|
157
|
+
entry_path: Path = Path(entry.path)
|
|
158
|
+
if ex_resolved and entry_path in ex_resolved:
|
|
159
|
+
continue
|
|
160
|
+
dirs_in_current.append(entry_path)
|
|
161
|
+
except OSError:
|
|
162
|
+
continue
|
|
163
|
+
|
|
164
|
+
if not is_repo:
|
|
165
|
+
# Append collected directories only if we are not inside a
|
|
166
|
+
# repository
|
|
167
|
+
dirs_to_visit.extend(dirs_in_current)
|
|
168
|
+
except OSError:
|
|
120
169
|
continue
|
|
121
170
|
|
|
122
|
-
# Clear dirnames to prevent os.walk from descending into
|
|
123
|
-
# subdirectories
|
|
124
|
-
dirnames.clear()
|
|
125
|
-
|
|
126
171
|
return repos, sub_repos
|
|
127
172
|
|
|
128
173
|
|
|
@@ -133,18 +178,19 @@ def get_sub_repos(repo_path: Path) -> set[Path]:
|
|
|
133
178
|
:return: A set of absolute paths to worktrees and submodules.
|
|
134
179
|
"""
|
|
135
180
|
sub_repos: set[Path] = set()
|
|
181
|
+
repo_path_resolved: Path = repo_path.resolve()
|
|
136
182
|
|
|
137
183
|
# Discover worktrees
|
|
138
184
|
# pylint: disable=too-many-try-statements
|
|
139
185
|
try:
|
|
140
|
-
wt_proc: subprocess.CompletedProcess[str] =
|
|
141
|
-
["git", "-C", str(repo_path),
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
current_wt:
|
|
186
|
+
wt_proc: subprocess.CompletedProcess[str] = \
|
|
187
|
+
subprocess.run(["git", "-C", str(repo_path),
|
|
188
|
+
"worktree", "list", "--porcelain"],
|
|
189
|
+
capture_output=True,
|
|
190
|
+
text=True,
|
|
191
|
+
check=True)
|
|
192
|
+
|
|
193
|
+
current_wt: Path | None = None
|
|
148
194
|
is_bare: bool = False
|
|
149
195
|
|
|
150
196
|
for line in wt_proc.stdout.splitlines():
|
|
@@ -152,8 +198,8 @@ def get_sub_repos(repo_path: Path) -> set[Path]:
|
|
|
152
198
|
|
|
153
199
|
# Git porcelain separates blocks with a blank line
|
|
154
200
|
if not line:
|
|
155
|
-
if current_wt and
|
|
156
|
-
and current_wt !=
|
|
201
|
+
if (current_wt and
|
|
202
|
+
not is_bare and current_wt != repo_path_resolved):
|
|
157
203
|
sub_repos.add(current_wt)
|
|
158
204
|
# Reset state for the next block
|
|
159
205
|
current_wt = None
|
|
@@ -166,7 +212,7 @@ def get_sub_repos(repo_path: Path) -> set[Path]:
|
|
|
166
212
|
is_bare = True
|
|
167
213
|
|
|
168
214
|
# Process the final block in case the output lacks a trailing newline
|
|
169
|
-
if current_wt and not is_bare and current_wt !=
|
|
215
|
+
if current_wt and not is_bare and current_wt != repo_path_resolved:
|
|
170
216
|
sub_repos.add(current_wt)
|
|
171
217
|
|
|
172
218
|
except subprocess.CalledProcessError:
|
|
@@ -174,13 +220,12 @@ def get_sub_repos(repo_path: Path) -> set[Path]:
|
|
|
174
220
|
|
|
175
221
|
# Discover submodules
|
|
176
222
|
try:
|
|
177
|
-
sm_proc: subprocess.CompletedProcess[str] =
|
|
178
|
-
["git", "-C", str(repo_path),
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
)
|
|
223
|
+
sm_proc: subprocess.CompletedProcess[str] = \
|
|
224
|
+
subprocess.run(["git", "-C", str(repo_path), "submodule",
|
|
225
|
+
"foreach", "--quiet", "pwd",],
|
|
226
|
+
capture_output=True,
|
|
227
|
+
text=True,
|
|
228
|
+
check=True)
|
|
184
229
|
|
|
185
230
|
for line in sm_proc.stdout.splitlines():
|
|
186
231
|
sm_path: Path = Path(line.strip()).resolve()
|
|
@@ -192,7 +237,8 @@ def get_sub_repos(repo_path: Path) -> set[Path]:
|
|
|
192
237
|
return sub_repos
|
|
193
238
|
|
|
194
239
|
|
|
195
|
-
def run_command_get_output(repo_path: Path,
|
|
240
|
+
def run_command_get_output(repo_path: Path,
|
|
241
|
+
cmd_list: list[str],
|
|
196
242
|
capture: bool = True) -> CommandResult:
|
|
197
243
|
"""Execute a shell command within a specific directory.
|
|
198
244
|
|
|
@@ -205,19 +251,13 @@ def run_command_get_output(repo_path: Path, cmd_list: list[str],
|
|
|
205
251
|
return CommandResult(command=[], returncode=0)
|
|
206
252
|
|
|
207
253
|
try:
|
|
208
|
-
proc: subprocess.CompletedProcess[str] =
|
|
209
|
-
cmd_list,
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
return CommandResult(
|
|
216
|
-
command=cmd_list,
|
|
217
|
-
returncode=proc.returncode,
|
|
218
|
-
stdout=proc.stdout if proc.stdout else "",
|
|
219
|
-
stderr=proc.stderr if proc.stderr else ""
|
|
220
|
-
)
|
|
254
|
+
proc: subprocess.CompletedProcess[str] = \
|
|
255
|
+
subprocess.run(cmd_list, cwd=repo_path, capture_output=capture,
|
|
256
|
+
text=True, check=False,)
|
|
257
|
+
return CommandResult(command=cmd_list,
|
|
258
|
+
returncode=proc.returncode,
|
|
259
|
+
stdout=proc.stdout if proc.stdout else "",
|
|
260
|
+
stderr=proc.stderr if proc.stderr else "")
|
|
221
261
|
except FileNotFoundError:
|
|
222
262
|
return CommandResult(
|
|
223
263
|
command=cmd_list,
|
|
@@ -226,8 +266,8 @@ def run_command_get_output(repo_path: Path, cmd_list: list[str],
|
|
|
226
266
|
)
|
|
227
267
|
|
|
228
268
|
|
|
229
|
-
def format_parallel_output(
|
|
230
|
-
|
|
269
|
+
def format_parallel_output(repo_path: Path,
|
|
270
|
+
result: CommandResult, quiet: bool = False) -> str:
|
|
231
271
|
"""Format the output of a parallel command for display.
|
|
232
272
|
|
|
233
273
|
:param repo_path: The repository path.
|
|
@@ -255,8 +295,9 @@ def format_parallel_output(
|
|
|
255
295
|
return header + indented_body
|
|
256
296
|
|
|
257
297
|
|
|
258
|
-
def process_repo(repo_path: Path,
|
|
259
|
-
|
|
298
|
+
def process_repo(repo_path: Path,
|
|
299
|
+
exec_parallel_cmd: list[str] | None,
|
|
300
|
+
if_exec_cmd: list[str] | None) -> RepoContext | None:
|
|
260
301
|
"""Process a single repository: check conditions and run parallel commands.
|
|
261
302
|
|
|
262
303
|
:param repo_path: The repository path.
|
|
@@ -265,22 +306,23 @@ def process_repo(repo_path: Path, exec_parallel_cmd: Optional[list[str]],
|
|
|
265
306
|
:return: RepoContext if processed successfully, None if filtered out.
|
|
266
307
|
"""
|
|
267
308
|
# Filter: if-exec
|
|
268
|
-
if_exec_cmd_list:
|
|
309
|
+
if_exec_cmd_list: list[str] | None = if_exec_cmd
|
|
269
310
|
if if_exec_cmd_list:
|
|
270
311
|
# We discard output for the filter check, only caring about exit code
|
|
271
|
-
filter_res: CommandResult = run_command_get_output(
|
|
272
|
-
|
|
273
|
-
|
|
312
|
+
filter_res: CommandResult = run_command_get_output(repo_path,
|
|
313
|
+
if_exec_cmd_list,
|
|
314
|
+
capture=True)
|
|
274
315
|
if filter_res.returncode != 0:
|
|
275
316
|
return None
|
|
276
317
|
|
|
277
318
|
# Action: exec-parallel
|
|
278
|
-
parallel_result:
|
|
279
|
-
exec_parallel_cmd_list:
|
|
319
|
+
parallel_result: CommandResult | None = None
|
|
320
|
+
exec_parallel_cmd_list: list[str] | None = exec_parallel_cmd
|
|
280
321
|
if exec_parallel_cmd_list:
|
|
281
|
-
parallel_result =
|
|
282
|
-
repo_path,
|
|
283
|
-
|
|
322
|
+
parallel_result = \
|
|
323
|
+
run_command_get_output(repo_path,
|
|
324
|
+
exec_parallel_cmd_list,
|
|
325
|
+
capture=True)
|
|
284
326
|
|
|
285
327
|
return RepoContext(path=repo_path, parallel_result=parallel_result)
|
|
286
328
|
|
|
@@ -300,12 +342,18 @@ def discover_repos(directory: Path,
|
|
|
300
342
|
repos: set[Path] = set()
|
|
301
343
|
worktrees: set[Path] = set()
|
|
302
344
|
|
|
345
|
+
list_ex_paths: list[Path] = (
|
|
346
|
+
[Path(ex_dir).resolve() for ex_dir in exclude_dirs]
|
|
347
|
+
if exclude_dirs
|
|
348
|
+
else []
|
|
349
|
+
)
|
|
350
|
+
|
|
303
351
|
# Discover all repositories
|
|
304
|
-
toplevel:
|
|
352
|
+
toplevel: Path | None = git_toplevel(directory)
|
|
305
353
|
if toplevel:
|
|
306
354
|
repos = {toplevel}
|
|
307
355
|
else:
|
|
308
|
-
repos, worktrees =
|
|
356
|
+
repos, worktrees = git_find_repo(directory, list_ex_paths)
|
|
309
357
|
|
|
310
358
|
# Extract and append sub-repositories concurrently if requested
|
|
311
359
|
if include_sub_repos:
|
|
@@ -321,25 +369,22 @@ def discover_repos(directory: Path,
|
|
|
321
369
|
|
|
322
370
|
repos.update(sub_repos)
|
|
323
371
|
|
|
324
|
-
# Filter out excluded directories
|
|
325
|
-
if not
|
|
372
|
+
# Filter out excluded directories (applies dynamically to sub-repos too)
|
|
373
|
+
if not list_ex_paths:
|
|
326
374
|
return repos
|
|
327
375
|
|
|
328
|
-
list_ex_paths: list[Path] = [Path(ex_dir).resolve()
|
|
329
|
-
for ex_dir in exclude_dirs]
|
|
330
376
|
return {
|
|
331
|
-
repo
|
|
332
|
-
|
|
377
|
+
repo
|
|
378
|
+
for repo in repos
|
|
379
|
+
if not any(repo.is_relative_to(ex_path)
|
|
333
380
|
for ex_path in list_ex_paths)
|
|
334
381
|
}
|
|
335
382
|
|
|
336
383
|
|
|
337
|
-
def execute_parallel_tasks(
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
max_workers: int
|
|
342
|
-
) -> list[RepoContext]:
|
|
384
|
+
def execute_parallel_tasks(repos: set[Path],
|
|
385
|
+
exec_parallel: list[str] | None,
|
|
386
|
+
if_exec: list[str] | None,
|
|
387
|
+
max_workers: int) -> list[RepoContext]:
|
|
343
388
|
"""Run discovery and parallel execution tasks.
|
|
344
389
|
|
|
345
390
|
:param repos: set of repositories to process.
|
|
@@ -351,16 +396,15 @@ def execute_parallel_tasks(
|
|
|
351
396
|
results: list[RepoContext] = []
|
|
352
397
|
|
|
353
398
|
with ThreadPoolExecutor(max_workers=max_workers) as executor:
|
|
354
|
-
futures: dict[Future[
|
|
355
|
-
executor.submit(
|
|
356
|
-
|
|
357
|
-
): repo for repo in repos
|
|
399
|
+
futures: dict[Future[RepoContext | None], Path] = {
|
|
400
|
+
executor.submit(process_repo, repo, exec_parallel, if_exec): repo
|
|
401
|
+
for repo in repos
|
|
358
402
|
}
|
|
359
403
|
|
|
360
404
|
for future in as_completed(futures):
|
|
361
405
|
repo_path: Path = futures[future]
|
|
362
406
|
try:
|
|
363
|
-
result:
|
|
407
|
+
result: RepoContext | None = future.result()
|
|
364
408
|
if result:
|
|
365
409
|
results.append(result)
|
|
366
410
|
except (OSError, ValueError) as exc:
|
|
@@ -378,70 +422,80 @@ def parse_args() -> argparse.Namespace:
|
|
|
378
422
|
parser: argparse.ArgumentParser = argparse.ArgumentParser(
|
|
379
423
|
description=("Find Git repositories and execute commands against "
|
|
380
424
|
"them in parallel."),
|
|
381
|
-
formatter_class=argparse.RawDescriptionHelpFormatter
|
|
425
|
+
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
382
426
|
)
|
|
383
427
|
parser.add_argument(
|
|
384
|
-
"-C",
|
|
428
|
+
"-C",
|
|
429
|
+
"--directory",
|
|
385
430
|
type=Path,
|
|
386
431
|
default=Path("."),
|
|
387
|
-
help="Root directory to search (defaults to current directory)"
|
|
432
|
+
help="Root directory to search (defaults to current directory)",
|
|
388
433
|
)
|
|
389
434
|
parser.add_argument(
|
|
390
435
|
"--exclude-dir",
|
|
391
436
|
action="append",
|
|
392
437
|
default=[],
|
|
393
|
-
help="Exclude a specific directory and all of its subdirectories"
|
|
438
|
+
help="Exclude a specific directory and all of its subdirectories",
|
|
394
439
|
)
|
|
395
440
|
parser.add_argument(
|
|
396
|
-
"-p",
|
|
441
|
+
"-p",
|
|
442
|
+
"--parallel",
|
|
397
443
|
action="store_true",
|
|
398
444
|
help="Execute the command in parallel using threads",
|
|
399
|
-
default=False
|
|
445
|
+
default=False,
|
|
400
446
|
)
|
|
401
447
|
parser.add_argument(
|
|
402
|
-
"-i",
|
|
448
|
+
"-i",
|
|
449
|
+
"--if-exec",
|
|
403
450
|
type=str,
|
|
404
451
|
help="Execute commands only if this check returns exit code 0.",
|
|
405
|
-
default=None
|
|
452
|
+
default=None,
|
|
406
453
|
)
|
|
407
454
|
parser.add_argument(
|
|
408
|
-
"-j",
|
|
455
|
+
"-j",
|
|
456
|
+
"--jobs",
|
|
409
457
|
type=int,
|
|
410
458
|
dest="max_workers",
|
|
411
459
|
help="Maximum number of processors/workers to use",
|
|
412
|
-
default=(cpu_count() or 1)
|
|
460
|
+
default=(cpu_count() or 1),
|
|
413
461
|
)
|
|
414
462
|
parser.add_argument(
|
|
415
|
-
"-q",
|
|
463
|
+
"-q",
|
|
464
|
+
"--quiet",
|
|
416
465
|
action="store_true",
|
|
417
|
-
help=(
|
|
418
|
-
|
|
419
|
-
|
|
466
|
+
help=(
|
|
467
|
+
"Quiet mode. Suppresses the informational log prefixes "
|
|
468
|
+
"([EXEC] and [EXEC-P]) that precede execution output."
|
|
469
|
+
),
|
|
470
|
+
default=False,
|
|
420
471
|
)
|
|
421
472
|
parser.add_argument(
|
|
422
|
-
"-s",
|
|
473
|
+
"-s",
|
|
474
|
+
"--include-sub-repos",
|
|
423
475
|
action="store_true",
|
|
424
476
|
help="Include sub-repositories (e.g., Git worktrees and submodules)",
|
|
425
|
-
default=False
|
|
477
|
+
default=False,
|
|
426
478
|
)
|
|
427
479
|
parser.add_argument(
|
|
428
480
|
"--print",
|
|
429
481
|
action="store_true",
|
|
430
482
|
help="Print the paths (only when no command is provided)",
|
|
431
|
-
default=False
|
|
483
|
+
default=False,
|
|
432
484
|
)
|
|
433
485
|
parser.add_argument(
|
|
434
486
|
"--print0",
|
|
435
487
|
action="store_true",
|
|
436
|
-
help=(
|
|
437
|
-
|
|
438
|
-
|
|
488
|
+
help=(
|
|
489
|
+
"Separate the paths with a null character (only when no "
|
|
490
|
+
"command is provided)"
|
|
491
|
+
),
|
|
492
|
+
default=False,
|
|
439
493
|
)
|
|
440
494
|
parser.add_argument(
|
|
441
495
|
"exec_cmd",
|
|
442
496
|
type=str,
|
|
443
497
|
nargs="*",
|
|
444
|
-
help="The command to execute. You can use -- to pass options."
|
|
498
|
+
help="The command to execute. You can use -- to pass options.",
|
|
445
499
|
)
|
|
446
500
|
|
|
447
501
|
args: argparse.Namespace = parser.parse_args()
|
|
@@ -485,6 +539,7 @@ def main() -> None:
|
|
|
485
539
|
try:
|
|
486
540
|
# pylint: disable=import-outside-toplevel
|
|
487
541
|
from setproctitle import setproctitle
|
|
542
|
+
|
|
488
543
|
setproctitle(Path(sys.argv[0]).name) # type: ignore
|
|
489
544
|
except ImportError:
|
|
490
545
|
# Optional dependency 'setproctitle' is not installed.
|
|
@@ -505,9 +560,8 @@ def main() -> None:
|
|
|
505
560
|
|
|
506
561
|
# Verify that the execution command exists
|
|
507
562
|
if args.exec_cmd and not shutil.which(args.exec_cmd[0]):
|
|
508
|
-
print(
|
|
509
|
-
|
|
510
|
-
file=sys.stderr)
|
|
563
|
+
print(f"Error: Command not found: '{args.exec_cmd[0]}'",
|
|
564
|
+
file=sys.stderr)
|
|
511
565
|
sys.exit(127)
|
|
512
566
|
|
|
513
567
|
# Discover Repositories
|
|
@@ -517,21 +571,20 @@ def main() -> None:
|
|
|
517
571
|
args.include_sub_repos)
|
|
518
572
|
|
|
519
573
|
# setup background command list if needed
|
|
520
|
-
exec_parallel_cmd:
|
|
574
|
+
exec_parallel_cmd: list[str] | None = None
|
|
521
575
|
if args.parallel and args.exec_cmd:
|
|
522
576
|
exec_parallel_cmd = args.exec_cmd
|
|
523
577
|
|
|
524
|
-
if_exec_cmd:
|
|
578
|
+
if_exec_cmd: list[str] | None = None
|
|
525
579
|
if args.if_exec:
|
|
526
580
|
if_exec_cmd = shlex.split(args.if_exec)
|
|
527
581
|
|
|
528
582
|
# Parallel Processing (Filter + Background Exec)
|
|
529
|
-
processed_repos: list[RepoContext] =
|
|
530
|
-
repos,
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
)
|
|
583
|
+
processed_repos: list[RepoContext] = \
|
|
584
|
+
execute_parallel_tasks(repos,
|
|
585
|
+
exec_parallel_cmd,
|
|
586
|
+
if_exec_cmd,
|
|
587
|
+
args.max_workers)
|
|
535
588
|
|
|
536
589
|
execution_errors: list[tuple[Path, CommandResult]] = []
|
|
537
590
|
|
|
@@ -541,8 +594,10 @@ def main() -> None:
|
|
|
541
594
|
|
|
542
595
|
# Handle Background Execution Results
|
|
543
596
|
if context.parallel_result:
|
|
544
|
-
output_display: str =
|
|
545
|
-
repo_path,
|
|
597
|
+
output_display: str = \
|
|
598
|
+
format_parallel_output(repo_path,
|
|
599
|
+
context.parallel_result,
|
|
600
|
+
args.quiet)
|
|
546
601
|
if output_display:
|
|
547
602
|
print(output_display, end="")
|
|
548
603
|
|
|
@@ -552,11 +607,9 @@ def main() -> None:
|
|
|
552
607
|
# Handle Foreground Execution
|
|
553
608
|
if args.exec_cmd and not args.parallel:
|
|
554
609
|
if not args.quiet:
|
|
555
|
-
print(
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
f"{Fore.RESET}"
|
|
559
|
-
)
|
|
610
|
+
print(f"{Fore.YELLOW}[EXEC] {repo_path}: "
|
|
611
|
+
f"{shlex.join(args.exec_cmd)}"
|
|
612
|
+
f"{Fore.RESET}")
|
|
560
613
|
|
|
561
614
|
# Run interactively/sequentially (capture=False allows interaction)
|
|
562
615
|
# However, for consistency with error tracking, we might want to
|
|
@@ -565,23 +618,18 @@ def main() -> None:
|
|
|
565
618
|
# We use subprocess.check_call to allow direct stdout/stderr
|
|
566
619
|
# flow unless we want to capture for error summary. To match
|
|
567
620
|
# previous logic, we let it flow to stdout.
|
|
568
|
-
subprocess.check_call(
|
|
569
|
-
args.exec_cmd,
|
|
570
|
-
cwd=repo_path
|
|
571
|
-
)
|
|
621
|
+
subprocess.check_call(args.exec_cmd, cwd=repo_path)
|
|
572
622
|
except subprocess.CalledProcessError as err:
|
|
573
623
|
# Construct a dummy result for the error summary
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
execution_errors.append((repo_path, failed_res))
|
|
624
|
+
execution_errors.append(
|
|
625
|
+
(repo_path,
|
|
626
|
+
CommandResult(command=args.exec_cmd,
|
|
627
|
+
returncode=err.returncode)))
|
|
579
628
|
except FileNotFoundError:
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
execution_errors.append((repo_path, failed_res))
|
|
629
|
+
execution_errors.append(
|
|
630
|
+
(repo_path,
|
|
631
|
+
CommandResult(command=args.exec_cmd,
|
|
632
|
+
returncode=127)))
|
|
585
633
|
|
|
586
634
|
# If no commands were run, just list the repo
|
|
587
635
|
if not args.exec_cmd:
|
|
@@ -598,8 +646,6 @@ if __name__ == "__main__":
|
|
|
598
646
|
try:
|
|
599
647
|
main()
|
|
600
648
|
except KeyboardInterrupt:
|
|
601
|
-
print("\nInterrupting...", file=sys.stderr)
|
|
602
649
|
sys.exit(130)
|
|
603
650
|
except BrokenPipeError:
|
|
604
|
-
print("\nInterrupting...", file=sys.stderr)
|
|
605
651
|
sys.exit(141)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: git-rexec
|
|
3
|
-
Version: 1.0.
|
|
3
|
+
Version: 1.0.4
|
|
4
4
|
Summary: Locate Git repositories and execute commands against them in parallel.
|
|
5
5
|
Author: James Cherti
|
|
6
6
|
License-Expression: GPL-3.0-or-later
|
|
@@ -21,36 +21,36 @@ Requires-Dist: setproctitle; extra == "extras"
|
|
|
21
21
|
The [git-rexec](https://github.com/jamescherti/git-rexec/) command-line tool recursively locates Git repositories within a directory and executes commands against them, either sequentially or in parallel.
|
|
22
22
|
|
|
23
23
|
Here are examples demonstrating how to use `git-rexec`:
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
24
|
+
|
|
25
|
+
Example 1: Execute `git status -s` across all discovered Git repositories (found by searching recursively under the current working directory) in parallel (`-p` or `--parallel`):
|
|
26
|
+
```
|
|
27
|
+
git-rexec -p -- git status -s
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Example 2: Fetch updates across all discovered repositories, limiting the concurrency to 5 background jobs (`-j 5`), which helps avoid network congestion or server rate limits when communicating with upstream Git remotes:
|
|
31
|
+
```bash
|
|
32
|
+
git-rexec -j 5 --parallel -- git fetch
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Example 3: Include sub-repositories (e.g., Git worktrees and submodules) alongside standard Git repositories during discovery (`-s` or `--include-sub-repos`), and execute `git status` against them:
|
|
36
|
+
```bash
|
|
37
|
+
git-rexec -s -p -- git status
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Example 4: Target a specific base directory (`~/projects`) using the `-C` flag to recursively discover repositories within it, while explicitly excluding a specific subfolder (`~/projects/archive`). This example executes `git status -s` in parallel for all discovered repositories except those within the excluded path:
|
|
41
|
+
```bash
|
|
42
|
+
git-rexec -C ~/projects --exclude-dir ~/projects/archive --parallel -- git status -s
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Example 5: Evaluate whether a `README.md` file exists in the repository (`sh -c "test -f README.md"`). If the condition returns an exit status of 0 (success), it counts the number of lines in that file (`wc -l README.md`):
|
|
46
|
+
```bash
|
|
47
|
+
git-rexec --if-exec 'sh -c "test -f README.md"' --parallel -- wc -l README.md
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Example 6: Print the paths of all discovered Git repositories:
|
|
51
|
+
```bash
|
|
52
|
+
git-rexec --print
|
|
53
|
+
```
|
|
54
54
|
|
|
55
55
|
If this helps your workflow, please support the project by **⭐ starring git-rexec on GitHub** and sharing it on your website, blog, Mastodon, Reddit, X, LinkedIn, or other social media platforms to help more Git users discover its benefits.
|
|
56
56
|
|
git_rexec-1.0.3/.gitignore
DELETED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|