batchfetch 1.2.7__tar.gz → 1.2.8__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {batchfetch-1.2.7 → batchfetch-1.2.8}/PKG-INFO +42 -6
- {batchfetch-1.2.7 → batchfetch-1.2.8}/README.md +41 -5
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch/batchfetch_base.py +3 -3
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch/batchfetch_cli.py +55 -38
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch/batchfetch_git.py +44 -67
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/PKG-INFO +42 -6
- {batchfetch-1.2.7 → batchfetch-1.2.8}/setup.py +1 -1
- {batchfetch-1.2.7 → batchfetch-1.2.8}/LICENSE +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch/__init__.py +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch/helpers.py +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/SOURCES.txt +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/dependency_links.txt +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/entry_points.txt +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/requires.txt +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/batchfetch.egg-info/top_level.txt +0 -0
- {batchfetch-1.2.7 → batchfetch-1.2.8}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: batchfetch
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.8
|
|
4
4
|
Summary: Efficiently clone and pull multiple Git repositories.
|
|
5
5
|
Home-page: https://github.com/jamescherti/batchfetch
|
|
6
6
|
Author: James Cherti
|
|
@@ -73,22 +73,33 @@ Execute the `batchfetch` command from the same directory as `batchfetch.yml` to
|
|
|
73
73
|
Here are the various options that `batchfetch` provides, along with descriptions of their usage:
|
|
74
74
|
|
|
75
75
|
```
|
|
76
|
-
usage: batchfetch [--option]
|
|
76
|
+
usage: batchfetch [--option] [TARGET]
|
|
77
77
|
|
|
78
78
|
Efficiently clone/pull multiple Git repositories in parallel.
|
|
79
79
|
|
|
80
|
+
positional arguments:
|
|
81
|
+
target This is a target path that batchfetch is supposed to
|
|
82
|
+
handle. When no target is specified, execute the tasks
|
|
83
|
+
of all target paths defined in the batchfetch.yml list
|
|
84
|
+
of tasks.
|
|
85
|
+
|
|
80
86
|
options:
|
|
81
87
|
-h, --help show this help message and exit
|
|
82
88
|
-f FILE, --file FILE Specify the batchfetch YAML file (default:
|
|
83
89
|
'./batchfetch.yaml').
|
|
84
90
|
-C DIRECTORY, --directory DIRECTORY
|
|
85
91
|
Change the working directory before reading the
|
|
86
|
-
batchfetch.yaml file. If not specified, the directory
|
|
87
|
-
set to the parent directory of the batchfetch.yaml
|
|
92
|
+
batchfetch.yaml file. If not specified, the directory
|
|
93
|
+
is set to the parent directory of the batchfetch.yaml
|
|
94
|
+
file.
|
|
88
95
|
-j JOBS, --jobs JOBS Run up to N parallel processes (default: 5).
|
|
89
|
-
Alternatively, the BATCHFETCH_JOBS environment
|
|
90
|
-
can be used to configure the number of jobs.
|
|
96
|
+
Alternatively, the BATCHFETCH_JOBS environment
|
|
97
|
+
variable can be used to configure the number of jobs.
|
|
91
98
|
-v, --verbose Enable verbose mode.
|
|
99
|
+
-u, --check-untracked
|
|
100
|
+
Abort if untracked files or directories exist.
|
|
101
|
+
Alternatively, set the BATCHFETCH_CHECK_UNTRACKED=1
|
|
102
|
+
environment variable to enable this check.
|
|
92
103
|
```
|
|
93
104
|
|
|
94
105
|
## Features
|
|
@@ -160,6 +171,31 @@ tasks:
|
|
|
160
171
|
exec_after: ["sh", "-c", "echo exec_after_task"]
|
|
161
172
|
```
|
|
162
173
|
|
|
174
|
+
### How to make batchfetch handle only one path?
|
|
175
|
+
|
|
176
|
+
To configure `batchfetch` to handle a specific path, you can define your tasks in a `batchfetch.yml` file and pass the desired path as an argument to the `batchfetch` command.
|
|
177
|
+
|
|
178
|
+
#### Example `batchfetch.yml` file:
|
|
179
|
+
|
|
180
|
+
In the following example, the `easysession` task clones two Git repositories:
|
|
181
|
+
```yaml
|
|
182
|
+
---
|
|
183
|
+
tasks:
|
|
184
|
+
- git: https://github.com/jamescherti/easysession.el
|
|
185
|
+
path: easysession
|
|
186
|
+
|
|
187
|
+
- git: https://github.com/jamescherti/outline-indent.el
|
|
188
|
+
revision: "1.1.0"
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
To make `batchfetch` clone only `easysession`, pass its path as an argument:
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
batchfetch easysession
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
This will execute only the task corresponding to the `easysession` path, skipping all others in the `batchfetch.yml` file.
|
|
198
|
+
|
|
163
199
|
## License
|
|
164
200
|
|
|
165
201
|
Copyright (C) 2024 [James Cherti](https://www.jamescherti.com)
|
|
@@ -53,22 +53,33 @@ Execute the `batchfetch` command from the same directory as `batchfetch.yml` to
|
|
|
53
53
|
Here are the various options that `batchfetch` provides, along with descriptions of their usage:
|
|
54
54
|
|
|
55
55
|
```
|
|
56
|
-
usage: batchfetch [--option]
|
|
56
|
+
usage: batchfetch [--option] [TARGET]
|
|
57
57
|
|
|
58
58
|
Efficiently clone/pull multiple Git repositories in parallel.
|
|
59
59
|
|
|
60
|
+
positional arguments:
|
|
61
|
+
target This is a target path that batchfetch is supposed to
|
|
62
|
+
handle. When no target is specified, execute the tasks
|
|
63
|
+
of all target paths defined in the batchfetch.yml list
|
|
64
|
+
of tasks.
|
|
65
|
+
|
|
60
66
|
options:
|
|
61
67
|
-h, --help show this help message and exit
|
|
62
68
|
-f FILE, --file FILE Specify the batchfetch YAML file (default:
|
|
63
69
|
'./batchfetch.yaml').
|
|
64
70
|
-C DIRECTORY, --directory DIRECTORY
|
|
65
71
|
Change the working directory before reading the
|
|
66
|
-
batchfetch.yaml file. If not specified, the directory
|
|
67
|
-
set to the parent directory of the batchfetch.yaml
|
|
72
|
+
batchfetch.yaml file. If not specified, the directory
|
|
73
|
+
is set to the parent directory of the batchfetch.yaml
|
|
74
|
+
file.
|
|
68
75
|
-j JOBS, --jobs JOBS Run up to N parallel processes (default: 5).
|
|
69
|
-
Alternatively, the BATCHFETCH_JOBS environment
|
|
70
|
-
can be used to configure the number of jobs.
|
|
76
|
+
Alternatively, the BATCHFETCH_JOBS environment
|
|
77
|
+
variable can be used to configure the number of jobs.
|
|
71
78
|
-v, --verbose Enable verbose mode.
|
|
79
|
+
-u, --check-untracked
|
|
80
|
+
Abort if untracked files or directories exist.
|
|
81
|
+
Alternatively, set the BATCHFETCH_CHECK_UNTRACKED=1
|
|
82
|
+
environment variable to enable this check.
|
|
72
83
|
```
|
|
73
84
|
|
|
74
85
|
## Features
|
|
@@ -140,6 +151,31 @@ tasks:
|
|
|
140
151
|
exec_after: ["sh", "-c", "echo exec_after_task"]
|
|
141
152
|
```
|
|
142
153
|
|
|
154
|
+
### How to make batchfetch handle only one path?
|
|
155
|
+
|
|
156
|
+
To configure `batchfetch` to handle a specific path, you can define your tasks in a `batchfetch.yml` file and pass the desired path as an argument to the `batchfetch` command.
|
|
157
|
+
|
|
158
|
+
#### Example `batchfetch.yml` file:
|
|
159
|
+
|
|
160
|
+
In the following example, the `easysession` task clones two Git repositories:
|
|
161
|
+
```yaml
|
|
162
|
+
---
|
|
163
|
+
tasks:
|
|
164
|
+
- git: https://github.com/jamescherti/easysession.el
|
|
165
|
+
path: easysession
|
|
166
|
+
|
|
167
|
+
- git: https://github.com/jamescherti/outline-indent.el
|
|
168
|
+
revision: "1.1.0"
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
To make `batchfetch` clone only `easysession`, pass its path as an argument:
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
batchfetch easysession
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
This will execute only the task corresponding to the `easysession` path, skipping all others in the `batchfetch.yml` file.
|
|
178
|
+
|
|
143
179
|
## License
|
|
144
180
|
|
|
145
181
|
Copyright (C) 2024 [James Cherti](https://www.jamescherti.com)
|
|
@@ -138,7 +138,7 @@ class TaskBatchFetch(TaskBase):
|
|
|
138
138
|
# Mark values as initialized
|
|
139
139
|
self._values_initialized = True
|
|
140
140
|
|
|
141
|
-
def
|
|
141
|
+
def _pre_run(self, *args, **kwargs):
|
|
142
142
|
stdout = run_indent_str(*args, **kwargs)
|
|
143
143
|
|
|
144
144
|
if not stdout.endswith("\n"):
|
|
@@ -151,7 +151,7 @@ class TaskBatchFetch(TaskBase):
|
|
|
151
151
|
if self["delete"] or not cmd:
|
|
152
152
|
return
|
|
153
153
|
|
|
154
|
-
self.
|
|
154
|
+
self._pre_run(cmd, cwd=str(cwd), env=self.env, spaces=self.indent)
|
|
155
155
|
|
|
156
156
|
def _run_post_exec(self, cwd: os.PathLike = Path(".")):
|
|
157
157
|
self._initialize_data()
|
|
@@ -159,7 +159,7 @@ class TaskBatchFetch(TaskBase):
|
|
|
159
159
|
if not cmd or self["delete"] or not self.is_changed():
|
|
160
160
|
return
|
|
161
161
|
|
|
162
|
-
self.
|
|
162
|
+
self._pre_run(cmd, cwd=str(cwd), env=self.env, spaces=self.indent)
|
|
163
163
|
|
|
164
164
|
def is_changed(self) -> bool:
|
|
165
165
|
self._initialize_data()
|
|
@@ -25,7 +25,7 @@ import subprocess
|
|
|
25
25
|
import sys
|
|
26
26
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
27
27
|
from pathlib import Path
|
|
28
|
-
from typing import Any, Dict, Set, Union
|
|
28
|
+
from typing import Any, Dict, List, Set, Union
|
|
29
29
|
|
|
30
30
|
import colorama
|
|
31
31
|
import yaml # type: ignore
|
|
@@ -40,15 +40,16 @@ from .batchfetch_git import BatchFetchGit
|
|
|
40
40
|
class BatchFetchCli:
|
|
41
41
|
"""Command-line-interface that downloads."""
|
|
42
42
|
|
|
43
|
-
def __init__(self, max_workers: int, verbose: bool, check_untracked: bool
|
|
43
|
+
def __init__(self, max_workers: int, verbose: bool, check_untracked: bool,
|
|
44
|
+
targets: List[str]):
|
|
45
|
+
self._logger = logging.getLogger(self.__class__.__name__)
|
|
46
|
+
self.targets = {Path(item).absolute() for item in targets}
|
|
44
47
|
self.cfg: dict = {}
|
|
45
|
-
self.folder = Path(".")
|
|
46
48
|
self.check_untracked = check_untracked
|
|
47
49
|
self.tracked_paths: Dict[Path, Set[str]] = {}
|
|
48
50
|
self.ignore_untracked_paths: Set[Path] = set()
|
|
49
51
|
self.verbose = verbose
|
|
50
52
|
self.max_workers = max_workers
|
|
51
|
-
self._logger = logging.getLogger(self.__class__.__name__)
|
|
52
53
|
self.dirs_relative_to_batchfetch: Set[str] = set()
|
|
53
54
|
|
|
54
55
|
# Plugin
|
|
@@ -183,10 +184,31 @@ class BatchFetchCli:
|
|
|
183
184
|
self.dirs_relative_to_batchfetch = set()
|
|
184
185
|
|
|
185
186
|
all_tasks = self.cfg["tasks"]
|
|
187
|
+
|
|
188
|
+
# Check targets
|
|
189
|
+
if self.targets:
|
|
190
|
+
targets_executed = set()
|
|
191
|
+
for task in all_tasks:
|
|
192
|
+
full_path = Path(task["path"]).absolute()
|
|
193
|
+
if full_path not in self.targets:
|
|
194
|
+
continue
|
|
195
|
+
|
|
196
|
+
targets_executed.add(full_path)
|
|
197
|
+
|
|
198
|
+
targets_not_found = self.targets - targets_executed
|
|
199
|
+
if targets_not_found:
|
|
200
|
+
err_str = \
|
|
201
|
+
f"Error: Target(s) not found: "
|
|
202
|
+
err_str += ", ".join([str(item)
|
|
203
|
+
for item in targets_not_found])
|
|
204
|
+
print(f"Error: {err_str}", file=sys.stderr)
|
|
205
|
+
sys.exit(1)
|
|
206
|
+
|
|
186
207
|
for task in all_tasks:
|
|
208
|
+
full_path = Path(task["path"]).absolute()
|
|
209
|
+
|
|
187
210
|
self.dirs_relative_to_batchfetch.add(str(task["path"]))
|
|
188
211
|
if not task["delete"]:
|
|
189
|
-
full_path = Path(task["path"]).absolute()
|
|
190
212
|
base_path = full_path.parent
|
|
191
213
|
try:
|
|
192
214
|
self.tracked_paths[base_path]
|
|
@@ -195,6 +217,8 @@ class BatchFetchCli:
|
|
|
195
217
|
|
|
196
218
|
self.tracked_paths[base_path].add(full_path.name)
|
|
197
219
|
|
|
220
|
+
if self.targets and full_path not in self.targets:
|
|
221
|
+
continue
|
|
198
222
|
threads.append(executor_update.submit(task.update))
|
|
199
223
|
|
|
200
224
|
for future in as_completed(threads):
|
|
@@ -292,9 +316,16 @@ def parse_args():
|
|
|
292
316
|
check_untracked = bool(check_untracked)
|
|
293
317
|
|
|
294
318
|
desc = "Efficiently clone/pull multiple Git repositories in parallel."
|
|
295
|
-
usage = "%(prog)s [--option]"
|
|
319
|
+
usage = "%(prog)s [--option] [TARGET]"
|
|
296
320
|
parser = argparse.ArgumentParser(description=desc, usage=usage)
|
|
297
321
|
|
|
322
|
+
parser.add_argument(
|
|
323
|
+
"targets", metavar="target", type=str, nargs="*",
|
|
324
|
+
help=("This is a target path that batchfetch is supposed to handle. "
|
|
325
|
+
"When no target is specified, execute the tasks of all target "
|
|
326
|
+
"paths defined in the batchfetch.yml list of tasks."),
|
|
327
|
+
)
|
|
328
|
+
|
|
298
329
|
parser.add_argument("-f",
|
|
299
330
|
"--file",
|
|
300
331
|
default=None,
|
|
@@ -350,31 +381,6 @@ def parse_args():
|
|
|
350
381
|
return args
|
|
351
382
|
|
|
352
383
|
|
|
353
|
-
def run_batchfetch_procedure(file: Path,
|
|
354
|
-
directory: Union[None, Path],
|
|
355
|
-
verbose: bool,
|
|
356
|
-
jobs: int,
|
|
357
|
-
check_untracked: bool) -> int:
|
|
358
|
-
errno = 0
|
|
359
|
-
batchfetch_cli = BatchFetchCli(verbose=verbose,
|
|
360
|
-
max_workers=int(jobs),
|
|
361
|
-
check_untracked=check_untracked)
|
|
362
|
-
os.chdir(directory)
|
|
363
|
-
batchfetch_cli.load(file)
|
|
364
|
-
|
|
365
|
-
try:
|
|
366
|
-
if not batchfetch_cli.run_tasks():
|
|
367
|
-
errno = 1
|
|
368
|
-
except KeyboardInterrupt:
|
|
369
|
-
print("Interrupted.", file=sys.stderr)
|
|
370
|
-
errno = 1
|
|
371
|
-
except BatchFetchError as err:
|
|
372
|
-
print(f"Error: {err}.", file=sys.stderr)
|
|
373
|
-
errno = 1
|
|
374
|
-
|
|
375
|
-
return errno
|
|
376
|
-
|
|
377
|
-
|
|
378
384
|
def command_line_interface():
|
|
379
385
|
"""Command line interface."""
|
|
380
386
|
try:
|
|
@@ -388,7 +394,7 @@ def command_line_interface():
|
|
|
388
394
|
|
|
389
395
|
args = parse_args()
|
|
390
396
|
done = []
|
|
391
|
-
file = Path(args.file)
|
|
397
|
+
file = Path(args.file).absolute()
|
|
392
398
|
file_resolved = file.absolute()
|
|
393
399
|
if not file_resolved:
|
|
394
400
|
print(f"Error: cannot resolve the path {file}",
|
|
@@ -405,12 +411,23 @@ def command_line_interface():
|
|
|
405
411
|
print(f"[CHECK UNTRACKED] {args.check_untracked}")
|
|
406
412
|
print()
|
|
407
413
|
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
414
|
+
os.chdir(args.directory)
|
|
415
|
+
batchfetch_cli = BatchFetchCli(verbose=args.verbose,
|
|
416
|
+
max_workers=args.jobs,
|
|
417
|
+
check_untracked=args.check_untracked,
|
|
418
|
+
targets=args.targets)
|
|
419
|
+
batchfetch_cli.load(file)
|
|
420
|
+
|
|
421
|
+
try:
|
|
422
|
+
if not batchfetch_cli.run_tasks():
|
|
423
|
+
errno = 1
|
|
424
|
+
except KeyboardInterrupt:
|
|
425
|
+
print("Interrupted.", file=sys.stderr)
|
|
426
|
+
errno = 1
|
|
427
|
+
except BatchFetchError as err:
|
|
428
|
+
print(f"Error: {err}.", file=sys.stderr)
|
|
429
|
+
errno = 1
|
|
413
430
|
|
|
414
431
|
sys.exit(errno)
|
|
415
432
|
except BrokenPipeError:
|
|
416
|
-
|
|
433
|
+
sys.exit(1)
|
|
@@ -18,13 +18,13 @@
|
|
|
18
18
|
#
|
|
19
19
|
"Clone and update Git repositories."
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
import os
|
|
22
22
|
import posixpath
|
|
23
23
|
import shutil
|
|
24
24
|
import subprocess
|
|
25
25
|
import textwrap
|
|
26
26
|
from pathlib import Path
|
|
27
|
-
from typing import List, Union
|
|
27
|
+
from typing import List, Tuple, Union
|
|
28
28
|
|
|
29
29
|
from schema import Optional
|
|
30
30
|
|
|
@@ -173,6 +173,24 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
173
173
|
|
|
174
174
|
return self.values
|
|
175
175
|
|
|
176
|
+
def _run(self, cmd: Union[List[str], str],
|
|
177
|
+
cwd: Union[None, os.PathLike] = None,
|
|
178
|
+
env: Union[None, dict] = None,
|
|
179
|
+
**kwargs) -> Tuple[List[str], List[str]]:
|
|
180
|
+
"""
|
|
181
|
+
Executes a command and returns stdout and stderr as separate lists of
|
|
182
|
+
strings.
|
|
183
|
+
|
|
184
|
+
:param cmd: Command to be executed. Can be a list or a string.
|
|
185
|
+
:param kwargs: Additional keyword arguments for Popen.
|
|
186
|
+
:return: Tuple containing two lists: stdout lines and stderr lines.
|
|
187
|
+
"""
|
|
188
|
+
if not cwd:
|
|
189
|
+
cwd = self.git_local_dir
|
|
190
|
+
if not env:
|
|
191
|
+
env = self.env
|
|
192
|
+
return run_simple(cmd=cmd, env=env, cwd=cwd, **kwargs)
|
|
193
|
+
|
|
176
194
|
def _git_ref(self, cwd: Union[None, Path] = None) -> str:
|
|
177
195
|
"""Get the commit revision of HEAD.
|
|
178
196
|
|
|
@@ -180,7 +198,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
180
198
|
"""
|
|
181
199
|
cmd = ["git", "show-ref", "--head", "--verify", "HEAD"]
|
|
182
200
|
try:
|
|
183
|
-
stdout, _ =
|
|
201
|
+
stdout, _ = self._run(cmd, cwd=cwd)
|
|
184
202
|
output = stdout[0].split(" ")[0]
|
|
185
203
|
except IndexError:
|
|
186
204
|
return ""
|
|
@@ -189,11 +207,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
189
207
|
def _update_current_branch_name(self):
|
|
190
208
|
try:
|
|
191
209
|
# This returns the branch name
|
|
192
|
-
stdout, _ =
|
|
193
|
-
["git", "symbolic-ref", "--short", "HEAD"],
|
|
194
|
-
env=self.env,
|
|
195
|
-
cwd=self.git_local_dir,
|
|
196
|
-
)
|
|
210
|
+
stdout, _ = self._run(["git", "symbolic-ref", "--short", "HEAD"])
|
|
197
211
|
self.current_branch = stdout[0]
|
|
198
212
|
self.is_branch = True
|
|
199
213
|
except (IndexError, subprocess.CalledProcessError):
|
|
@@ -230,17 +244,17 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
230
244
|
|
|
231
245
|
def _repo_clone(self):
|
|
232
246
|
git_clone_args = self["git_clone_args"]
|
|
233
|
-
git_clone_args += ["--recurse-submodules"]
|
|
247
|
+
# git_clone_args += ["--recurse-submodules"]
|
|
234
248
|
|
|
235
249
|
cmd = ["git", "clone"] + git_clone_args + \
|
|
236
250
|
[self[self.main_key], str(self.git_local_dir)]
|
|
237
|
-
self._run(cmd,
|
|
251
|
+
self._run(cmd, cwd=".")
|
|
238
252
|
self.set_changed(True)
|
|
239
253
|
|
|
240
254
|
def _repo_reset(self):
|
|
241
255
|
# Remove local changes
|
|
242
256
|
cmd = ["git", "reset", "--hard", "HEAD"]
|
|
243
|
-
self._run(cmd, cwd=
|
|
257
|
+
self._run(cmd, cwd=".")
|
|
244
258
|
|
|
245
259
|
def _repo_fetch(self):
|
|
246
260
|
# Merge
|
|
@@ -256,10 +270,9 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
256
270
|
|
|
257
271
|
try:
|
|
258
272
|
# Check if the revision such as
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
except GitRevisionDoesNotExist:
|
|
273
|
+
stdout, _ = self._run(["git", "cat-file", "-e",
|
|
274
|
+
self["revision"]])
|
|
275
|
+
except subprocess.CalledProcessError:
|
|
263
276
|
do_git_fetch = True
|
|
264
277
|
self.add_output(
|
|
265
278
|
self.indent_spaces
|
|
@@ -273,7 +286,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
273
286
|
# Check if the branch is a tag or a branch
|
|
274
287
|
cmd = ["git", "show-ref", "--verify", "--quiet",
|
|
275
288
|
f"refs/heads/{self['revision']}"]
|
|
276
|
-
|
|
289
|
+
self._run(cmd)
|
|
277
290
|
self.add_output(
|
|
278
291
|
self.indent_spaces
|
|
279
292
|
+ "[INFO] Git fetch origin reason: "
|
|
@@ -300,8 +313,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
300
313
|
# TODO: only merge when difference from upstream
|
|
301
314
|
commit_ref_head = self._git_ref(cwd=self.git_local_dir)
|
|
302
315
|
self._run(["git", "merge", "--ff-only",
|
|
303
|
-
f"origin/{self.current_branch}"]
|
|
304
|
-
cwd=str(self.git_local_dir), env=self.env)
|
|
316
|
+
f"origin/{self.current_branch}"])
|
|
305
317
|
git_ref_after_merge = self._git_ref(cwd=self.git_local_dir)
|
|
306
318
|
if commit_ref_head != git_ref_after_merge:
|
|
307
319
|
git_merge = True
|
|
@@ -309,19 +321,15 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
309
321
|
self._run(["git", "log",
|
|
310
322
|
'--pretty=format:"%h %ad %s [%cn]"',
|
|
311
323
|
"--decorate", "--date=short",
|
|
312
|
-
f"{commit_ref_head}..{git_ref_after_merge}"]
|
|
313
|
-
cwd=str(self.git_local_dir),
|
|
314
|
-
env=self.env)
|
|
324
|
+
f"{commit_ref_head}..{git_ref_after_merge}"])
|
|
315
325
|
|
|
316
326
|
return git_merge
|
|
317
327
|
|
|
318
328
|
def _git_get_remote_url(self, remote_name: str = "origin") -> str:
|
|
319
329
|
origin_url = ""
|
|
320
330
|
try:
|
|
321
|
-
stdout, _ =
|
|
322
|
-
|
|
323
|
-
env=self.env,
|
|
324
|
-
cwd=self.git_local_dir)
|
|
331
|
+
stdout, _ = self._run(["git", "config",
|
|
332
|
+
f"remote.{remote_name}.url"])
|
|
325
333
|
origin_url = stdout[0]
|
|
326
334
|
except (subprocess.CalledProcessError, IndexError) as err:
|
|
327
335
|
raise GitRemoteError(
|
|
@@ -333,18 +341,13 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
333
341
|
remote_name: str = "origin") -> str:
|
|
334
342
|
origin_url = ""
|
|
335
343
|
try:
|
|
336
|
-
|
|
337
|
-
env=self.env, cwd=self.git_local_dir)
|
|
344
|
+
self._run(["git", "remote", "remove", remote_name])
|
|
338
345
|
except subprocess.CalledProcessError:
|
|
339
346
|
# Ignore when it cannot be removed when it does not exist
|
|
340
347
|
pass
|
|
341
348
|
|
|
342
349
|
try:
|
|
343
|
-
stdout, _ =
|
|
344
|
-
["git", "remote", "add", remote_name, url],
|
|
345
|
-
env=self.env,
|
|
346
|
-
cwd=self.git_local_dir,
|
|
347
|
-
)
|
|
350
|
+
stdout, _ = self._run(["git", "remote", "add", remote_name, url])
|
|
348
351
|
origin_url = stdout[0]
|
|
349
352
|
except (subprocess.CalledProcessError, IndexError) as err:
|
|
350
353
|
raise GitRemoteError(
|
|
@@ -355,9 +358,8 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
355
358
|
def _git_is_local_branch(self, branch: str) -> bool:
|
|
356
359
|
"Return True if it is a local branch that exists."
|
|
357
360
|
try:
|
|
358
|
-
stdout, _ =
|
|
359
|
-
|
|
360
|
-
cwd=self.git_local_dir)
|
|
361
|
+
stdout, _ = self._run(["git", "rev-parse", "--symbolic-full-name",
|
|
362
|
+
branch])
|
|
361
363
|
if not stdout:
|
|
362
364
|
return False
|
|
363
365
|
|
|
@@ -374,9 +376,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
374
376
|
stdout: List[str] = []
|
|
375
377
|
error = False
|
|
376
378
|
try:
|
|
377
|
-
stdout, _ =
|
|
378
|
-
env=self.env,
|
|
379
|
-
cwd=self.git_local_dir)
|
|
379
|
+
stdout, _ = self._run(["git", "rev-parse", "--verify", revision])
|
|
380
380
|
if not stdout:
|
|
381
381
|
error = True
|
|
382
382
|
except subprocess.CalledProcessError:
|
|
@@ -394,11 +394,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
394
394
|
if self["revision"]:
|
|
395
395
|
# We also need tags because sometimes, a branch
|
|
396
396
|
# returns a different commit revision
|
|
397
|
-
git_tags, _ =
|
|
398
|
-
["git", "tag", "--points-at", "HEAD"],
|
|
399
|
-
env=self.env,
|
|
400
|
-
cwd=self.git_local_dir,
|
|
401
|
-
)
|
|
397
|
+
git_tags, _ = self._run(["git", "tag", "--points-at", "HEAD"])
|
|
402
398
|
|
|
403
399
|
# Also check the commit revision in case
|
|
404
400
|
# branch is a commit revision instead of a tag
|
|
@@ -410,8 +406,8 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
410
406
|
except GitRevisionDoesNotExist:
|
|
411
407
|
# Check if the commit ref exists
|
|
412
408
|
try:
|
|
413
|
-
git_ref_branch =
|
|
414
|
-
0]
|
|
409
|
+
git_ref_branch = \
|
|
410
|
+
self._git_rev_parse_verify(self["revision"])[0]
|
|
415
411
|
except GitRevisionDoesNotExist as err:
|
|
416
412
|
raise BatchFetchError(f"The branch '{self['revision']}' "
|
|
417
413
|
"does not exist.") from err
|
|
@@ -419,8 +415,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
419
415
|
if git_ref_after_merge != git_ref_branch and \
|
|
420
416
|
self["revision"] not in git_tags:
|
|
421
417
|
# Update the branch
|
|
422
|
-
self._run(["git", "checkout"] + [self["revision"]]
|
|
423
|
-
cwd=str(self.git_local_dir), env=self.env)
|
|
418
|
+
self._run(["git", "checkout"] + [self["revision"]])
|
|
424
419
|
self.add_output(self.indent_spaces
|
|
425
420
|
+ "[INFO] Branch changed to "
|
|
426
421
|
+ self["revision"] + "\n")
|
|
@@ -436,26 +431,9 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
436
431
|
# Fetch
|
|
437
432
|
if not self._git_fetch_origin_done:
|
|
438
433
|
cmd = ["git", "fetch", "origin"]
|
|
439
|
-
self._run(cmd
|
|
434
|
+
self._run(cmd)
|
|
440
435
|
self._git_fetch_origin_done = True
|
|
441
436
|
|
|
442
|
-
# def _repo_update_submodules(self):
|
|
443
|
-
# # This parameter instructs Git to initiate the update
|
|
444
|
-
# # process for submodules:
|
|
445
|
-
# # 1. Git fetches the commits specified in the parent
|
|
446
|
-
# # repository's configuration for each submodule.
|
|
447
|
-
# # 2. Updates are based solely on the commit pointers stored
|
|
448
|
-
# # within the parent repository's submodule configuration.
|
|
449
|
-
# # 3. It does not directly consult the upstream repositories
|
|
450
|
-
# # of the submodules.
|
|
451
|
-
# # 4. Submodules are updated to reflect the exact commits
|
|
452
|
-
# # revision in the parent repository's configuration,
|
|
453
|
-
# # potentially lagging behind the latest changes made in the
|
|
454
|
-
# # upstream repositories.
|
|
455
|
-
# if self.git_local_dir.joinpath(".gitmodules").is_file():
|
|
456
|
-
# cmd = ["git", "submodule", "update", "--recursive"]
|
|
457
|
-
# self._run(cmd, cwd=str(self.git_local_dir), env=self.env)
|
|
458
|
-
|
|
459
437
|
def _repo_fix_remote_origin(self):
|
|
460
438
|
correct_origin_url = self[self.main_key]
|
|
461
439
|
update_remote_origin = False
|
|
@@ -489,8 +467,7 @@ class BatchFetchGit(TaskBatchFetch):
|
|
|
489
467
|
|
|
490
468
|
cmd = ["git", "branch",
|
|
491
469
|
f"--set-upstream-to=origin/{self.current_branch}"]
|
|
492
|
-
_, _ =
|
|
493
|
-
cwd=self.git_local_dir)
|
|
470
|
+
_, _ = self._run(cmd)
|
|
494
471
|
# TODO: handle errors
|
|
495
472
|
except subprocess.CalledProcessError as err:
|
|
496
473
|
raise BatchFetchError(str(err)) from err
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: batchfetch
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.8
|
|
4
4
|
Summary: Efficiently clone and pull multiple Git repositories.
|
|
5
5
|
Home-page: https://github.com/jamescherti/batchfetch
|
|
6
6
|
Author: James Cherti
|
|
@@ -73,22 +73,33 @@ Execute the `batchfetch` command from the same directory as `batchfetch.yml` to
|
|
|
73
73
|
Here are the various options that `batchfetch` provides, along with descriptions of their usage:
|
|
74
74
|
|
|
75
75
|
```
|
|
76
|
-
usage: batchfetch [--option]
|
|
76
|
+
usage: batchfetch [--option] [TARGET]
|
|
77
77
|
|
|
78
78
|
Efficiently clone/pull multiple Git repositories in parallel.
|
|
79
79
|
|
|
80
|
+
positional arguments:
|
|
81
|
+
target This is a target path that batchfetch is supposed to
|
|
82
|
+
handle. When no target is specified, execute the tasks
|
|
83
|
+
of all target paths defined in the batchfetch.yml list
|
|
84
|
+
of tasks.
|
|
85
|
+
|
|
80
86
|
options:
|
|
81
87
|
-h, --help show this help message and exit
|
|
82
88
|
-f FILE, --file FILE Specify the batchfetch YAML file (default:
|
|
83
89
|
'./batchfetch.yaml').
|
|
84
90
|
-C DIRECTORY, --directory DIRECTORY
|
|
85
91
|
Change the working directory before reading the
|
|
86
|
-
batchfetch.yaml file. If not specified, the directory
|
|
87
|
-
set to the parent directory of the batchfetch.yaml
|
|
92
|
+
batchfetch.yaml file. If not specified, the directory
|
|
93
|
+
is set to the parent directory of the batchfetch.yaml
|
|
94
|
+
file.
|
|
88
95
|
-j JOBS, --jobs JOBS Run up to N parallel processes (default: 5).
|
|
89
|
-
Alternatively, the BATCHFETCH_JOBS environment
|
|
90
|
-
can be used to configure the number of jobs.
|
|
96
|
+
Alternatively, the BATCHFETCH_JOBS environment
|
|
97
|
+
variable can be used to configure the number of jobs.
|
|
91
98
|
-v, --verbose Enable verbose mode.
|
|
99
|
+
-u, --check-untracked
|
|
100
|
+
Abort if untracked files or directories exist.
|
|
101
|
+
Alternatively, set the BATCHFETCH_CHECK_UNTRACKED=1
|
|
102
|
+
environment variable to enable this check.
|
|
92
103
|
```
|
|
93
104
|
|
|
94
105
|
## Features
|
|
@@ -160,6 +171,31 @@ tasks:
|
|
|
160
171
|
exec_after: ["sh", "-c", "echo exec_after_task"]
|
|
161
172
|
```
|
|
162
173
|
|
|
174
|
+
### How to make batchfetch handle only one path?
|
|
175
|
+
|
|
176
|
+
To configure `batchfetch` to handle a specific path, you can define your tasks in a `batchfetch.yml` file and pass the desired path as an argument to the `batchfetch` command.
|
|
177
|
+
|
|
178
|
+
#### Example `batchfetch.yml` file:
|
|
179
|
+
|
|
180
|
+
In the following example, the `easysession` task clones two Git repositories:
|
|
181
|
+
```yaml
|
|
182
|
+
---
|
|
183
|
+
tasks:
|
|
184
|
+
- git: https://github.com/jamescherti/easysession.el
|
|
185
|
+
path: easysession
|
|
186
|
+
|
|
187
|
+
- git: https://github.com/jamescherti/outline-indent.el
|
|
188
|
+
revision: "1.1.0"
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
To make `batchfetch` clone only `easysession`, pass its path as an argument:
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
batchfetch easysession
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
This will execute only the task corresponding to the `easysession` path, skipping all others in the `batchfetch.yml` file.
|
|
198
|
+
|
|
163
199
|
## License
|
|
164
200
|
|
|
165
201
|
Copyright (C) 2024 [James Cherti](https://www.jamescherti.com)
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|