batchfetch 1.3.6__tar.gz → 1.3.7__tar.gz

{batchfetch-1.3.6 → batchfetch-1.3.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: batchfetch
-Version: 1.3.6
+Version: 1.3.7
 Summary: Efficiently clone and pull multiple Git repositories.
 Home-page: https://github.com/jamescherti/batchfetch
 Author: James Cherti
@@ -117,6 +117,7 @@ options:
 ```
 
 ## Features
+
 - Git Clone and Fetch/Merge: Clones the repositories and their submodules, ensuring that all the repositories are always up-to-date by fetching and merging changes.
 - Parallel Operations: Utilizes threads to simultaneously Git clone or pull multiple repositories, dramatically reducing wait times.
 - User-Friendly Interface: Provides simple and straightforward command-line options that make it easy to get started and effectively manage your repositories.
@@ -135,7 +136,7 @@ When *batchfetch* encounters an untracked file, it displays an error message to
 
 Here is an example of a *batchfetch.yaml* file that enables *batchfetch* to accept a list of untracked files:
 
-``` yaml
+```yaml
 options:
   ignore_untracked:
     - ./test
@@ -161,7 +162,8 @@ Batchfetch is fast, not only because it runs Git commands in parallel, but also
 When the user has specifies a revision (branch or commit reference), Batchfetch only performs a `git fetch` if that revision does not exist locally. If the revision is already up to date, it simply proceeds to the next repository in the queue.
 
 That's why it is highly recommended to always specify the revision to speed up Batchfetch, if speed is important to you. Here is an example of a `batchfetch.yaml` file where the branch (`1.1.0`) or commit reference (`b9c6d9b6134b4981760893254f804a371ffbc899`) is specified:
-``` yaml
+
+```yaml
 tasks:
   - git: https://github.com/jamescherti/outline-indent.el
     revision: "1.1.0"
@@ -171,12 +173,59 @@ tasks:
     revision: b9c6d9b6134b4981760893254f804a371ffbc899
 ```
 
+### How to configure additional Git remotes?
+
+You can define additional remotes for a Git repository using the `remote` option. Pass a list of key-value pairs where the key is the remote name and the value is the corresponding URL. Batchfetch will automatically ensure these remotes are present and point to the correct URLs.
+
+```yaml
+---
+tasks:
+  - git: https://github.com/jamescherti/easysession.el
+    path: emacs/easysession
+    remote:
+      upstream: https://github.com/name/repo
+      upstream2: https://github.com/name/repo2
+```
+
+### How to customize Git clone, merge, and update strategies?
+
+Batchfetch allows you to define advanced options for interacting with Git repositories. These parameters can be declared globally under the `options` block to apply to all tasks, or individually within a specific task to override the global defaults.
+
+* `git_clone_args`: A list of extra arguments to pass to `git clone`.
+* `git_merge_args`: A list of extra arguments to pass to `git merge`.
+* `git_update_strategy`: Specifies how updates should be applied. Valid options are `"merge"` (default), `"rebase"`, or `"reset"`.
+* `git_pull`: A boolean indicating whether to run Git pull/fetch operations. Set to `false` to disable updating entirely (default: `true`).
+
+Here is an example demonstrating both global and local configurations:
+
+```yaml
+---
+
+options:
+  # Apply these defaults to all tasks
+  git_update_strategy: reset
+  git_clone_args:
+    - --recurse-submodules
+
+tasks:
+  - git: https://github.com/jamescherti/compile-angel.el
+    git_pull: false # Do not attempt to update this repo
+
+  - git: https://github.com/jamescherti/outline-indent.el
+    # Override global strategy specifically for this task
+    git_update_strategy: merge
+    git_merge_args:
+      - --no-ff
+
+```
+
 ### How to execute a command before and after a task?
 
 To execute a command both before and after a specific task, you can define the `exec_before` and `exec_after` directives within the task configuration. These directives specify commands to be executed at the respective stages of the task lifecycle.
 
 Here is an example:
-``` yaml
+
+```yaml
 ---
 tasks:
   - git: https://github.com/jamescherti/easysession.el
@@ -192,6 +241,7 @@ To configure `batchfetch` to handle a specific path, you can define your tasks i
 #### Example `batchfetch.yml` file:
 
 In the following example, the `easysession` task clones two Git repositories:
+
 ```yaml
 ---
 tasks:
@@ -227,7 +277,7 @@ batchfetch
 
 ## License
 
-Copyright (C) 2024-2025 [James Cherti](https://www.jamescherti.com)
+Copyright (C) 2024-2026 [James Cherti](https://www.jamescherti.com)
 
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 

{batchfetch-1.3.6 → batchfetch-1.3.7}/README.md

@@ -83,6 +83,7 @@ options:
 ```
 
 ## Features
+
 - Git Clone and Fetch/Merge: Clones the repositories and their submodules, ensuring that all the repositories are always up-to-date by fetching and merging changes.
 - Parallel Operations: Utilizes threads to simultaneously Git clone or pull multiple repositories, dramatically reducing wait times.
 - User-Friendly Interface: Provides simple and straightforward command-line options that make it easy to get started and effectively manage your repositories.
@@ -101,7 +102,7 @@ When *batchfetch* encounters an untracked file, it displays an error message to
 
 Here is an example of a *batchfetch.yaml* file that enables *batchfetch* to accept a list of untracked files:
 
-``` yaml
+```yaml
 options:
   ignore_untracked:
     - ./test
@@ -127,7 +128,8 @@ Batchfetch is fast, not only because it runs Git commands in parallel, but also
 When the user has specifies a revision (branch or commit reference), Batchfetch only performs a `git fetch` if that revision does not exist locally. If the revision is already up to date, it simply proceeds to the next repository in the queue.
 
 That's why it is highly recommended to always specify the revision to speed up Batchfetch, if speed is important to you. Here is an example of a `batchfetch.yaml` file where the branch (`1.1.0`) or commit reference (`b9c6d9b6134b4981760893254f804a371ffbc899`) is specified:
-``` yaml
+
+```yaml
 tasks:
   - git: https://github.com/jamescherti/outline-indent.el
     revision: "1.1.0"
@@ -137,12 +139,59 @@ tasks:
     revision: b9c6d9b6134b4981760893254f804a371ffbc899
 ```
 
+### How to configure additional Git remotes?
+
+You can define additional remotes for a Git repository using the `remote` option. Pass a list of key-value pairs where the key is the remote name and the value is the corresponding URL. Batchfetch will automatically ensure these remotes are present and point to the correct URLs.
+
+```yaml
+---
+tasks:
+  - git: https://github.com/jamescherti/easysession.el
+    path: emacs/easysession
+    remote:
+      upstream: https://github.com/name/repo
+      upstream2: https://github.com/name/repo2
+```
+
+### How to customize Git clone, merge, and update strategies?
+
+Batchfetch allows you to define advanced options for interacting with Git repositories. These parameters can be declared globally under the `options` block to apply to all tasks, or individually within a specific task to override the global defaults.
+
+* `git_clone_args`: A list of extra arguments to pass to `git clone`.
+* `git_merge_args`: A list of extra arguments to pass to `git merge`.
+* `git_update_strategy`: Specifies how updates should be applied. Valid options are `"merge"` (default), `"rebase"`, or `"reset"`.
+* `git_pull`: A boolean indicating whether to run Git pull/fetch operations. Set to `false` to disable updating entirely (default: `true`).
+
+Here is an example demonstrating both global and local configurations:
+
+```yaml
+---
+
+options:
+  # Apply these defaults to all tasks
+  git_update_strategy: reset
+  git_clone_args:
+    - --recurse-submodules
+
+tasks:
+  - git: https://github.com/jamescherti/compile-angel.el
+    git_pull: false # Do not attempt to update this repo
+
+  - git: https://github.com/jamescherti/outline-indent.el
+    # Override global strategy specifically for this task
+    git_update_strategy: merge
+    git_merge_args:
+      - --no-ff
+
+```
+
 ### How to execute a command before and after a task?
 
 To execute a command both before and after a specific task, you can define the `exec_before` and `exec_after` directives within the task configuration. These directives specify commands to be executed at the respective stages of the task lifecycle.
 
 Here is an example:
-``` yaml
+
+```yaml
 ---
 tasks:
   - git: https://github.com/jamescherti/easysession.el
@@ -158,6 +207,7 @@ To configure `batchfetch` to handle a specific path, you can define your tasks i
 #### Example `batchfetch.yml` file:
 
 In the following example, the `easysession` task clones two Git repositories:
+
 ```yaml
 ---
 tasks:
@@ -193,7 +243,7 @@ batchfetch
 
 ## License
 
-Copyright (C) 2024-2025 [James Cherti](https://www.jamescherti.com)
+Copyright (C) 2024-2026 [James Cherti](https://www.jamescherti.com)
 
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch/__init__.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch/batchfetch_base.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under
@@ -16,12 +16,14 @@
 # You should have received a copy of the GNU General Public License along with
 # this program. If not, see <https://www.gnu.org/licenses/>.
 #
-"Base class used to run tasks in parallel."
+"""Base class used to run tasks in parallel."""
+
+from __future__ import annotations
 
 import os
 from copy import deepcopy
 from pathlib import Path
-from typing import Any, Dict, List
+from typing import Any
 
 from schema import Optional, Or, Schema
 
@@ -37,21 +39,21 @@ class DataAlreadyInitialized(Exception):
 
 
 class TaskBase:
-    def __init__(self, data: Dict[str, Any], options: Dict[str, Any]):
-        self.global_options_schema: Dict[Any, Any] = {}
-        self.global_options_values: Dict[str, Any] = options
+    def __init__(self, data: dict[str, Any], options: dict[str, Any]) -> None:
+        self.global_options_schema: dict[Any, Any] = {}
+        self.global_options_values: dict[str, Any] = options
 
-        self.task_schema: Dict[Any, Any] = {}
-        self.task_default_values: Dict[str, Any] = {}
+        self.task_schema: dict[Any, Any] = {}
+        self.task_default_values: dict[str, Any] = {}
 
         self._item_values = data
 
         # Variables
-        self.values: Dict[str, Any] = {}
-        self.options: Dict[str, Any] = {}
+        self.values: dict[str, Any] = {}
+        self.options: dict[str, Any] = {}
         self._values_initialized = False
 
-    def _initialize_data(self):
+    def _initialize_data(self) -> None:
         if self._values_initialized:
             raise DataAlreadyInitialized
 
@@ -76,10 +78,10 @@ class TaskBase:
             "changed": False,
         }
 
-    def validate_schema(self):
+    def validate_schema(self) -> None:
         self._initialize_data()
 
-    def __getitem__(self, key):
+    def __getitem__(self, key: str) -> Any:
         self._initialize_data()
 
         if key in self.values:
@@ -94,8 +96,8 @@ class TaskBase:
 class TaskBatchFetch(TaskBase):
     """Plugin downloader base class."""
 
-    def __init__(self, data: Dict[str, Any], options: Dict[str, Any]):
-        new_options: Dict[str, Any] = {"exec_before": [],
+    def __init__(self, data: dict[str, Any], options: dict[str, Any]) -> None:
+        new_options: dict[str, Any] = {"exec_before": [],
                                        "exec_after": [],
                                        "ignore_untracked": []}
         new_options.update(options)
@@ -107,13 +109,13 @@ class TaskBatchFetch(TaskBase):
         self.env = os.environ.copy()
 
         # Default
-        self.global_options_schema: Dict[Any, Any] = {
+        self.global_options_schema: dict[Any, Any] = {
             Optional("exec_before"): Or([str], str),
             Optional("exec_after"): Or([str], str),
             Optional("ignore_untracked"): Or([str], str),
         }
 
-        self.task_schema: Dict[Any, Any] = {
+        self.task_schema: dict[Any, Any] = {
             Optional("path"): str,
             Optional("delete"): bool,
 
@@ -124,12 +126,12 @@ class TaskBatchFetch(TaskBase):
             Optional("ignore_untracked"): Or([str], str),
         }
 
-        self.task_default_values: Dict[str, Any] = {
+        self.task_default_values: dict[str, Any] = {
             # Optional items
             "delete": False,
         }
 
-    def _initialize_data(self):
+    def _initialize_data(self) -> None:
         try:
             super()._initialize_data()
         except DataAlreadyInitialized:
@@ -138,14 +140,14 @@ class TaskBatchFetch(TaskBase):
         # Mark values as initialized
         self._values_initialized = True
 
-    def _local_task_exec(self, *args, **kwargs):
+    def _local_task_exec(self, *args: Any, **kwargs: Any) -> None:
         stdout = run_indent_str(env=self.env, spaces=self.indent,
                                 *args, **kwargs)
         if not stdout.endswith("\n"):
             stdout += "\n"
         self.add_output(stdout)
 
-    def _exec_before(self, cwd: os.PathLike = Path(".")):
+    def _exec_before(self, cwd: os.PathLike = Path(".")) -> None:
         self._initialize_data()
         if self["delete"]:
             return
@@ -162,9 +164,9 @@ class TaskBatchFetch(TaskBase):
         if cmd:
             self._local_task_exec(cmd, cwd=str(cwd))
 
-    def _exec_after(self, cwd: os.PathLike = Path(".")):
+    def _exec_after(self, cwd: os.PathLike = Path(".")) -> None:
         self._initialize_data()
-        if self["delete"] or not self.is_changed():
+        if self["delete"] or self.is_error() or not self.is_changed():
             return
 
         # Local
@@ -183,7 +185,7 @@ class TaskBatchFetch(TaskBase):
         self._initialize_data()
         return bool(self.values["result"]["changed"])
 
-    def set_changed(self, changed: bool):
+    def set_changed(self, changed: bool) -> None:
         self._initialize_data()
         self.values["result"]["changed"] = changed
 
@@ -197,15 +199,15 @@ class TaskBatchFetch(TaskBase):
         self._initialize_data()
         return bool(self.values["result"]["error"])
 
-    def set_error(self, error: bool):
+    def set_error(self, error: bool) -> None:
         self._initialize_data()
         self.values["result"]["error"] = error
 
-    def set_output(self, output: str):
+    def set_output(self, output: str) -> None:
         self._initialize_data()
         self.values["result"]["output"] = output
 
-    def add_output(self, output: str):
+    def add_output(self, output: str) -> None:
         self._initialize_data()
         self.values["result"]["output"] += output
 
@@ -214,6 +216,6 @@ class TaskBatchFetch(TaskBase):
         self._initialize_data()
         return str(self.values["result"]["output"])
 
-    def update(self):
+    def update(self) -> dict[str, Any]:
         self._initialize_data()
         return self.values

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch/batchfetch_cli.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under
@@ -18,6 +18,8 @@
 #
 """Command line interface."""
 
+from __future__ import annotations
+
 import argparse
 import logging
 import os
@@ -25,7 +27,7 @@ import subprocess
 import sys
 from concurrent.futures import ThreadPoolExecutor, as_completed
 from pathlib import Path
-from typing import Any, Dict, List, Set, Union
+from typing import Any
 
 import colorama
 import yaml  # type: ignore
@@ -33,35 +35,37 @@ from colorama import Fore
 from schema import Optional, Or, Schema, SchemaError
 from setproctitle import setproctitle
 
-from .batchfetch_base import BatchFetchError, TaskBatchFetch
+from .batchfetch_base import BatchFetchError
 from .batchfetch_git import BatchFetchGit
 from .helpers import collect_parent_dirs
 
 
+# pylint: disable=too-many-instance-attributes
 class BatchFetchCli:
     """Command-line-interface that downloads."""
 
+    # pylint: disable=too-many-positional-arguments
     def __init__(self, max_workers: int, verbose: bool, check_untracked: bool,
-                 targets: List[str], cwd: os.PathLike):
+                 targets: list[str], cwd: os.PathLike) -> None:
         os.chdir(cwd)
         self._cwd = cwd
         self._logger = logging.getLogger(self.__class__.__name__)
         self.targets = {Path(item).absolute() for item in targets}
         self.cfg: dict = {}
         self.check_untracked = check_untracked
-        self.tracked_paths: Dict[Path, Set[str]] = {}
-        self.ignore_untracked: Set[Path] = set()
+        self.tracked_paths: dict[Path, set[str]] = {}
+        self.ignore_untracked: set[Path] = set()
         self.verbose = verbose
         self.max_workers = max_workers
-        self.dirs_relative_to_batchfetch: Set[str] = set()
+        self.dirs_relative_to_batchfetch: set[str] = set()
 
         # Plugin
-        self.batchfetch_schemas: Dict[Any, Any] = {}
-        self.batchfetch_classes: Dict[str, TaskBatchFetch] = {}
-        self.cfg_schema: Dict[Any, Any] = {}
+        self.batchfetch_schemas: dict[Any, Any] = {}
+        self.batchfetch_classes: dict[str, Any] = {}
+        self.cfg_schema: dict[Any, Any] = {}
         self._plugin_add("git", BatchFetchGit)
 
-    def _plugin_add(self, keyword: str, batchfetch_class: Any):
+    def _plugin_add(self, keyword: str, batchfetch_class: Any) -> None:
         batchfetch_instance = batchfetch_class(data={},
                                                options={})
         batchfetch_instance.validate_schema()
@@ -76,7 +80,7 @@ class BatchFetchCli:
             ]
         }
 
-    def _plugin_get(self, raw_data: dict) -> str:
+    def _plugin_get(self, raw_data: dict[str, Any]) -> str:
         keyword_found = None
         for keyword in self.batchfetch_classes:
             if keyword in raw_data:
@@ -94,7 +98,8 @@ class BatchFetchCli:
 
         return keyword_found
 
-    def load(self, path: Path):
+    def load(self, path: Path) -> None:
+        # pylint: disable=too-many-try-statements
         try:
             with open(path, "r", encoding="utf-8") as fhandler:
                 yaml_dict = yaml.load(fhandler, Loader=yaml.FullLoader)
@@ -136,7 +141,7 @@ class BatchFetchCli:
         except OSError as err:
             raise BatchFetchError(str(err)) from err
 
-    def _loads(self, data: dict):
+    def _loads(self, data: dict[str, Any]) -> None:
         schema = Schema(self.cfg_schema)
         try:
             schema.validate(data)
@@ -154,11 +159,11 @@ class BatchFetchCli:
 
         self._loads_tasks(data)
 
-    def _loads_tasks(self, data: dict):
+    def _loads_tasks(self, data: dict[str, Any]) -> None:
         if "tasks" not in data:
             return
 
-        dict_local_dir = {}  # type: ignore
+        dict_local_dir: dict[str, Any] = {}  # type: ignore
         for task in data["tasks"]:
             keyword = self._plugin_get(task)
             batchfetch_class = self.batchfetch_classes[keyword]
@@ -183,6 +188,8 @@ class BatchFetchCli:
 
             dict_local_dir[str(dest_path)] = batchfetch_instance[keyword]
 
+    # pylint: disable=too-many-branches
+    # pylint: disable=too-many-statements
     def run_tasks(self) -> bool:
         failed = []
         error = False
@@ -192,6 +199,7 @@ class BatchFetchCli:
 
         executor_update = ThreadPoolExecutor(max_workers=self.max_workers)
 
+        # pylint: disable=too-many-try-statements
         try:
             self.dirs_relative_to_batchfetch = set()
 
@@ -209,8 +217,7 @@ class BatchFetchCli:
 
                 targets_not_found = self.targets - targets_executed
                 if targets_not_found:
-                    err_str = \
-                        f"Error: Target(s) not found: "
+                    err_str = "Error: Target(s) not found: "
                     err_str += ", ".join([str(item)
                                          for item in targets_not_found])
                     print(f"Error: {err_str}", file=sys.stderr)
@@ -282,14 +289,13 @@ class BatchFetchCli:
 
         return True
 
-    def _find_untracked_paths(self):
-        "Find the files that are untracked and should be deleted."
+    def _find_untracked_paths(self) -> None:
+        """Find the files that are untracked and should be deleted."""
         untracked_paths = set()
         cwd = Path.cwd()
-        local_ignore_untracked = set()
+        local_ignore_untracked: set[Path] = set()
 
         for tracked_dir, tracked_filenames in self.tracked_paths.items():
-            actual_filenames = {file.name for file in tracked_dir.iterdir()}
             self.ignore_untracked |= {tracked_dir}
             parents = collect_parent_dirs(cwd, tracked_dir)
             local_ignore_untracked |= parents
@@ -311,7 +317,7 @@ class BatchFetchCli:
             for path in untracked_paths:
                 err_str += ("  - " +
                             str(path.relative_to(self._cwd)) +
-                            (os.sep if path.is_dir() else "") +
+                            ("/" if path.is_dir() else "") +
                             "\n")
             err_str += ("The paths above are not managed by batchfetch."
                         " To retain them, add them to the "
@@ -320,7 +326,7 @@ class BatchFetchCli:
             raise BatchFetchError(err_str)
 
 
-def parse_args():
+def parse_args() -> argparse.Namespace:
     """Parse the command line arguments."""
     # Batchfetch file
     try:
@@ -330,20 +336,19 @@ def parse_args():
 
     # Jobs
     try:
-        jobs = os.environ["BATCHFETCH_JOBS"]
+        jobs_env = os.environ["BATCHFETCH_JOBS"]
     except KeyError:
         jobs = 5
     else:
-        if jobs:
-            jobs = int(jobs)
+        jobs = int(jobs_env)
 
     # Check untracked
     try:
-        check_untracked = os.environ["BATCHFETCH_CHECK_UNTRACKED"]
+        check_untracked_env = os.environ["BATCHFETCH_CHECK_UNTRACKED"]
     except KeyError:
         check_untracked = False
     else:
-        check_untracked = bool(check_untracked)
+        check_untracked = bool(check_untracked_env)
 
     desc = "Efficiently clone/pull multiple Git repositories in parallel."
     usage = "%(prog)s [--option] [TARGET]"
@@ -415,8 +420,9 @@ def parse_args():
     return args
 
 
-def command_line_interface():
+def command_line_interface() -> None:
     """Command line interface."""
+    # pylint: disable=too-many-try-statements
     try:
         errno = 0
         logging.basicConfig(level=logging.INFO, stream=sys.stdout,

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch/batchfetch_git.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under
@@ -16,17 +16,18 @@
 # You should have received a copy of the GNU General Public License along with
 # this program. If not, see <https://www.gnu.org/licenses/>.
 #
-"Clone and update Git repositories."
+"""Clone and update Git repositories."""
+
+from __future__ import annotations
 
 import os
-import posixpath
 import shutil
 import subprocess
 import textwrap
-from pathlib import Path
-from typing import List, Tuple, Union
+from pathlib import Path, PurePosixPath
+from typing import Any, Union
 
-from schema import Optional
+from schema import Optional, Or
 
 from .batchfetch_base import BatchFetchError, TaskBatchFetch
 from .helpers import run_simple
@@ -43,7 +44,7 @@ class GitRemoteError(Exception):
 class BatchFetchGit(TaskBatchFetch):
     """Clone or update a Git repository."""
 
-    def __init__(self, *args, **kwargs):
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
         self._git_fetch_origin_done = False
 
         self.branch_commit_ref = None
@@ -59,47 +60,53 @@ class BatchFetchGit(TaskBatchFetch):
             # Local options
             self.main_key: str,
             Optional("revision"): str,
+            Optional("remote"): {Optional(str): str},
 
             # Same as global options
             Optional("git_clone_args"): [str],
+            Optional("git_merge_args"): [str],
             Optional("git_pull"): bool,
+            Optional("git_update_strategy"): Or("merge", "rebase", "reset"),
         })
 
         self.global_options_schema.update({
             # Global options
             Optional("git_clone_args"): [str],
+            Optional("git_merge_args"): [str],
             Optional("git_pull"): bool,
+            Optional("git_update_strategy"): Or("merge", "rebase", "reset"),
         })
 
         # Data
         self.global_options_values.update({"git_clone_args": [],
+                                           "git_merge_args": [],
                                            "git_pull": True})
 
         self.task_default_values.update({
             self.main_key: "",
             "revision": "",
             "delete": False,
+            "remote": {},
         })
 
         self.git_local_dir = Path(self["path"])
         self.current_branch = None
         self.current_commit_ref = None
 
-    def _initialize_data(self):
+    def _initialize_data(self) -> None:
         super()._initialize_data()
 
         self.values[self.main_key] = \
             self.values[self.main_key].rstrip("/")
 
         if "path" not in self.values:
-            path = \
-                posixpath.basename(self.values[self.main_key])  # type: ignore
+            path = PurePosixPath(self.values[self.main_key]).name
             # Remove .git from the file name
             if path.endswith(".git"):
                 path = path[:-4]
             self.values["path"] = path
 
-    def update(self):
+    def update(self) -> dict[str, Any]:
         """Clone or update a Git repository."""
         super().update()
 
@@ -117,8 +124,9 @@ class BatchFetchGit(TaskBatchFetch):
 
         self.add_output(f"[GIT {update_type}] {self[self.main_key]}"
                         + (f" (Ref: {self['revision']})"
-                         if self["revision"] else "") + "\n")
+                           if self["revision"] else "") + "\n")
 
+        # pylint: disable=too-many-try-statements
         try:
             # Delete
             if self["delete"]:
@@ -137,6 +145,7 @@ class BatchFetchGit(TaskBatchFetch):
                     self._update_current_branch_name()
 
                 self._repo_fix_remote_origin()
+                self._repo_fix_remotes()
                 self._exec_before(cwd=self.git_local_dir)
 
                 if not self["revision"]:
@@ -148,19 +157,19 @@ class BatchFetchGit(TaskBatchFetch):
                         )
 
                     self.add_output(self.indent_spaces +
-                                    "[INFO] Update revision to: '" +
+                                    "[INFO] Default revision resolved to: '" +
                                     self["revision"] + "'\n")
 
+                git_fetch_done = False
                 if do_git_fetch:
                     git_fetch_done = self._repo_fetch()
 
                 self._repo_fix_branch()
 
                 if git_fetch_done:
-                    self._git_merge()
+                    self._git_apply_update_strategy()
 
-                if self.get_changed():
-                    self._exec_after(cwd=self.git_local_dir)
+                self._exec_after(cwd=self.git_local_dir)
         except BatchFetchError as err:
             self.set_error(True)
             self.add_output(self.indent_spaces + "[ERROR] " + str(err) + "\n")
@@ -177,25 +186,26 @@ class BatchFetchGit(TaskBatchFetch):
 
         return self.values
 
-    def _run_get_firstline(self, *args, **kwargs):
+    def _run_get_firstline(self, *args: Any, **kwargs: Any) -> str:
         stdout, _ = self._run(*args, **kwargs)
         try:
             return stdout[0]
         except IndexError:
             return ""
 
-    def _run(self, cmd: Union[List[str], str],
+    def _run(self, cmd: Union[list[str], str],
              cwd: Union[None, os.PathLike] = None,
              env: Union[None, dict] = None,
-             **kwargs) -> Tuple[List[str], List[str]]:
-        """
+             **kwargs: Any) -> tuple[list[str], list[str]]:
+        """Execute a command and return stdout and stderr.
+
         Executes a command and returns stdout and stderr as separate lists of
         strings.
 
         :param cmd: Command to be executed. Can be a list or a string.
+        :param cwd: Current working directory.
+        :param env: Environment variables.
         :param kwargs: Additional keyword arguments for Popen.
-        :cwd: Current working directory.
-        :env: Environment variables.
         :return: Tuple containing two lists: stdout lines and stderr lines.
         """
         if not cwd:
@@ -217,7 +227,7 @@ class BatchFetchGit(TaskBatchFetch):
             return ""
         return output
 
-    def _update_current_branch_name(self):
+    def _update_current_branch_name(self) -> None:
         try:
             # This returns the branch name
             stdout, _ = self._run(["git", "symbolic-ref", "--short", "HEAD"])
@@ -239,7 +249,7 @@ class BatchFetchGit(TaskBatchFetch):
             except GitRevisionDoesNotExist:
                 pass
 
-    def _repo_delete(self):
+    def _repo_delete(self) -> None:
         if not self.git_local_dir.exists():
             self.add_output(self.indent_spaces + "[INFO] Already deleted\n")
         elif not self.git_local_dir.joinpath(".git").is_dir():
@@ -255,7 +265,7 @@ class BatchFetchGit(TaskBatchFetch):
                             + f"[INFO] Deleted: '{self.git_local_dir}'")
             self.set_changed(True)
 
-    def _repo_clone(self):
+    def _repo_clone(self) -> None:
         git_clone_args = self["git_clone_args"]
         # git_clone_args += ["--recurse-submodules"]
 
@@ -264,7 +274,7 @@ class BatchFetchGit(TaskBatchFetch):
         self._run(cmd, cwd=".")
         self.set_changed(True)
 
-    def _repo_fetch(self):
+    def _repo_fetch(self) -> bool:
         # Merge
         do_git_fetch = self["git_pull"]
         do_git_fetch = False
@@ -304,26 +314,61 @@ class BatchFetchGit(TaskBatchFetch):
         self._git_fetch_origin()
         return True
 
-    def _git_merge(self):
-        git_merge = False
+    def _is_working_tree_clean(self) -> bool:
+        """Return True if the git working tree is clean."""
+        try:
+            stdout, _ = self._run(["git", "status", "--porcelain"])
+            return len(stdout) == 0
+        except subprocess.CalledProcessError:
+            return False
+
+    def _git_apply_update_strategy(self) -> bool:
+        git_updated = False
 
-        # Merge
+        # Merge, Rebase, or Reset
         real_branch = self._git_is_local_branch("HEAD")
         if real_branch and self.current_branch:
             # TODO: only merge when difference from upstream
             commit_ref_head = self._git_ref(cwd=self.git_local_dir)
-            self._run(["git", "merge", "--ff-only",
-                       f"origin/{self.current_branch}"])
-            git_ref_after_merge = self._git_ref(cwd=self.git_local_dir)
-            if commit_ref_head != git_ref_after_merge:
-                git_merge = True
+
+            try:
+                strategy = self["git_update_strategy"]
+            except KeyError:
+                strategy = "merge"
+
+            if strategy in ("rebase", "reset"):
+                if not self._is_working_tree_clean():
+                    raise BatchFetchError(
+                        f"Working tree is not clean. Aborting {strategy} to "
+                        "prevent data loss."
+                    )
+
+            self.add_output(
+                self.indent_spaces
+                + f"[INFO] Applying update strategy: {strategy} from "
+                f"origin/{self.current_branch}\n"
+            )
+
+            if strategy == "rebase":
+                self._run(["git", "rebase"] +
+                          [f"origin/{self.current_branch}"])
+            elif strategy == "reset":
+                self._run(["git", "reset", "--hard",
+                           f"origin/{self.current_branch}"])
+            else:
+                self._run(["git", "merge"] + self["git_merge_args"] +
+                          [f"origin/{self.current_branch}"])
+
+            git_ref_after_update = self._git_ref(cwd=self.git_local_dir)
+            if commit_ref_head != git_ref_after_update:
+                git_updated = True
                 self.set_changed(True)
                 self._run(["git", "log",
                            '--pretty=format:"%h %ad %s [%cn]"',
                            "--decorate", "--date=short",
-                           f"{commit_ref_head}..{git_ref_after_merge}"])
+                           f"{commit_ref_head}..{git_ref_after_update}"])
 
-        return git_merge
+        return git_updated
 
     def _git_get_remote_url(self, remote_name: str = "origin") -> str:
         origin_url = ""
@@ -348,7 +393,7 @@ class BatchFetchGit(TaskBatchFetch):
 
         try:
             stdout, _ = self._run(["git", "remote", "add", remote_name, url])
-            origin_url = stdout[0]
+            origin_url = stdout[0] if stdout else ""
         except (subprocess.CalledProcessError, IndexError) as err:
             raise GitRemoteError(
                 f"Failed to modify the Git remote url: {remote_name}") from err
@@ -356,7 +401,7 @@ class BatchFetchGit(TaskBatchFetch):
         return origin_url
 
     def _git_is_local_branch(self, branch: str) -> bool:
-        "Return True if it is a local branch that exists."
+        """Return True if it is a local branch that exists."""
         try:
             stdout, _ = self._run(["git", "rev-parse", "--symbolic-full-name",
                                    branch])
@@ -372,8 +417,8 @@ class BatchFetchGit(TaskBatchFetch):
 
         return False
 
-    def _git_rev_parse_verify(self, revision: str) -> List[str]:
-        stdout: List[str] = []
+    def _git_rev_parse_verify(self, revision: str) -> list[str]:
+        stdout: list[str] = []
         error = False
         try:
             stdout, _ = self._run(["git", "rev-parse", "--verify", revision])
@@ -427,14 +472,14 @@ class BatchFetchGit(TaskBatchFetch):
 
         return branch_changed
 
-    def _git_fetch_origin(self):
+    def _git_fetch_origin(self) -> None:
         # Fetch
         if not self._git_fetch_origin_done:
             cmd = ["git", "fetch", "origin"]
             self._run(cmd)
             self._git_fetch_origin_done = True
 
-    def _repo_fix_remote_origin(self):
+    def _repo_fix_remote_origin(self) -> None:
         correct_origin_url = self[self.main_key]
         update_remote_origin = False
 
@@ -471,3 +516,30 @@ class BatchFetchGit(TaskBatchFetch):
                     # TODO: handle errors
                 except subprocess.CalledProcessError as err:
                     raise BatchFetchError(str(err)) from err
+
+    def _repo_fix_remotes(self) -> None:
+        """Configure additional Git remotes defined in the task."""
+        remotes: dict[str, str] = self["remote"]
+        if not remotes:
+            return
+
+        for remote_name, correct_url in remotes.items():
+            update_remote = False
+            try:
+                current_url = self._git_get_remote_url(remote_name)
+                if current_url != correct_url:
+                    update_remote = True
+            except GitRemoteError:
+                update_remote = True
+
+            if update_remote:
+                try:
+                    self._git_set_remote_url(
+                        url=correct_url, remote_name=remote_name)
+                    self.add_output(self.indent_spaces
+                                    + f"[INFO] Remote '{remote_name}' "
+                                    f"set to {correct_url}\n")
+                except GitRemoteError as err:
+                    self.add_output(self.indent_spaces
+                                    + "[ERROR] Failed to set "
+                                    f"remote '{remote_name}': {err}\n")

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch/helpers.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under
@@ -16,26 +16,26 @@
 # You should have received a copy of the GNU General Public License along with
 # this program. If not, see <https://www.gnu.org/licenses/>.
 #
-"Batchfetch helper functions."
+"""Batchfetch helper functions."""
+
+from __future__ import annotations
 
 import hashlib
 import os
 import shlex
 from pathlib import Path
 from subprocess import PIPE, CalledProcessError, Popen, list2cmdline
-from typing import List, Set, Tuple, Union
+from typing import Any, Union
 
 
-def md5sum(filename: os.PathLike):
-    """
-    Calculate and return the MD5 checksum of a file.
+def md5sum(filename: os.PathLike) -> str:
+    """Calculate and return the MD5 checksum of a file.
 
     Args:
         filename: Path to the file for which the MD5 checksum is calculated.
 
     Returns:
         str: The MD5 checksum of the file.
-
     """
     md5 = hashlib.md5()
     with open(filename, "rb") as file:
@@ -44,9 +44,10 @@ def md5sum(filename: os.PathLike):
     return md5.hexdigest()
 
 
-def run_simple(cmd: Union[List[str], str],
-               **kwargs) -> Tuple[List[str], List[str]]:
-    """
+def run_simple(cmd: Union[list[str], str],
+               **kwargs: Any) -> tuple[list[str], list[str]]:
+    """Execute a command and return stdout and stderr.
+
     Executes a command and returns stdout and stderr as separate lists of
     strings.
 
@@ -72,8 +73,9 @@ def run_simple(cmd: Union[List[str], str],
     return (stdout_lines, stderr_lines)
 
 
-def run_indent_str(cmd: Union[List[str], str], **kwargs) -> str:
-    """
+def run_indent_str(cmd: Union[list[str], str], **kwargs: Any) -> str:
+    """Execute a command and return its stdout output as a single string.
+
     Executes a command and returns its stdout output as a single string with
     preserved line breaks.
 
@@ -87,9 +89,8 @@ def run_indent_str(cmd: Union[List[str], str], **kwargs) -> str:
     return "\n".join(stdout) + "\n"
 
 
-def indent_raw_output(raw_output: List[str], spaces: int = 4) -> List[str]:
-    """
-    Indents each line of the given list of strings.
+def indent_raw_output(raw_output: list[str], spaces: int = 4) -> list[str]:
+    """Indent each line of the given list of strings.
 
     :param raw_output: List of strings to indent.
     :param spaces: Number of spaces to indent each line.
@@ -99,9 +100,10 @@ def indent_raw_output(raw_output: List[str], spaces: int = 4) -> List[str]:
     return [indentation + line for line in raw_output]
 
 
-def run_indent(cmd: Union[List[str], str], spaces: int = 4,
-               **kwargs) -> Tuple[List[str], List[str]]:
-    """
+def run_indent(cmd: Union[list[str], str], spaces: int = 4,
+               **kwargs: Any) -> tuple[list[str], list[str]]:
+    """Execute a command and return its stdout and stderr output indented.
+
     Executes a command and returns its stdout and stderr output, both indented.
 
     :param cmd: Command to be executed, either as a list of strings or a single
@@ -119,15 +121,15 @@ def run_indent(cmd: Union[List[str], str], spaces: int = 4,
     return (stdout, stderr)
 
 
-def collect_parent_dirs(base_dir: Path, dir: Path) -> Set[Path]:
+def collect_parent_dirs(base_dir: Path, directory: Path) -> set[Path]:
     """Collect all parent directories of 'dir' until 'base_dir' is reached.
 
     If 'dir' is not inside 'base_dir', return None.
     """
-    parents: Set[Path] = set()
+    parents: set[Path] = set()
 
-    for dir_path, base_path in [(dir.resolve(), base_dir.resolve()),
-                                (dir.absolute(), base_dir.absolute())]:
+    for dir_path, base_path in ((directory.resolve(), base_dir.resolve()),
+                                (directory.absolute(), base_dir.absolute())):
         try:
             # Check if dir is inside base_dir
             dir_path.relative_to(base_path)

{batchfetch-1.3.6 → batchfetch-1.3.7}/batchfetch.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: batchfetch
-Version: 1.3.6
+Version: 1.3.7
 Summary: Efficiently clone and pull multiple Git repositories.
 Home-page: https://github.com/jamescherti/batchfetch
 Author: James Cherti
@@ -117,6 +117,7 @@ options:
 ```
 
 ## Features
+
 - Git Clone and Fetch/Merge: Clones the repositories and their submodules, ensuring that all the repositories are always up-to-date by fetching and merging changes.
 - Parallel Operations: Utilizes threads to simultaneously Git clone or pull multiple repositories, dramatically reducing wait times.
 - User-Friendly Interface: Provides simple and straightforward command-line options that make it easy to get started and effectively manage your repositories.
@@ -135,7 +136,7 @@ When *batchfetch* encounters an untracked file, it displays an error message to
 
 Here is an example of a *batchfetch.yaml* file that enables *batchfetch* to accept a list of untracked files:
 
-``` yaml
+```yaml
 options:
   ignore_untracked:
     - ./test
@@ -161,7 +162,8 @@ Batchfetch is fast, not only because it runs Git commands in parallel, but also
 When the user has specifies a revision (branch or commit reference), Batchfetch only performs a `git fetch` if that revision does not exist locally. If the revision is already up to date, it simply proceeds to the next repository in the queue.
 
 That's why it is highly recommended to always specify the revision to speed up Batchfetch, if speed is important to you. Here is an example of a `batchfetch.yaml` file where the branch (`1.1.0`) or commit reference (`b9c6d9b6134b4981760893254f804a371ffbc899`) is specified:
-``` yaml
+
+```yaml
 tasks:
   - git: https://github.com/jamescherti/outline-indent.el
     revision: "1.1.0"
@@ -171,12 +173,59 @@ tasks:
     revision: b9c6d9b6134b4981760893254f804a371ffbc899
 ```
 
+### How to configure additional Git remotes?
+
+You can define additional remotes for a Git repository using the `remote` option. Pass a list of key-value pairs where the key is the remote name and the value is the corresponding URL. Batchfetch will automatically ensure these remotes are present and point to the correct URLs.
+
+```yaml
+---
+tasks:
+  - git: https://github.com/jamescherti/easysession.el
+    path: emacs/easysession
+    remote:
+      upstream: https://github.com/name/repo
+      upstream2: https://github.com/name/repo2
+```
+
+### How to customize Git clone, merge, and update strategies?
+
+Batchfetch allows you to define advanced options for interacting with Git repositories. These parameters can be declared globally under the `options` block to apply to all tasks, or individually within a specific task to override the global defaults.
+
+* `git_clone_args`: A list of extra arguments to pass to `git clone`.
+* `git_merge_args`: A list of extra arguments to pass to `git merge`.
+* `git_update_strategy`: Specifies how updates should be applied. Valid options are `"merge"` (default), `"rebase"`, or `"reset"`.
+* `git_pull`: A boolean indicating whether to run Git pull/fetch operations. Set to `false` to disable updating entirely (default: `true`).
+
+Here is an example demonstrating both global and local configurations:
+
+```yaml
+---
+
+options:
+  # Apply these defaults to all tasks
+  git_update_strategy: reset
+  git_clone_args:
+    - --recurse-submodules
+
+tasks:
+  - git: https://github.com/jamescherti/compile-angel.el
+    git_pull: false # Do not attempt to update this repo
+
+  - git: https://github.com/jamescherti/outline-indent.el
+    # Override global strategy specifically for this task
+    git_update_strategy: merge
+    git_merge_args:
+      - --no-ff
+
+```
+
 ### How to execute a command before and after a task?
 
 To execute a command both before and after a specific task, you can define the `exec_before` and `exec_after` directives within the task configuration. These directives specify commands to be executed at the respective stages of the task lifecycle.
 
 Here is an example:
-``` yaml
+
+```yaml
 ---
 tasks:
   - git: https://github.com/jamescherti/easysession.el
@@ -192,6 +241,7 @@ To configure `batchfetch` to handle a specific path, you can define your tasks i
 #### Example `batchfetch.yml` file:
 
 In the following example, the `easysession` task clones two Git repositories:
+
 ```yaml
 ---
 tasks:
@@ -227,7 +277,7 @@ batchfetch
 
 ## License
 
-Copyright (C) 2024-2025 [James Cherti](https://www.jamescherti.com)
+Copyright (C) 2024-2026 [James Cherti](https://www.jamescherti.com)
 
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 

{batchfetch-1.3.6 → batchfetch-1.3.7}/setup.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under
@@ -22,7 +22,7 @@ from setuptools import find_packages, setup
 
 setup(
     name="batchfetch",
-    version="1.3.6",
+    version="1.3.7",
     packages=find_packages(),
     description="Efficiently clone and pull multiple Git repositories.",
     license="GPLv3",

{batchfetch-1.3.6 → batchfetch-1.3.7}/tests/test_helpers.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright (C) 2024-2025 James Cherti
+# Copyright (C) 2024-2026 James Cherti
 # URL: https://github.com/jamescherti/batchfetch
 #
 # This program is free software: you can redistribute it and/or modify it under