yd-cli 0.9__tar.gz → 0.11__tar.gz

{yd_cli-0.9 → yd_cli-0.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: yd-cli
-Version: 0.9
+Version: 0.11
 Summary: CLI tool to synchronize directories using rsync.
 Author: Christian Heinze
 License-Expression: MIT
@@ -25,6 +25,26 @@ Description-Content-Type: text/markdown
 
 Build and execute `rsync` commands from *TOML* configuration files.
 
+## (Un)Install
+
+To install, run
+
+```bash
+uv tool install yd-cli
+```
+
+Then run
+
+```bash
+yd --install-completion
+```
+
+to install auto-completion in your shell.
+
+In `bash`, the completion code is stored in `~/.bash_completions/yd.sh` and that file is sourced from `~/.bashrc`.
+Remove both and call `uv tool uninstall yd-cli` to remove this tool.
+The below mentioned configuration files need to be removed separately.
+
 ## Create a config
 
 Create a new configuration called `photos` with:
@@ -72,7 +92,7 @@ Leading comment lines directly at the top of the file are treated as the configu
 | Key | Meaning |
 | --- | --- |
 | `src_home` | Base directory for all `src` paths. Relative paths are resolved from your home directory. |
-| `target_home` | Base directory for all target paths. Relative paths are resolved from your home directory. `strftime` placeholders such as `%Y-%m-%d` are supported. |
+| `target_home` | Base directory for all target paths. Relative paths are resolved from your home directory. |
 | `mtp_target` | Use in-place syncing for MTP targets. |
 | `backup` | Backup directory for replaced or deleted files. Relative paths are resolved from `target_home`. `strftime` placeholders are supported. |
 | `exclude` | Exclude patterns applied to every command. |
@@ -84,7 +104,7 @@ Leading comment lines directly at the top of the file are treated as the configu
 | Key | Meaning |
 | --- | --- |
 | `src` | Relative source directory below `src_home`. |
-| `target` | Relative target directory below `target_home`; defaults to `src`. `strftime` placeholders are supported. |
+| `target` | Relative target directory below `target_home`; defaults to `src`. `strftime` placeholders (`%Y`, `%m`, and `%d` only!) are supported. |
 | `delete_extra` | Delete files in the target that do not exist in the source. Defaults to `true`. |
 | `exclude` | Extra exclude patterns for this command only. |
 
@@ -95,6 +115,8 @@ Leading comment lines directly at the top of the file are treated as the configu
 - Exactly one of `--src-home -` and `--target-home -` may read from standard input in a single run.
 - If `backup` is omitted, `yd` creates a timestamped backup directory automatically unless `--no-backup` is specified.
 - If a configured source directory exists but is empty, `yd` reports that no synchronization was performed for that command. E.g., forgetting to mount an external drive does not delete all corresponding copies on harddrive).
+- If `strftime` placeholders are included in the `target`, then the `target_home` is scanned for directories that may have been created by this rule in the past. All matches are included as additional references in the synchronization: if a file in `src` is included in one of those reference directories, then it will not be copied to `target`.
+- If the `target` directory did not exist before and no files are transferred into it, then it will be removed again.
 
 ## Run a config
 

{yd_cli-0.9 → yd_cli-0.11}/README.md

@@ -2,6 +2,26 @@
 
 Build and execute `rsync` commands from *TOML* configuration files.
 
+## (Un)Install
+
+To install, run
+
+```bash
+uv tool install yd-cli
+```
+
+Then run
+
+```bash
+yd --install-completion
+```
+
+to install auto-completion in your shell.
+
+In `bash`, the completion code is stored in `~/.bash_completions/yd.sh` and that file is sourced from `~/.bashrc`.
+Remove both and call `uv tool uninstall yd-cli` to remove this tool.
+The below mentioned configuration files need to be removed separately.
+
 ## Create a config
 
 Create a new configuration called `photos` with:
@@ -49,7 +69,7 @@ Leading comment lines directly at the top of the file are treated as the configu
 | Key | Meaning |
 | --- | --- |
 | `src_home` | Base directory for all `src` paths. Relative paths are resolved from your home directory. |
-| `target_home` | Base directory for all target paths. Relative paths are resolved from your home directory. `strftime` placeholders such as `%Y-%m-%d` are supported. |
+| `target_home` | Base directory for all target paths. Relative paths are resolved from your home directory. |
 | `mtp_target` | Use in-place syncing for MTP targets. |
 | `backup` | Backup directory for replaced or deleted files. Relative paths are resolved from `target_home`. `strftime` placeholders are supported. |
 | `exclude` | Exclude patterns applied to every command. |
@@ -61,7 +81,7 @@ Leading comment lines directly at the top of the file are treated as the configu
 | Key | Meaning |
 | --- | --- |
 | `src` | Relative source directory below `src_home`. |
-| `target` | Relative target directory below `target_home`; defaults to `src`. `strftime` placeholders are supported. |
+| `target` | Relative target directory below `target_home`; defaults to `src`. `strftime` placeholders (`%Y`, `%m`, and `%d` only!) are supported. |
 | `delete_extra` | Delete files in the target that do not exist in the source. Defaults to `true`. |
 | `exclude` | Extra exclude patterns for this command only. |
 
@@ -72,6 +92,8 @@ Leading comment lines directly at the top of the file are treated as the configu
 - Exactly one of `--src-home -` and `--target-home -` may read from standard input in a single run.
 - If `backup` is omitted, `yd` creates a timestamped backup directory automatically unless `--no-backup` is specified.
 - If a configured source directory exists but is empty, `yd` reports that no synchronization was performed for that command. E.g., forgetting to mount an external drive does not delete all corresponding copies on harddrive).
+- If `strftime` placeholders are included in the `target`, then the `target_home` is scanned for directories that may have been created by this rule in the past. All matches are included as additional references in the synchronization: if a file in `src` is included in one of those reference directories, then it will not be copied to `target`.
+- If the `target` directory did not exist before and no files are transferred into it, then it will be removed again.
 
 ## Run a config
 

{yd_cli-0.9 → yd_cli-0.11}/pyproject.toml

@@ -4,7 +4,7 @@ requires = [ "uv-build>=0.10,<0.12" ]
 
 [project]
 name = "yd-cli"
-version = "0.9"
+version = "0.11"
 description = "CLI tool to synchronize directories using rsync."
 readme = "README.md"
 license = "MIT"

{yd_cli-0.9 → yd_cli-0.11}/src/yd/cli.py

@@ -27,6 +27,8 @@ if TYPE_CHECKING:
 
     from rich.text import Text
 
+log = logging.getLogger(__name__)
+
 env: yd.types.Environment
 console: rich.console.Console
 highlighter: StrEnumHighlighter
@@ -176,7 +178,6 @@ def _setup_io() -> None:
 
         _encode_json = msgspec.json.Encoder().encode
 
-        @dataclasses.dataclass(slots=True)
         class JsonLineFormatter(logging.Formatter):
             def format(self, record: logging.LogRecord) -> str:
                 return _encode_json(
@@ -267,7 +268,9 @@ def run(
 ) -> int:
     """Run a saved configuration by name."""
     import asyncio
-    import functools
+    import contextlib
+    import functools as ft
+    import shutil
     from pathlib import Path
 
     import rich.progress
@@ -291,7 +294,6 @@ def run(
                 keep_newer=keep_newer,
                 rename_speedup=rename_speedup,
                 no_backup=no_backup,
-                error_encoder=yd.io.encode_error,
             )
         )
     except yd.types.CommandBuildError as err:
@@ -312,7 +314,7 @@ def run(
             ),
             rich.progress.TextColumn(
                 "",
-                table_column=rich.table.Column(width=8),
+                table_column=rich.table.Column(min_width=8),
             ),
             *it.chain.from_iterable(
                 (
@@ -333,6 +335,8 @@ def run(
                 }
             ),
             console=console,
+            expand=True,
+            refresh_per_second=10,
         ) as progress,
     ):
         template = "{action} {filename} ({transfer_bytes} bytes)"
@@ -371,6 +375,13 @@ def run(
                 **counter,  # type: ignore[ty:invalid-argument-type]
                 total=1,
             )
+
+            target = Path(args[-1])
+            target_existed = target.is_dir()
+            progress_updater = ft.partial(
+                show_progress, counter=counter, taskid=taskid, target=target
+            )
+
             try:
                 runner.run(
                     yd.execution.run(
@@ -378,12 +389,7 @@ def run(
                         *args,
                         parser_stdout=yd.io.decode_from_stdout,
                         parser_stderr=yd.io.decode_from_stderr,
-                        consumer=functools.partial(
-                            show_progress,
-                            counter=counter,
-                            taskid=taskid,
-                            target=Path(args[-1]),
-                        ),
+                        consumer=progress_updater,
                     )
                 )
             except (KeyboardInterrupt, asyncio.CancelledError) as err:
@@ -396,6 +402,28 @@ def run(
             except (yd.types.OutputParseError, yd.types.OutputConsumeError) as err:
                 console.print("Failed to handle rsync output.", style="error")
                 raise typer.Exit(2) from err
+
+            # If target was created but no files were copied, then remove it.
+            if (
+                not target_existed
+                and target.is_dir()  # Ensure rglob can be used.
+                and not any(
+                    path.is_file() or path.is_symlink() for path in target.rglob("*")
+                )
+            ):
+                with contextlib.suppress(OSError):
+                    shutil.rmtree(target)
+                    log.info("Removed empty but newly created target `%s`.", target)
+                    runner.run(
+                        progress_updater(
+                            yd.types.Transaction(
+                                filename=target.as_posix(),
+                                transfer_bytes=0,
+                                info="*deleting",
+                            )
+                        )
+                    )
+
             progress.update(taskid, completed=1)
 
     return 0
@@ -437,17 +465,18 @@ def edit(
     ] = False,
 ) -> int:
     """Open an existing configuration or create a new one in the editor."""
-    import contextlib
     import subprocess
 
     if (editor_bin := env.editor_bin) is None:
         raise typer.BadParameter("environment variable `EDITOR` is not set")
 
-    is_nvim = False
-    with contextlib.suppress(subprocess.CalledProcessError):
+    try:
         vers = subprocess.run(
             [editor_bin, "--version"], encoding="utf-8", capture_output=True, check=True
         ).stdout
+    except subprocess.CalledProcessError:
+        is_nvim = False
+    else:
         is_nvim = re.match(r"\s*NVIM", vers) is not None
 
     config_path = env.config_dir / f"{cfg_name}.toml"

{yd_cli-0.9 → yd_cli-0.11}/src/yd/commands.py

@@ -5,17 +5,20 @@
 
 from __future__ import annotations
 
+import datetime as dt
 import itertools as it
 import logging
+import re
 import sys
 import uuid
+from dataclasses import dataclass
 from pathlib import Path
 
 from yd import types
 
 TYPE_CHECKING = False
 if TYPE_CHECKING:
-    from collections.abc import Callable, Iterable, Sequence
+    from collections.abc import Iterator, Sequence
 
 
 log = logging.getLogger(__name__)
@@ -47,7 +50,57 @@ def _format_path(path: Path, /) -> str:
     return f"{path.as_posix()}/"
 
 
-def _create_error_command(
+@dataclass(frozen=True, slots=True)
+class _DatePathMatch:
+    path: Path
+    relative_path: str
+    datetime: dt.datetime | None
+
+
+_PLACEHOLDER = re.compile("%Y|%m|%d")
+
+
+def _match_dirs(
+    *, rel_path_pattern: str, home: Path, env: types.Environment
+) -> Iterator[_DatePathMatch]:
+    """Find subdirectories matching a target pattern."""
+    if "%" not in rel_path_pattern:
+        log.debug("No date placeholder found in `%s`.", rel_path_pattern)
+        return
+
+    # Crude filter. More careful checking below.
+    glob_pattern = re.sub(
+        "(?P<f>[^/])$", r"\g<f>/", _PLACEHOLDER.sub("*", rel_path_pattern)
+    )
+    if "%" in glob_pattern:
+        raise ValueError("pattern contains unsupported date placeholders")
+    rel_path_pattern = re.sub("/$", "", rel_path_pattern)
+
+    log.debug("Searching for sub-directories of the form `%s`.", home / glob_pattern)
+    for path in home.glob(glob_pattern):
+        rel_path = path.relative_to(home).as_posix()
+        log.debug("Scanning for date in `%s`.", rel_path)
+        try:
+            parsed = dt.datetime.strptime(rel_path, rel_path_pattern)
+        except ValueError:
+            log.debug("Found no date in `%s`.", rel_path)
+            continue
+
+        log.info("Found date `%s` in `%s`.", parsed.strftime("%Y-%m-%d"), rel_path)
+        if (rendered := parsed.strftime(rel_path_pattern)) != rel_path:
+            log.debug("Path pattern rendered to `%s` != `%s`.", rendered, rel_path)
+            continue
+        if parsed.date() >= env.current_time.date():
+            log.debug(
+                "Date `%s` not before current date `%s`.",
+                parsed.strftime("%Y-%m-%d"),
+                env.current_time.strftime("%Y-%m-%d"),
+            )
+            continue
+        yield _DatePathMatch(path=path, relative_path=rel_path, datetime=parsed)
+
+
+def _create_echo_command(
     enc_msg: str, /, environment: types.Environment
 ) -> tuple[str, tuple[str, ...]]:
     if environment.echo_bin is not None:
@@ -67,8 +120,7 @@ def create(  # noqa: PLR0915
     rename_speedup: bool,
     no_backup: bool,
     out_format: str = '{"filename": "%n", "transfer_bytes": %b, "info": "%i"}',
-    error_encoder: Callable[[str], str],
-) -> Iterable[tuple[str, Sequence[str]]]:
+) -> Iterator[tuple[str, Sequence[str]]]:
     """Build CLI commands.
 
     If the source directory is empty, then an `echo` command will be created. If
@@ -91,7 +143,6 @@ def create(  # noqa: PLR0915
     if not (target_home := cg.target_home if target_home is None else target_home):
         raise types.CommandBuildError("missing target home specification")
 
-    target_home = env.current_time.strftime(target_home)
     src_hpath, target_hpath = (
         _complete_path(Path(src_home), home=env.home_dir),
         _complete_path(Path(target_home), home=env.home_dir),
@@ -160,9 +211,9 @@ def create(  # noqa: PLR0915
             msg = f"Source directory `{src}` is empty. No synchronization will be done."
             if cmd.delete_extra:
                 msg += " Delete the folder contents manually to synchronize."
-
             log.warning(msg)
-            yield _create_error_command(error_encoder(msg), environment=env)
+
+            yield _create_echo_command(msg, environment=env)
             continue
 
         target_spec = (
@@ -211,6 +262,29 @@ def create(  # noqa: PLR0915
             # A relative path will be considered relative to the destination directory.
             args.extend(("--backup", "--backup-dir", _format_path(target_backup_path)))
 
+        if cmd.target is not None and target_spec != cmd.target:
+            try:
+                cmp_dests = sorted(
+                    _match_dirs(
+                        rel_path_pattern=cmd.target, home=target_hpath, env=env
+                    ),
+                    key=lambda d: d.datetime,
+                    reverse=True,
+                )
+            except ValueError as err:
+                raise types.CommandBuildError(
+                    "unsupported date specifiers in target"
+                ) from err
+            # rsync restricts the number of `--compare-dest` specs to 20.
+            cmp_dests = [d.path.as_posix() for i, d in enumerate(cmp_dests) if i < 20]
+            log.info(
+                "Found reference targets `%s`.",
+                "`, `".join(d for d in cmp_dests),
+            )
+            args.extend(
+                it.chain.from_iterable(("--compare-dest", d) for d in cmp_dests)
+            )
+
         args.extend(
             f"--exclude={pattern}" for pattern in it.chain(cmd.exclude, cg.exclude)
         )

{yd_cli-0.9 → yd_cli-0.11}/src/yd/execution.py

@@ -116,6 +116,7 @@ async def run[T](
     consumer: Consumer[T],
 ) -> int:
     proc: asyncio.subprocess.Process | None = None
+    log.info("Running `%s`.", (binary, *args))
     try:
         proc = await asyncio.create_subprocess_exec(
             binary,

{yd_cli-0.9 → yd_cli-0.11}/src/yd/io.py

@@ -35,7 +35,6 @@ __all__ = [
     "list_available_configs",
     "decode_from_stdout",
     "decode_from_stderr",
-    "encode_error",
 ]
 
 
@@ -195,7 +194,3 @@ def decode_from_stderr(text: str, /) -> types.Transaction | str | None:
     if not (stripped_text := text.strip()):
         return None
     return stripped_text
-
-
-def encode_error(text: str, /) -> str:
-    return text

{yd_cli-0.9 → yd_cli-0.11}/src/yd/types.py

@@ -70,10 +70,6 @@ class CommandGroup(msgspec.Struct, kw_only=True, forbid_unknown_fields=True):
     src_home/target_home
         Absolute path used to turn relative src/target path in individual commands into
         absolute paths.
-
-        The `target_home` is passed to `datetime.strftime` via the `format` parameter,
-        that is, placeholders for date/time components like `%Y`, `%m`, etc., are
-        replaced with the actual values at runtime.
     mtp_target
         Set True if communication with the target works with MTP. Due to
         technical limitations of that protocol, synchronization needs to be done
@@ -84,7 +80,7 @@ class CommandGroup(msgspec.Struct, kw_only=True, forbid_unknown_fields=True):
         Path to backup directory. Added to each command and if relative, then taken
         as relative to target home. If None, a default relative path is used.
 
-        The `target` is passed to `datetime.strftime` via the `format` parameter, that
+        The `backup` is passed to `datetime.strftime` via the `format` parameter, that
         is, placeholders for date/time components like `%Y`, `%m`, etc., are replaced
         with the actual values at runtime.
     exclude