git-private2public 0.2.2__tar.gz → 0.2.3__tar.gz

{git_private2public-0.2.2 → git_private2public-0.2.3}/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## 0.2.3
+
+- Preserve existing public target tags unchanged during snapshot publication when explicitly enabled.
+- Continue rejecting unconfigured refs and verify preserved tag object IDs after the branch push.
+- Add current-tree scan path exemptions and prune unreachable pre-snapshot objects before the history scan.
+- Keep history scanning complete because one blob can be reachable through both exempt and non-exempt paths.
+
 ## 0.2.2
 
 - Fail closed with a concise redacted error when target-ref preflight times out.

{git_private2public-0.2.2 → git_private2public-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: git-private2public
-Version: 0.2.2
+Version: 0.2.3
 Summary: Like .gitignore, but for what goes public. Keep a sanitized public mirror of your private repo.
 Project-URL: Homepage, https://github.com/megamen32/git-private2public
 Project-URL: Repository, https://github.com/megamen32/git-private2public

git_private2public-0.2.3/README.zh.md

@@ -0,0 +1,46 @@
+# git-private2public
+
+[![PyPI version](https://img.shields.io/pypi/v/git-private2public.svg)](https://pypi.org/project/git-private2public/) [![CI](https://github.com/megamen32/git-private2public/actions/workflows/ci.yml/badge.svg)](https://github.com/megamen32/git-private2public/actions/workflows/ci.yml)
+
+**[English](./README.md)** · **[Русский](./README.ru.md)** · **简体中文**
+
+![git-private2public 仓库审查工作台](docs/assets/readme-hero.png)
+
+**像 `.gitignore` 一样，但控制哪些内容可以公开。**
+
+## 快速开始
+
+项目需要 Python 3.9+。使用一条命令安装 CLI 及其 Git 清理引擎：
+
+```bash
+python3 -m pip install git-private2public
+```
+
+在要同步的私有仓库中初始化配置：
+
+```bash
+git-private2public init
+```
+
+编辑 `.gitpublic/config` 设置 `source` 与 `target`，再编辑 `.gitpublic/ignore` 添加不应公开的文件：
+
+```text
+.env
+secrets/
+*.key
+```
+
+## 安全发布
+
+先执行 dry-run 查看计划，再发布：
+
+```bash
+git-private2public publish --dry-run
+git-private2public publish
+```
+
+工具基于 `git-filter-repo` 和 YAML 规则创建干净的公开镜像。完整规则见 [高级配置](./docs/ADVANCED.md)。
+
+## 许可证
+
+MIT

{git_private2public-0.2.2 → git_private2public-0.2.3}/docs/ADVANCED.md

@@ -24,10 +24,16 @@ Missing custom files mean “no custom rules for that step”. Built-in credenti
 | `.gitpublic/ignore` | Removes matching files/paths from all published history | No files are removed |
 | `.gitpublic/replace` | Rewrites matching text in remaining files | No text is rewritten |
 | `.gitpublic/scan` | Adds custom blocking patterns | Built-in credential rules still protect zero-config scans and guard mode |
+| `.gitpublic/scan_ignore` | Skips matching paths in the current sanitized tree scan | Every current-tree file is scanned |
 | `.gitpublic/allow` | Exceptions for domain-like matches found by `scan` | No scan exceptions exist |
 | `.gitpublic/domains` | Same as `allow` alias | Nothing is allowlisted |
 | `.gitpublic/public/` | Adds or replaces files in the sanitized public tree | No public-only overlay is applied |
 
+`scan_ignore` does not exempt history blobs. Git can share one blob across
+multiple paths, so a displayed history path cannot safely identify every use
+of that content. Snapshot mode prunes unreachable private history before the
+history scan; history mode still checks all reachable content.
+
 ## `.gitpublic/config`
 
 Example:
@@ -38,11 +44,19 @@ target = you/public-repo
 push_force = true
 push_branches = main
 push_tags = false
+preserve_target_tags = true
 ```
 
-For a first public release, `snapshot` mode publishes only the sanitized
-current tracked tree as one neutral root commit. It removes private commit
-messages, authors, branches, tags, and previous history before filtering:
+For a public snapshot, `snapshot` mode publishes the sanitized current tracked
+tree as one neutral root commit. It removes private commit messages, authors,
+branches, tags, and previous history before filtering. Set
+`preserve_target_tags = true` to keep tags already on the public target at their
+exact existing object IDs; private source tags are still never published. Other
+unconfigured refs remain a hard error, and the publisher verifies every retained
+tag after updating the configured branches. Existing tags are not sent in the
+push. A concurrent remote ref change after preflight is detected by the final
+check and can be reported after the branch update. Reconcile the target before
+retrying:
 
 ```ini
 mode = snapshot
@@ -50,7 +64,8 @@ snapshot_include_source_sha = false
 ```
 
 Set `snapshot_include_source_sha = true` only when intentionally exposing the
-private source commit ID. Snapshot mode rejects `push_tags = true`. Normal
+private source commit ID. Snapshot mode rejects `push_tags = true`; the source
+tag publisher and existing-target-tag preservation cannot be combined. Normal
 `history` mode remains the default.
 
 `source` and `target` can be:

{git_private2public-0.2.2 → git_private2public-0.2.3}/docs/ADVANCED.ru.md

@@ -24,10 +24,16 @@
 | `.gitpublic/ignore` | удаляет совпавшие файлы/пути из публичной истории | файлы не удаляются |
 | `.gitpublic/replace` | заменяет совпавший текст в оставшихся файлах | текст не заменяется |
 | `.gitpublic/scan` | добавляет пользовательские блокирующие правила | встроенные правила продолжают работать в zero-config и guard-режимах |
+| `.gitpublic/scan_ignore` | пропускает совпавшие пути при проверке текущего очищенного дерева | проверяются все файлы текущего дерева |
 | `.gitpublic/allow` | исключения для доменов, найденных проверкой `scan` | исключений для проверки нет |
 | `.gitpublic/domains` | alias для `allow` | ничего не разрешается |
 | `.gitpublic/public/` | добавляет или заменяет файлы в очищенном public-дереве | public-only overlay не применяется |
 
+`scan_ignore` не исключает содержимое истории. Один Git blob может быть доступен
+по нескольким путям, поэтому путь, показанный для объекта истории, не доказывает,
+что этот текст используется только в исключённом файле. Snapshot удаляет старую
+недостижимую историю до проверки; режим `history` проверяет все достижимые данные.
+
 ## `.gitpublic/config`
 
 Пример:
@@ -38,11 +44,18 @@ target = you/public-repo
 push_force = true
 push_branches = main
 push_tags = false
+preserve_target_tags = true
 ```
 
-Для первой публичной выкладки режим `snapshot` публикует только
-очищенное текущее дерево tracked-файлов одним нейтральным root-коммитом.
-История, авторы, сообщения коммитов, ветки и теги приватного repo в зеркало не попадают:
+Режим `snapshot` публикует очищенное текущее дерево tracked-файлов одним
+нейтральным root-коммитом. История, авторы, сообщения коммитов, ветки и теги
+приватного repo не попадают в зеркало. Параметр `preserve_target_tags = true`
+сохраняет теги, уже существующие в публичном repo, с теми же object ID; теги
+приватного исходника по-прежнему не публикуются. Неизвестные refs блокируют
+публикацию, а издатель сверяет SHA каждого сохранённого тега после обновления веток:
+Теги не отправляются в push. Если другой процесс изменит refs после проверки
+перед публикацией, итоговая сверка обнаружит конфликт; он может обнаружиться уже
+после обновления ветки. Перед повтором сначала сверьте зеркало:
 
 ```ini
 mode = snapshot
@@ -50,7 +63,8 @@ snapshot_include_source_sha = false
 ```
 
 `snapshot_include_source_sha = true` ставьте только если нужно сознательно раскрыть SHA
-приватного коммита. `snapshot` не разрешает `push_tags = true`. По умолчанию
+приватного коммита. `snapshot` не разрешает `push_tags = true`; публикацию тегов
+из приватного исходника нельзя совмещать с сохранением публичных тегов. По умолчанию
 остаётся обычный режим `history`.
 
 `source` и `target` могут быть такими:

git_private2public-0.2.3/docs/VERIFICATION.md

@@ -0,0 +1,19 @@
+# Verification notes
+
+These notes separate repository evidence from product intent. They are kept
+short so the README can lead with the user outcome.
+
+## Checked in
+
+- `tests/test_git_private2public.py` covers folder/YAML configuration, ignore
+  and replace rule translation, allowlisted domains, version consistency, and
+  guard detection of provider-style tokens.
+- `pyproject.toml` preserves the supported install paths shown in the README:
+  PyPI installation and local `python3 -m pip install .`.
+
+## Deliberate limits
+
+`publish` rewrites a temporary clone and pushes only after its configured scan;
+it is not a claim that the source repository or its prior public copies are
+secret-free. The guard catches its documented patterns and custom rules, but
+cannot identify every possible credential or undo a credential exposure.

{git_private2public-0.2.2 → git_private2public-0.2.3}/example.yaml

@@ -23,3 +23,5 @@ ignore:          # these won't be in the public repo
 push:
   force: true
   branches: [main]
+  # Keep existing public release tags unchanged; private source tags are not pushed.
+  preserve_target_tags: false

{git_private2public-0.2.2 → git_private2public-0.2.3}/git_private2public.py

@@ -35,7 +35,7 @@ from pathlib import Path
 import time
 from typing import Iterable
 
-__version__ = "0.2.2"
+__version__ = "0.2.3"
 
 try:
     import yaml
@@ -84,6 +84,14 @@ DEFAULT_SECRET_PATTERNS: list[str] = [
 # Config
 # --------------------------------------------------------------------------- #
 
+def _yaml_bool(value: object, name: str, default: bool) -> bool:
+    if value is None:
+        return default
+    if type(value) is not bool:
+        raise SystemExit(f"{name} must be a YAML boolean")
+    return value
+
+
 @dataclass
 class Config:
     source: str
@@ -92,16 +100,29 @@ class Config:
     replace: list[str] = field(default_factory=list)
     allow_domains: list[str] = field(default_factory=list)
     fail_on_match: list[str] = field(default_factory=list)
+    scan_ignore: list[str] = field(default_factory=list)
     push_force: bool = True
     push_branches: list[str] = field(default_factory=lambda: ["main"])
     push_tags: bool = False
+    preserve_target_tags: bool = False
     mode: str = "history"
     snapshot_include_source_sha: bool = False
 
     @classmethod
     def from_yaml(cls, path: Path) -> "Config":
         data = yaml.safe_load(path.read_text())
+        if not isinstance(data, dict):
+            raise SystemExit("YAML config must be a mapping")
         push = data.get("push") or {}
+        if not isinstance(push, dict):
+            raise SystemExit("push must be a mapping")
+        branches = push.get("branches")
+        if branches is None or branches == []:
+            branches = ["main"]
+        elif isinstance(branches, str):
+            branches = [branches]
+        elif not isinstance(branches, list):
+            raise SystemExit("push.branches must be a branch name or list of branch names")
         return cls(
             source=data["source"],
             target=data["target"],
@@ -109,11 +130,19 @@ class Config:
             replace=list(data.get("replace") or []),
             allow_domains=list(data.get("allow_domains") or data.get("allow") or data.get("domains") or []),
             fail_on_match=list(data.get("fail_on_match") or []),
-            push_force=push.get("force", True),
-            push_branches=list(push.get("branches") or ["main"]),
-            push_tags=push.get("tags", False),
+            scan_ignore=list(data.get("scan_ignore") or data.get("scan-ignore") or []),
+            push_force=_yaml_bool(push.get("force"), "push.force", True),
+            push_branches=branches,
+            push_tags=_yaml_bool(push.get("tags"), "push.tags", False),
+            preserve_target_tags=_yaml_bool(
+                push.get("preserve_target_tags", data.get("preserve_target_tags")),
+                "push.preserve_target_tags",
+                False,
+            ),
             mode=str(data.get("mode") or "history"),
-            snapshot_include_source_sha=bool(data.get("snapshot_include_source_sha", False)),
+            snapshot_include_source_sha=_yaml_bool(
+                data.get("snapshot_include_source_sha"), "snapshot_include_source_sha", False
+            ),
         )
 
     @classmethod
@@ -121,7 +150,7 @@ class Config:
         """Load from a .gitpublic/ folder — each file is one concern, gitignore-style.
 
         Files (all optional — if missing, that setting is empty):
-          config      — source=, target=, push_force=, push_branches= (key=value)
+          config      — source=, target=, push_force=, push_branches=, preserve_target_tags= (key=value)
           ignore      — one path/glob per line (# for comments)
           replace     — old==>new per line (regex: prefix supported, glob:*.ext: scoped)
           scan        — one regex/literal per line (fail_on_match)
@@ -145,6 +174,7 @@ class Config:
         push_force = True
         push_branches = ["main"]
         push_tags = False
+        preserve_target_tags = False
         mode = "history"
         snapshot_include_source_sha = False
         cfg_file = folder / "config"
@@ -165,6 +195,10 @@ class Config:
                     push_branches = [b.strip() for b in v.split(",") if b.strip()]
                 elif k == "push_tags":
                     push_tags = v.lower() in ("true", "yes", "1")
+                elif k == "preserve_target_tags":
+                    if v.lower() not in ("true", "false", "yes", "no", "1", "0"):
+                        raise SystemExit("preserve_target_tags must be true or false")
+                    preserve_target_tags = v.lower() in ("true", "yes", "1")
                 elif k == "mode":
                     mode = v
                 elif k == "snapshot_include_source_sha":
@@ -177,9 +211,11 @@ class Config:
             replace=read_lines("replace"),
             allow_domains=read_lines("allow") + read_lines("domains"),
             fail_on_match=read_lines("scan"),
+            scan_ignore=read_lines("scan_ignore"),
             push_force=push_force,
             push_branches=push_branches,
             push_tags=push_tags,
+            preserve_target_tags=preserve_target_tags,
             mode=mode,
             snapshot_include_source_sha=snapshot_include_source_sha,
         )
@@ -293,7 +329,15 @@ def expand_github_shorthand(value: str) -> str:
 
 
 def mask_url(url: str) -> str:
-    return re.sub(r"https://[^/@]+@", "https://***@", url)
+    return re.sub(r"(?i)(https?|ssh)://[^/\s@]+@", r"\1://***@", url)
+
+
+def redact_target_output(output: str, target_url: str, token: str | None = None) -> str:
+    """Remove URL credentials and explicit publisher tokens from Git diagnostics."""
+    safe = output.replace(target_url, mask_url(target_url))
+    if token:
+        safe = safe.replace(token, "***")
+    return re.sub(r"(?i)(https?|ssh)://[^/\s@]+@", r"\1://***@", safe)
 
 
 def validate_config(config: Config, cwd: str | None = None) -> None:
@@ -333,8 +377,21 @@ def validate_config(config: Config, cwd: str | None = None) -> None:
         raise SystemExit(f"missing required config value(s): {', '.join(missing)}")
     if config.mode not in {"history", "snapshot"}:
         raise SystemExit("mode must be 'history' or 'snapshot'")
+    for name, value in (
+        ("push_force", config.push_force),
+        ("push_tags", config.push_tags),
+        ("preserve_target_tags", config.preserve_target_tags),
+        ("snapshot_include_source_sha", config.snapshot_include_source_sha),
+    ):
+        if type(value) is not bool:
+            raise SystemExit(f"{name} must be a boolean")
+    if (not isinstance(config.push_branches, list)
+            or any(not isinstance(branch, str) or not branch for branch in config.push_branches)):
+        raise SystemExit("push_branches must be a list of non-empty branch names")
     if config.mode == "snapshot" and config.push_tags:
         raise SystemExit("snapshot mode cannot publish tags")
+    if config.push_tags and config.preserve_target_tags:
+        raise SystemExit("push_tags and preserve_target_tags cannot be enabled together")
 
 
 def clone_source(source: str, dest: Path) -> None:
@@ -515,6 +572,18 @@ def _allowed_match(matched: bytes, allow_domains: list[str]) -> bool:
 # Scanning
 # --------------------------------------------------------------------------- #
 
+def _scan_ignored(fpath: str, patterns: list[str]) -> bool:
+    """Return True when a tracked path is exempt from secret scanning."""
+    posix = fpath.replace("\\", "/")
+    for raw in patterns:
+        pat = raw.strip()
+        if not pat:
+            continue
+        if fnmatch.fnmatch(posix, pat) or fnmatch.fnmatch(posix, pat.rstrip("/") + "/*"):
+            return True
+    return False
+
+
 def scan_tree(repo: Path, config: Config) -> list[str]:
     """Return list of violations (pattern + file:line) in the current tree."""
     violations: list[str] = []
@@ -535,6 +604,8 @@ def scan_tree(repo: Path, config: Config) -> list[str]:
         full = repo / fpath
         if not full.is_file():
             continue
+        if _scan_ignored(fpath, config.scan_ignore):
+            continue
         try:
             data = full.read_bytes()
         except Exception:
@@ -776,13 +847,35 @@ def _remote_refs(url: str) -> dict[str, str]:
     return refs
 
 
-def _verify_target_ref_names(url: str, allowed_refs: set[str]) -> None:
-    """Block publication when the target retains any unconfigured ref."""
-    unexpected = sorted(set(_remote_refs(url)) - allowed_refs)
+def _verify_target_ref_names(
+    url: str,
+    allowed_refs: set[str],
+    *,
+    preserve_target_tags: bool = False,
+) -> dict[str, str]:
+    """Block unconfigured refs and return existing tags explicitly preserved by policy."""
+    target_refs = _remote_refs(url)
+    preserved_tags = {
+        ref: sha for ref, sha in target_refs.items()
+        if preserve_target_tags and ref.startswith("refs/tags/")
+    }
+    unexpected = sorted(set(target_refs) - allowed_refs - set(preserved_tags))
     if unexpected:
         preview = ", ".join(unexpected[:10])
         suffix = f" and {len(unexpected) - 10} more" if len(unexpected) > 10 else ""
         raise SystemExit(f"unexpected target refs block publication: {preview}{suffix}")
+    return preserved_tags
+
+
+def _verify_target_tags_unchanged(
+    expected: dict[str, str],
+    actual: dict[str, str],
+) -> None:
+    """Fail before publishing if an existing public tag changed during sanitization."""
+    if actual != expected:
+        raise SystemExit(
+            "public target tags changed during sanitization; nothing was pushed"
+        )
 
 
 def _verify_target_refs_exact(url: str, expected_refs: dict[str, str]) -> None:
@@ -931,10 +1024,15 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
 
         # Capture the exact target state before any expensive rewriting. A later
         # push is allowed only if the branch still has this SHA (or is still absent).
+        initial_target_refs = _remote_refs(target_url)
         expected_target_refs = {
-            branch: _remote_ref_sha(target_url, f"refs/heads/{branch}")
+            branch: initial_target_refs.get(f"refs/heads/{branch}", "")
             for branch in config.push_branches
         }
+        expected_target_tags = {
+            ref: sha for ref, sha in initial_target_refs.items()
+            if config.preserve_target_tags and ref.startswith("refs/tags/")
+        }
 
         # Backup source gitdir before any destructive rewrite
         backup_info = _backup_gitdir(source_url, cwd)
@@ -989,6 +1087,18 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
                 config.snapshot_include_source_sha,
                 source_sha=original_source_sha,
             )
+            # The snapshot intentionally destroys all other refs, but their
+            # objects stay in the clone until pruned; a fail-closed history
+            # scan would still flag them as reachable garbage.
+            run(["git", "reflog", "expire", "--expire=now", "--all"], cwd=str(work))
+            try:
+                subprocess.run(
+                    ["git", "gc", "--prune=now", "--quiet"],
+                    cwd=str(work), check=True, capture_output=True, text=True,
+                    timeout=600,
+                )
+            except subprocess.TimeoutExpired:
+                raise ScanError("snapshot gc timed out; refusing to publish")
         elif overlay_files:
             commit_public_overlay(work)
 
@@ -999,7 +1109,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
         try:
             violations = scan_tree(work, scan_config)
             violations.extend(
-                scan_history(work, scan_config.fail_on_match, scan_config.allow_domains)
+                scan_history(work, scan_config.fail_on_match, scan_config.allow_domains, scan_ignore=scan_config.scan_ignore)
             )
         except ScanError as exc:
             print(f"✗ Security scan incomplete: {exc}", file=sys.stderr)
@@ -1022,12 +1132,23 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
 
         local_head = run(["git", "rev-parse", "HEAD"], cwd=str(work)).stdout.strip()
         expected_public_refs = _local_publish_refs(work, config, local_head)
-        _verify_target_ref_names(target_url, set(expected_public_refs))
+        preserved_target_tags = _verify_target_ref_names(
+            target_url,
+            set(expected_public_refs),
+            preserve_target_tags=config.preserve_target_tags,
+        )
+        _verify_target_tags_unchanged(expected_target_tags, preserved_target_tags)
+        expected_public_refs.update(preserved_target_tags)
 
         # Push to target
         print(f"▸ Pushing to {mask_url(target_url)} ...", file=sys.stderr)
         run(["git", "remote", "add", "target", target_url], cwd=str(work))
 
+        if config.preserve_target_tags:
+            print(
+                f"▸ Publishing configured branches; leaving {len(preserved_target_tags)} existing public tag(s) untouched ...",
+                file=sys.stderr,
+            )
         for branch in config.push_branches:
             push_cmd = ["git", "push"]
             if config.push_force:
@@ -1037,7 +1158,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
             push_cmd.extend(["target", f"HEAD:refs/heads/{branch}"])
             res = subprocess.run(push_cmd, cwd=str(work), capture_output=True, text=True)
             if res.returncode != 0:
-                sys.stderr.write(res.stderr)
+                sys.stderr.write(redact_target_output(res.stderr, target_url, token))
                 sys.exit(
                     f"push of {branch} failed; target probably changed while sanitizing. "
                     "Nothing was overwritten. Run publish again."
@@ -1052,7 +1173,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
                 cwd=str(work), capture_output=True, text=True
             )
             if res.returncode != 0:
-                sys.stderr.write(res.stderr)
+                sys.stderr.write(redact_target_output(res.stderr, target_url, token))
                 sys.exit("tag push failed; existing target tags were left unchanged")
 
         _verify_target_refs_exact(target_url, expected_public_refs)
@@ -1069,7 +1190,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
             try:
                 verify_violations = scan_tree(verify_repo, verify_config)
                 verify_violations.extend(
-                    scan_history(verify_repo, verify_config.fail_on_match, verify_config.allow_domains)
+                    scan_history(verify_repo, verify_config.fail_on_match, verify_config.allow_domains, scan_ignore=verify_config.scan_ignore)
                 )
             except ScanError as exc:
                 raise SystemExit(f"post-push target scan incomplete: {exc}") from exc
@@ -1339,7 +1460,13 @@ def _reachable_scan_objects(repo: Path, refs: list[str] | None) -> dict[str, tup
         return {}
 
     reachable = _run_scan_command(["git", "rev-list", "--objects", *refs], repo)
-    object_ids = sorted({line.split(maxsplit=1)[0] for line in reachable.stdout.decode().splitlines() if line})
+    object_ids: set[str] = set()
+    for line in reachable.stdout.decode().splitlines():
+        if not line:
+            continue
+        object_id = line.split(maxsplit=1)[0]
+        object_ids.add(object_id)
+    object_ids = sorted(object_ids)
     if not object_ids:
         return {}
     probe = _run_scan_command(
@@ -1379,8 +1506,15 @@ def scan_history(
     rules: list[str],
     allow_domains: list[str],
     refs: list[str] | None = None,
+    scan_ignore: list[str] | None = None,
 ) -> list[str]:
-    """Scan every blob reachable from selected refs, failing closed on gaps."""
+    """Scan every blob reachable from selected refs, failing closed on gaps.
+
+    ``scan_ignore`` is accepted for API compatibility but never exempts history
+    objects: one blob can be reachable through both ignored and non-ignored
+    paths, while ``rev-list --objects`` reports only one display path per blob.
+    Path exemptions apply to the current tree scan only.
+    """
     if not rules:
         return []
     compiled = _compile_rules(rules)

{git_private2public-0.2.2 → git_private2public-0.2.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "git-private2public"
-version = "0.2.2"
+version = "0.2.3"
 description = "Like .gitignore, but for what goes public. Keep a sanitized public mirror of your private repo."
 readme = "README.md"
 license = {text = "MIT"}

{git_private2public-0.2.2 → git_private2public-0.2.3}/tests/test_git_private2public.py

@@ -11,7 +11,10 @@ import git_private2public as g
 def test_folder_config_loads_allow_and_domains_alias(tmp_path: Path):
     cfgdir = tmp_path / ".gitpublic"
     cfgdir.mkdir()
-    (cfgdir / "config").write_text("source = owner/private\ntarget = owner/public\npush_force = false\npush_branches = main,dev\n")
+    (cfgdir / "config").write_text(
+        "source = owner/private\ntarget = owner/public\npush_force = false\n"
+        "push_branches = main,dev\npreserve_target_tags = true\n"
+    )
     (cfgdir / "ignore").write_text("# comment\n.env\n*.key\n")
     (cfgdir / "allow").write_text("github.com\n")
     (cfgdir / "domains").write_text("example.com\n")
@@ -24,6 +27,7 @@ def test_folder_config_loads_allow_and_domains_alias(tmp_path: Path):
     assert cfg.allow_domains == ["github.com", "example.com"]
     assert cfg.push_force is False
     assert cfg.push_branches == ["main", "dev"]
+    assert cfg.preserve_target_tags is True
 
 
 def test_folder_config_loads_snapshot_mode(tmp_path: Path):
@@ -40,6 +44,15 @@ def test_folder_config_loads_snapshot_mode(tmp_path: Path):
     assert cfg.snapshot_include_source_sha is True
 
 
+def test_folder_config_loads_scan_ignore(tmp_path: Path):
+    cfgdir = tmp_path / ".gitpublic"
+    cfgdir.mkdir()
+    (cfgdir / "config").write_text("source = owner/private\ntarget = owner/public\n")
+    (cfgdir / "scan_ignore").write_text("# fixture globs\ntests/fixtures/**\n*.snapshot\n")
+
+    assert g.Config.load(cfgdir).scan_ignore == ["tests/fixtures/**", "*.snapshot"]
+
+
 def test_yaml_config_accepts_allow_alias(tmp_path: Path):
     path = tmp_path / "rules.yaml"
     path.write_text(
@@ -55,6 +68,63 @@ def test_yaml_config_accepts_allow_alias(tmp_path: Path):
     assert cfg.allow_domains == ["github.com"]
 
 
+def test_yaml_config_loads_scan_ignore(tmp_path: Path):
+    path = tmp_path / "rules.yaml"
+    path.write_text(
+        "source: owner/private\n"
+        "target: owner/public\n"
+        "scan_ignore: ['tests/fixtures/**']\n"
+    )
+
+    assert g.Config.load(path).scan_ignore == ["tests/fixtures/**"]
+
+
+def test_yaml_config_loads_preserve_target_tags(tmp_path: Path):
+    path = tmp_path / "rules.yaml"
+    path.write_text(
+        "source: owner/private\n"
+        "target: owner/public\n"
+        "mode: snapshot\n"
+        "push:\n"
+        "  preserve_target_tags: true\n"
+    )
+
+    assert g.Config.load(path).preserve_target_tags is True
+
+
+@pytest.mark.parametrize(
+    ("option", "expected_error"),
+    [
+        ('push:\n  preserve_target_tags: "false"\n', "push.preserve_target_tags"),
+        ('push:\n  force: "false"\n', "push.force"),
+        ('push:\n  tags: "false"\n', "push.tags"),
+        ('snapshot_include_source_sha: "false"\n', "snapshot_include_source_sha"),
+    ],
+)
+def test_yaml_config_rejects_quoted_boolean_options(tmp_path: Path, option: str, expected_error: str):
+    path = tmp_path / "rules.yaml"
+    path.write_text(
+        "source: owner/private\n"
+        "target: owner/public\n"
+        + option
+    )
+
+    with pytest.raises(SystemExit, match=f"{expected_error} must be a YAML boolean"):
+        g.Config.load(path)
+
+
+def test_yaml_config_treats_scalar_branch_as_one_branch_name(tmp_path: Path):
+    path = tmp_path / "rules.yaml"
+    path.write_text(
+        "source: owner/private\n"
+        "target: owner/public\n"
+        "push:\n"
+        "  branches: main\n"
+    )
+
+    assert g.Config.load(path).push_branches == ["main"]
+
+
 def test_delete_rules_map_to_filter_repo_args(tmp_path: Path):
     replace_file = tmp_path / "replace.txt"
     replace_file.write_text("literal==>***\n")
@@ -115,8 +185,8 @@ def test_replace_rule_strips_separator_whitespace():
 
 
 def test_version_constant_matches_package_version():
-    assert g.__version__ == "0.2.2"
-    assert 'version = "0.2.2"' in (Path(__file__).parent.parent / "pyproject.toml").read_text()
+    assert g.__version__ == "0.2.3"
+    assert 'version = "0.2.3"' in (Path(__file__).parent.parent / "pyproject.toml").read_text()
 
 
 def test_gitpublic_secret_check_finds_only_replace_and_scan(tmp_path: Path):
@@ -478,6 +548,33 @@ def test_remote_ref_sha_timeout_fails_closed_without_url_leak(monkeypatch):
     assert "super-secret-token" not in message
 
 
+@pytest.mark.parametrize(
+    ("url", "diagnostic", "secret"),
+    [
+        (
+            "https://x-access-token:private-publish-token@github.com/owner/public.git",
+            "fatal: repository 'https://x-access-token:private-publish-token@github.com/owner/public.git/' unavailable",
+            "private-publish-token",
+        ),
+        (
+            "http://mirror-user:mirror-password@mirror.example/private.git/",
+            "fatal: repository 'http://mirror-user:mirror-password@mirror.example/private.git' unavailable",
+            "mirror-password",
+        ),
+        (
+            "ssh://mirror-user:mirror-password@mirror.example/private.git",
+            "fatal: repository 'ssh://mirror-user:mirror-password@mirror.example/private.git' unavailable",
+            "mirror-password",
+        ),
+    ],
+)
+def test_target_push_error_redacts_url_credentials_and_token(url: str, diagnostic: str, secret: str):
+    safe = g.redact_target_output(diagnostic, url, secret)
+
+    assert secret not in safe
+    assert "***@" in safe
+
+
 @pytest.mark.parametrize(
     "unexpected_ref",
     ["refs/heads/stale", "refs/tags/stale", "refs/notes/private"],
@@ -500,6 +597,78 @@ def test_target_ref_allowlist_accepts_configured_head_and_tags(monkeypatch):
     g._verify_target_refs_exact("target", expected)
 
 
+def test_target_ref_allowlist_can_preserve_existing_target_tags(monkeypatch):
+    remote = {
+        "refs/heads/main": "a" * 40,
+        "refs/tags/node-v1024": "b" * 40,
+        "refs/tags/v1021": "c" * 40,
+    }
+    monkeypatch.setattr(g, "_remote_refs", lambda url: dict(remote))
+
+    preserved = g._verify_target_ref_names(
+        "target", {"refs/heads/main"}, preserve_target_tags=True
+    )
+
+    assert preserved == {
+        "refs/tags/node-v1024": "b" * 40,
+        "refs/tags/v1021": "c" * 40,
+    }
+
+
+def test_preserve_target_tags_does_not_allow_unconfigured_branches(monkeypatch):
+    monkeypatch.setattr(
+        g,
+        "_remote_refs",
+        lambda url: {"refs/heads/stale": "a" * 40, "refs/tags/v1": "b" * 40},
+    )
+
+    with pytest.raises(SystemExit, match="unexpected target refs"):
+        g._verify_target_ref_names(
+            "target", {"refs/heads/main"}, preserve_target_tags=True
+        )
+
+
+def test_preserved_target_tag_change_fails_post_publish_verification(monkeypatch):
+    expected = {
+        "refs/heads/main": "a" * 40,
+        "refs/tags/v1": "b" * 40,
+    }
+    monkeypatch.setattr(
+        g,
+        "_remote_refs",
+        lambda url: {"refs/heads/main": "a" * 40, "refs/tags/v1": "c" * 40},
+    )
+
+    with pytest.raises(SystemExit, match="post-push target refs differ.*changed=refs/tags/v1"):
+        g._verify_target_refs_exact("target", expected)
+
+
+@pytest.mark.parametrize(
+    "actual",
+    [
+        {"refs/tags/v1": "c" * 40},
+        {"refs/tags/v1": "b" * 40, "refs/tags/v2": "d" * 40},
+        {},
+    ],
+    ids=["changed", "added", "removed"],
+)
+def test_target_tag_snapshot_change_aborts_before_publication(actual):
+    with pytest.raises(SystemExit, match="public target tags changed during sanitization"):
+        g._verify_target_tags_unchanged({"refs/tags/v1": "b" * 40}, actual)
+
+
+def test_preserve_target_tags_cannot_be_combined_with_pushing_source_tags():
+    cfg = g.Config(
+        source="owner/private",
+        target="owner/public",
+        push_tags=True,
+        preserve_target_tags=True,
+    )
+
+    with pytest.raises(SystemExit, match="push_tags and preserve_target_tags"):
+        g.validate_config(cfg)
+
+
 def test_post_push_ref_verification_blocks_missing_or_changed_refs(monkeypatch):
     expected = {"refs/heads/main": "a" * 40, "refs/tags/v1.0.0": "b" * 40}
     monkeypatch.setattr(g, "_remote_refs", lambda url: {"refs/heads/main": "c" * 40})
@@ -525,6 +694,50 @@ def test_remote_refs_ignores_symbolic_head_and_peeled_tag(monkeypatch):
     assert g._remote_refs("target") == {"refs/tags/v1": tag}
 
 
+def test_reachable_scan_objects_returns_empty_mapping_for_unborn_repository(tmp_path: Path):
+    repo = tmp_path / "unborn"
+    subprocess.run(["git", "init", "-q", str(repo)], check=True)
+
+    assert g._reachable_scan_objects(repo, None) == {}
+
+
+def test_history_scan_never_uses_scan_ignore_to_skip_an_aliased_blob(tmp_path: Path):
+    repo = tmp_path / "private"
+    repo.mkdir()
+    subprocess.run(["git", "init", "-q", str(repo)], check=True)
+    subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
+    subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
+    fake_token = "ghp_" + "A" * 36
+    (repo / "ignored").mkdir()
+    (repo / "ignored" / "fixture.txt").write_text(fake_token + "\n")
+    (repo / "public.txt").write_text(fake_token + "\n")
+    subprocess.run(["git", "-C", str(repo), "add", "ignored/fixture.txt", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(repo), "commit", "-q", "-m", "fixture alias"], check=True)
+
+    violations = g.scan_history(
+        repo, g.DEFAULT_SECRET_PATTERNS, [], scan_ignore=["ignored/**"]
+    )
+
+    assert violations
+    assert any("<history:" in finding for finding in violations)
+
+
+def test_current_tree_scan_ignore_skips_matching_fixture_path(tmp_path: Path):
+    repo = tmp_path / "tree"
+    (repo / "tests" / "fixtures").mkdir(parents=True)
+    subprocess.run(["git", "init", "-q", str(repo)], check=True)
+    fake_token = "ghp_" + "C" * 36
+    (repo / "tests" / "fixtures" / "sample.txt").write_text(fake_token + "\n")
+    subprocess.run(["git", "-C", str(repo), "add", "tests/fixtures/sample.txt"], check=True)
+    cfg = g.Config(
+        source="owner/private", target="owner/public",
+        fail_on_match=g.DEFAULT_SECRET_PATTERNS,
+        scan_ignore=["tests/fixtures/**"],
+    )
+
+    assert g.scan_tree(repo, cfg) == []
+
+
 def test_hook_dispatcher_preserves_foreign_hook_and_combines_actions(tmp_path: Path, monkeypatch):
     repo = tmp_path / "repo"
     repo.mkdir()
@@ -824,6 +1037,191 @@ def test_snapshot_publish_creates_single_public_root_without_tags(tmp_path: Path
     assert "private current" not in log
 
 
+def test_snapshot_prunes_secret_in_removed_private_history_before_scan(tmp_path: Path):
+    source = tmp_path / "private"
+    target = tmp_path / "public.git"
+    source.mkdir()
+    subprocess.run(["git", "init", "-q", str(source)], check=True)
+    subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
+    subprocess.run(["git", "-C", str(source), "config", "user.name", "Private Person"], check=True)
+    subprocess.run(["git", "-C", str(source), "config", "user.email", "private@example.test"], check=True)
+    fake_token = "ghp_" + "B" * 36
+    (source / "old.txt").write_text(fake_token + "\n")
+    subprocess.run(["git", "-C", str(source), "add", "old.txt"], check=True)
+    subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private old"], check=True)
+    subprocess.run(["git", "-C", str(source), "rm", "-q", "old.txt"], check=True)
+    (source / "safe.txt").write_text("public content\n")
+    subprocess.run(["git", "-C", str(source), "add", "safe.txt"], check=True)
+    subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "current public source"], check=True)
+    cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True)
+
+    assert g.publish(cfg, cwd=str(source)) == 0
+
+    public_files = subprocess.run(
+        ["git", "--git-dir", str(target), "ls-tree", "-r", "--name-only", "refs/heads/main"],
+        capture_output=True, text=True, check=True,
+    ).stdout.splitlines()
+    assert public_files == ["safe.txt"]
+
+
+def test_snapshot_publish_preserves_existing_public_release_tags(tmp_path: Path):
+    source = tmp_path / "private"
+    seed = tmp_path / "seed"
+    target = tmp_path / "public.git"
+    source.mkdir()
+    seed.mkdir()
+    subprocess.run(["git", "init", "-q", str(source)], check=True)
+    subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
+    subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
+
+    for repo, name, email in (
+        (source, "Private Person", "private@example.test"),
+        (seed, "Public Release Bot", "public@example.test"),
+    ):
+        subprocess.run(["git", "-C", str(repo), "config", "user.name", name], check=True)
+        subprocess.run(["git", "-C", str(repo), "config", "user.email", email], check=True)
+
+    (seed / "public.txt").write_text("previous release\n")
+    subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "previous public snapshot"], check=True)
+    subprocess.run(["git", "-C", str(seed), "tag", "node-v1024"], check=True)
+    subprocess.run(["git", "-C", str(seed), "tag", "-a", "v1021", "-m", "release v1021"], check=True)
+    subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
+    before_tags = subprocess.run(
+        ["git", "--git-dir", str(target), "for-each-ref", "--format=%(objectname) %(refname)", "refs/tags/"],
+        capture_output=True, text=True, check=True,
+    ).stdout.splitlines()
+
+    (source / "public.txt").write_text("current sanitized source\n")
+    subprocess.run(["git", "-C", str(source), "config", "user.name", "Private Person"], check=True)
+    subprocess.run(["git", "-C", str(source), "config", "user.email", "private@example.test"], check=True)
+    subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current source"], check=True)
+    subprocess.run(["git", "-C", str(source), "tag", "private-only-tag"], check=True)
+
+    cfg = g.Config(
+        source=str(source), target=str(target), mode="snapshot",
+        push_force=True, preserve_target_tags=True,
+    )
+    assert g.publish(cfg, cwd=str(source)) == 0
+
+    after_tags = subprocess.run(
+        ["git", "--git-dir", str(target), "for-each-ref", "--format=%(objectname) %(refname)", "refs/tags/"],
+        capture_output=True, text=True, check=True,
+    ).stdout.splitlines()
+    target_refs = subprocess.run(
+        ["git", "--git-dir", str(target), "for-each-ref", "--format=%(refname)"],
+        capture_output=True, text=True, check=True,
+    ).stdout.splitlines()
+    public_log = subprocess.run(
+        ["git", "--git-dir", str(target), "log", "--format=%an%n%s", "refs/heads/main"],
+        capture_output=True, text=True, check=True,
+    ).stdout
+
+    assert after_tags == before_tags
+    assert "refs/tags/private-only-tag" not in target_refs
+    assert target_refs == ["refs/heads/main", "refs/tags/node-v1024", "refs/tags/v1021"]
+    assert "Private Person" not in public_log
+    assert "private current source" not in public_log
+    assert subprocess.run(
+        ["git", "--git-dir", str(target), "rev-list", "--count", "refs/heads/main"],
+        capture_output=True, text=True, check=True,
+    ).stdout.strip() == "1"
+
+
+def test_snapshot_publish_aborts_before_branch_push_if_target_tag_changes(tmp_path: Path, monkeypatch):
+    source = tmp_path / "private"
+    seed = tmp_path / "seed"
+    target = tmp_path / "public.git"
+    source.mkdir()
+    seed.mkdir()
+    subprocess.run(["git", "init", "-q", str(source)], check=True)
+    subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
+    subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
+    for repo in (source, seed):
+        subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
+        subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
+    (seed / "public.txt").write_text("first\n")
+    subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "first"], check=True)
+    first = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    subprocess.run(["git", "-C", str(seed), "tag", "v1"], check=True)
+    (seed / "public.txt").write_text("second\n")
+    subprocess.run(["git", "-C", str(seed), "commit", "-qam", "second"], check=True)
+    second = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
+    original_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
+    (source / "public.txt").write_text("sanitized current\n")
+    subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current"], check=True)
+
+    verify = g._verify_target_ref_names
+    changed = False
+
+    def change_target_tag(url, allowed_refs, *, preserve_target_tags=False):
+        nonlocal changed
+        if not changed:
+            subprocess.run(
+                ["git", "--git-dir", str(target), "update-ref", "refs/tags/v1", second],
+                check=True,
+            )
+            changed = True
+        return verify(url, allowed_refs, preserve_target_tags=preserve_target_tags)
+
+    monkeypatch.setattr(g, "_verify_target_ref_names", change_target_tag)
+    cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True, preserve_target_tags=True)
+
+    with pytest.raises(SystemExit, match="public target tags changed during sanitization"):
+        g.publish(cfg, cwd=str(source))
+
+    assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip() == original_main
+    assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/tags/v1"], capture_output=True, text=True, check=True).stdout.strip() == second
+    assert first != second
+
+
+def test_target_tag_change_after_preflight_is_reported_after_branch_push(tmp_path: Path):
+    source = tmp_path / "private"
+    seed = tmp_path / "seed"
+    target = tmp_path / "public.git"
+    source.mkdir()
+    seed.mkdir()
+    subprocess.run(["git", "init", "-q", str(source)], check=True)
+    subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
+    subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
+    for repo in (source, seed):
+        subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
+        subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
+    (seed / "public.txt").write_text("first\n")
+    subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "first"], check=True)
+    first = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    subprocess.run(["git", "-C", str(seed), "tag", "v1"], check=True)
+    (seed / "public.txt").write_text("second\n")
+    subprocess.run(["git", "-C", str(seed), "commit", "-qam", "second"], check=True)
+    second = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
+    original_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
+    (source / "public.txt").write_text("sanitized current\n")
+    subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
+    subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current"], check=True)
+
+    hook = target / "hooks" / "pre-receive"
+    hook.write_text(
+        f'#!/bin/sh\nunset GIT_QUARANTINE_PATH\n'
+        f'git --git-dir="$GIT_DIR" update-ref refs/tags/v1 {second}\n'
+    )
+    hook.chmod(0o755)
+    cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True, preserve_target_tags=True)
+
+    with pytest.raises(SystemExit, match="post-push target refs differ.*changed=refs/tags/v1"):
+        g.publish(cfg, cwd=str(source))
+
+    published_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
+    assert published_main != original_main
+    assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/tags/v1"], capture_output=True, text=True, check=True).stdout.strip() == second
+    assert first != second
+
+
 def test_publish_blocks_stale_target_ref_before_writing_configured_head(tmp_path: Path):
     source = tmp_path / "private"
     target = tmp_path / "public.git"