git-private2public 0.2.1__tar.gz → 0.2.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {git_private2public-0.2.1 → git_private2public-0.2.3}/CHANGELOG.md +12 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/PKG-INFO +2 -2
- git_private2public-0.2.3/README.zh.md +46 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/docs/ADVANCED.md +19 -4
- {git_private2public-0.2.1 → git_private2public-0.2.3}/docs/ADVANCED.ru.md +18 -4
- git_private2public-0.2.3/docs/VERIFICATION.md +19 -0
- git_private2public-0.2.3/docs/assets/readme-hero.png +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/example.yaml +2 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/git_private2public.py +161 -25
- {git_private2public-0.2.1 → git_private2public-0.2.3}/pyproject.toml +1 -1
- {git_private2public-0.2.1 → git_private2public-0.2.3}/tests/test_git_private2public.py +417 -3
- {git_private2public-0.2.1 → git_private2public-0.2.3}/.github/workflows/ci.yml +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/.github/workflows/release.yml +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/.gitignore +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/LICENSE +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/README.md +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/README.ru.md +0 -0
- {git_private2public-0.2.1 → git_private2public-0.2.3}/templates/publish.yml +0 -0
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.3
|
|
4
|
+
|
|
5
|
+
- Preserve existing public target tags unchanged during snapshot publication when explicitly enabled.
|
|
6
|
+
- Continue rejecting unconfigured refs and verify preserved tag object IDs after the branch push.
|
|
7
|
+
- Add current-tree scan path exemptions and prune unreachable pre-snapshot objects before the history scan.
|
|
8
|
+
- Keep history scanning complete because one blob can be reachable through both exempt and non-exempt paths.
|
|
9
|
+
|
|
10
|
+
## 0.2.2
|
|
11
|
+
|
|
12
|
+
- Fail closed with a concise redacted error when target-ref preflight times out.
|
|
13
|
+
- Prevent Git remote errors from echoing credential-bearing target URLs.
|
|
14
|
+
|
|
3
15
|
## 0.2.1
|
|
4
16
|
|
|
5
17
|
- Add `.gitpublic/public/` overlays for public-only files after sanitization.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
2
|
Name: git-private2public
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.3
|
|
4
4
|
Summary: Like .gitignore, but for what goes public. Keep a sanitized public mirror of your private repo.
|
|
5
5
|
Project-URL: Homepage, https://github.com/megamen32/git-private2public
|
|
6
6
|
Project-URL: Repository, https://github.com/megamen32/git-private2public
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# git-private2public
|
|
2
|
+
|
|
3
|
+
[](https://pypi.org/project/git-private2public/) [](https://github.com/megamen32/git-private2public/actions/workflows/ci.yml)
|
|
4
|
+
|
|
5
|
+
**[English](./README.md)** · **[Русский](./README.ru.md)** · **简体中文**
|
|
6
|
+
|
|
7
|
+

|
|
8
|
+
|
|
9
|
+
**像 `.gitignore` 一样,但控制哪些内容可以公开。**
|
|
10
|
+
|
|
11
|
+
## 快速开始
|
|
12
|
+
|
|
13
|
+
项目需要 Python 3.9+。使用一条命令安装 CLI 及其 Git 清理引擎:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
python3 -m pip install git-private2public
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
在要同步的私有仓库中初始化配置:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
git-private2public init
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
编辑 `.gitpublic/config` 设置 `source` 与 `target`,再编辑 `.gitpublic/ignore` 添加不应公开的文件:
|
|
26
|
+
|
|
27
|
+
```text
|
|
28
|
+
.env
|
|
29
|
+
secrets/
|
|
30
|
+
*.key
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## 安全发布
|
|
34
|
+
|
|
35
|
+
先执行 dry-run 查看计划,再发布:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
git-private2public publish --dry-run
|
|
39
|
+
git-private2public publish
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
工具基于 `git-filter-repo` 和 YAML 规则创建干净的公开镜像。完整规则见 [高级配置](./docs/ADVANCED.md)。
|
|
43
|
+
|
|
44
|
+
## 许可证
|
|
45
|
+
|
|
46
|
+
MIT
|
|
@@ -24,10 +24,16 @@ Missing custom files mean “no custom rules for that step”. Built-in credenti
|
|
|
24
24
|
| `.gitpublic/ignore` | Removes matching files/paths from all published history | No files are removed |
|
|
25
25
|
| `.gitpublic/replace` | Rewrites matching text in remaining files | No text is rewritten |
|
|
26
26
|
| `.gitpublic/scan` | Adds custom blocking patterns | Built-in credential rules still protect zero-config scans and guard mode |
|
|
27
|
+
| `.gitpublic/scan_ignore` | Skips matching paths in the current sanitized tree scan | Every current-tree file is scanned |
|
|
27
28
|
| `.gitpublic/allow` | Exceptions for domain-like matches found by `scan` | No scan exceptions exist |
|
|
28
29
|
| `.gitpublic/domains` | Same as `allow` alias | Nothing is allowlisted |
|
|
29
30
|
| `.gitpublic/public/` | Adds or replaces files in the sanitized public tree | No public-only overlay is applied |
|
|
30
31
|
|
|
32
|
+
`scan_ignore` does not exempt history blobs. Git can share one blob across
|
|
33
|
+
multiple paths, so a displayed history path cannot safely identify every use
|
|
34
|
+
of that content. Snapshot mode prunes unreachable private history before the
|
|
35
|
+
history scan; history mode still checks all reachable content.
|
|
36
|
+
|
|
31
37
|
## `.gitpublic/config`
|
|
32
38
|
|
|
33
39
|
Example:
|
|
@@ -38,11 +44,19 @@ target = you/public-repo
|
|
|
38
44
|
push_force = true
|
|
39
45
|
push_branches = main
|
|
40
46
|
push_tags = false
|
|
47
|
+
preserve_target_tags = true
|
|
41
48
|
```
|
|
42
49
|
|
|
43
|
-
For a
|
|
44
|
-
|
|
45
|
-
|
|
50
|
+
For a public snapshot, `snapshot` mode publishes the sanitized current tracked
|
|
51
|
+
tree as one neutral root commit. It removes private commit messages, authors,
|
|
52
|
+
branches, tags, and previous history before filtering. Set
|
|
53
|
+
`preserve_target_tags = true` to keep tags already on the public target at their
|
|
54
|
+
exact existing object IDs; private source tags are still never published. Other
|
|
55
|
+
unconfigured refs remain a hard error, and the publisher verifies every retained
|
|
56
|
+
tag after updating the configured branches. Existing tags are not sent in the
|
|
57
|
+
push. A concurrent remote ref change after preflight is detected by the final
|
|
58
|
+
check and can be reported after the branch update. Reconcile the target before
|
|
59
|
+
retrying:
|
|
46
60
|
|
|
47
61
|
```ini
|
|
48
62
|
mode = snapshot
|
|
@@ -50,7 +64,8 @@ snapshot_include_source_sha = false
|
|
|
50
64
|
```
|
|
51
65
|
|
|
52
66
|
Set `snapshot_include_source_sha = true` only when intentionally exposing the
|
|
53
|
-
private source commit ID. Snapshot mode rejects `push_tags = true
|
|
67
|
+
private source commit ID. Snapshot mode rejects `push_tags = true`; the source
|
|
68
|
+
tag publisher and existing-target-tag preservation cannot be combined. Normal
|
|
54
69
|
`history` mode remains the default.
|
|
55
70
|
|
|
56
71
|
`source` and `target` can be:
|
|
@@ -24,10 +24,16 @@
|
|
|
24
24
|
| `.gitpublic/ignore` | удаляет совпавшие файлы/пути из публичной истории | файлы не удаляются |
|
|
25
25
|
| `.gitpublic/replace` | заменяет совпавший текст в оставшихся файлах | текст не заменяется |
|
|
26
26
|
| `.gitpublic/scan` | добавляет пользовательские блокирующие правила | встроенные правила продолжают работать в zero-config и guard-режимах |
|
|
27
|
+
| `.gitpublic/scan_ignore` | пропускает совпавшие пути при проверке текущего очищенного дерева | проверяются все файлы текущего дерева |
|
|
27
28
|
| `.gitpublic/allow` | исключения для доменов, найденных проверкой `scan` | исключений для проверки нет |
|
|
28
29
|
| `.gitpublic/domains` | alias для `allow` | ничего не разрешается |
|
|
29
30
|
| `.gitpublic/public/` | добавляет или заменяет файлы в очищенном public-дереве | public-only overlay не применяется |
|
|
30
31
|
|
|
32
|
+
`scan_ignore` не исключает содержимое истории. Один Git blob может быть доступен
|
|
33
|
+
по нескольким путям, поэтому путь, показанный для объекта истории, не доказывает,
|
|
34
|
+
что этот текст используется только в исключённом файле. Snapshot удаляет старую
|
|
35
|
+
недостижимую историю до проверки; режим `history` проверяет все достижимые данные.
|
|
36
|
+
|
|
31
37
|
## `.gitpublic/config`
|
|
32
38
|
|
|
33
39
|
Пример:
|
|
@@ -38,11 +44,18 @@ target = you/public-repo
|
|
|
38
44
|
push_force = true
|
|
39
45
|
push_branches = main
|
|
40
46
|
push_tags = false
|
|
47
|
+
preserve_target_tags = true
|
|
41
48
|
```
|
|
42
49
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
50
|
+
Режим `snapshot` публикует очищенное текущее дерево tracked-файлов одним
|
|
51
|
+
нейтральным root-коммитом. История, авторы, сообщения коммитов, ветки и теги
|
|
52
|
+
приватного repo не попадают в зеркало. Параметр `preserve_target_tags = true`
|
|
53
|
+
сохраняет теги, уже существующие в публичном repo, с теми же object ID; теги
|
|
54
|
+
приватного исходника по-прежнему не публикуются. Неизвестные refs блокируют
|
|
55
|
+
публикацию, а издатель сверяет SHA каждого сохранённого тега после обновления веток:
|
|
56
|
+
Теги не отправляются в push. Если другой процесс изменит refs после проверки
|
|
57
|
+
перед публикацией, итоговая сверка обнаружит конфликт; он может обнаружиться уже
|
|
58
|
+
после обновления ветки. Перед повтором сначала сверьте зеркало:
|
|
46
59
|
|
|
47
60
|
```ini
|
|
48
61
|
mode = snapshot
|
|
@@ -50,7 +63,8 @@ snapshot_include_source_sha = false
|
|
|
50
63
|
```
|
|
51
64
|
|
|
52
65
|
`snapshot_include_source_sha = true` ставьте только если нужно сознательно раскрыть SHA
|
|
53
|
-
приватного коммита. `snapshot` не разрешает `push_tags = true
|
|
66
|
+
приватного коммита. `snapshot` не разрешает `push_tags = true`; публикацию тегов
|
|
67
|
+
из приватного исходника нельзя совмещать с сохранением публичных тегов. По умолчанию
|
|
54
68
|
остаётся обычный режим `history`.
|
|
55
69
|
|
|
56
70
|
`source` и `target` могут быть такими:
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Verification notes
|
|
2
|
+
|
|
3
|
+
These notes separate repository evidence from product intent. They are kept
|
|
4
|
+
short so the README can lead with the user outcome.
|
|
5
|
+
|
|
6
|
+
## Checked in
|
|
7
|
+
|
|
8
|
+
- `tests/test_git_private2public.py` covers folder/YAML configuration, ignore
|
|
9
|
+
and replace rule translation, allowlisted domains, version consistency, and
|
|
10
|
+
guard detection of provider-style tokens.
|
|
11
|
+
- `pyproject.toml` preserves the supported install paths shown in the README:
|
|
12
|
+
PyPI installation and local `python3 -m pip install .`.
|
|
13
|
+
|
|
14
|
+
## Deliberate limits
|
|
15
|
+
|
|
16
|
+
`publish` rewrites a temporary clone and pushes only after its configured scan;
|
|
17
|
+
it is not a claim that the source repository or its prior public copies are
|
|
18
|
+
secret-free. The guard catches its documented patterns and custom rules, but
|
|
19
|
+
cannot identify every possible credential or undo a credential exposure.
|
|
Binary file
|
|
@@ -35,7 +35,7 @@ from pathlib import Path
|
|
|
35
35
|
import time
|
|
36
36
|
from typing import Iterable
|
|
37
37
|
|
|
38
|
-
__version__ = "0.2.
|
|
38
|
+
__version__ = "0.2.3"
|
|
39
39
|
|
|
40
40
|
try:
|
|
41
41
|
import yaml
|
|
@@ -84,6 +84,14 @@ DEFAULT_SECRET_PATTERNS: list[str] = [
|
|
|
84
84
|
# Config
|
|
85
85
|
# --------------------------------------------------------------------------- #
|
|
86
86
|
|
|
87
|
+
def _yaml_bool(value: object, name: str, default: bool) -> bool:
|
|
88
|
+
if value is None:
|
|
89
|
+
return default
|
|
90
|
+
if type(value) is not bool:
|
|
91
|
+
raise SystemExit(f"{name} must be a YAML boolean")
|
|
92
|
+
return value
|
|
93
|
+
|
|
94
|
+
|
|
87
95
|
@dataclass
|
|
88
96
|
class Config:
|
|
89
97
|
source: str
|
|
@@ -92,16 +100,29 @@ class Config:
|
|
|
92
100
|
replace: list[str] = field(default_factory=list)
|
|
93
101
|
allow_domains: list[str] = field(default_factory=list)
|
|
94
102
|
fail_on_match: list[str] = field(default_factory=list)
|
|
103
|
+
scan_ignore: list[str] = field(default_factory=list)
|
|
95
104
|
push_force: bool = True
|
|
96
105
|
push_branches: list[str] = field(default_factory=lambda: ["main"])
|
|
97
106
|
push_tags: bool = False
|
|
107
|
+
preserve_target_tags: bool = False
|
|
98
108
|
mode: str = "history"
|
|
99
109
|
snapshot_include_source_sha: bool = False
|
|
100
110
|
|
|
101
111
|
@classmethod
|
|
102
112
|
def from_yaml(cls, path: Path) -> "Config":
|
|
103
113
|
data = yaml.safe_load(path.read_text())
|
|
114
|
+
if not isinstance(data, dict):
|
|
115
|
+
raise SystemExit("YAML config must be a mapping")
|
|
104
116
|
push = data.get("push") or {}
|
|
117
|
+
if not isinstance(push, dict):
|
|
118
|
+
raise SystemExit("push must be a mapping")
|
|
119
|
+
branches = push.get("branches")
|
|
120
|
+
if branches is None or branches == []:
|
|
121
|
+
branches = ["main"]
|
|
122
|
+
elif isinstance(branches, str):
|
|
123
|
+
branches = [branches]
|
|
124
|
+
elif not isinstance(branches, list):
|
|
125
|
+
raise SystemExit("push.branches must be a branch name or list of branch names")
|
|
105
126
|
return cls(
|
|
106
127
|
source=data["source"],
|
|
107
128
|
target=data["target"],
|
|
@@ -109,11 +130,19 @@ class Config:
|
|
|
109
130
|
replace=list(data.get("replace") or []),
|
|
110
131
|
allow_domains=list(data.get("allow_domains") or data.get("allow") or data.get("domains") or []),
|
|
111
132
|
fail_on_match=list(data.get("fail_on_match") or []),
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
133
|
+
scan_ignore=list(data.get("scan_ignore") or data.get("scan-ignore") or []),
|
|
134
|
+
push_force=_yaml_bool(push.get("force"), "push.force", True),
|
|
135
|
+
push_branches=branches,
|
|
136
|
+
push_tags=_yaml_bool(push.get("tags"), "push.tags", False),
|
|
137
|
+
preserve_target_tags=_yaml_bool(
|
|
138
|
+
push.get("preserve_target_tags", data.get("preserve_target_tags")),
|
|
139
|
+
"push.preserve_target_tags",
|
|
140
|
+
False,
|
|
141
|
+
),
|
|
115
142
|
mode=str(data.get("mode") or "history"),
|
|
116
|
-
snapshot_include_source_sha=
|
|
143
|
+
snapshot_include_source_sha=_yaml_bool(
|
|
144
|
+
data.get("snapshot_include_source_sha"), "snapshot_include_source_sha", False
|
|
145
|
+
),
|
|
117
146
|
)
|
|
118
147
|
|
|
119
148
|
@classmethod
|
|
@@ -121,7 +150,7 @@ class Config:
|
|
|
121
150
|
"""Load from a .gitpublic/ folder — each file is one concern, gitignore-style.
|
|
122
151
|
|
|
123
152
|
Files (all optional — if missing, that setting is empty):
|
|
124
|
-
config — source=, target=, push_force=, push_branches= (key=value)
|
|
153
|
+
config — source=, target=, push_force=, push_branches=, preserve_target_tags= (key=value)
|
|
125
154
|
ignore — one path/glob per line (# for comments)
|
|
126
155
|
replace — old==>new per line (regex: prefix supported, glob:*.ext: scoped)
|
|
127
156
|
scan — one regex/literal per line (fail_on_match)
|
|
@@ -145,6 +174,7 @@ class Config:
|
|
|
145
174
|
push_force = True
|
|
146
175
|
push_branches = ["main"]
|
|
147
176
|
push_tags = False
|
|
177
|
+
preserve_target_tags = False
|
|
148
178
|
mode = "history"
|
|
149
179
|
snapshot_include_source_sha = False
|
|
150
180
|
cfg_file = folder / "config"
|
|
@@ -165,6 +195,10 @@ class Config:
|
|
|
165
195
|
push_branches = [b.strip() for b in v.split(",") if b.strip()]
|
|
166
196
|
elif k == "push_tags":
|
|
167
197
|
push_tags = v.lower() in ("true", "yes", "1")
|
|
198
|
+
elif k == "preserve_target_tags":
|
|
199
|
+
if v.lower() not in ("true", "false", "yes", "no", "1", "0"):
|
|
200
|
+
raise SystemExit("preserve_target_tags must be true or false")
|
|
201
|
+
preserve_target_tags = v.lower() in ("true", "yes", "1")
|
|
168
202
|
elif k == "mode":
|
|
169
203
|
mode = v
|
|
170
204
|
elif k == "snapshot_include_source_sha":
|
|
@@ -177,9 +211,11 @@ class Config:
|
|
|
177
211
|
replace=read_lines("replace"),
|
|
178
212
|
allow_domains=read_lines("allow") + read_lines("domains"),
|
|
179
213
|
fail_on_match=read_lines("scan"),
|
|
214
|
+
scan_ignore=read_lines("scan_ignore"),
|
|
180
215
|
push_force=push_force,
|
|
181
216
|
push_branches=push_branches,
|
|
182
217
|
push_tags=push_tags,
|
|
218
|
+
preserve_target_tags=preserve_target_tags,
|
|
183
219
|
mode=mode,
|
|
184
220
|
snapshot_include_source_sha=snapshot_include_source_sha,
|
|
185
221
|
)
|
|
@@ -293,7 +329,15 @@ def expand_github_shorthand(value: str) -> str:
|
|
|
293
329
|
|
|
294
330
|
|
|
295
331
|
def mask_url(url: str) -> str:
|
|
296
|
-
return re.sub(r"https://[
|
|
332
|
+
return re.sub(r"(?i)(https?|ssh)://[^/\s@]+@", r"\1://***@", url)
|
|
333
|
+
|
|
334
|
+
|
|
335
|
+
def redact_target_output(output: str, target_url: str, token: str | None = None) -> str:
|
|
336
|
+
"""Remove URL credentials and explicit publisher tokens from Git diagnostics."""
|
|
337
|
+
safe = output.replace(target_url, mask_url(target_url))
|
|
338
|
+
if token:
|
|
339
|
+
safe = safe.replace(token, "***")
|
|
340
|
+
return re.sub(r"(?i)(https?|ssh)://[^/\s@]+@", r"\1://***@", safe)
|
|
297
341
|
|
|
298
342
|
|
|
299
343
|
def validate_config(config: Config, cwd: str | None = None) -> None:
|
|
@@ -333,8 +377,21 @@ def validate_config(config: Config, cwd: str | None = None) -> None:
|
|
|
333
377
|
raise SystemExit(f"missing required config value(s): {', '.join(missing)}")
|
|
334
378
|
if config.mode not in {"history", "snapshot"}:
|
|
335
379
|
raise SystemExit("mode must be 'history' or 'snapshot'")
|
|
380
|
+
for name, value in (
|
|
381
|
+
("push_force", config.push_force),
|
|
382
|
+
("push_tags", config.push_tags),
|
|
383
|
+
("preserve_target_tags", config.preserve_target_tags),
|
|
384
|
+
("snapshot_include_source_sha", config.snapshot_include_source_sha),
|
|
385
|
+
):
|
|
386
|
+
if type(value) is not bool:
|
|
387
|
+
raise SystemExit(f"{name} must be a boolean")
|
|
388
|
+
if (not isinstance(config.push_branches, list)
|
|
389
|
+
or any(not isinstance(branch, str) or not branch for branch in config.push_branches)):
|
|
390
|
+
raise SystemExit("push_branches must be a list of non-empty branch names")
|
|
336
391
|
if config.mode == "snapshot" and config.push_tags:
|
|
337
392
|
raise SystemExit("snapshot mode cannot publish tags")
|
|
393
|
+
if config.push_tags and config.preserve_target_tags:
|
|
394
|
+
raise SystemExit("push_tags and preserve_target_tags cannot be enabled together")
|
|
338
395
|
|
|
339
396
|
|
|
340
397
|
def clone_source(source: str, dest: Path) -> None:
|
|
@@ -515,6 +572,18 @@ def _allowed_match(matched: bytes, allow_domains: list[str]) -> bool:
|
|
|
515
572
|
# Scanning
|
|
516
573
|
# --------------------------------------------------------------------------- #
|
|
517
574
|
|
|
575
|
+
def _scan_ignored(fpath: str, patterns: list[str]) -> bool:
|
|
576
|
+
"""Return True when a tracked path is exempt from secret scanning."""
|
|
577
|
+
posix = fpath.replace("\\", "/")
|
|
578
|
+
for raw in patterns:
|
|
579
|
+
pat = raw.strip()
|
|
580
|
+
if not pat:
|
|
581
|
+
continue
|
|
582
|
+
if fnmatch.fnmatch(posix, pat) or fnmatch.fnmatch(posix, pat.rstrip("/") + "/*"):
|
|
583
|
+
return True
|
|
584
|
+
return False
|
|
585
|
+
|
|
586
|
+
|
|
518
587
|
def scan_tree(repo: Path, config: Config) -> list[str]:
|
|
519
588
|
"""Return list of violations (pattern + file:line) in the current tree."""
|
|
520
589
|
violations: list[str] = []
|
|
@@ -535,6 +604,8 @@ def scan_tree(repo: Path, config: Config) -> list[str]:
|
|
|
535
604
|
full = repo / fpath
|
|
536
605
|
if not full.is_file():
|
|
537
606
|
continue
|
|
607
|
+
if _scan_ignored(fpath, config.scan_ignore):
|
|
608
|
+
continue
|
|
538
609
|
try:
|
|
539
610
|
data = full.read_bytes()
|
|
540
611
|
except Exception:
|
|
@@ -739,12 +810,14 @@ def create_snapshot_root(
|
|
|
739
810
|
|
|
740
811
|
|
|
741
812
|
def _remote_ref_sha(url: str, ref: str) -> str:
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
813
|
+
try:
|
|
814
|
+
res = subprocess.run(
|
|
815
|
+
["git", "ls-remote", url, ref], capture_output=True, text=True, timeout=30
|
|
816
|
+
)
|
|
817
|
+
except subprocess.TimeoutExpired:
|
|
818
|
+
raise SystemExit("cannot read target ref: operation timed out") from None
|
|
745
819
|
if res.returncode != 0:
|
|
746
|
-
|
|
747
|
-
raise SystemExit(f"cannot read target ref {ref}: {err}")
|
|
820
|
+
raise SystemExit("cannot read target ref")
|
|
748
821
|
line = res.stdout.strip().splitlines()
|
|
749
822
|
return line[0].split()[0] if line else ""
|
|
750
823
|
|
|
@@ -758,8 +831,8 @@ def _remote_refs(url: str) -> dict[str, str]:
|
|
|
758
831
|
text=True,
|
|
759
832
|
timeout=30,
|
|
760
833
|
)
|
|
761
|
-
except subprocess.TimeoutExpired
|
|
762
|
-
raise SystemExit("cannot enumerate target refs: operation timed out") from
|
|
834
|
+
except subprocess.TimeoutExpired:
|
|
835
|
+
raise SystemExit("cannot enumerate target refs: operation timed out") from None
|
|
763
836
|
if result.returncode != 0:
|
|
764
837
|
raise SystemExit("cannot enumerate target refs")
|
|
765
838
|
refs: dict[str, str] = {}
|
|
@@ -774,13 +847,35 @@ def _remote_refs(url: str) -> dict[str, str]:
|
|
|
774
847
|
return refs
|
|
775
848
|
|
|
776
849
|
|
|
777
|
-
def _verify_target_ref_names(
|
|
778
|
-
|
|
779
|
-
|
|
850
|
+
def _verify_target_ref_names(
|
|
851
|
+
url: str,
|
|
852
|
+
allowed_refs: set[str],
|
|
853
|
+
*,
|
|
854
|
+
preserve_target_tags: bool = False,
|
|
855
|
+
) -> dict[str, str]:
|
|
856
|
+
"""Block unconfigured refs and return existing tags explicitly preserved by policy."""
|
|
857
|
+
target_refs = _remote_refs(url)
|
|
858
|
+
preserved_tags = {
|
|
859
|
+
ref: sha for ref, sha in target_refs.items()
|
|
860
|
+
if preserve_target_tags and ref.startswith("refs/tags/")
|
|
861
|
+
}
|
|
862
|
+
unexpected = sorted(set(target_refs) - allowed_refs - set(preserved_tags))
|
|
780
863
|
if unexpected:
|
|
781
864
|
preview = ", ".join(unexpected[:10])
|
|
782
865
|
suffix = f" and {len(unexpected) - 10} more" if len(unexpected) > 10 else ""
|
|
783
866
|
raise SystemExit(f"unexpected target refs block publication: {preview}{suffix}")
|
|
867
|
+
return preserved_tags
|
|
868
|
+
|
|
869
|
+
|
|
870
|
+
def _verify_target_tags_unchanged(
|
|
871
|
+
expected: dict[str, str],
|
|
872
|
+
actual: dict[str, str],
|
|
873
|
+
) -> None:
|
|
874
|
+
"""Fail before publishing if an existing public tag changed during sanitization."""
|
|
875
|
+
if actual != expected:
|
|
876
|
+
raise SystemExit(
|
|
877
|
+
"public target tags changed during sanitization; nothing was pushed"
|
|
878
|
+
)
|
|
784
879
|
|
|
785
880
|
|
|
786
881
|
def _verify_target_refs_exact(url: str, expected_refs: dict[str, str]) -> None:
|
|
@@ -929,10 +1024,15 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
929
1024
|
|
|
930
1025
|
# Capture the exact target state before any expensive rewriting. A later
|
|
931
1026
|
# push is allowed only if the branch still has this SHA (or is still absent).
|
|
1027
|
+
initial_target_refs = _remote_refs(target_url)
|
|
932
1028
|
expected_target_refs = {
|
|
933
|
-
branch:
|
|
1029
|
+
branch: initial_target_refs.get(f"refs/heads/{branch}", "")
|
|
934
1030
|
for branch in config.push_branches
|
|
935
1031
|
}
|
|
1032
|
+
expected_target_tags = {
|
|
1033
|
+
ref: sha for ref, sha in initial_target_refs.items()
|
|
1034
|
+
if config.preserve_target_tags and ref.startswith("refs/tags/")
|
|
1035
|
+
}
|
|
936
1036
|
|
|
937
1037
|
# Backup source gitdir before any destructive rewrite
|
|
938
1038
|
backup_info = _backup_gitdir(source_url, cwd)
|
|
@@ -987,6 +1087,18 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
987
1087
|
config.snapshot_include_source_sha,
|
|
988
1088
|
source_sha=original_source_sha,
|
|
989
1089
|
)
|
|
1090
|
+
# The snapshot intentionally destroys all other refs, but their
|
|
1091
|
+
# objects stay in the clone until pruned; a fail-closed history
|
|
1092
|
+
# scan would still flag them as reachable garbage.
|
|
1093
|
+
run(["git", "reflog", "expire", "--expire=now", "--all"], cwd=str(work))
|
|
1094
|
+
try:
|
|
1095
|
+
subprocess.run(
|
|
1096
|
+
["git", "gc", "--prune=now", "--quiet"],
|
|
1097
|
+
cwd=str(work), check=True, capture_output=True, text=True,
|
|
1098
|
+
timeout=600,
|
|
1099
|
+
)
|
|
1100
|
+
except subprocess.TimeoutExpired:
|
|
1101
|
+
raise ScanError("snapshot gc timed out; refusing to publish")
|
|
990
1102
|
elif overlay_files:
|
|
991
1103
|
commit_public_overlay(work)
|
|
992
1104
|
|
|
@@ -997,7 +1109,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
997
1109
|
try:
|
|
998
1110
|
violations = scan_tree(work, scan_config)
|
|
999
1111
|
violations.extend(
|
|
1000
|
-
scan_history(work, scan_config.fail_on_match, scan_config.allow_domains)
|
|
1112
|
+
scan_history(work, scan_config.fail_on_match, scan_config.allow_domains, scan_ignore=scan_config.scan_ignore)
|
|
1001
1113
|
)
|
|
1002
1114
|
except ScanError as exc:
|
|
1003
1115
|
print(f"✗ Security scan incomplete: {exc}", file=sys.stderr)
|
|
@@ -1020,12 +1132,23 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
1020
1132
|
|
|
1021
1133
|
local_head = run(["git", "rev-parse", "HEAD"], cwd=str(work)).stdout.strip()
|
|
1022
1134
|
expected_public_refs = _local_publish_refs(work, config, local_head)
|
|
1023
|
-
_verify_target_ref_names(
|
|
1135
|
+
preserved_target_tags = _verify_target_ref_names(
|
|
1136
|
+
target_url,
|
|
1137
|
+
set(expected_public_refs),
|
|
1138
|
+
preserve_target_tags=config.preserve_target_tags,
|
|
1139
|
+
)
|
|
1140
|
+
_verify_target_tags_unchanged(expected_target_tags, preserved_target_tags)
|
|
1141
|
+
expected_public_refs.update(preserved_target_tags)
|
|
1024
1142
|
|
|
1025
1143
|
# Push to target
|
|
1026
1144
|
print(f"▸ Pushing to {mask_url(target_url)} ...", file=sys.stderr)
|
|
1027
1145
|
run(["git", "remote", "add", "target", target_url], cwd=str(work))
|
|
1028
1146
|
|
|
1147
|
+
if config.preserve_target_tags:
|
|
1148
|
+
print(
|
|
1149
|
+
f"▸ Publishing configured branches; leaving {len(preserved_target_tags)} existing public tag(s) untouched ...",
|
|
1150
|
+
file=sys.stderr,
|
|
1151
|
+
)
|
|
1029
1152
|
for branch in config.push_branches:
|
|
1030
1153
|
push_cmd = ["git", "push"]
|
|
1031
1154
|
if config.push_force:
|
|
@@ -1035,7 +1158,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
1035
1158
|
push_cmd.extend(["target", f"HEAD:refs/heads/{branch}"])
|
|
1036
1159
|
res = subprocess.run(push_cmd, cwd=str(work), capture_output=True, text=True)
|
|
1037
1160
|
if res.returncode != 0:
|
|
1038
|
-
sys.stderr.write(res.stderr)
|
|
1161
|
+
sys.stderr.write(redact_target_output(res.stderr, target_url, token))
|
|
1039
1162
|
sys.exit(
|
|
1040
1163
|
f"push of {branch} failed; target probably changed while sanitizing. "
|
|
1041
1164
|
"Nothing was overwritten. Run publish again."
|
|
@@ -1050,7 +1173,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
1050
1173
|
cwd=str(work), capture_output=True, text=True
|
|
1051
1174
|
)
|
|
1052
1175
|
if res.returncode != 0:
|
|
1053
|
-
sys.stderr.write(res.stderr)
|
|
1176
|
+
sys.stderr.write(redact_target_output(res.stderr, target_url, token))
|
|
1054
1177
|
sys.exit("tag push failed; existing target tags were left unchanged")
|
|
1055
1178
|
|
|
1056
1179
|
_verify_target_refs_exact(target_url, expected_public_refs)
|
|
@@ -1067,7 +1190,7 @@ def publish(config: Config, scan_only: bool = False, cwd: str | None = None, out
|
|
|
1067
1190
|
try:
|
|
1068
1191
|
verify_violations = scan_tree(verify_repo, verify_config)
|
|
1069
1192
|
verify_violations.extend(
|
|
1070
|
-
scan_history(verify_repo, verify_config.fail_on_match, verify_config.allow_domains)
|
|
1193
|
+
scan_history(verify_repo, verify_config.fail_on_match, verify_config.allow_domains, scan_ignore=verify_config.scan_ignore)
|
|
1071
1194
|
)
|
|
1072
1195
|
except ScanError as exc:
|
|
1073
1196
|
raise SystemExit(f"post-push target scan incomplete: {exc}") from exc
|
|
@@ -1337,7 +1460,13 @@ def _reachable_scan_objects(repo: Path, refs: list[str] | None) -> dict[str, tup
|
|
|
1337
1460
|
return {}
|
|
1338
1461
|
|
|
1339
1462
|
reachable = _run_scan_command(["git", "rev-list", "--objects", *refs], repo)
|
|
1340
|
-
object_ids
|
|
1463
|
+
object_ids: set[str] = set()
|
|
1464
|
+
for line in reachable.stdout.decode().splitlines():
|
|
1465
|
+
if not line:
|
|
1466
|
+
continue
|
|
1467
|
+
object_id = line.split(maxsplit=1)[0]
|
|
1468
|
+
object_ids.add(object_id)
|
|
1469
|
+
object_ids = sorted(object_ids)
|
|
1341
1470
|
if not object_ids:
|
|
1342
1471
|
return {}
|
|
1343
1472
|
probe = _run_scan_command(
|
|
@@ -1377,8 +1506,15 @@ def scan_history(
|
|
|
1377
1506
|
rules: list[str],
|
|
1378
1507
|
allow_domains: list[str],
|
|
1379
1508
|
refs: list[str] | None = None,
|
|
1509
|
+
scan_ignore: list[str] | None = None,
|
|
1380
1510
|
) -> list[str]:
|
|
1381
|
-
"""Scan every blob reachable from selected refs, failing closed on gaps.
|
|
1511
|
+
"""Scan every blob reachable from selected refs, failing closed on gaps.
|
|
1512
|
+
|
|
1513
|
+
``scan_ignore`` is accepted for API compatibility but never exempts history
|
|
1514
|
+
objects: one blob can be reachable through both ignored and non-ignored
|
|
1515
|
+
paths, while ``rev-list --objects`` reports only one display path per blob.
|
|
1516
|
+
Path exemptions apply to the current tree scan only.
|
|
1517
|
+
"""
|
|
1382
1518
|
if not rules:
|
|
1383
1519
|
return []
|
|
1384
1520
|
compiled = _compile_rules(rules)
|
|
@@ -11,7 +11,10 @@ import git_private2public as g
|
|
|
11
11
|
def test_folder_config_loads_allow_and_domains_alias(tmp_path: Path):
|
|
12
12
|
cfgdir = tmp_path / ".gitpublic"
|
|
13
13
|
cfgdir.mkdir()
|
|
14
|
-
(cfgdir / "config").write_text(
|
|
14
|
+
(cfgdir / "config").write_text(
|
|
15
|
+
"source = owner/private\ntarget = owner/public\npush_force = false\n"
|
|
16
|
+
"push_branches = main,dev\npreserve_target_tags = true\n"
|
|
17
|
+
)
|
|
15
18
|
(cfgdir / "ignore").write_text("# comment\n.env\n*.key\n")
|
|
16
19
|
(cfgdir / "allow").write_text("github.com\n")
|
|
17
20
|
(cfgdir / "domains").write_text("example.com\n")
|
|
@@ -24,6 +27,7 @@ def test_folder_config_loads_allow_and_domains_alias(tmp_path: Path):
|
|
|
24
27
|
assert cfg.allow_domains == ["github.com", "example.com"]
|
|
25
28
|
assert cfg.push_force is False
|
|
26
29
|
assert cfg.push_branches == ["main", "dev"]
|
|
30
|
+
assert cfg.preserve_target_tags is True
|
|
27
31
|
|
|
28
32
|
|
|
29
33
|
def test_folder_config_loads_snapshot_mode(tmp_path: Path):
|
|
@@ -40,6 +44,15 @@ def test_folder_config_loads_snapshot_mode(tmp_path: Path):
|
|
|
40
44
|
assert cfg.snapshot_include_source_sha is True
|
|
41
45
|
|
|
42
46
|
|
|
47
|
+
def test_folder_config_loads_scan_ignore(tmp_path: Path):
|
|
48
|
+
cfgdir = tmp_path / ".gitpublic"
|
|
49
|
+
cfgdir.mkdir()
|
|
50
|
+
(cfgdir / "config").write_text("source = owner/private\ntarget = owner/public\n")
|
|
51
|
+
(cfgdir / "scan_ignore").write_text("# fixture globs\ntests/fixtures/**\n*.snapshot\n")
|
|
52
|
+
|
|
53
|
+
assert g.Config.load(cfgdir).scan_ignore == ["tests/fixtures/**", "*.snapshot"]
|
|
54
|
+
|
|
55
|
+
|
|
43
56
|
def test_yaml_config_accepts_allow_alias(tmp_path: Path):
|
|
44
57
|
path = tmp_path / "rules.yaml"
|
|
45
58
|
path.write_text(
|
|
@@ -55,6 +68,63 @@ def test_yaml_config_accepts_allow_alias(tmp_path: Path):
|
|
|
55
68
|
assert cfg.allow_domains == ["github.com"]
|
|
56
69
|
|
|
57
70
|
|
|
71
|
+
def test_yaml_config_loads_scan_ignore(tmp_path: Path):
|
|
72
|
+
path = tmp_path / "rules.yaml"
|
|
73
|
+
path.write_text(
|
|
74
|
+
"source: owner/private\n"
|
|
75
|
+
"target: owner/public\n"
|
|
76
|
+
"scan_ignore: ['tests/fixtures/**']\n"
|
|
77
|
+
)
|
|
78
|
+
|
|
79
|
+
assert g.Config.load(path).scan_ignore == ["tests/fixtures/**"]
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def test_yaml_config_loads_preserve_target_tags(tmp_path: Path):
|
|
83
|
+
path = tmp_path / "rules.yaml"
|
|
84
|
+
path.write_text(
|
|
85
|
+
"source: owner/private\n"
|
|
86
|
+
"target: owner/public\n"
|
|
87
|
+
"mode: snapshot\n"
|
|
88
|
+
"push:\n"
|
|
89
|
+
" preserve_target_tags: true\n"
|
|
90
|
+
)
|
|
91
|
+
|
|
92
|
+
assert g.Config.load(path).preserve_target_tags is True
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
@pytest.mark.parametrize(
|
|
96
|
+
("option", "expected_error"),
|
|
97
|
+
[
|
|
98
|
+
('push:\n preserve_target_tags: "false"\n', "push.preserve_target_tags"),
|
|
99
|
+
('push:\n force: "false"\n', "push.force"),
|
|
100
|
+
('push:\n tags: "false"\n', "push.tags"),
|
|
101
|
+
('snapshot_include_source_sha: "false"\n', "snapshot_include_source_sha"),
|
|
102
|
+
],
|
|
103
|
+
)
|
|
104
|
+
def test_yaml_config_rejects_quoted_boolean_options(tmp_path: Path, option: str, expected_error: str):
|
|
105
|
+
path = tmp_path / "rules.yaml"
|
|
106
|
+
path.write_text(
|
|
107
|
+
"source: owner/private\n"
|
|
108
|
+
"target: owner/public\n"
|
|
109
|
+
+ option
|
|
110
|
+
)
|
|
111
|
+
|
|
112
|
+
with pytest.raises(SystemExit, match=f"{expected_error} must be a YAML boolean"):
|
|
113
|
+
g.Config.load(path)
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
def test_yaml_config_treats_scalar_branch_as_one_branch_name(tmp_path: Path):
|
|
117
|
+
path = tmp_path / "rules.yaml"
|
|
118
|
+
path.write_text(
|
|
119
|
+
"source: owner/private\n"
|
|
120
|
+
"target: owner/public\n"
|
|
121
|
+
"push:\n"
|
|
122
|
+
" branches: main\n"
|
|
123
|
+
)
|
|
124
|
+
|
|
125
|
+
assert g.Config.load(path).push_branches == ["main"]
|
|
126
|
+
|
|
127
|
+
|
|
58
128
|
def test_delete_rules_map_to_filter_repo_args(tmp_path: Path):
|
|
59
129
|
replace_file = tmp_path / "replace.txt"
|
|
60
130
|
replace_file.write_text("literal==>***\n")
|
|
@@ -115,8 +185,8 @@ def test_replace_rule_strips_separator_whitespace():
|
|
|
115
185
|
|
|
116
186
|
|
|
117
187
|
def test_version_constant_matches_package_version():
|
|
118
|
-
assert g.__version__ == "0.2.
|
|
119
|
-
assert 'version = "0.2.
|
|
188
|
+
assert g.__version__ == "0.2.3"
|
|
189
|
+
assert 'version = "0.2.3"' in (Path(__file__).parent.parent / "pyproject.toml").read_text()
|
|
120
190
|
|
|
121
191
|
|
|
122
192
|
def test_gitpublic_secret_check_finds_only_replace_and_scan(tmp_path: Path):
|
|
@@ -462,6 +532,49 @@ def test_remote_ref_sha_reads_exact_sha(monkeypatch):
|
|
|
462
532
|
assert g._remote_ref_sha("example", "refs/heads/main") == sha
|
|
463
533
|
|
|
464
534
|
|
|
535
|
+
def test_remote_ref_sha_timeout_fails_closed_without_url_leak(monkeypatch):
|
|
536
|
+
secret_url = "https://x-access-token:super-secret-token@example.test/private.git"
|
|
537
|
+
|
|
538
|
+
def timeout(*args, **kwargs):
|
|
539
|
+
raise subprocess.TimeoutExpired(args[0], timeout=30)
|
|
540
|
+
|
|
541
|
+
monkeypatch.setattr(g.subprocess, "run", timeout)
|
|
542
|
+
with pytest.raises(SystemExit) as error:
|
|
543
|
+
g._remote_ref_sha(secret_url, "refs/heads/main")
|
|
544
|
+
|
|
545
|
+
message = str(error.value)
|
|
546
|
+
assert message == "cannot read target ref: operation timed out"
|
|
547
|
+
assert secret_url not in message
|
|
548
|
+
assert "super-secret-token" not in message
|
|
549
|
+
|
|
550
|
+
|
|
551
|
+
@pytest.mark.parametrize(
|
|
552
|
+
("url", "diagnostic", "secret"),
|
|
553
|
+
[
|
|
554
|
+
(
|
|
555
|
+
"https://x-access-token:private-publish-token@github.com/owner/public.git",
|
|
556
|
+
"fatal: repository 'https://x-access-token:private-publish-token@github.com/owner/public.git/' unavailable",
|
|
557
|
+
"private-publish-token",
|
|
558
|
+
),
|
|
559
|
+
(
|
|
560
|
+
"http://mirror-user:mirror-password@mirror.example/private.git/",
|
|
561
|
+
"fatal: repository 'http://mirror-user:mirror-password@mirror.example/private.git' unavailable",
|
|
562
|
+
"mirror-password",
|
|
563
|
+
),
|
|
564
|
+
(
|
|
565
|
+
"ssh://mirror-user:mirror-password@mirror.example/private.git",
|
|
566
|
+
"fatal: repository 'ssh://mirror-user:mirror-password@mirror.example/private.git' unavailable",
|
|
567
|
+
"mirror-password",
|
|
568
|
+
),
|
|
569
|
+
],
|
|
570
|
+
)
|
|
571
|
+
def test_target_push_error_redacts_url_credentials_and_token(url: str, diagnostic: str, secret: str):
|
|
572
|
+
safe = g.redact_target_output(diagnostic, url, secret)
|
|
573
|
+
|
|
574
|
+
assert secret not in safe
|
|
575
|
+
assert "***@" in safe
|
|
576
|
+
|
|
577
|
+
|
|
465
578
|
@pytest.mark.parametrize(
|
|
466
579
|
"unexpected_ref",
|
|
467
580
|
["refs/heads/stale", "refs/tags/stale", "refs/notes/private"],
|
|
@@ -484,6 +597,78 @@ def test_target_ref_allowlist_accepts_configured_head_and_tags(monkeypatch):
|
|
|
484
597
|
g._verify_target_refs_exact("target", expected)
|
|
485
598
|
|
|
486
599
|
|
|
600
|
+
def test_target_ref_allowlist_can_preserve_existing_target_tags(monkeypatch):
|
|
601
|
+
remote = {
|
|
602
|
+
"refs/heads/main": "a" * 40,
|
|
603
|
+
"refs/tags/node-v1024": "b" * 40,
|
|
604
|
+
"refs/tags/v1021": "c" * 40,
|
|
605
|
+
}
|
|
606
|
+
monkeypatch.setattr(g, "_remote_refs", lambda url: dict(remote))
|
|
607
|
+
|
|
608
|
+
preserved = g._verify_target_ref_names(
|
|
609
|
+
"target", {"refs/heads/main"}, preserve_target_tags=True
|
|
610
|
+
)
|
|
611
|
+
|
|
612
|
+
assert preserved == {
|
|
613
|
+
"refs/tags/node-v1024": "b" * 40,
|
|
614
|
+
"refs/tags/v1021": "c" * 40,
|
|
615
|
+
}
|
|
616
|
+
|
|
617
|
+
|
|
618
|
+
def test_preserve_target_tags_does_not_allow_unconfigured_branches(monkeypatch):
|
|
619
|
+
monkeypatch.setattr(
|
|
620
|
+
g,
|
|
621
|
+
"_remote_refs",
|
|
622
|
+
lambda url: {"refs/heads/stale": "a" * 40, "refs/tags/v1": "b" * 40},
|
|
623
|
+
)
|
|
624
|
+
|
|
625
|
+
with pytest.raises(SystemExit, match="unexpected target refs"):
|
|
626
|
+
g._verify_target_ref_names(
|
|
627
|
+
"target", {"refs/heads/main"}, preserve_target_tags=True
|
|
628
|
+
)
|
|
629
|
+
|
|
630
|
+
|
|
631
|
+
def test_preserved_target_tag_change_fails_post_publish_verification(monkeypatch):
|
|
632
|
+
expected = {
|
|
633
|
+
"refs/heads/main": "a" * 40,
|
|
634
|
+
"refs/tags/v1": "b" * 40,
|
|
635
|
+
}
|
|
636
|
+
monkeypatch.setattr(
|
|
637
|
+
g,
|
|
638
|
+
"_remote_refs",
|
|
639
|
+
lambda url: {"refs/heads/main": "a" * 40, "refs/tags/v1": "c" * 40},
|
|
640
|
+
)
|
|
641
|
+
|
|
642
|
+
with pytest.raises(SystemExit, match="post-push target refs differ.*changed=refs/tags/v1"):
|
|
643
|
+
g._verify_target_refs_exact("target", expected)
|
|
644
|
+
|
|
645
|
+
|
|
646
|
+
@pytest.mark.parametrize(
|
|
647
|
+
"actual",
|
|
648
|
+
[
|
|
649
|
+
{"refs/tags/v1": "c" * 40},
|
|
650
|
+
{"refs/tags/v1": "b" * 40, "refs/tags/v2": "d" * 40},
|
|
651
|
+
{},
|
|
652
|
+
],
|
|
653
|
+
ids=["changed", "added", "removed"],
|
|
654
|
+
)
|
|
655
|
+
def test_target_tag_snapshot_change_aborts_before_publication(actual):
|
|
656
|
+
with pytest.raises(SystemExit, match="public target tags changed during sanitization"):
|
|
657
|
+
g._verify_target_tags_unchanged({"refs/tags/v1": "b" * 40}, actual)
|
|
658
|
+
|
|
659
|
+
|
|
660
|
+
def test_preserve_target_tags_cannot_be_combined_with_pushing_source_tags():
|
|
661
|
+
cfg = g.Config(
|
|
662
|
+
source="owner/private",
|
|
663
|
+
target="owner/public",
|
|
664
|
+
push_tags=True,
|
|
665
|
+
preserve_target_tags=True,
|
|
666
|
+
)
|
|
667
|
+
|
|
668
|
+
with pytest.raises(SystemExit, match="push_tags and preserve_target_tags"):
|
|
669
|
+
g.validate_config(cfg)
|
|
670
|
+
|
|
671
|
+
|
|
487
672
|
def test_post_push_ref_verification_blocks_missing_or_changed_refs(monkeypatch):
|
|
488
673
|
expected = {"refs/heads/main": "a" * 40, "refs/tags/v1.0.0": "b" * 40}
|
|
489
674
|
monkeypatch.setattr(g, "_remote_refs", lambda url: {"refs/heads/main": "c" * 40})
|
|
@@ -509,6 +694,50 @@ def test_remote_refs_ignores_symbolic_head_and_peeled_tag(monkeypatch):
|
|
|
509
694
|
assert g._remote_refs("target") == {"refs/tags/v1": tag}
|
|
510
695
|
|
|
511
696
|
|
|
697
|
+
def test_reachable_scan_objects_returns_empty_mapping_for_unborn_repository(tmp_path: Path):
|
|
698
|
+
repo = tmp_path / "unborn"
|
|
699
|
+
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
|
700
|
+
|
|
701
|
+
assert g._reachable_scan_objects(repo, None) == {}
|
|
702
|
+
|
|
703
|
+
|
|
704
|
+
def test_history_scan_never_uses_scan_ignore_to_skip_an_aliased_blob(tmp_path: Path):
|
|
705
|
+
repo = tmp_path / "private"
|
|
706
|
+
repo.mkdir()
|
|
707
|
+
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
|
708
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
|
|
709
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
|
|
710
|
+
fake_token = "ghp_" + "A" * 36
|
|
711
|
+
(repo / "ignored").mkdir()
|
|
712
|
+
(repo / "ignored" / "fixture.txt").write_text(fake_token + "\n")
|
|
713
|
+
(repo / "public.txt").write_text(fake_token + "\n")
|
|
714
|
+
subprocess.run(["git", "-C", str(repo), "add", "ignored/fixture.txt", "public.txt"], check=True)
|
|
715
|
+
subprocess.run(["git", "-C", str(repo), "commit", "-q", "-m", "fixture alias"], check=True)
|
|
716
|
+
|
|
717
|
+
violations = g.scan_history(
|
|
718
|
+
repo, g.DEFAULT_SECRET_PATTERNS, [], scan_ignore=["ignored/**"]
|
|
719
|
+
)
|
|
720
|
+
|
|
721
|
+
assert violations
|
|
722
|
+
assert any("<history:" in finding for finding in violations)
|
|
723
|
+
|
|
724
|
+
|
|
725
|
+
def test_current_tree_scan_ignore_skips_matching_fixture_path(tmp_path: Path):
|
|
726
|
+
repo = tmp_path / "tree"
|
|
727
|
+
(repo / "tests" / "fixtures").mkdir(parents=True)
|
|
728
|
+
subprocess.run(["git", "init", "-q", str(repo)], check=True)
|
|
729
|
+
fake_token = "ghp_" + "C" * 36
|
|
730
|
+
(repo / "tests" / "fixtures" / "sample.txt").write_text(fake_token + "\n")
|
|
731
|
+
subprocess.run(["git", "-C", str(repo), "add", "tests/fixtures/sample.txt"], check=True)
|
|
732
|
+
cfg = g.Config(
|
|
733
|
+
source="owner/private", target="owner/public",
|
|
734
|
+
fail_on_match=g.DEFAULT_SECRET_PATTERNS,
|
|
735
|
+
scan_ignore=["tests/fixtures/**"],
|
|
736
|
+
)
|
|
737
|
+
|
|
738
|
+
assert g.scan_tree(repo, cfg) == []
|
|
739
|
+
|
|
740
|
+
|
|
512
741
|
def test_hook_dispatcher_preserves_foreign_hook_and_combines_actions(tmp_path: Path, monkeypatch):
|
|
513
742
|
repo = tmp_path / "repo"
|
|
514
743
|
repo.mkdir()
|
|
@@ -808,6 +1037,191 @@ def test_snapshot_publish_creates_single_public_root_without_tags(tmp_path: Path
|
|
|
808
1037
|
assert "private current" not in log
|
|
809
1038
|
|
|
810
1039
|
|
|
1040
|
+
def test_snapshot_prunes_secret_in_removed_private_history_before_scan(tmp_path: Path):
|
|
1041
|
+
source = tmp_path / "private"
|
|
1042
|
+
target = tmp_path / "public.git"
|
|
1043
|
+
source.mkdir()
|
|
1044
|
+
subprocess.run(["git", "init", "-q", str(source)], check=True)
|
|
1045
|
+
subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
|
|
1046
|
+
subprocess.run(["git", "-C", str(source), "config", "user.name", "Private Person"], check=True)
|
|
1047
|
+
subprocess.run(["git", "-C", str(source), "config", "user.email", "private@example.test"], check=True)
|
|
1048
|
+
fake_token = "ghp_" + "B" * 36
|
|
1049
|
+
(source / "old.txt").write_text(fake_token + "\n")
|
|
1050
|
+
subprocess.run(["git", "-C", str(source), "add", "old.txt"], check=True)
|
|
1051
|
+
subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private old"], check=True)
|
|
1052
|
+
subprocess.run(["git", "-C", str(source), "rm", "-q", "old.txt"], check=True)
|
|
1053
|
+
(source / "safe.txt").write_text("public content\n")
|
|
1054
|
+
subprocess.run(["git", "-C", str(source), "add", "safe.txt"], check=True)
|
|
1055
|
+
subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "current public source"], check=True)
|
|
1056
|
+
cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True)
|
|
1057
|
+
|
|
1058
|
+
assert g.publish(cfg, cwd=str(source)) == 0
|
|
1059
|
+
|
|
1060
|
+
public_files = subprocess.run(
|
|
1061
|
+
["git", "--git-dir", str(target), "ls-tree", "-r", "--name-only", "refs/heads/main"],
|
|
1062
|
+
capture_output=True, text=True, check=True,
|
|
1063
|
+
).stdout.splitlines()
|
|
1064
|
+
assert public_files == ["safe.txt"]
|
|
1065
|
+
|
|
1066
|
+
|
|
1067
|
+
def test_snapshot_publish_preserves_existing_public_release_tags(tmp_path: Path):
|
|
1068
|
+
source = tmp_path / "private"
|
|
1069
|
+
seed = tmp_path / "seed"
|
|
1070
|
+
target = tmp_path / "public.git"
|
|
1071
|
+
source.mkdir()
|
|
1072
|
+
seed.mkdir()
|
|
1073
|
+
subprocess.run(["git", "init", "-q", str(source)], check=True)
|
|
1074
|
+
subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
|
|
1075
|
+
subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
|
|
1076
|
+
|
|
1077
|
+
for repo, name, email in (
|
|
1078
|
+
(source, "Private Person", "private@example.test"),
|
|
1079
|
+
(seed, "Public Release Bot", "public@example.test"),
|
|
1080
|
+
):
|
|
1081
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.name", name], check=True)
|
|
1082
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.email", email], check=True)
|
|
1083
|
+
|
|
1084
|
+
(seed / "public.txt").write_text("previous release\n")
|
|
1085
|
+
subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
|
|
1086
|
+
subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "previous public snapshot"], check=True)
|
|
1087
|
+
subprocess.run(["git", "-C", str(seed), "tag", "node-v1024"], check=True)
|
|
1088
|
+
subprocess.run(["git", "-C", str(seed), "tag", "-a", "v1021", "-m", "release v1021"], check=True)
|
|
1089
|
+
subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
|
|
1090
|
+
before_tags = subprocess.run(
|
|
1091
|
+
["git", "--git-dir", str(target), "for-each-ref", "--format=%(objectname) %(refname)", "refs/tags/"],
|
|
1092
|
+
capture_output=True, text=True, check=True,
|
|
1093
|
+
).stdout.splitlines()
|
|
1094
|
+
|
|
1095
|
+
(source / "public.txt").write_text("current sanitized source\n")
|
|
1096
|
+
subprocess.run(["git", "-C", str(source), "config", "user.name", "Private Person"], check=True)
|
|
1097
|
+
subprocess.run(["git", "-C", str(source), "config", "user.email", "private@example.test"], check=True)
|
|
1098
|
+
subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
|
|
1099
|
+
subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current source"], check=True)
|
|
1100
|
+
subprocess.run(["git", "-C", str(source), "tag", "private-only-tag"], check=True)
|
|
1101
|
+
|
|
1102
|
+
cfg = g.Config(
|
|
1103
|
+
source=str(source), target=str(target), mode="snapshot",
|
|
1104
|
+
push_force=True, preserve_target_tags=True,
|
|
1105
|
+
)
|
|
1106
|
+
assert g.publish(cfg, cwd=str(source)) == 0
|
|
1107
|
+
|
|
1108
|
+
after_tags = subprocess.run(
|
|
1109
|
+
["git", "--git-dir", str(target), "for-each-ref", "--format=%(objectname) %(refname)", "refs/tags/"],
|
|
1110
|
+
capture_output=True, text=True, check=True,
|
|
1111
|
+
).stdout.splitlines()
|
|
1112
|
+
target_refs = subprocess.run(
|
|
1113
|
+
["git", "--git-dir", str(target), "for-each-ref", "--format=%(refname)"],
|
|
1114
|
+
capture_output=True, text=True, check=True,
|
|
1115
|
+
).stdout.splitlines()
|
|
1116
|
+
public_log = subprocess.run(
|
|
1117
|
+
["git", "--git-dir", str(target), "log", "--format=%an%n%s", "refs/heads/main"],
|
|
1118
|
+
capture_output=True, text=True, check=True,
|
|
1119
|
+
).stdout
|
|
1120
|
+
|
|
1121
|
+
assert after_tags == before_tags
|
|
1122
|
+
assert "refs/tags/private-only-tag" not in target_refs
|
|
1123
|
+
assert target_refs == ["refs/heads/main", "refs/tags/node-v1024", "refs/tags/v1021"]
|
|
1124
|
+
assert "Private Person" not in public_log
|
|
1125
|
+
assert "private current source" not in public_log
|
|
1126
|
+
assert subprocess.run(
|
|
1127
|
+
["git", "--git-dir", str(target), "rev-list", "--count", "refs/heads/main"],
|
|
1128
|
+
capture_output=True, text=True, check=True,
|
|
1129
|
+
).stdout.strip() == "1"
|
|
1130
|
+
|
|
1131
|
+
|
|
1132
|
+
def test_snapshot_publish_aborts_before_branch_push_if_target_tag_changes(tmp_path: Path, monkeypatch):
|
|
1133
|
+
source = tmp_path / "private"
|
|
1134
|
+
seed = tmp_path / "seed"
|
|
1135
|
+
target = tmp_path / "public.git"
|
|
1136
|
+
source.mkdir()
|
|
1137
|
+
seed.mkdir()
|
|
1138
|
+
subprocess.run(["git", "init", "-q", str(source)], check=True)
|
|
1139
|
+
subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
|
|
1140
|
+
subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
|
|
1141
|
+
for repo in (source, seed):
|
|
1142
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
|
|
1143
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
|
|
1144
|
+
(seed / "public.txt").write_text("first\n")
|
|
1145
|
+
subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
|
|
1146
|
+
subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "first"], check=True)
|
|
1147
|
+
first = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1148
|
+
subprocess.run(["git", "-C", str(seed), "tag", "v1"], check=True)
|
|
1149
|
+
(seed / "public.txt").write_text("second\n")
|
|
1150
|
+
subprocess.run(["git", "-C", str(seed), "commit", "-qam", "second"], check=True)
|
|
1151
|
+
second = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1152
|
+
subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
|
|
1153
|
+
original_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1154
|
+
(source / "public.txt").write_text("sanitized current\n")
|
|
1155
|
+
subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
|
|
1156
|
+
subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current"], check=True)
|
|
1157
|
+
|
|
1158
|
+
verify = g._verify_target_ref_names
|
|
1159
|
+
changed = False
|
|
1160
|
+
|
|
1161
|
+
def change_target_tag(url, allowed_refs, *, preserve_target_tags=False):
|
|
1162
|
+
nonlocal changed
|
|
1163
|
+
if not changed:
|
|
1164
|
+
subprocess.run(
|
|
1165
|
+
["git", "--git-dir", str(target), "update-ref", "refs/tags/v1", second],
|
|
1166
|
+
check=True,
|
|
1167
|
+
)
|
|
1168
|
+
changed = True
|
|
1169
|
+
return verify(url, allowed_refs, preserve_target_tags=preserve_target_tags)
|
|
1170
|
+
|
|
1171
|
+
monkeypatch.setattr(g, "_verify_target_ref_names", change_target_tag)
|
|
1172
|
+
cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True, preserve_target_tags=True)
|
|
1173
|
+
|
|
1174
|
+
with pytest.raises(SystemExit, match="public target tags changed during sanitization"):
|
|
1175
|
+
g.publish(cfg, cwd=str(source))
|
|
1176
|
+
|
|
1177
|
+
assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip() == original_main
|
|
1178
|
+
assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/tags/v1"], capture_output=True, text=True, check=True).stdout.strip() == second
|
|
1179
|
+
assert first != second
|
|
1180
|
+
|
|
1181
|
+
|
|
1182
|
+
def test_target_tag_change_after_preflight_is_reported_after_branch_push(tmp_path: Path):
|
|
1183
|
+
source = tmp_path / "private"
|
|
1184
|
+
seed = tmp_path / "seed"
|
|
1185
|
+
target = tmp_path / "public.git"
|
|
1186
|
+
source.mkdir()
|
|
1187
|
+
seed.mkdir()
|
|
1188
|
+
subprocess.run(["git", "init", "-q", str(source)], check=True)
|
|
1189
|
+
subprocess.run(["git", "init", "-q", "--initial-branch=main", str(seed)], check=True)
|
|
1190
|
+
subprocess.run(["git", "init", "-q", "--bare", str(target)], check=True)
|
|
1191
|
+
for repo in (source, seed):
|
|
1192
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.name", "Test"], check=True)
|
|
1193
|
+
subprocess.run(["git", "-C", str(repo), "config", "user.email", "test@example.test"], check=True)
|
|
1194
|
+
(seed / "public.txt").write_text("first\n")
|
|
1195
|
+
subprocess.run(["git", "-C", str(seed), "add", "public.txt"], check=True)
|
|
1196
|
+
subprocess.run(["git", "-C", str(seed), "commit", "-q", "-m", "first"], check=True)
|
|
1197
|
+
first = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1198
|
+
subprocess.run(["git", "-C", str(seed), "tag", "v1"], check=True)
|
|
1199
|
+
(seed / "public.txt").write_text("second\n")
|
|
1200
|
+
subprocess.run(["git", "-C", str(seed), "commit", "-qam", "second"], check=True)
|
|
1201
|
+
second = subprocess.run(["git", "-C", str(seed), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1202
|
+
subprocess.run(["git", "-C", str(seed), "push", "-q", str(target), "main", "--tags"], check=True)
|
|
1203
|
+
original_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1204
|
+
(source / "public.txt").write_text("sanitized current\n")
|
|
1205
|
+
subprocess.run(["git", "-C", str(source), "add", "public.txt"], check=True)
|
|
1206
|
+
subprocess.run(["git", "-C", str(source), "commit", "-q", "-m", "private current"], check=True)
|
|
1207
|
+
|
|
1208
|
+
hook = target / "hooks" / "pre-receive"
|
|
1209
|
+
hook.write_text(
|
|
1210
|
+
f'#!/bin/sh\nunset GIT_QUARANTINE_PATH\n'
|
|
1211
|
+
f'git --git-dir="$GIT_DIR" update-ref refs/tags/v1 {second}\n'
|
|
1212
|
+
)
|
|
1213
|
+
hook.chmod(0o755)
|
|
1214
|
+
cfg = g.Config(source=str(source), target=str(target), mode="snapshot", push_force=True, preserve_target_tags=True)
|
|
1215
|
+
|
|
1216
|
+
with pytest.raises(SystemExit, match="post-push target refs differ.*changed=refs/tags/v1"):
|
|
1217
|
+
g.publish(cfg, cwd=str(source))
|
|
1218
|
+
|
|
1219
|
+
published_main = subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/heads/main"], capture_output=True, text=True, check=True).stdout.strip()
|
|
1220
|
+
assert published_main != original_main
|
|
1221
|
+
assert subprocess.run(["git", "--git-dir", str(target), "rev-parse", "refs/tags/v1"], capture_output=True, text=True, check=True).stdout.strip() == second
|
|
1222
|
+
assert first != second
|
|
1223
|
+
|
|
1224
|
+
|
|
811
1225
|
def test_publish_blocks_stale_target_ref_before_writing_configured_head(tmp_path: Path):
|
|
812
1226
|
source = tmp_path / "private"
|
|
813
1227
|
target = tmp_path / "public.git"
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|