github-repository-sync 0.0.1__tar.gz → 0.0.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- github_repository_sync-0.0.2/PKG-INFO +188 -0
- github_repository_sync-0.0.2/README.md +177 -0
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/pyproject.toml +2 -2
- github_repository_sync-0.0.2/src/github_repository_sync.egg-info/PKG-INFO +188 -0
- github_repository_sync-0.0.2/src/github_repository_sync.py +1045 -0
- github_repository_sync-0.0.2/tests/test_github_repository_sync.py +876 -0
- github_repository_sync-0.0.1/PKG-INFO +0 -105
- github_repository_sync-0.0.1/README.md +0 -94
- github_repository_sync-0.0.1/src/github_repository_sync.egg-info/PKG-INFO +0 -105
- github_repository_sync-0.0.1/src/github_repository_sync.py +0 -396
- github_repository_sync-0.0.1/tests/test_github_repository_sync.py +0 -95
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/setup.cfg +0 -0
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/src/github_repository_sync.egg-info/SOURCES.txt +0 -0
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/src/github_repository_sync.egg-info/dependency_links.txt +0 -0
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/src/github_repository_sync.egg-info/entry_points.txt +0 -0
- {github_repository_sync-0.0.1 → github_repository_sync-0.0.2}/src/github_repository_sync.egg-info/top_level.txt +0 -0
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: github-repository-sync
|
|
3
|
+
Version: 0.0.2
|
|
4
|
+
Summary: Synchronize local repositories for a GitHub organization.
|
|
5
|
+
Author: Kaizten Analytics
|
|
6
|
+
License-Expression: LicenseRef-Proprietary
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
9
|
+
Requires-Python: >=3.10
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
|
|
12
|
+
# GitHub Organization Repository Sync
|
|
13
|
+
|
|
14
|
+
`github-repository-sync.py` synchronizes all repositories for a GitHub
|
|
15
|
+
organization with the clones found under a recursive search root.
|
|
16
|
+
|
|
17
|
+
Repositories that are missing locally are cloned into `<root>/<org>/<repo>` by
|
|
18
|
+
default. Existing repositories with clean working trees are updated with
|
|
19
|
+
`git pull --no-rebase --no-edit`. This allows compatible remote changes to be
|
|
20
|
+
merged when the local branch also has commits that need to be pushed.
|
|
21
|
+
Repositories with staged, unstaged, or untracked changes are left untouched and
|
|
22
|
+
reported as needing push/local attention. Conflicted pull attempts are aborted
|
|
23
|
+
so the repository is not left in a merge-conflict state.
|
|
24
|
+
|
|
25
|
+
The script detects existing local repositories by inspecting their Git remotes,
|
|
26
|
+
so repositories can be located in nested folders. A checked-out branch without
|
|
27
|
+
a configured upstream is not pulled and is reported as both needing attention
|
|
28
|
+
and having an error.
|
|
29
|
+
|
|
30
|
+
The script uses named arguments only. There are no positional arguments.
|
|
31
|
+
|
|
32
|
+
## Dependencies
|
|
33
|
+
|
|
34
|
+
- Python 3.10 or newer
|
|
35
|
+
- Git
|
|
36
|
+
- OpenSSH client (`ssh` and `ssh-keyscan`)
|
|
37
|
+
- GitHub CLI (`gh`), recommended for private repositories
|
|
38
|
+
|
|
39
|
+
At startup, every run verifies the SSH host key returned by `github.com`
|
|
40
|
+
against GitHub's published fingerprints and tests SSH authentication with the
|
|
41
|
+
current user's key. The program prints an error and exits before searching or
|
|
42
|
+
cloning when GitHub's authenticity cannot be established or the SSH connection
|
|
43
|
+
cannot authenticate. This preflight is also required when `--protocol https` is
|
|
44
|
+
used.
|
|
45
|
+
|
|
46
|
+
Authentication options:
|
|
47
|
+
|
|
48
|
+
- Use an authenticated GitHub CLI session with `gh auth login`.
|
|
49
|
+
- Or set `GITHUB_TOKEN` or `GH_TOKEN`.
|
|
50
|
+
- Or pass a token with `--token`.
|
|
51
|
+
|
|
52
|
+
Without authentication, GitHub only returns public repositories.
|
|
53
|
+
|
|
54
|
+
## How to use
|
|
55
|
+
|
|
56
|
+
Run the script with the organization name:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
./github-repository-sync.py --org kaizten
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
By default, the script searches recursively under `~` and clones missing
|
|
63
|
+
repositories into `~/kaizten`.
|
|
64
|
+
|
|
65
|
+
Use `--dry-run` to inspect repositories and preview all pulls, missing
|
|
66
|
+
repositories, and clone destinations without pulling or cloning anything:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Use `--limit` to process only the first repositories selected by the saved
|
|
73
|
+
unknown-first, newest-to-oldest ordering. Both argument forms are supported:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
./github-repository-sync.py --org kaizten --limit=10
|
|
77
|
+
./github-repository-sync.py --org kaizten --limit 10
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
`LIMIT` must be a positive integer. If it exceeds the number of eligible
|
|
81
|
+
repositories, every eligible repository is processed.
|
|
82
|
+
|
|
83
|
+
Use `--root` to search and clone under a different directory:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Use `--clone-dir` to clone missing repositories somewhere other than
|
|
90
|
+
`<root>/<org>`:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
./github-repository-sync.py --org kaizten --clone-dir /home/christopher/repos
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
If a clone destination already exists and is not empty, the script chooses the
|
|
97
|
+
next available suffixed directory, such as `repo-1`.
|
|
98
|
+
|
|
99
|
+
## Repository processing order
|
|
100
|
+
|
|
101
|
+
The script stores change history separately for each GitHub organization at:
|
|
102
|
+
|
|
103
|
+
```text
|
|
104
|
+
~/.github-repository-sync/<organization>/state.json
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Repositories without saved history are processed first. The remaining
|
|
108
|
+
repositories are processed from the newest observed change to the oldest, with
|
|
109
|
+
repository name used as a deterministic tie-breaker.
|
|
110
|
+
|
|
111
|
+
While processing, each repository is prefixed with its position in the current
|
|
112
|
+
run, for example `[3/42] Processing repository-name`. When `--limit` is used,
|
|
113
|
+
the denominator and all missing-repository and summary counts reflect only the
|
|
114
|
+
selected subset. Limited runs always select the strict beginning of the saved
|
|
115
|
+
ordering, so older repositories may remain unchecked across repeated runs.
|
|
116
|
+
|
|
117
|
+
The timestamp changes when the script observes a different Git-visible state:
|
|
118
|
+
the checked-out commit, staged or unstaged tracked content, or non-ignored
|
|
119
|
+
untracked files. An unchanged check does not refresh the timestamp. Because the
|
|
120
|
+
script can only observe changes while running, the timestamp records discovery
|
|
121
|
+
time rather than necessarily the exact time a change was made.
|
|
122
|
+
|
|
123
|
+
State files are written atomically with restrictive permissions. If a state
|
|
124
|
+
file is invalid or unreadable, the script prints a warning, processes every
|
|
125
|
+
repository as having no history, and replaces the file after processing.
|
|
126
|
+
`--dry-run` reads saved state to determine order but never creates or updates
|
|
127
|
+
state files.
|
|
128
|
+
|
|
129
|
+
## End-of-run summary
|
|
130
|
+
|
|
131
|
+
After repository processing starts, every run prints aggregate counts for:
|
|
132
|
+
|
|
133
|
+
- repositories processed;
|
|
134
|
+
- repositories already local and missing locally;
|
|
135
|
+
- repositories cloned and pulled successfully;
|
|
136
|
+
- repositories needing push/local attention;
|
|
137
|
+
- repositories skipped because their working trees are dirty; and
|
|
138
|
+
- repositories with errors.
|
|
139
|
+
|
|
140
|
+
Successful pulls include repositories that were already up to date. A
|
|
141
|
+
repository is counted as needing push/local attention when it has commits ahead
|
|
142
|
+
of its upstream, staged or unstaged changes, untracked files, or no configured
|
|
143
|
+
upstream. Each repository is counted at most once in the error total, even if
|
|
144
|
+
more than one operation failed. Dry runs additionally report the numbers of
|
|
145
|
+
planned pulls and clones.
|
|
146
|
+
|
|
147
|
+
Failures during repository processing produce a nonzero exit status after the
|
|
148
|
+
summary is printed. Startup failures, such as an invalid root or failed SSH
|
|
149
|
+
preflight, occur before processing and therefore do not print a summary.
|
|
150
|
+
|
|
151
|
+
## Examples
|
|
152
|
+
|
|
153
|
+
Preview missing repositories for `kaizten`:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
Clone all missing repositories for `kaizten` under the default root:
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
./github-repository-sync.py --org kaizten
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Search under `/home/christopher` and clone missing repositories under
|
|
166
|
+
`/home/christopher/kaizten`:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Clone using SSH URLs:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
./github-repository-sync.py --org kaizten --protocol ssh
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Skip archived repositories:
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
./github-repository-sync.py --org kaizten --exclude-archived
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Use an explicit GitHub token:
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
./github-repository-sync.py --org kaizten --token YOUR_GITHUB_TOKEN
|
|
188
|
+
```
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
# GitHub Organization Repository Sync
|
|
2
|
+
|
|
3
|
+
`github-repository-sync.py` synchronizes all repositories for a GitHub
|
|
4
|
+
organization with the clones found under a recursive search root.
|
|
5
|
+
|
|
6
|
+
Repositories that are missing locally are cloned into `<root>/<org>/<repo>` by
|
|
7
|
+
default. Existing repositories with clean working trees are updated with
|
|
8
|
+
`git pull --no-rebase --no-edit`. This allows compatible remote changes to be
|
|
9
|
+
merged when the local branch also has commits that need to be pushed.
|
|
10
|
+
Repositories with staged, unstaged, or untracked changes are left untouched and
|
|
11
|
+
reported as needing push/local attention. Conflicted pull attempts are aborted
|
|
12
|
+
so the repository is not left in a merge-conflict state.
|
|
13
|
+
|
|
14
|
+
The script detects existing local repositories by inspecting their Git remotes,
|
|
15
|
+
so repositories can be located in nested folders. A checked-out branch without
|
|
16
|
+
a configured upstream is not pulled and is reported as both needing attention
|
|
17
|
+
and having an error.
|
|
18
|
+
|
|
19
|
+
The script uses named arguments only. There are no positional arguments.
|
|
20
|
+
|
|
21
|
+
## Dependencies
|
|
22
|
+
|
|
23
|
+
- Python 3.10 or newer
|
|
24
|
+
- Git
|
|
25
|
+
- OpenSSH client (`ssh` and `ssh-keyscan`)
|
|
26
|
+
- GitHub CLI (`gh`), recommended for private repositories
|
|
27
|
+
|
|
28
|
+
At startup, every run verifies the SSH host key returned by `github.com`
|
|
29
|
+
against GitHub's published fingerprints and tests SSH authentication with the
|
|
30
|
+
current user's key. The program prints an error and exits before searching or
|
|
31
|
+
cloning when GitHub's authenticity cannot be established or the SSH connection
|
|
32
|
+
cannot authenticate. This preflight is also required when `--protocol https` is
|
|
33
|
+
used.
|
|
34
|
+
|
|
35
|
+
Authentication options:
|
|
36
|
+
|
|
37
|
+
- Use an authenticated GitHub CLI session with `gh auth login`.
|
|
38
|
+
- Or set `GITHUB_TOKEN` or `GH_TOKEN`.
|
|
39
|
+
- Or pass a token with `--token`.
|
|
40
|
+
|
|
41
|
+
Without authentication, GitHub only returns public repositories.
|
|
42
|
+
|
|
43
|
+
## How to use
|
|
44
|
+
|
|
45
|
+
Run the script with the organization name:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
./github-repository-sync.py --org kaizten
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
By default, the script searches recursively under `~` and clones missing
|
|
52
|
+
repositories into `~/kaizten`.
|
|
53
|
+
|
|
54
|
+
Use `--dry-run` to inspect repositories and preview all pulls, missing
|
|
55
|
+
repositories, and clone destinations without pulling or cloning anything:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Use `--limit` to process only the first repositories selected by the saved
|
|
62
|
+
unknown-first, newest-to-oldest ordering. Both argument forms are supported:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
./github-repository-sync.py --org kaizten --limit=10
|
|
66
|
+
./github-repository-sync.py --org kaizten --limit 10
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`LIMIT` must be a positive integer. If it exceeds the number of eligible
|
|
70
|
+
repositories, every eligible repository is processed.
|
|
71
|
+
|
|
72
|
+
Use `--root` to search and clone under a different directory:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Use `--clone-dir` to clone missing repositories somewhere other than
|
|
79
|
+
`<root>/<org>`:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
./github-repository-sync.py --org kaizten --clone-dir /home/christopher/repos
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
If a clone destination already exists and is not empty, the script chooses the
|
|
86
|
+
next available suffixed directory, such as `repo-1`.
|
|
87
|
+
|
|
88
|
+
## Repository processing order
|
|
89
|
+
|
|
90
|
+
The script stores change history separately for each GitHub organization at:
|
|
91
|
+
|
|
92
|
+
```text
|
|
93
|
+
~/.github-repository-sync/<organization>/state.json
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Repositories without saved history are processed first. The remaining
|
|
97
|
+
repositories are processed from the newest observed change to the oldest, with
|
|
98
|
+
repository name used as a deterministic tie-breaker.
|
|
99
|
+
|
|
100
|
+
While processing, each repository is prefixed with its position in the current
|
|
101
|
+
run, for example `[3/42] Processing repository-name`. When `--limit` is used,
|
|
102
|
+
the denominator and all missing-repository and summary counts reflect only the
|
|
103
|
+
selected subset. Limited runs always select the strict beginning of the saved
|
|
104
|
+
ordering, so older repositories may remain unchecked across repeated runs.
|
|
105
|
+
|
|
106
|
+
The timestamp changes when the script observes a different Git-visible state:
|
|
107
|
+
the checked-out commit, staged or unstaged tracked content, or non-ignored
|
|
108
|
+
untracked files. An unchanged check does not refresh the timestamp. Because the
|
|
109
|
+
script can only observe changes while running, the timestamp records discovery
|
|
110
|
+
time rather than necessarily the exact time a change was made.
|
|
111
|
+
|
|
112
|
+
State files are written atomically with restrictive permissions. If a state
|
|
113
|
+
file is invalid or unreadable, the script prints a warning, processes every
|
|
114
|
+
repository as having no history, and replaces the file after processing.
|
|
115
|
+
`--dry-run` reads saved state to determine order but never creates or updates
|
|
116
|
+
state files.
|
|
117
|
+
|
|
118
|
+
## End-of-run summary
|
|
119
|
+
|
|
120
|
+
After repository processing starts, every run prints aggregate counts for:
|
|
121
|
+
|
|
122
|
+
- repositories processed;
|
|
123
|
+
- repositories already local and missing locally;
|
|
124
|
+
- repositories cloned and pulled successfully;
|
|
125
|
+
- repositories needing push/local attention;
|
|
126
|
+
- repositories skipped because their working trees are dirty; and
|
|
127
|
+
- repositories with errors.
|
|
128
|
+
|
|
129
|
+
Successful pulls include repositories that were already up to date. A
|
|
130
|
+
repository is counted as needing push/local attention when it has commits ahead
|
|
131
|
+
of its upstream, staged or unstaged changes, untracked files, or no configured
|
|
132
|
+
upstream. Each repository is counted at most once in the error total, even if
|
|
133
|
+
more than one operation failed. Dry runs additionally report the numbers of
|
|
134
|
+
planned pulls and clones.
|
|
135
|
+
|
|
136
|
+
Failures during repository processing produce a nonzero exit status after the
|
|
137
|
+
summary is printed. Startup failures, such as an invalid root or failed SSH
|
|
138
|
+
preflight, occur before processing and therefore do not print a summary.
|
|
139
|
+
|
|
140
|
+
## Examples
|
|
141
|
+
|
|
142
|
+
Preview missing repositories for `kaizten`:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Clone all missing repositories for `kaizten` under the default root:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
./github-repository-sync.py --org kaizten
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Search under `/home/christopher` and clone missing repositories under
|
|
155
|
+
`/home/christopher/kaizten`:
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Clone using SSH URLs:
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
./github-repository-sync.py --org kaizten --protocol ssh
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
Skip archived repositories:
|
|
168
|
+
|
|
169
|
+
```bash
|
|
170
|
+
./github-repository-sync.py --org kaizten --exclude-archived
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Use an explicit GitHub token:
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
./github-repository-sync.py --org kaizten --token YOUR_GITHUB_TOKEN
|
|
177
|
+
```
|
|
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "github-repository-sync"
|
|
7
|
-
version = "0.0.
|
|
8
|
-
description = "
|
|
7
|
+
version = "0.0.2"
|
|
8
|
+
description = "Synchronize local repositories for a GitHub organization."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.10"
|
|
11
11
|
license = "LicenseRef-Proprietary"
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: github-repository-sync
|
|
3
|
+
Version: 0.0.2
|
|
4
|
+
Summary: Synchronize local repositories for a GitHub organization.
|
|
5
|
+
Author: Kaizten Analytics
|
|
6
|
+
License-Expression: LicenseRef-Proprietary
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
9
|
+
Requires-Python: >=3.10
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
|
|
12
|
+
# GitHub Organization Repository Sync
|
|
13
|
+
|
|
14
|
+
`github-repository-sync.py` synchronizes all repositories for a GitHub
|
|
15
|
+
organization with the clones found under a recursive search root.
|
|
16
|
+
|
|
17
|
+
Repositories that are missing locally are cloned into `<root>/<org>/<repo>` by
|
|
18
|
+
default. Existing repositories with clean working trees are updated with
|
|
19
|
+
`git pull --no-rebase --no-edit`. This allows compatible remote changes to be
|
|
20
|
+
merged when the local branch also has commits that need to be pushed.
|
|
21
|
+
Repositories with staged, unstaged, or untracked changes are left untouched and
|
|
22
|
+
reported as needing push/local attention. Conflicted pull attempts are aborted
|
|
23
|
+
so the repository is not left in a merge-conflict state.
|
|
24
|
+
|
|
25
|
+
The script detects existing local repositories by inspecting their Git remotes,
|
|
26
|
+
so repositories can be located in nested folders. A checked-out branch without
|
|
27
|
+
a configured upstream is not pulled and is reported as both needing attention
|
|
28
|
+
and having an error.
|
|
29
|
+
|
|
30
|
+
The script uses named arguments only. There are no positional arguments.
|
|
31
|
+
|
|
32
|
+
## Dependencies
|
|
33
|
+
|
|
34
|
+
- Python 3.10 or newer
|
|
35
|
+
- Git
|
|
36
|
+
- OpenSSH client (`ssh` and `ssh-keyscan`)
|
|
37
|
+
- GitHub CLI (`gh`), recommended for private repositories
|
|
38
|
+
|
|
39
|
+
At startup, every run verifies the SSH host key returned by `github.com`
|
|
40
|
+
against GitHub's published fingerprints and tests SSH authentication with the
|
|
41
|
+
current user's key. The program prints an error and exits before searching or
|
|
42
|
+
cloning when GitHub's authenticity cannot be established or the SSH connection
|
|
43
|
+
cannot authenticate. This preflight is also required when `--protocol https` is
|
|
44
|
+
used.
|
|
45
|
+
|
|
46
|
+
Authentication options:
|
|
47
|
+
|
|
48
|
+
- Use an authenticated GitHub CLI session with `gh auth login`.
|
|
49
|
+
- Or set `GITHUB_TOKEN` or `GH_TOKEN`.
|
|
50
|
+
- Or pass a token with `--token`.
|
|
51
|
+
|
|
52
|
+
Without authentication, GitHub only returns public repositories.
|
|
53
|
+
|
|
54
|
+
## How to use
|
|
55
|
+
|
|
56
|
+
Run the script with the organization name:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
./github-repository-sync.py --org kaizten
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
By default, the script searches recursively under `~` and clones missing
|
|
63
|
+
repositories into `~/kaizten`.
|
|
64
|
+
|
|
65
|
+
Use `--dry-run` to inspect repositories and preview all pulls, missing
|
|
66
|
+
repositories, and clone destinations without pulling or cloning anything:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Use `--limit` to process only the first repositories selected by the saved
|
|
73
|
+
unknown-first, newest-to-oldest ordering. Both argument forms are supported:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
./github-repository-sync.py --org kaizten --limit=10
|
|
77
|
+
./github-repository-sync.py --org kaizten --limit 10
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
`LIMIT` must be a positive integer. If it exceeds the number of eligible
|
|
81
|
+
repositories, every eligible repository is processed.
|
|
82
|
+
|
|
83
|
+
Use `--root` to search and clone under a different directory:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Use `--clone-dir` to clone missing repositories somewhere other than
|
|
90
|
+
`<root>/<org>`:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
./github-repository-sync.py --org kaizten --clone-dir /home/christopher/repos
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
If a clone destination already exists and is not empty, the script chooses the
|
|
97
|
+
next available suffixed directory, such as `repo-1`.
|
|
98
|
+
|
|
99
|
+
## Repository processing order
|
|
100
|
+
|
|
101
|
+
The script stores change history separately for each GitHub organization at:
|
|
102
|
+
|
|
103
|
+
```text
|
|
104
|
+
~/.github-repository-sync/<organization>/state.json
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Repositories without saved history are processed first. The remaining
|
|
108
|
+
repositories are processed from the newest observed change to the oldest, with
|
|
109
|
+
repository name used as a deterministic tie-breaker.
|
|
110
|
+
|
|
111
|
+
While processing, each repository is prefixed with its position in the current
|
|
112
|
+
run, for example `[3/42] Processing repository-name`. When `--limit` is used,
|
|
113
|
+
the denominator and all missing-repository and summary counts reflect only the
|
|
114
|
+
selected subset. Limited runs always select the strict beginning of the saved
|
|
115
|
+
ordering, so older repositories may remain unchecked across repeated runs.
|
|
116
|
+
|
|
117
|
+
The timestamp changes when the script observes a different Git-visible state:
|
|
118
|
+
the checked-out commit, staged or unstaged tracked content, or non-ignored
|
|
119
|
+
untracked files. An unchanged check does not refresh the timestamp. Because the
|
|
120
|
+
script can only observe changes while running, the timestamp records discovery
|
|
121
|
+
time rather than necessarily the exact time a change was made.
|
|
122
|
+
|
|
123
|
+
State files are written atomically with restrictive permissions. If a state
|
|
124
|
+
file is invalid or unreadable, the script prints a warning, processes every
|
|
125
|
+
repository as having no history, and replaces the file after processing.
|
|
126
|
+
`--dry-run` reads saved state to determine order but never creates or updates
|
|
127
|
+
state files.
|
|
128
|
+
|
|
129
|
+
## End-of-run summary
|
|
130
|
+
|
|
131
|
+
After repository processing starts, every run prints aggregate counts for:
|
|
132
|
+
|
|
133
|
+
- repositories processed;
|
|
134
|
+
- repositories already local and missing locally;
|
|
135
|
+
- repositories cloned and pulled successfully;
|
|
136
|
+
- repositories needing push/local attention;
|
|
137
|
+
- repositories skipped because their working trees are dirty; and
|
|
138
|
+
- repositories with errors.
|
|
139
|
+
|
|
140
|
+
Successful pulls include repositories that were already up to date. A
|
|
141
|
+
repository is counted as needing push/local attention when it has commits ahead
|
|
142
|
+
of its upstream, staged or unstaged changes, untracked files, or no configured
|
|
143
|
+
upstream. Each repository is counted at most once in the error total, even if
|
|
144
|
+
more than one operation failed. Dry runs additionally report the numbers of
|
|
145
|
+
planned pulls and clones.
|
|
146
|
+
|
|
147
|
+
Failures during repository processing produce a nonzero exit status after the
|
|
148
|
+
summary is printed. Startup failures, such as an invalid root or failed SSH
|
|
149
|
+
preflight, occur before processing and therefore do not print a summary.
|
|
150
|
+
|
|
151
|
+
## Examples
|
|
152
|
+
|
|
153
|
+
Preview missing repositories for `kaizten`:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
./github-repository-sync.py --org kaizten --dry-run
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
Clone all missing repositories for `kaizten` under the default root:
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
./github-repository-sync.py --org kaizten
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Search under `/home/christopher` and clone missing repositories under
|
|
166
|
+
`/home/christopher/kaizten`:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
./github-repository-sync.py --org kaizten --root /home/christopher
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Clone using SSH URLs:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
./github-repository-sync.py --org kaizten --protocol ssh
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Skip archived repositories:
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
./github-repository-sync.py --org kaizten --exclude-archived
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Use an explicit GitHub token:
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
./github-repository-sync.py --org kaizten --token YOUR_GITHUB_TOKEN
|
|
188
|
+
```
|