@ebalzan/ci 0.0.0-stage → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +302 -2
- package/bin/_json.mjs +95 -0
- package/bin/_lib.sh +153 -0
- package/bin/cli.mjs +105 -0
- package/bin/gh-bootstrap.sh +622 -0
- package/bin/gh-reset.sh +148 -0
- package/bin/onboard.sh +340 -0
- package/bin/sync-managed.sh +81 -0
- package/config/actions.env +33 -0
- package/config/repo-settings.json +10 -0
- package/config/repos.env +25 -0
- package/config/rulesets/protect-dev.json +10 -0
- package/config/rulesets/protect-main.json +10 -0
- package/config/rulesets/require-pull-request.json +33 -0
- package/config/secrets.env.example +31 -0
- package/docs/ARCHITECTURE.md +477 -0
- package/docs/AUTO_WORK_ITEMS_WORKFLOW.md +305 -0
- package/docs/CI_FLOW.md +205 -0
- package/docs/CLI.md +229 -0
- package/docs/COMMIT_GUIDE.md +133 -0
- package/docs/CONFIGURATION.md +177 -0
- package/docs/GITHUB_ACTIONS_SETUP.md +315 -0
- package/docs/MIGRATION.md +175 -0
- package/docs/ONBOARDING.md +166 -0
- package/docs/REPO_RULES.md +116 -0
- package/managed/scripts/release/validate-conventional-subject.sh +32 -0
- package/package.json +66 -4
- package/renovate/tooling.json +45 -0
- package/templates/.github/actions.env +17 -0
- package/templates/.github/workflows/auto-pr.yml +33 -0
- package/templates/.github/workflows/pr-checks.yml +48 -0
- package/templates/.github/workflows/promote-dev-to-main.yml +26 -0
- package/templates/.github/workflows/release.yml +36 -0
- package/templates/.husky/commit-msg +18 -0
- package/templates/eslint.config.mjs +6 -0
- package/templates/renovate.json +4 -0
|
@@ -0,0 +1,315 @@
|
|
|
1
|
+
# GitHub Actions — secrets & variables
|
|
2
|
+
|
|
3
|
+
Where every Actions secret and variable lives, **why it lives there**, how
|
|
4
|
+
shared values reach every repo, and how one command syncs it all. Read
|
|
5
|
+
[§1](#1-the-rule) before adding any new value, and [§2](#2-shared-vs-repo-the-layers)
|
|
6
|
+
before deciding whether it goes in the shared layer or in a repo.
|
|
7
|
+
|
|
8
|
+
The tools: `ebalzan-ci bootstrap` sets values, `ebalzan-ci reset` deletes them
|
|
9
|
+
([CLI.md](CLI.md)). Every key, with its default: [CONFIGURATION.md](CONFIGURATION.md).
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 1. The rule
|
|
14
|
+
|
|
15
|
+
> **Environments** hold config that describes **a specific running system you
|
|
16
|
+
> deploy to**.
|
|
17
|
+
> **Repository** ("Settings → Secrets and variables → Actions") holds config
|
|
18
|
+
> that describes **the repository's own automation**.
|
|
19
|
+
|
|
20
|
+
That is the whole decision. It is _not_ "production goes in Environments" and
|
|
21
|
+
_not_ "the sensitive ones go in Environments". The only question is:
|
|
22
|
+
|
|
23
|
+
**Does this value point at, or grant access to, one particular deployed
|
|
24
|
+
system?**
|
|
25
|
+
|
|
26
|
+
- **Yes** → environment secret/variable, in the environment named after that
|
|
27
|
+
system (`production`, `staging`). Examples: a host address, an SSH key, a
|
|
28
|
+
database URL, a registry pull token used _by that host_, live-vs-test API keys
|
|
29
|
+
of a third party.
|
|
30
|
+
- **No — it configures the repo's own bots and CI** → repository
|
|
31
|
+
secret/variable. Examples: an LLM key used to write PR descriptions, a PAT for
|
|
32
|
+
the project board, lint/label tuning.
|
|
33
|
+
|
|
34
|
+
```mermaid
|
|
35
|
+
flowchart TD
|
|
36
|
+
A["New value to configure"] --> B{"Would you mind seeing it<br/>printed in a workflow log?"}
|
|
37
|
+
B -- No --> V["VARIABLE"]
|
|
38
|
+
B -- Yes --> S["SECRET"]
|
|
39
|
+
V --> C{"Does the value depend on WHICH<br/>deployed system the job touches?"}
|
|
40
|
+
S --> C
|
|
41
|
+
C -- "Yes (host, DB, live keys)" --> E["Environment scope<br/>production / staging"]
|
|
42
|
+
C -- "No (same for all runs)" --> R["Repository scope"]
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
### Why the rule is the rule
|
|
46
|
+
|
|
47
|
+
Environments are not a folder. They change _who can read the value_:
|
|
48
|
+
|
|
49
|
+
1. **Reachability gating.** An environment secret is invisible to any job that
|
|
50
|
+
does not declare `environment: <name>`. `AWS_DEPLOY_ROLE_ARN` cannot be read by
|
|
51
|
+
`pr-checks.yml` — not by convention, by mechanism. A repository secret is
|
|
52
|
+
readable by **every job in every workflow** in the repo.
|
|
53
|
+
2. **Deployment history.** The Environments page records which release went to
|
|
54
|
+
`production` and when.
|
|
55
|
+
3. **Protection rules** (required reviewers, wait timer, branch policy) — free
|
|
56
|
+
on public repos, **paid on private ones**, see [§3](#3-plan-reality).
|
|
57
|
+
|
|
58
|
+
Point 1 is the reason to use environments at all. Everything else follows from
|
|
59
|
+
it: put a value in an environment when you want it _unreachable_ from jobs that
|
|
60
|
+
have no business with that system.
|
|
61
|
+
|
|
62
|
+
### Precedence — most specific wins
|
|
63
|
+
|
|
64
|
+
Both secrets and variables exist at two scopes here, and they are **not
|
|
65
|
+
merged**; the nearest one wins:
|
|
66
|
+
|
|
67
|
+
```text
|
|
68
|
+
environment > repository
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
(GitHub has a third, wider scope — organization — but personal accounts have
|
|
72
|
+
none, and the tools never write to it. That is why the shared layer here is
|
|
73
|
+
read at run time for variables and merged in at sync time for secrets: see
|
|
74
|
+
§2. It works the same for repos owned by an organization.)
|
|
75
|
+
|
|
76
|
+
So defining e.g. `DEPLOY_HOST` at the repo _and_ on the `production`
|
|
77
|
+
environment means `environment: production` jobs get the environment value
|
|
78
|
+
and everyone else gets the repo value. That is a legitimate **override**, but
|
|
79
|
+
only do it on purpose — an accidental duplicate is the hardest Actions bug to see,
|
|
80
|
+
because both values are masked in logs.
|
|
81
|
+
|
|
82
|
+
### Secret or variable?
|
|
83
|
+
|
|
84
|
+
Same two scopes, different visibility. **Variables are plain text** — printed
|
|
85
|
+
in logs, readable by anyone who can view repo settings. Secrets are masked and
|
|
86
|
+
write-only after creation. If you would not paste it into a public issue, it is
|
|
87
|
+
a secret. Non-secret-but-environment-specific config (a region, a health-check
|
|
88
|
+
URL, a log level) is a legitimate **environment variable** — the two axes are
|
|
89
|
+
independent.
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## 2. Shared vs repo: the layers
|
|
94
|
+
|
|
95
|
+
> **A value goes in the shared layer only when it is the same for every repo.**
|
|
96
|
+
> Anything that differs — a board number, a deploy host, a milestone — is that
|
|
97
|
+
> repo's, and lives in that repo.
|
|
98
|
+
|
|
99
|
+
```text
|
|
100
|
+
shared (lower precedence) <repo>/.github/ (wins)
|
|
101
|
+
variables config/actions.env + actions.env → shared read at run time; repo's → repository variables
|
|
102
|
+
repo secrets <shared secrets dir>/secrets.env + secrets.env ¹ → repository secrets
|
|
103
|
+
env secrets <shared secrets dir>/secrets.<env>.env + secrets.<env>.env ¹ → environment <env> secrets
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
`config/actions.env` ships inside this repo (and the npm package). The
|
|
107
|
+
**shared secrets dir** is wherever you keep your shared secrets — they are never
|
|
108
|
+
committed: `$CI_SECRETS_DIR` if set, else `config/` when you run the tools from
|
|
109
|
+
a git checkout of ebalzan/ci, else `~/.config/ebalzan-ci/`.
|
|
110
|
+
|
|
111
|
+
¹ A repo's own secrets files live in `repos/<owner>/<repo>/` inside the shared
|
|
112
|
+
secrets dir when that folder exists — recommended: every repo's secrets in one
|
|
113
|
+
place, outside any git working tree, synced without checkouts. Otherwise they
|
|
114
|
+
are the gitignored files in the repo's `.github/`.
|
|
115
|
+
|
|
116
|
+
The two columns combine key by key — **the repo's value wins**. For secrets,
|
|
117
|
+
`bootstrap` merges them and writes the result onto the repository. For
|
|
118
|
+
variables it pushes only the repo's column; the shared one is read at run time
|
|
119
|
+
(below). So a repo can:
|
|
120
|
+
|
|
121
|
+
- **inherit** a shared key by not mentioning it;
|
|
122
|
+
- **override** it by setting the same key in its own file;
|
|
123
|
+
- **add** any key of its own, in any scope (including a new environment: the
|
|
124
|
+
file `secrets.staging.env` creates and fills environment `staging`).
|
|
125
|
+
|
|
126
|
+
The dry run labels every key `shared`, `repo`, `repo (overrides shared)` or
|
|
127
|
+
`repo (same as shared: redundant)` — delete the redundant ones from the repo's
|
|
128
|
+
file so the repo keeps only what is really its own.
|
|
129
|
+
|
|
130
|
+
**Shared variables are read at run time.** In a reusable workflow `vars` are
|
|
131
|
+
the caller repo's — but `config/actions.env` is committed, and the runner
|
|
132
|
+
downloads the whole of ebalzan/ci for `uses: ebalzan/ci/actions/…@v2`. So the
|
|
133
|
+
actions (`actions/auto-work-items/src/sharedVars.ts`, also used by
|
|
134
|
+
`scripts/release/promote-dev-to-main.mjs`) load that file and fill every key
|
|
135
|
+
the repo leaves unset. Consequences:
|
|
136
|
+
|
|
137
|
+
- A shared variable changes when it is released and **the major tag moves** —
|
|
138
|
+
no bootstrap run, and nothing stale is left on any repo.
|
|
139
|
+
- A repo's Settings page shows only its own variables; the run log line
|
|
140
|
+
`Shared variables from config/actions.env: …` lists what was filled.
|
|
141
|
+
- The shared workflows pass the repo's variables whole
|
|
142
|
+
(`REPO_VARS_JSON: ${{ toJSON(vars) }}`), so any `AUTO_*` key a repo sets
|
|
143
|
+
reaches the actions with no workflow edit.
|
|
144
|
+
- Only the shared actions see shared variables. A workflow that lives in a repo
|
|
145
|
+
and reads `vars.X` itself must have `X` in that repo's `actions.env`.
|
|
146
|
+
- An empty repo variable counts as unset, so a repo cannot blank out a shared
|
|
147
|
+
value — override it with a real one.
|
|
148
|
+
|
|
149
|
+
**Shared secrets are merged at sync time** — they are not committed, so there
|
|
150
|
+
is nothing to read at run time. After changing one, re-run
|
|
151
|
+
`ebalzan-ci bootstrap --all`.
|
|
152
|
+
|
|
153
|
+
**Removing a key** from a file does not delete it on GitHub. `--prune` deletes
|
|
154
|
+
every variable/secret in a synced scope that no layer defines (after listing
|
|
155
|
+
them and asking). Environments that no file names are never touched.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## 3. Plan reality
|
|
160
|
+
|
|
161
|
+
What the GitHub plan decides, for the repos this targets:
|
|
162
|
+
|
|
163
|
+
| Feature | Public repo | Private, free plan | Private, Pro/Team |
|
|
164
|
+
| ------------------------------------------------------------- | ----------- | ------------------ | ----------------- |
|
|
165
|
+
| Environments and environment secrets | ✅ | ✅ | ✅ |
|
|
166
|
+
| Environment protection rules (required reviewers, wait timer) | ✅ | ❌ | ¹ |
|
|
167
|
+
| Branch rulesets ([REPO_RULES.md](REPO_RULES.md)) | ✅ | ❌ (skipped) | ✅ |
|
|
168
|
+
| Organization-level secrets/variables | org only | org only | org only |
|
|
169
|
+
|
|
170
|
+
¹ At the time of writing GitHub's docs reserve required reviewers and wait
|
|
171
|
+
timers on private repos for GitHub Enterprise; check the current plan matrix
|
|
172
|
+
before relying on them. Environments themselves (and their secrets) were
|
|
173
|
+
verified working on private free-plan repos.
|
|
174
|
+
|
|
175
|
+
Without required reviewers, **merging the `dev → main` promotion PR is the
|
|
176
|
+
deploy approval**: review it as the approval it is.
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## 4. File mechanics
|
|
181
|
+
|
|
182
|
+
| File | Contains | Committed? |
|
|
183
|
+
| ------------------------------------------------- | ------------------------------------ | ------------ |
|
|
184
|
+
| `config/actions.env`, `.github/actions.env` | variables | ✅ Yes |
|
|
185
|
+
| `secrets.env` (shared dir), `.github/secrets.env` | repository secrets | ❌ **Never** |
|
|
186
|
+
| `…/secrets.<env>.env` | secrets for GitHub environment `env` | ❌ **Never** |
|
|
187
|
+
| `…/secrets*.env.example` | placeholders / templates | ✅ Yes |
|
|
188
|
+
| `.github/bootstrap.conf` | per-repo tool settings (below) | ✅ Yes |
|
|
189
|
+
|
|
190
|
+
Secrets files kept in a checkout must be gitignored — `onboard` adds
|
|
191
|
+
`/.github/secrets*.env` (keeping the `.example` files) to `.gitignore`.
|
|
192
|
+
|
|
193
|
+
> **The filename is the destination.** `actions.env` / `secrets.env` land on
|
|
194
|
+
> the repository; a `.<name>` infix always targets the GitHub environment it is
|
|
195
|
+
> named after.
|
|
196
|
+
|
|
197
|
+
- **`_FILE` suffix.** A secret key ending in `_FILE` is a path: the secret is
|
|
198
|
+
created under the stripped name from that file's **contents** (dotenv cannot
|
|
199
|
+
hold a multiline value). `~` expands; a relative path is relative to the env
|
|
200
|
+
file. `FOO_FILE` in one layer and `FOO` in the other are the same key.
|
|
201
|
+
- **Local-only keys.** `LOCAL_ONLY_KEYS=A B C` in a repo's
|
|
202
|
+
`.github/bootstrap.conf` are never pushed — for values only a local tool
|
|
203
|
+
reads (a provisioning script, say).
|
|
204
|
+
- **Guards.** Empty values and placeholders (`xxxx`, `sk-ant-xxx`,
|
|
205
|
+
`github_pat_xxx`) are refused: a secret cannot be read back to notice a bad
|
|
206
|
+
write. A missing secrets file is seeded from its `.example` on first run.
|
|
207
|
+
- **Remote mode** (`bootstrap -R owner/repo` with no local checkout) reads the
|
|
208
|
+
repo's committed files through the API and its secrets from
|
|
209
|
+
`repos/<owner>/<repo>/`. Without that folder it syncs only the shared secrets,
|
|
210
|
+
and never prunes secrets.
|
|
211
|
+
|
|
212
|
+
Real secrets never enter git. These are **Actions** secrets, unrelated to any
|
|
213
|
+
app's runtime `.env`.
|
|
214
|
+
|
|
215
|
+
---
|
|
216
|
+
|
|
217
|
+
## 5. Setup
|
|
218
|
+
|
|
219
|
+
### Prerequisites
|
|
220
|
+
|
|
221
|
+
1. **GitHub CLI** — <https://cli.github.com> (`gh --version`), authenticated
|
|
222
|
+
with the `project` scope if you use boards: `gh auth login && gh auth refresh -s project`.
|
|
223
|
+
2. **Admin** on every target repo.
|
|
224
|
+
3. Each repo's own secrets in `~/.config/ebalzan-ci/repos/<owner>/<repo>/` —
|
|
225
|
+
then `--all` needs no checkouts. (Alternatively in each checkout's
|
|
226
|
+
`.github/`, with the repos side by side under `CI_REPOS_ROOT`.)
|
|
227
|
+
|
|
228
|
+
### Fill in the secrets
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
mkdir -p ~/.config/ebalzan-ci
|
|
232
|
+
cp "$(npx --no-install ebalzan-ci path)/config/secrets.env.example" ~/.config/ebalzan-ci/secrets.env
|
|
233
|
+
$EDITOR ~/.config/ebalzan-ci/secrets.env
|
|
234
|
+
# plus each repo's own, e.g.:
|
|
235
|
+
mkdir -p ~/.config/ebalzan-ci/repos/owner/my-api
|
|
236
|
+
$EDITOR ~/.config/ebalzan-ci/repos/owner/my-api/secrets.production.env
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
> **Why a separate `GH_PROJECT_TOKEN`?** The built-in `GITHUB_TOKEN` cannot write
|
|
240
|
+
> to Projects (v2) boards, which are owned by the account, not the repo — and a
|
|
241
|
+
> PR it opens does not trigger other workflows, so PR Checks would not run on
|
|
242
|
+
> the bots' PRs.
|
|
243
|
+
>
|
|
244
|
+
> **Why classic and not fine-grained?** Fine-grained PATs expose a `Projects`
|
|
245
|
+
> permission only under **Organization** permissions. There is no account-level
|
|
246
|
+
> equivalent, so a fine-grained token cannot reach a board owned by a personal
|
|
247
|
+
> account. A classic PAT with the `project` scope is the only option there. It
|
|
248
|
+
> also needs `repo`, because the automation creates issues and pull requests,
|
|
249
|
+
> not just reads them.
|
|
250
|
+
>
|
|
251
|
+
> The trade-off: `repo` on a classic PAT covers **every** repository the account
|
|
252
|
+
> can reach and cannot be narrowed to selected repos. Keep the expiry short, and
|
|
253
|
+
> reissue rather than extend. One token serves every repo; the bootstrap writes
|
|
254
|
+
> it as a repository secret in each of them, from one line in the shared
|
|
255
|
+
> `secrets.env`.
|
|
256
|
+
|
|
257
|
+
### Run the bootstrap
|
|
258
|
+
|
|
259
|
+
```bash
|
|
260
|
+
ebalzan-ci bootstrap --all --dry-run # preview every repo — run this first
|
|
261
|
+
ebalzan-ci bootstrap --all # sync every repo
|
|
262
|
+
ebalzan-ci bootstrap -C ../my-repo # one repo, from its checkout
|
|
263
|
+
ebalzan-ci bootstrap -R owner/my-repo # one repo, no checkout
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
It is idempotent — re-run it to update any value; nothing needs clicking
|
|
267
|
+
together in the GitHub UI. Options: [CLI.md](CLI.md#bootstrap).
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## 6. Verify
|
|
272
|
+
|
|
273
|
+
```bash
|
|
274
|
+
gh variable list --repo owner/my-repo
|
|
275
|
+
gh secret list --repo owner/my-repo
|
|
276
|
+
gh secret list --repo owner/my-repo --env production
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
Repository values are under **Settings → Secrets and variables → Actions**;
|
|
280
|
+
environment values under **Settings → Environments → _name_**. Then the real
|
|
281
|
+
check: push a branch (work items appear) and open a PR (title check green).
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## 7. Resetting / deleting
|
|
286
|
+
|
|
287
|
+
Prefer `bootstrap --prune` to remove keys no file defines any more.
|
|
288
|
+
`ebalzan-ci reset` is the blunt tool: it lists what exists, then requires a
|
|
289
|
+
typed confirmation. Always `--dry-run` first.
|
|
290
|
+
|
|
291
|
+
```bash
|
|
292
|
+
ebalzan-ci reset -R owner/my-repo --dry-run # repo variables that would go
|
|
293
|
+
ebalzan-ci reset -R owner/my-repo --include-secrets # variables + repo secrets
|
|
294
|
+
ebalzan-ci reset -R owner/my-repo --match '^AUTO_' # only matching names
|
|
295
|
+
ebalzan-ci reset -R owner/my-repo --env production --secrets-only --dry-run
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
---
|
|
299
|
+
|
|
300
|
+
## 8. Troubleshooting
|
|
301
|
+
|
|
302
|
+
| Symptom | Cause / fix |
|
|
303
|
+
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
304
|
+
| Secret is set but the job sees an **empty string** | The job does not declare `environment:`. Environment secrets resolve **only** inside a job that names the environment — no error, just empty. |
|
|
305
|
+
| Shared change has no effect in a repo | Secret: not re-synced (`--all`). Variable: not released yet, or the repo overrides that key — the dry run says which. |
|
|
306
|
+
| Key removed from a file still on GitHub | Removal never deletes. Re-run with `--prune`. |
|
|
307
|
+
| Two scopes, wrong value wins | Precedence is environment > repository. Delete the duplicate you did not mean to create. |
|
|
308
|
+
| `gh is not authenticated` | `gh auth login`, then `gh auth refresh -s project`. |
|
|
309
|
+
| `… is still a placeholder` / `… has an empty value` | Fill the value in, or comment the line out. |
|
|
310
|
+
| Multiline secret arrives mangled | Use the `_FILE` suffix. |
|
|
311
|
+
| `Could not verify …/projects/N` | `gh auth refresh -s project`, or fix `AUTO_PROJECT_NUMBER` in the repo's `actions.env`. |
|
|
312
|
+
| Shared secrets not picked up | Check the `shared secrets:` path the bootstrap prints; set `CI_SECRETS_DIR` if it is not where your file is. |
|
|
313
|
+
|
|
314
|
+
Naming constraints: names are case-insensitive, cannot start with `GITHUB_` or a
|
|
315
|
+
digit, and secrets are capped at 48 KB (100 per repo, 100 per environment).
|
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# Migrating from `tracebim-ci` v1 to `ebalzan/ci` v2
|
|
2
|
+
|
|
3
|
+
v2 is the same pipeline under a new name, made generic and public:
|
|
4
|
+
|
|
5
|
+
| v1 (`ebalzan/tracebim-ci`, private) | v2 (`ebalzan/ci`, public, `@ebalzan/ci` on npm) |
|
|
6
|
+
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------- |
|
|
7
|
+
| `uses: ebalzan/tracebim-ci/.github/workflows/X.yml@v1` | `uses: ebalzan/ci/.github/workflows/X.yml@v2` |
|
|
8
|
+
| Only repos owned by `ebalzan` could call it | Any repo can call it |
|
|
9
|
+
| Hook ran `../tracebim-ci/managed/…` (or a synced copy) | Hook runs `ebalzan-ci commit-msg` from the npm devDependency |
|
|
10
|
+
| `bash ../tracebim-ci/bin/gh-bootstrap.sh --repo-dir .` | `npx ebalzan-ci bootstrap` |
|
|
11
|
+
| `REPOS=` checkout names next to the CI checkout | Repos tagged `ebalzan-ci`, plus optional `OWNER/REPO` slugs |
|
|
12
|
+
| Shared `AUTO_MILESTONE_TITLE=oct-26`, `AUTO_PROJECT_OWNER=ebalzan` | Removed: a milestone is per project; the board owner defaults to the repo owner |
|
|
13
|
+
| No board configured → the work-items run failed (strict mode) | No board configured → the board step is skipped |
|
|
14
|
+
| Release needed `package.json` + `package-lock.json` | Tag-only release without `package.json`; any lockfile |
|
|
15
|
+
| Settings and branch protection by hand | `bootstrap` applies `repo-settings.json` and `rulesets/` |
|
|
16
|
+
| `TRACEBIM_CI_DIR`, `TRACEBIM_REPOS_ROOT` | `CI_DIR`, `CI_REPOS_ROOT` (the old `TRACEBIM_REPOS_ROOT` still works) |
|
|
17
|
+
|
|
18
|
+
Repos stay on v1 until they change their `uses:` lines: `v1` is never moved
|
|
19
|
+
again, so nothing breaks on its own.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Part 1 — cutting v2 (this repo, once)
|
|
24
|
+
|
|
25
|
+
The code is ready on the `feat/generic-ci` branch. The GitHub side, in this
|
|
26
|
+
order:
|
|
27
|
+
|
|
28
|
+
1. **Rename** the repo, then point your checkout at the new name:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
gh repo rename ci -R ebalzan/tracebim-ci
|
|
32
|
+
git remote set-url origin git@github.com:ebalzan/ci.git
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Tags move with the repo, so `ebalzan/ci/actions/<name>@v1` resolves right
|
|
36
|
+
away — this repo's own workflows rely on that until step 5. GitHub redirects
|
|
37
|
+
the old name for git and the web; do not rely on the redirect for `uses:` in
|
|
38
|
+
the caller repos — migrate them (Part 2).
|
|
39
|
+
|
|
40
|
+
2. **Make it public.** The history was scanned for credentials before this
|
|
41
|
+
change: the only key-shaped string is the `sk-ant-xxxx…` placeholder in
|
|
42
|
+
`config/secrets.env.example`; `config/secrets.env` was never committed. To
|
|
43
|
+
re-check: `gitleaks git .` (or `git log --all -p | grep -E 'sk-ant-[A-Za-z0-9]{20}|ghp_[A-Za-z0-9]{30}'`).
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
gh repo edit ebalzan/ci --visibility public --accept-visibility-change-consequences
|
|
47
|
+
gh api -X PUT repos/ebalzan/ci/private-vulnerability-reporting # SECURITY.md points here
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
3. **Push the branch.** `self-auto-pr.yml` opens the PR into `dev`:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
git push -u origin feat/generic-ci
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
**Retitle the PR** `feat!: make the CI repo generic and public (ebalzan/ci)`
|
|
57
|
+
— the `!` is what makes the release `v2.0.0` instead of `v1.3.0`. The title
|
|
58
|
+
check accepts `!` on a `feat/` branch.
|
|
59
|
+
|
|
60
|
+
4. **Squash-merge** it into `dev`, then squash-merge the `dev → main` PR that
|
|
61
|
+
appears (titled `feat!: promote dev to main (v2.0.0)`). The release creates
|
|
62
|
+
`v2.0.0` and the `v2` tag, and warns that the internal refs still say `@v1`.
|
|
63
|
+
That is expected.
|
|
64
|
+
|
|
65
|
+
5. **Point the internal refs at v2** (a PR naming `@v2` could not pass its
|
|
66
|
+
checks before `v2` existed):
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
git switch dev && git pull
|
|
70
|
+
git switch -c ci/internal-refs-v2
|
|
71
|
+
sed -i 's#\(ebalzan/ci/actions/[a-z-]*\)@v1#\1@v2#' .github/workflows/*.yml
|
|
72
|
+
sed -i 's#renovate/tooling\#v1#renovate/tooling\#v2#' renovate.json
|
|
73
|
+
git commit -am "ci: point internal refs at v2" && git push -u origin ci/internal-refs-v2
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Merge it and its promotion PR: `v2.0.1`, with consistent refs.
|
|
77
|
+
|
|
78
|
+
6. **Apply the rules to this repo** (now public, so the rulesets apply too):
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
npm run gh:bootstrap -- --dry-run
|
|
82
|
+
npm run gh:bootstrap -- --prune
|
|
83
|
+
gh repo edit ebalzan/ci --add-topic ebalzan-ci # optional: include it in --all
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
7. **Publish to npm** — see [CONTRIBUTING.md → Publishing to
|
|
87
|
+
npm](../CONTRIBUTING.md#publishing-to-npm): one manual `npm publish`, then
|
|
88
|
+
trusted publishing and `NPM_PUBLISH=true` for every later release.
|
|
89
|
+
|
|
90
|
+
8. **Optional:** rename the local checkout to match (`mv tracebim-ci ci`).
|
|
91
|
+
Hooks in repos not migrated yet look for `../tracebim-ci`; until they are,
|
|
92
|
+
they warn instead of checking (CI still checks), or set
|
|
93
|
+
`TRACEBIM_CI_DIR=../ci`.
|
|
94
|
+
|
|
95
|
+
9. **Release `@ebalzan/config` 0.2.0**, which drops its copy of the commit rule
|
|
96
|
+
(the hook and the PR-title check use `@ebalzan/ci`'s). In its checkout, merge
|
|
97
|
+
the `refact/move-commit-rules-to-ci` branch into `main`, then
|
|
98
|
+
`npm version minor && git push --follow-tags` (its `publish.yml` publishes
|
|
99
|
+
the tag).
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Part 2 — moving a repo to v2
|
|
104
|
+
|
|
105
|
+
For each repo (e.g. `tracebim-api`), on a `ci/…` branch:
|
|
106
|
+
|
|
107
|
+
1. **Workflow refs:**
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
sed -i 's#ebalzan/tracebim-ci/#ebalzan/ci/#; s#\(ebalzan/ci/[^@]*\)@v1#\1@v2#' .github/workflows/*.yml
|
|
111
|
+
sed -i 's#github>ebalzan/tracebim-ci//renovate/tooling\#v1#github>ebalzan/ci//renovate/tooling\#v2#' renovate.json
|
|
112
|
+
grep -rn 'uses: .*tracebim-ci\|tracebim-ci//' .github/workflows renovate.json # must print nothing
|
|
113
|
+
grep -rn 'tracebim-ci' .github/workflows # only comments left: reword them
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
2. **Variables the shared file no longer sets.** If the repo used the shared
|
|
117
|
+
milestone, add it to the repo's `.github/actions.env`:
|
|
118
|
+
|
|
119
|
+
```dotenv
|
|
120
|
+
AUTO_MILESTONE_TITLE=oct-26
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`AUTO_PROJECT_OWNER` now defaults to the repo owner, which was the old shared
|
|
124
|
+
value for repos owned by `ebalzan` — nothing to do there.
|
|
125
|
+
|
|
126
|
+
3. **Hook and scripts** (Node repos):
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
npm install --save-dev @ebalzan/ci
|
|
130
|
+
npx ebalzan-ci path # check it is installed
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Replace `.husky/commit-msg` with [templates/.husky/commit-msg](../templates/.husky/commit-msg)
|
|
134
|
+
(placeholders: `ebalzan/ci`, `ci`, `@ebalzan/ci`), and delete any local copy
|
|
135
|
+
of `scripts/release/validate-conventional-subject.sh`.
|
|
136
|
+
|
|
137
|
+
**Remove the `gh:bootstrap` and `gh:reset` npm scripts.** In v2 the admin
|
|
138
|
+
commands run from your ebalzan/ci checkout (`npm run gh:bootstrap -- -R
|
|
139
|
+
owner/repo` or `--all`) or with `npx @ebalzan/ci bootstrap`, so the repo
|
|
140
|
+
needs none ([ARCHITECTURE.md §6–7](ARCHITECTURE.md#6-packagejson)).
|
|
141
|
+
|
|
142
|
+
Repo scripts that sourced `${TRACEBIM_CI_DIR:-../tracebim-ci}/bin/_lib.sh`
|
|
143
|
+
become `source "$(npx --no-install ebalzan-ci path)/bin/_lib.sh"`.
|
|
144
|
+
|
|
145
|
+
**Code style:** adopt [`@ebalzan/config`](https://www.npmjs.com/package/@ebalzan/config)
|
|
146
|
+
(≥ 0.2.0, which no longer has a commit rule of its own). For a repo whose
|
|
147
|
+
`.prettierrc.json` already holds the shared options, that is
|
|
148
|
+
`npm install --save-dev --save-exact @ebalzan/config`, deleting
|
|
149
|
+
`.prettierrc.json`, setting `"prettier": "@ebalzan/config/prettier"`, and
|
|
150
|
+
replacing the shared part of `eslint.config.*` with
|
|
151
|
+
`...node` from `@ebalzan/config/eslint/node` (keep the repo's own ignores
|
|
152
|
+
and blocks). `npx ebalzan-config verify` confirms it.
|
|
153
|
+
|
|
154
|
+
**Secrets (optional, recommended):** move the repo's `.github/secrets*.env`
|
|
155
|
+
into `~/.config/ebalzan-ci/repos/<owner>/<repo>/`. Local scripts that read
|
|
156
|
+
those files (via `LOCAL_ONLY_KEYS`) must then read them from there too, so
|
|
157
|
+
move a repo with such scripts last, or leave its secrets in the checkout.
|
|
158
|
+
|
|
159
|
+
4. **Topic**, so `--all` finds the repo: `gh repo edit owner/repo --add-topic ebalzan-ci`.
|
|
160
|
+
|
|
161
|
+
5. **Bootstrap** and read the plan — v2 adds repository settings (squash only,
|
|
162
|
+
PR-title commit subjects, delete merged branches) and rulesets:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
npx ebalzan-ci bootstrap --dry-run
|
|
166
|
+
npx ebalzan-ci bootstrap
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
A private repo on the free plan skips the rulesets with a warning.
|
|
170
|
+
|
|
171
|
+
6. Commit as `ci: move to ebalzan/ci v2`, push, merge. Done when the PR's
|
|
172
|
+
`title / Validate PR Title` check runs from `ebalzan/ci@v2`.
|
|
173
|
+
|
|
174
|
+
`--all` finds a repo by its topic from step 4 on; v2 ships with an empty
|
|
175
|
+
`REPOS` list in `config/repos.env`.
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# Onboarding a repo
|
|
2
|
+
|
|
3
|
+
Two ways: one command, or by hand. Both end in the same state.
|
|
4
|
+
|
|
5
|
+
- [Before the first repo](#before-the-first-repo)
|
|
6
|
+
- [With `onboard`](#with-onboard)
|
|
7
|
+
- [By hand](#by-hand)
|
|
8
|
+
- [What each generated file is for](#what-each-generated-file-is-for)
|
|
9
|
+
- [After onboarding](#after-onboarding)
|
|
10
|
+
- [Troubleshooting](#troubleshooting)
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Before the first repo
|
|
15
|
+
|
|
16
|
+
Once per machine:
|
|
17
|
+
|
|
18
|
+
1. **GitHub CLI**, logged in as an account with **admin** on the repos:
|
|
19
|
+
`gh auth login`. If you will use Projects (v2) boards, also
|
|
20
|
+
`gh auth refresh -s project`.
|
|
21
|
+
2. **Node.js ≥ 20**, git and bash.
|
|
22
|
+
3. **Shared secrets** in `~/.config/ebalzan-ci/secrets.env` (or any directory,
|
|
23
|
+
with `CI_SECRETS_DIR` pointing at it):
|
|
24
|
+
|
|
25
|
+
```dotenv
|
|
26
|
+
ANTHROPIC_API_KEY=sk-ant-... # optional: AI-written issue/PR descriptions
|
|
27
|
+
GH_PROJECT_TOKEN=ghp_... # recommended: classic PAT, scopes repo + project
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
[config/secrets.env.example](../config/secrets.env.example) explains each one,
|
|
31
|
+
and [CONFIGURATION.md](CONFIGURATION.md#secrets) explains why the token must be
|
|
32
|
+
classic. A repo's **own** secrets (deploy credentials, …) go next to them, in
|
|
33
|
+
`~/.config/ebalzan-ci/repos/<owner>/<repo>/secrets.env` and
|
|
34
|
+
`secrets.<environment>.env` ([ARCHITECTURE §7](ARCHITECTURE.md#7-where-you-run-commands)).
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## With `onboard`
|
|
39
|
+
|
|
40
|
+
From a clean checkout of the repo:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
npx @ebalzan/ci onboard --dry-run
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
```text
|
|
47
|
+
Onboard owner/my-repo ← ebalzan/ci@v2 (runs-on: ubuntu-latest)
|
|
48
|
+
|
|
49
|
+
files (branch ci/onboard-ebalzan-ci from origin/main)
|
|
50
|
+
create .github/actions.env
|
|
51
|
+
create .github/workflows/auto-pr.yml
|
|
52
|
+
create .github/workflows/pr-checks.yml
|
|
53
|
+
create .github/workflows/promote-dev-to-main.yml
|
|
54
|
+
create .github/workflows/release.yml
|
|
55
|
+
create .husky/commit-msg
|
|
56
|
+
create eslint.config.mjs
|
|
57
|
+
create renovate.json
|
|
58
|
+
update .gitignore (ignore .github/secrets*.env)
|
|
59
|
+
update package.json "prettier": "@ebalzan/config/prettier"
|
|
60
|
+
npm
|
|
61
|
+
install --save-dev @ebalzan/ci@^2.0.0 (and husky, if missing)
|
|
62
|
+
install --save-dev --save-exact @ebalzan/config (and eslint, prettier, if missing)
|
|
63
|
+
GitHub (owner/my-repo)
|
|
64
|
+
create branch dev from origin/main
|
|
65
|
+
add topic 'ebalzan-ci'
|
|
66
|
+
bootstrap: variables, secrets, settings, rulesets
|
|
67
|
+
push ci/onboard-ebalzan-ci → auto-pr.yml opens the PR into dev
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
If the plan looks right, run it without `--dry-run`. It asks once before
|
|
71
|
+
touching GitHub, and the bootstrap asks once more before pushing config.
|
|
72
|
+
|
|
73
|
+
Useful options: `--runs-on self-hosted` for self-hosted runners, `--no-push`
|
|
74
|
+
to only make the local commit (inspect it with `git show --stat`, push it
|
|
75
|
+
yourself later), `--force` to replace files the repo already has. All options:
|
|
76
|
+
[CLI.md](CLI.md#onboard).
|
|
77
|
+
|
|
78
|
+
**Existing files are kept.** If the repo already has, say, a `release.yml`,
|
|
79
|
+
`onboard` leaves it and says so; compare it with
|
|
80
|
+
[templates/](../templates/.github/workflows/) and merge by hand, or re-run with
|
|
81
|
+
`--force` on a fresh branch.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## By hand
|
|
86
|
+
|
|
87
|
+
1. **Branches.** Create `dev` from the default branch, and make `main` the
|
|
88
|
+
default branch if it is not.
|
|
89
|
+
2. **Workflows.** Copy the four files in
|
|
90
|
+
[templates/.github/workflows/](../templates/.github/workflows/) into the
|
|
91
|
+
repo's `.github/workflows/`, replacing `__CI_REPO__` with `ebalzan/ci`,
|
|
92
|
+
`__CI_REF__` with `v2` and `__RUNS_ON__` with your runner label.
|
|
93
|
+
3. **Variables.** Copy [templates/.github/actions.env](../templates/.github/actions.env)
|
|
94
|
+
to `.github/actions.env` and set what the repo needs (often nothing, or just
|
|
95
|
+
`AUTO_PROJECT_NUMBER`). Add `/.github/secrets.env`, `/.github/secrets.*.env`
|
|
96
|
+
and `!/.github/secrets*.env.example` to `.gitignore`.
|
|
97
|
+
4. **Hook** (Node repos): `npm install --save-dev @ebalzan/ci husky`, add
|
|
98
|
+
`"prepare": "husky"` to `scripts`, and copy
|
|
99
|
+
[templates/.husky/commit-msg](../templates/.husky/commit-msg) to
|
|
100
|
+
`.husky/commit-msg` (fill `__CI_REPO__`, `__CI_REPO_NAME__`, `__CI_PACKAGE__`).
|
|
101
|
+
**Code style** (Node repos): `npm install --save-dev --save-exact
|
|
102
|
+
@ebalzan/config eslint prettier`, set `"prettier": "@ebalzan/config/prettier"`
|
|
103
|
+
in `package.json`, copy [templates/eslint.config.mjs](../templates/eslint.config.mjs),
|
|
104
|
+
and make `tsconfig.json` extend `@ebalzan/config/tsconfig/node.json`. Check
|
|
105
|
+
with `npx ebalzan-config verify`.
|
|
106
|
+
5. **Renovate** (optional): install the [Renovate app](https://github.com/apps/renovate)
|
|
107
|
+
on the repo and commit `renovate.json` extending
|
|
108
|
+
`github>ebalzan/ci//renovate/tooling#v2`.
|
|
109
|
+
6. **Topic:** `gh repo edit owner/my-repo --add-topic ebalzan-ci`, so `--all`
|
|
110
|
+
finds it.
|
|
111
|
+
7. **Config:** `npx @ebalzan/ci bootstrap --dry-run`, then without `--dry-run`.
|
|
112
|
+
8. Push on a `ci/…` branch and merge the PR.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## What each generated file is for
|
|
117
|
+
|
|
118
|
+
| File | Runs on | Does |
|
|
119
|
+
| ------------------------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
120
|
+
| `.github/workflows/auto-pr.yml` | push to any work branch | Tracking issue + PR into `dev` + board card. [Details](AUTO_WORK_ITEMS_WORKFLOW.md) |
|
|
121
|
+
| `.github/workflows/pr-checks.yml` | PRs into `dev` and `main` | The shared title check, plus **your** build/test jobs — add them here. |
|
|
122
|
+
| `.github/workflows/promote-dev-to-main.yml` | push to `dev` | Opens/updates the single `dev → main` PR. |
|
|
123
|
+
| `.github/workflows/release.yml` | push to `main` | Bump, tag, GitHub Release, sync `dev` — plus **your** deploy job, chained on `needs.release.outputs.tag`. |
|
|
124
|
+
| `.github/actions.env` | `bootstrap` | The repo's own variables. |
|
|
125
|
+
| `.husky/commit-msg` | every local commit | Rejects non-conventional messages early. Never blocks when the package is missing. |
|
|
126
|
+
| `renovate.json` | Renovate, weekly | One `chore(deps): update dev tooling` PR into `dev`. |
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## After onboarding
|
|
131
|
+
|
|
132
|
+
What is shared and what is now the repo's own, stage by stage:
|
|
133
|
+
[ARCHITECTURE.md](ARCHITECTURE.md).
|
|
134
|
+
|
|
135
|
+
- **Add the repo's own checks** to `pr-checks.yml` — the template has a
|
|
136
|
+
commented `validate` job (`npm ci`, `ebalzan-config verify`, `npm run
|
|
137
|
+
validate`). They run in parallel with `title`.
|
|
138
|
+
- **Own Prettier/ESLint/TypeScript config?** `onboard` keeps it and says so.
|
|
139
|
+
Make it extend `@ebalzan/config` instead of copying rules, then
|
|
140
|
+
`npx ebalzan-config verify` passes ([ARCHITECTURE §10](ARCHITECTURE.md#10-the-two-packages-ebalzanci-and-ebalzanconfig)).
|
|
141
|
+
- **Add the deploy** to `release.yml` — the template has a commented `deploy`
|
|
142
|
+
job. Gate it with `if: needs.release.outputs.tag != ''`. For anything beyond
|
|
143
|
+
a one-liner, use a repo-local `deploy.yml` plus a manual `redeploy.yml`:
|
|
144
|
+
[ARCHITECTURE.md §4](ARCHITECTURE.md#4-deploys-are-yours).
|
|
145
|
+
- **A board?** Set `AUTO_PROJECT_NUMBER` in `.github/actions.env` and re-run
|
|
146
|
+
`ebalzan-ci bootstrap`. The board needs the fields listed in
|
|
147
|
+
[CONFIGURATION.md](CONFIGURATION.md#project-board).
|
|
148
|
+
- **Repo-specific values** (deploy hosts, keys) go in the repo's
|
|
149
|
+
`.github/secrets.<environment>.env` — see
|
|
150
|
+
[GITHUB_ACTIONS_SETUP.md](GITHUB_ACTIONS_SETUP.md).
|
|
151
|
+
- **Not a Node repo?** Releases work tag-only (no version commit). There is no
|
|
152
|
+
husky hook; the PR-title check still enforces the convention on every merge.
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## Troubleshooting
|
|
157
|
+
|
|
158
|
+
| Symptom | Cause / fix |
|
|
159
|
+
| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
160
|
+
| `… has uncommitted changes` | Commit or stash first; onboarding always works on a fresh branch. |
|
|
161
|
+
| `Branch ci/onboard-ebalzan-ci already exists` | A previous run left it. `git branch -D ci/onboard-ebalzan-ci` or pass `--branch`. |
|
|
162
|
+
| `Could not install @ebalzan/ci` | No network, or a version that is not published. The hook skips the local check until it is installed; CI still checks. |
|
|
163
|
+
| No PR appeared after the push | Open the Actions tab: `Auto Create Work Items` failed or did not run. Check the repo allows actions from other repositories (Settings → Actions → General → "Allow all actions"). Open the PR from the compare URL `onboard` printed. |
|
|
164
|
+
| `workflow was not found` / `… is not allowed to be used` | The repo's Actions policy blocks `ebalzan/ci`. Allow all actions, or add `ebalzan/ci/*` to the allowed list. |
|
|
165
|
+
| `Rulesets are not available … skipped` | Private repo on the free plan. Everything else was applied. [REPO_RULES.md](REPO_RULES.md#plans) |
|
|
166
|
+
| PR checks did not run on the bot's PR | Opened with `GITHUB_TOKEN` because `GH_PROJECT_TOKEN` is not set. Set it, or push to the PR. |
|