@quantile-co/factory-pi 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/dist/DEPENDENCY-LICENSES.txt +261 -0
- package/dist/index.d.ts +2 -0
- package/dist/index.js +3870 -0
- package/dist/settings.json +29 -0
- package/package.json +407 -3
- package/skills/day-zero-deploy/SKILL.md +32 -0
- package/skills/day-zero-deploy/references/local-state.md +267 -0
- package/skills/deploy/SKILL.md +34 -0
- package/skills/deploy/references/configuration.md +138 -0
- package/skills/deploy/references/plan-apply.md +189 -0
- package/skills/deploy/references/saved-plan-review.md +59 -0
- package/README.md +0 -3
|
@@ -0,0 +1,267 @@
|
|
|
1
|
+
# New-installation local state
|
|
2
|
+
|
|
3
|
+
This is the shared, executable-by-phase runbook for a genuinely new repository
|
|
4
|
+
using Factory's `tf/modules/project/{github,gcp}` and protected reusable
|
|
5
|
+
workflows. Each phase requires separate approval; do not paste the whole document
|
|
6
|
+
into a shell. Inventory the repository, support projects, buckets, WIF and
|
|
7
|
+
local/remote state before any mutation. If any installation exists, stop and
|
|
8
|
+
adopt its state rather than bootstrap again.
|
|
9
|
+
|
|
10
|
+
## 1. Identity and inputs
|
|
11
|
+
|
|
12
|
+
Use the pinned direnv shell; ordinary entry is credential-free.
|
|
13
|
+
Do not run protected Plan/Apply while bootstrapping: their CI inputs are not ready.
|
|
14
|
+
Keep credentials in the process environment, not Nix attributes or shell history.
|
|
15
|
+
Google CLI login, ADC, the operator's GitHub login and the provider PAT are distinct.
|
|
16
|
+
Read the approved billing account, parent folder and bucket location from the
|
|
17
|
+
concrete Terraform root. Verify access to those exact resources; do not substitute
|
|
18
|
+
Factory's values for a different installation. Reusable modules still accept
|
|
19
|
+
caller-selected values, but shared workflows do not supply billing/folder overrides.
|
|
20
|
+
Export local `TF_DIR` from the repository task/app's mapping of the workflow's
|
|
21
|
+
logical `--tf-layer`. The repository owns its directory and existing backend prefix.
|
|
22
|
+
Shared YAML does not prescribe either value.
|
|
23
|
+
Define reviewed `project_owners` and `project_members` locals in the concrete
|
|
24
|
+
Terraform root, pairing `gcp_email` with `gh_username`; use `[]` for no members.
|
|
25
|
+
Confirm each pair with its owner. Project emails into GCP and usernames into
|
|
26
|
+
GitHub. There are no runtime roster overrides or Actions owner lists. Members
|
|
27
|
+
receive repository write access only, with no added GCP or WIF grants. Names
|
|
28
|
+
describe project roles, not direct GCP Owner grants. See the [configuration contract](../../deploy/references/configuration.md).
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
set -euo pipefail
|
|
32
|
+
set +x
|
|
33
|
+
umask 077
|
|
34
|
+
|
|
35
|
+
# One-time Day Zero for the explicitly selected owner/repository. Publish the
|
|
36
|
+
# reviewed repo on main first (use a PR if required); confirm its actual Check
|
|
37
|
+
# status. Plan and Apply are manual-only; wait until prod is provisioned.
|
|
38
|
+
# Stop if this root already has local/remote state or a support project; adopt
|
|
39
|
+
# that state instead. Run in the repo's pinned direnv shell; verify project
|
|
40
|
+
# creator and billing access to the selected folder/account, not assumptions
|
|
41
|
+
# about a different repository's organization or credentials.
|
|
42
|
+
|
|
43
|
+
# Authenticate locally and check the operator-supplied inputs.
|
|
44
|
+
: "${TF_DIR:?set the reviewed repository Terraform root}"
|
|
45
|
+
gcloud auth login
|
|
46
|
+
gcloud auth application-default login # select the same Google account
|
|
47
|
+
: "${GCP_WIF_POOL:?set the Q0 GitHub WIF pool resource path}"
|
|
48
|
+
: "${GCP_WIF_PROVIDER:?set the Q0 quantile-co GitHub WIF provider path}"
|
|
49
|
+
: "${GH_REPO_TOKEN:?load a repository-scoped token from a secure local source}"
|
|
50
|
+
|
|
51
|
+
# Map public names to Terraform inputs; keep the GitHub token out of TF_VAR_*.
|
|
52
|
+
export TF_VAR_gcp_wif_pool="$GCP_WIF_POOL"
|
|
53
|
+
|
|
54
|
+
gh auth status # operator's GitHub CLI login, not the provider token
|
|
55
|
+
: "${GH_REPO:?set the intended owner/repo from an approved secure context}"
|
|
56
|
+
# Ignore the selector when verifying the checkout's own repository.
|
|
57
|
+
repo=$(env -u GH_REPO gh repo view --json nameWithOwner --jq .nameWithOwner)
|
|
58
|
+
test "$GH_REPO" = "$repo"
|
|
59
|
+
REPO_NAME=${repo#*/}
|
|
60
|
+
export GITHUB_TOKEN="$GH_REPO_TOKEN" # provider credential only for local Terraform
|
|
61
|
+
export REST_API_BEARER="$GH_REPO_TOKEN" # REST provider uses the same runtime token
|
|
62
|
+
# Verify the existing pool maps main_branch_repository_actor_id and the
|
|
63
|
+
# provider accepts this repository's owner. Do not impersonate Q0 break-glass.
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
## 2. Local seed and imports
|
|
68
|
+
|
|
69
|
+
Confirm no existing installation or backend initialization. Local file absence
|
|
70
|
+
alone is not sufficient. Approve each import separately; inspect other existing
|
|
71
|
+
resources before creating anything.
|
|
72
|
+
|
|
73
|
+
```sh
|
|
74
|
+
# Start with local state: the bucket does not exist yet.
|
|
75
|
+
test ! -e "$TF_DIR/backend_override.tf"
|
|
76
|
+
test ! -e "$TF_DIR/terraform.tfstate"
|
|
77
|
+
unset TF_DATA_DIR TF_WORKSPACE
|
|
78
|
+
printf 'terraform {\n backend "local" {}\n}\n' > "$TF_DIR/backend_override.tf"
|
|
79
|
+
tofu -chdir="$TF_DIR" init -input=false -lockfile=readonly
|
|
80
|
+
|
|
81
|
+
# The repository already exists. Import it ONLY when it has no state address.
|
|
82
|
+
tofu -chdir="$TF_DIR" import module.gh.github_repository.self "$REPO_NAME"
|
|
83
|
+
tofu -chdir="$TF_DIR" plan
|
|
84
|
+
|
|
85
|
+
# Review existing repo settings, the observed required Check context, and all changes.
|
|
86
|
+
# Import any other existing resources before approving a fresh local plan.
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
After separate bootstrap Apply approval, inspect the interactive plan again
|
|
90
|
+
before confirming it. This local bootstrap exception is not the ordinary CI path.
|
|
91
|
+
|
|
92
|
+
```sh
|
|
93
|
+
tofu -chdir="$TF_DIR" apply
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## 2a. Out-of-band project deletion lien
|
|
97
|
+
|
|
98
|
+
After the new project's approved local Apply, protect the state project from
|
|
99
|
+
accidental deletion **outside Terraform**. The lien is not a Terraform resource,
|
|
100
|
+
import, state address, or CI task. Use the operator's authenticated `gcloud`
|
|
101
|
+
identity, which must have `resourcemanager.projects.updateLiens` on the project
|
|
102
|
+
(for example, `roles/resourcemanager.lienModifier`); listing also requires
|
|
103
|
+
`resourcemanager.projects.get`. Google CLI authentication is separate from the
|
|
104
|
+
ADC used by Terraform. Do not grant these permissions or impersonate Q0
|
|
105
|
+
break-glass as an implicit part of bootstrap. Stop for separate authorization
|
|
106
|
+
if the selected identity lacks access.
|
|
107
|
+
|
|
108
|
+
Obtain the project ID from the applied local state, verify it is the intended
|
|
109
|
+
new project, and inspect existing liens before creating anything. If an
|
|
110
|
+
appropriate deletion lien already exists, stop rather than create a duplicate.
|
|
111
|
+
The read, credential choice, creation, and verification are separately approved
|
|
112
|
+
operations; do not run this block as one automated script.
|
|
113
|
+
|
|
114
|
+
```sh
|
|
115
|
+
GCP_PROJECT_ID=$(tofu -chdir="$TF_DIR" output -raw gcp_project_id)
|
|
116
|
+
: "${GCP_PROJECT_ID:?verify the new state project ID}"
|
|
117
|
+
gcloud config get-value account
|
|
118
|
+
# After read-only inspection is approved:
|
|
119
|
+
gcloud alpha resource-manager liens list --project="$GCP_PROJECT_ID" --format=json
|
|
120
|
+
# Only after separately approving creation for this exact project:
|
|
121
|
+
gcloud alpha resource-manager liens create \
|
|
122
|
+
--project="$GCP_PROJECT_ID" \
|
|
123
|
+
--restrictions=resourcemanager.projects.delete \
|
|
124
|
+
--reason="Protect this project from deletion." \
|
|
125
|
+
--origin=operator
|
|
126
|
+
# Verify the returned name, project, origin, reason, and restriction:
|
|
127
|
+
gcloud alpha resource-manager liens list --project="$GCP_PROJECT_ID" --format=json
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Using `--impersonate-service-account` and `--billing-project` requires separate
|
|
131
|
+
approval and verified token-creation, lien-modification and quota-project
|
|
132
|
+
permissions. A break-glass account is not the default bootstrap identity.
|
|
133
|
+
Keep the returned lien name in private operator records, never Git or Terraform
|
|
134
|
+
state. Do not remove the lien during normal state migration, CI handoff, or
|
|
135
|
+
Terraform teardown. An exceptional removal requires separate approval for the
|
|
136
|
+
**exact** lien name: inspect the project's liens again, then run
|
|
137
|
+
`gcloud alpha resource-manager liens delete "$GCP_LIEN_NAME"` for only that
|
|
138
|
+
verified name. Never bulk-delete the project's liens.
|
|
139
|
+
|
|
140
|
+
## 3. State migration
|
|
141
|
+
|
|
142
|
+
Authorize migration separately. Verify the output bucket and preserve the state
|
|
143
|
+
backup privately. Move this same lineage; never reset state or force-copy over
|
|
144
|
+
an existing backend. Stop if the post-migration plan is not clean.
|
|
145
|
+
|
|
146
|
+
```sh
|
|
147
|
+
# Move that same local state to its new, versioned GCS bucket.
|
|
148
|
+
GCP_TF_STATE_BUCKET=$(tofu -chdir="$TF_DIR" output -raw gcp_tf_state_bucket)
|
|
149
|
+
GCP_PROJECT_ID=$(tofu -chdir="$TF_DIR" output -raw gcp_project_id)
|
|
150
|
+
cp "$TF_DIR/terraform.tfstate" "$TF_DIR/terraform.tfstate.pre-gcs"
|
|
151
|
+
rm "$TF_DIR/backend_override.tf"
|
|
152
|
+
tofu -chdir="$TF_DIR" init -migrate-state -lockfile=readonly \
|
|
153
|
+
-backend-config="bucket=$GCP_TF_STATE_BUCKET" \
|
|
154
|
+
-backend-config="prefix=repository/prod"
|
|
155
|
+
tofu -chdir="$TF_DIR" state list
|
|
156
|
+
tofu -chdir="$TF_DIR" plan # expect no changes; keep the ignored backup securely until verified
|
|
157
|
+
unset GITHUB_TOKEN # gh run watch cannot use the Terraform fine-grained PAT
|
|
158
|
+
unset REST_API_BEARER
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
## 4. CI configuration
|
|
162
|
+
|
|
163
|
+
Verify main/prod restrictions, existing WIF/IAM rules and the Namespace runner.
|
|
164
|
+
Confirm `CACHIX_AUTH_TOKEN` is available to the protected workflow steps; do not
|
|
165
|
+
print or recreate existing secrets. Approve variable and secret changes before
|
|
166
|
+
running the following commands.
|
|
167
|
+
|
|
168
|
+
```sh
|
|
169
|
+
# Provision nonsecret repository variables for CI.
|
|
170
|
+
: "${GH_REPO:?use the same verified owner/repo as the local seed}"
|
|
171
|
+
gh variable set GCP_WIF_POOL --repo "$GH_REPO" --body "$GCP_WIF_POOL"
|
|
172
|
+
gh variable set GCP_WIF_PROVIDER --repo "$GH_REPO" --body "$GCP_WIF_PROVIDER"
|
|
173
|
+
gh variable set GCP_PROJECT_ID --repo "$GH_REPO" --body "$GCP_PROJECT_ID"
|
|
174
|
+
gh variable set GCP_TF_STATE_BUCKET --repo "$GH_REPO" --body "$GCP_TF_STATE_BUCKET"
|
|
175
|
+
|
|
176
|
+
# Verify prod permits ONLY main, not tags, before adding secrets.
|
|
177
|
+
printf '%s' "$GH_REPO_TOKEN" | gh secret set GH_REPO_TOKEN --repo "$GH_REPO" --env prod
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## 5. CI handoff
|
|
182
|
+
|
|
183
|
+
Read [Plan and Apply](../../deploy/references/plan-apply.md). The helper below
|
|
184
|
+
watches a single dispatch; an ambiguous result is not permission to dispatch
|
|
185
|
+
again. It forwards workflow inputs so Apply can select its saved-plan reference.
|
|
186
|
+
|
|
187
|
+
```sh
|
|
188
|
+
# Both workflows are published but manual-only. Dispatch only after prod is
|
|
189
|
+
# protected, populated, and the remote-state plan above is clean.
|
|
190
|
+
run_and_verify() {
|
|
191
|
+
output=$(gh workflow run "$@" --repo "$GH_REPO" --ref main 2>&1)
|
|
192
|
+
printf '%s\n' "$output"
|
|
193
|
+
url=$(printf '%s\n' "$output" |
|
|
194
|
+
grep -Eo 'https://github.com/[^[:space:]]+/actions/runs/[0-9]+' | head -1) || true
|
|
195
|
+
: "${url:?GitHub did not return a run URL; inspect the dispatched run manually}"
|
|
196
|
+
run_id=${url##*/}
|
|
197
|
+
gh run watch "$run_id" --repo "$GH_REPO" --exit-status
|
|
198
|
+
test "$(gh run view "$run_id" --repo "$GH_REPO" \
|
|
199
|
+
--json conclusion --jq .conclusion)" = success
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
# Review a successful CI Plan before dispatching CI Apply.
|
|
203
|
+
run_and_verify plan.yaml -f tf_layer=infrastructure
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Download and inspect the GCS plan using the deploy reference. Set `GH_PLAN_REF` to
|
|
207
|
+
its complete `RUN_ID/ATTEMPT/GENERATION` value from the Plan summary, verify its
|
|
208
|
+
exact producing attempt and obtain separate approval for Apply. Preserve the
|
|
209
|
+
same reference after review. A successful Plan is not Apply authorization.
|
|
210
|
+
|
|
211
|
+
```sh
|
|
212
|
+
: "${GH_PLAN_REF:?select the exact reviewed run/attempt/generation}"
|
|
213
|
+
[[ "$GH_PLAN_REF" =~ ^[1-9][0-9]*/[1-9][0-9]*/[1-9][0-9]*$ ]]
|
|
214
|
+
read -r -p 'Type APPLY after reviewing the successful CI Plan: ' answer </dev/tty
|
|
215
|
+
test "$answer" = APPLY
|
|
216
|
+
run_and_verify apply.yaml -f tf_layer=infrastructure -f "gh_plan_ref=$GH_PLAN_REF"
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
## 6. Remove bootstrap access
|
|
220
|
+
|
|
221
|
+
After successful CI handoff, verify WIF and Q0-admin recovery access, and the
|
|
222
|
+
Terraform-managed plan-reader bindings derived from `project_owners[].gcp_email`.
|
|
223
|
+
These grant object
|
|
224
|
+
read access only under `plans/`, not state access or Owner. Verify the operator's
|
|
225
|
+
Google account is included before removing bootstrap access. If read access is
|
|
226
|
+
missing, stop for a separately approved Terraform change; do not retain or
|
|
227
|
+
restore Owner merely to inspect plans.
|
|
228
|
+
|
|
229
|
+
Inspect IAM first, then obtain separate approval to remove only the bootstrap
|
|
230
|
+
human's direct, unconditional Owner grant. Preserve standing folder
|
|
231
|
+
`projectCreator`, billing `billing.user`, CI/WIF and recovery grants.
|
|
232
|
+
|
|
233
|
+
```sh
|
|
234
|
+
# After both CI runs succeed, inspect project IAM. Remove ONLY a direct
|
|
235
|
+
# roles/owner grant to the Day Zero human, if project creation gave them one;
|
|
236
|
+
# keep the standing folder projectCreator and billing.user domain grants.
|
|
237
|
+
creator=$(gcloud config get-value account)
|
|
238
|
+
: "${creator:?use the same Google account for gcloud and Terraform ADC}"
|
|
239
|
+
|
|
240
|
+
# Confirm this active gcloud account is the same human who used ADC above.
|
|
241
|
+
gcloud projects get-iam-policy "$GCP_PROJECT_ID" --format=json
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
Only after the narrow IAM removal is approved:
|
|
245
|
+
|
|
246
|
+
```sh
|
|
247
|
+
if gcloud projects get-iam-policy "$GCP_PROJECT_ID" --format=json |
|
|
248
|
+
jq -e --arg member "user:$creator" \
|
|
249
|
+
'any(.bindings[]?; .role == "roles/owner" and (.members | index($member)))' >/dev/null; then
|
|
250
|
+
gcloud projects remove-iam-policy-binding "$GCP_PROJECT_ID" \
|
|
251
|
+
--member="user:$creator" --role=roles/owner --condition=None
|
|
252
|
+
fi
|
|
253
|
+
# Recheck IAM and CI Plan after handoff. Preserve approved recovery admins.
|
|
254
|
+
# Never commit credentials, local state, .terraform/, or saved plans.
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
Reinspect IAM and verify private plan read access. Separately authorize a new CI
|
|
258
|
+
Plan to confirm the handoff; do not infer permission to Apply it.
|
|
259
|
+
|
|
260
|
+
```sh
|
|
261
|
+
gcloud projects get-iam-policy "$GCP_PROJECT_ID" --format=json
|
|
262
|
+
# After separate approval:
|
|
263
|
+
run_and_verify plan.yaml -f tf_layer=infrastructure
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Keep private backups until state and recovery are verified. Remove only your own
|
|
267
|
+
no-longer-needed local overrides and scratch; never commit them.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: deploy
|
|
3
|
+
description: Plan, privately review, apply or troubleshoot existing Factory-module infrastructure through protected reusable workflows and an exact saved-plan GCS generation. Not a new-installation bootstrap.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Deploy
|
|
7
|
+
|
|
8
|
+
Read [the configuration contract](references/configuration.md) and
|
|
9
|
+
[the full Plan and Apply procedure](references/plan-apply.md). Work from
|
|
10
|
+
the consuming repository root in its pinned direnv shell. Read `AGENTS.md`,
|
|
11
|
+
`.pi/memory/MEMORY.md`, the current Terraform root, its Factory module
|
|
12
|
+
references, the consumer's shell and task scripts, and the pinned caller **and** called workflows.
|
|
13
|
+
Use the repository task/app's mapping of the workflow's logical `--tf-layer` as
|
|
14
|
+
the reviewed `TF_DIR` for local inspection. Directory and backend-prefix mappings
|
|
15
|
+
belong to the repository, not the shared workflow.
|
|
16
|
+
|
|
17
|
+
Before planning, review the concrete root's source-owned repository identity,
|
|
18
|
+
paired owners/members, billing account, parent folder and bucket location. Adjust
|
|
19
|
+
source only for the intended installation or explicitly reviewed change; runtime
|
|
20
|
+
variables do not override these values. Existing-installation edits may change
|
|
21
|
+
access or replace resources, so inspect those effects rather than copying defaults.
|
|
22
|
+
|
|
23
|
+
Skill invocation, a source merge, a green Check or a successful Plan is not
|
|
24
|
+
authorization to Apply. Obtain separate approval for each dispatch, credential
|
|
25
|
+
operation, state repair or retry. Preserve main/prod restrictions, WIF,
|
|
26
|
+
backend locks, shared production concurrency and deletion protection.
|
|
27
|
+
|
|
28
|
+
Never reseed existing infrastructure or use local Apply to bypass CI. For a
|
|
29
|
+
confirmed new installation use [Day Zero](../day-zero-deploy/SKILL.md). Keep
|
|
30
|
+
credentials, state, plans and provider payloads out of Git and public logs.
|
|
31
|
+
|
|
32
|
+
On failure inspect the exact run and affected state before recovery. A failed
|
|
33
|
+
or timed-out Apply may have changed resources. Do not replay uncertain writes,
|
|
34
|
+
disable locking, cancel another operator's job or widen permissions.
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# Project configuration contract
|
|
2
|
+
|
|
3
|
+
## Operator settings
|
|
4
|
+
|
|
5
|
+
Use the consuming repository's pinned shell and deployment procedures. These
|
|
6
|
+
settings describe the current Factory contract; verify the consumer's configuration.
|
|
7
|
+
A source change, successful Check or Plan never authorizes a deployment.
|
|
8
|
+
|
|
9
|
+
| Setting | Value and purpose |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| `GH_REPO` | Verified `owner/repo` for the GitHub CLI; not a basename or numeric ID. |
|
|
12
|
+
| `TF_DIR` | Reviewed Terraform root path: `tf/infrastructure` for Factory, or the consumer's selected root. |
|
|
13
|
+
| `TF_LAYER` | Offered workflow layer: `infrastructure` or `application`; passed as `tf_layer`. |
|
|
14
|
+
| `GCP_WIF_POOL` | Trusted `projects/NUMBER/locations/global/workloadIdentityPools/POOL` resource path. |
|
|
15
|
+
| `GCP_WIF_PROVIDER` | Trusted pool path followed by `/providers/PROVIDER`, used by CI authentication. |
|
|
16
|
+
| `GH_PLAN_REF` | One complete `RUN_ID/ATTEMPT/GENERATION` value from the successful Plan summary; three positive decimal integers without leading zeros. |
|
|
17
|
+
|
|
18
|
+
The concrete Terraform root owns reviewed `project_owners` and `project_members`
|
|
19
|
+
locals. They are source configuration, not CLI/environment inputs or Actions
|
|
20
|
+
owner rosters. WIF values remain Actions configuration variables; Day Zero supplies
|
|
21
|
+
WIF at runtime before CI exists. `GH_REPO`, `TF_DIR`, `TF_LAYER` and `GH_PLAN_REF`
|
|
22
|
+
are procedure inputs, not additional standing repository variables. The workflow
|
|
23
|
+
bridges `gh_plan_ref` through a step-local `GH_PLAN_REF` variable before passing
|
|
24
|
+
it as a quoted `--gh-plan-ref` argument. Trusted repository variables go directly
|
|
25
|
+
to `--gcp-tf-state-bucket` and `--gcp-wif-pool`. Only the credential uses Commander
|
|
26
|
+
`.env()`; the plan-reference bridge keeps input data out of shell source. Native Actions repository/run/attempt IDs,
|
|
27
|
+
runner scratch and summary paths are read directly, not repeated as CLI flags.
|
|
28
|
+
Shared templates offer both layer jobs; `tf_layer` selects one per dispatch and
|
|
29
|
+
is passed as `--tf-layer` to the repository's pnpm task/app. That repository maps
|
|
30
|
+
the layer internally to its directory and existing backend prefix. Shared YAML
|
|
31
|
+
contains neither directory nor backend-prefix arguments. Set
|
|
32
|
+
`github.workflows.layers` to the supported subset. Factory selects Infrastructure
|
|
33
|
+
only, mapped by its app to `tf/infrastructure` and `repository/prod`. Retain each
|
|
34
|
+
consumer's existing state identity. Plan and Apply must select the same layer.
|
|
35
|
+
|
|
36
|
+
A synthetic owner-roster example, not identities to deploy:
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
[{"gcp_email":"owner@example.com","gh_username":"example-owner"}]
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Confirm every pairing. Emails and usernames must each be unique across owners
|
|
43
|
+
and members, ignoring case. Do not infer usernames from emails. Preserve spelling
|
|
44
|
+
for existing Terraform instance keys.
|
|
45
|
+
|
|
46
|
+
### Credentials stay separate
|
|
47
|
+
|
|
48
|
+
`GH_REPO_TOKEN` is the repository-scoped provider credential. Use this same name
|
|
49
|
+
for the protected environment secret, workflow environment and Commander `.env()`
|
|
50
|
+
option (`--gh-repo-token`). Workflows prefer environment-based credentials; users
|
|
51
|
+
may choose flags. Only the provider subprocess receives the internal native
|
|
52
|
+
`GITHUB_TOKEN` and `REST_API_BEARER` mappings. The storage subprocess receives
|
|
53
|
+
neither. Keep organization credentials distinct as `GH_ORG_TOKEN` or an
|
|
54
|
+
organization-qualified name such as `GH_QUANTILE_CO_ORG_TOKEN`; do not combine
|
|
55
|
+
privileges to standardize names. The operator's GitHub CLI login remains separate.
|
|
56
|
+
`CACHIX_AUTH_TOKEN` remains a separate workflow secret. Google CLI login, ADC
|
|
57
|
+
and CI WIF are distinct authentication contexts. Keep credentials out of Git,
|
|
58
|
+
Terraform inputs, Nix values and public logs.
|
|
59
|
+
|
|
60
|
+
### Concrete values are not overrides
|
|
61
|
+
|
|
62
|
+
Factory's `tf/infrastructure/main.tf` holds its approved billing account, parent
|
|
63
|
+
folder and bucket location. Billing/folder IDs identify resources; they do not
|
|
64
|
+
authorize access or charges. Publishing them still reveals account/hierarchy
|
|
65
|
+
metadata. Verify the deployed values and approve publication for each concrete
|
|
66
|
+
root; never copy private consumer configuration into public Factory.
|
|
67
|
+
|
|
68
|
+
Shared workflows and Day Zero do not supply billing/folder environment
|
|
69
|
+
variables. Reusable GCP module parameters `billing_account`, `folder_id` and
|
|
70
|
+
`tf_state_location` remain configurable for other roots. A new repository chooses
|
|
71
|
+
its own approved values; it does not inherit Factory's installation identity.
|
|
72
|
+
|
|
73
|
+
## Generated identifiers
|
|
74
|
+
|
|
75
|
+
Do not invent new identifiers for an existing installation. Record generated
|
|
76
|
+
project/bucket values as CI variables only after verifying the intended resources.
|
|
77
|
+
|
|
78
|
+
| Source | Consumer or meaning |
|
|
79
|
+
| --- | --- |
|
|
80
|
+
| `module.gcp.project_id` → root `gcp_project_id` | `GCP_PROJECT_ID` for CI authentication. |
|
|
81
|
+
| `module.gcp.project_number` | Numeric GCP project identity; distinct from its string project ID. |
|
|
82
|
+
| `module.gcp.tf_state_bucket` → root `gcp_tf_state_bucket` | `GCP_TF_STATE_BUCKET`, the existing backend and saved-plan bucket. |
|
|
83
|
+
| `module.gh.repo_id` → root `gh_repo_id` | Immutable numeric GitHub ID, also passed to GCP as `gh_repo_id`. |
|
|
84
|
+
| `module.gh.repo_name` | Repository basename within the configured provider owner. |
|
|
85
|
+
| `module.gh.owner_ids` | Owner usernames mapped to immutable GitHub actor IDs. |
|
|
86
|
+
|
|
87
|
+
`repo_name` is also the GitHub module's basename input. It is not `GH_REPO`.
|
|
88
|
+
Retain `tf_state` on the bucket: it identifies purpose, not a transport mechanism.
|
|
89
|
+
Keep role qualifiers where multiple projects coexist; do not collapse distinct
|
|
90
|
+
project identities merely to shorten names.
|
|
91
|
+
|
|
92
|
+
## Platform roles and native adapters
|
|
93
|
+
|
|
94
|
+
The Terraform root owns `local.project_owners` and `local.project_members`. It directly
|
|
95
|
+
projects emails into the GCP module and usernames into the GitHub module; there
|
|
96
|
+
is no intermediary identity module. Each platform module owns its permissions:
|
|
97
|
+
|
|
98
|
+
- GitHub owners are looked up for `owner_ids`; the lookup itself grants no
|
|
99
|
+
repository or organization membership. Members receive repository write
|
|
100
|
+
access (`push`), not administration.
|
|
101
|
+
- GCP owners receive object-read access only under `plans/`, not state access.
|
|
102
|
+
`gh_repo_id` and `gh_owner_ids` retain main/repository/actor-scoped WIF.
|
|
103
|
+
There is no direct human project Owner grant.
|
|
104
|
+
- Members receive no added GCP or WIF grants.
|
|
105
|
+
Platform modules reject overlap between owner and member identities.
|
|
106
|
+
|
|
107
|
+
The workflow CLI neither reads nor forwards owner/member rosters. Terraform
|
|
108
|
+
projects the source records directly into its platform modules. YAML has no owner
|
|
109
|
+
or triggering-actor list checks; dispatch access follows repository/environment
|
|
110
|
+
policy, while the existing WIF/IAM configuration restricts GCP access. Review parses
|
|
111
|
+
`GH_PLAN_REF` internally and selects its exact GCS generation; Apply receives the
|
|
112
|
+
same reference without replanning. Review and Apply require separate approval.
|
|
113
|
+
|
|
114
|
+
These grants do not revoke inherited organization, team or GCP permissions.
|
|
115
|
+
Review effective access, including collaborator removal when a member leaves
|
|
116
|
+
or changes roles. Repository writers remain within the existing branch-protection
|
|
117
|
+
and trusted-workflow model.
|
|
118
|
+
|
|
119
|
+
Use `GH_` / `gh_` and `GCP_` / `gcp_` for unscoped Quantile-owned names. Keep
|
|
120
|
+
`TF_VAR_*`, backend arguments and local plan paths inside adapters or exceptional
|
|
121
|
+
procedures. Preserve native provider/SDK attributes and variables, including
|
|
122
|
+
`repository_id`, `GITHUB_*`, `REST_API_BEARER` and
|
|
123
|
+
`GOOGLE_APPLICATION_CREDENTIALS`. Keep `GH_REPO` unset during independent checkout
|
|
124
|
+
verification, and explicit repository targets on mutations.
|
|
125
|
+
|
|
126
|
+
Same-named `STRIPE_API_KEY` or `RESEND_API_KEY` inputs in isolated contexts need
|
|
127
|
+
not have equal values or privileges. Qualify credentials only when they coexist;
|
|
128
|
+
never combine privileges to simplify names. There is no member GCP/admin-role
|
|
129
|
+
override or supported local-production Apply shortcut.
|
|
130
|
+
|
|
131
|
+
## Tool configuration files
|
|
132
|
+
|
|
133
|
+
Actionlint, clang-format and clang-tidy use native configuration discovery.
|
|
134
|
+
Other wrappers retain explicit paths where they inspect settings or change
|
|
135
|
+
working roots. Retained selectors use `<TOOL>_CONFIG_FILE`; upstream-native
|
|
136
|
+
names stay unchanged. `PREK_CONFIG_FILE` is shared by the Prek and keep-sorted
|
|
137
|
+
adapters. Nix `configFile` options select native templates, not a second policy
|
|
138
|
+
registry. Materialize files into their native locations, not runtime `etc/` paths.
|
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# Plan and Apply
|
|
2
|
+
|
|
3
|
+
Read the caller's current workflows, the pinned Factory reusable workflows,
|
|
4
|
+
the consumer's shell and task scripts, its Terraform root and declared Factory
|
|
5
|
+
Terraform module sources, and `nix/github/workflows/default.nix` from the pinned
|
|
6
|
+
Factory revision, plus its importable `packages/github` transport and thin
|
|
7
|
+
`apps/github` CLI. Its package tasks run TypeScript directly with locked `tsx`,
|
|
8
|
+
without a GitHub app/library build. Workflows run `pnpm install --frozen-lockfile`
|
|
9
|
+
immediately before the task in the same run block. Automatic installation is
|
|
10
|
+
disabled in the workspace. The credential uses
|
|
11
|
+
Commander `.env("GH_REPO_TOKEN")`; other inputs are quoted CLI arguments, with
|
|
12
|
+
native provider credential names mapped internally. Native Actions metadata needs no repeated
|
|
13
|
+
flags. Consumers own their task implementations and dependency delivery. Export local
|
|
14
|
+
`TF_DIR` for inspection/exception commands below from the repository task/app's
|
|
15
|
+
mapping of the workflow's logical `--tf-layer`. The repository owns its directory
|
|
16
|
+
and existing backend prefix, not the shared YAML.
|
|
17
|
+
The library gives custom callbacks the exact saved-plan path directly rather than
|
|
18
|
+
an ambient `TF_PLAN_FILE`.
|
|
19
|
+
Changing this source directory does not change the backend bucket or prefix.
|
|
20
|
+
Use the pinned direnv shell. Ordinary production changes use protected workflows,
|
|
21
|
+
not local Apply. Validate source without backend access:
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
pnpm run check
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Layout changes on an existing installation
|
|
28
|
+
|
|
29
|
+
For existing installations, compare the resource addresses in state with this
|
|
30
|
+
repository's modules and any `moved` blocks. Obtain separate
|
|
31
|
+
approval for any one-time address migration. Do not apply a plan that recreates
|
|
32
|
+
or deletes existing resources because of a refactor. When the original bucket
|
|
33
|
+
and existing prefix are retained, this is not a backend migration or a new
|
|
34
|
+
installation. Some installations also intentionally retire resources;
|
|
35
|
+
review those deletions individually. No compatibility blocks or state moves
|
|
36
|
+
run automatically.
|
|
37
|
+
|
|
38
|
+
## Local OpenTofu (exceptional, separately authorized)
|
|
39
|
+
|
|
40
|
+
Ordinary production Plan/Apply must use protected workflows. If an operator
|
|
41
|
+
explicitly authorizes a local exception, enter the pinned shell and supply
|
|
42
|
+
provider credentials at runtime from an approved source. Set
|
|
43
|
+
`TF_PLAN_FILE` to an absolute private path outside Git and the Nix store;
|
|
44
|
+
keep its bytes restricted for review. Use the existing bucket and prefix:
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
set -euo pipefail; set +x; umask 077
|
|
48
|
+
: "${GCP_TF_STATE_BUCKET:?Set the existing bucket}"
|
|
49
|
+
: "${TF_BACKEND_PREFIX:?Set the existing prefix}"
|
|
50
|
+
: "${TF_PLAN_FILE:?Set an absolute private plan path}"
|
|
51
|
+
case "$TF_PLAN_FILE" in /*) ;; *) exit 1 ;; esac
|
|
52
|
+
: "${TF_DIR:?Set the reviewed Terraform root}"
|
|
53
|
+
tofu -chdir="$TF_DIR" init -input=false -lockfile=readonly \
|
|
54
|
+
-backend-config="bucket=$GCP_TF_STATE_BUCKET" \
|
|
55
|
+
-backend-config="prefix=$TF_BACKEND_PREFIX"
|
|
56
|
+
test ! -e "$TF_PLAN_FILE"
|
|
57
|
+
tofu -chdir="$TF_DIR" plan -input=false -no-color -out="$TF_PLAN_FILE"
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
A *separately approved* local Apply uses the exact reviewed saved bytes, with
|
|
61
|
+
fresh runtime credentials and the same backend configuration. Never replan
|
|
62
|
+
in place of Apply:
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
set -euo pipefail; set +x; umask 077
|
|
66
|
+
: "${GCP_TF_STATE_BUCKET:?Set the reviewed bucket}"
|
|
67
|
+
: "${TF_BACKEND_PREFIX:?Set the reviewed prefix}"
|
|
68
|
+
: "${TF_PLAN_FILE:?Set the reviewed absolute private plan path}"
|
|
69
|
+
case "$TF_PLAN_FILE" in /*) ;; *) exit 1 ;; esac
|
|
70
|
+
test -s "$TF_PLAN_FILE"
|
|
71
|
+
: "${TF_DIR:?Set the reviewed Terraform root}"
|
|
72
|
+
tofu -chdir="$TF_DIR" init -input=false -lockfile=readonly \
|
|
73
|
+
-backend-config="bucket=$GCP_TF_STATE_BUCKET" \
|
|
74
|
+
-backend-config="prefix=$TF_BACKEND_PREFIX"
|
|
75
|
+
tofu -chdir="$TF_DIR" apply -input=false "$TF_PLAN_FILE"
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## 1. Confirm context
|
|
79
|
+
|
|
80
|
+
From a clean checkout of the current protected main revision:
|
|
81
|
+
|
|
82
|
+
```sh
|
|
83
|
+
set -euo pipefail
|
|
84
|
+
set +x
|
|
85
|
+
umask 077
|
|
86
|
+
: "${TF_DIR:?Set the reviewed Terraform root}"
|
|
87
|
+
: "${TF_LAYER:?Select an offered infrastructure or application layer}"
|
|
88
|
+
: "${GH_REPO:?Set and verify the intended owner/repo before any dispatch}"
|
|
89
|
+
repo=$(env -u GH_REPO gh repo view --json nameWithOwner --jq .nameWithOwner)
|
|
90
|
+
test "$GH_REPO" = "$repo"
|
|
91
|
+
SHA=$(git rev-parse HEAD)
|
|
92
|
+
test -z "$(git status --porcelain)"
|
|
93
|
+
test "$SHA" = "$(gh api "repos/$GH_REPO/git/ref/heads/main" --jq .object.sha)"
|
|
94
|
+
gh run list -R "$GH_REPO" -w check.yaml -c "$SHA" \
|
|
95
|
+
--json databaseId,headSha,status,conclusion,url
|
|
96
|
+
gh run list -R "$GH_REPO" --limit 10 \
|
|
97
|
+
--json databaseId,workflowName,headSha,status,conclusion,url
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Check may have run on the source branch before fast-forwarding main. Confirm its
|
|
101
|
+
exact SHA, not just the branch label. Inspect current production activity; never
|
|
102
|
+
cancel another operator's job. Verify main/prod restrictions, WIF and IAM,
|
|
103
|
+
the effective state bucket and backend prefix, including caller inputs and runtime
|
|
104
|
+
environment overrides. Factory defaults to `repository/prod`; consumers may retain
|
|
105
|
+
another existing prefix. Check the called workflow's `factory-production`
|
|
106
|
+
concurrency and any caller guard.
|
|
107
|
+
The Terraform root's reviewed paired records are the source of truth for platform
|
|
108
|
+
permissions, including plan-reader emails and immutable deployment actor IDs.
|
|
109
|
+
Neither YAML nor the CLI reads an Actions owner roster. The YAML owner and
|
|
110
|
+
triggering-actor conditions are intentionally absent; repository/environment
|
|
111
|
+
policy and the existing WIF/IAM rules govern access.
|
|
112
|
+
The GCP module grants humans object-read access only under `plans/`, not direct
|
|
113
|
+
project Owner access. Members must not be substituted into owner grants.
|
|
114
|
+
See the [configuration contract](configuration.md) for identity and role requirements.
|
|
115
|
+
The operator's GitHub CLI login is separate from the provider PAT;
|
|
116
|
+
do not let `GITHUB_TOKEN` or `GH_TOKEN` select the wrong credential.
|
|
117
|
+
|
|
118
|
+
Plan and Apply receive provider credentials and the Cachix token only at runtime.
|
|
119
|
+
Never copy those values into Terraform inputs, artifacts, source or Nix values.
|
|
120
|
+
|
|
121
|
+
## 2. Plan
|
|
122
|
+
|
|
123
|
+
After approval for this dispatch only:
|
|
124
|
+
|
|
125
|
+
```sh
|
|
126
|
+
gh workflow run plan.yaml -R "$GH_REPO" --ref main -f "tf_layer=$TF_LAYER"
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Capture its run ID and attempt from the result; an ambiguous dispatch is not
|
|
130
|
+
permission to dispatch again. Watch that exact run and require success, including
|
|
131
|
+
post-steps. Set `GH_PLAN_REF` to the complete `RUN_ID/ATTEMPT/GENERATION` value
|
|
132
|
+
from the Plan step summary; do not reconstruct it from separately chosen parts.
|
|
133
|
+
|
|
134
|
+
No plan or diagnostic is uploaded to GitHub. The saved plan and its private
|
|
135
|
+
Plan log stay in GCS under `plans/REPOSITORY_ID/RUN_ID/ATTEMPT/saved-plan/`.
|
|
136
|
+
An authorized reader needs access to those objects independently of CI's WIF
|
|
137
|
+
identity. Manage missing read access through the reviewed project role policy and
|
|
138
|
+
a separately approved Terraform Apply, not a manual IAM grant, Owner access or
|
|
139
|
+
public upload. Do not promote someone to a project owner solely to enable review.
|
|
140
|
+
An existing authorized reader must inspect the initial access-grant plan; setting
|
|
141
|
+
the input alone grants nothing.
|
|
142
|
+
|
|
143
|
+
## 3. Inspect the plan privately
|
|
144
|
+
|
|
145
|
+
Follow [the saved-plan review procedure](saved-plan-review.md) in full using
|
|
146
|
+
`GH_PLAN_REF`. It parses the components internally and verifies the producing
|
|
147
|
+
attempt; confirm its source and effective bucket before reviewing the plan.
|
|
148
|
+
Preserve that exact reference for the separately approved Apply; review does
|
|
149
|
+
not authorize it.
|
|
150
|
+
|
|
151
|
+
## 4. Apply
|
|
152
|
+
|
|
153
|
+
After separate approval for this exact run/attempt/generation:
|
|
154
|
+
|
|
155
|
+
```sh
|
|
156
|
+
: "${GH_PLAN_REF:?Set the exact reviewed run/attempt/generation}"
|
|
157
|
+
[[ "$GH_PLAN_REF" =~ ^[1-9][0-9]*/[1-9][0-9]*/[1-9][0-9]*$ ]]
|
|
158
|
+
gh workflow run apply.yaml -R "$GH_REPO" --ref main \
|
|
159
|
+
-f "tf_layer=$TF_LAYER" -f "gh_plan_ref=$GH_PLAN_REF"
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Select the same layer as the reviewed Plan. Each dispatch runs only that layer's
|
|
163
|
+
job, even when both jobs are available. If Infrastructure changes inputs consumed
|
|
164
|
+
by Application, apply Infrastructure first, then create and review a new
|
|
165
|
+
Application Plan; do not reuse the earlier Application plan.
|
|
166
|
+
|
|
167
|
+
Apply downloads that exact GCS plan generation into a local file, then runs
|
|
168
|
+
OpenTofu Apply without replanning. The protected Apply gate and the operator's
|
|
169
|
+
review of the selected reference are the authorization boundary. OpenTofu's
|
|
170
|
+
backend state locking and stale-plan checks remain active. If the source has
|
|
171
|
+
changed, the plan is missing, or the state has advanced, stop and review a new
|
|
172
|
+
Plan rather than applying a different object. GCS lifecycle cleanup is
|
|
173
|
+
asynchronous; do not assume old objects have already been deleted.
|
|
174
|
+
|
|
175
|
+
Watch the exact Apply run. A failed Plan stores a create-only `plan.log` under
|
|
176
|
+
`gs://$GCP_TF_STATE_BUCKET/plans/$GITHUB_REPOSITORY_ID/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/`.
|
|
177
|
+
A failed Apply stores `apply.log` under
|
|
178
|
+
`gs://$GCP_TF_STATE_BUCKET/plans/$GITHUB_REPOSITORY_ID/apply/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/`.
|
|
179
|
+
Fetch diagnostics only as an authorized reader into restricted scratch outside Git
|
|
180
|
+
and the Nix store. Provider output is redirected to local private logs, not
|
|
181
|
+
printed in the workflow. Diagnostic upload can
|
|
182
|
+
itself fail; absence is not proof that no infrastructure changed. GCS cleanup
|
|
183
|
+
follows the plans-prefix lifecycle.
|
|
184
|
+
|
|
185
|
+
Failure or timeout may follow a partial write: inspect state and completed
|
|
186
|
+
operations before proposing recovery. Do not rerun Day Zero, force-unlock an
|
|
187
|
+
active backend, replay an uncertain Apply or run workflows solely to warm caches.
|
|
188
|
+
Remove only your own private scratch after verification; preserve uncertain
|
|
189
|
+
results and backups securely until recovery is understood.
|