gowalk-cicd 1.0.168 → 1.0.170

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. package/README.md +25 -13
  2. package/action/scripts/mobile_artifact_files.py +14 -3
  3. package/action/scripts/mobile_artifact_inventory.cjs +32 -19
  4. package/action/scripts/version_utils.py +17 -7
  5. package/android-action/scripts/android_config.py +37 -0
  6. package/backend-action/action.yml +3 -38
  7. package/backend-action/configure_vhost.py +39 -21
  8. package/backend-action/edge_vhost.py +7 -7
  9. package/backend-action/ensure_certificate.sh +47 -123
  10. package/backend-action/remote_deploy.sh +26 -23
  11. package/bin/cli.mjs +1 -0
  12. package/docs/CANDIDATE_ARTIFACTS.md +14 -3
  13. package/docs/CONFIGURATION.md +4 -2
  14. package/docs/GRADLE_CACHE.md +14 -1
  15. package/package.json +1 -2
  16. package/src/config/merge.mjs +29 -1
  17. package/src/config/workflows.mjs +23 -12
  18. package/templates/deploy-backend.yml +2 -8
  19. package/templates/deploy.yml +27 -4
  20. package/templates/history.json.gz +0 -0
  21. package/templates/warm-android-cache.yml +256 -0
  22. package/backend-action/network_contract.py +0 -119
  23. package/backend-action/network_diagnostics.py +0 -182
  24. package/backend-action/network_inventory.py +0 -235
  25. package/backend-action/network_owned_inventory.py +0 -118
  26. package/backend-action/network_pool_facts.json +0 -214
  27. package/backend-action/network_pools.py +0 -267
  28. package/backend-action/network_prepare.py +0 -152
  29. package/backend-action/network_provider.py +0 -213
  30. package/backend-action/network_readback.py +0 -31
  31. package/backend-action/network_runtime.py +0 -62
  32. package/backend-action/network_selected.py +0 -96
  33. package/backend-action/network_selected_receipt.py +0 -60
  34. package/docs/BACKEND_NETWORKS.md +0 -159
package/README.md CHANGED
@@ -23,6 +23,14 @@ Writes three core things into your repo:
23
23
  - `.github/actions/android-app/` — Android build/sign action (Flutter or Gradle)
24
24
  - `.github/workflows/deploy.yml` — workflow that builds and deploys both platforms
25
25
 
26
+ It also writes:
27
+
28
+ - `.github/workflows/mobile-candidate.yml` — shipping binaries for ready task PRs
29
+ (dispatch-only while candidate artifacts are off)
30
+ - `.github/workflows/warm-android-cache.yml` — only while Android candidates are enabled: after this
31
+ app's successful default-branch release that includes Android, compiles the Android bundle once,
32
+ unsigned, when main has no Gradle cache for it ([Gradle dependency retention](docs/GRADLE_CACHE.md))
33
+
26
34
  The installer no longer adds `.github/workflows/deploy-web.yml`. On re-adoption it
27
35
  removes a `deploy-web.yml` that is still byte-for-byte the generated file and has no
28
36
  overrides in `.gowalk-cicd.yml`; a customized one keeps upgrading (see
@@ -183,10 +191,6 @@ in App Store Connect; see [First-time app setup](#first-time-app-setup).
183
191
 
184
192
  ## Backend deploy (Python + Postgres, Docker)
185
193
 
186
- New ordinary Compose projects receive small isolated networks inside Docker's verified existing pools.
187
- Existing/custom/external networks and authored IPAM retain their contracts. Read-only
188
- `network-diagnostics` uses the same deployment identity; see [capacity and diagnosis](docs/BACKEND_NETWORKS.md).
189
-
190
194
  When a repo carries a Dockerized backend, the installer also vendors a backend deploy path:
191
195
  `.github/actions/backend-app/` and `.github/workflows/deploy-backend.yml`. A
192
196
  mobile-only repo is unaffected (the backend path is not installed).
@@ -231,14 +235,21 @@ and a domain deploy fails unless its public HTTPS certificate and health route v
231
235
  Changing a container's published port preserves its existing HTTPS configuration; nginx must accept
232
236
  the change, or the previous vhost is restored and the deploy fails.
233
237
  When the domain's vhost has exactly one `ssl_certificate`, and that file exists, stays valid for more
234
- than a day and lists `DNS:<domain>` in its subjectAltName, the deploy runs neither Certbot nor an nginx
235
- reload and logs `certificate state=current`. Otherwise an existing certificate is installed for the
236
- domain (repairing older HTTP-only vhosts too) or a new one is issued, both with `certbot --nginx`.
237
- Every Certbot call runs under `flock -w 900 /run/gowalk-certbot.lock`, so deploys on one host take
238
- turns; when the host's renew timer holds Certbot's own lock, the call retries every 10 s for up to
239
- 10 minutes. Vhost writes with their nginx test and reload, and the host port reservation, run under
240
- the host nginx lock `/run/app-robot-nginx.lock` (`flock -w 120`), which the panel's domain applier also
241
- takes. After a healthy deploy the replaced build's images are pruned
238
+ than a day and lists `DNS:<domain>` in its subjectAltName, the deploy runs no Certbot, only follows the
239
+ published port, and logs `certificate state=current`. Otherwise the name is issued by webroot: the vhost
240
+ serves `/.well-known/acme-challenge/` from `/var/lib/gowalk-acme` on port 80 (and proxies the API there
241
+ until HTTPS exists); the deploy waits up to 20 s for this host's nginx to serve a probe file under the
242
+ domain's name; `certbot certonly --webroot --cert-name <domain> --keep-until-expiring` issues it (a lineage
243
+ of that name that is not due is kept); then the vhost is rendered with HTTPS: port 80 answers challenges
244
+ and redirects, port 443 proxies with `/etc/letsencrypt/live/<domain>/`. Certbot saves the webroot and a
245
+ deploy hook that reloads nginx under the host nginx lock, so the host's renew timer renews the lineage.
246
+ Vhosts that older releases configured with `certbot --nginx` keep their TLS directives; only their
247
+ upstream port changes. Every Certbot call runs under `flock -w 900 /run/gowalk-certbot.lock`, so deploys
248
+ on one host take turns; when the host's renew timer holds Certbot's own lock, the call retries every
249
+ 10 s for up to 10 minutes. Vhost writes with their nginx test and reload, and the host port reservation,
250
+ run under the host nginx lock `/run/app-robot-nginx.lock` (`flock -w 120`), which the panel's domain
251
+ applier also takes. New Compose networks come from Docker's own address pools; the deploy hosts'
252
+ `default-address-pools` hand out `/24`s. After a healthy deploy the replaced build's images are pruned
242
253
  (`docker image prune -f --filter label=com.docker.compose.project=<app>`); a refusal because another
243
254
  prune is running defers it to the next deploy.
244
255
 
@@ -851,7 +862,8 @@ is required. A completed CI run does not mean this deferred upload is complete.
851
862
 
852
863
  ### iOS delivery
853
864
 
854
- The iOS composite action runs on `macos-15` and:
865
+ The iOS composite action runs on `macos-15`; promoting a verified candidate IPA runs its first
866
+ attempt on `ubuntu-24.04` (see [Candidate artifacts](docs/CANDIDATE_ARTIFACTS.md)). It:
855
867
 
856
868
  GitHub Actions reaches Apple and Google **directly**. No store proxy is configured, read
857
869
  or inherited here: the account-pinned residential exit belongs to the panel and the runner
@@ -10,9 +10,12 @@ import shutil
10
10
  import sys
11
11
 
12
12
  SCHEMA = "gowalk-cicd/mobile-binary.v1"
13
+ # The source Info.plist travels checkout-relative, so a promotion on Linux (no xcodebuild) can
14
+ # persist a planned version into it. Receipts packed before it existed stay valid without it.
15
+ SOURCE_PLIST = "SWIFT_APP_SOURCE_INFOPLIST"
13
16
  IOS_ENV = ("CFG_BUNDLE_ID", "CFG_TEAM_ID", "CFG_APP_STORE_APPLE_ID", "CFG_CONFIGURATION",
14
17
  "CFG_PROJECT", "CFG_WORKSPACE", "CFG_SCHEME", "CFG_LOCALE", "CFG_USES_NON_EXEMPT",
15
- "CFG_WHATS_NEW", "MARKETING_VERSION", "BUILD_NUMBER", "SOURCE_MARKETING_VERSION")
18
+ "CFG_WHATS_NEW", "MARKETING_VERSION", "BUILD_NUMBER", "SOURCE_MARKETING_VERSION", SOURCE_PLIST)
16
19
  ANDROID_ENV = ("ANDROID_PACKAGE_NAME", "ANDROID_BUILD_NUMBER", "ANDROID_PROJECT_KIND")
17
20
 
18
21
 
@@ -69,13 +72,17 @@ def files(root: Path) -> dict:
69
72
  def metadata(platform: str) -> dict:
70
73
  event = json.loads(Path(os.environ["GITHUB_EVENT_PATH"]).read_text())
71
74
  keys = IOS_ENV if platform == "ios" else ANDROID_ENV
75
+ environment = {key: os.environ.get(key, "") for key in keys}
76
+ if environment.get(SOURCE_PLIST):
77
+ environment[SOURCE_PLIST] = os.path.relpath(Path(environment[SOURCE_PLIST]).resolve(),
78
+ Path(os.environ["GITHUB_WORKSPACE"]).resolve())
72
79
  return {"schema": SCHEMA, "platform": platform, "repository": os.environ["GITHUB_REPOSITORY"],
73
80
  "fingerprint": os.environ["MOBILE_ARTIFACT_FINGERPRINT"],
74
81
  "run_id": int(os.environ["GITHUB_RUN_ID"]), "run_attempt": int(os.environ["GITHUB_RUN_ATTEMPT"]),
75
82
  "source_sha": os.environ["GITHUB_SHA"],
76
83
  "head_sha": event.get("pull_request", {}).get("head", {}).get("sha", os.environ["GITHUB_SHA"]),
77
84
  "credential_fingerprints": credentials(platform),
78
- "environment": {key: os.environ.get(key, "") for key in keys}}
85
+ "environment": environment}
79
86
 
80
87
 
81
88
  def pack(platform: str, directory: Path) -> None:
@@ -115,7 +122,11 @@ def verify(platform: str, directory: Path) -> dict:
115
122
  raise ValueError("mobile_artifact_inputs_mismatch")
116
123
  expected = set(IOS_ENV if platform == "ios" else ANDROID_ENV)
117
124
  environment = value.get("environment", {})
118
- if set(environment) != expected or any(not isinstance(v, str) or len(v) > 16000 for v in environment.values()):
125
+ if set(environment) not in (expected, expected - {SOURCE_PLIST}) or any(
126
+ not isinstance(v, str) or len(v) > 16000 for v in environment.values()):
127
+ raise ValueError("mobile_artifact_environment_invalid")
128
+ plist = Path(environment.get(SOURCE_PLIST, ""))
129
+ if plist.is_absolute() or ".." in plist.parts:
119
130
  raise ValueError("mobile_artifact_environment_invalid")
120
131
  number = environment["BUILD_NUMBER" if platform == "ios" else "ANDROID_BUILD_NUMBER"]
121
132
  if not re.fullmatch(r"[1-9][0-9]{0,17}", number):
@@ -40,8 +40,8 @@ async function candidate(artifact) {
40
40
  return {...run, artifact_attempt: attempt};
41
41
  }
42
42
 
43
- async function restore(platform) {
44
- const deadline = Date.now() + 120000;
43
+ // The newest reusable candidate for this platform's current inputs, or null; reads only.
44
+ async function find(platform, deadline) {
45
45
  const fingerprint = identity(platform);
46
46
  const prefix = `mobile-binary-${platform}-${fingerprint}-`;
47
47
  const inventory = await api.request('GET', '/actions/artifacts?per_page=100', undefined, 5000);
@@ -49,23 +49,29 @@ async function restore(platform) {
49
49
  for (const artifact of inventory.artifacts.filter(row => row.name?.startsWith(prefix)).slice(0, 5)) {
50
50
  if (Date.now() >= deadline) break;
51
51
  const run = await candidate(artifact);
52
- if (!run || artifact.name !== `${prefix}${run.id}-${run.artifact_attempt}`) continue;
53
- const directory = mkdtempSync(join(process.env.RUNNER_TEMP, `mobile-binary-${platform}-`));
54
- execFileSync('gh', ['run', 'download', String(run.id), '--repo', process.env.GITHUB_REPOSITORY,
55
- '--name', artifact.name, '--dir', directory], {stdio: 'ignore', timeout: Math.max(1, deadline - Date.now())});
56
- const receipt = JSON.parse(readFileSync(join(directory, 'mobile-binary.json'), 'utf8'));
57
- if (receipt.run_id !== run.id || receipt.run_attempt !== run.artifact_attempt || receipt.head_sha !== run.head_sha) {
58
- throw Error('mobile_artifact_run_mismatch');
59
- }
60
- const operation = platform === 'ios' ? 'verify' : 'restore';
61
- execFileSync('python3', [join(__dirname, 'mobile_artifact_files.py'), operation, platform, directory],
62
- {stdio: ['ignore', 'inherit', 'pipe'], env: {...process.env, MOBILE_ARTIFACT_FINGERPRINT: fingerprint}});
63
- output('directory', directory);
64
- output('restored', 'true');
65
- console.log(`::notice::Reusing ${platform} binary from candidate run ${run.id}`);
66
- return;
52
+ if (run && artifact.name === `${prefix}${run.id}-${run.artifact_attempt}`) return {artifact, run, fingerprint};
53
+ }
54
+ return null;
55
+ }
56
+
57
+ async function restore(platform) {
58
+ const deadline = Date.now() + 120000;
59
+ const found = await find(platform, deadline);
60
+ if (!found) return output('restored', 'false');
61
+ const {artifact, run, fingerprint} = found;
62
+ const directory = mkdtempSync(join(process.env.RUNNER_TEMP, `mobile-binary-${platform}-`));
63
+ execFileSync('gh', ['run', 'download', String(run.id), '--repo', process.env.GITHUB_REPOSITORY,
64
+ '--name', artifact.name, '--dir', directory], {stdio: 'ignore', timeout: Math.max(1, deadline - Date.now())});
65
+ const receipt = JSON.parse(readFileSync(join(directory, 'mobile-binary.json'), 'utf8'));
66
+ if (receipt.run_id !== run.id || receipt.run_attempt !== run.artifact_attempt || receipt.head_sha !== run.head_sha) {
67
+ throw Error('mobile_artifact_run_mismatch');
67
68
  }
68
- output('restored', 'false');
69
+ const operation = platform === 'ios' ? 'verify' : 'restore';
70
+ execFileSync('python3', [join(__dirname, 'mobile_artifact_files.py'), operation, platform, directory],
71
+ {stdio: ['ignore', 'inherit', 'pipe'], env: {...process.env, MOBILE_ARTIFACT_FINGERPRINT: fingerprint}});
72
+ output('directory', directory);
73
+ output('restored', 'true');
74
+ console.log(`::notice::Reusing ${platform} binary from candidate run ${run.id}`);
69
75
  }
70
76
 
71
77
  function pack(platform) {
@@ -86,6 +92,13 @@ async function main([operation, platform, directory]) {
86
92
  return execFileSync('python3', [join(__dirname, 'mobile_artifact_files.py'), 'restore', platform, directory],
87
93
  {stdio: ['ignore', 'inherit', 'pipe'], env: {...process.env, MOBILE_ARTIFACT_FINGERPRINT: identity(platform)}});
88
94
  }
95
+ if (operation === 'lookup') {
96
+ // Admission picks the iOS runner from this answer; any doubt keeps the macOS build path.
97
+ let found = null;
98
+ try { found = await find(platform, Date.now() + 30000); } catch { /* optional optimization */ }
99
+ console.log(found ? `Candidate ${platform} binary in run ${found.run.id}` : `No candidate ${platform} binary`);
100
+ return output('found', found ? 'true' : 'false');
101
+ }
89
102
  if (operation !== 'restore') throw Error('mobile_artifact_operation_invalid');
90
103
  try { await restore(platform); }
91
104
  catch {
@@ -95,7 +108,7 @@ async function main([operation, platform, directory]) {
95
108
  }
96
109
  }
97
110
 
98
- module.exports = {identity, candidate, restore, pack};
111
+ module.exports = {identity, candidate, find, restore, pack, main};
99
112
  if (require.main === module) main(process.argv.slice(2)).catch(() => {
100
113
  console.error('::error::mobile_artifact_pack_failed');
101
114
  process.exitCode = 1;
@@ -295,10 +295,13 @@ _INFOPLIST_FILE_RE = re.compile(
295
295
  )
296
296
 
297
297
 
298
- def _write_infoplist_short_version(new_version: str) -> bool:
299
- """Fallback: stamp CFBundleShortVersionString into the project's
300
- resolved Info.plist. Returns True on success, False when no plist
301
- resolves."""
298
+ def _infoplist_candidates() -> list[Path]:
299
+ """The application's source Info.plist. A promotion carries it from the
300
+ candidate build (checkout-relative; empty when Xcode generates the plist),
301
+ so it never needs xcodebuild, which a Linux runner does not have."""
302
+ known = os.environ.get("SWIFT_APP_SOURCE_INFOPLIST")
303
+ if known is not None:
304
+ return [Path(known)] if known.strip() else []
302
305
  proj_args, base = _xcodebuild_args()
303
306
  scheme = os.environ.get("SCHEME", "").strip()
304
307
  config = os.environ.get("CONFIGURATION", "").strip() or "Release"
@@ -309,12 +312,19 @@ def _write_infoplist_short_version(new_version: str) -> bool:
309
312
  cmd, check=False, capture_output=True, text=True, timeout=120,
310
313
  )
311
314
  except (OSError, subprocess.SubprocessError):
312
- return False
315
+ return []
313
316
  m = _INFOPLIST_FILE_RE.search(out.stdout or "")
314
317
  if not m:
315
- return False
318
+ return []
316
319
  rel = m.group(1).strip()
317
- for path in (base / rel, Path(rel)):
320
+ return [base / rel, Path(rel)]
321
+
322
+
323
+ def _write_infoplist_short_version(new_version: str) -> bool:
324
+ """Fallback: stamp CFBundleShortVersionString into the project's
325
+ resolved Info.plist. Returns True on success, False when no plist
326
+ resolves."""
327
+ for path in _infoplist_candidates():
318
328
  if not path.is_file():
319
329
  continue
320
330
  try:
@@ -264,6 +264,21 @@ def detect_package_name(workspace: Path, build_file: Path | None = None) -> str:
264
264
  raise ConfigError(f"cannot detect Android applicationId from {gradle_files[0].parent}")
265
265
 
266
266
 
267
+ def build_inputs(workspace: Path) -> dict[str, str]:
268
+ """What the release build compiles, read from the checkout alone.
269
+
270
+ The default branch's Gradle cache warm-up (`templates/warm-android-cache.yml`) compiles
271
+ what a candidate compiles without creds/, a Play read or a version write, so it never runs
272
+ resolve_android.py (a Play edit). Raises ConfigError when there is no Android project.
273
+ """
274
+ if project_kind(workspace) == "flutter":
275
+ if not _first_existing(workspace / "android", SETTINGS_FILES):
276
+ raise ConfigError("no Android project: android/settings.gradle(.kts) is missing")
277
+ return {"project_kind": "flutter", "gradle_root": "", "gradle_module": ""}
278
+ project = find_gradle_project(workspace)
279
+ return {"project_kind": "gradle", "gradle_root": str(project.root), "gradle_module": project.module}
280
+
281
+
267
282
  def detect_bundle(workspace: Path, search_root: Path | None = None) -> Path:
268
283
  # Flutter collects every module's outputs under build/app/; a plain Gradle
269
284
  # build leaves them in the module's own build/outputs/bundle/.
@@ -304,3 +319,25 @@ def set_gradle_version(
304
319
  if patched != original:
305
320
  build_file.write_text(patched)
306
321
  return {"version_code": bool(code_hits), "version_name": bool(name_hits)}
322
+
323
+
324
+ if __name__ == "__main__":
325
+ import os
326
+ import sys
327
+
328
+ root = Path(os.environ.get("GITHUB_WORKSPACE", ".")).resolve()
329
+ if sys.argv[1:] == ["--flutter-toolchain"]:
330
+ import gradle_wrapper
331
+
332
+ # The same toolchain repair resolve_android.py applies before a Flutter release build.
333
+ for line in gradle_wrapper.ensure_toolchain(root, Path(os.environ["FLUTTER_ROOT"])):
334
+ print(line)
335
+ sys.exit(0)
336
+ try:
337
+ values = build_inputs(root)
338
+ except ConfigError as exc:
339
+ # Nothing to warm; an empty kind skips the rest without failing anything.
340
+ print(f"Nothing to compile: {exc}")
341
+ values = {"project_kind": "", "gradle_root": "", "gradle_module": ""}
342
+ with open(os.environ["GITHUB_OUTPUT"], "a") as output:
343
+ output.writelines(f"{name}={value}\n" for name, value in values.items())
@@ -7,7 +7,7 @@ description: >
7
7
 
8
8
  inputs:
9
9
  operation:
10
- description: deploy (default), identity-diagnostics, or read-only network-diagnostics.
10
+ description: deploy (default) or identity-diagnostics.
11
11
  required: false
12
12
  default: deploy
13
13
  diagnostic-service:
@@ -19,7 +19,7 @@ inputs:
19
19
  required: false
20
20
  default: ""
21
21
  diagnostic-provider:
22
- description: Optional fixed metadata source for network-diagnostics (none or digitalocean).
22
+ description: Ignored; network-diagnostics was removed with the network allocator.
23
23
  required: false
24
24
  default: none
25
25
  host:
@@ -105,7 +105,7 @@ runs:
105
105
  OPERATION: ${{ inputs.operation }}
106
106
  API_DOMAIN: ${{ inputs.api-domain }}
107
107
  run: |
108
- case "$OPERATION" in deploy|identity-diagnostics|network-diagnostics) ;; *) exit 1;; esac
108
+ case "$OPERATION" in deploy|identity-diagnostics) ;; *) exit 1;; esac
109
109
  # Only a repository with a bound origin holds the deploy key; the panel sets the domain when it binds one.
110
110
  if [ "$OPERATION" = deploy ] && [ -z "$API_DOMAIN" ]; then
111
111
  echo "::notice title=backend_origin_unbound::No api-domain: the backend origin is not bound yet, nothing deployed"
@@ -208,14 +208,6 @@ runs:
208
208
  "${{ github.action_path }}/runtime_env_parse.py" \
209
209
  "${{ github.action_path }}/edge_vhost.py" \
210
210
  "${{ github.action_path }}/edge_health.py" \
211
- "${{ github.action_path }}/identity_diagnostics.py" \
212
- "${{ github.action_path }}/network_runtime.py" \
213
- "${{ github.action_path }}/network_contract.py" \
214
- "${{ github.action_path }}/network_inventory.py" \
215
- "${{ github.action_path }}/network_owned_inventory.py" \
216
- "${{ github.action_path }}/network_readback.py" \
217
- "${{ github.action_path }}/network_pools.py" \
218
- "${{ github.action_path }}/network_prepare.py" \
219
211
  "${{ github.action_path }}/deploy_evidence.sh" \
220
212
  "${{ inputs.ssh-user }}@${{ inputs.host }}:$DEST/"
221
213
 
@@ -279,33 +271,6 @@ runs:
279
271
  path: ${{ steps.identity_diagnostics.outputs.result }}
280
272
  if-no-files-found: error
281
273
 
282
- - name: Read backend network diagnostics
283
- id: network_diagnostics
284
- if: inputs.operation == 'network-diagnostics'
285
- shell: bash
286
- env:
287
- DIAGNOSTIC_APP: ${{ inputs.app-name }}
288
- DIAGNOSTIC_HOST: ${{ inputs.host }}
289
- DIAGNOSTIC_USER: ${{ inputs.ssh-user }}
290
- DIAGNOSTIC_PROVIDER: ${{ inputs.diagnostic-provider }}
291
- run: |
292
- RESULT_DIR="$(mktemp -d "$RUNNER_TEMP/backend-network-diagnostics.XXXXXX")"
293
- echo "result=$RESULT_DIR/result.json" >> "$GITHUB_OUTPUT"
294
- python3 "${{ github.action_path }}/network_diagnostics.py" \
295
- --app "$DIAGNOSTIC_APP" --host "$DIAGNOSTIC_HOST" --user "$DIAGNOSTIC_USER" \
296
- --provider "$DIAGNOSTIC_PROVIDER" \
297
- --output "$RESULT_DIR/result.json"
298
-
299
- - name: Retain backend network diagnostics
300
- if: always() && inputs.operation == 'network-diagnostics'
301
- uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
302
- env:
303
- NODE_OPTIONS: --require "${{ github.action_path }}/scripts/artifact_env.cjs"
304
- with:
305
- name: backend-network-diagnostics-${{ github.job }}-${{ github.run_attempt }}
306
- path: ${{ steps.network_diagnostics.outputs.result }}
307
- if-no-files-found: error
308
-
309
274
  - name: Cleanup key
310
275
  if: always()
311
276
  shell: bash
@@ -1,4 +1,4 @@
1
- """Update managed nginx transport without discarding Certbot's HTTPS configuration."""
1
+ """Render the managed nginx vhost, or update its upstream without discarding Certbot's older HTTPS edits."""
2
2
  import hashlib
3
3
  import os
4
4
  from pathlib import Path
@@ -8,6 +8,10 @@ import subprocess
8
8
  import sys
9
9
  import tempfile
10
10
 
11
+ # Certbot's webroot for every managed name: first issuance and the host's renew timer answer HTTP-01 from here.
12
+ ACME_ROOT = "/var/lib/gowalk-acme"
13
+ STAGES = ("update", "acme", "tls")
14
+
11
15
 
12
16
  class NginxRejected(Exception):
13
17
  """nginx refused the updated vhost; the previous file is back in place."""
@@ -36,22 +40,10 @@ def transport(text, domain):
36
40
  return mapping + result
37
41
 
38
42
 
39
- def content(previous, domain, port):
40
- if not re.fullmatch(r"[a-z0-9](?:[a-z0-9.-]*[a-z0-9])?", domain) or not 0 < port < 65536:
41
- raise ValueError("invalid domain or port")
42
- if previous:
43
- names = re.findall(r"(?m)^\s*server_name\s+([^;]+);", previous)
44
- if not names or any(name.strip() != domain for name in names):
45
- raise ValueError("vhost belongs to another domain")
46
- pattern = r"(?m)^(\s*proxy_pass\s+http://127\.0\.0\.1:)\d+(;[^\n]*$)"
47
- result, count = re.subn(pattern, lambda match: f"{match[1]}{port}{match[2]}", previous)
48
- if count != 1:
49
- raise ValueError("managed upstream is not unique")
50
- return transport(result, domain)
51
- return transport(f"""server {{
52
- listen 80;
53
- server_name {domain};
54
- client_max_body_size 100m;
43
+ def render(domain, port, tls):
44
+ """Port 80 always answers ACME challenges. Before the certificate exists it proxies too; once it exists,
45
+ port 80 redirects and port 443 proxies with the lineage Certbot keeps at the domain's own name."""
46
+ proxy = f""" client_max_body_size 100m;
55
47
  location / {{
56
48
  proxy_pass http://127.0.0.1:{port};
57
49
  proxy_set_header Host $host;
@@ -61,7 +53,33 @@ def content(previous, domain, port):
61
53
  proxy_read_timeout 120s;
62
54
  }}
63
55
  }}
64
- """, domain)
56
+ """
57
+ http = (f"server {{\n listen 80;\n server_name {domain};\n"
58
+ f" location ^~ /.well-known/acme-challenge/ {{ root {ACME_ROOT}; }}\n")
59
+ if not tls:
60
+ return http + proxy
61
+ # One-line redirect: edge_vhost adds its ingress guard to the only multi-line `location / {`, the proxy.
62
+ live = f"/etc/letsencrypt/live/{domain}"
63
+ return (http + " location / { return 301 https://$host$request_uri; }\n}\n\n"
64
+ f"server {{\n listen 443 ssl;\n server_name {domain};\n"
65
+ f" ssl_certificate {live}/fullchain.pem;\n ssl_certificate_key {live}/privkey.pem;\n" + proxy)
66
+
67
+
68
+ def content(previous, domain, port, stage="update"):
69
+ """`update` changes only the upstream port of an existing vhost; `acme` and `tls` render it afresh."""
70
+ if not re.fullmatch(r"[a-z0-9](?:[a-z0-9.-]*[a-z0-9])?", domain) or not 0 < port < 65536 or stage not in STAGES:
71
+ raise ValueError("invalid domain, port or stage")
72
+ if previous:
73
+ names = re.findall(r"(?m)^\s*server_name\s+([^;]+);", previous)
74
+ if not names or any(name.strip() != domain for name in names):
75
+ raise ValueError("vhost belongs to another domain")
76
+ if stage != "update" or not previous:
77
+ return transport(render(domain, port, stage == "tls"), domain)
78
+ pattern = r"(?m)^(\s*proxy_pass\s+http://127\.0\.0\.1:)\d+(;[^\n]*$)"
79
+ result, count = re.subn(pattern, lambda match: f"{match[1]}{port}{match[2]}", previous)
80
+ if count != 1:
81
+ raise ValueError("managed upstream is not unique")
82
+ return transport(result, domain)
65
83
 
66
84
 
67
85
  def replace(path, data, mode):
@@ -77,7 +95,7 @@ def replace(path, data, mode):
77
95
  Path(temporary).unlink(missing_ok=True)
78
96
 
79
97
 
80
- def configure(path, domain, port, run=subprocess.run):
98
+ def configure(path, domain, port, run=subprocess.run, stage="update"):
81
99
  """Write the vhost, then test and reload nginx; on any failure restore the previous file, as edge does.
82
100
 
83
101
  A change nginx refuses must not stay on disk: every later `nginx -t` on the host, this deploy's own
@@ -88,7 +106,7 @@ def configure(path, domain, port, run=subprocess.run):
88
106
  if path.is_symlink():
89
107
  raise ValueError("managed vhost must not be a symlink")
90
108
  previous = path.read_text() if path.exists() else ""
91
- updated = content(previous, domain, port)
109
+ updated = content(previous, domain, port, stage)
92
110
  if previous == updated:
93
111
  return "unchanged"
94
112
  saved = path.read_bytes() if path.exists() else None
@@ -108,7 +126,7 @@ def configure(path, domain, port, run=subprocess.run):
108
126
 
109
127
  if __name__ == "__main__":
110
128
  try:
111
- print(configure(Path(sys.argv[1]), sys.argv[2], int(sys.argv[3])))
129
+ print(configure(Path(sys.argv[1]), sys.argv[2], int(sys.argv[3]), stage=(sys.argv[4:5] or ["update"])[0]))
112
130
  except NginxRejected:
113
131
  sys.exit("nginx rejected the managed vhost update; the previous vhost is restored")
114
132
  except (OSError, ValueError):
@@ -53,8 +53,8 @@ def validate(config, app, domain, nginx_dir=NGINX_DIR):
53
53
  return value
54
54
 
55
55
 
56
- def edge_content(text, domain, port, include):
57
- updated = content(text, domain, port)
56
+ def edge_content(text, domain, port, include, stage="update"):
57
+ updated = content(text, domain, port, stage)
58
58
  directive = f" include {include};\n"
59
59
  if re.search(r"[\s;{}]", str(include)):
60
60
  raise ValueError("backend_ingress_path_invalid")
@@ -69,7 +69,7 @@ def edge_content(text, domain, port, include):
69
69
  return updated
70
70
 
71
71
 
72
- def planned(config, app, domain, port, nginx_dir=NGINX_DIR):
72
+ def planned(config, app, domain, port, nginx_dir=NGINX_DIR, stage="update"):
73
73
  value = validate(config, app, domain, nginx_dir)
74
74
  if value["ingress_required"] != "true":
75
75
  raise ValueError("backend_ingress_configuration_required")
@@ -78,7 +78,7 @@ def planned(config, app, domain, port, nginx_dir=NGINX_DIR):
78
78
  guard = (f'if ($http_x_app_robot_backend_token != "{token}") {{ return 403; }}\n'
79
79
  'proxy_set_header X-App-Robot-Backend-Token "";\n')
80
80
  files = {include: (guard.encode(), 0o600),
81
- edge: (edge_content(previous(edge), domain, port, include).encode(), 0o644)}
81
+ edge: (edge_content(previous(edge), domain, port, include, stage).encode(), 0o644)}
82
82
  old_domain = recorded_domain(legacy)
83
83
  if old_domain:
84
84
  files[legacy] = (content(previous(legacy), old_domain, port).encode(),
@@ -86,8 +86,8 @@ def planned(config, app, domain, port, nginx_dir=NGINX_DIR):
86
86
  return files
87
87
 
88
88
 
89
- def configure(config, app, domain, port, nginx_dir=NGINX_DIR, run=subprocess.run):
90
- files = planned(config, app, domain, port, nginx_dir)
89
+ def configure(config, app, domain, port, nginx_dir=NGINX_DIR, run=subprocess.run, stage="update"):
90
+ files = planned(config, app, domain, port, nginx_dir, stage)
91
91
  saved = {path: ((path.read_bytes(), stat.S_IMODE(path.stat().st_mode)) if path.exists() else None)
92
92
  for path in files}
93
93
  changes = {path: value for path, value in files.items() if saved[path] != value}
@@ -113,7 +113,7 @@ def main():
113
113
  if operation == "validate":
114
114
  validate(config, app, domain)
115
115
  elif operation == "configure":
116
- print(configure(config, app, domain, int(sys.argv[5])))
116
+ print(configure(config, app, domain, int(sys.argv[5]), stage=(sys.argv[6:7] or ["update"])[0]))
117
117
  else:
118
118
  raise ValueError("backend_edge_operation_invalid")
119
119