gowalk-cicd 1.0.177 → 1.0.179

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -242,7 +242,7 @@ contents, contact a provider or deploy. Preserve app-owned configuration and gen
242
242
  in the same validated candidate. A layout receipt proves selection, not production readiness.
243
243
 
244
244
  The backend workflow runs on a GitHub-hosted runner and, on a push touching
245
- `backend/**`, deploys to the gowalk host (`138.197.36.107`, publicly reachable):
245
+ `backend/**`, deploys to the explicitly bound, publicly reachable `BACKEND_DEPLOY_HOST`:
246
246
  it rsyncs the backend dir to `/opt/gowalk-backends/<app>/`, runs
247
247
  `docker compose up -d --build`, auto-detects the published `127.0.0.1:<port>`,
248
248
  wires an nginx vhost + Let's Encrypt cert for the API domain, and health-checks.
@@ -304,9 +304,10 @@ Requirements on the consumer repo:
304
304
  **Backend Deploy runs only when it is set**: a repository with no bound backend
305
305
  origin holds no deploy key, so `deploy-backend.yml` is skipped and the feature-branch
306
306
  `backend-preview` deploy ends with a `backend_origin_unbound` notice.
307
+ - Repo **variable** `BACKEND_DEPLOY_HOST`: the bound SSH destination (DNS name or IPv4 address),
308
+ required for a bound backend; no panel host or historical address is used as a fallback.
307
309
  - Repo **variables** (optional): `BACKEND_APP_NAME` (defaults to the repo name),
308
- `BACKEND_HEALTH_PATH` (default `/health`), `BACKEND_DEPLOY_HOST` (default
309
- `138.197.36.107`). The action refuses an `api-domain` that is not a lowercase DNS
310
+ `BACKEND_HEALTH_PATH` (default `/health`). The action refuses an `api-domain` that is not a lowercase DNS
310
311
  name, a `health-path` outside `/[A-Za-z0-9_./-]*` and a malformed `cert-email`.
311
312
 
312
313
  The action directory is canonical here (like the iOS/Android actions): edit it in
@@ -319,6 +320,36 @@ only; normal default-branch backend deploys remain in `deploy-backend.yml`.
319
320
 
320
321
  For optional app-owned Cloudflare ingress, follow [backend edge delivery](docs/BACKEND_EDGE.md).
321
322
 
323
+ Backend starts use the resolved Compose model with effective CPU, memory and PID limits. Missing limits
324
+ default to 0.5 CPU, 256 MiB memory, a 64 MiB memory reservation (or the smaller hard limit), and 128 PIDs.
325
+ Explicit workload limits remain authoritative. Both Compose resource syntaxes agree in the generated
326
+ model; JSON/local logs default to 10 MiB with three files. Health probes run at least 60 seconds apart
327
+ in steady state, retaining fast startup with a 5-second start interval and at least 30 seconds of grace.
328
+ The app owns a cheap probe and effective worker/pool limits; this helper never rewrites application code.
329
+ Docker Engine 25+ and Compose 2.20.2+ support the startup interval.
330
+
331
+ Before building and again immediately before starting, deployment checks live host memory and pressure.
332
+ Other factory containers plus the proposed stack may reserve at most 70% of physical RAM; at least
333
+ 10% (and 512 MiB) stays available after additional reservations. One-minute load above twice the CPU
334
+ count or full memory pressure of 1% refuses a start. Starts share a host lock (120-second acquisition
335
+ limit); image builds stay parallel. A failed admission emits `backend_runtime_refused` with a fixed
336
+ `gowalk-cicd/backend-runtime-refused.v1 code=…` and leaves existing containers running.
337
+
338
+ An operator-managed `/opt/gowalk-backend-runtime/<app>.json`, outside the application mirror, may set
339
+ `schema: "gowalk-cicd/backend-runtime.v1"`, `disabled: true`, `remove_services: ["db"]`, and per-service
340
+ `services` overrides for `environment`, `command`, `entrypoint`, `mem_limit`, `mem_reservation`, `cpus`
341
+ and `pids_limit`. A disabled binding refuses builds/starts. After a verified database migration, this
342
+ file can remove the retired database service and override the app's connection environment on every
343
+ redeploy. Only dependencies on explicitly removed services are removed; volumes are preserved. Keep
344
+ the file private (0600), use distinct scoped database roles, and complete compatibility, restore and
345
+ isolation checks before removing a database. This helper provisions no shared database and deletes no
346
+ database volume. Resolved credentials stay in a private temporary file removed on success or failure;
347
+ literal dollars survive the second Compose parse.
348
+
349
+ For a shared host, use the [host-owned SSH deployment entry](docs/BACKEND_SSH_GATE.md). It scopes each panel's
350
+ key, keeps new project claims autonomous, and redirects older action invocations to the released host helpers.
351
+ Its global disabled gate permits readiness while refusing mutations during a migration.
352
+
322
353
  ## Android credentials
323
354
 
324
355
  Place these files under `creds/`:
@@ -524,6 +555,10 @@ remain readable by the panel. Masked fields are never reconstructed and credenti
524
555
  Protected vhost failures add `backend_vhost_failed` (`gowalk-cicd/backend-vhost-failed.v1`) with fixed
525
556
  brace-free `operation`, `stage`, `reason`, `child_exit` and `signals`; see `docs/BACKEND_EDGE.md`.
526
557
  It preserves the existing refusal and records only bounded observed facts, never private child output.
558
+ Edge health failures add `backend_health_failed` (`gowalk-cicd/backend-health-failed.v1`) with fixed
559
+ brace-free `probe`, `reason`, `curl_exit`, `http_status` and `attempt`; see `docs/BACKEND_EDGE.md`.
560
+ Only the final failed probe's bounded observations appear, with no URLs, bodies, headers or credentials.
561
+ Ingress refusal, strict TLS, exact-build acceptance, retry bounds and the original failed result remain.
527
562
  An interrupted locked Pod fetch can retry once within the existing 900-second install budget.
528
563
  `native_pods_retry` (`gowalk-cicd/native-pods-retry.v1`) reports `attempt:2` and `reason:partial_transfer`;
529
564
  successful preparation reports `pod_attempts`. Dependency locks, relay and TLS checks remain enforced.
@@ -1,7 +1,7 @@
1
1
  name: gowalk-cicd backend deploy
2
2
  description: >
3
3
  Build and deploy a Dockerized Python+Postgres backend to the gowalk deploy
4
- host (138.197.36.107) — rsync the backend dir, `docker compose up -d --build`,
4
+ configured host — rsync the backend dir, enforce resources, build/start the stack,
5
5
  wire an nginx vhost + Let's Encrypt cert for the API domain, and health-check.
6
6
  The deploy host is publicly reachable, so this runs on a GitHub-hosted runner.
7
7
 
@@ -23,9 +23,9 @@ inputs:
23
23
  required: false
24
24
  default: none
25
25
  host:
26
- description: SSH host of the deploy server.
27
- required: false
28
- default: "138.197.36.107"
26
+ description: Explicit SSH host from the backend binding; no infrastructure fallback.
27
+ required: true
28
+ default: ""
29
29
  ssh-user:
30
30
  description: SSH user on the deploy server.
31
31
  required: false
@@ -104,12 +104,16 @@ runs:
104
104
  env:
105
105
  OPERATION: ${{ inputs.operation }}
106
106
  API_DOMAIN: ${{ inputs.api-domain }}
107
+ DEPLOY_HOST: ${{ inputs.host }}
107
108
  run: |
108
109
  case "$OPERATION" in deploy|identity-diagnostics) ;; *) exit 1;; esac
109
110
  # Only a repository with a bound origin holds the deploy key; the panel sets the domain when it binds one.
110
111
  if [ "$OPERATION" = deploy ] && [ -z "$API_DOMAIN" ]; then
111
112
  echo "::notice title=backend_origin_unbound::No api-domain: the backend origin is not bound yet, nothing deployed"
112
113
  echo "skip=true" >> "$GITHUB_OUTPUT"
114
+ elif [[ ! "$DEPLOY_HOST" =~ ^[A-Za-z0-9][A-Za-z0-9.-]*$ ]]; then
115
+ echo "::error title=backend_host_required::Configure the backend binding's SSH host before deployment"
116
+ exit 1
113
117
  fi
114
118
 
115
119
  - name: Validate inputs
@@ -206,6 +210,10 @@ runs:
206
210
  "${{ github.action_path }}/edge_inputs.py" \
207
211
  "${{ github.action_path }}/runtime_env.py" \
208
212
  "${{ github.action_path }}/runtime_env_parse.py" \
213
+ "${{ github.action_path }}/runtime_model.py" \
214
+ "${{ github.action_path }}/runtime_host.py" \
215
+ "${{ github.action_path }}/native_profile.py" \
216
+ "${{ github.action_path }}/runtime_launch.py" \
209
217
  "${{ github.action_path }}/edge_vhost.py" \
210
218
  "${{ github.action_path }}/vhost_diagnostics.py" \
211
219
  "${{ github.action_path }}/edge_health.py" \
@@ -9,6 +9,25 @@ import time
9
9
  from edge_inputs import load, TOKEN_HEADER
10
10
  from edge_vhost import NGINX_DIR, paths, recorded_domain
11
11
 
12
+ PROBES = frozenset({"configuration", "origin_missing_token", "origin_invalid_token", "origin_authorized",
13
+ "legacy_identity", "legacy", "public"})
14
+ REASONS = frozenset({"unclassified", "path_invalid", "request_failed", "http_refused", "json_invalid",
15
+ "body_shape", "build_missing", "build_mismatch"})
16
+
17
+
18
+ def report(probe, reason, *, curl_exit=None, http_status=None, attempt=None):
19
+ probe = probe if type(probe) is str and probe in PROBES else "unclassified"
20
+ reason = reason if type(reason) is str and reason in REASONS else "unclassified"
21
+ code = str(curl_exit) if type(curl_exit) is int and -255 <= curl_exit <= 255 else "none"
22
+ status = str(http_status) if type(http_status) is int and 100 <= http_status <= 599 else "none"
23
+ count = str(attempt) if type(attempt) is int and 1 <= attempt <= 15 else "none"
24
+ message = ("::error title=backend_health_failed::gowalk-cicd/backend-health-failed.v1 "
25
+ "probe={} reason={} curl_exit={} http_status={} attempt={}\n")
26
+ try:
27
+ sys.stderr.write(message.format(probe, reason, code, status, count))
28
+ except (OSError, ValueError):
29
+ pass # Diagnostic delivery must not replace the original refusal.
30
+
12
31
 
13
32
  def request(url, *, headers=None, run=subprocess.run):
14
33
  # curl retains its strict TLS/time bounds. Private headers and bounded response
@@ -27,46 +46,89 @@ def request(url, *, headers=None, run=subprocess.run):
27
46
  return result.returncode, int(status) if status.isdigit() else 0, data
28
47
 
29
48
 
30
- def healthy(url, sha="", *, headers=None, fetch=request, sleep=time.sleep):
49
+ def response_reason(code, status, body, sha):
50
+ if code != 0:
51
+ return "request_failed"
52
+ if not 200 <= status < 300:
53
+ return "http_refused"
54
+ if sha:
55
+ try:
56
+ value = json.loads(body)
57
+ except ValueError:
58
+ return "json_invalid"
59
+ if not isinstance(value, dict):
60
+ return "body_shape"
61
+ if "build_sha" not in value:
62
+ return "build_missing"
63
+ if value.get("build_sha") != sha:
64
+ return "build_mismatch"
65
+ return None
66
+
67
+
68
+ def healthy(url, sha="", *, headers=None, fetch=request, sleep=time.sleep, probe="public"):
31
69
  for attempt in range(15):
70
+ code = status = None
71
+ reason = "unclassified"
32
72
  try:
33
73
  code, status, body = fetch(url, headers=headers)
34
- valid = code == 0 and 200 <= status < 300
35
- if valid and sha:
36
- value = json.loads(body)
37
- valid = isinstance(value, dict) and value.get("build_sha") == sha
38
- if valid:
74
+ reason = response_reason(code, status, body, sha)
75
+ if reason is None:
39
76
  return
40
77
  except (OSError, ValueError, subprocess.SubprocessError):
41
- pass
78
+ reason = "request_failed" if code is None else "unclassified"
42
79
  if attempt != 14:
43
80
  sleep(2)
81
+ report(probe, reason, curl_exit=code, http_status=status, attempt=15)
44
82
  raise ValueError("backend_public_health_failed")
45
83
 
46
84
 
85
+ def origin_refusal(url, headers, probe, fetch):
86
+ try:
87
+ code, status, _body = fetch(url, headers=headers)
88
+ except (OSError, ValueError, subprocess.SubprocessError):
89
+ report(probe, "request_failed", attempt=1)
90
+ raise
91
+ if code != 0 or status != 403:
92
+ report(probe, "request_failed" if code != 0 else "http_refused",
93
+ curl_exit=code, http_status=status, attempt=1)
94
+ raise ValueError("backend_origin_ingress_unverified")
95
+
96
+
47
97
  def verify(config, app, domain, health, *, nginx_dir=NGINX_DIR, fetch=request, sleep=time.sleep):
48
- value = load(config)
98
+ try:
99
+ value = load(config)
100
+ except (OSError, ValueError, subprocess.SubprocessError):
101
+ report("configuration", "unclassified")
102
+ raise
49
103
  if not health.startswith("/") or any(character in health for character in "\r\n?#"):
104
+ report("configuration", "path_invalid")
50
105
  raise ValueError("backend_health_path_invalid")
51
106
  origin_url = f"https://{domain}{health}"
52
107
  if value["ingress_required"] == "true":
53
- for headers in (None, {TOKEN_HEADER: "invalid-backend-ingress-token"}):
54
- code, status, _body = fetch(origin_url, headers=headers)
55
- if code != 0 or status != 403:
56
- raise ValueError("backend_origin_ingress_unverified")
108
+ for probe, headers in (("origin_missing_token", None),
109
+ ("origin_invalid_token", {TOKEN_HEADER: "invalid-backend-ingress-token"})):
110
+ origin_refusal(origin_url, headers, probe, fetch)
57
111
  healthy(origin_url, value["expected_build_sha"], headers={TOKEN_HEADER: value["ingress_token"]},
58
- fetch=fetch, sleep=sleep)
59
- legacy = recorded_domain(paths(config, app, nginx_dir)[0])
112
+ fetch=fetch, sleep=sleep, probe="origin_authorized")
113
+ try:
114
+ legacy = recorded_domain(paths(config, app, nginx_dir)[0])
115
+ except (OSError, ValueError, subprocess.SubprocessError):
116
+ report("legacy_identity", "unclassified")
117
+ raise
60
118
  if legacy:
61
- healthy(f"https://{legacy}{health}", fetch=fetch, sleep=sleep)
119
+ healthy(f"https://{legacy}{health}", fetch=fetch, sleep=sleep, probe="legacy")
62
120
  public = value["public_health_url"] or origin_url
63
121
  healthy(public, value["expected_build_sha"], fetch=fetch, sleep=sleep)
64
122
  return {"ok": True, "public_health_verified": True, "build_sha": value["expected_build_sha"],
65
123
  "ingress_verified": value["ingress_required"] == "true"}
66
124
 
67
125
 
68
- if __name__ == "__main__":
126
+ def main():
69
127
  try:
70
128
  print(json.dumps(verify(*sys.argv[1:5])))
71
129
  except (OSError, ValueError, subprocess.SubprocessError):
72
130
  sys.exit("backend_edge_health_failed")
131
+
132
+
133
+ if __name__ == "__main__":
134
+ main()
@@ -0,0 +1,143 @@
1
+ """Run only an operator-declared native deploy under shared host admission and resource limits."""
2
+ import http.client
3
+ import json
4
+ import os
5
+ import re
6
+ import subprocess
7
+ import sys
8
+ import time
9
+ from pathlib import Path
10
+ from urllib.parse import urlsplit
11
+
12
+ from native_profile import APPS, load_profile, model, unit_properties
13
+ from runtime_host import admit, host_facts, running_containers, start_lock
14
+ from runtime_launch import START_LOCK
15
+ from runtime_model import duration
16
+
17
+
18
+ def run(arguments, timeout=60):
19
+ return subprocess.run(arguments, capture_output=True, text=True, timeout=timeout, check=True)
20
+
21
+
22
+ def install(app, profile, phase='install'):
23
+ deadline = time.monotonic() + (600 if phase == 'install' else 60)
24
+ for index, arguments in enumerate(profile[phase]):
25
+ remaining = deadline - time.monotonic()
26
+ if remaining <= 0:
27
+ raise ValueError('native_install_timeout')
28
+ unit = 'gowalk-native-%s-%s-%s-%s' % (phase, app, os.getpid(), index)
29
+ try:
30
+ run(['systemd-run', '--quiet', '--wait', '--pipe', '--collect', '--unit=' + unit,
31
+ '--property=MemoryMax=512M', '--property=CPUQuota=100%', '--property=TasksMax=128',
32
+ '--property=RuntimeMaxSec=%g' % remaining, '--property=User=' + profile['install_user'],
33
+ '--property=WorkingDirectory=' + profile['directory'], '--', *arguments], timeout=remaining + 5)
34
+ except subprocess.SubprocessError:
35
+ run(['systemctl', 'stop', unit])
36
+ raise
37
+
38
+
39
+ def sync_source(app, profile):
40
+ source = APPS / app
41
+ if source.is_symlink() or not source.is_dir():
42
+ raise ValueError('native_source_invalid')
43
+ if str(source) == profile['directory']:
44
+ return
45
+ exclusions = sorted(set(['.env', '.runtime.env', '.git', 'env/', '.venv/', *profile['preserve']]))
46
+ run(['rsync', '-ac', '--delete', '--safe-links', *['--exclude=' + path for path in exclusions],
47
+ str(source) + '/', profile['directory'] + '/'], timeout=120)
48
+
49
+
50
+ def source_revision(directory):
51
+ with (Path(directory) / '.backend-source-sha').open() as stream:
52
+ revision = stream.read(42).strip()
53
+ if not re.fullmatch('[0-9a-f]{40}', revision):
54
+ raise ValueError('native_source_identity_invalid')
55
+ return revision
56
+
57
+
58
+ def held_state(profile):
59
+ units = sorted(set(profile['units']) - set(profile['restart_units']))
60
+ return unit_properties(units, ['ActiveState', 'UnitFileState', 'MainPID']) if units else {}
61
+
62
+
63
+ def apply_limits(document):
64
+ for unit, limits in document['services'].items():
65
+ run(['systemctl', 'set-property', unit, 'MemoryMax=' + str(limits['mem_limit']),
66
+ 'MemoryLow=' + str(limits['mem_reservation']), 'CPUQuota=%g%%' % (limits['cpus'] * 100),
67
+ 'TasksMax=' + str(limits['pids_limit'])])
68
+ states = unit_properties(list(document['services']), ['MemoryMax', 'MemoryLow', 'CPUQuotaPerSecUSec', 'TasksMax'])
69
+ for unit, limits in document['services'].items():
70
+ state = states[unit]
71
+ expected = (limits['mem_limit'], limits['mem_reservation'], limits['pids_limit'])
72
+ actual = tuple(int(state[name]) for name in ('MemoryMax', 'MemoryLow', 'TasksMax'))
73
+ if actual != expected or abs(duration(state['CPUQuotaPerSecUSec']) - limits['cpus']) > 0.000001:
74
+ raise ValueError('native_limits_unapplied')
75
+
76
+
77
+ def health(profile):
78
+ url = urlsplit(profile['health_url'])
79
+ deadline = time.monotonic() + 60
80
+ while time.monotonic() < deadline:
81
+ connection = http.client.HTTPConnection(url.hostname, url.port or 80, timeout=5)
82
+ try:
83
+ connection.request('GET', (url.path or '/') + ('?' + url.query if url.query else ''))
84
+ response = connection.getresponse()
85
+ payload = json.loads(response.read(65537)) if profile['health_json'] else {}
86
+ matches = isinstance(payload, dict) and all(payload.get(k) == v for k, v in profile['health_json'].items())
87
+ states = unit_properties(profile['restart_units'], ['ActiveState'])
88
+ active = all(item['ActiveState'] == 'active' for item in states.values())
89
+ if matches and 200 <= response.status < 300 and active:
90
+ return
91
+ except (OSError, ValueError, http.client.HTTPException):
92
+ pass
93
+ finally:
94
+ connection.close()
95
+ time.sleep(2)
96
+ raise ValueError('native_health_failed')
97
+
98
+
99
+ def deploy(app):
100
+ profile = load_profile(app)
101
+ if profile['disabled']:
102
+ raise ValueError('native_disabled')
103
+ revision = source_revision(APPS / app)
104
+ document = model(profile)
105
+ admit(app, document, host_facts(), running_containers())
106
+ preserved_state = held_state(profile)
107
+ sync_source(app, profile)
108
+ install(app, profile)
109
+ with start_lock(START_LOCK):
110
+ # Re-read the host's freeze after preparation and immediately before start.
111
+ if load_profile(app) != profile:
112
+ raise ValueError('native_profile_changed')
113
+ if held_state(profile) != preserved_state:
114
+ raise ValueError('native_held_unit_changed')
115
+ admit(app, document, host_facts(), running_containers())
116
+ apply_limits(document)
117
+ try:
118
+ run(['systemctl', 'restart', *profile['restart_units']])
119
+ health(profile)
120
+ install(app, profile, 'verify')
121
+ if source_revision(profile['directory']) != revision:
122
+ raise ValueError('native_source_changed')
123
+ if held_state(profile) != preserved_state:
124
+ raise ValueError('native_held_unit_changed')
125
+ except (ValueError, OSError, subprocess.SubprocessError):
126
+ run(['systemctl', 'stop', *profile['restart_units']])
127
+ raise
128
+ print(json.dumps({'schema': 'gowalk-cicd/backend-native-result.v1', 'project': app,
129
+ 'units': profile['restart_units'], 'health': 'passed', 'resources': 'verified',
130
+ 'verification': 'passed', 'source_sha': revision}))
131
+
132
+
133
+ def main():
134
+ try:
135
+ deploy(sys.argv[1])
136
+ except (ValueError, OSError, KeyError, IndexError, TypeError, subprocess.SubprocessError):
137
+ print('backend_native_refused', file=sys.stderr)
138
+ return 1
139
+ return 0
140
+
141
+
142
+ if __name__ == '__main__':
143
+ sys.exit(main())
@@ -0,0 +1,105 @@
1
+ """Root-owned native deployment declarations and reservations shared with Compose admission."""
2
+ import json
3
+ import re
4
+ import subprocess
5
+ from pathlib import Path
6
+ from urllib.parse import urlsplit
7
+
8
+ from runtime_model import configure
9
+
10
+ PROFILES = Path('/opt/gowalk-backend-native')
11
+ APPS = Path('/opt/gowalk-backends')
12
+ ROOT_UID = 0
13
+ FIELDS = {'schema', 'disabled', 'directory', 'units', 'restart_units', 'install', 'install_user',
14
+ 'health_url', 'health_json', 'resources', 'preserve', 'verify'}
15
+
16
+
17
+ def load_profile(app):
18
+ if not re.fullmatch('[a-z0-9-]+', app):
19
+ raise ValueError('native_project_invalid')
20
+ path = PROFILES / (app + '.json')
21
+ info = path.stat()
22
+ if path.is_symlink() or info.st_uid != ROOT_UID or info.st_mode & 0o022:
23
+ raise ValueError('native_profile_permissions')
24
+ profile = json.loads(path.read_text())
25
+ if not isinstance(profile, dict) or set(profile) != FIELDS:
26
+ raise ValueError('native_profile_invalid')
27
+ if profile['schema'] != 'gowalk-cicd/backend-native.v1' or not isinstance(profile['disabled'], bool):
28
+ raise ValueError('native_profile_invalid')
29
+ directory = Path(profile['directory'])
30
+ if directory not in (APPS / app, Path('/opt') / app) or directory.is_symlink():
31
+ raise ValueError('native_directory_invalid')
32
+ units = profile['units']
33
+ if not isinstance(units, list) or not units or len(set(units)) != len(units):
34
+ raise ValueError('native_units_invalid')
35
+ if any(not re.fullmatch('[a-zA-Z0-9_-]+[.]service', unit) for unit in units):
36
+ raise ValueError('native_units_invalid')
37
+ restarting = profile['restart_units']
38
+ if not isinstance(restarting, list) or not restarting or set(restarting) - set(units):
39
+ raise ValueError('native_units_invalid')
40
+ validate_deployment(profile)
41
+ model(profile)
42
+ return profile
43
+
44
+
45
+ def validate_deployment(profile):
46
+ for field in ['install', 'verify']:
47
+ commands = profile[field]
48
+ if not isinstance(commands, list) or len(commands) > 8:
49
+ raise ValueError('native_install_invalid')
50
+ for command in commands:
51
+ if not isinstance(command, list) or not command or not all(isinstance(word, str) for word in command):
52
+ raise ValueError('native_install_invalid')
53
+ if not command[0].startswith('/') or any('\x00' in word for word in command):
54
+ raise ValueError('native_install_invalid')
55
+ if not isinstance(profile['install_user'], str) or not re.fullmatch('[a-z_][a-z0-9_-]*', profile['install_user']):
56
+ raise ValueError('native_install_user_invalid')
57
+ url = urlsplit(profile['health_url'])
58
+ if url.scheme != 'http' or url.hostname not in ('127.0.0.1', 'localhost', '::1') or url.username or url.password:
59
+ raise ValueError('native_health_invalid')
60
+ if not isinstance(profile['health_json'], dict):
61
+ raise ValueError('native_health_invalid')
62
+ preserve = profile['preserve']
63
+ if not isinstance(preserve, list) or any(not re.fullmatch('[a-zA-Z0-9_.-]+/?', item) for item in preserve):
64
+ raise ValueError('native_preserve_invalid')
65
+ if any(item in ('.', '..', './', '../') for item in preserve):
66
+ raise ValueError('native_preserve_invalid')
67
+
68
+
69
+ def model(profile):
70
+ resources = profile['resources']
71
+ if not isinstance(resources, dict) or set(resources) - set(profile['units']):
72
+ raise ValueError('native_resources_invalid')
73
+ services = {}
74
+ for unit in profile['units']:
75
+ values = resources.get(unit, {})
76
+ if not isinstance(values, dict) or set(values) - {'mem_limit', 'mem_reservation', 'cpus', 'pids_limit'}:
77
+ raise ValueError('native_resources_invalid')
78
+ services[unit] = values
79
+ return configure({'services': services})
80
+
81
+
82
+ def unit_properties(units, properties):
83
+ result = subprocess.run(['systemctl', 'show', '--property=' + ','.join(['Id', *properties]), *units],
84
+ capture_output=True, text=True, timeout=30, check=True)
85
+ records = [dict(line.split('=', 1) for line in block.splitlines() if '=' in line)
86
+ for block in result.stdout.strip().split('\n\n')]
87
+ mapped = {record['Id']: record for record in records if record.get('Id')}
88
+ if set(mapped) != set(units):
89
+ raise ValueError('native_unit_missing')
90
+ return mapped
91
+
92
+
93
+ def running_native():
94
+ containers = []
95
+ for path in sorted(PROFILES.glob('*.json')):
96
+ profile = load_profile(path.stem)
97
+ states = unit_properties(profile['units'], ['ActiveState', 'MemoryLow'])
98
+ for unit, limits in model(profile)['services'].items():
99
+ if states[unit]['ActiveState'] not in ('active', 'activating', 'reloading'):
100
+ continue
101
+ reservation = max(int(states[unit]['MemoryLow']), limits['mem_reservation'])
102
+ containers.append({'Config': {'Labels': {'com.docker.compose.project': path.stem,
103
+ 'com.docker.compose.project.working_dir': str(APPS / path.stem)}},
104
+ 'HostConfig': {'MemoryReservation': reservation}})
105
+ return containers
@@ -9,7 +9,8 @@ DOMAIN="${2:-}" # api.<app-domain>, or empty to skip nginx/cert
9
9
  HEALTH="${3:-/health}"
10
10
  CERT_EMAIL="${4:-admin@gowalk.com}"
11
11
  DIR="/opt/gowalk-backends/$APP"
12
- source "$(dirname "${BASH_SOURCE[0]}")/deploy_evidence.sh"
12
+ HELPERS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
13
+ source "$HELPERS/deploy_evidence.sh"
13
14
  BACKEND_DEPLOY_PHASE=remote_setup
14
15
  cd "$DIR"
15
16
  export COMPOSE_FILE="${5:-docker-compose.yml}"
@@ -18,8 +19,8 @@ case "$COMPOSE_FILE" in
18
19
  *) echo "::error::unsupported backend compose filename"; exit 1;;
19
20
  esac
20
21
  test -f "$COMPOSE_FILE" || { echo "::error::selected backend compose file is missing"; exit 1; }
21
- source "$DIR/select_certbot_account.sh"
22
- source "$DIR/ensure_certificate.sh"
22
+ source "$HELPERS/select_certbot_account.sh"
23
+ source "$HELPERS/ensure_certificate.sh"
23
24
 
24
25
  log() { echo "[deploy $APP] $*"; }
25
26
  log "host $(python3 --version)"
@@ -38,9 +39,9 @@ EDGE_CONFIG="/opt/gowalk-backend-ingress/$APP/.backend-edge.json"
38
39
  EDGE_VHOST="/etc/nginx/conf.d/backend-$APP-edge.conf"
39
40
  EDGE_MODE=legacy
40
41
  if [ -f "$EDGE_CONFIG" ] || [ -e "$EDGE_VHOST" ]; then
41
- python3 "$DIR/edge_vhost.py" validate "$EDGE_CONFIG" "$APP" "$DOMAIN"
42
- EDGE_MODE="$(python3 "$DIR/edge_inputs.py" mode "$EDGE_CONFIG")"
43
- python3 "$DIR/edge_inputs.py" runtime "$EDGE_CONFIG"
42
+ python3 "$HELPERS/edge_vhost.py" validate "$EDGE_CONFIG" "$APP" "$DOMAIN"
43
+ EDGE_MODE="$(python3 "$HELPERS/edge_inputs.py" mode "$EDGE_CONFIG")"
44
+ python3 "$HELPERS/edge_inputs.py" runtime "$EDGE_CONFIG"
44
45
  fi
45
46
 
46
47
  # ── secrets: generate a per-app .env once, then preserve it (rsync excludes it)
@@ -83,8 +84,7 @@ fi
83
84
  # ── bring the stack up
84
85
  BACKEND_DEPLOY_PHASE=compose_up
85
86
  log "docker compose up -d --build"
86
- docker compose "${COMPOSE_ENV[@]}" --project-name "$APP" \
87
- up -d --build --remove-orphans
87
+ python3 "$HELPERS/runtime_launch.py" "$APP" "${COMPOSE_ENV[@]}"
88
88
 
89
89
  # ── discover the host port the service published on 127.0.0.1.
90
90
  # The text `ps` PORTS column reliably shows `127.0.0.1:<port>-><target>/tcp`.
@@ -136,9 +136,9 @@ CERT_VHOST="$VHOST"
136
136
  write_vhost() {
137
137
  local state
138
138
  if [ "${EDGE_MODE:-legacy}" = edge ]; then
139
- state="$(nginx_locked python3 "$DIR/edge_vhost.py" configure "$EDGE_CONFIG" "$APP" "$DOMAIN" "$PORT" "$1")"
139
+ state="$(nginx_locked python3 "$HELPERS/edge_vhost.py" configure "$EDGE_CONFIG" "$APP" "$DOMAIN" "$PORT" "$1")"
140
140
  else
141
- state="$(nginx_locked python3 "$DIR/configure_vhost.py" "$VHOST" "$DOMAIN" "$PORT" "$1")"
141
+ state="$(nginx_locked python3 "$HELPERS/configure_vhost.py" "$VHOST" "$DOMAIN" "$PORT" "$1")"
142
142
  fi
143
143
  [ "$state" != changed ] || log "nginx vhost for $DOMAIN ($1) → 127.0.0.1:$PORT"
144
144
  }
@@ -162,7 +162,7 @@ fi
162
162
  # and the application together; a green deploy must mean the public API works.
163
163
  BACKEND_DEPLOY_PHASE=public_health
164
164
  if [ "$EDGE_MODE" != legacy ]; then
165
- python3 "$DIR/edge_health.py" "$EDGE_CONFIG" "$APP" "$DOMAIN" "$HEALTH"
165
+ python3 "$HELPERS/edge_health.py" "$EDGE_CONFIG" "$APP" "$DOMAIN" "$HEALTH"
166
166
  log "done"
167
167
  exit 0
168
168
  fi
@@ -0,0 +1,95 @@
1
+ """Admit backend starts against live host headroom and existing container reservations."""
2
+ import json
3
+ import os
4
+ import subprocess
5
+ import time
6
+ import fcntl
7
+ from contextlib import contextmanager
8
+ from pathlib import Path
9
+
10
+ from runtime_model import MIB
11
+ from native_profile import running_native
12
+
13
+ HOST_POLICY = Path('/etc/gowalk-backend-ssh.json')
14
+ ROOT_UID = 0
15
+
16
+
17
+ def host_enabled():
18
+ if not HOST_POLICY.exists():
19
+ return
20
+ info = HOST_POLICY.stat()
21
+ policy = json.loads(HOST_POLICY.read_text())
22
+ if info.st_uid != ROOT_UID or info.st_mode & 0o022:
23
+ raise ValueError('host_policy_permissions')
24
+ if not isinstance(policy, dict) or policy.get('schema') != 'gowalk-cicd/backend-ssh.v1':
25
+ raise ValueError('host_backend_disabled')
26
+ if policy.get('disabled') is not False:
27
+ raise ValueError('host_backend_disabled')
28
+
29
+
30
+ @contextmanager
31
+ def start_lock(path):
32
+ with path.open("a") as lock:
33
+ deadline = time.monotonic() + 120
34
+ while True:
35
+ try:
36
+ fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
37
+ break
38
+ except BlockingIOError:
39
+ if time.monotonic() >= deadline:
40
+ raise ValueError("host_start_busy")
41
+ time.sleep(0.1)
42
+ yield
43
+
44
+
45
+ def host_facts(root=Path("/proc")):
46
+ memory = {}
47
+ for line in (root / "meminfo").read_text().splitlines():
48
+ key, value = line.split(":", 1)
49
+ memory[key] = int(value.strip().split()[0]) * 1024
50
+ pressure = 0.0
51
+ path = root / "pressure/memory"
52
+ if path.exists():
53
+ for line in path.read_text().splitlines():
54
+ if line.startswith("full "):
55
+ pressure = float(dict(part.split("=") for part in line.split()[1:])["avg10"])
56
+ return {"total": memory["MemTotal"], "available": memory["MemAvailable"],
57
+ "load": float((root / "loadavg").read_text().split()[0]),
58
+ "cpus": os.cpu_count() or 1, "memory_pressure": pressure}
59
+
60
+
61
+ def running_containers():
62
+ result = subprocess.run(["docker", "ps", "-q"], capture_output=True, text=True, timeout=30, check=True)
63
+ ids = result.stdout.split()
64
+ if not ids:
65
+ return running_native()
66
+ template = ('{"Config":{"Labels":{"com.docker.compose.project":'
67
+ '{{json (index .Config.Labels "com.docker.compose.project")}},'
68
+ '"com.docker.compose.project.working_dir":'
69
+ '{{json (index .Config.Labels "com.docker.compose.project.working_dir")}}}},'
70
+ '"HostConfig":{"MemoryReservation":{{.HostConfig.MemoryReservation}}}}')
71
+ result = subprocess.run(["docker", "inspect", "--format", template, *ids],
72
+ capture_output=True, text=True, timeout=30, check=True)
73
+ return [json.loads(line) for line in result.stdout.splitlines()] + running_native()
74
+
75
+
76
+ def admit(app, model, facts, containers):
77
+ host_enabled()
78
+ required = sum(service["mem_reservation"] for service in model["services"].values())
79
+ reserved, current = 0, 0
80
+ for container in containers:
81
+ labels = container.get("Config", {}).get("Labels") or {}
82
+ if not labels.get("com.docker.compose.project.working_dir", "").startswith("/opt/gowalk-backends/"):
83
+ continue
84
+ amount = container["HostConfig"].get("MemoryReservation") or 64 * MIB
85
+ if labels.get("com.docker.compose.project") == app:
86
+ current += amount
87
+ else:
88
+ reserved += amount
89
+ if reserved + required > facts["total"] * 0.7:
90
+ raise ValueError("host_reservations_exhausted")
91
+ if facts["available"] - max(0, required - current) < max(512 * MIB, facts["total"] * 0.1):
92
+ raise ValueError("host_memory_exhausted")
93
+ if facts["load"] > facts["cpus"] * 2 or facts["memory_pressure"] >= 1:
94
+ raise ValueError("host_pressure_high")
95
+ return required
@@ -0,0 +1,71 @@
1
+ """Resolve private Compose inputs, enforce limits and serialize only admitted container starts."""
2
+ import json
3
+ import os
4
+ import re
5
+ import subprocess
6
+ import sys
7
+ import tempfile
8
+ from pathlib import Path
9
+
10
+ from runtime_host import admit, host_facts, running_containers, start_lock
11
+ from runtime_model import configure
12
+
13
+ PROFILE_ROOT = Path("/opt/gowalk-backend-runtime")
14
+ START_LOCK = Path("/run/lock/gowalk-backend-start.lock")
15
+
16
+
17
+ def profile_snapshot(app):
18
+ if not re.fullmatch(r"[a-z0-9-]+", app):
19
+ raise ValueError("runtime_app_invalid")
20
+ profile = PROFILE_ROOT / (app + '.json')
21
+ return profile.read_bytes() if profile.exists() else None
22
+
23
+
24
+ def compose_model(app, arguments):
25
+ raw_profile = profile_snapshot(app)
26
+ result = subprocess.run(["docker", "compose", *arguments, "--project-name", app, "config", "--format", "json"],
27
+ capture_output=True, text=True, timeout=60, check=True)
28
+ return configure(json.loads(result.stdout), json.loads(raw_profile) if raw_profile is not None else None)
29
+
30
+
31
+ def launch(app, arguments):
32
+ original_profile = profile_snapshot(app)
33
+ model = compose_model(app, arguments)
34
+ admit(app, model, host_facts(), running_containers())
35
+ descriptor, filename = tempfile.mkstemp(prefix=".runtime-compose-", suffix=".json", dir=".")
36
+ try:
37
+ with os.fdopen(descriptor, "w") as stream:
38
+ json.dump(model, stream)
39
+ command = ["docker", "compose", "--file", filename, "--project-name", app]
40
+ if any(service.get("build") for service in model["services"].values()):
41
+ subprocess.run(command + ["build"], check=True)
42
+ with start_lock(START_LOCK):
43
+ if profile_snapshot(app) != original_profile:
44
+ raise ValueError('runtime_profile_changed')
45
+ admit(app, model, host_facts(), running_containers())
46
+ subprocess.run(command + ["up", "-d", "--no-build", "--remove-orphans"], check=True)
47
+ finally:
48
+ Path(filename).unlink()
49
+
50
+
51
+ def main():
52
+ try:
53
+ launch(sys.argv[1], sys.argv[2:])
54
+ except ValueError as error:
55
+ code = str(error) if str(error).replace("_", "").isalnum() else "runtime_invalid"
56
+ print("::error title=backend_runtime_refused::gowalk-cicd/backend-runtime-refused.v1 code=" + code,
57
+ file=sys.stderr)
58
+ return 1
59
+ except subprocess.CalledProcessError as error:
60
+ print("::error title=backend_runtime_refused::gowalk-cicd/backend-runtime-refused.v1 code=runtime_failed",
61
+ file=sys.stderr)
62
+ return error.returncode if 0 < error.returncode < 256 else 1
63
+ except (OSError, subprocess.SubprocessError, KeyError, TypeError):
64
+ print("::error title=backend_runtime_refused::gowalk-cicd/backend-runtime-refused.v1 code=runtime_failed",
65
+ file=sys.stderr)
66
+ return 1
67
+ return 0
68
+
69
+
70
+ if __name__ == "__main__":
71
+ sys.exit(main())
@@ -0,0 +1,111 @@
1
+ """Bound a resolved Compose model while preserving app code and host-owned DB bindings."""
2
+ import copy
3
+ import math
4
+ import re
5
+
6
+ MIB = 1024 * 1024
7
+ SCHEMA = "gowalk-cicd/backend-runtime.v1"
8
+ SERVICE_FIELDS = {"environment", "command", "entrypoint", "mem_limit", "mem_reservation", "cpus", "pids_limit"}
9
+
10
+
11
+ def quantity(value, units):
12
+ match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)\s*([a-z]*)", str(value).strip().lower())
13
+ if not match or match[2] not in units:
14
+ raise ValueError("resource_value_invalid")
15
+ result = float(match[1]) * units[match[2]]
16
+ if not math.isfinite(result) or result <= 0:
17
+ raise ValueError("resource_value_invalid")
18
+ return int(result)
19
+
20
+
21
+ def memory(value):
22
+ return quantity(value, {"": 1, "b": 1, "k": 1024, "kb": 1024, "m": MIB, "mb": MIB,
23
+ "g": 1024 * MIB, "gb": 1024 * MIB})
24
+
25
+
26
+ def duration(value):
27
+ parts = re.findall(r"([0-9]+(?:\.[0-9]+)?)(ns|us|ms|s|m|h)", str(value))
28
+ if not parts or "".join(number + unit for number, unit in parts) != str(value):
29
+ raise ValueError("health_duration_invalid")
30
+ units = {"ns": 1e-9, "us": 1e-6, "ms": 1e-3, "s": 1, "m": 60, "h": 3600}
31
+ return sum(float(number) * units[unit] for number, unit in parts)
32
+
33
+
34
+ def bind_profile(model, profile):
35
+ if not profile:
36
+ return
37
+ if not isinstance(profile, dict) or profile.get("schema") != SCHEMA:
38
+ raise ValueError("runtime_profile_invalid")
39
+ if set(profile) - {"schema", "disabled", "remove_services", "services"}:
40
+ raise ValueError("runtime_profile_invalid")
41
+ if profile.get("disabled"):
42
+ raise ValueError("runtime_disabled")
43
+ removed = profile.get("remove_services", [])
44
+ if not isinstance(removed, list) or any(not isinstance(name, str) for name in removed):
45
+ raise ValueError("runtime_profile_invalid")
46
+ for name in removed:
47
+ model["services"].pop(name, None)
48
+ for service in model["services"].values():
49
+ dependencies = service.get("depends_on", {})
50
+ service["depends_on"] = {key: value for key, value in dependencies.items() if key not in removed}
51
+ for name, values in profile.get("services", {}).items():
52
+ if name not in model["services"] or not isinstance(values, dict) or set(values) - SERVICE_FIELDS:
53
+ raise ValueError("runtime_service_invalid")
54
+ service = model["services"][name]
55
+ for key, value in values.items():
56
+ if key == "environment":
57
+ if not isinstance(value, dict):
58
+ raise ValueError("runtime_environment_invalid")
59
+ service.setdefault(key, {}).update(escape_interpolation(value))
60
+ else:
61
+ service[key] = escape_interpolation(value)
62
+
63
+
64
+ def bound_service(service):
65
+ resources = service.setdefault("deploy", {}).setdefault("resources", {})
66
+ limits, reservations = resources.setdefault("limits", {}), resources.setdefault("reservations", {})
67
+ cap = memory(service.get("mem_limit") or limits.get("memory") or 256 * MIB)
68
+ reserve = memory(service.get("mem_reservation") or reservations.get("memory") or min(64 * MIB, cap))
69
+ cpu = float(service.get("cpus") or limits.get("cpus") or 0.5)
70
+ pids = int(service.get("pids_limit") or limits.get("pids") or 128)
71
+ if reserve > cap or not math.isfinite(cpu) or cpu <= 0 or pids <= 0:
72
+ raise ValueError("resource_limits_invalid")
73
+ service.update(mem_limit=cap, mem_reservation=reserve, cpus=cpu, pids_limit=pids)
74
+ limits.update(memory=str(cap), cpus=str(cpu), pids=pids)
75
+ reservations["memory"] = str(reserve)
76
+ health = service.setdefault("healthcheck", {})
77
+ if not health.get("disable") and health.get("test") != ["NONE"]:
78
+ if duration(health.get("interval", "30s")) < 60:
79
+ health["interval"] = "60s"
80
+ if duration(health.get("start_period", "0s")) < 30:
81
+ health["start_period"] = "30s"
82
+ if duration(health.get("start_interval", "5s")) > 5 or "start_interval" not in health:
83
+ health["start_interval"] = "5s"
84
+ logging = service.setdefault("logging", {"driver": "json-file"})
85
+ if logging.get("driver") in ("json-file", "local"):
86
+ options = logging.setdefault("options", {})
87
+ options.setdefault("max-size", "10m")
88
+ options.setdefault("max-file", "3")
89
+ return reserve
90
+
91
+
92
+ def configure(document, profile=None):
93
+ model = copy.deepcopy(document)
94
+ if not isinstance(model.get("services"), dict) or not model["services"]:
95
+ raise ValueError("compose_services_required")
96
+ bind_profile(model, profile)
97
+ if not model["services"]:
98
+ raise ValueError("compose_services_required")
99
+ for service in model["services"].values():
100
+ bound_service(service)
101
+ return model
102
+
103
+
104
+ def escape_interpolation(value):
105
+ if isinstance(value, str):
106
+ return value.replace("$", "$$")
107
+ if isinstance(value, dict):
108
+ return {key: escape_interpolation(item) for key, item in value.items()}
109
+ if isinstance(value, list):
110
+ return [escape_interpolation(item) for item in value]
111
+ return value
@@ -0,0 +1,171 @@
1
+ """Forced-command entry for panel deploy keys; never execute helper code from app mirrors."""
2
+ import json
3
+ import os
4
+ import re
5
+ import shlex
6
+ import subprocess
7
+ import sys
8
+ import tempfile
9
+ from pathlib import Path
10
+
11
+ from native_profile import PROFILES as NATIVE_PROFILES
12
+
13
+ HELPERS = Path(__file__).resolve().parent
14
+ POLICY = Path('/etc/gowalk-backend-ssh.json')
15
+ OWNERS = Path('/var/lib/gowalk-backend-ownership')
16
+ APPS = Path('/opt/gowalk-backends')
17
+ INGRESS = Path('/opt/gowalk-backend-ingress')
18
+ ROOT_UID = 0
19
+ READY = ('command -v docker >/dev/null && docker info >/dev/null 2>&1 '
20
+ '&& docker compose version >/dev/null 2>&1')
21
+
22
+
23
+ def load_policy(owner):
24
+ info = POLICY.stat()
25
+ if info.st_uid != ROOT_UID or info.st_mode & 0o022:
26
+ raise ValueError('ssh_policy_permissions')
27
+ policy = json.loads(POLICY.read_text())
28
+ if not isinstance(policy, dict) or policy.get('schema') != 'gowalk-cicd/backend-ssh.v1':
29
+ raise ValueError('ssh_policy_invalid')
30
+ if not isinstance(policy.get('owners'), list) or not isinstance(policy.get('allow_new_projects'), bool):
31
+ raise ValueError('ssh_policy_invalid')
32
+ if not isinstance(policy.get('disabled'), bool) or owner not in policy['owners']:
33
+ raise ValueError('ssh_owner_refused')
34
+ return policy
35
+
36
+
37
+ def scoped_path(value, base=None):
38
+ base = base or APPS
39
+ path = Path(value)
40
+ if not path.is_absolute() or '..' in path.parts:
41
+ raise ValueError('ssh_path_refused')
42
+ try:
43
+ relative = path.relative_to(base)
44
+ except ValueError:
45
+ raise ValueError('ssh_path_refused') from None
46
+ if not relative.parts or not re.fullmatch('[a-z0-9-]+', relative.parts[0]):
47
+ raise ValueError('ssh_project_refused')
48
+ project = relative.parts[0]
49
+ root = base / project
50
+ resolved_root = base.resolve() / project
51
+ resolved = path.resolve()
52
+ if root.is_symlink() or (resolved != resolved_root and resolved_root not in resolved.parents):
53
+ raise ValueError('ssh_path_refused')
54
+ return project, path
55
+
56
+
57
+ def authorize(owner, project, policy, may_claim):
58
+ path = OWNERS / (project + '.json')
59
+ if not path.exists():
60
+ if not may_claim or not policy.get('allow_new_projects') or (APPS / project).exists():
61
+ raise ValueError('ssh_project_unregistered')
62
+ OWNERS.mkdir(mode=0o700, parents=True, exist_ok=True)
63
+ descriptor, staging = tempfile.mkstemp(prefix='.claim-', dir=str(OWNERS))
64
+ try:
65
+ with os.fdopen(descriptor, 'w') as stream:
66
+ json.dump({'project': project, 'owner': owner}, stream)
67
+ try:
68
+ os.link(staging, str(path))
69
+ except FileExistsError:
70
+ pass
71
+ finally:
72
+ Path(staging).unlink()
73
+ info = path.stat()
74
+ if info.st_uid != ROOT_UID or info.st_mode & 0o022:
75
+ raise ValueError('ssh_owner_permissions')
76
+ binding = json.loads(path.read_text())
77
+ if binding != {'project': project, 'owner': owner}:
78
+ raise ValueError('ssh_project_owned_elsewhere')
79
+
80
+
81
+ def helper_command(words):
82
+ if len(words) == 3 and words[:2] == ['gowalk-backend', 'native-deploy']:
83
+ if not re.fullmatch('[a-z0-9-]+', words[2]):
84
+ raise ValueError('ssh_project_refused')
85
+ return words[2], False, [sys.executable, str(HELPERS / 'native_deploy.py'), words[2]]
86
+ if len(words) in (4, 5, 6, 7) and words[0] == 'bash':
87
+ project, script = scoped_path(words[1])
88
+ if script != APPS / project / 'remote_deploy.sh' or words[2] != project:
89
+ raise ValueError('ssh_helper_refused')
90
+ patterns = [r'[a-z0-9.-]*', r'/[A-Za-z0-9_./-]*', r'[A-Za-z0-9._%+@-]+']
91
+ if any(not re.fullmatch(pattern, value) for pattern, value in zip(patterns, words[3:6])):
92
+ raise ValueError('ssh_arguments_refused')
93
+ compose_names = ('compose.yaml', 'compose.yml', 'docker-compose.yaml', 'docker-compose.yml')
94
+ if len(words) == 7 and words[6] not in compose_names:
95
+ raise ValueError('ssh_arguments_refused')
96
+ return project, False, ['/bin/bash', str(HELPERS / 'remote_deploy.sh'), *words[2:]]
97
+ if len(words) == 4 and words[0] == 'python3' and words[2] == 'receive':
98
+ project, script = scoped_path(words[1])
99
+ if script != APPS / project / 'runtime_env.py' or words[3] != str(APPS / project / '.runtime.env'):
100
+ raise ValueError('ssh_helper_refused')
101
+ return project, False, [sys.executable, str(HELPERS / 'runtime_env.py'), 'receive', words[3]]
102
+ if len(words) == 9 and words[:3] == ['python3', '-', '--remote']:
103
+ if words[3::2] != ['--app', '--service', '--since'] or not re.fullmatch('[a-z0-9-]+', words[4]):
104
+ raise ValueError('ssh_helper_refused')
105
+ return words[4], False, [sys.executable, str(HELPERS / 'identity_diagnostics.py'), *words[2:]]
106
+ raise ValueError('ssh_operation_refused')
107
+
108
+
109
+ def transfer_command(words):
110
+ if len(words) == 3 and words[:2] == ['mkdir', '-p']:
111
+ project, path = scoped_path(words[2])
112
+ return project, path == APPS / project, ['/usr/bin/mkdir', '-p', str(path)]
113
+ if len(words) >= 5 and words[:2] == ['rsync', '--server'] and words[-2] == '.':
114
+ project, path = scoped_path(words[-1])
115
+ for option in words[2:-2]:
116
+ if option not in ('--delete', '--delete-after', '--delete-during', '--delete-delay'):
117
+ if not re.fullmatch(r'-[vlogDtprz]+(?:e[.a-zA-Z]+)?', option):
118
+ raise ValueError('ssh_rsync_refused')
119
+ return project, False, ['/usr/bin/rsync', *words[1:-1], str(path)]
120
+ if len(words) == 11 and words[:5] == ['umask', '077', ';', 'mkdir', '-p']:
121
+ project, destination = scoped_path(words[5], INGRESS)
122
+ expected = [';', 'python3', str(APPS / project / 'edge_inputs.py'), 'receive',
123
+ str(INGRESS / project / '.backend-edge.json')]
124
+ if words[6:] != expected or destination != INGRESS / project:
125
+ raise ValueError('ssh_helper_refused')
126
+ return project, False, [sys.executable, str(HELPERS / 'edge_inputs.py'), 'receive', expected[-1]]
127
+ return helper_command(words)
128
+
129
+
130
+ def plan(command):
131
+ lexer = shlex.shlex(command, posix=True, punctuation_chars=';&|<>')
132
+ lexer.whitespace_split = True
133
+ lexer.commenters = ''
134
+ words = list(lexer)
135
+ if not words or len(command) > 8192:
136
+ raise ValueError('ssh_operation_refused')
137
+ return transfer_command(words)
138
+
139
+
140
+ def readiness():
141
+ for command in [['docker', 'info'], ['docker', 'compose', 'version']]:
142
+ subprocess.run(command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=15, check=True)
143
+ return 0
144
+
145
+
146
+ def main():
147
+ try:
148
+ os.umask(0o077)
149
+ owner = sys.argv[1]
150
+ policy = load_policy(owner)
151
+ command = os.environ.get('SSH_ORIGINAL_COMMAND', '')
152
+ if ' '.join(command.split()) in (READY, 'gowalk-backend readiness'):
153
+ return readiness()
154
+ if policy.get('disabled', True):
155
+ raise ValueError('ssh_backend_disabled')
156
+ project, may_claim, arguments = plan(command)
157
+ authorize(owner, project, policy, may_claim)
158
+ if arguments[:2] == ['/bin/bash', str(HELPERS / 'remote_deploy.sh')]:
159
+ if (NATIVE_PROFILES / (project + '.json')).exists():
160
+ raise ValueError('ssh_native_project_required')
161
+ if arguments[1:3] == [str(HELPERS / 'edge_inputs.py'), 'receive']:
162
+ Path(arguments[3]).parent.mkdir(mode=0o700, parents=True, exist_ok=True)
163
+ os.execve(arguments[0], arguments, {'PATH': '/usr/sbin:/usr/bin:/sbin:/bin', 'LANG': 'C.UTF-8'})
164
+ except (OSError, ValueError, KeyError, IndexError, subprocess.SubprocessError):
165
+ print('backend_ssh_refused', file=sys.stderr)
166
+ return 1
167
+ return 0
168
+
169
+
170
+ if __name__ == '__main__':
171
+ sys.exit(main())
@@ -57,3 +57,19 @@ They are observed patterns, not a unique cause or retry permission. No raw error
57
57
  token or child output is included. A failed restoration adds a separate rollback refusal while keeping
58
58
  the original failure visible; absence of that extra line alone does not prove successful restoration.
59
59
  The generic `backend_edge_vhost_refused`, changed/unchanged stdout, ingress, TLS and deploy gates remain.
60
+
61
+ Health refusals add the fixed stderr annotation `backend_health_failed` with schema
62
+ `gowalk-cicd/backend-health-failed.v1`. Its brace-free fields are `probe`, `reason`, `curl_exit`,
63
+ `http_status` and `attempt`. Probes are configuration, origin_missing_token, origin_invalid_token,
64
+ origin_authorized, legacy_identity, legacy and public; unknown metadata becomes unclassified.
65
+ Reasons are unclassified, path_invalid, request_failed, http_refused, json_invalid, body_shape,
66
+ build_missing and build_mismatch. No URL, response body, build identity, header, credential, path,
67
+ exception text or subprocess output enters the annotation. Curl exit is an observed integer from
68
+ -255 to 255, HTTP status from 100 to 599 and attempt from 1 to 15; other values become `none`.
69
+ An exhausted health check emits only its final attempt's facts, clearing any previous observations.
70
+ These facts identify the failed probe, not a unique cause or permission to retry or bypass it.
71
+ Missing-token and invalid-token checks remain single requests requiring completed 403 responses.
72
+ Healthy-origin, retained-legacy and public checks retain 15 attempts with two seconds between them,
73
+ strict TLS, existing curl time/body bounds and exact build checks. A successful check emits no
74
+ failure annotation and returns the existing result. Original exceptions and the generic CLI refusal
75
+ `backend_edge_health_failed` with exit 1 remain unchanged, including when diagnostic delivery fails.
@@ -0,0 +1,60 @@
1
+ # Profiled native backend compatibility
2
+
3
+ Existing native services can retain their interpreters, private environment, database and systemd units.
4
+ The same forced SSH entry accepts exactly `gowalk-backend native-deploy <project>` and dispatches the
5
+ release-owned `native_deploy.py`; callers cannot supply commands, units or resource overrides. The host
6
+ must already have the compatible interpreter, dependencies, root-owned units and ownership registration.
7
+ A native profile prevents that project from falling back to the Compose deploy entry.
8
+
9
+ Upload source to `/opt/gowalk-backends/<project>` through the scoped receiver. Include `.backend-source-sha`
10
+ containing the exact 40-character lowercase commit SHA. Create a root-owned, non-writable-by-others
11
+ `/opt/gowalk-backend-native/<project>.json` outside that mirror:
12
+
13
+ ```json
14
+ {
15
+ "schema": "gowalk-cicd/backend-native.v1",
16
+ "disabled": true,
17
+ "directory": "/opt/example",
18
+ "units": ["example.service", "example-worker.service"],
19
+ "restart_units": ["example.service"],
20
+ "install_user": "root",
21
+ "install": [
22
+ ["/opt/example/env/bin/python", "-m", "pip", "install", "-r", "requirements.txt"],
23
+ ["/opt/example/env/bin/python", "scripts/apply_migrations.py"]
24
+ ],
25
+ "verify": [],
26
+ "health_url": "http://127.0.0.1:6061/health",
27
+ "health_json": {"status": "ok"},
28
+ "preserve": ["static/", "logs/"],
29
+ "resources": {}
30
+ }
31
+ ```
32
+
33
+ The profile's directory must be the project's standard mirror or exact legacy `/opt/<project>` path.
34
+ The helper synchronizes legacy checkouts with content checksums, preserving `.env`, `.runtime.env`, `.git`,
35
+ `env/`, `.venv/` and the explicit production-state paths. Declare every persistent app-owned path before
36
+ activation; external state is never synchronized. Source deletion applies only outside those exclusions.
37
+ Private runtime rotations can use the existing runtime-env envelope into the mirror's `.runtime.env`.
38
+ Provision an optional systemd `EnvironmentFile=-/opt/gowalk-backends/<project>/.runtime.env` for apps whose
39
+ settings honor process-environment precedence. Do not replace their original private `.env` file.
40
+
41
+ `units` declares every managed unit; `restart_units` declares only the units this deployment starts.
42
+ All managed units receive persistent limits, but an inactive worker remains inactive and an active worker
43
+ is not restarted unless listed. ActiveState, UnitFileState and MainPID of held units must remain unchanged
44
+ from before preparation through final verification. Resource fields per unit are `mem_limit`, `mem_reservation`, `cpus` and
45
+ `pids_limit`, with the same defaults as Compose. The helper reads back MemoryMax, MemoryLow, CPUQuota and
46
+ TasksMax and refuses if they differ. Every active native unit contributes its reservation to admission
47
+ for both native and Compose deployments.
48
+
49
+ Install and post-health verification commands are fixed argument arrays from the trusted profile,
50
+ including an app's established idempotent migration entry. They run in the legacy checkout under the
51
+ specified existing user, inside transient units bounded to 512 MiB, one CPU and 128 PIDs. Installation
52
+ shares a ten-minute budget and verification a one-minute budget; a failed transient unit is stopped.
53
+ Captured output stays private. Verification programs must return nonzero when their contract fails.
54
+
55
+ Both global and per-project disabled gates must open before deployment. Admission runs before preparation
56
+ and again under the shared start lock, rereading the global freeze both times. A changed profile refuses start. Local health has a 60-second bound,
57
+ requires 2xx plus the declared JSON fields, and requires only the restarted units to be active. A failed
58
+ restart, health or verification stops exactly those units and fails CI; subsequent corrected deployments
59
+ use the same bounded path. No worker is enabled, no notification is sent and no source rollback is claimed.
60
+ A successful receipt reports the exact transferred source SHA, verified resources, health and verification.
@@ -0,0 +1,57 @@
1
+ # Host-owned backend deployment entry
2
+
3
+ Install `backend-action/` from an integrity-verified published package in a root-owned release directory,
4
+ outside `/opt/gowalk-backends/`. Deployment transport must not be able to replace that directory.
5
+ The forced-command entry runs every supported helper from its own release directory, including when an
6
+ older action uploaded an older `remote_deploy.sh` into an app mirror. No consumer repository update is
7
+ needed for the transport protocol below.
8
+
9
+ Each panel gets its own dedicated SSH key. Its `authorized_keys` entry uses `restrict` and a fixed owner:
10
+
11
+ ```text
12
+ restrict,command="/usr/bin/python3 /opt/gowalk-cicd-runtime/1.0.179/backend-action/ssh_gate.py robot" ssh-ed25519 <key>
13
+ ```
14
+
15
+ Keep operator and release credentials separate. Never install an unrestricted backend key as a fallback.
16
+ Create root-owned mode-0600 `/etc/gowalk-backend-ssh.json`:
17
+
18
+ ```json
19
+ {
20
+ "schema": "gowalk-cicd/backend-ssh.v1",
21
+ "disabled": true,
22
+ "owners": ["robot", "vpnrobot"],
23
+ "allow_new_projects": true
24
+ }
25
+ ```
26
+
27
+ `disabled: true` refuses every mutation while permitting the read-only readiness probe. Keep it set
28
+ through migration and validation. Existing per-app runtime `disabled` profiles remain an independent
29
+ start fence when the global transport gate opens.
30
+
31
+ Register each existing app as `/var/lib/gowalk-backend-ownership/<project>.json`, root-owned and mode0600,
32
+ with exactly `{"project":"<project>","owner":"robot"}` (or its actual panel owner). Do not infer ownership
33
+ from an SSH caller. Existing app directories without registration refuse first claims. After activation,
34
+ the action's first `mkdir -p /opt/gowalk-backends/<new-project>` atomically claims an unused safe project
35
+ name for that panel. A competing panel cannot replace the binding. This permits ordinary new backend
36
+ provisioning without a manual allowlist update; existing ownership and private directories remain fenced.
37
+
38
+ The entry permits only:
39
+
40
+ - the panel's exact existing Docker/Compose readiness probe, or `gowalk-backend readiness`;
41
+ - scoped app-directory creation and the action's rsync **receiver** protocol inside that app;
42
+ - the standard private runtime environment and edge-input receive envelopes;
43
+ - the standard backend deploy invocation, redirected to the host-owned released helper;
44
+ - the standard bounded identity-diagnostic invocation, also redirected to the released helper;
45
+ - `gowalk-backend native-deploy <project>` for an existing [root-profiled native service](BACKEND_NATIVE.md).
46
+
47
+ It refuses arbitrary shell commands, interactive shells, rsync sender/download operations, traversal,
48
+ directory symlinks escaping an app and any attempt to execute an app-mirror helper. App code still owns
49
+ its container build and product implementation; the released launcher owns runtime limits and admission.
50
+ Custom native deployment workflows use their fixed host-owned profile before enabling their key;
51
+ they cannot use this key to run arbitrary systemd commands.
52
+
53
+ Verify with a fresh connection under each real panel key: readiness succeeds during a global freeze;
54
+ mutations refuse; after the gate opens, the key can transfer only its own app's source and another panel's
55
+ project refuses. A disabled runtime profile must still refuse a legacy deploy invocation. Verify a new
56
+ task-owned project can be claimed and replayed, then remove only that verification project and binding.
57
+ Failures emit only `backend_ssh_refused`, never the original command or private stdin.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "gowalk-cicd",
3
- "version": "1.0.177",
3
+ "version": "1.0.179",
4
4
  "description": "Zero-config GitHub Actions delivery for iOS TestFlight and Android Google Play.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -39,6 +39,8 @@
39
39
  "docs/GRADLE_CACHE.md",
40
40
  "docs/CANDIDATE_ARTIFACTS.md",
41
41
  "docs/BACKEND_EDGE.md",
42
+ "docs/BACKEND_SSH_GATE.md",
43
+ "docs/BACKEND_NATIVE.md",
42
44
  "docs/CONFIGURATION.md",
43
45
  "docs/CI_PERFORMANCE.md"
44
46
  ],
@@ -60,7 +60,7 @@ jobs:
60
60
  diagnostic-since: ${{ inputs.diagnostic-since }}
61
61
  ssh-key: ${{ secrets.BACKEND_DEPLOY_SSH_KEY }}
62
62
  known-hosts: ${{ vars.BACKEND_DEPLOY_KNOWN_HOSTS }}
63
- host: ${{ vars.BACKEND_DEPLOY_HOST || '138.197.36.107' }}
63
+ host: ${{ vars.BACKEND_DEPLOY_HOST }}
64
64
  app-name: ${{ vars.BACKEND_APP_NAME || github.event.repository.name }}
65
65
  api-domain: ${{ vars.BACKEND_API_DOMAIN }}
66
66
  backend-dir: ${{ steps.dir.outputs.path }}
@@ -839,7 +839,7 @@ jobs:
839
839
  with:
840
840
  ssh-key: ${{ secrets.BACKEND_DEPLOY_SSH_KEY }}
841
841
  known-hosts: ${{ vars.BACKEND_DEPLOY_KNOWN_HOSTS }}
842
- host: ${{ vars.BACKEND_DEPLOY_HOST || '138.197.36.107' }}
842
+ host: ${{ vars.BACKEND_DEPLOY_HOST }}
843
843
  app-name: ${{ vars.BACKEND_APP_NAME || github.event.repository.name }}
844
844
  api-domain: ${{ vars.BACKEND_API_DOMAIN }}
845
845
  backend-dir: ${{ steps.dir.outputs.path }}
Binary file